2024-09-19T04:42:59.499 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-19T04:42:59.524 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T04:42:59.613 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911648 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/suites/iozone}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7911648' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1747 sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 sleep_before_teardown: 0 subset: 176/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi043.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMmEosclhvw7cTo/CMA/UFbCsz/ZCMQN4KsFOWOT709DwWOf3AKsbRVLftAIJsPw334v0/XW5YuBe/f7WySXE5w= smithi096.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBENHaflXfJQbHnvZUaDLotKckyg1WELt3bpPMEWB2BgZ0gWanGJ0Vw7UUIn5E0fWU8nDMxoUalpBm0/PVp/A+GM= smithi151.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFo8XIWX2hE260JVRzUEEApI5YUQwS/625/Bh4l077wN5uB886cLy61P66QshCC5jJyHvbc0lYL9SQgut+fOybg= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - suites/iozone.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-18_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-19T04:42:59.613 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa; will attempt to use it 2024-09-19T04:42:59.614 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks 2024-09-19T04:42:59.614 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-19T04:42:59.615 INFO:teuthology.task.internal:Checking packages... 2024-09-19T04:42:59.636 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '2f599a46ad10e37c929e5c43da2f6907ddcd1961' 2024-09-19T04:42:59.636 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-19T04:42:59.637 INFO:teuthology.packaging:ref: None 2024-09-19T04:42:59.637 INFO:teuthology.packaging:tag: None 2024-09-19T04:42:59.637 INFO:teuthology.packaging:branch: squid 2024-09-19T04:42:59.637 INFO:teuthology.packaging:sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:42:59.637 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-19T04:42:59.846 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-363.g6fad5dd7 2024-09-19T04:42:59.848 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-19T04:42:59.854 INFO:teuthology.task.internal:no buildpackages task found 2024-09-19T04:42:59.854 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-19T04:42:59.880 INFO:teuthology.task.internal:Saving configuration 2024-09-19T04:42:59.900 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-19T04:42:59.906 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-19T04:42:59.932 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi043.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911648', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 04:37:46.506921', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMmEosclhvw7cTo/CMA/UFbCsz/ZCMQN4KsFOWOT709DwWOf3AKsbRVLftAIJsPw334v0/XW5YuBe/f7WySXE5w='} 2024-09-19T04:42:59.957 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi096.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911648', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 04:37:46.505673', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBENHaflXfJQbHnvZUaDLotKckyg1WELt3bpPMEWB2BgZ0gWanGJ0Vw7UUIn5E0fWU8nDMxoUalpBm0/PVp/A+GM='} 2024-09-19T04:42:59.981 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi151.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911648', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 04:37:46.504186', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFo8XIWX2hE260JVRzUEEApI5YUQwS/625/Bh4l077wN5uB886cLy61P66QshCC5jJyHvbc0lYL9SQgut+fOybg='} 2024-09-19T04:42:59.982 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-19T04:42:59.987 INFO:teuthology.task.internal:roles: ubuntu@smithi043.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-19T04:42:59.988 INFO:teuthology.task.internal:roles: ubuntu@smithi096.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-19T04:42:59.988 INFO:teuthology.task.internal:roles: ubuntu@smithi151.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-19T04:42:59.988 INFO:teuthology.run_tasks:Running task console_log... 2024-09-19T04:43:00.101 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f5a479cbbe0>, signals=[15]) 2024-09-19T04:43:00.101 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-19T04:43:00.107 INFO:teuthology.task.internal:Opening connections... 2024-09-19T04:43:00.107 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi043.front.sepia.ceph.com 2024-09-19T04:43:00.108 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:43:00.184 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi096.front.sepia.ceph.com 2024-09-19T04:43:00.185 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:43:00.258 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi151.front.sepia.ceph.com 2024-09-19T04:43:00.259 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:43:00.333 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-19T04:43:00.340 DEBUG:teuthology.orchestra.run.smithi043:> uname -m 2024-09-19T04:43:00.358 INFO:teuthology.orchestra.run.smithi043.stdout:x86_64 2024-09-19T04:43:00.359 DEBUG:teuthology.orchestra.run.smithi043:> cat /etc/os-release 2024-09-19T04:43:00.415 INFO:teuthology.orchestra.run.smithi043.stdout:NAME="CentOS Stream" 2024-09-19T04:43:00.415 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION="9" 2024-09-19T04:43:00.416 INFO:teuthology.orchestra.run.smithi043.stdout:ID="centos" 2024-09-19T04:43:00.416 INFO:teuthology.orchestra.run.smithi043.stdout:ID_LIKE="rhel fedora" 2024-09-19T04:43:00.416 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION_ID="9" 2024-09-19T04:43:00.416 INFO:teuthology.orchestra.run.smithi043.stdout:PLATFORM_ID="platform:el9" 2024-09-19T04:43:00.416 INFO:teuthology.orchestra.run.smithi043.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T04:43:00.416 INFO:teuthology.orchestra.run.smithi043.stdout:ANSI_COLOR="0;31" 2024-09-19T04:43:00.416 INFO:teuthology.orchestra.run.smithi043.stdout:LOGO="fedora-logo-icon" 2024-09-19T04:43:00.416 INFO:teuthology.orchestra.run.smithi043.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T04:43:00.416 INFO:teuthology.orchestra.run.smithi043.stdout:HOME_URL="https://centos.org/" 2024-09-19T04:43:00.416 INFO:teuthology.orchestra.run.smithi043.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T04:43:00.416 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T04:43:00.416 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T04:43:00.417 INFO:teuthology.lock.ops:Updating smithi043.front.sepia.ceph.com on lock server 2024-09-19T04:43:00.532 DEBUG:teuthology.orchestra.run.smithi096:> uname -m 2024-09-19T04:43:00.551 INFO:teuthology.orchestra.run.smithi096.stdout:x86_64 2024-09-19T04:43:00.551 DEBUG:teuthology.orchestra.run.smithi096:> cat /etc/os-release 2024-09-19T04:43:00.608 INFO:teuthology.orchestra.run.smithi096.stdout:NAME="CentOS Stream" 2024-09-19T04:43:00.609 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION="9" 2024-09-19T04:43:00.609 INFO:teuthology.orchestra.run.smithi096.stdout:ID="centos" 2024-09-19T04:43:00.609 INFO:teuthology.orchestra.run.smithi096.stdout:ID_LIKE="rhel fedora" 2024-09-19T04:43:00.609 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_ID="9" 2024-09-19T04:43:00.609 INFO:teuthology.orchestra.run.smithi096.stdout:PLATFORM_ID="platform:el9" 2024-09-19T04:43:00.609 INFO:teuthology.orchestra.run.smithi096.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T04:43:00.609 INFO:teuthology.orchestra.run.smithi096.stdout:ANSI_COLOR="0;31" 2024-09-19T04:43:00.609 INFO:teuthology.orchestra.run.smithi096.stdout:LOGO="fedora-logo-icon" 2024-09-19T04:43:00.609 INFO:teuthology.orchestra.run.smithi096.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T04:43:00.609 INFO:teuthology.orchestra.run.smithi096.stdout:HOME_URL="https://centos.org/" 2024-09-19T04:43:00.609 INFO:teuthology.orchestra.run.smithi096.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T04:43:00.610 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T04:43:00.610 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T04:43:00.610 INFO:teuthology.lock.ops:Updating smithi096.front.sepia.ceph.com on lock server 2024-09-19T04:43:00.640 DEBUG:teuthology.orchestra.run.smithi151:> uname -m 2024-09-19T04:43:00.659 INFO:teuthology.orchestra.run.smithi151.stdout:x86_64 2024-09-19T04:43:00.659 DEBUG:teuthology.orchestra.run.smithi151:> cat /etc/os-release 2024-09-19T04:43:00.717 INFO:teuthology.orchestra.run.smithi151.stdout:NAME="CentOS Stream" 2024-09-19T04:43:00.717 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION="9" 2024-09-19T04:43:00.717 INFO:teuthology.orchestra.run.smithi151.stdout:ID="centos" 2024-09-19T04:43:00.717 INFO:teuthology.orchestra.run.smithi151.stdout:ID_LIKE="rhel fedora" 2024-09-19T04:43:00.717 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION_ID="9" 2024-09-19T04:43:00.717 INFO:teuthology.orchestra.run.smithi151.stdout:PLATFORM_ID="platform:el9" 2024-09-19T04:43:00.717 INFO:teuthology.orchestra.run.smithi151.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T04:43:00.717 INFO:teuthology.orchestra.run.smithi151.stdout:ANSI_COLOR="0;31" 2024-09-19T04:43:00.717 INFO:teuthology.orchestra.run.smithi151.stdout:LOGO="fedora-logo-icon" 2024-09-19T04:43:00.717 INFO:teuthology.orchestra.run.smithi151.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T04:43:00.718 INFO:teuthology.orchestra.run.smithi151.stdout:HOME_URL="https://centos.org/" 2024-09-19T04:43:00.718 INFO:teuthology.orchestra.run.smithi151.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T04:43:00.718 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T04:43:00.718 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T04:43:00.718 INFO:teuthology.lock.ops:Updating smithi151.front.sepia.ceph.com on lock server 2024-09-19T04:43:00.740 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-19T04:43:00.750 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-19T04:43:00.780 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-19T04:43:00.780 DEBUG:teuthology.orchestra.run.smithi043:> test '!' -e /home/ubuntu/cephtest 2024-09-19T04:43:00.783 DEBUG:teuthology.orchestra.run.smithi096:> test '!' -e /home/ubuntu/cephtest 2024-09-19T04:43:00.785 DEBUG:teuthology.orchestra.run.smithi151:> test '!' -e /home/ubuntu/cephtest 2024-09-19T04:43:00.802 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-19T04:43:00.808 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-19T04:43:00.808 DEBUG:teuthology.orchestra.run.smithi043:> test -z $(ls -A /var/lib/ceph) 2024-09-19T04:43:00.840 DEBUG:teuthology.orchestra.run.smithi096:> test -z $(ls -A /var/lib/ceph) 2024-09-19T04:43:00.843 DEBUG:teuthology.orchestra.run.smithi151:> test -z $(ls -A /var/lib/ceph) 2024-09-19T04:43:00.961 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-19T04:43:01.038 INFO:teuthology.run_tasks:Running task kernel... 2024-09-19T04:43:01.053 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-19T04:43:01.053 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-09-19T04:43:01.053 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-09-19T04:43:01.054 DEBUG:teuthology.orchestra.run.smithi043:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T04:43:01.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:43:01.071 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-09-19T04:43:01.237 DEBUG:teuthology.task.kernel:sha1 for client.0 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-19T04:43:01.237 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-19T04:43:01.237 DEBUG:teuthology.orchestra.run.smithi043:> uname -r 2024-09-19T04:43:01.255 INFO:teuthology.orchestra.run.smithi043.stdout:5.14.0-503.el9.x86_64 2024-09-19T04:43:01.255 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-19T04:43:01.255 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-19T04:43:01.256 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum upgrade -y linux-firmware 2024-09-19T04:43:05.545 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - BaseOS 3.3 MB/s | 8.3 MB 00:02 2024-09-19T04:43:14.653 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - AppStream 2.6 MB/s | 20 MB 00:07 2024-09-19T04:43:18.987 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.5 MB 00:00 2024-09-19T04:43:20.696 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 19 kB 00:00 2024-09-19T04:43:21.825 INFO:teuthology.orchestra.run.smithi043.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-19T04:43:27.357 INFO:teuthology.orchestra.run.smithi043.stdout:lab-extras 43 kB/s | 1.7 kB 00:00 2024-09-19T04:43:29.441 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-19T04:43:29.446 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-19T04:43:29.447 INFO:teuthology.orchestra.run.smithi043.stdout: Package Architecture Version Repository Size 2024-09-19T04:43:29.447 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-19T04:43:29.447 INFO:teuthology.orchestra.run.smithi043.stdout:Upgrading: 2024-09-19T04:43:29.447 INFO:teuthology.orchestra.run.smithi043.stdout: linux-firmware noarch 20240905-146.el9 baseos 388 M 2024-09-19T04:43:29.447 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T04:43:29.447 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-19T04:43:29.447 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-19T04:43:29.447 INFO:teuthology.orchestra.run.smithi043.stdout:Upgrade 1 Package 2024-09-19T04:43:29.447 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T04:43:29.447 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 388 M 2024-09-19T04:43:29.447 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-19T04:43:37.283 INFO:teuthology.orchestra.run.smithi043.stdout:linux-firmware-20240905-146.el9.noarch.rpm 51 MB/s | 388 MB 00:07 2024-09-19T04:43:37.283 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-19T04:43:37.283 INFO:teuthology.orchestra.run.smithi043.stdout:Total 50 MB/s | 388 MB 00:07 2024-09-19T04:43:40.557 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-19T04:43:40.640 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-19T04:43:40.641 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-19T04:43:44.064 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-19T04:43:44.065 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-19T04:43:53.665 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-19T04:43:54.301 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : linux-firmware-20240905-146.el9.noarch 1/2 2024-09-19T04:43:54.384 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : linux-firmware-20240716-145.el9.noarch 2/2 2024-09-19T04:43:55.045 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: linux-firmware-20240716-145.el9.noarch 2/2 2024-09-19T04:43:55.045 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : linux-firmware-20240905-146.el9.noarch 1/2 2024-09-19T04:43:55.856 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : linux-firmware-20240716-145.el9.noarch 2/2 2024-09-19T04:43:55.856 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T04:43:55.856 INFO:teuthology.orchestra.run.smithi043.stdout:Upgraded: 2024-09-19T04:43:55.856 INFO:teuthology.orchestra.run.smithi043.stdout: linux-firmware-20240905-146.el9.noarch 2024-09-19T04:43:55.856 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T04:43:55.856 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-19T04:43:56.190 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-19T04:43:56.191 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-19T04:43:56.351 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-19T04:43:56.351 DEBUG:teuthology.orchestra.run.smithi043:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-19T04:43:56.354 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi043.front.sepia.ceph.com to complete... 2024-09-19T04:44:02.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19 04:44:02 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-19T04:44:02.661 INFO:teuthology.orchestra.run.smithi043.stderr:FINISHED --2024-09-19 04:44:02-- 2024-09-19T04:44:02.661 INFO:teuthology.orchestra.run.smithi043.stderr:Total wall clock time: 6.3s 2024-09-19T04:44:02.661 INFO:teuthology.orchestra.run.smithi043.stderr:Downloaded: 1 files, 343M in 6.0s (57.2 MB/s) 2024-09-19T04:44:02.663 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-19T04:44:02.664 DEBUG:teuthology.orchestra.run.smithi043:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-19T04:44:04.823 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying... ######################################## 2024-09-19T04:44:04.880 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing... ######################################## 2024-09-19T04:44:04.944 INFO:teuthology.orchestra.run.smithi043.stdout:Updating / installing... 2024-09-19T04:44:13.241 INFO:teuthology.orchestra.run.smithi043.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-19T04:44:53.649 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi043.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-19T04:44:53.650 DEBUG:teuthology.orchestra.run.smithi043:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-19T04:44:53.740 INFO:teuthology.orchestra.run.smithi043.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-19T04:44:53.741 INFO:teuthology.orchestra.run.smithi043.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-19T04:44:53.741 INFO:teuthology.orchestra.run.smithi043.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-19T04:44:53.741 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-19T04:44:53.741 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-19T04:44:53.741 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-19T04:44:53.741 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-19T04:44:53.741 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-19T04:44:53.741 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-19T04:44:53.741 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-19T04:44:53.741 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-19T04:44:53.741 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-19T04:44:53.741 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-19T04:44:53.741 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-19T04:44:53.742 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-19T04:44:53.742 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-19T04:44:53.742 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-19T04:44:53.742 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-19T04:44:53.742 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-19T04:44:53.742 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-19T04:44:53.742 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-19T04:44:53.742 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-19T04:44:53.742 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-19T04:44:53.742 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-19T04:44:53.742 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-19T04:44:53.742 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-19T04:44:53.742 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-19T04:44:53.742 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-19T04:44:53.742 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-19T04:44:53.743 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-19T04:44:53.743 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-19T04:44:53.743 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-19T04:44:53.743 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-19T04:44:53.743 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-19T04:44:53.743 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-19T04:44:53.743 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-19T04:44:53.743 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-19T04:44:53.743 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-19T04:44:53.743 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-19T04:44:53.744 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-19T04:44:53.744 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-19T04:44:53.744 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-19T04:44:53.744 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-19T04:44:53.744 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-19T04:44:53.744 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-19T04:44:53.744 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-19T04:44:53.744 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-19T04:44:53.744 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-19T04:44:53.744 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-19T04:44:53.744 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-19T04:44:53.745 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-19T04:44:53.745 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-19T04:44:53.745 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-19T04:44:53.745 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-19T04:44:53.745 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-19T04:44:53.745 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-19T04:44:53.745 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-19T04:44:53.745 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-19T04:44:53.745 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-19T04:44:53.745 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-19T04:44:53.745 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-19T04:44:53.746 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-19T04:44:53.746 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-19T04:44:53.746 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-19T04:44:53.746 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-19T04:44:53.746 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-19T04:44:53.746 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-19T04:44:53.746 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-19T04:44:53.746 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-19T04:44:53.747 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-19T04:44:53.747 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-19T04:44:53.747 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-19T04:44:53.747 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-19T04:44:53.747 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-19T04:44:53.747 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-19T04:44:53.747 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-19T04:44:53.747 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-19T04:44:53.747 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-19T04:44:53.747 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-19T04:44:53.747 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-19T04:44:53.747 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-19T04:44:53.747 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-19T04:44:53.747 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-19T04:44:53.747 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-19T04:44:53.747 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-19T04:44:53.748 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-19T04:44:53.748 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-19T04:44:53.748 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-19T04:44:53.748 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-19T04:44:53.748 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-19T04:44:53.748 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-19T04:44:53.748 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-19T04:44:53.748 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-19T04:44:53.748 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-19T04:44:53.748 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-19T04:44:53.748 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-19T04:44:53.748 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-19T04:44:53.748 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-19T04:44:53.748 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-19T04:44:53.748 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-19T04:44:53.749 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-19T04:44:53.749 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-19T04:44:53.749 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-19T04:44:53.749 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-19T04:44:53.749 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-19T04:44:53.749 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-19T04:44:53.749 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-19T04:44:53.749 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-19T04:44:53.749 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-19T04:44:53.749 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-19T04:44:53.749 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-19T04:44:53.750 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-19T04:44:53.750 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-19T04:44:53.750 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-19T04:44:53.750 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-19T04:44:53.750 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-19T04:44:53.750 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-19T04:44:53.750 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-19T04:44:53.750 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-19T04:44:53.750 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-19T04:44:53.750 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-19T04:44:53.751 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-19T04:44:53.751 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-19T04:44:53.751 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-19T04:44:53.751 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-19T04:44:53.751 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-19T04:44:53.751 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-19T04:44:53.751 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-19T04:44:53.751 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-19T04:44:53.751 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-19T04:44:53.751 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-19T04:44:53.752 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-19T04:44:53.752 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-19T04:44:53.752 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-19T04:44:53.752 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-19T04:44:53.752 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-19T04:44:53.752 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-19T04:44:53.752 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-19T04:44:53.752 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-19T04:44:53.752 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-19T04:44:53.752 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-19T04:44:53.752 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-19T04:44:53.753 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-19T04:44:53.753 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-19T04:44:53.753 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-19T04:44:53.753 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-19T04:44:53.753 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-19T04:44:53.753 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-19T04:44:53.753 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-19T04:44:53.753 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-19T04:44:53.753 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-19T04:44:53.754 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-19T04:44:53.754 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-19T04:44:53.754 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-19T04:44:53.754 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-19T04:44:53.754 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-19T04:44:53.754 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-19T04:44:53.754 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-19T04:44:53.754 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-19T04:44:53.754 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-19T04:44:53.754 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-19T04:44:53.754 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-19T04:44:53.754 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-19T04:44:53.754 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-19T04:44:53.755 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-19T04:44:53.755 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-19T04:44:53.755 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-19T04:44:53.755 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-19T04:44:53.755 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-19T04:44:53.755 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-19T04:44:53.755 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-19T04:44:53.755 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-19T04:44:53.755 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-19T04:44:53.755 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-19T04:44:53.755 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-19T04:44:53.755 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-19T04:44:53.756 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-19T04:44:53.756 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-19T04:44:53.756 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-19T04:44:53.756 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-19T04:44:53.756 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-19T04:44:53.756 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-19T04:44:53.756 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-19T04:44:53.756 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-19T04:44:53.756 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-19T04:44:53.756 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-19T04:44:53.756 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-19T04:44:53.756 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-19T04:44:53.756 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-19T04:44:53.756 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-19T04:44:53.757 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-19T04:44:53.757 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-19T04:44:53.757 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-19T04:44:53.757 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-19T04:44:53.757 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-19T04:44:53.757 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-19T04:44:53.757 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-19T04:44:53.757 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-19T04:44:53.757 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-19T04:44:53.757 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-19T04:44:53.757 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-19T04:44:53.757 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-19T04:44:53.757 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-19T04:44:53.757 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-19T04:44:53.757 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-19T04:44:53.758 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-19T04:44:53.758 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-19T04:44:53.758 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-19T04:44:53.758 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-19T04:44:53.758 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-19T04:44:53.758 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-19T04:44:53.758 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-19T04:44:53.758 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-19T04:44:53.758 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-19T04:44:53.758 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-19T04:44:53.758 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-19T04:44:53.758 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-19T04:44:53.758 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-19T04:44:53.758 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-19T04:44:53.758 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-19T04:44:53.758 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-19T04:44:53.759 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-19T04:44:53.759 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-19T04:44:53.759 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-19T04:44:53.759 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-19T04:44:53.759 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-19T04:44:53.759 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-19T04:44:53.759 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-19T04:44:53.759 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-19T04:44:53.759 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-19T04:44:53.759 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-19T04:44:53.759 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-19T04:44:53.759 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-19T04:44:53.759 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-19T04:44:53.759 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-19T04:44:53.760 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-19T04:44:53.760 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-19T04:44:53.760 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-19T04:44:53.760 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-19T04:44:53.760 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-19T04:44:53.760 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-19T04:44:53.760 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-19T04:44:53.760 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-19T04:44:53.760 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-19T04:44:53.760 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-19T04:44:53.760 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-19T04:44:53.760 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-19T04:44:53.760 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-19T04:44:53.760 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-19T04:44:53.761 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-19T04:44:53.761 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-19T04:44:53.761 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-19T04:44:53.761 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-19T04:44:53.761 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-19T04:44:53.761 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-19T04:44:53.761 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-19T04:44:53.761 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-19T04:44:53.761 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-19T04:44:53.761 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-19T04:44:53.762 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-19T04:44:53.762 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-19T04:44:53.762 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-19T04:44:53.762 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-19T04:44:53.762 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-19T04:44:53.762 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-19T04:44:53.762 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-19T04:44:53.762 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-19T04:44:53.762 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-19T04:44:53.762 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-19T04:44:53.762 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-19T04:44:53.762 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-19T04:44:53.762 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-19T04:44:53.763 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-19T04:44:53.763 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-19T04:44:53.763 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-19T04:44:53.763 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-19T04:44:53.763 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-19T04:44:53.763 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-19T04:44:53.763 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-19T04:44:53.763 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-19T04:44:53.763 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-19T04:44:53.763 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-19T04:44:53.763 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-19T04:44:53.763 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-19T04:44:53.763 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-19T04:44:53.763 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-19T04:44:53.764 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-19T04:44:53.764 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-19T04:44:53.764 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-19T04:44:53.764 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-19T04:44:53.764 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-19T04:44:53.764 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-19T04:44:53.764 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-19T04:44:53.764 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-19T04:44:53.764 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-19T04:44:53.764 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-19T04:44:53.764 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-19T04:44:53.765 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-19T04:44:53.765 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-19T04:44:53.765 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-19T04:44:53.765 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-19T04:44:53.765 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-19T04:44:53.765 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-19T04:44:53.765 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-19T04:44:53.765 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-19T04:44:53.765 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-19T04:44:53.765 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-19T04:44:53.765 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-19T04:44:53.765 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-19T04:44:53.765 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-19T04:44:53.765 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-19T04:44:53.766 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-19T04:44:53.766 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-19T04:44:53.766 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-19T04:44:53.766 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-19T04:44:53.766 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-19T04:44:53.766 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-19T04:44:53.766 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-19T04:44:53.766 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-19T04:44:53.766 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-19T04:44:53.766 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-19T04:44:53.766 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-19T04:44:53.766 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-19T04:44:53.766 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-19T04:44:53.766 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-19T04:44:53.767 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-19T04:44:53.767 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-19T04:44:53.767 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-19T04:44:53.767 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-19T04:44:53.767 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-19T04:44:53.767 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-19T04:44:53.767 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-19T04:44:53.767 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-19T04:44:53.767 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-19T04:44:53.767 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-19T04:44:53.767 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-19T04:44:53.768 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-19T04:44:53.768 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-19T04:44:53.768 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-19T04:44:53.768 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-19T04:44:53.768 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-19T04:44:53.768 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-19T04:44:53.768 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-19T04:44:53.768 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-19T04:44:53.768 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-19T04:44:53.768 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-19T04:44:53.768 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-19T04:44:53.768 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-19T04:44:53.768 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-19T04:44:53.769 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-19T04:44:53.769 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-19T04:44:53.769 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-19T04:44:53.769 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-19T04:44:53.769 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-19T04:44:53.769 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-19T04:44:53.769 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-19T04:44:53.769 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-19T04:44:53.769 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-19T04:44:53.769 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-19T04:44:53.769 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-19T04:44:53.769 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-19T04:44:53.770 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-19T04:44:53.770 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-19T04:44:53.770 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-19T04:44:53.770 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-19T04:44:53.770 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-19T04:44:53.770 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-19T04:44:53.770 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-19T04:44:53.770 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-19T04:44:53.770 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-19T04:44:53.770 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-19T04:44:53.770 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-19T04:44:53.770 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-19T04:44:53.770 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-19T04:44:53.771 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-19T04:44:53.771 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-19T04:44:53.771 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-19T04:44:53.771 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-19T04:44:53.771 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-19T04:44:53.771 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-19T04:44:53.771 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-19T04:44:53.771 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-19T04:44:53.771 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-19T04:44:53.771 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-19T04:44:53.771 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-19T04:44:53.771 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-19T04:44:53.771 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-19T04:44:53.771 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-19T04:44:53.771 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-19T04:44:53.772 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-19T04:44:53.772 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-19T04:44:53.772 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-19T04:44:53.772 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-19T04:44:53.772 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-19T04:44:53.772 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-19T04:44:53.772 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-19T04:44:53.772 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-19T04:44:53.772 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-19T04:44:53.772 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-19T04:44:53.772 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-19T04:44:53.772 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-19T04:44:53.772 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-19T04:44:53.772 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-19T04:44:53.772 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-19T04:44:53.773 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-19T04:44:53.773 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-19T04:44:53.773 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-19T04:44:53.773 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-19T04:44:53.773 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-19T04:44:53.773 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-19T04:44:53.773 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-19T04:44:53.773 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-19T04:44:53.773 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-19T04:44:53.773 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-19T04:44:53.773 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-19T04:44:53.774 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-19T04:44:53.774 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-19T04:44:53.774 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-19T04:44:53.774 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-19T04:44:53.774 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-19T04:44:53.774 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-19T04:44:53.774 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-19T04:44:53.774 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-19T04:44:53.774 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-19T04:44:53.774 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-19T04:44:53.774 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-19T04:44:53.774 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-19T04:44:53.774 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-19T04:44:53.775 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-19T04:44:53.775 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-19T04:44:53.775 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-19T04:44:53.775 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-19T04:44:53.775 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-19T04:44:53.775 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-19T04:44:53.775 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-19T04:44:53.775 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-19T04:44:53.775 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-19T04:44:53.775 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-19T04:44:53.776 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-19T04:44:53.776 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-19T04:44:53.776 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-19T04:44:53.776 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-19T04:44:53.776 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-19T04:44:53.776 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-19T04:44:53.776 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-19T04:44:53.776 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-19T04:44:53.776 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-19T04:44:53.776 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-19T04:44:53.776 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-19T04:44:53.776 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-19T04:44:53.777 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-19T04:44:53.777 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-19T04:44:53.777 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-19T04:44:53.777 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-19T04:44:53.777 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-19T04:44:53.777 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-19T04:44:53.777 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-19T04:44:53.777 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-19T04:44:53.777 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-19T04:44:53.777 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-19T04:44:53.777 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-19T04:44:53.777 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-19T04:44:53.778 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-19T04:44:53.778 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-19T04:44:53.778 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-19T04:44:53.778 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-19T04:44:53.778 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-19T04:44:53.778 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-19T04:44:53.778 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-19T04:44:53.778 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-19T04:44:53.778 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-19T04:44:53.778 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-19T04:44:53.778 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-19T04:44:53.778 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-19T04:44:53.778 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-19T04:44:53.778 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-19T04:44:53.779 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-19T04:44:53.779 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-19T04:44:53.779 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-19T04:44:53.779 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-19T04:44:53.779 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-19T04:44:53.779 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-19T04:44:53.779 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-19T04:44:53.779 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-19T04:44:53.779 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-19T04:44:53.779 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-19T04:44:53.779 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-19T04:44:53.779 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-19T04:44:53.779 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-19T04:44:53.779 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-19T04:44:53.780 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-19T04:44:53.780 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-19T04:44:53.780 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-19T04:44:53.780 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-19T04:44:53.780 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-19T04:44:53.780 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-19T04:44:53.780 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-19T04:44:53.780 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-19T04:44:53.780 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-19T04:44:53.780 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-19T04:44:53.781 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-19T04:44:53.781 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-19T04:44:53.781 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-19T04:44:53.781 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-19T04:44:53.781 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-19T04:44:53.781 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-19T04:44:53.781 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-19T04:44:53.781 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-19T04:44:53.782 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-19T04:44:53.782 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-19T04:44:53.782 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-19T04:44:53.782 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-19T04:44:53.782 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-19T04:44:53.783 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-19T04:44:53.783 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-19T04:44:53.783 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-19T04:44:53.783 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-19T04:44:53.783 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-19T04:44:53.784 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-19T04:44:53.784 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-19T04:44:53.784 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-19T04:44:53.784 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-19T04:44:53.784 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-19T04:44:53.784 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-19T04:44:53.784 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-19T04:44:53.784 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-19T04:44:53.784 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-19T04:44:53.784 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-19T04:44:53.784 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-19T04:44:53.785 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-19T04:44:53.785 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-19T04:44:53.785 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-19T04:44:53.785 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-19T04:44:53.785 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-19T04:44:53.785 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-19T04:44:53.785 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-19T04:44:53.785 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-19T04:44:53.785 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-19T04:44:53.785 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-19T04:44:53.785 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-19T04:44:53.785 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-19T04:44:53.785 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-19T04:44:53.785 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-19T04:44:53.786 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-19T04:44:53.786 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-19T04:44:53.786 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-19T04:44:53.786 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-19T04:44:53.786 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-19T04:44:53.786 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-19T04:44:53.786 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-19T04:44:53.786 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-19T04:44:53.786 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-19T04:44:53.786 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-19T04:44:53.786 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-19T04:44:53.786 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-19T04:44:53.786 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-19T04:44:53.786 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-19T04:44:53.786 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-19T04:44:53.786 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-19T04:44:53.787 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-19T04:44:53.787 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-19T04:44:53.787 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-19T04:44:53.787 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-19T04:44:53.787 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-19T04:44:53.787 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-19T04:44:53.787 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-19T04:44:53.787 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-19T04:44:53.787 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-19T04:44:53.787 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-19T04:44:53.787 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-19T04:44:53.787 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-19T04:44:53.787 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-19T04:44:53.788 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-19T04:44:53.788 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-19T04:44:53.788 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-19T04:44:53.788 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-19T04:44:53.788 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-19T04:44:53.788 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-19T04:44:53.788 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-19T04:44:53.788 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-19T04:44:53.788 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-19T04:44:53.788 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-19T04:44:53.788 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-19T04:44:53.788 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-19T04:44:53.789 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-19T04:44:53.789 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-19T04:44:53.789 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-19T04:44:53.789 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-19T04:44:53.789 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-19T04:44:53.789 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-19T04:44:53.789 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-19T04:44:53.789 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-19T04:44:53.789 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-19T04:44:53.789 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-19T04:44:53.789 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-19T04:44:53.789 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-19T04:44:53.789 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-19T04:44:53.789 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-19T04:44:53.790 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-19T04:44:53.790 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-19T04:44:53.790 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-19T04:44:53.790 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-19T04:44:53.790 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-19T04:44:53.790 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-19T04:44:53.791 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-19T04:44:53.791 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-19T04:44:53.791 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-19T04:44:53.791 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-19T04:44:53.791 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-19T04:44:53.792 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-19T04:44:53.792 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-19T04:44:53.792 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-19T04:44:53.792 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-19T04:44:53.792 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-19T04:44:53.792 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-19T04:44:53.792 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-19T04:44:53.793 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-19T04:44:53.793 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-19T04:44:53.793 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-19T04:44:53.793 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-19T04:44:53.793 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-19T04:44:53.793 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-19T04:44:53.793 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-19T04:44:53.793 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-19T04:44:53.793 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-19T04:44:53.793 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-19T04:44:53.793 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-19T04:44:53.793 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-19T04:44:53.793 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-19T04:44:53.794 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-19T04:44:53.794 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-19T04:44:53.794 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-19T04:44:53.794 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-19T04:44:53.794 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-19T04:44:53.794 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-19T04:44:53.794 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-19T04:44:53.794 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-19T04:44:53.794 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-19T04:44:53.794 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-19T04:44:53.794 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-19T04:44:53.794 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-19T04:44:53.794 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-19T04:44:53.794 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-19T04:44:53.794 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-19T04:44:53.795 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-19T04:44:53.795 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-19T04:44:53.795 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-19T04:44:53.795 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-19T04:44:53.795 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-19T04:44:53.795 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-19T04:44:53.795 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-19T04:44:53.795 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-19T04:44:53.795 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-19T04:44:53.795 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-19T04:44:53.796 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-19T04:44:53.796 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-19T04:44:53.796 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-19T04:44:53.796 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-19T04:44:53.796 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-19T04:44:53.796 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-19T04:44:53.796 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-19T04:44:53.796 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-19T04:44:53.796 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-19T04:44:53.797 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-19T04:44:53.797 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-19T04:44:53.797 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-19T04:44:53.797 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-19T04:44:53.797 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-19T04:44:53.797 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-19T04:44:53.798 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-19T04:44:53.798 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-19T04:44:53.798 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-19T04:44:53.798 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-19T04:44:53.798 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-19T04:44:53.798 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-19T04:44:53.798 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-19T04:44:53.798 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-19T04:44:53.798 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-19T04:44:53.798 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-19T04:44:53.798 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-19T04:44:53.798 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-19T04:44:53.798 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-19T04:44:53.799 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-19T04:44:53.799 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-19T04:44:53.799 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-19T04:44:53.799 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-19T04:44:53.799 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-19T04:44:53.800 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-19T04:44:53.800 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-19T04:44:53.800 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-19T04:44:53.800 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-19T04:44:53.800 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-19T04:44:53.800 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-19T04:44:53.800 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-19T04:44:53.800 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-19T04:44:53.800 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-19T04:44:53.800 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-19T04:44:53.800 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-19T04:44:53.801 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-19T04:44:53.801 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-19T04:44:53.801 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-19T04:44:53.801 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-19T04:44:53.801 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-19T04:44:53.801 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-19T04:44:53.801 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-19T04:44:53.801 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-19T04:44:53.801 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-19T04:44:53.801 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-19T04:44:53.801 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-19T04:44:53.801 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-19T04:44:53.801 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-19T04:44:53.801 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-19T04:44:53.802 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-19T04:44:53.802 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-19T04:44:53.802 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-19T04:44:53.802 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-19T04:44:53.802 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-19T04:44:53.802 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-19T04:44:53.802 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-19T04:44:53.802 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-19T04:44:53.802 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-19T04:44:53.802 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-19T04:44:53.802 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-19T04:44:53.802 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-19T04:44:53.802 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-19T04:44:53.802 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-19T04:44:53.802 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-19T04:44:53.803 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-19T04:44:53.803 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-19T04:44:53.803 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-19T04:44:53.803 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-19T04:44:53.803 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-19T04:44:53.803 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-19T04:44:53.803 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-19T04:44:53.803 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-19T04:44:53.803 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-19T04:44:53.803 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-19T04:44:53.803 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-19T04:44:53.803 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-19T04:44:53.803 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-19T04:44:53.803 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-19T04:44:53.804 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-19T04:44:53.804 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-19T04:44:53.804 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-19T04:44:53.804 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-19T04:44:53.804 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-19T04:44:53.804 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-19T04:44:53.804 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-19T04:44:53.804 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-19T04:44:53.804 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-19T04:44:53.804 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-19T04:44:53.804 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-19T04:44:53.805 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-19T04:44:53.805 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-19T04:44:53.805 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-19T04:44:53.805 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-19T04:44:53.805 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-19T04:44:53.805 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-19T04:44:53.806 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-19T04:44:53.806 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-19T04:44:53.806 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-19T04:44:53.806 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-19T04:44:53.806 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-19T04:44:53.806 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-19T04:44:53.806 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-19T04:44:53.806 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-19T04:44:53.807 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-19T04:44:53.807 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-19T04:44:53.807 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-19T04:44:53.807 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-19T04:44:53.807 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-19T04:44:53.807 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-19T04:44:53.807 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-19T04:44:53.807 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-19T04:44:53.807 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-19T04:44:53.807 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-19T04:44:53.807 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-19T04:44:53.807 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-19T04:44:53.807 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-19T04:44:53.808 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-19T04:44:53.808 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-19T04:44:53.808 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-19T04:44:53.808 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-19T04:44:53.808 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-19T04:44:53.808 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-19T04:44:53.809 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-19T04:44:53.809 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-19T04:44:53.809 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-19T04:44:53.809 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-19T04:44:53.809 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-19T04:44:53.809 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-19T04:44:53.809 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-19T04:44:53.809 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-19T04:44:53.809 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-19T04:44:53.809 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-19T04:44:53.809 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-19T04:44:53.809 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-19T04:44:53.809 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-19T04:44:53.810 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-19T04:44:53.810 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-19T04:44:53.810 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-19T04:44:53.810 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-19T04:44:53.810 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-19T04:44:53.810 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-19T04:44:53.810 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-19T04:44:53.810 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-19T04:44:53.810 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-19T04:44:53.810 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-19T04:44:53.810 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-19T04:44:53.810 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-19T04:44:53.810 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-19T04:44:53.810 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-19T04:44:53.811 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-19T04:44:53.811 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-19T04:44:53.811 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-19T04:44:53.811 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-19T04:44:53.811 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-19T04:44:53.811 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-19T04:44:53.811 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-19T04:44:53.811 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-19T04:44:53.811 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-19T04:44:53.811 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-19T04:44:53.811 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-19T04:44:53.811 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-19T04:44:53.811 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-19T04:44:53.811 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-19T04:44:53.812 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-19T04:44:53.812 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-19T04:44:53.812 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-19T04:44:53.812 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-19T04:44:53.812 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-19T04:44:53.812 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-19T04:44:53.812 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-19T04:44:53.812 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-19T04:44:53.812 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-19T04:44:53.812 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-19T04:44:53.812 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-19T04:44:53.812 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-19T04:44:53.812 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-19T04:44:53.812 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-19T04:44:53.813 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-19T04:44:53.813 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-19T04:44:53.813 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-19T04:44:53.813 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-19T04:44:53.813 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-19T04:44:53.813 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-19T04:44:53.813 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-19T04:44:53.813 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-19T04:44:53.813 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-19T04:44:53.813 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-19T04:44:53.813 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-19T04:44:53.813 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-19T04:44:53.813 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-19T04:44:53.814 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-19T04:44:53.814 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-19T04:44:53.814 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-19T04:44:53.814 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-19T04:44:53.814 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-19T04:44:53.814 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-19T04:44:53.814 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-19T04:44:53.814 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-19T04:44:53.814 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-19T04:44:53.814 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-19T04:44:53.814 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-19T04:44:53.814 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-19T04:44:53.814 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-19T04:44:53.814 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-19T04:44:53.815 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-19T04:44:53.815 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-19T04:44:53.815 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-19T04:44:53.815 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-19T04:44:53.815 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-19T04:44:53.815 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-19T04:44:53.815 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-19T04:44:53.815 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-19T04:44:53.815 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-19T04:44:53.815 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-19T04:44:53.815 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-19T04:44:53.815 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-19T04:44:53.816 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-19T04:44:53.816 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-19T04:44:53.816 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-19T04:44:53.816 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-19T04:44:53.816 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-19T04:44:53.816 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-19T04:44:53.816 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-19T04:44:53.816 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-19T04:44:53.816 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-19T04:44:53.816 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-19T04:44:53.816 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-19T04:44:53.817 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-19T04:44:53.817 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-19T04:44:53.817 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-19T04:44:53.817 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-19T04:44:53.817 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-19T04:44:53.817 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-19T04:44:53.817 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-19T04:44:53.817 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-19T04:44:53.817 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-19T04:44:53.817 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-19T04:44:53.817 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-19T04:44:53.817 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-19T04:44:53.817 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-19T04:44:53.818 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-19T04:44:53.818 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-19T04:44:53.818 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-19T04:44:53.818 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-19T04:44:53.818 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-19T04:44:53.818 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-19T04:44:53.818 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-19T04:44:53.818 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-19T04:44:53.818 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-19T04:44:53.818 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-19T04:44:53.818 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-19T04:44:53.818 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-19T04:44:53.818 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-19T04:44:53.818 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-19T04:44:53.819 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-19T04:44:53.819 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-19T04:44:53.819 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-19T04:44:53.819 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-19T04:44:53.819 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-19T04:44:53.819 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-19T04:44:53.819 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-19T04:44:53.819 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-19T04:44:53.819 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-19T04:44:53.819 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-19T04:44:53.819 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-19T04:44:53.819 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-19T04:44:53.819 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-19T04:44:53.819 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-19T04:44:53.819 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-19T04:44:53.820 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-19T04:44:53.820 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-19T04:44:53.820 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-19T04:44:53.820 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-19T04:44:53.820 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-19T04:44:53.820 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-19T04:44:53.820 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-19T04:44:53.820 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-19T04:44:53.820 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-19T04:44:53.820 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-19T04:44:53.820 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-19T04:44:53.820 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-19T04:44:53.820 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-19T04:44:53.821 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-19T04:44:53.821 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-19T04:44:53.821 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-19T04:44:53.821 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-19T04:44:53.821 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-19T04:44:53.821 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-19T04:44:53.821 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-19T04:44:53.821 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-19T04:44:53.821 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-19T04:44:53.821 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-19T04:44:53.821 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-19T04:44:53.821 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-19T04:44:53.821 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-19T04:44:53.821 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-19T04:44:53.822 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-19T04:44:53.822 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-19T04:44:53.822 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-19T04:44:53.822 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-19T04:44:53.822 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-19T04:44:53.822 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-19T04:44:53.822 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-19T04:44:53.822 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-19T04:44:53.822 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-19T04:44:53.822 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-19T04:44:53.822 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-19T04:44:53.822 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-19T04:44:53.822 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-19T04:44:53.822 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-19T04:44:53.823 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-19T04:44:53.823 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-19T04:44:53.823 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-19T04:44:53.823 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-19T04:44:53.823 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-19T04:44:53.823 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-19T04:44:53.823 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-19T04:44:53.823 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-19T04:44:53.823 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-19T04:44:53.823 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-19T04:44:53.823 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-19T04:44:53.823 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-19T04:44:53.823 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-19T04:44:53.823 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-19T04:44:53.824 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-19T04:44:53.824 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-19T04:44:53.824 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-19T04:44:53.824 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-19T04:44:53.824 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-19T04:44:53.824 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-19T04:44:53.824 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-19T04:44:53.824 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-19T04:44:53.824 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-19T04:44:53.824 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-19T04:44:53.824 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-19T04:44:53.824 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-19T04:44:53.824 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-19T04:44:53.825 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-19T04:44:53.825 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-19T04:44:53.825 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-19T04:44:53.825 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-19T04:44:53.825 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-19T04:44:53.825 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-19T04:44:53.825 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-19T04:44:53.825 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-19T04:44:53.825 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-19T04:44:53.826 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-19T04:44:53.826 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-19T04:44:53.826 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-19T04:44:53.826 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-19T04:44:53.826 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-19T04:44:53.826 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-19T04:44:53.826 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-19T04:44:53.826 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-19T04:44:53.827 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-19T04:44:53.827 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-19T04:44:53.827 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-19T04:44:53.827 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-19T04:44:53.827 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-19T04:44:53.827 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-19T04:44:53.827 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-19T04:44:53.827 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-19T04:44:53.828 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-19T04:44:53.828 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-19T04:44:53.828 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-19T04:44:53.828 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-19T04:44:53.828 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-19T04:44:53.828 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-19T04:44:53.828 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-19T04:44:53.828 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-19T04:44:53.828 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-19T04:44:53.829 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-19T04:44:53.829 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-19T04:44:53.829 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-19T04:44:53.829 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-19T04:44:53.829 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-19T04:44:53.829 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-19T04:44:53.829 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-19T04:44:53.829 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-19T04:44:53.830 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-19T04:44:53.830 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-19T04:44:53.830 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-19T04:44:53.830 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-19T04:44:53.830 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-19T04:44:53.830 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-19T04:44:53.830 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-19T04:44:53.830 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-19T04:44:53.831 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-19T04:44:53.831 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-19T04:44:53.831 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-19T04:44:53.831 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-19T04:44:53.831 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-19T04:44:53.831 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-19T04:44:53.831 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-19T04:44:53.831 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-19T04:44:53.831 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-19T04:44:53.832 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-19T04:44:53.832 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-19T04:44:53.832 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-19T04:44:53.832 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-19T04:44:53.832 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-19T04:44:53.832 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-19T04:44:53.832 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-19T04:44:53.832 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-19T04:44:53.833 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-19T04:44:53.833 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-19T04:44:53.833 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-19T04:44:53.833 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-19T04:44:53.833 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-19T04:44:53.833 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-19T04:44:53.833 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-19T04:44:53.833 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-19T04:44:53.833 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-19T04:44:53.834 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-19T04:44:53.834 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-19T04:44:53.834 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-19T04:44:53.834 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-19T04:44:53.834 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-19T04:44:53.834 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-19T04:44:53.834 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-19T04:44:53.834 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-19T04:44:53.834 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-19T04:44:53.835 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-19T04:44:53.835 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-19T04:44:53.835 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-19T04:44:53.835 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-19T04:44:53.835 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-19T04:44:53.835 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-19T04:44:53.835 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-19T04:44:53.835 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-19T04:44:53.835 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-19T04:44:53.836 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-19T04:44:53.836 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-19T04:44:53.836 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-19T04:44:53.836 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-19T04:44:53.836 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-19T04:44:53.836 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-19T04:44:53.836 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-19T04:44:53.836 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-19T04:44:53.836 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-19T04:44:53.837 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-19T04:44:53.837 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-19T04:44:53.837 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-19T04:44:53.837 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-19T04:44:53.837 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-19T04:44:53.837 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-19T04:44:53.837 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-19T04:44:53.837 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-19T04:44:53.838 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-19T04:44:53.838 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-19T04:44:53.838 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-19T04:44:53.838 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-19T04:44:53.838 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-19T04:44:53.838 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-19T04:44:53.838 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-19T04:44:53.838 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-19T04:44:53.838 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-19T04:44:53.839 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-19T04:44:53.839 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-19T04:44:53.839 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-19T04:44:53.839 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-19T04:44:53.839 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-19T04:44:53.839 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-19T04:44:53.839 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-19T04:44:53.839 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-19T04:44:53.839 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-19T04:44:53.840 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-19T04:44:53.840 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-19T04:44:53.840 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-19T04:44:53.840 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-19T04:44:53.840 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-19T04:44:53.840 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-19T04:44:53.840 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-19T04:44:53.840 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-19T04:44:53.840 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-19T04:44:53.840 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-19T04:44:53.840 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-19T04:44:53.840 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-19T04:44:53.840 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-19T04:44:53.840 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-19T04:44:53.841 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-19T04:44:53.841 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-19T04:44:53.841 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-19T04:44:53.841 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-19T04:44:53.841 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-19T04:44:53.841 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-19T04:44:53.841 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-19T04:44:53.841 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-19T04:44:53.841 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-19T04:44:53.841 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-19T04:44:53.841 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-19T04:44:53.841 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-19T04:44:53.841 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-19T04:44:53.841 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-19T04:44:53.842 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-19T04:44:53.842 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-19T04:44:53.842 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-19T04:44:53.842 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-19T04:44:53.842 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-19T04:44:53.842 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-19T04:44:53.842 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-19T04:44:53.842 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-19T04:44:53.842 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-19T04:44:53.842 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-19T04:44:53.842 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-19T04:44:53.842 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-19T04:44:53.842 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-19T04:44:53.842 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-19T04:44:53.843 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-19T04:44:53.843 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-19T04:44:53.843 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-19T04:44:53.843 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-19T04:44:53.843 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-19T04:44:53.843 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-19T04:44:53.843 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-19T04:44:53.843 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-19T04:44:53.843 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-19T04:44:53.843 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-19T04:44:53.843 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-19T04:44:53.843 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-19T04:44:53.844 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-19T04:44:53.844 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-19T04:44:53.844 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-19T04:44:53.844 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-19T04:44:53.844 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-19T04:44:53.844 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-19T04:44:53.844 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-19T04:44:53.844 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-19T04:44:53.844 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-19T04:44:53.844 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-19T04:44:53.844 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-19T04:44:53.844 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-19T04:44:53.844 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-19T04:44:53.845 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-19T04:44:53.845 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-19T04:44:53.845 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-19T04:44:53.845 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-19T04:44:53.845 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-19T04:44:53.845 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-19T04:44:53.845 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-19T04:44:53.845 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-19T04:44:53.845 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-19T04:44:53.845 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-19T04:44:53.845 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-19T04:44:53.845 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-19T04:44:53.845 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-19T04:44:53.846 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-19T04:44:53.846 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-19T04:44:53.846 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-19T04:44:53.846 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-19T04:44:53.846 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-19T04:44:53.846 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-19T04:44:53.846 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-19T04:44:53.846 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-19T04:44:53.846 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-19T04:44:53.846 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-19T04:44:53.846 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-19T04:44:53.846 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-19T04:44:53.846 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-19T04:44:53.847 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-19T04:44:53.847 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-19T04:44:53.847 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-19T04:44:53.847 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-19T04:44:53.847 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-19T04:44:53.848 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-19T04:44:53.848 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-19T04:44:53.848 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-19T04:44:53.848 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-19T04:44:53.848 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-19T04:44:53.849 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-19T04:44:53.849 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-19T04:44:53.849 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-19T04:44:53.849 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-19T04:44:53.849 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-19T04:44:53.849 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-19T04:44:53.850 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-19T04:44:53.850 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-19T04:44:53.850 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-19T04:44:53.850 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-19T04:44:53.850 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-19T04:44:53.850 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-19T04:44:53.850 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-19T04:44:53.850 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-19T04:44:53.850 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-19T04:44:53.850 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-19T04:44:53.850 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-19T04:44:53.850 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-19T04:44:53.850 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-19T04:44:53.850 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-19T04:44:53.851 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-19T04:44:53.851 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-19T04:44:53.851 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-19T04:44:53.851 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-19T04:44:53.851 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-19T04:44:53.852 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-19T04:44:53.852 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-19T04:44:53.852 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-19T04:44:53.852 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-19T04:44:53.852 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-19T04:44:53.852 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-19T04:44:53.852 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-19T04:44:53.852 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-19T04:44:53.852 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-19T04:44:53.852 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-19T04:44:53.852 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-19T04:44:53.852 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-19T04:44:53.852 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-19T04:44:53.853 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-19T04:44:53.853 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-19T04:44:53.853 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-19T04:44:53.853 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-19T04:44:53.853 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-19T04:44:53.853 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-19T04:44:53.853 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-19T04:44:53.853 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-19T04:44:53.853 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-19T04:44:53.853 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-19T04:44:53.853 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-19T04:44:53.853 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-19T04:44:53.853 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-19T04:44:53.854 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-19T04:44:53.854 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-19T04:44:53.854 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-19T04:44:53.854 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-19T04:44:53.854 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-19T04:44:53.854 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-19T04:44:53.854 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-19T04:44:53.854 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-19T04:44:53.854 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-19T04:44:53.854 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-19T04:44:53.854 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-19T04:44:53.854 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-19T04:44:53.854 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-19T04:44:53.854 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-19T04:44:53.855 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-19T04:44:53.855 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-19T04:44:53.855 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-19T04:44:53.855 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-19T04:44:53.855 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-19T04:44:53.855 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-19T04:44:53.855 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-19T04:44:53.855 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-19T04:44:53.855 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-19T04:44:53.855 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-19T04:44:53.855 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-19T04:44:53.856 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-19T04:44:53.856 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-19T04:44:53.856 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-19T04:44:53.856 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-19T04:44:53.856 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-19T04:44:53.856 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-19T04:44:53.856 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-19T04:44:53.856 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-19T04:44:53.856 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-19T04:44:53.856 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-19T04:44:53.856 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-19T04:44:53.856 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-19T04:44:53.857 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-19T04:44:53.857 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-19T04:44:53.857 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-19T04:44:53.857 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-19T04:44:53.857 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-19T04:44:53.857 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-19T04:44:53.857 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-19T04:44:53.857 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-19T04:44:53.857 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-19T04:44:53.857 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-19T04:44:53.857 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-19T04:44:53.857 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-19T04:44:53.858 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-19T04:44:53.858 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-19T04:44:53.858 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-19T04:44:53.858 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-19T04:44:53.858 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-19T04:44:53.858 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-19T04:44:53.858 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-19T04:44:53.858 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-19T04:44:53.858 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-19T04:44:53.858 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-19T04:44:53.858 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-19T04:44:53.858 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-19T04:44:53.858 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-19T04:44:53.859 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-19T04:44:53.859 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-19T04:44:53.859 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-19T04:44:53.859 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-19T04:44:53.859 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-19T04:44:53.859 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-19T04:44:53.859 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-19T04:44:53.859 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-19T04:44:53.859 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-19T04:44:53.859 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-19T04:44:53.859 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-19T04:44:53.859 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-19T04:44:53.859 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-19T04:44:53.859 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-19T04:44:53.860 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-19T04:44:53.860 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-19T04:44:53.860 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-19T04:44:53.860 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-19T04:44:53.860 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-19T04:44:53.860 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-19T04:44:53.860 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-19T04:44:53.860 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-19T04:44:53.860 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-19T04:44:53.860 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-19T04:44:53.860 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-19T04:44:53.860 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-19T04:44:53.860 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-19T04:44:53.860 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-19T04:44:53.861 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-19T04:44:53.861 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-19T04:44:53.861 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-19T04:44:53.861 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-19T04:44:53.861 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-19T04:44:53.861 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-19T04:44:53.861 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-19T04:44:53.861 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-19T04:44:53.861 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-19T04:44:53.861 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-19T04:44:53.861 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-19T04:44:53.861 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-19T04:44:53.861 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-19T04:44:53.861 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-19T04:44:53.862 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-19T04:44:53.862 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-19T04:44:53.862 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-19T04:44:53.862 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-19T04:44:53.862 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-19T04:44:53.862 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-19T04:44:53.862 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-19T04:44:53.862 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-19T04:44:53.862 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-19T04:44:53.862 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-19T04:44:53.862 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-19T04:44:53.862 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-19T04:44:53.862 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-19T04:44:53.862 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-19T04:44:53.863 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-19T04:44:53.863 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-19T04:44:53.863 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-19T04:44:53.863 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-19T04:44:53.863 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-19T04:44:53.863 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-19T04:44:53.863 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-19T04:44:53.863 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-19T04:44:53.863 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-19T04:44:53.863 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-19T04:44:53.863 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-19T04:44:53.863 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-19T04:44:53.863 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-19T04:44:53.863 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-19T04:44:53.863 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-19T04:44:53.864 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-19T04:44:53.864 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-19T04:44:53.864 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-19T04:44:53.864 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-19T04:44:53.864 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-19T04:44:53.864 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-19T04:44:53.864 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-19T04:44:53.864 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-19T04:44:53.864 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-19T04:44:53.864 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-19T04:44:53.864 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-19T04:44:53.864 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-19T04:44:53.865 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-19T04:44:53.865 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-19T04:44:53.865 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-19T04:44:53.865 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-19T04:44:53.865 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-19T04:44:53.865 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-19T04:44:53.865 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-19T04:44:53.865 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-19T04:44:53.865 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-19T04:44:53.865 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-19T04:44:53.865 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-19T04:44:53.865 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-19T04:44:53.865 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-19T04:44:53.866 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-19T04:44:53.866 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-19T04:44:53.866 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-19T04:44:53.866 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-19T04:44:53.866 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-19T04:44:53.866 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-19T04:44:53.866 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-19T04:44:53.866 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-19T04:44:53.866 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-19T04:44:53.866 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-19T04:44:53.866 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-19T04:44:53.866 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-19T04:44:53.866 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-19T04:44:53.866 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-19T04:44:53.867 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-19T04:44:53.867 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-19T04:44:53.867 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-19T04:44:53.867 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-19T04:44:53.867 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-19T04:44:53.867 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-19T04:44:53.867 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-19T04:44:53.867 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-19T04:44:53.867 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-19T04:44:53.867 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-19T04:44:53.867 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-19T04:44:53.867 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-19T04:44:53.868 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-19T04:44:53.868 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-19T04:44:53.868 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-19T04:44:53.868 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-19T04:44:53.868 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-19T04:44:53.868 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-19T04:44:53.868 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-19T04:44:53.868 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-19T04:44:53.868 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-19T04:44:53.868 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-19T04:44:53.868 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-19T04:44:53.868 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-19T04:44:53.869 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-19T04:44:53.869 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-19T04:44:53.869 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-19T04:44:53.869 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-19T04:44:53.869 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-19T04:44:53.869 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-19T04:44:53.869 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-19T04:44:53.869 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-19T04:44:53.869 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-19T04:44:53.869 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-19T04:44:53.869 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-19T04:44:53.869 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-19T04:44:53.869 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-19T04:44:53.869 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-19T04:44:53.869 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-19T04:44:53.870 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-19T04:44:53.870 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-19T04:44:53.870 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-19T04:44:53.870 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-19T04:44:53.870 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-19T04:44:53.870 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-19T04:44:53.870 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-19T04:44:53.870 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-19T04:44:53.870 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-19T04:44:53.870 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-19T04:44:53.870 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-19T04:44:53.871 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-19T04:44:53.871 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-19T04:44:53.871 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-19T04:44:53.871 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-19T04:44:53.871 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-19T04:44:53.871 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-19T04:44:53.871 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-19T04:44:53.871 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-19T04:44:53.871 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-19T04:44:53.871 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-19T04:44:53.871 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-19T04:44:53.871 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-19T04:44:53.871 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-19T04:44:53.871 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-19T04:44:53.871 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-19T04:44:53.871 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-19T04:44:53.872 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-19T04:44:53.872 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-19T04:44:53.872 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-19T04:44:53.872 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-19T04:44:53.872 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-19T04:44:53.872 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-19T04:44:53.872 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-19T04:44:53.872 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-19T04:44:53.872 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-19T04:44:53.872 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-19T04:44:53.872 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-19T04:44:53.872 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-19T04:44:53.872 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-19T04:44:53.872 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-19T04:44:53.872 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-19T04:44:53.872 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-19T04:44:53.873 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-19T04:44:53.873 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-19T04:44:53.873 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-19T04:44:53.873 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-19T04:44:53.873 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-19T04:44:53.873 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-19T04:44:53.873 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-19T04:44:53.873 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-19T04:44:53.873 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-19T04:44:53.873 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-19T04:44:53.873 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-19T04:44:53.873 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-19T04:44:53.873 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-19T04:44:53.873 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-19T04:44:53.874 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-19T04:44:53.874 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-19T04:44:53.874 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-19T04:44:53.874 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-19T04:44:53.874 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-19T04:44:53.874 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-19T04:44:53.874 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-19T04:44:53.874 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-19T04:44:53.874 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-19T04:44:53.874 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-19T04:44:53.874 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-19T04:44:53.875 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-19T04:44:53.875 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-19T04:44:53.875 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-19T04:44:53.875 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-19T04:44:53.875 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-19T04:44:53.875 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-19T04:44:53.875 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-19T04:44:53.875 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-19T04:44:53.875 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-19T04:44:53.875 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-19T04:44:53.875 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-19T04:44:53.875 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-19T04:44:53.875 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-19T04:44:53.875 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-19T04:44:53.875 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-19T04:44:53.875 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-19T04:44:53.876 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-19T04:44:53.876 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-19T04:44:53.876 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-19T04:44:53.876 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-19T04:44:53.876 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-19T04:44:53.876 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-19T04:44:53.876 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-19T04:44:53.876 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-19T04:44:53.876 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-19T04:44:53.876 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-19T04:44:53.876 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-19T04:44:53.876 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-19T04:44:53.876 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-19T04:44:53.876 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-19T04:44:53.876 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-19T04:44:53.877 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-19T04:44:53.877 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-19T04:44:53.877 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-19T04:44:53.877 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-19T04:44:53.877 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-19T04:44:53.877 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-19T04:44:53.877 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-19T04:44:53.877 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-19T04:44:53.877 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-19T04:44:53.877 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-19T04:44:53.877 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-19T04:44:53.877 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-19T04:44:53.877 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-19T04:44:53.877 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-19T04:44:53.878 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-19T04:44:53.878 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-19T04:44:53.878 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-19T04:44:53.878 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-19T04:44:53.878 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-19T04:44:53.878 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-19T04:44:53.878 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-19T04:44:53.878 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-19T04:44:53.878 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-19T04:44:53.878 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-19T04:44:53.878 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-19T04:44:53.878 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-19T04:44:53.878 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-19T04:44:53.878 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-19T04:44:53.879 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-19T04:44:53.879 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-19T04:44:53.879 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-19T04:44:53.879 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-19T04:44:53.879 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-19T04:44:53.879 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-19T04:44:53.879 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-19T04:44:53.879 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-19T04:44:53.879 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-19T04:44:53.879 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-19T04:44:53.879 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-19T04:44:53.879 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-19T04:44:53.879 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-19T04:44:53.880 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-19T04:44:53.880 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-19T04:44:53.880 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-19T04:44:53.880 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-19T04:44:53.880 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-19T04:44:53.880 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-19T04:44:53.880 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-19T04:44:53.880 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-19T04:44:53.880 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-19T04:44:53.880 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-19T04:44:53.880 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-19T04:44:53.880 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-19T04:44:53.881 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-19T04:44:53.881 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-19T04:44:53.881 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-19T04:44:53.881 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-19T04:44:53.881 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-19T04:44:53.881 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-19T04:44:53.881 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-19T04:44:53.881 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-19T04:44:53.881 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-19T04:44:53.881 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-19T04:44:53.881 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-19T04:44:53.881 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-19T04:44:53.881 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-19T04:44:53.881 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-19T04:44:53.881 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-19T04:44:53.882 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-19T04:44:53.882 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-19T04:44:53.882 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-19T04:44:53.882 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-19T04:44:53.882 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-19T04:44:53.882 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-19T04:44:53.882 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-19T04:44:53.882 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-19T04:44:53.882 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-19T04:44:53.882 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-19T04:44:53.882 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-19T04:44:53.882 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-19T04:44:53.882 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-19T04:44:53.882 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-19T04:44:53.883 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-19T04:44:53.883 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-19T04:44:53.883 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-19T04:44:53.883 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-19T04:44:53.883 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-19T04:44:53.883 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-19T04:44:53.883 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-19T04:44:53.883 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-19T04:44:53.883 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-19T04:44:53.883 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-19T04:44:53.883 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-19T04:44:53.883 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-19T04:44:53.883 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-19T04:44:53.883 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-19T04:44:53.884 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-19T04:44:53.884 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-19T04:44:53.884 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-19T04:44:53.884 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-19T04:44:53.884 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-19T04:44:53.884 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-19T04:44:53.884 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-19T04:44:53.884 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-19T04:44:53.884 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-19T04:44:53.884 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-19T04:44:53.884 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-19T04:44:53.884 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-19T04:44:53.884 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-19T04:44:53.884 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-19T04:44:53.885 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-19T04:44:53.885 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-19T04:44:53.885 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-19T04:44:53.885 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-19T04:44:53.885 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-19T04:44:53.885 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-19T04:44:53.885 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-19T04:44:53.885 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-19T04:44:53.885 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-19T04:44:53.885 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-19T04:44:53.885 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-19T04:44:53.885 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-19T04:44:53.886 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-19T04:44:53.886 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-19T04:44:53.886 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-19T04:44:53.886 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-19T04:44:53.886 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-19T04:44:53.886 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-19T04:44:53.886 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-19T04:44:53.886 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-19T04:44:53.886 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-19T04:44:53.886 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-19T04:44:53.886 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-19T04:44:53.886 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-19T04:44:53.886 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-19T04:44:53.887 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-19T04:44:53.887 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-19T04:44:53.887 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-19T04:44:53.887 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-19T04:44:53.887 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-19T04:44:53.887 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-19T04:44:53.887 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-19T04:44:53.887 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-19T04:44:53.887 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-19T04:44:53.887 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-19T04:44:53.887 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-19T04:44:53.887 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-19T04:44:53.887 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-19T04:44:53.888 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-19T04:44:53.888 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-19T04:44:53.888 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-19T04:44:53.888 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-19T04:44:53.888 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-19T04:44:53.888 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-19T04:44:53.888 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-19T04:44:53.888 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-19T04:44:53.888 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-19T04:44:53.888 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-19T04:44:53.888 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-19T04:44:53.888 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-19T04:44:53.888 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-19T04:44:53.888 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-19T04:44:53.889 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-19T04:44:53.889 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-19T04:44:53.889 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-19T04:44:53.889 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-19T04:44:53.889 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-19T04:44:53.889 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-19T04:44:53.889 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-19T04:44:53.889 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-19T04:44:53.889 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-19T04:44:53.889 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-19T04:44:53.889 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-19T04:44:53.889 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-19T04:44:53.889 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-19T04:44:53.890 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-19T04:44:53.890 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-19T04:44:53.890 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-19T04:44:53.890 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-19T04:44:53.890 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-19T04:44:53.890 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-19T04:44:53.890 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-19T04:44:53.890 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-19T04:44:53.890 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-19T04:44:53.890 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-19T04:44:53.890 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-19T04:44:53.890 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-19T04:44:53.890 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-19T04:44:53.890 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-19T04:44:53.891 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-19T04:44:53.891 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-19T04:44:53.891 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-19T04:44:53.891 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-19T04:44:53.891 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-19T04:44:53.891 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-19T04:44:53.891 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-19T04:44:53.891 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-19T04:44:53.891 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-19T04:44:53.891 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-19T04:44:53.891 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-19T04:44:53.892 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-19T04:44:53.892 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-19T04:44:53.892 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-19T04:44:53.892 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-19T04:44:53.892 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-19T04:44:53.892 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-19T04:44:53.892 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-19T04:44:53.892 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-19T04:44:53.892 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-19T04:44:53.892 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-19T04:44:53.892 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-19T04:44:53.892 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-19T04:44:53.893 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-19T04:44:53.893 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-19T04:44:53.893 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-19T04:44:53.893 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-19T04:44:53.893 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-19T04:44:53.893 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-19T04:44:53.893 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-19T04:44:53.893 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-19T04:44:53.893 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-19T04:44:53.893 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-19T04:44:53.893 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-19T04:44:53.893 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-19T04:44:53.893 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-19T04:44:53.894 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-19T04:44:53.894 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-19T04:44:53.894 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-19T04:44:53.894 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-19T04:44:53.894 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-19T04:44:53.894 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-19T04:44:53.894 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-19T04:44:53.894 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-19T04:44:53.894 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-19T04:44:53.894 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-19T04:44:53.894 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-19T04:44:53.894 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-19T04:44:53.894 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-19T04:44:53.894 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-19T04:44:53.894 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-19T04:44:53.894 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-19T04:44:53.895 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-19T04:44:53.895 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-19T04:44:53.895 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-19T04:44:53.895 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-19T04:44:53.895 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-19T04:44:53.895 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-19T04:44:53.895 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-19T04:44:53.895 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-19T04:44:53.895 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-19T04:44:53.895 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-19T04:44:53.895 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-19T04:44:53.895 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-19T04:44:53.896 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-19T04:44:53.896 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-19T04:44:53.896 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-19T04:44:53.896 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-19T04:44:53.896 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-19T04:44:53.896 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-19T04:44:53.896 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-19T04:44:53.896 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-19T04:44:53.896 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-19T04:44:53.896 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-19T04:44:53.897 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-19T04:44:53.897 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-19T04:44:53.897 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-19T04:44:53.897 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-19T04:44:53.897 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-19T04:44:53.897 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-19T04:44:53.897 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-19T04:44:53.897 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-19T04:44:53.897 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-19T04:44:53.897 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-19T04:44:53.897 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-19T04:44:53.897 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-19T04:44:53.897 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-19T04:44:53.898 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-19T04:44:53.898 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-19T04:44:53.898 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-19T04:44:53.898 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-19T04:44:53.898 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-19T04:44:53.898 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-19T04:44:53.898 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-19T04:44:53.898 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-19T04:44:53.898 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-19T04:44:53.898 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-19T04:44:53.898 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-19T04:44:53.898 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-19T04:44:53.898 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-19T04:44:53.898 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-19T04:44:53.898 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-19T04:44:53.899 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-19T04:44:53.899 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-19T04:44:53.899 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-19T04:44:53.899 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-19T04:44:53.899 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-19T04:44:53.899 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-19T04:44:53.899 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-19T04:44:53.899 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-19T04:44:53.899 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-19T04:44:53.899 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-19T04:44:53.899 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-19T04:44:53.899 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-19T04:44:53.899 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-19T04:44:53.899 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-19T04:44:53.900 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-19T04:44:53.900 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-19T04:44:53.900 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-19T04:44:53.900 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-19T04:44:53.900 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-19T04:44:53.900 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-19T04:44:53.900 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-19T04:44:53.900 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-19T04:44:53.900 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-19T04:44:53.900 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-19T04:44:53.900 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-19T04:44:53.900 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-19T04:44:53.900 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-19T04:44:53.900 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-19T04:44:53.900 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-19T04:44:53.900 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-19T04:44:53.900 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-19T04:44:53.901 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-19T04:44:53.901 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-19T04:44:53.901 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-19T04:44:53.901 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-19T04:44:53.901 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-19T04:44:53.901 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-19T04:44:53.901 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-19T04:44:53.901 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-19T04:44:53.901 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-19T04:44:53.901 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-19T04:44:53.901 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-19T04:44:53.901 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-19T04:44:53.901 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-19T04:44:53.901 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-19T04:44:53.901 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-19T04:44:53.901 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-19T04:44:53.902 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-19T04:44:53.902 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-19T04:44:53.902 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-19T04:44:53.902 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-19T04:44:53.902 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-19T04:44:53.902 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-19T04:44:53.902 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-19T04:44:53.902 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-19T04:44:53.902 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-19T04:44:53.902 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-19T04:44:53.902 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-19T04:44:53.902 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-19T04:44:53.902 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-19T04:44:53.902 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-19T04:44:53.902 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-19T04:44:53.902 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-19T04:44:53.903 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-19T04:44:53.903 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-19T04:44:53.903 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-19T04:44:53.903 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-19T04:44:53.903 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-19T04:44:53.903 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-19T04:44:53.903 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-19T04:44:53.903 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-19T04:44:53.907 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-19T04:44:53.907 DEBUG:teuthology.orchestra.run.smithi043:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-19T04:44:53.931 INFO:teuthology.orchestra.run.smithi043.stdout:postinstall scriptlet (using /bin/sh): 2024-09-19T04:44:53.931 INFO:teuthology.orchestra.run.smithi043.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-19T04:44:53.931 INFO:teuthology.orchestra.run.smithi043.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-19T04:44:53.931 INFO:teuthology.orchestra.run.smithi043.stdout:fi 2024-09-19T04:44:53.931 INFO:teuthology.orchestra.run.smithi043.stdout:for file in vmlinuz System.map config; do 2024-09-19T04:44:53.931 INFO:teuthology.orchestra.run.smithi043.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-19T04:44:53.931 INFO:teuthology.orchestra.run.smithi043.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-19T04:44:53.931 INFO:teuthology.orchestra.run.smithi043.stdout: fi 2024-09-19T04:44:53.931 INFO:teuthology.orchestra.run.smithi043.stdout:done 2024-09-19T04:44:53.931 INFO:teuthology.orchestra.run.smithi043.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-19T04:44:53.931 INFO:teuthology.orchestra.run.smithi043.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-19T04:44:53.932 INFO:teuthology.orchestra.run.smithi043.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-19T04:44:53.932 INFO:teuthology.orchestra.run.smithi043.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-19T04:44:53.932 INFO:teuthology.orchestra.run.smithi043.stdout:fi 2024-09-19T04:44:53.932 INFO:teuthology.orchestra.run.smithi043.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-19T04:44:53.932 INFO:teuthology.orchestra.run.smithi043.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-19T04:44:53.932 INFO:teuthology.orchestra.run.smithi043.stdout:fi 2024-09-19T04:44:53.932 INFO:teuthology.orchestra.run.smithi043.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-19T04:44:53.932 INFO:teuthology.orchestra.run.smithi043.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-19T04:44:53.932 INFO:teuthology.orchestra.run.smithi043.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-19T04:44:53.932 INFO:teuthology.orchestra.run.smithi043.stdout:fi 2024-09-19T04:44:53.933 DEBUG:teuthology.orchestra.run.smithi043:> sudo rpm -qi grub2-tools 2024-09-19T04:44:54.018 INFO:teuthology.orchestra.run.smithi043.stdout:Name : grub2-tools 2024-09-19T04:44:54.018 INFO:teuthology.orchestra.run.smithi043.stdout:Epoch : 1 2024-09-19T04:44:54.018 INFO:teuthology.orchestra.run.smithi043.stdout:Version : 2.06 2024-09-19T04:44:54.018 INFO:teuthology.orchestra.run.smithi043.stdout:Release : 82.el9 2024-09-19T04:44:54.018 INFO:teuthology.orchestra.run.smithi043.stdout:Architecture: x86_64 2024-09-19T04:44:54.018 INFO:teuthology.orchestra.run.smithi043.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-19T04:44:54.018 INFO:teuthology.orchestra.run.smithi043.stdout:Group : Unspecified 2024-09-19T04:44:54.018 INFO:teuthology.orchestra.run.smithi043.stdout:Size : 8274670 2024-09-19T04:44:54.018 INFO:teuthology.orchestra.run.smithi043.stdout:License : GPLv3+ 2024-09-19T04:44:54.018 INFO:teuthology.orchestra.run.smithi043.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-19T04:44:54.018 INFO:teuthology.orchestra.run.smithi043.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-19T04:44:54.018 INFO:teuthology.orchestra.run.smithi043.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-19T04:44:54.019 INFO:teuthology.orchestra.run.smithi043.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-19T04:44:54.019 INFO:teuthology.orchestra.run.smithi043.stdout:Packager : builder@centos.org 2024-09-19T04:44:54.019 INFO:teuthology.orchestra.run.smithi043.stdout:Vendor : CentOS 2024-09-19T04:44:54.019 INFO:teuthology.orchestra.run.smithi043.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-19T04:44:54.019 INFO:teuthology.orchestra.run.smithi043.stdout:Summary : Support tools for GRUB. 2024-09-19T04:44:54.019 INFO:teuthology.orchestra.run.smithi043.stdout:Description : 2024-09-19T04:44:54.019 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T04:44:54.019 INFO:teuthology.orchestra.run.smithi043.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-19T04:44:54.019 INFO:teuthology.orchestra.run.smithi043.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-19T04:44:54.019 INFO:teuthology.orchestra.run.smithi043.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-19T04:44:54.019 INFO:teuthology.orchestra.run.smithi043.stdout:hardware devices. 2024-09-19T04:44:54.019 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T04:44:54.019 INFO:teuthology.orchestra.run.smithi043.stdout:This subpackage provides tools for support of all platforms. 2024-09-19T04:44:54.020 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-19T04:44:54.020 INFO:teuthology.task.kernel:Updating grub on smithi043 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-19T04:44:54.021 DEBUG:teuthology.orchestra.run.smithi043:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-19T04:44:54.570 INFO:teuthology.orchestra.run.smithi043.stderr:Generating grub configuration file ... 2024-09-19T04:44:56.052 INFO:teuthology.orchestra.run.smithi043.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-19T04:44:56.080 INFO:teuthology.orchestra.run.smithi043.stderr:done 2024-09-19T04:44:56.089 DEBUG:teuthology.orchestra.run.smithi043:> mktemp 2024-09-19T04:44:56.103 INFO:teuthology.orchestra.run.smithi043.stdout:/tmp/tmp.ayeCwVwbk3 2024-09-19T04:44:56.104 DEBUG:teuthology.orchestra.run.smithi043:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ayeCwVwbk3 2024-09-19T04:44:56.171 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0666 /tmp/tmp.ayeCwVwbk3 2024-09-19T04:44:56.341 DEBUG:teuthology.orchestra.remote:smithi043:/tmp/tmp.ayeCwVwbk3 is 6KB 2024-09-19T04:44:56.390 DEBUG:teuthology.orchestra.run.smithi043:> rm -fr /tmp/tmp.ayeCwVwbk3 2024-09-19T04:44:56.404 DEBUG:teuthology.orchestra.run.smithi043:> sudo /bin/ls /boot/loader/entries || true 2024-09-19T04:44:56.467 INFO:teuthology.orchestra.run.smithi043.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-19T04:44:56.467 INFO:teuthology.orchestra.run.smithi043.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-19T04:44:56.467 INFO:teuthology.orchestra.run.smithi043.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-19T04:44:56.467 INFO:teuthology.orchestra.run.smithi043.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-19T04:44:56.468 DEBUG:teuthology.orchestra.run.smithi043:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-19T04:44:56.650 DEBUG:teuthology.orchestra.run.smithi043:> sudo shutdown -r now 2024-09-19T04:45:26.654 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-19T04:45:26.654 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-19T04:45:26.655 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:46:26.659 DEBUG:teuthology.orchestra.remote:timed out 2024-09-19T04:46:35.660 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-19T04:46:35.661 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:46:39.220 DEBUG:teuthology.orchestra.run.smithi043:> true 2024-09-19T04:46:39.712 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-19T04:46:39.712 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-19T04:46:39.712 DEBUG:teuthology.orchestra.run.smithi043:> uname -r 2024-09-19T04:46:39.746 INFO:teuthology.orchestra.run.smithi043.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-19T04:46:39.746 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-19T04:46:39.746 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-19T04:46:39.746 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-19T04:46:40.749 DEBUG:teuthology.parallel:result is None 2024-09-19T04:46:40.750 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-19T04:46:40.757 INFO:teuthology.task.internal:Creating test directory... 2024-09-19T04:46:40.757 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T04:46:40.763 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T04:46:40.765 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T04:46:40.794 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-19T04:46:40.804 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-19T04:46:40.835 INFO:teuthology.task.internal:Creating archive directory... 2024-09-19T04:46:40.835 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T04:46:40.839 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T04:46:40.841 DEBUG:teuthology.orchestra.run.smithi151:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T04:46:40.972 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-19T04:46:40.979 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-19T04:46:40.979 DEBUG:teuthology.orchestra.run.smithi096:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T04:46:40.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:46:40.997 DEBUG:teuthology.orchestra.run.smithi151:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T04:46:41.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:46:41.014 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T04:46:41.018 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T04:46:41.039 DEBUG:teuthology.orchestra.run.smithi151:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T04:46:41.076 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T04:46:41.097 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T04:46:41.114 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T04:46:41.120 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T04:46:41.132 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T04:46:41.162 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T04:46:41.165 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-19T04:46:41.178 INFO:teuthology.task.internal:Configuring sudo... 2024-09-19T04:46:41.178 DEBUG:teuthology.orchestra.run.smithi043:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T04:46:41.182 DEBUG:teuthology.orchestra.run.smithi096:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T04:46:41.185 DEBUG:teuthology.orchestra.run.smithi151:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T04:46:41.236 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-19T04:46:41.245 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-19T04:46:41.245 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T04:46:41.248 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T04:46:41.252 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T04:46:41.279 DEBUG:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T04:46:41.382 DEBUG:teuthology.orchestra.run.smithi043:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T04:46:41.460 DEBUG:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T04:46:41.522 DEBUG:teuthology.orchestra.run.smithi043:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T04:46:41.595 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-19T04:46:41.595 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T04:46:41.675 DEBUG:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T04:46:41.718 DEBUG:teuthology.orchestra.run.smithi096:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T04:46:41.771 DEBUG:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T04:46:41.854 DEBUG:teuthology.orchestra.run.smithi096:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T04:46:41.903 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-19T04:46:41.903 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T04:46:41.971 DEBUG:teuthology.orchestra.run.smithi151:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T04:46:42.016 DEBUG:teuthology.orchestra.run.smithi151:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T04:46:42.068 DEBUG:teuthology.orchestra.run.smithi151:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T04:46:42.151 DEBUG:teuthology.orchestra.run.smithi151:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T04:46:42.197 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-19T04:46:42.197 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T04:46:42.266 DEBUG:teuthology.orchestra.run.smithi043:> sudo service rsyslog restart 2024-09-19T04:46:42.270 DEBUG:teuthology.orchestra.run.smithi096:> sudo service rsyslog restart 2024-09-19T04:46:42.273 DEBUG:teuthology.orchestra.run.smithi151:> sudo service rsyslog restart 2024-09-19T04:46:42.307 INFO:teuthology.orchestra.run.smithi096.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T04:46:42.345 INFO:teuthology.orchestra.run.smithi043.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T04:46:42.349 INFO:teuthology.orchestra.run.smithi151.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T04:46:42.838 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-19T04:46:42.846 INFO:teuthology.task.internal:Starting timer... 2024-09-19T04:46:42.846 INFO:teuthology.run_tasks:Running task pcp... 2024-09-19T04:46:42.947 INFO:teuthology.run_tasks:Running task selinux... 2024-09-19T04:46:42.988 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-19T04:46:42.988 DEBUG:teuthology.orchestra.run.smithi043:> sudo service auditd rotate 2024-09-19T04:46:43.103 INFO:teuthology.orchestra.run.smithi043.stdout:Rotating logs: 2024-09-19T04:46:43.106 DEBUG:teuthology.orchestra.run.smithi096:> sudo service auditd rotate 2024-09-19T04:46:43.207 INFO:teuthology.orchestra.run.smithi096.stdout:Rotating logs: 2024-09-19T04:46:43.209 DEBUG:teuthology.orchestra.run.smithi151:> sudo service auditd rotate 2024-09-19T04:46:43.280 INFO:teuthology.orchestra.run.smithi151.stdout:Rotating logs: 2024-09-19T04:46:43.282 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-19T04:46:43.282 DEBUG:teuthology.orchestra.run.smithi043:> /usr/sbin/getenforce 2024-09-19T04:46:43.342 INFO:teuthology.orchestra.run.smithi043.stdout:Permissive 2024-09-19T04:46:43.343 DEBUG:teuthology.orchestra.run.smithi096:> /usr/sbin/getenforce 2024-09-19T04:46:43.372 INFO:teuthology.orchestra.run.smithi096.stdout:Permissive 2024-09-19T04:46:43.372 DEBUG:teuthology.orchestra.run.smithi151:> /usr/sbin/getenforce 2024-09-19T04:46:43.394 INFO:teuthology.orchestra.run.smithi151.stdout:Permissive 2024-09-19T04:46:43.395 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi043.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi096.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi151.front.sepia.ceph.com': 'permissive'} 2024-09-19T04:46:43.395 DEBUG:teuthology.orchestra.run.smithi043:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T04:46:43.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:46:43.453 DEBUG:teuthology.orchestra.run.smithi096:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T04:46:43.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:46:43.482 DEBUG:teuthology.orchestra.run.smithi151:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T04:46:43.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:46:43.510 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-19T04:46:43.510 DEBUG:teuthology.orchestra.run.smithi043:> sudo /usr/sbin/setenforce permissive 2024-09-19T04:46:43.618 DEBUG:teuthology.orchestra.run.smithi096:> sudo /usr/sbin/setenforce permissive 2024-09-19T04:46:43.653 DEBUG:teuthology.orchestra.run.smithi151:> sudo /usr/sbin/setenforce permissive 2024-09-19T04:46:43.682 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-19T04:46:43.693 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-19T04:46:43.697 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-19T04:46:43.820 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-19T04:46:43.830 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-19T04:46:43.831 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi043.front.sepia.ceph.com,smithi096.front.sepia.ceph.com,smithi151.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-19T04:51:27.352 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi043.front.sepia.ceph.com'), Remote(name='ubuntu@smithi096.front.sepia.ceph.com'), Remote(name='ubuntu@smithi151.front.sepia.ceph.com')] 2024-09-19T04:51:27.354 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-19T04:51:27.355 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:51:27.450 DEBUG:teuthology.orchestra.run.smithi043:> true 2024-09-19T04:51:27.573 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-19T04:51:27.573 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-09-19T04:51:27.574 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:51:27.648 DEBUG:teuthology.orchestra.run.smithi096:> true 2024-09-19T04:51:27.740 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-09-19T04:51:27.741 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-09-19T04:51:27.741 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:51:27.815 DEBUG:teuthology.orchestra.run.smithi151:> true 2024-09-19T04:51:27.892 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-09-19T04:51:27.893 INFO:teuthology.run_tasks:Running task clock... 2024-09-19T04:51:27.905 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-19T04:51:27.905 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T04:51:27.905 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T04:51:27.910 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T04:51:27.910 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T04:51:27.912 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T04:51:27.912 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T04:51:27.939 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T04:51:27.953 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T04:51:27.962 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T04:51:27.974 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T04:51:27.977 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T04:51:27.977 INFO:teuthology.orchestra.run.smithi096.stderr:sudo: ntpd: command not found 2024-09-19T04:51:27.988 INFO:teuthology.orchestra.run.smithi096.stdout:506 Cannot talk to daemon 2024-09-19T04:51:28.001 INFO:teuthology.orchestra.run.smithi151.stderr:sudo: ntpd: command not found 2024-09-19T04:51:28.002 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T04:51:28.006 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T04:51:28.012 INFO:teuthology.orchestra.run.smithi151.stdout:506 Cannot talk to daemon 2024-09-19T04:51:28.016 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T04:51:28.027 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T04:51:28.041 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T04:51:28.059 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: ntpd: command not found 2024-09-19T04:51:28.078 INFO:teuthology.orchestra.run.smithi096.stderr:bash: line 1: ntpq: command not found 2024-09-19T04:51:28.081 INFO:teuthology.orchestra.run.smithi096.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T04:51:28.081 INFO:teuthology.orchestra.run.smithi096.stdout:=============================================================================== 2024-09-19T04:51:28.081 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:51:28.081 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:51:28.081 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:51:28.082 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:51:28.084 INFO:teuthology.orchestra.run.smithi043.stdout:506 Cannot talk to daemon 2024-09-19T04:51:28.100 INFO:teuthology.orchestra.run.smithi151.stderr:bash: line 1: ntpq: command not found 2024-09-19T04:51:28.104 INFO:teuthology.orchestra.run.smithi151.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T04:51:28.104 INFO:teuthology.orchestra.run.smithi151.stdout:=============================================================================== 2024-09-19T04:51:28.104 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:51:28.104 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:51:28.104 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:51:28.104 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:51:28.113 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T04:51:28.143 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T04:51:28.245 INFO:teuthology.orchestra.run.smithi043.stderr:bash: line 1: ntpq: command not found 2024-09-19T04:51:28.251 INFO:teuthology.orchestra.run.smithi043.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T04:51:28.251 INFO:teuthology.orchestra.run.smithi043.stdout:=============================================================================== 2024-09-19T04:51:28.251 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:51:28.251 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:51:28.252 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:51:28.252 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T04:51:28.253 INFO:teuthology.run_tasks:Running task install... 2024-09-19T04:51:28.262 DEBUG:teuthology.task.install:project ceph 2024-09-19T04:51:28.263 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'}} 2024-09-19T04:51:28.263 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'} 2024-09-19T04:51:28.263 INFO:teuthology.task.install:Using flavor: default 2024-09-19T04:51:28.268 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-19T04:51:28.268 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-19T04:51:28.269 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T04:51:28.269 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:51:28.271 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T04:51:28.272 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:51:28.273 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T04:51:28.273 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:51:28.426 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T04:51:28.426 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T04:51:28.430 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T04:51:28.430 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T04:51:28.431 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T04:51:28.431 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T04:51:28.537 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T04:51:28.538 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-19T04:51:28.538 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T04:51:28.543 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T04:51:28.543 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-19T04:51:28.543 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T04:51:28.547 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T04:51:28.547 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-19T04:51:28.547 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T04:51:28.570 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T04:51:28.570 DEBUG:teuthology.orchestra.run.smithi151:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T04:51:28.574 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T04:51:28.575 DEBUG:teuthology.orchestra.run.smithi096:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T04:51:28.595 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T04:51:28.595 DEBUG:teuthology.orchestra.run.smithi043:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T04:51:28.636 DEBUG:teuthology.orchestra.run.smithi151:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T04:51:28.643 DEBUG:teuthology.orchestra.run.smithi096:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T04:51:28.693 DEBUG:teuthology.orchestra.run.smithi043:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T04:51:28.712 DEBUG:teuthology.orchestra.run.smithi151:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T04:51:28.719 DEBUG:teuthology.orchestra.run.smithi096:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T04:51:28.773 DEBUG:teuthology.orchestra.run.smithi043:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T04:51:28.776 INFO:teuthology.orchestra.run.smithi151.stdout:check_obsoletes = 1 2024-09-19T04:51:28.778 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum clean all 2024-09-19T04:51:28.785 INFO:teuthology.orchestra.run.smithi096.stdout:check_obsoletes = 1 2024-09-19T04:51:28.786 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum clean all 2024-09-19T04:51:28.829 INFO:teuthology.orchestra.run.smithi043.stdout:check_obsoletes = 1 2024-09-19T04:51:28.832 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum clean all 2024-09-19T04:51:29.002 INFO:teuthology.orchestra.run.smithi151.stdout:45 files removed 2024-09-19T04:51:29.010 INFO:teuthology.orchestra.run.smithi096.stdout:45 files removed 2024-09-19T04:51:29.029 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T04:51:29.037 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T04:51:29.108 INFO:teuthology.orchestra.run.smithi043.stdout:45 files removed 2024-09-19T04:51:29.142 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T04:51:29.570 INFO:teuthology.orchestra.run.smithi096.stdout:ceph packages for x86_64 327 kB/s | 84 kB 00:00 2024-09-19T04:51:29.572 INFO:teuthology.orchestra.run.smithi151.stdout:ceph packages for x86_64 314 kB/s | 84 kB 00:00 2024-09-19T04:51:29.834 INFO:teuthology.orchestra.run.smithi043.stdout:ceph packages for x86_64 296 kB/s | 84 kB 00:00 2024-09-19T04:51:29.848 INFO:teuthology.orchestra.run.smithi151.stdout:ceph noarch packages 50 kB/s | 12 kB 00:00 2024-09-19T04:51:29.854 INFO:teuthology.orchestra.run.smithi096.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-09-19T04:51:30.115 INFO:teuthology.orchestra.run.smithi151.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-19T04:51:30.130 INFO:teuthology.orchestra.run.smithi096.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-19T04:51:30.135 INFO:teuthology.orchestra.run.smithi043.stdout:ceph noarch packages 46 kB/s | 12 kB 00:00 2024-09-19T04:51:30.419 INFO:teuthology.orchestra.run.smithi043.stdout:ceph source packages 7.8 kB/s | 1.9 kB 00:00 2024-09-19T04:51:30.663 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.3 MB 00:00 2024-09-19T04:51:30.741 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.3 MB 00:00 2024-09-19T04:51:30.952 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.3 MB 00:00 2024-09-19T04:51:32.590 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - AppStream 23 MB/s | 20 MB 00:00 2024-09-19T04:51:33.395 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - AppStream 16 MB/s | 20 MB 00:01 2024-09-19T04:51:36.734 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - AppStream 4.1 MB/s | 20 MB 00:04 2024-09-19T04:51:37.124 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - CRB 8.7 MB/s | 6.5 MB 00:00 2024-09-19T04:51:39.374 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 19 kB 00:01 2024-09-19T04:51:39.612 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - CRB 3.2 MB/s | 6.5 MB 00:02 2024-09-19T04:51:40.412 INFO:teuthology.orchestra.run.smithi096.stdout:Extra Packages for Enterprise Linux 25 MB/s | 23 MB 00:00 2024-09-19T04:51:41.271 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - Extras packages 61 kB/s | 19 kB 00:00 2024-09-19T04:51:41.585 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - CRB 8.6 MB/s | 6.5 MB 00:00 2024-09-19T04:51:42.550 INFO:teuthology.orchestra.run.smithi043.stdout:Extra Packages for Enterprise Linux 20 MB/s | 23 MB 00:01 2024-09-19T04:51:43.344 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 19 kB 00:00 2024-09-19T04:51:44.349 INFO:teuthology.orchestra.run.smithi151.stdout:Extra Packages for Enterprise Linux 26 MB/s | 23 MB 00:00 2024-09-19T04:51:45.867 INFO:teuthology.orchestra.run.smithi096.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-19T04:51:47.962 INFO:teuthology.orchestra.run.smithi096.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T04:51:47.963 INFO:teuthology.orchestra.run.smithi096.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T04:51:47.976 INFO:teuthology.orchestra.run.smithi096.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T04:51:47.977 INFO:teuthology.orchestra.run.smithi096.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T04:51:47.978 INFO:teuthology.orchestra.run.smithi096.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T04:51:47.979 INFO:teuthology.orchestra.run.smithi096.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T04:51:47.981 INFO:teuthology.orchestra.run.smithi096.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T04:51:47.984 INFO:teuthology.orchestra.run.smithi096.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T04:51:47.985 INFO:teuthology.orchestra.run.smithi096.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T04:51:48.047 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-09-19T04:51:48.061 INFO:teuthology.orchestra.run.smithi096.stdout:====================================================================================== 2024-09-19T04:51:48.061 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-09-19T04:51:48.061 INFO:teuthology.orchestra.run.smithi096.stdout:====================================================================================== 2024-09-19T04:51:48.061 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-09-19T04:51:48.061 INFO:teuthology.orchestra.run.smithi096.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T04:51:48.061 INFO:teuthology.orchestra.run.smithi096.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T04:51:48.062 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T04:51:48.062 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T04:51:48.062 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T04:51:48.062 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T04:51:48.062 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T04:51:48.062 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T04:51:48.062 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T04:51:48.062 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T04:51:48.062 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T04:51:48.062 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T04:51:48.062 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T04:51:48.063 INFO:teuthology.orchestra.run.smithi096.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T04:51:48.063 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T04:51:48.063 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T04:51:48.063 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T04:51:48.063 INFO:teuthology.orchestra.run.smithi096.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T04:51:48.063 INFO:teuthology.orchestra.run.smithi096.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T04:51:48.063 INFO:teuthology.orchestra.run.smithi096.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T04:51:48.063 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T04:51:48.063 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T04:51:48.063 INFO:teuthology.orchestra.run.smithi096.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T04:51:48.063 INFO:teuthology.orchestra.run.smithi096.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T04:51:48.063 INFO:teuthology.orchestra.run.smithi096.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T04:51:48.064 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T04:51:48.064 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T04:51:48.064 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T04:51:48.064 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T04:51:48.064 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T04:51:48.064 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T04:51:48.064 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T04:51:48.064 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T04:51:48.064 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T04:51:48.064 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T04:51:48.064 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T04:51:48.064 INFO:teuthology.orchestra.run.smithi096.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T04:51:48.064 INFO:teuthology.orchestra.run.smithi096.stdout:Upgrading: 2024-09-19T04:51:48.065 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T04:51:48.065 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T04:51:48.065 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T04:51:48.065 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T04:51:48.065 INFO:teuthology.orchestra.run.smithi096.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T04:51:48.065 INFO:teuthology.orchestra.run.smithi096.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T04:51:48.065 INFO:teuthology.orchestra.run.smithi096.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T04:51:48.065 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T04:51:48.065 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-09-19T04:51:48.065 INFO:teuthology.orchestra.run.smithi096.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T04:51:48.065 INFO:teuthology.orchestra.run.smithi096.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T04:51:48.066 INFO:teuthology.orchestra.run.smithi096.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T04:51:48.066 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T04:51:48.066 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T04:51:48.066 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T04:51:48.066 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T04:51:48.066 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T04:51:48.066 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T04:51:48.066 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T04:51:48.066 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T04:51:48.066 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T04:51:48.066 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T04:51:48.066 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T04:51:48.067 INFO:teuthology.orchestra.run.smithi096.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T04:51:48.067 INFO:teuthology.orchestra.run.smithi096.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T04:51:48.067 INFO:teuthology.orchestra.run.smithi096.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T04:51:48.067 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T04:51:48.067 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T04:51:48.067 INFO:teuthology.orchestra.run.smithi096.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T04:51:48.067 INFO:teuthology.orchestra.run.smithi096.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T04:51:48.067 INFO:teuthology.orchestra.run.smithi096.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T04:51:48.067 INFO:teuthology.orchestra.run.smithi096.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T04:51:48.067 INFO:teuthology.orchestra.run.smithi096.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T04:51:48.067 INFO:teuthology.orchestra.run.smithi096.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T04:51:48.067 INFO:teuthology.orchestra.run.smithi096.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T04:51:48.067 INFO:teuthology.orchestra.run.smithi096.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T04:51:48.068 INFO:teuthology.orchestra.run.smithi096.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T04:51:48.068 INFO:teuthology.orchestra.run.smithi096.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T04:51:48.068 INFO:teuthology.orchestra.run.smithi096.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T04:51:48.068 INFO:teuthology.orchestra.run.smithi096.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T04:51:48.068 INFO:teuthology.orchestra.run.smithi096.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T04:51:48.068 INFO:teuthology.orchestra.run.smithi096.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T04:51:48.068 INFO:teuthology.orchestra.run.smithi096.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T04:51:48.068 INFO:teuthology.orchestra.run.smithi096.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T04:51:48.068 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T04:51:48.069 INFO:teuthology.orchestra.run.smithi096.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T04:51:48.069 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T04:51:48.069 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T04:51:48.069 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T04:51:48.069 INFO:teuthology.orchestra.run.smithi096.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T04:51:48.069 INFO:teuthology.orchestra.run.smithi096.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T04:51:48.070 INFO:teuthology.orchestra.run.smithi096.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T04:51:48.070 INFO:teuthology.orchestra.run.smithi096.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T04:51:48.070 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T04:51:48.070 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T04:51:48.070 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T04:51:48.070 INFO:teuthology.orchestra.run.smithi096.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T04:51:48.070 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T04:51:48.070 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T04:51:48.070 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T04:51:48.071 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T04:51:48.071 INFO:teuthology.orchestra.run.smithi096.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T04:51:48.071 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T04:51:48.071 INFO:teuthology.orchestra.run.smithi096.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T04:51:48.071 INFO:teuthology.orchestra.run.smithi096.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T04:51:48.071 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T04:51:48.071 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T04:51:48.071 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T04:51:48.072 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T04:51:48.072 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T04:51:48.072 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T04:51:48.072 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T04:51:48.072 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T04:51:48.072 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T04:51:48.072 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T04:51:48.073 INFO:teuthology.orchestra.run.smithi096.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T04:51:48.073 INFO:teuthology.orchestra.run.smithi096.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T04:51:48.073 INFO:teuthology.orchestra.run.smithi096.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T04:51:48.073 INFO:teuthology.orchestra.run.smithi096.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T04:51:48.073 INFO:teuthology.orchestra.run.smithi096.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T04:51:48.073 INFO:teuthology.orchestra.run.smithi096.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T04:51:48.073 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T04:51:48.073 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T04:51:48.073 INFO:teuthology.orchestra.run.smithi096.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T04:51:48.073 INFO:teuthology.orchestra.run.smithi096.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T04:51:48.074 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T04:51:48.074 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T04:51:48.074 INFO:teuthology.orchestra.run.smithi096.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T04:51:48.074 INFO:teuthology.orchestra.run.smithi096.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T04:51:48.074 INFO:teuthology.orchestra.run.smithi096.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T04:51:48.074 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T04:51:48.074 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T04:51:48.074 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T04:51:48.074 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T04:51:48.074 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T04:51:48.074 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T04:51:48.074 INFO:teuthology.orchestra.run.smithi096.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T04:51:48.074 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T04:51:48.075 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T04:51:48.075 INFO:teuthology.orchestra.run.smithi096.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T04:51:48.075 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T04:51:48.075 INFO:teuthology.orchestra.run.smithi096.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T04:51:48.075 INFO:teuthology.orchestra.run.smithi096.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T04:51:48.075 INFO:teuthology.orchestra.run.smithi096.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T04:51:48.075 INFO:teuthology.orchestra.run.smithi096.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T04:51:48.075 INFO:teuthology.orchestra.run.smithi096.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T04:51:48.075 INFO:teuthology.orchestra.run.smithi096.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T04:51:48.075 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T04:51:48.075 INFO:teuthology.orchestra.run.smithi096.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T04:51:48.075 INFO:teuthology.orchestra.run.smithi096.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T04:51:48.075 INFO:teuthology.orchestra.run.smithi096.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T04:51:48.075 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T04:51:48.076 INFO:teuthology.orchestra.run.smithi096.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T04:51:48.076 INFO:teuthology.orchestra.run.smithi096.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T04:51:48.076 INFO:teuthology.orchestra.run.smithi096.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T04:51:48.076 INFO:teuthology.orchestra.run.smithi096.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T04:51:48.076 INFO:teuthology.orchestra.run.smithi096.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T04:51:48.076 INFO:teuthology.orchestra.run.smithi096.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T04:51:48.076 INFO:teuthology.orchestra.run.smithi096.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T04:51:48.076 INFO:teuthology.orchestra.run.smithi096.stdout:Installing weak dependencies: 2024-09-19T04:51:48.076 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T04:51:48.076 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-19T04:51:48.076 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-09-19T04:51:48.076 INFO:teuthology.orchestra.run.smithi096.stdout:====================================================================================== 2024-09-19T04:51:48.076 INFO:teuthology.orchestra.run.smithi096.stdout:Install 148 Packages 2024-09-19T04:51:48.077 INFO:teuthology.orchestra.run.smithi096.stdout:Upgrade 8 Packages 2024-09-19T04:51:48.077 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-19T04:51:48.077 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 226 M 2024-09-19T04:51:48.077 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-09-19T04:51:48.535 INFO:teuthology.orchestra.run.smithi096.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 54 kB/s | 6.6 kB 00:00 2024-09-19T04:51:48.760 INFO:teuthology.orchestra.run.smithi096.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 3.9 MB/s | 896 kB 00:00 2024-09-19T04:51:48.844 INFO:teuthology.orchestra.run.smithi096.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 1.7 MB/s | 145 kB 00:00 2024-09-19T04:51:48.928 INFO:teuthology.orchestra.run.smithi096.stdout:(4/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 11 MB/s | 5.4 MB 00:00 2024-09-19T04:51:49.020 INFO:teuthology.orchestra.run.smithi096.stdout:(5/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 17 MB/s | 1.5 MB 00:00 2024-09-19T04:51:49.061 INFO:teuthology.orchestra.run.smithi096.stdout:(6/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 11 MB/s | 2.4 MB 00:00 2024-09-19T04:51:49.097 INFO:teuthology.orchestra.run.smithi043.stdout:lab-extras 41 kB/s | 1.7 kB 00:00 2024-09-19T04:51:49.303 INFO:teuthology.orchestra.run.smithi096.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 17 MB/s | 4.7 MB 00:00 2024-09-19T04:51:49.512 INFO:teuthology.orchestra.run.smithi096.stdout:(8/156): ceph-common-19.1.1-361.g2f599a46.el9.x 19 MB/s | 21 MB 00:01 2024-09-19T04:51:49.563 INFO:teuthology.orchestra.run.smithi096.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 506 kB/s | 25 kB 00:00 2024-09-19T04:51:49.722 INFO:teuthology.orchestra.run.smithi151.stdout:lab-extras 37 kB/s | 1.7 kB 00:00 2024-09-19T04:51:50.480 INFO:teuthology.orchestra.run.smithi096.stdout:(10/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 12 MB/s | 17 MB 00:01 2024-09-19T04:51:50.529 INFO:teuthology.orchestra.run.smithi096.stdout:(11/156): cephfs-mirror-19.1.1-361.g2f599a46.el 4.5 MB/s | 228 kB 00:00 2024-09-19T04:51:50.638 INFO:teuthology.orchestra.run.smithi096.stdout:(12/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 7.9 MB/s | 11 MB 00:01 2024-09-19T04:51:50.655 INFO:teuthology.orchestra.run.smithi096.stdout:(13/156): cephfs-shell-19.1.1-361.g2f599a46.el9 216 kB/s | 27 kB 00:00 2024-09-19T04:51:50.672 INFO:teuthology.orchestra.run.smithi096.stdout:(14/156): libcephfs-devel-19.1.1-361.g2f599a46. 946 kB/s | 32 kB 00:00 2024-09-19T04:51:50.722 INFO:teuthology.orchestra.run.smithi096.stdout:(15/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 11 MB/s | 751 kB 00:00 2024-09-19T04:51:50.748 INFO:teuthology.orchestra.run.smithi096.stdout:(16/156): libcephsqlite-19.1.1-361.g2f599a46.el 2.1 MB/s | 163 kB 00:00 2024-09-19T04:51:50.773 INFO:teuthology.orchestra.run.smithi096.stdout:(17/156): librados-devel-19.1.1-361.g2f599a46.e 2.5 MB/s | 127 kB 00:00 2024-09-19T04:51:50.806 INFO:teuthology.orchestra.run.smithi096.stdout:(18/156): libradosstriper1-19.1.1-361.g2f599a46 8.5 MB/s | 506 kB 00:00 2024-09-19T04:51:50.865 INFO:teuthology.orchestra.run.smithi096.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 773 kB/s | 45 kB 00:00 2024-09-19T04:51:50.957 INFO:teuthology.orchestra.run.smithi096.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 1.5 MB/s | 136 kB 00:00 2024-09-19T04:51:51.408 INFO:teuthology.orchestra.run.smithi096.stdout:(21/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 26 MB/s | 49 MB 00:01 2024-09-19T04:51:51.432 INFO:teuthology.orchestra.run.smithi096.stdout:(22/156): python3-cephfs-19.1.1-361.g2f599a46.e 340 kB/s | 161 kB 00:00 2024-09-19T04:51:51.458 INFO:teuthology.orchestra.run.smithi096.stdout:(23/156): python3-rados-19.1.1-361.g2f599a46.el 6.3 MB/s | 320 kB 00:00 2024-09-19T04:51:51.491 INFO:teuthology.orchestra.run.smithi096.stdout:(24/156): python3-rbd-19.1.1-361.g2f599a46.el9. 5.0 MB/s | 300 kB 00:00 2024-09-19T04:51:51.521 INFO:teuthology.orchestra.run.smithi043.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T04:51:51.522 INFO:teuthology.orchestra.run.smithi043.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T04:51:51.533 INFO:teuthology.orchestra.run.smithi096.stdout:(25/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.3 MB/s | 99 kB 00:00 2024-09-19T04:51:51.537 INFO:teuthology.orchestra.run.smithi043.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T04:51:51.538 INFO:teuthology.orchestra.run.smithi043.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T04:51:51.539 INFO:teuthology.orchestra.run.smithi043.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T04:51:51.540 INFO:teuthology.orchestra.run.smithi043.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T04:51:51.542 INFO:teuthology.orchestra.run.smithi043.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T04:51:51.545 INFO:teuthology.orchestra.run.smithi043.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T04:51:51.546 INFO:teuthology.orchestra.run.smithi043.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T04:51:51.559 INFO:teuthology.orchestra.run.smithi096.stdout:(26/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.2 MB/s | 85 kB 00:00 2024-09-19T04:51:51.616 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-19T04:51:51.629 INFO:teuthology.orchestra.run.smithi043.stdout:====================================================================================== 2024-09-19T04:51:51.629 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-19T04:51:51.629 INFO:teuthology.orchestra.run.smithi043.stdout:====================================================================================== 2024-09-19T04:51:51.629 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-19T04:51:51.629 INFO:teuthology.orchestra.run.smithi043.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T04:51:51.629 INFO:teuthology.orchestra.run.smithi043.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T04:51:51.629 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T04:51:51.629 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T04:51:51.629 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T04:51:51.630 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T04:51:51.630 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T04:51:51.630 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T04:51:51.630 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T04:51:51.630 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T04:51:51.630 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T04:51:51.630 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T04:51:51.630 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T04:51:51.630 INFO:teuthology.orchestra.run.smithi043.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T04:51:51.630 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T04:51:51.630 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T04:51:51.630 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T04:51:51.631 INFO:teuthology.orchestra.run.smithi043.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T04:51:51.631 INFO:teuthology.orchestra.run.smithi043.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T04:51:51.631 INFO:teuthology.orchestra.run.smithi043.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T04:51:51.631 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T04:51:51.631 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T04:51:51.631 INFO:teuthology.orchestra.run.smithi043.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T04:51:51.631 INFO:teuthology.orchestra.run.smithi043.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T04:51:51.631 INFO:teuthology.orchestra.run.smithi043.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T04:51:51.631 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T04:51:51.631 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T04:51:51.631 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T04:51:51.631 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T04:51:51.632 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T04:51:51.632 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T04:51:51.632 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T04:51:51.632 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T04:51:51.632 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T04:51:51.632 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T04:51:51.632 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T04:51:51.632 INFO:teuthology.orchestra.run.smithi043.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T04:51:51.632 INFO:teuthology.orchestra.run.smithi043.stdout:Upgrading: 2024-09-19T04:51:51.632 INFO:teuthology.orchestra.run.smithi043.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T04:51:51.632 INFO:teuthology.orchestra.run.smithi043.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T04:51:51.632 INFO:teuthology.orchestra.run.smithi043.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T04:51:51.633 INFO:teuthology.orchestra.run.smithi043.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T04:51:51.633 INFO:teuthology.orchestra.run.smithi043.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T04:51:51.633 INFO:teuthology.orchestra.run.smithi043.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T04:51:51.633 INFO:teuthology.orchestra.run.smithi043.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T04:51:51.633 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T04:51:51.633 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-19T04:51:51.633 INFO:teuthology.orchestra.run.smithi043.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T04:51:51.633 INFO:teuthology.orchestra.run.smithi043.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T04:51:51.633 INFO:teuthology.orchestra.run.smithi043.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T04:51:51.633 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T04:51:51.633 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T04:51:51.634 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T04:51:51.634 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T04:51:51.634 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T04:51:51.634 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T04:51:51.634 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T04:51:51.634 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T04:51:51.634 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T04:51:51.634 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T04:51:51.634 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T04:51:51.634 INFO:teuthology.orchestra.run.smithi043.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T04:51:51.634 INFO:teuthology.orchestra.run.smithi043.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T04:51:51.634 INFO:teuthology.orchestra.run.smithi043.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T04:51:51.634 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T04:51:51.635 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T04:51:51.635 INFO:teuthology.orchestra.run.smithi043.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T04:51:51.635 INFO:teuthology.orchestra.run.smithi043.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T04:51:51.635 INFO:teuthology.orchestra.run.smithi043.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T04:51:51.635 INFO:teuthology.orchestra.run.smithi043.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T04:51:51.635 INFO:teuthology.orchestra.run.smithi043.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T04:51:51.635 INFO:teuthology.orchestra.run.smithi043.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T04:51:51.635 INFO:teuthology.orchestra.run.smithi043.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T04:51:51.635 INFO:teuthology.orchestra.run.smithi043.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T04:51:51.635 INFO:teuthology.orchestra.run.smithi043.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T04:51:51.635 INFO:teuthology.orchestra.run.smithi043.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T04:51:51.635 INFO:teuthology.orchestra.run.smithi043.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T04:51:51.635 INFO:teuthology.orchestra.run.smithi043.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T04:51:51.636 INFO:teuthology.orchestra.run.smithi043.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T04:51:51.636 INFO:teuthology.orchestra.run.smithi043.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T04:51:51.636 INFO:teuthology.orchestra.run.smithi043.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T04:51:51.636 INFO:teuthology.orchestra.run.smithi043.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T04:51:51.636 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T04:51:51.636 INFO:teuthology.orchestra.run.smithi043.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T04:51:51.636 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T04:51:51.636 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T04:51:51.636 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T04:51:51.636 INFO:teuthology.orchestra.run.smithi043.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T04:51:51.636 INFO:teuthology.orchestra.run.smithi043.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T04:51:51.636 INFO:teuthology.orchestra.run.smithi043.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T04:51:51.636 INFO:teuthology.orchestra.run.smithi043.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T04:51:51.636 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T04:51:51.637 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T04:51:51.637 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T04:51:51.637 INFO:teuthology.orchestra.run.smithi043.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T04:51:51.637 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T04:51:51.637 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T04:51:51.637 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T04:51:51.637 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T04:51:51.637 INFO:teuthology.orchestra.run.smithi043.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T04:51:51.637 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T04:51:51.637 INFO:teuthology.orchestra.run.smithi043.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T04:51:51.637 INFO:teuthology.orchestra.run.smithi043.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T04:51:51.637 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T04:51:51.637 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T04:51:51.638 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T04:51:51.638 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T04:51:51.638 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T04:51:51.638 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T04:51:51.638 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T04:51:51.638 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T04:51:51.638 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T04:51:51.638 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T04:51:51.638 INFO:teuthology.orchestra.run.smithi043.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T04:51:51.638 INFO:teuthology.orchestra.run.smithi043.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T04:51:51.638 INFO:teuthology.orchestra.run.smithi043.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T04:51:51.638 INFO:teuthology.orchestra.run.smithi043.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T04:51:51.638 INFO:teuthology.orchestra.run.smithi043.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T04:51:51.639 INFO:teuthology.orchestra.run.smithi043.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T04:51:51.639 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T04:51:51.639 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T04:51:51.639 INFO:teuthology.orchestra.run.smithi043.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T04:51:51.639 INFO:teuthology.orchestra.run.smithi043.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T04:51:51.639 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T04:51:51.639 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T04:51:51.639 INFO:teuthology.orchestra.run.smithi043.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T04:51:51.639 INFO:teuthology.orchestra.run.smithi043.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T04:51:51.639 INFO:teuthology.orchestra.run.smithi043.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T04:51:51.639 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T04:51:51.639 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T04:51:51.639 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T04:51:51.640 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T04:51:51.640 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T04:51:51.640 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T04:51:51.640 INFO:teuthology.orchestra.run.smithi043.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T04:51:51.640 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T04:51:51.640 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T04:51:51.640 INFO:teuthology.orchestra.run.smithi043.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T04:51:51.640 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T04:51:51.640 INFO:teuthology.orchestra.run.smithi043.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T04:51:51.640 INFO:teuthology.orchestra.run.smithi043.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T04:51:51.640 INFO:teuthology.orchestra.run.smithi043.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T04:51:51.640 INFO:teuthology.orchestra.run.smithi043.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T04:51:51.640 INFO:teuthology.orchestra.run.smithi043.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T04:51:51.641 INFO:teuthology.orchestra.run.smithi043.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T04:51:51.641 INFO:teuthology.orchestra.run.smithi043.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T04:51:51.641 INFO:teuthology.orchestra.run.smithi043.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T04:51:51.641 INFO:teuthology.orchestra.run.smithi043.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T04:51:51.641 INFO:teuthology.orchestra.run.smithi043.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T04:51:51.641 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T04:51:51.641 INFO:teuthology.orchestra.run.smithi043.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T04:51:51.641 INFO:teuthology.orchestra.run.smithi043.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T04:51:51.641 INFO:teuthology.orchestra.run.smithi043.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T04:51:51.641 INFO:teuthology.orchestra.run.smithi043.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T04:51:51.641 INFO:teuthology.orchestra.run.smithi043.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T04:51:51.641 INFO:teuthology.orchestra.run.smithi043.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T04:51:51.641 INFO:teuthology.orchestra.run.smithi043.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T04:51:51.641 INFO:teuthology.orchestra.run.smithi043.stdout:Installing weak dependencies: 2024-09-19T04:51:51.642 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T04:51:51.642 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T04:51:51.642 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-19T04:51:51.642 INFO:teuthology.orchestra.run.smithi043.stdout:====================================================================================== 2024-09-19T04:51:51.642 INFO:teuthology.orchestra.run.smithi043.stdout:Install 148 Packages 2024-09-19T04:51:51.642 INFO:teuthology.orchestra.run.smithi043.stdout:Upgrade 8 Packages 2024-09-19T04:51:51.642 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T04:51:51.643 INFO:teuthology.orchestra.run.smithi096.stdout:(27/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 6.2 MB/s | 5.3 MB 00:00 2024-09-19T04:51:51.643 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 226 M 2024-09-19T04:51:51.643 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-19T04:51:51.659 INFO:teuthology.orchestra.run.smithi096.stdout:(28/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 1.7 MB/s | 171 kB 00:00 2024-09-19T04:51:51.684 INFO:teuthology.orchestra.run.smithi096.stdout:(29/156): ceph-grafana-dashboards-19.1.1-361.g2 637 kB/s | 32 kB 00:00 2024-09-19T04:51:51.710 INFO:teuthology.orchestra.run.smithi096.stdout:(30/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 2.8 MB/s | 146 kB 00:00 2024-09-19T04:51:51.768 INFO:teuthology.orchestra.run.smithi096.stdout:(31/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 13 MB/s | 3.1 MB 00:00 2024-09-19T04:51:51.835 INFO:teuthology.orchestra.run.smithi096.stdout:(32/156): ceph-mgr-modules-core-19.1.1-361.g2f5 3.7 MB/s | 249 kB 00:00 2024-09-19T04:51:51.886 INFO:teuthology.orchestra.run.smithi151.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T04:51:51.886 INFO:teuthology.orchestra.run.smithi151.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T04:51:51.897 INFO:teuthology.orchestra.run.smithi151.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T04:51:51.898 INFO:teuthology.orchestra.run.smithi151.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T04:51:51.899 INFO:teuthology.orchestra.run.smithi151.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T04:51:51.900 INFO:teuthology.orchestra.run.smithi151.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T04:51:51.901 INFO:teuthology.orchestra.run.smithi151.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T04:51:51.902 INFO:teuthology.orchestra.run.smithi096.stdout:(33/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 742 kB/s | 49 kB 00:00 2024-09-19T04:51:51.903 INFO:teuthology.orchestra.run.smithi151.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T04:51:51.905 INFO:teuthology.orchestra.run.smithi151.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T04:51:51.961 INFO:teuthology.orchestra.run.smithi096.stdout:(34/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 13 MB/s | 3.6 MB 00:00 2024-09-19T04:51:51.966 INFO:teuthology.orchestra.run.smithi151.stdout:Dependencies resolved. 2024-09-19T04:51:51.983 INFO:teuthology.orchestra.run.smithi151.stdout:====================================================================================== 2024-09-19T04:51:51.983 INFO:teuthology.orchestra.run.smithi151.stdout: Package Arch Version Repository Size 2024-09-19T04:51:51.983 INFO:teuthology.orchestra.run.smithi151.stdout:====================================================================================== 2024-09-19T04:51:51.983 INFO:teuthology.orchestra.run.smithi151.stdout:Installing: 2024-09-19T04:51:51.983 INFO:teuthology.orchestra.run.smithi151.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T04:51:51.984 INFO:teuthology.orchestra.run.smithi151.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T04:51:51.984 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T04:51:51.984 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T04:51:51.984 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T04:51:51.984 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T04:51:51.984 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T04:51:51.984 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T04:51:51.984 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T04:51:51.984 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T04:51:51.984 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T04:51:51.984 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T04:51:51.984 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T04:51:51.985 INFO:teuthology.orchestra.run.smithi151.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T04:51:51.985 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T04:51:51.985 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T04:51:51.985 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T04:51:51.985 INFO:teuthology.orchestra.run.smithi151.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T04:51:51.985 INFO:teuthology.orchestra.run.smithi151.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T04:51:51.985 INFO:teuthology.orchestra.run.smithi151.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T04:51:51.985 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T04:51:51.985 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T04:51:51.985 INFO:teuthology.orchestra.run.smithi151.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T04:51:51.985 INFO:teuthology.orchestra.run.smithi151.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T04:51:51.985 INFO:teuthology.orchestra.run.smithi151.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T04:51:51.985 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T04:51:51.986 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T04:51:51.986 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T04:51:51.986 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T04:51:51.986 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T04:51:51.986 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T04:51:51.986 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T04:51:51.986 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T04:51:51.986 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T04:51:51.986 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T04:51:51.986 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T04:51:51.986 INFO:teuthology.orchestra.run.smithi151.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T04:51:51.986 INFO:teuthology.orchestra.run.smithi151.stdout:Upgrading: 2024-09-19T04:51:51.986 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T04:51:51.987 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T04:51:51.987 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T04:51:51.987 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T04:51:51.987 INFO:teuthology.orchestra.run.smithi151.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T04:51:51.987 INFO:teuthology.orchestra.run.smithi151.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T04:51:51.987 INFO:teuthology.orchestra.run.smithi151.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T04:51:51.987 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T04:51:51.987 INFO:teuthology.orchestra.run.smithi151.stdout:Installing dependencies: 2024-09-19T04:51:51.987 INFO:teuthology.orchestra.run.smithi151.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T04:51:51.987 INFO:teuthology.orchestra.run.smithi151.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T04:51:51.987 INFO:teuthology.orchestra.run.smithi151.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T04:51:51.987 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T04:51:51.987 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T04:51:51.988 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T04:51:51.988 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T04:51:51.988 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T04:51:51.988 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T04:51:51.988 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T04:51:51.988 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T04:51:51.988 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T04:51:51.988 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T04:51:51.988 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T04:51:51.988 INFO:teuthology.orchestra.run.smithi151.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T04:51:51.988 INFO:teuthology.orchestra.run.smithi151.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T04:51:51.988 INFO:teuthology.orchestra.run.smithi151.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T04:51:51.988 INFO:teuthology.orchestra.run.smithi151.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T04:51:51.989 INFO:teuthology.orchestra.run.smithi151.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T04:51:51.989 INFO:teuthology.orchestra.run.smithi151.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T04:51:51.989 INFO:teuthology.orchestra.run.smithi151.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T04:51:51.989 INFO:teuthology.orchestra.run.smithi151.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T04:51:51.989 INFO:teuthology.orchestra.run.smithi151.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T04:51:51.989 INFO:teuthology.orchestra.run.smithi151.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T04:51:51.989 INFO:teuthology.orchestra.run.smithi151.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T04:51:51.989 INFO:teuthology.orchestra.run.smithi151.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T04:51:51.989 INFO:teuthology.orchestra.run.smithi151.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T04:51:51.989 INFO:teuthology.orchestra.run.smithi151.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T04:51:51.989 INFO:teuthology.orchestra.run.smithi151.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T04:51:51.989 INFO:teuthology.orchestra.run.smithi151.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T04:51:51.989 INFO:teuthology.orchestra.run.smithi151.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T04:51:51.990 INFO:teuthology.orchestra.run.smithi151.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T04:51:51.990 INFO:teuthology.orchestra.run.smithi151.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T04:51:51.990 INFO:teuthology.orchestra.run.smithi151.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T04:51:51.990 INFO:teuthology.orchestra.run.smithi151.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T04:51:51.990 INFO:teuthology.orchestra.run.smithi151.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T04:51:51.990 INFO:teuthology.orchestra.run.smithi151.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T04:51:51.990 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T04:51:51.990 INFO:teuthology.orchestra.run.smithi151.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T04:51:51.990 INFO:teuthology.orchestra.run.smithi151.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T04:51:51.990 INFO:teuthology.orchestra.run.smithi151.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T04:51:51.990 INFO:teuthology.orchestra.run.smithi151.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T04:51:51.990 INFO:teuthology.orchestra.run.smithi151.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T04:51:51.990 INFO:teuthology.orchestra.run.smithi151.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T04:51:51.991 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T04:51:51.991 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T04:51:51.991 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T04:51:51.991 INFO:teuthology.orchestra.run.smithi151.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T04:51:51.991 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T04:51:51.991 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T04:51:51.991 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T04:51:51.991 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T04:51:51.991 INFO:teuthology.orchestra.run.smithi151.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T04:51:51.991 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T04:51:51.991 INFO:teuthology.orchestra.run.smithi151.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T04:51:51.991 INFO:teuthology.orchestra.run.smithi151.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T04:51:51.991 INFO:teuthology.orchestra.run.smithi151.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T04:51:51.992 INFO:teuthology.orchestra.run.smithi151.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T04:51:51.992 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T04:51:51.992 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T04:51:51.992 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T04:51:51.992 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T04:51:51.992 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T04:51:51.992 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T04:51:51.992 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T04:51:51.992 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T04:51:51.992 INFO:teuthology.orchestra.run.smithi151.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T04:51:51.992 INFO:teuthology.orchestra.run.smithi151.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T04:51:51.992 INFO:teuthology.orchestra.run.smithi151.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T04:51:51.992 INFO:teuthology.orchestra.run.smithi151.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T04:51:51.993 INFO:teuthology.orchestra.run.smithi151.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T04:51:51.993 INFO:teuthology.orchestra.run.smithi151.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T04:51:51.993 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T04:51:51.993 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T04:51:51.993 INFO:teuthology.orchestra.run.smithi151.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T04:51:51.993 INFO:teuthology.orchestra.run.smithi151.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T04:51:51.993 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T04:51:51.993 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T04:51:51.993 INFO:teuthology.orchestra.run.smithi151.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T04:51:51.993 INFO:teuthology.orchestra.run.smithi151.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T04:51:51.993 INFO:teuthology.orchestra.run.smithi151.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T04:51:51.993 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T04:51:51.993 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T04:51:51.994 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T04:51:51.994 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T04:51:51.994 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T04:51:51.994 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T04:51:51.994 INFO:teuthology.orchestra.run.smithi151.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T04:51:51.994 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T04:51:51.994 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T04:51:51.994 INFO:teuthology.orchestra.run.smithi151.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T04:51:51.994 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T04:51:51.994 INFO:teuthology.orchestra.run.smithi151.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T04:51:51.994 INFO:teuthology.orchestra.run.smithi151.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T04:51:51.994 INFO:teuthology.orchestra.run.smithi151.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T04:51:51.994 INFO:teuthology.orchestra.run.smithi151.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T04:51:51.995 INFO:teuthology.orchestra.run.smithi151.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T04:51:51.995 INFO:teuthology.orchestra.run.smithi151.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T04:51:51.995 INFO:teuthology.orchestra.run.smithi151.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T04:51:51.995 INFO:teuthology.orchestra.run.smithi151.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T04:51:51.995 INFO:teuthology.orchestra.run.smithi151.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T04:51:51.995 INFO:teuthology.orchestra.run.smithi151.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T04:51:51.995 INFO:teuthology.orchestra.run.smithi151.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T04:51:51.995 INFO:teuthology.orchestra.run.smithi151.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T04:51:51.995 INFO:teuthology.orchestra.run.smithi151.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T04:51:51.995 INFO:teuthology.orchestra.run.smithi151.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T04:51:51.995 INFO:teuthology.orchestra.run.smithi151.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T04:51:51.995 INFO:teuthology.orchestra.run.smithi151.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T04:51:51.995 INFO:teuthology.orchestra.run.smithi151.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T04:51:51.996 INFO:teuthology.orchestra.run.smithi151.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T04:51:51.996 INFO:teuthology.orchestra.run.smithi151.stdout:Installing weak dependencies: 2024-09-19T04:51:51.996 INFO:teuthology.orchestra.run.smithi151.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T04:51:51.996 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T04:51:51.996 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction Summary 2024-09-19T04:51:51.996 INFO:teuthology.orchestra.run.smithi151.stdout:====================================================================================== 2024-09-19T04:51:51.996 INFO:teuthology.orchestra.run.smithi151.stdout:Install 148 Packages 2024-09-19T04:51:51.996 INFO:teuthology.orchestra.run.smithi151.stdout:Upgrade 8 Packages 2024-09-19T04:51:51.996 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T04:51:51.997 INFO:teuthology.orchestra.run.smithi096.stdout:(35/156): ceph-prometheus-alerts-19.1.1-361.g2f 197 kB/s | 16 kB 00:00 2024-09-19T04:51:51.997 INFO:teuthology.orchestra.run.smithi151.stdout:Total download size: 226 M 2024-09-19T04:51:51.997 INFO:teuthology.orchestra.run.smithi151.stdout:Downloading Packages: 2024-09-19T04:51:52.011 INFO:teuthology.orchestra.run.smithi096.stdout:(36/156): ceph-volume-19.1.1-361.g2f599a46.el9. 5.5 MB/s | 285 kB 00:00 2024-09-19T04:51:52.062 INFO:teuthology.orchestra.run.smithi096.stdout:(37/156): cephadm-19.1.1-361.g2f599a46.el9.noar 9.9 MB/s | 765 kB 00:00 2024-09-19T04:51:52.087 INFO:teuthology.orchestra.run.smithi096.stdout:(38/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 242 kB/s | 18 kB 00:00 2024-09-19T04:51:52.196 INFO:teuthology.orchestra.run.smithi096.stdout:(39/156): ceph-mgr-diskprediction-local-19.1.1- 15 MB/s | 7.4 MB 00:00 2024-09-19T04:51:52.346 INFO:teuthology.orchestra.run.smithi096.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 281 kB/s | 80 kB 00:00 2024-09-19T04:51:52.433 INFO:teuthology.orchestra.run.smithi096.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 819 kB/s | 191 kB 00:00 2024-09-19T04:51:52.441 INFO:teuthology.orchestra.run.smithi043.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 46 kB/s | 6.6 kB 00:00 2024-09-19T04:51:52.463 INFO:teuthology.orchestra.run.smithi096.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.1 MB/s | 801 kB 00:00 2024-09-19T04:51:52.497 INFO:teuthology.orchestra.run.smithi096.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.6 MB/s | 253 kB 00:00 2024-09-19T04:51:52.530 INFO:teuthology.orchestra.run.smithi096.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-09-19T04:51:52.557 INFO:teuthology.orchestra.run.smithi151.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 53 kB/s | 6.6 kB 00:00 2024-09-19T04:51:52.568 INFO:teuthology.orchestra.run.smithi096.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.9 MB/s | 135 kB 00:00 2024-09-19T04:51:52.610 INFO:teuthology.orchestra.run.smithi096.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-09-19T04:51:52.641 INFO:teuthology.orchestra.run.smithi043.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 4.4 MB/s | 896 kB 00:00 2024-09-19T04:51:52.652 INFO:teuthology.orchestra.run.smithi096.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-19T04:51:52.766 INFO:teuthology.orchestra.run.smithi043.stdout:(3/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 12 MB/s | 5.4 MB 00:00 2024-09-19T04:51:52.791 INFO:teuthology.orchestra.run.smithi043.stdout:(4/156): ceph-immutable-object-cache-19.1.1-361 968 kB/s | 145 kB 00:00 2024-09-19T04:51:52.869 INFO:teuthology.orchestra.run.smithi096.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 2.8 MB/s | 1.2 MB 00:00 2024-09-19T04:51:52.916 INFO:teuthology.orchestra.run.smithi043.stdout:(5/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 16 MB/s | 2.4 MB 00:00 2024-09-19T04:51:53.024 INFO:teuthology.orchestra.run.smithi151.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 1.9 MB/s | 896 kB 00:00 2024-09-19T04:51:53.067 INFO:teuthology.orchestra.run.smithi043.stdout:(6/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 5.5 MB/s | 1.5 MB 00:00 2024-09-19T04:51:53.175 INFO:teuthology.orchestra.run.smithi151.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 968 kB/s | 145 kB 00:00 2024-09-19T04:51:53.184 INFO:teuthology.orchestra.run.smithi043.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 18 MB/s | 4.7 MB 00:00 2024-09-19T04:51:53.478 INFO:teuthology.orchestra.run.smithi096.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 795 kB/s | 689 kB 00:00 2024-09-19T04:51:53.501 INFO:teuthology.orchestra.run.smithi043.stdout:(8/156): ceph-common-19.1.1-361.g2f599a46.el9.x 18 MB/s | 21 MB 00:01 2024-09-19T04:51:53.536 INFO:teuthology.orchestra.run.smithi096.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 162 kB/s | 107 kB 00:00 2024-09-19T04:51:53.551 INFO:teuthology.orchestra.run.smithi043.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 505 kB/s | 25 kB 00:00 2024-09-19T04:51:53.586 INFO:teuthology.orchestra.run.smithi096.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 1.0 MB/s | 943 kB 00:00 2024-09-19T04:51:53.750 INFO:teuthology.orchestra.run.smithi151.stdout:(4/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 4.1 MB/s | 2.4 MB 00:00 2024-09-19T04:51:53.885 INFO:teuthology.orchestra.run.smithi043.stdout:(10/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 15 MB/s | 11 MB 00:00 2024-09-19T04:51:53.975 INFO:teuthology.orchestra.run.smithi151.stdout:(5/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 3.5 MB/s | 5.4 MB 00:01 2024-09-19T04:51:53.993 INFO:teuthology.orchestra.run.smithi043.stdout:(11/156): cephfs-mirror-19.1.1-361.g2f599a46.el 2.1 MB/s | 228 kB 00:00 2024-09-19T04:51:54.011 INFO:teuthology.orchestra.run.smithi096.stdout:(52/156): elfutils-libelf-devel-0.191-4.el9.x86 41 kB/s | 22 kB 00:00 2024-09-19T04:51:54.026 INFO:teuthology.orchestra.run.smithi151.stdout:(6/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 5.5 MB/s | 1.5 MB 00:00 2024-09-19T04:51:54.068 INFO:teuthology.orchestra.run.smithi043.stdout:(12/156): cephfs-shell-19.1.1-361.g2f599a46.el9 362 kB/s | 27 kB 00:00 2024-09-19T04:51:54.137 INFO:teuthology.orchestra.run.smithi096.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 59 kB/s | 33 kB 00:00 2024-09-19T04:51:54.152 INFO:teuthology.orchestra.run.smithi043.stdout:(13/156): libcephfs-devel-19.1.1-361.g2f599a46. 382 kB/s | 32 kB 00:00 2024-09-19T04:51:54.312 INFO:teuthology.orchestra.run.smithi096.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 404 kB/s | 313 kB 00:00 2024-09-19T04:51:54.411 INFO:teuthology.orchestra.run.smithi043.stdout:(14/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 13 MB/s | 17 MB 00:01 2024-09-19T04:51:54.452 INFO:teuthology.orchestra.run.smithi043.stdout:(15/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 2.4 MB/s | 751 kB 00:00 2024-09-19T04:51:54.478 INFO:teuthology.orchestra.run.smithi043.stdout:(16/156): libcephsqlite-19.1.1-361.g2f599a46.el 2.4 MB/s | 163 kB 00:00 2024-09-19T04:51:54.503 INFO:teuthology.orchestra.run.smithi043.stdout:(17/156): librados-devel-19.1.1-361.g2f599a46.e 2.5 MB/s | 127 kB 00:00 2024-09-19T04:51:54.526 INFO:teuthology.orchestra.run.smithi151.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 8.6 MB/s | 4.7 MB 00:00 2024-09-19T04:51:54.537 INFO:teuthology.orchestra.run.smithi043.stdout:(18/156): libradosstriper1-19.1.1-361.g2f599a46 8.4 MB/s | 506 kB 00:00 2024-09-19T04:51:54.595 INFO:teuthology.orchestra.run.smithi043.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 777 kB/s | 45 kB 00:00 2024-09-19T04:51:54.612 INFO:teuthology.orchestra.run.smithi096.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 38 kB/s | 18 kB 00:00 2024-09-19T04:51:54.701 INFO:teuthology.orchestra.run.smithi043.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 1.3 MB/s | 136 kB 00:00 2024-09-19T04:51:54.787 INFO:teuthology.orchestra.run.smithi096.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 23 kB/s | 11 kB 00:00 2024-09-19T04:51:54.795 INFO:teuthology.orchestra.run.smithi043.stdout:(21/156): python3-cephfs-19.1.1-361.g2f599a46.e 1.7 MB/s | 161 kB 00:00 2024-09-19T04:51:54.887 INFO:teuthology.orchestra.run.smithi043.stdout:(22/156): python3-rados-19.1.1-361.g2f599a46.el 3.4 MB/s | 320 kB 00:00 2024-09-19T04:51:54.988 INFO:teuthology.orchestra.run.smithi043.stdout:(23/156): python3-rbd-19.1.1-361.g2f599a46.el9. 2.9 MB/s | 300 kB 00:00 2024-09-19T04:51:55.080 INFO:teuthology.orchestra.run.smithi043.stdout:(24/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.1 MB/s | 99 kB 00:00 2024-09-19T04:51:55.121 INFO:teuthology.orchestra.run.smithi096.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.7 MB/s | 3.0 MB 00:01 2024-09-19T04:51:55.180 INFO:teuthology.orchestra.run.smithi043.stdout:(25/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 855 kB/s | 85 kB 00:00 2024-09-19T04:51:55.330 INFO:teuthology.orchestra.run.smithi096.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 224 kB/s | 160 kB 00:00 2024-09-19T04:51:55.397 INFO:teuthology.orchestra.run.smithi096.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 74 kB/s | 45 kB 00:00 2024-09-19T04:51:55.572 INFO:teuthology.orchestra.run.smithi043.stdout:(26/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 5.0 MB/s | 5.3 MB 00:01 2024-09-19T04:51:55.593 INFO:teuthology.orchestra.run.smithi151.stdout:(8/156): ceph-radosgw-19.1.1-361.g2f599a46.el9. 9.9 MB/s | 11 MB 00:01 2024-09-19T04:51:55.665 INFO:teuthology.orchestra.run.smithi043.stdout:(27/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 6.4 MB/s | 3.1 MB 00:00 2024-09-19T04:51:55.689 INFO:teuthology.orchestra.run.smithi043.stdout:(28/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 1.4 MB/s | 171 kB 00:00 2024-09-19T04:51:55.708 INFO:teuthology.orchestra.run.smithi043.stdout:(29/156): ceph-grafana-dashboards-19.1.1-361.g2 768 kB/s | 32 kB 00:00 2024-09-19T04:51:55.740 INFO:teuthology.orchestra.run.smithi043.stdout:(30/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 2.9 MB/s | 146 kB 00:00 2024-09-19T04:51:55.760 INFO:teuthology.orchestra.run.smithi151.stdout:(9/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86_ 9.7 MB/s | 17 MB 00:01 2024-09-19T04:51:55.785 INFO:teuthology.orchestra.run.smithi151.stdout:(10/156): ceph-selinux-19.1.1-361.g2f599a46.el9 131 kB/s | 25 kB 00:00 2024-09-19T04:51:55.916 INFO:teuthology.orchestra.run.smithi043.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 17 MB/s | 3.6 MB 00:00 2024-09-19T04:51:55.930 INFO:teuthology.orchestra.run.smithi096.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 46 kB/s | 24 kB 00:00 2024-09-19T04:51:55.972 INFO:teuthology.orchestra.run.smithi096.stdout:(61/156): librdkafka-1.6.1-102.el9.x86_64.rpm 780 kB/s | 662 kB 00:00 2024-09-19T04:51:55.977 INFO:teuthology.orchestra.run.smithi151.stdout:(11/156): ceph-common-19.1.1-361.g2f599a46.el9. 6.0 MB/s | 21 MB 00:03 2024-09-19T04:51:55.991 INFO:teuthology.orchestra.run.smithi043.stdout:(32/156): ceph-mgr-modules-core-19.1.1-361.g2f5 3.3 MB/s | 249 kB 00:00 2024-09-19T04:51:56.003 INFO:teuthology.orchestra.run.smithi151.stdout:(12/156): cephfs-mirror-19.1.1-361.g2f599a46.el 1.0 MB/s | 228 kB 00:00 2024-09-19T04:51:56.028 INFO:teuthology.orchestra.run.smithi151.stdout:(13/156): cephfs-shell-19.1.1-361.g2f599a46.el9 539 kB/s | 27 kB 00:00 2024-09-19T04:51:56.041 INFO:teuthology.orchestra.run.smithi043.stdout:(33/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 991 kB/s | 49 kB 00:00 2024-09-19T04:51:56.061 INFO:teuthology.orchestra.run.smithi151.stdout:(14/156): libcephfs-devel-19.1.1-361.g2f599a46. 542 kB/s | 32 kB 00:00 2024-09-19T04:51:56.099 INFO:teuthology.orchestra.run.smithi043.stdout:(34/156): ceph-prometheus-alerts-19.1.1-361.g2f 283 kB/s | 16 kB 00:00 2024-09-19T04:51:56.128 INFO:teuthology.orchestra.run.smithi151.stdout:(15/156): libcephsqlite-19.1.1-361.g2f599a46.el 2.4 MB/s | 163 kB 00:00 2024-09-19T04:51:56.162 INFO:teuthology.orchestra.run.smithi151.stdout:(16/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 5.5 MB/s | 751 kB 00:00 2024-09-19T04:51:56.181 INFO:teuthology.orchestra.run.smithi096.stdout:(62/156): libtool-2.4.6-46.el9.x86_64.rpm 683 kB/s | 581 kB 00:00 2024-09-19T04:51:56.183 INFO:teuthology.orchestra.run.smithi043.stdout:(35/156): ceph-volume-19.1.1-361.g2f599a46.el9. 3.3 MB/s | 285 kB 00:00 2024-09-19T04:51:56.205 INFO:teuthology.orchestra.run.smithi151.stdout:(17/156): librados-devel-19.1.1-361.g2f599a46.e 1.6 MB/s | 127 kB 00:00 2024-09-19T04:51:56.247 INFO:teuthology.orchestra.run.smithi151.stdout:(18/156): libradosstriper1-19.1.1-361.g2f599a46 5.9 MB/s | 506 kB 00:00 2024-09-19T04:51:56.314 INFO:teuthology.orchestra.run.smithi151.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 679 kB/s | 45 kB 00:00 2024-09-19T04:51:56.381 INFO:teuthology.orchestra.run.smithi151.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 2.0 MB/s | 136 kB 00:00 2024-09-19T04:51:56.447 INFO:teuthology.orchestra.run.smithi151.stdout:(21/156): python3-cephfs-19.1.1-361.g2f599a46.e 2.4 MB/s | 161 kB 00:00 2024-09-19T04:51:56.497 INFO:teuthology.orchestra.run.smithi096.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 84 kB/s | 47 kB 00:00 2024-09-19T04:51:56.548 INFO:teuthology.orchestra.run.smithi151.stdout:(22/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 16 MB/s | 5.3 MB 00:00 2024-09-19T04:51:56.573 INFO:teuthology.orchestra.run.smithi151.stdout:(23/156): python3-rados-19.1.1-361.g2f599a46.el 2.5 MB/s | 320 kB 00:00 2024-09-19T04:51:56.615 INFO:teuthology.orchestra.run.smithi151.stdout:(24/156): python3-rbd-19.1.1-361.g2f599a46.el9. 4.4 MB/s | 300 kB 00:00 2024-09-19T04:51:56.641 INFO:teuthology.orchestra.run.smithi151.stdout:(25/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.5 MB/s | 99 kB 00:00 2024-09-19T04:51:56.666 INFO:teuthology.orchestra.run.smithi151.stdout:(26/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-19T04:51:56.667 INFO:teuthology.orchestra.run.smithi043.stdout:(36/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 16 MB/s | 49 MB 00:03 2024-09-19T04:51:56.709 INFO:teuthology.orchestra.run.smithi043.stdout:(37/156): cephadm-19.1.1-361.g2f599a46.el9.noar 1.4 MB/s | 765 kB 00:00 2024-09-19T04:51:56.717 INFO:teuthology.orchestra.run.smithi151.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 3.3 MB/s | 171 kB 00:00 2024-09-19T04:51:56.725 INFO:teuthology.orchestra.run.smithi043.stdout:(38/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 314 kB/s | 18 kB 00:00 2024-09-19T04:51:56.748 INFO:teuthology.orchestra.run.smithi096.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 377 kB/s | 292 kB 00:00 2024-09-19T04:51:56.775 INFO:teuthology.orchestra.run.smithi151.stdout:(28/156): ceph-grafana-dashboards-19.1.1-361.g2 548 kB/s | 32 kB 00:00 2024-09-19T04:51:56.851 INFO:teuthology.orchestra.run.smithi043.stdout:(39/156): ceph-mgr-diskprediction-local-19.1.1- 6.7 MB/s | 7.4 MB 00:01 2024-09-19T04:51:56.859 INFO:teuthology.orchestra.run.smithi151.stdout:(29/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 14 MB/s | 3.1 MB 00:00 2024-09-19T04:51:56.884 INFO:teuthology.orchestra.run.smithi151.stdout:(30/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 1.3 MB/s | 146 kB 00:00 2024-09-19T04:51:56.884 INFO:teuthology.orchestra.run.smithi043.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 456 kB/s | 80 kB 00:00 2024-09-19T04:51:56.956 INFO:teuthology.orchestra.run.smithi096.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 243 kB/s | 188 kB 00:00 2024-09-19T04:51:56.959 INFO:teuthology.orchestra.run.smithi043.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.3 MB/s | 253 kB 00:00 2024-09-19T04:51:57.010 INFO:teuthology.orchestra.run.smithi043.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.8 MB/s | 801 kB 00:00 2024-09-19T04:51:57.056 INFO:teuthology.orchestra.run.smithi096.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 69 kB/s | 38 kB 00:00 2024-09-19T04:51:57.068 INFO:teuthology.orchestra.run.smithi043.stdout:(43/156): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-09-19T04:51:57.093 INFO:teuthology.orchestra.run.smithi043.stdout:(44/156): libquadmath-11.5.0-2.el9.x86_64.rpm 789 kB/s | 191 kB 00:00 2024-09-19T04:51:57.118 INFO:teuthology.orchestra.run.smithi151.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 14 MB/s | 3.6 MB 00:00 2024-09-19T04:51:57.119 INFO:teuthology.orchestra.run.smithi043.stdout:(45/156): python3-ply-3.11-14.el9.noarch.rpm 978 kB/s | 106 kB 00:00 2024-09-19T04:51:57.144 INFO:teuthology.orchestra.run.smithi043.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 1.7 MB/s | 135 kB 00:00 2024-09-19T04:51:57.177 INFO:teuthology.orchestra.run.smithi151.stdout:(32/156): ceph-mgr-modules-core-19.1.1-361.g2f5 4.2 MB/s | 249 kB 00:00 2024-09-19T04:51:57.203 INFO:teuthology.orchestra.run.smithi043.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-09-19T04:51:57.228 INFO:teuthology.orchestra.run.smithi043.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-09-19T04:51:57.244 INFO:teuthology.orchestra.run.smithi151.stdout:(33/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 741 kB/s | 49 kB 00:00 2024-09-19T04:51:57.308 INFO:teuthology.orchestra.run.smithi151.stdout:(34/156): ceph-prometheus-alerts-19.1.1-361.g2f 258 kB/s | 16 kB 00:00 2024-09-19T04:51:57.403 INFO:teuthology.orchestra.run.smithi043.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 611 kB/s | 107 kB 00:00 2024-09-19T04:51:57.408 INFO:teuthology.orchestra.run.smithi151.stdout:(35/156): ceph-mgr-diskprediction-local-19.1.1- 14 MB/s | 7.4 MB 00:00 2024-09-19T04:51:57.433 INFO:teuthology.orchestra.run.smithi151.stdout:(36/156): ceph-volume-19.1.1-361.g2f599a46.el9. 2.2 MB/s | 285 kB 00:00 2024-09-19T04:51:57.437 INFO:teuthology.orchestra.run.smithi043.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 2.3 MB/s | 689 kB 00:00 2024-09-19T04:51:57.492 INFO:teuthology.orchestra.run.smithi151.stdout:(37/156): cephadm-19.1.1-361.g2f599a46.el9.noar 9.0 MB/s | 765 kB 00:00 2024-09-19T04:51:57.518 INFO:teuthology.orchestra.run.smithi151.stdout:(38/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 218 kB/s | 18 kB 00:00 2024-09-19T04:51:57.587 INFO:teuthology.orchestra.run.smithi043.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 118 kB/s | 22 kB 00:00 2024-09-19T04:51:57.612 INFO:teuthology.orchestra.run.smithi043.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 1.7 MB/s | 313 kB 00:00 2024-09-19T04:51:57.626 INFO:teuthology.orchestra.run.smithi151.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 597 kB/s | 80 kB 00:00 2024-09-19T04:51:57.646 INFO:teuthology.orchestra.run.smithi043.stdout:(53/156): bison-3.7.4-5.el9.x86_64.rpm 2.1 MB/s | 943 kB 00:00 2024-09-19T04:51:57.663 INFO:teuthology.orchestra.run.smithi043.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 434 kB/s | 33 kB 00:00 2024-09-19T04:51:57.676 INFO:teuthology.orchestra.run.smithi151.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-19T04:51:57.688 INFO:teuthology.orchestra.run.smithi043.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 437 kB/s | 18 kB 00:00 2024-09-19T04:51:57.705 INFO:teuthology.orchestra.run.smithi043.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 267 kB/s | 11 kB 00:00 2024-09-19T04:51:57.719 INFO:teuthology.orchestra.run.smithi151.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.9 MB/s | 801 kB 00:00 2024-09-19T04:51:57.739 INFO:teuthology.orchestra.run.smithi043.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-19T04:51:57.761 INFO:teuthology.orchestra.run.smithi151.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-09-19T04:51:57.764 INFO:teuthology.orchestra.run.smithi043.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 768 kB/s | 45 kB 00:00 2024-09-19T04:51:57.803 INFO:teuthology.orchestra.run.smithi151.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-19T04:51:57.831 INFO:teuthology.orchestra.run.smithi043.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 14 MB/s | 3.0 MB 00:00 2024-09-19T04:51:57.845 INFO:teuthology.orchestra.run.smithi151.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-09-19T04:51:57.865 INFO:teuthology.orchestra.run.smithi043.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 5.1 MB/s | 662 kB 00:00 2024-09-19T04:51:57.882 INFO:teuthology.orchestra.run.smithi043.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 490 kB/s | 24 kB 00:00 2024-09-19T04:51:57.886 INFO:teuthology.orchestra.run.smithi151.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-09-19T04:51:57.907 INFO:teuthology.orchestra.run.smithi043.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-09-19T04:51:57.929 INFO:teuthology.orchestra.run.smithi151.stdout:(46/156): python3-urllib3-1.26.5-6.el9.noarch.r 5.0 MB/s | 215 kB 00:00 2024-09-19T04:51:57.941 INFO:teuthology.orchestra.run.smithi043.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-09-19T04:51:57.948 INFO:teuthology.orchestra.run.smithi096.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 4.1 MB/s | 4.9 MB 00:01 2024-09-19T04:51:57.962 INFO:teuthology.orchestra.run.smithi151.stdout:(47/156): python3-cryptography-36.0.1-4.el9.x86 5.0 MB/s | 1.2 MB 00:00 2024-09-19T04:51:57.974 INFO:teuthology.orchestra.run.smithi043.stdout:(64/156): libtool-2.4.6-46.el9.x86_64.rpm 2.7 MB/s | 581 kB 00:00 2024-09-19T04:51:57.991 INFO:teuthology.orchestra.run.smithi043.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 770 kB/s | 38 kB 00:00 2024-09-19T04:51:58.016 INFO:teuthology.orchestra.run.smithi096.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 1.7 MB/s | 1.6 MB 00:00 2024-09-19T04:51:58.033 INFO:teuthology.orchestra.run.smithi043.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-09-19T04:51:58.082 INFO:teuthology.orchestra.run.smithi096.stdout:(69/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.9 MB/s | 4.4 MB 00:01 2024-09-19T04:51:58.184 INFO:teuthology.orchestra.run.smithi043.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-19T04:51:58.300 INFO:teuthology.orchestra.run.smithi043.stdout:(68/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-09-19T04:51:58.384 INFO:teuthology.orchestra.run.smithi043.stdout:(69/156): openblas-openmp-0.3.26-2.el9.x86_64.r 12 MB/s | 4.9 MB 00:00 2024-09-19T04:51:58.409 INFO:teuthology.orchestra.run.smithi043.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 1.2 MB/s | 137 kB 00:00 2024-09-19T04:51:58.443 INFO:teuthology.orchestra.run.smithi043.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 3.4 MB/s | 910 kB 00:00 2024-09-19T04:51:58.501 INFO:teuthology.orchestra.run.smithi043.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-09-19T04:51:58.521 INFO:teuthology.orchestra.run.smithi151.stdout:(48/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 18 MB/s | 49 MB 00:02 2024-09-19T04:51:58.526 INFO:teuthology.orchestra.run.smithi043.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-09-19T04:51:58.555 INFO:teuthology.orchestra.run.smithi151.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 1.1 MB/s | 689 kB 00:00 2024-09-19T04:51:58.568 INFO:teuthology.orchestra.run.smithi043.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-19T04:51:58.597 INFO:teuthology.orchestra.run.smithi151.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 1.5 MB/s | 943 kB 00:00 2024-09-19T04:51:58.622 INFO:teuthology.orchestra.run.smithi151.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 324 kB/s | 22 kB 00:00 2024-09-19T04:51:58.635 INFO:teuthology.orchestra.run.smithi043.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 3.6 MB/s | 245 kB 00:00 2024-09-19T04:51:58.656 INFO:teuthology.orchestra.run.smithi151.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-09-19T04:51:58.682 INFO:teuthology.orchestra.run.smithi151.stdout:(53/156): boost-program-options-1.75.0-8.el9.x8 669 kB/s | 107 kB 00:00 2024-09-19T04:51:58.707 INFO:teuthology.orchestra.run.smithi096.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 199 kB/s | 137 kB 00:00 2024-09-19T04:51:58.723 INFO:teuthology.orchestra.run.smithi151.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 324 kB/s | 33 kB 00:00 2024-09-19T04:51:58.766 INFO:teuthology.orchestra.run.smithi151.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 269 kB/s | 11 kB 00:00 2024-09-19T04:51:58.791 INFO:teuthology.orchestra.run.smithi151.stdout:(56/156): flexiblas-openblas-openmp-3.0.4-8.el9 273 kB/s | 18 kB 00:00 2024-09-19T04:51:58.833 INFO:teuthology.orchestra.run.smithi151.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-09-19T04:51:58.858 INFO:teuthology.orchestra.run.smithi151.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-09-19T04:51:58.892 INFO:teuthology.orchestra.run.smithi043.stdout:(76/156): postgresql-server-13.16-1.el9.x86_64. 11 MB/s | 5.8 MB 00:00 2024-09-19T04:51:58.916 INFO:teuthology.orchestra.run.smithi096.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 941 kB/s | 910 kB 00:00 2024-09-19T04:51:58.919 INFO:teuthology.orchestra.run.smithi043.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 875 kB/s | 248 kB 00:00 2024-09-19T04:51:58.942 INFO:teuthology.orchestra.run.smithi151.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 10 MB/s | 3.0 MB 00:00 2024-09-19T04:51:58.944 INFO:teuthology.orchestra.run.smithi043.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 492 kB/s | 26 kB 00:00 2024-09-19T04:51:58.969 INFO:teuthology.orchestra.run.smithi043.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-09-19T04:51:58.975 INFO:teuthology.orchestra.run.smithi151.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 4.5 MB/s | 662 kB 00:00 2024-09-19T04:51:59.001 INFO:teuthology.orchestra.run.smithi151.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 418 kB/s | 24 kB 00:00 2024-09-19T04:51:59.003 INFO:teuthology.orchestra.run.smithi043.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-19T04:51:59.028 INFO:teuthology.orchestra.run.smithi043.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 593 kB/s | 35 kB 00:00 2024-09-19T04:51:59.034 INFO:teuthology.orchestra.run.smithi151.stdout:(62/156): libtool-2.4.6-46.el9.x86_64.rpm 3.2 MB/s | 581 kB 00:00 2024-09-19T04:51:59.060 INFO:teuthology.orchestra.run.smithi151.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 564 kB/s | 47 kB 00:00 2024-09-19T04:51:59.085 INFO:teuthology.orchestra.run.smithi151.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-09-19T04:51:59.110 INFO:teuthology.orchestra.run.smithi151.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 767 kB/s | 38 kB 00:00 2024-09-19T04:51:59.120 INFO:teuthology.orchestra.run.smithi043.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 9.6 MB/s | 6.0 MB 00:00 2024-09-19T04:51:59.136 INFO:teuthology.orchestra.run.smithi151.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-09-19T04:51:59.154 INFO:teuthology.orchestra.run.smithi043.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.0 MB/s | 250 kB 00:00 2024-09-19T04:51:59.188 INFO:teuthology.orchestra.run.smithi043.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.2 MB/s | 222 kB 00:00 2024-09-19T04:51:59.213 INFO:teuthology.orchestra.run.smithi043.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-19T04:51:59.238 INFO:teuthology.orchestra.run.smithi043.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 844 kB/s | 42 kB 00:00 2024-09-19T04:51:59.245 INFO:teuthology.orchestra.run.smithi151.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 31 MB/s | 4.9 MB 00:00 2024-09-19T04:51:59.266 INFO:teuthology.orchestra.run.smithi096.stdout:(72/156): postgresql-server-13.16-1.el9.x86_64. 4.9 MB/s | 5.8 MB 00:01 2024-09-19T04:51:59.271 INFO:teuthology.orchestra.run.smithi043.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-19T04:51:59.297 INFO:teuthology.orchestra.run.smithi043.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-09-19T04:51:59.303 INFO:teuthology.orchestra.run.smithi151.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 15 MB/s | 910 kB 00:00 2024-09-19T04:51:59.331 INFO:teuthology.orchestra.run.smithi043.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.6 MB/s | 279 kB 00:00 2024-09-19T04:51:59.345 INFO:teuthology.orchestra.run.smithi151.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 3.2 MB/s | 137 kB 00:00 2024-09-19T04:51:59.356 INFO:teuthology.orchestra.run.smithi043.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 868 kB/s | 51 kB 00:00 2024-09-19T04:51:59.440 INFO:teuthology.orchestra.run.smithi043.stdout:(91/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-09-19T04:51:59.457 INFO:teuthology.orchestra.run.smithi043.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 428 kB/s | 54 kB 00:00 2024-09-19T04:51:59.490 INFO:teuthology.orchestra.run.smithi043.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 831 kB/s | 42 kB 00:00 2024-09-19T04:51:59.507 INFO:teuthology.orchestra.run.smithi043.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 879 kB/s | 44 kB 00:00 2024-09-19T04:51:59.512 INFO:teuthology.orchestra.run.smithi151.stdout:(70/156): postgresql-server-13.16-1.el9.x86_64. 35 MB/s | 5.8 MB 00:00 2024-09-19T04:51:59.549 INFO:teuthology.orchestra.run.smithi043.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-19T04:51:59.554 INFO:teuthology.orchestra.run.smithi151.stdout:(71/156): postgresql-13.16-1.el9.x86_64.rpm 3.8 MB/s | 1.6 MB 00:00 2024-09-19T04:51:59.566 INFO:teuthology.orchestra.run.smithi043.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 614 kB/s | 36 kB 00:00 2024-09-19T04:51:59.596 INFO:teuthology.orchestra.run.smithi151.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-09-19T04:51:59.599 INFO:teuthology.orchestra.run.smithi043.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 950 kB/s | 47 kB 00:00 2024-09-19T04:51:59.621 INFO:teuthology.orchestra.run.smithi151.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-19T04:51:59.625 INFO:teuthology.orchestra.run.smithi096.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 123 kB/s | 87 kB 00:00 2024-09-19T04:51:59.625 INFO:teuthology.orchestra.run.smithi043.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-09-19T04:51:59.658 INFO:teuthology.orchestra.run.smithi096.stdout:(74/156): protobuf-3.14.0-13.el9.x86_64.rpm 1.1 MB/s | 1.0 MB 00:00 2024-09-19T04:51:59.675 INFO:teuthology.orchestra.run.smithi043.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 767 kB/s | 58 kB 00:00 2024-09-19T04:51:59.689 INFO:teuthology.orchestra.run.smithi151.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 916 kB/s | 61 kB 00:00 2024-09-19T04:51:59.709 INFO:teuthology.orchestra.run.smithi043.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.1 MB/s | 96 kB 00:00 2024-09-19T04:51:59.752 INFO:teuthology.orchestra.run.smithi043.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 829 kB/s | 64 kB 00:00 2024-09-19T04:51:59.764 INFO:teuthology.orchestra.run.smithi151.stdout:(75/156): python3-babel-2.9.1-2.el9.noarch.rpm 36 MB/s | 6.0 MB 00:00 2024-09-19T04:51:59.776 INFO:teuthology.orchestra.run.smithi043.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 669 kB/s | 45 kB 00:00 2024-09-19T04:51:59.823 INFO:teuthology.orchestra.run.smithi151.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.1 MB/s | 248 kB 00:00 2024-09-19T04:51:59.865 INFO:teuthology.orchestra.run.smithi151.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 613 kB/s | 26 kB 00:00 2024-09-19T04:51:59.890 INFO:teuthology.orchestra.run.smithi151.stdout:(78/156): python3-devel-3.9.19-8.el9.x86_64.rpm 1.2 MB/s | 245 kB 00:00 2024-09-19T04:51:59.910 INFO:teuthology.orchestra.run.smithi043.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 142 kB/s | 22 kB 00:00 2024-09-19T04:51:59.915 INFO:teuthology.orchestra.run.smithi151.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-09-19T04:51:59.958 INFO:teuthology.orchestra.run.smithi151.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-09-19T04:51:59.968 INFO:teuthology.orchestra.run.smithi043.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-19T04:51:59.983 INFO:teuthology.orchestra.run.smithi151.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 517 kB/s | 35 kB 00:00 2024-09-19T04:52:00.050 INFO:teuthology.orchestra.run.smithi151.stdout:(82/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 4.7 MB/s | 4.4 MB 00:00 2024-09-19T04:52:00.075 INFO:teuthology.orchestra.run.smithi151.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-09-19T04:52:00.101 INFO:teuthology.orchestra.run.smithi151.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-19T04:52:00.126 INFO:teuthology.orchestra.run.smithi151.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-19T04:52:00.151 INFO:teuthology.orchestra.run.smithi151.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 838 kB/s | 42 kB 00:00 2024-09-19T04:52:00.161 INFO:teuthology.orchestra.run.smithi043.stdout:(105/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-09-19T04:52:00.202 INFO:teuthology.orchestra.run.smithi043.stdout:(106/156): gperftools-libs-2.9.1-3.el9.x86_64.r 1.3 MB/s | 308 kB 00:00 2024-09-19T04:52:00.218 INFO:teuthology.orchestra.run.smithi151.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 2.9 MB/s | 269 kB 00:00 2024-09-19T04:52:00.236 INFO:teuthology.orchestra.run.smithi043.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 260 kB/s | 19 kB 00:00 2024-09-19T04:52:00.244 INFO:teuthology.orchestra.run.smithi151.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-09-19T04:52:00.250 INFO:teuthology.orchestra.run.smithi096.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 98 kB/s | 61 kB 00:00 2024-09-19T04:52:00.269 INFO:teuthology.orchestra.run.smithi151.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-19T04:52:00.278 INFO:teuthology.orchestra.run.smithi043.stdout:(108/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.7 MB/s | 863 kB 00:00 2024-09-19T04:52:00.294 INFO:teuthology.orchestra.run.smithi151.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-19T04:52:00.295 INFO:teuthology.orchestra.run.smithi043.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 459 kB/s | 26 kB 00:00 2024-09-19T04:52:00.320 INFO:teuthology.orchestra.run.smithi151.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-19T04:52:00.337 INFO:teuthology.orchestra.run.smithi043.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.1 MB/s | 67 kB 00:00 2024-09-19T04:52:00.362 INFO:teuthology.orchestra.run.smithi043.stdout:(111/156): liboath-2.6.7-2.el9.x86_64.rpm 581 kB/s | 49 kB 00:00 2024-09-19T04:52:00.387 INFO:teuthology.orchestra.run.smithi043.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 3.1 MB/s | 151 kB 00:00 2024-09-19T04:52:00.395 INFO:teuthology.orchestra.run.smithi151.stdout:(92/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-09-19T04:52:00.417 INFO:teuthology.orchestra.run.smithi096.stdout:(76/156): python3-babel-2.9.1-2.el9.noarch.rpm 5.2 MB/s | 6.0 MB 00:01 2024-09-19T04:52:00.420 INFO:teuthology.orchestra.run.smithi151.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 415 kB/s | 42 kB 00:00 2024-09-19T04:52:00.442 INFO:teuthology.orchestra.run.smithi096.stdout:(77/156): python3-devel-3.9.19-8.el9.x86_64.rpm 312 kB/s | 245 kB 00:00 2024-09-19T04:52:00.445 INFO:teuthology.orchestra.run.smithi151.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 880 kB/s | 44 kB 00:00 2024-09-19T04:52:00.599 INFO:teuthology.orchestra.run.smithi043.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 8.0 MB/s | 548 kB 00:00 2024-09-19T04:52:00.599 INFO:teuthology.orchestra.run.smithi043.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-19T04:52:00.599 INFO:teuthology.orchestra.run.smithi043.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-19T04:52:00.599 INFO:teuthology.orchestra.run.smithi043.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 847 kB/s | 14 kB 00:00 2024-09-19T04:52:00.600 INFO:teuthology.orchestra.run.smithi043.stdout:(117/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.9 MB/s | 839 kB 00:00 2024-09-19T04:52:00.600 INFO:teuthology.orchestra.run.smithi151.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-19T04:52:00.600 INFO:teuthology.orchestra.run.smithi151.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 611 kB/s | 36 kB 00:00 2024-09-19T04:52:00.600 INFO:teuthology.orchestra.run.smithi151.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-09-19T04:52:00.600 INFO:teuthology.orchestra.run.smithi151.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-19T04:52:00.600 INFO:teuthology.orchestra.run.smithi151.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-19T04:52:00.605 INFO:teuthology.orchestra.run.smithi043.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-09-19T04:52:00.615 INFO:teuthology.orchestra.run.smithi151.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-19T04:52:00.634 INFO:teuthology.orchestra.run.smithi151.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-19T04:52:00.649 INFO:teuthology.orchestra.run.smithi151.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-09-19T04:52:00.681 INFO:teuthology.orchestra.run.smithi043.stdout:(119/156): libarrow-9.0.0-11.el9.x86_64.rpm 9.3 MB/s | 4.4 MB 00:00 2024-09-19T04:52:00.706 INFO:teuthology.orchestra.run.smithi043.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-09-19T04:52:00.731 INFO:teuthology.orchestra.run.smithi043.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-09-19T04:52:00.757 INFO:teuthology.orchestra.run.smithi043.stdout:(122/156): python3-google-auth-2.34.0-1.el9.noa 2.9 MB/s | 220 kB 00:00 2024-09-19T04:52:00.782 INFO:teuthology.orchestra.run.smithi043.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-09-19T04:52:00.808 INFO:teuthology.orchestra.run.smithi043.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 212 kB/s | 11 kB 00:00 2024-09-19T04:52:00.833 INFO:teuthology.orchestra.run.smithi043.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 353 kB/s | 18 kB 00:00 2024-09-19T04:52:00.858 INFO:teuthology.orchestra.run.smithi043.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 463 kB/s | 23 kB 00:00 2024-09-19T04:52:00.883 INFO:teuthology.orchestra.run.smithi043.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-09-19T04:52:00.907 INFO:teuthology.orchestra.run.smithi151.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-09-19T04:52:00.908 INFO:teuthology.orchestra.run.smithi043.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-19T04:52:00.924 INFO:teuthology.orchestra.run.smithi151.stdout:(104/156): lua-devel-5.4.4-4.el9.x86_64.rpm 76 kB/s | 22 kB 00:00 2024-09-19T04:52:00.926 INFO:teuthology.orchestra.run.smithi096.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 40 kB/s | 19 kB 00:00 2024-09-19T04:52:00.941 INFO:teuthology.orchestra.run.smithi043.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-09-19T04:52:00.960 INFO:teuthology.orchestra.run.smithi096.stdout:(79/156): python3-jsonpatch-1.21-16.el9.noarch. 47 kB/s | 26 kB 00:00 2024-09-19T04:52:00.975 INFO:teuthology.orchestra.run.smithi043.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-19T04:52:01.000 INFO:teuthology.orchestra.run.smithi151.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 4.0 MB/s | 308 kB 00:00 2024-09-19T04:52:01.009 INFO:teuthology.orchestra.run.smithi043.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 8.1 MB/s | 1.0 MB 00:00 2024-09-19T04:52:01.034 INFO:teuthology.orchestra.run.smithi043.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 989 kB/s | 58 kB 00:00 2024-09-19T04:52:01.035 INFO:teuthology.orchestra.run.smithi096.stdout:(80/156): python3-jinja2-2.11.3-6.el9.noarch.rp 317 kB/s | 248 kB 00:00 2024-09-19T04:52:01.060 INFO:teuthology.orchestra.run.smithi043.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 654 kB/s | 16 kB 00:00 2024-09-19T04:52:01.084 INFO:teuthology.orchestra.run.smithi151.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 233 kB/s | 19 kB 00:00 2024-09-19T04:52:01.085 INFO:teuthology.orchestra.run.smithi043.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-19T04:52:01.109 INFO:teuthology.orchestra.run.smithi151.stdout:(107/156): abseil-cpp-20211102.0-3.el9.x86_64.r 2.7 MB/s | 548 kB 00:00 2024-09-19T04:52:01.118 INFO:teuthology.orchestra.run.smithi043.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-09-19T04:52:01.143 INFO:teuthology.orchestra.run.smithi043.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 484 kB/s | 28 kB 00:00 2024-09-19T04:52:01.151 INFO:teuthology.orchestra.run.smithi151.stdout:(108/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.7 MB/s | 863 kB 00:00 2024-09-19T04:52:01.169 INFO:teuthology.orchestra.run.smithi043.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 1.2 MB/s | 31 kB 00:00 2024-09-19T04:52:01.176 INFO:teuthology.orchestra.run.smithi151.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 388 kB/s | 26 kB 00:00 2024-09-19T04:52:01.201 INFO:teuthology.orchestra.run.smithi151.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-19T04:52:01.210 INFO:teuthology.orchestra.run.smithi043.stdout:(138/156): python3-grpcio-1.46.7-10.el9.x86_64. 4.0 MB/s | 2.0 MB 00:00 2024-09-19T04:52:01.228 INFO:teuthology.orchestra.run.smithi043.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-09-19T04:52:01.243 INFO:teuthology.orchestra.run.smithi151.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 4.3 MB/s | 151 kB 00:00 2024-09-19T04:52:01.253 INFO:teuthology.orchestra.run.smithi043.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 851 kB/s | 36 kB 00:00 2024-09-19T04:52:01.269 INFO:teuthology.orchestra.run.smithi151.stdout:(112/156): libunwind-1.6.2-1.el9.x86_64.rpm 729 kB/s | 67 kB 00:00 2024-09-19T04:52:01.278 INFO:teuthology.orchestra.run.smithi043.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-09-19T04:52:01.303 INFO:teuthology.orchestra.run.smithi043.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 991 kB/s | 74 kB 00:00 2024-09-19T04:52:01.329 INFO:teuthology.orchestra.run.smithi043.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-19T04:52:01.335 INFO:teuthology.orchestra.run.smithi151.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-09-19T04:52:01.354 INFO:teuthology.orchestra.run.smithi043.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-19T04:52:01.369 INFO:teuthology.orchestra.run.smithi151.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 5.3 MB/s | 548 kB 00:00 2024-09-19T04:52:01.379 INFO:teuthology.orchestra.run.smithi043.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-09-19T04:52:01.395 INFO:teuthology.orchestra.run.smithi151.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 740 kB/s | 43 kB 00:00 2024-09-19T04:52:01.420 INFO:teuthology.orchestra.run.smithi151.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 636 kB/s | 32 kB 00:00 2024-09-19T04:52:01.429 INFO:teuthology.orchestra.run.smithi043.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 2.5 MB/s | 191 kB 00:00 2024-09-19T04:52:01.445 INFO:teuthology.orchestra.run.smithi151.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 280 kB/s | 14 kB 00:00 2024-09-19T04:52:01.455 INFO:teuthology.orchestra.run.smithi043.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.8 MB/s | 427 kB 00:00 2024-09-19T04:52:01.479 INFO:teuthology.orchestra.run.smithi151.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-19T04:52:01.513 INFO:teuthology.orchestra.run.smithi043.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-19T04:52:01.518 INFO:teuthology.orchestra.run.smithi096.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 62 kB/s | 35 kB 00:00 2024-09-19T04:52:01.521 INFO:teuthology.orchestra.run.smithi151.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-09-19T04:52:01.588 INFO:teuthology.orchestra.run.smithi151.stdout:(120/156): libarrow-9.0.0-11.el9.x86_64.rpm 8.8 MB/s | 4.4 MB 00:00 2024-09-19T04:52:01.613 INFO:teuthology.orchestra.run.smithi151.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.2 MB/s | 303 kB 00:00 2024-09-19T04:52:01.635 INFO:teuthology.orchestra.run.smithi096.stdout:(82/156): python3-mako-1.1.4-6.el9.noarch.rpm 243 kB/s | 172 kB 00:00 2024-09-19T04:52:01.638 INFO:teuthology.orchestra.run.smithi151.stdout:(122/156): python3-google-auth-2.34.0-1.el9.noa 1.8 MB/s | 220 kB 00:00 2024-09-19T04:52:01.665 INFO:teuthology.orchestra.run.smithi151.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-09-19T04:52:01.690 INFO:teuthology.orchestra.run.smithi151.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 211 kB/s | 11 kB 00:00 2024-09-19T04:52:01.707 INFO:teuthology.orchestra.run.smithi151.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 421 kB/s | 18 kB 00:00 2024-09-19T04:52:01.722 INFO:teuthology.orchestra.run.smithi043.stdout:(149/156): NetworkManager-1.51.0-1.el9.x86_64.r 12 MB/s | 2.3 MB 00:00 2024-09-19T04:52:01.723 INFO:teuthology.orchestra.run.smithi151.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 689 kB/s | 23 kB 00:00 2024-09-19T04:52:01.741 INFO:teuthology.orchestra.run.smithi151.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 576 kB/s | 19 kB 00:00 2024-09-19T04:52:01.758 INFO:teuthology.orchestra.run.smithi151.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 584 kB/s | 20 kB 00:00 2024-09-19T04:52:01.783 INFO:teuthology.orchestra.run.smithi151.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-19T04:52:01.833 INFO:teuthology.orchestra.run.smithi151.stdout:(130/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.3 MB/s | 2.0 MB 00:00 2024-09-19T04:52:01.839 INFO:teuthology.orchestra.run.smithi043.stdout:(150/156): NetworkManager-libnm-1.51.0-1.el9.x8 16 MB/s | 1.8 MB 00:00 2024-09-19T04:52:01.860 INFO:teuthology.orchestra.run.smithi151.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-09-19T04:52:01.884 INFO:teuthology.orchestra.run.smithi151.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-09-19T04:52:01.910 INFO:teuthology.orchestra.run.smithi151.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-09-19T04:52:01.935 INFO:teuthology.orchestra.run.smithi151.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 326 kB/s | 16 kB 00:00 2024-09-19T04:52:01.956 INFO:teuthology.orchestra.run.smithi043.stdout:(151/156): librbd1-19.1.1-361.g2f599a46.el9.x86 6.3 MB/s | 3.2 MB 00:00 2024-09-19T04:52:01.960 INFO:teuthology.orchestra.run.smithi151.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-19T04:52:01.981 INFO:teuthology.orchestra.run.smithi043.stdout:(152/156): NetworkManager-team-1.51.0-1.el9.x86 288 kB/s | 41 kB 00:00 2024-09-19T04:52:02.003 INFO:teuthology.orchestra.run.smithi151.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 973 kB/s | 90 kB 00:00 2024-09-19T04:52:02.028 INFO:teuthology.orchestra.run.smithi151.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 458 kB/s | 31 kB 00:00 2024-09-19T04:52:02.048 INFO:teuthology.orchestra.run.smithi043.stdout:(153/156): librados2-19.1.1-361.g2f599a46.el9.x 5.5 MB/s | 3.4 MB 00:00 2024-09-19T04:52:02.062 INFO:teuthology.orchestra.run.smithi151.stdout:(138/156): python3-kubernetes-26.1.0-2.el9.noar 3.2 MB/s | 1.0 MB 00:00 2024-09-19T04:52:02.074 INFO:teuthology.orchestra.run.smithi043.stdout:(154/156): NetworkManager-tui-1.51.0-1.el9.x86_ 2.1 MB/s | 249 kB 00:00 2024-09-19T04:52:02.087 INFO:teuthology.orchestra.run.smithi151.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-09-19T04:52:02.107 INFO:teuthology.orchestra.run.smithi043.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 11 MB/s | 1.4 MB 00:00 2024-09-19T04:52:02.112 INFO:teuthology.orchestra.run.smithi151.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-09-19T04:52:02.137 INFO:teuthology.orchestra.run.smithi151.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 473 kB/s | 36 kB 00:00 2024-09-19T04:52:02.161 INFO:teuthology.orchestra.run.smithi096.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.5 MB/s | 5.1 MB 00:01 2024-09-19T04:52:02.163 INFO:teuthology.orchestra.run.smithi151.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 984 kB/s | 74 kB 00:00 2024-09-19T04:52:02.188 INFO:teuthology.orchestra.run.smithi151.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-19T04:52:02.213 INFO:teuthology.orchestra.run.smithi151.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-19T04:52:02.239 INFO:teuthology.orchestra.run.smithi151.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 395 kB/s | 20 kB 00:00 2024-09-19T04:52:02.249 INFO:teuthology.orchestra.run.smithi043.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 12 MB/s | 2.4 MB 00:00 2024-09-19T04:52:02.255 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-19T04:52:02.256 INFO:teuthology.orchestra.run.smithi043.stdout:Total 21 MB/s | 226 MB 00:10 2024-09-19T04:52:02.264 INFO:teuthology.orchestra.run.smithi151.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-19T04:52:02.286 INFO:teuthology.orchestra.run.smithi096.stdout:(84/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 327 kB/s | 250 kB 00:00 2024-09-19T04:52:02.290 INFO:teuthology.orchestra.run.smithi151.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-19T04:52:02.386 INFO:teuthology.orchestra.run.smithi096.stdout:(85/156): python3-oauthlib-3.1.1-5.el9.noarch.r 296 kB/s | 222 kB 00:00 2024-09-19T04:52:02.390 INFO:teuthology.orchestra.run.smithi151.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-09-19T04:52:02.549 INFO:teuthology.orchestra.run.smithi151.stdout:(149/156): librados2-19.1.1-361.g2f599a46.el9.x 12 MB/s | 3.4 MB 00:00 2024-09-19T04:52:02.599 INFO:teuthology.orchestra.run.smithi151.stdout:(150/156): NetworkManager-1.51.0-1.el9.x86_64.r 11 MB/s | 2.3 MB 00:00 2024-09-19T04:52:02.649 INFO:teuthology.orchestra.run.smithi151.stdout:(151/156): librbd1-19.1.1-361.g2f599a46.el9.x86 8.8 MB/s | 3.2 MB 00:00 2024-09-19T04:52:02.675 INFO:teuthology.orchestra.run.smithi151.stdout:(152/156): NetworkManager-team-1.51.0-1.el9.x86 543 kB/s | 41 kB 00:00 2024-09-19T04:52:02.734 INFO:teuthology.orchestra.run.smithi151.stdout:(153/156): openssl-3.2.2-6.el9.x86_64.rpm 23 MB/s | 1.4 MB 00:00 2024-09-19T04:52:02.759 INFO:teuthology.orchestra.run.smithi151.stdout:(154/156): NetworkManager-tui-1.51.0-1.el9.x86_ 2.2 MB/s | 249 kB 00:00 2024-09-19T04:52:02.795 INFO:teuthology.orchestra.run.smithi096.stdout:(86/156): python3-packaging-20.9-5.el9.noarch.r 122 kB/s | 77 kB 00:00 2024-09-19T04:52:02.801 INFO:teuthology.orchestra.run.smithi151.stdout:(155/156): NetworkManager-libnm-1.51.0-1.el9.x8 7.2 MB/s | 1.8 MB 00:00 2024-09-19T04:52:02.845 INFO:teuthology.orchestra.run.smithi096.stdout:(87/156): python3-prettytable-0.7.2-27.el9.noar 76 kB/s | 42 kB 00:00 2024-09-19T04:52:02.851 INFO:teuthology.orchestra.run.smithi151.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 20 MB/s | 2.4 MB 00:00 2024-09-19T04:52:02.855 INFO:teuthology.orchestra.run.smithi151.stdout:-------------------------------------------------------------------------------- 2024-09-19T04:52:02.855 INFO:teuthology.orchestra.run.smithi151.stdout:Total 21 MB/s | 226 MB 00:10 2024-09-19T04:52:03.162 INFO:teuthology.orchestra.run.smithi096.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 351 kB/s | 269 kB 00:00 2024-09-19T04:52:03.504 INFO:teuthology.orchestra.run.smithi096.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 224 kB/s | 159 kB 00:00 2024-09-19T04:52:03.637 INFO:teuthology.orchestra.run.smithi096.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 352 kB/s | 279 kB 00:00 2024-09-19T04:52:03.754 INFO:teuthology.orchestra.run.smithi096.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 86 kB/s | 51 kB 00:00 2024-09-19T04:52:04.032 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction check 2024-09-19T04:52:04.105 INFO:teuthology.orchestra.run.smithi096.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 89 kB/s | 54 kB 00:00 2024-09-19T04:52:04.184 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction check succeeded. 2024-09-19T04:52:04.184 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction test 2024-09-19T04:52:04.313 INFO:teuthology.orchestra.run.smithi096.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 75 kB/s | 42 kB 00:00 2024-09-19T04:52:04.379 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-19T04:52:04.581 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-19T04:52:04.581 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-19T04:52:04.672 INFO:teuthology.orchestra.run.smithi096.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 78 kB/s | 44 kB 00:00 2024-09-19T04:52:05.064 INFO:teuthology.orchestra.run.smithi096.stdout:(95/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-19T04:52:05.089 INFO:teuthology.orchestra.run.smithi096.stdout:(96/156): qatlib-24.02.0-1.el9.x86_64.rpm 285 kB/s | 221 kB 00:00 2024-09-19T04:52:05.331 INFO:teuthology.orchestra.run.smithi096.stdout:(97/156): qatlib-service-24.02.0-1.el9.x86_64.r 54 kB/s | 36 kB 00:00 2024-09-19T04:52:05.639 INFO:teuthology.orchestra.run.smithi096.stdout:(98/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 83 kB/s | 47 kB 00:00 2024-09-19T04:52:05.865 INFO:teuthology.orchestra.run.smithi096.stdout:(99/156): socat-1.7.4.1-6.el9.x86_64.rpm 392 kB/s | 304 kB 00:00 2024-09-19T04:52:05.957 INFO:teuthology.orchestra.run.smithi096.stdout:(100/156): uuid-1.6.2-55.el9.x86_64.rpm 92 kB/s | 58 kB 00:00 2024-09-19T04:52:05.981 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction test succeeded. 2024-09-19T04:52:05.981 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction 2024-09-19T04:52:06.323 INFO:teuthology.orchestra.run.smithi096.stdout:(101/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 140 kB/s | 96 kB 00:00 2024-09-19T04:52:06.407 INFO:teuthology.orchestra.run.smithi096.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 269 kB/s | 22 kB 00:00 2024-09-19T04:52:06.507 INFO:teuthology.orchestra.run.smithi096.stdout:(103/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 100 kB/s | 64 kB 00:00 2024-09-19T04:52:06.532 INFO:teuthology.orchestra.run.smithi096.stdout:(104/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 78 kB/s | 45 kB 00:00 2024-09-19T04:52:06.574 INFO:teuthology.orchestra.run.smithi096.stdout:(105/156): protobuf-compiler-3.14.0-13.el9.x86_ 5.0 MB/s | 863 kB 00:00 2024-09-19T04:52:06.633 INFO:teuthology.orchestra.run.smithi096.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 398 kB/s | 19 kB 00:00 2024-09-19T04:52:06.658 INFO:teuthology.orchestra.run.smithi096.stdout:(107/156): gperftools-libs-2.9.1-3.el9.x86_64.r 2.6 MB/s | 308 kB 00:00 2024-09-19T04:52:06.692 INFO:teuthology.orchestra.run.smithi096.stdout:(108/156): abseil-cpp-20211102.0-3.el9.x86_64.r 2.9 MB/s | 548 kB 00:00 2024-09-19T04:52:06.717 INFO:teuthology.orchestra.run.smithi096.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 444 kB/s | 26 kB 00:00 2024-09-19T04:52:06.743 INFO:teuthology.orchestra.run.smithi096.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 968 kB/s | 49 kB 00:00 2024-09-19T04:52:06.768 INFO:teuthology.orchestra.run.smithi096.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-19T04:52:06.793 INFO:teuthology.orchestra.run.smithi096.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-09-19T04:52:06.877 INFO:teuthology.orchestra.run.smithi096.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.6 MB/s | 839 kB 00:00 2024-09-19T04:52:06.902 INFO:teuthology.orchestra.run.smithi096.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 4.9 MB/s | 548 kB 00:00 2024-09-19T04:52:06.927 INFO:teuthology.orchestra.run.smithi096.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 861 kB/s | 43 kB 00:00 2024-09-19T04:52:06.953 INFO:teuthology.orchestra.run.smithi096.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 637 kB/s | 32 kB 00:00 2024-09-19T04:52:06.978 INFO:teuthology.orchestra.run.smithi096.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 282 kB/s | 14 kB 00:00 2024-09-19T04:52:07.020 INFO:teuthology.orchestra.run.smithi096.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-09-19T04:52:07.054 INFO:teuthology.orchestra.run.smithi096.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 4.6 MB/s | 358 kB 00:00 2024-09-19T04:52:07.064 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-19T04:52:07.064 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-19T04:52:07.087 INFO:teuthology.orchestra.run.smithi096.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-19T04:52:07.171 INFO:teuthology.orchestra.run.smithi096.stdout:(121/156): libarrow-9.0.0-11.el9.x86_64.rpm 8.2 MB/s | 4.4 MB 00:00 2024-09-19T04:52:07.196 INFO:teuthology.orchestra.run.smithi096.stdout:(122/156): python3-google-auth-2.34.0-1.el9.noa 1.5 MB/s | 220 kB 00:00 2024-09-19T04:52:07.222 INFO:teuthology.orchestra.run.smithi096.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-09-19T04:52:07.247 INFO:teuthology.orchestra.run.smithi096.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 214 kB/s | 11 kB 00:00 2024-09-19T04:52:07.272 INFO:teuthology.orchestra.run.smithi096.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 353 kB/s | 18 kB 00:00 2024-09-19T04:52:07.297 INFO:teuthology.orchestra.run.smithi096.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-09-19T04:52:07.314 INFO:teuthology.orchestra.run.smithi096.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 466 kB/s | 19 kB 00:00 2024-09-19T04:52:07.331 INFO:teuthology.orchestra.run.smithi096.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 587 kB/s | 20 kB 00:00 2024-09-19T04:52:07.357 INFO:teuthology.orchestra.run.smithi096.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-19T04:52:07.390 INFO:teuthology.orchestra.run.smithi096.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 14 MB/s | 1.0 MB 00:00 2024-09-19T04:52:07.416 INFO:teuthology.orchestra.run.smithi096.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-09-19T04:52:07.432 INFO:teuthology.orchestra.run.smithi096.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.3 MB/s | 58 kB 00:00 2024-09-19T04:52:07.449 INFO:teuthology.orchestra.run.smithi096.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 992 kB/s | 16 kB 00:00 2024-09-19T04:52:07.475 INFO:teuthology.orchestra.run.smithi096.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-19T04:52:07.500 INFO:teuthology.orchestra.run.smithi096.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-19T04:52:07.525 INFO:teuthology.orchestra.run.smithi096.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-09-19T04:52:07.550 INFO:teuthology.orchestra.run.smithi096.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-09-19T04:52:07.576 INFO:teuthology.orchestra.run.smithi096.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-19T04:52:07.601 INFO:teuthology.orchestra.run.smithi096.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-19T04:52:07.626 INFO:teuthology.orchestra.run.smithi096.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-09-19T04:52:07.652 INFO:teuthology.orchestra.run.smithi096.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-19T04:52:07.685 INFO:teuthology.orchestra.run.smithi096.stdout:(142/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.4 MB/s | 2.0 MB 00:00 2024-09-19T04:52:07.711 INFO:teuthology.orchestra.run.smithi096.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-09-19T04:52:07.736 INFO:teuthology.orchestra.run.smithi096.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-19T04:52:07.761 INFO:teuthology.orchestra.run.smithi096.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-19T04:52:07.787 INFO:teuthology.orchestra.run.smithi096.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-19T04:52:07.812 INFO:teuthology.orchestra.run.smithi096.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-19T04:52:07.863 INFO:teuthology.orchestra.run.smithi096.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-19T04:52:08.063 INFO:teuthology.orchestra.run.smithi096.stdout:(149/156): librados2-19.1.1-361.g2f599a46.el9.x 12 MB/s | 3.4 MB 00:00 2024-09-19T04:52:08.113 INFO:teuthology.orchestra.run.smithi096.stdout:(150/156): NetworkManager-1.51.0-1.el9.x86_64.r 9.3 MB/s | 2.3 MB 00:00 2024-09-19T04:52:08.188 INFO:teuthology.orchestra.run.smithi096.stdout:(151/156): librbd1-19.1.1-361.g2f599a46.el9.x86 8.4 MB/s | 3.2 MB 00:00 2024-09-19T04:52:08.213 INFO:teuthology.orchestra.run.smithi096.stdout:(152/156): NetworkManager-team-1.51.0-1.el9.x86 409 kB/s | 41 kB 00:00 2024-09-19T04:52:08.255 INFO:teuthology.orchestra.run.smithi096.stdout:(153/156): NetworkManager-libnm-1.51.0-1.el9.x8 9.4 MB/s | 1.8 MB 00:00 2024-09-19T04:52:08.281 INFO:teuthology.orchestra.run.smithi096.stdout:(154/156): NetworkManager-tui-1.51.0-1.el9.x86_ 2.6 MB/s | 249 kB 00:00 2024-09-19T04:52:08.314 INFO:teuthology.orchestra.run.smithi096.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 13 MB/s | 1.4 MB 00:00 2024-09-19T04:52:08.356 INFO:teuthology.orchestra.run.smithi096.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 24 MB/s | 2.4 MB 00:00 2024-09-19T04:52:08.362 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-09-19T04:52:08.362 INFO:teuthology.orchestra.run.smithi096.stdout:Total 11 MB/s | 226 MB 00:20 2024-09-19T04:52:09.592 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-09-19T04:52:09.741 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-09-19T04:52:09.741 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-09-19T04:52:11.532 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-09-19T04:52:11.532 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-09-19T04:52:12.209 INFO:teuthology.orchestra.run.smithi151.stdout: Preparing : 1/1 2024-09-19T04:52:12.322 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T04:52:12.453 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T04:52:12.629 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T04:52:12.675 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T04:52:12.699 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T04:52:12.756 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T04:52:13.091 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T04:52:13.218 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:52:13.340 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:52:13.369 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:52:13.460 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:52:13.490 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T04:52:13.640 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T04:52:13.688 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T04:52:13.725 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:52:13.827 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:52:13.860 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T04:52:13.922 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T04:52:13.961 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T04:52:13.984 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T04:52:14.005 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-19T04:52:14.146 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T04:52:14.180 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T04:52:14.197 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T04:52:14.280 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T04:52:14.329 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T04:52:14.332 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T04:52:14.363 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T04:52:14.419 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T04:52:14.453 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T04:52:14.554 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:52:14.630 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:52:14.804 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T04:52:14.861 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:52:14.922 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:52:14.991 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T04:52:15.078 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:52:15.091 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T04:52:15.106 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:52:15.127 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T04:52:15.163 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T04:52:15.221 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:52:15.236 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T04:52:15.265 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T04:52:15.295 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T04:52:15.336 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T04:52:15.380 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T04:52:15.412 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T04:52:15.440 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T04:52:15.488 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T04:52:15.510 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T04:52:15.571 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T04:52:15.576 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T04:52:15.602 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:52:15.602 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T04:52:15.646 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T04:52:15.722 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:52:15.722 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T04:52:15.755 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T04:52:15.757 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T04:52:15.801 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T04:52:15.829 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T04:52:15.829 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T04:52:15.862 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T04:52:15.899 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T04:52:15.902 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T04:52:15.953 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T04:52:16.027 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T04:52:16.132 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T04:52:16.160 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T04:52:16.214 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T04:52:16.313 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T04:52:16.530 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T04:52:16.651 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:52:16.740 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T04:52:16.745 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:52:16.819 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T04:52:16.858 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T04:52:16.909 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T04:52:16.955 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T04:52:17.005 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T04:52:17.029 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:52:17.179 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T04:52:17.184 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T04:52:17.240 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T04:52:17.294 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T04:52:17.300 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T04:52:17.361 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T04:52:17.367 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T04:52:17.399 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T04:52:17.406 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T04:52:17.436 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T04:52:17.483 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T04:52:17.491 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T04:52:17.528 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T04:52:17.558 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T04:52:17.570 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T04:52:17.597 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T04:52:17.612 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T04:52:17.652 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T04:52:17.690 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T04:52:17.724 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T04:52:17.747 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T04:52:17.790 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T04:52:17.815 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T04:52:17.831 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T04:52:17.851 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-09-19T04:52:17.877 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T04:52:17.910 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T04:52:17.936 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T04:52:17.951 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T04:52:17.958 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T04:52:18.045 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T04:52:18.048 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T04:52:18.089 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T04:52:18.134 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T04:52:18.175 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T04:52:18.179 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T04:52:18.216 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T04:52:18.249 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T04:52:18.271 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T04:52:18.274 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T04:52:18.289 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T04:52:18.323 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T04:52:18.344 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T04:52:18.353 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T04:52:18.369 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T04:52:18.416 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T04:52:18.422 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T04:52:18.478 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T04:52:18.508 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T04:52:18.537 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T04:52:18.570 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T04:52:18.580 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T04:52:18.618 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T04:52:18.651 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T04:52:18.658 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T04:52:18.688 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T04:52:18.723 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T04:52:18.752 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T04:52:18.756 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T04:52:18.798 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T04:52:18.833 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T04:52:18.853 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:52:18.920 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T04:52:18.963 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T04:52:18.974 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:52:18.995 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T04:52:19.004 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:52:19.095 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:52:19.098 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T04:52:19.125 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T04:52:19.155 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T04:52:19.270 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T04:52:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T04:52:19.325 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T04:52:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:52:19.429 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T04:52:19.436 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:52:19.460 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T04:52:19.470 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T04:52:19.506 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T04:52:19.522 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T04:52:19.528 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T04:52:19.548 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T04:52:19.561 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T04:52:19.615 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T04:52:19.746 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T04:52:19.873 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T04:52:19.879 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T04:52:19.929 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T04:52:19.962 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T04:52:20.006 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T04:52:20.061 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T04:52:20.092 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T04:52:20.109 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T04:52:20.110 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:52:20.154 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T04:52:20.178 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:52:20.180 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T04:52:20.242 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T04:52:20.259 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T04:52:20.291 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T04:52:20.329 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T04:52:20.364 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T04:52:20.409 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T04:52:20.414 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T04:52:20.477 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T04:52:20.482 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T04:52:20.517 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T04:52:20.522 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T04:52:20.538 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T04:52:20.563 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T04:52:20.608 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T04:52:20.613 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T04:52:20.626 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T04:52:20.658 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T04:52:20.692 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T04:52:20.756 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T04:52:20.794 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T04:52:20.801 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T04:52:20.834 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T04:52:20.845 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T04:52:20.877 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T04:52:20.888 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T04:52:20.918 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T04:52:20.975 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T04:52:21.019 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T04:52:21.048 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T04:52:21.070 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T04:52:21.105 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T04:52:21.113 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T04:52:21.136 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T04:52:21.142 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T04:52:21.167 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T04:52:21.171 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T04:52:21.238 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T04:52:21.252 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T04:52:21.274 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T04:52:21.288 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T04:52:21.317 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T04:52:21.346 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T04:52:21.372 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T04:52:21.415 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T04:52:21.461 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T04:52:21.462 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T04:52:21.511 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T04:52:21.534 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T04:52:21.565 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T04:52:21.611 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T04:52:21.664 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T04:52:21.682 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T04:52:21.721 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T04:52:21.721 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T04:52:21.758 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T04:52:21.793 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T04:52:21.826 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T04:52:21.861 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T04:52:21.893 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T04:52:21.997 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T04:52:22.053 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T04:52:22.088 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T04:52:22.214 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T04:52:22.232 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T04:52:22.278 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T04:52:22.308 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T04:52:22.347 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T04:52:22.381 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T04:52:22.419 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T04:52:22.461 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T04:52:22.591 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T04:52:22.610 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T04:52:22.655 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T04:52:22.709 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T04:52:22.775 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T04:52:22.815 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T04:52:22.851 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T04:52:22.898 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T04:52:22.934 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T04:52:22.977 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T04:52:22.982 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T04:52:23.010 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T04:52:23.024 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T04:52:23.044 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T04:52:23.167 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T04:52:23.210 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T04:52:23.213 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T04:52:23.251 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T04:52:23.270 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T04:52:23.328 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T04:52:23.352 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T04:52:23.352 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T04:52:23.385 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T04:52:23.398 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T04:52:23.425 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T04:52:23.449 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T04:52:23.456 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T04:52:23.476 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T04:52:23.489 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T04:52:23.512 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T04:52:23.520 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T04:52:23.540 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T04:52:23.574 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T04:52:23.633 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T04:52:23.667 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T04:52:23.681 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T04:52:23.717 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T04:52:23.722 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T04:52:23.745 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T04:52:23.779 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T04:52:23.818 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T04:52:23.849 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T04:52:23.879 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T04:52:23.914 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T04:52:23.947 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T04:52:23.971 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T04:52:23.989 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T04:52:24.003 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T04:52:24.023 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T04:52:24.100 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T04:52:24.101 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T04:52:24.135 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T04:52:24.145 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T04:52:24.169 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T04:52:24.177 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T04:52:24.240 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T04:52:24.271 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T04:52:24.295 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T04:52:24.344 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T04:52:24.960 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T04:52:25.056 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T04:52:25.100 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T04:52:25.102 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T04:52:25.134 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T04:52:25.231 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T04:52:25.280 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T04:52:25.334 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T04:52:25.412 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T04:52:25.445 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T04:52:25.619 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T04:52:25.651 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T04:52:25.728 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T04:52:26.363 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T04:52:26.482 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T04:52:26.585 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:52:26.619 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:52:26.771 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T04:52:26.865 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T04:52:26.902 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T04:52:26.935 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T04:52:27.162 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:52:27.199 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T04:52:27.287 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T04:52:27.292 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T04:52:27.292 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T04:52:27.292 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T04:52:27.292 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T04:52:27.326 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T04:52:27.337 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T04:52:27.459 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T04:52:27.493 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T04:52:27.543 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T04:52:27.684 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T04:52:27.773 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T04:52:27.801 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T04:52:27.833 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T04:52:28.097 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T04:52:28.140 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T04:52:28.164 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T04:52:28.214 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T04:52:28.235 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T04:52:28.294 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T04:52:28.312 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T04:52:28.343 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T04:52:28.379 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T04:52:28.859 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T04:52:28.883 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T04:52:28.938 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T04:52:28.968 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T04:52:30.931 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:52:30.981 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:52:31.230 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:52:31.273 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:52:31.653 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:52:31.702 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T04:52:31.815 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T04:52:31.840 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T04:52:31.840 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T04:52:31.840 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T04:52:31.840 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T04:52:31.878 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T04:52:31.985 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T04:52:32.062 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T04:52:32.063 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T04:52:32.063 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T04:52:32.063 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-19T04:52:32.095 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T04:52:39.219 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T04:52:39.219 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /sys 2024-09-19T04:52:39.219 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /proc 2024-09-19T04:52:39.219 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /mnt 2024-09-19T04:52:39.219 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /var/tmp 2024-09-19T04:52:39.219 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /home 2024-09-19T04:52:39.219 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /root 2024-09-19T04:52:39.220 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /tmp 2024-09-19T04:52:39.220 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T04:52:39.399 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T04:52:39.415 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T04:52:39.415 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:52:39.415 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T04:52:39.415 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T04:52:40.878 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T04:52:40.904 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T04:52:40.904 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:52:40.905 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T04:52:40.905 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T04:52:40.905 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T04:52:40.905 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T04:52:41.129 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T04:52:41.153 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T04:52:41.153 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:52:41.153 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T04:52:41.153 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T04:52:41.154 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T04:52:41.154 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T04:52:41.588 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T04:52:41.611 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T04:52:41.612 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:52:41.612 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T04:52:41.612 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T04:52:41.612 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T04:52:41.612 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T04:52:41.710 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T04:52:41.767 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T04:52:41.831 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T04:52:42.025 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T04:52:42.082 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T04:52:42.160 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T04:52:42.257 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T04:52:42.397 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T04:52:42.450 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T04:52:42.482 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T04:52:42.510 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T04:52:42.940 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T04:52:42.969 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T04:52:43.000 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T04:52:43.063 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T04:52:43.092 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T04:52:43.757 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T04:52:43.757 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /sys 2024-09-19T04:52:43.757 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /proc 2024-09-19T04:52:43.757 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /mnt 2024-09-19T04:52:43.757 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /var/tmp 2024-09-19T04:52:43.757 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /home 2024-09-19T04:52:43.758 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /root 2024-09-19T04:52:43.758 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /tmp 2024-09-19T04:52:43.758 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-19T04:52:43.933 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T04:52:43.949 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T04:52:43.949 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:52:43.949 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T04:52:43.949 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-19T04:52:44.139 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T04:52:44.170 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T04:52:45.126 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T04:52:45.153 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T04:52:45.254 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T04:52:45.403 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T04:52:45.406 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T04:52:45.430 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T04:52:45.430 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:52:45.430 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T04:52:45.430 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T04:52:45.430 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T04:52:45.430 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-19T04:52:45.443 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T04:52:45.466 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T04:52:45.466 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:52:45.466 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T04:52:45.466 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T04:52:45.467 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T04:52:45.467 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T04:52:45.515 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T04:52:45.528 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T04:52:45.538 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T04:52:45.538 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /sys 2024-09-19T04:52:45.538 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /proc 2024-09-19T04:52:45.538 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /mnt 2024-09-19T04:52:45.538 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /var/tmp 2024-09-19T04:52:45.538 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /home 2024-09-19T04:52:45.538 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /root 2024-09-19T04:52:45.538 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /tmp 2024-09-19T04:52:45.538 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T04:52:45.625 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T04:52:45.653 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T04:52:45.672 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T04:52:45.677 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T04:52:45.677 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:52:45.677 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T04:52:45.677 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T04:52:45.677 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T04:52:45.677 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-19T04:52:45.747 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T04:52:45.784 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T04:52:45.784 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:52:45.784 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T04:52:45.784 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T04:52:46.113 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T04:52:46.136 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T04:52:46.136 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:52:46.136 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T04:52:46.137 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T04:52:46.137 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T04:52:46.137 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-19T04:52:46.243 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T04:52:46.299 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T04:52:46.354 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T04:52:46.546 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T04:52:46.593 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T04:52:46.596 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T04:52:46.628 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T04:52:46.651 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T04:52:46.652 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:52:46.652 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T04:52:46.652 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T04:52:46.652 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T04:52:46.652 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T04:52:46.665 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T04:52:46.703 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T04:52:46.726 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T04:52:46.726 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:52:46.726 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T04:52:46.726 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T04:52:46.753 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T04:52:46.770 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T04:52:46.793 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T04:52:46.793 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:52:46.794 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T04:52:46.794 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T04:52:46.794 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T04:52:46.794 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T04:52:46.884 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T04:52:46.922 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T04:52:46.962 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T04:52:46.990 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T04:52:47.105 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T04:52:47.128 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T04:52:47.128 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:52:47.128 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T04:52:47.129 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T04:52:47.129 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T04:52:47.129 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T04:52:47.385 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T04:52:47.414 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T04:52:47.434 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T04:52:47.445 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T04:52:47.493 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T04:52:47.493 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:52:47.493 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T04:52:47.493 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T04:52:47.493 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T04:52:47.493 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T04:52:47.508 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T04:52:47.537 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T04:52:47.751 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T04:52:47.803 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T04:52:47.803 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:52:47.803 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T04:52:47.803 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T04:52:47.803 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T04:52:47.803 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T04:52:48.302 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T04:52:48.353 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T04:52:48.353 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:52:48.353 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T04:52:48.353 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T04:52:48.353 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T04:52:48.353 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T04:52:48.495 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T04:52:48.556 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T04:52:48.572 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T04:52:48.606 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T04:52:48.629 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T04:52:48.876 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T04:52:48.940 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T04:52:49.029 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T04:52:49.140 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T04:52:49.326 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T04:52:49.556 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T04:52:49.559 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T04:52:49.595 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T04:52:49.598 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T04:52:49.629 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T04:52:49.700 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T04:52:49.859 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T04:52:49.883 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T04:52:49.907 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T04:52:49.907 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:52:49.907 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T04:52:49.907 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T04:52:49.907 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T04:52:49.907 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-19T04:52:49.951 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T04:52:49.962 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T04:52:50.059 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T04:52:50.106 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T04:52:50.145 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T04:52:50.178 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T04:52:50.220 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T04:52:50.298 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T04:52:50.325 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T04:52:51.029 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T04:52:51.070 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T04:52:51.094 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T04:52:51.094 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:52:51.094 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T04:52:51.094 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T04:52:51.094 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T04:52:51.094 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-19T04:52:51.137 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T04:52:51.160 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T04:52:51.160 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:52:51.161 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T04:52:51.161 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-19T04:52:51.205 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T04:52:51.228 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T04:52:51.228 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:52:51.228 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T04:52:51.229 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T04:52:51.229 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T04:52:51.229 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-19T04:52:51.530 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T04:52:51.554 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T04:52:51.554 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:52:51.554 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T04:52:51.554 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T04:52:51.554 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T04:52:51.555 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-19T04:52:51.633 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T04:52:51.652 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T04:52:51.669 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T04:52:51.693 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T04:52:51.734 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T04:52:51.766 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T04:52:51.794 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T04:52:51.982 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T04:52:52.025 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T04:52:52.056 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T04:52:52.086 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T04:52:52.132 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T04:52:52.721 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:52:52.757 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:52:52.777 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:52:52.821 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T04:52:52.844 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T04:52:52.988 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T04:52:53.176 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T04:52:53.204 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T04:52:53.255 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T04:52:53.256 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:52:53.256 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T04:52:53.256 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T04:52:53.256 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T04:52:53.256 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T04:52:53.313 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T04:52:53.339 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T04:52:53.452 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T04:52:53.525 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T04:52:53.630 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T04:52:53.725 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T04:52:53.774 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T04:52:53.864 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T04:52:53.903 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T04:52:53.928 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T04:52:53.955 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T04:52:53.955 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T04:52:54.046 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T04:52:54.047 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T04:52:54.139 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T04:52:54.164 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T04:52:54.169 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:52:54.169 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:52:54.353 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:52:54.353 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T04:52:54.417 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T04:52:54.488 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:52:54.591 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T04:52:54.623 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T04:52:54.676 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T04:52:54.676 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:52:54.676 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T04:52:54.676 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T04:52:54.676 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T04:52:54.677 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T04:52:54.725 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T04:52:54.775 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T04:52:54.775 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:52:54.775 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T04:52:54.775 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T04:52:54.825 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T04:52:54.877 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T04:52:54.877 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:52:54.877 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T04:52:54.877 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T04:52:54.877 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T04:52:54.877 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T04:52:55.234 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T04:52:55.287 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T04:52:55.287 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T04:52:55.287 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T04:52:55.287 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T04:52:55.287 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T04:52:55.287 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T04:52:56.059 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T04:52:56.100 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T04:52:56.141 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T04:52:56.165 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T04:52:56.184 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T04:52:56.361 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T04:52:56.406 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T04:52:56.436 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T04:52:56.466 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T04:52:56.512 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T04:52:57.081 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:52:57.119 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:52:57.139 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:52:57.958 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T04:52:58.049 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T04:52:58.089 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T04:52:58.170 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T04:52:58.202 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T04:52:58.227 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T04:52:58.253 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T04:52:58.254 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T04:52:58.346 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T04:52:58.346 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T04:52:58.439 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T04:52:58.464 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T04:52:58.469 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:52:58.469 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:52:58.649 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:52:58.649 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T04:52:58.724 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T04:52:58.787 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:53:00.451 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:53:00.451 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T04:53:00.451 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T04:53:00.451 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T04:53:00.451 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T04:53:00.451 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T04:53:00.451 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T04:53:00.451 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T04:53:00.451 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:53:00.451 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:53:00.452 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T04:53:00.452 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T04:53:00.452 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T04:53:00.452 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:53:00.452 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T04:53:00.452 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T04:53:00.452 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T04:53:00.452 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T04:53:00.452 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T04:53:00.452 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T04:53:00.452 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T04:53:00.452 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T04:53:00.453 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T04:53:00.453 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T04:53:00.453 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T04:53:00.453 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T04:53:00.453 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T04:53:00.453 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T04:53:00.453 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T04:53:00.453 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T04:53:00.454 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T04:53:00.466 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T04:53:00.466 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T04:53:00.466 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T04:53:00.466 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T04:53:00.466 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T04:53:00.466 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T04:53:00.466 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T04:53:00.466 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T04:53:00.466 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T04:53:00.466 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T04:53:00.466 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T04:53:00.467 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T04:53:00.467 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T04:53:00.467 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T04:53:00.467 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T04:53:00.467 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T04:53:00.467 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T04:53:00.467 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T04:53:00.467 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T04:53:00.467 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T04:53:00.467 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T04:53:00.467 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T04:53:00.467 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T04:53:00.467 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T04:53:00.468 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T04:53:00.468 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T04:53:00.468 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T04:53:00.468 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T04:53:00.468 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T04:53:00.468 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T04:53:00.468 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T04:53:00.468 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T04:53:00.468 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T04:53:00.468 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T04:53:00.468 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T04:53:00.468 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T04:53:00.469 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T04:53:00.469 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T04:53:00.469 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T04:53:00.469 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T04:53:00.469 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T04:53:00.469 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T04:53:00.469 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T04:53:00.469 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T04:53:00.469 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T04:53:00.469 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T04:53:00.469 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T04:53:00.469 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T04:53:00.469 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T04:53:00.470 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T04:53:00.470 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T04:53:00.470 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T04:53:00.470 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T04:53:00.470 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T04:53:00.470 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T04:53:00.470 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T04:53:00.470 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T04:53:00.470 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T04:53:00.470 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T04:53:00.471 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T04:53:00.471 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T04:53:00.471 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T04:53:00.471 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T04:53:00.471 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T04:53:00.471 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T04:53:00.471 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T04:53:00.471 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T04:53:00.472 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T04:53:00.472 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T04:53:00.472 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T04:53:00.472 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T04:53:00.472 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T04:53:00.472 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T04:53:00.472 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T04:53:00.472 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T04:53:00.472 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T04:53:00.472 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T04:53:00.472 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T04:53:00.472 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T04:53:00.472 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T04:53:00.473 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T04:53:00.473 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T04:53:00.473 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T04:53:00.473 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T04:53:00.473 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T04:53:00.473 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T04:53:00.473 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T04:53:00.473 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T04:53:00.473 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T04:53:00.473 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T04:53:00.473 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T04:53:00.473 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T04:53:00.474 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T04:53:00.474 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T04:53:00.474 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T04:53:00.474 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T04:53:00.474 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T04:53:00.474 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T04:53:00.474 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T04:53:00.474 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T04:53:00.474 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T04:53:00.474 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T04:53:00.474 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T04:53:00.475 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T04:53:00.475 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T04:53:00.475 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T04:53:00.475 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T04:53:00.475 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T04:53:00.475 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T04:53:00.475 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T04:53:00.475 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T04:53:00.475 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T04:53:00.475 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T04:53:00.475 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T04:53:00.475 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T04:53:00.476 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T04:53:00.476 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T04:53:00.476 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T04:53:00.476 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T04:53:00.476 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T04:53:00.476 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T04:53:00.476 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T04:53:00.476 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T04:53:00.476 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T04:53:00.476 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T04:53:00.476 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T04:53:00.476 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T04:53:00.476 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T04:53:00.477 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T04:53:00.477 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T04:53:00.477 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T04:53:00.477 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T04:53:00.477 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T04:53:00.548 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T04:53:00.589 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T04:53:00.633 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T04:53:00.670 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T04:53:00.705 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T04:53:00.958 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T04:53:01.018 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T04:53:01.048 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T04:53:01.083 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T04:53:01.138 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T04:53:01.776 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:53:01.862 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:53:01.905 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T04:53:03.156 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:53:03.156 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T04:53:03.156 INFO:teuthology.orchestra.run.smithi151.stdout:Upgraded: 2024-09-19T04:53:03.156 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T04:53:03.156 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T04:53:03.156 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T04:53:03.156 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T04:53:03.156 INFO:teuthology.orchestra.run.smithi151.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:03.156 INFO:teuthology.orchestra.run.smithi151.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:03.156 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T04:53:03.157 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T04:53:03.157 INFO:teuthology.orchestra.run.smithi151.stdout:Installed: 2024-09-19T04:53:03.157 INFO:teuthology.orchestra.run.smithi151.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T04:53:03.157 INFO:teuthology.orchestra.run.smithi151.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T04:53:03.157 INFO:teuthology.orchestra.run.smithi151.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T04:53:03.157 INFO:teuthology.orchestra.run.smithi151.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T04:53:03.157 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:03.157 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:03.157 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:03.157 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:03.157 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:53:03.157 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:03.157 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:03.157 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:03.158 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:53:03.158 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:53:03.158 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:53:03.158 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:53:03.158 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:53:03.158 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:03.158 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:03.158 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:53:03.158 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:03.158 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:03.158 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:03.158 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:53:03.158 INFO:teuthology.orchestra.run.smithi151.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:53:03.159 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:03.159 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:03.159 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:53:03.159 INFO:teuthology.orchestra.run.smithi151.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T04:53:03.159 INFO:teuthology.orchestra.run.smithi151.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T04:53:03.159 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T04:53:03.159 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T04:53:03.159 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T04:53:03.159 INFO:teuthology.orchestra.run.smithi151.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T04:53:03.159 INFO:teuthology.orchestra.run.smithi151.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T04:53:03.159 INFO:teuthology.orchestra.run.smithi151.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T04:53:03.159 INFO:teuthology.orchestra.run.smithi151.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T04:53:03.159 INFO:teuthology.orchestra.run.smithi151.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T04:53:03.160 INFO:teuthology.orchestra.run.smithi151.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T04:53:03.160 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:03.160 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:03.160 INFO:teuthology.orchestra.run.smithi151.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:03.160 INFO:teuthology.orchestra.run.smithi151.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T04:53:03.160 INFO:teuthology.orchestra.run.smithi151.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T04:53:03.160 INFO:teuthology.orchestra.run.smithi151.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T04:53:03.160 INFO:teuthology.orchestra.run.smithi151.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T04:53:03.160 INFO:teuthology.orchestra.run.smithi151.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T04:53:03.160 INFO:teuthology.orchestra.run.smithi151.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:03.160 INFO:teuthology.orchestra.run.smithi151.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:03.160 INFO:teuthology.orchestra.run.smithi151.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T04:53:03.160 INFO:teuthology.orchestra.run.smithi151.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:03.160 INFO:teuthology.orchestra.run.smithi151.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T04:53:03.161 INFO:teuthology.orchestra.run.smithi151.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T04:53:03.161 INFO:teuthology.orchestra.run.smithi151.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T04:53:03.161 INFO:teuthology.orchestra.run.smithi151.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T04:53:03.161 INFO:teuthology.orchestra.run.smithi151.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T04:53:03.161 INFO:teuthology.orchestra.run.smithi151.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T04:53:03.161 INFO:teuthology.orchestra.run.smithi151.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T04:53:03.161 INFO:teuthology.orchestra.run.smithi151.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T04:53:03.161 INFO:teuthology.orchestra.run.smithi151.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T04:53:03.161 INFO:teuthology.orchestra.run.smithi151.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T04:53:03.161 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T04:53:03.161 INFO:teuthology.orchestra.run.smithi151.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T04:53:03.161 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T04:53:03.161 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T04:53:03.162 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T04:53:03.162 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T04:53:03.162 INFO:teuthology.orchestra.run.smithi151.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T04:53:03.162 INFO:teuthology.orchestra.run.smithi151.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T04:53:03.162 INFO:teuthology.orchestra.run.smithi151.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T04:53:03.162 INFO:teuthology.orchestra.run.smithi151.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T04:53:03.162 INFO:teuthology.orchestra.run.smithi151.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T04:53:03.162 INFO:teuthology.orchestra.run.smithi151.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T04:53:03.162 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T04:53:03.162 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:03.162 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:03.162 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:03.162 INFO:teuthology.orchestra.run.smithi151.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T04:53:03.162 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T04:53:03.163 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T04:53:03.163 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T04:53:03.163 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T04:53:03.163 INFO:teuthology.orchestra.run.smithi151.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T04:53:03.163 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T04:53:03.163 INFO:teuthology.orchestra.run.smithi151.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T04:53:03.163 INFO:teuthology.orchestra.run.smithi151.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T04:53:03.163 INFO:teuthology.orchestra.run.smithi151.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T04:53:03.163 INFO:teuthology.orchestra.run.smithi151.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T04:53:03.163 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T04:53:03.163 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T04:53:03.163 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T04:53:03.163 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T04:53:03.164 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T04:53:03.164 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T04:53:03.164 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T04:53:03.164 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T04:53:03.164 INFO:teuthology.orchestra.run.smithi151.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T04:53:03.164 INFO:teuthology.orchestra.run.smithi151.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T04:53:03.164 INFO:teuthology.orchestra.run.smithi151.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T04:53:03.164 INFO:teuthology.orchestra.run.smithi151.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T04:53:03.164 INFO:teuthology.orchestra.run.smithi151.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T04:53:03.164 INFO:teuthology.orchestra.run.smithi151.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T04:53:03.165 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T04:53:03.165 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T04:53:03.165 INFO:teuthology.orchestra.run.smithi151.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T04:53:03.165 INFO:teuthology.orchestra.run.smithi151.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T04:53:03.165 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T04:53:03.165 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T04:53:03.165 INFO:teuthology.orchestra.run.smithi151.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T04:53:03.165 INFO:teuthology.orchestra.run.smithi151.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T04:53:03.165 INFO:teuthology.orchestra.run.smithi151.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T04:53:03.165 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T04:53:03.165 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T04:53:03.165 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T04:53:03.165 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T04:53:03.165 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T04:53:03.165 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T04:53:03.166 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:03.166 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:03.166 INFO:teuthology.orchestra.run.smithi151.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T04:53:03.166 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T04:53:03.166 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T04:53:03.166 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:03.166 INFO:teuthology.orchestra.run.smithi151.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T04:53:03.166 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T04:53:03.166 INFO:teuthology.orchestra.run.smithi151.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T04:53:03.166 INFO:teuthology.orchestra.run.smithi151.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T04:53:03.166 INFO:teuthology.orchestra.run.smithi151.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T04:53:03.166 INFO:teuthology.orchestra.run.smithi151.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T04:53:03.166 INFO:teuthology.orchestra.run.smithi151.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T04:53:03.166 INFO:teuthology.orchestra.run.smithi151.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T04:53:03.167 INFO:teuthology.orchestra.run.smithi151.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T04:53:03.167 INFO:teuthology.orchestra.run.smithi151.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T04:53:03.167 INFO:teuthology.orchestra.run.smithi151.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T04:53:03.167 INFO:teuthology.orchestra.run.smithi151.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T04:53:03.167 INFO:teuthology.orchestra.run.smithi151.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T04:53:03.167 INFO:teuthology.orchestra.run.smithi151.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T04:53:03.167 INFO:teuthology.orchestra.run.smithi151.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T04:53:03.167 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:03.167 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:03.167 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:03.167 INFO:teuthology.orchestra.run.smithi151.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T04:53:03.167 INFO:teuthology.orchestra.run.smithi151.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T04:53:03.167 INFO:teuthology.orchestra.run.smithi151.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T04:53:03.168 INFO:teuthology.orchestra.run.smithi151.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T04:53:03.168 INFO:teuthology.orchestra.run.smithi151.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T04:53:03.168 INFO:teuthology.orchestra.run.smithi151.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T04:53:03.168 INFO:teuthology.orchestra.run.smithi151.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T04:53:03.168 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T04:53:03.168 INFO:teuthology.orchestra.run.smithi151.stdout:Complete! 2024-09-19T04:53:03.292 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T04:53:03.403 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T04:53:03.474 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T04:53:03.589 DEBUG:teuthology.parallel:result is None 2024-09-19T04:53:03.594 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T04:53:03.630 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T04:53:03.656 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T04:53:03.692 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T04:53:03.692 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T04:53:03.791 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T04:53:03.791 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T04:53:03.883 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T04:53:03.907 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T04:53:03.924 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:53:03.924 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:53:04.168 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T04:53:04.168 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T04:53:04.278 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T04:53:04.344 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:53:04.344 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T04:53:04.344 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T04:53:04.344 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T04:53:04.344 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T04:53:04.345 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T04:53:04.345 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T04:53:04.345 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T04:53:04.345 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:53:04.345 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:53:04.345 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T04:53:04.345 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T04:53:04.345 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T04:53:04.345 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:53:04.345 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T04:53:04.345 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T04:53:04.345 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T04:53:04.346 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T04:53:04.346 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T04:53:04.346 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T04:53:04.346 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T04:53:04.346 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T04:53:04.346 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T04:53:04.346 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T04:53:04.346 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T04:53:04.346 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T04:53:04.346 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T04:53:04.346 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T04:53:04.359 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T04:53:04.359 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T04:53:04.359 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T04:53:04.359 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T04:53:04.359 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T04:53:04.360 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T04:53:04.360 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T04:53:04.360 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T04:53:04.360 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T04:53:04.360 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T04:53:04.360 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T04:53:04.360 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T04:53:04.360 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T04:53:04.360 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T04:53:04.360 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T04:53:04.361 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T04:53:04.361 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T04:53:04.361 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T04:53:04.361 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T04:53:04.361 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T04:53:04.361 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T04:53:04.361 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T04:53:04.361 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T04:53:04.361 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T04:53:04.361 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T04:53:04.361 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T04:53:04.361 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T04:53:04.361 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T04:53:04.362 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T04:53:04.362 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T04:53:04.362 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T04:53:04.362 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T04:53:04.362 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T04:53:04.362 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T04:53:04.362 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T04:53:04.362 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T04:53:04.362 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T04:53:04.362 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T04:53:04.362 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T04:53:04.363 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T04:53:04.363 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T04:53:04.363 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T04:53:04.363 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T04:53:04.363 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T04:53:04.363 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T04:53:04.363 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T04:53:04.363 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T04:53:04.363 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T04:53:04.363 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T04:53:04.363 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T04:53:04.364 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T04:53:04.364 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T04:53:04.364 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T04:53:04.364 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T04:53:04.364 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T04:53:04.364 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T04:53:04.364 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T04:53:04.364 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T04:53:04.364 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T04:53:04.364 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T04:53:04.364 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T04:53:04.364 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T04:53:04.365 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T04:53:04.365 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T04:53:04.365 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T04:53:04.365 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T04:53:04.365 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T04:53:04.365 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T04:53:04.365 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T04:53:04.365 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T04:53:04.365 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T04:53:04.365 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T04:53:04.365 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T04:53:04.365 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T04:53:04.365 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T04:53:04.366 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T04:53:04.366 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T04:53:04.366 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T04:53:04.366 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T04:53:04.366 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T04:53:04.366 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T04:53:04.366 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T04:53:04.366 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T04:53:04.366 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T04:53:04.366 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T04:53:04.366 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T04:53:04.367 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T04:53:04.367 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T04:53:04.367 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T04:53:04.367 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T04:53:04.367 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T04:53:04.367 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T04:53:04.367 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T04:53:04.367 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T04:53:04.367 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T04:53:04.367 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T04:53:04.367 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T04:53:04.367 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T04:53:04.367 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T04:53:04.368 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T04:53:04.368 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T04:53:04.368 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T04:53:04.368 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T04:53:04.368 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T04:53:04.368 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T04:53:04.368 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T04:53:04.368 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T04:53:04.368 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T04:53:04.368 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T04:53:04.368 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T04:53:04.369 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T04:53:04.369 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T04:53:04.369 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T04:53:04.369 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T04:53:04.369 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T04:53:04.369 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T04:53:04.369 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T04:53:04.369 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T04:53:04.369 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T04:53:04.369 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T04:53:04.369 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T04:53:04.369 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T04:53:04.369 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T04:53:04.369 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T04:53:04.370 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T04:53:04.370 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T04:53:04.370 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T04:53:04.370 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T04:53:04.370 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T04:53:04.370 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T04:53:04.370 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T04:53:04.370 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T04:53:04.370 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T04:53:04.370 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T04:53:04.370 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T04:53:04.370 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T04:53:04.371 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:53:04.557 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:53:04.558 INFO:teuthology.orchestra.run.smithi043.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-19T04:53:04.558 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T04:53:07.029 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:53:07.029 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-19T04:53:07.029 INFO:teuthology.orchestra.run.smithi096.stdout:Upgraded: 2024-09-19T04:53:07.029 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T04:53:07.029 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T04:53:07.029 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T04:53:07.029 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T04:53:07.029 INFO:teuthology.orchestra.run.smithi096.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:07.029 INFO:teuthology.orchestra.run.smithi096.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:07.029 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T04:53:07.030 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T04:53:07.030 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-09-19T04:53:07.030 INFO:teuthology.orchestra.run.smithi096.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T04:53:07.030 INFO:teuthology.orchestra.run.smithi096.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T04:53:07.030 INFO:teuthology.orchestra.run.smithi096.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T04:53:07.030 INFO:teuthology.orchestra.run.smithi096.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T04:53:07.030 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:07.030 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:07.030 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:07.030 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:07.030 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:53:07.030 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:07.030 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:07.031 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:07.031 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:53:07.031 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:53:07.031 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:53:07.031 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:53:07.031 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:53:07.031 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:07.031 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:07.031 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:53:07.031 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:07.032 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:07.032 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:07.032 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:53:07.032 INFO:teuthology.orchestra.run.smithi096.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:53:07.032 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:07.032 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:07.032 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:53:07.032 INFO:teuthology.orchestra.run.smithi096.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T04:53:07.032 INFO:teuthology.orchestra.run.smithi096.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T04:53:07.032 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T04:53:07.032 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T04:53:07.032 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T04:53:07.032 INFO:teuthology.orchestra.run.smithi096.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T04:53:07.033 INFO:teuthology.orchestra.run.smithi096.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T04:53:07.033 INFO:teuthology.orchestra.run.smithi096.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T04:53:07.033 INFO:teuthology.orchestra.run.smithi096.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T04:53:07.033 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T04:53:07.033 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T04:53:07.033 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:07.033 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:07.033 INFO:teuthology.orchestra.run.smithi096.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:07.033 INFO:teuthology.orchestra.run.smithi096.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T04:53:07.034 INFO:teuthology.orchestra.run.smithi096.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T04:53:07.034 INFO:teuthology.orchestra.run.smithi096.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T04:53:07.034 INFO:teuthology.orchestra.run.smithi096.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T04:53:07.034 INFO:teuthology.orchestra.run.smithi096.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T04:53:07.034 INFO:teuthology.orchestra.run.smithi096.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:07.034 INFO:teuthology.orchestra.run.smithi096.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:07.034 INFO:teuthology.orchestra.run.smithi096.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T04:53:07.034 INFO:teuthology.orchestra.run.smithi096.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:07.034 INFO:teuthology.orchestra.run.smithi096.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T04:53:07.034 INFO:teuthology.orchestra.run.smithi096.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T04:53:07.034 INFO:teuthology.orchestra.run.smithi096.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T04:53:07.034 INFO:teuthology.orchestra.run.smithi096.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T04:53:07.034 INFO:teuthology.orchestra.run.smithi096.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T04:53:07.035 INFO:teuthology.orchestra.run.smithi096.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T04:53:07.035 INFO:teuthology.orchestra.run.smithi096.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T04:53:07.035 INFO:teuthology.orchestra.run.smithi096.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T04:53:07.035 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T04:53:07.035 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T04:53:07.035 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T04:53:07.035 INFO:teuthology.orchestra.run.smithi096.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T04:53:07.035 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T04:53:07.035 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T04:53:07.035 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T04:53:07.035 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T04:53:07.035 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T04:53:07.035 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T04:53:07.036 INFO:teuthology.orchestra.run.smithi096.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T04:53:07.036 INFO:teuthology.orchestra.run.smithi096.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T04:53:07.036 INFO:teuthology.orchestra.run.smithi096.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T04:53:07.036 INFO:teuthology.orchestra.run.smithi096.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T04:53:07.036 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T04:53:07.036 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:07.036 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:07.036 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:07.036 INFO:teuthology.orchestra.run.smithi096.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T04:53:07.036 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T04:53:07.036 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T04:53:07.036 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T04:53:07.036 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T04:53:07.036 INFO:teuthology.orchestra.run.smithi096.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T04:53:07.037 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T04:53:07.037 INFO:teuthology.orchestra.run.smithi096.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T04:53:07.037 INFO:teuthology.orchestra.run.smithi096.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T04:53:07.037 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T04:53:07.037 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T04:53:07.037 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T04:53:07.037 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T04:53:07.037 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T04:53:07.037 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T04:53:07.037 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T04:53:07.037 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T04:53:07.037 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T04:53:07.037 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T04:53:07.038 INFO:teuthology.orchestra.run.smithi096.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T04:53:07.038 INFO:teuthology.orchestra.run.smithi096.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T04:53:07.038 INFO:teuthology.orchestra.run.smithi096.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T04:53:07.038 INFO:teuthology.orchestra.run.smithi096.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T04:53:07.038 INFO:teuthology.orchestra.run.smithi096.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T04:53:07.038 INFO:teuthology.orchestra.run.smithi096.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T04:53:07.038 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T04:53:07.038 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T04:53:07.038 INFO:teuthology.orchestra.run.smithi096.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T04:53:07.038 INFO:teuthology.orchestra.run.smithi096.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T04:53:07.038 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T04:53:07.038 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T04:53:07.038 INFO:teuthology.orchestra.run.smithi096.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T04:53:07.038 INFO:teuthology.orchestra.run.smithi096.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T04:53:07.039 INFO:teuthology.orchestra.run.smithi096.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T04:53:07.039 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T04:53:07.039 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T04:53:07.039 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T04:53:07.039 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T04:53:07.039 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T04:53:07.039 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T04:53:07.039 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:07.039 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:07.039 INFO:teuthology.orchestra.run.smithi096.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T04:53:07.039 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T04:53:07.039 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T04:53:07.039 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:07.039 INFO:teuthology.orchestra.run.smithi096.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T04:53:07.040 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T04:53:07.040 INFO:teuthology.orchestra.run.smithi096.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T04:53:07.040 INFO:teuthology.orchestra.run.smithi096.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T04:53:07.040 INFO:teuthology.orchestra.run.smithi096.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T04:53:07.040 INFO:teuthology.orchestra.run.smithi096.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T04:53:07.040 INFO:teuthology.orchestra.run.smithi096.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T04:53:07.040 INFO:teuthology.orchestra.run.smithi096.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T04:53:07.040 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T04:53:07.040 INFO:teuthology.orchestra.run.smithi096.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T04:53:07.040 INFO:teuthology.orchestra.run.smithi096.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T04:53:07.040 INFO:teuthology.orchestra.run.smithi096.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T04:53:07.040 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T04:53:07.040 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T04:53:07.040 INFO:teuthology.orchestra.run.smithi096.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T04:53:07.041 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:07.041 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:07.041 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:07.041 INFO:teuthology.orchestra.run.smithi096.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T04:53:07.041 INFO:teuthology.orchestra.run.smithi096.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T04:53:07.041 INFO:teuthology.orchestra.run.smithi096.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T04:53:07.041 INFO:teuthology.orchestra.run.smithi096.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T04:53:07.041 INFO:teuthology.orchestra.run.smithi096.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T04:53:07.041 INFO:teuthology.orchestra.run.smithi096.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T04:53:07.044 INFO:teuthology.orchestra.run.smithi096.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T04:53:07.044 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-19T04:53:07.044 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-09-19T04:53:07.446 DEBUG:teuthology.parallel:result is None 2024-09-19T04:53:10.178 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T04:53:10.179 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T04:53:10.179 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T04:53:10.179 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T04:53:10.179 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T04:53:10.179 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T04:53:10.179 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T04:53:10.179 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T04:53:10.179 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T04:53:10.179 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T04:53:10.179 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T04:53:10.180 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T04:53:10.180 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T04:53:10.180 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T04:53:10.180 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T04:53:10.180 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T04:53:10.180 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T04:53:10.180 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T04:53:10.180 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T04:53:10.180 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T04:53:10.180 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T04:53:10.180 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T04:53:10.180 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T04:53:10.181 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T04:53:10.181 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T04:53:10.181 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T04:53:10.181 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T04:53:10.194 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T04:53:10.194 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T04:53:10.194 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T04:53:10.194 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T04:53:10.194 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T04:53:10.194 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T04:53:10.195 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T04:53:10.195 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T04:53:10.195 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T04:53:10.195 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T04:53:10.195 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T04:53:10.195 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T04:53:10.195 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T04:53:10.195 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T04:53:10.195 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T04:53:10.195 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T04:53:10.196 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T04:53:10.196 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T04:53:10.196 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T04:53:10.196 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T04:53:10.196 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T04:53:10.196 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T04:53:10.196 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T04:53:10.196 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T04:53:10.196 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T04:53:10.196 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T04:53:10.196 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T04:53:10.196 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T04:53:10.197 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T04:53:10.197 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T04:53:10.197 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T04:53:10.197 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T04:53:10.197 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T04:53:10.197 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T04:53:10.197 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T04:53:10.197 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T04:53:10.197 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T04:53:10.197 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T04:53:10.197 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T04:53:10.198 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T04:53:10.198 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T04:53:10.198 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T04:53:10.198 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T04:53:10.198 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T04:53:10.198 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T04:53:10.198 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T04:53:10.198 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T04:53:10.198 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T04:53:10.198 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T04:53:10.198 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T04:53:10.198 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T04:53:10.199 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T04:53:10.199 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T04:53:10.199 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T04:53:10.199 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T04:53:10.199 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T04:53:10.199 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T04:53:10.199 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T04:53:10.199 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T04:53:10.199 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T04:53:10.199 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T04:53:10.199 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T04:53:10.199 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T04:53:10.200 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T04:53:10.200 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T04:53:10.200 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T04:53:10.200 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T04:53:10.200 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T04:53:10.200 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T04:53:10.200 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T04:53:10.200 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T04:53:10.200 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T04:53:10.200 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T04:53:10.200 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T04:53:10.201 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T04:53:10.201 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T04:53:10.201 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T04:53:10.201 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T04:53:10.201 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T04:53:10.201 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T04:53:10.201 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T04:53:10.201 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T04:53:10.201 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T04:53:10.201 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T04:53:10.201 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T04:53:10.202 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T04:53:10.202 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T04:53:10.202 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T04:53:10.202 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T04:53:10.202 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T04:53:10.202 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T04:53:10.202 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T04:53:10.202 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T04:53:10.202 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T04:53:10.202 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T04:53:10.202 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T04:53:10.203 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T04:53:10.203 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T04:53:10.203 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T04:53:10.203 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T04:53:10.203 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T04:53:10.203 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T04:53:10.203 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T04:53:10.203 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T04:53:10.203 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T04:53:10.203 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T04:53:10.203 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T04:53:10.204 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T04:53:10.204 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T04:53:10.204 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T04:53:10.204 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T04:53:10.204 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T04:53:10.204 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T04:53:10.204 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T04:53:10.204 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T04:53:10.204 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T04:53:10.204 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T04:53:10.204 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T04:53:10.205 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T04:53:10.205 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T04:53:10.205 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T04:53:10.205 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T04:53:10.205 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T04:53:10.205 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T04:53:10.205 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T04:53:10.205 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T04:53:10.205 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T04:53:10.205 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T04:53:10.205 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T04:53:10.206 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T04:53:10.206 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T04:53:10.206 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T04:53:10.206 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T04:53:10.206 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T04:53:10.206 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T04:53:10.206 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T04:53:12.347 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T04:53:12.347 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T04:53:12.347 INFO:teuthology.orchestra.run.smithi043.stdout:Upgraded: 2024-09-19T04:53:12.347 INFO:teuthology.orchestra.run.smithi043.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T04:53:12.347 INFO:teuthology.orchestra.run.smithi043.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T04:53:12.347 INFO:teuthology.orchestra.run.smithi043.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T04:53:12.347 INFO:teuthology.orchestra.run.smithi043.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T04:53:12.347 INFO:teuthology.orchestra.run.smithi043.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:12.347 INFO:teuthology.orchestra.run.smithi043.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:12.348 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T04:53:12.348 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T04:53:12.348 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-19T04:53:12.348 INFO:teuthology.orchestra.run.smithi043.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T04:53:12.348 INFO:teuthology.orchestra.run.smithi043.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T04:53:12.348 INFO:teuthology.orchestra.run.smithi043.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T04:53:12.348 INFO:teuthology.orchestra.run.smithi043.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T04:53:12.348 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:12.348 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:12.348 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:12.348 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:12.348 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:53:12.349 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:12.349 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:12.349 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:12.349 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:53:12.349 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:53:12.349 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:53:12.349 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:53:12.349 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:53:12.349 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:12.349 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:12.349 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:53:12.349 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:12.349 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:12.350 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:12.350 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:53:12.350 INFO:teuthology.orchestra.run.smithi043.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:53:12.350 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:12.350 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:12.350 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T04:53:12.350 INFO:teuthology.orchestra.run.smithi043.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T04:53:12.350 INFO:teuthology.orchestra.run.smithi043.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T04:53:12.350 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T04:53:12.350 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T04:53:12.350 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T04:53:12.350 INFO:teuthology.orchestra.run.smithi043.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T04:53:12.350 INFO:teuthology.orchestra.run.smithi043.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T04:53:12.351 INFO:teuthology.orchestra.run.smithi043.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T04:53:12.351 INFO:teuthology.orchestra.run.smithi043.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T04:53:12.351 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T04:53:12.351 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T04:53:12.351 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:12.351 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:12.351 INFO:teuthology.orchestra.run.smithi043.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:12.351 INFO:teuthology.orchestra.run.smithi043.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T04:53:12.351 INFO:teuthology.orchestra.run.smithi043.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T04:53:12.351 INFO:teuthology.orchestra.run.smithi043.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T04:53:12.351 INFO:teuthology.orchestra.run.smithi043.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T04:53:12.351 INFO:teuthology.orchestra.run.smithi043.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T04:53:12.351 INFO:teuthology.orchestra.run.smithi043.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:12.352 INFO:teuthology.orchestra.run.smithi043.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:12.352 INFO:teuthology.orchestra.run.smithi043.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T04:53:12.352 INFO:teuthology.orchestra.run.smithi043.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:12.352 INFO:teuthology.orchestra.run.smithi043.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T04:53:12.352 INFO:teuthology.orchestra.run.smithi043.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T04:53:12.352 INFO:teuthology.orchestra.run.smithi043.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T04:53:12.352 INFO:teuthology.orchestra.run.smithi043.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T04:53:12.352 INFO:teuthology.orchestra.run.smithi043.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T04:53:12.352 INFO:teuthology.orchestra.run.smithi043.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T04:53:12.352 INFO:teuthology.orchestra.run.smithi043.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T04:53:12.352 INFO:teuthology.orchestra.run.smithi043.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T04:53:12.352 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T04:53:12.353 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T04:53:12.353 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T04:53:12.353 INFO:teuthology.orchestra.run.smithi043.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T04:53:12.353 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T04:53:12.353 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T04:53:12.353 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T04:53:12.353 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T04:53:12.353 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T04:53:12.353 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T04:53:12.353 INFO:teuthology.orchestra.run.smithi043.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T04:53:12.353 INFO:teuthology.orchestra.run.smithi043.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T04:53:12.353 INFO:teuthology.orchestra.run.smithi043.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T04:53:12.353 INFO:teuthology.orchestra.run.smithi043.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T04:53:12.353 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T04:53:12.354 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:12.354 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:12.354 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:12.354 INFO:teuthology.orchestra.run.smithi043.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T04:53:12.354 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T04:53:12.354 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T04:53:12.354 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T04:53:12.354 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T04:53:12.354 INFO:teuthology.orchestra.run.smithi043.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T04:53:12.354 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T04:53:12.354 INFO:teuthology.orchestra.run.smithi043.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T04:53:12.355 INFO:teuthology.orchestra.run.smithi043.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T04:53:12.355 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T04:53:12.355 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T04:53:12.355 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T04:53:12.355 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T04:53:12.355 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T04:53:12.355 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T04:53:12.355 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T04:53:12.355 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T04:53:12.355 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T04:53:12.355 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T04:53:12.355 INFO:teuthology.orchestra.run.smithi043.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T04:53:12.356 INFO:teuthology.orchestra.run.smithi043.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T04:53:12.356 INFO:teuthology.orchestra.run.smithi043.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T04:53:12.356 INFO:teuthology.orchestra.run.smithi043.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T04:53:12.356 INFO:teuthology.orchestra.run.smithi043.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T04:53:12.356 INFO:teuthology.orchestra.run.smithi043.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T04:53:12.356 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T04:53:12.356 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T04:53:12.356 INFO:teuthology.orchestra.run.smithi043.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T04:53:12.356 INFO:teuthology.orchestra.run.smithi043.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T04:53:12.356 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T04:53:12.356 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T04:53:12.357 INFO:teuthology.orchestra.run.smithi043.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T04:53:12.357 INFO:teuthology.orchestra.run.smithi043.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T04:53:12.357 INFO:teuthology.orchestra.run.smithi043.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T04:53:12.357 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T04:53:12.357 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T04:53:12.357 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T04:53:12.357 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T04:53:12.357 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T04:53:12.357 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T04:53:12.357 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:12.360 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:12.361 INFO:teuthology.orchestra.run.smithi043.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T04:53:12.361 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T04:53:12.361 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T04:53:12.361 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:12.361 INFO:teuthology.orchestra.run.smithi043.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T04:53:12.361 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T04:53:12.361 INFO:teuthology.orchestra.run.smithi043.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T04:53:12.361 INFO:teuthology.orchestra.run.smithi043.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T04:53:12.361 INFO:teuthology.orchestra.run.smithi043.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T04:53:12.361 INFO:teuthology.orchestra.run.smithi043.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T04:53:12.362 INFO:teuthology.orchestra.run.smithi043.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T04:53:12.362 INFO:teuthology.orchestra.run.smithi043.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T04:53:12.362 INFO:teuthology.orchestra.run.smithi043.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T04:53:12.362 INFO:teuthology.orchestra.run.smithi043.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T04:53:12.362 INFO:teuthology.orchestra.run.smithi043.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T04:53:12.363 INFO:teuthology.orchestra.run.smithi043.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T04:53:12.363 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T04:53:12.363 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T04:53:12.363 INFO:teuthology.orchestra.run.smithi043.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T04:53:12.363 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:12.363 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:12.364 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T04:53:12.364 INFO:teuthology.orchestra.run.smithi043.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T04:53:12.364 INFO:teuthology.orchestra.run.smithi043.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T04:53:12.364 INFO:teuthology.orchestra.run.smithi043.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T04:53:12.364 INFO:teuthology.orchestra.run.smithi043.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T04:53:12.364 INFO:teuthology.orchestra.run.smithi043.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T04:53:12.365 INFO:teuthology.orchestra.run.smithi043.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T04:53:12.365 INFO:teuthology.orchestra.run.smithi043.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T04:53:12.365 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T04:53:12.365 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-19T04:53:12.941 DEBUG:teuthology.parallel:result is None 2024-09-19T04:53:12.941 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:53:13.087 DEBUG:teuthology.orchestra.run.smithi043:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T04:53:13.133 INFO:teuthology.orchestra.run.smithi043.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T04:53:13.134 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T04:53:13.134 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T04:53:13.136 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:53:13.279 DEBUG:teuthology.orchestra.run.smithi096:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T04:53:13.306 INFO:teuthology.orchestra.run.smithi096.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T04:53:13.306 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T04:53:13.306 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T04:53:13.309 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:53:13.461 DEBUG:teuthology.orchestra.run.smithi151:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T04:53:13.488 INFO:teuthology.orchestra.run.smithi151.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T04:53:13.489 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T04:53:13.489 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T04:53:13.491 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-19T04:53:13.491 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-19T04:53:13.491 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T04:53:13.550 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-19T04:53:13.550 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T04:53:13.583 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-19T04:53:13.583 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T04:53:13.615 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-19T04:53:13.615 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-19T04:53:13.615 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T04:53:13.672 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T04:53:13.769 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-19T04:53:13.769 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T04:53:13.802 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T04:53:13.871 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-19T04:53:13.871 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T04:53:13.903 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T04:53:13.972 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-19T04:53:13.972 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-19T04:53:13.972 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T04:53:14.028 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T04:53:14.128 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-19T04:53:14.128 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T04:53:14.157 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T04:53:14.226 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-19T04:53:14.226 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T04:53:14.256 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T04:53:14.327 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-19T04:53:14.328 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-19T04:53:14.328 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T04:53:14.385 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T04:53:14.490 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-19T04:53:14.490 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T04:53:14.521 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T04:53:14.590 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-19T04:53:14.590 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T04:53:14.620 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T04:53:14.690 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-19T04:53:14.787 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-19T04:53:14.788 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:53:14.788 INFO:tasks.cephadm:Cluster fsid is 1456a810-7643-11ef-bceb-c7b262605968 2024-09-19T04:53:14.788 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-19T04:53:14.788 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.43', 'mon.b': '172.21.15.96', 'mon.c': '172.21.15.151'} 2024-09-19T04:53:14.789 INFO:tasks.cephadm:First mon is mon.a on smithi043 2024-09-19T04:53:14.789 INFO:tasks.cephadm:First mgr is x 2024-09-19T04:53:14.789 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-19T04:53:14.789 DEBUG:teuthology.orchestra.run.smithi043:> sudo hostname $(hostname -s) 2024-09-19T04:53:14.847 DEBUG:teuthology.orchestra.run.smithi096:> sudo hostname $(hostname -s) 2024-09-19T04:53:14.877 DEBUG:teuthology.orchestra.run.smithi151:> sudo hostname $(hostname -s) 2024-09-19T04:53:14.906 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-19T04:53:14.906 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:53:15.055 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56106/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-361-g2f599a46', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-361.g2f599a46'}, 'url': 'https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-18 17:52:24.287651', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-19T04:53:15.198 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref squid, sha1 2f599a46ad10e37c929e5c43da2f6907ddcd1961 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:53:15.199 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm 2024-09-19T04:53:15.199 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm 2024-09-19T04:53:15.199 DEBUG:teuthology.orchestra.run.smithi043:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T04:53:15.925 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 04:53 /home/ubuntu/cephtest/cephadm 2024-09-19T04:53:15.925 DEBUG:teuthology.orchestra.run.smithi096:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T04:53:16.464 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 04:53 /home/ubuntu/cephtest/cephadm 2024-09-19T04:53:16.465 DEBUG:teuthology.orchestra.run.smithi151:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T04:53:16.942 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 04:53 /home/ubuntu/cephtest/cephadm 2024-09-19T04:53:16.943 DEBUG:teuthology.orchestra.run.smithi043:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T04:53:16.981 DEBUG:teuthology.orchestra.run.smithi096:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T04:53:17.001 DEBUG:teuthology.orchestra.run.smithi151:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T04:53:17.032 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 on all hosts... 2024-09-19T04:53:17.032 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T04:53:17.036 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T04:53:17.043 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T04:53:18.590 INFO:teuthology.orchestra.run.smithi096.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T04:53:18.628 INFO:teuthology.orchestra.run.smithi151.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T04:53:18.998 INFO:teuthology.orchestra.run.smithi043.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T04:53:47.935 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-19T04:53:47.936 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T04:53:47.936 INFO:teuthology.orchestra.run.smithi151.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T04:53:47.936 INFO:teuthology.orchestra.run.smithi151.stdout: "repo_digests": [ 2024-09-19T04:53:47.936 INFO:teuthology.orchestra.run.smithi151.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T04:53:47.936 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-09-19T04:53:47.936 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-19T04:53:50.261 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-19T04:53:50.262 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T04:53:50.262 INFO:teuthology.orchestra.run.smithi096.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T04:53:50.262 INFO:teuthology.orchestra.run.smithi096.stdout: "repo_digests": [ 2024-09-19T04:53:50.262 INFO:teuthology.orchestra.run.smithi096.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T04:53:50.262 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-19T04:53:50.262 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-19T04:53:53.457 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T04:53:53.457 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T04:53:53.457 INFO:teuthology.orchestra.run.smithi043.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T04:53:53.457 INFO:teuthology.orchestra.run.smithi043.stdout: "repo_digests": [ 2024-09-19T04:53:53.457 INFO:teuthology.orchestra.run.smithi043.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T04:53:53.457 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-19T04:53:53.457 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T04:53:53.488 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /etc/ceph 2024-09-19T04:53:53.546 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /etc/ceph 2024-09-19T04:53:53.580 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /etc/ceph 2024-09-19T04:53:53.613 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 777 /etc/ceph 2024-09-19T04:53:53.669 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 777 /etc/ceph 2024-09-19T04:53:53.698 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 777 /etc/ceph 2024-09-19T04:53:53.726 INFO:tasks.cephadm:Writing seed config... 2024-09-19T04:53:53.727 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-19T04:53:53.728 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-19T04:53:53.728 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-19T04:53:53.728 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-19T04:53:53.728 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-19T04:53:53.728 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-19T04:53:53.728 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-19T04:53:53.728 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-19T04:53:53.728 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-19T04:53:53.728 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-19T04:53:53.728 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-19T04:53:53.728 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-19T04:53:53.728 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-19T04:53:53.728 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-19T04:53:53.728 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-19T04:53:53.729 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-19T04:53:53.729 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-19T04:53:53.729 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-09-19T04:53:53.729 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-19T04:53:53.729 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-19T04:53:53.729 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-19T04:53:53.729 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-19T04:53:53.729 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-09-19T04:53:53.729 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-19T04:53:53.729 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-19T04:53:53.729 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-19T04:53:53.729 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-19T04:53:53.729 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-19T04:53:53.730 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-19T04:53:53.730 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-19T04:53:53.730 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-19T04:53:53.730 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-19T04:53:53.730 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-19T04:53:53.730 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-19T04:53:53.730 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-19T04:53:53.730 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-19T04:53:53.730 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-19T04:53:53.730 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-19T04:53:53.730 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-19T04:53:53.730 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-19T04:53:53.730 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-19T04:53:53.730 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-19T04:53:53.731 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-19T04:53:53.731 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-19T04:53:53.731 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-19T04:53:53.731 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-19T04:53:53.731 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-19T04:53:53.731 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-19T04:53:53.732 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-19T04:53:53.732 DEBUG:teuthology.orchestra.run.smithi043:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-19T04:53:53.767 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 1456a810-7643-11ef-bceb-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-19T04:53:53.767 DEBUG:teuthology.orchestra.run.smithi043:mon.a> sudo journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@mon.a.service 2024-09-19T04:53:53.812 DEBUG:teuthology.orchestra.run.smithi043:mgr.x> sudo journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@mgr.x.service 2024-09-19T04:53:53.857 INFO:tasks.cephadm:Bootstrapping... 2024-09-19T04:53:53.857 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 -v bootstrap --fsid 1456a810-7643-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.43 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-19T04:53:54.123 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-19T04:53:54.123 INFO:teuthology.orchestra.run.smithi043.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961', '-v', 'bootstrap', '--fsid', '1456a810-7643-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.43', '--skip-admin-label'] 2024-09-19T04:53:54.123 INFO:teuthology.orchestra.run.smithi043.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-19T04:53:54.123 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying podman|docker is present... 2024-09-19T04:53:54.169 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stdout 5.2.2 2024-09-19T04:53:54.169 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying lvm2 is present... 2024-09-19T04:53:54.169 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying time synchronization is in place... 2024-09-19T04:53:54.183 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-19T04:53:54.183 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-19T04:53:54.195 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-19T04:53:54.195 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-09-19T04:53:54.209 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout enabled 2024-09-19T04:53:54.221 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout active 2024-09-19T04:53:54.221 INFO:teuthology.orchestra.run.smithi043.stdout:Unit chronyd.service is enabled and running 2024-09-19T04:53:54.221 INFO:teuthology.orchestra.run.smithi043.stdout:Repeating the final host check... 2024-09-19T04:53:54.265 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stdout 5.2.2 2024-09-19T04:53:54.265 INFO:teuthology.orchestra.run.smithi043.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-19T04:53:54.265 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl is present 2024-09-19T04:53:54.265 INFO:teuthology.orchestra.run.smithi043.stdout:lvcreate is present 2024-09-19T04:53:54.277 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-19T04:53:54.277 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-19T04:53:54.289 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-19T04:53:54.289 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-09-19T04:53:54.303 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout enabled 2024-09-19T04:53:54.315 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout active 2024-09-19T04:53:54.315 INFO:teuthology.orchestra.run.smithi043.stdout:Unit chronyd.service is enabled and running 2024-09-19T04:53:54.316 INFO:teuthology.orchestra.run.smithi043.stdout:Host looks OK 2024-09-19T04:53:54.316 INFO:teuthology.orchestra.run.smithi043.stdout:Cluster fsid: 1456a810-7643-11ef-bceb-c7b262605968 2024-09-19T04:53:54.316 INFO:teuthology.orchestra.run.smithi043.stdout:Acquiring lock 140593856549888 on /run/cephadm/1456a810-7643-11ef-bceb-c7b262605968.lock 2024-09-19T04:53:54.316 INFO:teuthology.orchestra.run.smithi043.stdout:Lock 140593856549888 acquired on /run/cephadm/1456a810-7643-11ef-bceb-c7b262605968.lock 2024-09-19T04:53:54.316 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying IP 172.21.15.43 port 3300 ... 2024-09-19T04:53:54.317 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying IP 172.21.15.43 port 6789 ... 2024-09-19T04:53:54.317 INFO:teuthology.orchestra.run.smithi043.stdout:Base mon IP(s) is [172.21.15.43:3300, 172.21.15.43:6789], mon addrv is [v2:172.21.15.43:3300,v1:172.21.15.43:6789] 2024-09-19T04:53:54.324 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.43 metric 100 2024-09-19T04:53:54.324 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.43 metric 100 2024-09-19T04:53:54.330 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-19T04:53:54.330 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-19T04:53:54.330 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1781sec hoplimit 64 pref medium 2024-09-19T04:53:54.336 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-19T04:53:54.336 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-19T04:53:54.336 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-19T04:53:54.336 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-19T04:53:54.336 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1358/64 scope link 2024-09-19T04:53:54.336 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-19T04:53:54.337 INFO:teuthology.orchestra.run.smithi043.stdout:Mon IP `172.21.15.43` is in CIDR network `172.21.0.0/20` 2024-09-19T04:53:54.337 INFO:teuthology.orchestra.run.smithi043.stdout:Mon IP `172.21.15.43` is in CIDR network `172.21.0.0/20` 2024-09-19T04:53:54.337 INFO:teuthology.orchestra.run.smithi043.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-19T04:53:54.337 INFO:teuthology.orchestra.run.smithi043.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-19T04:53:54.338 INFO:teuthology.orchestra.run.smithi043.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T04:53:54.865 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stdout d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d 2024-09-19T04:53:54.865 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T04:53:54.865 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Getting image source signatures 2024-09-19T04:53:54.865 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Copying blob sha256:c4221751d16d26d2a4b7031ebd71ecffffff13c8cf7401cf95ce80dab3b6d67a 2024-09-19T04:53:54.866 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-19T04:53:54.866 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Copying config sha256:d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d 2024-09-19T04:53:54.866 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-19T04:53:55.886 INFO:teuthology.orchestra.run.smithi043.stdout:ceph: stdout ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable) 2024-09-19T04:53:55.886 INFO:teuthology.orchestra.run.smithi043.stdout:Ceph version: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable) 2024-09-19T04:53:55.886 INFO:teuthology.orchestra.run.smithi043.stdout:Extracting ceph user uid/gid from container image... 2024-09-19T04:54:07.594 INFO:teuthology.orchestra.run.smithi043.stdout:stat: stdout 167 167 2024-09-19T04:54:07.594 INFO:teuthology.orchestra.run.smithi043.stdout:Creating initial keys... 2024-09-19T04:54:08.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph-authtool: stdout AQBvrutm10J9OhAAhe/S59gVjcGhKPUyctYV6A== 2024-09-19T04:54:09.612 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph-authtool: stdout AQBwrutmmVS4OhAA8aR06ZP+PXCQdMdpjEseeQ== 2024-09-19T04:54:10.625 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph-authtool: stdout AQByrutmYgBuABAAGGSlNZJpAsRZwfv+hmA8Xg== 2024-09-19T04:54:10.626 INFO:teuthology.orchestra.run.smithi043.stdout:Creating initial monmap... 2024-09-19T04:54:11.678 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-19T04:54:11.679 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-19T04:54:11.679 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 1456a810-7643-11ef-bceb-c7b262605968 2024-09-19T04:54:11.679 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-19T04:54:11.679 INFO:teuthology.orchestra.run.smithi043.stdout:monmaptool for a [v2:172.21.15.43:3300,v1:172.21.15.43:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-19T04:54:11.679 INFO:teuthology.orchestra.run.smithi043.stdout:setting min_mon_release = quincy 2024-09-19T04:54:11.679 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: set fsid to 1456a810-7643-11ef-bceb-c7b262605968 2024-09-19T04:54:11.679 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-19T04:54:11.679 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T04:54:11.679 INFO:teuthology.orchestra.run.smithi043.stdout:Creating mon... 2024-09-19T04:54:12.645 INFO:teuthology.orchestra.run.smithi043.stdout:create mon.a on 2024-09-19T04:54:12.964 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-19T04:54:13.195 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-19T04:54:13.480 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-1456a810-7643-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-1456a810-7643-11ef-bceb-c7b262605968.target. 2024-09-19T04:54:13.480 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-1456a810-7643-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-1456a810-7643-11ef-bceb-c7b262605968.target. 2024-09-19T04:54:13.818 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1456a810-7643-11ef-bceb-c7b262605968@mon.a 2024-09-19T04:54:13.818 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1456a810-7643-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-1456a810-7643-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-19T04:54:14.091 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1456a810-7643-11ef-bceb-c7b262605968.target.wants/ceph-1456a810-7643-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-1456a810-7643-11ef-bceb-c7b262605968@.service. 2024-09-19T04:54:14.710 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:14 smithi043 podman[30419]: 2024-09-19 04:54:14.647558678 +0000 UTC m=+0.323683360 container start 07f242987487feae1db27d8dfdf5f669a4f24c332e6f280afbc1d0d5609adbd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mon-a, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4) 2024-09-19T04:54:14.710 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:14 smithi043 ceph-mon[30437]: mkfs 1456a810-7643-11ef-bceb-c7b262605968 2024-09-19T04:54:14.770 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T04:54:14.770 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout disabled 2024-09-19T04:54:14.783 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T04:54:14.783 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-09-19T04:54:14.783 INFO:teuthology.orchestra.run.smithi043.stdout:firewalld.service is not enabled 2024-09-19T04:54:14.783 INFO:teuthology.orchestra.run.smithi043.stdout:Not possible to enable service . firewalld.service is not available 2024-09-19T04:54:14.784 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mon to start... 2024-09-19T04:54:14.784 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mon... 2024-09-19T04:54:15.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:14 smithi043 ceph-mon[30437]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T04:54:15.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:14 smithi043 bash[30419]: 07f242987487feae1db27d8dfdf5f669a4f24c332e6f280afbc1d0d5609adbd6 2024-09-19T04:54:15.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:14 smithi043 systemd[1]: Started Ceph mon.a for 1456a810-7643-11ef-bceb-c7b262605968. 2024-09-19T04:54:15.956 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:15 smithi043 ceph-mon[30437]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T04:54:15.956 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:15 smithi043 ceph-mon[30437]: monmap epoch 1 2024-09-19T04:54:15.956 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:15 smithi043 ceph-mon[30437]: fsid 1456a810-7643-11ef-bceb-c7b262605968 2024-09-19T04:54:15.956 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:15 smithi043 ceph-mon[30437]: last_changed 2024-09-19T04:54:11.079879+0000 2024-09-19T04:54:15.957 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:15 smithi043 ceph-mon[30437]: created 2024-09-19T04:54:11.079879+0000 2024-09-19T04:54:15.957 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:15 smithi043 ceph-mon[30437]: min_mon_release 19 (squid) 2024-09-19T04:54:15.957 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:15 smithi043 ceph-mon[30437]: election_strategy: 1 2024-09-19T04:54:15.957 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:15 smithi043 ceph-mon[30437]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-09-19T04:54:15.957 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:15 smithi043 ceph-mon[30437]: fsmap 2024-09-19T04:54:15.957 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:15 smithi043 ceph-mon[30437]: osdmap e1: 0 total, 0 up, 0 in 2024-09-19T04:54:15.957 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:15 smithi043 ceph-mon[30437]: mgrmap e1: no daemons active 2024-09-19T04:54:15.957 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:15 smithi043 ceph-mon[30437]: from='client.? 172.21.15.43:0/3553944717' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-19T04:54:15.971 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout cluster: 2024-09-19T04:54:15.971 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout id: 1456a810-7643-11ef-bceb-c7b262605968 2024-09-19T04:54:15.972 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-19T04:54:15.972 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-19T04:54:15.972 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout services: 2024-09-19T04:54:15.972 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.667746s) 2024-09-19T04:54:15.972 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-19T04:54:15.973 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-19T04:54:15.973 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-19T04:54:15.973 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout data: 2024-09-19T04:54:15.973 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-19T04:54:15.973 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-19T04:54:15.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-19T04:54:15.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout pgs: 2024-09-19T04:54:15.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-19T04:54:15.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.294+0000 7ff408e00640 1 Processor -- start 2024-09-19T04:54:15.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.294+0000 7ff408e00640 1 -- start start 2024-09-19T04:54:15.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.294+0000 7ff408e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff40410ab40 0x7ff40410af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:15.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.294+0000 7ff408e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff40410b510 con 0x7ff40410ab40 2024-09-19T04:54:15.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.295+0000 7ff403400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff40410ab40 0x7ff40410af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:15.975 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.295+0000 7ff403400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff40410ab40 0x7ff40410af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51042/0 (socket says 172.21.15.43:51042) 2024-09-19T04:54:15.975 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.295+0000 7ff403400640 1 -- 172.21.15.43:0/1023911019 learned_addr learned my addr 172.21.15.43:0/1023911019 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:54:15.975 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.296+0000 7ff403400640 1 -- 172.21.15.43:0/1023911019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff40410b690 con 0x7ff40410ab40 2024-09-19T04:54:15.975 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.298+0000 7ff403400640 1 --2- 172.21.15.43:0/1023911019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff40410ab40 0x7ff40410af40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4009820 tx=0x7ff3f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc8065af312901c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:15.975 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.299+0000 7ff402a00640 1 -- 172.21.15.43:0/1023911019 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3f4037070 con 0x7ff40410ab40 2024-09-19T04:54:15.975 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.299+0000 7ff402a00640 1 -- 172.21.15.43:0/1023911019 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3f4032440 con 0x7ff40410ab40 2024-09-19T04:54:15.975 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.299+0000 7ff402a00640 1 -- 172.21.15.43:0/1023911019 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3f4032740 con 0x7ff40410ab40 2024-09-19T04:54:15.975 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.299+0000 7ff408e00640 1 -- 172.21.15.43:0/1023911019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff40410ab40 msgr2=0x7ff40410af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:15.975 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.299+0000 7ff408e00640 1 --2- 172.21.15.43:0/1023911019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff40410ab40 0x7ff40410af40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4009820 tx=0x7ff3f402a270 comp rx=0 tx=0).stop 2024-09-19T04:54:15.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.299+0000 7ff408e00640 1 -- 172.21.15.43:0/1023911019 shutdown_connections 2024-09-19T04:54:15.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.299+0000 7ff408e00640 1 --2- 172.21.15.43:0/1023911019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff40410ab40 0x7ff40410af40 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:15.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.299+0000 7ff408e00640 1 -- 172.21.15.43:0/1023911019 >> 172.21.15.43:0/1023911019 conn(0x7ff40407cd70 msgr2=0x7ff40407d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:15.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.299+0000 7ff408e00640 1 -- 172.21.15.43:0/1023911019 shutdown_connections 2024-09-19T04:54:15.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.299+0000 7ff408e00640 1 -- 172.21.15.43:0/1023911019 wait complete. 2024-09-19T04:54:15.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.300+0000 7ff408e00640 1 Processor -- start 2024-09-19T04:54:15.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.300+0000 7ff408e00640 1 -- start start 2024-09-19T04:54:15.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.301+0000 7ff408e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff40410ab40 0x7ff40419cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:15.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.301+0000 7ff408e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff404109980 con 0x7ff40410ab40 2024-09-19T04:54:15.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.301+0000 7ff403400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff40410ab40 0x7ff40419cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:15.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.301+0000 7ff403400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff40410ab40 0x7ff40419cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51052/0 (socket says 172.21.15.43:51052) 2024-09-19T04:54:15.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.301+0000 7ff403400640 1 -- 172.21.15.43:0/3553944717 learned_addr learned my addr 172.21.15.43:0/3553944717 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:54:15.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.301+0000 7ff403400640 1 -- 172.21.15.43:0/3553944717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff40419d420 con 0x7ff40410ab40 2024-09-19T04:54:15.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.302+0000 7ff403400640 1 --2- 172.21.15.43:0/3553944717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff40410ab40 0x7ff40419cee0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff3f40090b0 tx=0x7ff3f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:15.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.302+0000 7ff400c00640 1 -- 172.21.15.43:0/3553944717 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3f4041070 con 0x7ff40410ab40 2024-09-19T04:54:15.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.302+0000 7ff400c00640 1 -- 172.21.15.43:0/3553944717 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3f403c7a0 con 0x7ff40410ab40 2024-09-19T04:54:15.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.302+0000 7ff400c00640 1 -- 172.21.15.43:0/3553944717 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3f4037040 con 0x7ff40410ab40 2024-09-19T04:54:15.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.302+0000 7ff408e00640 1 -- 172.21.15.43:0/3553944717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff40419d6b0 con 0x7ff40410ab40 2024-09-19T04:54:15.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.302+0000 7ff408e00640 1 -- 172.21.15.43:0/3553944717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4041a03a0 con 0x7ff40410ab40 2024-09-19T04:54:15.978 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.302+0000 7ff400c00640 1 -- 172.21.15.43:0/3553944717 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff3f4044070 con 0x7ff40410ab40 2024-09-19T04:54:15.978 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.302+0000 7ff400c00640 1 -- 172.21.15.43:0/3553944717 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff3f403d3f0 con 0x7ff40410ab40 2024-09-19T04:54:15.978 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.303+0000 7ff408e00640 1 -- 172.21.15.43:0/3553944717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff40410afc0 con 0x7ff40410ab40 2024-09-19T04:54:15.978 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.306+0000 7ff400c00640 1 -- 172.21.15.43:0/3553944717 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7ff3f4046450 con 0x7ff40410ab40 2024-09-19T04:54:15.978 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.371+0000 7ff408e00640 1 -- 172.21.15.43:0/3553944717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7ff4041a08f0 con 0x7ff40410ab40 2024-09-19T04:54:15.978 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.371+0000 7ff400c00640 1 -- 172.21.15.43:0/3553944717 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7ff3f4046630 con 0x7ff40410ab40 2024-09-19T04:54:15.978 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.372+0000 7ff408e00640 1 -- 172.21.15.43:0/3553944717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff40410ab40 msgr2=0x7ff40419cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:15.978 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.372+0000 7ff408e00640 1 --2- 172.21.15.43:0/3553944717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff40410ab40 0x7ff40419cee0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff3f40090b0 tx=0x7ff3f40046c0 comp rx=0 tx=0).stop 2024-09-19T04:54:15.978 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.372+0000 7ff408e00640 1 -- 172.21.15.43:0/3553944717 shutdown_connections 2024-09-19T04:54:15.978 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.372+0000 7ff408e00640 1 --2- 172.21.15.43:0/3553944717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff40410ab40 0x7ff40419cee0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:15.978 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.372+0000 7ff408e00640 1 -- 172.21.15.43:0/3553944717 >> 172.21.15.43:0/3553944717 conn(0x7ff40407cd70 msgr2=0x7ff404191200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:15.978 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.373+0000 7ff408e00640 1 -- 172.21.15.43:0/3553944717 shutdown_connections 2024-09-19T04:54:15.979 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:15.373+0000 7ff408e00640 1 -- 172.21.15.43:0/3553944717 wait complete. 2024-09-19T04:54:15.979 INFO:teuthology.orchestra.run.smithi043.stdout:mon is available 2024-09-19T04:54:15.979 INFO:teuthology.orchestra.run.smithi043.stdout:Assimilating anything we can from ceph.conf... 2024-09-19T04:54:16.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:16 smithi043 ceph-mon[30437]: from='client.? 172.21.15.43:0/937706371' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-19T04:54:16.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:16 smithi043 ceph-mon[30437]: from='client.? 172.21.15.43:0/937706371' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-19T04:54:17.159 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-19T04:54:17.160 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [global] 2024-09-19T04:54:17.160 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout fsid = 1456a810-7643-11ef-bceb-c7b262605968 2024-09-19T04:54:17.160 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.43:3300,v1:172.21.15.43:6789] 2024-09-19T04:54:17.160 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-19T04:54:17.160 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-19T04:54:17.160 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-19T04:54:17.160 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-19T04:54:17.160 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-19T04:54:17.160 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-19T04:54:17.160 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-19T04:54:17.160 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-19T04:54:17.160 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [osd] 2024-09-19T04:54:17.160 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-19T04:54:17.161 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-19T04:54:17.161 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-19T04:54:17.161 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-19T04:54:17.161 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-19T04:54:17.161 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.433+0000 7faf09600640 1 Processor -- start 2024-09-19T04:54:17.161 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.433+0000 7faf09600640 1 -- start start 2024-09-19T04:54:17.161 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.433+0000 7faf09600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf04109960 0x7faf04109d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:17.161 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.433+0000 7faf09600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faf0410a330 con 0x7faf04109960 2024-09-19T04:54:17.161 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.434+0000 7faf02a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf04109960 0x7faf04109d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:17.161 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.434+0000 7faf02a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf04109960 0x7faf04109d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53290/0 (socket says 172.21.15.43:53290) 2024-09-19T04:54:17.161 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.434+0000 7faf02a00640 1 -- 172.21.15.43:0/413339663 learned_addr learned my addr 172.21.15.43:0/413339663 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:54:17.161 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.434+0000 7faf02a00640 1 -- 172.21.15.43:0/413339663 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf0410aad0 con 0x7faf04109960 2024-09-19T04:54:17.162 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.435+0000 7faf02a00640 1 --2- 172.21.15.43:0/413339663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf04109960 0x7faf04109d60 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7faef4009e80 tx=0x7faef402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a88be84c5069552 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:17.162 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.435+0000 7faf02000640 1 -- 172.21.15.43:0/413339663 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faef4036070 con 0x7faf04109960 2024-09-19T04:54:17.162 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.435+0000 7faf02000640 1 -- 172.21.15.43:0/413339663 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faef4031400 con 0x7faf04109960 2024-09-19T04:54:17.162 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.435+0000 7faf02000640 1 -- 172.21.15.43:0/413339663 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faef4031700 con 0x7faf04109960 2024-09-19T04:54:17.162 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.436+0000 7faf09600640 1 -- 172.21.15.43:0/413339663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf04109960 msgr2=0x7faf04109d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:17.162 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.436+0000 7faf09600640 1 --2- 172.21.15.43:0/413339663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf04109960 0x7faf04109d60 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7faef4009e80 tx=0x7faef402a7a0 comp rx=0 tx=0).stop 2024-09-19T04:54:17.162 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.436+0000 7faf09600640 1 -- 172.21.15.43:0/413339663 shutdown_connections 2024-09-19T04:54:17.162 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.436+0000 7faf09600640 1 --2- 172.21.15.43:0/413339663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf04109960 0x7faf04109d60 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:17.162 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.436+0000 7faf09600640 1 -- 172.21.15.43:0/413339663 >> 172.21.15.43:0/413339663 conn(0x7faf0407cf30 msgr2=0x7faf0407d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:17.162 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.436+0000 7faf09600640 1 -- 172.21.15.43:0/413339663 shutdown_connections 2024-09-19T04:54:17.162 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.436+0000 7faf09600640 1 -- 172.21.15.43:0/413339663 wait complete. 2024-09-19T04:54:17.162 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.437+0000 7faf09600640 1 Processor -- start 2024-09-19T04:54:17.163 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.437+0000 7faf09600640 1 -- start start 2024-09-19T04:54:17.163 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.437+0000 7faf09600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf04109960 0x7faf0419d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:17.163 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.437+0000 7faf09600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faf0410b070 con 0x7faf04109960 2024-09-19T04:54:17.163 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.438+0000 7faf02a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf04109960 0x7faf0419d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:17.163 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.438+0000 7faf02a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf04109960 0x7faf0419d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53300/0 (socket says 172.21.15.43:53300) 2024-09-19T04:54:17.163 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.438+0000 7faf02a00640 1 -- 172.21.15.43:0/937706371 learned_addr learned my addr 172.21.15.43:0/937706371 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:54:17.163 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.438+0000 7faf02a00640 1 -- 172.21.15.43:0/937706371 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf0419d590 con 0x7faf04109960 2024-09-19T04:54:17.163 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.438+0000 7faf02a00640 1 --2- 172.21.15.43:0/937706371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf04109960 0x7faf0419d050 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7faef4034040 tx=0x7faef4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:17.163 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.439+0000 7faefbe00640 1 -- 172.21.15.43:0/937706371 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faef4041070 con 0x7faf04109960 2024-09-19T04:54:17.163 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.439+0000 7faefbe00640 1 -- 172.21.15.43:0/937706371 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faef403c650 con 0x7faf04109960 2024-09-19T04:54:17.163 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.439+0000 7faefbe00640 1 -- 172.21.15.43:0/937706371 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faef4036040 con 0x7faf04109960 2024-09-19T04:54:17.163 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.439+0000 7faf09600640 1 -- 172.21.15.43:0/937706371 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf0419d820 con 0x7faf04109960 2024-09-19T04:54:17.164 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.439+0000 7faf09600640 1 -- 172.21.15.43:0/937706371 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf041a0510 con 0x7faf04109960 2024-09-19T04:54:17.164 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.439+0000 7faefbe00640 1 -- 172.21.15.43:0/937706371 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7faef4044070 con 0x7faf04109960 2024-09-19T04:54:17.164 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.439+0000 7faefbe00640 1 -- 172.21.15.43:0/937706371 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7faef4046c10 con 0x7faf04109960 2024-09-19T04:54:17.164 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.440+0000 7faf09600640 1 -- 172.21.15.43:0/937706371 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf0410b6c0 con 0x7faf04109960 2024-09-19T04:54:17.164 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.442+0000 7faefbe00640 1 -- 172.21.15.43:0/937706371 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7faef403c950 con 0x7faf04109960 2024-09-19T04:54:17.164 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.506+0000 7faf09600640 1 -- 172.21.15.43:0/937706371 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7faf0402d910 con 0x7faf04109960 2024-09-19T04:54:17.164 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.509+0000 7faefbe00640 1 -- 172.21.15.43:0/937706371 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7faef403cb30 con 0x7faf04109960 2024-09-19T04:54:17.164 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.510+0000 7faefbe00640 1 -- 172.21.15.43:0/937706371 <== mon.0 v2:172.21.15.43:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7faef4056070 con 0x7faf04109960 2024-09-19T04:54:17.164 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.511+0000 7faf09600640 1 -- 172.21.15.43:0/937706371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf04109960 msgr2=0x7faf0419d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:17.164 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.511+0000 7faf09600640 1 --2- 172.21.15.43:0/937706371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf04109960 0x7faf0419d050 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7faef4034040 tx=0x7faef4004a40 comp rx=0 tx=0).stop 2024-09-19T04:54:17.165 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.511+0000 7faf09600640 1 -- 172.21.15.43:0/937706371 shutdown_connections 2024-09-19T04:54:17.165 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.511+0000 7faf09600640 1 --2- 172.21.15.43:0/937706371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf04109960 0x7faf0419d050 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:17.165 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.511+0000 7faf09600640 1 -- 172.21.15.43:0/937706371 >> 172.21.15.43:0/937706371 conn(0x7faf0407cf30 msgr2=0x7faf04191230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:17.165 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.512+0000 7faf09600640 1 -- 172.21.15.43:0/937706371 shutdown_connections 2024-09-19T04:54:17.165 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:16.512+0000 7faf09600640 1 -- 172.21.15.43:0/937706371 wait complete. 2024-09-19T04:54:17.165 INFO:teuthology.orchestra.run.smithi043.stdout:Generating new minimal ceph.conf... 2024-09-19T04:54:18.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:17 smithi043 ceph-mon[30437]: from='client.? 172.21.15.43:0/1669763046' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:54:18.361 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.663+0000 7fb4f0e00640 1 Processor -- start 2024-09-19T04:54:18.361 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.663+0000 7fb4f0e00640 1 -- start start 2024-09-19T04:54:18.361 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.663+0000 7fb4f0e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4ec1075c0 0x7fb4ec1079c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:18.362 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.663+0000 7fb4f0e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ec107f90 con 0x7fb4ec1075c0 2024-09-19T04:54:18.362 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.663+0000 7fb4eb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4ec1075c0 0x7fb4ec1079c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:18.362 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.664+0000 7fb4eb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4ec1075c0 0x7fb4ec1079c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53314/0 (socket says 172.21.15.43:53314) 2024-09-19T04:54:18.362 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.664+0000 7fb4eb400640 1 -- 172.21.15.43:0/2181140741 learned_addr learned my addr 172.21.15.43:0/2181140741 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:54:18.362 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.664+0000 7fb4eb400640 1 -- 172.21.15.43:0/2181140741 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4ec108720 con 0x7fb4ec1075c0 2024-09-19T04:54:18.362 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.665+0000 7fb4eb400640 1 --2- 172.21.15.43:0/2181140741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4ec1075c0 0x7fb4ec1079c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc009820 tx=0x7fb4dc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45c786a94599469 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:18.362 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.665+0000 7fb4eaa00640 1 -- 172.21.15.43:0/2181140741 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4dc037070 con 0x7fb4ec1075c0 2024-09-19T04:54:18.362 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.665+0000 7fb4eaa00640 1 -- 172.21.15.43:0/2181140741 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb4dc032900 con 0x7fb4ec1075c0 2024-09-19T04:54:18.362 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.665+0000 7fb4eaa00640 1 -- 172.21.15.43:0/2181140741 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4dc030850 con 0x7fb4ec1075c0 2024-09-19T04:54:18.362 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.666+0000 7fb4f0e00640 1 -- 172.21.15.43:0/2181140741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4ec1075c0 msgr2=0x7fb4ec1079c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:18.362 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.666+0000 7fb4f0e00640 1 --2- 172.21.15.43:0/2181140741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4ec1075c0 0x7fb4ec1079c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc009820 tx=0x7fb4dc02a270 comp rx=0 tx=0).stop 2024-09-19T04:54:18.362 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.666+0000 7fb4f0e00640 1 -- 172.21.15.43:0/2181140741 shutdown_connections 2024-09-19T04:54:18.363 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.666+0000 7fb4f0e00640 1 --2- 172.21.15.43:0/2181140741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4ec1075c0 0x7fb4ec1079c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:18.363 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.666+0000 7fb4f0e00640 1 -- 172.21.15.43:0/2181140741 >> 172.21.15.43:0/2181140741 conn(0x7fb4ec102d90 msgr2=0x7fb4ec1051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:18.363 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.666+0000 7fb4f0e00640 1 -- 172.21.15.43:0/2181140741 shutdown_connections 2024-09-19T04:54:18.363 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.666+0000 7fb4f0e00640 1 -- 172.21.15.43:0/2181140741 wait complete. 2024-09-19T04:54:18.363 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.667+0000 7fb4f0e00640 1 Processor -- start 2024-09-19T04:54:18.363 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.667+0000 7fb4f0e00640 1 -- start start 2024-09-19T04:54:18.363 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.667+0000 7fb4f0e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4ec1075c0 0x7fb4ec19d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:18.363 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.667+0000 7fb4f0e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ec10bad0 con 0x7fb4ec1075c0 2024-09-19T04:54:18.363 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.667+0000 7fb4eb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4ec1075c0 0x7fb4ec19d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:18.363 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.668+0000 7fb4eb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4ec1075c0 0x7fb4ec19d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53318/0 (socket says 172.21.15.43:53318) 2024-09-19T04:54:18.363 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.668+0000 7fb4eb400640 1 -- 172.21.15.43:0/1669763046 learned_addr learned my addr 172.21.15.43:0/1669763046 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:54:18.363 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.668+0000 7fb4eb400640 1 -- 172.21.15.43:0/1669763046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4ec19dcc0 con 0x7fb4ec1075c0 2024-09-19T04:54:18.363 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.668+0000 7fb4eb400640 1 --2- 172.21.15.43:0/1669763046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4ec1075c0 0x7fb4ec19d780 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc0097f0 tx=0x7fb4dc030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:18.364 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.668+0000 7fb4e8c00640 1 -- 172.21.15.43:0/1669763046 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4dc037040 con 0x7fb4ec1075c0 2024-09-19T04:54:18.364 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.668+0000 7fb4e8c00640 1 -- 172.21.15.43:0/1669763046 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb4dc040070 con 0x7fb4ec1075c0 2024-09-19T04:54:18.364 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.668+0000 7fb4e8c00640 1 -- 172.21.15.43:0/1669763046 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4dc03a390 con 0x7fb4ec1075c0 2024-09-19T04:54:18.364 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.668+0000 7fb4f0e00640 1 -- 172.21.15.43:0/1669763046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4ec19df50 con 0x7fb4ec1075c0 2024-09-19T04:54:18.364 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.669+0000 7fb4f0e00640 1 -- 172.21.15.43:0/1669763046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4ec1a0c40 con 0x7fb4ec1075c0 2024-09-19T04:54:18.364 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.669+0000 7fb4e8c00640 1 -- 172.21.15.43:0/1669763046 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb4dc03a5e0 con 0x7fb4ec1075c0 2024-09-19T04:54:18.364 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.669+0000 7fb4e8c00640 1 -- 172.21.15.43:0/1669763046 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb4dc045950 con 0x7fb4ec1075c0 2024-09-19T04:54:18.364 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.670+0000 7fb4f0e00640 1 -- 172.21.15.43:0/1669763046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4ec107a40 con 0x7fb4ec1075c0 2024-09-19T04:54:18.364 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.672+0000 7fb4e8c00640 1 -- 172.21.15.43:0/1669763046 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fb4dc045460 con 0x7fb4ec1075c0 2024-09-19T04:54:18.364 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.735+0000 7fb4f0e00640 1 -- 172.21.15.43:0/1669763046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fb4ec1a0e10 con 0x7fb4ec1075c0 2024-09-19T04:54:18.364 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.735+0000 7fb4e8c00640 1 -- 172.21.15.43:0/1669763046 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fb4dc03b430 con 0x7fb4ec1075c0 2024-09-19T04:54:18.364 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.736+0000 7fb4f0e00640 1 -- 172.21.15.43:0/1669763046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4ec1075c0 msgr2=0x7fb4ec19d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:18.365 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.736+0000 7fb4f0e00640 1 --2- 172.21.15.43:0/1669763046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4ec1075c0 0x7fb4ec19d780 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc0097f0 tx=0x7fb4dc030d80 comp rx=0 tx=0).stop 2024-09-19T04:54:18.365 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.736+0000 7fb4f0e00640 1 -- 172.21.15.43:0/1669763046 shutdown_connections 2024-09-19T04:54:18.365 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.736+0000 7fb4f0e00640 1 --2- 172.21.15.43:0/1669763046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4ec1075c0 0x7fb4ec19d780 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:18.365 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.736+0000 7fb4f0e00640 1 -- 172.21.15.43:0/1669763046 >> 172.21.15.43:0/1669763046 conn(0x7fb4ec102d90 msgr2=0x7fb4ec1038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:18.365 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.737+0000 7fb4f0e00640 1 -- 172.21.15.43:0/1669763046 shutdown_connections 2024-09-19T04:54:18.365 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:17.737+0000 7fb4f0e00640 1 -- 172.21.15.43:0/1669763046 wait complete. 2024-09-19T04:54:18.365 INFO:teuthology.orchestra.run.smithi043.stdout:Restarting the monitor... 2024-09-19T04:54:18.725 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:18 smithi043 systemd[1]: Stopping Ceph mon.a for 1456a810-7643-11ef-bceb-c7b262605968... 2024-09-19T04:54:19.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:18 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mon-a[30433]: 2024-09-19T04:54:18.722+0000 7fb769600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T04:54:19.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:18 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mon-a[30433]: 2024-09-19T04:54:18.722+0000 7fb769600640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-19T04:54:19.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:18 smithi043 podman[30678]: 2024-09-19 04:54:18.918837795 +0000 UTC m=+0.317282586 container died 07f242987487feae1db27d8dfdf5f669a4f24c332e6f280afbc1d0d5609adbd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mon-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-19T04:54:19.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:19 smithi043 podman[30678]: 2024-09-19 04:54:19.2951637 +0000 UTC m=+0.693608491 container cleanup 07f242987487feae1db27d8dfdf5f669a4f24c332e6f280afbc1d0d5609adbd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mon-a, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4) 2024-09-19T04:54:19.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:19 smithi043 bash[30678]: ceph-1456a810-7643-11ef-bceb-c7b262605968-mon-a 2024-09-19T04:54:20.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:19 smithi043 podman[30691]: 2024-09-19 04:54:19.563821126 +0000 UTC m=+0.636840289 container remove 07f242987487feae1db27d8dfdf5f669a4f24c332e6f280afbc1d0d5609adbd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mon-a, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-19T04:54:20.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:19 smithi043 systemd[1]: ceph-1456a810-7643-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-19T04:54:20.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:19 smithi043 systemd[1]: Stopped Ceph mon.a for 1456a810-7643-11ef-bceb-c7b262605968. 2024-09-19T04:54:20.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:19 smithi043 systemd[1]: ceph-1456a810-7643-11ef-bceb-c7b262605968@mon.a.service: Consumed 1.239s CPU time. 2024-09-19T04:54:20.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:19 smithi043 systemd[1]: Starting Ceph mon.a for 1456a810-7643-11ef-bceb-c7b262605968... 2024-09-19T04:54:20.307 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 podman[30781]: 2024-09-19 04:54:19.935652242 +0000 UTC m=+0.027186221 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:54:20.307 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 podman[30781]: 2024-09-19 04:54:20.063623913 +0000 UTC m=+0.155157882 container create cd395eb8a2c34a2d7fedb2538263765d6b841bfe0aeab36999c549b57daf5a67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mon-a, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-19T04:54:20.307 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 podman[30781]: 2024-09-19 04:54:20.265773497 +0000 UTC m=+0.357307488 container init cd395eb8a2c34a2d7fedb2538263765d6b841bfe0aeab36999c549b57daf5a67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mon-a, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-19T04:54:20.307 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 podman[30781]: 2024-09-19 04:54:20.272440982 +0000 UTC m=+0.363974952 container start cd395eb8a2c34a2d7fedb2538263765d6b841bfe0aeab36999c549b57daf5a67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mon-a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-19T04:54:20.367 INFO:teuthology.orchestra.run.smithi043.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-19T04:54:20.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: set uid:gid to 167:167 (ceph:ceph) 2024-09-19T04:54:20.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable), process ceph-mon, pid 2 2024-09-19T04:54:20.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: pidfile_write: ignore empty --pid-file 2024-09-19T04:54:20.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: load: jerasure load: lrc 2024-09-19T04:54:20.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: RocksDB version: 7.9.2 2024-09-19T04:54:20.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Git sha 0 2024-09-19T04:54:20.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Compile date 2024-09-18 16:29:48 2024-09-19T04:54:20.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: DB SUMMARY 2024-09-19T04:54:20.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: DB Session ID: 4RVZP09GFNPL5RRTL1U8 2024-09-19T04:54:20.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: CURRENT file: CURRENT 2024-09-19T04:54:20.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: IDENTITY file: IDENTITY 2024-09-19T04:54:20.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-19T04:54:20.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-19T04:54:20.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105836 ; 2024-09-19T04:54:20.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.error_if_exists: 0 2024-09-19T04:54:20.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.create_if_missing: 0 2024-09-19T04:54:20.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.paranoid_checks: 1 2024-09-19T04:54:20.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-19T04:54:20.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-19T04:54:20.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-19T04:54:20.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.env: 0x56221098e940 2024-09-19T04:54:20.613 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.fs: PosixFileSystem 2024-09-19T04:54:20.613 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.info_log: 0x562234b116a0 2024-09-19T04:54:20.613 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.max_file_opening_threads: 16 2024-09-19T04:54:20.613 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.statistics: (nil) 2024-09-19T04:54:20.613 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.use_fsync: 0 2024-09-19T04:54:20.613 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.max_log_file_size: 0 2024-09-19T04:54:20.613 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-19T04:54:20.613 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-19T04:54:20.613 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.keep_log_file_num: 1000 2024-09-19T04:54:20.613 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.recycle_log_file_num: 0 2024-09-19T04:54:20.613 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.allow_fallocate: 1 2024-09-19T04:54:20.613 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.allow_mmap_reads: 0 2024-09-19T04:54:20.613 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.allow_mmap_writes: 0 2024-09-19T04:54:20.614 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.use_direct_reads: 0 2024-09-19T04:54:20.614 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-19T04:54:20.614 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.create_missing_column_families: 0 2024-09-19T04:54:20.614 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.db_log_dir: 2024-09-19T04:54:20.614 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.wal_dir: 2024-09-19T04:54:20.614 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-19T04:54:20.614 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-19T04:54:20.614 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-19T04:54:20.614 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-19T04:54:20.614 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-19T04:54:20.614 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-19T04:54:20.614 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.advise_random_on_open: 1 2024-09-19T04:54:20.614 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.db_write_buffer_size: 0 2024-09-19T04:54:20.615 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.write_buffer_manager: 0x562234b20dc0 2024-09-19T04:54:20.615 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-19T04:54:20.615 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-19T04:54:20.615 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-19T04:54:20.615 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.rate_limiter: (nil) 2024-09-19T04:54:20.615 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-19T04:54:20.615 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.wal_recovery_mode: 2 2024-09-19T04:54:20.615 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.enable_thread_tracking: 0 2024-09-19T04:54:20.615 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.enable_pipelined_write: 0 2024-09-19T04:54:20.615 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.unordered_write: 0 2024-09-19T04:54:20.615 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-19T04:54:20.615 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-19T04:54:20.615 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-19T04:54:20.615 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-19T04:54:20.616 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.row_cache: None 2024-09-19T04:54:20.616 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.wal_filter: None 2024-09-19T04:54:20.616 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-19T04:54:20.616 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.allow_ingest_behind: 0 2024-09-19T04:54:20.616 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.two_write_queues: 0 2024-09-19T04:54:20.616 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.manual_wal_flush: 0 2024-09-19T04:54:20.616 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.wal_compression: 0 2024-09-19T04:54:20.616 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.atomic_flush: 0 2024-09-19T04:54:20.617 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-19T04:54:20.617 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-19T04:54:20.617 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-19T04:54:20.617 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.log_readahead_size: 0 2024-09-19T04:54:20.617 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-19T04:54:20.617 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.best_efforts_recovery: 0 2024-09-19T04:54:20.617 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-19T04:54:20.617 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-19T04:54:20.617 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.allow_data_in_errors: 0 2024-09-19T04:54:20.617 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.db_host_id: __hostname__ 2024-09-19T04:54:20.617 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-19T04:54:20.619 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.max_background_jobs: 2 2024-09-19T04:54:20.619 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.max_background_compactions: -1 2024-09-19T04:54:20.619 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.max_subcompactions: 1 2024-09-19T04:54:20.619 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-19T04:54:20.619 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-19T04:54:20.619 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-19T04:54:20.619 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.max_total_wal_size: 0 2024-09-19T04:54:20.619 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-19T04:54:20.619 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-19T04:54:20.619 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-19T04:54:20.619 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-19T04:54:20.620 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.max_open_files: -1 2024-09-19T04:54:20.620 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.bytes_per_sync: 0 2024-09-19T04:54:20.620 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-19T04:54:20.620 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-19T04:54:20.620 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.compaction_readahead_size: 0 2024-09-19T04:54:20.620 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.max_background_flushes: -1 2024-09-19T04:54:20.620 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Compression algorithms supported: 2024-09-19T04:54:20.620 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: kZSTD supported: 0 2024-09-19T04:54:20.620 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: kXpressCompression supported: 0 2024-09-19T04:54:20.620 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: kBZip2Compression supported: 0 2024-09-19T04:54:20.620 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-19T04:54:20.620 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: kLZ4Compression supported: 1 2024-09-19T04:54:20.620 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: kZlibCompression supported: 1 2024-09-19T04:54:20.621 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: kLZ4HCCompression supported: 1 2024-09-19T04:54:20.621 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: kSnappyCompression supported: 1 2024-09-19T04:54:20.621 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-19T04:54:20.621 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-19T04:54:20.621 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-19T04:54:20.621 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-19T04:54:20.621 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-19T04:54:20.621 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.merge_operator: 2024-09-19T04:54:20.621 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.compaction_filter: None 2024-09-19T04:54:20.621 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.compaction_filter_factory: None 2024-09-19T04:54:20.621 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.sst_partitioner_factory: None 2024-09-19T04:54:20.621 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-19T04:54:20.621 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-19T04:54:20.622 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562234b11300) 2024-09-19T04:54:20.622 INFO:journalctl@ceph.mon.a.smithi043.stdout: cache_index_and_filter_blocks: 1 2024-09-19T04:54:20.622 INFO:journalctl@ceph.mon.a.smithi043.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-19T04:54:20.622 INFO:journalctl@ceph.mon.a.smithi043.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-19T04:54:20.622 INFO:journalctl@ceph.mon.a.smithi043.stdout: pin_top_level_index_and_filter: 1 2024-09-19T04:54:20.622 INFO:journalctl@ceph.mon.a.smithi043.stdout: index_type: 0 2024-09-19T04:54:20.622 INFO:journalctl@ceph.mon.a.smithi043.stdout: data_block_index_type: 0 2024-09-19T04:54:20.622 INFO:journalctl@ceph.mon.a.smithi043.stdout: index_shortening: 1 2024-09-19T04:54:20.622 INFO:journalctl@ceph.mon.a.smithi043.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-19T04:54:20.622 INFO:journalctl@ceph.mon.a.smithi043.stdout: checksum: 4 2024-09-19T04:54:20.622 INFO:journalctl@ceph.mon.a.smithi043.stdout: no_block_cache: 0 2024-09-19T04:54:20.622 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_cache: 0x562234b051f0 2024-09-19T04:54:20.622 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_cache_name: BinnedLRUCache 2024-09-19T04:54:20.623 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_cache_options: 2024-09-19T04:54:20.623 INFO:journalctl@ceph.mon.a.smithi043.stdout: capacity : 536870912 2024-09-19T04:54:20.623 INFO:journalctl@ceph.mon.a.smithi043.stdout: num_shard_bits : 4 2024-09-19T04:54:20.623 INFO:journalctl@ceph.mon.a.smithi043.stdout: strict_capacity_limit : 0 2024-09-19T04:54:20.623 INFO:journalctl@ceph.mon.a.smithi043.stdout: high_pri_pool_ratio: 0.000 2024-09-19T04:54:20.623 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_cache_compressed: (nil) 2024-09-19T04:54:20.623 INFO:journalctl@ceph.mon.a.smithi043.stdout: persistent_cache: (nil) 2024-09-19T04:54:20.623 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_size: 4096 2024-09-19T04:54:20.623 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_size_deviation: 10 2024-09-19T04:54:20.623 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_restart_interval: 16 2024-09-19T04:54:20.623 INFO:journalctl@ceph.mon.a.smithi043.stdout: index_block_restart_interval: 1 2024-09-19T04:54:20.623 INFO:journalctl@ceph.mon.a.smithi043.stdout: metadata_block_size: 4096 2024-09-19T04:54:20.623 INFO:journalctl@ceph.mon.a.smithi043.stdout: partition_filters: 0 2024-09-19T04:54:20.623 INFO:journalctl@ceph.mon.a.smithi043.stdout: use_delta_encoding: 1 2024-09-19T04:54:20.624 INFO:journalctl@ceph.mon.a.smithi043.stdout: filter_policy: bloomfilter 2024-09-19T04:54:20.624 INFO:journalctl@ceph.mon.a.smithi043.stdout: whole_key_filtering: 1 2024-09-19T04:54:20.624 INFO:journalctl@ceph.mon.a.smithi043.stdout: verify_compression: 0 2024-09-19T04:54:20.624 INFO:journalctl@ceph.mon.a.smithi043.stdout: read_amp_bytes_per_bit: 0 2024-09-19T04:54:20.624 INFO:journalctl@ceph.mon.a.smithi043.stdout: format_version: 5 2024-09-19T04:54:20.624 INFO:journalctl@ceph.mon.a.smithi043.stdout: enable_index_compression: 1 2024-09-19T04:54:20.624 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_align: 0 2024-09-19T04:54:20.624 INFO:journalctl@ceph.mon.a.smithi043.stdout: max_auto_readahead_size: 262144 2024-09-19T04:54:20.624 INFO:journalctl@ceph.mon.a.smithi043.stdout: prepopulate_block_cache: 0 2024-09-19T04:54:20.624 INFO:journalctl@ceph.mon.a.smithi043.stdout: initial_auto_readahead_size: 8192 2024-09-19T04:54:20.624 INFO:journalctl@ceph.mon.a.smithi043.stdout: num_file_reads_for_auto_readahead: 2 2024-09-19T04:54:20.624 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.write_buffer_size: 33554432 2024-09-19T04:54:20.624 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.max_write_buffer_number: 2 2024-09-19T04:54:20.624 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.compression: NoCompression 2024-09-19T04:54:20.625 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.bottommost_compression: Disabled 2024-09-19T04:54:20.625 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.prefix_extractor: nullptr 2024-09-19T04:54:20.625 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-19T04:54:20.625 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.num_levels: 7 2024-09-19T04:54:20.625 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-19T04:54:20.625 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-19T04:54:20.625 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-19T04:54:20.625 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-19T04:54:20.625 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-19T04:54:20.625 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-19T04:54:20.625 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-19T04:54:20.625 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-19T04:54:20.625 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-19T04:54:20.625 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-19T04:54:20.626 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-19T04:54:20.626 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-19T04:54:20.626 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-19T04:54:20.626 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.compression_opts.level: 32767 2024-09-19T04:54:20.626 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.compression_opts.strategy: 0 2024-09-19T04:54:20.626 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-19T04:54:20.626 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-19T04:54:20.626 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-19T04:54:20.626 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-19T04:54:20.626 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.compression_opts.enabled: false 2024-09-19T04:54:20.626 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-19T04:54:20.626 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-19T04:54:20.626 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-19T04:54:20.627 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-19T04:54:20.627 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.target_file_size_base: 67108864 2024-09-19T04:54:20.627 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-19T04:54:20.627 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-19T04:54:20.627 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-19T04:54:20.627 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-19T04:54:20.627 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-19T04:54:20.627 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-19T04:54:20.627 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-19T04:54:20.627 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-19T04:54:20.627 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-19T04:54:20.627 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-19T04:54:20.627 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-19T04:54:20.627 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-19T04:54:20.628 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-19T04:54:20.628 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-19T04:54:20.628 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.arena_block_size: 1048576 2024-09-19T04:54:20.628 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-19T04:54:20.628 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-19T04:54:20.628 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.disable_auto_compactions: 0 2024-09-19T04:54:20.628 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-19T04:54:20.628 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-19T04:54:20.628 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-19T04:54:20.628 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-19T04:54:20.628 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-19T04:54:20.628 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-19T04:54:20.628 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-19T04:54:20.629 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-19T04:54:20.629 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-19T04:54:20.629 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-19T04:54:20.629 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-19T04:54:20.629 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.inplace_update_support: 0 2024-09-19T04:54:20.629 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-19T04:54:20.629 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-19T04:54:20.629 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-19T04:54:20.629 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-19T04:54:20.629 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.bloom_locality: 0 2024-09-19T04:54:20.629 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.max_successive_merges: 0 2024-09-19T04:54:20.629 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-19T04:54:20.629 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.paranoid_file_checks: 0 2024-09-19T04:54:20.630 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.force_consistency_checks: 1 2024-09-19T04:54:20.630 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.report_bg_io_stats: 0 2024-09-19T04:54:20.630 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.ttl: 2592000 2024-09-19T04:54:20.630 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-19T04:54:20.630 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-19T04:54:20.630 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-19T04:54:20.630 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.enable_blob_files: false 2024-09-19T04:54:20.630 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.min_blob_size: 0 2024-09-19T04:54:20.630 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.blob_file_size: 268435456 2024-09-19T04:54:20.630 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-19T04:54:20.630 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-19T04:54:20.630 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-19T04:54:20.631 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-19T04:54:20.631 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-19T04:54:20.631 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.blob_file_starting_level: 0 2024-09-19T04:54:20.631 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-19T04:54:20.631 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-19T04:54:20.631 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-19T04:54:20.631 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e8ee601b-aa56-4568-b82d-4fbfc6476149 2024-09-19T04:54:20.631 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726721660314872, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-19T04:54:20.631 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-19T04:54:20.631 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726721660316580, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100845, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 330, "table_properties": {"data_size": 98730, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14431, "raw_average_key_size": 49, "raw_value_size": 90930, "raw_average_value_size": 314, "num_data_blocks": 12, "num_entries": 289, "num_filter_entries": 289, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726721660, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e8ee601b-aa56-4568-b82d-4fbfc6476149", "db_session_id": "4RVZP09GFNPL5RRTL1U8", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-19T04:54:20.631 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726721660316795, "job": 1, "event": "recovery_finished"} 2024-09-19T04:54:20.631 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-19T04:54:20.632 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-19T04:54:20.632 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562234b34e00 2024-09-19T04:54:20.632 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: DB pointer 0x562234c30000 2024-09-19T04:54:20.632 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-19T04:54:20.632 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-19T04:54:20.632 INFO:journalctl@ceph.mon.a.smithi043.stdout: ** DB Stats ** 2024-09-19T04:54:20.632 INFO:journalctl@ceph.mon.a.smithi043.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T04:54:20.632 INFO:journalctl@ceph.mon.a.smithi043.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-19T04:54:20.632 INFO:journalctl@ceph.mon.a.smithi043.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T04:54:20.632 INFO:journalctl@ceph.mon.a.smithi043.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T04:54:20.632 INFO:journalctl@ceph.mon.a.smithi043.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-19T04:54:20.632 INFO:journalctl@ceph.mon.a.smithi043.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T04:54:20.633 INFO:journalctl@ceph.mon.a.smithi043.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T04:54:20.633 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-09-19T04:54:20.633 INFO:journalctl@ceph.mon.a.smithi043.stdout: ** Compaction Stats [default] ** 2024-09-19T04:54:20.633 INFO:journalctl@ceph.mon.a.smithi043.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T04:54:20.633 INFO:journalctl@ceph.mon.a.smithi043.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-19T04:54:20.633 INFO:journalctl@ceph.mon.a.smithi043.stdout: L0 2/0 100.34 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 91.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T04:54:20.633 INFO:journalctl@ceph.mon.a.smithi043.stdout: Sum 2/0 100.34 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 91.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T04:54:20.633 INFO:journalctl@ceph.mon.a.smithi043.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 91.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T04:54:20.633 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-09-19T04:54:20.633 INFO:journalctl@ceph.mon.a.smithi043.stdout: ** Compaction Stats [default] ** 2024-09-19T04:54:20.633 INFO:journalctl@ceph.mon.a.smithi043.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T04:54:20.633 INFO:journalctl@ceph.mon.a.smithi043.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-19T04:54:20.633 INFO:journalctl@ceph.mon.a.smithi043.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 91.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T04:54:20.634 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-09-19T04:54:20.634 INFO:journalctl@ceph.mon.a.smithi043.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-19T04:54:20.634 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-09-19T04:54:20.634 INFO:journalctl@ceph.mon.a.smithi043.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T04:54:20.634 INFO:journalctl@ceph.mon.a.smithi043.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-19T04:54:20.634 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-19T04:54:20.634 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-19T04:54:20.634 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-19T04:54:20.634 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-19T04:54:20.634 INFO:journalctl@ceph.mon.a.smithi043.stdout: Cumulative compaction: 0.00 GB write, 16.87 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T04:54:20.634 INFO:journalctl@ceph.mon.a.smithi043.stdout: Interval compaction: 0.00 GB write, 16.87 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T04:54:20.634 INFO:journalctl@ceph.mon.a.smithi043.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-19T04:54:20.634 INFO:journalctl@ceph.mon.a.smithi043.stdout: Block cache BinnedLRUCache@0x562234b051f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-19T04:54:20.634 INFO:journalctl@ceph.mon.a.smithi043.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-19T04:54:20.635 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-09-19T04:54:20.635 INFO:journalctl@ceph.mon.a.smithi043.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-19T04:54:20.635 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: starting mon.a rank 0 at public addrs [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] at bind addrs [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 1456a810-7643-11ef-bceb-c7b262605968 2024-09-19T04:54:20.635 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: mon.a@-1(???) e1 preinit fsid 1456a810-7643-11ef-bceb-c7b262605968 2024-09-19T04:54:20.635 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: mon.a@-1(???).mds e1 new map 2024-09-19T04:54:20.635 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: mon.a@-1(???).mds e1 print_map 2024-09-19T04:54:20.635 INFO:journalctl@ceph.mon.a.smithi043.stdout: e1 2024-09-19T04:54:20.635 INFO:journalctl@ceph.mon.a.smithi043.stdout: btime 2024-09-19T04:54:14:706422+0000 2024-09-19T04:54:20.635 INFO:journalctl@ceph.mon.a.smithi043.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T04:54:20.635 INFO:journalctl@ceph.mon.a.smithi043.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T04:54:20.635 INFO:journalctl@ceph.mon.a.smithi043.stdout: legacy client fscid: -1 2024-09-19T04:54:20.635 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-09-19T04:54:20.636 INFO:journalctl@ceph.mon.a.smithi043.stdout: No filesystems configured 2024-09-19T04:54:20.636 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-19T04:54:20.636 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T04:54:20.636 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T04:54:20.636 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T04:54:20.636 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-19T04:54:20.636 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 bash[30781]: cd395eb8a2c34a2d7fedb2538263765d6b841bfe0aeab36999c549b57daf5a67 2024-09-19T04:54:20.636 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 systemd[1]: Started Ceph mon.a for 1456a810-7643-11ef-bceb-c7b262605968. 2024-09-19T04:54:20.636 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T04:54:20.636 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: monmap epoch 1 2024-09-19T04:54:20.636 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: fsid 1456a810-7643-11ef-bceb-c7b262605968 2024-09-19T04:54:20.636 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: last_changed 2024-09-19T04:54:11.079879+0000 2024-09-19T04:54:20.636 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: created 2024-09-19T04:54:11.079879+0000 2024-09-19T04:54:20.637 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: min_mon_release 19 (squid) 2024-09-19T04:54:20.637 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: election_strategy: 1 2024-09-19T04:54:20.637 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-09-19T04:54:20.637 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: fsmap 2024-09-19T04:54:20.637 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: osdmap e1: 0 total, 0 up, 0 in 2024-09-19T04:54:20.637 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:20 smithi043 ceph-mon[30800]: mgrmap e1: no daemons active 2024-09-19T04:54:21.415 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.823+0000 7f19d6a00640 1 Processor -- start 2024-09-19T04:54:21.415 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.823+0000 7f19d6a00640 1 -- start start 2024-09-19T04:54:21.415 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.823+0000 7f19d6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19d0107560 0x7f19d0107960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:21.415 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.823+0000 7f19d6a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f19d0107f30 con 0x7f19d0107560 2024-09-19T04:54:21.415 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.824+0000 7f19d5600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19d0107560 0x7f19d0107960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:21.416 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.824+0000 7f19d5600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19d0107560 0x7f19d0107960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53330/0 (socket says 172.21.15.43:53330) 2024-09-19T04:54:21.416 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.824+0000 7f19d5600640 1 -- 172.21.15.43:0/1102617101 learned_addr learned my addr 172.21.15.43:0/1102617101 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:54:21.416 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.824+0000 7f19d5600640 1 -- 172.21.15.43:0/1102617101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19d0108710 con 0x7f19d0107560 2024-09-19T04:54:21.416 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.827+0000 7f19d5600640 1 --2- 172.21.15.43:0/1102617101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19d0107560 0x7f19d0107960 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f19c4009e80 tx=0x7f19c402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=109a08909ee8bdf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:21.416 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.827+0000 7f19cfe00640 1 -- 172.21.15.43:0/1102617101 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19c4036070 con 0x7f19d0107560 2024-09-19T04:54:21.416 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.827+0000 7f19cfe00640 1 -- 172.21.15.43:0/1102617101 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f19c40318c0 con 0x7f19d0107560 2024-09-19T04:54:21.416 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.827+0000 7f19cfe00640 1 -- 172.21.15.43:0/1102617101 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19c4007ab0 con 0x7f19d0107560 2024-09-19T04:54:21.416 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.827+0000 7f19d6a00640 1 -- 172.21.15.43:0/1102617101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19d0107560 msgr2=0x7f19d0107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:21.416 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.827+0000 7f19d6a00640 1 --2- 172.21.15.43:0/1102617101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19d0107560 0x7f19d0107960 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f19c4009e80 tx=0x7f19c402a7a0 comp rx=0 tx=0).stop 2024-09-19T04:54:21.416 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.828+0000 7f19d6a00640 1 -- 172.21.15.43:0/1102617101 shutdown_connections 2024-09-19T04:54:21.416 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.828+0000 7f19d6a00640 1 --2- 172.21.15.43:0/1102617101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19d0107560 0x7f19d0107960 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:21.417 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.828+0000 7f19d6a00640 1 -- 172.21.15.43:0/1102617101 >> 172.21.15.43:0/1102617101 conn(0x7f19d0102cf0 msgr2=0x7f19d0105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:21.417 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.828+0000 7f19d6a00640 1 -- 172.21.15.43:0/1102617101 shutdown_connections 2024-09-19T04:54:21.417 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.828+0000 7f19d6a00640 1 -- 172.21.15.43:0/1102617101 wait complete. 2024-09-19T04:54:21.417 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.829+0000 7f19d6a00640 1 Processor -- start 2024-09-19T04:54:21.417 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.829+0000 7f19d6a00640 1 -- start start 2024-09-19T04:54:21.417 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.829+0000 7f19d6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19d0107560 0x7f19d019d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:21.417 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.829+0000 7f19d6a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f19d010bac0 con 0x7f19d0107560 2024-09-19T04:54:21.417 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.829+0000 7f19d5600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19d0107560 0x7f19d019d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:21.417 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.829+0000 7f19d5600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19d0107560 0x7f19d019d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53332/0 (socket says 172.21.15.43:53332) 2024-09-19T04:54:21.417 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.829+0000 7f19d5600640 1 -- 172.21.15.43:0/2797753031 learned_addr learned my addr 172.21.15.43:0/2797753031 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:54:21.417 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.830+0000 7f19d5600640 1 -- 172.21.15.43:0/2797753031 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19d019dca0 con 0x7f19d0107560 2024-09-19T04:54:21.417 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.830+0000 7f19d5600640 1 --2- 172.21.15.43:0/2797753031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19d0107560 0x7f19d019d760 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f19c402acc0 tx=0x7f19c4031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:21.418 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.830+0000 7f19ce000640 1 -- 172.21.15.43:0/2797753031 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19c4036050 con 0x7f19d0107560 2024-09-19T04:54:21.418 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.831+0000 7f19d6a00640 1 -- 172.21.15.43:0/2797753031 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19d019df30 con 0x7f19d0107560 2024-09-19T04:54:21.418 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.831+0000 7f19ce000640 1 -- 172.21.15.43:0/2797753031 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f19c4037040 con 0x7f19d0107560 2024-09-19T04:54:21.418 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.831+0000 7f19ce000640 1 -- 172.21.15.43:0/2797753031 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19c403d8c0 con 0x7f19d0107560 2024-09-19T04:54:21.418 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.831+0000 7f19d6a00640 1 -- 172.21.15.43:0/2797753031 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19d01a0c20 con 0x7f19d0107560 2024-09-19T04:54:21.418 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.831+0000 7f19ce000640 1 -- 172.21.15.43:0/2797753031 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f19c4044020 con 0x7f19d0107560 2024-09-19T04:54:21.418 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.831+0000 7f19ce000640 1 -- 172.21.15.43:0/2797753031 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f19c4042020 con 0x7f19d0107560 2024-09-19T04:54:21.418 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.832+0000 7f19d6a00640 1 -- 172.21.15.43:0/2797753031 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19d010bc40 con 0x7f19d0107560 2024-09-19T04:54:21.418 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.835+0000 7f19ce000640 1 -- 172.21.15.43:0/2797753031 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f19c404b020 con 0x7f19d0107560 2024-09-19T04:54:21.418 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.897+0000 7f19d6a00640 1 -- 172.21.15.43:0/2797753031 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f19d01a1210 con 0x7f19d0107560 2024-09-19T04:54:21.418 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.899+0000 7f19ce000640 1 -- 172.21.15.43:0/2797753031 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f19c40073b0 con 0x7f19d0107560 2024-09-19T04:54:21.418 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.900+0000 7f19d6a00640 1 -- 172.21.15.43:0/2797753031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19d0107560 msgr2=0x7f19d019d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:21.418 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.901+0000 7f19d6a00640 1 --2- 172.21.15.43:0/2797753031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19d0107560 0x7f19d019d760 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f19c402acc0 tx=0x7f19c4031ca0 comp rx=0 tx=0).stop 2024-09-19T04:54:21.419 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.901+0000 7f19d6a00640 1 -- 172.21.15.43:0/2797753031 shutdown_connections 2024-09-19T04:54:21.419 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.901+0000 7f19d6a00640 1 --2- 172.21.15.43:0/2797753031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19d0107560 0x7f19d019d760 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:21.419 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.901+0000 7f19d6a00640 1 -- 172.21.15.43:0/2797753031 >> 172.21.15.43:0/2797753031 conn(0x7f19d0102cf0 msgr2=0x7f19d0103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:21.419 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.901+0000 7f19d6a00640 1 -- 172.21.15.43:0/2797753031 shutdown_connections 2024-09-19T04:54:21.419 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:20.901+0000 7f19d6a00640 1 -- 172.21.15.43:0/2797753031 wait complete. 2024-09-19T04:54:21.419 INFO:teuthology.orchestra.run.smithi043.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-19T04:54:21.438 INFO:teuthology.orchestra.run.smithi043.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-19T04:54:21.438 INFO:teuthology.orchestra.run.smithi043.stdout:Creating mgr... 2024-09-19T04:54:21.439 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-19T04:54:21.440 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-19T04:54:21.774 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1456a810-7643-11ef-bceb-c7b262605968@mgr.x 2024-09-19T04:54:21.774 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1456a810-7643-11ef-bceb-c7b262605968@mgr.x.service: Unit ceph-1456a810-7643-11ef-bceb-c7b262605968@mgr.x.service not loaded. 2024-09-19T04:54:22.011 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1456a810-7643-11ef-bceb-c7b262605968.target.wants/ceph-1456a810-7643-11ef-bceb-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-1456a810-7643-11ef-bceb-c7b262605968@.service. 2024-09-19T04:54:22.041 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:21 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/2797753031' entity='client.admin' 2024-09-19T04:54:22.717 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:22 smithi043 podman[31010]: 2024-09-19 04:54:22.257899494 +0000 UTC m=+0.027006099 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:54:22.717 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:22 smithi043 podman[31010]: 2024-09-19 04:54:22.716203137 +0000 UTC m=+0.485309731 container create 185a2dbed7db0f598ffc0d6685cfceb3b2ccf3610b2e2972c06173250d00123d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, ceph=True) 2024-09-19T04:54:22.986 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:22 smithi043 podman[31010]: 2024-09-19 04:54:22.92742582 +0000 UTC m=+0.696532415 container init 185a2dbed7db0f598ffc0d6685cfceb3b2ccf3610b2e2972c06173250d00123d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True) 2024-09-19T04:54:22.986 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:22 smithi043 podman[31010]: 2024-09-19 04:54:22.93425275 +0000 UTC m=+0.703359344 container start 185a2dbed7db0f598ffc0d6685cfceb3b2ccf3610b2e2972c06173250d00123d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T04:54:23.017 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T04:54:23.017 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout disabled 2024-09-19T04:54:23.033 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T04:54:23.033 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-09-19T04:54:23.033 INFO:teuthology.orchestra.run.smithi043.stdout:firewalld.service is not enabled 2024-09-19T04:54:23.033 INFO:teuthology.orchestra.run.smithi043.stdout:Not possible to enable service . firewalld.service is not available 2024-09-19T04:54:23.053 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T04:54:23.053 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout disabled 2024-09-19T04:54:23.068 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T04:54:23.069 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-09-19T04:54:23.069 INFO:teuthology.orchestra.run.smithi043.stdout:firewalld.service is not enabled 2024-09-19T04:54:23.069 INFO:teuthology.orchestra.run.smithi043.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-19T04:54:23.069 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mgr to start... 2024-09-19T04:54:23.069 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mgr... 2024-09-19T04:54:23.236 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:22 smithi043 bash[31010]: 185a2dbed7db0f598ffc0d6685cfceb3b2ccf3610b2e2972c06173250d00123d 2024-09-19T04:54:23.237 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:22 smithi043 systemd[1]: Started Ceph mgr.x for 1456a810-7643-11ef-bceb-c7b262605968. 2024-09-19T04:54:23.237 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:23 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:23.137+0000 7f98f85e1040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T04:54:23.237 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:23 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:23.209+0000 7f98f85e1040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T04:54:23.502 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:23 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:23.412+0000 7f98f85e1040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T04:54:23.502 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:23 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:23.499+0000 7f98f85e1040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T04:54:23.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:23 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/1390072621' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T04:54:23.780 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:23 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:23.691+0000 7f98f85e1040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:54:24.279 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:23 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:23.873+0000 7f98f85e1040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T04:54:24.390 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-19T04:54:24.393 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-09-19T04:54:24.393 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsid": "1456a810-7643-11ef-bceb-c7b262605968", 2024-09-19T04:54:24.393 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T04:54:24.393 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T04:54:24.393 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T04:54:24.393 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T04:54:24.394 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:54:24.394 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T04:54:24.394 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T04:54:24.394 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 0 2024-09-19T04:54:24.394 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-19T04:54:24.394 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T04:54:24.395 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T04:54:24.395 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-19T04:54:24.395 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-09-19T04:54:24.395 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T04:54:24.395 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:54:24.395 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T04:54:24.395 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T04:54:24.396 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:54:24.396 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T04:54:24.396 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:54:24.396 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T04:54:24.396 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T04:54:24.405 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T04:54:24.405 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T04:54:24.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T04:54:24.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T04:54:24.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:54:24.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T04:54:24.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T04:54:24.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T04:54:24.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T04:54:24.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T04:54:24.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T04:54:24.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T04:54:24.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T04:54:24.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T04:54:24.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:54:24.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T04:54:24.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:54:24.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T04:54:14:706422+0000", 2024-09-19T04:54:24.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T04:54:24.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T04:54:24.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:54:24.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T04:54:24.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-19T04:54:24.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T04:54:24.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T04:54:24.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T04:54:24.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T04:54:24.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T04:54:24.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-19T04:54:24.408 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T04:54:24.408 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:54:24.408 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T04:54:24.408 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:54:24.408 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T04:54:14.707187+0000", 2024-09-19T04:54:24.408 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T04:54:24.408 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:54:24.408 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T04:54:24.408 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-09-19T04:54:24.408 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.574+0000 7fb708e00640 1 Processor -- start 2024-09-19T04:54:24.408 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.574+0000 7fb708e00640 1 -- start start 2024-09-19T04:54:24.408 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.575+0000 7fb708e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7041099c0 0x7fb704109dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:24.408 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.575+0000 7fb708e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb70410a390 con 0x7fb7041099c0 2024-09-19T04:54:24.409 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.575+0000 7fb701e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7041099c0 0x7fb704109dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:24.409 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.575+0000 7fb701e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7041099c0 0x7fb704109dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53338/0 (socket says 172.21.15.43:53338) 2024-09-19T04:54:24.409 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.575+0000 7fb701e00640 1 -- 172.21.15.43:0/1452883201 learned_addr learned my addr 172.21.15.43:0/1452883201 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:54:24.409 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.576+0000 7fb701e00640 1 -- 172.21.15.43:0/1452883201 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb70410ab00 con 0x7fb7041099c0 2024-09-19T04:54:24.409 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.577+0000 7fb701e00640 1 --2- 172.21.15.43:0/1452883201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7041099c0 0x7fb704109dc0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4009820 tx=0x7fb6f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78888ae6b68e1fa8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:24.409 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.577+0000 7fb700a00640 1 -- 172.21.15.43:0/1452883201 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6f4037070 con 0x7fb7041099c0 2024-09-19T04:54:24.409 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.577+0000 7fb700a00640 1 -- 172.21.15.43:0/1452883201 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb6f4032900 con 0x7fb7041099c0 2024-09-19T04:54:24.409 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.577+0000 7fb700a00640 1 -- 172.21.15.43:0/1452883201 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6f4030850 con 0x7fb7041099c0 2024-09-19T04:54:24.409 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.578+0000 7fb708e00640 1 -- 172.21.15.43:0/1452883201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7041099c0 msgr2=0x7fb704109dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:24.409 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.578+0000 7fb708e00640 1 --2- 172.21.15.43:0/1452883201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7041099c0 0x7fb704109dc0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4009820 tx=0x7fb6f402a270 comp rx=0 tx=0).stop 2024-09-19T04:54:24.409 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.578+0000 7fb708e00640 1 -- 172.21.15.43:0/1452883201 shutdown_connections 2024-09-19T04:54:24.409 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.578+0000 7fb708e00640 1 --2- 172.21.15.43:0/1452883201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7041099c0 0x7fb704109dc0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:24.409 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.578+0000 7fb708e00640 1 -- 172.21.15.43:0/1452883201 >> 172.21.15.43:0/1452883201 conn(0x7fb70407ceb0 msgr2=0x7fb70407d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:24.410 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.578+0000 7fb708e00640 1 -- 172.21.15.43:0/1452883201 shutdown_connections 2024-09-19T04:54:24.410 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.578+0000 7fb708e00640 1 -- 172.21.15.43:0/1452883201 wait complete. 2024-09-19T04:54:24.410 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.579+0000 7fb708e00640 1 Processor -- start 2024-09-19T04:54:24.410 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.579+0000 7fb708e00640 1 -- start start 2024-09-19T04:54:24.410 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.579+0000 7fb708e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7041099c0 0x7fb70419d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:24.410 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.579+0000 7fb708e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb70410deb0 con 0x7fb7041099c0 2024-09-19T04:54:24.410 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.580+0000 7fb701e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7041099c0 0x7fb70419d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:24.410 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.580+0000 7fb701e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7041099c0 0x7fb70419d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53354/0 (socket says 172.21.15.43:53354) 2024-09-19T04:54:24.410 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.580+0000 7fb701e00640 1 -- 172.21.15.43:0/1390072621 learned_addr learned my addr 172.21.15.43:0/1390072621 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:54:24.410 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.580+0000 7fb701e00640 1 -- 172.21.15.43:0/1390072621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb70419de60 con 0x7fb7041099c0 2024-09-19T04:54:24.410 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.580+0000 7fb701e00640 1 --2- 172.21.15.43:0/1390072621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7041099c0 0x7fb70419d920 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb6f40097f0 tx=0x7fb6f4030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:24.410 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.581+0000 7fb6faa00640 1 -- 172.21.15.43:0/1390072621 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6f4042070 con 0x7fb7041099c0 2024-09-19T04:54:24.410 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.581+0000 7fb6faa00640 1 -- 172.21.15.43:0/1390072621 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb6f4032ca0 con 0x7fb7041099c0 2024-09-19T04:54:24.410 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.581+0000 7fb6faa00640 1 -- 172.21.15.43:0/1390072621 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6f4037040 con 0x7fb7041099c0 2024-09-19T04:54:24.411 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.581+0000 7fb708e00640 1 -- 172.21.15.43:0/1390072621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb70419e0f0 con 0x7fb7041099c0 2024-09-19T04:54:24.411 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.581+0000 7fb708e00640 1 -- 172.21.15.43:0/1390072621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7041a0de0 con 0x7fb7041099c0 2024-09-19T04:54:24.411 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.581+0000 7fb6faa00640 1 -- 172.21.15.43:0/1390072621 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb6f4045070 con 0x7fb7041099c0 2024-09-19T04:54:24.411 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.581+0000 7fb6faa00640 1 -- 172.21.15.43:0/1390072621 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb6f4047a30 con 0x7fb7041099c0 2024-09-19T04:54:24.411 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.582+0000 7fb708e00640 1 -- 172.21.15.43:0/1390072621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb70410e030 con 0x7fb7041099c0 2024-09-19T04:54:24.411 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.585+0000 7fb6faa00640 1 -- 172.21.15.43:0/1390072621 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fb6f403da60 con 0x7fb7041099c0 2024-09-19T04:54:24.411 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.651+0000 7fb708e00640 1 -- 172.21.15.43:0/1390072621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb7041a12c0 con 0x7fb7041099c0 2024-09-19T04:54:24.411 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.651+0000 7fb6faa00640 1 -- 172.21.15.43:0/1390072621 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fb6f403d430 con 0x7fb7041099c0 2024-09-19T04:54:24.411 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.652+0000 7fb708e00640 1 -- 172.21.15.43:0/1390072621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7041099c0 msgr2=0x7fb70419d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:24.411 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.652+0000 7fb708e00640 1 --2- 172.21.15.43:0/1390072621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7041099c0 0x7fb70419d920 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb6f40097f0 tx=0x7fb6f4030ec0 comp rx=0 tx=0).stop 2024-09-19T04:54:24.411 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.653+0000 7fb708e00640 1 -- 172.21.15.43:0/1390072621 shutdown_connections 2024-09-19T04:54:24.411 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.653+0000 7fb708e00640 1 --2- 172.21.15.43:0/1390072621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7041099c0 0x7fb70419d920 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:24.411 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.653+0000 7fb708e00640 1 -- 172.21.15.43:0/1390072621 >> 172.21.15.43:0/1390072621 conn(0x7fb70407ceb0 msgr2=0x7fb704107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:24.412 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.653+0000 7fb708e00640 1 -- 172.21.15.43:0/1390072621 shutdown_connections 2024-09-19T04:54:24.412 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:23.653+0000 7fb708e00640 1 -- 172.21.15.43:0/1390072621 wait complete. 2024-09-19T04:54:24.412 INFO:teuthology.orchestra.run.smithi043.stdout:mgr not available, waiting (1/15)... 2024-09-19T04:54:24.530 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:24 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:24.277+0000 7f98f85e1040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T04:54:24.530 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:24 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:24.352+0000 7f98f85e1040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T04:54:24.878 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:24 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:24.578+0000 7f98f85e1040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T04:54:26.030 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:25 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:25.734+0000 7f98f85e1040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T04:54:26.030 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:25 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:25.817+0000 7f98f85e1040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T04:54:27.136 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:27 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/732456437' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T04:54:27.136 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:26 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:26.810+0000 7f98f85e1040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T04:54:27.137 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:26 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:26.956+0000 7f98f85e1040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T04:54:27.137 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:27 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:27.134+0000 7f98f85e1040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:54:27.401 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:27 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:27.217+0000 7f98f85e1040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T04:54:27.754 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:27 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:27.400+0000 7f98f85e1040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T04:54:27.755 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:27 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:27.485+0000 7f98f85e1040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T04:54:27.755 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:27 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:27.562+0000 7f98f85e1040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T04:54:27.755 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:27 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:27.752+0000 7f98f85e1040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T04:54:27.972 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-19T04:54:27.973 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-09-19T04:54:27.973 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsid": "1456a810-7643-11ef-bceb-c7b262605968", 2024-09-19T04:54:27.973 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T04:54:27.973 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T04:54:27.973 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T04:54:27.973 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T04:54:27.973 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:54:27.973 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T04:54:27.973 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T04:54:27.973 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 0 2024-09-19T04:54:27.973 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-19T04:54:27.973 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T04:54:27.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T04:54:27.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-19T04:54:27.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-09-19T04:54:27.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T04:54:27.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:54:27.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T04:54:27.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T04:54:27.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:54:27.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T04:54:27.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:54:27.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T04:54:27.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T04:54:27.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T04:54:27.982 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T04:54:27.982 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T04:54:27.982 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T04:54:27.982 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:54:27.982 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T04:54:27.982 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T04:54:27.982 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T04:54:27.982 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T04:54:27.982 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T04:54:27.982 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T04:54:27.983 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T04:54:27.983 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T04:54:27.983 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T04:54:27.983 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:54:27.983 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T04:54:27.983 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:54:27.983 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T04:54:14:706422+0000", 2024-09-19T04:54:27.983 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T04:54:27.983 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T04:54:27.983 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:54:27.983 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T04:54:27.983 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-19T04:54:27.983 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T04:54:27.983 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T04:54:27.984 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T04:54:27.984 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T04:54:27.984 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T04:54:27.984 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-19T04:54:27.984 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T04:54:27.984 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:54:27.984 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T04:54:27.984 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:54:27.984 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T04:54:14.707187+0000", 2024-09-19T04:54:27.984 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T04:54:27.984 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:54:27.984 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T04:54:27.984 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-09-19T04:54:27.984 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.879+0000 7f03d9600640 1 Processor -- start 2024-09-19T04:54:27.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.879+0000 7f03d9600640 1 -- start start 2024-09-19T04:54:27.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.879+0000 7f03d9600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03d41099c0 0x7f03d4109dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:27.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.879+0000 7f03d9600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f03d410a390 con 0x7f03d41099c0 2024-09-19T04:54:27.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.880+0000 7f03d2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03d41099c0 0x7f03d4109dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:27.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.880+0000 7f03d2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03d41099c0 0x7f03d4109dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40980/0 (socket says 172.21.15.43:40980) 2024-09-19T04:54:27.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.880+0000 7f03d2a00640 1 -- 172.21.15.43:0/638868114 learned_addr learned my addr 172.21.15.43:0/638868114 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:54:27.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.880+0000 7f03d2a00640 1 -- 172.21.15.43:0/638868114 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d410ab00 con 0x7f03d41099c0 2024-09-19T04:54:27.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.881+0000 7f03d2a00640 1 --2- 172.21.15.43:0/638868114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03d41099c0 0x7f03d4109dc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f03c4009820 tx=0x7f03c402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d84040456393fcb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:27.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.882+0000 7f03d1600640 1 -- 172.21.15.43:0/638868114 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03c4037070 con 0x7f03d41099c0 2024-09-19T04:54:27.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.882+0000 7f03d1600640 1 -- 172.21.15.43:0/638868114 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f03c4032900 con 0x7f03d41099c0 2024-09-19T04:54:27.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.882+0000 7f03d1600640 1 -- 172.21.15.43:0/638868114 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03c4030850 con 0x7f03d41099c0 2024-09-19T04:54:27.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.882+0000 7f03d9600640 1 -- 172.21.15.43:0/638868114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03d41099c0 msgr2=0x7f03d4109dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:27.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.882+0000 7f03d9600640 1 --2- 172.21.15.43:0/638868114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03d41099c0 0x7f03d4109dc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f03c4009820 tx=0x7f03c402a270 comp rx=0 tx=0).stop 2024-09-19T04:54:27.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.882+0000 7f03d9600640 1 -- 172.21.15.43:0/638868114 shutdown_connections 2024-09-19T04:54:27.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.882+0000 7f03d9600640 1 --2- 172.21.15.43:0/638868114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03d41099c0 0x7f03d4109dc0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:27.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.882+0000 7f03d9600640 1 -- 172.21.15.43:0/638868114 >> 172.21.15.43:0/638868114 conn(0x7f03d407ceb0 msgr2=0x7f03d407d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:27.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.883+0000 7f03d9600640 1 -- 172.21.15.43:0/638868114 shutdown_connections 2024-09-19T04:54:27.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.883+0000 7f03d9600640 1 -- 172.21.15.43:0/638868114 wait complete. 2024-09-19T04:54:27.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.884+0000 7f03d9600640 1 Processor -- start 2024-09-19T04:54:27.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.884+0000 7f03d9600640 1 -- start start 2024-09-19T04:54:27.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.885+0000 7f03d9600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03d41099c0 0x7f03d419d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:27.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.885+0000 7f03d9600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f03d410deb0 con 0x7f03d41099c0 2024-09-19T04:54:27.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.885+0000 7f03d2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03d41099c0 0x7f03d419d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:27.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.885+0000 7f03d2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03d41099c0 0x7f03d419d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40984/0 (socket says 172.21.15.43:40984) 2024-09-19T04:54:27.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.885+0000 7f03d2a00640 1 -- 172.21.15.43:0/732456437 learned_addr learned my addr 172.21.15.43:0/732456437 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:54:27.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.885+0000 7f03d2a00640 1 -- 172.21.15.43:0/732456437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d419de60 con 0x7f03d41099c0 2024-09-19T04:54:27.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.885+0000 7f03d2a00640 1 --2- 172.21.15.43:0/732456437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03d41099c0 0x7f03d419d920 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f03c40097f0 tx=0x7f03c4030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:27.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.886+0000 7f03cb400640 1 -- 172.21.15.43:0/732456437 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03c4042070 con 0x7f03d41099c0 2024-09-19T04:54:27.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.886+0000 7f03cb400640 1 -- 172.21.15.43:0/732456437 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f03c4032ca0 con 0x7f03d41099c0 2024-09-19T04:54:27.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.886+0000 7f03cb400640 1 -- 172.21.15.43:0/732456437 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03c4037040 con 0x7f03d41099c0 2024-09-19T04:54:27.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.886+0000 7f03d9600640 1 -- 172.21.15.43:0/732456437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03d419e0f0 con 0x7f03d41099c0 2024-09-19T04:54:27.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.886+0000 7f03d9600640 1 -- 172.21.15.43:0/732456437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03d41a0de0 con 0x7f03d41099c0 2024-09-19T04:54:27.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.886+0000 7f03cb400640 1 -- 172.21.15.43:0/732456437 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f03c4045070 con 0x7f03d41099c0 2024-09-19T04:54:27.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.886+0000 7f03cb400640 1 -- 172.21.15.43:0/732456437 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f03c40463b0 con 0x7f03d41099c0 2024-09-19T04:54:27.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.887+0000 7f03d9600640 1 -- 172.21.15.43:0/732456437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03d410e030 con 0x7f03d41099c0 2024-09-19T04:54:27.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.890+0000 7f03cb400640 1 -- 172.21.15.43:0/732456437 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f03c4047440 con 0x7f03d41099c0 2024-09-19T04:54:27.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.956+0000 7f03d9600640 1 -- 172.21.15.43:0/732456437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f03d41a12c0 con 0x7f03d41099c0 2024-09-19T04:54:27.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.957+0000 7f03cb400640 1 -- 172.21.15.43:0/732456437 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f03c403da60 con 0x7f03d41099c0 2024-09-19T04:54:27.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.958+0000 7f03d9600640 1 -- 172.21.15.43:0/732456437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03d41099c0 msgr2=0x7f03d419d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:27.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.958+0000 7f03d9600640 1 --2- 172.21.15.43:0/732456437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03d41099c0 0x7f03d419d920 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f03c40097f0 tx=0x7f03c4030ec0 comp rx=0 tx=0).stop 2024-09-19T04:54:27.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.958+0000 7f03d9600640 1 -- 172.21.15.43:0/732456437 shutdown_connections 2024-09-19T04:54:27.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.958+0000 7f03d9600640 1 --2- 172.21.15.43:0/732456437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03d41099c0 0x7f03d419d920 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:27.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.958+0000 7f03d9600640 1 -- 172.21.15.43:0/732456437 >> 172.21.15.43:0/732456437 conn(0x7f03d407ceb0 msgr2=0x7f03d4107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:27.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.958+0000 7f03d9600640 1 -- 172.21.15.43:0/732456437 shutdown_connections 2024-09-19T04:54:27.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:26.958+0000 7f03d9600640 1 -- 172.21.15.43:0/732456437 wait complete. 2024-09-19T04:54:27.988 INFO:teuthology.orchestra.run.smithi043.stdout:mgr not available, waiting (2/15)... 2024-09-19T04:54:28.024 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:27 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:27.834+0000 7f98f85e1040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T04:54:28.024 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:27 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:27.916+0000 7f98f85e1040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T04:54:28.025 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:28 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:28.022+0000 7f98f85e1040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T04:54:28.280 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:28 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:28.096+0000 7f98f85e1040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T04:54:28.280 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:28 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:28.169+0000 7f98f85e1040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T04:54:29.530 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:29 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:29.130+0000 7f98f85e1040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T04:54:29.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:29 smithi043 ceph-mon[30800]: Activating manager daemon x 2024-09-19T04:54:29.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:29 smithi043 ceph-mon[30800]: mgrmap e2: x(active, starting, since 0.0039737s) 2024-09-19T04:54:29.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:29 smithi043 ceph-mon[30800]: from='mgr.14100 172.21.15.43:0/3348056712' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T04:54:29.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:29 smithi043 ceph-mon[30800]: from='mgr.14100 172.21.15.43:0/3348056712' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T04:54:29.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:29 smithi043 ceph-mon[30800]: from='mgr.14100 172.21.15.43:0/3348056712' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T04:54:29.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:29 smithi043 ceph-mon[30800]: from='mgr.14100 172.21.15.43:0/3348056712' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:54:29.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:29 smithi043 ceph-mon[30800]: from='mgr.14100 172.21.15.43:0/3348056712' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T04:54:29.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:29 smithi043 ceph-mon[30800]: Manager daemon x is now available 2024-09-19T04:54:29.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:29 smithi043 ceph-mon[30800]: from='mgr.14100 172.21.15.43:0/3348056712' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T04:54:29.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:29 smithi043 ceph-mon[30800]: from='mgr.14100 172.21.15.43:0/3348056712' entity='mgr.x' 2024-09-19T04:54:29.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:29 smithi043 ceph-mon[30800]: from='mgr.14100 172.21.15.43:0/3348056712' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T04:54:29.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:29 smithi043 ceph-mon[30800]: from='mgr.14100 172.21.15.43:0/3348056712' entity='mgr.x' 2024-09-19T04:54:29.532 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:29 smithi043 ceph-mon[30800]: from='mgr.14100 172.21.15.43:0/3348056712' entity='mgr.x' 2024-09-19T04:54:31.388 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:31 smithi043 ceph-mon[30800]: mgrmap e3: x(active, since 1.00919s) 2024-09-19T04:54:31.388 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:31 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/435106856' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T04:54:31.404 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-19T04:54:31.404 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-09-19T04:54:31.404 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsid": "1456a810-7643-11ef-bceb-c7b262605968", 2024-09-19T04:54:31.404 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T04:54:31.405 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T04:54:31.405 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T04:54:31.405 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T04:54:31.405 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:54:31.405 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T04:54:31.405 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T04:54:31.405 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 0 2024-09-19T04:54:31.405 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-19T04:54:31.405 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T04:54:31.405 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T04:54:31.405 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-19T04:54:31.405 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-09-19T04:54:31.405 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T04:54:31.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:54:31.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T04:54:31.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T04:54:31.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:54:31.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T04:54:31.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:54:31.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T04:54:31.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T04:54:31.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T04:54:31.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T04:54:31.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T04:54:31.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T04:54:31.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:54:31.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T04:54:31.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T04:54:31.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T04:54:31.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T04:54:31.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T04:54:31.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T04:54:31.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T04:54:31.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T04:54:31.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T04:54:31.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:54:31.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T04:54:31.408 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:54:31.408 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T04:54:14:706422+0000", 2024-09-19T04:54:31.408 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T04:54:31.408 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T04:54:31.408 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:54:31.408 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T04:54:31.408 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T04:54:31.408 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T04:54:31.408 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T04:54:31.408 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T04:54:31.408 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T04:54:31.408 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T04:54:31.409 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-19T04:54:31.409 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T04:54:31.409 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:54:31.409 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T04:54:31.409 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T04:54:31.409 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T04:54:14.707187+0000", 2024-09-19T04:54:31.409 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T04:54:31.409 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-19T04:54:31.409 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T04:54:31.409 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-09-19T04:54:31.409 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.457+0000 7f8477a00640 1 Processor -- start 2024-09-19T04:54:31.409 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.457+0000 7f8477a00640 1 -- start start 2024-09-19T04:54:31.409 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.457+0000 7f8477a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8470109960 0x7f8470109d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:31.409 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.457+0000 7f8477a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f847010a330 con 0x7f8470109960 2024-09-19T04:54:31.410 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.457+0000 7f8475000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8470109960 0x7f8470109d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:31.410 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.458+0000 7f8475000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8470109960 0x7f8470109d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41066/0 (socket says 172.21.15.43:41066) 2024-09-19T04:54:31.410 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.458+0000 7f8475000640 1 -- 172.21.15.43:0/2652317213 learned_addr learned my addr 172.21.15.43:0/2652317213 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:54:31.410 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.458+0000 7f8475000640 1 -- 172.21.15.43:0/2652317213 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f847010aad0 con 0x7f8470109960 2024-09-19T04:54:31.410 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.459+0000 7f8475000640 1 --2- 172.21.15.43:0/2652317213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8470109960 0x7f8470109d60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8460009820 tx=0x7f846002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f4938ebc34bfc6f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:31.410 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.459+0000 7f846fe00640 1 -- 172.21.15.43:0/2652317213 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8460037070 con 0x7f8470109960 2024-09-19T04:54:31.410 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.459+0000 7f846fe00640 1 -- 172.21.15.43:0/2652317213 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8460032900 con 0x7f8470109960 2024-09-19T04:54:31.410 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.459+0000 7f846fe00640 1 -- 172.21.15.43:0/2652317213 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8460030850 con 0x7f8470109960 2024-09-19T04:54:31.410 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.460+0000 7f8477a00640 1 -- 172.21.15.43:0/2652317213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8470109960 msgr2=0x7f8470109d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:31.410 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.460+0000 7f8477a00640 1 --2- 172.21.15.43:0/2652317213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8470109960 0x7f8470109d60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8460009820 tx=0x7f846002a270 comp rx=0 tx=0).stop 2024-09-19T04:54:31.410 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.460+0000 7f8477a00640 1 -- 172.21.15.43:0/2652317213 shutdown_connections 2024-09-19T04:54:31.410 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.460+0000 7f8477a00640 1 --2- 172.21.15.43:0/2652317213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8470109960 0x7f8470109d60 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:31.410 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.460+0000 7f8477a00640 1 -- 172.21.15.43:0/2652317213 >> 172.21.15.43:0/2652317213 conn(0x7f847007cf30 msgr2=0x7f847007d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:31.411 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.460+0000 7f8477a00640 1 -- 172.21.15.43:0/2652317213 shutdown_connections 2024-09-19T04:54:31.411 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.460+0000 7f8477a00640 1 -- 172.21.15.43:0/2652317213 wait complete. 2024-09-19T04:54:31.411 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.461+0000 7f8477a00640 1 Processor -- start 2024-09-19T04:54:31.411 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.461+0000 7f8477a00640 1 -- start start 2024-09-19T04:54:31.411 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.461+0000 7f8477a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8470109960 0x7f847019d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:31.411 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.461+0000 7f8477a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f847010de80 con 0x7f8470109960 2024-09-19T04:54:31.411 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.462+0000 7f8475000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8470109960 0x7f847019d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:31.411 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.462+0000 7f8475000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8470109960 0x7f847019d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41082/0 (socket says 172.21.15.43:41082) 2024-09-19T04:54:31.411 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.462+0000 7f8475000640 1 -- 172.21.15.43:0/435106856 learned_addr learned my addr 172.21.15.43:0/435106856 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:54:31.411 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.462+0000 7f8475000640 1 -- 172.21.15.43:0/435106856 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f847019de60 con 0x7f8470109960 2024-09-19T04:54:31.411 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.462+0000 7f8475000640 1 --2- 172.21.15.43:0/435106856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8470109960 0x7f847019d920 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f84600097f0 tx=0x7f8460030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:31.411 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.463+0000 7f846e000640 1 -- 172.21.15.43:0/435106856 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8460042070 con 0x7f8470109960 2024-09-19T04:54:31.411 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.463+0000 7f846e000640 1 -- 172.21.15.43:0/435106856 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8460032ca0 con 0x7f8470109960 2024-09-19T04:54:31.412 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.463+0000 7f8477a00640 1 -- 172.21.15.43:0/435106856 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f847019e0f0 con 0x7f8470109960 2024-09-19T04:54:31.412 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.463+0000 7f846e000640 1 -- 172.21.15.43:0/435106856 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8460037040 con 0x7f8470109960 2024-09-19T04:54:31.412 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.463+0000 7f8477a00640 1 -- 172.21.15.43:0/435106856 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84701a0de0 con 0x7f8470109960 2024-09-19T04:54:31.412 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.464+0000 7f846e000640 1 -- 172.21.15.43:0/435106856 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 3) ==== 46344+0+0 (secure 0 0 0) 0x7f8460045070 con 0x7f8470109960 2024-09-19T04:54:31.412 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.464+0000 7f8477a00640 1 -- 172.21.15.43:0/435106856 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f847010e000 con 0x7f8470109960 2024-09-19T04:54:31.412 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.464+0000 7f846e000640 1 --2- 172.21.15.43:0/435106856 >> [v2:172.21.15.43:6800/587463881,v1:172.21.15.43:6801/587463881] conn(0x7f8454039050 0x7f845403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:31.412 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.464+0000 7f846e000640 1 -- 172.21.15.43:0/435106856 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f846006bf60 con 0x7f8470109960 2024-09-19T04:54:31.412 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.465+0000 7f8467e00640 1 --2- 172.21.15.43:0/435106856 >> [v2:172.21.15.43:6800/587463881,v1:172.21.15.43:6801/587463881] conn(0x7f8454039050 0x7f845403b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:31.412 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.465+0000 7f8467e00640 1 --2- 172.21.15.43:0/435106856 >> [v2:172.21.15.43:6800/587463881,v1:172.21.15.43:6801/587463881] conn(0x7f8454039050 0x7f845403b510 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f84580098c0 tx=0x7f8458006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:31.412 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.470+0000 7f846e000640 1 -- 172.21.15.43:0/435106856 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f846003d430 con 0x7f8470109960 2024-09-19T04:54:31.412 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.721+0000 7f8477a00640 1 -- 172.21.15.43:0/435106856 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f84701a1470 con 0x7f8470109960 2024-09-19T04:54:31.412 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.722+0000 7f846e000640 1 -- 172.21.15.43:0/435106856 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f846003da60 con 0x7f8470109960 2024-09-19T04:54:31.413 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.724+0000 7f8477a00640 1 -- 172.21.15.43:0/435106856 >> [v2:172.21.15.43:6800/587463881,v1:172.21.15.43:6801/587463881] conn(0x7f8454039050 msgr2=0x7f845403b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:31.413 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.724+0000 7f8477a00640 1 --2- 172.21.15.43:0/435106856 >> [v2:172.21.15.43:6800/587463881,v1:172.21.15.43:6801/587463881] conn(0x7f8454039050 0x7f845403b510 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f84580098c0 tx=0x7f8458006db0 comp rx=0 tx=0).stop 2024-09-19T04:54:31.413 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.724+0000 7f8477a00640 1 -- 172.21.15.43:0/435106856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8470109960 msgr2=0x7f847019d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:31.413 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.724+0000 7f8477a00640 1 --2- 172.21.15.43:0/435106856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8470109960 0x7f847019d920 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f84600097f0 tx=0x7f8460030ec0 comp rx=0 tx=0).stop 2024-09-19T04:54:31.413 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.724+0000 7f8477a00640 1 -- 172.21.15.43:0/435106856 shutdown_connections 2024-09-19T04:54:31.413 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.724+0000 7f8477a00640 1 --2- 172.21.15.43:0/435106856 >> [v2:172.21.15.43:6800/587463881,v1:172.21.15.43:6801/587463881] conn(0x7f8454039050 0x7f845403b510 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:31.413 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.724+0000 7f8477a00640 1 --2- 172.21.15.43:0/435106856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8470109960 0x7f847019d920 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:31.413 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.724+0000 7f8477a00640 1 -- 172.21.15.43:0/435106856 >> 172.21.15.43:0/435106856 conn(0x7f847007cf30 msgr2=0x7f8470107570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:31.413 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.725+0000 7f8477a00640 1 -- 172.21.15.43:0/435106856 shutdown_connections 2024-09-19T04:54:31.413 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:30.725+0000 7f8477a00640 1 -- 172.21.15.43:0/435106856 wait complete. 2024-09-19T04:54:31.413 INFO:teuthology.orchestra.run.smithi043.stdout:mgr is available 2024-09-19T04:54:32.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:32 smithi043 ceph-mon[30800]: mgrmap e4: x(active, since 2s) 2024-09-19T04:54:32.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:32 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/3421189021' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-19T04:54:32.614 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-19T04:54:32.615 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [global] 2024-09-19T04:54:32.615 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout fsid = 1456a810-7643-11ef-bceb-c7b262605968 2024-09-19T04:54:32.615 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.43:3300,v1:172.21.15.43:6789] 2024-09-19T04:54:32.615 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-19T04:54:32.615 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-19T04:54:32.615 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-19T04:54:32.615 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-19T04:54:32.615 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-19T04:54:32.615 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-19T04:54:32.615 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-19T04:54:32.616 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-19T04:54:32.616 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [osd] 2024-09-19T04:54:32.616 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-19T04:54:32.616 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-19T04:54:32.616 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-19T04:54:32.616 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-19T04:54:32.616 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-19T04:54:32.616 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:31.898+0000 7fd025600640 1 Processor -- start 2024-09-19T04:54:32.616 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:31.898+0000 7fd025600640 1 -- start start 2024-09-19T04:54:32.616 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:31.898+0000 7fd025600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0201076f0 0x7fd020107af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:32.616 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:31.898+0000 7fd025600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd0201080c0 con 0x7fd0201076f0 2024-09-19T04:54:32.616 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:31.899+0000 7fd01ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0201076f0 0x7fd020107af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:32.616 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:31.899+0000 7fd01ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0201076f0 0x7fd020107af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41088/0 (socket says 172.21.15.43:41088) 2024-09-19T04:54:32.616 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:31.899+0000 7fd01ea00640 1 -- 172.21.15.43:0/1334110894 learned_addr learned my addr 172.21.15.43:0/1334110894 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:54:32.617 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:31.899+0000 7fd01ea00640 1 -- 172.21.15.43:0/1334110894 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0201088f0 con 0x7fd0201076f0 2024-09-19T04:54:32.617 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:31.900+0000 7fd01ea00640 1 --2- 172.21.15.43:0/1334110894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0201076f0 0x7fd020107af0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd010009820 tx=0x7fd01002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfbd695259a87b37 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:32.617 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:31.900+0000 7fd01d600640 1 -- 172.21.15.43:0/1334110894 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd010037070 con 0x7fd0201076f0 2024-09-19T04:54:32.617 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:31.900+0000 7fd01d600640 1 -- 172.21.15.43:0/1334110894 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd010032900 con 0x7fd0201076f0 2024-09-19T04:54:32.617 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:31.900+0000 7fd01d600640 1 -- 172.21.15.43:0/1334110894 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd010030850 con 0x7fd0201076f0 2024-09-19T04:54:32.617 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:31.901+0000 7fd025600640 1 -- 172.21.15.43:0/1334110894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0201076f0 msgr2=0x7fd020107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:32.617 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:31.901+0000 7fd025600640 1 --2- 172.21.15.43:0/1334110894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0201076f0 0x7fd020107af0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd010009820 tx=0x7fd01002a270 comp rx=0 tx=0).stop 2024-09-19T04:54:32.617 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:31.901+0000 7fd025600640 1 -- 172.21.15.43:0/1334110894 shutdown_connections 2024-09-19T04:54:32.617 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:31.901+0000 7fd025600640 1 --2- 172.21.15.43:0/1334110894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0201076f0 0x7fd020107af0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:32.617 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:31.901+0000 7fd025600640 1 -- 172.21.15.43:0/1334110894 >> 172.21.15.43:0/1334110894 conn(0x7fd020102ec0 msgr2=0x7fd0201052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:32.617 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:31.901+0000 7fd025600640 1 -- 172.21.15.43:0/1334110894 shutdown_connections 2024-09-19T04:54:32.617 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:31.901+0000 7fd025600640 1 -- 172.21.15.43:0/1334110894 wait complete. 2024-09-19T04:54:32.617 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:31.902+0000 7fd025600640 1 Processor -- start 2024-09-19T04:54:32.617 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:31.902+0000 7fd025600640 1 -- start start 2024-09-19T04:54:32.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:31.903+0000 7fd025600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0201076f0 0x7fd02019d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:32.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:31.903+0000 7fd025600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd02010bca0 con 0x7fd0201076f0 2024-09-19T04:54:32.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:31.903+0000 7fd01ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0201076f0 0x7fd02019d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:32.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:31.903+0000 7fd01ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0201076f0 0x7fd02019d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41092/0 (socket says 172.21.15.43:41092) 2024-09-19T04:54:32.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:31.903+0000 7fd01ea00640 1 -- 172.21.15.43:0/3421189021 learned_addr learned my addr 172.21.15.43:0/3421189021 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:54:32.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:31.903+0000 7fd01ea00640 1 -- 172.21.15.43:0/3421189021 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd02019de50 con 0x7fd0201076f0 2024-09-19T04:54:32.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:31.904+0000 7fd01ea00640 1 --2- 172.21.15.43:0/3421189021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0201076f0 0x7fd02019d910 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd0100097f0 tx=0x7fd010030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:32.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:31.904+0000 7fd017400640 1 -- 172.21.15.43:0/3421189021 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd010037040 con 0x7fd0201076f0 2024-09-19T04:54:32.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:31.904+0000 7fd017400640 1 -- 172.21.15.43:0/3421189021 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd010040070 con 0x7fd0201076f0 2024-09-19T04:54:32.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:31.904+0000 7fd017400640 1 -- 172.21.15.43:0/3421189021 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd01003a390 con 0x7fd0201076f0 2024-09-19T04:54:32.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:31.904+0000 7fd025600640 1 -- 172.21.15.43:0/3421189021 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd02019e0e0 con 0x7fd0201076f0 2024-09-19T04:54:32.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:31.904+0000 7fd025600640 1 -- 172.21.15.43:0/3421189021 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0201a0dd0 con 0x7fd0201076f0 2024-09-19T04:54:32.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:31.905+0000 7fd017400640 1 -- 172.21.15.43:0/3421189021 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7fd01003b430 con 0x7fd0201076f0 2024-09-19T04:54:32.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:31.905+0000 7fd025600640 1 -- 172.21.15.43:0/3421189021 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd020107b70 con 0x7fd0201076f0 2024-09-19T04:54:32.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:31.905+0000 7fd017400640 1 --2- 172.21.15.43:0/3421189021 >> [v2:172.21.15.43:6800/587463881,v1:172.21.15.43:6801/587463881] conn(0x7fd0040391b0 0x7fd00403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:32.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:31.905+0000 7fd017400640 1 -- 172.21.15.43:0/3421189021 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd01006c400 con 0x7fd0201076f0 2024-09-19T04:54:32.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:31.906+0000 7fd01e000640 1 --2- 172.21.15.43:0/3421189021 >> [v2:172.21.15.43:6800/587463881,v1:172.21.15.43:6801/587463881] conn(0x7fd0040391b0 0x7fd00403b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:32.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:31.906+0000 7fd01e000640 1 --2- 172.21.15.43:0/3421189021 >> [v2:172.21.15.43:6800/587463881,v1:172.21.15.43:6801/587463881] conn(0x7fd0040391b0 0x7fd00403b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd00c0098c0 tx=0x7fd00c006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:32.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:31.911+0000 7fd017400640 1 -- 172.21.15.43:0/3421189021 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd0100371e0 con 0x7fd0201076f0 2024-09-19T04:54:32.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:32.094+0000 7fd025600640 1 -- 172.21.15.43:0/3421189021 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fd02019e560 con 0x7fd0201076f0 2024-09-19T04:54:32.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:32.095+0000 7fd017400640 1 -- 172.21.15.43:0/3421189021 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+429 (secure 0 0 0) 0x7fd010032dd0 con 0x7fd0201076f0 2024-09-19T04:54:32.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:32.098+0000 7fd025600640 1 -- 172.21.15.43:0/3421189021 >> [v2:172.21.15.43:6800/587463881,v1:172.21.15.43:6801/587463881] conn(0x7fd0040391b0 msgr2=0x7fd00403b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:32.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:32.098+0000 7fd025600640 1 --2- 172.21.15.43:0/3421189021 >> [v2:172.21.15.43:6800/587463881,v1:172.21.15.43:6801/587463881] conn(0x7fd0040391b0 0x7fd00403b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd00c0098c0 tx=0x7fd00c006db0 comp rx=0 tx=0).stop 2024-09-19T04:54:32.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:32.098+0000 7fd025600640 1 -- 172.21.15.43:0/3421189021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0201076f0 msgr2=0x7fd02019d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:32.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:32.098+0000 7fd025600640 1 --2- 172.21.15.43:0/3421189021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0201076f0 0x7fd02019d910 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd0100097f0 tx=0x7fd010030d80 comp rx=0 tx=0).stop 2024-09-19T04:54:32.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:32.098+0000 7fd025600640 1 -- 172.21.15.43:0/3421189021 shutdown_connections 2024-09-19T04:54:32.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:32.098+0000 7fd025600640 1 --2- 172.21.15.43:0/3421189021 >> [v2:172.21.15.43:6800/587463881,v1:172.21.15.43:6801/587463881] conn(0x7fd0040391b0 0x7fd00403b670 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:32.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:32.098+0000 7fd025600640 1 --2- 172.21.15.43:0/3421189021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0201076f0 0x7fd02019d910 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:32.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:32.098+0000 7fd025600640 1 -- 172.21.15.43:0/3421189021 >> 172.21.15.43:0/3421189021 conn(0x7fd020102ec0 msgr2=0x7fd020103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:32.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:32.098+0000 7fd025600640 1 -- 172.21.15.43:0/3421189021 shutdown_connections 2024-09-19T04:54:32.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:32.098+0000 7fd025600640 1 -- 172.21.15.43:0/3421189021 wait complete. 2024-09-19T04:54:32.620 INFO:teuthology.orchestra.run.smithi043.stdout:Enabling cephadm module... 2024-09-19T04:54:33.658 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:33 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/2357706799' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-19T04:54:33.658 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:33 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: ignoring --setuser ceph since I am not root 2024-09-19T04:54:33.658 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:33 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: ignoring --setgroup ceph since I am not root 2024-09-19T04:54:33.658 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:33 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:33.656+0000 7f93289da040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T04:54:33.923 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:33 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:33.730+0000 7f93289da040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T04:54:33.923 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:33 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:33.921+0000 7f93289da040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T04:54:34.122 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.130+0000 7f1ad5c00640 1 Processor -- start 2024-09-19T04:54:34.122 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.130+0000 7f1ad5c00640 1 -- start start 2024-09-19T04:54:34.122 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.130+0000 7f1ad5c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ad0109960 0x7f1ad0109d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:34.122 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.130+0000 7f1ad5c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad010a330 con 0x7f1ad0109960 2024-09-19T04:54:34.122 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.131+0000 7f1acea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ad0109960 0x7f1ad0109d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:34.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.131+0000 7f1acea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ad0109960 0x7f1ad0109d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41100/0 (socket says 172.21.15.43:41100) 2024-09-19T04:54:34.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.131+0000 7f1acea00640 1 -- 172.21.15.43:0/1765762267 learned_addr learned my addr 172.21.15.43:0/1765762267 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:54:34.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.131+0000 7f1acea00640 1 -- 172.21.15.43:0/1765762267 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ad010aad0 con 0x7f1ad0109960 2024-09-19T04:54:34.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.132+0000 7f1acea00640 1 --2- 172.21.15.43:0/1765762267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ad0109960 0x7f1ad0109d60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0009820 tx=0x7f1ac002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb99f2f981d9bcd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:34.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.132+0000 7f1ace000640 1 -- 172.21.15.43:0/1765762267 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ac0037070 con 0x7f1ad0109960 2024-09-19T04:54:34.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.133+0000 7f1ace000640 1 -- 172.21.15.43:0/1765762267 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1ac0032900 con 0x7f1ad0109960 2024-09-19T04:54:34.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.133+0000 7f1ace000640 1 -- 172.21.15.43:0/1765762267 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ac0030850 con 0x7f1ad0109960 2024-09-19T04:54:34.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.133+0000 7f1ad5c00640 1 -- 172.21.15.43:0/1765762267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ad0109960 msgr2=0x7f1ad0109d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:34.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.133+0000 7f1ad5c00640 1 --2- 172.21.15.43:0/1765762267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ad0109960 0x7f1ad0109d60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0009820 tx=0x7f1ac002a270 comp rx=0 tx=0).stop 2024-09-19T04:54:34.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.133+0000 7f1ad5c00640 1 -- 172.21.15.43:0/1765762267 shutdown_connections 2024-09-19T04:54:34.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.133+0000 7f1ad5c00640 1 --2- 172.21.15.43:0/1765762267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ad0109960 0x7f1ad0109d60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:34.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.133+0000 7f1ad5c00640 1 -- 172.21.15.43:0/1765762267 >> 172.21.15.43:0/1765762267 conn(0x7f1ad007cf30 msgr2=0x7f1ad007d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:34.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.133+0000 7f1ad5c00640 1 -- 172.21.15.43:0/1765762267 shutdown_connections 2024-09-19T04:54:34.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.133+0000 7f1ad5c00640 1 -- 172.21.15.43:0/1765762267 wait complete. 2024-09-19T04:54:34.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.134+0000 7f1ad5c00640 1 Processor -- start 2024-09-19T04:54:34.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.135+0000 7f1ad5c00640 1 -- start start 2024-09-19T04:54:34.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.135+0000 7f1ad5c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ad0109960 0x7f1ad019d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:34.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.135+0000 7f1ad5c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad010de80 con 0x7f1ad0109960 2024-09-19T04:54:34.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.135+0000 7f1acea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ad0109960 0x7f1ad019d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:34.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.135+0000 7f1acea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ad0109960 0x7f1ad019d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41104/0 (socket says 172.21.15.43:41104) 2024-09-19T04:54:34.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.135+0000 7f1acea00640 1 -- 172.21.15.43:0/2357706799 learned_addr learned my addr 172.21.15.43:0/2357706799 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:54:34.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.135+0000 7f1acea00640 1 -- 172.21.15.43:0/2357706799 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ad019de60 con 0x7f1ad0109960 2024-09-19T04:54:34.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.136+0000 7f1acea00640 1 --2- 172.21.15.43:0/2357706799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ad0109960 0x7f1ad019d920 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1ac00097f0 tx=0x7f1ac0030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:34.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.136+0000 7f1ac7e00640 1 -- 172.21.15.43:0/2357706799 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ac0042070 con 0x7f1ad0109960 2024-09-19T04:54:34.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.136+0000 7f1ac7e00640 1 -- 172.21.15.43:0/2357706799 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1ac0032ca0 con 0x7f1ad0109960 2024-09-19T04:54:34.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.136+0000 7f1ac7e00640 1 -- 172.21.15.43:0/2357706799 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ac0037040 con 0x7f1ad0109960 2024-09-19T04:54:34.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.136+0000 7f1ad5c00640 1 -- 172.21.15.43:0/2357706799 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ad019e0f0 con 0x7f1ad0109960 2024-09-19T04:54:34.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.136+0000 7f1ad5c00640 1 -- 172.21.15.43:0/2357706799 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ad01a0de0 con 0x7f1ad0109960 2024-09-19T04:54:34.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.137+0000 7f1ac7e00640 1 -- 172.21.15.43:0/2357706799 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f1ac0045070 con 0x7f1ad0109960 2024-09-19T04:54:34.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.137+0000 7f1ad5c00640 1 -- 172.21.15.43:0/2357706799 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ad010e000 con 0x7f1ad0109960 2024-09-19T04:54:34.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.138+0000 7f1ac7e00640 1 --2- 172.21.15.43:0/2357706799 >> [v2:172.21.15.43:6800/587463881,v1:172.21.15.43:6801/587463881] conn(0x7f1ab40391b0 0x7f1ab403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:34.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.138+0000 7f1ac7e00640 1 -- 172.21.15.43:0/2357706799 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1ac006c140 con 0x7f1ad0109960 2024-09-19T04:54:34.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.138+0000 7f1ac6000640 1 --2- 172.21.15.43:0/2357706799 >> [v2:172.21.15.43:6800/587463881,v1:172.21.15.43:6801/587463881] conn(0x7f1ab40391b0 0x7f1ab403b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:34.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.139+0000 7f1ac6000640 1 --2- 172.21.15.43:0/2357706799 >> [v2:172.21.15.43:6800/587463881,v1:172.21.15.43:6801/587463881] conn(0x7f1ab40391b0 0x7f1ab403b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1ab80098c0 tx=0x7f1ab8006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:34.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.142+0000 7f1ac7e00640 1 -- 172.21.15.43:0/2357706799 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1ac004b170 con 0x7f1ad0109960 2024-09-19T04:54:34.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.377+0000 7f1ad5c00640 1 -- 172.21.15.43:0/2357706799 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f1ad01a1470 con 0x7f1ad0109960 2024-09-19T04:54:34.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.429+0000 7f1ac7e00640 1 -- 172.21.15.43:0/2357706799 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f1ac006bb60 con 0x7f1ad0109960 2024-09-19T04:54:34.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.430+0000 7f1ac7e00640 1 -- 172.21.15.43:0/2357706799 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f1ac003d430 con 0x7f1ad0109960 2024-09-19T04:54:34.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.432+0000 7f1ad5c00640 1 -- 172.21.15.43:0/2357706799 >> [v2:172.21.15.43:6800/587463881,v1:172.21.15.43:6801/587463881] conn(0x7f1ab40391b0 msgr2=0x7f1ab403b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:34.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.432+0000 7f1ad5c00640 1 --2- 172.21.15.43:0/2357706799 >> [v2:172.21.15.43:6800/587463881,v1:172.21.15.43:6801/587463881] conn(0x7f1ab40391b0 0x7f1ab403b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1ab80098c0 tx=0x7f1ab8006db0 comp rx=0 tx=0).stop 2024-09-19T04:54:34.126 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.432+0000 7f1ad5c00640 1 -- 172.21.15.43:0/2357706799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ad0109960 msgr2=0x7f1ad019d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:34.126 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.432+0000 7f1ad5c00640 1 --2- 172.21.15.43:0/2357706799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ad0109960 0x7f1ad019d920 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1ac00097f0 tx=0x7f1ac0030ec0 comp rx=0 tx=0).stop 2024-09-19T04:54:34.126 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.432+0000 7f1ad5c00640 1 -- 172.21.15.43:0/2357706799 shutdown_connections 2024-09-19T04:54:34.126 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.432+0000 7f1ad5c00640 1 --2- 172.21.15.43:0/2357706799 >> [v2:172.21.15.43:6800/587463881,v1:172.21.15.43:6801/587463881] conn(0x7f1ab40391b0 0x7f1ab403b670 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:34.126 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.432+0000 7f1ad5c00640 1 --2- 172.21.15.43:0/2357706799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ad0109960 0x7f1ad019d920 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:34.126 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.432+0000 7f1ad5c00640 1 -- 172.21.15.43:0/2357706799 >> 172.21.15.43:0/2357706799 conn(0x7f1ad007cf30 msgr2=0x7f1ad0107570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:34.126 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.432+0000 7f1ad5c00640 1 -- 172.21.15.43:0/2357706799 shutdown_connections 2024-09-19T04:54:34.126 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:33.432+0000 7f1ad5c00640 1 -- 172.21.15.43:0/2357706799 wait complete. 2024-09-19T04:54:34.202 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:34 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:34.005+0000 7f93289da040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T04:54:34.202 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:34 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:34.199+0000 7f93289da040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:54:34.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:34 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/2357706799' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-19T04:54:34.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:34 smithi043 ceph-mon[30800]: mgrmap e5: x(active, since 4s) 2024-09-19T04:54:34.531 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:34 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:34.372+0000 7f93289da040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T04:54:34.847 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:34 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:34.770+0000 7f93289da040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T04:54:34.847 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:34 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:34.844+0000 7f93289da040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T04:54:35.237 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:35 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:35.071+0000 7f93289da040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T04:54:35.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:35 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/3583316345' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-19T04:54:35.563 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-09-19T04:54:35.569 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-19T04:54:35.569 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T04:54:35.569 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-19T04:54:35.569 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-19T04:54:35.569 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-09-19T04:54:35.569 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.646+0000 7ff307e00640 1 Processor -- start 2024-09-19T04:54:35.569 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.646+0000 7ff307e00640 1 -- start start 2024-09-19T04:54:35.569 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.646+0000 7ff307e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff30807de40 0x7ff30807c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:35.569 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.646+0000 7ff307e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff30807c880 con 0x7ff30807de40 2024-09-19T04:54:35.570 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.646+0000 7ff306a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff30807de40 0x7ff30807c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:35.570 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.647+0000 7ff306a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff30807de40 0x7ff30807c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41146/0 (socket says 172.21.15.43:41146) 2024-09-19T04:54:35.570 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.647+0000 7ff306a00640 1 -- 172.21.15.43:0/3586963840 learned_addr learned my addr 172.21.15.43:0/3586963840 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:54:35.570 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.647+0000 7ff306a00640 1 -- 172.21.15.43:0/3586963840 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff30807ca00 con 0x7ff30807de40 2024-09-19T04:54:35.570 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.648+0000 7ff306a00640 1 --2- 172.21.15.43:0/3586963840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff30807de40 0x7ff30807c340 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8009820 tx=0x7ff2f802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14e185b5709cde09 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:35.570 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.648+0000 7ff306000640 1 -- 172.21.15.43:0/3586963840 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2f8037070 con 0x7ff30807de40 2024-09-19T04:54:35.570 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.648+0000 7ff306000640 1 -- 172.21.15.43:0/3586963840 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff2f8032900 con 0x7ff30807de40 2024-09-19T04:54:35.570 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.648+0000 7ff306000640 1 -- 172.21.15.43:0/3586963840 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2f8030850 con 0x7ff30807de40 2024-09-19T04:54:35.570 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.649+0000 7ff307e00640 1 -- 172.21.15.43:0/3586963840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff30807de40 msgr2=0x7ff30807c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:35.570 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.649+0000 7ff307e00640 1 --2- 172.21.15.43:0/3586963840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff30807de40 0x7ff30807c340 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8009820 tx=0x7ff2f802a270 comp rx=0 tx=0).stop 2024-09-19T04:54:35.570 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.649+0000 7ff307e00640 1 -- 172.21.15.43:0/3586963840 shutdown_connections 2024-09-19T04:54:35.570 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.649+0000 7ff307e00640 1 --2- 172.21.15.43:0/3586963840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff30807de40 0x7ff30807c340 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:35.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.649+0000 7ff307e00640 1 -- 172.21.15.43:0/3586963840 >> 172.21.15.43:0/3586963840 conn(0x7ff308102d90 msgr2=0x7ff3081051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:35.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.649+0000 7ff307e00640 1 -- 172.21.15.43:0/3586963840 shutdown_connections 2024-09-19T04:54:35.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.649+0000 7ff307e00640 1 -- 172.21.15.43:0/3586963840 wait complete. 2024-09-19T04:54:35.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.650+0000 7ff307e00640 1 Processor -- start 2024-09-19T04:54:35.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.650+0000 7ff307e00640 1 -- start start 2024-09-19T04:54:35.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.651+0000 7ff307e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff30807de40 0x7ff3081a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:35.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.651+0000 7ff307e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff308109780 con 0x7ff30807de40 2024-09-19T04:54:35.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.651+0000 7ff306a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff30807de40 0x7ff3081a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:35.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.651+0000 7ff306a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff30807de40 0x7ff3081a1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41154/0 (socket says 172.21.15.43:41154) 2024-09-19T04:54:35.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.651+0000 7ff306a00640 1 -- 172.21.15.43:0/3583316345 learned_addr learned my addr 172.21.15.43:0/3583316345 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:54:35.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.651+0000 7ff306a00640 1 -- 172.21.15.43:0/3583316345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3081a2050 con 0x7ff30807de40 2024-09-19T04:54:35.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.652+0000 7ff306a00640 1 --2- 172.21.15.43:0/3583316345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff30807de40 0x7ff3081a1b10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff2f80097f0 tx=0x7ff2f8030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:35.572 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.652+0000 7ff30ca00640 1 -- 172.21.15.43:0/3583316345 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2f8037040 con 0x7ff30807de40 2024-09-19T04:54:35.572 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.652+0000 7ff30ca00640 1 -- 172.21.15.43:0/3583316345 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff2f8040070 con 0x7ff30807de40 2024-09-19T04:54:35.572 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.652+0000 7ff30ca00640 1 -- 172.21.15.43:0/3583316345 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2f803a390 con 0x7ff30807de40 2024-09-19T04:54:35.572 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.652+0000 7ff307e00640 1 -- 172.21.15.43:0/3583316345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3081a22e0 con 0x7ff30807de40 2024-09-19T04:54:35.572 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.652+0000 7ff307e00640 1 -- 172.21.15.43:0/3583316345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3081a4fd0 con 0x7ff30807de40 2024-09-19T04:54:35.573 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.653+0000 7ff30ca00640 1 -- 172.21.15.43:0/3583316345 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7ff2f8032900 con 0x7ff30807de40 2024-09-19T04:54:35.573 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.653+0000 7ff307e00640 1 -- 172.21.15.43:0/3583316345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff30807c3c0 con 0x7ff30807de40 2024-09-19T04:54:35.573 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.654+0000 7ff30ca00640 1 --2- 172.21.15.43:0/3583316345 >> [v2:172.21.15.43:6800/587463881,v1:172.21.15.43:6801/587463881] conn(0x7ff2ec039200 0x7ff2ec03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:35.573 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.654+0000 7ff30ca00640 1 -- 172.21.15.43:0/3583316345 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff2f806c450 con 0x7ff30807de40 2024-09-19T04:54:35.573 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.654+0000 7ff2fe000640 1 -- 172.21.15.43:0/3583316345 >> [v2:172.21.15.43:6800/587463881,v1:172.21.15.43:6801/587463881] conn(0x7ff2ec039200 msgr2=0x7ff2ec03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/587463881 2024-09-19T04:54:35.573 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.654+0000 7ff2fe000640 1 --2- 172.21.15.43:0/3583316345 >> [v2:172.21.15.43:6800/587463881,v1:172.21.15.43:6801/587463881] conn(0x7ff2ec039200 0x7ff2ec03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:54:35.573 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.663+0000 7ff30ca00640 1 -- 172.21.15.43:0/3583316345 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff2f806c1b0 con 0x7ff30807de40 2024-09-19T04:54:35.573 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.855+0000 7ff2fe000640 1 -- 172.21.15.43:0/3583316345 >> [v2:172.21.15.43:6800/587463881,v1:172.21.15.43:6801/587463881] conn(0x7ff2ec039200 msgr2=0x7ff2ec03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/587463881 2024-09-19T04:54:35.573 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.855+0000 7ff2fe000640 1 --2- 172.21.15.43:0/3583316345 >> [v2:172.21.15.43:6800/587463881,v1:172.21.15.43:6801/587463881] conn(0x7ff2ec039200 0x7ff2ec03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T04:54:35.573 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.896+0000 7ff307e00640 1 -- 172.21.15.43:0/3583316345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7ff30802da90 con 0x7ff30807de40 2024-09-19T04:54:35.574 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.897+0000 7ff30ca00640 1 -- 172.21.15.43:0/3583316345 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7ff2f8040ce0 con 0x7ff30807de40 2024-09-19T04:54:35.574 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.899+0000 7ff307e00640 1 -- 172.21.15.43:0/3583316345 >> [v2:172.21.15.43:6800/587463881,v1:172.21.15.43:6801/587463881] conn(0x7ff2ec039200 msgr2=0x7ff2ec03b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:54:35.574 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.899+0000 7ff307e00640 1 --2- 172.21.15.43:0/3583316345 >> [v2:172.21.15.43:6800/587463881,v1:172.21.15.43:6801/587463881] conn(0x7ff2ec039200 0x7ff2ec03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:35.574 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.899+0000 7ff307e00640 1 -- 172.21.15.43:0/3583316345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff30807de40 msgr2=0x7ff3081a1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:35.574 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.899+0000 7ff307e00640 1 --2- 172.21.15.43:0/3583316345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff30807de40 0x7ff3081a1b10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff2f80097f0 tx=0x7ff2f8030d80 comp rx=0 tx=0).stop 2024-09-19T04:54:35.574 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.899+0000 7ff307e00640 1 -- 172.21.15.43:0/3583316345 shutdown_connections 2024-09-19T04:54:35.574 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.899+0000 7ff307e00640 1 --2- 172.21.15.43:0/3583316345 >> [v2:172.21.15.43:6800/587463881,v1:172.21.15.43:6801/587463881] conn(0x7ff2ec039200 0x7ff2ec03b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:35.574 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.899+0000 7ff307e00640 1 --2- 172.21.15.43:0/3583316345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff30807de40 0x7ff3081a1b10 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:35.574 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.899+0000 7ff307e00640 1 -- 172.21.15.43:0/3583316345 >> 172.21.15.43:0/3583316345 conn(0x7ff308102d90 msgr2=0x7ff308105180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:35.574 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.899+0000 7ff307e00640 1 -- 172.21.15.43:0/3583316345 shutdown_connections 2024-09-19T04:54:35.574 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:34.899+0000 7ff307e00640 1 -- 172.21.15.43:0/3583316345 wait complete. 2024-09-19T04:54:35.574 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for the mgr to restart... 2024-09-19T04:54:35.574 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mgr epoch 5... 2024-09-19T04:54:36.530 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:36 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:36.238+0000 7f93289da040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T04:54:36.530 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:36 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:36.321+0000 7f93289da040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T04:54:37.530 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:37 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:37.270+0000 7f93289da040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T04:54:37.530 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:37 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:37.412+0000 7f93289da040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T04:54:37.837 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:37 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:37.582+0000 7f93289da040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:54:37.837 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:37 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:37.657+0000 7f93289da040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T04:54:37.837 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:37 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:37.834+0000 7f93289da040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T04:54:38.184 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:37 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:37.919+0000 7f93289da040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T04:54:38.184 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:37 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:37.994+0000 7f93289da040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T04:54:38.185 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:38 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:38.182+0000 7f93289da040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T04:54:38.449 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:38 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:38.262+0000 7f93289da040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T04:54:38.449 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:38 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:38.344+0000 7f93289da040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T04:54:38.449 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:38 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:38.445+0000 7f93289da040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T04:54:38.780 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:38 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:38.519+0000 7f93289da040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T04:54:38.780 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:38 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:38.591+0000 7f93289da040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T04:54:40.030 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:39 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:39.545+0000 7f93289da040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T04:54:40.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:39 smithi043 ceph-mon[30800]: Active manager daemon x restarted 2024-09-19T04:54:40.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:39 smithi043 ceph-mon[30800]: Activating manager daemon x 2024-09-19T04:54:40.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:39 smithi043 ceph-mon[30800]: osdmap e2: 0 total, 0 up, 0 in 2024-09-19T04:54:40.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:39 smithi043 ceph-mon[30800]: mgrmap e6: x(active, starting, since 0.00409905s) 2024-09-19T04:54:40.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:39 smithi043 ceph-mon[30800]: from='mgr.14118 172.21.15.43:0/568310870' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:54:40.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:39 smithi043 ceph-mon[30800]: from='mgr.14118 172.21.15.43:0/568310870' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T04:54:40.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:39 smithi043 ceph-mon[30800]: from='mgr.14118 172.21.15.43:0/568310870' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T04:54:40.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:39 smithi043 ceph-mon[30800]: from='mgr.14118 172.21.15.43:0/568310870' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T04:54:40.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:39 smithi043 ceph-mon[30800]: from='mgr.14118 172.21.15.43:0/568310870' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T04:54:40.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:39 smithi043 ceph-mon[30800]: Manager daemon x is now available 2024-09-19T04:54:40.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:39 smithi043 ceph-mon[30800]: from='mgr.14118 172.21.15.43:0/568310870' entity='mgr.x' 2024-09-19T04:54:40.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:39 smithi043 ceph-mon[30800]: from='mgr.14118 172.21.15.43:0/568310870' entity='mgr.x' 2024-09-19T04:54:40.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:39 smithi043 ceph-mon[30800]: from='mgr.14118 172.21.15.43:0/568310870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:54:40.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:39 smithi043 ceph-mon[30800]: from='mgr.14118 172.21.15.43:0/568310870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:54:40.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:40 smithi043 ceph-mon[30800]: Found migration_current of "None". Setting to last migration. 2024-09-19T04:54:40.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:40 smithi043 ceph-mon[30800]: from='mgr.14118 172.21.15.43:0/568310870' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T04:54:40.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:40 smithi043 ceph-mon[30800]: from='mgr.14118 172.21.15.43:0/568310870' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T04:54:40.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:40 smithi043 ceph-mon[30800]: from='mgr.14118 172.21.15.43:0/568310870' entity='mgr.x' 2024-09-19T04:54:40.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:40 smithi043 ceph-mon[30800]: from='mgr.14118 172.21.15.43:0/568310870' entity='mgr.x' 2024-09-19T04:54:40.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:40 smithi043 ceph-mon[30800]: mgrmap e7: x(active, since 1.00667s) 2024-09-19T04:54:41.585 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-09-19T04:54:41.585 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-19T04:54:41.585 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-19T04:54:41.585 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-09-19T04:54:41.585 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:36.017+0000 7fb66f200640 1 Processor -- start 2024-09-19T04:54:41.585 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:36.018+0000 7fb66f200640 1 -- start start 2024-09-19T04:54:41.585 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:36.018+0000 7fb66f200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb67007de40 0x7fb67007c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:41.585 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:36.018+0000 7fb66f200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb67007c880 con 0x7fb67007de40 2024-09-19T04:54:41.585 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:36.018+0000 7fb66de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb67007de40 0x7fb67007c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:41.585 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:36.018+0000 7fb66de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb67007de40 0x7fb67007c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32994/0 (socket says 172.21.15.43:32994) 2024-09-19T04:54:41.586 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:36.018+0000 7fb66de00640 1 -- 172.21.15.43:0/367402048 learned_addr learned my addr 172.21.15.43:0/367402048 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:54:41.586 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:36.019+0000 7fb66de00640 1 -- 172.21.15.43:0/367402048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb67007ca00 con 0x7fb67007de40 2024-09-19T04:54:41.586 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:36.020+0000 7fb66de00640 1 --2- 172.21.15.43:0/367402048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb67007de40 0x7fb67007c340 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb660009820 tx=0x7fb66002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc501b388d1929e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:41.586 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:36.020+0000 7fb66ca00640 1 -- 172.21.15.43:0/367402048 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb660037070 con 0x7fb67007de40 2024-09-19T04:54:41.586 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:36.020+0000 7fb66ca00640 1 -- 172.21.15.43:0/367402048 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb660032900 con 0x7fb67007de40 2024-09-19T04:54:41.586 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:36.020+0000 7fb66ca00640 1 -- 172.21.15.43:0/367402048 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb660030850 con 0x7fb67007de40 2024-09-19T04:54:41.586 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:36.021+0000 7fb66f200640 1 -- 172.21.15.43:0/367402048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb67007de40 msgr2=0x7fb67007c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:41.586 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:36.021+0000 7fb66f200640 1 --2- 172.21.15.43:0/367402048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb67007de40 0x7fb67007c340 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb660009820 tx=0x7fb66002a270 comp rx=0 tx=0).stop 2024-09-19T04:54:41.586 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:36.021+0000 7fb66f200640 1 -- 172.21.15.43:0/367402048 shutdown_connections 2024-09-19T04:54:41.586 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:36.021+0000 7fb66f200640 1 --2- 172.21.15.43:0/367402048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb67007de40 0x7fb67007c340 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:41.586 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:36.021+0000 7fb66f200640 1 -- 172.21.15.43:0/367402048 >> 172.21.15.43:0/367402048 conn(0x7fb670102d90 msgr2=0x7fb6701051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:41.586 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:36.021+0000 7fb66f200640 1 -- 172.21.15.43:0/367402048 shutdown_connections 2024-09-19T04:54:41.587 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:36.021+0000 7fb66f200640 1 -- 172.21.15.43:0/367402048 wait complete. 2024-09-19T04:54:41.587 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:36.022+0000 7fb66f200640 1 Processor -- start 2024-09-19T04:54:41.587 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:36.022+0000 7fb66f200640 1 -- start start 2024-09-19T04:54:41.587 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:36.022+0000 7fb66f200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb67007de40 0x7fb67019d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:41.587 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:36.022+0000 7fb66f200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb67010db90 con 0x7fb67007de40 2024-09-19T04:54:41.587 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:36.022+0000 7fb66de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb67007de40 0x7fb67019d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:41.587 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:36.022+0000 7fb66de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb67007de40 0x7fb67019d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33010/0 (socket says 172.21.15.43:33010) 2024-09-19T04:54:41.587 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:36.023+0000 7fb66de00640 1 -- 172.21.15.43:0/2310968267 learned_addr learned my addr 172.21.15.43:0/2310968267 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:54:41.587 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:36.023+0000 7fb66de00640 1 -- 172.21.15.43:0/2310968267 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb67019dc20 con 0x7fb67007de40 2024-09-19T04:54:41.587 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:36.023+0000 7fb66de00640 1 --2- 172.21.15.43:0/2310968267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb67007de40 0x7fb67019d6e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb660032ce0 tx=0x7fb660032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:41.587 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:36.023+0000 7fb666a00640 1 -- 172.21.15.43:0/2310968267 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb660042070 con 0x7fb67007de40 2024-09-19T04:54:41.587 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:36.023+0000 7fb666a00640 1 -- 172.21.15.43:0/2310968267 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb6600308c0 con 0x7fb67007de40 2024-09-19T04:54:41.587 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:36.024+0000 7fb666a00640 1 -- 172.21.15.43:0/2310968267 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb660037040 con 0x7fb67007de40 2024-09-19T04:54:41.588 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:36.024+0000 7fb66f200640 1 -- 172.21.15.43:0/2310968267 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb67019deb0 con 0x7fb67007de40 2024-09-19T04:54:41.588 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:36.024+0000 7fb66f200640 1 -- 172.21.15.43:0/2310968267 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6701a0ba0 con 0x7fb67007de40 2024-09-19T04:54:41.588 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:36.025+0000 7fb666a00640 1 -- 172.21.15.43:0/2310968267 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7fb660045070 con 0x7fb67007de40 2024-09-19T04:54:41.588 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:36.025+0000 7fb666a00640 1 --2- 172.21.15.43:0/2310968267 >> [v2:172.21.15.43:6800/587463881,v1:172.21.15.43:6801/587463881] conn(0x7fb654039200 0x7fb65403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:41.588 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:36.025+0000 7fb666a00640 1 -- 172.21.15.43:0/2310968267 --> [v2:172.21.15.43:6800/587463881,v1:172.21.15.43:6801/587463881] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb65403bdd0 con 0x7fb654039200 2024-09-19T04:54:41.588 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:36.026+0000 7fb666a00640 1 -- 172.21.15.43:0/2310968267 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb66006c330 con 0x7fb67007de40 2024-09-19T04:54:41.588 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:36.026+0000 7fb66d400640 1 -- 172.21.15.43:0/2310968267 >> [v2:172.21.15.43:6800/587463881,v1:172.21.15.43:6801/587463881] conn(0x7fb654039200 msgr2=0x7fb65403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/587463881 2024-09-19T04:54:41.588 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:36.026+0000 7fb66d400640 1 --2- 172.21.15.43:0/2310968267 >> [v2:172.21.15.43:6800/587463881,v1:172.21.15.43:6801/587463881] conn(0x7fb654039200 0x7fb65403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:54:41.588 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:36.227+0000 7fb66d400640 1 -- 172.21.15.43:0/2310968267 >> [v2:172.21.15.43:6800/587463881,v1:172.21.15.43:6801/587463881] conn(0x7fb654039200 msgr2=0x7fb65403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/587463881 2024-09-19T04:54:41.588 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:36.227+0000 7fb66d400640 1 --2- 172.21.15.43:0/2310968267 >> [v2:172.21.15.43:6800/587463881,v1:172.21.15.43:6801/587463881] conn(0x7fb654039200 0x7fb65403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T04:54:41.588 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:36.628+0000 7fb66d400640 1 -- 172.21.15.43:0/2310968267 >> [v2:172.21.15.43:6800/587463881,v1:172.21.15.43:6801/587463881] conn(0x7fb654039200 msgr2=0x7fb65403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/587463881 2024-09-19T04:54:41.588 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:36.628+0000 7fb66d400640 1 --2- 172.21.15.43:0/2310968267 >> [v2:172.21.15.43:6800/587463881,v1:172.21.15.43:6801/587463881] conn(0x7fb654039200 0x7fb65403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-19T04:54:41.589 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:37.429+0000 7fb66d400640 1 -- 172.21.15.43:0/2310968267 >> [v2:172.21.15.43:6800/587463881,v1:172.21.15.43:6801/587463881] conn(0x7fb654039200 msgr2=0x7fb65403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/587463881 2024-09-19T04:54:41.589 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:37.429+0000 7fb66d400640 1 --2- 172.21.15.43:0/2310968267 >> [v2:172.21.15.43:6800/587463881,v1:172.21.15.43:6801/587463881] conn(0x7fb654039200 0x7fb65403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-19T04:54:41.589 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:39.030+0000 7fb66d400640 1 -- 172.21.15.43:0/2310968267 >> [v2:172.21.15.43:6800/587463881,v1:172.21.15.43:6801/587463881] conn(0x7fb654039200 msgr2=0x7fb65403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/587463881 2024-09-19T04:54:41.589 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:39.030+0000 7fb66d400640 1 --2- 172.21.15.43:0/2310968267 >> [v2:172.21.15.43:6800/587463881,v1:172.21.15.43:6801/587463881] conn(0x7fb654039200 0x7fb65403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-09-19T04:54:41.589 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:39.549+0000 7fb666a00640 1 -- 172.21.15.43:0/2310968267 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mgrmap(e 6) ==== 46228+0+0 (secure 0 0 0) 0x7fb66003c720 con 0x7fb67007de40 2024-09-19T04:54:41.589 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:39.549+0000 7fb666a00640 1 -- 172.21.15.43:0/2310968267 >> [v2:172.21.15.43:6800/587463881,v1:172.21.15.43:6801/587463881] conn(0x7fb654039200 msgr2=0x7fb65403b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:54:41.589 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:39.549+0000 7fb666a00640 1 --2- 172.21.15.43:0/2310968267 >> [v2:172.21.15.43:6800/587463881,v1:172.21.15.43:6801/587463881] conn(0x7fb654039200 0x7fb65403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:41.589 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:40.551+0000 7fb666a00640 1 -- 172.21.15.43:0/2310968267 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7fb66006dca0 con 0x7fb67007de40 2024-09-19T04:54:41.589 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:40.551+0000 7fb666a00640 1 --2- 172.21.15.43:0/2310968267 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7fb65403cb30 0x7fb65403ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:41.589 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:40.551+0000 7fb666a00640 1 -- 172.21.15.43:0/2310968267 --> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb660065460 con 0x7fb65403cb30 2024-09-19T04:54:41.589 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:40.552+0000 7fb66d400640 1 --2- 172.21.15.43:0/2310968267 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7fb65403cb30 0x7fb65403ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:41.589 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:40.553+0000 7fb66d400640 1 --2- 172.21.15.43:0/2310968267 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7fb65403cb30 0x7fb65403ef20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb65c003980 tx=0x7fb65c0092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:41.590 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:40.554+0000 7fb666a00640 1 -- 172.21.15.43:0/2310968267 <== mgr.14118 v2:172.21.15.43:6800/3241799377 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fb660065460 con 0x7fb65403cb30 2024-09-19T04:54:41.590 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:40.561+0000 7fb66f200640 1 -- 172.21.15.43:0/2310968267 --> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fb67007c3c0 con 0x7fb65403cb30 2024-09-19T04:54:41.590 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:40.562+0000 7fb666a00640 1 -- 172.21.15.43:0/2310968267 <== mgr.14118 v2:172.21.15.43:6800/3241799377 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fb67007c3c0 con 0x7fb65403cb30 2024-09-19T04:54:41.590 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:40.562+0000 7fb66f200640 1 -- 172.21.15.43:0/2310968267 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7fb65403cb30 msgr2=0x7fb65403ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:41.590 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:40.562+0000 7fb66f200640 1 --2- 172.21.15.43:0/2310968267 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7fb65403cb30 0x7fb65403ef20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb65c003980 tx=0x7fb65c0092b0 comp rx=0 tx=0).stop 2024-09-19T04:54:41.590 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:40.562+0000 7fb66f200640 1 -- 172.21.15.43:0/2310968267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb67007de40 msgr2=0x7fb67019d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:41.590 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:40.562+0000 7fb66f200640 1 --2- 172.21.15.43:0/2310968267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb67007de40 0x7fb67019d6e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb660032ce0 tx=0x7fb660032d10 comp rx=0 tx=0).stop 2024-09-19T04:54:41.590 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:40.562+0000 7fb66f200640 1 -- 172.21.15.43:0/2310968267 shutdown_connections 2024-09-19T04:54:41.590 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:40.562+0000 7fb66f200640 1 --2- 172.21.15.43:0/2310968267 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7fb65403cb30 0x7fb65403ef20 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:41.590 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:40.562+0000 7fb66f200640 1 --2- 172.21.15.43:0/2310968267 >> [v2:172.21.15.43:6800/587463881,v1:172.21.15.43:6801/587463881] conn(0x7fb654039200 0x7fb65403b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:41.590 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:40.562+0000 7fb66f200640 1 --2- 172.21.15.43:0/2310968267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb67007de40 0x7fb67019d6e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:41.590 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:40.562+0000 7fb66f200640 1 -- 172.21.15.43:0/2310968267 >> 172.21.15.43:0/2310968267 conn(0x7fb670102d90 msgr2=0x7fb6701038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:41.591 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:40.562+0000 7fb66f200640 1 -- 172.21.15.43:0/2310968267 shutdown_connections 2024-09-19T04:54:41.591 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:40.562+0000 7fb66f200640 1 -- 172.21.15.43:0/2310968267 wait complete. 2024-09-19T04:54:41.591 INFO:teuthology.orchestra.run.smithi043.stdout:mgr epoch 5 is available 2024-09-19T04:54:41.591 INFO:teuthology.orchestra.run.smithi043.stdout:Setting orchestrator backend to cephadm... 2024-09-19T04:54:41.833 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:41 smithi043 ceph-mon[30800]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-19T04:54:41.833 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:41 smithi043 ceph-mon[30800]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-19T04:54:41.834 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:41 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:41.777+0000 7f9287000640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:04:54:41] ENGINE Error in HTTPServer.serve 2024-09-19T04:54:41.834 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:41 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: Traceback (most recent call last): 2024-09-19T04:54:41.834 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:41 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T04:54:41.834 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:41 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: self._connections.run(self.expiration_interval) 2024-09-19T04:54:41.834 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:41 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T04:54:41.834 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:41 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: self._run(expiration_interval) 2024-09-19T04:54:41.834 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:41 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T04:54:41.835 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:41 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T04:54:41.835 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:41 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T04:54:41.835 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:41 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T04:54:41.835 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:41 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T04:54:41.835 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:41 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: s = self.context.wrap_socket( 2024-09-19T04:54:41.835 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:41 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T04:54:41.835 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:41 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: return self.sslsocket_class._create( 2024-09-19T04:54:41.835 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:41 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T04:54:41.835 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:41 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: self.do_handshake() 2024-09-19T04:54:41.835 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:41 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T04:54:41.835 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:41 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: self._sslobj.do_handshake() 2024-09-19T04:54:41.835 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:41 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T04:54:41.835 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:41 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:54:42.953 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:42 smithi043 ceph-mon[30800]: mgrmap e8: x(active, since 2s) 2024-09-19T04:54:42.953 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:42 smithi043 ceph-mon[30800]: [19/Sep/2024:04:54:41] ENGINE Bus STARTING 2024-09-19T04:54:42.953 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:42 smithi043 ceph-mon[30800]: [19/Sep/2024:04:54:41] ENGINE Serving on https://172.21.15.43:7150 2024-09-19T04:54:42.953 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:42 smithi043 ceph-mon[30800]: [19/Sep/2024:04:54:41] ENGINE Error in HTTPServer.serve 2024-09-19T04:54:42.953 INFO:journalctl@ceph.mon.a.smithi043.stdout: Traceback (most recent call last): 2024-09-19T04:54:42.953 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T04:54:42.953 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._connections.run(self.expiration_interval) 2024-09-19T04:54:42.953 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T04:54:42.953 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._run(expiration_interval) 2024-09-19T04:54:42.954 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T04:54:42.954 INFO:journalctl@ceph.mon.a.smithi043.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T04:54:42.954 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T04:54:42.954 INFO:journalctl@ceph.mon.a.smithi043.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T04:54:42.954 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T04:54:42.954 INFO:journalctl@ceph.mon.a.smithi043.stdout: s = self.context.wrap_socket( 2024-09-19T04:54:42.954 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T04:54:42.954 INFO:journalctl@ceph.mon.a.smithi043.stdout: return self.sslsocket_class._create( 2024-09-19T04:54:42.954 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T04:54:42.954 INFO:journalctl@ceph.mon.a.smithi043.stdout: self.do_handshake() 2024-09-19T04:54:42.954 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T04:54:42.954 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._sslobj.do_handshake() 2024-09-19T04:54:42.954 INFO:journalctl@ceph.mon.a.smithi043.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T04:54:42.954 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:42 smithi043 ceph-mon[30800]: [19/Sep/2024:04:54:41] ENGINE Serving on http://172.21.15.43:8765 2024-09-19T04:54:42.955 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:42 smithi043 ceph-mon[30800]: [19/Sep/2024:04:54:41] ENGINE Bus STARTED 2024-09-19T04:54:42.955 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:42 smithi043 ceph-mon[30800]: from='mgr.14118 172.21.15.43:0/568310870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:54:42.955 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:42 smithi043 ceph-mon[30800]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:54:42.955 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:42 smithi043 ceph-mon[30800]: from='mgr.14118 172.21.15.43:0/568310870' entity='mgr.x' 2024-09-19T04:54:42.955 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:42 smithi043 ceph-mon[30800]: from='mgr.14118 172.21.15.43:0/568310870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:54:42.963 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.099+0000 7f3efb000640 1 Processor -- start 2024-09-19T04:54:42.963 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.099+0000 7f3efb000640 1 -- start start 2024-09-19T04:54:42.963 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.099+0000 7f3efb000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ef4107720 0x7f3ef4107b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:42.963 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.099+0000 7f3efb000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef41080f0 con 0x7f3ef4107720 2024-09-19T04:54:42.963 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.100+0000 7f3ef3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ef4107720 0x7f3ef4107b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:42.963 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.100+0000 7f3ef3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ef4107720 0x7f3ef4107b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33056/0 (socket says 172.21.15.43:33056) 2024-09-19T04:54:42.963 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.100+0000 7f3ef3e00640 1 -- 172.21.15.43:0/900942946 learned_addr learned my addr 172.21.15.43:0/900942946 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:54:42.963 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.100+0000 7f3ef3e00640 1 -- 172.21.15.43:0/900942946 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef41088d0 con 0x7f3ef4107720 2024-09-19T04:54:42.963 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.101+0000 7f3ef3e00640 1 --2- 172.21.15.43:0/900942946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ef4107720 0x7f3ef4107b20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009820 tx=0x7f3ee402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a8194ebf8aa2d0f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:42.963 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.101+0000 7f3ef3400640 1 -- 172.21.15.43:0/900942946 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ee4037070 con 0x7f3ef4107720 2024-09-19T04:54:42.964 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.101+0000 7f3ef3400640 1 -- 172.21.15.43:0/900942946 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3ee4032900 con 0x7f3ef4107720 2024-09-19T04:54:42.964 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.101+0000 7f3ef3400640 1 -- 172.21.15.43:0/900942946 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ee4030850 con 0x7f3ef4107720 2024-09-19T04:54:42.964 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.102+0000 7f3efb000640 1 -- 172.21.15.43:0/900942946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ef4107720 msgr2=0x7f3ef4107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:42.964 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.102+0000 7f3efb000640 1 --2- 172.21.15.43:0/900942946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ef4107720 0x7f3ef4107b20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009820 tx=0x7f3ee402a270 comp rx=0 tx=0).stop 2024-09-19T04:54:42.964 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.102+0000 7f3efb000640 1 -- 172.21.15.43:0/900942946 shutdown_connections 2024-09-19T04:54:42.964 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.102+0000 7f3efb000640 1 --2- 172.21.15.43:0/900942946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ef4107720 0x7f3ef4107b20 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:42.964 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.102+0000 7f3efb000640 1 -- 172.21.15.43:0/900942946 >> 172.21.15.43:0/900942946 conn(0x7f3ef4102ef0 msgr2=0x7f3ef4105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:42.964 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.102+0000 7f3efb000640 1 -- 172.21.15.43:0/900942946 shutdown_connections 2024-09-19T04:54:42.964 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.102+0000 7f3efb000640 1 -- 172.21.15.43:0/900942946 wait complete. 2024-09-19T04:54:42.964 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.103+0000 7f3efb000640 1 Processor -- start 2024-09-19T04:54:42.964 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.103+0000 7f3efb000640 1 -- start start 2024-09-19T04:54:42.964 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.103+0000 7f3efb000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ef4107720 0x7f3ef419d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:42.965 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.103+0000 7f3efb000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef410bc80 con 0x7f3ef4107720 2024-09-19T04:54:42.965 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.104+0000 7f3ef3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ef4107720 0x7f3ef419d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:42.965 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.104+0000 7f3ef3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ef4107720 0x7f3ef419d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33064/0 (socket says 172.21.15.43:33064) 2024-09-19T04:54:42.965 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.104+0000 7f3ef3e00640 1 -- 172.21.15.43:0/2654433894 learned_addr learned my addr 172.21.15.43:0/2654433894 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:54:42.965 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.104+0000 7f3ef3e00640 1 -- 172.21.15.43:0/2654433894 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef419de70 con 0x7f3ef4107720 2024-09-19T04:54:42.965 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.104+0000 7f3ef3e00640 1 --2- 172.21.15.43:0/2654433894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ef4107720 0x7f3ef419d930 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009940 tx=0x7f3ee4032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:42.965 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.105+0000 7f3ef1600640 1 -- 172.21.15.43:0/2654433894 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ee4042070 con 0x7f3ef4107720 2024-09-19T04:54:42.965 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.105+0000 7f3ef1600640 1 -- 172.21.15.43:0/2654433894 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3ee40308c0 con 0x7f3ef4107720 2024-09-19T04:54:42.965 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.105+0000 7f3ef1600640 1 -- 172.21.15.43:0/2654433894 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ee4037040 con 0x7f3ef4107720 2024-09-19T04:54:42.965 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.105+0000 7f3efb000640 1 -- 172.21.15.43:0/2654433894 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ef419e100 con 0x7f3ef4107720 2024-09-19T04:54:42.965 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.105+0000 7f3efb000640 1 -- 172.21.15.43:0/2654433894 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ef41a0df0 con 0x7f3ef4107720 2024-09-19T04:54:42.965 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.106+0000 7f3ef1600640 1 -- 172.21.15.43:0/2654433894 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f3ee4045070 con 0x7f3ef4107720 2024-09-19T04:54:42.965 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.106+0000 7f3efb000640 1 -- 172.21.15.43:0/2654433894 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ef410be00 con 0x7f3ef4107720 2024-09-19T04:54:42.966 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.106+0000 7f3ef1600640 1 --2- 172.21.15.43:0/2654433894 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f3ed80391b0 0x7f3ed803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:42.966 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.106+0000 7f3ef1600640 1 -- 172.21.15.43:0/2654433894 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3ee406c3f0 con 0x7f3ef4107720 2024-09-19T04:54:42.966 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.106+0000 7f3eebe00640 1 --2- 172.21.15.43:0/2654433894 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f3ed80391b0 0x7f3ed803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:42.966 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.107+0000 7f3eebe00640 1 --2- 172.21.15.43:0/2654433894 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f3ed80391b0 0x7f3ed803b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3edc0098c0 tx=0x7f3edc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:42.966 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.112+0000 7f3ef1600640 1 -- 172.21.15.43:0/2654433894 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3ee406bd40 con 0x7f3ef4107720 2024-09-19T04:54:42.966 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.312+0000 7f3efb000640 1 -- 172.21.15.43:0/2654433894 --> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f3ef41040b0 con 0x7f3ed80391b0 2024-09-19T04:54:42.966 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.318+0000 7f3ef1600640 1 -- 172.21.15.43:0/2654433894 <== mgr.14118 v2:172.21.15.43:6800/3241799377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3ef41040b0 con 0x7f3ed80391b0 2024-09-19T04:54:42.966 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.320+0000 7f3efb000640 1 -- 172.21.15.43:0/2654433894 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f3ed80391b0 msgr2=0x7f3ed803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:42.966 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.320+0000 7f3efb000640 1 --2- 172.21.15.43:0/2654433894 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f3ed80391b0 0x7f3ed803b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3edc0098c0 tx=0x7f3edc006db0 comp rx=0 tx=0).stop 2024-09-19T04:54:42.966 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.320+0000 7f3efb000640 1 -- 172.21.15.43:0/2654433894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ef4107720 msgr2=0x7f3ef419d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:42.966 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.320+0000 7f3efb000640 1 --2- 172.21.15.43:0/2654433894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ef4107720 0x7f3ef419d930 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009940 tx=0x7f3ee4032ce0 comp rx=0 tx=0).stop 2024-09-19T04:54:42.966 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.321+0000 7f3efb000640 1 -- 172.21.15.43:0/2654433894 shutdown_connections 2024-09-19T04:54:42.966 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.321+0000 7f3efb000640 1 --2- 172.21.15.43:0/2654433894 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f3ed80391b0 0x7f3ed803b670 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:42.967 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.321+0000 7f3efb000640 1 --2- 172.21.15.43:0/2654433894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ef4107720 0x7f3ef419d930 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:42.967 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.321+0000 7f3efb000640 1 -- 172.21.15.43:0/2654433894 >> 172.21.15.43:0/2654433894 conn(0x7f3ef4102ef0 msgr2=0x7f3ef41039a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:42.967 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.321+0000 7f3efb000640 1 -- 172.21.15.43:0/2654433894 shutdown_connections 2024-09-19T04:54:42.967 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:42.321+0000 7f3efb000640 1 -- 172.21.15.43:0/2654433894 wait complete. 2024-09-19T04:54:44.295 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-19T04:54:44.295 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.453+0000 7f6caf200640 1 Processor -- start 2024-09-19T04:54:44.295 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.454+0000 7f6caf200640 1 -- start start 2024-09-19T04:54:44.295 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.454+0000 7f6caf200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb007de10 0x7f6cb007c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:44.295 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.454+0000 7f6caf200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb007c800 con 0x7f6cb007de10 2024-09-19T04:54:44.295 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.454+0000 7f6cade00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb007de10 0x7f6cb007c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:44.295 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.454+0000 7f6cade00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb007de10 0x7f6cb007c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33074/0 (socket says 172.21.15.43:33074) 2024-09-19T04:54:44.296 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.454+0000 7f6cade00640 1 -- 172.21.15.43:0/1567590227 learned_addr learned my addr 172.21.15.43:0/1567590227 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:54:44.296 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.455+0000 7f6cade00640 1 -- 172.21.15.43:0/1567590227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb007c980 con 0x7f6cb007de10 2024-09-19T04:54:44.296 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.456+0000 7f6cade00640 1 --2- 172.21.15.43:0/1567590227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb007de10 0x7f6cb007c2c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0009820 tx=0x7f6ca002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c7d26b246fae5f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:44.296 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.456+0000 7f6cad400640 1 -- 172.21.15.43:0/1567590227 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ca0037070 con 0x7f6cb007de10 2024-09-19T04:54:44.296 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.456+0000 7f6cad400640 1 -- 172.21.15.43:0/1567590227 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6ca0032900 con 0x7f6cb007de10 2024-09-19T04:54:44.296 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.456+0000 7f6cad400640 1 -- 172.21.15.43:0/1567590227 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ca0030850 con 0x7f6cb007de10 2024-09-19T04:54:44.296 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.457+0000 7f6caf200640 1 -- 172.21.15.43:0/1567590227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb007de10 msgr2=0x7f6cb007c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:44.296 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.457+0000 7f6caf200640 1 --2- 172.21.15.43:0/1567590227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb007de10 0x7f6cb007c2c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0009820 tx=0x7f6ca002a270 comp rx=0 tx=0).stop 2024-09-19T04:54:44.296 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.457+0000 7f6caf200640 1 -- 172.21.15.43:0/1567590227 shutdown_connections 2024-09-19T04:54:44.296 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.457+0000 7f6caf200640 1 --2- 172.21.15.43:0/1567590227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb007de10 0x7f6cb007c2c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:44.296 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.457+0000 7f6caf200640 1 -- 172.21.15.43:0/1567590227 >> 172.21.15.43:0/1567590227 conn(0x7f6cb0102cf0 msgr2=0x7f6cb0105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:44.296 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.457+0000 7f6caf200640 1 -- 172.21.15.43:0/1567590227 shutdown_connections 2024-09-19T04:54:44.297 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.457+0000 7f6caf200640 1 -- 172.21.15.43:0/1567590227 wait complete. 2024-09-19T04:54:44.297 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.458+0000 7f6caf200640 1 Processor -- start 2024-09-19T04:54:44.297 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.458+0000 7f6caf200640 1 -- start start 2024-09-19T04:54:44.297 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.458+0000 7f6caf200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb007de10 0x7f6cb019d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:44.297 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.458+0000 7f6caf200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb010dbb0 con 0x7f6cb007de10 2024-09-19T04:54:44.297 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.459+0000 7f6cade00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb007de10 0x7f6cb019d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:44.297 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.459+0000 7f6cade00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb007de10 0x7f6cb019d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33076/0 (socket says 172.21.15.43:33076) 2024-09-19T04:54:44.298 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.459+0000 7f6cade00640 1 -- 172.21.15.43:0/2594247101 learned_addr learned my addr 172.21.15.43:0/2594247101 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:54:44.298 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.459+0000 7f6cade00640 1 -- 172.21.15.43:0/2594247101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb019dc40 con 0x7f6cb007de10 2024-09-19T04:54:44.298 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.459+0000 7f6cade00640 1 --2- 172.21.15.43:0/2594247101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb007de10 0x7f6cb019d700 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6ca00097f0 tx=0x7f6ca0030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:44.298 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.459+0000 7f6ca6a00640 1 -- 172.21.15.43:0/2594247101 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ca0037040 con 0x7f6cb007de10 2024-09-19T04:54:44.298 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.459+0000 7f6ca6a00640 1 -- 172.21.15.43:0/2594247101 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6ca0040070 con 0x7f6cb007de10 2024-09-19T04:54:44.298 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.459+0000 7f6caf200640 1 -- 172.21.15.43:0/2594247101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cb019ded0 con 0x7f6cb007de10 2024-09-19T04:54:44.298 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.460+0000 7f6ca6a00640 1 -- 172.21.15.43:0/2594247101 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ca003a390 con 0x7f6cb007de10 2024-09-19T04:54:44.298 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.460+0000 7f6caf200640 1 -- 172.21.15.43:0/2594247101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cb01a0bc0 con 0x7f6cb007de10 2024-09-19T04:54:44.298 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.461+0000 7f6ca6a00640 1 -- 172.21.15.43:0/2594247101 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f6ca0032900 con 0x7f6cb007de10 2024-09-19T04:54:44.298 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.461+0000 7f6caf200640 1 -- 172.21.15.43:0/2594247101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cb007c340 con 0x7f6cb007de10 2024-09-19T04:54:44.299 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.461+0000 7f6ca6a00640 1 --2- 172.21.15.43:0/2594247101 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f6c94039200 0x7f6c9403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:44.299 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.461+0000 7f6ca6a00640 1 -- 172.21.15.43:0/2594247101 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6ca006c810 con 0x7f6cb007de10 2024-09-19T04:54:44.299 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.461+0000 7f6ca7e00640 1 --2- 172.21.15.43:0/2594247101 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f6c94039200 0x7f6c9403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:44.299 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.462+0000 7f6ca7e00640 1 --2- 172.21.15.43:0/2594247101 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f6c94039200 0x7f6c9403b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6c980098c0 tx=0x7f6c98006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:44.299 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.466+0000 7f6ca6a00640 1 -- 172.21.15.43:0/2594247101 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6ca006c160 con 0x7f6cb007de10 2024-09-19T04:54:44.299 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.654+0000 7f6caf200640 1 -- 172.21.15.43:0/2594247101 --> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f6cb0103fa0 con 0x7f6c94039200 2024-09-19T04:54:44.299 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.655+0000 7f6ca6a00640 1 -- 172.21.15.43:0/2594247101 <== mgr.14118 v2:172.21.15.43:6800/3241799377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f6cb0103fa0 con 0x7f6c94039200 2024-09-19T04:54:44.299 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.658+0000 7f6caf200640 1 -- 172.21.15.43:0/2594247101 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f6c94039200 msgr2=0x7f6c9403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:44.299 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.658+0000 7f6caf200640 1 --2- 172.21.15.43:0/2594247101 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f6c94039200 0x7f6c9403b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6c980098c0 tx=0x7f6c98006db0 comp rx=0 tx=0).stop 2024-09-19T04:54:44.299 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.658+0000 7f6caf200640 1 -- 172.21.15.43:0/2594247101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb007de10 msgr2=0x7f6cb019d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:44.299 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.658+0000 7f6caf200640 1 --2- 172.21.15.43:0/2594247101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb007de10 0x7f6cb019d700 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6ca00097f0 tx=0x7f6ca0030d80 comp rx=0 tx=0).stop 2024-09-19T04:54:44.299 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.658+0000 7f6caf200640 1 -- 172.21.15.43:0/2594247101 shutdown_connections 2024-09-19T04:54:44.300 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.658+0000 7f6caf200640 1 --2- 172.21.15.43:0/2594247101 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f6c94039200 0x7f6c9403b6c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:44.300 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.658+0000 7f6caf200640 1 --2- 172.21.15.43:0/2594247101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb007de10 0x7f6cb019d700 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:44.300 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.658+0000 7f6caf200640 1 -- 172.21.15.43:0/2594247101 >> 172.21.15.43:0/2594247101 conn(0x7f6cb0102cf0 msgr2=0x7f6cb0103890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:44.300 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.658+0000 7f6caf200640 1 -- 172.21.15.43:0/2594247101 shutdown_connections 2024-09-19T04:54:44.300 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:43.658+0000 7f6caf200640 1 -- 172.21.15.43:0/2594247101 wait complete. 2024-09-19T04:54:44.300 INFO:teuthology.orchestra.run.smithi043.stdout:Generating ssh key... 2024-09-19T04:54:44.841 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:44 smithi043 ceph-mon[30800]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:54:45.091 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:45 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: Generating public/private rsa key pair. 2024-09-19T04:54:45.092 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:45 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: Your identification has been saved in /tmp/tmpxaqoqszk/key 2024-09-19T04:54:45.092 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:45 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: Your public key has been saved in /tmp/tmpxaqoqszk/key.pub 2024-09-19T04:54:45.092 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:45 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: The key fingerprint is: 2024-09-19T04:54:45.092 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:45 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: SHA256:grguUwQzcNL/mTQdvsy/zNRoZKjzVa6Qjh9o6Ql/WHE ceph-1456a810-7643-11ef-bceb-c7b262605968 2024-09-19T04:54:45.092 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:45 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: The key's randomart image is: 2024-09-19T04:54:45.092 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:45 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: +---[RSA 3072]----+ 2024-09-19T04:54:45.092 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:45 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: |+.. | 2024-09-19T04:54:45.092 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:45 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: |+o. . | 2024-09-19T04:54:45.426 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:45 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: | + . o . | 2024-09-19T04:54:45.426 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:45 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: | ....o +.E | 2024-09-19T04:54:45.426 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:45 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: | .. .o.*S+o . | 2024-09-19T04:54:45.426 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:45 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: | .. +==+ = | 2024-09-19T04:54:45.426 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:45 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: | .. . *o+.= o | 2024-09-19T04:54:45.426 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:45 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: |o. =.*.O.. | 2024-09-19T04:54:45.427 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:45 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: | o. =o+ =. | 2024-09-19T04:54:45.427 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:54:45 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: +----[SHA256]-----+ 2024-09-19T04:54:45.735 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:44.826+0000 7f202fa00640 1 Processor -- start 2024-09-19T04:54:45.735 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:44.826+0000 7f202fa00640 1 -- start start 2024-09-19T04:54:45.735 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:44.827+0000 7f202fa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f202807de70 0x7f202807c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:45.735 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:44.827+0000 7f202fa00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f202807c8f0 con 0x7f202807de70 2024-09-19T04:54:45.735 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:44.827+0000 7f202d000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f202807de70 0x7f202807c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:45.735 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:44.827+0000 7f202d000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f202807de70 0x7f202807c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33092/0 (socket says 172.21.15.43:33092) 2024-09-19T04:54:45.735 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:44.827+0000 7f202d000640 1 -- 172.21.15.43:0/1845743641 learned_addr learned my addr 172.21.15.43:0/1845743641 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:54:45.735 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:44.827+0000 7f202d000640 1 -- 172.21.15.43:0/1845743641 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f202807ca70 con 0x7f202807de70 2024-09-19T04:54:45.735 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:44.828+0000 7f202d000640 1 --2- 172.21.15.43:0/1845743641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f202807de70 0x7f202807c320 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2018009820 tx=0x7f201802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d16f37f873df707 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:45.735 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:44.829+0000 7f2027e00640 1 -- 172.21.15.43:0/1845743641 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2018037070 con 0x7f202807de70 2024-09-19T04:54:45.736 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:44.829+0000 7f2027e00640 1 -- 172.21.15.43:0/1845743641 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2018032900 con 0x7f202807de70 2024-09-19T04:54:45.736 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:44.829+0000 7f2027e00640 1 -- 172.21.15.43:0/1845743641 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2018030850 con 0x7f202807de70 2024-09-19T04:54:45.736 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:44.829+0000 7f202fa00640 1 -- 172.21.15.43:0/1845743641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f202807de70 msgr2=0x7f202807c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:45.736 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:44.829+0000 7f202fa00640 1 --2- 172.21.15.43:0/1845743641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f202807de70 0x7f202807c320 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2018009820 tx=0x7f201802a270 comp rx=0 tx=0).stop 2024-09-19T04:54:45.736 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:44.829+0000 7f202fa00640 1 -- 172.21.15.43:0/1845743641 shutdown_connections 2024-09-19T04:54:45.736 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:44.829+0000 7f202fa00640 1 --2- 172.21.15.43:0/1845743641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f202807de70 0x7f202807c320 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:45.736 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:44.829+0000 7f202fa00640 1 -- 172.21.15.43:0/1845743641 >> 172.21.15.43:0/1845743641 conn(0x7f2028102ef0 msgr2=0x7f2028105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:45.736 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:44.829+0000 7f202fa00640 1 -- 172.21.15.43:0/1845743641 shutdown_connections 2024-09-19T04:54:45.736 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:44.829+0000 7f202fa00640 1 -- 172.21.15.43:0/1845743641 wait complete. 2024-09-19T04:54:45.736 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:44.830+0000 7f202fa00640 1 Processor -- start 2024-09-19T04:54:45.736 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:44.831+0000 7f202fa00640 1 -- start start 2024-09-19T04:54:45.736 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:44.831+0000 7f202fa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f202807de70 0x7f202819d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:45.736 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:44.831+0000 7f202fa00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f202810dd30 con 0x7f202807de70 2024-09-19T04:54:45.737 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:44.831+0000 7f202d000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f202807de70 0x7f202819d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:45.737 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:44.831+0000 7f202d000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f202807de70 0x7f202819d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33098/0 (socket says 172.21.15.43:33098) 2024-09-19T04:54:45.737 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:44.831+0000 7f202d000640 1 -- 172.21.15.43:0/77369753 learned_addr learned my addr 172.21.15.43:0/77369753 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:54:45.737 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:44.831+0000 7f202d000640 1 -- 172.21.15.43:0/77369753 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f202819de40 con 0x7f202807de70 2024-09-19T04:54:45.737 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:44.832+0000 7f202d000640 1 --2- 172.21.15.43:0/77369753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f202807de70 0x7f202819d900 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f20180097f0 tx=0x7f2018030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:45.737 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:44.832+0000 7f2026000640 1 -- 172.21.15.43:0/77369753 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2018042070 con 0x7f202807de70 2024-09-19T04:54:45.737 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:44.832+0000 7f2026000640 1 -- 172.21.15.43:0/77369753 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2018032ca0 con 0x7f202807de70 2024-09-19T04:54:45.737 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:44.832+0000 7f2026000640 1 -- 172.21.15.43:0/77369753 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2018037040 con 0x7f202807de70 2024-09-19T04:54:45.737 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:44.832+0000 7f202fa00640 1 -- 172.21.15.43:0/77369753 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f202819e0d0 con 0x7f202807de70 2024-09-19T04:54:45.737 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:44.832+0000 7f202fa00640 1 -- 172.21.15.43:0/77369753 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20281a0dc0 con 0x7f202807de70 2024-09-19T04:54:45.737 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:44.833+0000 7f2026000640 1 -- 172.21.15.43:0/77369753 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f2018045070 con 0x7f202807de70 2024-09-19T04:54:45.737 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:44.833+0000 7f202fa00640 1 -- 172.21.15.43:0/77369753 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f202807c3a0 con 0x7f202807de70 2024-09-19T04:54:45.737 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:44.834+0000 7f2026000640 1 --2- 172.21.15.43:0/77369753 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f200c039200 0x7f200c03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:45.738 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:44.834+0000 7f2026000640 1 -- 172.21.15.43:0/77369753 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f201806c2d0 con 0x7f202807de70 2024-09-19T04:54:45.738 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:44.834+0000 7f201fe00640 1 --2- 172.21.15.43:0/77369753 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f200c039200 0x7f200c03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:45.738 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:44.835+0000 7f201fe00640 1 --2- 172.21.15.43:0/77369753 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f200c039200 0x7f200c03b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f20100098c0 tx=0x7f2010006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:45.738 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:44.839+0000 7f2026000640 1 -- 172.21.15.43:0/77369753 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2018045360 con 0x7f202807de70 2024-09-19T04:54:45.738 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:45.027+0000 7f202fa00640 1 -- 172.21.15.43:0/77369753 --> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f2028104190 con 0x7f200c039200 2024-09-19T04:54:45.738 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:45.097+0000 7f2026000640 1 -- 172.21.15.43:0/77369753 <== mgr.14118 v2:172.21.15.43:6800/3241799377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2028104190 con 0x7f200c039200 2024-09-19T04:54:45.738 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:45.100+0000 7f202fa00640 1 -- 172.21.15.43:0/77369753 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f200c039200 msgr2=0x7f200c03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:45.738 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:45.100+0000 7f202fa00640 1 --2- 172.21.15.43:0/77369753 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f200c039200 0x7f200c03b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f20100098c0 tx=0x7f2010006db0 comp rx=0 tx=0).stop 2024-09-19T04:54:45.738 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:45.100+0000 7f202fa00640 1 -- 172.21.15.43:0/77369753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f202807de70 msgr2=0x7f202819d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:45.738 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:45.100+0000 7f202fa00640 1 --2- 172.21.15.43:0/77369753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f202807de70 0x7f202819d900 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f20180097f0 tx=0x7f2018030ec0 comp rx=0 tx=0).stop 2024-09-19T04:54:45.738 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:45.100+0000 7f202fa00640 1 -- 172.21.15.43:0/77369753 shutdown_connections 2024-09-19T04:54:45.738 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:45.100+0000 7f202fa00640 1 --2- 172.21.15.43:0/77369753 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f200c039200 0x7f200c03b6c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:45.738 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:45.100+0000 7f202fa00640 1 --2- 172.21.15.43:0/77369753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f202807de70 0x7f202819d900 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:45.738 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:45.100+0000 7f202fa00640 1 -- 172.21.15.43:0/77369753 >> 172.21.15.43:0/77369753 conn(0x7f2028102ef0 msgr2=0x7f2028103a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:45.739 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:45.100+0000 7f202fa00640 1 -- 172.21.15.43:0/77369753 shutdown_connections 2024-09-19T04:54:45.739 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:45.100+0000 7f202fa00640 1 -- 172.21.15.43:0/77369753 wait complete. 2024-09-19T04:54:46.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:46 smithi043 ceph-mon[30800]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:54:46.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:46 smithi043 ceph-mon[30800]: Generating ssh key... 2024-09-19T04:54:46.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:46 smithi043 ceph-mon[30800]: from='mgr.14118 172.21.15.43:0/568310870' entity='mgr.x' 2024-09-19T04:54:46.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:46 smithi043 ceph-mon[30800]: from='mgr.14118 172.21.15.43:0/568310870' entity='mgr.x' 2024-09-19T04:54:47.065 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCjTG2qRtuHcxvKcvb7xgsggkW7AmLgD/NygkD0vY+IXyL1Z05y7l+TN/YThYUnBJnfZWwmrKTWWpjskXoODzpLa+JVWMiXc1hdbrYpvAqbgUza1Y7Xx7z/GKpZn7BoghAPLj4tI+JYRKml9DzXr0Tsud4MVDcAu6/xkODaOlG4gsjqqNIQmIZ1jpynRiygFz6ivSgwhsz3gEdpqHnppykroCJLjBrnvNW1VxrKpUcZmYjuRieDzC2u2BsiawGXAOQGx+0gijZXctZNYSzo1yRxgf7r9WlPoL+HbjOVH/4gqmwyId9Kv4UTQ1TKXstsypnVMtfZhLgQt/sEOmieoMIbIYZ8S/8TUi3vMtGWSfMRHZCDDiWgqAKRNLr4ARgk3RopIBzBezPdURqTyi6LpJS7n+zkKpFmpCJMR2B59APTNbYFCVbloXzUh1HW18dVly8H9v49uFecc4Bef0I2IuggJyZx6jAe6Mo1ekThMs8ibdtPkX8v01gZJzBW9CEGQGs= ceph-1456a810-7643-11ef-bceb-c7b262605968 2024-09-19T04:54:47.065 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.208+0000 7f5994e00640 1 Processor -- start 2024-09-19T04:54:47.065 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.208+0000 7f5994e00640 1 -- start start 2024-09-19T04:54:47.065 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.208+0000 7f5994e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5990109960 0x7f5990109d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:47.065 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.208+0000 7f5994e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f599010a330 con 0x7f5990109960 2024-09-19T04:54:47.065 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.209+0000 7f598de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5990109960 0x7f5990109d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:47.065 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.209+0000 7f598de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5990109960 0x7f5990109d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37980/0 (socket says 172.21.15.43:37980) 2024-09-19T04:54:47.066 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.209+0000 7f598de00640 1 -- 172.21.15.43:0/751470962 learned_addr learned my addr 172.21.15.43:0/751470962 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:54:47.066 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.209+0000 7f598de00640 1 -- 172.21.15.43:0/751470962 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f599010aad0 con 0x7f5990109960 2024-09-19T04:54:47.066 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.210+0000 7f598de00640 1 --2- 172.21.15.43:0/751470962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5990109960 0x7f5990109d60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5980009820 tx=0x7f598002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf36a0f8ba2cc44e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:47.066 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.210+0000 7f598ca00640 1 -- 172.21.15.43:0/751470962 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5980037070 con 0x7f5990109960 2024-09-19T04:54:47.066 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.210+0000 7f598ca00640 1 -- 172.21.15.43:0/751470962 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5980032900 con 0x7f5990109960 2024-09-19T04:54:47.066 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.210+0000 7f598ca00640 1 -- 172.21.15.43:0/751470962 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5980030850 con 0x7f5990109960 2024-09-19T04:54:47.066 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.211+0000 7f5994e00640 1 -- 172.21.15.43:0/751470962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5990109960 msgr2=0x7f5990109d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:47.066 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.211+0000 7f5994e00640 1 --2- 172.21.15.43:0/751470962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5990109960 0x7f5990109d60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5980009820 tx=0x7f598002a270 comp rx=0 tx=0).stop 2024-09-19T04:54:47.066 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.211+0000 7f5994e00640 1 -- 172.21.15.43:0/751470962 shutdown_connections 2024-09-19T04:54:47.066 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.211+0000 7f5994e00640 1 --2- 172.21.15.43:0/751470962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5990109960 0x7f5990109d60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:47.066 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.211+0000 7f5994e00640 1 -- 172.21.15.43:0/751470962 >> 172.21.15.43:0/751470962 conn(0x7f599007cf30 msgr2=0x7f599007d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:47.066 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.211+0000 7f5994e00640 1 -- 172.21.15.43:0/751470962 shutdown_connections 2024-09-19T04:54:47.066 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.211+0000 7f5994e00640 1 -- 172.21.15.43:0/751470962 wait complete. 2024-09-19T04:54:47.067 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.212+0000 7f5994e00640 1 Processor -- start 2024-09-19T04:54:47.067 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.212+0000 7f5994e00640 1 -- start start 2024-09-19T04:54:47.067 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.212+0000 7f5994e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5990109960 0x7f599019d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:47.067 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.212+0000 7f5994e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f599010de80 con 0x7f5990109960 2024-09-19T04:54:47.067 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.212+0000 7f598de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5990109960 0x7f599019d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:47.067 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.212+0000 7f598de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5990109960 0x7f599019d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37990/0 (socket says 172.21.15.43:37990) 2024-09-19T04:54:47.067 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.212+0000 7f598de00640 1 -- 172.21.15.43:0/3450182124 learned_addr learned my addr 172.21.15.43:0/3450182124 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:54:47.067 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.213+0000 7f598de00640 1 -- 172.21.15.43:0/3450182124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f599019de40 con 0x7f5990109960 2024-09-19T04:54:47.067 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.213+0000 7f598de00640 1 --2- 172.21.15.43:0/3450182124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5990109960 0x7f599019d900 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f59800097f0 tx=0x7f5980030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:47.067 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.213+0000 7f5986a00640 1 -- 172.21.15.43:0/3450182124 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5980042070 con 0x7f5990109960 2024-09-19T04:54:47.067 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.213+0000 7f5986a00640 1 -- 172.21.15.43:0/3450182124 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5980032ca0 con 0x7f5990109960 2024-09-19T04:54:47.067 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.213+0000 7f5986a00640 1 -- 172.21.15.43:0/3450182124 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5980037040 con 0x7f5990109960 2024-09-19T04:54:47.067 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.213+0000 7f5994e00640 1 -- 172.21.15.43:0/3450182124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f599019e0d0 con 0x7f5990109960 2024-09-19T04:54:47.068 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.213+0000 7f5994e00640 1 -- 172.21.15.43:0/3450182124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59901a0dc0 con 0x7f5990109960 2024-09-19T04:54:47.068 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.214+0000 7f5986a00640 1 -- 172.21.15.43:0/3450182124 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f5980045070 con 0x7f5990109960 2024-09-19T04:54:47.068 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.214+0000 7f5994e00640 1 -- 172.21.15.43:0/3450182124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f599010e000 con 0x7f5990109960 2024-09-19T04:54:47.068 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.215+0000 7f5986a00640 1 --2- 172.21.15.43:0/3450182124 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f5974039200 0x7f597403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:47.068 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.215+0000 7f5986a00640 1 -- 172.21.15.43:0/3450182124 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f598006c230 con 0x7f5990109960 2024-09-19T04:54:47.068 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.215+0000 7f598d400640 1 --2- 172.21.15.43:0/3450182124 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f5974039200 0x7f597403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:47.068 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.216+0000 7f598d400640 1 --2- 172.21.15.43:0/3450182124 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f5974039200 0x7f597403b6c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f597c0098c0 tx=0x7f597c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:47.068 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.221+0000 7f5986a00640 1 -- 172.21.15.43:0/3450182124 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f598004b170 con 0x7f5990109960 2024-09-19T04:54:47.068 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.411+0000 7f5994e00640 1 -- 172.21.15.43:0/3450182124 --> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f5990107c20 con 0x7f5974039200 2024-09-19T04:54:47.068 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.411+0000 7f5986a00640 1 -- 172.21.15.43:0/3450182124 <== mgr.14118 v2:172.21.15.43:6800/3241799377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f5990107c20 con 0x7f5974039200 2024-09-19T04:54:47.068 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.414+0000 7f5994e00640 1 -- 172.21.15.43:0/3450182124 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f5974039200 msgr2=0x7f597403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:47.068 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.414+0000 7f5994e00640 1 --2- 172.21.15.43:0/3450182124 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f5974039200 0x7f597403b6c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f597c0098c0 tx=0x7f597c006db0 comp rx=0 tx=0).stop 2024-09-19T04:54:47.069 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.414+0000 7f5994e00640 1 -- 172.21.15.43:0/3450182124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5990109960 msgr2=0x7f599019d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:47.069 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.414+0000 7f5994e00640 1 --2- 172.21.15.43:0/3450182124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5990109960 0x7f599019d900 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f59800097f0 tx=0x7f5980030ec0 comp rx=0 tx=0).stop 2024-09-19T04:54:47.069 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.414+0000 7f5994e00640 1 -- 172.21.15.43:0/3450182124 shutdown_connections 2024-09-19T04:54:47.069 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.414+0000 7f5994e00640 1 --2- 172.21.15.43:0/3450182124 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f5974039200 0x7f597403b6c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:47.069 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.414+0000 7f5994e00640 1 --2- 172.21.15.43:0/3450182124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5990109960 0x7f599019d900 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:47.069 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.414+0000 7f5994e00640 1 -- 172.21.15.43:0/3450182124 >> 172.21.15.43:0/3450182124 conn(0x7f599007cf30 msgr2=0x7f5990107510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:47.069 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.414+0000 7f5994e00640 1 -- 172.21.15.43:0/3450182124 shutdown_connections 2024-09-19T04:54:47.069 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:46.414+0000 7f5994e00640 1 -- 172.21.15.43:0/3450182124 wait complete. 2024-09-19T04:54:47.069 INFO:teuthology.orchestra.run.smithi043.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-19T04:54:47.069 INFO:teuthology.orchestra.run.smithi043.stdout:Adding key to root@localhost authorized_keys... 2024-09-19T04:54:47.070 INFO:teuthology.orchestra.run.smithi043.stdout:Adding host smithi043... 2024-09-19T04:54:47.329 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:47 smithi043 ceph-mon[30800]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:54:48.920 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:48 smithi043 ceph-mon[30800]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi043", "addr": "172.21.15.43", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:54:50.021 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:49 smithi043 ceph-mon[30800]: Deploying cephadm binary to smithi043 2024-09-19T04:54:52.965 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout Added host 'smithi043' with addr '172.21.15.43' 2024-09-19T04:54:52.965 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:47.544+0000 7f8a42800640 1 Processor -- start 2024-09-19T04:54:52.965 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:47.544+0000 7f8a42800640 1 -- start start 2024-09-19T04:54:52.965 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:47.545+0000 7f8a42800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a3c108220 0x7f8a3c10a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:52.966 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:47.545+0000 7f8a42800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c07bdf0 con 0x7f8a3c108220 2024-09-19T04:54:52.966 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:47.545+0000 7f8a3b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a3c108220 0x7f8a3c10a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:52.966 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:47.545+0000 7f8a3b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a3c108220 0x7f8a3c10a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37998/0 (socket says 172.21.15.43:37998) 2024-09-19T04:54:52.966 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:47.545+0000 7f8a3b400640 1 -- 172.21.15.43:0/3926097961 learned_addr learned my addr 172.21.15.43:0/3926097961 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:54:52.966 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:47.545+0000 7f8a3b400640 1 -- 172.21.15.43:0/3926097961 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a3c10ab50 con 0x7f8a3c108220 2024-09-19T04:54:52.966 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:47.546+0000 7f8a3b400640 1 --2- 172.21.15.43:0/3926097961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a3c108220 0x7f8a3c10a610 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c009820 tx=0x7f8a2c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69652d803d895b5a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:52.966 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:47.547+0000 7f8a3aa00640 1 -- 172.21.15.43:0/3926097961 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a2c037070 con 0x7f8a3c108220 2024-09-19T04:54:52.966 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:47.547+0000 7f8a3aa00640 1 -- 172.21.15.43:0/3926097961 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8a2c032900 con 0x7f8a3c108220 2024-09-19T04:54:52.966 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:47.547+0000 7f8a3aa00640 1 -- 172.21.15.43:0/3926097961 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a2c030850 con 0x7f8a3c108220 2024-09-19T04:54:52.966 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:47.547+0000 7f8a42800640 1 -- 172.21.15.43:0/3926097961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a3c108220 msgr2=0x7f8a3c10a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:52.966 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:47.547+0000 7f8a42800640 1 --2- 172.21.15.43:0/3926097961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a3c108220 0x7f8a3c10a610 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c009820 tx=0x7f8a2c02a270 comp rx=0 tx=0).stop 2024-09-19T04:54:52.967 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:47.547+0000 7f8a42800640 1 -- 172.21.15.43:0/3926097961 shutdown_connections 2024-09-19T04:54:52.967 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:47.547+0000 7f8a42800640 1 --2- 172.21.15.43:0/3926097961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a3c108220 0x7f8a3c10a610 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:52.967 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:47.547+0000 7f8a42800640 1 -- 172.21.15.43:0/3926097961 >> 172.21.15.43:0/3926097961 conn(0x7f8a3c101e20 msgr2=0x7f8a3c104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:52.967 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:47.547+0000 7f8a42800640 1 -- 172.21.15.43:0/3926097961 shutdown_connections 2024-09-19T04:54:52.967 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:47.547+0000 7f8a42800640 1 -- 172.21.15.43:0/3926097961 wait complete. 2024-09-19T04:54:52.967 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:47.548+0000 7f8a42800640 1 Processor -- start 2024-09-19T04:54:52.967 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:47.549+0000 7f8a42800640 1 -- start start 2024-09-19T04:54:52.967 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:47.549+0000 7f8a42800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a3c108220 0x7f8a3c107aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:52.967 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:47.549+0000 7f8a42800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c10bb30 con 0x7f8a3c108220 2024-09-19T04:54:52.967 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:47.549+0000 7f8a3b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a3c108220 0x7f8a3c107aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:52.967 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:47.549+0000 7f8a3b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a3c108220 0x7f8a3c107aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38008/0 (socket says 172.21.15.43:38008) 2024-09-19T04:54:52.967 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:47.549+0000 7f8a3b400640 1 -- 172.21.15.43:0/3127999279 learned_addr learned my addr 172.21.15.43:0/3127999279 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:54:52.967 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:47.550+0000 7f8a3b400640 1 -- 172.21.15.43:0/3127999279 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a3c107fe0 con 0x7f8a3c108220 2024-09-19T04:54:52.968 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:47.550+0000 7f8a3b400640 1 --2- 172.21.15.43:0/3127999279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a3c108220 0x7f8a3c107aa0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c0097f0 tx=0x7f8a2c030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:52.968 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:47.550+0000 7f8a38c00640 1 -- 172.21.15.43:0/3127999279 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a2c042070 con 0x7f8a3c108220 2024-09-19T04:54:52.968 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:47.550+0000 7f8a38c00640 1 -- 172.21.15.43:0/3127999279 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8a2c032ca0 con 0x7f8a3c108220 2024-09-19T04:54:52.968 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:47.550+0000 7f8a38c00640 1 -- 172.21.15.43:0/3127999279 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a2c037040 con 0x7f8a3c108220 2024-09-19T04:54:52.968 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:47.550+0000 7f8a42800640 1 -- 172.21.15.43:0/3127999279 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a3c1061d0 con 0x7f8a3c108220 2024-09-19T04:54:52.968 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:47.550+0000 7f8a42800640 1 -- 172.21.15.43:0/3127999279 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a3c106710 con 0x7f8a3c108220 2024-09-19T04:54:52.968 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:47.551+0000 7f8a31600640 1 -- 172.21.15.43:0/3127999279 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a3c108680 con 0x7f8a3c108220 2024-09-19T04:54:52.968 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:47.552+0000 7f8a38c00640 1 -- 172.21.15.43:0/3127999279 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f8a2c045070 con 0x7f8a3c108220 2024-09-19T04:54:52.968 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:47.552+0000 7f8a38c00640 1 --2- 172.21.15.43:0/3127999279 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f8a20039200 0x7f8a2003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:52.968 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:47.552+0000 7f8a38c00640 1 -- 172.21.15.43:0/3127999279 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8a2c06c1f0 con 0x7f8a3c108220 2024-09-19T04:54:52.968 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:47.553+0000 7f8a32a00640 1 --2- 172.21.15.43:0/3127999279 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f8a20039200 0x7f8a2003b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:52.968 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:47.553+0000 7f8a32a00640 1 --2- 172.21.15.43:0/3127999279 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f8a20039200 0x7f8a2003b6c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8a240098c0 tx=0x7f8a24006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:52.968 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:47.558+0000 7f8a38c00640 1 -- 172.21.15.43:0/3127999279 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8a2c045360 con 0x7f8a3c108220 2024-09-19T04:54:52.969 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:47.747+0000 7f8a31600640 1 -- 172.21.15.43:0/3127999279 --> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi043", "addr": "172.21.15.43", "target": ["mon-mgr", ""]}) -- 0x7f8a3c106a50 con 0x7f8a20039200 2024-09-19T04:54:52.969 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:52.253+0000 7f8a38c00640 1 -- 172.21.15.43:0/3127999279 <== mgr.14118 v2:172.21.15.43:6800/3241799377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f8a3c106a50 con 0x7f8a20039200 2024-09-19T04:54:52.969 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:52.257+0000 7f8a31600640 1 -- 172.21.15.43:0/3127999279 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f8a20039200 msgr2=0x7f8a2003b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:52.969 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:52.257+0000 7f8a31600640 1 --2- 172.21.15.43:0/3127999279 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f8a20039200 0x7f8a2003b6c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8a240098c0 tx=0x7f8a24006db0 comp rx=0 tx=0).stop 2024-09-19T04:54:52.969 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:52.257+0000 7f8a31600640 1 -- 172.21.15.43:0/3127999279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a3c108220 msgr2=0x7f8a3c107aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:52.969 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:52.257+0000 7f8a31600640 1 --2- 172.21.15.43:0/3127999279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a3c108220 0x7f8a3c107aa0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c0097f0 tx=0x7f8a2c030ec0 comp rx=0 tx=0).stop 2024-09-19T04:54:52.969 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:52.257+0000 7f8a31600640 1 -- 172.21.15.43:0/3127999279 shutdown_connections 2024-09-19T04:54:52.969 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:52.257+0000 7f8a31600640 1 --2- 172.21.15.43:0/3127999279 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f8a20039200 0x7f8a2003b6c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:52.969 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:52.257+0000 7f8a31600640 1 --2- 172.21.15.43:0/3127999279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a3c108220 0x7f8a3c107aa0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:52.969 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:52.257+0000 7f8a31600640 1 -- 172.21.15.43:0/3127999279 >> 172.21.15.43:0/3127999279 conn(0x7f8a3c101e20 msgr2=0x7f8a3c102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:52.969 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:52.257+0000 7f8a31600640 1 -- 172.21.15.43:0/3127999279 shutdown_connections 2024-09-19T04:54:52.969 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:52.257+0000 7f8a31600640 1 -- 172.21.15.43:0/3127999279 wait complete. 2024-09-19T04:54:52.969 INFO:teuthology.orchestra.run.smithi043.stdout:Deploying unmanaged mon service... 2024-09-19T04:54:53.477 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:53 smithi043 ceph-mon[30800]: from='mgr.14118 172.21.15.43:0/568310870' entity='mgr.x' 2024-09-19T04:54:53.477 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:53 smithi043 ceph-mon[30800]: Added host smithi043 2024-09-19T04:54:53.477 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:53 smithi043 ceph-mon[30800]: from='mgr.14118 172.21.15.43:0/568310870' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:54:54.391 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-19T04:54:54.391 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.475+0000 7fae46400640 1 Processor -- start 2024-09-19T04:54:54.391 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.475+0000 7fae46400640 1 -- start start 2024-09-19T04:54:54.391 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.475+0000 7fae46400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae401099c0 0x7fae40109dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:54.392 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.475+0000 7fae46400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fae4010a390 con 0x7fae401099c0 2024-09-19T04:54:54.392 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.476+0000 7fae3f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae401099c0 0x7fae40109dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:54.392 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.476+0000 7fae3f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae401099c0 0x7fae40109dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38024/0 (socket says 172.21.15.43:38024) 2024-09-19T04:54:54.392 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.476+0000 7fae3f400640 1 -- 172.21.15.43:0/2523947238 learned_addr learned my addr 172.21.15.43:0/2523947238 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:54:54.392 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.477+0000 7fae3f400640 1 -- 172.21.15.43:0/2523947238 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae4010ab00 con 0x7fae401099c0 2024-09-19T04:54:54.392 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.478+0000 7fae3f400640 1 --2- 172.21.15.43:0/2523947238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae401099c0 0x7fae40109dc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fae30009820 tx=0x7fae3002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5359573fcf7a5498 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:54.393 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.478+0000 7fae3e000640 1 -- 172.21.15.43:0/2523947238 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae30037070 con 0x7fae401099c0 2024-09-19T04:54:54.393 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.478+0000 7fae3e000640 1 -- 172.21.15.43:0/2523947238 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fae30032900 con 0x7fae401099c0 2024-09-19T04:54:54.393 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.478+0000 7fae3e000640 1 -- 172.21.15.43:0/2523947238 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae30030850 con 0x7fae401099c0 2024-09-19T04:54:54.394 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.479+0000 7fae46400640 1 -- 172.21.15.43:0/2523947238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae401099c0 msgr2=0x7fae40109dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:54.394 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.479+0000 7fae46400640 1 --2- 172.21.15.43:0/2523947238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae401099c0 0x7fae40109dc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fae30009820 tx=0x7fae3002a270 comp rx=0 tx=0).stop 2024-09-19T04:54:54.394 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.479+0000 7fae46400640 1 -- 172.21.15.43:0/2523947238 shutdown_connections 2024-09-19T04:54:54.394 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.479+0000 7fae46400640 1 --2- 172.21.15.43:0/2523947238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae401099c0 0x7fae40109dc0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:54.395 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.479+0000 7fae46400640 1 -- 172.21.15.43:0/2523947238 >> 172.21.15.43:0/2523947238 conn(0x7fae4007ceb0 msgr2=0x7fae4007d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:54.395 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.479+0000 7fae46400640 1 -- 172.21.15.43:0/2523947238 shutdown_connections 2024-09-19T04:54:54.395 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.479+0000 7fae46400640 1 -- 172.21.15.43:0/2523947238 wait complete. 2024-09-19T04:54:54.395 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.480+0000 7fae46400640 1 Processor -- start 2024-09-19T04:54:54.396 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.480+0000 7fae46400640 1 -- start start 2024-09-19T04:54:54.396 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.481+0000 7fae46400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae401099c0 0x7fae4019d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:54.396 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.481+0000 7fae46400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fae4010deb0 con 0x7fae401099c0 2024-09-19T04:54:54.396 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.481+0000 7fae3f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae401099c0 0x7fae4019d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:54.396 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.481+0000 7fae3f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae401099c0 0x7fae4019d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38032/0 (socket says 172.21.15.43:38032) 2024-09-19T04:54:54.396 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.481+0000 7fae3f400640 1 -- 172.21.15.43:0/4139084972 learned_addr learned my addr 172.21.15.43:0/4139084972 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:54:54.397 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.481+0000 7fae3f400640 1 -- 172.21.15.43:0/4139084972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae4019de60 con 0x7fae401099c0 2024-09-19T04:54:54.397 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.482+0000 7fae3f400640 1 --2- 172.21.15.43:0/4139084972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae401099c0 0x7fae4019d920 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fae300097f0 tx=0x7fae30030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:54.397 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.482+0000 7fae37e00640 1 -- 172.21.15.43:0/4139084972 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae30042070 con 0x7fae401099c0 2024-09-19T04:54:54.397 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.482+0000 7fae46400640 1 -- 172.21.15.43:0/4139084972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae4019e0f0 con 0x7fae401099c0 2024-09-19T04:54:54.397 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.482+0000 7fae37e00640 1 -- 172.21.15.43:0/4139084972 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fae30032ca0 con 0x7fae401099c0 2024-09-19T04:54:54.397 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.482+0000 7fae37e00640 1 -- 172.21.15.43:0/4139084972 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae30037040 con 0x7fae401099c0 2024-09-19T04:54:54.398 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.482+0000 7fae46400640 1 -- 172.21.15.43:0/4139084972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae401a0de0 con 0x7fae401099c0 2024-09-19T04:54:54.398 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.483+0000 7fae37e00640 1 -- 172.21.15.43:0/4139084972 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fae30045070 con 0x7fae401099c0 2024-09-19T04:54:54.398 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.484+0000 7fae35600640 1 -- 172.21.15.43:0/4139084972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae18005290 con 0x7fae401099c0 2024-09-19T04:54:54.398 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.484+0000 7fae37e00640 1 --2- 172.21.15.43:0/4139084972 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7fae24039200 0x7fae2403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:54.398 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.484+0000 7fae37e00640 1 -- 172.21.15.43:0/4139084972 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fae3006c550 con 0x7fae401099c0 2024-09-19T04:54:54.398 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.484+0000 7fae3ea00640 1 --2- 172.21.15.43:0/4139084972 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7fae24039200 0x7fae2403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:54.398 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.485+0000 7fae3ea00640 1 --2- 172.21.15.43:0/4139084972 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7fae24039200 0x7fae2403b6c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0098c0 tx=0x7fae2c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:54.399 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.490+0000 7fae37e00640 1 -- 172.21.15.43:0/4139084972 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fae3004b170 con 0x7fae401099c0 2024-09-19T04:54:54.399 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.689+0000 7fae35600640 1 -- 172.21.15.43:0/4139084972 --> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fae18000bc0 con 0x7fae24039200 2024-09-19T04:54:54.399 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.694+0000 7fae37e00640 1 -- 172.21.15.43:0/4139084972 <== mgr.14118 v2:172.21.15.43:6800/3241799377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fae18000bc0 con 0x7fae24039200 2024-09-19T04:54:54.399 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.697+0000 7fae35600640 1 -- 172.21.15.43:0/4139084972 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7fae24039200 msgr2=0x7fae2403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:54.399 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.697+0000 7fae35600640 1 --2- 172.21.15.43:0/4139084972 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7fae24039200 0x7fae2403b6c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0098c0 tx=0x7fae2c006db0 comp rx=0 tx=0).stop 2024-09-19T04:54:54.399 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.697+0000 7fae35600640 1 -- 172.21.15.43:0/4139084972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae401099c0 msgr2=0x7fae4019d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:54.399 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.697+0000 7fae35600640 1 --2- 172.21.15.43:0/4139084972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae401099c0 0x7fae4019d920 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fae300097f0 tx=0x7fae30030ec0 comp rx=0 tx=0).stop 2024-09-19T04:54:54.399 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.697+0000 7fae35600640 1 -- 172.21.15.43:0/4139084972 shutdown_connections 2024-09-19T04:54:54.400 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.697+0000 7fae35600640 1 --2- 172.21.15.43:0/4139084972 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7fae24039200 0x7fae2403b6c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:54.400 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.697+0000 7fae35600640 1 --2- 172.21.15.43:0/4139084972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae401099c0 0x7fae4019d920 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:54.400 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.697+0000 7fae35600640 1 -- 172.21.15.43:0/4139084972 >> 172.21.15.43:0/4139084972 conn(0x7fae4007ceb0 msgr2=0x7fae40107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:54.400 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.697+0000 7fae35600640 1 -- 172.21.15.43:0/4139084972 shutdown_connections 2024-09-19T04:54:54.400 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:53.697+0000 7fae35600640 1 -- 172.21.15.43:0/4139084972 wait complete. 2024-09-19T04:54:54.400 INFO:teuthology.orchestra.run.smithi043.stdout:Deploying unmanaged mgr service... 2024-09-19T04:54:54.913 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:54 smithi043 ceph-mon[30800]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:54:54.913 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:54 smithi043 ceph-mon[30800]: Saving service mon spec with placement count:5 2024-09-19T04:54:54.913 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:54 smithi043 ceph-mon[30800]: from='mgr.14118 172.21.15.43:0/568310870' entity='mgr.x' 2024-09-19T04:54:56.078 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-19T04:54:56.079 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.090+0000 7f2e10a00640 1 Processor -- start 2024-09-19T04:54:56.079 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.090+0000 7f2e10a00640 1 -- start start 2024-09-19T04:54:56.079 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.090+0000 7f2e10a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e0c108020 0x7f2e0c10a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:56.079 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.090+0000 7f2e10a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0c07bdf0 con 0x7f2e0c108020 2024-09-19T04:54:56.079 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.090+0000 7f2e0b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e0c108020 0x7f2e0c10a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:56.079 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.090+0000 7f2e0b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e0c108020 0x7f2e0c10a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38048/0 (socket says 172.21.15.43:38048) 2024-09-19T04:54:56.079 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.090+0000 7f2e0b400640 1 -- 172.21.15.43:0/1674854066 learned_addr learned my addr 172.21.15.43:0/1674854066 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:54:56.079 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.091+0000 7f2e0b400640 1 -- 172.21.15.43:0/1674854066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e0c10a950 con 0x7f2e0c108020 2024-09-19T04:54:56.079 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.092+0000 7f2e0b400640 1 --2- 172.21.15.43:0/1674854066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e0c108020 0x7f2e0c10a410 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc009e80 tx=0x7f2dfc02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4345aa0f182a20f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:56.079 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.092+0000 7f2e0aa00640 1 -- 172.21.15.43:0/1674854066 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2dfc036070 con 0x7f2e0c108020 2024-09-19T04:54:56.079 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.092+0000 7f2e0aa00640 1 -- 172.21.15.43:0/1674854066 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2dfc0318c0 con 0x7f2e0c108020 2024-09-19T04:54:56.079 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.092+0000 7f2e0aa00640 1 -- 172.21.15.43:0/1674854066 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2dfc007ab0 con 0x7f2e0c108020 2024-09-19T04:54:56.080 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.093+0000 7f2e10a00640 1 -- 172.21.15.43:0/1674854066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e0c108020 msgr2=0x7f2e0c10a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:56.080 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.093+0000 7f2e10a00640 1 --2- 172.21.15.43:0/1674854066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e0c108020 0x7f2e0c10a410 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc009e80 tx=0x7f2dfc02a7a0 comp rx=0 tx=0).stop 2024-09-19T04:54:56.080 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.093+0000 7f2e10a00640 1 -- 172.21.15.43:0/1674854066 shutdown_connections 2024-09-19T04:54:56.080 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.093+0000 7f2e10a00640 1 --2- 172.21.15.43:0/1674854066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e0c108020 0x7f2e0c10a410 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:56.080 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.093+0000 7f2e10a00640 1 -- 172.21.15.43:0/1674854066 >> 172.21.15.43:0/1674854066 conn(0x7f2e0c101c40 msgr2=0x7f2e0c104060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:56.080 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.093+0000 7f2e10a00640 1 -- 172.21.15.43:0/1674854066 shutdown_connections 2024-09-19T04:54:56.080 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.093+0000 7f2e10a00640 1 -- 172.21.15.43:0/1674854066 wait complete. 2024-09-19T04:54:56.080 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.094+0000 7f2e10a00640 1 Processor -- start 2024-09-19T04:54:56.080 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.094+0000 7f2e10a00640 1 -- start start 2024-09-19T04:54:56.080 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.094+0000 7f2e10a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e0c108020 0x7f2e0c1078a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:56.080 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.094+0000 7f2e10a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0c10b930 con 0x7f2e0c108020 2024-09-19T04:54:56.080 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.094+0000 7f2e0b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e0c108020 0x7f2e0c1078a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:56.080 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.094+0000 7f2e0b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e0c108020 0x7f2e0c1078a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38052/0 (socket says 172.21.15.43:38052) 2024-09-19T04:54:56.081 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.094+0000 7f2e0b400640 1 -- 172.21.15.43:0/1794247527 learned_addr learned my addr 172.21.15.43:0/1794247527 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:54:56.081 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.095+0000 7f2e0b400640 1 -- 172.21.15.43:0/1794247527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e0c107de0 con 0x7f2e0c108020 2024-09-19T04:54:56.081 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.095+0000 7f2e0b400640 1 --2- 172.21.15.43:0/1794247527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e0c108020 0x7f2e0c1078a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc02acc0 tx=0x7f2dfc031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:56.081 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.095+0000 7f2e08c00640 1 -- 172.21.15.43:0/1794247527 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2dfc036050 con 0x7f2e0c108020 2024-09-19T04:54:56.081 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.095+0000 7f2e08c00640 1 -- 172.21.15.43:0/1794247527 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2dfc037040 con 0x7f2e0c108020 2024-09-19T04:54:56.081 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.095+0000 7f2e08c00640 1 -- 172.21.15.43:0/1794247527 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2dfc03d850 con 0x7f2e0c108020 2024-09-19T04:54:56.081 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.095+0000 7f2e10a00640 1 -- 172.21.15.43:0/1794247527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e0c105fd0 con 0x7f2e0c108020 2024-09-19T04:54:56.081 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.096+0000 7f2e10a00640 1 -- 172.21.15.43:0/1794247527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e0c106510 con 0x7f2e0c108020 2024-09-19T04:54:56.081 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.097+0000 7f2e08c00640 1 -- 172.21.15.43:0/1794247527 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f2dfc044020 con 0x7f2e0c108020 2024-09-19T04:54:56.081 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.097+0000 7f2e10a00640 1 -- 172.21.15.43:0/1794247527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e0c108480 con 0x7f2e0c108020 2024-09-19T04:54:56.081 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.097+0000 7f2e08c00640 1 --2- 172.21.15.43:0/1794247527 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f2df0039200 0x7f2df003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:56.081 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.097+0000 7f2e08c00640 1 -- 172.21.15.43:0/1794247527 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2dfc042020 con 0x7f2e0c108020 2024-09-19T04:54:56.081 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.097+0000 7f2e02a00640 1 --2- 172.21.15.43:0/1794247527 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f2df0039200 0x7f2df003b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:56.081 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.098+0000 7f2e02a00640 1 --2- 172.21.15.43:0/1794247527 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f2df0039200 0x7f2df003b6c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2df40098c0 tx=0x7f2df4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:56.082 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.102+0000 7f2e08c00640 1 -- 172.21.15.43:0/1794247527 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2dfc04b020 con 0x7f2e0c108020 2024-09-19T04:54:56.082 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.301+0000 7f2e10a00640 1 -- 172.21.15.43:0/1794247527 --> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f2e0c106e60 con 0x7f2df0039200 2024-09-19T04:54:56.082 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.306+0000 7f2e08c00640 1 -- 172.21.15.43:0/1794247527 <== mgr.14118 v2:172.21.15.43:6800/3241799377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2e0c106e60 con 0x7f2df0039200 2024-09-19T04:54:56.082 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.308+0000 7f2e10a00640 1 -- 172.21.15.43:0/1794247527 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f2df0039200 msgr2=0x7f2df003b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:56.082 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.308+0000 7f2e10a00640 1 --2- 172.21.15.43:0/1794247527 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f2df0039200 0x7f2df003b6c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2df40098c0 tx=0x7f2df4006db0 comp rx=0 tx=0).stop 2024-09-19T04:54:56.082 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.308+0000 7f2e10a00640 1 -- 172.21.15.43:0/1794247527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e0c108020 msgr2=0x7f2e0c1078a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:56.082 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.308+0000 7f2e10a00640 1 --2- 172.21.15.43:0/1794247527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e0c108020 0x7f2e0c1078a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc02acc0 tx=0x7f2dfc031ca0 comp rx=0 tx=0).stop 2024-09-19T04:54:56.082 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.308+0000 7f2e10a00640 1 -- 172.21.15.43:0/1794247527 shutdown_connections 2024-09-19T04:54:56.082 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.308+0000 7f2e10a00640 1 --2- 172.21.15.43:0/1794247527 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f2df0039200 0x7f2df003b6c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:56.082 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.308+0000 7f2e10a00640 1 --2- 172.21.15.43:0/1794247527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e0c108020 0x7f2e0c1078a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:56.082 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.308+0000 7f2e10a00640 1 -- 172.21.15.43:0/1794247527 >> 172.21.15.43:0/1794247527 conn(0x7f2e0c101c40 msgr2=0x7f2e0c102800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:56.082 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.308+0000 7f2e10a00640 1 -- 172.21.15.43:0/1794247527 shutdown_connections 2024-09-19T04:54:56.082 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:55.309+0000 7f2e10a00640 1 -- 172.21.15.43:0/1794247527 wait complete. 2024-09-19T04:54:56.346 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:56 smithi043 ceph-mon[30800]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:54:56.346 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:56 smithi043 ceph-mon[30800]: Saving service mgr spec with placement count:2 2024-09-19T04:54:56.346 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:56 smithi043 ceph-mon[30800]: from='mgr.14118 172.21.15.43:0/568310870' entity='mgr.x' 2024-09-19T04:54:57.688 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.552+0000 7fc274800640 1 Processor -- start 2024-09-19T04:54:57.688 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.553+0000 7fc274800640 1 -- start start 2024-09-19T04:54:57.688 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.553+0000 7fc274800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc26c107c50 0x7fc26c108050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:57.688 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.553+0000 7fc274800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c108620 con 0x7fc26c107c50 2024-09-19T04:54:57.688 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.553+0000 7fc271e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc26c107c50 0x7fc26c108050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:57.688 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.553+0000 7fc271e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc26c107c50 0x7fc26c108050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53508/0 (socket says 172.21.15.43:53508) 2024-09-19T04:54:57.688 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.553+0000 7fc271e00640 1 -- 172.21.15.43:0/2537158868 learned_addr learned my addr 172.21.15.43:0/2537158868 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:54:57.688 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.554+0000 7fc271e00640 1 -- 172.21.15.43:0/2537158868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc26c106820 con 0x7fc26c107c50 2024-09-19T04:54:57.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.555+0000 7fc271e00640 1 --2- 172.21.15.43:0/2537158868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc26c107c50 0x7fc26c108050 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc260009820 tx=0x7fc26002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=702cf45fee0d7f0f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:57.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.555+0000 7fc270a00640 1 -- 172.21.15.43:0/2537158868 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc260037070 con 0x7fc26c107c50 2024-09-19T04:54:57.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.555+0000 7fc270a00640 1 -- 172.21.15.43:0/2537158868 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc260032900 con 0x7fc26c107c50 2024-09-19T04:54:57.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.555+0000 7fc270a00640 1 -- 172.21.15.43:0/2537158868 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc260030850 con 0x7fc26c107c50 2024-09-19T04:54:57.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.555+0000 7fc274800640 1 -- 172.21.15.43:0/2537158868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc26c107c50 msgr2=0x7fc26c108050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:57.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.555+0000 7fc274800640 1 --2- 172.21.15.43:0/2537158868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc26c107c50 0x7fc26c108050 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc260009820 tx=0x7fc26002a270 comp rx=0 tx=0).stop 2024-09-19T04:54:57.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.556+0000 7fc274800640 1 -- 172.21.15.43:0/2537158868 shutdown_connections 2024-09-19T04:54:57.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.556+0000 7fc274800640 1 --2- 172.21.15.43:0/2537158868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc26c107c50 0x7fc26c108050 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:57.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.556+0000 7fc274800640 1 -- 172.21.15.43:0/2537158868 >> 172.21.15.43:0/2537158868 conn(0x7fc26c101580 msgr2=0x7fc26c1039a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:57.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.556+0000 7fc274800640 1 -- 172.21.15.43:0/2537158868 shutdown_connections 2024-09-19T04:54:57.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.556+0000 7fc274800640 1 -- 172.21.15.43:0/2537158868 wait complete. 2024-09-19T04:54:57.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.557+0000 7fc274800640 1 Processor -- start 2024-09-19T04:54:57.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.557+0000 7fc274800640 1 -- start start 2024-09-19T04:54:57.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.557+0000 7fc274800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc26c107c50 0x7fc26c1994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:57.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.557+0000 7fc274800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c105890 con 0x7fc26c107c50 2024-09-19T04:54:57.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.557+0000 7fc271e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc26c107c50 0x7fc26c1994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:57.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.557+0000 7fc271e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc26c107c50 0x7fc26c1994a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53520/0 (socket says 172.21.15.43:53520) 2024-09-19T04:54:57.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.557+0000 7fc271e00640 1 -- 172.21.15.43:0/2870436750 learned_addr learned my addr 172.21.15.43:0/2870436750 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:54:57.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.558+0000 7fc271e00640 1 -- 172.21.15.43:0/2870436750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc26c1999e0 con 0x7fc26c107c50 2024-09-19T04:54:57.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.558+0000 7fc271e00640 1 --2- 172.21.15.43:0/2870436750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc26c107c50 0x7fc26c1994a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc2600097f0 tx=0x7fc260030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:57.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.558+0000 7fc266a00640 1 -- 172.21.15.43:0/2870436750 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc260042070 con 0x7fc26c107c50 2024-09-19T04:54:57.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.558+0000 7fc266a00640 1 -- 172.21.15.43:0/2870436750 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc260032ca0 con 0x7fc26c107c50 2024-09-19T04:54:57.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.558+0000 7fc266a00640 1 -- 172.21.15.43:0/2870436750 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc260037040 con 0x7fc26c107c50 2024-09-19T04:54:57.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.558+0000 7fc274800640 1 -- 172.21.15.43:0/2870436750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc26c199c70 con 0x7fc26c107c50 2024-09-19T04:54:57.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.558+0000 7fc274800640 1 -- 172.21.15.43:0/2870436750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc26c19c960 con 0x7fc26c107c50 2024-09-19T04:54:57.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.559+0000 7fc274800640 1 -- 172.21.15.43:0/2870436750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc26c108050 con 0x7fc26c107c50 2024-09-19T04:54:57.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.559+0000 7fc266a00640 1 -- 172.21.15.43:0/2870436750 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fc260045070 con 0x7fc26c107c50 2024-09-19T04:54:57.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.560+0000 7fc266a00640 1 --2- 172.21.15.43:0/2870436750 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7fc25403d620 0x7fc25403fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:57.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.560+0000 7fc266a00640 1 -- 172.21.15.43:0/2870436750 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc26006c1f0 con 0x7fc26c107c50 2024-09-19T04:54:57.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.560+0000 7fc271400640 1 --2- 172.21.15.43:0/2870436750 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7fc25403d620 0x7fc25403fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:57.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.561+0000 7fc271400640 1 --2- 172.21.15.43:0/2870436750 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7fc25403d620 0x7fc25403fae0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc25c0098c0 tx=0x7fc25c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:57.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.565+0000 7fc266a00640 1 -- 172.21.15.43:0/2870436750 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc260045360 con 0x7fc26c107c50 2024-09-19T04:54:57.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.755+0000 7fc274800640 1 -- 172.21.15.43:0/2870436750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fc26c192a00 con 0x7fc26c107c50 2024-09-19T04:54:57.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.758+0000 7fc266a00640 1 -- 172.21.15.43:0/2870436750 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v6)=0 v6) ==== 142+0+0 (secure 0 0 0) 0x7fc260043260 con 0x7fc26c107c50 2024-09-19T04:54:57.693 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.760+0000 7fc274800640 1 -- 172.21.15.43:0/2870436750 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7fc25403d620 msgr2=0x7fc25403fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:57.693 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.760+0000 7fc274800640 1 --2- 172.21.15.43:0/2870436750 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7fc25403d620 0x7fc25403fae0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc25c0098c0 tx=0x7fc25c006db0 comp rx=0 tx=0).stop 2024-09-19T04:54:57.693 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.760+0000 7fc274800640 1 -- 172.21.15.43:0/2870436750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc26c107c50 msgr2=0x7fc26c1994a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:57.693 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.760+0000 7fc274800640 1 --2- 172.21.15.43:0/2870436750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc26c107c50 0x7fc26c1994a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc2600097f0 tx=0x7fc260030ec0 comp rx=0 tx=0).stop 2024-09-19T04:54:57.693 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.760+0000 7fc274800640 1 -- 172.21.15.43:0/2870436750 shutdown_connections 2024-09-19T04:54:57.693 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.760+0000 7fc274800640 1 --2- 172.21.15.43:0/2870436750 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7fc25403d620 0x7fc25403fae0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:57.693 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.760+0000 7fc274800640 1 --2- 172.21.15.43:0/2870436750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc26c107c50 0x7fc26c1994a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:57.693 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.760+0000 7fc274800640 1 -- 172.21.15.43:0/2870436750 >> 172.21.15.43:0/2870436750 conn(0x7fc26c101580 msgr2=0x7fc26c18fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:57.693 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.761+0000 7fc274800640 1 -- 172.21.15.43:0/2870436750 shutdown_connections 2024-09-19T04:54:57.693 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:56.761+0000 7fc274800640 1 -- 172.21.15.43:0/2870436750 wait complete. 2024-09-19T04:54:57.990 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:57 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/2870436750' entity='client.admin' 2024-09-19T04:54:57.990 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:57 smithi043 ceph-mon[30800]: from='mgr.14118 172.21.15.43:0/568310870' entity='mgr.x' 2024-09-19T04:54:59.079 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:58 smithi043 ceph-mon[30800]: from='mgr.14118 172.21.15.43:0/568310870' entity='mgr.x' 2024-09-19T04:54:59.080 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:54:58 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/1872235433' entity='client.admin' 2024-09-19T04:54:59.844 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.227+0000 7fbeef400640 1 Processor -- start 2024-09-19T04:54:59.845 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.228+0000 7fbeef400640 1 -- start start 2024-09-19T04:54:59.845 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.228+0000 7fbeef400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbee8109960 0x7fbee8109d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:59.845 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.228+0000 7fbeef400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbee810a330 con 0x7fbee8109960 2024-09-19T04:54:59.845 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.229+0000 7fbeeca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbee8109960 0x7fbee8109d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:59.845 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.229+0000 7fbeeca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbee8109960 0x7fbee8109d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53528/0 (socket says 172.21.15.43:53528) 2024-09-19T04:54:59.845 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.229+0000 7fbeeca00640 1 -- 172.21.15.43:0/1602872170 learned_addr learned my addr 172.21.15.43:0/1602872170 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:54:59.845 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.229+0000 7fbeeca00640 1 -- 172.21.15.43:0/1602872170 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee810aad0 con 0x7fbee8109960 2024-09-19T04:54:59.854 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.230+0000 7fbeeca00640 1 --2- 172.21.15.43:0/1602872170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbee8109960 0x7fbee8109d60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbedc009820 tx=0x7fbedc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffd028c6f70d770f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:59.854 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.230+0000 7fbee7400640 1 -- 172.21.15.43:0/1602872170 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbedc037070 con 0x7fbee8109960 2024-09-19T04:54:59.854 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.230+0000 7fbee7400640 1 -- 172.21.15.43:0/1602872170 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbedc032920 con 0x7fbee8109960 2024-09-19T04:54:59.854 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.230+0000 7fbee7400640 1 -- 172.21.15.43:0/1602872170 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbedc030810 con 0x7fbee8109960 2024-09-19T04:54:59.854 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.231+0000 7fbeef400640 1 -- 172.21.15.43:0/1602872170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbee8109960 msgr2=0x7fbee8109d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:59.854 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.231+0000 7fbeef400640 1 --2- 172.21.15.43:0/1602872170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbee8109960 0x7fbee8109d60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbedc009820 tx=0x7fbedc02a270 comp rx=0 tx=0).stop 2024-09-19T04:54:59.855 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.231+0000 7fbeef400640 1 -- 172.21.15.43:0/1602872170 shutdown_connections 2024-09-19T04:54:59.855 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.231+0000 7fbeef400640 1 --2- 172.21.15.43:0/1602872170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbee8109960 0x7fbee8109d60 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:59.855 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.231+0000 7fbeef400640 1 -- 172.21.15.43:0/1602872170 >> 172.21.15.43:0/1602872170 conn(0x7fbee807cf30 msgr2=0x7fbee807d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:59.855 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.231+0000 7fbeef400640 1 -- 172.21.15.43:0/1602872170 shutdown_connections 2024-09-19T04:54:59.855 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.231+0000 7fbeef400640 1 -- 172.21.15.43:0/1602872170 wait complete. 2024-09-19T04:54:59.855 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.232+0000 7fbeef400640 1 Processor -- start 2024-09-19T04:54:59.855 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.233+0000 7fbeef400640 1 -- start start 2024-09-19T04:54:59.855 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.233+0000 7fbeef400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbee8109960 0x7fbee819d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:59.855 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.233+0000 7fbeef400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbee810de80 con 0x7fbee8109960 2024-09-19T04:54:59.856 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.233+0000 7fbeeca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbee8109960 0x7fbee819d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:59.856 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.233+0000 7fbeeca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbee8109960 0x7fbee819d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53532/0 (socket says 172.21.15.43:53532) 2024-09-19T04:54:59.856 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.233+0000 7fbeeca00640 1 -- 172.21.15.43:0/1872235433 learned_addr learned my addr 172.21.15.43:0/1872235433 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:54:59.856 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.233+0000 7fbeeca00640 1 -- 172.21.15.43:0/1872235433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee819de80 con 0x7fbee8109960 2024-09-19T04:54:59.856 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.234+0000 7fbeeca00640 1 --2- 172.21.15.43:0/1872235433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbee8109960 0x7fbee819d940 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0097f0 tx=0x7fbedc0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:59.856 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.234+0000 7fbee5600640 1 -- 172.21.15.43:0/1872235433 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbedc037040 con 0x7fbee8109960 2024-09-19T04:54:59.856 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.234+0000 7fbee5600640 1 -- 172.21.15.43:0/1872235433 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbedc030c40 con 0x7fbee8109960 2024-09-19T04:54:59.856 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.234+0000 7fbeef400640 1 -- 172.21.15.43:0/1872235433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbee819e110 con 0x7fbee8109960 2024-09-19T04:54:59.857 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.234+0000 7fbee5600640 1 -- 172.21.15.43:0/1872235433 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbedc03a5b0 con 0x7fbee8109960 2024-09-19T04:54:59.857 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.234+0000 7fbeef400640 1 -- 172.21.15.43:0/1872235433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbee81a0e00 con 0x7fbee8109960 2024-09-19T04:54:59.857 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.235+0000 7fbee5600640 1 -- 172.21.15.43:0/1872235433 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fbedc03b3f0 con 0x7fbee8109960 2024-09-19T04:54:59.857 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.236+0000 7fbee5600640 1 --2- 172.21.15.43:0/1872235433 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7fbed00391b0 0x7fbed003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:54:59.857 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.236+0000 7fbee5600640 1 -- 172.21.15.43:0/1872235433 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbedc06c6b0 con 0x7fbee8109960 2024-09-19T04:54:59.857 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.236+0000 7fbeef400640 1 -- 172.21.15.43:0/1872235433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbee810e000 con 0x7fbee8109960 2024-09-19T04:54:59.857 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.236+0000 7fbee7e00640 1 --2- 172.21.15.43:0/1872235433 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7fbed00391b0 0x7fbed003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:54:59.857 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.237+0000 7fbee7e00640 1 --2- 172.21.15.43:0/1872235433 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7fbed00391b0 0x7fbed003b670 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbed80098c0 tx=0x7fbed8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:54:59.858 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.241+0000 7fbee5600640 1 -- 172.21.15.43:0/1872235433 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbedc0313a0 con 0x7fbee8109960 2024-09-19T04:54:59.858 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.436+0000 7fbeef400640 1 -- 172.21.15.43:0/1872235433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fbee8109de0 con 0x7fbee8109960 2024-09-19T04:54:59.858 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.438+0000 7fbee5600640 1 -- 172.21.15.43:0/1872235433 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fbedc041d10 con 0x7fbee8109960 2024-09-19T04:54:59.858 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.441+0000 7fbeef400640 1 -- 172.21.15.43:0/1872235433 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7fbed00391b0 msgr2=0x7fbed003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:59.858 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.441+0000 7fbeef400640 1 --2- 172.21.15.43:0/1872235433 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7fbed00391b0 0x7fbed003b670 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbed80098c0 tx=0x7fbed8006db0 comp rx=0 tx=0).stop 2024-09-19T04:54:59.858 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.441+0000 7fbeef400640 1 -- 172.21.15.43:0/1872235433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbee8109960 msgr2=0x7fbee819d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:54:59.858 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.441+0000 7fbeef400640 1 --2- 172.21.15.43:0/1872235433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbee8109960 0x7fbee819d940 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0097f0 tx=0x7fbedc0327b0 comp rx=0 tx=0).stop 2024-09-19T04:54:59.858 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.441+0000 7fbeef400640 1 -- 172.21.15.43:0/1872235433 shutdown_connections 2024-09-19T04:54:59.859 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.441+0000 7fbeef400640 1 --2- 172.21.15.43:0/1872235433 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7fbed00391b0 0x7fbed003b670 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:59.859 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.441+0000 7fbeef400640 1 --2- 172.21.15.43:0/1872235433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbee8109960 0x7fbee819d940 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:54:59.859 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.441+0000 7fbeef400640 1 -- 172.21.15.43:0/1872235433 >> 172.21.15.43:0/1872235433 conn(0x7fbee807cf30 msgr2=0x7fbee8107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:54:59.859 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.441+0000 7fbeef400640 1 -- 172.21.15.43:0/1872235433 shutdown_connections 2024-09-19T04:54:59.859 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:54:58.441+0000 7fbeef400640 1 -- 172.21.15.43:0/1872235433 wait complete. 2024-09-19T04:54:59.859 INFO:teuthology.orchestra.run.smithi043.stdout:Enabling the dashboard module... 2024-09-19T04:55:01.184 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:00 smithi043 ceph-mon[30800]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:55:01.185 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:00 smithi043 ceph-mon[30800]: from='mgr.14118 172.21.15.43:0/568310870' entity='mgr.x' 2024-09-19T04:55:01.185 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:00 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/564159042' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-19T04:55:01.185 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:01 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: ignoring --setuser ceph since I am not root 2024-09-19T04:55:01.185 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:01 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: ignoring --setgroup ceph since I am not root 2024-09-19T04:55:01.185 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:01 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:55:01.183+0000 7fafba62d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T04:55:01.450 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:01 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:55:01.257+0000 7fafba62d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T04:55:01.450 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:01 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:55:01.449+0000 7fafba62d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T04:55:01.648 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.387+0000 7f65fd000640 1 Processor -- start 2024-09-19T04:55:01.648 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.387+0000 7f65fd000640 1 -- start start 2024-09-19T04:55:01.648 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.387+0000 7f65fd000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65f81099c0 0x7f65f8109dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:01.648 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.387+0000 7f65fd000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f65f810a390 con 0x7f65f81099c0 2024-09-19T04:55:01.648 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.387+0000 7f65f5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65f81099c0 0x7f65f8109dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:01.648 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.388+0000 7f65f5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65f81099c0 0x7f65f8109dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53542/0 (socket says 172.21.15.43:53542) 2024-09-19T04:55:01.648 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.388+0000 7f65f5e00640 1 -- 172.21.15.43:0/1415224188 learned_addr learned my addr 172.21.15.43:0/1415224188 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:55:01.648 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.388+0000 7f65f5e00640 1 -- 172.21.15.43:0/1415224188 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f810ab00 con 0x7f65f81099c0 2024-09-19T04:55:01.649 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.389+0000 7f65f5e00640 1 --2- 172.21.15.43:0/1415224188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65f81099c0 0x7f65f8109dc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f65e8009820 tx=0x7f65e802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b86fbc67fdb87870 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:01.649 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.389+0000 7f65f4a00640 1 -- 172.21.15.43:0/1415224188 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65e8037070 con 0x7f65f81099c0 2024-09-19T04:55:01.649 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.389+0000 7f65f4a00640 1 -- 172.21.15.43:0/1415224188 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65e8032920 con 0x7f65f81099c0 2024-09-19T04:55:01.649 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.389+0000 7f65f4a00640 1 -- 172.21.15.43:0/1415224188 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65e8030810 con 0x7f65f81099c0 2024-09-19T04:55:01.649 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.390+0000 7f65fd000640 1 -- 172.21.15.43:0/1415224188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65f81099c0 msgr2=0x7f65f8109dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:01.649 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.390+0000 7f65fd000640 1 --2- 172.21.15.43:0/1415224188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65f81099c0 0x7f65f8109dc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f65e8009820 tx=0x7f65e802a270 comp rx=0 tx=0).stop 2024-09-19T04:55:01.649 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.390+0000 7f65fd000640 1 -- 172.21.15.43:0/1415224188 shutdown_connections 2024-09-19T04:55:01.649 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.390+0000 7f65fd000640 1 --2- 172.21.15.43:0/1415224188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65f81099c0 0x7f65f8109dc0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:01.649 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.390+0000 7f65fd000640 1 -- 172.21.15.43:0/1415224188 >> 172.21.15.43:0/1415224188 conn(0x7f65f807ceb0 msgr2=0x7f65f807d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:01.649 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.390+0000 7f65fd000640 1 -- 172.21.15.43:0/1415224188 shutdown_connections 2024-09-19T04:55:01.649 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.390+0000 7f65fd000640 1 -- 172.21.15.43:0/1415224188 wait complete. 2024-09-19T04:55:01.649 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.391+0000 7f65fd000640 1 Processor -- start 2024-09-19T04:55:01.649 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.391+0000 7f65fd000640 1 -- start start 2024-09-19T04:55:01.650 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.391+0000 7f65fd000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65f81099c0 0x7f65f819d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:01.650 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.391+0000 7f65fd000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f65f810deb0 con 0x7f65f81099c0 2024-09-19T04:55:01.650 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.392+0000 7f65f5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65f81099c0 0x7f65f819d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:01.650 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.392+0000 7f65f5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65f81099c0 0x7f65f819d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53558/0 (socket says 172.21.15.43:53558) 2024-09-19T04:55:01.650 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.392+0000 7f65f5e00640 1 -- 172.21.15.43:0/564159042 learned_addr learned my addr 172.21.15.43:0/564159042 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:55:01.650 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.392+0000 7f65f5e00640 1 -- 172.21.15.43:0/564159042 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f819deb0 con 0x7f65f81099c0 2024-09-19T04:55:01.650 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.392+0000 7f65f5e00640 1 --2- 172.21.15.43:0/564159042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65f81099c0 0x7f65f819d970 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f65e80097f0 tx=0x7f65e8030d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:01.650 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.392+0000 7f65eea00640 1 -- 172.21.15.43:0/564159042 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65e8037040 con 0x7f65f81099c0 2024-09-19T04:55:01.650 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.393+0000 7f65fd000640 1 -- 172.21.15.43:0/564159042 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65f819e140 con 0x7f65f81099c0 2024-09-19T04:55:01.650 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.393+0000 7f65eea00640 1 -- 172.21.15.43:0/564159042 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65e8040070 con 0x7f65f81099c0 2024-09-19T04:55:01.650 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.393+0000 7f65eea00640 1 -- 172.21.15.43:0/564159042 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65e803a610 con 0x7f65f81099c0 2024-09-19T04:55:01.650 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.393+0000 7f65fd000640 1 -- 172.21.15.43:0/564159042 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65f81a0e30 con 0x7f65f81099c0 2024-09-19T04:55:01.650 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.394+0000 7f65eea00640 1 -- 172.21.15.43:0/564159042 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f65e80327b0 con 0x7f65f81099c0 2024-09-19T04:55:01.650 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.394+0000 7f65fd000640 1 -- 172.21.15.43:0/564159042 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65f810e030 con 0x7f65f81099c0 2024-09-19T04:55:01.650 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.394+0000 7f65eea00640 1 --2- 172.21.15.43:0/564159042 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f65dc0391b0 0x7f65dc03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:01.651 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.394+0000 7f65eea00640 1 -- 172.21.15.43:0/564159042 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f65e806c8f0 con 0x7f65f81099c0 2024-09-19T04:55:01.651 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.395+0000 7f65f5400640 1 --2- 172.21.15.43:0/564159042 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f65dc0391b0 0x7f65dc03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:01.651 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.395+0000 7f65f5400640 1 --2- 172.21.15.43:0/564159042 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f65dc0391b0 0x7f65dc03b670 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f65e40098c0 tx=0x7f65e4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:01.651 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.400+0000 7f65eea00640 1 -- 172.21.15.43:0/564159042 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f65e804a200 con 0x7f65f81099c0 2024-09-19T04:55:01.651 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.634+0000 7f65fd000640 1 -- 172.21.15.43:0/564159042 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f65f81a13d0 con 0x7f65f81099c0 2024-09-19T04:55:01.651 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.927+0000 7f65eea00640 1 -- 172.21.15.43:0/564159042 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f65e8031e80 con 0x7f65f81099c0 2024-09-19T04:55:01.651 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.930+0000 7f65eea00640 1 -- 172.21.15.43:0/564159042 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f65e803b420 con 0x7f65f81099c0 2024-09-19T04:55:01.651 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.931+0000 7f65fd000640 1 -- 172.21.15.43:0/564159042 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f65dc0391b0 msgr2=0x7f65dc03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:01.651 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.931+0000 7f65fd000640 1 --2- 172.21.15.43:0/564159042 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f65dc0391b0 0x7f65dc03b670 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f65e40098c0 tx=0x7f65e4006db0 comp rx=0 tx=0).stop 2024-09-19T04:55:01.651 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.931+0000 7f65fd000640 1 -- 172.21.15.43:0/564159042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65f81099c0 msgr2=0x7f65f819d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:01.651 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.931+0000 7f65fd000640 1 --2- 172.21.15.43:0/564159042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65f81099c0 0x7f65f819d970 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f65e80097f0 tx=0x7f65e8030d40 comp rx=0 tx=0).stop 2024-09-19T04:55:01.651 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.931+0000 7f65fd000640 1 -- 172.21.15.43:0/564159042 shutdown_connections 2024-09-19T04:55:01.651 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.931+0000 7f65fd000640 1 --2- 172.21.15.43:0/564159042 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f65dc0391b0 0x7f65dc03b670 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:01.651 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.931+0000 7f65fd000640 1 --2- 172.21.15.43:0/564159042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65f81099c0 0x7f65f819d970 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:01.652 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.931+0000 7f65fd000640 1 -- 172.21.15.43:0/564159042 >> 172.21.15.43:0/564159042 conn(0x7f65f807ceb0 msgr2=0x7f65f8107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:01.652 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.932+0000 7f65fd000640 1 -- 172.21.15.43:0/564159042 shutdown_connections 2024-09-19T04:55:01.652 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:00.932+0000 7f65fd000640 1 -- 172.21.15.43:0/564159042 wait complete. 2024-09-19T04:55:01.714 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:01 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:55:01.531+0000 7fafba62d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T04:55:01.714 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:01 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:55:01.713+0000 7fafba62d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:55:01.965 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:01 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:55:01.884+0000 7fafba62d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T04:55:01.965 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:01 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/564159042' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-19T04:55:01.965 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:01 smithi043 ceph-mon[30800]: mgrmap e9: x(active, since 21s) 2024-09-19T04:55:02.350 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:02 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:55:02.274+0000 7fafba62d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T04:55:02.350 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:02 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:55:02.348+0000 7fafba62d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T04:55:02.780 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:02 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:55:02.570+0000 7fafba62d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T04:55:03.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:02 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/3464799473' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-19T04:55:03.122 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-09-19T04:55:03.122 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-19T04:55:03.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T04:55:03.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-19T04:55:03.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-19T04:55:03.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-09-19T04:55:03.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.207+0000 7f3535800640 1 Processor -- start 2024-09-19T04:55:03.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.207+0000 7f3535800640 1 -- start start 2024-09-19T04:55:03.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.208+0000 7f3535800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35301076f0 0x7f3530107af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:03.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.208+0000 7f3535800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f35301080c0 con 0x7f35301076f0 2024-09-19T04:55:03.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.208+0000 7f352ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35301076f0 0x7f3530107af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:03.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.208+0000 7f352ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35301076f0 0x7f3530107af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53574/0 (socket says 172.21.15.43:53574) 2024-09-19T04:55:03.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.208+0000 7f352ea00640 1 -- 172.21.15.43:0/2414311197 learned_addr learned my addr 172.21.15.43:0/2414311197 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:55:03.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.209+0000 7f352ea00640 1 -- 172.21.15.43:0/2414311197 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35301088f0 con 0x7f35301076f0 2024-09-19T04:55:03.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.210+0000 7f352ea00640 1 --2- 172.21.15.43:0/2414311197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35301076f0 0x7f3530107af0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3520009820 tx=0x7f352002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e502dc26aac338 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:03.123 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.210+0000 7f352e000640 1 -- 172.21.15.43:0/2414311197 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3520037070 con 0x7f35301076f0 2024-09-19T04:55:03.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.210+0000 7f352e000640 1 -- 172.21.15.43:0/2414311197 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3520032920 con 0x7f35301076f0 2024-09-19T04:55:03.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.210+0000 7f352e000640 1 -- 172.21.15.43:0/2414311197 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3520030810 con 0x7f35301076f0 2024-09-19T04:55:03.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.211+0000 7f3535800640 1 -- 172.21.15.43:0/2414311197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35301076f0 msgr2=0x7f3530107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:03.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.211+0000 7f3535800640 1 --2- 172.21.15.43:0/2414311197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35301076f0 0x7f3530107af0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3520009820 tx=0x7f352002a270 comp rx=0 tx=0).stop 2024-09-19T04:55:03.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.211+0000 7f3535800640 1 -- 172.21.15.43:0/2414311197 shutdown_connections 2024-09-19T04:55:03.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.211+0000 7f3535800640 1 --2- 172.21.15.43:0/2414311197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35301076f0 0x7f3530107af0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:03.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.211+0000 7f3535800640 1 -- 172.21.15.43:0/2414311197 >> 172.21.15.43:0/2414311197 conn(0x7f3530102ec0 msgr2=0x7f35301052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:03.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.211+0000 7f3535800640 1 -- 172.21.15.43:0/2414311197 shutdown_connections 2024-09-19T04:55:03.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.211+0000 7f3535800640 1 -- 172.21.15.43:0/2414311197 wait complete. 2024-09-19T04:55:03.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.212+0000 7f3535800640 1 Processor -- start 2024-09-19T04:55:03.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.212+0000 7f3535800640 1 -- start start 2024-09-19T04:55:03.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.213+0000 7f3535800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35301076f0 0x7f353019b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:03.124 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.213+0000 7f3535800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f353010bca0 con 0x7f35301076f0 2024-09-19T04:55:03.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.213+0000 7f352ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35301076f0 0x7f353019b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:03.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.213+0000 7f352ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35301076f0 0x7f353019b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53590/0 (socket says 172.21.15.43:53590) 2024-09-19T04:55:03.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.213+0000 7f352ea00640 1 -- 172.21.15.43:0/3464799473 learned_addr learned my addr 172.21.15.43:0/3464799473 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:55:03.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.213+0000 7f352ea00640 1 -- 172.21.15.43:0/3464799473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f353019bc30 con 0x7f35301076f0 2024-09-19T04:55:03.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.213+0000 7f352ea00640 1 --2- 172.21.15.43:0/3464799473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35301076f0 0x7f353019b6f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f35200097f0 tx=0x7f35200327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:03.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.214+0000 7f3527400640 1 -- 172.21.15.43:0/3464799473 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3520037040 con 0x7f35301076f0 2024-09-19T04:55:03.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.214+0000 7f3527400640 1 -- 172.21.15.43:0/3464799473 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3520030cc0 con 0x7f35301076f0 2024-09-19T04:55:03.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.214+0000 7f3527400640 1 -- 172.21.15.43:0/3464799473 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f352003a640 con 0x7f35301076f0 2024-09-19T04:55:03.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.214+0000 7f3535800640 1 -- 172.21.15.43:0/3464799473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f353019bec0 con 0x7f35301076f0 2024-09-19T04:55:03.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.214+0000 7f3535800640 1 -- 172.21.15.43:0/3464799473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f353019ebb0 con 0x7f35301076f0 2024-09-19T04:55:03.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.215+0000 7f3535800640 1 -- 172.21.15.43:0/3464799473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f353010be20 con 0x7f35301076f0 2024-09-19T04:55:03.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.215+0000 7f3527400640 1 -- 172.21.15.43:0/3464799473 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f352003b3f0 con 0x7f35301076f0 2024-09-19T04:55:03.125 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.216+0000 7f3527400640 1 --2- 172.21.15.43:0/3464799473 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f3514039260 0x7f351403b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:03.126 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.216+0000 7f3527400640 1 -- 172.21.15.43:0/3464799473 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f352006c8b0 con 0x7f35301076f0 2024-09-19T04:55:03.126 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.216+0000 7f3527e00640 1 -- 172.21.15.43:0/3464799473 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f3514039260 msgr2=0x7f351403b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/3241799377 2024-09-19T04:55:03.126 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.216+0000 7f3527e00640 1 --2- 172.21.15.43:0/3464799473 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f3514039260 0x7f351403b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:55:03.126 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.221+0000 7f3527400640 1 -- 172.21.15.43:0/3464799473 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3520042dc0 con 0x7f35301076f0 2024-09-19T04:55:03.126 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.417+0000 7f3527e00640 1 -- 172.21.15.43:0/3464799473 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f3514039260 msgr2=0x7f351403b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/3241799377 2024-09-19T04:55:03.126 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.417+0000 7f3527e00640 1 --2- 172.21.15.43:0/3464799473 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f3514039260 0x7f351403b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T04:55:03.126 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.453+0000 7f3535800640 1 -- 172.21.15.43:0/3464799473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f353019ed80 con 0x7f35301076f0 2024-09-19T04:55:03.126 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.454+0000 7f3527400640 1 -- 172.21.15.43:0/3464799473 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f352006c200 con 0x7f35301076f0 2024-09-19T04:55:03.126 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.456+0000 7f3535800640 1 -- 172.21.15.43:0/3464799473 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f3514039260 msgr2=0x7f351403b720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:55:03.126 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.456+0000 7f3535800640 1 --2- 172.21.15.43:0/3464799473 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f3514039260 0x7f351403b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:03.126 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.456+0000 7f3535800640 1 -- 172.21.15.43:0/3464799473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35301076f0 msgr2=0x7f353019b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:03.126 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.456+0000 7f3535800640 1 --2- 172.21.15.43:0/3464799473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35301076f0 0x7f353019b6f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f35200097f0 tx=0x7f35200327b0 comp rx=0 tx=0).stop 2024-09-19T04:55:03.126 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.456+0000 7f3535800640 1 -- 172.21.15.43:0/3464799473 shutdown_connections 2024-09-19T04:55:03.126 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.456+0000 7f3535800640 1 --2- 172.21.15.43:0/3464799473 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f3514039260 0x7f351403b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:03.127 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.456+0000 7f3535800640 1 --2- 172.21.15.43:0/3464799473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35301076f0 0x7f353019b6f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:03.127 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.456+0000 7f3535800640 1 -- 172.21.15.43:0/3464799473 >> 172.21.15.43:0/3464799473 conn(0x7f3530102ec0 msgr2=0x7f35301039f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:03.127 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.456+0000 7f3535800640 1 -- 172.21.15.43:0/3464799473 shutdown_connections 2024-09-19T04:55:03.127 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:02.456+0000 7f3535800640 1 -- 172.21.15.43:0/3464799473 wait complete. 2024-09-19T04:55:03.127 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for the mgr to restart... 2024-09-19T04:55:03.127 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mgr epoch 9... 2024-09-19T04:55:04.030 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:03 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:55:03.721+0000 7fafba62d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T04:55:04.030 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:03 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:55:03.804+0000 7fafba62d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T04:55:05.018 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:04 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:55:04.749+0000 7fafba62d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T04:55:05.018 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:04 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:55:04.890+0000 7fafba62d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T04:55:05.280 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:05 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:55:05.060+0000 7fafba62d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:55:05.280 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:05 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:55:05.135+0000 7fafba62d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T04:55:05.681 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:05 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:55:05.312+0000 7fafba62d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T04:55:05.681 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:05 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:55:05.417+0000 7fafba62d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T04:55:05.682 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:05 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:55:05.491+0000 7fafba62d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T04:55:05.682 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:05 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:55:05.680+0000 7fafba62d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T04:55:05.943 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:05 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:55:05.760+0000 7fafba62d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T04:55:05.944 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:05 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:55:05.842+0000 7fafba62d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T04:55:05.944 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:05 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:55:05.942+0000 7fafba62d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T04:55:06.280 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:06 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:55:06.015+0000 7fafba62d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T04:55:06.281 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:06 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:55:06.088+0000 7fafba62d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T04:55:07.306 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:07 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:55:07.034+0000 7fafba62d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T04:55:07.307 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:07 smithi043 ceph-mon[30800]: Active manager daemon x restarted 2024-09-19T04:55:07.307 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:07 smithi043 ceph-mon[30800]: Activating manager daemon x 2024-09-19T04:55:07.307 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:07 smithi043 ceph-mon[30800]: osdmap e3: 0 total, 0 up, 0 in 2024-09-19T04:55:07.307 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:07 smithi043 ceph-mon[30800]: mgrmap e10: x(active, starting, since 0.00372069s) 2024-09-19T04:55:07.307 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:07 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:55:07.307 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:07 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T04:55:07.307 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:07 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T04:55:07.307 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:07 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T04:55:07.307 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:07 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T04:55:07.307 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:07 smithi043 ceph-mon[30800]: Manager daemon x is now available 2024-09-19T04:55:07.308 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:07 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:55:08.326 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:08 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T04:55:08.326 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:08 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T04:55:08.326 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:08 smithi043 ceph-mon[30800]: mgrmap e11: x(active, since 1.00702s) 2024-09-19T04:55:08.327 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:08 smithi043 ceph-mon[30800]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-19T04:55:08.327 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:08 smithi043 ceph-mon[30800]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-19T04:55:08.996 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:08 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:55:08.941+0000 7faf17c00640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:04:55:08] ENGINE Error in HTTPServer.serve 2024-09-19T04:55:08.996 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:08 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: Traceback (most recent call last): 2024-09-19T04:55:08.997 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:08 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T04:55:08.997 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:08 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: self._connections.run(self.expiration_interval) 2024-09-19T04:55:08.997 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:08 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T04:55:08.997 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:08 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: self._run(expiration_interval) 2024-09-19T04:55:08.997 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:08 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T04:55:08.997 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:08 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T04:55:08.997 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:08 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T04:55:08.997 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:08 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T04:55:08.997 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:08 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T04:55:08.997 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:08 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: s = self.context.wrap_socket( 2024-09-19T04:55:08.997 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:08 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T04:55:08.997 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:08 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: return self.sslsocket_class._create( 2024-09-19T04:55:08.997 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:08 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T04:55:08.998 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:08 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: self.do_handshake() 2024-09-19T04:55:08.998 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:08 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T04:55:08.998 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:08 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: self._sslobj.do_handshake() 2024-09-19T04:55:08.998 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:08 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T04:55:08.998 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 04:55:08 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T04:55:09.324 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-09-19T04:55:09.324 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-19T04:55:09.324 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-19T04:55:09.324 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-09-19T04:55:09.324 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:03.606+0000 7f7027400640 1 Processor -- start 2024-09-19T04:55:09.324 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:03.606+0000 7f7027400640 1 -- start start 2024-09-19T04:55:09.324 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:03.607+0000 7f7027400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70201098d0 0x7f7020109cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:09.324 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:03.607+0000 7f7027400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f702010a2a0 con 0x7f70201098d0 2024-09-19T04:55:09.324 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:03.607+0000 7f7024a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70201098d0 0x7f7020109cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:09.324 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:03.607+0000 7f7024a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70201098d0 0x7f7020109cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53604/0 (socket says 172.21.15.43:53604) 2024-09-19T04:55:09.324 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:03.607+0000 7f7024a00640 1 -- 172.21.15.43:0/3861551632 learned_addr learned my addr 172.21.15.43:0/3861551632 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:55:09.325 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:03.607+0000 7f7024a00640 1 -- 172.21.15.43:0/3861551632 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f702010aae0 con 0x7f70201098d0 2024-09-19T04:55:09.325 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:03.608+0000 7f7024a00640 1 --2- 172.21.15.43:0/3861551632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70201098d0 0x7f7020109cd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7014009820 tx=0x7f701402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d91c39bfb3b960b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:09.325 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:03.609+0000 7f701f400640 1 -- 172.21.15.43:0/3861551632 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7014037070 con 0x7f70201098d0 2024-09-19T04:55:09.325 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:03.609+0000 7f701f400640 1 -- 172.21.15.43:0/3861551632 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7014032920 con 0x7f70201098d0 2024-09-19T04:55:09.325 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:03.609+0000 7f701f400640 1 -- 172.21.15.43:0/3861551632 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7014030810 con 0x7f70201098d0 2024-09-19T04:55:09.325 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:03.609+0000 7f7027400640 1 -- 172.21.15.43:0/3861551632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70201098d0 msgr2=0x7f7020109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:09.325 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:03.609+0000 7f7027400640 1 --2- 172.21.15.43:0/3861551632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70201098d0 0x7f7020109cd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7014009820 tx=0x7f701402a270 comp rx=0 tx=0).stop 2024-09-19T04:55:09.325 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:03.610+0000 7f7027400640 1 -- 172.21.15.43:0/3861551632 shutdown_connections 2024-09-19T04:55:09.326 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:03.610+0000 7f7027400640 1 --2- 172.21.15.43:0/3861551632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70201098d0 0x7f7020109cd0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:09.326 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:03.610+0000 7f7027400640 1 -- 172.21.15.43:0/3861551632 >> 172.21.15.43:0/3861551632 conn(0x7f702007cee0 msgr2=0x7f702007d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:09.326 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:03.610+0000 7f7027400640 1 -- 172.21.15.43:0/3861551632 shutdown_connections 2024-09-19T04:55:09.326 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:03.610+0000 7f7027400640 1 -- 172.21.15.43:0/3861551632 wait complete. 2024-09-19T04:55:09.326 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:03.611+0000 7f7027400640 1 Processor -- start 2024-09-19T04:55:09.326 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:03.611+0000 7f7027400640 1 -- start start 2024-09-19T04:55:09.326 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:03.611+0000 7f7027400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70201098d0 0x7f702019d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:09.326 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:03.611+0000 7f7027400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f702010de90 con 0x7f70201098d0 2024-09-19T04:55:09.326 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:03.611+0000 7f7024a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70201098d0 0x7f702019d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:09.326 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:03.611+0000 7f7024a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70201098d0 0x7f702019d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53614/0 (socket says 172.21.15.43:53614) 2024-09-19T04:55:09.326 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:03.611+0000 7f7024a00640 1 -- 172.21.15.43:0/3872558153 learned_addr learned my addr 172.21.15.43:0/3872558153 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:55:09.326 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:03.612+0000 7f7024a00640 1 -- 172.21.15.43:0/3872558153 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f702019de80 con 0x7f70201098d0 2024-09-19T04:55:09.326 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:03.612+0000 7f7024a00640 1 --2- 172.21.15.43:0/3872558153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70201098d0 0x7f702019d940 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f70140097f0 tx=0x7f70140327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:09.327 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:03.612+0000 7f701d600640 1 -- 172.21.15.43:0/3872558153 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7014037040 con 0x7f70201098d0 2024-09-19T04:55:09.327 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:03.612+0000 7f701d600640 1 -- 172.21.15.43:0/3872558153 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7014030c40 con 0x7f70201098d0 2024-09-19T04:55:09.327 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:03.612+0000 7f701d600640 1 -- 172.21.15.43:0/3872558153 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f701403a5b0 con 0x7f70201098d0 2024-09-19T04:55:09.327 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:03.612+0000 7f7027400640 1 -- 172.21.15.43:0/3872558153 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f702019e110 con 0x7f70201098d0 2024-09-19T04:55:09.327 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:03.613+0000 7f7027400640 1 -- 172.21.15.43:0/3872558153 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70201a0e00 con 0x7f70201098d0 2024-09-19T04:55:09.327 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:03.613+0000 7f701d600640 1 -- 172.21.15.43:0/3872558153 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f701403a7f0 con 0x7f70201098d0 2024-09-19T04:55:09.327 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:03.614+0000 7f701d600640 1 --2- 172.21.15.43:0/3872558153 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f7008039200 0x7f700803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:09.327 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:03.614+0000 7f7027400640 1 -- 172.21.15.43:0/3872558153 --> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7020109cd0 con 0x7f7008039200 2024-09-19T04:55:09.327 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:03.614+0000 7f701d600640 1 -- 172.21.15.43:0/3872558153 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f701406c920 con 0x7f70201098d0 2024-09-19T04:55:09.327 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:03.614+0000 7f701fe00640 1 -- 172.21.15.43:0/3872558153 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f7008039200 msgr2=0x7f700803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/3241799377 2024-09-19T04:55:09.327 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:03.614+0000 7f701fe00640 1 --2- 172.21.15.43:0/3872558153 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f7008039200 0x7f700803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:55:09.327 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:03.815+0000 7f701fe00640 1 -- 172.21.15.43:0/3872558153 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f7008039200 msgr2=0x7f700803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/3241799377 2024-09-19T04:55:09.327 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:03.815+0000 7f701fe00640 1 --2- 172.21.15.43:0/3872558153 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f7008039200 0x7f700803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T04:55:09.328 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:04.216+0000 7f701fe00640 1 -- 172.21.15.43:0/3872558153 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f7008039200 msgr2=0x7f700803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/3241799377 2024-09-19T04:55:09.328 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:04.216+0000 7f701fe00640 1 --2- 172.21.15.43:0/3872558153 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f7008039200 0x7f700803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-19T04:55:09.328 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:05.017+0000 7f701fe00640 1 -- 172.21.15.43:0/3872558153 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f7008039200 msgr2=0x7f700803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/3241799377 2024-09-19T04:55:09.328 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:05.017+0000 7f701fe00640 1 --2- 172.21.15.43:0/3872558153 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f7008039200 0x7f700803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-19T04:55:09.328 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:06.618+0000 7f701fe00640 1 -- 172.21.15.43:0/3872558153 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f7008039200 msgr2=0x7f700803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/3241799377 2024-09-19T04:55:09.328 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:06.619+0000 7f701fe00640 1 --2- 172.21.15.43:0/3872558153 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f7008039200 0x7f700803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-09-19T04:55:09.328 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:07.039+0000 7f701d600640 1 -- 172.21.15.43:0/3872558153 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mgrmap(e 10) ==== 46241+0+0 (secure 0 0 0) 0x7f701402fd60 con 0x7f70201098d0 2024-09-19T04:55:09.328 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:07.039+0000 7f701d600640 1 -- 172.21.15.43:0/3872558153 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f7008039200 msgr2=0x7f700803b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:55:09.328 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:07.039+0000 7f701d600640 1 --2- 172.21.15.43:0/3872558153 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f7008039200 0x7f700803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:09.328 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:08.041+0000 7f701d600640 1 -- 172.21.15.43:0/3872558153 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 11) ==== 46368+0+0 (secure 0 0 0) 0x7f701406e610 con 0x7f70201098d0 2024-09-19T04:55:09.328 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:08.041+0000 7f701d600640 1 --2- 172.21.15.43:0/3872558153 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f700803cad0 0x7f700803eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:09.328 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:08.041+0000 7f701d600640 1 -- 172.21.15.43:0/3872558153 --> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f701400af30 con 0x7f700803cad0 2024-09-19T04:55:09.329 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:08.042+0000 7f701fe00640 1 --2- 172.21.15.43:0/3872558153 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f700803cad0 0x7f700803eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:09.329 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:08.043+0000 7f701fe00640 1 --2- 172.21.15.43:0/3872558153 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f700803cad0 0x7f700803eec0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f700c003980 tx=0x7f700c009290 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:09.329 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:08.044+0000 7f701d600640 1 -- 172.21.15.43:0/3872558153 <== mgr.14150 v2:172.21.15.43:6800/3167108925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f701400af30 con 0x7f700803cad0 2024-09-19T04:55:09.329 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:08.051+0000 7f7027400640 1 -- 172.21.15.43:0/3872558153 --> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f7020107be0 con 0x7f700803cad0 2024-09-19T04:55:09.329 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:08.052+0000 7f701d600640 1 -- 172.21.15.43:0/3872558153 <== mgr.14150 v2:172.21.15.43:6800/3167108925 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f7020107be0 con 0x7f700803cad0 2024-09-19T04:55:09.329 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:08.052+0000 7f7027400640 1 -- 172.21.15.43:0/3872558153 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f700803cad0 msgr2=0x7f700803eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:09.329 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:08.052+0000 7f7027400640 1 --2- 172.21.15.43:0/3872558153 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f700803cad0 0x7f700803eec0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f700c003980 tx=0x7f700c009290 comp rx=0 tx=0).stop 2024-09-19T04:55:09.329 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:08.052+0000 7f7027400640 1 -- 172.21.15.43:0/3872558153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70201098d0 msgr2=0x7f702019d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:09.329 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:08.052+0000 7f7027400640 1 --2- 172.21.15.43:0/3872558153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70201098d0 0x7f702019d940 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f70140097f0 tx=0x7f70140327b0 comp rx=0 tx=0).stop 2024-09-19T04:55:09.329 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:08.052+0000 7f7027400640 1 -- 172.21.15.43:0/3872558153 shutdown_connections 2024-09-19T04:55:09.329 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:08.052+0000 7f7027400640 1 --2- 172.21.15.43:0/3872558153 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f700803cad0 0x7f700803eec0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:09.329 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:08.052+0000 7f7027400640 1 --2- 172.21.15.43:0/3872558153 >> [v2:172.21.15.43:6800/3241799377,v1:172.21.15.43:6801/3241799377] conn(0x7f7008039200 0x7f700803b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:09.329 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:08.052+0000 7f7027400640 1 --2- 172.21.15.43:0/3872558153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70201098d0 0x7f702019d940 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:09.330 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:08.052+0000 7f7027400640 1 -- 172.21.15.43:0/3872558153 >> 172.21.15.43:0/3872558153 conn(0x7f702007cee0 msgr2=0x7f70201074d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:09.330 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:08.052+0000 7f7027400640 1 -- 172.21.15.43:0/3872558153 shutdown_connections 2024-09-19T04:55:09.330 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:08.053+0000 7f7027400640 1 -- 172.21.15.43:0/3872558153 wait complete. 2024-09-19T04:55:09.330 INFO:teuthology.orchestra.run.smithi043.stdout:mgr epoch 9 is available 2024-09-19T04:55:09.330 INFO:teuthology.orchestra.run.smithi043.stdout:Generating a dashboard self-signed certificate... 2024-09-19T04:55:09.819 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:09 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:09.819 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:09 smithi043 ceph-mon[30800]: [19/Sep/2024:04:55:08] ENGINE Bus STARTING 2024-09-19T04:55:09.819 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:09 smithi043 ceph-mon[30800]: [19/Sep/2024:04:55:08] ENGINE Serving on https://172.21.15.43:7150 2024-09-19T04:55:09.820 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:09 smithi043 ceph-mon[30800]: [19/Sep/2024:04:55:08] ENGINE Error in HTTPServer.serve 2024-09-19T04:55:09.820 INFO:journalctl@ceph.mon.a.smithi043.stdout: Traceback (most recent call last): 2024-09-19T04:55:09.820 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T04:55:09.820 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._connections.run(self.expiration_interval) 2024-09-19T04:55:09.820 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T04:55:09.820 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._run(expiration_interval) 2024-09-19T04:55:09.820 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T04:55:09.820 INFO:journalctl@ceph.mon.a.smithi043.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T04:55:09.820 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T04:55:09.820 INFO:journalctl@ceph.mon.a.smithi043.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T04:55:09.820 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T04:55:09.820 INFO:journalctl@ceph.mon.a.smithi043.stdout: s = self.context.wrap_socket( 2024-09-19T04:55:09.820 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T04:55:09.821 INFO:journalctl@ceph.mon.a.smithi043.stdout: return self.sslsocket_class._create( 2024-09-19T04:55:09.821 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T04:55:09.821 INFO:journalctl@ceph.mon.a.smithi043.stdout: self.do_handshake() 2024-09-19T04:55:09.821 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T04:55:09.821 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._sslobj.do_handshake() 2024-09-19T04:55:09.821 INFO:journalctl@ceph.mon.a.smithi043.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T04:55:09.821 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:09 smithi043 ceph-mon[30800]: [19/Sep/2024:04:55:09] ENGINE Serving on http://172.21.15.43:8765 2024-09-19T04:55:09.821 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:09 smithi043 ceph-mon[30800]: [19/Sep/2024:04:55:09] ENGINE Bus STARTED 2024-09-19T04:55:10.753 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-19T04:55:10.753 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:09.810+0000 7f385c800640 1 Processor -- start 2024-09-19T04:55:10.753 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:09.810+0000 7f385c800640 1 -- start start 2024-09-19T04:55:10.753 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:09.810+0000 7f385c800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38541099c0 0x7f3854109dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:10.753 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:09.810+0000 7f385c800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f385410a390 con 0x7f38541099c0 2024-09-19T04:55:10.753 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:09.811+0000 7f3859e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38541099c0 0x7f3854109dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:10.753 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:09.811+0000 7f3859e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38541099c0 0x7f3854109dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60146/0 (socket says 172.21.15.43:60146) 2024-09-19T04:55:10.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:09.811+0000 7f3859e00640 1 -- 172.21.15.43:0/1052303912 learned_addr learned my addr 172.21.15.43:0/1052303912 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:55:10.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:09.812+0000 7f3859e00640 1 -- 172.21.15.43:0/1052303912 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f385410ab00 con 0x7f38541099c0 2024-09-19T04:55:10.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:09.813+0000 7f3859e00640 1 --2- 172.21.15.43:0/1052303912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38541099c0 0x7f3854109dc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3848009820 tx=0x7f384802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a43e75b6704d1feb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:10.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:09.813+0000 7f3858a00640 1 -- 172.21.15.43:0/1052303912 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3848037070 con 0x7f38541099c0 2024-09-19T04:55:10.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:09.813+0000 7f3858a00640 1 -- 172.21.15.43:0/1052303912 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3848032920 con 0x7f38541099c0 2024-09-19T04:55:10.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:09.813+0000 7f3858a00640 1 -- 172.21.15.43:0/1052303912 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3848030810 con 0x7f38541099c0 2024-09-19T04:55:10.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:09.814+0000 7f385c800640 1 -- 172.21.15.43:0/1052303912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38541099c0 msgr2=0x7f3854109dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:10.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:09.814+0000 7f385c800640 1 --2- 172.21.15.43:0/1052303912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38541099c0 0x7f3854109dc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3848009820 tx=0x7f384802a270 comp rx=0 tx=0).stop 2024-09-19T04:55:10.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:09.814+0000 7f385c800640 1 -- 172.21.15.43:0/1052303912 shutdown_connections 2024-09-19T04:55:10.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:09.814+0000 7f385c800640 1 --2- 172.21.15.43:0/1052303912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38541099c0 0x7f3854109dc0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:10.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:09.814+0000 7f385c800640 1 -- 172.21.15.43:0/1052303912 >> 172.21.15.43:0/1052303912 conn(0x7f385407ceb0 msgr2=0x7f385407d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:10.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:09.814+0000 7f385c800640 1 -- 172.21.15.43:0/1052303912 shutdown_connections 2024-09-19T04:55:10.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:09.814+0000 7f385c800640 1 -- 172.21.15.43:0/1052303912 wait complete. 2024-09-19T04:55:10.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:09.815+0000 7f385c800640 1 Processor -- start 2024-09-19T04:55:10.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:09.815+0000 7f385c800640 1 -- start start 2024-09-19T04:55:10.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:09.816+0000 7f385c800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38541099c0 0x7f385419d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:10.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:09.816+0000 7f385c800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f385410deb0 con 0x7f38541099c0 2024-09-19T04:55:10.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:09.816+0000 7f3859e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38541099c0 0x7f385419d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:10.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:09.816+0000 7f3859e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38541099c0 0x7f385419d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60152/0 (socket says 172.21.15.43:60152) 2024-09-19T04:55:10.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:09.816+0000 7f3859e00640 1 -- 172.21.15.43:0/4070296229 learned_addr learned my addr 172.21.15.43:0/4070296229 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:55:10.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:09.816+0000 7f3859e00640 1 -- 172.21.15.43:0/4070296229 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f385419deb0 con 0x7f38541099c0 2024-09-19T04:55:10.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:09.817+0000 7f3859e00640 1 --2- 172.21.15.43:0/4070296229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38541099c0 0x7f385419d970 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3848030560 tx=0x7f3848032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:10.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:09.817+0000 7f384ea00640 1 -- 172.21.15.43:0/4070296229 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3848037040 con 0x7f38541099c0 2024-09-19T04:55:10.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:09.817+0000 7f384ea00640 1 -- 172.21.15.43:0/4070296229 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3848040070 con 0x7f38541099c0 2024-09-19T04:55:10.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:09.817+0000 7f385c800640 1 -- 172.21.15.43:0/4070296229 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f385419e140 con 0x7f38541099c0 2024-09-19T04:55:10.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:09.817+0000 7f384ea00640 1 -- 172.21.15.43:0/4070296229 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f384803a6d0 con 0x7f38541099c0 2024-09-19T04:55:10.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:09.817+0000 7f385c800640 1 -- 172.21.15.43:0/4070296229 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38541a0e30 con 0x7f38541099c0 2024-09-19T04:55:10.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:09.819+0000 7f385c800640 1 -- 172.21.15.43:0/4070296229 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f385410e030 con 0x7f38541099c0 2024-09-19T04:55:10.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:09.819+0000 7f384ea00640 1 -- 172.21.15.43:0/4070296229 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f384803b420 con 0x7f38541099c0 2024-09-19T04:55:10.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:09.819+0000 7f384ea00640 1 --2- 172.21.15.43:0/4070296229 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f383c039200 0x7f383c03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:10.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:09.819+0000 7f384ea00640 1 -- 172.21.15.43:0/4070296229 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f384806ca90 con 0x7f38541099c0 2024-09-19T04:55:10.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:09.820+0000 7f3859400640 1 --2- 172.21.15.43:0/4070296229 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f383c039200 0x7f383c03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:10.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:09.820+0000 7f3859400640 1 --2- 172.21.15.43:0/4070296229 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f383c039200 0x7f383c03b6c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f38440098c0 tx=0x7f3844006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:10.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:09.825+0000 7f384ea00640 1 -- 172.21.15.43:0/4070296229 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3848043360 con 0x7f38541099c0 2024-09-19T04:55:10.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:10.018+0000 7f385c800640 1 -- 172.21.15.43:0/4070296229 --> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f3854107c40 con 0x7f383c039200 2024-09-19T04:55:10.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:10.072+0000 7f384ea00640 1 -- 172.21.15.43:0/4070296229 <== mgr.14150 v2:172.21.15.43:6800/3167108925 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f3854107c40 con 0x7f383c039200 2024-09-19T04:55:10.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:10.075+0000 7f385c800640 1 -- 172.21.15.43:0/4070296229 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f383c039200 msgr2=0x7f383c03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:10.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:10.075+0000 7f385c800640 1 --2- 172.21.15.43:0/4070296229 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f383c039200 0x7f383c03b6c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f38440098c0 tx=0x7f3844006db0 comp rx=0 tx=0).stop 2024-09-19T04:55:10.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:10.075+0000 7f385c800640 1 -- 172.21.15.43:0/4070296229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38541099c0 msgr2=0x7f385419d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:10.757 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:10.075+0000 7f385c800640 1 --2- 172.21.15.43:0/4070296229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38541099c0 0x7f385419d970 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3848030560 tx=0x7f3848032670 comp rx=0 tx=0).stop 2024-09-19T04:55:10.757 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:10.075+0000 7f385c800640 1 -- 172.21.15.43:0/4070296229 shutdown_connections 2024-09-19T04:55:10.757 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:10.075+0000 7f385c800640 1 --2- 172.21.15.43:0/4070296229 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f383c039200 0x7f383c03b6c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:10.757 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:10.075+0000 7f385c800640 1 --2- 172.21.15.43:0/4070296229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38541099c0 0x7f385419d970 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:10.757 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:10.075+0000 7f385c800640 1 -- 172.21.15.43:0/4070296229 >> 172.21.15.43:0/4070296229 conn(0x7f385407ceb0 msgr2=0x7f3854107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:10.757 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:10.075+0000 7f385c800640 1 -- 172.21.15.43:0/4070296229 shutdown_connections 2024-09-19T04:55:10.757 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:10.075+0000 7f385c800640 1 -- 172.21.15.43:0/4070296229 wait complete. 2024-09-19T04:55:10.757 INFO:teuthology.orchestra.run.smithi043.stdout:Creating initial admin user... 2024-09-19T04:55:11.012 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:10 smithi043 ceph-mon[30800]: mgrmap e12: x(active, since 2s) 2024-09-19T04:55:11.012 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:10 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:11.012 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:10 smithi043 ceph-mon[30800]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:55:11.012 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:10 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:11.012 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:10 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:12.960 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$nY80MOe8f63C5YHRvlPF7uV8KMiFDK0/uX7anPgPxhv52Js198a1G", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726721711, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-19T04:55:12.960 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.494+0000 7fdf9c800640 1 Processor -- start 2024-09-19T04:55:12.960 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.495+0000 7fdf9c800640 1 -- start start 2024-09-19T04:55:12.960 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.495+0000 7fdf9c800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf9407de70 0x7fdf9407c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:12.960 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.495+0000 7fdf9c800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9407c8f0 con 0x7fdf9407de70 2024-09-19T04:55:12.961 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.495+0000 7fdf99e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf9407de70 0x7fdf9407c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:12.961 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.495+0000 7fdf99e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf9407de70 0x7fdf9407c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60164/0 (socket says 172.21.15.43:60164) 2024-09-19T04:55:12.961 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.495+0000 7fdf99e00640 1 -- 172.21.15.43:0/85509854 learned_addr learned my addr 172.21.15.43:0/85509854 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:55:12.961 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.496+0000 7fdf99e00640 1 -- 172.21.15.43:0/85509854 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9407ca70 con 0x7fdf9407de70 2024-09-19T04:55:12.961 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.497+0000 7fdf99e00640 1 --2- 172.21.15.43:0/85509854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf9407de70 0x7fdf9407c320 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdf88009820 tx=0x7fdf8802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d20dda284ea73129 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:12.961 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.497+0000 7fdf98a00640 1 -- 172.21.15.43:0/85509854 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf88037070 con 0x7fdf9407de70 2024-09-19T04:55:12.961 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.497+0000 7fdf98a00640 1 -- 172.21.15.43:0/85509854 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf88032920 con 0x7fdf9407de70 2024-09-19T04:55:12.961 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.497+0000 7fdf98a00640 1 -- 172.21.15.43:0/85509854 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf88030810 con 0x7fdf9407de70 2024-09-19T04:55:12.961 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.498+0000 7fdf9c800640 1 -- 172.21.15.43:0/85509854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf9407de70 msgr2=0x7fdf9407c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:12.961 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.498+0000 7fdf9c800640 1 --2- 172.21.15.43:0/85509854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf9407de70 0x7fdf9407c320 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdf88009820 tx=0x7fdf8802a270 comp rx=0 tx=0).stop 2024-09-19T04:55:12.961 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.498+0000 7fdf9c800640 1 -- 172.21.15.43:0/85509854 shutdown_connections 2024-09-19T04:55:12.961 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.498+0000 7fdf9c800640 1 --2- 172.21.15.43:0/85509854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf9407de70 0x7fdf9407c320 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:12.962 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.498+0000 7fdf9c800640 1 -- 172.21.15.43:0/85509854 >> 172.21.15.43:0/85509854 conn(0x7fdf94102ef0 msgr2=0x7fdf94105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:12.962 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.498+0000 7fdf9c800640 1 -- 172.21.15.43:0/85509854 shutdown_connections 2024-09-19T04:55:12.962 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.498+0000 7fdf9c800640 1 -- 172.21.15.43:0/85509854 wait complete. 2024-09-19T04:55:12.962 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.499+0000 7fdf9c800640 1 Processor -- start 2024-09-19T04:55:12.962 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.499+0000 7fdf9c800640 1 -- start start 2024-09-19T04:55:12.962 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.499+0000 7fdf9c800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf9407de70 0x7fdf9419d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:12.962 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.499+0000 7fdf9c800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9410dd90 con 0x7fdf9407de70 2024-09-19T04:55:12.962 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.499+0000 7fdf99e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf9407de70 0x7fdf9419d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:12.962 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.499+0000 7fdf99e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf9407de70 0x7fdf9419d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60178/0 (socket says 172.21.15.43:60178) 2024-09-19T04:55:12.962 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.499+0000 7fdf99e00640 1 -- 172.21.15.43:0/2730116597 learned_addr learned my addr 172.21.15.43:0/2730116597 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:55:12.962 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.500+0000 7fdf99e00640 1 -- 172.21.15.43:0/2730116597 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9419de50 con 0x7fdf9407de70 2024-09-19T04:55:12.962 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.500+0000 7fdf99e00640 1 --2- 172.21.15.43:0/2730116597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf9407de70 0x7fdf9419d910 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdf880097f0 tx=0x7fdf880327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:12.962 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.500+0000 7fdf8ea00640 1 -- 172.21.15.43:0/2730116597 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf88037040 con 0x7fdf9407de70 2024-09-19T04:55:12.963 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.500+0000 7fdf8ea00640 1 -- 172.21.15.43:0/2730116597 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf88030c40 con 0x7fdf9407de70 2024-09-19T04:55:12.963 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.500+0000 7fdf9c800640 1 -- 172.21.15.43:0/2730116597 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf9419e0e0 con 0x7fdf9407de70 2024-09-19T04:55:12.963 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.500+0000 7fdf8ea00640 1 -- 172.21.15.43:0/2730116597 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf8803a5d0 con 0x7fdf9407de70 2024-09-19T04:55:12.963 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.500+0000 7fdf9c800640 1 -- 172.21.15.43:0/2730116597 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf941a0dd0 con 0x7fdf9407de70 2024-09-19T04:55:12.963 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.502+0000 7fdf8ea00640 1 -- 172.21.15.43:0/2730116597 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7fdf8803b3f0 con 0x7fdf9407de70 2024-09-19T04:55:12.963 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.502+0000 7fdf9c800640 1 -- 172.21.15.43:0/2730116597 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf9407c3a0 con 0x7fdf9407de70 2024-09-19T04:55:12.963 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.502+0000 7fdf8ea00640 1 --2- 172.21.15.43:0/2730116597 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fdf7c039260 0x7fdf7c03b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:12.963 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.502+0000 7fdf8ea00640 1 -- 172.21.15.43:0/2730116597 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdf8806c840 con 0x7fdf9407de70 2024-09-19T04:55:12.963 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.502+0000 7fdf99400640 1 --2- 172.21.15.43:0/2730116597 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fdf7c039260 0x7fdf7c03b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:12.963 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.503+0000 7fdf99400640 1 --2- 172.21.15.43:0/2730116597 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fdf7c039260 0x7fdf7c03b720 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdf840098c0 tx=0x7fdf84006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:12.963 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.507+0000 7fdf8ea00640 1 -- 172.21.15.43:0/2730116597 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdf8803b6e0 con 0x7fdf9407de70 2024-09-19T04:55:12.963 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.711+0000 7fdf9c800640 1 -- 172.21.15.43:0/2730116597 --> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fdf9419e5c0 con 0x7fdf7c039260 2024-09-19T04:55:12.963 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.961+0000 7fdf8ea00640 1 -- 172.21.15.43:0/2730116597 <== mgr.14150 v2:172.21.15.43:6800/3167108925 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fdf9419e5c0 con 0x7fdf7c039260 2024-09-19T04:55:12.963 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.963+0000 7fdf9c800640 1 -- 172.21.15.43:0/2730116597 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fdf7c039260 msgr2=0x7fdf7c03b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:12.963 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.963+0000 7fdf9c800640 1 --2- 172.21.15.43:0/2730116597 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fdf7c039260 0x7fdf7c03b720 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdf840098c0 tx=0x7fdf84006db0 comp rx=0 tx=0).stop 2024-09-19T04:55:12.964 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.963+0000 7fdf9c800640 1 -- 172.21.15.43:0/2730116597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf9407de70 msgr2=0x7fdf9419d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:12.964 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.963+0000 7fdf9c800640 1 --2- 172.21.15.43:0/2730116597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf9407de70 0x7fdf9419d910 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdf880097f0 tx=0x7fdf880327b0 comp rx=0 tx=0).stop 2024-09-19T04:55:12.964 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.963+0000 7fdf9c800640 1 -- 172.21.15.43:0/2730116597 shutdown_connections 2024-09-19T04:55:12.964 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.963+0000 7fdf9c800640 1 --2- 172.21.15.43:0/2730116597 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fdf7c039260 0x7fdf7c03b720 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:12.964 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.963+0000 7fdf9c800640 1 --2- 172.21.15.43:0/2730116597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf9407de70 0x7fdf9419d910 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:12.964 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.963+0000 7fdf9c800640 1 -- 172.21.15.43:0/2730116597 >> 172.21.15.43:0/2730116597 conn(0x7fdf94102ef0 msgr2=0x7fdf94103a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:12.964 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.963+0000 7fdf9c800640 1 -- 172.21.15.43:0/2730116597 shutdown_connections 2024-09-19T04:55:12.964 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:11.963+0000 7fdf9c800640 1 -- 172.21.15.43:0/2730116597 wait complete. 2024-09-19T04:55:12.964 INFO:teuthology.orchestra.run.smithi043.stdout:Fetching dashboard port number... 2024-09-19T04:55:13.224 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:12 smithi043 ceph-mon[30800]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:55:13.224 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:12 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:14.221 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:13 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/4010045309' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-19T04:55:14.231 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 8443 2024-09-19T04:55:14.232 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.424+0000 7fae96800640 1 Processor -- start 2024-09-19T04:55:14.232 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.424+0000 7fae96800640 1 -- start start 2024-09-19T04:55:14.232 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.425+0000 7fae96800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae90107740 0x7fae90107b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:14.232 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.425+0000 7fae96800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fae90108110 con 0x7fae90107740 2024-09-19T04:55:14.232 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.425+0000 7fae8f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae90107740 0x7fae90107b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:14.232 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.425+0000 7fae8f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae90107740 0x7fae90107b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60186/0 (socket says 172.21.15.43:60186) 2024-09-19T04:55:14.232 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.425+0000 7fae8f400640 1 -- 172.21.15.43:0/4190491262 learned_addr learned my addr 172.21.15.43:0/4190491262 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:55:14.232 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.425+0000 7fae8f400640 1 -- 172.21.15.43:0/4190491262 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae901088f0 con 0x7fae90107740 2024-09-19T04:55:14.233 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.426+0000 7fae8f400640 1 --2- 172.21.15.43:0/4190491262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae90107740 0x7fae90107b40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fae80009820 tx=0x7fae8002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad9d1dd595cb860 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:14.233 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.427+0000 7fae8e000640 1 -- 172.21.15.43:0/4190491262 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae80037070 con 0x7fae90107740 2024-09-19T04:55:14.233 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.427+0000 7fae8e000640 1 -- 172.21.15.43:0/4190491262 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae80032920 con 0x7fae90107740 2024-09-19T04:55:14.233 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.427+0000 7fae8e000640 1 -- 172.21.15.43:0/4190491262 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae80030810 con 0x7fae90107740 2024-09-19T04:55:14.233 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.427+0000 7fae96800640 1 -- 172.21.15.43:0/4190491262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae90107740 msgr2=0x7fae90107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:14.233 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.427+0000 7fae96800640 1 --2- 172.21.15.43:0/4190491262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae90107740 0x7fae90107b40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fae80009820 tx=0x7fae8002a270 comp rx=0 tx=0).stop 2024-09-19T04:55:14.233 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.427+0000 7fae96800640 1 -- 172.21.15.43:0/4190491262 shutdown_connections 2024-09-19T04:55:14.233 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.427+0000 7fae96800640 1 --2- 172.21.15.43:0/4190491262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae90107740 0x7fae90107b40 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:14.233 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.427+0000 7fae96800640 1 -- 172.21.15.43:0/4190491262 >> 172.21.15.43:0/4190491262 conn(0x7fae90102ef0 msgr2=0x7fae90105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:14.233 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.427+0000 7fae96800640 1 -- 172.21.15.43:0/4190491262 shutdown_connections 2024-09-19T04:55:14.233 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.427+0000 7fae96800640 1 -- 172.21.15.43:0/4190491262 wait complete. 2024-09-19T04:55:14.233 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.428+0000 7fae96800640 1 Processor -- start 2024-09-19T04:55:14.233 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.429+0000 7fae96800640 1 -- start start 2024-09-19T04:55:14.234 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.429+0000 7fae96800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae90107740 0x7fae9019b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:14.234 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.429+0000 7fae96800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fae9010bca0 con 0x7fae90107740 2024-09-19T04:55:14.234 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.429+0000 7fae8f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae90107740 0x7fae9019b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:14.234 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.429+0000 7fae8f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae90107740 0x7fae9019b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60198/0 (socket says 172.21.15.43:60198) 2024-09-19T04:55:14.234 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.429+0000 7fae8f400640 1 -- 172.21.15.43:0/4010045309 learned_addr learned my addr 172.21.15.43:0/4010045309 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:55:14.234 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.429+0000 7fae8f400640 1 -- 172.21.15.43:0/4010045309 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae9019bcc0 con 0x7fae90107740 2024-09-19T04:55:14.234 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.430+0000 7fae8f400640 1 --2- 172.21.15.43:0/4010045309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae90107740 0x7fae9019b780 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fae800097f0 tx=0x7fae800327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:14.234 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.430+0000 7fae87e00640 1 -- 172.21.15.43:0/4010045309 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae80037040 con 0x7fae90107740 2024-09-19T04:55:14.234 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.430+0000 7fae87e00640 1 -- 172.21.15.43:0/4010045309 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae80030c40 con 0x7fae90107740 2024-09-19T04:55:14.234 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.430+0000 7fae87e00640 1 -- 172.21.15.43:0/4010045309 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae8003a5b0 con 0x7fae90107740 2024-09-19T04:55:14.234 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.430+0000 7fae96800640 1 -- 172.21.15.43:0/4010045309 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae9019bf50 con 0x7fae90107740 2024-09-19T04:55:14.234 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.430+0000 7fae96800640 1 -- 172.21.15.43:0/4010045309 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae9019ec40 con 0x7fae90107740 2024-09-19T04:55:14.234 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.431+0000 7fae87e00640 1 -- 172.21.15.43:0/4010045309 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7fae8003a7a0 con 0x7fae90107740 2024-09-19T04:55:14.235 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.431+0000 7fae96800640 1 -- 172.21.15.43:0/4010045309 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae9010be20 con 0x7fae90107740 2024-09-19T04:55:14.235 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.432+0000 7fae87e00640 1 --2- 172.21.15.43:0/4010045309 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fae74039260 0x7fae7403b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:14.235 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.432+0000 7fae87e00640 1 -- 172.21.15.43:0/4010045309 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fae8006c7d0 con 0x7fae90107740 2024-09-19T04:55:14.235 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.432+0000 7fae8ea00640 1 --2- 172.21.15.43:0/4010045309 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fae74039260 0x7fae7403b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:14.235 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.433+0000 7fae8ea00640 1 --2- 172.21.15.43:0/4010045309 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fae74039260 0x7fae7403b720 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fae7c0098c0 tx=0x7fae7c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:14.235 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.437+0000 7fae87e00640 1 -- 172.21.15.43:0/4010045309 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fae800371e0 con 0x7fae90107740 2024-09-19T04:55:14.235 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.623+0000 7fae96800640 1 -- 172.21.15.43:0/4010045309 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fae9019ee10 con 0x7fae90107740 2024-09-19T04:55:14.235 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.624+0000 7fae87e00640 1 -- 172.21.15.43:0/4010045309 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fae8003b3f0 con 0x7fae90107740 2024-09-19T04:55:14.235 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.626+0000 7fae96800640 1 -- 172.21.15.43:0/4010045309 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fae74039260 msgr2=0x7fae7403b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:14.235 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.626+0000 7fae96800640 1 --2- 172.21.15.43:0/4010045309 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fae74039260 0x7fae7403b720 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fae7c0098c0 tx=0x7fae7c006db0 comp rx=0 tx=0).stop 2024-09-19T04:55:14.235 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.626+0000 7fae96800640 1 -- 172.21.15.43:0/4010045309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae90107740 msgr2=0x7fae9019b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:14.235 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.626+0000 7fae96800640 1 --2- 172.21.15.43:0/4010045309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae90107740 0x7fae9019b780 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fae800097f0 tx=0x7fae800327b0 comp rx=0 tx=0).stop 2024-09-19T04:55:14.235 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.626+0000 7fae96800640 1 -- 172.21.15.43:0/4010045309 shutdown_connections 2024-09-19T04:55:14.235 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.626+0000 7fae96800640 1 --2- 172.21.15.43:0/4010045309 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fae74039260 0x7fae7403b720 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:14.236 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.626+0000 7fae96800640 1 --2- 172.21.15.43:0/4010045309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae90107740 0x7fae9019b780 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:14.236 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.626+0000 7fae96800640 1 -- 172.21.15.43:0/4010045309 >> 172.21.15.43:0/4010045309 conn(0x7fae90102ef0 msgr2=0x7fae90103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:14.236 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.626+0000 7fae96800640 1 -- 172.21.15.43:0/4010045309 shutdown_connections 2024-09-19T04:55:14.236 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:13.626+0000 7fae96800640 1 -- 172.21.15.43:0/4010045309 wait complete. 2024-09-19T04:55:14.248 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T04:55:14.248 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout disabled 2024-09-19T04:55:14.262 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T04:55:14.262 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-09-19T04:55:14.262 INFO:teuthology.orchestra.run.smithi043.stdout:firewalld.service is not enabled 2024-09-19T04:55:14.262 INFO:teuthology.orchestra.run.smithi043.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-19T04:55:14.264 INFO:teuthology.orchestra.run.smithi043.stdout:Ceph Dashboard is now available at: 2024-09-19T04:55:14.264 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T04:55:14.264 INFO:teuthology.orchestra.run.smithi043.stdout: URL: https://smithi043.front.sepia.ceph.com:8443/ 2024-09-19T04:55:14.265 INFO:teuthology.orchestra.run.smithi043.stdout: User: admin 2024-09-19T04:55:14.265 INFO:teuthology.orchestra.run.smithi043.stdout: Password: 1wsrhxxjad 2024-09-19T04:55:14.265 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T04:55:14.265 INFO:teuthology.orchestra.run.smithi043.stdout:Saving cluster configuration to /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/config directory 2024-09-19T04:55:15.070 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:14 smithi043 ceph-mon[30800]: mgrmap e13: x(active, since 6s) 2024-09-19T04:55:15.651 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:14.774+0000 7fbb49800640 1 Processor -- start 2024-09-19T04:55:15.651 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:14.775+0000 7fbb49800640 1 -- start start 2024-09-19T04:55:15.651 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:14.775+0000 7fbb49800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb44109960 0x7fbb44109d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:15.651 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:14.775+0000 7fbb49800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4410a330 con 0x7fbb44109960 2024-09-19T04:55:15.652 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:14.775+0000 7fbb42a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb44109960 0x7fbb44109d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:15.652 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:14.775+0000 7fbb42a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb44109960 0x7fbb44109d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60214/0 (socket says 172.21.15.43:60214) 2024-09-19T04:55:15.652 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:14.775+0000 7fbb42a00640 1 -- 172.21.15.43:0/3673253029 learned_addr learned my addr 172.21.15.43:0/3673253029 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:55:15.652 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:14.776+0000 7fbb42a00640 1 -- 172.21.15.43:0/3673253029 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb4410aad0 con 0x7fbb44109960 2024-09-19T04:55:15.652 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:14.777+0000 7fbb42a00640 1 --2- 172.21.15.43:0/3673253029 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb44109960 0x7fbb44109d60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbb34009820 tx=0x7fbb3402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6227c5eeb9c9ef5e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:15.652 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:14.777+0000 7fbb42000640 1 -- 172.21.15.43:0/3673253029 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb34037070 con 0x7fbb44109960 2024-09-19T04:55:15.652 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:14.777+0000 7fbb42000640 1 -- 172.21.15.43:0/3673253029 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb34032920 con 0x7fbb44109960 2024-09-19T04:55:15.652 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:14.777+0000 7fbb42000640 1 -- 172.21.15.43:0/3673253029 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb34030810 con 0x7fbb44109960 2024-09-19T04:55:15.652 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:14.777+0000 7fbb49800640 1 -- 172.21.15.43:0/3673253029 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb44109960 msgr2=0x7fbb44109d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:15.652 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:14.777+0000 7fbb49800640 1 --2- 172.21.15.43:0/3673253029 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb44109960 0x7fbb44109d60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbb34009820 tx=0x7fbb3402a270 comp rx=0 tx=0).stop 2024-09-19T04:55:15.652 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:14.778+0000 7fbb49800640 1 -- 172.21.15.43:0/3673253029 shutdown_connections 2024-09-19T04:55:15.652 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:14.778+0000 7fbb49800640 1 --2- 172.21.15.43:0/3673253029 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb44109960 0x7fbb44109d60 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:15.653 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:14.778+0000 7fbb49800640 1 -- 172.21.15.43:0/3673253029 >> 172.21.15.43:0/3673253029 conn(0x7fbb4407cf30 msgr2=0x7fbb4407d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:15.653 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:14.778+0000 7fbb49800640 1 -- 172.21.15.43:0/3673253029 shutdown_connections 2024-09-19T04:55:15.653 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:14.778+0000 7fbb49800640 1 -- 172.21.15.43:0/3673253029 wait complete. 2024-09-19T04:55:15.653 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:14.779+0000 7fbb49800640 1 Processor -- start 2024-09-19T04:55:15.653 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:14.779+0000 7fbb49800640 1 -- start start 2024-09-19T04:55:15.653 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:14.779+0000 7fbb49800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb44109960 0x7fbb4419d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:15.653 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:14.779+0000 7fbb49800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4410de80 con 0x7fbb44109960 2024-09-19T04:55:15.653 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:14.779+0000 7fbb42a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb44109960 0x7fbb4419d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:15.653 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:14.779+0000 7fbb42a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb44109960 0x7fbb4419d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60218/0 (socket says 172.21.15.43:60218) 2024-09-19T04:55:15.653 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:14.779+0000 7fbb42a00640 1 -- 172.21.15.43:0/316425054 learned_addr learned my addr 172.21.15.43:0/316425054 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:55:15.653 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:14.780+0000 7fbb42a00640 1 -- 172.21.15.43:0/316425054 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb4419de80 con 0x7fbb44109960 2024-09-19T04:55:15.653 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:14.780+0000 7fbb42a00640 1 --2- 172.21.15.43:0/316425054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb44109960 0x7fbb4419d940 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbb34030560 tx=0x7fbb34032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:15.653 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:14.780+0000 7fbb3b400640 1 -- 172.21.15.43:0/316425054 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb34037040 con 0x7fbb44109960 2024-09-19T04:55:15.654 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:14.780+0000 7fbb3b400640 1 -- 172.21.15.43:0/316425054 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb34040070 con 0x7fbb44109960 2024-09-19T04:55:15.654 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:14.780+0000 7fbb49800640 1 -- 172.21.15.43:0/316425054 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb4419e110 con 0x7fbb44109960 2024-09-19T04:55:15.654 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:14.780+0000 7fbb3b400640 1 -- 172.21.15.43:0/316425054 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb3403a6d0 con 0x7fbb44109960 2024-09-19T04:55:15.654 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:14.780+0000 7fbb49800640 1 -- 172.21.15.43:0/316425054 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb441a0e00 con 0x7fbb44109960 2024-09-19T04:55:15.654 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:14.782+0000 7fbb3b400640 1 -- 172.21.15.43:0/316425054 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fbb3403b420 con 0x7fbb44109960 2024-09-19T04:55:15.654 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:14.781+0000 7fbb49800640 1 -- 172.21.15.43:0/316425054 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb4410e000 con 0x7fbb44109960 2024-09-19T04:55:15.654 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:14.782+0000 7fbb3b400640 1 --2- 172.21.15.43:0/316425054 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fbb28039300 0x7fbb2803b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:15.654 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:14.782+0000 7fbb3b400640 1 -- 172.21.15.43:0/316425054 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbb3406ca30 con 0x7fbb44109960 2024-09-19T04:55:15.654 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:14.782+0000 7fbb3be00640 1 --2- 172.21.15.43:0/316425054 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fbb28039300 0x7fbb2803b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:15.654 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:14.783+0000 7fbb3be00640 1 --2- 172.21.15.43:0/316425054 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fbb28039300 0x7fbb2803b7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c0098c0 tx=0x7fbb2c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:15.654 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:14.787+0000 7fbb3b400640 1 -- 172.21.15.43:0/316425054 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbb34042b50 con 0x7fbb44109960 2024-09-19T04:55:15.654 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:15.068+0000 7fbb49800640 1 -- 172.21.15.43:0/316425054 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fbb441a0fd0 con 0x7fbb44109960 2024-09-19T04:55:15.654 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:15.071+0000 7fbb3b400640 1 -- 172.21.15.43:0/316425054 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7fbb34043360 con 0x7fbb44109960 2024-09-19T04:55:15.655 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-19T04:55:15.655 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:15.073+0000 7fbb49800640 1 -- 172.21.15.43:0/316425054 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fbb28039300 msgr2=0x7fbb2803b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:15.655 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:15.073+0000 7fbb49800640 1 --2- 172.21.15.43:0/316425054 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fbb28039300 0x7fbb2803b7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c0098c0 tx=0x7fbb2c006db0 comp rx=0 tx=0).stop 2024-09-19T04:55:15.655 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:15.073+0000 7fbb49800640 1 -- 172.21.15.43:0/316425054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb44109960 msgr2=0x7fbb4419d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:15.655 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:15.073+0000 7fbb49800640 1 --2- 172.21.15.43:0/316425054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb44109960 0x7fbb4419d940 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbb34030560 tx=0x7fbb34032670 comp rx=0 tx=0).stop 2024-09-19T04:55:15.655 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:15.073+0000 7fbb49800640 1 -- 172.21.15.43:0/316425054 shutdown_connections 2024-09-19T04:55:15.655 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:15.073+0000 7fbb49800640 1 --2- 172.21.15.43:0/316425054 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fbb28039300 0x7fbb2803b7c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:15.655 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:15.073+0000 7fbb49800640 1 --2- 172.21.15.43:0/316425054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb44109960 0x7fbb4419d940 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:15.655 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:15.073+0000 7fbb49800640 1 -- 172.21.15.43:0/316425054 >> 172.21.15.43:0/316425054 conn(0x7fbb4407cf30 msgr2=0x7fbb44107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:15.655 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:15.073+0000 7fbb49800640 1 -- 172.21.15.43:0/316425054 shutdown_connections 2024-09-19T04:55:15.655 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-19T04:55:15.073+0000 7fbb49800640 1 -- 172.21.15.43:0/316425054 wait complete. 2024-09-19T04:55:15.655 INFO:teuthology.orchestra.run.smithi043.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-19T04:55:15.655 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T04:55:15.655 INFO:teuthology.orchestra.run.smithi043.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-19T04:55:15.656 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T04:55:15.656 INFO:teuthology.orchestra.run.smithi043.stdout:Or, if you are only running a single cluster on this host: 2024-09-19T04:55:15.656 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T04:55:15.656 INFO:teuthology.orchestra.run.smithi043.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-19T04:55:15.656 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T04:55:15.656 INFO:teuthology.orchestra.run.smithi043.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-19T04:55:15.656 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T04:55:15.656 INFO:teuthology.orchestra.run.smithi043.stdout: ceph telemetry on 2024-09-19T04:55:15.656 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T04:55:15.656 INFO:teuthology.orchestra.run.smithi043.stdout:For more information see: 2024-09-19T04:55:15.656 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T04:55:15.656 INFO:teuthology.orchestra.run.smithi043.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-19T04:55:15.656 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T04:55:15.657 INFO:teuthology.orchestra.run.smithi043.stdout:Bootstrap complete. 2024-09-19T04:55:15.703 INFO:tasks.cephadm:Fetching config... 2024-09-19T04:55:15.703 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-19T04:55:15.703 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-19T04:55:15.739 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-19T04:55:15.739 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-19T04:55:15.739 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-19T04:55:15.815 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-19T04:55:15.815 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-19T04:55:15.815 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-19T04:55:15.912 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-19T04:55:15.912 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-19T04:55:15.912 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-19T04:55:15.989 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-19T04:55:15.990 DEBUG:teuthology.orchestra.run.smithi043:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCjTG2qRtuHcxvKcvb7xgsggkW7AmLgD/NygkD0vY+IXyL1Z05y7l+TN/YThYUnBJnfZWwmrKTWWpjskXoODzpLa+JVWMiXc1hdbrYpvAqbgUza1Y7Xx7z/GKpZn7BoghAPLj4tI+JYRKml9DzXr0Tsud4MVDcAu6/xkODaOlG4gsjqqNIQmIZ1jpynRiygFz6ivSgwhsz3gEdpqHnppykroCJLjBrnvNW1VxrKpUcZmYjuRieDzC2u2BsiawGXAOQGx+0gijZXctZNYSzo1yRxgf7r9WlPoL+HbjOVH/4gqmwyId9Kv4UTQ1TKXstsypnVMtfZhLgQt/sEOmieoMIbIYZ8S/8TUi3vMtGWSfMRHZCDDiWgqAKRNLr4ARgk3RopIBzBezPdURqTyi6LpJS7n+zkKpFmpCJMR2B59APTNbYFCVbloXzUh1HW18dVly8H9v49uFecc4Bef0I2IuggJyZx6jAe6Mo1ekThMs8ibdtPkX8v01gZJzBW9CEGQGs= ceph-1456a810-7643-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T04:55:16.127 INFO:teuthology.orchestra.run.smithi043.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCjTG2qRtuHcxvKcvb7xgsggkW7AmLgD/NygkD0vY+IXyL1Z05y7l+TN/YThYUnBJnfZWwmrKTWWpjskXoODzpLa+JVWMiXc1hdbrYpvAqbgUza1Y7Xx7z/GKpZn7BoghAPLj4tI+JYRKml9DzXr0Tsud4MVDcAu6/xkODaOlG4gsjqqNIQmIZ1jpynRiygFz6ivSgwhsz3gEdpqHnppykroCJLjBrnvNW1VxrKpUcZmYjuRieDzC2u2BsiawGXAOQGx+0gijZXctZNYSzo1yRxgf7r9WlPoL+HbjOVH/4gqmwyId9Kv4UTQ1TKXstsypnVMtfZhLgQt/sEOmieoMIbIYZ8S/8TUi3vMtGWSfMRHZCDDiWgqAKRNLr4ARgk3RopIBzBezPdURqTyi6LpJS7n+zkKpFmpCJMR2B59APTNbYFCVbloXzUh1HW18dVly8H9v49uFecc4Bef0I2IuggJyZx6jAe6Mo1ekThMs8ibdtPkX8v01gZJzBW9CEGQGs= ceph-1456a810-7643-11ef-bceb-c7b262605968 2024-09-19T04:55:16.152 DEBUG:teuthology.orchestra.run.smithi096:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCjTG2qRtuHcxvKcvb7xgsggkW7AmLgD/NygkD0vY+IXyL1Z05y7l+TN/YThYUnBJnfZWwmrKTWWpjskXoODzpLa+JVWMiXc1hdbrYpvAqbgUza1Y7Xx7z/GKpZn7BoghAPLj4tI+JYRKml9DzXr0Tsud4MVDcAu6/xkODaOlG4gsjqqNIQmIZ1jpynRiygFz6ivSgwhsz3gEdpqHnppykroCJLjBrnvNW1VxrKpUcZmYjuRieDzC2u2BsiawGXAOQGx+0gijZXctZNYSzo1yRxgf7r9WlPoL+HbjOVH/4gqmwyId9Kv4UTQ1TKXstsypnVMtfZhLgQt/sEOmieoMIbIYZ8S/8TUi3vMtGWSfMRHZCDDiWgqAKRNLr4ARgk3RopIBzBezPdURqTyi6LpJS7n+zkKpFmpCJMR2B59APTNbYFCVbloXzUh1HW18dVly8H9v49uFecc4Bef0I2IuggJyZx6jAe6Mo1ekThMs8ibdtPkX8v01gZJzBW9CEGQGs= ceph-1456a810-7643-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T04:55:16.193 INFO:teuthology.orchestra.run.smithi096.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCjTG2qRtuHcxvKcvb7xgsggkW7AmLgD/NygkD0vY+IXyL1Z05y7l+TN/YThYUnBJnfZWwmrKTWWpjskXoODzpLa+JVWMiXc1hdbrYpvAqbgUza1Y7Xx7z/GKpZn7BoghAPLj4tI+JYRKml9DzXr0Tsud4MVDcAu6/xkODaOlG4gsjqqNIQmIZ1jpynRiygFz6ivSgwhsz3gEdpqHnppykroCJLjBrnvNW1VxrKpUcZmYjuRieDzC2u2BsiawGXAOQGx+0gijZXctZNYSzo1yRxgf7r9WlPoL+HbjOVH/4gqmwyId9Kv4UTQ1TKXstsypnVMtfZhLgQt/sEOmieoMIbIYZ8S/8TUi3vMtGWSfMRHZCDDiWgqAKRNLr4ARgk3RopIBzBezPdURqTyi6LpJS7n+zkKpFmpCJMR2B59APTNbYFCVbloXzUh1HW18dVly8H9v49uFecc4Bef0I2IuggJyZx6jAe6Mo1ekThMs8ibdtPkX8v01gZJzBW9CEGQGs= ceph-1456a810-7643-11ef-bceb-c7b262605968 2024-09-19T04:55:16.206 DEBUG:teuthology.orchestra.run.smithi151:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCjTG2qRtuHcxvKcvb7xgsggkW7AmLgD/NygkD0vY+IXyL1Z05y7l+TN/YThYUnBJnfZWwmrKTWWpjskXoODzpLa+JVWMiXc1hdbrYpvAqbgUza1Y7Xx7z/GKpZn7BoghAPLj4tI+JYRKml9DzXr0Tsud4MVDcAu6/xkODaOlG4gsjqqNIQmIZ1jpynRiygFz6ivSgwhsz3gEdpqHnppykroCJLjBrnvNW1VxrKpUcZmYjuRieDzC2u2BsiawGXAOQGx+0gijZXctZNYSzo1yRxgf7r9WlPoL+HbjOVH/4gqmwyId9Kv4UTQ1TKXstsypnVMtfZhLgQt/sEOmieoMIbIYZ8S/8TUi3vMtGWSfMRHZCDDiWgqAKRNLr4ARgk3RopIBzBezPdURqTyi6LpJS7n+zkKpFmpCJMR2B59APTNbYFCVbloXzUh1HW18dVly8H9v49uFecc4Bef0I2IuggJyZx6jAe6Mo1ekThMs8ibdtPkX8v01gZJzBW9CEGQGs= ceph-1456a810-7643-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T04:55:16.247 INFO:teuthology.orchestra.run.smithi151.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCjTG2qRtuHcxvKcvb7xgsggkW7AmLgD/NygkD0vY+IXyL1Z05y7l+TN/YThYUnBJnfZWwmrKTWWpjskXoODzpLa+JVWMiXc1hdbrYpvAqbgUza1Y7Xx7z/GKpZn7BoghAPLj4tI+JYRKml9DzXr0Tsud4MVDcAu6/xkODaOlG4gsjqqNIQmIZ1jpynRiygFz6ivSgwhsz3gEdpqHnppykroCJLjBrnvNW1VxrKpUcZmYjuRieDzC2u2BsiawGXAOQGx+0gijZXctZNYSzo1yRxgf7r9WlPoL+HbjOVH/4gqmwyId9Kv4UTQ1TKXstsypnVMtfZhLgQt/sEOmieoMIbIYZ8S/8TUi3vMtGWSfMRHZCDDiWgqAKRNLr4ARgk3RopIBzBezPdURqTyi6LpJS7n+zkKpFmpCJMR2B59APTNbYFCVbloXzUh1HW18dVly8H9v49uFecc4Bef0I2IuggJyZx6jAe6Mo1ekThMs8ibdtPkX8v01gZJzBW9CEGQGs= ceph-1456a810-7643-11ef-bceb-c7b262605968 2024-09-19T04:55:16.260 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-19T04:55:16.265 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:16 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/316425054' entity='client.admin' 2024-09-19T04:55:16.633 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:55:17.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:17.201+0000 7f3d49a00640 1 -- 172.21.15.43:0/258016256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d44101420 msgr2=0x7f3d44101820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:17.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:17.201+0000 7f3d49a00640 1 --2- 172.21.15.43:0/258016256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d44101420 0x7f3d44101820 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3d34009880 tx=0x7f3d3402a5d0 comp rx=0 tx=0).stop 2024-09-19T04:55:17.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:17.201+0000 7f3d49a00640 1 -- 172.21.15.43:0/258016256 shutdown_connections 2024-09-19T04:55:17.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:17.201+0000 7f3d49a00640 1 --2- 172.21.15.43:0/258016256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d44101420 0x7f3d44101820 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:17.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:17.201+0000 7f3d49a00640 1 -- 172.21.15.43:0/258016256 >> 172.21.15.43:0/258016256 conn(0x7f3d440fcbd0 msgr2=0x7f3d440feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:17.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:17.201+0000 7f3d49a00640 1 -- 172.21.15.43:0/258016256 shutdown_connections 2024-09-19T04:55:17.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:17.201+0000 7f3d49a00640 1 -- 172.21.15.43:0/258016256 wait complete. 2024-09-19T04:55:17.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:17.202+0000 7f3d49a00640 1 Processor -- start 2024-09-19T04:55:17.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:17.202+0000 7f3d49a00640 1 -- start start 2024-09-19T04:55:17.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:17.203+0000 7f3d49a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d44101420 0x7f3d44075eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:17.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:17.203+0000 7f3d49a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3d44107530 con 0x7f3d44101420 2024-09-19T04:55:17.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:17.203+0000 7f3d43e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d44101420 0x7f3d44075eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:17.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:17.203+0000 7f3d43e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d44101420 0x7f3d44075eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57246/0 (socket says 172.21.15.43:57246) 2024-09-19T04:55:17.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:17.203+0000 7f3d43e00640 1 -- 172.21.15.43:0/3057086831 learned_addr learned my addr 172.21.15.43:0/3057086831 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:55:17.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:17.203+0000 7f3d43e00640 1 -- 172.21.15.43:0/3057086831 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d440763f0 con 0x7f3d44101420 2024-09-19T04:55:17.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:17.204+0000 7f3d43e00640 1 --2- 172.21.15.43:0/3057086831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d44101420 0x7f3d44075eb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3d34004f60 tx=0x7f3d340048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:17.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:17.204+0000 7f3d40c00640 1 -- 172.21.15.43:0/3057086831 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d3403f070 con 0x7f3d44101420 2024-09-19T04:55:17.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:17.204+0000 7f3d40c00640 1 -- 172.21.15.43:0/3057086831 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d34004cc0 con 0x7f3d44101420 2024-09-19T04:55:17.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:17.204+0000 7f3d49a00640 1 -- 172.21.15.43:0/3057086831 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d440745e0 con 0x7f3d44101420 2024-09-19T04:55:17.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:17.204+0000 7f3d40c00640 1 -- 172.21.15.43:0/3057086831 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d34035030 con 0x7f3d44101420 2024-09-19T04:55:17.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:17.204+0000 7f3d49a00640 1 -- 172.21.15.43:0/3057086831 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d44074b20 con 0x7f3d44101420 2024-09-19T04:55:17.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:17.205+0000 7f3d49a00640 1 -- 172.21.15.43:0/3057086831 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d44107cb0 con 0x7f3d44101420 2024-09-19T04:55:17.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:17.206+0000 7f3d40c00640 1 -- 172.21.15.43:0/3057086831 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f3d34042070 con 0x7f3d44101420 2024-09-19T04:55:17.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:17.206+0000 7f3d40c00640 1 --2- 172.21.15.43:0/3057086831 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f3d28039300 0x7f3d2803b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:17.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:17.206+0000 7f3d40c00640 1 -- 172.21.15.43:0/3057086831 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3d3406d0c0 con 0x7f3d44101420 2024-09-19T04:55:17.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:17.207+0000 7f3d43400640 1 --2- 172.21.15.43:0/3057086831 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f3d28039300 0x7f3d2803b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:17.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:17.207+0000 7f3d43400640 1 --2- 172.21.15.43:0/3057086831 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f3d28039300 0x7f3d2803b7c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3d300098c0 tx=0x7f3d30006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:17.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:17.212+0000 7f3d40c00640 1 -- 172.21.15.43:0/3057086831 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3d34040260 con 0x7f3d44101420 2024-09-19T04:55:17.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:17.404+0000 7f3d49a00640 1 -- 172.21.15.43:0/3057086831 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f3d4418ffd0 con 0x7f3d44101420 2024-09-19T04:55:17.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:17.407+0000 7f3d40c00640 1 -- 172.21.15.43:0/3057086831 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f3d34003dd0 con 0x7f3d44101420 2024-09-19T04:55:17.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:17.409+0000 7f3d49a00640 1 -- 172.21.15.43:0/3057086831 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f3d28039300 msgr2=0x7f3d2803b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:17.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:17.409+0000 7f3d49a00640 1 --2- 172.21.15.43:0/3057086831 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f3d28039300 0x7f3d2803b7c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3d300098c0 tx=0x7f3d30006db0 comp rx=0 tx=0).stop 2024-09-19T04:55:17.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:17.409+0000 7f3d49a00640 1 -- 172.21.15.43:0/3057086831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d44101420 msgr2=0x7f3d44075eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:17.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:17.409+0000 7f3d49a00640 1 --2- 172.21.15.43:0/3057086831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d44101420 0x7f3d44075eb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3d34004f60 tx=0x7f3d340048f0 comp rx=0 tx=0).stop 2024-09-19T04:55:17.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:17.410+0000 7f3d49a00640 1 -- 172.21.15.43:0/3057086831 shutdown_connections 2024-09-19T04:55:17.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:17.410+0000 7f3d49a00640 1 --2- 172.21.15.43:0/3057086831 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f3d28039300 0x7f3d2803b7c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:17.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:17.410+0000 7f3d49a00640 1 --2- 172.21.15.43:0/3057086831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d44101420 0x7f3d44075eb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:17.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:17.410+0000 7f3d49a00640 1 -- 172.21.15.43:0/3057086831 >> 172.21.15.43:0/3057086831 conn(0x7f3d440fcbd0 msgr2=0x7f3d440fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:17.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:17.410+0000 7f3d49a00640 1 -- 172.21.15.43:0/3057086831 shutdown_connections 2024-09-19T04:55:17.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:17.410+0000 7f3d49a00640 1 -- 172.21.15.43:0/3057086831 wait complete. 2024-09-19T04:55:18.170 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-19T04:55:18.170 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-19T04:55:18.770 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:55:18.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:18 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/3057086831' entity='client.admin' 2024-09-19T04:55:19.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:19.288+0000 7ff38ba00640 1 -- 172.21.15.43:0/3099109268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff384076220 msgr2=0x7ff384074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:19.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:19.288+0000 7ff38ba00640 1 --2- 172.21.15.43:0/3099109268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff384076220 0x7ff384074680 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff378009880 tx=0x7ff37802a5d0 comp rx=0 tx=0).stop 2024-09-19T04:55:19.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:19.288+0000 7ff38ba00640 1 -- 172.21.15.43:0/3099109268 shutdown_connections 2024-09-19T04:55:19.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:19.288+0000 7ff38ba00640 1 --2- 172.21.15.43:0/3099109268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff384076220 0x7ff384074680 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:19.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:19.288+0000 7ff38ba00640 1 -- 172.21.15.43:0/3099109268 >> 172.21.15.43:0/3099109268 conn(0x7ff3840fcd80 msgr2=0x7ff3840ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:19.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:19.288+0000 7ff38ba00640 1 -- 172.21.15.43:0/3099109268 shutdown_connections 2024-09-19T04:55:19.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:19.288+0000 7ff38ba00640 1 -- 172.21.15.43:0/3099109268 wait complete. 2024-09-19T04:55:19.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:19.289+0000 7ff38ba00640 1 Processor -- start 2024-09-19T04:55:19.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:19.290+0000 7ff38ba00640 1 -- start start 2024-09-19T04:55:19.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:19.290+0000 7ff38ba00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff384076220 0x7ff38419d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:19.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:19.290+0000 7ff38ba00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff38410dd50 con 0x7ff384076220 2024-09-19T04:55:19.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:19.290+0000 7ff389000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff384076220 0x7ff38419d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:19.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:19.290+0000 7ff389000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff384076220 0x7ff38419d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57260/0 (socket says 172.21.15.43:57260) 2024-09-19T04:55:19.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:19.290+0000 7ff389000640 1 -- 172.21.15.43:0/4074828140 learned_addr learned my addr 172.21.15.43:0/4074828140 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:55:19.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:19.290+0000 7ff389000640 1 -- 172.21.15.43:0/4074828140 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff38419d8f0 con 0x7ff384076220 2024-09-19T04:55:19.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:19.291+0000 7ff389000640 1 --2- 172.21.15.43:0/4074828140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff384076220 0x7ff38419d3b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff378004f60 tx=0x7ff3780048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:19.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:19.291+0000 7ff381600640 1 -- 172.21.15.43:0/4074828140 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff37803f070 con 0x7ff384076220 2024-09-19T04:55:19.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:19.291+0000 7ff381600640 1 -- 172.21.15.43:0/4074828140 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff378004cc0 con 0x7ff384076220 2024-09-19T04:55:19.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:19.291+0000 7ff38ba00640 1 -- 172.21.15.43:0/4074828140 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff38419db80 con 0x7ff384076220 2024-09-19T04:55:19.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:19.291+0000 7ff381600640 1 -- 172.21.15.43:0/4074828140 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff378035030 con 0x7ff384076220 2024-09-19T04:55:19.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:19.291+0000 7ff38ba00640 1 -- 172.21.15.43:0/4074828140 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3841a0870 con 0x7ff384076220 2024-09-19T04:55:19.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:19.292+0000 7ff381600640 1 -- 172.21.15.43:0/4074828140 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7ff378042070 con 0x7ff384076220 2024-09-19T04:55:19.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:19.292+0000 7ff38ba00640 1 -- 172.21.15.43:0/4074828140 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff38410e490 con 0x7ff384076220 2024-09-19T04:55:19.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:19.293+0000 7ff381600640 1 --2- 172.21.15.43:0/4074828140 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff36c039300 0x7ff36c03b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:19.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:19.293+0000 7ff381600640 1 -- 172.21.15.43:0/4074828140 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff37806d0f0 con 0x7ff384076220 2024-09-19T04:55:19.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:19.293+0000 7ff383e00640 1 --2- 172.21.15.43:0/4074828140 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff36c039300 0x7ff36c03b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:19.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:19.294+0000 7ff383e00640 1 --2- 172.21.15.43:0/4074828140 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff36c039300 0x7ff36c03b7c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff3700098c0 tx=0x7ff370006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:19.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:19.298+0000 7ff381600640 1 -- 172.21.15.43:0/4074828140 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff37803a3c0 con 0x7ff384076220 2024-09-19T04:55:19.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:19.488+0000 7ff38ba00640 1 -- 172.21.15.43:0/4074828140 --> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7ff38402d830 con 0x7ff36c039300 2024-09-19T04:55:19.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:19.492+0000 7ff381600640 1 -- 172.21.15.43:0/4074828140 <== mgr.14150 v2:172.21.15.43:6800/3167108925 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff38402d830 con 0x7ff36c039300 2024-09-19T04:55:19.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:19.494+0000 7ff38ba00640 1 -- 172.21.15.43:0/4074828140 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff36c039300 msgr2=0x7ff36c03b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:19.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:19.494+0000 7ff38ba00640 1 --2- 172.21.15.43:0/4074828140 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff36c039300 0x7ff36c03b7c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff3700098c0 tx=0x7ff370006db0 comp rx=0 tx=0).stop 2024-09-19T04:55:19.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:19.494+0000 7ff38ba00640 1 -- 172.21.15.43:0/4074828140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff384076220 msgr2=0x7ff38419d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:19.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:19.494+0000 7ff38ba00640 1 --2- 172.21.15.43:0/4074828140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff384076220 0x7ff38419d3b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff378004f60 tx=0x7ff3780048f0 comp rx=0 tx=0).stop 2024-09-19T04:55:19.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:19.495+0000 7ff38ba00640 1 -- 172.21.15.43:0/4074828140 shutdown_connections 2024-09-19T04:55:19.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:19.495+0000 7ff38ba00640 1 --2- 172.21.15.43:0/4074828140 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff36c039300 0x7ff36c03b7c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:19.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:19.495+0000 7ff38ba00640 1 --2- 172.21.15.43:0/4074828140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff384076220 0x7ff38419d3b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:19.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:19.495+0000 7ff38ba00640 1 -- 172.21.15.43:0/4074828140 >> 172.21.15.43:0/4074828140 conn(0x7ff3840fcd80 msgr2=0x7ff3840fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:19.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:19.495+0000 7ff38ba00640 1 -- 172.21.15.43:0/4074828140 shutdown_connections 2024-09-19T04:55:19.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:19.495+0000 7ff38ba00640 1 -- 172.21.15.43:0/4074828140 wait complete. 2024-09-19T04:55:19.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:19 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:19.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:19 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:19.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:19 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi043", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:55:19.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:19 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:19.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:19 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:55:19.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:19 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:19.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:19 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:19.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:19 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:55:19.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:19 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:55:19.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:19 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:55:20.251 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi096 2024-09-19T04:55:20.251 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-19T04:55:20.251 DEBUG:teuthology.orchestra.run.smithi096:> dd of=/etc/ceph/ceph.conf 2024-09-19T04:55:20.270 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-19T04:55:20.270 DEBUG:teuthology.orchestra.run.smithi096:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:55:20.327 INFO:tasks.cephadm:Adding host smithi096 to orchestrator... 2024-09-19T04:55:20.328 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph orch host add smithi096 2024-09-19T04:55:20.709 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:55:20.982 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:20 smithi043 ceph-mon[30800]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:55:20.982 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:20 smithi043 ceph-mon[30800]: Updating smithi043:/etc/ceph/ceph.conf 2024-09-19T04:55:20.982 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:20 smithi043 ceph-mon[30800]: Updating smithi043:/var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:55:21.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:21.281+0000 7f48d4c00640 1 -- 172.21.15.43:0/429630124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48d0101600 msgr2=0x7f48d01019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:21.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:21.281+0000 7f48d4c00640 1 --2- 172.21.15.43:0/429630124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48d0101600 0x7f48d01019e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f48c0009880 tx=0x7f48c002a5d0 comp rx=0 tx=0).stop 2024-09-19T04:55:21.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:21.281+0000 7f48d4c00640 1 -- 172.21.15.43:0/429630124 shutdown_connections 2024-09-19T04:55:21.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:21.281+0000 7f48d4c00640 1 --2- 172.21.15.43:0/429630124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48d0101600 0x7f48d01019e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:21.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:21.282+0000 7f48d4c00640 1 -- 172.21.15.43:0/429630124 >> 172.21.15.43:0/429630124 conn(0x7f48d00fd0a0 msgr2=0x7f48d00ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:21.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:21.282+0000 7f48d4c00640 1 -- 172.21.15.43:0/429630124 shutdown_connections 2024-09-19T04:55:21.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:21.282+0000 7f48d4c00640 1 -- 172.21.15.43:0/429630124 wait complete. 2024-09-19T04:55:21.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:21.283+0000 7f48d4c00640 1 Processor -- start 2024-09-19T04:55:21.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:21.283+0000 7f48d4c00640 1 -- start start 2024-09-19T04:55:21.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:21.284+0000 7f48d4c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48d0101600 0x7f48d0196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:21.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:21.284+0000 7f48d4c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f48d010ba50 con 0x7f48d0101600 2024-09-19T04:55:21.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:21.284+0000 7f48cf400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48d0101600 0x7f48d0196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:21.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:21.284+0000 7f48cf400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48d0101600 0x7f48d0196f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57276/0 (socket says 172.21.15.43:57276) 2024-09-19T04:55:21.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:21.284+0000 7f48cf400640 1 -- 172.21.15.43:0/3895076589 learned_addr learned my addr 172.21.15.43:0/3895076589 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:55:21.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:21.284+0000 7f48cf400640 1 -- 172.21.15.43:0/3895076589 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48d01974c0 con 0x7f48d0101600 2024-09-19T04:55:21.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:21.285+0000 7f48cf400640 1 --2- 172.21.15.43:0/3895076589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48d0101600 0x7f48d0196f80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f48c0009230 tx=0x7f48c0003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:21.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:21.285+0000 7f48c7e00640 1 -- 172.21.15.43:0/3895076589 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48c003f070 con 0x7f48d0101600 2024-09-19T04:55:21.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:21.285+0000 7f48c7e00640 1 -- 172.21.15.43:0/3895076589 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48c003bd40 con 0x7f48d0101600 2024-09-19T04:55:21.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:21.285+0000 7f48c7e00640 1 -- 172.21.15.43:0/3895076589 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48c0035030 con 0x7f48d0101600 2024-09-19T04:55:21.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:21.285+0000 7f48d4c00640 1 -- 172.21.15.43:0/3895076589 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48d01977b0 con 0x7f48d0101600 2024-09-19T04:55:21.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:21.285+0000 7f48d4c00640 1 -- 172.21.15.43:0/3895076589 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48d019b850 con 0x7f48d0101600 2024-09-19T04:55:21.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:21.286+0000 7f48c7e00640 1 -- 172.21.15.43:0/3895076589 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f48c0042070 con 0x7f48d0101600 2024-09-19T04:55:21.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:21.286+0000 7f48d4c00640 1 -- 172.21.15.43:0/3895076589 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48d0102ae0 con 0x7f48d0101600 2024-09-19T04:55:21.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:21.287+0000 7f48c7e00640 1 --2- 172.21.15.43:0/3895076589 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f48b4039300 0x7f48b403b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:21.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:21.287+0000 7f48c7e00640 1 -- 172.21.15.43:0/3895076589 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f48c006cc10 con 0x7f48d0101600 2024-09-19T04:55:21.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:21.287+0000 7f48cea00640 1 --2- 172.21.15.43:0/3895076589 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f48b4039300 0x7f48b403b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:21.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:21.288+0000 7f48cea00640 1 --2- 172.21.15.43:0/3895076589 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f48b4039300 0x7f48b403b7c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f48bc0098c0 tx=0x7f48bc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:21.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:21.292+0000 7f48c7e00640 1 -- 172.21.15.43:0/3895076589 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f48c003ac70 con 0x7f48d0101600 2024-09-19T04:55:21.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:21.486+0000 7f48d4c00640 1 -- 172.21.15.43:0/3895076589 --> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi096", "target": ["mon-mgr", ""]}) -- 0x7f48d01982e0 con 0x7f48b4039300 2024-09-19T04:55:22.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:21 smithi043 ceph-mon[30800]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:55:22.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:21 smithi043 ceph-mon[30800]: Updating smithi043:/var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:55:22.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:21 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:22.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:21 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:22.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:21 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:22.931 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:22 smithi043 ceph-mon[30800]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi096", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:55:23.959 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:23 smithi043 ceph-mon[30800]: Deploying cephadm binary to smithi096 2024-09-19T04:55:25.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:25.372+0000 7f48c7e00640 1 -- 172.21.15.43:0/3895076589 <== mgr.14150 v2:172.21.15.43:6800/3167108925 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f48d01982e0 con 0x7f48b4039300 2024-09-19T04:55:25.374 INFO:teuthology.orchestra.run.smithi043.stdout:Added host 'smithi096' with addr '172.21.15.96' 2024-09-19T04:55:25.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:25.375+0000 7f48d4c00640 1 -- 172.21.15.43:0/3895076589 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f48b4039300 msgr2=0x7f48b403b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:25.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:25.375+0000 7f48d4c00640 1 --2- 172.21.15.43:0/3895076589 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f48b4039300 0x7f48b403b7c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f48bc0098c0 tx=0x7f48bc006db0 comp rx=0 tx=0).stop 2024-09-19T04:55:25.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:25.375+0000 7f48d4c00640 1 -- 172.21.15.43:0/3895076589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48d0101600 msgr2=0x7f48d0196f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:25.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:25.375+0000 7f48d4c00640 1 --2- 172.21.15.43:0/3895076589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48d0101600 0x7f48d0196f80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f48c0009230 tx=0x7f48c0003d80 comp rx=0 tx=0).stop 2024-09-19T04:55:25.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:25.375+0000 7f48d4c00640 1 -- 172.21.15.43:0/3895076589 shutdown_connections 2024-09-19T04:55:25.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:25.375+0000 7f48d4c00640 1 --2- 172.21.15.43:0/3895076589 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f48b4039300 0x7f48b403b7c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:25.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:25.375+0000 7f48d4c00640 1 --2- 172.21.15.43:0/3895076589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48d0101600 0x7f48d0196f80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:25.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:25.375+0000 7f48d4c00640 1 -- 172.21.15.43:0/3895076589 >> 172.21.15.43:0/3895076589 conn(0x7f48d00fd0a0 msgr2=0x7f48d0108fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:25.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:25.375+0000 7f48d4c00640 1 -- 172.21.15.43:0/3895076589 shutdown_connections 2024-09-19T04:55:25.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:25.375+0000 7f48d4c00640 1 -- 172.21.15.43:0/3895076589 wait complete. 2024-09-19T04:55:26.073 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-19T04:55:26.438 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:55:26.727 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:26 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:26.727 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:26 smithi043 ceph-mon[30800]: Added host smithi096 2024-09-19T04:55:26.727 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:26 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:55:26.727 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:26 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:26.728 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:26 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:26.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:26.968+0000 7ff119a00640 1 -- 172.21.15.43:0/3108205460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff114107660 msgr2=0x7ff114107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:26.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:26.968+0000 7ff119a00640 1 --2- 172.21.15.43:0/3108205460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff114107660 0x7ff114107a40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff104009c20 tx=0x7ff10402aa90 comp rx=0 tx=0).stop 2024-09-19T04:55:26.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:26.968+0000 7ff119a00640 1 -- 172.21.15.43:0/3108205460 shutdown_connections 2024-09-19T04:55:26.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:26.968+0000 7ff119a00640 1 --2- 172.21.15.43:0/3108205460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff114107660 0x7ff114107a40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:26.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:26.968+0000 7ff119a00640 1 -- 172.21.15.43:0/3108205460 >> 172.21.15.43:0/3108205460 conn(0x7ff1140fd290 msgr2=0x7ff1140ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:26.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:26.968+0000 7ff119a00640 1 -- 172.21.15.43:0/3108205460 shutdown_connections 2024-09-19T04:55:26.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:26.968+0000 7ff119a00640 1 -- 172.21.15.43:0/3108205460 wait complete. 2024-09-19T04:55:26.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:26.969+0000 7ff119a00640 1 Processor -- start 2024-09-19T04:55:26.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:26.970+0000 7ff119a00640 1 -- start start 2024-09-19T04:55:26.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:26.970+0000 7ff119a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff114107660 0x7ff114199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:26.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:26.970+0000 7ff119a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff11410bc90 con 0x7ff114107660 2024-09-19T04:55:26.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:26.970+0000 7ff112a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff114107660 0x7ff114199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:26.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:26.970+0000 7ff112a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff114107660 0x7ff114199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44344/0 (socket says 172.21.15.43:44344) 2024-09-19T04:55:26.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:26.970+0000 7ff112a00640 1 -- 172.21.15.43:0/3163644892 learned_addr learned my addr 172.21.15.43:0/3163644892 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:55:26.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:26.971+0000 7ff112a00640 1 -- 172.21.15.43:0/3163644892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff114199860 con 0x7ff114107660 2024-09-19T04:55:26.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:26.971+0000 7ff112a00640 1 --2- 172.21.15.43:0/3163644892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff114107660 0x7ff114199320 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff104009d40 tx=0x7ff10402fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:26.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:26.971+0000 7ff10be00640 1 -- 172.21.15.43:0/3163644892 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff104047070 con 0x7ff114107660 2024-09-19T04:55:26.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:26.971+0000 7ff10be00640 1 -- 172.21.15.43:0/3163644892 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff104003e40 con 0x7ff114107660 2024-09-19T04:55:26.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:26.971+0000 7ff10be00640 1 -- 172.21.15.43:0/3163644892 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff104033050 con 0x7ff114107660 2024-09-19T04:55:26.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:26.971+0000 7ff119a00640 1 -- 172.21.15.43:0/3163644892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff114199af0 con 0x7ff114107660 2024-09-19T04:55:26.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:26.971+0000 7ff119a00640 1 -- 172.21.15.43:0/3163644892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff114199f70 con 0x7ff114107660 2024-09-19T04:55:26.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:26.972+0000 7ff10be00640 1 -- 172.21.15.43:0/3163644892 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7ff10404a070 con 0x7ff114107660 2024-09-19T04:55:26.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:26.972+0000 7ff119a00640 1 -- 172.21.15.43:0/3163644892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff114102cd0 con 0x7ff114107660 2024-09-19T04:55:26.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:26.973+0000 7ff10be00640 1 --2- 172.21.15.43:0/3163644892 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff0f8039300 0x7ff0f803b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:26.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:26.973+0000 7ff10be00640 1 -- 172.21.15.43:0/3163644892 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff10403e030 con 0x7ff114107660 2024-09-19T04:55:26.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:26.973+0000 7ff10a000640 1 --2- 172.21.15.43:0/3163644892 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff0f8039300 0x7ff0f803b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:26.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:26.974+0000 7ff10a000640 1 --2- 172.21.15.43:0/3163644892 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff0f8039300 0x7ff0f803b7c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc0098c0 tx=0x7ff0fc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:26.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:26.978+0000 7ff10be00640 1 -- 172.21.15.43:0/3163644892 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff104004180 con 0x7ff114107660 2024-09-19T04:55:27.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:27.174+0000 7ff119a00640 1 -- 172.21.15.43:0/3163644892 --> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff11419dbf0 con 0x7ff0f8039300 2024-09-19T04:55:27.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:27.175+0000 7ff10be00640 1 -- 172.21.15.43:0/3163644892 <== mgr.14150 v2:172.21.15.43:6800/3167108925 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7ff11419dbf0 con 0x7ff0f8039300 2024-09-19T04:55:27.176 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T04:55:27.176 INFO:teuthology.orchestra.run.smithi043.stdout:[{"addr": "172.21.15.43", "hostname": "smithi043", "labels": [], "status": ""}, {"addr": "172.21.15.96", "hostname": "smithi096", "labels": [], "status": ""}] 2024-09-19T04:55:27.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:27.177+0000 7ff119a00640 1 -- 172.21.15.43:0/3163644892 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff0f8039300 msgr2=0x7ff0f803b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:27.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:27.177+0000 7ff119a00640 1 --2- 172.21.15.43:0/3163644892 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff0f8039300 0x7ff0f803b7c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc0098c0 tx=0x7ff0fc006db0 comp rx=0 tx=0).stop 2024-09-19T04:55:27.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:27.177+0000 7ff119a00640 1 -- 172.21.15.43:0/3163644892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff114107660 msgr2=0x7ff114199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:27.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:27.177+0000 7ff119a00640 1 --2- 172.21.15.43:0/3163644892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff114107660 0x7ff114199320 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff104009d40 tx=0x7ff10402fc10 comp rx=0 tx=0).stop 2024-09-19T04:55:27.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:27.177+0000 7ff119a00640 1 -- 172.21.15.43:0/3163644892 shutdown_connections 2024-09-19T04:55:27.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:27.177+0000 7ff119a00640 1 --2- 172.21.15.43:0/3163644892 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff0f8039300 0x7ff0f803b7c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:27.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:27.177+0000 7ff119a00640 1 --2- 172.21.15.43:0/3163644892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff114107660 0x7ff114199320 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:27.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:27.177+0000 7ff119a00640 1 -- 172.21.15.43:0/3163644892 >> 172.21.15.43:0/3163644892 conn(0x7ff1140fd290 msgr2=0x7ff114075e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:27.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:27.178+0000 7ff119a00640 1 -- 172.21.15.43:0/3163644892 shutdown_connections 2024-09-19T04:55:27.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:27.178+0000 7ff119a00640 1 -- 172.21.15.43:0/3163644892 wait complete. 2024-09-19T04:55:27.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:27 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:27.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:27 smithi043 ceph-mon[30800]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:55:27.849 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi151 2024-09-19T04:55:27.849 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-19T04:55:27.849 DEBUG:teuthology.orchestra.run.smithi151:> dd of=/etc/ceph/ceph.conf 2024-09-19T04:55:27.868 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-19T04:55:27.868 DEBUG:teuthology.orchestra.run.smithi151:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:55:27.927 INFO:tasks.cephadm:Adding host smithi151 to orchestrator... 2024-09-19T04:55:27.928 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph orch host add smithi151 2024-09-19T04:55:28.293 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:55:28.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:28.859+0000 7fa987e00640 1 -- 172.21.15.43:0/181548357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa988109740 msgr2=0x7fa988109b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:28.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:28.859+0000 7fa987e00640 1 --2- 172.21.15.43:0/181548357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa988109740 0x7fa988109b20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa978009880 tx=0x7fa97802a5d0 comp rx=0 tx=0).stop 2024-09-19T04:55:28.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:28.859+0000 7fa987e00640 1 -- 172.21.15.43:0/181548357 shutdown_connections 2024-09-19T04:55:28.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:28.859+0000 7fa987e00640 1 --2- 172.21.15.43:0/181548357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa988109740 0x7fa988109b20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:28.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:28.859+0000 7fa987e00640 1 -- 172.21.15.43:0/181548357 >> 172.21.15.43:0/181548357 conn(0x7fa9880ff2c0 msgr2=0x7fa9881016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:28.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:28.859+0000 7fa987e00640 1 -- 172.21.15.43:0/181548357 shutdown_connections 2024-09-19T04:55:28.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:28.859+0000 7fa987e00640 1 -- 172.21.15.43:0/181548357 wait complete. 2024-09-19T04:55:28.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:28.860+0000 7fa987e00640 1 Processor -- start 2024-09-19T04:55:28.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:28.860+0000 7fa987e00640 1 -- start start 2024-09-19T04:55:28.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:28.860+0000 7fa987e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa988109740 0x7fa9881991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:28.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:28.860+0000 7fa987e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa98810dd50 con 0x7fa988109740 2024-09-19T04:55:28.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:28.861+0000 7fa986a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa988109740 0x7fa9881991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:28.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:28.861+0000 7fa986a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa988109740 0x7fa9881991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44352/0 (socket says 172.21.15.43:44352) 2024-09-19T04:55:28.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:28.861+0000 7fa986a00640 1 -- 172.21.15.43:0/2484352830 learned_addr learned my addr 172.21.15.43:0/2484352830 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:55:28.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:28.861+0000 7fa986a00640 1 -- 172.21.15.43:0/2484352830 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa988199730 con 0x7fa988109740 2024-09-19T04:55:28.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:28.861+0000 7fa986a00640 1 --2- 172.21.15.43:0/2484352830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa988109740 0x7fa9881991f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa978009230 tx=0x7fa978003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:28.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:28.861+0000 7fa98ca00640 1 -- 172.21.15.43:0/2484352830 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa97803f070 con 0x7fa988109740 2024-09-19T04:55:28.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:28.862+0000 7fa98ca00640 1 -- 172.21.15.43:0/2484352830 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa97803bc80 con 0x7fa988109740 2024-09-19T04:55:28.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:28.862+0000 7fa987e00640 1 -- 172.21.15.43:0/2484352830 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa988199a20 con 0x7fa988109740 2024-09-19T04:55:28.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:28.862+0000 7fa98ca00640 1 -- 172.21.15.43:0/2484352830 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa978035030 con 0x7fa988109740 2024-09-19T04:55:28.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:28.862+0000 7fa987e00640 1 -- 172.21.15.43:0/2484352830 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa98819dac0 con 0x7fa988109740 2024-09-19T04:55:28.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:28.863+0000 7fa98ca00640 1 -- 172.21.15.43:0/2484352830 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fa978042070 con 0x7fa988109740 2024-09-19T04:55:28.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:28.863+0000 7fa987e00640 1 -- 172.21.15.43:0/2484352830 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa988104ce0 con 0x7fa988109740 2024-09-19T04:55:28.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:28.863+0000 7fa98ca00640 1 --2- 172.21.15.43:0/2484352830 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa96c039300 0x7fa96c03b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:28.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:28.863+0000 7fa98ca00640 1 -- 172.21.15.43:0/2484352830 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa97806cff0 con 0x7fa988109740 2024-09-19T04:55:28.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:28.864+0000 7fa97fe00640 1 --2- 172.21.15.43:0/2484352830 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa96c039300 0x7fa96c03b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:28.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:28.864+0000 7fa97fe00640 1 --2- 172.21.15.43:0/2484352830 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa96c039300 0x7fa96c03b7c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa9700098c0 tx=0x7fa970006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:28.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:28.868+0000 7fa98ca00640 1 -- 172.21.15.43:0/2484352830 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa9780372d0 con 0x7fa988109740 2024-09-19T04:55:29.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:28 smithi043 ceph-mon[30800]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:55:29.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:29.059+0000 7fa987e00640 1 -- 172.21.15.43:0/2484352830 --> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi151", "target": ["mon-mgr", ""]}) -- 0x7fa988109bd0 con 0x7fa96c039300 2024-09-19T04:55:30.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:29 smithi043 ceph-mon[30800]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:55:30.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:29 smithi043 ceph-mon[30800]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi151", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:55:30.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:29 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:30.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:29 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:30.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:29 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:30.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:29 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:30.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:29 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi096", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:55:30.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:29 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:55:30.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:29 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:55:31.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:30 smithi043 ceph-mon[30800]: Updating smithi096:/etc/ceph/ceph.conf 2024-09-19T04:55:31.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:30 smithi043 ceph-mon[30800]: Updating smithi096:/var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:55:31.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:30 smithi043 ceph-mon[30800]: Updating smithi096:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:55:31.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:30 smithi043 ceph-mon[30800]: Updating smithi096:/var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:55:31.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:30 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:31.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:30 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:31.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:30 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:32.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:31 smithi043 ceph-mon[30800]: Deploying cephadm binary to smithi151 2024-09-19T04:55:32.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:31 smithi043 ceph-mon[30800]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:55:32.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:32.954+0000 7fa98ca00640 1 -- 172.21.15.43:0/2484352830 <== mgr.14150 v2:172.21.15.43:6800/3167108925 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fa988109bd0 con 0x7fa96c039300 2024-09-19T04:55:32.956 INFO:teuthology.orchestra.run.smithi043.stdout:Added host 'smithi151' with addr '172.21.15.151' 2024-09-19T04:55:32.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:32.956+0000 7fa987e00640 1 -- 172.21.15.43:0/2484352830 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa96c039300 msgr2=0x7fa96c03b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:32.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:32.956+0000 7fa987e00640 1 --2- 172.21.15.43:0/2484352830 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa96c039300 0x7fa96c03b7c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa9700098c0 tx=0x7fa970006db0 comp rx=0 tx=0).stop 2024-09-19T04:55:32.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:32.956+0000 7fa987e00640 1 -- 172.21.15.43:0/2484352830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa988109740 msgr2=0x7fa9881991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:32.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:32.956+0000 7fa987e00640 1 --2- 172.21.15.43:0/2484352830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa988109740 0x7fa9881991f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa978009230 tx=0x7fa978003d40 comp rx=0 tx=0).stop 2024-09-19T04:55:32.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:32.956+0000 7fa987e00640 1 -- 172.21.15.43:0/2484352830 shutdown_connections 2024-09-19T04:55:32.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:32.956+0000 7fa987e00640 1 --2- 172.21.15.43:0/2484352830 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa96c039300 0x7fa96c03b7c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:32.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:32.956+0000 7fa987e00640 1 --2- 172.21.15.43:0/2484352830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa988109740 0x7fa9881991f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:32.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:32.956+0000 7fa987e00640 1 -- 172.21.15.43:0/2484352830 >> 172.21.15.43:0/2484352830 conn(0x7fa9880ff2c0 msgr2=0x7fa988100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:32.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:32.957+0000 7fa987e00640 1 -- 172.21.15.43:0/2484352830 shutdown_connections 2024-09-19T04:55:32.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:32.957+0000 7fa987e00640 1 -- 172.21.15.43:0/2484352830 wait complete. 2024-09-19T04:55:33.625 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-19T04:55:33.990 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:55:34.249 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:33 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:34.249 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:33 smithi043 ceph-mon[30800]: Added host smithi151 2024-09-19T04:55:34.249 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:33 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:55:34.249 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:33 smithi043 ceph-mon[30800]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:55:34.249 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:33 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:34.250 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:33 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:34.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:34.503+0000 7f2eab800640 1 -- 172.21.15.43:0/3647813256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ea41097a0 msgr2=0x7f2ea4109b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:34.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:34.503+0000 7f2eab800640 1 --2- 172.21.15.43:0/3647813256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ea41097a0 0x7f2ea4109b80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2e98009880 tx=0x7f2e9802a5d0 comp rx=0 tx=0).stop 2024-09-19T04:55:34.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:34.503+0000 7f2eab800640 1 -- 172.21.15.43:0/3647813256 shutdown_connections 2024-09-19T04:55:34.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:34.503+0000 7f2eab800640 1 --2- 172.21.15.43:0/3647813256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ea41097a0 0x7f2ea4109b80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:34.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:34.503+0000 7f2eab800640 1 -- 172.21.15.43:0/3647813256 >> 172.21.15.43:0/3647813256 conn(0x7f2ea40ff480 msgr2=0x7f2ea41018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:34.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:34.503+0000 7f2eab800640 1 -- 172.21.15.43:0/3647813256 shutdown_connections 2024-09-19T04:55:34.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:34.503+0000 7f2eab800640 1 -- 172.21.15.43:0/3647813256 wait complete. 2024-09-19T04:55:34.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:34.505+0000 7f2eab800640 1 Processor -- start 2024-09-19T04:55:34.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:34.505+0000 7f2eab800640 1 -- start start 2024-09-19T04:55:34.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:34.505+0000 7f2eab800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ea41097a0 0x7f2ea4199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:34.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:34.505+0000 7f2eab800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea410de60 con 0x7f2ea41097a0 2024-09-19T04:55:34.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:34.505+0000 7f2ea8e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ea41097a0 0x7f2ea4199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:34.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:34.505+0000 7f2ea8e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ea41097a0 0x7f2ea4199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44370/0 (socket says 172.21.15.43:44370) 2024-09-19T04:55:34.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:34.506+0000 7f2ea8e00640 1 -- 172.21.15.43:0/1255788178 learned_addr learned my addr 172.21.15.43:0/1255788178 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:55:34.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:34.506+0000 7f2ea8e00640 1 -- 172.21.15.43:0/1255788178 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ea4199890 con 0x7f2ea41097a0 2024-09-19T04:55:34.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:34.506+0000 7f2ea8e00640 1 --2- 172.21.15.43:0/1255788178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ea41097a0 0x7f2ea4199350 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2e98009230 tx=0x7f2e98003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:34.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:34.506+0000 7f2ea1600640 1 -- 172.21.15.43:0/1255788178 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e9803f070 con 0x7f2ea41097a0 2024-09-19T04:55:34.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:34.506+0000 7f2ea1600640 1 -- 172.21.15.43:0/1255788178 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e9803bc80 con 0x7f2ea41097a0 2024-09-19T04:55:34.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:34.507+0000 7f2eab800640 1 -- 172.21.15.43:0/1255788178 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ea4199b80 con 0x7f2ea41097a0 2024-09-19T04:55:34.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:34.507+0000 7f2ea1600640 1 -- 172.21.15.43:0/1255788178 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e98035030 con 0x7f2ea41097a0 2024-09-19T04:55:34.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:34.507+0000 7f2eab800640 1 -- 172.21.15.43:0/1255788178 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ea419dc20 con 0x7f2ea41097a0 2024-09-19T04:55:34.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:34.508+0000 7f2ea1600640 1 -- 172.21.15.43:0/1255788178 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f2e98042070 con 0x7f2ea41097a0 2024-09-19T04:55:34.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:34.508+0000 7f2eab800640 1 -- 172.21.15.43:0/1255788178 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ea4104ee0 con 0x7f2ea41097a0 2024-09-19T04:55:34.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:34.509+0000 7f2ea1600640 1 --2- 172.21.15.43:0/1255788178 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f2e8c039300 0x7f2e8c03b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:34.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:34.509+0000 7f2ea1600640 1 -- 172.21.15.43:0/1255788178 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2e9806cff0 con 0x7f2ea41097a0 2024-09-19T04:55:34.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:34.509+0000 7f2ea3e00640 1 --2- 172.21.15.43:0/1255788178 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f2e8c039300 0x7f2e8c03b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:34.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:34.509+0000 7f2ea3e00640 1 --2- 172.21.15.43:0/1255788178 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f2e8c039300 0x7f2e8c03b7c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2e900098c0 tx=0x7f2e90006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:34.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:34.514+0000 7f2ea1600640 1 -- 172.21.15.43:0/1255788178 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2e980372d0 con 0x7f2ea41097a0 2024-09-19T04:55:34.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:34.702+0000 7f2eab800640 1 -- 172.21.15.43:0/1255788178 --> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2ea419a6b0 con 0x7f2e8c039300 2024-09-19T04:55:34.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:34.703+0000 7f2ea1600640 1 -- 172.21.15.43:0/1255788178 <== mgr.14150 v2:172.21.15.43:6800/3167108925 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f2ea419a6b0 con 0x7f2e8c039300 2024-09-19T04:55:34.705 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T04:55:34.705 INFO:teuthology.orchestra.run.smithi043.stdout:[{"addr": "172.21.15.43", "hostname": "smithi043", "labels": [], "status": ""}, {"addr": "172.21.15.96", "hostname": "smithi096", "labels": [], "status": ""}, {"addr": "172.21.15.151", "hostname": "smithi151", "labels": [], "status": ""}] 2024-09-19T04:55:34.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:34.705+0000 7f2eab800640 1 -- 172.21.15.43:0/1255788178 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f2e8c039300 msgr2=0x7f2e8c03b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:34.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:34.705+0000 7f2eab800640 1 --2- 172.21.15.43:0/1255788178 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f2e8c039300 0x7f2e8c03b7c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2e900098c0 tx=0x7f2e90006db0 comp rx=0 tx=0).stop 2024-09-19T04:55:34.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:34.705+0000 7f2eab800640 1 -- 172.21.15.43:0/1255788178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ea41097a0 msgr2=0x7f2ea4199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:34.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:34.705+0000 7f2eab800640 1 --2- 172.21.15.43:0/1255788178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ea41097a0 0x7f2ea4199350 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2e98009230 tx=0x7f2e98003d40 comp rx=0 tx=0).stop 2024-09-19T04:55:34.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:34.705+0000 7f2eab800640 1 -- 172.21.15.43:0/1255788178 shutdown_connections 2024-09-19T04:55:34.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:34.705+0000 7f2eab800640 1 --2- 172.21.15.43:0/1255788178 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f2e8c039300 0x7f2e8c03b7c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:34.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:34.705+0000 7f2eab800640 1 --2- 172.21.15.43:0/1255788178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ea41097a0 0x7f2ea4199350 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:34.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:34.705+0000 7f2eab800640 1 -- 172.21.15.43:0/1255788178 >> 172.21.15.43:0/1255788178 conn(0x7f2ea40ff480 msgr2=0x7f2ea40ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:34.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:34.705+0000 7f2eab800640 1 -- 172.21.15.43:0/1255788178 shutdown_connections 2024-09-19T04:55:34.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:34.705+0000 7f2eab800640 1 -- 172.21.15.43:0/1255788178 wait complete. 2024-09-19T04:55:35.411 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-19T04:55:35.412 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-19T04:55:35.776 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:55:35.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:35 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:35.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:35 smithi043 ceph-mon[30800]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:55:35.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:35 smithi043 ceph-mon[30800]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:55:36.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:36.279+0000 7f9212e00640 1 -- 172.21.15.43:0/1190960053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f920c102560 msgr2=0x7f920c102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:36.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:36.279+0000 7f9212e00640 1 --2- 172.21.15.43:0/1190960053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f920c102560 0x7f920c102940 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9200009880 tx=0x7f920002a670 comp rx=0 tx=0).stop 2024-09-19T04:55:36.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:36.279+0000 7f9212e00640 1 -- 172.21.15.43:0/1190960053 shutdown_connections 2024-09-19T04:55:36.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:36.279+0000 7f9212e00640 1 --2- 172.21.15.43:0/1190960053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f920c102560 0x7f920c102940 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:36.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:36.279+0000 7f9212e00640 1 -- 172.21.15.43:0/1190960053 >> 172.21.15.43:0/1190960053 conn(0x7f920c077170 msgr2=0x7f920c0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:36.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:36.279+0000 7f9212e00640 1 -- 172.21.15.43:0/1190960053 shutdown_connections 2024-09-19T04:55:36.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:36.279+0000 7f9212e00640 1 -- 172.21.15.43:0/1190960053 wait complete. 2024-09-19T04:55:36.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:36.280+0000 7f9212e00640 1 Processor -- start 2024-09-19T04:55:36.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:36.280+0000 7f9212e00640 1 -- start start 2024-09-19T04:55:36.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:36.280+0000 7f9212e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f920c102560 0x7f920c197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:36.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:36.280+0000 7f9212e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f920c10bad0 con 0x7f920c102560 2024-09-19T04:55:36.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:36.281+0000 7f9211a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f920c102560 0x7f920c197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:36.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:36.281+0000 7f9211a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f920c102560 0x7f920c197040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60280/0 (socket says 172.21.15.43:60280) 2024-09-19T04:55:36.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:36.281+0000 7f9211a00640 1 -- 172.21.15.43:0/1745632619 learned_addr learned my addr 172.21.15.43:0/1745632619 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:55:36.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:36.281+0000 7f9211a00640 1 -- 172.21.15.43:0/1745632619 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f920c197580 con 0x7f920c102560 2024-09-19T04:55:36.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:36.281+0000 7f9211a00640 1 --2- 172.21.15.43:0/1745632619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f920c102560 0x7f920c197040 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9200009230 tx=0x7f9200003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:36.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:36.281+0000 7f920a000640 1 -- 172.21.15.43:0/1745632619 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f920003f070 con 0x7f920c102560 2024-09-19T04:55:36.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:36.282+0000 7f920a000640 1 -- 172.21.15.43:0/1745632619 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f920003bd40 con 0x7f920c102560 2024-09-19T04:55:36.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:36.282+0000 7f920a000640 1 -- 172.21.15.43:0/1745632619 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9200035030 con 0x7f920c102560 2024-09-19T04:55:36.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:36.282+0000 7f9212e00640 1 -- 172.21.15.43:0/1745632619 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f920c197810 con 0x7f920c102560 2024-09-19T04:55:36.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:36.282+0000 7f9212e00640 1 -- 172.21.15.43:0/1745632619 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f920c197c70 con 0x7f920c102560 2024-09-19T04:55:36.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:36.283+0000 7f920a000640 1 -- 172.21.15.43:0/1745632619 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f9200042070 con 0x7f920c102560 2024-09-19T04:55:36.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:36.283+0000 7f9212e00640 1 -- 172.21.15.43:0/1745632619 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f920c069f90 con 0x7f920c102560 2024-09-19T04:55:36.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:36.283+0000 7f920a000640 1 --2- 172.21.15.43:0/1745632619 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f91f4039300 0x7f91f403b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:36.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:36.283+0000 7f920a000640 1 -- 172.21.15.43:0/1745632619 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f920006cc00 con 0x7f920c102560 2024-09-19T04:55:36.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:36.283+0000 7f9211000640 1 --2- 172.21.15.43:0/1745632619 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f91f4039300 0x7f91f403b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:36.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:36.284+0000 7f9211000640 1 --2- 172.21.15.43:0/1745632619 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f91f4039300 0x7f91f403b7c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f91f80098c0 tx=0x7f91f8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:36.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:36.288+0000 7f920a000640 1 -- 172.21.15.43:0/1745632619 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9200044e10 con 0x7f920c102560 2024-09-19T04:55:36.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:36.479+0000 7f9212e00640 1 -- 172.21.15.43:0/1745632619 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f920c198400 con 0x7f920c102560 2024-09-19T04:55:36.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:36.531+0000 7f920a000640 1 -- 172.21.15.43:0/1745632619 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f9200040d50 con 0x7f920c102560 2024-09-19T04:55:36.534 INFO:teuthology.orchestra.run.smithi043.stderr:adjusted tunables profile to default 2024-09-19T04:55:36.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:36.534+0000 7f9212e00640 1 -- 172.21.15.43:0/1745632619 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f91f4039300 msgr2=0x7f91f403b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:36.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:36.534+0000 7f9212e00640 1 --2- 172.21.15.43:0/1745632619 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f91f4039300 0x7f91f403b7c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f91f80098c0 tx=0x7f91f8006db0 comp rx=0 tx=0).stop 2024-09-19T04:55:36.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:36.534+0000 7f9212e00640 1 -- 172.21.15.43:0/1745632619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f920c102560 msgr2=0x7f920c197040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:36.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:36.534+0000 7f9212e00640 1 --2- 172.21.15.43:0/1745632619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f920c102560 0x7f920c197040 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9200009230 tx=0x7f9200003d80 comp rx=0 tx=0).stop 2024-09-19T04:55:36.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:36.534+0000 7f9212e00640 1 -- 172.21.15.43:0/1745632619 shutdown_connections 2024-09-19T04:55:36.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:36.534+0000 7f9212e00640 1 --2- 172.21.15.43:0/1745632619 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f91f4039300 0x7f91f403b7c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:36.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:36.534+0000 7f9212e00640 1 --2- 172.21.15.43:0/1745632619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f920c102560 0x7f920c197040 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:36.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:36.534+0000 7f9212e00640 1 -- 172.21.15.43:0/1745632619 >> 172.21.15.43:0/1745632619 conn(0x7f920c077170 msgr2=0x7f920c105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:36.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:36.534+0000 7f9212e00640 1 -- 172.21.15.43:0/1745632619 shutdown_connections 2024-09-19T04:55:36.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:36.534+0000 7f9212e00640 1 -- 172.21.15.43:0/1745632619 wait complete. 2024-09-19T04:55:36.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:36 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/1745632619' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-19T04:55:37.282 INFO:tasks.cephadm:Adding mon.a on smithi043 2024-09-19T04:55:37.282 INFO:tasks.cephadm:Adding mon.b on smithi096 2024-09-19T04:55:37.282 INFO:tasks.cephadm:Adding mon.c on smithi151 2024-09-19T04:55:37.282 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi043:172.21.15.43=a;smithi096:172.21.15.96=b;smithi151:172.21.15.151=c' 2024-09-19T04:55:37.526 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:55:38.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:38.018+0000 7f7e95f11640 1 -- 172.21.15.151:0/594726098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e901065f0 msgr2=0x7f7e901069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:38.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:38.018+0000 7f7e95f11640 1 --2- 172.21.15.151:0/594726098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e901065f0 0x7f7e901069d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7e80009880 tx=0x7f7e8002a5d0 comp rx=0 tx=0).stop 2024-09-19T04:55:38.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:38.018+0000 7f7e95f11640 1 -- 172.21.15.151:0/594726098 shutdown_connections 2024-09-19T04:55:38.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:38.018+0000 7f7e95f11640 1 --2- 172.21.15.151:0/594726098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e901065f0 0x7f7e901069d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:38.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:38.018+0000 7f7e95f11640 1 -- 172.21.15.151:0/594726098 >> 172.21.15.151:0/594726098 conn(0x7f7e900fb930 msgr2=0x7f7e900fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:38.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:38.018+0000 7f7e95f11640 1 -- 172.21.15.151:0/594726098 shutdown_connections 2024-09-19T04:55:38.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:38.018+0000 7f7e95f11640 1 -- 172.21.15.151:0/594726098 wait complete. 2024-09-19T04:55:38.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:38.018+0000 7f7e95f11640 1 Processor -- start 2024-09-19T04:55:38.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:38.019+0000 7f7e95f11640 1 -- start start 2024-09-19T04:55:38.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:38.019+0000 7f7e95f11640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e901065f0 0x7f7e90196010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:38.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:38.019+0000 7f7e95f11640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9010ac00 con 0x7f7e901065f0 2024-09-19T04:55:38.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:38.019+0000 7f7e8f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e901065f0 0x7f7e90196010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:38.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:38.019+0000 7f7e8f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e901065f0 0x7f7e90196010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.151:52832/0 (socket says 172.21.15.151:52832) 2024-09-19T04:55:38.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:38.019+0000 7f7e8f7fe640 1 -- 172.21.15.151:0/255675752 learned_addr learned my addr 172.21.15.151:0/255675752 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-19T04:55:38.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:38.020+0000 7f7e8f7fe640 1 -- 172.21.15.151:0/255675752 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e90196550 con 0x7f7e901065f0 2024-09-19T04:55:38.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:38.020+0000 7f7e8f7fe640 1 --2- 172.21.15.151:0/255675752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e901065f0 0x7f7e90196010 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7e80009230 tx=0x7f7e80003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:38.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:38.020+0000 7f7e8cff9640 1 -- 172.21.15.151:0/255675752 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e8003f070 con 0x7f7e901065f0 2024-09-19T04:55:38.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:38.020+0000 7f7e95f11640 1 -- 172.21.15.151:0/255675752 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e901967e0 con 0x7f7e901065f0 2024-09-19T04:55:38.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:38.020+0000 7f7e8cff9640 1 -- 172.21.15.151:0/255675752 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e8003bc80 con 0x7f7e901065f0 2024-09-19T04:55:38.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:38.020+0000 7f7e8cff9640 1 -- 172.21.15.151:0/255675752 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e80035030 con 0x7f7e901065f0 2024-09-19T04:55:38.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:38.020+0000 7f7e95f11640 1 -- 172.21.15.151:0/255675752 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e9019a8e0 con 0x7f7e901065f0 2024-09-19T04:55:38.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:38.021+0000 7f7e8cff9640 1 -- 172.21.15.151:0/255675752 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f7e80042070 con 0x7f7e901065f0 2024-09-19T04:55:38.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:38.021+0000 7f7e95f11640 1 -- 172.21.15.151:0/255675752 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e90101c40 con 0x7f7e901065f0 2024-09-19T04:55:38.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:38.021+0000 7f7e8cff9640 1 --2- 172.21.15.151:0/255675752 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f7e780392a0 0x7f7e7803b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:38.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:38.021+0000 7f7e8cff9640 1 -- 172.21.15.151:0/255675752 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7e8006cff0 con 0x7f7e901065f0 2024-09-19T04:55:38.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:38.021+0000 7f7e8effd640 1 --2- 172.21.15.151:0/255675752 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f7e780392a0 0x7f7e7803b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:38.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:38.022+0000 7f7e8effd640 1 --2- 172.21.15.151:0/255675752 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f7e780392a0 0x7f7e7803b760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7e840098c0 tx=0x7f7e84006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:38.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:38.026+0000 7f7e8cff9640 1 -- 172.21.15.151:0/255675752 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7e8004c350 con 0x7f7e901065f0 2024-09-19T04:55:38.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:37 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/1745632619' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-19T04:55:38.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:37 smithi043 ceph-mon[30800]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:55:38.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:37 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:38.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:37 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:38.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:37 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:38.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:37 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:38.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:37 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi151", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:55:38.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:37 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:55:38.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:37 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:55:38.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:37 smithi043 ceph-mon[30800]: Updating smithi151:/etc/ceph/ceph.conf 2024-09-19T04:55:38.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:37 smithi043 ceph-mon[30800]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:55:38.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:38.201+0000 7f7e95f11640 1 -- 172.21.15.151:0/255675752 --> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi043:172.21.15.43=a;smithi096:172.21.15.96=b;smithi151:172.21.15.151=c", "target": ["mon-mgr", ""]}) -- 0x7f7e90197370 con 0x7f7e780392a0 2024-09-19T04:55:38.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:38.207+0000 7f7e8cff9640 1 -- 172.21.15.151:0/255675752 <== mgr.14150 v2:172.21.15.43:6800/3167108925 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f7e90197370 con 0x7f7e780392a0 2024-09-19T04:55:38.208 INFO:teuthology.orchestra.run.smithi151.stdout:Scheduled mon update... 2024-09-19T04:55:38.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:38.209+0000 7f7e95f11640 1 -- 172.21.15.151:0/255675752 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f7e780392a0 msgr2=0x7f7e7803b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:38.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:38.209+0000 7f7e95f11640 1 --2- 172.21.15.151:0/255675752 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f7e780392a0 0x7f7e7803b760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7e840098c0 tx=0x7f7e84006db0 comp rx=0 tx=0).stop 2024-09-19T04:55:38.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:38.209+0000 7f7e95f11640 1 -- 172.21.15.151:0/255675752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e901065f0 msgr2=0x7f7e90196010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:38.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:38.209+0000 7f7e95f11640 1 --2- 172.21.15.151:0/255675752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e901065f0 0x7f7e90196010 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7e80009230 tx=0x7f7e80003d40 comp rx=0 tx=0).stop 2024-09-19T04:55:38.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:38.209+0000 7f7e95f11640 1 -- 172.21.15.151:0/255675752 shutdown_connections 2024-09-19T04:55:38.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:38.209+0000 7f7e95f11640 1 --2- 172.21.15.151:0/255675752 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f7e780392a0 0x7f7e7803b760 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:38.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:38.209+0000 7f7e95f11640 1 --2- 172.21.15.151:0/255675752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e901065f0 0x7f7e90196010 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:38.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:38.209+0000 7f7e95f11640 1 -- 172.21.15.151:0/255675752 >> 172.21.15.151:0/255675752 conn(0x7f7e900fb930 msgr2=0x7f7e900fd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:38.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:38.209+0000 7f7e95f11640 1 -- 172.21.15.151:0/255675752 shutdown_connections 2024-09-19T04:55:38.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:38.209+0000 7f7e95f11640 1 -- 172.21.15.151:0/255675752 wait complete. 2024-09-19T04:55:38.859 DEBUG:teuthology.orchestra.run.smithi096:mon.b> sudo journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@mon.b.service 2024-09-19T04:55:38.862 DEBUG:teuthology.orchestra.run.smithi151:mon.c> sudo journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@mon.c.service 2024-09-19T04:55:38.865 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T04:55:38.865 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T04:55:39.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:38 smithi043 ceph-mon[30800]: Updating smithi151:/var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:55:39.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:38 smithi043 ceph-mon[30800]: Updating smithi151:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:55:39.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:38 smithi043 ceph-mon[30800]: Updating smithi151:/var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:55:39.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:38 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:39.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:38 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:39.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:38 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:39.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:38 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:39.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:38 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:55:39.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:38 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:55:39.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:38 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:55:39.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:38 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:39.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:38 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:55:39.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:38 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:55:39.841 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:55:40.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:39 smithi043 ceph-mon[30800]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi043:172.21.15.43=a;smithi096:172.21.15.96=b;smithi151:172.21.15.151=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:55:40.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:39 smithi043 ceph-mon[30800]: Saving service mon spec with placement smithi043:172.21.15.43=a;smithi096:172.21.15.96=b;smithi151:172.21.15.151=c;count:3 2024-09-19T04:55:40.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:39 smithi043 ceph-mon[30800]: Deploying daemon mon.c on smithi151 2024-09-19T04:55:40.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:39 smithi043 ceph-mon[30800]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:55:41.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:41.503+0000 7f404bbff640 1 -- 172.21.15.151:0/2709436375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4044102730 msgr2=0x7f4044102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:41.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:41.503+0000 7f404bbff640 1 --2- 172.21.15.151:0/2709436375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4044102730 0x7f4044102b10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4040009810 tx=0x7f404002a540 comp rx=0 tx=0).stop 2024-09-19T04:55:41.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:41.503+0000 7f404bbff640 1 -- 172.21.15.151:0/2709436375 shutdown_connections 2024-09-19T04:55:41.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:41.503+0000 7f404bbff640 1 --2- 172.21.15.151:0/2709436375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4044102730 0x7f4044102b10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:41.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:41.503+0000 7f404bbff640 1 -- 172.21.15.151:0/2709436375 >> 172.21.15.151:0/2709436375 conn(0x7f4044077310 msgr2=0x7f40440fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:41.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:41.503+0000 7f404bbff640 1 -- 172.21.15.151:0/2709436375 shutdown_connections 2024-09-19T04:55:41.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:41.503+0000 7f404bbff640 1 -- 172.21.15.151:0/2709436375 wait complete. 2024-09-19T04:55:41.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:41.504+0000 7f404bbff640 1 Processor -- start 2024-09-19T04:55:41.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:41.504+0000 7f404bbff640 1 -- start start 2024-09-19T04:55:41.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:41.504+0000 7f404bbff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4044102730 0x7f40441971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:41.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:41.504+0000 7f404bbff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f404410bc80 con 0x7f4044102730 2024-09-19T04:55:41.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:41.505+0000 7f404999b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4044102730 0x7f40441971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:41.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:41.505+0000 7f404999b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4044102730 0x7f40441971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.151:52842/0 (socket says 172.21.15.151:52842) 2024-09-19T04:55:41.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:41.505+0000 7f404999b640 1 -- 172.21.15.151:0/115911064 learned_addr learned my addr 172.21.15.151:0/115911064 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-19T04:55:41.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:41.505+0000 7f404999b640 1 -- 172.21.15.151:0/115911064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40441976f0 con 0x7f4044102730 2024-09-19T04:55:41.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:41.505+0000 7f404999b640 1 --2- 172.21.15.151:0/115911064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4044102730 0x7f40441971b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f40400045e0 tx=0x7f4040004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:41.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:41.505+0000 7f403affd640 1 -- 172.21.15.151:0/115911064 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4040030e70 con 0x7f4044102730 2024-09-19T04:55:41.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:41.505+0000 7f404bbff640 1 -- 172.21.15.151:0/115911064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4044197980 con 0x7f4044102730 2024-09-19T04:55:41.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:41.505+0000 7f403affd640 1 -- 172.21.15.151:0/115911064 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40400318e0 con 0x7f4044102730 2024-09-19T04:55:41.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:41.505+0000 7f403affd640 1 -- 172.21.15.151:0/115911064 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40400049c0 con 0x7f4044102730 2024-09-19T04:55:41.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:41.506+0000 7f404bbff640 1 -- 172.21.15.151:0/115911064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4044197de0 con 0x7f4044102730 2024-09-19T04:55:41.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:41.506+0000 7f404bbff640 1 -- 172.21.15.151:0/115911064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4044069f90 con 0x7f4044102730 2024-09-19T04:55:41.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:41.507+0000 7f403affd640 1 -- 172.21.15.151:0/115911064 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f4040039040 con 0x7f4044102730 2024-09-19T04:55:41.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:41.507+0000 7f403affd640 1 --2- 172.21.15.151:0/115911064 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f40300392a0 0x7f403003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:41.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:41.507+0000 7f403affd640 1 -- 172.21.15.151:0/115911064 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f404006d670 con 0x7f4044102730 2024-09-19T04:55:41.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:41.507+0000 7f404919a640 1 --2- 172.21.15.151:0/115911064 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f40300392a0 0x7f403003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:41.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:41.508+0000 7f404919a640 1 --2- 172.21.15.151:0/115911064 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f40300392a0 0x7f403003b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f40340098c0 tx=0x7f4034006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:41.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:41.512+0000 7f403affd640 1 -- 172.21.15.151:0/115911064 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4040040030 con 0x7f4044102730 2024-09-19T04:55:41.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:41 smithi043 ceph-mon[30800]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:55:41.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:41.770+0000 7f404bbff640 1 -- 172.21.15.151:0/115911064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f4044198510 con 0x7f4044102730 2024-09-19T04:55:41.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:41.770+0000 7f403affd640 1 -- 172.21.15.151:0/115911064 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f404003b3b0 con 0x7f4044102730 2024-09-19T04:55:41.772 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T04:55:41.772 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":1,"fsid":"1456a810-7643-11ef-bceb-c7b262605968","modified":"2024-09-19T04:54:11.079879Z","created":"2024-09-19T04:54:11.079879Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-19T04:55:41.772 INFO:teuthology.orchestra.run.smithi151.stderr:dumped monmap epoch 1 2024-09-19T04:55:41.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:41.772+0000 7f404bbff640 1 -- 172.21.15.151:0/115911064 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f40300392a0 msgr2=0x7f403003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:41.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:41.772+0000 7f404bbff640 1 --2- 172.21.15.151:0/115911064 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f40300392a0 0x7f403003b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f40340098c0 tx=0x7f4034006db0 comp rx=0 tx=0).stop 2024-09-19T04:55:41.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:41.772+0000 7f404bbff640 1 -- 172.21.15.151:0/115911064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4044102730 msgr2=0x7f40441971b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:41.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:41.772+0000 7f404bbff640 1 --2- 172.21.15.151:0/115911064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4044102730 0x7f40441971b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f40400045e0 tx=0x7f4040004610 comp rx=0 tx=0).stop 2024-09-19T04:55:41.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:41.772+0000 7f404bbff640 1 -- 172.21.15.151:0/115911064 shutdown_connections 2024-09-19T04:55:41.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:41.772+0000 7f404bbff640 1 --2- 172.21.15.151:0/115911064 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f40300392a0 0x7f403003b760 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:41.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:41.772+0000 7f404bbff640 1 --2- 172.21.15.151:0/115911064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4044102730 0x7f40441971b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:41.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:41.773+0000 7f404bbff640 1 -- 172.21.15.151:0/115911064 >> 172.21.15.151:0/115911064 conn(0x7f4044077310 msgr2=0x7f40441054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:41.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:41.773+0000 7f404bbff640 1 -- 172.21.15.151:0/115911064 shutdown_connections 2024-09-19T04:55:41.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:41.773+0000 7f404bbff640 1 -- 172.21.15.151:0/115911064 wait complete. 2024-09-19T04:55:42.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:42 smithi043 ceph-mon[30800]: from='client.? 172.21.15.151:0/115911064' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:55:42.870 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 podman[35407]: 2024-09-19 04:55:42.8476081 +0000 UTC m=+0.347093242 container start 92705e2799e7cc20a1ac0925b0f6ff3a51a0330bb6d9dc505889143a6264327c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2) 2024-09-19T04:55:42.870 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: set uid:gid to 167:167 (ceph:ceph) 2024-09-19T04:55:42.871 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable), process ceph-mon, pid 2 2024-09-19T04:55:42.871 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: pidfile_write: ignore empty --pid-file 2024-09-19T04:55:43.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: load: jerasure load: lrc 2024-09-19T04:55:43.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: RocksDB version: 7.9.2 2024-09-19T04:55:43.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Git sha 0 2024-09-19T04:55:43.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Compile date 2024-09-18 16:29:48 2024-09-19T04:55:43.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: DB SUMMARY 2024-09-19T04:55:43.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: DB Session ID: 99SPJIAC282WOCXX6NU1 2024-09-19T04:55:43.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: CURRENT file: CURRENT 2024-09-19T04:55:43.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: IDENTITY file: IDENTITY 2024-09-19T04:55:43.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-19T04:55:43.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-19T04:55:43.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-19T04:55:43.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.error_if_exists: 0 2024-09-19T04:55:43.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.create_if_missing: 0 2024-09-19T04:55:43.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.paranoid_checks: 1 2024-09-19T04:55:43.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-19T04:55:43.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-19T04:55:43.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-19T04:55:43.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.env: 0x55645e0c0940 2024-09-19T04:55:43.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.fs: PosixFileSystem 2024-09-19T04:55:43.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.info_log: 0x5564601036e0 2024-09-19T04:55:43.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.max_file_opening_threads: 16 2024-09-19T04:55:43.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.statistics: (nil) 2024-09-19T04:55:43.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.use_fsync: 0 2024-09-19T04:55:43.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.max_log_file_size: 0 2024-09-19T04:55:43.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-19T04:55:43.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-19T04:55:43.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.keep_log_file_num: 1000 2024-09-19T04:55:43.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.recycle_log_file_num: 0 2024-09-19T04:55:43.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.allow_fallocate: 1 2024-09-19T04:55:43.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.allow_mmap_reads: 0 2024-09-19T04:55:43.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.allow_mmap_writes: 0 2024-09-19T04:55:43.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.use_direct_reads: 0 2024-09-19T04:55:43.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-19T04:55:43.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.create_missing_column_families: 0 2024-09-19T04:55:43.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.db_log_dir: 2024-09-19T04:55:43.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.wal_dir: 2024-09-19T04:55:43.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-19T04:55:43.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-19T04:55:43.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-19T04:55:43.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-19T04:55:43.160 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-19T04:55:43.160 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-19T04:55:43.160 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.advise_random_on_open: 1 2024-09-19T04:55:43.160 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.db_write_buffer_size: 0 2024-09-19T04:55:43.160 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.write_buffer_manager: 0x556460112dc0 2024-09-19T04:55:43.160 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-19T04:55:43.160 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-19T04:55:43.161 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-19T04:55:43.161 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.rate_limiter: (nil) 2024-09-19T04:55:43.161 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-19T04:55:43.161 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.wal_recovery_mode: 2 2024-09-19T04:55:43.161 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.enable_thread_tracking: 0 2024-09-19T04:55:43.161 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.enable_pipelined_write: 0 2024-09-19T04:55:43.161 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.unordered_write: 0 2024-09-19T04:55:43.161 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-19T04:55:43.161 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-19T04:55:43.161 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-19T04:55:43.161 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-19T04:55:43.161 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.row_cache: None 2024-09-19T04:55:43.161 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.wal_filter: None 2024-09-19T04:55:43.162 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-19T04:55:43.162 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.allow_ingest_behind: 0 2024-09-19T04:55:43.162 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.two_write_queues: 0 2024-09-19T04:55:43.162 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.manual_wal_flush: 0 2024-09-19T04:55:43.162 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.wal_compression: 0 2024-09-19T04:55:43.162 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.atomic_flush: 0 2024-09-19T04:55:43.162 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-19T04:55:43.162 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-19T04:55:43.162 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-19T04:55:43.162 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.log_readahead_size: 0 2024-09-19T04:55:43.162 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-19T04:55:43.162 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.best_efforts_recovery: 0 2024-09-19T04:55:43.163 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-19T04:55:43.163 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-19T04:55:43.163 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.allow_data_in_errors: 0 2024-09-19T04:55:43.163 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.db_host_id: __hostname__ 2024-09-19T04:55:43.163 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-19T04:55:43.163 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.max_background_jobs: 2 2024-09-19T04:55:43.163 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.max_background_compactions: -1 2024-09-19T04:55:43.163 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.max_subcompactions: 1 2024-09-19T04:55:43.163 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-19T04:55:43.163 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-19T04:55:43.163 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-19T04:55:43.163 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.max_total_wal_size: 0 2024-09-19T04:55:43.163 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-19T04:55:43.164 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-19T04:55:43.164 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-19T04:55:43.164 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-19T04:55:43.164 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.max_open_files: -1 2024-09-19T04:55:43.164 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.bytes_per_sync: 0 2024-09-19T04:55:43.164 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-19T04:55:43.164 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-19T04:55:43.164 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.compaction_readahead_size: 0 2024-09-19T04:55:43.164 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.max_background_flushes: -1 2024-09-19T04:55:43.164 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Compression algorithms supported: 2024-09-19T04:55:43.164 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: kZSTD supported: 0 2024-09-19T04:55:43.164 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: kXpressCompression supported: 0 2024-09-19T04:55:43.164 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: kBZip2Compression supported: 0 2024-09-19T04:55:43.164 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-19T04:55:43.165 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: kLZ4Compression supported: 1 2024-09-19T04:55:43.165 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: kZlibCompression supported: 1 2024-09-19T04:55:43.165 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: kLZ4HCCompression supported: 1 2024-09-19T04:55:43.165 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: kSnappyCompression supported: 1 2024-09-19T04:55:43.165 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-19T04:55:43.165 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-19T04:55:43.165 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-19T04:55:43.165 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-19T04:55:43.165 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-19T04:55:43.165 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.merge_operator: 2024-09-19T04:55:43.165 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.compaction_filter: None 2024-09-19T04:55:43.165 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.compaction_filter_factory: None 2024-09-19T04:55:43.165 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.sst_partitioner_factory: None 2024-09-19T04:55:43.165 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-19T04:55:43.166 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-19T04:55:43.166 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556460103320) 2024-09-19T04:55:43.166 INFO:journalctl@ceph.mon.c.smithi151.stdout: cache_index_and_filter_blocks: 1 2024-09-19T04:55:43.166 INFO:journalctl@ceph.mon.c.smithi151.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-19T04:55:43.166 INFO:journalctl@ceph.mon.c.smithi151.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-19T04:55:43.166 INFO:journalctl@ceph.mon.c.smithi151.stdout: pin_top_level_index_and_filter: 1 2024-09-19T04:55:43.166 INFO:journalctl@ceph.mon.c.smithi151.stdout: index_type: 0 2024-09-19T04:55:43.166 INFO:journalctl@ceph.mon.c.smithi151.stdout: data_block_index_type: 0 2024-09-19T04:55:43.166 INFO:journalctl@ceph.mon.c.smithi151.stdout: index_shortening: 1 2024-09-19T04:55:43.166 INFO:journalctl@ceph.mon.c.smithi151.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-19T04:55:43.166 INFO:journalctl@ceph.mon.c.smithi151.stdout: checksum: 4 2024-09-19T04:55:43.166 INFO:journalctl@ceph.mon.c.smithi151.stdout: no_block_cache: 0 2024-09-19T04:55:43.166 INFO:journalctl@ceph.mon.c.smithi151.stdout: block_cache: 0x5564600f71f0 2024-09-19T04:55:43.167 INFO:journalctl@ceph.mon.c.smithi151.stdout: block_cache_name: BinnedLRUCache 2024-09-19T04:55:43.167 INFO:journalctl@ceph.mon.c.smithi151.stdout: block_cache_options: 2024-09-19T04:55:43.167 INFO:journalctl@ceph.mon.c.smithi151.stdout: capacity : 536870912 2024-09-19T04:55:43.167 INFO:journalctl@ceph.mon.c.smithi151.stdout: num_shard_bits : 4 2024-09-19T04:55:43.167 INFO:journalctl@ceph.mon.c.smithi151.stdout: strict_capacity_limit : 0 2024-09-19T04:55:43.167 INFO:journalctl@ceph.mon.c.smithi151.stdout: high_pri_pool_ratio: 0.000 2024-09-19T04:55:43.167 INFO:journalctl@ceph.mon.c.smithi151.stdout: block_cache_compressed: (nil) 2024-09-19T04:55:43.167 INFO:journalctl@ceph.mon.c.smithi151.stdout: persistent_cache: (nil) 2024-09-19T04:55:43.167 INFO:journalctl@ceph.mon.c.smithi151.stdout: block_size: 4096 2024-09-19T04:55:43.167 INFO:journalctl@ceph.mon.c.smithi151.stdout: block_size_deviation: 10 2024-09-19T04:55:43.167 INFO:journalctl@ceph.mon.c.smithi151.stdout: block_restart_interval: 16 2024-09-19T04:55:43.167 INFO:journalctl@ceph.mon.c.smithi151.stdout: index_block_restart_interval: 1 2024-09-19T04:55:43.167 INFO:journalctl@ceph.mon.c.smithi151.stdout: metadata_block_size: 4096 2024-09-19T04:55:43.167 INFO:journalctl@ceph.mon.c.smithi151.stdout: partition_filters: 0 2024-09-19T04:55:43.168 INFO:journalctl@ceph.mon.c.smithi151.stdout: use_delta_encoding: 1 2024-09-19T04:55:43.168 INFO:journalctl@ceph.mon.c.smithi151.stdout: filter_policy: bloomfilter 2024-09-19T04:55:43.168 INFO:journalctl@ceph.mon.c.smithi151.stdout: whole_key_filtering: 1 2024-09-19T04:55:43.168 INFO:journalctl@ceph.mon.c.smithi151.stdout: verify_compression: 0 2024-09-19T04:55:43.168 INFO:journalctl@ceph.mon.c.smithi151.stdout: read_amp_bytes_per_bit: 0 2024-09-19T04:55:43.168 INFO:journalctl@ceph.mon.c.smithi151.stdout: format_version: 5 2024-09-19T04:55:43.168 INFO:journalctl@ceph.mon.c.smithi151.stdout: enable_index_compression: 1 2024-09-19T04:55:43.168 INFO:journalctl@ceph.mon.c.smithi151.stdout: block_align: 0 2024-09-19T04:55:43.168 INFO:journalctl@ceph.mon.c.smithi151.stdout: max_auto_readahead_size: 262144 2024-09-19T04:55:43.168 INFO:journalctl@ceph.mon.c.smithi151.stdout: prepopulate_block_cache: 0 2024-09-19T04:55:43.168 INFO:journalctl@ceph.mon.c.smithi151.stdout: initial_auto_readahead_size: 8192 2024-09-19T04:55:43.168 INFO:journalctl@ceph.mon.c.smithi151.stdout: num_file_reads_for_auto_readahead: 2 2024-09-19T04:55:43.168 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.write_buffer_size: 33554432 2024-09-19T04:55:43.168 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.max_write_buffer_number: 2 2024-09-19T04:55:43.169 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.compression: NoCompression 2024-09-19T04:55:43.169 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.bottommost_compression: Disabled 2024-09-19T04:55:43.169 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.prefix_extractor: nullptr 2024-09-19T04:55:43.169 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-19T04:55:43.169 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.num_levels: 7 2024-09-19T04:55:43.169 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-19T04:55:43.169 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-19T04:55:43.169 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-19T04:55:43.169 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-19T04:55:43.169 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-19T04:55:43.169 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-19T04:55:43.169 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-19T04:55:43.170 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-19T04:55:43.170 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-19T04:55:43.170 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-19T04:55:43.170 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-19T04:55:43.170 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-19T04:55:43.170 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-19T04:55:43.170 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.compression_opts.level: 32767 2024-09-19T04:55:43.170 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.compression_opts.strategy: 0 2024-09-19T04:55:43.170 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-19T04:55:43.170 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-19T04:55:43.170 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-19T04:55:43.170 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-19T04:55:43.170 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.compression_opts.enabled: false 2024-09-19T04:55:43.171 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-19T04:55:43.171 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-19T04:55:43.171 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-19T04:55:43.171 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-19T04:55:43.171 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.target_file_size_base: 67108864 2024-09-19T04:55:43.171 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-19T04:55:43.171 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-19T04:55:43.171 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-19T04:55:43.171 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-19T04:55:43.171 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-19T04:55:43.171 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-19T04:55:43.171 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-19T04:55:43.171 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-19T04:55:43.171 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-19T04:55:43.172 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-19T04:55:43.172 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-19T04:55:43.172 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-19T04:55:43.172 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-19T04:55:43.172 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-19T04:55:43.172 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.arena_block_size: 1048576 2024-09-19T04:55:43.172 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-19T04:55:43.172 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-19T04:55:43.172 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.disable_auto_compactions: 0 2024-09-19T04:55:43.172 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-19T04:55:43.172 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-19T04:55:43.172 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-19T04:55:43.172 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-19T04:55:43.172 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-19T04:55:43.173 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-19T04:55:43.173 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-19T04:55:43.173 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-19T04:55:43.173 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-19T04:55:43.173 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-19T04:55:43.173 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-19T04:55:43.173 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.inplace_update_support: 0 2024-09-19T04:55:43.173 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-19T04:55:43.173 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-19T04:55:43.173 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-19T04:55:43.173 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-19T04:55:43.173 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.bloom_locality: 0 2024-09-19T04:55:43.173 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.max_successive_merges: 0 2024-09-19T04:55:43.174 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-19T04:55:43.174 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.paranoid_file_checks: 0 2024-09-19T04:55:43.174 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.force_consistency_checks: 1 2024-09-19T04:55:43.174 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.report_bg_io_stats: 0 2024-09-19T04:55:43.174 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.ttl: 2592000 2024-09-19T04:55:43.174 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-19T04:55:43.174 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-19T04:55:43.174 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-19T04:55:43.174 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.enable_blob_files: false 2024-09-19T04:55:43.174 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.min_blob_size: 0 2024-09-19T04:55:43.174 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.blob_file_size: 268435456 2024-09-19T04:55:43.174 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-19T04:55:43.174 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-19T04:55:43.174 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-19T04:55:43.175 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-19T04:55:43.175 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-19T04:55:43.175 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.blob_file_starting_level: 0 2024-09-19T04:55:43.175 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-19T04:55:43.175 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-19T04:55:43.175 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-19T04:55:43.175 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: b5b7ba98-bb7e-4adc-bf1d-26f1268bcd3c 2024-09-19T04:55:43.175 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726721742875351, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-19T04:55:43.175 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-19T04:55:43.175 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726721742875910, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726721742, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "b5b7ba98-bb7e-4adc-bf1d-26f1268bcd3c", "db_session_id": "99SPJIAC282WOCXX6NU1", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-19T04:55:43.175 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726721742876034, "job": 1, "event": "recovery_finished"} 2024-09-19T04:55:43.175 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-19T04:55:43.175 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-19T04:55:43.176 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556460124e00 2024-09-19T04:55:43.176 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: DB pointer 0x556460220000 2024-09-19T04:55:43.176 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-19T04:55:43.176 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-19T04:55:43.176 INFO:journalctl@ceph.mon.c.smithi151.stdout: ** DB Stats ** 2024-09-19T04:55:43.176 INFO:journalctl@ceph.mon.c.smithi151.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T04:55:43.176 INFO:journalctl@ceph.mon.c.smithi151.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-19T04:55:43.176 INFO:journalctl@ceph.mon.c.smithi151.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T04:55:43.176 INFO:journalctl@ceph.mon.c.smithi151.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T04:55:43.176 INFO:journalctl@ceph.mon.c.smithi151.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-19T04:55:43.176 INFO:journalctl@ceph.mon.c.smithi151.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T04:55:43.176 INFO:journalctl@ceph.mon.c.smithi151.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T04:55:43.176 INFO:journalctl@ceph.mon.c.smithi151.stdout: 2024-09-19T04:55:43.176 INFO:journalctl@ceph.mon.c.smithi151.stdout: ** Compaction Stats [default] ** 2024-09-19T04:55:43.177 INFO:journalctl@ceph.mon.c.smithi151.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T04:55:43.177 INFO:journalctl@ceph.mon.c.smithi151.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-19T04:55:43.177 INFO:journalctl@ceph.mon.c.smithi151.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T04:55:43.177 INFO:journalctl@ceph.mon.c.smithi151.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T04:55:43.177 INFO:journalctl@ceph.mon.c.smithi151.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T04:55:43.177 INFO:journalctl@ceph.mon.c.smithi151.stdout: 2024-09-19T04:55:43.177 INFO:journalctl@ceph.mon.c.smithi151.stdout: ** Compaction Stats [default] ** 2024-09-19T04:55:43.177 INFO:journalctl@ceph.mon.c.smithi151.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T04:55:43.177 INFO:journalctl@ceph.mon.c.smithi151.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-19T04:55:43.177 INFO:journalctl@ceph.mon.c.smithi151.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T04:55:43.177 INFO:journalctl@ceph.mon.c.smithi151.stdout: 2024-09-19T04:55:43.177 INFO:journalctl@ceph.mon.c.smithi151.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-19T04:55:43.177 INFO:journalctl@ceph.mon.c.smithi151.stdout: 2024-09-19T04:55:43.178 INFO:journalctl@ceph.mon.c.smithi151.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T04:55:43.178 INFO:journalctl@ceph.mon.c.smithi151.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-19T04:55:43.178 INFO:journalctl@ceph.mon.c.smithi151.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-19T04:55:43.178 INFO:journalctl@ceph.mon.c.smithi151.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-19T04:55:43.178 INFO:journalctl@ceph.mon.c.smithi151.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-19T04:55:43.178 INFO:journalctl@ceph.mon.c.smithi151.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-19T04:55:43.178 INFO:journalctl@ceph.mon.c.smithi151.stdout: Cumulative compaction: 0.00 GB write, 0.43 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T04:55:43.178 INFO:journalctl@ceph.mon.c.smithi151.stdout: Interval compaction: 0.00 GB write, 0.43 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T04:55:43.178 INFO:journalctl@ceph.mon.c.smithi151.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-19T04:55:43.178 INFO:journalctl@ceph.mon.c.smithi151.stdout: Block cache BinnedLRUCache@0x5564600f71f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-19T04:55:43.178 INFO:journalctl@ceph.mon.c.smithi151.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-19T04:55:43.178 INFO:journalctl@ceph.mon.c.smithi151.stdout: 2024-09-19T04:55:43.178 INFO:journalctl@ceph.mon.c.smithi151.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-19T04:55:43.178 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-19T04:55:43.179 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: using public_addr v2:172.21.15.151:0/0 -> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] 2024-09-19T04:55:43.179 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: starting mon.c rank -1 at public addrs [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] at bind addrs [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 1456a810-7643-11ef-bceb-c7b262605968 2024-09-19T04:55:43.179 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: mon.c@-1(???) e0 preinit fsid 1456a810-7643-11ef-bceb-c7b262605968 2024-09-19T04:55:43.179 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: mon.c@-1(synchronizing).mds e1 new map 2024-09-19T04:55:43.179 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-19T04:55:43.179 INFO:journalctl@ceph.mon.c.smithi151.stdout: e1 2024-09-19T04:55:43.179 INFO:journalctl@ceph.mon.c.smithi151.stdout: btime 2024-09-19T04:54:14:706422+0000 2024-09-19T04:55:43.179 INFO:journalctl@ceph.mon.c.smithi151.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T04:55:43.179 INFO:journalctl@ceph.mon.c.smithi151.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T04:55:43.179 INFO:journalctl@ceph.mon.c.smithi151.stdout: legacy client fscid: -1 2024-09-19T04:55:43.179 INFO:journalctl@ceph.mon.c.smithi151.stdout: 2024-09-19T04:55:43.179 INFO:journalctl@ceph.mon.c.smithi151.stdout: No filesystems configured 2024-09-19T04:55:43.179 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-19T04:55:43.180 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-19T04:55:43.180 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-19T04:55:43.180 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-19T04:55:43.180 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-19T04:55:43.180 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-19T04:55:43.180 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-19T04:55:43.180 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T04:55:43.180 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T04:55:43.180 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T04:55:43.180 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: Deploying cephadm binary to smithi151 2024-09-19T04:55:43.180 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:55:43.180 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:43.180 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: Added host smithi151 2024-09-19T04:55:43.180 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:55:43.181 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:55:43.181 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:43.181 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:43.181 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:43.181 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T04:55:43.181 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:55:43.181 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/1745632619' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-19T04:55:43.181 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/1745632619' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-19T04:55:43.181 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:55:43.181 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:43.181 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:43.181 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:43.181 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:43.181 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi151", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:55:43.182 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:55:43.182 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:55:43.182 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: Updating smithi151:/etc/ceph/ceph.conf 2024-09-19T04:55:43.182 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:55:43.182 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: Updating smithi151:/var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:55:43.182 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: Updating smithi151:/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:55:43.182 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: Updating smithi151:/var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T04:55:43.182 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:43.182 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:43.182 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:43.182 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:43.182 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:55:43.182 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:55:43.183 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:55:43.183 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:43.183 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:55:43.183 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:55:43.183 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi043:172.21.15.43=a;smithi096:172.21.15.96=b;smithi151:172.21.15.151=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:55:43.183 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: Saving service mon spec with placement smithi043:172.21.15.43=a;smithi096:172.21.15.96=b;smithi151:172.21.15.151=c;count:3 2024-09-19T04:55:43.183 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: Deploying daemon mon.c on smithi151 2024-09-19T04:55:43.183 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:55:43.183 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:55:43.183 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: from='client.? 172.21.15.151:0/115911064' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:55:43.183 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 ceph-mon[35425]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-19T04:55:43.183 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 bash[35407]: 92705e2799e7cc20a1ac0925b0f6ff3a51a0330bb6d9dc505889143a6264327c 2024-09-19T04:55:43.183 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:42 smithi151 systemd[1]: Started Ceph mon.c for 1456a810-7643-11ef-bceb-c7b262605968. 2024-09-19T04:55:43.363 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T04:55:43.364 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T04:55:43.619 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:55:48.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:47 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:55:48.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:47 smithi043 ceph-mon[30800]: mon.a calling monitor election 2024-09-19T04:55:48.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:47 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:55:48.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:47 smithi043 ceph-mon[30800]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:55:48.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:47 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:55:48.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:47 smithi043 ceph-mon[30800]: mon.c calling monitor election 2024-09-19T04:55:48.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:47 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:55:48.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:47 smithi043 ceph-mon[30800]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:55:48.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:47 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:55:48.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:47 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:55:48.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:47 smithi043 ceph-mon[30800]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:55:48.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:47 smithi043 ceph-mon[30800]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T04:55:48.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:47 smithi043 ceph-mon[30800]: monmap epoch 2 2024-09-19T04:55:48.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:47 smithi043 ceph-mon[30800]: fsid 1456a810-7643-11ef-bceb-c7b262605968 2024-09-19T04:55:48.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:47 smithi043 ceph-mon[30800]: last_changed 2024-09-19T04:55:42.904907+0000 2024-09-19T04:55:48.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:47 smithi043 ceph-mon[30800]: created 2024-09-19T04:54:11.079879+0000 2024-09-19T04:55:48.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:47 smithi043 ceph-mon[30800]: min_mon_release 19 (squid) 2024-09-19T04:55:48.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:47 smithi043 ceph-mon[30800]: election_strategy: 1 2024-09-19T04:55:48.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:47 smithi043 ceph-mon[30800]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-09-19T04:55:48.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:47 smithi043 ceph-mon[30800]: 1: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.c 2024-09-19T04:55:48.283 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:47 smithi043 ceph-mon[30800]: fsmap 2024-09-19T04:55:48.283 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:47 smithi043 ceph-mon[30800]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:55:48.283 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:47 smithi043 ceph-mon[30800]: mgrmap e13: x(active, since 40s) 2024-09-19T04:55:48.283 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:47 smithi043 ceph-mon[30800]: overall HEALTH_OK 2024-09-19T04:55:48.283 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:47 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:55:48.283 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:47 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:48.283 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:47 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:48.283 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:47 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:48.283 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:47 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:55:48.283 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:47 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:55:48.409 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:47 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:55:48.409 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:47 smithi151 ceph-mon[35425]: mon.a calling monitor election 2024-09-19T04:55:48.409 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:47 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:55:48.409 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:47 smithi151 ceph-mon[35425]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:55:48.410 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:47 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:55:48.410 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:47 smithi151 ceph-mon[35425]: mon.c calling monitor election 2024-09-19T04:55:48.410 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:47 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:55:48.410 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:47 smithi151 ceph-mon[35425]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:55:48.410 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:47 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:55:48.410 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:47 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:55:48.410 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:47 smithi151 ceph-mon[35425]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:55:48.410 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:47 smithi151 ceph-mon[35425]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T04:55:48.410 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:47 smithi151 ceph-mon[35425]: monmap epoch 2 2024-09-19T04:55:48.410 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:47 smithi151 ceph-mon[35425]: fsid 1456a810-7643-11ef-bceb-c7b262605968 2024-09-19T04:55:48.410 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:47 smithi151 ceph-mon[35425]: last_changed 2024-09-19T04:55:42.904907+0000 2024-09-19T04:55:48.410 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:47 smithi151 ceph-mon[35425]: created 2024-09-19T04:54:11.079879+0000 2024-09-19T04:55:48.411 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:47 smithi151 ceph-mon[35425]: min_mon_release 19 (squid) 2024-09-19T04:55:48.411 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:47 smithi151 ceph-mon[35425]: election_strategy: 1 2024-09-19T04:55:48.411 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:47 smithi151 ceph-mon[35425]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-09-19T04:55:48.411 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:47 smithi151 ceph-mon[35425]: 1: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.c 2024-09-19T04:55:48.411 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:47 smithi151 ceph-mon[35425]: fsmap 2024-09-19T04:55:48.411 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:47 smithi151 ceph-mon[35425]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:55:48.411 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:47 smithi151 ceph-mon[35425]: mgrmap e13: x(active, since 40s) 2024-09-19T04:55:48.411 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:47 smithi151 ceph-mon[35425]: overall HEALTH_OK 2024-09-19T04:55:48.411 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:47 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:55:48.411 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:47 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:48.411 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:47 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:48.411 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:47 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:48.412 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:47 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:55:48.412 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:47 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:55:48.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.483+0000 7f77242ec640 1 -- 172.21.15.151:0/3884701292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7708003530 msgr2=0x7f77080059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:48.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.483+0000 7f77242ec640 1 --2- 172.21.15.151:0/3884701292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7708003530 0x7f77080059c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f771c102af0 tx=0x7f771802c490 comp rx=0 tx=0).stop 2024-09-19T04:55:48.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.483+0000 7f77242ec640 1 -- 172.21.15.151:0/3884701292 shutdown_connections 2024-09-19T04:55:48.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.483+0000 7f77242ec640 1 --2- 172.21.15.151:0/3884701292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7708003530 0x7f77080059c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:48.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.483+0000 7f77242ec640 1 -- 172.21.15.151:0/3884701292 >> 172.21.15.151:0/3884701292 conn(0x7f771c0fd320 msgr2=0x7f771c0ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:48.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.483+0000 7f77242ec640 1 -- 172.21.15.151:0/3884701292 shutdown_connections 2024-09-19T04:55:48.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.483+0000 7f77242ec640 1 -- 172.21.15.151:0/3884701292 wait complete. 2024-09-19T04:55:48.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.485+0000 7f77242ec640 1 Processor -- start 2024-09-19T04:55:48.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.485+0000 7f77242ec640 1 -- start start 2024-09-19T04:55:48.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.485+0000 7f77242ec640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7708003530 0x7f771c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:48.488 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.485+0000 7f77242ec640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f771c074a30 0x7f771c074e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:48.488 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.485+0000 7f77242ec640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f771c109780 con 0x7f7708003530 2024-09-19T04:55:48.488 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.485+0000 7f77242ec640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f771c109600 con 0x7f771c074a30 2024-09-19T04:55:48.488 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.485+0000 7f7721887640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f771c074a30 0x7f771c074e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:48.488 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.485+0000 7f7721887640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f771c074a30 0x7f771c074e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43744/0 (socket says 172.21.15.151:43744) 2024-09-19T04:55:48.488 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.485+0000 7f7721887640 1 -- 172.21.15.151:0/2375773816 learned_addr learned my addr 172.21.15.151:0/2375773816 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-19T04:55:48.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.485+0000 7f7721887640 1 -- 172.21.15.151:0/2375773816 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f771c074a30 msgr2=0x7f771c074e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-19T04:55:48.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.485+0000 7f7721887640 1 -- 172.21.15.151:0/2375773816 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f771c074a30 msgr2=0x7f771c074e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T04:55:48.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.485+0000 7f7721887640 1 --2- 172.21.15.151:0/2375773816 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f771c074a30 0x7f771c074e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T04:55:48.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.485+0000 7f7721887640 1 --2- 172.21.15.151:0/2375773816 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f771c074a30 0x7f771c074e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:55:48.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.486+0000 7f7722088640 1 --2- 172.21.15.151:0/2375773816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7708003530 0x7f771c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:48.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.486+0000 7f7722088640 1 -- 172.21.15.151:0/2375773816 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f771c074a30 msgr2=0x7f771c074e90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:55:48.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.486+0000 7f7722088640 1 --2- 172.21.15.151:0/2375773816 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f771c074a30 0x7f771c074e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:48.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.486+0000 7f7722088640 1 -- 172.21.15.151:0/2375773816 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f771c0763e0 con 0x7f7708003530 2024-09-19T04:55:48.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.486+0000 7f7722088640 1 --2- 172.21.15.151:0/2375773816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7708003530 0x7f771c0744f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7718036040 tx=0x7f7718033680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:48.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.487+0000 7f77137fe640 1 -- 172.21.15.151:0/2375773816 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f7718032920 con 0x7f7708003530 2024-09-19T04:55:48.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.487+0000 7f77137fe640 1 -- 172.21.15.151:0/2375773816 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7718041070 con 0x7f7708003530 2024-09-19T04:55:48.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.487+0000 7f77242ec640 1 -- 172.21.15.151:0/2375773816 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f771c1a34d0 con 0x7f7708003530 2024-09-19T04:55:48.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.487+0000 7f77137fe640 1 -- 172.21.15.151:0/2375773816 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f7718004970 con 0x7f7708003530 2024-09-19T04:55:48.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.487+0000 7f77242ec640 1 -- 172.21.15.151:0/2375773816 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f771c1a3a60 con 0x7f7708003530 2024-09-19T04:55:48.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.488+0000 7f77137fe640 1 -- 172.21.15.151:0/2375773816 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f7718004c70 con 0x7f7708003530 2024-09-19T04:55:48.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.488+0000 7f77242ec640 1 -- 172.21.15.151:0/2375773816 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f771c102da0 con 0x7f7708003530 2024-09-19T04:55:48.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.488+0000 7f77137fe640 1 --2- 172.21.15.151:0/2375773816 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f7700039040 0x7f770003b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:48.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.488+0000 7f77137fe640 1 -- 172.21.15.151:0/2375773816 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f771806e720 con 0x7f7708003530 2024-09-19T04:55:48.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.489+0000 7f7721887640 1 --2- 172.21.15.151:0/2375773816 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f7700039040 0x7f770003b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:48.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.489+0000 7f7721887640 1 --2- 172.21.15.151:0/2375773816 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f7700039040 0x7f770003b500 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f770c009100 tx=0x7f770c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:48.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.494+0000 7f77137fe640 1 -- 172.21.15.151:0/2375773816 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f771803c430 con 0x7f7708003530 2024-09-19T04:55:48.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.724+0000 7f77242ec640 1 -- 172.21.15.151:0/2375773816 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f771c10a930 con 0x7f7708003530 2024-09-19T04:55:48.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.725+0000 7f77137fe640 1 -- 172.21.15.151:0/2375773816 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f7718044360 con 0x7f7708003530 2024-09-19T04:55:48.727 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T04:55:48.727 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":2,"fsid":"1456a810-7643-11ef-bceb-c7b262605968","modified":"2024-09-19T04:55:42.904907Z","created":"2024-09-19T04:54:11.079879Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:3300","nonce":0},{"type":"v1","addr":"172.21.15.151:6789","nonce":0}]},"addr":"172.21.15.151:6789/0","public_addr":"172.21.15.151:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T04:55:48.727 INFO:teuthology.orchestra.run.smithi151.stderr:dumped monmap epoch 2 2024-09-19T04:55:48.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.727+0000 7f77242ec640 1 -- 172.21.15.151:0/2375773816 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f7700039040 msgr2=0x7f770003b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:48.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.727+0000 7f77242ec640 1 --2- 172.21.15.151:0/2375773816 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f7700039040 0x7f770003b500 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f770c009100 tx=0x7f770c008040 comp rx=0 tx=0).stop 2024-09-19T04:55:48.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.727+0000 7f77242ec640 1 -- 172.21.15.151:0/2375773816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7708003530 msgr2=0x7f771c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:48.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.727+0000 7f77242ec640 1 --2- 172.21.15.151:0/2375773816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7708003530 0x7f771c0744f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7718036040 tx=0x7f7718033680 comp rx=0 tx=0).stop 2024-09-19T04:55:48.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.727+0000 7f77242ec640 1 -- 172.21.15.151:0/2375773816 shutdown_connections 2024-09-19T04:55:48.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.727+0000 7f77242ec640 1 --2- 172.21.15.151:0/2375773816 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f7700039040 0x7f770003b500 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:48.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.727+0000 7f77242ec640 1 --2- 172.21.15.151:0/2375773816 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f771c074a30 0x7f771c074e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:48.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.727+0000 7f77242ec640 1 --2- 172.21.15.151:0/2375773816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7708003530 0x7f771c0744f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:48.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.727+0000 7f77242ec640 1 -- 172.21.15.151:0/2375773816 >> 172.21.15.151:0/2375773816 conn(0x7f771c0fd320 msgr2=0x7f771c0ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:48.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.727+0000 7f77242ec640 1 -- 172.21.15.151:0/2375773816 shutdown_connections 2024-09-19T04:55:48.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:48.727+0000 7f77242ec640 1 -- 172.21.15.151:0/2375773816 wait complete. 2024-09-19T04:55:49.026 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:48 smithi151 ceph-mon[35425]: Deploying daemon mon.b on smithi096 2024-09-19T04:55:49.026 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:48 smithi151 ceph-mon[35425]: from='client.? 172.21.15.151:0/2375773816' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:55:49.027 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:48 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:55:49.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:48 smithi043 ceph-mon[30800]: Deploying daemon mon.b on smithi096 2024-09-19T04:55:49.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:48 smithi043 ceph-mon[30800]: from='client.? 172.21.15.151:0/2375773816' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:55:49.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:48 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:55:50.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:49 smithi043 ceph-mon[30800]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:55:50.314 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T04:55:50.314 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T04:55:50.342 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:49 smithi151 ceph-mon[35425]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:55:50.566 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:55:51.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.075+0000 7ff1e9349640 1 -- 172.21.15.151:0/2276300053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1e4106650 msgr2=0x7ff1e4106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:51.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.075+0000 7ff1e9349640 1 --2- 172.21.15.151:0/2276300053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1e4106650 0x7ff1e4106a30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4009880 tx=0x7ff1d402a5d0 comp rx=0 tx=0).stop 2024-09-19T04:55:51.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.075+0000 7ff1e9349640 1 -- 172.21.15.151:0/2276300053 shutdown_connections 2024-09-19T04:55:51.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.075+0000 7ff1e9349640 1 --2- 172.21.15.151:0/2276300053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1e4106650 0x7ff1e4106a30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:51.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.076+0000 7ff1e9349640 1 -- 172.21.15.151:0/2276300053 >> 172.21.15.151:0/2276300053 conn(0x7ff1e40fc280 msgr2=0x7ff1e40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:51.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.076+0000 7ff1e9349640 1 -- 172.21.15.151:0/2276300053 shutdown_connections 2024-09-19T04:55:51.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.076+0000 7ff1e9349640 1 -- 172.21.15.151:0/2276300053 wait complete. 2024-09-19T04:55:51.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.076+0000 7ff1e9349640 1 Processor -- start 2024-09-19T04:55:51.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.076+0000 7ff1e9349640 1 -- start start 2024-09-19T04:55:51.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.077+0000 7ff1e9349640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1e4106650 0x7ff1e4198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:51.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.077+0000 7ff1e9349640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff1e4198780 0x7ff1e4198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:51.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.077+0000 7ff1e9349640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e410ac80 con 0x7ff1e4106650 2024-09-19T04:55:51.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.077+0000 7ff1e9349640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e410ab00 con 0x7ff1e4198780 2024-09-19T04:55:51.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.077+0000 7ff1e27fc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff1e4198780 0x7ff1e4198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:51.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.077+0000 7ff1e27fc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff1e4198780 0x7ff1e4198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43750/0 (socket says 172.21.15.151:43750) 2024-09-19T04:55:51.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.077+0000 7ff1e27fc640 1 -- 172.21.15.151:0/1343372102 learned_addr learned my addr 172.21.15.151:0/1343372102 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-19T04:55:51.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.077+0000 7ff1e27fc640 1 -- 172.21.15.151:0/1343372102 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff1e4198780 msgr2=0x7ff1e4198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-19T04:55:51.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.077+0000 7ff1e27fc640 1 -- 172.21.15.151:0/1343372102 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff1e4198780 msgr2=0x7ff1e4198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T04:55:51.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.077+0000 7ff1e27fc640 1 --2- 172.21.15.151:0/1343372102 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff1e4198780 0x7ff1e4198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T04:55:51.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.077+0000 7ff1e2ffd640 1 --2- 172.21.15.151:0/1343372102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1e4106650 0x7ff1e4198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:51.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.077+0000 7ff1e27fc640 1 --2- 172.21.15.151:0/1343372102 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff1e4198780 0x7ff1e4198be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:55:51.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.077+0000 7ff1e2ffd640 1 -- 172.21.15.151:0/1343372102 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff1e4198780 msgr2=0x7ff1e4198be0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:55:51.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.077+0000 7ff1e2ffd640 1 --2- 172.21.15.151:0/1343372102 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff1e4198780 0x7ff1e4198be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:51.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.077+0000 7ff1e2ffd640 1 -- 172.21.15.151:0/1343372102 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1e41a12b0 con 0x7ff1e4106650 2024-09-19T04:55:51.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.078+0000 7ff1e2ffd640 1 --2- 172.21.15.151:0/1343372102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1e4106650 0x7ff1e4198240 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4000c00 tx=0x7ff1d4032c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:51.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.078+0000 7ff1dbfff640 1 -- 172.21.15.151:0/1343372102 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7ff1d4035070 con 0x7ff1e4106650 2024-09-19T04:55:51.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.078+0000 7ff1dbfff640 1 -- 172.21.15.151:0/1343372102 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1d4032e40 con 0x7ff1e4106650 2024-09-19T04:55:51.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.078+0000 7ff1e9349640 1 -- 172.21.15.151:0/1343372102 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1e41a1480 con 0x7ff1e4106650 2024-09-19T04:55:51.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.078+0000 7ff1dbfff640 1 -- 172.21.15.151:0/1343372102 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7ff1d403bc30 con 0x7ff1e4106650 2024-09-19T04:55:51.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.078+0000 7ff1e9349640 1 -- 172.21.15.151:0/1343372102 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1e41a1930 con 0x7ff1e4106650 2024-09-19T04:55:51.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.079+0000 7ff1e9349640 1 -- 172.21.15.151:0/1343372102 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1e4101cc0 con 0x7ff1e4106650 2024-09-19T04:55:51.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.079+0000 7ff1dbfff640 1 -- 172.21.15.151:0/1343372102 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7ff1d4042030 con 0x7ff1e4106650 2024-09-19T04:55:51.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.079+0000 7ff1dbfff640 1 --2- 172.21.15.151:0/1343372102 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff1d0039040 0x7ff1d003b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:51.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.079+0000 7ff1dbfff640 1 -- 172.21.15.151:0/1343372102 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff1d4040070 con 0x7ff1e4106650 2024-09-19T04:55:51.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.080+0000 7ff1e27fc640 1 --2- 172.21.15.151:0/1343372102 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff1d0039040 0x7ff1d003b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:51.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.080+0000 7ff1e27fc640 1 --2- 172.21.15.151:0/1343372102 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff1d0039040 0x7ff1d003b500 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc009100 tx=0x7ff1cc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:51.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.085+0000 7ff1dbfff640 1 -- 172.21.15.151:0/1343372102 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff1d404a260 con 0x7ff1e4106650 2024-09-19T04:55:51.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.315+0000 7ff1e9349640 1 -- 172.21.15.151:0/1343372102 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7ff1e41a1be0 con 0x7ff1e4106650 2024-09-19T04:55:51.317 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:51 smithi151 ceph-mon[35425]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:55:51.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.315+0000 7ff1dbfff640 1 -- 172.21.15.151:0/1343372102 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7ff1d403bdd0 con 0x7ff1e4106650 2024-09-19T04:55:51.317 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T04:55:51.317 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":2,"fsid":"1456a810-7643-11ef-bceb-c7b262605968","modified":"2024-09-19T04:55:42.904907Z","created":"2024-09-19T04:54:11.079879Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:3300","nonce":0},{"type":"v1","addr":"172.21.15.151:6789","nonce":0}]},"addr":"172.21.15.151:6789/0","public_addr":"172.21.15.151:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T04:55:51.317 INFO:teuthology.orchestra.run.smithi151.stderr:dumped monmap epoch 2 2024-09-19T04:55:51.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.317+0000 7ff1e9349640 1 -- 172.21.15.151:0/1343372102 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff1d0039040 msgr2=0x7ff1d003b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:51.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.317+0000 7ff1e9349640 1 --2- 172.21.15.151:0/1343372102 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff1d0039040 0x7ff1d003b500 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc009100 tx=0x7ff1cc008040 comp rx=0 tx=0).stop 2024-09-19T04:55:51.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.317+0000 7ff1e9349640 1 -- 172.21.15.151:0/1343372102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1e4106650 msgr2=0x7ff1e4198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:51.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.317+0000 7ff1e9349640 1 --2- 172.21.15.151:0/1343372102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1e4106650 0x7ff1e4198240 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4000c00 tx=0x7ff1d4032c50 comp rx=0 tx=0).stop 2024-09-19T04:55:51.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.317+0000 7ff1e9349640 1 -- 172.21.15.151:0/1343372102 shutdown_connections 2024-09-19T04:55:51.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.317+0000 7ff1e9349640 1 --2- 172.21.15.151:0/1343372102 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff1d0039040 0x7ff1d003b500 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:51.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.317+0000 7ff1e9349640 1 --2- 172.21.15.151:0/1343372102 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff1e4198780 0x7ff1e4198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:51.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.317+0000 7ff1e9349640 1 --2- 172.21.15.151:0/1343372102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1e4106650 0x7ff1e4198240 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:51.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.317+0000 7ff1e9349640 1 -- 172.21.15.151:0/1343372102 >> 172.21.15.151:0/1343372102 conn(0x7ff1e40fc280 msgr2=0x7ff1e40fc660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:51.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.318+0000 7ff1e9349640 1 -- 172.21.15.151:0/1343372102 shutdown_connections 2024-09-19T04:55:51.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:51.318+0000 7ff1e9349640 1 -- 172.21.15.151:0/1343372102 wait complete. 2024-09-19T04:55:51.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:51 smithi043 ceph-mon[30800]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:55:51.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:51 smithi096 systemd[1]: Started Ceph mon.b for 1456a810-7643-11ef-bceb-c7b262605968. 2024-09-19T04:55:52.920 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T04:55:52.921 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T04:55:53.174 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:55:56.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.567+0000 7faf3a7ec640 1 -- 172.21.15.151:0/1640920405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf1c003530 msgr2=0x7faf1c0059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:56.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.567+0000 7faf3a7ec640 1 --2- 172.21.15.151:0/1640920405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf1c003530 0x7faf1c0059c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7faf34102af0 tx=0x7faf2402bce0 comp rx=0 tx=0).stop 2024-09-19T04:55:56.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.567+0000 7faf3a7ec640 1 -- 172.21.15.151:0/1640920405 shutdown_connections 2024-09-19T04:55:56.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.567+0000 7faf3a7ec640 1 --2- 172.21.15.151:0/1640920405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf1c003530 0x7faf1c0059c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:56.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.567+0000 7faf3a7ec640 1 -- 172.21.15.151:0/1640920405 >> 172.21.15.151:0/1640920405 conn(0x7faf340fd320 msgr2=0x7faf340ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:56.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.567+0000 7faf3a7ec640 1 -- 172.21.15.151:0/1640920405 shutdown_connections 2024-09-19T04:55:56.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.567+0000 7faf3a7ec640 1 -- 172.21.15.151:0/1640920405 wait complete. 2024-09-19T04:55:56.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.568+0000 7faf3a7ec640 1 Processor -- start 2024-09-19T04:55:56.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.568+0000 7faf3a7ec640 1 -- start start 2024-09-19T04:55:56.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.569+0000 7faf3a7ec640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf1c003530 0x7faf340744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:56.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.569+0000 7faf3a7ec640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf34074a30 0x7faf34074e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:56.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.569+0000 7faf3a7ec640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf341a3540 0x7faf341a5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:56.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.569+0000 7faf3a7ec640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faf34109720 con 0x7faf34074a30 2024-09-19T04:55:56.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.569+0000 7faf3a7ec640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faf341095a0 con 0x7faf341a3540 2024-09-19T04:55:56.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.569+0000 7faf3a7ec640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faf341098a0 con 0x7faf1c003530 2024-09-19T04:55:56.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.569+0000 7faf33fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf1c003530 0x7faf340744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:56.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.569+0000 7faf33fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf1c003530 0x7faf340744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:50158/0 (socket says 172.21.15.151:50158) 2024-09-19T04:55:56.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.569+0000 7faf33fff640 1 -- 172.21.15.151:0/2109579671 learned_addr learned my addr 172.21.15.151:0/2109579671 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-19T04:55:56.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.569+0000 7faf337fe640 1 --2- 172.21.15.151:0/2109579671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf34074a30 0x7faf34074e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:56.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.569+0000 7faf33fff640 1 -- 172.21.15.151:0/2109579671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf1c003530 msgr2=0x7faf340744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-19T04:55:56.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.569+0000 7faf38d89640 1 --2- 172.21.15.151:0/2109579671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf341a3540 0x7faf341a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:56.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.569+0000 7faf33fff640 1 -- 172.21.15.151:0/2109579671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf1c003530 msgr2=0x7faf340744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T04:55:56.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.569+0000 7faf33fff640 1 --2- 172.21.15.151:0/2109579671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf1c003530 0x7faf340744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T04:55:56.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.569+0000 7faf33fff640 1 --2- 172.21.15.151:0/2109579671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf1c003530 0x7faf340744f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:55:56.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.569+0000 7faf38d89640 1 -- 172.21.15.151:0/2109579671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf341a3540 msgr2=0x7faf341a5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-19T04:55:56.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.569+0000 7faf38d89640 1 -- 172.21.15.151:0/2109579671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf341a3540 msgr2=0x7faf341a5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T04:55:56.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.569+0000 7faf38d89640 1 --2- 172.21.15.151:0/2109579671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf341a3540 0x7faf341a5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T04:55:56.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.569+0000 7faf38d89640 1 --2- 172.21.15.151:0/2109579671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf341a3540 0x7faf341a5930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:55:56.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.569+0000 7faf337fe640 1 -- 172.21.15.151:0/2109579671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf1c003530 msgr2=0x7faf340744f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:55:56.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.569+0000 7faf337fe640 1 --2- 172.21.15.151:0/2109579671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf1c003530 0x7faf340744f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:56.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.569+0000 7faf337fe640 1 -- 172.21.15.151:0/2109579671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf341a3540 msgr2=0x7faf341a5930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:55:56.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.569+0000 7faf337fe640 1 --2- 172.21.15.151:0/2109579671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf341a3540 0x7faf341a5930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:56.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.569+0000 7faf337fe640 1 -- 172.21.15.151:0/2109579671 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf341a6000 con 0x7faf34074a30 2024-09-19T04:55:56.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.570+0000 7faf337fe640 1 --2- 172.21.15.151:0/2109579671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf34074a30 0x7faf34074e90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7faf2800b450 tx=0x7faf2800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:56.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.570+0000 7faf317fa640 1 -- 172.21.15.151:0/2109579671 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf28004230 con 0x7faf34074a30 2024-09-19T04:55:56.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.570+0000 7faf317fa640 1 -- 172.21.15.151:0/2109579671 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf280043d0 con 0x7faf34074a30 2024-09-19T04:55:56.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.570+0000 7faf3a7ec640 1 -- 172.21.15.151:0/2109579671 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf341a62f0 con 0x7faf34074a30 2024-09-19T04:55:56.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.570+0000 7faf317fa640 1 -- 172.21.15.151:0/2109579671 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf28003cb0 con 0x7faf34074a30 2024-09-19T04:55:56.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.570+0000 7faf3a7ec640 1 -- 172.21.15.151:0/2109579671 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf341a6830 con 0x7faf34074a30 2024-09-19T04:55:56.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.571+0000 7faf317fa640 1 -- 172.21.15.151:0/2109579671 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7faf2800f050 con 0x7faf34074a30 2024-09-19T04:55:56.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.571+0000 7faf3a7ec640 1 -- 172.21.15.151:0/2109579671 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf3410a8d0 con 0x7faf34074a30 2024-09-19T04:55:56.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.571+0000 7faf317fa640 1 --2- 172.21.15.151:0/2109579671 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7faf20039730 0x7faf2003bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:56.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.571+0000 7faf317fa640 1 -- 172.21.15.151:0/2109579671 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7faf2804c1b0 con 0x7faf34074a30 2024-09-19T04:55:56.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.572+0000 7faf33fff640 1 --2- 172.21.15.151:0/2109579671 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7faf20039730 0x7faf2003bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:56.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.572+0000 7faf33fff640 1 --2- 172.21.15.151:0/2109579671 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7faf20039730 0x7faf2003bbf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7faf24005940 tx=0x7faf240058b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:56.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.577+0000 7faf317fa640 1 -- 172.21.15.151:0/2109579671 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7faf2800d040 con 0x7faf34074a30 2024-09-19T04:55:56.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.814+0000 7faf3a7ec640 1 -- 172.21.15.151:0/2109579671 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7faf3402d7c0 con 0x7faf34074a30 2024-09-19T04:55:56.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.815+0000 7faf317fa640 1 -- 172.21.15.151:0/2109579671 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7faf2800d540 con 0x7faf34074a30 2024-09-19T04:55:56.816 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T04:55:56.817 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":3,"fsid":"1456a810-7643-11ef-bceb-c7b262605968","modified":"2024-09-19T04:55:51.536852Z","created":"2024-09-19T04:54:11.079879Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:3300","nonce":0},{"type":"v1","addr":"172.21.15.151:6789","nonce":0}]},"addr":"172.21.15.151:6789/0","public_addr":"172.21.15.151:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:3300","nonce":0},{"type":"v1","addr":"172.21.15.96:6789","nonce":0}]},"addr":"172.21.15.96:6789/0","public_addr":"172.21.15.96:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T04:55:56.817 INFO:teuthology.orchestra.run.smithi151.stderr:dumped monmap epoch 3 2024-09-19T04:55:56.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.817+0000 7faf3a7ec640 1 -- 172.21.15.151:0/2109579671 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7faf20039730 msgr2=0x7faf2003bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:56.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.817+0000 7faf3a7ec640 1 --2- 172.21.15.151:0/2109579671 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7faf20039730 0x7faf2003bbf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7faf24005940 tx=0x7faf240058b0 comp rx=0 tx=0).stop 2024-09-19T04:55:56.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.817+0000 7faf3a7ec640 1 -- 172.21.15.151:0/2109579671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf34074a30 msgr2=0x7faf34074e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:56.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.817+0000 7faf3a7ec640 1 --2- 172.21.15.151:0/2109579671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf34074a30 0x7faf34074e90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7faf2800b450 tx=0x7faf2800b910 comp rx=0 tx=0).stop 2024-09-19T04:55:56.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.817+0000 7faf3a7ec640 1 -- 172.21.15.151:0/2109579671 shutdown_connections 2024-09-19T04:55:56.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.817+0000 7faf3a7ec640 1 --2- 172.21.15.151:0/2109579671 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7faf20039730 0x7faf2003bbf0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:56.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.817+0000 7faf3a7ec640 1 --2- 172.21.15.151:0/2109579671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf341a3540 0x7faf341a5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:56.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.817+0000 7faf3a7ec640 1 --2- 172.21.15.151:0/2109579671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf34074a30 0x7faf34074e90 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:56.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.817+0000 7faf3a7ec640 1 --2- 172.21.15.151:0/2109579671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf1c003530 0x7faf340744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:56.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.817+0000 7faf3a7ec640 1 -- 172.21.15.151:0/2109579671 >> 172.21.15.151:0/2109579671 conn(0x7faf340fd320 msgr2=0x7faf340ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:56.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.817+0000 7faf3a7ec640 1 -- 172.21.15.151:0/2109579671 shutdown_connections 2024-09-19T04:55:56.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:55:56.817+0000 7faf3a7ec640 1 -- 172.21.15.151:0/2109579671 wait complete. 2024-09-19T04:55:56.819 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:56 smithi151 ceph-mon[35425]: mon.c calling monitor election 2024-09-19T04:55:56.819 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:56 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:55:56.819 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:56 smithi151 ceph-mon[35425]: mon.a calling monitor election 2024-09-19T04:55:56.820 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:56 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:55:56.820 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:56 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:55:56.820 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:56 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:55:56.820 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:56 smithi151 ceph-mon[35425]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:55:56.820 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:56 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:55:56.820 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:56 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:55:56.820 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:56 smithi151 ceph-mon[35425]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:55:56.820 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:56 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:55:56.820 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:56 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:55:56.820 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:56 smithi151 ceph-mon[35425]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T04:55:56.820 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:56 smithi151 ceph-mon[35425]: monmap epoch 3 2024-09-19T04:55:56.820 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:56 smithi151 ceph-mon[35425]: fsid 1456a810-7643-11ef-bceb-c7b262605968 2024-09-19T04:55:56.820 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:56 smithi151 ceph-mon[35425]: last_changed 2024-09-19T04:55:51.536852+0000 2024-09-19T04:55:56.820 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:56 smithi151 ceph-mon[35425]: created 2024-09-19T04:54:11.079879+0000 2024-09-19T04:55:56.821 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:56 smithi151 ceph-mon[35425]: min_mon_release 19 (squid) 2024-09-19T04:55:56.821 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:56 smithi151 ceph-mon[35425]: election_strategy: 1 2024-09-19T04:55:56.821 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:56 smithi151 ceph-mon[35425]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-09-19T04:55:56.821 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:56 smithi151 ceph-mon[35425]: 1: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.c 2024-09-19T04:55:56.821 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:56 smithi151 ceph-mon[35425]: 2: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.b 2024-09-19T04:55:56.821 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:56 smithi151 ceph-mon[35425]: fsmap 2024-09-19T04:55:56.821 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:56 smithi151 ceph-mon[35425]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:55:56.821 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:56 smithi151 ceph-mon[35425]: mgrmap e13: x(active, since 49s) 2024-09-19T04:55:56.821 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:56 smithi151 ceph-mon[35425]: overall HEALTH_OK 2024-09-19T04:55:56.821 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:56 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:56.821 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:56 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:56.821 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:56 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:56.821 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:56 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:56.937 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:56 smithi043 ceph-mon[30800]: mon.c calling monitor election 2024-09-19T04:55:56.937 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:56 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:55:56.937 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:56 smithi043 ceph-mon[30800]: mon.a calling monitor election 2024-09-19T04:55:56.937 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:56 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:55:56.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:56 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:55:56.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:56 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:55:56.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:56 smithi043 ceph-mon[30800]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:55:56.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:56 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:55:56.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:56 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:55:56.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:56 smithi043 ceph-mon[30800]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:55:56.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:56 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:55:56.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:56 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:55:56.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:56 smithi043 ceph-mon[30800]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T04:55:56.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:56 smithi043 ceph-mon[30800]: monmap epoch 3 2024-09-19T04:55:56.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:56 smithi043 ceph-mon[30800]: fsid 1456a810-7643-11ef-bceb-c7b262605968 2024-09-19T04:55:56.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:56 smithi043 ceph-mon[30800]: last_changed 2024-09-19T04:55:51.536852+0000 2024-09-19T04:55:56.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:56 smithi043 ceph-mon[30800]: created 2024-09-19T04:54:11.079879+0000 2024-09-19T04:55:56.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:56 smithi043 ceph-mon[30800]: min_mon_release 19 (squid) 2024-09-19T04:55:56.939 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:56 smithi043 ceph-mon[30800]: election_strategy: 1 2024-09-19T04:55:56.939 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:56 smithi043 ceph-mon[30800]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-09-19T04:55:56.939 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:56 smithi043 ceph-mon[30800]: 1: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.c 2024-09-19T04:55:56.939 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:56 smithi043 ceph-mon[30800]: 2: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.b 2024-09-19T04:55:56.939 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:56 smithi043 ceph-mon[30800]: fsmap 2024-09-19T04:55:56.939 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:56 smithi043 ceph-mon[30800]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:55:56.939 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:56 smithi043 ceph-mon[30800]: mgrmap e13: x(active, since 49s) 2024-09-19T04:55:56.939 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:56 smithi043 ceph-mon[30800]: overall HEALTH_OK 2024-09-19T04:55:56.939 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:56 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:56.939 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:56 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:56.939 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:56 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:56.939 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:56 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:57.431 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-19T04:55:57.431 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-19T04:55:57.700 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:57 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:55:57.700 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:57 smithi151 ceph-mon[35425]: from='client.? 172.21.15.151:0/2109579671' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:55:57.700 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:57 smithi151 ceph-mon[35425]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:55:57.700 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:57 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:57.701 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:57 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:55:57.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:57 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:55:57.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:57 smithi043 ceph-mon[30800]: from='client.? 172.21.15.151:0/2109579671' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:55:57.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:57 smithi043 ceph-mon[30800]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:55:57.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:57 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:57.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:57 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:55:57.834 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:55:58.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.362+0000 7f03b1c00640 1 -- 172.21.15.43:0/931574994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03ac074590 msgr2=0x7f03ac074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:58.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.362+0000 7f03b1c00640 1 --2- 172.21.15.43:0/931574994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03ac074590 0x7f03ac074970 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f039c009880 tx=0x7f039c02a670 comp rx=0 tx=0).stop 2024-09-19T04:55:58.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.362+0000 7f03b1c00640 1 -- 172.21.15.43:0/931574994 shutdown_connections 2024-09-19T04:55:58.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.362+0000 7f03b1c00640 1 --2- 172.21.15.43:0/931574994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03ac074590 0x7f03ac074970 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:58.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.362+0000 7f03b1c00640 1 -- 172.21.15.43:0/931574994 >> 172.21.15.43:0/931574994 conn(0x7f03ac0fd0a0 msgr2=0x7f03ac0ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:58.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.362+0000 7f03b1c00640 1 -- 172.21.15.43:0/931574994 shutdown_connections 2024-09-19T04:55:58.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.362+0000 7f03b1c00640 1 -- 172.21.15.43:0/931574994 wait complete. 2024-09-19T04:55:58.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.363+0000 7f03b1c00640 1 Processor -- start 2024-09-19T04:55:58.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.363+0000 7f03b1c00640 1 -- start start 2024-09-19T04:55:58.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.363+0000 7f03b1c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03ac074590 0x7f03ac073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:58.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.363+0000 7f03b1c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03ac071ce0 0x7f03ac072140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:58.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.363+0000 7f03b1c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03ac072680 0x7f03ac1b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:58.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.363+0000 7f03b1c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac1120c0 con 0x7f03ac072680 2024-09-19T04:55:58.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.364+0000 7f03b1c00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac111f40 con 0x7f03ac074590 2024-09-19T04:55:58.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.364+0000 7f03b1c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac112240 con 0x7f03ac071ce0 2024-09-19T04:55:58.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.364+0000 7f03ab400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03ac071ce0 0x7f03ac072140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:58.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.364+0000 7f03ab400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03ac071ce0 0x7f03ac072140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:44060/0 (socket says 172.21.15.43:44060) 2024-09-19T04:55:58.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.364+0000 7f03ab400640 1 -- 172.21.15.43:0/2579061060 learned_addr learned my addr 172.21.15.43:0/2579061060 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:55:58.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.364+0000 7f03abe00640 1 --2- 172.21.15.43:0/2579061060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03ac074590 0x7f03ac073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:58.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.364+0000 7f03b1200640 1 --2- 172.21.15.43:0/2579061060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03ac072680 0x7f03ac1b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:58.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.364+0000 7f03abe00640 1 -- 172.21.15.43:0/2579061060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03ac074590 msgr2=0x7f03ac073bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-19T04:55:58.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.364+0000 7f03abe00640 1 -- 172.21.15.43:0/2579061060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03ac074590 msgr2=0x7f03ac073bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T04:55:58.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.364+0000 7f03abe00640 1 --2- 172.21.15.43:0/2579061060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03ac074590 0x7f03ac073bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T04:55:58.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.364+0000 7f03abe00640 1 --2- 172.21.15.43:0/2579061060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03ac074590 0x7f03ac073bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:55:58.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.364+0000 7f03ab400640 1 -- 172.21.15.43:0/2579061060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03ac074590 msgr2=0x7f03ac073bd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:55:58.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.365+0000 7f03ab400640 1 --2- 172.21.15.43:0/2579061060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03ac074590 0x7f03ac073bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:58.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.365+0000 7f03ab400640 1 -- 172.21.15.43:0/2579061060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03ac072680 msgr2=0x7f03ac1b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:58.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.365+0000 7f03ab400640 1 --2- 172.21.15.43:0/2579061060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03ac072680 0x7f03ac1b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:58.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.365+0000 7f03ab400640 1 -- 172.21.15.43:0/2579061060 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03ac105640 con 0x7f03ac071ce0 2024-09-19T04:55:58.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.365+0000 7f03ab400640 1 --2- 172.21.15.43:0/2579061060 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03ac071ce0 0x7f03ac072140 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f039800b520 tx=0x7f039800b9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:58.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.365+0000 7f03a8c00640 1 -- 172.21.15.43:0/2579061060 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0398004230 con 0x7f03ac071ce0 2024-09-19T04:55:58.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.365+0000 7f03a8c00640 1 -- 172.21.15.43:0/2579061060 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03980043d0 con 0x7f03ac071ce0 2024-09-19T04:55:58.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.365+0000 7f03a8c00640 1 -- 172.21.15.43:0/2579061060 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0398003d60 con 0x7f03ac071ce0 2024-09-19T04:55:58.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.365+0000 7f03b1c00640 1 -- 172.21.15.43:0/2579061060 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03ac105930 con 0x7f03ac071ce0 2024-09-19T04:55:58.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.366+0000 7f03b1c00640 1 -- 172.21.15.43:0/2579061060 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03ac105e20 con 0x7f03ac071ce0 2024-09-19T04:55:58.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.367+0000 7f03b1c00640 1 -- 172.21.15.43:0/2579061060 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03ac0749f0 con 0x7f03ac071ce0 2024-09-19T04:55:58.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.367+0000 7f03a8c00640 1 -- 172.21.15.43:0/2579061060 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f039800e070 con 0x7f03ac071ce0 2024-09-19T04:55:58.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.368+0000 7f03a8c00640 1 --2- 172.21.15.43:0/2579061060 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f0390039730 0x7f039003bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:55:58.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.368+0000 7f03a8c00640 1 -- 172.21.15.43:0/2579061060 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f039804bdc0 con 0x7f03ac071ce0 2024-09-19T04:55:58.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.368+0000 7f03abe00640 1 --2- 172.21.15.43:0/2579061060 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f0390039730 0x7f039003bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:55:58.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.369+0000 7f03abe00640 1 --2- 172.21.15.43:0/2579061060 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f0390039730 0x7f039003bbf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f039c005ab0 tx=0x7f039c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:55:58.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.373+0000 7f03a8c00640 1 -- 172.21.15.43:0/2579061060 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0398014020 con 0x7f03ac071ce0 2024-09-19T04:55:58.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.565+0000 7f03b1c00640 1 -- 172.21.15.43:0/2579061060 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f03ac106140 con 0x7f03ac071ce0 2024-09-19T04:55:58.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.565+0000 7f03a8c00640 1 -- 172.21.15.43:0/2579061060 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f0398010690 con 0x7f03ac071ce0 2024-09-19T04:55:58.568 INFO:teuthology.orchestra.run.smithi043.stdout:# minimal ceph.conf for 1456a810-7643-11ef-bceb-c7b262605968 2024-09-19T04:55:58.568 INFO:teuthology.orchestra.run.smithi043.stdout:[global] 2024-09-19T04:55:58.568 INFO:teuthology.orchestra.run.smithi043.stdout: fsid = 1456a810-7643-11ef-bceb-c7b262605968 2024-09-19T04:55:58.568 INFO:teuthology.orchestra.run.smithi043.stdout: mon_host = [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] 2024-09-19T04:55:58.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.568+0000 7f03b1c00640 1 -- 172.21.15.43:0/2579061060 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f0390039730 msgr2=0x7f039003bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:58.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.568+0000 7f03b1c00640 1 --2- 172.21.15.43:0/2579061060 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f0390039730 0x7f039003bbf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f039c005ab0 tx=0x7f039c005a20 comp rx=0 tx=0).stop 2024-09-19T04:55:58.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.568+0000 7f03b1c00640 1 -- 172.21.15.43:0/2579061060 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03ac071ce0 msgr2=0x7f03ac072140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:55:58.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.568+0000 7f03b1c00640 1 --2- 172.21.15.43:0/2579061060 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03ac071ce0 0x7f03ac072140 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f039800b520 tx=0x7f039800b9e0 comp rx=0 tx=0).stop 2024-09-19T04:55:58.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.569+0000 7f03b1c00640 1 -- 172.21.15.43:0/2579061060 shutdown_connections 2024-09-19T04:55:58.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.569+0000 7f03b1c00640 1 --2- 172.21.15.43:0/2579061060 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f0390039730 0x7f039003bbf0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:58.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.569+0000 7f03b1c00640 1 --2- 172.21.15.43:0/2579061060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03ac072680 0x7f03ac1b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:58.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.569+0000 7f03b1c00640 1 --2- 172.21.15.43:0/2579061060 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03ac071ce0 0x7f03ac072140 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:58.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.569+0000 7f03b1c00640 1 --2- 172.21.15.43:0/2579061060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03ac074590 0x7f03ac073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:55:58.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.569+0000 7f03b1c00640 1 -- 172.21.15.43:0/2579061060 >> 172.21.15.43:0/2579061060 conn(0x7f03ac0fd0a0 msgr2=0x7f03ac0fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:55:58.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.569+0000 7f03b1c00640 1 -- 172.21.15.43:0/2579061060 shutdown_connections 2024-09-19T04:55:58.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:55:58.569+0000 7f03b1c00640 1 -- 172.21.15.43:0/2579061060 wait complete. 2024-09-19T04:55:58.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:58 smithi096 ceph-mon[35085]: mon.c calling monitor election 2024-09-19T04:55:58.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:58 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:55:58.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:58 smithi096 ceph-mon[35085]: mon.a calling monitor election 2024-09-19T04:55:58.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:58 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:55:58.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:58 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:55:58.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:58 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:55:58.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:58 smithi096 ceph-mon[35085]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:55:58.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:58 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:55:58.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:58 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:55:58.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:58 smithi096 ceph-mon[35085]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:55:58.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:58 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:55:58.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:58 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:55:58.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:58 smithi096 ceph-mon[35085]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T04:55:58.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:58 smithi096 ceph-mon[35085]: monmap epoch 3 2024-09-19T04:55:58.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:58 smithi096 ceph-mon[35085]: fsid 1456a810-7643-11ef-bceb-c7b262605968 2024-09-19T04:55:58.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:58 smithi096 ceph-mon[35085]: last_changed 2024-09-19T04:55:51.536852+0000 2024-09-19T04:55:58.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:58 smithi096 ceph-mon[35085]: created 2024-09-19T04:54:11.079879+0000 2024-09-19T04:55:58.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:58 smithi096 ceph-mon[35085]: min_mon_release 19 (squid) 2024-09-19T04:55:58.975 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:58 smithi096 ceph-mon[35085]: election_strategy: 1 2024-09-19T04:55:58.975 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:58 smithi096 ceph-mon[35085]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-09-19T04:55:58.975 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:58 smithi096 ceph-mon[35085]: 1: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.c 2024-09-19T04:55:58.975 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:58 smithi096 ceph-mon[35085]: 2: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.b 2024-09-19T04:55:58.975 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:58 smithi096 ceph-mon[35085]: fsmap 2024-09-19T04:55:58.975 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:58 smithi096 ceph-mon[35085]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:55:58.975 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:58 smithi096 ceph-mon[35085]: mgrmap e13: x(active, since 49s) 2024-09-19T04:55:58.975 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:58 smithi096 ceph-mon[35085]: overall HEALTH_OK 2024-09-19T04:55:58.975 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:58 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:58.975 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:58 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:58.975 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:58 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:58.975 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:58 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:58.975 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:58 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:55:58.975 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:58 smithi096 ceph-mon[35085]: from='client.? 172.21.15.151:0/2109579671' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T04:55:58.975 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:58 smithi096 ceph-mon[35085]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:55:58.976 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:58 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:58.976 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:58 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:55:59.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:59 smithi043 ceph-mon[30800]: mon.b calling monitor election 2024-09-19T04:55:59.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:59 smithi043 ceph-mon[30800]: Updating smithi043:/etc/ceph/ceph.conf 2024-09-19T04:55:59.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:59 smithi043 ceph-mon[30800]: Updating smithi096:/etc/ceph/ceph.conf 2024-09-19T04:55:59.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:59 smithi043 ceph-mon[30800]: Updating smithi151:/etc/ceph/ceph.conf 2024-09-19T04:55:59.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:59 smithi043 ceph-mon[30800]: Updating smithi096:/var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:55:59.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:59 smithi043 ceph-mon[30800]: Updating smithi151:/var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:55:59.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:59 smithi043 ceph-mon[30800]: mon.b calling monitor election 2024-09-19T04:55:59.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:59 smithi043 ceph-mon[30800]: mon.c calling monitor election 2024-09-19T04:55:59.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:59 smithi043 ceph-mon[30800]: mon.a calling monitor election 2024-09-19T04:55:59.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:59 smithi043 ceph-mon[30800]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T04:55:59.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:59 smithi043 ceph-mon[30800]: monmap epoch 3 2024-09-19T04:55:59.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:59 smithi043 ceph-mon[30800]: fsid 1456a810-7643-11ef-bceb-c7b262605968 2024-09-19T04:55:59.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:59 smithi043 ceph-mon[30800]: last_changed 2024-09-19T04:55:51.536852+0000 2024-09-19T04:55:59.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:59 smithi043 ceph-mon[30800]: created 2024-09-19T04:54:11.079879+0000 2024-09-19T04:55:59.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:59 smithi043 ceph-mon[30800]: min_mon_release 19 (squid) 2024-09-19T04:55:59.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:59 smithi043 ceph-mon[30800]: election_strategy: 1 2024-09-19T04:55:59.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:59 smithi043 ceph-mon[30800]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-09-19T04:55:59.283 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:59 smithi043 ceph-mon[30800]: 1: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.c 2024-09-19T04:55:59.283 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:59 smithi043 ceph-mon[30800]: 2: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.b 2024-09-19T04:55:59.283 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:59 smithi043 ceph-mon[30800]: fsmap 2024-09-19T04:55:59.283 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:59 smithi043 ceph-mon[30800]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:55:59.283 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:59 smithi043 ceph-mon[30800]: mgrmap e13: x(active, since 51s) 2024-09-19T04:55:59.283 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:59 smithi043 ceph-mon[30800]: overall HEALTH_OK 2024-09-19T04:55:59.283 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:59 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/2579061060' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:55:59.283 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:59 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:59.283 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:59 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:59.283 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:59 smithi043 ceph-mon[30800]: Updating smithi043:/var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:55:59.283 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:59 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:59.284 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:59 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:59.284 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:55:59 smithi043 ceph-mon[30800]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:55:59.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:59 smithi151 ceph-mon[35425]: mon.b calling monitor election 2024-09-19T04:55:59.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:59 smithi151 ceph-mon[35425]: Updating smithi043:/etc/ceph/ceph.conf 2024-09-19T04:55:59.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:59 smithi151 ceph-mon[35425]: Updating smithi096:/etc/ceph/ceph.conf 2024-09-19T04:55:59.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:59 smithi151 ceph-mon[35425]: Updating smithi151:/etc/ceph/ceph.conf 2024-09-19T04:55:59.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:59 smithi151 ceph-mon[35425]: Updating smithi096:/var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:55:59.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:59 smithi151 ceph-mon[35425]: Updating smithi151:/var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:55:59.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:59 smithi151 ceph-mon[35425]: mon.b calling monitor election 2024-09-19T04:55:59.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:59 smithi151 ceph-mon[35425]: mon.c calling monitor election 2024-09-19T04:55:59.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:59 smithi151 ceph-mon[35425]: mon.a calling monitor election 2024-09-19T04:55:59.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:59 smithi151 ceph-mon[35425]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T04:55:59.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:59 smithi151 ceph-mon[35425]: monmap epoch 3 2024-09-19T04:55:59.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:59 smithi151 ceph-mon[35425]: fsid 1456a810-7643-11ef-bceb-c7b262605968 2024-09-19T04:55:59.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:59 smithi151 ceph-mon[35425]: last_changed 2024-09-19T04:55:51.536852+0000 2024-09-19T04:55:59.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:59 smithi151 ceph-mon[35425]: created 2024-09-19T04:54:11.079879+0000 2024-09-19T04:55:59.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:59 smithi151 ceph-mon[35425]: min_mon_release 19 (squid) 2024-09-19T04:55:59.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:59 smithi151 ceph-mon[35425]: election_strategy: 1 2024-09-19T04:55:59.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:59 smithi151 ceph-mon[35425]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-09-19T04:55:59.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:59 smithi151 ceph-mon[35425]: 1: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.c 2024-09-19T04:55:59.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:59 smithi151 ceph-mon[35425]: 2: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.b 2024-09-19T04:55:59.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:59 smithi151 ceph-mon[35425]: fsmap 2024-09-19T04:55:59.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:59 smithi151 ceph-mon[35425]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:55:59.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:59 smithi151 ceph-mon[35425]: mgrmap e13: x(active, since 51s) 2024-09-19T04:55:59.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:59 smithi151 ceph-mon[35425]: overall HEALTH_OK 2024-09-19T04:55:59.407 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:59 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/2579061060' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:55:59.407 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:59 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:59.407 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:59 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:59.407 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:59 smithi151 ceph-mon[35425]: Updating smithi043:/var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:55:59.407 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:59 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:59.407 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:59 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:59.407 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:55:59 smithi151 ceph-mon[35425]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:55:59.417 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-19T04:55:59.417 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-19T04:55:59.417 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T04:55:59.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:59 smithi096 ceph-mon[35085]: mon.b calling monitor election 2024-09-19T04:55:59.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:59 smithi096 ceph-mon[35085]: Updating smithi043:/etc/ceph/ceph.conf 2024-09-19T04:55:59.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:59 smithi096 ceph-mon[35085]: Updating smithi096:/etc/ceph/ceph.conf 2024-09-19T04:55:59.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:59 smithi096 ceph-mon[35085]: Updating smithi151:/etc/ceph/ceph.conf 2024-09-19T04:55:59.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:59 smithi096 ceph-mon[35085]: Updating smithi096:/var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:55:59.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:59 smithi096 ceph-mon[35085]: Updating smithi151:/var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:55:59.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:59 smithi096 ceph-mon[35085]: mon.b calling monitor election 2024-09-19T04:55:59.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:59 smithi096 ceph-mon[35085]: mon.c calling monitor election 2024-09-19T04:55:59.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:59 smithi096 ceph-mon[35085]: mon.a calling monitor election 2024-09-19T04:55:59.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:59 smithi096 ceph-mon[35085]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T04:55:59.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:59 smithi096 ceph-mon[35085]: monmap epoch 3 2024-09-19T04:55:59.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:59 smithi096 ceph-mon[35085]: fsid 1456a810-7643-11ef-bceb-c7b262605968 2024-09-19T04:55:59.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:59 smithi096 ceph-mon[35085]: last_changed 2024-09-19T04:55:51.536852+0000 2024-09-19T04:55:59.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:59 smithi096 ceph-mon[35085]: created 2024-09-19T04:54:11.079879+0000 2024-09-19T04:55:59.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:59 smithi096 ceph-mon[35085]: min_mon_release 19 (squid) 2024-09-19T04:55:59.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:59 smithi096 ceph-mon[35085]: election_strategy: 1 2024-09-19T04:55:59.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:59 smithi096 ceph-mon[35085]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-09-19T04:55:59.475 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:59 smithi096 ceph-mon[35085]: 1: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.c 2024-09-19T04:55:59.475 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:59 smithi096 ceph-mon[35085]: 2: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.b 2024-09-19T04:55:59.475 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:59 smithi096 ceph-mon[35085]: fsmap 2024-09-19T04:55:59.475 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:59 smithi096 ceph-mon[35085]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T04:55:59.475 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:59 smithi096 ceph-mon[35085]: mgrmap e13: x(active, since 51s) 2024-09-19T04:55:59.475 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:59 smithi096 ceph-mon[35085]: overall HEALTH_OK 2024-09-19T04:55:59.475 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:59 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/2579061060' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:55:59.475 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:59 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:59.475 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:59 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:59.475 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:59 smithi096 ceph-mon[35085]: Updating smithi043:/var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T04:55:59.475 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:59 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:59.475 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:59 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:55:59.475 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:55:59 smithi096 ceph-mon[35085]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:55:59.477 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-19T04:55:59.477 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:55:59.574 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-19T04:55:59.575 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T04:55:59.604 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-19T04:55:59.604 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:55:59.675 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-19T04:55:59.675 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T04:55:59.703 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-19T04:55:59.703 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T04:55:59.769 INFO:tasks.cephadm:Adding mgr.x on smithi043 2024-09-19T04:55:59.769 INFO:tasks.cephadm:Adding mgr.y on smithi096 2024-09-19T04:55:59.769 INFO:tasks.cephadm:Adding mgr.z on smithi151 2024-09-19T04:55:59.769 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph orch apply mgr '3;smithi043=x;smithi096=y;smithi151=z' 2024-09-19T04:56:00.045 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:56:00.424 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:00 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:00.424 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:00 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:00.424 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:00 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:00.425 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:00 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:00.425 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:00 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:00.425 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:00 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:00.425 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:00 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:00.425 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:00 smithi151 ceph-mon[35425]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T04:56:00.425 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:00 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:56:00.425 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:00 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:56:00.425 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:00 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:00.425 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:00 smithi151 ceph-mon[35425]: Reconfiguring daemon mon.a on smithi043 2024-09-19T04:56:00.425 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:00 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:56:00.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:00 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:00.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:00 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:00.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:00 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:00.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:00 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:00.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:00 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:00.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:00 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:00.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:00 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:00.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:00 smithi096 ceph-mon[35085]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T04:56:00.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:00 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:56:00.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:00 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:56:00.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:00 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:00.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:00 smithi096 ceph-mon[35085]: Reconfiguring daemon mon.a on smithi043 2024-09-19T04:56:00.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:00 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:56:00.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:00 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:00.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:00 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:00.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:00 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:00.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:00 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:00.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:00 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:00.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:00 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:00.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:00 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:00.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:00 smithi043 ceph-mon[30800]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T04:56:00.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:00 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:56:00.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:00 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:56:00.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:00 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:00.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:00 smithi043 ceph-mon[30800]: Reconfiguring daemon mon.a on smithi043 2024-09-19T04:56:00.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:00 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:56:00.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.612+0000 7f6cb2925640 1 -- 172.21.15.151:0/2215575835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cac1005e0 msgr2=0x7f6cac1009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:00.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.612+0000 7f6cb2925640 1 --2- 172.21.15.151:0/2215575835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cac1005e0 0x7f6cac1009e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c009880 tx=0x7f6c9c02a5d0 comp rx=0 tx=0).stop 2024-09-19T04:56:00.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.612+0000 7f6cb2925640 1 -- 172.21.15.151:0/2215575835 shutdown_connections 2024-09-19T04:56:00.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.612+0000 7f6cb2925640 1 --2- 172.21.15.151:0/2215575835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cac1005e0 0x7f6cac1009e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:00.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.612+0000 7f6cb2925640 1 -- 172.21.15.151:0/2215575835 >> 172.21.15.151:0/2215575835 conn(0x7f6cac0fbdb0 msgr2=0x7f6cac0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:00.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.612+0000 7f6cb2925640 1 -- 172.21.15.151:0/2215575835 shutdown_connections 2024-09-19T04:56:00.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.612+0000 7f6cb2925640 1 -- 172.21.15.151:0/2215575835 wait complete. 2024-09-19T04:56:00.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.612+0000 7f6cb2925640 1 Processor -- start 2024-09-19T04:56:00.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.612+0000 7f6cb2925640 1 -- start start 2024-09-19T04:56:00.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.613+0000 7f6cb2925640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6cac1005e0 0x7f6cac06a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:00.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.613+0000 7f6cb2925640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cac199d00 0x7f6cac19c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:00.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.613+0000 7f6cb2925640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6cac19c630 0x7f6cac19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:00.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.613+0000 7f6cb2925640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6cac1066f0 con 0x7f6cac199d00 2024-09-19T04:56:00.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.613+0000 7f6cb2925640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6cac106570 con 0x7f6cac1005e0 2024-09-19T04:56:00.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.613+0000 7f6cb2925640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6cac106870 con 0x7f6cac19c630 2024-09-19T04:56:00.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.613+0000 7f6cb0ec2640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6cac19c630 0x7f6cac19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:00.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.613+0000 7f6cb0ec2640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6cac19c630 0x7f6cac19eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:50168/0 (socket says 172.21.15.151:50168) 2024-09-19T04:56:00.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.613+0000 7f6cb0ec2640 1 -- 172.21.15.151:0/926357312 learned_addr learned my addr 172.21.15.151:0/926357312 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-19T04:56:00.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.613+0000 7f6cab7fe640 1 --2- 172.21.15.151:0/926357312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cac199d00 0x7f6cac19c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:00.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.613+0000 7f6cabfff640 1 --2- 172.21.15.151:0/926357312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6cac1005e0 0x7f6cac06a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:00.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.613+0000 7f6cb0ec2640 1 -- 172.21.15.151:0/926357312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6cac1005e0 msgr2=0x7f6cac06a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:00.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.613+0000 7f6cb0ec2640 1 --2- 172.21.15.151:0/926357312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6cac1005e0 0x7f6cac06a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:00.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.613+0000 7f6cb0ec2640 1 -- 172.21.15.151:0/926357312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cac199d00 msgr2=0x7f6cac19c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:00.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.613+0000 7f6cb0ec2640 1 --2- 172.21.15.151:0/926357312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cac199d00 0x7f6cac19c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:00.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.613+0000 7f6cb0ec2640 1 -- 172.21.15.151:0/926357312 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cac19f400 con 0x7f6cac19c630 2024-09-19T04:56:00.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.614+0000 7f6cb0ec2640 1 --2- 172.21.15.151:0/926357312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6cac19c630 0x7f6cac19eaf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6ca400ba50 tx=0x7f6ca400bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:00.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.614+0000 7f6ca97fa640 1 -- 172.21.15.151:0/926357312 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4019070 con 0x7f6cac19c630 2024-09-19T04:56:00.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.614+0000 7f6cb2925640 1 -- 172.21.15.151:0/926357312 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cac19f690 con 0x7f6cac19c630 2024-09-19T04:56:00.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.614+0000 7f6ca97fa640 1 -- 172.21.15.151:0/926357312 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ca400cbe0 con 0x7f6cac19c630 2024-09-19T04:56:00.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.614+0000 7f6ca97fa640 1 -- 172.21.15.151:0/926357312 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca40076a0 con 0x7f6cac19c630 2024-09-19T04:56:00.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.614+0000 7f6cb2925640 1 -- 172.21.15.151:0/926357312 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cac19fc00 con 0x7f6cac19c630 2024-09-19T04:56:00.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.615+0000 7f6ca97fa640 1 -- 172.21.15.151:0/926357312 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f6ca401c070 con 0x7f6cac19c630 2024-09-19T04:56:00.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.615+0000 7f6cb2925640 1 -- 172.21.15.151:0/926357312 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cac18f4b0 con 0x7f6cac19c630 2024-09-19T04:56:00.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.615+0000 7f6ca97fa640 1 --2- 172.21.15.151:0/926357312 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f6c94039730 0x7f6c9403bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:00.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.615+0000 7f6ca97fa640 1 -- 172.21.15.151:0/926357312 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6ca404bd90 con 0x7f6cac19c630 2024-09-19T04:56:00.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.615+0000 7f6cabfff640 1 --2- 172.21.15.151:0/926357312 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f6c94039730 0x7f6c9403bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:00.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.616+0000 7f6cabfff640 1 --2- 172.21.15.151:0/926357312 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f6c94039730 0x7f6c9403bbf0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c02abc0 tx=0x7f6c9c031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:00.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.619+0000 7f6ca97fa640 1 -- 172.21.15.151:0/926357312 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6ca40112a0 con 0x7f6cac19c630 2024-09-19T04:56:00.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.796+0000 7f6cb2925640 1 -- 172.21.15.151:0/926357312 --> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi043=x;smithi096=y;smithi151=z", "target": ["mon-mgr", ""]}) -- 0x7f6cac0626c0 con 0x7f6c94039730 2024-09-19T04:56:00.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.802+0000 7f6ca97fa640 1 -- 172.21.15.151:0/926357312 <== mgr.14150 v2:172.21.15.43:6800/3167108925 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6cac0626c0 con 0x7f6c94039730 2024-09-19T04:56:00.803 INFO:teuthology.orchestra.run.smithi151.stdout:Scheduled mgr update... 2024-09-19T04:56:00.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.804+0000 7f6cb2925640 1 -- 172.21.15.151:0/926357312 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f6c94039730 msgr2=0x7f6c9403bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:00.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.804+0000 7f6cb2925640 1 --2- 172.21.15.151:0/926357312 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f6c94039730 0x7f6c9403bbf0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c02abc0 tx=0x7f6c9c031400 comp rx=0 tx=0).stop 2024-09-19T04:56:00.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.804+0000 7f6cb2925640 1 -- 172.21.15.151:0/926357312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6cac19c630 msgr2=0x7f6cac19eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:00.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.804+0000 7f6cb2925640 1 --2- 172.21.15.151:0/926357312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6cac19c630 0x7f6cac19eaf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6ca400ba50 tx=0x7f6ca400bf10 comp rx=0 tx=0).stop 2024-09-19T04:56:00.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.804+0000 7f6cb2925640 1 -- 172.21.15.151:0/926357312 shutdown_connections 2024-09-19T04:56:00.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.804+0000 7f6cb2925640 1 --2- 172.21.15.151:0/926357312 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f6c94039730 0x7f6c9403bbf0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:00.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.804+0000 7f6cb2925640 1 --2- 172.21.15.151:0/926357312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6cac19c630 0x7f6cac19eaf0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:00.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.804+0000 7f6cb2925640 1 --2- 172.21.15.151:0/926357312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cac199d00 0x7f6cac19c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:00.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.804+0000 7f6cb2925640 1 --2- 172.21.15.151:0/926357312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6cac1005e0 0x7f6cac06a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:00.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.804+0000 7f6cb2925640 1 -- 172.21.15.151:0/926357312 >> 172.21.15.151:0/926357312 conn(0x7f6cac0fbdb0 msgr2=0x7f6cac0fc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:00.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.804+0000 7f6cb2925640 1 -- 172.21.15.151:0/926357312 shutdown_connections 2024-09-19T04:56:00.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:56:00.804+0000 7f6cb2925640 1 -- 172.21.15.151:0/926357312 wait complete. 2024-09-19T04:56:01.443 DEBUG:teuthology.orchestra.run.smithi096:mgr.y> sudo journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@mgr.y.service 2024-09-19T04:56:01.446 DEBUG:teuthology.orchestra.run.smithi151:mgr.z> sudo journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@mgr.z.service 2024-09-19T04:56:01.448 INFO:tasks.cephadm:Deploying OSDs... 2024-09-19T04:56:01.448 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-19T04:56:01.448 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T04:56:01.484 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T04:56:01.485 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_1 2024-09-19T04:56:01.562 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T04:56:01.562 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:56:01.562 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-09-19T04:56:01.562 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:56:01.562 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:56:01.562 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-19 04:55:16.979205099 +0000 2024-09-19T04:56:01.562 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-19 04:51:02.318685452 +0000 2024-09-19T04:56:01.562 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-19 04:51:02.318685452 +0000 2024-09-19T04:56:01.562 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-09-19 04:51:02.318685452 +0000 2024-09-19T04:56:01.562 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T04:56:01.658 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-09-19T04:56:01.658 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-09-19T04:56:01.658 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.00018771 s, 2.7 MB/s 2024-09-19T04:56:01.660 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T04:56:01.701 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_2 2024-09-19T04:56:01.777 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T04:56:01.777 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:56:01.777 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-09-19T04:56:01.778 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:56:01.778 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:56:01.778 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-19 04:55:17.018204634 +0000 2024-09-19T04:56:01.778 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-19 04:51:02.780680012 +0000 2024-09-19T04:56:01.778 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-19 04:51:02.780680012 +0000 2024-09-19T04:56:01.778 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-09-19 04:51:02.780680012 +0000 2024-09-19T04:56:01.778 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T04:56:01.872 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-09-19T04:56:01.873 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-09-19T04:56:01.873 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000177856 s, 2.9 MB/s 2024-09-19T04:56:01.875 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T04:56:01.912 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_3 2024-09-19T04:56:01.988 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T04:56:01.989 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:56:01.989 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-09-19T04:56:01.989 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:56:01.989 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:56:01.989 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-19 04:55:17.058204157 +0000 2024-09-19T04:56:01.989 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-19 04:51:03.241674584 +0000 2024-09-19T04:56:01.989 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-19 04:51:03.241674584 +0000 2024-09-19T04:56:01.989 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-09-19 04:51:03.241674584 +0000 2024-09-19T04:56:01.989 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T04:56:02.077 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:01 smithi043 ceph-mon[30800]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi043=x;smithi096=y;smithi151=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:56:02.078 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:01 smithi043 ceph-mon[30800]: Saving service mgr spec with placement smithi043=x;smithi096=y;smithi151=z;count:3 2024-09-19T04:56:02.078 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:01 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:02.078 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:01 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:02.078 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:01 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:02.078 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:01 smithi043 ceph-mon[30800]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T04:56:02.078 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:01 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:56:02.078 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:01 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:56:02.078 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:01 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:02.078 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:01 smithi043 ceph-mon[30800]: Reconfiguring daemon mgr.x on smithi043 2024-09-19T04:56:02.078 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:01 smithi043 ceph-mon[30800]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:02.084 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-09-19T04:56:02.084 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-09-19T04:56:02.084 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000249015 s, 2.1 MB/s 2024-09-19T04:56:02.088 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T04:56:02.125 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_4 2024-09-19T04:56:02.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:01 smithi151 ceph-mon[35425]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi043=x;smithi096=y;smithi151=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:56:02.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:01 smithi151 ceph-mon[35425]: Saving service mgr spec with placement smithi043=x;smithi096=y;smithi151=z;count:3 2024-09-19T04:56:02.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:01 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:02.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:01 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:02.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:01 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:02.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:01 smithi151 ceph-mon[35425]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T04:56:02.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:01 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:56:02.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:01 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:56:02.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:01 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:02.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:01 smithi151 ceph-mon[35425]: Reconfiguring daemon mgr.x on smithi043 2024-09-19T04:56:02.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:01 smithi151 ceph-mon[35425]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:02.202 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T04:56:02.202 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:56:02.202 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-09-19T04:56:02.202 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:56:02.202 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:56:02.202 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-19 04:55:17.099203668 +0000 2024-09-19T04:56:02.202 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-19 04:51:03.741668697 +0000 2024-09-19T04:56:02.202 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-19 04:51:03.741668697 +0000 2024-09-19T04:56:02.202 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-09-19 04:51:03.741668697 +0000 2024-09-19T04:56:02.203 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T04:56:02.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:01 smithi096 ceph-mon[35085]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi043=x;smithi096=y;smithi151=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:56:02.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:01 smithi096 ceph-mon[35085]: Saving service mgr spec with placement smithi043=x;smithi096=y;smithi151=z;count:3 2024-09-19T04:56:02.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:01 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:02.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:01 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:02.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:01 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:02.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:01 smithi096 ceph-mon[35085]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T04:56:02.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:01 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:56:02.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:01 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:56:02.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:01 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:02.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:01 smithi096 ceph-mon[35085]: Reconfiguring daemon mgr.x on smithi043 2024-09-19T04:56:02.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:01 smithi096 ceph-mon[35085]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:02.297 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-09-19T04:56:02.297 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-09-19T04:56:02.297 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000168484 s, 3.0 MB/s 2024-09-19T04:56:02.299 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T04:56:02.336 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-19T04:56:02.337 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T04:56:02.357 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T04:56:02.357 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_1 2024-09-19T04:56:02.416 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T04:56:02.416 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:56:02.416 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-19T04:56:02.416 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:56:02.416 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:56:02.416 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-09-19 04:55:51.020187157 +0000 2024-09-19T04:56:02.416 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-09-19 04:50:23.388288394 +0000 2024-09-19T04:56:02.416 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-09-19 04:50:23.388288394 +0000 2024-09-19T04:56:02.417 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-09-19 04:50:23.388288394 +0000 2024-09-19T04:56:02.417 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T04:56:02.483 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-09-19T04:56:02.483 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-09-19T04:56:02.483 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000233217 s, 2.2 MB/s 2024-09-19T04:56:02.485 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T04:56:02.544 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_2 2024-09-19T04:56:02.602 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T04:56:02.602 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:56:02.602 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-09-19T04:56:02.602 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:56:02.603 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:56:02.603 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-09-19 04:55:51.020187157 +0000 2024-09-19T04:56:02.603 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-09-19 04:50:23.705280915 +0000 2024-09-19T04:56:02.603 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-09-19 04:50:23.705280915 +0000 2024-09-19T04:56:02.603 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-09-19 04:50:23.705280915 +0000 2024-09-19T04:56:02.603 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T04:56:02.669 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-09-19T04:56:02.670 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-09-19T04:56:02.670 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000214695 s, 2.4 MB/s 2024-09-19T04:56:02.671 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T04:56:02.727 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_3 2024-09-19T04:56:02.783 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T04:56:02.783 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:56:02.783 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-19T04:56:02.784 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:56:02.784 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:56:02.784 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-09-19 04:55:51.020187157 +0000 2024-09-19T04:56:02.784 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-09-19 04:50:24.018273531 +0000 2024-09-19T04:56:02.784 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-09-19 04:50:24.018273531 +0000 2024-09-19T04:56:02.784 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-09-19 04:50:24.018273531 +0000 2024-09-19T04:56:02.784 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T04:56:02.847 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-09-19T04:56:02.847 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-09-19T04:56:02.848 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000212305 s, 2.4 MB/s 2024-09-19T04:56:02.849 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T04:56:02.905 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_4 2024-09-19T04:56:02.961 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T04:56:02.961 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:56:02.961 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-19T04:56:02.961 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:56:02.961 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:56:02.961 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-09-19 04:55:51.021187134 +0000 2024-09-19T04:56:02.961 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-09-19 04:50:24.333266099 +0000 2024-09-19T04:56:02.961 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-09-19 04:50:24.333266099 +0000 2024-09-19T04:56:02.961 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-09-19 04:50:24.333266099 +0000 2024-09-19T04:56:02.962 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T04:56:03.024 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-09-19T04:56:03.024 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-09-19T04:56:03.024 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.00018081 s, 2.8 MB/s 2024-09-19T04:56:03.025 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T04:56:03.082 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-19T04:56:03.082 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T04:56:03.102 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T04:56:03.102 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_1 2024-09-19T04:56:03.161 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T04:56:03.161 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:56:03.161 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-19T04:56:03.161 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:56:03.161 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:56:03.161 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-09-19 04:55:41.892250518 +0000 2024-09-19T04:56:03.161 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-09-19 04:50:24.285452249 +0000 2024-09-19T04:56:03.161 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-09-19 04:50:24.285452249 +0000 2024-09-19T04:56:03.161 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-09-19 04:50:24.285452249 +0000 2024-09-19T04:56:03.161 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T04:56:03.230 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-09-19T04:56:03.230 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-09-19T04:56:03.230 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000271997 s, 1.9 MB/s 2024-09-19T04:56:03.231 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T04:56:03.291 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_2 2024-09-19T04:56:03.348 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T04:56:03.349 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:56:03.349 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-09-19T04:56:03.349 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:56:03.349 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:56:03.349 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-09-19 04:55:41.892250518 +0000 2024-09-19T04:56:03.349 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-09-19 04:50:24.635447955 +0000 2024-09-19T04:56:03.349 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-09-19 04:50:24.635447955 +0000 2024-09-19T04:56:03.349 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-09-19 04:50:24.635447955 +0000 2024-09-19T04:56:03.349 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T04:56:03.415 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-09-19T04:56:03.415 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-09-19T04:56:03.415 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000183425 s, 2.8 MB/s 2024-09-19T04:56:03.416 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T04:56:03.474 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_3 2024-09-19T04:56:03.531 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T04:56:03.531 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:56:03.531 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-09-19T04:56:03.532 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:56:03.532 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:56:03.532 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-09-19 04:55:41.892250518 +0000 2024-09-19T04:56:03.532 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-09-19 04:50:24.967443882 +0000 2024-09-19T04:56:03.532 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-09-19 04:50:24.967443882 +0000 2024-09-19T04:56:03.532 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-09-19 04:50:24.967443882 +0000 2024-09-19T04:56:03.532 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T04:56:03.596 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-09-19T04:56:03.597 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-09-19T04:56:03.597 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000203566 s, 2.5 MB/s 2024-09-19T04:56:03.598 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T04:56:03.655 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_4 2024-09-19T04:56:03.711 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T04:56:03.711 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:56:03.711 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-09-19T04:56:03.711 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:56:03.711 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T04:56:03.711 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-09-19 04:55:41.893250505 +0000 2024-09-19T04:56:03.711 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-09-19 04:50:25.297439834 +0000 2024-09-19T04:56:03.711 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-09-19 04:50:25.297439834 +0000 2024-09-19T04:56:03.711 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-09-19 04:50:25.297439834 +0000 2024-09-19T04:56:03.712 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T04:56:03.774 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-09-19T04:56:03.774 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-09-19T04:56:03.774 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000205848 s, 2.5 MB/s 2024-09-19T04:56:03.775 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T04:56:03.833 INFO:tasks.cephadm:Deploying osd.0 on smithi043 with /dev/vg_nvme/lv_4... 2024-09-19T04:56:03.833 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T04:56:03.887 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:03 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:03.888 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:03 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:03.888 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:03 smithi043 ceph-mon[30800]: Reconfiguring mon.b (monmap changed)... 2024-09-19T04:56:03.888 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:03 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:56:03.888 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:03 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:56:03.888 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:03 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:03.888 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:03 smithi043 ceph-mon[30800]: Reconfiguring daemon mon.b on smithi096 2024-09-19T04:56:03.888 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:03 smithi043 ceph-mon[30800]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:03.904 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:03 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:03.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:03 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:03.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:03 smithi151 ceph-mon[35425]: Reconfiguring mon.b (monmap changed)... 2024-09-19T04:56:03.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:03 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:56:03.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:03 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:56:03.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:03 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:03.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:03 smithi151 ceph-mon[35425]: Reconfiguring daemon mon.b on smithi096 2024-09-19T04:56:03.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:03 smithi151 ceph-mon[35425]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:03.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:03 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:03.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:03 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:03.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:03 smithi096 ceph-mon[35085]: Reconfiguring mon.b (monmap changed)... 2024-09-19T04:56:03.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:03 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:56:03.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:03 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:56:03.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:03 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:03.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:03 smithi096 ceph-mon[35085]: Reconfiguring daemon mon.b on smithi096 2024-09-19T04:56:03.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:03 smithi096 ceph-mon[35085]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:04.211 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:56:05.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:05 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:05.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:05 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:05.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:05 smithi043 ceph-mon[30800]: Reconfiguring mon.c (monmap changed)... 2024-09-19T04:56:05.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:05 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:56:05.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:05 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:56:05.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:05 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:05.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:05 smithi043 ceph-mon[30800]: Reconfiguring daemon mon.c on smithi151 2024-09-19T04:56:05.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:05 smithi043 ceph-mon[30800]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:05.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:05 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:05.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:05 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:05.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:05 smithi151 ceph-mon[35425]: Reconfiguring mon.c (monmap changed)... 2024-09-19T04:56:05.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:05 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:56:05.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:05 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:56:05.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:05 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:05.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:05 smithi151 ceph-mon[35425]: Reconfiguring daemon mon.c on smithi151 2024-09-19T04:56:05.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:05 smithi151 ceph-mon[35425]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:05.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:05 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:05.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:05 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:05.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:05 smithi096 ceph-mon[35085]: Reconfiguring mon.c (monmap changed)... 2024-09-19T04:56:05.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:05 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T04:56:05.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:05 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T04:56:05.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:05 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:05.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:05 smithi096 ceph-mon[35085]: Reconfiguring daemon mon.c on smithi151 2024-09-19T04:56:05.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:05 smithi096 ceph-mon[35085]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:06.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:06 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:06.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:06 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:06.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:06 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:56:06.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:06 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:06.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:06 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:56:06.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:06 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:06.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:06 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:56:06.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:06 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T04:56:06.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:06 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:56:06.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:06 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:06.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:06 smithi151 ceph-mon[35425]: Deploying daemon mgr.z on smithi151 2024-09-19T04:56:06.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:06 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:06.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:06 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:06.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:06 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:56:06.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:06 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:06.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:06 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:56:06.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:06 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:06.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:06 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:56:06.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:06 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T04:56:06.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:06 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:56:06.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:06 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:06.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:06 smithi096 ceph-mon[35085]: Deploying daemon mgr.z on smithi151 2024-09-19T04:56:07.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:06 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:07.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:06 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:07.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:06 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:56:07.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:06 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:07.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:06 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:56:07.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:06 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:07.032 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:06 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:56:07.032 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:06 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T04:56:07.032 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:06 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:56:07.032 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:06 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:07.032 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:06 smithi043 ceph-mon[30800]: Deploying daemon mgr.z on smithi151 2024-09-19T04:56:07.078 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T04:56:07.111 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi043:vg_nvme/lv_4 2024-09-19T04:56:07.477 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:56:07.564 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 04:56:07 smithi151 systemd[1]: Starting Ceph mgr.z for 1456a810-7643-11ef-bceb-c7b262605968... 2024-09-19T04:56:07.826 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:07 smithi043 ceph-mon[30800]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:07.828 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:07 smithi151 ceph-mon[35425]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:07.828 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 04:56:07 smithi151 podman[37221]: 2024-09-19 04:56:07.466741998 +0000 UTC m=+0.015870272 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:56:07.828 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 04:56:07 smithi151 podman[37221]: 2024-09-19 04:56:07.564505611 +0000 UTC m=+0.113633875 container create 1e3c4197daf2699d5423f0bd348688b12841ab2a1270a9562ccf4ecb38c511d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-19T04:56:07.829 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 04:56:07 smithi151 podman[37221]: 2024-09-19 04:56:07.800415958 +0000 UTC m=+0.349544245 container init 1e3c4197daf2699d5423f0bd348688b12841ab2a1270a9562ccf4ecb38c511d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-19T04:56:07.829 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 04:56:07 smithi151 podman[37221]: 2024-09-19 04:56:07.803306724 +0000 UTC m=+0.352434989 container start 1e3c4197daf2699d5423f0bd348688b12841ab2a1270a9562ccf4ecb38c511d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-19T04:56:07.829 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 04:56:07 smithi151 ceph-mgr[37240]: -- 172.21.15.151:0/2973563422 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5618526dd0e0 con 0x5618526bb000 2024-09-19T04:56:07.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:07 smithi096 ceph-mon[35085]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:08.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:08.026+0000 7f79dea00640 1 -- 172.21.15.43:0/460555263 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79d8077df0 msgr2=0x7f79d8078250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:08.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:08.026+0000 7f79dea00640 1 --2- 172.21.15.43:0/460555263 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79d8077df0 0x7f79d8078250 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f79c4009900 tx=0x7f79c402a3f0 comp rx=0 tx=0).stop 2024-09-19T04:56:08.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:08.026+0000 7f79dea00640 1 -- 172.21.15.43:0/460555263 shutdown_connections 2024-09-19T04:56:08.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:08.026+0000 7f79dea00640 1 --2- 172.21.15.43:0/460555263 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79d8078790 0x7f79d8079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:08.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:08.026+0000 7f79dea00640 1 --2- 172.21.15.43:0/460555263 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79d8077df0 0x7f79d8078250 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:08.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:08.026+0000 7f79dea00640 1 --2- 172.21.15.43:0/460555263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79d8076ba0 0x7f79d8076fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:08.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:08.026+0000 7f79dea00640 1 -- 172.21.15.43:0/460555263 >> 172.21.15.43:0/460555263 conn(0x7f79d80ff3e0 msgr2=0x7f79d8101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:08.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:08.026+0000 7f79dea00640 1 -- 172.21.15.43:0/460555263 shutdown_connections 2024-09-19T04:56:08.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:08.026+0000 7f79dea00640 1 -- 172.21.15.43:0/460555263 wait complete. 2024-09-19T04:56:08.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:08.027+0000 7f79dea00640 1 Processor -- start 2024-09-19T04:56:08.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:08.027+0000 7f79dea00640 1 -- start start 2024-09-19T04:56:08.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:08.028+0000 7f79dea00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79d8076ba0 0x7f79d819e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:08.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:08.028+0000 7f79dea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79d8077df0 0x7f79d819e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:08.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:08.028+0000 7f79dea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79d8078790 0x7f79d81a58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:08.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:08.028+0000 7f79dea00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f79d8113cf0 con 0x7f79d8078790 2024-09-19T04:56:08.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:08.028+0000 7f79dea00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f79d8113b70 con 0x7f79d8076ba0 2024-09-19T04:56:08.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:08.028+0000 7f79dea00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f79d8113e70 con 0x7f79d8077df0 2024-09-19T04:56:08.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:08.028+0000 7f79dd600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79d8076ba0 0x7f79d819e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:08.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:08.028+0000 7f79de000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79d8078790 0x7f79d81a58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:08.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:08.028+0000 7f79dcc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79d8077df0 0x7f79d819e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:08.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:08.028+0000 7f79dd600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79d8076ba0 0x7f79d819e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:36152/0 (socket says 172.21.15.43:36152) 2024-09-19T04:56:08.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:08.028+0000 7f79dd600640 1 -- 172.21.15.43:0/2956059828 learned_addr learned my addr 172.21.15.43:0/2956059828 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:56:08.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:08.028+0000 7f79dd600640 1 -- 172.21.15.43:0/2956059828 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79d8077df0 msgr2=0x7f79d819e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:08.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:08.028+0000 7f79dd600640 1 --2- 172.21.15.43:0/2956059828 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79d8077df0 0x7f79d819e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:08.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:08.028+0000 7f79dd600640 1 -- 172.21.15.43:0/2956059828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79d8078790 msgr2=0x7f79d81a58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:08.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:08.028+0000 7f79dd600640 1 --2- 172.21.15.43:0/2956059828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79d8078790 0x7f79d81a58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:08.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:08.028+0000 7f79dd600640 1 -- 172.21.15.43:0/2956059828 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79d81a5f20 con 0x7f79d8076ba0 2024-09-19T04:56:08.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:08.028+0000 7f79de000640 1 --2- 172.21.15.43:0/2956059828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79d8078790 0x7f79d81a58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:56:08.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:08.029+0000 7f79dd600640 1 --2- 172.21.15.43:0/2956059828 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79d8076ba0 0x7f79d819e2f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f79cc00e9f0 tx=0x7f79cc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:08.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:08.029+0000 7f79d6000640 1 -- 172.21.15.43:0/2956059828 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79cc00cd80 con 0x7f79d8076ba0 2024-09-19T04:56:08.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:08.029+0000 7f79d6000640 1 -- 172.21.15.43:0/2956059828 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79cc010910 con 0x7f79d8076ba0 2024-09-19T04:56:08.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:08.029+0000 7f79dea00640 1 -- 172.21.15.43:0/2956059828 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79d81a6210 con 0x7f79d8076ba0 2024-09-19T04:56:08.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:08.029+0000 7f79d6000640 1 -- 172.21.15.43:0/2956059828 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79cc018990 con 0x7f79d8076ba0 2024-09-19T04:56:08.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:08.029+0000 7f79dea00640 1 -- 172.21.15.43:0/2956059828 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79d81a6770 con 0x7f79d8076ba0 2024-09-19T04:56:08.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:08.030+0000 7f79d6000640 1 -- 172.21.15.43:0/2956059828 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f79cc018b30 con 0x7f79d8076ba0 2024-09-19T04:56:08.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:08.030+0000 7f79dea00640 1 -- 172.21.15.43:0/2956059828 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79d81105d0 con 0x7f79d8076ba0 2024-09-19T04:56:08.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:08.031+0000 7f79d6000640 1 --2- 172.21.15.43:0/2956059828 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f79c0039500 0x7f79c003b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:08.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:08.031+0000 7f79d6000640 1 -- 172.21.15.43:0/2956059828 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f79cc04cdf0 con 0x7f79d8076ba0 2024-09-19T04:56:08.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:08.031+0000 7f79dcc00640 1 --2- 172.21.15.43:0/2956059828 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f79c0039500 0x7f79c003b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:08.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:08.032+0000 7f79dcc00640 1 --2- 172.21.15.43:0/2956059828 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f79c0039500 0x7f79c003b9c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f79c40314e0 tx=0x7f79c4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:08.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:08.037+0000 7f79d6000640 1 -- 172.21.15.43:0/2956059828 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f79cc018e20 con 0x7f79d8076ba0 2024-09-19T04:56:08.138 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 04:56:07 smithi151 bash[37221]: 1e3c4197daf2699d5423f0bd348688b12841ab2a1270a9562ccf4ecb38c511d7 2024-09-19T04:56:08.138 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 04:56:07 smithi151 systemd[1]: Started Ceph mgr.z for 1456a810-7643-11ef-bceb-c7b262605968. 2024-09-19T04:56:08.138 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 04:56:07 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T04:56:07.923+0000 7fd07bf3e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T04:56:08.138 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 04:56:07 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T04:56:07.989+0000 7fd07bf3e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T04:56:08.138 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 04:56:08 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T04:56:08.135+0000 7fd07bf3e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T04:56:08.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:08.234+0000 7f79dea00640 1 -- 172.21.15.43:0/2956059828 --> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f79d8062630 con 0x7f79c0039500 2024-09-19T04:56:08.404 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 04:56:08 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T04:56:08.201+0000 7fd07bf3e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T04:56:08.405 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 04:56:08 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T04:56:08.346+0000 7fd07bf3e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:56:08.797 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 04:56:08 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T04:56:08.484+0000 7fd07bf3e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T04:56:09.083 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:08 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:09.083 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:08 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:09.083 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:08 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:09.083 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:08 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:56:09.083 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:08 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T04:56:09.083 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:08 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:56:09.083 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:08 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:09.084 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:08 smithi043 ceph-mon[30800]: Deploying daemon mgr.y on smithi096 2024-09-19T04:56:09.084 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:08 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:56:09.084 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:08 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:56:09.084 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:08 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:09.155 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 04:56:08 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T04:56:08.794+0000 7fd07bf3e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T04:56:09.155 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 04:56:08 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T04:56:08.855+0000 7fd07bf3e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T04:56:09.155 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 04:56:09 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T04:56:09.041+0000 7fd07bf3e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T04:56:09.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:08 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:09.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:08 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:09.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:08 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:09.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:08 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:56:09.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:08 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T04:56:09.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:08 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:56:09.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:08 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:09.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:08 smithi151 ceph-mon[35425]: Deploying daemon mgr.y on smithi096 2024-09-19T04:56:09.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:08 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:56:09.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:08 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:56:09.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:08 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:09.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:08 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:09.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:08 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:09.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:08 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:09.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:08 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T04:56:09.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:08 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T04:56:09.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:08 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T04:56:09.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:08 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:09.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:08 smithi096 ceph-mon[35085]: Deploying daemon mgr.y on smithi096 2024-09-19T04:56:09.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:08 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:56:09.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:08 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:56:09.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:08 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:09.723 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 04:56:09 smithi096 systemd[1]: Starting Ceph mgr.y for 1456a810-7643-11ef-bceb-c7b262605968... 2024-09-19T04:56:09.997 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:09 smithi043 ceph-mon[30800]: from='client.24107 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:56:09.998 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:09 smithi043 ceph-mon[30800]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:10.023 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:09 smithi096 ceph-mon[35085]: from='client.24107 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:56:10.023 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:09 smithi096 ceph-mon[35085]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:10.023 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 04:56:09 smithi096 podman[36436]: 2024-09-19 04:56:09.625417455 +0000 UTC m=+0.015618453 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T04:56:10.023 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 04:56:09 smithi096 podman[36436]: 2024-09-19 04:56:09.728370238 +0000 UTC m=+0.118571232 container create 97a90af90dc6cd3617342bc404baae1e68f917691bb326280f741eb9daee1cbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-19T04:56:10.023 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 04:56:09 smithi096 podman[36436]: 2024-09-19 04:56:09.939263762 +0000 UTC m=+0.329464759 container init 97a90af90dc6cd3617342bc404baae1e68f917691bb326280f741eb9daee1cbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4) 2024-09-19T04:56:10.023 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 04:56:09 smithi096 podman[36436]: 2024-09-19 04:56:09.941898228 +0000 UTC m=+0.332099232 container start 97a90af90dc6cd3617342bc404baae1e68f917691bb326280f741eb9daee1cbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-19T04:56:10.024 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 04:56:09 smithi096 ceph-mgr[36455]: -- 172.21.15.96:0/2372842597 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55abde9310e0 con 0x55abde90f400 2024-09-19T04:56:10.024 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 04:56:10 smithi096 bash[36436]: 97a90af90dc6cd3617342bc404baae1e68f917691bb326280f741eb9daee1cbe 2024-09-19T04:56:10.232 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:09 smithi151 ceph-mon[35425]: from='client.24107 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:56:10.232 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:09 smithi151 ceph-mon[35425]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:10.233 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 04:56:09 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T04:56:09.917+0000 7fd07bf3e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T04:56:10.233 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 04:56:09 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T04:56:09.988+0000 7fd07bf3e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T04:56:10.276 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 04:56:10 smithi096 systemd[1]: Started Ceph mgr.y for 1456a810-7643-11ef-bceb-c7b262605968. 2024-09-19T04:56:10.276 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 04:56:10 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T04:56:10.061+0000 7fd0081ee040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T04:56:10.276 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 04:56:10 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T04:56:10.120+0000 7fd0081ee040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T04:56:10.276 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 04:56:10 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T04:56:10.273+0000 7fd0081ee040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T04:56:10.689 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 04:56:10 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T04:56:10.344+0000 7fd0081ee040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T04:56:10.689 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 04:56:10 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T04:56:10.503+0000 7fd0081ee040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:56:10.689 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 04:56:10 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T04:56:10.686+0000 7fd0081ee040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T04:56:11.050 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 04:56:10 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T04:56:10.773+0000 7fd07bf3e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T04:56:11.050 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 04:56:10 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T04:56:10.894+0000 7fd07bf3e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T04:56:11.050 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 04:56:11 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T04:56:11.047+0000 7fd07bf3e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:56:11.269 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:11 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:11.269 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:11 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:11.270 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:11 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:11.270 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:11 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:11.270 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:11 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:56:11.270 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:11 smithi096 ceph-mon[35085]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:11.270 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 04:56:11 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T04:56:11.014+0000 7fd0081ee040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T04:56:11.270 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 04:56:11 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T04:56:11.074+0000 7fd0081ee040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T04:56:11.270 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 04:56:11 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T04:56:11.267+0000 7fd0081ee040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T04:56:11.349 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:11 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:11.349 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:11 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:11.349 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:11 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:11.349 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:11 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:11.349 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:11 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:56:11.350 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:11 smithi151 ceph-mon[35425]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:11.350 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 04:56:11 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T04:56:11.113+0000 7fd07bf3e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T04:56:11.350 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 04:56:11 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T04:56:11.266+0000 7fd07bf3e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T04:56:11.350 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 04:56:11 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T04:56:11.346+0000 7fd07bf3e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T04:56:11.381 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:11 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:11.381 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:11 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:11.381 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:11 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:11.381 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:11 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:11.381 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:11 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:56:11.381 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:11 smithi043 ceph-mon[30800]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:11.635 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 04:56:11 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T04:56:11.407+0000 7fd07bf3e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T04:56:11.636 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 04:56:11 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T04:56:11.567+0000 7fd07bf3e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T04:56:11.636 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 04:56:11 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T04:56:11.633+0000 7fd07bf3e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T04:56:11.904 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 04:56:11 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T04:56:11.702+0000 7fd07bf3e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T04:56:11.904 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 04:56:11 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T04:56:11.784+0000 7fd07bf3e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T04:56:11.904 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 04:56:11 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T04:56:11.842+0000 7fd07bf3e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T04:56:11.904 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 04:56:11 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T04:56:11.901+0000 7fd07bf3e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T04:56:12.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:12 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/587650831' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "81a317a8-7ec8-45c3-bc10-7da023283635"}]: dispatch 2024-09-19T04:56:12.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:12 smithi151 ceph-mon[35425]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "81a317a8-7ec8-45c3-bc10-7da023283635"}]: dispatch 2024-09-19T04:56:12.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:12 smithi151 ceph-mon[35425]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "81a317a8-7ec8-45c3-bc10-7da023283635"}]': finished 2024-09-19T04:56:12.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:12 smithi151 ceph-mon[35425]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T04:56:12.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:12 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:56:12.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:12 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:12.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:12 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:12.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:12 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:12.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:12 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:12.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:12 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:12.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:12 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:56:12.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:12 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:12.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:12 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:56:12.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:12 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:12.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:12 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:56:12.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:12 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:12.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:12 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/360525628' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:56:12.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:12 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:12.473 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 04:56:12 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T04:56:12.179+0000 7fd0081ee040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T04:56:12.473 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 04:56:12 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T04:56:12.245+0000 7fd0081ee040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T04:56:12.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:12 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/587650831' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "81a317a8-7ec8-45c3-bc10-7da023283635"}]: dispatch 2024-09-19T04:56:12.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:12 smithi096 ceph-mon[35085]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "81a317a8-7ec8-45c3-bc10-7da023283635"}]: dispatch 2024-09-19T04:56:12.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:12 smithi096 ceph-mon[35085]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "81a317a8-7ec8-45c3-bc10-7da023283635"}]': finished 2024-09-19T04:56:12.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:12 smithi096 ceph-mon[35085]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T04:56:12.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:12 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:56:12.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:12 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:12.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:12 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:12.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:12 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:12.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:12 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:12.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:12 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:12.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:12 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:56:12.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:12 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:12.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:12 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:56:12.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:12 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:12.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:12 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:56:12.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:12 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:12.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:12 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/360525628' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:56:12.475 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:12 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:12.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:12 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/587650831' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "81a317a8-7ec8-45c3-bc10-7da023283635"}]: dispatch 2024-09-19T04:56:12.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:12 smithi043 ceph-mon[30800]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "81a317a8-7ec8-45c3-bc10-7da023283635"}]: dispatch 2024-09-19T04:56:12.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:12 smithi043 ceph-mon[30800]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "81a317a8-7ec8-45c3-bc10-7da023283635"}]': finished 2024-09-19T04:56:12.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:12 smithi043 ceph-mon[30800]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T04:56:12.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:12 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:56:12.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:12 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:12.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:12 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:12.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:12 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:12.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:12 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:12.532 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:12 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:12.532 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:12 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:56:12.532 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:12 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:12.532 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:12 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:56:12.532 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:12 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:12.532 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:12 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:56:12.532 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:12 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:12.532 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:12 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/360525628' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:56:12.532 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:12 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:13.109 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 04:56:12 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T04:56:12.694+0000 7fd07bf3e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T04:56:13.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:13.116+0000 7f79d6000640 1 -- 172.21.15.43:0/2956059828 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 14) ==== 92408+0+0 (secure 0 0 0) 0x7f79cc082210 con 0x7f79d8076ba0 2024-09-19T04:56:13.204 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:13 smithi096 ceph-mon[35085]: Standby manager daemon z started 2024-09-19T04:56:13.204 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:13 smithi096 ceph-mon[35085]: from='mgr.? 172.21.15.151:0/1325658344' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T04:56:13.204 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:13 smithi096 ceph-mon[35085]: from='mgr.? 172.21.15.151:0/1325658344' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:56:13.204 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:13 smithi096 ceph-mon[35085]: from='mgr.? 172.21.15.151:0/1325658344' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T04:56:13.204 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:13 smithi096 ceph-mon[35085]: from='mgr.? 172.21.15.151:0/1325658344' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:56:13.204 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:13 smithi096 ceph-mon[35085]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:13.205 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 04:56:12 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T04:56:12.948+0000 7fd0081ee040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T04:56:13.205 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 04:56:13 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T04:56:13.060+0000 7fd0081ee040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T04:56:13.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:13 smithi151 ceph-mon[35425]: Standby manager daemon z started 2024-09-19T04:56:13.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:13 smithi151 ceph-mon[35425]: from='mgr.? 172.21.15.151:0/1325658344' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T04:56:13.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:13 smithi151 ceph-mon[35425]: from='mgr.? 172.21.15.151:0/1325658344' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:56:13.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:13 smithi151 ceph-mon[35425]: from='mgr.? 172.21.15.151:0/1325658344' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T04:56:13.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:13 smithi151 ceph-mon[35425]: from='mgr.? 172.21.15.151:0/1325658344' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:56:13.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:13 smithi151 ceph-mon[35425]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:13.473 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 04:56:13 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T04:56:13.202+0000 7fd0081ee040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:56:13.473 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 04:56:13 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T04:56:13.263+0000 7fd0081ee040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T04:56:13.473 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 04:56:13 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T04:56:13.411+0000 7fd0081ee040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T04:56:13.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:13 smithi043 ceph-mon[30800]: Standby manager daemon z started 2024-09-19T04:56:13.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:13 smithi043 ceph-mon[30800]: from='mgr.? 172.21.15.151:0/1325658344' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T04:56:13.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:13 smithi043 ceph-mon[30800]: from='mgr.? 172.21.15.151:0/1325658344' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:56:13.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:13 smithi043 ceph-mon[30800]: from='mgr.? 172.21.15.151:0/1325658344' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T04:56:13.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:13 smithi043 ceph-mon[30800]: from='mgr.? 172.21.15.151:0/1325658344' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:56:13.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:13 smithi043 ceph-mon[30800]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:13.761 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 04:56:13 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T04:56:13.478+0000 7fd0081ee040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T04:56:13.761 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 04:56:13 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T04:56:13.538+0000 7fd0081ee040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T04:56:13.761 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 04:56:13 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T04:56:13.694+0000 7fd0081ee040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T04:56:13.761 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 04:56:13 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T04:56:13.759+0000 7fd0081ee040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T04:56:14.025 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 04:56:13 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T04:56:13.825+0000 7fd0081ee040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T04:56:14.026 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 04:56:13 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T04:56:13.907+0000 7fd0081ee040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T04:56:14.026 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 04:56:13 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T04:56:13.965+0000 7fd0081ee040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T04:56:14.026 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 04:56:14 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T04:56:14.024+0000 7fd0081ee040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T04:56:14.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:14 smithi151 ceph-mon[35425]: mgrmap e14: x(active, since 66s), standbys: z 2024-09-19T04:56:14.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:14 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T04:56:14.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:14 smithi096 ceph-mon[35085]: mgrmap e14: x(active, since 66s), standbys: z 2024-09-19T04:56:14.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:14 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T04:56:14.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:14 smithi043 ceph-mon[30800]: mgrmap e14: x(active, since 66s), standbys: z 2024-09-19T04:56:14.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:14 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T04:56:15.118 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 04:56:14 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T04:56:14.807+0000 7fd0081ee040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T04:56:15.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:15.126+0000 7f79d6000640 1 -- 172.21.15.43:0/2956059828 <== mon.2 v2:172.21.15.96:3300/0 8 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7f79cc01f310 con 0x7f79d8076ba0 2024-09-19T04:56:15.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:15 smithi151 ceph-mon[35425]: Standby manager daemon y started 2024-09-19T04:56:15.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:15 smithi151 ceph-mon[35425]: from='mgr.? 172.21.15.96:0/3201151770' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T04:56:15.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:15 smithi151 ceph-mon[35425]: from='mgr.? 172.21.15.96:0/3201151770' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:56:15.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:15 smithi151 ceph-mon[35425]: from='mgr.? 172.21.15.96:0/3201151770' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T04:56:15.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:15 smithi151 ceph-mon[35425]: from='mgr.? 172.21.15.96:0/3201151770' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:56:15.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:15 smithi151 ceph-mon[35425]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:15.464 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:15 smithi043 ceph-mon[30800]: Standby manager daemon y started 2024-09-19T04:56:15.464 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:15 smithi043 ceph-mon[30800]: from='mgr.? 172.21.15.96:0/3201151770' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T04:56:15.464 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:15 smithi043 ceph-mon[30800]: from='mgr.? 172.21.15.96:0/3201151770' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:56:15.464 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:15 smithi043 ceph-mon[30800]: from='mgr.? 172.21.15.96:0/3201151770' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T04:56:15.464 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:15 smithi043 ceph-mon[30800]: from='mgr.? 172.21.15.96:0/3201151770' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:56:15.464 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:15 smithi043 ceph-mon[30800]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:15.472 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:15 smithi096 ceph-mon[35085]: Standby manager daemon y started 2024-09-19T04:56:15.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:15 smithi096 ceph-mon[35085]: from='mgr.? 172.21.15.96:0/3201151770' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T04:56:15.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:15 smithi096 ceph-mon[35085]: from='mgr.? 172.21.15.96:0/3201151770' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T04:56:15.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:15 smithi096 ceph-mon[35085]: from='mgr.? 172.21.15.96:0/3201151770' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T04:56:15.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:15 smithi096 ceph-mon[35085]: from='mgr.? 172.21.15.96:0/3201151770' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T04:56:15.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:15 smithi096 ceph-mon[35085]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:16.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:16 smithi151 ceph-mon[35425]: mgrmap e15: x(active, since 68s), standbys: z, y 2024-09-19T04:56:16.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:16 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T04:56:16.472 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:16 smithi043 ceph-mon[30800]: mgrmap e15: x(active, since 68s), standbys: z, y 2024-09-19T04:56:16.472 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:16 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T04:56:16.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:16 smithi096 ceph-mon[35085]: mgrmap e15: x(active, since 68s), standbys: z, y 2024-09-19T04:56:16.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:16 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T04:56:17.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:17 smithi151 ceph-mon[35425]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:17.433 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:17 smithi043 ceph-mon[30800]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:17.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:17 smithi096 ceph-mon[35085]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:19.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:19 smithi151 ceph-mon[35425]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:19.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:19 smithi096 ceph-mon[35085]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:19.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:19 smithi043 ceph-mon[30800]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:20.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:20 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T04:56:20.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:20 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:20.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:20 smithi043 ceph-mon[30800]: Deploying daemon osd.0 on smithi043 2024-09-19T04:56:20.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:20 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T04:56:20.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:20 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:20.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:20 smithi151 ceph-mon[35425]: Deploying daemon osd.0 on smithi043 2024-09-19T04:56:20.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:20 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T04:56:20.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:20 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:20.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:20 smithi096 ceph-mon[35085]: Deploying daemon osd.0 on smithi043 2024-09-19T04:56:21.191 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:21 smithi043 ceph-mon[30800]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:21.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:21 smithi151 ceph-mon[35425]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:21.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:21 smithi096 ceph-mon[35085]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:23.392 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:23 smithi043 ceph-mon[30800]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:23.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:23 smithi151 ceph-mon[35425]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:23.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:23 smithi096 ceph-mon[35085]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:25.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:25 smithi151 ceph-mon[35425]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:25.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:25 smithi096 ceph-mon[35085]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:25.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:25 smithi043 ceph-mon[30800]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:27.208 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:27 smithi043 ceph-mon[30800]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:27.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:27 smithi151 ceph-mon[35425]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:27.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:27 smithi096 ceph-mon[35085]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:28.331 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:28 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:56:28.349 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:28 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:28.349 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:28 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:28.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:28 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:56:28.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:28 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:28.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:28 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:28.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:28 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:56:28.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:28 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:28.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:28 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:29.278 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:29 smithi043 ceph-mon[30800]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:29.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:29 smithi151 ceph-mon[35425]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:29.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:29 smithi096 ceph-mon[35085]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:31.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:31 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:31.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:31 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:31.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:31 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:31.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:31 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:56:31.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:31 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:31.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:31 smithi151 ceph-mon[35425]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:31.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:31 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:31.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:31 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:31.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:31 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:31.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:31 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:56:31.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:31 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:31.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:31 smithi096 ceph-mon[35085]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:31.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:31 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:31.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:31 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:31.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:31 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:31.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:31 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:56:31.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:31 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:31.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:31 smithi043 ceph-mon[30800]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:32.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:32.140+0000 7f79d6000640 1 -- 172.21.15.43:0/2956059828 <== mgr.14150 v2:172.21.15.43:6800/3167108925 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f79d8062630 con 0x7f79c0039500 2024-09-19T04:56:32.142 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 0 on host 'smithi043' 2024-09-19T04:56:32.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:32.142+0000 7f79dea00640 1 -- 172.21.15.43:0/2956059828 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f79c0039500 msgr2=0x7f79c003b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:32.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:32.142+0000 7f79dea00640 1 --2- 172.21.15.43:0/2956059828 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f79c0039500 0x7f79c003b9c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f79c40314e0 tx=0x7f79c4000bc0 comp rx=0 tx=0).stop 2024-09-19T04:56:32.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:32.142+0000 7f79dea00640 1 -- 172.21.15.43:0/2956059828 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79d8076ba0 msgr2=0x7f79d819e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:32.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:32.142+0000 7f79dea00640 1 --2- 172.21.15.43:0/2956059828 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79d8076ba0 0x7f79d819e2f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f79cc00e9f0 tx=0x7f79cc00eeb0 comp rx=0 tx=0).stop 2024-09-19T04:56:32.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:32.143+0000 7f79dea00640 1 -- 172.21.15.43:0/2956059828 shutdown_connections 2024-09-19T04:56:32.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:32.143+0000 7f79dea00640 1 --2- 172.21.15.43:0/2956059828 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f79c0039500 0x7f79c003b9c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:32.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:32.143+0000 7f79dea00640 1 --2- 172.21.15.43:0/2956059828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79d8078790 0x7f79d81a58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:32.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:32.143+0000 7f79dea00640 1 --2- 172.21.15.43:0/2956059828 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79d8077df0 0x7f79d819e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:32.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:32.143+0000 7f79dea00640 1 --2- 172.21.15.43:0/2956059828 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79d8076ba0 0x7f79d819e2f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:32.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:32.143+0000 7f79dea00640 1 -- 172.21.15.43:0/2956059828 >> 172.21.15.43:0/2956059828 conn(0x7f79d80ff3e0 msgr2=0x7f79d8100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:32.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:32.143+0000 7f79dea00640 1 -- 172.21.15.43:0/2956059828 shutdown_connections 2024-09-19T04:56:32.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:56:32.143+0000 7f79dea00640 1 -- 172.21.15.43:0/2956059828 wait complete. 2024-09-19T04:56:32.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:32 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:56:32.496 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:32 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:32.496 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:32 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:32.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:32 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:56:32.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:32 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:32.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:32 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:32.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:32 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:56:32.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:32 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:32.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:32 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:32.900 DEBUG:teuthology.orchestra.run.smithi043:osd.0> sudo journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@osd.0.service 2024-09-19T04:56:32.905 INFO:tasks.cephadm:Deploying osd.1 on smithi096 with /dev/vg_nvme/lv_4... 2024-09-19T04:56:32.905 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T04:56:33.155 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:56:33.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:33 smithi043 ceph-mon[30800]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:33.562 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:33 smithi096 ceph-mon[35085]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:33.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:33 smithi151 ceph-mon[35425]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:34.031 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 19 04:56:33 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-0[41934]: 2024-09-19T04:56:33.719+0000 7f177044e640 -1 osd.0 0 log_to_monitors true 2024-09-19T04:56:34.336 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:34 smithi096 ceph-mon[35085]: from='osd.0 [v2:172.21.15.43:6802/825351486,v1:172.21.15.43:6803/825351486]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T04:56:34.504 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:34 smithi043 ceph-mon[30800]: from='osd.0 [v2:172.21.15.43:6802/825351486,v1:172.21.15.43:6803/825351486]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T04:56:34.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:34 smithi151 ceph-mon[35425]: from='osd.0 [v2:172.21.15.43:6802/825351486,v1:172.21.15.43:6803/825351486]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T04:56:35.383 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-19T04:56:35.408 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi096:vg_nvme/lv_4 2024-09-19T04:56:35.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:35 smithi151 ceph-mon[35425]: from='osd.0 [v2:172.21.15.43:6802/825351486,v1:172.21.15.43:6803/825351486]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T04:56:35.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:35 smithi151 ceph-mon[35425]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T04:56:35.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:35 smithi151 ceph-mon[35425]: from='osd.0 [v2:172.21.15.43:6802/825351486,v1:172.21.15.43:6803/825351486]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-19T04:56:35.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:35 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:56:35.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:35 smithi151 ceph-mon[35425]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:35.658 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:56:35.683 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:35 smithi096 ceph-mon[35085]: from='osd.0 [v2:172.21.15.43:6802/825351486,v1:172.21.15.43:6803/825351486]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T04:56:35.683 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:35 smithi096 ceph-mon[35085]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T04:56:35.683 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:35 smithi096 ceph-mon[35085]: from='osd.0 [v2:172.21.15.43:6802/825351486,v1:172.21.15.43:6803/825351486]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-19T04:56:35.683 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:35 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:56:35.683 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:35 smithi096 ceph-mon[35085]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:35.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:35 smithi043 ceph-mon[30800]: from='osd.0 [v2:172.21.15.43:6802/825351486,v1:172.21.15.43:6803/825351486]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T04:56:35.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:35 smithi043 ceph-mon[30800]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T04:56:35.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:35 smithi043 ceph-mon[30800]: from='osd.0 [v2:172.21.15.43:6802/825351486,v1:172.21.15.43:6803/825351486]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-19T04:56:35.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:35 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:56:35.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:35 smithi043 ceph-mon[30800]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:36.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:36.092+0000 7f16984e6640 1 -- 172.21.15.96:0/3495216601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16901026b0 msgr2=0x7f1690108f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:36.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:36.092+0000 7f16984e6640 1 --2- 172.21.15.96:0/3495216601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16901026b0 0x7f1690108f40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1688009ca0 tx=0x7f168802a8c0 comp rx=0 tx=0).stop 2024-09-19T04:56:36.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:36.092+0000 7f16984e6640 1 -- 172.21.15.96:0/3495216601 shutdown_connections 2024-09-19T04:56:36.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:36.092+0000 7f16984e6640 1 --2- 172.21.15.96:0/3495216601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16901026b0 0x7f1690108f40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:36.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:36.092+0000 7f16984e6640 1 --2- 172.21.15.96:0/3495216601 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1690101cf0 0x7f1690102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:36.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:36.092+0000 7f16984e6640 1 --2- 172.21.15.96:0/3495216601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1690100af0 0x7f1690100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:36.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:36.092+0000 7f16984e6640 1 -- 172.21.15.96:0/3495216601 >> 172.21.15.96:0/3495216601 conn(0x7f16900fc280 msgr2=0x7f16900fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:36.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:36.092+0000 7f16984e6640 1 -- 172.21.15.96:0/3495216601 shutdown_connections 2024-09-19T04:56:36.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:36.092+0000 7f16984e6640 1 -- 172.21.15.96:0/3495216601 wait complete. 2024-09-19T04:56:36.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:36.092+0000 7f16984e6640 1 Processor -- start 2024-09-19T04:56:36.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:36.092+0000 7f16984e6640 1 -- start start 2024-09-19T04:56:36.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:36.092+0000 7f16984e6640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1690100af0 0x7f1690197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:36.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:36.093+0000 7f16984e6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1690101cf0 0x7f1690198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:36.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:36.093+0000 7f16984e6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16901026b0 0x7f169019f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:36.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:36.093+0000 7f16984e6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f169010c770 con 0x7f1690101cf0 2024-09-19T04:56:36.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:36.093+0000 7f16984e6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f169010c5f0 con 0x7f16901026b0 2024-09-19T04:56:36.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:36.093+0000 7f16984e6640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f169010c8f0 con 0x7f1690100af0 2024-09-19T04:56:36.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:36.093+0000 7f1696a83640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16901026b0 0x7f169019f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:36.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:36.093+0000 7f1696a83640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16901026b0 0x7f169019f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59962/0 (socket says 172.21.15.96:59962) 2024-09-19T04:56:36.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:36.093+0000 7f1696a83640 1 -- 172.21.15.96:0/1161473387 learned_addr learned my addr 172.21.15.96:0/1161473387 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-19T04:56:36.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:36.093+0000 7f1696282640 1 --2- 172.21.15.96:0/1161473387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1690100af0 0x7f1690197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:36.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:36.093+0000 7f1695a81640 1 --2- 172.21.15.96:0/1161473387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1690101cf0 0x7f1690198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:36.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:36.093+0000 7f1696a83640 1 -- 172.21.15.96:0/1161473387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1690100af0 msgr2=0x7f1690197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:36.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:36.093+0000 7f1696a83640 1 --2- 172.21.15.96:0/1161473387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1690100af0 0x7f1690197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:36.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:36.093+0000 7f1696a83640 1 -- 172.21.15.96:0/1161473387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1690101cf0 msgr2=0x7f1690198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:36.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:36.093+0000 7f1696a83640 1 --2- 172.21.15.96:0/1161473387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1690101cf0 0x7f1690198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:36.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:36.093+0000 7f1696a83640 1 -- 172.21.15.96:0/1161473387 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f169019fb10 con 0x7f16901026b0 2024-09-19T04:56:36.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:36.093+0000 7f1695a81640 1 --2- 172.21.15.96:0/1161473387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1690101cf0 0x7f1690198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:56:36.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:36.093+0000 7f1696a83640 1 --2- 172.21.15.96:0/1161473387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16901026b0 0x7f169019f4a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1688005e30 tx=0x7f1688031ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:36.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:36.093+0000 7f16877fe640 1 -- 172.21.15.96:0/1161473387 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1688003c30 con 0x7f16901026b0 2024-09-19T04:56:36.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:36.093+0000 7f16877fe640 1 -- 172.21.15.96:0/1161473387 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1688003dd0 con 0x7f16901026b0 2024-09-19T04:56:36.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:36.093+0000 7f16984e6640 1 -- 172.21.15.96:0/1161473387 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f169019fda0 con 0x7f16901026b0 2024-09-19T04:56:36.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:36.093+0000 7f16877fe640 1 -- 172.21.15.96:0/1161473387 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168802fde0 con 0x7f16901026b0 2024-09-19T04:56:36.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:36.093+0000 7f16984e6640 1 -- 172.21.15.96:0/1161473387 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16901a0250 con 0x7f16901026b0 2024-09-19T04:56:36.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:36.095+0000 7f16984e6640 1 -- 172.21.15.96:0/1161473387 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16901a04b0 con 0x7f16901026b0 2024-09-19T04:56:36.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:36.096+0000 7f16877fe640 1 -- 172.21.15.96:0/1161473387 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7f1688004050 con 0x7f16901026b0 2024-09-19T04:56:36.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:36.097+0000 7f16877fe640 1 --2- 172.21.15.96:0/1161473387 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f167c0a31e0 0x7f167c0a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:56:36.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:36.097+0000 7f16877fe640 1 -- 172.21.15.96:0/1161473387 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7f16880ef010 con 0x7f16901026b0 2024-09-19T04:56:36.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:36.097+0000 7f1696282640 1 --2- 172.21.15.96:0/1161473387 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f167c0a31e0 0x7f167c0a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:56:36.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:36.098+0000 7f1696282640 1 --2- 172.21.15.96:0/1161473387 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f167c0a31e0 0x7f167c0a56a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1680005fd0 tx=0x7f1680005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:56:36.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:36.101+0000 7f16877fe640 1 -- 172.21.15.96:0/1161473387 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f16880bd8c0 con 0x7f16901026b0 2024-09-19T04:56:36.170 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 19 04:56:35 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-0[41934]: 2024-09-19T04:56:35.886+0000 7f176b000640 -1 osd.0 0 waiting for initial osdmap 2024-09-19T04:56:36.170 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 19 04:56:35 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-0[41934]: 2024-09-19T04:56:35.901+0000 7f1761000640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:56:36.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:36.283+0000 7f16984e6640 1 -- 172.21.15.96:0/1161473387 --> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f169006aef0 con 0x7f167c0a31e0 2024-09-19T04:56:36.511 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:36 smithi043 ceph-mon[30800]: from='osd.0 [v2:172.21.15.43:6802/825351486,v1:172.21.15.43:6803/825351486]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-19T04:56:36.512 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:36 smithi043 ceph-mon[30800]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T04:56:36.512 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:36 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:56:36.512 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:36 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:56:36.512 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:36 smithi043 ceph-mon[30800]: from='osd.0 [v2:172.21.15.43:6802/825351486,v1:172.21.15.43:6803/825351486]' entity='osd.0' 2024-09-19T04:56:36.512 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:36 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:56:36.512 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:36 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:56:36.512 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:36 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:36.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:36 smithi151 ceph-mon[35425]: from='osd.0 [v2:172.21.15.43:6802/825351486,v1:172.21.15.43:6803/825351486]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-19T04:56:36.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:36 smithi151 ceph-mon[35425]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T04:56:36.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:36 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:56:36.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:36 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:56:36.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:36 smithi151 ceph-mon[35425]: from='osd.0 [v2:172.21.15.43:6802/825351486,v1:172.21.15.43:6803/825351486]' entity='osd.0' 2024-09-19T04:56:36.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:36 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:56:36.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:36 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:56:36.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:36 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:36.667 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:36 smithi096 ceph-mon[35085]: from='osd.0 [v2:172.21.15.43:6802/825351486,v1:172.21.15.43:6803/825351486]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-19T04:56:36.667 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:36 smithi096 ceph-mon[35085]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T04:56:36.667 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:36 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:56:36.667 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:36 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:56:36.667 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:36 smithi096 ceph-mon[35085]: from='osd.0 [v2:172.21.15.43:6802/825351486,v1:172.21.15.43:6803/825351486]' entity='osd.0' 2024-09-19T04:56:36.667 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:36 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:56:36.667 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:36 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:56:36.667 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:36 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:37.646 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:37 smithi096 ceph-mon[35085]: purged_snaps scrub starts 2024-09-19T04:56:37.646 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:37 smithi096 ceph-mon[35085]: purged_snaps scrub ok 2024-09-19T04:56:37.647 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:37 smithi096 ceph-mon[35085]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:56:37.647 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:37 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:56:37.647 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:37 smithi096 ceph-mon[35085]: Detected new or changed devices on smithi043 2024-09-19T04:56:37.647 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:37 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:37.647 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:37 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:37.647 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:37 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:56:37.647 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:37 smithi096 ceph-mon[35085]: Adjusting osd_memory_target on smithi043 to 17094M 2024-09-19T04:56:37.647 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:37 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:37.647 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:37 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:37.647 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:37 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:56:37.647 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:37 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:37.647 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:37 smithi096 ceph-mon[35085]: osd.0 [v2:172.21.15.43:6802/825351486,v1:172.21.15.43:6803/825351486] boot 2024-09-19T04:56:37.647 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:37 smithi096 ceph-mon[35085]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T04:56:37.648 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:37 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:56:37.648 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:37 smithi096 ceph-mon[35085]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:37.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:37 smithi151 ceph-mon[35425]: purged_snaps scrub starts 2024-09-19T04:56:37.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:37 smithi151 ceph-mon[35425]: purged_snaps scrub ok 2024-09-19T04:56:37.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:37 smithi151 ceph-mon[35425]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:56:37.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:37 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:56:37.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:37 smithi151 ceph-mon[35425]: Detected new or changed devices on smithi043 2024-09-19T04:56:37.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:37 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:37.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:37 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:37.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:37 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:56:37.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:37 smithi151 ceph-mon[35425]: Adjusting osd_memory_target on smithi043 to 17094M 2024-09-19T04:56:37.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:37 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:37.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:37 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:37.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:37 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:56:37.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:37 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:37.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:37 smithi151 ceph-mon[35425]: osd.0 [v2:172.21.15.43:6802/825351486,v1:172.21.15.43:6803/825351486] boot 2024-09-19T04:56:37.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:37 smithi151 ceph-mon[35425]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T04:56:37.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:37 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:56:37.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:37 smithi151 ceph-mon[35425]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:37.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:37 smithi043 ceph-mon[30800]: purged_snaps scrub starts 2024-09-19T04:56:37.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:37 smithi043 ceph-mon[30800]: purged_snaps scrub ok 2024-09-19T04:56:37.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:37 smithi043 ceph-mon[30800]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:56:37.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:37 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:56:37.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:37 smithi043 ceph-mon[30800]: Detected new or changed devices on smithi043 2024-09-19T04:56:37.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:37 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:37.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:37 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:37.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:37 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:56:37.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:37 smithi043 ceph-mon[30800]: Adjusting osd_memory_target on smithi043 to 17094M 2024-09-19T04:56:37.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:37 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:37.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:37 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:37.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:37 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:56:37.782 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:37 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:37.782 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:37 smithi043 ceph-mon[30800]: osd.0 [v2:172.21.15.43:6802/825351486,v1:172.21.15.43:6803/825351486] boot 2024-09-19T04:56:37.782 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:37 smithi043 ceph-mon[30800]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T04:56:37.782 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:37 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T04:56:37.782 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:37 smithi043 ceph-mon[30800]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T04:56:39.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:39 smithi151 ceph-mon[35425]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T04:56:39.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:39 smithi151 ceph-mon[35425]: from='client.? 172.21.15.96:0/3210221478' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e96fcfac-4fff-4a36-b38b-aebf57842740"}]: dispatch 2024-09-19T04:56:39.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:39 smithi151 ceph-mon[35425]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e96fcfac-4fff-4a36-b38b-aebf57842740"}]: dispatch 2024-09-19T04:56:39.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:39 smithi151 ceph-mon[35425]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e96fcfac-4fff-4a36-b38b-aebf57842740"}]': finished 2024-09-19T04:56:39.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:39 smithi151 ceph-mon[35425]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T04:56:39.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:39 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:56:39.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:39 smithi151 ceph-mon[35425]: from='client.? 172.21.15.96:0/1848294269' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:56:39.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:39 smithi151 ceph-mon[35425]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:56:39.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:39 smithi096 ceph-mon[35085]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T04:56:39.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:39 smithi096 ceph-mon[35085]: from='client.? 172.21.15.96:0/3210221478' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e96fcfac-4fff-4a36-b38b-aebf57842740"}]: dispatch 2024-09-19T04:56:39.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:39 smithi096 ceph-mon[35085]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e96fcfac-4fff-4a36-b38b-aebf57842740"}]: dispatch 2024-09-19T04:56:39.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:39 smithi096 ceph-mon[35085]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e96fcfac-4fff-4a36-b38b-aebf57842740"}]': finished 2024-09-19T04:56:39.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:39 smithi096 ceph-mon[35085]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T04:56:39.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:39 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:56:39.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:39 smithi096 ceph-mon[35085]: from='client.? 172.21.15.96:0/1848294269' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:56:39.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:39 smithi096 ceph-mon[35085]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:56:39.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:39 smithi043 ceph-mon[30800]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T04:56:39.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:39 smithi043 ceph-mon[30800]: from='client.? 172.21.15.96:0/3210221478' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e96fcfac-4fff-4a36-b38b-aebf57842740"}]: dispatch 2024-09-19T04:56:39.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:39 smithi043 ceph-mon[30800]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e96fcfac-4fff-4a36-b38b-aebf57842740"}]: dispatch 2024-09-19T04:56:39.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:39 smithi043 ceph-mon[30800]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e96fcfac-4fff-4a36-b38b-aebf57842740"}]': finished 2024-09-19T04:56:39.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:39 smithi043 ceph-mon[30800]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T04:56:39.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:39 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:56:39.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:39 smithi043 ceph-mon[30800]: from='client.? 172.21.15.96:0/1848294269' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:56:39.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:39 smithi043 ceph-mon[30800]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:56:41.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:41 smithi151 ceph-mon[35425]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:56:41.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:41 smithi096 ceph-mon[35085]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:56:41.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:41 smithi043 ceph-mon[30800]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:56:43.370 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:43 smithi096 ceph-mon[35085]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:56:43.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:43 smithi151 ceph-mon[35425]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:56:43.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:43 smithi043 ceph-mon[30800]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:56:45.337 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:45 smithi096 ceph-mon[35085]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:56:45.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:45 smithi151 ceph-mon[35425]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:56:45.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:45 smithi043 ceph-mon[30800]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:56:46.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:46 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T04:56:46.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:46 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:46.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:46 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T04:56:46.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:46 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:46.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:46 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T04:56:46.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:46 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:47.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:47 smithi151 ceph-mon[35425]: Deploying daemon osd.1 on smithi096 2024-09-19T04:56:47.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:47 smithi151 ceph-mon[35425]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:56:47.413 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:47 smithi096 ceph-mon[35085]: Deploying daemon osd.1 on smithi096 2024-09-19T04:56:47.414 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:47 smithi096 ceph-mon[35085]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:56:47.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:47 smithi043 ceph-mon[30800]: Deploying daemon osd.1 on smithi096 2024-09-19T04:56:47.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:47 smithi043 ceph-mon[30800]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:56:49.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:49 smithi151 ceph-mon[35425]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:56:49.440 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:49 smithi096 ceph-mon[35085]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:56:49.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:49 smithi043 ceph-mon[30800]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:56:51.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:51 smithi151 ceph-mon[35425]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:56:51.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:51 smithi096 ceph-mon[35085]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:56:51.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:51 smithi043 ceph-mon[30800]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:56:52.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:52 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:56:52.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:52 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:52.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:52 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:52.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:52 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:56:52.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:52 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:52.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:52 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:52.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:52 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:56:52.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:52 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:52.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:52 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:53.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:53 smithi151 ceph-mon[35425]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:56:53.701 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:53 smithi096 ceph-mon[35085]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:56:53.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:53 smithi043 ceph-mon[30800]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:56:55.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:55 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:55.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:55 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:55.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:55 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:55.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:55 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:56:55.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:55 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:55.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:55 smithi096 ceph-mon[35085]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:56:55.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:55 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:55.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:55 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:55.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:55 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:55.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:55 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:56:55.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:55 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:55.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:55 smithi043 ceph-mon[30800]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:56:55.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:55 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:55.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:55 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:55.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:55 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:55.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:55 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:56:55.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:55 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:55.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:55 smithi151 ceph-mon[35425]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:56:56.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:56.119+0000 7f16877fe640 1 -- 172.21.15.96:0/1161473387 <== mgr.14150 v2:172.21.15.43:6800/3167108925 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f169006aef0 con 0x7f167c0a31e0 2024-09-19T04:56:56.121 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 1 on host 'smithi096' 2024-09-19T04:56:56.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:56.122+0000 7f16984e6640 1 -- 172.21.15.96:0/1161473387 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f167c0a31e0 msgr2=0x7f167c0a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:56.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:56.122+0000 7f16984e6640 1 --2- 172.21.15.96:0/1161473387 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f167c0a31e0 0x7f167c0a56a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1680005fd0 tx=0x7f1680005850 comp rx=0 tx=0).stop 2024-09-19T04:56:56.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:56.122+0000 7f16984e6640 1 -- 172.21.15.96:0/1161473387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16901026b0 msgr2=0x7f169019f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:56:56.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:56.122+0000 7f16984e6640 1 --2- 172.21.15.96:0/1161473387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16901026b0 0x7f169019f4a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1688005e30 tx=0x7f1688031ee0 comp rx=0 tx=0).stop 2024-09-19T04:56:56.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:56.122+0000 7f16984e6640 1 -- 172.21.15.96:0/1161473387 shutdown_connections 2024-09-19T04:56:56.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:56.122+0000 7f16984e6640 1 --2- 172.21.15.96:0/1161473387 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f167c0a31e0 0x7f167c0a56a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:56.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:56.122+0000 7f16984e6640 1 --2- 172.21.15.96:0/1161473387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16901026b0 0x7f169019f4a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:56.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:56.122+0000 7f16984e6640 1 --2- 172.21.15.96:0/1161473387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1690101cf0 0x7f1690198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:56.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:56.122+0000 7f16984e6640 1 --2- 172.21.15.96:0/1161473387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1690100af0 0x7f1690197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:56:56.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:56.122+0000 7f16984e6640 1 -- 172.21.15.96:0/1161473387 >> 172.21.15.96:0/1161473387 conn(0x7f16900fc280 msgr2=0x7f169006a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:56:56.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:56.122+0000 7f16984e6640 1 -- 172.21.15.96:0/1161473387 shutdown_connections 2024-09-19T04:56:56.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:56:56.122+0000 7f16984e6640 1 -- 172.21.15.96:0/1161473387 wait complete. 2024-09-19T04:56:56.706 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:56 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:56:56.707 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:56 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:56.707 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:56 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:56.735 DEBUG:teuthology.orchestra.run.smithi096:osd.1> sudo journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@osd.1.service 2024-09-19T04:56:56.741 INFO:tasks.cephadm:Deploying osd.2 on smithi151 with /dev/vg_nvme/lv_4... 2024-09-19T04:56:56.741 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T04:56:56.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:56 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:56:56.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:56 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:56.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:56 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:56.993 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:56:57.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:56 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:56:57.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:56 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:57.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:56 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:57.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:57 smithi151 ceph-mon[35425]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:56:57.713 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:57 smithi096 ceph-mon[35085]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:56:58.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:57 smithi043 ceph-mon[30800]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:56:58.639 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:58 smithi096 ceph-mon[35085]: from='osd.1 [v2:172.21.15.96:6800/2140095130,v1:172.21.15.96:6801/2140095130]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T04:56:58.639 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:58 smithi096 ceph-mon[35085]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T04:56:58.639 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 04:56:58 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-1[39579]: 2024-09-19T04:56:58.316+0000 7fbee6593640 -1 osd.1 0 log_to_monitors true 2024-09-19T04:56:58.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:58 smithi043 ceph-mon[30800]: from='osd.1 [v2:172.21.15.96:6800/2140095130,v1:172.21.15.96:6801/2140095130]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T04:56:58.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:58 smithi043 ceph-mon[30800]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T04:56:58.800 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:58 smithi151 ceph-mon[35425]: from='osd.1 [v2:172.21.15.96:6800/2140095130,v1:172.21.15.96:6801/2140095130]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T04:56:58.800 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:58 smithi151 ceph-mon[35425]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T04:56:59.376 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T04:56:59.403 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi151:vg_nvme/lv_4 2024-09-19T04:56:59.654 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:56:59.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:59 smithi151 ceph-mon[35425]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T04:56:59.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:59 smithi151 ceph-mon[35425]: from='osd.1 [v2:172.21.15.96:6800/2140095130,v1:172.21.15.96:6801/2140095130]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T04:56:59.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:59 smithi151 ceph-mon[35425]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T04:56:59.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:59 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:56:59.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:59 smithi151 ceph-mon[35425]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T04:56:59.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:59 smithi151 ceph-mon[35425]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:56:59.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:59 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:59.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:59 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:59.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:59 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:56:59.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:59 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:59.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:59 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:59.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:59 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:56:59.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:56:59 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:59.948 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:59 smithi096 ceph-mon[35085]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T04:56:59.948 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:59 smithi096 ceph-mon[35085]: from='osd.1 [v2:172.21.15.96:6800/2140095130,v1:172.21.15.96:6801/2140095130]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T04:56:59.948 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:59 smithi096 ceph-mon[35085]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T04:56:59.948 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:59 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:56:59.948 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:59 smithi096 ceph-mon[35085]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T04:56:59.948 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:59 smithi096 ceph-mon[35085]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:56:59.949 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:59 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:59.949 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:59 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:59.949 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:59 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:56:59.949 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:59 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:56:59.949 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:59 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:56:59.949 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:59 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:56:59.949 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:56:59 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:00.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:59 smithi043 ceph-mon[30800]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T04:57:00.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:59 smithi043 ceph-mon[30800]: from='osd.1 [v2:172.21.15.96:6800/2140095130,v1:172.21.15.96:6801/2140095130]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T04:57:00.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:59 smithi043 ceph-mon[30800]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T04:57:00.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:59 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:57:00.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:59 smithi043 ceph-mon[30800]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T04:57:00.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:59 smithi043 ceph-mon[30800]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:57:00.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:59 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:00.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:59 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:00.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:59 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:00.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:59 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:00.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:59 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:57:00.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:59 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:57:00.032 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:56:59 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:00.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:00.153+0000 7fee9f671640 1 -- 172.21.15.151:0/428055116 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee98104f90 msgr2=0x7fee98107380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:00.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:00.153+0000 7fee9f671640 1 --2- 172.21.15.151:0/428055116 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee98104f90 0x7fee98107380 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fee94009900 tx=0x7fee9402a410 comp rx=0 tx=0).stop 2024-09-19T04:57:00.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:00.153+0000 7fee9f671640 1 -- 172.21.15.151:0/428055116 shutdown_connections 2024-09-19T04:57:00.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:00.153+0000 7fee9f671640 1 --2- 172.21.15.151:0/428055116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee981078c0 0x7fee98109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:00.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:00.153+0000 7fee9f671640 1 --2- 172.21.15.151:0/428055116 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee98104f90 0x7fee98107380 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:00.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:00.153+0000 7fee9f671640 1 --2- 172.21.15.151:0/428055116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee98102660 0x7fee98104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:00.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:00.153+0000 7fee9f671640 1 -- 172.21.15.151:0/428055116 >> 172.21.15.151:0/428055116 conn(0x7fee980fc280 msgr2=0x7fee980fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:00.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:00.153+0000 7fee9f671640 1 -- 172.21.15.151:0/428055116 shutdown_connections 2024-09-19T04:57:00.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:00.153+0000 7fee9f671640 1 -- 172.21.15.151:0/428055116 wait complete. 2024-09-19T04:57:00.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:00.154+0000 7fee9f671640 1 Processor -- start 2024-09-19T04:57:00.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:00.154+0000 7fee9f671640 1 -- start start 2024-09-19T04:57:00.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:00.154+0000 7fee9f671640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee98102660 0x7fee98197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:00.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:00.154+0000 7fee9f671640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee98104f90 0x7fee981984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:00.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:00.154+0000 7fee9f671640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee981078c0 0x7fee9819f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:00.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:00.154+0000 7fee9f671640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fee9810d8b0 con 0x7fee981078c0 2024-09-19T04:57:00.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:00.154+0000 7fee9f671640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fee9810d730 con 0x7fee98102660 2024-09-19T04:57:00.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:00.154+0000 7fee9f671640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fee9810da30 con 0x7fee98104f90 2024-09-19T04:57:00.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:00.154+0000 7fee9cc0c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee98104f90 0x7fee981984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:00.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:00.154+0000 7fee9cc0c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee98104f90 0x7fee981984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:58194/0 (socket says 172.21.15.151:58194) 2024-09-19T04:57:00.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:00.154+0000 7fee9cc0c640 1 -- 172.21.15.151:0/186416627 learned_addr learned my addr 172.21.15.151:0/186416627 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-19T04:57:00.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:00.154+0000 7fee9d40d640 1 --2- 172.21.15.151:0/186416627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee98102660 0x7fee98197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:00.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:00.154+0000 7fee9dc0e640 1 --2- 172.21.15.151:0/186416627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee981078c0 0x7fee9819f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:00.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:00.155+0000 7fee9cc0c640 1 -- 172.21.15.151:0/186416627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee98102660 msgr2=0x7fee98197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:00.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:00.155+0000 7fee9cc0c640 1 --2- 172.21.15.151:0/186416627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee98102660 0x7fee98197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:00.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:00.155+0000 7fee9cc0c640 1 -- 172.21.15.151:0/186416627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee981078c0 msgr2=0x7fee9819f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:00.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:00.155+0000 7fee9cc0c640 1 --2- 172.21.15.151:0/186416627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee981078c0 0x7fee9819f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:00.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:00.155+0000 7fee9cc0c640 1 -- 172.21.15.151:0/186416627 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee9819fbc0 con 0x7fee98104f90 2024-09-19T04:57:00.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:00.155+0000 7fee9cc0c640 1 --2- 172.21.15.151:0/186416627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee98104f90 0x7fee981984d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fee94031610 tx=0x7fee940043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:00.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:00.155+0000 7fee8e7fc640 1 -- 172.21.15.151:0/186416627 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9403f030 con 0x7fee98104f90 2024-09-19T04:57:00.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:00.155+0000 7fee8e7fc640 1 -- 172.21.15.151:0/186416627 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee94003ce0 con 0x7fee98104f90 2024-09-19T04:57:00.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:00.155+0000 7fee9f671640 1 -- 172.21.15.151:0/186416627 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee9819fe50 con 0x7fee98104f90 2024-09-19T04:57:00.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:00.155+0000 7fee8e7fc640 1 -- 172.21.15.151:0/186416627 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee94035070 con 0x7fee98104f90 2024-09-19T04:57:00.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:00.155+0000 7fee9f671640 1 -- 172.21.15.151:0/186416627 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee981a02b0 con 0x7fee98104f90 2024-09-19T04:57:00.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:00.156+0000 7fee9f671640 1 -- 172.21.15.151:0/186416627 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee980fdd70 con 0x7fee98104f90 2024-09-19T04:57:00.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:00.157+0000 7fee8e7fc640 1 -- 172.21.15.151:0/186416627 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7fee94042030 con 0x7fee98104f90 2024-09-19T04:57:00.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:00.158+0000 7fee8e7fc640 1 --2- 172.21.15.151:0/186416627 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fee840a3260 0x7fee840a5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:00.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:00.158+0000 7fee8e7fc640 1 -- 172.21.15.151:0/186416627 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7fee940ee650 con 0x7fee98104f90 2024-09-19T04:57:00.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:00.158+0000 7fee9d40d640 1 --2- 172.21.15.151:0/186416627 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fee840a3260 0x7fee840a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:00.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:00.159+0000 7fee9d40d640 1 --2- 172.21.15.151:0/186416627 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fee840a3260 0x7fee840a5720 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fee88005d00 tx=0x7fee8800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:00.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:00.162+0000 7fee8e7fc640 1 -- 172.21.15.151:0/186416627 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fee940bcce0 con 0x7fee98104f90 2024-09-19T04:57:00.223 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 04:56:59 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-1[39579]: 2024-09-19T04:56:59.946+0000 7fbee2d2a640 -1 osd.1 0 waiting for initial osdmap 2024-09-19T04:57:00.223 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 04:56:59 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-1[39579]: 2024-09-19T04:56:59.950+0000 7fbeda336640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:57:00.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:00.337+0000 7fee9f671640 1 -- 172.21.15.151:0/186416627 --> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fee98062630 con 0x7fee840a3260 2024-09-19T04:57:00.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:00 smithi151 ceph-mon[35425]: Detected new or changed devices on smithi096 2024-09-19T04:57:00.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:00 smithi151 ceph-mon[35425]: Adjusting osd_memory_target on smithi096 to 17088M 2024-09-19T04:57:00.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:00 smithi151 ceph-mon[35425]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-19T04:57:00.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:00 smithi151 ceph-mon[35425]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T04:57:00.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:00 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:57:00.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:00 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:57:00.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:00 smithi151 ceph-mon[35425]: from='osd.1 ' entity='osd.1' 2024-09-19T04:57:00.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:00 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:57:00.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:00 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:57:00.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:00 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:57:00.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:00 smithi096 ceph-mon[35085]: Detected new or changed devices on smithi096 2024-09-19T04:57:00.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:00 smithi096 ceph-mon[35085]: Adjusting osd_memory_target on smithi096 to 17088M 2024-09-19T04:57:00.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:00 smithi096 ceph-mon[35085]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-19T04:57:00.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:00 smithi096 ceph-mon[35085]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T04:57:00.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:00 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:57:00.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:00 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:57:00.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:00 smithi096 ceph-mon[35085]: from='osd.1 ' entity='osd.1' 2024-09-19T04:57:00.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:00 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:57:00.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:00 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:57:00.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:00 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:57:01.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:00 smithi043 ceph-mon[30800]: Detected new or changed devices on smithi096 2024-09-19T04:57:01.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:00 smithi043 ceph-mon[30800]: Adjusting osd_memory_target on smithi096 to 17088M 2024-09-19T04:57:01.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:00 smithi043 ceph-mon[30800]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-19T04:57:01.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:00 smithi043 ceph-mon[30800]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T04:57:01.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:00 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:57:01.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:00 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:57:01.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:00 smithi043 ceph-mon[30800]: from='osd.1 ' entity='osd.1' 2024-09-19T04:57:01.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:00 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:57:01.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:00 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:57:01.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:00 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:57:01.833 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:01 smithi151 ceph-mon[35425]: purged_snaps scrub starts 2024-09-19T04:57:01.833 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:01 smithi151 ceph-mon[35425]: purged_snaps scrub ok 2024-09-19T04:57:01.833 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:01 smithi151 ceph-mon[35425]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:57:01.834 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:01 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:57:01.834 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:01 smithi151 ceph-mon[35425]: osd.1 [v2:172.21.15.96:6800/2140095130,v1:172.21.15.96:6801/2140095130] boot 2024-09-19T04:57:01.834 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:01 smithi151 ceph-mon[35425]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T04:57:01.834 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:01 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:57:01.834 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:01 smithi151 ceph-mon[35425]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:57:01.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:01 smithi096 ceph-mon[35085]: purged_snaps scrub starts 2024-09-19T04:57:01.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:01 smithi096 ceph-mon[35085]: purged_snaps scrub ok 2024-09-19T04:57:01.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:01 smithi096 ceph-mon[35085]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:57:01.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:01 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:57:01.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:01 smithi096 ceph-mon[35085]: osd.1 [v2:172.21.15.96:6800/2140095130,v1:172.21.15.96:6801/2140095130] boot 2024-09-19T04:57:01.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:01 smithi096 ceph-mon[35085]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T04:57:01.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:01 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:57:01.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:01 smithi096 ceph-mon[35085]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:57:02.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:01 smithi043 ceph-mon[30800]: purged_snaps scrub starts 2024-09-19T04:57:02.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:01 smithi043 ceph-mon[30800]: purged_snaps scrub ok 2024-09-19T04:57:02.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:01 smithi043 ceph-mon[30800]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:57:02.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:01 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:57:02.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:01 smithi043 ceph-mon[30800]: osd.1 [v2:172.21.15.96:6800/2140095130,v1:172.21.15.96:6801/2140095130] boot 2024-09-19T04:57:02.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:01 smithi043 ceph-mon[30800]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T04:57:02.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:01 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T04:57:02.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:01 smithi043 ceph-mon[30800]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T04:57:03.221 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:02 smithi151 ceph-mon[35425]: osdmap e14: 2 total, 2 up, 2 in 2024-09-19T04:57:03.221 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:02 smithi151 ceph-mon[35425]: from='client.? 172.21.15.151:0/965408439' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b659869e-114a-4f56-8aac-a2ed602b4058"}]: dispatch 2024-09-19T04:57:03.221 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:02 smithi151 ceph-mon[35425]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b659869e-114a-4f56-8aac-a2ed602b4058"}]: dispatch 2024-09-19T04:57:03.221 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:02 smithi151 ceph-mon[35425]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b659869e-114a-4f56-8aac-a2ed602b4058"}]': finished 2024-09-19T04:57:03.221 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:02 smithi151 ceph-mon[35425]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T04:57:03.221 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:02 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:57:03.222 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:02 smithi096 ceph-mon[35085]: osdmap e14: 2 total, 2 up, 2 in 2024-09-19T04:57:03.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:02 smithi096 ceph-mon[35085]: from='client.? 172.21.15.151:0/965408439' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b659869e-114a-4f56-8aac-a2ed602b4058"}]: dispatch 2024-09-19T04:57:03.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:02 smithi096 ceph-mon[35085]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b659869e-114a-4f56-8aac-a2ed602b4058"}]: dispatch 2024-09-19T04:57:03.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:02 smithi096 ceph-mon[35085]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b659869e-114a-4f56-8aac-a2ed602b4058"}]': finished 2024-09-19T04:57:03.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:02 smithi096 ceph-mon[35085]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T04:57:03.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:02 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:57:03.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:02 smithi043 ceph-mon[30800]: osdmap e14: 2 total, 2 up, 2 in 2024-09-19T04:57:03.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:02 smithi043 ceph-mon[30800]: from='client.? 172.21.15.151:0/965408439' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b659869e-114a-4f56-8aac-a2ed602b4058"}]: dispatch 2024-09-19T04:57:03.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:02 smithi043 ceph-mon[30800]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b659869e-114a-4f56-8aac-a2ed602b4058"}]: dispatch 2024-09-19T04:57:03.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:02 smithi043 ceph-mon[30800]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b659869e-114a-4f56-8aac-a2ed602b4058"}]': finished 2024-09-19T04:57:03.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:02 smithi043 ceph-mon[30800]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T04:57:03.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:02 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:57:04.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:03 smithi096 ceph-mon[35085]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:57:04.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:03 smithi096 ceph-mon[35085]: from='client.? 172.21.15.151:0/1354016204' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:57:04.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:03 smithi043 ceph-mon[30800]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:57:04.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:03 smithi043 ceph-mon[30800]: from='client.? 172.21.15.151:0/1354016204' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:57:04.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:03 smithi151 ceph-mon[35425]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:57:04.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:03 smithi151 ceph-mon[35425]: from='client.? 172.21.15.151:0/1354016204' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:57:05.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:05 smithi151 ceph-mon[35425]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:57:05.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:05 smithi096 ceph-mon[35085]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:57:05.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:05 smithi043 ceph-mon[30800]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:57:07.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:07 smithi151 ceph-mon[35425]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:57:07.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:07 smithi096 ceph-mon[35085]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:57:07.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:07 smithi043 ceph-mon[30800]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:57:09.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:09 smithi151 ceph-mon[35425]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:57:09.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:09 smithi096 ceph-mon[35085]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:57:09.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:09 smithi043 ceph-mon[30800]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:57:11.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:11 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T04:57:11.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:11 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:57:11.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:11 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T04:57:11.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:11 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:57:11.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:11 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T04:57:11.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:11 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:57:12.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:12 smithi043 ceph-mon[30800]: Deploying daemon osd.2 on smithi151 2024-09-19T04:57:12.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:12 smithi043 ceph-mon[30800]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:57:12.301 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:12 smithi151 ceph-mon[35425]: Deploying daemon osd.2 on smithi151 2024-09-19T04:57:12.301 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:12 smithi151 ceph-mon[35425]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:57:12.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:12 smithi096 ceph-mon[35085]: Deploying daemon osd.2 on smithi151 2024-09-19T04:57:12.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:12 smithi096 ceph-mon[35085]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:57:13.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:13 smithi151 ceph-mon[35425]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:57:13.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:13 smithi096 ceph-mon[35085]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:57:13.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:13 smithi043 ceph-mon[30800]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:57:15.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:15 smithi151 ceph-mon[35425]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:57:15.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:15 smithi096 ceph-mon[35085]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:57:15.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:15 smithi043 ceph-mon[30800]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:57:17.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:17 smithi151 ceph-mon[35425]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:57:17.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:17 smithi096 ceph-mon[35085]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:57:17.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:17 smithi043 ceph-mon[30800]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:57:18.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:18 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:57:18.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:18 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:18.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:18 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:18.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:18 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:57:18.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:18 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:18.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:18 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:18.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:18 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:57:18.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:18 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:18.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:18 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:19.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:19 smithi151 ceph-mon[35425]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:57:19.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:19 smithi096 ceph-mon[35085]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:57:19.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:19 smithi043 ceph-mon[30800]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:57:20.985 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:20 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:20.986 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:20 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:20.986 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:20 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:57:20.986 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:20 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:57:20.986 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:20 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:21.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:21.024+0000 7fee8e7fc640 1 -- 172.21.15.151:0/186416627 <== mgr.14150 v2:172.21.15.43:6800/3167108925 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fee98062630 con 0x7fee840a3260 2024-09-19T04:57:21.026 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 2 on host 'smithi151' 2024-09-19T04:57:21.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:21.027+0000 7fee9f671640 1 -- 172.21.15.151:0/186416627 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fee840a3260 msgr2=0x7fee840a5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:21.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:21.027+0000 7fee9f671640 1 --2- 172.21.15.151:0/186416627 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fee840a3260 0x7fee840a5720 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fee88005d00 tx=0x7fee8800b410 comp rx=0 tx=0).stop 2024-09-19T04:57:21.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:21.027+0000 7fee9f671640 1 -- 172.21.15.151:0/186416627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee98104f90 msgr2=0x7fee981984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:21.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:21.027+0000 7fee9f671640 1 --2- 172.21.15.151:0/186416627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee98104f90 0x7fee981984d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fee94031610 tx=0x7fee940043b0 comp rx=0 tx=0).stop 2024-09-19T04:57:21.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:21.027+0000 7fee9f671640 1 -- 172.21.15.151:0/186416627 shutdown_connections 2024-09-19T04:57:21.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:21.027+0000 7fee9f671640 1 --2- 172.21.15.151:0/186416627 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fee840a3260 0x7fee840a5720 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:21.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:21.027+0000 7fee9f671640 1 --2- 172.21.15.151:0/186416627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee981078c0 0x7fee9819f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:21.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:21.027+0000 7fee9f671640 1 --2- 172.21.15.151:0/186416627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee98104f90 0x7fee981984d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:21.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:21.027+0000 7fee9f671640 1 --2- 172.21.15.151:0/186416627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee98102660 0x7fee98197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:21.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:21.027+0000 7fee9f671640 1 -- 172.21.15.151:0/186416627 >> 172.21.15.151:0/186416627 conn(0x7fee980fc280 msgr2=0x7fee98103040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:21.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:21.027+0000 7fee9f671640 1 -- 172.21.15.151:0/186416627 shutdown_connections 2024-09-19T04:57:21.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:57:21.027+0000 7fee9f671640 1 -- 172.21.15.151:0/186416627 wait complete. 2024-09-19T04:57:21.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:20 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:21.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:20 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:21.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:20 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:57:21.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:20 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:57:21.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:20 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:21.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:20 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:21.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:20 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:21.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:20 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:57:21.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:20 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:57:21.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:20 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:21.695 DEBUG:teuthology.orchestra.run.smithi151:osd.2> sudo journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@osd.2.service 2024-09-19T04:57:21.697 INFO:tasks.cephadm:Deploying osd.3 on smithi043 with /dev/vg_nvme/lv_3... 2024-09-19T04:57:21.697 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T04:57:21.934 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:21 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:57:21.934 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:21 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:21.934 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:21 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:21.934 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:21 smithi151 ceph-mon[35425]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:57:22.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:21 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:57:22.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:21 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:22.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:21 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:22.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:21 smithi043 ceph-mon[30800]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:57:22.080 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:57:22.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:21 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:57:22.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:21 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:22.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:21 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:22.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:21 smithi096 ceph-mon[35085]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:57:23.399 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:23 smithi043 ceph-mon[30800]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:57:23.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:23 smithi151 ceph-mon[35425]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:57:23.405 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 04:57:23 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-2[40388]: 2024-09-19T04:57:23.286+0000 7ff6c7c91640 -1 osd.2 0 log_to_monitors true 2024-09-19T04:57:23.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:23 smithi096 ceph-mon[35085]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:57:24.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:24 smithi043 ceph-mon[30800]: from='osd.2 [v2:172.21.15.151:6800/2047147996,v1:172.21.15.151:6801/2047147996]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T04:57:24.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:24 smithi043 ceph-mon[30800]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T04:57:24.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:24 smithi151 ceph-mon[35425]: from='osd.2 [v2:172.21.15.151:6800/2047147996,v1:172.21.15.151:6801/2047147996]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T04:57:24.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:24 smithi151 ceph-mon[35425]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T04:57:24.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:24 smithi096 ceph-mon[35085]: from='osd.2 [v2:172.21.15.151:6800/2047147996,v1:172.21.15.151:6801/2047147996]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T04:57:24.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:24 smithi096 ceph-mon[35085]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T04:57:25.339 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T04:57:25.382 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi043:vg_nvme/lv_3 2024-09-19T04:57:25.405 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 04:57:25 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-2[40388]: 2024-09-19T04:57:25.151+0000 7ff6c3c14640 -1 osd.2 0 waiting for initial osdmap 2024-09-19T04:57:25.405 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 04:57:25 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-2[40388]: 2024-09-19T04:57:25.155+0000 7ff6bfa3c640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:57:25.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:25 smithi151 ceph-mon[35425]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T04:57:25.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:25 smithi151 ceph-mon[35425]: from='osd.2 [v2:172.21.15.151:6800/2047147996,v1:172.21.15.151:6801/2047147996]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T04:57:25.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:25 smithi151 ceph-mon[35425]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T04:57:25.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:25 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:57:25.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:25 smithi151 ceph-mon[35425]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T04:57:25.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:25 smithi151 ceph-mon[35425]: Detected new or changed devices on smithi151 2024-09-19T04:57:25.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:25 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:25.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:25 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:25.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:25 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:25.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:25 smithi151 ceph-mon[35425]: Adjusting osd_memory_target on smithi151 to 17088M 2024-09-19T04:57:25.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:25 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:25.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:25 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:57:25.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:25 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:57:25.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:25 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:25.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:25 smithi151 ceph-mon[35425]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:57:25.432 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:25 smithi043 ceph-mon[30800]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T04:57:25.432 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:25 smithi043 ceph-mon[30800]: from='osd.2 [v2:172.21.15.151:6800/2047147996,v1:172.21.15.151:6801/2047147996]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T04:57:25.432 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:25 smithi043 ceph-mon[30800]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T04:57:25.432 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:25 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:57:25.432 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:25 smithi043 ceph-mon[30800]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T04:57:25.432 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:25 smithi043 ceph-mon[30800]: Detected new or changed devices on smithi151 2024-09-19T04:57:25.432 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:25 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:25.432 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:25 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:25.432 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:25 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:25.432 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:25 smithi043 ceph-mon[30800]: Adjusting osd_memory_target on smithi151 to 17088M 2024-09-19T04:57:25.432 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:25 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:25.433 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:25 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:57:25.433 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:25 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:57:25.433 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:25 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:25.433 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:25 smithi043 ceph-mon[30800]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:57:25.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:25 smithi096 ceph-mon[35085]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T04:57:25.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:25 smithi096 ceph-mon[35085]: from='osd.2 [v2:172.21.15.151:6800/2047147996,v1:172.21.15.151:6801/2047147996]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T04:57:25.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:25 smithi096 ceph-mon[35085]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T04:57:25.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:25 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:57:25.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:25 smithi096 ceph-mon[35085]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T04:57:25.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:25 smithi096 ceph-mon[35085]: Detected new or changed devices on smithi151 2024-09-19T04:57:25.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:25 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:25.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:25 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:25.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:25 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:25.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:25 smithi096 ceph-mon[35085]: Adjusting osd_memory_target on smithi151 to 17088M 2024-09-19T04:57:25.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:25 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:25.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:25 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:57:25.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:25 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:57:25.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:25 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:25.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:25 smithi096 ceph-mon[35085]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:57:25.762 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:57:26.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:26.291+0000 7f1376a00640 1 -- 172.21.15.43:0/3000784392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1370105810 msgr2=0x7f137010c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:26.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:26.291+0000 7f1376a00640 1 --2- 172.21.15.43:0/3000784392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1370105810 0x7f137010c0a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1368009ca0 tx=0x7f136802a8c0 comp rx=0 tx=0).stop 2024-09-19T04:57:26.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:26.291+0000 7f1376a00640 1 -- 172.21.15.43:0/3000784392 shutdown_connections 2024-09-19T04:57:26.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:26.291+0000 7f1376a00640 1 --2- 172.21.15.43:0/3000784392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1370105810 0x7f137010c0a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:26.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:26.291+0000 7f1376a00640 1 --2- 172.21.15.43:0/3000784392 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1370104e50 0x7f13701052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:26.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:26.291+0000 7f1376a00640 1 --2- 172.21.15.43:0/3000784392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1370103c50 0x7f1370104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:26.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:26.291+0000 7f1376a00640 1 -- 172.21.15.43:0/3000784392 >> 172.21.15.43:0/3000784392 conn(0x7f13700ff3e0 msgr2=0x7f1370101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:26.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:26.291+0000 7f1376a00640 1 -- 172.21.15.43:0/3000784392 shutdown_connections 2024-09-19T04:57:26.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:26.291+0000 7f1376a00640 1 -- 172.21.15.43:0/3000784392 wait complete. 2024-09-19T04:57:26.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:26.292+0000 7f1376a00640 1 Processor -- start 2024-09-19T04:57:26.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:26.292+0000 7f1376a00640 1 -- start start 2024-09-19T04:57:26.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:26.293+0000 7f1376a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1370103c50 0x7f137019a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:26.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:26.293+0000 7f1376a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1370104e50 0x7f137019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:26.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:26.293+0000 7f1376a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1370105810 0x7f13701a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:26.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:26.293+0000 7f1376a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f137010fa20 con 0x7f1370103c50 2024-09-19T04:57:26.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:26.293+0000 7f1376a00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f137010f8a0 con 0x7f1370104e50 2024-09-19T04:57:26.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:26.293+0000 7f1376a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f137010fba0 con 0x7f1370105810 2024-09-19T04:57:26.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:26.293+0000 7f1375600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1370103c50 0x7f137019a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:26.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:26.293+0000 7f1375600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1370103c50 0x7f137019a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37994/0 (socket says 172.21.15.43:37994) 2024-09-19T04:57:26.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:26.293+0000 7f1375600640 1 -- 172.21.15.43:0/2192381220 learned_addr learned my addr 172.21.15.43:0/2192381220 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:57:26.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:26.293+0000 7f1374c00640 1 --2- 172.21.15.43:0/2192381220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1370104e50 0x7f137019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:26.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:26.293+0000 7f1376000640 1 --2- 172.21.15.43:0/2192381220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1370105810 0x7f13701a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:26.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:26.293+0000 7f1375600640 1 -- 172.21.15.43:0/2192381220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1370105810 msgr2=0x7f13701a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:26.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:26.293+0000 7f1375600640 1 --2- 172.21.15.43:0/2192381220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1370105810 0x7f13701a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:26.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:26.293+0000 7f1375600640 1 -- 172.21.15.43:0/2192381220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1370104e50 msgr2=0x7f137019a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:26.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:26.293+0000 7f1375600640 1 --2- 172.21.15.43:0/2192381220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1370104e50 0x7f137019a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:26.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:26.293+0000 7f1375600640 1 -- 172.21.15.43:0/2192381220 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13701a1c90 con 0x7f1370103c50 2024-09-19T04:57:26.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:26.294+0000 7f1375600640 1 --2- 172.21.15.43:0/2192381220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1370103c50 0x7f137019a060 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f136400b450 tx=0x7f136400b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:26.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:26.294+0000 7f136e000640 1 -- 172.21.15.43:0/2192381220 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1364015020 con 0x7f1370103c50 2024-09-19T04:57:26.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:26.294+0000 7f136e000640 1 -- 172.21.15.43:0/2192381220 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1364011d10 con 0x7f1370103c50 2024-09-19T04:57:26.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:26.294+0000 7f1376a00640 1 -- 172.21.15.43:0/2192381220 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13701a1f80 con 0x7f1370103c50 2024-09-19T04:57:26.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:26.294+0000 7f136e000640 1 -- 172.21.15.43:0/2192381220 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136400f2e0 con 0x7f1370103c50 2024-09-19T04:57:26.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:26.294+0000 7f1376a00640 1 -- 172.21.15.43:0/2192381220 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13701a2450 con 0x7f1370103c50 2024-09-19T04:57:26.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:26.295+0000 7f1376a00640 1 -- 172.21.15.43:0/2192381220 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f137010aff0 con 0x7f1370103c50 2024-09-19T04:57:26.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:26.297+0000 7f136e000640 1 -- 172.21.15.43:0/2192381220 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7f1364018020 con 0x7f1370103c50 2024-09-19T04:57:26.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:26.298+0000 7f136e000640 1 --2- 172.21.15.43:0/2192381220 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f13580a31e0 0x7f13580a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:26.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:26.299+0000 7f136e000640 1 -- 172.21.15.43:0/2192381220 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2851+0+0 (secure 0 0 0) 0x7f13640cdb50 con 0x7f1370103c50 2024-09-19T04:57:26.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:26.299+0000 7f1374c00640 1 --2- 172.21.15.43:0/2192381220 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f13580a31e0 0x7f13580a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:26.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:26.299+0000 7f1374c00640 1 --2- 172.21.15.43:0/2192381220 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f13580a31e0 0x7f13580a56a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f137019b580 tx=0x7f135c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:26.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:26.303+0000 7f136e000640 1 -- 172.21.15.43:0/2192381220 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f136401f020 con 0x7f1370103c50 2024-09-19T04:57:26.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:26 smithi096 ceph-mon[35085]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-19T04:57:26.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:26 smithi096 ceph-mon[35085]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T04:57:26.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:26 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:57:26.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:26 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:57:26.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:26.498+0000 7f1376a00640 1 -- 172.21.15.43:0/2192381220 --> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f1370062630 con 0x7f13580a31e0 2024-09-19T04:57:26.501 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:26 smithi043 ceph-mon[30800]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-19T04:57:26.501 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:26 smithi043 ceph-mon[30800]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T04:57:26.501 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:26 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:57:26.501 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:26 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:57:26.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:26 smithi151 ceph-mon[35425]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-19T04:57:26.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:26 smithi151 ceph-mon[35425]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T04:57:26.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:26 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:57:26.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:26 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:57:27.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:27 smithi043 ceph-mon[30800]: purged_snaps scrub starts 2024-09-19T04:57:27.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:27 smithi043 ceph-mon[30800]: purged_snaps scrub ok 2024-09-19T04:57:27.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:27 smithi043 ceph-mon[30800]: osd.2 [v2:172.21.15.151:6800/2047147996,v1:172.21.15.151:6801/2047147996] boot 2024-09-19T04:57:27.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:27 smithi043 ceph-mon[30800]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T04:57:27.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:27 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:57:27.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:27 smithi043 ceph-mon[30800]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:57:27.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:27 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:57:27.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:27 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:57:27.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:27 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:57:27.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:27 smithi043 ceph-mon[30800]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:57:27.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:27 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:57:27.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:27 smithi151 ceph-mon[35425]: purged_snaps scrub starts 2024-09-19T04:57:27.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:27 smithi151 ceph-mon[35425]: purged_snaps scrub ok 2024-09-19T04:57:27.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:27 smithi151 ceph-mon[35425]: osd.2 [v2:172.21.15.151:6800/2047147996,v1:172.21.15.151:6801/2047147996] boot 2024-09-19T04:57:27.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:27 smithi151 ceph-mon[35425]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T04:57:27.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:27 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:57:27.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:27 smithi151 ceph-mon[35425]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:57:27.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:27 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:57:27.906 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:27 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:57:27.906 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:27 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:57:27.906 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:27 smithi151 ceph-mon[35425]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:57:27.906 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:27 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:57:27.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:27 smithi096 ceph-mon[35085]: purged_snaps scrub starts 2024-09-19T04:57:27.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:27 smithi096 ceph-mon[35085]: purged_snaps scrub ok 2024-09-19T04:57:27.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:27 smithi096 ceph-mon[35085]: osd.2 [v2:172.21.15.151:6800/2047147996,v1:172.21.15.151:6801/2047147996] boot 2024-09-19T04:57:27.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:27 smithi096 ceph-mon[35085]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T04:57:27.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:27 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T04:57:27.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:27 smithi096 ceph-mon[35085]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:57:27.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:27 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:57:27.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:27 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:57:27.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:27 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:57:27.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:27 smithi096 ceph-mon[35085]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-19T04:57:27.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:27 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:57:28.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:28 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T04:57:28.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:28 smithi096 ceph-mon[35085]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T04:57:28.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:28 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:57:29.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:28 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T04:57:29.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:28 smithi043 ceph-mon[30800]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T04:57:29.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:28 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:57:29.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:28 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T04:57:29.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:28 smithi151 ceph-mon[35425]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T04:57:29.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:28 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T04:57:29.530 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 19 04:57:29 smithi043 sudo[46786]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T04:57:29.530 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 19 04:57:29 smithi043 sudo[46786]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:57:29.531 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 19 04:57:29 smithi043 sudo[46786]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:57:29.531 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 19 04:57:29 smithi043 sudo[46786]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:57:29.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:29.668+0000 7f136e000640 1 -- 172.21.15.43:0/2192381220 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f1364097d70 con 0x7f1370103c50 2024-09-19T04:57:29.950 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 19 04:57:29 smithi043 sudo[46820]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T04:57:30.003 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 19 04:57:29 smithi043 sudo[46820]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:57:30.004 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 19 04:57:29 smithi043 sudo[46820]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:57:30.004 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 19 04:57:29 smithi043 sudo[46820]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:57:30.004 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:29 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T04:57:30.004 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:29 smithi043 ceph-mon[30800]: osdmap e20: 3 total, 3 up, 3 in 2024-09-19T04:57:30.004 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:29 smithi043 ceph-mon[30800]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:57:30.004 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:29 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/2376081529' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7be64b2c-e45c-493d-a610-4ce41196061f"}]: dispatch 2024-09-19T04:57:30.004 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:29 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/2376081529' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7be64b2c-e45c-493d-a610-4ce41196061f"}]': finished 2024-09-19T04:57:30.004 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:29 smithi043 ceph-mon[30800]: osdmap e21: 4 total, 3 up, 4 in 2024-09-19T04:57:30.005 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:29 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:57:30.005 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:29 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T04:57:30.005 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:29 smithi096 ceph-mon[35085]: osdmap e20: 3 total, 3 up, 3 in 2024-09-19T04:57:30.005 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:29 smithi096 ceph-mon[35085]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:57:30.005 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:29 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/2376081529' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7be64b2c-e45c-493d-a610-4ce41196061f"}]: dispatch 2024-09-19T04:57:30.006 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:29 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/2376081529' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7be64b2c-e45c-493d-a610-4ce41196061f"}]': finished 2024-09-19T04:57:30.006 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:29 smithi096 ceph-mon[35085]: osdmap e21: 4 total, 3 up, 4 in 2024-09-19T04:57:30.006 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:29 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:57:30.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:29 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T04:57:30.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:29 smithi151 ceph-mon[35425]: osdmap e20: 3 total, 3 up, 3 in 2024-09-19T04:57:30.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:29 smithi151 ceph-mon[35425]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:57:30.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:29 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/2376081529' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7be64b2c-e45c-493d-a610-4ce41196061f"}]: dispatch 2024-09-19T04:57:30.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:29 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/2376081529' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7be64b2c-e45c-493d-a610-4ce41196061f"}]': finished 2024-09-19T04:57:30.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:29 smithi151 ceph-mon[35425]: osdmap e21: 4 total, 3 up, 4 in 2024-09-19T04:57:30.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:29 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:57:30.423 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 04:57:30 smithi096 sudo[43305]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T04:57:30.423 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 04:57:30 smithi096 sudo[43305]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:57:30.423 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 04:57:30 smithi096 sudo[43305]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:57:30.423 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 04:57:30 smithi096 sudo[43305]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:57:30.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:30 smithi096 ceph-mon[35085]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T04:57:30.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:30 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:30.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:30 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/3039152101' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:57:30.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:30 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:30.723 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 04:57:30 smithi096 sudo[43308]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T04:57:30.724 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 04:57:30 smithi096 sudo[43308]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:57:30.724 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 04:57:30 smithi096 sudo[43308]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:57:30.724 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 04:57:30 smithi096 sudo[43308]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:57:30.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:30 smithi151 ceph-mon[35425]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T04:57:30.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:30 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:30.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:30 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/3039152101' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:57:30.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:30 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:30.905 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 04:57:30 smithi151 sudo[44016]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T04:57:30.905 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 04:57:30 smithi151 sudo[44016]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:57:30.905 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 04:57:30 smithi151 sudo[44016]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:57:30.905 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 04:57:30 smithi151 sudo[44016]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:57:31.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:30 smithi043 ceph-mon[30800]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T04:57:31.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:30 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:31.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:30 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/3039152101' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:57:31.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:30 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:31.405 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 04:57:30 smithi151 sudo[44019]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T04:57:31.405 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 04:57:30 smithi151 sudo[44019]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:57:31.405 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 04:57:30 smithi151 sudo[44019]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:57:31.405 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 04:57:30 smithi151 sudo[44019]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:57:32.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:31 smithi043 sudo[47260]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T04:57:32.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:31 smithi043 sudo[47260]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:57:32.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:31 smithi043 sudo[47260]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:57:32.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:31 smithi043 sudo[47260]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:57:32.384 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:32 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:32.385 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:32 smithi151 ceph-mon[35425]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:57:32.385 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:32 smithi151 sudo[44022]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T04:57:32.385 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:32 smithi151 sudo[44022]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:57:32.385 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:32 smithi151 sudo[44022]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:57:32.385 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:32 smithi151 sudo[44022]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:57:32.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:32 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:32.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:32 smithi096 ceph-mon[35085]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:57:32.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:32 smithi043 sudo[47270]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T04:57:32.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:32 smithi043 sudo[47270]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:57:32.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:32 smithi043 sudo[47270]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:57:32.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:32 smithi043 sudo[47270]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:57:32.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:32 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:32.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:32 smithi043 ceph-mon[30800]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:57:32.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:32 smithi151 sudo[44025]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T04:57:32.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:32 smithi151 sudo[44025]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:57:32.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:32 smithi151 sudo[44025]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:57:32.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:32 smithi151 sudo[44025]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:57:33.121 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:32 smithi096 sudo[43311]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T04:57:33.121 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:32 smithi096 sudo[43311]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:57:33.121 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:32 smithi096 sudo[43311]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:57:33.121 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:32 smithi096 sudo[43311]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:57:33.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:33 smithi151 ceph-mon[35425]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:57:33.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:33 smithi151 ceph-mon[35425]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:57:33.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:33 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:57:33.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:33 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:57:33.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:33 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:57:33.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:33 smithi151 ceph-mon[35425]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:57:33.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:33 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:57:33.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:33 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:57:33.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:33 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:57:33.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:33 smithi151 ceph-mon[35425]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:57:33.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:33 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:57:33.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:33 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:57:33.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:33 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:57:33.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:33 smithi151 ceph-mon[35425]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:57:33.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:33 smithi096 sudo[43314]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T04:57:33.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:33 smithi096 sudo[43314]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T04:57:33.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:33 smithi096 sudo[43314]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T04:57:33.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:33 smithi096 sudo[43314]: pam_unix(sudo:session): session closed for user root 2024-09-19T04:57:33.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:33 smithi096 ceph-mon[35085]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:57:33.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:33 smithi096 ceph-mon[35085]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:57:33.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:33 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:57:33.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:33 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:57:33.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:33 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:57:33.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:33 smithi096 ceph-mon[35085]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:57:33.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:33 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:57:33.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:33 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:57:33.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:33 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:57:33.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:33 smithi096 ceph-mon[35085]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:57:33.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:33 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:57:33.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:33 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:57:33.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:33 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:57:33.475 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:33 smithi096 ceph-mon[35085]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:57:33.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:33 smithi043 ceph-mon[30800]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:57:33.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:33 smithi043 ceph-mon[30800]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:57:33.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:33 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:57:33.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:33 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:57:33.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:33 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:57:33.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:33 smithi043 ceph-mon[30800]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:57:33.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:33 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:57:33.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:33 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:57:33.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:33 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:57:33.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:33 smithi043 ceph-mon[30800]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:57:33.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:33 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T04:57:33.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:33 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T04:57:33.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:33 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T04:57:33.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:33 smithi043 ceph-mon[30800]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:57:34.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:34 smithi151 ceph-mon[35425]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:57:34.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:34 smithi151 ceph-mon[35425]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:57:34.413 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:34 smithi043 ceph-mon[30800]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:57:34.413 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:34 smithi043 ceph-mon[30800]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:57:34.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:34 smithi096 ceph-mon[35085]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T04:57:34.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:34 smithi096 ceph-mon[35085]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T04:57:35.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:35 smithi151 ceph-mon[35425]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:57:35.459 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:35 smithi043 ceph-mon[30800]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:57:35.472 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:35 smithi096 ceph-mon[35085]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:57:37.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:37 smithi043 ceph-mon[30800]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-09-19T04:57:37.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:37 smithi151 ceph-mon[35425]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-09-19T04:57:37.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:37 smithi096 ceph-mon[35085]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-09-19T04:57:39.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:38 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T04:57:39.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:38 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:57:39.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:38 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T04:57:39.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:38 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:57:39.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:38 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T04:57:39.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:38 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:57:40.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:39 smithi043 ceph-mon[30800]: Deploying daemon osd.3 on smithi043 2024-09-19T04:57:40.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:39 smithi043 ceph-mon[30800]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-19T04:57:40.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:39 smithi151 ceph-mon[35425]: Deploying daemon osd.3 on smithi043 2024-09-19T04:57:40.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:39 smithi151 ceph-mon[35425]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-19T04:57:40.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:39 smithi096 ceph-mon[35085]: Deploying daemon osd.3 on smithi043 2024-09-19T04:57:40.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:39 smithi096 ceph-mon[35085]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-19T04:57:41.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:41 smithi043 ceph-mon[30800]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-19T04:57:41.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:41 smithi151 ceph-mon[35425]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-19T04:57:41.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:41 smithi096 ceph-mon[35085]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-19T04:57:43.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:43 smithi043 ceph-mon[30800]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-19T04:57:43.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:43 smithi151 ceph-mon[35425]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-19T04:57:43.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:43 smithi096 ceph-mon[35085]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-19T04:57:45.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:45 smithi043 ceph-mon[30800]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-19T04:57:45.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:45 smithi151 ceph-mon[35425]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-19T04:57:45.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:45 smithi096 ceph-mon[35085]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-19T04:57:46.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:46 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:57:46.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:46 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:46.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:46 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:46.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:46 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:57:46.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:46 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:46.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:46 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:46.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:46 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:57:46.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:46 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:46.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:46 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:47.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:47 smithi043 ceph-mon[30800]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-19T04:57:47.904 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:47 smithi151 ceph-mon[35425]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-19T04:57:47.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:47 smithi096 ceph-mon[35085]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-19T04:57:49.212 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:49 smithi043 ceph-mon[30800]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:57:49.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:49 smithi151 ceph-mon[35425]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:57:49.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:49 smithi096 ceph-mon[35085]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:57:50.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:50.812+0000 7f136e000640 1 -- 172.21.15.43:0/2192381220 <== mgr.14150 v2:172.21.15.43:6800/3167108925 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1370062630 con 0x7f13580a31e0 2024-09-19T04:57:50.814 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 3 on host 'smithi043' 2024-09-19T04:57:50.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:50.814+0000 7f1376a00640 1 -- 172.21.15.43:0/2192381220 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f13580a31e0 msgr2=0x7f13580a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:50.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:50.814+0000 7f1376a00640 1 --2- 172.21.15.43:0/2192381220 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f13580a31e0 0x7f13580a56a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f137019b580 tx=0x7f135c009110 comp rx=0 tx=0).stop 2024-09-19T04:57:50.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:50.814+0000 7f1376a00640 1 -- 172.21.15.43:0/2192381220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1370103c50 msgr2=0x7f137019a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:50.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:50.814+0000 7f1376a00640 1 --2- 172.21.15.43:0/2192381220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1370103c50 0x7f137019a060 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f136400b450 tx=0x7f136400b850 comp rx=0 tx=0).stop 2024-09-19T04:57:50.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:50.814+0000 7f1376a00640 1 -- 172.21.15.43:0/2192381220 shutdown_connections 2024-09-19T04:57:50.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:50.814+0000 7f1376a00640 1 --2- 172.21.15.43:0/2192381220 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f13580a31e0 0x7f13580a56a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:50.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:50.814+0000 7f1376a00640 1 --2- 172.21.15.43:0/2192381220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1370105810 0x7f13701a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:50.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:50.814+0000 7f1376a00640 1 --2- 172.21.15.43:0/2192381220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1370104e50 0x7f137019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:50.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:50.815+0000 7f1376a00640 1 --2- 172.21.15.43:0/2192381220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1370103c50 0x7f137019a060 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:50.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:50.815+0000 7f1376a00640 1 -- 172.21.15.43:0/2192381220 >> 172.21.15.43:0/2192381220 conn(0x7f13700ff3e0 msgr2=0x7f13701081c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:50.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:50.815+0000 7f1376a00640 1 -- 172.21.15.43:0/2192381220 shutdown_connections 2024-09-19T04:57:50.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:57:50.815+0000 7f1376a00640 1 -- 172.21.15.43:0/2192381220 wait complete. 2024-09-19T04:57:50.904 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:50 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:50.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:50 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:50.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:50 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:57:50.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:50 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:57:50.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:50 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:50.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:50 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:50.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:50 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:50.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:50 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:57:50.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:50 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:57:50.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:50 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:51.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:50 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:51.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:50 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:51.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:50 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:57:51.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:50 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:57:51.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:50 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:51.531 DEBUG:teuthology.orchestra.run.smithi043:osd.3> sudo journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@osd.3.service 2024-09-19T04:57:51.535 INFO:tasks.cephadm:Deploying osd.4 on smithi096 with /dev/vg_nvme/lv_3... 2024-09-19T04:57:51.535 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T04:57:51.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:51 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:57:51.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:51 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:51.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:51 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:51.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:51 smithi043 ceph-mon[30800]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:57:51.788 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:57:51.909 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:51 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:57:51.909 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:51 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:51.910 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:51 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:51.910 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:51 smithi096 ceph-mon[35085]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:57:52.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:51 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:57:52.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:51 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:52.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:51 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:52.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:51 smithi151 ceph-mon[35425]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:57:52.781 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 19 04:57:52 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-3[48868]: 2024-09-19T04:57:52.437+0000 7fdacfa5a640 -1 osd.3 0 log_to_monitors true 2024-09-19T04:57:53.081 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:52 smithi096 ceph-mon[35085]: from='osd.3 [v2:172.21.15.43:6810/1734673762,v1:172.21.15.43:6811/1734673762]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T04:57:53.136 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:52 smithi043 ceph-mon[30800]: from='osd.3 [v2:172.21.15.43:6810/1734673762,v1:172.21.15.43:6811/1734673762]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T04:57:53.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:52 smithi151 ceph-mon[35425]: from='osd.3 [v2:172.21.15.43:6810/1734673762,v1:172.21.15.43:6811/1734673762]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T04:57:53.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:53 smithi096 ceph-mon[35085]: from='osd.3 [v2:172.21.15.43:6810/1734673762,v1:172.21.15.43:6811/1734673762]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T04:57:53.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:53 smithi096 ceph-mon[35085]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T04:57:53.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:53 smithi096 ceph-mon[35085]: from='osd.3 [v2:172.21.15.43:6810/1734673762,v1:172.21.15.43:6811/1734673762]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-19T04:57:53.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:53 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:57:53.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:53 smithi096 ceph-mon[35085]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:57:54.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:53 smithi151 ceph-mon[35425]: from='osd.3 [v2:172.21.15.43:6810/1734673762,v1:172.21.15.43:6811/1734673762]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T04:57:54.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:53 smithi151 ceph-mon[35425]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T04:57:54.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:53 smithi151 ceph-mon[35425]: from='osd.3 [v2:172.21.15.43:6810/1734673762,v1:172.21.15.43:6811/1734673762]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-19T04:57:54.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:53 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:57:54.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:53 smithi151 ceph-mon[35425]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:57:54.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:53 smithi043 ceph-mon[30800]: from='osd.3 [v2:172.21.15.43:6810/1734673762,v1:172.21.15.43:6811/1734673762]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T04:57:54.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:53 smithi043 ceph-mon[30800]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T04:57:54.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:53 smithi043 ceph-mon[30800]: from='osd.3 [v2:172.21.15.43:6810/1734673762,v1:172.21.15.43:6811/1734673762]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-19T04:57:54.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:53 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:57:54.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:53 smithi043 ceph-mon[30800]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-19T04:57:54.304 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-19T04:57:54.331 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi096:vg_nvme/lv_3 2024-09-19T04:57:54.574 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:57:54.780 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 19 04:57:54 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-3[48868]: 2024-09-19T04:57:54.404+0000 7fdaca600640 -1 osd.3 0 waiting for initial osdmap 2024-09-19T04:57:54.781 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 19 04:57:54 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-3[48868]: 2024-09-19T04:57:54.423+0000 7fdac0600640 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:57:54.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:54 smithi096 ceph-mon[35085]: from='osd.3 [v2:172.21.15.43:6810/1734673762,v1:172.21.15.43:6811/1734673762]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-19T04:57:54.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:54 smithi096 ceph-mon[35085]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T04:57:54.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:54 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:57:54.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:54 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:57:54.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:54 smithi096 ceph-mon[35085]: from='osd.3 [v2:172.21.15.43:6810/1734673762,v1:172.21.15.43:6811/1734673762]' entity='osd.3' 2024-09-19T04:57:55.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:57:55.021+0000 7f74907ce640 1 -- 172.21.15.96:0/892182409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74881037c0 msgr2=0x7f748810a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:55.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:57:55.021+0000 7f74907ce640 1 --2- 172.21.15.96:0/892182409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74881037c0 0x7f748810a050 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7480009ca0 tx=0x7f748002a8c0 comp rx=0 tx=0).stop 2024-09-19T04:57:55.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:57:55.021+0000 7f74907ce640 1 -- 172.21.15.96:0/892182409 shutdown_connections 2024-09-19T04:57:55.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:57:55.021+0000 7f74907ce640 1 --2- 172.21.15.96:0/892182409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74881037c0 0x7f748810a050 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:55.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:57:55.021+0000 7f74907ce640 1 --2- 172.21.15.96:0/892182409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7488102e00 0x7f7488103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:55.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:57:55.021+0000 7f74907ce640 1 --2- 172.21.15.96:0/892182409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7488101c00 0x7f7488102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:55.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:57:55.021+0000 7f74907ce640 1 -- 172.21.15.96:0/892182409 >> 172.21.15.96:0/892182409 conn(0x7f74880fd390 msgr2=0x7f74880ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:57:55.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:57:55.021+0000 7f74907ce640 1 -- 172.21.15.96:0/892182409 shutdown_connections 2024-09-19T04:57:55.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:57:55.021+0000 7f74907ce640 1 -- 172.21.15.96:0/892182409 wait complete. 2024-09-19T04:57:55.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:57:55.022+0000 7f74907ce640 1 Processor -- start 2024-09-19T04:57:55.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:57:55.022+0000 7f74907ce640 1 -- start start 2024-09-19T04:57:55.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:57:55.022+0000 7f74907ce640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7488101c00 0x7f7488197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:55.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:57:55.022+0000 7f74907ce640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7488102e00 0x7f7488198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:55.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:57:55.022+0000 7f74907ce640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74881037c0 0x7f748819f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:55.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:57:55.022+0000 7f74907ce640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f748810d980 con 0x7f7488101c00 2024-09-19T04:57:55.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:57:55.022+0000 7f74907ce640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f748810d800 con 0x7f7488102e00 2024-09-19T04:57:55.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:57:55.022+0000 7f74907ce640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f748810db00 con 0x7f74881037c0 2024-09-19T04:57:55.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:57:55.022+0000 7f748dd69640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7488102e00 0x7f7488198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:55.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:57:55.022+0000 7f748dd69640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7488102e00 0x7f7488198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60488/0 (socket says 172.21.15.96:60488) 2024-09-19T04:57:55.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:57:55.022+0000 7f748dd69640 1 -- 172.21.15.96:0/3932384667 learned_addr learned my addr 172.21.15.96:0/3932384667 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-19T04:57:55.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:57:55.022+0000 7f748ed6b640 1 --2- 172.21.15.96:0/3932384667 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74881037c0 0x7f748819f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:55.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:57:55.022+0000 7f748e56a640 1 --2- 172.21.15.96:0/3932384667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7488101c00 0x7f7488197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:55.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:57:55.022+0000 7f748dd69640 1 -- 172.21.15.96:0/3932384667 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74881037c0 msgr2=0x7f748819f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:55.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:57:55.022+0000 7f748dd69640 1 --2- 172.21.15.96:0/3932384667 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74881037c0 0x7f748819f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:55.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:57:55.022+0000 7f748dd69640 1 -- 172.21.15.96:0/3932384667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7488101c00 msgr2=0x7f7488197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:57:55.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:57:55.022+0000 7f748dd69640 1 --2- 172.21.15.96:0/3932384667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7488101c00 0x7f7488197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:57:55.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:57:55.022+0000 7f748dd69640 1 -- 172.21.15.96:0/3932384667 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f748819fb70 con 0x7f7488102e00 2024-09-19T04:57:55.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:57:55.022+0000 7f748e56a640 1 --2- 172.21.15.96:0/3932384667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7488101c00 0x7f7488197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:57:55.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:57:55.023+0000 7f748dd69640 1 --2- 172.21.15.96:0/3932384667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7488102e00 0x7f7488198480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f747800d770 tx=0x7f747800dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:55.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:57:55.023+0000 7f747f7fe640 1 -- 172.21.15.96:0/3932384667 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7478017070 con 0x7f7488102e00 2024-09-19T04:57:55.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:57:55.023+0000 7f747f7fe640 1 -- 172.21.15.96:0/3932384667 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7478005500 con 0x7f7488102e00 2024-09-19T04:57:55.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:57:55.023+0000 7f74907ce640 1 -- 172.21.15.96:0/3932384667 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f748819fe60 con 0x7f7488102e00 2024-09-19T04:57:55.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:57:55.023+0000 7f747f7fe640 1 -- 172.21.15.96:0/3932384667 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7478004e00 con 0x7f7488102e00 2024-09-19T04:57:55.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:57:55.023+0000 7f74907ce640 1 -- 172.21.15.96:0/3932384667 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74881a0390 con 0x7f7488102e00 2024-09-19T04:57:55.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:57:55.024+0000 7f74907ce640 1 -- 172.21.15.96:0/3932384667 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7488108f30 con 0x7f7488102e00 2024-09-19T04:57:55.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:57:55.025+0000 7f747f7fe640 1 -- 172.21.15.96:0/3932384667 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f7478004fa0 con 0x7f7488102e00 2024-09-19T04:57:55.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:57:55.026+0000 7f747f7fe640 1 --2- 172.21.15.96:0/3932384667 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f74740a32f0 0x7f74740a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:57:55.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:57:55.026+0000 7f747f7fe640 1 -- 172.21.15.96:0/3932384667 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 3702+0+0 (secure 0 0 0) 0x7f74780ce120 con 0x7f7488102e00 2024-09-19T04:57:55.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:57:55.026+0000 7f748e56a640 1 --2- 172.21.15.96:0/3932384667 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f74740a32f0 0x7f74740a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:57:55.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:57:55.027+0000 7f748e56a640 1 --2- 172.21.15.96:0/3932384667 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f74740a32f0 0x7f74740a57b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7484006fd0 tx=0x7f7484008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:57:55.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:57:55.030+0000 7f747f7fe640 1 -- 172.21.15.96:0/3932384667 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f747809c040 con 0x7f7488102e00 2024-09-19T04:57:55.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:54 smithi151 ceph-mon[35425]: from='osd.3 [v2:172.21.15.43:6810/1734673762,v1:172.21.15.43:6811/1734673762]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-19T04:57:55.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:54 smithi151 ceph-mon[35425]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T04:57:55.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:54 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:57:55.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:54 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:57:55.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:54 smithi151 ceph-mon[35425]: from='osd.3 [v2:172.21.15.43:6810/1734673762,v1:172.21.15.43:6811/1734673762]' entity='osd.3' 2024-09-19T04:57:55.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:57:55.204+0000 7f74907ce640 1 -- 172.21.15.96:0/3932384667 --> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f7488062630 con 0x7f74740a32f0 2024-09-19T04:57:55.227 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:54 smithi043 ceph-mon[30800]: from='osd.3 [v2:172.21.15.43:6810/1734673762,v1:172.21.15.43:6811/1734673762]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-19T04:57:55.228 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:54 smithi043 ceph-mon[30800]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T04:57:55.228 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:54 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:57:55.228 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:54 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:57:55.228 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:54 smithi043 ceph-mon[30800]: from='osd.3 [v2:172.21.15.43:6810/1734673762,v1:172.21.15.43:6811/1734673762]' entity='osd.3' 2024-09-19T04:57:55.874 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:55 smithi096 ceph-mon[35085]: purged_snaps scrub starts 2024-09-19T04:57:55.874 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:55 smithi096 ceph-mon[35085]: purged_snaps scrub ok 2024-09-19T04:57:55.874 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:55 smithi096 ceph-mon[35085]: osd.3 [v2:172.21.15.43:6810/1734673762,v1:172.21.15.43:6811/1734673762] boot 2024-09-19T04:57:55.874 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:55 smithi096 ceph-mon[35085]: osdmap e24: 4 total, 4 up, 4 in 2024-09-19T04:57:55.875 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:55 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:57:55.875 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:55 smithi096 ceph-mon[35085]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:57:55.875 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:55 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:57:55.875 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:55 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:57:55.875 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:55 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:57:55.875 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:55 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:55.875 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:55 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:55.875 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:55 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:55.875 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:55 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:55.875 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:55 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:55.875 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:55 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:57:55.875 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:55 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:57:55.876 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:55 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:55.876 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:55 smithi096 ceph-mon[35085]: osdmap e25: 4 total, 4 up, 4 in 2024-09-19T04:57:56.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:55 smithi151 ceph-mon[35425]: purged_snaps scrub starts 2024-09-19T04:57:56.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:55 smithi151 ceph-mon[35425]: purged_snaps scrub ok 2024-09-19T04:57:56.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:55 smithi151 ceph-mon[35425]: osd.3 [v2:172.21.15.43:6810/1734673762,v1:172.21.15.43:6811/1734673762] boot 2024-09-19T04:57:56.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:55 smithi151 ceph-mon[35425]: osdmap e24: 4 total, 4 up, 4 in 2024-09-19T04:57:56.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:55 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:57:56.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:55 smithi151 ceph-mon[35425]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:57:56.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:55 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:57:56.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:55 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:57:56.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:55 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:57:56.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:55 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:56.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:55 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:56.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:55 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:56.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:55 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:56.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:55 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:56.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:55 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:57:56.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:55 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:57:56.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:55 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:56.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:55 smithi151 ceph-mon[35425]: osdmap e25: 4 total, 4 up, 4 in 2024-09-19T04:57:56.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:55 smithi043 ceph-mon[30800]: purged_snaps scrub starts 2024-09-19T04:57:56.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:55 smithi043 ceph-mon[30800]: purged_snaps scrub ok 2024-09-19T04:57:56.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:55 smithi043 ceph-mon[30800]: osd.3 [v2:172.21.15.43:6810/1734673762,v1:172.21.15.43:6811/1734673762] boot 2024-09-19T04:57:56.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:55 smithi043 ceph-mon[30800]: osdmap e24: 4 total, 4 up, 4 in 2024-09-19T04:57:56.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:55 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T04:57:56.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:55 smithi043 ceph-mon[30800]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:57:56.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:55 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:57:56.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:55 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:57:56.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:55 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:57:56.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:55 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:56.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:55 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:56.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:55 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:56.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:55 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:57:56.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:55 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:56.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:55 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:57:56.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:55 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:57:56.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:55 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:57:56.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:55 smithi043 ceph-mon[30800]: osdmap e25: 4 total, 4 up, 4 in 2024-09-19T04:57:56.872 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:56 smithi096 ceph-mon[35085]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:57:56.872 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:56 smithi096 ceph-mon[35085]: Detected new or changed devices on smithi043 2024-09-19T04:57:56.872 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:56 smithi096 ceph-mon[35085]: Adjusting osd_memory_target on smithi043 to 8547M 2024-09-19T04:57:56.872 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:56 smithi096 ceph-mon[35085]: osdmap e26: 4 total, 4 up, 4 in 2024-09-19T04:57:57.024 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:56 smithi043 ceph-mon[30800]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:57:57.024 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:56 smithi043 ceph-mon[30800]: Detected new or changed devices on smithi043 2024-09-19T04:57:57.024 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:56 smithi043 ceph-mon[30800]: Adjusting osd_memory_target on smithi043 to 8547M 2024-09-19T04:57:57.024 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:56 smithi043 ceph-mon[30800]: osdmap e26: 4 total, 4 up, 4 in 2024-09-19T04:57:57.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:56 smithi151 ceph-mon[35425]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:57:57.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:56 smithi151 ceph-mon[35425]: Detected new or changed devices on smithi043 2024-09-19T04:57:57.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:56 smithi151 ceph-mon[35425]: Adjusting osd_memory_target on smithi043 to 8547M 2024-09-19T04:57:57.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:56 smithi151 ceph-mon[35425]: osdmap e26: 4 total, 4 up, 4 in 2024-09-19T04:57:58.148 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:57 smithi096 ceph-mon[35085]: pgmap v101: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T04:57:58.148 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:57 smithi096 ceph-mon[35085]: osdmap e27: 4 total, 4 up, 4 in 2024-09-19T04:57:58.148 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:57 smithi096 ceph-mon[35085]: from='client.? 172.21.15.96:0/2604086307' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "196d5af1-a6e5-496c-8933-4e58892f4360"}]: dispatch 2024-09-19T04:57:58.148 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:57 smithi096 ceph-mon[35085]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "196d5af1-a6e5-496c-8933-4e58892f4360"}]: dispatch 2024-09-19T04:57:58.149 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:57 smithi096 ceph-mon[35085]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "196d5af1-a6e5-496c-8933-4e58892f4360"}]': finished 2024-09-19T04:57:58.149 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:57 smithi096 ceph-mon[35085]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T04:57:58.149 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:57 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:57:58.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:57 smithi151 ceph-mon[35425]: pgmap v101: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T04:57:58.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:57 smithi151 ceph-mon[35425]: osdmap e27: 4 total, 4 up, 4 in 2024-09-19T04:57:58.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:57 smithi151 ceph-mon[35425]: from='client.? 172.21.15.96:0/2604086307' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "196d5af1-a6e5-496c-8933-4e58892f4360"}]: dispatch 2024-09-19T04:57:58.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:57 smithi151 ceph-mon[35425]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "196d5af1-a6e5-496c-8933-4e58892f4360"}]: dispatch 2024-09-19T04:57:58.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:57 smithi151 ceph-mon[35425]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "196d5af1-a6e5-496c-8933-4e58892f4360"}]': finished 2024-09-19T04:57:58.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:57 smithi151 ceph-mon[35425]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T04:57:58.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:57 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:57:58.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:57 smithi043 ceph-mon[30800]: pgmap v101: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T04:57:58.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:57 smithi043 ceph-mon[30800]: osdmap e27: 4 total, 4 up, 4 in 2024-09-19T04:57:58.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:57 smithi043 ceph-mon[30800]: from='client.? 172.21.15.96:0/2604086307' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "196d5af1-a6e5-496c-8933-4e58892f4360"}]: dispatch 2024-09-19T04:57:58.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:57 smithi043 ceph-mon[30800]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "196d5af1-a6e5-496c-8933-4e58892f4360"}]: dispatch 2024-09-19T04:57:58.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:57 smithi043 ceph-mon[30800]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "196d5af1-a6e5-496c-8933-4e58892f4360"}]': finished 2024-09-19T04:57:58.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:57 smithi043 ceph-mon[30800]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T04:57:58.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:57 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:57:59.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:58 smithi151 ceph-mon[35425]: from='client.? 172.21.15.96:0/2589774489' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:57:59.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:58 smithi096 ceph-mon[35085]: from='client.? 172.21.15.96:0/2589774489' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:57:59.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:58 smithi043 ceph-mon[30800]: from='client.? 172.21.15.96:0/2589774489' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:58:00.146 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:57:59 smithi151 ceph-mon[35425]: pgmap v104: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T04:58:00.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:57:59 smithi096 ceph-mon[35085]: pgmap v104: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T04:58:00.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:57:59 smithi043 ceph-mon[30800]: pgmap v104: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T04:58:01.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:01 smithi151 ceph-mon[35425]: pgmap v105: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-19T04:58:01.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:01 smithi096 ceph-mon[35085]: pgmap v105: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-19T04:58:01.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:01 smithi043 ceph-mon[30800]: pgmap v105: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-19T04:58:03.335 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:03 smithi096 ceph-mon[35085]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-19T04:58:03.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:03 smithi151 ceph-mon[35425]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-19T04:58:03.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:03 smithi043 ceph-mon[30800]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-19T04:58:05.335 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:05 smithi096 ceph-mon[35085]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:58:05.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:05 smithi151 ceph-mon[35425]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:58:05.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:05 smithi043 ceph-mon[30800]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:58:06.222 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:06 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T04:58:06.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:06 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:06.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:06 smithi096 ceph-mon[35085]: Deploying daemon osd.4 on smithi096 2024-09-19T04:58:06.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:06 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T04:58:06.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:06 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:06.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:06 smithi151 ceph-mon[35425]: Deploying daemon osd.4 on smithi096 2024-09-19T04:58:06.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:06 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T04:58:06.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:06 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:06.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:06 smithi043 ceph-mon[30800]: Deploying daemon osd.4 on smithi096 2024-09-19T04:58:07.288 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:07 smithi096 ceph-mon[35085]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:58:07.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:07 smithi151 ceph-mon[35425]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:58:07.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:07 smithi043 ceph-mon[30800]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:58:09.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:09 smithi151 ceph-mon[35425]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:58:09.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:09 smithi096 ceph-mon[35085]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:58:09.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:09 smithi043 ceph-mon[30800]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:58:11.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:11 smithi096 ceph-mon[35085]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:58:11.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:11 smithi151 ceph-mon[35425]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:58:11.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:11 smithi043 ceph-mon[30800]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:58:12.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:12 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:58:12.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:12 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:12.472 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:12 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:58:12.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:12 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:12.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:12 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:58:12.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:12 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:13.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:13 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:13.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:13 smithi151 ceph-mon[35425]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:58:13.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:13 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:13.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:13 smithi096 ceph-mon[35085]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:58:13.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:13 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:13.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:13 smithi043 ceph-mon[30800]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:58:15.905 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 4 on host 'smithi096' 2024-09-19T04:58:15.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:58:15.903+0000 7f747f7fe640 1 -- 172.21.15.96:0/3932384667 <== mgr.14150 v2:172.21.15.43:6800/3167108925 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7488062630 con 0x7f74740a32f0 2024-09-19T04:58:15.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:58:15.905+0000 7f74907ce640 1 -- 172.21.15.96:0/3932384667 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f74740a32f0 msgr2=0x7f74740a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:58:15.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:58:15.905+0000 7f74907ce640 1 --2- 172.21.15.96:0/3932384667 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f74740a32f0 0x7f74740a57b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7484006fd0 tx=0x7f7484008040 comp rx=0 tx=0).stop 2024-09-19T04:58:15.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:58:15.905+0000 7f74907ce640 1 -- 172.21.15.96:0/3932384667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7488102e00 msgr2=0x7f7488198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:58:15.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:58:15.905+0000 7f74907ce640 1 --2- 172.21.15.96:0/3932384667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7488102e00 0x7f7488198480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f747800d770 tx=0x7f747800dc30 comp rx=0 tx=0).stop 2024-09-19T04:58:15.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:58:15.905+0000 7f74907ce640 1 -- 172.21.15.96:0/3932384667 shutdown_connections 2024-09-19T04:58:15.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:58:15.905+0000 7f74907ce640 1 --2- 172.21.15.96:0/3932384667 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f74740a32f0 0x7f74740a57b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:15.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:58:15.905+0000 7f74907ce640 1 --2- 172.21.15.96:0/3932384667 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74881037c0 0x7f748819f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:15.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:58:15.905+0000 7f74907ce640 1 --2- 172.21.15.96:0/3932384667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7488102e00 0x7f7488198480 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:15.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:58:15.905+0000 7f74907ce640 1 --2- 172.21.15.96:0/3932384667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7488101c00 0x7f7488197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:15.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:58:15.905+0000 7f74907ce640 1 -- 172.21.15.96:0/3932384667 >> 172.21.15.96:0/3932384667 conn(0x7f74880fd390 msgr2=0x7f7488106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:58:15.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:58:15.905+0000 7f74907ce640 1 -- 172.21.15.96:0/3932384667 shutdown_connections 2024-09-19T04:58:15.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:58:15.905+0000 7f74907ce640 1 -- 172.21.15.96:0/3932384667 wait complete. 2024-09-19T04:58:16.209 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:15 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:16.209 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:15 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:16.209 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:15 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:16.209 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:15 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:58:16.209 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:15 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:16.210 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:15 smithi096 ceph-mon[35085]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:58:16.210 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:15 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:58:16.210 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:15 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:16.210 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:15 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:16.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:15 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:16.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:15 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:16.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:15 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:16.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:15 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:58:16.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:15 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:16.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:15 smithi043 ceph-mon[30800]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:58:16.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:15 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:58:16.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:15 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:16.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:15 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:16.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:15 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:16.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:15 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:16.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:15 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:16.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:15 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:58:16.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:15 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:16.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:15 smithi151 ceph-mon[35425]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:58:16.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:15 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:58:16.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:15 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:16.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:15 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:16.545 DEBUG:teuthology.orchestra.run.smithi096:osd.4> sudo journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@osd.4.service 2024-09-19T04:58:16.548 INFO:tasks.cephadm:Deploying osd.5 on smithi151 with /dev/vg_nvme/lv_3... 2024-09-19T04:58:16.548 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T04:58:16.801 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:58:17.244 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:17 smithi151 ceph-mon[35425]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:58:17.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:17 smithi096 ceph-mon[35085]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:58:17.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:17 smithi043 ceph-mon[30800]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:58:18.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:18 smithi096 ceph-mon[35085]: from='osd.4 [v2:172.21.15.96:6808/3184785362,v1:172.21.15.96:6809/3184785362]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T04:58:18.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:18 smithi096 ceph-mon[35085]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T04:58:18.473 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 19 04:58:18 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-4[46023]: 2024-09-19T04:58:18.189+0000 7f714ce47640 -1 osd.4 0 log_to_monitors true 2024-09-19T04:58:18.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:18 smithi043 ceph-mon[30800]: from='osd.4 [v2:172.21.15.96:6808/3184785362,v1:172.21.15.96:6809/3184785362]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T04:58:18.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:18 smithi043 ceph-mon[30800]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T04:58:18.629 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:18 smithi151 ceph-mon[35425]: from='osd.4 [v2:172.21.15.96:6808/3184785362,v1:172.21.15.96:6809/3184785362]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T04:58:18.629 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:18 smithi151 ceph-mon[35425]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T04:58:19.323 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:19 smithi151 ceph-mon[35425]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T04:58:19.323 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:19 smithi151 ceph-mon[35425]: from='osd.4 [v2:172.21.15.96:6808/3184785362,v1:172.21.15.96:6809/3184785362]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T04:58:19.323 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:19 smithi151 ceph-mon[35425]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T04:58:19.323 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:19 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:58:19.323 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:19 smithi151 ceph-mon[35425]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T04:58:19.323 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:19 smithi151 ceph-mon[35425]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:58:19.324 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:19 smithi151 ceph-mon[35425]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-19T04:58:19.324 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:19 smithi151 ceph-mon[35425]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T04:58:19.326 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T04:58:19.352 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi151:vg_nvme/lv_3 2024-09-19T04:58:19.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:19 smithi043 ceph-mon[30800]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T04:58:19.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:19 smithi043 ceph-mon[30800]: from='osd.4 [v2:172.21.15.96:6808/3184785362,v1:172.21.15.96:6809/3184785362]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T04:58:19.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:19 smithi043 ceph-mon[30800]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T04:58:19.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:19 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:58:19.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:19 smithi043 ceph-mon[30800]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T04:58:19.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:19 smithi043 ceph-mon[30800]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:58:19.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:19 smithi043 ceph-mon[30800]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-19T04:58:19.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:19 smithi043 ceph-mon[30800]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T04:58:19.608 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:58:19.723 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 19 04:58:19 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-4[46023]: 2024-09-19T04:58:19.253+0000 7f7148dca640 -1 osd.4 0 waiting for initial osdmap 2024-09-19T04:58:19.723 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 19 04:58:19 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-4[46023]: 2024-09-19T04:58:19.260+0000 7f7144bf2640 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:58:19.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:19 smithi096 ceph-mon[35085]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T04:58:19.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:19 smithi096 ceph-mon[35085]: from='osd.4 [v2:172.21.15.96:6808/3184785362,v1:172.21.15.96:6809/3184785362]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T04:58:19.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:19 smithi096 ceph-mon[35085]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T04:58:19.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:19 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:58:19.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:19 smithi096 ceph-mon[35085]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T04:58:19.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:19 smithi096 ceph-mon[35085]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:58:19.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:19 smithi096 ceph-mon[35085]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-19T04:58:19.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:19 smithi096 ceph-mon[35085]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T04:58:20.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:20.061+0000 7ff6be8da640 1 -- 172.21.15.151:0/2184191896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6b8101bf0 msgr2=0x7ff6b8101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:58:20.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:20.061+0000 7ff6be8da640 1 --2- 172.21.15.151:0/2184191896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6b8101bf0 0x7ff6b8101ff0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8009900 tx=0x7ff6a802a410 comp rx=0 tx=0).stop 2024-09-19T04:58:20.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:20.061+0000 7ff6be8da640 1 -- 172.21.15.151:0/2184191896 shutdown_connections 2024-09-19T04:58:20.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:20.061+0000 7ff6be8da640 1 --2- 172.21.15.151:0/2184191896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6b81037b0 0x7ff6b810a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:20.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:20.061+0000 7ff6be8da640 1 --2- 172.21.15.151:0/2184191896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6b8102df0 0x7ff6b8103270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:20.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:20.061+0000 7ff6be8da640 1 --2- 172.21.15.151:0/2184191896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6b8101bf0 0x7ff6b8101ff0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:20.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:20.061+0000 7ff6be8da640 1 -- 172.21.15.151:0/2184191896 >> 172.21.15.151:0/2184191896 conn(0x7ff6b80fd3c0 msgr2=0x7ff6b80ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:58:20.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:20.061+0000 7ff6be8da640 1 -- 172.21.15.151:0/2184191896 shutdown_connections 2024-09-19T04:58:20.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:20.061+0000 7ff6be8da640 1 -- 172.21.15.151:0/2184191896 wait complete. 2024-09-19T04:58:20.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:20.061+0000 7ff6be8da640 1 Processor -- start 2024-09-19T04:58:20.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:20.062+0000 7ff6be8da640 1 -- start start 2024-09-19T04:58:20.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:20.062+0000 7ff6be8da640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6b8101bf0 0x7ff6b8197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:58:20.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:20.062+0000 7ff6be8da640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6b8102df0 0x7ff6b8198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:58:20.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:20.062+0000 7ff6be8da640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6b81037b0 0x7ff6b819f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:58:20.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:20.062+0000 7ff6be8da640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b810d940 con 0x7ff6b8101bf0 2024-09-19T04:58:20.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:20.062+0000 7ff6be8da640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b810d7c0 con 0x7ff6b81037b0 2024-09-19T04:58:20.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:20.062+0000 7ff6be8da640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b810dac0 con 0x7ff6b8102df0 2024-09-19T04:58:20.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:20.062+0000 7ff6b77fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6b8102df0 0x7ff6b8198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:58:20.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:20.062+0000 7ff6b77fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6b8102df0 0x7ff6b8198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:32852/0 (socket says 172.21.15.151:32852) 2024-09-19T04:58:20.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:20.062+0000 7ff6b77fe640 1 -- 172.21.15.151:0/2581200 learned_addr learned my addr 172.21.15.151:0/2581200 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-19T04:58:20.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:20.062+0000 7ff6b7fff640 1 --2- 172.21.15.151:0/2581200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6b8101bf0 0x7ff6b8197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:58:20.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:20.062+0000 7ff6b77fe640 1 -- 172.21.15.151:0/2581200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6b81037b0 msgr2=0x7ff6b819f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:58:20.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:20.062+0000 7ff6b77fe640 1 --2- 172.21.15.151:0/2581200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6b81037b0 0x7ff6b819f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:20.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:20.062+0000 7ff6b77fe640 1 -- 172.21.15.151:0/2581200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6b8101bf0 msgr2=0x7ff6b8197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:58:20.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:20.062+0000 7ff6b77fe640 1 --2- 172.21.15.151:0/2581200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6b8101bf0 0x7ff6b8197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:20.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:20.062+0000 7ff6b77fe640 1 -- 172.21.15.151:0/2581200 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6b819fb70 con 0x7ff6b8102df0 2024-09-19T04:58:20.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:20.062+0000 7ff6b7fff640 1 --2- 172.21.15.151:0/2581200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6b8101bf0 0x7ff6b8197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:58:20.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:20.063+0000 7ff6b77fe640 1 --2- 172.21.15.151:0/2581200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6b8102df0 0x7ff6b8198480 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00d770 tx=0x7ff6ac00dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:58:20.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:20.063+0000 7ff6b57fa640 1 -- 172.21.15.151:0/2581200 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac017070 con 0x7ff6b8102df0 2024-09-19T04:58:20.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:20.063+0000 7ff6b57fa640 1 -- 172.21.15.151:0/2581200 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6ac005500 con 0x7ff6b8102df0 2024-09-19T04:58:20.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:20.063+0000 7ff6be8da640 1 -- 172.21.15.151:0/2581200 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6b819fe60 con 0x7ff6b8102df0 2024-09-19T04:58:20.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:20.063+0000 7ff6b57fa640 1 -- 172.21.15.151:0/2581200 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac004e00 con 0x7ff6b8102df0 2024-09-19T04:58:20.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:20.063+0000 7ff6be8da640 1 -- 172.21.15.151:0/2581200 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6b81a0390 con 0x7ff6b8102df0 2024-09-19T04:58:20.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:20.064+0000 7ff6be8da640 1 -- 172.21.15.151:0/2581200 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6b8108fa0 con 0x7ff6b8102df0 2024-09-19T04:58:20.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:20.065+0000 7ff6b57fa640 1 -- 172.21.15.151:0/2581200 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7ff6ac004fa0 con 0x7ff6b8102df0 2024-09-19T04:58:20.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:20.066+0000 7ff6b57fa640 1 --2- 172.21.15.151:0/2581200 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff6a00a32f0 0x7ff6a00a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:58:20.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:20.066+0000 7ff6b57fa640 1 -- 172.21.15.151:0/2581200 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 3821+0+0 (secure 0 0 0) 0x7ff6ac0ce1e0 con 0x7ff6b8102df0 2024-09-19T04:58:20.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:20.066+0000 7ff6b7fff640 1 --2- 172.21.15.151:0/2581200 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff6a00a32f0 0x7ff6a00a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:58:20.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:20.067+0000 7ff6b7fff640 1 --2- 172.21.15.151:0/2581200 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff6a00a32f0 0x7ff6a00a57b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff6a802a930 tx=0x7ff6a8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:58:20.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:20.070+0000 7ff6b57fa640 1 -- 172.21.15.151:0/2581200 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff6ac09c090 con 0x7ff6b8102df0 2024-09-19T04:58:20.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:20.269+0000 7ff6be8da640 1 -- 172.21.15.151:0/2581200 --> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7ff6b8062630 con 0x7ff6a00a32f0 2024-09-19T04:58:20.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:20 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:58:20.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:20 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:58:20.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:20 smithi043 ceph-mon[30800]: Detected new or changed devices on smithi096 2024-09-19T04:58:20.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:20 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:20.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:20 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:20.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:20 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:58:20.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:20 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:58:20.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:20 smithi043 ceph-mon[30800]: Adjusting osd_memory_target on smithi096 to 8544M 2024-09-19T04:58:20.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:20 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:20.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:20 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:20.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:20 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:58:20.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:20 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:20.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:20 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:58:20.638 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:20 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:58:20.638 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:20 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:58:20.638 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:20 smithi151 ceph-mon[35425]: Detected new or changed devices on smithi096 2024-09-19T04:58:20.638 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:20 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:20.639 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:20 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:20.639 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:20 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:58:20.639 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:20 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:58:20.639 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:20 smithi151 ceph-mon[35425]: Adjusting osd_memory_target on smithi096 to 8544M 2024-09-19T04:58:20.639 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:20 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:20.639 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:20 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:20.639 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:20 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:58:20.639 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:20 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:20.639 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:20 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:58:20.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:20 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:58:20.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:20 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:58:20.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:20 smithi096 ceph-mon[35085]: Detected new or changed devices on smithi096 2024-09-19T04:58:20.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:20 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:20.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:20 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:20.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:20 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:58:20.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:20 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:58:20.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:20 smithi096 ceph-mon[35085]: Adjusting osd_memory_target on smithi096 to 8544M 2024-09-19T04:58:20.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:20 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:20.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:20 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:20.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:20 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:58:20.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:20 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:20.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:20 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:58:21.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:21 smithi043 ceph-mon[30800]: purged_snaps scrub starts 2024-09-19T04:58:21.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:21 smithi043 ceph-mon[30800]: purged_snaps scrub ok 2024-09-19T04:58:21.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:21 smithi043 ceph-mon[30800]: osd.4 [v2:172.21.15.96:6808/3184785362,v1:172.21.15.96:6809/3184785362] boot 2024-09-19T04:58:21.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:21 smithi043 ceph-mon[30800]: osdmap e31: 5 total, 5 up, 5 in 2024-09-19T04:58:21.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:21 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:58:21.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:21 smithi043 ceph-mon[30800]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:58:21.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:21 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:58:21.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:21 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:58:21.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:21 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:21.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:21 smithi043 ceph-mon[30800]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:58:21.642 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:21 smithi151 ceph-mon[35425]: purged_snaps scrub starts 2024-09-19T04:58:21.642 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:21 smithi151 ceph-mon[35425]: purged_snaps scrub ok 2024-09-19T04:58:21.642 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:21 smithi151 ceph-mon[35425]: osd.4 [v2:172.21.15.96:6808/3184785362,v1:172.21.15.96:6809/3184785362] boot 2024-09-19T04:58:21.642 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:21 smithi151 ceph-mon[35425]: osdmap e31: 5 total, 5 up, 5 in 2024-09-19T04:58:21.642 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:21 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:58:21.642 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:21 smithi151 ceph-mon[35425]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:58:21.642 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:21 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:58:21.642 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:21 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:58:21.642 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:21 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:21.642 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:21 smithi151 ceph-mon[35425]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:58:21.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:21 smithi096 ceph-mon[35085]: purged_snaps scrub starts 2024-09-19T04:58:21.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:21 smithi096 ceph-mon[35085]: purged_snaps scrub ok 2024-09-19T04:58:21.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:21 smithi096 ceph-mon[35085]: osd.4 [v2:172.21.15.96:6808/3184785362,v1:172.21.15.96:6809/3184785362] boot 2024-09-19T04:58:21.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:21 smithi096 ceph-mon[35085]: osdmap e31: 5 total, 5 up, 5 in 2024-09-19T04:58:21.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:21 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T04:58:21.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:21 smithi096 ceph-mon[35085]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:58:21.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:21 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:58:21.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:21 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:58:21.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:21 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:21.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:21 smithi096 ceph-mon[35085]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-19T04:58:22.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:22 smithi043 ceph-mon[30800]: osdmap e32: 5 total, 5 up, 5 in 2024-09-19T04:58:22.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:22 smithi151 ceph-mon[35425]: osdmap e32: 5 total, 5 up, 5 in 2024-09-19T04:58:22.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:22 smithi096 ceph-mon[35085]: osdmap e32: 5 total, 5 up, 5 in 2024-09-19T04:58:23.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:23 smithi151 ceph-mon[35425]: from='client.? 172.21.15.151:0/410479252' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5492d474-8509-4865-bc2e-d08e31558a75"}]: dispatch 2024-09-19T04:58:23.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:23 smithi151 ceph-mon[35425]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5492d474-8509-4865-bc2e-d08e31558a75"}]: dispatch 2024-09-19T04:58:23.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:23 smithi151 ceph-mon[35425]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5492d474-8509-4865-bc2e-d08e31558a75"}]': finished 2024-09-19T04:58:23.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:23 smithi151 ceph-mon[35425]: osdmap e33: 6 total, 5 up, 6 in 2024-09-19T04:58:23.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:23 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:58:23.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:23 smithi151 ceph-mon[35425]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:58:23.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:23 smithi151 ceph-mon[35425]: from='client.? 172.21.15.151:0/858934426' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:58:23.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:23 smithi043 ceph-mon[30800]: from='client.? 172.21.15.151:0/410479252' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5492d474-8509-4865-bc2e-d08e31558a75"}]: dispatch 2024-09-19T04:58:23.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:23 smithi043 ceph-mon[30800]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5492d474-8509-4865-bc2e-d08e31558a75"}]: dispatch 2024-09-19T04:58:23.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:23 smithi043 ceph-mon[30800]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5492d474-8509-4865-bc2e-d08e31558a75"}]': finished 2024-09-19T04:58:23.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:23 smithi043 ceph-mon[30800]: osdmap e33: 6 total, 5 up, 6 in 2024-09-19T04:58:23.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:23 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:58:23.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:23 smithi043 ceph-mon[30800]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:58:23.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:23 smithi043 ceph-mon[30800]: from='client.? 172.21.15.151:0/858934426' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:58:23.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:23 smithi096 ceph-mon[35085]: from='client.? 172.21.15.151:0/410479252' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5492d474-8509-4865-bc2e-d08e31558a75"}]: dispatch 2024-09-19T04:58:23.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:23 smithi096 ceph-mon[35085]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5492d474-8509-4865-bc2e-d08e31558a75"}]: dispatch 2024-09-19T04:58:23.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:23 smithi096 ceph-mon[35085]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5492d474-8509-4865-bc2e-d08e31558a75"}]': finished 2024-09-19T04:58:23.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:23 smithi096 ceph-mon[35085]: osdmap e33: 6 total, 5 up, 6 in 2024-09-19T04:58:23.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:23 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:58:23.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:23 smithi096 ceph-mon[35085]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:58:23.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:23 smithi096 ceph-mon[35085]: from='client.? 172.21.15.151:0/858934426' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:58:25.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:25 smithi151 ceph-mon[35425]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:58:25.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:25 smithi096 ceph-mon[35085]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:58:25.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:25 smithi043 ceph-mon[30800]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:58:27.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:27 smithi151 ceph-mon[35425]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:58:27.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:27 smithi096 ceph-mon[35085]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:58:27.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:27 smithi043 ceph-mon[30800]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:58:29.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:29 smithi151 ceph-mon[35425]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:58:29.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:29 smithi043 ceph-mon[30800]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:58:29.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:29 smithi096 ceph-mon[35085]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:58:31.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:31 smithi151 ceph-mon[35425]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:58:31.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:31 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T04:58:31.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:31 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:31.454 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:31 smithi096 ceph-mon[35085]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:58:31.454 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:31 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T04:58:31.454 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:31 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:31.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:31 smithi043 ceph-mon[30800]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:58:31.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:31 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T04:58:31.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:31 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:32.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:32 smithi151 ceph-mon[35425]: Deploying daemon osd.5 on smithi151 2024-09-19T04:58:32.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:32 smithi096 ceph-mon[35085]: Deploying daemon osd.5 on smithi151 2024-09-19T04:58:32.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:32 smithi043 ceph-mon[30800]: Deploying daemon osd.5 on smithi151 2024-09-19T04:58:33.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:33 smithi151 ceph-mon[35425]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:58:33.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:33 smithi096 ceph-mon[35085]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:58:33.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:33 smithi043 ceph-mon[30800]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:58:35.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:35 smithi151 ceph-mon[35425]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:58:35.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:35 smithi096 ceph-mon[35085]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:58:35.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:35 smithi043 ceph-mon[30800]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:58:37.350 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:37 smithi151 ceph-mon[35425]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:58:37.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:37 smithi096 ceph-mon[35085]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:58:37.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:37 smithi043 ceph-mon[30800]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:58:38.340 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:38 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:58:38.340 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:38 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:38.340 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:38 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:38.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:38 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:58:38.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:38 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:38.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:38 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:38.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:38 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:58:38.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:38 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:38.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:38 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:39.294 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:39 smithi151 ceph-mon[35425]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:58:39.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:39 smithi096 ceph-mon[35085]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:58:39.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:39 smithi043 ceph-mon[30800]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:58:41.635 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:41 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:41.636 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:41 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:41.636 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:41 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:41.636 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:41 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:58:41.636 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:41 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:41.636 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:41 smithi151 ceph-mon[35425]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:58:41.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:41.678+0000 7ff6b57fa640 1 -- 172.21.15.151:0/2581200 <== mgr.14150 v2:172.21.15.43:6800/3167108925 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff6b8062630 con 0x7ff6a00a32f0 2024-09-19T04:58:41.680 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 5 on host 'smithi151' 2024-09-19T04:58:41.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:41.681+0000 7ff6be8da640 1 -- 172.21.15.151:0/2581200 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff6a00a32f0 msgr2=0x7ff6a00a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:58:41.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:41.681+0000 7ff6be8da640 1 --2- 172.21.15.151:0/2581200 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff6a00a32f0 0x7ff6a00a57b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff6a802a930 tx=0x7ff6a8000bc0 comp rx=0 tx=0).stop 2024-09-19T04:58:41.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:41.681+0000 7ff6be8da640 1 -- 172.21.15.151:0/2581200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6b8102df0 msgr2=0x7ff6b8198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:58:41.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:41.681+0000 7ff6be8da640 1 --2- 172.21.15.151:0/2581200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6b8102df0 0x7ff6b8198480 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00d770 tx=0x7ff6ac00dc30 comp rx=0 tx=0).stop 2024-09-19T04:58:41.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:41.681+0000 7ff6be8da640 1 -- 172.21.15.151:0/2581200 shutdown_connections 2024-09-19T04:58:41.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:41.681+0000 7ff6be8da640 1 --2- 172.21.15.151:0/2581200 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff6a00a32f0 0x7ff6a00a57b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:41.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:41.681+0000 7ff6be8da640 1 --2- 172.21.15.151:0/2581200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6b81037b0 0x7ff6b819f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:41.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:41.681+0000 7ff6be8da640 1 --2- 172.21.15.151:0/2581200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6b8102df0 0x7ff6b8198480 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:41.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:41.681+0000 7ff6be8da640 1 --2- 172.21.15.151:0/2581200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6b8101bf0 0x7ff6b8197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:41.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:41.681+0000 7ff6be8da640 1 -- 172.21.15.151:0/2581200 >> 172.21.15.151:0/2581200 conn(0x7ff6b80fd3c0 msgr2=0x7ff6b8106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:58:41.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:41.681+0000 7ff6be8da640 1 -- 172.21.15.151:0/2581200 shutdown_connections 2024-09-19T04:58:41.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:58:41.682+0000 7ff6be8da640 1 -- 172.21.15.151:0/2581200 wait complete. 2024-09-19T04:58:41.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:41 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:41.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:41 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:41.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:41 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:41.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:41 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:58:41.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:41 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:41.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:41 smithi096 ceph-mon[35085]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:58:41.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:41 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:41.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:41 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:41.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:41 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:41.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:41 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:58:41.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:41 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:41.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:41 smithi043 ceph-mon[30800]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:58:42.336 DEBUG:teuthology.orchestra.run.smithi151:osd.5> sudo journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@osd.5.service 2024-09-19T04:58:42.339 INFO:tasks.cephadm:Deploying osd.6 on smithi043 with /dev/vg_nvme/lv_2... 2024-09-19T04:58:42.339 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T04:58:42.557 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:42 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:58:42.557 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:42 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:42.558 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:42 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:42.719 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:58:42.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:42 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:58:42.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:42 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:42.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:42 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:42.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:42 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:58:42.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:42 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:42.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:42 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:43.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:43 smithi043 ceph-mon[30800]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:58:43.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:43 smithi151 ceph-mon[35425]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:58:43.655 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 04:58:43 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5[46846]: 2024-09-19T04:58:43.513+0000 7fa5d764b640 -1 osd.5 0 log_to_monitors true 2024-09-19T04:58:43.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:43 smithi096 ceph-mon[35085]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:58:44.510 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:44 smithi043 ceph-mon[30800]: from='osd.5 [v2:172.21.15.151:6808/2878451924,v1:172.21.15.151:6809/2878451924]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T04:58:44.510 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:44 smithi043 ceph-mon[30800]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T04:58:44.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:44 smithi151 ceph-mon[35425]: from='osd.5 [v2:172.21.15.151:6808/2878451924,v1:172.21.15.151:6809/2878451924]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T04:58:44.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:44 smithi151 ceph-mon[35425]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T04:58:44.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:44 smithi096 ceph-mon[35085]: from='osd.5 [v2:172.21.15.151:6808/2878451924,v1:172.21.15.151:6809/2878451924]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T04:58:44.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:44 smithi096 ceph-mon[35085]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T04:58:45.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:45 smithi151 ceph-mon[35425]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T04:58:45.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:45 smithi151 ceph-mon[35425]: osdmap e34: 6 total, 5 up, 6 in 2024-09-19T04:58:45.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:45 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:58:45.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:45 smithi151 ceph-mon[35425]: from='osd.5 [v2:172.21.15.151:6808/2878451924,v1:172.21.15.151:6809/2878451924]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T04:58:45.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:45 smithi151 ceph-mon[35425]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T04:58:45.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:45 smithi151 ceph-mon[35425]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:58:45.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:45 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:45.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:45 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:45.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:45 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:58:45.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:45 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:58:45.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:45 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:45.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:45 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:45.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:45 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:58:45.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:45 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:45.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:45 smithi096 ceph-mon[35085]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T04:58:45.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:45 smithi096 ceph-mon[35085]: osdmap e34: 6 total, 5 up, 6 in 2024-09-19T04:58:45.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:45 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:58:45.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:45 smithi096 ceph-mon[35085]: from='osd.5 [v2:172.21.15.151:6808/2878451924,v1:172.21.15.151:6809/2878451924]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T04:58:45.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:45 smithi096 ceph-mon[35085]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T04:58:45.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:45 smithi096 ceph-mon[35085]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:58:45.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:45 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:45.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:45 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:45.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:45 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:58:45.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:45 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:58:45.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:45 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:45.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:45 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:45.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:45 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:58:45.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:45 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:45.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:45 smithi043 ceph-mon[30800]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T04:58:45.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:45 smithi043 ceph-mon[30800]: osdmap e34: 6 total, 5 up, 6 in 2024-09-19T04:58:45.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:45 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:58:45.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:45 smithi043 ceph-mon[30800]: from='osd.5 [v2:172.21.15.151:6808/2878451924,v1:172.21.15.151:6809/2878451924]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T04:58:45.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:45 smithi043 ceph-mon[30800]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T04:58:45.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:45 smithi043 ceph-mon[30800]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:58:45.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:45 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:45.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:45 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:45.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:45 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:58:45.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:45 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:58:45.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:45 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:45.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:45 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:45.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:45 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:58:45.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:45 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:58:45.832 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T04:58:45.863 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi043:vg_nvme/lv_2 2024-09-19T04:58:46.155 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 04:58:45 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5[46846]: 2024-09-19T04:58:45.756+0000 7fa5d35ce640 -1 osd.5 0 waiting for initial osdmap 2024-09-19T04:58:46.155 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 04:58:45 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5[46846]: 2024-09-19T04:58:45.761+0000 7fa5cb3ee640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:58:46.231 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T04:58:46.636 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:46 smithi043 ceph-mon[30800]: Detected new or changed devices on smithi151 2024-09-19T04:58:46.636 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:46 smithi043 ceph-mon[30800]: Adjusting osd_memory_target on smithi151 to 8544M 2024-09-19T04:58:46.636 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:46 smithi043 ceph-mon[30800]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-19T04:58:46.636 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:46 smithi043 ceph-mon[30800]: osdmap e35: 6 total, 5 up, 6 in 2024-09-19T04:58:46.636 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:46 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:58:46.636 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:46 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:58:46.636 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:46 smithi043 ceph-mon[30800]: osdmap e36: 6 total, 5 up, 6 in 2024-09-19T04:58:46.636 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:46 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:58:46.636 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:46 smithi043 ceph-mon[30800]: from='osd.5 ' entity='osd.5' 2024-09-19T04:58:46.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:46 smithi151 ceph-mon[35425]: Detected new or changed devices on smithi151 2024-09-19T04:58:46.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:46 smithi151 ceph-mon[35425]: Adjusting osd_memory_target on smithi151 to 8544M 2024-09-19T04:58:46.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:46 smithi151 ceph-mon[35425]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-19T04:58:46.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:46 smithi151 ceph-mon[35425]: osdmap e35: 6 total, 5 up, 6 in 2024-09-19T04:58:46.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:46 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:58:46.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:46 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:58:46.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:46 smithi151 ceph-mon[35425]: osdmap e36: 6 total, 5 up, 6 in 2024-09-19T04:58:46.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:46 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:58:46.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:46 smithi151 ceph-mon[35425]: from='osd.5 ' entity='osd.5' 2024-09-19T04:58:46.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:46 smithi096 ceph-mon[35085]: Detected new or changed devices on smithi151 2024-09-19T04:58:46.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:46 smithi096 ceph-mon[35085]: Adjusting osd_memory_target on smithi151 to 8544M 2024-09-19T04:58:46.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:46 smithi096 ceph-mon[35085]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-19T04:58:46.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:46 smithi096 ceph-mon[35085]: osdmap e35: 6 total, 5 up, 6 in 2024-09-19T04:58:46.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:46 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:58:46.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:46 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:58:46.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:46 smithi096 ceph-mon[35085]: osdmap e36: 6 total, 5 up, 6 in 2024-09-19T04:58:46.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:46 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:58:46.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:46 smithi096 ceph-mon[35085]: from='osd.5 ' entity='osd.5' 2024-09-19T04:58:46.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:58:46.749+0000 7f7607600640 1 -- 172.21.15.43:0/3390786050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7600103e00 msgr2=0x7f7600104200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:58:46.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:58:46.749+0000 7f7607600640 1 --2- 172.21.15.43:0/3390786050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7600103e00 0x7f7600104200 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f75f4009860 tx=0x7f75f402a340 comp rx=0 tx=0).stop 2024-09-19T04:58:46.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:58:46.749+0000 7f7607600640 1 -- 172.21.15.43:0/3390786050 shutdown_connections 2024-09-19T04:58:46.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:58:46.749+0000 7f7607600640 1 --2- 172.21.15.43:0/3390786050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7600105920 0x7f760010c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:46.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:58:46.749+0000 7f7607600640 1 --2- 172.21.15.43:0/3390786050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7600104f60 0x7f76001053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:46.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:58:46.749+0000 7f7607600640 1 --2- 172.21.15.43:0/3390786050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7600103e00 0x7f7600104200 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:46.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:58:46.749+0000 7f7607600640 1 -- 172.21.15.43:0/3390786050 >> 172.21.15.43:0/3390786050 conn(0x7f76000ff5b0 msgr2=0x7f76001019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:58:46.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:58:46.749+0000 7f7607600640 1 -- 172.21.15.43:0/3390786050 shutdown_connections 2024-09-19T04:58:46.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:58:46.749+0000 7f7607600640 1 -- 172.21.15.43:0/3390786050 wait complete. 2024-09-19T04:58:46.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:58:46.750+0000 7f7607600640 1 Processor -- start 2024-09-19T04:58:46.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:58:46.751+0000 7f7607600640 1 -- start start 2024-09-19T04:58:46.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:58:46.751+0000 7f7607600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7600103e00 0x7f760019a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:58:46.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:58:46.751+0000 7f7607600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7600104f60 0x7f760019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:58:46.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:58:46.751+0000 7f7607600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7600105920 0x7f76001a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:58:46.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:58:46.751+0000 7f7607600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f760010fab0 con 0x7f7600105920 2024-09-19T04:58:46.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:58:46.751+0000 7f7607600640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f760010f930 con 0x7f7600104f60 2024-09-19T04:58:46.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:58:46.751+0000 7f7607600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f760010fc30 con 0x7f7600103e00 2024-09-19T04:58:46.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:58:46.751+0000 7f7605600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7600105920 0x7f76001a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:58:46.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:58:46.751+0000 7f75ffe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7600104f60 0x7f760019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:58:46.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:58:46.751+0000 7f7604c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7600103e00 0x7f760019a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:58:46.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:58:46.751+0000 7f7605600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7600105920 0x7f76001a1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54008/0 (socket says 172.21.15.43:54008) 2024-09-19T04:58:46.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:58:46.751+0000 7f75ffe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7600104f60 0x7f760019a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:40866/0 (socket says 172.21.15.43:40866) 2024-09-19T04:58:46.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:58:46.751+0000 7f7605600640 1 -- 172.21.15.43:0/3551469555 learned_addr learned my addr 172.21.15.43:0/3551469555 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T04:58:46.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:58:46.752+0000 7f7604c00640 1 -- 172.21.15.43:0/3551469555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7600104f60 msgr2=0x7f760019a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:58:46.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:58:46.752+0000 7f7604c00640 1 --2- 172.21.15.43:0/3551469555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7600104f60 0x7f760019a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:46.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:58:46.752+0000 7f7604c00640 1 -- 172.21.15.43:0/3551469555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7600105920 msgr2=0x7f76001a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:58:46.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:58:46.752+0000 7f7604c00640 1 --2- 172.21.15.43:0/3551469555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7600105920 0x7f76001a1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:58:46.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:58:46.752+0000 7f7604c00640 1 -- 172.21.15.43:0/3551469555 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76001a1cd0 con 0x7f7600103e00 2024-09-19T04:58:46.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:58:46.752+0000 7f7604c00640 1 --2- 172.21.15.43:0/3551469555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7600103e00 0x7f760019a0a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f75f4035040 tx=0x7f75f4033d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:58:46.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:58:46.752+0000 7f75fd600640 1 -- 172.21.15.43:0/3551469555 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f4037040 con 0x7f7600103e00 2024-09-19T04:58:46.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:58:46.752+0000 7f75fd600640 1 -- 172.21.15.43:0/3551469555 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75f4030850 con 0x7f7600103e00 2024-09-19T04:58:46.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:58:46.752+0000 7f7607600640 1 -- 172.21.15.43:0/3551469555 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76001a1f60 con 0x7f7600103e00 2024-09-19T04:58:46.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:58:46.752+0000 7f75fd600640 1 -- 172.21.15.43:0/3551469555 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f4005650 con 0x7f7600103e00 2024-09-19T04:58:46.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:58:46.752+0000 7f7607600640 1 -- 172.21.15.43:0/3551469555 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76001a23b0 con 0x7f7600103e00 2024-09-19T04:58:46.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:58:46.753+0000 7f7607600640 1 -- 172.21.15.43:0/3551469555 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f760010b1a0 con 0x7f7600103e00 2024-09-19T04:58:46.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:58:46.756+0000 7f75fd600640 1 -- 172.21.15.43:0/3551469555 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f75f4004050 con 0x7f7600103e00 2024-09-19T04:58:46.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:58:46.757+0000 7f75fd600640 1 --2- 172.21.15.43:0/3551469555 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f75e80a32f0 0x7f75e80a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:58:46.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:58:46.757+0000 7f75fd600640 1 -- 172.21.15.43:0/3551469555 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4566+0+0 (secure 0 0 0) 0x7f75f40ee890 con 0x7f7600103e00 2024-09-19T04:58:46.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:58:46.757+0000 7f75ffe00640 1 --2- 172.21.15.43:0/3551469555 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f75e80a32f0 0x7f75e80a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:58:46.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:58:46.758+0000 7f75ffe00640 1 --2- 172.21.15.43:0/3551469555 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f75e80a32f0 0x7f75e80a57b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f760019b5c0 tx=0x7f75ec00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:58:46.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:58:46.759+0000 7f75fd600640 1 -- 172.21.15.43:0/3551469555 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f75f40bc3b0 con 0x7f7600103e00 2024-09-19T04:58:46.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:58:46.959+0000 7f7607600640 1 -- 172.21.15.43:0/3551469555 --> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f7600062630 con 0x7f75e80a32f0 2024-09-19T04:58:47.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:47 smithi151 ceph-mon[35425]: purged_snaps scrub starts 2024-09-19T04:58:47.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:47 smithi151 ceph-mon[35425]: purged_snaps scrub ok 2024-09-19T04:58:47.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:47 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:58:47.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:47 smithi151 ceph-mon[35425]: osd.5 [v2:172.21.15.151:6808/2878451924,v1:172.21.15.151:6809/2878451924] boot 2024-09-19T04:58:47.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:47 smithi151 ceph-mon[35425]: osdmap e37: 6 total, 6 up, 6 in 2024-09-19T04:58:47.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:47 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:58:47.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:47 smithi151 ceph-mon[35425]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:58:47.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:47 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:58:47.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:47 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:58:47.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:47 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:47.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:47 smithi151 ceph-mon[35425]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:58:47.687 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:47 smithi043 ceph-mon[30800]: purged_snaps scrub starts 2024-09-19T04:58:47.687 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:47 smithi043 ceph-mon[30800]: purged_snaps scrub ok 2024-09-19T04:58:47.687 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:47 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:58:47.687 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:47 smithi043 ceph-mon[30800]: osd.5 [v2:172.21.15.151:6808/2878451924,v1:172.21.15.151:6809/2878451924] boot 2024-09-19T04:58:47.687 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:47 smithi043 ceph-mon[30800]: osdmap e37: 6 total, 6 up, 6 in 2024-09-19T04:58:47.687 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:47 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:58:47.688 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:47 smithi043 ceph-mon[30800]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:58:47.688 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:47 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:58:47.688 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:47 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:58:47.688 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:47 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:47.688 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:47 smithi043 ceph-mon[30800]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:58:47.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:47 smithi096 ceph-mon[35085]: purged_snaps scrub starts 2024-09-19T04:58:47.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:47 smithi096 ceph-mon[35085]: purged_snaps scrub ok 2024-09-19T04:58:47.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:47 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:58:47.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:47 smithi096 ceph-mon[35085]: osd.5 [v2:172.21.15.151:6808/2878451924,v1:172.21.15.151:6809/2878451924] boot 2024-09-19T04:58:47.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:47 smithi096 ceph-mon[35085]: osdmap e37: 6 total, 6 up, 6 in 2024-09-19T04:58:47.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:47 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T04:58:47.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:47 smithi096 ceph-mon[35085]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:58:47.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:47 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:58:47.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:47 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:58:47.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:47 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:58:47.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:47 smithi096 ceph-mon[35085]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-19T04:58:48.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:48 smithi151 ceph-mon[35425]: osdmap e38: 6 total, 6 up, 6 in 2024-09-19T04:58:48.676 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:48 smithi043 ceph-mon[30800]: osdmap e38: 6 total, 6 up, 6 in 2024-09-19T04:58:48.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:48 smithi096 ceph-mon[35085]: osdmap e38: 6 total, 6 up, 6 in 2024-09-19T04:58:49.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:49 smithi151 ceph-mon[35425]: osdmap e39: 6 total, 6 up, 6 in 2024-09-19T04:58:49.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:49 smithi151 ceph-mon[35425]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:58:49.710 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:49 smithi043 ceph-mon[30800]: osdmap e39: 6 total, 6 up, 6 in 2024-09-19T04:58:49.710 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:49 smithi043 ceph-mon[30800]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:58:49.722 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:49 smithi096 ceph-mon[35085]: osdmap e39: 6 total, 6 up, 6 in 2024-09-19T04:58:49.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:49 smithi096 ceph-mon[35085]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:58:50.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:50 smithi151 ceph-mon[35425]: osdmap e40: 6 total, 6 up, 6 in 2024-09-19T04:58:50.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:50 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/1503326219' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb8457ed-7bb9-4435-9d15-c1cbfbddfcf3"}]: dispatch 2024-09-19T04:58:50.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:50 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/1503326219' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb8457ed-7bb9-4435-9d15-c1cbfbddfcf3"}]': finished 2024-09-19T04:58:50.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:50 smithi151 ceph-mon[35425]: osdmap e41: 7 total, 6 up, 7 in 2024-09-19T04:58:50.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:50 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:58:50.690 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:50 smithi043 ceph-mon[30800]: osdmap e40: 6 total, 6 up, 6 in 2024-09-19T04:58:50.691 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:50 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/1503326219' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb8457ed-7bb9-4435-9d15-c1cbfbddfcf3"}]: dispatch 2024-09-19T04:58:50.691 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:50 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/1503326219' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb8457ed-7bb9-4435-9d15-c1cbfbddfcf3"}]': finished 2024-09-19T04:58:50.691 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:50 smithi043 ceph-mon[30800]: osdmap e41: 7 total, 6 up, 7 in 2024-09-19T04:58:50.691 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:50 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:58:50.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:50 smithi096 ceph-mon[35085]: osdmap e40: 6 total, 6 up, 6 in 2024-09-19T04:58:50.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:50 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/1503326219' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb8457ed-7bb9-4435-9d15-c1cbfbddfcf3"}]: dispatch 2024-09-19T04:58:50.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:50 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/1503326219' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb8457ed-7bb9-4435-9d15-c1cbfbddfcf3"}]': finished 2024-09-19T04:58:50.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:50 smithi096 ceph-mon[35085]: osdmap e41: 7 total, 6 up, 7 in 2024-09-19T04:58:50.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:50 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:58:51.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:51 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/2718859190' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:58:51.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:51 smithi151 ceph-mon[35425]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:58:51.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:51 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/2718859190' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:58:51.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:51 smithi096 ceph-mon[35085]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:58:51.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:51 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/2718859190' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:58:51.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:51 smithi043 ceph-mon[30800]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:58:53.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:53 smithi151 ceph-mon[35425]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-19T04:58:53.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:53 smithi096 ceph-mon[35085]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-19T04:58:53.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:53 smithi043 ceph-mon[30800]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-19T04:58:55.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:55 smithi043 ceph-mon[30800]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-19T04:58:55.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:55 smithi151 ceph-mon[35425]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-19T04:58:55.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:55 smithi096 ceph-mon[35085]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-19T04:58:57.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:57 smithi151 ceph-mon[35425]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T04:58:57.450 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:57 smithi043 ceph-mon[30800]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T04:58:57.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:57 smithi096 ceph-mon[35085]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T04:58:59.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:58:59 smithi151 ceph-mon[35425]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-19T04:58:59.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:58:59 smithi096 ceph-mon[35085]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-19T04:58:59.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:58:59 smithi043 ceph-mon[30800]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-19T04:59:00.188 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:00 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T04:59:00.188 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:00 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:00.188 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:00 smithi043 ceph-mon[30800]: Deploying daemon osd.6 on smithi043 2024-09-19T04:59:00.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:00 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T04:59:00.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:00 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:00.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:00 smithi151 ceph-mon[35425]: Deploying daemon osd.6 on smithi043 2024-09-19T04:59:00.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:00 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T04:59:00.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:00 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:00.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:00 smithi096 ceph-mon[35085]: Deploying daemon osd.6 on smithi043 2024-09-19T04:59:01.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:01 smithi043 ceph-mon[30800]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-19T04:59:01.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:01 smithi096 ceph-mon[35085]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-19T04:59:01.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:01 smithi151 ceph-mon[35425]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-19T04:59:03.332 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:03 smithi043 ceph-mon[30800]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T04:59:03.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:03 smithi151 ceph-mon[35425]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T04:59:03.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:03 smithi096 ceph-mon[35085]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T04:59:05.374 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:05 smithi043 ceph-mon[30800]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:59:05.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:05 smithi151 ceph-mon[35425]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:59:05.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:05 smithi096 ceph-mon[35085]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:59:07.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:06 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:59:07.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:06 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:07.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:06 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:07.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:06 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:59:07.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:06 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:07.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:06 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:07.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:06 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:59:07.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:06 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:07.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:06 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:08.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:07 smithi043 ceph-mon[30800]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:59:08.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:07 smithi151 ceph-mon[35425]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:59:08.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:07 smithi096 ceph-mon[35085]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:59:09.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:09 smithi043 ceph-mon[30800]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:59:09.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:09 smithi151 ceph-mon[35425]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:59:09.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:09 smithi096 ceph-mon[35085]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:59:11.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:11 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:11.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:11 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:11.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:11 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:11.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:11 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:59:11.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:11 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:11.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:11 smithi096 ceph-mon[35085]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:59:11.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:11 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:11.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:11 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:11.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:11 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:11.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:11 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:59:11.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:11 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:11.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:11 smithi043 ceph-mon[30800]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:59:11.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:59:11.628+0000 7f75fd600640 1 -- 172.21.15.43:0/3551469555 <== mgr.14150 v2:172.21.15.43:6800/3167108925 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7600062630 con 0x7f75e80a32f0 2024-09-19T04:59:11.629 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 6 on host 'smithi043' 2024-09-19T04:59:11.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:59:11.630+0000 7f7607600640 1 -- 172.21.15.43:0/3551469555 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f75e80a32f0 msgr2=0x7f75e80a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:59:11.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:59:11.630+0000 7f7607600640 1 --2- 172.21.15.43:0/3551469555 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f75e80a32f0 0x7f75e80a57b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f760019b5c0 tx=0x7f75ec00b410 comp rx=0 tx=0).stop 2024-09-19T04:59:11.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:59:11.630+0000 7f7607600640 1 -- 172.21.15.43:0/3551469555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7600103e00 msgr2=0x7f760019a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:59:11.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:59:11.630+0000 7f7607600640 1 --2- 172.21.15.43:0/3551469555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7600103e00 0x7f760019a0a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f75f4035040 tx=0x7f75f4033d90 comp rx=0 tx=0).stop 2024-09-19T04:59:11.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:59:11.630+0000 7f7607600640 1 -- 172.21.15.43:0/3551469555 shutdown_connections 2024-09-19T04:59:11.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:59:11.630+0000 7f7607600640 1 --2- 172.21.15.43:0/3551469555 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f75e80a32f0 0x7f75e80a57b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:11.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:59:11.630+0000 7f7607600640 1 --2- 172.21.15.43:0/3551469555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7600105920 0x7f76001a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:11.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:59:11.630+0000 7f7607600640 1 --2- 172.21.15.43:0/3551469555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7600104f60 0x7f760019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:11.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:59:11.630+0000 7f7607600640 1 --2- 172.21.15.43:0/3551469555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7600103e00 0x7f760019a0a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:11.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:59:11.630+0000 7f7607600640 1 -- 172.21.15.43:0/3551469555 >> 172.21.15.43:0/3551469555 conn(0x7f76000ff5b0 msgr2=0x7f7600108370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:59:11.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:59:11.631+0000 7f7607600640 1 -- 172.21.15.43:0/3551469555 shutdown_connections 2024-09-19T04:59:11.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T04:59:11.631+0000 7f7607600640 1 -- 172.21.15.43:0/3551469555 wait complete. 2024-09-19T04:59:11.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:11 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:11.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:11 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:11.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:11 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:11.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:11 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:59:11.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:11 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:11.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:11 smithi151 ceph-mon[35425]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:59:12.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:12 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:59:12.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:12 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:12.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:12 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:12.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:12 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:59:12.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:12 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:12.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:12 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:12.511 DEBUG:teuthology.orchestra.run.smithi043:osd.6> sudo journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@osd.6.service 2024-09-19T04:59:12.515 INFO:tasks.cephadm:Deploying osd.7 on smithi096 with /dev/vg_nvme/lv_2... 2024-09-19T04:59:12.515 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T04:59:12.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:12 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:59:12.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:12 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:12.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:12 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:12.771 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:59:12.780 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 04:59:12 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T04:59:12.696+0000 7fdd3f241640 -1 osd.6 0 log_to_monitors true 2024-09-19T04:59:13.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:13 smithi096 ceph-mon[35085]: from='osd.6 [v2:172.21.15.43:6818/2008630290,v1:172.21.15.43:6819/2008630290]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T04:59:13.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:13 smithi096 ceph-mon[35085]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:59:13.475 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:13 smithi043 ceph-mon[30800]: from='osd.6 [v2:172.21.15.43:6818/2008630290,v1:172.21.15.43:6819/2008630290]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T04:59:13.475 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:13 smithi043 ceph-mon[30800]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:59:13.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:13 smithi151 ceph-mon[35425]: from='osd.6 [v2:172.21.15.43:6818/2008630290,v1:172.21.15.43:6819/2008630290]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T04:59:13.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:13 smithi151 ceph-mon[35425]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:59:14.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:14 smithi096 ceph-mon[35085]: from='osd.6 [v2:172.21.15.43:6818/2008630290,v1:172.21.15.43:6819/2008630290]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T04:59:14.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:14 smithi096 ceph-mon[35085]: osdmap e42: 7 total, 6 up, 7 in 2024-09-19T04:59:14.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:14 smithi096 ceph-mon[35085]: from='osd.6 [v2:172.21.15.43:6818/2008630290,v1:172.21.15.43:6819/2008630290]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-19T04:59:14.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:14 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:59:14.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:14 smithi043 ceph-mon[30800]: from='osd.6 [v2:172.21.15.43:6818/2008630290,v1:172.21.15.43:6819/2008630290]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T04:59:14.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:14 smithi043 ceph-mon[30800]: osdmap e42: 7 total, 6 up, 7 in 2024-09-19T04:59:14.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:14 smithi043 ceph-mon[30800]: from='osd.6 [v2:172.21.15.43:6818/2008630290,v1:172.21.15.43:6819/2008630290]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-19T04:59:14.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:14 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:59:14.531 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 04:59:14 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T04:59:14.209+0000 7fdd39e00640 -1 osd.6 0 waiting for initial osdmap 2024-09-19T04:59:14.531 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 04:59:14 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T04:59:14.218+0000 7fdd34e00640 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:59:14.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:14 smithi151 ceph-mon[35425]: from='osd.6 [v2:172.21.15.43:6818/2008630290,v1:172.21.15.43:6819/2008630290]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T04:59:14.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:14 smithi151 ceph-mon[35425]: osdmap e42: 7 total, 6 up, 7 in 2024-09-19T04:59:14.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:14 smithi151 ceph-mon[35425]: from='osd.6 [v2:172.21.15.43:6818/2008630290,v1:172.21.15.43:6819/2008630290]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-19T04:59:14.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:14 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:59:15.235 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-19T04:59:15.261 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi096:vg_nvme/lv_2 2024-09-19T04:59:15.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:15 smithi096 ceph-mon[35085]: from='osd.6 [v2:172.21.15.43:6818/2008630290,v1:172.21.15.43:6819/2008630290]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-19T04:59:15.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:15 smithi096 ceph-mon[35085]: osdmap e43: 7 total, 6 up, 7 in 2024-09-19T04:59:15.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:15 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:59:15.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:15 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:59:15.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:15 smithi096 ceph-mon[35085]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:59:15.523 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T04:59:15.529 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:15 smithi043 ceph-mon[30800]: from='osd.6 [v2:172.21.15.43:6818/2008630290,v1:172.21.15.43:6819/2008630290]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-19T04:59:15.529 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:15 smithi043 ceph-mon[30800]: osdmap e43: 7 total, 6 up, 7 in 2024-09-19T04:59:15.529 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:15 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:59:15.529 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:15 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:59:15.529 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:15 smithi043 ceph-mon[30800]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:59:15.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:15 smithi151 ceph-mon[35425]: from='osd.6 [v2:172.21.15.43:6818/2008630290,v1:172.21.15.43:6819/2008630290]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-19T04:59:15.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:15 smithi151 ceph-mon[35425]: osdmap e43: 7 total, 6 up, 7 in 2024-09-19T04:59:15.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:15 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:59:15.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:15 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:59:15.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:15 smithi151 ceph-mon[35425]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-19T04:59:16.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:16.011+0000 7f5015844640 1 -- 172.21.15.96:0/3768393387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50101078f0 msgr2=0x7f5010109d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:59:16.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:16.011+0000 7f5015844640 1 --2- 172.21.15.96:0/3768393387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50101078f0 0x7f5010109d40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5008009ca0 tx=0x7f500802a8c0 comp rx=0 tx=0).stop 2024-09-19T04:59:16.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:16.011+0000 7f5015844640 1 -- 172.21.15.96:0/3768393387 shutdown_connections 2024-09-19T04:59:16.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:16.011+0000 7f5015844640 1 --2- 172.21.15.96:0/3768393387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50101078f0 0x7f5010109d40 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:16.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:16.011+0000 7f5015844640 1 --2- 172.21.15.96:0/3768393387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5010104fc0 0x7f50101073b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:16.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:16.011+0000 7f5015844640 1 --2- 172.21.15.96:0/3768393387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5010102690 0x7f5010104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:16.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:16.011+0000 7f5015844640 1 -- 172.21.15.96:0/3768393387 >> 172.21.15.96:0/3768393387 conn(0x7f50100fc290 msgr2=0x7f50100fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:59:16.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:16.011+0000 7f5015844640 1 -- 172.21.15.96:0/3768393387 shutdown_connections 2024-09-19T04:59:16.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:16.011+0000 7f5015844640 1 -- 172.21.15.96:0/3768393387 wait complete. 2024-09-19T04:59:16.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:16.012+0000 7f5015844640 1 Processor -- start 2024-09-19T04:59:16.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:16.012+0000 7f5015844640 1 -- start start 2024-09-19T04:59:16.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:16.012+0000 7f5015844640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5010102690 0x7f5010197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:59:16.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:16.012+0000 7f5015844640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5010104fc0 0x7f50101983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:59:16.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:16.012+0000 7f5015844640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50101078f0 0x7f501019f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:59:16.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:16.012+0000 7f5015844640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f501010d8e0 con 0x7f50101078f0 2024-09-19T04:59:16.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:16.012+0000 7f5015844640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f501010d760 con 0x7f5010102690 2024-09-19T04:59:16.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:16.012+0000 7f5015844640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f501010da60 con 0x7f5010104fc0 2024-09-19T04:59:16.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:16.012+0000 7f500effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5010102690 0x7f5010197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:59:16.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:16.012+0000 7f500effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5010102690 0x7f5010197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51132/0 (socket says 172.21.15.96:51132) 2024-09-19T04:59:16.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:16.013+0000 7f500effd640 1 -- 172.21.15.96:0/64658581 learned_addr learned my addr 172.21.15.96:0/64658581 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-19T04:59:16.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:16.013+0000 7f500f7fe640 1 --2- 172.21.15.96:0/64658581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50101078f0 0x7f501019f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:59:16.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:16.013+0000 7f5007fff640 1 --2- 172.21.15.96:0/64658581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5010104fc0 0x7f50101983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:59:16.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:16.013+0000 7f500effd640 1 -- 172.21.15.96:0/64658581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5010104fc0 msgr2=0x7f50101983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:59:16.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:16.013+0000 7f500effd640 1 --2- 172.21.15.96:0/64658581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5010104fc0 0x7f50101983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:16.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:16.013+0000 7f500effd640 1 -- 172.21.15.96:0/64658581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50101078f0 msgr2=0x7f501019f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:59:16.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:16.013+0000 7f500effd640 1 --2- 172.21.15.96:0/64658581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50101078f0 0x7f501019f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:16.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:16.013+0000 7f500effd640 1 -- 172.21.15.96:0/64658581 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f501019fab0 con 0x7f5010102690 2024-09-19T04:59:16.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:16.013+0000 7f5007fff640 1 --2- 172.21.15.96:0/64658581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5010104fc0 0x7f50101983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:59:16.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:16.013+0000 7f500effd640 1 --2- 172.21.15.96:0/64658581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5010102690 0x7f5010197e80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f500000c920 tx=0x7f500000cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:59:16.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:16.013+0000 7f5014842640 1 -- 172.21.15.96:0/64658581 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5000007c70 con 0x7f5010102690 2024-09-19T04:59:16.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:16.013+0000 7f5014842640 1 -- 172.21.15.96:0/64658581 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5000007e10 con 0x7f5010102690 2024-09-19T04:59:16.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:16.014+0000 7f5014842640 1 -- 172.21.15.96:0/64658581 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5000018890 con 0x7f5010102690 2024-09-19T04:59:16.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:16.014+0000 7f5015844640 1 -- 172.21.15.96:0/64658581 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f501019fda0 con 0x7f5010102690 2024-09-19T04:59:16.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:16.014+0000 7f5015844640 1 -- 172.21.15.96:0/64658581 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50101a0200 con 0x7f5010102690 2024-09-19T04:59:16.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:16.015+0000 7f5015844640 1 -- 172.21.15.96:0/64658581 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50100fdd80 con 0x7f5010102690 2024-09-19T04:59:16.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:16.016+0000 7f5014842640 1 -- 172.21.15.96:0/64658581 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f5000018a30 con 0x7f5010102690 2024-09-19T04:59:16.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:16.016+0000 7f5014842640 1 --2- 172.21.15.96:0/64658581 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f4ffc0a3270 0x7f4ffc0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:59:16.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:16.017+0000 7f5014842640 1 -- 172.21.15.96:0/64658581 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4965+0+0 (secure 0 0 0) 0x7f50000ced20 con 0x7f5010102690 2024-09-19T04:59:16.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:16.017+0000 7f5007fff640 1 --2- 172.21.15.96:0/64658581 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f4ffc0a3270 0x7f4ffc0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:59:16.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:16.017+0000 7f5007fff640 1 --2- 172.21.15.96:0/64658581 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f4ffc0a3270 0x7f4ffc0a5730 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f50101993a0 tx=0x7f4ff8005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:59:16.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:16.021+0000 7f5014842640 1 -- 172.21.15.96:0/64658581 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5000014030 con 0x7f5010102690 2024-09-19T04:59:16.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:16.199+0000 7f5015844640 1 -- 172.21.15.96:0/64658581 --> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f5010062630 con 0x7f4ffc0a3270 2024-09-19T04:59:16.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:16 smithi096 ceph-mon[35085]: purged_snaps scrub starts 2024-09-19T04:59:16.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:16 smithi096 ceph-mon[35085]: purged_snaps scrub ok 2024-09-19T04:59:16.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:16 smithi096 ceph-mon[35085]: osd.6 [v2:172.21.15.43:6818/2008630290,v1:172.21.15.43:6819/2008630290] boot 2024-09-19T04:59:16.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:16 smithi096 ceph-mon[35085]: osdmap e44: 7 total, 7 up, 7 in 2024-09-19T04:59:16.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:16 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:59:16.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:16 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:59:16.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:16 smithi043 ceph-mon[30800]: purged_snaps scrub starts 2024-09-19T04:59:16.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:16 smithi043 ceph-mon[30800]: purged_snaps scrub ok 2024-09-19T04:59:16.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:16 smithi043 ceph-mon[30800]: osd.6 [v2:172.21.15.43:6818/2008630290,v1:172.21.15.43:6819/2008630290] boot 2024-09-19T04:59:16.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:16 smithi043 ceph-mon[30800]: osdmap e44: 7 total, 7 up, 7 in 2024-09-19T04:59:16.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:16 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:59:16.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:16 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:59:16.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:16 smithi151 ceph-mon[35425]: purged_snaps scrub starts 2024-09-19T04:59:16.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:16 smithi151 ceph-mon[35425]: purged_snaps scrub ok 2024-09-19T04:59:16.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:16 smithi151 ceph-mon[35425]: osd.6 [v2:172.21.15.43:6818/2008630290,v1:172.21.15.43:6819/2008630290] boot 2024-09-19T04:59:16.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:16 smithi151 ceph-mon[35425]: osdmap e44: 7 total, 7 up, 7 in 2024-09-19T04:59:16.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:16 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T04:59:16.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:16 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:59:17.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:17 smithi096 ceph-mon[35085]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:59:17.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:17 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:59:17.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:17 smithi096 ceph-mon[35085]: osdmap e45: 7 total, 7 up, 7 in 2024-09-19T04:59:17.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:17 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:17.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:17 smithi096 ceph-mon[35085]: Detected new or changed devices on smithi043 2024-09-19T04:59:17.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:17 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:17.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:17 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:17.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:17 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:59:17.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:17 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:59:17.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:17 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:59:17.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:17 smithi096 ceph-mon[35085]: Adjusting osd_memory_target on smithi043 to 5698M 2024-09-19T04:59:17.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:17 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:17.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:17 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:17.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:17 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:59:17.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:17 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:17.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:17 smithi096 ceph-mon[35085]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:59:17.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:17 smithi043 ceph-mon[30800]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:59:17.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:17 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:59:17.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:17 smithi043 ceph-mon[30800]: osdmap e45: 7 total, 7 up, 7 in 2024-09-19T04:59:17.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:17 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:17.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:17 smithi043 ceph-mon[30800]: Detected new or changed devices on smithi043 2024-09-19T04:59:17.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:17 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:17.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:17 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:17.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:17 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:59:17.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:17 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:59:17.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:17 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:59:17.532 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:17 smithi043 ceph-mon[30800]: Adjusting osd_memory_target on smithi043 to 5698M 2024-09-19T04:59:17.532 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:17 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:17.532 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:17 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:17.532 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:17 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:59:17.532 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:17 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:17.532 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:17 smithi043 ceph-mon[30800]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:59:17.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:17 smithi151 ceph-mon[35425]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:59:17.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:17 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:59:17.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:17 smithi151 ceph-mon[35425]: osdmap e45: 7 total, 7 up, 7 in 2024-09-19T04:59:17.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:17 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:17.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:17 smithi151 ceph-mon[35425]: Detected new or changed devices on smithi043 2024-09-19T04:59:17.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:17 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:17.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:17 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:17.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:17 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:59:17.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:17 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:59:17.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:17 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:59:17.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:17 smithi151 ceph-mon[35425]: Adjusting osd_memory_target on smithi043 to 5698M 2024-09-19T04:59:17.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:17 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:17.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:17 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:17.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:17 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:59:17.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:17 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:17.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:17 smithi151 ceph-mon[35425]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:59:18.904 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:18 smithi151 ceph-mon[35425]: from='client.? 172.21.15.96:0/3252574600' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02046db8-30de-4e92-b25a-1f07d6190e91"}]: dispatch 2024-09-19T04:59:18.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:18 smithi151 ceph-mon[35425]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02046db8-30de-4e92-b25a-1f07d6190e91"}]: dispatch 2024-09-19T04:59:18.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:18 smithi151 ceph-mon[35425]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "02046db8-30de-4e92-b25a-1f07d6190e91"}]': finished 2024-09-19T04:59:18.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:18 smithi151 ceph-mon[35425]: osdmap e46: 8 total, 7 up, 8 in 2024-09-19T04:59:18.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:18 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:59:18.922 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:18 smithi096 ceph-mon[35085]: from='client.? 172.21.15.96:0/3252574600' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02046db8-30de-4e92-b25a-1f07d6190e91"}]: dispatch 2024-09-19T04:59:18.923 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:18 smithi096 ceph-mon[35085]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02046db8-30de-4e92-b25a-1f07d6190e91"}]: dispatch 2024-09-19T04:59:18.923 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:18 smithi096 ceph-mon[35085]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "02046db8-30de-4e92-b25a-1f07d6190e91"}]': finished 2024-09-19T04:59:18.923 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:18 smithi096 ceph-mon[35085]: osdmap e46: 8 total, 7 up, 8 in 2024-09-19T04:59:18.923 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:18 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:59:19.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:18 smithi043 ceph-mon[30800]: from='client.? 172.21.15.96:0/3252574600' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02046db8-30de-4e92-b25a-1f07d6190e91"}]: dispatch 2024-09-19T04:59:19.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:18 smithi043 ceph-mon[30800]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02046db8-30de-4e92-b25a-1f07d6190e91"}]: dispatch 2024-09-19T04:59:19.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:18 smithi043 ceph-mon[30800]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "02046db8-30de-4e92-b25a-1f07d6190e91"}]': finished 2024-09-19T04:59:19.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:18 smithi043 ceph-mon[30800]: osdmap e46: 8 total, 7 up, 8 in 2024-09-19T04:59:19.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:18 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:59:19.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:19 smithi151 ceph-mon[35425]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:59:19.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:19 smithi151 ceph-mon[35425]: from='client.? 172.21.15.96:0/4117448294' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:59:19.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:19 smithi096 ceph-mon[35085]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:59:19.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:19 smithi096 ceph-mon[35085]: from='client.? 172.21.15.96:0/4117448294' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:59:20.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:19 smithi043 ceph-mon[30800]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:59:20.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:19 smithi043 ceph-mon[30800]: from='client.? 172.21.15.96:0/4117448294' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:59:21.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:21 smithi096 ceph-mon[35085]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:59:21.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:21 smithi043 ceph-mon[30800]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:59:21.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:21 smithi151 ceph-mon[35425]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:59:23.409 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:23 smithi096 ceph-mon[35085]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:59:23.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:23 smithi043 ceph-mon[30800]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:59:23.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:23 smithi151 ceph-mon[35425]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:59:25.379 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:25 smithi096 ceph-mon[35085]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:59:25.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:25 smithi043 ceph-mon[30800]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:59:25.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:25 smithi151 ceph-mon[35425]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:59:26.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:26 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T04:59:26.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:26 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:26.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:26 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T04:59:26.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:26 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:26.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:26 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T04:59:26.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:26 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:27.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:27 smithi096 ceph-mon[35085]: Deploying daemon osd.7 on smithi096 2024-09-19T04:59:27.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:27 smithi096 ceph-mon[35085]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:59:27.507 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:27 smithi043 ceph-mon[30800]: Deploying daemon osd.7 on smithi096 2024-09-19T04:59:27.507 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:27 smithi043 ceph-mon[30800]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:59:27.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:27 smithi151 ceph-mon[35425]: Deploying daemon osd.7 on smithi096 2024-09-19T04:59:27.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:27 smithi151 ceph-mon[35425]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:59:29.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:29 smithi096 ceph-mon[35085]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:59:29.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:29 smithi043 ceph-mon[30800]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:59:29.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:29 smithi151 ceph-mon[35425]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:59:31.446 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:31 smithi096 ceph-mon[35085]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:59:31.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:31 smithi043 ceph-mon[30800]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:59:31.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:31 smithi151 ceph-mon[35425]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:59:32.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:32 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:59:32.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:32 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:32.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:32 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:32.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:32 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:59:32.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:32 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:32.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:32 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:32.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:32 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:59:32.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:32 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:32.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:32 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:33.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:33 smithi096 ceph-mon[35085]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:59:33.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:33 smithi043 ceph-mon[30800]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:59:33.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:33 smithi151 ceph-mon[35425]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:59:35.846 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 7 on host 'smithi096' 2024-09-19T04:59:35.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:35.844+0000 7f5014842640 1 -- 172.21.15.96:0/64658581 <== mgr.14150 v2:172.21.15.43:6800/3167108925 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5010062630 con 0x7f4ffc0a3270 2024-09-19T04:59:35.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:35.846+0000 7f5015844640 1 -- 172.21.15.96:0/64658581 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f4ffc0a3270 msgr2=0x7f4ffc0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:59:35.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:35.846+0000 7f5015844640 1 --2- 172.21.15.96:0/64658581 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f4ffc0a3270 0x7f4ffc0a5730 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f50101993a0 tx=0x7f4ff8005e50 comp rx=0 tx=0).stop 2024-09-19T04:59:35.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:35.846+0000 7f5015844640 1 -- 172.21.15.96:0/64658581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5010102690 msgr2=0x7f5010197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:59:35.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:35.846+0000 7f5015844640 1 --2- 172.21.15.96:0/64658581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5010102690 0x7f5010197e80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f500000c920 tx=0x7f500000cde0 comp rx=0 tx=0).stop 2024-09-19T04:59:35.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:35.846+0000 7f5015844640 1 -- 172.21.15.96:0/64658581 shutdown_connections 2024-09-19T04:59:35.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:35.846+0000 7f5015844640 1 --2- 172.21.15.96:0/64658581 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f4ffc0a3270 0x7f4ffc0a5730 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:35.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:35.846+0000 7f5015844640 1 --2- 172.21.15.96:0/64658581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50101078f0 0x7f501019f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:35.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:35.846+0000 7f5015844640 1 --2- 172.21.15.96:0/64658581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5010104fc0 0x7f50101983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:35.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:35.846+0000 7f5015844640 1 --2- 172.21.15.96:0/64658581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5010102690 0x7f5010197e80 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:35.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:35.846+0000 7f5015844640 1 -- 172.21.15.96:0/64658581 >> 172.21.15.96:0/64658581 conn(0x7f50100fc290 msgr2=0x7f5010102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:59:35.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:35.846+0000 7f5015844640 1 -- 172.21.15.96:0/64658581 shutdown_connections 2024-09-19T04:59:35.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T04:59:35.846+0000 7f5015844640 1 -- 172.21.15.96:0/64658581 wait complete. 2024-09-19T04:59:36.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:35 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:36.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:35 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:36.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:35 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:36.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:35 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:59:36.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:35 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:36.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:35 smithi151 ceph-mon[35425]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:59:36.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:35 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:59:36.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:35 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:36.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:35 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:36.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:35 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:36.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:35 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:36.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:35 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:36.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:35 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:59:36.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:35 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:36.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:35 smithi096 ceph-mon[35085]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:59:36.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:35 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:59:36.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:35 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:36.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:35 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:36.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:35 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:36.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:35 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:36.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:35 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:36.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:35 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:59:36.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:35 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:36.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:35 smithi043 ceph-mon[30800]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:59:36.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:35 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:59:36.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:35 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:36.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:35 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:36.518 DEBUG:teuthology.orchestra.run.smithi096:osd.7> sudo journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@osd.7.service 2024-09-19T04:59:36.521 INFO:tasks.cephadm:Deploying osd.8 on smithi151 with /dev/vg_nvme/lv_2... 2024-09-19T04:59:36.521 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T04:59:36.775 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:59:37.390 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:37 smithi096 ceph-mon[35085]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:59:37.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:37 smithi151 ceph-mon[35425]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:59:37.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:37 smithi043 ceph-mon[30800]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:59:38.223 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 04:59:38 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T04:59:38.174+0000 7fae21c96640 -1 osd.7 0 log_to_monitors true 2024-09-19T04:59:38.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:38 smithi043 ceph-mon[30800]: from='osd.7 [v2:172.21.15.96:6816/3588713275,v1:172.21.15.96:6817/3588713275]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T04:59:38.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:38 smithi043 ceph-mon[30800]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T04:59:38.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:38 smithi151 ceph-mon[35425]: from='osd.7 [v2:172.21.15.96:6816/3588713275,v1:172.21.15.96:6817/3588713275]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T04:59:38.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:38 smithi151 ceph-mon[35425]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T04:59:38.715 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:38 smithi096 ceph-mon[35085]: from='osd.7 [v2:172.21.15.96:6816/3588713275,v1:172.21.15.96:6817/3588713275]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T04:59:38.716 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:38 smithi096 ceph-mon[35085]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T04:59:39.356 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 04:59:39 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T04:59:39.251+0000 7fae1dc19640 -1 osd.7 0 waiting for initial osdmap 2024-09-19T04:59:39.356 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 04:59:39 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T04:59:39.259+0000 7fae19a41640 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:59:39.356 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:39 smithi096 ceph-mon[35085]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T04:59:39.356 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:39 smithi096 ceph-mon[35085]: osdmap e47: 8 total, 7 up, 8 in 2024-09-19T04:59:39.356 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:39 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:59:39.357 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:39 smithi096 ceph-mon[35085]: from='osd.7 [v2:172.21.15.96:6816/3588713275,v1:172.21.15.96:6817/3588713275]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T04:59:39.357 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:39 smithi096 ceph-mon[35085]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T04:59:39.357 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:39 smithi096 ceph-mon[35085]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:59:39.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:39 smithi151 ceph-mon[35425]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T04:59:39.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:39 smithi151 ceph-mon[35425]: osdmap e47: 8 total, 7 up, 8 in 2024-09-19T04:59:39.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:39 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:59:39.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:39 smithi151 ceph-mon[35425]: from='osd.7 [v2:172.21.15.96:6816/3588713275,v1:172.21.15.96:6817/3588713275]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T04:59:39.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:39 smithi151 ceph-mon[35425]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T04:59:39.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:39 smithi151 ceph-mon[35425]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:59:39.461 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T04:59:39.485 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi151:vg_nvme/lv_2 2024-09-19T04:59:39.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:39 smithi043 ceph-mon[30800]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T04:59:39.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:39 smithi043 ceph-mon[30800]: osdmap e47: 8 total, 7 up, 8 in 2024-09-19T04:59:39.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:39 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:59:39.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:39 smithi043 ceph-mon[30800]: from='osd.7 [v2:172.21.15.96:6816/3588713275,v1:172.21.15.96:6817/3588713275]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T04:59:39.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:39 smithi043 ceph-mon[30800]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T04:59:39.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:39 smithi043 ceph-mon[30800]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-19T04:59:39.739 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T04:59:40.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:59:40.249+0000 7fa459532640 1 -- 172.21.15.151:0/2298117427 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa454101cd0 msgr2=0x7fa454102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:59:40.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:59:40.249+0000 7fa459532640 1 --2- 172.21.15.151:0/2298117427 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa454101cd0 0x7fa454102150 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa448009900 tx=0x7fa44802a410 comp rx=0 tx=0).stop 2024-09-19T04:59:40.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:59:40.249+0000 7fa459532640 1 -- 172.21.15.151:0/2298117427 shutdown_connections 2024-09-19T04:59:40.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:59:40.249+0000 7fa459532640 1 --2- 172.21.15.151:0/2298117427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa454102690 0x7fa454108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:40.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:59:40.249+0000 7fa459532640 1 --2- 172.21.15.151:0/2298117427 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa454101cd0 0x7fa454102150 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:40.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:59:40.249+0000 7fa459532640 1 --2- 172.21.15.151:0/2298117427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa454100ad0 0x7fa454100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:40.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:59:40.249+0000 7fa459532640 1 -- 172.21.15.151:0/2298117427 >> 172.21.15.151:0/2298117427 conn(0x7fa4540fc280 msgr2=0x7fa4540fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:59:40.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:59:40.249+0000 7fa459532640 1 -- 172.21.15.151:0/2298117427 shutdown_connections 2024-09-19T04:59:40.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:59:40.249+0000 7fa459532640 1 -- 172.21.15.151:0/2298117427 wait complete. 2024-09-19T04:59:40.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:59:40.250+0000 7fa459532640 1 Processor -- start 2024-09-19T04:59:40.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:59:40.250+0000 7fa459532640 1 -- start start 2024-09-19T04:59:40.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:59:40.250+0000 7fa459532640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa454100ad0 0x7fa45406aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:59:40.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:59:40.250+0000 7fa459532640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa454101cd0 0x7fa45419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:59:40.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:59:40.250+0000 7fa459532640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa454102690 0x7fa45419f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:59:40.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:59:40.250+0000 7fa459532640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa45410c750 con 0x7fa454102690 2024-09-19T04:59:40.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:59:40.250+0000 7fa459532640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa45410c5d0 con 0x7fa454100ad0 2024-09-19T04:59:40.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:59:40.250+0000 7fa4527fc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa454101cd0 0x7fa45419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:59:40.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:59:40.250+0000 7fa459532640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa45410c8d0 con 0x7fa454101cd0 2024-09-19T04:59:40.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:59:40.250+0000 7fa4527fc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa454101cd0 0x7fa45419cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48772/0 (socket says 172.21.15.151:48772) 2024-09-19T04:59:40.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:59:40.250+0000 7fa4527fc640 1 -- 172.21.15.151:0/3443575481 learned_addr learned my addr 172.21.15.151:0/3443575481 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-19T04:59:40.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:59:40.250+0000 7fa452ffd640 1 --2- 172.21.15.151:0/3443575481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa454100ad0 0x7fa45406aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:59:40.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:59:40.250+0000 7fa4537fe640 1 --2- 172.21.15.151:0/3443575481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa454102690 0x7fa45419f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:59:40.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:59:40.250+0000 7fa4527fc640 1 -- 172.21.15.151:0/3443575481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa454100ad0 msgr2=0x7fa45406aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:59:40.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:59:40.250+0000 7fa4527fc640 1 --2- 172.21.15.151:0/3443575481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa454100ad0 0x7fa45406aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:40.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:59:40.250+0000 7fa4527fc640 1 -- 172.21.15.151:0/3443575481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa454102690 msgr2=0x7fa45419f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:59:40.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:59:40.250+0000 7fa4527fc640 1 --2- 172.21.15.151:0/3443575481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa454102690 0x7fa45419f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:59:40.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:59:40.250+0000 7fa4527fc640 1 -- 172.21.15.151:0/3443575481 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa45419fb60 con 0x7fa454101cd0 2024-09-19T04:59:40.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:59:40.251+0000 7fa4527fc640 1 --2- 172.21.15.151:0/3443575481 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa454101cd0 0x7fa45419cc30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa448009230 tx=0x7fa448031b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:59:40.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:59:40.256+0000 7fa43bfff640 1 -- 172.21.15.151:0/3443575481 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44803f030 con 0x7fa454101cd0 2024-09-19T04:59:40.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:59:40.256+0000 7fa459532640 1 -- 172.21.15.151:0/3443575481 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa45419fd90 con 0x7fa454101cd0 2024-09-19T04:59:40.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:59:40.256+0000 7fa459532640 1 -- 172.21.15.151:0/3443575481 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4541a0260 con 0x7fa454101cd0 2024-09-19T04:59:40.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:59:40.256+0000 7fa43bfff640 1 -- 172.21.15.151:0/3443575481 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa448003ad0 con 0x7fa454101cd0 2024-09-19T04:59:40.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:59:40.256+0000 7fa43bfff640 1 -- 172.21.15.151:0/3443575481 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa448049060 con 0x7fa454101cd0 2024-09-19T04:59:40.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:59:40.257+0000 7fa459532640 1 -- 172.21.15.151:0/3443575481 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4400052c0 con 0x7fa454101cd0 2024-09-19T04:59:40.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:59:40.258+0000 7fa43bfff640 1 -- 172.21.15.151:0/3443575481 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fa448042030 con 0x7fa454101cd0 2024-09-19T04:59:40.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:59:40.258+0000 7fa43bfff640 1 --2- 172.21.15.151:0/3443575481 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa43c0a3240 0x7fa43c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:59:40.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:59:40.259+0000 7fa43bfff640 1 -- 172.21.15.151:0/3443575481 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5430+0+0 (secure 0 0 0) 0x7fa4480eefa0 con 0x7fa454101cd0 2024-09-19T04:59:40.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:59:40.259+0000 7fa452ffd640 1 --2- 172.21.15.151:0/3443575481 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa43c0a3240 0x7fa43c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:59:40.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:59:40.259+0000 7fa452ffd640 1 --2- 172.21.15.151:0/3443575481 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa43c0a3240 0x7fa43c0a5700 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa454101b30 tx=0x7fa44400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:59:40.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:59:40.267+0000 7fa43bfff640 1 -- 172.21.15.151:0/3443575481 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa4480bc800 con 0x7fa454101cd0 2024-09-19T04:59:40.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:40 smithi151 ceph-mon[35425]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-19T04:59:40.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:40 smithi151 ceph-mon[35425]: osdmap e48: 8 total, 7 up, 8 in 2024-09-19T04:59:40.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:40 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:59:40.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:40 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:59:40.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:40 smithi151 ceph-mon[35425]: Detected new or changed devices on smithi096 2024-09-19T04:59:40.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:40 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:40.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:40 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:40.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:40 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:59:40.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:40 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:59:40.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:40 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:59:40.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:40 smithi151 ceph-mon[35425]: Adjusting osd_memory_target on smithi096 to 5696M 2024-09-19T04:59:40.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:40 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:40.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:40 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:40.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:40 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:59:40.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:40 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:40.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:40 smithi151 ceph-mon[35425]: osd.7 [v2:172.21.15.96:6816/3588713275,v1:172.21.15.96:6817/3588713275] boot 2024-09-19T04:59:40.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:40 smithi151 ceph-mon[35425]: osdmap e49: 8 total, 8 up, 8 in 2024-09-19T04:59:40.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:40 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:59:40.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T04:59:40.450+0000 7fa459532640 1 -- 172.21.15.151:0/3443575481 --> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fa440000bf0 con 0x7fa43c0a3240 2024-09-19T04:59:40.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:40 smithi043 ceph-mon[30800]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-19T04:59:40.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:40 smithi043 ceph-mon[30800]: osdmap e48: 8 total, 7 up, 8 in 2024-09-19T04:59:40.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:40 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:59:40.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:40 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:59:40.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:40 smithi043 ceph-mon[30800]: Detected new or changed devices on smithi096 2024-09-19T04:59:40.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:40 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:40.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:40 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:40.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:40 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:59:40.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:40 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:59:40.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:40 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:59:40.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:40 smithi043 ceph-mon[30800]: Adjusting osd_memory_target on smithi096 to 5696M 2024-09-19T04:59:40.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:40 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:40.532 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:40 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:40.532 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:40 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:59:40.532 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:40 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:40.532 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:40 smithi043 ceph-mon[30800]: osd.7 [v2:172.21.15.96:6816/3588713275,v1:172.21.15.96:6817/3588713275] boot 2024-09-19T04:59:40.532 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:40 smithi043 ceph-mon[30800]: osdmap e49: 8 total, 8 up, 8 in 2024-09-19T04:59:40.532 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:40 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:59:40.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:40 smithi096 ceph-mon[35085]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-19T04:59:40.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:40 smithi096 ceph-mon[35085]: osdmap e48: 8 total, 7 up, 8 in 2024-09-19T04:59:40.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:40 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:59:40.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:40 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:59:40.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:40 smithi096 ceph-mon[35085]: Detected new or changed devices on smithi096 2024-09-19T04:59:40.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:40 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:40.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:40 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:40.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:40 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:59:40.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:40 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:59:40.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:40 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T04:59:40.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:40 smithi096 ceph-mon[35085]: Adjusting osd_memory_target on smithi096 to 5696M 2024-09-19T04:59:40.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:40 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:40.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:40 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:40.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:40 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T04:59:40.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:40 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:40.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:40 smithi096 ceph-mon[35085]: osd.7 [v2:172.21.15.96:6816/3588713275,v1:172.21.15.96:6817/3588713275] boot 2024-09-19T04:59:40.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:40 smithi096 ceph-mon[35085]: osdmap e49: 8 total, 8 up, 8 in 2024-09-19T04:59:40.725 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:40 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T04:59:41.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:41 smithi151 ceph-mon[35425]: purged_snaps scrub starts 2024-09-19T04:59:41.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:41 smithi151 ceph-mon[35425]: purged_snaps scrub ok 2024-09-19T04:59:41.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:41 smithi151 ceph-mon[35425]: osdmap e50: 8 total, 8 up, 8 in 2024-09-19T04:59:41.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:41 smithi151 ceph-mon[35425]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:59:41.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:41 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:59:41.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:41 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:59:41.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:41 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:41.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:41 smithi151 ceph-mon[35425]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:59:41.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:41 smithi096 ceph-mon[35085]: purged_snaps scrub starts 2024-09-19T04:59:41.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:41 smithi096 ceph-mon[35085]: purged_snaps scrub ok 2024-09-19T04:59:41.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:41 smithi096 ceph-mon[35085]: osdmap e50: 8 total, 8 up, 8 in 2024-09-19T04:59:41.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:41 smithi096 ceph-mon[35085]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:59:41.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:41 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:59:41.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:41 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:59:41.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:41 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:41.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:41 smithi096 ceph-mon[35085]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:59:41.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:41 smithi043 ceph-mon[30800]: purged_snaps scrub starts 2024-09-19T04:59:41.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:41 smithi043 ceph-mon[30800]: purged_snaps scrub ok 2024-09-19T04:59:41.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:41 smithi043 ceph-mon[30800]: osdmap e50: 8 total, 8 up, 8 in 2024-09-19T04:59:41.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:41 smithi043 ceph-mon[30800]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T04:59:41.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:41 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T04:59:41.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:41 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T04:59:41.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:41 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:41.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:41 smithi043 ceph-mon[30800]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:59:42.430 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:42 smithi151 ceph-mon[35425]: osdmap e51: 8 total, 8 up, 8 in 2024-09-19T04:59:42.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:42 smithi096 ceph-mon[35085]: osdmap e51: 8 total, 8 up, 8 in 2024-09-19T04:59:42.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:42 smithi043 ceph-mon[30800]: osdmap e51: 8 total, 8 up, 8 in 2024-09-19T04:59:43.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:43 smithi151 ceph-mon[35425]: osdmap e52: 8 total, 8 up, 8 in 2024-09-19T04:59:43.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:43 smithi151 ceph-mon[35425]: from='client.? 172.21.15.151:0/345525037' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a5bb4218-51f3-4044-b43e-ebfc764f49ad"}]: dispatch 2024-09-19T04:59:43.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:43 smithi151 ceph-mon[35425]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a5bb4218-51f3-4044-b43e-ebfc764f49ad"}]: dispatch 2024-09-19T04:59:43.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:43 smithi151 ceph-mon[35425]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a5bb4218-51f3-4044-b43e-ebfc764f49ad"}]': finished 2024-09-19T04:59:43.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:43 smithi151 ceph-mon[35425]: osdmap e53: 9 total, 8 up, 9 in 2024-09-19T04:59:43.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:43 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:59:43.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:43 smithi151 ceph-mon[35425]: pgmap v181: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:59:43.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:43 smithi096 ceph-mon[35085]: osdmap e52: 8 total, 8 up, 8 in 2024-09-19T04:59:43.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:43 smithi096 ceph-mon[35085]: from='client.? 172.21.15.151:0/345525037' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a5bb4218-51f3-4044-b43e-ebfc764f49ad"}]: dispatch 2024-09-19T04:59:43.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:43 smithi096 ceph-mon[35085]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a5bb4218-51f3-4044-b43e-ebfc764f49ad"}]: dispatch 2024-09-19T04:59:43.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:43 smithi096 ceph-mon[35085]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a5bb4218-51f3-4044-b43e-ebfc764f49ad"}]': finished 2024-09-19T04:59:43.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:43 smithi096 ceph-mon[35085]: osdmap e53: 9 total, 8 up, 9 in 2024-09-19T04:59:43.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:43 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:59:43.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:43 smithi096 ceph-mon[35085]: pgmap v181: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:59:43.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:43 smithi043 ceph-mon[30800]: osdmap e52: 8 total, 8 up, 8 in 2024-09-19T04:59:43.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:43 smithi043 ceph-mon[30800]: from='client.? 172.21.15.151:0/345525037' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a5bb4218-51f3-4044-b43e-ebfc764f49ad"}]: dispatch 2024-09-19T04:59:43.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:43 smithi043 ceph-mon[30800]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a5bb4218-51f3-4044-b43e-ebfc764f49ad"}]: dispatch 2024-09-19T04:59:43.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:43 smithi043 ceph-mon[30800]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a5bb4218-51f3-4044-b43e-ebfc764f49ad"}]': finished 2024-09-19T04:59:43.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:43 smithi043 ceph-mon[30800]: osdmap e53: 9 total, 8 up, 9 in 2024-09-19T04:59:43.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:43 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T04:59:43.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:43 smithi043 ceph-mon[30800]: pgmap v181: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:59:44.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:44 smithi151 ceph-mon[35425]: from='client.? 172.21.15.151:0/4069817853' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:59:44.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:44 smithi096 ceph-mon[35085]: from='client.? 172.21.15.151:0/4069817853' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:59:44.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:44 smithi043 ceph-mon[30800]: from='client.? 172.21.15.151:0/4069817853' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T04:59:45.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:45 smithi096 ceph-mon[35085]: pgmap v182: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:59:45.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:45 smithi043 ceph-mon[30800]: pgmap v182: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:59:45.904 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:45 smithi151 ceph-mon[35425]: pgmap v182: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:59:47.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:47 smithi096 ceph-mon[35085]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:59:47.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:47 smithi043 ceph-mon[30800]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:59:47.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:47 smithi151 ceph-mon[35425]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:59:49.389 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:49 smithi151 ceph-mon[35425]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:59:49.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:49 smithi096 ceph-mon[35085]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:59:49.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:49 smithi043 ceph-mon[30800]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:59:51.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:51 smithi096 ceph-mon[35085]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:59:51.501 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:51 smithi151 ceph-mon[35425]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:59:51.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:51 smithi043 ceph-mon[30800]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:59:52.279 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:52 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T04:59:52.279 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:52 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:52.279 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:52 smithi151 ceph-mon[35425]: Deploying daemon osd.8 on smithi151 2024-09-19T04:59:52.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:52 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T04:59:52.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:52 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:52.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:52 smithi096 ceph-mon[35085]: Deploying daemon osd.8 on smithi151 2024-09-19T04:59:52.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:52 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T04:59:52.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:52 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T04:59:52.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:52 smithi043 ceph-mon[30800]: Deploying daemon osd.8 on smithi151 2024-09-19T04:59:53.284 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:53 smithi151 ceph-mon[35425]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:59:53.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:53 smithi096 ceph-mon[35085]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:59:53.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:53 smithi043 ceph-mon[30800]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:59:55.344 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:55 smithi151 ceph-mon[35425]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:59:55.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:55 smithi096 ceph-mon[35085]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:59:55.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:55 smithi043 ceph-mon[30800]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:59:57.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:57 smithi151 ceph-mon[35425]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:59:57.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:57 smithi096 ceph-mon[35085]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:59:57.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:57 smithi043 ceph-mon[30800]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:59:58.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:58 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:59:58.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:58 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:58.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:58 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:58.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:58 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:59:58.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:58 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:58.476 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:58 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:58.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:58 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T04:59:58.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:58 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:58.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:58 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T04:59:59.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 04:59:59 smithi151 ceph-mon[35425]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:59:59.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 04:59:59 smithi096 ceph-mon[35085]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-19T04:59:59.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 04:59:59 smithi043 ceph-mon[30800]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:00:00.309 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:00 smithi151 ceph-mon[35425]: overall HEALTH_OK 2024-09-19T05:00:00.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:00 smithi096 ceph-mon[35085]: overall HEALTH_OK 2024-09-19T05:00:00.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:00 smithi043 ceph-mon[30800]: overall HEALTH_OK 2024-09-19T05:00:01.421 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 8 on host 'smithi151' 2024-09-19T05:00:01.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:00:01.419+0000 7fa43bfff640 1 -- 172.21.15.151:0/3443575481 <== mgr.14150 v2:172.21.15.43:6800/3167108925 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa440000bf0 con 0x7fa43c0a3240 2024-09-19T05:00:01.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:00:01.422+0000 7fa459532640 1 -- 172.21.15.151:0/3443575481 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa43c0a3240 msgr2=0x7fa43c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:00:01.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:00:01.422+0000 7fa459532640 1 --2- 172.21.15.151:0/3443575481 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa43c0a3240 0x7fa43c0a5700 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa454101b30 tx=0x7fa44400b390 comp rx=0 tx=0).stop 2024-09-19T05:00:01.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:00:01.422+0000 7fa459532640 1 -- 172.21.15.151:0/3443575481 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa454101cd0 msgr2=0x7fa45419cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:00:01.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:00:01.422+0000 7fa459532640 1 --2- 172.21.15.151:0/3443575481 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa454101cd0 0x7fa45419cc30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa448009230 tx=0x7fa448031b60 comp rx=0 tx=0).stop 2024-09-19T05:00:01.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:00:01.422+0000 7fa459532640 1 -- 172.21.15.151:0/3443575481 shutdown_connections 2024-09-19T05:00:01.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:00:01.422+0000 7fa459532640 1 --2- 172.21.15.151:0/3443575481 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa43c0a3240 0x7fa43c0a5700 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:00:01.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:00:01.422+0000 7fa459532640 1 --2- 172.21.15.151:0/3443575481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa454102690 0x7fa45419f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:00:01.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:00:01.422+0000 7fa459532640 1 --2- 172.21.15.151:0/3443575481 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa454101cd0 0x7fa45419cc30 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:00:01.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:00:01.422+0000 7fa459532640 1 --2- 172.21.15.151:0/3443575481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa454100ad0 0x7fa45406aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:00:01.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:00:01.422+0000 7fa459532640 1 -- 172.21.15.151:0/3443575481 >> 172.21.15.151:0/3443575481 conn(0x7fa4540fc280 msgr2=0x7fa4541050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:00:01.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:00:01.422+0000 7fa459532640 1 -- 172.21.15.151:0/3443575481 shutdown_connections 2024-09-19T05:00:01.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:00:01.422+0000 7fa459532640 1 -- 172.21.15.151:0/3443575481 wait complete. 2024-09-19T05:00:01.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:01 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:01.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:01 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:01.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:01 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:01.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:01 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:00:01.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:01 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:01.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:01 smithi151 ceph-mon[35425]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:00:01.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:01 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:00:01.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:01 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:01.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:01 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:01.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:01 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:01.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:01 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:01.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:01 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:01.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:01 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:00:01.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:01 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:01.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:01 smithi043 ceph-mon[30800]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:00:01.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:01 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:00:01.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:01 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:01.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:01 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:01.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:01 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:01.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:01 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:01.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:01 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:01.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:01 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:00:01.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:01 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:01.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:01 smithi096 ceph-mon[35085]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:00:01.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:01 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:00:01.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:01 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:01.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:01 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:02.110 DEBUG:teuthology.orchestra.run.smithi151:osd.8> sudo journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@osd.8.service 2024-09-19T05:00:02.112 INFO:tasks.cephadm:Deploying osd.9 on smithi043 with /dev/vg_nvme/lv_1... 2024-09-19T05:00:02.112 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T05:00:02.501 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:00:03.273 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:03 smithi151 ceph-mon[35425]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:00:03.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:03 smithi096 ceph-mon[35085]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:00:03.506 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:03 smithi043 ceph-mon[30800]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:00:03.851 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:00:03 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:00:03.551+0000 7f9299ca9640 -1 osd.8 0 log_to_monitors true 2024-09-19T05:00:04.228 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:04 smithi043 ceph-mon[30800]: from='osd.8 [v2:172.21.15.151:6816/725316393,v1:172.21.15.151:6817/725316393]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T05:00:04.228 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:04 smithi043 ceph-mon[30800]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T05:00:04.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:04 smithi096 ceph-mon[35085]: from='osd.8 [v2:172.21.15.151:6816/725316393,v1:172.21.15.151:6817/725316393]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T05:00:04.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:04 smithi096 ceph-mon[35085]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T05:00:04.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:04 smithi151 ceph-mon[35425]: from='osd.8 [v2:172.21.15.151:6816/725316393,v1:172.21.15.151:6817/725316393]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T05:00:04.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:04 smithi151 ceph-mon[35425]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T05:00:05.237 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:05 smithi043 ceph-mon[30800]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T05:00:05.237 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:05 smithi043 ceph-mon[30800]: osdmap e54: 9 total, 8 up, 9 in 2024-09-19T05:00:05.237 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:05 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:00:05.237 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:05 smithi043 ceph-mon[30800]: from='osd.8 [v2:172.21.15.151:6816/725316393,v1:172.21.15.151:6817/725316393]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T05:00:05.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:05 smithi043 ceph-mon[30800]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T05:00:05.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:05 smithi043 ceph-mon[30800]: Detected new or changed devices on smithi151 2024-09-19T05:00:05.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:05 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:05.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:05 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:05.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:05 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:00:05.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:05 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:00:05.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:05 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:00:05.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:05 smithi043 ceph-mon[30800]: Adjusting osd_memory_target on smithi151 to 5696M 2024-09-19T05:00:05.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:05 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:05.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:05 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:05.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:05 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:00:05.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:05 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:05.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:05 smithi043 ceph-mon[30800]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:00:05.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:05 smithi151 ceph-mon[35425]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T05:00:05.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:05 smithi151 ceph-mon[35425]: osdmap e54: 9 total, 8 up, 9 in 2024-09-19T05:00:05.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:05 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:00:05.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:05 smithi151 ceph-mon[35425]: from='osd.8 [v2:172.21.15.151:6816/725316393,v1:172.21.15.151:6817/725316393]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T05:00:05.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:05 smithi151 ceph-mon[35425]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T05:00:05.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:05 smithi151 ceph-mon[35425]: Detected new or changed devices on smithi151 2024-09-19T05:00:05.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:05 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:05.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:05 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:05.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:05 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:00:05.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:05 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:00:05.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:05 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:00:05.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:05 smithi151 ceph-mon[35425]: Adjusting osd_memory_target on smithi151 to 5696M 2024-09-19T05:00:05.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:05 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:05.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:05 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:05.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:05 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:00:05.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:05 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:05.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:05 smithi151 ceph-mon[35425]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:00:05.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:05 smithi096 ceph-mon[35085]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T05:00:05.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:05 smithi096 ceph-mon[35085]: osdmap e54: 9 total, 8 up, 9 in 2024-09-19T05:00:05.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:05 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:00:05.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:05 smithi096 ceph-mon[35085]: from='osd.8 [v2:172.21.15.151:6816/725316393,v1:172.21.15.151:6817/725316393]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T05:00:05.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:05 smithi096 ceph-mon[35085]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T05:00:05.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:05 smithi096 ceph-mon[35085]: Detected new or changed devices on smithi151 2024-09-19T05:00:05.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:05 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:05.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:05 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:05.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:05 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:00:05.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:05 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:00:05.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:05 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:00:05.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:05 smithi096 ceph-mon[35085]: Adjusting osd_memory_target on smithi151 to 5696M 2024-09-19T05:00:05.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:05 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:05.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:05 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:05.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:05 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:00:05.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:05 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:05.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:05 smithi096 ceph-mon[35085]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:00:05.905 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:00:05 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:00:05.569+0000 7f9295c2c640 -1 osd.8 0 waiting for initial osdmap 2024-09-19T05:00:05.905 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:00:05 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:00:05.580+0000 7f928da4c640 -1 osd.8 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:00:05.928 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:00:05.962 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi043:vg_nvme/lv_1 2024-09-19T05:00:06.336 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:00:06.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:06 smithi096 ceph-mon[35085]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-19T05:00:06.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:06 smithi096 ceph-mon[35085]: osdmap e55: 9 total, 8 up, 9 in 2024-09-19T05:00:06.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:06 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:00:06.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:06 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:00:06.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:06 smithi096 ceph-mon[35085]: osdmap e56: 9 total, 8 up, 9 in 2024-09-19T05:00:06.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:06 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:00:06.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:06 smithi096 ceph-mon[35085]: from='osd.8 ' entity='osd.8' 2024-09-19T05:00:06.475 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:06 smithi043 ceph-mon[30800]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-19T05:00:06.475 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:06 smithi043 ceph-mon[30800]: osdmap e55: 9 total, 8 up, 9 in 2024-09-19T05:00:06.475 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:06 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:00:06.475 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:06 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:00:06.475 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:06 smithi043 ceph-mon[30800]: osdmap e56: 9 total, 8 up, 9 in 2024-09-19T05:00:06.475 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:06 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:00:06.475 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:06 smithi043 ceph-mon[30800]: from='osd.8 ' entity='osd.8' 2024-09-19T05:00:06.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:06 smithi151 ceph-mon[35425]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-19T05:00:06.657 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:06 smithi151 ceph-mon[35425]: osdmap e55: 9 total, 8 up, 9 in 2024-09-19T05:00:06.657 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:06 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:00:06.657 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:06 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:00:06.657 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:06 smithi151 ceph-mon[35425]: osdmap e56: 9 total, 8 up, 9 in 2024-09-19T05:00:06.657 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:06 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:00:06.657 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:06 smithi151 ceph-mon[35425]: from='osd.8 ' entity='osd.8' 2024-09-19T05:00:06.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:06.939+0000 7f0b3f800640 1 -- 172.21.15.43:0/352227199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b38103e00 msgr2=0x7f0b38104200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:00:06.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:06.939+0000 7f0b3f800640 1 --2- 172.21.15.43:0/352227199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b38103e00 0x7f0b38104200 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0b28009900 tx=0x7f0b2802a410 comp rx=0 tx=0).stop 2024-09-19T05:00:06.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:06.939+0000 7f0b3f800640 1 -- 172.21.15.43:0/352227199 shutdown_connections 2024-09-19T05:00:06.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:06.939+0000 7f0b3f800640 1 --2- 172.21.15.43:0/352227199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b38105920 0x7f0b3810c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:00:06.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:06.939+0000 7f0b3f800640 1 --2- 172.21.15.43:0/352227199 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b38104f60 0x7f0b381053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:00:06.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:06.939+0000 7f0b3f800640 1 --2- 172.21.15.43:0/352227199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b38103e00 0x7f0b38104200 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:00:06.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:06.939+0000 7f0b3f800640 1 -- 172.21.15.43:0/352227199 >> 172.21.15.43:0/352227199 conn(0x7f0b380ff5b0 msgr2=0x7f0b381019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:00:06.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:06.940+0000 7f0b3f800640 1 -- 172.21.15.43:0/352227199 shutdown_connections 2024-09-19T05:00:06.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:06.940+0000 7f0b3f800640 1 -- 172.21.15.43:0/352227199 wait complete. 2024-09-19T05:00:06.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:06.940+0000 7f0b3f800640 1 Processor -- start 2024-09-19T05:00:06.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:06.941+0000 7f0b3f800640 1 -- start start 2024-09-19T05:00:06.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:06.941+0000 7f0b3f800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b38103e00 0x7f0b3819a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:00:06.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:06.941+0000 7f0b3f800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b38104f60 0x7f0b3819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:00:06.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:06.941+0000 7f0b3f800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b38105920 0x7f0b381a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:00:06.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:06.941+0000 7f0b3f800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0b3810fab0 con 0x7f0b38103e00 2024-09-19T05:00:06.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:06.941+0000 7f0b3f800640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0b3810f930 con 0x7f0b38104f60 2024-09-19T05:00:06.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:06.941+0000 7f0b3f800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0b3810fc30 con 0x7f0b38105920 2024-09-19T05:00:06.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:06.941+0000 7f0b3ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b38103e00 0x7f0b3819a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:00:06.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:06.941+0000 7f0b3ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b38103e00 0x7f0b3819a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35260/0 (socket says 172.21.15.43:35260) 2024-09-19T05:00:06.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:06.941+0000 7f0b3ce00640 1 -- 172.21.15.43:0/1401484676 learned_addr learned my addr 172.21.15.43:0/1401484676 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:00:06.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:06.941+0000 7f0b3d800640 1 --2- 172.21.15.43:0/1401484676 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b38105920 0x7f0b381a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:00:06.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:06.941+0000 7f0b2fe00640 1 --2- 172.21.15.43:0/1401484676 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b38104f60 0x7f0b3819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:00:06.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:06.942+0000 7f0b3ce00640 1 -- 172.21.15.43:0/1401484676 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b38105920 msgr2=0x7f0b381a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:00:06.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:06.942+0000 7f0b3ce00640 1 --2- 172.21.15.43:0/1401484676 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b38105920 0x7f0b381a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:00:06.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:06.942+0000 7f0b3ce00640 1 -- 172.21.15.43:0/1401484676 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b38104f60 msgr2=0x7f0b3819a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:00:06.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:06.942+0000 7f0b3ce00640 1 --2- 172.21.15.43:0/1401484676 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b38104f60 0x7f0b3819a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:00:06.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:06.942+0000 7f0b3ce00640 1 -- 172.21.15.43:0/1401484676 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b381a1d30 con 0x7f0b38103e00 2024-09-19T05:00:06.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:06.942+0000 7f0b2fe00640 1 --2- 172.21.15.43:0/1401484676 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b38104f60 0x7f0b3819a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:00:06.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:06.942+0000 7f0b3ce00640 1 --2- 172.21.15.43:0/1401484676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b38103e00 0x7f0b3819a100 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0b28009230 tx=0x7f0b280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:00:06.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:06.942+0000 7f0b36000640 1 -- 172.21.15.43:0/1401484676 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2803f030 con 0x7f0b38103e00 2024-09-19T05:00:06.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:06.942+0000 7f0b36000640 1 -- 172.21.15.43:0/1401484676 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b28003ce0 con 0x7f0b38103e00 2024-09-19T05:00:06.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:06.943+0000 7f0b3f800640 1 -- 172.21.15.43:0/1401484676 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b381a1fc0 con 0x7f0b38103e00 2024-09-19T05:00:06.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:06.943+0000 7f0b36000640 1 -- 172.21.15.43:0/1401484676 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b28035070 con 0x7f0b38103e00 2024-09-19T05:00:06.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:06.943+0000 7f0b3f800640 1 -- 172.21.15.43:0/1401484676 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b381a23c0 con 0x7f0b38103e00 2024-09-19T05:00:06.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:06.944+0000 7f0b3f800640 1 -- 172.21.15.43:0/1401484676 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b3810b1a0 con 0x7f0b38103e00 2024-09-19T05:00:06.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:06.946+0000 7f0b36000640 1 -- 172.21.15.43:0/1401484676 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f0b28042030 con 0x7f0b38103e00 2024-09-19T05:00:06.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:06.947+0000 7f0b36000640 1 --2- 172.21.15.43:0/1401484676 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f0b1c0a3270 0x7f0b1c0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:00:06.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:06.947+0000 7f0b36000640 1 -- 172.21.15.43:0/1401484676 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5862+0+0 (secure 0 0 0) 0x7f0b280efc00 con 0x7f0b38103e00 2024-09-19T05:00:06.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:06.947+0000 7f0b2fe00640 1 --2- 172.21.15.43:0/1401484676 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f0b1c0a3270 0x7f0b1c0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:00:06.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:06.948+0000 7f0b2fe00640 1 --2- 172.21.15.43:0/1401484676 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f0b1c0a3270 0x7f0b1c0a5730 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0b3819b620 tx=0x7f0b20005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:00:06.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:06.950+0000 7f0b36000640 1 -- 172.21.15.43:0/1401484676 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0b28049030 con 0x7f0b38103e00 2024-09-19T05:00:07.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:07.150+0000 7f0b3f800640 1 -- 172.21.15.43:0/1401484676 --> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f0b38062630 con 0x7f0b1c0a3270 2024-09-19T05:00:07.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:07 smithi096 ceph-mon[35085]: purged_snaps scrub starts 2024-09-19T05:00:07.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:07 smithi096 ceph-mon[35085]: purged_snaps scrub ok 2024-09-19T05:00:07.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:07 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:00:07.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:07 smithi096 ceph-mon[35085]: osd.8 [v2:172.21.15.151:6816/725316393,v1:172.21.15.151:6817/725316393] boot 2024-09-19T05:00:07.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:07 smithi096 ceph-mon[35085]: osdmap e57: 9 total, 9 up, 9 in 2024-09-19T05:00:07.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:07 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:00:07.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:07 smithi096 ceph-mon[35085]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:00:07.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:07 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:00:07.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:07 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:00:07.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:07 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:07.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:07 smithi043 ceph-mon[30800]: purged_snaps scrub starts 2024-09-19T05:00:07.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:07 smithi043 ceph-mon[30800]: purged_snaps scrub ok 2024-09-19T05:00:07.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:07 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:00:07.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:07 smithi043 ceph-mon[30800]: osd.8 [v2:172.21.15.151:6816/725316393,v1:172.21.15.151:6817/725316393] boot 2024-09-19T05:00:07.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:07 smithi043 ceph-mon[30800]: osdmap e57: 9 total, 9 up, 9 in 2024-09-19T05:00:07.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:07 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:00:07.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:07 smithi043 ceph-mon[30800]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:00:07.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:07 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:00:07.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:07 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:00:07.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:07 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:07.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:07 smithi151 ceph-mon[35425]: purged_snaps scrub starts 2024-09-19T05:00:07.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:07 smithi151 ceph-mon[35425]: purged_snaps scrub ok 2024-09-19T05:00:07.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:07 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:00:07.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:07 smithi151 ceph-mon[35425]: osd.8 [v2:172.21.15.151:6816/725316393,v1:172.21.15.151:6817/725316393] boot 2024-09-19T05:00:07.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:07 smithi151 ceph-mon[35425]: osdmap e57: 9 total, 9 up, 9 in 2024-09-19T05:00:07.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:07 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:00:07.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:07 smithi151 ceph-mon[35425]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-19T05:00:07.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:07 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:00:07.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:07 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:00:07.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:07 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:08.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:08 smithi151 ceph-mon[35425]: from='client.14382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:00:08.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:08 smithi151 ceph-mon[35425]: osdmap e58: 9 total, 9 up, 9 in 2024-09-19T05:00:08.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:08 smithi096 ceph-mon[35085]: from='client.14382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:00:08.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:08 smithi096 ceph-mon[35085]: osdmap e58: 9 total, 9 up, 9 in 2024-09-19T05:00:08.770 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:08 smithi043 ceph-mon[30800]: from='client.14382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:00:08.770 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:08 smithi043 ceph-mon[30800]: osdmap e58: 9 total, 9 up, 9 in 2024-09-19T05:00:09.481 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:09 smithi043 ceph-mon[30800]: osdmap e59: 9 total, 9 up, 9 in 2024-09-19T05:00:09.481 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:09 smithi043 ceph-mon[30800]: pgmap v200: 1 pgs: 1 peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:00:09.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:09 smithi151 ceph-mon[35425]: osdmap e59: 9 total, 9 up, 9 in 2024-09-19T05:00:09.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:09 smithi151 ceph-mon[35425]: pgmap v200: 1 pgs: 1 peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:00:09.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:09 smithi096 ceph-mon[35085]: osdmap e59: 9 total, 9 up, 9 in 2024-09-19T05:00:09.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:09 smithi096 ceph-mon[35085]: pgmap v200: 1 pgs: 1 peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:00:10.701 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:10 smithi043 ceph-mon[30800]: osdmap e60: 9 total, 9 up, 9 in 2024-09-19T05:00:10.701 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:10 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/3530505607' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c19f65c-9750-4baa-9d63-7b8bd03e6955"}]: dispatch 2024-09-19T05:00:10.701 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:10 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/3530505607' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c19f65c-9750-4baa-9d63-7b8bd03e6955"}]': finished 2024-09-19T05:00:10.702 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:10 smithi043 ceph-mon[30800]: osdmap e61: 10 total, 9 up, 10 in 2024-09-19T05:00:10.702 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:10 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:00:10.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:10 smithi096 ceph-mon[35085]: osdmap e60: 9 total, 9 up, 9 in 2024-09-19T05:00:10.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:10 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/3530505607' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c19f65c-9750-4baa-9d63-7b8bd03e6955"}]: dispatch 2024-09-19T05:00:10.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:10 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/3530505607' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c19f65c-9750-4baa-9d63-7b8bd03e6955"}]': finished 2024-09-19T05:00:10.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:10 smithi096 ceph-mon[35085]: osdmap e61: 10 total, 9 up, 10 in 2024-09-19T05:00:10.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:10 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:00:10.904 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:10 smithi151 ceph-mon[35425]: osdmap e60: 9 total, 9 up, 9 in 2024-09-19T05:00:10.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:10 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/3530505607' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c19f65c-9750-4baa-9d63-7b8bd03e6955"}]: dispatch 2024-09-19T05:00:10.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:10 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/3530505607' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c19f65c-9750-4baa-9d63-7b8bd03e6955"}]': finished 2024-09-19T05:00:10.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:10 smithi151 ceph-mon[35425]: osdmap e61: 10 total, 9 up, 10 in 2024-09-19T05:00:10.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:10 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:00:11.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:11 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/923847390' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:00:11.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:11 smithi096 ceph-mon[35085]: pgmap v203: 1 pgs: 1 peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T05:00:11.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:11 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/923847390' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:00:11.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:11 smithi043 ceph-mon[30800]: pgmap v203: 1 pgs: 1 peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T05:00:11.904 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:11 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/923847390' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:00:11.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:11 smithi151 ceph-mon[35425]: pgmap v203: 1 pgs: 1 peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T05:00:13.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:13 smithi096 ceph-mon[35085]: pgmap v204: 1 pgs: 1 peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-19T05:00:13.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:13 smithi043 ceph-mon[30800]: pgmap v204: 1 pgs: 1 peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-19T05:00:13.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:13 smithi151 ceph-mon[35425]: pgmap v204: 1 pgs: 1 peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-19T05:00:15.424 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:15 smithi043 ceph-mon[30800]: pgmap v205: 1 pgs: 1 peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-19T05:00:15.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:15 smithi096 ceph-mon[35085]: pgmap v205: 1 pgs: 1 peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-19T05:00:15.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:15 smithi151 ceph-mon[35425]: pgmap v205: 1 pgs: 1 peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-19T05:00:17.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:17 smithi043 ceph-mon[30800]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T05:00:17.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:17 smithi096 ceph-mon[35085]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T05:00:17.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:17 smithi151 ceph-mon[35425]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T05:00:19.457 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:19 smithi151 ceph-mon[35425]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-19T05:00:19.472 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:19 smithi096 ceph-mon[35085]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-19T05:00:19.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:19 smithi043 ceph-mon[30800]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-19T05:00:20.422 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:20 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T05:00:20.422 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:20 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:20.422 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:20 smithi043 ceph-mon[30800]: Deploying daemon osd.9 on smithi043 2024-09-19T05:00:20.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:20 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T05:00:20.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:20 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:20.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:20 smithi096 ceph-mon[35085]: Deploying daemon osd.9 on smithi043 2024-09-19T05:00:20.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:20 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T05:00:20.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:20 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:20.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:20 smithi151 ceph-mon[35425]: Deploying daemon osd.9 on smithi043 2024-09-19T05:00:21.448 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:21 smithi043 ceph-mon[30800]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:00:21.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:21 smithi096 ceph-mon[35085]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:00:21.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:21 smithi151 ceph-mon[35425]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:00:23.259 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:23 smithi043 ceph-mon[30800]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:00:23.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:23 smithi096 ceph-mon[35085]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:00:23.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:23 smithi151 ceph-mon[35425]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:00:25.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:25 smithi096 ceph-mon[35085]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:00:25.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:25 smithi043 ceph-mon[30800]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:00:25.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:25 smithi151 ceph-mon[35425]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:00:27.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:27 smithi043 ceph-mon[30800]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:00:27.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:27 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:00:27.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:27 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:27.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:27 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:27.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:27 smithi096 ceph-mon[35085]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:00:27.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:27 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:00:27.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:27 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:27.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:27 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:27.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:27 smithi151 ceph-mon[35425]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:00:27.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:27 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:00:27.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:27 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:27.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:27 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:29.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:29 smithi096 ceph-mon[35085]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:00:29.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:29 smithi043 ceph-mon[30800]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:00:29.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:29 smithi151 ceph-mon[35425]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:00:31.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:31.467+0000 7f0b36000640 1 -- 172.21.15.43:0/1401484676 <== mgr.14150 v2:172.21.15.43:6800/3167108925 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0b38062630 con 0x7f0b1c0a3270 2024-09-19T05:00:31.469 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 9 on host 'smithi043' 2024-09-19T05:00:31.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:31.470+0000 7f0b3f800640 1 -- 172.21.15.43:0/1401484676 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f0b1c0a3270 msgr2=0x7f0b1c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:00:31.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:31.470+0000 7f0b3f800640 1 --2- 172.21.15.43:0/1401484676 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f0b1c0a3270 0x7f0b1c0a5730 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0b3819b620 tx=0x7f0b20005f90 comp rx=0 tx=0).stop 2024-09-19T05:00:31.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:31.470+0000 7f0b3f800640 1 -- 172.21.15.43:0/1401484676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b38103e00 msgr2=0x7f0b3819a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:00:31.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:31.470+0000 7f0b3f800640 1 --2- 172.21.15.43:0/1401484676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b38103e00 0x7f0b3819a100 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0b28009230 tx=0x7f0b280042f0 comp rx=0 tx=0).stop 2024-09-19T05:00:31.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:31.470+0000 7f0b3f800640 1 -- 172.21.15.43:0/1401484676 shutdown_connections 2024-09-19T05:00:31.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:31.470+0000 7f0b3f800640 1 --2- 172.21.15.43:0/1401484676 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f0b1c0a3270 0x7f0b1c0a5730 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:00:31.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:31.470+0000 7f0b3f800640 1 --2- 172.21.15.43:0/1401484676 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b38105920 0x7f0b381a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:00:31.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:31.470+0000 7f0b3f800640 1 --2- 172.21.15.43:0/1401484676 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b38104f60 0x7f0b3819a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:00:31.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:31.470+0000 7f0b3f800640 1 --2- 172.21.15.43:0/1401484676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b38103e00 0x7f0b3819a100 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:00:31.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:31.470+0000 7f0b3f800640 1 -- 172.21.15.43:0/1401484676 >> 172.21.15.43:0/1401484676 conn(0x7f0b380ff5b0 msgr2=0x7f0b38108370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:00:31.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:31.470+0000 7f0b3f800640 1 -- 172.21.15.43:0/1401484676 shutdown_connections 2024-09-19T05:00:31.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:00:31.471+0000 7f0b3f800640 1 -- 172.21.15.43:0/1401484676 wait complete. 2024-09-19T05:00:31.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:31 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:31.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:31 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:31.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:31 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:31.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:31 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:00:31.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:31 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:31.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:31 smithi043 ceph-mon[30800]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:00:31.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:31 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:00:31.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:31 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:31.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:31 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:31.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:31 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:31.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:31 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:31.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:31 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:31.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:31 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:00:31.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:31 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:31.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:31 smithi151 ceph-mon[35425]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:00:31.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:31 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:00:31.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:31 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:31.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:31 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:31.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:31 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:31.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:31 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:31.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:31 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:31.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:31 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:00:31.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:31 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:31.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:31 smithi096 ceph-mon[35085]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:00:31.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:31 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:00:31.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:31 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:31.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:31 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:32.262 DEBUG:teuthology.orchestra.run.smithi043:osd.9> sudo journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@osd.9.service 2024-09-19T05:00:32.266 INFO:tasks.cephadm:Deploying osd.10 on smithi096 with /dev/vg_nvme/lv_1... 2024-09-19T05:00:32.266 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T05:00:32.521 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T05:00:33.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:33 smithi096 ceph-mon[35085]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:00:33.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:33 smithi043 ceph-mon[30800]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:00:33.530 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:00:33 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:00:33.345+0000 7f5bc703b640 -1 osd.9 0 log_to_monitors true 2024-09-19T05:00:33.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:33 smithi151 ceph-mon[35425]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:00:34.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:34 smithi096 ceph-mon[35085]: from='osd.9 [v2:172.21.15.43:6826/4053438621,v1:172.21.15.43:6827/4053438621]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T05:00:34.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:34 smithi096 ceph-mon[35085]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T05:00:34.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:34 smithi043 ceph-mon[30800]: from='osd.9 [v2:172.21.15.43:6826/4053438621,v1:172.21.15.43:6827/4053438621]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T05:00:34.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:34 smithi043 ceph-mon[30800]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T05:00:34.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:34 smithi151 ceph-mon[35425]: from='osd.9 [v2:172.21.15.43:6826/4053438621,v1:172.21.15.43:6827/4053438621]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T05:00:34.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:34 smithi151 ceph-mon[35425]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T05:00:35.151 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-19T05:00:35.177 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi096:vg_nvme/lv_1 2024-09-19T05:00:35.426 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T05:00:35.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:35 smithi096 ceph-mon[35085]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T05:00:35.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:35 smithi096 ceph-mon[35085]: osdmap e62: 10 total, 9 up, 10 in 2024-09-19T05:00:35.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:35 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:00:35.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:35 smithi096 ceph-mon[35085]: from='osd.9 [v2:172.21.15.43:6826/4053438621,v1:172.21.15.43:6827/4053438621]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-19T05:00:35.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:35 smithi096 ceph-mon[35085]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-19T05:00:35.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:35 smithi096 ceph-mon[35085]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:00:35.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:35 smithi043 ceph-mon[30800]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T05:00:35.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:35 smithi043 ceph-mon[30800]: osdmap e62: 10 total, 9 up, 10 in 2024-09-19T05:00:35.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:35 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:00:35.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:35 smithi043 ceph-mon[30800]: from='osd.9 [v2:172.21.15.43:6826/4053438621,v1:172.21.15.43:6827/4053438621]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-19T05:00:35.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:35 smithi043 ceph-mon[30800]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-19T05:00:35.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:35 smithi043 ceph-mon[30800]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:00:35.531 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:00:35 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:00:35.203+0000 7f5bc1800640 -1 osd.9 0 waiting for initial osdmap 2024-09-19T05:00:35.531 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:00:35 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:00:35.223+0000 7f5bbc800640 -1 osd.9 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:00:35.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:35 smithi151 ceph-mon[35425]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T05:00:35.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:35 smithi151 ceph-mon[35425]: osdmap e62: 10 total, 9 up, 10 in 2024-09-19T05:00:35.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:35 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:00:35.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:35 smithi151 ceph-mon[35425]: from='osd.9 [v2:172.21.15.43:6826/4053438621,v1:172.21.15.43:6827/4053438621]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-19T05:00:35.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:35 smithi151 ceph-mon[35425]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-19T05:00:35.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:35 smithi151 ceph-mon[35425]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-19T05:00:35.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:35.917+0000 7fae1ca4b640 1 -- 172.21.15.96:0/3132646575 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae18102690 msgr2=0x7fae18104a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:00:35.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:35.917+0000 7fae1ca4b640 1 --2- 172.21.15.96:0/3132646575 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae18102690 0x7fae18104a80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fae08009900 tx=0x7fae0802a410 comp rx=0 tx=0).stop 2024-09-19T05:00:35.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:35.917+0000 7fae1ca4b640 1 -- 172.21.15.96:0/3132646575 shutdown_connections 2024-09-19T05:00:35.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:35.917+0000 7fae1ca4b640 1 --2- 172.21.15.96:0/3132646575 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae181078f0 0x7fae18109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:00:35.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:35.917+0000 7fae1ca4b640 1 --2- 172.21.15.96:0/3132646575 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae18104fc0 0x7fae181073b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:00:35.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:35.917+0000 7fae1ca4b640 1 --2- 172.21.15.96:0/3132646575 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae18102690 0x7fae18104a80 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:00:35.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:35.917+0000 7fae1ca4b640 1 -- 172.21.15.96:0/3132646575 >> 172.21.15.96:0/3132646575 conn(0x7fae180fc290 msgr2=0x7fae180fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:00:35.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:35.917+0000 7fae1ca4b640 1 -- 172.21.15.96:0/3132646575 shutdown_connections 2024-09-19T05:00:35.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:35.917+0000 7fae1ca4b640 1 -- 172.21.15.96:0/3132646575 wait complete. 2024-09-19T05:00:35.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:35.918+0000 7fae1ca4b640 1 Processor -- start 2024-09-19T05:00:35.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:35.918+0000 7fae1ca4b640 1 -- start start 2024-09-19T05:00:35.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:35.918+0000 7fae1ca4b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae18102690 0x7fae18101ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:00:35.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:35.918+0000 7fae1ca4b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae18104fc0 0x7fae18100640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:00:35.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:35.918+0000 7fae1ca4b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae181078f0 0x7fae18100b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:00:35.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:35.918+0000 7fae1ca4b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fae1810d8c0 con 0x7fae18104fc0 2024-09-19T05:00:35.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:35.918+0000 7fae1ca4b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fae1810d740 con 0x7fae18102690 2024-09-19T05:00:35.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:35.918+0000 7fae1ca4b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fae1810da40 con 0x7fae181078f0 2024-09-19T05:00:35.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:35.918+0000 7fae1659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae18102690 0x7fae18101ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:00:35.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:35.918+0000 7fae1659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae18102690 0x7fae18101ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59572/0 (socket says 172.21.15.96:59572) 2024-09-19T05:00:35.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:35.918+0000 7fae1659c640 1 -- 172.21.15.96:0/762013853 learned_addr learned my addr 172.21.15.96:0/762013853 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-19T05:00:35.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:35.919+0000 7fae15d9b640 1 --2- 172.21.15.96:0/762013853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae18104fc0 0x7fae18100640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:00:35.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:35.919+0000 7fae16d9d640 1 --2- 172.21.15.96:0/762013853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae181078f0 0x7fae18100b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:00:35.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:35.919+0000 7fae1659c640 1 -- 172.21.15.96:0/762013853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae181078f0 msgr2=0x7fae18100b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:00:35.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:35.919+0000 7fae1659c640 1 --2- 172.21.15.96:0/762013853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae181078f0 0x7fae18100b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:00:35.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:35.919+0000 7fae1659c640 1 -- 172.21.15.96:0/762013853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae18104fc0 msgr2=0x7fae18100640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:00:35.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:35.919+0000 7fae1659c640 1 --2- 172.21.15.96:0/762013853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae18104fc0 0x7fae18100640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:00:35.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:35.919+0000 7fae1659c640 1 -- 172.21.15.96:0/762013853 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae18101440 con 0x7fae18102690 2024-09-19T05:00:35.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:35.919+0000 7fae1659c640 1 --2- 172.21.15.96:0/762013853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae18102690 0x7fae18101ff0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fae08009230 tx=0x7fae08004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:00:35.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:35.919+0000 7fae077fe640 1 -- 172.21.15.96:0/762013853 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae0803f030 con 0x7fae18102690 2024-09-19T05:00:35.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:35.919+0000 7fae1ca4b640 1 -- 172.21.15.96:0/762013853 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae181a2070 con 0x7fae18102690 2024-09-19T05:00:35.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:35.919+0000 7fae077fe640 1 -- 172.21.15.96:0/762013853 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae08003ce0 con 0x7fae18102690 2024-09-19T05:00:35.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:35.919+0000 7fae077fe640 1 -- 172.21.15.96:0/762013853 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae08035070 con 0x7fae18102690 2024-09-19T05:00:35.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:35.919+0000 7fae1ca4b640 1 -- 172.21.15.96:0/762013853 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae181a2520 con 0x7fae18102690 2024-09-19T05:00:35.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:35.920+0000 7fae1ca4b640 1 -- 172.21.15.96:0/762013853 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae180fdd80 con 0x7fae18102690 2024-09-19T05:00:35.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:35.921+0000 7fae077fe640 1 -- 172.21.15.96:0/762013853 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fae08042030 con 0x7fae18102690 2024-09-19T05:00:35.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:35.922+0000 7fae077fe640 1 --2- 172.21.15.96:0/762013853 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fae000a3240 0x7fae000a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:00:35.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:35.922+0000 7fae077fe640 1 -- 172.21.15.96:0/762013853 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5981+0+0 (secure 0 0 0) 0x7fae080efa90 con 0x7fae18102690 2024-09-19T05:00:35.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:35.922+0000 7fae15d9b640 1 --2- 172.21.15.96:0/762013853 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fae000a3240 0x7fae000a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:00:35.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:35.923+0000 7fae15d9b640 1 --2- 172.21.15.96:0/762013853 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fae000a3240 0x7fae000a5700 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fae0c005fd0 tx=0x7fae0c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:00:35.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:35.926+0000 7fae077fe640 1 -- 172.21.15.96:0/762013853 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fae080bd0d0 con 0x7fae18102690 2024-09-19T05:00:36.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:36.103+0000 7fae1ca4b640 1 -- 172.21.15.96:0/762013853 --> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fae18062630 con 0x7fae000a3240 2024-09-19T05:00:36.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:36 smithi043 ceph-mon[30800]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-19T05:00:36.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:36 smithi043 ceph-mon[30800]: osdmap e63: 10 total, 9 up, 10 in 2024-09-19T05:00:36.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:36 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:00:36.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:36 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:00:36.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:36 smithi043 ceph-mon[30800]: Detected new or changed devices on smithi043 2024-09-19T05:00:36.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:36 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:36.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:36 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:36.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:36 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:00:36.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:36 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:00:36.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:36 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:00:36.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:36 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:00:36.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:36 smithi043 ceph-mon[30800]: Adjusting osd_memory_target on smithi043 to 4273M 2024-09-19T05:00:36.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:36 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:36.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:36 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:36.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:36 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:00:36.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:36 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:36.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:36 smithi043 ceph-mon[30800]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:00:36.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:36 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:00:36.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:36 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:00:36.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:36 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:36.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:36 smithi096 ceph-mon[35085]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-19T05:00:36.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:36 smithi096 ceph-mon[35085]: osdmap e63: 10 total, 9 up, 10 in 2024-09-19T05:00:36.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:36 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:00:36.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:36 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:00:36.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:36 smithi096 ceph-mon[35085]: Detected new or changed devices on smithi043 2024-09-19T05:00:36.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:36 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:36.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:36 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:36.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:36 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:00:36.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:36 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:00:36.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:36 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:00:36.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:36 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:00:36.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:36 smithi096 ceph-mon[35085]: Adjusting osd_memory_target on smithi043 to 4273M 2024-09-19T05:00:36.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:36 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:36.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:36 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:36.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:36 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:00:36.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:36 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:36.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:36 smithi096 ceph-mon[35085]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:00:36.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:36 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:00:36.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:36 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:00:36.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:36 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:36.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:36 smithi151 ceph-mon[35425]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-19T05:00:36.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:36 smithi151 ceph-mon[35425]: osdmap e63: 10 total, 9 up, 10 in 2024-09-19T05:00:36.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:36 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:00:36.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:36 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:00:36.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:36 smithi151 ceph-mon[35425]: Detected new or changed devices on smithi043 2024-09-19T05:00:36.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:36 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:36.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:36 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:36.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:36 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:00:36.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:36 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:00:36.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:36 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:00:36.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:36 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:00:36.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:36 smithi151 ceph-mon[35425]: Adjusting osd_memory_target on smithi043 to 4273M 2024-09-19T05:00:36.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:36 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:36.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:36 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:36.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:36 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:00:36.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:36 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:36.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:36 smithi151 ceph-mon[35425]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:00:36.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:36 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:00:36.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:36 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:00:36.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:36 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:37.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:37 smithi096 ceph-mon[35085]: purged_snaps scrub starts 2024-09-19T05:00:37.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:37 smithi096 ceph-mon[35085]: purged_snaps scrub ok 2024-09-19T05:00:37.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:37 smithi096 ceph-mon[35085]: osd.9 [v2:172.21.15.43:6826/4053438621,v1:172.21.15.43:6827/4053438621] boot 2024-09-19T05:00:37.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:37 smithi096 ceph-mon[35085]: osdmap e64: 10 total, 10 up, 10 in 2024-09-19T05:00:37.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:37 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:00:37.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:37 smithi096 ceph-mon[35085]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:00:37.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:37 smithi043 ceph-mon[30800]: purged_snaps scrub starts 2024-09-19T05:00:37.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:37 smithi043 ceph-mon[30800]: purged_snaps scrub ok 2024-09-19T05:00:37.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:37 smithi043 ceph-mon[30800]: osd.9 [v2:172.21.15.43:6826/4053438621,v1:172.21.15.43:6827/4053438621] boot 2024-09-19T05:00:37.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:37 smithi043 ceph-mon[30800]: osdmap e64: 10 total, 10 up, 10 in 2024-09-19T05:00:37.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:37 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:00:37.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:37 smithi043 ceph-mon[30800]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:00:37.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:37 smithi151 ceph-mon[35425]: purged_snaps scrub starts 2024-09-19T05:00:37.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:37 smithi151 ceph-mon[35425]: purged_snaps scrub ok 2024-09-19T05:00:37.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:37 smithi151 ceph-mon[35425]: osd.9 [v2:172.21.15.43:6826/4053438621,v1:172.21.15.43:6827/4053438621] boot 2024-09-19T05:00:37.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:37 smithi151 ceph-mon[35425]: osdmap e64: 10 total, 10 up, 10 in 2024-09-19T05:00:37.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:37 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:00:37.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:37 smithi151 ceph-mon[35425]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:00:38.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:38 smithi096 ceph-mon[35085]: osdmap e65: 10 total, 10 up, 10 in 2024-09-19T05:00:38.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:38 smithi043 ceph-mon[30800]: osdmap e65: 10 total, 10 up, 10 in 2024-09-19T05:00:38.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:38 smithi151 ceph-mon[35425]: osdmap e65: 10 total, 10 up, 10 in 2024-09-19T05:00:39.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:39 smithi096 ceph-mon[35085]: from='client.? 172.21.15.96:0/3668612292' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7aa7da3-8eab-47fe-a351-298b6ff75f14"}]: dispatch 2024-09-19T05:00:39.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:39 smithi096 ceph-mon[35085]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7aa7da3-8eab-47fe-a351-298b6ff75f14"}]: dispatch 2024-09-19T05:00:39.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:39 smithi096 ceph-mon[35085]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7aa7da3-8eab-47fe-a351-298b6ff75f14"}]': finished 2024-09-19T05:00:39.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:39 smithi096 ceph-mon[35085]: osdmap e66: 11 total, 10 up, 11 in 2024-09-19T05:00:39.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:39 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:00:39.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:39 smithi096 ceph-mon[35085]: from='client.? 172.21.15.96:0/3661599393' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:00:39.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:39 smithi096 ceph-mon[35085]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:00:39.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:39 smithi043 ceph-mon[30800]: from='client.? 172.21.15.96:0/3668612292' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7aa7da3-8eab-47fe-a351-298b6ff75f14"}]: dispatch 2024-09-19T05:00:39.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:39 smithi043 ceph-mon[30800]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7aa7da3-8eab-47fe-a351-298b6ff75f14"}]: dispatch 2024-09-19T05:00:39.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:39 smithi043 ceph-mon[30800]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7aa7da3-8eab-47fe-a351-298b6ff75f14"}]': finished 2024-09-19T05:00:39.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:39 smithi043 ceph-mon[30800]: osdmap e66: 11 total, 10 up, 11 in 2024-09-19T05:00:39.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:39 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:00:39.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:39 smithi043 ceph-mon[30800]: from='client.? 172.21.15.96:0/3661599393' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:00:39.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:39 smithi043 ceph-mon[30800]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:00:39.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:39 smithi151 ceph-mon[35425]: from='client.? 172.21.15.96:0/3668612292' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7aa7da3-8eab-47fe-a351-298b6ff75f14"}]: dispatch 2024-09-19T05:00:39.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:39 smithi151 ceph-mon[35425]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7aa7da3-8eab-47fe-a351-298b6ff75f14"}]: dispatch 2024-09-19T05:00:39.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:39 smithi151 ceph-mon[35425]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7aa7da3-8eab-47fe-a351-298b6ff75f14"}]': finished 2024-09-19T05:00:39.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:39 smithi151 ceph-mon[35425]: osdmap e66: 11 total, 10 up, 11 in 2024-09-19T05:00:39.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:39 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:00:39.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:39 smithi151 ceph-mon[35425]: from='client.? 172.21.15.96:0/3661599393' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:00:39.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:39 smithi151 ceph-mon[35425]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:00:41.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:41 smithi096 ceph-mon[35085]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:00:41.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:41 smithi043 ceph-mon[30800]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:00:41.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:41 smithi151 ceph-mon[35425]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:00:43.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:43 smithi096 ceph-mon[35085]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:00:43.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:43 smithi043 ceph-mon[30800]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:00:43.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:43 smithi151 ceph-mon[35425]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:00:45.302 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:45 smithi096 ceph-mon[35085]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:00:45.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:45 smithi043 ceph-mon[30800]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:00:45.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:45 smithi151 ceph-mon[35425]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:00:46.722 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:46 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T05:00:46.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:46 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:46.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:46 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T05:00:46.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:46 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:46.904 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:46 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T05:00:46.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:46 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:47.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:47 smithi043 ceph-mon[30800]: Deploying daemon osd.10 on smithi096 2024-09-19T05:00:47.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:47 smithi043 ceph-mon[30800]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:00:47.842 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:47 smithi096 ceph-mon[35085]: Deploying daemon osd.10 on smithi096 2024-09-19T05:00:47.842 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:47 smithi096 ceph-mon[35085]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:00:47.904 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:47 smithi151 ceph-mon[35425]: Deploying daemon osd.10 on smithi096 2024-09-19T05:00:47.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:47 smithi151 ceph-mon[35425]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:00:49.383 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:49 smithi096 ceph-mon[35085]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:00:49.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:49 smithi043 ceph-mon[30800]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:00:49.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:49 smithi151 ceph-mon[35425]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:00:51.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:51 smithi096 ceph-mon[35085]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:00:51.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:51 smithi043 ceph-mon[30800]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:00:51.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:51 smithi151 ceph-mon[35425]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:00:52.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:52 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:00:52.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:52 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:52.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:52 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:53.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:52 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:00:53.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:52 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:53.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:52 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:53.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:52 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:00:53.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:52 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:53.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:52 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:53.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:53 smithi096 ceph-mon[35085]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:00:54.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:53 smithi043 ceph-mon[30800]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:00:54.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:53 smithi151 ceph-mon[35425]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:00:55.422 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:55 smithi096 ceph-mon[35085]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:00:55.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:55 smithi043 ceph-mon[30800]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:00:55.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:55 smithi151 ceph-mon[35425]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:00:56.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:56.056+0000 7fae077fe640 1 -- 172.21.15.96:0/762013853 <== mgr.14150 v2:172.21.15.43:6800/3167108925 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fae18062630 con 0x7fae000a3240 2024-09-19T05:00:56.058 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 10 on host 'smithi096' 2024-09-19T05:00:56.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:56.058+0000 7fae1ca4b640 1 -- 172.21.15.96:0/762013853 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fae000a3240 msgr2=0x7fae000a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:00:56.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:56.058+0000 7fae1ca4b640 1 --2- 172.21.15.96:0/762013853 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fae000a3240 0x7fae000a5700 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fae0c005fd0 tx=0x7fae0c005850 comp rx=0 tx=0).stop 2024-09-19T05:00:56.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:56.058+0000 7fae1ca4b640 1 -- 172.21.15.96:0/762013853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae18102690 msgr2=0x7fae18101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:00:56.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:56.058+0000 7fae1ca4b640 1 --2- 172.21.15.96:0/762013853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae18102690 0x7fae18101ff0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fae08009230 tx=0x7fae08004360 comp rx=0 tx=0).stop 2024-09-19T05:00:56.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:56.059+0000 7fae1ca4b640 1 -- 172.21.15.96:0/762013853 shutdown_connections 2024-09-19T05:00:56.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:56.059+0000 7fae1ca4b640 1 --2- 172.21.15.96:0/762013853 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fae000a3240 0x7fae000a5700 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:00:56.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:56.059+0000 7fae1ca4b640 1 --2- 172.21.15.96:0/762013853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae181078f0 0x7fae18100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:00:56.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:56.059+0000 7fae1ca4b640 1 --2- 172.21.15.96:0/762013853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae18104fc0 0x7fae18100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:00:56.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:56.059+0000 7fae1ca4b640 1 --2- 172.21.15.96:0/762013853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae18102690 0x7fae18101ff0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:00:56.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:56.059+0000 7fae1ca4b640 1 -- 172.21.15.96:0/762013853 >> 172.21.15.96:0/762013853 conn(0x7fae180fc290 msgr2=0x7fae181030f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:00:56.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:56.059+0000 7fae1ca4b640 1 -- 172.21.15.96:0/762013853 shutdown_connections 2024-09-19T05:00:56.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T05:00:56.059+0000 7fae1ca4b640 1 -- 172.21.15.96:0/762013853 wait complete. 2024-09-19T05:00:56.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:56 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:56.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:56 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:56.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:56 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:56.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:56 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:00:56.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:56 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:56.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:56 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:00:56.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:56 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:56.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:56 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:56.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:56 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:56.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:56 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:56.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:56 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:56.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:56 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:00:56.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:56 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:56.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:56 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:00:56.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:56 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:56.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:56 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:56.792 DEBUG:teuthology.orchestra.run.smithi096:osd.10> sudo journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@osd.10.service 2024-09-19T05:00:56.795 INFO:tasks.cephadm:Deploying osd.11 on smithi151 with /dev/vg_nvme/lv_1... 2024-09-19T05:00:56.795 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T05:00:56.823 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:56 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:56.823 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:56 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:56.823 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:56 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:00:56.823 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:56 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:00:56.824 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:56 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:56.824 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:56 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:00:56.824 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:56 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:56.824 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:56 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:00:57.049 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:00:57.684 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:57 smithi043 ceph-mon[30800]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:00:57.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:57 smithi096 ceph-mon[35085]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:00:57.904 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:57 smithi151 ceph-mon[35425]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:00:58.464 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:58 smithi151 ceph-mon[35425]: from='osd.10 [v2:172.21.15.96:6824/2403210086,v1:172.21.15.96:6825/2403210086]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T05:00:58.464 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:58 smithi151 ceph-mon[35425]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T05:00:58.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:58 smithi096 ceph-mon[35085]: from='osd.10 [v2:172.21.15.96:6824/2403210086,v1:172.21.15.96:6825/2403210086]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T05:00:58.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:58 smithi096 ceph-mon[35085]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T05:00:58.723 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:00:58 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:00:58.411+0000 7faac2d40640 -1 osd.10 0 log_to_monitors true 2024-09-19T05:00:58.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:58 smithi043 ceph-mon[30800]: from='osd.10 [v2:172.21.15.96:6824/2403210086,v1:172.21.15.96:6825/2403210086]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T05:00:58.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:58 smithi043 ceph-mon[30800]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T05:00:59.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:59 smithi043 ceph-mon[30800]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T05:00:59.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:59 smithi043 ceph-mon[30800]: osdmap e67: 11 total, 10 up, 11 in 2024-09-19T05:00:59.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:59 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:00:59.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:59 smithi043 ceph-mon[30800]: from='osd.10 [v2:172.21.15.96:6824/2403210086,v1:172.21.15.96:6825/2403210086]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T05:00:59.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:59 smithi043 ceph-mon[30800]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T05:00:59.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:00:59 smithi043 ceph-mon[30800]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:00:59.671 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:59 smithi096 ceph-mon[35085]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T05:00:59.672 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:59 smithi096 ceph-mon[35085]: osdmap e67: 11 total, 10 up, 11 in 2024-09-19T05:00:59.672 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:59 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:00:59.672 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:59 smithi096 ceph-mon[35085]: from='osd.10 [v2:172.21.15.96:6824/2403210086,v1:172.21.15.96:6825/2403210086]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T05:00:59.672 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:59 smithi096 ceph-mon[35085]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T05:00:59.672 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:00:59 smithi096 ceph-mon[35085]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:00:59.780 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:59 smithi151 ceph-mon[35425]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T05:00:59.781 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:59 smithi151 ceph-mon[35425]: osdmap e67: 11 total, 10 up, 11 in 2024-09-19T05:00:59.781 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:59 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:00:59.781 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:59 smithi151 ceph-mon[35425]: from='osd.10 [v2:172.21.15.96:6824/2403210086,v1:172.21.15.96:6825/2403210086]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T05:00:59.781 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:59 smithi151 ceph-mon[35425]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T05:00:59.781 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:00:59 smithi151 ceph-mon[35425]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T05:00:59.781 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T05:00:59.806 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi151:vg_nvme/lv_1 2024-09-19T05:00:59.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:00:59 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:00:59.828+0000 7faabecc3640 -1 osd.10 0 waiting for initial osdmap 2024-09-19T05:00:59.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:00:59 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:00:59.842+0000 7faab6ae3640 -1 osd.10 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:01:00.091 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:01:00.498 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:00 smithi151 ceph-mon[35425]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-19T05:01:00.499 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:00 smithi151 ceph-mon[35425]: osdmap e68: 11 total, 10 up, 11 in 2024-09-19T05:01:00.499 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:00 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:01:00.499 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:00 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:01:00.499 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:00 smithi151 ceph-mon[35425]: Detected new or changed devices on smithi096 2024-09-19T05:01:00.499 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:00 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:00.499 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:00 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:00.499 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:00 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:01:00.499 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:00 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:01:00.499 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:00 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:01:00.499 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:00 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:01:00.499 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:00 smithi151 ceph-mon[35425]: Adjusting osd_memory_target on smithi096 to 4272M 2024-09-19T05:01:00.499 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:00 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:00.499 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:00 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:00.500 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:00 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:01:00.500 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:00 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:00.500 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:00 smithi151 ceph-mon[35425]: from='osd.10 ' entity='osd.10' 2024-09-19T05:01:00.500 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:00 smithi151 ceph-mon[35425]: osd.10 [v2:172.21.15.96:6824/2403210086,v1:172.21.15.96:6825/2403210086] boot 2024-09-19T05:01:00.500 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:00 smithi151 ceph-mon[35425]: osdmap e69: 11 total, 11 up, 11 in 2024-09-19T05:01:00.500 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:00 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:01:00.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:00.580+0000 7f9bbe4fe640 1 -- 172.21.15.151:0/2423350077 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bb81037c0 msgr2=0x7f9bb810a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:00.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:00.580+0000 7f9bbe4fe640 1 --2- 172.21.15.151:0/2423350077 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bb81037c0 0x7f9bb810a050 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0009ca0 tx=0x7f9bb002a8c0 comp rx=0 tx=0).stop 2024-09-19T05:01:00.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:00.580+0000 7f9bbe4fe640 1 -- 172.21.15.151:0/2423350077 shutdown_connections 2024-09-19T05:01:00.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:00.580+0000 7f9bbe4fe640 1 --2- 172.21.15.151:0/2423350077 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bb81037c0 0x7f9bb810a050 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:00.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:00.580+0000 7f9bbe4fe640 1 --2- 172.21.15.151:0/2423350077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bb8102e00 0x7f9bb8103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:00.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:00.580+0000 7f9bbe4fe640 1 --2- 172.21.15.151:0/2423350077 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bb8101c00 0x7f9bb8102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:00.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:00.580+0000 7f9bbe4fe640 1 -- 172.21.15.151:0/2423350077 >> 172.21.15.151:0/2423350077 conn(0x7f9bb80fd390 msgr2=0x7f9bb80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:01:00.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:00.580+0000 7f9bbe4fe640 1 -- 172.21.15.151:0/2423350077 shutdown_connections 2024-09-19T05:01:00.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:00.580+0000 7f9bbe4fe640 1 -- 172.21.15.151:0/2423350077 wait complete. 2024-09-19T05:01:00.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:00.581+0000 7f9bbe4fe640 1 Processor -- start 2024-09-19T05:01:00.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:00.581+0000 7f9bbe4fe640 1 -- start start 2024-09-19T05:01:00.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:00.581+0000 7f9bbe4fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bb8101c00 0x7f9bb8197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:00.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:00.581+0000 7f9bbe4fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bb8102e00 0x7f9bb8198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:00.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:00.581+0000 7f9bbe4fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bb81037c0 0x7f9bb819f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:00.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:00.581+0000 7f9bbe4fe640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb810d860 con 0x7f9bb8101c00 2024-09-19T05:01:00.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:00.581+0000 7f9bbe4fe640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb810d6e0 con 0x7f9bb8102e00 2024-09-19T05:01:00.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:00.581+0000 7f9bbe4fe640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb810d9e0 con 0x7f9bb81037c0 2024-09-19T05:01:00.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:00.581+0000 7f9bbca9b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bb81037c0 0x7f9bb819f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:00.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:00.581+0000 7f9bbca9b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bb81037c0 0x7f9bb819f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45734/0 (socket says 172.21.15.151:45734) 2024-09-19T05:01:00.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:00.581+0000 7f9bbca9b640 1 -- 172.21.15.151:0/406543680 learned_addr learned my addr 172.21.15.151:0/406543680 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-19T05:01:00.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:00.582+0000 7f9bb77fe640 1 --2- 172.21.15.151:0/406543680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bb8102e00 0x7f9bb8198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:00.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:00.582+0000 7f9bb7fff640 1 --2- 172.21.15.151:0/406543680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bb8101c00 0x7f9bb8197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:00.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:00.582+0000 7f9bbca9b640 1 -- 172.21.15.151:0/406543680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bb8102e00 msgr2=0x7f9bb8198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:00.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:00.582+0000 7f9bbca9b640 1 --2- 172.21.15.151:0/406543680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bb8102e00 0x7f9bb8198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:00.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:00.582+0000 7f9bbca9b640 1 -- 172.21.15.151:0/406543680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bb8101c00 msgr2=0x7f9bb8197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:00.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:00.582+0000 7f9bbca9b640 1 --2- 172.21.15.151:0/406543680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bb8101c00 0x7f9bb8197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:00.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:00.582+0000 7f9bbca9b640 1 -- 172.21.15.151:0/406543680 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bb819fa30 con 0x7f9bb81037c0 2024-09-19T05:01:00.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:00.582+0000 7f9bbca9b640 1 --2- 172.21.15.151:0/406543680 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bb81037c0 0x7f9bb819f3c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0005e30 tx=0x7f9bb0031ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:01:00.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:00.582+0000 7f9bb57fa640 1 -- 172.21.15.151:0/406543680 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb0003c30 con 0x7f9bb81037c0 2024-09-19T05:01:00.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:00.582+0000 7f9bb57fa640 1 -- 172.21.15.151:0/406543680 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bb0003dd0 con 0x7f9bb81037c0 2024-09-19T05:01:00.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:00.582+0000 7f9bbe4fe640 1 -- 172.21.15.151:0/406543680 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bb819fcc0 con 0x7f9bb81037c0 2024-09-19T05:01:00.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:00.582+0000 7f9bb57fa640 1 -- 172.21.15.151:0/406543680 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb002fde0 con 0x7f9bb81037c0 2024-09-19T05:01:00.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:00.582+0000 7f9bbe4fe640 1 -- 172.21.15.151:0/406543680 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bb81a01b0 con 0x7f9bb81037c0 2024-09-19T05:01:00.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:00.583+0000 7f9bbe4fe640 1 -- 172.21.15.151:0/406543680 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bb8108f30 con 0x7f9bb81037c0 2024-09-19T05:01:00.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:00.584+0000 7f9bb57fa640 1 -- 172.21.15.151:0/406543680 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f9bb0004050 con 0x7f9bb81037c0 2024-09-19T05:01:00.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:00.585+0000 7f9bb57fa640 1 --2- 172.21.15.151:0/406543680 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f9ba00a3270 0x7f9ba00a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:00.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:00.585+0000 7f9bb57fa640 1 -- 172.21.15.151:0/406543680 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6727+0+0 (secure 0 0 0) 0x7f9bb00f0510 con 0x7f9bb81037c0 2024-09-19T05:01:00.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:00.585+0000 7f9bb7fff640 1 --2- 172.21.15.151:0/406543680 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f9ba00a3270 0x7f9ba00a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:00.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:00.586+0000 7f9bb7fff640 1 --2- 172.21.15.151:0/406543680 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f9ba00a3270 0x7f9ba00a5730 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9ba80058c0 tx=0x7f9ba8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:01:00.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:00.589+0000 7f9bb57fa640 1 -- 172.21.15.151:0/406543680 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9bb00bd860 con 0x7f9bb81037c0 2024-09-19T05:01:00.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:00 smithi096 ceph-mon[35085]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-19T05:01:00.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:00 smithi096 ceph-mon[35085]: osdmap e68: 11 total, 10 up, 11 in 2024-09-19T05:01:00.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:00 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:01:00.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:00 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:01:00.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:00 smithi096 ceph-mon[35085]: Detected new or changed devices on smithi096 2024-09-19T05:01:00.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:00 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:00.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:00 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:00.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:00 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:01:00.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:00 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:01:00.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:00 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:01:00.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:00 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:01:00.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:00 smithi096 ceph-mon[35085]: Adjusting osd_memory_target on smithi096 to 4272M 2024-09-19T05:01:00.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:00 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:00.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:00 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:00.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:00 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:01:00.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:00 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:00.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:00 smithi096 ceph-mon[35085]: from='osd.10 ' entity='osd.10' 2024-09-19T05:01:00.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:00 smithi096 ceph-mon[35085]: osd.10 [v2:172.21.15.96:6824/2403210086,v1:172.21.15.96:6825/2403210086] boot 2024-09-19T05:01:00.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:00 smithi096 ceph-mon[35085]: osdmap e69: 11 total, 11 up, 11 in 2024-09-19T05:01:00.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:00 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:01:00.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:00.765+0000 7f9bbe4fe640 1 -- 172.21.15.151:0/406543680 --> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f9bb8062630 con 0x7f9ba00a3270 2024-09-19T05:01:00.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:00 smithi043 ceph-mon[30800]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-19T05:01:00.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:00 smithi043 ceph-mon[30800]: osdmap e68: 11 total, 10 up, 11 in 2024-09-19T05:01:00.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:00 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:01:00.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:00 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:01:00.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:00 smithi043 ceph-mon[30800]: Detected new or changed devices on smithi096 2024-09-19T05:01:00.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:00 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:00.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:00 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:00.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:00 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:01:00.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:00 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:01:00.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:00 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:01:00.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:00 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:01:00.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:00 smithi043 ceph-mon[30800]: Adjusting osd_memory_target on smithi096 to 4272M 2024-09-19T05:01:00.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:00 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:00.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:00 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:00.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:00 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:01:00.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:00 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:00.782 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:00 smithi043 ceph-mon[30800]: from='osd.10 ' entity='osd.10' 2024-09-19T05:01:00.782 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:00 smithi043 ceph-mon[30800]: osd.10 [v2:172.21.15.96:6824/2403210086,v1:172.21.15.96:6825/2403210086] boot 2024-09-19T05:01:00.782 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:00 smithi043 ceph-mon[30800]: osdmap e69: 11 total, 11 up, 11 in 2024-09-19T05:01:00.782 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:00 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:01:01.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:01 smithi096 ceph-mon[35085]: purged_snaps scrub starts 2024-09-19T05:01:01.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:01 smithi096 ceph-mon[35085]: purged_snaps scrub ok 2024-09-19T05:01:01.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:01 smithi096 ceph-mon[35085]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:01:01.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:01 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:01:01.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:01 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:01:01.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:01 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:01.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:01 smithi096 ceph-mon[35085]: pgmap v236: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:01:01.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:01 smithi096 ceph-mon[35085]: osdmap e70: 11 total, 11 up, 11 in 2024-09-19T05:01:01.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:01 smithi043 ceph-mon[30800]: purged_snaps scrub starts 2024-09-19T05:01:01.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:01 smithi043 ceph-mon[30800]: purged_snaps scrub ok 2024-09-19T05:01:01.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:01 smithi043 ceph-mon[30800]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:01:01.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:01 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:01:01.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:01 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:01:01.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:01 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:01.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:01 smithi043 ceph-mon[30800]: pgmap v236: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:01:01.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:01 smithi043 ceph-mon[30800]: osdmap e70: 11 total, 11 up, 11 in 2024-09-19T05:01:01.904 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:01 smithi151 ceph-mon[35425]: purged_snaps scrub starts 2024-09-19T05:01:01.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:01 smithi151 ceph-mon[35425]: purged_snaps scrub ok 2024-09-19T05:01:01.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:01 smithi151 ceph-mon[35425]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:01:01.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:01 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T05:01:01.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:01 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T05:01:01.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:01 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:01.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:01 smithi151 ceph-mon[35425]: pgmap v236: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:01:01.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:01 smithi151 ceph-mon[35425]: osdmap e70: 11 total, 11 up, 11 in 2024-09-19T05:01:03.719 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:03 smithi151 ceph-mon[35425]: osdmap e71: 11 total, 11 up, 11 in 2024-09-19T05:01:03.719 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:03 smithi151 ceph-mon[35425]: pgmap v239: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:01:03.719 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:03 smithi151 ceph-mon[35425]: from='client.? 172.21.15.151:0/2047723791' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "760b6afc-253d-44dd-88e9-dcc97f67c9a5"}]: dispatch 2024-09-19T05:01:03.719 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:03 smithi151 ceph-mon[35425]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "760b6afc-253d-44dd-88e9-dcc97f67c9a5"}]: dispatch 2024-09-19T05:01:03.719 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:03 smithi151 ceph-mon[35425]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "760b6afc-253d-44dd-88e9-dcc97f67c9a5"}]': finished 2024-09-19T05:01:03.719 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:03 smithi151 ceph-mon[35425]: osdmap e72: 12 total, 11 up, 12 in 2024-09-19T05:01:03.719 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:03 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:01:03.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:03 smithi096 ceph-mon[35085]: osdmap e71: 11 total, 11 up, 11 in 2024-09-19T05:01:03.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:03 smithi096 ceph-mon[35085]: pgmap v239: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:01:03.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:03 smithi096 ceph-mon[35085]: from='client.? 172.21.15.151:0/2047723791' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "760b6afc-253d-44dd-88e9-dcc97f67c9a5"}]: dispatch 2024-09-19T05:01:03.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:03 smithi096 ceph-mon[35085]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "760b6afc-253d-44dd-88e9-dcc97f67c9a5"}]: dispatch 2024-09-19T05:01:03.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:03 smithi096 ceph-mon[35085]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "760b6afc-253d-44dd-88e9-dcc97f67c9a5"}]': finished 2024-09-19T05:01:03.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:03 smithi096 ceph-mon[35085]: osdmap e72: 12 total, 11 up, 12 in 2024-09-19T05:01:03.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:03 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:01:03.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:03 smithi043 ceph-mon[30800]: osdmap e71: 11 total, 11 up, 11 in 2024-09-19T05:01:03.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:03 smithi043 ceph-mon[30800]: pgmap v239: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:01:03.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:03 smithi043 ceph-mon[30800]: from='client.? 172.21.15.151:0/2047723791' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "760b6afc-253d-44dd-88e9-dcc97f67c9a5"}]: dispatch 2024-09-19T05:01:03.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:03 smithi043 ceph-mon[30800]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "760b6afc-253d-44dd-88e9-dcc97f67c9a5"}]: dispatch 2024-09-19T05:01:03.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:03 smithi043 ceph-mon[30800]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "760b6afc-253d-44dd-88e9-dcc97f67c9a5"}]': finished 2024-09-19T05:01:03.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:03 smithi043 ceph-mon[30800]: osdmap e72: 12 total, 11 up, 12 in 2024-09-19T05:01:03.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:03 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:01:04.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:04 smithi096 ceph-mon[35085]: from='client.? 172.21.15.151:0/3861707935' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:01:04.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:04 smithi096 ceph-mon[35085]: osdmap e73: 12 total, 11 up, 12 in 2024-09-19T05:01:04.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:04 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:01:04.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:04 smithi043 ceph-mon[30800]: from='client.? 172.21.15.151:0/3861707935' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:01:04.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:04 smithi043 ceph-mon[30800]: osdmap e73: 12 total, 11 up, 12 in 2024-09-19T05:01:04.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:04 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:01:04.904 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:04 smithi151 ceph-mon[35425]: from='client.? 172.21.15.151:0/3861707935' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T05:01:04.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:04 smithi151 ceph-mon[35425]: osdmap e73: 12 total, 11 up, 12 in 2024-09-19T05:01:04.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:04 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:01:05.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:05 smithi096 ceph-mon[35085]: pgmap v242: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:01:05.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:05 smithi043 ceph-mon[30800]: pgmap v242: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:01:05.904 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:05 smithi151 ceph-mon[35425]: pgmap v242: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:01:07.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:07 smithi096 ceph-mon[35085]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-19T05:01:07.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:07 smithi043 ceph-mon[30800]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-19T05:01:07.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:07 smithi151 ceph-mon[35425]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-19T05:01:09.358 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:09 smithi151 ceph-mon[35425]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-19T05:01:09.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:09 smithi096 ceph-mon[35085]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-19T05:01:09.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:09 smithi043 ceph-mon[30800]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-19T05:01:11.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:11 smithi151 ceph-mon[35425]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T05:01:11.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:11 smithi096 ceph-mon[35085]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T05:01:11.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:11 smithi043 ceph-mon[30800]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T05:01:12.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:12 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T05:01:12.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:12 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:12.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:12 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T05:01:12.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:12 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:12.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:12 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T05:01:12.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:12 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:13.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:13 smithi151 ceph-mon[35425]: Deploying daemon osd.11 on smithi151 2024-09-19T05:01:13.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:13 smithi151 ceph-mon[35425]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-19T05:01:13.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:13 smithi096 ceph-mon[35085]: Deploying daemon osd.11 on smithi151 2024-09-19T05:01:13.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:13 smithi096 ceph-mon[35085]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-19T05:01:13.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:13 smithi043 ceph-mon[30800]: Deploying daemon osd.11 on smithi151 2024-09-19T05:01:13.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:13 smithi043 ceph-mon[30800]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-19T05:01:15.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:15 smithi151 ceph-mon[35425]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-19T05:01:15.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:15 smithi096 ceph-mon[35085]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-19T05:01:15.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:15 smithi043 ceph-mon[30800]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-19T05:01:17.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:17 smithi151 ceph-mon[35425]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T05:01:17.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:17 smithi096 ceph-mon[35085]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T05:01:17.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:17 smithi043 ceph-mon[30800]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T05:01:18.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:18 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:01:18.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:18 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:18.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:18 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:18.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:18 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:01:18.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:18 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:18.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:18 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:18.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:18 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:01:18.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:18 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:18.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:18 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:19.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:19 smithi151 ceph-mon[35425]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:01:19.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:19 smithi096 ceph-mon[35085]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:01:19.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:19 smithi043 ceph-mon[30800]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:01:21.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:21 smithi151 ceph-mon[35425]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:01:21.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:21 smithi096 ceph-mon[35085]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:01:21.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:21 smithi043 ceph-mon[30800]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:01:21.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:21.866+0000 7f9bb57fa640 1 -- 172.21.15.151:0/406543680 <== mgr.14150 v2:172.21.15.43:6800/3167108925 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f9bb8062630 con 0x7f9ba00a3270 2024-09-19T05:01:21.868 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 11 on host 'smithi151' 2024-09-19T05:01:21.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:21.869+0000 7f9bbe4fe640 1 -- 172.21.15.151:0/406543680 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f9ba00a3270 msgr2=0x7f9ba00a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:21.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:21.869+0000 7f9bbe4fe640 1 --2- 172.21.15.151:0/406543680 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f9ba00a3270 0x7f9ba00a5730 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9ba80058c0 tx=0x7f9ba8005850 comp rx=0 tx=0).stop 2024-09-19T05:01:21.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:21.869+0000 7f9bbe4fe640 1 -- 172.21.15.151:0/406543680 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bb81037c0 msgr2=0x7f9bb819f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:21.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:21.869+0000 7f9bbe4fe640 1 --2- 172.21.15.151:0/406543680 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bb81037c0 0x7f9bb819f3c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0005e30 tx=0x7f9bb0031ee0 comp rx=0 tx=0).stop 2024-09-19T05:01:21.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:21.869+0000 7f9bbe4fe640 1 -- 172.21.15.151:0/406543680 shutdown_connections 2024-09-19T05:01:21.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:21.869+0000 7f9bbe4fe640 1 --2- 172.21.15.151:0/406543680 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f9ba00a3270 0x7f9ba00a5730 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:21.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:21.869+0000 7f9bbe4fe640 1 --2- 172.21.15.151:0/406543680 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bb81037c0 0x7f9bb819f3c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:21.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:21.869+0000 7f9bbe4fe640 1 --2- 172.21.15.151:0/406543680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bb8102e00 0x7f9bb8198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:21.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:21.869+0000 7f9bbe4fe640 1 --2- 172.21.15.151:0/406543680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bb8101c00 0x7f9bb8197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:21.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:21.869+0000 7f9bbe4fe640 1 -- 172.21.15.151:0/406543680 >> 172.21.15.151:0/406543680 conn(0x7f9bb80fd390 msgr2=0x7f9bb8106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:01:21.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:21.869+0000 7f9bbe4fe640 1 -- 172.21.15.151:0/406543680 shutdown_connections 2024-09-19T05:01:21.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:21.869+0000 7f9bbe4fe640 1 -- 172.21.15.151:0/406543680 wait complete. 2024-09-19T05:01:22.617 DEBUG:teuthology.orchestra.run.smithi151:osd.11> sudo journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@osd.11.service 2024-09-19T05:01:22.619 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-19T05:01:22.619 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T05:01:23.010 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:01:23.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:22 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:23.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:22 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:23.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:22 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:23.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:22 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:23.153 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:22 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:23.153 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:22 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:23.153 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:22 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:23.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:22 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:23.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:22 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:23.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:22 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:23.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:22 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:23.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:22 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:23.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.652+0000 7f02ce200640 1 -- 172.21.15.43:0/2921760974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02c8103e00 msgr2=0x7f02c8104200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:23.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.652+0000 7f02ce200640 1 --2- 172.21.15.43:0/2921760974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02c8103e00 0x7f02c8104200 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f02b8009900 tx=0x7f02b802a410 comp rx=0 tx=0).stop 2024-09-19T05:01:23.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.652+0000 7f02ce200640 1 -- 172.21.15.43:0/2921760974 shutdown_connections 2024-09-19T05:01:23.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.652+0000 7f02ce200640 1 --2- 172.21.15.43:0/2921760974 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02c8105920 0x7f02c810c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:23.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.652+0000 7f02ce200640 1 --2- 172.21.15.43:0/2921760974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02c8104f60 0x7f02c81053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:23.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.652+0000 7f02ce200640 1 --2- 172.21.15.43:0/2921760974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02c8103e00 0x7f02c8104200 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:23.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.652+0000 7f02ce200640 1 -- 172.21.15.43:0/2921760974 >> 172.21.15.43:0/2921760974 conn(0x7f02c80ff5b0 msgr2=0x7f02c81019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:01:23.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.652+0000 7f02ce200640 1 -- 172.21.15.43:0/2921760974 shutdown_connections 2024-09-19T05:01:23.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.652+0000 7f02ce200640 1 -- 172.21.15.43:0/2921760974 wait complete. 2024-09-19T05:01:23.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.653+0000 7f02ce200640 1 Processor -- start 2024-09-19T05:01:23.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.654+0000 7f02ce200640 1 -- start start 2024-09-19T05:01:23.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.654+0000 7f02ce200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02c8103e00 0x7f02c819a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:23.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.654+0000 7f02ce200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02c8104f60 0x7f02c819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:23.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.654+0000 7f02ce200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02c8105920 0x7f02c81a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:23.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.654+0000 7f02ce200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f02c810fa90 con 0x7f02c8103e00 2024-09-19T05:01:23.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.654+0000 7f02ce200640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f02c810f910 con 0x7f02c8104f60 2024-09-19T05:01:23.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.654+0000 7f02ce200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f02c810fc10 con 0x7f02c8105920 2024-09-19T05:01:23.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.654+0000 7f02c7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02c8103e00 0x7f02c819a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:23.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.654+0000 7f02c6a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02c8104f60 0x7f02c819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:23.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.654+0000 7f02c7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02c8103e00 0x7f02c819a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42784/0 (socket says 172.21.15.43:42784) 2024-09-19T05:01:23.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.654+0000 7f02c7400640 1 -- 172.21.15.43:0/375385302 learned_addr learned my addr 172.21.15.43:0/375385302 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:01:23.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.654+0000 7f02c6a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02c8104f60 0x7f02c819a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:56106/0 (socket says 172.21.15.43:56106) 2024-09-19T05:01:23.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.654+0000 7f02c7e00640 1 --2- 172.21.15.43:0/375385302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02c8105920 0x7f02c81a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:23.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.654+0000 7f02c6a00640 1 -- 172.21.15.43:0/375385302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02c8105920 msgr2=0x7f02c81a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:23.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.654+0000 7f02c6a00640 1 --2- 172.21.15.43:0/375385302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02c8105920 0x7f02c81a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:23.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.654+0000 7f02c6a00640 1 -- 172.21.15.43:0/375385302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02c8103e00 msgr2=0x7f02c819a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:23.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.655+0000 7f02c6a00640 1 --2- 172.21.15.43:0/375385302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02c8103e00 0x7f02c819a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:23.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.655+0000 7f02c6a00640 1 -- 172.21.15.43:0/375385302 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c81a1d10 con 0x7f02c8104f60 2024-09-19T05:01:23.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.655+0000 7f02c6a00640 1 --2- 172.21.15.43:0/375385302 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02c8104f60 0x7f02c819a620 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f02b400e9c0 tx=0x7f02b400edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:01:23.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.655+0000 7f02bfe00640 1 -- 172.21.15.43:0/375385302 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b400ccc0 con 0x7f02c8104f60 2024-09-19T05:01:23.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.655+0000 7f02bfe00640 1 -- 172.21.15.43:0/375385302 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02b40035f0 con 0x7f02c8104f60 2024-09-19T05:01:23.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.655+0000 7f02ce200640 1 -- 172.21.15.43:0/375385302 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02c81a1fa0 con 0x7f02c8104f60 2024-09-19T05:01:23.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.655+0000 7f02bfe00640 1 -- 172.21.15.43:0/375385302 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b400ce60 con 0x7f02c8104f60 2024-09-19T05:01:23.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.655+0000 7f02ce200640 1 -- 172.21.15.43:0/375385302 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02c81a24b0 con 0x7f02c8104f60 2024-09-19T05:01:23.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.656+0000 7f02ce200640 1 -- 172.21.15.43:0/375385302 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02c810b190 con 0x7f02c8104f60 2024-09-19T05:01:23.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.658+0000 7f02bfe00640 1 -- 172.21.15.43:0/375385302 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f02b40037a0 con 0x7f02c8104f60 2024-09-19T05:01:23.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.660+0000 7f02bfe00640 1 --2- 172.21.15.43:0/375385302 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f02ac0a3270 0x7f02ac0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:23.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.660+0000 7f02bfe00640 1 -- 172.21.15.43:0/375385302 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6809+0+0 (secure 0 0 0) 0x7f02b40cf6e0 con 0x7f02c8104f60 2024-09-19T05:01:23.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.660+0000 7f02c7400640 1 --2- 172.21.15.43:0/375385302 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f02ac0a3270 0x7f02ac0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:23.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.660+0000 7f02c7400640 1 --2- 172.21.15.43:0/375385302 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f02ac0a3270 0x7f02ac0a5730 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f02b8031440 tx=0x7f02b80315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:01:23.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.662+0000 7f02bfe00640 1 -- 172.21.15.43:0/375385302 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f02b409c9d0 con 0x7f02c8104f60 2024-09-19T05:01:23.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.850+0000 7f02ce200640 1 -- 172.21.15.43:0/375385302 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f02c802d9c0 con 0x7f02c8104f60 2024-09-19T05:01:23.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.851+0000 7f02bfe00640 1 -- 172.21.15.43:0/375385302 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) ==== 74+0+133 (secure 0 0 0) 0x7f02b409c2d0 con 0x7f02c8104f60 2024-09-19T05:01:23.854 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:01:23.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.853+0000 7f02ce200640 1 -- 172.21.15.43:0/375385302 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f02ac0a3270 msgr2=0x7f02ac0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:23.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.853+0000 7f02ce200640 1 --2- 172.21.15.43:0/375385302 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f02ac0a3270 0x7f02ac0a5730 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f02b8031440 tx=0x7f02b80315b0 comp rx=0 tx=0).stop 2024-09-19T05:01:23.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.853+0000 7f02ce200640 1 -- 172.21.15.43:0/375385302 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02c8104f60 msgr2=0x7f02c819a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:23.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.853+0000 7f02ce200640 1 --2- 172.21.15.43:0/375385302 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02c8104f60 0x7f02c819a620 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f02b400e9c0 tx=0x7f02b400edc0 comp rx=0 tx=0).stop 2024-09-19T05:01:23.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.854+0000 7f02ce200640 1 -- 172.21.15.43:0/375385302 shutdown_connections 2024-09-19T05:01:23.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.854+0000 7f02ce200640 1 --2- 172.21.15.43:0/375385302 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f02ac0a3270 0x7f02ac0a5730 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:23.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.854+0000 7f02ce200640 1 --2- 172.21.15.43:0/375385302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02c8105920 0x7f02c81a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:23.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.854+0000 7f02ce200640 1 --2- 172.21.15.43:0/375385302 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02c8104f60 0x7f02c819a620 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:23.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.854+0000 7f02ce200640 1 --2- 172.21.15.43:0/375385302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02c8103e00 0x7f02c819a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:23.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.854+0000 7f02ce200640 1 -- 172.21.15.43:0/375385302 >> 172.21.15.43:0/375385302 conn(0x7f02c80ff5b0 msgr2=0x7f02c8108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:01:23.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.854+0000 7f02ce200640 1 -- 172.21.15.43:0/375385302 shutdown_connections 2024-09-19T05:01:23.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:23.854+0000 7f02ce200640 1 -- 172.21.15.43:0/375385302 wait complete. 2024-09-19T05:01:24.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:23 smithi151 ceph-mon[35425]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:01:24.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:23 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/375385302' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T05:01:24.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:01:23 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:01:23.914+0000 7fda8f640640 -1 osd.11 0 log_to_monitors true 2024-09-19T05:01:24.203 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:23 smithi043 ceph-mon[30800]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:01:24.203 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:23 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/375385302' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T05:01:24.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:23 smithi096 ceph-mon[35085]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:01:24.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:23 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/375385302' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T05:01:24.702 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":73,"num_osds":12,"num_up_osds":11,"osd_up_since":1726722060,"num_in_osds":12,"osd_in_since":1726722063,"num_remapped_pgs":0} 2024-09-19T05:01:25.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:24 smithi151 ceph-mon[35425]: from='osd.11 [v2:172.21.15.151:6824/292202881,v1:172.21.15.151:6825/292202881]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T05:01:25.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:24 smithi151 ceph-mon[35425]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T05:01:25.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:24 smithi096 ceph-mon[35085]: from='osd.11 [v2:172.21.15.151:6824/292202881,v1:172.21.15.151:6825/292202881]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T05:01:25.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:24 smithi096 ceph-mon[35085]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T05:01:25.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:24 smithi043 ceph-mon[30800]: from='osd.11 [v2:172.21.15.151:6824/292202881,v1:172.21.15.151:6825/292202881]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T05:01:25.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:24 smithi043 ceph-mon[30800]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T05:01:25.703 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T05:01:26.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:25 smithi043 ceph-mon[30800]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T05:01:26.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:25 smithi043 ceph-mon[30800]: osdmap e74: 12 total, 11 up, 12 in 2024-09-19T05:01:26.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:25 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:01:26.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:25 smithi043 ceph-mon[30800]: from='osd.11 [v2:172.21.15.151:6824/292202881,v1:172.21.15.151:6825/292202881]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T05:01:26.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:25 smithi043 ceph-mon[30800]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T05:01:26.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:25 smithi043 ceph-mon[30800]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:01:26.113 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:01:26.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:25 smithi151 ceph-mon[35425]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T05:01:26.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:25 smithi151 ceph-mon[35425]: osdmap e74: 12 total, 11 up, 12 in 2024-09-19T05:01:26.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:25 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:01:26.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:25 smithi151 ceph-mon[35425]: from='osd.11 [v2:172.21.15.151:6824/292202881,v1:172.21.15.151:6825/292202881]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T05:01:26.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:25 smithi151 ceph-mon[35425]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T05:01:26.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:25 smithi151 ceph-mon[35425]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:01:26.222 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:25 smithi096 ceph-mon[35085]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T05:01:26.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:25 smithi096 ceph-mon[35085]: osdmap e74: 12 total, 11 up, 12 in 2024-09-19T05:01:26.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:25 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:01:26.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:25 smithi096 ceph-mon[35085]: from='osd.11 [v2:172.21.15.151:6824/292202881,v1:172.21.15.151:6825/292202881]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T05:01:26.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:25 smithi096 ceph-mon[35085]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T05:01:26.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:25 smithi096 ceph-mon[35085]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:01:26.654 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:01:26 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:01:26.345+0000 7fda8b5c3640 -1 osd.11 0 waiting for initial osdmap 2024-09-19T05:01:26.654 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:01:26 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:01:26.361+0000 7fda833e3640 -1 osd.11 75 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:01:26.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.778+0000 7fab51c00640 1 -- 172.21.15.43:0/2154361464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab4c103c50 msgr2=0x7fab4c104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:26.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.778+0000 7fab51c00640 1 --2- 172.21.15.43:0/2154361464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab4c103c50 0x7fab4c104050 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fab3c009900 tx=0x7fab3c02a3f0 comp rx=0 tx=0).stop 2024-09-19T05:01:26.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.778+0000 7fab51c00640 1 -- 172.21.15.43:0/2154361464 shutdown_connections 2024-09-19T05:01:26.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.778+0000 7fab51c00640 1 --2- 172.21.15.43:0/2154361464 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab4c105810 0x7fab4c10c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:26.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.778+0000 7fab51c00640 1 --2- 172.21.15.43:0/2154361464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab4c104e50 0x7fab4c1052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:26.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.778+0000 7fab51c00640 1 --2- 172.21.15.43:0/2154361464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab4c103c50 0x7fab4c104050 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:26.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.778+0000 7fab51c00640 1 -- 172.21.15.43:0/2154361464 >> 172.21.15.43:0/2154361464 conn(0x7fab4c0ff3e0 msgr2=0x7fab4c101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:01:26.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.778+0000 7fab51c00640 1 -- 172.21.15.43:0/2154361464 shutdown_connections 2024-09-19T05:01:26.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.778+0000 7fab51c00640 1 -- 172.21.15.43:0/2154361464 wait complete. 2024-09-19T05:01:26.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.779+0000 7fab51c00640 1 Processor -- start 2024-09-19T05:01:26.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.779+0000 7fab51c00640 1 -- start start 2024-09-19T05:01:26.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.779+0000 7fab51c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab4c103c50 0x7fab4c199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:26.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.780+0000 7fab51c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab4c104e50 0x7fab4c19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:26.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.780+0000 7fab51c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab4c105810 0x7fab4c1a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:26.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.780+0000 7fab51c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c10f8b0 con 0x7fab4c104e50 2024-09-19T05:01:26.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.780+0000 7fab51c00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c10f730 con 0x7fab4c105810 2024-09-19T05:01:26.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.780+0000 7fab51c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c10fa30 con 0x7fab4c103c50 2024-09-19T05:01:26.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.780+0000 7fab4b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab4c104e50 0x7fab4c19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:26.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.780+0000 7fab4be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab4c103c50 0x7fab4c199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:26.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.780+0000 7fab4be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab4c103c50 0x7fab4c199f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:52146/0 (socket says 172.21.15.43:52146) 2024-09-19T05:01:26.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.780+0000 7fab4be00640 1 -- 172.21.15.43:0/1806133010 learned_addr learned my addr 172.21.15.43:0/1806133010 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:01:26.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.780+0000 7fab4b400640 1 --2- 172.21.15.43:0/1806133010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab4c104e50 0x7fab4c19a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49550/0 (socket says 172.21.15.43:49550) 2024-09-19T05:01:26.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.780+0000 7fab51200640 1 --2- 172.21.15.43:0/1806133010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab4c105810 0x7fab4c1a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:26.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.780+0000 7fab4be00640 1 -- 172.21.15.43:0/1806133010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab4c105810 msgr2=0x7fab4c1a14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:26.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.780+0000 7fab4be00640 1 --2- 172.21.15.43:0/1806133010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab4c105810 0x7fab4c1a14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:26.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.780+0000 7fab4be00640 1 -- 172.21.15.43:0/1806133010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab4c104e50 msgr2=0x7fab4c19a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:26.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.780+0000 7fab4be00640 1 --2- 172.21.15.43:0/1806133010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab4c104e50 0x7fab4c19a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:26.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.780+0000 7fab4be00640 1 -- 172.21.15.43:0/1806133010 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab4c1a1b60 con 0x7fab4c103c50 2024-09-19T05:01:26.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.781+0000 7fab4be00640 1 --2- 172.21.15.43:0/1806133010 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab4c103c50 0x7fab4c199f30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fab3c031610 tx=0x7fab3c004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:01:26.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.781+0000 7fab48c00640 1 -- 172.21.15.43:0/1806133010 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab3c03f030 con 0x7fab4c103c50 2024-09-19T05:01:26.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.781+0000 7fab48c00640 1 -- 172.21.15.43:0/1806133010 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab3c003ad0 con 0x7fab4c103c50 2024-09-19T05:01:26.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.781+0000 7fab48c00640 1 -- 172.21.15.43:0/1806133010 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab3c035070 con 0x7fab4c103c50 2024-09-19T05:01:26.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.781+0000 7fab51c00640 1 -- 172.21.15.43:0/1806133010 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab4c1a1df0 con 0x7fab4c103c50 2024-09-19T05:01:26.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.781+0000 7fab51c00640 1 -- 172.21.15.43:0/1806133010 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab4c1a21c0 con 0x7fab4c103c50 2024-09-19T05:01:26.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.782+0000 7fab51c00640 1 -- 172.21.15.43:0/1806133010 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab4c10b110 con 0x7fab4c103c50 2024-09-19T05:01:26.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.784+0000 7fab48c00640 1 -- 172.21.15.43:0/1806133010 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fab3c042030 con 0x7fab4c103c50 2024-09-19T05:01:26.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.785+0000 7fab48c00640 1 --2- 172.21.15.43:0/1806133010 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fab300a32f0 0x7fab300a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:26.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.785+0000 7fab48c00640 1 -- 172.21.15.43:0/1806133010 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6847+0+0 (secure 0 0 0) 0x7fab3c0efb30 con 0x7fab4c103c50 2024-09-19T05:01:26.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.786+0000 7fab4b400640 1 --2- 172.21.15.43:0/1806133010 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fab300a32f0 0x7fab300a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:26.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.786+0000 7fab4b400640 1 --2- 172.21.15.43:0/1806133010 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fab300a32f0 0x7fab300a57b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fab4c19b450 tx=0x7fab38005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:01:26.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.787+0000 7fab48c00640 1 -- 172.21.15.43:0/1806133010 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fab3c0bcd70 con 0x7fab4c103c50 2024-09-19T05:01:26.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.975+0000 7fab51c00640 1 -- 172.21.15.43:0/1806133010 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fab4c062ef0 con 0x7fab4c103c50 2024-09-19T05:01:26.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.976+0000 7fab48c00640 1 -- 172.21.15.43:0/1806133010 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) ==== 74+0+133 (secure 0 0 0) 0x7fab3c0bc670 con 0x7fab4c103c50 2024-09-19T05:01:26.978 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:01:26.978 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:26 smithi043 ceph-mon[30800]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-19T05:01:26.978 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:26 smithi043 ceph-mon[30800]: osdmap e75: 12 total, 11 up, 12 in 2024-09-19T05:01:26.978 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:26 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:01:26.979 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:26 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:01:26.979 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:26 smithi043 ceph-mon[30800]: Detected new or changed devices on smithi151 2024-09-19T05:01:26.979 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:26 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:26.979 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:26 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:26.979 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:26 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:01:26.979 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:26 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:01:26.979 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:26 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:01:26.979 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:26 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:01:26.979 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:26 smithi043 ceph-mon[30800]: Adjusting osd_memory_target on smithi151 to 4272M 2024-09-19T05:01:26.979 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:26 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:26.979 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:26 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:26.979 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:26 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:01:26.980 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:26 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:26.980 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:26 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:01:26.980 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:26 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:26.980 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:26 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:01:26.980 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:26 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:26.980 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:26 smithi043 ceph-mon[30800]: from='osd.11 ' entity='osd.11' 2024-09-19T05:01:26.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.978+0000 7fab51c00640 1 -- 172.21.15.43:0/1806133010 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fab300a32f0 msgr2=0x7fab300a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:26.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.978+0000 7fab51c00640 1 --2- 172.21.15.43:0/1806133010 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fab300a32f0 0x7fab300a57b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fab4c19b450 tx=0x7fab38005f90 comp rx=0 tx=0).stop 2024-09-19T05:01:26.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.978+0000 7fab51c00640 1 -- 172.21.15.43:0/1806133010 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab4c103c50 msgr2=0x7fab4c199f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:26.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.978+0000 7fab51c00640 1 --2- 172.21.15.43:0/1806133010 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab4c103c50 0x7fab4c199f30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fab3c031610 tx=0x7fab3c004270 comp rx=0 tx=0).stop 2024-09-19T05:01:26.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.978+0000 7fab51c00640 1 -- 172.21.15.43:0/1806133010 shutdown_connections 2024-09-19T05:01:26.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.978+0000 7fab51c00640 1 --2- 172.21.15.43:0/1806133010 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fab300a32f0 0x7fab300a57b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:26.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.978+0000 7fab51c00640 1 --2- 172.21.15.43:0/1806133010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab4c105810 0x7fab4c1a14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:26.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.978+0000 7fab51c00640 1 --2- 172.21.15.43:0/1806133010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab4c104e50 0x7fab4c19a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:26.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.978+0000 7fab51c00640 1 --2- 172.21.15.43:0/1806133010 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab4c103c50 0x7fab4c199f30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:26.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.978+0000 7fab51c00640 1 -- 172.21.15.43:0/1806133010 >> 172.21.15.43:0/1806133010 conn(0x7fab4c0ff3e0 msgr2=0x7fab4c1082e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:01:26.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.979+0000 7fab51c00640 1 -- 172.21.15.43:0/1806133010 shutdown_connections 2024-09-19T05:01:26.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:26.979+0000 7fab51c00640 1 -- 172.21.15.43:0/1806133010 wait complete. 2024-09-19T05:01:27.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:26 smithi151 ceph-mon[35425]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-19T05:01:27.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:26 smithi151 ceph-mon[35425]: osdmap e75: 12 total, 11 up, 12 in 2024-09-19T05:01:27.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:26 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:01:27.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:26 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:01:27.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:26 smithi151 ceph-mon[35425]: Detected new or changed devices on smithi151 2024-09-19T05:01:27.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:26 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:27.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:26 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:27.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:26 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:01:27.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:26 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:01:27.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:26 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:01:27.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:26 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:01:27.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:26 smithi151 ceph-mon[35425]: Adjusting osd_memory_target on smithi151 to 4272M 2024-09-19T05:01:27.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:26 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:27.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:26 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:27.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:26 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:01:27.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:26 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:27.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:26 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:01:27.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:26 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:27.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:26 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:01:27.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:26 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:27.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:26 smithi151 ceph-mon[35425]: from='osd.11 ' entity='osd.11' 2024-09-19T05:01:27.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:26 smithi096 ceph-mon[35085]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-19T05:01:27.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:26 smithi096 ceph-mon[35085]: osdmap e75: 12 total, 11 up, 12 in 2024-09-19T05:01:27.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:26 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:01:27.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:26 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:01:27.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:26 smithi096 ceph-mon[35085]: Detected new or changed devices on smithi151 2024-09-19T05:01:27.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:26 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:27.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:26 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:27.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:26 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:01:27.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:26 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:01:27.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:26 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:01:27.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:26 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:01:27.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:26 smithi096 ceph-mon[35085]: Adjusting osd_memory_target on smithi151 to 4272M 2024-09-19T05:01:27.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:26 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:27.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:26 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:27.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:26 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:01:27.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:26 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:27.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:26 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:01:27.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:26 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:27.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:26 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:01:27.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:26 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:27.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:26 smithi096 ceph-mon[35085]: from='osd.11 ' entity='osd.11' 2024-09-19T05:01:27.766 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":75,"num_osds":12,"num_up_osds":11,"osd_up_since":1726722060,"num_in_osds":12,"osd_in_since":1726722063,"num_remapped_pgs":0} 2024-09-19T05:01:28.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:27 smithi043 ceph-mon[30800]: purged_snaps scrub starts 2024-09-19T05:01:28.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:27 smithi043 ceph-mon[30800]: purged_snaps scrub ok 2024-09-19T05:01:28.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:27 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:01:28.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:27 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/1806133010' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T05:01:28.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:27 smithi043 ceph-mon[30800]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:01:28.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:27 smithi043 ceph-mon[30800]: osd.11 [v2:172.21.15.151:6824/292202881,v1:172.21.15.151:6825/292202881] boot 2024-09-19T05:01:28.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:27 smithi043 ceph-mon[30800]: osdmap e76: 12 total, 12 up, 12 in 2024-09-19T05:01:28.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:27 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:01:28.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:27 smithi151 ceph-mon[35425]: purged_snaps scrub starts 2024-09-19T05:01:28.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:27 smithi151 ceph-mon[35425]: purged_snaps scrub ok 2024-09-19T05:01:28.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:27 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:01:28.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:27 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/1806133010' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T05:01:28.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:27 smithi151 ceph-mon[35425]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:01:28.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:27 smithi151 ceph-mon[35425]: osd.11 [v2:172.21.15.151:6824/292202881,v1:172.21.15.151:6825/292202881] boot 2024-09-19T05:01:28.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:27 smithi151 ceph-mon[35425]: osdmap e76: 12 total, 12 up, 12 in 2024-09-19T05:01:28.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:27 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:01:28.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:27 smithi096 ceph-mon[35085]: purged_snaps scrub starts 2024-09-19T05:01:28.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:27 smithi096 ceph-mon[35085]: purged_snaps scrub ok 2024-09-19T05:01:28.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:27 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:01:28.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:27 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/1806133010' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T05:01:28.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:27 smithi096 ceph-mon[35085]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-19T05:01:28.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:27 smithi096 ceph-mon[35085]: osd.11 [v2:172.21.15.151:6824/292202881,v1:172.21.15.151:6825/292202881] boot 2024-09-19T05:01:28.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:27 smithi096 ceph-mon[35085]: osdmap e76: 12 total, 12 up, 12 in 2024-09-19T05:01:28.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:27 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:01:28.767 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T05:01:29.155 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:01:29.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.743+0000 7f2761000640 1 -- 172.21.15.43:0/1301651200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f275c078790 msgr2=0x7f275c079030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:29.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.743+0000 7f2761000640 1 --2- 172.21.15.43:0/1301651200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f275c078790 0x7f275c079030 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2754009ca0 tx=0x7f275402a8c0 comp rx=0 tx=0).stop 2024-09-19T05:01:29.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.743+0000 7f2761000640 1 -- 172.21.15.43:0/1301651200 shutdown_connections 2024-09-19T05:01:29.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.743+0000 7f2761000640 1 --2- 172.21.15.43:0/1301651200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f275c078790 0x7f275c079030 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:29.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.743+0000 7f2761000640 1 --2- 172.21.15.43:0/1301651200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f275c077df0 0x7f275c078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:29.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.743+0000 7f2761000640 1 --2- 172.21.15.43:0/1301651200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f275c076ba0 0x7f275c076fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:29.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.743+0000 7f2761000640 1 -- 172.21.15.43:0/1301651200 >> 172.21.15.43:0/1301651200 conn(0x7f275c0ff5b0 msgr2=0x7f275c1019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:01:29.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.743+0000 7f2761000640 1 -- 172.21.15.43:0/1301651200 shutdown_connections 2024-09-19T05:01:29.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.743+0000 7f2761000640 1 -- 172.21.15.43:0/1301651200 wait complete. 2024-09-19T05:01:29.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.744+0000 7f2761000640 1 Processor -- start 2024-09-19T05:01:29.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.744+0000 7f2761000640 1 -- start start 2024-09-19T05:01:29.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.745+0000 7f2761000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f275c076ba0 0x7f275c19e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:29.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.745+0000 7f2761000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f275c077df0 0x7f275c19ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:29.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.745+0000 7f2761000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f275c078790 0x7f275c1a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:29.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.745+0000 7f2761000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f275c113f10 con 0x7f275c078790 2024-09-19T05:01:29.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.745+0000 7f2761000640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f275c113d90 con 0x7f275c076ba0 2024-09-19T05:01:29.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.745+0000 7f2761000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f275c114090 con 0x7f275c077df0 2024-09-19T05:01:29.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.745+0000 7f275a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f275c078790 0x7f275c1a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:29.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.745+0000 7f2759e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f275c076ba0 0x7f275c19e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:29.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.745+0000 7f275a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f275c078790 0x7f275c1a5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49574/0 (socket says 172.21.15.43:49574) 2024-09-19T05:01:29.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.745+0000 7f275a800640 1 -- 172.21.15.43:0/3200825885 learned_addr learned my addr 172.21.15.43:0/3200825885 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:01:29.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.745+0000 7f2753e00640 1 --2- 172.21.15.43:0/3200825885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f275c077df0 0x7f275c19ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:29.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.746+0000 7f2759e00640 1 -- 172.21.15.43:0/3200825885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f275c077df0 msgr2=0x7f275c19ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:29.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.746+0000 7f2759e00640 1 --2- 172.21.15.43:0/3200825885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f275c077df0 0x7f275c19ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:29.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.746+0000 7f2759e00640 1 -- 172.21.15.43:0/3200825885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f275c078790 msgr2=0x7f275c1a5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:29.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.746+0000 7f2759e00640 1 --2- 172.21.15.43:0/3200825885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f275c078790 0x7f275c1a5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:29.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.746+0000 7f2759e00640 1 -- 172.21.15.43:0/3200825885 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f275c1a6160 con 0x7f275c076ba0 2024-09-19T05:01:29.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.746+0000 7f2753e00640 1 --2- 172.21.15.43:0/3200825885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f275c077df0 0x7f275c19ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:01:29.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.746+0000 7f2759e00640 1 --2- 172.21.15.43:0/3200825885 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f275c076ba0 0x7f275c19e530 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f274c00e9f0 tx=0x7f274c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:01:29.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.746+0000 7f2752a00640 1 -- 172.21.15.43:0/3200825885 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274c00ccc0 con 0x7f275c076ba0 2024-09-19T05:01:29.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.746+0000 7f2761000640 1 -- 172.21.15.43:0/3200825885 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f275c1a6450 con 0x7f275c076ba0 2024-09-19T05:01:29.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.746+0000 7f2761000640 1 -- 172.21.15.43:0/3200825885 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f275c1a69e0 con 0x7f275c076ba0 2024-09-19T05:01:29.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.746+0000 7f2752a00640 1 -- 172.21.15.43:0/3200825885 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f274c010910 con 0x7f275c076ba0 2024-09-19T05:01:29.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.746+0000 7f2752a00640 1 -- 172.21.15.43:0/3200825885 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274c018960 con 0x7f275c076ba0 2024-09-19T05:01:29.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.748+0000 7f2761000640 1 -- 172.21.15.43:0/3200825885 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27380052c0 con 0x7f275c076ba0 2024-09-19T05:01:29.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.749+0000 7f2752a00640 1 -- 172.21.15.43:0/3200825885 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f274c010ac0 con 0x7f275c076ba0 2024-09-19T05:01:29.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.751+0000 7f2752a00640 1 --2- 172.21.15.43:0/3200825885 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f27400a32f0 0x7f27400a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:29.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.751+0000 7f2752a00640 1 -- 172.21.15.43:0/3200825885 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f274c0cf7a0 con 0x7f275c076ba0 2024-09-19T05:01:29.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.751+0000 7f2753e00640 1 --2- 172.21.15.43:0/3200825885 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f27400a32f0 0x7f27400a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:29.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.752+0000 7f2753e00640 1 --2- 172.21.15.43:0/3200825885 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f27400a32f0 0x7f27400a57b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f275c19fa50 tx=0x7f2744005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:01:29.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.755+0000 7f2752a00640 1 -- 172.21.15.43:0/3200825885 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f274c054210 con 0x7f275c076ba0 2024-09-19T05:01:29.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.944+0000 7f2761000640 1 -- 172.21.15.43:0/3200825885 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f27380055b0 con 0x7f275c076ba0 2024-09-19T05:01:29.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.945+0000 7f2752a00640 1 -- 172.21.15.43:0/3200825885 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v77) ==== 74+0+133 (secure 0 0 0) 0x7f274c09c270 con 0x7f275c076ba0 2024-09-19T05:01:29.947 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:01:29.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.947+0000 7f2761000640 1 -- 172.21.15.43:0/3200825885 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f27400a32f0 msgr2=0x7f27400a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:29.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.947+0000 7f2761000640 1 --2- 172.21.15.43:0/3200825885 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f27400a32f0 0x7f27400a57b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f275c19fa50 tx=0x7f2744005f90 comp rx=0 tx=0).stop 2024-09-19T05:01:29.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.947+0000 7f2761000640 1 -- 172.21.15.43:0/3200825885 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f275c076ba0 msgr2=0x7f275c19e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:29.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.947+0000 7f2761000640 1 --2- 172.21.15.43:0/3200825885 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f275c076ba0 0x7f275c19e530 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f274c00e9f0 tx=0x7f274c00eeb0 comp rx=0 tx=0).stop 2024-09-19T05:01:29.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.947+0000 7f2761000640 1 -- 172.21.15.43:0/3200825885 shutdown_connections 2024-09-19T05:01:29.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.947+0000 7f2761000640 1 --2- 172.21.15.43:0/3200825885 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f27400a32f0 0x7f27400a57b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:29.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.947+0000 7f2761000640 1 --2- 172.21.15.43:0/3200825885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f275c078790 0x7f275c1a5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:29.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.947+0000 7f2761000640 1 --2- 172.21.15.43:0/3200825885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f275c077df0 0x7f275c19ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:29.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.947+0000 7f2761000640 1 --2- 172.21.15.43:0/3200825885 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f275c076ba0 0x7f275c19e530 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:29.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.947+0000 7f2761000640 1 -- 172.21.15.43:0/3200825885 >> 172.21.15.43:0/3200825885 conn(0x7f275c0ff5b0 msgr2=0x7f275c101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:01:29.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.948+0000 7f2761000640 1 -- 172.21.15.43:0/3200825885 shutdown_connections 2024-09-19T05:01:29.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:29.948+0000 7f2761000640 1 -- 172.21.15.43:0/3200825885 wait complete. 2024-09-19T05:01:30.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:29 smithi151 ceph-mon[35425]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T05:01:30.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:29 smithi151 ceph-mon[35425]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:30.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:29 smithi096 ceph-mon[35085]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T05:01:30.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:29 smithi096 ceph-mon[35085]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:30.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:29 smithi043 ceph-mon[30800]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T05:01:30.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:29 smithi043 ceph-mon[30800]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:30.844 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":77,"num_osds":12,"num_up_osds":12,"osd_up_since":1726722087,"num_in_osds":12,"osd_in_since":1726722063,"num_remapped_pgs":0} 2024-09-19T05:01:30.845 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T05:01:31.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:30 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/3200825885' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T05:01:31.219 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:01:31.222 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:30 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/3200825885' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T05:01:31.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:30 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/3200825885' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T05:01:31.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.776+0000 7f43f1400640 1 -- 172.21.15.43:0/867881962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43ec077040 msgr2=0x7f43ec1124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:31.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.776+0000 7f43f1400640 1 --2- 172.21.15.43:0/867881962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43ec077040 0x7f43ec1124f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f43d8009900 tx=0x7f43d802a400 comp rx=0 tx=0).stop 2024-09-19T05:01:31.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.776+0000 7f43f1400640 1 -- 172.21.15.43:0/867881962 shutdown_connections 2024-09-19T05:01:31.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.776+0000 7f43f1400640 1 --2- 172.21.15.43:0/867881962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43ec112a30 0x7f43ec114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:31.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.776+0000 7f43f1400640 1 --2- 172.21.15.43:0/867881962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43ec077040 0x7f43ec1124f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:31.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.776+0000 7f43f1400640 1 --2- 172.21.15.43:0/867881962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43ec076720 0x7f43ec076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:31.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.776+0000 7f43f1400640 1 -- 172.21.15.43:0/867881962 >> 172.21.15.43:0/867881962 conn(0x7f43ec0ffa90 msgr2=0x7f43ec101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:01:31.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.776+0000 7f43f1400640 1 -- 172.21.15.43:0/867881962 shutdown_connections 2024-09-19T05:01:31.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.776+0000 7f43f1400640 1 -- 172.21.15.43:0/867881962 wait complete. 2024-09-19T05:01:31.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.777+0000 7f43f1400640 1 Processor -- start 2024-09-19T05:01:31.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.777+0000 7f43f1400640 1 -- start start 2024-09-19T05:01:31.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.777+0000 7f43f1400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43ec076720 0x7f43ec1a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:31.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.777+0000 7f43f1400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43ec077040 0x7f43ec1a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:31.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.777+0000 7f43f1400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43ec112a30 0x7f43ec19e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:31.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.777+0000 7f43f1400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f43ec118610 con 0x7f43ec076720 2024-09-19T05:01:31.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.778+0000 7f43f1400640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f43ec118490 con 0x7f43ec077040 2024-09-19T05:01:31.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.778+0000 7f43f1400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f43ec118790 con 0x7f43ec112a30 2024-09-19T05:01:31.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.778+0000 7f43ea800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43ec076720 0x7f43ec1a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:31.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.778+0000 7f43ea800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43ec076720 0x7f43ec1a4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49596/0 (socket says 172.21.15.43:49596) 2024-09-19T05:01:31.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.778+0000 7f43ea800640 1 -- 172.21.15.43:0/2175523218 learned_addr learned my addr 172.21.15.43:0/2175523218 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:01:31.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.778+0000 7f43e9e00640 1 --2- 172.21.15.43:0/2175523218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43ec077040 0x7f43ec1a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:31.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.778+0000 7f43eb200640 1 --2- 172.21.15.43:0/2175523218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43ec112a30 0x7f43ec19e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:31.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.778+0000 7f43ea800640 1 -- 172.21.15.43:0/2175523218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43ec112a30 msgr2=0x7f43ec19e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:31.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.778+0000 7f43ea800640 1 --2- 172.21.15.43:0/2175523218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43ec112a30 0x7f43ec19e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:31.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.778+0000 7f43ea800640 1 -- 172.21.15.43:0/2175523218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43ec077040 msgr2=0x7f43ec1a4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:31.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.778+0000 7f43ea800640 1 --2- 172.21.15.43:0/2175523218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43ec077040 0x7f43ec1a4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:31.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.778+0000 7f43ea800640 1 -- 172.21.15.43:0/2175523218 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43ec19f050 con 0x7f43ec076720 2024-09-19T05:01:31.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.779+0000 7f43ea800640 1 --2- 172.21.15.43:0/2175523218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43ec076720 0x7f43ec1a4760 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f43dc00d770 tx=0x7f43dc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:01:31.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.779+0000 7f43e3400640 1 -- 172.21.15.43:0/2175523218 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc017070 con 0x7f43ec076720 2024-09-19T05:01:31.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.779+0000 7f43e3400640 1 -- 172.21.15.43:0/2175523218 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43dc005500 con 0x7f43ec076720 2024-09-19T05:01:31.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.779+0000 7f43e3400640 1 -- 172.21.15.43:0/2175523218 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc004e00 con 0x7f43ec076720 2024-09-19T05:01:31.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.779+0000 7f43f1400640 1 -- 172.21.15.43:0/2175523218 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43ec19f340 con 0x7f43ec076720 2024-09-19T05:01:31.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.779+0000 7f43f1400640 1 -- 172.21.15.43:0/2175523218 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43ec1ab630 con 0x7f43ec076720 2024-09-19T05:01:31.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.780+0000 7f43f1400640 1 -- 172.21.15.43:0/2175523218 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43ec076b00 con 0x7f43ec076720 2024-09-19T05:01:31.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.782+0000 7f43e3400640 1 -- 172.21.15.43:0/2175523218 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f43dc004fa0 con 0x7f43ec076720 2024-09-19T05:01:31.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.784+0000 7f43e3400640 1 --2- 172.21.15.43:0/2175523218 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f43d00a32c0 0x7f43d00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:31.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.784+0000 7f43e3400640 1 -- 172.21.15.43:0/2175523218 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f43dc0cf2a0 con 0x7f43ec076720 2024-09-19T05:01:31.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.784+0000 7f43e9e00640 1 --2- 172.21.15.43:0/2175523218 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f43d00a32c0 0x7f43d00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:31.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.785+0000 7f43e9e00640 1 --2- 172.21.15.43:0/2175523218 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f43d00a32c0 0x7f43d00a5780 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f43d8000c00 tx=0x7f43d8005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:01:31.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.786+0000 7f43e3400640 1 -- 172.21.15.43:0/2175523218 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f43dc09bdf0 con 0x7f43ec076720 2024-09-19T05:01:31.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.975+0000 7f43f1400640 1 -- 172.21.15.43:0/2175523218 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f43ec19fe50 con 0x7f43ec076720 2024-09-19T05:01:31.977 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:31 smithi043 ceph-mon[30800]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:31.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.976+0000 7f43e3400640 1 -- 172.21.15.43:0/2175523218 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+18426 (secure 0 0 0) 0x7f43dc0a00f0 con 0x7f43ec076720 2024-09-19T05:01:31.978 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:01:31.979 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":77,"fsid":"1456a810-7643-11ef-bceb-c7b262605968","created":"2024-09-19T04:54:14.706644+0000","modified":"2024-09-19T05:01:28.888089+0000","last_up_change":"2024-09-19T05:01:27.348740+0000","last_in_change":"2024-09-19T05:01:03.396477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:57:27.642705+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"81a317a8-7ec8-45c3-bc10-7da023283635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6803","nonce":825351486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6805","nonce":825351486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6809","nonce":825351486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6807","nonce":825351486}]},"public_addr":"172.21.15.43:6803/825351486","cluster_addr":"172.21.15.43:6805/825351486","heartbeat_back_addr":"172.21.15.43:6809/825351486","heartbeat_front_addr":"172.21.15.43:6807/825351486","state":["exists","up"]},{"osd":1,"uuid":"e96fcfac-4fff-4a36-b38b-aebf57842740","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6801","nonce":2140095130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6803","nonce":2140095130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6807","nonce":2140095130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6805","nonce":2140095130}]},"public_addr":"172.21.15.96:6801/2140095130","cluster_addr":"172.21.15.96:6803/2140095130","heartbeat_back_addr":"172.21.15.96:6807/2140095130","heartbeat_front_addr":"172.21.15.96:6805/2140095130","state":["exists","up"]},{"osd":2,"uuid":"b659869e-114a-4f56-8aac-a2ed602b4058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6801","nonce":2047147996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6803","nonce":2047147996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6807","nonce":2047147996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6805","nonce":2047147996}]},"public_addr":"172.21.15.151:6801/2047147996","cluster_addr":"172.21.15.151:6803/2047147996","heartbeat_back_addr":"172.21.15.151:6807/2047147996","heartbeat_front_addr":"172.21.15.151:6805/2047147996","state":["exists","up"]},{"osd":3,"uuid":"7be64b2c-e45c-493d-a610-4ce41196061f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6811","nonce":1734673762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6813","nonce":1734673762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6817","nonce":1734673762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6815","nonce":1734673762}]},"public_addr":"172.21.15.43:6811/1734673762","cluster_addr":"172.21.15.43:6813/1734673762","heartbeat_back_addr":"172.21.15.43:6817/1734673762","heartbeat_front_addr":"172.21.15.43:6815/1734673762","state":["exists","up"]},{"osd":4,"uuid":"196d5af1-a6e5-496c-8933-4e58892f4360","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6809","nonce":3184785362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6811","nonce":3184785362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6815","nonce":3184785362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6813","nonce":3184785362}]},"public_addr":"172.21.15.96:6809/3184785362","cluster_addr":"172.21.15.96:6811/3184785362","heartbeat_back_addr":"172.21.15.96:6815/3184785362","heartbeat_front_addr":"172.21.15.96:6813/3184785362","state":["exists","up"]},{"osd":5,"uuid":"5492d474-8509-4865-bc2e-d08e31558a75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6809","nonce":2878451924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6811","nonce":2878451924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6815","nonce":2878451924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6813","nonce":2878451924}]},"public_addr":"172.21.15.151:6809/2878451924","cluster_addr":"172.21.15.151:6811/2878451924","heartbeat_back_addr":"172.21.15.151:6815/2878451924","heartbeat_front_addr":"172.21.15.151:6813/2878451924","state":["exists","up"]},{"osd":6,"uuid":"fb8457ed-7bb9-4435-9d15-c1cbfbddfcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6819","nonce":2008630290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6821","nonce":2008630290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6825","nonce":2008630290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6823","nonce":2008630290}]},"public_addr":"172.21.15.43:6819/2008630290","cluster_addr":"172.21.15.43:6821/2008630290","heartbeat_back_addr":"172.21.15.43:6825/2008630290","heartbeat_front_addr":"172.21.15.43:6823/2008630290","state":["exists","up"]},{"osd":7,"uuid":"02046db8-30de-4e92-b25a-1f07d6190e91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6817","nonce":3588713275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6819","nonce":3588713275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6823","nonce":3588713275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6821","nonce":3588713275}]},"public_addr":"172.21.15.96:6817/3588713275","cluster_addr":"172.21.15.96:6819/3588713275","heartbeat_back_addr":"172.21.15.96:6823/3588713275","heartbeat_front_addr":"172.21.15.96:6821/3588713275","state":["exists","up"]},{"osd":8,"uuid":"a5bb4218-51f3-4044-b43e-ebfc764f49ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6817","nonce":725316393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6819","nonce":725316393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6823","nonce":725316393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6821","nonce":725316393}]},"public_addr":"172.21.15.151:6817/725316393","cluster_addr":"172.21.15.151:6819/725316393","heartbeat_back_addr":"172.21.15.151:6823/725316393","heartbeat_front_addr":"172.21.15.151:6821/725316393","state":["exists","up"]},{"osd":9,"uuid":"6c19f65c-9750-4baa-9d63-7b8bd03e6955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6827","nonce":4053438621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6829","nonce":4053438621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6833","nonce":4053438621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6831","nonce":4053438621}]},"public_addr":"172.21.15.43:6827/4053438621","cluster_addr":"172.21.15.43:6829/4053438621","heartbeat_back_addr":"172.21.15.43:6833/4053438621","heartbeat_front_addr":"172.21.15.43:6831/4053438621","state":["exists","up"]},{"osd":10,"uuid":"a7aa7da3-8eab-47fe-a351-298b6ff75f14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6825","nonce":2403210086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6827","nonce":2403210086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6831","nonce":2403210086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6829","nonce":2403210086}]},"public_addr":"172.21.15.96:6825/2403210086","cluster_addr":"172.21.15.96:6827/2403210086","heartbeat_back_addr":"172.21.15.96:6831/2403210086","heartbeat_front_addr":"172.21.15.96:6829/2403210086","state":["exists","up"]},{"osd":11,"uuid":"760b6afc-253d-44dd-88e9-dcc97f67c9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6825","nonce":292202881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6827","nonce":292202881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6831","nonce":292202881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6829","nonce":292202881}]},"public_addr":"172.21.15.151:6825/292202881","cluster_addr":"172.21.15.151:6827/292202881","heartbeat_back_addr":"172.21.15.151:6831/292202881","heartbeat_front_addr":"172.21.15.151:6829/292202881","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:56:34.716991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:56:59.321174+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:57:24.281696+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:57:53.443880+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:58:19.205873+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:58:44.504873+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:13.650731+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:39.137876+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:04.579893+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:34.337312+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:59.434658+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:01:24.930529+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/898247657":"2024-09-20T04:55:07.037117+0000","172.21.15.43:6801/3241799377":"2024-09-20T04:55:07.037117+0000","172.21.15.43:6800/3241799377":"2024-09-20T04:55:07.037117+0000","172.21.15.43:0/3256350596":"2024-09-20T04:55:07.037117+0000","172.21.15.43:6801/587463881":"2024-09-20T04:54:39.548065+0000","172.21.15.43:6800/587463881":"2024-09-20T04:54:39.548065+0000","172.21.15.43:0/4271788315":"2024-09-20T04:55:07.037117+0000","172.21.15.43:0/6132665":"2024-09-20T04:54:39.548065+0000","172.21.15.43:0/632163478":"2024-09-20T04:54:39.548065+0000","172.21.15.43:0/330854185":"2024-09-20T04:54:39.548065+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:01:31.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.979+0000 7f43f1400640 1 -- 172.21.15.43:0/2175523218 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f43d00a32c0 msgr2=0x7f43d00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:31.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.979+0000 7f43f1400640 1 --2- 172.21.15.43:0/2175523218 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f43d00a32c0 0x7f43d00a5780 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f43d8000c00 tx=0x7f43d8005b50 comp rx=0 tx=0).stop 2024-09-19T05:01:31.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.979+0000 7f43f1400640 1 -- 172.21.15.43:0/2175523218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43ec076720 msgr2=0x7f43ec1a4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:31.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.979+0000 7f43f1400640 1 --2- 172.21.15.43:0/2175523218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43ec076720 0x7f43ec1a4760 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f43dc00d770 tx=0x7f43dc00dc30 comp rx=0 tx=0).stop 2024-09-19T05:01:31.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.979+0000 7f43f1400640 1 -- 172.21.15.43:0/2175523218 shutdown_connections 2024-09-19T05:01:31.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.979+0000 7f43f1400640 1 --2- 172.21.15.43:0/2175523218 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f43d00a32c0 0x7f43d00a5780 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:31.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.979+0000 7f43f1400640 1 --2- 172.21.15.43:0/2175523218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43ec112a30 0x7f43ec19e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:31.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.979+0000 7f43f1400640 1 --2- 172.21.15.43:0/2175523218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43ec077040 0x7f43ec1a4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:31.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.979+0000 7f43f1400640 1 --2- 172.21.15.43:0/2175523218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43ec076720 0x7f43ec1a4760 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:31.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.979+0000 7f43f1400640 1 -- 172.21.15.43:0/2175523218 >> 172.21.15.43:0/2175523218 conn(0x7f43ec0ffa90 msgr2=0x7f43ec101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:01:31.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.979+0000 7f43f1400640 1 -- 172.21.15.43:0/2175523218 shutdown_connections 2024-09-19T05:01:31.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:31.979+0000 7f43f1400640 1 -- 172.21.15.43:0/2175523218 wait complete. 2024-09-19T05:01:32.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:31 smithi096 ceph-mon[35085]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:32.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:31 smithi151 ceph-mon[35425]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:32.966 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T04:57:27.642705+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T05:01:32.967 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-19T05:01:33.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:32 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/2175523218' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:01:33.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:32 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/2175523218' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:01:33.341 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:01:33.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:32 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/2175523218' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:01:34.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.008+0000 7f920f800640 1 -- 172.21.15.43:0/3577285202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9208103ff0 msgr2=0x7f92081043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:34.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.008+0000 7f920f800640 1 --2- 172.21.15.43:0/3577285202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9208103ff0 0x7f92081043d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f91f8009900 tx=0x7f91f802a410 comp rx=0 tx=0).stop 2024-09-19T05:01:34.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.008+0000 7f920f800640 1 -- 172.21.15.43:0/3577285202 shutdown_connections 2024-09-19T05:01:34.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.008+0000 7f920f800640 1 --2- 172.21.15.43:0/3577285202 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9208109120 0x7f9208110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:34.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.008+0000 7f920f800640 1 --2- 172.21.15.43:0/3577285202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92081049a0 0x7f92081089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:34.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.008+0000 7f920f800640 1 --2- 172.21.15.43:0/3577285202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9208103ff0 0x7f92081043d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:34.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.008+0000 7f920f800640 1 -- 172.21.15.43:0/3577285202 >> 172.21.15.43:0/3577285202 conn(0x7f92080ffa90 msgr2=0x7f9208101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:01:34.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.008+0000 7f920f800640 1 -- 172.21.15.43:0/3577285202 shutdown_connections 2024-09-19T05:01:34.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.008+0000 7f920f800640 1 -- 172.21.15.43:0/3577285202 wait complete. 2024-09-19T05:01:34.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.009+0000 7f920f800640 1 Processor -- start 2024-09-19T05:01:34.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.009+0000 7f920f800640 1 -- start start 2024-09-19T05:01:34.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.009+0000 7f920f800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9208103ff0 0x7f920819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:34.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.009+0000 7f920f800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92081049a0 0x7f920819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:34.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.009+0000 7f920f800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9208109120 0x7f920819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:34.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.009+0000 7f920f800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9208114170 con 0x7f9208103ff0 2024-09-19T05:01:34.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.009+0000 7f920f800640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9208113ff0 con 0x7f92081049a0 2024-09-19T05:01:34.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.009+0000 7f920f800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f92081142f0 con 0x7f9208109120 2024-09-19T05:01:34.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.010+0000 7f920ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9208103ff0 0x7f920819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:34.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.010+0000 7f920ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9208103ff0 0x7f920819a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49614/0 (socket says 172.21.15.43:49614) 2024-09-19T05:01:34.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.010+0000 7f920ce00640 1 -- 172.21.15.43:0/2806019578 learned_addr learned my addr 172.21.15.43:0/2806019578 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:01:34.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.010+0000 7f91ffe00640 1 --2- 172.21.15.43:0/2806019578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92081049a0 0x7f920819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:34.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.010+0000 7f920d800640 1 --2- 172.21.15.43:0/2806019578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9208109120 0x7f920819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:34.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.010+0000 7f920ce00640 1 -- 172.21.15.43:0/2806019578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9208109120 msgr2=0x7f920819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:34.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.010+0000 7f920ce00640 1 --2- 172.21.15.43:0/2806019578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9208109120 0x7f920819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:34.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.010+0000 7f920ce00640 1 -- 172.21.15.43:0/2806019578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92081049a0 msgr2=0x7f920819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:34.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.010+0000 7f920ce00640 1 --2- 172.21.15.43:0/2806019578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92081049a0 0x7f920819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:34.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.010+0000 7f920ce00640 1 -- 172.21.15.43:0/2806019578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f920819f620 con 0x7f9208103ff0 2024-09-19T05:01:34.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.010+0000 7f91ffe00640 1 --2- 172.21.15.43:0/2806019578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92081049a0 0x7f920819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:01:34.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.011+0000 7f920ce00640 1 --2- 172.21.15.43:0/2806019578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9208103ff0 0x7f920819a670 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f91f8004770 tx=0x7f91f8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:01:34.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.011+0000 7f9206000640 1 -- 172.21.15.43:0/2806019578 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f803f070 con 0x7f9208103ff0 2024-09-19T05:01:34.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.011+0000 7f9206000640 1 -- 172.21.15.43:0/2806019578 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91f8031ca0 con 0x7f9208103ff0 2024-09-19T05:01:34.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.011+0000 7f9206000640 1 -- 172.21.15.43:0/2806019578 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f8035030 con 0x7f9208103ff0 2024-09-19T05:01:34.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.011+0000 7f920f800640 1 -- 172.21.15.43:0/2806019578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f920819f8b0 con 0x7f9208103ff0 2024-09-19T05:01:34.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.011+0000 7f920f800640 1 -- 172.21.15.43:0/2806019578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92081a71f0 con 0x7f9208103ff0 2024-09-19T05:01:34.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.012+0000 7f920f800640 1 -- 172.21.15.43:0/2806019578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92081043d0 con 0x7f9208103ff0 2024-09-19T05:01:34.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.014+0000 7f9206000640 1 -- 172.21.15.43:0/2806019578 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f91f8042070 con 0x7f9208103ff0 2024-09-19T05:01:34.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.015+0000 7f9206000640 1 --2- 172.21.15.43:0/2806019578 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f91ec0a32c0 0x7f91ec0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:34.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.016+0000 7f9206000640 1 -- 172.21.15.43:0/2806019578 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f91f80f0100 con 0x7f9208103ff0 2024-09-19T05:01:34.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.016+0000 7f91ffe00640 1 --2- 172.21.15.43:0/2806019578 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f91ec0a32c0 0x7f91ec0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:34.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.017+0000 7f91ffe00640 1 --2- 172.21.15.43:0/2806019578 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f91ec0a32c0 0x7f91ec0a5780 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f92080fff40 tx=0x7f91f000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:01:34.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.018+0000 7f9206000640 1 -- 172.21.15.43:0/2806019578 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f91f80bcbd0 con 0x7f9208103ff0 2024-09-19T05:01:34.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.203+0000 7f920f800640 1 -- 172.21.15.43:0/2806019578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f920810c300 con 0x7f9208103ff0 2024-09-19T05:01:34.205 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:33 smithi043 ceph-mon[30800]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:34.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.203+0000 7f9206000640 1 -- 172.21.15.43:0/2806019578 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7f91f80c0ed0 con 0x7f9208103ff0 2024-09-19T05:01:34.206 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-09-19T05:01:34.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.205+0000 7f920f800640 1 -- 172.21.15.43:0/2806019578 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f91ec0a32c0 msgr2=0x7f91ec0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:34.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.205+0000 7f920f800640 1 --2- 172.21.15.43:0/2806019578 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f91ec0a32c0 0x7f91ec0a5780 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f92080fff40 tx=0x7f91f000b410 comp rx=0 tx=0).stop 2024-09-19T05:01:34.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.205+0000 7f920f800640 1 -- 172.21.15.43:0/2806019578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9208103ff0 msgr2=0x7f920819a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:34.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.205+0000 7f920f800640 1 --2- 172.21.15.43:0/2806019578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9208103ff0 0x7f920819a670 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f91f8004770 tx=0x7f91f8003c10 comp rx=0 tx=0).stop 2024-09-19T05:01:34.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.205+0000 7f920f800640 1 -- 172.21.15.43:0/2806019578 shutdown_connections 2024-09-19T05:01:34.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.205+0000 7f920f800640 1 --2- 172.21.15.43:0/2806019578 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f91ec0a32c0 0x7f91ec0a5780 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:34.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.205+0000 7f920f800640 1 --2- 172.21.15.43:0/2806019578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9208109120 0x7f920819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:34.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.205+0000 7f920f800640 1 --2- 172.21.15.43:0/2806019578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92081049a0 0x7f920819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:34.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.205+0000 7f920f800640 1 --2- 172.21.15.43:0/2806019578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9208103ff0 0x7f920819a670 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:34.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.205+0000 7f920f800640 1 -- 172.21.15.43:0/2806019578 >> 172.21.15.43:0/2806019578 conn(0x7f92080ffa90 msgr2=0x7f9208100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:01:34.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.206+0000 7f920f800640 1 -- 172.21.15.43:0/2806019578 shutdown_connections 2024-09-19T05:01:34.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:34.206+0000 7f920f800640 1 -- 172.21.15.43:0/2806019578 wait complete. 2024-09-19T05:01:34.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:33 smithi096 ceph-mon[35085]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:34.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:33 smithi151 ceph-mon[35425]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:35.090 INFO:tasks.cephadm:Adding mds.a on smithi043 2024-09-19T05:01:35.090 INFO:tasks.cephadm:Adding mds.d on smithi043 2024-09-19T05:01:35.090 INFO:tasks.cephadm:Adding mds.g on smithi043 2024-09-19T05:01:35.090 INFO:tasks.cephadm:Adding mds.j on smithi043 2024-09-19T05:01:35.090 INFO:tasks.cephadm:Adding mds.b on smithi096 2024-09-19T05:01:35.091 INFO:tasks.cephadm:Adding mds.e on smithi096 2024-09-19T05:01:35.091 INFO:tasks.cephadm:Adding mds.h on smithi096 2024-09-19T05:01:35.091 INFO:tasks.cephadm:Adding mds.k on smithi096 2024-09-19T05:01:35.091 INFO:tasks.cephadm:Adding mds.c on smithi151 2024-09-19T05:01:35.091 INFO:tasks.cephadm:Adding mds.f on smithi151 2024-09-19T05:01:35.091 INFO:tasks.cephadm:Adding mds.i on smithi151 2024-09-19T05:01:35.091 INFO:tasks.cephadm:Adding mds.l on smithi151 2024-09-19T05:01:35.091 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph orch apply mds all '12;smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l' 2024-09-19T05:01:35.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:34 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/2806019578' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T05:01:35.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:34 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/2806019578' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T05:01:35.351 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:01:35.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:34 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/2806019578' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T05:01:35.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:35.833+0000 7fca6fe25640 1 -- 172.21.15.151:0/1852991579 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca68102df0 msgr2=0x7fca68103270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:35.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:35.833+0000 7fca6fe25640 1 --2- 172.21.15.151:0/1852991579 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca68102df0 0x7fca68103270 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fca64009900 tx=0x7fca6402a410 comp rx=0 tx=0).stop 2024-09-19T05:01:35.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:35.833+0000 7fca6fe25640 1 -- 172.21.15.151:0/1852991579 shutdown_connections 2024-09-19T05:01:35.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:35.833+0000 7fca6fe25640 1 --2- 172.21.15.151:0/1852991579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca681037b0 0x7fca6810a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:35.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:35.833+0000 7fca6fe25640 1 --2- 172.21.15.151:0/1852991579 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca68102df0 0x7fca68103270 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:35.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:35.833+0000 7fca6fe25640 1 --2- 172.21.15.151:0/1852991579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca68101bf0 0x7fca68101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:35.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:35.833+0000 7fca6fe25640 1 -- 172.21.15.151:0/1852991579 >> 172.21.15.151:0/1852991579 conn(0x7fca680fd3c0 msgr2=0x7fca680ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:01:35.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:35.833+0000 7fca6fe25640 1 -- 172.21.15.151:0/1852991579 shutdown_connections 2024-09-19T05:01:35.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:35.833+0000 7fca6fe25640 1 -- 172.21.15.151:0/1852991579 wait complete. 2024-09-19T05:01:35.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:35.833+0000 7fca6fe25640 1 Processor -- start 2024-09-19T05:01:35.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:35.833+0000 7fca6fe25640 1 -- start start 2024-09-19T05:01:35.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:35.834+0000 7fca6fe25640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca68101bf0 0x7fca68075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:35.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:35.834+0000 7fca6fe25640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca68102df0 0x7fca680745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:35.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:35.834+0000 7fca6fe25640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca681037b0 0x7fca68074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:35.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:35.834+0000 7fca6fe25640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fca6810d8f0 con 0x7fca68102df0 2024-09-19T05:01:35.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:35.834+0000 7fca6fe25640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fca6810d770 con 0x7fca68101bf0 2024-09-19T05:01:35.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:35.834+0000 7fca6fe25640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fca6810da70 con 0x7fca681037b0 2024-09-19T05:01:35.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:35.834+0000 7fca6e3c2640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca681037b0 0x7fca68074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:35.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:35.834+0000 7fca6dbc1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca68101bf0 0x7fca68075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:35.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:35.834+0000 7fca6dbc1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca68101bf0 0x7fca68075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.151:36706/0 (socket says 172.21.15.151:36706) 2024-09-19T05:01:35.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:35.834+0000 7fca6dbc1640 1 -- 172.21.15.151:0/403616009 learned_addr learned my addr 172.21.15.151:0/403616009 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-19T05:01:35.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:35.834+0000 7fca6d3c0640 1 --2- 172.21.15.151:0/403616009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca68102df0 0x7fca680745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:35.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:35.834+0000 7fca6dbc1640 1 -- 172.21.15.151:0/403616009 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca681037b0 msgr2=0x7fca68074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:35.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:35.834+0000 7fca6dbc1640 1 --2- 172.21.15.151:0/403616009 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca681037b0 0x7fca68074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:35.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:35.834+0000 7fca6dbc1640 1 -- 172.21.15.151:0/403616009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca68102df0 msgr2=0x7fca680745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:35.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:35.834+0000 7fca6dbc1640 1 --2- 172.21.15.151:0/403616009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca68102df0 0x7fca680745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:35.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:35.834+0000 7fca6dbc1640 1 -- 172.21.15.151:0/403616009 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca680753e0 con 0x7fca68101bf0 2024-09-19T05:01:35.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:35.834+0000 7fca6e3c2640 1 --2- 172.21.15.151:0/403616009 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca681037b0 0x7fca68074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:01:35.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:35.834+0000 7fca6d3c0640 1 --2- 172.21.15.151:0/403616009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca68102df0 0x7fca680745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:01:35.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:35.834+0000 7fca6dbc1640 1 --2- 172.21.15.151:0/403616009 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca68101bf0 0x7fca68075f90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fca5800d950 tx=0x7fca5800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:01:35.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:35.834+0000 7fca5effd640 1 -- 172.21.15.151:0/403616009 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca58017070 con 0x7fca68101bf0 2024-09-19T05:01:35.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:35.835+0000 7fca6fe25640 1 -- 172.21.15.151:0/403616009 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca681a20d0 con 0x7fca68101bf0 2024-09-19T05:01:35.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:35.835+0000 7fca5effd640 1 -- 172.21.15.151:0/403616009 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca58009c30 con 0x7fca68101bf0 2024-09-19T05:01:35.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:35.835+0000 7fca5effd640 1 -- 172.21.15.151:0/403616009 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5800f600 con 0x7fca68101bf0 2024-09-19T05:01:35.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:35.835+0000 7fca6fe25640 1 -- 172.21.15.151:0/403616009 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca681a2660 con 0x7fca68101bf0 2024-09-19T05:01:35.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:35.835+0000 7fca6fe25640 1 -- 172.21.15.151:0/403616009 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca68108fa0 con 0x7fca68101bf0 2024-09-19T05:01:35.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:35.837+0000 7fca5effd640 1 -- 172.21.15.151:0/403616009 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fca5801a070 con 0x7fca68101bf0 2024-09-19T05:01:35.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:35.838+0000 7fca5effd640 1 --2- 172.21.15.151:0/403616009 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fca540a3240 0x7fca540a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:35.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:35.838+0000 7fca5effd640 1 -- 172.21.15.151:0/403616009 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fca580cef40 con 0x7fca68101bf0 2024-09-19T05:01:35.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:35.838+0000 7fca6d3c0640 1 --2- 172.21.15.151:0/403616009 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fca540a3240 0x7fca540a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:35.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:35.838+0000 7fca6d3c0640 1 --2- 172.21.15.151:0/403616009 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fca540a3240 0x7fca540a5700 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fca64000c00 tx=0x7fca640095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:01:35.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:35.841+0000 7fca5effd640 1 -- 172.21.15.151:0/403616009 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fca58009de0 con 0x7fca68101bf0 2024-09-19T05:01:36.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:36.024+0000 7fca6fe25640 1 -- 172.21.15.151:0/403616009 --> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l", "target": ["mon-mgr", ""]}) -- 0x7fca680627f0 con 0x7fca540a3240 2024-09-19T05:01:36.026 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:35 smithi151 ceph-mon[35425]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:36.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:36.032+0000 7fca5effd640 1 -- 172.21.15.151:0/403616009 <== mgr.14150 v2:172.21.15.43:6800/3167108925 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fca680627f0 con 0x7fca540a3240 2024-09-19T05:01:36.034 INFO:teuthology.orchestra.run.smithi151.stdout:Scheduled mds.all update... 2024-09-19T05:01:36.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:36.034+0000 7fca6fe25640 1 -- 172.21.15.151:0/403616009 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fca540a3240 msgr2=0x7fca540a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:36.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:36.034+0000 7fca6fe25640 1 --2- 172.21.15.151:0/403616009 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fca540a3240 0x7fca540a5700 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fca64000c00 tx=0x7fca640095f0 comp rx=0 tx=0).stop 2024-09-19T05:01:36.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:36.034+0000 7fca6fe25640 1 -- 172.21.15.151:0/403616009 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca68101bf0 msgr2=0x7fca68075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:36.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:36.034+0000 7fca6fe25640 1 --2- 172.21.15.151:0/403616009 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca68101bf0 0x7fca68075f90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fca5800d950 tx=0x7fca5800de10 comp rx=0 tx=0).stop 2024-09-19T05:01:36.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:36.034+0000 7fca6fe25640 1 -- 172.21.15.151:0/403616009 shutdown_connections 2024-09-19T05:01:36.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:36.034+0000 7fca6fe25640 1 --2- 172.21.15.151:0/403616009 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fca540a3240 0x7fca540a5700 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:36.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:36.034+0000 7fca6fe25640 1 --2- 172.21.15.151:0/403616009 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca681037b0 0x7fca68074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:36.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:36.034+0000 7fca6fe25640 1 --2- 172.21.15.151:0/403616009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca68102df0 0x7fca680745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:36.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:36.034+0000 7fca6fe25640 1 --2- 172.21.15.151:0/403616009 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca68101bf0 0x7fca68075f90 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:36.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:36.034+0000 7fca6fe25640 1 -- 172.21.15.151:0/403616009 >> 172.21.15.151:0/403616009 conn(0x7fca680fd3c0 msgr2=0x7fca68106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:01:36.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:36.034+0000 7fca6fe25640 1 -- 172.21.15.151:0/403616009 shutdown_connections 2024-09-19T05:01:36.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T05:01:36.034+0000 7fca6fe25640 1 -- 172.21.15.151:0/403616009 wait complete. 2024-09-19T05:01:36.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:35 smithi096 ceph-mon[35085]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:36.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:35 smithi043 ceph-mon[30800]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:36.925 DEBUG:teuthology.orchestra.run.smithi043:mds.a> sudo journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@mds.a.service 2024-09-19T05:01:36.929 DEBUG:teuthology.orchestra.run.smithi043:mds.d> sudo journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@mds.d.service 2024-09-19T05:01:36.974 DEBUG:teuthology.orchestra.run.smithi043:mds.g> sudo journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@mds.g.service 2024-09-19T05:01:37.019 DEBUG:teuthology.orchestra.run.smithi043:mds.j> sudo journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@mds.j.service 2024-09-19T05:01:37.064 DEBUG:teuthology.orchestra.run.smithi096:mds.b> sudo journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@mds.b.service 2024-09-19T05:01:37.067 DEBUG:teuthology.orchestra.run.smithi096:mds.e> sudo journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@mds.e.service 2024-09-19T05:01:37.110 DEBUG:teuthology.orchestra.run.smithi096:mds.h> sudo journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@mds.h.service 2024-09-19T05:01:37.154 DEBUG:teuthology.orchestra.run.smithi096:mds.k> sudo journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@mds.k.service 2024-09-19T05:01:37.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:37 smithi151 ceph-mon[35425]: from='client.24437 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:01:37.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:37 smithi151 ceph-mon[35425]: Saving service mds.all spec with placement smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l;count:12 2024-09-19T05:01:37.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:37 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:37.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:37 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:01:37.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:37 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:37.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:37 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:01:37.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:37 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:37.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:37 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:01:37.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:37 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:01:37.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:37 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:37.197 DEBUG:teuthology.orchestra.run.smithi151:mds.c> sudo journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@mds.c.service 2024-09-19T05:01:37.199 DEBUG:teuthology.orchestra.run.smithi151:mds.f> sudo journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@mds.f.service 2024-09-19T05:01:37.242 DEBUG:teuthology.orchestra.run.smithi151:mds.i> sudo journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@mds.i.service 2024-09-19T05:01:37.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:37 smithi043 ceph-mon[30800]: from='client.24437 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:01:37.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:37 smithi043 ceph-mon[30800]: Saving service mds.all spec with placement smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l;count:12 2024-09-19T05:01:37.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:37 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:37.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:37 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:01:37.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:37 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:37.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:37 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:01:37.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:37 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:37.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:37 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:01:37.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:37 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:01:37.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:37 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:37.285 DEBUG:teuthology.orchestra.run.smithi151:mds.l> sudo journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@mds.l.service 2024-09-19T05:01:37.329 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-19T05:01:37.329 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:01:37.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:37 smithi096 ceph-mon[35085]: from='client.24437 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:01:37.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:37 smithi096 ceph-mon[35085]: Saving service mds.all spec with placement smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l;count:12 2024-09-19T05:01:37.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:37 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:37.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:37 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:01:37.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:37 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:37.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:37 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:01:37.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:37 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:37.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:37 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:01:37.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:37 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:01:37.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:37 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:37.700 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:01:38.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:38 smithi043 ceph-mon[30800]: Deploying daemon mds.l on smithi151 2024-09-19T05:01:38.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:38 smithi043 ceph-mon[30800]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:38.242 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:38 smithi151 ceph-mon[35425]: Deploying daemon mds.l on smithi151 2024-09-19T05:01:38.242 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:38 smithi151 ceph-mon[35425]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:38.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.261+0000 7f10cd400640 1 -- 172.21.15.43:0/3076097756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10c8100a80 msgr2=0x7f10c8100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:38.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.261+0000 7f10cd400640 1 --2- 172.21.15.43:0/3076097756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10c8100a80 0x7f10c8100ee0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f10b0009930 tx=0x7f10b002a420 comp rx=0 tx=0).stop 2024-09-19T05:01:38.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.261+0000 7f10cd400640 1 -- 172.21.15.43:0/3076097756 shutdown_connections 2024-09-19T05:01:38.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.261+0000 7f10cd400640 1 --2- 172.21.15.43:0/3076097756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10c8101420 0x7f10c810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:38.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.261+0000 7f10cd400640 1 --2- 172.21.15.43:0/3076097756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10c8100a80 0x7f10c8100ee0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:38.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.261+0000 7f10cd400640 1 --2- 172.21.15.43:0/3076097756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10c8106a80 0x7f10c8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:38.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.261+0000 7f10cd400640 1 -- 172.21.15.43:0/3076097756 >> 172.21.15.43:0/3076097756 conn(0x7f10c80fc7a0 msgr2=0x7f10c80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:01:38.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.261+0000 7f10cd400640 1 -- 172.21.15.43:0/3076097756 shutdown_connections 2024-09-19T05:01:38.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.261+0000 7f10cd400640 1 -- 172.21.15.43:0/3076097756 wait complete. 2024-09-19T05:01:38.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.262+0000 7f10cd400640 1 Processor -- start 2024-09-19T05:01:38.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.262+0000 7f10cd400640 1 -- start start 2024-09-19T05:01:38.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.262+0000 7f10cd400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10c8100a80 0x7f10c80fce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:38.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.262+0000 7f10cd400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10c8101420 0x7f10c80fd370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:38.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.262+0000 7f10cd400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10c8106a80 0x7f10c80fd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:38.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.262+0000 7f10cd400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f10c8110e80 con 0x7f10c8106a80 2024-09-19T05:01:38.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.262+0000 7f10cd400640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f10c8110d00 con 0x7f10c8101420 2024-09-19T05:01:38.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.262+0000 7f10cd400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f10c8111000 con 0x7f10c8100a80 2024-09-19T05:01:38.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.263+0000 7f10c7200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10c8106a80 0x7f10c80fd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:38.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.263+0000 7f10c6800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10c8100a80 0x7f10c80fce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:38.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.263+0000 7f10bde00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10c8101420 0x7f10c80fd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:38.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.263+0000 7f10c7200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10c8106a80 0x7f10c80fd8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37730/0 (socket says 172.21.15.43:37730) 2024-09-19T05:01:38.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.263+0000 7f10c6800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10c8100a80 0x7f10c80fce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:56864/0 (socket says 172.21.15.43:56864) 2024-09-19T05:01:38.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.263+0000 7f10bde00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10c8101420 0x7f10c80fd370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:42436/0 (socket says 172.21.15.43:42436) 2024-09-19T05:01:38.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.263+0000 7f10c6800640 1 -- 172.21.15.43:0/3632938627 learned_addr learned my addr 172.21.15.43:0/3632938627 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:01:38.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.263+0000 7f10c7200640 1 -- 172.21.15.43:0/3632938627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10c8100a80 msgr2=0x7f10c80fce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:38.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.263+0000 7f10c7200640 1 --2- 172.21.15.43:0/3632938627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10c8100a80 0x7f10c80fce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:38.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.263+0000 7f10c7200640 1 -- 172.21.15.43:0/3632938627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10c8101420 msgr2=0x7f10c80fd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:38.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.263+0000 7f10c7200640 1 --2- 172.21.15.43:0/3632938627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10c8101420 0x7f10c80fd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:38.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.263+0000 7f10c7200640 1 -- 172.21.15.43:0/3632938627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c81a68d0 con 0x7f10c8106a80 2024-09-19T05:01:38.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.264+0000 7f10c7200640 1 --2- 172.21.15.43:0/3632938627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10c8106a80 0x7f10c80fd8b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f10c000e940 tx=0x7f10c000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:01:38.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.264+0000 7f10bfe00640 1 -- 172.21.15.43:0/3632938627 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c0004d20 con 0x7f10c8106a80 2024-09-19T05:01:38.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.264+0000 7f10bfe00640 1 -- 172.21.15.43:0/3632938627 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10c0005020 con 0x7f10c8106a80 2024-09-19T05:01:38.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.264+0000 7f10bfe00640 1 -- 172.21.15.43:0/3632938627 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c00116b0 con 0x7f10c8106a80 2024-09-19T05:01:38.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.264+0000 7f10cd400640 1 -- 172.21.15.43:0/3632938627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10c81a6bc0 con 0x7f10c8106a80 2024-09-19T05:01:38.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.264+0000 7f10cd400640 1 -- 172.21.15.43:0/3632938627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10c81a7080 con 0x7f10c8106a80 2024-09-19T05:01:38.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.265+0000 7f10cd400640 1 -- 172.21.15.43:0/3632938627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10c8100ee0 con 0x7f10c8106a80 2024-09-19T05:01:38.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.267+0000 7f10bfe00640 1 -- 172.21.15.43:0/3632938627 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f10c00054d0 con 0x7f10c8106a80 2024-09-19T05:01:38.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.268+0000 7f10bfe00640 1 --2- 172.21.15.43:0/3632938627 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f10ac0a32c0 0x7f10ac0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:38.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.268+0000 7f10bfe00640 1 -- 172.21.15.43:0/3632938627 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f10c00cf4f0 con 0x7f10c8106a80 2024-09-19T05:01:38.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.269+0000 7f10c6800640 1 --2- 172.21.15.43:0/3632938627 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f10ac0a32c0 0x7f10ac0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:38.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.269+0000 7f10c6800640 1 --2- 172.21.15.43:0/3632938627 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f10ac0a32c0 0x7f10ac0a5780 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f10b8009c10 tx=0x7f10b8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:01:38.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.273+0000 7f10bfe00640 1 -- 172.21.15.43:0/3632938627 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f10c0016030 con 0x7f10c8106a80 2024-09-19T05:01:38.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:38 smithi096 ceph-mon[35085]: Deploying daemon mds.l on smithi151 2024-09-19T05:01:38.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:38 smithi096 ceph-mon[35085]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:38.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.500+0000 7f10cd400640 1 -- 172.21.15.43:0/3632938627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10c8109a70 con 0x7f10c8106a80 2024-09-19T05:01:38.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.500+0000 7f10bfe00640 1 -- 172.21.15.43:0/3632938627 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f10c009bfc0 con 0x7f10c8106a80 2024-09-19T05:01:38.502 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:01:38.503 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":1,"btime":"2024-09-19T04:54:14:706422+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-19T05:01:38.503 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 1 2024-09-19T05:01:38.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.503+0000 7f10cd400640 1 -- 172.21.15.43:0/3632938627 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f10ac0a32c0 msgr2=0x7f10ac0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:38.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.503+0000 7f10cd400640 1 --2- 172.21.15.43:0/3632938627 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f10ac0a32c0 0x7f10ac0a5780 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f10b8009c10 tx=0x7f10b8005f90 comp rx=0 tx=0).stop 2024-09-19T05:01:38.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.503+0000 7f10cd400640 1 -- 172.21.15.43:0/3632938627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10c8106a80 msgr2=0x7f10c80fd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:38.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.503+0000 7f10cd400640 1 --2- 172.21.15.43:0/3632938627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10c8106a80 0x7f10c80fd8b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f10c000e940 tx=0x7f10c000ee00 comp rx=0 tx=0).stop 2024-09-19T05:01:38.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.503+0000 7f10cd400640 1 -- 172.21.15.43:0/3632938627 shutdown_connections 2024-09-19T05:01:38.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.503+0000 7f10cd400640 1 --2- 172.21.15.43:0/3632938627 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f10ac0a32c0 0x7f10ac0a5780 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:38.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.503+0000 7f10cd400640 1 --2- 172.21.15.43:0/3632938627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10c8106a80 0x7f10c80fd8b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:38.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.503+0000 7f10cd400640 1 --2- 172.21.15.43:0/3632938627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10c8101420 0x7f10c80fd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:38.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.503+0000 7f10cd400640 1 --2- 172.21.15.43:0/3632938627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10c8100a80 0x7f10c80fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:38.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.503+0000 7f10cd400640 1 -- 172.21.15.43:0/3632938627 >> 172.21.15.43:0/3632938627 conn(0x7f10c80fc7a0 msgr2=0x7f10c806aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:01:38.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.503+0000 7f10cd400640 1 -- 172.21.15.43:0/3632938627 shutdown_connections 2024-09-19T05:01:38.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:38.503+0000 7f10cd400640 1 -- 172.21.15.43:0/3632938627 wait complete. 2024-09-19T05:01:38.905 INFO:journalctl@ceph.mds.l.smithi151.stdout:Sep 19 05:01:38 smithi151 systemd[1]: Starting Ceph mds.l for 1456a810-7643-11ef-bceb-c7b262605968... 2024-09-19T05:01:39.205 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:39 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/3632938627' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:01:39.205 INFO:journalctl@ceph.mds.l.smithi151.stdout:Sep 19 05:01:38 smithi151 podman[64855]: 2024-09-19 05:01:38.839325765 +0000 UTC m=+0.015809099 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:01:39.206 INFO:journalctl@ceph.mds.l.smithi151.stdout:Sep 19 05:01:38 smithi151 podman[64855]: 2024-09-19 05:01:38.951939001 +0000 UTC m=+0.128422324 container create f4341073aa7ac136530694e85b28ad8df89b0ea7d5cf7080676490e82081d478 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-l, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:01:39.206 INFO:journalctl@ceph.mds.l.smithi151.stdout:Sep 19 05:01:39 smithi151 podman[64855]: 2024-09-19 05:01:39.204511986 +0000 UTC m=+0.380995309 container init f4341073aa7ac136530694e85b28ad8df89b0ea7d5cf7080676490e82081d478 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-l, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T05:01:39.256 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:39 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/3632938627' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:01:39.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:39 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/3632938627' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:01:39.655 INFO:journalctl@ceph.mds.l.smithi151.stdout:Sep 19 05:01:39 smithi151 podman[64855]: 2024-09-19 05:01:39.207364367 +0000 UTC m=+0.383847700 container start f4341073aa7ac136530694e85b28ad8df89b0ea7d5cf7080676490e82081d478 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-l, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-19T05:01:39.655 INFO:journalctl@ceph.mds.l.smithi151.stdout:Sep 19 05:01:39 smithi151 ceph-mds[64874]: -- 172.21.15.151:0/3813423585 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55edb6c4f860 con 0x55edb6c2d000 2024-09-19T05:01:39.655 INFO:journalctl@ceph.mds.l.smithi151.stdout:Sep 19 05:01:39 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-l[64870]: starting mds.l at 2024-09-19T05:01:39.655 INFO:journalctl@ceph.mds.l.smithi151.stdout:Sep 19 05:01:39 smithi151 bash[64855]: f4341073aa7ac136530694e85b28ad8df89b0ea7d5cf7080676490e82081d478 2024-09-19T05:01:39.656 INFO:journalctl@ceph.mds.l.smithi151.stdout:Sep 19 05:01:39 smithi151 systemd[1]: Started Ceph mds.l for 1456a810-7643-11ef-bceb-c7b262605968. 2024-09-19T05:01:40.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:40 smithi151 ceph-mon[35425]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:40.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:40 smithi151 ceph-mon[35425]: mds.? [v2:172.21.15.151:6832/2836453158,v1:172.21.15.151:6833/2836453158] up:boot 2024-09-19T05:01:40.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:40 smithi151 ceph-mon[35425]: fsmap 1 up:standby 2024-09-19T05:01:40.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:40 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T05:01:40.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:40 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:40.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:40 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:40.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:40 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:40.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:40 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:01:40.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:40 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:01:40.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:40 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:40.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:40 smithi096 ceph-mon[35085]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:40.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:40 smithi096 ceph-mon[35085]: mds.? [v2:172.21.15.151:6832/2836453158,v1:172.21.15.151:6833/2836453158] up:boot 2024-09-19T05:01:40.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:40 smithi096 ceph-mon[35085]: fsmap 1 up:standby 2024-09-19T05:01:40.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:40 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T05:01:40.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:40 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:40.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:40 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:40.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:40 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:40.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:40 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:01:40.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:40 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:01:40.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:40 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:40.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:40 smithi043 ceph-mon[30800]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:40.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:40 smithi043 ceph-mon[30800]: mds.? [v2:172.21.15.151:6832/2836453158,v1:172.21.15.151:6833/2836453158] up:boot 2024-09-19T05:01:40.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:40 smithi043 ceph-mon[30800]: fsmap 1 up:standby 2024-09-19T05:01:40.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:40 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T05:01:40.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:40 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:40.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:40 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:40.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:40 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:40.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:40 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:01:40.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:40 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:01:40.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:40 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:41.295 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:01:41.327 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:41 smithi096 ceph-mon[35085]: Deploying daemon mds.k on smithi096 2024-09-19T05:01:41.350 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:41 smithi043 ceph-mon[30800]: Deploying daemon mds.k on smithi096 2024-09-19T05:01:41.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:41 smithi151 ceph-mon[35425]: Deploying daemon mds.k on smithi096 2024-09-19T05:01:41.599 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 19 05:01:41 smithi096 podman[64236]: 2024-09-19 05:01:41.441209577 +0000 UTC m=+0.015632944 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:01:41.599 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 19 05:01:41 smithi096 podman[64236]: 2024-09-19 05:01:41.598008126 +0000 UTC m=+0.172431484 container create 843ac97aa1db88033d069b41a1d43f3dc11f82d507562afc9dd5c431e03918d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-19T05:01:41.682 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:01:42.117 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:42 smithi043 ceph-mon[30800]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:42.117 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:42 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:42.118 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:42 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:42.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:42 smithi096 ceph-mon[35085]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:42.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:42 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:42.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:42 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:42.224 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 19 05:01:41 smithi096 podman[64236]: 2024-09-19 05:01:41.896483403 +0000 UTC m=+0.470906760 container init 843ac97aa1db88033d069b41a1d43f3dc11f82d507562afc9dd5c431e03918d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-k, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-19T05:01:42.224 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 19 05:01:41 smithi096 podman[64236]: 2024-09-19 05:01:41.899561157 +0000 UTC m=+0.473984525 container start 843ac97aa1db88033d069b41a1d43f3dc11f82d507562afc9dd5c431e03918d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-19T05:01:42.224 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 19 05:01:41 smithi096 ceph-mds[64256]: -- 172.21.15.96:0/3202274723 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ddcaa89c20 con 0x55ddcaa67400 2024-09-19T05:01:42.224 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 19 05:01:41 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-k[64252]: starting mds.k at 2024-09-19T05:01:42.224 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 19 05:01:41 smithi096 bash[64236]: 843ac97aa1db88033d069b41a1d43f3dc11f82d507562afc9dd5c431e03918d9 2024-09-19T05:01:42.224 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 19 05:01:41 smithi096 systemd[1]: Started Ceph mds.k for 1456a810-7643-11ef-bceb-c7b262605968. 2024-09-19T05:01:42.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.238+0000 7f8801000640 1 -- 172.21.15.43:0/1553893872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87fc1019c0 msgr2=0x7f87fc101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:42.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.238+0000 7f8801000640 1 --2- 172.21.15.43:0/1553893872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87fc1019c0 0x7f87fc101e20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f87e8009930 tx=0x7f87e802a440 comp rx=0 tx=0).stop 2024-09-19T05:01:42.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.238+0000 7f8801000640 1 -- 172.21.15.43:0/1553893872 shutdown_connections 2024-09-19T05:01:42.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.238+0000 7f8801000640 1 --2- 172.21.15.43:0/1553893872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87fc102360 0x7f87fc10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:42.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.238+0000 7f8801000640 1 --2- 172.21.15.43:0/1553893872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87fc1019c0 0x7f87fc101e20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:42.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.238+0000 7f8801000640 1 --2- 172.21.15.43:0/1553893872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87fc1079c0 0x7f87fc107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:42.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.238+0000 7f8801000640 1 -- 172.21.15.43:0/1553893872 >> 172.21.15.43:0/1553893872 conn(0x7f87fc0fd740 msgr2=0x7f87fc0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:01:42.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.238+0000 7f8801000640 1 -- 172.21.15.43:0/1553893872 shutdown_connections 2024-09-19T05:01:42.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.238+0000 7f8801000640 1 -- 172.21.15.43:0/1553893872 wait complete. 2024-09-19T05:01:42.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.239+0000 7f8801000640 1 Processor -- start 2024-09-19T05:01:42.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.239+0000 7f8801000640 1 -- start start 2024-09-19T05:01:42.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.240+0000 7f8801000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87fc1019c0 0x7f87fc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:42.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.240+0000 7f8801000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87fc102360 0x7f87fc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:42.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.240+0000 7f8801000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87fc1079c0 0x7f87fc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:42.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.240+0000 7f8801000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc111dd0 con 0x7f87fc1019c0 2024-09-19T05:01:42.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.240+0000 7f8801000640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc111c50 con 0x7f87fc102360 2024-09-19T05:01:42.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.240+0000 7f8801000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc111f50 con 0x7f87fc1079c0 2024-09-19T05:01:42.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.240+0000 7f87faa00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87fc102360 0x7f87fc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:42.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.240+0000 7f87fb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87fc1019c0 0x7f87fc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:42.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.240+0000 7f87faa00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87fc102360 0x7f87fc074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:42456/0 (socket says 172.21.15.43:42456) 2024-09-19T05:01:42.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.240+0000 7f87faa00640 1 -- 172.21.15.43:0/3168328427 learned_addr learned my addr 172.21.15.43:0/3168328427 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:01:42.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.240+0000 7f87fbe00640 1 --2- 172.21.15.43:0/3168328427 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87fc1079c0 0x7f87fc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:42.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.241+0000 7f87fb400640 1 -- 172.21.15.43:0/3168328427 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87fc1079c0 msgr2=0x7f87fc074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:42.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.241+0000 7f87fb400640 1 --2- 172.21.15.43:0/3168328427 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87fc1079c0 0x7f87fc074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:42.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.241+0000 7f87fb400640 1 -- 172.21.15.43:0/3168328427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87fc102360 msgr2=0x7f87fc074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:42.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.241+0000 7f87fb400640 1 --2- 172.21.15.43:0/3168328427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87fc102360 0x7f87fc074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:42.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.241+0000 7f87fb400640 1 -- 172.21.15.43:0/3168328427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87fc1a6790 con 0x7f87fc1019c0 2024-09-19T05:01:42.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.241+0000 7f87faa00640 1 --2- 172.21.15.43:0/3168328427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87fc102360 0x7f87fc074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:01:42.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.241+0000 7f87fb400640 1 --2- 172.21.15.43:0/3168328427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87fc1019c0 0x7f87fc0744f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f87ec00e9f0 tx=0x7f87ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:01:42.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.241+0000 7f87f3e00640 1 -- 172.21.15.43:0/3168328427 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ec00ccc0 con 0x7f87fc1019c0 2024-09-19T05:01:42.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.242+0000 7f87f3e00640 1 -- 172.21.15.43:0/3168328427 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87ec010910 con 0x7f87fc1019c0 2024-09-19T05:01:42.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.242+0000 7f8801000640 1 -- 172.21.15.43:0/3168328427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87fc1a6a80 con 0x7f87fc1019c0 2024-09-19T05:01:42.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.242+0000 7f87f3e00640 1 -- 172.21.15.43:0/3168328427 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ec018960 con 0x7f87fc1019c0 2024-09-19T05:01:42.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.242+0000 7f8801000640 1 -- 172.21.15.43:0/3168328427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87fc1a6fc0 con 0x7f87fc1019c0 2024-09-19T05:01:42.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.244+0000 7f8801000640 1 -- 172.21.15.43:0/3168328427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87fc101e20 con 0x7f87fc1019c0 2024-09-19T05:01:42.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.245+0000 7f87f3e00640 1 -- 172.21.15.43:0/3168328427 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f87ec010ac0 con 0x7f87fc1019c0 2024-09-19T05:01:42.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.247+0000 7f87f3e00640 1 --2- 172.21.15.43:0/3168328427 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f87e00a3240 0x7f87e00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:42.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.248+0000 7f87f3e00640 1 -- 172.21.15.43:0/3168328427 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f87ec0cf9d0 con 0x7f87fc1019c0 2024-09-19T05:01:42.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.248+0000 7f87faa00640 1 --2- 172.21.15.43:0/3168328427 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f87e00a3240 0x7f87e00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:42.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.249+0000 7f87faa00640 1 --2- 172.21.15.43:0/3168328427 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f87e00a3240 0x7f87e00a5700 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f87fc075f80 tx=0x7f87e8031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:01:42.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.252+0000 7f87f3e00640 1 -- 172.21.15.43:0/3168328427 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f87ec054210 con 0x7f87fc1019c0 2024-09-19T05:01:42.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:42 smithi151 ceph-mon[35425]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:42.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:42 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:42.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:42 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:42.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.476+0000 7f8801000640 1 -- 172.21.15.43:0/3168328427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87fc109f80 con 0x7f87fc1019c0 2024-09-19T05:01:42.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.477+0000 7f87f3e00640 1 -- 172.21.15.43:0/3168328427 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2260 (secure 0 0 0) 0x7f87ec09caf0 con 0x7f87fc1019c0 2024-09-19T05:01:42.479 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:01:42.479 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":3,"btime":"2024-09-19T05:01:42:047339+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/2836453158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2836453158},{"type":"v1","addr":"172.21.15.151:6833","nonce":2836453158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/178785812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":178785812},{"type":"v1","addr":"172.21.15.96:6833","nonce":178785812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-19T05:01:42.479 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 3 2024-09-19T05:01:42.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.480+0000 7f8801000640 1 -- 172.21.15.43:0/3168328427 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f87e00a3240 msgr2=0x7f87e00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:42.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.480+0000 7f8801000640 1 --2- 172.21.15.43:0/3168328427 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f87e00a3240 0x7f87e00a5700 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f87fc075f80 tx=0x7f87e8031510 comp rx=0 tx=0).stop 2024-09-19T05:01:42.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.480+0000 7f8801000640 1 -- 172.21.15.43:0/3168328427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87fc1019c0 msgr2=0x7f87fc0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:42.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.480+0000 7f8801000640 1 --2- 172.21.15.43:0/3168328427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87fc1019c0 0x7f87fc0744f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f87ec00e9f0 tx=0x7f87ec00eeb0 comp rx=0 tx=0).stop 2024-09-19T05:01:42.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.480+0000 7f8801000640 1 -- 172.21.15.43:0/3168328427 shutdown_connections 2024-09-19T05:01:42.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.480+0000 7f8801000640 1 --2- 172.21.15.43:0/3168328427 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f87e00a3240 0x7f87e00a5700 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:42.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.480+0000 7f8801000640 1 --2- 172.21.15.43:0/3168328427 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87fc1079c0 0x7f87fc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:42.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.480+0000 7f8801000640 1 --2- 172.21.15.43:0/3168328427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87fc102360 0x7f87fc074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:42.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.480+0000 7f8801000640 1 --2- 172.21.15.43:0/3168328427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87fc1019c0 0x7f87fc0744f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:42.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.480+0000 7f8801000640 1 -- 172.21.15.43:0/3168328427 >> 172.21.15.43:0/3168328427 conn(0x7f87fc0fd740 msgr2=0x7f87fc0fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:01:42.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.480+0000 7f8801000640 1 -- 172.21.15.43:0/3168328427 shutdown_connections 2024-09-19T05:01:42.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:42.480+0000 7f8801000640 1 -- 172.21.15.43:0/3168328427 wait complete. 2024-09-19T05:01:43.156 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:43 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:43.156 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:43 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:01:43.156 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:43 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:01:43.156 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:43 smithi043 ceph-mon[30800]: mds.? [v2:172.21.15.96:6832/178785812,v1:172.21.15.96:6833/178785812] up:boot 2024-09-19T05:01:43.156 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:43 smithi043 ceph-mon[30800]: fsmap 2 up:standby 2024-09-19T05:01:43.157 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:43 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T05:01:43.157 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:43 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:43.157 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:43 smithi043 ceph-mon[30800]: Deploying daemon mds.j on smithi043 2024-09-19T05:01:43.157 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:43 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/3168328427' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:01:43.315 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:43 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:43.315 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:43 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:01:43.315 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:43 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:01:43.315 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:43 smithi096 ceph-mon[35085]: mds.? [v2:172.21.15.96:6832/178785812,v1:172.21.15.96:6833/178785812] up:boot 2024-09-19T05:01:43.315 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:43 smithi096 ceph-mon[35085]: fsmap 2 up:standby 2024-09-19T05:01:43.315 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:43 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T05:01:43.315 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:43 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:43.315 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:43 smithi096 ceph-mon[35085]: Deploying daemon mds.j on smithi043 2024-09-19T05:01:43.315 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:43 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/3168328427' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:01:43.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:43 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:43.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:43 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:01:43.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:43 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:01:43.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:43 smithi151 ceph-mon[35425]: mds.? [v2:172.21.15.96:6832/178785812,v1:172.21.15.96:6833/178785812] up:boot 2024-09-19T05:01:43.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:43 smithi151 ceph-mon[35425]: fsmap 2 up:standby 2024-09-19T05:01:43.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:43 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T05:01:43.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:43 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:43.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:43 smithi151 ceph-mon[35425]: Deploying daemon mds.j on smithi043 2024-09-19T05:01:43.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:43 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/3168328427' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:01:44.389 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:44 smithi043 ceph-mon[30800]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:44.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:44 smithi151 ceph-mon[35425]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:44.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:44 smithi096 ceph-mon[35085]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:45.204 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:01:45.383 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:45 smithi043 ceph-mon[30800]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:45.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:45 smithi096 ceph-mon[35085]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:45.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:45 smithi151 ceph-mon[35425]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:45.677 INFO:journalctl@ceph.mds.j.smithi043.stdout:Sep 19 05:01:45 smithi043 podman[68087]: 2024-09-19 05:01:45.423685614 +0000 UTC m=+0.167597569 container create 80bca36d1ddde68859eade052168b5a31ad0dfd4cc5f442cd9a9ad2dbad1d6aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-j, ceph=True, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-19T05:01:45.678 INFO:journalctl@ceph.mds.j.smithi043.stdout:Sep 19 05:01:45 smithi043 podman[68087]: 2024-09-19 05:01:45.676577312 +0000 UTC m=+0.420489258 container init 80bca36d1ddde68859eade052168b5a31ad0dfd4cc5f442cd9a9ad2dbad1d6aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-j, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-19T05:01:45.886 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:01:46.027 INFO:journalctl@ceph.mds.j.smithi043.stdout:Sep 19 05:01:45 smithi043 podman[68087]: 2024-09-19 05:01:45.682914885 +0000 UTC m=+0.426826831 container start 80bca36d1ddde68859eade052168b5a31ad0dfd4cc5f442cd9a9ad2dbad1d6aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-j, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T05:01:46.027 INFO:journalctl@ceph.mds.j.smithi043.stdout:Sep 19 05:01:45 smithi043 ceph-mds[68133]: -- 172.21.15.43:0/4213764848 <== mon.0 v2:172.21.15.43:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557d591ddc20 con 0x557d591bb400 2024-09-19T05:01:46.027 INFO:journalctl@ceph.mds.j.smithi043.stdout:Sep 19 05:01:45 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-j[68120]: starting mds.j at 2024-09-19T05:01:46.027 INFO:journalctl@ceph.mds.j.smithi043.stdout:Sep 19 05:01:45 smithi043 bash[68087]: 80bca36d1ddde68859eade052168b5a31ad0dfd4cc5f442cd9a9ad2dbad1d6aa 2024-09-19T05:01:46.027 INFO:journalctl@ceph.mds.j.smithi043.stdout:Sep 19 05:01:45 smithi043 systemd[1]: Started Ceph mds.j for 1456a810-7643-11ef-bceb-c7b262605968. 2024-09-19T05:01:46.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.447+0000 7f2d17e00640 1 -- 172.21.15.43:0/2073581014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d180745c0 msgr2=0x7f2d180749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:46.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.447+0000 7f2d17e00640 1 --2- 172.21.15.43:0/2073581014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d180745c0 0x7f2d180749a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2d08009900 tx=0x7f2d0802a480 comp rx=0 tx=0).stop 2024-09-19T05:01:46.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.447+0000 7f2d17e00640 1 -- 172.21.15.43:0/2073581014 shutdown_connections 2024-09-19T05:01:46.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.447+0000 7f2d17e00640 1 --2- 172.21.15.43:0/2073581014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d1810e260 0x7f2d18110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:46.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.447+0000 7f2d17e00640 1 --2- 172.21.15.43:0/2073581014 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d18074ee0 0x7f2d1810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:46.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.447+0000 7f2d17e00640 1 --2- 172.21.15.43:0/2073581014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d180745c0 0x7f2d180749a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:46.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.447+0000 7f2d17e00640 1 -- 172.21.15.43:0/2073581014 >> 172.21.15.43:0/2073581014 conn(0x7f2d180fd740 msgr2=0x7f2d180ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:01:46.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.447+0000 7f2d17e00640 1 -- 172.21.15.43:0/2073581014 shutdown_connections 2024-09-19T05:01:46.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.447+0000 7f2d17e00640 1 -- 172.21.15.43:0/2073581014 wait complete. 2024-09-19T05:01:46.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.448+0000 7f2d17e00640 1 Processor -- start 2024-09-19T05:01:46.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.448+0000 7f2d17e00640 1 -- start start 2024-09-19T05:01:46.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.449+0000 7f2d17e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d180745c0 0x7f2d1819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:46.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.449+0000 7f2d17e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d18074ee0 0x7f2d1819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:46.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.449+0000 7f2d17e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d1810e260 0x7f2d1819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:46.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.449+0000 7f2d17e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2d18113fb0 con 0x7f2d1810e260 2024-09-19T05:01:46.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.449+0000 7f2d17e00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2d18113e30 con 0x7f2d180745c0 2024-09-19T05:01:46.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.449+0000 7f2d17e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2d18114130 con 0x7f2d18074ee0 2024-09-19T05:01:46.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.449+0000 7f2d16a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d180745c0 0x7f2d1819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:46.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.449+0000 7f2d16a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d180745c0 0x7f2d1819a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:34210/0 (socket says 172.21.15.43:34210) 2024-09-19T05:01:46.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.449+0000 7f2d16000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d18074ee0 0x7f2d1819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:46.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.449+0000 7f2d16a00640 1 -- 172.21.15.43:0/1963138808 learned_addr learned my addr 172.21.15.43:0/1963138808 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:01:46.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.449+0000 7f2d17400640 1 --2- 172.21.15.43:0/1963138808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d1810e260 0x7f2d1819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:46.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.449+0000 7f2d16a00640 1 -- 172.21.15.43:0/1963138808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d18074ee0 msgr2=0x7f2d1819a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:46.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.449+0000 7f2d16a00640 1 --2- 172.21.15.43:0/1963138808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d18074ee0 0x7f2d1819a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:46.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.449+0000 7f2d16a00640 1 -- 172.21.15.43:0/1963138808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d1810e260 msgr2=0x7f2d1819ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:46.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.449+0000 7f2d16a00640 1 --2- 172.21.15.43:0/1963138808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d1810e260 0x7f2d1819ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:46.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.449+0000 7f2d16a00640 1 -- 172.21.15.43:0/1963138808 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d1819f420 con 0x7f2d180745c0 2024-09-19T05:01:46.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.449+0000 7f2d16000640 1 --2- 172.21.15.43:0/1963138808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d18074ee0 0x7f2d1819a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:01:46.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.450+0000 7f2d17400640 1 --2- 172.21.15.43:0/1963138808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d1810e260 0x7f2d1819ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:01:46.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.450+0000 7f2d16a00640 1 --2- 172.21.15.43:0/1963138808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d180745c0 0x7f2d1819a470 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2d08006010 tx=0x7f2d080042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:01:46.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.450+0000 7f2d0f400640 1 -- 172.21.15.43:0/1963138808 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0803f030 con 0x7f2d180745c0 2024-09-19T05:01:46.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.450+0000 7f2d17e00640 1 -- 172.21.15.43:0/1963138808 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d1819f6b0 con 0x7f2d180745c0 2024-09-19T05:01:46.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.450+0000 7f2d0f400640 1 -- 172.21.15.43:0/1963138808 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d08003ce0 con 0x7f2d180745c0 2024-09-19T05:01:46.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.450+0000 7f2d0f400640 1 -- 172.21.15.43:0/1963138808 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d08035070 con 0x7f2d180745c0 2024-09-19T05:01:46.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.450+0000 7f2d17e00640 1 -- 172.21.15.43:0/1963138808 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d181a6ff0 con 0x7f2d180745c0 2024-09-19T05:01:46.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.451+0000 7f2d17e00640 1 -- 172.21.15.43:0/1963138808 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d18075c70 con 0x7f2d180745c0 2024-09-19T05:01:46.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.453+0000 7f2d0f400640 1 -- 172.21.15.43:0/1963138808 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f2d08042030 con 0x7f2d180745c0 2024-09-19T05:01:46.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.454+0000 7f2d0f400640 1 --2- 172.21.15.43:0/1963138808 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f2cfc0a3240 0x7f2cfc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:46.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.455+0000 7f2d0f400640 1 -- 172.21.15.43:0/1963138808 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f2d080efa40 con 0x7f2d180745c0 2024-09-19T05:01:46.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.455+0000 7f2d16000640 1 --2- 172.21.15.43:0/1963138808 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f2cfc0a3240 0x7f2cfc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:46.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.455+0000 7f2d16000640 1 --2- 172.21.15.43:0/1963138808 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f2cfc0a3240 0x7f2cfc0a5700 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2d1810f5f0 tx=0x7f2d04005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:01:46.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.459+0000 7f2d0f400640 1 -- 172.21.15.43:0/1963138808 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2d080bc5d0 con 0x7f2d180745c0 2024-09-19T05:01:46.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.702+0000 7f2d17e00640 1 -- 172.21.15.43:0/1963138808 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d181093e0 con 0x7f2d180745c0 2024-09-19T05:01:46.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.703+0000 7f2d0f400640 1 -- 172.21.15.43:0/1963138808 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3082 (secure 0 0 0) 0x7f2d080c08d0 con 0x7f2d180745c0 2024-09-19T05:01:46.705 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:01:46.706 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":4,"btime":"2024-09-19T05:01:46:180264+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/773801169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":773801169},{"type":"v1","addr":"172.21.15.43:6835","nonce":773801169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/2836453158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2836453158},{"type":"v1","addr":"172.21.15.151:6833","nonce":2836453158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/178785812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":178785812},{"type":"v1","addr":"172.21.15.96:6833","nonce":178785812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-19T05:01:46.706 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 4 2024-09-19T05:01:46.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.705+0000 7f2d17e00640 1 -- 172.21.15.43:0/1963138808 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f2cfc0a3240 msgr2=0x7f2cfc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:46.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.705+0000 7f2d17e00640 1 --2- 172.21.15.43:0/1963138808 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f2cfc0a3240 0x7f2cfc0a5700 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2d1810f5f0 tx=0x7f2d04005f90 comp rx=0 tx=0).stop 2024-09-19T05:01:46.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.706+0000 7f2d17e00640 1 -- 172.21.15.43:0/1963138808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d180745c0 msgr2=0x7f2d1819a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:46.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.706+0000 7f2d17e00640 1 --2- 172.21.15.43:0/1963138808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d180745c0 0x7f2d1819a470 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2d08006010 tx=0x7f2d080042f0 comp rx=0 tx=0).stop 2024-09-19T05:01:46.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.706+0000 7f2d17e00640 1 -- 172.21.15.43:0/1963138808 shutdown_connections 2024-09-19T05:01:46.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.706+0000 7f2d17e00640 1 --2- 172.21.15.43:0/1963138808 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f2cfc0a3240 0x7f2cfc0a5700 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:46.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.706+0000 7f2d17e00640 1 --2- 172.21.15.43:0/1963138808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d1810e260 0x7f2d1819ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:46.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.706+0000 7f2d17e00640 1 --2- 172.21.15.43:0/1963138808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d18074ee0 0x7f2d1819a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:46.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.706+0000 7f2d17e00640 1 --2- 172.21.15.43:0/1963138808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d180745c0 0x7f2d1819a470 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:46.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.706+0000 7f2d17e00640 1 -- 172.21.15.43:0/1963138808 >> 172.21.15.43:0/1963138808 conn(0x7f2d180fd740 msgr2=0x7f2d180fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:01:46.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.706+0000 7f2d17e00640 1 -- 172.21.15.43:0/1963138808 shutdown_connections 2024-09-19T05:01:46.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:46.706+0000 7f2d17e00640 1 -- 172.21.15.43:0/1963138808 wait complete. 2024-09-19T05:01:47.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:46 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:47.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:46 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:47.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:46 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:47.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:46 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:01:47.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:46 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:01:47.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:46 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:47.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:46 smithi043 ceph-mon[30800]: Deploying daemon mds.d on smithi043 2024-09-19T05:01:47.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:46 smithi043 ceph-mon[30800]: mds.? [v2:172.21.15.43:6834/773801169,v1:172.21.15.43:6835/773801169] up:boot 2024-09-19T05:01:47.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:46 smithi043 ceph-mon[30800]: fsmap 3 up:standby 2024-09-19T05:01:47.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:46 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T05:01:47.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:46 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/1963138808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:01:47.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:46 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:47.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:46 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:47.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:46 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:47.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:46 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:01:47.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:46 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:01:47.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:46 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:47.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:46 smithi151 ceph-mon[35425]: Deploying daemon mds.d on smithi043 2024-09-19T05:01:47.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:46 smithi151 ceph-mon[35425]: mds.? [v2:172.21.15.43:6834/773801169,v1:172.21.15.43:6835/773801169] up:boot 2024-09-19T05:01:47.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:46 smithi151 ceph-mon[35425]: fsmap 3 up:standby 2024-09-19T05:01:47.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:46 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T05:01:47.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:46 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/1963138808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:01:47.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:46 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:47.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:46 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:47.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:46 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:47.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:46 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:01:47.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:46 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:01:47.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:46 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:47.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:46 smithi096 ceph-mon[35085]: Deploying daemon mds.d on smithi043 2024-09-19T05:01:47.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:46 smithi096 ceph-mon[35085]: mds.? [v2:172.21.15.43:6834/773801169,v1:172.21.15.43:6835/773801169] up:boot 2024-09-19T05:01:47.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:46 smithi096 ceph-mon[35085]: fsmap 3 up:standby 2024-09-19T05:01:47.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:46 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T05:01:47.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:46 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/1963138808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:01:48.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:47 smithi043 ceph-mon[30800]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:48.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:47 smithi151 ceph-mon[35425]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:48.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:47 smithi096 ceph-mon[35085]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:49.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:49 smithi096 ceph-mon[35085]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:49.500 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:49 smithi043 ceph-mon[30800]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:49.500 INFO:journalctl@ceph.mds.d.smithi043.stdout:Sep 19 05:01:49 smithi043 systemd[1]: Starting Ceph mds.d for 1456a810-7643-11ef-bceb-c7b262605968... 2024-09-19T05:01:49.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:49 smithi151 ceph-mon[35425]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:49.679 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:01:49.780 INFO:journalctl@ceph.mds.d.smithi043.stdout:Sep 19 05:01:49 smithi043 podman[68480]: 2024-09-19 05:01:49.402828527 +0000 UTC m=+0.027485615 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:01:49.780 INFO:journalctl@ceph.mds.d.smithi043.stdout:Sep 19 05:01:49 smithi043 podman[68480]: 2024-09-19 05:01:49.522973101 +0000 UTC m=+0.147630188 container create d91e4fed3c4c6c4ff379235adfb37e7906daf1fc90de45d45116b44893cf90ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-d, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T05:01:50.080 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:01:50.220 INFO:journalctl@ceph.mds.d.smithi043.stdout:Sep 19 05:01:49 smithi043 podman[68480]: 2024-09-19 05:01:49.805104054 +0000 UTC m=+0.429761142 container init d91e4fed3c4c6c4ff379235adfb37e7906daf1fc90de45d45116b44893cf90ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-d, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:01:50.220 INFO:journalctl@ceph.mds.d.smithi043.stdout:Sep 19 05:01:49 smithi043 podman[68480]: 2024-09-19 05:01:49.811888678 +0000 UTC m=+0.436545774 container start d91e4fed3c4c6c4ff379235adfb37e7906daf1fc90de45d45116b44893cf90ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-d, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-19T05:01:50.220 INFO:journalctl@ceph.mds.d.smithi043.stdout:Sep 19 05:01:49 smithi043 ceph-mds[68517]: -- 172.21.15.43:0/231704017 <== mon.0 v2:172.21.15.43:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b9eb219860 con 0x55b9eb1f6800 2024-09-19T05:01:50.220 INFO:journalctl@ceph.mds.d.smithi043.stdout:Sep 19 05:01:49 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-d[68495]: starting mds.d at 2024-09-19T05:01:50.221 INFO:journalctl@ceph.mds.d.smithi043.stdout:Sep 19 05:01:49 smithi043 bash[68480]: d91e4fed3c4c6c4ff379235adfb37e7906daf1fc90de45d45116b44893cf90ef 2024-09-19T05:01:50.221 INFO:journalctl@ceph.mds.d.smithi043.stdout:Sep 19 05:01:49 smithi043 systemd[1]: Started Ceph mds.d for 1456a810-7643-11ef-bceb-c7b262605968. 2024-09-19T05:01:50.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.634+0000 7f9402400640 1 -- 172.21.15.43:0/1192757366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93fc1049c0 msgr2=0x7f93fc108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:50.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.634+0000 7f9402400640 1 --2- 172.21.15.43:0/1192757366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93fc1049c0 0x7f93fc108a10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f93e4009900 tx=0x7f93e402a410 comp rx=0 tx=0).stop 2024-09-19T05:01:50.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.635+0000 7f9402400640 1 -- 172.21.15.43:0/1192757366 shutdown_connections 2024-09-19T05:01:50.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.635+0000 7f9402400640 1 --2- 172.21.15.43:0/1192757366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93fc109140 0x7f93fc110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:50.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.635+0000 7f9402400640 1 --2- 172.21.15.43:0/1192757366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93fc1049c0 0x7f93fc108a10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:50.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.635+0000 7f9402400640 1 --2- 172.21.15.43:0/1192757366 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f93fc104010 0x7f93fc1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:50.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.635+0000 7f9402400640 1 -- 172.21.15.43:0/1192757366 >> 172.21.15.43:0/1192757366 conn(0x7f93fc0ffa50 msgr2=0x7f93fc101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:01:50.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.635+0000 7f9402400640 1 -- 172.21.15.43:0/1192757366 shutdown_connections 2024-09-19T05:01:50.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.635+0000 7f9402400640 1 -- 172.21.15.43:0/1192757366 wait complete. 2024-09-19T05:01:50.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.636+0000 7f9402400640 1 Processor -- start 2024-09-19T05:01:50.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.636+0000 7f9402400640 1 -- start start 2024-09-19T05:01:50.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.637+0000 7f9402400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93fc104010 0x7f93fc19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:50.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.637+0000 7f9402400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93fc1049c0 0x7f93fc19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:50.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.637+0000 7f9402400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f93fc109140 0x7f93fc19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:50.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.637+0000 7f9402400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f93fc114140 con 0x7f93fc1049c0 2024-09-19T05:01:50.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.637+0000 7f9402400640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f93fc113fc0 con 0x7f93fc104010 2024-09-19T05:01:50.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.637+0000 7f9402400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f93fc1142c0 con 0x7f93fc109140 2024-09-19T05:01:50.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.637+0000 7f93fb400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93fc104010 0x7f93fc19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:50.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.637+0000 7f93fbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f93fc109140 0x7f93fc19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:50.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.637+0000 7f93fb400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93fc104010 0x7f93fc19a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:34246/0 (socket says 172.21.15.43:34246) 2024-09-19T05:01:50.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.637+0000 7f93fb400640 1 -- 172.21.15.43:0/174641639 learned_addr learned my addr 172.21.15.43:0/174641639 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:01:50.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.637+0000 7f93f2a00640 1 --2- 172.21.15.43:0/174641639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93fc1049c0 0x7f93fc19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:50.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.637+0000 7f93fbe00640 1 -- 172.21.15.43:0/174641639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93fc104010 msgr2=0x7f93fc19a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:50.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.637+0000 7f93fbe00640 1 --2- 172.21.15.43:0/174641639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93fc104010 0x7f93fc19a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:50.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.637+0000 7f93fbe00640 1 -- 172.21.15.43:0/174641639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93fc1049c0 msgr2=0x7f93fc19ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:50.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.637+0000 7f93fbe00640 1 --2- 172.21.15.43:0/174641639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93fc1049c0 0x7f93fc19ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:50.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.637+0000 7f93fbe00640 1 -- 172.21.15.43:0/174641639 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93fc19f5d0 con 0x7f93fc109140 2024-09-19T05:01:50.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.638+0000 7f93fbe00640 1 --2- 172.21.15.43:0/174641639 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f93fc109140 0x7f93fc19eef0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f93f4009fd0 tx=0x7f93f400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:01:50.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.638+0000 7f93f8c00640 1 -- 172.21.15.43:0/174641639 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f400ce20 con 0x7f93fc109140 2024-09-19T05:01:50.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.638+0000 7f93f8c00640 1 -- 172.21.15.43:0/174641639 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93f4005020 con 0x7f93fc109140 2024-09-19T05:01:50.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.638+0000 7f9402400640 1 -- 172.21.15.43:0/174641639 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93fc19f8c0 con 0x7f93fc109140 2024-09-19T05:01:50.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.638+0000 7f93f8c00640 1 -- 172.21.15.43:0/174641639 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f4010720 con 0x7f93fc109140 2024-09-19T05:01:50.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.638+0000 7f9402400640 1 -- 172.21.15.43:0/174641639 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93fc1a71a0 con 0x7f93fc109140 2024-09-19T05:01:50.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.639+0000 7f9402400640 1 -- 172.21.15.43:0/174641639 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93fc1043f0 con 0x7f93fc109140 2024-09-19T05:01:50.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.641+0000 7f93f8c00640 1 -- 172.21.15.43:0/174641639 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f93f4005540 con 0x7f93fc109140 2024-09-19T05:01:50.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.642+0000 7f93f8c00640 1 --2- 172.21.15.43:0/174641639 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f93e00a32c0 0x7f93e00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:50.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.643+0000 7f93f8c00640 1 -- 172.21.15.43:0/174641639 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f93f40cfa80 con 0x7f93fc109140 2024-09-19T05:01:50.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.643+0000 7f93fb400640 1 --2- 172.21.15.43:0/174641639 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f93e00a32c0 0x7f93e00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:50.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.643+0000 7f93fb400640 1 --2- 172.21.15.43:0/174641639 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f93e00a32c0 0x7f93e00a5780 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f93ec009c90 tx=0x7f93ec005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:01:50.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.644+0000 7f93f8c00640 1 -- 172.21.15.43:0/174641639 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f93f409cba0 con 0x7f93fc109140 2024-09-19T05:01:50.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.865+0000 7f9402400640 1 -- 172.21.15.43:0/174641639 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f93fc02d7c0 con 0x7f93fc109140 2024-09-19T05:01:50.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.866+0000 7f93f8c00640 1 -- 172.21.15.43:0/174641639 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3907 (secure 0 0 0) 0x7f93f409c4a0 con 0x7f93fc109140 2024-09-19T05:01:50.868 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:01:50.868 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":5,"btime":"2024-09-19T05:01:50:181765+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/773801169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":773801169},{"type":"v1","addr":"172.21.15.43:6835","nonce":773801169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1799137902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1799137902},{"type":"v1","addr":"172.21.15.43:6837","nonce":1799137902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/2836453158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2836453158},{"type":"v1","addr":"172.21.15.151:6833","nonce":2836453158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/178785812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":178785812},{"type":"v1","addr":"172.21.15.96:6833","nonce":178785812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-19T05:01:50.868 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 5 2024-09-19T05:01:50.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.868+0000 7f9402400640 1 -- 172.21.15.43:0/174641639 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f93e00a32c0 msgr2=0x7f93e00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:50.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.868+0000 7f9402400640 1 --2- 172.21.15.43:0/174641639 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f93e00a32c0 0x7f93e00a5780 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f93ec009c90 tx=0x7f93ec005f90 comp rx=0 tx=0).stop 2024-09-19T05:01:50.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.868+0000 7f9402400640 1 -- 172.21.15.43:0/174641639 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f93fc109140 msgr2=0x7f93fc19eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:50.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.868+0000 7f9402400640 1 --2- 172.21.15.43:0/174641639 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f93fc109140 0x7f93fc19eef0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f93f4009fd0 tx=0x7f93f400ef10 comp rx=0 tx=0).stop 2024-09-19T05:01:50.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.868+0000 7f9402400640 1 -- 172.21.15.43:0/174641639 shutdown_connections 2024-09-19T05:01:50.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.868+0000 7f9402400640 1 --2- 172.21.15.43:0/174641639 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f93e00a32c0 0x7f93e00a5780 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:50.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.868+0000 7f9402400640 1 --2- 172.21.15.43:0/174641639 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f93fc109140 0x7f93fc19eef0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:50.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.868+0000 7f9402400640 1 --2- 172.21.15.43:0/174641639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93fc1049c0 0x7f93fc19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:50.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.868+0000 7f9402400640 1 --2- 172.21.15.43:0/174641639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93fc104010 0x7f93fc19a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:50.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.868+0000 7f9402400640 1 -- 172.21.15.43:0/174641639 >> 172.21.15.43:0/174641639 conn(0x7f93fc0ffa50 msgr2=0x7f93fc100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:01:50.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.868+0000 7f9402400640 1 -- 172.21.15.43:0/174641639 shutdown_connections 2024-09-19T05:01:50.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:50.868+0000 7f9402400640 1 -- 172.21.15.43:0/174641639 wait complete. 2024-09-19T05:01:51.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:51 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:51.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:51 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:51.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:51 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:51.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:51 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:01:51.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:51 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:01:51.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:51 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:51.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:51 smithi151 ceph-mon[35425]: Deploying daemon mds.e on smithi096 2024-09-19T05:01:51.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:51 smithi151 ceph-mon[35425]: mds.? [v2:172.21.15.43:6836/1799137902,v1:172.21.15.43:6837/1799137902] up:boot 2024-09-19T05:01:51.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:51 smithi151 ceph-mon[35425]: fsmap 4 up:standby 2024-09-19T05:01:51.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:51 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T05:01:51.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:51 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/174641639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:01:51.161 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:51 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:51.161 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:51 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:51.161 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:51 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:51.161 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:51 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:01:51.162 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:51 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:01:51.162 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:51 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:51.162 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:51 smithi096 ceph-mon[35085]: Deploying daemon mds.e on smithi096 2024-09-19T05:01:51.162 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:51 smithi096 ceph-mon[35085]: mds.? [v2:172.21.15.43:6836/1799137902,v1:172.21.15.43:6837/1799137902] up:boot 2024-09-19T05:01:51.162 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:51 smithi096 ceph-mon[35085]: fsmap 4 up:standby 2024-09-19T05:01:51.162 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:51 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T05:01:51.162 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:51 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/174641639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:01:51.248 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:51 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:51.249 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:51 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:51.249 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:51 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:51.249 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:51 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:01:51.249 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:51 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:01:51.249 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:51 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:51.249 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:51 smithi043 ceph-mon[30800]: Deploying daemon mds.e on smithi096 2024-09-19T05:01:51.249 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:51 smithi043 ceph-mon[30800]: mds.? [v2:172.21.15.43:6836/1799137902,v1:172.21.15.43:6837/1799137902] up:boot 2024-09-19T05:01:51.249 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:51 smithi043 ceph-mon[30800]: fsmap 4 up:standby 2024-09-19T05:01:51.249 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:51 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T05:01:51.249 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:51 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/174641639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:01:52.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:52 smithi096 ceph-mon[35085]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:52.223 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 19 05:01:52 smithi096 systemd[1]: Starting Ceph mds.e for 1456a810-7643-11ef-bceb-c7b262605968... 2024-09-19T05:01:52.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:52 smithi043 ceph-mon[30800]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:52.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:52 smithi151 ceph-mon[35425]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:52.585 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 19 05:01:52 smithi096 podman[64550]: 2024-09-19 05:01:52.248673363 +0000 UTC m=+0.108868581 container create 06f3d3813f3932e9b4d91d0172fc96dc1e180ed5de878eff54d45b7862a44c48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-e, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T05:01:52.585 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 19 05:01:52 smithi096 podman[64550]: 2024-09-19 05:01:52.155815676 +0000 UTC m=+0.016010907 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:01:52.585 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 19 05:01:52 smithi096 podman[64550]: 2024-09-19 05:01:52.584465025 +0000 UTC m=+0.444660243 container init 06f3d3813f3932e9b4d91d0172fc96dc1e180ed5de878eff54d45b7862a44c48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-e, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T05:01:52.973 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 19 05:01:52 smithi096 podman[64550]: 2024-09-19 05:01:52.587556803 +0000 UTC m=+0.447752024 container start 06f3d3813f3932e9b4d91d0172fc96dc1e180ed5de878eff54d45b7862a44c48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-19T05:01:52.973 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 19 05:01:52 smithi096 ceph-mds[64569]: -- 172.21.15.96:0/4084441192 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564cf27c9860 con 0x564cf27a7000 2024-09-19T05:01:52.973 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 19 05:01:52 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-e[64565]: starting mds.e at 2024-09-19T05:01:52.973 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 19 05:01:52 smithi096 bash[64550]: 06f3d3813f3932e9b4d91d0172fc96dc1e180ed5de878eff54d45b7862a44c48 2024-09-19T05:01:52.973 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 19 05:01:52 smithi096 systemd[1]: Started Ceph mds.e for 1456a810-7643-11ef-bceb-c7b262605968. 2024-09-19T05:01:53.617 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:01:54.007 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:01:54.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:53 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:54.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:53 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:54.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:53 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:54.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:53 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:01:54.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:53 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:01:54.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:53 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:54.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:53 smithi043 ceph-mon[30800]: Deploying daemon mds.f on smithi151 2024-09-19T05:01:54.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:53 smithi043 ceph-mon[30800]: mds.? [v2:172.21.15.96:6834/1671043820,v1:172.21.15.96:6835/1671043820] up:boot 2024-09-19T05:01:54.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:53 smithi043 ceph-mon[30800]: fsmap 5 up:standby 2024-09-19T05:01:54.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:53 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T05:01:54.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:53 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:54.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:53 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:54.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:53 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:54.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:53 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:01:54.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:53 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:01:54.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:53 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:54.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:53 smithi151 ceph-mon[35425]: Deploying daemon mds.f on smithi151 2024-09-19T05:01:54.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:53 smithi151 ceph-mon[35425]: mds.? [v2:172.21.15.96:6834/1671043820,v1:172.21.15.96:6835/1671043820] up:boot 2024-09-19T05:01:54.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:53 smithi151 ceph-mon[35425]: fsmap 5 up:standby 2024-09-19T05:01:54.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:53 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T05:01:54.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:53 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:54.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:53 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:54.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:53 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:54.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:53 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:01:54.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:53 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:01:54.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:53 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:54.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:53 smithi096 ceph-mon[35085]: Deploying daemon mds.f on smithi151 2024-09-19T05:01:54.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:53 smithi096 ceph-mon[35085]: mds.? [v2:172.21.15.96:6834/1671043820,v1:172.21.15.96:6835/1671043820] up:boot 2024-09-19T05:01:54.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:53 smithi096 ceph-mon[35085]: fsmap 5 up:standby 2024-09-19T05:01:54.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:53 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T05:01:54.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.549+0000 7f2b89a00640 1 -- 172.21.15.43:0/2858862641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b84103e30 msgr2=0x7f2b84104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:54.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.549+0000 7f2b89a00640 1 --2- 172.21.15.43:0/2858862641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b84103e30 0x7f2b84104210 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2b74009900 tx=0x7f2b7402a3f0 comp rx=0 tx=0).stop 2024-09-19T05:01:54.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.549+0000 7f2b89a00640 1 -- 172.21.15.43:0/2858862641 shutdown_connections 2024-09-19T05:01:54.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.549+0000 7f2b89a00640 1 --2- 172.21.15.43:0/2858862641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b84108f60 0x7f2b84110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:54.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.549+0000 7f2b89a00640 1 --2- 172.21.15.43:0/2858862641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b841047e0 0x7f2b84108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:54.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.549+0000 7f2b89a00640 1 --2- 172.21.15.43:0/2858862641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b84103e30 0x7f2b84104210 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:54.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.549+0000 7f2b89a00640 1 -- 172.21.15.43:0/2858862641 >> 172.21.15.43:0/2858862641 conn(0x7f2b840ff8d0 msgr2=0x7f2b84101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:01:54.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.549+0000 7f2b89a00640 1 -- 172.21.15.43:0/2858862641 shutdown_connections 2024-09-19T05:01:54.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.549+0000 7f2b89a00640 1 -- 172.21.15.43:0/2858862641 wait complete. 2024-09-19T05:01:54.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.550+0000 7f2b89a00640 1 Processor -- start 2024-09-19T05:01:54.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.550+0000 7f2b89a00640 1 -- start start 2024-09-19T05:01:54.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.550+0000 7f2b89a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b84103e30 0x7f2b8419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:54.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.550+0000 7f2b89a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b841047e0 0x7f2b8419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:54.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.550+0000 7f2b89a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b84108f60 0x7f2b8419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:54.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.550+0000 7f2b89a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2b84114010 con 0x7f2b84108f60 2024-09-19T05:01:54.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.550+0000 7f2b89a00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2b84113e90 con 0x7f2b84103e30 2024-09-19T05:01:54.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.550+0000 7f2b89a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2b84114190 con 0x7f2b841047e0 2024-09-19T05:01:54.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.551+0000 7f2b7b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b841047e0 0x7f2b8419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:54.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.551+0000 7f2b7b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b841047e0 0x7f2b8419a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:32896/0 (socket says 172.21.15.43:32896) 2024-09-19T05:01:54.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.551+0000 7f2b7b400640 1 -- 172.21.15.43:0/2022344034 learned_addr learned my addr 172.21.15.43:0/2022344034 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:01:54.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.551+0000 7f2b89000640 1 --2- 172.21.15.43:0/2022344034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b84108f60 0x7f2b8419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:54.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.551+0000 7f2b83e00640 1 --2- 172.21.15.43:0/2022344034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b84103e30 0x7f2b8419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:54.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.551+0000 7f2b7b400640 1 -- 172.21.15.43:0/2022344034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b84103e30 msgr2=0x7f2b8419a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:54.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.551+0000 7f2b7b400640 1 --2- 172.21.15.43:0/2022344034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b84103e30 0x7f2b8419a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:54.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.551+0000 7f2b7b400640 1 -- 172.21.15.43:0/2022344034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b84108f60 msgr2=0x7f2b8419ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:54.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.551+0000 7f2b7b400640 1 --2- 172.21.15.43:0/2022344034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b84108f60 0x7f2b8419ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:54.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.551+0000 7f2b7b400640 1 -- 172.21.15.43:0/2022344034 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b8419f400 con 0x7f2b841047e0 2024-09-19T05:01:54.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.551+0000 7f2b7b400640 1 --2- 172.21.15.43:0/2022344034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b841047e0 0x7f2b8419a990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c00e9f0 tx=0x7f2b6c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:01:54.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.552+0000 7f2b81600640 1 -- 172.21.15.43:0/2022344034 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c00cd80 con 0x7f2b841047e0 2024-09-19T05:01:54.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.552+0000 7f2b81600640 1 -- 172.21.15.43:0/2022344034 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b6c005500 con 0x7f2b841047e0 2024-09-19T05:01:54.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.552+0000 7f2b81600640 1 -- 172.21.15.43:0/2022344034 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c010690 con 0x7f2b841047e0 2024-09-19T05:01:54.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.552+0000 7f2b89a00640 1 -- 172.21.15.43:0/2022344034 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b8419f6f0 con 0x7f2b841047e0 2024-09-19T05:01:54.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.552+0000 7f2b89a00640 1 -- 172.21.15.43:0/2022344034 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b841a6fd0 con 0x7f2b841047e0 2024-09-19T05:01:54.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.553+0000 7f2b89a00640 1 -- 172.21.15.43:0/2022344034 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b84104210 con 0x7f2b841047e0 2024-09-19T05:01:54.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.555+0000 7f2b81600640 1 -- 172.21.15.43:0/2022344034 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f2b6c0046c0 con 0x7f2b841047e0 2024-09-19T05:01:54.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.556+0000 7f2b81600640 1 --2- 172.21.15.43:0/2022344034 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f2b680a32c0 0x7f2b680a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:54.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.556+0000 7f2b81600640 1 -- 172.21.15.43:0/2022344034 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f2b6c0cfb50 con 0x7f2b841047e0 2024-09-19T05:01:54.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.556+0000 7f2b83e00640 1 --2- 172.21.15.43:0/2022344034 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f2b680a32c0 0x7f2b680a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:54.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.557+0000 7f2b83e00640 1 --2- 172.21.15.43:0/2022344034 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f2b680a32c0 0x7f2b680a5780 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2b74000c00 tx=0x7f2b74005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:01:54.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.558+0000 7f2b81600640 1 -- 172.21.15.43:0/2022344034 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2b6c09cc70 con 0x7f2b841047e0 2024-09-19T05:01:54.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.781+0000 7f2b89a00640 1 -- 172.21.15.43:0/2022344034 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b8402d7c0 con 0x7f2b841047e0 2024-09-19T05:01:54.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.782+0000 7f2b81600640 1 -- 172.21.15.43:0/2022344034 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4732 (secure 0 0 0) 0x7f2b6c0bc030 con 0x7f2b841047e0 2024-09-19T05:01:54.783 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:01:54.783 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":6,"btime":"2024-09-19T05:01:53:012455+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/773801169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":773801169},{"type":"v1","addr":"172.21.15.43:6835","nonce":773801169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1799137902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1799137902},{"type":"v1","addr":"172.21.15.43:6837","nonce":1799137902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/2836453158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2836453158},{"type":"v1","addr":"172.21.15.151:6833","nonce":2836453158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/178785812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":178785812},{"type":"v1","addr":"172.21.15.96:6833","nonce":178785812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1671043820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1671043820},{"type":"v1","addr":"172.21.15.96:6835","nonce":1671043820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-19T05:01:54.784 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 6 2024-09-19T05:01:54.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.784+0000 7f2b89a00640 1 -- 172.21.15.43:0/2022344034 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f2b680a32c0 msgr2=0x7f2b680a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:54.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.784+0000 7f2b89a00640 1 --2- 172.21.15.43:0/2022344034 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f2b680a32c0 0x7f2b680a5780 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2b74000c00 tx=0x7f2b74005b50 comp rx=0 tx=0).stop 2024-09-19T05:01:54.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.784+0000 7f2b89a00640 1 -- 172.21.15.43:0/2022344034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b841047e0 msgr2=0x7f2b8419a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:54.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.784+0000 7f2b89a00640 1 --2- 172.21.15.43:0/2022344034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b841047e0 0x7f2b8419a990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c00e9f0 tx=0x7f2b6c00edf0 comp rx=0 tx=0).stop 2024-09-19T05:01:54.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.785+0000 7f2b89a00640 1 -- 172.21.15.43:0/2022344034 shutdown_connections 2024-09-19T05:01:54.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.785+0000 7f2b89a00640 1 --2- 172.21.15.43:0/2022344034 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f2b680a32c0 0x7f2b680a5780 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:54.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.785+0000 7f2b89a00640 1 --2- 172.21.15.43:0/2022344034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b84108f60 0x7f2b8419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:54.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.785+0000 7f2b89a00640 1 --2- 172.21.15.43:0/2022344034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b841047e0 0x7f2b8419a990 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:54.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.785+0000 7f2b89a00640 1 --2- 172.21.15.43:0/2022344034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b84103e30 0x7f2b8419a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:54.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.785+0000 7f2b89a00640 1 -- 172.21.15.43:0/2022344034 >> 172.21.15.43:0/2022344034 conn(0x7f2b840ff8d0 msgr2=0x7f2b84100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:01:54.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.785+0000 7f2b89a00640 1 -- 172.21.15.43:0/2022344034 shutdown_connections 2024-09-19T05:01:54.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:54.785+0000 7f2b89a00640 1 -- 172.21.15.43:0/2022344034 wait complete. 2024-09-19T05:01:54.887 INFO:journalctl@ceph.mds.f.smithi151.stdout:Sep 19 05:01:54 smithi151 systemd[1]: Starting Ceph mds.f for 1456a810-7643-11ef-bceb-c7b262605968... 2024-09-19T05:01:55.143 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:55 smithi043 ceph-mon[30800]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:55.439 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:55 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/2022344034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:01:55.439 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:55 smithi151 ceph-mon[35425]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:55.440 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:55 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/2022344034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:01:55.440 INFO:journalctl@ceph.mds.f.smithi151.stdout:Sep 19 05:01:54 smithi151 podman[65168]: 2024-09-19 05:01:54.789020947 +0000 UTC m=+0.015890324 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:01:55.440 INFO:journalctl@ceph.mds.f.smithi151.stdout:Sep 19 05:01:54 smithi151 podman[65168]: 2024-09-19 05:01:54.913517521 +0000 UTC m=+0.140386892 container create 3bdb1ef911fa3ac241c97ee037bf051fa668866ee0287b3ea03ed09fb7bdee00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-f, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:01:55.440 INFO:journalctl@ceph.mds.f.smithi151.stdout:Sep 19 05:01:55 smithi151 podman[65168]: 2024-09-19 05:01:55.149478144 +0000 UTC m=+0.376347508 container init 3bdb1ef911fa3ac241c97ee037bf051fa668866ee0287b3ea03ed09fb7bdee00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-f, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_CLEAN=True) 2024-09-19T05:01:55.440 INFO:journalctl@ceph.mds.f.smithi151.stdout:Sep 19 05:01:55 smithi151 podman[65168]: 2024-09-19 05:01:55.15238287 +0000 UTC m=+0.379252253 container start 3bdb1ef911fa3ac241c97ee037bf051fa668866ee0287b3ea03ed09fb7bdee00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-f, RELEASE=squid-2f599a4, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-19T05:01:55.440 INFO:journalctl@ceph.mds.f.smithi151.stdout:Sep 19 05:01:55 smithi151 ceph-mds[65187]: -- 172.21.15.151:0/2247352562 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563cae3d7860 con 0x563cae3b5000 2024-09-19T05:01:55.441 INFO:journalctl@ceph.mds.f.smithi151.stdout:Sep 19 05:01:55 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-f[65183]: starting mds.f at 2024-09-19T05:01:55.441 INFO:journalctl@ceph.mds.f.smithi151.stdout:Sep 19 05:01:55 smithi151 bash[65168]: 3bdb1ef911fa3ac241c97ee037bf051fa668866ee0287b3ea03ed09fb7bdee00 2024-09-19T05:01:55.441 INFO:journalctl@ceph.mds.f.smithi151.stdout:Sep 19 05:01:55 smithi151 systemd[1]: Started Ceph mds.f for 1456a810-7643-11ef-bceb-c7b262605968. 2024-09-19T05:01:55.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:55 smithi096 ceph-mon[35085]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:55.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:55 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/2022344034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:01:56.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:56 smithi151 ceph-mon[35425]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:56.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:56 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:56.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:56 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:56.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:56 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:56.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:56 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:01:56.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:56 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:01:56.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:56 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:56.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:56 smithi151 ceph-mon[35425]: Deploying daemon mds.c on smithi151 2024-09-19T05:01:56.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:56 smithi151 ceph-mon[35425]: mds.? [v2:172.21.15.151:6834/2972514328,v1:172.21.15.151:6835/2972514328] up:boot 2024-09-19T05:01:56.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:56 smithi151 ceph-mon[35425]: fsmap 6 up:standby 2024-09-19T05:01:56.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:56 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T05:01:56.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:56 smithi096 ceph-mon[35085]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:56.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:56 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:56.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:56 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:56.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:56 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:56.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:56 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:01:56.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:56 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:01:56.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:56 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:56.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:56 smithi096 ceph-mon[35085]: Deploying daemon mds.c on smithi151 2024-09-19T05:01:56.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:56 smithi096 ceph-mon[35085]: mds.? [v2:172.21.15.151:6834/2972514328,v1:172.21.15.151:6835/2972514328] up:boot 2024-09-19T05:01:56.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:56 smithi096 ceph-mon[35085]: fsmap 6 up:standby 2024-09-19T05:01:56.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:56 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T05:01:56.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:56 smithi043 ceph-mon[30800]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:56.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:56 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:56.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:56 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:56.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:56 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:56.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:56 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:01:56.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:56 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:01:56.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:56 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:56.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:56 smithi043 ceph-mon[30800]: Deploying daemon mds.c on smithi151 2024-09-19T05:01:56.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:56 smithi043 ceph-mon[30800]: mds.? [v2:172.21.15.151:6834/2972514328,v1:172.21.15.151:6835/2972514328] up:boot 2024-09-19T05:01:56.782 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:56 smithi043 ceph-mon[30800]: fsmap 6 up:standby 2024-09-19T05:01:56.782 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:56 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T05:01:57.405 INFO:journalctl@ceph.mds.c.smithi151.stdout:Sep 19 05:01:57 smithi151 systemd[1]: Starting Ceph mds.c for 1456a810-7643-11ef-bceb-c7b262605968... 2024-09-19T05:01:57.532 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:01:57.762 INFO:journalctl@ceph.mds.c.smithi151.stdout:Sep 19 05:01:57 smithi151 podman[65472]: 2024-09-19 05:01:57.346227695 +0000 UTC m=+0.016377671 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:01:57.762 INFO:journalctl@ceph.mds.c.smithi151.stdout:Sep 19 05:01:57 smithi151 podman[65472]: 2024-09-19 05:01:57.46706246 +0000 UTC m=+0.137222149 container create b2c85bce4d908712ab79c6be5738175ffb5194f947d9065bc9ccbcd7023a88c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-19T05:01:57.762 INFO:journalctl@ceph.mds.c.smithi151.stdout:Sep 19 05:01:57 smithi151 podman[65472]: 2024-09-19 05:01:57.761457892 +0000 UTC m=+0.431607852 container init b2c85bce4d908712ab79c6be5738175ffb5194f947d9065bc9ccbcd7023a88c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-c, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-19T05:01:57.917 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:01:58.155 INFO:journalctl@ceph.mds.c.smithi151.stdout:Sep 19 05:01:57 smithi151 podman[65472]: 2024-09-19 05:01:57.764440645 +0000 UTC m=+0.434590606 container start b2c85bce4d908712ab79c6be5738175ffb5194f947d9065bc9ccbcd7023a88c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-19T05:01:58.155 INFO:journalctl@ceph.mds.c.smithi151.stdout:Sep 19 05:01:57 smithi151 ceph-mds[65492]: -- 172.21.15.151:0/3237688315 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556150a5d860 con 0x556150a3b400 2024-09-19T05:01:58.155 INFO:journalctl@ceph.mds.c.smithi151.stdout:Sep 19 05:01:57 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-c[65488]: starting mds.c at 2024-09-19T05:01:58.155 INFO:journalctl@ceph.mds.c.smithi151.stdout:Sep 19 05:01:57 smithi151 bash[65472]: b2c85bce4d908712ab79c6be5738175ffb5194f947d9065bc9ccbcd7023a88c1 2024-09-19T05:01:58.155 INFO:journalctl@ceph.mds.c.smithi151.stdout:Sep 19 05:01:57 smithi151 systemd[1]: Started Ceph mds.c for 1456a810-7643-11ef-bceb-c7b262605968. 2024-09-19T05:01:58.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.526+0000 7fa26be00640 1 -- 172.21.15.43:0/1509996455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa26c112870 msgr2=0x7fa26c114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:58.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.526+0000 7fa26be00640 1 --2- 172.21.15.43:0/1509996455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa26c112870 0x7fa26c114c60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa264009ca0 tx=0x7fa26402a8c0 comp rx=0 tx=0).stop 2024-09-19T05:01:58.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.526+0000 7fa26be00640 1 -- 172.21.15.43:0/1509996455 shutdown_connections 2024-09-19T05:01:58.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.526+0000 7fa26be00640 1 --2- 172.21.15.43:0/1509996455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa26c112870 0x7fa26c114c60 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:58.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.526+0000 7fa26be00640 1 --2- 172.21.15.43:0/1509996455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa26c077040 0x7fa26c112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:58.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.526+0000 7fa26be00640 1 --2- 172.21.15.43:0/1509996455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa26c076720 0x7fa26c076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:58.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.526+0000 7fa26be00640 1 -- 172.21.15.43:0/1509996455 >> 172.21.15.43:0/1509996455 conn(0x7fa26c0ff910 msgr2=0x7fa26c101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:01:58.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.526+0000 7fa26be00640 1 -- 172.21.15.43:0/1509996455 shutdown_connections 2024-09-19T05:01:58.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.526+0000 7fa26be00640 1 -- 172.21.15.43:0/1509996455 wait complete. 2024-09-19T05:01:58.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.528+0000 7fa26be00640 1 Processor -- start 2024-09-19T05:01:58.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.528+0000 7fa26be00640 1 -- start start 2024-09-19T05:01:58.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.528+0000 7fa26be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa26c076720 0x7fa26c1a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:58.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.528+0000 7fa26be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa26c077040 0x7fa26c1a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:58.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.528+0000 7fa26be00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa26c112870 0x7fa26c19e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:58.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.528+0000 7fa26be00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa26c118420 con 0x7fa26c076720 2024-09-19T05:01:58.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.528+0000 7fa26be00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa26c1182a0 con 0x7fa26c112870 2024-09-19T05:01:58.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.528+0000 7fa26be00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa26c1185a0 con 0x7fa26c077040 2024-09-19T05:01:58.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.528+0000 7fa26aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa26c076720 0x7fa26c1a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:58.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.528+0000 7fa26a000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa26c077040 0x7fa26c1a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:58.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.528+0000 7fa26aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa26c076720 0x7fa26c1a4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43464/0 (socket says 172.21.15.43:43464) 2024-09-19T05:01:58.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.528+0000 7fa26aa00640 1 -- 172.21.15.43:0/221509161 learned_addr learned my addr 172.21.15.43:0/221509161 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:01:58.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.529+0000 7fa26b400640 1 --2- 172.21.15.43:0/221509161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa26c112870 0x7fa26c19e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:58.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.529+0000 7fa26a000640 1 -- 172.21.15.43:0/221509161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa26c112870 msgr2=0x7fa26c19e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:58.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.529+0000 7fa26a000640 1 --2- 172.21.15.43:0/221509161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa26c112870 0x7fa26c19e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:58.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.529+0000 7fa26a000640 1 -- 172.21.15.43:0/221509161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa26c076720 msgr2=0x7fa26c1a4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:58.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.529+0000 7fa26a000640 1 --2- 172.21.15.43:0/221509161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa26c076720 0x7fa26c1a4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:58.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.529+0000 7fa26a000640 1 -- 172.21.15.43:0/221509161 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa26c19ef40 con 0x7fa26c077040 2024-09-19T05:01:58.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.529+0000 7fa26b400640 1 --2- 172.21.15.43:0/221509161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa26c112870 0x7fa26c19e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:01:58.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.529+0000 7fa26aa00640 1 --2- 172.21.15.43:0/221509161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa26c076720 0x7fa26c1a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:01:58.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.529+0000 7fa26a000640 1 --2- 172.21.15.43:0/221509161 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa26c077040 0x7fa26c1a4b40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa25800e9f0 tx=0x7fa25800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:01:58.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.529+0000 7fa263400640 1 -- 172.21.15.43:0/221509161 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25800ccc0 con 0x7fa26c077040 2024-09-19T05:01:58.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.529+0000 7fa263400640 1 -- 172.21.15.43:0/221509161 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa258005020 con 0x7fa26c077040 2024-09-19T05:01:58.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.529+0000 7fa263400640 1 -- 172.21.15.43:0/221509161 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2580049e0 con 0x7fa26c077040 2024-09-19T05:01:58.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.529+0000 7fa26be00640 1 -- 172.21.15.43:0/221509161 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa26c19f230 con 0x7fa26c077040 2024-09-19T05:01:58.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.530+0000 7fa26be00640 1 -- 172.21.15.43:0/221509161 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa26c1ab580 con 0x7fa26c077040 2024-09-19T05:01:58.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.531+0000 7fa26be00640 1 -- 172.21.15.43:0/221509161 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa26c077e60 con 0x7fa26c077040 2024-09-19T05:01:58.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.533+0000 7fa263400640 1 -- 172.21.15.43:0/221509161 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fa258005560 con 0x7fa26c077040 2024-09-19T05:01:58.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.534+0000 7fa263400640 1 --2- 172.21.15.43:0/221509161 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa2500a32c0 0x7fa2500a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:01:58.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.534+0000 7fa263400640 1 -- 172.21.15.43:0/221509161 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fa2580cf3e0 con 0x7fa26c077040 2024-09-19T05:01:58.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.534+0000 7fa26aa00640 1 --2- 172.21.15.43:0/221509161 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa2500a32c0 0x7fa2500a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:01:58.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.535+0000 7fa26aa00640 1 --2- 172.21.15.43:0/221509161 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa2500a32c0 0x7fa2500a5780 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa25c005fd0 tx=0x7fa25c00b480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:01:58.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.538+0000 7fa263400640 1 -- 172.21.15.43:0/221509161 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa2580051d0 con 0x7fa26c077040 2024-09-19T05:01:58.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.762+0000 7fa26be00640 1 -- 172.21.15.43:0/221509161 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa26c10da00 con 0x7fa26c077040 2024-09-19T05:01:58.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.763+0000 7fa263400640 1 -- 172.21.15.43:0/221509161 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6388 (secure 0 0 0) 0x7fa25809c500 con 0x7fa26c077040 2024-09-19T05:01:58.764 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:01:58.764 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":8,"btime":"2024-09-19T05:01:58:308027+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/773801169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":773801169},{"type":"v1","addr":"172.21.15.43:6835","nonce":773801169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1799137902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1799137902},{"type":"v1","addr":"172.21.15.43:6837","nonce":1799137902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/2836453158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2836453158},{"type":"v1","addr":"172.21.15.151:6833","nonce":2836453158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/178785812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":178785812},{"type":"v1","addr":"172.21.15.96:6833","nonce":178785812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1671043820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1671043820},{"type":"v1","addr":"172.21.15.96:6835","nonce":1671043820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2972514328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2972514328},{"type":"v1","addr":"172.21.15.151:6835","nonce":2972514328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24493,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/3960378010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3960378010},{"type":"v1","addr":"172.21.15.151:6837","nonce":3960378010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-19T05:01:58.764 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 8 2024-09-19T05:01:58.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.765+0000 7fa26be00640 1 -- 172.21.15.43:0/221509161 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa2500a32c0 msgr2=0x7fa2500a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:58.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.765+0000 7fa26be00640 1 --2- 172.21.15.43:0/221509161 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa2500a32c0 0x7fa2500a5780 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa25c005fd0 tx=0x7fa25c00b480 comp rx=0 tx=0).stop 2024-09-19T05:01:58.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.765+0000 7fa26be00640 1 -- 172.21.15.43:0/221509161 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa26c077040 msgr2=0x7fa26c1a4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:01:58.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.765+0000 7fa26be00640 1 --2- 172.21.15.43:0/221509161 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa26c077040 0x7fa26c1a4b40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa25800e9f0 tx=0x7fa25800eeb0 comp rx=0 tx=0).stop 2024-09-19T05:01:58.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.765+0000 7fa26be00640 1 -- 172.21.15.43:0/221509161 shutdown_connections 2024-09-19T05:01:58.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.765+0000 7fa26be00640 1 --2- 172.21.15.43:0/221509161 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa2500a32c0 0x7fa2500a5780 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:58.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.765+0000 7fa26be00640 1 --2- 172.21.15.43:0/221509161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa26c112870 0x7fa26c19e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:58.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.765+0000 7fa26be00640 1 --2- 172.21.15.43:0/221509161 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa26c077040 0x7fa26c1a4b40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:58.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.765+0000 7fa26be00640 1 --2- 172.21.15.43:0/221509161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa26c076720 0x7fa26c1a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:01:58.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.765+0000 7fa26be00640 1 -- 172.21.15.43:0/221509161 >> 172.21.15.43:0/221509161 conn(0x7fa26c0ff910 msgr2=0x7fa26c101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:01:58.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.765+0000 7fa26be00640 1 -- 172.21.15.43:0/221509161 shutdown_connections 2024-09-19T05:01:58.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:01:58.766+0000 7fa26be00640 1 -- 172.21.15.43:0/221509161 wait complete. 2024-09-19T05:01:59.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:58 smithi043 ceph-mon[30800]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:59.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:58 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:59.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:58 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:59.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:58 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:59.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:58 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:01:59.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:58 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:01:59.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:58 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:59.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:58 smithi043 ceph-mon[30800]: Deploying daemon mds.h on smithi096 2024-09-19T05:01:59.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:58 smithi043 ceph-mon[30800]: mds.? [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] up:boot 2024-09-19T05:01:59.032 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:58 smithi043 ceph-mon[30800]: fsmap 7 up:standby 2024-09-19T05:01:59.032 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:58 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T05:01:59.032 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:01:58 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/221509161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:01:59.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:58 smithi096 ceph-mon[35085]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:59.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:58 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:59.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:58 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:59.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:58 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:59.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:58 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:01:59.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:58 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:01:59.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:58 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:59.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:58 smithi096 ceph-mon[35085]: Deploying daemon mds.h on smithi096 2024-09-19T05:01:59.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:58 smithi096 ceph-mon[35085]: mds.? [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] up:boot 2024-09-19T05:01:59.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:58 smithi096 ceph-mon[35085]: fsmap 7 up:standby 2024-09-19T05:01:59.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:58 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T05:01:59.225 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:01:58 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/221509161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:01:59.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:58 smithi151 ceph-mon[35425]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:01:59.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:58 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:59.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:58 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:59.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:58 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:01:59.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:58 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:01:59.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:58 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:01:59.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:58 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:01:59.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:58 smithi151 ceph-mon[35425]: Deploying daemon mds.h on smithi096 2024-09-19T05:01:59.407 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:58 smithi151 ceph-mon[35425]: mds.? [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] up:boot 2024-09-19T05:01:59.407 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:58 smithi151 ceph-mon[35425]: fsmap 7 up:standby 2024-09-19T05:01:59.407 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:58 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T05:01:59.407 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:01:58 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/221509161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:02:00.131 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 19 05:01:59 smithi096 systemd[1]: Starting Ceph mds.h for 1456a810-7643-11ef-bceb-c7b262605968... 2024-09-19T05:02:00.432 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 19 05:02:00 smithi096 podman[64850]: 2024-09-19 05:02:00.032676825 +0000 UTC m=+0.016127654 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:02:00.432 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 19 05:02:00 smithi096 podman[64850]: 2024-09-19 05:02:00.145423723 +0000 UTC m=+0.128874554 container create 3bd8c04d61ff9f3d9ac7ab0c52ea54f582e55d3ac025a72c58ee5ddcc5b7676e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-h, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-19T05:02:00.433 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 19 05:02:00 smithi096 podman[64850]: 2024-09-19 05:02:00.431767423 +0000 UTC m=+0.415218251 container init 3bd8c04d61ff9f3d9ac7ab0c52ea54f582e55d3ac025a72c58ee5ddcc5b7676e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-h, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_CLEAN=True) 2024-09-19T05:02:00.723 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 19 05:02:00 smithi096 podman[64850]: 2024-09-19 05:02:00.43457653 +0000 UTC m=+0.418027359 container start 3bd8c04d61ff9f3d9ac7ab0c52ea54f582e55d3ac025a72c58ee5ddcc5b7676e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-h, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-19T05:02:00.723 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 19 05:02:00 smithi096 ceph-mds[64869]: -- 172.21.15.96:0/63828300 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5603bb281860 con 0x5603bb25e800 2024-09-19T05:02:00.723 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 19 05:02:00 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-h[64865]: starting mds.h at 2024-09-19T05:02:00.723 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 19 05:02:00 smithi096 bash[64850]: 3bd8c04d61ff9f3d9ac7ab0c52ea54f582e55d3ac025a72c58ee5ddcc5b7676e 2024-09-19T05:02:00.723 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 19 05:02:00 smithi096 systemd[1]: Started Ceph mds.h for 1456a810-7643-11ef-bceb-c7b262605968. 2024-09-19T05:02:01.172 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:00 smithi151 ceph-mon[35425]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:01.172 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:00 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:01.172 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:00 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:01.172 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:00 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:01.172 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:00 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:02:01.172 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:00 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:02:01.172 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:00 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:01.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:00 smithi096 ceph-mon[35085]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:01.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:00 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:01.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:00 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:01.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:00 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:01.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:00 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:02:01.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:00 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:02:01.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:00 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:01.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:00 smithi043 ceph-mon[30800]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:01.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:00 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:01.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:00 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:01.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:00 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:01.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:00 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:02:01.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:00 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:02:01.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:00 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:01.653 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:02:02.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:01 smithi043 ceph-mon[30800]: Deploying daemon mds.i on smithi151 2024-09-19T05:02:02.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:01 smithi043 ceph-mon[30800]: mds.? [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] up:boot 2024-09-19T05:02:02.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:01 smithi043 ceph-mon[30800]: fsmap 8 up:standby 2024-09-19T05:02:02.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:01 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T05:02:02.044 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:02:02.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:01 smithi096 ceph-mon[35085]: Deploying daemon mds.i on smithi151 2024-09-19T05:02:02.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:01 smithi096 ceph-mon[35085]: mds.? [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] up:boot 2024-09-19T05:02:02.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:01 smithi096 ceph-mon[35085]: fsmap 8 up:standby 2024-09-19T05:02:02.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:01 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T05:02:02.253 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:01 smithi151 ceph-mon[35425]: Deploying daemon mds.i on smithi151 2024-09-19T05:02:02.253 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:01 smithi151 ceph-mon[35425]: mds.? [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] up:boot 2024-09-19T05:02:02.253 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:01 smithi151 ceph-mon[35425]: fsmap 8 up:standby 2024-09-19T05:02:02.254 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:01 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T05:02:02.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.577+0000 7f01f6000640 1 -- 172.21.15.43:0/3914529564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f0103ff0 msgr2=0x7f01f01043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:02.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.577+0000 7f01f6000640 1 --2- 172.21.15.43:0/3914529564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f0103ff0 0x7f01f01043d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f01e0009900 tx=0x7f01e002a3f0 comp rx=0 tx=0).stop 2024-09-19T05:02:02.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.577+0000 7f01f6000640 1 -- 172.21.15.43:0/3914529564 shutdown_connections 2024-09-19T05:02:02.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.577+0000 7f01f6000640 1 --2- 172.21.15.43:0/3914529564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01f0109120 0x7f01f0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:02.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.577+0000 7f01f6000640 1 --2- 172.21.15.43:0/3914529564 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01f01049a0 0x7f01f01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:02.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.577+0000 7f01f6000640 1 --2- 172.21.15.43:0/3914529564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f0103ff0 0x7f01f01043d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:02.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.577+0000 7f01f6000640 1 -- 172.21.15.43:0/3914529564 >> 172.21.15.43:0/3914529564 conn(0x7f01f00ffa90 msgr2=0x7f01f0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:02.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.578+0000 7f01f6000640 1 -- 172.21.15.43:0/3914529564 shutdown_connections 2024-09-19T05:02:02.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.578+0000 7f01f6000640 1 -- 172.21.15.43:0/3914529564 wait complete. 2024-09-19T05:02:02.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.578+0000 7f01f6000640 1 Processor -- start 2024-09-19T05:02:02.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.579+0000 7f01f6000640 1 -- start start 2024-09-19T05:02:02.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.579+0000 7f01f6000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f0103ff0 0x7f01f019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:02.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.579+0000 7f01f6000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01f01049a0 0x7f01f019abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:02.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.579+0000 7f01f6000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01f0109120 0x7f01f019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:02.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.579+0000 7f01f6000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f01f0114170 con 0x7f01f0103ff0 2024-09-19T05:02:02.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.579+0000 7f01f6000640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f01f0113ff0 con 0x7f01f01049a0 2024-09-19T05:02:02.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.579+0000 7f01f6000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f01f01142f0 con 0x7f01f0109120 2024-09-19T05:02:02.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.579+0000 7f01ef400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f0103ff0 0x7f01f019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:02.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.579+0000 7f01ef400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f0103ff0 0x7f01f019a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43480/0 (socket says 172.21.15.43:43480) 2024-09-19T05:02:02.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.579+0000 7f01ef400640 1 -- 172.21.15.43:0/2549785545 learned_addr learned my addr 172.21.15.43:0/2549785545 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:02:02.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.579+0000 7f01efe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01f0109120 0x7f01f019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:02.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.579+0000 7f01eea00640 1 --2- 172.21.15.43:0/2549785545 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01f01049a0 0x7f01f019abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:02.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.580+0000 7f01efe00640 1 -- 172.21.15.43:0/2549785545 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01f01049a0 msgr2=0x7f01f019abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:02.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.580+0000 7f01efe00640 1 --2- 172.21.15.43:0/2549785545 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01f01049a0 0x7f01f019abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:02.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.580+0000 7f01efe00640 1 -- 172.21.15.43:0/2549785545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f0103ff0 msgr2=0x7f01f019a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:02.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.580+0000 7f01efe00640 1 --2- 172.21.15.43:0/2549785545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f0103ff0 0x7f01f019a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:02.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.580+0000 7f01efe00640 1 -- 172.21.15.43:0/2549785545 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f019f640 con 0x7f01f0109120 2024-09-19T05:02:02.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.580+0000 7f01efe00640 1 --2- 172.21.15.43:0/2549785545 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01f0109120 0x7f01f019ef60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f01e800da10 tx=0x7f01e800ded0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:02.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.580+0000 7f01e7e00640 1 -- 172.21.15.43:0/2549785545 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e800bd70 con 0x7f01f0109120 2024-09-19T05:02:02.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.580+0000 7f01f6000640 1 -- 172.21.15.43:0/2549785545 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01f019f930 con 0x7f01f0109120 2024-09-19T05:02:02.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.580+0000 7f01e7e00640 1 -- 172.21.15.43:0/2549785545 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01e8003e50 con 0x7f01f0109120 2024-09-19T05:02:02.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.580+0000 7f01e7e00640 1 -- 172.21.15.43:0/2549785545 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e80106a0 con 0x7f01f0109120 2024-09-19T05:02:02.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.580+0000 7f01f6000640 1 -- 172.21.15.43:0/2549785545 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01f01a7210 con 0x7f01f0109120 2024-09-19T05:02:02.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.581+0000 7f01f6000640 1 -- 172.21.15.43:0/2549785545 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01f01043d0 con 0x7f01f0109120 2024-09-19T05:02:02.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.583+0000 7f01e7e00640 1 -- 172.21.15.43:0/2549785545 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f01e8003970 con 0x7f01f0109120 2024-09-19T05:02:02.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.584+0000 7f01e7e00640 1 --2- 172.21.15.43:0/2549785545 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f01d40a3240 0x7f01d40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:02.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.585+0000 7f01e7e00640 1 -- 172.21.15.43:0/2549785545 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f01e80cf850 con 0x7f01f0109120 2024-09-19T05:02:02.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.585+0000 7f01ef400640 1 --2- 172.21.15.43:0/2549785545 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f01d40a3240 0x7f01d40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:02.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.585+0000 7f01ef400640 1 --2- 172.21.15.43:0/2549785545 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f01d40a3240 0x7f01d40a5700 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f01e00096c0 tx=0x7f01e00046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:02.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.588+0000 7f01e7e00640 1 -- 172.21.15.43:0/2549785545 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f01e809c970 con 0x7f01f0109120 2024-09-19T05:02:02.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.810+0000 7f01f6000640 1 -- 172.21.15.43:0/2549785545 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01f002d7c0 con 0x7f01f0109120 2024-09-19T05:02:02.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.811+0000 7f01e7e00640 1 -- 172.21.15.43:0/2549785545 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7213 (secure 0 0 0) 0x7f01e809c270 con 0x7f01f0109120 2024-09-19T05:02:02.812 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:02:02.812 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":9,"btime":"2024-09-19T05:02:00:913957+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/773801169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":773801169},{"type":"v1","addr":"172.21.15.43:6835","nonce":773801169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1799137902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1799137902},{"type":"v1","addr":"172.21.15.43:6837","nonce":1799137902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/2836453158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2836453158},{"type":"v1","addr":"172.21.15.151:6833","nonce":2836453158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/178785812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":178785812},{"type":"v1","addr":"172.21.15.96:6833","nonce":178785812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1671043820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1671043820},{"type":"v1","addr":"172.21.15.96:6835","nonce":1671043820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2972514328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2972514328},{"type":"v1","addr":"172.21.15.151:6835","nonce":2972514328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24493,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/3960378010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3960378010},{"type":"v1","addr":"172.21.15.151:6837","nonce":3960378010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/3978388286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3978388286},{"type":"v1","addr":"172.21.15.96:6837","nonce":3978388286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-19T05:02:02.812 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 9 2024-09-19T05:02:02.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.813+0000 7f01f6000640 1 -- 172.21.15.43:0/2549785545 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f01d40a3240 msgr2=0x7f01d40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:02.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.813+0000 7f01f6000640 1 --2- 172.21.15.43:0/2549785545 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f01d40a3240 0x7f01d40a5700 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f01e00096c0 tx=0x7f01e00046c0 comp rx=0 tx=0).stop 2024-09-19T05:02:02.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.813+0000 7f01f6000640 1 -- 172.21.15.43:0/2549785545 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01f0109120 msgr2=0x7f01f019ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:02.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.813+0000 7f01f6000640 1 --2- 172.21.15.43:0/2549785545 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01f0109120 0x7f01f019ef60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f01e800da10 tx=0x7f01e800ded0 comp rx=0 tx=0).stop 2024-09-19T05:02:02.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.813+0000 7f01f6000640 1 -- 172.21.15.43:0/2549785545 shutdown_connections 2024-09-19T05:02:02.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.813+0000 7f01f6000640 1 --2- 172.21.15.43:0/2549785545 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f01d40a3240 0x7f01d40a5700 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:02.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.813+0000 7f01f6000640 1 --2- 172.21.15.43:0/2549785545 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01f0109120 0x7f01f019ef60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:02.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.813+0000 7f01f6000640 1 --2- 172.21.15.43:0/2549785545 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01f01049a0 0x7f01f019abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:02.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.813+0000 7f01f6000640 1 --2- 172.21.15.43:0/2549785545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f0103ff0 0x7f01f019a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:02.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.813+0000 7f01f6000640 1 -- 172.21.15.43:0/2549785545 >> 172.21.15.43:0/2549785545 conn(0x7f01f00ffa90 msgr2=0x7f01f0100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:02.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.813+0000 7f01f6000640 1 -- 172.21.15.43:0/2549785545 shutdown_connections 2024-09-19T05:02:02.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:02.813+0000 7f01f6000640 1 -- 172.21.15.43:0/2549785545 wait complete. 2024-09-19T05:02:02.905 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 05:02:02 smithi151 podman[65771]: 2024-09-19 05:02:02.86346153 +0000 UTC m=+0.128172486 container create 17ab155834124caaf0e12d97e98808ec5bfb213853248f1f254006a4fb83b975 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-i, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-2f599a4) 2024-09-19T05:02:03.210 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:02 smithi043 ceph-mon[30800]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:03.210 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:02 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/2549785545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:02:03.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:02 smithi096 ceph-mon[35085]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:03.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:02 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/2549785545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:02:03.225 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:02 smithi151 ceph-mon[35425]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:03.225 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:02 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/2549785545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:02:03.226 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 05:02:03 smithi151 podman[65771]: 2024-09-19 05:02:03.116333109 +0000 UTC m=+0.381044055 container init 17ab155834124caaf0e12d97e98808ec5bfb213853248f1f254006a4fb83b975 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-i, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-19T05:02:03.226 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 05:02:03 smithi151 podman[65771]: 2024-09-19 05:02:03.119247496 +0000 UTC m=+0.383958447 container start 17ab155834124caaf0e12d97e98808ec5bfb213853248f1f254006a4fb83b975 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-19T05:02:03.226 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 05:02:03 smithi151 ceph-mds[65791]: -- 172.21.15.151:0/200297290 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d915583860 con 0x55d915560800 2024-09-19T05:02:03.226 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 05:02:03 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-i[65787]: starting mds.i at 2024-09-19T05:02:03.654 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 05:02:03 smithi151 bash[65771]: 17ab155834124caaf0e12d97e98808ec5bfb213853248f1f254006a4fb83b975 2024-09-19T05:02:03.655 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 05:02:03 smithi151 systemd[1]: Started Ceph mds.i for 1456a810-7643-11ef-bceb-c7b262605968. 2024-09-19T05:02:04.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:04 smithi043 ceph-mon[30800]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:04.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:04 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:04.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:04 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:04.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:04 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:04.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:04 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:02:04.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:04 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:02:04.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:04 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:04.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:04 smithi043 ceph-mon[30800]: Deploying daemon mds.g on smithi043 2024-09-19T05:02:04.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:04 smithi043 ceph-mon[30800]: mds.? [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] up:boot 2024-09-19T05:02:04.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:04 smithi043 ceph-mon[30800]: fsmap 9 up:standby 2024-09-19T05:02:04.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:04 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T05:02:04.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:04 smithi151 ceph-mon[35425]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:04.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:04 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:04.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:04 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:04.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:04 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:04.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:04 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:02:04.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:04 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:02:04.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:04 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:04.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:04 smithi151 ceph-mon[35425]: Deploying daemon mds.g on smithi043 2024-09-19T05:02:04.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:04 smithi151 ceph-mon[35425]: mds.? [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] up:boot 2024-09-19T05:02:04.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:04 smithi151 ceph-mon[35425]: fsmap 9 up:standby 2024-09-19T05:02:04.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:04 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T05:02:04.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:04 smithi096 ceph-mon[35085]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:04.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:04 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:04.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:04 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:04.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:04 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:04.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:04 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:02:04.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:04 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:02:04.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:04 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:04.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:04 smithi096 ceph-mon[35085]: Deploying daemon mds.g on smithi043 2024-09-19T05:02:04.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:04 smithi096 ceph-mon[35085]: mds.? [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] up:boot 2024-09-19T05:02:04.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:04 smithi096 ceph-mon[35085]: fsmap 9 up:standby 2024-09-19T05:02:04.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:04 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T05:02:05.626 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:02:06.278 INFO:journalctl@ceph.mds.g.smithi043.stdout:Sep 19 05:02:06 smithi043 podman[69258]: 2024-09-19 05:02:06.021394812 +0000 UTC m=+0.136343341 container create c1495dc06331c24efa0347774080a4dd469b732e38411afe17185eed9faae0b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-g, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-19T05:02:06.493 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:02:06.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:06 smithi043 ceph-mon[30800]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:06.530 INFO:journalctl@ceph.mds.g.smithi043.stdout:Sep 19 05:02:06 smithi043 podman[69258]: 2024-09-19 05:02:06.299314483 +0000 UTC m=+0.414263023 container init c1495dc06331c24efa0347774080a4dd469b732e38411afe17185eed9faae0b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-g, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T05:02:06.531 INFO:journalctl@ceph.mds.g.smithi043.stdout:Sep 19 05:02:06 smithi043 podman[69258]: 2024-09-19 05:02:06.305993055 +0000 UTC m=+0.420941584 container start c1495dc06331c24efa0347774080a4dd469b732e38411afe17185eed9faae0b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-g, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-19T05:02:06.531 INFO:journalctl@ceph.mds.g.smithi043.stdout:Sep 19 05:02:06 smithi043 ceph-mds[69300]: -- 172.21.15.43:0/1496669437 <== mon.0 v2:172.21.15.43:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56243b29b860 con 0x56243b279400 2024-09-19T05:02:06.531 INFO:journalctl@ceph.mds.g.smithi043.stdout:Sep 19 05:02:06 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-g[69296]: starting mds.g at 2024-09-19T05:02:06.531 INFO:journalctl@ceph.mds.g.smithi043.stdout:Sep 19 05:02:06 smithi043 bash[69258]: c1495dc06331c24efa0347774080a4dd469b732e38411afe17185eed9faae0b0 2024-09-19T05:02:06.531 INFO:journalctl@ceph.mds.g.smithi043.stdout:Sep 19 05:02:06 smithi043 systemd[1]: Started Ceph mds.g for 1456a810-7643-11ef-bceb-c7b262605968. 2024-09-19T05:02:06.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:06 smithi151 ceph-mon[35425]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:06.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:06 smithi096 ceph-mon[35085]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:07.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.029+0000 7fc165a00640 1 -- 172.21.15.43:0/977156811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc160103e30 msgr2=0x7fc160104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:07.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.029+0000 7fc165a00640 1 --2- 172.21.15.43:0/977156811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc160103e30 0x7fc160104210 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc150009860 tx=0x7fc15002a340 comp rx=0 tx=0).stop 2024-09-19T05:02:07.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.029+0000 7fc165a00640 1 -- 172.21.15.43:0/977156811 shutdown_connections 2024-09-19T05:02:07.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.029+0000 7fc165a00640 1 --2- 172.21.15.43:0/977156811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc160108f60 0x7fc160110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:07.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.029+0000 7fc165a00640 1 --2- 172.21.15.43:0/977156811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1601047e0 0x7fc160108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:07.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.029+0000 7fc165a00640 1 --2- 172.21.15.43:0/977156811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc160103e30 0x7fc160104210 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:07.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.029+0000 7fc165a00640 1 -- 172.21.15.43:0/977156811 >> 172.21.15.43:0/977156811 conn(0x7fc1600ff910 msgr2=0x7fc160101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:07.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.029+0000 7fc165a00640 1 -- 172.21.15.43:0/977156811 shutdown_connections 2024-09-19T05:02:07.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.029+0000 7fc165a00640 1 -- 172.21.15.43:0/977156811 wait complete. 2024-09-19T05:02:07.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.030+0000 7fc165a00640 1 Processor -- start 2024-09-19T05:02:07.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.030+0000 7fc165a00640 1 -- start start 2024-09-19T05:02:07.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.030+0000 7fc165a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc160103e30 0x7fc16019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:07.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.030+0000 7fc165a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1601047e0 0x7fc16019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:07.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.030+0000 7fc165a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc160108f60 0x7fc16019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:07.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.030+0000 7fc165a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc160113fc0 con 0x7fc1601047e0 2024-09-19T05:02:07.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.030+0000 7fc165a00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc160113e40 con 0x7fc160108f60 2024-09-19T05:02:07.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.030+0000 7fc165a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc160114140 con 0x7fc160103e30 2024-09-19T05:02:07.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.030+0000 7fc15f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1601047e0 0x7fc16019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:07.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.030+0000 7fc15f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1601047e0 0x7fc16019a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44976/0 (socket says 172.21.15.43:44976) 2024-09-19T05:02:07.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.030+0000 7fc15f400640 1 -- 172.21.15.43:0/3075737070 learned_addr learned my addr 172.21.15.43:0/3075737070 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:02:07.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.031+0000 7fc15fe00640 1 --2- 172.21.15.43:0/3075737070 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc160103e30 0x7fc16019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:07.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.031+0000 7fc165000640 1 --2- 172.21.15.43:0/3075737070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc160108f60 0x7fc16019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:07.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.031+0000 7fc15f400640 1 -- 172.21.15.43:0/3075737070 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc160103e30 msgr2=0x7fc16019a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:07.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.031+0000 7fc15f400640 1 --2- 172.21.15.43:0/3075737070 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc160103e30 0x7fc16019a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:07.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.031+0000 7fc15f400640 1 -- 172.21.15.43:0/3075737070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc160108f60 msgr2=0x7fc16019ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:07.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.031+0000 7fc15f400640 1 --2- 172.21.15.43:0/3075737070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc160108f60 0x7fc16019ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:07.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.031+0000 7fc15f400640 1 -- 172.21.15.43:0/3075737070 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc16019f430 con 0x7fc1601047e0 2024-09-19T05:02:07.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.031+0000 7fc15f400640 1 --2- 172.21.15.43:0/3075737070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1601047e0 0x7fc16019a9c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc14c00e9c0 tx=0x7fc14c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:07.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.031+0000 7fc15cc00640 1 -- 172.21.15.43:0/3075737070 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc14c00cde0 con 0x7fc1601047e0 2024-09-19T05:02:07.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.031+0000 7fc15cc00640 1 -- 172.21.15.43:0/3075737070 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc14c0035f0 con 0x7fc1601047e0 2024-09-19T05:02:07.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.031+0000 7fc165a00640 1 -- 172.21.15.43:0/3075737070 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc16019f720 con 0x7fc1601047e0 2024-09-19T05:02:07.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.031+0000 7fc15cc00640 1 -- 172.21.15.43:0/3075737070 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc14c010690 con 0x7fc1601047e0 2024-09-19T05:02:07.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.032+0000 7fc165a00640 1 -- 172.21.15.43:0/3075737070 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1601a7000 con 0x7fc1601047e0 2024-09-19T05:02:07.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.033+0000 7fc165a00640 1 -- 172.21.15.43:0/3075737070 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc160105310 con 0x7fc1601047e0 2024-09-19T05:02:07.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.035+0000 7fc15cc00640 1 -- 172.21.15.43:0/3075737070 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fc14c0037a0 con 0x7fc1601047e0 2024-09-19T05:02:07.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.037+0000 7fc15cc00640 1 --2- 172.21.15.43:0/3075737070 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fc1440a32c0 0x7fc1440a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:07.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.037+0000 7fc15cc00640 1 -- 172.21.15.43:0/3075737070 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fc14c0cf930 con 0x7fc1601047e0 2024-09-19T05:02:07.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.037+0000 7fc15fe00640 1 --2- 172.21.15.43:0/3075737070 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fc1440a32c0 0x7fc1440a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:07.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.038+0000 7fc15fe00640 1 --2- 172.21.15.43:0/3075737070 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fc1440a32c0 0x7fc1440a5780 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc15002a930 tx=0x7fc1500056d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:07.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.038+0000 7fc15cc00640 1 -- 172.21.15.43:0/3075737070 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc14c09cae0 con 0x7fc1601047e0 2024-09-19T05:02:07.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.273+0000 7fc165a00640 1 -- 172.21.15.43:0/3075737070 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc160104210 con 0x7fc1601047e0 2024-09-19T05:02:07.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.274+0000 7fc15cc00640 1 -- 172.21.15.43:0/3075737070 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8043 (secure 0 0 0) 0x7fc160104210 con 0x7fc1601047e0 2024-09-19T05:02:07.275 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:02:07.275 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":10,"btime":"2024-09-19T05:02:03:928236+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/773801169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":773801169},{"type":"v1","addr":"172.21.15.43:6835","nonce":773801169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1799137902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1799137902},{"type":"v1","addr":"172.21.15.43:6837","nonce":1799137902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/2836453158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2836453158},{"type":"v1","addr":"172.21.15.151:6833","nonce":2836453158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/178785812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":178785812},{"type":"v1","addr":"172.21.15.96:6833","nonce":178785812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1671043820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1671043820},{"type":"v1","addr":"172.21.15.96:6835","nonce":1671043820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2972514328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2972514328},{"type":"v1","addr":"172.21.15.151:6835","nonce":2972514328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24493,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/3960378010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3960378010},{"type":"v1","addr":"172.21.15.151:6837","nonce":3960378010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/3978388286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3978388286},{"type":"v1","addr":"172.21.15.96:6837","nonce":3978388286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/3794104313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3794104313},{"type":"v1","addr":"172.21.15.151:6839","nonce":3794104313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-19T05:02:07.275 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 10 2024-09-19T05:02:07.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.276+0000 7fc165a00640 1 -- 172.21.15.43:0/3075737070 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fc1440a32c0 msgr2=0x7fc1440a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:07.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.276+0000 7fc165a00640 1 --2- 172.21.15.43:0/3075737070 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fc1440a32c0 0x7fc1440a5780 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc15002a930 tx=0x7fc1500056d0 comp rx=0 tx=0).stop 2024-09-19T05:02:07.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.276+0000 7fc165a00640 1 -- 172.21.15.43:0/3075737070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1601047e0 msgr2=0x7fc16019a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:07.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.276+0000 7fc165a00640 1 --2- 172.21.15.43:0/3075737070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1601047e0 0x7fc16019a9c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc14c00e9c0 tx=0x7fc14c00ee80 comp rx=0 tx=0).stop 2024-09-19T05:02:07.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.276+0000 7fc165a00640 1 -- 172.21.15.43:0/3075737070 shutdown_connections 2024-09-19T05:02:07.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.276+0000 7fc165a00640 1 --2- 172.21.15.43:0/3075737070 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fc1440a32c0 0x7fc1440a5780 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:07.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.276+0000 7fc165a00640 1 --2- 172.21.15.43:0/3075737070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc160108f60 0x7fc16019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:07.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.276+0000 7fc165a00640 1 --2- 172.21.15.43:0/3075737070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1601047e0 0x7fc16019a9c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:07.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.276+0000 7fc165a00640 1 --2- 172.21.15.43:0/3075737070 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc160103e30 0x7fc16019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:07.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.276+0000 7fc165a00640 1 -- 172.21.15.43:0/3075737070 >> 172.21.15.43:0/3075737070 conn(0x7fc1600ff910 msgr2=0x7fc160100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:07.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.277+0000 7fc165a00640 1 -- 172.21.15.43:0/3075737070 shutdown_connections 2024-09-19T05:02:07.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:07.277+0000 7fc165a00640 1 -- 172.21.15.43:0/3075737070 wait complete. 2024-09-19T05:02:07.515 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:07 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:07.516 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:07 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:07.516 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:07 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:07.516 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:07 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:02:07.516 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:07 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:02:07.516 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:07 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:07.516 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:07 smithi043 ceph-mon[30800]: Deploying daemon mds.a on smithi043 2024-09-19T05:02:07.516 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:07 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/3075737070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:02:07.516 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:07 smithi043 ceph-mon[30800]: mds.? [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] up:boot 2024-09-19T05:02:07.517 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:07 smithi043 ceph-mon[30800]: fsmap 10 up:standby 2024-09-19T05:02:07.517 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:07 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T05:02:07.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:07 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:07.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:07 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:07.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:07 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:07.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:07 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:02:07.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:07 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:02:07.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:07 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:07.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:07 smithi151 ceph-mon[35425]: Deploying daemon mds.a on smithi043 2024-09-19T05:02:07.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:07 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/3075737070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:02:07.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:07 smithi151 ceph-mon[35425]: mds.? [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] up:boot 2024-09-19T05:02:07.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:07 smithi151 ceph-mon[35425]: fsmap 10 up:standby 2024-09-19T05:02:07.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:07 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T05:02:07.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:07 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:07.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:07 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:07.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:07 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:07.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:07 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:02:07.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:07 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:02:07.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:07 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:07.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:07 smithi096 ceph-mon[35085]: Deploying daemon mds.a on smithi043 2024-09-19T05:02:07.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:07 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/3075737070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:02:07.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:07 smithi096 ceph-mon[35085]: mds.? [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] up:boot 2024-09-19T05:02:07.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:07 smithi096 ceph-mon[35085]: fsmap 10 up:standby 2024-09-19T05:02:07.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:07 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T05:02:08.683 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:08 smithi043 ceph-mon[30800]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:08.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:08 smithi151 ceph-mon[35425]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:08.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:08 smithi096 ceph-mon[35085]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:10.009 INFO:journalctl@ceph.mds.a.smithi043.stdout:Sep 19 05:02:09 smithi043 systemd[1]: Starting Ceph mds.a for 1456a810-7643-11ef-bceb-c7b262605968... 2024-09-19T05:02:10.109 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:02:10.281 INFO:journalctl@ceph.mds.a.smithi043.stdout:Sep 19 05:02:10 smithi043 podman[69648]: 2024-09-19 05:02:09.913164694 +0000 UTC m=+0.027875467 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:02:10.281 INFO:journalctl@ceph.mds.a.smithi043.stdout:Sep 19 05:02:10 smithi043 podman[69648]: 2024-09-19 05:02:10.018717581 +0000 UTC m=+0.133428335 container create d6a90fee2ab115c5f79e1f4af0de8da4550119b6cadb392c90dc37da2f4a72f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-a, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-19T05:02:10.281 INFO:journalctl@ceph.mds.a.smithi043.stdout:Sep 19 05:02:10 smithi043 podman[69648]: 2024-09-19 05:02:10.246895077 +0000 UTC m=+0.361605829 container init d6a90fee2ab115c5f79e1f4af0de8da4550119b6cadb392c90dc37da2f4a72f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-a, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS) 2024-09-19T05:02:10.281 INFO:journalctl@ceph.mds.a.smithi043.stdout:Sep 19 05:02:10 smithi043 podman[69648]: 2024-09-19 05:02:10.253467032 +0000 UTC m=+0.368177793 container start d6a90fee2ab115c5f79e1f4af0de8da4550119b6cadb392c90dc37da2f4a72f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-a, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-19T05:02:10.518 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:02:10.660 INFO:journalctl@ceph.mds.a.smithi043.stdout:Sep 19 05:02:10 smithi043 ceph-mds[69684]: -- 172.21.15.43:0/2350134202 <== mon.0 v2:172.21.15.43:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555eb4cdbc20 con 0x555eb4cb9000 2024-09-19T05:02:10.660 INFO:journalctl@ceph.mds.a.smithi043.stdout:Sep 19 05:02:10 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-a[69678]: starting mds.a at 2024-09-19T05:02:10.660 INFO:journalctl@ceph.mds.a.smithi043.stdout:Sep 19 05:02:10 smithi043 bash[69648]: d6a90fee2ab115c5f79e1f4af0de8da4550119b6cadb392c90dc37da2f4a72f0 2024-09-19T05:02:10.660 INFO:journalctl@ceph.mds.a.smithi043.stdout:Sep 19 05:02:10 smithi043 systemd[1]: Started Ceph mds.a for 1456a810-7643-11ef-bceb-c7b262605968. 2024-09-19T05:02:10.660 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:10 smithi043 ceph-mon[30800]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:10.661 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:10 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:10.661 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:10 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:10.661 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:10 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:10.661 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:10 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:02:10.661 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:10 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:02:10.661 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:10 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:10.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:10 smithi151 ceph-mon[35425]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:10.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:10 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:10.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:10 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:10.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:10 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:10.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:10 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:02:10.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:10 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:02:10.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:10 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:10.911 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:10 smithi096 ceph-mon[35085]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:10.911 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:10 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:10.911 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:10 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:10.911 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:10 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:10.912 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:10 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T05:02:10.912 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:10 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T05:02:10.912 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:10 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:11.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.055+0000 7f2a41e00640 1 -- 172.21.15.43:0/3572512328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a3c10e3c0 msgr2=0x7f2a3c110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:11.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.055+0000 7f2a41e00640 1 --2- 172.21.15.43:0/3572512328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a3c10e3c0 0x7f2a3c110810 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2a34009ca0 tx=0x7f2a3402a950 comp rx=0 tx=0).stop 2024-09-19T05:02:11.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.056+0000 7f2a41e00640 1 -- 172.21.15.43:0/3572512328 shutdown_connections 2024-09-19T05:02:11.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.056+0000 7f2a41e00640 1 --2- 172.21.15.43:0/3572512328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a3c10e3c0 0x7f2a3c110810 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:11.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.056+0000 7f2a41e00640 1 --2- 172.21.15.43:0/3572512328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a3c074f40 0x7f2a3c10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:11.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.056+0000 7f2a41e00640 1 --2- 172.21.15.43:0/3572512328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a3c074620 0x7f2a3c074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:11.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.056+0000 7f2a41e00640 1 -- 172.21.15.43:0/3572512328 >> 172.21.15.43:0/3572512328 conn(0x7f2a3c0fd8c0 msgr2=0x7f2a3c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:11.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.056+0000 7f2a41e00640 1 -- 172.21.15.43:0/3572512328 shutdown_connections 2024-09-19T05:02:11.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.056+0000 7f2a41e00640 1 -- 172.21.15.43:0/3572512328 wait complete. 2024-09-19T05:02:11.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.057+0000 7f2a41e00640 1 Processor -- start 2024-09-19T05:02:11.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.057+0000 7f2a41e00640 1 -- start start 2024-09-19T05:02:11.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.057+0000 7f2a41e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a3c074620 0x7f2a3c19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:11.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.057+0000 7f2a41e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a3c074f40 0x7f2a3c19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:11.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.057+0000 7f2a41e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a3c10e3c0 0x7f2a3c19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:11.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.057+0000 7f2a41e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2a3c1141c0 con 0x7f2a3c074f40 2024-09-19T05:02:11.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.057+0000 7f2a41e00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2a3c114040 con 0x7f2a3c10e3c0 2024-09-19T05:02:11.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.057+0000 7f2a41e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2a3c114340 con 0x7f2a3c074620 2024-09-19T05:02:11.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.057+0000 7f2a33e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a3c074f40 0x7f2a3c19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:11.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.058+0000 7f2a3b400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a3c10e3c0 0x7f2a3c19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:11.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.057+0000 7f2a33e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a3c074f40 0x7f2a3c19aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45008/0 (socket says 172.21.15.43:45008) 2024-09-19T05:02:11.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.058+0000 7f2a33e00640 1 -- 172.21.15.43:0/762208465 learned_addr learned my addr 172.21.15.43:0/762208465 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:02:11.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.058+0000 7f2a3aa00640 1 --2- 172.21.15.43:0/762208465 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a3c074620 0x7f2a3c19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:11.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.058+0000 7f2a3b400640 1 -- 172.21.15.43:0/762208465 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a3c074620 msgr2=0x7f2a3c19a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:11.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.058+0000 7f2a3b400640 1 --2- 172.21.15.43:0/762208465 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a3c074620 0x7f2a3c19a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:11.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.058+0000 7f2a3b400640 1 -- 172.21.15.43:0/762208465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a3c074f40 msgr2=0x7f2a3c19aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:11.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.058+0000 7f2a3b400640 1 --2- 172.21.15.43:0/762208465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a3c074f40 0x7f2a3c19aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:11.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.058+0000 7f2a3b400640 1 -- 172.21.15.43:0/762208465 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a3c19f610 con 0x7f2a3c10e3c0 2024-09-19T05:02:11.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.058+0000 7f2a33e00640 1 --2- 172.21.15.43:0/762208465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a3c074f40 0x7f2a3c19aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:02:11.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.058+0000 7f2a3b400640 1 --2- 172.21.15.43:0/762208465 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a3c10e3c0 0x7f2a3c19ef30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2a34005e30 tx=0x7f2a34003b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:11.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.059+0000 7f2a40a00640 1 -- 172.21.15.43:0/762208465 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a34034040 con 0x7f2a3c10e3c0 2024-09-19T05:02:11.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.059+0000 7f2a40a00640 1 -- 172.21.15.43:0/762208465 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a3402f940 con 0x7f2a3c10e3c0 2024-09-19T05:02:11.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.059+0000 7f2a41e00640 1 -- 172.21.15.43:0/762208465 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a3c19f8a0 con 0x7f2a3c10e3c0 2024-09-19T05:02:11.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.059+0000 7f2a40a00640 1 -- 172.21.15.43:0/762208465 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a340045e0 con 0x7f2a3c10e3c0 2024-09-19T05:02:11.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.059+0000 7f2a41e00640 1 -- 172.21.15.43:0/762208465 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a3c1a71e0 con 0x7f2a3c10e3c0 2024-09-19T05:02:11.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.060+0000 7f2a41e00640 1 -- 172.21.15.43:0/762208465 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a3c074a00 con 0x7f2a3c10e3c0 2024-09-19T05:02:11.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.061+0000 7f2a40a00640 1 -- 172.21.15.43:0/762208465 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f2a34037020 con 0x7f2a3c10e3c0 2024-09-19T05:02:11.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.063+0000 7f2a40a00640 1 --2- 172.21.15.43:0/762208465 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f2a200a32c0 0x7f2a200a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:11.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.063+0000 7f2a40a00640 1 -- 172.21.15.43:0/762208465 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f2a340f0130 con 0x7f2a3c10e3c0 2024-09-19T05:02:11.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.063+0000 7f2a3aa00640 1 --2- 172.21.15.43:0/762208465 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f2a200a32c0 0x7f2a200a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:11.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.064+0000 7f2a3aa00640 1 --2- 172.21.15.43:0/762208465 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f2a200a32c0 0x7f2a200a5780 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c0095f0 tx=0x7f2a2c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:11.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.067+0000 7f2a40a00640 1 -- 172.21.15.43:0/762208465 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2a340bd250 con 0x7f2a3c10e3c0 2024-09-19T05:02:11.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.289+0000 7f2a41e00640 1 -- 172.21.15.43:0/762208465 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a3c109540 con 0x7f2a3c10e3c0 2024-09-19T05:02:11.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.289+0000 7f2a40a00640 1 -- 172.21.15.43:0/762208465 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9692 (secure 0 0 0) 0x7f2a340bcb50 con 0x7f2a3c10e3c0 2024-09-19T05:02:11.291 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:02:11.291 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":12,"btime":"2024-09-19T05:02:10:521384+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/773801169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":773801169},{"type":"v1","addr":"172.21.15.43:6835","nonce":773801169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1799137902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1799137902},{"type":"v1","addr":"172.21.15.43:6837","nonce":1799137902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/794881135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":794881135},{"type":"v1","addr":"172.21.15.43:6841","nonce":794881135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/2836453158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2836453158},{"type":"v1","addr":"172.21.15.151:6833","nonce":2836453158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/178785812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":178785812},{"type":"v1","addr":"172.21.15.96:6833","nonce":178785812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1671043820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1671043820},{"type":"v1","addr":"172.21.15.96:6835","nonce":1671043820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2972514328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2972514328},{"type":"v1","addr":"172.21.15.151:6835","nonce":2972514328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24493,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/3960378010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3960378010},{"type":"v1","addr":"172.21.15.151:6837","nonce":3960378010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/3978388286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3978388286},{"type":"v1","addr":"172.21.15.96:6837","nonce":3978388286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/3794104313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3794104313},{"type":"v1","addr":"172.21.15.151:6839","nonce":3794104313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24518,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3404555030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3404555030},{"type":"v1","addr":"172.21.15.43:6839","nonce":3404555030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-09-19T05:02:11.291 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 12 2024-09-19T05:02:11.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.292+0000 7f2a41e00640 1 -- 172.21.15.43:0/762208465 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f2a200a32c0 msgr2=0x7f2a200a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:11.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.292+0000 7f2a41e00640 1 --2- 172.21.15.43:0/762208465 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f2a200a32c0 0x7f2a200a5780 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c0095f0 tx=0x7f2a2c005f90 comp rx=0 tx=0).stop 2024-09-19T05:02:11.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.292+0000 7f2a41e00640 1 -- 172.21.15.43:0/762208465 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a3c10e3c0 msgr2=0x7f2a3c19ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:11.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.292+0000 7f2a41e00640 1 --2- 172.21.15.43:0/762208465 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a3c10e3c0 0x7f2a3c19ef30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2a34005e30 tx=0x7f2a34003b80 comp rx=0 tx=0).stop 2024-09-19T05:02:11.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.292+0000 7f2a41e00640 1 -- 172.21.15.43:0/762208465 shutdown_connections 2024-09-19T05:02:11.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.292+0000 7f2a41e00640 1 --2- 172.21.15.43:0/762208465 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f2a200a32c0 0x7f2a200a5780 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:11.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.292+0000 7f2a41e00640 1 --2- 172.21.15.43:0/762208465 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a3c10e3c0 0x7f2a3c19ef30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:11.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.292+0000 7f2a41e00640 1 --2- 172.21.15.43:0/762208465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a3c074f40 0x7f2a3c19aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:11.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.292+0000 7f2a41e00640 1 --2- 172.21.15.43:0/762208465 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a3c074620 0x7f2a3c19a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:11.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.292+0000 7f2a41e00640 1 -- 172.21.15.43:0/762208465 >> 172.21.15.43:0/762208465 conn(0x7f2a3c0fd8c0 msgr2=0x7f2a3c0fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:11.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.292+0000 7f2a41e00640 1 -- 172.21.15.43:0/762208465 shutdown_connections 2024-09-19T05:02:11.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:11.292+0000 7f2a41e00640 1 -- 172.21.15.43:0/762208465 wait complete. 2024-09-19T05:02:11.631 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:11 smithi043 ceph-mon[30800]: Deploying daemon mds.b on smithi096 2024-09-19T05:02:11.631 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:11 smithi043 ceph-mon[30800]: mds.? [v2:172.21.15.43:6840/794881135,v1:172.21.15.43:6841/794881135] up:boot 2024-09-19T05:02:11.631 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:11 smithi043 ceph-mon[30800]: fsmap 11 up:standby 2024-09-19T05:02:11.631 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:11 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T05:02:11.631 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:11 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/762208465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:02:11.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:11 smithi151 ceph-mon[35425]: Deploying daemon mds.b on smithi096 2024-09-19T05:02:11.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:11 smithi151 ceph-mon[35425]: mds.? [v2:172.21.15.43:6840/794881135,v1:172.21.15.43:6841/794881135] up:boot 2024-09-19T05:02:11.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:11 smithi151 ceph-mon[35425]: fsmap 11 up:standby 2024-09-19T05:02:11.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:11 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T05:02:11.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:11 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/762208465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:02:11.936 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:11 smithi096 ceph-mon[35085]: Deploying daemon mds.b on smithi096 2024-09-19T05:02:11.936 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:11 smithi096 ceph-mon[35085]: mds.? [v2:172.21.15.43:6840/794881135,v1:172.21.15.43:6841/794881135] up:boot 2024-09-19T05:02:11.936 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:11 smithi096 ceph-mon[35085]: fsmap 11 up:standby 2024-09-19T05:02:11.936 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:11 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T05:02:11.936 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:11 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/762208465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:02:12.473 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 05:02:12 smithi096 systemd[1]: Starting Ceph mds.b for 1456a810-7643-11ef-bceb-c7b262605968... 2024-09-19T05:02:12.877 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:12 smithi096 ceph-mon[35085]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:12.878 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 05:02:12 smithi096 podman[65149]: 2024-09-19 05:02:12.512620979 +0000 UTC m=+0.015695563 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:02:12.878 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 05:02:12 smithi096 podman[65149]: 2024-09-19 05:02:12.624667286 +0000 UTC m=+0.127741865 container create 1102336ec1069b3325bc3a082bdb99c2d8b7f553d7840070373005006062688c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-b, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T05:02:12.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:12 smithi151 ceph-mon[35425]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:13.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:12 smithi043 ceph-mon[30800]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:13.223 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 05:02:12 smithi096 podman[65149]: 2024-09-19 05:02:12.877226461 +0000 UTC m=+0.380301037 container init 1102336ec1069b3325bc3a082bdb99c2d8b7f553d7840070373005006062688c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-b, ceph=True, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-19T05:02:13.223 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 05:02:12 smithi096 podman[65149]: 2024-09-19 05:02:12.880221504 +0000 UTC m=+0.383296083 container start 1102336ec1069b3325bc3a082bdb99c2d8b7f553d7840070373005006062688c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-19T05:02:13.223 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 05:02:12 smithi096 ceph-mds[65168]: -- 172.21.15.96:0/2289185167 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5587b7ded860 con 0x5587b7dcb400 2024-09-19T05:02:13.223 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 05:02:12 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-b[65164]: starting mds.b at 2024-09-19T05:02:13.223 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 05:02:12 smithi096 bash[65149]: 1102336ec1069b3325bc3a082bdb99c2d8b7f553d7840070373005006062688c 2024-09-19T05:02:13.223 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 05:02:12 smithi096 systemd[1]: Started Ceph mds.b for 1456a810-7643-11ef-bceb-c7b262605968. 2024-09-19T05:02:13.986 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:02:14.081 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:14 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:14.081 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:14 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:14.081 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:14 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:14.081 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:14 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:14.081 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:14 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:14.081 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:14 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:02:14.081 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:14 smithi096 ceph-mon[35085]: mds.? [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] up:boot 2024-09-19T05:02:14.081 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:14 smithi096 ceph-mon[35085]: fsmap 12 up:standby 2024-09-19T05:02:14.081 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:14 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T05:02:14.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:14 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:14.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:14 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:14.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:14 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:14.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:14 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:14.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:14 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:14.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:14 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:02:14.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:14 smithi151 ceph-mon[35425]: mds.? [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] up:boot 2024-09-19T05:02:14.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:14 smithi151 ceph-mon[35425]: fsmap 12 up:standby 2024-09-19T05:02:14.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:14 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T05:02:14.352 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:14 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:14.352 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:14 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:14.352 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:14 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:14.352 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:14 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:14.352 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:14 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:14.352 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:14 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:02:14.352 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:14 smithi043 ceph-mon[30800]: mds.? [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] up:boot 2024-09-19T05:02:14.353 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:14 smithi043 ceph-mon[30800]: fsmap 12 up:standby 2024-09-19T05:02:14.353 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:14 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T05:02:14.718 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:02:15.198 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:15 smithi043 ceph-mon[30800]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:15.198 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:15 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:15.198 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:15 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:15.198 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:15 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:15.198 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:15 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:15.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.311+0000 7f2e4a800640 1 -- 172.21.15.43:0/3157769186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e44076720 msgr2=0x7f2e44076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:15.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.311+0000 7f2e4a800640 1 --2- 172.21.15.43:0/3157769186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e44076720 0x7f2e44076b00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2e34009900 tx=0x7f2e3402a3f0 comp rx=0 tx=0).stop 2024-09-19T05:02:15.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.311+0000 7f2e4a800640 1 -- 172.21.15.43:0/3157769186 shutdown_connections 2024-09-19T05:02:15.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.311+0000 7f2e4a800640 1 --2- 172.21.15.43:0/3157769186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e44112a30 0x7f2e44114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:15.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.311+0000 7f2e4a800640 1 --2- 172.21.15.43:0/3157769186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2e44077040 0x7f2e441124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:15.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.311+0000 7f2e4a800640 1 --2- 172.21.15.43:0/3157769186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e44076720 0x7f2e44076b00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:15.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.311+0000 7f2e4a800640 1 -- 172.21.15.43:0/3157769186 >> 172.21.15.43:0/3157769186 conn(0x7f2e440ffa90 msgr2=0x7f2e44101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:15.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.311+0000 7f2e4a800640 1 -- 172.21.15.43:0/3157769186 shutdown_connections 2024-09-19T05:02:15.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.311+0000 7f2e4a800640 1 -- 172.21.15.43:0/3157769186 wait complete. 2024-09-19T05:02:15.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.312+0000 7f2e4a800640 1 Processor -- start 2024-09-19T05:02:15.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.312+0000 7f2e4a800640 1 -- start start 2024-09-19T05:02:15.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.313+0000 7f2e4a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2e44076720 0x7f2e441a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:15.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.313+0000 7f2e4a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e44077040 0x7f2e441a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:15.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.313+0000 7f2e4a800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e44112a30 0x7f2e4419e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:15.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.313+0000 7f2e4a800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2e441185f0 con 0x7f2e44077040 2024-09-19T05:02:15.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.313+0000 7f2e4a800640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2e44118470 con 0x7f2e44112a30 2024-09-19T05:02:15.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.313+0000 7f2e4a800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2e44118770 con 0x7f2e44076720 2024-09-19T05:02:15.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.313+0000 7f2e42a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e44077040 0x7f2e441a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:15.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.313+0000 7f2e42a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e44077040 0x7f2e441a4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45036/0 (socket says 172.21.15.43:45036) 2024-09-19T05:02:15.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.313+0000 7f2e42a00640 1 -- 172.21.15.43:0/4066086919 learned_addr learned my addr 172.21.15.43:0/4066086919 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:02:15.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.313+0000 7f2e43e00640 1 --2- 172.21.15.43:0/4066086919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e44112a30 0x7f2e4419e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:15.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.313+0000 7f2e43400640 1 --2- 172.21.15.43:0/4066086919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2e44076720 0x7f2e441a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:15.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.313+0000 7f2e42a00640 1 -- 172.21.15.43:0/4066086919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2e44076720 msgr2=0x7f2e441a46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:15.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.313+0000 7f2e42a00640 1 --2- 172.21.15.43:0/4066086919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2e44076720 0x7f2e441a46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:15.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.313+0000 7f2e42a00640 1 -- 172.21.15.43:0/4066086919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e44112a30 msgr2=0x7f2e4419e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:15.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.313+0000 7f2e42a00640 1 --2- 172.21.15.43:0/4066086919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e44112a30 0x7f2e4419e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:15.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.313+0000 7f2e42a00640 1 -- 172.21.15.43:0/4066086919 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4419efb0 con 0x7f2e44077040 2024-09-19T05:02:15.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.313+0000 7f2e43e00640 1 --2- 172.21.15.43:0/4066086919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e44112a30 0x7f2e4419e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:02:15.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.314+0000 7f2e42a00640 1 --2- 172.21.15.43:0/4066086919 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e44077040 0x7f2e441a4c00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2e3000d950 tx=0x7f2e3000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:15.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.314+0000 7f2e3be00640 1 -- 172.21.15.43:0/4066086919 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e30017070 con 0x7f2e44077040 2024-09-19T05:02:15.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.314+0000 7f2e3be00640 1 -- 172.21.15.43:0/4066086919 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e3000de80 con 0x7f2e44077040 2024-09-19T05:02:15.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.314+0000 7f2e4a800640 1 -- 172.21.15.43:0/4066086919 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e4419f2a0 con 0x7f2e44077040 2024-09-19T05:02:15.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.314+0000 7f2e3be00640 1 -- 172.21.15.43:0/4066086919 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3000f620 con 0x7f2e44077040 2024-09-19T05:02:15.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.314+0000 7f2e4a800640 1 -- 172.21.15.43:0/4066086919 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e441ab590 con 0x7f2e44077040 2024-09-19T05:02:15.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.315+0000 7f2e4a800640 1 -- 172.21.15.43:0/4066086919 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e44076b00 con 0x7f2e44077040 2024-09-19T05:02:15.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.318+0000 7f2e3be00640 1 -- 172.21.15.43:0/4066086919 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f2e30009c30 con 0x7f2e44077040 2024-09-19T05:02:15.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.319+0000 7f2e3be00640 1 --2- 172.21.15.43:0/4066086919 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f2e280a3240 0x7f2e280a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:15.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.319+0000 7f2e3be00640 1 -- 172.21.15.43:0/4066086919 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f2e300cf110 con 0x7f2e44077040 2024-09-19T05:02:15.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.320+0000 7f2e43400640 1 --2- 172.21.15.43:0/4066086919 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f2e280a3240 0x7f2e280a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:15.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.320+0000 7f2e43400640 1 --2- 172.21.15.43:0/4066086919 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f2e280a3240 0x7f2e280a5700 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2e34006010 tx=0x7f2e34000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:15.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.324+0000 7f2e3be00640 1 -- 172.21.15.43:0/4066086919 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2e3009c230 con 0x7f2e44077040 2024-09-19T05:02:15.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:15 smithi151 ceph-mon[35425]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:15.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:15 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:15.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:15 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:15.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:15 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:15.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:15 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:15.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:15 smithi096 ceph-mon[35085]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:15.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:15 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:15.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:15 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:15.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:15 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:15.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:15 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:15.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.615+0000 7f2e4a800640 1 -- 172.21.15.43:0/4066086919 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2e441ab880 con 0x7f2e44077040 2024-09-19T05:02:15.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.616+0000 7f2e3be00640 1 -- 172.21.15.43:0/4066086919 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10515 (secure 0 0 0) 0x7f2e3009bb30 con 0x7f2e44077040 2024-09-19T05:02:15.617 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:02:15.617 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":13,"btime":"2024-09-19T05:02:13:535638+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/773801169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":773801169},{"type":"v1","addr":"172.21.15.43:6835","nonce":773801169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1799137902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1799137902},{"type":"v1","addr":"172.21.15.43:6837","nonce":1799137902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/794881135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":794881135},{"type":"v1","addr":"172.21.15.43:6841","nonce":794881135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/2836453158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2836453158},{"type":"v1","addr":"172.21.15.151:6833","nonce":2836453158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/178785812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":178785812},{"type":"v1","addr":"172.21.15.96:6833","nonce":178785812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1671043820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1671043820},{"type":"v1","addr":"172.21.15.96:6835","nonce":1671043820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2972514328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2972514328},{"type":"v1","addr":"172.21.15.151:6835","nonce":2972514328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24493,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/3960378010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3960378010},{"type":"v1","addr":"172.21.15.151:6837","nonce":3960378010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/3978388286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3978388286},{"type":"v1","addr":"172.21.15.96:6837","nonce":3978388286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/3794104313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3794104313},{"type":"v1","addr":"172.21.15.151:6839","nonce":3794104313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24518,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/3404555030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3404555030},{"type":"v1","addr":"172.21.15.43:6839","nonce":3404555030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24542,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/605527621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":605527621},{"type":"v1","addr":"172.21.15.96:6839","nonce":605527621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-19T05:02:15.617 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 13 2024-09-19T05:02:15.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.618+0000 7f2e4a800640 1 -- 172.21.15.43:0/4066086919 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f2e280a3240 msgr2=0x7f2e280a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:15.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.618+0000 7f2e4a800640 1 --2- 172.21.15.43:0/4066086919 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f2e280a3240 0x7f2e280a5700 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2e34006010 tx=0x7f2e34000bc0 comp rx=0 tx=0).stop 2024-09-19T05:02:15.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.618+0000 7f2e4a800640 1 -- 172.21.15.43:0/4066086919 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e44077040 msgr2=0x7f2e441a4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:15.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.618+0000 7f2e4a800640 1 --2- 172.21.15.43:0/4066086919 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e44077040 0x7f2e441a4c00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2e3000d950 tx=0x7f2e3000dd50 comp rx=0 tx=0).stop 2024-09-19T05:02:15.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.619+0000 7f2e4a800640 1 -- 172.21.15.43:0/4066086919 shutdown_connections 2024-09-19T05:02:15.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.619+0000 7f2e4a800640 1 --2- 172.21.15.43:0/4066086919 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f2e280a3240 0x7f2e280a5700 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:15.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.619+0000 7f2e4a800640 1 --2- 172.21.15.43:0/4066086919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e44112a30 0x7f2e4419e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:15.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.619+0000 7f2e4a800640 1 --2- 172.21.15.43:0/4066086919 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e44077040 0x7f2e441a4c00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:15.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.619+0000 7f2e4a800640 1 --2- 172.21.15.43:0/4066086919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2e44076720 0x7f2e441a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:15.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.619+0000 7f2e4a800640 1 -- 172.21.15.43:0/4066086919 >> 172.21.15.43:0/4066086919 conn(0x7f2e440ffa90 msgr2=0x7f2e44101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:15.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.619+0000 7f2e4a800640 1 -- 172.21.15.43:0/4066086919 shutdown_connections 2024-09-19T05:02:15.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:15.619+0000 7f2e4a800640 1 -- 172.21.15.43:0/4066086919 wait complete. 2024-09-19T05:02:16.352 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-19T05:02:16.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:16 smithi096 ceph-mon[35085]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:16.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:16 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/4066086919' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:02:16.760 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:02:16.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:16 smithi043 ceph-mon[30800]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:16.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:16 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/4066086919' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:02:16.904 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:16 smithi151 ceph-mon[35425]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:16.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:16 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/4066086919' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:02:17.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.344+0000 7f55ab400640 1 -- 172.21.15.43:0/3330393597 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55980319b0 con 0x7f55ac074ee0 2024-09-19T05:02:17.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.344+0000 7f55b3c00640 1 -- 172.21.15.43:0/3330393597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55ac074ee0 msgr2=0x7f55ac10def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:17.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.344+0000 7f55b3c00640 1 --2- 172.21.15.43:0/3330393597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55ac074ee0 0x7f55ac10def0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5598009900 tx=0x7f559802a480 comp rx=0 tx=0).stop 2024-09-19T05:02:17.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.344+0000 7f55b3c00640 1 -- 172.21.15.43:0/3330393597 shutdown_connections 2024-09-19T05:02:17.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.344+0000 7f55b3c00640 1 --2- 172.21.15.43:0/3330393597 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55ac10e430 0x7f55ac110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:17.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.344+0000 7f55b3c00640 1 --2- 172.21.15.43:0/3330393597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55ac074ee0 0x7f55ac10def0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:17.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.344+0000 7f55b3c00640 1 --2- 172.21.15.43:0/3330393597 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55ac0745c0 0x7f55ac0749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:17.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.344+0000 7f55b3c00640 1 -- 172.21.15.43:0/3330393597 >> 172.21.15.43:0/3330393597 conn(0x7f55ac0fd8b0 msgr2=0x7f55ac0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:17.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.345+0000 7f55b3c00640 1 -- 172.21.15.43:0/3330393597 shutdown_connections 2024-09-19T05:02:17.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.345+0000 7f55b3c00640 1 -- 172.21.15.43:0/3330393597 wait complete. 2024-09-19T05:02:17.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.346+0000 7f55b3c00640 1 Processor -- start 2024-09-19T05:02:17.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.346+0000 7f55b3c00640 1 -- start start 2024-09-19T05:02:17.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.346+0000 7f55b3c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55ac0745c0 0x7f55ac1a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:17.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.346+0000 7f55b3c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55ac074ee0 0x7f55ac1a4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:17.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.346+0000 7f55b3c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55ac10e430 0x7f55ac10c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:17.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.346+0000 7f55b3c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f55ac102d80 con 0x7f55ac0745c0 2024-09-19T05:02:17.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.346+0000 7f55b3c00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f55ac102c00 con 0x7f55ac074ee0 2024-09-19T05:02:17.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.346+0000 7f55b3c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f55ac102f00 con 0x7f55ac10e430 2024-09-19T05:02:17.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.346+0000 7f55b1200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55ac0745c0 0x7f55ac1a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:17.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.346+0000 7f55b1c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55ac10e430 0x7f55ac10c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:17.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.346+0000 7f55b1200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55ac0745c0 0x7f55ac1a4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46336/0 (socket says 172.21.15.43:46336) 2024-09-19T05:02:17.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.346+0000 7f55b1200640 1 -- 172.21.15.43:0/733166872 learned_addr learned my addr 172.21.15.43:0/733166872 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:02:17.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.347+0000 7f55abe00640 1 --2- 172.21.15.43:0/733166872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55ac074ee0 0x7f55ac1a4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:17.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.347+0000 7f55b1c00640 1 -- 172.21.15.43:0/733166872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55ac074ee0 msgr2=0x7f55ac1a4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:17.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.347+0000 7f55b1c00640 1 --2- 172.21.15.43:0/733166872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55ac074ee0 0x7f55ac1a4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:17.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.347+0000 7f55b1c00640 1 -- 172.21.15.43:0/733166872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55ac0745c0 msgr2=0x7f55ac1a4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:17.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.347+0000 7f55b1c00640 1 --2- 172.21.15.43:0/733166872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55ac0745c0 0x7f55ac1a4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:17.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.347+0000 7f55b1c00640 1 -- 172.21.15.43:0/733166872 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55ac10d080 con 0x7f55ac10e430 2024-09-19T05:02:17.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.347+0000 7f55b1c00640 1 --2- 172.21.15.43:0/733166872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55ac10e430 0x7f55ac10c8b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f55a4005fd0 tx=0x7f55a400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:17.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.347+0000 7f55a9600640 1 -- 172.21.15.43:0/733166872 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a400cdf0 con 0x7f55ac10e430 2024-09-19T05:02:17.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.347+0000 7f55a9600640 1 -- 172.21.15.43:0/733166872 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55a4005020 con 0x7f55ac10e430 2024-09-19T05:02:17.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.347+0000 7f55b3c00640 1 -- 172.21.15.43:0/733166872 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55ac10d370 con 0x7f55ac10e430 2024-09-19T05:02:17.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.348+0000 7f55a9600640 1 -- 172.21.15.43:0/733166872 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a40106d0 con 0x7f55ac10e430 2024-09-19T05:02:17.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.348+0000 7f55b3c00640 1 -- 172.21.15.43:0/733166872 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55ac1ab590 con 0x7f55ac10e430 2024-09-19T05:02:17.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.349+0000 7f55b3c00640 1 -- 172.21.15.43:0/733166872 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55ac0749a0 con 0x7f55ac10e430 2024-09-19T05:02:17.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.351+0000 7f55a9600640 1 -- 172.21.15.43:0/733166872 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f55a40054d0 con 0x7f55ac10e430 2024-09-19T05:02:17.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.354+0000 7f55a9600640 1 --2- 172.21.15.43:0/733166872 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f55940a3240 0x7f55940a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:17.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.354+0000 7f55a9600640 1 -- 172.21.15.43:0/733166872 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f55a40cfab0 con 0x7f55ac10e430 2024-09-19T05:02:17.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.354+0000 7f55b1200640 1 --2- 172.21.15.43:0/733166872 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f55940a3240 0x7f55940a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:17.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.355+0000 7f55b1200640 1 --2- 172.21.15.43:0/733166872 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f55940a3240 0x7f55940a5700 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f55a0005fd0 tx=0x7f55a000b4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:17.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.357+0000 7f55a9600640 1 -- 172.21.15.43:0/733166872 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f55a409cbd0 con 0x7f55ac10e430 2024-09-19T05:02:17.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:17 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:17.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:17 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:17.492 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:17 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:17.492 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:17 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:02:17.492 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:17 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:17.492 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:17 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:02:17.493 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:17 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:17.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:17 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:17.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:17 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:17.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:17 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:17.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:17 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:02:17.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:17 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:17.532 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:17 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:02:17.532 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:17 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:17.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.548+0000 7f55b3c00640 1 -- 172.21.15.43:0/733166872 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f55ac106c60 con 0x7f55ac10e430 2024-09-19T05:02:17.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.549+0000 7f55a9600640 1 -- 172.21.15.43:0/733166872 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v77) ==== 84+0+65 (secure 0 0 0) 0x7f55a409c4d0 con 0x7f55ac10e430 2024-09-19T05:02:17.551 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:02:17.551 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2024-09-19T05:02:17.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:02:17.551 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 1, 2024-09-19T05:02:17.551 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": ".mgr" 2024-09-19T05:02:17.552 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:02:17.552 INFO:teuthology.orchestra.run.smithi043.stdout:] 2024-09-19T05:02:17.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.551+0000 7f55b3c00640 1 -- 172.21.15.43:0/733166872 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f55940a3240 msgr2=0x7f55940a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:17.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.551+0000 7f55b3c00640 1 --2- 172.21.15.43:0/733166872 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f55940a3240 0x7f55940a5700 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f55a0005fd0 tx=0x7f55a000b4a0 comp rx=0 tx=0).stop 2024-09-19T05:02:17.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.551+0000 7f55b3c00640 1 -- 172.21.15.43:0/733166872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55ac10e430 msgr2=0x7f55ac10c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:17.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.551+0000 7f55b3c00640 1 --2- 172.21.15.43:0/733166872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55ac10e430 0x7f55ac10c8b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f55a4005fd0 tx=0x7f55a400ef10 comp rx=0 tx=0).stop 2024-09-19T05:02:17.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.551+0000 7f55b3c00640 1 -- 172.21.15.43:0/733166872 shutdown_connections 2024-09-19T05:02:17.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.551+0000 7f55b3c00640 1 --2- 172.21.15.43:0/733166872 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f55940a3240 0x7f55940a5700 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:17.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.551+0000 7f55b3c00640 1 --2- 172.21.15.43:0/733166872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55ac10e430 0x7f55ac10c8b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:17.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.551+0000 7f55b3c00640 1 --2- 172.21.15.43:0/733166872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55ac074ee0 0x7f55ac1a4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:17.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.551+0000 7f55b3c00640 1 --2- 172.21.15.43:0/733166872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55ac0745c0 0x7f55ac1a4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.551+0000 7f55b3c00640 1 -- 172.21.15.43:0/733166872 >> 172.21.15.43:0/733166872 conn(0x7f55ac0fd8b0 msgr2=0x7f55ac0fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.552+0000 7f55b3c00640 1 -- 172.21.15.43:0/733166872 shutdown_connections 2024-09-19T05:02:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:17.552+0000 7f55b3c00640 1 -- 172.21.15.43:0/733166872 wait complete. 2024-09-19T05:02:17.685 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:17 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:17.685 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:17 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:17.685 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:17 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:17.685 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:17 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:02:17.686 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:17 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:17.686 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:17 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:02:17.686 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:17 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:18.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:18 smithi096 ceph-mon[35085]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:18.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:18 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/733166872' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T05:02:18.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:18 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:18.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:18 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:18.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:18 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:18.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:18 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:18.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:18 smithi043 ceph-mon[30800]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:18.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:18 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/733166872' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T05:02:18.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:18 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:18.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:18 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:18.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:18 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:18.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:18 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:18.904 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:18 smithi151 ceph-mon[35425]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:18.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:18 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/733166872' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T05:02:18.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:18 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:18.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:18 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:18.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:18 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:18.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:18 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:18.945 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T05:02:18.945 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T05:02:19.355 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:02:19.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:19.927+0000 7f38eec00640 1 -- 172.21.15.43:0/2387900665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38e8101de0 msgr2=0x7f38e81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:19.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:19.927+0000 7f38eec00640 1 --2- 172.21.15.43:0/2387900665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38e8101de0 0x7f38e81021c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f38d8009900 tx=0x7f38d802a3f0 comp rx=0 tx=0).stop 2024-09-19T05:02:19.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:19.928+0000 7f38eec00640 1 -- 172.21.15.43:0/2387900665 shutdown_connections 2024-09-19T05:02:19.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:19.928+0000 7f38eec00640 1 --2- 172.21.15.43:0/2387900665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f38e8106f10 0x7f38e810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:19.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:19.928+0000 7f38eec00640 1 --2- 172.21.15.43:0/2387900665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f38e8102790 0x7f38e81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:19.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:19.928+0000 7f38eec00640 1 --2- 172.21.15.43:0/2387900665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38e8101de0 0x7f38e81021c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:19.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:19.928+0000 7f38eec00640 1 -- 172.21.15.43:0/2387900665 >> 172.21.15.43:0/2387900665 conn(0x7f38e80fd8c0 msgr2=0x7f38e80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:19.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:19.928+0000 7f38eec00640 1 -- 172.21.15.43:0/2387900665 shutdown_connections 2024-09-19T05:02:19.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:19.928+0000 7f38eec00640 1 -- 172.21.15.43:0/2387900665 wait complete. 2024-09-19T05:02:19.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:19.929+0000 7f38eec00640 1 Processor -- start 2024-09-19T05:02:19.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:19.929+0000 7f38eec00640 1 -- start start 2024-09-19T05:02:19.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:19.929+0000 7f38eec00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f38e8101de0 0x7f38e80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:19.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:19.929+0000 7f38eec00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38e8102790 0x7f38e8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:19.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:19.929+0000 7f38eec00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f38e8106f10 0x7f38e8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:19.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:19.929+0000 7f38eec00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f38e8111fc0 con 0x7f38e8102790 2024-09-19T05:02:19.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:19.929+0000 7f38eec00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f38e8111e40 con 0x7f38e8101de0 2024-09-19T05:02:19.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:19.929+0000 7f38eec00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f38e8112140 con 0x7f38e8106f10 2024-09-19T05:02:19.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:19.930+0000 7f38dfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38e8102790 0x7f38e8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:19.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:19.930+0000 7f38e7e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f38e8101de0 0x7f38e80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:19.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:19.930+0000 7f38dfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38e8102790 0x7f38e8074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46348/0 (socket says 172.21.15.43:46348) 2024-09-19T05:02:19.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:19.930+0000 7f38e7e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f38e8101de0 0x7f38e80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:40716/0 (socket says 172.21.15.43:40716) 2024-09-19T05:02:19.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:19.930+0000 7f38dfe00640 1 -- 172.21.15.43:0/1746821554 learned_addr learned my addr 172.21.15.43:0/1746821554 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:02:19.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:19.930+0000 7f38ecc00640 1 --2- 172.21.15.43:0/1746821554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f38e8106f10 0x7f38e8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:19.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:19.930+0000 7f38dfe00640 1 -- 172.21.15.43:0/1746821554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f38e8106f10 msgr2=0x7f38e8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:19.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:19.930+0000 7f38dfe00640 1 --2- 172.21.15.43:0/1746821554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f38e8106f10 0x7f38e8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:19.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:19.930+0000 7f38dfe00640 1 -- 172.21.15.43:0/1746821554 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f38e8101de0 msgr2=0x7f38e80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:19.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:19.930+0000 7f38dfe00640 1 --2- 172.21.15.43:0/1746821554 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f38e8101de0 0x7f38e80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:19.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:19.930+0000 7f38dfe00640 1 -- 172.21.15.43:0/1746821554 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38e81a69a0 con 0x7f38e8102790 2024-09-19T05:02:19.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:19.930+0000 7f38dfe00640 1 --2- 172.21.15.43:0/1746821554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38e8102790 0x7f38e8074a30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f38d000e9f0 tx=0x7f38d000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:19.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:19.931+0000 7f38e5600640 1 -- 172.21.15.43:0/1746821554 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d000cd80 con 0x7f38e8102790 2024-09-19T05:02:19.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:19.931+0000 7f38e5600640 1 -- 172.21.15.43:0/1746821554 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38d0005500 con 0x7f38e8102790 2024-09-19T05:02:19.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:19.931+0000 7f38e5600640 1 -- 172.21.15.43:0/1746821554 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d0010690 con 0x7f38e8102790 2024-09-19T05:02:19.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:19.931+0000 7f38eec00640 1 -- 172.21.15.43:0/1746821554 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38e81a6c90 con 0x7f38e8102790 2024-09-19T05:02:19.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:19.931+0000 7f38eec00640 1 -- 172.21.15.43:0/1746821554 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38e81a7180 con 0x7f38e8102790 2024-09-19T05:02:19.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:19.932+0000 7f38eec00640 1 -- 172.21.15.43:0/1746821554 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38e81021c0 con 0x7f38e8102790 2024-09-19T05:02:19.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:19.934+0000 7f38e5600640 1 -- 172.21.15.43:0/1746821554 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f38d00046c0 con 0x7f38e8102790 2024-09-19T05:02:19.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:19.935+0000 7f38e5600640 1 --2- 172.21.15.43:0/1746821554 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f38cc0a32c0 0x7f38cc0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:19.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:19.936+0000 7f38e5600640 1 -- 172.21.15.43:0/1746821554 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f38d00cf890 con 0x7f38e8102790 2024-09-19T05:02:19.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:19.936+0000 7f38e7e00640 1 --2- 172.21.15.43:0/1746821554 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f38cc0a32c0 0x7f38cc0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:19.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:19.936+0000 7f38e7e00640 1 --2- 172.21.15.43:0/1746821554 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f38cc0a32c0 0x7f38cc0a5780 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f38d8000c00 tx=0x7f38d8005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:19.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:19.940+0000 7f38e5600640 1 -- 172.21.15.43:0/1746821554 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f38d009c9b0 con 0x7f38e8102790 2024-09-19T05:02:20.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:20.154+0000 7f38eec00640 1 -- 172.21.15.43:0/1746821554 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f38e810a0f0 con 0x7f38e8102790 2024-09-19T05:02:21.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:21.004+0000 7f38e5600640 1 -- 172.21.15.43:0/1746821554 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v78) ==== 138+0+0 (secure 0 0 0) 0x7f38d0096e20 con 0x7f38e8102790 2024-09-19T05:02:21.006 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' created 2024-09-19T05:02:21.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:21.007+0000 7f38eec00640 1 -- 172.21.15.43:0/1746821554 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f38cc0a32c0 msgr2=0x7f38cc0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:21.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:21.007+0000 7f38eec00640 1 --2- 172.21.15.43:0/1746821554 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f38cc0a32c0 0x7f38cc0a5780 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f38d8000c00 tx=0x7f38d8005b50 comp rx=0 tx=0).stop 2024-09-19T05:02:21.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:21.007+0000 7f38eec00640 1 -- 172.21.15.43:0/1746821554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38e8102790 msgr2=0x7f38e8074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:21.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:21.007+0000 7f38eec00640 1 --2- 172.21.15.43:0/1746821554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38e8102790 0x7f38e8074a30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f38d000e9f0 tx=0x7f38d000edf0 comp rx=0 tx=0).stop 2024-09-19T05:02:21.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:21.008+0000 7f38eec00640 1 -- 172.21.15.43:0/1746821554 shutdown_connections 2024-09-19T05:02:21.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:21.008+0000 7f38eec00640 1 --2- 172.21.15.43:0/1746821554 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f38cc0a32c0 0x7f38cc0a5780 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:21.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:21.008+0000 7f38eec00640 1 --2- 172.21.15.43:0/1746821554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f38e8106f10 0x7f38e8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:21.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:21.008+0000 7f38eec00640 1 --2- 172.21.15.43:0/1746821554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38e8102790 0x7f38e8074a30 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:21.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:21.008+0000 7f38eec00640 1 --2- 172.21.15.43:0/1746821554 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f38e8101de0 0x7f38e80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:21.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:21.008+0000 7f38eec00640 1 -- 172.21.15.43:0/1746821554 >> 172.21.15.43:0/1746821554 conn(0x7f38e80fd8c0 msgr2=0x7f38e80febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:21.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:21.008+0000 7f38eec00640 1 -- 172.21.15.43:0/1746821554 shutdown_connections 2024-09-19T05:02:21.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:21.008+0000 7f38eec00640 1 -- 172.21.15.43:0/1746821554 wait complete. 2024-09-19T05:02:21.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:20 smithi043 ceph-mon[30800]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:21.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:20 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:21.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:20 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:21.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:20 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/1746821554' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T05:02:21.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:20 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:21.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:20 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:02:21.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:20 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:21.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:20 smithi151 ceph-mon[35425]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:21.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:20 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:21.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:20 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:21.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:20 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/1746821554' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T05:02:21.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:20 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:21.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:20 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:02:21.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:20 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:21.427 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:20 smithi096 ceph-mon[35085]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:21.427 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:20 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:21.427 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:20 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:21.427 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:20 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/1746821554' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T05:02:21.427 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:20 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:02:21.427 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:20 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:02:21.427 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:20 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:02:21.744 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T05:02:22.133 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:02:22.273 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:22 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/1746821554' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T05:02:22.273 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:22 smithi043 ceph-mon[30800]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T05:02:22.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:22 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/1746821554' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T05:02:22.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:22 smithi151 ceph-mon[35425]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T05:02:22.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:22 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/1746821554' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T05:02:22.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:22 smithi096 ceph-mon[35085]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T05:02:22.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:22.664+0000 7f84c4000640 1 -- 172.21.15.43:0/1892691171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84bc109120 msgr2=0x7f84bc110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:22.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:22.664+0000 7f84c4000640 1 --2- 172.21.15.43:0/1892691171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84bc109120 0x7f84bc110ca0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f84b4009ca0 tx=0x7f84b402a8c0 comp rx=0 tx=0).stop 2024-09-19T05:02:22.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:22.664+0000 7f84c4000640 1 -- 172.21.15.43:0/1892691171 shutdown_connections 2024-09-19T05:02:22.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:22.664+0000 7f84c4000640 1 --2- 172.21.15.43:0/1892691171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84bc109120 0x7f84bc110ca0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:22.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:22.664+0000 7f84c4000640 1 --2- 172.21.15.43:0/1892691171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84bc1049a0 0x7f84bc1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:22.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:22.664+0000 7f84c4000640 1 --2- 172.21.15.43:0/1892691171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84bc103ff0 0x7f84bc1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:22.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:22.664+0000 7f84c4000640 1 -- 172.21.15.43:0/1892691171 >> 172.21.15.43:0/1892691171 conn(0x7f84bc0ffa90 msgr2=0x7f84bc101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:22.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:22.664+0000 7f84c4000640 1 -- 172.21.15.43:0/1892691171 shutdown_connections 2024-09-19T05:02:22.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:22.664+0000 7f84c4000640 1 -- 172.21.15.43:0/1892691171 wait complete. 2024-09-19T05:02:22.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:22.665+0000 7f84c4000640 1 Processor -- start 2024-09-19T05:02:22.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:22.665+0000 7f84c4000640 1 -- start start 2024-09-19T05:02:22.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:22.666+0000 7f84c4000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84bc103ff0 0x7f84bc19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:22.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:22.666+0000 7f84c4000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84bc1049a0 0x7f84bc19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:22.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:22.666+0000 7f84c4000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84bc109120 0x7f84bc19ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:22.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:22.666+0000 7f84c4000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f84bc114180 con 0x7f84bc103ff0 2024-09-19T05:02:22.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:22.666+0000 7f84c4000640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f84bc114000 con 0x7f84bc109120 2024-09-19T05:02:22.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:22.666+0000 7f84c4000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f84bc114300 con 0x7f84bc1049a0 2024-09-19T05:02:22.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:22.666+0000 7f84c1600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84bc103ff0 0x7f84bc19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:22.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:22.666+0000 7f84c1600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84bc103ff0 0x7f84bc19a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46360/0 (socket says 172.21.15.43:46360) 2024-09-19T05:02:22.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:22.666+0000 7f84c1600640 1 -- 172.21.15.43:0/3318974200 learned_addr learned my addr 172.21.15.43:0/3318974200 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:02:22.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:22.666+0000 7f84c0c00640 1 --2- 172.21.15.43:0/3318974200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84bc1049a0 0x7f84bc19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:22.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:22.666+0000 7f84c2000640 1 --2- 172.21.15.43:0/3318974200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84bc109120 0x7f84bc19ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:22.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:22.666+0000 7f84c1600640 1 -- 172.21.15.43:0/3318974200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84bc1049a0 msgr2=0x7f84bc19ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:22.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:22.666+0000 7f84c1600640 1 --2- 172.21.15.43:0/3318974200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84bc1049a0 0x7f84bc19ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:22.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:22.666+0000 7f84c1600640 1 -- 172.21.15.43:0/3318974200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84bc109120 msgr2=0x7f84bc19ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:22.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:22.666+0000 7f84c1600640 1 --2- 172.21.15.43:0/3318974200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84bc109120 0x7f84bc19ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:22.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:22.666+0000 7f84c1600640 1 -- 172.21.15.43:0/3318974200 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84bc19f570 con 0x7f84bc103ff0 2024-09-19T05:02:22.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:22.667+0000 7f84c1600640 1 --2- 172.21.15.43:0/3318974200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84bc103ff0 0x7f84bc19a5c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f84b000d950 tx=0x7f84b000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:22.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:22.667+0000 7f84ba000640 1 -- 172.21.15.43:0/3318974200 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b0017070 con 0x7f84bc103ff0 2024-09-19T05:02:22.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:22.667+0000 7f84ba000640 1 -- 172.21.15.43:0/3318974200 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84b0009c30 con 0x7f84bc103ff0 2024-09-19T05:02:22.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:22.667+0000 7f84c4000640 1 -- 172.21.15.43:0/3318974200 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84bc19f860 con 0x7f84bc103ff0 2024-09-19T05:02:22.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:22.667+0000 7f84ba000640 1 -- 172.21.15.43:0/3318974200 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b000f6b0 con 0x7f84bc103ff0 2024-09-19T05:02:22.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:22.667+0000 7f84c4000640 1 -- 172.21.15.43:0/3318974200 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84bc1a7140 con 0x7f84bc103ff0 2024-09-19T05:02:22.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:22.668+0000 7f84c4000640 1 -- 172.21.15.43:0/3318974200 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84bc1043d0 con 0x7f84bc103ff0 2024-09-19T05:02:22.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:22.670+0000 7f84ba000640 1 -- 172.21.15.43:0/3318974200 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f84b00109d0 con 0x7f84bc103ff0 2024-09-19T05:02:22.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:22.671+0000 7f84ba000640 1 --2- 172.21.15.43:0/3318974200 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f84a40a3240 0x7f84a40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:22.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:22.672+0000 7f84ba000640 1 -- 172.21.15.43:0/3318974200 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7497+0+0 (secure 0 0 0) 0x7f84b00cf200 con 0x7f84bc103ff0 2024-09-19T05:02:22.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:22.672+0000 7f84c0c00640 1 --2- 172.21.15.43:0/3318974200 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f84a40a3240 0x7f84a40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:22.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:22.672+0000 7f84c0c00640 1 --2- 172.21.15.43:0/3318974200 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f84a40a3240 0x7f84a40a5700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f84a80095f0 tx=0x7f84a8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:22.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:22.674+0000 7f84ba000640 1 -- 172.21.15.43:0/3318974200 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f84b0009de0 con 0x7f84bc103ff0 2024-09-19T05:02:22.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:22.862+0000 7f84c4000640 1 -- 172.21.15.43:0/3318974200 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f84bc1a73e0 con 0x7f84bc103ff0 2024-09-19T05:02:23.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:23.401+0000 7f84ba000640 1 -- 172.21.15.43:0/3318974200 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v80) ==== 170+0+0 (secure 0 0 0) 0x7f84b009bb60 con 0x7f84bc103ff0 2024-09-19T05:02:23.403 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' created 2024-09-19T05:02:23.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:23.404+0000 7f84c4000640 1 -- 172.21.15.43:0/3318974200 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f84a40a3240 msgr2=0x7f84a40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:23.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:23.404+0000 7f84c4000640 1 --2- 172.21.15.43:0/3318974200 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f84a40a3240 0x7f84a40a5700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f84a80095f0 tx=0x7f84a8005f90 comp rx=0 tx=0).stop 2024-09-19T05:02:23.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:23.404+0000 7f84c4000640 1 -- 172.21.15.43:0/3318974200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84bc103ff0 msgr2=0x7f84bc19a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:23.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:23.404+0000 7f84c4000640 1 --2- 172.21.15.43:0/3318974200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84bc103ff0 0x7f84bc19a5c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f84b000d950 tx=0x7f84b000de10 comp rx=0 tx=0).stop 2024-09-19T05:02:23.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:23.404+0000 7f84c4000640 1 -- 172.21.15.43:0/3318974200 shutdown_connections 2024-09-19T05:02:23.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:23.404+0000 7f84c4000640 1 --2- 172.21.15.43:0/3318974200 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f84a40a3240 0x7f84a40a5700 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:23.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:23.404+0000 7f84c4000640 1 --2- 172.21.15.43:0/3318974200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84bc109120 0x7f84bc19ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:23.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:23.404+0000 7f84c4000640 1 --2- 172.21.15.43:0/3318974200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84bc1049a0 0x7f84bc19ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:23.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:23.404+0000 7f84c4000640 1 --2- 172.21.15.43:0/3318974200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84bc103ff0 0x7f84bc19a5c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:23.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:23.404+0000 7f84c4000640 1 -- 172.21.15.43:0/3318974200 >> 172.21.15.43:0/3318974200 conn(0x7f84bc0ffa90 msgr2=0x7f84bc1005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:23.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:23.405+0000 7f84c4000640 1 -- 172.21.15.43:0/3318974200 shutdown_connections 2024-09-19T05:02:23.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:23.405+0000 7f84c4000640 1 -- 172.21.15.43:0/3318974200 wait complete. 2024-09-19T05:02:23.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:23 smithi151 ceph-mon[35425]: pgmap v285: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:23.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:23 smithi151 ceph-mon[35425]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T05:02:23.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:23 smithi151 ceph-mon[35425]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T05:02:23.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:23 smithi096 ceph-mon[35085]: pgmap v285: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:23.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:23 smithi096 ceph-mon[35085]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T05:02:23.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:23 smithi096 ceph-mon[35085]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T05:02:23.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:23 smithi043 ceph-mon[30800]: pgmap v285: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:23.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:23 smithi043 ceph-mon[30800]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T05:02:23.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:23 smithi043 ceph-mon[30800]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T05:02:24.306 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T05:02:24.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:24 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/3318974200' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T05:02:24.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:24 smithi043 ceph-mon[30800]: pgmap v287: 33 pgs: 10 creating+peering, 8 unknown, 15 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:24.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:24 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/3318974200' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T05:02:24.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:24 smithi043 ceph-mon[30800]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T05:02:24.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:24 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/3318974200' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T05:02:24.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:24 smithi151 ceph-mon[35425]: pgmap v287: 33 pgs: 10 creating+peering, 8 unknown, 15 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:24.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:24 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/3318974200' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T05:02:24.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:24 smithi151 ceph-mon[35425]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T05:02:24.702 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:02:24.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:24 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/3318974200' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T05:02:24.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:24 smithi096 ceph-mon[35085]: pgmap v287: 33 pgs: 10 creating+peering, 8 unknown, 15 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:24.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:24 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/3318974200' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T05:02:24.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:24 smithi096 ceph-mon[35085]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T05:02:25.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:25.226+0000 7faf9ae00640 1 -- 172.21.15.43:0/2419929479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf94101de0 msgr2=0x7faf941021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:25.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:25.226+0000 7faf9ae00640 1 --2- 172.21.15.43:0/2419929479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf94101de0 0x7faf941021c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faf84009900 tx=0x7faf8402a3f0 comp rx=0 tx=0).stop 2024-09-19T05:02:25.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:25.226+0000 7faf9ae00640 1 -- 172.21.15.43:0/2419929479 shutdown_connections 2024-09-19T05:02:25.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:25.226+0000 7faf9ae00640 1 --2- 172.21.15.43:0/2419929479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf94106f10 0x7faf9410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:25.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:25.226+0000 7faf9ae00640 1 --2- 172.21.15.43:0/2419929479 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf94102790 0x7faf941067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:25.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:25.226+0000 7faf9ae00640 1 --2- 172.21.15.43:0/2419929479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf94101de0 0x7faf941021c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:25.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:25.226+0000 7faf9ae00640 1 -- 172.21.15.43:0/2419929479 >> 172.21.15.43:0/2419929479 conn(0x7faf940fd8c0 msgr2=0x7faf940ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:25.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:25.226+0000 7faf9ae00640 1 -- 172.21.15.43:0/2419929479 shutdown_connections 2024-09-19T05:02:25.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:25.226+0000 7faf9ae00640 1 -- 172.21.15.43:0/2419929479 wait complete. 2024-09-19T05:02:25.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:25.227+0000 7faf9ae00640 1 Processor -- start 2024-09-19T05:02:25.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:25.227+0000 7faf9ae00640 1 -- start start 2024-09-19T05:02:25.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:25.228+0000 7faf9ae00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf94101de0 0x7faf941984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:25.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:25.228+0000 7faf9ae00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf94102790 0x7faf941989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:25.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:25.228+0000 7faf9ae00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf94106f10 0x7faf9419cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:25.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:25.228+0000 7faf9ae00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faf94111fc0 con 0x7faf94101de0 2024-09-19T05:02:25.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:25.228+0000 7faf9ae00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faf94111e40 con 0x7faf94106f10 2024-09-19T05:02:25.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:25.228+0000 7faf9ae00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faf94112140 con 0x7faf94102790 2024-09-19T05:02:25.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:25.228+0000 7faf98e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf94106f10 0x7faf9419cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:25.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:25.228+0000 7faf8b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf94102790 0x7faf941989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:25.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:25.228+0000 7faf98e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf94106f10 0x7faf9419cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:40750/0 (socket says 172.21.15.43:40750) 2024-09-19T05:02:25.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:25.228+0000 7faf98e00640 1 -- 172.21.15.43:0/290816456 learned_addr learned my addr 172.21.15.43:0/290816456 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:02:25.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:25.228+0000 7faf93e00640 1 --2- 172.21.15.43:0/290816456 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf94101de0 0x7faf941984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:25.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:25.229+0000 7faf98e00640 1 -- 172.21.15.43:0/290816456 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf94102790 msgr2=0x7faf941989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:25.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:25.229+0000 7faf98e00640 1 --2- 172.21.15.43:0/290816456 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf94102790 0x7faf941989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:25.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:25.229+0000 7faf98e00640 1 -- 172.21.15.43:0/290816456 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf94101de0 msgr2=0x7faf941984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:25.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:25.229+0000 7faf98e00640 1 --2- 172.21.15.43:0/290816456 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf94101de0 0x7faf941984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:25.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:25.229+0000 7faf98e00640 1 -- 172.21.15.43:0/290816456 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf9419d450 con 0x7faf94106f10 2024-09-19T05:02:25.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:25.229+0000 7faf8b400640 1 --2- 172.21.15.43:0/290816456 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf94102790 0x7faf941989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:02:25.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:25.229+0000 7faf98e00640 1 --2- 172.21.15.43:0/290816456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf94106f10 0x7faf9419cd70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faf8c00eb10 tx=0x7faf8c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:25.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:25.229+0000 7faf91600640 1 -- 172.21.15.43:0/290816456 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8c004b20 con 0x7faf94106f10 2024-09-19T05:02:25.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:25.229+0000 7faf91600640 1 -- 172.21.15.43:0/290816456 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf8c004cc0 con 0x7faf94106f10 2024-09-19T05:02:25.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:25.229+0000 7faf9ae00640 1 -- 172.21.15.43:0/290816456 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf9419d740 con 0x7faf94106f10 2024-09-19T05:02:25.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:25.229+0000 7faf91600640 1 -- 172.21.15.43:0/290816456 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8c010760 con 0x7faf94106f10 2024-09-19T05:02:25.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:25.229+0000 7faf9ae00640 1 -- 172.21.15.43:0/290816456 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf941a5020 con 0x7faf94106f10 2024-09-19T05:02:25.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:25.230+0000 7faf9ae00640 1 -- 172.21.15.43:0/290816456 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf941021c0 con 0x7faf94106f10 2024-09-19T05:02:25.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:25.232+0000 7faf91600640 1 -- 172.21.15.43:0/290816456 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7faf8c010900 con 0x7faf94106f10 2024-09-19T05:02:25.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:25.234+0000 7faf91600640 1 --2- 172.21.15.43:0/290816456 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7faf780a32c0 0x7faf780a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:25.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:25.234+0000 7faf91600640 1 -- 172.21.15.43:0/290816456 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7879+0+0 (secure 0 0 0) 0x7faf8c0cff40 con 0x7faf94106f10 2024-09-19T05:02:25.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:25.234+0000 7faf93e00640 1 --2- 172.21.15.43:0/290816456 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7faf780a32c0 0x7faf780a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:25.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:25.235+0000 7faf93e00640 1 --2- 172.21.15.43:0/290816456 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7faf780a32c0 0x7faf780a5780 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7faf84000c00 tx=0x7faf84005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:25.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:25.236+0000 7faf91600640 1 -- 172.21.15.43:0/290816456 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7faf8c09cd70 con 0x7faf94106f10 2024-09-19T05:02:25.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:25.461+0000 7faf9ae00640 1 -- 172.21.15.43:0/290816456 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7faf94199800 con 0x7faf94106f10 2024-09-19T05:02:25.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:25 smithi096 ceph-mon[35085]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T05:02:25.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:25 smithi043 ceph-mon[30800]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T05:02:25.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:25 smithi151 ceph-mon[35425]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T05:02:26.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:26.416+0000 7faf91600640 1 -- 172.21.15.43:0/290816456 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7faf94199800 con 0x7faf94106f10 2024-09-19T05:02:26.418 INFO:teuthology.orchestra.run.smithi043.stderr:new fs with metadata pool 2 and data pool 3 2024-09-19T05:02:26.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:26.421+0000 7faf9ae00640 1 -- 172.21.15.43:0/290816456 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7faf780a32c0 msgr2=0x7faf780a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:26.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:26.421+0000 7faf9ae00640 1 --2- 172.21.15.43:0/290816456 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7faf780a32c0 0x7faf780a5780 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7faf84000c00 tx=0x7faf84005b50 comp rx=0 tx=0).stop 2024-09-19T05:02:26.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:26.421+0000 7faf9ae00640 1 -- 172.21.15.43:0/290816456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf94106f10 msgr2=0x7faf9419cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:26.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:26.421+0000 7faf9ae00640 1 --2- 172.21.15.43:0/290816456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf94106f10 0x7faf9419cd70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faf8c00eb10 tx=0x7faf8c00c2d0 comp rx=0 tx=0).stop 2024-09-19T05:02:26.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:26.422+0000 7faf9ae00640 1 -- 172.21.15.43:0/290816456 shutdown_connections 2024-09-19T05:02:26.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:26.422+0000 7faf9ae00640 1 --2- 172.21.15.43:0/290816456 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7faf780a32c0 0x7faf780a5780 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:26.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:26.422+0000 7faf9ae00640 1 --2- 172.21.15.43:0/290816456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf94106f10 0x7faf9419cd70 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:26.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:26.422+0000 7faf9ae00640 1 --2- 172.21.15.43:0/290816456 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf94102790 0x7faf941989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:26.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:26.422+0000 7faf9ae00640 1 --2- 172.21.15.43:0/290816456 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf94101de0 0x7faf941984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:26.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:26.422+0000 7faf9ae00640 1 -- 172.21.15.43:0/290816456 >> 172.21.15.43:0/290816456 conn(0x7faf940fd8c0 msgr2=0x7faf940febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:26.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:26.422+0000 7faf9ae00640 1 -- 172.21.15.43:0/290816456 shutdown_connections 2024-09-19T05:02:26.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:26.422+0000 7faf9ae00640 1 -- 172.21.15.43:0/290816456 wait complete. 2024-09-19T05:02:26.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:26 smithi096 ceph-mon[35085]: pgmap v290: 97 pgs: 10 creating+peering, 62 unknown, 25 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:26.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:26 smithi096 ceph-mon[35085]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T05:02:26.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:26 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/290816456' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T05:02:26.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:26 smithi096 ceph-mon[35085]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T05:02:26.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:26 smithi096 ceph-mon[35085]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-19T05:02:26.774 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:26 smithi043 ceph-mon[30800]: pgmap v290: 97 pgs: 10 creating+peering, 62 unknown, 25 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:26.774 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:26 smithi043 ceph-mon[30800]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T05:02:26.775 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:26 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/290816456' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T05:02:26.775 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:26 smithi043 ceph-mon[30800]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T05:02:26.775 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:26 smithi043 ceph-mon[30800]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-19T05:02:26.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:26 smithi151 ceph-mon[35425]: pgmap v290: 97 pgs: 10 creating+peering, 62 unknown, 25 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:26.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:26 smithi151 ceph-mon[35425]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T05:02:26.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:26 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/290816456' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T05:02:26.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:26 smithi151 ceph-mon[35425]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T05:02:26.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:26 smithi151 ceph-mon[35425]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-19T05:02:27.294 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T05:02:27.677 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:02:27.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:27 smithi096 ceph-mon[35085]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T05:02:27.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:27 smithi096 ceph-mon[35085]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T05:02:27.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:27 smithi096 ceph-mon[35085]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-19T05:02:27.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:27 smithi096 ceph-mon[35085]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-19T05:02:27.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:27 smithi043 ceph-mon[30800]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T05:02:27.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:27 smithi043 ceph-mon[30800]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T05:02:27.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:27 smithi043 ceph-mon[30800]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-19T05:02:27.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:27 smithi043 ceph-mon[30800]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-19T05:02:27.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:27 smithi151 ceph-mon[35425]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T05:02:27.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:27 smithi151 ceph-mon[35425]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T05:02:27.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:27 smithi151 ceph-mon[35425]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-19T05:02:27.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:27 smithi151 ceph-mon[35425]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-19T05:02:28.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.244+0000 7f8a27e00640 1 -- 172.21.15.43:0/3558672495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a28074eb0 msgr2=0x7f8a281124a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:28.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.244+0000 7f8a27e00640 1 --2- 172.21.15.43:0/3558672495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a28074eb0 0x7f8a281124a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8a14009900 tx=0x7f8a1402a480 comp rx=0 tx=0).stop 2024-09-19T05:02:28.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.244+0000 7f8a27e00640 1 -- 172.21.15.43:0/3558672495 shutdown_connections 2024-09-19T05:02:28.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.244+0000 7f8a27e00640 1 --2- 172.21.15.43:0/3558672495 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a281129e0 0x7f8a28114dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:28.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.244+0000 7f8a27e00640 1 --2- 172.21.15.43:0/3558672495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a28074eb0 0x7f8a281124a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:28.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.244+0000 7f8a27e00640 1 --2- 172.21.15.43:0/3558672495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a28074590 0x7f8a28074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:28.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.244+0000 7f8a27e00640 1 -- 172.21.15.43:0/3558672495 >> 172.21.15.43:0/3558672495 conn(0x7f8a280fd710 msgr2=0x7f8a280ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:28.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.244+0000 7f8a27e00640 1 -- 172.21.15.43:0/3558672495 shutdown_connections 2024-09-19T05:02:28.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.244+0000 7f8a27e00640 1 -- 172.21.15.43:0/3558672495 wait complete. 2024-09-19T05:02:28.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.245+0000 7f8a27e00640 1 Processor -- start 2024-09-19T05:02:28.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.245+0000 7f8a27e00640 1 -- start start 2024-09-19T05:02:28.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.245+0000 7f8a27e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a28074590 0x7f8a281a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:28.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.245+0000 7f8a27e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a28074eb0 0x7f8a281a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:28.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.246+0000 7f8a27e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a281129e0 0x7f8a2819e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:28.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.246+0000 7f8a27e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8a28118400 con 0x7f8a28074eb0 2024-09-19T05:02:28.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.246+0000 7f8a27e00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8a28118280 con 0x7f8a281129e0 2024-09-19T05:02:28.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.246+0000 7f8a27e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8a28118580 con 0x7f8a28074590 2024-09-19T05:02:28.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.246+0000 7f8a26000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a28074eb0 0x7f8a281a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:28.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.246+0000 7f8a26000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a28074eb0 0x7f8a281a4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40592/0 (socket says 172.21.15.43:40592) 2024-09-19T05:02:28.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.246+0000 7f8a26000640 1 -- 172.21.15.43:0/1941887440 learned_addr learned my addr 172.21.15.43:0/1941887440 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:02:28.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.246+0000 7f8a27400640 1 --2- 172.21.15.43:0/1941887440 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a281129e0 0x7f8a2819e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:28.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.246+0000 7f8a26a00640 1 --2- 172.21.15.43:0/1941887440 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a28074590 0x7f8a281a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:28.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.246+0000 7f8a26000640 1 -- 172.21.15.43:0/1941887440 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a28074590 msgr2=0x7f8a281a4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:28.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.246+0000 7f8a26000640 1 --2- 172.21.15.43:0/1941887440 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a28074590 0x7f8a281a4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:28.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.246+0000 7f8a26000640 1 -- 172.21.15.43:0/1941887440 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a281129e0 msgr2=0x7f8a2819e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:28.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.246+0000 7f8a26000640 1 --2- 172.21.15.43:0/1941887440 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a281129e0 0x7f8a2819e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:28.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.246+0000 7f8a26000640 1 -- 172.21.15.43:0/1941887440 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a2819efc0 con 0x7f8a28074eb0 2024-09-19T05:02:28.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.247+0000 7f8a26000640 1 --2- 172.21.15.43:0/1941887440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a28074eb0 0x7f8a281a4bc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8a14004770 tx=0x7f8a14003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:28.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.247+0000 7f8a1f400640 1 -- 172.21.15.43:0/1941887440 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1403f070 con 0x7f8a28074eb0 2024-09-19T05:02:28.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.247+0000 7f8a1f400640 1 -- 172.21.15.43:0/1941887440 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a14031ca0 con 0x7f8a28074eb0 2024-09-19T05:02:28.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.247+0000 7f8a27e00640 1 -- 172.21.15.43:0/1941887440 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a2819f250 con 0x7f8a28074eb0 2024-09-19T05:02:28.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.247+0000 7f8a1f400640 1 -- 172.21.15.43:0/1941887440 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a14035030 con 0x7f8a28074eb0 2024-09-19T05:02:28.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.247+0000 7f8a27e00640 1 -- 172.21.15.43:0/1941887440 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a281ab430 con 0x7f8a28074eb0 2024-09-19T05:02:28.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.249+0000 7f8a27e00640 1 -- 172.21.15.43:0/1941887440 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a28075cd0 con 0x7f8a28074eb0 2024-09-19T05:02:28.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.250+0000 7f8a1f400640 1 -- 172.21.15.43:0/1941887440 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f8a14042070 con 0x7f8a28074eb0 2024-09-19T05:02:28.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.252+0000 7f8a1f400640 1 --2- 172.21.15.43:0/1941887440 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f8a0c0a32c0 0x7f8a0c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:28.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.252+0000 7f8a1f400640 1 -- 172.21.15.43:0/1941887440 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f8a140efa30 con 0x7f8a28074eb0 2024-09-19T05:02:28.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.252+0000 7f8a26a00640 1 --2- 172.21.15.43:0/1941887440 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f8a0c0a32c0 0x7f8a0c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:28.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.253+0000 7f8a26a00640 1 --2- 172.21.15.43:0/1941887440 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f8a0c0a32c0 0x7f8a0c0a5780 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8a18009bd0 tx=0x7f8a18005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:28.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.256+0000 7f8a1f400640 1 -- 172.21.15.43:0/1941887440 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8a140bc270 con 0x7f8a28074eb0 2024-09-19T05:02:28.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.446+0000 7f8a27e00640 1 -- 172.21.15.43:0/1941887440 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8a28104e50 con 0x7f8a28074eb0 2024-09-19T05:02:28.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.448+0000 7f8a1f400640 1 -- 172.21.15.43:0/1941887440 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22237 (secure 0 0 0) 0x7f8a140c0570 con 0x7f8a28074eb0 2024-09-19T05:02:28.449 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:02:28.450 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":83,"fsid":"1456a810-7643-11ef-bceb-c7b262605968","created":"2024-09-19T04:54:14.706644+0000","modified":"2024-09-19T05:02:25.462897+0000","last_up_change":"2024-09-19T05:01:27.348740+0000","last_in_change":"2024-09-19T05:01:03.396477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:57:27.642705+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:02:20.685123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:02:23.393596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"81a317a8-7ec8-45c3-bc10-7da023283635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6803","nonce":825351486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6805","nonce":825351486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6809","nonce":825351486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6807","nonce":825351486}]},"public_addr":"172.21.15.43:6803/825351486","cluster_addr":"172.21.15.43:6805/825351486","heartbeat_back_addr":"172.21.15.43:6809/825351486","heartbeat_front_addr":"172.21.15.43:6807/825351486","state":["exists","up"]},{"osd":1,"uuid":"e96fcfac-4fff-4a36-b38b-aebf57842740","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6801","nonce":2140095130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6803","nonce":2140095130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6807","nonce":2140095130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6805","nonce":2140095130}]},"public_addr":"172.21.15.96:6801/2140095130","cluster_addr":"172.21.15.96:6803/2140095130","heartbeat_back_addr":"172.21.15.96:6807/2140095130","heartbeat_front_addr":"172.21.15.96:6805/2140095130","state":["exists","up"]},{"osd":2,"uuid":"b659869e-114a-4f56-8aac-a2ed602b4058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6801","nonce":2047147996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6803","nonce":2047147996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6807","nonce":2047147996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6805","nonce":2047147996}]},"public_addr":"172.21.15.151:6801/2047147996","cluster_addr":"172.21.15.151:6803/2047147996","heartbeat_back_addr":"172.21.15.151:6807/2047147996","heartbeat_front_addr":"172.21.15.151:6805/2047147996","state":["exists","up"]},{"osd":3,"uuid":"7be64b2c-e45c-493d-a610-4ce41196061f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6811","nonce":1734673762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6813","nonce":1734673762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6817","nonce":1734673762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6815","nonce":1734673762}]},"public_addr":"172.21.15.43:6811/1734673762","cluster_addr":"172.21.15.43:6813/1734673762","heartbeat_back_addr":"172.21.15.43:6817/1734673762","heartbeat_front_addr":"172.21.15.43:6815/1734673762","state":["exists","up"]},{"osd":4,"uuid":"196d5af1-a6e5-496c-8933-4e58892f4360","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6809","nonce":3184785362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6811","nonce":3184785362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6815","nonce":3184785362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6813","nonce":3184785362}]},"public_addr":"172.21.15.96:6809/3184785362","cluster_addr":"172.21.15.96:6811/3184785362","heartbeat_back_addr":"172.21.15.96:6815/3184785362","heartbeat_front_addr":"172.21.15.96:6813/3184785362","state":["exists","up"]},{"osd":5,"uuid":"5492d474-8509-4865-bc2e-d08e31558a75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6809","nonce":2878451924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6811","nonce":2878451924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6815","nonce":2878451924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6813","nonce":2878451924}]},"public_addr":"172.21.15.151:6809/2878451924","cluster_addr":"172.21.15.151:6811/2878451924","heartbeat_back_addr":"172.21.15.151:6815/2878451924","heartbeat_front_addr":"172.21.15.151:6813/2878451924","state":["exists","up"]},{"osd":6,"uuid":"fb8457ed-7bb9-4435-9d15-c1cbfbddfcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6819","nonce":2008630290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6821","nonce":2008630290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6825","nonce":2008630290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6823","nonce":2008630290}]},"public_addr":"172.21.15.43:6819/2008630290","cluster_addr":"172.21.15.43:6821/2008630290","heartbeat_back_addr":"172.21.15.43:6825/2008630290","heartbeat_front_addr":"172.21.15.43:6823/2008630290","state":["exists","up"]},{"osd":7,"uuid":"02046db8-30de-4e92-b25a-1f07d6190e91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6817","nonce":3588713275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6819","nonce":3588713275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6823","nonce":3588713275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6821","nonce":3588713275}]},"public_addr":"172.21.15.96:6817/3588713275","cluster_addr":"172.21.15.96:6819/3588713275","heartbeat_back_addr":"172.21.15.96:6823/3588713275","heartbeat_front_addr":"172.21.15.96:6821/3588713275","state":["exists","up"]},{"osd":8,"uuid":"a5bb4218-51f3-4044-b43e-ebfc764f49ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6817","nonce":725316393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6819","nonce":725316393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6823","nonce":725316393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6821","nonce":725316393}]},"public_addr":"172.21.15.151:6817/725316393","cluster_addr":"172.21.15.151:6819/725316393","heartbeat_back_addr":"172.21.15.151:6823/725316393","heartbeat_front_addr":"172.21.15.151:6821/725316393","state":["exists","up"]},{"osd":9,"uuid":"6c19f65c-9750-4baa-9d63-7b8bd03e6955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6827","nonce":4053438621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6829","nonce":4053438621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6833","nonce":4053438621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6831","nonce":4053438621}]},"public_addr":"172.21.15.43:6827/4053438621","cluster_addr":"172.21.15.43:6829/4053438621","heartbeat_back_addr":"172.21.15.43:6833/4053438621","heartbeat_front_addr":"172.21.15.43:6831/4053438621","state":["exists","up"]},{"osd":10,"uuid":"a7aa7da3-8eab-47fe-a351-298b6ff75f14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6825","nonce":2403210086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6827","nonce":2403210086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6831","nonce":2403210086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6829","nonce":2403210086}]},"public_addr":"172.21.15.96:6825/2403210086","cluster_addr":"172.21.15.96:6827/2403210086","heartbeat_back_addr":"172.21.15.96:6831/2403210086","heartbeat_front_addr":"172.21.15.96:6829/2403210086","state":["exists","up"]},{"osd":11,"uuid":"760b6afc-253d-44dd-88e9-dcc97f67c9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6825","nonce":292202881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6827","nonce":292202881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6831","nonce":292202881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6829","nonce":292202881}]},"public_addr":"172.21.15.151:6825/292202881","cluster_addr":"172.21.15.151:6827/292202881","heartbeat_back_addr":"172.21.15.151:6831/292202881","heartbeat_front_addr":"172.21.15.151:6829/292202881","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:56:34.716991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:56:59.321174+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:57:24.281696+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:57:53.443880+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:58:19.205873+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:58:44.504873+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:13.650731+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:39.137876+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:04.579893+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:34.337312+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:59.434658+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:01:24.930529+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/898247657":"2024-09-20T04:55:07.037117+0000","172.21.15.43:6801/3241799377":"2024-09-20T04:55:07.037117+0000","172.21.15.43:6800/3241799377":"2024-09-20T04:55:07.037117+0000","172.21.15.43:0/3256350596":"2024-09-20T04:55:07.037117+0000","172.21.15.43:6801/587463881":"2024-09-20T04:54:39.548065+0000","172.21.15.43:6800/587463881":"2024-09-20T04:54:39.548065+0000","172.21.15.43:0/4271788315":"2024-09-20T04:55:07.037117+0000","172.21.15.43:0/6132665":"2024-09-20T04:54:39.548065+0000","172.21.15.43:0/632163478":"2024-09-20T04:54:39.548065+0000","172.21.15.43:0/330854185":"2024-09-20T04:54:39.548065+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:02:28.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.450+0000 7f8a27e00640 1 -- 172.21.15.43:0/1941887440 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f8a0c0a32c0 msgr2=0x7f8a0c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:28.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.450+0000 7f8a27e00640 1 --2- 172.21.15.43:0/1941887440 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f8a0c0a32c0 0x7f8a0c0a5780 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8a18009bd0 tx=0x7f8a18005f90 comp rx=0 tx=0).stop 2024-09-19T05:02:28.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.450+0000 7f8a27e00640 1 -- 172.21.15.43:0/1941887440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a28074eb0 msgr2=0x7f8a281a4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:28.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.450+0000 7f8a27e00640 1 --2- 172.21.15.43:0/1941887440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a28074eb0 0x7f8a281a4bc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8a14004770 tx=0x7f8a14003c10 comp rx=0 tx=0).stop 2024-09-19T05:02:28.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.450+0000 7f8a27e00640 1 -- 172.21.15.43:0/1941887440 shutdown_connections 2024-09-19T05:02:28.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.450+0000 7f8a27e00640 1 --2- 172.21.15.43:0/1941887440 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f8a0c0a32c0 0x7f8a0c0a5780 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:28.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.450+0000 7f8a27e00640 1 --2- 172.21.15.43:0/1941887440 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a281129e0 0x7f8a2819e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:28.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.450+0000 7f8a27e00640 1 --2- 172.21.15.43:0/1941887440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a28074eb0 0x7f8a281a4bc0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:28.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.450+0000 7f8a27e00640 1 --2- 172.21.15.43:0/1941887440 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a28074590 0x7f8a281a4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:28.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.450+0000 7f8a27e00640 1 -- 172.21.15.43:0/1941887440 >> 172.21.15.43:0/1941887440 conn(0x7f8a280fd710 msgr2=0x7f8a280ff120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:28.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.451+0000 7f8a27e00640 1 -- 172.21.15.43:0/1941887440 shutdown_connections 2024-09-19T05:02:28.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:28.451+0000 7f8a27e00640 1 -- 172.21.15.43:0/1941887440 wait complete. 2024-09-19T05:02:28.681 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:28 smithi043 ceph-mon[30800]: pgmap v293: 97 pgs: 28 unknown, 69 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:28.682 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:28 smithi043 ceph-mon[30800]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T05:02:28.682 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:28 smithi043 ceph-mon[30800]: Cluster is now healthy 2024-09-19T05:02:28.682 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:28 smithi043 ceph-mon[30800]: mds.? [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] up:active 2024-09-19T05:02:28.682 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:28 smithi043 ceph-mon[30800]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T05:02:28.682 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:28 smithi043 ceph-mon[30800]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T05:02:28.723 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 05:02:28 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-b[65164]: 2024-09-19T05:02:28.543+0000 7f6a0a343640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T05:02:28.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:28 smithi096 ceph-mon[35085]: pgmap v293: 97 pgs: 28 unknown, 69 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:28.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:28 smithi096 ceph-mon[35085]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T05:02:28.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:28 smithi096 ceph-mon[35085]: Cluster is now healthy 2024-09-19T05:02:28.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:28 smithi096 ceph-mon[35085]: mds.? [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] up:active 2024-09-19T05:02:28.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:28 smithi096 ceph-mon[35085]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T05:02:28.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:28 smithi096 ceph-mon[35085]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T05:02:28.906 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:28 smithi151 ceph-mon[35425]: pgmap v293: 97 pgs: 28 unknown, 69 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:02:28.906 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:28 smithi151 ceph-mon[35425]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T05:02:28.906 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:28 smithi151 ceph-mon[35425]: Cluster is now healthy 2024-09-19T05:02:28.906 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:28 smithi151 ceph-mon[35425]: mds.? [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] up:active 2024-09-19T05:02:28.906 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:28 smithi151 ceph-mon[35425]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T05:02:28.906 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:28 smithi151 ceph-mon[35425]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T05:02:29.332 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T05:02:29.559 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:29 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/1941887440' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:02:29.712 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:02:29.722 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:29 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/1941887440' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:02:29.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:29 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/1941887440' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:02:30.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.427+0000 7f01f1e00640 1 -- 172.21.15.43:0/3299909772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01ec1049a0 msgr2=0x7f01ec1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:30.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.427+0000 7f01f1e00640 1 --2- 172.21.15.43:0/3299909772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01ec1049a0 0x7f01ec1089f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f01d4009900 tx=0x7f01d402a3f0 comp rx=0 tx=0).stop 2024-09-19T05:02:30.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.427+0000 7f01f1e00640 1 -- 172.21.15.43:0/3299909772 shutdown_connections 2024-09-19T05:02:30.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.427+0000 7f01f1e00640 1 --2- 172.21.15.43:0/3299909772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01ec109120 0x7f01ec110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:30.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.427+0000 7f01f1e00640 1 --2- 172.21.15.43:0/3299909772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01ec1049a0 0x7f01ec1089f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:30.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.427+0000 7f01f1e00640 1 --2- 172.21.15.43:0/3299909772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01ec103ff0 0x7f01ec1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:30.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.427+0000 7f01f1e00640 1 -- 172.21.15.43:0/3299909772 >> 172.21.15.43:0/3299909772 conn(0x7f01ec0ffa90 msgr2=0x7f01ec101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:30.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.427+0000 7f01f1e00640 1 -- 172.21.15.43:0/3299909772 shutdown_connections 2024-09-19T05:02:30.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.427+0000 7f01f1e00640 1 -- 172.21.15.43:0/3299909772 wait complete. 2024-09-19T05:02:30.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.428+0000 7f01f1e00640 1 Processor -- start 2024-09-19T05:02:30.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.428+0000 7f01f1e00640 1 -- start start 2024-09-19T05:02:30.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.429+0000 7f01f1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01ec103ff0 0x7f01ec19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:30.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.429+0000 7f01f1e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01ec1049a0 0x7f01ec19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:30.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.429+0000 7f01f1e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01ec109120 0x7f01ec19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:30.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.429+0000 7f01f1e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f01ec114190 con 0x7f01ec103ff0 2024-09-19T05:02:30.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.429+0000 7f01f1e00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f01ec114010 con 0x7f01ec109120 2024-09-19T05:02:30.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.429+0000 7f01f1e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f01ec114310 con 0x7f01ec1049a0 2024-09-19T05:02:30.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.429+0000 7f01eaa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01ec103ff0 0x7f01ec19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:30.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.429+0000 7f01eaa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01ec103ff0 0x7f01ec19a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40598/0 (socket says 172.21.15.43:40598) 2024-09-19T05:02:30.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.429+0000 7f01eaa00640 1 -- 172.21.15.43:0/375680993 learned_addr learned my addr 172.21.15.43:0/375680993 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:02:30.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.429+0000 7f01e2000640 1 --2- 172.21.15.43:0/375680993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01ec1049a0 0x7f01ec19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:30.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.429+0000 7f01eb400640 1 --2- 172.21.15.43:0/375680993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01ec109120 0x7f01ec19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:30.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.429+0000 7f01eaa00640 1 -- 172.21.15.43:0/375680993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01ec1049a0 msgr2=0x7f01ec19ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:30.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.429+0000 7f01eaa00640 1 --2- 172.21.15.43:0/375680993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01ec1049a0 0x7f01ec19ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:30.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.429+0000 7f01eaa00640 1 -- 172.21.15.43:0/375680993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01ec109120 msgr2=0x7f01ec19eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:30.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.429+0000 7f01eaa00640 1 --2- 172.21.15.43:0/375680993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01ec109120 0x7f01ec19eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:30.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.429+0000 7f01eaa00640 1 -- 172.21.15.43:0/375680993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ec19f5c0 con 0x7f01ec103ff0 2024-09-19T05:02:30.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.430+0000 7f01eaa00640 1 --2- 172.21.15.43:0/375680993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01ec103ff0 0x7f01ec19a610 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f01dc00eb90 tx=0x7f01dc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:30.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.430+0000 7f01f0a00640 1 -- 172.21.15.43:0/375680993 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01dc004d20 con 0x7f01ec103ff0 2024-09-19T05:02:30.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.430+0000 7f01f0a00640 1 -- 172.21.15.43:0/375680993 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01dc00ee90 con 0x7f01ec103ff0 2024-09-19T05:02:30.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.430+0000 7f01f0a00640 1 -- 172.21.15.43:0/375680993 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01dc010830 con 0x7f01ec103ff0 2024-09-19T05:02:30.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.430+0000 7f01f1e00640 1 -- 172.21.15.43:0/375680993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01ec19f8b0 con 0x7f01ec103ff0 2024-09-19T05:02:30.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.430+0000 7f01f1e00640 1 -- 172.21.15.43:0/375680993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01ec1a7190 con 0x7f01ec103ff0 2024-09-19T05:02:30.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.431+0000 7f01f1e00640 1 -- 172.21.15.43:0/375680993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01ec1043d0 con 0x7f01ec103ff0 2024-09-19T05:02:30.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.433+0000 7f01f0a00640 1 -- 172.21.15.43:0/375680993 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f01dc005020 con 0x7f01ec103ff0 2024-09-19T05:02:30.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.435+0000 7f01f0a00640 1 --2- 172.21.15.43:0/375680993 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f01d00a32c0 0x7f01d00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:30.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.436+0000 7f01f0a00640 1 -- 172.21.15.43:0/375680993 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f01dc0cfe00 con 0x7f01ec103ff0 2024-09-19T05:02:30.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.436+0000 7f01e2000640 1 --2- 172.21.15.43:0/375680993 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f01d00a32c0 0x7f01d00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:30.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.436+0000 7f01e2000640 1 --2- 172.21.15.43:0/375680993 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f01d00a32c0 0x7f01d00a5780 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f01d4000c00 tx=0x7f01d4005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:30.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.440+0000 7f01f0a00640 1 -- 172.21.15.43:0/375680993 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f01dc09cbd0 con 0x7f01ec103ff0 2024-09-19T05:02:30.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.625+0000 7f01f1e00640 1 -- 172.21.15.43:0/375680993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f01ec10c3c0 con 0x7f01ec103ff0 2024-09-19T05:02:30.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.626+0000 7f01f0a00640 1 -- 172.21.15.43:0/375680993 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22237 (secure 0 0 0) 0x7f01dc09c4d0 con 0x7f01ec103ff0 2024-09-19T05:02:30.627 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:02:30.628 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":83,"fsid":"1456a810-7643-11ef-bceb-c7b262605968","created":"2024-09-19T04:54:14.706644+0000","modified":"2024-09-19T05:02:25.462897+0000","last_up_change":"2024-09-19T05:01:27.348740+0000","last_in_change":"2024-09-19T05:01:03.396477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:57:27.642705+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:02:20.685123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:02:23.393596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"81a317a8-7ec8-45c3-bc10-7da023283635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6803","nonce":825351486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6805","nonce":825351486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6809","nonce":825351486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6807","nonce":825351486}]},"public_addr":"172.21.15.43:6803/825351486","cluster_addr":"172.21.15.43:6805/825351486","heartbeat_back_addr":"172.21.15.43:6809/825351486","heartbeat_front_addr":"172.21.15.43:6807/825351486","state":["exists","up"]},{"osd":1,"uuid":"e96fcfac-4fff-4a36-b38b-aebf57842740","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6801","nonce":2140095130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6803","nonce":2140095130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6807","nonce":2140095130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6805","nonce":2140095130}]},"public_addr":"172.21.15.96:6801/2140095130","cluster_addr":"172.21.15.96:6803/2140095130","heartbeat_back_addr":"172.21.15.96:6807/2140095130","heartbeat_front_addr":"172.21.15.96:6805/2140095130","state":["exists","up"]},{"osd":2,"uuid":"b659869e-114a-4f56-8aac-a2ed602b4058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6801","nonce":2047147996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6803","nonce":2047147996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6807","nonce":2047147996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6805","nonce":2047147996}]},"public_addr":"172.21.15.151:6801/2047147996","cluster_addr":"172.21.15.151:6803/2047147996","heartbeat_back_addr":"172.21.15.151:6807/2047147996","heartbeat_front_addr":"172.21.15.151:6805/2047147996","state":["exists","up"]},{"osd":3,"uuid":"7be64b2c-e45c-493d-a610-4ce41196061f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6811","nonce":1734673762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6813","nonce":1734673762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6817","nonce":1734673762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6815","nonce":1734673762}]},"public_addr":"172.21.15.43:6811/1734673762","cluster_addr":"172.21.15.43:6813/1734673762","heartbeat_back_addr":"172.21.15.43:6817/1734673762","heartbeat_front_addr":"172.21.15.43:6815/1734673762","state":["exists","up"]},{"osd":4,"uuid":"196d5af1-a6e5-496c-8933-4e58892f4360","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6809","nonce":3184785362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6811","nonce":3184785362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6815","nonce":3184785362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6813","nonce":3184785362}]},"public_addr":"172.21.15.96:6809/3184785362","cluster_addr":"172.21.15.96:6811/3184785362","heartbeat_back_addr":"172.21.15.96:6815/3184785362","heartbeat_front_addr":"172.21.15.96:6813/3184785362","state":["exists","up"]},{"osd":5,"uuid":"5492d474-8509-4865-bc2e-d08e31558a75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6809","nonce":2878451924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6811","nonce":2878451924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6815","nonce":2878451924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6813","nonce":2878451924}]},"public_addr":"172.21.15.151:6809/2878451924","cluster_addr":"172.21.15.151:6811/2878451924","heartbeat_back_addr":"172.21.15.151:6815/2878451924","heartbeat_front_addr":"172.21.15.151:6813/2878451924","state":["exists","up"]},{"osd":6,"uuid":"fb8457ed-7bb9-4435-9d15-c1cbfbddfcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6819","nonce":2008630290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6821","nonce":2008630290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6825","nonce":2008630290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6823","nonce":2008630290}]},"public_addr":"172.21.15.43:6819/2008630290","cluster_addr":"172.21.15.43:6821/2008630290","heartbeat_back_addr":"172.21.15.43:6825/2008630290","heartbeat_front_addr":"172.21.15.43:6823/2008630290","state":["exists","up"]},{"osd":7,"uuid":"02046db8-30de-4e92-b25a-1f07d6190e91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6817","nonce":3588713275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6819","nonce":3588713275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6823","nonce":3588713275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6821","nonce":3588713275}]},"public_addr":"172.21.15.96:6817/3588713275","cluster_addr":"172.21.15.96:6819/3588713275","heartbeat_back_addr":"172.21.15.96:6823/3588713275","heartbeat_front_addr":"172.21.15.96:6821/3588713275","state":["exists","up"]},{"osd":8,"uuid":"a5bb4218-51f3-4044-b43e-ebfc764f49ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6817","nonce":725316393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6819","nonce":725316393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6823","nonce":725316393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6821","nonce":725316393}]},"public_addr":"172.21.15.151:6817/725316393","cluster_addr":"172.21.15.151:6819/725316393","heartbeat_back_addr":"172.21.15.151:6823/725316393","heartbeat_front_addr":"172.21.15.151:6821/725316393","state":["exists","up"]},{"osd":9,"uuid":"6c19f65c-9750-4baa-9d63-7b8bd03e6955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6827","nonce":4053438621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6829","nonce":4053438621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6833","nonce":4053438621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6831","nonce":4053438621}]},"public_addr":"172.21.15.43:6827/4053438621","cluster_addr":"172.21.15.43:6829/4053438621","heartbeat_back_addr":"172.21.15.43:6833/4053438621","heartbeat_front_addr":"172.21.15.43:6831/4053438621","state":["exists","up"]},{"osd":10,"uuid":"a7aa7da3-8eab-47fe-a351-298b6ff75f14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6825","nonce":2403210086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6827","nonce":2403210086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6831","nonce":2403210086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6829","nonce":2403210086}]},"public_addr":"172.21.15.96:6825/2403210086","cluster_addr":"172.21.15.96:6827/2403210086","heartbeat_back_addr":"172.21.15.96:6831/2403210086","heartbeat_front_addr":"172.21.15.96:6829/2403210086","state":["exists","up"]},{"osd":11,"uuid":"760b6afc-253d-44dd-88e9-dcc97f67c9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6825","nonce":292202881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6827","nonce":292202881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6831","nonce":292202881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6829","nonce":292202881}]},"public_addr":"172.21.15.151:6825/292202881","cluster_addr":"172.21.15.151:6827/292202881","heartbeat_back_addr":"172.21.15.151:6831/292202881","heartbeat_front_addr":"172.21.15.151:6829/292202881","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:56:34.716991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:56:59.321174+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:57:24.281696+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:57:53.443880+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:58:19.205873+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:58:44.504873+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:13.650731+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:39.137876+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:04.579893+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:34.337312+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:59.434658+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:01:24.930529+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/898247657":"2024-09-20T04:55:07.037117+0000","172.21.15.43:6801/3241799377":"2024-09-20T04:55:07.037117+0000","172.21.15.43:6800/3241799377":"2024-09-20T04:55:07.037117+0000","172.21.15.43:0/3256350596":"2024-09-20T04:55:07.037117+0000","172.21.15.43:6801/587463881":"2024-09-20T04:54:39.548065+0000","172.21.15.43:6800/587463881":"2024-09-20T04:54:39.548065+0000","172.21.15.43:0/4271788315":"2024-09-20T04:55:07.037117+0000","172.21.15.43:0/6132665":"2024-09-20T04:54:39.548065+0000","172.21.15.43:0/632163478":"2024-09-20T04:54:39.548065+0000","172.21.15.43:0/330854185":"2024-09-20T04:54:39.548065+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:02:30.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.628+0000 7f01f1e00640 1 -- 172.21.15.43:0/375680993 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f01d00a32c0 msgr2=0x7f01d00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:30.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.628+0000 7f01f1e00640 1 --2- 172.21.15.43:0/375680993 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f01d00a32c0 0x7f01d00a5780 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f01d4000c00 tx=0x7f01d4005b50 comp rx=0 tx=0).stop 2024-09-19T05:02:30.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.628+0000 7f01f1e00640 1 -- 172.21.15.43:0/375680993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01ec103ff0 msgr2=0x7f01ec19a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:30.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.628+0000 7f01f1e00640 1 --2- 172.21.15.43:0/375680993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01ec103ff0 0x7f01ec19a610 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f01dc00eb90 tx=0x7f01dc00c6a0 comp rx=0 tx=0).stop 2024-09-19T05:02:30.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.629+0000 7f01f1e00640 1 -- 172.21.15.43:0/375680993 shutdown_connections 2024-09-19T05:02:30.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.629+0000 7f01f1e00640 1 --2- 172.21.15.43:0/375680993 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f01d00a32c0 0x7f01d00a5780 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:30.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.629+0000 7f01f1e00640 1 --2- 172.21.15.43:0/375680993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01ec109120 0x7f01ec19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:30.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.629+0000 7f01f1e00640 1 --2- 172.21.15.43:0/375680993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01ec1049a0 0x7f01ec19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:30.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.629+0000 7f01f1e00640 1 --2- 172.21.15.43:0/375680993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01ec103ff0 0x7f01ec19a610 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:30.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.629+0000 7f01f1e00640 1 -- 172.21.15.43:0/375680993 >> 172.21.15.43:0/375680993 conn(0x7f01ec0ffa90 msgr2=0x7f01ec100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:30.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.629+0000 7f01f1e00640 1 -- 172.21.15.43:0/375680993 shutdown_connections 2024-09-19T05:02:30.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:30.629+0000 7f01f1e00640 1 -- 172.21.15.43:0/375680993 wait complete. 2024-09-19T05:02:30.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:30 smithi096 ceph-mon[35085]: pgmap v294: 97 pgs: 97 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 4 op/s 2024-09-19T05:02:30.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:30 smithi043 ceph-mon[30800]: pgmap v294: 97 pgs: 97 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 4 op/s 2024-09-19T05:02:30.904 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:30 smithi151 ceph-mon[35425]: pgmap v294: 97 pgs: 97 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 4 op/s 2024-09-19T05:02:31.382 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-19T05:02:31.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:31 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/375680993' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:02:31.764 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:02:31.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:31 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/375680993' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:02:31.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:31 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/375680993' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:02:32.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:32.278+0000 7f3cf3e00640 1 -- 172.21.15.43:0/1237185726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3cf41019c0 msgr2=0x7f3cf4101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:32.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:32.278+0000 7f3cf3e00640 1 --2- 172.21.15.43:0/1237185726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3cf41019c0 0x7f3cf4101e20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3ce0009930 tx=0x7f3ce002a420 comp rx=0 tx=0).stop 2024-09-19T05:02:32.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:32.278+0000 7f3cf3e00640 1 -- 172.21.15.43:0/1237185726 shutdown_connections 2024-09-19T05:02:32.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:32.278+0000 7f3cf3e00640 1 --2- 172.21.15.43:0/1237185726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3cf4102360 0x7f3cf410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:32.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:32.278+0000 7f3cf3e00640 1 --2- 172.21.15.43:0/1237185726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3cf41019c0 0x7f3cf4101e20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:32.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:32.278+0000 7f3cf3e00640 1 --2- 172.21.15.43:0/1237185726 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3cf41079c0 0x7f3cf4107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:32.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:32.278+0000 7f3cf3e00640 1 -- 172.21.15.43:0/1237185726 >> 172.21.15.43:0/1237185726 conn(0x7f3cf40fd6e0 msgr2=0x7f3cf40ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:32.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:32.278+0000 7f3cf3e00640 1 -- 172.21.15.43:0/1237185726 shutdown_connections 2024-09-19T05:02:32.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:32.278+0000 7f3cf3e00640 1 -- 172.21.15.43:0/1237185726 wait complete. 2024-09-19T05:02:32.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:32.279+0000 7f3cf3e00640 1 Processor -- start 2024-09-19T05:02:32.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:32.280+0000 7f3cf3e00640 1 -- start start 2024-09-19T05:02:32.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:32.280+0000 7f3cf3e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3cf41019c0 0x7f3cf419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:32.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:32.280+0000 7f3cf3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3cf4102360 0x7f3cf419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:32.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:32.280+0000 7f3cf3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3cf41079c0 0x7f3cf419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:32.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:32.280+0000 7f3cf3e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf4111e10 con 0x7f3cf41079c0 2024-09-19T05:02:32.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:32.280+0000 7f3cf3e00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf4111c90 con 0x7f3cf41019c0 2024-09-19T05:02:32.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:32.280+0000 7f3cf3e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf4111f90 con 0x7f3cf4102360 2024-09-19T05:02:32.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:32.280+0000 7f3cf2a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3cf41019c0 0x7f3cf419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:32.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:32.280+0000 7f3cf2000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3cf4102360 0x7f3cf419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:32.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:32.280+0000 7f3cf3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3cf41079c0 0x7f3cf419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:32.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:32.280+0000 7f3cf2a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3cf41019c0 0x7f3cf419a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:37184/0 (socket says 172.21.15.43:37184) 2024-09-19T05:02:32.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:32.280+0000 7f3cf2a00640 1 -- 172.21.15.43:0/4183400404 learned_addr learned my addr 172.21.15.43:0/4183400404 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:02:32.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:32.281+0000 7f3cf2000640 1 -- 172.21.15.43:0/4183400404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3cf41019c0 msgr2=0x7f3cf419a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:32.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:32.281+0000 7f3cf2000640 1 --2- 172.21.15.43:0/4183400404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3cf41019c0 0x7f3cf419a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:32.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:32.281+0000 7f3cf2000640 1 -- 172.21.15.43:0/4183400404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3cf41079c0 msgr2=0x7f3cf419eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:32.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:32.281+0000 7f3cf2000640 1 --2- 172.21.15.43:0/4183400404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3cf41079c0 0x7f3cf419eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:32.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:32.281+0000 7f3cf2000640 1 -- 172.21.15.43:0/4183400404 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cf419f480 con 0x7f3cf4102360 2024-09-19T05:02:32.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:32.281+0000 7f3cf2000640 1 --2- 172.21.15.43:0/4183400404 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3cf4102360 0x7f3cf419aa10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3ce002a940 tx=0x7f3ce0003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:32.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:32.281+0000 7f3ceb400640 1 -- 172.21.15.43:0/4183400404 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce003f070 con 0x7f3cf4102360 2024-09-19T05:02:32.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:32.281+0000 7f3ceb400640 1 -- 172.21.15.43:0/4183400404 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ce0031c90 con 0x7f3cf4102360 2024-09-19T05:02:32.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:32.281+0000 7f3cf3e00640 1 -- 172.21.15.43:0/4183400404 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cf419f710 con 0x7f3cf4102360 2024-09-19T05:02:32.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:32.281+0000 7f3ceb400640 1 -- 172.21.15.43:0/4183400404 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce0035030 con 0x7f3cf4102360 2024-09-19T05:02:32.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:32.282+0000 7f3cf3e00640 1 -- 172.21.15.43:0/4183400404 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cf41a7050 con 0x7f3cf4102360 2024-09-19T05:02:32.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:32.283+0000 7f3cf3e00640 1 -- 172.21.15.43:0/4183400404 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cf4103100 con 0x7f3cf4102360 2024-09-19T05:02:32.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:32.286+0000 7f3ceb400640 1 -- 172.21.15.43:0/4183400404 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f3ce0042070 con 0x7f3cf4102360 2024-09-19T05:02:32.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:32.287+0000 7f3ceb400640 1 --2- 172.21.15.43:0/4183400404 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f3cd80a3240 0x7f3cd80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:32.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:32.287+0000 7f3ceb400640 1 -- 172.21.15.43:0/4183400404 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f3ce00f0370 con 0x7f3cf4102360 2024-09-19T05:02:32.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:32.287+0000 7f3cf2a00640 1 --2- 172.21.15.43:0/4183400404 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f3cd80a3240 0x7f3cd80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:32.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:32.288+0000 7f3cf2a00640 1 --2- 172.21.15.43:0/4183400404 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f3cd80a3240 0x7f3cd80a5700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4009c50 tx=0x7f3ce4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:32.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:32.288+0000 7f3ceb400640 1 -- 172.21.15.43:0/4183400404 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3ce00bcaf0 con 0x7f3cf4102360 2024-09-19T05:02:32.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:32.517+0000 7f3cf3e00640 1 -- 172.21.15.43:0/4183400404 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f3cf4101e20 con 0x7f3cf4102360 2024-09-19T05:02:32.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:32 smithi096 ceph-mon[35085]: pgmap v295: 97 pgs: 97 active+clean; 578 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-09-19T05:02:32.723 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:32 smithi151 ceph-mon[35425]: pgmap v295: 97 pgs: 97 active+clean; 578 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-09-19T05:02:32.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:32 smithi043 ceph-mon[30800]: pgmap v295: 97 pgs: 97 active+clean; 578 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-09-19T05:02:33.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:33.454+0000 7f3ceb400640 1 -- 172.21.15.43:0/4183400404 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f3cf4101e20 con 0x7f3cf4102360 2024-09-19T05:02:33.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:33.457+0000 7f3cf3e00640 1 -- 172.21.15.43:0/4183400404 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f3cd80a3240 msgr2=0x7f3cd80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:33.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:33.457+0000 7f3cf3e00640 1 --2- 172.21.15.43:0/4183400404 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f3cd80a3240 0x7f3cd80a5700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4009c50 tx=0x7f3ce4005f90 comp rx=0 tx=0).stop 2024-09-19T05:02:33.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:33.458+0000 7f3cf3e00640 1 -- 172.21.15.43:0/4183400404 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3cf4102360 msgr2=0x7f3cf419aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:33.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:33.458+0000 7f3cf3e00640 1 --2- 172.21.15.43:0/4183400404 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3cf4102360 0x7f3cf419aa10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3ce002a940 tx=0x7f3ce0003b90 comp rx=0 tx=0).stop 2024-09-19T05:02:33.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:33.458+0000 7f3cf3e00640 1 -- 172.21.15.43:0/4183400404 shutdown_connections 2024-09-19T05:02:33.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:33.458+0000 7f3cf3e00640 1 --2- 172.21.15.43:0/4183400404 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f3cd80a3240 0x7f3cd80a5700 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:33.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:33.458+0000 7f3cf3e00640 1 --2- 172.21.15.43:0/4183400404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3cf41079c0 0x7f3cf419eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:33.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:33.458+0000 7f3cf3e00640 1 --2- 172.21.15.43:0/4183400404 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3cf4102360 0x7f3cf419aa10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:33.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:33.458+0000 7f3cf3e00640 1 --2- 172.21.15.43:0/4183400404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3cf41019c0 0x7f3cf419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:33.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:33.458+0000 7f3cf3e00640 1 -- 172.21.15.43:0/4183400404 >> 172.21.15.43:0/4183400404 conn(0x7f3cf40fd6e0 msgr2=0x7f3cf40fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:33.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:33.458+0000 7f3cf3e00640 1 -- 172.21.15.43:0/4183400404 shutdown_connections 2024-09-19T05:02:33.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:33.458+0000 7f3cf3e00640 1 -- 172.21.15.43:0/4183400404 wait complete. 2024-09-19T05:02:33.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:33 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/4183400404' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T05:02:33.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:33 smithi096 ceph-mon[35085]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T05:02:33.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:33 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/4183400404' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T05:02:33.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:33 smithi043 ceph-mon[30800]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T05:02:33.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:33 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/4183400404' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T05:02:33.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:33 smithi151 ceph-mon[35425]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T05:02:34.177 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-19T05:02:34.178 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-09-19T05:02:34.557 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:02:34.711 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:34 smithi043 ceph-mon[30800]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-19T05:02:34.711 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:34 smithi043 ceph-mon[30800]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T05:02:34.711 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:34 smithi043 ceph-mon[30800]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T05:02:34.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:34 smithi096 ceph-mon[35085]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-19T05:02:34.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:34 smithi096 ceph-mon[35085]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T05:02:34.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:34 smithi096 ceph-mon[35085]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T05:02:34.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:34 smithi151 ceph-mon[35425]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-19T05:02:34.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:34 smithi151 ceph-mon[35425]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T05:02:34.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:34 smithi151 ceph-mon[35425]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T05:02:35.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.103+0000 7f21fcc00640 1 -- 172.21.15.43:0/263192705 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21f8106f10 msgr2=0x7f21f810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:35.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.103+0000 7f21fcc00640 1 --2- 172.21.15.43:0/263192705 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21f8106f10 0x7f21f810ea90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f21f0009ca0 tx=0x7f21f002a8c0 comp rx=0 tx=0).stop 2024-09-19T05:02:35.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.103+0000 7f21fcc00640 1 -- 172.21.15.43:0/263192705 shutdown_connections 2024-09-19T05:02:35.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.103+0000 7f21fcc00640 1 --2- 172.21.15.43:0/263192705 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21f8106f10 0x7f21f810ea90 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:35.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.103+0000 7f21fcc00640 1 --2- 172.21.15.43:0/263192705 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21f8102790 0x7f21f81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:35.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.103+0000 7f21fcc00640 1 --2- 172.21.15.43:0/263192705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21f8101de0 0x7f21f81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:35.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.103+0000 7f21fcc00640 1 -- 172.21.15.43:0/263192705 >> 172.21.15.43:0/263192705 conn(0x7f21f80fd8c0 msgr2=0x7f21f80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:35.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.103+0000 7f21fcc00640 1 -- 172.21.15.43:0/263192705 shutdown_connections 2024-09-19T05:02:35.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.103+0000 7f21fcc00640 1 -- 172.21.15.43:0/263192705 wait complete. 2024-09-19T05:02:35.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.104+0000 7f21fcc00640 1 Processor -- start 2024-09-19T05:02:35.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.104+0000 7f21fcc00640 1 -- start start 2024-09-19T05:02:35.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.104+0000 7f21fcc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21f8101de0 0x7f21f80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:35.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.104+0000 7f21fcc00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21f8102790 0x7f21f8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:35.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.104+0000 7f21fcc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21f8106f10 0x7f21f8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:35.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.104+0000 7f21fcc00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f21f8111fc0 con 0x7f21f8101de0 2024-09-19T05:02:35.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.104+0000 7f21fcc00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f21f8111e40 con 0x7f21f8102790 2024-09-19T05:02:35.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.104+0000 7f21fcc00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f21f8112140 con 0x7f21f8106f10 2024-09-19T05:02:35.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.105+0000 7f21f5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21f8101de0 0x7f21f80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:35.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.105+0000 7f21f5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21f8101de0 0x7f21f80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40640/0 (socket says 172.21.15.43:40640) 2024-09-19T05:02:35.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.105+0000 7f21f5e00640 1 -- 172.21.15.43:0/3339296026 learned_addr learned my addr 172.21.15.43:0/3339296026 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:02:35.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.105+0000 7f21efe00640 1 --2- 172.21.15.43:0/3339296026 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21f8102790 0x7f21f8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:35.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.105+0000 7f21f6800640 1 --2- 172.21.15.43:0/3339296026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21f8106f10 0x7f21f8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:35.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.105+0000 7f21f5e00640 1 -- 172.21.15.43:0/3339296026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21f8106f10 msgr2=0x7f21f8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:35.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.105+0000 7f21f5e00640 1 --2- 172.21.15.43:0/3339296026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21f8106f10 0x7f21f8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:35.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.105+0000 7f21f5e00640 1 -- 172.21.15.43:0/3339296026 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21f8102790 msgr2=0x7f21f8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:35.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.105+0000 7f21f5e00640 1 --2- 172.21.15.43:0/3339296026 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21f8102790 0x7f21f8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:35.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.105+0000 7f21f5e00640 1 -- 172.21.15.43:0/3339296026 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f81a6890 con 0x7f21f8101de0 2024-09-19T05:02:35.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.105+0000 7f21f6800640 1 --2- 172.21.15.43:0/3339296026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21f8106f10 0x7f21f8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:02:35.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.105+0000 7f21f5e00640 1 --2- 172.21.15.43:0/3339296026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21f8101de0 0x7f21f80744f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f21e800e9f0 tx=0x7f21e800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:35.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.106+0000 7f21eea00640 1 -- 172.21.15.43:0/3339296026 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e800cd80 con 0x7f21f8101de0 2024-09-19T05:02:35.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.106+0000 7f21fcc00640 1 -- 172.21.15.43:0/3339296026 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21f81a6b80 con 0x7f21f8101de0 2024-09-19T05:02:35.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.106+0000 7f21eea00640 1 -- 172.21.15.43:0/3339296026 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21e8005500 con 0x7f21f8101de0 2024-09-19T05:02:35.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.106+0000 7f21eea00640 1 -- 172.21.15.43:0/3339296026 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e8010690 con 0x7f21f8101de0 2024-09-19T05:02:35.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.106+0000 7f21fcc00640 1 -- 172.21.15.43:0/3339296026 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21f81a70c0 con 0x7f21f8101de0 2024-09-19T05:02:35.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.107+0000 7f21fcc00640 1 -- 172.21.15.43:0/3339296026 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21f81021c0 con 0x7f21f8101de0 2024-09-19T05:02:35.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.109+0000 7f21eea00640 1 -- 172.21.15.43:0/3339296026 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f21e80046c0 con 0x7f21f8101de0 2024-09-19T05:02:35.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.110+0000 7f21eea00640 1 --2- 172.21.15.43:0/3339296026 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f21dc0a32c0 0x7f21dc0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:35.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.110+0000 7f21eea00640 1 -- 172.21.15.43:0/3339296026 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f21e80cfc90 con 0x7f21f8101de0 2024-09-19T05:02:35.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.110+0000 7f21efe00640 1 --2- 172.21.15.43:0/3339296026 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f21dc0a32c0 0x7f21dc0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:35.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.111+0000 7f21efe00640 1 --2- 172.21.15.43:0/3339296026 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f21dc0a32c0 0x7f21dc0a5780 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f21f8075f80 tx=0x7f21e000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:35.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.114+0000 7f21eea00640 1 -- 172.21.15.43:0/3339296026 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f21e809ca60 con 0x7f21f8101de0 2024-09-19T05:02:35.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.344+0000 7f21fcc00640 1 -- 172.21.15.43:0/3339296026 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f21f8109bc0 con 0x7f21f8101de0 2024-09-19T05:02:35.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.462+0000 7f21eea00640 1 -- 172.21.15.43:0/3339296026 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f21e809c360 con 0x7f21f8101de0 2024-09-19T05:02:35.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.465+0000 7f21fcc00640 1 -- 172.21.15.43:0/3339296026 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f21dc0a32c0 msgr2=0x7f21dc0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:35.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.465+0000 7f21fcc00640 1 --2- 172.21.15.43:0/3339296026 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f21dc0a32c0 0x7f21dc0a5780 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f21f8075f80 tx=0x7f21e000b390 comp rx=0 tx=0).stop 2024-09-19T05:02:35.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.465+0000 7f21fcc00640 1 -- 172.21.15.43:0/3339296026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21f8101de0 msgr2=0x7f21f80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:35.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.465+0000 7f21fcc00640 1 --2- 172.21.15.43:0/3339296026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21f8101de0 0x7f21f80744f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f21e800e9f0 tx=0x7f21e800edf0 comp rx=0 tx=0).stop 2024-09-19T05:02:35.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.465+0000 7f21fcc00640 1 -- 172.21.15.43:0/3339296026 shutdown_connections 2024-09-19T05:02:35.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.465+0000 7f21fcc00640 1 --2- 172.21.15.43:0/3339296026 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f21dc0a32c0 0x7f21dc0a5780 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:35.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.465+0000 7f21fcc00640 1 --2- 172.21.15.43:0/3339296026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21f8106f10 0x7f21f8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:35.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.465+0000 7f21fcc00640 1 --2- 172.21.15.43:0/3339296026 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21f8102790 0x7f21f8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:35.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.465+0000 7f21fcc00640 1 --2- 172.21.15.43:0/3339296026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21f8101de0 0x7f21f80744f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:35.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.465+0000 7f21fcc00640 1 -- 172.21.15.43:0/3339296026 >> 172.21.15.43:0/3339296026 conn(0x7f21f80fd8c0 msgr2=0x7f21f80fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:35.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.465+0000 7f21fcc00640 1 -- 172.21.15.43:0/3339296026 shutdown_connections 2024-09-19T05:02:35.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:35.465+0000 7f21fcc00640 1 -- 172.21.15.43:0/3339296026 wait complete. 2024-09-19T05:02:35.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:35 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/3339296026' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-19T05:02:35.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:35 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/3339296026' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-19T05:02:35.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:35 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/3339296026' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-19T05:02:36.245 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-19T05:02:36.629 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:02:36.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:36 smithi096 ceph-mon[35085]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-09-19T05:02:36.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:36 smithi096 ceph-mon[35085]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T05:02:36.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:36 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/3339296026' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-19T05:02:36.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:36 smithi096 ceph-mon[35085]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T05:02:36.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:36 smithi096 ceph-mon[35085]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T05:02:36.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:36 smithi096 ceph-mon[35085]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-09-19T05:02:36.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:36 smithi096 ceph-mon[35085]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-09-19T05:02:36.768 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:36 smithi043 ceph-mon[30800]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-09-19T05:02:36.769 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:36 smithi043 ceph-mon[30800]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T05:02:36.769 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:36 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/3339296026' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-19T05:02:36.769 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:36 smithi043 ceph-mon[30800]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T05:02:36.769 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:36 smithi043 ceph-mon[30800]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T05:02:36.769 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:36 smithi043 ceph-mon[30800]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-09-19T05:02:36.769 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:36 smithi043 ceph-mon[30800]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-09-19T05:02:36.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:36 smithi151 ceph-mon[35425]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-09-19T05:02:36.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:36 smithi151 ceph-mon[35425]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T05:02:36.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:36 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/3339296026' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-19T05:02:36.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:36 smithi151 ceph-mon[35425]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T05:02:36.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:36 smithi151 ceph-mon[35425]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T05:02:36.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:36 smithi151 ceph-mon[35425]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-09-19T05:02:36.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:36 smithi151 ceph-mon[35425]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-09-19T05:02:37.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.204+0000 7f7b03e00640 1 -- 172.21.15.43:0/3755405032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b04102360 msgr2=0x7f7b0410e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:37.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.204+0000 7f7b03e00640 1 --2- 172.21.15.43:0/3755405032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b04102360 0x7f7b0410e890 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7afc009cd0 tx=0x7f7afc02a8e0 comp rx=0 tx=0).stop 2024-09-19T05:02:37.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.204+0000 7f7b03e00640 1 -- 172.21.15.43:0/3755405032 shutdown_connections 2024-09-19T05:02:37.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.204+0000 7f7b03e00640 1 --2- 172.21.15.43:0/3755405032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b04102360 0x7f7b0410e890 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:37.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.204+0000 7f7b03e00640 1 --2- 172.21.15.43:0/3755405032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b041019c0 0x7f7b04101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:37.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.204+0000 7f7b03e00640 1 --2- 172.21.15.43:0/3755405032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b041079c0 0x7f7b04107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:37.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.204+0000 7f7b03e00640 1 -- 172.21.15.43:0/3755405032 >> 172.21.15.43:0/3755405032 conn(0x7f7b040fd740 msgr2=0x7f7b040ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:37.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.204+0000 7f7b03e00640 1 -- 172.21.15.43:0/3755405032 shutdown_connections 2024-09-19T05:02:37.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.204+0000 7f7b03e00640 1 -- 172.21.15.43:0/3755405032 wait complete. 2024-09-19T05:02:37.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.205+0000 7f7b03e00640 1 Processor -- start 2024-09-19T05:02:37.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.205+0000 7f7b03e00640 1 -- start start 2024-09-19T05:02:37.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.205+0000 7f7b03e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b041019c0 0x7f7b041982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:37.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.206+0000 7f7b03e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b04102360 0x7f7b041987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:37.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.206+0000 7f7b03e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b041079c0 0x7f7b0419cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:37.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.206+0000 7f7b03e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7b04111dc0 con 0x7f7b041019c0 2024-09-19T05:02:37.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.206+0000 7f7b03e00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7b04111c40 con 0x7f7b04102360 2024-09-19T05:02:37.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.206+0000 7f7b03e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7b04111f40 con 0x7f7b041079c0 2024-09-19T05:02:37.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.206+0000 7f7b02a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b041019c0 0x7f7b041982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:37.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.206+0000 7f7b02a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b041019c0 0x7f7b041982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59576/0 (socket says 172.21.15.43:59576) 2024-09-19T05:02:37.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.206+0000 7f7b02a00640 1 -- 172.21.15.43:0/2335439822 learned_addr learned my addr 172.21.15.43:0/2335439822 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:02:37.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.206+0000 7f7b02000640 1 --2- 172.21.15.43:0/2335439822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b04102360 0x7f7b041987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:37.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.206+0000 7f7b03400640 1 --2- 172.21.15.43:0/2335439822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b041079c0 0x7f7b0419cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:37.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.206+0000 7f7b02000640 1 -- 172.21.15.43:0/2335439822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b041079c0 msgr2=0x7f7b0419cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:37.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.206+0000 7f7b02000640 1 --2- 172.21.15.43:0/2335439822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b041079c0 0x7f7b0419cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:37.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.206+0000 7f7b02000640 1 -- 172.21.15.43:0/2335439822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b041019c0 msgr2=0x7f7b041982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:37.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.206+0000 7f7b02000640 1 --2- 172.21.15.43:0/2335439822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b041019c0 0x7f7b041982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:37.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.206+0000 7f7b02000640 1 -- 172.21.15.43:0/2335439822 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b0419d250 con 0x7f7b04102360 2024-09-19T05:02:37.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.206+0000 7f7b02a00640 1 --2- 172.21.15.43:0/2335439822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b041019c0 0x7f7b041982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:02:37.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.207+0000 7f7b02000640 1 --2- 172.21.15.43:0/2335439822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b04102360 0x7f7b041987e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7af000e9f0 tx=0x7f7af000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:37.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.207+0000 7f7afb400640 1 -- 172.21.15.43:0/2335439822 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af000ccc0 con 0x7f7b04102360 2024-09-19T05:02:37.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.207+0000 7f7b03e00640 1 -- 172.21.15.43:0/2335439822 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b0419d540 con 0x7f7b04102360 2024-09-19T05:02:37.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.207+0000 7f7afb400640 1 -- 172.21.15.43:0/2335439822 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7af0005020 con 0x7f7b04102360 2024-09-19T05:02:37.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.207+0000 7f7afb400640 1 -- 172.21.15.43:0/2335439822 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af00049e0 con 0x7f7b04102360 2024-09-19T05:02:37.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.207+0000 7f7b03e00640 1 -- 172.21.15.43:0/2335439822 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b041a4e20 con 0x7f7b04102360 2024-09-19T05:02:37.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.208+0000 7f7b03e00640 1 -- 172.21.15.43:0/2335439822 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b04101e20 con 0x7f7b04102360 2024-09-19T05:02:37.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.211+0000 7f7afb400640 1 -- 172.21.15.43:0/2335439822 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f7af0005560 con 0x7f7b04102360 2024-09-19T05:02:37.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.212+0000 7f7afb400640 1 --2- 172.21.15.43:0/2335439822 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f7ae80a3240 0x7f7ae80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:37.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.212+0000 7f7afb400640 1 -- 172.21.15.43:0/2335439822 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f7af00cf880 con 0x7f7b04102360 2024-09-19T05:02:37.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.212+0000 7f7b02a00640 1 --2- 172.21.15.43:0/2335439822 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f7ae80a3240 0x7f7ae80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:37.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.213+0000 7f7b02a00640 1 --2- 172.21.15.43:0/2335439822 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f7ae80a3240 0x7f7ae80a5700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7af4011440 tx=0x7f7af4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:37.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.214+0000 7f7afb400640 1 -- 172.21.15.43:0/2335439822 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7af00051d0 con 0x7f7b04102360 2024-09-19T05:02:37.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.438+0000 7f7b03e00640 1 -- 172.21.15.43:0/2335439822 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f7b0402d7c0 con 0x7f7b04102360 2024-09-19T05:02:37.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.480+0000 7f7afb400640 1 -- 172.21.15.43:0/2335439822 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f7af009c6e0 con 0x7f7b04102360 2024-09-19T05:02:37.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.483+0000 7f7b03e00640 1 -- 172.21.15.43:0/2335439822 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f7ae80a3240 msgr2=0x7f7ae80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:37.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.483+0000 7f7b03e00640 1 --2- 172.21.15.43:0/2335439822 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f7ae80a3240 0x7f7ae80a5700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7af4011440 tx=0x7f7af4005f90 comp rx=0 tx=0).stop 2024-09-19T05:02:37.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.483+0000 7f7b03e00640 1 -- 172.21.15.43:0/2335439822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b04102360 msgr2=0x7f7b041987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:37.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.483+0000 7f7b03e00640 1 --2- 172.21.15.43:0/2335439822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b04102360 0x7f7b041987e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7af000e9f0 tx=0x7f7af000eeb0 comp rx=0 tx=0).stop 2024-09-19T05:02:37.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.483+0000 7f7b03e00640 1 -- 172.21.15.43:0/2335439822 shutdown_connections 2024-09-19T05:02:37.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.483+0000 7f7b03e00640 1 --2- 172.21.15.43:0/2335439822 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f7ae80a3240 0x7f7ae80a5700 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:37.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.483+0000 7f7b03e00640 1 --2- 172.21.15.43:0/2335439822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b041079c0 0x7f7b0419cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:37.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.483+0000 7f7b03e00640 1 --2- 172.21.15.43:0/2335439822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b04102360 0x7f7b041987e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:37.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.483+0000 7f7b03e00640 1 --2- 172.21.15.43:0/2335439822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b041019c0 0x7f7b041982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:37.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.483+0000 7f7b03e00640 1 -- 172.21.15.43:0/2335439822 >> 172.21.15.43:0/2335439822 conn(0x7f7b040fd740 msgr2=0x7f7b040fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:37.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.483+0000 7f7b03e00640 1 -- 172.21.15.43:0/2335439822 shutdown_connections 2024-09-19T05:02:37.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:37.483+0000 7f7b03e00640 1 -- 172.21.15.43:0/2335439822 wait complete. 2024-09-19T05:02:37.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:37 smithi096 ceph-mon[35085]: mds.? [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] up:active 2024-09-19T05:02:37.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:37 smithi096 ceph-mon[35085]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T05:02:37.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:37 smithi096 ceph-mon[35085]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-09-19T05:02:37.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:37 smithi096 ceph-mon[35085]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:creating} 9 up:standby 2024-09-19T05:02:37.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:37 smithi096 ceph-mon[35085]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-19T05:02:37.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:37 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/2335439822' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T05:02:37.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:37 smithi096 ceph-mon[35085]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T05:02:37.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:37 smithi043 ceph-mon[30800]: mds.? [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] up:active 2024-09-19T05:02:37.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:37 smithi043 ceph-mon[30800]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T05:02:37.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:37 smithi043 ceph-mon[30800]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-09-19T05:02:37.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:37 smithi043 ceph-mon[30800]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:creating} 9 up:standby 2024-09-19T05:02:37.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:37 smithi043 ceph-mon[30800]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-19T05:02:37.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:37 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/2335439822' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T05:02:37.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:37 smithi043 ceph-mon[30800]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T05:02:37.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:37 smithi151 ceph-mon[35425]: mds.? [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] up:active 2024-09-19T05:02:37.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:37 smithi151 ceph-mon[35425]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T05:02:37.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:37 smithi151 ceph-mon[35425]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-09-19T05:02:37.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:37 smithi151 ceph-mon[35425]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:creating} 9 up:standby 2024-09-19T05:02:37.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:37 smithi151 ceph-mon[35425]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-19T05:02:37.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:37 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/2335439822' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T05:02:37.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:37 smithi151 ceph-mon[35425]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T05:02:38.334 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-19T05:02:38.719 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:02:38.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:38 smithi043 ceph-mon[30800]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-09-19T05:02:38.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:38 smithi043 ceph-mon[30800]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T05:02:38.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:38 smithi043 ceph-mon[30800]: mds.? [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] up:active 2024-09-19T05:02:38.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:38 smithi043 ceph-mon[30800]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-19T05:02:38.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:38 smithi043 ceph-mon[30800]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 9 up:standby 2024-09-19T05:02:38.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:38 smithi043 ceph-mon[30800]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=i=up:active,3=h=up:creating} 8 up:standby 2024-09-19T05:02:38.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:38 smithi043 ceph-mon[30800]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-09-19T05:02:38.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:38 smithi151 ceph-mon[35425]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-09-19T05:02:38.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:38 smithi151 ceph-mon[35425]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T05:02:38.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:38 smithi151 ceph-mon[35425]: mds.? [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] up:active 2024-09-19T05:02:38.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:38 smithi151 ceph-mon[35425]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-19T05:02:38.906 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:38 smithi151 ceph-mon[35425]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 9 up:standby 2024-09-19T05:02:38.906 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:38 smithi151 ceph-mon[35425]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=i=up:active,3=h=up:creating} 8 up:standby 2024-09-19T05:02:38.906 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:38 smithi151 ceph-mon[35425]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-09-19T05:02:38.973 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 05:02:38 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-b[65164]: 2024-09-19T05:02:38.544+0000 7f6a0a343640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-19T05:02:38.973 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 05:02:38 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-b[65164]: 2024-09-19T05:02:38.544+0000 7f6a0a343640 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-09-19T05:02:38.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:38 smithi096 ceph-mon[35085]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-09-19T05:02:38.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:38 smithi096 ceph-mon[35085]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T05:02:38.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:38 smithi096 ceph-mon[35085]: mds.? [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] up:active 2024-09-19T05:02:38.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:38 smithi096 ceph-mon[35085]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-19T05:02:38.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:38 smithi096 ceph-mon[35085]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 9 up:standby 2024-09-19T05:02:38.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:38 smithi096 ceph-mon[35085]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=i=up:active,3=h=up:creating} 8 up:standby 2024-09-19T05:02:38.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:38 smithi096 ceph-mon[35085]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-09-19T05:02:39.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:39.287+0000 7fa84ac00640 1 -- 172.21.15.43:0/484790953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa844103e30 msgr2=0x7fa844104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:39.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:39.287+0000 7fa84ac00640 1 --2- 172.21.15.43:0/484790953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa844103e30 0x7fa844104210 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa838009900 tx=0x7fa83802a3f0 comp rx=0 tx=0).stop 2024-09-19T05:02:39.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:39.287+0000 7fa84ac00640 1 -- 172.21.15.43:0/484790953 shutdown_connections 2024-09-19T05:02:39.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:39.287+0000 7fa84ac00640 1 --2- 172.21.15.43:0/484790953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa844108f60 0x7fa844110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:39.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:39.287+0000 7fa84ac00640 1 --2- 172.21.15.43:0/484790953 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8441047e0 0x7fa844108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:39.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:39.287+0000 7fa84ac00640 1 --2- 172.21.15.43:0/484790953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa844103e30 0x7fa844104210 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:39.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:39.287+0000 7fa84ac00640 1 -- 172.21.15.43:0/484790953 >> 172.21.15.43:0/484790953 conn(0x7fa8440ff910 msgr2=0x7fa844101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:39.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:39.287+0000 7fa84ac00640 1 -- 172.21.15.43:0/484790953 shutdown_connections 2024-09-19T05:02:39.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:39.287+0000 7fa84ac00640 1 -- 172.21.15.43:0/484790953 wait complete. 2024-09-19T05:02:39.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:39.288+0000 7fa84ac00640 1 Processor -- start 2024-09-19T05:02:39.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:39.288+0000 7fa84ac00640 1 -- start start 2024-09-19T05:02:39.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:39.289+0000 7fa84ac00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa844103e30 0x7fa84419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:39.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:39.289+0000 7fa84ac00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8441047e0 0x7fa84419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:39.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:39.289+0000 7fa84ac00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa844108f60 0x7fa84419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:39.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:39.289+0000 7fa84ac00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa844114010 con 0x7fa844103e30 2024-09-19T05:02:39.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:39.289+0000 7fa84ac00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa844113e90 con 0x7fa844108f60 2024-09-19T05:02:39.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:39.289+0000 7fa84ac00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa844114190 con 0x7fa8441047e0 2024-09-19T05:02:39.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:39.289+0000 7fa849800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa844103e30 0x7fa84419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:39.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:39.289+0000 7fa849800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa844103e30 0x7fa84419a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59606/0 (socket says 172.21.15.43:59606) 2024-09-19T05:02:39.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:39.289+0000 7fa849800640 1 -- 172.21.15.43:0/3709974647 learned_addr learned my addr 172.21.15.43:0/3709974647 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:02:39.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:39.289+0000 7fa84a200640 1 --2- 172.21.15.43:0/3709974647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa844108f60 0x7fa84419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:39.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:39.289+0000 7fa848e00640 1 --2- 172.21.15.43:0/3709974647 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8441047e0 0x7fa84419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:39.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:39.290+0000 7fa849800640 1 -- 172.21.15.43:0/3709974647 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8441047e0 msgr2=0x7fa84419a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:39.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:39.290+0000 7fa849800640 1 --2- 172.21.15.43:0/3709974647 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8441047e0 0x7fa84419a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:39.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:39.290+0000 7fa849800640 1 -- 172.21.15.43:0/3709974647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa844108f60 msgr2=0x7fa84419ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:39.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:39.290+0000 7fa849800640 1 --2- 172.21.15.43:0/3709974647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa844108f60 0x7fa84419ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:39.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:39.290+0000 7fa849800640 1 -- 172.21.15.43:0/3709974647 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa84419f420 con 0x7fa844103e30 2024-09-19T05:02:39.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:39.290+0000 7fa849800640 1 --2- 172.21.15.43:0/3709974647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa844103e30 0x7fa84419a470 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa838004770 tx=0x7fa838003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:39.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:39.290+0000 7fa842000640 1 -- 172.21.15.43:0/3709974647 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83803f070 con 0x7fa844103e30 2024-09-19T05:02:39.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:39.290+0000 7fa84ac00640 1 -- 172.21.15.43:0/3709974647 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa84419f6b0 con 0x7fa844103e30 2024-09-19T05:02:39.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:39.290+0000 7fa842000640 1 -- 172.21.15.43:0/3709974647 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa838031ca0 con 0x7fa844103e30 2024-09-19T05:02:39.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:39.290+0000 7fa842000640 1 -- 172.21.15.43:0/3709974647 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa838035030 con 0x7fa844103e30 2024-09-19T05:02:39.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:39.290+0000 7fa84ac00640 1 -- 172.21.15.43:0/3709974647 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8441a6ff0 con 0x7fa844103e30 2024-09-19T05:02:39.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:39.292+0000 7fa84ac00640 1 -- 172.21.15.43:0/3709974647 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa844105310 con 0x7fa844103e30 2024-09-19T05:02:39.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:39.293+0000 7fa842000640 1 -- 172.21.15.43:0/3709974647 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fa838042070 con 0x7fa844103e30 2024-09-19T05:02:39.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:39.295+0000 7fa842000640 1 --2- 172.21.15.43:0/3709974647 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa82c0a3240 0x7fa82c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:39.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:39.295+0000 7fa842000640 1 -- 172.21.15.43:0/3709974647 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7fa8380eff40 con 0x7fa844103e30 2024-09-19T05:02:39.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:39.295+0000 7fa848e00640 1 --2- 172.21.15.43:0/3709974647 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa82c0a3240 0x7fa82c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:39.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:39.296+0000 7fa848e00640 1 --2- 172.21.15.43:0/3709974647 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa82c0a3240 0x7fa82c0a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa8440ffd30 tx=0x7fa83000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:39.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:39.299+0000 7fa842000640 1 -- 172.21.15.43:0/3709974647 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa8380bcd10 con 0x7fa844103e30 2024-09-19T05:02:39.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:39.528+0000 7fa84ac00640 1 -- 172.21.15.43:0/3709974647 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fa84410c140 con 0x7fa844103e30 2024-09-19T05:02:39.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:39 smithi043 ceph-mon[30800]: mds.? [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] up:active 2024-09-19T05:02:39.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:39 smithi043 ceph-mon[30800]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-19T05:02:39.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:39 smithi043 ceph-mon[30800]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T05:02:39.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:39 smithi043 ceph-mon[30800]: Cluster is now healthy 2024-09-19T05:02:39.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:39 smithi043 ceph-mon[30800]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=i=up:active,3=h=up:active} 8 up:standby 2024-09-19T05:02:39.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:39 smithi043 ceph-mon[30800]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-19T05:02:39.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:39 smithi043 ceph-mon[30800]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-09-19T05:02:39.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:39 smithi151 ceph-mon[35425]: mds.? [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] up:active 2024-09-19T05:02:39.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:39 smithi151 ceph-mon[35425]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-19T05:02:39.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:39 smithi151 ceph-mon[35425]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T05:02:39.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:39 smithi151 ceph-mon[35425]: Cluster is now healthy 2024-09-19T05:02:39.906 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:39 smithi151 ceph-mon[35425]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=i=up:active,3=h=up:active} 8 up:standby 2024-09-19T05:02:39.906 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:39 smithi151 ceph-mon[35425]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-19T05:02:39.906 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:39 smithi151 ceph-mon[35425]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-09-19T05:02:39.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:39 smithi096 ceph-mon[35085]: mds.? [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] up:active 2024-09-19T05:02:39.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:39 smithi096 ceph-mon[35085]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-19T05:02:39.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:39 smithi096 ceph-mon[35085]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T05:02:39.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:39 smithi096 ceph-mon[35085]: Cluster is now healthy 2024-09-19T05:02:39.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:39 smithi096 ceph-mon[35085]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=i=up:active,3=h=up:active} 8 up:standby 2024-09-19T05:02:39.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:39 smithi096 ceph-mon[35085]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-19T05:02:39.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:39 smithi096 ceph-mon[35085]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-09-19T05:02:40.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:40.505+0000 7fa842000640 1 -- 172.21.15.43:0/3709974647 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v28) ==== 115+0+0 (secure 0 0 0) 0x7fa8380bc610 con 0x7fa844103e30 2024-09-19T05:02:40.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:40.509+0000 7fa84ac00640 1 -- 172.21.15.43:0/3709974647 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa82c0a3240 msgr2=0x7fa82c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:40.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:40.509+0000 7fa84ac00640 1 --2- 172.21.15.43:0/3709974647 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa82c0a3240 0x7fa82c0a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa8440ffd30 tx=0x7fa83000b410 comp rx=0 tx=0).stop 2024-09-19T05:02:40.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:40.509+0000 7fa84ac00640 1 -- 172.21.15.43:0/3709974647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa844103e30 msgr2=0x7fa84419a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:40.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:40.509+0000 7fa84ac00640 1 --2- 172.21.15.43:0/3709974647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa844103e30 0x7fa84419a470 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa838004770 tx=0x7fa838003c10 comp rx=0 tx=0).stop 2024-09-19T05:02:40.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:40.509+0000 7fa84ac00640 1 -- 172.21.15.43:0/3709974647 shutdown_connections 2024-09-19T05:02:40.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:40.509+0000 7fa84ac00640 1 --2- 172.21.15.43:0/3709974647 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa82c0a3240 0x7fa82c0a5700 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:40.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:40.509+0000 7fa84ac00640 1 --2- 172.21.15.43:0/3709974647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa844108f60 0x7fa84419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:40.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:40.509+0000 7fa84ac00640 1 --2- 172.21.15.43:0/3709974647 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8441047e0 0x7fa84419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:40.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:40.509+0000 7fa84ac00640 1 --2- 172.21.15.43:0/3709974647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa844103e30 0x7fa84419a470 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:40.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:40.509+0000 7fa84ac00640 1 -- 172.21.15.43:0/3709974647 >> 172.21.15.43:0/3709974647 conn(0x7fa8440ff910 msgr2=0x7fa844100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:40.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:40.509+0000 7fa84ac00640 1 -- 172.21.15.43:0/3709974647 shutdown_connections 2024-09-19T05:02:40.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:40.509+0000 7fa84ac00640 1 -- 172.21.15.43:0/3709974647 wait complete. 2024-09-19T05:02:40.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:40 smithi043 ceph-mon[30800]: pgmap v299: 97 pgs: 97 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 7 op/s 2024-09-19T05:02:40.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:40 smithi043 ceph-mon[30800]: mds.? [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] up:active 2024-09-19T05:02:40.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:40 smithi043 ceph-mon[30800]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-19T05:02:40.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:40 smithi043 ceph-mon[30800]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T05:02:40.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:40 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/3709974647' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T05:02:40.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:40 smithi151 ceph-mon[35425]: pgmap v299: 97 pgs: 97 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 7 op/s 2024-09-19T05:02:40.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:40 smithi151 ceph-mon[35425]: mds.? [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] up:active 2024-09-19T05:02:40.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:40 smithi151 ceph-mon[35425]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-19T05:02:40.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:40 smithi151 ceph-mon[35425]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T05:02:40.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:40 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/3709974647' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T05:02:40.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:40 smithi096 ceph-mon[35085]: pgmap v299: 97 pgs: 97 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 7 op/s 2024-09-19T05:02:40.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:40 smithi096 ceph-mon[35085]: mds.? [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] up:active 2024-09-19T05:02:40.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:40 smithi096 ceph-mon[35085]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-19T05:02:40.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:40 smithi096 ceph-mon[35085]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T05:02:40.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:40 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/3709974647' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T05:02:41.389 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-09-19T05:02:41.389 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-19T05:02:41.776 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:02:41.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:41 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/3709974647' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T05:02:41.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:41 smithi043 ceph-mon[30800]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T05:02:41.904 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:41 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/3709974647' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T05:02:41.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:41 smithi151 ceph-mon[35425]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T05:02:41.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:41 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/3709974647' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T05:02:41.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:41 smithi096 ceph-mon[35085]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T05:02:42.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.389+0000 7f20cea00640 1 -- 172.21.15.43:0/101575930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20c8104010 msgr2=0x7f20c81043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:42.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.389+0000 7f20cea00640 1 --2- 172.21.15.43:0/101575930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20c8104010 0x7f20c81043f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f20b8009900 tx=0x7f20b802a3f0 comp rx=0 tx=0).stop 2024-09-19T05:02:42.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.389+0000 7f20cea00640 1 -- 172.21.15.43:0/101575930 shutdown_connections 2024-09-19T05:02:42.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.389+0000 7f20cea00640 1 --2- 172.21.15.43:0/101575930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20c8109140 0x7f20c8110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:42.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.389+0000 7f20cea00640 1 --2- 172.21.15.43:0/101575930 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f20c81049c0 0x7f20c8108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:42.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.389+0000 7f20cea00640 1 --2- 172.21.15.43:0/101575930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20c8104010 0x7f20c81043f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:42.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.389+0000 7f20cea00640 1 -- 172.21.15.43:0/101575930 >> 172.21.15.43:0/101575930 conn(0x7f20c80ffa50 msgr2=0x7f20c8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:42.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.389+0000 7f20cea00640 1 -- 172.21.15.43:0/101575930 shutdown_connections 2024-09-19T05:02:42.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.389+0000 7f20cea00640 1 -- 172.21.15.43:0/101575930 wait complete. 2024-09-19T05:02:42.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.390+0000 7f20cea00640 1 Processor -- start 2024-09-19T05:02:42.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.391+0000 7f20cea00640 1 -- start start 2024-09-19T05:02:42.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.391+0000 7f20cea00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20c8104010 0x7f20c819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:42.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.391+0000 7f20cea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f20c81049c0 0x7f20c819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:42.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.391+0000 7f20cea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20c8109140 0x7f20c819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:42.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.391+0000 7f20cea00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f20c8114140 con 0x7f20c8109140 2024-09-19T05:02:42.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.391+0000 7f20cea00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f20c8113fc0 con 0x7f20c8104010 2024-09-19T05:02:42.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.391+0000 7f20cea00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f20c81142c0 con 0x7f20c81049c0 2024-09-19T05:02:42.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.391+0000 7f20c7e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20c8104010 0x7f20c819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:42.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.391+0000 7f20c7e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20c8104010 0x7f20c819a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:43216/0 (socket says 172.21.15.43:43216) 2024-09-19T05:02:42.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.391+0000 7f20c7e00640 1 -- 172.21.15.43:0/1201097959 learned_addr learned my addr 172.21.15.43:0/1201097959 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:02:42.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.391+0000 7f20cca00640 1 --2- 172.21.15.43:0/1201097959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20c8109140 0x7f20c819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:42.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.392+0000 7f20c7400640 1 --2- 172.21.15.43:0/1201097959 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f20c81049c0 0x7f20c819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:42.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.392+0000 7f20cca00640 1 -- 172.21.15.43:0/1201097959 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f20c81049c0 msgr2=0x7f20c819ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:42.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.392+0000 7f20cca00640 1 --2- 172.21.15.43:0/1201097959 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f20c81049c0 0x7f20c819ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:42.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.392+0000 7f20cca00640 1 -- 172.21.15.43:0/1201097959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20c8104010 msgr2=0x7f20c819a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:42.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.392+0000 7f20cca00640 1 --2- 172.21.15.43:0/1201097959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20c8104010 0x7f20c819a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:42.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.392+0000 7f20cca00640 1 -- 172.21.15.43:0/1201097959 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20c819f5d0 con 0x7f20c8109140 2024-09-19T05:02:42.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.392+0000 7f20c7e00640 1 --2- 172.21.15.43:0/1201097959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20c8104010 0x7f20c819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:02:42.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.392+0000 7f20cca00640 1 --2- 172.21.15.43:0/1201097959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20c8109140 0x7f20c819eef0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f20c000e940 tx=0x7f20c000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:42.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.392+0000 7f20c4c00640 1 -- 172.21.15.43:0/1201097959 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c0004d20 con 0x7f20c8109140 2024-09-19T05:02:42.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.393+0000 7f20cea00640 1 -- 172.21.15.43:0/1201097959 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20c819f8c0 con 0x7f20c8109140 2024-09-19T05:02:42.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.393+0000 7f20c4c00640 1 -- 172.21.15.43:0/1201097959 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20c0005020 con 0x7f20c8109140 2024-09-19T05:02:42.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.393+0000 7f20c4c00640 1 -- 172.21.15.43:0/1201097959 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c00116b0 con 0x7f20c8109140 2024-09-19T05:02:42.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.393+0000 7f20cea00640 1 -- 172.21.15.43:0/1201097959 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20c81a71a0 con 0x7f20c8109140 2024-09-19T05:02:42.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.394+0000 7f20cea00640 1 -- 172.21.15.43:0/1201097959 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20c81043f0 con 0x7f20c8109140 2024-09-19T05:02:42.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.396+0000 7f20c4c00640 1 -- 172.21.15.43:0/1201097959 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f20c00054d0 con 0x7f20c8109140 2024-09-19T05:02:42.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.397+0000 7f20c4c00640 1 --2- 172.21.15.43:0/1201097959 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f20ac0a3240 0x7f20ac0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:42.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.397+0000 7f20c4c00640 1 -- 172.21.15.43:0/1201097959 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f20c00cf870 con 0x7f20c8109140 2024-09-19T05:02:42.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.397+0000 7f20c7e00640 1 --2- 172.21.15.43:0/1201097959 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f20ac0a3240 0x7f20ac0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:42.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.398+0000 7f20c7e00640 1 --2- 172.21.15.43:0/1201097959 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f20ac0a3240 0x7f20ac0a5700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f20b80096c0 tx=0x7f20b8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:42.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.401+0000 7f20c4c00640 1 -- 172.21.15.43:0/1201097959 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f20c0016030 con 0x7f20c8109140 2024-09-19T05:02:42.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:42 smithi043 ceph-mon[30800]: pgmap v300: 97 pgs: 97 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 2.2 KiB/s wr, 8 op/s 2024-09-19T05:02:42.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.615+0000 7f20cea00640 1 -- 172.21.15.43:0/1201097959 --> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f20c8062630 con 0x7f20ac0a3240 2024-09-19T05:02:42.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.652+0000 7f20c4c00640 1 -- 172.21.15.43:0/1201097959 <== mgr.14150 v2:172.21.15.43:6800/3167108925 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f20c8062630 con 0x7f20ac0a3240 2024-09-19T05:02:42.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.654+0000 7f20cea00640 1 -- 172.21.15.43:0/1201097959 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f20ac0a3240 msgr2=0x7f20ac0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:42.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.654+0000 7f20cea00640 1 --2- 172.21.15.43:0/1201097959 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f20ac0a3240 0x7f20ac0a5700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f20b80096c0 tx=0x7f20b8005a20 comp rx=0 tx=0).stop 2024-09-19T05:02:42.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.654+0000 7f20cea00640 1 -- 172.21.15.43:0/1201097959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20c8109140 msgr2=0x7f20c819eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:42.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.654+0000 7f20cea00640 1 --2- 172.21.15.43:0/1201097959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20c8109140 0x7f20c819eef0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f20c000e940 tx=0x7f20c000ee00 comp rx=0 tx=0).stop 2024-09-19T05:02:42.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.654+0000 7f20cea00640 1 -- 172.21.15.43:0/1201097959 shutdown_connections 2024-09-19T05:02:42.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.654+0000 7f20cea00640 1 --2- 172.21.15.43:0/1201097959 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f20ac0a3240 0x7f20ac0a5700 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:42.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.654+0000 7f20cea00640 1 --2- 172.21.15.43:0/1201097959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20c8109140 0x7f20c819eef0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:42.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.654+0000 7f20cea00640 1 --2- 172.21.15.43:0/1201097959 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f20c81049c0 0x7f20c819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:42.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.654+0000 7f20cea00640 1 --2- 172.21.15.43:0/1201097959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20c8104010 0x7f20c819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:42.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.654+0000 7f20cea00640 1 -- 172.21.15.43:0/1201097959 >> 172.21.15.43:0/1201097959 conn(0x7f20c80ffa50 msgr2=0x7f20c8100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:42.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.655+0000 7f20cea00640 1 -- 172.21.15.43:0/1201097959 shutdown_connections 2024-09-19T05:02:42.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:42.655+0000 7f20cea00640 1 -- 172.21.15.43:0/1201097959 wait complete. 2024-09-19T05:02:42.904 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:42 smithi151 ceph-mon[35425]: pgmap v300: 97 pgs: 97 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 2.2 KiB/s wr, 8 op/s 2024-09-19T05:02:42.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:42 smithi096 ceph-mon[35085]: pgmap v300: 97 pgs: 97 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 2.2 KiB/s wr, 8 op/s 2024-09-19T05:02:43.030 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:02:42 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:02:42.622+0000 7faf18600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:02:43.031 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:02:42 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:02:42.622+0000 7faf18600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:02:43.031 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:02:42 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:02:42.622+0000 7faf18600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:02:43.031 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:02:42 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:02:42.622+0000 7faf18600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:02:43.031 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:02:42 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:02:42.622+0000 7faf18600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:02:43.512 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-19T05:02:43.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:43 smithi043 ceph-mon[30800]: from='client.14622 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:02:43.897 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:02:43.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:43 smithi151 ceph-mon[35425]: from='client.14622 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:02:43.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:43 smithi096 ceph-mon[35085]: from='client.14622 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:02:44.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.436+0000 7f7b52800640 1 -- 172.21.15.43:0/2392932660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b4c077040 msgr2=0x7f7b4c1124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:44.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.436+0000 7f7b52800640 1 --2- 172.21.15.43:0/2392932660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b4c077040 0x7f7b4c1124f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7b34009900 tx=0x7f7b3402a410 comp rx=0 tx=0).stop 2024-09-19T05:02:44.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.436+0000 7f7b52800640 1 -- 172.21.15.43:0/2392932660 shutdown_connections 2024-09-19T05:02:44.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.436+0000 7f7b52800640 1 --2- 172.21.15.43:0/2392932660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b4c112a30 0x7f7b4c114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:44.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.436+0000 7f7b52800640 1 --2- 172.21.15.43:0/2392932660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b4c077040 0x7f7b4c1124f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:44.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.436+0000 7f7b52800640 1 --2- 172.21.15.43:0/2392932660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b4c076720 0x7f7b4c076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:44.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.436+0000 7f7b52800640 1 -- 172.21.15.43:0/2392932660 >> 172.21.15.43:0/2392932660 conn(0x7f7b4c0ffa90 msgr2=0x7f7b4c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:44.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.436+0000 7f7b52800640 1 -- 172.21.15.43:0/2392932660 shutdown_connections 2024-09-19T05:02:44.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.436+0000 7f7b52800640 1 -- 172.21.15.43:0/2392932660 wait complete. 2024-09-19T05:02:44.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.437+0000 7f7b52800640 1 Processor -- start 2024-09-19T05:02:44.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.437+0000 7f7b52800640 1 -- start start 2024-09-19T05:02:44.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.438+0000 7f7b52800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b4c076720 0x7f7b4c1a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:44.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.438+0000 7f7b52800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b4c077040 0x7f7b4c1a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:44.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.438+0000 7f7b52800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b4c112a30 0x7f7b4c19e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:44.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.438+0000 7f7b52800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4c118600 con 0x7f7b4c076720 2024-09-19T05:02:44.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.438+0000 7f7b52800640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4c118480 con 0x7f7b4c077040 2024-09-19T05:02:44.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.438+0000 7f7b52800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4c118780 con 0x7f7b4c112a30 2024-09-19T05:02:44.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.438+0000 7f7b4aa00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b4c077040 0x7f7b4c1a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:44.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.438+0000 7f7b4be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b4c112a30 0x7f7b4c19e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:44.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.438+0000 7f7b4aa00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b4c077040 0x7f7b4c1a4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:43254/0 (socket says 172.21.15.43:43254) 2024-09-19T05:02:44.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.438+0000 7f7b4b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b4c076720 0x7f7b4c1a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:44.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.438+0000 7f7b4aa00640 1 -- 172.21.15.43:0/1548580525 learned_addr learned my addr 172.21.15.43:0/1548580525 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:02:44.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.438+0000 7f7b4aa00640 1 -- 172.21.15.43:0/1548580525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b4c112a30 msgr2=0x7f7b4c19e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:44.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.438+0000 7f7b4aa00640 1 --2- 172.21.15.43:0/1548580525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b4c112a30 0x7f7b4c19e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:44.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.438+0000 7f7b4aa00640 1 -- 172.21.15.43:0/1548580525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b4c076720 msgr2=0x7f7b4c1a4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:44.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.438+0000 7f7b4aa00640 1 --2- 172.21.15.43:0/1548580525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b4c076720 0x7f7b4c1a4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:44.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.438+0000 7f7b4aa00640 1 -- 172.21.15.43:0/1548580525 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b4c19eff0 con 0x7f7b4c077040 2024-09-19T05:02:44.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.439+0000 7f7b4aa00640 1 --2- 172.21.15.43:0/1548580525 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b4c077040 0x7f7b4c1a4c40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7b34004770 tx=0x7f7b34003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:44.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.439+0000 7f7b43e00640 1 -- 172.21.15.43:0/1548580525 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3403f070 con 0x7f7b4c077040 2024-09-19T05:02:44.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.439+0000 7f7b52800640 1 -- 172.21.15.43:0/1548580525 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b4c19f280 con 0x7f7b4c077040 2024-09-19T05:02:44.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.439+0000 7f7b43e00640 1 -- 172.21.15.43:0/1548580525 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b34031ca0 con 0x7f7b4c077040 2024-09-19T05:02:44.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.439+0000 7f7b43e00640 1 -- 172.21.15.43:0/1548580525 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b34035030 con 0x7f7b4c077040 2024-09-19T05:02:44.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.439+0000 7f7b52800640 1 -- 172.21.15.43:0/1548580525 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b4c1ab5d0 con 0x7f7b4c077040 2024-09-19T05:02:44.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.440+0000 7f7b52800640 1 -- 172.21.15.43:0/1548580525 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b4c076b00 con 0x7f7b4c077040 2024-09-19T05:02:44.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.442+0000 7f7b43e00640 1 -- 172.21.15.43:0/1548580525 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f7b34042070 con 0x7f7b4c077040 2024-09-19T05:02:44.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.443+0000 7f7b43e00640 1 --2- 172.21.15.43:0/1548580525 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f7b300a32d0 0x7f7b300a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:44.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.444+0000 7f7b43e00640 1 -- 172.21.15.43:0/1548580525 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f7b340f0090 con 0x7f7b4c077040 2024-09-19T05:02:44.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.444+0000 7f7b4b400640 1 --2- 172.21.15.43:0/1548580525 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f7b300a32d0 0x7f7b300a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:44.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.444+0000 7f7b4b400640 1 --2- 172.21.15.43:0/1548580525 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f7b300a32d0 0x7f7b300a5790 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c19fa00 tx=0x7f7b3c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:44.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.448+0000 7f7b43e00640 1 -- 172.21.15.43:0/1548580525 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7b340bc6b0 con 0x7f7b4c077040 2024-09-19T05:02:44.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.663+0000 7f7b52800640 1 -- 172.21.15.43:0/1548580525 --> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f7b4c062630 con 0x7f7b300a32d0 2024-09-19T05:02:44.665 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:44 smithi043 ceph-mon[30800]: pgmap v301: 97 pgs: 97 active+clean; 584 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 4.1 KiB/s wr, 18 op/s 2024-09-19T05:02:44.665 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:44 smithi043 ceph-mon[30800]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T05:02:44.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.691+0000 7f7b43e00640 1 -- 172.21.15.43:0/1548580525 <== mgr.14150 v2:172.21.15.43:6800/3167108925 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7b4c062630 con 0x7f7b300a32d0 2024-09-19T05:02:44.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.693+0000 7f7b52800640 1 -- 172.21.15.43:0/1548580525 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f7b300a32d0 msgr2=0x7f7b300a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:44.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.693+0000 7f7b52800640 1 --2- 172.21.15.43:0/1548580525 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f7b300a32d0 0x7f7b300a5790 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c19fa00 tx=0x7f7b3c009500 comp rx=0 tx=0).stop 2024-09-19T05:02:44.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.693+0000 7f7b52800640 1 -- 172.21.15.43:0/1548580525 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b4c077040 msgr2=0x7f7b4c1a4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:44.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.693+0000 7f7b52800640 1 --2- 172.21.15.43:0/1548580525 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b4c077040 0x7f7b4c1a4c40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7b34004770 tx=0x7f7b34003c10 comp rx=0 tx=0).stop 2024-09-19T05:02:44.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.693+0000 7f7b52800640 1 -- 172.21.15.43:0/1548580525 shutdown_connections 2024-09-19T05:02:44.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.693+0000 7f7b52800640 1 --2- 172.21.15.43:0/1548580525 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f7b300a32d0 0x7f7b300a5790 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:44.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.693+0000 7f7b52800640 1 --2- 172.21.15.43:0/1548580525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b4c112a30 0x7f7b4c19e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:44.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.693+0000 7f7b52800640 1 --2- 172.21.15.43:0/1548580525 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b4c077040 0x7f7b4c1a4c40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:44.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.693+0000 7f7b52800640 1 --2- 172.21.15.43:0/1548580525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b4c076720 0x7f7b4c1a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:44.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.694+0000 7f7b52800640 1 -- 172.21.15.43:0/1548580525 >> 172.21.15.43:0/1548580525 conn(0x7f7b4c0ffa90 msgr2=0x7f7b4c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:44.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.694+0000 7f7b52800640 1 -- 172.21.15.43:0/1548580525 shutdown_connections 2024-09-19T05:02:44.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:44.694+0000 7f7b52800640 1 -- 172.21.15.43:0/1548580525 wait complete. 2024-09-19T05:02:44.904 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:44 smithi151 ceph-mon[35425]: pgmap v301: 97 pgs: 97 active+clean; 584 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 4.1 KiB/s wr, 18 op/s 2024-09-19T05:02:44.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:44 smithi151 ceph-mon[35425]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T05:02:44.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:44 smithi096 ceph-mon[35085]: pgmap v301: 97 pgs: 97 active+clean; 584 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 4.1 KiB/s wr, 18 op/s 2024-09-19T05:02:44.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:44 smithi096 ceph-mon[35085]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T05:02:45.552 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-19T05:02:45.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:45 smithi043 ceph-mon[30800]: from='client.24626 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:02:45.904 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:45 smithi151 ceph-mon[35425]: from='client.24626 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:02:45.931 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:02:45.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:45 smithi096 ceph-mon[35085]: from='client.24626 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:02:46.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.554+0000 7fd295200640 1 -- 172.21.15.43:0/1619209871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd290076720 msgr2=0x7fd290076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:46.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.554+0000 7fd295200640 1 --2- 172.21.15.43:0/1619209871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd290076720 0x7fd290076b00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd280009900 tx=0x7fd28002a410 comp rx=0 tx=0).stop 2024-09-19T05:02:46.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.554+0000 7fd295200640 1 -- 172.21.15.43:0/1619209871 shutdown_connections 2024-09-19T05:02:46.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.554+0000 7fd295200640 1 --2- 172.21.15.43:0/1619209871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd290112a50 0x7fd290114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:46.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.554+0000 7fd295200640 1 --2- 172.21.15.43:0/1619209871 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd290077040 0x7fd290112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:46.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.554+0000 7fd295200640 1 --2- 172.21.15.43:0/1619209871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd290076720 0x7fd290076b00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:46.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.554+0000 7fd295200640 1 -- 172.21.15.43:0/1619209871 >> 172.21.15.43:0/1619209871 conn(0x7fd2900ffa50 msgr2=0x7fd290101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:46.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.554+0000 7fd295200640 1 -- 172.21.15.43:0/1619209871 shutdown_connections 2024-09-19T05:02:46.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.554+0000 7fd295200640 1 -- 172.21.15.43:0/1619209871 wait complete. 2024-09-19T05:02:46.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.555+0000 7fd295200640 1 Processor -- start 2024-09-19T05:02:46.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.556+0000 7fd295200640 1 -- start start 2024-09-19T05:02:46.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.556+0000 7fd295200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd290076720 0x7fd2901a47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:46.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.556+0000 7fd295200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd290077040 0x7fd2901a4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:46.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.556+0000 7fd295200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd290112a50 0x7fd29019e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:46.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.556+0000 7fd295200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd290118610 con 0x7fd290112a50 2024-09-19T05:02:46.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.556+0000 7fd295200640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd290118490 con 0x7fd290076720 2024-09-19T05:02:46.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.556+0000 7fd295200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd290118790 con 0x7fd290077040 2024-09-19T05:02:46.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.556+0000 7fd28de00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd290076720 0x7fd2901a47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:46.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.556+0000 7fd28de00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd290076720 0x7fd2901a47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:42422/0 (socket says 172.21.15.43:42422) 2024-09-19T05:02:46.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.557+0000 7fd28de00640 1 -- 172.21.15.43:0/3080375908 learned_addr learned my addr 172.21.15.43:0/3080375908 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:02:46.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.557+0000 7fd28e800640 1 --2- 172.21.15.43:0/3080375908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd290112a50 0x7fd29019e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:46.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.557+0000 7fd287e00640 1 --2- 172.21.15.43:0/3080375908 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd290077040 0x7fd2901a4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:46.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.557+0000 7fd28de00640 1 -- 172.21.15.43:0/3080375908 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd290077040 msgr2=0x7fd2901a4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:46.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.557+0000 7fd28de00640 1 --2- 172.21.15.43:0/3080375908 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd290077040 0x7fd2901a4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:46.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.557+0000 7fd28de00640 1 -- 172.21.15.43:0/3080375908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd290112a50 msgr2=0x7fd29019e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:46.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.557+0000 7fd28de00640 1 --2- 172.21.15.43:0/3080375908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd290112a50 0x7fd29019e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:46.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.557+0000 7fd28de00640 1 -- 172.21.15.43:0/3080375908 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd29019f0d0 con 0x7fd290076720 2024-09-19T05:02:46.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.557+0000 7fd28e800640 1 --2- 172.21.15.43:0/3080375908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd290112a50 0x7fd29019e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:02:46.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.557+0000 7fd28de00640 1 --2- 172.21.15.43:0/3080375908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd290076720 0x7fd2901a47e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd280004770 tx=0x7fd280003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:46.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.558+0000 7fd286a00640 1 -- 172.21.15.43:0/3080375908 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28003f070 con 0x7fd290076720 2024-09-19T05:02:46.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.558+0000 7fd286a00640 1 -- 172.21.15.43:0/3080375908 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd280031ca0 con 0x7fd290076720 2024-09-19T05:02:46.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.558+0000 7fd295200640 1 -- 172.21.15.43:0/3080375908 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd29019f360 con 0x7fd290076720 2024-09-19T05:02:46.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.558+0000 7fd286a00640 1 -- 172.21.15.43:0/3080375908 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd280035030 con 0x7fd290076720 2024-09-19T05:02:46.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.558+0000 7fd295200640 1 -- 172.21.15.43:0/3080375908 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2901ab6b0 con 0x7fd290076720 2024-09-19T05:02:46.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.559+0000 7fd295200640 1 -- 172.21.15.43:0/3080375908 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd290076b00 con 0x7fd290076720 2024-09-19T05:02:46.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.561+0000 7fd286a00640 1 -- 172.21.15.43:0/3080375908 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fd280042070 con 0x7fd290076720 2024-09-19T05:02:46.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.562+0000 7fd286a00640 1 --2- 172.21.15.43:0/3080375908 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fd2740a32d0 0x7fd2740a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:46.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.562+0000 7fd286a00640 1 -- 172.21.15.43:0/3080375908 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7fd2800f01b0 con 0x7fd290076720 2024-09-19T05:02:46.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.562+0000 7fd287e00640 1 --2- 172.21.15.43:0/3080375908 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fd2740a32d0 0x7fd2740a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:46.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.563+0000 7fd287e00640 1 --2- 172.21.15.43:0/3080375908 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fd2740a32d0 0x7fd2740a5790 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd290079220 tx=0x7fd27800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:46.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.564+0000 7fd286a00640 1 -- 172.21.15.43:0/3080375908 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd2800bc8e0 con 0x7fd290076720 2024-09-19T05:02:46.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.777+0000 7fd295200640 1 -- 172.21.15.43:0/3080375908 --> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fd290062630 con 0x7fd2740a32d0 2024-09-19T05:02:46.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:46 smithi043 ceph-mon[30800]: pgmap v302: 97 pgs: 97 active+clean; 584 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 17 op/s 2024-09-19T05:02:46.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.783+0000 7fd286a00640 1 -- 172.21.15.43:0/3080375908 <== mgr.14150 v2:172.21.15.43:6800/3167108925 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fd290062630 con 0x7fd2740a32d0 2024-09-19T05:02:46.785 INFO:teuthology.orchestra.run.smithi043.stdout:/volumes/qa/sv_0/29aa0ea4-ba54-4984-9ef4-c278593c8812 2024-09-19T05:02:46.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.785+0000 7fd295200640 1 -- 172.21.15.43:0/3080375908 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fd2740a32d0 msgr2=0x7fd2740a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:46.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.785+0000 7fd295200640 1 --2- 172.21.15.43:0/3080375908 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fd2740a32d0 0x7fd2740a5790 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd290079220 tx=0x7fd27800b390 comp rx=0 tx=0).stop 2024-09-19T05:02:46.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.785+0000 7fd295200640 1 -- 172.21.15.43:0/3080375908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd290076720 msgr2=0x7fd2901a47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:46.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.785+0000 7fd295200640 1 --2- 172.21.15.43:0/3080375908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd290076720 0x7fd2901a47e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd280004770 tx=0x7fd280003c10 comp rx=0 tx=0).stop 2024-09-19T05:02:46.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.785+0000 7fd295200640 1 -- 172.21.15.43:0/3080375908 shutdown_connections 2024-09-19T05:02:46.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.785+0000 7fd295200640 1 --2- 172.21.15.43:0/3080375908 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fd2740a32d0 0x7fd2740a5790 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:46.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.785+0000 7fd295200640 1 --2- 172.21.15.43:0/3080375908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd290112a50 0x7fd29019e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:46.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.785+0000 7fd295200640 1 --2- 172.21.15.43:0/3080375908 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd290077040 0x7fd2901a4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:46.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.785+0000 7fd295200640 1 --2- 172.21.15.43:0/3080375908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd290076720 0x7fd2901a47e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:46.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.785+0000 7fd295200640 1 -- 172.21.15.43:0/3080375908 >> 172.21.15.43:0/3080375908 conn(0x7fd2900ffa50 msgr2=0x7fd290101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:46.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.785+0000 7fd295200640 1 -- 172.21.15.43:0/3080375908 shutdown_connections 2024-09-19T05:02:46.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:46.785+0000 7fd295200640 1 -- 172.21.15.43:0/3080375908 wait complete. 2024-09-19T05:02:46.904 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:46 smithi151 ceph-mon[35425]: pgmap v302: 97 pgs: 97 active+clean; 584 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 17 op/s 2024-09-19T05:02:46.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:46 smithi096 ceph-mon[35085]: pgmap v302: 97 pgs: 97 active+clean; 584 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 17 op/s 2024-09-19T05:02:47.625 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-19T05:02:47.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:47 smithi043 ceph-mon[30800]: from='client.24632 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:02:47.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:47 smithi151 ceph-mon[35425]: from='client.24632 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:02:47.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:47 smithi096 ceph-mon[35085]: from='client.24632 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:02:48.001 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:02:48.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.548+0000 7f1537e00640 1 -- 172.21.15.43:0/3076640617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15381047e0 msgr2=0x7f1538108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:48.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.548+0000 7f1537e00640 1 --2- 172.21.15.43:0/3076640617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15381047e0 0x7f1538108830 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1524009900 tx=0x7f152402a410 comp rx=0 tx=0).stop 2024-09-19T05:02:48.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.548+0000 7f1537e00640 1 -- 172.21.15.43:0/3076640617 shutdown_connections 2024-09-19T05:02:48.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.548+0000 7f1537e00640 1 --2- 172.21.15.43:0/3076640617 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1538108f60 0x7f1538110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:48.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.548+0000 7f1537e00640 1 --2- 172.21.15.43:0/3076640617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15381047e0 0x7f1538108830 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:48.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.548+0000 7f1537e00640 1 --2- 172.21.15.43:0/3076640617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1538103e30 0x7f1538104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:48.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.548+0000 7f1537e00640 1 -- 172.21.15.43:0/3076640617 >> 172.21.15.43:0/3076640617 conn(0x7f15380ff8d0 msgr2=0x7f1538101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:48.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.548+0000 7f1537e00640 1 -- 172.21.15.43:0/3076640617 shutdown_connections 2024-09-19T05:02:48.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.548+0000 7f1537e00640 1 -- 172.21.15.43:0/3076640617 wait complete. 2024-09-19T05:02:48.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.549+0000 7f1537e00640 1 Processor -- start 2024-09-19T05:02:48.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.549+0000 7f1537e00640 1 -- start start 2024-09-19T05:02:48.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.549+0000 7f1537e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1538103e30 0x7f153819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:48.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.549+0000 7f1537e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15381047e0 0x7f153819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:48.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.549+0000 7f1537e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1538108f60 0x7f153819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:48.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.549+0000 7f1537e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1538114010 con 0x7f1538108f60 2024-09-19T05:02:48.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.549+0000 7f1537e00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1538113e90 con 0x7f1538103e30 2024-09-19T05:02:48.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.549+0000 7f1537e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1538114190 con 0x7f15381047e0 2024-09-19T05:02:48.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.550+0000 7f1536a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1538103e30 0x7f153819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:48.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.550+0000 7f1536a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1538103e30 0x7f153819a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:42450/0 (socket says 172.21.15.43:42450) 2024-09-19T05:02:48.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.550+0000 7f1536a00640 1 -- 172.21.15.43:0/901502602 learned_addr learned my addr 172.21.15.43:0/901502602 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:02:48.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.550+0000 7f1537400640 1 --2- 172.21.15.43:0/901502602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1538108f60 0x7f153819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:48.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.550+0000 7f1536000640 1 --2- 172.21.15.43:0/901502602 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15381047e0 0x7f153819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:48.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.550+0000 7f1536a00640 1 -- 172.21.15.43:0/901502602 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15381047e0 msgr2=0x7f153819a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:48.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.550+0000 7f1536a00640 1 --2- 172.21.15.43:0/901502602 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15381047e0 0x7f153819a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:48.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.550+0000 7f1536a00640 1 -- 172.21.15.43:0/901502602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1538108f60 msgr2=0x7f153819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:48.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.550+0000 7f1536a00640 1 --2- 172.21.15.43:0/901502602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1538108f60 0x7f153819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:48.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.550+0000 7f1536a00640 1 -- 172.21.15.43:0/901502602 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f153819f460 con 0x7f1538103e30 2024-09-19T05:02:48.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.550+0000 7f1536a00640 1 --2- 172.21.15.43:0/901502602 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1538103e30 0x7f153819a4b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f152800e9f0 tx=0x7f152800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:48.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.550+0000 7f152f400640 1 -- 172.21.15.43:0/901502602 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f152800cd80 con 0x7f1538103e30 2024-09-19T05:02:48.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.551+0000 7f152f400640 1 -- 172.21.15.43:0/901502602 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1528005500 con 0x7f1538103e30 2024-09-19T05:02:48.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.551+0000 7f1537e00640 1 -- 172.21.15.43:0/901502602 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f153819f750 con 0x7f1538103e30 2024-09-19T05:02:48.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.551+0000 7f152f400640 1 -- 172.21.15.43:0/901502602 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1528010690 con 0x7f1538103e30 2024-09-19T05:02:48.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.551+0000 7f1537e00640 1 -- 172.21.15.43:0/901502602 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15381a7030 con 0x7f1538103e30 2024-09-19T05:02:48.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.552+0000 7f1537e00640 1 -- 172.21.15.43:0/901502602 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1538104210 con 0x7f1538103e30 2024-09-19T05:02:48.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.553+0000 7f152f400640 1 -- 172.21.15.43:0/901502602 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f15280046c0 con 0x7f1538103e30 2024-09-19T05:02:48.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.555+0000 7f152f400640 1 --2- 172.21.15.43:0/901502602 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f151c0a32d0 0x7f151c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:48.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.555+0000 7f152f400640 1 -- 172.21.15.43:0/901502602 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f15280cfdd0 con 0x7f1538103e30 2024-09-19T05:02:48.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.555+0000 7f1536000640 1 --2- 172.21.15.43:0/901502602 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f151c0a32d0 0x7f151c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:48.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.556+0000 7f1536000640 1 --2- 172.21.15.43:0/901502602 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f151c0a32d0 0x7f151c0a5790 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1524000c00 tx=0x7f1524005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:48.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.558+0000 7f152f400640 1 -- 172.21.15.43:0/901502602 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f152809ca40 con 0x7f1538103e30 2024-09-19T05:02:48.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.768+0000 7f1537e00640 1 -- 172.21.15.43:0/901502602 --> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f1538062630 con 0x7f151c0a32d0 2024-09-19T05:02:48.771 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:48 smithi043 ceph-mon[30800]: pgmap v303: 97 pgs: 97 active+clean; 605 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 6.3 KiB/s wr, 22 op/s 2024-09-19T05:02:48.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.794+0000 7f152f400640 1 -- 172.21.15.43:0/901502602 <== mgr.14150 v2:172.21.15.43:6800/3167108925 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1538062630 con 0x7f151c0a32d0 2024-09-19T05:02:48.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.796+0000 7f1537e00640 1 -- 172.21.15.43:0/901502602 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f151c0a32d0 msgr2=0x7f151c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:48.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.796+0000 7f1537e00640 1 --2- 172.21.15.43:0/901502602 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f151c0a32d0 0x7f151c0a5790 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1524000c00 tx=0x7f1524005b50 comp rx=0 tx=0).stop 2024-09-19T05:02:48.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.796+0000 7f1537e00640 1 -- 172.21.15.43:0/901502602 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1538103e30 msgr2=0x7f153819a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:48.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.796+0000 7f1537e00640 1 --2- 172.21.15.43:0/901502602 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1538103e30 0x7f153819a4b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f152800e9f0 tx=0x7f152800edf0 comp rx=0 tx=0).stop 2024-09-19T05:02:48.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.796+0000 7f1537e00640 1 -- 172.21.15.43:0/901502602 shutdown_connections 2024-09-19T05:02:48.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.796+0000 7f1537e00640 1 --2- 172.21.15.43:0/901502602 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f151c0a32d0 0x7f151c0a5790 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:48.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.796+0000 7f1537e00640 1 --2- 172.21.15.43:0/901502602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1538108f60 0x7f153819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:48.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.796+0000 7f1537e00640 1 --2- 172.21.15.43:0/901502602 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15381047e0 0x7f153819a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:48.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.796+0000 7f1537e00640 1 --2- 172.21.15.43:0/901502602 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1538103e30 0x7f153819a4b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:48.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.796+0000 7f1537e00640 1 -- 172.21.15.43:0/901502602 >> 172.21.15.43:0/901502602 conn(0x7f15380ff8d0 msgr2=0x7f1538100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:48.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.796+0000 7f1537e00640 1 -- 172.21.15.43:0/901502602 shutdown_connections 2024-09-19T05:02:48.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:48.796+0000 7f1537e00640 1 -- 172.21.15.43:0/901502602 wait complete. 2024-09-19T05:02:48.904 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:48 smithi151 ceph-mon[35425]: pgmap v303: 97 pgs: 97 active+clean; 605 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 6.3 KiB/s wr, 22 op/s 2024-09-19T05:02:48.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:48 smithi096 ceph-mon[35085]: pgmap v303: 97 pgs: 97 active+clean; 605 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 6.3 KiB/s wr, 22 op/s 2024-09-19T05:02:49.499 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-19T05:02:49.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:49 smithi043 ceph-mon[30800]: from='client.24638 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:02:49.889 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:02:49.904 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:49 smithi151 ceph-mon[35425]: from='client.24638 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:02:49.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:49 smithi096 ceph-mon[35085]: from='client.24638 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:02:50.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.454+0000 7f8c3e800640 1 -- 172.21.15.43:0/1750901602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c381049c0 msgr2=0x7f8c38108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:50.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.454+0000 7f8c3e800640 1 --2- 172.21.15.43:0/1750901602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c381049c0 0x7f8c38108a10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8c20009900 tx=0x7f8c2002a3f0 comp rx=0 tx=0).stop 2024-09-19T05:02:50.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.455+0000 7f8c3e800640 1 -- 172.21.15.43:0/1750901602 shutdown_connections 2024-09-19T05:02:50.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.455+0000 7f8c3e800640 1 --2- 172.21.15.43:0/1750901602 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c38109140 0x7f8c38110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:50.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.455+0000 7f8c3e800640 1 --2- 172.21.15.43:0/1750901602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c381049c0 0x7f8c38108a10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:50.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.455+0000 7f8c3e800640 1 --2- 172.21.15.43:0/1750901602 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c38104010 0x7f8c381043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:50.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.455+0000 7f8c3e800640 1 -- 172.21.15.43:0/1750901602 >> 172.21.15.43:0/1750901602 conn(0x7f8c380ffa50 msgr2=0x7f8c38101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:50.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.455+0000 7f8c3e800640 1 -- 172.21.15.43:0/1750901602 shutdown_connections 2024-09-19T05:02:50.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.455+0000 7f8c3e800640 1 -- 172.21.15.43:0/1750901602 wait complete. 2024-09-19T05:02:50.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.456+0000 7f8c3e800640 1 Processor -- start 2024-09-19T05:02:50.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.456+0000 7f8c3e800640 1 -- start start 2024-09-19T05:02:50.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.456+0000 7f8c3e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c38104010 0x7f8c3819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:50.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.456+0000 7f8c3e800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c381049c0 0x7f8c3819ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:50.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.456+0000 7f8c3e800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c38109140 0x7f8c3819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:50.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.456+0000 7f8c3e800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8c38114160 con 0x7f8c38104010 2024-09-19T05:02:50.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.456+0000 7f8c3e800640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8c38113fe0 con 0x7f8c38109140 2024-09-19T05:02:50.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.456+0000 7f8c3e800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8c381142e0 con 0x7f8c381049c0 2024-09-19T05:02:50.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.456+0000 7f8c37400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c38104010 0x7f8c3819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:50.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.456+0000 7f8c37400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c38104010 0x7f8c3819a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60898/0 (socket says 172.21.15.43:60898) 2024-09-19T05:02:50.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.456+0000 7f8c37400640 1 -- 172.21.15.43:0/1044907269 learned_addr learned my addr 172.21.15.43:0/1044907269 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:02:50.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.457+0000 7f8c2ea00640 1 --2- 172.21.15.43:0/1044907269 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c381049c0 0x7f8c3819ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:50.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.457+0000 7f8c37e00640 1 --2- 172.21.15.43:0/1044907269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c38109140 0x7f8c3819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:50.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.457+0000 7f8c37400640 1 -- 172.21.15.43:0/1044907269 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c381049c0 msgr2=0x7f8c3819ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:50.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.457+0000 7f8c37400640 1 --2- 172.21.15.43:0/1044907269 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c381049c0 0x7f8c3819ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:50.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.457+0000 7f8c37400640 1 -- 172.21.15.43:0/1044907269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c38109140 msgr2=0x7f8c3819efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:50.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.457+0000 7f8c37400640 1 --2- 172.21.15.43:0/1044907269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c38109140 0x7f8c3819efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:50.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.457+0000 7f8c37400640 1 -- 172.21.15.43:0/1044907269 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c3819f690 con 0x7f8c38104010 2024-09-19T05:02:50.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.457+0000 7f8c37e00640 1 --2- 172.21.15.43:0/1044907269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c38109140 0x7f8c3819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:02:50.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.457+0000 7f8c37400640 1 --2- 172.21.15.43:0/1044907269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c38104010 0x7f8c3819a6e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8c2800eb90 tx=0x7f8c2800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:50.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.457+0000 7f8c34c00640 1 -- 172.21.15.43:0/1044907269 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c28004d20 con 0x7f8c38104010 2024-09-19T05:02:50.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.457+0000 7f8c3e800640 1 -- 172.21.15.43:0/1044907269 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c3819f980 con 0x7f8c38104010 2024-09-19T05:02:50.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.457+0000 7f8c34c00640 1 -- 172.21.15.43:0/1044907269 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c2800ee90 con 0x7f8c38104010 2024-09-19T05:02:50.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.457+0000 7f8c34c00640 1 -- 172.21.15.43:0/1044907269 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c28010830 con 0x7f8c38104010 2024-09-19T05:02:50.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.458+0000 7f8c3e800640 1 -- 172.21.15.43:0/1044907269 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c381a7260 con 0x7f8c38104010 2024-09-19T05:02:50.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.459+0000 7f8c3e800640 1 -- 172.21.15.43:0/1044907269 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c381043f0 con 0x7f8c38104010 2024-09-19T05:02:50.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.460+0000 7f8c34c00640 1 -- 172.21.15.43:0/1044907269 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f8c28005020 con 0x7f8c38104010 2024-09-19T05:02:50.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.462+0000 7f8c34c00640 1 --2- 172.21.15.43:0/1044907269 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f8c1c0a32d0 0x7f8c1c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:50.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.462+0000 7f8c34c00640 1 -- 172.21.15.43:0/1044907269 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f8c280d00e0 con 0x7f8c38104010 2024-09-19T05:02:50.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.462+0000 7f8c2ea00640 1 --2- 172.21.15.43:0/1044907269 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f8c1c0a32d0 0x7f8c1c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:50.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.463+0000 7f8c2ea00640 1 --2- 172.21.15.43:0/1044907269 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f8c1c0a32d0 0x7f8c1c0a5790 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8c20009230 tx=0x7f8c200058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:50.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.466+0000 7f8c34c00640 1 -- 172.21.15.43:0/1044907269 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8c2809cd50 con 0x7f8c38104010 2024-09-19T05:02:50.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.684+0000 7f8c3e800640 1 -- 172.21.15.43:0/1044907269 --> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f8c38062630 con 0x7f8c1c0a32d0 2024-09-19T05:02:50.686 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:50 smithi043 ceph-mon[30800]: pgmap v304: 97 pgs: 97 active+clean; 606 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.0 KiB/s wr, 22 op/s 2024-09-19T05:02:50.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.688+0000 7f8c34c00640 1 -- 172.21.15.43:0/1044907269 <== mgr.14150 v2:172.21.15.43:6800/3167108925 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f8c38062630 con 0x7f8c1c0a32d0 2024-09-19T05:02:50.691 INFO:teuthology.orchestra.run.smithi043.stdout:/volumes/qa/sv_1/0bec0f8f-1c83-4286-a8b9-aa1bb4d77684 2024-09-19T05:02:50.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.691+0000 7f8c3e800640 1 -- 172.21.15.43:0/1044907269 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f8c1c0a32d0 msgr2=0x7f8c1c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:50.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.691+0000 7f8c3e800640 1 --2- 172.21.15.43:0/1044907269 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f8c1c0a32d0 0x7f8c1c0a5790 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8c20009230 tx=0x7f8c200058d0 comp rx=0 tx=0).stop 2024-09-19T05:02:50.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.691+0000 7f8c3e800640 1 -- 172.21.15.43:0/1044907269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c38104010 msgr2=0x7f8c3819a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:50.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.691+0000 7f8c3e800640 1 --2- 172.21.15.43:0/1044907269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c38104010 0x7f8c3819a6e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8c2800eb90 tx=0x7f8c2800c6a0 comp rx=0 tx=0).stop 2024-09-19T05:02:50.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.691+0000 7f8c3e800640 1 -- 172.21.15.43:0/1044907269 shutdown_connections 2024-09-19T05:02:50.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.691+0000 7f8c3e800640 1 --2- 172.21.15.43:0/1044907269 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f8c1c0a32d0 0x7f8c1c0a5790 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:50.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.691+0000 7f8c3e800640 1 --2- 172.21.15.43:0/1044907269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c38109140 0x7f8c3819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:50.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.691+0000 7f8c3e800640 1 --2- 172.21.15.43:0/1044907269 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c381049c0 0x7f8c3819ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:50.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.691+0000 7f8c3e800640 1 --2- 172.21.15.43:0/1044907269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c38104010 0x7f8c3819a6e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:50.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.691+0000 7f8c3e800640 1 -- 172.21.15.43:0/1044907269 >> 172.21.15.43:0/1044907269 conn(0x7f8c380ffa50 msgr2=0x7f8c38100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:50.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.691+0000 7f8c3e800640 1 -- 172.21.15.43:0/1044907269 shutdown_connections 2024-09-19T05:02:50.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:50.691+0000 7f8c3e800640 1 -- 172.21.15.43:0/1044907269 wait complete. 2024-09-19T05:02:50.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:50 smithi151 ceph-mon[35425]: pgmap v304: 97 pgs: 97 active+clean; 606 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.0 KiB/s wr, 22 op/s 2024-09-19T05:02:50.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:50 smithi096 ceph-mon[35085]: pgmap v304: 97 pgs: 97 active+clean; 606 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.0 KiB/s wr, 22 op/s 2024-09-19T05:02:51.532 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:02:51.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:51 smithi043 ceph-mon[30800]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:02:51.806 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:51 smithi151 ceph-mon[35425]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:02:51.874 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:51 smithi096 ceph-mon[35085]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:02:51.910 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:02:52.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.483+0000 7f271e800640 1 -- 172.21.15.43:0/1605354041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2718103ff0 msgr2=0x7f27181043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:52.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.483+0000 7f271e800640 1 --2- 172.21.15.43:0/1605354041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2718103ff0 0x7f27181043d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2708009900 tx=0x7f270802a3f0 comp rx=0 tx=0).stop 2024-09-19T05:02:52.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.483+0000 7f271e800640 1 -- 172.21.15.43:0/1605354041 shutdown_connections 2024-09-19T05:02:52.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.483+0000 7f271e800640 1 --2- 172.21.15.43:0/1605354041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2718109120 0x7f2718110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:52.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.483+0000 7f271e800640 1 --2- 172.21.15.43:0/1605354041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27181049a0 0x7f27181089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:52.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.483+0000 7f271e800640 1 --2- 172.21.15.43:0/1605354041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2718103ff0 0x7f27181043d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:52.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.483+0000 7f271e800640 1 -- 172.21.15.43:0/1605354041 >> 172.21.15.43:0/1605354041 conn(0x7f27180ffa90 msgr2=0x7f2718101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:52.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.483+0000 7f271e800640 1 -- 172.21.15.43:0/1605354041 shutdown_connections 2024-09-19T05:02:52.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.483+0000 7f271e800640 1 -- 172.21.15.43:0/1605354041 wait complete. 2024-09-19T05:02:52.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.484+0000 7f271e800640 1 Processor -- start 2024-09-19T05:02:52.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.484+0000 7f271e800640 1 -- start start 2024-09-19T05:02:52.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.485+0000 7f271e800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2718103ff0 0x7f271819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:52.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.485+0000 7f271e800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27181049a0 0x7f271819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:52.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.485+0000 7f271e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2718109120 0x7f271819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:52.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.485+0000 7f271e800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f27181141a0 con 0x7f2718109120 2024-09-19T05:02:52.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.485+0000 7f271e800640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2718114020 con 0x7f2718103ff0 2024-09-19T05:02:52.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.485+0000 7f271e800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2718114320 con 0x7f27181049a0 2024-09-19T05:02:52.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.485+0000 7f2717400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2718103ff0 0x7f271819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:52.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.485+0000 7f2717e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2718109120 0x7f271819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:52.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.485+0000 7f2716a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27181049a0 0x7f271819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:52.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.485+0000 7f2717400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2718103ff0 0x7f271819a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:42474/0 (socket says 172.21.15.43:42474) 2024-09-19T05:02:52.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.485+0000 7f2717400640 1 -- 172.21.15.43:0/3358738297 learned_addr learned my addr 172.21.15.43:0/3358738297 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:02:52.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.485+0000 7f2717400640 1 -- 172.21.15.43:0/3358738297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27181049a0 msgr2=0x7f271819abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:52.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.485+0000 7f2717400640 1 --2- 172.21.15.43:0/3358738297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27181049a0 0x7f271819abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:52.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.485+0000 7f2717400640 1 -- 172.21.15.43:0/3358738297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2718109120 msgr2=0x7f271819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:52.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.485+0000 7f2717400640 1 --2- 172.21.15.43:0/3358738297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2718109120 0x7f271819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:52.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.485+0000 7f2717400640 1 -- 172.21.15.43:0/3358738297 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f271819f650 con 0x7f2718103ff0 2024-09-19T05:02:52.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.486+0000 7f2717400640 1 --2- 172.21.15.43:0/3358738297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2718103ff0 0x7f271819a6a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2708004770 tx=0x7f2708003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:52.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.486+0000 7f270fe00640 1 -- 172.21.15.43:0/3358738297 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270803f070 con 0x7f2718103ff0 2024-09-19T05:02:52.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.486+0000 7f271e800640 1 -- 172.21.15.43:0/3358738297 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f271819f8e0 con 0x7f2718103ff0 2024-09-19T05:02:52.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.486+0000 7f270fe00640 1 -- 172.21.15.43:0/3358738297 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2708031ca0 con 0x7f2718103ff0 2024-09-19T05:02:52.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.486+0000 7f270fe00640 1 -- 172.21.15.43:0/3358738297 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2708035030 con 0x7f2718103ff0 2024-09-19T05:02:52.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.486+0000 7f271e800640 1 -- 172.21.15.43:0/3358738297 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27181a7220 con 0x7f2718103ff0 2024-09-19T05:02:52.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.487+0000 7f271e800640 1 -- 172.21.15.43:0/3358738297 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27181054d0 con 0x7f2718103ff0 2024-09-19T05:02:52.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.489+0000 7f270fe00640 1 -- 172.21.15.43:0/3358738297 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f2708042070 con 0x7f2718103ff0 2024-09-19T05:02:52.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.490+0000 7f270fe00640 1 --2- 172.21.15.43:0/3358738297 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f26fc0a32d0 0x7f26fc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:52.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.491+0000 7f270fe00640 1 -- 172.21.15.43:0/3358738297 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f27080f04d0 con 0x7f2718103ff0 2024-09-19T05:02:52.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.491+0000 7f2716a00640 1 --2- 172.21.15.43:0/3358738297 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f26fc0a32d0 0x7f26fc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:52.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.491+0000 7f2716a00640 1 --2- 172.21.15.43:0/3358738297 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f26fc0a32d0 0x7f26fc0a5790 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f27180fffb0 tx=0x7f270400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:52.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.493+0000 7f270fe00640 1 -- 172.21.15.43:0/3358738297 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f27080bcc00 con 0x7f2718103ff0 2024-09-19T05:02:52.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.714+0000 7f271e800640 1 -- 172.21.15.43:0/3358738297 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27181043d0 con 0x7f2718103ff0 2024-09-19T05:02:52.717 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:52 smithi043 ceph-mon[30800]: pgmap v305: 97 pgs: 97 active+clean; 626 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.7 KiB/s wr, 22 op/s 2024-09-19T05:02:52.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.715+0000 7f270fe00640 1 -- 172.21.15.43:0/3358738297 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11965 (secure 0 0 0) 0x7f27080c0f00 con 0x7f2718103ff0 2024-09-19T05:02:52.718 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:02:52.718 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":28,"btime":"2024-09-19T05:02:40:505009+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/773801169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":773801169},{"type":"v1","addr":"172.21.15.43:6835","nonce":773801169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1799137902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1799137902},{"type":"v1","addr":"172.21.15.43:6837","nonce":1799137902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:02:25.463041+0000","modified":"2024-09-19T05:02:39.530912+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24518,"mds_2":24511,"mds_3":24506,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14541":{"gid":14541,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/794881135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":794881135},{"type":"v1","addr":"172.21.15.43:6841","nonce":794881135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/2836453158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2836453158},{"type":"v1","addr":"172.21.15.151:6833","nonce":2836453158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/178785812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":178785812},{"type":"v1","addr":"172.21.15.96:6833","nonce":178785812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1671043820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1671043820},{"type":"v1","addr":"172.21.15.96:6835","nonce":1671043820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2972514328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2972514328},{"type":"v1","addr":"172.21.15.151:6835","nonce":2972514328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.151:6837/3960378010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3960378010},{"type":"v1","addr":"172.21.15.151:6837","nonce":3960378010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6837/3978388286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3978388286},{"type":"v1","addr":"172.21.15.96:6837","nonce":3978388286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.151:6839/3794104313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3794104313},{"type":"v1","addr":"172.21.15.151:6839","nonce":3794104313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.43:6839/3404555030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3404555030},{"type":"v1","addr":"172.21.15.43:6839","nonce":3404555030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/605527621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":605527621},{"type":"v1","addr":"172.21.15.96:6839","nonce":605527621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24493,24506,24511,24518,24542]},"id":1}]} 2024-09-19T05:02:52.719 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 28 2024-09-19T05:02:52.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.718+0000 7f271e800640 1 -- 172.21.15.43:0/3358738297 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f26fc0a32d0 msgr2=0x7f26fc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:52.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.718+0000 7f271e800640 1 --2- 172.21.15.43:0/3358738297 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f26fc0a32d0 0x7f26fc0a5790 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f27180fffb0 tx=0x7f270400b410 comp rx=0 tx=0).stop 2024-09-19T05:02:52.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.718+0000 7f271e800640 1 -- 172.21.15.43:0/3358738297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2718103ff0 msgr2=0x7f271819a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:52.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.718+0000 7f271e800640 1 --2- 172.21.15.43:0/3358738297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2718103ff0 0x7f271819a6a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2708004770 tx=0x7f2708003c10 comp rx=0 tx=0).stop 2024-09-19T05:02:52.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.718+0000 7f271e800640 1 -- 172.21.15.43:0/3358738297 shutdown_connections 2024-09-19T05:02:52.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.718+0000 7f271e800640 1 --2- 172.21.15.43:0/3358738297 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f26fc0a32d0 0x7f26fc0a5790 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:52.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.718+0000 7f271e800640 1 --2- 172.21.15.43:0/3358738297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2718109120 0x7f271819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:52.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.718+0000 7f271e800640 1 --2- 172.21.15.43:0/3358738297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27181049a0 0x7f271819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:52.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.718+0000 7f271e800640 1 --2- 172.21.15.43:0/3358738297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2718103ff0 0x7f271819a6a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:52.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.718+0000 7f271e800640 1 -- 172.21.15.43:0/3358738297 >> 172.21.15.43:0/3358738297 conn(0x7f27180ffa90 msgr2=0x7f2718100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:52.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.719+0000 7f271e800640 1 -- 172.21.15.43:0/3358738297 shutdown_connections 2024-09-19T05:02:52.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:52.719+0000 7f271e800640 1 -- 172.21.15.43:0/3358738297 wait complete. 2024-09-19T05:02:52.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:52 smithi151 ceph-mon[35425]: pgmap v305: 97 pgs: 97 active+clean; 626 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.7 KiB/s wr, 22 op/s 2024-09-19T05:02:52.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:52 smithi096 ceph-mon[35085]: pgmap v305: 97 pgs: 97 active+clean; 626 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.7 KiB/s wr, 22 op/s 2024-09-19T05:02:53.625 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T05:02:53.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:53 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/3358738297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:02:53.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:53 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/3358738297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:02:54.001 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:02:54.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:53 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/3358738297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:02:54.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.510+0000 7f147ac00640 1 -- 172.21.15.43:0/1160440052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1474108f60 msgr2=0x7f1474110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:54.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.510+0000 7f147ac00640 1 --2- 172.21.15.43:0/1160440052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1474108f60 0x7f1474110ae0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f146c009900 tx=0x7f146c02a3f0 comp rx=0 tx=0).stop 2024-09-19T05:02:54.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.510+0000 7f147ac00640 1 -- 172.21.15.43:0/1160440052 shutdown_connections 2024-09-19T05:02:54.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.510+0000 7f147ac00640 1 --2- 172.21.15.43:0/1160440052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1474108f60 0x7f1474110ae0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:54.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.510+0000 7f147ac00640 1 --2- 172.21.15.43:0/1160440052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14741047e0 0x7f1474108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:54.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.510+0000 7f147ac00640 1 --2- 172.21.15.43:0/1160440052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1474103e30 0x7f1474104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:54.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.510+0000 7f147ac00640 1 -- 172.21.15.43:0/1160440052 >> 172.21.15.43:0/1160440052 conn(0x7f14740ff910 msgr2=0x7f1474101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:54.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.510+0000 7f147ac00640 1 -- 172.21.15.43:0/1160440052 shutdown_connections 2024-09-19T05:02:54.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.510+0000 7f147ac00640 1 -- 172.21.15.43:0/1160440052 wait complete. 2024-09-19T05:02:54.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.511+0000 7f147ac00640 1 Processor -- start 2024-09-19T05:02:54.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.511+0000 7f147ac00640 1 -- start start 2024-09-19T05:02:54.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.511+0000 7f147ac00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1474103e30 0x7f147419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:54.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.511+0000 7f147ac00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14741047e0 0x7f147419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:54.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.511+0000 7f147ac00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1474108f60 0x7f147419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:54.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.511+0000 7f147ac00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1474114010 con 0x7f1474108f60 2024-09-19T05:02:54.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.511+0000 7f147ac00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1474113e90 con 0x7f14741047e0 2024-09-19T05:02:54.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.511+0000 7f147ac00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1474114190 con 0x7f1474103e30 2024-09-19T05:02:54.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.512+0000 7f147a200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1474108f60 0x7f147419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:54.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.512+0000 7f1479800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1474103e30 0x7f147419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:54.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.512+0000 7f147a200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1474108f60 0x7f147419edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60942/0 (socket says 172.21.15.43:60942) 2024-09-19T05:02:54.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.512+0000 7f1479800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1474103e30 0x7f147419a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:58054/0 (socket says 172.21.15.43:58054) 2024-09-19T05:02:54.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.512+0000 7f147a200640 1 -- 172.21.15.43:0/2293131510 learned_addr learned my addr 172.21.15.43:0/2293131510 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:02:54.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.512+0000 7f1478e00640 1 --2- 172.21.15.43:0/2293131510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14741047e0 0x7f147419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:54.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.512+0000 7f147a200640 1 -- 172.21.15.43:0/2293131510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1474103e30 msgr2=0x7f147419a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:54.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.512+0000 7f147a200640 1 --2- 172.21.15.43:0/2293131510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1474103e30 0x7f147419a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:54.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.512+0000 7f147a200640 1 -- 172.21.15.43:0/2293131510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14741047e0 msgr2=0x7f147419aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:54.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.512+0000 7f147a200640 1 --2- 172.21.15.43:0/2293131510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14741047e0 0x7f147419aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:54.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.512+0000 7f147a200640 1 -- 172.21.15.43:0/2293131510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f147419f4a0 con 0x7f1474108f60 2024-09-19T05:02:54.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.512+0000 7f147a200640 1 --2- 172.21.15.43:0/2293131510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1474108f60 0x7f147419edc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f146c004770 tx=0x7f146c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:54.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.513+0000 7f1472000640 1 -- 172.21.15.43:0/2293131510 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146c03f070 con 0x7f1474108f60 2024-09-19T05:02:54.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.513+0000 7f1472000640 1 -- 172.21.15.43:0/2293131510 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f146c031c90 con 0x7f1474108f60 2024-09-19T05:02:54.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.513+0000 7f1472000640 1 -- 172.21.15.43:0/2293131510 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146c035030 con 0x7f1474108f60 2024-09-19T05:02:54.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.513+0000 7f147ac00640 1 -- 172.21.15.43:0/2293131510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f147419f730 con 0x7f1474108f60 2024-09-19T05:02:54.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.513+0000 7f147ac00640 1 -- 172.21.15.43:0/2293131510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14741a7070 con 0x7f1474108f60 2024-09-19T05:02:54.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.514+0000 7f147ac00640 1 -- 172.21.15.43:0/2293131510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1474105310 con 0x7f1474108f60 2024-09-19T05:02:54.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.516+0000 7f1472000640 1 -- 172.21.15.43:0/2293131510 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f146c042070 con 0x7f1474108f60 2024-09-19T05:02:54.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.517+0000 7f1472000640 1 --2- 172.21.15.43:0/2293131510 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f145c0a3350 0x7f145c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:54.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.517+0000 7f1472000640 1 -- 172.21.15.43:0/2293131510 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f146c0eff00 con 0x7f1474108f60 2024-09-19T05:02:54.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.517+0000 7f1479800640 1 --2- 172.21.15.43:0/2293131510 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f145c0a3350 0x7f145c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:54.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.518+0000 7f1479800640 1 --2- 172.21.15.43:0/2293131510 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f145c0a3350 0x7f145c0a5810 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1468005fd0 tx=0x7f1468009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:54.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.521+0000 7f1472000640 1 -- 172.21.15.43:0/2293131510 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f146c0bc5e0 con 0x7f1474108f60 2024-09-19T05:02:54.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.709+0000 7f147ac00640 1 -- 172.21.15.43:0/2293131510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f14741a7240 con 0x7f1474108f60 2024-09-19T05:02:54.711 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:54 smithi043 ceph-mon[30800]: pgmap v306: 97 pgs: 97 active+clean; 626 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 KiB/s wr, 21 op/s 2024-09-19T05:02:54.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.711+0000 7f1472000640 1 -- 172.21.15.43:0/2293131510 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22237 (secure 0 0 0) 0x7f146c0c08e0 con 0x7f1474108f60 2024-09-19T05:02:54.713 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:02:54.713 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":83,"fsid":"1456a810-7643-11ef-bceb-c7b262605968","created":"2024-09-19T04:54:14.706644+0000","modified":"2024-09-19T05:02:25.462897+0000","last_up_change":"2024-09-19T05:01:27.348740+0000","last_in_change":"2024-09-19T05:01:03.396477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:57:27.642705+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:02:20.685123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:02:23.393596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"81a317a8-7ec8-45c3-bc10-7da023283635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6803","nonce":825351486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6805","nonce":825351486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6809","nonce":825351486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6807","nonce":825351486}]},"public_addr":"172.21.15.43:6803/825351486","cluster_addr":"172.21.15.43:6805/825351486","heartbeat_back_addr":"172.21.15.43:6809/825351486","heartbeat_front_addr":"172.21.15.43:6807/825351486","state":["exists","up"]},{"osd":1,"uuid":"e96fcfac-4fff-4a36-b38b-aebf57842740","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6801","nonce":2140095130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6803","nonce":2140095130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6807","nonce":2140095130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6805","nonce":2140095130}]},"public_addr":"172.21.15.96:6801/2140095130","cluster_addr":"172.21.15.96:6803/2140095130","heartbeat_back_addr":"172.21.15.96:6807/2140095130","heartbeat_front_addr":"172.21.15.96:6805/2140095130","state":["exists","up"]},{"osd":2,"uuid":"b659869e-114a-4f56-8aac-a2ed602b4058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6801","nonce":2047147996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6803","nonce":2047147996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6807","nonce":2047147996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6805","nonce":2047147996}]},"public_addr":"172.21.15.151:6801/2047147996","cluster_addr":"172.21.15.151:6803/2047147996","heartbeat_back_addr":"172.21.15.151:6807/2047147996","heartbeat_front_addr":"172.21.15.151:6805/2047147996","state":["exists","up"]},{"osd":3,"uuid":"7be64b2c-e45c-493d-a610-4ce41196061f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6811","nonce":1734673762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6813","nonce":1734673762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6817","nonce":1734673762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6815","nonce":1734673762}]},"public_addr":"172.21.15.43:6811/1734673762","cluster_addr":"172.21.15.43:6813/1734673762","heartbeat_back_addr":"172.21.15.43:6817/1734673762","heartbeat_front_addr":"172.21.15.43:6815/1734673762","state":["exists","up"]},{"osd":4,"uuid":"196d5af1-a6e5-496c-8933-4e58892f4360","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6809","nonce":3184785362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6811","nonce":3184785362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6815","nonce":3184785362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6813","nonce":3184785362}]},"public_addr":"172.21.15.96:6809/3184785362","cluster_addr":"172.21.15.96:6811/3184785362","heartbeat_back_addr":"172.21.15.96:6815/3184785362","heartbeat_front_addr":"172.21.15.96:6813/3184785362","state":["exists","up"]},{"osd":5,"uuid":"5492d474-8509-4865-bc2e-d08e31558a75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6809","nonce":2878451924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6811","nonce":2878451924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6815","nonce":2878451924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6813","nonce":2878451924}]},"public_addr":"172.21.15.151:6809/2878451924","cluster_addr":"172.21.15.151:6811/2878451924","heartbeat_back_addr":"172.21.15.151:6815/2878451924","heartbeat_front_addr":"172.21.15.151:6813/2878451924","state":["exists","up"]},{"osd":6,"uuid":"fb8457ed-7bb9-4435-9d15-c1cbfbddfcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6819","nonce":2008630290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6821","nonce":2008630290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6825","nonce":2008630290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6823","nonce":2008630290}]},"public_addr":"172.21.15.43:6819/2008630290","cluster_addr":"172.21.15.43:6821/2008630290","heartbeat_back_addr":"172.21.15.43:6825/2008630290","heartbeat_front_addr":"172.21.15.43:6823/2008630290","state":["exists","up"]},{"osd":7,"uuid":"02046db8-30de-4e92-b25a-1f07d6190e91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6817","nonce":3588713275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6819","nonce":3588713275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6823","nonce":3588713275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6821","nonce":3588713275}]},"public_addr":"172.21.15.96:6817/3588713275","cluster_addr":"172.21.15.96:6819/3588713275","heartbeat_back_addr":"172.21.15.96:6823/3588713275","heartbeat_front_addr":"172.21.15.96:6821/3588713275","state":["exists","up"]},{"osd":8,"uuid":"a5bb4218-51f3-4044-b43e-ebfc764f49ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6817","nonce":725316393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6819","nonce":725316393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6823","nonce":725316393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6821","nonce":725316393}]},"public_addr":"172.21.15.151:6817/725316393","cluster_addr":"172.21.15.151:6819/725316393","heartbeat_back_addr":"172.21.15.151:6823/725316393","heartbeat_front_addr":"172.21.15.151:6821/725316393","state":["exists","up"]},{"osd":9,"uuid":"6c19f65c-9750-4baa-9d63-7b8bd03e6955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6827","nonce":4053438621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6829","nonce":4053438621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6833","nonce":4053438621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6831","nonce":4053438621}]},"public_addr":"172.21.15.43:6827/4053438621","cluster_addr":"172.21.15.43:6829/4053438621","heartbeat_back_addr":"172.21.15.43:6833/4053438621","heartbeat_front_addr":"172.21.15.43:6831/4053438621","state":["exists","up"]},{"osd":10,"uuid":"a7aa7da3-8eab-47fe-a351-298b6ff75f14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6825","nonce":2403210086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6827","nonce":2403210086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6831","nonce":2403210086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6829","nonce":2403210086}]},"public_addr":"172.21.15.96:6825/2403210086","cluster_addr":"172.21.15.96:6827/2403210086","heartbeat_back_addr":"172.21.15.96:6831/2403210086","heartbeat_front_addr":"172.21.15.96:6829/2403210086","state":["exists","up"]},{"osd":11,"uuid":"760b6afc-253d-44dd-88e9-dcc97f67c9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6825","nonce":292202881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6827","nonce":292202881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6831","nonce":292202881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6829","nonce":292202881}]},"public_addr":"172.21.15.151:6825/292202881","cluster_addr":"172.21.15.151:6827/292202881","heartbeat_back_addr":"172.21.15.151:6831/292202881","heartbeat_front_addr":"172.21.15.151:6829/292202881","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:56:34.716991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:56:59.321174+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:57:24.281696+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:57:53.443880+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:58:19.205873+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:58:44.504873+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:13.650731+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:39.137876+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:04.579893+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:34.337312+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:59.434658+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:01:24.930529+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/898247657":"2024-09-20T04:55:07.037117+0000","172.21.15.43:6801/3241799377":"2024-09-20T04:55:07.037117+0000","172.21.15.43:6800/3241799377":"2024-09-20T04:55:07.037117+0000","172.21.15.43:0/3256350596":"2024-09-20T04:55:07.037117+0000","172.21.15.43:6801/587463881":"2024-09-20T04:54:39.548065+0000","172.21.15.43:6800/587463881":"2024-09-20T04:54:39.548065+0000","172.21.15.43:0/4271788315":"2024-09-20T04:55:07.037117+0000","172.21.15.43:0/6132665":"2024-09-20T04:54:39.548065+0000","172.21.15.43:0/632163478":"2024-09-20T04:54:39.548065+0000","172.21.15.43:0/330854185":"2024-09-20T04:54:39.548065+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:02:54.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.713+0000 7f147ac00640 1 -- 172.21.15.43:0/2293131510 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f145c0a3350 msgr2=0x7f145c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:54.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.713+0000 7f147ac00640 1 --2- 172.21.15.43:0/2293131510 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f145c0a3350 0x7f145c0a5810 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1468005fd0 tx=0x7f1468009500 comp rx=0 tx=0).stop 2024-09-19T05:02:54.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.713+0000 7f147ac00640 1 -- 172.21.15.43:0/2293131510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1474108f60 msgr2=0x7f147419edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:54.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.713+0000 7f147ac00640 1 --2- 172.21.15.43:0/2293131510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1474108f60 0x7f147419edc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f146c004770 tx=0x7f146c003b90 comp rx=0 tx=0).stop 2024-09-19T05:02:54.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.714+0000 7f147ac00640 1 -- 172.21.15.43:0/2293131510 shutdown_connections 2024-09-19T05:02:54.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.714+0000 7f147ac00640 1 --2- 172.21.15.43:0/2293131510 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f145c0a3350 0x7f145c0a5810 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:54.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.714+0000 7f147ac00640 1 --2- 172.21.15.43:0/2293131510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1474108f60 0x7f147419edc0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:54.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.714+0000 7f147ac00640 1 --2- 172.21.15.43:0/2293131510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14741047e0 0x7f147419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:54.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.714+0000 7f147ac00640 1 --2- 172.21.15.43:0/2293131510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1474103e30 0x7f147419a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:54.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.714+0000 7f147ac00640 1 -- 172.21.15.43:0/2293131510 >> 172.21.15.43:0/2293131510 conn(0x7f14740ff910 msgr2=0x7f1474101d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:54.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.714+0000 7f147ac00640 1 -- 172.21.15.43:0/2293131510 shutdown_connections 2024-09-19T05:02:54.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:54.714+0000 7f147ac00640 1 -- 172.21.15.43:0/2293131510 wait complete. 2024-09-19T05:02:54.904 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:54 smithi151 ceph-mon[35425]: pgmap v306: 97 pgs: 97 active+clean; 626 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 KiB/s wr, 21 op/s 2024-09-19T05:02:54.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:54 smithi096 ceph-mon[35085]: pgmap v306: 97 pgs: 97 active+clean; 626 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 KiB/s wr, 21 op/s 2024-09-19T05:02:55.622 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T05:02:55.622 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T05:02:55.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:55 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/2293131510' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:02:55.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:55 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/2293131510' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:02:55.999 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:02:56.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:55 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/2293131510' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:02:56.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.571+0000 7ff5b5200640 1 -- 172.21.15.43:0/301512179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5b010c180 msgr2=0x7ff5b010e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:56.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.571+0000 7ff5b5200640 1 --2- 172.21.15.43:0/301512179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5b010c180 0x7ff5b010e620 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8009860 tx=0x7ff5a802a3d0 comp rx=0 tx=0).stop 2024-09-19T05:02:56.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.571+0000 7ff5b5200640 1 -- 172.21.15.43:0/301512179 shutdown_connections 2024-09-19T05:02:56.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.571+0000 7ff5b5200640 1 --2- 172.21.15.43:0/301512179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5b010c180 0x7ff5b010e620 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:56.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.571+0000 7ff5b5200640 1 --2- 172.21.15.43:0/301512179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5b0103670 0x7ff5b010bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:56.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.571+0000 7ff5b5200640 1 --2- 172.21.15.43:0/301512179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5b0102d50 0x7ff5b0103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:56.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.571+0000 7ff5b5200640 1 -- 172.21.15.43:0/301512179 >> 172.21.15.43:0/301512179 conn(0x7ff5b00fc7d0 msgr2=0x7ff5b00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:56.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.571+0000 7ff5b5200640 1 -- 172.21.15.43:0/301512179 shutdown_connections 2024-09-19T05:02:56.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.571+0000 7ff5b5200640 1 -- 172.21.15.43:0/301512179 wait complete. 2024-09-19T05:02:56.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.572+0000 7ff5b5200640 1 Processor -- start 2024-09-19T05:02:56.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.572+0000 7ff5b5200640 1 -- start start 2024-09-19T05:02:56.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.573+0000 7ff5b5200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5b0102d50 0x7ff5b01009b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:56.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.573+0000 7ff5b5200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5b0103670 0x7ff5b0100ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:56.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.573+0000 7ff5b5200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5b010c180 0x7ff5b0101430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:56.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.573+0000 7ff5b5200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b0111f90 con 0x7ff5b0103670 2024-09-19T05:02:56.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.573+0000 7ff5b5200640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b0111e10 con 0x7ff5b010c180 2024-09-19T05:02:56.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.573+0000 7ff5b5200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b0112110 con 0x7ff5b0102d50 2024-09-19T05:02:56.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.573+0000 7ff5a5400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5b0103670 0x7ff5b0100ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:56.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.573+0000 7ff5a5400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5b0103670 0x7ff5b0100ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59112/0 (socket says 172.21.15.43:59112) 2024-09-19T05:02:56.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.573+0000 7ff5a5400640 1 -- 172.21.15.43:0/3013813366 learned_addr learned my addr 172.21.15.43:0/3013813366 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:02:56.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.573+0000 7ff5ae800640 1 --2- 172.21.15.43:0/3013813366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5b010c180 0x7ff5b0101430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:56.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.573+0000 7ff5ade00640 1 --2- 172.21.15.43:0/3013813366 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5b0102d50 0x7ff5b01009b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:56.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.574+0000 7ff5a5400640 1 -- 172.21.15.43:0/3013813366 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5b0102d50 msgr2=0x7ff5b01009b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:56.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.574+0000 7ff5a5400640 1 --2- 172.21.15.43:0/3013813366 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5b0102d50 0x7ff5b01009b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:56.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.574+0000 7ff5a5400640 1 -- 172.21.15.43:0/3013813366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5b010c180 msgr2=0x7ff5b0101430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:56.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.574+0000 7ff5a5400640 1 --2- 172.21.15.43:0/3013813366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5b010c180 0x7ff5b0101430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:56.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.574+0000 7ff5a5400640 1 -- 172.21.15.43:0/3013813366 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5b01a68d0 con 0x7ff5b0103670 2024-09-19T05:02:56.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.574+0000 7ff5ae800640 1 --2- 172.21.15.43:0/3013813366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5b010c180 0x7ff5b0101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:02:56.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.574+0000 7ff5a5400640 1 --2- 172.21.15.43:0/3013813366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5b0103670 0x7ff5b0100ef0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff59800e9f0 tx=0x7ff59800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:56.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.574+0000 7ff5a7400640 1 -- 172.21.15.43:0/3013813366 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59800ccc0 con 0x7ff5b0103670 2024-09-19T05:02:56.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.574+0000 7ff5a7400640 1 -- 172.21.15.43:0/3013813366 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff598010910 con 0x7ff5b0103670 2024-09-19T05:02:56.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.574+0000 7ff5b5200640 1 -- 172.21.15.43:0/3013813366 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5b01a6bc0 con 0x7ff5b0103670 2024-09-19T05:02:56.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.574+0000 7ff5a7400640 1 -- 172.21.15.43:0/3013813366 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff598018920 con 0x7ff5b0103670 2024-09-19T05:02:56.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.574+0000 7ff5b5200640 1 -- 172.21.15.43:0/3013813366 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5b01a7150 con 0x7ff5b0103670 2024-09-19T05:02:56.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.575+0000 7ff5b5200640 1 -- 172.21.15.43:0/3013813366 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5b0103130 con 0x7ff5b0103670 2024-09-19T05:02:56.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.577+0000 7ff5a7400640 1 -- 172.21.15.43:0/3013813366 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7ff598010430 con 0x7ff5b0103670 2024-09-19T05:02:56.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.579+0000 7ff5a7400640 1 --2- 172.21.15.43:0/3013813366 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff5940a3350 0x7ff5940a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:56.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.579+0000 7ff5a7400640 1 -- 172.21.15.43:0/3013813366 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7ff5980cfb20 con 0x7ff5b0103670 2024-09-19T05:02:56.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.579+0000 7ff5ade00640 1 --2- 172.21.15.43:0/3013813366 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff5940a3350 0x7ff5940a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:56.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.579+0000 7ff5ade00640 1 --2- 172.21.15.43:0/3013813366 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff5940a3350 0x7ff5940a5810 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0009a60 tx=0x7ff5a0010040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:56.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.583+0000 7ff5a7400640 1 -- 172.21.15.43:0/3013813366 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff59809c140 con 0x7ff5b0103670 2024-09-19T05:02:56.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.766+0000 7ff5b5200640 1 -- 172.21.15.43:0/3013813366 --> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff5b0062630 con 0x7ff5940a3350 2024-09-19T05:02:56.768 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:56 smithi043 ceph-mon[30800]: pgmap v307: 97 pgs: 97 active+clean; 626 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.5 KiB/s wr, 11 op/s 2024-09-19T05:02:56.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.775+0000 7ff5a7400640 1 -- 172.21.15.43:0/3013813366 <== mgr.14150 v2:172.21.15.43:6800/3167108925 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258910 (secure 0 0 0) 0x7ff5b0062630 con 0x7ff5940a3350 2024-09-19T05:02:56.780 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:02:56.781 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-19T05:02:56.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.779+0000 7ff5b5200640 1 -- 172.21.15.43:0/3013813366 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff5940a3350 msgr2=0x7ff5940a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:56.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.779+0000 7ff5b5200640 1 --2- 172.21.15.43:0/3013813366 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff5940a3350 0x7ff5940a5810 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0009a60 tx=0x7ff5a0010040 comp rx=0 tx=0).stop 2024-09-19T05:02:56.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.779+0000 7ff5b5200640 1 -- 172.21.15.43:0/3013813366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5b0103670 msgr2=0x7ff5b0100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:56.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.779+0000 7ff5b5200640 1 --2- 172.21.15.43:0/3013813366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5b0103670 0x7ff5b0100ef0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff59800e9f0 tx=0x7ff59800eeb0 comp rx=0 tx=0).stop 2024-09-19T05:02:56.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.779+0000 7ff5b5200640 1 -- 172.21.15.43:0/3013813366 shutdown_connections 2024-09-19T05:02:56.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.779+0000 7ff5b5200640 1 --2- 172.21.15.43:0/3013813366 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff5940a3350 0x7ff5940a5810 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:56.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.779+0000 7ff5b5200640 1 --2- 172.21.15.43:0/3013813366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5b010c180 0x7ff5b0101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:56.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.779+0000 7ff5b5200640 1 --2- 172.21.15.43:0/3013813366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5b0103670 0x7ff5b0100ef0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:56.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.779+0000 7ff5b5200640 1 --2- 172.21.15.43:0/3013813366 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5b0102d50 0x7ff5b01009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:56.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.779+0000 7ff5b5200640 1 -- 172.21.15.43:0/3013813366 >> 172.21.15.43:0/3013813366 conn(0x7ff5b00fc7d0 msgr2=0x7ff5b00fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:56.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.779+0000 7ff5b5200640 1 -- 172.21.15.43:0/3013813366 shutdown_connections 2024-09-19T05:02:56.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:56.779+0000 7ff5b5200640 1 -- 172.21.15.43:0/3013813366 wait complete. 2024-09-19T05:02:56.904 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:56 smithi151 ceph-mon[35425]: pgmap v307: 97 pgs: 97 active+clean; 626 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.5 KiB/s wr, 11 op/s 2024-09-19T05:02:56.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:56 smithi096 ceph-mon[35085]: pgmap v307: 97 pgs: 97 active+clean; 626 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.5 KiB/s wr, 11 op/s 2024-09-19T05:02:57.545 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-09-19T05:02:55.146658+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":640799,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":293,"num_read_kb":253,"num_write":414,"num_write_kb":3054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":340,"ondisk_log_size":340,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368752,"kb_used_data":11184,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124573072,"statfs":{"total":1151940427776,"available":1151562825728,"internally_reserved":0,"allocated":11452416,"data_stored":7134573,"data_compressed":108990,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":42397,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":95,"num_write":28,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009664"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.428048+0000","last_change":"2024-09-19T05:02:24.426572+0000","last_active":"2024-09-19T05:02:26.428048+0000","last_peered":"2024-09-19T05:02:26.428048+0000","last_clean":"2024-09-19T05:02:26.428048+0000","last_became_active":"2024-09-19T05:02:24.426241+0000","last_became_peered":"2024-09-19T05:02:24.426241+0000","last_unstale":"2024-09-19T05:02:26.428048+0000","last_undegraded":"2024-09-19T05:02:26.428048+0000","last_fullsized":"2024-09-19T05:02:26.428048+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:42.511495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.429332+0000","last_change":"2024-09-19T05:02:24.427263+0000","last_active":"2024-09-19T05:02:26.429332+0000","last_peered":"2024-09-19T05:02:26.429332+0000","last_clean":"2024-09-19T05:02:26.429332+0000","last_became_active":"2024-09-19T05:02:24.426928+0000","last_became_peered":"2024-09-19T05:02:24.426928+0000","last_unstale":"2024-09-19T05:02:26.429332+0000","last_undegraded":"2024-09-19T05:02:26.429332+0000","last_fullsized":"2024-09-19T05:02:26.429332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:03.177020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.429185+0000","last_change":"2024-09-19T05:02:24.432709+0000","last_active":"2024-09-19T05:02:26.429185+0000","last_peered":"2024-09-19T05:02:26.429185+0000","last_clean":"2024-09-19T05:02:26.429185+0000","last_became_active":"2024-09-19T05:02:24.431114+0000","last_became_peered":"2024-09-19T05:02:24.431114+0000","last_unstale":"2024-09-19T05:02:26.429185+0000","last_undegraded":"2024-09-19T05:02:26.429185+0000","last_fullsized":"2024-09-19T05:02:26.429185+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:57.244919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.422150+0000","last_change":"2024-09-19T05:02:24.432174+0000","last_active":"2024-09-19T05:02:26.422150+0000","last_peered":"2024-09-19T05:02:26.422150+0000","last_clean":"2024-09-19T05:02:26.422150+0000","last_became_active":"2024-09-19T05:02:24.431445+0000","last_became_peered":"2024-09-19T05:02:24.431445+0000","last_unstale":"2024-09-19T05:02:26.422150+0000","last_undegraded":"2024-09-19T05:02:26.422150+0000","last_fullsized":"2024-09-19T05:02:26.422150+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:01:35.959119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.421670+0000","last_change":"2024-09-19T05:02:24.419992+0000","last_active":"2024-09-19T05:02:26.421670+0000","last_peered":"2024-09-19T05:02:26.421670+0000","last_clean":"2024-09-19T05:02:26.421670+0000","last_became_active":"2024-09-19T05:02:24.419643+0000","last_became_peered":"2024-09-19T05:02:24.419643+0000","last_unstale":"2024-09-19T05:02:26.421670+0000","last_undegraded":"2024-09-19T05:02:26.421670+0000","last_fullsized":"2024-09-19T05:02:26.421670+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:54:40.548216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.421437+0000","last_change":"2024-09-19T05:02:24.434106+0000","last_active":"2024-09-19T05:02:26.421437+0000","last_peered":"2024-09-19T05:02:26.421437+0000","last_clean":"2024-09-19T05:02:26.421437+0000","last_became_active":"2024-09-19T05:02:24.433124+0000","last_became_peered":"2024-09-19T05:02:24.433124+0000","last_unstale":"2024-09-19T05:02:26.421437+0000","last_undegraded":"2024-09-19T05:02:26.421437+0000","last_fullsized":"2024-09-19T05:02:26.421437+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:09:46.502506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.424981+0000","last_change":"2024-09-19T05:02:24.429640+0000","last_active":"2024-09-19T05:02:26.424981+0000","last_peered":"2024-09-19T05:02:26.424981+0000","last_clean":"2024-09-19T05:02:26.424981+0000","last_became_active":"2024-09-19T05:02:24.429326+0000","last_became_peered":"2024-09-19T05:02:24.429326+0000","last_unstale":"2024-09-19T05:02:26.424981+0000","last_undegraded":"2024-09-19T05:02:26.424981+0000","last_fullsized":"2024-09-19T05:02:26.424981+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:27.330449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.415057+0000","last_change":"2024-09-19T05:02:24.430933+0000","last_active":"2024-09-19T05:02:26.415057+0000","last_peered":"2024-09-19T05:02:26.415057+0000","last_clean":"2024-09-19T05:02:26.415057+0000","last_became_active":"2024-09-19T05:02:24.430478+0000","last_became_peered":"2024-09-19T05:02:24.430478+0000","last_unstale":"2024-09-19T05:02:26.415057+0000","last_undegraded":"2024-09-19T05:02:26.415057+0000","last_fullsized":"2024-09-19T05:02:26.415057+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:45:22.171049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.415974+0000","last_change":"2024-09-19T05:02:24.433207+0000","last_active":"2024-09-19T05:02:26.415974+0000","last_peered":"2024-09-19T05:02:26.415974+0000","last_clean":"2024-09-19T05:02:26.415974+0000","last_became_active":"2024-09-19T05:02:24.432882+0000","last_became_peered":"2024-09-19T05:02:24.432882+0000","last_unstale":"2024-09-19T05:02:26.415974+0000","last_undegraded":"2024-09-19T05:02:26.415974+0000","last_fullsized":"2024-09-19T05:02:26.415974+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:06:42.748227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.421944+0000","last_change":"2024-09-19T05:02:24.433825+0000","last_active":"2024-09-19T05:02:26.421944+0000","last_peered":"2024-09-19T05:02:26.421944+0000","last_clean":"2024-09-19T05:02:26.421944+0000","last_became_active":"2024-09-19T05:02:24.433088+0000","last_became_peered":"2024-09-19T05:02:24.433088+0000","last_unstale":"2024-09-19T05:02:26.421944+0000","last_undegraded":"2024-09-19T05:02:26.421944+0000","last_fullsized":"2024-09-19T05:02:26.421944+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:12:53.039028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.425611+0000","last_change":"2024-09-19T05:02:24.427661+0000","last_active":"2024-09-19T05:02:26.425611+0000","last_peered":"2024-09-19T05:02:26.425611+0000","last_clean":"2024-09-19T05:02:26.425611+0000","last_became_active":"2024-09-19T05:02:24.427093+0000","last_became_peered":"2024-09-19T05:02:24.427093+0000","last_unstale":"2024-09-19T05:02:26.425611+0000","last_undegraded":"2024-09-19T05:02:26.425611+0000","last_fullsized":"2024-09-19T05:02:26.425611+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:43.091041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.422330+0000","last_change":"2024-09-19T05:02:24.430820+0000","last_active":"2024-09-19T05:02:26.422330+0000","last_peered":"2024-09-19T05:02:26.422330+0000","last_clean":"2024-09-19T05:02:26.422330+0000","last_became_active":"2024-09-19T05:02:24.429945+0000","last_became_peered":"2024-09-19T05:02:24.429945+0000","last_unstale":"2024-09-19T05:02:26.422330+0000","last_undegraded":"2024-09-19T05:02:26.422330+0000","last_fullsized":"2024-09-19T05:02:26.422330+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:21:59.400725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.424811+0000","last_change":"2024-09-19T05:02:24.428695+0000","last_active":"2024-09-19T05:02:26.424811+0000","last_peered":"2024-09-19T05:02:26.424811+0000","last_clean":"2024-09-19T05:02:26.424811+0000","last_became_active":"2024-09-19T05:02:24.426385+0000","last_became_peered":"2024-09-19T05:02:24.426385+0000","last_unstale":"2024-09-19T05:02:26.424811+0000","last_undegraded":"2024-09-19T05:02:26.424811+0000","last_fullsized":"2024-09-19T05:02:26.424811+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:44:00.893313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.428369+0000","last_change":"2024-09-19T05:02:24.424935+0000","last_active":"2024-09-19T05:02:26.428369+0000","last_peered":"2024-09-19T05:02:26.428369+0000","last_clean":"2024-09-19T05:02:26.428369+0000","last_became_active":"2024-09-19T05:02:24.424219+0000","last_became_peered":"2024-09-19T05:02:24.424219+0000","last_unstale":"2024-09-19T05:02:26.428369+0000","last_undegraded":"2024-09-19T05:02:26.428369+0000","last_fullsized":"2024-09-19T05:02:26.428369+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:00.890778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.427577+0000","last_change":"2024-09-19T05:02:24.425407+0000","last_active":"2024-09-19T05:02:26.427577+0000","last_peered":"2024-09-19T05:02:26.427577+0000","last_clean":"2024-09-19T05:02:26.427577+0000","last_became_active":"2024-09-19T05:02:24.425053+0000","last_became_peered":"2024-09-19T05:02:24.425053+0000","last_unstale":"2024-09-19T05:02:26.427577+0000","last_undegraded":"2024-09-19T05:02:26.427577+0000","last_fullsized":"2024-09-19T05:02:26.427577+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:54:10.024183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.415130+0000","last_change":"2024-09-19T05:02:24.427888+0000","last_active":"2024-09-19T05:02:26.415130+0000","last_peered":"2024-09-19T05:02:26.415130+0000","last_clean":"2024-09-19T05:02:26.415130+0000","last_became_active":"2024-09-19T05:02:24.427589+0000","last_became_peered":"2024-09-19T05:02:24.427589+0000","last_unstale":"2024-09-19T05:02:26.415130+0000","last_undegraded":"2024-09-19T05:02:26.415130+0000","last_fullsized":"2024-09-19T05:02:26.415130+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:02:02.654416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.427843+0000","last_change":"2024-09-19T05:02:24.426654+0000","last_active":"2024-09-19T05:02:26.427843+0000","last_peered":"2024-09-19T05:02:26.427843+0000","last_clean":"2024-09-19T05:02:26.427843+0000","last_became_active":"2024-09-19T05:02:24.426352+0000","last_became_peered":"2024-09-19T05:02:24.426352+0000","last_unstale":"2024-09-19T05:02:26.427843+0000","last_undegraded":"2024-09-19T05:02:26.427843+0000","last_fullsized":"2024-09-19T05:02:26.427843+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:20.055222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.423666+0000","last_change":"2024-09-19T05:02:24.432178+0000","last_active":"2024-09-19T05:02:26.423666+0000","last_peered":"2024-09-19T05:02:26.423666+0000","last_clean":"2024-09-19T05:02:26.423666+0000","last_became_active":"2024-09-19T05:02:24.429058+0000","last_became_peered":"2024-09-19T05:02:24.429058+0000","last_unstale":"2024-09-19T05:02:26.423666+0000","last_undegraded":"2024-09-19T05:02:26.423666+0000","last_fullsized":"2024-09-19T05:02:26.423666+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:08.001134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.417818+0000","last_change":"2024-09-19T05:02:24.428202+0000","last_active":"2024-09-19T05:02:26.417818+0000","last_peered":"2024-09-19T05:02:26.417818+0000","last_clean":"2024-09-19T05:02:26.417818+0000","last_became_active":"2024-09-19T05:02:24.427741+0000","last_became_peered":"2024-09-19T05:02:24.427741+0000","last_unstale":"2024-09-19T05:02:26.417818+0000","last_undegraded":"2024-09-19T05:02:26.417818+0000","last_fullsized":"2024-09-19T05:02:26.417818+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:42:51.076056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.426673+0000","last_change":"2024-09-19T05:02:24.425007+0000","last_active":"2024-09-19T05:02:26.426673+0000","last_peered":"2024-09-19T05:02:26.426673+0000","last_clean":"2024-09-19T05:02:26.426673+0000","last_became_active":"2024-09-19T05:02:24.424612+0000","last_became_peered":"2024-09-19T05:02:24.424612+0000","last_unstale":"2024-09-19T05:02:26.426673+0000","last_undegraded":"2024-09-19T05:02:26.426673+0000","last_fullsized":"2024-09-19T05:02:26.426673+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:36:06.115757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.424994+0000","last_change":"2024-09-19T05:02:24.428724+0000","last_active":"2024-09-19T05:02:26.424994+0000","last_peered":"2024-09-19T05:02:26.424994+0000","last_clean":"2024-09-19T05:02:26.424994+0000","last_became_active":"2024-09-19T05:02:24.427730+0000","last_became_peered":"2024-09-19T05:02:24.427730+0000","last_unstale":"2024-09-19T05:02:26.424994+0000","last_undegraded":"2024-09-19T05:02:26.424994+0000","last_fullsized":"2024-09-19T05:02:26.424994+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:15:25.466553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.415384+0000","last_change":"2024-09-19T05:02:24.426238+0000","last_active":"2024-09-19T05:02:26.415384+0000","last_peered":"2024-09-19T05:02:26.415384+0000","last_clean":"2024-09-19T05:02:26.415384+0000","last_became_active":"2024-09-19T05:02:24.425915+0000","last_became_peered":"2024-09-19T05:02:24.425915+0000","last_unstale":"2024-09-19T05:02:26.415384+0000","last_undegraded":"2024-09-19T05:02:26.415384+0000","last_fullsized":"2024-09-19T05:02:26.415384+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:37:18.444042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.415085+0000","last_change":"2024-09-19T05:02:24.425726+0000","last_active":"2024-09-19T05:02:26.415085+0000","last_peered":"2024-09-19T05:02:26.415085+0000","last_clean":"2024-09-19T05:02:26.415085+0000","last_became_active":"2024-09-19T05:02:24.425409+0000","last_became_peered":"2024-09-19T05:02:24.425409+0000","last_unstale":"2024-09-19T05:02:26.415085+0000","last_undegraded":"2024-09-19T05:02:26.415085+0000","last_fullsized":"2024-09-19T05:02:26.415085+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:12.068304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.425175+0000","last_change":"2024-09-19T05:02:24.425267+0000","last_active":"2024-09-19T05:02:26.425175+0000","last_peered":"2024-09-19T05:02:26.425175+0000","last_clean":"2024-09-19T05:02:26.425175+0000","last_became_active":"2024-09-19T05:02:24.424933+0000","last_became_peered":"2024-09-19T05:02:24.424933+0000","last_unstale":"2024-09-19T05:02:26.425175+0000","last_undegraded":"2024-09-19T05:02:26.425175+0000","last_fullsized":"2024-09-19T05:02:26.425175+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:51:55.274870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.415742+0000","last_change":"2024-09-19T05:02:24.427909+0000","last_active":"2024-09-19T05:02:26.415742+0000","last_peered":"2024-09-19T05:02:26.415742+0000","last_clean":"2024-09-19T05:02:26.415742+0000","last_became_active":"2024-09-19T05:02:24.425515+0000","last_became_peered":"2024-09-19T05:02:24.425515+0000","last_unstale":"2024-09-19T05:02:26.415742+0000","last_undegraded":"2024-09-19T05:02:26.415742+0000","last_fullsized":"2024-09-19T05:02:26.415742+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:12:22.390754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.418042+0000","last_change":"2024-09-19T05:02:24.426201+0000","last_active":"2024-09-19T05:02:26.418042+0000","last_peered":"2024-09-19T05:02:26.418042+0000","last_clean":"2024-09-19T05:02:26.418042+0000","last_became_active":"2024-09-19T05:02:24.425890+0000","last_became_peered":"2024-09-19T05:02:24.425890+0000","last_unstale":"2024-09-19T05:02:26.418042+0000","last_undegraded":"2024-09-19T05:02:26.418042+0000","last_fullsized":"2024-09-19T05:02:26.418042+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:36.680002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.427337+0000","last_change":"2024-09-19T05:02:24.416037+0000","last_active":"2024-09-19T05:02:26.427337+0000","last_peered":"2024-09-19T05:02:26.427337+0000","last_clean":"2024-09-19T05:02:26.427337+0000","last_became_active":"2024-09-19T05:02:24.415535+0000","last_became_peered":"2024-09-19T05:02:24.415535+0000","last_unstale":"2024-09-19T05:02:26.427337+0000","last_undegraded":"2024-09-19T05:02:26.427337+0000","last_fullsized":"2024-09-19T05:02:26.427337+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:14.756410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.422856+0000","last_change":"2024-09-19T05:02:24.431137+0000","last_active":"2024-09-19T05:02:26.422856+0000","last_peered":"2024-09-19T05:02:26.422856+0000","last_clean":"2024-09-19T05:02:26.422856+0000","last_became_active":"2024-09-19T05:02:24.424962+0000","last_became_peered":"2024-09-19T05:02:24.424962+0000","last_unstale":"2024-09-19T05:02:26.422856+0000","last_undegraded":"2024-09-19T05:02:26.422856+0000","last_fullsized":"2024-09-19T05:02:26.422856+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:36:39.192239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.429147+0000","last_change":"2024-09-19T05:02:24.423776+0000","last_active":"2024-09-19T05:02:26.429147+0000","last_peered":"2024-09-19T05:02:26.429147+0000","last_clean":"2024-09-19T05:02:26.429147+0000","last_became_active":"2024-09-19T05:02:24.423446+0000","last_became_peered":"2024-09-19T05:02:24.423446+0000","last_unstale":"2024-09-19T05:02:26.429147+0000","last_undegraded":"2024-09-19T05:02:26.429147+0000","last_fullsized":"2024-09-19T05:02:26.429147+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:47:02.747049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.417615+0000","last_change":"2024-09-19T05:02:24.420534+0000","last_active":"2024-09-19T05:02:26.417615+0000","last_peered":"2024-09-19T05:02:26.417615+0000","last_clean":"2024-09-19T05:02:26.417615+0000","last_became_active":"2024-09-19T05:02:24.419980+0000","last_became_peered":"2024-09-19T05:02:24.419980+0000","last_unstale":"2024-09-19T05:02:26.417615+0000","last_undegraded":"2024-09-19T05:02:26.417615+0000","last_fullsized":"2024-09-19T05:02:26.417615+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:33:49.537258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.417972+0000","last_change":"2024-09-19T05:02:24.424859+0000","last_active":"2024-09-19T05:02:26.417972+0000","last_peered":"2024-09-19T05:02:26.417972+0000","last_clean":"2024-09-19T05:02:26.417972+0000","last_became_active":"2024-09-19T05:02:24.424514+0000","last_became_peered":"2024-09-19T05:02:24.424514+0000","last_unstale":"2024-09-19T05:02:26.417972+0000","last_undegraded":"2024-09-19T05:02:26.417972+0000","last_fullsized":"2024-09-19T05:02:26.417972+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:41:41.929578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.426176+0000","last_change":"2024-09-19T05:02:24.421756+0000","last_active":"2024-09-19T05:02:26.426176+0000","last_peered":"2024-09-19T05:02:26.426176+0000","last_clean":"2024-09-19T05:02:26.426176+0000","last_became_active":"2024-09-19T05:02:24.420910+0000","last_became_peered":"2024-09-19T05:02:24.420910+0000","last_unstale":"2024-09-19T05:02:26.426176+0000","last_undegraded":"2024-09-19T05:02:26.426176+0000","last_fullsized":"2024-09-19T05:02:26.426176+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:33.759745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.423804+0000","last_change":"2024-09-19T05:02:24.428514+0000","last_active":"2024-09-19T05:02:26.423804+0000","last_peered":"2024-09-19T05:02:26.423804+0000","last_clean":"2024-09-19T05:02:26.423804+0000","last_became_active":"2024-09-19T05:02:24.428206+0000","last_became_peered":"2024-09-19T05:02:24.428206+0000","last_unstale":"2024-09-19T05:02:26.423804+0000","last_undegraded":"2024-09-19T05:02:26.423804+0000","last_fullsized":"2024-09-19T05:02:26.423804+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:33.045739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:50.528984+0000","last_change":"2024-09-19T05:02:22.026683+0000","last_active":"2024-09-19T05:02:50.528984+0000","last_peered":"2024-09-19T05:02:50.528984+0000","last_clean":"2024-09-19T05:02:50.528984+0000","last_became_active":"2024-09-19T05:02:22.026342+0000","last_became_peered":"2024-09-19T05:02:22.026342+0000","last_unstale":"2024-09-19T05:02:50.528984+0000","last_undegraded":"2024-09-19T05:02:50.528984+0000","last_fullsized":"2024-09-19T05:02:50.528984+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:01:37.462838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.422187+0000","last_change":"2024-09-19T05:02:24.421753+0000","last_active":"2024-09-19T05:02:26.422187+0000","last_peered":"2024-09-19T05:02:26.422187+0000","last_clean":"2024-09-19T05:02:26.422187+0000","last_became_active":"2024-09-19T05:02:24.421455+0000","last_became_peered":"2024-09-19T05:02:24.421455+0000","last_unstale":"2024-09-19T05:02:26.422187+0000","last_undegraded":"2024-09-19T05:02:26.422187+0000","last_fullsized":"2024-09-19T05:02:26.422187+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:06.416550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:37.493191+0000","last_change":"2024-09-19T05:02:22.027188+0000","last_active":"2024-09-19T05:02:37.493191+0000","last_peered":"2024-09-19T05:02:37.493191+0000","last_clean":"2024-09-19T05:02:37.493191+0000","last_became_active":"2024-09-19T05:02:22.026414+0000","last_became_peered":"2024-09-19T05:02:22.026414+0000","last_unstale":"2024-09-19T05:02:37.493191+0000","last_undegraded":"2024-09-19T05:02:37.493191+0000","last_fullsized":"2024-09-19T05:02:37.493191+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:37:21.777913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:36.486962+0000","last_change":"2024-09-19T05:02:22.026599+0000","last_active":"2024-09-19T05:02:36.486962+0000","last_peered":"2024-09-19T05:02:36.486962+0000","last_clean":"2024-09-19T05:02:36.486962+0000","last_became_active":"2024-09-19T05:02:22.024652+0000","last_became_peered":"2024-09-19T05:02:22.024652+0000","last_unstale":"2024-09-19T05:02:36.486962+0000","last_undegraded":"2024-09-19T05:02:36.486962+0000","last_fullsized":"2024-09-19T05:02:36.486962+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:14:32.508333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.415203+0000","last_change":"2024-09-19T05:02:24.419262+0000","last_active":"2024-09-19T05:02:26.415203+0000","last_peered":"2024-09-19T05:02:26.415203+0000","last_clean":"2024-09-19T05:02:26.415203+0000","last_became_active":"2024-09-19T05:02:24.418589+0000","last_became_peered":"2024-09-19T05:02:24.418589+0000","last_unstale":"2024-09-19T05:02:26.415203+0000","last_undegraded":"2024-09-19T05:02:26.415203+0000","last_fullsized":"2024-09-19T05:02:26.415203+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:27.256676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:38.499696+0000","last_change":"2024-09-19T05:02:22.024225+0000","last_active":"2024-09-19T05:02:38.499696+0000","last_peered":"2024-09-19T05:02:38.499696+0000","last_clean":"2024-09-19T05:02:38.499696+0000","last_became_active":"2024-09-19T05:02:22.023882+0000","last_became_peered":"2024-09-19T05:02:22.023882+0000","last_unstale":"2024-09-19T05:02:38.499696+0000","last_undegraded":"2024-09-19T05:02:38.499696+0000","last_fullsized":"2024-09-19T05:02:38.499696+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:29.619634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.423285+0000","last_change":"2024-09-19T05:02:24.436511+0000","last_active":"2024-09-19T05:02:26.423285+0000","last_peered":"2024-09-19T05:02:26.423285+0000","last_clean":"2024-09-19T05:02:26.423285+0000","last_became_active":"2024-09-19T05:02:24.435565+0000","last_became_peered":"2024-09-19T05:02:24.435565+0000","last_unstale":"2024-09-19T05:02:26.423285+0000","last_undegraded":"2024-09-19T05:02:26.423285+0000","last_fullsized":"2024-09-19T05:02:26.423285+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:16:43.779420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:49.527108+0000","last_change":"2024-09-19T05:02:22.024281+0000","last_active":"2024-09-19T05:02:49.527108+0000","last_peered":"2024-09-19T05:02:49.527108+0000","last_clean":"2024-09-19T05:02:49.527108+0000","last_became_active":"2024-09-19T05:02:22.023868+0000","last_became_peered":"2024-09-19T05:02:22.023868+0000","last_unstale":"2024-09-19T05:02:49.527108+0000","last_undegraded":"2024-09-19T05:02:49.527108+0000","last_fullsized":"2024-09-19T05:02:49.527108+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:29:47.597570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.415817+0000","last_change":"2024-09-19T05:02:24.436383+0000","last_active":"2024-09-19T05:02:26.415817+0000","last_peered":"2024-09-19T05:02:26.415817+0000","last_clean":"2024-09-19T05:02:26.415817+0000","last_became_active":"2024-09-19T05:02:24.436062+0000","last_became_peered":"2024-09-19T05:02:24.436062+0000","last_unstale":"2024-09-19T05:02:26.415817+0000","last_undegraded":"2024-09-19T05:02:26.415817+0000","last_fullsized":"2024-09-19T05:02:26.415817+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:48:42.574431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:38.499021+0000","last_change":"2024-09-19T05:02:22.026950+0000","last_active":"2024-09-19T05:02:38.499021+0000","last_peered":"2024-09-19T05:02:38.499021+0000","last_clean":"2024-09-19T05:02:38.499021+0000","last_became_active":"2024-09-19T05:02:22.026498+0000","last_became_peered":"2024-09-19T05:02:22.026498+0000","last_unstale":"2024-09-19T05:02:38.499021+0000","last_undegraded":"2024-09-19T05:02:38.499021+0000","last_fullsized":"2024-09-19T05:02:38.499021+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:55:31.833714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.430242+0000","last_change":"2024-09-19T05:02:24.428958+0000","last_active":"2024-09-19T05:02:26.430242+0000","last_peered":"2024-09-19T05:02:26.430242+0000","last_clean":"2024-09-19T05:02:26.430242+0000","last_became_active":"2024-09-19T05:02:24.428203+0000","last_became_peered":"2024-09-19T05:02:24.428203+0000","last_unstale":"2024-09-19T05:02:26.430242+0000","last_undegraded":"2024-09-19T05:02:26.430242+0000","last_fullsized":"2024-09-19T05:02:26.430242+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:59:37.225357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:37.493615+0000","last_change":"2024-09-19T05:02:22.075901+0000","last_active":"2024-09-19T05:02:37.493615+0000","last_peered":"2024-09-19T05:02:37.493615+0000","last_clean":"2024-09-19T05:02:37.493615+0000","last_became_active":"2024-09-19T05:02:22.075512+0000","last_became_peered":"2024-09-19T05:02:22.075512+0000","last_unstale":"2024-09-19T05:02:37.493615+0000","last_undegraded":"2024-09-19T05:02:37.493615+0000","last_fullsized":"2024-09-19T05:02:37.493615+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:09:26.976888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.415511+0000","last_change":"2024-09-19T05:02:24.436797+0000","last_active":"2024-09-19T05:02:26.415511+0000","last_peered":"2024-09-19T05:02:26.415511+0000","last_clean":"2024-09-19T05:02:26.415511+0000","last_became_active":"2024-09-19T05:02:24.436457+0000","last_became_peered":"2024-09-19T05:02:24.436457+0000","last_unstale":"2024-09-19T05:02:26.415511+0000","last_undegraded":"2024-09-19T05:02:26.415511+0000","last_fullsized":"2024-09-19T05:02:26.415511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:32:44.702601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'24","reported_seq":72,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:49.534860+0000","last_change":"2024-09-19T05:02:22.027415+0000","last_active":"2024-09-19T05:02:49.534860+0000","last_peered":"2024-09-19T05:02:49.534860+0000","last_clean":"2024-09-19T05:02:49.534860+0000","last_became_active":"2024-09-19T05:02:22.027065+0000","last_became_peered":"2024-09-19T05:02:22.027065+0000","last_unstale":"2024-09-19T05:02:49.534860+0000","last_undegraded":"2024-09-19T05:02:49.534860+0000","last_fullsized":"2024-09-19T05:02:49.534860+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:02.599310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42203,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":28,"num_read_kb":54,"num_write":26,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.421996+0000","last_change":"2024-09-19T05:02:24.419718+0000","last_active":"2024-09-19T05:02:26.421996+0000","last_peered":"2024-09-19T05:02:26.421996+0000","last_clean":"2024-09-19T05:02:26.421996+0000","last_became_active":"2024-09-19T05:02:24.419388+0000","last_became_peered":"2024-09-19T05:02:24.419388+0000","last_unstale":"2024-09-19T05:02:26.421996+0000","last_undegraded":"2024-09-19T05:02:26.421996+0000","last_fullsized":"2024-09-19T05:02:26.421996+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:09.453551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:39.511271+0000","last_change":"2024-09-19T05:02:22.026220+0000","last_active":"2024-09-19T05:02:39.511271+0000","last_peered":"2024-09-19T05:02:39.511271+0000","last_clean":"2024-09-19T05:02:39.511271+0000","last_became_active":"2024-09-19T05:02:22.025838+0000","last_became_peered":"2024-09-19T05:02:22.025838+0000","last_unstale":"2024-09-19T05:02:39.511271+0000","last_undegraded":"2024-09-19T05:02:39.511271+0000","last_fullsized":"2024-09-19T05:02:39.511271+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:02.192799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.424167+0000","last_change":"2024-09-19T05:02:24.431872+0000","last_active":"2024-09-19T05:02:26.424167+0000","last_peered":"2024-09-19T05:02:26.424167+0000","last_clean":"2024-09-19T05:02:26.424167+0000","last_became_active":"2024-09-19T05:02:24.427227+0000","last_became_peered":"2024-09-19T05:02:24.427227+0000","last_unstale":"2024-09-19T05:02:26.424167+0000","last_undegraded":"2024-09-19T05:02:26.424167+0000","last_fullsized":"2024-09-19T05:02:26.424167+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:52.655108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:38.499901+0000","last_change":"2024-09-19T05:02:22.025136+0000","last_active":"2024-09-19T05:02:38.499901+0000","last_peered":"2024-09-19T05:02:38.499901+0000","last_clean":"2024-09-19T05:02:38.499901+0000","last_became_active":"2024-09-19T05:02:22.024813+0000","last_became_peered":"2024-09-19T05:02:22.024813+0000","last_unstale":"2024-09-19T05:02:38.499901+0000","last_undegraded":"2024-09-19T05:02:38.499901+0000","last_fullsized":"2024-09-19T05:02:38.499901+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:47:03.956661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.425183+0000","last_change":"2024-09-19T05:02:24.428497+0000","last_active":"2024-09-19T05:02:26.425183+0000","last_peered":"2024-09-19T05:02:26.425183+0000","last_clean":"2024-09-19T05:02:26.425183+0000","last_became_active":"2024-09-19T05:02:24.427774+0000","last_became_peered":"2024-09-19T05:02:24.427774+0000","last_unstale":"2024-09-19T05:02:26.425183+0000","last_undegraded":"2024-09-19T05:02:26.425183+0000","last_fullsized":"2024-09-19T05:02:26.425183+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:32:02.160354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'6","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:49.527427+0000","last_change":"2024-09-19T05:02:22.023789+0000","last_active":"2024-09-19T05:02:49.527427+0000","last_peered":"2024-09-19T05:02:49.527427+0000","last_clean":"2024-09-19T05:02:49.527427+0000","last_became_active":"2024-09-19T05:02:22.023371+0000","last_became_peered":"2024-09-19T05:02:22.023371+0000","last_unstale":"2024-09-19T05:02:49.527427+0000","last_undegraded":"2024-09-19T05:02:49.527427+0000","last_fullsized":"2024-09-19T05:02:49.527427+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:26:29.644878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.424639+0000","last_change":"2024-09-19T05:02:24.433098+0000","last_active":"2024-09-19T05:02:26.424639+0000","last_peered":"2024-09-19T05:02:26.424639+0000","last_clean":"2024-09-19T05:02:26.424639+0000","last_became_active":"2024-09-19T05:02:24.432800+0000","last_became_peered":"2024-09-19T05:02:24.432800+0000","last_unstale":"2024-09-19T05:02:26.424639+0000","last_undegraded":"2024-09-19T05:02:26.424639+0000","last_fullsized":"2024-09-19T05:02:26.424639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:50:30.163842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:38.500001+0000","last_change":"2024-09-19T05:02:22.075865+0000","last_active":"2024-09-19T05:02:38.500001+0000","last_peered":"2024-09-19T05:02:38.500001+0000","last_clean":"2024-09-19T05:02:38.500001+0000","last_became_active":"2024-09-19T05:02:22.075185+0000","last_became_peered":"2024-09-19T05:02:22.075185+0000","last_unstale":"2024-09-19T05:02:38.500001+0000","last_undegraded":"2024-09-19T05:02:38.500001+0000","last_fullsized":"2024-09-19T05:02:38.500001+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:19.150155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.421855+0000","last_change":"2024-09-19T05:02:24.422759+0000","last_active":"2024-09-19T05:02:26.421855+0000","last_peered":"2024-09-19T05:02:26.421855+0000","last_clean":"2024-09-19T05:02:26.421855+0000","last_became_active":"2024-09-19T05:02:24.422421+0000","last_became_peered":"2024-09-19T05:02:24.422421+0000","last_unstale":"2024-09-19T05:02:26.421855+0000","last_undegraded":"2024-09-19T05:02:26.421855+0000","last_fullsized":"2024-09-19T05:02:26.421855+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:58:50.083017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:38.480917+0000","last_change":"2024-09-19T05:02:22.024305+0000","last_active":"2024-09-19T05:02:38.480917+0000","last_peered":"2024-09-19T05:02:38.480917+0000","last_clean":"2024-09-19T05:02:38.480917+0000","last_became_active":"2024-09-19T05:02:22.023616+0000","last_became_peered":"2024-09-19T05:02:22.023616+0000","last_unstale":"2024-09-19T05:02:38.480917+0000","last_undegraded":"2024-09-19T05:02:38.480917+0000","last_fullsized":"2024-09-19T05:02:38.480917+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:05.665219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.427007+0000","last_change":"2024-09-19T05:02:24.423859+0000","last_active":"2024-09-19T05:02:26.427007+0000","last_peered":"2024-09-19T05:02:26.427007+0000","last_clean":"2024-09-19T05:02:26.427007+0000","last_became_active":"2024-09-19T05:02:24.423517+0000","last_became_peered":"2024-09-19T05:02:24.423517+0000","last_unstale":"2024-09-19T05:02:26.427007+0000","last_undegraded":"2024-09-19T05:02:26.427007+0000","last_fullsized":"2024-09-19T05:02:26.427007+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:39:50.474469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:38.500041+0000","last_change":"2024-09-19T05:02:22.024194+0000","last_active":"2024-09-19T05:02:38.500041+0000","last_peered":"2024-09-19T05:02:38.500041+0000","last_clean":"2024-09-19T05:02:38.500041+0000","last_became_active":"2024-09-19T05:02:22.023777+0000","last_became_peered":"2024-09-19T05:02:22.023777+0000","last_unstale":"2024-09-19T05:02:38.500041+0000","last_undegraded":"2024-09-19T05:02:38.500041+0000","last_fullsized":"2024-09-19T05:02:38.500041+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:37:51.814732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.425694+0000","last_change":"2024-09-19T05:02:24.419360+0000","last_active":"2024-09-19T05:02:26.425694+0000","last_peered":"2024-09-19T05:02:26.425694+0000","last_clean":"2024-09-19T05:02:26.425694+0000","last_became_active":"2024-09-19T05:02:24.419040+0000","last_became_peered":"2024-09-19T05:02:24.419040+0000","last_unstale":"2024-09-19T05:02:26.425694+0000","last_undegraded":"2024-09-19T05:02:26.425694+0000","last_fullsized":"2024-09-19T05:02:26.425694+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:48:12.922676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:38.499231+0000","last_change":"2024-09-19T05:02:22.023124+0000","last_active":"2024-09-19T05:02:38.499231+0000","last_peered":"2024-09-19T05:02:38.499231+0000","last_clean":"2024-09-19T05:02:38.499231+0000","last_became_active":"2024-09-19T05:02:22.022830+0000","last_became_peered":"2024-09-19T05:02:22.022830+0000","last_unstale":"2024-09-19T05:02:38.499231+0000","last_undegraded":"2024-09-19T05:02:38.499231+0000","last_fullsized":"2024-09-19T05:02:38.499231+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:22:13.208616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.425279+0000","last_change":"2024-09-19T05:02:24.426169+0000","last_active":"2024-09-19T05:02:26.425279+0000","last_peered":"2024-09-19T05:02:26.425279+0000","last_clean":"2024-09-19T05:02:26.425279+0000","last_became_active":"2024-09-19T05:02:24.425816+0000","last_became_peered":"2024-09-19T05:02:24.425816+0000","last_unstale":"2024-09-19T05:02:26.425279+0000","last_undegraded":"2024-09-19T05:02:26.425279+0000","last_fullsized":"2024-09-19T05:02:26.425279+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:56:11.735351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'6","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:50.529124+0000","last_change":"2024-09-19T05:02:22.154391+0000","last_active":"2024-09-19T05:02:50.529124+0000","last_peered":"2024-09-19T05:02:50.529124+0000","last_clean":"2024-09-19T05:02:50.529124+0000","last_became_active":"2024-09-19T05:02:22.154008+0000","last_became_peered":"2024-09-19T05:02:22.154008+0000","last_unstale":"2024-09-19T05:02:50.529124+0000","last_undegraded":"2024-09-19T05:02:50.529124+0000","last_fullsized":"2024-09-19T05:02:50.529124+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:29.997649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.425785+0000","last_change":"2024-09-19T05:02:24.427416+0000","last_active":"2024-09-19T05:02:26.425785+0000","last_peered":"2024-09-19T05:02:26.425785+0000","last_clean":"2024-09-19T05:02:26.425785+0000","last_became_active":"2024-09-19T05:02:24.426645+0000","last_became_peered":"2024-09-19T05:02:24.426645+0000","last_unstale":"2024-09-19T05:02:26.425785+0000","last_undegraded":"2024-09-19T05:02:26.425785+0000","last_fullsized":"2024-09-19T05:02:26.425785+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:21.841013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":344,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.425141+0000","last_change":"2024-09-19T05:01:04.407740+0000","last_active":"2024-09-19T05:02:26.425141+0000","last_peered":"2024-09-19T05:02:26.425141+0000","last_clean":"2024-09-19T05:02:26.425141+0000","last_became_active":"2024-09-19T05:01:04.407281+0000","last_became_peered":"2024-09-19T05:01:04.407281+0000","last_unstale":"2024-09-19T05:02:26.425141+0000","last_undegraded":"2024-09-19T05:02:26.425141+0000","last_fullsized":"2024-09-19T05:02:26.425141+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:57:27.643438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:57:27.643438+0000","last_clean_scrub_stamp":"2024-09-19T04:57:27.643438+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:54:50.242926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:39.509813+0000","last_change":"2024-09-19T05:02:22.023991+0000","last_active":"2024-09-19T05:02:39.509813+0000","last_peered":"2024-09-19T05:02:39.509813+0000","last_clean":"2024-09-19T05:02:39.509813+0000","last_became_active":"2024-09-19T05:02:22.023361+0000","last_became_peered":"2024-09-19T05:02:22.023361+0000","last_unstale":"2024-09-19T05:02:39.509813+0000","last_undegraded":"2024-09-19T05:02:39.509813+0000","last_fullsized":"2024-09-19T05:02:39.509813+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:37:05.551318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.415403+0000","last_change":"2024-09-19T05:02:24.424473+0000","last_active":"2024-09-19T05:02:26.415403+0000","last_peered":"2024-09-19T05:02:26.415403+0000","last_clean":"2024-09-19T05:02:26.415403+0000","last_became_active":"2024-09-19T05:02:24.423968+0000","last_became_peered":"2024-09-19T05:02:24.423968+0000","last_unstale":"2024-09-19T05:02:26.415403+0000","last_undegraded":"2024-09-19T05:02:26.415403+0000","last_fullsized":"2024-09-19T05:02:26.415403+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:29:53.305036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:36.484572+0000","last_change":"2024-09-19T05:02:22.026278+0000","last_active":"2024-09-19T05:02:36.484572+0000","last_peered":"2024-09-19T05:02:36.484572+0000","last_clean":"2024-09-19T05:02:36.484572+0000","last_became_active":"2024-09-19T05:02:22.025759+0000","last_became_peered":"2024-09-19T05:02:22.025759+0000","last_unstale":"2024-09-19T05:02:36.484572+0000","last_undegraded":"2024-09-19T05:02:36.484572+0000","last_fullsized":"2024-09-19T05:02:36.484572+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:21:21.696382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.422877+0000","last_change":"2024-09-19T05:02:24.426880+0000","last_active":"2024-09-19T05:02:26.422877+0000","last_peered":"2024-09-19T05:02:26.422877+0000","last_clean":"2024-09-19T05:02:26.422877+0000","last_became_active":"2024-09-19T05:02:24.426336+0000","last_became_peered":"2024-09-19T05:02:24.426336+0000","last_unstale":"2024-09-19T05:02:26.422877+0000","last_undegraded":"2024-09-19T05:02:26.422877+0000","last_fullsized":"2024-09-19T05:02:26.422877+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:51:07.110738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:49.526926+0000","last_change":"2024-09-19T05:02:22.075789+0000","last_active":"2024-09-19T05:02:49.526926+0000","last_peered":"2024-09-19T05:02:49.526926+0000","last_clean":"2024-09-19T05:02:49.526926+0000","last_became_active":"2024-09-19T05:02:22.075415+0000","last_became_peered":"2024-09-19T05:02:22.075415+0000","last_unstale":"2024-09-19T05:02:49.526926+0000","last_undegraded":"2024-09-19T05:02:49.526926+0000","last_fullsized":"2024-09-19T05:02:49.526926+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:11:43.843627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.425383+0000","last_change":"2024-09-19T05:02:24.422661+0000","last_active":"2024-09-19T05:02:26.425383+0000","last_peered":"2024-09-19T05:02:26.425383+0000","last_clean":"2024-09-19T05:02:26.425383+0000","last_became_active":"2024-09-19T05:02:24.422313+0000","last_became_peered":"2024-09-19T05:02:24.422313+0000","last_unstale":"2024-09-19T05:02:26.425383+0000","last_undegraded":"2024-09-19T05:02:26.425383+0000","last_fullsized":"2024-09-19T05:02:26.425383+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:07:39.729388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'6","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:48.528876+0000","last_change":"2024-09-19T05:02:22.028008+0000","last_active":"2024-09-19T05:02:48.528876+0000","last_peered":"2024-09-19T05:02:48.528876+0000","last_clean":"2024-09-19T05:02:48.528876+0000","last_became_active":"2024-09-19T05:02:22.027635+0000","last_became_peered":"2024-09-19T05:02:22.027635+0000","last_unstale":"2024-09-19T05:02:48.528876+0000","last_undegraded":"2024-09-19T05:02:48.528876+0000","last_fullsized":"2024-09-19T05:02:48.528876+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:41:36.902282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.421366+0000","last_change":"2024-09-19T05:02:24.435362+0000","last_active":"2024-09-19T05:02:26.421366+0000","last_peered":"2024-09-19T05:02:26.421366+0000","last_clean":"2024-09-19T05:02:26.421366+0000","last_became_active":"2024-09-19T05:02:24.435056+0000","last_became_peered":"2024-09-19T05:02:24.435056+0000","last_unstale":"2024-09-19T05:02:26.421366+0000","last_undegraded":"2024-09-19T05:02:26.421366+0000","last_fullsized":"2024-09-19T05:02:26.421366+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:58:40.092466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.424059+0000","last_change":"2024-09-19T05:02:22.077651+0000","last_active":"2024-09-19T05:02:26.424059+0000","last_peered":"2024-09-19T05:02:26.424059+0000","last_clean":"2024-09-19T05:02:26.424059+0000","last_became_active":"2024-09-19T05:02:22.076273+0000","last_became_peered":"2024-09-19T05:02:22.076273+0000","last_unstale":"2024-09-19T05:02:26.424059+0000","last_undegraded":"2024-09-19T05:02:26.424059+0000","last_fullsized":"2024-09-19T05:02:26.424059+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:28:42.064126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.421778+0000","last_change":"2024-09-19T05:02:24.438125+0000","last_active":"2024-09-19T05:02:26.421778+0000","last_peered":"2024-09-19T05:02:26.421778+0000","last_clean":"2024-09-19T05:02:26.421778+0000","last_became_active":"2024-09-19T05:02:24.437806+0000","last_became_peered":"2024-09-19T05:02:24.437806+0000","last_unstale":"2024-09-19T05:02:26.421778+0000","last_undegraded":"2024-09-19T05:02:26.421778+0000","last_fullsized":"2024-09-19T05:02:26.421778+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:46:08.338501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:38.500675+0000","last_change":"2024-09-19T05:02:22.075920+0000","last_active":"2024-09-19T05:02:38.500675+0000","last_peered":"2024-09-19T05:02:38.500675+0000","last_clean":"2024-09-19T05:02:38.500675+0000","last_became_active":"2024-09-19T05:02:22.075609+0000","last_became_peered":"2024-09-19T05:02:22.075609+0000","last_unstale":"2024-09-19T05:02:38.500675+0000","last_undegraded":"2024-09-19T05:02:38.500675+0000","last_fullsized":"2024-09-19T05:02:38.500675+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:45.915625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.423025+0000","last_change":"2024-09-19T05:02:24.434681+0000","last_active":"2024-09-19T05:02:26.423025+0000","last_peered":"2024-09-19T05:02:26.423025+0000","last_clean":"2024-09-19T05:02:26.423025+0000","last_became_active":"2024-09-19T05:02:24.434257+0000","last_became_peered":"2024-09-19T05:02:24.434257+0000","last_unstale":"2024-09-19T05:02:26.423025+0000","last_undegraded":"2024-09-19T05:02:26.423025+0000","last_fullsized":"2024-09-19T05:02:26.423025+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:06:09.133112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'11","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:48.422693+0000","last_change":"2024-09-19T05:02:22.024314+0000","last_active":"2024-09-19T05:02:48.422693+0000","last_peered":"2024-09-19T05:02:48.422693+0000","last_clean":"2024-09-19T05:02:48.422693+0000","last_became_active":"2024-09-19T05:02:22.023733+0000","last_became_peered":"2024-09-19T05:02:22.023733+0000","last_unstale":"2024-09-19T05:02:48.422693+0000","last_undegraded":"2024-09-19T05:02:48.422693+0000","last_fullsized":"2024-09-19T05:02:48.422693+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:39:31.251257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.415756+0000","last_change":"2024-09-19T05:02:24.429807+0000","last_active":"2024-09-19T05:02:26.415756+0000","last_peered":"2024-09-19T05:02:26.415756+0000","last_clean":"2024-09-19T05:02:26.415756+0000","last_became_active":"2024-09-19T05:02:24.429477+0000","last_became_peered":"2024-09-19T05:02:24.429477+0000","last_unstale":"2024-09-19T05:02:26.415756+0000","last_undegraded":"2024-09-19T05:02:26.415756+0000","last_fullsized":"2024-09-19T05:02:26.415756+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:02.314851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:38.499588+0000","last_change":"2024-09-19T05:02:22.154534+0000","last_active":"2024-09-19T05:02:38.499588+0000","last_peered":"2024-09-19T05:02:38.499588+0000","last_clean":"2024-09-19T05:02:38.499588+0000","last_became_active":"2024-09-19T05:02:22.154238+0000","last_became_peered":"2024-09-19T05:02:22.154238+0000","last_unstale":"2024-09-19T05:02:38.499588+0000","last_undegraded":"2024-09-19T05:02:38.499588+0000","last_fullsized":"2024-09-19T05:02:38.499588+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:32:30.963503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.429485+0000","last_change":"2024-09-19T05:02:24.422817+0000","last_active":"2024-09-19T05:02:26.429485+0000","last_peered":"2024-09-19T05:02:26.429485+0000","last_clean":"2024-09-19T05:02:26.429485+0000","last_became_active":"2024-09-19T05:02:24.422478+0000","last_became_peered":"2024-09-19T05:02:24.422478+0000","last_unstale":"2024-09-19T05:02:26.429485+0000","last_undegraded":"2024-09-19T05:02:26.429485+0000","last_fullsized":"2024-09-19T05:02:26.429485+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:15:35.442149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:35.481268+0000","last_change":"2024-09-19T05:02:22.074396+0000","last_active":"2024-09-19T05:02:35.481268+0000","last_peered":"2024-09-19T05:02:35.481268+0000","last_clean":"2024-09-19T05:02:35.481268+0000","last_became_active":"2024-09-19T05:02:22.074108+0000","last_became_peered":"2024-09-19T05:02:22.074108+0000","last_unstale":"2024-09-19T05:02:35.481268+0000","last_undegraded":"2024-09-19T05:02:35.481268+0000","last_fullsized":"2024-09-19T05:02:35.481268+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:58:32.718702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.421653+0000","last_change":"2024-09-19T05:02:24.429599+0000","last_active":"2024-09-19T05:02:26.421653+0000","last_peered":"2024-09-19T05:02:26.421653+0000","last_clean":"2024-09-19T05:02:26.421653+0000","last_became_active":"2024-09-19T05:02:24.429268+0000","last_became_peered":"2024-09-19T05:02:24.429268+0000","last_unstale":"2024-09-19T05:02:26.421653+0000","last_undegraded":"2024-09-19T05:02:26.421653+0000","last_fullsized":"2024-09-19T05:02:26.421653+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:57.123516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:38.498197+0000","last_change":"2024-09-19T05:02:22.021900+0000","last_active":"2024-09-19T05:02:38.498197+0000","last_peered":"2024-09-19T05:02:38.498197+0000","last_clean":"2024-09-19T05:02:38.498197+0000","last_became_active":"2024-09-19T05:02:22.020576+0000","last_became_peered":"2024-09-19T05:02:22.020576+0000","last_unstale":"2024-09-19T05:02:38.498197+0000","last_undegraded":"2024-09-19T05:02:38.498197+0000","last_fullsized":"2024-09-19T05:02:38.498197+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:44:14.466475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.426331+0000","last_change":"2024-09-19T05:02:24.420362+0000","last_active":"2024-09-19T05:02:26.426331+0000","last_peered":"2024-09-19T05:02:26.426331+0000","last_clean":"2024-09-19T05:02:26.426331+0000","last_became_active":"2024-09-19T05:02:24.420047+0000","last_became_peered":"2024-09-19T05:02:24.420047+0000","last_unstale":"2024-09-19T05:02:26.426331+0000","last_undegraded":"2024-09-19T05:02:26.426331+0000","last_fullsized":"2024-09-19T05:02:26.426331+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:28:56.977946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:37.493114+0000","last_change":"2024-09-19T05:02:22.024565+0000","last_active":"2024-09-19T05:02:37.493114+0000","last_peered":"2024-09-19T05:02:37.493114+0000","last_clean":"2024-09-19T05:02:37.493114+0000","last_became_active":"2024-09-19T05:02:22.024246+0000","last_became_peered":"2024-09-19T05:02:22.024246+0000","last_unstale":"2024-09-19T05:02:37.493114+0000","last_undegraded":"2024-09-19T05:02:37.493114+0000","last_fullsized":"2024-09-19T05:02:37.493114+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:58:27.491975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.415472+0000","last_change":"2024-09-19T05:02:24.428046+0000","last_active":"2024-09-19T05:02:26.415472+0000","last_peered":"2024-09-19T05:02:26.415472+0000","last_clean":"2024-09-19T05:02:26.415472+0000","last_became_active":"2024-09-19T05:02:24.427724+0000","last_became_peered":"2024-09-19T05:02:24.427724+0000","last_unstale":"2024-09-19T05:02:26.415472+0000","last_undegraded":"2024-09-19T05:02:26.415472+0000","last_fullsized":"2024-09-19T05:02:26.415472+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:11:54.252876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:35.481601+0000","last_change":"2024-09-19T05:02:22.023054+0000","last_active":"2024-09-19T05:02:35.481601+0000","last_peered":"2024-09-19T05:02:35.481601+0000","last_clean":"2024-09-19T05:02:35.481601+0000","last_became_active":"2024-09-19T05:02:22.022700+0000","last_became_peered":"2024-09-19T05:02:22.022700+0000","last_unstale":"2024-09-19T05:02:35.481601+0000","last_undegraded":"2024-09-19T05:02:35.481601+0000","last_fullsized":"2024-09-19T05:02:35.481601+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:44:19.714938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.425315+0000","last_change":"2024-09-19T05:02:24.423045+0000","last_active":"2024-09-19T05:02:26.425315+0000","last_peered":"2024-09-19T05:02:26.425315+0000","last_clean":"2024-09-19T05:02:26.425315+0000","last_became_active":"2024-09-19T05:02:24.422719+0000","last_became_peered":"2024-09-19T05:02:24.422719+0000","last_unstale":"2024-09-19T05:02:26.425315+0000","last_undegraded":"2024-09-19T05:02:26.425315+0000","last_fullsized":"2024-09-19T05:02:26.425315+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:45.299603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.422314+0000","last_change":"2024-09-19T05:02:24.426942+0000","last_active":"2024-09-19T05:02:26.422314+0000","last_peered":"2024-09-19T05:02:26.422314+0000","last_clean":"2024-09-19T05:02:26.422314+0000","last_became_active":"2024-09-19T05:02:24.425896+0000","last_became_peered":"2024-09-19T05:02:24.425896+0000","last_unstale":"2024-09-19T05:02:26.422314+0000","last_undegraded":"2024-09-19T05:02:26.422314+0000","last_fullsized":"2024-09-19T05:02:26.422314+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:54:15.384309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:38.499891+0000","last_change":"2024-09-19T05:02:22.021415+0000","last_active":"2024-09-19T05:02:38.499891+0000","last_peered":"2024-09-19T05:02:38.499891+0000","last_clean":"2024-09-19T05:02:38.499891+0000","last_became_active":"2024-09-19T05:02:22.020508+0000","last_became_peered":"2024-09-19T05:02:22.020508+0000","last_unstale":"2024-09-19T05:02:38.499891+0000","last_undegraded":"2024-09-19T05:02:38.499891+0000","last_fullsized":"2024-09-19T05:02:38.499891+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:35.912386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.415456+0000","last_change":"2024-09-19T05:02:24.435538+0000","last_active":"2024-09-19T05:02:26.415456+0000","last_peered":"2024-09-19T05:02:26.415456+0000","last_clean":"2024-09-19T05:02:26.415456+0000","last_became_active":"2024-09-19T05:02:24.435209+0000","last_became_peered":"2024-09-19T05:02:24.435209+0000","last_unstale":"2024-09-19T05:02:26.415456+0000","last_undegraded":"2024-09-19T05:02:26.415456+0000","last_fullsized":"2024-09-19T05:02:26.415456+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:55:30.075113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:39.511585+0000","last_change":"2024-09-19T05:02:22.024153+0000","last_active":"2024-09-19T05:02:39.511585+0000","last_peered":"2024-09-19T05:02:39.511585+0000","last_clean":"2024-09-19T05:02:39.511585+0000","last_became_active":"2024-09-19T05:02:22.023879+0000","last_became_peered":"2024-09-19T05:02:22.023879+0000","last_unstale":"2024-09-19T05:02:39.511585+0000","last_undegraded":"2024-09-19T05:02:39.511585+0000","last_fullsized":"2024-09-19T05:02:39.511585+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:39:32.878079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.430074+0000","last_change":"2024-09-19T05:02:24.428958+0000","last_active":"2024-09-19T05:02:26.430074+0000","last_peered":"2024-09-19T05:02:26.430074+0000","last_clean":"2024-09-19T05:02:26.430074+0000","last_became_active":"2024-09-19T05:02:24.428672+0000","last_became_peered":"2024-09-19T05:02:24.428672+0000","last_unstale":"2024-09-19T05:02:26.430074+0000","last_undegraded":"2024-09-19T05:02:26.430074+0000","last_fullsized":"2024-09-19T05:02:26.430074+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:06:23.529706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.428814+0000","last_change":"2024-09-19T05:02:22.077034+0000","last_active":"2024-09-19T05:02:26.428814+0000","last_peered":"2024-09-19T05:02:26.428814+0000","last_clean":"2024-09-19T05:02:26.428814+0000","last_became_active":"2024-09-19T05:02:22.076182+0000","last_became_peered":"2024-09-19T05:02:22.076182+0000","last_unstale":"2024-09-19T05:02:26.428814+0000","last_undegraded":"2024-09-19T05:02:26.428814+0000","last_fullsized":"2024-09-19T05:02:26.428814+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:43:31.076464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.415813+0000","last_change":"2024-09-19T05:02:24.423765+0000","last_active":"2024-09-19T05:02:26.415813+0000","last_peered":"2024-09-19T05:02:26.415813+0000","last_clean":"2024-09-19T05:02:26.415813+0000","last_became_active":"2024-09-19T05:02:24.423445+0000","last_became_peered":"2024-09-19T05:02:24.423445+0000","last_unstale":"2024-09-19T05:02:26.415813+0000","last_undegraded":"2024-09-19T05:02:26.415813+0000","last_fullsized":"2024-09-19T05:02:26.415813+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:10:07.806697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'13","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:49.529595+0000","last_change":"2024-09-19T05:02:22.023413+0000","last_active":"2024-09-19T05:02:49.529595+0000","last_peered":"2024-09-19T05:02:49.529595+0000","last_clean":"2024-09-19T05:02:49.529595+0000","last_became_active":"2024-09-19T05:02:22.023078+0000","last_became_peered":"2024-09-19T05:02:22.023078+0000","last_unstale":"2024-09-19T05:02:49.529595+0000","last_undegraded":"2024-09-19T05:02:49.529595+0000","last_fullsized":"2024-09-19T05:02:49.529595+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:11:09.185011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":50197,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":157,"num_read_kb":134,"num_write":231,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":516096,"data_stored":150591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":201,"ondisk_log_size":201,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285699,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514516,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":435933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743448,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":1069165,"data_compressed":36330,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906973,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":435470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135907,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":435848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397544,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":1029154,"data_compressed":36330,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561069,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":435580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790003,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":995328,"data_stored":1028046,"data_compressed":36330,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986231,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":435441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215166,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":933888,"data_stored":435012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411395,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":479309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":962560,"data_stored":478696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738446,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":937984,"data_stored":436919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34683,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34683,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34683,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36330,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36330,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36330,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36330,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36330,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":44348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":44961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":44332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T05:02:57.551 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T05:02:57.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:57 smithi151 ceph-mon[35425]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:02:57.929 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:02:57.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:57 smithi096 ceph-mon[35085]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:02:58.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:57 smithi043 ceph-mon[30800]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:02:58.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.513+0000 7f335c800640 1 -- 172.21.15.43:0/1733641571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3354074e30 msgr2=0x7f335410df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:58.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.513+0000 7f335c800640 1 --2- 172.21.15.43:0/1733641571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3354074e30 0x7f335410df10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3344009900 tx=0x7f334402a490 comp rx=0 tx=0).stop 2024-09-19T05:02:58.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.513+0000 7f335c800640 1 -- 172.21.15.43:0/1733641571 shutdown_connections 2024-09-19T05:02:58.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.513+0000 7f335c800640 1 --2- 172.21.15.43:0/1733641571 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f335410e450 0x7f3354110840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:58.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.513+0000 7f335c800640 1 --2- 172.21.15.43:0/1733641571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3354074e30 0x7f335410df10 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:58.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.513+0000 7f335c800640 1 --2- 172.21.15.43:0/1733641571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3354074510 0x7f33540748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:58.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.513+0000 7f335c800640 1 -- 172.21.15.43:0/1733641571 >> 172.21.15.43:0/1733641571 conn(0x7f33540fd8b0 msgr2=0x7f33540ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:58.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.513+0000 7f335c800640 1 -- 172.21.15.43:0/1733641571 shutdown_connections 2024-09-19T05:02:58.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.513+0000 7f335c800640 1 -- 172.21.15.43:0/1733641571 wait complete. 2024-09-19T05:02:58.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.514+0000 7f335c800640 1 Processor -- start 2024-09-19T05:02:58.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.514+0000 7f335c800640 1 -- start start 2024-09-19T05:02:58.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.515+0000 7f335c800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3354074510 0x7f335419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:58.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.515+0000 7f335c800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3354074e30 0x7f335419ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:58.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.515+0000 7f335c800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f335410e450 0x7f335419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:58.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.515+0000 7f335c800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f33541141c0 con 0x7f3354074510 2024-09-19T05:02:58.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.515+0000 7f335c800640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3354114040 con 0x7f335410e450 2024-09-19T05:02:58.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.515+0000 7f335c800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3354114340 con 0x7f3354074e30 2024-09-19T05:02:58.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.515+0000 7f3359e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3354074510 0x7f335419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:58.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.515+0000 7f3359400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3354074e30 0x7f335419ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:58.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.515+0000 7f3359e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3354074510 0x7f335419a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59136/0 (socket says 172.21.15.43:59136) 2024-09-19T05:02:58.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.515+0000 7f3359400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3354074e30 0x7f335419ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:42158/0 (socket says 172.21.15.43:42158) 2024-09-19T05:02:58.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.515+0000 7f3359e00640 1 -- 172.21.15.43:0/531897474 learned_addr learned my addr 172.21.15.43:0/531897474 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:02:58.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.515+0000 7f335a800640 1 --2- 172.21.15.43:0/531897474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f335410e450 0x7f335419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:58.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.516+0000 7f3359400640 1 -- 172.21.15.43:0/531897474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f335410e450 msgr2=0x7f335419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:58.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.516+0000 7f3359400640 1 --2- 172.21.15.43:0/531897474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f335410e450 0x7f335419ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:58.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.516+0000 7f3359400640 1 -- 172.21.15.43:0/531897474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3354074510 msgr2=0x7f335419a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:58.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.516+0000 7f3359400640 1 --2- 172.21.15.43:0/531897474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3354074510 0x7f335419a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:58.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.516+0000 7f3359400640 1 -- 172.21.15.43:0/531897474 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f335419f670 con 0x7f3354074e30 2024-09-19T05:02:58.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.516+0000 7f335a800640 1 --2- 172.21.15.43:0/531897474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f335410e450 0x7f335419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:02:58.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.516+0000 7f3359400640 1 --2- 172.21.15.43:0/531897474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3354074e30 0x7f335419ac00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f334402a9b0 tx=0x7f3344003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:58.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.516+0000 7f334ea00640 1 -- 172.21.15.43:0/531897474 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334403f070 con 0x7f3354074e30 2024-09-19T05:02:58.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.516+0000 7f335c800640 1 -- 172.21.15.43:0/531897474 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f335419f900 con 0x7f3354074e30 2024-09-19T05:02:58.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.517+0000 7f334ea00640 1 -- 172.21.15.43:0/531897474 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3344031c90 con 0x7f3354074e30 2024-09-19T05:02:58.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.517+0000 7f334ea00640 1 -- 172.21.15.43:0/531897474 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3344035030 con 0x7f3354074e30 2024-09-19T05:02:58.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.517+0000 7f335c800640 1 -- 172.21.15.43:0/531897474 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33541a7240 con 0x7f3354074e30 2024-09-19T05:02:58.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.518+0000 7f335c800640 1 -- 172.21.15.43:0/531897474 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33540748f0 con 0x7f3354074e30 2024-09-19T05:02:58.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.520+0000 7f334ea00640 1 -- 172.21.15.43:0/531897474 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f3344042070 con 0x7f3354074e30 2024-09-19T05:02:58.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.521+0000 7f334ea00640 1 --2- 172.21.15.43:0/531897474 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f333c0a3350 0x7f333c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:02:58.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.521+0000 7f334ea00640 1 -- 172.21.15.43:0/531897474 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f33440f05d0 con 0x7f3354074e30 2024-09-19T05:02:58.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.521+0000 7f3359e00640 1 --2- 172.21.15.43:0/531897474 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f333c0a3350 0x7f333c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:02:58.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.522+0000 7f3359e00640 1 --2- 172.21.15.43:0/531897474 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f333c0a3350 0x7f333c0a5810 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3348009cd0 tx=0x7f3348005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:02:58.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.523+0000 7f334ea00640 1 -- 172.21.15.43:0/531897474 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f33440bd240 con 0x7f3354074e30 2024-09-19T05:02:58.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.711+0000 7f335c800640 1 -- 172.21.15.43:0/531897474 --> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3354109640 con 0x7f333c0a3350 2024-09-19T05:02:58.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:58 smithi043 ceph-mon[30800]: pgmap v308: 97 pgs: 97 active+clean; 638 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.7 KiB/s wr, 15 op/s 2024-09-19T05:02:58.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.719+0000 7f334ea00640 1 -- 172.21.15.43:0/531897474 <== mgr.14150 v2:172.21.15.43:6800/3167108925 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258912 (secure 0 0 0) 0x7f3348009dc0 con 0x7f333c0a3350 2024-09-19T05:02:58.725 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:02:58.726 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-19T05:02:58.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.723+0000 7f335c800640 1 -- 172.21.15.43:0/531897474 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f333c0a3350 msgr2=0x7f333c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:58.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.723+0000 7f335c800640 1 --2- 172.21.15.43:0/531897474 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f333c0a3350 0x7f333c0a5810 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3348009cd0 tx=0x7f3348005f90 comp rx=0 tx=0).stop 2024-09-19T05:02:58.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.723+0000 7f335c800640 1 -- 172.21.15.43:0/531897474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3354074e30 msgr2=0x7f335419ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:02:58.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.723+0000 7f335c800640 1 --2- 172.21.15.43:0/531897474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3354074e30 0x7f335419ac00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f334402a9b0 tx=0x7f3344003b90 comp rx=0 tx=0).stop 2024-09-19T05:02:58.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.723+0000 7f335c800640 1 -- 172.21.15.43:0/531897474 shutdown_connections 2024-09-19T05:02:58.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.723+0000 7f335c800640 1 --2- 172.21.15.43:0/531897474 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f333c0a3350 0x7f333c0a5810 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:58.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.723+0000 7f335c800640 1 --2- 172.21.15.43:0/531897474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f335410e450 0x7f335419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:58.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.723+0000 7f335c800640 1 --2- 172.21.15.43:0/531897474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3354074e30 0x7f335419ac00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:58.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.723+0000 7f335c800640 1 --2- 172.21.15.43:0/531897474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3354074510 0x7f335419a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:02:58.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.723+0000 7f335c800640 1 -- 172.21.15.43:0/531897474 >> 172.21.15.43:0/531897474 conn(0x7f33540fd8b0 msgr2=0x7f33540fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:02:58.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.723+0000 7f335c800640 1 -- 172.21.15.43:0/531897474 shutdown_connections 2024-09-19T05:02:58.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:02:58.723+0000 7f335c800640 1 -- 172.21.15.43:0/531897474 wait complete. 2024-09-19T05:02:58.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:58 smithi151 ceph-mon[35425]: pgmap v308: 97 pgs: 97 active+clean; 638 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.7 KiB/s wr, 15 op/s 2024-09-19T05:02:58.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:58 smithi096 ceph-mon[35085]: pgmap v308: 97 pgs: 97 active+clean; 638 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.7 KiB/s wr, 15 op/s 2024-09-19T05:02:59.456 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2024-09-19T05:02:57.147119+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653041,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":334,"num_read_kb":290,"num_write":424,"num_write_kb":3068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":346,"ondisk_log_size":346,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368776,"kb_used_data":11208,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124573048,"statfs":{"total":1151940427776,"available":1151562801152,"internally_reserved":0,"allocated":11476992,"data_stored":7159057,"data_compressed":108990,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":54639,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":149,"num_read_kb":132,"num_write":38,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009448"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.428048+0000","last_change":"2024-09-19T05:02:24.426572+0000","last_active":"2024-09-19T05:02:26.428048+0000","last_peered":"2024-09-19T05:02:26.428048+0000","last_clean":"2024-09-19T05:02:26.428048+0000","last_became_active":"2024-09-19T05:02:24.426241+0000","last_became_peered":"2024-09-19T05:02:24.426241+0000","last_unstale":"2024-09-19T05:02:26.428048+0000","last_undegraded":"2024-09-19T05:02:26.428048+0000","last_fullsized":"2024-09-19T05:02:26.428048+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:42.511495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.429332+0000","last_change":"2024-09-19T05:02:24.427263+0000","last_active":"2024-09-19T05:02:26.429332+0000","last_peered":"2024-09-19T05:02:26.429332+0000","last_clean":"2024-09-19T05:02:26.429332+0000","last_became_active":"2024-09-19T05:02:24.426928+0000","last_became_peered":"2024-09-19T05:02:24.426928+0000","last_unstale":"2024-09-19T05:02:26.429332+0000","last_undegraded":"2024-09-19T05:02:26.429332+0000","last_fullsized":"2024-09-19T05:02:26.429332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:03.177020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.429185+0000","last_change":"2024-09-19T05:02:24.432709+0000","last_active":"2024-09-19T05:02:26.429185+0000","last_peered":"2024-09-19T05:02:26.429185+0000","last_clean":"2024-09-19T05:02:26.429185+0000","last_became_active":"2024-09-19T05:02:24.431114+0000","last_became_peered":"2024-09-19T05:02:24.431114+0000","last_unstale":"2024-09-19T05:02:26.429185+0000","last_undegraded":"2024-09-19T05:02:26.429185+0000","last_fullsized":"2024-09-19T05:02:26.429185+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:57.244919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.422150+0000","last_change":"2024-09-19T05:02:24.432174+0000","last_active":"2024-09-19T05:02:26.422150+0000","last_peered":"2024-09-19T05:02:26.422150+0000","last_clean":"2024-09-19T05:02:26.422150+0000","last_became_active":"2024-09-19T05:02:24.431445+0000","last_became_peered":"2024-09-19T05:02:24.431445+0000","last_unstale":"2024-09-19T05:02:26.422150+0000","last_undegraded":"2024-09-19T05:02:26.422150+0000","last_fullsized":"2024-09-19T05:02:26.422150+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:01:35.959119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.421670+0000","last_change":"2024-09-19T05:02:24.419992+0000","last_active":"2024-09-19T05:02:26.421670+0000","last_peered":"2024-09-19T05:02:26.421670+0000","last_clean":"2024-09-19T05:02:26.421670+0000","last_became_active":"2024-09-19T05:02:24.419643+0000","last_became_peered":"2024-09-19T05:02:24.419643+0000","last_unstale":"2024-09-19T05:02:26.421670+0000","last_undegraded":"2024-09-19T05:02:26.421670+0000","last_fullsized":"2024-09-19T05:02:26.421670+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:54:40.548216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.421437+0000","last_change":"2024-09-19T05:02:24.434106+0000","last_active":"2024-09-19T05:02:26.421437+0000","last_peered":"2024-09-19T05:02:26.421437+0000","last_clean":"2024-09-19T05:02:26.421437+0000","last_became_active":"2024-09-19T05:02:24.433124+0000","last_became_peered":"2024-09-19T05:02:24.433124+0000","last_unstale":"2024-09-19T05:02:26.421437+0000","last_undegraded":"2024-09-19T05:02:26.421437+0000","last_fullsized":"2024-09-19T05:02:26.421437+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:09:46.502506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.424981+0000","last_change":"2024-09-19T05:02:24.429640+0000","last_active":"2024-09-19T05:02:26.424981+0000","last_peered":"2024-09-19T05:02:26.424981+0000","last_clean":"2024-09-19T05:02:26.424981+0000","last_became_active":"2024-09-19T05:02:24.429326+0000","last_became_peered":"2024-09-19T05:02:24.429326+0000","last_unstale":"2024-09-19T05:02:26.424981+0000","last_undegraded":"2024-09-19T05:02:26.424981+0000","last_fullsized":"2024-09-19T05:02:26.424981+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:27.330449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.415057+0000","last_change":"2024-09-19T05:02:24.430933+0000","last_active":"2024-09-19T05:02:26.415057+0000","last_peered":"2024-09-19T05:02:26.415057+0000","last_clean":"2024-09-19T05:02:26.415057+0000","last_became_active":"2024-09-19T05:02:24.430478+0000","last_became_peered":"2024-09-19T05:02:24.430478+0000","last_unstale":"2024-09-19T05:02:26.415057+0000","last_undegraded":"2024-09-19T05:02:26.415057+0000","last_fullsized":"2024-09-19T05:02:26.415057+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:45:22.171049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.415974+0000","last_change":"2024-09-19T05:02:24.433207+0000","last_active":"2024-09-19T05:02:26.415974+0000","last_peered":"2024-09-19T05:02:26.415974+0000","last_clean":"2024-09-19T05:02:26.415974+0000","last_became_active":"2024-09-19T05:02:24.432882+0000","last_became_peered":"2024-09-19T05:02:24.432882+0000","last_unstale":"2024-09-19T05:02:26.415974+0000","last_undegraded":"2024-09-19T05:02:26.415974+0000","last_fullsized":"2024-09-19T05:02:26.415974+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:06:42.748227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.421944+0000","last_change":"2024-09-19T05:02:24.433825+0000","last_active":"2024-09-19T05:02:26.421944+0000","last_peered":"2024-09-19T05:02:26.421944+0000","last_clean":"2024-09-19T05:02:26.421944+0000","last_became_active":"2024-09-19T05:02:24.433088+0000","last_became_peered":"2024-09-19T05:02:24.433088+0000","last_unstale":"2024-09-19T05:02:26.421944+0000","last_undegraded":"2024-09-19T05:02:26.421944+0000","last_fullsized":"2024-09-19T05:02:26.421944+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:12:53.039028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.425611+0000","last_change":"2024-09-19T05:02:24.427661+0000","last_active":"2024-09-19T05:02:26.425611+0000","last_peered":"2024-09-19T05:02:26.425611+0000","last_clean":"2024-09-19T05:02:26.425611+0000","last_became_active":"2024-09-19T05:02:24.427093+0000","last_became_peered":"2024-09-19T05:02:24.427093+0000","last_unstale":"2024-09-19T05:02:26.425611+0000","last_undegraded":"2024-09-19T05:02:26.425611+0000","last_fullsized":"2024-09-19T05:02:26.425611+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:43.091041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.422330+0000","last_change":"2024-09-19T05:02:24.430820+0000","last_active":"2024-09-19T05:02:26.422330+0000","last_peered":"2024-09-19T05:02:26.422330+0000","last_clean":"2024-09-19T05:02:26.422330+0000","last_became_active":"2024-09-19T05:02:24.429945+0000","last_became_peered":"2024-09-19T05:02:24.429945+0000","last_unstale":"2024-09-19T05:02:26.422330+0000","last_undegraded":"2024-09-19T05:02:26.422330+0000","last_fullsized":"2024-09-19T05:02:26.422330+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:21:59.400725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.424811+0000","last_change":"2024-09-19T05:02:24.428695+0000","last_active":"2024-09-19T05:02:26.424811+0000","last_peered":"2024-09-19T05:02:26.424811+0000","last_clean":"2024-09-19T05:02:26.424811+0000","last_became_active":"2024-09-19T05:02:24.426385+0000","last_became_peered":"2024-09-19T05:02:24.426385+0000","last_unstale":"2024-09-19T05:02:26.424811+0000","last_undegraded":"2024-09-19T05:02:26.424811+0000","last_fullsized":"2024-09-19T05:02:26.424811+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:44:00.893313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.428369+0000","last_change":"2024-09-19T05:02:24.424935+0000","last_active":"2024-09-19T05:02:26.428369+0000","last_peered":"2024-09-19T05:02:26.428369+0000","last_clean":"2024-09-19T05:02:26.428369+0000","last_became_active":"2024-09-19T05:02:24.424219+0000","last_became_peered":"2024-09-19T05:02:24.424219+0000","last_unstale":"2024-09-19T05:02:26.428369+0000","last_undegraded":"2024-09-19T05:02:26.428369+0000","last_fullsized":"2024-09-19T05:02:26.428369+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:00.890778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.427577+0000","last_change":"2024-09-19T05:02:24.425407+0000","last_active":"2024-09-19T05:02:26.427577+0000","last_peered":"2024-09-19T05:02:26.427577+0000","last_clean":"2024-09-19T05:02:26.427577+0000","last_became_active":"2024-09-19T05:02:24.425053+0000","last_became_peered":"2024-09-19T05:02:24.425053+0000","last_unstale":"2024-09-19T05:02:26.427577+0000","last_undegraded":"2024-09-19T05:02:26.427577+0000","last_fullsized":"2024-09-19T05:02:26.427577+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:54:10.024183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.415130+0000","last_change":"2024-09-19T05:02:24.427888+0000","last_active":"2024-09-19T05:02:26.415130+0000","last_peered":"2024-09-19T05:02:26.415130+0000","last_clean":"2024-09-19T05:02:26.415130+0000","last_became_active":"2024-09-19T05:02:24.427589+0000","last_became_peered":"2024-09-19T05:02:24.427589+0000","last_unstale":"2024-09-19T05:02:26.415130+0000","last_undegraded":"2024-09-19T05:02:26.415130+0000","last_fullsized":"2024-09-19T05:02:26.415130+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:02:02.654416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.427843+0000","last_change":"2024-09-19T05:02:24.426654+0000","last_active":"2024-09-19T05:02:26.427843+0000","last_peered":"2024-09-19T05:02:26.427843+0000","last_clean":"2024-09-19T05:02:26.427843+0000","last_became_active":"2024-09-19T05:02:24.426352+0000","last_became_peered":"2024-09-19T05:02:24.426352+0000","last_unstale":"2024-09-19T05:02:26.427843+0000","last_undegraded":"2024-09-19T05:02:26.427843+0000","last_fullsized":"2024-09-19T05:02:26.427843+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:20.055222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.423666+0000","last_change":"2024-09-19T05:02:24.432178+0000","last_active":"2024-09-19T05:02:26.423666+0000","last_peered":"2024-09-19T05:02:26.423666+0000","last_clean":"2024-09-19T05:02:26.423666+0000","last_became_active":"2024-09-19T05:02:24.429058+0000","last_became_peered":"2024-09-19T05:02:24.429058+0000","last_unstale":"2024-09-19T05:02:26.423666+0000","last_undegraded":"2024-09-19T05:02:26.423666+0000","last_fullsized":"2024-09-19T05:02:26.423666+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:08.001134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.417818+0000","last_change":"2024-09-19T05:02:24.428202+0000","last_active":"2024-09-19T05:02:26.417818+0000","last_peered":"2024-09-19T05:02:26.417818+0000","last_clean":"2024-09-19T05:02:26.417818+0000","last_became_active":"2024-09-19T05:02:24.427741+0000","last_became_peered":"2024-09-19T05:02:24.427741+0000","last_unstale":"2024-09-19T05:02:26.417818+0000","last_undegraded":"2024-09-19T05:02:26.417818+0000","last_fullsized":"2024-09-19T05:02:26.417818+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:42:51.076056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.426673+0000","last_change":"2024-09-19T05:02:24.425007+0000","last_active":"2024-09-19T05:02:26.426673+0000","last_peered":"2024-09-19T05:02:26.426673+0000","last_clean":"2024-09-19T05:02:26.426673+0000","last_became_active":"2024-09-19T05:02:24.424612+0000","last_became_peered":"2024-09-19T05:02:24.424612+0000","last_unstale":"2024-09-19T05:02:26.426673+0000","last_undegraded":"2024-09-19T05:02:26.426673+0000","last_fullsized":"2024-09-19T05:02:26.426673+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:36:06.115757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.424994+0000","last_change":"2024-09-19T05:02:24.428724+0000","last_active":"2024-09-19T05:02:26.424994+0000","last_peered":"2024-09-19T05:02:26.424994+0000","last_clean":"2024-09-19T05:02:26.424994+0000","last_became_active":"2024-09-19T05:02:24.427730+0000","last_became_peered":"2024-09-19T05:02:24.427730+0000","last_unstale":"2024-09-19T05:02:26.424994+0000","last_undegraded":"2024-09-19T05:02:26.424994+0000","last_fullsized":"2024-09-19T05:02:26.424994+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:15:25.466553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.415384+0000","last_change":"2024-09-19T05:02:24.426238+0000","last_active":"2024-09-19T05:02:26.415384+0000","last_peered":"2024-09-19T05:02:26.415384+0000","last_clean":"2024-09-19T05:02:26.415384+0000","last_became_active":"2024-09-19T05:02:24.425915+0000","last_became_peered":"2024-09-19T05:02:24.425915+0000","last_unstale":"2024-09-19T05:02:26.415384+0000","last_undegraded":"2024-09-19T05:02:26.415384+0000","last_fullsized":"2024-09-19T05:02:26.415384+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:37:18.444042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.415085+0000","last_change":"2024-09-19T05:02:24.425726+0000","last_active":"2024-09-19T05:02:26.415085+0000","last_peered":"2024-09-19T05:02:26.415085+0000","last_clean":"2024-09-19T05:02:26.415085+0000","last_became_active":"2024-09-19T05:02:24.425409+0000","last_became_peered":"2024-09-19T05:02:24.425409+0000","last_unstale":"2024-09-19T05:02:26.415085+0000","last_undegraded":"2024-09-19T05:02:26.415085+0000","last_fullsized":"2024-09-19T05:02:26.415085+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:12.068304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.425175+0000","last_change":"2024-09-19T05:02:24.425267+0000","last_active":"2024-09-19T05:02:26.425175+0000","last_peered":"2024-09-19T05:02:26.425175+0000","last_clean":"2024-09-19T05:02:26.425175+0000","last_became_active":"2024-09-19T05:02:24.424933+0000","last_became_peered":"2024-09-19T05:02:24.424933+0000","last_unstale":"2024-09-19T05:02:26.425175+0000","last_undegraded":"2024-09-19T05:02:26.425175+0000","last_fullsized":"2024-09-19T05:02:26.425175+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:51:55.274870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.415742+0000","last_change":"2024-09-19T05:02:24.427909+0000","last_active":"2024-09-19T05:02:26.415742+0000","last_peered":"2024-09-19T05:02:26.415742+0000","last_clean":"2024-09-19T05:02:26.415742+0000","last_became_active":"2024-09-19T05:02:24.425515+0000","last_became_peered":"2024-09-19T05:02:24.425515+0000","last_unstale":"2024-09-19T05:02:26.415742+0000","last_undegraded":"2024-09-19T05:02:26.415742+0000","last_fullsized":"2024-09-19T05:02:26.415742+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:12:22.390754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.418042+0000","last_change":"2024-09-19T05:02:24.426201+0000","last_active":"2024-09-19T05:02:26.418042+0000","last_peered":"2024-09-19T05:02:26.418042+0000","last_clean":"2024-09-19T05:02:26.418042+0000","last_became_active":"2024-09-19T05:02:24.425890+0000","last_became_peered":"2024-09-19T05:02:24.425890+0000","last_unstale":"2024-09-19T05:02:26.418042+0000","last_undegraded":"2024-09-19T05:02:26.418042+0000","last_fullsized":"2024-09-19T05:02:26.418042+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:36.680002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.427337+0000","last_change":"2024-09-19T05:02:24.416037+0000","last_active":"2024-09-19T05:02:26.427337+0000","last_peered":"2024-09-19T05:02:26.427337+0000","last_clean":"2024-09-19T05:02:26.427337+0000","last_became_active":"2024-09-19T05:02:24.415535+0000","last_became_peered":"2024-09-19T05:02:24.415535+0000","last_unstale":"2024-09-19T05:02:26.427337+0000","last_undegraded":"2024-09-19T05:02:26.427337+0000","last_fullsized":"2024-09-19T05:02:26.427337+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:14.756410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.422856+0000","last_change":"2024-09-19T05:02:24.431137+0000","last_active":"2024-09-19T05:02:26.422856+0000","last_peered":"2024-09-19T05:02:26.422856+0000","last_clean":"2024-09-19T05:02:26.422856+0000","last_became_active":"2024-09-19T05:02:24.424962+0000","last_became_peered":"2024-09-19T05:02:24.424962+0000","last_unstale":"2024-09-19T05:02:26.422856+0000","last_undegraded":"2024-09-19T05:02:26.422856+0000","last_fullsized":"2024-09-19T05:02:26.422856+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:36:39.192239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.429147+0000","last_change":"2024-09-19T05:02:24.423776+0000","last_active":"2024-09-19T05:02:26.429147+0000","last_peered":"2024-09-19T05:02:26.429147+0000","last_clean":"2024-09-19T05:02:26.429147+0000","last_became_active":"2024-09-19T05:02:24.423446+0000","last_became_peered":"2024-09-19T05:02:24.423446+0000","last_unstale":"2024-09-19T05:02:26.429147+0000","last_undegraded":"2024-09-19T05:02:26.429147+0000","last_fullsized":"2024-09-19T05:02:26.429147+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:47:02.747049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.417615+0000","last_change":"2024-09-19T05:02:24.420534+0000","last_active":"2024-09-19T05:02:26.417615+0000","last_peered":"2024-09-19T05:02:26.417615+0000","last_clean":"2024-09-19T05:02:26.417615+0000","last_became_active":"2024-09-19T05:02:24.419980+0000","last_became_peered":"2024-09-19T05:02:24.419980+0000","last_unstale":"2024-09-19T05:02:26.417615+0000","last_undegraded":"2024-09-19T05:02:26.417615+0000","last_fullsized":"2024-09-19T05:02:26.417615+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:33:49.537258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.417972+0000","last_change":"2024-09-19T05:02:24.424859+0000","last_active":"2024-09-19T05:02:26.417972+0000","last_peered":"2024-09-19T05:02:26.417972+0000","last_clean":"2024-09-19T05:02:26.417972+0000","last_became_active":"2024-09-19T05:02:24.424514+0000","last_became_peered":"2024-09-19T05:02:24.424514+0000","last_unstale":"2024-09-19T05:02:26.417972+0000","last_undegraded":"2024-09-19T05:02:26.417972+0000","last_fullsized":"2024-09-19T05:02:26.417972+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:41:41.929578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.426176+0000","last_change":"2024-09-19T05:02:24.421756+0000","last_active":"2024-09-19T05:02:26.426176+0000","last_peered":"2024-09-19T05:02:26.426176+0000","last_clean":"2024-09-19T05:02:26.426176+0000","last_became_active":"2024-09-19T05:02:24.420910+0000","last_became_peered":"2024-09-19T05:02:24.420910+0000","last_unstale":"2024-09-19T05:02:26.426176+0000","last_undegraded":"2024-09-19T05:02:26.426176+0000","last_fullsized":"2024-09-19T05:02:26.426176+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:33.759745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.423804+0000","last_change":"2024-09-19T05:02:24.428514+0000","last_active":"2024-09-19T05:02:26.423804+0000","last_peered":"2024-09-19T05:02:26.423804+0000","last_clean":"2024-09-19T05:02:26.423804+0000","last_became_active":"2024-09-19T05:02:24.428206+0000","last_became_peered":"2024-09-19T05:02:24.428206+0000","last_unstale":"2024-09-19T05:02:26.423804+0000","last_undegraded":"2024-09-19T05:02:26.423804+0000","last_fullsized":"2024-09-19T05:02:26.423804+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:33.045739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:50.528984+0000","last_change":"2024-09-19T05:02:22.026683+0000","last_active":"2024-09-19T05:02:50.528984+0000","last_peered":"2024-09-19T05:02:50.528984+0000","last_clean":"2024-09-19T05:02:50.528984+0000","last_became_active":"2024-09-19T05:02:22.026342+0000","last_became_peered":"2024-09-19T05:02:22.026342+0000","last_unstale":"2024-09-19T05:02:50.528984+0000","last_undegraded":"2024-09-19T05:02:50.528984+0000","last_fullsized":"2024-09-19T05:02:50.528984+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:01:37.462838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.422187+0000","last_change":"2024-09-19T05:02:24.421753+0000","last_active":"2024-09-19T05:02:26.422187+0000","last_peered":"2024-09-19T05:02:26.422187+0000","last_clean":"2024-09-19T05:02:26.422187+0000","last_became_active":"2024-09-19T05:02:24.421455+0000","last_became_peered":"2024-09-19T05:02:24.421455+0000","last_unstale":"2024-09-19T05:02:26.422187+0000","last_undegraded":"2024-09-19T05:02:26.422187+0000","last_fullsized":"2024-09-19T05:02:26.422187+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:06.416550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:37.493191+0000","last_change":"2024-09-19T05:02:22.027188+0000","last_active":"2024-09-19T05:02:37.493191+0000","last_peered":"2024-09-19T05:02:37.493191+0000","last_clean":"2024-09-19T05:02:37.493191+0000","last_became_active":"2024-09-19T05:02:22.026414+0000","last_became_peered":"2024-09-19T05:02:22.026414+0000","last_unstale":"2024-09-19T05:02:37.493191+0000","last_undegraded":"2024-09-19T05:02:37.493191+0000","last_fullsized":"2024-09-19T05:02:37.493191+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:37:21.777913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:36.486962+0000","last_change":"2024-09-19T05:02:22.026599+0000","last_active":"2024-09-19T05:02:36.486962+0000","last_peered":"2024-09-19T05:02:36.486962+0000","last_clean":"2024-09-19T05:02:36.486962+0000","last_became_active":"2024-09-19T05:02:22.024652+0000","last_became_peered":"2024-09-19T05:02:22.024652+0000","last_unstale":"2024-09-19T05:02:36.486962+0000","last_undegraded":"2024-09-19T05:02:36.486962+0000","last_fullsized":"2024-09-19T05:02:36.486962+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:14:32.508333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.415203+0000","last_change":"2024-09-19T05:02:24.419262+0000","last_active":"2024-09-19T05:02:26.415203+0000","last_peered":"2024-09-19T05:02:26.415203+0000","last_clean":"2024-09-19T05:02:26.415203+0000","last_became_active":"2024-09-19T05:02:24.418589+0000","last_became_peered":"2024-09-19T05:02:24.418589+0000","last_unstale":"2024-09-19T05:02:26.415203+0000","last_undegraded":"2024-09-19T05:02:26.415203+0000","last_fullsized":"2024-09-19T05:02:26.415203+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:27.256676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:38.499696+0000","last_change":"2024-09-19T05:02:22.024225+0000","last_active":"2024-09-19T05:02:38.499696+0000","last_peered":"2024-09-19T05:02:38.499696+0000","last_clean":"2024-09-19T05:02:38.499696+0000","last_became_active":"2024-09-19T05:02:22.023882+0000","last_became_peered":"2024-09-19T05:02:22.023882+0000","last_unstale":"2024-09-19T05:02:38.499696+0000","last_undegraded":"2024-09-19T05:02:38.499696+0000","last_fullsized":"2024-09-19T05:02:38.499696+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:29.619634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.423285+0000","last_change":"2024-09-19T05:02:24.436511+0000","last_active":"2024-09-19T05:02:26.423285+0000","last_peered":"2024-09-19T05:02:26.423285+0000","last_clean":"2024-09-19T05:02:26.423285+0000","last_became_active":"2024-09-19T05:02:24.435565+0000","last_became_peered":"2024-09-19T05:02:24.435565+0000","last_unstale":"2024-09-19T05:02:26.423285+0000","last_undegraded":"2024-09-19T05:02:26.423285+0000","last_fullsized":"2024-09-19T05:02:26.423285+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:16:43.779420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":56,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:54.534182+0000","last_change":"2024-09-19T05:02:22.024281+0000","last_active":"2024-09-19T05:02:54.534182+0000","last_peered":"2024-09-19T05:02:54.534182+0000","last_clean":"2024-09-19T05:02:54.534182+0000","last_became_active":"2024-09-19T05:02:22.023868+0000","last_became_peered":"2024-09-19T05:02:22.023868+0000","last_unstale":"2024-09-19T05:02:54.534182+0000","last_undegraded":"2024-09-19T05:02:54.534182+0000","last_fullsized":"2024-09-19T05:02:54.534182+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:29:47.597570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.415817+0000","last_change":"2024-09-19T05:02:24.436383+0000","last_active":"2024-09-19T05:02:26.415817+0000","last_peered":"2024-09-19T05:02:26.415817+0000","last_clean":"2024-09-19T05:02:26.415817+0000","last_became_active":"2024-09-19T05:02:24.436062+0000","last_became_peered":"2024-09-19T05:02:24.436062+0000","last_unstale":"2024-09-19T05:02:26.415817+0000","last_undegraded":"2024-09-19T05:02:26.415817+0000","last_fullsized":"2024-09-19T05:02:26.415817+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:48:42.574431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:38.499021+0000","last_change":"2024-09-19T05:02:22.026950+0000","last_active":"2024-09-19T05:02:38.499021+0000","last_peered":"2024-09-19T05:02:38.499021+0000","last_clean":"2024-09-19T05:02:38.499021+0000","last_became_active":"2024-09-19T05:02:22.026498+0000","last_became_peered":"2024-09-19T05:02:22.026498+0000","last_unstale":"2024-09-19T05:02:38.499021+0000","last_undegraded":"2024-09-19T05:02:38.499021+0000","last_fullsized":"2024-09-19T05:02:38.499021+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:55:31.833714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.430242+0000","last_change":"2024-09-19T05:02:24.428958+0000","last_active":"2024-09-19T05:02:26.430242+0000","last_peered":"2024-09-19T05:02:26.430242+0000","last_clean":"2024-09-19T05:02:26.430242+0000","last_became_active":"2024-09-19T05:02:24.428203+0000","last_became_peered":"2024-09-19T05:02:24.428203+0000","last_unstale":"2024-09-19T05:02:26.430242+0000","last_undegraded":"2024-09-19T05:02:26.430242+0000","last_fullsized":"2024-09-19T05:02:26.430242+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:59:37.225357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:37.493615+0000","last_change":"2024-09-19T05:02:22.075901+0000","last_active":"2024-09-19T05:02:37.493615+0000","last_peered":"2024-09-19T05:02:37.493615+0000","last_clean":"2024-09-19T05:02:37.493615+0000","last_became_active":"2024-09-19T05:02:22.075512+0000","last_became_peered":"2024-09-19T05:02:22.075512+0000","last_unstale":"2024-09-19T05:02:37.493615+0000","last_undegraded":"2024-09-19T05:02:37.493615+0000","last_fullsized":"2024-09-19T05:02:37.493615+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:09:26.976888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.415511+0000","last_change":"2024-09-19T05:02:24.436797+0000","last_active":"2024-09-19T05:02:26.415511+0000","last_peered":"2024-09-19T05:02:26.415511+0000","last_clean":"2024-09-19T05:02:26.415511+0000","last_became_active":"2024-09-19T05:02:24.436457+0000","last_became_peered":"2024-09-19T05:02:24.436457+0000","last_unstale":"2024-09-19T05:02:26.415511+0000","last_undegraded":"2024-09-19T05:02:26.415511+0000","last_fullsized":"2024-09-19T05:02:26.415511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:32:44.702601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'27","reported_seq":86,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:54.542867+0000","last_change":"2024-09-19T05:02:22.027415+0000","last_active":"2024-09-19T05:02:54.542867+0000","last_peered":"2024-09-19T05:02:54.542867+0000","last_clean":"2024-09-19T05:02:54.542867+0000","last_became_active":"2024-09-19T05:02:22.027065+0000","last_became_peered":"2024-09-19T05:02:22.027065+0000","last_unstale":"2024-09-19T05:02:54.542867+0000","last_undegraded":"2024-09-19T05:02:54.542867+0000","last_fullsized":"2024-09-19T05:02:54.542867+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:02.599310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54445,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":39,"num_read_kb":71,"num_write":29,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.421996+0000","last_change":"2024-09-19T05:02:24.419718+0000","last_active":"2024-09-19T05:02:26.421996+0000","last_peered":"2024-09-19T05:02:26.421996+0000","last_clean":"2024-09-19T05:02:26.421996+0000","last_became_active":"2024-09-19T05:02:24.419388+0000","last_became_peered":"2024-09-19T05:02:24.419388+0000","last_unstale":"2024-09-19T05:02:26.421996+0000","last_undegraded":"2024-09-19T05:02:26.421996+0000","last_fullsized":"2024-09-19T05:02:26.421996+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:09.453551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:39.511271+0000","last_change":"2024-09-19T05:02:22.026220+0000","last_active":"2024-09-19T05:02:39.511271+0000","last_peered":"2024-09-19T05:02:39.511271+0000","last_clean":"2024-09-19T05:02:39.511271+0000","last_became_active":"2024-09-19T05:02:22.025838+0000","last_became_peered":"2024-09-19T05:02:22.025838+0000","last_unstale":"2024-09-19T05:02:39.511271+0000","last_undegraded":"2024-09-19T05:02:39.511271+0000","last_fullsized":"2024-09-19T05:02:39.511271+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:02.192799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.424167+0000","last_change":"2024-09-19T05:02:24.431872+0000","last_active":"2024-09-19T05:02:26.424167+0000","last_peered":"2024-09-19T05:02:26.424167+0000","last_clean":"2024-09-19T05:02:26.424167+0000","last_became_active":"2024-09-19T05:02:24.427227+0000","last_became_peered":"2024-09-19T05:02:24.427227+0000","last_unstale":"2024-09-19T05:02:26.424167+0000","last_undegraded":"2024-09-19T05:02:26.424167+0000","last_fullsized":"2024-09-19T05:02:26.424167+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:52.655108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:38.499901+0000","last_change":"2024-09-19T05:02:22.025136+0000","last_active":"2024-09-19T05:02:38.499901+0000","last_peered":"2024-09-19T05:02:38.499901+0000","last_clean":"2024-09-19T05:02:38.499901+0000","last_became_active":"2024-09-19T05:02:22.024813+0000","last_became_peered":"2024-09-19T05:02:22.024813+0000","last_unstale":"2024-09-19T05:02:38.499901+0000","last_undegraded":"2024-09-19T05:02:38.499901+0000","last_fullsized":"2024-09-19T05:02:38.499901+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:47:03.956661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.425183+0000","last_change":"2024-09-19T05:02:24.428497+0000","last_active":"2024-09-19T05:02:26.425183+0000","last_peered":"2024-09-19T05:02:26.425183+0000","last_clean":"2024-09-19T05:02:26.425183+0000","last_became_active":"2024-09-19T05:02:24.427774+0000","last_became_peered":"2024-09-19T05:02:24.427774+0000","last_unstale":"2024-09-19T05:02:26.425183+0000","last_undegraded":"2024-09-19T05:02:26.425183+0000","last_fullsized":"2024-09-19T05:02:26.425183+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:32:02.160354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'6","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:54.534225+0000","last_change":"2024-09-19T05:02:22.023789+0000","last_active":"2024-09-19T05:02:54.534225+0000","last_peered":"2024-09-19T05:02:54.534225+0000","last_clean":"2024-09-19T05:02:54.534225+0000","last_became_active":"2024-09-19T05:02:22.023371+0000","last_became_peered":"2024-09-19T05:02:22.023371+0000","last_unstale":"2024-09-19T05:02:54.534225+0000","last_undegraded":"2024-09-19T05:02:54.534225+0000","last_fullsized":"2024-09-19T05:02:54.534225+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:26:29.644878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":21,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.424639+0000","last_change":"2024-09-19T05:02:24.433098+0000","last_active":"2024-09-19T05:02:26.424639+0000","last_peered":"2024-09-19T05:02:26.424639+0000","last_clean":"2024-09-19T05:02:26.424639+0000","last_became_active":"2024-09-19T05:02:24.432800+0000","last_became_peered":"2024-09-19T05:02:24.432800+0000","last_unstale":"2024-09-19T05:02:26.424639+0000","last_undegraded":"2024-09-19T05:02:26.424639+0000","last_fullsized":"2024-09-19T05:02:26.424639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:50:30.163842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:38.500001+0000","last_change":"2024-09-19T05:02:22.075865+0000","last_active":"2024-09-19T05:02:38.500001+0000","last_peered":"2024-09-19T05:02:38.500001+0000","last_clean":"2024-09-19T05:02:38.500001+0000","last_became_active":"2024-09-19T05:02:22.075185+0000","last_became_peered":"2024-09-19T05:02:22.075185+0000","last_unstale":"2024-09-19T05:02:38.500001+0000","last_undegraded":"2024-09-19T05:02:38.500001+0000","last_fullsized":"2024-09-19T05:02:38.500001+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:19.150155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.421855+0000","last_change":"2024-09-19T05:02:24.422759+0000","last_active":"2024-09-19T05:02:26.421855+0000","last_peered":"2024-09-19T05:02:26.421855+0000","last_clean":"2024-09-19T05:02:26.421855+0000","last_became_active":"2024-09-19T05:02:24.422421+0000","last_became_peered":"2024-09-19T05:02:24.422421+0000","last_unstale":"2024-09-19T05:02:26.421855+0000","last_undegraded":"2024-09-19T05:02:26.421855+0000","last_fullsized":"2024-09-19T05:02:26.421855+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:58:50.083017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:38.480917+0000","last_change":"2024-09-19T05:02:22.024305+0000","last_active":"2024-09-19T05:02:38.480917+0000","last_peered":"2024-09-19T05:02:38.480917+0000","last_clean":"2024-09-19T05:02:38.480917+0000","last_became_active":"2024-09-19T05:02:22.023616+0000","last_became_peered":"2024-09-19T05:02:22.023616+0000","last_unstale":"2024-09-19T05:02:38.480917+0000","last_undegraded":"2024-09-19T05:02:38.480917+0000","last_fullsized":"2024-09-19T05:02:38.480917+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:05.665219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.427007+0000","last_change":"2024-09-19T05:02:24.423859+0000","last_active":"2024-09-19T05:02:26.427007+0000","last_peered":"2024-09-19T05:02:26.427007+0000","last_clean":"2024-09-19T05:02:26.427007+0000","last_became_active":"2024-09-19T05:02:24.423517+0000","last_became_peered":"2024-09-19T05:02:24.423517+0000","last_unstale":"2024-09-19T05:02:26.427007+0000","last_undegraded":"2024-09-19T05:02:26.427007+0000","last_fullsized":"2024-09-19T05:02:26.427007+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:39:50.474469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:38.500041+0000","last_change":"2024-09-19T05:02:22.024194+0000","last_active":"2024-09-19T05:02:38.500041+0000","last_peered":"2024-09-19T05:02:38.500041+0000","last_clean":"2024-09-19T05:02:38.500041+0000","last_became_active":"2024-09-19T05:02:22.023777+0000","last_became_peered":"2024-09-19T05:02:22.023777+0000","last_unstale":"2024-09-19T05:02:38.500041+0000","last_undegraded":"2024-09-19T05:02:38.500041+0000","last_fullsized":"2024-09-19T05:02:38.500041+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:37:51.814732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.425694+0000","last_change":"2024-09-19T05:02:24.419360+0000","last_active":"2024-09-19T05:02:26.425694+0000","last_peered":"2024-09-19T05:02:26.425694+0000","last_clean":"2024-09-19T05:02:26.425694+0000","last_became_active":"2024-09-19T05:02:24.419040+0000","last_became_peered":"2024-09-19T05:02:24.419040+0000","last_unstale":"2024-09-19T05:02:26.425694+0000","last_undegraded":"2024-09-19T05:02:26.425694+0000","last_fullsized":"2024-09-19T05:02:26.425694+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:48:12.922676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:38.499231+0000","last_change":"2024-09-19T05:02:22.023124+0000","last_active":"2024-09-19T05:02:38.499231+0000","last_peered":"2024-09-19T05:02:38.499231+0000","last_clean":"2024-09-19T05:02:38.499231+0000","last_became_active":"2024-09-19T05:02:22.022830+0000","last_became_peered":"2024-09-19T05:02:22.022830+0000","last_unstale":"2024-09-19T05:02:38.499231+0000","last_undegraded":"2024-09-19T05:02:38.499231+0000","last_fullsized":"2024-09-19T05:02:38.499231+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:22:13.208616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.425279+0000","last_change":"2024-09-19T05:02:24.426169+0000","last_active":"2024-09-19T05:02:26.425279+0000","last_peered":"2024-09-19T05:02:26.425279+0000","last_clean":"2024-09-19T05:02:26.425279+0000","last_became_active":"2024-09-19T05:02:24.425816+0000","last_became_peered":"2024-09-19T05:02:24.425816+0000","last_unstale":"2024-09-19T05:02:26.425279+0000","last_undegraded":"2024-09-19T05:02:26.425279+0000","last_fullsized":"2024-09-19T05:02:26.425279+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:56:11.735351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'6","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:50.529124+0000","last_change":"2024-09-19T05:02:22.154391+0000","last_active":"2024-09-19T05:02:50.529124+0000","last_peered":"2024-09-19T05:02:50.529124+0000","last_clean":"2024-09-19T05:02:50.529124+0000","last_became_active":"2024-09-19T05:02:22.154008+0000","last_became_peered":"2024-09-19T05:02:22.154008+0000","last_unstale":"2024-09-19T05:02:50.529124+0000","last_undegraded":"2024-09-19T05:02:50.529124+0000","last_fullsized":"2024-09-19T05:02:50.529124+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:29.997649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.425785+0000","last_change":"2024-09-19T05:02:24.427416+0000","last_active":"2024-09-19T05:02:26.425785+0000","last_peered":"2024-09-19T05:02:26.425785+0000","last_clean":"2024-09-19T05:02:26.425785+0000","last_became_active":"2024-09-19T05:02:24.426645+0000","last_became_peered":"2024-09-19T05:02:24.426645+0000","last_unstale":"2024-09-19T05:02:26.425785+0000","last_undegraded":"2024-09-19T05:02:26.425785+0000","last_fullsized":"2024-09-19T05:02:26.425785+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:21.841013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":344,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.425141+0000","last_change":"2024-09-19T05:01:04.407740+0000","last_active":"2024-09-19T05:02:26.425141+0000","last_peered":"2024-09-19T05:02:26.425141+0000","last_clean":"2024-09-19T05:02:26.425141+0000","last_became_active":"2024-09-19T05:01:04.407281+0000","last_became_peered":"2024-09-19T05:01:04.407281+0000","last_unstale":"2024-09-19T05:02:26.425141+0000","last_undegraded":"2024-09-19T05:02:26.425141+0000","last_fullsized":"2024-09-19T05:02:26.425141+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:57:27.643438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:57:27.643438+0000","last_clean_scrub_stamp":"2024-09-19T04:57:27.643438+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:54:50.242926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:39.509813+0000","last_change":"2024-09-19T05:02:22.023991+0000","last_active":"2024-09-19T05:02:39.509813+0000","last_peered":"2024-09-19T05:02:39.509813+0000","last_clean":"2024-09-19T05:02:39.509813+0000","last_became_active":"2024-09-19T05:02:22.023361+0000","last_became_peered":"2024-09-19T05:02:22.023361+0000","last_unstale":"2024-09-19T05:02:39.509813+0000","last_undegraded":"2024-09-19T05:02:39.509813+0000","last_fullsized":"2024-09-19T05:02:39.509813+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:37:05.551318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.415403+0000","last_change":"2024-09-19T05:02:24.424473+0000","last_active":"2024-09-19T05:02:26.415403+0000","last_peered":"2024-09-19T05:02:26.415403+0000","last_clean":"2024-09-19T05:02:26.415403+0000","last_became_active":"2024-09-19T05:02:24.423968+0000","last_became_peered":"2024-09-19T05:02:24.423968+0000","last_unstale":"2024-09-19T05:02:26.415403+0000","last_undegraded":"2024-09-19T05:02:26.415403+0000","last_fullsized":"2024-09-19T05:02:26.415403+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:29:53.305036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:36.484572+0000","last_change":"2024-09-19T05:02:22.026278+0000","last_active":"2024-09-19T05:02:36.484572+0000","last_peered":"2024-09-19T05:02:36.484572+0000","last_clean":"2024-09-19T05:02:36.484572+0000","last_became_active":"2024-09-19T05:02:22.025759+0000","last_became_peered":"2024-09-19T05:02:22.025759+0000","last_unstale":"2024-09-19T05:02:36.484572+0000","last_undegraded":"2024-09-19T05:02:36.484572+0000","last_fullsized":"2024-09-19T05:02:36.484572+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:21:21.696382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.422877+0000","last_change":"2024-09-19T05:02:24.426880+0000","last_active":"2024-09-19T05:02:26.422877+0000","last_peered":"2024-09-19T05:02:26.422877+0000","last_clean":"2024-09-19T05:02:26.422877+0000","last_became_active":"2024-09-19T05:02:24.426336+0000","last_became_peered":"2024-09-19T05:02:24.426336+0000","last_unstale":"2024-09-19T05:02:26.422877+0000","last_undegraded":"2024-09-19T05:02:26.422877+0000","last_fullsized":"2024-09-19T05:02:26.422877+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:51:07.110738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:54.533603+0000","last_change":"2024-09-19T05:02:22.075789+0000","last_active":"2024-09-19T05:02:54.533603+0000","last_peered":"2024-09-19T05:02:54.533603+0000","last_clean":"2024-09-19T05:02:54.533603+0000","last_became_active":"2024-09-19T05:02:22.075415+0000","last_became_peered":"2024-09-19T05:02:22.075415+0000","last_unstale":"2024-09-19T05:02:54.533603+0000","last_undegraded":"2024-09-19T05:02:54.533603+0000","last_fullsized":"2024-09-19T05:02:54.533603+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:11:43.843627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.425383+0000","last_change":"2024-09-19T05:02:24.422661+0000","last_active":"2024-09-19T05:02:26.425383+0000","last_peered":"2024-09-19T05:02:26.425383+0000","last_clean":"2024-09-19T05:02:26.425383+0000","last_became_active":"2024-09-19T05:02:24.422313+0000","last_became_peered":"2024-09-19T05:02:24.422313+0000","last_unstale":"2024-09-19T05:02:26.425383+0000","last_undegraded":"2024-09-19T05:02:26.425383+0000","last_fullsized":"2024-09-19T05:02:26.425383+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:07:39.729388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'6","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:53.535317+0000","last_change":"2024-09-19T05:02:22.028008+0000","last_active":"2024-09-19T05:02:53.535317+0000","last_peered":"2024-09-19T05:02:53.535317+0000","last_clean":"2024-09-19T05:02:53.535317+0000","last_became_active":"2024-09-19T05:02:22.027635+0000","last_became_peered":"2024-09-19T05:02:22.027635+0000","last_unstale":"2024-09-19T05:02:53.535317+0000","last_undegraded":"2024-09-19T05:02:53.535317+0000","last_fullsized":"2024-09-19T05:02:53.535317+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:41:36.902282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.421366+0000","last_change":"2024-09-19T05:02:24.435362+0000","last_active":"2024-09-19T05:02:26.421366+0000","last_peered":"2024-09-19T05:02:26.421366+0000","last_clean":"2024-09-19T05:02:26.421366+0000","last_became_active":"2024-09-19T05:02:24.435056+0000","last_became_peered":"2024-09-19T05:02:24.435056+0000","last_unstale":"2024-09-19T05:02:26.421366+0000","last_undegraded":"2024-09-19T05:02:26.421366+0000","last_fullsized":"2024-09-19T05:02:26.421366+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:58:40.092466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.424059+0000","last_change":"2024-09-19T05:02:22.077651+0000","last_active":"2024-09-19T05:02:26.424059+0000","last_peered":"2024-09-19T05:02:26.424059+0000","last_clean":"2024-09-19T05:02:26.424059+0000","last_became_active":"2024-09-19T05:02:22.076273+0000","last_became_peered":"2024-09-19T05:02:22.076273+0000","last_unstale":"2024-09-19T05:02:26.424059+0000","last_undegraded":"2024-09-19T05:02:26.424059+0000","last_fullsized":"2024-09-19T05:02:26.424059+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:28:42.064126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.421778+0000","last_change":"2024-09-19T05:02:24.438125+0000","last_active":"2024-09-19T05:02:26.421778+0000","last_peered":"2024-09-19T05:02:26.421778+0000","last_clean":"2024-09-19T05:02:26.421778+0000","last_became_active":"2024-09-19T05:02:24.437806+0000","last_became_peered":"2024-09-19T05:02:24.437806+0000","last_unstale":"2024-09-19T05:02:26.421778+0000","last_undegraded":"2024-09-19T05:02:26.421778+0000","last_fullsized":"2024-09-19T05:02:26.421778+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:46:08.338501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:38.500675+0000","last_change":"2024-09-19T05:02:22.075920+0000","last_active":"2024-09-19T05:02:38.500675+0000","last_peered":"2024-09-19T05:02:38.500675+0000","last_clean":"2024-09-19T05:02:38.500675+0000","last_became_active":"2024-09-19T05:02:22.075609+0000","last_became_peered":"2024-09-19T05:02:22.075609+0000","last_unstale":"2024-09-19T05:02:38.500675+0000","last_undegraded":"2024-09-19T05:02:38.500675+0000","last_fullsized":"2024-09-19T05:02:38.500675+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:45.915625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.423025+0000","last_change":"2024-09-19T05:02:24.434681+0000","last_active":"2024-09-19T05:02:26.423025+0000","last_peered":"2024-09-19T05:02:26.423025+0000","last_clean":"2024-09-19T05:02:26.423025+0000","last_became_active":"2024-09-19T05:02:24.434257+0000","last_became_peered":"2024-09-19T05:02:24.434257+0000","last_unstale":"2024-09-19T05:02:26.423025+0000","last_undegraded":"2024-09-19T05:02:26.423025+0000","last_fullsized":"2024-09-19T05:02:26.423025+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:06:09.133112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'14","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:53.424214+0000","last_change":"2024-09-19T05:02:22.024314+0000","last_active":"2024-09-19T05:02:53.424214+0000","last_peered":"2024-09-19T05:02:53.424214+0000","last_clean":"2024-09-19T05:02:53.424214+0000","last_became_active":"2024-09-19T05:02:22.023733+0000","last_became_peered":"2024-09-19T05:02:22.023733+0000","last_unstale":"2024-09-19T05:02:53.424214+0000","last_undegraded":"2024-09-19T05:02:53.424214+0000","last_fullsized":"2024-09-19T05:02:53.424214+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:39:31.251257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.415756+0000","last_change":"2024-09-19T05:02:24.429807+0000","last_active":"2024-09-19T05:02:26.415756+0000","last_peered":"2024-09-19T05:02:26.415756+0000","last_clean":"2024-09-19T05:02:26.415756+0000","last_became_active":"2024-09-19T05:02:24.429477+0000","last_became_peered":"2024-09-19T05:02:24.429477+0000","last_unstale":"2024-09-19T05:02:26.415756+0000","last_undegraded":"2024-09-19T05:02:26.415756+0000","last_fullsized":"2024-09-19T05:02:26.415756+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:02.314851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:38.499588+0000","last_change":"2024-09-19T05:02:22.154534+0000","last_active":"2024-09-19T05:02:38.499588+0000","last_peered":"2024-09-19T05:02:38.499588+0000","last_clean":"2024-09-19T05:02:38.499588+0000","last_became_active":"2024-09-19T05:02:22.154238+0000","last_became_peered":"2024-09-19T05:02:22.154238+0000","last_unstale":"2024-09-19T05:02:38.499588+0000","last_undegraded":"2024-09-19T05:02:38.499588+0000","last_fullsized":"2024-09-19T05:02:38.499588+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:32:30.963503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.429485+0000","last_change":"2024-09-19T05:02:24.422817+0000","last_active":"2024-09-19T05:02:26.429485+0000","last_peered":"2024-09-19T05:02:26.429485+0000","last_clean":"2024-09-19T05:02:26.429485+0000","last_became_active":"2024-09-19T05:02:24.422478+0000","last_became_peered":"2024-09-19T05:02:24.422478+0000","last_unstale":"2024-09-19T05:02:26.429485+0000","last_undegraded":"2024-09-19T05:02:26.429485+0000","last_fullsized":"2024-09-19T05:02:26.429485+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:15:35.442149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:35.481268+0000","last_change":"2024-09-19T05:02:22.074396+0000","last_active":"2024-09-19T05:02:35.481268+0000","last_peered":"2024-09-19T05:02:35.481268+0000","last_clean":"2024-09-19T05:02:35.481268+0000","last_became_active":"2024-09-19T05:02:22.074108+0000","last_became_peered":"2024-09-19T05:02:22.074108+0000","last_unstale":"2024-09-19T05:02:35.481268+0000","last_undegraded":"2024-09-19T05:02:35.481268+0000","last_fullsized":"2024-09-19T05:02:35.481268+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:58:32.718702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.421653+0000","last_change":"2024-09-19T05:02:24.429599+0000","last_active":"2024-09-19T05:02:26.421653+0000","last_peered":"2024-09-19T05:02:26.421653+0000","last_clean":"2024-09-19T05:02:26.421653+0000","last_became_active":"2024-09-19T05:02:24.429268+0000","last_became_peered":"2024-09-19T05:02:24.429268+0000","last_unstale":"2024-09-19T05:02:26.421653+0000","last_undegraded":"2024-09-19T05:02:26.421653+0000","last_fullsized":"2024-09-19T05:02:26.421653+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:57.123516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:38.498197+0000","last_change":"2024-09-19T05:02:22.021900+0000","last_active":"2024-09-19T05:02:38.498197+0000","last_peered":"2024-09-19T05:02:38.498197+0000","last_clean":"2024-09-19T05:02:38.498197+0000","last_became_active":"2024-09-19T05:02:22.020576+0000","last_became_peered":"2024-09-19T05:02:22.020576+0000","last_unstale":"2024-09-19T05:02:38.498197+0000","last_undegraded":"2024-09-19T05:02:38.498197+0000","last_fullsized":"2024-09-19T05:02:38.498197+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:44:14.466475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.426331+0000","last_change":"2024-09-19T05:02:24.420362+0000","last_active":"2024-09-19T05:02:26.426331+0000","last_peered":"2024-09-19T05:02:26.426331+0000","last_clean":"2024-09-19T05:02:26.426331+0000","last_became_active":"2024-09-19T05:02:24.420047+0000","last_became_peered":"2024-09-19T05:02:24.420047+0000","last_unstale":"2024-09-19T05:02:26.426331+0000","last_undegraded":"2024-09-19T05:02:26.426331+0000","last_fullsized":"2024-09-19T05:02:26.426331+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:28:56.977946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:37.493114+0000","last_change":"2024-09-19T05:02:22.024565+0000","last_active":"2024-09-19T05:02:37.493114+0000","last_peered":"2024-09-19T05:02:37.493114+0000","last_clean":"2024-09-19T05:02:37.493114+0000","last_became_active":"2024-09-19T05:02:22.024246+0000","last_became_peered":"2024-09-19T05:02:22.024246+0000","last_unstale":"2024-09-19T05:02:37.493114+0000","last_undegraded":"2024-09-19T05:02:37.493114+0000","last_fullsized":"2024-09-19T05:02:37.493114+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:58:27.491975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.415472+0000","last_change":"2024-09-19T05:02:24.428046+0000","last_active":"2024-09-19T05:02:26.415472+0000","last_peered":"2024-09-19T05:02:26.415472+0000","last_clean":"2024-09-19T05:02:26.415472+0000","last_became_active":"2024-09-19T05:02:24.427724+0000","last_became_peered":"2024-09-19T05:02:24.427724+0000","last_unstale":"2024-09-19T05:02:26.415472+0000","last_undegraded":"2024-09-19T05:02:26.415472+0000","last_fullsized":"2024-09-19T05:02:26.415472+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:11:54.252876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:35.481601+0000","last_change":"2024-09-19T05:02:22.023054+0000","last_active":"2024-09-19T05:02:35.481601+0000","last_peered":"2024-09-19T05:02:35.481601+0000","last_clean":"2024-09-19T05:02:35.481601+0000","last_became_active":"2024-09-19T05:02:22.022700+0000","last_became_peered":"2024-09-19T05:02:22.022700+0000","last_unstale":"2024-09-19T05:02:35.481601+0000","last_undegraded":"2024-09-19T05:02:35.481601+0000","last_fullsized":"2024-09-19T05:02:35.481601+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:44:19.714938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.425315+0000","last_change":"2024-09-19T05:02:24.423045+0000","last_active":"2024-09-19T05:02:26.425315+0000","last_peered":"2024-09-19T05:02:26.425315+0000","last_clean":"2024-09-19T05:02:26.425315+0000","last_became_active":"2024-09-19T05:02:24.422719+0000","last_became_peered":"2024-09-19T05:02:24.422719+0000","last_unstale":"2024-09-19T05:02:26.425315+0000","last_undegraded":"2024-09-19T05:02:26.425315+0000","last_fullsized":"2024-09-19T05:02:26.425315+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:45.299603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.422314+0000","last_change":"2024-09-19T05:02:24.426942+0000","last_active":"2024-09-19T05:02:26.422314+0000","last_peered":"2024-09-19T05:02:26.422314+0000","last_clean":"2024-09-19T05:02:26.422314+0000","last_became_active":"2024-09-19T05:02:24.425896+0000","last_became_peered":"2024-09-19T05:02:24.425896+0000","last_unstale":"2024-09-19T05:02:26.422314+0000","last_undegraded":"2024-09-19T05:02:26.422314+0000","last_fullsized":"2024-09-19T05:02:26.422314+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:54:15.384309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:38.499891+0000","last_change":"2024-09-19T05:02:22.021415+0000","last_active":"2024-09-19T05:02:38.499891+0000","last_peered":"2024-09-19T05:02:38.499891+0000","last_clean":"2024-09-19T05:02:38.499891+0000","last_became_active":"2024-09-19T05:02:22.020508+0000","last_became_peered":"2024-09-19T05:02:22.020508+0000","last_unstale":"2024-09-19T05:02:38.499891+0000","last_undegraded":"2024-09-19T05:02:38.499891+0000","last_fullsized":"2024-09-19T05:02:38.499891+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:35.912386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.415456+0000","last_change":"2024-09-19T05:02:24.435538+0000","last_active":"2024-09-19T05:02:26.415456+0000","last_peered":"2024-09-19T05:02:26.415456+0000","last_clean":"2024-09-19T05:02:26.415456+0000","last_became_active":"2024-09-19T05:02:24.435209+0000","last_became_peered":"2024-09-19T05:02:24.435209+0000","last_unstale":"2024-09-19T05:02:26.415456+0000","last_undegraded":"2024-09-19T05:02:26.415456+0000","last_fullsized":"2024-09-19T05:02:26.415456+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:55:30.075113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:39.511585+0000","last_change":"2024-09-19T05:02:22.024153+0000","last_active":"2024-09-19T05:02:39.511585+0000","last_peered":"2024-09-19T05:02:39.511585+0000","last_clean":"2024-09-19T05:02:39.511585+0000","last_became_active":"2024-09-19T05:02:22.023879+0000","last_became_peered":"2024-09-19T05:02:22.023879+0000","last_unstale":"2024-09-19T05:02:39.511585+0000","last_undegraded":"2024-09-19T05:02:39.511585+0000","last_fullsized":"2024-09-19T05:02:39.511585+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:39:32.878079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.430074+0000","last_change":"2024-09-19T05:02:24.428958+0000","last_active":"2024-09-19T05:02:26.430074+0000","last_peered":"2024-09-19T05:02:26.430074+0000","last_clean":"2024-09-19T05:02:26.430074+0000","last_became_active":"2024-09-19T05:02:24.428672+0000","last_became_peered":"2024-09-19T05:02:24.428672+0000","last_unstale":"2024-09-19T05:02:26.430074+0000","last_undegraded":"2024-09-19T05:02:26.430074+0000","last_fullsized":"2024-09-19T05:02:26.430074+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:06:23.529706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.428814+0000","last_change":"2024-09-19T05:02:22.077034+0000","last_active":"2024-09-19T05:02:26.428814+0000","last_peered":"2024-09-19T05:02:26.428814+0000","last_clean":"2024-09-19T05:02:26.428814+0000","last_became_active":"2024-09-19T05:02:22.076182+0000","last_became_peered":"2024-09-19T05:02:22.076182+0000","last_unstale":"2024-09-19T05:02:26.428814+0000","last_undegraded":"2024-09-19T05:02:26.428814+0000","last_fullsized":"2024-09-19T05:02:26.428814+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:43:31.076464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:26.415813+0000","last_change":"2024-09-19T05:02:24.423765+0000","last_active":"2024-09-19T05:02:26.415813+0000","last_peered":"2024-09-19T05:02:26.415813+0000","last_clean":"2024-09-19T05:02:26.415813+0000","last_became_active":"2024-09-19T05:02:24.423445+0000","last_became_peered":"2024-09-19T05:02:24.423445+0000","last_unstale":"2024-09-19T05:02:26.415813+0000","last_undegraded":"2024-09-19T05:02:26.415813+0000","last_fullsized":"2024-09-19T05:02:26.415813+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:10:07.806697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'13","reported_seq":51,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-19T05:02:54.535955+0000","last_change":"2024-09-19T05:02:22.023413+0000","last_active":"2024-09-19T05:02:54.535955+0000","last_peered":"2024-09-19T05:02:54.535955+0000","last_clean":"2024-09-19T05:02:54.535955+0000","last_became_active":"2024-09-19T05:02:22.023078+0000","last_became_peered":"2024-09-19T05:02:22.023078+0000","last_unstale":"2024-09-19T05:02:54.535955+0000","last_undegraded":"2024-09-19T05:02:54.535955+0000","last_fullsized":"2024-09-19T05:02:54.535955+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:11:09.185011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62439,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":198,"num_read_kb":171,"num_write":241,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":175075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":207,"ondisk_log_size":207,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285699,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514516,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":435933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743449,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":1081407,"data_compressed":36330,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906974,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":435470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135907,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":435848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397545,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":1029154,"data_compressed":36330,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561070,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":435580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790003,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":995328,"data_stored":1028046,"data_compressed":36330,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986232,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":435441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215166,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":933888,"data_stored":435012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411395,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":479309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":974848,"data_stored":490938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738446,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":937984,"data_stored":436919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34683,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34683,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34683,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36330,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36330,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36330,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36330,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36330,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":56590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":44961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":56574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T05:02:59.460 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T05:02:59.460 INFO:tasks.cephadm:Setting up client nodes... 2024-09-19T05:02:59.461 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-19T05:02:59.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:02:59 smithi043 ceph-mon[30800]: from='client.24664 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:02:59.860 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:02:59.910 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:02:59 smithi151 ceph-mon[35425]: from='client.24664 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:02:59.983 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:02:59 smithi096 ceph-mon[35085]: from='client.24664 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:03:00.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.393+0000 7ff46c200640 1 -- 172.21.15.43:0/2057765736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff464104f60 msgr2=0x7ff4641053e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:00.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.393+0000 7ff46c200640 1 --2- 172.21.15.43:0/2057765736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff464104f60 0x7ff4641053e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff450009900 tx=0x7ff45002a410 comp rx=0 tx=0).stop 2024-09-19T05:03:00.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.393+0000 7ff46c200640 1 -- 172.21.15.43:0/2057765736 shutdown_connections 2024-09-19T05:03:00.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.393+0000 7ff46c200640 1 --2- 172.21.15.43:0/2057765736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff464105920 0x7ff46410c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:00.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.393+0000 7ff46c200640 1 --2- 172.21.15.43:0/2057765736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff464104f60 0x7ff4641053e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:00.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.393+0000 7ff46c200640 1 --2- 172.21.15.43:0/2057765736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff464103e00 0x7ff464104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:00.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.393+0000 7ff46c200640 1 -- 172.21.15.43:0/2057765736 >> 172.21.15.43:0/2057765736 conn(0x7ff4640ff5b0 msgr2=0x7ff4641019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:00.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.393+0000 7ff46c200640 1 -- 172.21.15.43:0/2057765736 shutdown_connections 2024-09-19T05:03:00.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.393+0000 7ff46c200640 1 -- 172.21.15.43:0/2057765736 wait complete. 2024-09-19T05:03:00.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.394+0000 7ff46c200640 1 Processor -- start 2024-09-19T05:03:00.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.395+0000 7ff46c200640 1 -- start start 2024-09-19T05:03:00.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.395+0000 7ff46c200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff464103e00 0x7ff46419e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:00.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.395+0000 7ff46c200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff464104f60 0x7ff46419ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:00.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.395+0000 7ff46c200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff464105920 0x7ff4641a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:00.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.395+0000 7ff46c200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff46410fae0 con 0x7ff464103e00 2024-09-19T05:03:00.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.395+0000 7ff46c200640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff46410f960 con 0x7ff464104f60 2024-09-19T05:03:00.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.395+0000 7ff46c200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff46410fc60 con 0x7ff464105920 2024-09-19T05:03:00.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.395+0000 7ff469800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff464103e00 0x7ff46419e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:00.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.395+0000 7ff469800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff464103e00 0x7ff46419e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59154/0 (socket says 172.21.15.43:59154) 2024-09-19T05:03:00.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.395+0000 7ff469800640 1 -- 172.21.15.43:0/3522992331 learned_addr learned my addr 172.21.15.43:0/3522992331 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:00.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.395+0000 7ff468e00640 1 --2- 172.21.15.43:0/3522992331 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff464104f60 0x7ff46419ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:00.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.395+0000 7ff46a200640 1 --2- 172.21.15.43:0/3522992331 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff464105920 0x7ff4641a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:00.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.396+0000 7ff46a200640 1 -- 172.21.15.43:0/3522992331 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff464104f60 msgr2=0x7ff46419ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:00.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.396+0000 7ff46a200640 1 --2- 172.21.15.43:0/3522992331 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff464104f60 0x7ff46419ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:00.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.396+0000 7ff46a200640 1 -- 172.21.15.43:0/3522992331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff464103e00 msgr2=0x7ff46419e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:00.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.396+0000 7ff46a200640 1 --2- 172.21.15.43:0/3522992331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff464103e00 0x7ff46419e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:00.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.396+0000 7ff46a200640 1 -- 172.21.15.43:0/3522992331 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4641a6160 con 0x7ff464105920 2024-09-19T05:03:00.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.396+0000 7ff469800640 1 --2- 172.21.15.43:0/3522992331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff464103e00 0x7ff46419e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:03:00.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.396+0000 7ff46a200640 1 --2- 172.21.15.43:0/3522992331 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff464105920 0x7ff4641a5af0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff45c009fd0 tx=0x7ff45c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:00.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.396+0000 7ff462000640 1 -- 172.21.15.43:0/3522992331 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c00ce20 con 0x7ff464105920 2024-09-19T05:03:00.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.396+0000 7ff46c200640 1 -- 172.21.15.43:0/3522992331 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4641a6450 con 0x7ff464105920 2024-09-19T05:03:00.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.396+0000 7ff462000640 1 -- 172.21.15.43:0/3522992331 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff45c005020 con 0x7ff464105920 2024-09-19T05:03:00.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.396+0000 7ff462000640 1 -- 172.21.15.43:0/3522992331 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c010720 con 0x7ff464105920 2024-09-19T05:03:00.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.397+0000 7ff46c200640 1 -- 172.21.15.43:0/3522992331 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4641a6960 con 0x7ff464105920 2024-09-19T05:03:00.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.398+0000 7ff46c200640 1 -- 172.21.15.43:0/3522992331 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4641a6c20 con 0x7ff464105920 2024-09-19T05:03:00.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.400+0000 7ff462000640 1 -- 172.21.15.43:0/3522992331 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7ff45c005540 con 0x7ff464105920 2024-09-19T05:03:00.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.402+0000 7ff462000640 1 --2- 172.21.15.43:0/3522992331 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff44c0a3300 0x7ff44c0a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:00.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.402+0000 7ff462000640 1 -- 172.21.15.43:0/3522992331 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7ff45c0d00b0 con 0x7ff464105920 2024-09-19T05:03:00.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.402+0000 7ff469800640 1 --2- 172.21.15.43:0/3522992331 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff44c0a3300 0x7ff44c0a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:00.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.403+0000 7ff469800640 1 --2- 172.21.15.43:0/3522992331 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff44c0a3300 0x7ff44c0a57c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff458011550 tx=0x7ff458005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:00.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.405+0000 7ff462000640 1 -- 172.21.15.43:0/3522992331 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff45c09cd20 con 0x7ff464105920 2024-09-19T05:03:00.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.658+0000 7ff46c200640 1 -- 172.21.15.43:0/3522992331 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7ff4641a6df0 con 0x7ff464105920 2024-09-19T05:03:00.661 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:00 smithi043 ceph-mon[30800]: pgmap v309: 97 pgs: 97 active+clean; 638 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-09-19T05:03:00.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.664+0000 7ff462000640 1 -- 172.21.15.43:0/3522992331 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7ff45c09c620 con 0x7ff464105920 2024-09-19T05:03:00.666 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2024-09-19T05:03:00.666 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCEsOtmtwZtJxAA9IeEnzaM8NnhbTIMZ8wcnw== 2024-09-19T05:03:00.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.666+0000 7ff46c200640 1 -- 172.21.15.43:0/3522992331 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff44c0a3300 msgr2=0x7ff44c0a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:00.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.666+0000 7ff46c200640 1 --2- 172.21.15.43:0/3522992331 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff44c0a3300 0x7ff44c0a57c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff458011550 tx=0x7ff458005f90 comp rx=0 tx=0).stop 2024-09-19T05:03:00.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.666+0000 7ff46c200640 1 -- 172.21.15.43:0/3522992331 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff464105920 msgr2=0x7ff4641a5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:00.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.666+0000 7ff46c200640 1 --2- 172.21.15.43:0/3522992331 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff464105920 0x7ff4641a5af0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff45c009fd0 tx=0x7ff45c00ef10 comp rx=0 tx=0).stop 2024-09-19T05:03:00.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.666+0000 7ff46c200640 1 -- 172.21.15.43:0/3522992331 shutdown_connections 2024-09-19T05:03:00.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.666+0000 7ff46c200640 1 --2- 172.21.15.43:0/3522992331 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff44c0a3300 0x7ff44c0a57c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:00.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.666+0000 7ff46c200640 1 --2- 172.21.15.43:0/3522992331 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff464105920 0x7ff4641a5af0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:00.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.666+0000 7ff46c200640 1 --2- 172.21.15.43:0/3522992331 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff464104f60 0x7ff46419ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:00.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.666+0000 7ff46c200640 1 --2- 172.21.15.43:0/3522992331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff464103e00 0x7ff46419e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:00.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.666+0000 7ff46c200640 1 -- 172.21.15.43:0/3522992331 >> 172.21.15.43:0/3522992331 conn(0x7ff4640ff5b0 msgr2=0x7ff464078640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:00.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.666+0000 7ff46c200640 1 -- 172.21.15.43:0/3522992331 shutdown_connections 2024-09-19T05:03:00.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:00.667+0000 7ff46c200640 1 -- 172.21.15.43:0/3522992331 wait complete. 2024-09-19T05:03:00.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:00 smithi151 ceph-mon[35425]: pgmap v309: 97 pgs: 97 active+clean; 638 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-09-19T05:03:00.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:00 smithi096 ceph-mon[35085]: pgmap v309: 97 pgs: 97 active+clean; 638 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-09-19T05:03:01.539 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-19T05:03:01.539 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-19T05:03:01.539 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-19T05:03:01.624 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-19T05:03:01.624 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-19T05:03:01.624 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-19T05:03:01.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:01 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/3522992331' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T05:03:01.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:01 smithi043 ceph-mon[30800]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T05:03:01.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:01 smithi043 ceph-mon[30800]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T05:03:01.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:01 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/3522992331' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T05:03:01.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:01 smithi151 ceph-mon[35425]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T05:03:01.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:01 smithi151 ceph-mon[35425]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T05:03:01.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:01 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/3522992331' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T05:03:01.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:01 smithi096 ceph-mon[35085]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T05:03:01.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:01 smithi096 ceph-mon[35085]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T05:03:02.042 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:02.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.573+0000 7fda86400640 1 -- 172.21.15.43:0/1916055034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda801049c0 msgr2=0x7fda80108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:02.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.573+0000 7fda86400640 1 --2- 172.21.15.43:0/1916055034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda801049c0 0x7fda80108a10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fda6c009900 tx=0x7fda6c02a410 comp rx=0 tx=0).stop 2024-09-19T05:03:02.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.573+0000 7fda86400640 1 -- 172.21.15.43:0/1916055034 shutdown_connections 2024-09-19T05:03:02.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.573+0000 7fda86400640 1 --2- 172.21.15.43:0/1916055034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda80109140 0x7fda80110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:02.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.573+0000 7fda86400640 1 --2- 172.21.15.43:0/1916055034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda801049c0 0x7fda80108a10 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:02.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.573+0000 7fda86400640 1 --2- 172.21.15.43:0/1916055034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda80104010 0x7fda801043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:02.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.573+0000 7fda86400640 1 -- 172.21.15.43:0/1916055034 >> 172.21.15.43:0/1916055034 conn(0x7fda800ffa50 msgr2=0x7fda80101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:02.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.573+0000 7fda86400640 1 -- 172.21.15.43:0/1916055034 shutdown_connections 2024-09-19T05:03:02.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.573+0000 7fda86400640 1 -- 172.21.15.43:0/1916055034 wait complete. 2024-09-19T05:03:02.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.574+0000 7fda86400640 1 Processor -- start 2024-09-19T05:03:02.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.574+0000 7fda86400640 1 -- start start 2024-09-19T05:03:02.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.574+0000 7fda86400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda80104010 0x7fda8019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:02.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.574+0000 7fda86400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda801049c0 0x7fda8019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:02.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.575+0000 7fda86400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda80109140 0x7fda8019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:02.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.575+0000 7fda86400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fda80114190 con 0x7fda80104010 2024-09-19T05:03:02.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.575+0000 7fda86400640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fda80114010 con 0x7fda801049c0 2024-09-19T05:03:02.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.575+0000 7fda86400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fda80114310 con 0x7fda80109140 2024-09-19T05:03:02.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.575+0000 7fda7f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda80104010 0x7fda8019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:02.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.575+0000 7fda7f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda80104010 0x7fda8019a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59172/0 (socket says 172.21.15.43:59172) 2024-09-19T05:03:02.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.575+0000 7fda7f400640 1 -- 172.21.15.43:0/3812542427 learned_addr learned my addr 172.21.15.43:0/3812542427 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:02.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.575+0000 7fda7ea00640 1 --2- 172.21.15.43:0/3812542427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda801049c0 0x7fda8019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:02.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.575+0000 7fda7fe00640 1 --2- 172.21.15.43:0/3812542427 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda80109140 0x7fda8019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:02.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.575+0000 7fda7f400640 1 -- 172.21.15.43:0/3812542427 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda80109140 msgr2=0x7fda8019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:02.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.575+0000 7fda7f400640 1 --2- 172.21.15.43:0/3812542427 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda80109140 0x7fda8019ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:02.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.575+0000 7fda7f400640 1 -- 172.21.15.43:0/3812542427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda801049c0 msgr2=0x7fda8019ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:02.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.575+0000 7fda7f400640 1 --2- 172.21.15.43:0/3812542427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda801049c0 0x7fda8019ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:02.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.575+0000 7fda7f400640 1 -- 172.21.15.43:0/3812542427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda8019f600 con 0x7fda80104010 2024-09-19T05:03:02.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.576+0000 7fda7f400640 1 --2- 172.21.15.43:0/3812542427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda80104010 0x7fda8019a650 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fda7000d950 tx=0x7fda7000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:02.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.576+0000 7fda77e00640 1 -- 172.21.15.43:0/3812542427 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda70017030 con 0x7fda80104010 2024-09-19T05:03:02.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.576+0000 7fda77e00640 1 -- 172.21.15.43:0/3812542427 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda70004020 con 0x7fda80104010 2024-09-19T05:03:02.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.576+0000 7fda77e00640 1 -- 172.21.15.43:0/3812542427 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda700047a0 con 0x7fda80104010 2024-09-19T05:03:02.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.576+0000 7fda86400640 1 -- 172.21.15.43:0/3812542427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda8019f8f0 con 0x7fda80104010 2024-09-19T05:03:02.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.576+0000 7fda86400640 1 -- 172.21.15.43:0/3812542427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda801a71d0 con 0x7fda80104010 2024-09-19T05:03:02.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.577+0000 7fda86400640 1 -- 172.21.15.43:0/3812542427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda801043f0 con 0x7fda80104010 2024-09-19T05:03:02.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.579+0000 7fda77e00640 1 -- 172.21.15.43:0/3812542427 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fda700041d0 con 0x7fda80104010 2024-09-19T05:03:02.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.580+0000 7fda77e00640 1 --2- 172.21.15.43:0/3812542427 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fda640a3350 0x7fda640a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:02.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.581+0000 7fda77e00640 1 -- 172.21.15.43:0/3812542427 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7fda700cf2b0 con 0x7fda80104010 2024-09-19T05:03:02.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.581+0000 7fda7ea00640 1 --2- 172.21.15.43:0/3812542427 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fda640a3350 0x7fda640a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:02.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.581+0000 7fda7ea00640 1 --2- 172.21.15.43:0/3812542427 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fda640a3350 0x7fda640a5810 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fda6c000c00 tx=0x7fda6c005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:02.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.585+0000 7fda77e00640 1 -- 172.21.15.43:0/3812542427 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fda7009bf20 con 0x7fda80104010 2024-09-19T05:03:02.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:02 smithi043 ceph-mon[30800]: pgmap v310: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.8 KiB/s wr, 12 op/s 2024-09-19T05:03:02.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.820+0000 7fda86400640 1 -- 172.21.15.43:0/3812542427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fda8019b9b0 con 0x7fda80104010 2024-09-19T05:03:02.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.826+0000 7fda77e00640 1 -- 172.21.15.43:0/3812542427 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264049 (secure 0 0 0) 0x7fda7009b820 con 0x7fda80104010 2024-09-19T05:03:02.833 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:03:02.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.830+0000 7fda86400640 1 -- 172.21.15.43:0/3812542427 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fda640a3350 msgr2=0x7fda640a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:02.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.830+0000 7fda86400640 1 --2- 172.21.15.43:0/3812542427 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fda640a3350 0x7fda640a5810 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fda6c000c00 tx=0x7fda6c005b50 comp rx=0 tx=0).stop 2024-09-19T05:03:02.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.830+0000 7fda86400640 1 -- 172.21.15.43:0/3812542427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda80104010 msgr2=0x7fda8019a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:02.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.830+0000 7fda86400640 1 --2- 172.21.15.43:0/3812542427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda80104010 0x7fda8019a650 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fda7000d950 tx=0x7fda7000de10 comp rx=0 tx=0).stop 2024-09-19T05:03:02.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.830+0000 7fda86400640 1 -- 172.21.15.43:0/3812542427 shutdown_connections 2024-09-19T05:03:02.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.830+0000 7fda86400640 1 --2- 172.21.15.43:0/3812542427 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fda640a3350 0x7fda640a5810 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:02.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.830+0000 7fda86400640 1 --2- 172.21.15.43:0/3812542427 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda80109140 0x7fda8019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:02.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.830+0000 7fda86400640 1 --2- 172.21.15.43:0/3812542427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda801049c0 0x7fda8019ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:02.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.830+0000 7fda86400640 1 --2- 172.21.15.43:0/3812542427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda80104010 0x7fda8019a650 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:02.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.830+0000 7fda86400640 1 -- 172.21.15.43:0/3812542427 >> 172.21.15.43:0/3812542427 conn(0x7fda800ffa50 msgr2=0x7fda80100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:02.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.830+0000 7fda86400640 1 -- 172.21.15.43:0/3812542427 shutdown_connections 2024-09-19T05:03:02.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:02.830+0000 7fda86400640 1 -- 172.21.15.43:0/3812542427 wait complete. 2024-09-19T05:03:02.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:02 smithi096 ceph-mon[35085]: pgmap v310: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.8 KiB/s wr, 12 op/s 2024-09-19T05:03:03.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:02 smithi151 ceph-mon[35425]: pgmap v310: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.8 KiB/s wr, 12 op/s 2024-09-19T05:03:03.600 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3167108925},{"type":"v1","addr":"172.21.15.43:6801","nonce":3167108925}]},"active_addr":"172.21.15.43:6801/3167108925","active_change":"2024-09-19T04:55:07.037228+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.43:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":3819455723}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":3712369735}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":515912755}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":3730884138}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":4018217102}]}]} 2024-09-19T05:03:03.606 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-19T05:03:03.606 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-19T05:03:03.606 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T05:03:03.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:03 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/3812542427' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T05:03:03.985 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:04.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:03 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/3812542427' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T05:03:04.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:03 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/3812542427' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T05:03:04.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.523+0000 7fac1aa00640 1 -- 172.21.15.43:0/2363289709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac141079c0 msgr2=0x7fac14107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:04.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.523+0000 7fac1aa00640 1 --2- 172.21.15.43:0/2363289709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac141079c0 0x7fac14107da0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fac08009930 tx=0x7fac0802a420 comp rx=0 tx=0).stop 2024-09-19T05:03:04.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.523+0000 7fac1aa00640 1 -- 172.21.15.43:0/2363289709 shutdown_connections 2024-09-19T05:03:04.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.523+0000 7fac1aa00640 1 --2- 172.21.15.43:0/2363289709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac14102360 0x7fac1410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:04.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.523+0000 7fac1aa00640 1 --2- 172.21.15.43:0/2363289709 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac141019c0 0x7fac14101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:04.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.523+0000 7fac1aa00640 1 --2- 172.21.15.43:0/2363289709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac141079c0 0x7fac14107da0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:04.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.523+0000 7fac1aa00640 1 -- 172.21.15.43:0/2363289709 >> 172.21.15.43:0/2363289709 conn(0x7fac140fd740 msgr2=0x7fac140ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:04.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.523+0000 7fac1aa00640 1 -- 172.21.15.43:0/2363289709 shutdown_connections 2024-09-19T05:03:04.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.523+0000 7fac1aa00640 1 -- 172.21.15.43:0/2363289709 wait complete. 2024-09-19T05:03:04.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.524+0000 7fac1aa00640 1 Processor -- start 2024-09-19T05:03:04.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.524+0000 7fac1aa00640 1 -- start start 2024-09-19T05:03:04.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.525+0000 7fac1aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac141019c0 0x7fac140744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:04.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.525+0000 7fac1aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac14102360 0x7fac14074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:04.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.525+0000 7fac1aa00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac141079c0 0x7fac14074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:04.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.525+0000 7fac1aa00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fac14111e10 con 0x7fac14102360 2024-09-19T05:03:04.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.525+0000 7fac1aa00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fac14111c90 con 0x7fac141079c0 2024-09-19T05:03:04.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.525+0000 7fac1aa00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fac14111f90 con 0x7fac141019c0 2024-09-19T05:03:04.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.525+0000 7fac18c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac14102360 0x7fac14074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:04.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.525+0000 7fac18c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac14102360 0x7fac14074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59192/0 (socket says 172.21.15.43:59192) 2024-09-19T05:03:04.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.525+0000 7fac18c00640 1 -- 172.21.15.43:0/1021170114 learned_addr learned my addr 172.21.15.43:0/1021170114 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:04.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.525+0000 7fac19600640 1 --2- 172.21.15.43:0/1021170114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac141019c0 0x7fac140744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:04.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.525+0000 7fac1a000640 1 --2- 172.21.15.43:0/1021170114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac141079c0 0x7fac14074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:04.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.525+0000 7fac18c00640 1 -- 172.21.15.43:0/1021170114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac141019c0 msgr2=0x7fac140744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:04.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.525+0000 7fac18c00640 1 --2- 172.21.15.43:0/1021170114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac141019c0 0x7fac140744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:04.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.525+0000 7fac18c00640 1 -- 172.21.15.43:0/1021170114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac141079c0 msgr2=0x7fac14074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:04.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.525+0000 7fac18c00640 1 --2- 172.21.15.43:0/1021170114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac141079c0 0x7fac14074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:04.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.525+0000 7fac18c00640 1 -- 172.21.15.43:0/1021170114 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac141a6770 con 0x7fac14102360 2024-09-19T05:03:04.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.526+0000 7fac18c00640 1 --2- 172.21.15.43:0/1021170114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac14102360 0x7fac14074a30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fac0000d950 tx=0x7fac0000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:04.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.526+0000 7fac12000640 1 -- 172.21.15.43:0/1021170114 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac00017030 con 0x7fac14102360 2024-09-19T05:03:04.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.526+0000 7fac12000640 1 -- 172.21.15.43:0/1021170114 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac00004020 con 0x7fac14102360 2024-09-19T05:03:04.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.526+0000 7fac12000640 1 -- 172.21.15.43:0/1021170114 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac000047a0 con 0x7fac14102360 2024-09-19T05:03:04.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.526+0000 7fac1aa00640 1 -- 172.21.15.43:0/1021170114 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac141a6a60 con 0x7fac14102360 2024-09-19T05:03:04.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.526+0000 7fac1aa00640 1 -- 172.21.15.43:0/1021170114 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac141a6ff0 con 0x7fac14102360 2024-09-19T05:03:04.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.527+0000 7fac1aa00640 1 -- 172.21.15.43:0/1021170114 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac14101e20 con 0x7fac14102360 2024-09-19T05:03:04.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.529+0000 7fac12000640 1 -- 172.21.15.43:0/1021170114 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fac000041d0 con 0x7fac14102360 2024-09-19T05:03:04.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.531+0000 7fac12000640 1 --2- 172.21.15.43:0/1021170114 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fabfc0a3350 0x7fabfc0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:04.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.531+0000 7fac12000640 1 -- 172.21.15.43:0/1021170114 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7fac000cf2b0 con 0x7fac14102360 2024-09-19T05:03:04.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.531+0000 7fac19600640 1 --2- 172.21.15.43:0/1021170114 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fabfc0a3350 0x7fabfc0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:04.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.532+0000 7fac19600640 1 --2- 172.21.15.43:0/1021170114 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fabfc0a3350 0x7fabfc0a5810 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fac08006010 tx=0x7fac08000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:04.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.534+0000 7fac12000640 1 -- 172.21.15.43:0/1021170114 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fac0009bf20 con 0x7fac14102360 2024-09-19T05:03:04.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.724+0000 7fac1aa00640 1 -- 172.21.15.43:0/1021170114 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fac14109f30 con 0x7fac14102360 2024-09-19T05:03:04.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.725+0000 7fac12000640 1 -- 172.21.15.43:0/1021170114 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22237 (secure 0 0 0) 0x7fac0009b820 con 0x7fac14102360 2024-09-19T05:03:04.728 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:03:04.728 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":83,"fsid":"1456a810-7643-11ef-bceb-c7b262605968","created":"2024-09-19T04:54:14.706644+0000","modified":"2024-09-19T05:02:25.462897+0000","last_up_change":"2024-09-19T05:01:27.348740+0000","last_in_change":"2024-09-19T05:01:03.396477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:57:27.642705+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:02:20.685123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:02:23.393596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"81a317a8-7ec8-45c3-bc10-7da023283635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6803","nonce":825351486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6805","nonce":825351486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6809","nonce":825351486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6807","nonce":825351486}]},"public_addr":"172.21.15.43:6803/825351486","cluster_addr":"172.21.15.43:6805/825351486","heartbeat_back_addr":"172.21.15.43:6809/825351486","heartbeat_front_addr":"172.21.15.43:6807/825351486","state":["exists","up"]},{"osd":1,"uuid":"e96fcfac-4fff-4a36-b38b-aebf57842740","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6801","nonce":2140095130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6803","nonce":2140095130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6807","nonce":2140095130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6805","nonce":2140095130}]},"public_addr":"172.21.15.96:6801/2140095130","cluster_addr":"172.21.15.96:6803/2140095130","heartbeat_back_addr":"172.21.15.96:6807/2140095130","heartbeat_front_addr":"172.21.15.96:6805/2140095130","state":["exists","up"]},{"osd":2,"uuid":"b659869e-114a-4f56-8aac-a2ed602b4058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6801","nonce":2047147996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6803","nonce":2047147996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6807","nonce":2047147996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6805","nonce":2047147996}]},"public_addr":"172.21.15.151:6801/2047147996","cluster_addr":"172.21.15.151:6803/2047147996","heartbeat_back_addr":"172.21.15.151:6807/2047147996","heartbeat_front_addr":"172.21.15.151:6805/2047147996","state":["exists","up"]},{"osd":3,"uuid":"7be64b2c-e45c-493d-a610-4ce41196061f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6811","nonce":1734673762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6813","nonce":1734673762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6817","nonce":1734673762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6815","nonce":1734673762}]},"public_addr":"172.21.15.43:6811/1734673762","cluster_addr":"172.21.15.43:6813/1734673762","heartbeat_back_addr":"172.21.15.43:6817/1734673762","heartbeat_front_addr":"172.21.15.43:6815/1734673762","state":["exists","up"]},{"osd":4,"uuid":"196d5af1-a6e5-496c-8933-4e58892f4360","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6809","nonce":3184785362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6811","nonce":3184785362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6815","nonce":3184785362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6813","nonce":3184785362}]},"public_addr":"172.21.15.96:6809/3184785362","cluster_addr":"172.21.15.96:6811/3184785362","heartbeat_back_addr":"172.21.15.96:6815/3184785362","heartbeat_front_addr":"172.21.15.96:6813/3184785362","state":["exists","up"]},{"osd":5,"uuid":"5492d474-8509-4865-bc2e-d08e31558a75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6809","nonce":2878451924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6811","nonce":2878451924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6815","nonce":2878451924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6813","nonce":2878451924}]},"public_addr":"172.21.15.151:6809/2878451924","cluster_addr":"172.21.15.151:6811/2878451924","heartbeat_back_addr":"172.21.15.151:6815/2878451924","heartbeat_front_addr":"172.21.15.151:6813/2878451924","state":["exists","up"]},{"osd":6,"uuid":"fb8457ed-7bb9-4435-9d15-c1cbfbddfcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6819","nonce":2008630290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6821","nonce":2008630290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6825","nonce":2008630290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6823","nonce":2008630290}]},"public_addr":"172.21.15.43:6819/2008630290","cluster_addr":"172.21.15.43:6821/2008630290","heartbeat_back_addr":"172.21.15.43:6825/2008630290","heartbeat_front_addr":"172.21.15.43:6823/2008630290","state":["exists","up"]},{"osd":7,"uuid":"02046db8-30de-4e92-b25a-1f07d6190e91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6817","nonce":3588713275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6819","nonce":3588713275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6823","nonce":3588713275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6821","nonce":3588713275}]},"public_addr":"172.21.15.96:6817/3588713275","cluster_addr":"172.21.15.96:6819/3588713275","heartbeat_back_addr":"172.21.15.96:6823/3588713275","heartbeat_front_addr":"172.21.15.96:6821/3588713275","state":["exists","up"]},{"osd":8,"uuid":"a5bb4218-51f3-4044-b43e-ebfc764f49ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6817","nonce":725316393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6819","nonce":725316393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6823","nonce":725316393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6821","nonce":725316393}]},"public_addr":"172.21.15.151:6817/725316393","cluster_addr":"172.21.15.151:6819/725316393","heartbeat_back_addr":"172.21.15.151:6823/725316393","heartbeat_front_addr":"172.21.15.151:6821/725316393","state":["exists","up"]},{"osd":9,"uuid":"6c19f65c-9750-4baa-9d63-7b8bd03e6955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6827","nonce":4053438621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6829","nonce":4053438621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6833","nonce":4053438621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6831","nonce":4053438621}]},"public_addr":"172.21.15.43:6827/4053438621","cluster_addr":"172.21.15.43:6829/4053438621","heartbeat_back_addr":"172.21.15.43:6833/4053438621","heartbeat_front_addr":"172.21.15.43:6831/4053438621","state":["exists","up"]},{"osd":10,"uuid":"a7aa7da3-8eab-47fe-a351-298b6ff75f14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6825","nonce":2403210086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6827","nonce":2403210086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6831","nonce":2403210086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6829","nonce":2403210086}]},"public_addr":"172.21.15.96:6825/2403210086","cluster_addr":"172.21.15.96:6827/2403210086","heartbeat_back_addr":"172.21.15.96:6831/2403210086","heartbeat_front_addr":"172.21.15.96:6829/2403210086","state":["exists","up"]},{"osd":11,"uuid":"760b6afc-253d-44dd-88e9-dcc97f67c9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6825","nonce":292202881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6827","nonce":292202881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6831","nonce":292202881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6829","nonce":292202881}]},"public_addr":"172.21.15.151:6825/292202881","cluster_addr":"172.21.15.151:6827/292202881","heartbeat_back_addr":"172.21.15.151:6831/292202881","heartbeat_front_addr":"172.21.15.151:6829/292202881","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:56:34.716991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:56:59.321174+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:57:24.281696+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:57:53.443880+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:58:19.205873+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:58:44.504873+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:13.650731+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:39.137876+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:04.579893+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:34.337312+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:59.434658+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:01:24.930529+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/898247657":"2024-09-20T04:55:07.037117+0000","172.21.15.43:6801/3241799377":"2024-09-20T04:55:07.037117+0000","172.21.15.43:6800/3241799377":"2024-09-20T04:55:07.037117+0000","172.21.15.43:0/3256350596":"2024-09-20T04:55:07.037117+0000","172.21.15.43:6801/587463881":"2024-09-20T04:54:39.548065+0000","172.21.15.43:6800/587463881":"2024-09-20T04:54:39.548065+0000","172.21.15.43:0/4271788315":"2024-09-20T04:55:07.037117+0000","172.21.15.43:0/6132665":"2024-09-20T04:54:39.548065+0000","172.21.15.43:0/632163478":"2024-09-20T04:54:39.548065+0000","172.21.15.43:0/330854185":"2024-09-20T04:54:39.548065+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:03:04.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.728+0000 7fac1aa00640 1 -- 172.21.15.43:0/1021170114 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fabfc0a3350 msgr2=0x7fabfc0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:04.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.728+0000 7fac1aa00640 1 --2- 172.21.15.43:0/1021170114 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fabfc0a3350 0x7fabfc0a5810 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fac08006010 tx=0x7fac08000bc0 comp rx=0 tx=0).stop 2024-09-19T05:03:04.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.728+0000 7fac1aa00640 1 -- 172.21.15.43:0/1021170114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac14102360 msgr2=0x7fac14074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:04.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.728+0000 7fac1aa00640 1 --2- 172.21.15.43:0/1021170114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac14102360 0x7fac14074a30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fac0000d950 tx=0x7fac0000de10 comp rx=0 tx=0).stop 2024-09-19T05:03:04.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.728+0000 7fac1aa00640 1 -- 172.21.15.43:0/1021170114 shutdown_connections 2024-09-19T05:03:04.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.728+0000 7fac1aa00640 1 --2- 172.21.15.43:0/1021170114 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fabfc0a3350 0x7fabfc0a5810 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:04.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.728+0000 7fac1aa00640 1 --2- 172.21.15.43:0/1021170114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac141079c0 0x7fac14074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:04.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.728+0000 7fac1aa00640 1 --2- 172.21.15.43:0/1021170114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac14102360 0x7fac14074a30 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:04.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.728+0000 7fac1aa00640 1 --2- 172.21.15.43:0/1021170114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac141019c0 0x7fac140744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:04.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.728+0000 7fac1aa00640 1 -- 172.21.15.43:0/1021170114 >> 172.21.15.43:0/1021170114 conn(0x7fac140fd740 msgr2=0x7fac140ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:04.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.728+0000 7fac1aa00640 1 -- 172.21.15.43:0/1021170114 shutdown_connections 2024-09-19T05:03:04.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:04.728+0000 7fac1aa00640 1 -- 172.21.15.43:0/1021170114 wait complete. 2024-09-19T05:03:04.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:04 smithi096 ceph-mon[35085]: pgmap v311: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-09-19T05:03:05.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:04 smithi043 ceph-mon[30800]: pgmap v311: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-09-19T05:03:05.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:04 smithi151 ceph-mon[35425]: pgmap v311: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-09-19T05:03:05.526 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-19T05:03:05.527 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T05:03:05.908 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:05.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:05 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/1021170114' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:03:06.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:05 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/1021170114' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:03:06.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:05 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/1021170114' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:03:06.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.495+0000 7f69ff200640 1 -- 172.21.15.43:0/2095925250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69f8103ff0 msgr2=0x7f69f81043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:06.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.495+0000 7f69ff200640 1 --2- 172.21.15.43:0/2095925250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69f8103ff0 0x7f69f81043d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f69e8009900 tx=0x7f69e802a410 comp rx=0 tx=0).stop 2024-09-19T05:03:06.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.496+0000 7f69ff200640 1 -- 172.21.15.43:0/2095925250 shutdown_connections 2024-09-19T05:03:06.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.496+0000 7f69ff200640 1 --2- 172.21.15.43:0/2095925250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69f8109120 0x7f69f8110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:06.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.496+0000 7f69ff200640 1 --2- 172.21.15.43:0/2095925250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f69f81049a0 0x7f69f81089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:06.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.496+0000 7f69ff200640 1 --2- 172.21.15.43:0/2095925250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69f8103ff0 0x7f69f81043d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:06.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.496+0000 7f69ff200640 1 -- 172.21.15.43:0/2095925250 >> 172.21.15.43:0/2095925250 conn(0x7f69f80ffa90 msgr2=0x7f69f8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:06.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.496+0000 7f69ff200640 1 -- 172.21.15.43:0/2095925250 shutdown_connections 2024-09-19T05:03:06.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.496+0000 7f69ff200640 1 -- 172.21.15.43:0/2095925250 wait complete. 2024-09-19T05:03:06.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.497+0000 7f69ff200640 1 Processor -- start 2024-09-19T05:03:06.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.497+0000 7f69ff200640 1 -- start start 2024-09-19T05:03:06.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.498+0000 7f69ff200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69f8103ff0 0x7f69f819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:06.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.498+0000 7f69ff200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69f81049a0 0x7f69f819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:06.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.498+0000 7f69ff200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f69f8109120 0x7f69f819eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:06.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.498+0000 7f69ff200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f69f8114120 con 0x7f69f8103ff0 2024-09-19T05:03:06.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.498+0000 7f69ff200640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f69f8113fa0 con 0x7f69f8109120 2024-09-19T05:03:06.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.498+0000 7f69ff200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f69f81142a0 con 0x7f69f81049a0 2024-09-19T05:03:06.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.498+0000 7f69f7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69f8103ff0 0x7f69f819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:06.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.498+0000 7f69f7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69f8103ff0 0x7f69f819a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45906/0 (socket says 172.21.15.43:45906) 2024-09-19T05:03:06.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.498+0000 7f69f7e00640 1 -- 172.21.15.43:0/1194300703 learned_addr learned my addr 172.21.15.43:0/1194300703 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:06.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.498+0000 7f69efe00640 1 --2- 172.21.15.43:0/1194300703 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69f81049a0 0x7f69f819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:06.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.498+0000 7f69fd200640 1 --2- 172.21.15.43:0/1194300703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f69f8109120 0x7f69f819eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:06.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.498+0000 7f69f7e00640 1 -- 172.21.15.43:0/1194300703 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69f81049a0 msgr2=0x7f69f819ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:06.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.498+0000 7f69f7e00640 1 --2- 172.21.15.43:0/1194300703 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69f81049a0 0x7f69f819ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:06.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.498+0000 7f69f7e00640 1 -- 172.21.15.43:0/1194300703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f69f8109120 msgr2=0x7f69f819eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:06.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.498+0000 7f69f7e00640 1 --2- 172.21.15.43:0/1194300703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f69f8109120 0x7f69f819eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:06.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.498+0000 7f69f7e00640 1 -- 172.21.15.43:0/1194300703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69f819f590 con 0x7f69f8103ff0 2024-09-19T05:03:06.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.499+0000 7f69f7e00640 1 --2- 172.21.15.43:0/1194300703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69f8103ff0 0x7f69f819a5e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f69e8006010 tx=0x7f69e80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:06.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.499+0000 7f69f5600640 1 -- 172.21.15.43:0/1194300703 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e803f030 con 0x7f69f8103ff0 2024-09-19T05:03:06.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.499+0000 7f69f5600640 1 -- 172.21.15.43:0/1194300703 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69e8003ce0 con 0x7f69f8103ff0 2024-09-19T05:03:06.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.499+0000 7f69f5600640 1 -- 172.21.15.43:0/1194300703 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e8035070 con 0x7f69f8103ff0 2024-09-19T05:03:06.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.499+0000 7f69ff200640 1 -- 172.21.15.43:0/1194300703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69f819f820 con 0x7f69f8103ff0 2024-09-19T05:03:06.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.499+0000 7f69ff200640 1 -- 172.21.15.43:0/1194300703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69f81a7160 con 0x7f69f8103ff0 2024-09-19T05:03:06.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.501+0000 7f69ff200640 1 -- 172.21.15.43:0/1194300703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69f81054d0 con 0x7f69f8103ff0 2024-09-19T05:03:06.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.502+0000 7f69f5600640 1 -- 172.21.15.43:0/1194300703 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f69e8042030 con 0x7f69f8103ff0 2024-09-19T05:03:06.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.504+0000 7f69f5600640 1 --2- 172.21.15.43:0/1194300703 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f69dc0a32d0 0x7f69dc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:06.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.504+0000 7f69f5600640 1 -- 172.21.15.43:0/1194300703 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f69e80efb20 con 0x7f69f8103ff0 2024-09-19T05:03:06.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.504+0000 7f69efe00640 1 --2- 172.21.15.43:0/1194300703 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f69dc0a32d0 0x7f69dc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:06.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.505+0000 7f69efe00640 1 --2- 172.21.15.43:0/1194300703 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f69dc0a32d0 0x7f69dc0a5790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f69f80ffeb0 tx=0x7f69e000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:06.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.508+0000 7f69f5600640 1 -- 172.21.15.43:0/1194300703 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f69e80bc200 con 0x7f69f8103ff0 2024-09-19T05:03:06.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.696+0000 7f69ff200640 1 -- 172.21.15.43:0/1194300703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f69f810c300 con 0x7f69f8103ff0 2024-09-19T05:03:06.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.697+0000 7f69f5600640 1 -- 172.21.15.43:0/1194300703 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22237 (secure 0 0 0) 0x7f69e80c0500 con 0x7f69f8103ff0 2024-09-19T05:03:06.700 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:03:06.700 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":83,"fsid":"1456a810-7643-11ef-bceb-c7b262605968","created":"2024-09-19T04:54:14.706644+0000","modified":"2024-09-19T05:02:25.462897+0000","last_up_change":"2024-09-19T05:01:27.348740+0000","last_in_change":"2024-09-19T05:01:03.396477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:57:27.642705+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:02:20.685123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:02:23.393596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"81a317a8-7ec8-45c3-bc10-7da023283635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6803","nonce":825351486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6805","nonce":825351486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6809","nonce":825351486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6807","nonce":825351486}]},"public_addr":"172.21.15.43:6803/825351486","cluster_addr":"172.21.15.43:6805/825351486","heartbeat_back_addr":"172.21.15.43:6809/825351486","heartbeat_front_addr":"172.21.15.43:6807/825351486","state":["exists","up"]},{"osd":1,"uuid":"e96fcfac-4fff-4a36-b38b-aebf57842740","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6801","nonce":2140095130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6803","nonce":2140095130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6807","nonce":2140095130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6805","nonce":2140095130}]},"public_addr":"172.21.15.96:6801/2140095130","cluster_addr":"172.21.15.96:6803/2140095130","heartbeat_back_addr":"172.21.15.96:6807/2140095130","heartbeat_front_addr":"172.21.15.96:6805/2140095130","state":["exists","up"]},{"osd":2,"uuid":"b659869e-114a-4f56-8aac-a2ed602b4058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6801","nonce":2047147996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6803","nonce":2047147996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6807","nonce":2047147996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6805","nonce":2047147996}]},"public_addr":"172.21.15.151:6801/2047147996","cluster_addr":"172.21.15.151:6803/2047147996","heartbeat_back_addr":"172.21.15.151:6807/2047147996","heartbeat_front_addr":"172.21.15.151:6805/2047147996","state":["exists","up"]},{"osd":3,"uuid":"7be64b2c-e45c-493d-a610-4ce41196061f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6811","nonce":1734673762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6813","nonce":1734673762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6817","nonce":1734673762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6815","nonce":1734673762}]},"public_addr":"172.21.15.43:6811/1734673762","cluster_addr":"172.21.15.43:6813/1734673762","heartbeat_back_addr":"172.21.15.43:6817/1734673762","heartbeat_front_addr":"172.21.15.43:6815/1734673762","state":["exists","up"]},{"osd":4,"uuid":"196d5af1-a6e5-496c-8933-4e58892f4360","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6809","nonce":3184785362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6811","nonce":3184785362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6815","nonce":3184785362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6813","nonce":3184785362}]},"public_addr":"172.21.15.96:6809/3184785362","cluster_addr":"172.21.15.96:6811/3184785362","heartbeat_back_addr":"172.21.15.96:6815/3184785362","heartbeat_front_addr":"172.21.15.96:6813/3184785362","state":["exists","up"]},{"osd":5,"uuid":"5492d474-8509-4865-bc2e-d08e31558a75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6809","nonce":2878451924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6811","nonce":2878451924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6815","nonce":2878451924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6813","nonce":2878451924}]},"public_addr":"172.21.15.151:6809/2878451924","cluster_addr":"172.21.15.151:6811/2878451924","heartbeat_back_addr":"172.21.15.151:6815/2878451924","heartbeat_front_addr":"172.21.15.151:6813/2878451924","state":["exists","up"]},{"osd":6,"uuid":"fb8457ed-7bb9-4435-9d15-c1cbfbddfcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6819","nonce":2008630290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6821","nonce":2008630290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6825","nonce":2008630290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6823","nonce":2008630290}]},"public_addr":"172.21.15.43:6819/2008630290","cluster_addr":"172.21.15.43:6821/2008630290","heartbeat_back_addr":"172.21.15.43:6825/2008630290","heartbeat_front_addr":"172.21.15.43:6823/2008630290","state":["exists","up"]},{"osd":7,"uuid":"02046db8-30de-4e92-b25a-1f07d6190e91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6817","nonce":3588713275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6819","nonce":3588713275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6823","nonce":3588713275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6821","nonce":3588713275}]},"public_addr":"172.21.15.96:6817/3588713275","cluster_addr":"172.21.15.96:6819/3588713275","heartbeat_back_addr":"172.21.15.96:6823/3588713275","heartbeat_front_addr":"172.21.15.96:6821/3588713275","state":["exists","up"]},{"osd":8,"uuid":"a5bb4218-51f3-4044-b43e-ebfc764f49ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6817","nonce":725316393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6819","nonce":725316393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6823","nonce":725316393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6821","nonce":725316393}]},"public_addr":"172.21.15.151:6817/725316393","cluster_addr":"172.21.15.151:6819/725316393","heartbeat_back_addr":"172.21.15.151:6823/725316393","heartbeat_front_addr":"172.21.15.151:6821/725316393","state":["exists","up"]},{"osd":9,"uuid":"6c19f65c-9750-4baa-9d63-7b8bd03e6955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6827","nonce":4053438621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6829","nonce":4053438621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6833","nonce":4053438621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6831","nonce":4053438621}]},"public_addr":"172.21.15.43:6827/4053438621","cluster_addr":"172.21.15.43:6829/4053438621","heartbeat_back_addr":"172.21.15.43:6833/4053438621","heartbeat_front_addr":"172.21.15.43:6831/4053438621","state":["exists","up"]},{"osd":10,"uuid":"a7aa7da3-8eab-47fe-a351-298b6ff75f14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6825","nonce":2403210086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6827","nonce":2403210086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6831","nonce":2403210086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6829","nonce":2403210086}]},"public_addr":"172.21.15.96:6825/2403210086","cluster_addr":"172.21.15.96:6827/2403210086","heartbeat_back_addr":"172.21.15.96:6831/2403210086","heartbeat_front_addr":"172.21.15.96:6829/2403210086","state":["exists","up"]},{"osd":11,"uuid":"760b6afc-253d-44dd-88e9-dcc97f67c9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6825","nonce":292202881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6827","nonce":292202881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6831","nonce":292202881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6829","nonce":292202881}]},"public_addr":"172.21.15.151:6825/292202881","cluster_addr":"172.21.15.151:6827/292202881","heartbeat_back_addr":"172.21.15.151:6831/292202881","heartbeat_front_addr":"172.21.15.151:6829/292202881","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:56:34.716991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:56:59.321174+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:57:24.281696+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:57:53.443880+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:58:19.205873+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:58:44.504873+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:13.650731+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:39.137876+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:04.579893+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:34.337312+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:59.434658+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:01:24.930529+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/898247657":"2024-09-20T04:55:07.037117+0000","172.21.15.43:6801/3241799377":"2024-09-20T04:55:07.037117+0000","172.21.15.43:6800/3241799377":"2024-09-20T04:55:07.037117+0000","172.21.15.43:0/3256350596":"2024-09-20T04:55:07.037117+0000","172.21.15.43:6801/587463881":"2024-09-20T04:54:39.548065+0000","172.21.15.43:6800/587463881":"2024-09-20T04:54:39.548065+0000","172.21.15.43:0/4271788315":"2024-09-20T04:55:07.037117+0000","172.21.15.43:0/6132665":"2024-09-20T04:54:39.548065+0000","172.21.15.43:0/632163478":"2024-09-20T04:54:39.548065+0000","172.21.15.43:0/330854185":"2024-09-20T04:54:39.548065+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:03:06.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.700+0000 7f69ff200640 1 -- 172.21.15.43:0/1194300703 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f69dc0a32d0 msgr2=0x7f69dc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:06.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.700+0000 7f69ff200640 1 --2- 172.21.15.43:0/1194300703 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f69dc0a32d0 0x7f69dc0a5790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f69f80ffeb0 tx=0x7f69e000b390 comp rx=0 tx=0).stop 2024-09-19T05:03:06.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.700+0000 7f69ff200640 1 -- 172.21.15.43:0/1194300703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69f8103ff0 msgr2=0x7f69f819a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:06.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.700+0000 7f69ff200640 1 --2- 172.21.15.43:0/1194300703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69f8103ff0 0x7f69f819a5e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f69e8006010 tx=0x7f69e80042f0 comp rx=0 tx=0).stop 2024-09-19T05:03:06.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.700+0000 7f69ff200640 1 -- 172.21.15.43:0/1194300703 shutdown_connections 2024-09-19T05:03:06.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.700+0000 7f69ff200640 1 --2- 172.21.15.43:0/1194300703 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f69dc0a32d0 0x7f69dc0a5790 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:06.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.700+0000 7f69ff200640 1 --2- 172.21.15.43:0/1194300703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f69f8109120 0x7f69f819eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:06.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.700+0000 7f69ff200640 1 --2- 172.21.15.43:0/1194300703 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69f81049a0 0x7f69f819ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:06.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.700+0000 7f69ff200640 1 --2- 172.21.15.43:0/1194300703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69f8103ff0 0x7f69f819a5e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:06.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.700+0000 7f69ff200640 1 -- 172.21.15.43:0/1194300703 >> 172.21.15.43:0/1194300703 conn(0x7f69f80ffa90 msgr2=0x7f69f8100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:06.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.700+0000 7f69ff200640 1 -- 172.21.15.43:0/1194300703 shutdown_connections 2024-09-19T05:03:06.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:06.700+0000 7f69ff200640 1 -- 172.21.15.43:0/1194300703 wait complete. 2024-09-19T05:03:06.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:06 smithi096 ceph-mon[35085]: pgmap v312: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-09-19T05:03:07.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:06 smithi043 ceph-mon[30800]: pgmap v312: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-09-19T05:03:07.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:06 smithi151 ceph-mon[35425]: pgmap v312: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-09-19T05:03:07.445 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-19T05:03:07.446 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-19T05:03:07.446 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-19T05:03:07.446 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-19T05:03:07.447 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-19T05:03:07.447 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-19T05:03:07.448 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-19T05:03:07.448 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-19T05:03:07.449 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-19T05:03:07.449 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-19T05:03:07.450 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-19T05:03:07.450 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-19T05:03:07.709 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:07 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/1194300703' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:03:07.709 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:07 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-09-19T05:03:07.709 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:07 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-09-19T05:03:07.709 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:07 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.11", "id": [7, 11]}]: dispatch 2024-09-19T05:03:07.709 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:07 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-09-19T05:03:07.709 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:07 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1]}]: dispatch 2024-09-19T05:03:07.709 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:07 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-09-19T05:03:07.709 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:07 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-09-19T05:03:07.710 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:07 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-09-19T05:03:07.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:07 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/1194300703' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:03:07.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:07 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-09-19T05:03:07.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:07 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-09-19T05:03:07.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:07 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.11", "id": [7, 11]}]: dispatch 2024-09-19T05:03:07.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:07 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-09-19T05:03:07.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:07 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1]}]: dispatch 2024-09-19T05:03:07.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:07 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-09-19T05:03:07.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:07 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-09-19T05:03:07.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:07 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-09-19T05:03:07.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:07 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-09-19T05:03:08.033 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:07 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-09-19T05:03:08.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:07 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/1194300703' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:03:08.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:07 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-09-19T05:03:08.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:07 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-09-19T05:03:08.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:07 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.11", "id": [7, 11]}]: dispatch 2024-09-19T05:03:08.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:07 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-09-19T05:03:08.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:07 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1]}]: dispatch 2024-09-19T05:03:08.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:07 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-09-19T05:03:08.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:07 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-09-19T05:03:08.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:07 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-09-19T05:03:08.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:07 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-09-19T05:03:08.402 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:08.417 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:08.483 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:08.544 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:08.882 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:08 smithi043 ceph-mon[30800]: pgmap v313: 97 pgs: 97 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-09-19T05:03:08.882 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:08 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-09-19T05:03:08.882 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:08 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-09-19T05:03:08.882 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:08 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.11", "id": [7, 11]}]': finished 2024-09-19T05:03:08.882 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:08 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-09-19T05:03:08.882 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:08 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1]}]': finished 2024-09-19T05:03:08.882 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:08 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-09-19T05:03:08.882 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:08 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-09-19T05:03:08.883 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:08 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-09-19T05:03:08.883 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:08 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-19T05:03:08.883 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:08 smithi043 ceph-mon[30800]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T05:03:08.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:08 smithi096 ceph-mon[35085]: pgmap v313: 97 pgs: 97 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-09-19T05:03:08.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:08 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-09-19T05:03:08.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:08 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-09-19T05:03:08.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:08 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.11", "id": [7, 11]}]': finished 2024-09-19T05:03:08.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:08 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-09-19T05:03:08.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:08 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1]}]': finished 2024-09-19T05:03:08.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:08 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-09-19T05:03:08.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:08 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-09-19T05:03:08.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:08 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-09-19T05:03:08.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:08 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-19T05:03:08.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:08 smithi096 ceph-mon[35085]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T05:03:09.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:08 smithi151 ceph-mon[35425]: pgmap v313: 97 pgs: 97 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-09-19T05:03:09.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:08 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-09-19T05:03:09.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:08 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-09-19T05:03:09.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:08 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.11", "id": [7, 11]}]': finished 2024-09-19T05:03:09.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:08 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-09-19T05:03:09.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:08 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1]}]': finished 2024-09-19T05:03:09.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:08 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-09-19T05:03:09.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:08 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-09-19T05:03:09.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:08 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-09-19T05:03:09.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:08 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-19T05:03:09.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:08 smithi151 ceph-mon[35425]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T05:03:09.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.329+0000 7fcf4b400640 1 -- 172.21.15.43:0/2430776084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf441049c0 msgr2=0x7fcf44108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:09.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.329+0000 7fcf4b400640 1 --2- 172.21.15.43:0/2430776084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf441049c0 0x7fcf44108a10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcf30009900 tx=0x7fcf3002a3f0 comp rx=0 tx=0).stop 2024-09-19T05:03:09.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.329+0000 7fcf4b400640 1 -- 172.21.15.43:0/2430776084 shutdown_connections 2024-09-19T05:03:09.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.329+0000 7fcf4b400640 1 --2- 172.21.15.43:0/2430776084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf44109140 0x7fcf44110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:09.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.329+0000 7fcf4b400640 1 --2- 172.21.15.43:0/2430776084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf441049c0 0x7fcf44108a10 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:09.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.329+0000 7fcf4b400640 1 --2- 172.21.15.43:0/2430776084 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf44104010 0x7fcf441043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:09.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.329+0000 7fcf4b400640 1 -- 172.21.15.43:0/2430776084 >> 172.21.15.43:0/2430776084 conn(0x7fcf440ffa50 msgr2=0x7fcf44101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:09.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.330+0000 7fcf4b400640 1 -- 172.21.15.43:0/2430776084 shutdown_connections 2024-09-19T05:03:09.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.330+0000 7fcf4b400640 1 -- 172.21.15.43:0/2430776084 wait complete. 2024-09-19T05:03:09.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.330+0000 7fcf4b400640 1 Processor -- start 2024-09-19T05:03:09.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.331+0000 7fcf4b400640 1 -- start start 2024-09-19T05:03:09.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.331+0000 7fcf4b400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf44104010 0x7fcf4419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:09.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.331+0000 7fcf4b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf441049c0 0x7fcf4419abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:09.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.331+0000 7fcf4b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf44109140 0x7fcf4419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:09.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.331+0000 7fcf4b400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcf441141b0 con 0x7fcf44109140 2024-09-19T05:03:09.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.331+0000 7fcf4b400640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcf44114030 con 0x7fcf44104010 2024-09-19T05:03:09.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.331+0000 7fcf4b400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcf44114330 con 0x7fcf441049c0 2024-09-19T05:03:09.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.331+0000 7fcf48a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf44104010 0x7fcf4419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:09.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.331+0000 7fcf48a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf44104010 0x7fcf4419a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:50312/0 (socket says 172.21.15.43:50312) 2024-09-19T05:03:09.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.331+0000 7fcf48a00640 1 -- 172.21.15.43:0/2252157234 learned_addr learned my addr 172.21.15.43:0/2252157234 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:09.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.331+0000 7fcf43e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf441049c0 0x7fcf4419abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:09.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.331+0000 7fcf49400640 1 --2- 172.21.15.43:0/2252157234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf44109140 0x7fcf4419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:09.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.332+0000 7fcf48a00640 1 -- 172.21.15.43:0/2252157234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf441049c0 msgr2=0x7fcf4419abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:09.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.332+0000 7fcf48a00640 1 --2- 172.21.15.43:0/2252157234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf441049c0 0x7fcf4419abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:09.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.332+0000 7fcf48a00640 1 -- 172.21.15.43:0/2252157234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf44109140 msgr2=0x7fcf4419ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:09.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.332+0000 7fcf48a00640 1 --2- 172.21.15.43:0/2252157234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf44109140 0x7fcf4419ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:09.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.332+0000 7fcf48a00640 1 -- 172.21.15.43:0/2252157234 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf4419f630 con 0x7fcf44104010 2024-09-19T05:03:09.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.332+0000 7fcf48a00640 1 --2- 172.21.15.43:0/2252157234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf44104010 0x7fcf4419a680 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcf3800e9f0 tx=0x7fcf3800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:09.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.332+0000 7fcf41600640 1 -- 172.21.15.43:0/2252157234 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3800ccc0 con 0x7fcf44104010 2024-09-19T05:03:09.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.333+0000 7fcf4b400640 1 -- 172.21.15.43:0/2252157234 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf4419f920 con 0x7fcf44104010 2024-09-19T05:03:09.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.333+0000 7fcf41600640 1 -- 172.21.15.43:0/2252157234 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf3800ce60 con 0x7fcf44104010 2024-09-19T05:03:09.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.333+0000 7fcf41600640 1 -- 172.21.15.43:0/2252157234 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf38010690 con 0x7fcf44104010 2024-09-19T05:03:09.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.333+0000 7fcf4b400640 1 -- 172.21.15.43:0/2252157234 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf441a7200 con 0x7fcf44104010 2024-09-19T05:03:09.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.333+0000 7fcf4b400640 1 -- 172.21.15.43:0/2252157234 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcf44062940 con 0x7fcf44104010 2024-09-19T05:03:09.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.336+0000 7fcf41600640 1 -- 172.21.15.43:0/2252157234 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fcf380046c0 con 0x7fcf44104010 2024-09-19T05:03:09.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.337+0000 7fcf41600640 1 --2- 172.21.15.43:0/2252157234 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fcf2c0a32d0 0x7fcf2c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:09.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.337+0000 7fcf41600640 1 -- 172.21.15.43:0/2252157234 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fcf380d0330 con 0x7fcf44104010 2024-09-19T05:03:09.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.337+0000 7fcf43e00640 1 --2- 172.21.15.43:0/2252157234 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fcf2c0a32d0 0x7fcf2c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:09.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.337+0000 7fcf41600640 1 --2- 172.21.15.43:0/2252157234 >> [v2:172.21.15.43:6810/1734673762,v1:172.21.15.43:6811/1734673762] conn(0x7fcf2c0acf00 0x7fcf2c0af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:09.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.338+0000 7fcf41600640 1 -- 172.21.15.43:0/2252157234 --> [v2:172.21.15.43:6810/1734673762,v1:172.21.15.43:6811/1734673762] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcf2c0afa70 con 0x7fcf2c0acf00 2024-09-19T05:03:09.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.338+0000 7fcf41600640 1 -- 172.21.15.43:0/2252157234 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fcf380d06e0 con 0x7fcf44104010 2024-09-19T05:03:09.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.338+0000 7fcf49400640 1 --2- 172.21.15.43:0/2252157234 >> [v2:172.21.15.43:6810/1734673762,v1:172.21.15.43:6811/1734673762] conn(0x7fcf2c0acf00 0x7fcf2c0af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:09.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.338+0000 7fcf43e00640 1 --2- 172.21.15.43:0/2252157234 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fcf2c0a32d0 0x7fcf2c0a5790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcf30000c00 tx=0x7fcf30005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:09.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.338+0000 7fcf49400640 1 --2- 172.21.15.43:0/2252157234 >> [v2:172.21.15.43:6810/1734673762,v1:172.21.15.43:6811/1734673762] conn(0x7fcf2c0acf00 0x7fcf2c0af3a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:09.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.339+0000 7fcf41600640 1 -- 172.21.15.43:0/2252157234 <== osd.3 v2:172.21.15.43:6810/1734673762 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcf2c0afa70 con 0x7fcf2c0acf00 2024-09-19T05:03:09.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.354+0000 7fcf4b400640 1 -- 172.21.15.43:0/2252157234 --> [v2:172.21.15.43:6810/1734673762,v1:172.21.15.43:6811/1734673762] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcf44062dc0 con 0x7fcf2c0acf00 2024-09-19T05:03:09.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.354+0000 7fcf41600640 1 -- 172.21.15.43:0/2252157234 <== osd.3 v2:172.21.15.43:6810/1734673762 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fcf44062dc0 con 0x7fcf2c0acf00 2024-09-19T05:03:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.354+0000 7fcf4b400640 1 -- 172.21.15.43:0/2252157234 >> [v2:172.21.15.43:6810/1734673762,v1:172.21.15.43:6811/1734673762] conn(0x7fcf2c0acf00 msgr2=0x7fcf2c0af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.354+0000 7fcf4b400640 1 --2- 172.21.15.43:0/2252157234 >> [v2:172.21.15.43:6810/1734673762,v1:172.21.15.43:6811/1734673762] conn(0x7fcf2c0acf00 0x7fcf2c0af3a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.354+0000 7fcf4b400640 1 -- 172.21.15.43:0/2252157234 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fcf2c0a32d0 msgr2=0x7fcf2c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.355+0000 7fcf4b400640 1 --2- 172.21.15.43:0/2252157234 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fcf2c0a32d0 0x7fcf2c0a5790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcf30000c00 tx=0x7fcf30005b50 comp rx=0 tx=0).stop 2024-09-19T05:03:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.355+0000 7fcf4b400640 1 -- 172.21.15.43:0/2252157234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf44104010 msgr2=0x7fcf4419a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.355+0000 7fcf4b400640 1 --2- 172.21.15.43:0/2252157234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf44104010 0x7fcf4419a680 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcf3800e9f0 tx=0x7fcf3800edf0 comp rx=0 tx=0).stop 2024-09-19T05:03:09.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.355+0000 7fcf49400640 1 -- 172.21.15.43:0/2252157234 reap_dead start 2024-09-19T05:03:09.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.355+0000 7fcf4b400640 1 -- 172.21.15.43:0/2252157234 shutdown_connections 2024-09-19T05:03:09.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.355+0000 7fcf4b400640 1 -- 172.21.15.43:0/2252157234 >> 172.21.15.43:0/2252157234 conn(0x7fcf440ffa50 msgr2=0x7fcf44100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:09.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.355+0000 7fcf4b400640 1 -- 172.21.15.43:0/2252157234 shutdown_connections 2024-09-19T05:03:09.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.355+0000 7fcf4b400640 1 -- 172.21.15.43:0/2252157234 wait complete. 2024-09-19T05:03:09.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.564+0000 7f40d8000640 1 -- 172.21.15.43:0/2239757286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40d0109140 msgr2=0x7f40d0110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:09.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.564+0000 7f40d8000640 1 --2- 172.21.15.43:0/2239757286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40d0109140 0x7f40d0110cc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f40c8009ca0 tx=0x7f40c802a8c0 comp rx=0 tx=0).stop 2024-09-19T05:03:09.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.564+0000 7f40d8000640 1 -- 172.21.15.43:0/2239757286 shutdown_connections 2024-09-19T05:03:09.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.564+0000 7f40d8000640 1 --2- 172.21.15.43:0/2239757286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40d0109140 0x7f40d0110cc0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:09.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.564+0000 7f40d8000640 1 --2- 172.21.15.43:0/2239757286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40d01049c0 0x7f40d0108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:09.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.564+0000 7f40d8000640 1 --2- 172.21.15.43:0/2239757286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40d0104010 0x7f40d01043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:09.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.564+0000 7f40d8000640 1 -- 172.21.15.43:0/2239757286 >> 172.21.15.43:0/2239757286 conn(0x7f40d00ffa50 msgr2=0x7f40d0101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:09.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.564+0000 7f40d8000640 1 -- 172.21.15.43:0/2239757286 shutdown_connections 2024-09-19T05:03:09.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.564+0000 7f40d8000640 1 -- 172.21.15.43:0/2239757286 wait complete. 2024-09-19T05:03:09.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.565+0000 7f40d8000640 1 Processor -- start 2024-09-19T05:03:09.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.565+0000 7f40d8000640 1 -- start start 2024-09-19T05:03:09.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.566+0000 7f40d8000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40d0104010 0x7f40d019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:09.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.566+0000 7f40d8000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40d01049c0 0x7f40d019ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:09.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.566+0000 7f40d8000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40d0109140 0x7f40d019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:09.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.566+0000 7f40d8000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f40d01141c0 con 0x7f40d0109140 2024-09-19T05:03:09.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.566+0000 7f40d8000640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f40d0114040 con 0x7f40d01049c0 2024-09-19T05:03:09.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.566+0000 7f40d8000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f40d0114340 con 0x7f40d0104010 2024-09-19T05:03:09.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.566+0000 7f40d4c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40d01049c0 0x7f40d019ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:09.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.566+0000 7f40d4c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40d01049c0 0x7f40d019ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:50326/0 (socket says 172.21.15.43:50326) 2024-09-19T05:03:09.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.566+0000 7f40d6000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40d0109140 0x7f40d019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:09.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.566+0000 7f40d4c00640 1 -- 172.21.15.43:0/987600287 learned_addr learned my addr 172.21.15.43:0/987600287 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:09.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.566+0000 7f40d5600640 1 --2- 172.21.15.43:0/987600287 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40d0104010 0x7f40d019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:09.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.566+0000 7f40d4c00640 1 -- 172.21.15.43:0/987600287 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40d0104010 msgr2=0x7f40d019a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:09.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.566+0000 7f40d4c00640 1 --2- 172.21.15.43:0/987600287 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40d0104010 0x7f40d019a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:09.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.566+0000 7f40d4c00640 1 -- 172.21.15.43:0/987600287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40d0109140 msgr2=0x7f40d019ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:09.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.566+0000 7f40d4c00640 1 --2- 172.21.15.43:0/987600287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40d0109140 0x7f40d019ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:09.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.566+0000 7f40d4c00640 1 -- 172.21.15.43:0/987600287 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40d019f670 con 0x7f40d01049c0 2024-09-19T05:03:09.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.567+0000 7f40d4c00640 1 --2- 172.21.15.43:0/987600287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40d01049c0 0x7f40d019ac00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f40bc00e9f0 tx=0x7f40bc00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:09.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.567+0000 7f40ce000640 1 -- 172.21.15.43:0/987600287 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40bc00cd80 con 0x7f40d01049c0 2024-09-19T05:03:09.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.567+0000 7f40ce000640 1 -- 172.21.15.43:0/987600287 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40bc005500 con 0x7f40d01049c0 2024-09-19T05:03:09.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.567+0000 7f40ce000640 1 -- 172.21.15.43:0/987600287 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40bc010690 con 0x7f40d01049c0 2024-09-19T05:03:09.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.567+0000 7f40d8000640 1 -- 172.21.15.43:0/987600287 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40d019f960 con 0x7f40d01049c0 2024-09-19T05:03:09.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.567+0000 7f40d8000640 1 -- 172.21.15.43:0/987600287 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40d01a7240 con 0x7f40d01049c0 2024-09-19T05:03:09.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.568+0000 7f40d8000640 1 -- 172.21.15.43:0/987600287 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f40d0062900 con 0x7f40d01049c0 2024-09-19T05:03:09.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.570+0000 7f40ce000640 1 -- 172.21.15.43:0/987600287 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f40bc0046c0 con 0x7f40d01049c0 2024-09-19T05:03:09.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.572+0000 7f40ce000640 1 --2- 172.21.15.43:0/987600287 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f40b80a3350 0x7f40b80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:09.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.572+0000 7f40ce000640 1 -- 172.21.15.43:0/987600287 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f40bc0d0230 con 0x7f40d01049c0 2024-09-19T05:03:09.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.572+0000 7f40d5600640 1 --2- 172.21.15.43:0/987600287 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f40b80a3350 0x7f40b80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:09.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.572+0000 7f40ce000640 1 --2- 172.21.15.43:0/987600287 >> [v2:172.21.15.151:6808/2878451924,v1:172.21.15.151:6809/2878451924] conn(0x7f40b80acf80 0x7f40b80af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:09.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.572+0000 7f40ce000640 1 -- 172.21.15.43:0/987600287 --> [v2:172.21.15.151:6808/2878451924,v1:172.21.15.151:6809/2878451924] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f40bc01aec0 con 0x7f40b80acf80 2024-09-19T05:03:09.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.572+0000 7f40ce000640 1 -- 172.21.15.43:0/987600287 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f40bc0d05e0 con 0x7f40d01049c0 2024-09-19T05:03:09.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.572+0000 7f40d5600640 1 --2- 172.21.15.43:0/987600287 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f40b80a3350 0x7f40b80a5810 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f40c4005d00 tx=0x7f40c400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:09.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.573+0000 7f40d6000640 1 --2- 172.21.15.43:0/987600287 >> [v2:172.21.15.151:6808/2878451924,v1:172.21.15.151:6809/2878451924] conn(0x7f40b80acf80 0x7f40b80af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:09.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.573+0000 7f40d6000640 1 --2- 172.21.15.43:0/987600287 >> [v2:172.21.15.151:6808/2878451924,v1:172.21.15.151:6809/2878451924] conn(0x7f40b80acf80 0x7f40b80af420 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:09.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.574+0000 7f40ce000640 1 -- 172.21.15.43:0/987600287 <== osd.5 v2:172.21.15.151:6808/2878451924 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f40bc01aec0 con 0x7f40b80acf80 2024-09-19T05:03:09.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.591+0000 7f40d8000640 1 -- 172.21.15.43:0/987600287 --> [v2:172.21.15.151:6808/2878451924,v1:172.21.15.151:6809/2878451924] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f40d0062d80 con 0x7f40b80acf80 2024-09-19T05:03:09.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.591+0000 7f40ce000640 1 -- 172.21.15.43:0/987600287 <== osd.5 v2:172.21.15.151:6808/2878451924 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f40d0062d80 con 0x7f40b80acf80 2024-09-19T05:03:09.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.592+0000 7f40d8000640 1 -- 172.21.15.43:0/987600287 >> [v2:172.21.15.151:6808/2878451924,v1:172.21.15.151:6809/2878451924] conn(0x7f40b80acf80 msgr2=0x7f40b80af420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:09.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.592+0000 7f40d8000640 1 --2- 172.21.15.43:0/987600287 >> [v2:172.21.15.151:6808/2878451924,v1:172.21.15.151:6809/2878451924] conn(0x7f40b80acf80 0x7f40b80af420 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:09.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.592+0000 7f40d8000640 1 -- 172.21.15.43:0/987600287 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f40b80a3350 msgr2=0x7f40b80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:09.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.592+0000 7f40d8000640 1 --2- 172.21.15.43:0/987600287 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f40b80a3350 0x7f40b80a5810 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f40c4005d00 tx=0x7f40c400b410 comp rx=0 tx=0).stop 2024-09-19T05:03:09.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.592+0000 7f40d8000640 1 -- 172.21.15.43:0/987600287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40d01049c0 msgr2=0x7f40d019ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:09.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.592+0000 7f40d8000640 1 --2- 172.21.15.43:0/987600287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40d01049c0 0x7f40d019ac00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f40bc00e9f0 tx=0x7f40bc00edf0 comp rx=0 tx=0).stop 2024-09-19T05:03:09.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.592+0000 7f40d6000640 1 -- 172.21.15.43:0/987600287 reap_dead start 2024-09-19T05:03:09.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.592+0000 7f40d8000640 1 -- 172.21.15.43:0/987600287 shutdown_connections 2024-09-19T05:03:09.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.592+0000 7f40d8000640 1 -- 172.21.15.43:0/987600287 >> 172.21.15.43:0/987600287 conn(0x7f40d00ffa50 msgr2=0x7f40d0100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:09.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.592+0000 7f40d8000640 1 -- 172.21.15.43:0/987600287 shutdown_connections 2024-09-19T05:03:09.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:09.592+0000 7f40d8000640 1 -- 172.21.15.43:0/987600287 wait complete. 2024-09-19T05:03:09.909 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:09 smithi043 ceph-mon[30800]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T05:03:09.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:09 smithi096 ceph-mon[35085]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T05:03:10.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:09 smithi151 ceph-mon[35425]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T05:03:10.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.467+0000 7f5756a00640 1 -- 172.21.15.43:0/1377473830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57501079c0 msgr2=0x7f5750107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:10.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.467+0000 7f5756a00640 1 --2- 172.21.15.43:0/1377473830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57501079c0 0x7f5750107da0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5740009cd0 tx=0x7f574002a8f0 comp rx=0 tx=0).stop 2024-09-19T05:03:10.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.467+0000 7f5756a00640 1 -- 172.21.15.43:0/1377473830 shutdown_connections 2024-09-19T05:03:10.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.467+0000 7f5756a00640 1 --2- 172.21.15.43:0/1377473830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5750102360 0x7f575010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:10.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.467+0000 7f5756a00640 1 --2- 172.21.15.43:0/1377473830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57501019c0 0x7f5750101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:10.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.467+0000 7f5756a00640 1 --2- 172.21.15.43:0/1377473830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57501079c0 0x7f5750107da0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:10.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.467+0000 7f5756a00640 1 -- 172.21.15.43:0/1377473830 >> 172.21.15.43:0/1377473830 conn(0x7f57500fd740 msgr2=0x7f57500ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:10.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.467+0000 7f5756a00640 1 -- 172.21.15.43:0/1377473830 shutdown_connections 2024-09-19T05:03:10.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.467+0000 7f5756a00640 1 -- 172.21.15.43:0/1377473830 wait complete. 2024-09-19T05:03:10.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.468+0000 7f5756a00640 1 Processor -- start 2024-09-19T05:03:10.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.468+0000 7f5756a00640 1 -- start start 2024-09-19T05:03:10.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.468+0000 7f5756a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57501019c0 0x7f57500744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:10.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.468+0000 7f5756a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5750102360 0x7f5750074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:10.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.468+0000 7f5756a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57501079c0 0x7f5750074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:10.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.468+0000 7f5756a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5750111e10 con 0x7f5750102360 2024-09-19T05:03:10.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.468+0000 7f5756a00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5750111c90 con 0x7f57501019c0 2024-09-19T05:03:10.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.468+0000 7f5756a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5750111f90 con 0x7f57501079c0 2024-09-19T05:03:10.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.469+0000 7f5755600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57501019c0 0x7f57500744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:10.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.469+0000 7f5755600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57501019c0 0x7f57500744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:50332/0 (socket says 172.21.15.43:50332) 2024-09-19T05:03:10.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.469+0000 7f5754c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5750102360 0x7f5750074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:10.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.469+0000 7f5755600640 1 -- 172.21.15.43:0/1285794010 learned_addr learned my addr 172.21.15.43:0/1285794010 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:10.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.469+0000 7f5756000640 1 --2- 172.21.15.43:0/1285794010 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57501079c0 0x7f5750074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:10.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.469+0000 7f5755600640 1 -- 172.21.15.43:0/1285794010 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57501079c0 msgr2=0x7f5750074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:10.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.469+0000 7f5755600640 1 --2- 172.21.15.43:0/1285794010 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57501079c0 0x7f5750074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:10.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.469+0000 7f5755600640 1 -- 172.21.15.43:0/1285794010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5750102360 msgr2=0x7f5750074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:10.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.469+0000 7f5755600640 1 --2- 172.21.15.43:0/1285794010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5750102360 0x7f5750074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:10.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.469+0000 7f5755600640 1 -- 172.21.15.43:0/1285794010 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57501a67e0 con 0x7f57501019c0 2024-09-19T05:03:10.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.469+0000 7f5756000640 1 --2- 172.21.15.43:0/1285794010 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57501079c0 0x7f5750074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:03:10.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.469+0000 7f5755600640 1 --2- 172.21.15.43:0/1285794010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57501019c0 0x7f57500744f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5740000f80 tx=0x7f5740030b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:10.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.470+0000 7f574e000640 1 -- 172.21.15.43:0/1285794010 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5740004240 con 0x7f57501019c0 2024-09-19T05:03:10.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.470+0000 7f574e000640 1 -- 172.21.15.43:0/1285794010 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5740040070 con 0x7f57501019c0 2024-09-19T05:03:10.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.470+0000 7f574e000640 1 -- 172.21.15.43:0/1285794010 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5740045ce0 con 0x7f57501019c0 2024-09-19T05:03:10.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.470+0000 7f5756a00640 1 -- 172.21.15.43:0/1285794010 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57501a6a70 con 0x7f57501019c0 2024-09-19T05:03:10.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.470+0000 7f5756a00640 1 -- 172.21.15.43:0/1285794010 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57501a7000 con 0x7f57501019c0 2024-09-19T05:03:10.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.471+0000 7f5756a00640 1 -- 172.21.15.43:0/1285794010 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f57500628c0 con 0x7f57501019c0 2024-09-19T05:03:10.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.473+0000 7f574e000640 1 -- 172.21.15.43:0/1285794010 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f5740045470 con 0x7f57501019c0 2024-09-19T05:03:10.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.474+0000 7f574e000640 1 --2- 172.21.15.43:0/1285794010 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f57340a3350 0x7f57340a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:10.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.474+0000 7f574e000640 1 -- 172.21.15.43:0/1285794010 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f57400f0ea0 con 0x7f57501019c0 2024-09-19T05:03:10.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.475+0000 7f5754c00640 1 --2- 172.21.15.43:0/1285794010 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f57340a3350 0x7f57340a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:10.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.475+0000 7f574e000640 1 --2- 172.21.15.43:0/1285794010 >> [v2:172.21.15.96:6824/2403210086,v1:172.21.15.96:6825/2403210086] conn(0x7f57340acf80 0x7f57340af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:10.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.475+0000 7f574e000640 1 -- 172.21.15.43:0/1285794010 --> [v2:172.21.15.96:6824/2403210086,v1:172.21.15.96:6825/2403210086] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f574003ce50 con 0x7f57340acf80 2024-09-19T05:03:10.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.475+0000 7f574e000640 1 -- 172.21.15.43:0/1285794010 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f57400f1250 con 0x7f57501019c0 2024-09-19T05:03:10.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.475+0000 7f5756000640 1 --2- 172.21.15.43:0/1285794010 >> [v2:172.21.15.96:6824/2403210086,v1:172.21.15.96:6825/2403210086] conn(0x7f57340acf80 0x7f57340af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:10.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.475+0000 7f5754c00640 1 --2- 172.21.15.43:0/1285794010 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f57340a3350 0x7f57340a5810 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5750106f40 tx=0x7f573c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:10.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.476+0000 7f5756000640 1 --2- 172.21.15.43:0/1285794010 >> [v2:172.21.15.96:6824/2403210086,v1:172.21.15.96:6825/2403210086] conn(0x7f57340acf80 0x7f57340af420 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:10.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.477+0000 7f574e000640 1 -- 172.21.15.43:0/1285794010 <== osd.10 v2:172.21.15.96:6824/2403210086 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f574003ce50 con 0x7f57340acf80 2024-09-19T05:03:10.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.491+0000 7f5756a00640 1 -- 172.21.15.43:0/1285794010 --> [v2:172.21.15.96:6824/2403210086,v1:172.21.15.96:6825/2403210086] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5750062cc0 con 0x7f57340acf80 2024-09-19T05:03:10.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.492+0000 7f574e000640 1 -- 172.21.15.43:0/1285794010 <== osd.10 v2:172.21.15.96:6824/2403210086 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5750062cc0 con 0x7f57340acf80 2024-09-19T05:03:10.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.492+0000 7f5756a00640 1 -- 172.21.15.43:0/1285794010 >> [v2:172.21.15.96:6824/2403210086,v1:172.21.15.96:6825/2403210086] conn(0x7f57340acf80 msgr2=0x7f57340af420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:10.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.492+0000 7f5756a00640 1 --2- 172.21.15.43:0/1285794010 >> [v2:172.21.15.96:6824/2403210086,v1:172.21.15.96:6825/2403210086] conn(0x7f57340acf80 0x7f57340af420 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:10.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.492+0000 7f5756a00640 1 -- 172.21.15.43:0/1285794010 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f57340a3350 msgr2=0x7f57340a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:10.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.492+0000 7f5756a00640 1 --2- 172.21.15.43:0/1285794010 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f57340a3350 0x7f57340a5810 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5750106f40 tx=0x7f573c00b390 comp rx=0 tx=0).stop 2024-09-19T05:03:10.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.492+0000 7f5756a00640 1 -- 172.21.15.43:0/1285794010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57501019c0 msgr2=0x7f57500744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:10.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.492+0000 7f5756a00640 1 --2- 172.21.15.43:0/1285794010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57501019c0 0x7f57500744f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5740000f80 tx=0x7f5740030b60 comp rx=0 tx=0).stop 2024-09-19T05:03:10.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.492+0000 7f5756000640 1 -- 172.21.15.43:0/1285794010 reap_dead start 2024-09-19T05:03:10.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.492+0000 7f5756a00640 1 -- 172.21.15.43:0/1285794010 shutdown_connections 2024-09-19T05:03:10.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.492+0000 7f5756a00640 1 -- 172.21.15.43:0/1285794010 >> 172.21.15.43:0/1285794010 conn(0x7f57500fd740 msgr2=0x7f57500fefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:10.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.493+0000 7f5756a00640 1 -- 172.21.15.43:0/1285794010 shutdown_connections 2024-09-19T05:03:10.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:10.493+0000 7f5756a00640 1 -- 172.21.15.43:0/1285794010 wait complete. 2024-09-19T05:03:10.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:10 smithi043 ceph-mon[30800]: pgmap v316: 97 pgs: 1 peering, 96 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T05:03:10.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:10 smithi096 ceph-mon[35085]: pgmap v316: 97 pgs: 1 peering, 96 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T05:03:11.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:10 smithi151 ceph-mon[35425]: pgmap v316: 97 pgs: 1 peering, 96 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T05:03:11.722 INFO:teuthology.orchestra.run.smithi043.stdout:103079215169 2024-09-19T05:03:11.722 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-19T05:03:11.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.462+0000 7f5027400640 1 -- 172.21.15.43:0/526942065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5020112a50 msgr2=0x7f5020114e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:11.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.462+0000 7f5027400640 1 --2- 172.21.15.43:0/526942065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5020112a50 0x7f5020114e40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5018009ca0 tx=0x7f501802a8c0 comp rx=0 tx=0).stop 2024-09-19T05:03:11.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.462+0000 7f5027400640 1 -- 172.21.15.43:0/526942065 shutdown_connections 2024-09-19T05:03:11.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.462+0000 7f5027400640 1 --2- 172.21.15.43:0/526942065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5020112a50 0x7f5020114e40 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:11.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.462+0000 7f5027400640 1 --2- 172.21.15.43:0/526942065 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5020077040 0x7f5020112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:11.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.462+0000 7f5027400640 1 --2- 172.21.15.43:0/526942065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5020076720 0x7f5020076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:11.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.462+0000 7f5027400640 1 -- 172.21.15.43:0/526942065 >> 172.21.15.43:0/526942065 conn(0x7f50200ffa50 msgr2=0x7f5020101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:11.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.462+0000 7f5027400640 1 -- 172.21.15.43:0/526942065 shutdown_connections 2024-09-19T05:03:11.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.463+0000 7f5027400640 1 -- 172.21.15.43:0/526942065 wait complete. 2024-09-19T05:03:11.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.464+0000 7f5027400640 1 Processor -- start 2024-09-19T05:03:11.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.464+0000 7f5027400640 1 -- start start 2024-09-19T05:03:11.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.464+0000 7f5027400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5020076720 0x7f50201a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:11.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.464+0000 7f5027400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5020077040 0x7f50201a4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:11.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.464+0000 7f5027400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5020112a50 0x7f502019e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:11.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.464+0000 7f5027400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5020118600 con 0x7f5020076720 2024-09-19T05:03:11.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.464+0000 7f5027400640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5020118480 con 0x7f5020077040 2024-09-19T05:03:11.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.464+0000 7f5027400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5020118780 con 0x7f5020112a50 2024-09-19T05:03:11.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.465+0000 7f5024a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5020076720 0x7f50201a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:11.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.465+0000 7f5024a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5020076720 0x7f50201a4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45994/0 (socket says 172.21.15.43:45994) 2024-09-19T05:03:11.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.465+0000 7f5024a00640 1 -- 172.21.15.43:0/1102096964 learned_addr learned my addr 172.21.15.43:0/1102096964 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:11.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.465+0000 7f5025400640 1 --2- 172.21.15.43:0/1102096964 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5020112a50 0x7f502019e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:11.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.465+0000 7f501fe00640 1 --2- 172.21.15.43:0/1102096964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5020077040 0x7f50201a4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:11.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.465+0000 7f5024a00640 1 -- 172.21.15.43:0/1102096964 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5020112a50 msgr2=0x7f502019e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:11.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.465+0000 7f5024a00640 1 --2- 172.21.15.43:0/1102096964 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5020112a50 0x7f502019e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:11.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.465+0000 7f5024a00640 1 -- 172.21.15.43:0/1102096964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5020077040 msgr2=0x7f50201a4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:11.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.465+0000 7f5024a00640 1 --2- 172.21.15.43:0/1102096964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5020077040 0x7f50201a4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:11.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.465+0000 7f5024a00640 1 -- 172.21.15.43:0/1102096964 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f502019f080 con 0x7f5020076720 2024-09-19T05:03:11.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.465+0000 7f5024a00640 1 --2- 172.21.15.43:0/1102096964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5020076720 0x7f50201a4790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f501400e9f0 tx=0x7f501400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:11.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.466+0000 7f501d600640 1 -- 172.21.15.43:0/1102096964 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501400cd80 con 0x7f5020076720 2024-09-19T05:03:11.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.466+0000 7f5027400640 1 -- 172.21.15.43:0/1102096964 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f502019f370 con 0x7f5020076720 2024-09-19T05:03:11.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.466+0000 7f501d600640 1 -- 172.21.15.43:0/1102096964 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5014010910 con 0x7f5020076720 2024-09-19T05:03:11.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.466+0000 7f501d600640 1 -- 172.21.15.43:0/1102096964 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50140188f0 con 0x7f5020076720 2024-09-19T05:03:11.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.466+0000 7f5027400640 1 -- 172.21.15.43:0/1102096964 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50201ab660 con 0x7f5020076720 2024-09-19T05:03:11.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.467+0000 7f5027400640 1 -- 172.21.15.43:0/1102096964 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5020062900 con 0x7f5020076720 2024-09-19T05:03:11.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.469+0000 7f501d600640 1 -- 172.21.15.43:0/1102096964 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f5014010ac0 con 0x7f5020076720 2024-09-19T05:03:11.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.470+0000 7f501d600640 1 --2- 172.21.15.43:0/1102096964 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f50080a32d0 0x7f50080a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:11.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.470+0000 7f501d600640 1 -- 172.21.15.43:0/1102096964 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f50140d04f0 con 0x7f5020076720 2024-09-19T05:03:11.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.470+0000 7f501fe00640 1 --2- 172.21.15.43:0/1102096964 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f50080a32d0 0x7f50080a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:11.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.470+0000 7f501d600640 1 --2- 172.21.15.43:0/1102096964 >> [v2:172.21.15.151:6800/2047147996,v1:172.21.15.151:6801/2047147996] conn(0x7f50080acf00 0x7f50080af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:11.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.470+0000 7f501d600640 1 -- 172.21.15.43:0/1102096964 --> [v2:172.21.15.151:6800/2047147996,v1:172.21.15.151:6801/2047147996] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f50080afa70 con 0x7f50080acf00 2024-09-19T05:03:11.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.470+0000 7f501d600640 1 -- 172.21.15.43:0/1102096964 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f50140d08a0 con 0x7f5020076720 2024-09-19T05:03:11.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.471+0000 7f501fe00640 1 --2- 172.21.15.43:0/1102096964 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f50080a32d0 0x7f50080a5790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5020079390 tx=0x7f500c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:11.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.471+0000 7f5025400640 1 --2- 172.21.15.43:0/1102096964 >> [v2:172.21.15.151:6800/2047147996,v1:172.21.15.151:6801/2047147996] conn(0x7f50080acf00 0x7f50080af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:11.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.471+0000 7f5025400640 1 --2- 172.21.15.43:0/1102096964 >> [v2:172.21.15.151:6800/2047147996,v1:172.21.15.151:6801/2047147996] conn(0x7f50080acf00 0x7f50080af3a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:11.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.472+0000 7f501d600640 1 -- 172.21.15.43:0/1102096964 <== osd.2 v2:172.21.15.151:6800/2047147996 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f50080afa70 con 0x7f50080acf00 2024-09-19T05:03:11.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.487+0000 7f5027400640 1 -- 172.21.15.43:0/1102096964 --> [v2:172.21.15.151:6800/2047147996,v1:172.21.15.151:6801/2047147996] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5020062d00 con 0x7f50080acf00 2024-09-19T05:03:11.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.487+0000 7f501d600640 1 -- 172.21.15.43:0/1102096964 <== osd.2 v2:172.21.15.151:6800/2047147996 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5020062d00 con 0x7f50080acf00 2024-09-19T05:03:11.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.487+0000 7f5027400640 1 -- 172.21.15.43:0/1102096964 >> [v2:172.21.15.151:6800/2047147996,v1:172.21.15.151:6801/2047147996] conn(0x7f50080acf00 msgr2=0x7f50080af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:11.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.487+0000 7f5027400640 1 --2- 172.21.15.43:0/1102096964 >> [v2:172.21.15.151:6800/2047147996,v1:172.21.15.151:6801/2047147996] conn(0x7f50080acf00 0x7f50080af3a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:11.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.488+0000 7f5027400640 1 -- 172.21.15.43:0/1102096964 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f50080a32d0 msgr2=0x7f50080a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:11.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.488+0000 7f5027400640 1 --2- 172.21.15.43:0/1102096964 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f50080a32d0 0x7f50080a5790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5020079390 tx=0x7f500c00b390 comp rx=0 tx=0).stop 2024-09-19T05:03:11.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.488+0000 7f5027400640 1 -- 172.21.15.43:0/1102096964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5020076720 msgr2=0x7f50201a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:11.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.488+0000 7f5027400640 1 --2- 172.21.15.43:0/1102096964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5020076720 0x7f50201a4790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f501400e9f0 tx=0x7f501400eeb0 comp rx=0 tx=0).stop 2024-09-19T05:03:11.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.488+0000 7f5025400640 1 -- 172.21.15.43:0/1102096964 reap_dead start 2024-09-19T05:03:11.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.488+0000 7f5027400640 1 -- 172.21.15.43:0/1102096964 shutdown_connections 2024-09-19T05:03:11.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.488+0000 7f5027400640 1 -- 172.21.15.43:0/1102096964 >> 172.21.15.43:0/1102096964 conn(0x7f50200ffa50 msgr2=0x7f5020101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:11.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.488+0000 7f5027400640 1 -- 172.21.15.43:0/1102096964 shutdown_connections 2024-09-19T05:03:11.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:11.488+0000 7f5027400640 1 -- 172.21.15.43:0/1102096964 wait complete. 2024-09-19T05:03:12.044 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:12.044 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:12.044 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:12.047 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:12.053 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:12.053 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:12.396 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:12.396 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:12.435 INFO:teuthology.orchestra.run.smithi043.stdout:158913790007 2024-09-19T05:03:12.436 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-19T05:03:12.935 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:12 smithi043 ceph-mon[30800]: pgmap v317: 97 pgs: 1 peering, 96 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T05:03:12.937 INFO:teuthology.orchestra.run.smithi043.stdout:77309411399 2024-09-19T05:03:12.938 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-19T05:03:12.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:12 smithi096 ceph-mon[35085]: pgmap v317: 97 pgs: 1 peering, 96 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T05:03:13.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:12 smithi151 ceph-mon[35425]: pgmap v317: 97 pgs: 1 peering, 96 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T05:03:14.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.151+0000 7f8c64e00640 1 -- 172.21.15.43:0/1256280349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c60102790 msgr2=0x7f8c601067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:14.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.151+0000 7f8c64e00640 1 --2- 172.21.15.43:0/1256280349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c60102790 0x7f8c601067e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8c48009900 tx=0x7f8c4802a410 comp rx=0 tx=0).stop 2024-09-19T05:03:14.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.151+0000 7f8c64e00640 1 -- 172.21.15.43:0/1256280349 shutdown_connections 2024-09-19T05:03:14.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.151+0000 7f8c64e00640 1 --2- 172.21.15.43:0/1256280349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c60106f10 0x7f8c6010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:14.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.151+0000 7f8c64e00640 1 --2- 172.21.15.43:0/1256280349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c60102790 0x7f8c601067e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:14.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.151+0000 7f8c64e00640 1 --2- 172.21.15.43:0/1256280349 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c60101de0 0x7f8c601021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:14.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.151+0000 7f8c64e00640 1 -- 172.21.15.43:0/1256280349 >> 172.21.15.43:0/1256280349 conn(0x7f8c600fd8c0 msgr2=0x7f8c600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:14.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.151+0000 7f8c64e00640 1 -- 172.21.15.43:0/1256280349 shutdown_connections 2024-09-19T05:03:14.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.151+0000 7f8c64e00640 1 -- 172.21.15.43:0/1256280349 wait complete. 2024-09-19T05:03:14.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.152+0000 7f8c64e00640 1 Processor -- start 2024-09-19T05:03:14.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.152+0000 7f8c64e00640 1 -- start start 2024-09-19T05:03:14.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.152+0000 7f8c64e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c60101de0 0x7f8c600744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:14.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.152+0000 7f8c64e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c60102790 0x7f8c60074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:14.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.153+0000 7f8c64e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c60106f10 0x7f8c60074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:14.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.153+0000 7f8c64e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8c60111fc0 con 0x7f8c60102790 2024-09-19T05:03:14.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.153+0000 7f8c64e00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8c60111e40 con 0x7f8c60101de0 2024-09-19T05:03:14.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.153+0000 7f8c64e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8c60112140 con 0x7f8c60106f10 2024-09-19T05:03:14.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.153+0000 7f8c57e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c60102790 0x7f8c60074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:14.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.153+0000 7f8c57e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c60102790 0x7f8c60074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46020/0 (socket says 172.21.15.43:46020) 2024-09-19T05:03:14.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.153+0000 7f8c57e00640 1 -- 172.21.15.43:0/3305737458 learned_addr learned my addr 172.21.15.43:0/3305737458 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:14.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.153+0000 7f8c5de00640 1 --2- 172.21.15.43:0/3305737458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c60101de0 0x7f8c600744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:14.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.153+0000 7f8c5e800640 1 --2- 172.21.15.43:0/3305737458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c60106f10 0x7f8c60074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:14.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.153+0000 7f8c57e00640 1 -- 172.21.15.43:0/3305737458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c60106f10 msgr2=0x7f8c60074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:14.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.153+0000 7f8c57e00640 1 --2- 172.21.15.43:0/3305737458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c60106f10 0x7f8c60074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:14.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.153+0000 7f8c57e00640 1 -- 172.21.15.43:0/3305737458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c60101de0 msgr2=0x7f8c600744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:14.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.153+0000 7f8c57e00640 1 --2- 172.21.15.43:0/3305737458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c60101de0 0x7f8c600744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:14.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.153+0000 7f8c57e00640 1 -- 172.21.15.43:0/3305737458 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c601a6930 con 0x7f8c60102790 2024-09-19T05:03:14.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.153+0000 7f8c5de00640 1 --2- 172.21.15.43:0/3305737458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c60101de0 0x7f8c600744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:03:14.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.154+0000 7f8c57e00640 1 --2- 172.21.15.43:0/3305737458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c60102790 0x7f8c60074a30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8c48004770 tx=0x7f8c48003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:14.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.154+0000 7f8c56a00640 1 -- 172.21.15.43:0/3305737458 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4803f070 con 0x7f8c60102790 2024-09-19T05:03:14.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.154+0000 7f8c64e00640 1 -- 172.21.15.43:0/3305737458 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c601a6bc0 con 0x7f8c60102790 2024-09-19T05:03:14.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.154+0000 7f8c56a00640 1 -- 172.21.15.43:0/3305737458 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c48031ca0 con 0x7f8c60102790 2024-09-19T05:03:14.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.154+0000 7f8c56a00640 1 -- 172.21.15.43:0/3305737458 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c48035030 con 0x7f8c60102790 2024-09-19T05:03:14.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.154+0000 7f8c64e00640 1 -- 172.21.15.43:0/3305737458 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c601a70f0 con 0x7f8c60102790 2024-09-19T05:03:14.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.155+0000 7f8c64e00640 1 -- 172.21.15.43:0/3305737458 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8c600628c0 con 0x7f8c60102790 2024-09-19T05:03:14.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.157+0000 7f8c56a00640 1 -- 172.21.15.43:0/3305737458 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f8c48042070 con 0x7f8c60102790 2024-09-19T05:03:14.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.158+0000 7f8c56a00640 1 --2- 172.21.15.43:0/3305737458 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f8c440a32d0 0x7f8c440a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:14.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.159+0000 7f8c56a00640 1 -- 172.21.15.43:0/3305737458 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f8c480f0210 con 0x7f8c60102790 2024-09-19T05:03:14.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.159+0000 7f8c5de00640 1 --2- 172.21.15.43:0/3305737458 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f8c440a32d0 0x7f8c440a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:14.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.159+0000 7f8c56a00640 1 --2- 172.21.15.43:0/3305737458 >> [v2:172.21.15.96:6808/3184785362,v1:172.21.15.96:6809/3184785362] conn(0x7f8c440acf00 0x7f8c440af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:14.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.159+0000 7f8c56a00640 1 -- 172.21.15.43:0/3305737458 --> [v2:172.21.15.96:6808/3184785362,v1:172.21.15.96:6809/3184785362] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8c4802fec0 con 0x7f8c440acf00 2024-09-19T05:03:14.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.159+0000 7f8c56a00640 1 -- 172.21.15.43:0/3305737458 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f8c480f7050 con 0x7f8c60102790 2024-09-19T05:03:14.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.159+0000 7f8c5e800640 1 --2- 172.21.15.43:0/3305737458 >> [v2:172.21.15.96:6808/3184785362,v1:172.21.15.96:6809/3184785362] conn(0x7f8c440acf00 0x7f8c440af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:14.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.160+0000 7f8c5de00640 1 --2- 172.21.15.43:0/3305737458 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f8c440a32d0 0x7f8c440a5790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8c50005fd0 tx=0x7f8c5000b5f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:14.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.160+0000 7f8c5e800640 1 --2- 172.21.15.43:0/3305737458 >> [v2:172.21.15.96:6808/3184785362,v1:172.21.15.96:6809/3184785362] conn(0x7f8c440acf00 0x7f8c440af3a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:14.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.161+0000 7f8c56a00640 1 -- 172.21.15.43:0/3305737458 <== osd.4 v2:172.21.15.96:6808/3184785362 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8c4802fec0 con 0x7f8c440acf00 2024-09-19T05:03:14.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.175+0000 7f8c64e00640 1 -- 172.21.15.43:0/3305737458 --> [v2:172.21.15.96:6808/3184785362,v1:172.21.15.96:6809/3184785362] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8c60062cf0 con 0x7f8c440acf00 2024-09-19T05:03:14.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.176+0000 7f8c56a00640 1 -- 172.21.15.43:0/3305737458 <== osd.4 v2:172.21.15.96:6808/3184785362 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8c60062cf0 con 0x7f8c440acf00 2024-09-19T05:03:14.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.177+0000 7f8c64e00640 1 -- 172.21.15.43:0/3305737458 >> [v2:172.21.15.96:6808/3184785362,v1:172.21.15.96:6809/3184785362] conn(0x7f8c440acf00 msgr2=0x7f8c440af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:14.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.177+0000 7f8c64e00640 1 --2- 172.21.15.43:0/3305737458 >> [v2:172.21.15.96:6808/3184785362,v1:172.21.15.96:6809/3184785362] conn(0x7f8c440acf00 0x7f8c440af3a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:14.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.177+0000 7f8c64e00640 1 -- 172.21.15.43:0/3305737458 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f8c440a32d0 msgr2=0x7f8c440a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:14.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.177+0000 7f8c64e00640 1 --2- 172.21.15.43:0/3305737458 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f8c440a32d0 0x7f8c440a5790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8c50005fd0 tx=0x7f8c5000b5f0 comp rx=0 tx=0).stop 2024-09-19T05:03:14.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.177+0000 7f8c64e00640 1 -- 172.21.15.43:0/3305737458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c60102790 msgr2=0x7f8c60074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:14.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.177+0000 7f8c64e00640 1 --2- 172.21.15.43:0/3305737458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c60102790 0x7f8c60074a30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8c48004770 tx=0x7f8c48003c10 comp rx=0 tx=0).stop 2024-09-19T05:03:14.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.177+0000 7f8c5e800640 1 -- 172.21.15.43:0/3305737458 reap_dead start 2024-09-19T05:03:14.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.177+0000 7f8c64e00640 1 -- 172.21.15.43:0/3305737458 shutdown_connections 2024-09-19T05:03:14.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.177+0000 7f8c64e00640 1 -- 172.21.15.43:0/3305737458 >> 172.21.15.43:0/3305737458 conn(0x7f8c600fd8c0 msgr2=0x7f8c600feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:14.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.177+0000 7f8c64e00640 1 -- 172.21.15.43:0/3305737458 shutdown_connections 2024-09-19T05:03:14.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.177+0000 7f8c64e00640 1 -- 172.21.15.43:0/3305737458 wait complete. 2024-09-19T05:03:14.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.237+0000 7f2395e00640 1 -- 172.21.15.43:0/3496498091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23901019c0 msgr2=0x7f2390101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:14.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.237+0000 7f2395e00640 1 --2- 172.21.15.43:0/3496498091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23901019c0 0x7f2390101e20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f237c009900 tx=0x7f237c02a410 comp rx=0 tx=0).stop 2024-09-19T05:03:14.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.237+0000 7f2395e00640 1 -- 172.21.15.43:0/3496498091 shutdown_connections 2024-09-19T05:03:14.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.237+0000 7f2395e00640 1 --2- 172.21.15.43:0/3496498091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2390102360 0x7f239010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:14.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.237+0000 7f2395e00640 1 --2- 172.21.15.43:0/3496498091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23901019c0 0x7f2390101e20 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:14.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.237+0000 7f2395e00640 1 --2- 172.21.15.43:0/3496498091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23901079c0 0x7f2390107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:14.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.237+0000 7f2395e00640 1 -- 172.21.15.43:0/3496498091 >> 172.21.15.43:0/3496498091 conn(0x7f23900fd740 msgr2=0x7f23900ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:14.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.237+0000 7f2395e00640 1 -- 172.21.15.43:0/3496498091 shutdown_connections 2024-09-19T05:03:14.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.237+0000 7f2395e00640 1 -- 172.21.15.43:0/3496498091 wait complete. 2024-09-19T05:03:14.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.238+0000 7f2395e00640 1 Processor -- start 2024-09-19T05:03:14.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.238+0000 7f2395e00640 1 -- start start 2024-09-19T05:03:14.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.239+0000 7f2395e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23901019c0 0x7f239019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:14.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.239+0000 7f2395e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2390102360 0x7f239019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:14.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.239+0000 7f2395e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23901079c0 0x7f239019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:14.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.239+0000 7f2395e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2390111e10 con 0x7f23901079c0 2024-09-19T05:03:14.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.239+0000 7f2395e00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2390111c90 con 0x7f23901019c0 2024-09-19T05:03:14.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.239+0000 7f2395e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2390111f90 con 0x7f2390102360 2024-09-19T05:03:14.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.239+0000 7f2394a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23901019c0 0x7f239019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:14.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.239+0000 7f2395400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23901079c0 0x7f239019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:14.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.239+0000 7f2394a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23901019c0 0x7f239019a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:50380/0 (socket says 172.21.15.43:50380) 2024-09-19T05:03:14.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.239+0000 7f2394a00640 1 -- 172.21.15.43:0/724899521 learned_addr learned my addr 172.21.15.43:0/724899521 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:14.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.239+0000 7f238fe00640 1 --2- 172.21.15.43:0/724899521 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2390102360 0x7f239019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:14.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.239+0000 7f2394a00640 1 -- 172.21.15.43:0/724899521 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2390102360 msgr2=0x7f239019a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:14.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.239+0000 7f2394a00640 1 --2- 172.21.15.43:0/724899521 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2390102360 0x7f239019a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:14.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.239+0000 7f2394a00640 1 -- 172.21.15.43:0/724899521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23901079c0 msgr2=0x7f239019ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:14.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.239+0000 7f2394a00640 1 --2- 172.21.15.43:0/724899521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23901079c0 0x7f239019ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:14.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.239+0000 7f2394a00640 1 -- 172.21.15.43:0/724899521 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f239019f450 con 0x7f23901019c0 2024-09-19T05:03:14.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.240+0000 7f2394a00640 1 --2- 172.21.15.43:0/724899521 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23901019c0 0x7f239019a4a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f238400e9f0 tx=0x7f238400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:14.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.240+0000 7f238d600640 1 -- 172.21.15.43:0/724899521 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238400cd80 con 0x7f23901019c0 2024-09-19T05:03:14.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.240+0000 7f238d600640 1 -- 172.21.15.43:0/724899521 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2384005500 con 0x7f23901019c0 2024-09-19T05:03:14.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.240+0000 7f2395e00640 1 -- 172.21.15.43:0/724899521 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f239019f740 con 0x7f23901019c0 2024-09-19T05:03:14.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.240+0000 7f238d600640 1 -- 172.21.15.43:0/724899521 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2384010690 con 0x7f23901019c0 2024-09-19T05:03:14.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.240+0000 7f2395e00640 1 -- 172.21.15.43:0/724899521 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23901a7020 con 0x7f23901019c0 2024-09-19T05:03:14.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.241+0000 7f2395e00640 1 -- 172.21.15.43:0/724899521 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f23900fefd0 con 0x7f23901019c0 2024-09-19T05:03:14.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.243+0000 7f238d600640 1 -- 172.21.15.43:0/724899521 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f23840046c0 con 0x7f23901019c0 2024-09-19T05:03:14.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.245+0000 7f238d600640 1 --2- 172.21.15.43:0/724899521 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f23780a32d0 0x7f23780a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:14.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.245+0000 7f238d600640 1 -- 172.21.15.43:0/724899521 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f23840d02c0 con 0x7f23901019c0 2024-09-19T05:03:14.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.245+0000 7f238fe00640 1 --2- 172.21.15.43:0/724899521 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f23780a32d0 0x7f23780a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:14.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.245+0000 7f238d600640 1 --2- 172.21.15.43:0/724899521 >> [v2:172.21.15.151:6816/725316393,v1:172.21.15.151:6817/725316393] conn(0x7f23780acf00 0x7f23780af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:14.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.245+0000 7f238d600640 1 -- 172.21.15.43:0/724899521 --> [v2:172.21.15.151:6816/725316393,v1:172.21.15.151:6817/725316393] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2384004470 con 0x7f23780acf00 2024-09-19T05:03:14.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.245+0000 7f238d600640 1 -- 172.21.15.43:0/724899521 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f23840d0670 con 0x7f23901019c0 2024-09-19T05:03:14.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.245+0000 7f2395400640 1 --2- 172.21.15.43:0/724899521 >> [v2:172.21.15.151:6816/725316393,v1:172.21.15.151:6817/725316393] conn(0x7f23780acf00 0x7f23780af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:14.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.246+0000 7f238fe00640 1 --2- 172.21.15.43:0/724899521 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f23780a32d0 0x7f23780a5790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f237c000c00 tx=0x7f237c005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:14.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.246+0000 7f2395400640 1 --2- 172.21.15.43:0/724899521 >> [v2:172.21.15.151:6816/725316393,v1:172.21.15.151:6817/725316393] conn(0x7f23780acf00 0x7f23780af3a0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:14.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.247+0000 7f238d600640 1 -- 172.21.15.43:0/724899521 <== osd.8 v2:172.21.15.151:6816/725316393 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2384004470 con 0x7f23780acf00 2024-09-19T05:03:14.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.261+0000 7f2395e00640 1 -- 172.21.15.43:0/724899521 --> [v2:172.21.15.151:6816/725316393,v1:172.21.15.151:6817/725316393] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2390103100 con 0x7f23780acf00 2024-09-19T05:03:14.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.262+0000 7f238d600640 1 -- 172.21.15.43:0/724899521 <== osd.8 v2:172.21.15.151:6816/725316393 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2390103100 con 0x7f23780acf00 2024-09-19T05:03:14.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.262+0000 7f2395e00640 1 -- 172.21.15.43:0/724899521 >> [v2:172.21.15.151:6816/725316393,v1:172.21.15.151:6817/725316393] conn(0x7f23780acf00 msgr2=0x7f23780af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:14.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.262+0000 7f2395e00640 1 --2- 172.21.15.43:0/724899521 >> [v2:172.21.15.151:6816/725316393,v1:172.21.15.151:6817/725316393] conn(0x7f23780acf00 0x7f23780af3a0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:14.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.262+0000 7f2395e00640 1 -- 172.21.15.43:0/724899521 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f23780a32d0 msgr2=0x7f23780a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:14.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.262+0000 7f2395e00640 1 --2- 172.21.15.43:0/724899521 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f23780a32d0 0x7f23780a5790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f237c000c00 tx=0x7f237c005b50 comp rx=0 tx=0).stop 2024-09-19T05:03:14.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.262+0000 7f2395e00640 1 -- 172.21.15.43:0/724899521 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23901019c0 msgr2=0x7f239019a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:14.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.262+0000 7f2395e00640 1 --2- 172.21.15.43:0/724899521 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23901019c0 0x7f239019a4a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f238400e9f0 tx=0x7f238400edf0 comp rx=0 tx=0).stop 2024-09-19T05:03:14.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.262+0000 7f2395400640 1 -- 172.21.15.43:0/724899521 reap_dead start 2024-09-19T05:03:14.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.262+0000 7f2395e00640 1 -- 172.21.15.43:0/724899521 shutdown_connections 2024-09-19T05:03:14.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.262+0000 7f2395e00640 1 -- 172.21.15.43:0/724899521 >> 172.21.15.43:0/724899521 conn(0x7f23900fd740 msgr2=0x7f239010d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:14.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.263+0000 7f2395e00640 1 -- 172.21.15.43:0/724899521 shutdown_connections 2024-09-19T05:03:14.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:14.263+0000 7f2395e00640 1 -- 172.21.15.43:0/724899521 wait complete. 2024-09-19T05:03:14.908 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:14 smithi043 ceph-mon[30800]: pgmap v318: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 11 op/s; 122 B/s, 2 keys/s, 2 objects/s recovering 2024-09-19T05:03:14.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:14 smithi096 ceph-mon[35085]: pgmap v318: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 11 op/s; 122 B/s, 2 keys/s, 2 objects/s recovering 2024-09-19T05:03:15.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.144+0000 7fb936200640 1 -- 172.21.15.43:0/2369435156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9301019c0 msgr2=0x7fb930101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:15.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.144+0000 7fb936200640 1 --2- 172.21.15.43:0/2369435156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9301019c0 0x7fb930101e20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb918009890 tx=0x7fb91802a360 comp rx=0 tx=0).stop 2024-09-19T05:03:15.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.144+0000 7fb936200640 1 -- 172.21.15.43:0/2369435156 shutdown_connections 2024-09-19T05:03:15.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.144+0000 7fb936200640 1 --2- 172.21.15.43:0/2369435156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb930102360 0x7fb93010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:15.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.144+0000 7fb936200640 1 --2- 172.21.15.43:0/2369435156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9301019c0 0x7fb930101e20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:15.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.144+0000 7fb936200640 1 --2- 172.21.15.43:0/2369435156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9301079c0 0x7fb930107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:15.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.144+0000 7fb936200640 1 -- 172.21.15.43:0/2369435156 >> 172.21.15.43:0/2369435156 conn(0x7fb9300fd740 msgr2=0x7fb9300ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:15.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.145+0000 7fb936200640 1 -- 172.21.15.43:0/2369435156 shutdown_connections 2024-09-19T05:03:15.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.145+0000 7fb936200640 1 -- 172.21.15.43:0/2369435156 wait complete. 2024-09-19T05:03:15.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.146+0000 7fb936200640 1 Processor -- start 2024-09-19T05:03:15.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.146+0000 7fb936200640 1 -- start start 2024-09-19T05:03:15.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.146+0000 7fb936200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9301019c0 0x7fb93019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:15.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.146+0000 7fb936200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb930102360 0x7fb93019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:15.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.146+0000 7fb936200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9301079c0 0x7fb93019edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:15.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.146+0000 7fb936200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb930111dc0 con 0x7fb9301019c0 2024-09-19T05:03:15.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.146+0000 7fb936200640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb930111c40 con 0x7fb930102360 2024-09-19T05:03:15.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.146+0000 7fb936200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb930111f40 con 0x7fb9301079c0 2024-09-19T05:03:15.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.146+0000 7fb934e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9301019c0 0x7fb93019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:15.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.147+0000 7fb934e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9301019c0 0x7fb93019a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46058/0 (socket says 172.21.15.43:46058) 2024-09-19T05:03:15.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.147+0000 7fb934e00640 1 -- 172.21.15.43:0/339217117 learned_addr learned my addr 172.21.15.43:0/339217117 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:15.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.147+0000 7fb927e00640 1 --2- 172.21.15.43:0/339217117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb930102360 0x7fb93019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:15.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.147+0000 7fb935800640 1 --2- 172.21.15.43:0/339217117 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9301079c0 0x7fb93019edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:15.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.147+0000 7fb927e00640 1 -- 172.21.15.43:0/339217117 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9301079c0 msgr2=0x7fb93019edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:15.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.147+0000 7fb927e00640 1 --2- 172.21.15.43:0/339217117 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9301079c0 0x7fb93019edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:15.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.147+0000 7fb927e00640 1 -- 172.21.15.43:0/339217117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9301019c0 msgr2=0x7fb93019a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:15.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.147+0000 7fb927e00640 1 --2- 172.21.15.43:0/339217117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9301019c0 0x7fb93019a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:15.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.147+0000 7fb927e00640 1 -- 172.21.15.43:0/339217117 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93019f4d0 con 0x7fb930102360 2024-09-19T05:03:15.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.147+0000 7fb927e00640 1 --2- 172.21.15.43:0/339217117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb930102360 0x7fb93019aa60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb91802a950 tx=0x7fb918033b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:15.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.148+0000 7fb92e000640 1 -- 172.21.15.43:0/339217117 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91803f020 con 0x7fb930102360 2024-09-19T05:03:15.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.148+0000 7fb92e000640 1 -- 172.21.15.43:0/339217117 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb918030580 con 0x7fb930102360 2024-09-19T05:03:15.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.148+0000 7fb936200640 1 -- 172.21.15.43:0/339217117 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb93019f760 con 0x7fb930102360 2024-09-19T05:03:15.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.148+0000 7fb92e000640 1 -- 172.21.15.43:0/339217117 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb918037040 con 0x7fb930102360 2024-09-19T05:03:15.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.148+0000 7fb936200640 1 -- 172.21.15.43:0/339217117 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9301a70a0 con 0x7fb930102360 2024-09-19T05:03:15.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.149+0000 7fb936200640 1 -- 172.21.15.43:0/339217117 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb9300fe8b0 con 0x7fb930102360 2024-09-19T05:03:15.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.153+0000 7fb92e000640 1 -- 172.21.15.43:0/339217117 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fb918042020 con 0x7fb930102360 2024-09-19T05:03:15.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:14 smithi151 ceph-mon[35425]: pgmap v318: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 11 op/s; 122 B/s, 2 keys/s, 2 objects/s recovering 2024-09-19T05:03:15.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.153+0000 7fb92e000640 1 --2- 172.21.15.43:0/339217117 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fb9140a32d0 0x7fb9140a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:15.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.153+0000 7fb92e000640 1 -- 172.21.15.43:0/339217117 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fb9180ef690 con 0x7fb930102360 2024-09-19T05:03:15.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.153+0000 7fb934e00640 1 --2- 172.21.15.43:0/339217117 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fb9140a32d0 0x7fb9140a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:15.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.153+0000 7fb92e000640 1 --2- 172.21.15.43:0/339217117 >> [v2:172.21.15.43:6818/2008630290,v1:172.21.15.43:6819/2008630290] conn(0x7fb9140acf00 0x7fb9140af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:15.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.153+0000 7fb92e000640 1 -- 172.21.15.43:0/339217117 --> [v2:172.21.15.43:6818/2008630290,v1:172.21.15.43:6819/2008630290] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb9140afa70 con 0x7fb9140acf00 2024-09-19T05:03:15.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.153+0000 7fb92e000640 1 -- 172.21.15.43:0/339217117 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fb9180efa40 con 0x7fb930102360 2024-09-19T05:03:15.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.154+0000 7fb935800640 1 --2- 172.21.15.43:0/339217117 >> [v2:172.21.15.43:6818/2008630290,v1:172.21.15.43:6819/2008630290] conn(0x7fb9140acf00 0x7fb9140af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:15.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.155+0000 7fb935800640 1 --2- 172.21.15.43:0/339217117 >> [v2:172.21.15.43:6818/2008630290,v1:172.21.15.43:6819/2008630290] conn(0x7fb9140acf00 0x7fb9140af3a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:15.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.155+0000 7fb934e00640 1 --2- 172.21.15.43:0/339217117 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fb9140a32d0 0x7fb9140a5790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb920009c50 tx=0x7fb920005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:15.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.156+0000 7fb92e000640 1 -- 172.21.15.43:0/339217117 <== osd.6 v2:172.21.15.43:6818/2008630290 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb9140afa70 con 0x7fb9140acf00 2024-09-19T05:03:15.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.171+0000 7fb936200640 1 -- 172.21.15.43:0/339217117 --> [v2:172.21.15.43:6818/2008630290,v1:172.21.15.43:6819/2008630290] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb930103100 con 0x7fb9140acf00 2024-09-19T05:03:15.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.172+0000 7fb92e000640 1 -- 172.21.15.43:0/339217117 <== osd.6 v2:172.21.15.43:6818/2008630290 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb930103100 con 0x7fb9140acf00 2024-09-19T05:03:15.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.172+0000 7fb936200640 1 -- 172.21.15.43:0/339217117 >> [v2:172.21.15.43:6818/2008630290,v1:172.21.15.43:6819/2008630290] conn(0x7fb9140acf00 msgr2=0x7fb9140af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:15.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.172+0000 7fb936200640 1 --2- 172.21.15.43:0/339217117 >> [v2:172.21.15.43:6818/2008630290,v1:172.21.15.43:6819/2008630290] conn(0x7fb9140acf00 0x7fb9140af3a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:15.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.172+0000 7fb936200640 1 -- 172.21.15.43:0/339217117 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fb9140a32d0 msgr2=0x7fb9140a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:15.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.172+0000 7fb936200640 1 --2- 172.21.15.43:0/339217117 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fb9140a32d0 0x7fb9140a5790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb920009c50 tx=0x7fb920005f90 comp rx=0 tx=0).stop 2024-09-19T05:03:15.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.172+0000 7fb936200640 1 -- 172.21.15.43:0/339217117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb930102360 msgr2=0x7fb93019aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:15.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.172+0000 7fb936200640 1 --2- 172.21.15.43:0/339217117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb930102360 0x7fb93019aa60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb91802a950 tx=0x7fb918033b60 comp rx=0 tx=0).stop 2024-09-19T05:03:15.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.172+0000 7fb935800640 1 -- 172.21.15.43:0/339217117 reap_dead start 2024-09-19T05:03:15.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.173+0000 7fb936200640 1 -- 172.21.15.43:0/339217117 shutdown_connections 2024-09-19T05:03:15.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.173+0000 7fb936200640 1 -- 172.21.15.43:0/339217117 >> 172.21.15.43:0/339217117 conn(0x7fb9300fd740 msgr2=0x7fb9300fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:15.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.173+0000 7fb936200640 1 -- 172.21.15.43:0/339217117 shutdown_connections 2024-09-19T05:03:15.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.173+0000 7fb936200640 1 -- 172.21.15.43:0/339217117 wait complete. 2024-09-19T05:03:15.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.610+0000 7fa890200640 1 -- 172.21.15.43:0/2316464505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa8881049c0 msgr2=0x7fa888108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:15.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.610+0000 7fa890200640 1 --2- 172.21.15.43:0/2316464505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa8881049c0 0x7fa888108a10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa874009900 tx=0x7fa87402a3f0 comp rx=0 tx=0).stop 2024-09-19T05:03:15.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.610+0000 7fa890200640 1 -- 172.21.15.43:0/2316464505 shutdown_connections 2024-09-19T05:03:15.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.610+0000 7fa890200640 1 --2- 172.21.15.43:0/2316464505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa888109140 0x7fa888110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:15.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.610+0000 7fa890200640 1 --2- 172.21.15.43:0/2316464505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa8881049c0 0x7fa888108a10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:15.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.610+0000 7fa890200640 1 --2- 172.21.15.43:0/2316464505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa888104010 0x7fa8881043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:15.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.610+0000 7fa890200640 1 -- 172.21.15.43:0/2316464505 >> 172.21.15.43:0/2316464505 conn(0x7fa8880ffa50 msgr2=0x7fa888101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:15.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.610+0000 7fa890200640 1 -- 172.21.15.43:0/2316464505 shutdown_connections 2024-09-19T05:03:15.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.610+0000 7fa890200640 1 -- 172.21.15.43:0/2316464505 wait complete. 2024-09-19T05:03:15.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.611+0000 7fa890200640 1 Processor -- start 2024-09-19T05:03:15.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.611+0000 7fa890200640 1 -- start start 2024-09-19T05:03:15.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.611+0000 7fa890200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa888104010 0x7fa88819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:15.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.612+0000 7fa890200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8881049c0 0x7fa88819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:15.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.612+0000 7fa890200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa888109140 0x7fa88819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:15.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.612+0000 7fa890200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa888114140 con 0x7fa888109140 2024-09-19T05:03:15.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.612+0000 7fa890200640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa888113fc0 con 0x7fa888104010 2024-09-19T05:03:15.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.612+0000 7fa890200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa8881142c0 con 0x7fa8881049c0 2024-09-19T05:03:15.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.612+0000 7fa88d800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa888104010 0x7fa88819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:15.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.612+0000 7fa88d800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa888104010 0x7fa88819a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:50416/0 (socket says 172.21.15.43:50416) 2024-09-19T05:03:15.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.612+0000 7fa88d800640 1 -- 172.21.15.43:0/734921768 learned_addr learned my addr 172.21.15.43:0/734921768 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:15.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.612+0000 7fa88e200640 1 --2- 172.21.15.43:0/734921768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa888109140 0x7fa88819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:15.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.612+0000 7fa88ce00640 1 --2- 172.21.15.43:0/734921768 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8881049c0 0x7fa88819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:15.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.612+0000 7fa88ce00640 1 -- 172.21.15.43:0/734921768 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa888104010 msgr2=0x7fa88819a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:15.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.612+0000 7fa88ce00640 1 --2- 172.21.15.43:0/734921768 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa888104010 0x7fa88819a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:15.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.612+0000 7fa88ce00640 1 -- 172.21.15.43:0/734921768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa888109140 msgr2=0x7fa88819ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:15.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.612+0000 7fa88ce00640 1 --2- 172.21.15.43:0/734921768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa888109140 0x7fa88819ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:15.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.612+0000 7fa88ce00640 1 -- 172.21.15.43:0/734921768 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa88819f600 con 0x7fa8881049c0 2024-09-19T05:03:15.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.613+0000 7fa88ce00640 1 --2- 172.21.15.43:0/734921768 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8881049c0 0x7fa88819ab90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa874005940 tx=0x7fa8740040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:15.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.613+0000 7fa886000640 1 -- 172.21.15.43:0/734921768 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87403f030 con 0x7fa8881049c0 2024-09-19T05:03:15.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.613+0000 7fa890200640 1 -- 172.21.15.43:0/734921768 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa88819f890 con 0x7fa8881049c0 2024-09-19T05:03:15.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.613+0000 7fa886000640 1 -- 172.21.15.43:0/734921768 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8740044c0 con 0x7fa8881049c0 2024-09-19T05:03:15.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.613+0000 7fa886000640 1 -- 172.21.15.43:0/734921768 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa874035070 con 0x7fa8881049c0 2024-09-19T05:03:15.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.613+0000 7fa890200640 1 -- 172.21.15.43:0/734921768 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8881a71d0 con 0x7fa8881049c0 2024-09-19T05:03:15.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.614+0000 7fa890200640 1 -- 172.21.15.43:0/734921768 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa888062980 con 0x7fa8881049c0 2024-09-19T05:03:15.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.617+0000 7fa886000640 1 -- 172.21.15.43:0/734921768 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fa874042030 con 0x7fa8881049c0 2024-09-19T05:03:15.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.619+0000 7fa886000640 1 --2- 172.21.15.43:0/734921768 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa8700a32d0 0x7fa8700a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:15.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.620+0000 7fa886000640 1 -- 172.21.15.43:0/734921768 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fa8740f0470 con 0x7fa8881049c0 2024-09-19T05:03:15.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.620+0000 7fa88d800640 1 --2- 172.21.15.43:0/734921768 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa8700a32d0 0x7fa8700a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:15.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.620+0000 7fa886000640 1 --2- 172.21.15.43:0/734921768 >> [v2:172.21.15.151:6824/292202881,v1:172.21.15.151:6825/292202881] conn(0x7fa8700acdd0 0x7fa8700af270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:15.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.620+0000 7fa886000640 1 -- 172.21.15.43:0/734921768 --> [v2:172.21.15.151:6824/292202881,v1:172.21.15.151:6825/292202881] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa8700af940 con 0x7fa8700acdd0 2024-09-19T05:03:15.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.620+0000 7fa886000640 1 -- 172.21.15.43:0/734921768 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fa8740f0820 con 0x7fa8881049c0 2024-09-19T05:03:15.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.620+0000 7fa88e200640 1 --2- 172.21.15.43:0/734921768 >> [v2:172.21.15.151:6824/292202881,v1:172.21.15.151:6825/292202881] conn(0x7fa8700acdd0 0x7fa8700af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:15.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.620+0000 7fa88d800640 1 --2- 172.21.15.43:0/734921768 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa8700a32d0 0x7fa8700a5790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa87c009c90 tx=0x7fa87c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:15.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.621+0000 7fa88e200640 1 --2- 172.21.15.43:0/734921768 >> [v2:172.21.15.151:6824/292202881,v1:172.21.15.151:6825/292202881] conn(0x7fa8700acdd0 0x7fa8700af270 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:15.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.622+0000 7fa886000640 1 -- 172.21.15.43:0/734921768 <== osd.11 v2:172.21.15.151:6824/292202881 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa8700af940 con 0x7fa8700acdd0 2024-09-19T05:03:15.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.637+0000 7fa890200640 1 -- 172.21.15.43:0/734921768 --> [v2:172.21.15.151:6824/292202881,v1:172.21.15.151:6825/292202881] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa888062d80 con 0x7fa8700acdd0 2024-09-19T05:03:15.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.638+0000 7fa886000640 1 -- 172.21.15.43:0/734921768 <== osd.11 v2:172.21.15.151:6824/292202881 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa888062d80 con 0x7fa8700acdd0 2024-09-19T05:03:15.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.638+0000 7fa890200640 1 -- 172.21.15.43:0/734921768 >> [v2:172.21.15.151:6824/292202881,v1:172.21.15.151:6825/292202881] conn(0x7fa8700acdd0 msgr2=0x7fa8700af270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:15.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.638+0000 7fa890200640 1 --2- 172.21.15.43:0/734921768 >> [v2:172.21.15.151:6824/292202881,v1:172.21.15.151:6825/292202881] conn(0x7fa8700acdd0 0x7fa8700af270 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:15.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.639+0000 7fa890200640 1 -- 172.21.15.43:0/734921768 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa8700a32d0 msgr2=0x7fa8700a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:15.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.639+0000 7fa890200640 1 --2- 172.21.15.43:0/734921768 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa8700a32d0 0x7fa8700a5790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa87c009c90 tx=0x7fa87c005f90 comp rx=0 tx=0).stop 2024-09-19T05:03:15.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.639+0000 7fa890200640 1 -- 172.21.15.43:0/734921768 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8881049c0 msgr2=0x7fa88819ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:15.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.639+0000 7fa890200640 1 --2- 172.21.15.43:0/734921768 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8881049c0 0x7fa88819ab90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa874005940 tx=0x7fa8740040c0 comp rx=0 tx=0).stop 2024-09-19T05:03:15.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.639+0000 7fa88e200640 1 -- 172.21.15.43:0/734921768 reap_dead start 2024-09-19T05:03:15.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.639+0000 7fa890200640 1 -- 172.21.15.43:0/734921768 shutdown_connections 2024-09-19T05:03:15.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.639+0000 7fa890200640 1 -- 172.21.15.43:0/734921768 >> 172.21.15.43:0/734921768 conn(0x7fa8880ffa50 msgr2=0x7fa888100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:15.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.639+0000 7fa890200640 1 -- 172.21.15.43:0/734921768 shutdown_connections 2024-09-19T05:03:15.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:15.639+0000 7fa890200640 1 -- 172.21.15.43:0/734921768 wait complete. 2024-09-19T05:03:16.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.534+0000 7fb2dfe00640 1 -- 172.21.15.43:0/84166500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2d8076720 msgr2=0x7fb2d8076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:16.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.534+0000 7fb2dfe00640 1 --2- 172.21.15.43:0/84166500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2d8076720 0x7fb2d8076b00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc009900 tx=0x7fb2cc02a400 comp rx=0 tx=0).stop 2024-09-19T05:03:16.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.534+0000 7fb2dfe00640 1 -- 172.21.15.43:0/84166500 shutdown_connections 2024-09-19T05:03:16.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.534+0000 7fb2dfe00640 1 --2- 172.21.15.43:0/84166500 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2d8112a30 0x7fb2d8114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:16.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.534+0000 7fb2dfe00640 1 --2- 172.21.15.43:0/84166500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2d8077040 0x7fb2d81124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:16.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.534+0000 7fb2dfe00640 1 --2- 172.21.15.43:0/84166500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2d8076720 0x7fb2d8076b00 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:16.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.534+0000 7fb2dfe00640 1 -- 172.21.15.43:0/84166500 >> 172.21.15.43:0/84166500 conn(0x7fb2d80ffa90 msgr2=0x7fb2d8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:16.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.535+0000 7fb2dfe00640 1 -- 172.21.15.43:0/84166500 shutdown_connections 2024-09-19T05:03:16.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.535+0000 7fb2dfe00640 1 -- 172.21.15.43:0/84166500 wait complete. 2024-09-19T05:03:16.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.535+0000 7fb2dfe00640 1 Processor -- start 2024-09-19T05:03:16.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.536+0000 7fb2dfe00640 1 -- start start 2024-09-19T05:03:16.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.536+0000 7fb2dfe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2d8076720 0x7fb2d81a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:16.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.536+0000 7fb2dfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2d8077040 0x7fb2d81a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:16.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.536+0000 7fb2dfe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2d8112a30 0x7fb2d819e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:16.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.536+0000 7fb2dfe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d8118610 con 0x7fb2d8077040 2024-09-19T05:03:16.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.536+0000 7fb2dfe00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d8118490 con 0x7fb2d8112a30 2024-09-19T05:03:16.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.536+0000 7fb2dfe00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d8118790 con 0x7fb2d8076720 2024-09-19T05:03:16.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.536+0000 7fb2dd400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2d8076720 0x7fb2d81a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:16.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.536+0000 7fb2dca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2d8077040 0x7fb2d81a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:16.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.536+0000 7fb2dd400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2d8076720 0x7fb2d81a4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:52514/0 (socket says 172.21.15.43:52514) 2024-09-19T05:03:16.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.536+0000 7fb2dd400640 1 -- 172.21.15.43:0/2920532603 learned_addr learned my addr 172.21.15.43:0/2920532603 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:16.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.536+0000 7fb2dde00640 1 --2- 172.21.15.43:0/2920532603 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2d8112a30 0x7fb2d819e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:16.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.537+0000 7fb2dde00640 1 -- 172.21.15.43:0/2920532603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2d8076720 msgr2=0x7fb2d81a4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:16.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.537+0000 7fb2dde00640 1 --2- 172.21.15.43:0/2920532603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2d8076720 0x7fb2d81a4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:16.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.537+0000 7fb2dde00640 1 -- 172.21.15.43:0/2920532603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2d8077040 msgr2=0x7fb2d81a4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:16.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.537+0000 7fb2dde00640 1 --2- 172.21.15.43:0/2920532603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2d8077040 0x7fb2d81a4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:16.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.537+0000 7fb2dde00640 1 -- 172.21.15.43:0/2920532603 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2d819f010 con 0x7fb2d8112a30 2024-09-19T05:03:16.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.537+0000 7fb2dca00640 1 --2- 172.21.15.43:0/2920532603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2d8077040 0x7fb2d81a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:03:16.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.537+0000 7fb2dde00640 1 --2- 172.21.15.43:0/2920532603 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2d8112a30 0x7fb2d819e8e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb2d807aa10 tx=0x7fb2d000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:16.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.538+0000 7fb2d6000640 1 -- 172.21.15.43:0/2920532603 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d000cdf0 con 0x7fb2d8112a30 2024-09-19T05:03:16.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.538+0000 7fb2dfe00640 1 -- 172.21.15.43:0/2920532603 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2d819f300 con 0x7fb2d8112a30 2024-09-19T05:03:16.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.538+0000 7fb2d6000640 1 -- 172.21.15.43:0/2920532603 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2d0005020 con 0x7fb2d8112a30 2024-09-19T05:03:16.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.538+0000 7fb2d6000640 1 -- 172.21.15.43:0/2920532603 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0010690 con 0x7fb2d8112a30 2024-09-19T05:03:16.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.538+0000 7fb2dfe00640 1 -- 172.21.15.43:0/2920532603 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2d81ab5f0 con 0x7fb2d8112a30 2024-09-19T05:03:16.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.539+0000 7fb2dfe00640 1 -- 172.21.15.43:0/2920532603 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb2d8062900 con 0x7fb2d8112a30 2024-09-19T05:03:16.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.541+0000 7fb2d6000640 1 -- 172.21.15.43:0/2920532603 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fb2d00054d0 con 0x7fb2d8112a30 2024-09-19T05:03:16.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.542+0000 7fb2d6000640 1 --2- 172.21.15.43:0/2920532603 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fb2c00a32d0 0x7fb2c00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:16.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.542+0000 7fb2d6000640 1 -- 172.21.15.43:0/2920532603 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fb2d00d0230 con 0x7fb2d8112a30 2024-09-19T05:03:16.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.542+0000 7fb2dd400640 1 --2- 172.21.15.43:0/2920532603 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fb2c00a32d0 0x7fb2c00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:16.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.543+0000 7fb2d6000640 1 --2- 172.21.15.43:0/2920532603 >> [v2:172.21.15.96:6800/2140095130,v1:172.21.15.96:6801/2140095130] conn(0x7fb2c00acf00 0x7fb2c00af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:16.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.543+0000 7fb2d6000640 1 -- 172.21.15.43:0/2920532603 --> [v2:172.21.15.96:6800/2140095130,v1:172.21.15.96:6801/2140095130] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb2c00afa70 con 0x7fb2c00acf00 2024-09-19T05:03:16.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.543+0000 7fb2d6000640 1 -- 172.21.15.43:0/2920532603 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fb2d00d05e0 con 0x7fb2d8112a30 2024-09-19T05:03:16.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.543+0000 7fb2dca00640 1 --2- 172.21.15.43:0/2920532603 >> [v2:172.21.15.96:6800/2140095130,v1:172.21.15.96:6801/2140095130] conn(0x7fb2c00acf00 0x7fb2c00af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:16.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.543+0000 7fb2dd400640 1 --2- 172.21.15.43:0/2920532603 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fb2c00a32d0 0x7fb2c00a5790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc0096c0 tx=0x7fb2cc0046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:16.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.543+0000 7fb2dca00640 1 --2- 172.21.15.43:0/2920532603 >> [v2:172.21.15.96:6800/2140095130,v1:172.21.15.96:6801/2140095130] conn(0x7fb2c00acf00 0x7fb2c00af3a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:16.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.544+0000 7fb2d6000640 1 -- 172.21.15.43:0/2920532603 <== osd.1 v2:172.21.15.96:6800/2140095130 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb2c00afa70 con 0x7fb2c00acf00 2024-09-19T05:03:16.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.559+0000 7fb2dfe00640 1 -- 172.21.15.43:0/2920532603 --> [v2:172.21.15.96:6800/2140095130,v1:172.21.15.96:6801/2140095130] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb2d8062d00 con 0x7fb2c00acf00 2024-09-19T05:03:16.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.559+0000 7fb2d6000640 1 -- 172.21.15.43:0/2920532603 <== osd.1 v2:172.21.15.96:6800/2140095130 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb2d8062d00 con 0x7fb2c00acf00 2024-09-19T05:03:16.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.560+0000 7fb2dfe00640 1 -- 172.21.15.43:0/2920532603 >> [v2:172.21.15.96:6800/2140095130,v1:172.21.15.96:6801/2140095130] conn(0x7fb2c00acf00 msgr2=0x7fb2c00af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:16.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.560+0000 7fb2dfe00640 1 --2- 172.21.15.43:0/2920532603 >> [v2:172.21.15.96:6800/2140095130,v1:172.21.15.96:6801/2140095130] conn(0x7fb2c00acf00 0x7fb2c00af3a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:16.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.560+0000 7fb2dfe00640 1 -- 172.21.15.43:0/2920532603 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fb2c00a32d0 msgr2=0x7fb2c00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:16.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.560+0000 7fb2dfe00640 1 --2- 172.21.15.43:0/2920532603 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fb2c00a32d0 0x7fb2c00a5790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc0096c0 tx=0x7fb2cc0046c0 comp rx=0 tx=0).stop 2024-09-19T05:03:16.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.560+0000 7fb2dfe00640 1 -- 172.21.15.43:0/2920532603 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2d8112a30 msgr2=0x7fb2d819e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:16.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.560+0000 7fb2dfe00640 1 --2- 172.21.15.43:0/2920532603 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2d8112a30 0x7fb2d819e8e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb2d807aa10 tx=0x7fb2d000ef10 comp rx=0 tx=0).stop 2024-09-19T05:03:16.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.560+0000 7fb2dde00640 1 -- 172.21.15.43:0/2920532603 reap_dead start 2024-09-19T05:03:16.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.560+0000 7fb2dfe00640 1 -- 172.21.15.43:0/2920532603 shutdown_connections 2024-09-19T05:03:16.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.560+0000 7fb2dfe00640 1 -- 172.21.15.43:0/2920532603 >> 172.21.15.43:0/2920532603 conn(0x7fb2d80ffa90 msgr2=0x7fb2d8101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:16.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.560+0000 7fb2dfe00640 1 -- 172.21.15.43:0/2920532603 shutdown_connections 2024-09-19T05:03:16.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:16.561+0000 7fb2dfe00640 1 -- 172.21.15.43:0/2920532603 wait complete. 2024-09-19T05:03:16.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:16 smithi096 ceph-mon[35085]: pgmap v319: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s; 122 B/s, 2 keys/s, 2 objects/s recovering 2024-09-19T05:03:17.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:16 smithi043 ceph-mon[30800]: pgmap v319: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s; 122 B/s, 2 keys/s, 2 objects/s recovering 2024-09-19T05:03:17.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:16 smithi151 ceph-mon[35425]: pgmap v319: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s; 122 B/s, 2 keys/s, 2 objects/s recovering 2024-09-19T05:03:17.233 INFO:teuthology.orchestra.run.smithi043.stdout:296352743453 2024-09-19T05:03:17.233 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-19T05:03:18.151 INFO:teuthology.orchestra.run.smithi043.stdout:188978561074 2024-09-19T05:03:18.151 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-19T05:03:18.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.617+0000 7f105a000640 1 -- 172.21.15.43:0/4073737093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1054102370 msgr2=0x7f105410e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:18.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.617+0000 7f105a000640 1 --2- 172.21.15.43:0/4073737093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1054102370 0x7f105410e8a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f104c009cd0 tx=0x7f104c02a8e0 comp rx=0 tx=0).stop 2024-09-19T05:03:18.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.617+0000 7f105a000640 1 -- 172.21.15.43:0/4073737093 shutdown_connections 2024-09-19T05:03:18.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.617+0000 7f105a000640 1 --2- 172.21.15.43:0/4073737093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1054102370 0x7f105410e8a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:18.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.617+0000 7f105a000640 1 --2- 172.21.15.43:0/4073737093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10541019d0 0x7f1054101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:18.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.617+0000 7f105a000640 1 --2- 172.21.15.43:0/4073737093 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10541079d0 0x7f1054107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:18.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.617+0000 7f105a000640 1 -- 172.21.15.43:0/4073737093 >> 172.21.15.43:0/4073737093 conn(0x7f10540fd710 msgr2=0x7f10540ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:18.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.618+0000 7f105a000640 1 -- 172.21.15.43:0/4073737093 shutdown_connections 2024-09-19T05:03:18.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.618+0000 7f105a000640 1 -- 172.21.15.43:0/4073737093 wait complete. 2024-09-19T05:03:18.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.618+0000 7f105a000640 1 Processor -- start 2024-09-19T05:03:18.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.619+0000 7f105a000640 1 -- start start 2024-09-19T05:03:18.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.619+0000 7f105a000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10541019d0 0x7f105419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:18.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.619+0000 7f105a000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1054102370 0x7f105419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:18.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.619+0000 7f105a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10541079d0 0x7f105419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:18.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.619+0000 7f105a000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1054111e20 con 0x7f10541079d0 2024-09-19T05:03:18.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.619+0000 7f105a000640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1054111ca0 con 0x7f10541019d0 2024-09-19T05:03:18.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.619+0000 7f105a000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1054111fa0 con 0x7f1054102370 2024-09-19T05:03:18.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.619+0000 7f1058c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10541019d0 0x7f105419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:18.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.619+0000 7f1059600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10541079d0 0x7f105419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:18.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.619+0000 7f1058c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10541019d0 0x7f105419a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:53230/0 (socket says 172.21.15.43:53230) 2024-09-19T05:03:18.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.619+0000 7f1058c00640 1 -- 172.21.15.43:0/3445504132 learned_addr learned my addr 172.21.15.43:0/3445504132 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:18.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.620+0000 7f1058c00640 1 -- 172.21.15.43:0/3445504132 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1054102370 msgr2=0x7f105419a9a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T05:03:18.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.620+0000 7f1058c00640 1 --2- 172.21.15.43:0/3445504132 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1054102370 0x7f105419a9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:18.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.620+0000 7f1058c00640 1 -- 172.21.15.43:0/3445504132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10541079d0 msgr2=0x7f105419ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:18.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.620+0000 7f1058c00640 1 --2- 172.21.15.43:0/3445504132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10541079d0 0x7f105419ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:18.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.620+0000 7f1058c00640 1 -- 172.21.15.43:0/3445504132 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f105419f410 con 0x7f10541019d0 2024-09-19T05:03:18.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.620+0000 7f1058c00640 1 --2- 172.21.15.43:0/3445504132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10541019d0 0x7f105419a460 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f104800e9f0 tx=0x7f104800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:18.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.620+0000 7f1051600640 1 -- 172.21.15.43:0/3445504132 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104800cd80 con 0x7f10541019d0 2024-09-19T05:03:18.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.620+0000 7f105a000640 1 -- 172.21.15.43:0/3445504132 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f105419f700 con 0x7f10541019d0 2024-09-19T05:03:18.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.620+0000 7f1051600640 1 -- 172.21.15.43:0/3445504132 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1048005500 con 0x7f10541019d0 2024-09-19T05:03:18.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.620+0000 7f1051600640 1 -- 172.21.15.43:0/3445504132 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1048010690 con 0x7f10541019d0 2024-09-19T05:03:18.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.621+0000 7f105a000640 1 -- 172.21.15.43:0/3445504132 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10541a6fe0 con 0x7f10541019d0 2024-09-19T05:03:18.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.621+0000 7f105a000640 1 -- 172.21.15.43:0/3445504132 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f10540fef60 con 0x7f10541019d0 2024-09-19T05:03:18.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.625+0000 7f1051600640 1 -- 172.21.15.43:0/3445504132 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f10480046c0 con 0x7f10541019d0 2024-09-19T05:03:18.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.626+0000 7f1051600640 1 --2- 172.21.15.43:0/3445504132 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f103c0a3350 0x7f103c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:18.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.626+0000 7f1051600640 1 -- 172.21.15.43:0/3445504132 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f10480d02e0 con 0x7f10541019d0 2024-09-19T05:03:18.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.626+0000 7f1053e00640 1 --2- 172.21.15.43:0/3445504132 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f103c0a3350 0x7f103c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:18.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.626+0000 7f1051600640 1 --2- 172.21.15.43:0/3445504132 >> [v2:172.21.15.43:6802/825351486,v1:172.21.15.43:6803/825351486] conn(0x7f103c0acf80 0x7f103c0af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:18.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.626+0000 7f1051600640 1 -- 172.21.15.43:0/3445504132 --> [v2:172.21.15.43:6802/825351486,v1:172.21.15.43:6803/825351486] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f104801aec0 con 0x7f103c0acf80 2024-09-19T05:03:18.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.626+0000 7f1051600640 1 -- 172.21.15.43:0/3445504132 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f10480d0690 con 0x7f10541019d0 2024-09-19T05:03:18.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.626+0000 7f1059600640 1 --2- 172.21.15.43:0/3445504132 >> [v2:172.21.15.43:6802/825351486,v1:172.21.15.43:6803/825351486] conn(0x7f103c0acf80 0x7f103c0af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:18.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.627+0000 7f1053e00640 1 --2- 172.21.15.43:0/3445504132 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f103c0a3350 0x7f103c0a5810 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f105410eb10 tx=0x7f1040005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:18.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.627+0000 7f1059600640 1 --2- 172.21.15.43:0/3445504132 >> [v2:172.21.15.43:6802/825351486,v1:172.21.15.43:6803/825351486] conn(0x7f103c0acf80 0x7f103c0af420 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:18.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.628+0000 7f1051600640 1 -- 172.21.15.43:0/3445504132 <== osd.0 v2:172.21.15.43:6802/825351486 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f104801aec0 con 0x7f103c0acf80 2024-09-19T05:03:18.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.643+0000 7f105a000640 1 -- 172.21.15.43:0/3445504132 --> [v2:172.21.15.43:6802/825351486,v1:172.21.15.43:6803/825351486] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1054103110 con 0x7f103c0acf80 2024-09-19T05:03:18.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.643+0000 7f1051600640 1 -- 172.21.15.43:0/3445504132 <== osd.0 v2:172.21.15.43:6802/825351486 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1054103110 con 0x7f103c0acf80 2024-09-19T05:03:18.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.644+0000 7f105a000640 1 -- 172.21.15.43:0/3445504132 >> [v2:172.21.15.43:6802/825351486,v1:172.21.15.43:6803/825351486] conn(0x7f103c0acf80 msgr2=0x7f103c0af420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:18.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.644+0000 7f105a000640 1 --2- 172.21.15.43:0/3445504132 >> [v2:172.21.15.43:6802/825351486,v1:172.21.15.43:6803/825351486] conn(0x7f103c0acf80 0x7f103c0af420 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:18.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.644+0000 7f105a000640 1 -- 172.21.15.43:0/3445504132 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f103c0a3350 msgr2=0x7f103c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:18.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.644+0000 7f105a000640 1 --2- 172.21.15.43:0/3445504132 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f103c0a3350 0x7f103c0a5810 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f105410eb10 tx=0x7f1040005f90 comp rx=0 tx=0).stop 2024-09-19T05:03:18.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.644+0000 7f105a000640 1 -- 172.21.15.43:0/3445504132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10541019d0 msgr2=0x7f105419a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:18.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.644+0000 7f105a000640 1 --2- 172.21.15.43:0/3445504132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10541019d0 0x7f105419a460 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f104800e9f0 tx=0x7f104800edf0 comp rx=0 tx=0).stop 2024-09-19T05:03:18.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.644+0000 7f1059600640 1 -- 172.21.15.43:0/3445504132 reap_dead start 2024-09-19T05:03:18.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.644+0000 7f105a000640 1 -- 172.21.15.43:0/3445504132 shutdown_connections 2024-09-19T05:03:18.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.644+0000 7f105a000640 1 -- 172.21.15.43:0/3445504132 >> 172.21.15.43:0/3445504132 conn(0x7f10540fd710 msgr2=0x7f105410d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:18.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.644+0000 7f105a000640 1 -- 172.21.15.43:0/3445504132 shutdown_connections 2024-09-19T05:03:18.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:17.644+0000 7f105a000640 1 -- 172.21.15.43:0/3445504132 wait complete. 2024-09-19T05:03:18.945 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:18 smithi043 ceph-mon[30800]: pgmap v320: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 8 op/s; 103 B/s, 1 keys/s, 2 objects/s recovering 2024-09-19T05:03:18.980 INFO:teuthology.orchestra.run.smithi043.stdout:244813135912 2024-09-19T05:03:18.980 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-19T05:03:19.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:18 smithi151 ceph-mon[35425]: pgmap v320: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 8 op/s; 103 B/s, 1 keys/s, 2 objects/s recovering 2024-09-19T05:03:19.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:18 smithi096 ceph-mon[35085]: pgmap v320: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 8 op/s; 103 B/s, 1 keys/s, 2 objects/s recovering 2024-09-19T05:03:19.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.297+0000 7f282d200640 1 -- 172.21.15.43:0/987443884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2828103e30 msgr2=0x7f2828104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:19.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.297+0000 7f282d200640 1 --2- 172.21.15.43:0/987443884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2828103e30 0x7f2828104210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2818009900 tx=0x7f281802a410 comp rx=0 tx=0).stop 2024-09-19T05:03:19.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.297+0000 7f282d200640 1 -- 172.21.15.43:0/987443884 shutdown_connections 2024-09-19T05:03:19.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.297+0000 7f282d200640 1 --2- 172.21.15.43:0/987443884 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2828108f60 0x7f2828110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:19.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.297+0000 7f282d200640 1 --2- 172.21.15.43:0/987443884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28281047e0 0x7f2828108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:19.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.297+0000 7f282d200640 1 --2- 172.21.15.43:0/987443884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2828103e30 0x7f2828104210 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:19.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.297+0000 7f282d200640 1 -- 172.21.15.43:0/987443884 >> 172.21.15.43:0/987443884 conn(0x7f28280ff910 msgr2=0x7f2828101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:19.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.297+0000 7f282d200640 1 -- 172.21.15.43:0/987443884 shutdown_connections 2024-09-19T05:03:19.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.297+0000 7f282d200640 1 -- 172.21.15.43:0/987443884 wait complete. 2024-09-19T05:03:19.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.298+0000 7f282d200640 1 Processor -- start 2024-09-19T05:03:19.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.298+0000 7f282d200640 1 -- start start 2024-09-19T05:03:19.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.298+0000 7f282d200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2828103e30 0x7f282819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:19.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.299+0000 7f282d200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28281047e0 0x7f282819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:19.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.299+0000 7f282d200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2828108f60 0x7f282819edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:19.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.299+0000 7f282d200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2828113fc0 con 0x7f28281047e0 2024-09-19T05:03:19.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.299+0000 7f282d200640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2828113e40 con 0x7f2828108f60 2024-09-19T05:03:19.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.299+0000 7f282d200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2828114140 con 0x7f2828103e30 2024-09-19T05:03:19.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.299+0000 7f281fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28281047e0 0x7f282819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:19.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.299+0000 7f281fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28281047e0 0x7f282819aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35832/0 (socket says 172.21.15.43:35832) 2024-09-19T05:03:19.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.299+0000 7f281fe00640 1 -- 172.21.15.43:0/3405271454 learned_addr learned my addr 172.21.15.43:0/3405271454 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:19.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.299+0000 7f2827400640 1 --2- 172.21.15.43:0/3405271454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2828103e30 0x7f282819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:19.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.299+0000 7f2827e00640 1 --2- 172.21.15.43:0/3405271454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2828108f60 0x7f282819edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:19.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.299+0000 7f281fe00640 1 -- 172.21.15.43:0/3405271454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2828103e30 msgr2=0x7f282819a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:19.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.299+0000 7f281fe00640 1 --2- 172.21.15.43:0/3405271454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2828103e30 0x7f282819a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:19.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.299+0000 7f281fe00640 1 -- 172.21.15.43:0/3405271454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2828108f60 msgr2=0x7f282819edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:19.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.299+0000 7f281fe00640 1 --2- 172.21.15.43:0/3405271454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2828108f60 0x7f282819edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:19.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.299+0000 7f281fe00640 1 -- 172.21.15.43:0/3405271454 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f282819f490 con 0x7f28281047e0 2024-09-19T05:03:19.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.300+0000 7f281fe00640 1 --2- 172.21.15.43:0/3405271454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28281047e0 0x7f282819aa20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f281000e9f0 tx=0x7f281000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:19.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.300+0000 7f2824c00640 1 -- 172.21.15.43:0/3405271454 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281000cd80 con 0x7f28281047e0 2024-09-19T05:03:19.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.300+0000 7f2824c00640 1 -- 172.21.15.43:0/3405271454 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2810005500 con 0x7f28281047e0 2024-09-19T05:03:19.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.300+0000 7f282d200640 1 -- 172.21.15.43:0/3405271454 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f282819f780 con 0x7f28281047e0 2024-09-19T05:03:19.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.300+0000 7f2824c00640 1 -- 172.21.15.43:0/3405271454 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2810010690 con 0x7f28281047e0 2024-09-19T05:03:19.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.300+0000 7f282d200640 1 -- 172.21.15.43:0/3405271454 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28281a7060 con 0x7f28281047e0 2024-09-19T05:03:19.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.301+0000 7f282d200640 1 -- 172.21.15.43:0/3405271454 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2828062980 con 0x7f28281047e0 2024-09-19T05:03:19.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.303+0000 7f2824c00640 1 -- 172.21.15.43:0/3405271454 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f28100046c0 con 0x7f28281047e0 2024-09-19T05:03:19.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.304+0000 7f2824c00640 1 --2- 172.21.15.43:0/3405271454 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f280c0a32d0 0x7f280c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:19.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.304+0000 7f2824c00640 1 -- 172.21.15.43:0/3405271454 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f28100d0320 con 0x7f28281047e0 2024-09-19T05:03:19.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.305+0000 7f2827400640 1 --2- 172.21.15.43:0/3405271454 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f280c0a32d0 0x7f280c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:19.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.305+0000 7f2824c00640 1 --2- 172.21.15.43:0/3405271454 >> [v2:172.21.15.43:6826/4053438621,v1:172.21.15.43:6827/4053438621] conn(0x7f280c0acf00 0x7f280c0af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:19.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.305+0000 7f2824c00640 1 -- 172.21.15.43:0/3405271454 --> [v2:172.21.15.43:6826/4053438621,v1:172.21.15.43:6827/4053438621] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2810005350 con 0x7f280c0acf00 2024-09-19T05:03:19.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.305+0000 7f2824c00640 1 -- 172.21.15.43:0/3405271454 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f28100d06d0 con 0x7f28281047e0 2024-09-19T05:03:19.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.305+0000 7f2827e00640 1 --2- 172.21.15.43:0/3405271454 >> [v2:172.21.15.43:6826/4053438621,v1:172.21.15.43:6827/4053438621] conn(0x7f280c0acf00 0x7f280c0af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:19.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.305+0000 7f2827400640 1 --2- 172.21.15.43:0/3405271454 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f280c0a32d0 0x7f280c0a5790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2818009230 tx=0x7f28180058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:19.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.306+0000 7f2827e00640 1 --2- 172.21.15.43:0/3405271454 >> [v2:172.21.15.43:6826/4053438621,v1:172.21.15.43:6827/4053438621] conn(0x7f280c0acf00 0x7f280c0af3a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:19.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.307+0000 7f2824c00640 1 -- 172.21.15.43:0/3405271454 <== osd.9 v2:172.21.15.43:6826/4053438621 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2810005350 con 0x7f280c0acf00 2024-09-19T05:03:19.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.321+0000 7f282d200640 1 -- 172.21.15.43:0/3405271454 --> [v2:172.21.15.43:6826/4053438621,v1:172.21.15.43:6827/4053438621] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2828062e00 con 0x7f280c0acf00 2024-09-19T05:03:19.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.321+0000 7f2824c00640 1 -- 172.21.15.43:0/3405271454 <== osd.9 v2:172.21.15.43:6826/4053438621 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2828062e00 con 0x7f280c0acf00 2024-09-19T05:03:19.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.322+0000 7f282d200640 1 -- 172.21.15.43:0/3405271454 >> [v2:172.21.15.43:6826/4053438621,v1:172.21.15.43:6827/4053438621] conn(0x7f280c0acf00 msgr2=0x7f280c0af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:19.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.322+0000 7f282d200640 1 --2- 172.21.15.43:0/3405271454 >> [v2:172.21.15.43:6826/4053438621,v1:172.21.15.43:6827/4053438621] conn(0x7f280c0acf00 0x7f280c0af3a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:19.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.322+0000 7f282d200640 1 -- 172.21.15.43:0/3405271454 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f280c0a32d0 msgr2=0x7f280c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:19.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.322+0000 7f282d200640 1 --2- 172.21.15.43:0/3405271454 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f280c0a32d0 0x7f280c0a5790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2818009230 tx=0x7f28180058d0 comp rx=0 tx=0).stop 2024-09-19T05:03:19.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.322+0000 7f282d200640 1 -- 172.21.15.43:0/3405271454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28281047e0 msgr2=0x7f282819aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:19.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.322+0000 7f282d200640 1 --2- 172.21.15.43:0/3405271454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28281047e0 0x7f282819aa20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f281000e9f0 tx=0x7f281000edf0 comp rx=0 tx=0).stop 2024-09-19T05:03:19.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.322+0000 7f2827e00640 1 -- 172.21.15.43:0/3405271454 reap_dead start 2024-09-19T05:03:19.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.322+0000 7f282d200640 1 -- 172.21.15.43:0/3405271454 shutdown_connections 2024-09-19T05:03:19.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.322+0000 7f282d200640 1 -- 172.21.15.43:0/3405271454 >> 172.21.15.43:0/3405271454 conn(0x7f28280ff910 msgr2=0x7f2828100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:19.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.322+0000 7f282d200640 1 -- 172.21.15.43:0/3405271454 shutdown_connections 2024-09-19T05:03:19.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.322+0000 7f282d200640 1 -- 172.21.15.43:0/3405271454 wait complete. 2024-09-19T05:03:20.009 INFO:teuthology.orchestra.run.smithi043.stdout:34359738452 2024-09-19T05:03:20.009 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-19T05:03:20.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.501+0000 7f1471e00640 1 -- 172.21.15.43:0/1077452838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f146c112a30 msgr2=0x7f146c114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:20.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.501+0000 7f1471e00640 1 --2- 172.21.15.43:0/1077452838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f146c112a30 0x7f146c114e20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1464009ca0 tx=0x7f146402a8c0 comp rx=0 tx=0).stop 2024-09-19T05:03:20.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.501+0000 7f1471e00640 1 -- 172.21.15.43:0/1077452838 shutdown_connections 2024-09-19T05:03:20.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.501+0000 7f1471e00640 1 --2- 172.21.15.43:0/1077452838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f146c112a30 0x7f146c114e20 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:20.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.501+0000 7f1471e00640 1 --2- 172.21.15.43:0/1077452838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f146c077040 0x7f146c1124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:20.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.501+0000 7f1471e00640 1 --2- 172.21.15.43:0/1077452838 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f146c076720 0x7f146c076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:20.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.501+0000 7f1471e00640 1 -- 172.21.15.43:0/1077452838 >> 172.21.15.43:0/1077452838 conn(0x7f146c0ffa90 msgr2=0x7f146c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:20.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.501+0000 7f1471e00640 1 -- 172.21.15.43:0/1077452838 shutdown_connections 2024-09-19T05:03:20.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.501+0000 7f1471e00640 1 -- 172.21.15.43:0/1077452838 wait complete. 2024-09-19T05:03:20.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.502+0000 7f1471e00640 1 Processor -- start 2024-09-19T05:03:20.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.502+0000 7f1471e00640 1 -- start start 2024-09-19T05:03:20.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.503+0000 7f1471e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f146c076720 0x7f146c1a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:20.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.503+0000 7f1471e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f146c077040 0x7f146c1a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:20.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.503+0000 7f1471e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f146c112a30 0x7f146c19e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:20.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.503+0000 7f1471e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f146c1185d0 con 0x7f146c076720 2024-09-19T05:03:20.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.503+0000 7f1471e00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f146c118450 con 0x7f146c112a30 2024-09-19T05:03:20.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.503+0000 7f1471e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f146c118750 con 0x7f146c077040 2024-09-19T05:03:20.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.503+0000 7f146aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f146c076720 0x7f146c1a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:20.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.503+0000 7f146b400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f146c112a30 0x7f146c19e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:20.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.503+0000 7f146aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f146c076720 0x7f146c1a46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35850/0 (socket says 172.21.15.43:35850) 2024-09-19T05:03:20.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.503+0000 7f146aa00640 1 -- 172.21.15.43:0/4260765947 learned_addr learned my addr 172.21.15.43:0/4260765947 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.503+0000 7f146a000640 1 --2- 172.21.15.43:0/4260765947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f146c077040 0x7f146c1a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.503+0000 7f146aa00640 1 -- 172.21.15.43:0/4260765947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f146c077040 msgr2=0x7f146c1a4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.503+0000 7f146aa00640 1 --2- 172.21.15.43:0/4260765947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f146c077040 0x7f146c1a4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.503+0000 7f146aa00640 1 -- 172.21.15.43:0/4260765947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f146c112a30 msgr2=0x7f146c19e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.503+0000 7f146aa00640 1 --2- 172.21.15.43:0/4260765947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f146c112a30 0x7f146c19e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.503+0000 7f146aa00640 1 -- 172.21.15.43:0/4260765947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f146c19ef90 con 0x7f146c076720 2024-09-19T05:03:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.503+0000 7f146b400640 1 --2- 172.21.15.43:0/4260765947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f146c112a30 0x7f146c19e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:03:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.504+0000 7f146aa00640 1 --2- 172.21.15.43:0/4260765947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f146c076720 0x7f146c1a46a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f145c00e9f0 tx=0x7f145c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.504+0000 7f1470a00640 1 -- 172.21.15.43:0/4260765947 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145c00cd80 con 0x7f146c076720 2024-09-19T05:03:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.504+0000 7f1470a00640 1 -- 172.21.15.43:0/4260765947 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f145c005500 con 0x7f146c076720 2024-09-19T05:03:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.504+0000 7f1470a00640 1 -- 172.21.15.43:0/4260765947 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145c010690 con 0x7f146c076720 2024-09-19T05:03:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.504+0000 7f1471e00640 1 -- 172.21.15.43:0/4260765947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f146c19f280 con 0x7f146c076720 2024-09-19T05:03:20.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.504+0000 7f1471e00640 1 -- 172.21.15.43:0/4260765947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f146c1ab570 con 0x7f146c076720 2024-09-19T05:03:20.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.505+0000 7f1471e00640 1 -- 172.21.15.43:0/4260765947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f146c062980 con 0x7f146c076720 2024-09-19T05:03:20.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.507+0000 7f1470a00640 1 -- 172.21.15.43:0/4260765947 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f145c0046c0 con 0x7f146c076720 2024-09-19T05:03:20.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.508+0000 7f1470a00640 1 --2- 172.21.15.43:0/4260765947 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f14500a32d0 0x7f14500a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:20.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.509+0000 7f1470a00640 1 -- 172.21.15.43:0/4260765947 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f145c0d0320 con 0x7f146c076720 2024-09-19T05:03:20.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.509+0000 7f146a000640 1 --2- 172.21.15.43:0/4260765947 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f14500a32d0 0x7f14500a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:20.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.509+0000 7f1470a00640 1 --2- 172.21.15.43:0/4260765947 >> [v2:172.21.15.96:6816/3588713275,v1:172.21.15.96:6817/3588713275] conn(0x7f14500acf00 0x7f14500af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:20.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.509+0000 7f1470a00640 1 -- 172.21.15.43:0/4260765947 --> [v2:172.21.15.96:6816/3588713275,v1:172.21.15.96:6817/3588713275] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f145c005350 con 0x7f14500acf00 2024-09-19T05:03:20.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.509+0000 7f1470a00640 1 -- 172.21.15.43:0/4260765947 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f145c0d06d0 con 0x7f146c076720 2024-09-19T05:03:20.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.509+0000 7f146b400640 1 --2- 172.21.15.43:0/4260765947 >> [v2:172.21.15.96:6816/3588713275,v1:172.21.15.96:6817/3588713275] conn(0x7f14500acf00 0x7f14500af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:20.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.509+0000 7f146a000640 1 --2- 172.21.15.43:0/4260765947 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f14500a32d0 0x7f14500a5790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f146c079290 tx=0x7f1458005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:20.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.510+0000 7f146b400640 1 --2- 172.21.15.43:0/4260765947 >> [v2:172.21.15.96:6816/3588713275,v1:172.21.15.96:6817/3588713275] conn(0x7f14500acf00 0x7f14500af3a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:20.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.511+0000 7f1470a00640 1 -- 172.21.15.43:0/4260765947 <== osd.7 v2:172.21.15.96:6816/3588713275 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f145c005350 con 0x7f14500acf00 2024-09-19T05:03:20.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.525+0000 7f1471e00640 1 -- 172.21.15.43:0/4260765947 --> [v2:172.21.15.96:6816/3588713275,v1:172.21.15.96:6817/3588713275] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f146c062e00 con 0x7f14500acf00 2024-09-19T05:03:20.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.526+0000 7f1470a00640 1 -- 172.21.15.43:0/4260765947 <== osd.7 v2:172.21.15.96:6816/3588713275 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f146c062e00 con 0x7f14500acf00 2024-09-19T05:03:20.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.526+0000 7f1471e00640 1 -- 172.21.15.43:0/4260765947 >> [v2:172.21.15.96:6816/3588713275,v1:172.21.15.96:6817/3588713275] conn(0x7f14500acf00 msgr2=0x7f14500af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:20.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.526+0000 7f1471e00640 1 --2- 172.21.15.43:0/4260765947 >> [v2:172.21.15.96:6816/3588713275,v1:172.21.15.96:6817/3588713275] conn(0x7f14500acf00 0x7f14500af3a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:20.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.526+0000 7f1471e00640 1 -- 172.21.15.43:0/4260765947 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f14500a32d0 msgr2=0x7f14500a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:20.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.526+0000 7f1471e00640 1 --2- 172.21.15.43:0/4260765947 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f14500a32d0 0x7f14500a5790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f146c079290 tx=0x7f1458005f90 comp rx=0 tx=0).stop 2024-09-19T05:03:20.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.526+0000 7f1471e00640 1 -- 172.21.15.43:0/4260765947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f146c076720 msgr2=0x7f146c1a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:20.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.526+0000 7f1471e00640 1 --2- 172.21.15.43:0/4260765947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f146c076720 0x7f146c1a46a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f145c00e9f0 tx=0x7f145c00edf0 comp rx=0 tx=0).stop 2024-09-19T05:03:20.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.526+0000 7f146b400640 1 -- 172.21.15.43:0/4260765947 reap_dead start 2024-09-19T05:03:20.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.527+0000 7f1471e00640 1 -- 172.21.15.43:0/4260765947 shutdown_connections 2024-09-19T05:03:20.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.527+0000 7f1471e00640 1 -- 172.21.15.43:0/4260765947 >> 172.21.15.43:0/4260765947 conn(0x7f146c0ffa90 msgr2=0x7f146c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:20.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.527+0000 7f1471e00640 1 -- 172.21.15.43:0/4260765947 shutdown_connections 2024-09-19T05:03:20.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:19.527+0000 7f1471e00640 1 -- 172.21.15.43:0/4260765947 wait complete. 2024-09-19T05:03:20.813 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:20.853 INFO:teuthology.orchestra.run.smithi043.stdout:326417514521 2024-09-19T05:03:20.853 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-19T05:03:21.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:20 smithi096 ceph-mon[35085]: pgmap v321: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s; 93 B/s, 1 keys/s, 1 objects/s recovering 2024-09-19T05:03:21.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:20 smithi043 ceph-mon[30800]: pgmap v321: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s; 93 B/s, 1 keys/s, 1 objects/s recovering 2024-09-19T05:03:21.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:20 smithi151 ceph-mon[35425]: pgmap v321: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s; 93 B/s, 1 keys/s, 1 objects/s recovering 2024-09-19T05:03:21.608 INFO:teuthology.orchestra.run.smithi043.stdout:55834574926 2024-09-19T05:03:21.609 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-19T05:03:21.613 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:22.005 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:21 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:03:22.042 INFO:teuthology.orchestra.run.smithi043.stdout:133143986236 2024-09-19T05:03:22.042 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-19T05:03:22.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:21 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:03:22.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:21 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:03:22.466 INFO:teuthology.orchestra.run.smithi043.stdout:274877906979 2024-09-19T05:03:22.467 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-19T05:03:22.998 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:22 smithi043 ceph-mon[30800]: pgmap v322: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s; 81 B/s, 1 keys/s, 1 objects/s recovering 2024-09-19T05:03:22.998 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:22 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:03:22.998 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:22 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:03:22.998 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:22 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:03:22.999 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:22 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:03:23.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:22 smithi151 ceph-mon[35425]: pgmap v322: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s; 81 B/s, 1 keys/s, 1 objects/s recovering 2024-09-19T05:03:23.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:22 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:03:23.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:22 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:03:23.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:22 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:03:23.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:22 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:03:23.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:22 smithi096 ceph-mon[35085]: pgmap v322: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s; 81 B/s, 1 keys/s, 1 objects/s recovering 2024-09-19T05:03:23.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:22 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:03:23.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:22 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:03:23.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:22 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:03:23.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:22 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:03:23.300 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:23.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.301+0000 7f68bf000640 1 -- 172.21.15.43:0/3059161624 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68b810e460 msgr2=0x7f68b8110850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:23.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.301+0000 7f68bf000640 1 --2- 172.21.15.43:0/3059161624 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68b810e460 0x7f68b8110850 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f68b0009be0 tx=0x7f68b002a6c0 comp rx=0 tx=0).stop 2024-09-19T05:03:23.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.302+0000 7f68bf000640 1 -- 172.21.15.43:0/3059161624 shutdown_connections 2024-09-19T05:03:23.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.302+0000 7f68bf000640 1 --2- 172.21.15.43:0/3059161624 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68b810e460 0x7f68b8110850 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:23.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.302+0000 7f68bf000640 1 --2- 172.21.15.43:0/3059161624 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f68b8101260 0x7f68b810df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:23.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.302+0000 7f68bf000640 1 --2- 172.21.15.43:0/3059161624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68b81008b0 0x7f68b8100c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:23.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.302+0000 7f68bf000640 1 -- 172.21.15.43:0/3059161624 >> 172.21.15.43:0/3059161624 conn(0x7f68b80fc780 msgr2=0x7f68b80feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:23.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.303+0000 7f68bf000640 1 -- 172.21.15.43:0/3059161624 shutdown_connections 2024-09-19T05:03:23.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.303+0000 7f68bf000640 1 -- 172.21.15.43:0/3059161624 wait complete. 2024-09-19T05:03:23.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.305+0000 7f68bf000640 1 Processor -- start 2024-09-19T05:03:23.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.306+0000 7f68bf000640 1 -- start start 2024-09-19T05:03:23.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.307+0000 7f68bf000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f68b81008b0 0x7f68b810c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:23.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.307+0000 7f68bf000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68b8101260 0x7f68b8073c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:23.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.307+0000 7f68bf000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68b810e460 0x7f68b810ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:23.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.308+0000 7f68bf000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f68b8103a20 con 0x7f68b810e460 2024-09-19T05:03:23.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.308+0000 7f68bf000640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f68b81038a0 con 0x7f68b8101260 2024-09-19T05:03:23.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.308+0000 7f68bf000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f68b8103ba0 con 0x7f68b81008b0 2024-09-19T05:03:23.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.308+0000 7f68b7e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f68b81008b0 0x7f68b810c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:23.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.308+0000 7f68b7e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f68b81008b0 0x7f68b810c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:52594/0 (socket says 172.21.15.43:52594) 2024-09-19T05:03:23.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.308+0000 7f68b7400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68b8101260 0x7f68b8073c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:23.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.308+0000 7f68b7400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68b8101260 0x7f68b8073c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:53288/0 (socket says 172.21.15.43:53288) 2024-09-19T05:03:23.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.308+0000 7f68b7400640 1 -- 172.21.15.43:0/510118751 learned_addr learned my addr 172.21.15.43:0/510118751 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:23.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.308+0000 7f68bd000640 1 --2- 172.21.15.43:0/510118751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68b810e460 0x7f68b810ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:23.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.309+0000 7f68bd000640 1 -- 172.21.15.43:0/510118751 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f68b81008b0 msgr2=0x7f68b810c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:23.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.309+0000 7f68bd000640 1 --2- 172.21.15.43:0/510118751 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f68b81008b0 0x7f68b810c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:23.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.309+0000 7f68bd000640 1 -- 172.21.15.43:0/510118751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68b8101260 msgr2=0x7f68b8073c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:23.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.309+0000 7f68bd000640 1 --2- 172.21.15.43:0/510118751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68b8101260 0x7f68b8073c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:23.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.309+0000 7f68bd000640 1 -- 172.21.15.43:0/510118751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b810d3e0 con 0x7f68b810e460 2024-09-19T05:03:23.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.310+0000 7f68bd000640 1 --2- 172.21.15.43:0/510118751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68b810e460 0x7f68b810ccb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f68b002acb0 tx=0x7f68b00305b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:23.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.311+0000 7f68b4c00640 1 -- 172.21.15.43:0/510118751 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b0041020 con 0x7f68b810e460 2024-09-19T05:03:23.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.311+0000 7f68bf000640 1 -- 172.21.15.43:0/510118751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68b81b3780 con 0x7f68b810e460 2024-09-19T05:03:23.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.311+0000 7f68bf000640 1 -- 172.21.15.43:0/510118751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68b81b3c80 con 0x7f68b810e460 2024-09-19T05:03:23.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.311+0000 7f68b4c00640 1 -- 172.21.15.43:0/510118751 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68b0003e50 con 0x7f68b810e460 2024-09-19T05:03:23.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.311+0000 7f68b4c00640 1 -- 172.21.15.43:0/510118751 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b0036040 con 0x7f68b810e460 2024-09-19T05:03:23.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.314+0000 7f68bf000640 1 -- 172.21.15.43:0/510118751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68b8100c90 con 0x7f68b810e460 2024-09-19T05:03:23.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.318+0000 7f68b4c00640 1 -- 172.21.15.43:0/510118751 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f68b0044020 con 0x7f68b810e460 2024-09-19T05:03:23.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.320+0000 7f68b4c00640 1 --2- 172.21.15.43:0/510118751 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f689c0a3350 0x7f689c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:23.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.321+0000 7f68b4c00640 1 -- 172.21.15.43:0/510118751 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f68b00f06b0 con 0x7f68b810e460 2024-09-19T05:03:23.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.322+0000 7f68b7e00640 1 --2- 172.21.15.43:0/510118751 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f689c0a3350 0x7f689c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:23.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.323+0000 7f68b7e00640 1 --2- 172.21.15.43:0/510118751 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f689c0a3350 0x7f689c0a5810 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f68a8005fd0 tx=0x7f68a8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:23.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.328+0000 7f68b4c00640 1 -- 172.21.15.43:0/510118751 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f68b004b020 con 0x7f68b810e460 2024-09-19T05:03:23.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.532+0000 7f68bf000640 1 -- 172.21.15.43:0/510118751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f68b8106900 con 0x7f68b810e460 2024-09-19T05:03:23.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.533+0000 7f68b4c00640 1 -- 172.21.15.43:0/510118751 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f68b0031440 con 0x7f68b810e460 2024-09-19T05:03:23.535 INFO:teuthology.orchestra.run.smithi043.stdout:103079215172 2024-09-19T05:03:23.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.536+0000 7f68bf000640 1 -- 172.21.15.43:0/510118751 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f689c0a3350 msgr2=0x7f689c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:23.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.536+0000 7f68bf000640 1 --2- 172.21.15.43:0/510118751 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f689c0a3350 0x7f689c0a5810 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f68a8005fd0 tx=0x7f68a8009260 comp rx=0 tx=0).stop 2024-09-19T05:03:23.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.536+0000 7f68bf000640 1 -- 172.21.15.43:0/510118751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68b810e460 msgr2=0x7f68b810ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:23.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.536+0000 7f68bf000640 1 --2- 172.21.15.43:0/510118751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68b810e460 0x7f68b810ccb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f68b002acb0 tx=0x7f68b00305b0 comp rx=0 tx=0).stop 2024-09-19T05:03:23.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.536+0000 7f68bf000640 1 -- 172.21.15.43:0/510118751 shutdown_connections 2024-09-19T05:03:23.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.536+0000 7f68bf000640 1 --2- 172.21.15.43:0/510118751 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f689c0a3350 0x7f689c0a5810 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:23.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.536+0000 7f68bf000640 1 --2- 172.21.15.43:0/510118751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68b810e460 0x7f68b810ccb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:23.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.536+0000 7f68bf000640 1 --2- 172.21.15.43:0/510118751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68b8101260 0x7f68b8073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:23.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.536+0000 7f68bf000640 1 --2- 172.21.15.43:0/510118751 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f68b81008b0 0x7f68b810c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:23.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.536+0000 7f68bf000640 1 -- 172.21.15.43:0/510118751 >> 172.21.15.43:0/510118751 conn(0x7f68b80fc780 msgr2=0x7f68b80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:23.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.536+0000 7f68bf000640 1 -- 172.21.15.43:0/510118751 shutdown_connections 2024-09-19T05:03:23.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:23.536+0000 7f68bf000640 1 -- 172.21.15.43:0/510118751 wait complete. 2024-09-19T05:03:23.639 INFO:teuthology.orchestra.run.smithi043.stdout:210453397550 2024-09-19T05:03:23.639 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-19T05:03:23.920 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:23 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/510118751' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T05:03:24.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:23 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/510118751' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T05:03:24.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:23 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/510118751' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T05:03:24.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.487+0000 7ff80a800640 1 -- 172.21.15.43:0/4137669587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff804074ee0 msgr2=0x7ff80410dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:24.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.487+0000 7ff80a800640 1 --2- 172.21.15.43:0/4137669587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff804074ee0 0x7ff80410dee0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0009900 tx=0x7ff7f002a480 comp rx=0 tx=0).stop 2024-09-19T05:03:24.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.488+0000 7ff80a800640 1 -- 172.21.15.43:0/4137669587 shutdown_connections 2024-09-19T05:03:24.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.488+0000 7ff80a800640 1 --2- 172.21.15.43:0/4137669587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff80410e420 0x7ff804110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:24.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.488+0000 7ff80a800640 1 --2- 172.21.15.43:0/4137669587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff804074ee0 0x7ff80410dee0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:24.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.488+0000 7ff80a800640 1 --2- 172.21.15.43:0/4137669587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8040745c0 0x7ff8040749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:24.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.488+0000 7ff80a800640 1 -- 172.21.15.43:0/4137669587 >> 172.21.15.43:0/4137669587 conn(0x7ff8040fd8c0 msgr2=0x7ff8040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:24.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.488+0000 7ff80a800640 1 -- 172.21.15.43:0/4137669587 shutdown_connections 2024-09-19T05:03:24.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.488+0000 7ff80a800640 1 -- 172.21.15.43:0/4137669587 wait complete. 2024-09-19T05:03:24.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.489+0000 7ff80a800640 1 Processor -- start 2024-09-19T05:03:24.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.489+0000 7ff80a800640 1 -- start start 2024-09-19T05:03:24.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.489+0000 7ff80a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8040745c0 0x7ff80419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:24.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.489+0000 7ff80a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff804074ee0 0x7ff80419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:24.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.489+0000 7ff80a800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff80410e420 0x7ff80419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:24.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.490+0000 7ff80a800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff8041141b0 con 0x7ff8040745c0 2024-09-19T05:03:24.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.490+0000 7ff80a800640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff804114030 con 0x7ff80410e420 2024-09-19T05:03:24.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.490+0000 7ff80a800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff804114330 con 0x7ff804074ee0 2024-09-19T05:03:24.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.490+0000 7ff803400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8040745c0 0x7ff80419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:24.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.490+0000 7ff803400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8040745c0 0x7ff80419a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35890/0 (socket says 172.21.15.43:35890) 2024-09-19T05:03:24.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.490+0000 7ff803400640 1 -- 172.21.15.43:0/3859316565 learned_addr learned my addr 172.21.15.43:0/3859316565 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:24.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.490+0000 7ff802a00640 1 --2- 172.21.15.43:0/3859316565 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff804074ee0 0x7ff80419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:24.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.490+0000 7ff803e00640 1 --2- 172.21.15.43:0/3859316565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff80410e420 0x7ff80419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:24.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.491+0000 7ff803e00640 1 -- 172.21.15.43:0/3859316565 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff804074ee0 msgr2=0x7ff80419abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:24.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.491+0000 7ff803e00640 1 --2- 172.21.15.43:0/3859316565 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff804074ee0 0x7ff80419abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:24.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.491+0000 7ff803e00640 1 -- 172.21.15.43:0/3859316565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8040745c0 msgr2=0x7ff80419a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:24.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.491+0000 7ff803e00640 1 --2- 172.21.15.43:0/3859316565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8040745c0 0x7ff80419a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:24.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.491+0000 7ff803e00640 1 -- 172.21.15.43:0/3859316565 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff80419f620 con 0x7ff80410e420 2024-09-19T05:03:24.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.491+0000 7ff803e00640 1 --2- 172.21.15.43:0/3859316565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff80410e420 0x7ff80419ef40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc005fd0 tx=0x7ff7fc00edd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:24.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.491+0000 7ff7fbe00640 1 -- 172.21.15.43:0/3859316565 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc004d20 con 0x7ff80410e420 2024-09-19T05:03:24.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.491+0000 7ff7fbe00640 1 -- 172.21.15.43:0/3859316565 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7fc005020 con 0x7ff80410e420 2024-09-19T05:03:24.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.491+0000 7ff80a800640 1 -- 172.21.15.43:0/3859316565 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff80419f910 con 0x7ff80410e420 2024-09-19T05:03:24.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.491+0000 7ff7fbe00640 1 -- 172.21.15.43:0/3859316565 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc0116d0 con 0x7ff80410e420 2024-09-19T05:03:24.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.492+0000 7ff80a800640 1 -- 172.21.15.43:0/3859316565 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8041a71f0 con 0x7ff80410e420 2024-09-19T05:03:24.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.493+0000 7ff80a800640 1 -- 172.21.15.43:0/3859316565 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8040749a0 con 0x7ff80410e420 2024-09-19T05:03:24.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.495+0000 7ff7fbe00640 1 -- 172.21.15.43:0/3859316565 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7ff7fc005540 con 0x7ff80410e420 2024-09-19T05:03:24.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.496+0000 7ff7fbe00640 1 --2- 172.21.15.43:0/3859316565 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff7e80a3320 0x7ff7e80a57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:24.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.496+0000 7ff7fbe00640 1 -- 172.21.15.43:0/3859316565 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ff7fc0cfad0 con 0x7ff80410e420 2024-09-19T05:03:24.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.496+0000 7ff803400640 1 --2- 172.21.15.43:0/3859316565 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff7e80a3320 0x7ff7e80a57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:24.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.497+0000 7ff803400640 1 --2- 172.21.15.43:0/3859316565 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff7e80a3320 0x7ff7e80a57e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4009c90 tx=0x7ff7f4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:24.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.500+0000 7ff7fbe00640 1 -- 172.21.15.43:0/3859316565 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff7fc016030 con 0x7ff80410e420 2024-09-19T05:03:24.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.692+0000 7ff80a800640 1 -- 172.21.15.43:0/3859316565 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7ff804109010 con 0x7ff80410e420 2024-09-19T05:03:24.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.692+0000 7ff7fbe00640 1 -- 172.21.15.43:0/3859316565 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff7fc09c000 con 0x7ff80410e420 2024-09-19T05:03:24.693 INFO:teuthology.orchestra.run.smithi043.stdout:77309411402 2024-09-19T05:03:24.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.694+0000 7ff80a800640 1 -- 172.21.15.43:0/3859316565 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff7e80a3320 msgr2=0x7ff7e80a57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:24.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.694+0000 7ff80a800640 1 --2- 172.21.15.43:0/3859316565 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff7e80a3320 0x7ff7e80a57e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4009c90 tx=0x7ff7f4005f90 comp rx=0 tx=0).stop 2024-09-19T05:03:24.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.694+0000 7ff80a800640 1 -- 172.21.15.43:0/3859316565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff80410e420 msgr2=0x7ff80419ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:24.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.694+0000 7ff80a800640 1 --2- 172.21.15.43:0/3859316565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff80410e420 0x7ff80419ef40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc005fd0 tx=0x7ff7fc00edd0 comp rx=0 tx=0).stop 2024-09-19T05:03:24.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.695+0000 7ff80a800640 1 -- 172.21.15.43:0/3859316565 shutdown_connections 2024-09-19T05:03:24.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.695+0000 7ff80a800640 1 --2- 172.21.15.43:0/3859316565 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff7e80a3320 0x7ff7e80a57e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:24.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.695+0000 7ff80a800640 1 --2- 172.21.15.43:0/3859316565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff80410e420 0x7ff80419ef40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:24.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.695+0000 7ff80a800640 1 --2- 172.21.15.43:0/3859316565 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff804074ee0 0x7ff80419abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:24.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.695+0000 7ff80a800640 1 --2- 172.21.15.43:0/3859316565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8040745c0 0x7ff80419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:24.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.695+0000 7ff80a800640 1 -- 172.21.15.43:0/3859316565 >> 172.21.15.43:0/3859316565 conn(0x7ff8040fd8c0 msgr2=0x7ff8040fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:24.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.695+0000 7ff80a800640 1 -- 172.21.15.43:0/3859316565 shutdown_connections 2024-09-19T05:03:24.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.695+0000 7ff80a800640 1 -- 172.21.15.43:0/3859316565 wait complete. 2024-09-19T05:03:25.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:24 smithi043 ceph-mon[30800]: pgmap v323: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 81 B/s, 1 keys/s, 1 objects/s recovering 2024-09-19T05:03:25.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:24 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/3859316565' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T05:03:25.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.871+0000 7f236aa00640 1 -- 172.21.15.43:0/2548126529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2364104010 msgr2=0x7f23641043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:25.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.871+0000 7f236aa00640 1 --2- 172.21.15.43:0/2548126529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2364104010 0x7f23641043f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2354009900 tx=0x7f235402a410 comp rx=0 tx=0).stop 2024-09-19T05:03:25.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.871+0000 7f236aa00640 1 -- 172.21.15.43:0/2548126529 shutdown_connections 2024-09-19T05:03:25.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.871+0000 7f236aa00640 1 --2- 172.21.15.43:0/2548126529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2364109140 0x7f2364110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:25.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.871+0000 7f236aa00640 1 --2- 172.21.15.43:0/2548126529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23641049c0 0x7f2364108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:25.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.871+0000 7f236aa00640 1 --2- 172.21.15.43:0/2548126529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2364104010 0x7f23641043f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:25.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.871+0000 7f236aa00640 1 -- 172.21.15.43:0/2548126529 >> 172.21.15.43:0/2548126529 conn(0x7f23640ffa50 msgr2=0x7f2364101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:25.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.871+0000 7f236aa00640 1 -- 172.21.15.43:0/2548126529 shutdown_connections 2024-09-19T05:03:25.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.871+0000 7f236aa00640 1 -- 172.21.15.43:0/2548126529 wait complete. 2024-09-19T05:03:25.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.872+0000 7f236aa00640 1 Processor -- start 2024-09-19T05:03:25.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.873+0000 7f236aa00640 1 -- start start 2024-09-19T05:03:25.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.873+0000 7f236aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2364104010 0x7f236419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:25.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.873+0000 7f236aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23641049c0 0x7f236419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:25.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.873+0000 7f236aa00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2364109140 0x7f236419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:25.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.873+0000 7f236aa00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f23641141c0 con 0x7f23641049c0 2024-09-19T05:03:25.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.873+0000 7f236aa00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2364114040 con 0x7f2364109140 2024-09-19T05:03:25.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.873+0000 7f236aa00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2364114340 con 0x7f2364104010 2024-09-19T05:03:25.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.873+0000 7f2363400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23641049c0 0x7f236419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:25.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.873+0000 7f2363400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23641049c0 0x7f236419ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35914/0 (socket says 172.21.15.43:35914) 2024-09-19T05:03:25.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.873+0000 7f2363400640 1 -- 172.21.15.43:0/3937814980 learned_addr learned my addr 172.21.15.43:0/3937814980 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.873+0000 7f2363e00640 1 --2- 172.21.15.43:0/3937814980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2364104010 0x7f236419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.873+0000 7f2368a00640 1 --2- 172.21.15.43:0/3937814980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2364109140 0x7f236419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.874+0000 7f2363400640 1 -- 172.21.15.43:0/3937814980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2364104010 msgr2=0x7f236419a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.874+0000 7f2363400640 1 --2- 172.21.15.43:0/3937814980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2364104010 0x7f236419a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.874+0000 7f2363400640 1 -- 172.21.15.43:0/3937814980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2364109140 msgr2=0x7f236419eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.874+0000 7f2363400640 1 --2- 172.21.15.43:0/3937814980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2364109140 0x7f236419eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.874+0000 7f2363400640 1 -- 172.21.15.43:0/3937814980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f236419f5d0 con 0x7f23641049c0 2024-09-19T05:03:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.874+0000 7f2363400640 1 --2- 172.21.15.43:0/3937814980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23641049c0 0x7f236419ab60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f235000e9f0 tx=0x7f235000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.874+0000 7f2360c00640 1 -- 172.21.15.43:0/3937814980 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235000cd80 con 0x7f23641049c0 2024-09-19T05:03:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.874+0000 7f2360c00640 1 -- 172.21.15.43:0/3937814980 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2350010910 con 0x7f23641049c0 2024-09-19T05:03:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.875+0000 7f2360c00640 1 -- 172.21.15.43:0/3937814980 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2350018990 con 0x7f23641049c0 2024-09-19T05:03:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.875+0000 7f236aa00640 1 -- 172.21.15.43:0/3937814980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f236419f8c0 con 0x7f23641049c0 2024-09-19T05:03:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.875+0000 7f236aa00640 1 -- 172.21.15.43:0/3937814980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23641a71a0 con 0x7f23641049c0 2024-09-19T05:03:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.876+0000 7f236aa00640 1 -- 172.21.15.43:0/3937814980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23641043f0 con 0x7f23641049c0 2024-09-19T05:03:25.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.878+0000 7f2360c00640 1 -- 172.21.15.43:0/3937814980 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f2350010ac0 con 0x7f23641049c0 2024-09-19T05:03:25.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.880+0000 7f2360c00640 1 --2- 172.21.15.43:0/3937814980 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f23480a3350 0x7f23480a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:25.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.880+0000 7f2360c00640 1 -- 172.21.15.43:0/3937814980 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f23500cfd50 con 0x7f23641049c0 2024-09-19T05:03:25.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.880+0000 7f2363e00640 1 --2- 172.21.15.43:0/3937814980 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f23480a3350 0x7f23480a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:25.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.881+0000 7f2363e00640 1 --2- 172.21.15.43:0/3937814980 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f23480a3350 0x7f23480a5810 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2354000c00 tx=0x7f2354005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:25.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:24.884+0000 7f2360c00640 1 -- 172.21.15.43:0/3937814980 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f235009c280 con 0x7f23641049c0 2024-09-19T05:03:25.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:25.072+0000 7f236aa00640 1 -- 172.21.15.43:0/3937814980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f236402d7c0 con 0x7f23641049c0 2024-09-19T05:03:25.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:25.072+0000 7f2360c00640 1 -- 172.21.15.43:0/3937814980 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f23500a0580 con 0x7f23641049c0 2024-09-19T05:03:25.089 INFO:teuthology.orchestra.run.smithi043.stdout:158913790010 2024-09-19T05:03:25.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:25.075+0000 7f236aa00640 1 -- 172.21.15.43:0/3937814980 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f23480a3350 msgr2=0x7f23480a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:25.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:25.075+0000 7f236aa00640 1 --2- 172.21.15.43:0/3937814980 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f23480a3350 0x7f23480a5810 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2354000c00 tx=0x7f2354005b50 comp rx=0 tx=0).stop 2024-09-19T05:03:25.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:25.075+0000 7f236aa00640 1 -- 172.21.15.43:0/3937814980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23641049c0 msgr2=0x7f236419ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:25.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:25.075+0000 7f236aa00640 1 --2- 172.21.15.43:0/3937814980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23641049c0 0x7f236419ab60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f235000e9f0 tx=0x7f235000eeb0 comp rx=0 tx=0).stop 2024-09-19T05:03:25.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:25.075+0000 7f236aa00640 1 -- 172.21.15.43:0/3937814980 shutdown_connections 2024-09-19T05:03:25.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:25.075+0000 7f236aa00640 1 --2- 172.21.15.43:0/3937814980 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f23480a3350 0x7f23480a5810 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:25.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:25.075+0000 7f236aa00640 1 --2- 172.21.15.43:0/3937814980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2364109140 0x7f236419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:25.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:25.075+0000 7f236aa00640 1 --2- 172.21.15.43:0/3937814980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23641049c0 0x7f236419ab60 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:25.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:25.075+0000 7f236aa00640 1 --2- 172.21.15.43:0/3937814980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2364104010 0x7f236419a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:25.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:25.075+0000 7f236aa00640 1 -- 172.21.15.43:0/3937814980 >> 172.21.15.43:0/3937814980 conn(0x7f23640ffa50 msgr2=0x7f2364100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:25.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:25.075+0000 7f236aa00640 1 -- 172.21.15.43:0/3937814980 shutdown_connections 2024-09-19T05:03:25.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:25.075+0000 7f236aa00640 1 -- 172.21.15.43:0/3937814980 wait complete. 2024-09-19T05:03:25.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:24 smithi151 ceph-mon[35425]: pgmap v323: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 81 B/s, 1 keys/s, 1 objects/s recovering 2024-09-19T05:03:25.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:24 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/3859316565' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T05:03:25.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:24 smithi096 ceph-mon[35085]: pgmap v323: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 81 B/s, 1 keys/s, 1 objects/s recovering 2024-09-19T05:03:25.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:24 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/3859316565' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T05:03:25.439 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:25.809 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215169 got 103079215172 for osd.3 2024-09-19T05:03:25.809 DEBUG:teuthology.parallel:result is None 2024-09-19T05:03:26.024 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:25 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/3937814980' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T05:03:26.047 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:26.048 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:26.048 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:26.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:25 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/3937814980' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T05:03:26.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:25 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/3937814980' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T05:03:26.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.448+0000 7f7c44600640 1 -- 172.21.15.43:0/2993175047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c3c077040 msgr2=0x7f7c3c1124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:26.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.448+0000 7f7c44600640 1 --2- 172.21.15.43:0/2993175047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c3c077040 0x7f7c3c1124f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7c28009900 tx=0x7f7c2802a3f0 comp rx=0 tx=0).stop 2024-09-19T05:03:26.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.448+0000 7f7c44600640 1 -- 172.21.15.43:0/2993175047 shutdown_connections 2024-09-19T05:03:26.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.448+0000 7f7c44600640 1 --2- 172.21.15.43:0/2993175047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c3c112a30 0x7f7c3c114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:26.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.448+0000 7f7c44600640 1 --2- 172.21.15.43:0/2993175047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c3c077040 0x7f7c3c1124f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:26.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.448+0000 7f7c44600640 1 --2- 172.21.15.43:0/2993175047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c3c076720 0x7f7c3c076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:26.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.448+0000 7f7c44600640 1 -- 172.21.15.43:0/2993175047 >> 172.21.15.43:0/2993175047 conn(0x7f7c3c0ffa90 msgr2=0x7f7c3c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:26.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.448+0000 7f7c44600640 1 -- 172.21.15.43:0/2993175047 shutdown_connections 2024-09-19T05:03:26.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.448+0000 7f7c44600640 1 -- 172.21.15.43:0/2993175047 wait complete. 2024-09-19T05:03:26.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.449+0000 7f7c44600640 1 Processor -- start 2024-09-19T05:03:26.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.449+0000 7f7c44600640 1 -- start start 2024-09-19T05:03:26.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.450+0000 7f7c44600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c3c076720 0x7f7c3c1a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:26.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.450+0000 7f7c44600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c3c077040 0x7f7c3c1a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:26.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.450+0000 7f7c44600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c3c112a30 0x7f7c3c19e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:26.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.450+0000 7f7c44600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3c1185a0 con 0x7f7c3c112a30 2024-09-19T05:03:26.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.450+0000 7f7c44600640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3c118420 con 0x7f7c3c076720 2024-09-19T05:03:26.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.450+0000 7f7c44600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3c118720 con 0x7f7c3c077040 2024-09-19T05:03:26.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.450+0000 7f7c42600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c3c112a30 0x7f7c3c19e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:26.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.450+0000 7f7c42600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c3c112a30 0x7f7c3c19e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58746/0 (socket says 172.21.15.43:58746) 2024-09-19T05:03:26.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.450+0000 7f7c42600640 1 -- 172.21.15.43:0/2192130169 learned_addr learned my addr 172.21.15.43:0/2192130169 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:26.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.450+0000 7f7c41c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c3c076720 0x7f7c3c1a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:26.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.450+0000 7f7c41200640 1 --2- 172.21.15.43:0/2192130169 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c3c077040 0x7f7c3c1a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:26.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.451+0000 7f7c42600640 1 -- 172.21.15.43:0/2192130169 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c3c077040 msgr2=0x7f7c3c1a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:26.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.451+0000 7f7c42600640 1 --2- 172.21.15.43:0/2192130169 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c3c077040 0x7f7c3c1a4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:26.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.451+0000 7f7c42600640 1 -- 172.21.15.43:0/2192130169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c3c076720 msgr2=0x7f7c3c1a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:26.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.451+0000 7f7c42600640 1 --2- 172.21.15.43:0/2192130169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c3c076720 0x7f7c3c1a46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:26.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.451+0000 7f7c42600640 1 -- 172.21.15.43:0/2192130169 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c3c19efc0 con 0x7f7c3c112a30 2024-09-19T05:03:26.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.451+0000 7f7c41200640 1 --2- 172.21.15.43:0/2192130169 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c3c077040 0x7f7c3c1a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:03:26.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.451+0000 7f7c41c00640 1 --2- 172.21.15.43:0/2192130169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c3c076720 0x7f7c3c1a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:03:26.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.451+0000 7f7c42600640 1 --2- 172.21.15.43:0/2192130169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c3c112a30 0x7f7c3c19e890 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7c3400cce0 tx=0x7f7c3400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:26.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.451+0000 7f7c3a000640 1 -- 172.21.15.43:0/2192130169 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3400f450 con 0x7f7c3c112a30 2024-09-19T05:03:26.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.451+0000 7f7c3a000640 1 -- 172.21.15.43:0/2192130169 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c3400fad0 con 0x7f7c3c112a30 2024-09-19T05:03:26.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.451+0000 7f7c3a000640 1 -- 172.21.15.43:0/2192130169 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c34018bd0 con 0x7f7c3c112a30 2024-09-19T05:03:26.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.451+0000 7f7c44600640 1 -- 172.21.15.43:0/2192130169 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c3c19f2b0 con 0x7f7c3c112a30 2024-09-19T05:03:26.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.452+0000 7f7c44600640 1 -- 172.21.15.43:0/2192130169 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c3c1ab5a0 con 0x7f7c3c112a30 2024-09-19T05:03:26.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.453+0000 7f7c44600640 1 -- 172.21.15.43:0/2192130169 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c3c076b00 con 0x7f7c3c112a30 2024-09-19T05:03:26.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.455+0000 7f7c3a000640 1 -- 172.21.15.43:0/2192130169 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f7c3400fc80 con 0x7f7c3c112a30 2024-09-19T05:03:26.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.456+0000 7f7c3a000640 1 --2- 172.21.15.43:0/2192130169 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f7c240a3350 0x7f7c240a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:26.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.456+0000 7f7c3a000640 1 -- 172.21.15.43:0/2192130169 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f7c340d0100 con 0x7f7c3c112a30 2024-09-19T05:03:26.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.456+0000 7f7c41c00640 1 --2- 172.21.15.43:0/2192130169 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f7c240a3350 0x7f7c240a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:26.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.457+0000 7f7c41c00640 1 --2- 172.21.15.43:0/2192130169 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f7c240a3350 0x7f7c240a5810 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c19f9d0 tx=0x7f7c30005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:26.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.460+0000 7f7c3a000640 1 -- 172.21.15.43:0/2192130169 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7c34014030 con 0x7f7c3c112a30 2024-09-19T05:03:26.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.647+0000 7f7c44600640 1 -- 172.21.15.43:0/2192130169 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f7c3c10d630 con 0x7f7c3c112a30 2024-09-19T05:03:26.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.647+0000 7f7c3a000640 1 -- 172.21.15.43:0/2192130169 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7c3409cc80 con 0x7f7c3c112a30 2024-09-19T05:03:26.649 INFO:teuthology.orchestra.run.smithi043.stdout:188978561076 2024-09-19T05:03:26.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.650+0000 7f7c44600640 1 -- 172.21.15.43:0/2192130169 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f7c240a3350 msgr2=0x7f7c240a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:26.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.650+0000 7f7c44600640 1 --2- 172.21.15.43:0/2192130169 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f7c240a3350 0x7f7c240a5810 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c19f9d0 tx=0x7f7c30005f90 comp rx=0 tx=0).stop 2024-09-19T05:03:26.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.650+0000 7f7c44600640 1 -- 172.21.15.43:0/2192130169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c3c112a30 msgr2=0x7f7c3c19e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:26.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.650+0000 7f7c44600640 1 --2- 172.21.15.43:0/2192130169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c3c112a30 0x7f7c3c19e890 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7c3400cce0 tx=0x7f7c3400a620 comp rx=0 tx=0).stop 2024-09-19T05:03:26.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.650+0000 7f7c44600640 1 -- 172.21.15.43:0/2192130169 shutdown_connections 2024-09-19T05:03:26.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.650+0000 7f7c44600640 1 --2- 172.21.15.43:0/2192130169 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f7c240a3350 0x7f7c240a5810 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:26.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.650+0000 7f7c44600640 1 --2- 172.21.15.43:0/2192130169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c3c112a30 0x7f7c3c19e890 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:26.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.650+0000 7f7c44600640 1 --2- 172.21.15.43:0/2192130169 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c3c077040 0x7f7c3c1a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:26.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.650+0000 7f7c44600640 1 --2- 172.21.15.43:0/2192130169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c3c076720 0x7f7c3c1a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:26.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.650+0000 7f7c44600640 1 -- 172.21.15.43:0/2192130169 >> 172.21.15.43:0/2192130169 conn(0x7f7c3c0ffa90 msgr2=0x7f7c3c113630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:26.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.650+0000 7f7c44600640 1 -- 172.21.15.43:0/2192130169 shutdown_connections 2024-09-19T05:03:26.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:26.650+0000 7f7c44600640 1 -- 172.21.15.43:0/2192130169 wait complete. 2024-09-19T05:03:26.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:26 smithi043 ceph-mon[30800]: pgmap v324: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:03:26.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:26 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/2192130169' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T05:03:27.020 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790007 got 158913790010 for osd.5 2024-09-19T05:03:27.021 DEBUG:teuthology.parallel:result is None 2024-09-19T05:03:27.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:26 smithi151 ceph-mon[35425]: pgmap v324: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:03:27.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:26 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/2192130169' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T05:03:27.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:26 smithi096 ceph-mon[35085]: pgmap v324: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:03:27.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:26 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/2192130169' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T05:03:27.390 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411399 got 77309411402 for osd.2 2024-09-19T05:03:27.390 DEBUG:teuthology.parallel:result is None 2024-09-19T05:03:28.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.245+0000 7fb426800640 1 -- 172.21.15.43:0/3195327855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4201019c0 msgr2=0x7fb420101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:28.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.245+0000 7fb426800640 1 --2- 172.21.15.43:0/3195327855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4201019c0 0x7fb420101e20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009930 tx=0x7fb40c02a420 comp rx=0 tx=0).stop 2024-09-19T05:03:28.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.245+0000 7fb426800640 1 -- 172.21.15.43:0/3195327855 shutdown_connections 2024-09-19T05:03:28.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.245+0000 7fb426800640 1 --2- 172.21.15.43:0/3195327855 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb420102360 0x7fb42010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:28.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.245+0000 7fb426800640 1 --2- 172.21.15.43:0/3195327855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4201019c0 0x7fb420101e20 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:28.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.245+0000 7fb426800640 1 --2- 172.21.15.43:0/3195327855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4201079c0 0x7fb420107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:28.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.245+0000 7fb426800640 1 -- 172.21.15.43:0/3195327855 >> 172.21.15.43:0/3195327855 conn(0x7fb4200fd740 msgr2=0x7fb4200ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:28.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.245+0000 7fb426800640 1 -- 172.21.15.43:0/3195327855 shutdown_connections 2024-09-19T05:03:28.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.245+0000 7fb426800640 1 -- 172.21.15.43:0/3195327855 wait complete. 2024-09-19T05:03:28.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.246+0000 7fb426800640 1 Processor -- start 2024-09-19T05:03:28.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.246+0000 7fb426800640 1 -- start start 2024-09-19T05:03:28.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.247+0000 7fb426800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4201019c0 0x7fb42019a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:28.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.247+0000 7fb426800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb420102360 0x7fb42019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:28.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.247+0000 7fb426800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4201079c0 0x7fb42019ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:28.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.247+0000 7fb426800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb420111e10 con 0x7fb4201019c0 2024-09-19T05:03:28.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.247+0000 7fb426800640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb420111c90 con 0x7fb4201079c0 2024-09-19T05:03:28.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.247+0000 7fb426800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb420111f90 con 0x7fb420102360 2024-09-19T05:03:28.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.247+0000 7fb425400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4201019c0 0x7fb42019a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:28.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.247+0000 7fb425400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4201019c0 0x7fb42019a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58778/0 (socket says 172.21.15.43:58778) 2024-09-19T05:03:28.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.247+0000 7fb425400640 1 -- 172.21.15.43:0/277505245 learned_addr learned my addr 172.21.15.43:0/277505245 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:28.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.247+0000 7fb424a00640 1 --2- 172.21.15.43:0/277505245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb420102360 0x7fb42019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:28.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.247+0000 7fb425e00640 1 --2- 172.21.15.43:0/277505245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4201079c0 0x7fb42019ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:28.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.248+0000 7fb425400640 1 -- 172.21.15.43:0/277505245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb420102360 msgr2=0x7fb42019aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:28.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.248+0000 7fb425400640 1 --2- 172.21.15.43:0/277505245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb420102360 0x7fb42019aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:28.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.248+0000 7fb425400640 1 -- 172.21.15.43:0/277505245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4201079c0 msgr2=0x7fb42019ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:28.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.248+0000 7fb425400640 1 --2- 172.21.15.43:0/277505245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4201079c0 0x7fb42019ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:28.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.248+0000 7fb425400640 1 -- 172.21.15.43:0/277505245 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb42019f4e0 con 0x7fb4201019c0 2024-09-19T05:03:28.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.248+0000 7fb425400640 1 --2- 172.21.15.43:0/277505245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4201019c0 0x7fb42019a530 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb41400e9f0 tx=0x7fb41400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:28.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.248+0000 7fb41e000640 1 -- 172.21.15.43:0/277505245 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41400cd80 con 0x7fb4201019c0 2024-09-19T05:03:28.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.248+0000 7fb426800640 1 -- 172.21.15.43:0/277505245 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb42019f7d0 con 0x7fb4201019c0 2024-09-19T05:03:28.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.248+0000 7fb426800640 1 -- 172.21.15.43:0/277505245 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4201a70b0 con 0x7fb4201019c0 2024-09-19T05:03:28.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.248+0000 7fb41e000640 1 -- 172.21.15.43:0/277505245 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb414005500 con 0x7fb4201019c0 2024-09-19T05:03:28.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.248+0000 7fb41e000640 1 -- 172.21.15.43:0/277505245 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb414010690 con 0x7fb4201019c0 2024-09-19T05:03:28.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.250+0000 7fb426800640 1 -- 172.21.15.43:0/277505245 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb420101e20 con 0x7fb4201019c0 2024-09-19T05:03:28.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.252+0000 7fb41e000640 1 -- 172.21.15.43:0/277505245 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fb4140046c0 con 0x7fb4201019c0 2024-09-19T05:03:28.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.253+0000 7fb41e000640 1 --2- 172.21.15.43:0/277505245 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fb4080a3350 0x7fb4080a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:28.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.253+0000 7fb41e000640 1 -- 172.21.15.43:0/277505245 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fb4140cfe30 con 0x7fb4201019c0 2024-09-19T05:03:28.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.253+0000 7fb424a00640 1 --2- 172.21.15.43:0/277505245 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fb4080a3350 0x7fb4080a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:28.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.254+0000 7fb424a00640 1 --2- 172.21.15.43:0/277505245 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fb4080a3350 0x7fb4080a5810 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb40c006010 tx=0x7fb40c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:28.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.258+0000 7fb41e000640 1 -- 172.21.15.43:0/277505245 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb41409c9b0 con 0x7fb4201019c0 2024-09-19T05:03:28.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.456+0000 7fb426800640 1 -- 172.21.15.43:0/277505245 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fb4201a72e0 con 0x7fb4201019c0 2024-09-19T05:03:28.459 INFO:teuthology.orchestra.run.smithi043.stdout:34359738453 2024-09-19T05:03:28.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.457+0000 7fb41e000640 1 -- 172.21.15.43:0/277505245 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb414096e20 con 0x7fb4201019c0 2024-09-19T05:03:28.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.459+0000 7fb426800640 1 -- 172.21.15.43:0/277505245 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fb4080a3350 msgr2=0x7fb4080a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:28.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.459+0000 7fb426800640 1 --2- 172.21.15.43:0/277505245 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fb4080a3350 0x7fb4080a5810 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb40c006010 tx=0x7fb40c000bc0 comp rx=0 tx=0).stop 2024-09-19T05:03:28.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.459+0000 7fb426800640 1 -- 172.21.15.43:0/277505245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4201019c0 msgr2=0x7fb42019a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:28.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.459+0000 7fb426800640 1 --2- 172.21.15.43:0/277505245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4201019c0 0x7fb42019a530 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb41400e9f0 tx=0x7fb41400edf0 comp rx=0 tx=0).stop 2024-09-19T05:03:28.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.459+0000 7fb426800640 1 -- 172.21.15.43:0/277505245 shutdown_connections 2024-09-19T05:03:28.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.459+0000 7fb426800640 1 --2- 172.21.15.43:0/277505245 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fb4080a3350 0x7fb4080a5810 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:28.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.459+0000 7fb426800640 1 --2- 172.21.15.43:0/277505245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4201079c0 0x7fb42019ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:28.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.459+0000 7fb426800640 1 --2- 172.21.15.43:0/277505245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb420102360 0x7fb42019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:28.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.459+0000 7fb426800640 1 --2- 172.21.15.43:0/277505245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4201019c0 0x7fb42019a530 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:28.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.459+0000 7fb426800640 1 -- 172.21.15.43:0/277505245 >> 172.21.15.43:0/277505245 conn(0x7fb4200fd740 msgr2=0x7fb4200fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:28.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.460+0000 7fb426800640 1 -- 172.21.15.43:0/277505245 shutdown_connections 2024-09-19T05:03:28.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.460+0000 7fb426800640 1 -- 172.21.15.43:0/277505245 wait complete. 2024-09-19T05:03:28.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:28 smithi043 ceph-mon[30800]: pgmap v325: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T05:03:28.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:28 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/3640195681' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T05:03:28.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:28 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/277505245' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T05:03:29.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.975+0000 7fb3a6600640 1 -- 172.21.15.43:0/2598605217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3a010e2a0 msgr2=0x7fb3a0110690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:29.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.975+0000 7fb3a6600640 1 --2- 172.21.15.43:0/2598605217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3a010e2a0 0x7fb3a0110690 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb398009be0 tx=0x7fb39802a6f0 comp rx=0 tx=0).stop 2024-09-19T05:03:29.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.975+0000 7fb3a6600640 1 -- 172.21.15.43:0/2598605217 shutdown_connections 2024-09-19T05:03:29.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.975+0000 7fb3a6600640 1 --2- 172.21.15.43:0/2598605217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3a010e2a0 0x7fb3a0110690 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:29.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.975+0000 7fb3a6600640 1 --2- 172.21.15.43:0/2598605217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3a01010a0 0x7fb3a010dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:29.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.975+0000 7fb3a6600640 1 --2- 172.21.15.43:0/2598605217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3a01006f0 0x7fb3a0100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:29.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.975+0000 7fb3a6600640 1 -- 172.21.15.43:0/2598605217 >> 172.21.15.43:0/2598605217 conn(0x7fb3a00fc5c0 msgr2=0x7fb3a00fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:29.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.975+0000 7fb3a6600640 1 -- 172.21.15.43:0/2598605217 shutdown_connections 2024-09-19T05:03:29.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.975+0000 7fb3a6600640 1 -- 172.21.15.43:0/2598605217 wait complete. 2024-09-19T05:03:29.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.976+0000 7fb3a6600640 1 Processor -- start 2024-09-19T05:03:29.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.976+0000 7fb3a6600640 1 -- start start 2024-09-19T05:03:29.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.976+0000 7fb3a6600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3a01006f0 0x7fb3a019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:29.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.976+0000 7fb3a6600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3a01010a0 0x7fb3a019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:29.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.976+0000 7fb3a6600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3a010e2a0 0x7fb3a019edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:29.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.976+0000 7fb3a6600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a0114000 con 0x7fb3a01006f0 2024-09-19T05:03:29.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.976+0000 7fb3a6600640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a0113e80 con 0x7fb3a010e2a0 2024-09-19T05:03:29.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.976+0000 7fb3a6600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a0114180 con 0x7fb3a01010a0 2024-09-19T05:03:29.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.977+0000 7fb3a5200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3a01006f0 0x7fb3a019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:29.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.977+0000 7fb3a5200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3a01006f0 0x7fb3a019a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58792/0 (socket says 172.21.15.43:58792) 2024-09-19T05:03:29.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.977+0000 7fb3a5200640 1 -- 172.21.15.43:0/1558001333 learned_addr learned my addr 172.21.15.43:0/1558001333 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:29.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.977+0000 7fb39fe00640 1 --2- 172.21.15.43:0/1558001333 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3a01010a0 0x7fb3a019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:29.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.977+0000 7fb3a5c00640 1 --2- 172.21.15.43:0/1558001333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3a010e2a0 0x7fb3a019edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:29.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.977+0000 7fb39fe00640 1 -- 172.21.15.43:0/1558001333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3a010e2a0 msgr2=0x7fb3a019edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:29.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.977+0000 7fb39fe00640 1 --2- 172.21.15.43:0/1558001333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3a010e2a0 0x7fb3a019edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:29.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.977+0000 7fb39fe00640 1 -- 172.21.15.43:0/1558001333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3a01006f0 msgr2=0x7fb3a019a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:29.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.977+0000 7fb39fe00640 1 --2- 172.21.15.43:0/1558001333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3a01006f0 0x7fb3a019a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:29.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.977+0000 7fb39fe00640 1 -- 172.21.15.43:0/1558001333 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3a019f490 con 0x7fb3a01010a0 2024-09-19T05:03:29.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.977+0000 7fb3a5c00640 1 --2- 172.21.15.43:0/1558001333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3a010e2a0 0x7fb3a019edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:03:29.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.978+0000 7fb39fe00640 1 --2- 172.21.15.43:0/1558001333 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3a01010a0 0x7fb3a019aa20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb38c005fd0 tx=0x7fb38c00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:29.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.978+0000 7fb39d600640 1 -- 172.21.15.43:0/1558001333 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38c00cd00 con 0x7fb3a01010a0 2024-09-19T05:03:29.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.978+0000 7fb39d600640 1 -- 172.21.15.43:0/1558001333 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb38c0035f0 con 0x7fb3a01010a0 2024-09-19T05:03:29.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.978+0000 7fb39d600640 1 -- 172.21.15.43:0/1558001333 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38c010690 con 0x7fb3a01010a0 2024-09-19T05:03:29.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.978+0000 7fb3a6600640 1 -- 172.21.15.43:0/1558001333 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3a019f780 con 0x7fb3a01010a0 2024-09-19T05:03:29.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.978+0000 7fb3a6600640 1 -- 172.21.15.43:0/1558001333 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3a01a7060 con 0x7fb3a01010a0 2024-09-19T05:03:29.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.979+0000 7fb3a6600640 1 -- 172.21.15.43:0/1558001333 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3a0100ad0 con 0x7fb3a01010a0 2024-09-19T05:03:29.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.987+0000 7fb39d600640 1 -- 172.21.15.43:0/1558001333 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fb38c0037a0 con 0x7fb3a01010a0 2024-09-19T05:03:29.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.990+0000 7fb39d600640 1 --2- 172.21.15.43:0/1558001333 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fb3880a32d0 0x7fb3880a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:29.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.990+0000 7fb39d600640 1 -- 172.21.15.43:0/1558001333 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fb38c0d0050 con 0x7fb3a01010a0 2024-09-19T05:03:29.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.990+0000 7fb39d600640 1 -- 172.21.15.43:0/1558001333 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb38c0d0550 con 0x7fb3a01010a0 2024-09-19T05:03:29.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.991+0000 7fb3a5200640 1 --2- 172.21.15.43:0/1558001333 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fb3880a32d0 0x7fb3880a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:29.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.992+0000 7fb3a5200640 1 --2- 172.21.15.43:0/1558001333 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fb3880a32d0 0x7fb3880a5790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb394009a30 tx=0x7fb394009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:29.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:28 smithi151 ceph-mon[35425]: pgmap v325: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T05:03:29.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:28 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/3640195681' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T05:03:29.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:28 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/277505245' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T05:03:29.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:29.187+0000 7fb3a6600640 1 -- 172.21.15.43:0/1558001333 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fb3a019b840 con 0x7fb3a01010a0 2024-09-19T05:03:29.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:29.187+0000 7fb39d600640 1 -- 172.21.15.43:0/1558001333 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb3a019b840 con 0x7fb3a01010a0 2024-09-19T05:03:29.189 INFO:teuthology.orchestra.run.smithi043.stdout:133143986239 2024-09-19T05:03:29.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:29.190+0000 7fb3a6600640 1 -- 172.21.15.43:0/1558001333 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fb3880a32d0 msgr2=0x7fb3880a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:29.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:29.190+0000 7fb3a6600640 1 --2- 172.21.15.43:0/1558001333 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fb3880a32d0 0x7fb3880a5790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb394009a30 tx=0x7fb394009190 comp rx=0 tx=0).stop 2024-09-19T05:03:29.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:29.190+0000 7fb3a6600640 1 -- 172.21.15.43:0/1558001333 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3a01010a0 msgr2=0x7fb3a019aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:29.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:29.190+0000 7fb3a6600640 1 --2- 172.21.15.43:0/1558001333 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3a01010a0 0x7fb3a019aa20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb38c005fd0 tx=0x7fb38c00ee50 comp rx=0 tx=0).stop 2024-09-19T05:03:29.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:29.190+0000 7fb3a6600640 1 -- 172.21.15.43:0/1558001333 shutdown_connections 2024-09-19T05:03:29.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:29.190+0000 7fb3a6600640 1 --2- 172.21.15.43:0/1558001333 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fb3880a32d0 0x7fb3880a5790 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:29.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:29.190+0000 7fb3a6600640 1 --2- 172.21.15.43:0/1558001333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3a010e2a0 0x7fb3a019edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:29.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:29.190+0000 7fb3a6600640 1 --2- 172.21.15.43:0/1558001333 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3a01010a0 0x7fb3a019aa20 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:29.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:29.190+0000 7fb3a6600640 1 --2- 172.21.15.43:0/1558001333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3a01006f0 0x7fb3a019a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:29.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:29.190+0000 7fb3a6600640 1 -- 172.21.15.43:0/1558001333 >> 172.21.15.43:0/1558001333 conn(0x7fb3a00fc5c0 msgr2=0x7fb3a00fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:29.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:29.190+0000 7fb3a6600640 1 -- 172.21.15.43:0/1558001333 shutdown_connections 2024-09-19T05:03:29.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:29.190+0000 7fb3a6600640 1 -- 172.21.15.43:0/1558001333 wait complete. 2024-09-19T05:03:29.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:28 smithi096 ceph-mon[35085]: pgmap v325: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T05:03:29.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:28 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/3640195681' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T05:03:29.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:28 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/277505245' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T05:03:29.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.120+0000 7f5921800640 1 -- 172.21.15.43:0/2526545152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f591c077040 msgr2=0x7f591c112510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:29.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.120+0000 7f5921800640 1 --2- 172.21.15.43:0/2526545152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f591c077040 0x7f591c112510 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5904009900 tx=0x7f590402a410 comp rx=0 tx=0).stop 2024-09-19T05:03:29.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.120+0000 7f5921800640 1 -- 172.21.15.43:0/2526545152 shutdown_connections 2024-09-19T05:03:29.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.120+0000 7f5921800640 1 --2- 172.21.15.43:0/2526545152 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f591c112a50 0x7f591c114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:29.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.120+0000 7f5921800640 1 --2- 172.21.15.43:0/2526545152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f591c077040 0x7f591c112510 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:29.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.120+0000 7f5921800640 1 --2- 172.21.15.43:0/2526545152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f591c076720 0x7f591c076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:29.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.120+0000 7f5921800640 1 -- 172.21.15.43:0/2526545152 >> 172.21.15.43:0/2526545152 conn(0x7f591c0ffa50 msgr2=0x7f591c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:29.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.120+0000 7f5921800640 1 -- 172.21.15.43:0/2526545152 shutdown_connections 2024-09-19T05:03:29.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.120+0000 7f5921800640 1 -- 172.21.15.43:0/2526545152 wait complete. 2024-09-19T05:03:29.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.121+0000 7f5921800640 1 Processor -- start 2024-09-19T05:03:29.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.121+0000 7f5921800640 1 -- start start 2024-09-19T05:03:29.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.122+0000 7f5921800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f591c076720 0x7f591c1a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:29.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.122+0000 7f5921800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f591c077040 0x7f591c1a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:29.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.122+0000 7f5921800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f591c112a50 0x7f591c19e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:29.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.122+0000 7f5921800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f591c1185d0 con 0x7f591c076720 2024-09-19T05:03:29.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.122+0000 7f5921800640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f591c118450 con 0x7f591c112a50 2024-09-19T05:03:29.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.122+0000 7f5921800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f591c118750 con 0x7f591c077040 2024-09-19T05:03:29.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.122+0000 7f591aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f591c076720 0x7f591c1a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:29.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.122+0000 7f591b400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f591c112a50 0x7f591c19e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:29.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.122+0000 7f591aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f591c076720 0x7f591c1a46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58760/0 (socket says 172.21.15.43:58760) 2024-09-19T05:03:29.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.122+0000 7f591aa00640 1 -- 172.21.15.43:0/3640195681 learned_addr learned my addr 172.21.15.43:0/3640195681 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:29.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.122+0000 7f5913e00640 1 --2- 172.21.15.43:0/3640195681 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f591c077040 0x7f591c1a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:29.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.122+0000 7f591aa00640 1 -- 172.21.15.43:0/3640195681 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f591c077040 msgr2=0x7f591c1a4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:29.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.122+0000 7f591aa00640 1 --2- 172.21.15.43:0/3640195681 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f591c077040 0x7f591c1a4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:29.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.122+0000 7f591aa00640 1 -- 172.21.15.43:0/3640195681 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f591c112a50 msgr2=0x7f591c19e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:29.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.122+0000 7f591aa00640 1 --2- 172.21.15.43:0/3640195681 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f591c112a50 0x7f591c19e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:29.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.122+0000 7f591aa00640 1 -- 172.21.15.43:0/3640195681 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f591c19efe0 con 0x7f591c076720 2024-09-19T05:03:29.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.122+0000 7f5913e00640 1 --2- 172.21.15.43:0/3640195681 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f591c077040 0x7f591c1a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:03:29.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.123+0000 7f591aa00640 1 --2- 172.21.15.43:0/3640195681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f591c076720 0x7f591c1a46f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f590c00d950 tx=0x7f590c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:29.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.123+0000 7f5913400640 1 -- 172.21.15.43:0/3640195681 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590c017070 con 0x7f591c076720 2024-09-19T05:03:29.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.123+0000 7f5913400640 1 -- 172.21.15.43:0/3640195681 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f590c009c30 con 0x7f591c076720 2024-09-19T05:03:29.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.123+0000 7f5913400640 1 -- 172.21.15.43:0/3640195681 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590c00f6b0 con 0x7f591c076720 2024-09-19T05:03:29.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.123+0000 7f5921800640 1 -- 172.21.15.43:0/3640195681 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f591c19f2d0 con 0x7f591c076720 2024-09-19T05:03:29.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.123+0000 7f5921800640 1 -- 172.21.15.43:0/3640195681 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f591c1ab5c0 con 0x7f591c076720 2024-09-19T05:03:29.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.124+0000 7f5921800640 1 -- 172.21.15.43:0/3640195681 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f591c077e60 con 0x7f591c076720 2024-09-19T05:03:29.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.126+0000 7f5913400640 1 -- 172.21.15.43:0/3640195681 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f590c0109d0 con 0x7f591c076720 2024-09-19T05:03:29.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.128+0000 7f5913400640 1 --2- 172.21.15.43:0/3640195681 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f59000a3350 0x7f59000a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:29.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.128+0000 7f5913400640 1 -- 172.21.15.43:0/3640195681 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f590c0cf6d0 con 0x7f591c076720 2024-09-19T05:03:29.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.128+0000 7f5913e00640 1 --2- 172.21.15.43:0/3640195681 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f59000a3350 0x7f59000a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:29.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.129+0000 7f5913e00640 1 --2- 172.21.15.43:0/3640195681 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f59000a3350 0x7f59000a5810 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f591c1a00a0 tx=0x7f5904005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:29.449 INFO:teuthology.orchestra.run.smithi043.stdout:55834574929 2024-09-19T05:03:29.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.132+0000 7f5913400640 1 -- 172.21.15.43:0/3640195681 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f590c009de0 con 0x7f591c076720 2024-09-19T05:03:29.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.325+0000 7f5921800640 1 -- 172.21.15.43:0/3640195681 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f591c1ab7f0 con 0x7f591c076720 2024-09-19T05:03:29.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.325+0000 7f5913400640 1 -- 172.21.15.43:0/3640195681 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f590c09c250 con 0x7f591c076720 2024-09-19T05:03:29.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.328+0000 7f5921800640 1 -- 172.21.15.43:0/3640195681 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f59000a3350 msgr2=0x7f59000a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:29.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.328+0000 7f5921800640 1 --2- 172.21.15.43:0/3640195681 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f59000a3350 0x7f59000a5810 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f591c1a00a0 tx=0x7f5904005a20 comp rx=0 tx=0).stop 2024-09-19T05:03:29.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.328+0000 7f5921800640 1 -- 172.21.15.43:0/3640195681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f591c076720 msgr2=0x7f591c1a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:29.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.328+0000 7f5921800640 1 --2- 172.21.15.43:0/3640195681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f591c076720 0x7f591c1a46f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f590c00d950 tx=0x7f590c00de10 comp rx=0 tx=0).stop 2024-09-19T05:03:29.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.328+0000 7f5921800640 1 -- 172.21.15.43:0/3640195681 shutdown_connections 2024-09-19T05:03:29.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.328+0000 7f5921800640 1 --2- 172.21.15.43:0/3640195681 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f59000a3350 0x7f59000a5810 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:29.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.328+0000 7f5921800640 1 --2- 172.21.15.43:0/3640195681 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f591c112a50 0x7f591c19e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:29.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.328+0000 7f5921800640 1 --2- 172.21.15.43:0/3640195681 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f591c077040 0x7f591c1a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:29.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.328+0000 7f5921800640 1 --2- 172.21.15.43:0/3640195681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f591c076720 0x7f591c1a46f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:29.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.328+0000 7f5921800640 1 -- 172.21.15.43:0/3640195681 >> 172.21.15.43:0/3640195681 conn(0x7f591c0ffa50 msgr2=0x7f591c101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:29.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.328+0000 7f5921800640 1 -- 172.21.15.43:0/3640195681 shutdown_connections 2024-09-19T05:03:29.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:28.328+0000 7f5921800640 1 -- 172.21.15.43:0/3640195681 wait complete. 2024-09-19T05:03:29.809 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:29.811 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:29.832 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738452 got 34359738453 for osd.0 2024-09-19T05:03:29.832 DEBUG:teuthology.parallel:result is None 2024-09-19T05:03:30.017 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:29 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/1558001333' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T05:03:30.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:29 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/1558001333' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T05:03:30.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:29 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/1558001333' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T05:03:31.011 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:30 smithi043 ceph-mon[30800]: pgmap v326: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:03:31.097 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:31.097 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:31.131 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561074 got 188978561076 for osd.6 2024-09-19T05:03:31.131 DEBUG:teuthology.parallel:result is None 2024-09-19T05:03:31.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:30 smithi151 ceph-mon[35425]: pgmap v326: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:03:31.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:30 smithi096 ceph-mon[35085]: pgmap v326: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:03:31.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.316+0000 7fedaf200640 1 -- 172.21.15.43:0/1600275166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feda81079c0 msgr2=0x7feda8107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:31.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.316+0000 7fedaf200640 1 --2- 172.21.15.43:0/1600275166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feda81079c0 0x7feda8107da0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fed9c009930 tx=0x7fed9c02a420 comp rx=0 tx=0).stop 2024-09-19T05:03:31.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.316+0000 7fedaf200640 1 -- 172.21.15.43:0/1600275166 shutdown_connections 2024-09-19T05:03:31.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.316+0000 7fedaf200640 1 --2- 172.21.15.43:0/1600275166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feda8102360 0x7feda810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:31.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.316+0000 7fedaf200640 1 --2- 172.21.15.43:0/1600275166 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feda81019c0 0x7feda8101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:31.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.316+0000 7fedaf200640 1 --2- 172.21.15.43:0/1600275166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feda81079c0 0x7feda8107da0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:31.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.316+0000 7fedaf200640 1 -- 172.21.15.43:0/1600275166 >> 172.21.15.43:0/1600275166 conn(0x7feda80fd740 msgr2=0x7feda80ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:31.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.316+0000 7fedaf200640 1 -- 172.21.15.43:0/1600275166 shutdown_connections 2024-09-19T05:03:31.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.316+0000 7fedaf200640 1 -- 172.21.15.43:0/1600275166 wait complete. 2024-09-19T05:03:31.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.317+0000 7fedaf200640 1 Processor -- start 2024-09-19T05:03:31.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.317+0000 7fedaf200640 1 -- start start 2024-09-19T05:03:31.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.318+0000 7fedaf200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feda81019c0 0x7feda80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:31.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.318+0000 7fedaf200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feda8102360 0x7feda8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:31.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.318+0000 7fedaf200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feda81079c0 0x7feda8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:31.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.318+0000 7fedaf200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7feda8111dc0 con 0x7feda81019c0 2024-09-19T05:03:31.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.318+0000 7fedaf200640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7feda8111c40 con 0x7feda81079c0 2024-09-19T05:03:31.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.318+0000 7fedaf200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7feda8111f40 con 0x7feda8102360 2024-09-19T05:03:31.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.318+0000 7fedade00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feda81019c0 0x7feda80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:31.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.318+0000 7fedad400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feda8102360 0x7feda8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:31.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.318+0000 7fedade00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feda81019c0 0x7feda80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58812/0 (socket says 172.21.15.43:58812) 2024-09-19T05:03:31.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.318+0000 7fedade00640 1 -- 172.21.15.43:0/3924882371 learned_addr learned my addr 172.21.15.43:0/3924882371 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:31.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.318+0000 7fedae800640 1 --2- 172.21.15.43:0/3924882371 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feda81079c0 0x7feda8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:31.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.318+0000 7fedad400640 1 -- 172.21.15.43:0/3924882371 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feda81079c0 msgr2=0x7feda8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:31.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.318+0000 7fedad400640 1 --2- 172.21.15.43:0/3924882371 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feda81079c0 0x7feda8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:31.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.318+0000 7fedad400640 1 -- 172.21.15.43:0/3924882371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feda81019c0 msgr2=0x7feda80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:31.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.319+0000 7fedad400640 1 --2- 172.21.15.43:0/3924882371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feda81019c0 0x7feda80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:31.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.319+0000 7fedad400640 1 -- 172.21.15.43:0/3924882371 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda81a6800 con 0x7feda8102360 2024-09-19T05:03:31.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.319+0000 7fedad400640 1 --2- 172.21.15.43:0/3924882371 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feda8102360 0x7feda8074a30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fed9800d720 tx=0x7fed9800dbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:31.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.319+0000 7feda2a00640 1 -- 172.21.15.43:0/3924882371 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed98017070 con 0x7feda8102360 2024-09-19T05:03:31.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.319+0000 7fedaf200640 1 -- 172.21.15.43:0/3924882371 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feda81a6af0 con 0x7feda8102360 2024-09-19T05:03:31.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.319+0000 7feda2a00640 1 -- 172.21.15.43:0/3924882371 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed9800de30 con 0x7feda8102360 2024-09-19T05:03:31.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.319+0000 7feda2a00640 1 -- 172.21.15.43:0/3924882371 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed980047a0 con 0x7feda8102360 2024-09-19T05:03:31.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.319+0000 7fedaf200640 1 -- 172.21.15.43:0/3924882371 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feda81a7030 con 0x7feda8102360 2024-09-19T05:03:31.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.320+0000 7fedaf200640 1 -- 172.21.15.43:0/3924882371 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feda8101e20 con 0x7feda8102360 2024-09-19T05:03:31.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.322+0000 7feda2a00640 1 -- 172.21.15.43:0/3924882371 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fed980099d0 con 0x7feda8102360 2024-09-19T05:03:31.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.324+0000 7feda2a00640 1 --2- 172.21.15.43:0/3924882371 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fed900a3350 0x7fed900a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:31.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.324+0000 7feda2a00640 1 -- 172.21.15.43:0/3924882371 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fed980cfa30 con 0x7feda8102360 2024-09-19T05:03:31.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.324+0000 7fedade00640 1 --2- 172.21.15.43:0/3924882371 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fed900a3350 0x7fed900a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:31.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.324+0000 7fedade00640 1 --2- 172.21.15.43:0/3924882371 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fed900a3350 0x7fed900a5810 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fed9c000c00 tx=0x7fed9c009560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:31.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.326+0000 7feda2a00640 1 -- 172.21.15.43:0/3924882371 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fed98020020 con 0x7feda8102360 2024-09-19T05:03:31.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.512+0000 7fedaf200640 1 -- 172.21.15.43:0/3924882371 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7feda8109f30 con 0x7feda8102360 2024-09-19T05:03:31.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.513+0000 7feda2a00640 1 -- 172.21.15.43:0/3924882371 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fed9809c630 con 0x7feda8102360 2024-09-19T05:03:31.514 INFO:teuthology.orchestra.run.smithi043.stdout:244813135915 2024-09-19T05:03:31.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.515+0000 7fedaf200640 1 -- 172.21.15.43:0/3924882371 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fed900a3350 msgr2=0x7fed900a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:31.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.515+0000 7fedaf200640 1 --2- 172.21.15.43:0/3924882371 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fed900a3350 0x7fed900a5810 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fed9c000c00 tx=0x7fed9c009560 comp rx=0 tx=0).stop 2024-09-19T05:03:31.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.515+0000 7fedaf200640 1 -- 172.21.15.43:0/3924882371 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feda8102360 msgr2=0x7feda8074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:31.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.515+0000 7fedaf200640 1 --2- 172.21.15.43:0/3924882371 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feda8102360 0x7feda8074a30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fed9800d720 tx=0x7fed9800dbe0 comp rx=0 tx=0).stop 2024-09-19T05:03:31.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.515+0000 7fedaf200640 1 -- 172.21.15.43:0/3924882371 shutdown_connections 2024-09-19T05:03:31.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.515+0000 7fedaf200640 1 --2- 172.21.15.43:0/3924882371 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fed900a3350 0x7fed900a5810 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:31.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.515+0000 7fedaf200640 1 --2- 172.21.15.43:0/3924882371 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feda81079c0 0x7feda8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:31.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.515+0000 7fedaf200640 1 --2- 172.21.15.43:0/3924882371 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feda8102360 0x7feda8074a30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:31.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.515+0000 7fedaf200640 1 --2- 172.21.15.43:0/3924882371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feda81019c0 0x7feda80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:31.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.515+0000 7fedaf200640 1 -- 172.21.15.43:0/3924882371 >> 172.21.15.43:0/3924882371 conn(0x7feda80fd740 msgr2=0x7feda80fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:31.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.515+0000 7fedaf200640 1 -- 172.21.15.43:0/3924882371 shutdown_connections 2024-09-19T05:03:31.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.515+0000 7fedaf200640 1 -- 172.21.15.43:0/3924882371 wait complete. 2024-09-19T05:03:31.705 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574926 got 55834574929 for osd.1 2024-09-19T05:03:31.705 DEBUG:teuthology.parallel:result is None 2024-09-19T05:03:31.959 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:31 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/3924882371' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T05:03:31.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.979+0000 7fb51fc00640 1 -- 172.21.15.43:0/1814720003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb518102790 msgr2=0x7fb5181067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:31.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.979+0000 7fb51fc00640 1 --2- 172.21.15.43:0/1814720003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb518102790 0x7fb5181067e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb504009900 tx=0x7fb50402a410 comp rx=0 tx=0).stop 2024-09-19T05:03:31.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.980+0000 7fb51fc00640 1 -- 172.21.15.43:0/1814720003 shutdown_connections 2024-09-19T05:03:31.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.980+0000 7fb51fc00640 1 --2- 172.21.15.43:0/1814720003 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb518106f10 0x7fb51810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:31.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.980+0000 7fb51fc00640 1 --2- 172.21.15.43:0/1814720003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb518102790 0x7fb5181067e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:31.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.980+0000 7fb51fc00640 1 --2- 172.21.15.43:0/1814720003 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb518101de0 0x7fb5181021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:31.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.980+0000 7fb51fc00640 1 -- 172.21.15.43:0/1814720003 >> 172.21.15.43:0/1814720003 conn(0x7fb5180fd8c0 msgr2=0x7fb5180ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:31.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.980+0000 7fb51fc00640 1 -- 172.21.15.43:0/1814720003 shutdown_connections 2024-09-19T05:03:31.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.980+0000 7fb51fc00640 1 -- 172.21.15.43:0/1814720003 wait complete. 2024-09-19T05:03:31.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.981+0000 7fb51fc00640 1 Processor -- start 2024-09-19T05:03:31.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.981+0000 7fb51fc00640 1 -- start start 2024-09-19T05:03:31.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.981+0000 7fb51fc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb518101de0 0x7fb51819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:31.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.981+0000 7fb51fc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb518102790 0x7fb51819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:31.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.981+0000 7fb51fc00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb518106f10 0x7fb51819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:31.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.981+0000 7fb51fc00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb518111fc0 con 0x7fb518102790 2024-09-19T05:03:31.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.981+0000 7fb51fc00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb518111e40 con 0x7fb518106f10 2024-09-19T05:03:31.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.981+0000 7fb51fc00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb518112140 con 0x7fb518101de0 2024-09-19T05:03:31.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.981+0000 7fb517e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb518102790 0x7fb51819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:31.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.981+0000 7fb517e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb518102790 0x7fb51819ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58826/0 (socket says 172.21.15.43:58826) 2024-09-19T05:03:31.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.981+0000 7fb517e00640 1 -- 172.21.15.43:0/2205368498 learned_addr learned my addr 172.21.15.43:0/2205368498 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:31.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.981+0000 7fb51d200640 1 --2- 172.21.15.43:0/2205368498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb518101de0 0x7fb51819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:31.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.982+0000 7fb51dc00640 1 --2- 172.21.15.43:0/2205368498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb518106f10 0x7fb51819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:31.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.982+0000 7fb517e00640 1 -- 172.21.15.43:0/2205368498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb518101de0 msgr2=0x7fb51819a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:31.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.982+0000 7fb517e00640 1 --2- 172.21.15.43:0/2205368498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb518101de0 0x7fb51819a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:31.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.982+0000 7fb517e00640 1 -- 172.21.15.43:0/2205368498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb518106f10 msgr2=0x7fb51819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:31.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.982+0000 7fb517e00640 1 --2- 172.21.15.43:0/2205368498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb518106f10 0x7fb51819ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:31.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.982+0000 7fb517e00640 1 -- 172.21.15.43:0/2205368498 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb51819f5f0 con 0x7fb518102790 2024-09-19T05:03:31.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.982+0000 7fb517e00640 1 --2- 172.21.15.43:0/2205368498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb518102790 0x7fb51819ab80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb504004770 tx=0x7fb504003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:31.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.982+0000 7fb515600640 1 -- 172.21.15.43:0/2205368498 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50403f070 con 0x7fb518102790 2024-09-19T05:03:31.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.983+0000 7fb515600640 1 -- 172.21.15.43:0/2205368498 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb504031ca0 con 0x7fb518102790 2024-09-19T05:03:31.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.983+0000 7fb51fc00640 1 -- 172.21.15.43:0/2205368498 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb51819f880 con 0x7fb518102790 2024-09-19T05:03:31.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.983+0000 7fb515600640 1 -- 172.21.15.43:0/2205368498 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb504035030 con 0x7fb518102790 2024-09-19T05:03:31.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.983+0000 7fb51fc00640 1 -- 172.21.15.43:0/2205368498 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5181a71c0 con 0x7fb518102790 2024-09-19T05:03:31.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.984+0000 7fb51fc00640 1 -- 172.21.15.43:0/2205368498 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5181021c0 con 0x7fb518102790 2024-09-19T05:03:31.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.986+0000 7fb515600640 1 -- 172.21.15.43:0/2205368498 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fb504042070 con 0x7fb518102790 2024-09-19T05:03:31.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.987+0000 7fb515600640 1 --2- 172.21.15.43:0/2205368498 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fb5000a3350 0x7fb5000a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:31.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.987+0000 7fb515600640 1 -- 172.21.15.43:0/2205368498 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fb5040f0070 con 0x7fb518102790 2024-09-19T05:03:31.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.987+0000 7fb51d200640 1 --2- 172.21.15.43:0/2205368498 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fb5000a3350 0x7fb5000a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:31.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.988+0000 7fb51d200640 1 --2- 172.21.15.43:0/2205368498 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fb5000a3350 0x7fb5000a5810 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00bbe0 tx=0x7fb50c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:31.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:31.991+0000 7fb515600640 1 -- 172.21.15.43:0/2205368498 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb5040bc5e0 con 0x7fb518102790 2024-09-19T05:03:32.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:31 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/3924882371' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T05:03:32.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:32.177+0000 7fb51fc00640 1 -- 172.21.15.43:0/2205368498 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fb5181a73c0 con 0x7fb518102790 2024-09-19T05:03:32.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:32.178+0000 7fb515600640 1 -- 172.21.15.43:0/2205368498 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fb5040c08e0 con 0x7fb518102790 2024-09-19T05:03:32.179 INFO:teuthology.orchestra.run.smithi043.stdout:296352743457 2024-09-19T05:03:32.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:32.180+0000 7fb51fc00640 1 -- 172.21.15.43:0/2205368498 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fb5000a3350 msgr2=0x7fb5000a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:32.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:32.180+0000 7fb51fc00640 1 --2- 172.21.15.43:0/2205368498 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fb5000a3350 0x7fb5000a5810 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00bbe0 tx=0x7fb50c00b390 comp rx=0 tx=0).stop 2024-09-19T05:03:32.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:32.180+0000 7fb51fc00640 1 -- 172.21.15.43:0/2205368498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb518102790 msgr2=0x7fb51819ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:32.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:32.180+0000 7fb51fc00640 1 --2- 172.21.15.43:0/2205368498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb518102790 0x7fb51819ab80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb504004770 tx=0x7fb504003c10 comp rx=0 tx=0).stop 2024-09-19T05:03:32.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:32.180+0000 7fb51fc00640 1 -- 172.21.15.43:0/2205368498 shutdown_connections 2024-09-19T05:03:32.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:32.180+0000 7fb51fc00640 1 --2- 172.21.15.43:0/2205368498 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fb5000a3350 0x7fb5000a5810 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:32.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:32.180+0000 7fb51fc00640 1 --2- 172.21.15.43:0/2205368498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb518106f10 0x7fb51819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:32.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:32.180+0000 7fb51fc00640 1 --2- 172.21.15.43:0/2205368498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb518102790 0x7fb51819ab80 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:32.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:32.180+0000 7fb51fc00640 1 --2- 172.21.15.43:0/2205368498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb518101de0 0x7fb51819a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:32.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:32.180+0000 7fb51fc00640 1 -- 172.21.15.43:0/2205368498 >> 172.21.15.43:0/2205368498 conn(0x7fb5180fd8c0 msgr2=0x7fb5180ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:32.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:32.180+0000 7fb51fc00640 1 -- 172.21.15.43:0/2205368498 shutdown_connections 2024-09-19T05:03:32.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:32.180+0000 7fb51fc00640 1 -- 172.21.15.43:0/2205368498 wait complete. 2024-09-19T05:03:32.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:31 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/3924882371' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T05:03:32.972 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:32 smithi043 ceph-mon[30800]: pgmap v327: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T05:03:32.972 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:32 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/2205368498' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T05:03:33.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:32 smithi151 ceph-mon[35425]: pgmap v327: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T05:03:33.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:32 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/2205368498' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T05:03:33.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:32 smithi096 ceph-mon[35085]: pgmap v327: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T05:03:33.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:32 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/2205368498' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T05:03:33.278 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743453 got 296352743457 for osd.10 2024-09-19T05:03:33.278 DEBUG:teuthology.parallel:result is None 2024-09-19T05:03:33.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.446+0000 7f8368400640 1 -- 172.21.15.43:0/4000946599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8360109120 msgr2=0x7f8360110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:33.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.446+0000 7f8368400640 1 --2- 172.21.15.43:0/4000946599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8360109120 0x7f8360110ca0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8358009ca0 tx=0x7f835802a8c0 comp rx=0 tx=0).stop 2024-09-19T05:03:33.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.446+0000 7f8368400640 1 -- 172.21.15.43:0/4000946599 shutdown_connections 2024-09-19T05:03:33.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.446+0000 7f8368400640 1 --2- 172.21.15.43:0/4000946599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8360109120 0x7f8360110ca0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:33.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.446+0000 7f8368400640 1 --2- 172.21.15.43:0/4000946599 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83601049a0 0x7f83601089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:33.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.446+0000 7f8368400640 1 --2- 172.21.15.43:0/4000946599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8360103ff0 0x7f83601043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:33.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.446+0000 7f8368400640 1 -- 172.21.15.43:0/4000946599 >> 172.21.15.43:0/4000946599 conn(0x7f83600ffa90 msgr2=0x7f8360101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:33.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.446+0000 7f8368400640 1 -- 172.21.15.43:0/4000946599 shutdown_connections 2024-09-19T05:03:33.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.446+0000 7f8368400640 1 -- 172.21.15.43:0/4000946599 wait complete. 2024-09-19T05:03:33.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.447+0000 7f8368400640 1 Processor -- start 2024-09-19T05:03:33.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.447+0000 7f8368400640 1 -- start start 2024-09-19T05:03:33.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.448+0000 7f8368400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8360103ff0 0x7f836019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:33.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.448+0000 7f8368400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83601049a0 0x7f836019ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:33.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.448+0000 7f8368400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8360109120 0x7f836019eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:33.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.448+0000 7f8368400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8360114130 con 0x7f83601049a0 2024-09-19T05:03:33.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.448+0000 7f8368400640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8360113fb0 con 0x7f8360109120 2024-09-19T05:03:33.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.448+0000 7f8368400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f83601142b0 con 0x7f8360103ff0 2024-09-19T05:03:33.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.448+0000 7f8365000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83601049a0 0x7f836019ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:33.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.448+0000 7f8366400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8360109120 0x7f836019eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:33.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.448+0000 7f8365000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83601049a0 0x7f836019ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58854/0 (socket says 172.21.15.43:58854) 2024-09-19T05:03:33.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.448+0000 7f8365000640 1 -- 172.21.15.43:0/4247024363 learned_addr learned my addr 172.21.15.43:0/4247024363 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:33.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.448+0000 7f8365a00640 1 --2- 172.21.15.43:0/4247024363 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8360103ff0 0x7f836019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:33.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.448+0000 7f8366400640 1 -- 172.21.15.43:0/4247024363 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8360103ff0 msgr2=0x7f836019a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:33.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.448+0000 7f8366400640 1 --2- 172.21.15.43:0/4247024363 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8360103ff0 0x7f836019a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:33.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.448+0000 7f8366400640 1 -- 172.21.15.43:0/4247024363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83601049a0 msgr2=0x7f836019ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:33.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.448+0000 7f8366400640 1 --2- 172.21.15.43:0/4247024363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83601049a0 0x7f836019ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:33.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.449+0000 7f8366400640 1 -- 172.21.15.43:0/4247024363 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f836019f580 con 0x7f8360109120 2024-09-19T05:03:33.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.449+0000 7f8366400640 1 --2- 172.21.15.43:0/4247024363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8360109120 0x7f836019eea0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f835802ade0 tx=0x7f83580319f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:33.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.449+0000 7f835e000640 1 -- 172.21.15.43:0/4247024363 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8358003b70 con 0x7f8360109120 2024-09-19T05:03:33.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.449+0000 7f835e000640 1 -- 172.21.15.43:0/4247024363 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8358003d10 con 0x7f8360109120 2024-09-19T05:03:33.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.449+0000 7f8368400640 1 -- 172.21.15.43:0/4247024363 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f836019f810 con 0x7f8360109120 2024-09-19T05:03:33.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.449+0000 7f835e000640 1 -- 172.21.15.43:0/4247024363 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835802fcb0 con 0x7f8360109120 2024-09-19T05:03:33.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.449+0000 7f8368400640 1 -- 172.21.15.43:0/4247024363 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83601a7150 con 0x7f8360109120 2024-09-19T05:03:33.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.450+0000 7f8368400640 1 -- 172.21.15.43:0/4247024363 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83601043d0 con 0x7f8360109120 2024-09-19T05:03:33.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.452+0000 7f835e000640 1 -- 172.21.15.43:0/4247024363 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f8358004050 con 0x7f8360109120 2024-09-19T05:03:33.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.454+0000 7f835e000640 1 --2- 172.21.15.43:0/4247024363 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f83480a32d0 0x7f83480a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:33.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.454+0000 7f835e000640 1 -- 172.21.15.43:0/4247024363 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f83580f0b50 con 0x7f8360109120 2024-09-19T05:03:33.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.454+0000 7f8365a00640 1 --2- 172.21.15.43:0/4247024363 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f83480a32d0 0x7f83480a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:33.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.455+0000 7f8365a00640 1 --2- 172.21.15.43:0/4247024363 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f83480a32d0 0x7f83480a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8354009bd0 tx=0x7f8354005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:33.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.458+0000 7f835e000640 1 -- 172.21.15.43:0/4247024363 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f83580bd900 con 0x7f8360109120 2024-09-19T05:03:33.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.646+0000 7f8368400640 1 -- 172.21.15.43:0/4247024363 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f836010c3c0 con 0x7f8360109120 2024-09-19T05:03:33.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.647+0000 7f835e000640 1 -- 172.21.15.43:0/4247024363 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f83580bd200 con 0x7f8360109120 2024-09-19T05:03:33.648 INFO:teuthology.orchestra.run.smithi043.stdout:210453397553 2024-09-19T05:03:33.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.649+0000 7f8368400640 1 -- 172.21.15.43:0/4247024363 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f83480a32d0 msgr2=0x7f83480a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:33.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.649+0000 7f8368400640 1 --2- 172.21.15.43:0/4247024363 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f83480a32d0 0x7f83480a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8354009bd0 tx=0x7f8354005f90 comp rx=0 tx=0).stop 2024-09-19T05:03:33.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.649+0000 7f8368400640 1 -- 172.21.15.43:0/4247024363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8360109120 msgr2=0x7f836019eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:33.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.649+0000 7f8368400640 1 --2- 172.21.15.43:0/4247024363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8360109120 0x7f836019eea0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f835802ade0 tx=0x7f83580319f0 comp rx=0 tx=0).stop 2024-09-19T05:03:33.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.649+0000 7f8368400640 1 -- 172.21.15.43:0/4247024363 shutdown_connections 2024-09-19T05:03:33.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.649+0000 7f8368400640 1 --2- 172.21.15.43:0/4247024363 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f83480a32d0 0x7f83480a5790 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:33.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.649+0000 7f8368400640 1 --2- 172.21.15.43:0/4247024363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8360109120 0x7f836019eea0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:33.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.649+0000 7f8368400640 1 --2- 172.21.15.43:0/4247024363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83601049a0 0x7f836019ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:33.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.649+0000 7f8368400640 1 --2- 172.21.15.43:0/4247024363 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8360103ff0 0x7f836019a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:33.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.649+0000 7f8368400640 1 -- 172.21.15.43:0/4247024363 >> 172.21.15.43:0/4247024363 conn(0x7f83600ffa90 msgr2=0x7f8360100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:33.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.649+0000 7f8368400640 1 -- 172.21.15.43:0/4247024363 shutdown_connections 2024-09-19T05:03:33.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:33.649+0000 7f8368400640 1 -- 172.21.15.43:0/4247024363 wait complete. 2024-09-19T05:03:33.860 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:33.891 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986236 got 133143986239 for osd.4 2024-09-19T05:03:33.892 DEBUG:teuthology.parallel:result is None 2024-09-19T05:03:34.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:33 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/4247024363' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T05:03:34.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:33 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/4247024363' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T05:03:34.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:33 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/4247024363' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T05:03:34.773 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397550 got 210453397553 for osd.7 2024-09-19T05:03:34.773 DEBUG:teuthology.parallel:result is None 2024-09-19T05:03:35.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:34 smithi043 ceph-mon[30800]: pgmap v328: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-19T05:03:35.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:34 smithi151 ceph-mon[35425]: pgmap v328: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-19T05:03:35.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:34 smithi096 ceph-mon[35085]: pgmap v328: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-19T05:03:35.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.631+0000 7fbb4d800640 1 -- 172.21.15.43:0/1085293921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb48074e30 msgr2=0x7fbb4810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:35.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.631+0000 7fbb4d800640 1 --2- 172.21.15.43:0/1085293921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb48074e30 0x7fbb4810dd40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbb30009900 tx=0x7fbb3002a480 comp rx=0 tx=0).stop 2024-09-19T05:03:35.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.631+0000 7fbb4d800640 1 -- 172.21.15.43:0/1085293921 shutdown_connections 2024-09-19T05:03:35.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.631+0000 7fbb4d800640 1 --2- 172.21.15.43:0/1085293921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb4810e280 0x7fbb48110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:35.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.631+0000 7fbb4d800640 1 --2- 172.21.15.43:0/1085293921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb48074e30 0x7fbb4810dd40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:35.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.631+0000 7fbb4d800640 1 --2- 172.21.15.43:0/1085293921 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb48074510 0x7fbb480748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:35.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.631+0000 7fbb4d800640 1 -- 172.21.15.43:0/1085293921 >> 172.21.15.43:0/1085293921 conn(0x7fbb480fd740 msgr2=0x7fbb480ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:35.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.632+0000 7fbb4d800640 1 -- 172.21.15.43:0/1085293921 shutdown_connections 2024-09-19T05:03:35.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.632+0000 7fbb4d800640 1 -- 172.21.15.43:0/1085293921 wait complete. 2024-09-19T05:03:35.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.633+0000 7fbb4d800640 1 Processor -- start 2024-09-19T05:03:35.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.634+0000 7fbb4d800640 1 -- start start 2024-09-19T05:03:35.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.634+0000 7fbb4d800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb48074510 0x7fbb4819a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:35.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.634+0000 7fbb4d800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb48074e30 0x7fbb4819aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:35.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.634+0000 7fbb4d800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb4810e280 0x7fbb4819ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:35.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.634+0000 7fbb4d800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbb48114040 con 0x7fbb4810e280 2024-09-19T05:03:35.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.634+0000 7fbb4d800640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbb48113ec0 con 0x7fbb48074510 2024-09-19T05:03:35.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.634+0000 7fbb4d800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbb481141c0 con 0x7fbb48074e30 2024-09-19T05:03:35.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.634+0000 7fbb4ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb4810e280 0x7fbb4819ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:35.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.634+0000 7fbb47e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb48074510 0x7fbb4819a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:35.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.634+0000 7fbb4ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb4810e280 0x7fbb4819ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58880/0 (socket says 172.21.15.43:58880) 2024-09-19T05:03:35.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.634+0000 7fbb47e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb48074510 0x7fbb4819a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:56232/0 (socket says 172.21.15.43:56232) 2024-09-19T05:03:35.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.634+0000 7fbb4ce00640 1 -- 172.21.15.43:0/2834429331 learned_addr learned my addr 172.21.15.43:0/2834429331 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:35.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.634+0000 7fbb3f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb48074e30 0x7fbb4819aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:35.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.635+0000 7fbb47e00640 1 -- 172.21.15.43:0/2834429331 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb48074e30 msgr2=0x7fbb4819aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:35.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.635+0000 7fbb47e00640 1 --2- 172.21.15.43:0/2834429331 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb48074e30 0x7fbb4819aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:35.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.635+0000 7fbb47e00640 1 -- 172.21.15.43:0/2834429331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb4810e280 msgr2=0x7fbb4819ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:35.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.635+0000 7fbb47e00640 1 --2- 172.21.15.43:0/2834429331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb4810e280 0x7fbb4819ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:35.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.635+0000 7fbb47e00640 1 -- 172.21.15.43:0/2834429331 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb4819f500 con 0x7fbb48074510 2024-09-19T05:03:35.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.635+0000 7fbb47e00640 1 --2- 172.21.15.43:0/2834429331 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb48074510 0x7fbb4819a550 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbb3800bd70 tx=0x7fbb3800be60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:35.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.636+0000 7fbb45600640 1 -- 172.21.15.43:0/2834429331 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb38019070 con 0x7fbb48074510 2024-09-19T05:03:35.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.636+0000 7fbb4d800640 1 -- 172.21.15.43:0/2834429331 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb4819f7f0 con 0x7fbb48074510 2024-09-19T05:03:35.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.636+0000 7fbb45600640 1 -- 172.21.15.43:0/2834429331 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb38014920 con 0x7fbb48074510 2024-09-19T05:03:35.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.636+0000 7fbb45600640 1 -- 172.21.15.43:0/2834429331 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb380128e0 con 0x7fbb48074510 2024-09-19T05:03:35.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.636+0000 7fbb4d800640 1 -- 172.21.15.43:0/2834429331 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb481a70d0 con 0x7fbb48074510 2024-09-19T05:03:35.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.637+0000 7fbb4d800640 1 -- 172.21.15.43:0/2834429331 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb480748f0 con 0x7fbb48074510 2024-09-19T05:03:35.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.639+0000 7fbb45600640 1 -- 172.21.15.43:0/2834429331 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fbb3801c070 con 0x7fbb48074510 2024-09-19T05:03:35.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.641+0000 7fbb45600640 1 --2- 172.21.15.43:0/2834429331 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fbb2c0a3350 0x7fbb2c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:35.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.641+0000 7fbb45600640 1 -- 172.21.15.43:0/2834429331 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fbb380cf490 con 0x7fbb48074510 2024-09-19T05:03:35.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.641+0000 7fbb3f400640 1 --2- 172.21.15.43:0/2834429331 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fbb2c0a3350 0x7fbb2c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:35.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.642+0000 7fbb3f400640 1 --2- 172.21.15.43:0/2834429331 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fbb2c0a3350 0x7fbb2c0a5810 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbb30000c00 tx=0x7fbb30005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:35.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.642+0000 7fbb45600640 1 -- 172.21.15.43:0/2834429331 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbb3809bad0 con 0x7fbb48074510 2024-09-19T05:03:35.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.881+0000 7fbb4d800640 1 -- 172.21.15.43:0/2834429331 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fbb48109430 con 0x7fbb48074510 2024-09-19T05:03:35.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.882+0000 7fbb45600640 1 -- 172.21.15.43:0/2834429331 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fbb3809fdd0 con 0x7fbb48074510 2024-09-19T05:03:35.883 INFO:teuthology.orchestra.run.smithi043.stdout:326417514525 2024-09-19T05:03:35.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.884+0000 7fbb4d800640 1 -- 172.21.15.43:0/2834429331 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fbb2c0a3350 msgr2=0x7fbb2c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:35.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.884+0000 7fbb4d800640 1 --2- 172.21.15.43:0/2834429331 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fbb2c0a3350 0x7fbb2c0a5810 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbb30000c00 tx=0x7fbb30005b50 comp rx=0 tx=0).stop 2024-09-19T05:03:35.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.884+0000 7fbb4d800640 1 -- 172.21.15.43:0/2834429331 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb48074510 msgr2=0x7fbb4819a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:35.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.884+0000 7fbb4d800640 1 --2- 172.21.15.43:0/2834429331 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb48074510 0x7fbb4819a550 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbb3800bd70 tx=0x7fbb3800be60 comp rx=0 tx=0).stop 2024-09-19T05:03:35.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.884+0000 7fbb4d800640 1 -- 172.21.15.43:0/2834429331 shutdown_connections 2024-09-19T05:03:35.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.884+0000 7fbb4d800640 1 --2- 172.21.15.43:0/2834429331 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fbb2c0a3350 0x7fbb2c0a5810 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:35.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.884+0000 7fbb4d800640 1 --2- 172.21.15.43:0/2834429331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb4810e280 0x7fbb4819ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:35.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.884+0000 7fbb4d800640 1 --2- 172.21.15.43:0/2834429331 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb48074e30 0x7fbb4819aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:35.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.884+0000 7fbb4d800640 1 --2- 172.21.15.43:0/2834429331 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb48074510 0x7fbb4819a550 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:35.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.884+0000 7fbb4d800640 1 -- 172.21.15.43:0/2834429331 >> 172.21.15.43:0/2834429331 conn(0x7fbb480fd740 msgr2=0x7fbb480feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:35.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.885+0000 7fbb4d800640 1 -- 172.21.15.43:0/2834429331 shutdown_connections 2024-09-19T05:03:35.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:35.885+0000 7fbb4d800640 1 -- 172.21.15.43:0/2834429331 wait complete. 2024-09-19T05:03:36.123 INFO:tasks.cephadm.ceph_manager.ceph:need seq 244813135912 got 244813135915 for osd.8 2024-09-19T05:03:36.123 DEBUG:teuthology.parallel:result is None 2024-09-19T05:03:36.821 INFO:tasks.cephadm.ceph_manager.ceph:need seq 326417514521 got 326417514525 for osd.11 2024-09-19T05:03:36.821 DEBUG:teuthology.parallel:result is None 2024-09-19T05:03:37.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:36 smithi043 ceph-mon[30800]: pgmap v329: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:03:37.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:36 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/2834429331' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T05:03:37.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:36 smithi151 ceph-mon[35425]: pgmap v329: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:03:37.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:36 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/2834429331' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T05:03:37.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:36 smithi096 ceph-mon[35085]: pgmap v329: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:03:37.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:36 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/2834429331' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T05:03:37.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.435+0000 7fd1ee800640 1 -- 172.21.15.43:0/2145883379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1e8109120 msgr2=0x7fd1e8110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:37.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.435+0000 7fd1ee800640 1 --2- 172.21.15.43:0/2145883379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1e8109120 0x7fd1e8110ca0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0009ca0 tx=0x7fd1e002a8c0 comp rx=0 tx=0).stop 2024-09-19T05:03:37.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.435+0000 7fd1ee800640 1 -- 172.21.15.43:0/2145883379 shutdown_connections 2024-09-19T05:03:37.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.435+0000 7fd1ee800640 1 --2- 172.21.15.43:0/2145883379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1e8109120 0x7fd1e8110ca0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:37.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.435+0000 7fd1ee800640 1 --2- 172.21.15.43:0/2145883379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1e81049a0 0x7fd1e81089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:37.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.435+0000 7fd1ee800640 1 --2- 172.21.15.43:0/2145883379 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1e8103ff0 0x7fd1e81043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:37.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.435+0000 7fd1ee800640 1 -- 172.21.15.43:0/2145883379 >> 172.21.15.43:0/2145883379 conn(0x7fd1e80ffa90 msgr2=0x7fd1e8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:37.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.436+0000 7fd1ee800640 1 -- 172.21.15.43:0/2145883379 shutdown_connections 2024-09-19T05:03:37.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.436+0000 7fd1ee800640 1 -- 172.21.15.43:0/2145883379 wait complete. 2024-09-19T05:03:37.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.436+0000 7fd1ee800640 1 Processor -- start 2024-09-19T05:03:37.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.437+0000 7fd1ee800640 1 -- start start 2024-09-19T05:03:37.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.437+0000 7fd1ee800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1e8103ff0 0x7fd1e819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:37.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.437+0000 7fd1ee800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1e81049a0 0x7fd1e819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:37.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.437+0000 7fd1ee800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1e8109120 0x7fd1e819eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:37.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.437+0000 7fd1ee800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e81141a0 con 0x7fd1e8103ff0 2024-09-19T05:03:37.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.437+0000 7fd1ee800640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e8114020 con 0x7fd1e8109120 2024-09-19T05:03:37.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.437+0000 7fd1ee800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e8114320 con 0x7fd1e81049a0 2024-09-19T05:03:37.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.437+0000 7fd1e7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1e8103ff0 0x7fd1e819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:37.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.437+0000 7fd1e7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1e8103ff0 0x7fd1e819a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37626/0 (socket says 172.21.15.43:37626) 2024-09-19T05:03:37.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.437+0000 7fd1e7400640 1 -- 172.21.15.43:0/1536407809 learned_addr learned my addr 172.21.15.43:0/1536407809 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:37.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.437+0000 7fd1e6a00640 1 --2- 172.21.15.43:0/1536407809 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1e81049a0 0x7fd1e819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:37.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.438+0000 7fd1e7e00640 1 --2- 172.21.15.43:0/1536407809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1e8109120 0x7fd1e819eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:37.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.438+0000 7fd1e6a00640 1 -- 172.21.15.43:0/1536407809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1e8109120 msgr2=0x7fd1e819eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:37.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.438+0000 7fd1e6a00640 1 --2- 172.21.15.43:0/1536407809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1e8109120 0x7fd1e819eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:37.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.438+0000 7fd1e6a00640 1 -- 172.21.15.43:0/1536407809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1e8103ff0 msgr2=0x7fd1e819a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:37.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.438+0000 7fd1e6a00640 1 --2- 172.21.15.43:0/1536407809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1e8103ff0 0x7fd1e819a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:37.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.438+0000 7fd1e6a00640 1 -- 172.21.15.43:0/1536407809 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1e819f590 con 0x7fd1e81049a0 2024-09-19T05:03:37.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.438+0000 7fd1e7400640 1 --2- 172.21.15.43:0/1536407809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1e8103ff0 0x7fd1e819a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:03:37.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.438+0000 7fd1e6a00640 1 --2- 172.21.15.43:0/1536407809 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1e81049a0 0x7fd1e819ab20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd1d400e9f0 tx=0x7fd1d400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:37.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.438+0000 7fd1dfe00640 1 -- 172.21.15.43:0/1536407809 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d400cd80 con 0x7fd1e81049a0 2024-09-19T05:03:37.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.438+0000 7fd1ee800640 1 -- 172.21.15.43:0/1536407809 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1e819f880 con 0x7fd1e81049a0 2024-09-19T05:03:37.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.439+0000 7fd1dfe00640 1 -- 172.21.15.43:0/1536407809 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1d40035f0 con 0x7fd1e81049a0 2024-09-19T05:03:37.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.439+0000 7fd1dfe00640 1 -- 172.21.15.43:0/1536407809 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4010670 con 0x7fd1e81049a0 2024-09-19T05:03:37.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.439+0000 7fd1ee800640 1 -- 172.21.15.43:0/1536407809 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1e81a7160 con 0x7fd1e81049a0 2024-09-19T05:03:37.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.440+0000 7fd1ee800640 1 -- 172.21.15.43:0/1536407809 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1e81043d0 con 0x7fd1e81049a0 2024-09-19T05:03:37.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.442+0000 7fd1dfe00640 1 -- 172.21.15.43:0/1536407809 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fd1d40037a0 con 0x7fd1e81049a0 2024-09-19T05:03:37.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.443+0000 7fd1dfe00640 1 --2- 172.21.15.43:0/1536407809 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fd1cc0a3350 0x7fd1cc0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:37.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.444+0000 7fd1dfe00640 1 -- 172.21.15.43:0/1536407809 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd1d40d0240 con 0x7fd1e81049a0 2024-09-19T05:03:37.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.444+0000 7fd1e7400640 1 --2- 172.21.15.43:0/1536407809 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fd1cc0a3350 0x7fd1cc0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:37.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.444+0000 7fd1e7400640 1 --2- 172.21.15.43:0/1536407809 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fd1cc0a3350 0x7fd1cc0a5810 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8011500 tx=0x7fd1d8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:37.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.445+0000 7fd1dfe00640 1 -- 172.21.15.43:0/1536407809 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd1d409cdc0 con 0x7fd1e81049a0 2024-09-19T05:03:37.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.681+0000 7fd1ee800640 1 -- 172.21.15.43:0/1536407809 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fd1e819b940 con 0x7fd1e81049a0 2024-09-19T05:03:37.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.682+0000 7fd1dfe00640 1 -- 172.21.15.43:0/1536407809 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd1e819b940 con 0x7fd1e81049a0 2024-09-19T05:03:37.683 INFO:teuthology.orchestra.run.smithi043.stdout:274877906982 2024-09-19T05:03:37.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.685+0000 7fd1ee800640 1 -- 172.21.15.43:0/1536407809 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fd1cc0a3350 msgr2=0x7fd1cc0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:37.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.685+0000 7fd1ee800640 1 --2- 172.21.15.43:0/1536407809 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fd1cc0a3350 0x7fd1cc0a5810 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8011500 tx=0x7fd1d8005f90 comp rx=0 tx=0).stop 2024-09-19T05:03:37.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.685+0000 7fd1ee800640 1 -- 172.21.15.43:0/1536407809 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1e81049a0 msgr2=0x7fd1e819ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:37.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.685+0000 7fd1ee800640 1 --2- 172.21.15.43:0/1536407809 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1e81049a0 0x7fd1e819ab20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd1d400e9f0 tx=0x7fd1d400eeb0 comp rx=0 tx=0).stop 2024-09-19T05:03:37.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.685+0000 7fd1ee800640 1 -- 172.21.15.43:0/1536407809 shutdown_connections 2024-09-19T05:03:37.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.685+0000 7fd1ee800640 1 --2- 172.21.15.43:0/1536407809 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fd1cc0a3350 0x7fd1cc0a5810 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:37.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.685+0000 7fd1ee800640 1 --2- 172.21.15.43:0/1536407809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1e8109120 0x7fd1e819eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:37.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.685+0000 7fd1ee800640 1 --2- 172.21.15.43:0/1536407809 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1e81049a0 0x7fd1e819ab20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:37.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.685+0000 7fd1ee800640 1 --2- 172.21.15.43:0/1536407809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1e8103ff0 0x7fd1e819a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:37.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.685+0000 7fd1ee800640 1 -- 172.21.15.43:0/1536407809 >> 172.21.15.43:0/1536407809 conn(0x7fd1e80ffa90 msgr2=0x7fd1e81005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:37.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.685+0000 7fd1ee800640 1 -- 172.21.15.43:0/1536407809 shutdown_connections 2024-09-19T05:03:37.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:37.685+0000 7fd1ee800640 1 -- 172.21.15.43:0/1536407809 wait complete. 2024-09-19T05:03:38.015 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:37 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/1536407809' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T05:03:38.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:37 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/1536407809' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T05:03:38.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:37 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/1536407809' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T05:03:38.387 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906979 got 274877906982 for osd.9 2024-09-19T05:03:38.388 DEBUG:teuthology.parallel:result is None 2024-09-19T05:03:38.388 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T05:03:38.388 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T05:03:38.788 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:39.071 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:38 smithi043 ceph-mon[30800]: pgmap v330: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T05:03:39.071 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:38 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:03:39.072 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:38 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:03:39.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:38 smithi151 ceph-mon[35425]: pgmap v330: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T05:03:39.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:38 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:03:39.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:38 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:03:39.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:38 smithi096 ceph-mon[35085]: pgmap v330: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T05:03:39.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:38 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:03:39.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:38 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:03:39.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.326+0000 7fcea5e00640 1 -- 172.21.15.43:0/1417376665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcea0103ff0 msgr2=0x7fcea01043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:39.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.326+0000 7fcea5e00640 1 --2- 172.21.15.43:0/1417376665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcea0103ff0 0x7fcea01043d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fce90009900 tx=0x7fce9002a3f0 comp rx=0 tx=0).stop 2024-09-19T05:03:39.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.327+0000 7fcea5e00640 1 -- 172.21.15.43:0/1417376665 shutdown_connections 2024-09-19T05:03:39.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.327+0000 7fcea5e00640 1 --2- 172.21.15.43:0/1417376665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcea0109120 0x7fcea0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:39.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.327+0000 7fcea5e00640 1 --2- 172.21.15.43:0/1417376665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcea01049a0 0x7fcea01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:39.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.327+0000 7fcea5e00640 1 --2- 172.21.15.43:0/1417376665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcea0103ff0 0x7fcea01043d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:39.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.327+0000 7fcea5e00640 1 -- 172.21.15.43:0/1417376665 >> 172.21.15.43:0/1417376665 conn(0x7fcea00ffa90 msgr2=0x7fcea0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:39.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.327+0000 7fcea5e00640 1 -- 172.21.15.43:0/1417376665 shutdown_connections 2024-09-19T05:03:39.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.327+0000 7fcea5e00640 1 -- 172.21.15.43:0/1417376665 wait complete. 2024-09-19T05:03:39.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.327+0000 7fcea5e00640 1 Processor -- start 2024-09-19T05:03:39.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.328+0000 7fcea5e00640 1 -- start start 2024-09-19T05:03:39.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.328+0000 7fcea5e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcea0103ff0 0x7fcea019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:39.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.328+0000 7fcea5e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcea01049a0 0x7fcea019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:39.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.328+0000 7fcea5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcea0109120 0x7fcea019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:39.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.328+0000 7fcea5e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcea0114140 con 0x7fcea0109120 2024-09-19T05:03:39.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.328+0000 7fcea5e00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcea0113fc0 con 0x7fcea0103ff0 2024-09-19T05:03:39.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.328+0000 7fcea5e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcea01142c0 con 0x7fcea01049a0 2024-09-19T05:03:39.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.329+0000 7fce9ea00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcea0103ff0 0x7fcea019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:39.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.329+0000 7fce9ea00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcea0103ff0 0x7fcea019a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:53592/0 (socket says 172.21.15.43:53592) 2024-09-19T05:03:39.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.329+0000 7fce9ea00640 1 -- 172.21.15.43:0/3975035325 learned_addr learned my addr 172.21.15.43:0/3975035325 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:39.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.329+0000 7fce9f400640 1 --2- 172.21.15.43:0/3975035325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcea0109120 0x7fcea019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:39.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.329+0000 7fce97e00640 1 --2- 172.21.15.43:0/3975035325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcea01049a0 0x7fcea019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:39.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.329+0000 7fce9ea00640 1 -- 172.21.15.43:0/3975035325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcea01049a0 msgr2=0x7fcea019ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:39.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.329+0000 7fce9ea00640 1 --2- 172.21.15.43:0/3975035325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcea01049a0 0x7fcea019ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:39.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.329+0000 7fce9ea00640 1 -- 172.21.15.43:0/3975035325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcea0109120 msgr2=0x7fcea019eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:39.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.329+0000 7fce9ea00640 1 --2- 172.21.15.43:0/3975035325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcea0109120 0x7fcea019eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:39.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.329+0000 7fce9ea00640 1 -- 172.21.15.43:0/3975035325 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcea019f5b0 con 0x7fcea0103ff0 2024-09-19T05:03:39.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.329+0000 7fce9f400640 1 --2- 172.21.15.43:0/3975035325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcea0109120 0x7fcea019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:03:39.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.330+0000 7fce9ea00640 1 --2- 172.21.15.43:0/3975035325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcea0103ff0 0x7fcea019a600 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fce90006010 tx=0x7fce90004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:39.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.330+0000 7fcea4a00640 1 -- 172.21.15.43:0/3975035325 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9003f030 con 0x7fcea0103ff0 2024-09-19T05:03:39.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.330+0000 7fcea5e00640 1 -- 172.21.15.43:0/3975035325 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcea019f840 con 0x7fcea0103ff0 2024-09-19T05:03:39.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.330+0000 7fcea4a00640 1 -- 172.21.15.43:0/3975035325 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce90004440 con 0x7fcea0103ff0 2024-09-19T05:03:39.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.330+0000 7fcea4a00640 1 -- 172.21.15.43:0/3975035325 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce90035070 con 0x7fcea0103ff0 2024-09-19T05:03:39.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.330+0000 7fcea5e00640 1 -- 172.21.15.43:0/3975035325 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcea01a7180 con 0x7fcea0103ff0 2024-09-19T05:03:39.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.331+0000 7fcea5e00640 1 -- 172.21.15.43:0/3975035325 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcea01043d0 con 0x7fcea0103ff0 2024-09-19T05:03:39.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.333+0000 7fcea4a00640 1 -- 172.21.15.43:0/3975035325 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fce90042030 con 0x7fcea0103ff0 2024-09-19T05:03:39.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.335+0000 7fcea4a00640 1 --2- 172.21.15.43:0/3975035325 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fce840a3350 0x7fce840a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:39.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.335+0000 7fcea4a00640 1 -- 172.21.15.43:0/3975035325 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fce900efdb0 con 0x7fcea0103ff0 2024-09-19T05:03:39.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.335+0000 7fce97e00640 1 --2- 172.21.15.43:0/3975035325 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fce840a3350 0x7fce840a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:39.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.335+0000 7fce97e00640 1 --2- 172.21.15.43:0/3975035325 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fce840a3350 0x7fce840a5810 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcea00fff20 tx=0x7fce88005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:39.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.339+0000 7fcea4a00640 1 -- 172.21.15.43:0/3975035325 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fce900f4070 con 0x7fcea0103ff0 2024-09-19T05:03:39.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.527+0000 7fcea5e00640 1 -- 172.21.15.43:0/3975035325 --> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcea0062630 con 0x7fce840a3350 2024-09-19T05:03:39.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.534+0000 7fcea4a00640 1 -- 172.21.15.43:0/3975035325 <== mgr.14150 v2:172.21.15.43:6800/3167108925 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259022 (secure 0 0 0) 0x7fcea0062630 con 0x7fce840a3350 2024-09-19T05:03:39.537 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:03:39.538 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-19T05:03:39.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.538+0000 7fcea5e00640 1 -- 172.21.15.43:0/3975035325 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fce840a3350 msgr2=0x7fce840a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:39.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.538+0000 7fcea5e00640 1 --2- 172.21.15.43:0/3975035325 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fce840a3350 0x7fce840a5810 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcea00fff20 tx=0x7fce88005f90 comp rx=0 tx=0).stop 2024-09-19T05:03:39.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.538+0000 7fcea5e00640 1 -- 172.21.15.43:0/3975035325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcea0103ff0 msgr2=0x7fcea019a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:39.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.538+0000 7fcea5e00640 1 --2- 172.21.15.43:0/3975035325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcea0103ff0 0x7fcea019a600 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fce90006010 tx=0x7fce90004040 comp rx=0 tx=0).stop 2024-09-19T05:03:39.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.538+0000 7fcea5e00640 1 -- 172.21.15.43:0/3975035325 shutdown_connections 2024-09-19T05:03:39.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.538+0000 7fcea5e00640 1 --2- 172.21.15.43:0/3975035325 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fce840a3350 0x7fce840a5810 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:39.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.538+0000 7fcea5e00640 1 --2- 172.21.15.43:0/3975035325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcea0109120 0x7fcea019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:39.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.538+0000 7fcea5e00640 1 --2- 172.21.15.43:0/3975035325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcea01049a0 0x7fcea019ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:39.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.538+0000 7fcea5e00640 1 --2- 172.21.15.43:0/3975035325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcea0103ff0 0x7fcea019a600 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:39.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.538+0000 7fcea5e00640 1 -- 172.21.15.43:0/3975035325 >> 172.21.15.43:0/3975035325 conn(0x7fcea00ffa90 msgr2=0x7fcea0100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:39.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.538+0000 7fcea5e00640 1 -- 172.21.15.43:0/3975035325 shutdown_connections 2024-09-19T05:03:39.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:39.538+0000 7fcea5e00640 1 -- 172.21.15.43:0/3975035325 wait complete. 2024-09-19T05:03:40.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:39 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:40.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:39 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:03:40.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:39 smithi151 ceph-mon[35425]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:03:40.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:39 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:40.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:39 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:03:40.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:39 smithi096 ceph-mon[35085]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:03:40.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:39 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:03:40.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:39 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:03:40.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:39 smithi043 ceph-mon[30800]: from='mgr.14150 172.21.15.43:0/3246419152' entity='mgr.x' 2024-09-19T05:03:40.321 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-09-19T05:03:39.157880+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660424,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":740,"num_read_kb":498,"num_write":430,"num_write_kb":3078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":1181717,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":351,"ondisk_log_size":351,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373336,"kb_used_data":11520,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568488,"statfs":{"total":1151940427776,"available":1151558131712,"internally_reserved":0,"allocated":11796480,"data_stored":7399812,"data_compressed":178482,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002967"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.708643+0000","last_change":"2024-09-19T05:02:24.426572+0000","last_active":"2024-09-19T05:03:08.708643+0000","last_peered":"2024-09-19T05:03:08.708643+0000","last_clean":"2024-09-19T05:03:08.708643+0000","last_became_active":"2024-09-19T05:02:24.426241+0000","last_became_peered":"2024-09-19T05:02:24.426241+0000","last_unstale":"2024-09-19T05:03:08.708643+0000","last_undegraded":"2024-09-19T05:03:08.708643+0000","last_fullsized":"2024-09-19T05:03:08.708643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:42.511495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.701611+0000","last_change":"2024-09-19T05:02:24.427263+0000","last_active":"2024-09-19T05:03:08.701611+0000","last_peered":"2024-09-19T05:03:08.701611+0000","last_clean":"2024-09-19T05:03:08.701611+0000","last_became_active":"2024-09-19T05:02:24.426928+0000","last_became_peered":"2024-09-19T05:02:24.426928+0000","last_unstale":"2024-09-19T05:03:08.701611+0000","last_undegraded":"2024-09-19T05:03:08.701611+0000","last_fullsized":"2024-09-19T05:03:08.701611+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:03.177020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.701980+0000","last_change":"2024-09-19T05:02:24.432709+0000","last_active":"2024-09-19T05:03:08.701980+0000","last_peered":"2024-09-19T05:03:08.701980+0000","last_clean":"2024-09-19T05:03:08.701980+0000","last_became_active":"2024-09-19T05:02:24.431114+0000","last_became_peered":"2024-09-19T05:02:24.431114+0000","last_unstale":"2024-09-19T05:03:08.701980+0000","last_undegraded":"2024-09-19T05:03:08.701980+0000","last_fullsized":"2024-09-19T05:03:08.701980+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:57.244919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.714711+0000","last_change":"2024-09-19T05:03:08.713389+0000","last_active":"2024-09-19T05:03:08.714711+0000","last_peered":"2024-09-19T05:03:08.714711+0000","last_clean":"2024-09-19T05:03:08.714711+0000","last_became_active":"2024-09-19T05:03:08.713058+0000","last_became_peered":"2024-09-19T05:03:08.713058+0000","last_unstale":"2024-09-19T05:03:08.714711+0000","last_undegraded":"2024-09-19T05:03:08.714711+0000","last_fullsized":"2024-09-19T05:03:08.714711+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:52:10.993733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.704408+0000","last_change":"2024-09-19T05:02:24.419992+0000","last_active":"2024-09-19T05:03:08.704408+0000","last_peered":"2024-09-19T05:03:08.704408+0000","last_clean":"2024-09-19T05:03:08.704408+0000","last_became_active":"2024-09-19T05:02:24.419643+0000","last_became_peered":"2024-09-19T05:02:24.419643+0000","last_unstale":"2024-09-19T05:03:08.704408+0000","last_undegraded":"2024-09-19T05:03:08.704408+0000","last_fullsized":"2024-09-19T05:03:08.704408+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:54:40.548216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.706084+0000","last_change":"2024-09-19T05:02:24.434106+0000","last_active":"2024-09-19T05:03:08.706084+0000","last_peered":"2024-09-19T05:03:08.706084+0000","last_clean":"2024-09-19T05:03:08.706084+0000","last_became_active":"2024-09-19T05:02:24.433124+0000","last_became_peered":"2024-09-19T05:02:24.433124+0000","last_unstale":"2024-09-19T05:03:08.706084+0000","last_undegraded":"2024-09-19T05:03:08.706084+0000","last_fullsized":"2024-09-19T05:03:08.706084+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:09:46.502506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.707062+0000","last_change":"2024-09-19T05:02:24.429640+0000","last_active":"2024-09-19T05:03:08.707062+0000","last_peered":"2024-09-19T05:03:08.707062+0000","last_clean":"2024-09-19T05:03:08.707062+0000","last_became_active":"2024-09-19T05:02:24.429326+0000","last_became_peered":"2024-09-19T05:02:24.429326+0000","last_unstale":"2024-09-19T05:03:08.707062+0000","last_undegraded":"2024-09-19T05:03:08.707062+0000","last_fullsized":"2024-09-19T05:03:08.707062+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:27.330449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.711936+0000","last_change":"2024-09-19T05:02:24.430933+0000","last_active":"2024-09-19T05:03:08.711936+0000","last_peered":"2024-09-19T05:03:08.711936+0000","last_clean":"2024-09-19T05:03:08.711936+0000","last_became_active":"2024-09-19T05:02:24.430478+0000","last_became_peered":"2024-09-19T05:02:24.430478+0000","last_unstale":"2024-09-19T05:03:08.711936+0000","last_undegraded":"2024-09-19T05:03:08.711936+0000","last_fullsized":"2024-09-19T05:03:08.711936+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:45:22.171049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.702535+0000","last_change":"2024-09-19T05:02:24.433207+0000","last_active":"2024-09-19T05:03:08.702535+0000","last_peered":"2024-09-19T05:03:08.702535+0000","last_clean":"2024-09-19T05:03:08.702535+0000","last_became_active":"2024-09-19T05:02:24.432882+0000","last_became_peered":"2024-09-19T05:02:24.432882+0000","last_unstale":"2024-09-19T05:03:08.702535+0000","last_undegraded":"2024-09-19T05:03:08.702535+0000","last_fullsized":"2024-09-19T05:03:08.702535+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:06:42.748227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.706778+0000","last_change":"2024-09-19T05:02:24.433825+0000","last_active":"2024-09-19T05:03:08.706778+0000","last_peered":"2024-09-19T05:03:08.706778+0000","last_clean":"2024-09-19T05:03:08.706778+0000","last_became_active":"2024-09-19T05:02:24.433088+0000","last_became_peered":"2024-09-19T05:02:24.433088+0000","last_unstale":"2024-09-19T05:03:08.706778+0000","last_undegraded":"2024-09-19T05:03:08.706778+0000","last_fullsized":"2024-09-19T05:03:08.706778+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:12:53.039028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.707237+0000","last_change":"2024-09-19T05:02:24.427661+0000","last_active":"2024-09-19T05:03:08.707237+0000","last_peered":"2024-09-19T05:03:08.707237+0000","last_clean":"2024-09-19T05:03:08.707237+0000","last_became_active":"2024-09-19T05:02:24.427093+0000","last_became_peered":"2024-09-19T05:02:24.427093+0000","last_unstale":"2024-09-19T05:03:08.707237+0000","last_undegraded":"2024-09-19T05:03:08.707237+0000","last_fullsized":"2024-09-19T05:03:08.707237+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:43.091041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.707657+0000","last_change":"2024-09-19T05:02:24.430820+0000","last_active":"2024-09-19T05:03:08.707657+0000","last_peered":"2024-09-19T05:03:08.707657+0000","last_clean":"2024-09-19T05:03:08.707657+0000","last_became_active":"2024-09-19T05:02:24.429945+0000","last_became_peered":"2024-09-19T05:02:24.429945+0000","last_unstale":"2024-09-19T05:03:08.707657+0000","last_undegraded":"2024-09-19T05:03:08.707657+0000","last_fullsized":"2024-09-19T05:03:08.707657+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:21:59.400725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.705892+0000","last_change":"2024-09-19T05:02:24.428695+0000","last_active":"2024-09-19T05:03:08.705892+0000","last_peered":"2024-09-19T05:03:08.705892+0000","last_clean":"2024-09-19T05:03:08.705892+0000","last_became_active":"2024-09-19T05:02:24.426385+0000","last_became_peered":"2024-09-19T05:02:24.426385+0000","last_unstale":"2024-09-19T05:03:08.705892+0000","last_undegraded":"2024-09-19T05:03:08.705892+0000","last_fullsized":"2024-09-19T05:03:08.705892+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:44:00.893313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.701499+0000","last_change":"2024-09-19T05:02:24.424935+0000","last_active":"2024-09-19T05:03:08.701499+0000","last_peered":"2024-09-19T05:03:08.701499+0000","last_clean":"2024-09-19T05:03:08.701499+0000","last_became_active":"2024-09-19T05:02:24.424219+0000","last_became_peered":"2024-09-19T05:02:24.424219+0000","last_unstale":"2024-09-19T05:03:08.701499+0000","last_undegraded":"2024-09-19T05:03:08.701499+0000","last_fullsized":"2024-09-19T05:03:08.701499+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:00.890778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.701387+0000","last_change":"2024-09-19T05:02:24.425407+0000","last_active":"2024-09-19T05:03:08.701387+0000","last_peered":"2024-09-19T05:03:08.701387+0000","last_clean":"2024-09-19T05:03:08.701387+0000","last_became_active":"2024-09-19T05:02:24.425053+0000","last_became_peered":"2024-09-19T05:02:24.425053+0000","last_unstale":"2024-09-19T05:03:08.701387+0000","last_undegraded":"2024-09-19T05:03:08.701387+0000","last_fullsized":"2024-09-19T05:03:08.701387+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:54:10.024183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.701951+0000","last_change":"2024-09-19T05:02:24.427888+0000","last_active":"2024-09-19T05:03:08.701951+0000","last_peered":"2024-09-19T05:03:08.701951+0000","last_clean":"2024-09-19T05:03:08.701951+0000","last_became_active":"2024-09-19T05:02:24.427589+0000","last_became_peered":"2024-09-19T05:02:24.427589+0000","last_unstale":"2024-09-19T05:03:08.701951+0000","last_undegraded":"2024-09-19T05:03:08.701951+0000","last_fullsized":"2024-09-19T05:03:08.701951+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:02:02.654416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.708819+0000","last_change":"2024-09-19T05:02:24.426654+0000","last_active":"2024-09-19T05:03:08.708819+0000","last_peered":"2024-09-19T05:03:08.708819+0000","last_clean":"2024-09-19T05:03:08.708819+0000","last_became_active":"2024-09-19T05:02:24.426352+0000","last_became_peered":"2024-09-19T05:02:24.426352+0000","last_unstale":"2024-09-19T05:03:08.708819+0000","last_undegraded":"2024-09-19T05:03:08.708819+0000","last_fullsized":"2024-09-19T05:03:08.708819+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:20.055222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.706137+0000","last_change":"2024-09-19T05:02:24.432178+0000","last_active":"2024-09-19T05:03:08.706137+0000","last_peered":"2024-09-19T05:03:08.706137+0000","last_clean":"2024-09-19T05:03:08.706137+0000","last_became_active":"2024-09-19T05:02:24.429058+0000","last_became_peered":"2024-09-19T05:02:24.429058+0000","last_unstale":"2024-09-19T05:03:08.706137+0000","last_undegraded":"2024-09-19T05:03:08.706137+0000","last_fullsized":"2024-09-19T05:03:08.706137+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:08.001134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.702195+0000","last_change":"2024-09-19T05:02:24.428202+0000","last_active":"2024-09-19T05:03:08.702195+0000","last_peered":"2024-09-19T05:03:08.702195+0000","last_clean":"2024-09-19T05:03:08.702195+0000","last_became_active":"2024-09-19T05:02:24.427741+0000","last_became_peered":"2024-09-19T05:02:24.427741+0000","last_unstale":"2024-09-19T05:03:08.702195+0000","last_undegraded":"2024-09-19T05:03:08.702195+0000","last_fullsized":"2024-09-19T05:03:08.702195+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:42:51.076056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.708596+0000","last_change":"2024-09-19T05:02:24.425007+0000","last_active":"2024-09-19T05:03:08.708596+0000","last_peered":"2024-09-19T05:03:08.708596+0000","last_clean":"2024-09-19T05:03:08.708596+0000","last_became_active":"2024-09-19T05:02:24.424612+0000","last_became_peered":"2024-09-19T05:02:24.424612+0000","last_unstale":"2024-09-19T05:03:08.708596+0000","last_undegraded":"2024-09-19T05:03:08.708596+0000","last_fullsized":"2024-09-19T05:03:08.708596+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:36:06.115757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.705989+0000","last_change":"2024-09-19T05:02:24.428724+0000","last_active":"2024-09-19T05:03:08.705989+0000","last_peered":"2024-09-19T05:03:08.705989+0000","last_clean":"2024-09-19T05:03:08.705989+0000","last_became_active":"2024-09-19T05:02:24.427730+0000","last_became_peered":"2024-09-19T05:02:24.427730+0000","last_unstale":"2024-09-19T05:03:08.705989+0000","last_undegraded":"2024-09-19T05:03:08.705989+0000","last_fullsized":"2024-09-19T05:03:08.705989+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:15:25.466553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.702112+0000","last_change":"2024-09-19T05:02:24.426238+0000","last_active":"2024-09-19T05:03:08.702112+0000","last_peered":"2024-09-19T05:03:08.702112+0000","last_clean":"2024-09-19T05:03:08.702112+0000","last_became_active":"2024-09-19T05:02:24.425915+0000","last_became_peered":"2024-09-19T05:02:24.425915+0000","last_unstale":"2024-09-19T05:03:08.702112+0000","last_undegraded":"2024-09-19T05:03:08.702112+0000","last_fullsized":"2024-09-19T05:03:08.702112+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:37:18.444042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.712573+0000","last_change":"2024-09-19T05:02:24.425726+0000","last_active":"2024-09-19T05:03:08.712573+0000","last_peered":"2024-09-19T05:03:08.712573+0000","last_clean":"2024-09-19T05:03:08.712573+0000","last_became_active":"2024-09-19T05:02:24.425409+0000","last_became_peered":"2024-09-19T05:02:24.425409+0000","last_unstale":"2024-09-19T05:03:08.712573+0000","last_undegraded":"2024-09-19T05:03:08.712573+0000","last_fullsized":"2024-09-19T05:03:08.712573+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:12.068304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.896269+0000","last_change":"2024-09-19T05:02:24.425267+0000","last_active":"2024-09-19T05:03:08.896269+0000","last_peered":"2024-09-19T05:03:08.896269+0000","last_clean":"2024-09-19T05:03:08.896269+0000","last_became_active":"2024-09-19T05:02:24.424933+0000","last_became_peered":"2024-09-19T05:02:24.424933+0000","last_unstale":"2024-09-19T05:03:08.896269+0000","last_undegraded":"2024-09-19T05:03:08.896269+0000","last_fullsized":"2024-09-19T05:03:08.896269+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:51:55.274870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.714845+0000","last_change":"2024-09-19T05:02:24.427909+0000","last_active":"2024-09-19T05:03:08.714845+0000","last_peered":"2024-09-19T05:03:08.714845+0000","last_clean":"2024-09-19T05:03:08.714845+0000","last_became_active":"2024-09-19T05:02:24.425515+0000","last_became_peered":"2024-09-19T05:02:24.425515+0000","last_unstale":"2024-09-19T05:03:08.714845+0000","last_undegraded":"2024-09-19T05:03:08.714845+0000","last_fullsized":"2024-09-19T05:03:08.714845+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:12:22.390754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.702279+0000","last_change":"2024-09-19T05:02:24.426201+0000","last_active":"2024-09-19T05:03:08.702279+0000","last_peered":"2024-09-19T05:03:08.702279+0000","last_clean":"2024-09-19T05:03:08.702279+0000","last_became_active":"2024-09-19T05:02:24.425890+0000","last_became_peered":"2024-09-19T05:02:24.425890+0000","last_unstale":"2024-09-19T05:03:08.702279+0000","last_undegraded":"2024-09-19T05:03:08.702279+0000","last_fullsized":"2024-09-19T05:03:08.702279+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:36.680002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.708581+0000","last_change":"2024-09-19T05:02:24.416037+0000","last_active":"2024-09-19T05:03:08.708581+0000","last_peered":"2024-09-19T05:03:08.708581+0000","last_clean":"2024-09-19T05:03:08.708581+0000","last_became_active":"2024-09-19T05:02:24.415535+0000","last_became_peered":"2024-09-19T05:02:24.415535+0000","last_unstale":"2024-09-19T05:03:08.708581+0000","last_undegraded":"2024-09-19T05:03:08.708581+0000","last_fullsized":"2024-09-19T05:03:08.708581+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:14.756410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.708063+0000","last_change":"2024-09-19T05:03:08.708063+0000","last_active":"2024-09-19T05:03:08.708063+0000","last_peered":"2024-09-19T05:03:08.708063+0000","last_clean":"2024-09-19T05:03:08.708063+0000","last_became_active":"2024-09-19T05:03:08.707474+0000","last_became_peered":"2024-09-19T05:03:08.707474+0000","last_unstale":"2024-09-19T05:03:08.708063+0000","last_undegraded":"2024-09-19T05:03:08.708063+0000","last_fullsized":"2024-09-19T05:03:08.708063+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:20:55.604156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.702331+0000","last_change":"2024-09-19T05:02:24.423776+0000","last_active":"2024-09-19T05:03:08.702331+0000","last_peered":"2024-09-19T05:03:08.702331+0000","last_clean":"2024-09-19T05:03:08.702331+0000","last_became_active":"2024-09-19T05:02:24.423446+0000","last_became_peered":"2024-09-19T05:02:24.423446+0000","last_unstale":"2024-09-19T05:03:08.702331+0000","last_undegraded":"2024-09-19T05:03:08.702331+0000","last_fullsized":"2024-09-19T05:03:08.702331+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:47:02.747049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.702005+0000","last_change":"2024-09-19T05:02:24.420534+0000","last_active":"2024-09-19T05:03:08.702005+0000","last_peered":"2024-09-19T05:03:08.702005+0000","last_clean":"2024-09-19T05:03:08.702005+0000","last_became_active":"2024-09-19T05:02:24.419980+0000","last_became_peered":"2024-09-19T05:02:24.419980+0000","last_unstale":"2024-09-19T05:03:08.702005+0000","last_undegraded":"2024-09-19T05:03:08.702005+0000","last_fullsized":"2024-09-19T05:03:08.702005+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:33:49.537258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.702152+0000","last_change":"2024-09-19T05:02:24.424859+0000","last_active":"2024-09-19T05:03:08.702152+0000","last_peered":"2024-09-19T05:03:08.702152+0000","last_clean":"2024-09-19T05:03:08.702152+0000","last_became_active":"2024-09-19T05:02:24.424514+0000","last_became_peered":"2024-09-19T05:02:24.424514+0000","last_unstale":"2024-09-19T05:03:08.702152+0000","last_undegraded":"2024-09-19T05:03:08.702152+0000","last_fullsized":"2024-09-19T05:03:08.702152+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:41:41.929578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.708322+0000","last_change":"2024-09-19T05:02:24.421756+0000","last_active":"2024-09-19T05:03:08.708322+0000","last_peered":"2024-09-19T05:03:08.708322+0000","last_clean":"2024-09-19T05:03:08.708322+0000","last_became_active":"2024-09-19T05:02:24.420910+0000","last_became_peered":"2024-09-19T05:02:24.420910+0000","last_unstale":"2024-09-19T05:03:08.708322+0000","last_undegraded":"2024-09-19T05:03:08.708322+0000","last_fullsized":"2024-09-19T05:03:08.708322+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:33.759745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.704511+0000","last_change":"2024-09-19T05:02:24.428514+0000","last_active":"2024-09-19T05:03:08.704511+0000","last_peered":"2024-09-19T05:03:08.704511+0000","last_clean":"2024-09-19T05:03:08.704511+0000","last_became_active":"2024-09-19T05:02:24.428206+0000","last_became_peered":"2024-09-19T05:02:24.428206+0000","last_unstale":"2024-09-19T05:03:08.704511+0000","last_undegraded":"2024-09-19T05:03:08.704511+0000","last_fullsized":"2024-09-19T05:03:08.704511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:33.045739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":91,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:35.595525+0000","last_change":"2024-09-19T05:02:22.026683+0000","last_active":"2024-09-19T05:03:35.595525+0000","last_peered":"2024-09-19T05:03:35.595525+0000","last_clean":"2024-09-19T05:03:35.595525+0000","last_became_active":"2024-09-19T05:02:22.026342+0000","last_became_peered":"2024-09-19T05:02:22.026342+0000","last_unstale":"2024-09-19T05:03:35.595525+0000","last_undegraded":"2024-09-19T05:03:35.595525+0000","last_fullsized":"2024-09-19T05:03:35.595525+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:01:37.462838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":61,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.896757+0000","last_change":"2024-09-19T05:02:24.421753+0000","last_active":"2024-09-19T05:03:08.896757+0000","last_peered":"2024-09-19T05:03:08.896757+0000","last_clean":"2024-09-19T05:03:08.896757+0000","last_became_active":"2024-09-19T05:02:24.421455+0000","last_became_peered":"2024-09-19T05:02:24.421455+0000","last_unstale":"2024-09-19T05:03:08.896757+0000","last_undegraded":"2024-09-19T05:03:08.896757+0000","last_fullsized":"2024-09-19T05:03:08.896757+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:06.416550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.716866+0000","last_change":"2024-09-19T05:03:08.716866+0000","last_active":"2024-09-19T05:03:08.716866+0000","last_peered":"2024-09-19T05:03:08.716866+0000","last_clean":"2024-09-19T05:03:08.716866+0000","last_became_active":"2024-09-19T05:03:08.709783+0000","last_became_peered":"2024-09-19T05:03:08.709783+0000","last_unstale":"2024-09-19T05:03:08.716866+0000","last_undegraded":"2024-09-19T05:03:08.716866+0000","last_fullsized":"2024-09-19T05:03:08.716866+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:26.103651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.704707+0000","last_change":"2024-09-19T05:02:22.026599+0000","last_active":"2024-09-19T05:03:08.704707+0000","last_peered":"2024-09-19T05:03:08.704707+0000","last_clean":"2024-09-19T05:03:08.704707+0000","last_became_active":"2024-09-19T05:02:22.024652+0000","last_became_peered":"2024-09-19T05:02:22.024652+0000","last_unstale":"2024-09-19T05:03:08.704707+0000","last_undegraded":"2024-09-19T05:03:08.704707+0000","last_fullsized":"2024-09-19T05:03:08.704707+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:14:32.508333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.711885+0000","last_change":"2024-09-19T05:02:24.419262+0000","last_active":"2024-09-19T05:03:08.711885+0000","last_peered":"2024-09-19T05:03:08.711885+0000","last_clean":"2024-09-19T05:03:08.711885+0000","last_became_active":"2024-09-19T05:02:24.418589+0000","last_became_peered":"2024-09-19T05:02:24.418589+0000","last_unstale":"2024-09-19T05:03:08.711885+0000","last_undegraded":"2024-09-19T05:03:08.711885+0000","last_fullsized":"2024-09-19T05:03:08.711885+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:27.256676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.896552+0000","last_change":"2024-09-19T05:02:22.024225+0000","last_active":"2024-09-19T05:03:08.896552+0000","last_peered":"2024-09-19T05:03:08.896552+0000","last_clean":"2024-09-19T05:03:08.896552+0000","last_became_active":"2024-09-19T05:02:22.023882+0000","last_became_peered":"2024-09-19T05:02:22.023882+0000","last_unstale":"2024-09-19T05:03:08.896552+0000","last_undegraded":"2024-09-19T05:03:08.896552+0000","last_fullsized":"2024-09-19T05:03:08.896552+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:29.619634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.705735+0000","last_change":"2024-09-19T05:02:24.436511+0000","last_active":"2024-09-19T05:03:08.705735+0000","last_peered":"2024-09-19T05:03:08.705735+0000","last_clean":"2024-09-19T05:03:08.705735+0000","last_became_active":"2024-09-19T05:02:24.435565+0000","last_became_peered":"2024-09-19T05:02:24.435565+0000","last_unstale":"2024-09-19T05:03:08.705735+0000","last_undegraded":"2024-09-19T05:03:08.705735+0000","last_fullsized":"2024-09-19T05:03:08.705735+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:16:43.779420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":140,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:34.593239+0000","last_change":"2024-09-19T05:02:22.024281+0000","last_active":"2024-09-19T05:03:34.593239+0000","last_peered":"2024-09-19T05:03:34.593239+0000","last_clean":"2024-09-19T05:03:34.593239+0000","last_became_active":"2024-09-19T05:02:22.023868+0000","last_became_peered":"2024-09-19T05:02:22.023868+0000","last_unstale":"2024-09-19T05:03:34.593239+0000","last_undegraded":"2024-09-19T05:03:34.593239+0000","last_fullsized":"2024-09-19T05:03:34.593239+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:29:47.597570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.702411+0000","last_change":"2024-09-19T05:02:24.436383+0000","last_active":"2024-09-19T05:03:08.702411+0000","last_peered":"2024-09-19T05:03:08.702411+0000","last_clean":"2024-09-19T05:03:08.702411+0000","last_became_active":"2024-09-19T05:02:24.436062+0000","last_became_peered":"2024-09-19T05:02:24.436062+0000","last_unstale":"2024-09-19T05:03:08.702411+0000","last_undegraded":"2024-09-19T05:03:08.702411+0000","last_fullsized":"2024-09-19T05:03:08.702411+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:48:42.574431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.707591+0000","last_change":"2024-09-19T05:02:22.026950+0000","last_active":"2024-09-19T05:03:08.707591+0000","last_peered":"2024-09-19T05:03:08.707591+0000","last_clean":"2024-09-19T05:03:08.707591+0000","last_became_active":"2024-09-19T05:02:22.026498+0000","last_became_peered":"2024-09-19T05:02:22.026498+0000","last_unstale":"2024-09-19T05:03:08.707591+0000","last_undegraded":"2024-09-19T05:03:08.707591+0000","last_fullsized":"2024-09-19T05:03:08.707591+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:55:31.833714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.897727+0000","last_change":"2024-09-19T05:03:08.897727+0000","last_active":"2024-09-19T05:03:08.897727+0000","last_peered":"2024-09-19T05:03:08.897727+0000","last_clean":"2024-09-19T05:03:08.897727+0000","last_became_active":"2024-09-19T05:03:08.897382+0000","last_became_peered":"2024-09-19T05:03:08.897382+0000","last_unstale":"2024-09-19T05:03:08.897727+0000","last_undegraded":"2024-09-19T05:03:08.897727+0000","last_fullsized":"2024-09-19T05:03:08.897727+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:13.319291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.896688+0000","last_change":"2024-09-19T05:02:22.075901+0000","last_active":"2024-09-19T05:03:08.896688+0000","last_peered":"2024-09-19T05:03:08.896688+0000","last_clean":"2024-09-19T05:03:08.896688+0000","last_became_active":"2024-09-19T05:02:22.075512+0000","last_became_peered":"2024-09-19T05:02:22.075512+0000","last_unstale":"2024-09-19T05:03:08.896688+0000","last_undegraded":"2024-09-19T05:03:08.896688+0000","last_fullsized":"2024-09-19T05:03:08.896688+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:09:26.976888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.702172+0000","last_change":"2024-09-19T05:02:24.436797+0000","last_active":"2024-09-19T05:03:08.702172+0000","last_peered":"2024-09-19T05:03:08.702172+0000","last_clean":"2024-09-19T05:03:08.702172+0000","last_became_active":"2024-09-19T05:02:24.436457+0000","last_became_peered":"2024-09-19T05:02:24.436457+0000","last_unstale":"2024-09-19T05:03:08.702172+0000","last_undegraded":"2024-09-19T05:03:08.702172+0000","last_fullsized":"2024-09-19T05:03:08.702172+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:32:44.702601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'31","reported_seq":176,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:34.607763+0000","last_change":"2024-09-19T05:02:22.027415+0000","last_active":"2024-09-19T05:03:34.607763+0000","last_peered":"2024-09-19T05:03:34.607763+0000","last_clean":"2024-09-19T05:03:34.607763+0000","last_became_active":"2024-09-19T05:02:22.027065+0000","last_became_peered":"2024-09-19T05:02:22.027065+0000","last_unstale":"2024-09-19T05:03:34.607763+0000","last_undegraded":"2024-09-19T05:03:34.607763+0000","last_fullsized":"2024-09-19T05:03:34.607763+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:02.599310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61828,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":121,"num_read_kb":119,"num_write":33,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.705180+0000","last_change":"2024-09-19T05:02:24.419718+0000","last_active":"2024-09-19T05:03:08.705180+0000","last_peered":"2024-09-19T05:03:08.705180+0000","last_clean":"2024-09-19T05:03:08.705180+0000","last_became_active":"2024-09-19T05:02:24.419388+0000","last_became_peered":"2024-09-19T05:02:24.419388+0000","last_unstale":"2024-09-19T05:03:08.705180+0000","last_undegraded":"2024-09-19T05:03:08.705180+0000","last_fullsized":"2024-09-19T05:03:08.705180+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:09.453551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.702317+0000","last_change":"2024-09-19T05:02:22.026220+0000","last_active":"2024-09-19T05:03:08.702317+0000","last_peered":"2024-09-19T05:03:08.702317+0000","last_clean":"2024-09-19T05:03:08.702317+0000","last_became_active":"2024-09-19T05:02:22.025838+0000","last_became_peered":"2024-09-19T05:02:22.025838+0000","last_unstale":"2024-09-19T05:03:08.702317+0000","last_undegraded":"2024-09-19T05:03:08.702317+0000","last_fullsized":"2024-09-19T05:03:08.702317+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:02.192799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.704907+0000","last_change":"2024-09-19T05:02:24.431872+0000","last_active":"2024-09-19T05:03:08.704907+0000","last_peered":"2024-09-19T05:03:08.704907+0000","last_clean":"2024-09-19T05:03:08.704907+0000","last_became_active":"2024-09-19T05:02:24.427227+0000","last_became_peered":"2024-09-19T05:02:24.427227+0000","last_unstale":"2024-09-19T05:03:08.704907+0000","last_undegraded":"2024-09-19T05:03:08.704907+0000","last_fullsized":"2024-09-19T05:03:08.704907+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:52.655108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.896545+0000","last_change":"2024-09-19T05:02:22.025136+0000","last_active":"2024-09-19T05:03:08.896545+0000","last_peered":"2024-09-19T05:03:08.896545+0000","last_clean":"2024-09-19T05:03:08.896545+0000","last_became_active":"2024-09-19T05:02:22.024813+0000","last_became_peered":"2024-09-19T05:02:22.024813+0000","last_unstale":"2024-09-19T05:03:08.896545+0000","last_undegraded":"2024-09-19T05:03:08.896545+0000","last_fullsized":"2024-09-19T05:03:08.896545+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:47:03.956661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.708019+0000","last_change":"2024-09-19T05:02:24.428497+0000","last_active":"2024-09-19T05:03:08.708019+0000","last_peered":"2024-09-19T05:03:08.708019+0000","last_clean":"2024-09-19T05:03:08.708019+0000","last_became_active":"2024-09-19T05:02:24.427774+0000","last_became_peered":"2024-09-19T05:02:24.427774+0000","last_unstale":"2024-09-19T05:03:08.708019+0000","last_undegraded":"2024-09-19T05:03:08.708019+0000","last_fullsized":"2024-09-19T05:03:08.708019+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:32:02.160354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'6","reported_seq":104,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:34.749129+0000","last_change":"2024-09-19T05:03:08.718249+0000","last_active":"2024-09-19T05:03:34.749129+0000","last_peered":"2024-09-19T05:03:34.749129+0000","last_clean":"2024-09-19T05:03:34.749129+0000","last_became_active":"2024-09-19T05:03:08.713169+0000","last_became_peered":"2024-09-19T05:03:08.713169+0000","last_unstale":"2024-09-19T05:03:34.749129+0000","last_undegraded":"2024-09-19T05:03:34.749129+0000","last_fullsized":"2024-09-19T05:03:34.749129+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:34.640353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":55,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":981,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,10],"acting":[4,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.706821+0000","last_change":"2024-09-19T05:02:24.433098+0000","last_active":"2024-09-19T05:03:08.706821+0000","last_peered":"2024-09-19T05:03:08.706821+0000","last_clean":"2024-09-19T05:03:08.706821+0000","last_became_active":"2024-09-19T05:02:24.432800+0000","last_became_peered":"2024-09-19T05:02:24.432800+0000","last_unstale":"2024-09-19T05:03:08.706821+0000","last_undegraded":"2024-09-19T05:03:08.706821+0000","last_fullsized":"2024-09-19T05:03:08.706821+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:50:30.163842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.704340+0000","last_change":"2024-09-19T05:02:22.075865+0000","last_active":"2024-09-19T05:03:08.704340+0000","last_peered":"2024-09-19T05:03:08.704340+0000","last_clean":"2024-09-19T05:03:08.704340+0000","last_became_active":"2024-09-19T05:02:22.075185+0000","last_became_peered":"2024-09-19T05:02:22.075185+0000","last_unstale":"2024-09-19T05:03:08.704340+0000","last_undegraded":"2024-09-19T05:03:08.704340+0000","last_fullsized":"2024-09-19T05:03:08.704340+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:19.150155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.896357+0000","last_change":"2024-09-19T05:02:24.422759+0000","last_active":"2024-09-19T05:03:08.896357+0000","last_peered":"2024-09-19T05:03:08.896357+0000","last_clean":"2024-09-19T05:03:08.896357+0000","last_became_active":"2024-09-19T05:02:24.422421+0000","last_became_peered":"2024-09-19T05:02:24.422421+0000","last_unstale":"2024-09-19T05:03:08.896357+0000","last_undegraded":"2024-09-19T05:03:08.896357+0000","last_fullsized":"2024-09-19T05:03:08.896357+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:58:50.083017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.705327+0000","last_change":"2024-09-19T05:02:22.024305+0000","last_active":"2024-09-19T05:03:08.705327+0000","last_peered":"2024-09-19T05:03:08.705327+0000","last_clean":"2024-09-19T05:03:08.705327+0000","last_became_active":"2024-09-19T05:02:22.023616+0000","last_became_peered":"2024-09-19T05:02:22.023616+0000","last_unstale":"2024-09-19T05:03:08.705327+0000","last_undegraded":"2024-09-19T05:03:08.705327+0000","last_fullsized":"2024-09-19T05:03:08.705327+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:05.665219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.713219+0000","last_change":"2024-09-19T05:03:08.713219+0000","last_active":"2024-09-19T05:03:08.713219+0000","last_peered":"2024-09-19T05:03:08.713219+0000","last_clean":"2024-09-19T05:03:08.713219+0000","last_became_active":"2024-09-19T05:03:08.712908+0000","last_became_peered":"2024-09-19T05:03:08.712908+0000","last_unstale":"2024-09-19T05:03:08.713219+0000","last_undegraded":"2024-09-19T05:03:08.713219+0000","last_fullsized":"2024-09-19T05:03:08.713219+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:23.366098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.720385+0000","last_change":"2024-09-19T05:03:08.720385+0000","last_active":"2024-09-19T05:03:08.720385+0000","last_peered":"2024-09-19T05:03:08.720385+0000","last_clean":"2024-09-19T05:03:08.720385+0000","last_became_active":"2024-09-19T05:03:08.715635+0000","last_became_peered":"2024-09-19T05:03:08.715635+0000","last_unstale":"2024-09-19T05:03:08.720385+0000","last_undegraded":"2024-09-19T05:03:08.720385+0000","last_fullsized":"2024-09-19T05:03:08.720385+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:09:57.627667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.707970+0000","last_change":"2024-09-19T05:02:24.419360+0000","last_active":"2024-09-19T05:03:08.707970+0000","last_peered":"2024-09-19T05:03:08.707970+0000","last_clean":"2024-09-19T05:03:08.707970+0000","last_became_active":"2024-09-19T05:02:24.419040+0000","last_became_peered":"2024-09-19T05:02:24.419040+0000","last_unstale":"2024-09-19T05:03:08.707970+0000","last_undegraded":"2024-09-19T05:03:08.707970+0000","last_fullsized":"2024-09-19T05:03:08.707970+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:48:12.922676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.896749+0000","last_change":"2024-09-19T05:02:22.023124+0000","last_active":"2024-09-19T05:03:08.896749+0000","last_peered":"2024-09-19T05:03:08.896749+0000","last_clean":"2024-09-19T05:03:08.896749+0000","last_became_active":"2024-09-19T05:02:22.022830+0000","last_became_peered":"2024-09-19T05:02:22.022830+0000","last_unstale":"2024-09-19T05:03:08.896749+0000","last_undegraded":"2024-09-19T05:03:08.896749+0000","last_fullsized":"2024-09-19T05:03:08.896749+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:22:13.208616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.707468+0000","last_change":"2024-09-19T05:02:24.426169+0000","last_active":"2024-09-19T05:03:08.707468+0000","last_peered":"2024-09-19T05:03:08.707468+0000","last_clean":"2024-09-19T05:03:08.707468+0000","last_became_active":"2024-09-19T05:02:24.425816+0000","last_became_peered":"2024-09-19T05:02:24.425816+0000","last_unstale":"2024-09-19T05:03:08.707468+0000","last_undegraded":"2024-09-19T05:03:08.707468+0000","last_fullsized":"2024-09-19T05:03:08.707468+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:56:11.735351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'6","reported_seq":91,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:35.595664+0000","last_change":"2024-09-19T05:02:22.154391+0000","last_active":"2024-09-19T05:03:35.595664+0000","last_peered":"2024-09-19T05:03:35.595664+0000","last_clean":"2024-09-19T05:03:35.595664+0000","last_became_active":"2024-09-19T05:02:22.154008+0000","last_became_peered":"2024-09-19T05:02:22.154008+0000","last_unstale":"2024-09-19T05:03:35.595664+0000","last_undegraded":"2024-09-19T05:03:35.595664+0000","last_fullsized":"2024-09-19T05:03:35.595664+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:29.997649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":61,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.707331+0000","last_change":"2024-09-19T05:02:24.427416+0000","last_active":"2024-09-19T05:03:08.707331+0000","last_peered":"2024-09-19T05:03:08.707331+0000","last_clean":"2024-09-19T05:03:08.707331+0000","last_became_active":"2024-09-19T05:02:24.426645+0000","last_became_peered":"2024-09-19T05:02:24.426645+0000","last_unstale":"2024-09-19T05:03:08.707331+0000","last_undegraded":"2024-09-19T05:03:08.707331+0000","last_fullsized":"2024-09-19T05:03:08.707331+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:21.841013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":348,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.707816+0000","last_change":"2024-09-19T05:01:04.407740+0000","last_active":"2024-09-19T05:03:08.707816+0000","last_peered":"2024-09-19T05:03:08.707816+0000","last_clean":"2024-09-19T05:03:08.707816+0000","last_became_active":"2024-09-19T05:01:04.407281+0000","last_became_peered":"2024-09-19T05:01:04.407281+0000","last_unstale":"2024-09-19T05:03:08.707816+0000","last_undegraded":"2024-09-19T05:03:08.707816+0000","last_fullsized":"2024-09-19T05:03:08.707816+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:57:27.643438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:57:27.643438+0000","last_clean_scrub_stamp":"2024-09-19T04:57:27.643438+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:54:50.242926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.706019+0000","last_change":"2024-09-19T05:02:22.023991+0000","last_active":"2024-09-19T05:03:08.706019+0000","last_peered":"2024-09-19T05:03:08.706019+0000","last_clean":"2024-09-19T05:03:08.706019+0000","last_became_active":"2024-09-19T05:02:22.023361+0000","last_became_peered":"2024-09-19T05:02:22.023361+0000","last_unstale":"2024-09-19T05:03:08.706019+0000","last_undegraded":"2024-09-19T05:03:08.706019+0000","last_fullsized":"2024-09-19T05:03:08.706019+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:37:05.551318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.714133+0000","last_change":"2024-09-19T05:02:24.424473+0000","last_active":"2024-09-19T05:03:08.714133+0000","last_peered":"2024-09-19T05:03:08.714133+0000","last_clean":"2024-09-19T05:03:08.714133+0000","last_became_active":"2024-09-19T05:02:24.423968+0000","last_became_peered":"2024-09-19T05:02:24.423968+0000","last_unstale":"2024-09-19T05:03:08.714133+0000","last_undegraded":"2024-09-19T05:03:08.714133+0000","last_fullsized":"2024-09-19T05:03:08.714133+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:29:53.305036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.707708+0000","last_change":"2024-09-19T05:02:22.026278+0000","last_active":"2024-09-19T05:03:08.707708+0000","last_peered":"2024-09-19T05:03:08.707708+0000","last_clean":"2024-09-19T05:03:08.707708+0000","last_became_active":"2024-09-19T05:02:22.025759+0000","last_became_peered":"2024-09-19T05:02:22.025759+0000","last_unstale":"2024-09-19T05:03:08.707708+0000","last_undegraded":"2024-09-19T05:03:08.707708+0000","last_fullsized":"2024-09-19T05:03:08.707708+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:21:21.696382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.704478+0000","last_change":"2024-09-19T05:02:24.426880+0000","last_active":"2024-09-19T05:03:08.704478+0000","last_peered":"2024-09-19T05:03:08.704478+0000","last_clean":"2024-09-19T05:03:08.704478+0000","last_became_active":"2024-09-19T05:02:24.426336+0000","last_became_peered":"2024-09-19T05:02:24.426336+0000","last_unstale":"2024-09-19T05:03:08.704478+0000","last_undegraded":"2024-09-19T05:03:08.704478+0000","last_fullsized":"2024-09-19T05:03:08.704478+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:51:07.110738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":87,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:34.748529+0000","last_change":"2024-09-19T05:02:22.075789+0000","last_active":"2024-09-19T05:03:34.748529+0000","last_peered":"2024-09-19T05:03:34.748529+0000","last_clean":"2024-09-19T05:03:34.748529+0000","last_became_active":"2024-09-19T05:02:22.075415+0000","last_became_peered":"2024-09-19T05:02:22.075415+0000","last_unstale":"2024-09-19T05:03:34.748529+0000","last_undegraded":"2024-09-19T05:03:34.748529+0000","last_fullsized":"2024-09-19T05:03:34.748529+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:11:43.843627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":58,"num_read_kb":58,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.896317+0000","last_change":"2024-09-19T05:02:24.422661+0000","last_active":"2024-09-19T05:03:08.896317+0000","last_peered":"2024-09-19T05:03:08.896317+0000","last_clean":"2024-09-19T05:03:08.896317+0000","last_became_active":"2024-09-19T05:02:24.422313+0000","last_became_peered":"2024-09-19T05:02:24.422313+0000","last_unstale":"2024-09-19T05:03:08.896317+0000","last_undegraded":"2024-09-19T05:03:08.896317+0000","last_fullsized":"2024-09-19T05:03:08.896317+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:07:39.729388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'6","reported_seq":88,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:33.591386+0000","last_change":"2024-09-19T05:02:22.028008+0000","last_active":"2024-09-19T05:03:33.591386+0000","last_peered":"2024-09-19T05:03:33.591386+0000","last_clean":"2024-09-19T05:03:33.591386+0000","last_became_active":"2024-09-19T05:02:22.027635+0000","last_became_peered":"2024-09-19T05:02:22.027635+0000","last_unstale":"2024-09-19T05:03:33.591386+0000","last_undegraded":"2024-09-19T05:03:33.591386+0000","last_fullsized":"2024-09-19T05:03:33.591386+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:41:36.902282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.705952+0000","last_change":"2024-09-19T05:02:24.435362+0000","last_active":"2024-09-19T05:03:08.705952+0000","last_peered":"2024-09-19T05:03:08.705952+0000","last_clean":"2024-09-19T05:03:08.705952+0000","last_became_active":"2024-09-19T05:02:24.435056+0000","last_became_peered":"2024-09-19T05:02:24.435056+0000","last_unstale":"2024-09-19T05:03:08.705952+0000","last_undegraded":"2024-09-19T05:03:08.705952+0000","last_fullsized":"2024-09-19T05:03:08.705952+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:58:40.092466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.705908+0000","last_change":"2024-09-19T05:02:22.077651+0000","last_active":"2024-09-19T05:03:08.705908+0000","last_peered":"2024-09-19T05:03:08.705908+0000","last_clean":"2024-09-19T05:03:08.705908+0000","last_became_active":"2024-09-19T05:02:22.076273+0000","last_became_peered":"2024-09-19T05:02:22.076273+0000","last_unstale":"2024-09-19T05:03:08.705908+0000","last_undegraded":"2024-09-19T05:03:08.705908+0000","last_fullsized":"2024-09-19T05:03:08.705908+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:28:42.064126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.706228+0000","last_change":"2024-09-19T05:02:24.438125+0000","last_active":"2024-09-19T05:03:08.706228+0000","last_peered":"2024-09-19T05:03:08.706228+0000","last_clean":"2024-09-19T05:03:08.706228+0000","last_became_active":"2024-09-19T05:02:24.437806+0000","last_became_peered":"2024-09-19T05:02:24.437806+0000","last_unstale":"2024-09-19T05:03:08.706228+0000","last_undegraded":"2024-09-19T05:03:08.706228+0000","last_fullsized":"2024-09-19T05:03:08.706228+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:46:08.338501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.705207+0000","last_change":"2024-09-19T05:02:22.075920+0000","last_active":"2024-09-19T05:03:08.705207+0000","last_peered":"2024-09-19T05:03:08.705207+0000","last_clean":"2024-09-19T05:03:08.705207+0000","last_became_active":"2024-09-19T05:02:22.075609+0000","last_became_peered":"2024-09-19T05:02:22.075609+0000","last_unstale":"2024-09-19T05:03:08.705207+0000","last_undegraded":"2024-09-19T05:03:08.705207+0000","last_fullsized":"2024-09-19T05:03:08.705207+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:45.915625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.706035+0000","last_change":"2024-09-19T05:02:24.434681+0000","last_active":"2024-09-19T05:03:08.706035+0000","last_peered":"2024-09-19T05:03:08.706035+0000","last_clean":"2024-09-19T05:03:08.706035+0000","last_became_active":"2024-09-19T05:02:24.434257+0000","last_became_peered":"2024-09-19T05:02:24.434257+0000","last_unstale":"2024-09-19T05:03:08.706035+0000","last_undegraded":"2024-09-19T05:03:08.706035+0000","last_fullsized":"2024-09-19T05:03:08.706035+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:06:09.133112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'15","reported_seq":55,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.725679+0000","last_change":"2024-09-19T05:03:08.725679+0000","last_active":"2024-09-19T05:03:08.725679+0000","last_peered":"2024-09-19T05:03:08.725679+0000","last_clean":"2024-09-19T05:03:08.725679+0000","last_became_active":"2024-09-19T05:03:08.715651+0000","last_became_peered":"2024-09-19T05:03:08.715651+0000","last_unstale":"2024-09-19T05:03:08.725679+0000","last_undegraded":"2024-09-19T05:03:08.725679+0000","last_fullsized":"2024-09-19T05:03:08.725679+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:18:32.003686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.714968+0000","last_change":"2024-09-19T05:02:24.429807+0000","last_active":"2024-09-19T05:03:08.714968+0000","last_peered":"2024-09-19T05:03:08.714968+0000","last_clean":"2024-09-19T05:03:08.714968+0000","last_became_active":"2024-09-19T05:02:24.429477+0000","last_became_peered":"2024-09-19T05:02:24.429477+0000","last_unstale":"2024-09-19T05:03:08.714968+0000","last_undegraded":"2024-09-19T05:03:08.714968+0000","last_fullsized":"2024-09-19T05:03:08.714968+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:02.314851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.707123+0000","last_change":"2024-09-19T05:02:22.154534+0000","last_active":"2024-09-19T05:03:08.707123+0000","last_peered":"2024-09-19T05:03:08.707123+0000","last_clean":"2024-09-19T05:03:08.707123+0000","last_became_active":"2024-09-19T05:02:22.154238+0000","last_became_peered":"2024-09-19T05:02:22.154238+0000","last_unstale":"2024-09-19T05:03:08.707123+0000","last_undegraded":"2024-09-19T05:03:08.707123+0000","last_fullsized":"2024-09-19T05:03:08.707123+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:32:30.963503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.701708+0000","last_change":"2024-09-19T05:02:24.422817+0000","last_active":"2024-09-19T05:03:08.701708+0000","last_peered":"2024-09-19T05:03:08.701708+0000","last_clean":"2024-09-19T05:03:08.701708+0000","last_became_active":"2024-09-19T05:02:24.422478+0000","last_became_peered":"2024-09-19T05:02:24.422478+0000","last_unstale":"2024-09-19T05:03:08.701708+0000","last_undegraded":"2024-09-19T05:03:08.701708+0000","last_fullsized":"2024-09-19T05:03:08.701708+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:15:35.442149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.702099+0000","last_change":"2024-09-19T05:02:22.074396+0000","last_active":"2024-09-19T05:03:08.702099+0000","last_peered":"2024-09-19T05:03:08.702099+0000","last_clean":"2024-09-19T05:03:08.702099+0000","last_became_active":"2024-09-19T05:02:22.074108+0000","last_became_peered":"2024-09-19T05:02:22.074108+0000","last_unstale":"2024-09-19T05:03:08.702099+0000","last_undegraded":"2024-09-19T05:03:08.702099+0000","last_fullsized":"2024-09-19T05:03:08.702099+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:58:32.718702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.704336+0000","last_change":"2024-09-19T05:02:24.429599+0000","last_active":"2024-09-19T05:03:08.704336+0000","last_peered":"2024-09-19T05:03:08.704336+0000","last_clean":"2024-09-19T05:03:08.704336+0000","last_became_active":"2024-09-19T05:02:24.429268+0000","last_became_peered":"2024-09-19T05:02:24.429268+0000","last_unstale":"2024-09-19T05:03:08.704336+0000","last_undegraded":"2024-09-19T05:03:08.704336+0000","last_fullsized":"2024-09-19T05:03:08.704336+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:57.123516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.713357+0000","last_change":"2024-09-19T05:02:22.021900+0000","last_active":"2024-09-19T05:03:08.713357+0000","last_peered":"2024-09-19T05:03:08.713357+0000","last_clean":"2024-09-19T05:03:08.713357+0000","last_became_active":"2024-09-19T05:02:22.020576+0000","last_became_peered":"2024-09-19T05:02:22.020576+0000","last_unstale":"2024-09-19T05:03:08.713357+0000","last_undegraded":"2024-09-19T05:03:08.713357+0000","last_fullsized":"2024-09-19T05:03:08.713357+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:44:14.466475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.708358+0000","last_change":"2024-09-19T05:02:24.420362+0000","last_active":"2024-09-19T05:03:08.708358+0000","last_peered":"2024-09-19T05:03:08.708358+0000","last_clean":"2024-09-19T05:03:08.708358+0000","last_became_active":"2024-09-19T05:02:24.420047+0000","last_became_peered":"2024-09-19T05:02:24.420047+0000","last_unstale":"2024-09-19T05:03:08.708358+0000","last_undegraded":"2024-09-19T05:03:08.708358+0000","last_fullsized":"2024-09-19T05:03:08.708358+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:28:56.977946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.702013+0000","last_change":"2024-09-19T05:02:22.024565+0000","last_active":"2024-09-19T05:03:08.702013+0000","last_peered":"2024-09-19T05:03:08.702013+0000","last_clean":"2024-09-19T05:03:08.702013+0000","last_became_active":"2024-09-19T05:02:22.024246+0000","last_became_peered":"2024-09-19T05:02:22.024246+0000","last_unstale":"2024-09-19T05:03:08.702013+0000","last_undegraded":"2024-09-19T05:03:08.702013+0000","last_fullsized":"2024-09-19T05:03:08.702013+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:58:27.491975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.714394+0000","last_change":"2024-09-19T05:02:24.428046+0000","last_active":"2024-09-19T05:03:08.714394+0000","last_peered":"2024-09-19T05:03:08.714394+0000","last_clean":"2024-09-19T05:03:08.714394+0000","last_became_active":"2024-09-19T05:02:24.427724+0000","last_became_peered":"2024-09-19T05:02:24.427724+0000","last_unstale":"2024-09-19T05:03:08.714394+0000","last_undegraded":"2024-09-19T05:03:08.714394+0000","last_fullsized":"2024-09-19T05:03:08.714394+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:11:54.252876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.704539+0000","last_change":"2024-09-19T05:02:22.023054+0000","last_active":"2024-09-19T05:03:08.704539+0000","last_peered":"2024-09-19T05:03:08.704539+0000","last_clean":"2024-09-19T05:03:08.704539+0000","last_became_active":"2024-09-19T05:02:22.022700+0000","last_became_peered":"2024-09-19T05:02:22.022700+0000","last_unstale":"2024-09-19T05:03:08.704539+0000","last_undegraded":"2024-09-19T05:03:08.704539+0000","last_fullsized":"2024-09-19T05:03:08.704539+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:44:19.714938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.707641+0000","last_change":"2024-09-19T05:02:24.423045+0000","last_active":"2024-09-19T05:03:08.707641+0000","last_peered":"2024-09-19T05:03:08.707641+0000","last_clean":"2024-09-19T05:03:08.707641+0000","last_became_active":"2024-09-19T05:02:24.422719+0000","last_became_peered":"2024-09-19T05:02:24.422719+0000","last_unstale":"2024-09-19T05:03:08.707641+0000","last_undegraded":"2024-09-19T05:03:08.707641+0000","last_fullsized":"2024-09-19T05:03:08.707641+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:45.299603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.896555+0000","last_change":"2024-09-19T05:02:24.426942+0000","last_active":"2024-09-19T05:03:08.896555+0000","last_peered":"2024-09-19T05:03:08.896555+0000","last_clean":"2024-09-19T05:03:08.896555+0000","last_became_active":"2024-09-19T05:02:24.425896+0000","last_became_peered":"2024-09-19T05:02:24.425896+0000","last_unstale":"2024-09-19T05:03:08.896555+0000","last_undegraded":"2024-09-19T05:03:08.896555+0000","last_fullsized":"2024-09-19T05:03:08.896555+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:54:15.384309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.705029+0000","last_change":"2024-09-19T05:02:22.021415+0000","last_active":"2024-09-19T05:03:08.705029+0000","last_peered":"2024-09-19T05:03:08.705029+0000","last_clean":"2024-09-19T05:03:08.705029+0000","last_became_active":"2024-09-19T05:02:22.020508+0000","last_became_peered":"2024-09-19T05:02:22.020508+0000","last_unstale":"2024-09-19T05:03:08.705029+0000","last_undegraded":"2024-09-19T05:03:08.705029+0000","last_fullsized":"2024-09-19T05:03:08.705029+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:35.912386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.702168+0000","last_change":"2024-09-19T05:02:24.435538+0000","last_active":"2024-09-19T05:03:08.702168+0000","last_peered":"2024-09-19T05:03:08.702168+0000","last_clean":"2024-09-19T05:03:08.702168+0000","last_became_active":"2024-09-19T05:02:24.435209+0000","last_became_peered":"2024-09-19T05:02:24.435209+0000","last_unstale":"2024-09-19T05:03:08.702168+0000","last_undegraded":"2024-09-19T05:03:08.702168+0000","last_fullsized":"2024-09-19T05:03:08.702168+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:55:30.075113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":59,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.720605+0000","last_change":"2024-09-19T05:03:08.720605+0000","last_active":"2024-09-19T05:03:08.720605+0000","last_peered":"2024-09-19T05:03:08.720605+0000","last_clean":"2024-09-19T05:03:08.720605+0000","last_became_active":"2024-09-19T05:03:08.709207+0000","last_became_peered":"2024-09-19T05:03:08.709207+0000","last_unstale":"2024-09-19T05:03:08.720605+0000","last_undegraded":"2024-09-19T05:03:08.720605+0000","last_fullsized":"2024-09-19T05:03:08.720605+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:52:26.691476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.702464+0000","last_change":"2024-09-19T05:02:24.428958+0000","last_active":"2024-09-19T05:03:08.702464+0000","last_peered":"2024-09-19T05:03:08.702464+0000","last_clean":"2024-09-19T05:03:08.702464+0000","last_became_active":"2024-09-19T05:02:24.428672+0000","last_became_peered":"2024-09-19T05:02:24.428672+0000","last_unstale":"2024-09-19T05:03:08.702464+0000","last_undegraded":"2024-09-19T05:03:08.702464+0000","last_fullsized":"2024-09-19T05:03:08.702464+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:06:23.529706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.715004+0000","last_change":"2024-09-19T05:02:22.077034+0000","last_active":"2024-09-19T05:03:08.715004+0000","last_peered":"2024-09-19T05:03:08.715004+0000","last_clean":"2024-09-19T05:03:08.715004+0000","last_became_active":"2024-09-19T05:02:22.076182+0000","last_became_peered":"2024-09-19T05:02:22.076182+0000","last_unstale":"2024-09-19T05:03:08.715004+0000","last_undegraded":"2024-09-19T05:03:08.715004+0000","last_fullsized":"2024-09-19T05:03:08.715004+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:43:31.076464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.714845+0000","last_change":"2024-09-19T05:02:24.423765+0000","last_active":"2024-09-19T05:03:08.714845+0000","last_peered":"2024-09-19T05:03:08.714845+0000","last_clean":"2024-09-19T05:03:08.714845+0000","last_became_active":"2024-09-19T05:02:24.423445+0000","last_became_peered":"2024-09-19T05:02:24.423445+0000","last_unstale":"2024-09-19T05:03:08.714845+0000","last_undegraded":"2024-09-19T05:03:08.714845+0000","last_fullsized":"2024-09-19T05:03:08.714845+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:10:07.806697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'13","reported_seq":95,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:34.592105+0000","last_change":"2024-09-19T05:02:22.023413+0000","last_active":"2024-09-19T05:03:34.592105+0000","last_peered":"2024-09-19T05:03:34.592105+0000","last_clean":"2024-09-19T05:03:34.592105+0000","last_became_active":"2024-09-19T05:02:22.023078+0000","last_became_peered":"2024-09-19T05:02:22.023078+0000","last_unstale":"2024-09-19T05:03:34.592105+0000","last_undegraded":"2024-09-19T05:03:34.592105+0000","last_fullsized":"2024-09-19T05:03:34.592105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:11:09.185011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":58,"num_read_kb":58,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69822,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":604,"num_read_kb":379,"num_write":247,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":981,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":209466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285699,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514526,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":937984,"data_stored":454111,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743458,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1089536,"data_stored":1105987,"data_compressed":42121,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906983,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":958464,"data_stored":452667,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135917,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":954368,"data_stored":453045,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397554,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1003520,"data_stored":1045370,"data_compressed":42121,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561079,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":942080,"data_stored":452777,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790013,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1019904,"data_stored":1045243,"data_compressed":42121,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986241,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":942080,"data_stored":452638,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215175,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":958464,"data_stored":452209,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411405,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1024000,"data_stored":516131,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1003520,"data_stored":515518,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":962560,"data_stored":454116,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34683,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34683,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34683,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36330,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36330,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36330,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36330,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36330,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T05:03:40.326 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T05:03:40.722 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:41.011 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:40 smithi043 ceph-mon[30800]: pgmap v331: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:03:41.011 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:40 smithi043 ceph-mon[30800]: from='client.24839 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:03:41.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:40 smithi151 ceph-mon[35425]: pgmap v331: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:03:41.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:40 smithi151 ceph-mon[35425]: from='client.24839 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:03:41.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:40 smithi096 ceph-mon[35085]: pgmap v331: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:03:41.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:40 smithi096 ceph-mon[35085]: from='client.24839 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:03:41.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.277+0000 7fcaa1200640 1 -- 172.21.15.43:0/1008217703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca9c074590 msgr2=0x7fca9c074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:41.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.277+0000 7fcaa1200640 1 --2- 172.21.15.43:0/1008217703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca9c074590 0x7fca9c074970 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fca8c009900 tx=0x7fca8c02a480 comp rx=0 tx=0).stop 2024-09-19T05:03:41.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.277+0000 7fcaa1200640 1 -- 172.21.15.43:0/1008217703 shutdown_connections 2024-09-19T05:03:41.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.277+0000 7fcaa1200640 1 --2- 172.21.15.43:0/1008217703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca9c112b90 0x7fca9c114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:41.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.277+0000 7fcaa1200640 1 --2- 172.21.15.43:0/1008217703 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca9c074eb0 0x7fca9c112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:41.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.277+0000 7fcaa1200640 1 --2- 172.21.15.43:0/1008217703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca9c074590 0x7fca9c074970 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:41.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.277+0000 7fcaa1200640 1 -- 172.21.15.43:0/1008217703 >> 172.21.15.43:0/1008217703 conn(0x7fca9c0fd8c0 msgr2=0x7fca9c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:41.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.277+0000 7fcaa1200640 1 -- 172.21.15.43:0/1008217703 shutdown_connections 2024-09-19T05:03:41.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.277+0000 7fcaa1200640 1 -- 172.21.15.43:0/1008217703 wait complete. 2024-09-19T05:03:41.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.278+0000 7fcaa1200640 1 Processor -- start 2024-09-19T05:03:41.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.278+0000 7fcaa1200640 1 -- start start 2024-09-19T05:03:41.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.278+0000 7fcaa1200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca9c074590 0x7fca9c1a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:41.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.278+0000 7fcaa1200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca9c074eb0 0x7fca9c1a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:41.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.279+0000 7fcaa1200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca9c112b90 0x7fca9c19e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:41.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.279+0000 7fcaa1200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c1185a0 con 0x7fca9c074590 2024-09-19T05:03:41.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.279+0000 7fcaa1200640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c118420 con 0x7fca9c112b90 2024-09-19T05:03:41.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.279+0000 7fcaa1200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c118720 con 0x7fca9c074eb0 2024-09-19T05:03:41.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.279+0000 7fca99e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca9c074590 0x7fca9c1a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:41.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.279+0000 7fca99e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca9c074590 0x7fca9c1a46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37652/0 (socket says 172.21.15.43:37652) 2024-09-19T05:03:41.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.279+0000 7fca99e00640 1 -- 172.21.15.43:0/4260546328 learned_addr learned my addr 172.21.15.43:0/4260546328 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:41.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.279+0000 7fca9a800640 1 --2- 172.21.15.43:0/4260546328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca9c112b90 0x7fca9c19e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:41.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.279+0000 7fca93e00640 1 --2- 172.21.15.43:0/4260546328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca9c074eb0 0x7fca9c1a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:41.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.279+0000 7fca99e00640 1 -- 172.21.15.43:0/4260546328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca9c074eb0 msgr2=0x7fca9c1a4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:41.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.279+0000 7fca99e00640 1 --2- 172.21.15.43:0/4260546328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca9c074eb0 0x7fca9c1a4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:41.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.279+0000 7fca99e00640 1 -- 172.21.15.43:0/4260546328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca9c112b90 msgr2=0x7fca9c19e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:41.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.279+0000 7fca99e00640 1 --2- 172.21.15.43:0/4260546328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca9c112b90 0x7fca9c19e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:41.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.279+0000 7fca99e00640 1 -- 172.21.15.43:0/4260546328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca9c19efe0 con 0x7fca9c074590 2024-09-19T05:03:41.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.280+0000 7fca99e00640 1 --2- 172.21.15.43:0/4260546328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca9c074590 0x7fca9c1a46f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fca8c004770 tx=0x7fca8c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:41.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.280+0000 7fca92a00640 1 -- 172.21.15.43:0/4260546328 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca8c03f070 con 0x7fca9c074590 2024-09-19T05:03:41.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.280+0000 7fca92a00640 1 -- 172.21.15.43:0/4260546328 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca8c031ca0 con 0x7fca9c074590 2024-09-19T05:03:41.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.280+0000 7fca92a00640 1 -- 172.21.15.43:0/4260546328 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca8c035030 con 0x7fca9c074590 2024-09-19T05:03:41.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.280+0000 7fcaa1200640 1 -- 172.21.15.43:0/4260546328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca9c19f270 con 0x7fca9c074590 2024-09-19T05:03:41.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.280+0000 7fcaa1200640 1 -- 172.21.15.43:0/4260546328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca9c1ab5c0 con 0x7fca9c074590 2024-09-19T05:03:41.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.281+0000 7fcaa1200640 1 -- 172.21.15.43:0/4260546328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca9c074970 con 0x7fca9c074590 2024-09-19T05:03:41.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.283+0000 7fca92a00640 1 -- 172.21.15.43:0/4260546328 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fca8c042070 con 0x7fca9c074590 2024-09-19T05:03:41.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.284+0000 7fca92a00640 1 --2- 172.21.15.43:0/4260546328 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fca800a32d0 0x7fca800a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:41.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.285+0000 7fca92a00640 1 -- 172.21.15.43:0/4260546328 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fca8c0f0630 con 0x7fca9c074590 2024-09-19T05:03:41.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.285+0000 7fca93e00640 1 --2- 172.21.15.43:0/4260546328 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fca800a32d0 0x7fca800a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:41.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.285+0000 7fca93e00640 1 --2- 172.21.15.43:0/4260546328 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fca800a32d0 0x7fca800a5790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fca9c0ffec0 tx=0x7fca84005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:41.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.287+0000 7fca92a00640 1 -- 172.21.15.43:0/4260546328 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fca8c049030 con 0x7fca9c074590 2024-09-19T05:03:41.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.468+0000 7fcaa1200640 1 -- 172.21.15.43:0/4260546328 --> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fca9c0ff6c0 con 0x7fca800a32d0 2024-09-19T05:03:41.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.476+0000 7fca92a00640 1 -- 172.21.15.43:0/4260546328 <== mgr.14150 v2:172.21.15.43:6800/3167108925 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259022 (secure 0 0 0) 0x7fca9c0ff6c0 con 0x7fca800a32d0 2024-09-19T05:03:41.478 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:03:41.488 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-19T05:03:41.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.481+0000 7fcaa1200640 1 -- 172.21.15.43:0/4260546328 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fca800a32d0 msgr2=0x7fca800a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:41.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.481+0000 7fcaa1200640 1 --2- 172.21.15.43:0/4260546328 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fca800a32d0 0x7fca800a5790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fca9c0ffec0 tx=0x7fca84005f90 comp rx=0 tx=0).stop 2024-09-19T05:03:41.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.481+0000 7fcaa1200640 1 -- 172.21.15.43:0/4260546328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca9c074590 msgr2=0x7fca9c1a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:41.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.481+0000 7fcaa1200640 1 --2- 172.21.15.43:0/4260546328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca9c074590 0x7fca9c1a46f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fca8c004770 tx=0x7fca8c003c10 comp rx=0 tx=0).stop 2024-09-19T05:03:41.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.481+0000 7fcaa1200640 1 -- 172.21.15.43:0/4260546328 shutdown_connections 2024-09-19T05:03:41.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.481+0000 7fcaa1200640 1 --2- 172.21.15.43:0/4260546328 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fca800a32d0 0x7fca800a5790 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:41.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.481+0000 7fcaa1200640 1 --2- 172.21.15.43:0/4260546328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca9c112b90 0x7fca9c19e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:41.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.481+0000 7fcaa1200640 1 --2- 172.21.15.43:0/4260546328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca9c074eb0 0x7fca9c1a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:41.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.481+0000 7fcaa1200640 1 --2- 172.21.15.43:0/4260546328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca9c074590 0x7fca9c1a46f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:41.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.481+0000 7fcaa1200640 1 -- 172.21.15.43:0/4260546328 >> 172.21.15.43:0/4260546328 conn(0x7fca9c0fd8c0 msgr2=0x7fca9c0fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:41.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.481+0000 7fcaa1200640 1 -- 172.21.15.43:0/4260546328 shutdown_connections 2024-09-19T05:03:41.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:41.481+0000 7fcaa1200640 1 -- 172.21.15.43:0/4260546328 wait complete. 2024-09-19T05:03:42.289 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-09-19T05:03:41.158423+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660424,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":765,"num_read_kb":513,"num_write":430,"num_write_kb":3078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":1181717,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":351,"ondisk_log_size":351,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373336,"kb_used_data":11520,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568488,"statfs":{"total":1151940427776,"available":1151558131712,"internally_reserved":0,"allocated":11796480,"data_stored":7399812,"data_compressed":178482,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":125,"num_read_kb":65,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003084"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.708643+0000","last_change":"2024-09-19T05:02:24.426572+0000","last_active":"2024-09-19T05:03:08.708643+0000","last_peered":"2024-09-19T05:03:08.708643+0000","last_clean":"2024-09-19T05:03:08.708643+0000","last_became_active":"2024-09-19T05:02:24.426241+0000","last_became_peered":"2024-09-19T05:02:24.426241+0000","last_unstale":"2024-09-19T05:03:08.708643+0000","last_undegraded":"2024-09-19T05:03:08.708643+0000","last_fullsized":"2024-09-19T05:03:08.708643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:42.511495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.701611+0000","last_change":"2024-09-19T05:02:24.427263+0000","last_active":"2024-09-19T05:03:08.701611+0000","last_peered":"2024-09-19T05:03:08.701611+0000","last_clean":"2024-09-19T05:03:08.701611+0000","last_became_active":"2024-09-19T05:02:24.426928+0000","last_became_peered":"2024-09-19T05:02:24.426928+0000","last_unstale":"2024-09-19T05:03:08.701611+0000","last_undegraded":"2024-09-19T05:03:08.701611+0000","last_fullsized":"2024-09-19T05:03:08.701611+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:03.177020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.701980+0000","last_change":"2024-09-19T05:02:24.432709+0000","last_active":"2024-09-19T05:03:08.701980+0000","last_peered":"2024-09-19T05:03:08.701980+0000","last_clean":"2024-09-19T05:03:08.701980+0000","last_became_active":"2024-09-19T05:02:24.431114+0000","last_became_peered":"2024-09-19T05:02:24.431114+0000","last_unstale":"2024-09-19T05:03:08.701980+0000","last_undegraded":"2024-09-19T05:03:08.701980+0000","last_fullsized":"2024-09-19T05:03:08.701980+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:57.244919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.714711+0000","last_change":"2024-09-19T05:03:08.713389+0000","last_active":"2024-09-19T05:03:08.714711+0000","last_peered":"2024-09-19T05:03:08.714711+0000","last_clean":"2024-09-19T05:03:08.714711+0000","last_became_active":"2024-09-19T05:03:08.713058+0000","last_became_peered":"2024-09-19T05:03:08.713058+0000","last_unstale":"2024-09-19T05:03:08.714711+0000","last_undegraded":"2024-09-19T05:03:08.714711+0000","last_fullsized":"2024-09-19T05:03:08.714711+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:52:10.993733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.704408+0000","last_change":"2024-09-19T05:02:24.419992+0000","last_active":"2024-09-19T05:03:08.704408+0000","last_peered":"2024-09-19T05:03:08.704408+0000","last_clean":"2024-09-19T05:03:08.704408+0000","last_became_active":"2024-09-19T05:02:24.419643+0000","last_became_peered":"2024-09-19T05:02:24.419643+0000","last_unstale":"2024-09-19T05:03:08.704408+0000","last_undegraded":"2024-09-19T05:03:08.704408+0000","last_fullsized":"2024-09-19T05:03:08.704408+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:54:40.548216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.706084+0000","last_change":"2024-09-19T05:02:24.434106+0000","last_active":"2024-09-19T05:03:08.706084+0000","last_peered":"2024-09-19T05:03:08.706084+0000","last_clean":"2024-09-19T05:03:08.706084+0000","last_became_active":"2024-09-19T05:02:24.433124+0000","last_became_peered":"2024-09-19T05:02:24.433124+0000","last_unstale":"2024-09-19T05:03:08.706084+0000","last_undegraded":"2024-09-19T05:03:08.706084+0000","last_fullsized":"2024-09-19T05:03:08.706084+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:09:46.502506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.707062+0000","last_change":"2024-09-19T05:02:24.429640+0000","last_active":"2024-09-19T05:03:08.707062+0000","last_peered":"2024-09-19T05:03:08.707062+0000","last_clean":"2024-09-19T05:03:08.707062+0000","last_became_active":"2024-09-19T05:02:24.429326+0000","last_became_peered":"2024-09-19T05:02:24.429326+0000","last_unstale":"2024-09-19T05:03:08.707062+0000","last_undegraded":"2024-09-19T05:03:08.707062+0000","last_fullsized":"2024-09-19T05:03:08.707062+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:27.330449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.711936+0000","last_change":"2024-09-19T05:02:24.430933+0000","last_active":"2024-09-19T05:03:08.711936+0000","last_peered":"2024-09-19T05:03:08.711936+0000","last_clean":"2024-09-19T05:03:08.711936+0000","last_became_active":"2024-09-19T05:02:24.430478+0000","last_became_peered":"2024-09-19T05:02:24.430478+0000","last_unstale":"2024-09-19T05:03:08.711936+0000","last_undegraded":"2024-09-19T05:03:08.711936+0000","last_fullsized":"2024-09-19T05:03:08.711936+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:45:22.171049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.702535+0000","last_change":"2024-09-19T05:02:24.433207+0000","last_active":"2024-09-19T05:03:08.702535+0000","last_peered":"2024-09-19T05:03:08.702535+0000","last_clean":"2024-09-19T05:03:08.702535+0000","last_became_active":"2024-09-19T05:02:24.432882+0000","last_became_peered":"2024-09-19T05:02:24.432882+0000","last_unstale":"2024-09-19T05:03:08.702535+0000","last_undegraded":"2024-09-19T05:03:08.702535+0000","last_fullsized":"2024-09-19T05:03:08.702535+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:06:42.748227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.706778+0000","last_change":"2024-09-19T05:02:24.433825+0000","last_active":"2024-09-19T05:03:08.706778+0000","last_peered":"2024-09-19T05:03:08.706778+0000","last_clean":"2024-09-19T05:03:08.706778+0000","last_became_active":"2024-09-19T05:02:24.433088+0000","last_became_peered":"2024-09-19T05:02:24.433088+0000","last_unstale":"2024-09-19T05:03:08.706778+0000","last_undegraded":"2024-09-19T05:03:08.706778+0000","last_fullsized":"2024-09-19T05:03:08.706778+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:12:53.039028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.707237+0000","last_change":"2024-09-19T05:02:24.427661+0000","last_active":"2024-09-19T05:03:08.707237+0000","last_peered":"2024-09-19T05:03:08.707237+0000","last_clean":"2024-09-19T05:03:08.707237+0000","last_became_active":"2024-09-19T05:02:24.427093+0000","last_became_peered":"2024-09-19T05:02:24.427093+0000","last_unstale":"2024-09-19T05:03:08.707237+0000","last_undegraded":"2024-09-19T05:03:08.707237+0000","last_fullsized":"2024-09-19T05:03:08.707237+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:43.091041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.707657+0000","last_change":"2024-09-19T05:02:24.430820+0000","last_active":"2024-09-19T05:03:08.707657+0000","last_peered":"2024-09-19T05:03:08.707657+0000","last_clean":"2024-09-19T05:03:08.707657+0000","last_became_active":"2024-09-19T05:02:24.429945+0000","last_became_peered":"2024-09-19T05:02:24.429945+0000","last_unstale":"2024-09-19T05:03:08.707657+0000","last_undegraded":"2024-09-19T05:03:08.707657+0000","last_fullsized":"2024-09-19T05:03:08.707657+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:21:59.400725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.705892+0000","last_change":"2024-09-19T05:02:24.428695+0000","last_active":"2024-09-19T05:03:08.705892+0000","last_peered":"2024-09-19T05:03:08.705892+0000","last_clean":"2024-09-19T05:03:08.705892+0000","last_became_active":"2024-09-19T05:02:24.426385+0000","last_became_peered":"2024-09-19T05:02:24.426385+0000","last_unstale":"2024-09-19T05:03:08.705892+0000","last_undegraded":"2024-09-19T05:03:08.705892+0000","last_fullsized":"2024-09-19T05:03:08.705892+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:44:00.893313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.701499+0000","last_change":"2024-09-19T05:02:24.424935+0000","last_active":"2024-09-19T05:03:08.701499+0000","last_peered":"2024-09-19T05:03:08.701499+0000","last_clean":"2024-09-19T05:03:08.701499+0000","last_became_active":"2024-09-19T05:02:24.424219+0000","last_became_peered":"2024-09-19T05:02:24.424219+0000","last_unstale":"2024-09-19T05:03:08.701499+0000","last_undegraded":"2024-09-19T05:03:08.701499+0000","last_fullsized":"2024-09-19T05:03:08.701499+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:00.890778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.701387+0000","last_change":"2024-09-19T05:02:24.425407+0000","last_active":"2024-09-19T05:03:08.701387+0000","last_peered":"2024-09-19T05:03:08.701387+0000","last_clean":"2024-09-19T05:03:08.701387+0000","last_became_active":"2024-09-19T05:02:24.425053+0000","last_became_peered":"2024-09-19T05:02:24.425053+0000","last_unstale":"2024-09-19T05:03:08.701387+0000","last_undegraded":"2024-09-19T05:03:08.701387+0000","last_fullsized":"2024-09-19T05:03:08.701387+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:54:10.024183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.701951+0000","last_change":"2024-09-19T05:02:24.427888+0000","last_active":"2024-09-19T05:03:08.701951+0000","last_peered":"2024-09-19T05:03:08.701951+0000","last_clean":"2024-09-19T05:03:08.701951+0000","last_became_active":"2024-09-19T05:02:24.427589+0000","last_became_peered":"2024-09-19T05:02:24.427589+0000","last_unstale":"2024-09-19T05:03:08.701951+0000","last_undegraded":"2024-09-19T05:03:08.701951+0000","last_fullsized":"2024-09-19T05:03:08.701951+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:02:02.654416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.708819+0000","last_change":"2024-09-19T05:02:24.426654+0000","last_active":"2024-09-19T05:03:08.708819+0000","last_peered":"2024-09-19T05:03:08.708819+0000","last_clean":"2024-09-19T05:03:08.708819+0000","last_became_active":"2024-09-19T05:02:24.426352+0000","last_became_peered":"2024-09-19T05:02:24.426352+0000","last_unstale":"2024-09-19T05:03:08.708819+0000","last_undegraded":"2024-09-19T05:03:08.708819+0000","last_fullsized":"2024-09-19T05:03:08.708819+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:20.055222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.706137+0000","last_change":"2024-09-19T05:02:24.432178+0000","last_active":"2024-09-19T05:03:08.706137+0000","last_peered":"2024-09-19T05:03:08.706137+0000","last_clean":"2024-09-19T05:03:08.706137+0000","last_became_active":"2024-09-19T05:02:24.429058+0000","last_became_peered":"2024-09-19T05:02:24.429058+0000","last_unstale":"2024-09-19T05:03:08.706137+0000","last_undegraded":"2024-09-19T05:03:08.706137+0000","last_fullsized":"2024-09-19T05:03:08.706137+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:08.001134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.702195+0000","last_change":"2024-09-19T05:02:24.428202+0000","last_active":"2024-09-19T05:03:08.702195+0000","last_peered":"2024-09-19T05:03:08.702195+0000","last_clean":"2024-09-19T05:03:08.702195+0000","last_became_active":"2024-09-19T05:02:24.427741+0000","last_became_peered":"2024-09-19T05:02:24.427741+0000","last_unstale":"2024-09-19T05:03:08.702195+0000","last_undegraded":"2024-09-19T05:03:08.702195+0000","last_fullsized":"2024-09-19T05:03:08.702195+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:42:51.076056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.708596+0000","last_change":"2024-09-19T05:02:24.425007+0000","last_active":"2024-09-19T05:03:08.708596+0000","last_peered":"2024-09-19T05:03:08.708596+0000","last_clean":"2024-09-19T05:03:08.708596+0000","last_became_active":"2024-09-19T05:02:24.424612+0000","last_became_peered":"2024-09-19T05:02:24.424612+0000","last_unstale":"2024-09-19T05:03:08.708596+0000","last_undegraded":"2024-09-19T05:03:08.708596+0000","last_fullsized":"2024-09-19T05:03:08.708596+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:36:06.115757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.705989+0000","last_change":"2024-09-19T05:02:24.428724+0000","last_active":"2024-09-19T05:03:08.705989+0000","last_peered":"2024-09-19T05:03:08.705989+0000","last_clean":"2024-09-19T05:03:08.705989+0000","last_became_active":"2024-09-19T05:02:24.427730+0000","last_became_peered":"2024-09-19T05:02:24.427730+0000","last_unstale":"2024-09-19T05:03:08.705989+0000","last_undegraded":"2024-09-19T05:03:08.705989+0000","last_fullsized":"2024-09-19T05:03:08.705989+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:15:25.466553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.702112+0000","last_change":"2024-09-19T05:02:24.426238+0000","last_active":"2024-09-19T05:03:08.702112+0000","last_peered":"2024-09-19T05:03:08.702112+0000","last_clean":"2024-09-19T05:03:08.702112+0000","last_became_active":"2024-09-19T05:02:24.425915+0000","last_became_peered":"2024-09-19T05:02:24.425915+0000","last_unstale":"2024-09-19T05:03:08.702112+0000","last_undegraded":"2024-09-19T05:03:08.702112+0000","last_fullsized":"2024-09-19T05:03:08.702112+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:37:18.444042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.712573+0000","last_change":"2024-09-19T05:02:24.425726+0000","last_active":"2024-09-19T05:03:08.712573+0000","last_peered":"2024-09-19T05:03:08.712573+0000","last_clean":"2024-09-19T05:03:08.712573+0000","last_became_active":"2024-09-19T05:02:24.425409+0000","last_became_peered":"2024-09-19T05:02:24.425409+0000","last_unstale":"2024-09-19T05:03:08.712573+0000","last_undegraded":"2024-09-19T05:03:08.712573+0000","last_fullsized":"2024-09-19T05:03:08.712573+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:12.068304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.896269+0000","last_change":"2024-09-19T05:02:24.425267+0000","last_active":"2024-09-19T05:03:08.896269+0000","last_peered":"2024-09-19T05:03:08.896269+0000","last_clean":"2024-09-19T05:03:08.896269+0000","last_became_active":"2024-09-19T05:02:24.424933+0000","last_became_peered":"2024-09-19T05:02:24.424933+0000","last_unstale":"2024-09-19T05:03:08.896269+0000","last_undegraded":"2024-09-19T05:03:08.896269+0000","last_fullsized":"2024-09-19T05:03:08.896269+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:51:55.274870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.714845+0000","last_change":"2024-09-19T05:02:24.427909+0000","last_active":"2024-09-19T05:03:08.714845+0000","last_peered":"2024-09-19T05:03:08.714845+0000","last_clean":"2024-09-19T05:03:08.714845+0000","last_became_active":"2024-09-19T05:02:24.425515+0000","last_became_peered":"2024-09-19T05:02:24.425515+0000","last_unstale":"2024-09-19T05:03:08.714845+0000","last_undegraded":"2024-09-19T05:03:08.714845+0000","last_fullsized":"2024-09-19T05:03:08.714845+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:12:22.390754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.702279+0000","last_change":"2024-09-19T05:02:24.426201+0000","last_active":"2024-09-19T05:03:08.702279+0000","last_peered":"2024-09-19T05:03:08.702279+0000","last_clean":"2024-09-19T05:03:08.702279+0000","last_became_active":"2024-09-19T05:02:24.425890+0000","last_became_peered":"2024-09-19T05:02:24.425890+0000","last_unstale":"2024-09-19T05:03:08.702279+0000","last_undegraded":"2024-09-19T05:03:08.702279+0000","last_fullsized":"2024-09-19T05:03:08.702279+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:36.680002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.708581+0000","last_change":"2024-09-19T05:02:24.416037+0000","last_active":"2024-09-19T05:03:08.708581+0000","last_peered":"2024-09-19T05:03:08.708581+0000","last_clean":"2024-09-19T05:03:08.708581+0000","last_became_active":"2024-09-19T05:02:24.415535+0000","last_became_peered":"2024-09-19T05:02:24.415535+0000","last_unstale":"2024-09-19T05:03:08.708581+0000","last_undegraded":"2024-09-19T05:03:08.708581+0000","last_fullsized":"2024-09-19T05:03:08.708581+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:14.756410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.708063+0000","last_change":"2024-09-19T05:03:08.708063+0000","last_active":"2024-09-19T05:03:08.708063+0000","last_peered":"2024-09-19T05:03:08.708063+0000","last_clean":"2024-09-19T05:03:08.708063+0000","last_became_active":"2024-09-19T05:03:08.707474+0000","last_became_peered":"2024-09-19T05:03:08.707474+0000","last_unstale":"2024-09-19T05:03:08.708063+0000","last_undegraded":"2024-09-19T05:03:08.708063+0000","last_fullsized":"2024-09-19T05:03:08.708063+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:20:55.604156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.702331+0000","last_change":"2024-09-19T05:02:24.423776+0000","last_active":"2024-09-19T05:03:08.702331+0000","last_peered":"2024-09-19T05:03:08.702331+0000","last_clean":"2024-09-19T05:03:08.702331+0000","last_became_active":"2024-09-19T05:02:24.423446+0000","last_became_peered":"2024-09-19T05:02:24.423446+0000","last_unstale":"2024-09-19T05:03:08.702331+0000","last_undegraded":"2024-09-19T05:03:08.702331+0000","last_fullsized":"2024-09-19T05:03:08.702331+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:47:02.747049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.702005+0000","last_change":"2024-09-19T05:02:24.420534+0000","last_active":"2024-09-19T05:03:08.702005+0000","last_peered":"2024-09-19T05:03:08.702005+0000","last_clean":"2024-09-19T05:03:08.702005+0000","last_became_active":"2024-09-19T05:02:24.419980+0000","last_became_peered":"2024-09-19T05:02:24.419980+0000","last_unstale":"2024-09-19T05:03:08.702005+0000","last_undegraded":"2024-09-19T05:03:08.702005+0000","last_fullsized":"2024-09-19T05:03:08.702005+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:33:49.537258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.702152+0000","last_change":"2024-09-19T05:02:24.424859+0000","last_active":"2024-09-19T05:03:08.702152+0000","last_peered":"2024-09-19T05:03:08.702152+0000","last_clean":"2024-09-19T05:03:08.702152+0000","last_became_active":"2024-09-19T05:02:24.424514+0000","last_became_peered":"2024-09-19T05:02:24.424514+0000","last_unstale":"2024-09-19T05:03:08.702152+0000","last_undegraded":"2024-09-19T05:03:08.702152+0000","last_fullsized":"2024-09-19T05:03:08.702152+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:41:41.929578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.708322+0000","last_change":"2024-09-19T05:02:24.421756+0000","last_active":"2024-09-19T05:03:08.708322+0000","last_peered":"2024-09-19T05:03:08.708322+0000","last_clean":"2024-09-19T05:03:08.708322+0000","last_became_active":"2024-09-19T05:02:24.420910+0000","last_became_peered":"2024-09-19T05:02:24.420910+0000","last_unstale":"2024-09-19T05:03:08.708322+0000","last_undegraded":"2024-09-19T05:03:08.708322+0000","last_fullsized":"2024-09-19T05:03:08.708322+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:33.759745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.704511+0000","last_change":"2024-09-19T05:02:24.428514+0000","last_active":"2024-09-19T05:03:08.704511+0000","last_peered":"2024-09-19T05:03:08.704511+0000","last_clean":"2024-09-19T05:03:08.704511+0000","last_became_active":"2024-09-19T05:02:24.428206+0000","last_became_peered":"2024-09-19T05:02:24.428206+0000","last_unstale":"2024-09-19T05:03:08.704511+0000","last_undegraded":"2024-09-19T05:03:08.704511+0000","last_fullsized":"2024-09-19T05:03:08.704511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:33.045739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":91,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:35.595525+0000","last_change":"2024-09-19T05:02:22.026683+0000","last_active":"2024-09-19T05:03:35.595525+0000","last_peered":"2024-09-19T05:03:35.595525+0000","last_clean":"2024-09-19T05:03:35.595525+0000","last_became_active":"2024-09-19T05:02:22.026342+0000","last_became_peered":"2024-09-19T05:02:22.026342+0000","last_unstale":"2024-09-19T05:03:35.595525+0000","last_undegraded":"2024-09-19T05:03:35.595525+0000","last_fullsized":"2024-09-19T05:03:35.595525+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:01:37.462838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":61,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.896757+0000","last_change":"2024-09-19T05:02:24.421753+0000","last_active":"2024-09-19T05:03:08.896757+0000","last_peered":"2024-09-19T05:03:08.896757+0000","last_clean":"2024-09-19T05:03:08.896757+0000","last_became_active":"2024-09-19T05:02:24.421455+0000","last_became_peered":"2024-09-19T05:02:24.421455+0000","last_unstale":"2024-09-19T05:03:08.896757+0000","last_undegraded":"2024-09-19T05:03:08.896757+0000","last_fullsized":"2024-09-19T05:03:08.896757+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:06.416550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.716866+0000","last_change":"2024-09-19T05:03:08.716866+0000","last_active":"2024-09-19T05:03:08.716866+0000","last_peered":"2024-09-19T05:03:08.716866+0000","last_clean":"2024-09-19T05:03:08.716866+0000","last_became_active":"2024-09-19T05:03:08.709783+0000","last_became_peered":"2024-09-19T05:03:08.709783+0000","last_unstale":"2024-09-19T05:03:08.716866+0000","last_undegraded":"2024-09-19T05:03:08.716866+0000","last_fullsized":"2024-09-19T05:03:08.716866+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:26.103651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.704707+0000","last_change":"2024-09-19T05:02:22.026599+0000","last_active":"2024-09-19T05:03:08.704707+0000","last_peered":"2024-09-19T05:03:08.704707+0000","last_clean":"2024-09-19T05:03:08.704707+0000","last_became_active":"2024-09-19T05:02:22.024652+0000","last_became_peered":"2024-09-19T05:02:22.024652+0000","last_unstale":"2024-09-19T05:03:08.704707+0000","last_undegraded":"2024-09-19T05:03:08.704707+0000","last_fullsized":"2024-09-19T05:03:08.704707+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:14:32.508333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.711885+0000","last_change":"2024-09-19T05:02:24.419262+0000","last_active":"2024-09-19T05:03:08.711885+0000","last_peered":"2024-09-19T05:03:08.711885+0000","last_clean":"2024-09-19T05:03:08.711885+0000","last_became_active":"2024-09-19T05:02:24.418589+0000","last_became_peered":"2024-09-19T05:02:24.418589+0000","last_unstale":"2024-09-19T05:03:08.711885+0000","last_undegraded":"2024-09-19T05:03:08.711885+0000","last_fullsized":"2024-09-19T05:03:08.711885+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:27.256676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.896552+0000","last_change":"2024-09-19T05:02:22.024225+0000","last_active":"2024-09-19T05:03:08.896552+0000","last_peered":"2024-09-19T05:03:08.896552+0000","last_clean":"2024-09-19T05:03:08.896552+0000","last_became_active":"2024-09-19T05:02:22.023882+0000","last_became_peered":"2024-09-19T05:02:22.023882+0000","last_unstale":"2024-09-19T05:03:08.896552+0000","last_undegraded":"2024-09-19T05:03:08.896552+0000","last_fullsized":"2024-09-19T05:03:08.896552+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:29.619634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.705735+0000","last_change":"2024-09-19T05:02:24.436511+0000","last_active":"2024-09-19T05:03:08.705735+0000","last_peered":"2024-09-19T05:03:08.705735+0000","last_clean":"2024-09-19T05:03:08.705735+0000","last_became_active":"2024-09-19T05:02:24.435565+0000","last_became_peered":"2024-09-19T05:02:24.435565+0000","last_unstale":"2024-09-19T05:03:08.705735+0000","last_undegraded":"2024-09-19T05:03:08.705735+0000","last_fullsized":"2024-09-19T05:03:08.705735+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:16:43.779420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":150,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:39.600252+0000","last_change":"2024-09-19T05:02:22.024281+0000","last_active":"2024-09-19T05:03:39.600252+0000","last_peered":"2024-09-19T05:03:39.600252+0000","last_clean":"2024-09-19T05:03:39.600252+0000","last_became_active":"2024-09-19T05:02:22.023868+0000","last_became_peered":"2024-09-19T05:02:22.023868+0000","last_unstale":"2024-09-19T05:03:39.600252+0000","last_undegraded":"2024-09-19T05:03:39.600252+0000","last_fullsized":"2024-09-19T05:03:39.600252+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:29:47.597570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":122,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.702411+0000","last_change":"2024-09-19T05:02:24.436383+0000","last_active":"2024-09-19T05:03:08.702411+0000","last_peered":"2024-09-19T05:03:08.702411+0000","last_clean":"2024-09-19T05:03:08.702411+0000","last_became_active":"2024-09-19T05:02:24.436062+0000","last_became_peered":"2024-09-19T05:02:24.436062+0000","last_unstale":"2024-09-19T05:03:08.702411+0000","last_undegraded":"2024-09-19T05:03:08.702411+0000","last_fullsized":"2024-09-19T05:03:08.702411+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:48:42.574431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.707591+0000","last_change":"2024-09-19T05:02:22.026950+0000","last_active":"2024-09-19T05:03:08.707591+0000","last_peered":"2024-09-19T05:03:08.707591+0000","last_clean":"2024-09-19T05:03:08.707591+0000","last_became_active":"2024-09-19T05:02:22.026498+0000","last_became_peered":"2024-09-19T05:02:22.026498+0000","last_unstale":"2024-09-19T05:03:08.707591+0000","last_undegraded":"2024-09-19T05:03:08.707591+0000","last_fullsized":"2024-09-19T05:03:08.707591+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:55:31.833714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.897727+0000","last_change":"2024-09-19T05:03:08.897727+0000","last_active":"2024-09-19T05:03:08.897727+0000","last_peered":"2024-09-19T05:03:08.897727+0000","last_clean":"2024-09-19T05:03:08.897727+0000","last_became_active":"2024-09-19T05:03:08.897382+0000","last_became_peered":"2024-09-19T05:03:08.897382+0000","last_unstale":"2024-09-19T05:03:08.897727+0000","last_undegraded":"2024-09-19T05:03:08.897727+0000","last_fullsized":"2024-09-19T05:03:08.897727+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:13.319291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.896688+0000","last_change":"2024-09-19T05:02:22.075901+0000","last_active":"2024-09-19T05:03:08.896688+0000","last_peered":"2024-09-19T05:03:08.896688+0000","last_clean":"2024-09-19T05:03:08.896688+0000","last_became_active":"2024-09-19T05:02:22.075512+0000","last_became_peered":"2024-09-19T05:02:22.075512+0000","last_unstale":"2024-09-19T05:03:08.896688+0000","last_undegraded":"2024-09-19T05:03:08.896688+0000","last_fullsized":"2024-09-19T05:03:08.896688+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:09:26.976888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.702172+0000","last_change":"2024-09-19T05:02:24.436797+0000","last_active":"2024-09-19T05:03:08.702172+0000","last_peered":"2024-09-19T05:03:08.702172+0000","last_clean":"2024-09-19T05:03:08.702172+0000","last_became_active":"2024-09-19T05:02:24.436457+0000","last_became_peered":"2024-09-19T05:02:24.436457+0000","last_unstale":"2024-09-19T05:03:08.702172+0000","last_undegraded":"2024-09-19T05:03:08.702172+0000","last_fullsized":"2024-09-19T05:03:08.702172+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:32:44.702601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'31","reported_seq":186,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:39.616270+0000","last_change":"2024-09-19T05:02:22.027415+0000","last_active":"2024-09-19T05:03:39.616270+0000","last_peered":"2024-09-19T05:03:39.616270+0000","last_clean":"2024-09-19T05:03:39.616270+0000","last_became_active":"2024-09-19T05:02:22.027065+0000","last_became_peered":"2024-09-19T05:02:22.027065+0000","last_unstale":"2024-09-19T05:03:39.616270+0000","last_undegraded":"2024-09-19T05:03:39.616270+0000","last_fullsized":"2024-09-19T05:03:39.616270+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:02.599310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61828,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":131,"num_read_kb":124,"num_write":33,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.705180+0000","last_change":"2024-09-19T05:02:24.419718+0000","last_active":"2024-09-19T05:03:08.705180+0000","last_peered":"2024-09-19T05:03:08.705180+0000","last_clean":"2024-09-19T05:03:08.705180+0000","last_became_active":"2024-09-19T05:02:24.419388+0000","last_became_peered":"2024-09-19T05:02:24.419388+0000","last_unstale":"2024-09-19T05:03:08.705180+0000","last_undegraded":"2024-09-19T05:03:08.705180+0000","last_fullsized":"2024-09-19T05:03:08.705180+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:09.453551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.702317+0000","last_change":"2024-09-19T05:02:22.026220+0000","last_active":"2024-09-19T05:03:08.702317+0000","last_peered":"2024-09-19T05:03:08.702317+0000","last_clean":"2024-09-19T05:03:08.702317+0000","last_became_active":"2024-09-19T05:02:22.025838+0000","last_became_peered":"2024-09-19T05:02:22.025838+0000","last_unstale":"2024-09-19T05:03:08.702317+0000","last_undegraded":"2024-09-19T05:03:08.702317+0000","last_fullsized":"2024-09-19T05:03:08.702317+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:02.192799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.704907+0000","last_change":"2024-09-19T05:02:24.431872+0000","last_active":"2024-09-19T05:03:08.704907+0000","last_peered":"2024-09-19T05:03:08.704907+0000","last_clean":"2024-09-19T05:03:08.704907+0000","last_became_active":"2024-09-19T05:02:24.427227+0000","last_became_peered":"2024-09-19T05:02:24.427227+0000","last_unstale":"2024-09-19T05:03:08.704907+0000","last_undegraded":"2024-09-19T05:03:08.704907+0000","last_fullsized":"2024-09-19T05:03:08.704907+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:52.655108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.896545+0000","last_change":"2024-09-19T05:02:22.025136+0000","last_active":"2024-09-19T05:03:08.896545+0000","last_peered":"2024-09-19T05:03:08.896545+0000","last_clean":"2024-09-19T05:03:08.896545+0000","last_became_active":"2024-09-19T05:02:22.024813+0000","last_became_peered":"2024-09-19T05:02:22.024813+0000","last_unstale":"2024-09-19T05:03:08.896545+0000","last_undegraded":"2024-09-19T05:03:08.896545+0000","last_fullsized":"2024-09-19T05:03:08.896545+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:47:03.956661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.708019+0000","last_change":"2024-09-19T05:02:24.428497+0000","last_active":"2024-09-19T05:03:08.708019+0000","last_peered":"2024-09-19T05:03:08.708019+0000","last_clean":"2024-09-19T05:03:08.708019+0000","last_became_active":"2024-09-19T05:02:24.427774+0000","last_became_peered":"2024-09-19T05:02:24.427774+0000","last_unstale":"2024-09-19T05:03:08.708019+0000","last_undegraded":"2024-09-19T05:03:08.708019+0000","last_fullsized":"2024-09-19T05:03:08.708019+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:32:02.160354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'6","reported_seq":104,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:34.749129+0000","last_change":"2024-09-19T05:03:08.718249+0000","last_active":"2024-09-19T05:03:34.749129+0000","last_peered":"2024-09-19T05:03:34.749129+0000","last_clean":"2024-09-19T05:03:34.749129+0000","last_became_active":"2024-09-19T05:03:08.713169+0000","last_became_peered":"2024-09-19T05:03:08.713169+0000","last_unstale":"2024-09-19T05:03:34.749129+0000","last_undegraded":"2024-09-19T05:03:34.749129+0000","last_fullsized":"2024-09-19T05:03:34.749129+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:34.640353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":55,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":981,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,10],"acting":[4,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.706821+0000","last_change":"2024-09-19T05:02:24.433098+0000","last_active":"2024-09-19T05:03:08.706821+0000","last_peered":"2024-09-19T05:03:08.706821+0000","last_clean":"2024-09-19T05:03:08.706821+0000","last_became_active":"2024-09-19T05:02:24.432800+0000","last_became_peered":"2024-09-19T05:02:24.432800+0000","last_unstale":"2024-09-19T05:03:08.706821+0000","last_undegraded":"2024-09-19T05:03:08.706821+0000","last_fullsized":"2024-09-19T05:03:08.706821+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:50:30.163842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.704340+0000","last_change":"2024-09-19T05:02:22.075865+0000","last_active":"2024-09-19T05:03:08.704340+0000","last_peered":"2024-09-19T05:03:08.704340+0000","last_clean":"2024-09-19T05:03:08.704340+0000","last_became_active":"2024-09-19T05:02:22.075185+0000","last_became_peered":"2024-09-19T05:02:22.075185+0000","last_unstale":"2024-09-19T05:03:08.704340+0000","last_undegraded":"2024-09-19T05:03:08.704340+0000","last_fullsized":"2024-09-19T05:03:08.704340+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:19.150155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.896357+0000","last_change":"2024-09-19T05:02:24.422759+0000","last_active":"2024-09-19T05:03:08.896357+0000","last_peered":"2024-09-19T05:03:08.896357+0000","last_clean":"2024-09-19T05:03:08.896357+0000","last_became_active":"2024-09-19T05:02:24.422421+0000","last_became_peered":"2024-09-19T05:02:24.422421+0000","last_unstale":"2024-09-19T05:03:08.896357+0000","last_undegraded":"2024-09-19T05:03:08.896357+0000","last_fullsized":"2024-09-19T05:03:08.896357+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:58:50.083017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.705327+0000","last_change":"2024-09-19T05:02:22.024305+0000","last_active":"2024-09-19T05:03:08.705327+0000","last_peered":"2024-09-19T05:03:08.705327+0000","last_clean":"2024-09-19T05:03:08.705327+0000","last_became_active":"2024-09-19T05:02:22.023616+0000","last_became_peered":"2024-09-19T05:02:22.023616+0000","last_unstale":"2024-09-19T05:03:08.705327+0000","last_undegraded":"2024-09-19T05:03:08.705327+0000","last_fullsized":"2024-09-19T05:03:08.705327+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:05.665219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.713219+0000","last_change":"2024-09-19T05:03:08.713219+0000","last_active":"2024-09-19T05:03:08.713219+0000","last_peered":"2024-09-19T05:03:08.713219+0000","last_clean":"2024-09-19T05:03:08.713219+0000","last_became_active":"2024-09-19T05:03:08.712908+0000","last_became_peered":"2024-09-19T05:03:08.712908+0000","last_unstale":"2024-09-19T05:03:08.713219+0000","last_undegraded":"2024-09-19T05:03:08.713219+0000","last_fullsized":"2024-09-19T05:03:08.713219+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:23.366098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.720385+0000","last_change":"2024-09-19T05:03:08.720385+0000","last_active":"2024-09-19T05:03:08.720385+0000","last_peered":"2024-09-19T05:03:08.720385+0000","last_clean":"2024-09-19T05:03:08.720385+0000","last_became_active":"2024-09-19T05:03:08.715635+0000","last_became_peered":"2024-09-19T05:03:08.715635+0000","last_unstale":"2024-09-19T05:03:08.720385+0000","last_undegraded":"2024-09-19T05:03:08.720385+0000","last_fullsized":"2024-09-19T05:03:08.720385+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:09:57.627667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.707970+0000","last_change":"2024-09-19T05:02:24.419360+0000","last_active":"2024-09-19T05:03:08.707970+0000","last_peered":"2024-09-19T05:03:08.707970+0000","last_clean":"2024-09-19T05:03:08.707970+0000","last_became_active":"2024-09-19T05:02:24.419040+0000","last_became_peered":"2024-09-19T05:02:24.419040+0000","last_unstale":"2024-09-19T05:03:08.707970+0000","last_undegraded":"2024-09-19T05:03:08.707970+0000","last_fullsized":"2024-09-19T05:03:08.707970+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:48:12.922676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.896749+0000","last_change":"2024-09-19T05:02:22.023124+0000","last_active":"2024-09-19T05:03:08.896749+0000","last_peered":"2024-09-19T05:03:08.896749+0000","last_clean":"2024-09-19T05:03:08.896749+0000","last_became_active":"2024-09-19T05:02:22.022830+0000","last_became_peered":"2024-09-19T05:02:22.022830+0000","last_unstale":"2024-09-19T05:03:08.896749+0000","last_undegraded":"2024-09-19T05:03:08.896749+0000","last_fullsized":"2024-09-19T05:03:08.896749+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:22:13.208616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.707468+0000","last_change":"2024-09-19T05:02:24.426169+0000","last_active":"2024-09-19T05:03:08.707468+0000","last_peered":"2024-09-19T05:03:08.707468+0000","last_clean":"2024-09-19T05:03:08.707468+0000","last_became_active":"2024-09-19T05:02:24.425816+0000","last_became_peered":"2024-09-19T05:02:24.425816+0000","last_unstale":"2024-09-19T05:03:08.707468+0000","last_undegraded":"2024-09-19T05:03:08.707468+0000","last_fullsized":"2024-09-19T05:03:08.707468+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:56:11.735351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'6","reported_seq":91,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:35.595664+0000","last_change":"2024-09-19T05:02:22.154391+0000","last_active":"2024-09-19T05:03:35.595664+0000","last_peered":"2024-09-19T05:03:35.595664+0000","last_clean":"2024-09-19T05:03:35.595664+0000","last_became_active":"2024-09-19T05:02:22.154008+0000","last_became_peered":"2024-09-19T05:02:22.154008+0000","last_unstale":"2024-09-19T05:03:35.595664+0000","last_undegraded":"2024-09-19T05:03:35.595664+0000","last_fullsized":"2024-09-19T05:03:35.595664+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:29.997649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":61,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.707331+0000","last_change":"2024-09-19T05:02:24.427416+0000","last_active":"2024-09-19T05:03:08.707331+0000","last_peered":"2024-09-19T05:03:08.707331+0000","last_clean":"2024-09-19T05:03:08.707331+0000","last_became_active":"2024-09-19T05:02:24.426645+0000","last_became_peered":"2024-09-19T05:02:24.426645+0000","last_unstale":"2024-09-19T05:03:08.707331+0000","last_undegraded":"2024-09-19T05:03:08.707331+0000","last_fullsized":"2024-09-19T05:03:08.707331+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:21.841013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":348,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.707816+0000","last_change":"2024-09-19T05:01:04.407740+0000","last_active":"2024-09-19T05:03:08.707816+0000","last_peered":"2024-09-19T05:03:08.707816+0000","last_clean":"2024-09-19T05:03:08.707816+0000","last_became_active":"2024-09-19T05:01:04.407281+0000","last_became_peered":"2024-09-19T05:01:04.407281+0000","last_unstale":"2024-09-19T05:03:08.707816+0000","last_undegraded":"2024-09-19T05:03:08.707816+0000","last_fullsized":"2024-09-19T05:03:08.707816+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:57:27.643438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:57:27.643438+0000","last_clean_scrub_stamp":"2024-09-19T04:57:27.643438+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:54:50.242926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.706019+0000","last_change":"2024-09-19T05:02:22.023991+0000","last_active":"2024-09-19T05:03:08.706019+0000","last_peered":"2024-09-19T05:03:08.706019+0000","last_clean":"2024-09-19T05:03:08.706019+0000","last_became_active":"2024-09-19T05:02:22.023361+0000","last_became_peered":"2024-09-19T05:02:22.023361+0000","last_unstale":"2024-09-19T05:03:08.706019+0000","last_undegraded":"2024-09-19T05:03:08.706019+0000","last_fullsized":"2024-09-19T05:03:08.706019+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:37:05.551318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.714133+0000","last_change":"2024-09-19T05:02:24.424473+0000","last_active":"2024-09-19T05:03:08.714133+0000","last_peered":"2024-09-19T05:03:08.714133+0000","last_clean":"2024-09-19T05:03:08.714133+0000","last_became_active":"2024-09-19T05:02:24.423968+0000","last_became_peered":"2024-09-19T05:02:24.423968+0000","last_unstale":"2024-09-19T05:03:08.714133+0000","last_undegraded":"2024-09-19T05:03:08.714133+0000","last_fullsized":"2024-09-19T05:03:08.714133+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:29:53.305036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.707708+0000","last_change":"2024-09-19T05:02:22.026278+0000","last_active":"2024-09-19T05:03:08.707708+0000","last_peered":"2024-09-19T05:03:08.707708+0000","last_clean":"2024-09-19T05:03:08.707708+0000","last_became_active":"2024-09-19T05:02:22.025759+0000","last_became_peered":"2024-09-19T05:02:22.025759+0000","last_unstale":"2024-09-19T05:03:08.707708+0000","last_undegraded":"2024-09-19T05:03:08.707708+0000","last_fullsized":"2024-09-19T05:03:08.707708+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:21:21.696382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.704478+0000","last_change":"2024-09-19T05:02:24.426880+0000","last_active":"2024-09-19T05:03:08.704478+0000","last_peered":"2024-09-19T05:03:08.704478+0000","last_clean":"2024-09-19T05:03:08.704478+0000","last_became_active":"2024-09-19T05:02:24.426336+0000","last_became_peered":"2024-09-19T05:02:24.426336+0000","last_unstale":"2024-09-19T05:03:08.704478+0000","last_undegraded":"2024-09-19T05:03:08.704478+0000","last_fullsized":"2024-09-19T05:03:08.704478+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:51:07.110738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":87,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:34.748529+0000","last_change":"2024-09-19T05:02:22.075789+0000","last_active":"2024-09-19T05:03:34.748529+0000","last_peered":"2024-09-19T05:03:34.748529+0000","last_clean":"2024-09-19T05:03:34.748529+0000","last_became_active":"2024-09-19T05:02:22.075415+0000","last_became_peered":"2024-09-19T05:02:22.075415+0000","last_unstale":"2024-09-19T05:03:34.748529+0000","last_undegraded":"2024-09-19T05:03:34.748529+0000","last_fullsized":"2024-09-19T05:03:34.748529+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:11:43.843627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":58,"num_read_kb":58,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.896317+0000","last_change":"2024-09-19T05:02:24.422661+0000","last_active":"2024-09-19T05:03:08.896317+0000","last_peered":"2024-09-19T05:03:08.896317+0000","last_clean":"2024-09-19T05:03:08.896317+0000","last_became_active":"2024-09-19T05:02:24.422313+0000","last_became_peered":"2024-09-19T05:02:24.422313+0000","last_unstale":"2024-09-19T05:03:08.896317+0000","last_undegraded":"2024-09-19T05:03:08.896317+0000","last_fullsized":"2024-09-19T05:03:08.896317+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:07:39.729388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'6","reported_seq":93,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:38.598726+0000","last_change":"2024-09-19T05:02:22.028008+0000","last_active":"2024-09-19T05:03:38.598726+0000","last_peered":"2024-09-19T05:03:38.598726+0000","last_clean":"2024-09-19T05:03:38.598726+0000","last_became_active":"2024-09-19T05:02:22.027635+0000","last_became_peered":"2024-09-19T05:02:22.027635+0000","last_unstale":"2024-09-19T05:03:38.598726+0000","last_undegraded":"2024-09-19T05:03:38.598726+0000","last_fullsized":"2024-09-19T05:03:38.598726+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:41:36.902282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":63,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.705952+0000","last_change":"2024-09-19T05:02:24.435362+0000","last_active":"2024-09-19T05:03:08.705952+0000","last_peered":"2024-09-19T05:03:08.705952+0000","last_clean":"2024-09-19T05:03:08.705952+0000","last_became_active":"2024-09-19T05:02:24.435056+0000","last_became_peered":"2024-09-19T05:02:24.435056+0000","last_unstale":"2024-09-19T05:03:08.705952+0000","last_undegraded":"2024-09-19T05:03:08.705952+0000","last_fullsized":"2024-09-19T05:03:08.705952+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:58:40.092466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.705908+0000","last_change":"2024-09-19T05:02:22.077651+0000","last_active":"2024-09-19T05:03:08.705908+0000","last_peered":"2024-09-19T05:03:08.705908+0000","last_clean":"2024-09-19T05:03:08.705908+0000","last_became_active":"2024-09-19T05:02:22.076273+0000","last_became_peered":"2024-09-19T05:02:22.076273+0000","last_unstale":"2024-09-19T05:03:08.705908+0000","last_undegraded":"2024-09-19T05:03:08.705908+0000","last_fullsized":"2024-09-19T05:03:08.705908+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:28:42.064126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.706228+0000","last_change":"2024-09-19T05:02:24.438125+0000","last_active":"2024-09-19T05:03:08.706228+0000","last_peered":"2024-09-19T05:03:08.706228+0000","last_clean":"2024-09-19T05:03:08.706228+0000","last_became_active":"2024-09-19T05:02:24.437806+0000","last_became_peered":"2024-09-19T05:02:24.437806+0000","last_unstale":"2024-09-19T05:03:08.706228+0000","last_undegraded":"2024-09-19T05:03:08.706228+0000","last_fullsized":"2024-09-19T05:03:08.706228+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:46:08.338501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.705207+0000","last_change":"2024-09-19T05:02:22.075920+0000","last_active":"2024-09-19T05:03:08.705207+0000","last_peered":"2024-09-19T05:03:08.705207+0000","last_clean":"2024-09-19T05:03:08.705207+0000","last_became_active":"2024-09-19T05:02:22.075609+0000","last_became_peered":"2024-09-19T05:02:22.075609+0000","last_unstale":"2024-09-19T05:03:08.705207+0000","last_undegraded":"2024-09-19T05:03:08.705207+0000","last_fullsized":"2024-09-19T05:03:08.705207+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:45.915625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.706035+0000","last_change":"2024-09-19T05:02:24.434681+0000","last_active":"2024-09-19T05:03:08.706035+0000","last_peered":"2024-09-19T05:03:08.706035+0000","last_clean":"2024-09-19T05:03:08.706035+0000","last_became_active":"2024-09-19T05:02:24.434257+0000","last_became_peered":"2024-09-19T05:02:24.434257+0000","last_unstale":"2024-09-19T05:03:08.706035+0000","last_undegraded":"2024-09-19T05:03:08.706035+0000","last_fullsized":"2024-09-19T05:03:08.706035+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:06:09.133112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'15","reported_seq":55,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.725679+0000","last_change":"2024-09-19T05:03:08.725679+0000","last_active":"2024-09-19T05:03:08.725679+0000","last_peered":"2024-09-19T05:03:08.725679+0000","last_clean":"2024-09-19T05:03:08.725679+0000","last_became_active":"2024-09-19T05:03:08.715651+0000","last_became_peered":"2024-09-19T05:03:08.715651+0000","last_unstale":"2024-09-19T05:03:08.725679+0000","last_undegraded":"2024-09-19T05:03:08.725679+0000","last_fullsized":"2024-09-19T05:03:08.725679+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:18:32.003686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.714968+0000","last_change":"2024-09-19T05:02:24.429807+0000","last_active":"2024-09-19T05:03:08.714968+0000","last_peered":"2024-09-19T05:03:08.714968+0000","last_clean":"2024-09-19T05:03:08.714968+0000","last_became_active":"2024-09-19T05:02:24.429477+0000","last_became_peered":"2024-09-19T05:02:24.429477+0000","last_unstale":"2024-09-19T05:03:08.714968+0000","last_undegraded":"2024-09-19T05:03:08.714968+0000","last_fullsized":"2024-09-19T05:03:08.714968+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:02.314851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.707123+0000","last_change":"2024-09-19T05:02:22.154534+0000","last_active":"2024-09-19T05:03:08.707123+0000","last_peered":"2024-09-19T05:03:08.707123+0000","last_clean":"2024-09-19T05:03:08.707123+0000","last_became_active":"2024-09-19T05:02:22.154238+0000","last_became_peered":"2024-09-19T05:02:22.154238+0000","last_unstale":"2024-09-19T05:03:08.707123+0000","last_undegraded":"2024-09-19T05:03:08.707123+0000","last_fullsized":"2024-09-19T05:03:08.707123+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:32:30.963503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.701708+0000","last_change":"2024-09-19T05:02:24.422817+0000","last_active":"2024-09-19T05:03:08.701708+0000","last_peered":"2024-09-19T05:03:08.701708+0000","last_clean":"2024-09-19T05:03:08.701708+0000","last_became_active":"2024-09-19T05:02:24.422478+0000","last_became_peered":"2024-09-19T05:02:24.422478+0000","last_unstale":"2024-09-19T05:03:08.701708+0000","last_undegraded":"2024-09-19T05:03:08.701708+0000","last_fullsized":"2024-09-19T05:03:08.701708+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:15:35.442149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.702099+0000","last_change":"2024-09-19T05:02:22.074396+0000","last_active":"2024-09-19T05:03:08.702099+0000","last_peered":"2024-09-19T05:03:08.702099+0000","last_clean":"2024-09-19T05:03:08.702099+0000","last_became_active":"2024-09-19T05:02:22.074108+0000","last_became_peered":"2024-09-19T05:02:22.074108+0000","last_unstale":"2024-09-19T05:03:08.702099+0000","last_undegraded":"2024-09-19T05:03:08.702099+0000","last_fullsized":"2024-09-19T05:03:08.702099+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:58:32.718702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.704336+0000","last_change":"2024-09-19T05:02:24.429599+0000","last_active":"2024-09-19T05:03:08.704336+0000","last_peered":"2024-09-19T05:03:08.704336+0000","last_clean":"2024-09-19T05:03:08.704336+0000","last_became_active":"2024-09-19T05:02:24.429268+0000","last_became_peered":"2024-09-19T05:02:24.429268+0000","last_unstale":"2024-09-19T05:03:08.704336+0000","last_undegraded":"2024-09-19T05:03:08.704336+0000","last_fullsized":"2024-09-19T05:03:08.704336+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:57.123516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.713357+0000","last_change":"2024-09-19T05:02:22.021900+0000","last_active":"2024-09-19T05:03:08.713357+0000","last_peered":"2024-09-19T05:03:08.713357+0000","last_clean":"2024-09-19T05:03:08.713357+0000","last_became_active":"2024-09-19T05:02:22.020576+0000","last_became_peered":"2024-09-19T05:02:22.020576+0000","last_unstale":"2024-09-19T05:03:08.713357+0000","last_undegraded":"2024-09-19T05:03:08.713357+0000","last_fullsized":"2024-09-19T05:03:08.713357+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:44:14.466475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.708358+0000","last_change":"2024-09-19T05:02:24.420362+0000","last_active":"2024-09-19T05:03:08.708358+0000","last_peered":"2024-09-19T05:03:08.708358+0000","last_clean":"2024-09-19T05:03:08.708358+0000","last_became_active":"2024-09-19T05:02:24.420047+0000","last_became_peered":"2024-09-19T05:02:24.420047+0000","last_unstale":"2024-09-19T05:03:08.708358+0000","last_undegraded":"2024-09-19T05:03:08.708358+0000","last_fullsized":"2024-09-19T05:03:08.708358+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:28:56.977946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.702013+0000","last_change":"2024-09-19T05:02:22.024565+0000","last_active":"2024-09-19T05:03:08.702013+0000","last_peered":"2024-09-19T05:03:08.702013+0000","last_clean":"2024-09-19T05:03:08.702013+0000","last_became_active":"2024-09-19T05:02:22.024246+0000","last_became_peered":"2024-09-19T05:02:22.024246+0000","last_unstale":"2024-09-19T05:03:08.702013+0000","last_undegraded":"2024-09-19T05:03:08.702013+0000","last_fullsized":"2024-09-19T05:03:08.702013+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:58:27.491975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.714394+0000","last_change":"2024-09-19T05:02:24.428046+0000","last_active":"2024-09-19T05:03:08.714394+0000","last_peered":"2024-09-19T05:03:08.714394+0000","last_clean":"2024-09-19T05:03:08.714394+0000","last_became_active":"2024-09-19T05:02:24.427724+0000","last_became_peered":"2024-09-19T05:02:24.427724+0000","last_unstale":"2024-09-19T05:03:08.714394+0000","last_undegraded":"2024-09-19T05:03:08.714394+0000","last_fullsized":"2024-09-19T05:03:08.714394+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:11:54.252876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.704539+0000","last_change":"2024-09-19T05:02:22.023054+0000","last_active":"2024-09-19T05:03:08.704539+0000","last_peered":"2024-09-19T05:03:08.704539+0000","last_clean":"2024-09-19T05:03:08.704539+0000","last_became_active":"2024-09-19T05:02:22.022700+0000","last_became_peered":"2024-09-19T05:02:22.022700+0000","last_unstale":"2024-09-19T05:03:08.704539+0000","last_undegraded":"2024-09-19T05:03:08.704539+0000","last_fullsized":"2024-09-19T05:03:08.704539+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:44:19.714938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.707641+0000","last_change":"2024-09-19T05:02:24.423045+0000","last_active":"2024-09-19T05:03:08.707641+0000","last_peered":"2024-09-19T05:03:08.707641+0000","last_clean":"2024-09-19T05:03:08.707641+0000","last_became_active":"2024-09-19T05:02:24.422719+0000","last_became_peered":"2024-09-19T05:02:24.422719+0000","last_unstale":"2024-09-19T05:03:08.707641+0000","last_undegraded":"2024-09-19T05:03:08.707641+0000","last_fullsized":"2024-09-19T05:03:08.707641+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:45.299603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.896555+0000","last_change":"2024-09-19T05:02:24.426942+0000","last_active":"2024-09-19T05:03:08.896555+0000","last_peered":"2024-09-19T05:03:08.896555+0000","last_clean":"2024-09-19T05:03:08.896555+0000","last_became_active":"2024-09-19T05:02:24.425896+0000","last_became_peered":"2024-09-19T05:02:24.425896+0000","last_unstale":"2024-09-19T05:03:08.896555+0000","last_undegraded":"2024-09-19T05:03:08.896555+0000","last_fullsized":"2024-09-19T05:03:08.896555+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:54:15.384309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.705029+0000","last_change":"2024-09-19T05:02:22.021415+0000","last_active":"2024-09-19T05:03:08.705029+0000","last_peered":"2024-09-19T05:03:08.705029+0000","last_clean":"2024-09-19T05:03:08.705029+0000","last_became_active":"2024-09-19T05:02:22.020508+0000","last_became_peered":"2024-09-19T05:02:22.020508+0000","last_unstale":"2024-09-19T05:03:08.705029+0000","last_undegraded":"2024-09-19T05:03:08.705029+0000","last_fullsized":"2024-09-19T05:03:08.705029+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:35.912386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.702168+0000","last_change":"2024-09-19T05:02:24.435538+0000","last_active":"2024-09-19T05:03:08.702168+0000","last_peered":"2024-09-19T05:03:08.702168+0000","last_clean":"2024-09-19T05:03:08.702168+0000","last_became_active":"2024-09-19T05:02:24.435209+0000","last_became_peered":"2024-09-19T05:02:24.435209+0000","last_unstale":"2024-09-19T05:03:08.702168+0000","last_undegraded":"2024-09-19T05:03:08.702168+0000","last_fullsized":"2024-09-19T05:03:08.702168+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:55:30.075113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":59,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.720605+0000","last_change":"2024-09-19T05:03:08.720605+0000","last_active":"2024-09-19T05:03:08.720605+0000","last_peered":"2024-09-19T05:03:08.720605+0000","last_clean":"2024-09-19T05:03:08.720605+0000","last_became_active":"2024-09-19T05:03:08.709207+0000","last_became_peered":"2024-09-19T05:03:08.709207+0000","last_unstale":"2024-09-19T05:03:08.720605+0000","last_undegraded":"2024-09-19T05:03:08.720605+0000","last_fullsized":"2024-09-19T05:03:08.720605+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:52:26.691476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.702464+0000","last_change":"2024-09-19T05:02:24.428958+0000","last_active":"2024-09-19T05:03:08.702464+0000","last_peered":"2024-09-19T05:03:08.702464+0000","last_clean":"2024-09-19T05:03:08.702464+0000","last_became_active":"2024-09-19T05:02:24.428672+0000","last_became_peered":"2024-09-19T05:02:24.428672+0000","last_unstale":"2024-09-19T05:03:08.702464+0000","last_undegraded":"2024-09-19T05:03:08.702464+0000","last_fullsized":"2024-09-19T05:03:08.702464+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:06:23.529706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.715004+0000","last_change":"2024-09-19T05:02:22.077034+0000","last_active":"2024-09-19T05:03:08.715004+0000","last_peered":"2024-09-19T05:03:08.715004+0000","last_clean":"2024-09-19T05:03:08.715004+0000","last_became_active":"2024-09-19T05:02:22.076182+0000","last_became_peered":"2024-09-19T05:02:22.076182+0000","last_unstale":"2024-09-19T05:03:08.715004+0000","last_undegraded":"2024-09-19T05:03:08.715004+0000","last_fullsized":"2024-09-19T05:03:08.715004+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:43:31.076464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:08.714845+0000","last_change":"2024-09-19T05:02:24.423765+0000","last_active":"2024-09-19T05:03:08.714845+0000","last_peered":"2024-09-19T05:03:08.714845+0000","last_clean":"2024-09-19T05:03:08.714845+0000","last_became_active":"2024-09-19T05:02:24.423445+0000","last_became_peered":"2024-09-19T05:02:24.423445+0000","last_unstale":"2024-09-19T05:03:08.714845+0000","last_undegraded":"2024-09-19T05:03:08.714845+0000","last_fullsized":"2024-09-19T05:03:08.714845+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:10:07.806697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'13","reported_seq":95,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T05:03:34.592105+0000","last_change":"2024-09-19T05:02:22.023413+0000","last_active":"2024-09-19T05:03:34.592105+0000","last_peered":"2024-09-19T05:03:34.592105+0000","last_clean":"2024-09-19T05:03:34.592105+0000","last_became_active":"2024-09-19T05:02:22.023078+0000","last_became_peered":"2024-09-19T05:02:22.023078+0000","last_unstale":"2024-09-19T05:03:34.592105+0000","last_undegraded":"2024-09-19T05:03:34.592105+0000","last_fullsized":"2024-09-19T05:03:34.592105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:11:09.185011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":58,"num_read_kb":58,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69822,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":629,"num_read_kb":394,"num_write":247,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":981,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":209466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285699,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514526,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":937984,"data_stored":454111,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743459,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1089536,"data_stored":1105987,"data_compressed":42121,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906983,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":958464,"data_stored":452667,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135917,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":954368,"data_stored":453045,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397555,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1003520,"data_stored":1045370,"data_compressed":42121,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561079,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":942080,"data_stored":452777,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790013,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1019904,"data_stored":1045243,"data_compressed":42121,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986241,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":942080,"data_stored":452638,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215176,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":958464,"data_stored":452209,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411405,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1024000,"data_stored":516131,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1003520,"data_stored":515518,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":962560,"data_stored":454116,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34683,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34683,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34683,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36330,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36330,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36330,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36330,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36330,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T05:03:42.293 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T05:03:42.293 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-19T05:03:42.293 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-19T05:03:42.294 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-19T05:03:42.672 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:43.122 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:42 smithi043 ceph-mon[30800]: pgmap v332: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T05:03:43.122 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:42 smithi043 ceph-mon[30800]: from='client.14856 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:03:43.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:42 smithi151 ceph-mon[35425]: pgmap v332: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T05:03:43.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:42 smithi151 ceph-mon[35425]: from='client.14856 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:03:43.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:42 smithi096 ceph-mon[35085]: pgmap v332: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T05:03:43.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:42 smithi096 ceph-mon[35085]: from='client.14856 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:03:43.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.233+0000 7f5839000640 1 -- 172.21.15.43:0/1343954417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58341079c0 msgr2=0x7f5834107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:43.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.233+0000 7f5839000640 1 --2- 172.21.15.43:0/1343954417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58341079c0 0x7f5834107da0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5824009930 tx=0x7f582402a420 comp rx=0 tx=0).stop 2024-09-19T05:03:43.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.233+0000 7f5839000640 1 -- 172.21.15.43:0/1343954417 shutdown_connections 2024-09-19T05:03:43.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.233+0000 7f5839000640 1 --2- 172.21.15.43:0/1343954417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5834102360 0x7f583410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:43.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.233+0000 7f5839000640 1 --2- 172.21.15.43:0/1343954417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58341019c0 0x7f5834101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:43.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.233+0000 7f5839000640 1 --2- 172.21.15.43:0/1343954417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58341079c0 0x7f5834107da0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:43.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.233+0000 7f5839000640 1 -- 172.21.15.43:0/1343954417 >> 172.21.15.43:0/1343954417 conn(0x7f58340fd740 msgr2=0x7f58340ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:43.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.233+0000 7f5839000640 1 -- 172.21.15.43:0/1343954417 shutdown_connections 2024-09-19T05:03:43.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.234+0000 7f5839000640 1 -- 172.21.15.43:0/1343954417 wait complete. 2024-09-19T05:03:43.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.235+0000 7f5839000640 1 Processor -- start 2024-09-19T05:03:43.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.235+0000 7f5839000640 1 -- start start 2024-09-19T05:03:43.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.235+0000 7f5839000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58341019c0 0x7f583419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:43.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.235+0000 7f5839000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5834102360 0x7f583419a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:43.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.235+0000 7f5839000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58341079c0 0x7f583419ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:43.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.235+0000 7f5839000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5834111e10 con 0x7f58341019c0 2024-09-19T05:03:43.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.235+0000 7f5839000640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5834111c90 con 0x7f58341079c0 2024-09-19T05:03:43.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.235+0000 7f5839000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5834111f90 con 0x7f5834102360 2024-09-19T05:03:43.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.236+0000 7f5833400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58341019c0 0x7f583419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:43.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.236+0000 7f5833400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58341019c0 0x7f583419a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37684/0 (socket says 172.21.15.43:37684) 2024-09-19T05:03:43.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.236+0000 7f5833400640 1 -- 172.21.15.43:0/4211353953 learned_addr learned my addr 172.21.15.43:0/4211353953 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:43.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.236+0000 7f5832a00640 1 --2- 172.21.15.43:0/4211353953 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5834102360 0x7f583419a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:43.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.236+0000 7f5833e00640 1 --2- 172.21.15.43:0/4211353953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58341079c0 0x7f583419ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:43.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.236+0000 7f5833400640 1 -- 172.21.15.43:0/4211353953 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5834102360 msgr2=0x7f583419a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:43.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.236+0000 7f5833400640 1 --2- 172.21.15.43:0/4211353953 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5834102360 0x7f583419a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:43.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.236+0000 7f5833400640 1 -- 172.21.15.43:0/4211353953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58341079c0 msgr2=0x7f583419ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:43.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.236+0000 7f5833400640 1 --2- 172.21.15.43:0/4211353953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58341079c0 0x7f583419ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:43.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.236+0000 7f5833400640 1 -- 172.21.15.43:0/4211353953 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f583419f3f0 con 0x7f58341019c0 2024-09-19T05:03:43.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.236+0000 7f5832a00640 1 --2- 172.21.15.43:0/4211353953 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5834102360 0x7f583419a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:03:43.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.236+0000 7f5833e00640 1 --2- 172.21.15.43:0/4211353953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58341079c0 0x7f583419ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:03:43.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.237+0000 7f5833400640 1 --2- 172.21.15.43:0/4211353953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58341019c0 0x7f583419a440 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f582402a940 tx=0x7f5824003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:43.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.237+0000 7f582be00640 1 -- 172.21.15.43:0/4211353953 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582403f070 con 0x7f58341019c0 2024-09-19T05:03:43.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.237+0000 7f582be00640 1 -- 172.21.15.43:0/4211353953 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5824031c90 con 0x7f58341019c0 2024-09-19T05:03:43.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.237+0000 7f5839000640 1 -- 172.21.15.43:0/4211353953 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f583419f680 con 0x7f58341019c0 2024-09-19T05:03:43.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.237+0000 7f582be00640 1 -- 172.21.15.43:0/4211353953 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5824035030 con 0x7f58341019c0 2024-09-19T05:03:43.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.237+0000 7f5839000640 1 -- 172.21.15.43:0/4211353953 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58341a6fc0 con 0x7f58341019c0 2024-09-19T05:03:43.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.238+0000 7f5839000640 1 -- 172.21.15.43:0/4211353953 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5834103100 con 0x7f58341019c0 2024-09-19T05:03:43.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.240+0000 7f582be00640 1 -- 172.21.15.43:0/4211353953 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f5824042070 con 0x7f58341019c0 2024-09-19T05:03:43.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.241+0000 7f582be00640 1 --2- 172.21.15.43:0/4211353953 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f58180a32d0 0x7f58180a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:43.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.242+0000 7f582be00640 1 -- 172.21.15.43:0/4211353953 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f58240f00e0 con 0x7f58341019c0 2024-09-19T05:03:43.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.242+0000 7f5832a00640 1 --2- 172.21.15.43:0/4211353953 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f58180a32d0 0x7f58180a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:43.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.242+0000 7f5832a00640 1 --2- 172.21.15.43:0/4211353953 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f58180a32d0 0x7f58180a5790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f58200095f0 tx=0x7f5820005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:43.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.245+0000 7f582be00640 1 -- 172.21.15.43:0/4211353953 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f58240bc610 con 0x7f58341019c0 2024-09-19T05:03:43.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.470+0000 7f5839000640 1 -- 172.21.15.43:0/4211353953 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f58341a7190 con 0x7f58341019c0 2024-09-19T05:03:43.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.471+0000 7f582be00640 1 -- 172.21.15.43:0/4211353953 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f58240c0910 con 0x7f58341019c0 2024-09-19T05:03:43.472 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:03:43.473 INFO:teuthology.orchestra.run.smithi043.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-19T05:03:43.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.473+0000 7f5839000640 1 -- 172.21.15.43:0/4211353953 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f58180a32d0 msgr2=0x7f58180a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:43.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.473+0000 7f5839000640 1 --2- 172.21.15.43:0/4211353953 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f58180a32d0 0x7f58180a5790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f58200095f0 tx=0x7f5820005f90 comp rx=0 tx=0).stop 2024-09-19T05:03:43.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.473+0000 7f5839000640 1 -- 172.21.15.43:0/4211353953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58341019c0 msgr2=0x7f583419a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:43.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.473+0000 7f5839000640 1 --2- 172.21.15.43:0/4211353953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58341019c0 0x7f583419a440 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f582402a940 tx=0x7f5824003b20 comp rx=0 tx=0).stop 2024-09-19T05:03:43.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.473+0000 7f5839000640 1 -- 172.21.15.43:0/4211353953 shutdown_connections 2024-09-19T05:03:43.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.474+0000 7f5839000640 1 --2- 172.21.15.43:0/4211353953 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f58180a32d0 0x7f58180a5790 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:43.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.474+0000 7f5839000640 1 --2- 172.21.15.43:0/4211353953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58341079c0 0x7f583419ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:43.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.474+0000 7f5839000640 1 --2- 172.21.15.43:0/4211353953 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5834102360 0x7f583419a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:43.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.474+0000 7f5839000640 1 --2- 172.21.15.43:0/4211353953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58341019c0 0x7f583419a440 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:43.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.474+0000 7f5839000640 1 -- 172.21.15.43:0/4211353953 >> 172.21.15.43:0/4211353953 conn(0x7f58340fd740 msgr2=0x7f583410d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:43.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.474+0000 7f5839000640 1 -- 172.21.15.43:0/4211353953 shutdown_connections 2024-09-19T05:03:43.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:43.474+0000 7f5839000640 1 -- 172.21.15.43:0/4211353953 wait complete. 2024-09-19T05:03:44.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:43 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/4211353953' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T05:03:44.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:43 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/4211353953' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T05:03:44.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:43 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/4211353953' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T05:03:44.327 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-19T05:03:44.328 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:03:44.709 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:45.143 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:44 smithi043 ceph-mon[30800]: pgmap v333: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-19T05:03:45.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:44 smithi151 ceph-mon[35425]: pgmap v333: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-19T05:03:45.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:44 smithi096 ceph-mon[35085]: pgmap v333: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-19T05:03:45.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.254+0000 7fe6c6000640 1 -- 172.21.15.43:0/2465377825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6c0112a30 msgr2=0x7fe6c0114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:45.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.254+0000 7fe6c6000640 1 --2- 172.21.15.43:0/2465377825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6c0112a30 0x7fe6c0114e20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8009ca0 tx=0x7fe6b802a8c0 comp rx=0 tx=0).stop 2024-09-19T05:03:45.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.255+0000 7fe6c6000640 1 -- 172.21.15.43:0/2465377825 shutdown_connections 2024-09-19T05:03:45.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.255+0000 7fe6c6000640 1 --2- 172.21.15.43:0/2465377825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6c0112a30 0x7fe6c0114e20 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:45.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.255+0000 7fe6c6000640 1 --2- 172.21.15.43:0/2465377825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6c0077040 0x7fe6c01124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:45.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.255+0000 7fe6c6000640 1 --2- 172.21.15.43:0/2465377825 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6c0076720 0x7fe6c0076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:45.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.255+0000 7fe6c6000640 1 -- 172.21.15.43:0/2465377825 >> 172.21.15.43:0/2465377825 conn(0x7fe6c00ffa90 msgr2=0x7fe6c0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:45.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.255+0000 7fe6c6000640 1 -- 172.21.15.43:0/2465377825 shutdown_connections 2024-09-19T05:03:45.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.255+0000 7fe6c6000640 1 -- 172.21.15.43:0/2465377825 wait complete. 2024-09-19T05:03:45.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.256+0000 7fe6c6000640 1 Processor -- start 2024-09-19T05:03:45.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.256+0000 7fe6c6000640 1 -- start start 2024-09-19T05:03:45.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.256+0000 7fe6c6000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6c0076720 0x7fe6c01a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:45.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.256+0000 7fe6c6000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6c0077040 0x7fe6c01a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:45.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.256+0000 7fe6c6000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6c0112a30 0x7fe6c019e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:45.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.256+0000 7fe6c6000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c01185d0 con 0x7fe6c0077040 2024-09-19T05:03:45.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.256+0000 7fe6c6000640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c0118450 con 0x7fe6c0112a30 2024-09-19T05:03:45.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.256+0000 7fe6c6000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c0118750 con 0x7fe6c0076720 2024-09-19T05:03:45.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.256+0000 7fe6b6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6c0077040 0x7fe6c01a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:45.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.256+0000 7fe6b6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6c0077040 0x7fe6c01a4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37710/0 (socket says 172.21.15.43:37710) 2024-09-19T05:03:45.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.256+0000 7fe6b6a00640 1 -- 172.21.15.43:0/1126997878 learned_addr learned my addr 172.21.15.43:0/1126997878 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:45.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.256+0000 7fe6bf400640 1 --2- 172.21.15.43:0/1126997878 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6c0076720 0x7fe6c01a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:45.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.257+0000 7fe6bfe00640 1 --2- 172.21.15.43:0/1126997878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6c0112a30 0x7fe6c019e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:45.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.257+0000 7fe6b6a00640 1 -- 172.21.15.43:0/1126997878 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6c0076720 msgr2=0x7fe6c01a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:45.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.257+0000 7fe6b6a00640 1 --2- 172.21.15.43:0/1126997878 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6c0076720 0x7fe6c01a46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:45.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.257+0000 7fe6b6a00640 1 -- 172.21.15.43:0/1126997878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6c0112a30 msgr2=0x7fe6c019e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:45.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.257+0000 7fe6b6a00640 1 --2- 172.21.15.43:0/1126997878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6c0112a30 0x7fe6c019e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:45.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.257+0000 7fe6b6a00640 1 -- 172.21.15.43:0/1126997878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6c019efd0 con 0x7fe6c0077040 2024-09-19T05:03:45.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.257+0000 7fe6b6a00640 1 --2- 172.21.15.43:0/1126997878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6c0077040 0x7fe6c01a4c20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe6a800e9f0 tx=0x7fe6a800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:45.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.257+0000 7fe6bcc00640 1 -- 172.21.15.43:0/1126997878 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a800cd80 con 0x7fe6c0077040 2024-09-19T05:03:45.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.257+0000 7fe6bcc00640 1 -- 172.21.15.43:0/1126997878 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6a8010910 con 0x7fe6c0077040 2024-09-19T05:03:45.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.257+0000 7fe6c6000640 1 -- 172.21.15.43:0/1126997878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6c019f2c0 con 0x7fe6c0077040 2024-09-19T05:03:45.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.257+0000 7fe6bcc00640 1 -- 172.21.15.43:0/1126997878 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a8018990 con 0x7fe6c0077040 2024-09-19T05:03:45.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.258+0000 7fe6c6000640 1 -- 172.21.15.43:0/1126997878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6c01ab5b0 con 0x7fe6c0077040 2024-09-19T05:03:45.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.259+0000 7fe6c6000640 1 -- 172.21.15.43:0/1126997878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6c0076b00 con 0x7fe6c0077040 2024-09-19T05:03:45.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.260+0000 7fe6bcc00640 1 -- 172.21.15.43:0/1126997878 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fe6a8010ac0 con 0x7fe6c0077040 2024-09-19T05:03:45.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.262+0000 7fe6bcc00640 1 --2- 172.21.15.43:0/1126997878 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fe6a40a3350 0x7fe6a40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:45.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.262+0000 7fe6bcc00640 1 -- 172.21.15.43:0/1126997878 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe6a80cfcb0 con 0x7fe6c0077040 2024-09-19T05:03:45.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.262+0000 7fe6bf400640 1 --2- 172.21.15.43:0/1126997878 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fe6a40a3350 0x7fe6a40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:45.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.263+0000 7fe6bf400640 1 --2- 172.21.15.43:0/1126997878 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fe6a40a3350 0x7fe6a40a5810 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe6c019f9e0 tx=0x7fe6b0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:45.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.266+0000 7fe6bcc00640 1 -- 172.21.15.43:0/1126997878 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe6a809c1e0 con 0x7fe6c0077040 2024-09-19T05:03:45.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.488+0000 7fe6c6000640 1 -- 172.21.15.43:0/1126997878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6c010dc20 con 0x7fe6c0077040 2024-09-19T05:03:45.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.489+0000 7fe6bcc00640 1 -- 172.21.15.43:0/1126997878 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11965 (secure 0 0 0) 0x7fe6a80a04e0 con 0x7fe6c0077040 2024-09-19T05:03:45.491 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:03:45.491 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":28,"btime":"2024-09-19T05:02:40:505009+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/773801169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":773801169},{"type":"v1","addr":"172.21.15.43:6835","nonce":773801169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1799137902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1799137902},{"type":"v1","addr":"172.21.15.43:6837","nonce":1799137902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:02:25.463041+0000","modified":"2024-09-19T05:02:39.530912+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24518,"mds_2":24511,"mds_3":24506,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14541":{"gid":14541,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/794881135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":794881135},{"type":"v1","addr":"172.21.15.43:6841","nonce":794881135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/2836453158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2836453158},{"type":"v1","addr":"172.21.15.151:6833","nonce":2836453158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/178785812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":178785812},{"type":"v1","addr":"172.21.15.96:6833","nonce":178785812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1671043820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1671043820},{"type":"v1","addr":"172.21.15.96:6835","nonce":1671043820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2972514328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2972514328},{"type":"v1","addr":"172.21.15.151:6835","nonce":2972514328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.151:6837/3960378010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3960378010},{"type":"v1","addr":"172.21.15.151:6837","nonce":3960378010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6837/3978388286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3978388286},{"type":"v1","addr":"172.21.15.96:6837","nonce":3978388286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.151:6839/3794104313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3794104313},{"type":"v1","addr":"172.21.15.151:6839","nonce":3794104313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.43:6839/3404555030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3404555030},{"type":"v1","addr":"172.21.15.43:6839","nonce":3404555030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/605527621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":605527621},{"type":"v1","addr":"172.21.15.96:6839","nonce":605527621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24493,24506,24511,24518,24542]},"id":1}]} 2024-09-19T05:03:45.491 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 28 2024-09-19T05:03:45.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.493+0000 7fe6c6000640 1 -- 172.21.15.43:0/1126997878 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fe6a40a3350 msgr2=0x7fe6a40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:45.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.493+0000 7fe6c6000640 1 --2- 172.21.15.43:0/1126997878 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fe6a40a3350 0x7fe6a40a5810 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe6c019f9e0 tx=0x7fe6b0005f90 comp rx=0 tx=0).stop 2024-09-19T05:03:45.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.493+0000 7fe6c6000640 1 -- 172.21.15.43:0/1126997878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6c0077040 msgr2=0x7fe6c01a4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:45.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.493+0000 7fe6c6000640 1 --2- 172.21.15.43:0/1126997878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6c0077040 0x7fe6c01a4c20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe6a800e9f0 tx=0x7fe6a800eeb0 comp rx=0 tx=0).stop 2024-09-19T05:03:45.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.493+0000 7fe6c6000640 1 -- 172.21.15.43:0/1126997878 shutdown_connections 2024-09-19T05:03:45.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.493+0000 7fe6c6000640 1 --2- 172.21.15.43:0/1126997878 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fe6a40a3350 0x7fe6a40a5810 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:45.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.493+0000 7fe6c6000640 1 --2- 172.21.15.43:0/1126997878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6c0112a30 0x7fe6c019e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:45.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.493+0000 7fe6c6000640 1 --2- 172.21.15.43:0/1126997878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6c0077040 0x7fe6c01a4c20 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:45.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.493+0000 7fe6c6000640 1 --2- 172.21.15.43:0/1126997878 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6c0076720 0x7fe6c01a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:45.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.493+0000 7fe6c6000640 1 -- 172.21.15.43:0/1126997878 >> 172.21.15.43:0/1126997878 conn(0x7fe6c00ffa90 msgr2=0x7fe6c0101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:45.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.493+0000 7fe6c6000640 1 -- 172.21.15.43:0/1126997878 shutdown_connections 2024-09-19T05:03:45.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:45.493+0000 7fe6c6000640 1 -- 172.21.15.43:0/1126997878 wait complete. 2024-09-19T05:03:46.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:45 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/1126997878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:03:46.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:45 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/1126997878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:03:46.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:45 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/1126997878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:03:46.345 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:03:46.722 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:47.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:46 smithi151 ceph-mon[35425]: pgmap v334: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:03:47.171 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:46 smithi043 ceph-mon[30800]: pgmap v334: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:03:47.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:46 smithi096 ceph-mon[35085]: pgmap v334: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:03:47.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.282+0000 7f9dbbc00640 1 -- 172.21.15.43:0/2634281930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9db41049a0 msgr2=0x7f9db41089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:47.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.282+0000 7f9dbbc00640 1 --2- 172.21.15.43:0/2634281930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9db41049a0 0x7f9db41089f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9da0009900 tx=0x7f9da002a410 comp rx=0 tx=0).stop 2024-09-19T05:03:47.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.282+0000 7f9dbbc00640 1 -- 172.21.15.43:0/2634281930 shutdown_connections 2024-09-19T05:03:47.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.282+0000 7f9dbbc00640 1 --2- 172.21.15.43:0/2634281930 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9db4109120 0x7f9db4110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:47.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.282+0000 7f9dbbc00640 1 --2- 172.21.15.43:0/2634281930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9db41049a0 0x7f9db41089f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:47.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.282+0000 7f9dbbc00640 1 --2- 172.21.15.43:0/2634281930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9db4103ff0 0x7f9db41043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:47.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.282+0000 7f9dbbc00640 1 -- 172.21.15.43:0/2634281930 >> 172.21.15.43:0/2634281930 conn(0x7f9db40ffa90 msgr2=0x7f9db4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:47.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.282+0000 7f9dbbc00640 1 -- 172.21.15.43:0/2634281930 shutdown_connections 2024-09-19T05:03:47.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.282+0000 7f9dbbc00640 1 -- 172.21.15.43:0/2634281930 wait complete. 2024-09-19T05:03:47.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.283+0000 7f9dbbc00640 1 Processor -- start 2024-09-19T05:03:47.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.283+0000 7f9dbbc00640 1 -- start start 2024-09-19T05:03:47.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.284+0000 7f9dbbc00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9db4103ff0 0x7f9db419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:47.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.284+0000 7f9dbbc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9db41049a0 0x7f9db419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:47.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.284+0000 7f9dbbc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9db4109120 0x7f9db419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:47.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.284+0000 7f9dbbc00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9db4114190 con 0x7f9db4109120 2024-09-19T05:03:47.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.284+0000 7f9dbbc00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9db4114010 con 0x7f9db4103ff0 2024-09-19T05:03:47.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.284+0000 7f9dbbc00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9db4114310 con 0x7f9db41049a0 2024-09-19T05:03:47.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.284+0000 7f9db9c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9db4109120 0x7f9db419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:47.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.284+0000 7f9db9200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9db4103ff0 0x7f9db419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:47.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.284+0000 7f9db9c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9db4109120 0x7f9db419eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55678/0 (socket says 172.21.15.43:55678) 2024-09-19T05:03:47.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.284+0000 7f9db9c00640 1 -- 172.21.15.43:0/4158756788 learned_addr learned my addr 172.21.15.43:0/4158756788 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:47.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.284+0000 7f9db3e00640 1 --2- 172.21.15.43:0/4158756788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9db41049a0 0x7f9db419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:47.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.285+0000 7f9db9200640 1 -- 172.21.15.43:0/4158756788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9db41049a0 msgr2=0x7f9db419ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:47.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.285+0000 7f9db9200640 1 --2- 172.21.15.43:0/4158756788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9db41049a0 0x7f9db419ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:47.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.285+0000 7f9db9200640 1 -- 172.21.15.43:0/4158756788 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9db4109120 msgr2=0x7f9db419eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:47.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.285+0000 7f9db9200640 1 --2- 172.21.15.43:0/4158756788 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9db4109120 0x7f9db419eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:47.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.285+0000 7f9db9200640 1 -- 172.21.15.43:0/4158756788 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9db419f5a0 con 0x7f9db4103ff0 2024-09-19T05:03:47.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.285+0000 7f9db9200640 1 --2- 172.21.15.43:0/4158756788 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9db4103ff0 0x7f9db419a5f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9da800e9f0 tx=0x7f9da800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:47.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.285+0000 7f9db1600640 1 -- 172.21.15.43:0/4158756788 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da800ccc0 con 0x7f9db4103ff0 2024-09-19T05:03:47.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.285+0000 7f9dbbc00640 1 -- 172.21.15.43:0/4158756788 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9db419f890 con 0x7f9db4103ff0 2024-09-19T05:03:47.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.285+0000 7f9db1600640 1 -- 172.21.15.43:0/4158756788 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9da8010910 con 0x7f9db4103ff0 2024-09-19T05:03:47.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.285+0000 7f9db1600640 1 -- 172.21.15.43:0/4158756788 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da8018960 con 0x7f9db4103ff0 2024-09-19T05:03:47.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.285+0000 7f9dbbc00640 1 -- 172.21.15.43:0/4158756788 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9db41a7170 con 0x7f9db4103ff0 2024-09-19T05:03:47.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.286+0000 7f9dbbc00640 1 -- 172.21.15.43:0/4158756788 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9db41043d0 con 0x7f9db4103ff0 2024-09-19T05:03:47.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.288+0000 7f9db1600640 1 -- 172.21.15.43:0/4158756788 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f9da8010ac0 con 0x7f9db4103ff0 2024-09-19T05:03:47.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.290+0000 7f9db1600640 1 --2- 172.21.15.43:0/4158756788 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f9d9c0a32d0 0x7f9d9c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:47.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.290+0000 7f9db1600640 1 -- 172.21.15.43:0/4158756788 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f9da80d0330 con 0x7f9db4103ff0 2024-09-19T05:03:47.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.290+0000 7f9db3e00640 1 --2- 172.21.15.43:0/4158756788 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f9d9c0a32d0 0x7f9d9c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:47.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.290+0000 7f9db3e00640 1 --2- 172.21.15.43:0/4158756788 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f9d9c0a32d0 0x7f9d9c0a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9da0000c00 tx=0x7f9da0005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:47.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.292+0000 7f9db1600640 1 -- 172.21.15.43:0/4158756788 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9da809c860 con 0x7f9db4103ff0 2024-09-19T05:03:47.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.513+0000 7f9dbbc00640 1 -- 172.21.15.43:0/4158756788 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9db41a7340 con 0x7f9db4103ff0 2024-09-19T05:03:47.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.514+0000 7f9db1600640 1 -- 172.21.15.43:0/4158756788 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11965 (secure 0 0 0) 0x7f9da80a0b60 con 0x7f9db4103ff0 2024-09-19T05:03:47.516 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:03:47.516 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":28,"btime":"2024-09-19T05:02:40:505009+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/773801169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":773801169},{"type":"v1","addr":"172.21.15.43:6835","nonce":773801169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1799137902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1799137902},{"type":"v1","addr":"172.21.15.43:6837","nonce":1799137902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:02:25.463041+0000","modified":"2024-09-19T05:02:39.530912+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24518,"mds_2":24511,"mds_3":24506,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14541":{"gid":14541,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/794881135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":794881135},{"type":"v1","addr":"172.21.15.43:6841","nonce":794881135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/2836453158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2836453158},{"type":"v1","addr":"172.21.15.151:6833","nonce":2836453158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/178785812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":178785812},{"type":"v1","addr":"172.21.15.96:6833","nonce":178785812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1671043820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1671043820},{"type":"v1","addr":"172.21.15.96:6835","nonce":1671043820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2972514328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2972514328},{"type":"v1","addr":"172.21.15.151:6835","nonce":2972514328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.151:6837/3960378010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3960378010},{"type":"v1","addr":"172.21.15.151:6837","nonce":3960378010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6837/3978388286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3978388286},{"type":"v1","addr":"172.21.15.96:6837","nonce":3978388286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.151:6839/3794104313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3794104313},{"type":"v1","addr":"172.21.15.151:6839","nonce":3794104313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.43:6839/3404555030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3404555030},{"type":"v1","addr":"172.21.15.43:6839","nonce":3404555030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/605527621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":605527621},{"type":"v1","addr":"172.21.15.96:6839","nonce":605527621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24493,24506,24511,24518,24542]},"id":1}]} 2024-09-19T05:03:47.516 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 28 2024-09-19T05:03:47.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.517+0000 7f9dbbc00640 1 -- 172.21.15.43:0/4158756788 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f9d9c0a32d0 msgr2=0x7f9d9c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:47.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.517+0000 7f9dbbc00640 1 --2- 172.21.15.43:0/4158756788 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f9d9c0a32d0 0x7f9d9c0a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9da0000c00 tx=0x7f9da0005b50 comp rx=0 tx=0).stop 2024-09-19T05:03:47.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.517+0000 7f9dbbc00640 1 -- 172.21.15.43:0/4158756788 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9db4103ff0 msgr2=0x7f9db419a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:47.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.517+0000 7f9dbbc00640 1 --2- 172.21.15.43:0/4158756788 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9db4103ff0 0x7f9db419a5f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9da800e9f0 tx=0x7f9da800eeb0 comp rx=0 tx=0).stop 2024-09-19T05:03:47.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.517+0000 7f9dbbc00640 1 -- 172.21.15.43:0/4158756788 shutdown_connections 2024-09-19T05:03:47.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.517+0000 7f9dbbc00640 1 --2- 172.21.15.43:0/4158756788 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f9d9c0a32d0 0x7f9d9c0a5790 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:47.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.517+0000 7f9dbbc00640 1 --2- 172.21.15.43:0/4158756788 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9db4109120 0x7f9db419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:47.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.517+0000 7f9dbbc00640 1 --2- 172.21.15.43:0/4158756788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9db41049a0 0x7f9db419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:47.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.517+0000 7f9dbbc00640 1 --2- 172.21.15.43:0/4158756788 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9db4103ff0 0x7f9db419a5f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:47.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.517+0000 7f9dbbc00640 1 -- 172.21.15.43:0/4158756788 >> 172.21.15.43:0/4158756788 conn(0x7f9db40ffa90 msgr2=0x7f9db4101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:47.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.517+0000 7f9dbbc00640 1 -- 172.21.15.43:0/4158756788 shutdown_connections 2024-09-19T05:03:47.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:47.517+0000 7f9dbbc00640 1 -- 172.21.15.43:0/4158756788 wait complete. 2024-09-19T05:03:48.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:47 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/4158756788' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:03:48.193 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:47 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/4158756788' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:03:48.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:47 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/4158756788' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:03:48.234 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T05:03:48.611 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:49.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:48 smithi043 ceph-mon[30800]: pgmap v335: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T05:03:49.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:48 smithi151 ceph-mon[35425]: pgmap v335: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T05:03:49.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.184+0000 7f09ab800640 1 -- 172.21.15.43:0/2064603947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09a4074620 msgr2=0x7f09a4074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:49.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.184+0000 7f09ab800640 1 --2- 172.21.15.43:0/2064603947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09a4074620 0x7f09a4074a00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0994009900 tx=0x7f099402a480 comp rx=0 tx=0).stop 2024-09-19T05:03:49.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.184+0000 7f09ab800640 1 -- 172.21.15.43:0/2064603947 shutdown_connections 2024-09-19T05:03:49.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.184+0000 7f09ab800640 1 --2- 172.21.15.43:0/2064603947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09a410e3c0 0x7f09a4110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:49.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.184+0000 7f09ab800640 1 --2- 172.21.15.43:0/2064603947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09a4074f40 0x7f09a410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:49.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.184+0000 7f09ab800640 1 --2- 172.21.15.43:0/2064603947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09a4074620 0x7f09a4074a00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:49.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.184+0000 7f09ab800640 1 -- 172.21.15.43:0/2064603947 >> 172.21.15.43:0/2064603947 conn(0x7f09a40fd8c0 msgr2=0x7f09a40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:49.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.184+0000 7f09ab800640 1 -- 172.21.15.43:0/2064603947 shutdown_connections 2024-09-19T05:03:49.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.184+0000 7f09ab800640 1 -- 172.21.15.43:0/2064603947 wait complete. 2024-09-19T05:03:49.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.185+0000 7f09ab800640 1 Processor -- start 2024-09-19T05:03:49.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.185+0000 7f09ab800640 1 -- start start 2024-09-19T05:03:49.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.185+0000 7f09ab800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09a4074620 0x7f09a419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:49.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.185+0000 7f09ab800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09a4074f40 0x7f09a419ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:49.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.186+0000 7f09ab800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09a410e3c0 0x7f09a419efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:49.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.186+0000 7f09ab800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f09a4114170 con 0x7f09a410e3c0 2024-09-19T05:03:49.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.186+0000 7f09ab800640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f09a4113ff0 con 0x7f09a4074f40 2024-09-19T05:03:49.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.186+0000 7f09ab800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f09a41142f0 con 0x7f09a4074620 2024-09-19T05:03:49.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.186+0000 7f09a9800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09a410e3c0 0x7f09a419efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:49.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.186+0000 7f09a9800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09a410e3c0 0x7f09a419efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55688/0 (socket says 172.21.15.43:55688) 2024-09-19T05:03:49.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.186+0000 7f09a9800640 1 -- 172.21.15.43:0/3874140990 learned_addr learned my addr 172.21.15.43:0/3874140990 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:49.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.186+0000 7f099be00640 1 --2- 172.21.15.43:0/3874140990 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09a4074f40 0x7f09a419ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:49.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.186+0000 7f09a8e00640 1 --2- 172.21.15.43:0/3874140990 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09a4074620 0x7f09a419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:49.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.186+0000 7f09a9800640 1 -- 172.21.15.43:0/3874140990 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09a4074620 msgr2=0x7f09a419a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:49.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.186+0000 7f09a9800640 1 --2- 172.21.15.43:0/3874140990 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09a4074620 0x7f09a419a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:49.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.186+0000 7f09a9800640 1 -- 172.21.15.43:0/3874140990 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09a4074f40 msgr2=0x7f09a419ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:49.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.186+0000 7f09a9800640 1 --2- 172.21.15.43:0/3874140990 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09a4074f40 0x7f09a419ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:49.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.186+0000 7f09a9800640 1 -- 172.21.15.43:0/3874140990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a419f6c0 con 0x7f09a410e3c0 2024-09-19T05:03:49.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.186+0000 7f099be00640 1 --2- 172.21.15.43:0/3874140990 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09a4074f40 0x7f09a419ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:03:49.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.187+0000 7f09a9800640 1 --2- 172.21.15.43:0/3874140990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09a410e3c0 0x7f09a419efe0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f099c00eb10 tx=0x7f099c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:49.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.187+0000 7f09a2000640 1 -- 172.21.15.43:0/3874140990 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c004b20 con 0x7f09a410e3c0 2024-09-19T05:03:49.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.187+0000 7f09ab800640 1 -- 172.21.15.43:0/3874140990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09a419f9b0 con 0x7f09a410e3c0 2024-09-19T05:03:49.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.187+0000 7f09a2000640 1 -- 172.21.15.43:0/3874140990 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f099c004cc0 con 0x7f09a410e3c0 2024-09-19T05:03:49.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.187+0000 7f09a2000640 1 -- 172.21.15.43:0/3874140990 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c010760 con 0x7f09a410e3c0 2024-09-19T05:03:49.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.187+0000 7f09ab800640 1 -- 172.21.15.43:0/3874140990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09a41a7290 con 0x7f09a410e3c0 2024-09-19T05:03:49.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.188+0000 7f09ab800640 1 -- 172.21.15.43:0/3874140990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09a4074a00 con 0x7f09a410e3c0 2024-09-19T05:03:49.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.190+0000 7f09a2000640 1 -- 172.21.15.43:0/3874140990 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f099c005020 con 0x7f09a410e3c0 2024-09-19T05:03:49.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.192+0000 7f09a2000640 1 --2- 172.21.15.43:0/3874140990 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f09880a3350 0x7f09880a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:49.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.192+0000 7f09a2000640 1 -- 172.21.15.43:0/3874140990 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f099c0d0050 con 0x7f09a410e3c0 2024-09-19T05:03:49.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.193+0000 7f09a8e00640 1 --2- 172.21.15.43:0/3874140990 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f09880a3350 0x7f09880a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:49.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.193+0000 7f09a8e00640 1 --2- 172.21.15.43:0/3874140990 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f09880a3350 0x7f09880a5810 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0994009230 tx=0x7f09940058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:49.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.197+0000 7f09a2000640 1 -- 172.21.15.43:0/3874140990 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f099c09cc10 con 0x7f09a410e3c0 2024-09-19T05:03:49.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:48 smithi096 ceph-mon[35085]: pgmap v335: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T05:03:49.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.380+0000 7f09ab800640 1 -- 172.21.15.43:0/3874140990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f09a4108fb0 con 0x7f09a410e3c0 2024-09-19T05:03:49.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.382+0000 7f09a2000640 1 -- 172.21.15.43:0/3874140990 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22661 (secure 0 0 0) 0x7f099c09c510 con 0x7f09a410e3c0 2024-09-19T05:03:49.384 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:03:49.384 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":85,"fsid":"1456a810-7643-11ef-bceb-c7b262605968","created":"2024-09-19T04:54:14.706644+0000","modified":"2024-09-19T05:03:08.697712+0000","last_up_change":"2024-09-19T05:01:27.348740+0000","last_in_change":"2024-09-19T05:01:03.396477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:57:27.642705+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:02:20.685123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:02:23.393596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"81a317a8-7ec8-45c3-bc10-7da023283635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6803","nonce":825351486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6805","nonce":825351486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6809","nonce":825351486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6807","nonce":825351486}]},"public_addr":"172.21.15.43:6803/825351486","cluster_addr":"172.21.15.43:6805/825351486","heartbeat_back_addr":"172.21.15.43:6809/825351486","heartbeat_front_addr":"172.21.15.43:6807/825351486","state":["exists","up"]},{"osd":1,"uuid":"e96fcfac-4fff-4a36-b38b-aebf57842740","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6801","nonce":2140095130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6803","nonce":2140095130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6807","nonce":2140095130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6805","nonce":2140095130}]},"public_addr":"172.21.15.96:6801/2140095130","cluster_addr":"172.21.15.96:6803/2140095130","heartbeat_back_addr":"172.21.15.96:6807/2140095130","heartbeat_front_addr":"172.21.15.96:6805/2140095130","state":["exists","up"]},{"osd":2,"uuid":"b659869e-114a-4f56-8aac-a2ed602b4058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6801","nonce":2047147996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6803","nonce":2047147996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6807","nonce":2047147996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6805","nonce":2047147996}]},"public_addr":"172.21.15.151:6801/2047147996","cluster_addr":"172.21.15.151:6803/2047147996","heartbeat_back_addr":"172.21.15.151:6807/2047147996","heartbeat_front_addr":"172.21.15.151:6805/2047147996","state":["exists","up"]},{"osd":3,"uuid":"7be64b2c-e45c-493d-a610-4ce41196061f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6811","nonce":1734673762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6813","nonce":1734673762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6817","nonce":1734673762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6815","nonce":1734673762}]},"public_addr":"172.21.15.43:6811/1734673762","cluster_addr":"172.21.15.43:6813/1734673762","heartbeat_back_addr":"172.21.15.43:6817/1734673762","heartbeat_front_addr":"172.21.15.43:6815/1734673762","state":["exists","up"]},{"osd":4,"uuid":"196d5af1-a6e5-496c-8933-4e58892f4360","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6809","nonce":3184785362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6811","nonce":3184785362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6815","nonce":3184785362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6813","nonce":3184785362}]},"public_addr":"172.21.15.96:6809/3184785362","cluster_addr":"172.21.15.96:6811/3184785362","heartbeat_back_addr":"172.21.15.96:6815/3184785362","heartbeat_front_addr":"172.21.15.96:6813/3184785362","state":["exists","up"]},{"osd":5,"uuid":"5492d474-8509-4865-bc2e-d08e31558a75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6809","nonce":2878451924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6811","nonce":2878451924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6815","nonce":2878451924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6813","nonce":2878451924}]},"public_addr":"172.21.15.151:6809/2878451924","cluster_addr":"172.21.15.151:6811/2878451924","heartbeat_back_addr":"172.21.15.151:6815/2878451924","heartbeat_front_addr":"172.21.15.151:6813/2878451924","state":["exists","up"]},{"osd":6,"uuid":"fb8457ed-7bb9-4435-9d15-c1cbfbddfcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6819","nonce":2008630290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6821","nonce":2008630290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6825","nonce":2008630290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6823","nonce":2008630290}]},"public_addr":"172.21.15.43:6819/2008630290","cluster_addr":"172.21.15.43:6821/2008630290","heartbeat_back_addr":"172.21.15.43:6825/2008630290","heartbeat_front_addr":"172.21.15.43:6823/2008630290","state":["exists","up"]},{"osd":7,"uuid":"02046db8-30de-4e92-b25a-1f07d6190e91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6817","nonce":3588713275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6819","nonce":3588713275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6823","nonce":3588713275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6821","nonce":3588713275}]},"public_addr":"172.21.15.96:6817/3588713275","cluster_addr":"172.21.15.96:6819/3588713275","heartbeat_back_addr":"172.21.15.96:6823/3588713275","heartbeat_front_addr":"172.21.15.96:6821/3588713275","state":["exists","up"]},{"osd":8,"uuid":"a5bb4218-51f3-4044-b43e-ebfc764f49ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6817","nonce":725316393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6819","nonce":725316393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6823","nonce":725316393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6821","nonce":725316393}]},"public_addr":"172.21.15.151:6817/725316393","cluster_addr":"172.21.15.151:6819/725316393","heartbeat_back_addr":"172.21.15.151:6823/725316393","heartbeat_front_addr":"172.21.15.151:6821/725316393","state":["exists","up"]},{"osd":9,"uuid":"6c19f65c-9750-4baa-9d63-7b8bd03e6955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6827","nonce":4053438621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6829","nonce":4053438621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6833","nonce":4053438621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6831","nonce":4053438621}]},"public_addr":"172.21.15.43:6827/4053438621","cluster_addr":"172.21.15.43:6829/4053438621","heartbeat_back_addr":"172.21.15.43:6833/4053438621","heartbeat_front_addr":"172.21.15.43:6831/4053438621","state":["exists","up"]},{"osd":10,"uuid":"a7aa7da3-8eab-47fe-a351-298b6ff75f14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6825","nonce":2403210086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6827","nonce":2403210086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6831","nonce":2403210086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6829","nonce":2403210086}]},"public_addr":"172.21.15.96:6825/2403210086","cluster_addr":"172.21.15.96:6827/2403210086","heartbeat_back_addr":"172.21.15.96:6831/2403210086","heartbeat_front_addr":"172.21.15.96:6829/2403210086","state":["exists","up"]},{"osd":11,"uuid":"760b6afc-253d-44dd-88e9-dcc97f67c9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6825","nonce":292202881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6827","nonce":292202881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6831","nonce":292202881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6829","nonce":292202881}]},"public_addr":"172.21.15.151:6825/292202881","cluster_addr":"172.21.15.151:6827/292202881","heartbeat_back_addr":"172.21.15.151:6831/292202881","heartbeat_front_addr":"172.21.15.151:6829/292202881","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:56:34.716991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:56:59.321174+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:57:24.281696+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:57:53.443880+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:58:19.205873+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:58:44.504873+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:13.650731+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:39.137876+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:04.579893+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:34.337312+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:59.434658+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:01:24.930529+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/898247657":"2024-09-20T04:55:07.037117+0000","172.21.15.43:6801/3241799377":"2024-09-20T04:55:07.037117+0000","172.21.15.43:6800/3241799377":"2024-09-20T04:55:07.037117+0000","172.21.15.43:0/3256350596":"2024-09-20T04:55:07.037117+0000","172.21.15.43:6801/587463881":"2024-09-20T04:54:39.548065+0000","172.21.15.43:6800/587463881":"2024-09-20T04:54:39.548065+0000","172.21.15.43:0/4271788315":"2024-09-20T04:55:07.037117+0000","172.21.15.43:0/6132665":"2024-09-20T04:54:39.548065+0000","172.21.15.43:0/632163478":"2024-09-20T04:54:39.548065+0000","172.21.15.43:0/330854185":"2024-09-20T04:54:39.548065+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:03:49.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.384+0000 7f09ab800640 1 -- 172.21.15.43:0/3874140990 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f09880a3350 msgr2=0x7f09880a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:49.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.384+0000 7f09ab800640 1 --2- 172.21.15.43:0/3874140990 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f09880a3350 0x7f09880a5810 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0994009230 tx=0x7f09940058d0 comp rx=0 tx=0).stop 2024-09-19T05:03:49.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.384+0000 7f09ab800640 1 -- 172.21.15.43:0/3874140990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09a410e3c0 msgr2=0x7f09a419efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:49.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.384+0000 7f09ab800640 1 --2- 172.21.15.43:0/3874140990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09a410e3c0 0x7f09a419efe0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f099c00eb10 tx=0x7f099c00c2d0 comp rx=0 tx=0).stop 2024-09-19T05:03:49.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.385+0000 7f09ab800640 1 -- 172.21.15.43:0/3874140990 shutdown_connections 2024-09-19T05:03:49.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.385+0000 7f09ab800640 1 --2- 172.21.15.43:0/3874140990 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f09880a3350 0x7f09880a5810 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:49.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.385+0000 7f09ab800640 1 --2- 172.21.15.43:0/3874140990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09a410e3c0 0x7f09a419efe0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:49.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.385+0000 7f09ab800640 1 --2- 172.21.15.43:0/3874140990 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09a4074f40 0x7f09a419ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:49.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.385+0000 7f09ab800640 1 --2- 172.21.15.43:0/3874140990 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09a4074620 0x7f09a419a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:49.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.385+0000 7f09ab800640 1 -- 172.21.15.43:0/3874140990 >> 172.21.15.43:0/3874140990 conn(0x7f09a40fd8c0 msgr2=0x7f09a40fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:49.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.385+0000 7f09ab800640 1 -- 172.21.15.43:0/3874140990 shutdown_connections 2024-09-19T05:03:49.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:49.385+0000 7f09ab800640 1 -- 172.21.15.43:0/3874140990 wait complete. 2024-09-19T05:03:50.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:49 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/3874140990' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:03:50.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:49 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/3874140990' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:03:50.193 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:03:50.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:49 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/3874140990' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:03:50.572 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:50.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:50 smithi043 ceph-mon[30800]: pgmap v336: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:03:51.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.104+0000 7fa99c400640 1 -- 172.21.15.43:0/3776213842 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa994103ff0 msgr2=0x7fa9941043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:51.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.104+0000 7fa99c400640 1 --2- 172.21.15.43:0/3776213842 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa994103ff0 0x7fa9941043d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa988009900 tx=0x7fa98802a410 comp rx=0 tx=0).stop 2024-09-19T05:03:51.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.105+0000 7fa99c400640 1 -- 172.21.15.43:0/3776213842 shutdown_connections 2024-09-19T05:03:51.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.105+0000 7fa99c400640 1 --2- 172.21.15.43:0/3776213842 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa994109120 0x7fa994110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:51.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.105+0000 7fa99c400640 1 --2- 172.21.15.43:0/3776213842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9941049a0 0x7fa9941089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:51.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.105+0000 7fa99c400640 1 --2- 172.21.15.43:0/3776213842 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa994103ff0 0x7fa9941043d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:51.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.105+0000 7fa99c400640 1 -- 172.21.15.43:0/3776213842 >> 172.21.15.43:0/3776213842 conn(0x7fa9940ffa90 msgr2=0x7fa994101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:51.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.105+0000 7fa99c400640 1 -- 172.21.15.43:0/3776213842 shutdown_connections 2024-09-19T05:03:51.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.105+0000 7fa99c400640 1 -- 172.21.15.43:0/3776213842 wait complete. 2024-09-19T05:03:51.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.106+0000 7fa99c400640 1 Processor -- start 2024-09-19T05:03:51.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.106+0000 7fa99c400640 1 -- start start 2024-09-19T05:03:51.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.106+0000 7fa99c400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa994103ff0 0x7fa99419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:51.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.106+0000 7fa99c400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9941049a0 0x7fa99419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:51.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.106+0000 7fa99c400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa994109120 0x7fa99419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:51.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.106+0000 7fa99c400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa994114170 con 0x7fa994103ff0 2024-09-19T05:03:51.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.106+0000 7fa99c400640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa994113ff0 con 0x7fa994109120 2024-09-19T05:03:51.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.106+0000 7fa99c400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa9941142f0 con 0x7fa9941049a0 2024-09-19T05:03:51.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.107+0000 7fa999a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa994103ff0 0x7fa99419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:51.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.107+0000 7fa999a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa994103ff0 0x7fa99419a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55706/0 (socket says 172.21.15.43:55706) 2024-09-19T05:03:51.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.107+0000 7fa999a00640 1 -- 172.21.15.43:0/2685522057 learned_addr learned my addr 172.21.15.43:0/2685522057 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:51.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.107+0000 7fa999000640 1 --2- 172.21.15.43:0/2685522057 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9941049a0 0x7fa99419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:51.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.107+0000 7fa99a400640 1 --2- 172.21.15.43:0/2685522057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa994109120 0x7fa99419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:51.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.107+0000 7fa999a00640 1 -- 172.21.15.43:0/2685522057 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9941049a0 msgr2=0x7fa99419abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:51.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.107+0000 7fa999a00640 1 --2- 172.21.15.43:0/2685522057 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9941049a0 0x7fa99419abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:51.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.107+0000 7fa999a00640 1 -- 172.21.15.43:0/2685522057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa994109120 msgr2=0x7fa99419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:51.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.107+0000 7fa999a00640 1 --2- 172.21.15.43:0/2685522057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa994109120 0x7fa99419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:51.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.107+0000 7fa999a00640 1 -- 172.21.15.43:0/2685522057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa99419f640 con 0x7fa994103ff0 2024-09-19T05:03:51.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.107+0000 7fa999a00640 1 --2- 172.21.15.43:0/2685522057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa994103ff0 0x7fa99419a690 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa988004770 tx=0x7fa988003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:51.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.108+0000 7fa992000640 1 -- 172.21.15.43:0/2685522057 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98803f070 con 0x7fa994103ff0 2024-09-19T05:03:51.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.108+0000 7fa992000640 1 -- 172.21.15.43:0/2685522057 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa988031ca0 con 0x7fa994103ff0 2024-09-19T05:03:51.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.108+0000 7fa99c400640 1 -- 172.21.15.43:0/2685522057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa99419f8d0 con 0x7fa994103ff0 2024-09-19T05:03:51.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.108+0000 7fa992000640 1 -- 172.21.15.43:0/2685522057 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa988035030 con 0x7fa994103ff0 2024-09-19T05:03:51.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.108+0000 7fa99c400640 1 -- 172.21.15.43:0/2685522057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9941a7210 con 0x7fa994103ff0 2024-09-19T05:03:51.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.109+0000 7fa99c400640 1 -- 172.21.15.43:0/2685522057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9941054d0 con 0x7fa994103ff0 2024-09-19T05:03:51.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.111+0000 7fa992000640 1 -- 172.21.15.43:0/2685522057 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fa988042070 con 0x7fa994103ff0 2024-09-19T05:03:51.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.112+0000 7fa992000640 1 --2- 172.21.15.43:0/2685522057 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa97c0a32d0 0x7fa97c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:51.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.112+0000 7fa992000640 1 -- 172.21.15.43:0/2685522057 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fa9880f0730 con 0x7fa994103ff0 2024-09-19T05:03:51.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.113+0000 7fa999000640 1 --2- 172.21.15.43:0/2685522057 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa97c0a32d0 0x7fa97c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:51.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.113+0000 7fa999000640 1 --2- 172.21.15.43:0/2685522057 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa97c0a32d0 0x7fa97c0a5790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa9940fffb0 tx=0x7fa980005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:51.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.114+0000 7fa992000640 1 -- 172.21.15.43:0/2685522057 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa9880bcc60 con 0x7fa994103ff0 2024-09-19T05:03:51.123 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:50 smithi096 ceph-mon[35085]: pgmap v336: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:03:51.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:50 smithi151 ceph-mon[35425]: pgmap v336: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:03:51.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.338+0000 7fa99c400640 1 -- 172.21.15.43:0/2685522057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9941043d0 con 0x7fa994103ff0 2024-09-19T05:03:51.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.339+0000 7fa992000640 1 -- 172.21.15.43:0/2685522057 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11965 (secure 0 0 0) 0x7fa9941043d0 con 0x7fa994103ff0 2024-09-19T05:03:51.340 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:03:51.341 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":28,"btime":"2024-09-19T05:02:40:505009+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/773801169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":773801169},{"type":"v1","addr":"172.21.15.43:6835","nonce":773801169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1799137902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1799137902},{"type":"v1","addr":"172.21.15.43:6837","nonce":1799137902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:02:25.463041+0000","modified":"2024-09-19T05:02:39.530912+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24518,"mds_2":24511,"mds_3":24506,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14541":{"gid":14541,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/794881135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":794881135},{"type":"v1","addr":"172.21.15.43:6841","nonce":794881135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/2836453158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2836453158},{"type":"v1","addr":"172.21.15.151:6833","nonce":2836453158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/178785812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":178785812},{"type":"v1","addr":"172.21.15.96:6833","nonce":178785812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1671043820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1671043820},{"type":"v1","addr":"172.21.15.96:6835","nonce":1671043820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2972514328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2972514328},{"type":"v1","addr":"172.21.15.151:6835","nonce":2972514328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.151:6837/3960378010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3960378010},{"type":"v1","addr":"172.21.15.151:6837","nonce":3960378010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6837/3978388286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3978388286},{"type":"v1","addr":"172.21.15.96:6837","nonce":3978388286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.151:6839/3794104313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3794104313},{"type":"v1","addr":"172.21.15.151:6839","nonce":3794104313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.43:6839/3404555030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3404555030},{"type":"v1","addr":"172.21.15.43:6839","nonce":3404555030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/605527621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":605527621},{"type":"v1","addr":"172.21.15.96:6839","nonce":605527621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24493,24506,24511,24518,24542]},"id":1}]} 2024-09-19T05:03:51.341 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 28 2024-09-19T05:03:51.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.341+0000 7fa99c400640 1 -- 172.21.15.43:0/2685522057 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa97c0a32d0 msgr2=0x7fa97c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:51.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.341+0000 7fa99c400640 1 --2- 172.21.15.43:0/2685522057 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa97c0a32d0 0x7fa97c0a5790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa9940fffb0 tx=0x7fa980005f90 comp rx=0 tx=0).stop 2024-09-19T05:03:51.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.341+0000 7fa99c400640 1 -- 172.21.15.43:0/2685522057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa994103ff0 msgr2=0x7fa99419a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:51.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.341+0000 7fa99c400640 1 --2- 172.21.15.43:0/2685522057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa994103ff0 0x7fa99419a690 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa988004770 tx=0x7fa988003c10 comp rx=0 tx=0).stop 2024-09-19T05:03:51.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.341+0000 7fa99c400640 1 -- 172.21.15.43:0/2685522057 shutdown_connections 2024-09-19T05:03:51.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.341+0000 7fa99c400640 1 --2- 172.21.15.43:0/2685522057 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa97c0a32d0 0x7fa97c0a5790 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:51.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.341+0000 7fa99c400640 1 --2- 172.21.15.43:0/2685522057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa994109120 0x7fa99419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:51.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.341+0000 7fa99c400640 1 --2- 172.21.15.43:0/2685522057 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9941049a0 0x7fa99419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:51.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.341+0000 7fa99c400640 1 --2- 172.21.15.43:0/2685522057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa994103ff0 0x7fa99419a690 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:51.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.341+0000 7fa99c400640 1 -- 172.21.15.43:0/2685522057 >> 172.21.15.43:0/2685522057 conn(0x7fa9940ffa90 msgr2=0x7fa994100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:51.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.341+0000 7fa99c400640 1 -- 172.21.15.43:0/2685522057 shutdown_connections 2024-09-19T05:03:51.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:51.341+0000 7fa99c400640 1 -- 172.21.15.43:0/2685522057 wait complete. 2024-09-19T05:03:52.023 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:51 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/2685522057' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:03:52.062 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T05:02:25.463041+0000', 'modified': '2024-09-19T05:02:39.530912+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24542, 'mds_1': 24518, 'mds_2': 24511, 'mds_3': 24506, 'mds_4': 24493}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14541': {'gid': 14541, 'name': 'a', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.43:6841/794881135', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6840', 'nonce': 794881135}, {'type': 'v1', 'addr': '172.21.15.43:6841', 'nonce': 794881135}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.151:6833/2836453158', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6832', 'nonce': 2836453158}, {'type': 'v1', 'addr': '172.21.15.151:6833', 'nonce': 2836453158}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.96:6833/178785812', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6832', 'nonce': 178785812}, {'type': 'v1', 'addr': '172.21.15.96:6833', 'nonce': 178785812}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.96:6835/1671043820', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6834', 'nonce': 1671043820}, {'type': 'v1', 'addr': '172.21.15.96:6835', 'nonce': 1671043820}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.151:6835/2972514328', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6834', 'nonce': 2972514328}, {'type': 'v1', 'addr': '172.21.15.151:6835', 'nonce': 2972514328}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'c', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.151:6837/3960378010', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6836', 'nonce': 3960378010}, {'type': 'v1', 'addr': '172.21.15.151:6837', 'nonce': 3960378010}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24506': {'gid': 24506, 'name': 'h', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.96:6837/3978388286', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6836', 'nonce': 3978388286}, {'type': 'v1', 'addr': '172.21.15.96:6837', 'nonce': 3978388286}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24511': {'gid': 24511, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.151:6839/3794104313', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6838', 'nonce': 3794104313}, {'type': 'v1', 'addr': '172.21.15.151:6839', 'nonce': 3794104313}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24518': {'gid': 24518, 'name': 'g', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.43:6839/3404555030', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6838', 'nonce': 3404555030}, {'type': 'v1', 'addr': '172.21.15.43:6839', 'nonce': 3404555030}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24542': {'gid': 24542, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.96:6839/605527621', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6838', 'nonce': 605527621}, {'type': 'v1', 'addr': '172.21.15.96:6839', 'nonce': 605527621}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24542, 'qdb_cluster': [24493, 24506, 24511, 24518, 24542]} 2024-09-19T05:03:52.062 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-19T05:03:52.062 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-19T05:03:52.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:51 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/2685522057' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:03:52.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:51 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/2685522057' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:03:52.441 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:53.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.059+0000 7fa626000640 1 -- 172.21.15.43:0/3141034771 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6140319b0 con 0x7fa6281047e0 2024-09-19T05:03:53.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.060+0000 7fa62ce00640 1 -- 172.21.15.43:0/3141034771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6281047e0 msgr2=0x7fa628108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:53.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.060+0000 7fa62ce00640 1 --2- 172.21.15.43:0/3141034771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6281047e0 0x7fa628108830 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa614009900 tx=0x7fa61402a3f0 comp rx=0 tx=0).stop 2024-09-19T05:03:53.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.060+0000 7fa62ce00640 1 -- 172.21.15.43:0/3141034771 shutdown_connections 2024-09-19T05:03:53.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.060+0000 7fa62ce00640 1 --2- 172.21.15.43:0/3141034771 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa628108f60 0x7fa628110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:53.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.060+0000 7fa62ce00640 1 --2- 172.21.15.43:0/3141034771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6281047e0 0x7fa628108830 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:53.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.060+0000 7fa62ce00640 1 --2- 172.21.15.43:0/3141034771 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa628103e30 0x7fa628104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:53.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.060+0000 7fa62ce00640 1 -- 172.21.15.43:0/3141034771 >> 172.21.15.43:0/3141034771 conn(0x7fa6280ff910 msgr2=0x7fa628101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:53.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.060+0000 7fa62ce00640 1 -- 172.21.15.43:0/3141034771 shutdown_connections 2024-09-19T05:03:53.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.060+0000 7fa62ce00640 1 -- 172.21.15.43:0/3141034771 wait complete. 2024-09-19T05:03:53.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.061+0000 7fa62ce00640 1 Processor -- start 2024-09-19T05:03:53.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.061+0000 7fa62ce00640 1 -- start start 2024-09-19T05:03:53.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.062+0000 7fa62ce00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa628103e30 0x7fa62819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:53.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.062+0000 7fa62ce00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6281047e0 0x7fa62819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:53.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.062+0000 7fa62ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa628108f60 0x7fa62819ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:53.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.062+0000 7fa62ce00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa628113fc0 con 0x7fa628108f60 2024-09-19T05:03:53.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.062+0000 7fa62ce00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa628113e40 con 0x7fa628103e30 2024-09-19T05:03:53.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.062+0000 7fa62ce00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa628114140 con 0x7fa6281047e0 2024-09-19T05:03:53.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.062+0000 7fa627400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa628103e30 0x7fa62819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:53.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.062+0000 7fa627400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa628103e30 0x7fa62819a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:59894/0 (socket says 172.21.15.43:59894) 2024-09-19T05:03:53.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.062+0000 7fa627400640 1 -- 172.21.15.43:0/1885533931 learned_addr learned my addr 172.21.15.43:0/1885533931 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:53.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.062+0000 7fa626a00640 1 --2- 172.21.15.43:0/1885533931 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6281047e0 0x7fa62819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:53.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.062+0000 7fa627e00640 1 --2- 172.21.15.43:0/1885533931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa628108f60 0x7fa62819ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:53.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.062+0000 7fa627400640 1 -- 172.21.15.43:0/1885533931 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6281047e0 msgr2=0x7fa62819a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:53.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.062+0000 7fa627400640 1 --2- 172.21.15.43:0/1885533931 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6281047e0 0x7fa62819a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:53.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.062+0000 7fa627400640 1 -- 172.21.15.43:0/1885533931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa628108f60 msgr2=0x7fa62819ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:53.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.062+0000 7fa627400640 1 --2- 172.21.15.43:0/1885533931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa628108f60 0x7fa62819ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:53.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.062+0000 7fa627400640 1 -- 172.21.15.43:0/1885533931 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa62819f430 con 0x7fa628103e30 2024-09-19T05:03:53.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.062+0000 7fa627e00640 1 --2- 172.21.15.43:0/1885533931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa628108f60 0x7fa62819ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:03:53.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.063+0000 7fa627400640 1 --2- 172.21.15.43:0/1885533931 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa628103e30 0x7fa62819a480 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa61800e9f0 tx=0x7fa61800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:53.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.063+0000 7fa61fe00640 1 -- 172.21.15.43:0/1885533931 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61800cd80 con 0x7fa628103e30 2024-09-19T05:03:53.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.063+0000 7fa62ce00640 1 -- 172.21.15.43:0/1885533931 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa62819f720 con 0x7fa628103e30 2024-09-19T05:03:53.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.063+0000 7fa61fe00640 1 -- 172.21.15.43:0/1885533931 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa618010910 con 0x7fa628103e30 2024-09-19T05:03:53.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.063+0000 7fa61fe00640 1 -- 172.21.15.43:0/1885533931 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6180188f0 con 0x7fa628103e30 2024-09-19T05:03:53.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.063+0000 7fa62ce00640 1 -- 172.21.15.43:0/1885533931 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6281a7000 con 0x7fa628103e30 2024-09-19T05:03:53.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.066+0000 7fa630ec1740 1 Processor -- start 2024-09-19T05:03:53.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.066+0000 7fa61fe00640 1 -- 172.21.15.43:0/1885533931 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fa618010ac0 con 0x7fa628103e30 2024-09-19T05:03:53.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.066+0000 7fa630ec1740 1 -- start start 2024-09-19T05:03:53.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.066+0000 7fa630ec1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587201904e0 0x5587201908c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:53.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.067+0000 7fa630ec1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558720190e00 0x55872019b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:53.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.067+0000 7fa630ec1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55872019b6c0 0x55872019dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:53.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.067+0000 7fa630ec1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5587200f60b0 con 0x55872019b6c0 2024-09-19T05:03:53.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.067+0000 7fa630ec1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55872009dab0 con 0x558720190e00 2024-09-19T05:03:53.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.067+0000 7fa630ec1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5587200f2070 con 0x5587201904e0 2024-09-19T05:03:53.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.067+0000 7fa626a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55872019b6c0 0x55872019dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:53.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.067+0000 7fa627400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558720190e00 0x55872019b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:53.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.067+0000 7fa626a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55872019b6c0 0x55872019dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55722/0 (socket says 172.21.15.43:55722) 2024-09-19T05:03:53.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.067+0000 7fa626a00640 1 -- 172.21.15.43:0/3909813087 learned_addr learned my addr 172.21.15.43:0/3909813087 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:53.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.067+0000 7fa627e00640 1 --2- 172.21.15.43:0/3909813087 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587201904e0 0x5587201908c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:53.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.067+0000 7fa627e00640 1 -- 172.21.15.43:0/3909813087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558720190e00 msgr2=0x55872019b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:03:53.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.067+0000 7fa627e00640 1 --2- 172.21.15.43:0/3909813087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558720190e00 0x55872019b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:53.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.067+0000 7fa627e00640 1 -- 172.21.15.43:0/3909813087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55872019b6c0 msgr2=0x55872019dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:03:53.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.067+0000 7fa627e00640 1 --2- 172.21.15.43:0/3909813087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55872019b6c0 0x55872019dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:53.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.067+0000 7fa627e00640 1 -- 172.21.15.43:0/3909813087 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587200aa4a0 con 0x5587201904e0 2024-09-19T05:03:53.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.067+0000 7fa626a00640 1 --2- 172.21.15.43:0/3909813087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55872019b6c0 0x55872019dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:03:53.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.068+0000 7fa627e00640 1 --2- 172.21.15.43:0/3909813087 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587201904e0 0x5587201908c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa620009f10 tx=0x7fa6200107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fbdf520e15a2758 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:53.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.068+0000 7fa61fe00640 1 --2- 172.21.15.43:0/1885533931 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa60c0a3350 0x7fa60c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:53.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.068+0000 7fa61cc00640 1 -- 172.21.15.43:0/3909813087 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62001c680 con 0x5587201904e0 2024-09-19T05:03:53.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.068+0000 7fa61cc00640 1 -- 172.21.15.43:0/3909813087 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa62001cd00 con 0x5587201904e0 2024-09-19T05:03:53.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.068+0000 7fa61cc00640 1 -- 172.21.15.43:0/3909813087 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62001a380 con 0x5587201904e0 2024-09-19T05:03:53.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.068+0000 7fa61fe00640 1 -- 172.21.15.43:0/1885533931 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fa6180cfed0 con 0x7fa628103e30 2024-09-19T05:03:53.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.068+0000 7fa627400640 1 --2- 172.21.15.43:0/1885533931 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa60c0a3350 0x7fa60c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:53.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.069+0000 7fa630ec1740 1 -- 172.21.15.43:0/3909813087 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587201904e0 msgr2=0x5587201908c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:53.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.069+0000 7fa630ec1740 1 --2- 172.21.15.43:0/3909813087 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587201904e0 0x5587201908c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa620009f10 tx=0x7fa6200107a0 comp rx=0 tx=0).stop 2024-09-19T05:03:53.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.069+0000 7fa627400640 1 --2- 172.21.15.43:0/1885533931 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa60c0a3350 0x7fa60c0a5810 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa61809c2e0 tx=0x7fa6180d3980 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:53.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.069+0000 7fa630ec1740 1 -- 172.21.15.43:0/3909813087 shutdown_connections 2024-09-19T05:03:53.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.069+0000 7fa630ec1740 1 --2- 172.21.15.43:0/3909813087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55872019b6c0 0x55872019dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:53.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.069+0000 7fa630ec1740 1 --2- 172.21.15.43:0/3909813087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558720190e00 0x55872019b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:53.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.069+0000 7fa630ec1740 1 --2- 172.21.15.43:0/3909813087 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587201904e0 0x5587201908c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:53.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.069+0000 7fa630ec1740 1 -- 172.21.15.43:0/3909813087 >> 172.21.15.43:0/3909813087 conn(0x55872018ca70 msgr2=0x55872018ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:53.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.069+0000 7fa630ec1740 1 -- 172.21.15.43:0/3909813087 shutdown_connections 2024-09-19T05:03:53.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.069+0000 7fa630ec1740 1 -- 172.21.15.43:0/3909813087 wait complete. 2024-09-19T05:03:53.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.070+0000 7fa630ec1740 1 -- start start 2024-09-19T05:03:53.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.070+0000 7fa630ec1740 1 Processor -- start 2024-09-19T05:03:53.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.070+0000 7fa630ec1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587201904e0 0x5587201985d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:53.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.070+0000 7fa630ec1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558720190e00 0x558720198b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:53.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.070+0000 7fa630ec1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55872019b6c0 0x558720199050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:53.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.070+0000 7fa630ec1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55872019e330 con 0x5587201904e0 2024-09-19T05:03:53.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.070+0000 7fa630ec1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55872007e920 con 0x55872019b6c0 2024-09-19T05:03:53.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.070+0000 7fa630ec1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55872019e4b0 con 0x558720190e00 2024-09-19T05:03:53.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.070+0000 7fa630ec1740 10 client.0 _ll_register_callbacks cb 0x5587200a28a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:03:53.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.071+0000 7fa626a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558720190e00 0x558720198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:53.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.071+0000 7fa626a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558720190e00 0x558720198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:41460/0 (socket says 172.21.15.43:41460) 2024-09-19T05:03:53.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.071+0000 7fa626a00640 1 -- 172.21.15.43:0/2127968704 learned_addr learned my addr 172.21.15.43:0/2127968704 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:53.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.071+0000 7fa627e00640 1 --2- 172.21.15.43:0/2127968704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587201904e0 0x5587201985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:53.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.071+0000 7fa627e00640 1 --2- 172.21.15.43:0/2127968704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55872019b6c0 0x558720199050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:53.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.071+0000 7fa626a00640 1 -- 172.21.15.43:0/2127968704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55872019b6c0 msgr2=0x558720199050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:03:53.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.071+0000 7fa626a00640 1 --2- 172.21.15.43:0/2127968704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55872019b6c0 0x558720199050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:53.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.071+0000 7fa626a00640 1 -- 172.21.15.43:0/2127968704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587201904e0 msgr2=0x5587201985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:03:53.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.071+0000 7fa626a00640 1 --2- 172.21.15.43:0/2127968704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587201904e0 0x5587201985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:53.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.071+0000 7fa626a00640 1 -- 172.21.15.43:0/2127968704 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55871fe672b0 con 0x558720190e00 2024-09-19T05:03:53.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.071+0000 7fa626a00640 1 --2- 172.21.15.43:0/2127968704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558720190e00 0x558720198b10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa614009230 tx=0x7fa6140317e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ddbceb71c541503 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:53.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.071+0000 7fa611600640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:03:53.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.072+0000 7fa611600640 1 -- 172.21.15.43:0/2127968704 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa614035040 con 0x558720190e00 2024-09-19T05:03:53.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.072+0000 7fa611600640 1 -- 172.21.15.43:0/2127968704 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fa614004240 con 0x558720190e00 2024-09-19T05:03:53.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.072+0000 7fa630ec1740 1 -- 172.21.15.43:0/2127968704 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa62001cd00 con 0x558720190e00 2024-09-19T05:03:53.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.072+0000 7fa611600640 1 -- 172.21.15.43:0/2127968704 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa614004440 con 0x558720190e00 2024-09-19T05:03:53.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.072+0000 7fa630ec1740 10 client.24877 fetch_fsmap learned FSMap version 28 2024-09-19T05:03:53.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.072+0000 7fa630ec1740 1 -- 172.21.15.43:0/2127968704 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5587201aa040 con 0x558720190e00 2024-09-19T05:03:53.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.072+0000 7fa611600640 1 -- 172.21.15.43:0/2127968704 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa62001cd00 con 0x558720190e00 2024-09-19T05:03:53.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.072+0000 7fa611600640 1 -- 172.21.15.43:0/2127968704 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa6140035f0 con 0x558720190e00 2024-09-19T05:03:53.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.073+0000 7fa630ec1740 10 client.24877 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:03:53.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.073+0000 7fa630ec1740 10 client.24877 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T05:03:53.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.073+0000 7fa630ec1740 20 client.24877 populate_metadata read hostname 'smithi043' 2024-09-19T05:03:53.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.073+0000 7fa630ec1740 1 --2- 172.21.15.43:0/2127968704 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x5587201ac6e0 0x5587201adb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:53.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.073+0000 7fa630ec1740 4 client.24877 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:03:53.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.073+0000 7fa630ec1740 1 -- 172.21.15.43:0/2127968704 --> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558720103620 con 0x5587201ac6e0 2024-09-19T05:03:53.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.073+0000 7fa627e00640 1 --2- 172.21.15.43:0/2127968704 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x5587201ac6e0 0x5587201adb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:53.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.073+0000 7fa627e00640 1 --2- 172.21.15.43:0/2127968704 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x5587201ac6e0 0x5587201adb50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=76efff80b120a089 server_cookie=1ef73491735f1bd in_seq=0 out_seq=0 2024-09-19T05:03:53.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.073+0000 7fa611600640 10 client.24877 ms_handle_connect on v2:172.21.15.151:6836/3960378010 2024-09-19T05:03:53.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.074+0000 7fa611600640 1 -- 172.21.15.43:0/2127968704 <== mds.4 v2:172.21.15.151:6836/3960378010 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558720103620 con 0x5587201ac6e0 2024-09-19T05:03:53.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.074+0000 7fa611600640 10 client.24877 handle_command_reply: tid=0 2024-09-19T05:03:53.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.074+0000 7fa630ec1740 1 client.24877 shutdown 2024-09-19T05:03:53.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.075+0000 7fa630ec1740 1 -- 172.21.15.43:0/2127968704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558720190e00 msgr2=0x558720198b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:53.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.075+0000 7fa630ec1740 1 --2- 172.21.15.43:0/2127968704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558720190e00 0x558720198b10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa614009230 tx=0x7fa6140317e0 comp rx=0 tx=0).stop 2024-09-19T05:03:53.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.075+0000 7fa630ec1740 1 -- 172.21.15.43:0/2127968704 shutdown_connections 2024-09-19T05:03:53.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.075+0000 7fa630ec1740 1 --2- 172.21.15.43:0/2127968704 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x5587201ac6e0 0x5587201adb50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:53.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.075+0000 7fa630ec1740 1 --2- 172.21.15.43:0/2127968704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55872019b6c0 0x558720199050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:53.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.075+0000 7fa630ec1740 1 --2- 172.21.15.43:0/2127968704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558720190e00 0x558720198b10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:53.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.075+0000 7fa630ec1740 1 --2- 172.21.15.43:0/2127968704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587201904e0 0x5587201985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:53.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.075+0000 7fa611600640 0 client.24877 ms_handle_reset on v2:172.21.15.151:6836/3960378010 2024-09-19T05:03:53.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.075+0000 7fa630ec1740 1 -- 172.21.15.43:0/2127968704 >> 172.21.15.43:0/2127968704 conn(0x55872018ca70 msgr2=0x558720192750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:53.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.075+0000 7fa630ec1740 1 -- 172.21.15.43:0/2127968704 shutdown_connections 2024-09-19T05:03:53.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.075+0000 7fa630ec1740 1 -- 172.21.15.43:0/2127968704 wait complete. 2024-09-19T05:03:53.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.075+0000 7fa630ec1740 20 client.24877 trim_cache size 0 max 16384 2024-09-19T05:03:53.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.093+0000 7fa630ec1740 1 Processor -- start 2024-09-19T05:03:53.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.094+0000 7fa630ec1740 1 -- start start 2024-09-19T05:03:53.095 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:52 smithi043 ceph-mon[30800]: pgmap v337: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T05:03:53.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.094+0000 7fa630ec1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587201904e0 0x558720187840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:53.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.094+0000 7fa630ec1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558720190e00 0x558720192f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:53.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.094+0000 7fa630ec1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55872019b6c0 0x558720198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:53.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.094+0000 7fa630ec1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5587201abe90 con 0x5587201904e0 2024-09-19T05:03:53.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.094+0000 7fa630ec1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5587201ac560 con 0x55872019b6c0 2024-09-19T05:03:53.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.094+0000 7fa630ec1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5587201acc30 con 0x558720190e00 2024-09-19T05:03:53.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.094+0000 7fa627e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587201904e0 0x558720187840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:53.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.094+0000 7fa627e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587201904e0 0x558720187840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55738/0 (socket says 172.21.15.43:55738) 2024-09-19T05:03:53.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.094+0000 7fa627e00640 1 -- 172.21.15.43:0/2018744862 learned_addr learned my addr 172.21.15.43:0/2018744862 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:53.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.094+0000 7fa626a00640 1 --2- 172.21.15.43:0/2018744862 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558720190e00 0x558720192f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:53.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.095+0000 7fa627e00640 1 --2- 172.21.15.43:0/2018744862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55872019b6c0 0x558720198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:53.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.095+0000 7fa627e00640 1 -- 172.21.15.43:0/2018744862 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558720190e00 msgr2=0x558720192f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:03:53.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.095+0000 7fa627e00640 1 --2- 172.21.15.43:0/2018744862 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558720190e00 0x558720192f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:53.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.095+0000 7fa627e00640 1 -- 172.21.15.43:0/2018744862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55872019b6c0 msgr2=0x558720198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:03:53.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.095+0000 7fa627e00640 1 --2- 172.21.15.43:0/2018744862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55872019b6c0 0x558720198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:53.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.095+0000 7fa627e00640 1 -- 172.21.15.43:0/2018744862 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558720193480 con 0x5587201904e0 2024-09-19T05:03:53.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.095+0000 7fa626a00640 1 --2- 172.21.15.43:0/2018744862 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558720190e00 0x558720192f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:03:53.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.095+0000 7fa627e00640 1 --2- 172.21.15.43:0/2018744862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587201904e0 0x558720187840 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa620009a40 tx=0x7fa620009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=407ab64beec29d0f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:53.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.095+0000 7fa613e00640 1 -- 172.21.15.43:0/2018744862 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa620025040 con 0x5587201904e0 2024-09-19T05:03:53.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.095+0000 7fa613e00640 1 -- 172.21.15.43:0/2018744862 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa620030070 con 0x5587201904e0 2024-09-19T05:03:53.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.095+0000 7fa613e00640 1 -- 172.21.15.43:0/2018744862 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62002c670 con 0x5587201904e0 2024-09-19T05:03:53.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.096+0000 7fa630ec1740 1 -- 172.21.15.43:0/2018744862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587201904e0 msgr2=0x558720187840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:53.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.096+0000 7fa630ec1740 1 --2- 172.21.15.43:0/2018744862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587201904e0 0x558720187840 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa620009a40 tx=0x7fa620009280 comp rx=0 tx=0).stop 2024-09-19T05:03:53.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.096+0000 7fa630ec1740 1 -- 172.21.15.43:0/2018744862 shutdown_connections 2024-09-19T05:03:53.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.096+0000 7fa630ec1740 1 --2- 172.21.15.43:0/2018744862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55872019b6c0 0x558720198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:53.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.096+0000 7fa630ec1740 1 --2- 172.21.15.43:0/2018744862 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558720190e00 0x558720192f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:53.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.096+0000 7fa630ec1740 1 --2- 172.21.15.43:0/2018744862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587201904e0 0x558720187840 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:53.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.096+0000 7fa630ec1740 1 -- 172.21.15.43:0/2018744862 >> 172.21.15.43:0/2018744862 conn(0x55872018ca70 msgr2=0x558720196f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:53.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.096+0000 7fa630ec1740 1 -- 172.21.15.43:0/2018744862 shutdown_connections 2024-09-19T05:03:53.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.096+0000 7fa630ec1740 1 -- 172.21.15.43:0/2018744862 wait complete. 2024-09-19T05:03:53.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.097+0000 7fa630ec1740 1 -- start start 2024-09-19T05:03:53.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.097+0000 7fa630ec1740 1 Processor -- start 2024-09-19T05:03:53.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.097+0000 7fa630ec1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587201904e0 0x5587201955e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:53.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.097+0000 7fa630ec1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558720190e00 0x558720195b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:53.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.098+0000 7fa630ec1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55872019b6c0 0x558720196060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:53.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.098+0000 7fa630ec1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5587201ad9d0 con 0x558720190e00 2024-09-19T05:03:53.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.098+0000 7fa630ec1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5587201ad300 con 0x55872019b6c0 2024-09-19T05:03:53.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.098+0000 7fa630ec1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5587201ae0a0 con 0x5587201904e0 2024-09-19T05:03:53.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.098+0000 7fa630ec1740 10 client.0 _ll_register_callbacks cb 0x7fa614005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:03:53.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.098+0000 7fa626a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558720190e00 0x558720195b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:53.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.098+0000 7fa627e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587201904e0 0x5587201955e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:53.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.098+0000 7fa626a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558720190e00 0x558720195b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55744/0 (socket says 172.21.15.43:55744) 2024-09-19T05:03:53.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.098+0000 7fa626a00640 1 -- 172.21.15.43:0/2797319462 learned_addr learned my addr 172.21.15.43:0/2797319462 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:53.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.098+0000 7fa627e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587201904e0 0x5587201955e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:41476/0 (socket says 172.21.15.43:41476) 2024-09-19T05:03:53.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.098+0000 7fa627e00640 1 --2- 172.21.15.43:0/2797319462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55872019b6c0 0x558720196060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:53.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.098+0000 7fa626a00640 1 -- 172.21.15.43:0/2797319462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587201904e0 msgr2=0x5587201955e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:03:53.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.098+0000 7fa626a00640 1 --2- 172.21.15.43:0/2797319462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587201904e0 0x5587201955e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:53.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.098+0000 7fa626a00640 1 -- 172.21.15.43:0/2797319462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55872019b6c0 msgr2=0x558720196060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:03:53.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.098+0000 7fa626a00640 1 --2- 172.21.15.43:0/2797319462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55872019b6c0 0x558720196060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:53.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.098+0000 7fa626a00640 1 -- 172.21.15.43:0/2797319462 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587201b4220 con 0x558720190e00 2024-09-19T05:03:53.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.099+0000 7fa626a00640 1 --2- 172.21.15.43:0/2797319462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558720190e00 0x558720195b20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa614003a90 tx=0x7fa614009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ddca1f8e8ce38a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:53.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.099+0000 7fa611600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:03:53.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.099+0000 7fa611600640 1 -- 172.21.15.43:0/2797319462 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa614035050 con 0x558720190e00 2024-09-19T05:03:53.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.099+0000 7fa611600640 1 -- 172.21.15.43:0/2797319462 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fa6140046c0 con 0x558720190e00 2024-09-19T05:03:53.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.099+0000 7fa630ec1740 1 -- 172.21.15.43:0/2797319462 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa620030070 con 0x558720190e00 2024-09-19T05:03:53.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.099+0000 7fa611600640 1 -- 172.21.15.43:0/2797319462 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa614062050 con 0x558720190e00 2024-09-19T05:03:53.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.099+0000 7fa630ec1740 10 client.14904 fetch_fsmap learned FSMap version 28 2024-09-19T05:03:53.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.099+0000 7fa630ec1740 1 -- 172.21.15.43:0/2797319462 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558720216fd0 con 0x558720190e00 2024-09-19T05:03:53.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.100+0000 7fa611600640 1 -- 172.21.15.43:0/2797319462 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa620030070 con 0x558720190e00 2024-09-19T05:03:53.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.100+0000 7fa611600640 1 -- 172.21.15.43:0/2797319462 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa614059070 con 0x558720190e00 2024-09-19T05:03:53.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.100+0000 7fa630ec1740 10 client.14904 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:03:53.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.100+0000 7fa630ec1740 10 client.14904 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T05:03:53.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.100+0000 7fa630ec1740 20 client.14904 populate_metadata read hostname 'smithi043' 2024-09-19T05:03:53.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.100+0000 7fa630ec1740 1 --2- 172.21.15.43:0/2797319462 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55872021bb00 0x55872021a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:53.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.100+0000 7fa630ec1740 4 client.14904 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T05:03:53.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.100+0000 7fa630ec1740 1 -- 172.21.15.43:0/2797319462 --> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] -- command(tid 0: {"prefix": "status"}) -- 0x5587200fdb30 con 0x55872021bb00 2024-09-19T05:03:53.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.100+0000 7fa627e00640 1 --2- 172.21.15.43:0/2797319462 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55872021bb00 0x55872021a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:53.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.101+0000 7fa627e00640 1 --2- 172.21.15.43:0/2797319462 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55872021bb00 0x55872021a810 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=67f36406988bf040 server_cookie=6ef8b8695a4f48 in_seq=0 out_seq=0 2024-09-19T05:03:53.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.101+0000 7fa611600640 10 client.14904 ms_handle_connect on v2:172.21.15.151:6836/3960378010 2024-09-19T05:03:53.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.101+0000 7fa611600640 1 -- 172.21.15.43:0/2797319462 <== mds.4 v2:172.21.15.151:6836/3960378010 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5587200fdb30 con 0x55872021bb00 2024-09-19T05:03:53.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.101+0000 7fa611600640 10 client.14904 handle_command_reply: tid=0 2024-09-19T05:03:53.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.101+0000 7fa630ec1740 1 client.14904 shutdown 2024-09-19T05:03:53.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.101+0000 7fa630ec1740 1 -- 172.21.15.43:0/2797319462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558720190e00 msgr2=0x558720195b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:53.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.101+0000 7fa630ec1740 1 --2- 172.21.15.43:0/2797319462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558720190e00 0x558720195b20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa614003a90 tx=0x7fa614009260 comp rx=0 tx=0).stop 2024-09-19T05:03:53.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.101+0000 7fa630ec1740 1 -- 172.21.15.43:0/2797319462 shutdown_connections 2024-09-19T05:03:53.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.101+0000 7fa630ec1740 1 --2- 172.21.15.43:0/2797319462 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55872021bb00 0x55872021a810 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:53.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.101+0000 7fa630ec1740 1 --2- 172.21.15.43:0/2797319462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55872019b6c0 0x558720196060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:53.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.101+0000 7fa630ec1740 1 --2- 172.21.15.43:0/2797319462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558720190e00 0x558720195b20 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:53.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.101+0000 7fa630ec1740 1 --2- 172.21.15.43:0/2797319462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587201904e0 0x5587201955e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:53.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.101+0000 7fa611600640 0 client.14904 ms_handle_reset on v2:172.21.15.151:6836/3960378010 2024-09-19T05:03:53.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.101+0000 7fa630ec1740 1 -- 172.21.15.43:0/2797319462 >> 172.21.15.43:0/2797319462 conn(0x55872018ca70 msgr2=0x558720196e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:53.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.102+0000 7fa630ec1740 1 -- 172.21.15.43:0/2797319462 shutdown_connections 2024-09-19T05:03:53.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.102+0000 7fa630ec1740 1 -- 172.21.15.43:0/2797319462 wait complete. 2024-09-19T05:03:53.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.102+0000 7fa630ec1740 20 client.14904 trim_cache size 0 max 16384 2024-09-19T05:03:53.106 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:03:53.106 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "1456a810-7643-11ef-bceb-c7b262605968", 2024-09-19T05:03:53.106 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 4, 2024-09-19T05:03:53.106 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 24493, 2024-09-19T05:03:53.106 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-09-19T05:03:53.106 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-09-19T05:03:53.106 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-09-19T05:03:53.106 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 74.611134149999998, 2024-09-19T05:03:53.107 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 28, 2024-09-19T05:03:53.107 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 85, 2024-09-19T05:03:53.107 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 83, 2024-09-19T05:03:53.107 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 115.31080173700001 2024-09-19T05:03:53.107 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:03:53.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.102+0000 7fa62ce00640 1 -- 172.21.15.43:0/1885533931 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa60c0a3350 msgr2=0x7fa60c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:53.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.102+0000 7fa62ce00640 1 --2- 172.21.15.43:0/1885533931 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa60c0a3350 0x7fa60c0a5810 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa61809c2e0 tx=0x7fa6180d3980 comp rx=0 tx=0).stop 2024-09-19T05:03:53.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.103+0000 7fa62ce00640 1 -- 172.21.15.43:0/1885533931 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa628103e30 msgr2=0x7fa62819a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:53.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.103+0000 7fa62ce00640 1 --2- 172.21.15.43:0/1885533931 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa628103e30 0x7fa62819a480 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa61800e9f0 tx=0x7fa61800eeb0 comp rx=0 tx=0).stop 2024-09-19T05:03:53.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.103+0000 7fa62ce00640 1 -- 172.21.15.43:0/1885533931 shutdown_connections 2024-09-19T05:03:53.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.103+0000 7fa62ce00640 1 --2- 172.21.15.43:0/1885533931 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa60c0a3350 0x7fa60c0a5810 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:53.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.103+0000 7fa62ce00640 1 --2- 172.21.15.43:0/1885533931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa628108f60 0x7fa62819ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:53.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.103+0000 7fa62ce00640 1 --2- 172.21.15.43:0/1885533931 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6281047e0 0x7fa62819a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:53.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.103+0000 7fa62ce00640 1 --2- 172.21.15.43:0/1885533931 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa628103e30 0x7fa62819a480 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:53.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.103+0000 7fa62ce00640 1 -- 172.21.15.43:0/1885533931 >> 172.21.15.43:0/1885533931 conn(0x7fa6280ff910 msgr2=0x7fa628100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:53.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.103+0000 7fa62ce00640 1 -- 172.21.15.43:0/1885533931 shutdown_connections 2024-09-19T05:03:53.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:53.103+0000 7fa62ce00640 1 -- 172.21.15.43:0/1885533931 wait complete. 2024-09-19T05:03:53.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:52 smithi151 ceph-mon[35425]: pgmap v337: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T05:03:53.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:52 smithi096 ceph-mon[35085]: pgmap v337: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T05:03:53.858 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-19T05:03:54.235 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:54.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.756+0000 7ff130200640 1 -- 172.21.15.43:0/212426445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff128102790 msgr2=0x7ff1281067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:54.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.756+0000 7ff130200640 1 --2- 172.21.15.43:0/212426445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff128102790 0x7ff1281067e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff114009900 tx=0x7ff11402a3f0 comp rx=0 tx=0).stop 2024-09-19T05:03:54.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.756+0000 7ff130200640 1 -- 172.21.15.43:0/212426445 shutdown_connections 2024-09-19T05:03:54.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.756+0000 7ff130200640 1 --2- 172.21.15.43:0/212426445 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff128106f10 0x7ff12810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:54.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.756+0000 7ff130200640 1 --2- 172.21.15.43:0/212426445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff128102790 0x7ff1281067e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:54.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.756+0000 7ff130200640 1 --2- 172.21.15.43:0/212426445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff128101de0 0x7ff1281021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:54.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.756+0000 7ff130200640 1 -- 172.21.15.43:0/212426445 >> 172.21.15.43:0/212426445 conn(0x7ff1280fd8c0 msgr2=0x7ff1280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:54.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.756+0000 7ff130200640 1 -- 172.21.15.43:0/212426445 shutdown_connections 2024-09-19T05:03:54.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.756+0000 7ff130200640 1 -- 172.21.15.43:0/212426445 wait complete. 2024-09-19T05:03:54.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.758+0000 7ff130200640 1 Processor -- start 2024-09-19T05:03:54.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.758+0000 7ff130200640 1 -- start start 2024-09-19T05:03:54.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.758+0000 7ff130200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff128101de0 0x7ff1280744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:54.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.758+0000 7ff130200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff128102790 0x7ff128074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:54.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.758+0000 7ff130200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff128106f10 0x7ff128074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:54.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.758+0000 7ff130200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff128111fc0 con 0x7ff128101de0 2024-09-19T05:03:54.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.758+0000 7ff130200640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff128111e40 con 0x7ff128102790 2024-09-19T05:03:54.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.758+0000 7ff130200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff128112140 con 0x7ff128106f10 2024-09-19T05:03:54.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.759+0000 7ff12d800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff128101de0 0x7ff1280744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:54.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.759+0000 7ff12d800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff128101de0 0x7ff1280744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55754/0 (socket says 172.21.15.43:55754) 2024-09-19T05:03:54.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.759+0000 7ff12d800640 1 -- 172.21.15.43:0/1945104495 learned_addr learned my addr 172.21.15.43:0/1945104495 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:54.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.759+0000 7ff12ce00640 1 --2- 172.21.15.43:0/1945104495 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff128102790 0x7ff128074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:54.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.759+0000 7ff12e200640 1 --2- 172.21.15.43:0/1945104495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff128106f10 0x7ff128074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:54.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.759+0000 7ff12d800640 1 -- 172.21.15.43:0/1945104495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff128106f10 msgr2=0x7ff128074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:54.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.759+0000 7ff12d800640 1 --2- 172.21.15.43:0/1945104495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff128106f10 0x7ff128074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:54.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.759+0000 7ff12d800640 1 -- 172.21.15.43:0/1945104495 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff128102790 msgr2=0x7ff128074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:54.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.759+0000 7ff12d800640 1 --2- 172.21.15.43:0/1945104495 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff128102790 0x7ff128074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:54.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.759+0000 7ff12d800640 1 -- 172.21.15.43:0/1945104495 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1281a6920 con 0x7ff128101de0 2024-09-19T05:03:54.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.759+0000 7ff12d800640 1 --2- 172.21.15.43:0/1945104495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff128101de0 0x7ff1280744f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00e9f0 tx=0x7ff11c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:54.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.760+0000 7ff126000640 1 -- 172.21.15.43:0/1945104495 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c00ccc0 con 0x7ff128101de0 2024-09-19T05:03:54.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.760+0000 7ff126000640 1 -- 172.21.15.43:0/1945104495 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff11c00ce60 con 0x7ff128101de0 2024-09-19T05:03:54.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.760+0000 7ff130200640 1 -- 172.21.15.43:0/1945104495 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1281a6c10 con 0x7ff128101de0 2024-09-19T05:03:54.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.760+0000 7ff126000640 1 -- 172.21.15.43:0/1945104495 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c010690 con 0x7ff128101de0 2024-09-19T05:03:54.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.760+0000 7ff130200640 1 -- 172.21.15.43:0/1945104495 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1281a7120 con 0x7ff128101de0 2024-09-19T05:03:54.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.763+0000 7ff132e4e740 1 Processor -- start 2024-09-19T05:03:54.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.763+0000 7ff126000640 1 -- 172.21.15.43:0/1945104495 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7ff11c0046c0 con 0x7ff128101de0 2024-09-19T05:03:54.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.763+0000 7ff132e4e740 1 -- start start 2024-09-19T05:03:54.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.764+0000 7ff132e4e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562fa293c330 0x562fa293c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:54.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.764+0000 7ff132e4e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562fa293cc50 0x562fa2946fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:54.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.764+0000 7ff132e4e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562fa2947510 0x562fa2949900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:54.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.764+0000 7ff132e4e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562fa28a20b0 con 0x562fa293cc50 2024-09-19T05:03:54.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.764+0000 7ff132e4e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562fa2849ab0 con 0x562fa2947510 2024-09-19T05:03:54.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.764+0000 7ff132e4e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562fa289e070 con 0x562fa293c330 2024-09-19T05:03:54.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.764+0000 7ff12d800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562fa293cc50 0x562fa2946fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:54.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.764+0000 7ff12d800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562fa293cc50 0x562fa2946fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55768/0 (socket says 172.21.15.43:55768) 2024-09-19T05:03:54.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.764+0000 7ff12d800640 1 -- 172.21.15.43:0/102038039 learned_addr learned my addr 172.21.15.43:0/102038039 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:54.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.764+0000 7ff12e200640 1 --2- 172.21.15.43:0/102038039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562fa293c330 0x562fa293c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:54.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.764+0000 7ff12ce00640 1 --2- 172.21.15.43:0/102038039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562fa2947510 0x562fa2949900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:54.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.764+0000 7ff12e200640 1 -- 172.21.15.43:0/102038039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562fa2947510 msgr2=0x562fa2949900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:03:54.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.764+0000 7ff12e200640 1 --2- 172.21.15.43:0/102038039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562fa2947510 0x562fa2949900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:54.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.764+0000 7ff12e200640 1 -- 172.21.15.43:0/102038039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562fa293cc50 msgr2=0x562fa2946fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:03:54.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.764+0000 7ff12e200640 1 --2- 172.21.15.43:0/102038039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562fa293cc50 0x562fa2946fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:54.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.765+0000 7ff12e200640 1 -- 172.21.15.43:0/102038039 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fa28564a0 con 0x562fa293c330 2024-09-19T05:03:54.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.765+0000 7ff12d800640 1 --2- 172.21.15.43:0/102038039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562fa293cc50 0x562fa2946fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:03:54.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.765+0000 7ff12e200640 1 --2- 172.21.15.43:0/102038039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562fa293c330 0x562fa293c710 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff120009f10 tx=0x7ff120011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cc5d0e30b12e9c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:54.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.765+0000 7ff126000640 1 --2- 172.21.15.43:0/1945104495 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff1100a3350 0x7ff1100a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:54.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.765+0000 7ff11aa00640 1 -- 172.21.15.43:0/102038039 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff120004d20 con 0x562fa293c330 2024-09-19T05:03:54.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.765+0000 7ff11aa00640 1 -- 172.21.15.43:0/102038039 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff120004050 con 0x562fa293c330 2024-09-19T05:03:54.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.765+0000 7ff11aa00640 1 -- 172.21.15.43:0/102038039 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff120005260 con 0x562fa293c330 2024-09-19T05:03:54.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.765+0000 7ff126000640 1 -- 172.21.15.43:0/1945104495 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ff11c0cfe40 con 0x7ff128101de0 2024-09-19T05:03:54.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.765+0000 7ff12d800640 1 --2- 172.21.15.43:0/1945104495 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff1100a3350 0x7ff1100a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:54.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.766+0000 7ff132e4e740 1 -- 172.21.15.43:0/102038039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562fa293c330 msgr2=0x562fa293c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:54.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.766+0000 7ff132e4e740 1 --2- 172.21.15.43:0/102038039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562fa293c330 0x562fa293c710 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff120009f10 tx=0x7ff120011810 comp rx=0 tx=0).stop 2024-09-19T05:03:54.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.766+0000 7ff12d800640 1 --2- 172.21.15.43:0/1945104495 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff1100a3350 0x7ff1100a5810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff11c09bfc0 tx=0x7ff11c0d38f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:54.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.766+0000 7ff132e4e740 1 -- 172.21.15.43:0/102038039 shutdown_connections 2024-09-19T05:03:54.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.766+0000 7ff132e4e740 1 --2- 172.21.15.43:0/102038039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562fa2947510 0x562fa2949900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:54.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.766+0000 7ff132e4e740 1 --2- 172.21.15.43:0/102038039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562fa293cc50 0x562fa2946fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:54.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.766+0000 7ff132e4e740 1 --2- 172.21.15.43:0/102038039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562fa293c330 0x562fa293c710 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:54.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.766+0000 7ff132e4e740 1 -- 172.21.15.43:0/102038039 >> 172.21.15.43:0/102038039 conn(0x562fa29388c0 msgr2=0x562fa2938ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:54.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.766+0000 7ff132e4e740 1 -- 172.21.15.43:0/102038039 shutdown_connections 2024-09-19T05:03:54.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.766+0000 7ff132e4e740 1 -- 172.21.15.43:0/102038039 wait complete. 2024-09-19T05:03:54.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.767+0000 7ff132e4e740 1 -- start start 2024-09-19T05:03:54.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.767+0000 7ff132e4e740 1 Processor -- start 2024-09-19T05:03:54.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.767+0000 7ff132e4e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562fa293c330 0x562fa2944520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:54.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.767+0000 7ff132e4e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562fa293cc50 0x562fa2944a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:54.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.767+0000 7ff132e4e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562fa2947510 0x562fa2944fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:54.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.767+0000 7ff132e4e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562fa294a060 con 0x562fa2947510 2024-09-19T05:03:54.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.767+0000 7ff132e4e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562fa282a920 con 0x562fa293cc50 2024-09-19T05:03:54.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.767+0000 7ff132e4e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562fa294a1e0 con 0x562fa293c330 2024-09-19T05:03:54.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.767+0000 7ff132e4e740 10 client.0 _ll_register_callbacks cb 0x562fa284e8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:03:54.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.768+0000 7ff12e200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562fa293c330 0x562fa2944520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:54.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.768+0000 7ff12e200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562fa293c330 0x562fa2944520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:41512/0 (socket says 172.21.15.43:41512) 2024-09-19T05:03:54.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.768+0000 7ff12e200640 1 -- 172.21.15.43:0/1099243575 learned_addr learned my addr 172.21.15.43:0/1099243575 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:54.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.768+0000 7ff12ce00640 1 --2- 172.21.15.43:0/1099243575 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562fa293cc50 0x562fa2944a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:54.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.768+0000 7ff12e200640 1 --2- 172.21.15.43:0/1099243575 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562fa2947510 0x562fa2944fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:54.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.768+0000 7ff12ce00640 1 -- 172.21.15.43:0/1099243575 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562fa293c330 msgr2=0x562fa2944520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:03:54.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.768+0000 7ff12ce00640 1 --2- 172.21.15.43:0/1099243575 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562fa293c330 0x562fa2944520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:54.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.768+0000 7ff12ce00640 1 -- 172.21.15.43:0/1099243575 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562fa2947510 msgr2=0x562fa2944fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:03:54.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.768+0000 7ff12ce00640 1 --2- 172.21.15.43:0/1099243575 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562fa2947510 0x562fa2944fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:54.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.768+0000 7ff12ce00640 1 -- 172.21.15.43:0/1099243575 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fa26132b0 con 0x562fa293cc50 2024-09-19T05:03:54.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.769+0000 7ff12ce00640 1 --2- 172.21.15.43:0/1099243575 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562fa293cc50 0x562fa2944a60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff11402a910 tx=0x7ff114004390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1c354d856e51a35 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:54.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.769+0000 7ff10f400640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:03:54.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.769+0000 7ff10f400640 1 -- 172.21.15.43:0/1099243575 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff114035040 con 0x562fa293cc50 2024-09-19T05:03:54.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.769+0000 7ff132e4e740 1 -- 172.21.15.43:0/1099243575 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff120004050 con 0x562fa293cc50 2024-09-19T05:03:54.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.769+0000 7ff10f400640 1 -- 172.21.15.43:0/1099243575 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ff114005450 con 0x562fa293cc50 2024-09-19T05:03:54.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.769+0000 7ff10f400640 1 -- 172.21.15.43:0/1099243575 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff114005650 con 0x562fa293cc50 2024-09-19T05:03:54.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.769+0000 7ff132e4e740 10 client.24905 fetch_fsmap learned FSMap version 28 2024-09-19T05:03:54.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.769+0000 7ff132e4e740 1 -- 172.21.15.43:0/1099243575 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562fa2955e20 con 0x562fa293cc50 2024-09-19T05:03:54.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.769+0000 7ff10f400640 1 -- 172.21.15.43:0/1099243575 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff120004050 con 0x562fa293cc50 2024-09-19T05:03:54.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.770+0000 7ff10f400640 1 -- 172.21.15.43:0/1099243575 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff1140319f0 con 0x562fa293cc50 2024-09-19T05:03:54.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.770+0000 7ff132e4e740 10 client.24905 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:03:54.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.770+0000 7ff132e4e740 10 client.24905 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T05:03:54.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.770+0000 7ff132e4e740 20 client.24905 populate_metadata read hostname 'smithi043' 2024-09-19T05:03:54.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.770+0000 7ff132e4e740 1 --2- 172.21.15.43:0/1099243575 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x562fa29583f0 0x562fa2959860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:54.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.770+0000 7ff132e4e740 4 client.24905 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:03:54.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.770+0000 7ff132e4e740 1 -- 172.21.15.43:0/1099243575 --> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562fa28af620 con 0x562fa29583f0 2024-09-19T05:03:54.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.770+0000 7ff12e200640 1 --2- 172.21.15.43:0/1099243575 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x562fa29583f0 0x562fa2959860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:54.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.771+0000 7ff12e200640 1 --2- 172.21.15.43:0/1099243575 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x562fa29583f0 0x562fa2959860 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d390e9514f4b8f90 server_cookie=b75f37753ae9f1c0 in_seq=0 out_seq=0 2024-09-19T05:03:54.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.771+0000 7ff10f400640 10 client.24905 ms_handle_connect on v2:172.21.15.96:6836/3978388286 2024-09-19T05:03:54.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.772+0000 7ff10f400640 1 -- 172.21.15.43:0/1099243575 <== mds.3 v2:172.21.15.96:6836/3978388286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562fa28af620 con 0x562fa29583f0 2024-09-19T05:03:54.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.772+0000 7ff10f400640 10 client.24905 handle_command_reply: tid=0 2024-09-19T05:03:54.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.772+0000 7ff132e4e740 1 client.24905 shutdown 2024-09-19T05:03:54.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.772+0000 7ff132e4e740 1 -- 172.21.15.43:0/1099243575 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562fa293cc50 msgr2=0x562fa2944a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:54.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.772+0000 7ff132e4e740 1 --2- 172.21.15.43:0/1099243575 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562fa293cc50 0x562fa2944a60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff11402a910 tx=0x7ff114004390 comp rx=0 tx=0).stop 2024-09-19T05:03:54.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.772+0000 7ff132e4e740 1 -- 172.21.15.43:0/1099243575 shutdown_connections 2024-09-19T05:03:54.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.772+0000 7ff132e4e740 1 --2- 172.21.15.43:0/1099243575 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x562fa29583f0 0x562fa2959860 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:54.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.772+0000 7ff132e4e740 1 --2- 172.21.15.43:0/1099243575 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562fa2947510 0x562fa2944fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:54.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.772+0000 7ff132e4e740 1 --2- 172.21.15.43:0/1099243575 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562fa293cc50 0x562fa2944a60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:54.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.772+0000 7ff10f400640 0 client.24905 ms_handle_reset on v2:172.21.15.96:6836/3978388286 2024-09-19T05:03:54.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.772+0000 7ff132e4e740 1 --2- 172.21.15.43:0/1099243575 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562fa293c330 0x562fa2944520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:54.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.772+0000 7ff132e4e740 1 -- 172.21.15.43:0/1099243575 >> 172.21.15.43:0/1099243575 conn(0x562fa29388c0 msgr2=0x562fa293e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:54.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.772+0000 7ff132e4e740 1 -- 172.21.15.43:0/1099243575 shutdown_connections 2024-09-19T05:03:54.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.772+0000 7ff132e4e740 1 -- 172.21.15.43:0/1099243575 wait complete. 2024-09-19T05:03:54.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.772+0000 7ff132e4e740 20 client.24905 trim_cache size 0 max 16384 2024-09-19T05:03:54.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.791+0000 7ff132e4e740 1 Processor -- start 2024-09-19T05:03:54.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.791+0000 7ff132e4e740 1 -- start start 2024-09-19T05:03:54.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.792+0000 7ff132e4e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562fa293c330 0x562fa2938ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:54.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.792+0000 7ff132e4e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562fa293cc50 0x562fa293ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:54.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.792+0000 7ff132e4e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562fa2947510 0x562fa29417c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:54.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.792+0000 7ff132e4e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562fa2957ba0 con 0x562fa293c330 2024-09-19T05:03:54.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.792+0000 7ff132e4e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562fa2958270 con 0x562fa2947510 2024-09-19T05:03:54.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.792+0000 7ff132e4e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562fa2958940 con 0x562fa293cc50 2024-09-19T05:03:54.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.792+0000 7ff12e200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562fa293c330 0x562fa2938ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:54.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.792+0000 7ff12e200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562fa293c330 0x562fa2938ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55782/0 (socket says 172.21.15.43:55782) 2024-09-19T05:03:54.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.792+0000 7ff12e200640 1 -- 172.21.15.43:0/316720625 learned_addr learned my addr 172.21.15.43:0/316720625 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:54.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.792+0000 7ff12e200640 1 --2- 172.21.15.43:0/316720625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562fa2947510 0x562fa29417c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:54.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.792+0000 7ff12ce00640 1 --2- 172.21.15.43:0/316720625 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562fa293cc50 0x562fa293ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:54.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.792+0000 7ff12e200640 1 -- 172.21.15.43:0/316720625 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562fa293cc50 msgr2=0x562fa293ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:03:54.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.792+0000 7ff12e200640 1 --2- 172.21.15.43:0/316720625 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562fa293cc50 0x562fa293ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:54.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.792+0000 7ff12e200640 1 -- 172.21.15.43:0/316720625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562fa2947510 msgr2=0x562fa29417c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:03:54.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.792+0000 7ff12e200640 1 --2- 172.21.15.43:0/316720625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562fa2947510 0x562fa29417c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:54.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.792+0000 7ff12e200640 1 -- 172.21.15.43:0/316720625 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fa2942a90 con 0x562fa293c330 2024-09-19T05:03:54.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.793+0000 7ff12e200640 1 --2- 172.21.15.43:0/316720625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562fa293c330 0x562fa2938ca0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff120014040 tx=0x7ff120009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c43fc02d59ebd4a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:54.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.793+0000 7ff11a000640 1 -- 172.21.15.43:0/316720625 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff120003cb0 con 0x562fa293c330 2024-09-19T05:03:54.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.793+0000 7ff11a000640 1 -- 172.21.15.43:0/316720625 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff120004d80 con 0x562fa293c330 2024-09-19T05:03:54.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.793+0000 7ff11a000640 1 -- 172.21.15.43:0/316720625 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff120005780 con 0x562fa293c330 2024-09-19T05:03:54.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.794+0000 7ff132e4e740 1 -- 172.21.15.43:0/316720625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562fa293c330 msgr2=0x562fa2938ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:54.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.794+0000 7ff132e4e740 1 --2- 172.21.15.43:0/316720625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562fa293c330 0x562fa2938ca0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff120014040 tx=0x7ff120009cd0 comp rx=0 tx=0).stop 2024-09-19T05:03:54.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.794+0000 7ff132e4e740 1 -- 172.21.15.43:0/316720625 shutdown_connections 2024-09-19T05:03:54.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.794+0000 7ff132e4e740 1 --2- 172.21.15.43:0/316720625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562fa2947510 0x562fa29417c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:54.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.794+0000 7ff132e4e740 1 --2- 172.21.15.43:0/316720625 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562fa293cc50 0x562fa293ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:54.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.794+0000 7ff132e4e740 1 --2- 172.21.15.43:0/316720625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562fa293c330 0x562fa2938ca0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:54.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.794+0000 7ff132e4e740 1 -- 172.21.15.43:0/316720625 >> 172.21.15.43:0/316720625 conn(0x562fa29388c0 msgr2=0x562fa2942e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:54.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.794+0000 7ff132e4e740 1 -- 172.21.15.43:0/316720625 shutdown_connections 2024-09-19T05:03:54.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.794+0000 7ff132e4e740 1 -- 172.21.15.43:0/316720625 wait complete. 2024-09-19T05:03:54.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.794+0000 7ff132e4e740 1 -- start start 2024-09-19T05:03:54.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.794+0000 7ff132e4e740 1 Processor -- start 2024-09-19T05:03:54.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.795+0000 7ff132e4e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562fa293c330 0x562fa2961ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:54.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.795+0000 7ff132e4e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562fa293cc50 0x562fa2962000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:54.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.795+0000 7ff132e4e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562fa2947510 0x562fa295ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:54.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.795+0000 7ff132e4e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562fa29596e0 con 0x562fa293c330 2024-09-19T05:03:54.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.795+0000 7ff132e4e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562fa2959010 con 0x562fa2947510 2024-09-19T05:03:54.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.795+0000 7ff132e4e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562fa2959db0 con 0x562fa293cc50 2024-09-19T05:03:54.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.795+0000 7ff132e4e740 10 client.0 _ll_register_callbacks cb 0x7ff114004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:03:54.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.795+0000 7ff12e200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562fa293c330 0x562fa2961ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:54.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.795+0000 7ff12e200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562fa293c330 0x562fa2961ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55786/0 (socket says 172.21.15.43:55786) 2024-09-19T05:03:54.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.795+0000 7ff12e200640 1 -- 172.21.15.43:0/2048622724 learned_addr learned my addr 172.21.15.43:0/2048622724 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:54.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.795+0000 7ff12ce00640 1 --2- 172.21.15.43:0/2048622724 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562fa293cc50 0x562fa2962000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:54.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.795+0000 7ff12e200640 1 --2- 172.21.15.43:0/2048622724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562fa2947510 0x562fa295ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:54.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.796+0000 7ff12e200640 1 -- 172.21.15.43:0/2048622724 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562fa293cc50 msgr2=0x562fa2962000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:03:54.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.796+0000 7ff12e200640 1 --2- 172.21.15.43:0/2048622724 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562fa293cc50 0x562fa2962000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:54.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.796+0000 7ff12e200640 1 -- 172.21.15.43:0/2048622724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562fa2947510 msgr2=0x562fa295ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:03:54.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.796+0000 7ff12e200640 1 --2- 172.21.15.43:0/2048622724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562fa2947510 0x562fa295ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:54.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.796+0000 7ff12e200640 1 -- 172.21.15.43:0/2048622724 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fa295cfb0 con 0x562fa293c330 2024-09-19T05:03:54.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.796+0000 7ff12e200640 1 --2- 172.21.15.43:0/2048622724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562fa293c330 0x562fa2961ac0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff120011850 tx=0x7ff120011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7697d020f764ce6e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:54.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.796+0000 7ff10f400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:03:54.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.796+0000 7ff10f400640 1 -- 172.21.15.43:0/2048622724 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1200046a0 con 0x562fa293c330 2024-09-19T05:03:54.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.796+0000 7ff10f400640 1 -- 172.21.15.43:0/2048622724 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ff120004840 con 0x562fa293c330 2024-09-19T05:03:54.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.796+0000 7ff132e4e740 1 -- 172.21.15.43:0/2048622724 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff120004d80 con 0x562fa293c330 2024-09-19T05:03:54.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.796+0000 7ff10f400640 1 -- 172.21.15.43:0/2048622724 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff120004a40 con 0x562fa293c330 2024-09-19T05:03:54.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.796+0000 7ff132e4e740 10 client.14922 fetch_fsmap learned FSMap version 28 2024-09-19T05:03:54.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.796+0000 7ff132e4e740 1 -- 172.21.15.43:0/2048622724 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562fa29c3260 con 0x562fa293c330 2024-09-19T05:03:54.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.797+0000 7ff10f400640 1 -- 172.21.15.43:0/2048622724 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff120004d80 con 0x562fa293c330 2024-09-19T05:03:54.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.797+0000 7ff10f400640 1 -- 172.21.15.43:0/2048622724 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff1200286c0 con 0x562fa293c330 2024-09-19T05:03:54.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.797+0000 7ff132e4e740 10 client.14922 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:03:54.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.797+0000 7ff132e4e740 10 client.14922 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T05:03:54.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.797+0000 7ff132e4e740 20 client.14922 populate_metadata read hostname 'smithi043' 2024-09-19T05:03:54.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.797+0000 7ff132e4e740 1 --2- 172.21.15.43:0/2048622724 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x562fa29c76b0 0x562fa29c63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:54.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.797+0000 7ff132e4e740 4 client.14922 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T05:03:54.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.797+0000 7ff132e4e740 1 -- 172.21.15.43:0/2048622724 --> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] -- command(tid 0: {"prefix": "status"}) -- 0x562fa28a9b30 con 0x562fa29c76b0 2024-09-19T05:03:54.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.797+0000 7ff12ce00640 1 --2- 172.21.15.43:0/2048622724 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x562fa29c76b0 0x562fa29c63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:54.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.798+0000 7ff12ce00640 1 --2- 172.21.15.43:0/2048622724 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x562fa29c76b0 0x562fa29c63c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b103065e3724b3ee server_cookie=2b64b35892e2d0e4 in_seq=0 out_seq=0 2024-09-19T05:03:54.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.798+0000 7ff10f400640 10 client.14922 ms_handle_connect on v2:172.21.15.96:6836/3978388286 2024-09-19T05:03:54.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.798+0000 7ff10f400640 1 -- 172.21.15.43:0/2048622724 <== mds.3 v2:172.21.15.96:6836/3978388286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x562fa28a9b30 con 0x562fa29c76b0 2024-09-19T05:03:54.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.798+0000 7ff10f400640 10 client.14922 handle_command_reply: tid=0 2024-09-19T05:03:54.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.798+0000 7ff132e4e740 1 client.14922 shutdown 2024-09-19T05:03:54.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.798+0000 7ff132e4e740 1 -- 172.21.15.43:0/2048622724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562fa293c330 msgr2=0x562fa2961ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:54.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.798+0000 7ff132e4e740 1 --2- 172.21.15.43:0/2048622724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562fa293c330 0x562fa2961ac0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff120011850 tx=0x7ff120011e00 comp rx=0 tx=0).stop 2024-09-19T05:03:54.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.798+0000 7ff132e4e740 1 -- 172.21.15.43:0/2048622724 shutdown_connections 2024-09-19T05:03:54.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.799+0000 7ff132e4e740 1 --2- 172.21.15.43:0/2048622724 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x562fa29c76b0 0x562fa29c63c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:54.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.799+0000 7ff132e4e740 1 --2- 172.21.15.43:0/2048622724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562fa2947510 0x562fa295ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:54.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.799+0000 7ff10f400640 0 client.14922 ms_handle_reset on v2:172.21.15.96:6836/3978388286 2024-09-19T05:03:54.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.799+0000 7ff132e4e740 1 --2- 172.21.15.43:0/2048622724 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562fa293cc50 0x562fa2962000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:54.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.799+0000 7ff132e4e740 1 --2- 172.21.15.43:0/2048622724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562fa293c330 0x562fa2961ac0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:54.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.799+0000 7ff132e4e740 1 -- 172.21.15.43:0/2048622724 >> 172.21.15.43:0/2048622724 conn(0x562fa29388c0 msgr2=0x562fa2937200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:54.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.799+0000 7ff132e4e740 1 -- 172.21.15.43:0/2048622724 shutdown_connections 2024-09-19T05:03:54.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.799+0000 7ff132e4e740 1 -- 172.21.15.43:0/2048622724 wait complete. 2024-09-19T05:03:54.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.799+0000 7ff132e4e740 20 client.14922 trim_cache size 0 max 16384 2024-09-19T05:03:54.804 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:03:54.804 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "1456a810-7643-11ef-bceb-c7b262605968", 2024-09-19T05:03:54.804 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 3, 2024-09-19T05:03:54.804 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 24506, 2024-09-19T05:03:54.804 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-09-19T05:03:54.804 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-09-19T05:03:54.804 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-09-19T05:03:54.804 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 77.314765858000001, 2024-09-19T05:03:54.804 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 28, 2024-09-19T05:03:54.804 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 85, 2024-09-19T05:03:54.804 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 83, 2024-09-19T05:03:54.804 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 114.337789786 2024-09-19T05:03:54.804 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:03:54.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.800+0000 7ff130200640 1 -- 172.21.15.43:0/1945104495 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff1100a3350 msgr2=0x7ff1100a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:54.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.800+0000 7ff130200640 1 --2- 172.21.15.43:0/1945104495 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff1100a3350 0x7ff1100a5810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff11c09bfc0 tx=0x7ff11c0d38f0 comp rx=0 tx=0).stop 2024-09-19T05:03:54.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.800+0000 7ff130200640 1 -- 172.21.15.43:0/1945104495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff128101de0 msgr2=0x7ff1280744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:54.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.800+0000 7ff130200640 1 --2- 172.21.15.43:0/1945104495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff128101de0 0x7ff1280744f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00e9f0 tx=0x7ff11c00edf0 comp rx=0 tx=0).stop 2024-09-19T05:03:54.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.800+0000 7ff130200640 1 -- 172.21.15.43:0/1945104495 shutdown_connections 2024-09-19T05:03:54.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.800+0000 7ff130200640 1 --2- 172.21.15.43:0/1945104495 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff1100a3350 0x7ff1100a5810 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:54.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.800+0000 7ff130200640 1 --2- 172.21.15.43:0/1945104495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff128106f10 0x7ff128074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:54.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.800+0000 7ff130200640 1 --2- 172.21.15.43:0/1945104495 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff128102790 0x7ff128074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:54.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.800+0000 7ff130200640 1 --2- 172.21.15.43:0/1945104495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff128101de0 0x7ff1280744f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:54.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.800+0000 7ff130200640 1 -- 172.21.15.43:0/1945104495 >> 172.21.15.43:0/1945104495 conn(0x7ff1280fd8c0 msgr2=0x7ff1280feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:54.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.800+0000 7ff130200640 1 -- 172.21.15.43:0/1945104495 shutdown_connections 2024-09-19T05:03:54.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:54.800+0000 7ff130200640 1 -- 172.21.15.43:0/1945104495 wait complete. 2024-09-19T05:03:55.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:54 smithi043 ceph-mon[30800]: pgmap v338: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-19T05:03:55.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:54 smithi151 ceph-mon[35425]: pgmap v338: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-19T05:03:55.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:54 smithi096 ceph-mon[35085]: pgmap v338: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-19T05:03:55.714 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-19T05:03:56.092 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:56.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.668+0000 7f2371200640 1 -- 172.21.15.43:0/1401084746 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f236c103ff0 msgr2=0x7f236c1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:56.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.668+0000 7f2371200640 1 --2- 172.21.15.43:0/1401084746 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f236c103ff0 0x7f236c1043d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f235c009ca0 tx=0x7f235c02a8c0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.668+0000 7f2371200640 1 -- 172.21.15.43:0/1401084746 shutdown_connections 2024-09-19T05:03:56.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.668+0000 7f2371200640 1 --2- 172.21.15.43:0/1401084746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f236c109120 0x7f236c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.668+0000 7f2371200640 1 --2- 172.21.15.43:0/1401084746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f236c1049a0 0x7f236c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.668+0000 7f2371200640 1 --2- 172.21.15.43:0/1401084746 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f236c103ff0 0x7f236c1043d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.668+0000 7f2371200640 1 -- 172.21.15.43:0/1401084746 >> 172.21.15.43:0/1401084746 conn(0x7f236c0ffa90 msgr2=0x7f236c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:56.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.668+0000 7f2371200640 1 -- 172.21.15.43:0/1401084746 shutdown_connections 2024-09-19T05:03:56.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.668+0000 7f2371200640 1 -- 172.21.15.43:0/1401084746 wait complete. 2024-09-19T05:03:56.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.669+0000 7f2371200640 1 Processor -- start 2024-09-19T05:03:56.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.669+0000 7f2371200640 1 -- start start 2024-09-19T05:03:56.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.669+0000 7f2371200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f236c103ff0 0x7f236c19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:56.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.669+0000 7f2371200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f236c1049a0 0x7f236c19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:56.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.669+0000 7f2371200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f236c109120 0x7f236c19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:56.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.669+0000 7f2371200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f236c114120 con 0x7f236c1049a0 2024-09-19T05:03:56.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.669+0000 7f2371200640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f236c113fa0 con 0x7f236c109120 2024-09-19T05:03:56.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.669+0000 7f2371200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f236c1142a0 con 0x7f236c103ff0 2024-09-19T05:03:56.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.670+0000 7f2369e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f236c103ff0 0x7f236c19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:56.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.670+0000 7f2361400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f236c1049a0 0x7f236c19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:56.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.670+0000 7f2369e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f236c103ff0 0x7f236c19a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:59362/0 (socket says 172.21.15.43:59362) 2024-09-19T05:03:56.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.670+0000 7f2369e00640 1 -- 172.21.15.43:0/3033676384 learned_addr learned my addr 172.21.15.43:0/3033676384 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:56.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.670+0000 7f236a800640 1 --2- 172.21.15.43:0/3033676384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f236c109120 0x7f236c19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:56.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.670+0000 7f2369e00640 1 -- 172.21.15.43:0/3033676384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f236c109120 msgr2=0x7f236c19eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:56.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.670+0000 7f2369e00640 1 --2- 172.21.15.43:0/3033676384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f236c109120 0x7f236c19eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.670+0000 7f2369e00640 1 -- 172.21.15.43:0/3033676384 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f236c1049a0 msgr2=0x7f236c19ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:56.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.670+0000 7f2369e00640 1 --2- 172.21.15.43:0/3033676384 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f236c1049a0 0x7f236c19ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.670+0000 7f2369e00640 1 -- 172.21.15.43:0/3033676384 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f236c19f5d0 con 0x7f236c103ff0 2024-09-19T05:03:56.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.670+0000 7f236a800640 1 --2- 172.21.15.43:0/3033676384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f236c109120 0x7f236c19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:03:56.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.671+0000 7f2369e00640 1 --2- 172.21.15.43:0/3033676384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f236c103ff0 0x7f236c19a620 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f235c02ade0 tx=0x7f235c0319f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:56.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.671+0000 7f2363400640 1 -- 172.21.15.43:0/3033676384 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235c02fa20 con 0x7f236c103ff0 2024-09-19T05:03:56.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.671+0000 7f2371200640 1 -- 172.21.15.43:0/3033676384 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f236c19f860 con 0x7f236c103ff0 2024-09-19T05:03:56.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.671+0000 7f2363400640 1 -- 172.21.15.43:0/3033676384 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f235c02fbc0 con 0x7f236c103ff0 2024-09-19T05:03:56.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.671+0000 7f2363400640 1 -- 172.21.15.43:0/3033676384 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235c0467b0 con 0x7f236c103ff0 2024-09-19T05:03:56.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.671+0000 7f2371200640 1 -- 172.21.15.43:0/3033676384 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f236c1a71a0 con 0x7f236c103ff0 2024-09-19T05:03:56.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.674+0000 7f2373e22740 1 Processor -- start 2024-09-19T05:03:56.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.674+0000 7f2373e22740 1 -- start start 2024-09-19T05:03:56.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.674+0000 7f2363400640 1 -- 172.21.15.43:0/3033676384 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f235c046950 con 0x7f236c103ff0 2024-09-19T05:03:56.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.675+0000 7f2373e22740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557588a91330 0x557588a91710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:56.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.675+0000 7f2373e22740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557588a91c50 0x557588a9bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:56.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.675+0000 7f2373e22740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557588a9c510 0x557588a9e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:56.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.675+0000 7f2373e22740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5575889f70b0 con 0x557588a9c510 2024-09-19T05:03:56.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.675+0000 7f2373e22740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55758899eab0 con 0x557588a91c50 2024-09-19T05:03:56.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.675+0000 7f2373e22740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5575889f3070 con 0x557588a91330 2024-09-19T05:03:56.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.675+0000 7f2361400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557588a9c510 0x557588a9e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:56.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.675+0000 7f236a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557588a91330 0x557588a91710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:56.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.675+0000 7f236a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557588a91330 0x557588a91710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:59366/0 (socket says 172.21.15.43:59366) 2024-09-19T05:03:56.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.675+0000 7f236a800640 1 -- 172.21.15.43:0/2734355671 learned_addr learned my addr 172.21.15.43:0/2734355671 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:56.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.675+0000 7f2369e00640 1 --2- 172.21.15.43:0/2734355671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557588a91c50 0x557588a9bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:56.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.675+0000 7f236a800640 1 -- 172.21.15.43:0/2734355671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557588a91c50 msgr2=0x557588a9bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:03:56.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.675+0000 7f236a800640 1 --2- 172.21.15.43:0/2734355671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557588a91c50 0x557588a9bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.675+0000 7f236a800640 1 -- 172.21.15.43:0/2734355671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557588a9c510 msgr2=0x557588a9e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:03:56.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.675+0000 7f236a800640 1 --2- 172.21.15.43:0/2734355671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557588a9c510 0x557588a9e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.675+0000 7f236a800640 1 -- 172.21.15.43:0/2734355671 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575889ab4a0 con 0x557588a91330 2024-09-19T05:03:56.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.675+0000 7f2361400640 1 --2- 172.21.15.43:0/2734355671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557588a9c510 0x557588a9e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:03:56.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.676+0000 7f236a800640 1 --2- 172.21.15.43:0/2734355671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557588a91330 0x557588a91710 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f236400bca0 tx=0x7f236400f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=759fb9693c0e55e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:56.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.676+0000 7f235b400640 1 -- 172.21.15.43:0/2734355671 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2364015070 con 0x557588a91330 2024-09-19T05:03:56.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.676+0000 7f2363400640 1 --2- 172.21.15.43:0/3033676384 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f23500a3350 0x7f23500a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:56.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.676+0000 7f235b400640 1 -- 172.21.15.43:0/2734355671 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2364005500 con 0x557588a91330 2024-09-19T05:03:56.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.676+0000 7f235b400640 1 -- 172.21.15.43:0/2734355671 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23640116a0 con 0x557588a91330 2024-09-19T05:03:56.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.676+0000 7f2363400640 1 -- 172.21.15.43:0/3033676384 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f235c0f0e50 con 0x7f236c103ff0 2024-09-19T05:03:56.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.676+0000 7f2369e00640 1 --2- 172.21.15.43:0/3033676384 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f23500a3350 0x7f23500a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:56.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.677+0000 7f2373e22740 1 -- 172.21.15.43:0/2734355671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557588a91330 msgr2=0x557588a91710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:56.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.677+0000 7f2373e22740 1 --2- 172.21.15.43:0/2734355671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557588a91330 0x557588a91710 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f236400bca0 tx=0x7f236400f730 comp rx=0 tx=0).stop 2024-09-19T05:03:56.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.677+0000 7f2369e00640 1 --2- 172.21.15.43:0/3033676384 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f23500a3350 0x7f23500a5810 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f235c0b98d0 tx=0x7f235c0b9040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:56.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.677+0000 7f2373e22740 1 -- 172.21.15.43:0/2734355671 shutdown_connections 2024-09-19T05:03:56.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.677+0000 7f2373e22740 1 --2- 172.21.15.43:0/2734355671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557588a9c510 0x557588a9e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.677+0000 7f2373e22740 1 --2- 172.21.15.43:0/2734355671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557588a91c50 0x557588a9bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.677+0000 7f2373e22740 1 --2- 172.21.15.43:0/2734355671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557588a91330 0x557588a91710 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.677+0000 7f2373e22740 1 -- 172.21.15.43:0/2734355671 >> 172.21.15.43:0/2734355671 conn(0x557588a8d8c0 msgr2=0x557588a8dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:56.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.677+0000 7f2373e22740 1 -- 172.21.15.43:0/2734355671 shutdown_connections 2024-09-19T05:03:56.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.677+0000 7f2373e22740 1 -- 172.21.15.43:0/2734355671 wait complete. 2024-09-19T05:03:56.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.678+0000 7f2373e22740 1 -- start start 2024-09-19T05:03:56.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.678+0000 7f2373e22740 1 Processor -- start 2024-09-19T05:03:56.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.678+0000 7f2373e22740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557588a91330 0x557588a99560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:56.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.678+0000 7f2373e22740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557588a91c50 0x557588a99aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:56.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.678+0000 7f2373e22740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557588a9c510 0x557588a99fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:56.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.678+0000 7f2373e22740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557588a9f060 con 0x557588a91c50 2024-09-19T05:03:56.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.678+0000 7f2373e22740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55758897f920 con 0x557588a91330 2024-09-19T05:03:56.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.678+0000 7f2373e22740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557588a9f1e0 con 0x557588a9c510 2024-09-19T05:03:56.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.678+0000 7f2373e22740 10 client.0 _ll_register_callbacks cb 0x5575889a38a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:03:56.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.679+0000 7f236a800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557588a91330 0x557588a99560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:56.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.679+0000 7f236a800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557588a91330 0x557588a99560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:40546/0 (socket says 172.21.15.43:40546) 2024-09-19T05:03:56.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.679+0000 7f236a800640 1 -- 172.21.15.43:0/532875188 learned_addr learned my addr 172.21.15.43:0/532875188 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:56.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.679+0000 7f2361400640 1 --2- 172.21.15.43:0/532875188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557588a91c50 0x557588a99aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:56.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.679+0000 7f236a800640 1 --2- 172.21.15.43:0/532875188 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557588a9c510 0x557588a99fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:56.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.679+0000 7f236a800640 1 -- 172.21.15.43:0/532875188 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557588a9c510 msgr2=0x557588a99fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:03:56.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.679+0000 7f236a800640 1 --2- 172.21.15.43:0/532875188 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557588a9c510 0x557588a99fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.679+0000 7f236a800640 1 -- 172.21.15.43:0/532875188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557588a91c50 msgr2=0x557588a99aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:03:56.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.679+0000 7f236a800640 1 --2- 172.21.15.43:0/532875188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557588a91c50 0x557588a99aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.679+0000 7f236a800640 1 -- 172.21.15.43:0/532875188 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575887682b0 con 0x557588a91330 2024-09-19T05:03:56.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.679+0000 7f236a800640 1 --2- 172.21.15.43:0/532875188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557588a91330 0x557588a99560 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2364005c80 tx=0x7f2364004ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7748ead4cb40e4e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:56.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.679+0000 7f234fe00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:03:56.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.679+0000 7f234fe00640 1 -- 172.21.15.43:0/532875188 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2364015070 con 0x557588a91330 2024-09-19T05:03:56.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.680+0000 7f2373e22740 1 -- 172.21.15.43:0/532875188 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2364005500 con 0x557588a91330 2024-09-19T05:03:56.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.680+0000 7f234fe00640 1 -- 172.21.15.43:0/532875188 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f2364005150 con 0x557588a91330 2024-09-19T05:03:56.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.680+0000 7f234fe00640 1 -- 172.21.15.43:0/532875188 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f236402f050 con 0x557588a91330 2024-09-19T05:03:56.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.680+0000 7f2373e22740 10 client.24923 fetch_fsmap learned FSMap version 28 2024-09-19T05:03:56.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.680+0000 7f2373e22740 1 -- 172.21.15.43:0/532875188 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557588aaadf0 con 0x557588a91330 2024-09-19T05:03:56.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.680+0000 7f234fe00640 1 -- 172.21.15.43:0/532875188 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2364005500 con 0x557588a91330 2024-09-19T05:03:56.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.680+0000 7f234fe00640 1 -- 172.21.15.43:0/532875188 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2364027020 con 0x557588a91330 2024-09-19T05:03:56.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.681+0000 7f2373e22740 10 client.24923 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:03:56.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.681+0000 7f2373e22740 10 client.24923 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T05:03:56.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.681+0000 7f2373e22740 20 client.24923 populate_metadata read hostname 'smithi043' 2024-09-19T05:03:56.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.681+0000 7f2373e22740 1 --2- 172.21.15.43:0/532875188 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x557588aad0d0 0x557588aae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:56.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.681+0000 7f2373e22740 4 client.24923 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:03:56.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.681+0000 7f2373e22740 1 -- 172.21.15.43:0/532875188 --> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557588a04620 con 0x557588aad0d0 2024-09-19T05:03:56.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.681+0000 7f2361400640 1 --2- 172.21.15.43:0/532875188 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x557588aad0d0 0x557588aae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:56.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.682+0000 7f2361400640 1 --2- 172.21.15.43:0/532875188 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x557588aad0d0 0x557588aae540 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=48fbe8bb80ea8a29 server_cookie=7e5898badbc5a0e6 in_seq=0 out_seq=0 2024-09-19T05:03:56.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.682+0000 7f234fe00640 10 client.24923 ms_handle_connect on v2:172.21.15.151:6838/3794104313 2024-09-19T05:03:56.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.683+0000 7f234fe00640 1 -- 172.21.15.43:0/532875188 <== mds.2 v2:172.21.15.151:6838/3794104313 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557588a04620 con 0x557588aad0d0 2024-09-19T05:03:56.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.683+0000 7f234fe00640 10 client.24923 handle_command_reply: tid=0 2024-09-19T05:03:56.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.683+0000 7f2373e22740 1 client.24923 shutdown 2024-09-19T05:03:56.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.683+0000 7f2373e22740 1 -- 172.21.15.43:0/532875188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557588a91330 msgr2=0x557588a99560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:56.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.683+0000 7f2373e22740 1 --2- 172.21.15.43:0/532875188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557588a91330 0x557588a99560 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2364005c80 tx=0x7f2364004ec0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.683+0000 7f2373e22740 1 -- 172.21.15.43:0/532875188 shutdown_connections 2024-09-19T05:03:56.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.683+0000 7f2373e22740 1 --2- 172.21.15.43:0/532875188 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x557588aad0d0 0x557588aae540 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.683+0000 7f2373e22740 1 --2- 172.21.15.43:0/532875188 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557588a9c510 0x557588a99fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.683+0000 7f2373e22740 1 --2- 172.21.15.43:0/532875188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557588a91c50 0x557588a99aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.683+0000 7f2373e22740 1 --2- 172.21.15.43:0/532875188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557588a91330 0x557588a99560 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.683+0000 7f234fe00640 0 client.24923 ms_handle_reset on v2:172.21.15.151:6838/3794104313 2024-09-19T05:03:56.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.683+0000 7f2373e22740 1 -- 172.21.15.43:0/532875188 >> 172.21.15.43:0/532875188 conn(0x557588a8d8c0 msgr2=0x557588a935a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:56.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.684+0000 7f2373e22740 1 -- 172.21.15.43:0/532875188 shutdown_connections 2024-09-19T05:03:56.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.684+0000 7f2373e22740 1 -- 172.21.15.43:0/532875188 wait complete. 2024-09-19T05:03:56.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.684+0000 7f2373e22740 20 client.24923 trim_cache size 0 max 16384 2024-09-19T05:03:56.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.703+0000 7f2373e22740 1 Processor -- start 2024-09-19T05:03:56.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.703+0000 7f2373e22740 1 -- start start 2024-09-19T05:03:56.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.703+0000 7f2373e22740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557588a91330 0x557588aaca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:56.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.703+0000 7f2373e22740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557588a91c50 0x557588a98e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:56.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.703+0000 7f2373e22740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557588a9c510 0x557588a96820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:56.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.703+0000 7f2373e22740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557588aac880 con 0x557588a91330 2024-09-19T05:03:56.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.703+0000 7f2373e22740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557588aacf50 con 0x557588a91c50 2024-09-19T05:03:56.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.703+0000 7f2373e22740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557588aad620 con 0x557588a9c510 2024-09-19T05:03:56.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.703+0000 7f236a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557588a91330 0x557588aaca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:56.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.704+0000 7f236a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557588a91330 0x557588aaca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38640/0 (socket says 172.21.15.43:38640) 2024-09-19T05:03:56.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.704+0000 7f236a800640 1 -- 172.21.15.43:0/724795033 learned_addr learned my addr 172.21.15.43:0/724795033 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:56.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.704+0000 7f2361400640 1 --2- 172.21.15.43:0/724795033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557588a91c50 0x557588a98e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:56.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.704+0000 7f236a800640 1 -- 172.21.15.43:0/724795033 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557588a9c510 msgr2=0x557588a96820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:03:56.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.704+0000 7f236a800640 1 --2- 172.21.15.43:0/724795033 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557588a9c510 0x557588a96820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.704+0000 7f236a800640 1 -- 172.21.15.43:0/724795033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557588a91c50 msgr2=0x557588a98e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:03:56.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.704+0000 7f236a800640 1 --2- 172.21.15.43:0/724795033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557588a91c50 0x557588a98e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.704+0000 7f236a800640 1 -- 172.21.15.43:0/724795033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557588a8e550 con 0x557588a91330 2024-09-19T05:03:56.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.704+0000 7f2361400640 1 --2- 172.21.15.43:0/724795033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557588a91c50 0x557588a98e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:03:56.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.704+0000 7f236a800640 1 --2- 172.21.15.43:0/724795033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557588a91330 0x557588aaca00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2364014040 tx=0x7f236400fd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28421bf5776b620f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:56.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.704+0000 7f235aa00640 1 -- 172.21.15.43:0/724795033 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2364038070 con 0x557588a91330 2024-09-19T05:03:56.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.705+0000 7f235aa00640 1 -- 172.21.15.43:0/724795033 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f236401e020 con 0x557588a91330 2024-09-19T05:03:56.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.705+0000 7f235aa00640 1 -- 172.21.15.43:0/724795033 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2364015040 con 0x557588a91330 2024-09-19T05:03:56.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.705+0000 7f2373e22740 1 -- 172.21.15.43:0/724795033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557588a91330 msgr2=0x557588aaca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:56.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.705+0000 7f2373e22740 1 --2- 172.21.15.43:0/724795033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557588a91330 0x557588aaca00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2364014040 tx=0x7f236400fd60 comp rx=0 tx=0).stop 2024-09-19T05:03:56.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.705+0000 7f2373e22740 1 -- 172.21.15.43:0/724795033 shutdown_connections 2024-09-19T05:03:56.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.705+0000 7f2373e22740 1 --2- 172.21.15.43:0/724795033 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557588a9c510 0x557588a96820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.705+0000 7f2373e22740 1 --2- 172.21.15.43:0/724795033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557588a91c50 0x557588a98e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.705+0000 7f2373e22740 1 --2- 172.21.15.43:0/724795033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557588a91330 0x557588aaca00 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.705+0000 7f2373e22740 1 -- 172.21.15.43:0/724795033 >> 172.21.15.43:0/724795033 conn(0x557588a8d8c0 msgr2=0x557588a97e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:56.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.705+0000 7f2373e22740 1 -- 172.21.15.43:0/724795033 shutdown_connections 2024-09-19T05:03:56.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.705+0000 7f2373e22740 1 -- 172.21.15.43:0/724795033 wait complete. 2024-09-19T05:03:56.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.706+0000 7f2373e22740 1 -- start start 2024-09-19T05:03:56.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.706+0000 7f2373e22740 1 Processor -- start 2024-09-19T05:03:56.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.706+0000 7f2373e22740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557588a91330 0x557588a96720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:56.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.706+0000 7f2373e22740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557588a91c50 0x557588a96c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:56.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.706+0000 7f2373e22740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557588a9c510 0x557588a971a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:56.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.706+0000 7f2373e22740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557588aae3c0 con 0x557588a91c50 2024-09-19T05:03:56.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.706+0000 7f2373e22740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557588aadcf0 con 0x557588a9c510 2024-09-19T05:03:56.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.706+0000 7f2373e22740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557588aaea90 con 0x557588a91330 2024-09-19T05:03:56.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.707+0000 7f2373e22740 10 client.0 _ll_register_callbacks cb 0x7f236400b900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:03:56.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.707+0000 7f2361400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557588a91c50 0x557588a96c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:56.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.707+0000 7f2361400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557588a91c50 0x557588a96c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38650/0 (socket says 172.21.15.43:38650) 2024-09-19T05:03:56.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.707+0000 7f2361400640 1 -- 172.21.15.43:0/265571147 learned_addr learned my addr 172.21.15.43:0/265571147 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:56.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.707+0000 7f236a800640 1 --2- 172.21.15.43:0/265571147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557588a91330 0x557588a96720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:56.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.707+0000 7f236a800640 1 --2- 172.21.15.43:0/265571147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557588a9c510 0x557588a971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:56.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.707+0000 7f2361400640 1 -- 172.21.15.43:0/265571147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557588a91330 msgr2=0x557588a96720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:03:56.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.707+0000 7f2361400640 1 --2- 172.21.15.43:0/265571147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557588a91330 0x557588a96720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.707+0000 7f2361400640 1 -- 172.21.15.43:0/265571147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557588a9c510 msgr2=0x557588a971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:03:56.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.707+0000 7f2361400640 1 --2- 172.21.15.43:0/265571147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557588a9c510 0x557588a971a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.707+0000 7f2361400640 1 -- 172.21.15.43:0/265571147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557588ab4f50 con 0x557588a91c50 2024-09-19T05:03:56.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.707+0000 7f236a800640 1 --2- 172.21.15.43:0/265571147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557588a9c510 0x557588a971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:03:56.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.708+0000 7f2361400640 1 --2- 172.21.15.43:0/265571147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557588a91c50 0x557588a96c60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f235400b740 tx=0x7f235400bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=498905e7dfa2bf17 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:56.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.708+0000 7f234fe00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:03:56.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.708+0000 7f234fe00640 1 -- 172.21.15.43:0/265571147 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2354011440 con 0x557588a91c50 2024-09-19T05:03:56.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.708+0000 7f234fe00640 1 -- 172.21.15.43:0/265571147 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f23540115e0 con 0x557588a91c50 2024-09-19T05:03:56.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.708+0000 7f2373e22740 1 -- 172.21.15.43:0/265571147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f236401e020 con 0x557588a91c50 2024-09-19T05:03:56.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.708+0000 7f234fe00640 1 -- 172.21.15.43:0/265571147 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f23540117e0 con 0x557588a91c50 2024-09-19T05:03:56.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.708+0000 7f2373e22740 10 client.14940 fetch_fsmap learned FSMap version 28 2024-09-19T05:03:56.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.708+0000 7f2373e22740 1 -- 172.21.15.43:0/265571147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557588b17be0 con 0x557588a91c50 2024-09-19T05:03:56.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.709+0000 7f234fe00640 1 -- 172.21.15.43:0/265571147 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f236401e020 con 0x557588a91c50 2024-09-19T05:03:56.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.709+0000 7f234fe00640 1 -- 172.21.15.43:0/265571147 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2354026b30 con 0x557588a91c50 2024-09-19T05:03:56.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.709+0000 7f2373e22740 10 client.14940 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:03:56.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.709+0000 7f2373e22740 10 client.14940 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T05:03:56.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.709+0000 7f2373e22740 20 client.14940 populate_metadata read hostname 'smithi043' 2024-09-19T05:03:56.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.709+0000 7f2373e22740 1 --2- 172.21.15.43:0/265571147 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x557588b1c840 0x557588b1b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:56.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.709+0000 7f2373e22740 4 client.14940 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T05:03:56.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.709+0000 7f2373e22740 1 -- 172.21.15.43:0/265571147 --> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] -- command(tid 0: {"prefix": "status"}) -- 0x5575889feb30 con 0x557588b1c840 2024-09-19T05:03:56.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.710+0000 7f236a800640 1 --2- 172.21.15.43:0/265571147 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x557588b1c840 0x557588b1b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:56.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.710+0000 7f236a800640 1 --2- 172.21.15.43:0/265571147 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x557588b1c840 0x557588b1b550 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a60d5f95d7c9d93b server_cookie=bea70a7216a46d5a in_seq=0 out_seq=0 2024-09-19T05:03:56.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.710+0000 7f234fe00640 10 client.14940 ms_handle_connect on v2:172.21.15.151:6838/3794104313 2024-09-19T05:03:56.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.710+0000 7f234fe00640 1 -- 172.21.15.43:0/265571147 <== mds.2 v2:172.21.15.151:6838/3794104313 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5575889feb30 con 0x557588b1c840 2024-09-19T05:03:56.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.710+0000 7f234fe00640 10 client.14940 handle_command_reply: tid=0 2024-09-19T05:03:56.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.710+0000 7f2373e22740 1 client.14940 shutdown 2024-09-19T05:03:56.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.711+0000 7f2373e22740 1 -- 172.21.15.43:0/265571147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557588a91c50 msgr2=0x557588a96c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:56.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.711+0000 7f2373e22740 1 --2- 172.21.15.43:0/265571147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557588a91c50 0x557588a96c60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f235400b740 tx=0x7f235400bee0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.711+0000 7f2373e22740 1 -- 172.21.15.43:0/265571147 shutdown_connections 2024-09-19T05:03:56.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.711+0000 7f2373e22740 1 --2- 172.21.15.43:0/265571147 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x557588b1c840 0x557588b1b550 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.711+0000 7f2373e22740 1 --2- 172.21.15.43:0/265571147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557588a9c510 0x557588a971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.711+0000 7f2373e22740 1 --2- 172.21.15.43:0/265571147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557588a91c50 0x557588a96c60 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.711+0000 7f2373e22740 1 --2- 172.21.15.43:0/265571147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557588a91330 0x557588a96720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.711+0000 7f234fe00640 0 client.14940 ms_handle_reset on v2:172.21.15.151:6838/3794104313 2024-09-19T05:03:56.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.711+0000 7f2373e22740 1 -- 172.21.15.43:0/265571147 >> 172.21.15.43:0/265571147 conn(0x557588a8d8c0 msgr2=0x557588aa7430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:56.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.711+0000 7f2373e22740 1 -- 172.21.15.43:0/265571147 shutdown_connections 2024-09-19T05:03:56.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.711+0000 7f2373e22740 1 -- 172.21.15.43:0/265571147 wait complete. 2024-09-19T05:03:56.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.711+0000 7f2373e22740 20 client.14940 trim_cache size 0 max 16384 2024-09-19T05:03:56.717 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:03:56.717 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "1456a810-7643-11ef-bceb-c7b262605968", 2024-09-19T05:03:56.717 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 2, 2024-09-19T05:03:56.717 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 24511, 2024-09-19T05:03:56.717 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-09-19T05:03:56.717 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-09-19T05:03:56.717 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-09-19T05:03:56.717 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 80.235960844999994, 2024-09-19T05:03:56.717 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 28, 2024-09-19T05:03:56.717 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 85, 2024-09-19T05:03:56.717 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 83, 2024-09-19T05:03:56.717 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 113.56746923199999 2024-09-19T05:03:56.717 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:03:56.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.712+0000 7f2371200640 1 -- 172.21.15.43:0/3033676384 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f23500a3350 msgr2=0x7f23500a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:56.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.712+0000 7f2371200640 1 --2- 172.21.15.43:0/3033676384 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f23500a3350 0x7f23500a5810 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f235c0b98d0 tx=0x7f235c0b9040 comp rx=0 tx=0).stop 2024-09-19T05:03:56.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.712+0000 7f2371200640 1 -- 172.21.15.43:0/3033676384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f236c103ff0 msgr2=0x7f236c19a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:56.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.712+0000 7f2371200640 1 --2- 172.21.15.43:0/3033676384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f236c103ff0 0x7f236c19a620 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f235c02ade0 tx=0x7f235c0319f0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.712+0000 7f2371200640 1 -- 172.21.15.43:0/3033676384 shutdown_connections 2024-09-19T05:03:56.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.712+0000 7f2371200640 1 --2- 172.21.15.43:0/3033676384 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f23500a3350 0x7f23500a5810 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.712+0000 7f2371200640 1 --2- 172.21.15.43:0/3033676384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f236c109120 0x7f236c19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.712+0000 7f2371200640 1 --2- 172.21.15.43:0/3033676384 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f236c1049a0 0x7f236c19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.712+0000 7f2371200640 1 --2- 172.21.15.43:0/3033676384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f236c103ff0 0x7f236c19a620 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:56.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.712+0000 7f2371200640 1 -- 172.21.15.43:0/3033676384 >> 172.21.15.43:0/3033676384 conn(0x7f236c0ffa90 msgr2=0x7f236c100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:56.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.712+0000 7f2371200640 1 -- 172.21.15.43:0/3033676384 shutdown_connections 2024-09-19T05:03:56.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:56.712+0000 7f2371200640 1 -- 172.21.15.43:0/3033676384 wait complete. 2024-09-19T05:03:57.124 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:56 smithi043 ceph-mon[30800]: pgmap v339: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:03:57.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:56 smithi151 ceph-mon[35425]: pgmap v339: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:03:57.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:56 smithi096 ceph-mon[35085]: pgmap v339: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:03:57.600 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.g status 2024-09-19T05:03:57.981 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:03:58.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.532+0000 7fae6e800640 1 -- 172.21.15.43:0/367586887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae68112a30 msgr2=0x7fae68114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:58.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.532+0000 7fae6e800640 1 --2- 172.21.15.43:0/367586887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae68112a30 0x7fae68114e20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fae60009ca0 tx=0x7fae6002a8c0 comp rx=0 tx=0).stop 2024-09-19T05:03:58.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.532+0000 7fae6e800640 1 -- 172.21.15.43:0/367586887 shutdown_connections 2024-09-19T05:03:58.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.532+0000 7fae6e800640 1 --2- 172.21.15.43:0/367586887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae68112a30 0x7fae68114e20 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:58.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.532+0000 7fae6e800640 1 --2- 172.21.15.43:0/367586887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae68077040 0x7fae681124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:58.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.532+0000 7fae6e800640 1 --2- 172.21.15.43:0/367586887 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae68076720 0x7fae68076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:58.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.532+0000 7fae6e800640 1 -- 172.21.15.43:0/367586887 >> 172.21.15.43:0/367586887 conn(0x7fae680ffa90 msgr2=0x7fae68101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:58.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.532+0000 7fae6e800640 1 -- 172.21.15.43:0/367586887 shutdown_connections 2024-09-19T05:03:58.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.532+0000 7fae6e800640 1 -- 172.21.15.43:0/367586887 wait complete. 2024-09-19T05:03:58.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.534+0000 7fae6e800640 1 Processor -- start 2024-09-19T05:03:58.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.534+0000 7fae6e800640 1 -- start start 2024-09-19T05:03:58.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.535+0000 7fae6e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae68076720 0x7fae681a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:58.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.535+0000 7fae6e800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae68077040 0x7fae681a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:58.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.535+0000 7fae6e800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae68112a30 0x7fae6819e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:58.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.535+0000 7fae6e800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fae681185c0 con 0x7fae68076720 2024-09-19T05:03:58.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.535+0000 7fae6e800640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fae68118440 con 0x7fae68077040 2024-09-19T05:03:58.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.535+0000 7fae6e800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fae68118740 con 0x7fae68112a30 2024-09-19T05:03:58.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.535+0000 7fae5fe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae68077040 0x7fae681a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:58.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.535+0000 7fae5fe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae68077040 0x7fae681a4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:40594/0 (socket says 172.21.15.43:40594) 2024-09-19T05:03:58.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.535+0000 7fae5fe00640 1 -- 172.21.15.43:0/3024253248 learned_addr learned my addr 172.21.15.43:0/3024253248 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:58.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.535+0000 7fae67e00640 1 --2- 172.21.15.43:0/3024253248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae68112a30 0x7fae6819e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:58.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.535+0000 7fae67400640 1 --2- 172.21.15.43:0/3024253248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae68076720 0x7fae681a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:58.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.535+0000 7fae5fe00640 1 -- 172.21.15.43:0/3024253248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae68112a30 msgr2=0x7fae6819e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:58.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.535+0000 7fae5fe00640 1 --2- 172.21.15.43:0/3024253248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae68112a30 0x7fae6819e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:58.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.536+0000 7fae5fe00640 1 -- 172.21.15.43:0/3024253248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae68076720 msgr2=0x7fae681a46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:58.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.536+0000 7fae5fe00640 1 --2- 172.21.15.43:0/3024253248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae68076720 0x7fae681a46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:58.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.536+0000 7fae5fe00640 1 -- 172.21.15.43:0/3024253248 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae6819ef90 con 0x7fae68077040 2024-09-19T05:03:58.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.536+0000 7fae67400640 1 --2- 172.21.15.43:0/3024253248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae68076720 0x7fae681a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:03:58.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.536+0000 7fae5fe00640 1 --2- 172.21.15.43:0/3024253248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae68077040 0x7fae681a4be0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fae5000e9f0 tx=0x7fae5000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:58.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.536+0000 7fae64c00640 1 -- 172.21.15.43:0/3024253248 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5000cd80 con 0x7fae68077040 2024-09-19T05:03:58.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.536+0000 7fae6e800640 1 -- 172.21.15.43:0/3024253248 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae6819f280 con 0x7fae68077040 2024-09-19T05:03:58.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.536+0000 7fae64c00640 1 -- 172.21.15.43:0/3024253248 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae50010910 con 0x7fae68077040 2024-09-19T05:03:58.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.536+0000 7fae64c00640 1 -- 172.21.15.43:0/3024253248 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae500188f0 con 0x7fae68077040 2024-09-19T05:03:58.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.536+0000 7fae6e800640 1 -- 172.21.15.43:0/3024253248 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae681ab570 con 0x7fae68077040 2024-09-19T05:03:58.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.540+0000 7fae7125b740 1 Processor -- start 2024-09-19T05:03:58.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.541+0000 7fae7125b740 1 -- start start 2024-09-19T05:03:58.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.541+0000 7fae7125b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626d1703330 0x5626d1703710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:58.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.541+0000 7fae7125b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d1703c50 0x5626d170dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:58.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.541+0000 7fae7125b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626d170e510 0x5626d1710900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:58.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.541+0000 7fae7125b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5626d16690b0 con 0x5626d1703330 2024-09-19T05:03:58.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.541+0000 7fae7125b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5626d1610ab0 con 0x5626d1703c50 2024-09-19T05:03:58.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.541+0000 7fae7125b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5626d1665070 con 0x5626d170e510 2024-09-19T05:03:58.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.541+0000 7fae64c00640 1 -- 172.21.15.43:0/3024253248 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fae50010ac0 con 0x7fae68077040 2024-09-19T05:03:58.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.541+0000 7fae67e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626d1703330 0x5626d1703710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:58.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.541+0000 7fae67400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d1703c50 0x5626d170dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:58.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.542+0000 7fae67400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d1703c50 0x5626d170dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:40600/0 (socket says 172.21.15.43:40600) 2024-09-19T05:03:58.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.542+0000 7fae67400640 1 -- 172.21.15.43:0/3225249316 learned_addr learned my addr 172.21.15.43:0/3225249316 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:58.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.542+0000 7fae5fe00640 1 --2- 172.21.15.43:0/3225249316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626d170e510 0x5626d1710900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:58.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.542+0000 7fae67400640 1 -- 172.21.15.43:0/3225249316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626d170e510 msgr2=0x5626d1710900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:03:58.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.542+0000 7fae67400640 1 --2- 172.21.15.43:0/3225249316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626d170e510 0x5626d1710900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:58.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.542+0000 7fae67400640 1 -- 172.21.15.43:0/3225249316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626d1703330 msgr2=0x5626d1703710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:03:58.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.542+0000 7fae67400640 1 --2- 172.21.15.43:0/3225249316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626d1703330 0x5626d1703710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:58.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.542+0000 7fae67400640 1 -- 172.21.15.43:0/3225249316 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626d161d4a0 con 0x5626d1703c50 2024-09-19T05:03:58.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.542+0000 7fae5fe00640 1 --2- 172.21.15.43:0/3225249316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626d170e510 0x5626d1710900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:03:58.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.542+0000 7fae67e00640 1 --2- 172.21.15.43:0/3225249316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626d1703330 0x5626d1703710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:03:58.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.542+0000 7fae67400640 1 --2- 172.21.15.43:0/3225249316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d1703c50 0x5626d170dfd0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fae58010450 tx=0x7fae58010910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=798d8fe8939157a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:58.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.543+0000 7fae5cc00640 1 -- 172.21.15.43:0/3225249316 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae58004220 con 0x5626d1703c50 2024-09-19T05:03:58.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.543+0000 7fae5cc00640 1 -- 172.21.15.43:0/3225249316 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae580043c0 con 0x5626d1703c50 2024-09-19T05:03:58.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.543+0000 7fae5cc00640 1 -- 172.21.15.43:0/3225249316 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae58011b70 con 0x5626d1703c50 2024-09-19T05:03:58.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.543+0000 7fae64c00640 1 --2- 172.21.15.43:0/3024253248 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fae4c0a3350 0x7fae4c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:58.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.543+0000 7fae64c00640 1 -- 172.21.15.43:0/3024253248 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fae500cfd80 con 0x7fae68077040 2024-09-19T05:03:58.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.543+0000 7fae67e00640 1 --2- 172.21.15.43:0/3024253248 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fae4c0a3350 0x7fae4c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:58.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.544+0000 7fae7125b740 1 -- 172.21.15.43:0/3225249316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d1703c50 msgr2=0x5626d170dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:58.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.544+0000 7fae7125b740 1 --2- 172.21.15.43:0/3225249316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d1703c50 0x5626d170dfd0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fae58010450 tx=0x7fae58010910 comp rx=0 tx=0).stop 2024-09-19T05:03:58.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.544+0000 7fae7125b740 1 -- 172.21.15.43:0/3225249316 shutdown_connections 2024-09-19T05:03:58.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.544+0000 7fae7125b740 1 --2- 172.21.15.43:0/3225249316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626d170e510 0x5626d1710900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:58.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.544+0000 7fae7125b740 1 --2- 172.21.15.43:0/3225249316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d1703c50 0x5626d170dfd0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:58.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.544+0000 7fae7125b740 1 --2- 172.21.15.43:0/3225249316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626d1703330 0x5626d1703710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:58.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.544+0000 7fae7125b740 1 -- 172.21.15.43:0/3225249316 >> 172.21.15.43:0/3225249316 conn(0x5626d16ff8c0 msgr2=0x5626d16ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:58.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.544+0000 7fae7125b740 1 -- 172.21.15.43:0/3225249316 shutdown_connections 2024-09-19T05:03:58.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.544+0000 7fae67e00640 1 --2- 172.21.15.43:0/3024253248 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fae4c0a3350 0x7fae4c0a5810 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fae60005e30 tx=0x7fae60005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:58.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.544+0000 7fae7125b740 1 -- 172.21.15.43:0/3225249316 wait complete. 2024-09-19T05:03:58.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.545+0000 7fae7125b740 1 -- start start 2024-09-19T05:03:58.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.545+0000 7fae7125b740 1 Processor -- start 2024-09-19T05:03:58.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.546+0000 7fae7125b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626d1703330 0x5626d170b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:58.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.546+0000 7fae7125b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d1703c50 0x5626d170baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:58.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.546+0000 7fae7125b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626d170e510 0x5626d170bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:58.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.546+0000 7fae7125b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5626d1711060 con 0x5626d1703330 2024-09-19T05:03:58.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.546+0000 7fae7125b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5626d15f1920 con 0x5626d1703c50 2024-09-19T05:03:58.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.546+0000 7fae7125b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5626d17111e0 con 0x5626d170e510 2024-09-19T05:03:58.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.546+0000 7fae7125b740 10 client.0 _ll_register_callbacks cb 0x5626d16158a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:03:58.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.546+0000 7fae67400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626d1703330 0x5626d170b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:58.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.546+0000 7fae67400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626d1703330 0x5626d170b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38670/0 (socket says 172.21.15.43:38670) 2024-09-19T05:03:58.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.546+0000 7fae5fe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d1703c50 0x5626d170baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:58.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.546+0000 7fae67400640 1 -- 172.21.15.43:0/566574886 learned_addr learned my addr 172.21.15.43:0/566574886 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:58.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.546+0000 7fae67e00640 1 --2- 172.21.15.43:0/566574886 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626d170e510 0x5626d170bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:58.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.547+0000 7fae67400640 1 -- 172.21.15.43:0/566574886 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626d170e510 msgr2=0x5626d170bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:03:58.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.547+0000 7fae67400640 1 --2- 172.21.15.43:0/566574886 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626d170e510 0x5626d170bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:58.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.547+0000 7fae67400640 1 -- 172.21.15.43:0/566574886 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d1703c50 msgr2=0x5626d170baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:03:58.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.547+0000 7fae67400640 1 --2- 172.21.15.43:0/566574886 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d1703c50 0x5626d170baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:58.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.547+0000 7fae67400640 1 -- 172.21.15.43:0/566574886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626d13da2b0 con 0x5626d1703330 2024-09-19T05:03:58.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.547+0000 7fae67e00640 1 --2- 172.21.15.43:0/566574886 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626d170e510 0x5626d170bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:03:58.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.547+0000 7fae5fe00640 1 --2- 172.21.15.43:0/566574886 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d1703c50 0x5626d170baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:03:58.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.547+0000 7fae67400640 1 --2- 172.21.15.43:0/566574886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626d1703330 0x5626d170b560 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fae580116d0 tx=0x7fae58004230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b14607741411468f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:58.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.547+0000 7fae55600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:03:58.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.547+0000 7fae55600640 1 -- 172.21.15.43:0/566574886 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5800bc50 con 0x5626d1703330 2024-09-19T05:03:58.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.548+0000 7fae55600640 1 -- 172.21.15.43:0/566574886 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fae58005250 con 0x5626d1703330 2024-09-19T05:03:58.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.548+0000 7fae7125b740 1 -- 172.21.15.43:0/566574886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae580043c0 con 0x5626d1703330 2024-09-19T05:03:58.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.548+0000 7fae55600640 1 -- 172.21.15.43:0/566574886 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fae58005450 con 0x5626d1703330 2024-09-19T05:03:58.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.548+0000 7fae7125b740 10 client.14952 fetch_fsmap learned FSMap version 28 2024-09-19T05:03:58.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.548+0000 7fae7125b740 1 -- 172.21.15.43:0/566574886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5626d171cd60 con 0x5626d1703330 2024-09-19T05:03:58.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.548+0000 7fae55600640 1 -- 172.21.15.43:0/566574886 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae580043c0 con 0x5626d1703330 2024-09-19T05:03:58.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.548+0000 7fae55600640 1 -- 172.21.15.43:0/566574886 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fae58011b10 con 0x5626d1703330 2024-09-19T05:03:58.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.548+0000 7fae7125b740 10 client.14952 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:03:58.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.549+0000 7fae7125b740 10 client.14952 resolve_mds: resolved name 'g' to daemon mds.g 2024-09-19T05:03:58.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.549+0000 7fae7125b740 20 client.14952 populate_metadata read hostname 'smithi043' 2024-09-19T05:03:58.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.549+0000 7fae7125b740 1 --2- 172.21.15.43:0/566574886 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5626d171f170 0x5626d17205e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:58.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.549+0000 7fae7125b740 4 client.14952 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:03:58.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.549+0000 7fae7125b740 1 -- 172.21.15.43:0/566574886 --> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626d1676620 con 0x5626d171f170 2024-09-19T05:03:58.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.549+0000 7fae5fe00640 1 --2- 172.21.15.43:0/566574886 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5626d171f170 0x5626d17205e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:58.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.549+0000 7fae55600640 10 client.14952 ms_handle_connect on v2:172.21.15.43:6838/3404555030 2024-09-19T05:03:58.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.549+0000 7fae5fe00640 1 --2- 172.21.15.43:0/566574886 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5626d171f170 0x5626d17205e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c0b54397f5ff6673 server_cookie=4703d4687cce2f26 in_seq=0 out_seq=0 2024-09-19T05:03:58.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.550+0000 7fae55600640 1 -- 172.21.15.43:0/566574886 <== mds.1 v2:172.21.15.43:6838/3404555030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5626d1676620 con 0x5626d171f170 2024-09-19T05:03:58.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.550+0000 7fae55600640 10 client.14952 handle_command_reply: tid=0 2024-09-19T05:03:58.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.550+0000 7fae7125b740 1 client.14952 shutdown 2024-09-19T05:03:58.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.551+0000 7fae7125b740 1 -- 172.21.15.43:0/566574886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626d1703330 msgr2=0x5626d170b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:58.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.551+0000 7fae7125b740 1 --2- 172.21.15.43:0/566574886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626d1703330 0x5626d170b560 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fae580116d0 tx=0x7fae58004230 comp rx=0 tx=0).stop 2024-09-19T05:03:58.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.551+0000 7fae7125b740 1 -- 172.21.15.43:0/566574886 shutdown_connections 2024-09-19T05:03:58.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.551+0000 7fae7125b740 1 --2- 172.21.15.43:0/566574886 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5626d171f170 0x5626d17205e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:58.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.551+0000 7fae7125b740 1 --2- 172.21.15.43:0/566574886 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626d170e510 0x5626d170bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:58.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.551+0000 7fae7125b740 1 --2- 172.21.15.43:0/566574886 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d1703c50 0x5626d170baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:58.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.551+0000 7fae7125b740 1 --2- 172.21.15.43:0/566574886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626d1703330 0x5626d170b560 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:58.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.551+0000 7fae7125b740 1 -- 172.21.15.43:0/566574886 >> 172.21.15.43:0/566574886 conn(0x5626d16ff8c0 msgr2=0x5626d17055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:58.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.551+0000 7fae55600640 0 client.14952 ms_handle_reset on v2:172.21.15.43:6838/3404555030 2024-09-19T05:03:58.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.551+0000 7fae7125b740 1 -- 172.21.15.43:0/566574886 shutdown_connections 2024-09-19T05:03:58.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.551+0000 7fae7125b740 1 -- 172.21.15.43:0/566574886 wait complete. 2024-09-19T05:03:58.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.551+0000 7fae7125b740 20 client.14952 trim_cache size 0 max 16384 2024-09-19T05:03:58.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.570+0000 7fae7125b740 1 Processor -- start 2024-09-19T05:03:58.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.570+0000 7fae7125b740 1 -- start start 2024-09-19T05:03:58.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.570+0000 7fae7125b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626d16ff8c0 0x5626d171f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:58.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.570+0000 7fae7125b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626d1703330 0x5626d1705d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:58.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.570+0000 7fae7125b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d1703c50 0x5626d170d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:58.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.570+0000 7fae7125b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5626d171e920 con 0x5626d1703330 2024-09-19T05:03:58.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.570+0000 7fae7125b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5626d171eff0 con 0x5626d1703c50 2024-09-19T05:03:58.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.570+0000 7fae7125b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5626d171f6c0 con 0x5626d16ff8c0 2024-09-19T05:03:58.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.571+0000 7fae5fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626d1703330 0x5626d1705d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:58.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.571+0000 7fae67e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d1703c50 0x5626d170d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:58.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.571+0000 7fae67e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d1703c50 0x5626d170d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:40628/0 (socket says 172.21.15.43:40628) 2024-09-19T05:03:58.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.571+0000 7fae5fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626d1703330 0x5626d1705d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38684/0 (socket says 172.21.15.43:38684) 2024-09-19T05:03:58.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.571+0000 7fae5fe00640 1 -- 172.21.15.43:0/1133954905 learned_addr learned my addr 172.21.15.43:0/1133954905 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:58.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.571+0000 7fae67400640 1 --2- 172.21.15.43:0/1133954905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626d16ff8c0 0x5626d171f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:58.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.571+0000 7fae5fe00640 1 -- 172.21.15.43:0/1133954905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626d16ff8c0 msgr2=0x5626d171f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:03:58.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.571+0000 7fae5fe00640 1 --2- 172.21.15.43:0/1133954905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626d16ff8c0 0x5626d171f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:58.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.571+0000 7fae5fe00640 1 -- 172.21.15.43:0/1133954905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d1703c50 msgr2=0x5626d170d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:03:58.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.571+0000 7fae5fe00640 1 --2- 172.21.15.43:0/1133954905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d1703c50 0x5626d170d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:58.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.571+0000 7fae5fe00640 1 -- 172.21.15.43:0/1133954905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626d170e2b0 con 0x5626d1703330 2024-09-19T05:03:58.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.571+0000 7fae5fe00640 1 --2- 172.21.15.43:0/1133954905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626d1703330 0x5626d1705d90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fae5000cfd0 tx=0x7fae500d21d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=470d0fed38ee30fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:58.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.572+0000 7fae57e00640 1 -- 172.21.15.43:0/1133954905 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae500d1040 con 0x5626d1703330 2024-09-19T05:03:58.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.572+0000 7fae57e00640 1 -- 172.21.15.43:0/1133954905 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae5009a5b0 con 0x5626d1703330 2024-09-19T05:03:58.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.572+0000 7fae57e00640 1 -- 172.21.15.43:0/1133954905 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae50018b90 con 0x5626d1703330 2024-09-19T05:03:58.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.572+0000 7fae7125b740 1 -- 172.21.15.43:0/1133954905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626d1703330 msgr2=0x5626d1705d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:58.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.572+0000 7fae7125b740 1 --2- 172.21.15.43:0/1133954905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626d1703330 0x5626d1705d90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fae5000cfd0 tx=0x7fae500d21d0 comp rx=0 tx=0).stop 2024-09-19T05:03:58.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.573+0000 7fae7125b740 1 -- 172.21.15.43:0/1133954905 shutdown_connections 2024-09-19T05:03:58.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.573+0000 7fae7125b740 1 --2- 172.21.15.43:0/1133954905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d1703c50 0x5626d170d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:58.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.573+0000 7fae7125b740 1 --2- 172.21.15.43:0/1133954905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626d1703330 0x5626d1705d90 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:58.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.573+0000 7fae7125b740 1 --2- 172.21.15.43:0/1133954905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626d16ff8c0 0x5626d171f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:58.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.573+0000 7fae7125b740 1 -- 172.21.15.43:0/1133954905 >> 172.21.15.43:0/1133954905 conn(0x5626d171e540 msgr2=0x5626d170cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:58.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.573+0000 7fae7125b740 1 -- 172.21.15.43:0/1133954905 shutdown_connections 2024-09-19T05:03:58.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.573+0000 7fae7125b740 1 -- 172.21.15.43:0/1133954905 wait complete. 2024-09-19T05:03:58.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.573+0000 7fae7125b740 1 -- start start 2024-09-19T05:03:58.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.573+0000 7fae7125b740 1 Processor -- start 2024-09-19T05:03:58.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.574+0000 7fae7125b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626d16ff8c0 0x5626d1728cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:58.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.574+0000 7fae7125b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626d1703330 0x5626d1722000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:58.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.574+0000 7fae7125b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d1703c50 0x5626d1722540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:58.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.574+0000 7fae7125b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5626d1720460 con 0x5626d1703330 2024-09-19T05:03:58.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.574+0000 7fae7125b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5626d171fd90 con 0x5626d1703c50 2024-09-19T05:03:58.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.574+0000 7fae7125b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5626d1720b30 con 0x5626d16ff8c0 2024-09-19T05:03:58.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.574+0000 7fae7125b740 10 client.0 _ll_register_callbacks cb 0x7fae58005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:03:58.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.574+0000 7fae67e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d1703c50 0x5626d1722540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:58.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.574+0000 7fae67400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626d16ff8c0 0x5626d1728cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:58.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.574+0000 7fae67e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d1703c50 0x5626d1722540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:40632/0 (socket says 172.21.15.43:40632) 2024-09-19T05:03:58.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.574+0000 7fae5fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626d1703330 0x5626d1722000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:58.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.574+0000 7fae67e00640 1 -- 172.21.15.43:0/3894528950 learned_addr learned my addr 172.21.15.43:0/3894528950 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:03:58.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.575+0000 7fae67400640 1 -- 172.21.15.43:0/3894528950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d1703c50 msgr2=0x5626d1722540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:03:58.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.575+0000 7fae67400640 1 --2- 172.21.15.43:0/3894528950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d1703c50 0x5626d1722540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:58.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.575+0000 7fae67400640 1 -- 172.21.15.43:0/3894528950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626d1703330 msgr2=0x5626d1722000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:03:58.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.575+0000 7fae67400640 1 --2- 172.21.15.43:0/3894528950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626d1703330 0x5626d1722000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:58.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.575+0000 7fae67400640 1 -- 172.21.15.43:0/3894528950 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626d1722a80 con 0x5626d16ff8c0 2024-09-19T05:03:58.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.575+0000 7fae67e00640 1 --2- 172.21.15.43:0/3894528950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d1703c50 0x5626d1722540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:03:58.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.575+0000 7fae67400640 1 --2- 172.21.15.43:0/3894528950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626d16ff8c0 0x5626d1728cd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fae58000c00 tx=0x7fae58004890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6006d5d141656c28 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:03:58.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.575+0000 7fae55600640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:03:58.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.575+0000 7fae55600640 1 -- 172.21.15.43:0/3894528950 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5800bc50 con 0x5626d16ff8c0 2024-09-19T05:03:58.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.576+0000 7fae7125b740 1 -- 172.21.15.43:0/3894528950 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae5009a5b0 con 0x5626d16ff8c0 2024-09-19T05:03:58.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.576+0000 7fae55600640 1 -- 172.21.15.43:0/3894528950 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fae58005250 con 0x5626d16ff8c0 2024-09-19T05:03:58.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.576+0000 7fae55600640 1 -- 172.21.15.43:0/3894528950 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fae580042a0 con 0x5626d16ff8c0 2024-09-19T05:03:58.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.576+0000 7fae7125b740 10 client.24931 fetch_fsmap learned FSMap version 28 2024-09-19T05:03:58.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.576+0000 7fae7125b740 1 -- 172.21.15.43:0/3894528950 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5626d17274e0 con 0x5626d16ff8c0 2024-09-19T05:03:58.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.576+0000 7fae55600640 1 -- 172.21.15.43:0/3894528950 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae5009a5b0 con 0x5626d16ff8c0 2024-09-19T05:03:58.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.576+0000 7fae55600640 1 -- 172.21.15.43:0/3894528950 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fae580035f0 con 0x5626d16ff8c0 2024-09-19T05:03:58.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.576+0000 7fae7125b740 10 client.24931 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:03:58.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.577+0000 7fae7125b740 10 client.24931 resolve_mds: resolved name 'g' to daemon mds.g 2024-09-19T05:03:58.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.577+0000 7fae7125b740 20 client.24931 populate_metadata read hostname 'smithi043' 2024-09-19T05:03:58.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.577+0000 7fae7125b740 1 --2- 172.21.15.43:0/3894528950 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5626d170e510 0x5626d178cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:03:58.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.577+0000 7fae7125b740 4 client.24931 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T05:03:58.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.577+0000 7fae7125b740 1 -- 172.21.15.43:0/3894528950 --> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] -- command(tid 0: {"prefix": "status"}) -- 0x5626d1670b30 con 0x5626d170e510 2024-09-19T05:03:58.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.577+0000 7fae5fe00640 1 --2- 172.21.15.43:0/3894528950 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5626d170e510 0x5626d178cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:03:58.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.577+0000 7fae55600640 10 client.24931 ms_handle_connect on v2:172.21.15.43:6838/3404555030 2024-09-19T05:03:58.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.577+0000 7fae5fe00640 1 --2- 172.21.15.43:0/3894528950 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5626d170e510 0x5626d178cdf0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b4eea23cea02e36a server_cookie=25021db3c7d019a5 in_seq=0 out_seq=0 2024-09-19T05:03:58.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.578+0000 7fae55600640 1 -- 172.21.15.43:0/3894528950 <== mds.1 v2:172.21.15.43:6838/3404555030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5626d1670b30 con 0x5626d170e510 2024-09-19T05:03:58.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.578+0000 7fae55600640 10 client.24931 handle_command_reply: tid=0 2024-09-19T05:03:58.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.578+0000 7fae7125b740 1 client.24931 shutdown 2024-09-19T05:03:58.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.578+0000 7fae7125b740 1 -- 172.21.15.43:0/3894528950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626d16ff8c0 msgr2=0x5626d1728cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:58.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.578+0000 7fae7125b740 1 --2- 172.21.15.43:0/3894528950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626d16ff8c0 0x5626d1728cd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fae58000c00 tx=0x7fae58004890 comp rx=0 tx=0).stop 2024-09-19T05:03:58.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.578+0000 7fae7125b740 1 -- 172.21.15.43:0/3894528950 shutdown_connections 2024-09-19T05:03:58.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.578+0000 7fae7125b740 1 --2- 172.21.15.43:0/3894528950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5626d1703c50 0x5626d1722540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:58.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.578+0000 7fae7125b740 1 --2- 172.21.15.43:0/3894528950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5626d1703330 0x5626d1722000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:58.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.578+0000 7fae7125b740 1 --2- 172.21.15.43:0/3894528950 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5626d170e510 0x5626d178cdf0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:58.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.578+0000 7fae7125b740 1 --2- 172.21.15.43:0/3894528950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626d16ff8c0 0x5626d1728cd0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:58.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.578+0000 7fae7125b740 1 -- 172.21.15.43:0/3894528950 >> 172.21.15.43:0/3894528950 conn(0x5626d171e540 msgr2=0x5626d16fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:58.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.578+0000 7fae55600640 0 client.24931 ms_handle_reset on v2:172.21.15.43:6838/3404555030 2024-09-19T05:03:58.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.578+0000 7fae7125b740 1 -- 172.21.15.43:0/3894528950 shutdown_connections 2024-09-19T05:03:58.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.578+0000 7fae7125b740 1 -- 172.21.15.43:0/3894528950 wait complete. 2024-09-19T05:03:58.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.578+0000 7fae7125b740 20 client.24931 trim_cache size 0 max 16384 2024-09-19T05:03:58.583 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:03:58.583 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "1456a810-7643-11ef-bceb-c7b262605968", 2024-09-19T05:03:58.583 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 1, 2024-09-19T05:03:58.583 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 24518, 2024-09-19T05:03:58.583 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-09-19T05:03:58.583 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-09-19T05:03:58.584 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-09-19T05:03:58.584 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 83.108260935000004, 2024-09-19T05:03:58.584 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 28, 2024-09-19T05:03:58.584 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 85, 2024-09-19T05:03:58.584 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 83, 2024-09-19T05:03:58.584 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 112.229583541 2024-09-19T05:03:58.584 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:03:58.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.579+0000 7fae6e800640 1 -- 172.21.15.43:0/3024253248 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fae4c0a3350 msgr2=0x7fae4c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:58.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.579+0000 7fae6e800640 1 --2- 172.21.15.43:0/3024253248 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fae4c0a3350 0x7fae4c0a5810 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fae60005e30 tx=0x7fae60005d80 comp rx=0 tx=0).stop 2024-09-19T05:03:58.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.579+0000 7fae6e800640 1 -- 172.21.15.43:0/3024253248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae68077040 msgr2=0x7fae681a4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:03:58.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.579+0000 7fae6e800640 1 --2- 172.21.15.43:0/3024253248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae68077040 0x7fae681a4be0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fae5000e9f0 tx=0x7fae5000eeb0 comp rx=0 tx=0).stop 2024-09-19T05:03:58.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.579+0000 7fae6e800640 1 -- 172.21.15.43:0/3024253248 shutdown_connections 2024-09-19T05:03:58.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.579+0000 7fae6e800640 1 --2- 172.21.15.43:0/3024253248 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fae4c0a3350 0x7fae4c0a5810 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:58.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.579+0000 7fae6e800640 1 --2- 172.21.15.43:0/3024253248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae68112a30 0x7fae6819e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:58.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.579+0000 7fae6e800640 1 --2- 172.21.15.43:0/3024253248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae68077040 0x7fae681a4be0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:58.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.579+0000 7fae6e800640 1 --2- 172.21.15.43:0/3024253248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae68076720 0x7fae681a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:03:58.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.579+0000 7fae6e800640 1 -- 172.21.15.43:0/3024253248 >> 172.21.15.43:0/3024253248 conn(0x7fae680ffa90 msgr2=0x7fae68101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:03:58.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.579+0000 7fae6e800640 1 -- 172.21.15.43:0/3024253248 shutdown_connections 2024-09-19T05:03:58.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:03:58.579+0000 7fae6e800640 1 -- 172.21.15.43:0/3024253248 wait complete. 2024-09-19T05:03:59.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:03:58 smithi151 ceph-mon[35425]: pgmap v340: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T05:03:59.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:03:58 smithi096 ceph-mon[35085]: pgmap v340: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T05:03:59.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:03:58 smithi043 ceph-mon[30800]: pgmap v340: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T05:03:59.427 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T05:03:59.819 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:00.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.372+0000 7f79ba200640 1 -- 172.21.15.43:0/3228536760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79b4102360 msgr2=0x7f79b410e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:00.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.372+0000 7f79ba200640 1 --2- 172.21.15.43:0/3228536760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79b4102360 0x7f79b410e890 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f79ac009cd0 tx=0x7f79ac02a8e0 comp rx=0 tx=0).stop 2024-09-19T05:04:00.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.372+0000 7f79ba200640 1 -- 172.21.15.43:0/3228536760 shutdown_connections 2024-09-19T05:04:00.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.372+0000 7f79ba200640 1 --2- 172.21.15.43:0/3228536760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79b4102360 0x7f79b410e890 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:00.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.372+0000 7f79ba200640 1 --2- 172.21.15.43:0/3228536760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79b41019c0 0x7f79b4101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:00.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.372+0000 7f79ba200640 1 --2- 172.21.15.43:0/3228536760 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79b41079c0 0x7f79b4107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:00.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.372+0000 7f79ba200640 1 -- 172.21.15.43:0/3228536760 >> 172.21.15.43:0/3228536760 conn(0x7f79b40fd740 msgr2=0x7f79b40ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:00.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.372+0000 7f79ba200640 1 -- 172.21.15.43:0/3228536760 shutdown_connections 2024-09-19T05:04:00.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.372+0000 7f79ba200640 1 -- 172.21.15.43:0/3228536760 wait complete. 2024-09-19T05:04:00.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.373+0000 7f79ba200640 1 Processor -- start 2024-09-19T05:04:00.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.373+0000 7f79ba200640 1 -- start start 2024-09-19T05:04:00.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.374+0000 7f79ba200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79b41019c0 0x7f79b419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:00.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.374+0000 7f79ba200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79b4102360 0x7f79b419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:00.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.374+0000 7f79ba200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79b41079c0 0x7f79b419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:00.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.374+0000 7f79ba200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f79b4111e10 con 0x7f79b4102360 2024-09-19T05:04:00.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.374+0000 7f79ba200640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f79b4111c90 con 0x7f79b41079c0 2024-09-19T05:04:00.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.374+0000 7f79ba200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f79b4111f90 con 0x7f79b41019c0 2024-09-19T05:04:00.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.374+0000 7f79b8e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79b41019c0 0x7f79b419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:00.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.374+0000 7f79b8e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79b41019c0 0x7f79b419a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:59464/0 (socket says 172.21.15.43:59464) 2024-09-19T05:04:00.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.374+0000 7f79b8e00640 1 -- 172.21.15.43:0/2678122604 learned_addr learned my addr 172.21.15.43:0/2678122604 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:00.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.374+0000 7f79abe00640 1 --2- 172.21.15.43:0/2678122604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79b4102360 0x7f79b419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:00.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.374+0000 7f79b9800640 1 --2- 172.21.15.43:0/2678122604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79b41079c0 0x7f79b419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:00.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.375+0000 7f79b9800640 1 -- 172.21.15.43:0/2678122604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79b41019c0 msgr2=0x7f79b419a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:00.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.375+0000 7f79b9800640 1 --2- 172.21.15.43:0/2678122604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79b41019c0 0x7f79b419a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:00.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.375+0000 7f79b9800640 1 -- 172.21.15.43:0/2678122604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79b4102360 msgr2=0x7f79b419a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:00.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.375+0000 7f79b9800640 1 --2- 172.21.15.43:0/2678122604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79b4102360 0x7f79b419a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:00.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.375+0000 7f79b9800640 1 -- 172.21.15.43:0/2678122604 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79b419f410 con 0x7f79b41079c0 2024-09-19T05:04:00.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.375+0000 7f79b8e00640 1 --2- 172.21.15.43:0/2678122604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79b41019c0 0x7f79b419a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:04:00.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.375+0000 7f79b9800640 1 --2- 172.21.15.43:0/2678122604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79b41079c0 0x7f79b419ed30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f79ac009ca0 tx=0x7f79ac031e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:00.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.375+0000 7f79b2000640 1 -- 172.21.15.43:0/2678122604 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ac004040 con 0x7f79b41079c0 2024-09-19T05:04:00.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.375+0000 7f79ba200640 1 -- 172.21.15.43:0/2678122604 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79b419f6a0 con 0x7f79b41079c0 2024-09-19T05:04:00.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.375+0000 7f79b2000640 1 -- 172.21.15.43:0/2678122604 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79ac0046c0 con 0x7f79b41079c0 2024-09-19T05:04:00.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.376+0000 7f79b2000640 1 -- 172.21.15.43:0/2678122604 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ac030e90 con 0x7f79b41079c0 2024-09-19T05:04:00.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.376+0000 7f79ba200640 1 -- 172.21.15.43:0/2678122604 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79b41a6fe0 con 0x7f79b41079c0 2024-09-19T05:04:00.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.379+0000 7f79b2000640 1 -- 172.21.15.43:0/2678122604 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f79ac0305a0 con 0x7f79b41079c0 2024-09-19T05:04:00.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.379+0000 7f79be261740 1 Processor -- start 2024-09-19T05:04:00.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.379+0000 7f79be261740 1 -- start start 2024-09-19T05:04:00.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.379+0000 7f79be261740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55868a7274e0 0x55868a7278c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:00.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.379+0000 7f79be261740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55868a727e00 0x55868a732180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:00.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.379+0000 7f79be261740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55868a7326c0 0x55868a734ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:00.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.379+0000 7f79be261740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55868a68d0b0 con 0x55868a7274e0 2024-09-19T05:04:00.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.379+0000 7f79be261740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55868a634ab0 con 0x55868a727e00 2024-09-19T05:04:00.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.379+0000 7f79be261740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55868a689070 con 0x55868a7326c0 2024-09-19T05:04:00.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.379+0000 7f79abe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55868a7274e0 0x55868a7278c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:00.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.379+0000 7f79abe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55868a7274e0 0x55868a7278c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38708/0 (socket says 172.21.15.43:38708) 2024-09-19T05:04:00.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.379+0000 7f79abe00640 1 -- 172.21.15.43:0/107760178 learned_addr learned my addr 172.21.15.43:0/107760178 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:00.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.379+0000 7f79b8e00640 1 --2- 172.21.15.43:0/107760178 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55868a727e00 0x55868a732180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:00.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.380+0000 7f79abe00640 1 --2- 172.21.15.43:0/107760178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55868a7326c0 0x55868a734ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:00.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.380+0000 7f79abe00640 1 -- 172.21.15.43:0/107760178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55868a7326c0 msgr2=0x55868a734ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:00.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.380+0000 7f79abe00640 1 --2- 172.21.15.43:0/107760178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55868a7326c0 0x55868a734ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:00.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.380+0000 7f79abe00640 1 -- 172.21.15.43:0/107760178 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55868a727e00 msgr2=0x55868a732180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:00.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.380+0000 7f79abe00640 1 --2- 172.21.15.43:0/107760178 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55868a727e00 0x55868a732180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:00.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.380+0000 7f79abe00640 1 -- 172.21.15.43:0/107760178 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55868a6414a0 con 0x55868a7274e0 2024-09-19T05:04:00.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.380+0000 7f79b8e00640 1 --2- 172.21.15.43:0/107760178 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55868a727e00 0x55868a732180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:04:00.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.380+0000 7f79abe00640 1 --2- 172.21.15.43:0/107760178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55868a7274e0 0x55868a7278c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f799c00b6c0 tx=0x7f799c00b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd231dd844a8795 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:00.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.380+0000 7f79b2000640 1 --2- 172.21.15.43:0/2678122604 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f79980a32d0 0x7f79980a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:00.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.380+0000 7f79aa000640 1 -- 172.21.15.43:0/107760178 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c005020 con 0x55868a7274e0 2024-09-19T05:04:00.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.380+0000 7f79aa000640 1 -- 172.21.15.43:0/107760178 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f799c0047a0 con 0x55868a7274e0 2024-09-19T05:04:00.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.380+0000 7f79aa000640 1 -- 172.21.15.43:0/107760178 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c00f6e0 con 0x55868a7274e0 2024-09-19T05:04:00.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.380+0000 7f79b2000640 1 -- 172.21.15.43:0/2678122604 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f79ac02e030 con 0x7f79b41079c0 2024-09-19T05:04:00.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.381+0000 7f79b8e00640 1 --2- 172.21.15.43:0/2678122604 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f79980a32d0 0x7f79980a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:00.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.381+0000 7f79be261740 1 -- 172.21.15.43:0/107760178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55868a7274e0 msgr2=0x55868a7278c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:00.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.381+0000 7f79be261740 1 --2- 172.21.15.43:0/107760178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55868a7274e0 0x55868a7278c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f799c00b6c0 tx=0x7f799c00b650 comp rx=0 tx=0).stop 2024-09-19T05:04:00.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.381+0000 7f79b8e00640 1 --2- 172.21.15.43:0/2678122604 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f79980a32d0 0x7f79980a5790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x55868a7248d0 tx=0x7f79a4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:00.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.381+0000 7f79be261740 1 -- 172.21.15.43:0/107760178 shutdown_connections 2024-09-19T05:04:00.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.381+0000 7f79be261740 1 --2- 172.21.15.43:0/107760178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55868a7326c0 0x55868a734ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:00.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.381+0000 7f79be261740 1 --2- 172.21.15.43:0/107760178 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55868a727e00 0x55868a732180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:00.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.381+0000 7f79be261740 1 --2- 172.21.15.43:0/107760178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55868a7274e0 0x55868a7278c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:00.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.381+0000 7f79be261740 1 -- 172.21.15.43:0/107760178 >> 172.21.15.43:0/107760178 conn(0x55868a723a70 msgr2=0x55868a723e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:00.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.381+0000 7f79be261740 1 -- 172.21.15.43:0/107760178 shutdown_connections 2024-09-19T05:04:00.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.381+0000 7f79be261740 1 -- 172.21.15.43:0/107760178 wait complete. 2024-09-19T05:04:00.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.382+0000 7f79be261740 1 -- start start 2024-09-19T05:04:00.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.382+0000 7f79be261740 1 Processor -- start 2024-09-19T05:04:00.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.383+0000 7f79be261740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55868a7274e0 0x55868a72f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:00.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.383+0000 7f79be261740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55868a727e00 0x55868a72fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:00.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.383+0000 7f79be261740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55868a7326c0 0x55868a72ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:00.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.383+0000 7f79be261740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55868a735330 con 0x55868a7274e0 2024-09-19T05:04:00.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.383+0000 7f79be261740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55868a615920 con 0x55868a727e00 2024-09-19T05:04:00.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.383+0000 7f79be261740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55868a7354b0 con 0x55868a7326c0 2024-09-19T05:04:00.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.383+0000 7f79be261740 10 client.0 _ll_register_callbacks cb 0x55868a6398a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:04:00.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.383+0000 7f79abe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55868a727e00 0x55868a72fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:00.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.383+0000 7f79abe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55868a727e00 0x55868a72fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:40656/0 (socket says 172.21.15.43:40656) 2024-09-19T05:04:00.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.383+0000 7f79abe00640 1 -- 172.21.15.43:0/942765991 learned_addr learned my addr 172.21.15.43:0/942765991 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:00.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.383+0000 7f79b9800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55868a7326c0 0x55868a72ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:00.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.383+0000 7f79b8e00640 1 --2- 172.21.15.43:0/942765991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55868a7274e0 0x55868a72f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:00.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.383+0000 7f79b9800640 1 -- 172.21.15.43:0/942765991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55868a727e00 msgr2=0x55868a72fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:00.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.383+0000 7f79b9800640 1 --2- 172.21.15.43:0/942765991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55868a727e00 0x55868a72fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:00.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.383+0000 7f79b9800640 1 -- 172.21.15.43:0/942765991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55868a7274e0 msgr2=0x55868a72f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:00.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.383+0000 7f79b9800640 1 --2- 172.21.15.43:0/942765991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55868a7274e0 0x55868a72f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:00.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.384+0000 7f79b9800640 1 -- 172.21.15.43:0/942765991 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55868a3fe2b0 con 0x55868a7326c0 2024-09-19T05:04:00.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.384+0000 7f79abe00640 1 --2- 172.21.15.43:0/942765991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55868a727e00 0x55868a72fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:04:00.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.384+0000 7f79b9800640 1 --2- 172.21.15.43:0/942765991 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55868a7326c0 0x55868a72ff50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f79ac0ee9d0 tx=0x7f79ac0b9780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=553abc37314c2c91 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:00.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.384+0000 7f79a2a00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:04:00.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.384+0000 7f79a2a00640 1 -- 172.21.15.43:0/942765991 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ac0d5020 con 0x55868a7326c0 2024-09-19T05:04:00.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.384+0000 7f79be261740 1 -- 172.21.15.43:0/942765991 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f799c0047a0 con 0x55868a7326c0 2024-09-19T05:04:00.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.384+0000 7f79a2a00640 1 -- 172.21.15.43:0/942765991 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f79ac0d6450 con 0x55868a7326c0 2024-09-19T05:04:00.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.384+0000 7f79a2a00640 1 -- 172.21.15.43:0/942765991 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f79ac0bce40 con 0x55868a7326c0 2024-09-19T05:04:00.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.384+0000 7f79be261740 10 client.24943 fetch_fsmap learned FSMap version 28 2024-09-19T05:04:00.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.384+0000 7f79be261740 1 -- 172.21.15.43:0/942765991 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55868a7353e0 con 0x55868a7326c0 2024-09-19T05:04:00.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.384+0000 7f79a2a00640 1 -- 172.21.15.43:0/942765991 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f799c0047a0 con 0x55868a7326c0 2024-09-19T05:04:00.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.385+0000 7f79a2a00640 1 -- 172.21.15.43:0/942765991 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f79ac04e070 con 0x55868a7326c0 2024-09-19T05:04:00.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.385+0000 7f79be261740 10 client.24943 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:04:00.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.385+0000 7f79be261740 10 client.24943 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T05:04:00.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.385+0000 7f79be261740 20 client.24943 populate_metadata read hostname 'smithi043' 2024-09-19T05:04:00.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.385+0000 7f79be261740 1 --2- 172.21.15.43:0/942765991 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55868a7435c0 0x55868a744a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:00.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.385+0000 7f79be261740 4 client.24943 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:04:00.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.385+0000 7f79be261740 1 -- 172.21.15.43:0/942765991 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55868a69a620 con 0x55868a7435c0 2024-09-19T05:04:00.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.385+0000 7f79b8e00640 1 --2- 172.21.15.43:0/942765991 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55868a7435c0 0x55868a744a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:00.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.386+0000 7f79b8e00640 1 --2- 172.21.15.43:0/942765991 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55868a7435c0 0x55868a744a30 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8ada81c1f3cace6 server_cookie=4eb5189f6bf413f0 in_seq=0 out_seq=0 2024-09-19T05:04:00.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.386+0000 7f79a2a00640 10 client.24943 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:04:00.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.387+0000 7f79a2a00640 1 -- 172.21.15.43:0/942765991 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55868a69a620 con 0x55868a7435c0 2024-09-19T05:04:00.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.387+0000 7f79a2a00640 10 client.24943 handle_command_reply: tid=0 2024-09-19T05:04:00.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.387+0000 7f79be261740 1 client.24943 shutdown 2024-09-19T05:04:00.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.387+0000 7f79be261740 1 -- 172.21.15.43:0/942765991 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55868a7326c0 msgr2=0x55868a72ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:00.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.387+0000 7f79be261740 1 --2- 172.21.15.43:0/942765991 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55868a7326c0 0x55868a72ff50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f79ac0ee9d0 tx=0x7f79ac0b9780 comp rx=0 tx=0).stop 2024-09-19T05:04:00.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.387+0000 7f79be261740 1 -- 172.21.15.43:0/942765991 shutdown_connections 2024-09-19T05:04:00.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.387+0000 7f79be261740 1 --2- 172.21.15.43:0/942765991 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55868a7435c0 0x55868a744a30 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:00.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.387+0000 7f79be261740 1 --2- 172.21.15.43:0/942765991 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55868a7326c0 0x55868a72ff50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:00.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.387+0000 7f79be261740 1 --2- 172.21.15.43:0/942765991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55868a727e00 0x55868a72fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:00.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.387+0000 7f79be261740 1 --2- 172.21.15.43:0/942765991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55868a7274e0 0x55868a72f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:00.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.387+0000 7f79be261740 1 -- 172.21.15.43:0/942765991 >> 172.21.15.43:0/942765991 conn(0x55868a723a70 msgr2=0x55868a7297a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:00.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.387+0000 7f79a2a00640 0 client.24943 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:04:00.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.388+0000 7f79be261740 1 -- 172.21.15.43:0/942765991 shutdown_connections 2024-09-19T05:04:00.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.388+0000 7f79be261740 1 -- 172.21.15.43:0/942765991 wait complete. 2024-09-19T05:04:00.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.388+0000 7f79be261740 20 client.24943 trim_cache size 0 max 16384 2024-09-19T05:04:00.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.407+0000 7f79be261740 1 Processor -- start 2024-09-19T05:04:00.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.407+0000 7f79be261740 1 -- start start 2024-09-19T05:04:00.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.407+0000 7f79be261740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55868a7274e0 0x55868a73e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:00.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.407+0000 7f79be261740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55868a727e00 0x55868a729f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:00.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.407+0000 7f79be261740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55868a7326c0 0x55868a72c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:00.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.407+0000 7f79be261740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55868a742d70 con 0x55868a7326c0 2024-09-19T05:04:00.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.407+0000 7f79be261740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55868a743440 con 0x55868a7274e0 2024-09-19T05:04:00.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.407+0000 7f79be261740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55868a743b10 con 0x55868a727e00 2024-09-19T05:04:00.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.407+0000 7f79b9800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55868a7326c0 0x55868a72c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:00.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.407+0000 7f79b8e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55868a7274e0 0x55868a73e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:00.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.407+0000 7f79abe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55868a727e00 0x55868a729f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:00.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.407+0000 7f79b9800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55868a7326c0 0x55868a72c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38722/0 (socket says 172.21.15.43:38722) 2024-09-19T05:04:00.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.407+0000 7f79b9800640 1 -- 172.21.15.43:0/2627539911 learned_addr learned my addr 172.21.15.43:0/2627539911 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:00.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.408+0000 7f79abe00640 1 -- 172.21.15.43:0/2627539911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55868a7274e0 msgr2=0x55868a73e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:00.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.408+0000 7f79abe00640 1 --2- 172.21.15.43:0/2627539911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55868a7274e0 0x55868a73e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:00.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.408+0000 7f79abe00640 1 -- 172.21.15.43:0/2627539911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55868a7326c0 msgr2=0x55868a72c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:00.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.408+0000 7f79abe00640 1 --2- 172.21.15.43:0/2627539911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55868a7326c0 0x55868a72c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:00.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.408+0000 7f79abe00640 1 -- 172.21.15.43:0/2627539911 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55868a727c30 con 0x55868a727e00 2024-09-19T05:04:00.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.408+0000 7f79b9800640 1 --2- 172.21.15.43:0/2627539911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55868a7326c0 0x55868a72c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:04:00.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.408+0000 7f79b8e00640 1 --2- 172.21.15.43:0/2627539911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55868a7274e0 0x55868a73e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:04:00.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.408+0000 7f79abe00640 1 --2- 172.21.15.43:0/2627539911 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55868a727e00 0x55868a729f40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55868a730b10 tx=0x7f799c00b8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f83e365f97f4c3d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:00.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.408+0000 7f79a9600640 1 -- 172.21.15.43:0/2627539911 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c004020 con 0x55868a727e00 2024-09-19T05:04:00.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.409+0000 7f79a9600640 1 -- 172.21.15.43:0/2627539911 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f799c0041c0 con 0x55868a727e00 2024-09-19T05:04:00.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.409+0000 7f79a9600640 1 -- 172.21.15.43:0/2627539911 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c0106e0 con 0x55868a727e00 2024-09-19T05:04:00.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.409+0000 7f79be261740 1 -- 172.21.15.43:0/2627539911 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55868a727e00 msgr2=0x55868a729f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:00.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.409+0000 7f79be261740 1 --2- 172.21.15.43:0/2627539911 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55868a727e00 0x55868a729f40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55868a730b10 tx=0x7f799c00b8c0 comp rx=0 tx=0).stop 2024-09-19T05:04:00.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.409+0000 7f79be261740 1 -- 172.21.15.43:0/2627539911 shutdown_connections 2024-09-19T05:04:00.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.409+0000 7f79be261740 1 --2- 172.21.15.43:0/2627539911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55868a7326c0 0x55868a72c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:00.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.409+0000 7f79be261740 1 --2- 172.21.15.43:0/2627539911 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55868a727e00 0x55868a729f40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:00.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.409+0000 7f79be261740 1 --2- 172.21.15.43:0/2627539911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55868a7274e0 0x55868a73e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:00.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.409+0000 7f79be261740 1 -- 172.21.15.43:0/2627539911 >> 172.21.15.43:0/2627539911 conn(0x55868a723a70 msgr2=0x55868a723e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:00.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.409+0000 7f79be261740 1 -- 172.21.15.43:0/2627539911 shutdown_connections 2024-09-19T05:04:00.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.409+0000 7f79be261740 1 -- 172.21.15.43:0/2627539911 wait complete. 2024-09-19T05:04:00.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.410+0000 7f79be261740 1 -- start start 2024-09-19T05:04:00.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.410+0000 7f79be261740 1 Processor -- start 2024-09-19T05:04:00.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.411+0000 7f79be261740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55868a7274e0 0x55868a72bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:00.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.411+0000 7f79be261740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55868a727e00 0x55868a72c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:00.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.411+0000 7f79be261740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55868a7326c0 0x55868a74bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:00.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.411+0000 7f79be261740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55868a7448b0 con 0x55868a7326c0 2024-09-19T05:04:00.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.411+0000 7f79be261740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55868a7441e0 con 0x55868a727e00 2024-09-19T05:04:00.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.411+0000 7f79be261740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55868a744f80 con 0x55868a7274e0 2024-09-19T05:04:00.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.411+0000 7f79be261740 10 client.0 _ll_register_callbacks cb 0x7f79ac009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:04:00.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.411+0000 7f79b9800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55868a7326c0 0x55868a74bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:00.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.411+0000 7f79abe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55868a727e00 0x55868a72c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:00.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.411+0000 7f79b9800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55868a7326c0 0x55868a74bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38730/0 (socket says 172.21.15.43:38730) 2024-09-19T05:04:00.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.411+0000 7f79b9800640 1 -- 172.21.15.43:0/4242107584 learned_addr learned my addr 172.21.15.43:0/4242107584 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:00.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.411+0000 7f79abe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55868a727e00 0x55868a72c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:40684/0 (socket says 172.21.15.43:40684) 2024-09-19T05:04:00.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.411+0000 7f79b8e00640 1 --2- 172.21.15.43:0/4242107584 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55868a7274e0 0x55868a72bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:00.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.411+0000 7f79abe00640 1 -- 172.21.15.43:0/4242107584 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55868a7274e0 msgr2=0x55868a72bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:00.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.411+0000 7f79abe00640 1 --2- 172.21.15.43:0/4242107584 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55868a7274e0 0x55868a72bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:00.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.411+0000 7f79abe00640 1 -- 172.21.15.43:0/4242107584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55868a7326c0 msgr2=0x55868a74bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:00.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.411+0000 7f79abe00640 1 --2- 172.21.15.43:0/4242107584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55868a7326c0 0x55868a74bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:00.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.411+0000 7f79abe00640 1 -- 172.21.15.43:0/4242107584 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55868a74c1a0 con 0x55868a727e00 2024-09-19T05:04:00.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.411+0000 7f79b9800640 1 --2- 172.21.15.43:0/4242107584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55868a7326c0 0x55868a74bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:04:00.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.412+0000 7f79abe00640 1 --2- 172.21.15.43:0/4242107584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55868a727e00 0x55868a72c440 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x55868a745150 tx=0x7f799c00bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c970aa5ae1c7560 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:00.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.412+0000 7f79a2a00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:04:00.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.412+0000 7f79a2a00640 1 -- 172.21.15.43:0/4242107584 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c02e070 con 0x55868a727e00 2024-09-19T05:04:00.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.412+0000 7f79be261740 1 -- 172.21.15.43:0/4242107584 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f799c0041c0 con 0x55868a727e00 2024-09-19T05:04:00.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.412+0000 7f79a2a00640 1 -- 172.21.15.43:0/4242107584 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f799c004370 con 0x55868a727e00 2024-09-19T05:04:00.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.412+0000 7f79a2a00640 1 -- 172.21.15.43:0/4242107584 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f799c003ee0 con 0x55868a727e00 2024-09-19T05:04:00.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.412+0000 7f79be261740 10 client.24965 fetch_fsmap learned FSMap version 28 2024-09-19T05:04:00.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.412+0000 7f79be261740 1 -- 172.21.15.43:0/4242107584 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55868a74d4a0 con 0x55868a727e00 2024-09-19T05:04:00.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.413+0000 7f79a2a00640 1 -- 172.21.15.43:0/4242107584 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f799c0041c0 con 0x55868a727e00 2024-09-19T05:04:00.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.413+0000 7f79a2a00640 1 -- 172.21.15.43:0/4242107584 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f799c031020 con 0x55868a727e00 2024-09-19T05:04:00.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.413+0000 7f79be261740 10 client.24965 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:04:00.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.413+0000 7f79be261740 10 client.24965 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T05:04:00.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.413+0000 7f79be261740 20 client.24965 populate_metadata read hostname 'smithi043' 2024-09-19T05:04:00.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.413+0000 7f79be261740 1 --2- 172.21.15.43:0/4242107584 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55868a7b1af0 0x55868a7b0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:00.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.413+0000 7f79be261740 4 client.24965 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T05:04:00.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.413+0000 7f79be261740 1 -- 172.21.15.43:0/4242107584 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "status"}) -- 0x55868a694b30 con 0x55868a7b1af0 2024-09-19T05:04:00.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.413+0000 7f79b8e00640 1 --2- 172.21.15.43:0/4242107584 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55868a7b1af0 0x55868a7b0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:00.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.414+0000 7f79b8e00640 1 --2- 172.21.15.43:0/4242107584 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55868a7b1af0 0x55868a7b0800 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=788c4579201b0585 server_cookie=4af9a68e6c666432 in_seq=0 out_seq=0 2024-09-19T05:04:00.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.414+0000 7f79a2a00640 10 client.24965 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:04:00.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.414+0000 7f79a2a00640 1 -- 172.21.15.43:0/4242107584 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x55868a694b30 con 0x55868a7b1af0 2024-09-19T05:04:00.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.414+0000 7f79a2a00640 10 client.24965 handle_command_reply: tid=0 2024-09-19T05:04:00.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.414+0000 7f79be261740 1 client.24965 shutdown 2024-09-19T05:04:00.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.414+0000 7f79be261740 1 -- 172.21.15.43:0/4242107584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55868a727e00 msgr2=0x55868a72c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:00.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.414+0000 7f79be261740 1 --2- 172.21.15.43:0/4242107584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55868a727e00 0x55868a72c440 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x55868a745150 tx=0x7f799c00bf90 comp rx=0 tx=0).stop 2024-09-19T05:04:00.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.414+0000 7f79be261740 1 -- 172.21.15.43:0/4242107584 shutdown_connections 2024-09-19T05:04:00.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.414+0000 7f79be261740 1 --2- 172.21.15.43:0/4242107584 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55868a7b1af0 0x55868a7b0800 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:00.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.414+0000 7f79be261740 1 --2- 172.21.15.43:0/4242107584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55868a7326c0 0x55868a74bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:00.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.414+0000 7f79be261740 1 --2- 172.21.15.43:0/4242107584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55868a727e00 0x55868a72c440 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:00.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.414+0000 7f79a2a00640 0 client.24965 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:04:00.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.414+0000 7f79be261740 1 --2- 172.21.15.43:0/4242107584 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55868a7274e0 0x55868a72bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:00.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.414+0000 7f79be261740 1 -- 172.21.15.43:0/4242107584 >> 172.21.15.43:0/4242107584 conn(0x55868a723a70 msgr2=0x55868a72dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:00.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.414+0000 7f79be261740 1 -- 172.21.15.43:0/4242107584 shutdown_connections 2024-09-19T05:04:00.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.414+0000 7f79be261740 1 -- 172.21.15.43:0/4242107584 wait complete. 2024-09-19T05:04:00.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.415+0000 7f79be261740 20 client.24965 trim_cache size 0 max 16384 2024-09-19T05:04:00.421 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:04:00.421 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "1456a810-7643-11ef-bceb-c7b262605968", 2024-09-19T05:04:00.421 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-09-19T05:04:00.421 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 24542, 2024-09-19T05:04:00.421 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-09-19T05:04:00.421 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-09-19T05:04:00.421 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-09-19T05:04:00.421 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 94.001411933, 2024-09-19T05:04:00.421 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 28, 2024-09-19T05:04:00.421 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 85, 2024-09-19T05:04:00.422 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 83, 2024-09-19T05:04:00.422 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 107.509410382 2024-09-19T05:04:00.422 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:04:00.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.415+0000 7f79ba200640 1 -- 172.21.15.43:0/2678122604 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f79980a32d0 msgr2=0x7f79980a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:00.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.415+0000 7f79ba200640 1 --2- 172.21.15.43:0/2678122604 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f79980a32d0 0x7f79980a5790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x55868a7248d0 tx=0x7f79a4005f90 comp rx=0 tx=0).stop 2024-09-19T05:04:00.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.415+0000 7f79ba200640 1 -- 172.21.15.43:0/2678122604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79b41079c0 msgr2=0x7f79b419ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:00.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.415+0000 7f79ba200640 1 --2- 172.21.15.43:0/2678122604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79b41079c0 0x7f79b419ed30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f79ac009ca0 tx=0x7f79ac031e60 comp rx=0 tx=0).stop 2024-09-19T05:04:00.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.415+0000 7f79ba200640 1 -- 172.21.15.43:0/2678122604 shutdown_connections 2024-09-19T05:04:00.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.415+0000 7f79ba200640 1 --2- 172.21.15.43:0/2678122604 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f79980a32d0 0x7f79980a5790 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:00.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.415+0000 7f79ba200640 1 --2- 172.21.15.43:0/2678122604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79b41079c0 0x7f79b419ed30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:00.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.415+0000 7f79ba200640 1 --2- 172.21.15.43:0/2678122604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79b4102360 0x7f79b419a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:00.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.415+0000 7f79ba200640 1 --2- 172.21.15.43:0/2678122604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79b41019c0 0x7f79b419a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:00.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.415+0000 7f79ba200640 1 -- 172.21.15.43:0/2678122604 >> 172.21.15.43:0/2678122604 conn(0x7f79b40fd740 msgr2=0x7f79b410d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:00.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.415+0000 7f79ba200640 1 -- 172.21.15.43:0/2678122604 shutdown_connections 2024-09-19T05:04:00.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:00.416+0000 7f79ba200640 1 -- 172.21.15.43:0/2678122604 wait complete. 2024-09-19T05:04:01.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:00 smithi151 ceph-mon[35425]: pgmap v341: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:04:01.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:00 smithi096 ceph-mon[35085]: pgmap v341: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:04:01.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:00 smithi043 ceph-mon[30800]: pgmap v341: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:04:01.281 INFO:tasks.cephadm:Setup complete, yielding 2024-09-19T05:04:01.281 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-19T05:04:01.291 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi043.front.sepia.ceph.com 2024-09-19T05:04:01.292 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- bash -c 'ceph orch status' 2024-09-19T05:04:01.671 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:02.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.251+0000 7f3cd1c00640 1 -- 172.21.15.43:0/1972556125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ccc105920 msgr2=0x7f3ccc10c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:02.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.251+0000 7f3cd1c00640 1 --2- 172.21.15.43:0/1972556125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ccc105920 0x7f3ccc10c240 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4009ca0 tx=0x7f3cc402a8c0 comp rx=0 tx=0).stop 2024-09-19T05:04:02.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.252+0000 7f3cd1c00640 1 -- 172.21.15.43:0/1972556125 shutdown_connections 2024-09-19T05:04:02.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.252+0000 7f3cd1c00640 1 --2- 172.21.15.43:0/1972556125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ccc105920 0x7f3ccc10c240 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:02.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.252+0000 7f3cd1c00640 1 --2- 172.21.15.43:0/1972556125 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ccc104f60 0x7f3ccc1053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:02.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.252+0000 7f3cd1c00640 1 --2- 172.21.15.43:0/1972556125 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ccc103e00 0x7f3ccc104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:02.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.252+0000 7f3cd1c00640 1 -- 172.21.15.43:0/1972556125 >> 172.21.15.43:0/1972556125 conn(0x7f3ccc0ff5b0 msgr2=0x7f3ccc1019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:02.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.252+0000 7f3cd1c00640 1 -- 172.21.15.43:0/1972556125 shutdown_connections 2024-09-19T05:04:02.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.252+0000 7f3cd1c00640 1 -- 172.21.15.43:0/1972556125 wait complete. 2024-09-19T05:04:02.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.252+0000 7f3cd1c00640 1 Processor -- start 2024-09-19T05:04:02.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.253+0000 7f3cd1c00640 1 -- start start 2024-09-19T05:04:02.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.253+0000 7f3cd1c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ccc103e00 0x7f3ccc19a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:02.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.253+0000 7f3cd1c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ccc104f60 0x7f3ccc19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:02.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.253+0000 7f3cd1c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ccc105920 0x7f3ccc1a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:02.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.253+0000 7f3cd1c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3ccc10fb00 con 0x7f3ccc104f60 2024-09-19T05:04:02.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.253+0000 7f3cd1c00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3ccc10f980 con 0x7f3ccc103e00 2024-09-19T05:04:02.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.253+0000 7f3cd1c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3ccc10fc80 con 0x7f3ccc105920 2024-09-19T05:04:02.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.253+0000 7f3cc2000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ccc104f60 0x7f3ccc19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:02.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.253+0000 7f3cc2000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ccc104f60 0x7f3ccc19a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38744/0 (socket says 172.21.15.43:38744) 2024-09-19T05:04:02.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.253+0000 7f3cc2000640 1 -- 172.21.15.43:0/1766000395 learned_addr learned my addr 172.21.15.43:0/1766000395 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:02.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.253+0000 7f3ccaa00640 1 --2- 172.21.15.43:0/1766000395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ccc103e00 0x7f3ccc19a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:02.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.253+0000 7f3ccb400640 1 --2- 172.21.15.43:0/1766000395 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ccc105920 0x7f3ccc1a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:02.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.254+0000 7f3cc2000640 1 -- 172.21.15.43:0/1766000395 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ccc105920 msgr2=0x7f3ccc1a1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:02.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.254+0000 7f3cc2000640 1 --2- 172.21.15.43:0/1766000395 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ccc105920 0x7f3ccc1a1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:02.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.254+0000 7f3cc2000640 1 -- 172.21.15.43:0/1766000395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ccc103e00 msgr2=0x7f3ccc19a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:02.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.254+0000 7f3cc2000640 1 --2- 172.21.15.43:0/1766000395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ccc103e00 0x7f3ccc19a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:02.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.254+0000 7f3cc2000640 1 -- 172.21.15.43:0/1766000395 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ccc1a1ca0 con 0x7f3ccc104f60 2024-09-19T05:04:02.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.254+0000 7f3cc2000640 1 --2- 172.21.15.43:0/1766000395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ccc104f60 0x7f3ccc19a5b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3cb400e9c0 tx=0x7f3cb400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:02.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.254+0000 7f3cc3e00640 1 -- 172.21.15.43:0/1766000395 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb400cd80 con 0x7f3ccc104f60 2024-09-19T05:04:02.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.254+0000 7f3cc3e00640 1 -- 172.21.15.43:0/1766000395 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cb40035f0 con 0x7f3ccc104f60 2024-09-19T05:04:02.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.254+0000 7f3cc3e00640 1 -- 172.21.15.43:0/1766000395 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb4010690 con 0x7f3ccc104f60 2024-09-19T05:04:02.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.254+0000 7f3cd1c00640 1 -- 172.21.15.43:0/1766000395 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ccc1a1f90 con 0x7f3ccc104f60 2024-09-19T05:04:02.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.255+0000 7f3cd1c00640 1 -- 172.21.15.43:0/1766000395 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ccc1a23f0 con 0x7f3ccc104f60 2024-09-19T05:04:02.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.256+0000 7f3cd1c00640 1 -- 172.21.15.43:0/1766000395 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ccc10b120 con 0x7f3ccc104f60 2024-09-19T05:04:02.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.258+0000 7f3cc3e00640 1 -- 172.21.15.43:0/1766000395 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f3cb40037a0 con 0x7f3ccc104f60 2024-09-19T05:04:02.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.259+0000 7f3cc3e00640 1 --2- 172.21.15.43:0/1766000395 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f3cb00a3380 0x7f3cb00a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:02.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.260+0000 7f3cc3e00640 1 -- 172.21.15.43:0/1766000395 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f3cb40cfcf0 con 0x7f3ccc104f60 2024-09-19T05:04:02.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.260+0000 7f3ccaa00640 1 --2- 172.21.15.43:0/1766000395 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f3cb00a3380 0x7f3cb00a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:02.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.260+0000 7f3ccaa00640 1 --2- 172.21.15.43:0/1766000395 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f3cb00a3380 0x7f3cb00a5840 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc005d00 tx=0x7f3cbc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:02.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.263+0000 7f3cc3e00640 1 -- 172.21.15.43:0/1766000395 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3cb409c870 con 0x7f3ccc104f60 2024-09-19T05:04:02.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.463+0000 7f3cd1c00640 1 -- 172.21.15.43:0/1766000395 --> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f3ccc062630 con 0x7f3cb00a3380 2024-09-19T05:04:02.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.464+0000 7f3cc3e00640 1 -- 172.21.15.43:0/1766000395 <== mgr.14150 v2:172.21.15.43:6800/3167108925 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f3ccc062630 con 0x7f3cb00a3380 2024-09-19T05:04:02.467 INFO:teuthology.orchestra.run.smithi043.stdout:Backend: cephadm 2024-09-19T05:04:02.467 INFO:teuthology.orchestra.run.smithi043.stdout:Available: Yes 2024-09-19T05:04:02.467 INFO:teuthology.orchestra.run.smithi043.stdout:Paused: No 2024-09-19T05:04:02.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.467+0000 7f3cd1c00640 1 -- 172.21.15.43:0/1766000395 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f3cb00a3380 msgr2=0x7f3cb00a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:02.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.467+0000 7f3cd1c00640 1 --2- 172.21.15.43:0/1766000395 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f3cb00a3380 0x7f3cb00a5840 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc005d00 tx=0x7f3cbc00b410 comp rx=0 tx=0).stop 2024-09-19T05:04:02.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.467+0000 7f3cd1c00640 1 -- 172.21.15.43:0/1766000395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ccc104f60 msgr2=0x7f3ccc19a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:02.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.467+0000 7f3cd1c00640 1 --2- 172.21.15.43:0/1766000395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ccc104f60 0x7f3ccc19a5b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3cb400e9c0 tx=0x7f3cb400ee80 comp rx=0 tx=0).stop 2024-09-19T05:04:02.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.467+0000 7f3cd1c00640 1 -- 172.21.15.43:0/1766000395 shutdown_connections 2024-09-19T05:04:02.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.467+0000 7f3cd1c00640 1 --2- 172.21.15.43:0/1766000395 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f3cb00a3380 0x7f3cb00a5840 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:02.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.467+0000 7f3cd1c00640 1 --2- 172.21.15.43:0/1766000395 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ccc105920 0x7f3ccc1a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:02.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.467+0000 7f3cd1c00640 1 --2- 172.21.15.43:0/1766000395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ccc104f60 0x7f3ccc19a5b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:02.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.467+0000 7f3cd1c00640 1 --2- 172.21.15.43:0/1766000395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ccc103e00 0x7f3ccc19a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:02.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.467+0000 7f3cd1c00640 1 -- 172.21.15.43:0/1766000395 >> 172.21.15.43:0/1766000395 conn(0x7f3ccc0ff5b0 msgr2=0x7f3ccc1082f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:02.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.467+0000 7f3cd1c00640 1 -- 172.21.15.43:0/1766000395 shutdown_connections 2024-09-19T05:04:02.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:02.467+0000 7f3cd1c00640 1 -- 172.21.15.43:0/1766000395 wait complete. 2024-09-19T05:04:03.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:02 smithi151 ceph-mon[35425]: pgmap v342: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T05:04:03.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:02 smithi096 ceph-mon[35085]: pgmap v342: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T05:04:03.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:02 smithi043 ceph-mon[30800]: pgmap v342: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T05:04:03.529 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-19T05:04:03.913 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:04.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:03 smithi151 ceph-mon[35425]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:04.215 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:03 smithi043 ceph-mon[30800]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:04.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:03 smithi096 ceph-mon[35085]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:04.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.494+0000 7fe0dbe00640 1 -- 172.21.15.43:0/93597208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0d4101c00 msgr2=0x7fe0d4102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:04.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.494+0000 7fe0dbe00640 1 --2- 172.21.15.43:0/93597208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0d4101c00 0x7fe0d4102000 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8009900 tx=0x7fe0c802a410 comp rx=0 tx=0).stop 2024-09-19T05:04:04.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.495+0000 7fe0dbe00640 1 -- 172.21.15.43:0/93597208 shutdown_connections 2024-09-19T05:04:04.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.495+0000 7fe0dbe00640 1 --2- 172.21.15.43:0/93597208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0d41037c0 0x7fe0d410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:04.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.495+0000 7fe0dbe00640 1 --2- 172.21.15.43:0/93597208 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0d4102e00 0x7fe0d4103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:04.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.495+0000 7fe0dbe00640 1 --2- 172.21.15.43:0/93597208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0d4101c00 0x7fe0d4102000 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:04.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.495+0000 7fe0dbe00640 1 -- 172.21.15.43:0/93597208 >> 172.21.15.43:0/93597208 conn(0x7fe0d40fd390 msgr2=0x7fe0d40ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:04.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.495+0000 7fe0dbe00640 1 -- 172.21.15.43:0/93597208 shutdown_connections 2024-09-19T05:04:04.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.495+0000 7fe0dbe00640 1 -- 172.21.15.43:0/93597208 wait complete. 2024-09-19T05:04:04.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.496+0000 7fe0dbe00640 1 Processor -- start 2024-09-19T05:04:04.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.496+0000 7fe0dbe00640 1 -- start start 2024-09-19T05:04:04.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.496+0000 7fe0dbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0d4101c00 0x7fe0d4075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:04.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.496+0000 7fe0dbe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0d4102e00 0x7fe0d40745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:04.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.496+0000 7fe0dbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0d41037c0 0x7fe0d4074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:04.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.496+0000 7fe0dbe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d410d880 con 0x7fe0d4101c00 2024-09-19T05:04:04.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.496+0000 7fe0dbe00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d410d700 con 0x7fe0d4102e00 2024-09-19T05:04:04.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.496+0000 7fe0dbe00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d410da00 con 0x7fe0d41037c0 2024-09-19T05:04:04.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.497+0000 7fe0d9400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0d4101c00 0x7fe0d4075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:04.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.497+0000 7fe0d9400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0d4101c00 0x7fe0d4075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38766/0 (socket says 172.21.15.43:38766) 2024-09-19T05:04:04.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.497+0000 7fe0d9400640 1 -- 172.21.15.43:0/3713791513 learned_addr learned my addr 172.21.15.43:0/3713791513 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:04.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.497+0000 7fe0d8a00640 1 --2- 172.21.15.43:0/3713791513 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0d4102e00 0x7fe0d40745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:04.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.497+0000 7fe0d9e00640 1 --2- 172.21.15.43:0/3713791513 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0d41037c0 0x7fe0d4074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:04.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.497+0000 7fe0d9400640 1 -- 172.21.15.43:0/3713791513 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0d41037c0 msgr2=0x7fe0d4074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:04.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.497+0000 7fe0d9400640 1 --2- 172.21.15.43:0/3713791513 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0d41037c0 0x7fe0d4074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:04.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.497+0000 7fe0d9400640 1 -- 172.21.15.43:0/3713791513 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0d4102e00 msgr2=0x7fe0d40745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:04.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.497+0000 7fe0d9400640 1 --2- 172.21.15.43:0/3713791513 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0d4102e00 0x7fe0d40745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:04.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.497+0000 7fe0d9400640 1 -- 172.21.15.43:0/3713791513 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0d40753e0 con 0x7fe0d4101c00 2024-09-19T05:04:04.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.497+0000 7fe0d8a00640 1 --2- 172.21.15.43:0/3713791513 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0d4102e00 0x7fe0d40745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:04:04.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.498+0000 7fe0d9400640 1 --2- 172.21.15.43:0/3713791513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0d4101c00 0x7fe0d4075f90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8031610 tx=0x7fe0c8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:04.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.498+0000 7fe0d2000640 1 -- 172.21.15.43:0/3713791513 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c803f030 con 0x7fe0d4101c00 2024-09-19T05:04:04.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.498+0000 7fe0d2000640 1 -- 172.21.15.43:0/3713791513 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0c8003ad0 con 0x7fe0d4101c00 2024-09-19T05:04:04.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.498+0000 7fe0d2000640 1 -- 172.21.15.43:0/3713791513 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8035070 con 0x7fe0d4101c00 2024-09-19T05:04:04.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.498+0000 7fe0dbe00640 1 -- 172.21.15.43:0/3713791513 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0d41a2000 con 0x7fe0d4101c00 2024-09-19T05:04:04.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.498+0000 7fe0dbe00640 1 -- 172.21.15.43:0/3713791513 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0d41a2500 con 0x7fe0d4101c00 2024-09-19T05:04:04.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.499+0000 7fe0dbe00640 1 -- 172.21.15.43:0/3713791513 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0d4108ff0 con 0x7fe0d4101c00 2024-09-19T05:04:04.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.501+0000 7fe0d2000640 1 -- 172.21.15.43:0/3713791513 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fe0c8042030 con 0x7fe0d4101c00 2024-09-19T05:04:04.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.502+0000 7fe0d2000640 1 --2- 172.21.15.43:0/3713791513 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fe0bc0a3350 0x7fe0bc0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:04.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.502+0000 7fe0d2000640 1 -- 172.21.15.43:0/3713791513 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe0c80f00a0 con 0x7fe0d4101c00 2024-09-19T05:04:04.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.503+0000 7fe0d8a00640 1 --2- 172.21.15.43:0/3713791513 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fe0bc0a3350 0x7fe0bc0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:04.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.503+0000 7fe0d8a00640 1 --2- 172.21.15.43:0/3713791513 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fe0bc0a3350 0x7fe0bc0a5810 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4075d10 tx=0x7fe0c000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:04.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.506+0000 7fe0d2000640 1 -- 172.21.15.43:0/3713791513 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe0c80bc5d0 con 0x7fe0d4101c00 2024-09-19T05:04:04.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.704+0000 7fe0dbe00640 1 -- 172.21.15.43:0/3713791513 --> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fe0d4062630 con 0x7fe0bc0a3350 2024-09-19T05:04:04.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.717+0000 7fe0d2000640 1 -- 172.21.15.43:0/3713791513 <== mgr.14150 v2:172.21.15.43:6800/3167108925 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4154 (secure 0 0 0) 0x7fe0d4062630 con 0x7fe0bc0a3350 2024-09-19T05:04:04.720 INFO:teuthology.orchestra.run.smithi043.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-19T05:04:04.720 INFO:teuthology.orchestra.run.smithi043.stdout:mds.a smithi043 running (114s) 26s ago 114s 15.4M - 19.1.1-361-g2f599a46 d2baa855c067 d6a90fee2ab1 2024-09-19T05:04:04.720 INFO:teuthology.orchestra.run.smithi043.stdout:mds.b smithi096 running (112s) 42s ago 111s 28.9M - 19.1.1-361-g2f599a46 d2baa855c067 1102336ec106 2024-09-19T05:04:04.720 INFO:teuthology.orchestra.run.smithi043.stdout:mds.c smithi151 running (2m) 42s ago 2m 17.6M - 19.1.1-361-g2f599a46 d2baa855c067 b2c85bce4d90 2024-09-19T05:04:04.720 INFO:teuthology.orchestra.run.smithi043.stdout:mds.d smithi043 running (2m) 26s ago 2m 12.7M - 19.1.1-361-g2f599a46 d2baa855c067 d91e4fed3c4c 2024-09-19T05:04:04.720 INFO:teuthology.orchestra.run.smithi043.stdout:mds.e smithi096 running (2m) 42s ago 2m 19.0M - 19.1.1-361-g2f599a46 d2baa855c067 06f3d3813f39 2024-09-19T05:04:04.720 INFO:teuthology.orchestra.run.smithi043.stdout:mds.f smithi151 running (2m) 42s ago 2m 20.0M - 19.1.1-361-g2f599a46 d2baa855c067 3bdb1ef911fa 2024-09-19T05:04:04.720 INFO:teuthology.orchestra.run.smithi043.stdout:mds.g smithi043 running (118s) 26s ago 118s 16.7M - 19.1.1-361-g2f599a46 d2baa855c067 c1495dc06331 2024-09-19T05:04:04.720 INFO:teuthology.orchestra.run.smithi043.stdout:mds.h smithi096 running (2m) 42s ago 2m 19.1M - 19.1.1-361-g2f599a46 d2baa855c067 3bd8c04d61ff 2024-09-19T05:04:04.721 INFO:teuthology.orchestra.run.smithi043.stdout:mds.i smithi151 running (2m) 42s ago 2m 17.9M - 19.1.1-361-g2f599a46 d2baa855c067 17ab15583412 2024-09-19T05:04:04.721 INFO:teuthology.orchestra.run.smithi043.stdout:mds.j smithi043 running (2m) 26s ago 2m 12.8M - 19.1.1-361-g2f599a46 d2baa855c067 80bca36d1ddd 2024-09-19T05:04:04.721 INFO:teuthology.orchestra.run.smithi043.stdout:mds.k smithi096 running (2m) 42s ago 2m 18.3M - 19.1.1-361-g2f599a46 d2baa855c067 843ac97aa1db 2024-09-19T05:04:04.721 INFO:teuthology.orchestra.run.smithi043.stdout:mds.l smithi151 running (2m) 42s ago 2m 16.7M - 19.1.1-361-g2f599a46 d2baa855c067 f4341073aa7a 2024-09-19T05:04:04.721 INFO:teuthology.orchestra.run.smithi043.stdout:mgr.x smithi043 *:9283,8765 running (9m) 26s ago 9m 508M - 19.1.1-361-g2f599a46 d2baa855c067 185a2dbed7db 2024-09-19T05:04:04.721 INFO:teuthology.orchestra.run.smithi043.stdout:mgr.y smithi096 *:8443,8765 running (7m) 42s ago 7m 442M - 19.1.1-361-g2f599a46 d2baa855c067 97a90af90dc6 2024-09-19T05:04:04.721 INFO:teuthology.orchestra.run.smithi043.stdout:mgr.z smithi151 *:8443,8765 running (7m) 42s ago 7m 441M - 19.1.1-361-g2f599a46 d2baa855c067 1e3c4197daf2 2024-09-19T05:04:04.721 INFO:teuthology.orchestra.run.smithi043.stdout:mon.a smithi043 running (9m) 26s ago 9m 49.2M 2048M 19.1.1-361-g2f599a46 d2baa855c067 cd395eb8a2c3 2024-09-19T05:04:04.721 INFO:teuthology.orchestra.run.smithi043.stdout:mon.b smithi096 running (8m) 42s ago 8m 41.2M 2048M 19.1.1-361-g2f599a46 d2baa855c067 7a4dd79a850d 2024-09-19T05:04:04.721 INFO:teuthology.orchestra.run.smithi043.stdout:mon.c smithi151 running (8m) 42s ago 8m 40.7M 2048M 19.1.1-361-g2f599a46 d2baa855c067 92705e2799e7 2024-09-19T05:04:04.721 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0 smithi043 running (7m) 26s ago 7m 54.8M 4273M 19.1.1-361-g2f599a46 d2baa855c067 d6d47d99edac 2024-09-19T05:04:04.721 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1 smithi096 running (7m) 42s ago 7m 56.4M 4272M 19.1.1-361-g2f599a46 d2baa855c067 74367a52818f 2024-09-19T05:04:04.721 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2 smithi151 running (6m) 42s ago 6m 47.0M 4272M 19.1.1-361-g2f599a46 d2baa855c067 81f63e983b99 2024-09-19T05:04:04.722 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3 smithi043 running (6m) 26s ago 6m 52.7M 4273M 19.1.1-361-g2f599a46 d2baa855c067 12571aea2f41 2024-09-19T05:04:04.722 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4 smithi096 running (5m) 42s ago 5m 45.2M 4272M 19.1.1-361-g2f599a46 d2baa855c067 39519f99e72f 2024-09-19T05:04:04.722 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5 smithi151 running (5m) 42s ago 5m 54.7M 4272M 19.1.1-361-g2f599a46 d2baa855c067 a48b682ae76f 2024-09-19T05:04:04.722 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6 smithi043 running (4m) 26s ago 4m 41.4M 4273M 19.1.1-361-g2f599a46 d2baa855c067 1abd2be77e5b 2024-09-19T05:04:04.722 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7 smithi096 running (4m) 42s ago 4m 46.8M 4272M 19.1.1-361-g2f599a46 d2baa855c067 179eb0c5de23 2024-09-19T05:04:04.722 INFO:teuthology.orchestra.run.smithi043.stdout:osd.8 smithi151 running (4m) 42s ago 4m 53.5M 4272M 19.1.1-361-g2f599a46 d2baa855c067 3567e6bc34cd 2024-09-19T05:04:04.722 INFO:teuthology.orchestra.run.smithi043.stdout:osd.9 smithi043 running (3m) 26s ago 3m 39.4M 4273M 19.1.1-361-g2f599a46 d2baa855c067 b3dc20aefb07 2024-09-19T05:04:04.722 INFO:teuthology.orchestra.run.smithi043.stdout:osd.10 smithi096 running (3m) 42s ago 3m 54.7M 4272M 19.1.1-361-g2f599a46 d2baa855c067 fce4e0704c19 2024-09-19T05:04:04.722 INFO:teuthology.orchestra.run.smithi043.stdout:osd.11 smithi151 running (2m) 42s ago 2m 54.7M 4272M 19.1.1-361-g2f599a46 d2baa855c067 bb8d55cff5fc 2024-09-19T05:04:04.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.720+0000 7fe0dbe00640 1 -- 172.21.15.43:0/3713791513 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fe0bc0a3350 msgr2=0x7fe0bc0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:04.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.720+0000 7fe0dbe00640 1 --2- 172.21.15.43:0/3713791513 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fe0bc0a3350 0x7fe0bc0a5810 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4075d10 tx=0x7fe0c000b390 comp rx=0 tx=0).stop 2024-09-19T05:04:04.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.720+0000 7fe0dbe00640 1 -- 172.21.15.43:0/3713791513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0d4101c00 msgr2=0x7fe0d4075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:04.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.720+0000 7fe0dbe00640 1 --2- 172.21.15.43:0/3713791513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0d4101c00 0x7fe0d4075f90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8031610 tx=0x7fe0c8004270 comp rx=0 tx=0).stop 2024-09-19T05:04:04.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.720+0000 7fe0dbe00640 1 -- 172.21.15.43:0/3713791513 shutdown_connections 2024-09-19T05:04:04.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.720+0000 7fe0dbe00640 1 --2- 172.21.15.43:0/3713791513 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fe0bc0a3350 0x7fe0bc0a5810 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:04.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.720+0000 7fe0dbe00640 1 --2- 172.21.15.43:0/3713791513 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0d41037c0 0x7fe0d4074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:04.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.720+0000 7fe0dbe00640 1 --2- 172.21.15.43:0/3713791513 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0d4102e00 0x7fe0d40745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:04.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.720+0000 7fe0dbe00640 1 --2- 172.21.15.43:0/3713791513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0d4101c00 0x7fe0d4075f90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:04.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.720+0000 7fe0dbe00640 1 -- 172.21.15.43:0/3713791513 >> 172.21.15.43:0/3713791513 conn(0x7fe0d40fd390 msgr2=0x7fe0d41061c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:04.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.720+0000 7fe0dbe00640 1 -- 172.21.15.43:0/3713791513 shutdown_connections 2024-09-19T05:04:04.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:04.720+0000 7fe0dbe00640 1 -- 172.21.15.43:0/3713791513 wait complete. 2024-09-19T05:04:05.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:04 smithi043 ceph-mon[30800]: pgmap v343: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-19T05:04:05.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:04 smithi151 ceph-mon[35425]: pgmap v343: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-19T05:04:05.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:04 smithi096 ceph-mon[35085]: pgmap v343: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-19T05:04:05.626 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-19T05:04:06.015 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:06.155 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:05 smithi043 ceph-mon[30800]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:06.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:05 smithi151 ceph-mon[35425]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:06.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:05 smithi096 ceph-mon[35085]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:06.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.629+0000 7f87ca400640 1 -- 172.21.15.43:0/913087627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87c4104e50 msgr2=0x7f87c41052d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:06.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.629+0000 7f87ca400640 1 --2- 172.21.15.43:0/913087627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87c4104e50 0x7f87c41052d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f87b0009860 tx=0x7f87b002a340 comp rx=0 tx=0).stop 2024-09-19T05:04:06.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.630+0000 7f87ca400640 1 -- 172.21.15.43:0/913087627 shutdown_connections 2024-09-19T05:04:06.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.630+0000 7f87ca400640 1 --2- 172.21.15.43:0/913087627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87c4105810 0x7f87c410c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:06.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.630+0000 7f87ca400640 1 --2- 172.21.15.43:0/913087627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87c4104e50 0x7f87c41052d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:06.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.630+0000 7f87ca400640 1 --2- 172.21.15.43:0/913087627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87c4103c50 0x7f87c4104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:06.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.630+0000 7f87ca400640 1 -- 172.21.15.43:0/913087627 >> 172.21.15.43:0/913087627 conn(0x7f87c40ff3e0 msgr2=0x7f87c4101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:06.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.630+0000 7f87ca400640 1 -- 172.21.15.43:0/913087627 shutdown_connections 2024-09-19T05:04:06.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.630+0000 7f87ca400640 1 -- 172.21.15.43:0/913087627 wait complete. 2024-09-19T05:04:06.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.631+0000 7f87ca400640 1 Processor -- start 2024-09-19T05:04:06.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.631+0000 7f87ca400640 1 -- start start 2024-09-19T05:04:06.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.631+0000 7f87ca400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87c4103c50 0x7f87c4199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:06.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.631+0000 7f87ca400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87c4104e50 0x7f87c419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:06.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.631+0000 7f87ca400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87c4105810 0x7f87c41a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:06.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.631+0000 7f87ca400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f87c410f8d0 con 0x7f87c4103c50 2024-09-19T05:04:06.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.631+0000 7f87ca400640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f87c410f750 con 0x7f87c4104e50 2024-09-19T05:04:06.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.631+0000 7f87ca400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f87c410fa50 con 0x7f87c4105810 2024-09-19T05:04:06.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.631+0000 7f87c9000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87c4103c50 0x7f87c4199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:06.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.631+0000 7f87c9000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87c4103c50 0x7f87c4199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40034/0 (socket says 172.21.15.43:40034) 2024-09-19T05:04:06.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.631+0000 7f87c3e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87c4104e50 0x7f87c419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:06.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.631+0000 7f87c9000640 1 -- 172.21.15.43:0/3491913923 learned_addr learned my addr 172.21.15.43:0/3491913923 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:06.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.631+0000 7f87c9a00640 1 --2- 172.21.15.43:0/3491913923 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87c4105810 0x7f87c41a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:06.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.632+0000 7f87c3e00640 1 -- 172.21.15.43:0/3491913923 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87c4105810 msgr2=0x7f87c41a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:06.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.632+0000 7f87c3e00640 1 --2- 172.21.15.43:0/3491913923 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87c4105810 0x7f87c41a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:06.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.632+0000 7f87c3e00640 1 -- 172.21.15.43:0/3491913923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87c4103c50 msgr2=0x7f87c4199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:06.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.632+0000 7f87c3e00640 1 --2- 172.21.15.43:0/3491913923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87c4103c50 0x7f87c4199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:06.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.632+0000 7f87c3e00640 1 -- 172.21.15.43:0/3491913923 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87c41a1b80 con 0x7f87c4104e50 2024-09-19T05:04:06.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.632+0000 7f87c3e00640 1 --2- 172.21.15.43:0/3491913923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87c4104e50 0x7f87c419a490 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f87b0035040 tx=0x7f87b0033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:06.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.632+0000 7f87c1600640 1 -- 172.21.15.43:0/3491913923 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b0037040 con 0x7f87c4104e50 2024-09-19T05:04:06.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.633+0000 7f87ca400640 1 -- 172.21.15.43:0/3491913923 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87c41a1e10 con 0x7f87c4104e50 2024-09-19T05:04:06.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.633+0000 7f87c1600640 1 -- 172.21.15.43:0/3491913923 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87b0030850 con 0x7f87c4104e50 2024-09-19T05:04:06.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.633+0000 7f87c1600640 1 -- 172.21.15.43:0/3491913923 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b0004920 con 0x7f87c4104e50 2024-09-19T05:04:06.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.633+0000 7f87ca400640 1 -- 172.21.15.43:0/3491913923 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87c41a2270 con 0x7f87c4104e50 2024-09-19T05:04:06.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.634+0000 7f87ca400640 1 -- 172.21.15.43:0/3491913923 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87c410b0a0 con 0x7f87c4104e50 2024-09-19T05:04:06.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.636+0000 7f87c1600640 1 -- 172.21.15.43:0/3491913923 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f87b0004050 con 0x7f87c4104e50 2024-09-19T05:04:06.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.637+0000 7f87c1600640 1 --2- 172.21.15.43:0/3491913923 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f87ac0a3300 0x7f87ac0a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:06.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.637+0000 7f87c1600640 1 -- 172.21.15.43:0/3491913923 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f87b00ef700 con 0x7f87c4104e50 2024-09-19T05:04:06.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.637+0000 7f87c9000640 1 --2- 172.21.15.43:0/3491913923 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f87ac0a3300 0x7f87ac0a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:06.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.638+0000 7f87c9000640 1 --2- 172.21.15.43:0/3491913923 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f87ac0a3300 0x7f87ac0a57c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f87b80058c0 tx=0x7f87b8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:06.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.639+0000 7f87c1600640 1 -- 172.21.15.43:0/3491913923 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f87b00bc390 con 0x7f87c4104e50 2024-09-19T05:04:06.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.836+0000 7f87ca400640 1 -- 172.21.15.43:0/3491913923 --> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f87c4062630 con 0x7f87ac0a3300 2024-09-19T05:04:06.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.842+0000 7f87c1600640 1 -- 172.21.15.43:0/3491913923 <== mgr.14150 v2:172.21.15.43:6800/3167108925 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f87c4062630 con 0x7f87ac0a3300 2024-09-19T05:04:06.844 INFO:teuthology.orchestra.run.smithi043.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-19T05:04:06.844 INFO:teuthology.orchestra.run.smithi043.stdout:mds.all 12/12 44s ago 2m smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l;count:12 2024-09-19T05:04:06.844 INFO:teuthology.orchestra.run.smithi043.stdout:mgr 3/3 44s ago 8m smithi043=x;smithi096=y;smithi151=z;count:3 2024-09-19T05:04:06.845 INFO:teuthology.orchestra.run.smithi043.stdout:mon 3/3 44s ago 8m smithi043:172.21.15.43=a;smithi096:172.21.15.96=b;smithi151:172.21.15.151=c;count:3 2024-09-19T05:04:06.845 INFO:teuthology.orchestra.run.smithi043.stdout:osd 12 44s ago - 2024-09-19T05:04:06.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.844+0000 7f87ca400640 1 -- 172.21.15.43:0/3491913923 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f87ac0a3300 msgr2=0x7f87ac0a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:06.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.844+0000 7f87ca400640 1 --2- 172.21.15.43:0/3491913923 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f87ac0a3300 0x7f87ac0a57c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f87b80058c0 tx=0x7f87b8005f90 comp rx=0 tx=0).stop 2024-09-19T05:04:06.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.844+0000 7f87ca400640 1 -- 172.21.15.43:0/3491913923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87c4104e50 msgr2=0x7f87c419a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:06.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.844+0000 7f87ca400640 1 --2- 172.21.15.43:0/3491913923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87c4104e50 0x7f87c419a490 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f87b0035040 tx=0x7f87b0033d90 comp rx=0 tx=0).stop 2024-09-19T05:04:06.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.844+0000 7f87ca400640 1 -- 172.21.15.43:0/3491913923 shutdown_connections 2024-09-19T05:04:06.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.844+0000 7f87ca400640 1 --2- 172.21.15.43:0/3491913923 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f87ac0a3300 0x7f87ac0a57c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:06.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.844+0000 7f87ca400640 1 --2- 172.21.15.43:0/3491913923 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87c4105810 0x7f87c41a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:06.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.844+0000 7f87ca400640 1 --2- 172.21.15.43:0/3491913923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87c4104e50 0x7f87c419a490 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:06.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.844+0000 7f87ca400640 1 --2- 172.21.15.43:0/3491913923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87c4103c50 0x7f87c4199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:06.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.844+0000 7f87ca400640 1 -- 172.21.15.43:0/3491913923 >> 172.21.15.43:0/3491913923 conn(0x7f87c40ff3e0 msgr2=0x7f87c4108270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:06.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.844+0000 7f87ca400640 1 -- 172.21.15.43:0/3491913923 shutdown_connections 2024-09-19T05:04:06.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:06.844+0000 7f87ca400640 1 -- 172.21.15.43:0/3491913923 wait complete. 2024-09-19T05:04:07.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:06 smithi151 ceph-mon[35425]: pgmap v344: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:04:07.218 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:06 smithi043 ceph-mon[30800]: pgmap v344: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:04:07.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:06 smithi096 ceph-mon[35085]: pgmap v344: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:04:07.574 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-19T05:04:07.960 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:08.123 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:07 smithi043 ceph-mon[30800]: from='client.24983 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:08.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:07 smithi151 ceph-mon[35425]: from='client.24983 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:08.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:07 smithi096 ceph-mon[35085]: from='client.24983 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:08.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.511+0000 7fbf58000640 1 -- 172.21.15.43:0/361807628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf50102e00 msgr2=0x7fbf50103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:08.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.511+0000 7fbf58000640 1 --2- 172.21.15.43:0/361807628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf50102e00 0x7fbf50103280 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c009900 tx=0x7fbf3c02a410 comp rx=0 tx=0).stop 2024-09-19T05:04:08.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.511+0000 7fbf58000640 1 -- 172.21.15.43:0/361807628 shutdown_connections 2024-09-19T05:04:08.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.511+0000 7fbf58000640 1 --2- 172.21.15.43:0/361807628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf501037c0 0x7fbf5010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:08.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.511+0000 7fbf58000640 1 --2- 172.21.15.43:0/361807628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf50102e00 0x7fbf50103280 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:08.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.511+0000 7fbf58000640 1 --2- 172.21.15.43:0/361807628 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf50101c00 0x7fbf50102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:08.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.511+0000 7fbf58000640 1 -- 172.21.15.43:0/361807628 >> 172.21.15.43:0/361807628 conn(0x7fbf500fd390 msgr2=0x7fbf500ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:08.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.511+0000 7fbf58000640 1 -- 172.21.15.43:0/361807628 shutdown_connections 2024-09-19T05:04:08.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.511+0000 7fbf58000640 1 -- 172.21.15.43:0/361807628 wait complete. 2024-09-19T05:04:08.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.512+0000 7fbf58000640 1 Processor -- start 2024-09-19T05:04:08.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.513+0000 7fbf58000640 1 -- start start 2024-09-19T05:04:08.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.513+0000 7fbf58000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf50101c00 0x7fbf5019a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:08.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.513+0000 7fbf58000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf50102e00 0x7fbf5019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:08.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.513+0000 7fbf58000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf501037c0 0x7fbf501a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:08.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.513+0000 7fbf58000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5010d880 con 0x7fbf50101c00 2024-09-19T05:04:08.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.513+0000 7fbf58000640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5010d700 con 0x7fbf50102e00 2024-09-19T05:04:08.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.513+0000 7fbf58000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5010da00 con 0x7fbf501037c0 2024-09-19T05:04:08.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.513+0000 7fbf55600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf50101c00 0x7fbf5019a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:08.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.513+0000 7fbf55600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf50101c00 0x7fbf5019a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40050/0 (socket says 172.21.15.43:40050) 2024-09-19T05:04:08.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.513+0000 7fbf55600640 1 -- 172.21.15.43:0/1957308013 learned_addr learned my addr 172.21.15.43:0/1957308013 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:08.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.514+0000 7fbf54c00640 1 --2- 172.21.15.43:0/1957308013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf50102e00 0x7fbf5019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:08.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.514+0000 7fbf56000640 1 --2- 172.21.15.43:0/1957308013 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf501037c0 0x7fbf501a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:08.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.514+0000 7fbf55600640 1 -- 172.21.15.43:0/1957308013 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf501037c0 msgr2=0x7fbf501a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:08.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.514+0000 7fbf55600640 1 --2- 172.21.15.43:0/1957308013 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf501037c0 0x7fbf501a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:08.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.514+0000 7fbf55600640 1 -- 172.21.15.43:0/1957308013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf50102e00 msgr2=0x7fbf5019a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:08.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.514+0000 7fbf55600640 1 --2- 172.21.15.43:0/1957308013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf50102e00 0x7fbf5019a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:08.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.514+0000 7fbf55600640 1 -- 172.21.15.43:0/1957308013 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf501a1db0 con 0x7fbf50101c00 2024-09-19T05:04:08.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.514+0000 7fbf55600640 1 --2- 172.21.15.43:0/1957308013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf50101c00 0x7fbf5019a180 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbf4400e9c0 tx=0x7fbf4400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:08.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.515+0000 7fbf4e000640 1 -- 172.21.15.43:0/1957308013 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4400cd80 con 0x7fbf50101c00 2024-09-19T05:04:08.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.515+0000 7fbf4e000640 1 -- 172.21.15.43:0/1957308013 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf440035f0 con 0x7fbf50101c00 2024-09-19T05:04:08.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.515+0000 7fbf58000640 1 -- 172.21.15.43:0/1957308013 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf501a20a0 con 0x7fbf50101c00 2024-09-19T05:04:08.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.515+0000 7fbf4e000640 1 -- 172.21.15.43:0/1957308013 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf44010690 con 0x7fbf50101c00 2024-09-19T05:04:08.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.515+0000 7fbf58000640 1 -- 172.21.15.43:0/1957308013 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf501a25b0 con 0x7fbf50101c00 2024-09-19T05:04:08.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.516+0000 7fbf58000640 1 -- 172.21.15.43:0/1957308013 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf501a2870 con 0x7fbf50101c00 2024-09-19T05:04:08.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.518+0000 7fbf4e000640 1 -- 172.21.15.43:0/1957308013 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fbf440037a0 con 0x7fbf50101c00 2024-09-19T05:04:08.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.520+0000 7fbf4e000640 1 --2- 172.21.15.43:0/1957308013 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fbf380a32f0 0x7fbf380a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:08.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.520+0000 7fbf4e000640 1 -- 172.21.15.43:0/1957308013 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fbf440cfc00 con 0x7fbf50101c00 2024-09-19T05:04:08.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.520+0000 7fbf54c00640 1 --2- 172.21.15.43:0/1957308013 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fbf380a32f0 0x7fbf380a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:08.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.521+0000 7fbf54c00640 1 --2- 172.21.15.43:0/1957308013 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fbf380a32f0 0x7fbf380a57b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c0314e0 tx=0x7fbf3c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:08.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.524+0000 7fbf4e000640 1 -- 172.21.15.43:0/1957308013 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbf4409c8e0 con 0x7fbf50101c00 2024-09-19T05:04:08.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.717+0000 7fbf58000640 1 -- 172.21.15.43:0/1957308013 --> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fbf50109cf0 con 0x7fbf380a32f0 2024-09-19T05:04:08.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.719+0000 7fbf4e000640 1 -- 172.21.15.43:0/1957308013 <== mgr.14150 v2:172.21.15.43:6800/3167108925 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fbf50109cf0 con 0x7fbf380a32f0 2024-09-19T05:04:08.721 INFO:teuthology.orchestra.run.smithi043.stdout:HOST ADDR LABELS STATUS 2024-09-19T05:04:08.721 INFO:teuthology.orchestra.run.smithi043.stdout:smithi043 172.21.15.43 2024-09-19T05:04:08.721 INFO:teuthology.orchestra.run.smithi043.stdout:smithi096 172.21.15.96 2024-09-19T05:04:08.721 INFO:teuthology.orchestra.run.smithi043.stdout:smithi151 172.21.15.151 2024-09-19T05:04:08.722 INFO:teuthology.orchestra.run.smithi043.stdout:3 hosts in cluster 2024-09-19T05:04:08.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.721+0000 7fbf58000640 1 -- 172.21.15.43:0/1957308013 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fbf380a32f0 msgr2=0x7fbf380a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:08.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.721+0000 7fbf58000640 1 --2- 172.21.15.43:0/1957308013 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fbf380a32f0 0x7fbf380a57b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c0314e0 tx=0x7fbf3c000bc0 comp rx=0 tx=0).stop 2024-09-19T05:04:08.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.721+0000 7fbf58000640 1 -- 172.21.15.43:0/1957308013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf50101c00 msgr2=0x7fbf5019a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:08.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.721+0000 7fbf58000640 1 --2- 172.21.15.43:0/1957308013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf50101c00 0x7fbf5019a180 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbf4400e9c0 tx=0x7fbf4400ee80 comp rx=0 tx=0).stop 2024-09-19T05:04:08.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.722+0000 7fbf58000640 1 -- 172.21.15.43:0/1957308013 shutdown_connections 2024-09-19T05:04:08.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.722+0000 7fbf58000640 1 --2- 172.21.15.43:0/1957308013 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fbf380a32f0 0x7fbf380a57b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:08.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.722+0000 7fbf58000640 1 --2- 172.21.15.43:0/1957308013 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf501037c0 0x7fbf501a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:08.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.722+0000 7fbf58000640 1 --2- 172.21.15.43:0/1957308013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf50102e00 0x7fbf5019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:08.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.722+0000 7fbf58000640 1 --2- 172.21.15.43:0/1957308013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf50101c00 0x7fbf5019a180 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:08.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.722+0000 7fbf58000640 1 -- 172.21.15.43:0/1957308013 >> 172.21.15.43:0/1957308013 conn(0x7fbf500fd390 msgr2=0x7fbf501085d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:08.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.722+0000 7fbf58000640 1 -- 172.21.15.43:0/1957308013 shutdown_connections 2024-09-19T05:04:08.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:08.722+0000 7fbf58000640 1 -- 172.21.15.43:0/1957308013 wait complete. 2024-09-19T05:04:09.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:08 smithi043 ceph-mon[30800]: pgmap v345: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T05:04:09.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:08 smithi043 ceph-mon[30800]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-19T05:04:09.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:08 smithi151 ceph-mon[35425]: pgmap v345: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T05:04:09.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:08 smithi151 ceph-mon[35425]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-19T05:04:09.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:08 smithi096 ceph-mon[35085]: pgmap v345: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T05:04:09.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:08 smithi096 ceph-mon[35085]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-19T05:04:09.694 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-19T05:04:10.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:09 smithi043 ceph-mon[30800]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:10.074 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:10.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:09 smithi151 ceph-mon[35425]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:10.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:09 smithi096 ceph-mon[35085]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:10.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.642+0000 7f4b32800640 1 -- 172.21.15.43:0/389118932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b2c074b40 msgr2=0x7f4b2c074fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:10.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.642+0000 7f4b32800640 1 --2- 172.21.15.43:0/389118932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b2c074b40 0x7f4b2c074fa0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4b18009900 tx=0x7f4b1802a3f0 comp rx=0 tx=0).stop 2024-09-19T05:04:10.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.642+0000 7f4b32800640 1 -- 172.21.15.43:0/389118932 shutdown_connections 2024-09-19T05:04:10.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.642+0000 7f4b32800640 1 --2- 172.21.15.43:0/389118932 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b2c1098b0 0x7f4b2c10bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:10.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.642+0000 7f4b32800640 1 --2- 172.21.15.43:0/389118932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b2c074b40 0x7f4b2c074fa0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:10.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.642+0000 7f4b32800640 1 --2- 172.21.15.43:0/389118932 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b2c0761a0 0x7f4b2c074600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:10.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.642+0000 7f4b32800640 1 -- 172.21.15.43:0/389118932 >> 172.21.15.43:0/389118932 conn(0x7f4b2c0fd1f0 msgr2=0x7f4b2c0ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:10.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.642+0000 7f4b32800640 1 -- 172.21.15.43:0/389118932 shutdown_connections 2024-09-19T05:04:10.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.642+0000 7f4b32800640 1 -- 172.21.15.43:0/389118932 wait complete. 2024-09-19T05:04:10.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.643+0000 7f4b32800640 1 Processor -- start 2024-09-19T05:04:10.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.643+0000 7f4b32800640 1 -- start start 2024-09-19T05:04:10.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.643+0000 7f4b32800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b2c074b40 0x7f4b2c199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:10.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.643+0000 7f4b32800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b2c0761a0 0x7f4b2c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:10.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.643+0000 7f4b32800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b2c1098b0 0x7f4b2c1a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:10.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.643+0000 7f4b32800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c10f900 con 0x7f4b2c0761a0 2024-09-19T05:04:10.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.643+0000 7f4b32800640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c10f780 con 0x7f4b2c1098b0 2024-09-19T05:04:10.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.643+0000 7f4b32800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c10fa80 con 0x7f4b2c074b40 2024-09-19T05:04:10.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.644+0000 7f4b31400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b2c074b40 0x7f4b2c199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:10.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.644+0000 7f4b31400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b2c074b40 0x7f4b2c199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:42556/0 (socket says 172.21.15.43:42556) 2024-09-19T05:04:10.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.644+0000 7f4b31400640 1 -- 172.21.15.43:0/923405105 learned_addr learned my addr 172.21.15.43:0/923405105 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:10.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.644+0000 7f4b30a00640 1 --2- 172.21.15.43:0/923405105 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b2c0761a0 0x7f4b2c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:10.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.644+0000 7f4b31e00640 1 --2- 172.21.15.43:0/923405105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b2c1098b0 0x7f4b2c1a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:10.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.644+0000 7f4b31400640 1 -- 172.21.15.43:0/923405105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b2c1098b0 msgr2=0x7f4b2c1a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:10.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.644+0000 7f4b31400640 1 --2- 172.21.15.43:0/923405105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b2c1098b0 0x7f4b2c1a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:10.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.644+0000 7f4b31400640 1 -- 172.21.15.43:0/923405105 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b2c0761a0 msgr2=0x7f4b2c19a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:10.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.644+0000 7f4b31400640 1 --2- 172.21.15.43:0/923405105 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b2c0761a0 0x7f4b2c19a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:10.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.644+0000 7f4b31400640 1 -- 172.21.15.43:0/923405105 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b2c1a1b80 con 0x7f4b2c074b40 2024-09-19T05:04:10.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.645+0000 7f4b31400640 1 --2- 172.21.15.43:0/923405105 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b2c074b40 0x7f4b2c199f50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4b2000e9f0 tx=0x7f4b2000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:10.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.645+0000 7f4b2a000640 1 -- 172.21.15.43:0/923405105 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2000cd80 con 0x7f4b2c074b40 2024-09-19T05:04:10.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.645+0000 7f4b2a000640 1 -- 172.21.15.43:0/923405105 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b20005500 con 0x7f4b2c074b40 2024-09-19T05:04:10.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.645+0000 7f4b32800640 1 -- 172.21.15.43:0/923405105 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b2c1a1e70 con 0x7f4b2c074b40 2024-09-19T05:04:10.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.645+0000 7f4b2a000640 1 -- 172.21.15.43:0/923405105 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b20010690 con 0x7f4b2c074b40 2024-09-19T05:04:10.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.645+0000 7f4b32800640 1 -- 172.21.15.43:0/923405105 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b2c1a2270 con 0x7f4b2c074b40 2024-09-19T05:04:10.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.646+0000 7f4b32800640 1 -- 172.21.15.43:0/923405105 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b2c1086c0 con 0x7f4b2c074b40 2024-09-19T05:04:10.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.648+0000 7f4b2a000640 1 -- 172.21.15.43:0/923405105 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f4b200046c0 con 0x7f4b2c074b40 2024-09-19T05:04:10.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.649+0000 7f4b2a000640 1 --2- 172.21.15.43:0/923405105 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f4b140a3270 0x7f4b140a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:10.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.649+0000 7f4b2a000640 1 -- 172.21.15.43:0/923405105 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f4b200cfd50 con 0x7f4b2c074b40 2024-09-19T05:04:10.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.650+0000 7f4b30a00640 1 --2- 172.21.15.43:0/923405105 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f4b140a3270 0x7f4b140a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:10.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.650+0000 7f4b30a00640 1 --2- 172.21.15.43:0/923405105 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f4b140a3270 0x7f4b140a5730 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4b18031440 tx=0x7f4b180315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:10.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.653+0000 7f4b2a000640 1 -- 172.21.15.43:0/923405105 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4b2009ca30 con 0x7f4b2c074b40 2024-09-19T05:04:10.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.848+0000 7f4b32800640 1 -- 172.21.15.43:0/923405105 --> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f4b2c062630 con 0x7f4b140a3270 2024-09-19T05:04:10.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.850+0000 7f4b2a000640 1 -- 172.21.15.43:0/923405105 <== mgr.14150 v2:172.21.15.43:6800/3167108925 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f4b2c062630 con 0x7f4b140a3270 2024-09-19T05:04:10.852 INFO:teuthology.orchestra.run.smithi043.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-19T05:04:10.853 INFO:teuthology.orchestra.run.smithi043.stdout:smithi043 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300GH480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-09-19T05:04:10.853 INFO:teuthology.orchestra.run.smithi043.stdout:smithi096 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NH400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-19T05:04:10.853 INFO:teuthology.orchestra.run.smithi043.stdout:smithi151 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300JJ400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-19T05:04:10.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.853+0000 7f4b32800640 1 -- 172.21.15.43:0/923405105 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f4b140a3270 msgr2=0x7f4b140a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:10.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.853+0000 7f4b32800640 1 --2- 172.21.15.43:0/923405105 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f4b140a3270 0x7f4b140a5730 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4b18031440 tx=0x7f4b180315b0 comp rx=0 tx=0).stop 2024-09-19T05:04:10.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.853+0000 7f4b32800640 1 -- 172.21.15.43:0/923405105 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b2c074b40 msgr2=0x7f4b2c199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:10.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.853+0000 7f4b32800640 1 --2- 172.21.15.43:0/923405105 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b2c074b40 0x7f4b2c199f50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4b2000e9f0 tx=0x7f4b2000edf0 comp rx=0 tx=0).stop 2024-09-19T05:04:10.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.853+0000 7f4b32800640 1 -- 172.21.15.43:0/923405105 shutdown_connections 2024-09-19T05:04:10.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.853+0000 7f4b32800640 1 --2- 172.21.15.43:0/923405105 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f4b140a3270 0x7f4b140a5730 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:10.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.853+0000 7f4b32800640 1 --2- 172.21.15.43:0/923405105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b2c1098b0 0x7f4b2c1a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:10.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.853+0000 7f4b32800640 1 --2- 172.21.15.43:0/923405105 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b2c0761a0 0x7f4b2c19a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:10.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.853+0000 7f4b32800640 1 --2- 172.21.15.43:0/923405105 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b2c074b40 0x7f4b2c199f50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:10.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.853+0000 7f4b32800640 1 -- 172.21.15.43:0/923405105 >> 172.21.15.43:0/923405105 conn(0x7f4b2c0fd1f0 msgr2=0x7f4b2c105890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:10.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.853+0000 7f4b32800640 1 -- 172.21.15.43:0/923405105 shutdown_connections 2024-09-19T05:04:10.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:10.853+0000 7f4b32800640 1 -- 172.21.15.43:0/923405105 wait complete. 2024-09-19T05:04:11.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:10 smithi151 ceph-mon[35425]: pgmap v346: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-19T05:04:11.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:10 smithi096 ceph-mon[35085]: pgmap v346: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-19T05:04:11.228 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:10 smithi043 ceph-mon[30800]: pgmap v346: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-19T05:04:11.722 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-19T05:04:11.731 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi043.front.sepia.ceph.com 2024-09-19T05:04:11.732 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-19T05:04:12.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:11 smithi043 ceph-mon[30800]: from='client.24979 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:12.109 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:12.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:11 smithi151 ceph-mon[35425]: from='client.24979 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:12.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:11 smithi096 ceph-mon[35085]: from='client.24979 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:12.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.683+0000 7f7d30c00640 1 -- 172.21.15.43:0/984495039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d2c101a40 msgr2=0x7f7d2c101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:12.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.683+0000 7f7d30c00640 1 --2- 172.21.15.43:0/984495039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d2c101a40 0x7f7d2c101e40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c009860 tx=0x7f7d1c02a340 comp rx=0 tx=0).stop 2024-09-19T05:04:12.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.683+0000 7f7d30c00640 1 -- 172.21.15.43:0/984495039 shutdown_connections 2024-09-19T05:04:12.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.683+0000 7f7d30c00640 1 --2- 172.21.15.43:0/984495039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d2c103600 0x7f7d2c109e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:12.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.683+0000 7f7d30c00640 1 --2- 172.21.15.43:0/984495039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d2c102c40 0x7f7d2c1030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:12.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.683+0000 7f7d30c00640 1 --2- 172.21.15.43:0/984495039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d2c101a40 0x7f7d2c101e40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:12.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.683+0000 7f7d30c00640 1 -- 172.21.15.43:0/984495039 >> 172.21.15.43:0/984495039 conn(0x7f7d2c0fd1f0 msgr2=0x7f7d2c0ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:12.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.683+0000 7f7d30c00640 1 -- 172.21.15.43:0/984495039 shutdown_connections 2024-09-19T05:04:12.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.683+0000 7f7d30c00640 1 -- 172.21.15.43:0/984495039 wait complete. 2024-09-19T05:04:12.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.684+0000 7f7d30c00640 1 Processor -- start 2024-09-19T05:04:12.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.684+0000 7f7d30c00640 1 -- start start 2024-09-19T05:04:12.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.685+0000 7f7d30c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d2c101a40 0x7f7d2c075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:12.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.685+0000 7f7d30c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d2c102c40 0x7f7d2c0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:12.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.685+0000 7f7d30c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d2c103600 0x7f7d2c074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:12.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.685+0000 7f7d30c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c10d6a0 con 0x7f7d2c102c40 2024-09-19T05:04:12.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.685+0000 7f7d30c00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c10d520 con 0x7f7d2c101a40 2024-09-19T05:04:12.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.685+0000 7f7d30c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c10d820 con 0x7f7d2c103600 2024-09-19T05:04:12.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.685+0000 7f7d2b400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d2c101a40 0x7f7d2c075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:12.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.685+0000 7f7d2b400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d2c101a40 0x7f7d2c075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:52494/0 (socket says 172.21.15.43:52494) 2024-09-19T05:04:12.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.685+0000 7f7d2b400640 1 -- 172.21.15.43:0/3456106216 learned_addr learned my addr 172.21.15.43:0/3456106216 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:12.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.685+0000 7f7d23e00640 1 --2- 172.21.15.43:0/3456106216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d2c102c40 0x7f7d2c0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:12.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.685+0000 7f7d2be00640 1 --2- 172.21.15.43:0/3456106216 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d2c103600 0x7f7d2c074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:12.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.685+0000 7f7d2b400640 1 -- 172.21.15.43:0/3456106216 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d2c103600 msgr2=0x7f7d2c074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:12.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.686+0000 7f7d2b400640 1 --2- 172.21.15.43:0/3456106216 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d2c103600 0x7f7d2c074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:12.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.686+0000 7f7d2b400640 1 -- 172.21.15.43:0/3456106216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d2c102c40 msgr2=0x7f7d2c0745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:12.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.686+0000 7f7d2b400640 1 --2- 172.21.15.43:0/3456106216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d2c102c40 0x7f7d2c0745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:12.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.686+0000 7f7d2b400640 1 -- 172.21.15.43:0/3456106216 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d2c0753e0 con 0x7f7d2c101a40 2024-09-19T05:04:12.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.686+0000 7f7d2be00640 1 --2- 172.21.15.43:0/3456106216 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d2c103600 0x7f7d2c074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:04:12.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.686+0000 7f7d2b400640 1 --2- 172.21.15.43:0/3456106216 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d2c101a40 0x7f7d2c075f90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c035040 tx=0x7f7d1c033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:12.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.686+0000 7f7d28c00640 1 -- 172.21.15.43:0/3456106216 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c037040 con 0x7f7d2c101a40 2024-09-19T05:04:12.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.686+0000 7f7d28c00640 1 -- 172.21.15.43:0/3456106216 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d1c030850 con 0x7f7d2c101a40 2024-09-19T05:04:12.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.686+0000 7f7d30c00640 1 -- 172.21.15.43:0/3456106216 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d2c1a1e60 con 0x7f7d2c101a40 2024-09-19T05:04:12.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.686+0000 7f7d28c00640 1 -- 172.21.15.43:0/3456106216 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c004920 con 0x7f7d2c101a40 2024-09-19T05:04:12.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.686+0000 7f7d30c00640 1 -- 172.21.15.43:0/3456106216 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d2c1a2200 con 0x7f7d2c101a40 2024-09-19T05:04:12.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.687+0000 7f7d30c00640 1 -- 172.21.15.43:0/3456106216 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d2c108e90 con 0x7f7d2c101a40 2024-09-19T05:04:12.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.689+0000 7f7d28c00640 1 -- 172.21.15.43:0/3456106216 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f7d1c004050 con 0x7f7d2c101a40 2024-09-19T05:04:12.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.691+0000 7f7d28c00640 1 --2- 172.21.15.43:0/3456106216 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f7d100a3240 0x7f7d100a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:12.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.691+0000 7f7d28c00640 1 -- 172.21.15.43:0/3456106216 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f7d1c0ef8b0 con 0x7f7d2c101a40 2024-09-19T05:04:12.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.691+0000 7f7d23e00640 1 --2- 172.21.15.43:0/3456106216 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f7d100a3240 0x7f7d100a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:12.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.691+0000 7f7d23e00640 1 --2- 172.21.15.43:0/3456106216 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f7d100a3240 0x7f7d100a5700 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7d140096c0 tx=0x7f7d14009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:12.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.693+0000 7f7d28c00640 1 -- 172.21.15.43:0/3456106216 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7d1c0bc620 con 0x7f7d2c101a40 2024-09-19T05:04:12.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.919+0000 7f7d30c00640 1 -- 172.21.15.43:0/3456106216 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f7d2c02d7c0 con 0x7f7d2c101a40 2024-09-19T05:04:12.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.920+0000 7f7d28c00640 1 -- 172.21.15.43:0/3456106216 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+3123 (secure 0 0 0) 0x7f7d1c0bbf20 con 0x7f7d2c101a40 2024-09-19T05:04:12.923 INFO:teuthology.orchestra.run.smithi043.stdout:e28 2024-09-19T05:04:12.923 INFO:teuthology.orchestra.run.smithi043.stdout:btime 2024-09-19T05:02:40:505009+0000 2024-09-19T05:04:12.923 INFO:teuthology.orchestra.run.smithi043.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T05:04:12.923 INFO:teuthology.orchestra.run.smithi043.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T05:04:12.924 INFO:teuthology.orchestra.run.smithi043.stdout:legacy client fscid: 1 2024-09-19T05:04:12.924 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:04:12.924 INFO:teuthology.orchestra.run.smithi043.stdout:Filesystem 'cephfs' (1) 2024-09-19T05:04:12.924 INFO:teuthology.orchestra.run.smithi043.stdout:fs_name cephfs 2024-09-19T05:04:12.924 INFO:teuthology.orchestra.run.smithi043.stdout:epoch 28 2024-09-19T05:04:12.924 INFO:teuthology.orchestra.run.smithi043.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-19T05:04:12.924 INFO:teuthology.orchestra.run.smithi043.stdout:created 2024-09-19T05:02:25.463041+0000 2024-09-19T05:04:12.924 INFO:teuthology.orchestra.run.smithi043.stdout:modified 2024-09-19T05:02:39.530912+0000 2024-09-19T05:04:12.924 INFO:teuthology.orchestra.run.smithi043.stdout:tableserver 0 2024-09-19T05:04:12.924 INFO:teuthology.orchestra.run.smithi043.stdout:root 0 2024-09-19T05:04:12.924 INFO:teuthology.orchestra.run.smithi043.stdout:session_timeout 300 2024-09-19T05:04:12.924 INFO:teuthology.orchestra.run.smithi043.stdout:session_autoclose 300 2024-09-19T05:04:12.924 INFO:teuthology.orchestra.run.smithi043.stdout:max_file_size 1099511627776 2024-09-19T05:04:12.925 INFO:teuthology.orchestra.run.smithi043.stdout:max_xattr_size 65536 2024-09-19T05:04:12.925 INFO:teuthology.orchestra.run.smithi043.stdout:required_client_features {} 2024-09-19T05:04:12.925 INFO:teuthology.orchestra.run.smithi043.stdout:last_failure 0 2024-09-19T05:04:12.925 INFO:teuthology.orchestra.run.smithi043.stdout:last_failure_osd_epoch 0 2024-09-19T05:04:12.925 INFO:teuthology.orchestra.run.smithi043.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T05:04:12.925 INFO:teuthology.orchestra.run.smithi043.stdout:max_mds 5 2024-09-19T05:04:12.925 INFO:teuthology.orchestra.run.smithi043.stdout:in 0,1,2,3,4 2024-09-19T05:04:12.925 INFO:teuthology.orchestra.run.smithi043.stdout:up {0=24542,1=24518,2=24511,3=24506,4=24493} 2024-09-19T05:04:12.925 INFO:teuthology.orchestra.run.smithi043.stdout:failed 2024-09-19T05:04:12.925 INFO:teuthology.orchestra.run.smithi043.stdout:damaged 2024-09-19T05:04:12.925 INFO:teuthology.orchestra.run.smithi043.stdout:stopped 2024-09-19T05:04:12.925 INFO:teuthology.orchestra.run.smithi043.stdout:data_pools [3] 2024-09-19T05:04:12.926 INFO:teuthology.orchestra.run.smithi043.stdout:metadata_pool 2 2024-09-19T05:04:12.926 INFO:teuthology.orchestra.run.smithi043.stdout:inline_data disabled 2024-09-19T05:04:12.926 INFO:teuthology.orchestra.run.smithi043.stdout:balancer 2024-09-19T05:04:12.926 INFO:teuthology.orchestra.run.smithi043.stdout:bal_rank_mask -1 2024-09-19T05:04:12.926 INFO:teuthology.orchestra.run.smithi043.stdout:standby_count_wanted 0 2024-09-19T05:04:12.926 INFO:teuthology.orchestra.run.smithi043.stdout:qdb_cluster leader: 24542 members: 24493,24506,24511,24518,24542 2024-09-19T05:04:12.926 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.b{0:24542} state up:active seq 5 addr [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T05:04:12.926 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.f{0:24487} state up:standby-replay seq 1 addr [v2:172.21.15.151:6834/2972514328,v1:172.21.15.151:6835/2972514328] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T05:04:12.926 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.g{1:24518} state up:active seq 9 addr [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T05:04:12.926 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.e{1:24485} state up:standby-replay seq 1 addr [v2:172.21.15.96:6834/1671043820,v1:172.21.15.96:6835/1671043820] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T05:04:12.926 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.i{2:24511} state up:active seq 10 addr [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T05:04:12.926 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.k{2:24449} state up:standby-replay seq 1 addr [v2:172.21.15.96:6832/178785812,v1:172.21.15.96:6833/178785812] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T05:04:12.926 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.h{3:24506} state up:active seq 11 addr [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T05:04:12.926 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.l{3:24442} state up:standby-replay seq 1 addr [v2:172.21.15.151:6832/2836453158,v1:172.21.15.151:6833/2836453158] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T05:04:12.927 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.c{4:24493} state up:active seq 12 addr [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T05:04:12.927 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.a{4:14541} state up:standby-replay seq 1 addr [v2:172.21.15.43:6840/794881135,v1:172.21.15.43:6841/794881135] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T05:04:12.927 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:04:12.927 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:04:12.927 INFO:teuthology.orchestra.run.smithi043.stdout:Standby daemons: 2024-09-19T05:04:12.927 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:04:12.927 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.j{-1:14481} state up:standby seq 1 addr [v2:172.21.15.43:6834/773801169,v1:172.21.15.43:6835/773801169] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T05:04:12.927 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.d{-1:14493} state up:standby seq 1 addr [v2:172.21.15.43:6836/1799137902,v1:172.21.15.43:6837/1799137902] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T05:04:12.928 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 28 2024-09-19T05:04:12.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.922+0000 7f7d30c00640 1 -- 172.21.15.43:0/3456106216 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f7d100a3240 msgr2=0x7f7d100a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:12.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.922+0000 7f7d30c00640 1 --2- 172.21.15.43:0/3456106216 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f7d100a3240 0x7f7d100a5700 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7d140096c0 tx=0x7f7d14009260 comp rx=0 tx=0).stop 2024-09-19T05:04:12.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.922+0000 7f7d30c00640 1 -- 172.21.15.43:0/3456106216 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d2c101a40 msgr2=0x7f7d2c075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:12.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.922+0000 7f7d30c00640 1 --2- 172.21.15.43:0/3456106216 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d2c101a40 0x7f7d2c075f90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c035040 tx=0x7f7d1c033d90 comp rx=0 tx=0).stop 2024-09-19T05:04:12.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.923+0000 7f7d30c00640 1 -- 172.21.15.43:0/3456106216 shutdown_connections 2024-09-19T05:04:12.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.923+0000 7f7d30c00640 1 --2- 172.21.15.43:0/3456106216 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f7d100a3240 0x7f7d100a5700 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:12.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.923+0000 7f7d30c00640 1 --2- 172.21.15.43:0/3456106216 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d2c103600 0x7f7d2c074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:12.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.923+0000 7f7d30c00640 1 --2- 172.21.15.43:0/3456106216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d2c102c40 0x7f7d2c0745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:12.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.923+0000 7f7d30c00640 1 --2- 172.21.15.43:0/3456106216 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d2c101a40 0x7f7d2c075f90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:12.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.923+0000 7f7d30c00640 1 -- 172.21.15.43:0/3456106216 >> 172.21.15.43:0/3456106216 conn(0x7f7d2c0fd1f0 msgr2=0x7f7d2c106060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:12.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.923+0000 7f7d30c00640 1 -- 172.21.15.43:0/3456106216 shutdown_connections 2024-09-19T05:04:12.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:12.923+0000 7f7d30c00640 1 -- 172.21.15.43:0/3456106216 wait complete. 2024-09-19T05:04:13.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:12 smithi151 ceph-mon[35425]: pgmap v347: 97 pgs: 97 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-19T05:04:13.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:12 smithi096 ceph-mon[35085]: pgmap v347: 97 pgs: 97 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-19T05:04:13.272 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:12 smithi043 ceph-mon[30800]: pgmap v347: 97 pgs: 97 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-19T05:04:13.738 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-19T05:04:14.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:13 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/3456106216' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T05:04:14.120 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:14.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:13 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/3456106216' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T05:04:14.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:13 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/3456106216' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T05:04:14.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.714+0000 7fe393800640 1 -- 172.21.15.43:0/813850641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe38c074c50 msgr2=0x7fe38c0750b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:14.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.714+0000 7fe393800640 1 --2- 172.21.15.43:0/813850641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe38c074c50 0x7fe38c0750b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe378009900 tx=0x7fe37802a410 comp rx=0 tx=0).stop 2024-09-19T05:04:14.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.714+0000 7fe393800640 1 -- 172.21.15.43:0/813850641 shutdown_connections 2024-09-19T05:04:14.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.714+0000 7fe393800640 1 --2- 172.21.15.43:0/813850641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe38c10e1e0 0x7fe38c1105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:14.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.714+0000 7fe393800640 1 --2- 172.21.15.43:0/813850641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe38c074c50 0x7fe38c0750b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:14.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.714+0000 7fe393800640 1 --2- 172.21.15.43:0/813850641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe38c076220 0x7fe38c074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:14.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.714+0000 7fe393800640 1 -- 172.21.15.43:0/813850641 >> 172.21.15.43:0/813850641 conn(0x7fe38c0fd390 msgr2=0x7fe38c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:14.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.715+0000 7fe393800640 1 -- 172.21.15.43:0/813850641 shutdown_connections 2024-09-19T05:04:14.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.715+0000 7fe393800640 1 -- 172.21.15.43:0/813850641 wait complete. 2024-09-19T05:04:14.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.715+0000 7fe393800640 1 Processor -- start 2024-09-19T05:04:14.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.716+0000 7fe393800640 1 -- start start 2024-09-19T05:04:14.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.716+0000 7fe393800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe38c074c50 0x7fe38c19e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:14.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.716+0000 7fe393800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe38c076220 0x7fe38c19ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:14.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.716+0000 7fe393800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe38c10e1e0 0x7fe38c1a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:14.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.716+0000 7fe393800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe38c113ea0 con 0x7fe38c10e1e0 2024-09-19T05:04:14.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.716+0000 7fe393800640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe38c113d20 con 0x7fe38c074c50 2024-09-19T05:04:14.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.716+0000 7fe393800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe38c114020 con 0x7fe38c076220 2024-09-19T05:04:14.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.716+0000 7fe390e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe38c074c50 0x7fe38c19e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:14.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.716+0000 7fe38be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe38c076220 0x7fe38c19ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:14.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.716+0000 7fe390e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe38c074c50 0x7fe38c19e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:52514/0 (socket says 172.21.15.43:52514) 2024-09-19T05:04:14.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.716+0000 7fe390e00640 1 -- 172.21.15.43:0/71753246 learned_addr learned my addr 172.21.15.43:0/71753246 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:14.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.716+0000 7fe391800640 1 --2- 172.21.15.43:0/71753246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe38c10e1e0 0x7fe38c1a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:14.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.717+0000 7fe390e00640 1 -- 172.21.15.43:0/71753246 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe38c076220 msgr2=0x7fe38c19ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:14.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.717+0000 7fe390e00640 1 --2- 172.21.15.43:0/71753246 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe38c076220 0x7fe38c19ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:14.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.717+0000 7fe390e00640 1 -- 172.21.15.43:0/71753246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe38c10e1e0 msgr2=0x7fe38c1a5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:14.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.717+0000 7fe390e00640 1 --2- 172.21.15.43:0/71753246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe38c10e1e0 0x7fe38c1a5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:14.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.717+0000 7fe390e00640 1 -- 172.21.15.43:0/71753246 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe38c1a6130 con 0x7fe38c074c50 2024-09-19T05:04:14.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.717+0000 7fe390e00640 1 --2- 172.21.15.43:0/71753246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe38c074c50 0x7fe38c19e500 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe38000e9c0 tx=0x7fe38000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:14.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.717+0000 7fe389600640 1 -- 172.21.15.43:0/71753246 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38000cd80 con 0x7fe38c074c50 2024-09-19T05:04:14.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.717+0000 7fe389600640 1 -- 172.21.15.43:0/71753246 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe380005500 con 0x7fe38c074c50 2024-09-19T05:04:14.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.717+0000 7fe393800640 1 -- 172.21.15.43:0/71753246 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe38c1a6420 con 0x7fe38c074c50 2024-09-19T05:04:14.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.717+0000 7fe389600640 1 -- 172.21.15.43:0/71753246 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe380010690 con 0x7fe38c074c50 2024-09-19T05:04:14.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.717+0000 7fe393800640 1 -- 172.21.15.43:0/71753246 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe38c1a6930 con 0x7fe38c074c50 2024-09-19T05:04:14.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.719+0000 7fe393800640 1 -- 172.21.15.43:0/71753246 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe38c103b80 con 0x7fe38c074c50 2024-09-19T05:04:14.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.720+0000 7fe389600640 1 -- 172.21.15.43:0/71753246 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fe3800046c0 con 0x7fe38c074c50 2024-09-19T05:04:14.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.722+0000 7fe389600640 1 --2- 172.21.15.43:0/71753246 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fe3740a3270 0x7fe3740a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:14.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.722+0000 7fe389600640 1 -- 172.21.15.43:0/71753246 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe3800cfca0 con 0x7fe38c074c50 2024-09-19T05:04:14.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.722+0000 7fe38be00640 1 --2- 172.21.15.43:0/71753246 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fe3740a3270 0x7fe3740a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:14.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.723+0000 7fe38be00640 1 --2- 172.21.15.43:0/71753246 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fe3740a3270 0x7fe3740a5730 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe3780314e0 tx=0x7fe378000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:14.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.726+0000 7fe389600640 1 -- 172.21.15.43:0/71753246 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe38009c980 con 0x7fe38c074c50 2024-09-19T05:04:14.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.913+0000 7fe393800640 1 -- 172.21.15.43:0/71753246 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fe38c062df0 con 0x7fe38c074c50 2024-09-19T05:04:14.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.915+0000 7fe389600640 1 -- 172.21.15.43:0/71753246 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v85) ==== 56+0+5287 (secure 0 0 0) 0x7fe38009c280 con 0x7fe38c074c50 2024-09-19T05:04:14.917 INFO:teuthology.orchestra.run.smithi043.stdout:epoch 85 2024-09-19T05:04:14.917 INFO:teuthology.orchestra.run.smithi043.stdout:fsid 1456a810-7643-11ef-bceb-c7b262605968 2024-09-19T05:04:14.917 INFO:teuthology.orchestra.run.smithi043.stdout:created 2024-09-19T04:54:14.706644+0000 2024-09-19T05:04:14.917 INFO:teuthology.orchestra.run.smithi043.stdout:modified 2024-09-19T05:03:08.697712+0000 2024-09-19T05:04:14.918 INFO:teuthology.orchestra.run.smithi043.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-19T05:04:14.918 INFO:teuthology.orchestra.run.smithi043.stdout:crush_version 26 2024-09-19T05:04:14.918 INFO:teuthology.orchestra.run.smithi043.stdout:full_ratio 0.95 2024-09-19T05:04:14.918 INFO:teuthology.orchestra.run.smithi043.stdout:backfillfull_ratio 0.9 2024-09-19T05:04:14.918 INFO:teuthology.orchestra.run.smithi043.stdout:nearfull_ratio 0.85 2024-09-19T05:04:14.918 INFO:teuthology.orchestra.run.smithi043.stdout:require_min_compat_client luminous 2024-09-19T05:04:14.918 INFO:teuthology.orchestra.run.smithi043.stdout:min_compat_client luminous 2024-09-19T05:04:14.918 INFO:teuthology.orchestra.run.smithi043.stdout:require_osd_release squid 2024-09-19T05:04:14.918 INFO:teuthology.orchestra.run.smithi043.stdout:stretch_mode_enabled false 2024-09-19T05:04:14.918 INFO:teuthology.orchestra.run.smithi043.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-19T05:04:14.918 INFO:teuthology.orchestra.run.smithi043.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 83 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-09-19T05:04:14.918 INFO:teuthology.orchestra.run.smithi043.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 83 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-19T05:04:14.918 INFO:teuthology.orchestra.run.smithi043.stdout:max_osd 12 2024-09-19T05:04:14.918 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0 up in weight 1 up_from 8 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.43:6802/825351486,v1:172.21.15.43:6803/825351486] [v2:172.21.15.43:6804/825351486,v1:172.21.15.43:6805/825351486] exists,up 81a317a8-7ec8-45c3-bc10-7da023283635 2024-09-19T05:04:14.919 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1 up in weight 1 up_from 13 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6800/2140095130,v1:172.21.15.96:6801/2140095130] [v2:172.21.15.96:6802/2140095130,v1:172.21.15.96:6803/2140095130] exists,up e96fcfac-4fff-4a36-b38b-aebf57842740 2024-09-19T05:04:14.919 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2 up in weight 1 up_from 18 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6800/2047147996,v1:172.21.15.151:6801/2047147996] [v2:172.21.15.151:6802/2047147996,v1:172.21.15.151:6803/2047147996] exists,up b659869e-114a-4f56-8aac-a2ed602b4058 2024-09-19T05:04:14.919 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3 up in weight 1 up_from 24 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.43:6810/1734673762,v1:172.21.15.43:6811/1734673762] [v2:172.21.15.43:6812/1734673762,v1:172.21.15.43:6813/1734673762] exists,up 7be64b2c-e45c-493d-a610-4ce41196061f 2024-09-19T05:04:14.919 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4 up in weight 1 up_from 31 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6808/3184785362,v1:172.21.15.96:6809/3184785362] [v2:172.21.15.96:6810/3184785362,v1:172.21.15.96:6811/3184785362] exists,up 196d5af1-a6e5-496c-8933-4e58892f4360 2024-09-19T05:04:14.919 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5 up in weight 1 up_from 37 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6808/2878451924,v1:172.21.15.151:6809/2878451924] [v2:172.21.15.151:6810/2878451924,v1:172.21.15.151:6811/2878451924] exists,up 5492d474-8509-4865-bc2e-d08e31558a75 2024-09-19T05:04:14.919 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6 up in weight 1 up_from 44 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.43:6818/2008630290,v1:172.21.15.43:6819/2008630290] [v2:172.21.15.43:6820/2008630290,v1:172.21.15.43:6821/2008630290] exists,up fb8457ed-7bb9-4435-9d15-c1cbfbddfcf3 2024-09-19T05:04:14.919 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7 up in weight 1 up_from 49 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6816/3588713275,v1:172.21.15.96:6817/3588713275] [v2:172.21.15.96:6818/3588713275,v1:172.21.15.96:6819/3588713275] exists,up 02046db8-30de-4e92-b25a-1f07d6190e91 2024-09-19T05:04:14.919 INFO:teuthology.orchestra.run.smithi043.stdout:osd.8 up in weight 1 up_from 57 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6816/725316393,v1:172.21.15.151:6817/725316393] [v2:172.21.15.151:6818/725316393,v1:172.21.15.151:6819/725316393] exists,up a5bb4218-51f3-4044-b43e-ebfc764f49ad 2024-09-19T05:04:14.919 INFO:teuthology.orchestra.run.smithi043.stdout:osd.9 up in weight 1 up_from 64 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.43:6826/4053438621,v1:172.21.15.43:6827/4053438621] [v2:172.21.15.43:6828/4053438621,v1:172.21.15.43:6829/4053438621] exists,up 6c19f65c-9750-4baa-9d63-7b8bd03e6955 2024-09-19T05:04:14.919 INFO:teuthology.orchestra.run.smithi043.stdout:osd.10 up in weight 1 up_from 69 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6824/2403210086,v1:172.21.15.96:6825/2403210086] [v2:172.21.15.96:6826/2403210086,v1:172.21.15.96:6827/2403210086] exists,up a7aa7da3-8eab-47fe-a351-298b6ff75f14 2024-09-19T05:04:14.919 INFO:teuthology.orchestra.run.smithi043.stdout:osd.11 up in weight 1 up_from 76 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6824/292202881,v1:172.21.15.151:6825/292202881] [v2:172.21.15.151:6826/292202881,v1:172.21.15.151:6827/292202881] exists,up 760b6afc-253d-44dd-88e9-dcc97f67c9a5 2024-09-19T05:04:14.919 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 2.6 [8,11] 2024-09-19T05:04:14.919 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 2.e [2,4] 2024-09-19T05:04:14.919 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 2.11 [7,11] 2024-09-19T05:04:14.920 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 2.1a [8,4] 2024-09-19T05:04:14.920 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 2.1d [2,1] 2024-09-19T05:04:14.920 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 3.e [5,9] 2024-09-19T05:04:14.920 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 3.17 [4,8] 2024-09-19T05:04:14.920 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 3.24 [4,9] 2024-09-19T05:04:14.920 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 3.3c [4,9] 2024-09-19T05:04:14.920 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:0/898247657 expires 2024-09-20T04:55:07.037117+0000 2024-09-19T05:04:14.920 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:6801/3241799377 expires 2024-09-20T04:55:07.037117+0000 2024-09-19T05:04:14.920 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:6800/3241799377 expires 2024-09-20T04:55:07.037117+0000 2024-09-19T05:04:14.921 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:0/3256350596 expires 2024-09-20T04:55:07.037117+0000 2024-09-19T05:04:14.921 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:6801/587463881 expires 2024-09-20T04:54:39.548065+0000 2024-09-19T05:04:14.921 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:6800/587463881 expires 2024-09-20T04:54:39.548065+0000 2024-09-19T05:04:14.921 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:0/4271788315 expires 2024-09-20T04:55:07.037117+0000 2024-09-19T05:04:14.921 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:0/6132665 expires 2024-09-20T04:54:39.548065+0000 2024-09-19T05:04:14.921 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:0/632163478 expires 2024-09-20T04:54:39.548065+0000 2024-09-19T05:04:14.921 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:0/330854185 expires 2024-09-20T04:54:39.548065+0000 2024-09-19T05:04:14.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.917+0000 7fe393800640 1 -- 172.21.15.43:0/71753246 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fe3740a3270 msgr2=0x7fe3740a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:14.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.917+0000 7fe393800640 1 --2- 172.21.15.43:0/71753246 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fe3740a3270 0x7fe3740a5730 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe3780314e0 tx=0x7fe378000bc0 comp rx=0 tx=0).stop 2024-09-19T05:04:14.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.917+0000 7fe393800640 1 -- 172.21.15.43:0/71753246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe38c074c50 msgr2=0x7fe38c19e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:14.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.917+0000 7fe393800640 1 --2- 172.21.15.43:0/71753246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe38c074c50 0x7fe38c19e500 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe38000e9c0 tx=0x7fe38000ee80 comp rx=0 tx=0).stop 2024-09-19T05:04:14.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.917+0000 7fe393800640 1 -- 172.21.15.43:0/71753246 shutdown_connections 2024-09-19T05:04:14.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.917+0000 7fe393800640 1 --2- 172.21.15.43:0/71753246 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fe3740a3270 0x7fe3740a5730 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:14.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.917+0000 7fe393800640 1 --2- 172.21.15.43:0/71753246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe38c10e1e0 0x7fe38c1a5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:14.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.917+0000 7fe393800640 1 --2- 172.21.15.43:0/71753246 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe38c076220 0x7fe38c19ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:14.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.917+0000 7fe393800640 1 --2- 172.21.15.43:0/71753246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe38c074c50 0x7fe38c19e500 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:14.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.917+0000 7fe393800640 1 -- 172.21.15.43:0/71753246 >> 172.21.15.43:0/71753246 conn(0x7fe38c0fd390 msgr2=0x7fe38c0ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:14.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.917+0000 7fe393800640 1 -- 172.21.15.43:0/71753246 shutdown_connections 2024-09-19T05:04:14.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:14.917+0000 7fe393800640 1 -- 172.21.15.43:0/71753246 wait complete. 2024-09-19T05:04:15.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:14 smithi151 ceph-mon[35425]: pgmap v348: 97 pgs: 97 active+clean; 646 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-19T05:04:15.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:14 smithi096 ceph-mon[35085]: pgmap v348: 97 pgs: 97 active+clean; 646 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-19T05:04:15.267 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:14 smithi043 ceph-mon[30800]: pgmap v348: 97 pgs: 97 active+clean; 646 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-19T05:04:15.773 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-19T05:04:15.785 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:04:16.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:15 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/71753246' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T05:04:16.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:15 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/71753246' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T05:04:16.161 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:16.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:15 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/71753246' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T05:04:16.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.732+0000 7fa079c00640 1 -- 172.21.15.43:0/836492072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa074074f40 msgr2=0x7fa07410de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:16.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.732+0000 7fa079c00640 1 --2- 172.21.15.43:0/836492072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa074074f40 0x7fa07410de80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa05c009900 tx=0x7fa05c02a480 comp rx=0 tx=0).stop 2024-09-19T05:04:16.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.732+0000 7fa079c00640 1 -- 172.21.15.43:0/836492072 shutdown_connections 2024-09-19T05:04:16.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.732+0000 7fa079c00640 1 --2- 172.21.15.43:0/836492072 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa07410e3c0 0x7fa074110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:16.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.732+0000 7fa079c00640 1 --2- 172.21.15.43:0/836492072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa074074f40 0x7fa07410de80 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:16.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.732+0000 7fa079c00640 1 --2- 172.21.15.43:0/836492072 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa074074620 0x7fa074074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:16.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.732+0000 7fa079c00640 1 -- 172.21.15.43:0/836492072 >> 172.21.15.43:0/836492072 conn(0x7fa0740fd8c0 msgr2=0x7fa0740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:16.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.732+0000 7fa079c00640 1 -- 172.21.15.43:0/836492072 shutdown_connections 2024-09-19T05:04:16.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.732+0000 7fa079c00640 1 -- 172.21.15.43:0/836492072 wait complete. 2024-09-19T05:04:16.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.734+0000 7fa079c00640 1 Processor -- start 2024-09-19T05:04:16.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.734+0000 7fa079c00640 1 -- start start 2024-09-19T05:04:16.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.734+0000 7fa079c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa074074620 0x7fa07419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:16.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.734+0000 7fa079c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa074074f40 0x7fa07419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:16.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.734+0000 7fa079c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa07410e3c0 0x7fa07419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:16.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.734+0000 7fa079c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa074114190 con 0x7fa074074f40 2024-09-19T05:04:16.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.734+0000 7fa079c00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa074114010 con 0x7fa07410e3c0 2024-09-19T05:04:16.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.734+0000 7fa079c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa074114310 con 0x7fa074074620 2024-09-19T05:04:16.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.735+0000 7fa06be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa074074f40 0x7fa07419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:16.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.735+0000 7fa06be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa074074f40 0x7fa07419abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46412/0 (socket says 172.21.15.43:46412) 2024-09-19T05:04:16.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.735+0000 7fa06be00640 1 -- 172.21.15.43:0/2289089351 learned_addr learned my addr 172.21.15.43:0/2289089351 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:16.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.735+0000 7fa072a00640 1 --2- 172.21.15.43:0/2289089351 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa074074620 0x7fa07419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:16.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.735+0000 7fa073400640 1 --2- 172.21.15.43:0/2289089351 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa07410e3c0 0x7fa07419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:16.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.735+0000 7fa06be00640 1 -- 172.21.15.43:0/2289089351 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa074074620 msgr2=0x7fa07419a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:16.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.735+0000 7fa06be00640 1 --2- 172.21.15.43:0/2289089351 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa074074620 0x7fa07419a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:16.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.735+0000 7fa06be00640 1 -- 172.21.15.43:0/2289089351 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa07410e3c0 msgr2=0x7fa07419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:16.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.735+0000 7fa06be00640 1 --2- 172.21.15.43:0/2289089351 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa07410e3c0 0x7fa07419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:16.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.735+0000 7fa06be00640 1 -- 172.21.15.43:0/2289089351 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa07419f640 con 0x7fa074074f40 2024-09-19T05:04:16.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.735+0000 7fa073400640 1 --2- 172.21.15.43:0/2289089351 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa07410e3c0 0x7fa07419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:04:16.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.735+0000 7fa06be00640 1 --2- 172.21.15.43:0/2289089351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa074074f40 0x7fa07419abd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa05c004770 tx=0x7fa05c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:16.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.736+0000 7fa06b400640 1 -- 172.21.15.43:0/2289089351 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c03f070 con 0x7fa074074f40 2024-09-19T05:04:16.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.736+0000 7fa06b400640 1 -- 172.21.15.43:0/2289089351 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa05c031ca0 con 0x7fa074074f40 2024-09-19T05:04:16.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.736+0000 7fa079c00640 1 -- 172.21.15.43:0/2289089351 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa07419f8d0 con 0x7fa074074f40 2024-09-19T05:04:16.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.736+0000 7fa06b400640 1 -- 172.21.15.43:0/2289089351 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c035030 con 0x7fa074074f40 2024-09-19T05:04:16.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.736+0000 7fa079c00640 1 -- 172.21.15.43:0/2289089351 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0741a7210 con 0x7fa074074f40 2024-09-19T05:04:16.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.737+0000 7fa079c00640 1 -- 172.21.15.43:0/2289089351 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa074075cd0 con 0x7fa074074f40 2024-09-19T05:04:16.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.739+0000 7fa06b400640 1 -- 172.21.15.43:0/2289089351 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fa05c042070 con 0x7fa074074f40 2024-09-19T05:04:16.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.740+0000 7fa06b400640 1 --2- 172.21.15.43:0/2289089351 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa0580a3240 0x7fa0580a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:16.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.740+0000 7fa06b400640 1 -- 172.21.15.43:0/2289089351 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fa05c0efd30 con 0x7fa074074f40 2024-09-19T05:04:16.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.740+0000 7fa072a00640 1 --2- 172.21.15.43:0/2289089351 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa0580a3240 0x7fa0580a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:16.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.741+0000 7fa072a00640 1 --2- 172.21.15.43:0/2289089351 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa0580a3240 0x7fa0580a5700 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa06400bbe0 tx=0x7fa06400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:16.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.745+0000 7fa06b400640 1 -- 172.21.15.43:0/2289089351 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa05c0bca10 con 0x7fa074074f40 2024-09-19T05:04:16.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.970+0000 7fa079c00640 1 -- 172.21.15.43:0/2289089351 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa074109540 con 0x7fa074074f40 2024-09-19T05:04:16.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.970+0000 7fa06b400640 1 -- 172.21.15.43:0/2289089351 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11965 (secure 0 0 0) 0x7fa05c0bc310 con 0x7fa074074f40 2024-09-19T05:04:16.973 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:04:16.973 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":28,"btime":"2024-09-19T05:02:40:505009+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/773801169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":773801169},{"type":"v1","addr":"172.21.15.43:6835","nonce":773801169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1799137902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1799137902},{"type":"v1","addr":"172.21.15.43:6837","nonce":1799137902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:02:25.463041+0000","modified":"2024-09-19T05:02:39.530912+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24518,"mds_2":24511,"mds_3":24506,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14541":{"gid":14541,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/794881135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":794881135},{"type":"v1","addr":"172.21.15.43:6841","nonce":794881135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/2836453158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2836453158},{"type":"v1","addr":"172.21.15.151:6833","nonce":2836453158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/178785812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":178785812},{"type":"v1","addr":"172.21.15.96:6833","nonce":178785812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1671043820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1671043820},{"type":"v1","addr":"172.21.15.96:6835","nonce":1671043820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2972514328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2972514328},{"type":"v1","addr":"172.21.15.151:6835","nonce":2972514328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.151:6837/3960378010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3960378010},{"type":"v1","addr":"172.21.15.151:6837","nonce":3960378010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6837/3978388286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3978388286},{"type":"v1","addr":"172.21.15.96:6837","nonce":3978388286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.151:6839/3794104313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3794104313},{"type":"v1","addr":"172.21.15.151:6839","nonce":3794104313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.43:6839/3404555030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3404555030},{"type":"v1","addr":"172.21.15.43:6839","nonce":3404555030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/605527621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":605527621},{"type":"v1","addr":"172.21.15.96:6839","nonce":605527621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24493,24506,24511,24518,24542]},"id":1}]} 2024-09-19T05:04:16.973 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 28 2024-09-19T05:04:16.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.973+0000 7fa079c00640 1 -- 172.21.15.43:0/2289089351 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa0580a3240 msgr2=0x7fa0580a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:16.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.973+0000 7fa079c00640 1 --2- 172.21.15.43:0/2289089351 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa0580a3240 0x7fa0580a5700 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa06400bbe0 tx=0x7fa06400b390 comp rx=0 tx=0).stop 2024-09-19T05:04:16.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.973+0000 7fa079c00640 1 -- 172.21.15.43:0/2289089351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa074074f40 msgr2=0x7fa07419abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:16.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.973+0000 7fa079c00640 1 --2- 172.21.15.43:0/2289089351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa074074f40 0x7fa07419abd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa05c004770 tx=0x7fa05c003c10 comp rx=0 tx=0).stop 2024-09-19T05:04:16.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.973+0000 7fa079c00640 1 -- 172.21.15.43:0/2289089351 shutdown_connections 2024-09-19T05:04:16.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.973+0000 7fa079c00640 1 --2- 172.21.15.43:0/2289089351 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa0580a3240 0x7fa0580a5700 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:16.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.973+0000 7fa079c00640 1 --2- 172.21.15.43:0/2289089351 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa07410e3c0 0x7fa07419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:16.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.973+0000 7fa079c00640 1 --2- 172.21.15.43:0/2289089351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa074074f40 0x7fa07419abd0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:16.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.973+0000 7fa079c00640 1 --2- 172.21.15.43:0/2289089351 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa074074620 0x7fa07419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:16.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.973+0000 7fa079c00640 1 -- 172.21.15.43:0/2289089351 >> 172.21.15.43:0/2289089351 conn(0x7fa0740fd8c0 msgr2=0x7fa0740fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:16.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.973+0000 7fa079c00640 1 -- 172.21.15.43:0/2289089351 shutdown_connections 2024-09-19T05:04:16.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:16.973+0000 7fa079c00640 1 -- 172.21.15.43:0/2289089351 wait complete. 2024-09-19T05:04:17.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:16 smithi151 ceph-mon[35425]: pgmap v349: 97 pgs: 97 active+clean; 646 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-19T05:04:17.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:16 smithi096 ceph-mon[35085]: pgmap v349: 97 pgs: 97 active+clean; 646 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-19T05:04:17.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:16 smithi043 ceph-mon[30800]: pgmap v349: 97 pgs: 97 active+clean; 646 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-19T05:04:17.709 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:04:18.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:17 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/2289089351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:04:18.088 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:18.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:17 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/2289089351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:04:18.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:17 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/2289089351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:04:18.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.671+0000 7fe7fa400640 1 -- 172.21.15.43:0/3406849487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7f4103e30 msgr2=0x7fe7f4104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:18.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.671+0000 7fe7fa400640 1 --2- 172.21.15.43:0/3406849487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7f4103e30 0x7fe7f4104210 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4009900 tx=0x7fe7e402a410 comp rx=0 tx=0).stop 2024-09-19T05:04:18.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.671+0000 7fe7fa400640 1 -- 172.21.15.43:0/3406849487 shutdown_connections 2024-09-19T05:04:18.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.671+0000 7fe7fa400640 1 --2- 172.21.15.43:0/3406849487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7f4108f60 0x7fe7f4110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:18.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.671+0000 7fe7fa400640 1 --2- 172.21.15.43:0/3406849487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe7f41047e0 0x7fe7f4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:18.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.671+0000 7fe7fa400640 1 --2- 172.21.15.43:0/3406849487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7f4103e30 0x7fe7f4104210 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:18.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.671+0000 7fe7fa400640 1 -- 172.21.15.43:0/3406849487 >> 172.21.15.43:0/3406849487 conn(0x7fe7f40ff8d0 msgr2=0x7fe7f4101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:18.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.671+0000 7fe7fa400640 1 -- 172.21.15.43:0/3406849487 shutdown_connections 2024-09-19T05:04:18.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.671+0000 7fe7fa400640 1 -- 172.21.15.43:0/3406849487 wait complete. 2024-09-19T05:04:18.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.672+0000 7fe7fa400640 1 Processor -- start 2024-09-19T05:04:18.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.672+0000 7fe7fa400640 1 -- start start 2024-09-19T05:04:18.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.672+0000 7fe7fa400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7f4103e30 0x7fe7f419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:18.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.672+0000 7fe7fa400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7f41047e0 0x7fe7f419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:18.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.672+0000 7fe7fa400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe7f4108f60 0x7fe7f419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:18.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.672+0000 7fe7fa400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f4113fd0 con 0x7fe7f41047e0 2024-09-19T05:04:18.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.672+0000 7fe7fa400640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f4113e50 con 0x7fe7f4103e30 2024-09-19T05:04:18.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.672+0000 7fe7fa400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f4114150 con 0x7fe7f4108f60 2024-09-19T05:04:18.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.673+0000 7fe7f9000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7f4103e30 0x7fe7f419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:18.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.673+0000 7fe7ebe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7f41047e0 0x7fe7f419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:18.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.673+0000 7fe7f9000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7f4103e30 0x7fe7f419a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:58114/0 (socket says 172.21.15.43:58114) 2024-09-19T05:04:18.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.673+0000 7fe7f9000640 1 -- 172.21.15.43:0/488240723 learned_addr learned my addr 172.21.15.43:0/488240723 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:18.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.673+0000 7fe7f9a00640 1 --2- 172.21.15.43:0/488240723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe7f4108f60 0x7fe7f419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:18.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.673+0000 7fe7ebe00640 1 -- 172.21.15.43:0/488240723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe7f4108f60 msgr2=0x7fe7f419ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:18.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.673+0000 7fe7ebe00640 1 --2- 172.21.15.43:0/488240723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe7f4108f60 0x7fe7f419ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:18.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.673+0000 7fe7ebe00640 1 -- 172.21.15.43:0/488240723 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7f4103e30 msgr2=0x7fe7f419a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:18.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.673+0000 7fe7ebe00640 1 --2- 172.21.15.43:0/488240723 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7f4103e30 0x7fe7f419a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:18.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.673+0000 7fe7ebe00640 1 -- 172.21.15.43:0/488240723 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f419f460 con 0x7fe7f41047e0 2024-09-19T05:04:18.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.674+0000 7fe7ebe00640 1 --2- 172.21.15.43:0/488240723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7f41047e0 0x7fe7f419a9f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc00d6d0 tx=0x7fe7dc00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:18.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.674+0000 7fe7f2000640 1 -- 172.21.15.43:0/488240723 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc017070 con 0x7fe7f41047e0 2024-09-19T05:04:18.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.674+0000 7fe7f2000640 1 -- 172.21.15.43:0/488240723 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7dc009820 con 0x7fe7f41047e0 2024-09-19T05:04:18.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.674+0000 7fe7fa400640 1 -- 172.21.15.43:0/488240723 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7f419f750 con 0x7fe7f41047e0 2024-09-19T05:04:18.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.674+0000 7fe7f2000640 1 -- 172.21.15.43:0/488240723 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc004d00 con 0x7fe7f41047e0 2024-09-19T05:04:18.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.674+0000 7fe7fa400640 1 -- 172.21.15.43:0/488240723 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7f41a7030 con 0x7fe7f41047e0 2024-09-19T05:04:18.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.675+0000 7fe7fa400640 1 -- 172.21.15.43:0/488240723 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7f4104210 con 0x7fe7f41047e0 2024-09-19T05:04:18.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.677+0000 7fe7f2000640 1 -- 172.21.15.43:0/488240723 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fe7dc0099d0 con 0x7fe7f41047e0 2024-09-19T05:04:18.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.679+0000 7fe7f2000640 1 --2- 172.21.15.43:0/488240723 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fe7d80a32c0 0x7fe7d80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:18.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.679+0000 7fe7f2000640 1 -- 172.21.15.43:0/488240723 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe7dc0cf3d0 con 0x7fe7f41047e0 2024-09-19T05:04:18.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.679+0000 7fe7f9000640 1 --2- 172.21.15.43:0/488240723 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fe7d80a32c0 0x7fe7d80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:18.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.680+0000 7fe7f9000640 1 --2- 172.21.15.43:0/488240723 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fe7d80a32c0 0x7fe7d80a5780 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe7e40096c0 tx=0x7fe7e40046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:18.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.683+0000 7fe7f2000640 1 -- 172.21.15.43:0/488240723 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe7dc09ba60 con 0x7fe7f41047e0 2024-09-19T05:04:18.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.908+0000 7fe7fa400640 1 -- 172.21.15.43:0/488240723 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7f402d7c0 con 0x7fe7f41047e0 2024-09-19T05:04:18.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.909+0000 7fe7f2000640 1 -- 172.21.15.43:0/488240723 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11965 (secure 0 0 0) 0x7fe7dc09fd60 con 0x7fe7f41047e0 2024-09-19T05:04:18.912 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:04:18.912 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":28,"btime":"2024-09-19T05:02:40:505009+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/773801169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":773801169},{"type":"v1","addr":"172.21.15.43:6835","nonce":773801169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1799137902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1799137902},{"type":"v1","addr":"172.21.15.43:6837","nonce":1799137902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:02:25.463041+0000","modified":"2024-09-19T05:02:39.530912+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24518,"mds_2":24511,"mds_3":24506,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14541":{"gid":14541,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/794881135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":794881135},{"type":"v1","addr":"172.21.15.43:6841","nonce":794881135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/2836453158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2836453158},{"type":"v1","addr":"172.21.15.151:6833","nonce":2836453158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/178785812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":178785812},{"type":"v1","addr":"172.21.15.96:6833","nonce":178785812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1671043820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1671043820},{"type":"v1","addr":"172.21.15.96:6835","nonce":1671043820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2972514328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2972514328},{"type":"v1","addr":"172.21.15.151:6835","nonce":2972514328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.151:6837/3960378010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3960378010},{"type":"v1","addr":"172.21.15.151:6837","nonce":3960378010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6837/3978388286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3978388286},{"type":"v1","addr":"172.21.15.96:6837","nonce":3978388286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.151:6839/3794104313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3794104313},{"type":"v1","addr":"172.21.15.151:6839","nonce":3794104313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.43:6839/3404555030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3404555030},{"type":"v1","addr":"172.21.15.43:6839","nonce":3404555030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/605527621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":605527621},{"type":"v1","addr":"172.21.15.96:6839","nonce":605527621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24493,24506,24511,24518,24542]},"id":1}]} 2024-09-19T05:04:18.912 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 28 2024-09-19T05:04:18.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.911+0000 7fe7fa400640 1 -- 172.21.15.43:0/488240723 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fe7d80a32c0 msgr2=0x7fe7d80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:18.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.911+0000 7fe7fa400640 1 --2- 172.21.15.43:0/488240723 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fe7d80a32c0 0x7fe7d80a5780 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe7e40096c0 tx=0x7fe7e40046c0 comp rx=0 tx=0).stop 2024-09-19T05:04:18.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.911+0000 7fe7fa400640 1 -- 172.21.15.43:0/488240723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7f41047e0 msgr2=0x7fe7f419a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:18.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.911+0000 7fe7fa400640 1 --2- 172.21.15.43:0/488240723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7f41047e0 0x7fe7f419a9f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc00d6d0 tx=0x7fe7dc00db90 comp rx=0 tx=0).stop 2024-09-19T05:04:18.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.911+0000 7fe7fa400640 1 -- 172.21.15.43:0/488240723 shutdown_connections 2024-09-19T05:04:18.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.911+0000 7fe7fa400640 1 --2- 172.21.15.43:0/488240723 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fe7d80a32c0 0x7fe7d80a5780 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:18.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.911+0000 7fe7fa400640 1 --2- 172.21.15.43:0/488240723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe7f4108f60 0x7fe7f419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:18.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.911+0000 7fe7fa400640 1 --2- 172.21.15.43:0/488240723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7f41047e0 0x7fe7f419a9f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:18.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.911+0000 7fe7fa400640 1 --2- 172.21.15.43:0/488240723 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7f4103e30 0x7fe7f419a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:18.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.912+0000 7fe7fa400640 1 -- 172.21.15.43:0/488240723 >> 172.21.15.43:0/488240723 conn(0x7fe7f40ff8d0 msgr2=0x7fe7f4100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:18.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.912+0000 7fe7fa400640 1 -- 172.21.15.43:0/488240723 shutdown_connections 2024-09-19T05:04:18.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:18.912+0000 7fe7fa400640 1 -- 172.21.15.43:0/488240723 wait complete. 2024-09-19T05:04:19.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:18 smithi096 ceph-mon[35085]: pgmap v350: 97 pgs: 97 active+clean; 646 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-19T05:04:19.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:18 smithi043 ceph-mon[30800]: pgmap v350: 97 pgs: 97 active+clean; 646 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-19T05:04:19.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:18 smithi151 ceph-mon[35425]: pgmap v350: 97 pgs: 97 active+clean; 646 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-19T05:04:19.738 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T05:04:20.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:19 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/488240723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:04:20.116 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:20.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:19 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/488240723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:04:20.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:19 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/488240723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:04:20.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.673+0000 7fa4ebc00640 1 -- 172.21.15.43:0/649682793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4e4076720 msgr2=0x7fa4e4076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:20.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.673+0000 7fa4ebc00640 1 --2- 172.21.15.43:0/649682793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4e4076720 0x7fa4e4076b00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4009900 tx=0x7fa4d402a3f0 comp rx=0 tx=0).stop 2024-09-19T05:04:20.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.673+0000 7fa4ebc00640 1 -- 172.21.15.43:0/649682793 shutdown_connections 2024-09-19T05:04:20.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.673+0000 7fa4ebc00640 1 --2- 172.21.15.43:0/649682793 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4e4112a50 0x7fa4e4114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:20.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.673+0000 7fa4ebc00640 1 --2- 172.21.15.43:0/649682793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4e4077040 0x7fa4e4112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:20.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.673+0000 7fa4ebc00640 1 --2- 172.21.15.43:0/649682793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4e4076720 0x7fa4e4076b00 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:20.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.673+0000 7fa4ebc00640 1 -- 172.21.15.43:0/649682793 >> 172.21.15.43:0/649682793 conn(0x7fa4e40ffa50 msgr2=0x7fa4e4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:20.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.673+0000 7fa4ebc00640 1 -- 172.21.15.43:0/649682793 shutdown_connections 2024-09-19T05:04:20.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.673+0000 7fa4ebc00640 1 -- 172.21.15.43:0/649682793 wait complete. 2024-09-19T05:04:20.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.674+0000 7fa4ebc00640 1 Processor -- start 2024-09-19T05:04:20.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.674+0000 7fa4ebc00640 1 -- start start 2024-09-19T05:04:20.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.674+0000 7fa4ebc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4e4076720 0x7fa4e41a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:20.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.674+0000 7fa4ebc00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4e4077040 0x7fa4e41a4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:20.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.674+0000 7fa4ebc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4e4112a50 0x7fa4e419e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:20.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.674+0000 7fa4ebc00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e4118610 con 0x7fa4e4076720 2024-09-19T05:04:20.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.674+0000 7fa4ebc00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e4118490 con 0x7fa4e4077040 2024-09-19T05:04:20.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.674+0000 7fa4ebc00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e4118790 con 0x7fa4e4112a50 2024-09-19T05:04:20.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.675+0000 7fa4e9200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4e4076720 0x7fa4e41a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:20.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.675+0000 7fa4e9200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4e4076720 0x7fa4e41a47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46462/0 (socket says 172.21.15.43:46462) 2024-09-19T05:04:20.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.675+0000 7fa4e9200640 1 -- 172.21.15.43:0/3769473157 learned_addr learned my addr 172.21.15.43:0/3769473157 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:20.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.675+0000 7fa4dbe00640 1 --2- 172.21.15.43:0/3769473157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4e4077040 0x7fa4e41a4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:20.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.675+0000 7fa4e9c00640 1 --2- 172.21.15.43:0/3769473157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4e4112a50 0x7fa4e419e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:20.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.675+0000 7fa4e9200640 1 -- 172.21.15.43:0/3769473157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4e4112a50 msgr2=0x7fa4e419e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:20.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.675+0000 7fa4e9200640 1 --2- 172.21.15.43:0/3769473157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4e4112a50 0x7fa4e419e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:20.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.675+0000 7fa4e9200640 1 -- 172.21.15.43:0/3769473157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4e4077040 msgr2=0x7fa4e41a4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:20.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.675+0000 7fa4e9200640 1 --2- 172.21.15.43:0/3769473157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4e4077040 0x7fa4e41a4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:20.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.675+0000 7fa4e9200640 1 -- 172.21.15.43:0/3769473157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4e419f090 con 0x7fa4e4076720 2024-09-19T05:04:20.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.675+0000 7fa4e9200640 1 --2- 172.21.15.43:0/3769473157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4e4076720 0x7fa4e41a47a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4004770 tx=0x7fa4d4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:20.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.676+0000 7fa4e2000640 1 -- 172.21.15.43:0/3769473157 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d403f070 con 0x7fa4e4076720 2024-09-19T05:04:20.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.676+0000 7fa4e2000640 1 -- 172.21.15.43:0/3769473157 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4d4031ca0 con 0x7fa4e4076720 2024-09-19T05:04:20.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.676+0000 7fa4e2000640 1 -- 172.21.15.43:0/3769473157 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4035030 con 0x7fa4e4076720 2024-09-19T05:04:20.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.676+0000 7fa4ebc00640 1 -- 172.21.15.43:0/3769473157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4e419f320 con 0x7fa4e4076720 2024-09-19T05:04:20.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.676+0000 7fa4ebc00640 1 -- 172.21.15.43:0/3769473157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4e41ab670 con 0x7fa4e4076720 2024-09-19T05:04:20.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.677+0000 7fa4ebc00640 1 -- 172.21.15.43:0/3769473157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4e4076b00 con 0x7fa4e4076720 2024-09-19T05:04:20.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.679+0000 7fa4e2000640 1 -- 172.21.15.43:0/3769473157 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fa4d4042070 con 0x7fa4e4076720 2024-09-19T05:04:20.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.680+0000 7fa4e2000640 1 --2- 172.21.15.43:0/3769473157 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa4c80a32c0 0x7fa4c80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:20.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.680+0000 7fa4e2000640 1 -- 172.21.15.43:0/3769473157 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fa4d40effe0 con 0x7fa4e4076720 2024-09-19T05:04:20.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.680+0000 7fa4dbe00640 1 --2- 172.21.15.43:0/3769473157 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa4c80a32c0 0x7fa4c80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:20.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.681+0000 7fa4dbe00640 1 --2- 172.21.15.43:0/3769473157 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa4c80a32c0 0x7fa4c80a5780 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc0095f0 tx=0x7fa4cc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:20.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.682+0000 7fa4e2000640 1 -- 172.21.15.43:0/3769473157 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa4d40bccc0 con 0x7fa4e4076720 2024-09-19T05:04:20.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.873+0000 7fa4ebc00640 1 -- 172.21.15.43:0/3769473157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa4e419fe90 con 0x7fa4e4076720 2024-09-19T05:04:20.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.875+0000 7fa4e2000640 1 -- 172.21.15.43:0/3769473157 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22661 (secure 0 0 0) 0x7fa4d40bc5c0 con 0x7fa4e4076720 2024-09-19T05:04:20.878 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:04:20.878 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":85,"fsid":"1456a810-7643-11ef-bceb-c7b262605968","created":"2024-09-19T04:54:14.706644+0000","modified":"2024-09-19T05:03:08.697712+0000","last_up_change":"2024-09-19T05:01:27.348740+0000","last_in_change":"2024-09-19T05:01:03.396477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:57:27.642705+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:02:20.685123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:02:23.393596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"81a317a8-7ec8-45c3-bc10-7da023283635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6803","nonce":825351486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6805","nonce":825351486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6809","nonce":825351486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6807","nonce":825351486}]},"public_addr":"172.21.15.43:6803/825351486","cluster_addr":"172.21.15.43:6805/825351486","heartbeat_back_addr":"172.21.15.43:6809/825351486","heartbeat_front_addr":"172.21.15.43:6807/825351486","state":["exists","up"]},{"osd":1,"uuid":"e96fcfac-4fff-4a36-b38b-aebf57842740","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6801","nonce":2140095130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6803","nonce":2140095130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6807","nonce":2140095130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6805","nonce":2140095130}]},"public_addr":"172.21.15.96:6801/2140095130","cluster_addr":"172.21.15.96:6803/2140095130","heartbeat_back_addr":"172.21.15.96:6807/2140095130","heartbeat_front_addr":"172.21.15.96:6805/2140095130","state":["exists","up"]},{"osd":2,"uuid":"b659869e-114a-4f56-8aac-a2ed602b4058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6801","nonce":2047147996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6803","nonce":2047147996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6807","nonce":2047147996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6805","nonce":2047147996}]},"public_addr":"172.21.15.151:6801/2047147996","cluster_addr":"172.21.15.151:6803/2047147996","heartbeat_back_addr":"172.21.15.151:6807/2047147996","heartbeat_front_addr":"172.21.15.151:6805/2047147996","state":["exists","up"]},{"osd":3,"uuid":"7be64b2c-e45c-493d-a610-4ce41196061f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6811","nonce":1734673762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6813","nonce":1734673762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6817","nonce":1734673762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6815","nonce":1734673762}]},"public_addr":"172.21.15.43:6811/1734673762","cluster_addr":"172.21.15.43:6813/1734673762","heartbeat_back_addr":"172.21.15.43:6817/1734673762","heartbeat_front_addr":"172.21.15.43:6815/1734673762","state":["exists","up"]},{"osd":4,"uuid":"196d5af1-a6e5-496c-8933-4e58892f4360","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6809","nonce":3184785362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6811","nonce":3184785362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6815","nonce":3184785362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6813","nonce":3184785362}]},"public_addr":"172.21.15.96:6809/3184785362","cluster_addr":"172.21.15.96:6811/3184785362","heartbeat_back_addr":"172.21.15.96:6815/3184785362","heartbeat_front_addr":"172.21.15.96:6813/3184785362","state":["exists","up"]},{"osd":5,"uuid":"5492d474-8509-4865-bc2e-d08e31558a75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6809","nonce":2878451924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6811","nonce":2878451924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6815","nonce":2878451924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6813","nonce":2878451924}]},"public_addr":"172.21.15.151:6809/2878451924","cluster_addr":"172.21.15.151:6811/2878451924","heartbeat_back_addr":"172.21.15.151:6815/2878451924","heartbeat_front_addr":"172.21.15.151:6813/2878451924","state":["exists","up"]},{"osd":6,"uuid":"fb8457ed-7bb9-4435-9d15-c1cbfbddfcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6819","nonce":2008630290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6821","nonce":2008630290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6825","nonce":2008630290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6823","nonce":2008630290}]},"public_addr":"172.21.15.43:6819/2008630290","cluster_addr":"172.21.15.43:6821/2008630290","heartbeat_back_addr":"172.21.15.43:6825/2008630290","heartbeat_front_addr":"172.21.15.43:6823/2008630290","state":["exists","up"]},{"osd":7,"uuid":"02046db8-30de-4e92-b25a-1f07d6190e91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6817","nonce":3588713275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6819","nonce":3588713275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6823","nonce":3588713275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6821","nonce":3588713275}]},"public_addr":"172.21.15.96:6817/3588713275","cluster_addr":"172.21.15.96:6819/3588713275","heartbeat_back_addr":"172.21.15.96:6823/3588713275","heartbeat_front_addr":"172.21.15.96:6821/3588713275","state":["exists","up"]},{"osd":8,"uuid":"a5bb4218-51f3-4044-b43e-ebfc764f49ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6817","nonce":725316393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6819","nonce":725316393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6823","nonce":725316393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6821","nonce":725316393}]},"public_addr":"172.21.15.151:6817/725316393","cluster_addr":"172.21.15.151:6819/725316393","heartbeat_back_addr":"172.21.15.151:6823/725316393","heartbeat_front_addr":"172.21.15.151:6821/725316393","state":["exists","up"]},{"osd":9,"uuid":"6c19f65c-9750-4baa-9d63-7b8bd03e6955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6827","nonce":4053438621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6829","nonce":4053438621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6833","nonce":4053438621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6831","nonce":4053438621}]},"public_addr":"172.21.15.43:6827/4053438621","cluster_addr":"172.21.15.43:6829/4053438621","heartbeat_back_addr":"172.21.15.43:6833/4053438621","heartbeat_front_addr":"172.21.15.43:6831/4053438621","state":["exists","up"]},{"osd":10,"uuid":"a7aa7da3-8eab-47fe-a351-298b6ff75f14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6825","nonce":2403210086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6827","nonce":2403210086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6831","nonce":2403210086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6829","nonce":2403210086}]},"public_addr":"172.21.15.96:6825/2403210086","cluster_addr":"172.21.15.96:6827/2403210086","heartbeat_back_addr":"172.21.15.96:6831/2403210086","heartbeat_front_addr":"172.21.15.96:6829/2403210086","state":["exists","up"]},{"osd":11,"uuid":"760b6afc-253d-44dd-88e9-dcc97f67c9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6825","nonce":292202881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6827","nonce":292202881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6831","nonce":292202881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6829","nonce":292202881}]},"public_addr":"172.21.15.151:6825/292202881","cluster_addr":"172.21.15.151:6827/292202881","heartbeat_back_addr":"172.21.15.151:6831/292202881","heartbeat_front_addr":"172.21.15.151:6829/292202881","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:56:34.716991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:56:59.321174+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:57:24.281696+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:57:53.443880+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:58:19.205873+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:58:44.504873+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:13.650731+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:39.137876+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:04.579893+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:34.337312+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:59.434658+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:01:24.930529+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/898247657":"2024-09-20T04:55:07.037117+0000","172.21.15.43:6801/3241799377":"2024-09-20T04:55:07.037117+0000","172.21.15.43:6800/3241799377":"2024-09-20T04:55:07.037117+0000","172.21.15.43:0/3256350596":"2024-09-20T04:55:07.037117+0000","172.21.15.43:6801/587463881":"2024-09-20T04:54:39.548065+0000","172.21.15.43:6800/587463881":"2024-09-20T04:54:39.548065+0000","172.21.15.43:0/4271788315":"2024-09-20T04:55:07.037117+0000","172.21.15.43:0/6132665":"2024-09-20T04:54:39.548065+0000","172.21.15.43:0/632163478":"2024-09-20T04:54:39.548065+0000","172.21.15.43:0/330854185":"2024-09-20T04:54:39.548065+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:04:20.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.878+0000 7fa4ebc00640 1 -- 172.21.15.43:0/3769473157 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa4c80a32c0 msgr2=0x7fa4c80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:20.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.878+0000 7fa4ebc00640 1 --2- 172.21.15.43:0/3769473157 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa4c80a32c0 0x7fa4c80a5780 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc0095f0 tx=0x7fa4cc005f90 comp rx=0 tx=0).stop 2024-09-19T05:04:20.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.878+0000 7fa4ebc00640 1 -- 172.21.15.43:0/3769473157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4e4076720 msgr2=0x7fa4e41a47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:20.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.878+0000 7fa4ebc00640 1 --2- 172.21.15.43:0/3769473157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4e4076720 0x7fa4e41a47a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4004770 tx=0x7fa4d4003c10 comp rx=0 tx=0).stop 2024-09-19T05:04:20.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.878+0000 7fa4ebc00640 1 -- 172.21.15.43:0/3769473157 shutdown_connections 2024-09-19T05:04:20.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.878+0000 7fa4ebc00640 1 --2- 172.21.15.43:0/3769473157 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fa4c80a32c0 0x7fa4c80a5780 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:20.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.878+0000 7fa4ebc00640 1 --2- 172.21.15.43:0/3769473157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4e4112a50 0x7fa4e419e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:20.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.878+0000 7fa4ebc00640 1 --2- 172.21.15.43:0/3769473157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4e4077040 0x7fa4e41a4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:20.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.878+0000 7fa4ebc00640 1 --2- 172.21.15.43:0/3769473157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4e4076720 0x7fa4e41a47a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:20.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.878+0000 7fa4ebc00640 1 -- 172.21.15.43:0/3769473157 >> 172.21.15.43:0/3769473157 conn(0x7fa4e40ffa50 msgr2=0x7fa4e4113650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:20.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.878+0000 7fa4ebc00640 1 -- 172.21.15.43:0/3769473157 shutdown_connections 2024-09-19T05:04:20.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:20.878+0000 7fa4ebc00640 1 -- 172.21.15.43:0/3769473157 wait complete. 2024-09-19T05:04:21.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:20 smithi096 ceph-mon[35085]: pgmap v351: 97 pgs: 97 active+clean; 646 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-19T05:04:21.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:20 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/3769473157' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:04:21.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:20 smithi043 ceph-mon[30800]: pgmap v351: 97 pgs: 97 active+clean; 646 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-19T05:04:21.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:20 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/3769473157' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:04:21.306 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:20 smithi151 ceph-mon[35425]: pgmap v351: 97 pgs: 97 active+clean; 646 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-19T05:04:21.306 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:20 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/3769473157' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:04:21.652 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T05:02:25.463041+0000', 'modified': '2024-09-19T05:02:39.530912+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24542, 'mds_1': 24518, 'mds_2': 24511, 'mds_3': 24506, 'mds_4': 24493}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14541': {'gid': 14541, 'name': 'a', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.43:6841/794881135', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6840', 'nonce': 794881135}, {'type': 'v1', 'addr': '172.21.15.43:6841', 'nonce': 794881135}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.151:6833/2836453158', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6832', 'nonce': 2836453158}, {'type': 'v1', 'addr': '172.21.15.151:6833', 'nonce': 2836453158}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.96:6833/178785812', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6832', 'nonce': 178785812}, {'type': 'v1', 'addr': '172.21.15.96:6833', 'nonce': 178785812}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.96:6835/1671043820', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6834', 'nonce': 1671043820}, {'type': 'v1', 'addr': '172.21.15.96:6835', 'nonce': 1671043820}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.151:6835/2972514328', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6834', 'nonce': 2972514328}, {'type': 'v1', 'addr': '172.21.15.151:6835', 'nonce': 2972514328}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'c', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.151:6837/3960378010', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6836', 'nonce': 3960378010}, {'type': 'v1', 'addr': '172.21.15.151:6837', 'nonce': 3960378010}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24506': {'gid': 24506, 'name': 'h', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.96:6837/3978388286', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6836', 'nonce': 3978388286}, {'type': 'v1', 'addr': '172.21.15.96:6837', 'nonce': 3978388286}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24511': {'gid': 24511, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.151:6839/3794104313', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6838', 'nonce': 3794104313}, {'type': 'v1', 'addr': '172.21.15.151:6839', 'nonce': 3794104313}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24518': {'gid': 24518, 'name': 'g', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.43:6839/3404555030', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6838', 'nonce': 3404555030}, {'type': 'v1', 'addr': '172.21.15.43:6839', 'nonce': 3404555030}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24542': {'gid': 24542, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.96:6839/605527621', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6838', 'nonce': 605527621}, {'type': 'v1', 'addr': '172.21.15.96:6839', 'nonce': 605527621}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24542, 'qdb_cluster': [24493, 24506, 24511, 24518, 24542]} 2024-09-19T05:04:21.652 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-19T05:04:21.652 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-19T05:04:22.030 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:22.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.597+0000 7f0541600640 1 -- 172.21.15.43:0/4193671697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f053c076720 msgr2=0x7f053c076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:22.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.597+0000 7f0541600640 1 --2- 172.21.15.43:0/4193671697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f053c076720 0x7f053c076b00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f052c009900 tx=0x7f052c02a410 comp rx=0 tx=0).stop 2024-09-19T05:04:22.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.597+0000 7f0541600640 1 -- 172.21.15.43:0/4193671697 shutdown_connections 2024-09-19T05:04:22.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.597+0000 7f0541600640 1 --2- 172.21.15.43:0/4193671697 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f053c112a30 0x7f053c114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:22.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.597+0000 7f0541600640 1 --2- 172.21.15.43:0/4193671697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f053c077040 0x7f053c1124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:22.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.597+0000 7f0541600640 1 --2- 172.21.15.43:0/4193671697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f053c076720 0x7f053c076b00 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:22.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.597+0000 7f0541600640 1 -- 172.21.15.43:0/4193671697 >> 172.21.15.43:0/4193671697 conn(0x7f053c0ffa90 msgr2=0x7f053c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:22.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.597+0000 7f0541600640 1 -- 172.21.15.43:0/4193671697 shutdown_connections 2024-09-19T05:04:22.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.597+0000 7f0541600640 1 -- 172.21.15.43:0/4193671697 wait complete. 2024-09-19T05:04:22.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.598+0000 7f0541600640 1 Processor -- start 2024-09-19T05:04:22.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.598+0000 7f0541600640 1 -- start start 2024-09-19T05:04:22.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.598+0000 7f0541600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f053c076720 0x7f053c1a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:22.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.598+0000 7f0541600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f053c077040 0x7f053c1a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:22.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.598+0000 7f0541600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f053c112a30 0x7f053c19e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:22.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.598+0000 7f0541600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f053c1185f0 con 0x7f053c076720 2024-09-19T05:04:22.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.598+0000 7f0541600640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f053c118470 con 0x7f053c077040 2024-09-19T05:04:22.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.598+0000 7f0541600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f053c118770 con 0x7f053c112a30 2024-09-19T05:04:22.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.598+0000 7f053aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f053c076720 0x7f053c1a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:22.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.598+0000 7f053aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f053c076720 0x7f053c1a46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46480/0 (socket says 172.21.15.43:46480) 2024-09-19T05:04:22.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.598+0000 7f053aa00640 1 -- 172.21.15.43:0/3799876698 learned_addr learned my addr 172.21.15.43:0/3799876698 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:22.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.599+0000 7f0532000640 1 --2- 172.21.15.43:0/3799876698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f053c077040 0x7f053c1a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:22.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.599+0000 7f053b400640 1 --2- 172.21.15.43:0/3799876698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f053c112a30 0x7f053c19e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:22.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.599+0000 7f053aa00640 1 -- 172.21.15.43:0/3799876698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f053c112a30 msgr2=0x7f053c19e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:22.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.599+0000 7f053aa00640 1 --2- 172.21.15.43:0/3799876698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f053c112a30 0x7f053c19e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:22.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.599+0000 7f053aa00640 1 -- 172.21.15.43:0/3799876698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f053c077040 msgr2=0x7f053c1a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:22.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.599+0000 7f053aa00640 1 --2- 172.21.15.43:0/3799876698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f053c077040 0x7f053c1a4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:22.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.599+0000 7f053aa00640 1 -- 172.21.15.43:0/3799876698 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f053c19efb0 con 0x7f053c076720 2024-09-19T05:04:22.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.599+0000 7f053aa00640 1 --2- 172.21.15.43:0/3799876698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f053c076720 0x7f053c1a46c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f052c004770 tx=0x7f052c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:22.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.599+0000 7f0533e00640 1 -- 172.21.15.43:0/3799876698 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c03f070 con 0x7f053c076720 2024-09-19T05:04:22.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.600+0000 7f0541600640 1 -- 172.21.15.43:0/3799876698 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f053c19f240 con 0x7f053c076720 2024-09-19T05:04:22.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.600+0000 7f0533e00640 1 -- 172.21.15.43:0/3799876698 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f052c031ca0 con 0x7f053c076720 2024-09-19T05:04:22.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.600+0000 7f0533e00640 1 -- 172.21.15.43:0/3799876698 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c035030 con 0x7f053c076720 2024-09-19T05:04:22.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.600+0000 7f0541600640 1 -- 172.21.15.43:0/3799876698 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f053c1ab590 con 0x7f053c076720 2024-09-19T05:04:22.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.603+0000 7f0533e00640 1 -- 172.21.15.43:0/3799876698 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f052c042070 con 0x7f053c076720 2024-09-19T05:04:22.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.603+0000 7f0543ffa740 1 Processor -- start 2024-09-19T05:04:22.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.603+0000 7f0543ffa740 1 -- start start 2024-09-19T05:04:22.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.603+0000 7f0543ffa740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557eb0cc4330 0x557eb0cc4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:22.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.603+0000 7f0543ffa740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557eb0cc4c50 0x557eb0ccefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:22.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.603+0000 7f0543ffa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eb0ccf510 0x557eb0cd1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:22.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.603+0000 7f0543ffa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557eb0c2a0b0 con 0x557eb0ccf510 2024-09-19T05:04:22.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.603+0000 7f0543ffa740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557eb0bd1ab0 con 0x557eb0cc4330 2024-09-19T05:04:22.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.603+0000 7f0543ffa740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557eb0c26070 con 0x557eb0cc4c50 2024-09-19T05:04:22.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.604+0000 7f053aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557eb0cc4c50 0x557eb0ccefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:22.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.604+0000 7f0532000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eb0ccf510 0x557eb0cd1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:22.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.604+0000 7f053aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557eb0cc4c50 0x557eb0ccefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:37804/0 (socket says 172.21.15.43:37804) 2024-09-19T05:04:22.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.604+0000 7f053aa00640 1 -- 172.21.15.43:0/2188944610 learned_addr learned my addr 172.21.15.43:0/2188944610 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:22.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.604+0000 7f053b400640 1 --2- 172.21.15.43:0/2188944610 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557eb0cc4330 0x557eb0cc4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:22.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.604+0000 7f053aa00640 1 -- 172.21.15.43:0/2188944610 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557eb0cc4330 msgr2=0x557eb0cc4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:22.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.604+0000 7f053aa00640 1 --2- 172.21.15.43:0/2188944610 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557eb0cc4330 0x557eb0cc4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:22.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.604+0000 7f053aa00640 1 -- 172.21.15.43:0/2188944610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eb0ccf510 msgr2=0x557eb0cd1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:22.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.604+0000 7f053aa00640 1 --2- 172.21.15.43:0/2188944610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eb0ccf510 0x557eb0cd1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:22.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.604+0000 7f053aa00640 1 -- 172.21.15.43:0/2188944610 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557eb0bde4a0 con 0x557eb0cc4c50 2024-09-19T05:04:22.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.604+0000 7f0532000640 1 --2- 172.21.15.43:0/2188944610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eb0ccf510 0x557eb0cd1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:04:22.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.604+0000 7f0533e00640 1 --2- 172.21.15.43:0/3799876698 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f05200a3240 0x7f05200a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:22.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.604+0000 7f053aa00640 1 --2- 172.21.15.43:0/2188944610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557eb0cc4c50 0x557eb0ccefd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f052c0f3fb0 tx=0x7f052c0f81a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f199149f2bd11cea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:22.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.605+0000 7f053b400640 1 --2- 172.21.15.43:0/3799876698 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f05200a3240 0x7f05200a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:22.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.605+0000 7f0533e00640 1 -- 172.21.15.43:0/3799876698 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f052c0eff10 con 0x7f053c076720 2024-09-19T05:04:22.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.605+0000 7f052b400640 1 -- 172.21.15.43:0/2188944610 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c0b8900 con 0x557eb0cc4c50 2024-09-19T05:04:22.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.605+0000 7f052b400640 1 -- 172.21.15.43:0/2188944610 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f052c0bc660 con 0x557eb0cc4c50 2024-09-19T05:04:22.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.605+0000 7f052b400640 1 -- 172.21.15.43:0/2188944610 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c0bbfc0 con 0x557eb0cc4c50 2024-09-19T05:04:22.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.606+0000 7f0543ffa740 1 -- 172.21.15.43:0/2188944610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557eb0cc4c50 msgr2=0x557eb0ccefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:22.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.606+0000 7f053b400640 1 --2- 172.21.15.43:0/3799876698 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f05200a3240 0x7f05200a5700 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f053c06b460 tx=0x7f0534009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:22.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.606+0000 7f0543ffa740 1 --2- 172.21.15.43:0/2188944610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557eb0cc4c50 0x557eb0ccefd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f052c0f3fb0 tx=0x7f052c0f81a0 comp rx=0 tx=0).stop 2024-09-19T05:04:22.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.606+0000 7f0543ffa740 1 -- 172.21.15.43:0/2188944610 shutdown_connections 2024-09-19T05:04:22.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.606+0000 7f0543ffa740 1 --2- 172.21.15.43:0/2188944610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eb0ccf510 0x557eb0cd1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:22.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.606+0000 7f0543ffa740 1 --2- 172.21.15.43:0/2188944610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557eb0cc4c50 0x557eb0ccefd0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:22.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.606+0000 7f0543ffa740 1 --2- 172.21.15.43:0/2188944610 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557eb0cc4330 0x557eb0cc4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:22.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.606+0000 7f0543ffa740 1 -- 172.21.15.43:0/2188944610 >> 172.21.15.43:0/2188944610 conn(0x557eb0cc08c0 msgr2=0x557eb0cc0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:22.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.606+0000 7f0543ffa740 1 -- 172.21.15.43:0/2188944610 shutdown_connections 2024-09-19T05:04:22.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.606+0000 7f0543ffa740 1 -- 172.21.15.43:0/2188944610 wait complete. 2024-09-19T05:04:22.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.606+0000 7f0543ffa740 1 -- start start 2024-09-19T05:04:22.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.607+0000 7f0543ffa740 1 Processor -- start 2024-09-19T05:04:22.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.607+0000 7f0543ffa740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557eb0cc4330 0x557eb0ccc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:22.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.607+0000 7f0543ffa740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557eb0cc4c50 0x557eb0ccc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:22.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.607+0000 7f0543ffa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eb0ccf510 0x557eb0cccf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:22.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.607+0000 7f0543ffa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557eb0cd1ea0 con 0x557eb0ccf510 2024-09-19T05:04:22.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.607+0000 7f0543ffa740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557eb0bb2920 con 0x557eb0cc4330 2024-09-19T05:04:22.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.607+0000 7f0543ffa740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557eb0cd2020 con 0x557eb0cc4c50 2024-09-19T05:04:22.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.607+0000 7f0543ffa740 10 client.0 _ll_register_callbacks cb 0x557eb0bd68a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:04:22.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.608+0000 7f053b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eb0ccf510 0x557eb0cccf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:22.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.608+0000 7f0532000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557eb0cc4c50 0x557eb0ccc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:22.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.608+0000 7f0532000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557eb0cc4c50 0x557eb0ccc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:37814/0 (socket says 172.21.15.43:37814) 2024-09-19T05:04:22.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.608+0000 7f0532000640 1 -- 172.21.15.43:0/36279801 learned_addr learned my addr 172.21.15.43:0/36279801 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:22.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.608+0000 7f053aa00640 1 --2- 172.21.15.43:0/36279801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557eb0cc4330 0x557eb0ccc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:22.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.608+0000 7f0532000640 1 -- 172.21.15.43:0/36279801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557eb0cc4330 msgr2=0x557eb0ccc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:22.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.608+0000 7f0532000640 1 --2- 172.21.15.43:0/36279801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557eb0cc4330 0x557eb0ccc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:22.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.608+0000 7f0532000640 1 -- 172.21.15.43:0/36279801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eb0ccf510 msgr2=0x557eb0cccf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:22.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.608+0000 7f0532000640 1 --2- 172.21.15.43:0/36279801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eb0ccf510 0x557eb0cccf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:22.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.608+0000 7f0532000640 1 -- 172.21.15.43:0/36279801 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557eb099b2b0 con 0x557eb0cc4c50 2024-09-19T05:04:22.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.608+0000 7f0532000640 1 --2- 172.21.15.43:0/36279801 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557eb0cc4c50 0x557eb0ccc9e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0524012700 tx=0x7f0524012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f877eb88b035019b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:22.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.608+0000 7f051fe00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:04:22.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.609+0000 7f051fe00640 1 -- 172.21.15.43:0/36279801 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052400f920 con 0x557eb0cc4c50 2024-09-19T05:04:22.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.609+0000 7f051fe00640 1 -- 172.21.15.43:0/36279801 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f052400fac0 con 0x557eb0cc4c50 2024-09-19T05:04:22.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.609+0000 7f0543ffa740 1 -- 172.21.15.43:0/36279801 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f052c0bc660 con 0x557eb0cc4c50 2024-09-19T05:04:22.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.609+0000 7f051fe00640 1 -- 172.21.15.43:0/36279801 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f052400fcc0 con 0x557eb0cc4c50 2024-09-19T05:04:22.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.609+0000 7f0543ffa740 10 client.25021 fetch_fsmap learned FSMap version 28 2024-09-19T05:04:22.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.609+0000 7f0543ffa740 1 -- 172.21.15.43:0/36279801 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557eb0cddb40 con 0x557eb0cc4c50 2024-09-19T05:04:22.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.609+0000 7f051fe00640 1 -- 172.21.15.43:0/36279801 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f052c0bc660 con 0x557eb0cc4c50 2024-09-19T05:04:22.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.609+0000 7f051fe00640 1 -- 172.21.15.43:0/36279801 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f052401c420 con 0x557eb0cc4c50 2024-09-19T05:04:22.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.609+0000 7f0543ffa740 10 client.25021 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:04:22.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.609+0000 7f0543ffa740 10 client.25021 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T05:04:22.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.609+0000 7f0543ffa740 20 client.25021 populate_metadata read hostname 'smithi043' 2024-09-19T05:04:22.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.610+0000 7f0543ffa740 1 --2- 172.21.15.43:0/36279801 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x557eb0ce0190 0x557eb0ce1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:22.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.610+0000 7f0543ffa740 4 client.25021 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:04:22.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.610+0000 7f0543ffa740 1 -- 172.21.15.43:0/36279801 --> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557eb0c37620 con 0x557eb0ce0190 2024-09-19T05:04:22.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.610+0000 7f053aa00640 1 --2- 172.21.15.43:0/36279801 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x557eb0ce0190 0x557eb0ce1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:22.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.610+0000 7f053aa00640 1 --2- 172.21.15.43:0/36279801 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x557eb0ce0190 0x557eb0ce1600 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9eda226f8ab201c server_cookie=18f6b33bf507de9a in_seq=0 out_seq=0 2024-09-19T05:04:22.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.610+0000 7f051fe00640 10 client.25021 ms_handle_connect on v2:172.21.15.151:6836/3960378010 2024-09-19T05:04:22.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.611+0000 7f051fe00640 1 -- 172.21.15.43:0/36279801 <== mds.4 v2:172.21.15.151:6836/3960378010 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557eb0c37620 con 0x557eb0ce0190 2024-09-19T05:04:22.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.611+0000 7f051fe00640 10 client.25021 handle_command_reply: tid=0 2024-09-19T05:04:22.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.611+0000 7f0543ffa740 1 client.25021 shutdown 2024-09-19T05:04:22.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.611+0000 7f0543ffa740 1 -- 172.21.15.43:0/36279801 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557eb0cc4c50 msgr2=0x557eb0ccc9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:22.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.611+0000 7f0543ffa740 1 --2- 172.21.15.43:0/36279801 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557eb0cc4c50 0x557eb0ccc9e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0524012700 tx=0x7f0524012bc0 comp rx=0 tx=0).stop 2024-09-19T05:04:22.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.611+0000 7f0543ffa740 1 -- 172.21.15.43:0/36279801 shutdown_connections 2024-09-19T05:04:22.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.611+0000 7f0543ffa740 1 --2- 172.21.15.43:0/36279801 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x557eb0ce0190 0x557eb0ce1600 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:22.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.611+0000 7f0543ffa740 1 --2- 172.21.15.43:0/36279801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eb0ccf510 0x557eb0cccf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:22.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.611+0000 7f0543ffa740 1 --2- 172.21.15.43:0/36279801 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557eb0cc4c50 0x557eb0ccc9e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:22.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.611+0000 7f0543ffa740 1 --2- 172.21.15.43:0/36279801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557eb0cc4330 0x557eb0ccc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:22.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.611+0000 7f051fe00640 0 client.25021 ms_handle_reset on v2:172.21.15.151:6836/3960378010 2024-09-19T05:04:22.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.611+0000 7f0543ffa740 1 -- 172.21.15.43:0/36279801 >> 172.21.15.43:0/36279801 conn(0x557eb0cc08c0 msgr2=0x557eb0cc65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:22.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.612+0000 7f0543ffa740 1 -- 172.21.15.43:0/36279801 shutdown_connections 2024-09-19T05:04:22.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.612+0000 7f0543ffa740 1 -- 172.21.15.43:0/36279801 wait complete. 2024-09-19T05:04:22.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.612+0000 7f0543ffa740 20 client.25021 trim_cache size 0 max 16384 2024-09-19T05:04:22.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.630+0000 7f0543ffa740 1 Processor -- start 2024-09-19T05:04:22.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.631+0000 7f0543ffa740 1 -- start start 2024-09-19T05:04:22.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.631+0000 7f0543ffa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eb0cc4330 0x557eb0cce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:22.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.631+0000 7f0543ffa740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557eb0cc4c50 0x557eb0cccd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:22.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.631+0000 7f0543ffa740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557eb0ccf510 0x557eb0ce98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:22.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.631+0000 7f0543ffa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557eb0cdf940 con 0x557eb0cc4330 2024-09-19T05:04:22.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.631+0000 7f0543ffa740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557eb0ce0010 con 0x557eb0cc4c50 2024-09-19T05:04:22.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.631+0000 7f0543ffa740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557eb0ce06e0 con 0x557eb0ccf510 2024-09-19T05:04:22.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.631+0000 7f053aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eb0cc4330 0x557eb0cce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:22.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.631+0000 7f053aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eb0cc4330 0x557eb0cce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46508/0 (socket says 172.21.15.43:46508) 2024-09-19T05:04:22.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.631+0000 7f053aa00640 1 -- 172.21.15.43:0/408332764 learned_addr learned my addr 172.21.15.43:0/408332764 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:22.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.631+0000 7f053b400640 1 --2- 172.21.15.43:0/408332764 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557eb0ccf510 0x557eb0ce98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:22.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.631+0000 7f0532000640 1 --2- 172.21.15.43:0/408332764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557eb0cc4c50 0x557eb0cccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:22.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.631+0000 7f053aa00640 1 -- 172.21.15.43:0/408332764 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557eb0ccf510 msgr2=0x557eb0ce98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:22.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.631+0000 7f053aa00640 1 --2- 172.21.15.43:0/408332764 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557eb0ccf510 0x557eb0ce98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:22.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.631+0000 7f053aa00640 1 -- 172.21.15.43:0/408332764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557eb0cc4c50 msgr2=0x557eb0cccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:22.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.631+0000 7f053aa00640 1 --2- 172.21.15.43:0/408332764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557eb0cc4c50 0x557eb0cccd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:22.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.631+0000 7f053aa00640 1 -- 172.21.15.43:0/408332764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557eb0cca9f0 con 0x557eb0cc4330 2024-09-19T05:04:22.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.632+0000 7f053aa00640 1 --2- 172.21.15.43:0/408332764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eb0cc4330 0x557eb0cce410 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f052c0b9030 tx=0x7f052c0f0190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d338dac329ef4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:22.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.632+0000 7f052aa00640 1 -- 172.21.15.43:0/408332764 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c049040 con 0x557eb0cc4330 2024-09-19T05:04:22.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.632+0000 7f052aa00640 1 -- 172.21.15.43:0/408332764 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f052c0c0960 con 0x557eb0cc4330 2024-09-19T05:04:22.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.632+0000 7f052aa00640 1 -- 172.21.15.43:0/408332764 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c0bc1a0 con 0x557eb0cc4330 2024-09-19T05:04:22.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.633+0000 7f0543ffa740 1 -- 172.21.15.43:0/408332764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eb0cc4330 msgr2=0x557eb0cce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:22.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.633+0000 7f0543ffa740 1 --2- 172.21.15.43:0/408332764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eb0cc4330 0x557eb0cce410 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f052c0b9030 tx=0x7f052c0f0190 comp rx=0 tx=0).stop 2024-09-19T05:04:22.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.633+0000 7f0543ffa740 1 -- 172.21.15.43:0/408332764 shutdown_connections 2024-09-19T05:04:22.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.633+0000 7f0543ffa740 1 --2- 172.21.15.43:0/408332764 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557eb0ccf510 0x557eb0ce98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:22.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.633+0000 7f0543ffa740 1 --2- 172.21.15.43:0/408332764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557eb0cc4c50 0x557eb0cccd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:22.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.633+0000 7f0543ffa740 1 --2- 172.21.15.43:0/408332764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eb0cc4330 0x557eb0cce410 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:22.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.633+0000 7f0543ffa740 1 -- 172.21.15.43:0/408332764 >> 172.21.15.43:0/408332764 conn(0x557eb0cc08c0 msgr2=0x557eb0ccdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:22.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.633+0000 7f0543ffa740 1 -- 172.21.15.43:0/408332764 shutdown_connections 2024-09-19T05:04:22.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.633+0000 7f0543ffa740 1 -- 172.21.15.43:0/408332764 wait complete. 2024-09-19T05:04:22.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.633+0000 7f0543ffa740 1 -- start start 2024-09-19T05:04:22.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.633+0000 7f0543ffa740 1 Processor -- start 2024-09-19T05:04:22.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.634+0000 7f0543ffa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eb0cc4330 0x557eb0cc9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:22.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.634+0000 7f0543ffa740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557eb0cc4c50 0x557eb0cc9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:22.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.634+0000 7f0543ffa740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557eb0ccf510 0x557eb0cc9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:22.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.634+0000 7f0543ffa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557eb0ce1480 con 0x557eb0cc4330 2024-09-19T05:04:22.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.634+0000 7f0543ffa740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557eb0ce0db0 con 0x557eb0cc4c50 2024-09-19T05:04:22.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.634+0000 7f0543ffa740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557eb0ce1b50 con 0x557eb0ccf510 2024-09-19T05:04:22.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.634+0000 7f0543ffa740 10 client.0 _ll_register_callbacks cb 0x7f0524009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:04:22.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.634+0000 7f0532000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557eb0cc4c50 0x557eb0cc9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:22.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.634+0000 7f0532000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557eb0cc4c50 0x557eb0cc9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:58190/0 (socket says 172.21.15.43:58190) 2024-09-19T05:04:22.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.634+0000 7f0532000640 1 -- 172.21.15.43:0/2992196171 learned_addr learned my addr 172.21.15.43:0/2992196171 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:22.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.634+0000 7f053aa00640 1 --2- 172.21.15.43:0/2992196171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eb0cc4330 0x557eb0cc9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:22.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.634+0000 7f053b400640 1 --2- 172.21.15.43:0/2992196171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557eb0ccf510 0x557eb0cc9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:22.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.635+0000 7f0532000640 1 -- 172.21.15.43:0/2992196171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557eb0ccf510 msgr2=0x557eb0cc9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:22.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.635+0000 7f0532000640 1 --2- 172.21.15.43:0/2992196171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557eb0ccf510 0x557eb0cc9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:22.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.635+0000 7f0532000640 1 -- 172.21.15.43:0/2992196171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eb0cc4330 msgr2=0x557eb0cc9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:22.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.635+0000 7f0532000640 1 --2- 172.21.15.43:0/2992196171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eb0cc4330 0x557eb0cc9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:22.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.635+0000 7f0532000640 1 -- 172.21.15.43:0/2992196171 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557eb0ce7da0 con 0x557eb0cc4c50 2024-09-19T05:04:22.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.635+0000 7f0532000640 1 --2- 172.21.15.43:0/2992196171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557eb0cc4c50 0x557eb0cc9980 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x557eb0ccdae0 tx=0x7f052400fbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e05ea269489bfda server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:22.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.635+0000 7f051fe00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:04:22.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.635+0000 7f051fe00640 1 -- 172.21.15.43:0/2992196171 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052402b060 con 0x557eb0cc4c50 2024-09-19T05:04:22.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.635+0000 7f0543ffa740 1 -- 172.21.15.43:0/2992196171 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f052c0c0960 con 0x557eb0cc4c50 2024-09-19T05:04:22.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.635+0000 7f051fe00640 1 -- 172.21.15.43:0/2992196171 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f052401ca60 con 0x557eb0cc4c50 2024-09-19T05:04:22.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.635+0000 7f051fe00640 1 -- 172.21.15.43:0/2992196171 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f052401c630 con 0x557eb0cc4c50 2024-09-19T05:04:22.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.636+0000 7f0543ffa740 10 client.25043 fetch_fsmap learned FSMap version 28 2024-09-19T05:04:22.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.636+0000 7f0543ffa740 1 -- 172.21.15.43:0/2992196171 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557eb0d4ab50 con 0x557eb0cc4c50 2024-09-19T05:04:22.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.636+0000 7f051fe00640 1 -- 172.21.15.43:0/2992196171 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f052c0c0960 con 0x557eb0cc4c50 2024-09-19T05:04:22.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.636+0000 7f051fe00640 1 -- 172.21.15.43:0/2992196171 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0524030070 con 0x557eb0cc4c50 2024-09-19T05:04:22.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.636+0000 7f0543ffa740 10 client.25043 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:04:22.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.636+0000 7f0543ffa740 10 client.25043 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T05:04:22.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.636+0000 7f0543ffa740 20 client.25043 populate_metadata read hostname 'smithi043' 2024-09-19T05:04:22.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.636+0000 7f0543ffa740 1 --2- 172.21.15.43:0/2992196171 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x557eb0d4f780 0x557eb0d4e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:22.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.636+0000 7f0543ffa740 4 client.25043 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T05:04:22.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.636+0000 7f0543ffa740 1 -- 172.21.15.43:0/2992196171 --> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] -- command(tid 0: {"prefix": "status"}) -- 0x557eb0c31b30 con 0x557eb0d4f780 2024-09-19T05:04:22.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.637+0000 7f053aa00640 1 --2- 172.21.15.43:0/2992196171 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x557eb0d4f780 0x557eb0d4e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:22.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.637+0000 7f053aa00640 1 --2- 172.21.15.43:0/2992196171 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x557eb0d4f780 0x557eb0d4e490 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9de95cfd900d48f2 server_cookie=ca6ed1f9397f83c in_seq=0 out_seq=0 2024-09-19T05:04:22.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.637+0000 7f051fe00640 10 client.25043 ms_handle_connect on v2:172.21.15.151:6836/3960378010 2024-09-19T05:04:22.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.637+0000 7f051fe00640 1 -- 172.21.15.43:0/2992196171 <== mds.4 v2:172.21.15.151:6836/3960378010 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x557eb0c31b30 con 0x557eb0d4f780 2024-09-19T05:04:22.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.637+0000 7f051fe00640 10 client.25043 handle_command_reply: tid=0 2024-09-19T05:04:22.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.637+0000 7f0543ffa740 1 client.25043 shutdown 2024-09-19T05:04:22.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.637+0000 7f0543ffa740 1 -- 172.21.15.43:0/2992196171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557eb0cc4c50 msgr2=0x557eb0cc9980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:22.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.637+0000 7f0543ffa740 1 --2- 172.21.15.43:0/2992196171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557eb0cc4c50 0x557eb0cc9980 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x557eb0ccdae0 tx=0x7f052400fbd0 comp rx=0 tx=0).stop 2024-09-19T05:04:22.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.638+0000 7f0543ffa740 1 -- 172.21.15.43:0/2992196171 shutdown_connections 2024-09-19T05:04:22.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.638+0000 7f0543ffa740 1 --2- 172.21.15.43:0/2992196171 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x557eb0d4f780 0x557eb0d4e490 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:22.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.638+0000 7f0543ffa740 1 --2- 172.21.15.43:0/2992196171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557eb0ccf510 0x557eb0cc9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:22.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.638+0000 7f0543ffa740 1 --2- 172.21.15.43:0/2992196171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557eb0cc4c50 0x557eb0cc9980 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:22.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.638+0000 7f0543ffa740 1 --2- 172.21.15.43:0/2992196171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557eb0cc4330 0x557eb0cc9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:22.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.638+0000 7f051fe00640 0 client.25043 ms_handle_reset on v2:172.21.15.151:6836/3960378010 2024-09-19T05:04:22.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.638+0000 7f0543ffa740 1 -- 172.21.15.43:0/2992196171 >> 172.21.15.43:0/2992196171 conn(0x557eb0cc08c0 msgr2=0x557eb0cc65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:22.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.638+0000 7f0543ffa740 1 -- 172.21.15.43:0/2992196171 shutdown_connections 2024-09-19T05:04:22.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.638+0000 7f0543ffa740 1 -- 172.21.15.43:0/2992196171 wait complete. 2024-09-19T05:04:22.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.638+0000 7f0543ffa740 20 client.25043 trim_cache size 0 max 16384 2024-09-19T05:04:22.648 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:04:22.649 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "1456a810-7643-11ef-bceb-c7b262605968", 2024-09-19T05:04:22.649 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 4, 2024-09-19T05:04:22.649 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 24493, 2024-09-19T05:04:22.649 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-09-19T05:04:22.649 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-09-19T05:04:22.650 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-09-19T05:04:22.650 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 104.148453128, 2024-09-19T05:04:22.650 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 28, 2024-09-19T05:04:22.650 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 85, 2024-09-19T05:04:22.650 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 83, 2024-09-19T05:04:22.650 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 144.84812030399999 2024-09-19T05:04:22.651 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:04:22.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.639+0000 7f0541600640 1 -- 172.21.15.43:0/3799876698 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f05200a3240 msgr2=0x7f05200a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:22.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.639+0000 7f0541600640 1 --2- 172.21.15.43:0/3799876698 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f05200a3240 0x7f05200a5700 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f053c06b460 tx=0x7f0534009500 comp rx=0 tx=0).stop 2024-09-19T05:04:22.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.639+0000 7f0541600640 1 -- 172.21.15.43:0/3799876698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f053c076720 msgr2=0x7f053c1a46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:22.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.639+0000 7f0541600640 1 --2- 172.21.15.43:0/3799876698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f053c076720 0x7f053c1a46c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f052c004770 tx=0x7f052c003c10 comp rx=0 tx=0).stop 2024-09-19T05:04:22.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.639+0000 7f0541600640 1 -- 172.21.15.43:0/3799876698 shutdown_connections 2024-09-19T05:04:22.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.639+0000 7f0541600640 1 --2- 172.21.15.43:0/3799876698 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f05200a3240 0x7f05200a5700 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:22.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.639+0000 7f0541600640 1 --2- 172.21.15.43:0/3799876698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f053c112a30 0x7f053c19e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:22.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.639+0000 7f0541600640 1 --2- 172.21.15.43:0/3799876698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f053c077040 0x7f053c1a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:22.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.639+0000 7f0541600640 1 --2- 172.21.15.43:0/3799876698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f053c076720 0x7f053c1a46c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:22.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.639+0000 7f0541600640 1 -- 172.21.15.43:0/3799876698 >> 172.21.15.43:0/3799876698 conn(0x7f053c0ffa90 msgr2=0x7f053c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:22.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.639+0000 7f0541600640 1 -- 172.21.15.43:0/3799876698 shutdown_connections 2024-09-19T05:04:22.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:22.639+0000 7f0541600640 1 -- 172.21.15.43:0/3799876698 wait complete. 2024-09-19T05:04:23.202 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:22 smithi043 ceph-mon[30800]: pgmap v352: 97 pgs: 97 active+clean; 646 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-19T05:04:23.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:22 smithi096 ceph-mon[35085]: pgmap v352: 97 pgs: 97 active+clean; 646 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-19T05:04:23.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:22 smithi151 ceph-mon[35425]: pgmap v352: 97 pgs: 97 active+clean; 646 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-19T05:04:23.411 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-19T05:04:23.788 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:24.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.345+0000 7ff607600640 1 -- 172.21.15.43:0/2326874684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6001049c0 msgr2=0x7ff600108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:24.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.345+0000 7ff607600640 1 --2- 172.21.15.43:0/2326874684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6001049c0 0x7ff600108a10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec009900 tx=0x7ff5ec02a410 comp rx=0 tx=0).stop 2024-09-19T05:04:24.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.346+0000 7ff607600640 1 -- 172.21.15.43:0/2326874684 shutdown_connections 2024-09-19T05:04:24.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.346+0000 7ff607600640 1 --2- 172.21.15.43:0/2326874684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff600109140 0x7ff600110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.346+0000 7ff607600640 1 --2- 172.21.15.43:0/2326874684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6001049c0 0x7ff600108a10 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.346+0000 7ff607600640 1 --2- 172.21.15.43:0/2326874684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff600104010 0x7ff6001043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.346+0000 7ff607600640 1 -- 172.21.15.43:0/2326874684 >> 172.21.15.43:0/2326874684 conn(0x7ff6000ffa50 msgr2=0x7ff600101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:24.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.346+0000 7ff607600640 1 -- 172.21.15.43:0/2326874684 shutdown_connections 2024-09-19T05:04:24.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.346+0000 7ff607600640 1 -- 172.21.15.43:0/2326874684 wait complete. 2024-09-19T05:04:24.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.347+0000 7ff607600640 1 Processor -- start 2024-09-19T05:04:24.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.347+0000 7ff607600640 1 -- start start 2024-09-19T05:04:24.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.347+0000 7ff607600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff600104010 0x7ff60019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:24.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.347+0000 7ff607600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6001049c0 0x7ff60019ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:24.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.347+0000 7ff607600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff600109140 0x7ff60019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:24.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.347+0000 7ff607600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff600114190 con 0x7ff6001049c0 2024-09-19T05:04:24.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.347+0000 7ff607600640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff600114010 con 0x7ff600104010 2024-09-19T05:04:24.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.347+0000 7ff607600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff600114310 con 0x7ff600109140 2024-09-19T05:04:24.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.348+0000 7ff604c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff600104010 0x7ff60019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:24.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.348+0000 7ff604c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff600104010 0x7ff60019a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:58210/0 (socket says 172.21.15.43:58210) 2024-09-19T05:04:24.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.348+0000 7ff604c00640 1 -- 172.21.15.43:0/2194833245 learned_addr learned my addr 172.21.15.43:0/2194833245 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:24.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.348+0000 7ff605600640 1 --2- 172.21.15.43:0/2194833245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff600109140 0x7ff60019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:24.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.348+0000 7ff5ffe00640 1 --2- 172.21.15.43:0/2194833245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6001049c0 0x7ff60019ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:24.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.348+0000 7ff604c00640 1 -- 172.21.15.43:0/2194833245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff600109140 msgr2=0x7ff60019efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:24.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.348+0000 7ff604c00640 1 --2- 172.21.15.43:0/2194833245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff600109140 0x7ff60019efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.348+0000 7ff604c00640 1 -- 172.21.15.43:0/2194833245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6001049c0 msgr2=0x7ff60019ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:24.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.348+0000 7ff604c00640 1 --2- 172.21.15.43:0/2194833245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6001049c0 0x7ff60019ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.348+0000 7ff604c00640 1 -- 172.21.15.43:0/2194833245 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff60019f6a0 con 0x7ff600104010 2024-09-19T05:04:24.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.348+0000 7ff604c00640 1 --2- 172.21.15.43:0/2194833245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff600104010 0x7ff60019a6f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff5f400e9f0 tx=0x7ff5f400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:24.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.349+0000 7ff5fd600640 1 -- 172.21.15.43:0/2194833245 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f400cd80 con 0x7ff600104010 2024-09-19T05:04:24.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.349+0000 7ff607600640 1 -- 172.21.15.43:0/2194833245 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff60019f990 con 0x7ff600104010 2024-09-19T05:04:24.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.349+0000 7ff5fd600640 1 -- 172.21.15.43:0/2194833245 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5f4010910 con 0x7ff600104010 2024-09-19T05:04:24.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.349+0000 7ff607600640 1 -- 172.21.15.43:0/2194833245 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6001a7270 con 0x7ff600104010 2024-09-19T05:04:24.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.349+0000 7ff5fd600640 1 -- 172.21.15.43:0/2194833245 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4018980 con 0x7ff600104010 2024-09-19T05:04:24.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.352+0000 7ff60a231740 1 Processor -- start 2024-09-19T05:04:24.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.352+0000 7ff60a231740 1 -- start start 2024-09-19T05:04:24.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.352+0000 7ff5fd600640 1 -- 172.21.15.43:0/2194833245 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7ff5f4010ac0 con 0x7ff600104010 2024-09-19T05:04:24.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.353+0000 7ff60a231740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603820dc330 0x5603820dc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:24.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.353+0000 7ff60a231740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603820dcc50 0x5603820e6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:24.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.353+0000 7ff60a231740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603820e7510 0x5603820e9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:24.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.353+0000 7ff60a231740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5603820420b0 con 0x5603820e7510 2024-09-19T05:04:24.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.353+0000 7ff60a231740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560381fe9ab0 con 0x5603820dcc50 2024-09-19T05:04:24.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.353+0000 7ff60a231740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56038203e070 con 0x5603820dc330 2024-09-19T05:04:24.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.353+0000 7ff5ffe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603820e7510 0x5603820e9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:24.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.353+0000 7ff605600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603820dc330 0x5603820dc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:24.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.353+0000 7ff605600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603820dc330 0x5603820dc710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:37856/0 (socket says 172.21.15.43:37856) 2024-09-19T05:04:24.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.353+0000 7ff605600640 1 -- 172.21.15.43:0/1410864444 learned_addr learned my addr 172.21.15.43:0/1410864444 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:24.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.353+0000 7ff5ffe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603820e7510 0x5603820e9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46542/0 (socket says 172.21.15.43:46542) 2024-09-19T05:04:24.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.353+0000 7ff604c00640 1 --2- 172.21.15.43:0/1410864444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603820dcc50 0x5603820e6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:24.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.353+0000 7ff605600640 1 -- 172.21.15.43:0/1410864444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603820dcc50 msgr2=0x5603820e6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:24.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.353+0000 7ff605600640 1 --2- 172.21.15.43:0/1410864444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603820dcc50 0x5603820e6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.353+0000 7ff605600640 1 -- 172.21.15.43:0/1410864444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603820e7510 msgr2=0x5603820e9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:24.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.353+0000 7ff605600640 1 --2- 172.21.15.43:0/1410864444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603820e7510 0x5603820e9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.353+0000 7ff605600640 1 -- 172.21.15.43:0/1410864444 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560381ff64a0 con 0x5603820dc330 2024-09-19T05:04:24.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.353+0000 7ff5ffe00640 1 --2- 172.21.15.43:0/1410864444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603820e7510 0x5603820e9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:04:24.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.354+0000 7ff605600640 1 --2- 172.21.15.43:0/1410864444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603820dc330 0x5603820dc710 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8009f50 tx=0x7ff5f80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b62b214d3a8510f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:24.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.354+0000 7ff5fd600640 1 --2- 172.21.15.43:0/2194833245 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff5e80a32c0 0x7ff5e80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:24.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.354+0000 7ff5f1c00640 1 -- 172.21.15.43:0/1410864444 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f8004d20 con 0x5603820dc330 2024-09-19T05:04:24.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.354+0000 7ff5f1c00640 1 -- 172.21.15.43:0/1410864444 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5f8004050 con 0x5603820dc330 2024-09-19T05:04:24.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.354+0000 7ff5f1c00640 1 -- 172.21.15.43:0/1410864444 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f8005280 con 0x5603820dc330 2024-09-19T05:04:24.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.354+0000 7ff5fd600640 1 -- 172.21.15.43:0/2194833245 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ff5f40cfc40 con 0x7ff600104010 2024-09-19T05:04:24.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.354+0000 7ff604c00640 1 --2- 172.21.15.43:0/2194833245 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff5e80a32c0 0x7ff5e80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:24.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.355+0000 7ff60a231740 1 -- 172.21.15.43:0/1410864444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603820dc330 msgr2=0x5603820dc710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:24.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.355+0000 7ff60a231740 1 --2- 172.21.15.43:0/1410864444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603820dc330 0x5603820dc710 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8009f50 tx=0x7ff5f80118d0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.355+0000 7ff604c00640 1 --2- 172.21.15.43:0/2194833245 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff5e80a32c0 0x7ff5e80a5780 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4098ce0 tx=0x7ff5f40d36c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:24.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.355+0000 7ff60a231740 1 -- 172.21.15.43:0/1410864444 shutdown_connections 2024-09-19T05:04:24.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.355+0000 7ff60a231740 1 --2- 172.21.15.43:0/1410864444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603820e7510 0x5603820e9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.355+0000 7ff60a231740 1 --2- 172.21.15.43:0/1410864444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603820dcc50 0x5603820e6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.355+0000 7ff60a231740 1 --2- 172.21.15.43:0/1410864444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603820dc330 0x5603820dc710 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.355+0000 7ff60a231740 1 -- 172.21.15.43:0/1410864444 >> 172.21.15.43:0/1410864444 conn(0x5603820d88c0 msgr2=0x5603820d8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:24.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.355+0000 7ff60a231740 1 -- 172.21.15.43:0/1410864444 shutdown_connections 2024-09-19T05:04:24.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.355+0000 7ff60a231740 1 -- 172.21.15.43:0/1410864444 wait complete. 2024-09-19T05:04:24.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.356+0000 7ff60a231740 1 -- start start 2024-09-19T05:04:24.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.356+0000 7ff60a231740 1 Processor -- start 2024-09-19T05:04:24.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.356+0000 7ff60a231740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603820dc330 0x5603820e4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:24.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.356+0000 7ff60a231740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603820dcc50 0x5603820e4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:24.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.356+0000 7ff60a231740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603820e7510 0x5603820e5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:24.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.356+0000 7ff60a231740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5603820ea060 con 0x5603820dcc50 2024-09-19T05:04:24.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.356+0000 7ff60a231740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560381fca920 con 0x5603820dc330 2024-09-19T05:04:24.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.356+0000 7ff60a231740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5603820ea1e0 con 0x5603820e7510 2024-09-19T05:04:24.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.356+0000 7ff60a231740 10 client.0 _ll_register_callbacks cb 0x560381fee8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:04:24.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.357+0000 7ff605600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603820e7510 0x5603820e5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:24.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.357+0000 7ff605600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603820e7510 0x5603820e5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:37866/0 (socket says 172.21.15.43:37866) 2024-09-19T05:04:24.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.357+0000 7ff605600640 1 -- 172.21.15.43:0/3109677843 learned_addr learned my addr 172.21.15.43:0/3109677843 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:24.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.357+0000 7ff5ffe00640 1 --2- 172.21.15.43:0/3109677843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603820dcc50 0x5603820e4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:24.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.357+0000 7ff605600640 1 --2- 172.21.15.43:0/3109677843 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603820dc330 0x5603820e4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:24.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.357+0000 7ff605600640 1 -- 172.21.15.43:0/3109677843 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603820dc330 msgr2=0x5603820e4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:24.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.357+0000 7ff605600640 1 --2- 172.21.15.43:0/3109677843 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603820dc330 0x5603820e4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.357+0000 7ff605600640 1 -- 172.21.15.43:0/3109677843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603820dcc50 msgr2=0x5603820e4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:24.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.357+0000 7ff605600640 1 --2- 172.21.15.43:0/3109677843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603820dcc50 0x5603820e4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.357+0000 7ff605600640 1 -- 172.21.15.43:0/3109677843 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560381db32b0 con 0x5603820e7510 2024-09-19T05:04:24.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.357+0000 7ff605600640 1 --2- 172.21.15.43:0/3109677843 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603820e7510 0x5603820e5080 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff5f80046a0 tx=0x7ff5f80047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98e9a0e140de4e14 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:24.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.357+0000 7ff5e6000640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:04:24.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.358+0000 7ff5e6000640 1 -- 172.21.15.43:0/3109677843 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f8028450 con 0x5603820e7510 2024-09-19T05:04:24.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.358+0000 7ff5e6000640 1 -- 172.21.15.43:0/3109677843 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ff5f80285f0 con 0x5603820e7510 2024-09-19T05:04:24.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.358+0000 7ff60a231740 1 -- 172.21.15.43:0/3109677843 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5f8004050 con 0x5603820e7510 2024-09-19T05:04:24.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.358+0000 7ff5e6000640 1 -- 172.21.15.43:0/3109677843 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff5f80287f0 con 0x5603820e7510 2024-09-19T05:04:24.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.358+0000 7ff60a231740 10 client.25039 fetch_fsmap learned FSMap version 28 2024-09-19T05:04:24.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.358+0000 7ff60a231740 1 -- 172.21.15.43:0/3109677843 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5603820f5e10 con 0x5603820e7510 2024-09-19T05:04:24.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.358+0000 7ff5e6000640 1 -- 172.21.15.43:0/3109677843 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5f8004050 con 0x5603820e7510 2024-09-19T05:04:24.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.358+0000 7ff5e6000640 1 -- 172.21.15.43:0/3109677843 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff5f800fa30 con 0x5603820e7510 2024-09-19T05:04:24.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.359+0000 7ff60a231740 10 client.25039 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:04:24.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.359+0000 7ff60a231740 10 client.25039 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T05:04:24.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.359+0000 7ff60a231740 20 client.25039 populate_metadata read hostname 'smithi043' 2024-09-19T05:04:24.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.359+0000 7ff60a231740 1 --2- 172.21.15.43:0/3109677843 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x5603820f81b0 0x5603820f9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:24.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.359+0000 7ff60a231740 4 client.25039 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:04:24.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.359+0000 7ff60a231740 1 -- 172.21.15.43:0/3109677843 --> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56038204f620 con 0x5603820f81b0 2024-09-19T05:04:24.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.359+0000 7ff5ffe00640 1 --2- 172.21.15.43:0/3109677843 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x5603820f81b0 0x5603820f9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:24.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.360+0000 7ff5ffe00640 1 --2- 172.21.15.43:0/3109677843 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x5603820f81b0 0x5603820f9620 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=25e357dd49ae270b server_cookie=d93e7dd2467685ad in_seq=0 out_seq=0 2024-09-19T05:04:24.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.360+0000 7ff5e6000640 10 client.25039 ms_handle_connect on v2:172.21.15.96:6836/3978388286 2024-09-19T05:04:24.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.360+0000 7ff5e6000640 1 -- 172.21.15.43:0/3109677843 <== mds.3 v2:172.21.15.96:6836/3978388286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56038204f620 con 0x5603820f81b0 2024-09-19T05:04:24.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.360+0000 7ff5e6000640 10 client.25039 handle_command_reply: tid=0 2024-09-19T05:04:24.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.361+0000 7ff60a231740 1 client.25039 shutdown 2024-09-19T05:04:24.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.361+0000 7ff60a231740 1 -- 172.21.15.43:0/3109677843 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603820e7510 msgr2=0x5603820e5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:24.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.361+0000 7ff60a231740 1 --2- 172.21.15.43:0/3109677843 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603820e7510 0x5603820e5080 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff5f80046a0 tx=0x7ff5f80047b0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.361+0000 7ff60a231740 1 -- 172.21.15.43:0/3109677843 shutdown_connections 2024-09-19T05:04:24.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.361+0000 7ff60a231740 1 --2- 172.21.15.43:0/3109677843 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x5603820f81b0 0x5603820f9620 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.361+0000 7ff60a231740 1 --2- 172.21.15.43:0/3109677843 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603820e7510 0x5603820e5080 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.361+0000 7ff60a231740 1 --2- 172.21.15.43:0/3109677843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603820dcc50 0x5603820e4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.361+0000 7ff5e6000640 0 client.25039 ms_handle_reset on v2:172.21.15.96:6836/3978388286 2024-09-19T05:04:24.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.361+0000 7ff60a231740 1 --2- 172.21.15.43:0/3109677843 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603820dc330 0x5603820e4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.361+0000 7ff60a231740 1 -- 172.21.15.43:0/3109677843 >> 172.21.15.43:0/3109677843 conn(0x5603820d88c0 msgr2=0x5603820de5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:24.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.361+0000 7ff60a231740 1 -- 172.21.15.43:0/3109677843 shutdown_connections 2024-09-19T05:04:24.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.361+0000 7ff60a231740 1 -- 172.21.15.43:0/3109677843 wait complete. 2024-09-19T05:04:24.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.361+0000 7ff60a231740 20 client.25039 trim_cache size 0 max 16384 2024-09-19T05:04:24.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.380+0000 7ff60a231740 1 Processor -- start 2024-09-19T05:04:24.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.380+0000 7ff60a231740 1 -- start start 2024-09-19T05:04:24.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.380+0000 7ff60a231740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603820dc330 0x5603820d8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:24.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.380+0000 7ff60a231740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603820dcc50 0x5603820e6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:24.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.380+0000 7ff60a231740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603820e7510 0x560382101900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:24.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.380+0000 7ff60a231740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5603820f7960 con 0x5603820e7510 2024-09-19T05:04:24.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.380+0000 7ff60a231740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5603820f8030 con 0x5603820dc330 2024-09-19T05:04:24.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.380+0000 7ff60a231740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5603820f8700 con 0x5603820dcc50 2024-09-19T05:04:24.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.381+0000 7ff605600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603820dc330 0x5603820d8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:24.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.381+0000 7ff5ffe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603820dcc50 0x5603820e6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:24.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.381+0000 7ff605600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603820dc330 0x5603820d8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:58238/0 (socket says 172.21.15.43:58238) 2024-09-19T05:04:24.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.381+0000 7ff605600640 1 -- 172.21.15.43:0/2991628981 learned_addr learned my addr 172.21.15.43:0/2991628981 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:24.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.381+0000 7ff5ffe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603820dcc50 0x5603820e6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:37872/0 (socket says 172.21.15.43:37872) 2024-09-19T05:04:24.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.381+0000 7ff605600640 1 --2- 172.21.15.43:0/2991628981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603820e7510 0x560382101900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:24.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.381+0000 7ff5ffe00640 1 -- 172.21.15.43:0/2991628981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603820dc330 msgr2=0x5603820d8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:24.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.381+0000 7ff5ffe00640 1 --2- 172.21.15.43:0/2991628981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603820dc330 0x5603820d8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.381+0000 7ff5ffe00640 1 -- 172.21.15.43:0/2991628981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603820e7510 msgr2=0x560382101900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:24.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.381+0000 7ff5ffe00640 1 --2- 172.21.15.43:0/2991628981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603820e7510 0x560382101900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.381+0000 7ff5ffe00640 1 -- 172.21.15.43:0/2991628981 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603820dc120 con 0x5603820dcc50 2024-09-19T05:04:24.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.382+0000 7ff5ffe00640 1 --2- 172.21.15.43:0/2991628981 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603820dcc50 0x5603820e6570 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec02aa00 tx=0x7ff5ec005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91ef740eb79d58ed server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:24.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.382+0000 7ff5f1200640 1 -- 172.21.15.43:0/2991628981 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec035040 con 0x5603820dcc50 2024-09-19T05:04:24.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.382+0000 7ff5f1200640 1 -- 172.21.15.43:0/2991628981 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5ec04b070 con 0x5603820dcc50 2024-09-19T05:04:24.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.382+0000 7ff5f1200640 1 -- 172.21.15.43:0/2991628981 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec0315b0 con 0x5603820dcc50 2024-09-19T05:04:24.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.382+0000 7ff60a231740 1 -- 172.21.15.43:0/2991628981 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603820dcc50 msgr2=0x5603820e6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:24.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.382+0000 7ff60a231740 1 --2- 172.21.15.43:0/2991628981 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603820dcc50 0x5603820e6570 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec02aa00 tx=0x7ff5ec005b50 comp rx=0 tx=0).stop 2024-09-19T05:04:24.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.382+0000 7ff60a231740 1 -- 172.21.15.43:0/2991628981 shutdown_connections 2024-09-19T05:04:24.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.382+0000 7ff60a231740 1 --2- 172.21.15.43:0/2991628981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603820e7510 0x560382101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.382+0000 7ff60a231740 1 --2- 172.21.15.43:0/2991628981 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603820dcc50 0x5603820e6570 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.382+0000 7ff60a231740 1 --2- 172.21.15.43:0/2991628981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603820dc330 0x5603820d8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.382+0000 7ff60a231740 1 -- 172.21.15.43:0/2991628981 >> 172.21.15.43:0/2991628981 conn(0x5603820d88c0 msgr2=0x5603820e5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:24.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.383+0000 7ff60a231740 1 -- 172.21.15.43:0/2991628981 shutdown_connections 2024-09-19T05:04:24.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.383+0000 7ff60a231740 1 -- 172.21.15.43:0/2991628981 wait complete. 2024-09-19T05:04:24.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.383+0000 7ff60a231740 1 -- start start 2024-09-19T05:04:24.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.383+0000 7ff60a231740 1 Processor -- start 2024-09-19T05:04:24.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.384+0000 7ff60a231740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603820dc330 0x5603820fc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:24.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.384+0000 7ff60a231740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603820dcc50 0x5603820fca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:24.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.384+0000 7ff60a231740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603820e7510 0x5603820fcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:24.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.384+0000 7ff60a231740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5603820f94a0 con 0x5603820dc330 2024-09-19T05:04:24.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.384+0000 7ff60a231740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5603820f8dd0 con 0x5603820dcc50 2024-09-19T05:04:24.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.384+0000 7ff60a231740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5603820f9b70 con 0x5603820e7510 2024-09-19T05:04:24.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.384+0000 7ff60a231740 10 client.0 _ll_register_callbacks cb 0x7ff5f8004d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:04:24.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.384+0000 7ff5ffe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603820dcc50 0x5603820fca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:24.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.384+0000 7ff605600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603820dc330 0x5603820fc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:24.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.384+0000 7ff605600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603820dc330 0x5603820fc540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46558/0 (socket says 172.21.15.43:46558) 2024-09-19T05:04:24.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.384+0000 7ff605600640 1 -- 172.21.15.43:0/1594343629 learned_addr learned my addr 172.21.15.43:0/1594343629 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:24.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.384+0000 7ff5ffe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603820dcc50 0x5603820fca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:58250/0 (socket says 172.21.15.43:58250) 2024-09-19T05:04:24.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.384+0000 7ff605600640 1 --2- 172.21.15.43:0/1594343629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603820e7510 0x5603820fcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:24.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.384+0000 7ff605600640 1 -- 172.21.15.43:0/1594343629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603820e7510 msgr2=0x5603820fcfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:24.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.384+0000 7ff605600640 1 --2- 172.21.15.43:0/1594343629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603820e7510 0x5603820fcfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.384+0000 7ff605600640 1 -- 172.21.15.43:0/1594343629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603820dcc50 msgr2=0x5603820fca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:24.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.384+0000 7ff605600640 1 --2- 172.21.15.43:0/1594343629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603820dcc50 0x5603820fca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.384+0000 7ff605600640 1 -- 172.21.15.43:0/1594343629 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560382101610 con 0x5603820dc330 2024-09-19T05:04:24.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.384+0000 7ff5ffe00640 1 --2- 172.21.15.43:0/1594343629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603820dcc50 0x5603820fca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:04:24.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.385+0000 7ff605600640 1 --2- 172.21.15.43:0/1594343629 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603820dc330 0x5603820fc540 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff5f800fdd0 tx=0x7ff5f8004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fa0f961e4279b33 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:24.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.385+0000 7ff5e6000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:04:24.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.385+0000 7ff5e6000640 1 -- 172.21.15.43:0/1594343629 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f8003e50 con 0x5603820dc330 2024-09-19T05:04:24.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.385+0000 7ff5e6000640 1 -- 172.21.15.43:0/1594343629 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ff5f8003ff0 con 0x5603820dc330 2024-09-19T05:04:24.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.385+0000 7ff60a231740 1 -- 172.21.15.43:0/1594343629 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5ec04b070 con 0x5603820dc330 2024-09-19T05:04:24.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.385+0000 7ff5e6000640 1 -- 172.21.15.43:0/1594343629 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff5f8028990 con 0x5603820dc330 2024-09-19T05:04:24.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.385+0000 7ff60a231740 10 client.15072 fetch_fsmap learned FSMap version 28 2024-09-19T05:04:24.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.385+0000 7ff60a231740 1 -- 172.21.15.43:0/1594343629 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560382162bb0 con 0x5603820dc330 2024-09-19T05:04:24.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.386+0000 7ff5e6000640 1 -- 172.21.15.43:0/1594343629 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5ec04b070 con 0x5603820dc330 2024-09-19T05:04:24.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.386+0000 7ff5e6000640 1 -- 172.21.15.43:0/1594343629 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff5f800f480 con 0x5603820dc330 2024-09-19T05:04:24.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.386+0000 7ff60a231740 10 client.15072 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:04:24.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.386+0000 7ff60a231740 10 client.15072 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T05:04:24.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.386+0000 7ff60a231740 20 client.15072 populate_metadata read hostname 'smithi043' 2024-09-19T05:04:24.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.386+0000 7ff60a231740 1 --2- 172.21.15.43:0/1594343629 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x5603821676b0 0x5603821663c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:24.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.386+0000 7ff60a231740 4 client.15072 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T05:04:24.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.386+0000 7ff60a231740 1 -- 172.21.15.43:0/1594343629 --> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] -- command(tid 0: {"prefix": "status"}) -- 0x560382049b30 con 0x5603821676b0 2024-09-19T05:04:24.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.386+0000 7ff5ffe00640 1 --2- 172.21.15.43:0/1594343629 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x5603821676b0 0x5603821663c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:24.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.387+0000 7ff5ffe00640 1 --2- 172.21.15.43:0/1594343629 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x5603821676b0 0x5603821663c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7c4a3419e61e2540 server_cookie=cbb890bbd56d6bcd in_seq=0 out_seq=0 2024-09-19T05:04:24.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.387+0000 7ff5e6000640 10 client.15072 ms_handle_connect on v2:172.21.15.96:6836/3978388286 2024-09-19T05:04:24.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.387+0000 7ff5e6000640 1 -- 172.21.15.43:0/1594343629 <== mds.3 v2:172.21.15.96:6836/3978388286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x560382049b30 con 0x5603821676b0 2024-09-19T05:04:24.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.387+0000 7ff5e6000640 10 client.15072 handle_command_reply: tid=0 2024-09-19T05:04:24.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.387+0000 7ff60a231740 1 client.15072 shutdown 2024-09-19T05:04:24.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.387+0000 7ff60a231740 1 -- 172.21.15.43:0/1594343629 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603820dc330 msgr2=0x5603820fc540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:24.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.387+0000 7ff60a231740 1 --2- 172.21.15.43:0/1594343629 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603820dc330 0x5603820fc540 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff5f800fdd0 tx=0x7ff5f8004700 comp rx=0 tx=0).stop 2024-09-19T05:04:24.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.388+0000 7ff60a231740 1 -- 172.21.15.43:0/1594343629 shutdown_connections 2024-09-19T05:04:24.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.388+0000 7ff60a231740 1 --2- 172.21.15.43:0/1594343629 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x5603821676b0 0x5603821663c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.388+0000 7ff60a231740 1 --2- 172.21.15.43:0/1594343629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603820e7510 0x5603820fcfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.388+0000 7ff60a231740 1 --2- 172.21.15.43:0/1594343629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603820dcc50 0x5603820fca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.388+0000 7ff60a231740 1 --2- 172.21.15.43:0/1594343629 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603820dc330 0x5603820fc540 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.388+0000 7ff60a231740 1 -- 172.21.15.43:0/1594343629 >> 172.21.15.43:0/1594343629 conn(0x5603820d88c0 msgr2=0x5603820ddcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:24.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.388+0000 7ff5e6000640 0 client.15072 ms_handle_reset on v2:172.21.15.96:6836/3978388286 2024-09-19T05:04:24.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.388+0000 7ff60a231740 1 -- 172.21.15.43:0/1594343629 shutdown_connections 2024-09-19T05:04:24.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.388+0000 7ff60a231740 1 -- 172.21.15.43:0/1594343629 wait complete. 2024-09-19T05:04:24.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.388+0000 7ff60a231740 20 client.15072 trim_cache size 0 max 16384 2024-09-19T05:04:24.396 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:04:24.396 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "1456a810-7643-11ef-bceb-c7b262605968", 2024-09-19T05:04:24.396 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 3, 2024-09-19T05:04:24.396 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 24506, 2024-09-19T05:04:24.396 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-09-19T05:04:24.396 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-09-19T05:04:24.396 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-09-19T05:04:24.396 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 106.904816314, 2024-09-19T05:04:24.396 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 28, 2024-09-19T05:04:24.397 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 85, 2024-09-19T05:04:24.397 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 83, 2024-09-19T05:04:24.397 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 143.927832827 2024-09-19T05:04:24.397 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:04:24.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.389+0000 7ff607600640 1 -- 172.21.15.43:0/2194833245 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff5e80a32c0 msgr2=0x7ff5e80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:24.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.389+0000 7ff607600640 1 --2- 172.21.15.43:0/2194833245 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff5e80a32c0 0x7ff5e80a5780 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4098ce0 tx=0x7ff5f40d36c0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.389+0000 7ff607600640 1 -- 172.21.15.43:0/2194833245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff600104010 msgr2=0x7ff60019a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:24.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.389+0000 7ff607600640 1 --2- 172.21.15.43:0/2194833245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff600104010 0x7ff60019a6f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff5f400e9f0 tx=0x7ff5f400eeb0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.389+0000 7ff607600640 1 -- 172.21.15.43:0/2194833245 shutdown_connections 2024-09-19T05:04:24.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.389+0000 7ff607600640 1 --2- 172.21.15.43:0/2194833245 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff5e80a32c0 0x7ff5e80a5780 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.389+0000 7ff607600640 1 --2- 172.21.15.43:0/2194833245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff600109140 0x7ff60019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.389+0000 7ff607600640 1 --2- 172.21.15.43:0/2194833245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6001049c0 0x7ff60019ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.389+0000 7ff607600640 1 --2- 172.21.15.43:0/2194833245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff600104010 0x7ff60019a6f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:24.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.389+0000 7ff607600640 1 -- 172.21.15.43:0/2194833245 >> 172.21.15.43:0/2194833245 conn(0x7ff6000ffa50 msgr2=0x7ff600100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:24.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.389+0000 7ff607600640 1 -- 172.21.15.43:0/2194833245 shutdown_connections 2024-09-19T05:04:24.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:24.389+0000 7ff607600640 1 -- 172.21.15.43:0/2194833245 wait complete. 2024-09-19T05:04:25.199 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:24 smithi043 ceph-mon[30800]: pgmap v353: 97 pgs: 97 active+clean; 646 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-19T05:04:25.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:24 smithi096 ceph-mon[35085]: pgmap v353: 97 pgs: 97 active+clean; 646 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-19T05:04:25.236 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-19T05:04:25.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:24 smithi151 ceph-mon[35425]: pgmap v353: 97 pgs: 97 active+clean; 646 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-19T05:04:25.613 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:26.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.147+0000 7f72d3800640 1 -- 172.21.15.43:0/3911150071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72cc109140 msgr2=0x7f72cc110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:26.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.147+0000 7f72d3800640 1 --2- 172.21.15.43:0/3911150071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72cc109140 0x7f72cc110cc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f72c4009ca0 tx=0x7f72c402a8c0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.148+0000 7f72d3800640 1 -- 172.21.15.43:0/3911150071 shutdown_connections 2024-09-19T05:04:26.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.148+0000 7f72d3800640 1 --2- 172.21.15.43:0/3911150071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72cc109140 0x7f72cc110cc0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.148+0000 7f72d3800640 1 --2- 172.21.15.43:0/3911150071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72cc1049c0 0x7f72cc108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.148+0000 7f72d3800640 1 --2- 172.21.15.43:0/3911150071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72cc104010 0x7f72cc1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.148+0000 7f72d3800640 1 -- 172.21.15.43:0/3911150071 >> 172.21.15.43:0/3911150071 conn(0x7f72cc0ffa50 msgr2=0x7f72cc101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:26.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.148+0000 7f72d3800640 1 -- 172.21.15.43:0/3911150071 shutdown_connections 2024-09-19T05:04:26.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.148+0000 7f72d3800640 1 -- 172.21.15.43:0/3911150071 wait complete. 2024-09-19T05:04:26.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.149+0000 7f72d3800640 1 Processor -- start 2024-09-19T05:04:26.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.149+0000 7f72d3800640 1 -- start start 2024-09-19T05:04:26.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.149+0000 7f72d3800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72cc104010 0x7f72cc19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:26.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.150+0000 7f72d3800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72cc1049c0 0x7f72cc19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:26.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.150+0000 7f72d3800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72cc109140 0x7f72cc19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:26.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.150+0000 7f72d3800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f72cc1141a0 con 0x7f72cc104010 2024-09-19T05:04:26.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.150+0000 7f72d3800640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f72cc114020 con 0x7f72cc109140 2024-09-19T05:04:26.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.150+0000 7f72d3800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f72cc114320 con 0x7f72cc1049c0 2024-09-19T05:04:26.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.150+0000 7f72d0e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72cc104010 0x7f72cc19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:26.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.150+0000 7f72c3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72cc1049c0 0x7f72cc19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:26.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.150+0000 7f72d1800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72cc109140 0x7f72cc19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:26.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.150+0000 7f72d0e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72cc104010 0x7f72cc19a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33394/0 (socket says 172.21.15.43:33394) 2024-09-19T05:04:26.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.150+0000 7f72c3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72cc1049c0 0x7f72cc19ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:37486/0 (socket says 172.21.15.43:37486) 2024-09-19T05:04:26.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.150+0000 7f72d0e00640 1 -- 172.21.15.43:0/1003042568 learned_addr learned my addr 172.21.15.43:0/1003042568 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:26.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.150+0000 7f72c3e00640 1 -- 172.21.15.43:0/1003042568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72cc109140 msgr2=0x7f72cc19eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:26.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.150+0000 7f72c3e00640 1 --2- 172.21.15.43:0/1003042568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72cc109140 0x7f72cc19eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.150+0000 7f72c3e00640 1 -- 172.21.15.43:0/1003042568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72cc104010 msgr2=0x7f72cc19a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:26.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.150+0000 7f72c3e00640 1 --2- 172.21.15.43:0/1003042568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72cc104010 0x7f72cc19a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.150+0000 7f72c3e00640 1 -- 172.21.15.43:0/1003042568 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72cc19f5d0 con 0x7f72cc1049c0 2024-09-19T05:04:26.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.151+0000 7f72c3e00640 1 --2- 172.21.15.43:0/1003042568 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72cc1049c0 0x7f72cc19ab60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f72b400e9f0 tx=0x7f72b400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:26.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.151+0000 7f72ca000640 1 -- 172.21.15.43:0/1003042568 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b400ccc0 con 0x7f72cc1049c0 2024-09-19T05:04:26.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.151+0000 7f72d3800640 1 -- 172.21.15.43:0/1003042568 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72cc19f8c0 con 0x7f72cc1049c0 2024-09-19T05:04:26.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.151+0000 7f72ca000640 1 -- 172.21.15.43:0/1003042568 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72b4010910 con 0x7f72cc1049c0 2024-09-19T05:04:26.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.151+0000 7f72ca000640 1 -- 172.21.15.43:0/1003042568 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b4018960 con 0x7f72cc1049c0 2024-09-19T05:04:26.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.151+0000 7f72d3800640 1 -- 172.21.15.43:0/1003042568 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72cc1a71a0 con 0x7f72cc1049c0 2024-09-19T05:04:26.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.154+0000 7f72d62bd740 1 Processor -- start 2024-09-19T05:04:26.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.154+0000 7f72ca000640 1 -- 172.21.15.43:0/1003042568 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f72b4010ac0 con 0x7f72cc1049c0 2024-09-19T05:04:26.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.154+0000 7f72d62bd740 1 -- start start 2024-09-19T05:04:26.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.154+0000 7f72d62bd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55823a249330 0x55823a249710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:26.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.154+0000 7f72d62bd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55823a249c50 0x55823a253fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:26.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.155+0000 7f72d62bd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823a254510 0x55823a256900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:26.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.155+0000 7f72d62bd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55823a1af0b0 con 0x55823a249330 2024-09-19T05:04:26.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.155+0000 7f72d62bd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55823a156ab0 con 0x55823a249c50 2024-09-19T05:04:26.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.155+0000 7f72d62bd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55823a1ab070 con 0x55823a254510 2024-09-19T05:04:26.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.155+0000 7f72d1800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55823a249330 0x55823a249710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:26.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.155+0000 7f72d0e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55823a249c50 0x55823a253fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:26.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.155+0000 7f72c3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823a254510 0x55823a256900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:26.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.155+0000 7f72d1800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55823a249330 0x55823a249710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33408/0 (socket says 172.21.15.43:33408) 2024-09-19T05:04:26.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.155+0000 7f72d1800640 1 -- 172.21.15.43:0/1524961569 learned_addr learned my addr 172.21.15.43:0/1524961569 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:26.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.155+0000 7f72d0e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55823a249c50 0x55823a253fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:38540/0 (socket says 172.21.15.43:38540) 2024-09-19T05:04:26.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.155+0000 7f72d0e00640 1 -- 172.21.15.43:0/1524961569 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823a254510 msgr2=0x55823a256900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:26.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.155+0000 7f72d0e00640 1 --2- 172.21.15.43:0/1524961569 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823a254510 0x55823a256900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.155+0000 7f72d0e00640 1 -- 172.21.15.43:0/1524961569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55823a249330 msgr2=0x55823a249710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:26.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.155+0000 7f72d0e00640 1 --2- 172.21.15.43:0/1524961569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55823a249330 0x55823a249710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.155+0000 7f72d0e00640 1 -- 172.21.15.43:0/1524961569 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55823a1634a0 con 0x55823a249c50 2024-09-19T05:04:26.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.155+0000 7f72d1800640 1 --2- 172.21.15.43:0/1524961569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55823a249330 0x55823a249710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:04:26.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.156+0000 7f72d0e00640 1 --2- 172.21.15.43:0/1524961569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55823a249c50 0x55823a253fd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f72bc004590 tx=0x7f72bc00bbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddd2e9330c4ff5d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:26.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.156+0000 7f72ca000640 1 --2- 172.21.15.43:0/1003042568 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f72b00a3240 0x7f72b00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:26.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.156+0000 7f72c2000640 1 -- 172.21.15.43:0/1524961569 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72bc003bd0 con 0x55823a249c50 2024-09-19T05:04:26.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.156+0000 7f72ca000640 1 -- 172.21.15.43:0/1003042568 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f72b40cfc70 con 0x7f72cc1049c0 2024-09-19T05:04:26.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.156+0000 7f72c2000640 1 -- 172.21.15.43:0/1524961569 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72bc015070 con 0x55823a249c50 2024-09-19T05:04:26.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.156+0000 7f72c2000640 1 -- 172.21.15.43:0/1524961569 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72bc004da0 con 0x55823a249c50 2024-09-19T05:04:26.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.156+0000 7f72d1800640 1 --2- 172.21.15.43:0/1003042568 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f72b00a3240 0x7f72b00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:26.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.157+0000 7f72d62bd740 1 -- 172.21.15.43:0/1524961569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55823a249c50 msgr2=0x55823a253fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:26.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.157+0000 7f72d62bd740 1 --2- 172.21.15.43:0/1524961569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55823a249c50 0x55823a253fd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f72bc004590 tx=0x7f72bc00bbb0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.157+0000 7f72d1800640 1 --2- 172.21.15.43:0/1003042568 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f72b00a3240 0x7f72b00a5700 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f72c402aeb0 tx=0x7f72c4005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:26.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.157+0000 7f72d62bd740 1 -- 172.21.15.43:0/1524961569 shutdown_connections 2024-09-19T05:04:26.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.157+0000 7f72d62bd740 1 --2- 172.21.15.43:0/1524961569 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823a254510 0x55823a256900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.157+0000 7f72d62bd740 1 --2- 172.21.15.43:0/1524961569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55823a249c50 0x55823a253fd0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.157+0000 7f72d62bd740 1 --2- 172.21.15.43:0/1524961569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55823a249330 0x55823a249710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.157+0000 7f72d62bd740 1 -- 172.21.15.43:0/1524961569 >> 172.21.15.43:0/1524961569 conn(0x55823a2458c0 msgr2=0x55823a245ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:26.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.157+0000 7f72d62bd740 1 -- 172.21.15.43:0/1524961569 shutdown_connections 2024-09-19T05:04:26.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.157+0000 7f72d62bd740 1 -- 172.21.15.43:0/1524961569 wait complete. 2024-09-19T05:04:26.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.158+0000 7f72d62bd740 1 -- start start 2024-09-19T05:04:26.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.158+0000 7f72d62bd740 1 Processor -- start 2024-09-19T05:04:26.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.158+0000 7f72d62bd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55823a249330 0x55823a251620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:26.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.158+0000 7f72d62bd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823a249c50 0x55823a251b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:26.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.158+0000 7f72d62bd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55823a254510 0x55823a2520a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:26.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.158+0000 7f72d62bd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55823a257060 con 0x55823a254510 2024-09-19T05:04:26.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.158+0000 7f72d62bd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55823a137920 con 0x55823a249330 2024-09-19T05:04:26.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.158+0000 7f72d62bd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55823a2571e0 con 0x55823a249c50 2024-09-19T05:04:26.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.158+0000 7f72d62bd740 10 client.0 _ll_register_callbacks cb 0x55823a15b8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:04:26.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.159+0000 7f72d1800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55823a254510 0x55823a2520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:26.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.159+0000 7f72d0e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55823a249330 0x55823a251620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:26.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.159+0000 7f72d1800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55823a254510 0x55823a2520a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33422/0 (socket says 172.21.15.43:33422) 2024-09-19T05:04:26.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.159+0000 7f72d1800640 1 -- 172.21.15.43:0/2317679566 learned_addr learned my addr 172.21.15.43:0/2317679566 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:26.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.159+0000 7f72c3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823a249c50 0x55823a251b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:26.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.159+0000 7f72d0e00640 1 --2- 172.21.15.43:0/2317679566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55823a249330 0x55823a251620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:38550/0 (socket says 172.21.15.43:38550) 2024-09-19T05:04:26.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.159+0000 7f72c3e00640 1 -- 172.21.15.43:0/2317679566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55823a249330 msgr2=0x55823a251620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:26.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.159+0000 7f72c3e00640 1 --2- 172.21.15.43:0/2317679566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55823a249330 0x55823a251620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.159+0000 7f72c3e00640 1 -- 172.21.15.43:0/2317679566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55823a254510 msgr2=0x55823a2520a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:26.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.159+0000 7f72c3e00640 1 --2- 172.21.15.43:0/2317679566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55823a254510 0x55823a2520a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.159+0000 7f72c3e00640 1 -- 172.21.15.43:0/2317679566 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558239f202b0 con 0x55823a249c50 2024-09-19T05:04:26.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.159+0000 7f72c3e00640 1 --2- 172.21.15.43:0/2317679566 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823a249c50 0x55823a251b60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x55823a246c40 tx=0x7f72b40985a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbfb2d447ec88506 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:26.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.159+0000 7f72baa00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:04:26.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.160+0000 7f72baa00640 1 -- 172.21.15.43:0/2317679566 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b409bc60 con 0x55823a249c50 2024-09-19T05:04:26.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.160+0000 7f72baa00640 1 -- 172.21.15.43:0/2317679566 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f72b40bc0d0 con 0x55823a249c50 2024-09-19T05:04:26.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.160+0000 7f72d62bd740 1 -- 172.21.15.43:0/2317679566 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72bc015070 con 0x55823a249c50 2024-09-19T05:04:26.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.160+0000 7f72baa00640 1 -- 172.21.15.43:0/2317679566 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f72b40d3020 con 0x55823a249c50 2024-09-19T05:04:26.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.160+0000 7f72d62bd740 10 client.25057 fetch_fsmap learned FSMap version 28 2024-09-19T05:04:26.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.160+0000 7f72d62bd740 1 -- 172.21.15.43:0/2317679566 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55823a262e20 con 0x55823a249c50 2024-09-19T05:04:26.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.160+0000 7f72baa00640 1 -- 172.21.15.43:0/2317679566 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72bc015070 con 0x55823a249c50 2024-09-19T05:04:26.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.160+0000 7f72baa00640 1 -- 172.21.15.43:0/2317679566 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f72b40d9040 con 0x55823a249c50 2024-09-19T05:04:26.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.160+0000 7f72d62bd740 10 client.25057 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:04:26.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.161+0000 7f72d62bd740 10 client.25057 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T05:04:26.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.161+0000 7f72d62bd740 20 client.25057 populate_metadata read hostname 'smithi043' 2024-09-19T05:04:26.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.161+0000 7f72d62bd740 1 --2- 172.21.15.43:0/2317679566 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55823a265470 0x55823a2668e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:26.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.161+0000 7f72d62bd740 4 client.25057 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:04:26.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.161+0000 7f72d62bd740 1 -- 172.21.15.43:0/2317679566 --> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55823a1bc620 con 0x55823a265470 2024-09-19T05:04:26.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.161+0000 7f72d0e00640 1 --2- 172.21.15.43:0/2317679566 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55823a265470 0x55823a2668e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:26.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.161+0000 7f72d0e00640 1 --2- 172.21.15.43:0/2317679566 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55823a265470 0x55823a2668e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2af41e7b562ab18f server_cookie=f7fe1a9eb716df7a in_seq=0 out_seq=0 2024-09-19T05:04:26.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.161+0000 7f72baa00640 10 client.25057 ms_handle_connect on v2:172.21.15.151:6838/3794104313 2024-09-19T05:04:26.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.162+0000 7f72baa00640 1 -- 172.21.15.43:0/2317679566 <== mds.2 v2:172.21.15.151:6838/3794104313 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55823a1bc620 con 0x55823a265470 2024-09-19T05:04:26.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.162+0000 7f72baa00640 10 client.25057 handle_command_reply: tid=0 2024-09-19T05:04:26.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.162+0000 7f72d62bd740 1 client.25057 shutdown 2024-09-19T05:04:26.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.163+0000 7f72d62bd740 1 -- 172.21.15.43:0/2317679566 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823a249c50 msgr2=0x55823a251b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:26.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.163+0000 7f72d62bd740 1 --2- 172.21.15.43:0/2317679566 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823a249c50 0x55823a251b60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x55823a246c40 tx=0x7f72b40985a0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.163+0000 7f72d62bd740 1 -- 172.21.15.43:0/2317679566 shutdown_connections 2024-09-19T05:04:26.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.163+0000 7f72d62bd740 1 --2- 172.21.15.43:0/2317679566 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55823a265470 0x55823a2668e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.163+0000 7f72d62bd740 1 --2- 172.21.15.43:0/2317679566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55823a254510 0x55823a2520a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.163+0000 7f72d62bd740 1 --2- 172.21.15.43:0/2317679566 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823a249c50 0x55823a251b60 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.163+0000 7f72d62bd740 1 --2- 172.21.15.43:0/2317679566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55823a249330 0x55823a251620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.163+0000 7f72baa00640 0 client.25057 ms_handle_reset on v2:172.21.15.151:6838/3794104313 2024-09-19T05:04:26.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.163+0000 7f72d62bd740 1 -- 172.21.15.43:0/2317679566 >> 172.21.15.43:0/2317679566 conn(0x55823a2458c0 msgr2=0x55823a24b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:26.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.163+0000 7f72d62bd740 1 -- 172.21.15.43:0/2317679566 shutdown_connections 2024-09-19T05:04:26.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.163+0000 7f72d62bd740 1 -- 172.21.15.43:0/2317679566 wait complete. 2024-09-19T05:04:26.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.163+0000 7f72d62bd740 20 client.25057 trim_cache size 0 max 16384 2024-09-19T05:04:26.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.185+0000 7f72d62bd740 1 Processor -- start 2024-09-19T05:04:26.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.185+0000 7f72d62bd740 1 -- start start 2024-09-19T05:04:26.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.185+0000 7f72d62bd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55823a249330 0x55823a2642e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:26.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.185+0000 7f72d62bd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55823a249c50 0x55823a250620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:26.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.185+0000 7f72d62bd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823a254510 0x55823a24e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:26.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.185+0000 7f72d62bd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55823a264c20 con 0x55823a249330 2024-09-19T05:04:26.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.185+0000 7f72d62bd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55823a2652f0 con 0x55823a249c50 2024-09-19T05:04:26.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.185+0000 7f72d62bd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55823a2659c0 con 0x55823a254510 2024-09-19T05:04:26.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.185+0000 7f72d0e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55823a249330 0x55823a2642e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:26.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.186+0000 7f72d0e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55823a249330 0x55823a2642e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33434/0 (socket says 172.21.15.43:33434) 2024-09-19T05:04:26.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.186+0000 7f72d0e00640 1 -- 172.21.15.43:0/1114374040 learned_addr learned my addr 172.21.15.43:0/1114374040 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:26.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.186+0000 7f72c3e00640 1 --2- 172.21.15.43:0/1114374040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55823a249c50 0x55823a250620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:26.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.186+0000 7f72d1800640 1 --2- 172.21.15.43:0/1114374040 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823a254510 0x55823a24e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:26.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.186+0000 7f72d0e00640 1 -- 172.21.15.43:0/1114374040 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823a254510 msgr2=0x55823a24e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:26.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.186+0000 7f72d0e00640 1 --2- 172.21.15.43:0/1114374040 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823a254510 0x55823a24e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.186+0000 7f72d0e00640 1 -- 172.21.15.43:0/1114374040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55823a249c50 msgr2=0x55823a250620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:26.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.186+0000 7f72d0e00640 1 --2- 172.21.15.43:0/1114374040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55823a249c50 0x55823a250620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.186+0000 7f72d0e00640 1 -- 172.21.15.43:0/1114374040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55823a249a40 con 0x55823a249330 2024-09-19T05:04:26.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.186+0000 7f72d0e00640 1 --2- 172.21.15.43:0/1114374040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55823a249330 0x55823a2642e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f72bc000be0 tx=0x7f72bc003af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f0089c91cab06bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:26.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.186+0000 7f72c1600640 1 -- 172.21.15.43:0/1114374040 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72bc003dd0 con 0x55823a249330 2024-09-19T05:04:26.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.187+0000 7f72c1600640 1 -- 172.21.15.43:0/1114374040 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72bc015040 con 0x55823a249330 2024-09-19T05:04:26.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.187+0000 7f72c1600640 1 -- 172.21.15.43:0/1114374040 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72bc019680 con 0x55823a249330 2024-09-19T05:04:26.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.187+0000 7f72d62bd740 1 -- 172.21.15.43:0/1114374040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55823a249330 msgr2=0x55823a2642e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:26.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.187+0000 7f72d62bd740 1 --2- 172.21.15.43:0/1114374040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55823a249330 0x55823a2642e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f72bc000be0 tx=0x7f72bc003af0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.187+0000 7f72d62bd740 1 -- 172.21.15.43:0/1114374040 shutdown_connections 2024-09-19T05:04:26.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.187+0000 7f72d62bd740 1 --2- 172.21.15.43:0/1114374040 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823a254510 0x55823a24e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.187+0000 7f72d62bd740 1 --2- 172.21.15.43:0/1114374040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55823a249c50 0x55823a250620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.187+0000 7f72d62bd740 1 --2- 172.21.15.43:0/1114374040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55823a249330 0x55823a2642e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.187+0000 7f72d62bd740 1 -- 172.21.15.43:0/1114374040 >> 172.21.15.43:0/1114374040 conn(0x55823a2458c0 msgr2=0x55823a24ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:26.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.187+0000 7f72d62bd740 1 -- 172.21.15.43:0/1114374040 shutdown_connections 2024-09-19T05:04:26.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.187+0000 7f72d62bd740 1 -- 172.21.15.43:0/1114374040 wait complete. 2024-09-19T05:04:26.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.188+0000 7f72d62bd740 1 -- start start 2024-09-19T05:04:26.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.188+0000 7f72d62bd740 1 Processor -- start 2024-09-19T05:04:26.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.188+0000 7f72d62bd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823a249330 0x55823a26f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:26.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.188+0000 7f72d62bd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55823a249c50 0x55823a24e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:26.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.188+0000 7f72d62bd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55823a254510 0x55823a24ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:26.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.188+0000 7f72d62bd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55823a266760 con 0x55823a254510 2024-09-19T05:04:26.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.188+0000 7f72d62bd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55823a266090 con 0x55823a249c50 2024-09-19T05:04:26.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.188+0000 7f72d62bd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55823a266e30 con 0x55823a249330 2024-09-19T05:04:26.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.189+0000 7f72d62bd740 10 client.0 _ll_register_callbacks cb 0x7f72b4010e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:04:26.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.189+0000 7f72d0e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823a249330 0x55823a26f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:26.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.189+0000 7f72d1800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55823a254510 0x55823a24ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:26.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.189+0000 7f72d1800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55823a254510 0x55823a24ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33450/0 (socket says 172.21.15.43:33450) 2024-09-19T05:04:26.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.189+0000 7f72c3e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55823a249c50 0x55823a24e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:26.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.189+0000 7f72d1800640 1 -- 172.21.15.43:0/13634199 learned_addr learned my addr 172.21.15.43:0/13634199 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:26.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.189+0000 7f72d1800640 1 -- 172.21.15.43:0/13634199 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823a249330 msgr2=0x55823a26f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:26.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.189+0000 7f72d1800640 1 --2- 172.21.15.43:0/13634199 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823a249330 0x55823a26f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.189+0000 7f72d1800640 1 -- 172.21.15.43:0/13634199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55823a249c50 msgr2=0x55823a24e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:26.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.189+0000 7f72d1800640 1 --2- 172.21.15.43:0/13634199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55823a249c50 0x55823a24e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.189+0000 7f72d1800640 1 -- 172.21.15.43:0/13634199 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55823a26cf90 con 0x55823a254510 2024-09-19T05:04:26.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.190+0000 7f72d1800640 1 --2- 172.21.15.43:0/13634199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55823a254510 0x55823a24ecd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f72c4006010 tx=0x7f72c402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ddba0d5e4cb5652 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:26.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.190+0000 7f72baa00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:04:26.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.190+0000 7f72baa00640 1 -- 172.21.15.43:0/13634199 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c4031720 con 0x55823a254510 2024-09-19T05:04:26.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.190+0000 7f72baa00640 1 -- 172.21.15.43:0/13634199 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f72c4042030 con 0x55823a254510 2024-09-19T05:04:26.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.190+0000 7f72d62bd740 1 -- 172.21.15.43:0/13634199 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72bc015040 con 0x55823a254510 2024-09-19T05:04:26.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.190+0000 7f72baa00640 1 -- 172.21.15.43:0/13634199 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f72c40318c0 con 0x55823a254510 2024-09-19T05:04:26.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.190+0000 7f72d62bd740 10 client.15090 fetch_fsmap learned FSMap version 28 2024-09-19T05:04:26.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.190+0000 7f72d62bd740 1 -- 172.21.15.43:0/13634199 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55823a2cfd60 con 0x55823a254510 2024-09-19T05:04:26.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.191+0000 7f72baa00640 1 -- 172.21.15.43:0/13634199 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72bc015040 con 0x55823a254510 2024-09-19T05:04:26.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.191+0000 7f72baa00640 1 -- 172.21.15.43:0/13634199 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f72c402f970 con 0x55823a254510 2024-09-19T05:04:26.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.191+0000 7f72d62bd740 10 client.15090 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:04:26.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.191+0000 7f72d62bd740 10 client.15090 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T05:04:26.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.191+0000 7f72d62bd740 20 client.15090 populate_metadata read hostname 'smithi043' 2024-09-19T05:04:26.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.191+0000 7f72d62bd740 1 --2- 172.21.15.43:0/13634199 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55823a2d4940 0x55823a2d3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:26.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.191+0000 7f72d62bd740 4 client.15090 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T05:04:26.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.191+0000 7f72d62bd740 1 -- 172.21.15.43:0/13634199 --> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] -- command(tid 0: {"prefix": "status"}) -- 0x55823a1b6b30 con 0x55823a2d4940 2024-09-19T05:04:26.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.191+0000 7f72d0e00640 1 --2- 172.21.15.43:0/13634199 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55823a2d4940 0x55823a2d3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:26.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.192+0000 7f72d0e00640 1 --2- 172.21.15.43:0/13634199 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55823a2d4940 0x55823a2d3650 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d1999419ad31435b server_cookie=a3b0effbdbf5c64b in_seq=0 out_seq=0 2024-09-19T05:04:26.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.192+0000 7f72baa00640 10 client.15090 ms_handle_connect on v2:172.21.15.151:6838/3794104313 2024-09-19T05:04:26.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.192+0000 7f72baa00640 1 -- 172.21.15.43:0/13634199 <== mds.2 v2:172.21.15.151:6838/3794104313 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55823a1b6b30 con 0x55823a2d4940 2024-09-19T05:04:26.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.192+0000 7f72baa00640 10 client.15090 handle_command_reply: tid=0 2024-09-19T05:04:26.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.192+0000 7f72d62bd740 1 client.15090 shutdown 2024-09-19T05:04:26.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.193+0000 7f72d62bd740 1 -- 172.21.15.43:0/13634199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55823a254510 msgr2=0x55823a24ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:26.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.193+0000 7f72d62bd740 1 --2- 172.21.15.43:0/13634199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55823a254510 0x55823a24ecd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f72c4006010 tx=0x7f72c402a8c0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.193+0000 7f72d62bd740 1 -- 172.21.15.43:0/13634199 shutdown_connections 2024-09-19T05:04:26.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.193+0000 7f72d62bd740 1 --2- 172.21.15.43:0/13634199 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55823a2d4940 0x55823a2d3650 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.193+0000 7f72d62bd740 1 --2- 172.21.15.43:0/13634199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55823a254510 0x55823a24ecd0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.193+0000 7f72d62bd740 1 --2- 172.21.15.43:0/13634199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55823a249c50 0x55823a24e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.193+0000 7f72baa00640 0 client.15090 ms_handle_reset on v2:172.21.15.151:6838/3794104313 2024-09-19T05:04:26.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.193+0000 7f72d62bd740 1 --2- 172.21.15.43:0/13634199 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823a249330 0x55823a26f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.193+0000 7f72d62bd740 1 -- 172.21.15.43:0/13634199 >> 172.21.15.43:0/13634199 conn(0x55823a2458c0 msgr2=0x55823a250730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:26.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.193+0000 7f72d62bd740 1 -- 172.21.15.43:0/13634199 shutdown_connections 2024-09-19T05:04:26.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.193+0000 7f72d62bd740 1 -- 172.21.15.43:0/13634199 wait complete. 2024-09-19T05:04:26.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.193+0000 7f72d62bd740 20 client.15090 trim_cache size 0 max 16384 2024-09-19T05:04:26.198 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:04:26.198 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "1456a810-7643-11ef-bceb-c7b262605968", 2024-09-19T05:04:26.198 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 2, 2024-09-19T05:04:26.198 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 24511, 2024-09-19T05:04:26.198 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-09-19T05:04:26.198 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-09-19T05:04:26.198 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-09-19T05:04:26.199 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 109.717988893, 2024-09-19T05:04:26.199 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 28, 2024-09-19T05:04:26.199 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 85, 2024-09-19T05:04:26.199 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 83, 2024-09-19T05:04:26.199 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 143.04949614399999 2024-09-19T05:04:26.199 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:04:26.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.194+0000 7f72d3800640 1 -- 172.21.15.43:0/1003042568 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f72b00a3240 msgr2=0x7f72b00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:26.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.194+0000 7f72d3800640 1 --2- 172.21.15.43:0/1003042568 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f72b00a3240 0x7f72b00a5700 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f72c402aeb0 tx=0x7f72c4005d80 comp rx=0 tx=0).stop 2024-09-19T05:04:26.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.194+0000 7f72d3800640 1 -- 172.21.15.43:0/1003042568 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72cc1049c0 msgr2=0x7f72cc19ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:26.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.194+0000 7f72d3800640 1 --2- 172.21.15.43:0/1003042568 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72cc1049c0 0x7f72cc19ab60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f72b400e9f0 tx=0x7f72b400eeb0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.194+0000 7f72d3800640 1 -- 172.21.15.43:0/1003042568 shutdown_connections 2024-09-19T05:04:26.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.194+0000 7f72d3800640 1 --2- 172.21.15.43:0/1003042568 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f72b00a3240 0x7f72b00a5700 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.194+0000 7f72d3800640 1 --2- 172.21.15.43:0/1003042568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72cc109140 0x7f72cc19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.194+0000 7f72d3800640 1 --2- 172.21.15.43:0/1003042568 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72cc1049c0 0x7f72cc19ab60 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.194+0000 7f72d3800640 1 --2- 172.21.15.43:0/1003042568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72cc104010 0x7f72cc19a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:26.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.194+0000 7f72d3800640 1 -- 172.21.15.43:0/1003042568 >> 172.21.15.43:0/1003042568 conn(0x7f72cc0ffa50 msgr2=0x7f72cc100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:26.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.194+0000 7f72d3800640 1 -- 172.21.15.43:0/1003042568 shutdown_connections 2024-09-19T05:04:26.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:26.194+0000 7f72d3800640 1 -- 172.21.15.43:0/1003042568 wait complete. 2024-09-19T05:04:26.980 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.g status 2024-09-19T05:04:27.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:26 smithi096 ceph-mon[35085]: pgmap v354: 97 pgs: 97 active+clean; 646 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:04:27.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:26 smithi043 ceph-mon[30800]: pgmap v354: 97 pgs: 97 active+clean; 646 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:04:27.359 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:27.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:26 smithi151 ceph-mon[35425]: pgmap v354: 97 pgs: 97 active+clean; 646 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:04:27.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.925+0000 7ff433e00640 1 -- 172.21.15.43:0/3496874747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff42c1027a0 msgr2=0x7ff42c1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:27.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.925+0000 7ff433e00640 1 --2- 172.21.15.43:0/3496874747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff42c1027a0 0x7ff42c1067f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff418009900 tx=0x7ff41802a410 comp rx=0 tx=0).stop 2024-09-19T05:04:27.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.925+0000 7ff433e00640 1 -- 172.21.15.43:0/3496874747 shutdown_connections 2024-09-19T05:04:27.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.925+0000 7ff433e00640 1 --2- 172.21.15.43:0/3496874747 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff42c106f20 0x7ff42c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:27.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.925+0000 7ff433e00640 1 --2- 172.21.15.43:0/3496874747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff42c1027a0 0x7ff42c1067f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:27.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.925+0000 7ff433e00640 1 --2- 172.21.15.43:0/3496874747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff42c101df0 0x7ff42c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:27.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.925+0000 7ff433e00640 1 -- 172.21.15.43:0/3496874747 >> 172.21.15.43:0/3496874747 conn(0x7ff42c0fd8b0 msgr2=0x7ff42c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:27.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.925+0000 7ff433e00640 1 -- 172.21.15.43:0/3496874747 shutdown_connections 2024-09-19T05:04:27.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.925+0000 7ff433e00640 1 -- 172.21.15.43:0/3496874747 wait complete. 2024-09-19T05:04:27.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.927+0000 7ff433e00640 1 Processor -- start 2024-09-19T05:04:27.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.927+0000 7ff433e00640 1 -- start start 2024-09-19T05:04:27.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.927+0000 7ff433e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff42c101df0 0x7ff42c19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:27.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.927+0000 7ff433e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff42c1027a0 0x7ff42c19abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:27.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.927+0000 7ff433e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff42c106f20 0x7ff42c19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:27.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.927+0000 7ff433e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c111fd0 con 0x7ff42c1027a0 2024-09-19T05:04:27.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.927+0000 7ff433e00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c111e50 con 0x7ff42c106f20 2024-09-19T05:04:27.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.927+0000 7ff433e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c112150 con 0x7ff42c101df0 2024-09-19T05:04:27.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.927+0000 7ff430a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff42c1027a0 0x7ff42c19abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:27.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.927+0000 7ff431400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff42c101df0 0x7ff42c19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:27.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.927+0000 7ff430a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff42c1027a0 0x7ff42c19abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33468/0 (socket says 172.21.15.43:33468) 2024-09-19T05:04:27.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.927+0000 7ff430a00640 1 -- 172.21.15.43:0/1226109563 learned_addr learned my addr 172.21.15.43:0/1226109563 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:27.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.927+0000 7ff431e00640 1 --2- 172.21.15.43:0/1226109563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff42c106f20 0x7ff42c19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:27.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.928+0000 7ff430a00640 1 -- 172.21.15.43:0/1226109563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff42c101df0 msgr2=0x7ff42c19a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:27.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.928+0000 7ff430a00640 1 --2- 172.21.15.43:0/1226109563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff42c101df0 0x7ff42c19a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:27.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.928+0000 7ff430a00640 1 -- 172.21.15.43:0/1226109563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff42c106f20 msgr2=0x7ff42c19ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:27.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.928+0000 7ff430a00640 1 --2- 172.21.15.43:0/1226109563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff42c106f20 0x7ff42c19ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:27.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.928+0000 7ff430a00640 1 -- 172.21.15.43:0/1226109563 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42c19f640 con 0x7ff42c1027a0 2024-09-19T05:04:27.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.928+0000 7ff430a00640 1 --2- 172.21.15.43:0/1226109563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff42c1027a0 0x7ff42c19abd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff418004770 tx=0x7ff418003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:27.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.928+0000 7ff42a000640 1 -- 172.21.15.43:0/1226109563 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41803f070 con 0x7ff42c1027a0 2024-09-19T05:04:27.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.929+0000 7ff42a000640 1 -- 172.21.15.43:0/1226109563 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff418031ca0 con 0x7ff42c1027a0 2024-09-19T05:04:27.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.929+0000 7ff42a000640 1 -- 172.21.15.43:0/1226109563 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff418035030 con 0x7ff42c1027a0 2024-09-19T05:04:27.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.929+0000 7ff433e00640 1 -- 172.21.15.43:0/1226109563 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff42c19f8d0 con 0x7ff42c1027a0 2024-09-19T05:04:27.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.929+0000 7ff433e00640 1 -- 172.21.15.43:0/1226109563 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff42c1a7210 con 0x7ff42c1027a0 2024-09-19T05:04:27.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.932+0000 7ff42a000640 1 -- 172.21.15.43:0/1226109563 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7ff418042070 con 0x7ff42c1027a0 2024-09-19T05:04:27.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.932+0000 7ff4369f2740 1 Processor -- start 2024-09-19T05:04:27.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.932+0000 7ff4369f2740 1 -- start start 2024-09-19T05:04:27.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.932+0000 7ff4369f2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5623c55a9330 0x5623c55a9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:27.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.932+0000 7ff4369f2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623c55a9c50 0x5623c55b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:27.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.932+0000 7ff4369f2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5623c55b4510 0x5623c55b6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:27.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.932+0000 7ff4369f2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5623c550f0b0 con 0x5623c55a9c50 2024-09-19T05:04:27.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.932+0000 7ff4369f2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5623c54b6ab0 con 0x5623c55b4510 2024-09-19T05:04:27.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.932+0000 7ff4369f2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5623c550b070 con 0x5623c55a9330 2024-09-19T05:04:27.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.933+0000 7ff431400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623c55a9c50 0x5623c55b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:27.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.933+0000 7ff430a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5623c55b4510 0x5623c55b6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:27.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.933+0000 7ff431400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623c55a9c50 0x5623c55b3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33478/0 (socket says 172.21.15.43:33478) 2024-09-19T05:04:27.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.933+0000 7ff431400640 1 -- 172.21.15.43:0/2232334539 learned_addr learned my addr 172.21.15.43:0/2232334539 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:27.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.933+0000 7ff431e00640 1 --2- 172.21.15.43:0/2232334539 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5623c55a9330 0x5623c55a9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:27.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.933+0000 7ff431400640 1 -- 172.21.15.43:0/2232334539 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5623c55a9330 msgr2=0x5623c55a9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:27.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.933+0000 7ff431400640 1 --2- 172.21.15.43:0/2232334539 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5623c55a9330 0x5623c55a9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:27.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.933+0000 7ff431400640 1 -- 172.21.15.43:0/2232334539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5623c55b4510 msgr2=0x5623c55b6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:27.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.933+0000 7ff431400640 1 --2- 172.21.15.43:0/2232334539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5623c55b4510 0x5623c55b6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:27.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.933+0000 7ff431400640 1 -- 172.21.15.43:0/2232334539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623c54c34a0 con 0x5623c55a9c50 2024-09-19T05:04:27.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.933+0000 7ff431400640 1 --2- 172.21.15.43:0/2232334539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623c55a9c50 0x5623c55b3fd0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff420009cf0 tx=0x7ff420011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f026d3ddc9a7a0a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:27.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.933+0000 7ff42a000640 1 --2- 172.21.15.43:0/1226109563 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff4140a3240 0x7ff4140a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:27.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.934+0000 7ff41e600640 1 -- 172.21.15.43:0/2232334539 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff420004230 con 0x5623c55a9c50 2024-09-19T05:04:27.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.934+0000 7ff42a000640 1 -- 172.21.15.43:0/1226109563 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ff4180efef0 con 0x7ff42c1027a0 2024-09-19T05:04:27.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.934+0000 7ff41e600640 1 -- 172.21.15.43:0/2232334539 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4200043d0 con 0x5623c55a9c50 2024-09-19T05:04:27.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.934+0000 7ff41e600640 1 -- 172.21.15.43:0/2232334539 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff420019960 con 0x5623c55a9c50 2024-09-19T05:04:27.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.934+0000 7ff431e00640 1 --2- 172.21.15.43:0/1226109563 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff4140a3240 0x7ff4140a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:27.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.935+0000 7ff431e00640 1 --2- 172.21.15.43:0/1226109563 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff4140a3240 0x7ff4140a5700 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff424005fd0 tx=0x7ff424009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:27.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.935+0000 7ff4369f2740 1 -- 172.21.15.43:0/2232334539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623c55a9c50 msgr2=0x5623c55b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:27.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.935+0000 7ff4369f2740 1 --2- 172.21.15.43:0/2232334539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623c55a9c50 0x5623c55b3fd0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff420009cf0 tx=0x7ff420011810 comp rx=0 tx=0).stop 2024-09-19T05:04:27.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.935+0000 7ff4369f2740 1 -- 172.21.15.43:0/2232334539 shutdown_connections 2024-09-19T05:04:27.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.935+0000 7ff4369f2740 1 --2- 172.21.15.43:0/2232334539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5623c55b4510 0x5623c55b6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:27.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.935+0000 7ff4369f2740 1 --2- 172.21.15.43:0/2232334539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623c55a9c50 0x5623c55b3fd0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:27.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.935+0000 7ff4369f2740 1 --2- 172.21.15.43:0/2232334539 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5623c55a9330 0x5623c55a9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:27.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.935+0000 7ff4369f2740 1 -- 172.21.15.43:0/2232334539 >> 172.21.15.43:0/2232334539 conn(0x5623c55a58c0 msgr2=0x5623c55a5ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:27.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.935+0000 7ff4369f2740 1 -- 172.21.15.43:0/2232334539 shutdown_connections 2024-09-19T05:04:27.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.935+0000 7ff4369f2740 1 -- 172.21.15.43:0/2232334539 wait complete. 2024-09-19T05:04:27.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.936+0000 7ff4369f2740 1 -- start start 2024-09-19T05:04:27.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.936+0000 7ff4369f2740 1 Processor -- start 2024-09-19T05:04:27.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.936+0000 7ff4369f2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623c55a9330 0x5623c55b1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:27.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.936+0000 7ff4369f2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5623c55a9c50 0x5623c55b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:27.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.936+0000 7ff4369f2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5623c55b4510 0x5623c55b2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:27.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.936+0000 7ff4369f2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5623c55b7060 con 0x5623c55a9330 2024-09-19T05:04:27.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.936+0000 7ff4369f2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5623c5497920 con 0x5623c55b4510 2024-09-19T05:04:27.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.936+0000 7ff4369f2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5623c55b71e0 con 0x5623c55a9c50 2024-09-19T05:04:27.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.936+0000 7ff4369f2740 10 client.0 _ll_register_callbacks cb 0x5623c54bb8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:04:27.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.937+0000 7ff431400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623c55a9330 0x5623c55b1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:27.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.937+0000 7ff431e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5623c55b4510 0x5623c55b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:27.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.937+0000 7ff431400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623c55a9330 0x5623c55b1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33490/0 (socket says 172.21.15.43:33490) 2024-09-19T05:04:27.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.937+0000 7ff431400640 1 -- 172.21.15.43:0/593859596 learned_addr learned my addr 172.21.15.43:0/593859596 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:27.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.937+0000 7ff431e00640 1 --2- 172.21.15.43:0/593859596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5623c55b4510 0x5623c55b2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:38586/0 (socket says 172.21.15.43:38586) 2024-09-19T05:04:27.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.937+0000 7ff430a00640 1 --2- 172.21.15.43:0/593859596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5623c55a9c50 0x5623c55b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:27.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.937+0000 7ff431400640 1 -- 172.21.15.43:0/593859596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5623c55a9c50 msgr2=0x5623c55b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:27.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.937+0000 7ff431400640 1 --2- 172.21.15.43:0/593859596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5623c55a9c50 0x5623c55b1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:27.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.937+0000 7ff431400640 1 -- 172.21.15.43:0/593859596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5623c55b4510 msgr2=0x5623c55b2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:27.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.937+0000 7ff431400640 1 --2- 172.21.15.43:0/593859596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5623c55b4510 0x5623c55b2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:27.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.937+0000 7ff431400640 1 -- 172.21.15.43:0/593859596 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623c52802b0 con 0x5623c55a9330 2024-09-19T05:04:27.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.937+0000 7ff431400640 1 --2- 172.21.15.43:0/593859596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623c55a9330 0x5623c55b1610 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff42001b040 tx=0x7ff420003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcaf9f59e952f358 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:27.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.938+0000 7ff412a00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:04:27.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.938+0000 7ff412a00640 1 -- 172.21.15.43:0/593859596 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff420027070 con 0x5623c55a9330 2024-09-19T05:04:27.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.938+0000 7ff4369f2740 1 -- 172.21.15.43:0/593859596 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4200043d0 con 0x5623c55a9330 2024-09-19T05:04:27.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.938+0000 7ff412a00640 1 -- 172.21.15.43:0/593859596 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ff420005240 con 0x5623c55a9330 2024-09-19T05:04:27.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.938+0000 7ff412a00640 1 -- 172.21.15.43:0/593859596 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff420033050 con 0x5623c55a9330 2024-09-19T05:04:27.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.938+0000 7ff4369f2740 10 client.15102 fetch_fsmap learned FSMap version 28 2024-09-19T05:04:27.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.938+0000 7ff4369f2740 1 -- 172.21.15.43:0/593859596 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5623c55c2e20 con 0x5623c55a9330 2024-09-19T05:04:27.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.938+0000 7ff412a00640 1 -- 172.21.15.43:0/593859596 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4200043d0 con 0x5623c55a9330 2024-09-19T05:04:27.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.939+0000 7ff412a00640 1 -- 172.21.15.43:0/593859596 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff420004ba0 con 0x5623c55a9330 2024-09-19T05:04:27.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.939+0000 7ff4369f2740 10 client.15102 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:04:27.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.939+0000 7ff4369f2740 10 client.15102 resolve_mds: resolved name 'g' to daemon mds.g 2024-09-19T05:04:27.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.939+0000 7ff4369f2740 20 client.15102 populate_metadata read hostname 'smithi043' 2024-09-19T05:04:27.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.939+0000 7ff4369f2740 1 --2- 172.21.15.43:0/593859596 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5623c55c51f0 0x5623c55c6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:27.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.939+0000 7ff4369f2740 4 client.15102 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:04:27.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.939+0000 7ff4369f2740 1 -- 172.21.15.43:0/593859596 --> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623c551c620 con 0x5623c55c51f0 2024-09-19T05:04:27.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.939+0000 7ff430a00640 1 --2- 172.21.15.43:0/593859596 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5623c55c51f0 0x5623c55c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:27.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.940+0000 7ff430a00640 1 --2- 172.21.15.43:0/593859596 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5623c55c51f0 0x5623c55c6660 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fb17b4e8b6715a0c server_cookie=9c42f6866f848235 in_seq=0 out_seq=0 2024-09-19T05:04:27.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.940+0000 7ff412a00640 10 client.15102 ms_handle_connect on v2:172.21.15.43:6838/3404555030 2024-09-19T05:04:27.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.940+0000 7ff412a00640 1 -- 172.21.15.43:0/593859596 <== mds.1 v2:172.21.15.43:6838/3404555030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5623c551c620 con 0x5623c55c51f0 2024-09-19T05:04:27.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.940+0000 7ff412a00640 10 client.15102 handle_command_reply: tid=0 2024-09-19T05:04:27.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.940+0000 7ff4369f2740 1 client.15102 shutdown 2024-09-19T05:04:27.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.941+0000 7ff4369f2740 1 -- 172.21.15.43:0/593859596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623c55a9330 msgr2=0x5623c55b1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:27.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.941+0000 7ff4369f2740 1 --2- 172.21.15.43:0/593859596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623c55a9330 0x5623c55b1610 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff42001b040 tx=0x7ff420003820 comp rx=0 tx=0).stop 2024-09-19T05:04:27.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.941+0000 7ff4369f2740 1 -- 172.21.15.43:0/593859596 shutdown_connections 2024-09-19T05:04:27.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.941+0000 7ff4369f2740 1 --2- 172.21.15.43:0/593859596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5623c55b4510 0x5623c55b2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:27.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.941+0000 7ff4369f2740 1 --2- 172.21.15.43:0/593859596 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5623c55c51f0 0x5623c55c6660 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:27.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.941+0000 7ff4369f2740 1 --2- 172.21.15.43:0/593859596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5623c55a9c50 0x5623c55b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:27.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.941+0000 7ff4369f2740 1 --2- 172.21.15.43:0/593859596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623c55a9330 0x5623c55b1610 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:27.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.941+0000 7ff4369f2740 1 -- 172.21.15.43:0/593859596 >> 172.21.15.43:0/593859596 conn(0x5623c55a58c0 msgr2=0x5623c55ab5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:27.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.941+0000 7ff412a00640 0 client.15102 ms_handle_reset on v2:172.21.15.43:6838/3404555030 2024-09-19T05:04:27.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.941+0000 7ff4369f2740 1 -- 172.21.15.43:0/593859596 shutdown_connections 2024-09-19T05:04:27.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.941+0000 7ff4369f2740 1 -- 172.21.15.43:0/593859596 wait complete. 2024-09-19T05:04:27.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.941+0000 7ff4369f2740 20 client.15102 trim_cache size 0 max 16384 2024-09-19T05:04:27.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.960+0000 7ff4369f2740 1 Processor -- start 2024-09-19T05:04:27.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.960+0000 7ff4369f2740 1 -- start start 2024-09-19T05:04:27.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.961+0000 7ff4369f2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5623c55a9330 0x5623c55b3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:27.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.961+0000 7ff4369f2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623c55a9c50 0x5623c55b1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:27.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.961+0000 7ff4369f2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5623c55b4510 0x5623c55b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:27.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.961+0000 7ff4369f2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5623c55c49a0 con 0x5623c55a9c50 2024-09-19T05:04:27.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.961+0000 7ff4369f2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5623c55c5070 con 0x5623c55b4510 2024-09-19T05:04:27.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.961+0000 7ff4369f2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5623c55c5740 con 0x5623c55a9330 2024-09-19T05:04:27.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.961+0000 7ff430a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623c55a9c50 0x5623c55b1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:27.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.961+0000 7ff430a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623c55a9c50 0x5623c55b1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33492/0 (socket says 172.21.15.43:33492) 2024-09-19T05:04:27.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.961+0000 7ff430a00640 1 -- 172.21.15.43:0/4000722582 learned_addr learned my addr 172.21.15.43:0/4000722582 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:27.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.961+0000 7ff431400640 1 --2- 172.21.15.43:0/4000722582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5623c55a9330 0x5623c55b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:27.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.961+0000 7ff431e00640 1 --2- 172.21.15.43:0/4000722582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5623c55b4510 0x5623c55b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:27.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.962+0000 7ff430a00640 1 -- 172.21.15.43:0/4000722582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5623c55a9330 msgr2=0x5623c55b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:27.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.962+0000 7ff430a00640 1 --2- 172.21.15.43:0/4000722582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5623c55a9330 0x5623c55b3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:27.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.962+0000 7ff430a00640 1 -- 172.21.15.43:0/4000722582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5623c55b4510 msgr2=0x5623c55b20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:27.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.962+0000 7ff430a00640 1 --2- 172.21.15.43:0/4000722582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5623c55b4510 0x5623c55b20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:27.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.962+0000 7ff430a00640 1 -- 172.21.15.43:0/4000722582 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623c55a6a50 con 0x5623c55a9c50 2024-09-19T05:04:27.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.962+0000 7ff431400640 1 --2- 172.21.15.43:0/4000722582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5623c55a9330 0x5623c55b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:04:27.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.962+0000 7ff430a00640 1 --2- 172.21.15.43:0/4000722582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623c55a9c50 0x5623c55b1b60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff4180b8ae0 tx=0x7ff4180f3970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9afa0b14c02b797 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:27.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.962+0000 7ff41dc00640 1 -- 172.21.15.43:0/4000722582 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff418049040 con 0x5623c55a9c50 2024-09-19T05:04:27.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.962+0000 7ff41dc00640 1 -- 172.21.15.43:0/4000722582 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4180bc680 con 0x5623c55a9c50 2024-09-19T05:04:27.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.962+0000 7ff41dc00640 1 -- 172.21.15.43:0/4000722582 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4180baa10 con 0x5623c55a9c50 2024-09-19T05:04:27.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.963+0000 7ff4369f2740 1 -- 172.21.15.43:0/4000722582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623c55a9c50 msgr2=0x5623c55b1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:27.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.963+0000 7ff4369f2740 1 --2- 172.21.15.43:0/4000722582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623c55a9c50 0x5623c55b1b60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff4180b8ae0 tx=0x7ff4180f3970 comp rx=0 tx=0).stop 2024-09-19T05:04:27.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.963+0000 7ff4369f2740 1 -- 172.21.15.43:0/4000722582 shutdown_connections 2024-09-19T05:04:27.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.963+0000 7ff4369f2740 1 --2- 172.21.15.43:0/4000722582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5623c55b4510 0x5623c55b20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:27.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.963+0000 7ff4369f2740 1 --2- 172.21.15.43:0/4000722582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623c55a9c50 0x5623c55b1b60 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:27.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.963+0000 7ff4369f2740 1 --2- 172.21.15.43:0/4000722582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5623c55a9330 0x5623c55b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:27.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.963+0000 7ff4369f2740 1 -- 172.21.15.43:0/4000722582 >> 172.21.15.43:0/4000722582 conn(0x5623c55a58c0 msgr2=0x5623c55b2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:27.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.963+0000 7ff4369f2740 1 -- 172.21.15.43:0/4000722582 shutdown_connections 2024-09-19T05:04:27.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.963+0000 7ff4369f2740 1 -- 172.21.15.43:0/4000722582 wait complete. 2024-09-19T05:04:27.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.964+0000 7ff4369f2740 1 -- start start 2024-09-19T05:04:27.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.964+0000 7ff4369f2740 1 Processor -- start 2024-09-19T05:04:27.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.964+0000 7ff4369f2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623c55a9330 0x5623c55cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:27.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.964+0000 7ff4369f2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5623c55a9c50 0x5623c55ae430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:27.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.964+0000 7ff4369f2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5623c55b4510 0x5623c55aeab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:27.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.964+0000 7ff4369f2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5623c55c64e0 con 0x5623c55a9330 2024-09-19T05:04:27.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.964+0000 7ff4369f2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5623c55c5e10 con 0x5623c55a9c50 2024-09-19T05:04:27.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.964+0000 7ff4369f2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5623c55c6bb0 con 0x5623c55b4510 2024-09-19T05:04:27.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.964+0000 7ff4369f2740 10 client.0 _ll_register_callbacks cb 0x7ff420009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:04:27.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.965+0000 7ff431400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623c55a9330 0x5623c55cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:27.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.965+0000 7ff431400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623c55a9330 0x5623c55cf160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33498/0 (socket says 172.21.15.43:33498) 2024-09-19T05:04:27.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.965+0000 7ff431400640 1 -- 172.21.15.43:0/2448871915 learned_addr learned my addr 172.21.15.43:0/2448871915 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:27.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.965+0000 7ff430a00640 1 --2- 172.21.15.43:0/2448871915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5623c55a9c50 0x5623c55ae430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:27.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.965+0000 7ff431e00640 1 --2- 172.21.15.43:0/2448871915 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5623c55b4510 0x5623c55aeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:27.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.965+0000 7ff430a00640 1 -- 172.21.15.43:0/2448871915 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5623c55b4510 msgr2=0x5623c55aeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:27.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.965+0000 7ff430a00640 1 --2- 172.21.15.43:0/2448871915 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5623c55b4510 0x5623c55aeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:27.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.965+0000 7ff430a00640 1 -- 172.21.15.43:0/2448871915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623c55a9330 msgr2=0x5623c55cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:27.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.965+0000 7ff430a00640 1 --2- 172.21.15.43:0/2448871915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623c55a9330 0x5623c55cf160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:27.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.965+0000 7ff430a00640 1 -- 172.21.15.43:0/2448871915 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623c55ccc80 con 0x5623c55a9c50 2024-09-19T05:04:27.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.965+0000 7ff430a00640 1 --2- 172.21.15.43:0/2448871915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5623c55a9c50 0x5623c55ae430 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x5623c55c5650 tx=0x7ff4180bbfe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a020b0a703ddc74 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:27.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.965+0000 7ff412a00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:04:27.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.965+0000 7ff412a00640 1 -- 172.21.15.43:0/2448871915 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4180da070 con 0x5623c55a9c50 2024-09-19T05:04:27.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.966+0000 7ff4369f2740 1 -- 172.21.15.43:0/2448871915 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4180bc680 con 0x5623c55a9c50 2024-09-19T05:04:27.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.966+0000 7ff412a00640 1 -- 172.21.15.43:0/2448871915 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ff4180edb00 con 0x5623c55a9c50 2024-09-19T05:04:27.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.966+0000 7ff412a00640 1 -- 172.21.15.43:0/2448871915 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff418049040 con 0x5623c55a9c50 2024-09-19T05:04:27.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.966+0000 7ff4369f2740 10 client.25097 fetch_fsmap learned FSMap version 28 2024-09-19T05:04:27.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.966+0000 7ff4369f2740 1 -- 172.21.15.43:0/2448871915 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5623c562fa80 con 0x5623c55a9c50 2024-09-19T05:04:27.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.966+0000 7ff412a00640 1 -- 172.21.15.43:0/2448871915 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4180bc680 con 0x5623c55a9c50 2024-09-19T05:04:27.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.966+0000 7ff412a00640 1 -- 172.21.15.43:0/2448871915 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff4180bc8e0 con 0x5623c55a9c50 2024-09-19T05:04:27.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.966+0000 7ff4369f2740 10 client.25097 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:04:27.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.966+0000 7ff4369f2740 10 client.25097 resolve_mds: resolved name 'g' to daemon mds.g 2024-09-19T05:04:27.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.966+0000 7ff4369f2740 20 client.25097 populate_metadata read hostname 'smithi043' 2024-09-19T05:04:27.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.966+0000 7ff4369f2740 1 --2- 172.21.15.43:0/2448871915 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5623c5634500 0x5623c5633210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:27.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.966+0000 7ff4369f2740 4 client.25097 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T05:04:27.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.966+0000 7ff4369f2740 1 -- 172.21.15.43:0/2448871915 --> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] -- command(tid 0: {"prefix": "status"}) -- 0x5623c5516b30 con 0x5623c5634500 2024-09-19T05:04:27.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.967+0000 7ff431400640 1 --2- 172.21.15.43:0/2448871915 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5623c5634500 0x5623c5633210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:27.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.967+0000 7ff412a00640 10 client.25097 ms_handle_connect on v2:172.21.15.43:6838/3404555030 2024-09-19T05:04:27.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.967+0000 7ff431400640 1 --2- 172.21.15.43:0/2448871915 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5623c5634500 0x5623c5633210 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8bea8b68f725626a server_cookie=6520269eaf0ff54a in_seq=0 out_seq=0 2024-09-19T05:04:27.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.967+0000 7ff412a00640 1 -- 172.21.15.43:0/2448871915 <== mds.1 v2:172.21.15.43:6838/3404555030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5623c5516b30 con 0x5623c5634500 2024-09-19T05:04:27.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.967+0000 7ff412a00640 10 client.25097 handle_command_reply: tid=0 2024-09-19T05:04:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.967+0000 7ff4369f2740 1 client.25097 shutdown 2024-09-19T05:04:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.968+0000 7ff4369f2740 1 -- 172.21.15.43:0/2448871915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5623c55a9c50 msgr2=0x5623c55ae430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.968+0000 7ff4369f2740 1 --2- 172.21.15.43:0/2448871915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5623c55a9c50 0x5623c55ae430 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x5623c55c5650 tx=0x7ff4180bbfe0 comp rx=0 tx=0).stop 2024-09-19T05:04:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.968+0000 7ff4369f2740 1 -- 172.21.15.43:0/2448871915 shutdown_connections 2024-09-19T05:04:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.968+0000 7ff4369f2740 1 --2- 172.21.15.43:0/2448871915 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5623c5634500 0x5623c5633210 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.968+0000 7ff4369f2740 1 --2- 172.21.15.43:0/2448871915 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5623c55b4510 0x5623c55aeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.968+0000 7ff4369f2740 1 --2- 172.21.15.43:0/2448871915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5623c55a9c50 0x5623c55ae430 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.968+0000 7ff412a00640 0 client.25097 ms_handle_reset on v2:172.21.15.43:6838/3404555030 2024-09-19T05:04:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.968+0000 7ff4369f2740 1 --2- 172.21.15.43:0/2448871915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5623c55a9330 0x5623c55cf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.968+0000 7ff4369f2740 1 -- 172.21.15.43:0/2448871915 >> 172.21.15.43:0/2448871915 conn(0x5623c55a58c0 msgr2=0x5623c55a4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.968+0000 7ff4369f2740 1 -- 172.21.15.43:0/2448871915 shutdown_connections 2024-09-19T05:04:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.968+0000 7ff4369f2740 1 -- 172.21.15.43:0/2448871915 wait complete. 2024-09-19T05:04:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.968+0000 7ff4369f2740 20 client.25097 trim_cache size 0 max 16384 2024-09-19T05:04:27.983 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:04:27.983 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "1456a810-7643-11ef-bceb-c7b262605968", 2024-09-19T05:04:27.983 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 1, 2024-09-19T05:04:27.983 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 24518, 2024-09-19T05:04:27.983 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-09-19T05:04:27.983 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-09-19T05:04:27.983 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-09-19T05:04:27.983 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 112.497929192, 2024-09-19T05:04:27.983 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 28, 2024-09-19T05:04:27.983 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 85, 2024-09-19T05:04:27.983 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 83, 2024-09-19T05:04:27.984 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 141.61925108700001 2024-09-19T05:04:27.984 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:04:27.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.969+0000 7ff433e00640 1 -- 172.21.15.43:0/1226109563 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff4140a3240 msgr2=0x7ff4140a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:27.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.969+0000 7ff433e00640 1 --2- 172.21.15.43:0/1226109563 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff4140a3240 0x7ff4140a5700 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff424005fd0 tx=0x7ff424009500 comp rx=0 tx=0).stop 2024-09-19T05:04:27.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.969+0000 7ff433e00640 1 -- 172.21.15.43:0/1226109563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff42c1027a0 msgr2=0x7ff42c19abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.969+0000 7ff433e00640 1 --2- 172.21.15.43:0/1226109563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff42c1027a0 0x7ff42c19abd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff418004770 tx=0x7ff418003c10 comp rx=0 tx=0).stop 2024-09-19T05:04:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.969+0000 7ff433e00640 1 -- 172.21.15.43:0/1226109563 shutdown_connections 2024-09-19T05:04:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.969+0000 7ff433e00640 1 --2- 172.21.15.43:0/1226109563 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff4140a3240 0x7ff4140a5700 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.969+0000 7ff433e00640 1 --2- 172.21.15.43:0/1226109563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff42c106f20 0x7ff42c19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.969+0000 7ff433e00640 1 --2- 172.21.15.43:0/1226109563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff42c1027a0 0x7ff42c19abd0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.969+0000 7ff433e00640 1 --2- 172.21.15.43:0/1226109563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff42c101df0 0x7ff42c19a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.969+0000 7ff433e00640 1 -- 172.21.15.43:0/1226109563 >> 172.21.15.43:0/1226109563 conn(0x7ff42c0fd8b0 msgr2=0x7ff42c0fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.969+0000 7ff433e00640 1 -- 172.21.15.43:0/1226109563 shutdown_connections 2024-09-19T05:04:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:27.969+0000 7ff433e00640 1 -- 172.21.15.43:0/1226109563 wait complete. 2024-09-19T05:04:28.742 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T05:04:29.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:28 smithi043 ceph-mon[30800]: pgmap v355: 97 pgs: 97 active+clean; 646 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T05:04:29.130 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:29.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:28 smithi096 ceph-mon[35085]: pgmap v355: 97 pgs: 97 active+clean; 646 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T05:04:29.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:28 smithi151 ceph-mon[35425]: pgmap v355: 97 pgs: 97 active+clean; 646 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T05:04:29.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.712+0000 7ff403200640 1 -- 172.21.15.43:0/2830799122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3fc10c170 msgr2=0x7ff3fc10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:29.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.712+0000 7ff403200640 1 --2- 172.21.15.43:0/2830799122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3fc10c170 0x7ff3fc10e610 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4009ca0 tx=0x7ff3f402a950 comp rx=0 tx=0).stop 2024-09-19T05:04:29.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.712+0000 7ff403200640 1 -- 172.21.15.43:0/2830799122 shutdown_connections 2024-09-19T05:04:29.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.712+0000 7ff403200640 1 --2- 172.21.15.43:0/2830799122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3fc10c170 0x7ff3fc10e610 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:29.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.712+0000 7ff403200640 1 --2- 172.21.15.43:0/2830799122 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3fc103660 0x7ff3fc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:29.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.712+0000 7ff403200640 1 --2- 172.21.15.43:0/2830799122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3fc102d40 0x7ff3fc103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:29.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.712+0000 7ff403200640 1 -- 172.21.15.43:0/2830799122 >> 172.21.15.43:0/2830799122 conn(0x7ff3fc0fc7a0 msgr2=0x7ff3fc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:29.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.712+0000 7ff403200640 1 -- 172.21.15.43:0/2830799122 shutdown_connections 2024-09-19T05:04:29.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.712+0000 7ff403200640 1 -- 172.21.15.43:0/2830799122 wait complete. 2024-09-19T05:04:29.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.713+0000 7ff403200640 1 Processor -- start 2024-09-19T05:04:29.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.713+0000 7ff403200640 1 -- start start 2024-09-19T05:04:29.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.714+0000 7ff403200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3fc102d40 0x7ff3fc1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:29.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.714+0000 7ff403200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3fc103660 0x7ff3fc100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:29.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.714+0000 7ff403200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3fc10c170 0x7ff3fc101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:29.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.714+0000 7ff403200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff3fc111f80 con 0x7ff3fc103660 2024-09-19T05:04:29.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.714+0000 7ff403200640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff3fc111e00 con 0x7ff3fc10c170 2024-09-19T05:04:29.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.714+0000 7ff403200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff3fc112100 con 0x7ff3fc102d40 2024-09-19T05:04:29.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.714+0000 7ff3f3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3fc103660 0x7ff3fc100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:29.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.714+0000 7ff3fbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3fc102d40 0x7ff3fc1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:29.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.714+0000 7ff3f3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3fc103660 0x7ff3fc100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33516/0 (socket says 172.21.15.43:33516) 2024-09-19T05:04:29.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.714+0000 7ff3f3400640 1 -- 172.21.15.43:0/294240758 learned_addr learned my addr 172.21.15.43:0/294240758 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:29.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.714+0000 7ff401200640 1 --2- 172.21.15.43:0/294240758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3fc10c170 0x7ff3fc101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:29.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.715+0000 7ff3fbe00640 1 -- 172.21.15.43:0/294240758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3fc10c170 msgr2=0x7ff3fc101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:29.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.715+0000 7ff3fbe00640 1 --2- 172.21.15.43:0/294240758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3fc10c170 0x7ff3fc101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:29.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.715+0000 7ff3fbe00640 1 -- 172.21.15.43:0/294240758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3fc103660 msgr2=0x7ff3fc100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:29.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.715+0000 7ff3fbe00640 1 --2- 172.21.15.43:0/294240758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3fc103660 0x7ff3fc100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:29.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.715+0000 7ff3fbe00640 1 -- 172.21.15.43:0/294240758 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3fc1a6880 con 0x7ff3fc102d40 2024-09-19T05:04:29.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.715+0000 7ff3fbe00640 1 --2- 172.21.15.43:0/294240758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3fc102d40 0x7ff3fc1009a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec00d6d0 tx=0x7ff3ec00db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:29.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.715+0000 7ff3f9600640 1 -- 172.21.15.43:0/294240758 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec017070 con 0x7ff3fc102d40 2024-09-19T05:04:29.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.715+0000 7ff3f9600640 1 -- 172.21.15.43:0/294240758 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3ec009820 con 0x7ff3fc102d40 2024-09-19T05:04:29.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.715+0000 7ff403200640 1 -- 172.21.15.43:0/294240758 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3fc1a6b70 con 0x7ff3fc102d40 2024-09-19T05:04:29.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.715+0000 7ff3f9600640 1 -- 172.21.15.43:0/294240758 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec004d00 con 0x7ff3fc102d40 2024-09-19T05:04:29.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.716+0000 7ff403200640 1 -- 172.21.15.43:0/294240758 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3fc1a70b0 con 0x7ff3fc102d40 2024-09-19T05:04:29.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.719+0000 7ff405e6f740 1 Processor -- start 2024-09-19T05:04:29.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.719+0000 7ff3f9600640 1 -- 172.21.15.43:0/294240758 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7ff3ec0099d0 con 0x7ff3fc102d40 2024-09-19T05:04:29.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.719+0000 7ff405e6f740 1 -- start start 2024-09-19T05:04:29.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.719+0000 7ff405e6f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ddefa0330 0x557ddefa0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:29.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.719+0000 7ff405e6f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ddefa0c50 0x557ddefaafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:29.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.719+0000 7ff405e6f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ddefab510 0x557ddefad900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:29.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.719+0000 7ff405e6f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557ddef060b0 con 0x557ddefa0c50 2024-09-19T05:04:29.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.719+0000 7ff405e6f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557ddeeadab0 con 0x557ddefa0330 2024-09-19T05:04:29.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.719+0000 7ff405e6f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557ddef02070 con 0x557ddefab510 2024-09-19T05:04:29.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.720+0000 7ff3fbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ddefa0c50 0x557ddefaafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:29.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.720+0000 7ff3f3400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ddefab510 0x557ddefad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:29.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.720+0000 7ff3fbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ddefa0c50 0x557ddefaafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33518/0 (socket says 172.21.15.43:33518) 2024-09-19T05:04:29.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.720+0000 7ff3fbe00640 1 -- 172.21.15.43:0/1741563958 learned_addr learned my addr 172.21.15.43:0/1741563958 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:29.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.720+0000 7ff401200640 1 --2- 172.21.15.43:0/1741563958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ddefa0330 0x557ddefa0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:29.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.720+0000 7ff401200640 1 -- 172.21.15.43:0/1741563958 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ddefab510 msgr2=0x557ddefad900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:29.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.720+0000 7ff401200640 1 --2- 172.21.15.43:0/1741563958 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ddefab510 0x557ddefad900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:29.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.720+0000 7ff401200640 1 -- 172.21.15.43:0/1741563958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ddefa0c50 msgr2=0x557ddefaafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:29.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.720+0000 7ff401200640 1 --2- 172.21.15.43:0/1741563958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ddefa0c50 0x557ddefaafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:29.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.720+0000 7ff401200640 1 -- 172.21.15.43:0/1741563958 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ddeeba4a0 con 0x557ddefa0330 2024-09-19T05:04:29.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.721+0000 7ff401200640 1 --2- 172.21.15.43:0/1741563958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ddefa0330 0x557ddefa0710 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4005e30 tx=0x7ff3f402f450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bf7c0ac7d37f01 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:29.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.721+0000 7ff3f9600640 1 --2- 172.21.15.43:0/294240758 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff3e00a3240 0x7ff3e00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:29.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.721+0000 7ff3f1600640 1 -- 172.21.15.43:0/1741563958 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4048070 con 0x557ddefa0330 2024-09-19T05:04:29.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.721+0000 7ff3f1600640 1 -- 172.21.15.43:0/1741563958 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3f4003e50 con 0x557ddefa0330 2024-09-19T05:04:29.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.721+0000 7ff3f1600640 1 -- 172.21.15.43:0/1741563958 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4034040 con 0x557ddefa0330 2024-09-19T05:04:29.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.721+0000 7ff3f9600640 1 -- 172.21.15.43:0/294240758 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ff3ec0cf420 con 0x7ff3fc102d40 2024-09-19T05:04:29.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.721+0000 7ff3fbe00640 1 --2- 172.21.15.43:0/294240758 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff3e00a3240 0x7ff3e00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:29.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.722+0000 7ff3fbe00640 1 --2- 172.21.15.43:0/294240758 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff3e00a3240 0x7ff3e00a5700 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec0047e0 tx=0x7ff3ec0d2e60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:29.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.722+0000 7ff405e6f740 1 -- 172.21.15.43:0/1741563958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ddefa0330 msgr2=0x557ddefa0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:29.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.722+0000 7ff405e6f740 1 --2- 172.21.15.43:0/1741563958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ddefa0330 0x557ddefa0710 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4005e30 tx=0x7ff3f402f450 comp rx=0 tx=0).stop 2024-09-19T05:04:29.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.722+0000 7ff405e6f740 1 -- 172.21.15.43:0/1741563958 shutdown_connections 2024-09-19T05:04:29.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.722+0000 7ff405e6f740 1 --2- 172.21.15.43:0/1741563958 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ddefab510 0x557ddefad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:29.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.722+0000 7ff405e6f740 1 --2- 172.21.15.43:0/1741563958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ddefa0c50 0x557ddefaafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:29.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.722+0000 7ff405e6f740 1 --2- 172.21.15.43:0/1741563958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ddefa0330 0x557ddefa0710 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:29.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.722+0000 7ff405e6f740 1 -- 172.21.15.43:0/1741563958 >> 172.21.15.43:0/1741563958 conn(0x557ddef9c8c0 msgr2=0x557ddef9cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:29.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.722+0000 7ff405e6f740 1 -- 172.21.15.43:0/1741563958 shutdown_connections 2024-09-19T05:04:29.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.722+0000 7ff405e6f740 1 -- 172.21.15.43:0/1741563958 wait complete. 2024-09-19T05:04:29.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.722+0000 7ff405e6f740 1 -- start start 2024-09-19T05:04:29.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.723+0000 7ff405e6f740 1 Processor -- start 2024-09-19T05:04:29.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.723+0000 7ff405e6f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ddefa0330 0x557ddefa85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:29.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.723+0000 7ff405e6f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ddefa0c50 0x557ddefa8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:29.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.723+0000 7ff405e6f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ddefab510 0x557ddefa9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:29.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.723+0000 7ff405e6f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557ddefae060 con 0x557ddefab510 2024-09-19T05:04:29.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.723+0000 7ff405e6f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557ddee8e920 con 0x557ddefa0c50 2024-09-19T05:04:29.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.723+0000 7ff405e6f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557ddefae1e0 con 0x557ddefa0330 2024-09-19T05:04:29.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.723+0000 7ff405e6f740 10 client.0 _ll_register_callbacks cb 0x557ddeeb28a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:04:29.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.723+0000 7ff401200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ddefab510 0x557ddefa9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:29.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.723+0000 7ff3f3400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ddefa0c50 0x557ddefa8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:29.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.723+0000 7ff3f3400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ddefa0c50 0x557ddefa8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:38646/0 (socket says 172.21.15.43:38646) 2024-09-19T05:04:29.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.723+0000 7ff3f3400640 1 -- 172.21.15.43:0/587911197 learned_addr learned my addr 172.21.15.43:0/587911197 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:29.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.723+0000 7ff401200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ddefab510 0x557ddefa9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33520/0 (socket says 172.21.15.43:33520) 2024-09-19T05:04:29.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.724+0000 7ff3f3400640 1 -- 172.21.15.43:0/587911197 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ddefa0330 msgr2=0x557ddefa85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:29.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.724+0000 7ff3f3400640 1 --2- 172.21.15.43:0/587911197 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ddefa0330 0x557ddefa85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:29.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.724+0000 7ff3f3400640 1 -- 172.21.15.43:0/587911197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ddefab510 msgr2=0x557ddefa9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:29.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.724+0000 7ff3f3400640 1 --2- 172.21.15.43:0/587911197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ddefab510 0x557ddefa9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:29.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.724+0000 7ff3f3400640 1 -- 172.21.15.43:0/587911197 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ddec772b0 con 0x557ddefa0c50 2024-09-19T05:04:29.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.724+0000 7ff401200640 1 --2- 172.21.15.43:0/587911197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ddefab510 0x557ddefa9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:04:29.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.724+0000 7ff3f3400640 1 --2- 172.21.15.43:0/587911197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ddefa0c50 0x557ddefa8b00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff3e400bc10 tx=0x7ff3e40039c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f039ce3e05432078 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:29.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.724+0000 7ff3ea000640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:04:29.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.724+0000 7ff3ea000640 1 -- 172.21.15.43:0/587911197 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e400be80 con 0x557ddefa0c50 2024-09-19T05:04:29.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.724+0000 7ff405e6f740 1 -- 172.21.15.43:0/587911197 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3f4003e50 con 0x557ddefa0c50 2024-09-19T05:04:29.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.725+0000 7ff3ea000640 1 -- 172.21.15.43:0/587911197 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ff3e4011650 con 0x557ddefa0c50 2024-09-19T05:04:29.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.725+0000 7ff3ea000640 1 -- 172.21.15.43:0/587911197 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff3e4011850 con 0x557ddefa0c50 2024-09-19T05:04:29.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.725+0000 7ff405e6f740 10 client.25109 fetch_fsmap learned FSMap version 28 2024-09-19T05:04:29.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.725+0000 7ff405e6f740 1 -- 172.21.15.43:0/587911197 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557ddefb9f50 con 0x557ddefa0c50 2024-09-19T05:04:29.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.725+0000 7ff3ea000640 1 -- 172.21.15.43:0/587911197 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3f4003e50 con 0x557ddefa0c50 2024-09-19T05:04:29.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.726+0000 7ff3ea000640 1 -- 172.21.15.43:0/587911197 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff3e401bb20 con 0x557ddefa0c50 2024-09-19T05:04:29.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.726+0000 7ff405e6f740 10 client.25109 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:04:29.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.726+0000 7ff405e6f740 10 client.25109 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T05:04:29.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.726+0000 7ff405e6f740 20 client.25109 populate_metadata read hostname 'smithi043' 2024-09-19T05:04:29.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.726+0000 7ff405e6f740 1 --2- 172.21.15.43:0/587911197 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x557ddefbc5c0 0x557ddefbda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:29.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.726+0000 7ff405e6f740 4 client.25109 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:04:29.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.726+0000 7ff405e6f740 1 -- 172.21.15.43:0/587911197 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ddef13620 con 0x557ddefbc5c0 2024-09-19T05:04:29.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.726+0000 7ff401200640 1 --2- 172.21.15.43:0/587911197 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x557ddefbc5c0 0x557ddefbda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:29.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.726+0000 7ff401200640 1 --2- 172.21.15.43:0/587911197 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x557ddefbc5c0 0x557ddefbda30 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2164f60251479bc server_cookie=ac834f410f8773e in_seq=0 out_seq=0 2024-09-19T05:04:29.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.726+0000 7ff3ea000640 10 client.25109 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:04:29.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.727+0000 7ff3ea000640 1 -- 172.21.15.43:0/587911197 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557ddef13620 con 0x557ddefbc5c0 2024-09-19T05:04:29.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.727+0000 7ff3ea000640 10 client.25109 handle_command_reply: tid=0 2024-09-19T05:04:29.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.727+0000 7ff405e6f740 1 client.25109 shutdown 2024-09-19T05:04:29.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.728+0000 7ff405e6f740 1 -- 172.21.15.43:0/587911197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ddefa0c50 msgr2=0x557ddefa8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:29.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.728+0000 7ff405e6f740 1 --2- 172.21.15.43:0/587911197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ddefa0c50 0x557ddefa8b00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff3e400bc10 tx=0x7ff3e40039c0 comp rx=0 tx=0).stop 2024-09-19T05:04:29.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.728+0000 7ff405e6f740 1 -- 172.21.15.43:0/587911197 shutdown_connections 2024-09-19T05:04:29.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.728+0000 7ff405e6f740 1 --2- 172.21.15.43:0/587911197 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x557ddefbc5c0 0x557ddefbda30 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:29.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.728+0000 7ff405e6f740 1 --2- 172.21.15.43:0/587911197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ddefab510 0x557ddefa9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:29.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.728+0000 7ff405e6f740 1 --2- 172.21.15.43:0/587911197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ddefa0c50 0x557ddefa8b00 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:29.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.728+0000 7ff405e6f740 1 --2- 172.21.15.43:0/587911197 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ddefa0330 0x557ddefa85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:29.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.728+0000 7ff405e6f740 1 -- 172.21.15.43:0/587911197 >> 172.21.15.43:0/587911197 conn(0x557ddef9c8c0 msgr2=0x557ddefa25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:29.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.728+0000 7ff3ea000640 0 client.25109 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:04:29.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.728+0000 7ff405e6f740 1 -- 172.21.15.43:0/587911197 shutdown_connections 2024-09-19T05:04:29.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.728+0000 7ff405e6f740 1 -- 172.21.15.43:0/587911197 wait complete. 2024-09-19T05:04:29.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.728+0000 7ff405e6f740 20 client.25109 trim_cache size 0 max 16384 2024-09-19T05:04:29.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.747+0000 7ff405e6f740 1 Processor -- start 2024-09-19T05:04:29.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.747+0000 7ff405e6f740 1 -- start start 2024-09-19T05:04:29.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.747+0000 7ff405e6f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ddefa0330 0x557ddefa8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:29.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.747+0000 7ff405e6f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ddefa0c50 0x557ddefa5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:29.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.747+0000 7ff405e6f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ddefab510 0x557ddefa5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:29.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.747+0000 7ff405e6f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557ddefbbd70 con 0x557ddefab510 2024-09-19T05:04:29.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.747+0000 7ff405e6f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557ddefbc440 con 0x557ddefa0330 2024-09-19T05:04:29.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.747+0000 7ff405e6f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557ddefbcb10 con 0x557ddefa0c50 2024-09-19T05:04:29.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.747+0000 7ff401200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ddefa0330 0x557ddefa8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:29.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.747+0000 7ff401200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ddefab510 0x557ddefa5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:29.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.747+0000 7ff401200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ddefab510 0x557ddefa5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33534/0 (socket says 172.21.15.43:33534) 2024-09-19T05:04:29.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.747+0000 7ff401200640 1 -- 172.21.15.43:0/4075109469 learned_addr learned my addr 172.21.15.43:0/4075109469 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:29.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.747+0000 7ff3f3400640 1 --2- 172.21.15.43:0/4075109469 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ddefa0c50 0x557ddefa5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:29.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.748+0000 7ff401200640 1 -- 172.21.15.43:0/4075109469 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ddefa0c50 msgr2=0x557ddefa5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:29.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.748+0000 7ff401200640 1 --2- 172.21.15.43:0/4075109469 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ddefa0c50 0x557ddefa5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:29.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.748+0000 7ff401200640 1 -- 172.21.15.43:0/4075109469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ddefab510 msgr2=0x557ddefa5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:29.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.748+0000 7ff401200640 1 --2- 172.21.15.43:0/4075109469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ddefab510 0x557ddefa5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:29.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.748+0000 7ff401200640 1 -- 172.21.15.43:0/4075109469 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ddefaaf30 con 0x557ddefa0330 2024-09-19T05:04:29.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.748+0000 7ff3f3400640 1 --2- 172.21.15.43:0/4075109469 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ddefa0c50 0x557ddefa5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:04:29.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.748+0000 7ff401200640 1 --2- 172.21.15.43:0/4075109469 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ddefa0330 0x557ddefa8e90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff3f402fe60 tx=0x7ff3f402ff20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4a2bca17d93f1a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:29.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.749+0000 7ff3f0c00640 1 -- 172.21.15.43:0/4075109469 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4048040 con 0x557ddefa0330 2024-09-19T05:04:29.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.749+0000 7ff3f0c00640 1 -- 172.21.15.43:0/4075109469 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3f404e7e0 con 0x557ddefa0330 2024-09-19T05:04:29.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.749+0000 7ff3f0c00640 1 -- 172.21.15.43:0/4075109469 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f40309a0 con 0x557ddefa0330 2024-09-19T05:04:29.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.749+0000 7ff405e6f740 1 -- 172.21.15.43:0/4075109469 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ddefa0330 msgr2=0x557ddefa8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:29.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.749+0000 7ff405e6f740 1 --2- 172.21.15.43:0/4075109469 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ddefa0330 0x557ddefa8e90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff3f402fe60 tx=0x7ff3f402ff20 comp rx=0 tx=0).stop 2024-09-19T05:04:29.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.749+0000 7ff405e6f740 1 -- 172.21.15.43:0/4075109469 shutdown_connections 2024-09-19T05:04:29.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.749+0000 7ff405e6f740 1 --2- 172.21.15.43:0/4075109469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ddefab510 0x557ddefa5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:29.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.749+0000 7ff405e6f740 1 --2- 172.21.15.43:0/4075109469 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ddefa0c50 0x557ddefa5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:29.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.749+0000 7ff405e6f740 1 --2- 172.21.15.43:0/4075109469 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ddefa0330 0x557ddefa8e90 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:29.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.749+0000 7ff405e6f740 1 -- 172.21.15.43:0/4075109469 >> 172.21.15.43:0/4075109469 conn(0x557ddef9c8c0 msgr2=0x557ddefa6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:29.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.749+0000 7ff405e6f740 1 -- 172.21.15.43:0/4075109469 shutdown_connections 2024-09-19T05:04:29.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.750+0000 7ff405e6f740 1 -- 172.21.15.43:0/4075109469 wait complete. 2024-09-19T05:04:29.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.750+0000 7ff405e6f740 1 -- start start 2024-09-19T05:04:29.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.750+0000 7ff405e6f740 1 Processor -- start 2024-09-19T05:04:29.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.750+0000 7ff405e6f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ddefa0330 0x557ddefa4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:29.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.751+0000 7ff405e6f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ddefa0c50 0x557ddefa54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:29.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.751+0000 7ff405e6f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ddefab510 0x557ddefc4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:29.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.751+0000 7ff405e6f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557ddefbd8b0 con 0x557ddefab510 2024-09-19T05:04:29.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.751+0000 7ff405e6f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557ddefbd1e0 con 0x557ddefa0c50 2024-09-19T05:04:29.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.751+0000 7ff405e6f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557ddefbdf80 con 0x557ddefa0330 2024-09-19T05:04:29.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.751+0000 7ff405e6f740 10 client.0 _ll_register_callbacks cb 0x7ff3e4004360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:04:29.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.751+0000 7ff401200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ddefab510 0x557ddefc4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:29.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.751+0000 7ff3f3400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ddefa0c50 0x557ddefa54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:29.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.751+0000 7ff401200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ddefa0330 0x557ddefa4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:29.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.751+0000 7ff3f3400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ddefa0c50 0x557ddefa54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:38672/0 (socket says 172.21.15.43:38672) 2024-09-19T05:04:29.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.751+0000 7ff3f3400640 1 -- 172.21.15.43:0/2178704881 learned_addr learned my addr 172.21.15.43:0/2178704881 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:29.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.751+0000 7ff401200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ddefa0330 0x557ddefa4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:37618/0 (socket says 172.21.15.43:37618) 2024-09-19T05:04:29.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.751+0000 7ff3f3400640 1 -- 172.21.15.43:0/2178704881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ddefa0330 msgr2=0x557ddefa4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:29.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.751+0000 7ff3f3400640 1 --2- 172.21.15.43:0/2178704881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ddefa0330 0x557ddefa4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:29.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.751+0000 7ff3f3400640 1 -- 172.21.15.43:0/2178704881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ddefab510 msgr2=0x557ddefc4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:29.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.751+0000 7ff3f3400640 1 --2- 172.21.15.43:0/2178704881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ddefab510 0x557ddefc4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:29.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.751+0000 7ff3f3400640 1 -- 172.21.15.43:0/2178704881 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ddefc5390 con 0x557ddefa0c50 2024-09-19T05:04:29.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.752+0000 7ff3f3400640 1 --2- 172.21.15.43:0/2178704881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ddefa0c50 0x557ddefa54f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x557ddefa7e50 tx=0x7ff3e400bef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=763815585f243697 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:29.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.752+0000 7ff3ea000640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:04:29.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.752+0000 7ff3ea000640 1 -- 172.21.15.43:0/2178704881 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e4035030 con 0x557ddefa0c50 2024-09-19T05:04:29.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.752+0000 7ff3ea000640 1 -- 172.21.15.43:0/2178704881 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ff3e4017040 con 0x557ddefa0c50 2024-09-19T05:04:29.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.752+0000 7ff405e6f740 1 -- 172.21.15.43:0/2178704881 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3f404e7e0 con 0x557ddefa0c50 2024-09-19T05:04:29.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.752+0000 7ff3ea000640 1 -- 172.21.15.43:0/2178704881 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff3e40119f0 con 0x557ddefa0c50 2024-09-19T05:04:29.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.752+0000 7ff405e6f740 10 client.25115 fetch_fsmap learned FSMap version 28 2024-09-19T05:04:29.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.752+0000 7ff405e6f740 1 -- 172.21.15.43:0/2178704881 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557ddefc66c0 con 0x557ddefa0c50 2024-09-19T05:04:29.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.753+0000 7ff3ea000640 1 -- 172.21.15.43:0/2178704881 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3f404e7e0 con 0x557ddefa0c50 2024-09-19T05:04:29.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.753+0000 7ff3ea000640 1 -- 172.21.15.43:0/2178704881 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff3e401b470 con 0x557ddefa0c50 2024-09-19T05:04:29.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.753+0000 7ff405e6f740 10 client.25115 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T05:04:29.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.753+0000 7ff405e6f740 10 client.25115 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T05:04:29.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.753+0000 7ff405e6f740 20 client.25115 populate_metadata read hostname 'smithi043' 2024-09-19T05:04:29.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.753+0000 7ff405e6f740 1 --2- 172.21.15.43:0/2178704881 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x557ddf02b470 0x557ddf02a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:29.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.753+0000 7ff405e6f740 4 client.25115 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T05:04:29.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.753+0000 7ff405e6f740 1 -- 172.21.15.43:0/2178704881 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "status"}) -- 0x557ddef0db30 con 0x557ddf02b470 2024-09-19T05:04:29.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.753+0000 7ff401200640 1 --2- 172.21.15.43:0/2178704881 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x557ddf02b470 0x557ddf02a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:29.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.754+0000 7ff401200640 1 --2- 172.21.15.43:0/2178704881 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x557ddf02b470 0x557ddf02a180 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c06c83472690aaa server_cookie=7a3649bf1b701576 in_seq=0 out_seq=0 2024-09-19T05:04:29.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.754+0000 7ff3ea000640 10 client.25115 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:04:29.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.754+0000 7ff3ea000640 1 -- 172.21.15.43:0/2178704881 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x557ddef0db30 con 0x557ddf02b470 2024-09-19T05:04:29.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.754+0000 7ff3ea000640 10 client.25115 handle_command_reply: tid=0 2024-09-19T05:04:29.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.754+0000 7ff405e6f740 1 client.25115 shutdown 2024-09-19T05:04:29.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.754+0000 7ff405e6f740 1 -- 172.21.15.43:0/2178704881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ddefa0c50 msgr2=0x557ddefa54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:29.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.754+0000 7ff405e6f740 1 --2- 172.21.15.43:0/2178704881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ddefa0c50 0x557ddefa54f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x557ddefa7e50 tx=0x7ff3e400bef0 comp rx=0 tx=0).stop 2024-09-19T05:04:29.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.754+0000 7ff405e6f740 1 -- 172.21.15.43:0/2178704881 shutdown_connections 2024-09-19T05:04:29.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.754+0000 7ff405e6f740 1 --2- 172.21.15.43:0/2178704881 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x557ddf02b470 0x557ddf02a180 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:29.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.754+0000 7ff405e6f740 1 --2- 172.21.15.43:0/2178704881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557ddefab510 0x557ddefc4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:29.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.754+0000 7ff405e6f740 1 --2- 172.21.15.43:0/2178704881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ddefa0c50 0x557ddefa54f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:29.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.754+0000 7ff405e6f740 1 --2- 172.21.15.43:0/2178704881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ddefa0330 0x557ddefa4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:29.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.754+0000 7ff3ea000640 0 client.25115 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:04:29.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.754+0000 7ff405e6f740 1 -- 172.21.15.43:0/2178704881 >> 172.21.15.43:0/2178704881 conn(0x557ddef9c8c0 msgr2=0x557ddefb6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:29.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.755+0000 7ff405e6f740 1 -- 172.21.15.43:0/2178704881 shutdown_connections 2024-09-19T05:04:29.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.755+0000 7ff405e6f740 1 -- 172.21.15.43:0/2178704881 wait complete. 2024-09-19T05:04:29.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.755+0000 7ff405e6f740 20 client.25115 trim_cache size 0 max 16384 2024-09-19T05:04:29.760 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:04:29.760 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "1456a810-7643-11ef-bceb-c7b262605968", 2024-09-19T05:04:29.760 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-09-19T05:04:29.760 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 24542, 2024-09-19T05:04:29.761 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-09-19T05:04:29.761 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-09-19T05:04:29.761 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-09-19T05:04:29.761 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 123.341531898, 2024-09-19T05:04:29.761 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 28, 2024-09-19T05:04:29.761 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 85, 2024-09-19T05:04:29.761 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 83, 2024-09-19T05:04:29.761 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 136.849528472 2024-09-19T05:04:29.761 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:04:29.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.755+0000 7ff403200640 1 -- 172.21.15.43:0/294240758 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff3e00a3240 msgr2=0x7ff3e00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:29.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.755+0000 7ff403200640 1 --2- 172.21.15.43:0/294240758 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff3e00a3240 0x7ff3e00a5700 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec0047e0 tx=0x7ff3ec0d2e60 comp rx=0 tx=0).stop 2024-09-19T05:04:29.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.755+0000 7ff403200640 1 -- 172.21.15.43:0/294240758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3fc102d40 msgr2=0x7ff3fc1009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:29.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.755+0000 7ff403200640 1 --2- 172.21.15.43:0/294240758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3fc102d40 0x7ff3fc1009a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec00d6d0 tx=0x7ff3ec00db90 comp rx=0 tx=0).stop 2024-09-19T05:04:29.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.756+0000 7ff403200640 1 -- 172.21.15.43:0/294240758 shutdown_connections 2024-09-19T05:04:29.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.756+0000 7ff403200640 1 --2- 172.21.15.43:0/294240758 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7ff3e00a3240 0x7ff3e00a5700 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:29.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.756+0000 7ff403200640 1 --2- 172.21.15.43:0/294240758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3fc10c170 0x7ff3fc101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:29.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.756+0000 7ff403200640 1 --2- 172.21.15.43:0/294240758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3fc103660 0x7ff3fc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:29.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.756+0000 7ff403200640 1 --2- 172.21.15.43:0/294240758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3fc102d40 0x7ff3fc1009a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:29.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.756+0000 7ff403200640 1 -- 172.21.15.43:0/294240758 >> 172.21.15.43:0/294240758 conn(0x7ff3fc0fc7a0 msgr2=0x7ff3fc0fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:29.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.756+0000 7ff403200640 1 -- 172.21.15.43:0/294240758 shutdown_connections 2024-09-19T05:04:29.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:29.756+0000 7ff403200640 1 -- 172.21.15.43:0/294240758 wait complete. 2024-09-19T05:04:30.551 INFO:teuthology.run_tasks:Running task sequential... 2024-09-19T05:04:30.561 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-19T05:04:30.561 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-19T05:04:30.562 INFO:teuthology.task.print:Enabling mgr modules 2024-09-19T05:04:30.562 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-09-19T05:04:30.563 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T05:04:30.563 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi043.front.sepia.ceph.com 2024-09-19T05:04:30.563 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-09-19T05:04:30.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:30.719+0000 7f05dfe00640 1 -- 172.21.15.43:0/1001436822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05d8104f50 msgr2=0x7f05d81053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:30.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:30.719+0000 7f05dfe00640 1 --2- 172.21.15.43:0/1001436822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05d8104f50 0x7f05d81053d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f05c4009900 tx=0x7f05c402a410 comp rx=0 tx=0).stop 2024-09-19T05:04:30.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:30.719+0000 7f05dfe00640 1 -- 172.21.15.43:0/1001436822 shutdown_connections 2024-09-19T05:04:30.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:30.719+0000 7f05dfe00640 1 --2- 172.21.15.43:0/1001436822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05d8105910 0x7f05d810c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:30.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:30.719+0000 7f05dfe00640 1 --2- 172.21.15.43:0/1001436822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05d8104f50 0x7f05d81053d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:30.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:30.719+0000 7f05dfe00640 1 --2- 172.21.15.43:0/1001436822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05d8103d60 0x7f05d8104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:30.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:30.719+0000 7f05dfe00640 1 -- 172.21.15.43:0/1001436822 >> 172.21.15.43:0/1001436822 conn(0x7f05d80ff4e0 msgr2=0x7f05d8101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:30.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:30.719+0000 7f05dfe00640 1 -- 172.21.15.43:0/1001436822 shutdown_connections 2024-09-19T05:04:30.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:30.719+0000 7f05dfe00640 1 -- 172.21.15.43:0/1001436822 wait complete. 2024-09-19T05:04:30.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:30.720+0000 7f05dfe00640 1 Processor -- start 2024-09-19T05:04:30.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:30.720+0000 7f05dfe00640 1 -- start start 2024-09-19T05:04:30.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:30.720+0000 7f05dfe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05d8103d60 0x7f05d8199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:30.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:30.720+0000 7f05dfe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05d8104f50 0x7f05d819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:30.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:30.721+0000 7f05dfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05d8105910 0x7f05d81a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:30.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:30.721+0000 7f05dfe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f05d810fa30 con 0x7f05d8105910 2024-09-19T05:04:30.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:30.721+0000 7f05dfe00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f05d810f8b0 con 0x7f05d8104f50 2024-09-19T05:04:30.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:30.721+0000 7f05dfe00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f05d810fbb0 con 0x7f05d8103d60 2024-09-19T05:04:30.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:30.721+0000 7f05dde00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05d8105910 0x7f05d81a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:30.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:30.721+0000 7f05dde00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05d8105910 0x7f05d81a1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33560/0 (socket says 172.21.15.43:33560) 2024-09-19T05:04:30.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:30.721+0000 7f05dde00640 1 -- 172.21.15.43:0/453549217 learned_addr learned my addr 172.21.15.43:0/453549217 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:30.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:30.721+0000 7f05dd400640 1 --2- 172.21.15.43:0/453549217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05d8103d60 0x7f05d8199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:30.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:30.721+0000 7f05dca00640 1 --2- 172.21.15.43:0/453549217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05d8104f50 0x7f05d819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:30.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:30.721+0000 7f05dde00640 1 -- 172.21.15.43:0/453549217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05d8103d60 msgr2=0x7f05d8199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:30.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:30.721+0000 7f05dde00640 1 --2- 172.21.15.43:0/453549217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05d8103d60 0x7f05d8199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:30.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:30.721+0000 7f05dde00640 1 -- 172.21.15.43:0/453549217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05d8104f50 msgr2=0x7f05d819a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:30.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:30.721+0000 7f05dde00640 1 --2- 172.21.15.43:0/453549217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05d8104f50 0x7f05d819a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:30.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:30.721+0000 7f05dde00640 1 -- 172.21.15.43:0/453549217 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05d81a1bc0 con 0x7f05d8105910 2024-09-19T05:04:30.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:30.721+0000 7f05dd400640 1 --2- 172.21.15.43:0/453549217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05d8103d60 0x7f05d8199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:04:30.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:30.721+0000 7f05dca00640 1 --2- 172.21.15.43:0/453549217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05d8104f50 0x7f05d819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:04:30.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:30.721+0000 7f05dde00640 1 --2- 172.21.15.43:0/453549217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05d8105910 0x7f05d81a1550 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f05d000eb10 tx=0x7f05d000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:30.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:30.722+0000 7f05d6000640 1 -- 172.21.15.43:0/453549217 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d0004b20 con 0x7f05d8105910 2024-09-19T05:04:30.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:30.722+0000 7f05d6000640 1 -- 172.21.15.43:0/453549217 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05d0004cc0 con 0x7f05d8105910 2024-09-19T05:04:30.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:30.722+0000 7f05d6000640 1 -- 172.21.15.43:0/453549217 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d0010760 con 0x7f05d8105910 2024-09-19T05:04:30.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:30.722+0000 7f05dfe00640 1 -- 172.21.15.43:0/453549217 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05d81a1eb0 con 0x7f05d8105910 2024-09-19T05:04:30.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:30.722+0000 7f05dfe00640 1 -- 172.21.15.43:0/453549217 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05d81a2370 con 0x7f05d8105910 2024-09-19T05:04:30.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:30.723+0000 7f05dfe00640 1 -- 172.21.15.43:0/453549217 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05d810b080 con 0x7f05d8105910 2024-09-19T05:04:30.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:30.725+0000 7f05d6000640 1 -- 172.21.15.43:0/453549217 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f05d0005020 con 0x7f05d8105910 2024-09-19T05:04:30.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:30.727+0000 7f05d6000640 1 --2- 172.21.15.43:0/453549217 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f05c00a3370 0x7f05c00a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:30.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:30.727+0000 7f05d6000640 1 -- 172.21.15.43:0/453549217 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f05d00cffb0 con 0x7f05d8105910 2024-09-19T05:04:30.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:30.727+0000 7f05dd400640 1 --2- 172.21.15.43:0/453549217 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f05c00a3370 0x7f05c00a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:30.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:30.728+0000 7f05dd400640 1 --2- 172.21.15.43:0/453549217 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f05c00a3370 0x7f05c00a5830 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f05cc0058c0 tx=0x7f05cc005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:30.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:30.731+0000 7f05d6000640 1 -- 172.21.15.43:0/453549217 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f05d009cc90 con 0x7f05d8105910 2024-09-19T05:04:30.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:30.967+0000 7f05dfe00640 1 -- 172.21.15.43:0/453549217 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f05d8062e90 con 0x7f05d8105910 2024-09-19T05:04:31.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:30 smithi096 ceph-mon[35085]: pgmap v356: 97 pgs: 97 active+clean; 646 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:04:31.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:30 smithi043 ceph-mon[30800]: pgmap v356: 97 pgs: 97 active+clean; 646 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:04:31.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:30 smithi151 ceph-mon[35425]: pgmap v356: 97 pgs: 97 active+clean; 646 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:04:31.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:31.935+0000 7f05d6000640 1 -- 172.21.15.43:0/453549217 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f05d009c590 con 0x7f05d8105910 2024-09-19T05:04:31.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:31.938+0000 7f05dfe00640 1 -- 172.21.15.43:0/453549217 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f05c00a3370 msgr2=0x7f05c00a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:31.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:31.938+0000 7f05dfe00640 1 --2- 172.21.15.43:0/453549217 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f05c00a3370 0x7f05c00a5830 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f05cc0058c0 tx=0x7f05cc005850 comp rx=0 tx=0).stop 2024-09-19T05:04:31.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:31.938+0000 7f05dfe00640 1 -- 172.21.15.43:0/453549217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05d8105910 msgr2=0x7f05d81a1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:31.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:31.938+0000 7f05dfe00640 1 --2- 172.21.15.43:0/453549217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05d8105910 0x7f05d81a1550 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f05d000eb10 tx=0x7f05d000c2d0 comp rx=0 tx=0).stop 2024-09-19T05:04:31.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:31.938+0000 7f05dfe00640 1 -- 172.21.15.43:0/453549217 shutdown_connections 2024-09-19T05:04:31.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:31.938+0000 7f05dfe00640 1 --2- 172.21.15.43:0/453549217 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7f05c00a3370 0x7f05c00a5830 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:31.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:31.938+0000 7f05dfe00640 1 --2- 172.21.15.43:0/453549217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05d8105910 0x7f05d81a1550 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:31.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:31.938+0000 7f05dfe00640 1 --2- 172.21.15.43:0/453549217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05d8104f50 0x7f05d819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:31.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:31.938+0000 7f05dfe00640 1 --2- 172.21.15.43:0/453549217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05d8103d60 0x7f05d8199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:31.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:31.938+0000 7f05dfe00640 1 -- 172.21.15.43:0/453549217 >> 172.21.15.43:0/453549217 conn(0x7f05d80ff4e0 msgr2=0x7f05d8108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:31.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:31.939+0000 7f05dfe00640 1 -- 172.21.15.43:0/453549217 shutdown_connections 2024-09-19T05:04:31.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:31.939+0000 7f05dfe00640 1 -- 172.21.15.43:0/453549217 wait complete. 2024-09-19T05:04:31.966 INFO:teuthology.run_tasks:Running task kclient... 2024-09-19T05:04:31.991 INFO:tasks.kclient:Mounting kernel clients... 2024-09-19T05:04:31.991 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-19T05:04:31.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:04:31.991 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-09-19T05:04:32.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:04:32.031 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link delete ceph-brx 2024-09-19T05:04:32.132 INFO:teuthology.orchestra.run.smithi043.stderr:Cannot find device "ceph-brx" 2024-09-19T05:04:32.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T05:04:32.135 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-09-19T05:04:32.135 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-19T05:04:32.135 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T05:04:32.136 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-19T05:04:32.184 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:31 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/453549217' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-19T05:04:32.185 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:32 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: ignoring --setuser ceph since I am not root 2024-09-19T05:04:32.185 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:32 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: ignoring --setgroup ceph since I am not root 2024-09-19T05:04:32.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:31 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/453549217' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-19T05:04:32.223 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 05:04:31 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: ignoring --setuser ceph since I am not root 2024-09-19T05:04:32.223 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 05:04:31 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: ignoring --setgroup ceph since I am not root 2024-09-19T05:04:32.223 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 05:04:31 smithi096 ceph-mgr[36455]: -- 172.21.15.96:0/1936007056 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559bea9bd0e0 con 0x559bea99b000 2024-09-19T05:04:32.223 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 05:04:32 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T05:04:32.088+0000 7feabc579040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T05:04:32.223 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 05:04:32 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T05:04:32.145+0000 7feabc579040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T05:04:32.291 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:31 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/453549217' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-19T05:04:32.291 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 05:04:31 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: ignoring --setuser ceph since I am not root 2024-09-19T05:04:32.291 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 05:04:31 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: ignoring --setgroup ceph since I am not root 2024-09-19T05:04:32.291 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 05:04:31 smithi151 ceph-mgr[37240]: -- 172.21.15.151:0/561335313 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e88b0710e0 con 0x55e88b04f400 2024-09-19T05:04:32.291 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 05:04:32 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T05:04:32.088+0000 7f824ac73040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T05:04:32.291 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 05:04:32 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T05:04:32.144+0000 7f824ac73040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T05:04:32.292 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 05:04:32 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T05:04:32.289+0000 7f824ac73040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T05:04:32.479 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:32 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:04:32.197+0000 7f829c41a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T05:04:32.480 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:32 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:04:32.271+0000 7f829c41a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T05:04:32.527 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:32.647 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 05:04:32 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T05:04:32.356+0000 7f824ac73040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T05:04:32.647 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 05:04:32 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T05:04:32.506+0000 7f824ac73040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T05:04:32.651 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 05:04:32 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T05:04:32.293+0000 7feabc579040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T05:04:32.651 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 05:04:32 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T05:04:32.358+0000 7feabc579040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T05:04:32.651 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 05:04:32 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T05:04:32.507+0000 7feabc579040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T05:04:32.651 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 05:04:32 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T05:04:32.649+0000 7feabc579040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T05:04:32.754 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:32 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:04:32.477+0000 7f829c41a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T05:04:32.754 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:32 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:04:32.565+0000 7f829c41a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T05:04:32.904 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 05:04:32 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T05:04:32.646+0000 7f824ac73040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T05:04:33.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:32 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/453549217' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-19T05:04:33.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:32 smithi043 ceph-mon[30800]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-19T05:04:33.031 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:32 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:04:32.751+0000 7f829c41a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T05:04:33.031 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:32 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:04:32.927+0000 7f829c41a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T05:04:33.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.147+0000 7fae98c00640 1 -- 172.21.15.43:0/112198494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae94103ff0 msgr2=0x7fae941043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:33.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.147+0000 7fae98c00640 1 --2- 172.21.15.43:0/112198494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae94103ff0 0x7fae941043d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fae84009860 tx=0x7fae8402a340 comp rx=0 tx=0).stop 2024-09-19T05:04:33.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.147+0000 7fae98c00640 1 -- 172.21.15.43:0/112198494 shutdown_connections 2024-09-19T05:04:33.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.147+0000 7fae98c00640 1 --2- 172.21.15.43:0/112198494 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae94109120 0x7fae94110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:33.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.147+0000 7fae98c00640 1 --2- 172.21.15.43:0/112198494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae941049a0 0x7fae941089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:33.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.147+0000 7fae98c00640 1 --2- 172.21.15.43:0/112198494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae94103ff0 0x7fae941043d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:33.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.147+0000 7fae98c00640 1 -- 172.21.15.43:0/112198494 >> 172.21.15.43:0/112198494 conn(0x7fae940ffa90 msgr2=0x7fae94101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:33.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.147+0000 7fae98c00640 1 -- 172.21.15.43:0/112198494 shutdown_connections 2024-09-19T05:04:33.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.147+0000 7fae98c00640 1 -- 172.21.15.43:0/112198494 wait complete. 2024-09-19T05:04:33.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:32 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/453549217' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-19T05:04:33.306 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:32 smithi096 ceph-mon[35085]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-19T05:04:33.307 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:32 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/453549217' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-19T05:04:33.307 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:32 smithi151 ceph-mon[35425]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-19T05:04:33.307 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 05:04:32 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T05:04:32.959+0000 7feabc579040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T05:04:33.307 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 05:04:33 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T05:04:33.019+0000 7feabc579040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T05:04:33.307 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 05:04:32 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T05:04:32.959+0000 7f824ac73040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T05:04:33.307 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 05:04:33 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T05:04:33.019+0000 7f824ac73040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T05:04:33.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.148+0000 7fae98c00640 1 Processor -- start 2024-09-19T05:04:33.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.148+0000 7fae98c00640 1 -- start start 2024-09-19T05:04:33.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.148+0000 7fae98c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae94103ff0 0x7fae9419a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:33.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.149+0000 7fae98c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae941049a0 0x7fae9419ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:33.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.149+0000 7fae98c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae94109120 0x7fae9419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:33.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.149+0000 7fae98c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fae94114120 con 0x7fae94103ff0 2024-09-19T05:04:33.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.149+0000 7fae98c00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fae94113fa0 con 0x7fae94109120 2024-09-19T05:04:33.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.149+0000 7fae98c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fae941142a0 con 0x7fae941049a0 2024-09-19T05:04:33.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.149+0000 7fae91e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae94103ff0 0x7fae9419a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:33.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.149+0000 7fae92800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae94109120 0x7fae9419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:33.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.149+0000 7fae92800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae94109120 0x7fae9419eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:38706/0 (socket says 172.21.15.43:38706) 2024-09-19T05:04:33.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.149+0000 7fae92800640 1 -- 172.21.15.43:0/3072277931 learned_addr learned my addr 172.21.15.43:0/3072277931 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:33.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.149+0000 7fae91400640 1 --2- 172.21.15.43:0/3072277931 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae941049a0 0x7fae9419ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:33.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.149+0000 7fae92800640 1 -- 172.21.15.43:0/3072277931 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae941049a0 msgr2=0x7fae9419ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:33.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.149+0000 7fae92800640 1 --2- 172.21.15.43:0/3072277931 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae941049a0 0x7fae9419ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:33.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.149+0000 7fae92800640 1 -- 172.21.15.43:0/3072277931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae94103ff0 msgr2=0x7fae9419a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:33.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.149+0000 7fae92800640 1 --2- 172.21.15.43:0/3072277931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae94103ff0 0x7fae9419a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:33.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.149+0000 7fae92800640 1 -- 172.21.15.43:0/3072277931 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae9419f590 con 0x7fae94109120 2024-09-19T05:04:33.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.149+0000 7fae91400640 1 --2- 172.21.15.43:0/3072277931 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae941049a0 0x7fae9419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:04:33.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.150+0000 7fae92800640 1 --2- 172.21.15.43:0/3072277931 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae94109120 0x7fae9419eeb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fae8c00eb10 tx=0x7fae8c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:33.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.150+0000 7fae8aa00640 1 -- 172.21.15.43:0/3072277931 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c00ce20 con 0x7fae94109120 2024-09-19T05:04:33.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.150+0000 7fae8aa00640 1 -- 172.21.15.43:0/3072277931 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae8c005020 con 0x7fae94109120 2024-09-19T05:04:33.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.150+0000 7fae8aa00640 1 -- 172.21.15.43:0/3072277931 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c010690 con 0x7fae94109120 2024-09-19T05:04:33.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.150+0000 7fae98c00640 1 -- 172.21.15.43:0/3072277931 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae9419f880 con 0x7fae94109120 2024-09-19T05:04:33.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.150+0000 7fae98c00640 1 -- 172.21.15.43:0/3072277931 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae941a7160 con 0x7fae94109120 2024-09-19T05:04:33.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.151+0000 7fae98c00640 1 -- 172.21.15.43:0/3072277931 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae941043d0 con 0x7fae94109120 2024-09-19T05:04:33.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.153+0000 7fae8aa00640 1 -- 172.21.15.43:0/3072277931 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 138372+0+0 (secure 0 0 0) 0x7fae8c005540 con 0x7fae94109120 2024-09-19T05:04:33.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.154+0000 7fae8aa00640 1 --2- 172.21.15.43:0/3072277931 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fae780a3310 0x7fae780a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:33.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.155+0000 7fae91e00640 1 -- 172.21.15.43:0/3072277931 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fae780a3310 msgr2=0x7fae780a57d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/3167108925 2024-09-19T05:04:33.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.155+0000 7fae8aa00640 1 -- 172.21.15.43:0/3072277931 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fae8c0cfe50 con 0x7fae94109120 2024-09-19T05:04:33.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.155+0000 7fae91e00640 1 --2- 172.21.15.43:0/3072277931 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fae780a3310 0x7fae780a57d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T05:04:33.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.158+0000 7fae8aa00640 1 -- 172.21.15.43:0/3072277931 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fae8c09cb30 con 0x7fae94109120 2024-09-19T05:04:33.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.356+0000 7fae91e00640 1 -- 172.21.15.43:0/3072277931 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fae780a3310 msgr2=0x7fae780a57d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/3167108925 2024-09-19T05:04:33.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.356+0000 7fae91e00640 1 --2- 172.21.15.43:0/3072277931 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fae780a3310 0x7fae780a57d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T05:04:33.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.383+0000 7fae98c00640 1 -- 172.21.15.43:0/3072277931 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fae9402d7c0 con 0x7fae94109120 2024-09-19T05:04:33.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.384+0000 7fae8aa00640 1 -- 172.21.15.43:0/3072277931 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) ==== 53+0+73 (secure 0 0 0) 0x7fae8c09c430 con 0x7fae94109120 2024-09-19T05:04:33.387 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T05:04:33.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.386+0000 7fae98c00640 1 -- 172.21.15.43:0/3072277931 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fae780a3310 msgr2=0x7fae780a57d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T05:04:33.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.386+0000 7fae98c00640 1 --2- 172.21.15.43:0/3072277931 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fae780a3310 0x7fae780a57d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:33.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.386+0000 7fae98c00640 1 -- 172.21.15.43:0/3072277931 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae94109120 msgr2=0x7fae9419eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:33.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.386+0000 7fae98c00640 1 --2- 172.21.15.43:0/3072277931 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae94109120 0x7fae9419eeb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fae8c00eb10 tx=0x7fae8c00ef10 comp rx=0 tx=0).stop 2024-09-19T05:04:33.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.386+0000 7fae98c00640 1 -- 172.21.15.43:0/3072277931 shutdown_connections 2024-09-19T05:04:33.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.386+0000 7fae98c00640 1 --2- 172.21.15.43:0/3072277931 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fae780a3310 0x7fae780a57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:33.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.386+0000 7fae98c00640 1 --2- 172.21.15.43:0/3072277931 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae94109120 0x7fae9419eeb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:33.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.386+0000 7fae98c00640 1 --2- 172.21.15.43:0/3072277931 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae941049a0 0x7fae9419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:33.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.386+0000 7fae98c00640 1 --2- 172.21.15.43:0/3072277931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae94103ff0 0x7fae9419a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:33.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.386+0000 7fae98c00640 1 -- 172.21.15.43:0/3072277931 >> 172.21.15.43:0/3072277931 conn(0x7fae940ffa90 msgr2=0x7fae94100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:33.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.386+0000 7fae98c00640 1 -- 172.21.15.43:0/3072277931 shutdown_connections 2024-09-19T05:04:33.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:33.386+0000 7fae98c00640 1 -- 172.21.15.43:0/3072277931 wait complete. 2024-09-19T05:04:33.475 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 05:04:33 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T05:04:33.204+0000 7feabc579040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T05:04:33.621 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:33 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:04:33.316+0000 7f829c41a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T05:04:33.621 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:33 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:04:33.394+0000 7f829c41a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T05:04:33.621 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:33 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:04:33.618+0000 7f829c41a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T05:04:33.655 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 05:04:33 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T05:04:33.202+0000 7f824ac73040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T05:04:34.194 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:33 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/3072277931' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T05:04:34.198 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T05:04:34.198 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T05:04:34.198 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-09-19T05:04:34.198 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T05:04:34.198 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T05:04:34.198 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T05:04:34.198 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T05:04:34.198 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T05:04:34.198 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T05:04:34.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:04:34.199 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-09-19T05:04:34.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:33 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/3072277931' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T05:04:34.223 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 05:04:34 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T05:04:34.078+0000 7feabc579040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T05:04:34.223 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 05:04:34 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T05:04:34.144+0000 7feabc579040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T05:04:34.237 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T05:04:34.237 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T05:04:34.237 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T05:04:34.237 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-19T05:04:34.238 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-09-19T05:04:34.238 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-19T05:04:34.238 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T05:04:34.238 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-09-19T05:04:34.238 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-09-19T05:04:34.238 INFO:teuthology.orchestra.run.smithi043.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T05:04:34.238 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-09-19T05:04:34.238 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-09-19T05:04:34.238 INFO:teuthology.orchestra.run.smithi043.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T05:04:34.238 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-09-19T05:04:34.239 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-09-19T05:04:34.239 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-19T05:04:34.239 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 42123sec preferred_lft 42123sec 2024-09-19T05:04:34.239 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2024-09-19T05:04:34.239 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-19T05:04:34.239 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T05:04:34.239 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-09-19T05:04:34.239 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-09-19T05:04:34.240 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-19T05:04:34.240 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T05:04:34.240 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-19T05:04:34.240 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link add name ceph-brx type bridge 2024-09-19T05:04:34.240 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip addr flush dev ceph-brx 2024-09-19T05:04:34.241 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set ceph-brx up 2024-09-19T05:04:34.241 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-19T05:04:34.241 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-19T05:04:34.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T05:04:34.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:33 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/3072277931' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T05:04:34.405 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 05:04:34 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T05:04:34.079+0000 7f824ac73040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T05:04:34.405 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 05:04:34 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T05:04:34.145+0000 7f824ac73040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T05:04:34.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T05:04:34.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:04:34.671 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T05:04:34.729 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-09-19T05:04:34.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:04:34.733 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-09-19T05:04:34.784 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-09-19T05:04:34.784 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T05:04:34.784 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-19T05:04:34.784 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-19T05:04:34.784 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T05:04:34.785 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T05:04:34.785 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-19T05:04:34.785 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-19T05:04:34.785 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-19T05:04:34.785 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-19T05:04:34.785 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-19T05:04:34.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T05:04:34.882 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:34 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:04:34.795+0000 7f829c41a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T05:04:35.107 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 05:04:34 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T05:04:34.847+0000 7feabc579040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T05:04:35.108 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 05:04:34 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T05:04:34.961+0000 7feabc579040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T05:04:35.108 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 05:04:34 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T05:04:34.851+0000 7f824ac73040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T05:04:35.108 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 05:04:34 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T05:04:34.965+0000 7f824ac73040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T05:04:35.135 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:34 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:04:34.883+0000 7f829c41a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T05:04:35.383 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 05:04:35 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T05:04:35.105+0000 7feabc579040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T05:04:35.383 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 05:04:35 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T05:04:35.165+0000 7feabc579040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T05:04:35.383 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 05:04:35 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T05:04:35.312+0000 7feabc579040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T05:04:35.383 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 05:04:35 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T05:04:35.380+0000 7feabc579040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T05:04:35.384 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 05:04:35 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T05:04:35.107+0000 7f824ac73040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T05:04:35.384 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 05:04:35 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T05:04:35.168+0000 7f824ac73040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T05:04:35.384 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 05:04:35 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T05:04:35.316+0000 7f824ac73040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T05:04:35.384 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 05:04:35 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T05:04:35.382+0000 7f824ac73040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T05:04:35.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T05:04:35.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:04:35.400 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-09-19T05:04:35.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:04:35.437 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list-id 2024-09-19T05:04:35.512 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T05:04:35.512 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-19T05:04:35.513 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T05:04:35.513 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-19T05:04:35.513 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-19T05:04:35.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T05:04:35.655 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 05:04:35 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T05:04:35.442+0000 7f824ac73040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T05:04:35.655 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 05:04:35 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T05:04:35.600+0000 7f824ac73040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T05:04:35.667 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 05:04:35 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T05:04:35.440+0000 7feabc579040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T05:04:35.667 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 05:04:35 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T05:04:35.599+0000 7feabc579040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T05:04:35.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T05:04:35.695 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-19T05:04:35.695 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T05:04:35.695 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-19T05:04:35.695 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-19T05:04:35.695 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-19T05:04:35.696 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-19T05:04:35.696 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-19T05:04:35.696 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-19T05:04:35.696 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-19T05:04:35.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T05:04:35.901 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:35 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:04:35.872+0000 7f829c41a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T05:04:35.931 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 05:04:35 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T05:04:35.665+0000 7f824ac73040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T05:04:35.931 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 05:04:35 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T05:04:35.732+0000 7f824ac73040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T05:04:35.931 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 05:04:35 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T05:04:35.813+0000 7f824ac73040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T05:04:35.931 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 05:04:35 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T05:04:35.871+0000 7f824ac73040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T05:04:35.931 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 05:04:35 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T05:04:35.929+0000 7f824ac73040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T05:04:35.935 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 05:04:35 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T05:04:35.664+0000 7feabc579040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T05:04:35.935 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 05:04:35 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T05:04:35.732+0000 7feabc579040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T05:04:35.935 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 05:04:35 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T05:04:35.814+0000 7feabc579040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T05:04:35.935 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 05:04:35 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T05:04:35.874+0000 7feabc579040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T05:04:35.935 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 05:04:35 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T05:04:35.933+0000 7feabc579040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T05:04:35.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T05:04:35.946 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T05:04:35.946 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-19T05:04:35.946 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-09-19T05:04:35.946 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set dev brx.0 master ceph-brx 2024-09-19T05:04:35.946 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-19T05:04:36.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T05:04:36.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T05:04:36.117 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T05:04:36.117 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T05:04:36.153 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T05:04:36.153 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T05:04:36.153 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T05:04:36.201 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:36 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:04:36.021+0000 7f829c41a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T05:04:36.228 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-19T05:04:36.228 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-19T05:04:36.228 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/0bec0f8f-1c83-4286-a8b9-aa1bb4d77684 2024-09-19T05:04:36.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:04:36.229 DEBUG:teuthology.orchestra.run.smithi043:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/0bec0f8f-1c83-4286-a8b9-aa1bb4d77684 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-09-19T05:04:36.376 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-09-19T05:04:36.377 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2024-09-19T05:04:36.463 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:36 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:04:36.198+0000 7f829c41a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T05:04:36.463 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:36 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:04:36.275+0000 7f829c41a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T05:04:36.762 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:36 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:04:36.460+0000 7f829c41a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T05:04:36.762 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:36 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:04:36.548+0000 7f829c41a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T05:04:36.762 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:36 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:04:36.624+0000 7f829c41a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T05:04:37.044 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:36 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:04:36.824+0000 7f829c41a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T05:04:37.044 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:36 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:04:36.906+0000 7f829c41a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T05:04:37.044 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:36 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:04:36.990+0000 7f829c41a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T05:04:37.044 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:36 smithi043 ceph-mon[30800]: Standby manager daemon z restarted 2024-09-19T05:04:37.044 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:36 smithi043 ceph-mon[30800]: Standby manager daemon z started 2024-09-19T05:04:37.045 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:36 smithi043 ceph-mon[30800]: from='mgr.? 172.21.15.151:0/3757189228' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T05:04:37.045 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:36 smithi043 ceph-mon[30800]: from='mgr.? 172.21.15.151:0/3757189228' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T05:04:37.045 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:36 smithi043 ceph-mon[30800]: from='mgr.? 172.21.15.151:0/3757189228' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T05:04:37.045 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:36 smithi043 ceph-mon[30800]: from='mgr.? 172.21.15.151:0/3757189228' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T05:04:37.045 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:36 smithi043 ceph-mon[30800]: Standby manager daemon y restarted 2024-09-19T05:04:37.045 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:36 smithi043 ceph-mon[30800]: Standby manager daemon y started 2024-09-19T05:04:37.045 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:36 smithi043 ceph-mon[30800]: from='mgr.? 172.21.15.96:0/1690296854' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T05:04:37.045 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:36 smithi043 ceph-mon[30800]: from='mgr.? 172.21.15.96:0/1690296854' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T05:04:37.045 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:36 smithi043 ceph-mon[30800]: from='mgr.? 172.21.15.96:0/1690296854' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T05:04:37.045 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:36 smithi043 ceph-mon[30800]: from='mgr.? 172.21.15.96:0/1690296854' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T05:04:37.061 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-09-19T05:04:37.061 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2024-09-19T05:04:37.061 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:6789,172.21.15.96:6789,172.21.15.151:6789" 2024-09-19T05:04:37.061 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: trying mount with new device syntax: 0@1456a810-7643-11ef-bceb-c7b262605968.cephfs=/volumes/qa/sv_1/0bec0f8f-1c83-4286-a8b9-aa1bb4d77684 2024-09-19T05:04:37.061 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync,name=0,key=0,mon_addr=172.21.15.43:6789/172.21.15.96:6789/172.21.15.151:6789" will pass to kernel 2024-09-19T05:04:37.064 INFO:teuthology.orchestra.run.smithi043.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-19T05:04:37.064 INFO:teuthology.orchestra.run.smithi043.stdout: You just mounted a file system that supports labels which does not 2024-09-19T05:04:37.064 INFO:teuthology.orchestra.run.smithi043.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-19T05:04:37.064 INFO:teuthology.orchestra.run.smithi043.stdout: applications will generate AVC messages and not be allowed access to 2024-09-19T05:04:37.064 INFO:teuthology.orchestra.run.smithi043.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-19T05:04:37.068 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-19T05:04:37.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:04:37.069 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T05:04:37.128 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T05:04:37.155 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 05:04:36 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-19T05:04:36.715+0000 7f824ac73040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T05:04:37.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:36 smithi151 ceph-mon[35425]: Standby manager daemon z restarted 2024-09-19T05:04:37.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:36 smithi151 ceph-mon[35425]: Standby manager daemon z started 2024-09-19T05:04:37.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:36 smithi151 ceph-mon[35425]: from='mgr.? 172.21.15.151:0/3757189228' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T05:04:37.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:36 smithi151 ceph-mon[35425]: from='mgr.? 172.21.15.151:0/3757189228' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T05:04:37.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:36 smithi151 ceph-mon[35425]: from='mgr.? 172.21.15.151:0/3757189228' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T05:04:37.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:36 smithi151 ceph-mon[35425]: from='mgr.? 172.21.15.151:0/3757189228' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T05:04:37.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:36 smithi151 ceph-mon[35425]: Standby manager daemon y restarted 2024-09-19T05:04:37.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:36 smithi151 ceph-mon[35425]: Standby manager daemon y started 2024-09-19T05:04:37.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:36 smithi151 ceph-mon[35425]: from='mgr.? 172.21.15.96:0/1690296854' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T05:04:37.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:36 smithi151 ceph-mon[35425]: from='mgr.? 172.21.15.96:0/1690296854' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T05:04:37.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:36 smithi151 ceph-mon[35425]: from='mgr.? 172.21.15.96:0/1690296854' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T05:04:37.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:36 smithi151 ceph-mon[35425]: from='mgr.? 172.21.15.96:0/1690296854' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T05:04:37.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T05:04:37.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T05:04:37.223 INFO:teuthology.orchestra.run.smithi043.stdout:client25145 2024-09-19T05:04:37.223 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-19T05:04:37.223 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T05:04:37.224 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 05:04:36 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y[36451]: 2024-09-19T05:04:36.722+0000 7feabc579040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T05:04:37.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:36 smithi096 ceph-mon[35085]: Standby manager daemon z restarted 2024-09-19T05:04:37.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:36 smithi096 ceph-mon[35085]: Standby manager daemon z started 2024-09-19T05:04:37.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:36 smithi096 ceph-mon[35085]: from='mgr.? 172.21.15.151:0/3757189228' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T05:04:37.225 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:36 smithi096 ceph-mon[35085]: from='mgr.? 172.21.15.151:0/3757189228' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T05:04:37.225 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:36 smithi096 ceph-mon[35085]: from='mgr.? 172.21.15.151:0/3757189228' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T05:04:37.225 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:36 smithi096 ceph-mon[35085]: from='mgr.? 172.21.15.151:0/3757189228' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T05:04:37.225 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:36 smithi096 ceph-mon[35085]: Standby manager daemon y restarted 2024-09-19T05:04:37.225 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:36 smithi096 ceph-mon[35085]: Standby manager daemon y started 2024-09-19T05:04:37.225 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:36 smithi096 ceph-mon[35085]: from='mgr.? 172.21.15.96:0/1690296854' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T05:04:37.225 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:36 smithi096 ceph-mon[35085]: from='mgr.? 172.21.15.96:0/1690296854' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T05:04:37.225 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:36 smithi096 ceph-mon[35085]: from='mgr.? 172.21.15.96:0/1690296854' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T05:04:37.225 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:36 smithi096 ceph-mon[35085]: from='mgr.? 172.21.15.96:0/1690296854' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T05:04:37.261 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T05:04:37.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T05:04:37.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T05:04:37.394 INFO:teuthology.orchestra.run.smithi043.stdout:client25145 2024-09-19T05:04:37.394 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T05:04:37.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T05:04:37.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T05:04:37.487 INFO:teuthology.orchestra.run.smithi043.stdout:client25145 2024-09-19T05:04:37.487 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/1456a810-7643-11ef-bceb-c7b262605968.client25145/status') 2024-09-19T05:04:37.530 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:37 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:04:37.100+0000 7f829c41a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T05:04:37.530 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:37 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:04:37.177+0000 7f829c41a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T05:04:37.530 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:37 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:04:37.252+0000 7f829c41a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T05:04:37.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T05:04:37.588 INFO:teuthology.orchestra.run.smithi043.stdout:instance: client.25145 (3)172.21.15.43:0/3338542334 2024-09-19T05:04:37.588 INFO:teuthology.orchestra.run.smithi043.stdout:blocklisted: false 2024-09-19T05:04:37.588 INFO:teuthology.orchestra.run.smithi043.stderr:0+1 records in 2024-09-19T05:04:37.589 INFO:teuthology.orchestra.run.smithi043.stderr:0+1 records out 2024-09-19T05:04:37.589 INFO:teuthology.orchestra.run.smithi043.stderr:71 bytes copied, 5.7582e-05 s, 1.2 MB/s 2024-09-19T05:04:37.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T05:04:37.599 INFO:teuthology.run_tasks:Running task exec... 2024-09-19T05:04:37.608 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T05:04:37.608 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi043.front.sepia.ceph.com 2024-09-19T05:04:37.608 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-09-19T05:04:37.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37.795+0000 7fb50bc00640 1 -- 172.21.15.43:0/2441277509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb504107830 msgr2=0x7fb504109ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:37.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37.795+0000 7fb50bc00640 1 --2- 172.21.15.43:0/2441277509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb504107830 0x7fb504109ca0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc009ca0 tx=0x7fb4fc02a8b0 comp rx=0 tx=0).stop 2024-09-19T05:04:37.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37.796+0000 7fb50bc00640 1 -- 172.21.15.43:0/2441277509 shutdown_connections 2024-09-19T05:04:37.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37.796+0000 7fb50bc00640 1 --2- 172.21.15.43:0/2441277509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb504107830 0x7fb504109ca0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:37.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37.796+0000 7fb50bc00640 1 --2- 172.21.15.43:0/2441277509 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb504104f00 0x7fb5041072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:37.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37.796+0000 7fb50bc00640 1 --2- 172.21.15.43:0/2441277509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5041025d0 0x7fb5041049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:37.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37.796+0000 7fb50bc00640 1 -- 172.21.15.43:0/2441277509 >> 172.21.15.43:0/2441277509 conn(0x7fb5040fc1b0 msgr2=0x7fb5040fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:37.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37.796+0000 7fb50bc00640 1 -- 172.21.15.43:0/2441277509 shutdown_connections 2024-09-19T05:04:37.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37.796+0000 7fb50bc00640 1 -- 172.21.15.43:0/2441277509 wait complete. 2024-09-19T05:04:37.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37.797+0000 7fb50bc00640 1 Processor -- start 2024-09-19T05:04:37.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37.797+0000 7fb50bc00640 1 -- start start 2024-09-19T05:04:37.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37.797+0000 7fb50bc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5041025d0 0x7fb50419a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:37.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37.797+0000 7fb50bc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb504104f00 0x7fb50419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:37.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37.797+0000 7fb50bc00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb504107830 0x7fb5041a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:37.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37.797+0000 7fb50bc00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb50410d840 con 0x7fb504104f00 2024-09-19T05:04:37.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37.797+0000 7fb50bc00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb50410d6c0 con 0x7fb504107830 2024-09-19T05:04:37.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37.797+0000 7fb50bc00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb50410d9c0 con 0x7fb5041025d0 2024-09-19T05:04:37.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37.797+0000 7fb503e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb504104f00 0x7fb50419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:37.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37.797+0000 7fb503e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb504104f00 0x7fb50419a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59850/0 (socket says 172.21.15.43:59850) 2024-09-19T05:04:37.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37.797+0000 7fb503e00640 1 -- 172.21.15.43:0/2623798103 learned_addr learned my addr 172.21.15.43:0/2623798103 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:37.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37.798+0000 7fb509c00640 1 --2- 172.21.15.43:0/2623798103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb504107830 0x7fb5041a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:37.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37.798+0000 7fb509200640 1 --2- 172.21.15.43:0/2623798103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5041025d0 0x7fb50419a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:37.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37.798+0000 7fb503e00640 1 -- 172.21.15.43:0/2623798103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5041025d0 msgr2=0x7fb50419a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:37.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37.798+0000 7fb503e00640 1 --2- 172.21.15.43:0/2623798103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5041025d0 0x7fb50419a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:37.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37.798+0000 7fb503e00640 1 -- 172.21.15.43:0/2623798103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb504107830 msgr2=0x7fb5041a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:37.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37.798+0000 7fb503e00640 1 --2- 172.21.15.43:0/2623798103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb504107830 0x7fb5041a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:37.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37.798+0000 7fb503e00640 1 -- 172.21.15.43:0/2623798103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5041a1c20 con 0x7fb504104f00 2024-09-19T05:04:37.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37.798+0000 7fb503e00640 1 --2- 172.21.15.43:0/2623798103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb504104f00 0x7fb50419a540 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb4f000d770 tx=0x7fb4f000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:37.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37.798+0000 7fb501600640 1 -- 172.21.15.43:0/2623798103 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f0017070 con 0x7fb504104f00 2024-09-19T05:04:37.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37.799+0000 7fb50bc00640 1 -- 172.21.15.43:0/2623798103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5041a1f10 con 0x7fb504104f00 2024-09-19T05:04:37.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37.799+0000 7fb501600640 1 -- 172.21.15.43:0/2623798103 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4f0009c30 con 0x7fb504104f00 2024-09-19T05:04:37.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37.799+0000 7fb501600640 1 -- 172.21.15.43:0/2623798103 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f000f620 con 0x7fb504104f00 2024-09-19T05:04:37.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37.799+0000 7fb50bc00640 1 -- 172.21.15.43:0/2623798103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5041a23d0 con 0x7fb504104f00 2024-09-19T05:04:37.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37.800+0000 7fb50bc00640 1 -- 172.21.15.43:0/2623798103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb504109320 con 0x7fb504104f00 2024-09-19T05:04:37.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37.804+0000 7fb501600640 1 -- 172.21.15.43:0/2623798103 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 20) ==== 138372+0+0 (secure 0 0 0) 0x7fb4f001a070 con 0x7fb504104f00 2024-09-19T05:04:37.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37.806+0000 7fb501600640 1 --2- 172.21.15.43:0/2623798103 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fb4ec0a3340 0x7fb4ec0a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:37.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37.806+0000 7fb509200640 1 -- 172.21.15.43:0/2623798103 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fb4ec0a3340 msgr2=0x7fb4ec0a5800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/3167108925 2024-09-19T05:04:37.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37.806+0000 7fb501600640 1 -- 172.21.15.43:0/2623798103 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fb4f00cf400 con 0x7fb504104f00 2024-09-19T05:04:37.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37.806+0000 7fb509200640 1 --2- 172.21.15.43:0/2623798103 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fb4ec0a3340 0x7fb4ec0a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T05:04:37.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:37.811+0000 7fb501600640 1 -- 172.21.15.43:0/2623798103 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb4f0009de0 con 0x7fb504104f00 2024-09-19T05:04:38.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.007+0000 7fb509200640 1 -- 172.21.15.43:0/2623798103 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fb4ec0a3340 msgr2=0x7fb4ec0a5800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/3167108925 2024-09-19T05:04:38.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.007+0000 7fb509200640 1 --2- 172.21.15.43:0/2623798103 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fb4ec0a3340 0x7fb4ec0a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T05:04:38.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:37 smithi043 ceph-mon[30800]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-09-19T05:04:38.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.037+0000 7fb50bc00640 1 -- 172.21.15.43:0/2623798103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7fb504062e90 con 0x7fb504104f00 2024-09-19T05:04:38.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:37 smithi151 ceph-mon[35425]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-09-19T05:04:38.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:37 smithi096 ceph-mon[35085]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-09-19T05:04:38.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.246+0000 7fb501600640 1 -- 172.21.15.43:0/2623798103 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 21) ==== 138040+0+0 (secure 0 0 0) 0x7fb4f0086100 con 0x7fb504104f00 2024-09-19T05:04:38.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.247+0000 7fb501600640 1 -- 172.21.15.43:0/2623798103 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fb4ec0a3340 msgr2=0x7fb4ec0a5800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T05:04:38.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.247+0000 7fb501600640 1 --2- 172.21.15.43:0/2623798103 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fb4ec0a3340 0x7fb4ec0a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:38.530 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:04:38.233+0000 7f829c41a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T05:04:38.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.791+0000 7fb501600640 1 -- 172.21.15.43:0/2623798103 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v29) ==== 118+0+0 (secure 0 0 0) 0x7fb4f008c0a0 con 0x7fb504104f00 2024-09-19T05:04:38.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.793+0000 7fb50bc00640 1 -- 172.21.15.43:0/2623798103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb504104f00 msgr2=0x7fb50419a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:38.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.793+0000 7fb50bc00640 1 --2- 172.21.15.43:0/2623798103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb504104f00 0x7fb50419a540 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb4f000d770 tx=0x7fb4f000dc30 comp rx=0 tx=0).stop 2024-09-19T05:04:38.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.793+0000 7fb50bc00640 1 -- 172.21.15.43:0/2623798103 shutdown_connections 2024-09-19T05:04:38.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.793+0000 7fb50bc00640 1 --2- 172.21.15.43:0/2623798103 >> [v2:172.21.15.43:6800/3167108925,v1:172.21.15.43:6801/3167108925] conn(0x7fb4ec0a3340 0x7fb4ec0a5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:38.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.793+0000 7fb50bc00640 1 --2- 172.21.15.43:0/2623798103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb504107830 0x7fb5041a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:38.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.793+0000 7fb50bc00640 1 --2- 172.21.15.43:0/2623798103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb504104f00 0x7fb50419a540 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:38.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.793+0000 7fb50bc00640 1 --2- 172.21.15.43:0/2623798103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5041025d0 0x7fb50419a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:38.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.793+0000 7fb50bc00640 1 -- 172.21.15.43:0/2623798103 >> 172.21.15.43:0/2623798103 conn(0x7fb5040fc1b0 msgr2=0x7fb504101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:38.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.794+0000 7fb50bc00640 1 -- 172.21.15.43:0/2623798103 shutdown_connections 2024-09-19T05:04:38.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.794+0000 7fb50bc00640 1 -- 172.21.15.43:0/2623798103 wait complete. 2024-09-19T05:04:38.813 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-09-19T05:04:38.873 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:04:38.574+0000 7f81f5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:04:38.873 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:04:38.574+0000 7f81f5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:04:38.874 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:04:38.574+0000 7f81f5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:04:38.874 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:04:38.574+0000 7f81f5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:04:38.874 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:04:38.574+0000 7f81f5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:04:38.874 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:04:38.575+0000 7f81f8200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:04:38.874 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:04:38.575+0000 7f81f8200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:04:38.874 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:04:38.575+0000 7f81f8200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:04:38.874 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:04:38.575+0000 7f81f8200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:04:38.874 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:04:38.575+0000 7f81f8200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:04:38.875 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/2623798103' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-19T05:04:38.875 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: Active manager daemon x restarted 2024-09-19T05:04:38.875 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: Activating manager daemon x 2024-09-19T05:04:38.875 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T05:04:38.875 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: mgrmap e21: x(active, starting, since 0.0120021s), standbys: z, y 2024-09-19T05:04:38.875 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T05:04:38.875 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:04:38.875 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:04:38.875 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T05:04:38.875 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T05:04:38.875 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T05:04:38.875 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T05:04:38.875 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T05:04:38.876 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T05:04:38.876 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T05:04:38.876 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T05:04:38.876 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T05:04:38.876 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T05:04:38.876 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T05:04:38.876 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T05:04:38.876 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T05:04:38.876 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T05:04:38.876 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T05:04:38.876 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T05:04:38.876 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T05:04:38.876 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T05:04:38.876 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:04:38.877 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:04:38.877 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:04:38.877 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:04:38.877 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:04:38.877 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:04:38.877 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:04:38.877 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:04:38.877 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:04:38.877 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T05:04:38.877 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T05:04:38.878 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T05:04:38.878 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: Manager daemon x is now available 2024-09-19T05:04:38.878 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:04:38.878 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T05:04:38.878 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T05:04:38.878 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T05:04:38.878 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:38 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T05:04:38.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/2623798103' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-19T05:04:38.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: Active manager daemon x restarted 2024-09-19T05:04:38.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: Activating manager daemon x 2024-09-19T05:04:38.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T05:04:38.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: mgrmap e21: x(active, starting, since 0.0120021s), standbys: z, y 2024-09-19T05:04:38.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T05:04:38.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:04:38.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:04:38.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T05:04:38.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T05:04:38.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T05:04:38.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T05:04:38.975 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T05:04:38.975 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T05:04:38.975 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T05:04:38.975 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T05:04:38.975 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T05:04:38.975 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T05:04:38.975 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T05:04:38.975 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T05:04:38.975 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T05:04:38.975 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T05:04:38.975 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T05:04:38.975 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T05:04:38.975 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T05:04:38.975 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T05:04:38.976 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:04:38.976 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:04:38.976 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:04:38.976 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:04:38.976 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:04:38.976 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:04:38.976 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:04:38.976 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:04:38.976 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:04:38.976 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T05:04:38.976 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T05:04:38.976 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T05:04:38.976 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: Manager daemon x is now available 2024-09-19T05:04:38.977 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:04:38.977 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T05:04:38.977 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T05:04:38.977 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T05:04:38.977 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:38 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T05:04:38.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.987+0000 7f9260c00640 1 -- 172.21.15.43:0/3984077719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f925c076230 msgr2=0x7f925c074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:38.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.987+0000 7f9260c00640 1 --2- 172.21.15.43:0/3984077719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f925c076230 0x7f925c074680 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f924c009860 tx=0x7f924c02a340 comp rx=0 tx=0).stop 2024-09-19T05:04:38.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.987+0000 7f9260c00640 1 -- 172.21.15.43:0/3984077719 shutdown_connections 2024-09-19T05:04:38.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.987+0000 7f9260c00640 1 --2- 172.21.15.43:0/3984077719 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f925c10e110 0x7f925c110500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:38.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.987+0000 7f9260c00640 1 --2- 172.21.15.43:0/3984077719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f925c074c50 0x7f925c0750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:38.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.987+0000 7f9260c00640 1 --2- 172.21.15.43:0/3984077719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f925c076230 0x7f925c074680 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:38.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.987+0000 7f9260c00640 1 -- 172.21.15.43:0/3984077719 >> 172.21.15.43:0/3984077719 conn(0x7f925c0fd2e0 msgr2=0x7f925c0ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:38.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.987+0000 7f9260c00640 1 -- 172.21.15.43:0/3984077719 shutdown_connections 2024-09-19T05:04:38.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.987+0000 7f9260c00640 1 -- 172.21.15.43:0/3984077719 wait complete. 2024-09-19T05:04:38.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.988+0000 7f9260c00640 1 Processor -- start 2024-09-19T05:04:38.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.988+0000 7f9260c00640 1 -- start start 2024-09-19T05:04:38.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.988+0000 7f9260c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f925c074c50 0x7f925c19e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:38.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.988+0000 7f9260c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f925c076230 0x7f925c19e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:38.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.988+0000 7f9260c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f925c10e110 0x7f925c1a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:38.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.988+0000 7f9260c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f925c113dd0 con 0x7f925c076230 2024-09-19T05:04:38.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.988+0000 7f9260c00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f925c113c50 con 0x7f925c074c50 2024-09-19T05:04:38.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.988+0000 7f9260c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f925c113f50 con 0x7f925c10e110 2024-09-19T05:04:38.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.989+0000 7f9259e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f925c074c50 0x7f925c19e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:38.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.989+0000 7f9259e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f925c074c50 0x7f925c19e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:59284/0 (socket says 172.21.15.43:59284) 2024-09-19T05:04:38.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.989+0000 7f9259e00640 1 -- 172.21.15.43:0/2299658285 learned_addr learned my addr 172.21.15.43:0/2299658285 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:38.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.989+0000 7f9259400640 1 --2- 172.21.15.43:0/2299658285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f925c076230 0x7f925c19e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:38.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.989+0000 7f925a800640 1 --2- 172.21.15.43:0/2299658285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f925c10e110 0x7f925c1a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:38.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.989+0000 7f9259e00640 1 -- 172.21.15.43:0/2299658285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f925c10e110 msgr2=0x7f925c1a58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:38.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.989+0000 7f9259e00640 1 --2- 172.21.15.43:0/2299658285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f925c10e110 0x7f925c1a58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:38.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.989+0000 7f9259e00640 1 -- 172.21.15.43:0/2299658285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f925c076230 msgr2=0x7f925c19e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:38.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.989+0000 7f9259e00640 1 --2- 172.21.15.43:0/2299658285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f925c076230 0x7f925c19e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:38.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.989+0000 7f9259e00640 1 -- 172.21.15.43:0/2299658285 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f925c1a5f40 con 0x7f925c074c50 2024-09-19T05:04:38.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.989+0000 7f9259e00640 1 --2- 172.21.15.43:0/2299658285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f925c074c50 0x7f925c19e320 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f924c035040 tx=0x7f924c033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:38.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.989+0000 7f9252a00640 1 -- 172.21.15.43:0/2299658285 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924c037040 con 0x7f925c074c50 2024-09-19T05:04:38.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.989+0000 7f9252a00640 1 -- 172.21.15.43:0/2299658285 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f924c030850 con 0x7f925c074c50 2024-09-19T05:04:38.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.990+0000 7f9260c00640 1 -- 172.21.15.43:0/2299658285 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f925c1a61d0 con 0x7f925c074c50 2024-09-19T05:04:38.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.990+0000 7f9252a00640 1 -- 172.21.15.43:0/2299658285 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924c004920 con 0x7f925c074c50 2024-09-19T05:04:38.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.990+0000 7f9260c00640 1 -- 172.21.15.43:0/2299658285 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f925c1a6710 con 0x7f925c074c50 2024-09-19T05:04:38.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.991+0000 7f9260c00640 1 -- 172.21.15.43:0/2299658285 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f925c1041c0 con 0x7f925c074c50 2024-09-19T05:04:38.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.992+0000 7f9252a00640 1 -- 172.21.15.43:0/2299658285 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 21) ==== 138040+0+0 (secure 0 0 0) 0x7f924c004050 con 0x7f925c074c50 2024-09-19T05:04:38.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.994+0000 7f9252a00640 1 -- 172.21.15.43:0/2299658285 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f924c0ef390 con 0x7f925c074c50 2024-09-19T05:04:38.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:38.998+0000 7f9252a00640 1 -- 172.21.15.43:0/2299658285 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f924c0bc340 con 0x7f925c074c50 2024-09-19T05:04:39.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/2623798103' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-19T05:04:39.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: Active manager daemon x restarted 2024-09-19T05:04:39.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: Activating manager daemon x 2024-09-19T05:04:39.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T05:04:39.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: mgrmap e21: x(active, starting, since 0.0120021s), standbys: z, y 2024-09-19T05:04:39.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T05:04:39.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T05:04:39.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T05:04:39.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T05:04:39.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T05:04:39.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T05:04:39.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T05:04:39.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T05:04:39.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T05:04:39.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T05:04:39.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T05:04:39.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T05:04:39.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T05:04:39.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T05:04:39.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T05:04:39.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T05:04:39.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T05:04:39.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T05:04:39.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T05:04:39.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T05:04:39.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T05:04:39.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T05:04:39.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T05:04:39.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T05:04:39.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T05:04:39.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T05:04:39.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T05:04:39.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T05:04:39.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T05:04:39.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T05:04:39.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T05:04:39.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T05:04:39.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T05:04:39.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: Manager daemon x is now available 2024-09-19T05:04:39.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:04:39.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T05:04:39.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T05:04:39.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T05:04:39.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:38 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T05:04:39.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.257+0000 7f9252a00640 1 -- 172.21.15.43:0/2299658285 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 22) ==== 138167+0+0 (secure 0 0 0) 0x7f924c0b7990 con 0x7f925c074c50 2024-09-19T05:04:39.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.258+0000 7f9252a00640 1 --2- 172.21.15.43:0/2299658285 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f92400aa0f0 0x7f92400ac4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:39.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.258+0000 7f9252a00640 1 -- 172.21.15.43:0/2299658285 --> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f924c039ec0 con 0x7f92400aa0f0 2024-09-19T05:04:39.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.260+0000 7f9259400640 1 --2- 172.21.15.43:0/2299658285 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f92400aa0f0 0x7f92400ac4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:39.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.265+0000 7f9259400640 1 --2- 172.21.15.43:0/2299658285 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f92400aa0f0 0x7f92400ac4e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f925c19f830 tx=0x7f924800a2d0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:39.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.295+0000 7f9252a00640 1 -- 172.21.15.43:0/2299658285 <== mgr.25136 v2:172.21.15.43:6800/2120444093 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f924c039ec0 con 0x7f92400aa0f0 2024-09-19T05:04:39.296 INFO:teuthology.orchestra.run.smithi043.stdout:{} 2024-09-19T05:04:39.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.297+0000 7f9260c00640 1 -- 172.21.15.43:0/2299658285 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f92400aa0f0 msgr2=0x7f92400ac4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:39.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.297+0000 7f9260c00640 1 --2- 172.21.15.43:0/2299658285 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f92400aa0f0 0x7f92400ac4e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f925c19f830 tx=0x7f924800a2d0 comp rx=0 tx=0).stop 2024-09-19T05:04:39.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.297+0000 7f9260c00640 1 -- 172.21.15.43:0/2299658285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f925c074c50 msgr2=0x7f925c19e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:39.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.297+0000 7f9260c00640 1 --2- 172.21.15.43:0/2299658285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f925c074c50 0x7f925c19e320 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f924c035040 tx=0x7f924c033d90 comp rx=0 tx=0).stop 2024-09-19T05:04:39.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.297+0000 7f9260c00640 1 -- 172.21.15.43:0/2299658285 shutdown_connections 2024-09-19T05:04:39.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.297+0000 7f9260c00640 1 --2- 172.21.15.43:0/2299658285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f925c10e110 0x7f925c1a58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:39.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.297+0000 7f9260c00640 1 --2- 172.21.15.43:0/2299658285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f925c076230 0x7f925c19e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:39.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.297+0000 7f9260c00640 1 --2- 172.21.15.43:0/2299658285 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f92400aa0f0 0x7f92400ac4e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:39.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.297+0000 7f9260c00640 1 --2- 172.21.15.43:0/2299658285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f925c074c50 0x7f925c19e320 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:39.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.297+0000 7f9260c00640 1 -- 172.21.15.43:0/2299658285 >> 172.21.15.43:0/2299658285 conn(0x7f925c0fd2e0 msgr2=0x7f925c101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:39.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.297+0000 7f9260c00640 1 -- 172.21.15.43:0/2299658285 shutdown_connections 2024-09-19T05:04:39.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.297+0000 7f9260c00640 1 -- 172.21.15.43:0/2299658285 wait complete. 2024-09-19T05:04:39.315 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-19T05:04:39.325 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-19T05:04:39.417 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T05:04:39.530 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:39 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:04:39.276+0000 7f81f1e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:04:39.531 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:39 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:04:39.276+0000 7f81f1e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:04:39.531 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:39 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:04:39.276+0000 7f81f1e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:04:39.531 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:39 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:04:39.276+0000 7f81f1e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:04:39.531 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:39 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:04:39.276+0000 7f81f1e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:04:39.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.587+0000 7fdfb2000640 1 -- 172.21.15.43:0/3299863167 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdfac104f20 msgr2=0x7fdfac1053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:39.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.587+0000 7fdfb2000640 1 --2- 172.21.15.43:0/3299863167 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdfac104f20 0x7fdfac1053a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdf98009860 tx=0x7fdf9802a340 comp rx=0 tx=0).stop 2024-09-19T05:04:39.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.587+0000 7fdfb2000640 1 -- 172.21.15.43:0/3299863167 shutdown_connections 2024-09-19T05:04:39.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.587+0000 7fdfb2000640 1 --2- 172.21.15.43:0/3299863167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfac1058e0 0x7fdfac10c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:39.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.587+0000 7fdfb2000640 1 --2- 172.21.15.43:0/3299863167 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdfac104f20 0x7fdfac1053a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:39.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.587+0000 7fdfb2000640 1 --2- 172.21.15.43:0/3299863167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfac103d30 0x7fdfac104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:39.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.587+0000 7fdfb2000640 1 -- 172.21.15.43:0/3299863167 >> 172.21.15.43:0/3299863167 conn(0x7fdfac0ff4b0 msgr2=0x7fdfac1018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:39.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.587+0000 7fdfb2000640 1 -- 172.21.15.43:0/3299863167 shutdown_connections 2024-09-19T05:04:39.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.587+0000 7fdfb2000640 1 -- 172.21.15.43:0/3299863167 wait complete. 2024-09-19T05:04:39.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.588+0000 7fdfb2000640 1 Processor -- start 2024-09-19T05:04:39.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.588+0000 7fdfb2000640 1 -- start start 2024-09-19T05:04:39.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.589+0000 7fdfb2000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfac103d30 0x7fdfac19a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:39.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.589+0000 7fdfb2000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdfac104f20 0x7fdfac19a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:39.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.589+0000 7fdfb2000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfac1058e0 0x7fdfac1a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:39.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.589+0000 7fdfb2000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdfac10f9b0 con 0x7fdfac1058e0 2024-09-19T05:04:39.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.589+0000 7fdfb2000640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdfac10f830 con 0x7fdfac103d30 2024-09-19T05:04:39.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.589+0000 7fdfb2000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdfac10fb30 con 0x7fdfac104f20 2024-09-19T05:04:39.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.589+0000 7fdfaaa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdfac104f20 0x7fdfac19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:39.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.589+0000 7fdfaaa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdfac104f20 0x7fdfac19a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:39268/0 (socket says 172.21.15.43:39268) 2024-09-19T05:04:39.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.589+0000 7fdfaaa00640 1 -- 172.21.15.43:0/2504992225 learned_addr learned my addr 172.21.15.43:0/2504992225 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:39.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.589+0000 7fdfabe00640 1 --2- 172.21.15.43:0/2504992225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfac1058e0 0x7fdfac1a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:39.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.589+0000 7fdfab400640 1 --2- 172.21.15.43:0/2504992225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfac103d30 0x7fdfac19a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:39.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.589+0000 7fdfaaa00640 1 -- 172.21.15.43:0/2504992225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfac103d30 msgr2=0x7fdfac19a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:39.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.589+0000 7fdfaaa00640 1 --2- 172.21.15.43:0/2504992225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfac103d30 0x7fdfac19a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:39.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.589+0000 7fdfaaa00640 1 -- 172.21.15.43:0/2504992225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfac1058e0 msgr2=0x7fdfac1a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:39.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.589+0000 7fdfaaa00640 1 --2- 172.21.15.43:0/2504992225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfac1058e0 0x7fdfac1a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:39.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.589+0000 7fdfaaa00640 1 -- 172.21.15.43:0/2504992225 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfac1a1c20 con 0x7fdfac104f20 2024-09-19T05:04:39.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.590+0000 7fdfaaa00640 1 --2- 172.21.15.43:0/2504992225 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdfac104f20 0x7fdfac19a540 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdf98035040 tx=0x7fdf98033d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:39.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.590+0000 7fdfa3e00640 1 -- 172.21.15.43:0/2504992225 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf98037040 con 0x7fdfac104f20 2024-09-19T05:04:39.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.590+0000 7fdfa3e00640 1 -- 172.21.15.43:0/2504992225 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf98030850 con 0x7fdfac104f20 2024-09-19T05:04:39.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.590+0000 7fdfb2000640 1 -- 172.21.15.43:0/2504992225 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfac1a1eb0 con 0x7fdfac104f20 2024-09-19T05:04:39.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.590+0000 7fdfa3e00640 1 -- 172.21.15.43:0/2504992225 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf98004920 con 0x7fdfac104f20 2024-09-19T05:04:39.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.590+0000 7fdfb2000640 1 -- 172.21.15.43:0/2504992225 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfac1a2320 con 0x7fdfac104f20 2024-09-19T05:04:39.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.592+0000 7fdfb2000640 1 -- 172.21.15.43:0/2504992225 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfac10b110 con 0x7fdfac104f20 2024-09-19T05:04:39.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.594+0000 7fdfa3e00640 1 -- 172.21.15.43:0/2504992225 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 22) ==== 138167+0+0 (secure 0 0 0) 0x7fdf98004050 con 0x7fdfac104f20 2024-09-19T05:04:39.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.595+0000 7fdfa3e00640 1 --2- 172.21.15.43:0/2504992225 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fdf900a30d0 0x7fdf900a5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:39.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.596+0000 7fdfa3e00640 1 -- 172.21.15.43:0/2504992225 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fdf980ef4c0 con 0x7fdfac104f20 2024-09-19T05:04:39.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.596+0000 7fdfab400640 1 --2- 172.21.15.43:0/2504992225 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fdf900a30d0 0x7fdf900a5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:39.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.596+0000 7fdfab400640 1 --2- 172.21.15.43:0/2504992225 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fdf900a30d0 0x7fdf900a5590 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c009530 tx=0x7fdf9c009260 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:39.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.599+0000 7fdfa3e00640 1 -- 172.21.15.43:0/2504992225 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdf980bc380 con 0x7fdfac104f20 2024-09-19T05:04:39.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.820+0000 7fdfb2000640 1 -- 172.21.15.43:0/2504992225 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdfac062df0 con 0x7fdfac104f20 2024-09-19T05:04:39.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.822+0000 7fdfa3e00640 1 -- 172.21.15.43:0/2504992225 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+23038 (secure 0 0 0) 0x7fdf980bbc80 con 0x7fdfac104f20 2024-09-19T05:04:39.824 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:04:39.824 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":86,"fsid":"1456a810-7643-11ef-bceb-c7b262605968","created":"2024-09-19T04:54:14.706644+0000","modified":"2024-09-19T05:04:38.237868+0000","last_up_change":"2024-09-19T05:01:27.348740+0000","last_in_change":"2024-09-19T05:01:03.396477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:57:27.642705+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:02:20.685123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:02:23.393596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"81a317a8-7ec8-45c3-bc10-7da023283635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6803","nonce":825351486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6805","nonce":825351486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6809","nonce":825351486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6807","nonce":825351486}]},"public_addr":"172.21.15.43:6803/825351486","cluster_addr":"172.21.15.43:6805/825351486","heartbeat_back_addr":"172.21.15.43:6809/825351486","heartbeat_front_addr":"172.21.15.43:6807/825351486","state":["exists","up"]},{"osd":1,"uuid":"e96fcfac-4fff-4a36-b38b-aebf57842740","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6801","nonce":2140095130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6803","nonce":2140095130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6807","nonce":2140095130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6805","nonce":2140095130}]},"public_addr":"172.21.15.96:6801/2140095130","cluster_addr":"172.21.15.96:6803/2140095130","heartbeat_back_addr":"172.21.15.96:6807/2140095130","heartbeat_front_addr":"172.21.15.96:6805/2140095130","state":["exists","up"]},{"osd":2,"uuid":"b659869e-114a-4f56-8aac-a2ed602b4058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6801","nonce":2047147996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6803","nonce":2047147996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6807","nonce":2047147996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6805","nonce":2047147996}]},"public_addr":"172.21.15.151:6801/2047147996","cluster_addr":"172.21.15.151:6803/2047147996","heartbeat_back_addr":"172.21.15.151:6807/2047147996","heartbeat_front_addr":"172.21.15.151:6805/2047147996","state":["exists","up"]},{"osd":3,"uuid":"7be64b2c-e45c-493d-a610-4ce41196061f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6811","nonce":1734673762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6813","nonce":1734673762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6817","nonce":1734673762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6815","nonce":1734673762}]},"public_addr":"172.21.15.43:6811/1734673762","cluster_addr":"172.21.15.43:6813/1734673762","heartbeat_back_addr":"172.21.15.43:6817/1734673762","heartbeat_front_addr":"172.21.15.43:6815/1734673762","state":["exists","up"]},{"osd":4,"uuid":"196d5af1-a6e5-496c-8933-4e58892f4360","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6809","nonce":3184785362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6811","nonce":3184785362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6815","nonce":3184785362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6813","nonce":3184785362}]},"public_addr":"172.21.15.96:6809/3184785362","cluster_addr":"172.21.15.96:6811/3184785362","heartbeat_back_addr":"172.21.15.96:6815/3184785362","heartbeat_front_addr":"172.21.15.96:6813/3184785362","state":["exists","up"]},{"osd":5,"uuid":"5492d474-8509-4865-bc2e-d08e31558a75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6809","nonce":2878451924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6811","nonce":2878451924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6815","nonce":2878451924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6813","nonce":2878451924}]},"public_addr":"172.21.15.151:6809/2878451924","cluster_addr":"172.21.15.151:6811/2878451924","heartbeat_back_addr":"172.21.15.151:6815/2878451924","heartbeat_front_addr":"172.21.15.151:6813/2878451924","state":["exists","up"]},{"osd":6,"uuid":"fb8457ed-7bb9-4435-9d15-c1cbfbddfcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6819","nonce":2008630290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6821","nonce":2008630290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6825","nonce":2008630290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6823","nonce":2008630290}]},"public_addr":"172.21.15.43:6819/2008630290","cluster_addr":"172.21.15.43:6821/2008630290","heartbeat_back_addr":"172.21.15.43:6825/2008630290","heartbeat_front_addr":"172.21.15.43:6823/2008630290","state":["exists","up"]},{"osd":7,"uuid":"02046db8-30de-4e92-b25a-1f07d6190e91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6817","nonce":3588713275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6819","nonce":3588713275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6823","nonce":3588713275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6821","nonce":3588713275}]},"public_addr":"172.21.15.96:6817/3588713275","cluster_addr":"172.21.15.96:6819/3588713275","heartbeat_back_addr":"172.21.15.96:6823/3588713275","heartbeat_front_addr":"172.21.15.96:6821/3588713275","state":["exists","up"]},{"osd":8,"uuid":"a5bb4218-51f3-4044-b43e-ebfc764f49ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6817","nonce":725316393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6819","nonce":725316393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6823","nonce":725316393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6821","nonce":725316393}]},"public_addr":"172.21.15.151:6817/725316393","cluster_addr":"172.21.15.151:6819/725316393","heartbeat_back_addr":"172.21.15.151:6823/725316393","heartbeat_front_addr":"172.21.15.151:6821/725316393","state":["exists","up"]},{"osd":9,"uuid":"6c19f65c-9750-4baa-9d63-7b8bd03e6955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6827","nonce":4053438621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6829","nonce":4053438621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6833","nonce":4053438621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6831","nonce":4053438621}]},"public_addr":"172.21.15.43:6827/4053438621","cluster_addr":"172.21.15.43:6829/4053438621","heartbeat_back_addr":"172.21.15.43:6833/4053438621","heartbeat_front_addr":"172.21.15.43:6831/4053438621","state":["exists","up"]},{"osd":10,"uuid":"a7aa7da3-8eab-47fe-a351-298b6ff75f14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6825","nonce":2403210086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6827","nonce":2403210086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6831","nonce":2403210086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6829","nonce":2403210086}]},"public_addr":"172.21.15.96:6825/2403210086","cluster_addr":"172.21.15.96:6827/2403210086","heartbeat_back_addr":"172.21.15.96:6831/2403210086","heartbeat_front_addr":"172.21.15.96:6829/2403210086","state":["exists","up"]},{"osd":11,"uuid":"760b6afc-253d-44dd-88e9-dcc97f67c9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6825","nonce":292202881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6827","nonce":292202881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6831","nonce":292202881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6829","nonce":292202881}]},"public_addr":"172.21.15.151:6825/292202881","cluster_addr":"172.21.15.151:6827/292202881","heartbeat_back_addr":"172.21.15.151:6831/292202881","heartbeat_front_addr":"172.21.15.151:6829/292202881","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:56:34.716991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:56:59.321174+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:57:24.281696+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:57:53.443880+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:58:19.205873+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:58:44.504873+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:13.650731+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:39.137876+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:04.579893+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:34.337312+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:59.434658+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:01:24.930529+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/3819455723":"2024-09-20T05:04:38.237695+0000","172.21.15.43:0/3712369735":"2024-09-20T05:04:38.237695+0000","172.21.15.43:0/330854185":"2024-09-20T04:54:39.548065+0000","172.21.15.43:0/6132665":"2024-09-20T04:54:39.548065+0000","172.21.15.43:0/3730884138":"2024-09-20T05:04:38.237695+0000","172.21.15.43:0/632163478":"2024-09-20T04:54:39.548065+0000","172.21.15.43:0/4271788315":"2024-09-20T04:55:07.037117+0000","172.21.15.43:6800/587463881":"2024-09-20T04:54:39.548065+0000","172.21.15.43:6800/3241799377":"2024-09-20T04:55:07.037117+0000","172.21.15.43:6801/3241799377":"2024-09-20T04:55:07.037117+0000","172.21.15.43:0/898247657":"2024-09-20T04:55:07.037117+0000","172.21.15.43:6801/3167108925":"2024-09-20T05:04:38.237695+0000","172.21.15.43:0/515912755":"2024-09-20T05:04:38.237695+0000","172.21.15.43:6801/587463881":"2024-09-20T04:54:39.548065+0000","172.21.15.43:0/3256350596":"2024-09-20T04:55:07.037117+0000","172.21.15.43:6800/3167108925":"2024-09-20T05:04:38.237695+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:04:39.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.824+0000 7fdfb2000640 1 -- 172.21.15.43:0/2504992225 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fdf900a30d0 msgr2=0x7fdf900a5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:39.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.824+0000 7fdfb2000640 1 --2- 172.21.15.43:0/2504992225 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fdf900a30d0 0x7fdf900a5590 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c009530 tx=0x7fdf9c009260 comp rx=0 tx=0).stop 2024-09-19T05:04:39.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.824+0000 7fdfb2000640 1 -- 172.21.15.43:0/2504992225 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdfac104f20 msgr2=0x7fdfac19a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:39.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.824+0000 7fdfb2000640 1 --2- 172.21.15.43:0/2504992225 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdfac104f20 0x7fdfac19a540 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdf98035040 tx=0x7fdf98033d90 comp rx=0 tx=0).stop 2024-09-19T05:04:39.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.824+0000 7fdfb2000640 1 -- 172.21.15.43:0/2504992225 shutdown_connections 2024-09-19T05:04:39.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.824+0000 7fdfb2000640 1 --2- 172.21.15.43:0/2504992225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfac1058e0 0x7fdfac1a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:39.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.824+0000 7fdfb2000640 1 --2- 172.21.15.43:0/2504992225 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdfac104f20 0x7fdfac19a540 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:39.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.824+0000 7fdfb2000640 1 --2- 172.21.15.43:0/2504992225 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fdf900a30d0 0x7fdf900a5590 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:39.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.824+0000 7fdfb2000640 1 --2- 172.21.15.43:0/2504992225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfac103d30 0x7fdfac19a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:39.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.825+0000 7fdfb2000640 1 -- 172.21.15.43:0/2504992225 >> 172.21.15.43:0/2504992225 conn(0x7fdfac0ff4b0 msgr2=0x7fdfac1082e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:39.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.825+0000 7fdfb2000640 1 -- 172.21.15.43:0/2504992225 shutdown_connections 2024-09-19T05:04:39.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:39.825+0000 7fdfb2000640 1 -- 172.21.15.43:0/2504992225 wait complete. 2024-09-19T05:04:39.841 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T04:57:27.642705+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-19T05:02:20.685123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-19T05:02:23.393596+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T05:04:39.842 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T05:04:39.849 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:39 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/2623798103' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-19T05:04:39.849 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:39 smithi151 ceph-mon[35425]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T05:04:39.849 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:39 smithi151 ceph-mon[35425]: mgrmap e22: x(active, since 1.01998s), standbys: z, y 2024-09-19T05:04:40.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.011+0000 7fe519400640 1 -- 172.21.15.43:0/3465516742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe514101b10 msgr2=0x7fe514101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:40.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.011+0000 7fe519400640 1 --2- 172.21.15.43:0/3465516742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe514101b10 0x7fe514101f10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe504009900 tx=0x7fe50402a400 comp rx=0 tx=0).stop 2024-09-19T05:04:40.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.011+0000 7fe519400640 1 -- 172.21.15.43:0/3465516742 shutdown_connections 2024-09-19T05:04:40.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.011+0000 7fe519400640 1 --2- 172.21.15.43:0/3465516742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe5141036c0 0x7fe514109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:40.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.011+0000 7fe519400640 1 --2- 172.21.15.43:0/3465516742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe514102d00 0x7fe514103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:40.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.011+0000 7fe519400640 1 --2- 172.21.15.43:0/3465516742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe514101b10 0x7fe514101f10 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:40.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.011+0000 7fe519400640 1 -- 172.21.15.43:0/3465516742 >> 172.21.15.43:0/3465516742 conn(0x7fe5140fd2b0 msgr2=0x7fe5140ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:40.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.011+0000 7fe519400640 1 -- 172.21.15.43:0/3465516742 shutdown_connections 2024-09-19T05:04:40.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.012+0000 7fe519400640 1 -- 172.21.15.43:0/3465516742 wait complete. 2024-09-19T05:04:40.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.012+0000 7fe519400640 1 Processor -- start 2024-09-19T05:04:40.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.013+0000 7fe519400640 1 -- start start 2024-09-19T05:04:40.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.013+0000 7fe519400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe514101b10 0x7fe514197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:40.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.013+0000 7fe519400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe514102d00 0x7fe5141983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:40.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.013+0000 7fe519400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5141036c0 0x7fe51419f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:40.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.013+0000 7fe519400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe51410d790 con 0x7fe5141036c0 2024-09-19T05:04:40.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.013+0000 7fe519400640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe51410d610 con 0x7fe514101b10 2024-09-19T05:04:40.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.013+0000 7fe519400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe51410d910 con 0x7fe514102d00 2024-09-19T05:04:40.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.013+0000 7fe512800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe514101b10 0x7fe514197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:40.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.013+0000 7fe512800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe514101b10 0x7fe514197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:59346/0 (socket says 172.21.15.43:59346) 2024-09-19T05:04:40.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.013+0000 7fe512800640 1 -- 172.21.15.43:0/261895473 learned_addr learned my addr 172.21.15.43:0/261895473 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:40.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.013+0000 7fe511e00640 1 --2- 172.21.15.43:0/261895473 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe514102d00 0x7fe5141983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:40.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.013+0000 7fe513200640 1 --2- 172.21.15.43:0/261895473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5141036c0 0x7fe51419f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:40.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.014+0000 7fe512800640 1 -- 172.21.15.43:0/261895473 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe514102d00 msgr2=0x7fe5141983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:40.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.014+0000 7fe512800640 1 --2- 172.21.15.43:0/261895473 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe514102d00 0x7fe5141983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:40.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.014+0000 7fe512800640 1 -- 172.21.15.43:0/261895473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5141036c0 msgr2=0x7fe51419f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:40.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.014+0000 7fe512800640 1 --2- 172.21.15.43:0/261895473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5141036c0 0x7fe51419f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:40.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.014+0000 7fe512800640 1 -- 172.21.15.43:0/261895473 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe51419fa90 con 0x7fe514101b10 2024-09-19T05:04:40.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.014+0000 7fe511e00640 1 --2- 172.21.15.43:0/261895473 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe514102d00 0x7fe5141983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:04:40.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.014+0000 7fe513200640 1 --2- 172.21.15.43:0/261895473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5141036c0 0x7fe51419f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:04:40.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.014+0000 7fe512800640 1 --2- 172.21.15.43:0/261895473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe514101b10 0x7fe514197e70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe504031610 tx=0x7fe504004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:40.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.014+0000 7fe50b400640 1 -- 172.21.15.43:0/261895473 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50403f030 con 0x7fe514101b10 2024-09-19T05:04:40.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.014+0000 7fe50b400640 1 -- 172.21.15.43:0/261895473 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe504003ad0 con 0x7fe514101b10 2024-09-19T05:04:40.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.014+0000 7fe519400640 1 -- 172.21.15.43:0/261895473 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe51419fd20 con 0x7fe514101b10 2024-09-19T05:04:40.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.014+0000 7fe50b400640 1 -- 172.21.15.43:0/261895473 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe504035070 con 0x7fe514101b10 2024-09-19T05:04:40.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.015+0000 7fe519400640 1 -- 172.21.15.43:0/261895473 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5141a01c0 con 0x7fe514101b10 2024-09-19T05:04:40.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.016+0000 7fe519400640 1 -- 172.21.15.43:0/261895473 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe514108f40 con 0x7fe514101b10 2024-09-19T05:04:40.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.019+0000 7fe50b400640 1 -- 172.21.15.43:0/261895473 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 22) ==== 138167+0+0 (secure 0 0 0) 0x7fe504042030 con 0x7fe514101b10 2024-09-19T05:04:40.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.020+0000 7fe50b400640 1 --2- 172.21.15.43:0/261895473 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fe4f80a3050 0x7fe4f80a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:40.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.021+0000 7fe50b400640 1 -- 172.21.15.43:0/261895473 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fe5040efc60 con 0x7fe514101b10 2024-09-19T05:04:40.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.021+0000 7fe511e00640 1 --2- 172.21.15.43:0/261895473 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fe4f80a3050 0x7fe4f80a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:40.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.021+0000 7fe511e00640 1 --2- 172.21.15.43:0/261895473 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fe4f80a3050 0x7fe4f80a5510 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe5000095f0 tx=0x7fe500005f90 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:40.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.024+0000 7fe50b400640 1 -- 172.21.15.43:0/261895473 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe5040bcb60 con 0x7fe514101b10 2024-09-19T05:04:40.044 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:39 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/2623798103' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-19T05:04:40.045 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:39 smithi043 ceph-mon[30800]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T05:04:40.045 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:39 smithi043 ceph-mon[30800]: mgrmap e22: x(active, since 1.01998s), standbys: z, y 2024-09-19T05:04:40.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.214+0000 7fe519400640 1 -- 172.21.15.43:0/261895473 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe514062e90 con 0x7fe514101b10 2024-09-19T05:04:40.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.214+0000 7fe50b400640 1 -- 172.21.15.43:0/261895473 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) ==== 93+0+10 (secure 0 0 0) 0x7fe5040bc460 con 0x7fe514101b10 2024-09-19T05:04:40.215 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-09-19T05:04:40.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.216+0000 7fe519400640 1 -- 172.21.15.43:0/261895473 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fe4f80a3050 msgr2=0x7fe4f80a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:40.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.216+0000 7fe519400640 1 --2- 172.21.15.43:0/261895473 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fe4f80a3050 0x7fe4f80a5510 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe5000095f0 tx=0x7fe500005f90 comp rx=0 tx=0).stop 2024-09-19T05:04:40.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.216+0000 7fe519400640 1 -- 172.21.15.43:0/261895473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe514101b10 msgr2=0x7fe514197e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:40.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.216+0000 7fe519400640 1 --2- 172.21.15.43:0/261895473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe514101b10 0x7fe514197e70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe504031610 tx=0x7fe504004270 comp rx=0 tx=0).stop 2024-09-19T05:04:40.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.217+0000 7fe519400640 1 -- 172.21.15.43:0/261895473 shutdown_connections 2024-09-19T05:04:40.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.217+0000 7fe519400640 1 --2- 172.21.15.43:0/261895473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5141036c0 0x7fe51419f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:40.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.217+0000 7fe519400640 1 --2- 172.21.15.43:0/261895473 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe514102d00 0x7fe5141983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:40.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.217+0000 7fe519400640 1 --2- 172.21.15.43:0/261895473 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fe4f80a3050 0x7fe4f80a5510 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:40.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.217+0000 7fe519400640 1 --2- 172.21.15.43:0/261895473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe514101b10 0x7fe514197e70 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:40.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.217+0000 7fe519400640 1 -- 172.21.15.43:0/261895473 >> 172.21.15.43:0/261895473 conn(0x7fe5140fd2b0 msgr2=0x7fe514106110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:40.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.217+0000 7fe519400640 1 -- 172.21.15.43:0/261895473 shutdown_connections 2024-09-19T05:04:40.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.217+0000 7fe519400640 1 -- 172.21.15.43:0/261895473 wait complete. 2024-09-19T05:04:40.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:39 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/2623798103' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-19T05:04:40.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:39 smithi096 ceph-mon[35085]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T05:04:40.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:39 smithi096 ceph-mon[35085]: mgrmap e22: x(active, since 1.01998s), standbys: z, y 2024-09-19T05:04:40.233 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-19T05:04:40.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.418+0000 7f980b000640 1 -- 172.21.15.43:0/4209754937 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9804076bb0 msgr2=0x7f9804076fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:40.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.418+0000 7f980b000640 1 --2- 172.21.15.43:0/4209754937 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9804076bb0 0x7f9804076fb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f97f4009900 tx=0x7f97f402a3f0 comp rx=0 tx=0).stop 2024-09-19T05:04:40.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.418+0000 7f980b000640 1 -- 172.21.15.43:0/4209754937 shutdown_connections 2024-09-19T05:04:40.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.418+0000 7f980b000640 1 --2- 172.21.15.43:0/4209754937 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9804078790 0x7f9804079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:40.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.418+0000 7f980b000640 1 --2- 172.21.15.43:0/4209754937 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9804077df0 0x7f9804078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:40.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.418+0000 7f980b000640 1 --2- 172.21.15.43:0/4209754937 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9804076bb0 0x7f9804076fb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:40.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.418+0000 7f980b000640 1 -- 172.21.15.43:0/4209754937 >> 172.21.15.43:0/4209754937 conn(0x7f98040ff470 msgr2=0x7f98041018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:40.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.418+0000 7f980b000640 1 -- 172.21.15.43:0/4209754937 shutdown_connections 2024-09-19T05:04:40.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.418+0000 7f980b000640 1 -- 172.21.15.43:0/4209754937 wait complete. 2024-09-19T05:04:40.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.419+0000 7f980b000640 1 Processor -- start 2024-09-19T05:04:40.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.420+0000 7f980b000640 1 -- start start 2024-09-19T05:04:40.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.420+0000 7f980b000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9804076bb0 0x7f980419e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:40.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.420+0000 7f980b000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9804077df0 0x7f980419e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:40.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.420+0000 7f980b000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9804078790 0x7f98041a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:40.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.420+0000 7f980b000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9804113e10 con 0x7f9804078790 2024-09-19T05:04:40.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.420+0000 7f980b000640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9804113c90 con 0x7f9804076bb0 2024-09-19T05:04:40.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.420+0000 7f980b000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9804113f90 con 0x7f9804077df0 2024-09-19T05:04:40.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.420+0000 7f9803e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9804076bb0 0x7f980419e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:40.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.420+0000 7f9803e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9804076bb0 0x7f980419e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:59396/0 (socket says 172.21.15.43:59396) 2024-09-19T05:04:40.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.420+0000 7f9803e00640 1 -- 172.21.15.43:0/3662732182 learned_addr learned my addr 172.21.15.43:0/3662732182 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:40.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.420+0000 7f9803400640 1 --2- 172.21.15.43:0/3662732182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9804077df0 0x7f980419e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:40.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.420+0000 7f9809000640 1 --2- 172.21.15.43:0/3662732182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9804078790 0x7f98041a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:40.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.421+0000 7f9803e00640 1 -- 172.21.15.43:0/3662732182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9804077df0 msgr2=0x7f980419e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:40.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.421+0000 7f9803e00640 1 --2- 172.21.15.43:0/3662732182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9804077df0 0x7f980419e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:40.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.421+0000 7f9803e00640 1 -- 172.21.15.43:0/3662732182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9804078790 msgr2=0x7f98041a5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:40.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.421+0000 7f9803e00640 1 --2- 172.21.15.43:0/3662732182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9804078790 0x7f98041a5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:40.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.421+0000 7f9803e00640 1 -- 172.21.15.43:0/3662732182 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98041a6090 con 0x7f9804076bb0 2024-09-19T05:04:40.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.421+0000 7f9803e00640 1 --2- 172.21.15.43:0/3662732182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9804076bb0 0x7f980419e470 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f97f4031440 tx=0x7f97f4032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:40.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.421+0000 7f9800c00640 1 -- 172.21.15.43:0/3662732182 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f403f030 con 0x7f9804076bb0 2024-09-19T05:04:40.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.421+0000 7f9800c00640 1 -- 172.21.15.43:0/3662732182 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97f4003ce0 con 0x7f9804076bb0 2024-09-19T05:04:40.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.421+0000 7f980b000640 1 -- 172.21.15.43:0/3662732182 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98041a6320 con 0x7f9804076bb0 2024-09-19T05:04:40.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.421+0000 7f9800c00640 1 -- 172.21.15.43:0/3662732182 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f4035070 con 0x7f9804076bb0 2024-09-19T05:04:40.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.421+0000 7f980b000640 1 -- 172.21.15.43:0/3662732182 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98041a6810 con 0x7f9804076bb0 2024-09-19T05:04:40.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.422+0000 7f980b000640 1 -- 172.21.15.43:0/3662732182 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98041107d0 con 0x7f9804076bb0 2024-09-19T05:04:40.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.426+0000 7f9800c00640 1 -- 172.21.15.43:0/3662732182 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 22) ==== 138167+0+0 (secure 0 0 0) 0x7f97f4042030 con 0x7f9804076bb0 2024-09-19T05:04:40.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.427+0000 7f9800c00640 1 --2- 172.21.15.43:0/3662732182 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f97e80a3050 0x7f97e80a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:40.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.428+0000 7f9800c00640 1 -- 172.21.15.43:0/3662732182 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f97f40efbc0 con 0x7f9804076bb0 2024-09-19T05:04:40.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.428+0000 7f9803400640 1 --2- 172.21.15.43:0/3662732182 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f97e80a3050 0x7f97e80a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:40.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.429+0000 7f9803400640 1 --2- 172.21.15.43:0/3662732182 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f97e80a3050 0x7f97e80a5510 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f97f00095f0 tx=0x7f97f0005f90 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:40.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.432+0000 7f9800c00640 1 -- 172.21.15.43:0/3662732182 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f97f40bca80 con 0x7f9804076bb0 2024-09-19T05:04:40.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.629+0000 7f980b000640 1 -- 172.21.15.43:0/3662732182 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9804062df0 con 0x7f9804076bb0 2024-09-19T05:04:40.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.629+0000 7f9800c00640 1 -- 172.21.15.43:0/3662732182 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) ==== 104+0+11 (secure 0 0 0) 0x7f97f40bc380 con 0x7f9804076bb0 2024-09-19T05:04:40.630 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-09-19T05:04:40.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.632+0000 7f980b000640 1 -- 172.21.15.43:0/3662732182 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f97e80a3050 msgr2=0x7f97e80a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:40.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.632+0000 7f980b000640 1 --2- 172.21.15.43:0/3662732182 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f97e80a3050 0x7f97e80a5510 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f97f00095f0 tx=0x7f97f0005f90 comp rx=0 tx=0).stop 2024-09-19T05:04:40.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.632+0000 7f980b000640 1 -- 172.21.15.43:0/3662732182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9804076bb0 msgr2=0x7f980419e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:40.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.632+0000 7f980b000640 1 --2- 172.21.15.43:0/3662732182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9804076bb0 0x7f980419e470 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f97f4031440 tx=0x7f97f4032ce0 comp rx=0 tx=0).stop 2024-09-19T05:04:40.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.632+0000 7f980b000640 1 -- 172.21.15.43:0/3662732182 shutdown_connections 2024-09-19T05:04:40.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.632+0000 7f980b000640 1 --2- 172.21.15.43:0/3662732182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9804078790 0x7f98041a5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:40.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.632+0000 7f980b000640 1 --2- 172.21.15.43:0/3662732182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9804077df0 0x7f980419e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:40.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.632+0000 7f980b000640 1 --2- 172.21.15.43:0/3662732182 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f97e80a3050 0x7f97e80a5510 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:40.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.632+0000 7f980b000640 1 --2- 172.21.15.43:0/3662732182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9804076bb0 0x7f980419e470 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:40.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.632+0000 7f980b000640 1 -- 172.21.15.43:0/3662732182 >> 172.21.15.43:0/3662732182 conn(0x7f98040ff470 msgr2=0x7f9804100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:40.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.632+0000 7f980b000640 1 -- 172.21.15.43:0/3662732182 shutdown_connections 2024-09-19T05:04:40.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.632+0000 7f980b000640 1 -- 172.21.15.43:0/3662732182 wait complete. 2024-09-19T05:04:40.651 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-19T05:04:40.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.855+0000 7f6a49200640 1 -- 172.21.15.43:0/2606369672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a44076bb0 msgr2=0x7f6a44076fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:40.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.855+0000 7f6a49200640 1 --2- 172.21.15.43:0/2606369672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a44076bb0 0x7f6a44076fb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6a34009860 tx=0x7f6a3402a340 comp rx=0 tx=0).stop 2024-09-19T05:04:40.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.855+0000 7f6a49200640 1 -- 172.21.15.43:0/2606369672 shutdown_connections 2024-09-19T05:04:40.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.855+0000 7f6a49200640 1 --2- 172.21.15.43:0/2606369672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a44078790 0x7f6a44079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:40.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.855+0000 7f6a49200640 1 --2- 172.21.15.43:0/2606369672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a44077df0 0x7f6a44078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:40.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.855+0000 7f6a49200640 1 --2- 172.21.15.43:0/2606369672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a44076bb0 0x7f6a44076fb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:40.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.855+0000 7f6a49200640 1 -- 172.21.15.43:0/2606369672 >> 172.21.15.43:0/2606369672 conn(0x7f6a440ff470 msgr2=0x7f6a441018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:40.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.855+0000 7f6a49200640 1 -- 172.21.15.43:0/2606369672 shutdown_connections 2024-09-19T05:04:40.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.855+0000 7f6a49200640 1 -- 172.21.15.43:0/2606369672 wait complete. 2024-09-19T05:04:40.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.856+0000 7f6a49200640 1 Processor -- start 2024-09-19T05:04:40.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.856+0000 7f6a49200640 1 -- start start 2024-09-19T05:04:40.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.856+0000 7f6a49200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a44076bb0 0x7f6a4419e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:40.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.856+0000 7f6a49200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a44077df0 0x7f6a4419e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:40.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.856+0000 7f6a49200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a44078790 0x7f6a441a5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:40.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.856+0000 7f6a49200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6a44113dc0 con 0x7f6a44077df0 2024-09-19T05:04:40.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.856+0000 7f6a49200640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6a44113c40 con 0x7f6a44076bb0 2024-09-19T05:04:40.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.856+0000 7f6a49200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6a44113f40 con 0x7f6a44078790 2024-09-19T05:04:40.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.857+0000 7f6a41e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a44076bb0 0x7f6a4419e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:40.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.857+0000 7f6a42800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a44078790 0x7f6a441a5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:40.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.857+0000 7f6a42800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a44078790 0x7f6a441a5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:39368/0 (socket says 172.21.15.43:39368) 2024-09-19T05:04:40.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.857+0000 7f6a41400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a44077df0 0x7f6a4419e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:40.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.857+0000 7f6a41e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a44076bb0 0x7f6a4419e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:59424/0 (socket says 172.21.15.43:59424) 2024-09-19T05:04:40.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.857+0000 7f6a42800640 1 -- 172.21.15.43:0/2657062502 learned_addr learned my addr 172.21.15.43:0/2657062502 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:40.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.857+0000 7f6a41e00640 1 -- 172.21.15.43:0/2657062502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a44078790 msgr2=0x7f6a441a5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:40.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.857+0000 7f6a41e00640 1 --2- 172.21.15.43:0/2657062502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a44078790 0x7f6a441a5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:40.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.857+0000 7f6a41e00640 1 -- 172.21.15.43:0/2657062502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a44077df0 msgr2=0x7f6a4419e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:40.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.857+0000 7f6a41e00640 1 --2- 172.21.15.43:0/2657062502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a44077df0 0x7f6a4419e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:40.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.857+0000 7f6a41e00640 1 -- 172.21.15.43:0/2657062502 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a441a60a0 con 0x7f6a44076bb0 2024-09-19T05:04:40.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.857+0000 7f6a41e00640 1 --2- 172.21.15.43:0/2657062502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a44076bb0 0x7f6a4419e480 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6a34035040 tx=0x7f6a34033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:40.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.857+0000 7f6a3aa00640 1 -- 172.21.15.43:0/2657062502 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a34037040 con 0x7f6a44076bb0 2024-09-19T05:04:40.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.858+0000 7f6a3aa00640 1 -- 172.21.15.43:0/2657062502 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a34004040 con 0x7f6a44076bb0 2024-09-19T05:04:40.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.858+0000 7f6a3aa00640 1 -- 172.21.15.43:0/2657062502 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a34005650 con 0x7f6a44076bb0 2024-09-19T05:04:40.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.858+0000 7f6a49200640 1 -- 172.21.15.43:0/2657062502 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a441a6330 con 0x7f6a44076bb0 2024-09-19T05:04:40.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.858+0000 7f6a49200640 1 -- 172.21.15.43:0/2657062502 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a441a67d0 con 0x7f6a44076bb0 2024-09-19T05:04:40.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.859+0000 7f6a49200640 1 -- 172.21.15.43:0/2657062502 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a44110840 con 0x7f6a44076bb0 2024-09-19T05:04:40.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.860+0000 7f6a3aa00640 1 -- 172.21.15.43:0/2657062502 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 23) ==== 138526+0+0 (secure 0 0 0) 0x7f6a34042020 con 0x7f6a44076bb0 2024-09-19T05:04:40.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.862+0000 7f6a3aa00640 1 --2- 172.21.15.43:0/2657062502 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f6a280a34d0 0x7f6a280a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:40.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.862+0000 7f6a3aa00640 1 -- 172.21.15.43:0/2657062502 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f6a340ef6a0 con 0x7f6a44076bb0 2024-09-19T05:04:40.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.862+0000 7f6a41400640 1 --2- 172.21.15.43:0/2657062502 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f6a280a34d0 0x7f6a280a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:40.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.863+0000 7f6a41400640 1 --2- 172.21.15.43:0/2657062502 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f6a280a34d0 0x7f6a280a5990 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6a4419f990 tx=0x7f6a3000b410 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:40.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:40.866+0000 7f6a3aa00640 1 -- 172.21.15.43:0/2657062502 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6a34049020 con 0x7f6a44076bb0 2024-09-19T05:04:41.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:40 smithi043 ceph-mon[30800]: from='client.25190 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:41.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:40 smithi043 ceph-mon[30800]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:04:41.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:40 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/2504992225' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:04:41.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:40 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/261895473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T05:04:41.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:40 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:41.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:40 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:41.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:40 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:41.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:40 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:41.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:40 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/3662732182' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T05:04:41.031 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:40 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:04:40.718+0000 7f81d2a00640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:05:04:40] ENGINE Error in HTTPServer.serve 2024-09-19T05:04:41.031 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:40 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: Traceback (most recent call last): 2024-09-19T05:04:41.032 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:40 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T05:04:41.032 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:40 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: self._connections.run(self.expiration_interval) 2024-09-19T05:04:41.032 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:40 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T05:04:41.032 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:40 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: self._run(expiration_interval) 2024-09-19T05:04:41.032 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:40 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T05:04:41.032 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:40 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T05:04:41.032 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:40 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T05:04:41.032 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:40 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T05:04:41.032 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:40 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T05:04:41.032 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:40 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: s = self.context.wrap_socket( 2024-09-19T05:04:41.032 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:40 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T05:04:41.032 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:40 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: return self.sslsocket_class._create( 2024-09-19T05:04:41.032 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:40 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T05:04:41.033 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:40 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: self.do_handshake() 2024-09-19T05:04:41.033 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:40 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T05:04:41.033 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:40 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: self._sslobj.do_handshake() 2024-09-19T05:04:41.033 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:40 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T05:04:41.033 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:04:40 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:04:41.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:41.070+0000 7f6a49200640 1 -- 172.21.15.43:0/2657062502 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6a44062df0 con 0x7f6a44076bb0 2024-09-19T05:04:41.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:41.071+0000 7f6a3aa00640 1 -- 172.21.15.43:0/2657062502 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) ==== 100+0+11 (secure 0 0 0) 0x7f6a340bbfd0 con 0x7f6a44076bb0 2024-09-19T05:04:41.072 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-09-19T05:04:41.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:41.073+0000 7f6a49200640 1 -- 172.21.15.43:0/2657062502 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f6a280a34d0 msgr2=0x7f6a280a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:41.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:41.073+0000 7f6a49200640 1 --2- 172.21.15.43:0/2657062502 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f6a280a34d0 0x7f6a280a5990 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6a4419f990 tx=0x7f6a3000b410 comp rx=0 tx=0).stop 2024-09-19T05:04:41.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:41.073+0000 7f6a49200640 1 -- 172.21.15.43:0/2657062502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a44076bb0 msgr2=0x7f6a4419e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:41.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:41.073+0000 7f6a49200640 1 --2- 172.21.15.43:0/2657062502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a44076bb0 0x7f6a4419e480 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6a34035040 tx=0x7f6a34033d90 comp rx=0 tx=0).stop 2024-09-19T05:04:41.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:41.074+0000 7f6a49200640 1 -- 172.21.15.43:0/2657062502 shutdown_connections 2024-09-19T05:04:41.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:41.074+0000 7f6a49200640 1 --2- 172.21.15.43:0/2657062502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a44078790 0x7f6a441a5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:41.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:41.074+0000 7f6a49200640 1 --2- 172.21.15.43:0/2657062502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a44077df0 0x7f6a4419e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:41.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:41.074+0000 7f6a49200640 1 --2- 172.21.15.43:0/2657062502 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f6a280a34d0 0x7f6a280a5990 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:41.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:41.074+0000 7f6a49200640 1 --2- 172.21.15.43:0/2657062502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a44076bb0 0x7f6a4419e480 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:41.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:41.074+0000 7f6a49200640 1 -- 172.21.15.43:0/2657062502 >> 172.21.15.43:0/2657062502 conn(0x7f6a440ff470 msgr2=0x7f6a44100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:41.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:41.074+0000 7f6a49200640 1 -- 172.21.15.43:0/2657062502 shutdown_connections 2024-09-19T05:04:41.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:41.074+0000 7f6a49200640 1 -- 172.21.15.43:0/2657062502 wait complete. 2024-09-19T05:04:41.093 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-19T05:04:41.094 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:04:41.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:40 smithi151 ceph-mon[35425]: from='client.25190 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:41.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:40 smithi151 ceph-mon[35425]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:04:41.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:40 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/2504992225' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:04:41.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:40 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/261895473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T05:04:41.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:40 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:41.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:40 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:41.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:40 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:41.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:40 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:41.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:40 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/3662732182' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T05:04:41.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:40 smithi096 ceph-mon[35085]: from='client.25190 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:41.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:40 smithi096 ceph-mon[35085]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:04:41.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:40 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/2504992225' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:04:41.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:40 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/261895473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T05:04:41.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:40 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:41.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:40 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:41.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:40 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:41.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:40 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:41.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:40 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/3662732182' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T05:04:41.488 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:41.900 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:41 smithi043 ceph-mon[30800]: [19/Sep/2024:05:04:40] ENGINE Bus STARTING 2024-09-19T05:04:41.900 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:41 smithi043 ceph-mon[30800]: [19/Sep/2024:05:04:40] ENGINE Serving on https://172.21.15.43:7150 2024-09-19T05:04:41.900 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:41 smithi043 ceph-mon[30800]: [19/Sep/2024:05:04:40] ENGINE Error in HTTPServer.serve 2024-09-19T05:04:41.900 INFO:journalctl@ceph.mon.a.smithi043.stdout: Traceback (most recent call last): 2024-09-19T05:04:41.900 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T05:04:41.900 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._connections.run(self.expiration_interval) 2024-09-19T05:04:41.900 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T05:04:41.900 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._run(expiration_interval) 2024-09-19T05:04:41.900 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T05:04:41.900 INFO:journalctl@ceph.mon.a.smithi043.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T05:04:41.901 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T05:04:41.901 INFO:journalctl@ceph.mon.a.smithi043.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T05:04:41.901 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T05:04:41.901 INFO:journalctl@ceph.mon.a.smithi043.stdout: s = self.context.wrap_socket( 2024-09-19T05:04:41.901 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T05:04:41.901 INFO:journalctl@ceph.mon.a.smithi043.stdout: return self.sslsocket_class._create( 2024-09-19T05:04:41.901 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T05:04:41.901 INFO:journalctl@ceph.mon.a.smithi043.stdout: self.do_handshake() 2024-09-19T05:04:41.901 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T05:04:41.901 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._sslobj.do_handshake() 2024-09-19T05:04:41.901 INFO:journalctl@ceph.mon.a.smithi043.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T05:04:41.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:41 smithi043 ceph-mon[30800]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-09-19T05:04:41.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:41 smithi043 ceph-mon[30800]: [19/Sep/2024:05:04:40] ENGINE Serving on http://172.21.15.43:8765 2024-09-19T05:04:41.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:41 smithi043 ceph-mon[30800]: [19/Sep/2024:05:04:40] ENGINE Bus STARTED 2024-09-19T05:04:41.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:41 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:41.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:41 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:41.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:41 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:41.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:41 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:41.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:41 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:41.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:41 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:41.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:41 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:41.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:41 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:41.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:41 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:41.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:41 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:41.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:41 smithi043 ceph-mon[30800]: Adjusting osd_memory_target on smithi096 to 176.0M 2024-09-19T05:04:41.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:41 smithi043 ceph-mon[30800]: Unable to set osd_memory_target on smithi096 to 184587468: error parsing value: Value '184587468' is below minimum 939524096 2024-09-19T05:04:41.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:41 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/2657062502' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T05:04:41.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:41 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:41.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:41 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:41.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:41 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:41.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:41 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:41.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:41 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:41.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:41 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:41.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:41 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:41.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:41 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:41.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:41 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:41.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:41 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:41.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:41 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:41.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:41 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:42.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:41 smithi151 ceph-mon[35425]: [19/Sep/2024:05:04:40] ENGINE Bus STARTING 2024-09-19T05:04:42.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:41 smithi151 ceph-mon[35425]: [19/Sep/2024:05:04:40] ENGINE Serving on https://172.21.15.43:7150 2024-09-19T05:04:42.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:41 smithi151 ceph-mon[35425]: [19/Sep/2024:05:04:40] ENGINE Error in HTTPServer.serve 2024-09-19T05:04:42.155 INFO:journalctl@ceph.mon.c.smithi151.stdout: Traceback (most recent call last): 2024-09-19T05:04:42.155 INFO:journalctl@ceph.mon.c.smithi151.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T05:04:42.156 INFO:journalctl@ceph.mon.c.smithi151.stdout: self._connections.run(self.expiration_interval) 2024-09-19T05:04:42.156 INFO:journalctl@ceph.mon.c.smithi151.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T05:04:42.156 INFO:journalctl@ceph.mon.c.smithi151.stdout: self._run(expiration_interval) 2024-09-19T05:04:42.156 INFO:journalctl@ceph.mon.c.smithi151.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T05:04:42.156 INFO:journalctl@ceph.mon.c.smithi151.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T05:04:42.156 INFO:journalctl@ceph.mon.c.smithi151.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T05:04:42.156 INFO:journalctl@ceph.mon.c.smithi151.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T05:04:42.156 INFO:journalctl@ceph.mon.c.smithi151.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T05:04:42.156 INFO:journalctl@ceph.mon.c.smithi151.stdout: s = self.context.wrap_socket( 2024-09-19T05:04:42.156 INFO:journalctl@ceph.mon.c.smithi151.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T05:04:42.156 INFO:journalctl@ceph.mon.c.smithi151.stdout: return self.sslsocket_class._create( 2024-09-19T05:04:42.156 INFO:journalctl@ceph.mon.c.smithi151.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T05:04:42.156 INFO:journalctl@ceph.mon.c.smithi151.stdout: self.do_handshake() 2024-09-19T05:04:42.157 INFO:journalctl@ceph.mon.c.smithi151.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T05:04:42.157 INFO:journalctl@ceph.mon.c.smithi151.stdout: self._sslobj.do_handshake() 2024-09-19T05:04:42.157 INFO:journalctl@ceph.mon.c.smithi151.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T05:04:42.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:41 smithi151 ceph-mon[35425]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-09-19T05:04:42.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:41 smithi151 ceph-mon[35425]: [19/Sep/2024:05:04:40] ENGINE Serving on http://172.21.15.43:8765 2024-09-19T05:04:42.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:41 smithi151 ceph-mon[35425]: [19/Sep/2024:05:04:40] ENGINE Bus STARTED 2024-09-19T05:04:42.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:41 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:42.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:41 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:42.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:41 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:42.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:41 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:42.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:41 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:42.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:41 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:42.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:41 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:42.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:41 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:42.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:41 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:42.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:41 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:42.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:41 smithi151 ceph-mon[35425]: Adjusting osd_memory_target on smithi096 to 176.0M 2024-09-19T05:04:42.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:41 smithi151 ceph-mon[35425]: Unable to set osd_memory_target on smithi096 to 184587468: error parsing value: Value '184587468' is below minimum 939524096 2024-09-19T05:04:42.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:41 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/2657062502' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T05:04:42.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:41 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:42.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:41 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:42.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:41 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:42.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:41 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:42.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:41 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:42.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:41 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:42.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:41 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:42.159 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:41 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:42.159 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:41 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:42.159 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:41 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:42.159 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:41 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:42.159 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:41 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:42.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.149+0000 7f11d6400640 1 -- 172.21.15.43:0/3170891457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d0102360 msgr2=0x7f11d010e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:42.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.149+0000 7f11d6400640 1 --2- 172.21.15.43:0/3170891457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d0102360 0x7f11d010e890 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f11c8009cd0 tx=0x7f11c802a8f0 comp rx=0 tx=0).stop 2024-09-19T05:04:42.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.149+0000 7f11d6400640 1 -- 172.21.15.43:0/3170891457 shutdown_connections 2024-09-19T05:04:42.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.149+0000 7f11d6400640 1 --2- 172.21.15.43:0/3170891457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d0102360 0x7f11d010e890 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:42.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.149+0000 7f11d6400640 1 --2- 172.21.15.43:0/3170891457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11d01019c0 0x7f11d0101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:42.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.149+0000 7f11d6400640 1 --2- 172.21.15.43:0/3170891457 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11d01079c0 0x7f11d0107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.149+0000 7f11d6400640 1 -- 172.21.15.43:0/3170891457 >> 172.21.15.43:0/3170891457 conn(0x7f11d00fd740 msgr2=0x7f11d00ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.149+0000 7f11d6400640 1 -- 172.21.15.43:0/3170891457 shutdown_connections 2024-09-19T05:04:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.149+0000 7f11d6400640 1 -- 172.21.15.43:0/3170891457 wait complete. 2024-09-19T05:04:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.151+0000 7f11d6400640 1 Processor -- start 2024-09-19T05:04:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.151+0000 7f11d6400640 1 -- start start 2024-09-19T05:04:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.151+0000 7f11d6400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11d01019c0 0x7f11d0198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.151+0000 7f11d6400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d0102360 0x7f11d01987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.151+0000 7f11d6400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11d01079c0 0x7f11d019cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.151+0000 7f11d6400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f11d0111dc0 con 0x7f11d0102360 2024-09-19T05:04:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.151+0000 7f11d6400640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f11d0111c40 con 0x7f11d01079c0 2024-09-19T05:04:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.151+0000 7f11d6400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f11d0111f40 con 0x7f11d01019c0 2024-09-19T05:04:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.152+0000 7f11cfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d0102360 0x7f11d01987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.152+0000 7f11cfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d0102360 0x7f11d01987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60094/0 (socket says 172.21.15.43:60094) 2024-09-19T05:04:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.152+0000 7f11cfe00640 1 -- 172.21.15.43:0/3906913917 learned_addr learned my addr 172.21.15.43:0/3906913917 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:42.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.152+0000 7f11d5a00640 1 --2- 172.21.15.43:0/3906913917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11d01079c0 0x7f11d019cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:42.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.152+0000 7f11d5000640 1 --2- 172.21.15.43:0/3906913917 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11d01019c0 0x7f11d0198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:42.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.152+0000 7f11cfe00640 1 -- 172.21.15.43:0/3906913917 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11d01019c0 msgr2=0x7f11d0198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:42.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.152+0000 7f11cfe00640 1 --2- 172.21.15.43:0/3906913917 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11d01019c0 0x7f11d0198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:42.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.152+0000 7f11cfe00640 1 -- 172.21.15.43:0/3906913917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11d01079c0 msgr2=0x7f11d019cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:42.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.152+0000 7f11cfe00640 1 --2- 172.21.15.43:0/3906913917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11d01079c0 0x7f11d019cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:42.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.152+0000 7f11cfe00640 1 -- 172.21.15.43:0/3906913917 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d019d210 con 0x7f11d0102360 2024-09-19T05:04:42.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.153+0000 7f11cfe00640 1 --2- 172.21.15.43:0/3906913917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d0102360 0x7f11d01987a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f11bc00ea60 tx=0x7f11bc00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:42.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.153+0000 7f11cd600640 1 -- 172.21.15.43:0/3906913917 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc00cde0 con 0x7f11d0102360 2024-09-19T05:04:42.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.153+0000 7f11d6400640 1 -- 172.21.15.43:0/3906913917 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11d019d500 con 0x7f11d0102360 2024-09-19T05:04:42.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.153+0000 7f11cd600640 1 -- 172.21.15.43:0/3906913917 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11bc0035f0 con 0x7f11d0102360 2024-09-19T05:04:42.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.153+0000 7f11cd600640 1 -- 172.21.15.43:0/3906913917 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc005220 con 0x7f11d0102360 2024-09-19T05:04:42.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.153+0000 7f11d6400640 1 -- 172.21.15.43:0/3906913917 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11d01a4de0 con 0x7f11d0102360 2024-09-19T05:04:42.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.154+0000 7f11d6400640 1 -- 172.21.15.43:0/3906913917 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11d0101e20 con 0x7f11d0102360 2024-09-19T05:04:42.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.157+0000 7f11cd600640 1 -- 172.21.15.43:0/3906913917 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 23) ==== 138526+0+0 (secure 0 0 0) 0x7f11bc0037a0 con 0x7f11d0102360 2024-09-19T05:04:42.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.159+0000 7f11cd600640 1 --2- 172.21.15.43:0/3906913917 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f11b80a3440 0x7f11b80a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:42.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.159+0000 7f11cd600640 1 -- 172.21.15.43:0/3906913917 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f11bc0d00f0 con 0x7f11d0102360 2024-09-19T05:04:42.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.159+0000 7f11d5000640 1 --2- 172.21.15.43:0/3906913917 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f11b80a3440 0x7f11b80a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:42.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.160+0000 7f11d5000640 1 --2- 172.21.15.43:0/3906913917 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f11b80a3440 0x7f11b80a5900 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f11c400bb60 tx=0x7f11c400b390 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:42.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.163+0000 7f11cd600640 1 -- 172.21.15.43:0/3906913917 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f11bc09c910 con 0x7f11d0102360 2024-09-19T05:04:42.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:41 smithi096 ceph-mon[35085]: [19/Sep/2024:05:04:40] ENGINE Bus STARTING 2024-09-19T05:04:42.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:41 smithi096 ceph-mon[35085]: [19/Sep/2024:05:04:40] ENGINE Serving on https://172.21.15.43:7150 2024-09-19T05:04:42.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:41 smithi096 ceph-mon[35085]: [19/Sep/2024:05:04:40] ENGINE Error in HTTPServer.serve 2024-09-19T05:04:42.224 INFO:journalctl@ceph.mon.b.smithi096.stdout: Traceback (most recent call last): 2024-09-19T05:04:42.224 INFO:journalctl@ceph.mon.b.smithi096.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T05:04:42.224 INFO:journalctl@ceph.mon.b.smithi096.stdout: self._connections.run(self.expiration_interval) 2024-09-19T05:04:42.224 INFO:journalctl@ceph.mon.b.smithi096.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T05:04:42.224 INFO:journalctl@ceph.mon.b.smithi096.stdout: self._run(expiration_interval) 2024-09-19T05:04:42.224 INFO:journalctl@ceph.mon.b.smithi096.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T05:04:42.224 INFO:journalctl@ceph.mon.b.smithi096.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T05:04:42.224 INFO:journalctl@ceph.mon.b.smithi096.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T05:04:42.224 INFO:journalctl@ceph.mon.b.smithi096.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T05:04:42.224 INFO:journalctl@ceph.mon.b.smithi096.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T05:04:42.225 INFO:journalctl@ceph.mon.b.smithi096.stdout: s = self.context.wrap_socket( 2024-09-19T05:04:42.225 INFO:journalctl@ceph.mon.b.smithi096.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T05:04:42.225 INFO:journalctl@ceph.mon.b.smithi096.stdout: return self.sslsocket_class._create( 2024-09-19T05:04:42.225 INFO:journalctl@ceph.mon.b.smithi096.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T05:04:42.225 INFO:journalctl@ceph.mon.b.smithi096.stdout: self.do_handshake() 2024-09-19T05:04:42.225 INFO:journalctl@ceph.mon.b.smithi096.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T05:04:42.225 INFO:journalctl@ceph.mon.b.smithi096.stdout: self._sslobj.do_handshake() 2024-09-19T05:04:42.225 INFO:journalctl@ceph.mon.b.smithi096.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T05:04:42.225 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:41 smithi096 ceph-mon[35085]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-09-19T05:04:42.225 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:41 smithi096 ceph-mon[35085]: [19/Sep/2024:05:04:40] ENGINE Serving on http://172.21.15.43:8765 2024-09-19T05:04:42.225 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:41 smithi096 ceph-mon[35085]: [19/Sep/2024:05:04:40] ENGINE Bus STARTED 2024-09-19T05:04:42.225 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:41 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:42.225 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:41 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:42.226 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:41 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:42.226 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:41 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:42.226 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:41 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:42.226 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:41 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:42.226 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:41 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:42.226 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:41 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:42.226 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:41 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:42.226 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:41 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:42.226 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:41 smithi096 ceph-mon[35085]: Adjusting osd_memory_target on smithi096 to 176.0M 2024-09-19T05:04:42.226 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:41 smithi096 ceph-mon[35085]: Unable to set osd_memory_target on smithi096 to 184587468: error parsing value: Value '184587468' is below minimum 939524096 2024-09-19T05:04:42.226 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:41 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/2657062502' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T05:04:42.226 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:41 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:42.226 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:41 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:42.227 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:41 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:42.227 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:41 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:42.227 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:41 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:42.227 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:41 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:42.227 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:41 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:42.227 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:41 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:42.227 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:41 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:42.227 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:41 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:42.227 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:41 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:42.227 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:41 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:42.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.388+0000 7f11d6400640 1 -- 172.21.15.43:0/3906913917 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11d002d7c0 con 0x7f11d0102360 2024-09-19T05:04:42.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.389+0000 7f11cd600640 1 -- 172.21.15.43:0/3906913917 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11965 (secure 0 0 0) 0x7f11bc09c210 con 0x7f11d0102360 2024-09-19T05:04:42.390 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:04:42.390 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":29,"btime":"2024-09-19T05:04:38:785407+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/773801169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":773801169},{"type":"v1","addr":"172.21.15.43:6835","nonce":773801169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1799137902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1799137902},{"type":"v1","addr":"172.21.15.43:6837","nonce":1799137902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:02:25.463041+0000","modified":"2024-09-19T05:04:38.039809+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24518,"mds_2":24511,"mds_3":24506,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14541":{"gid":14541,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/794881135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":794881135},{"type":"v1","addr":"172.21.15.43:6841","nonce":794881135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/2836453158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2836453158},{"type":"v1","addr":"172.21.15.151:6833","nonce":2836453158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/178785812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":178785812},{"type":"v1","addr":"172.21.15.96:6833","nonce":178785812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1671043820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1671043820},{"type":"v1","addr":"172.21.15.96:6835","nonce":1671043820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2972514328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2972514328},{"type":"v1","addr":"172.21.15.151:6835","nonce":2972514328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.151:6837/3960378010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3960378010},{"type":"v1","addr":"172.21.15.151:6837","nonce":3960378010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6837/3978388286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3978388286},{"type":"v1","addr":"172.21.15.96:6837","nonce":3978388286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.151:6839/3794104313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3794104313},{"type":"v1","addr":"172.21.15.151:6839","nonce":3794104313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.43:6839/3404555030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3404555030},{"type":"v1","addr":"172.21.15.43:6839","nonce":3404555030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/605527621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":605527621},{"type":"v1","addr":"172.21.15.96:6839","nonce":605527621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24518,24511,24506,24493]},"id":1}]} 2024-09-19T05:04:42.391 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 29 2024-09-19T05:04:42.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.392+0000 7f11d6400640 1 -- 172.21.15.43:0/3906913917 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f11b80a3440 msgr2=0x7f11b80a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:42.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.392+0000 7f11d6400640 1 --2- 172.21.15.43:0/3906913917 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f11b80a3440 0x7f11b80a5900 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f11c400bb60 tx=0x7f11c400b390 comp rx=0 tx=0).stop 2024-09-19T05:04:42.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.392+0000 7f11d6400640 1 -- 172.21.15.43:0/3906913917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d0102360 msgr2=0x7f11d01987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:42.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.392+0000 7f11d6400640 1 --2- 172.21.15.43:0/3906913917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d0102360 0x7f11d01987a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f11bc00ea60 tx=0x7f11bc00ef20 comp rx=0 tx=0).stop 2024-09-19T05:04:42.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.392+0000 7f11d6400640 1 -- 172.21.15.43:0/3906913917 shutdown_connections 2024-09-19T05:04:42.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.392+0000 7f11d6400640 1 --2- 172.21.15.43:0/3906913917 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f11b80a3440 0x7f11b80a5900 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:42.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.392+0000 7f11d6400640 1 --2- 172.21.15.43:0/3906913917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11d01079c0 0x7f11d019cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:42.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.392+0000 7f11d6400640 1 --2- 172.21.15.43:0/3906913917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d0102360 0x7f11d01987a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:42.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.392+0000 7f11d6400640 1 --2- 172.21.15.43:0/3906913917 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11d01019c0 0x7f11d0198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:42.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.392+0000 7f11d6400640 1 -- 172.21.15.43:0/3906913917 >> 172.21.15.43:0/3906913917 conn(0x7f11d00fd740 msgr2=0x7f11d00fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:42.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.392+0000 7f11d6400640 1 -- 172.21.15.43:0/3906913917 shutdown_connections 2024-09-19T05:04:42.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:42.392+0000 7f11d6400640 1 -- 172.21.15.43:0/3906913917 wait complete. 2024-09-19T05:04:43.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:42 smithi151 ceph-mon[35425]: Adjusting osd_memory_target on smithi151 to 176.0M 2024-09-19T05:04:43.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:42 smithi151 ceph-mon[35425]: Unable to set osd_memory_target on smithi151 to 184591052: error parsing value: Value '184591052' is below minimum 939524096 2024-09-19T05:04:43.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:42 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/3906913917' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:04:43.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:42 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:43.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:42 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:43.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:42 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:43.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:42 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:43.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:42 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:43.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:42 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:43.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:42 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:43.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:42 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:43.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:42 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:43.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:42 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:43.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:42 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:04:43.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:42 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:04:43.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:42 smithi096 ceph-mon[35085]: Adjusting osd_memory_target on smithi151 to 176.0M 2024-09-19T05:04:43.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:42 smithi096 ceph-mon[35085]: Unable to set osd_memory_target on smithi151 to 184591052: error parsing value: Value '184591052' is below minimum 939524096 2024-09-19T05:04:43.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:42 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/3906913917' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:04:43.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:42 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:43.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:42 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:43.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:42 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:43.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:42 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:43.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:42 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:43.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:42 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:43.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:42 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:43.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:42 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:43.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:42 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:43.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:42 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:43.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:42 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:04:43.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:42 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:04:43.251 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:42 smithi043 ceph-mon[30800]: Adjusting osd_memory_target on smithi151 to 176.0M 2024-09-19T05:04:43.251 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:42 smithi043 ceph-mon[30800]: Unable to set osd_memory_target on smithi151 to 184591052: error parsing value: Value '184591052' is below minimum 939524096 2024-09-19T05:04:43.251 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:42 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/3906913917' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:04:43.251 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:42 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:43.251 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:42 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:43.251 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:42 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:43.251 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:42 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:43.251 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:42 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:43.251 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:42 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:43.251 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:42 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:43.252 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:42 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:43.252 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:42 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:43.252 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:42 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:04:43.252 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:42 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:04:43.252 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:42 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:04:43.304 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-19T05:04:43.304 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-19T05:04:43.304 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T05:04:43.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.471+0000 7fa980a00640 1 -- 172.21.15.43:0/3983705589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa97c1001d0 msgr2=0x7fa97c1025e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:43.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.471+0000 7fa980a00640 1 --2- 172.21.15.43:0/3983705589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa97c1001d0 0x7fa97c1025e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa96c009900 tx=0x7fa96c02a410 comp rx=0 tx=0).stop 2024-09-19T05:04:43.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.471+0000 7fa980a00640 1 -- 172.21.15.43:0/3983705589 shutdown_connections 2024-09-19T05:04:43.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.471+0000 7fa980a00640 1 --2- 172.21.15.43:0/3983705589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa97c1054d0 0x7fa97c105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:43.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.471+0000 7fa980a00640 1 --2- 172.21.15.43:0/3983705589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa97c102b20 0x7fa97c104f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:43.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.471+0000 7fa980a00640 1 --2- 172.21.15.43:0/3983705589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa97c1001d0 0x7fa97c1025e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:43.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.471+0000 7fa980a00640 1 -- 172.21.15.43:0/3983705589 >> 172.21.15.43:0/3983705589 conn(0x7fa97c0fc000 msgr2=0x7fa97c0fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:43.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.471+0000 7fa980a00640 1 -- 172.21.15.43:0/3983705589 shutdown_connections 2024-09-19T05:04:43.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.471+0000 7fa980a00640 1 -- 172.21.15.43:0/3983705589 wait complete. 2024-09-19T05:04:43.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.472+0000 7fa980a00640 1 Processor -- start 2024-09-19T05:04:43.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.472+0000 7fa980a00640 1 -- start start 2024-09-19T05:04:43.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.473+0000 7fa980a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa97c1001d0 0x7fa97c19e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:43.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.473+0000 7fa980a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa97c102b20 0x7fa97c19e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:43.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.473+0000 7fa980a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa97c1054d0 0x7fa97c1a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:43.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.473+0000 7fa980a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa97c113c60 con 0x7fa97c102b20 2024-09-19T05:04:43.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.473+0000 7fa980a00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa97c113ae0 con 0x7fa97c1054d0 2024-09-19T05:04:43.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.473+0000 7fa980a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa97c113de0 con 0x7fa97c1001d0 2024-09-19T05:04:43.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.473+0000 7fa97b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa97c1001d0 0x7fa97c19e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:43.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.473+0000 7fa97b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa97c1001d0 0x7fa97c19e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:39398/0 (socket says 172.21.15.43:39398) 2024-09-19T05:04:43.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.473+0000 7fa97b400640 1 -- 172.21.15.43:0/1706908204 learned_addr learned my addr 172.21.15.43:0/1706908204 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:43.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.473+0000 7fa97be00640 1 --2- 172.21.15.43:0/1706908204 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa97c1054d0 0x7fa97c1a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:43.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.473+0000 7fa97aa00640 1 --2- 172.21.15.43:0/1706908204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa97c102b20 0x7fa97c19e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:43.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.473+0000 7fa97b400640 1 -- 172.21.15.43:0/1706908204 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa97c1054d0 msgr2=0x7fa97c1a5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:43.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.473+0000 7fa97b400640 1 --2- 172.21.15.43:0/1706908204 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa97c1054d0 0x7fa97c1a5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:43.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.473+0000 7fa97b400640 1 -- 172.21.15.43:0/1706908204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa97c102b20 msgr2=0x7fa97c19e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:43.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.473+0000 7fa97b400640 1 --2- 172.21.15.43:0/1706908204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa97c102b20 0x7fa97c19e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:43.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.473+0000 7fa97b400640 1 -- 172.21.15.43:0/1706908204 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa97c1a5e80 con 0x7fa97c1001d0 2024-09-19T05:04:43.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.473+0000 7fa97aa00640 1 --2- 172.21.15.43:0/1706908204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa97c102b20 0x7fa97c19e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:04:43.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.473+0000 7fa97be00640 1 --2- 172.21.15.43:0/1706908204 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa97c1054d0 0x7fa97c1a5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:04:43.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.474+0000 7fa97b400640 1 --2- 172.21.15.43:0/1706908204 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa97c1001d0 0x7fa97c19e2d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa96c031440 tx=0x7fa96c032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:43.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.474+0000 7fa973e00640 1 -- 172.21.15.43:0/1706908204 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c03f030 con 0x7fa97c1001d0 2024-09-19T05:04:43.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.474+0000 7fa973e00640 1 -- 172.21.15.43:0/1706908204 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa96c003ce0 con 0x7fa97c1001d0 2024-09-19T05:04:43.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.474+0000 7fa980a00640 1 -- 172.21.15.43:0/1706908204 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa97c1a6110 con 0x7fa97c1001d0 2024-09-19T05:04:43.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.474+0000 7fa973e00640 1 -- 172.21.15.43:0/1706908204 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c035070 con 0x7fa97c1001d0 2024-09-19T05:04:43.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.474+0000 7fa980a00640 1 -- 172.21.15.43:0/1706908204 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa97c1a6650 con 0x7fa97c1001d0 2024-09-19T05:04:43.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.476+0000 7fa980a00640 1 -- 172.21.15.43:0/1706908204 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa97c0fdaf0 con 0x7fa97c1001d0 2024-09-19T05:04:43.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.477+0000 7fa973e00640 1 -- 172.21.15.43:0/1706908204 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7fa96c042030 con 0x7fa97c1001d0 2024-09-19T05:04:43.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.479+0000 7fa973e00640 1 --2- 172.21.15.43:0/1706908204 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fa9600a3560 0x7fa9600a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:43.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.480+0000 7fa973e00640 1 -- 172.21.15.43:0/1706908204 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fa96c0f0430 con 0x7fa97c1001d0 2024-09-19T05:04:43.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.480+0000 7fa97aa00640 1 --2- 172.21.15.43:0/1706908204 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fa9600a3560 0x7fa9600a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:43.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.480+0000 7fa97aa00640 1 --2- 172.21.15.43:0/1706908204 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fa9600a3560 0x7fa9600a5a20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa97c19f770 tx=0x7fa968005f90 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:43.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.483+0000 7fa973e00640 1 -- 172.21.15.43:0/1706908204 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa96c0bcc40 con 0x7fa97c1001d0 2024-09-19T05:04:43.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.696+0000 7fa980a00640 1 -- 172.21.15.43:0/1706908204 --> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa97c062630 con 0x7fa9600a3560 2024-09-19T05:04:43.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.704+0000 7fa973e00640 1 -- 172.21.15.43:0/1706908204 <== mgr.25136 v2:172.21.15.43:6800/2120444093 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258096 (secure 0 0 0) 0x7fa97c062630 con 0x7fa9600a3560 2024-09-19T05:04:43.708 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:04:43.709 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-19T05:04:43.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.708+0000 7fa980a00640 1 -- 172.21.15.43:0/1706908204 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fa9600a3560 msgr2=0x7fa9600a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:43.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.708+0000 7fa980a00640 1 --2- 172.21.15.43:0/1706908204 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fa9600a3560 0x7fa9600a5a20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa97c19f770 tx=0x7fa968005f90 comp rx=0 tx=0).stop 2024-09-19T05:04:43.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.709+0000 7fa980a00640 1 -- 172.21.15.43:0/1706908204 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa97c1001d0 msgr2=0x7fa97c19e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:43.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.709+0000 7fa980a00640 1 --2- 172.21.15.43:0/1706908204 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa97c1001d0 0x7fa97c19e2d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa96c031440 tx=0x7fa96c032ce0 comp rx=0 tx=0).stop 2024-09-19T05:04:43.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.709+0000 7fa980a00640 1 -- 172.21.15.43:0/1706908204 shutdown_connections 2024-09-19T05:04:43.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.709+0000 7fa980a00640 1 --2- 172.21.15.43:0/1706908204 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fa9600a3560 0x7fa9600a5a20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:43.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.709+0000 7fa980a00640 1 --2- 172.21.15.43:0/1706908204 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa97c1054d0 0x7fa97c1a5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:43.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.709+0000 7fa980a00640 1 --2- 172.21.15.43:0/1706908204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa97c102b20 0x7fa97c19e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:43.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.709+0000 7fa980a00640 1 --2- 172.21.15.43:0/1706908204 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa97c1001d0 0x7fa97c19e2d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:43.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.709+0000 7fa980a00640 1 -- 172.21.15.43:0/1706908204 >> 172.21.15.43:0/1706908204 conn(0x7fa97c0fc000 msgr2=0x7fa97c1034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:43.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.709+0000 7fa980a00640 1 -- 172.21.15.43:0/1706908204 shutdown_connections 2024-09-19T05:04:43.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.709+0000 7fa980a00640 1 -- 172.21.15.43:0/1706908204 wait complete. 2024-09-19T05:04:43.729 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-19T05:04:42.261564+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1193702,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1475,"num_read_kb":911,"num_write":564,"num_write_kb":4412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":1181717,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":947,"ondisk_log_size":947,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":377696,"kb_used_data":11784,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124564128,"statfs":{"total":1151940427776,"available":1151553667072,"internally_reserved":0,"allocated":12066816,"data_stored":9078285,"data_compressed":280452,"data_compressed_allocated":368640,"data_compressed_original":3588096,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.982555"},"pg_stats":[{"pgid":"3.3f","version":"85'8","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.343604+0000","last_change":"2024-09-19T05:02:24.426572+0000","last_active":"2024-09-19T05:04:38.343604+0000","last_peered":"2024-09-19T05:04:38.343604+0000","last_clean":"2024-09-19T05:04:38.343604+0000","last_became_active":"2024-09-19T05:02:24.426241+0000","last_became_peered":"2024-09-19T05:02:24.426241+0000","last_unstale":"2024-09-19T05:04:38.343604+0000","last_undegraded":"2024-09-19T05:04:38.343604+0000","last_fullsized":"2024-09-19T05:04:38.343604+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:42.511495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.591304+0000","last_change":"2024-09-19T05:02:24.427263+0000","last_active":"2024-09-19T05:04:38.591304+0000","last_peered":"2024-09-19T05:04:38.591304+0000","last_clean":"2024-09-19T05:04:38.591304+0000","last_became_active":"2024-09-19T05:02:24.426928+0000","last_became_peered":"2024-09-19T05:02:24.426928+0000","last_unstale":"2024-09-19T05:04:38.591304+0000","last_undegraded":"2024-09-19T05:04:38.591304+0000","last_fullsized":"2024-09-19T05:04:38.591304+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:03.177020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.591094+0000","last_change":"2024-09-19T05:02:24.432709+0000","last_active":"2024-09-19T05:04:38.591094+0000","last_peered":"2024-09-19T05:04:38.591094+0000","last_clean":"2024-09-19T05:04:38.591094+0000","last_became_active":"2024-09-19T05:02:24.431114+0000","last_became_peered":"2024-09-19T05:02:24.431114+0000","last_unstale":"2024-09-19T05:04:38.591094+0000","last_undegraded":"2024-09-19T05:04:38.591094+0000","last_fullsized":"2024-09-19T05:04:38.591094+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:57.244919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.340848+0000","last_change":"2024-09-19T05:03:08.713389+0000","last_active":"2024-09-19T05:04:38.340848+0000","last_peered":"2024-09-19T05:04:38.340848+0000","last_clean":"2024-09-19T05:04:38.340848+0000","last_became_active":"2024-09-19T05:03:08.713058+0000","last_became_peered":"2024-09-19T05:03:08.713058+0000","last_unstale":"2024-09-19T05:04:38.340848+0000","last_undegraded":"2024-09-19T05:04:38.340848+0000","last_fullsized":"2024-09-19T05:04:38.340848+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:52:10.993733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587640+0000","last_change":"2024-09-19T05:02:24.419992+0000","last_active":"2024-09-19T05:04:38.587640+0000","last_peered":"2024-09-19T05:04:38.587640+0000","last_clean":"2024-09-19T05:04:38.587640+0000","last_became_active":"2024-09-19T05:02:24.419643+0000","last_became_peered":"2024-09-19T05:02:24.419643+0000","last_unstale":"2024-09-19T05:04:38.587640+0000","last_undegraded":"2024-09-19T05:04:38.587640+0000","last_fullsized":"2024-09-19T05:04:38.587640+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:54:40.548216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.356087+0000","last_change":"2024-09-19T05:02:24.434106+0000","last_active":"2024-09-19T05:04:38.356087+0000","last_peered":"2024-09-19T05:04:38.356087+0000","last_clean":"2024-09-19T05:04:38.356087+0000","last_became_active":"2024-09-19T05:02:24.433124+0000","last_became_peered":"2024-09-19T05:02:24.433124+0000","last_unstale":"2024-09-19T05:04:38.356087+0000","last_undegraded":"2024-09-19T05:04:38.356087+0000","last_fullsized":"2024-09-19T05:04:38.356087+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:09:46.502506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587605+0000","last_change":"2024-09-19T05:02:24.429640+0000","last_active":"2024-09-19T05:04:38.587605+0000","last_peered":"2024-09-19T05:04:38.587605+0000","last_clean":"2024-09-19T05:04:38.587605+0000","last_became_active":"2024-09-19T05:02:24.429326+0000","last_became_peered":"2024-09-19T05:02:24.429326+0000","last_unstale":"2024-09-19T05:04:38.587605+0000","last_undegraded":"2024-09-19T05:04:38.587605+0000","last_fullsized":"2024-09-19T05:04:38.587605+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:27.330449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.243840+0000","last_change":"2024-09-19T05:02:24.430933+0000","last_active":"2024-09-19T05:04:38.243840+0000","last_peered":"2024-09-19T05:04:38.243840+0000","last_clean":"2024-09-19T05:04:38.243840+0000","last_became_active":"2024-09-19T05:02:24.430478+0000","last_became_peered":"2024-09-19T05:02:24.430478+0000","last_unstale":"2024-09-19T05:04:38.243840+0000","last_undegraded":"2024-09-19T05:04:38.243840+0000","last_fullsized":"2024-09-19T05:04:38.243840+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:45:22.171049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.585189+0000","last_change":"2024-09-19T05:02:24.433207+0000","last_active":"2024-09-19T05:04:38.585189+0000","last_peered":"2024-09-19T05:04:38.585189+0000","last_clean":"2024-09-19T05:04:38.585189+0000","last_became_active":"2024-09-19T05:02:24.432882+0000","last_became_peered":"2024-09-19T05:02:24.432882+0000","last_unstale":"2024-09-19T05:04:38.585189+0000","last_undegraded":"2024-09-19T05:04:38.585189+0000","last_fullsized":"2024-09-19T05:04:38.585189+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:06:42.748227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.356558+0000","last_change":"2024-09-19T05:02:24.433825+0000","last_active":"2024-09-19T05:04:38.356558+0000","last_peered":"2024-09-19T05:04:38.356558+0000","last_clean":"2024-09-19T05:04:38.356558+0000","last_became_active":"2024-09-19T05:02:24.433088+0000","last_became_peered":"2024-09-19T05:02:24.433088+0000","last_unstale":"2024-09-19T05:04:38.356558+0000","last_undegraded":"2024-09-19T05:04:38.356558+0000","last_fullsized":"2024-09-19T05:04:38.356558+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:12:53.039028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587972+0000","last_change":"2024-09-19T05:02:24.427661+0000","last_active":"2024-09-19T05:04:38.587972+0000","last_peered":"2024-09-19T05:04:38.587972+0000","last_clean":"2024-09-19T05:04:38.587972+0000","last_became_active":"2024-09-19T05:02:24.427093+0000","last_became_peered":"2024-09-19T05:02:24.427093+0000","last_unstale":"2024-09-19T05:04:38.587972+0000","last_undegraded":"2024-09-19T05:04:38.587972+0000","last_fullsized":"2024-09-19T05:04:38.587972+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:43.091041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.245390+0000","last_change":"2024-09-19T05:02:24.430820+0000","last_active":"2024-09-19T05:04:38.245390+0000","last_peered":"2024-09-19T05:04:38.245390+0000","last_clean":"2024-09-19T05:04:38.245390+0000","last_became_active":"2024-09-19T05:02:24.429945+0000","last_became_peered":"2024-09-19T05:02:24.429945+0000","last_unstale":"2024-09-19T05:04:38.245390+0000","last_undegraded":"2024-09-19T05:04:38.245390+0000","last_fullsized":"2024-09-19T05:04:38.245390+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:21:59.400725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.564024+0000","last_change":"2024-09-19T05:02:24.428695+0000","last_active":"2024-09-19T05:04:38.564024+0000","last_peered":"2024-09-19T05:04:38.564024+0000","last_clean":"2024-09-19T05:04:38.564024+0000","last_became_active":"2024-09-19T05:02:24.426385+0000","last_became_peered":"2024-09-19T05:02:24.426385+0000","last_unstale":"2024-09-19T05:04:38.564024+0000","last_undegraded":"2024-09-19T05:04:38.564024+0000","last_fullsized":"2024-09-19T05:04:38.564024+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:44:00.893313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.590202+0000","last_change":"2024-09-19T05:02:24.424935+0000","last_active":"2024-09-19T05:04:38.590202+0000","last_peered":"2024-09-19T05:04:38.590202+0000","last_clean":"2024-09-19T05:04:38.590202+0000","last_became_active":"2024-09-19T05:02:24.424219+0000","last_became_peered":"2024-09-19T05:02:24.424219+0000","last_unstale":"2024-09-19T05:04:38.590202+0000","last_undegraded":"2024-09-19T05:04:38.590202+0000","last_fullsized":"2024-09-19T05:04:38.590202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:00.890778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.589476+0000","last_change":"2024-09-19T05:02:24.425407+0000","last_active":"2024-09-19T05:04:38.589476+0000","last_peered":"2024-09-19T05:04:38.589476+0000","last_clean":"2024-09-19T05:04:38.589476+0000","last_became_active":"2024-09-19T05:02:24.425053+0000","last_became_peered":"2024-09-19T05:02:24.425053+0000","last_unstale":"2024-09-19T05:04:38.589476+0000","last_undegraded":"2024-09-19T05:04:38.589476+0000","last_fullsized":"2024-09-19T05:04:38.589476+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:54:10.024183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.584494+0000","last_change":"2024-09-19T05:02:24.427888+0000","last_active":"2024-09-19T05:04:38.584494+0000","last_peered":"2024-09-19T05:04:38.584494+0000","last_clean":"2024-09-19T05:04:38.584494+0000","last_became_active":"2024-09-19T05:02:24.427589+0000","last_became_peered":"2024-09-19T05:02:24.427589+0000","last_unstale":"2024-09-19T05:04:38.584494+0000","last_undegraded":"2024-09-19T05:04:38.584494+0000","last_fullsized":"2024-09-19T05:04:38.584494+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:02:02.654416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.343559+0000","last_change":"2024-09-19T05:02:24.426654+0000","last_active":"2024-09-19T05:04:38.343559+0000","last_peered":"2024-09-19T05:04:38.343559+0000","last_clean":"2024-09-19T05:04:38.343559+0000","last_became_active":"2024-09-19T05:02:24.426352+0000","last_became_peered":"2024-09-19T05:02:24.426352+0000","last_unstale":"2024-09-19T05:04:38.343559+0000","last_undegraded":"2024-09-19T05:04:38.343559+0000","last_fullsized":"2024-09-19T05:04:38.343559+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:20.055222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.340926+0000","last_change":"2024-09-19T05:02:24.432178+0000","last_active":"2024-09-19T05:04:38.340926+0000","last_peered":"2024-09-19T05:04:38.340926+0000","last_clean":"2024-09-19T05:04:38.340926+0000","last_became_active":"2024-09-19T05:02:24.429058+0000","last_became_peered":"2024-09-19T05:02:24.429058+0000","last_unstale":"2024-09-19T05:04:38.340926+0000","last_undegraded":"2024-09-19T05:04:38.340926+0000","last_fullsized":"2024-09-19T05:04:38.340926+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:08.001134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.455282+0000","last_change":"2024-09-19T05:02:24.428202+0000","last_active":"2024-09-19T05:04:38.455282+0000","last_peered":"2024-09-19T05:04:38.455282+0000","last_clean":"2024-09-19T05:04:38.455282+0000","last_became_active":"2024-09-19T05:02:24.427741+0000","last_became_peered":"2024-09-19T05:02:24.427741+0000","last_unstale":"2024-09-19T05:04:38.455282+0000","last_undegraded":"2024-09-19T05:04:38.455282+0000","last_fullsized":"2024-09-19T05:04:38.455282+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:42:51.076056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.343183+0000","last_change":"2024-09-19T05:02:24.425007+0000","last_active":"2024-09-19T05:04:38.343183+0000","last_peered":"2024-09-19T05:04:38.343183+0000","last_clean":"2024-09-19T05:04:38.343183+0000","last_became_active":"2024-09-19T05:02:24.424612+0000","last_became_peered":"2024-09-19T05:02:24.424612+0000","last_unstale":"2024-09-19T05:04:38.343183+0000","last_undegraded":"2024-09-19T05:04:38.343183+0000","last_fullsized":"2024-09-19T05:04:38.343183+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:36:06.115757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.564225+0000","last_change":"2024-09-19T05:02:24.428724+0000","last_active":"2024-09-19T05:04:38.564225+0000","last_peered":"2024-09-19T05:04:38.564225+0000","last_clean":"2024-09-19T05:04:38.564225+0000","last_became_active":"2024-09-19T05:02:24.427730+0000","last_became_peered":"2024-09-19T05:02:24.427730+0000","last_unstale":"2024-09-19T05:04:38.564225+0000","last_undegraded":"2024-09-19T05:04:38.564225+0000","last_fullsized":"2024-09-19T05:04:38.564225+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:15:25.466553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.584736+0000","last_change":"2024-09-19T05:02:24.426238+0000","last_active":"2024-09-19T05:04:38.584736+0000","last_peered":"2024-09-19T05:04:38.584736+0000","last_clean":"2024-09-19T05:04:38.584736+0000","last_became_active":"2024-09-19T05:02:24.425915+0000","last_became_peered":"2024-09-19T05:02:24.425915+0000","last_unstale":"2024-09-19T05:04:38.584736+0000","last_undegraded":"2024-09-19T05:04:38.584736+0000","last_fullsized":"2024-09-19T05:04:38.584736+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:37:18.444042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.243918+0000","last_change":"2024-09-19T05:02:24.425726+0000","last_active":"2024-09-19T05:04:38.243918+0000","last_peered":"2024-09-19T05:04:38.243918+0000","last_clean":"2024-09-19T05:04:38.243918+0000","last_became_active":"2024-09-19T05:02:24.425409+0000","last_became_peered":"2024-09-19T05:02:24.425409+0000","last_unstale":"2024-09-19T05:04:38.243918+0000","last_undegraded":"2024-09-19T05:04:38.243918+0000","last_fullsized":"2024-09-19T05:04:38.243918+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:12.068304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.245729+0000","last_change":"2024-09-19T05:02:24.425267+0000","last_active":"2024-09-19T05:04:38.245729+0000","last_peered":"2024-09-19T05:04:38.245729+0000","last_clean":"2024-09-19T05:04:38.245729+0000","last_became_active":"2024-09-19T05:02:24.424933+0000","last_became_peered":"2024-09-19T05:02:24.424933+0000","last_unstale":"2024-09-19T05:04:38.245729+0000","last_undegraded":"2024-09-19T05:04:38.245729+0000","last_fullsized":"2024-09-19T05:04:38.245729+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:51:55.274870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.244581+0000","last_change":"2024-09-19T05:02:24.427909+0000","last_active":"2024-09-19T05:04:38.244581+0000","last_peered":"2024-09-19T05:04:38.244581+0000","last_clean":"2024-09-19T05:04:38.244581+0000","last_became_active":"2024-09-19T05:02:24.425515+0000","last_became_peered":"2024-09-19T05:02:24.425515+0000","last_unstale":"2024-09-19T05:04:38.244581+0000","last_undegraded":"2024-09-19T05:04:38.244581+0000","last_fullsized":"2024-09-19T05:04:38.244581+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:12:22.390754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.455426+0000","last_change":"2024-09-19T05:02:24.426201+0000","last_active":"2024-09-19T05:04:38.455426+0000","last_peered":"2024-09-19T05:04:38.455426+0000","last_clean":"2024-09-19T05:04:38.455426+0000","last_became_active":"2024-09-19T05:02:24.425890+0000","last_became_peered":"2024-09-19T05:02:24.425890+0000","last_unstale":"2024-09-19T05:04:38.455426+0000","last_undegraded":"2024-09-19T05:04:38.455426+0000","last_fullsized":"2024-09-19T05:04:38.455426+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:36.680002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.343493+0000","last_change":"2024-09-19T05:02:24.416037+0000","last_active":"2024-09-19T05:04:38.343493+0000","last_peered":"2024-09-19T05:04:38.343493+0000","last_clean":"2024-09-19T05:04:38.343493+0000","last_became_active":"2024-09-19T05:02:24.415535+0000","last_became_peered":"2024-09-19T05:02:24.415535+0000","last_unstale":"2024-09-19T05:04:38.343493+0000","last_undegraded":"2024-09-19T05:04:38.343493+0000","last_fullsized":"2024-09-19T05:04:38.343493+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:14.756410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.564248+0000","last_change":"2024-09-19T05:03:08.708063+0000","last_active":"2024-09-19T05:04:38.564248+0000","last_peered":"2024-09-19T05:04:38.564248+0000","last_clean":"2024-09-19T05:04:38.564248+0000","last_became_active":"2024-09-19T05:03:08.707474+0000","last_became_peered":"2024-09-19T05:03:08.707474+0000","last_unstale":"2024-09-19T05:04:38.564248+0000","last_undegraded":"2024-09-19T05:04:38.564248+0000","last_fullsized":"2024-09-19T05:04:38.564248+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:20:55.604156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.589950+0000","last_change":"2024-09-19T05:02:24.423776+0000","last_active":"2024-09-19T05:04:38.589950+0000","last_peered":"2024-09-19T05:04:38.589950+0000","last_clean":"2024-09-19T05:04:38.589950+0000","last_became_active":"2024-09-19T05:02:24.423446+0000","last_became_peered":"2024-09-19T05:02:24.423446+0000","last_unstale":"2024-09-19T05:04:38.589950+0000","last_undegraded":"2024-09-19T05:04:38.589950+0000","last_fullsized":"2024-09-19T05:04:38.589950+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:47:02.747049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.455107+0000","last_change":"2024-09-19T05:02:24.420534+0000","last_active":"2024-09-19T05:04:38.455107+0000","last_peered":"2024-09-19T05:04:38.455107+0000","last_clean":"2024-09-19T05:04:38.455107+0000","last_became_active":"2024-09-19T05:02:24.419980+0000","last_became_peered":"2024-09-19T05:02:24.419980+0000","last_unstale":"2024-09-19T05:04:38.455107+0000","last_undegraded":"2024-09-19T05:04:38.455107+0000","last_fullsized":"2024-09-19T05:04:38.455107+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:33:49.537258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.455289+0000","last_change":"2024-09-19T05:02:24.424859+0000","last_active":"2024-09-19T05:04:38.455289+0000","last_peered":"2024-09-19T05:04:38.455289+0000","last_clean":"2024-09-19T05:04:38.455289+0000","last_became_active":"2024-09-19T05:02:24.424514+0000","last_became_peered":"2024-09-19T05:02:24.424514+0000","last_unstale":"2024-09-19T05:04:38.455289+0000","last_undegraded":"2024-09-19T05:04:38.455289+0000","last_fullsized":"2024-09-19T05:04:38.455289+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:41:41.929578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.343253+0000","last_change":"2024-09-19T05:02:24.421756+0000","last_active":"2024-09-19T05:04:38.343253+0000","last_peered":"2024-09-19T05:04:38.343253+0000","last_clean":"2024-09-19T05:04:38.343253+0000","last_became_active":"2024-09-19T05:02:24.420910+0000","last_became_peered":"2024-09-19T05:02:24.420910+0000","last_unstale":"2024-09-19T05:04:38.343253+0000","last_undegraded":"2024-09-19T05:04:38.343253+0000","last_fullsized":"2024-09-19T05:04:38.343253+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:33.759745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587628+0000","last_change":"2024-09-19T05:02:24.428514+0000","last_active":"2024-09-19T05:04:38.587628+0000","last_peered":"2024-09-19T05:04:38.587628+0000","last_clean":"2024-09-19T05:04:38.587628+0000","last_became_active":"2024-09-19T05:02:24.428206+0000","last_became_peered":"2024-09-19T05:02:24.428206+0000","last_unstale":"2024-09-19T05:04:38.587628+0000","last_undegraded":"2024-09-19T05:04:38.587628+0000","last_fullsized":"2024-09-19T05:04:38.587628+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:33.045739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"85'7","reported_seq":157,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.246167+0000","last_change":"2024-09-19T05:02:22.026683+0000","last_active":"2024-09-19T05:04:38.246167+0000","last_peered":"2024-09-19T05:04:38.246167+0000","last_clean":"2024-09-19T05:04:38.246167+0000","last_became_active":"2024-09-19T05:02:22.026342+0000","last_became_peered":"2024-09-19T05:02:22.026342+0000","last_unstale":"2024-09-19T05:04:38.246167+0000","last_undegraded":"2024-09-19T05:04:38.246167+0000","last_fullsized":"2024-09-19T05:04:38.246167+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:01:37.462838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.245923+0000","last_change":"2024-09-19T05:02:24.421753+0000","last_active":"2024-09-19T05:04:38.245923+0000","last_peered":"2024-09-19T05:04:38.245923+0000","last_clean":"2024-09-19T05:04:38.245923+0000","last_became_active":"2024-09-19T05:02:24.421455+0000","last_became_peered":"2024-09-19T05:02:24.421455+0000","last_unstale":"2024-09-19T05:04:38.245923+0000","last_undegraded":"2024-09-19T05:04:38.245923+0000","last_fullsized":"2024-09-19T05:04:38.245923+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:06.416550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.356217+0000","last_change":"2024-09-19T05:03:08.716866+0000","last_active":"2024-09-19T05:04:38.356217+0000","last_peered":"2024-09-19T05:04:38.356217+0000","last_clean":"2024-09-19T05:04:38.356217+0000","last_became_active":"2024-09-19T05:03:08.709783+0000","last_became_peered":"2024-09-19T05:03:08.709783+0000","last_unstale":"2024-09-19T05:04:38.356217+0000","last_undegraded":"2024-09-19T05:04:38.356217+0000","last_fullsized":"2024-09-19T05:04:38.356217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:26.103651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.563762+0000","last_change":"2024-09-19T05:02:22.026599+0000","last_active":"2024-09-19T05:04:38.563762+0000","last_peered":"2024-09-19T05:04:38.563762+0000","last_clean":"2024-09-19T05:04:38.563762+0000","last_became_active":"2024-09-19T05:02:22.024652+0000","last_became_peered":"2024-09-19T05:02:22.024652+0000","last_unstale":"2024-09-19T05:04:38.563762+0000","last_undegraded":"2024-09-19T05:04:38.563762+0000","last_fullsized":"2024-09-19T05:04:38.563762+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:14:32.508333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.243898+0000","last_change":"2024-09-19T05:02:24.419262+0000","last_active":"2024-09-19T05:04:38.243898+0000","last_peered":"2024-09-19T05:04:38.243898+0000","last_clean":"2024-09-19T05:04:38.243898+0000","last_became_active":"2024-09-19T05:02:24.418589+0000","last_became_peered":"2024-09-19T05:02:24.418589+0000","last_unstale":"2024-09-19T05:04:38.243898+0000","last_undegraded":"2024-09-19T05:04:38.243898+0000","last_fullsized":"2024-09-19T05:04:38.243898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:27.256676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.245917+0000","last_change":"2024-09-19T05:02:22.024225+0000","last_active":"2024-09-19T05:04:38.245917+0000","last_peered":"2024-09-19T05:04:38.245917+0000","last_clean":"2024-09-19T05:04:38.245917+0000","last_became_active":"2024-09-19T05:02:22.023882+0000","last_became_peered":"2024-09-19T05:02:22.023882+0000","last_unstale":"2024-09-19T05:04:38.245917+0000","last_undegraded":"2024-09-19T05:04:38.245917+0000","last_fullsized":"2024-09-19T05:04:38.245917+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:29.619634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.340536+0000","last_change":"2024-09-19T05:02:24.436511+0000","last_active":"2024-09-19T05:04:38.340536+0000","last_peered":"2024-09-19T05:04:38.340536+0000","last_clean":"2024-09-19T05:04:38.340536+0000","last_became_active":"2024-09-19T05:02:24.435565+0000","last_became_peered":"2024-09-19T05:02:24.435565+0000","last_unstale":"2024-09-19T05:04:38.340536+0000","last_undegraded":"2024-09-19T05:04:38.340536+0000","last_fullsized":"2024-09-19T05:04:38.340536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:16:43.779420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'7","reported_seq":273,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.683927+0000","last_change":"2024-09-19T05:02:22.024281+0000","last_active":"2024-09-19T05:04:38.683927+0000","last_peered":"2024-09-19T05:04:38.683927+0000","last_clean":"2024-09-19T05:04:38.683927+0000","last_became_active":"2024-09-19T05:02:22.023868+0000","last_became_peered":"2024-09-19T05:02:22.023868+0000","last_unstale":"2024-09-19T05:04:38.683927+0000","last_undegraded":"2024-09-19T05:04:38.683927+0000","last_fullsized":"2024-09-19T05:04:38.683927+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:29:47.597570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":240,"num_read_kb":240,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.584958+0000","last_change":"2024-09-19T05:02:24.436383+0000","last_active":"2024-09-19T05:04:38.584958+0000","last_peered":"2024-09-19T05:04:38.584958+0000","last_clean":"2024-09-19T05:04:38.584958+0000","last_became_active":"2024-09-19T05:02:24.436062+0000","last_became_peered":"2024-09-19T05:02:24.436062+0000","last_unstale":"2024-09-19T05:04:38.584958+0000","last_undegraded":"2024-09-19T05:04:38.584958+0000","last_fullsized":"2024-09-19T05:04:38.584958+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:48:42.574431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587817+0000","last_change":"2024-09-19T05:02:22.026950+0000","last_active":"2024-09-19T05:04:38.587817+0000","last_peered":"2024-09-19T05:04:38.587817+0000","last_clean":"2024-09-19T05:04:38.587817+0000","last_became_active":"2024-09-19T05:02:22.026498+0000","last_became_peered":"2024-09-19T05:02:22.026498+0000","last_unstale":"2024-09-19T05:04:38.587817+0000","last_undegraded":"2024-09-19T05:04:38.587817+0000","last_fullsized":"2024-09-19T05:04:38.587817+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:55:31.833714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.591864+0000","last_change":"2024-09-19T05:03:08.897727+0000","last_active":"2024-09-19T05:04:38.591864+0000","last_peered":"2024-09-19T05:04:38.591864+0000","last_clean":"2024-09-19T05:04:38.591864+0000","last_became_active":"2024-09-19T05:03:08.897382+0000","last_became_peered":"2024-09-19T05:03:08.897382+0000","last_unstale":"2024-09-19T05:04:38.591864+0000","last_undegraded":"2024-09-19T05:04:38.591864+0000","last_fullsized":"2024-09-19T05:04:38.591864+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:13.319291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.246219+0000","last_change":"2024-09-19T05:02:22.075901+0000","last_active":"2024-09-19T05:04:38.246219+0000","last_peered":"2024-09-19T05:04:38.246219+0000","last_clean":"2024-09-19T05:04:38.246219+0000","last_became_active":"2024-09-19T05:02:22.075512+0000","last_became_peered":"2024-09-19T05:02:22.075512+0000","last_unstale":"2024-09-19T05:04:38.246219+0000","last_undegraded":"2024-09-19T05:04:38.246219+0000","last_fullsized":"2024-09-19T05:04:38.246219+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:09:26.976888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.584706+0000","last_change":"2024-09-19T05:02:24.436797+0000","last_active":"2024-09-19T05:04:38.584706+0000","last_peered":"2024-09-19T05:04:38.584706+0000","last_clean":"2024-09-19T05:04:38.584706+0000","last_became_active":"2024-09-19T05:02:24.436457+0000","last_became_peered":"2024-09-19T05:02:24.436457+0000","last_unstale":"2024-09-19T05:04:38.584706+0000","last_undegraded":"2024-09-19T05:04:38.584706+0000","last_fullsized":"2024-09-19T05:04:38.584706+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:32:44.702601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'39","reported_seq":317,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.705059+0000","last_change":"2024-09-19T05:02:22.027415+0000","last_active":"2024-09-19T05:04:38.705059+0000","last_peered":"2024-09-19T05:04:38.705059+0000","last_clean":"2024-09-19T05:04:38.705059+0000","last_became_active":"2024-09-19T05:02:22.027065+0000","last_became_peered":"2024-09-19T05:02:22.027065+0000","last_unstale":"2024-09-19T05:04:38.705059+0000","last_undegraded":"2024-09-19T05:04:38.705059+0000","last_fullsized":"2024-09-19T05:04:38.705059+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:02.599310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68547,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":252,"num_read_kb":192,"num_write":41,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587855+0000","last_change":"2024-09-19T05:02:24.419718+0000","last_active":"2024-09-19T05:04:38.587855+0000","last_peered":"2024-09-19T05:04:38.587855+0000","last_clean":"2024-09-19T05:04:38.587855+0000","last_became_active":"2024-09-19T05:02:24.419388+0000","last_became_peered":"2024-09-19T05:02:24.419388+0000","last_unstale":"2024-09-19T05:04:38.587855+0000","last_undegraded":"2024-09-19T05:04:38.587855+0000","last_fullsized":"2024-09-19T05:04:38.587855+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:09.453551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.455176+0000","last_change":"2024-09-19T05:02:22.026220+0000","last_active":"2024-09-19T05:04:38.455176+0000","last_peered":"2024-09-19T05:04:38.455176+0000","last_clean":"2024-09-19T05:04:38.455176+0000","last_became_active":"2024-09-19T05:02:22.025838+0000","last_became_peered":"2024-09-19T05:02:22.025838+0000","last_unstale":"2024-09-19T05:04:38.455176+0000","last_undegraded":"2024-09-19T05:04:38.455176+0000","last_fullsized":"2024-09-19T05:04:38.455176+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:02.192799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.563968+0000","last_change":"2024-09-19T05:02:24.431872+0000","last_active":"2024-09-19T05:04:38.563968+0000","last_peered":"2024-09-19T05:04:38.563968+0000","last_clean":"2024-09-19T05:04:38.563968+0000","last_became_active":"2024-09-19T05:02:24.427227+0000","last_became_peered":"2024-09-19T05:02:24.427227+0000","last_unstale":"2024-09-19T05:04:38.563968+0000","last_undegraded":"2024-09-19T05:04:38.563968+0000","last_fullsized":"2024-09-19T05:04:38.563968+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:52.655108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.245966+0000","last_change":"2024-09-19T05:02:22.025136+0000","last_active":"2024-09-19T05:04:38.245966+0000","last_peered":"2024-09-19T05:04:38.245966+0000","last_clean":"2024-09-19T05:04:38.245966+0000","last_became_active":"2024-09-19T05:02:22.024813+0000","last_became_peered":"2024-09-19T05:02:22.024813+0000","last_unstale":"2024-09-19T05:04:38.245966+0000","last_undegraded":"2024-09-19T05:04:38.245966+0000","last_fullsized":"2024-09-19T05:04:38.245966+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:47:03.956661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587730+0000","last_change":"2024-09-19T05:02:24.428497+0000","last_active":"2024-09-19T05:04:38.587730+0000","last_peered":"2024-09-19T05:04:38.587730+0000","last_clean":"2024-09-19T05:04:38.587730+0000","last_became_active":"2024-09-19T05:02:24.427774+0000","last_became_peered":"2024-09-19T05:02:24.427774+0000","last_unstale":"2024-09-19T05:04:38.587730+0000","last_undegraded":"2024-09-19T05:04:38.587730+0000","last_fullsized":"2024-09-19T05:04:38.587730+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:32:02.160354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'8","reported_seq":174,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.845746+0000","last_change":"2024-09-19T05:03:08.718249+0000","last_active":"2024-09-19T05:04:38.845746+0000","last_peered":"2024-09-19T05:04:38.845746+0000","last_clean":"2024-09-19T05:04:38.845746+0000","last_became_active":"2024-09-19T05:03:08.713169+0000","last_became_peered":"2024-09-19T05:03:08.713169+0000","last_unstale":"2024-09-19T05:04:38.845746+0000","last_undegraded":"2024-09-19T05:04:38.845746+0000","last_fullsized":"2024-09-19T05:04:38.845746+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:34.640353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1515,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":981,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,10],"acting":[4,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587563+0000","last_change":"2024-09-19T05:02:24.433098+0000","last_active":"2024-09-19T05:04:38.587563+0000","last_peered":"2024-09-19T05:04:38.587563+0000","last_clean":"2024-09-19T05:04:38.587563+0000","last_became_active":"2024-09-19T05:02:24.432800+0000","last_became_peered":"2024-09-19T05:02:24.432800+0000","last_unstale":"2024-09-19T05:04:38.587563+0000","last_undegraded":"2024-09-19T05:04:38.587563+0000","last_fullsized":"2024-09-19T05:04:38.587563+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:50:30.163842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587677+0000","last_change":"2024-09-19T05:02:22.075865+0000","last_active":"2024-09-19T05:04:38.587677+0000","last_peered":"2024-09-19T05:04:38.587677+0000","last_clean":"2024-09-19T05:04:38.587677+0000","last_became_active":"2024-09-19T05:02:22.075185+0000","last_became_peered":"2024-09-19T05:02:22.075185+0000","last_unstale":"2024-09-19T05:04:38.587677+0000","last_undegraded":"2024-09-19T05:04:38.587677+0000","last_fullsized":"2024-09-19T05:04:38.587677+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:19.150155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.245715+0000","last_change":"2024-09-19T05:02:24.422759+0000","last_active":"2024-09-19T05:04:38.245715+0000","last_peered":"2024-09-19T05:04:38.245715+0000","last_clean":"2024-09-19T05:04:38.245715+0000","last_became_active":"2024-09-19T05:02:24.422421+0000","last_became_peered":"2024-09-19T05:02:24.422421+0000","last_unstale":"2024-09-19T05:04:38.245715+0000","last_undegraded":"2024-09-19T05:04:38.245715+0000","last_fullsized":"2024-09-19T05:04:38.245715+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:58:50.083017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.588067+0000","last_change":"2024-09-19T05:02:22.024305+0000","last_active":"2024-09-19T05:04:38.588067+0000","last_peered":"2024-09-19T05:04:38.588067+0000","last_clean":"2024-09-19T05:04:38.588067+0000","last_became_active":"2024-09-19T05:02:22.023616+0000","last_became_peered":"2024-09-19T05:02:22.023616+0000","last_unstale":"2024-09-19T05:04:38.588067+0000","last_undegraded":"2024-09-19T05:04:38.588067+0000","last_fullsized":"2024-09-19T05:04:38.588067+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:05.665219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.343423+0000","last_change":"2024-09-19T05:03:08.713219+0000","last_active":"2024-09-19T05:04:38.343423+0000","last_peered":"2024-09-19T05:04:38.343423+0000","last_clean":"2024-09-19T05:04:38.343423+0000","last_became_active":"2024-09-19T05:03:08.712908+0000","last_became_peered":"2024-09-19T05:03:08.712908+0000","last_unstale":"2024-09-19T05:04:38.343423+0000","last_undegraded":"2024-09-19T05:04:38.343423+0000","last_fullsized":"2024-09-19T05:04:38.343423+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:23.366098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.564341+0000","last_change":"2024-09-19T05:03:08.720385+0000","last_active":"2024-09-19T05:04:38.564341+0000","last_peered":"2024-09-19T05:04:38.564341+0000","last_clean":"2024-09-19T05:04:38.564341+0000","last_became_active":"2024-09-19T05:03:08.715635+0000","last_became_peered":"2024-09-19T05:03:08.715635+0000","last_unstale":"2024-09-19T05:04:38.564341+0000","last_undegraded":"2024-09-19T05:04:38.564341+0000","last_fullsized":"2024-09-19T05:04:38.564341+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:09:57.627667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587953+0000","last_change":"2024-09-19T05:02:24.419360+0000","last_active":"2024-09-19T05:04:38.587953+0000","last_peered":"2024-09-19T05:04:38.587953+0000","last_clean":"2024-09-19T05:04:38.587953+0000","last_became_active":"2024-09-19T05:02:24.419040+0000","last_became_peered":"2024-09-19T05:02:24.419040+0000","last_unstale":"2024-09-19T05:04:38.587953+0000","last_undegraded":"2024-09-19T05:04:38.587953+0000","last_fullsized":"2024-09-19T05:04:38.587953+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:48:12.922676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'41","reported_seq":76,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.532840+0000","last_change":"2024-09-19T05:02:22.023124+0000","last_active":"2024-09-19T05:04:38.532840+0000","last_peered":"2024-09-19T05:04:38.532840+0000","last_clean":"2024-09-19T05:04:38.532840+0000","last_became_active":"2024-09-19T05:02:22.022830+0000","last_became_peered":"2024-09-19T05:02:22.022830+0000","last_unstale":"2024-09-19T05:04:38.532840+0000","last_undegraded":"2024-09-19T05:04:38.532840+0000","last_fullsized":"2024-09-19T05:04:38.532840+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:22:13.208616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'7","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587815+0000","last_change":"2024-09-19T05:02:24.426169+0000","last_active":"2024-09-19T05:04:38.587815+0000","last_peered":"2024-09-19T05:04:38.587815+0000","last_clean":"2024-09-19T05:04:38.587815+0000","last_became_active":"2024-09-19T05:02:24.425816+0000","last_became_peered":"2024-09-19T05:02:24.425816+0000","last_unstale":"2024-09-19T05:04:38.587815+0000","last_undegraded":"2024-09-19T05:04:38.587815+0000","last_fullsized":"2024-09-19T05:04:38.587815+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:56:11.735351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'8","reported_seq":160,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.684817+0000","last_change":"2024-09-19T05:02:22.154391+0000","last_active":"2024-09-19T05:04:38.684817+0000","last_peered":"2024-09-19T05:04:38.684817+0000","last_clean":"2024-09-19T05:04:38.684817+0000","last_became_active":"2024-09-19T05:02:22.154008+0000","last_became_peered":"2024-09-19T05:02:22.154008+0000","last_unstale":"2024-09-19T05:04:38.684817+0000","last_undegraded":"2024-09-19T05:04:38.684817+0000","last_fullsized":"2024-09-19T05:04:38.684817+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:29.997649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":125,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.588194+0000","last_change":"2024-09-19T05:02:24.427416+0000","last_active":"2024-09-19T05:04:38.588194+0000","last_peered":"2024-09-19T05:04:38.588194+0000","last_clean":"2024-09-19T05:04:38.588194+0000","last_became_active":"2024-09-19T05:02:24.426645+0000","last_became_peered":"2024-09-19T05:02:24.426645+0000","last_unstale":"2024-09-19T05:04:38.588194+0000","last_undegraded":"2024-09-19T05:04:38.588194+0000","last_fullsized":"2024-09-19T05:04:38.588194+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:21.841013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":351,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587790+0000","last_change":"2024-09-19T05:01:04.407740+0000","last_active":"2024-09-19T05:04:38.587790+0000","last_peered":"2024-09-19T05:04:38.587790+0000","last_clean":"2024-09-19T05:04:38.587790+0000","last_became_active":"2024-09-19T05:01:04.407281+0000","last_became_peered":"2024-09-19T05:01:04.407281+0000","last_unstale":"2024-09-19T05:04:38.587790+0000","last_undegraded":"2024-09-19T05:04:38.587790+0000","last_fullsized":"2024-09-19T05:04:38.587790+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:57:27.643438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:57:27.643438+0000","last_clean_scrub_stamp":"2024-09-19T04:57:27.643438+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:54:50.242926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.245478+0000","last_change":"2024-09-19T05:02:22.023991+0000","last_active":"2024-09-19T05:04:38.245478+0000","last_peered":"2024-09-19T05:04:38.245478+0000","last_clean":"2024-09-19T05:04:38.245478+0000","last_became_active":"2024-09-19T05:02:22.023361+0000","last_became_peered":"2024-09-19T05:02:22.023361+0000","last_unstale":"2024-09-19T05:04:38.245478+0000","last_undegraded":"2024-09-19T05:04:38.245478+0000","last_fullsized":"2024-09-19T05:04:38.245478+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:37:05.551318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.244168+0000","last_change":"2024-09-19T05:02:24.424473+0000","last_active":"2024-09-19T05:04:38.244168+0000","last_peered":"2024-09-19T05:04:38.244168+0000","last_clean":"2024-09-19T05:04:38.244168+0000","last_became_active":"2024-09-19T05:02:24.423968+0000","last_became_peered":"2024-09-19T05:02:24.423968+0000","last_unstale":"2024-09-19T05:04:38.244168+0000","last_undegraded":"2024-09-19T05:04:38.244168+0000","last_fullsized":"2024-09-19T05:04:38.244168+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:29:53.305036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587734+0000","last_change":"2024-09-19T05:02:22.026278+0000","last_active":"2024-09-19T05:04:38.587734+0000","last_peered":"2024-09-19T05:04:38.587734+0000","last_clean":"2024-09-19T05:04:38.587734+0000","last_became_active":"2024-09-19T05:02:22.025759+0000","last_became_peered":"2024-09-19T05:02:22.025759+0000","last_unstale":"2024-09-19T05:04:38.587734+0000","last_undegraded":"2024-09-19T05:04:38.587734+0000","last_fullsized":"2024-09-19T05:04:38.587734+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:21:21.696382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.563673+0000","last_change":"2024-09-19T05:02:24.426880+0000","last_active":"2024-09-19T05:04:38.563673+0000","last_peered":"2024-09-19T05:04:38.563673+0000","last_clean":"2024-09-19T05:04:38.563673+0000","last_became_active":"2024-09-19T05:02:24.426336+0000","last_became_peered":"2024-09-19T05:02:24.426336+0000","last_unstale":"2024-09-19T05:04:38.563673+0000","last_undegraded":"2024-09-19T05:04:38.563673+0000","last_fullsized":"2024-09-19T05:04:38.563673+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:51:07.110738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"86'7","reported_seq":155,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.594472+0000","last_change":"2024-09-19T05:02:22.075789+0000","last_active":"2024-09-19T05:04:38.594472+0000","last_peered":"2024-09-19T05:04:38.594472+0000","last_clean":"2024-09-19T05:04:38.594472+0000","last_became_active":"2024-09-19T05:02:22.075415+0000","last_became_peered":"2024-09-19T05:02:22.075415+0000","last_unstale":"2024-09-19T05:04:38.594472+0000","last_undegraded":"2024-09-19T05:04:38.594472+0000","last_fullsized":"2024-09-19T05:04:38.594472+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:11:43.843627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.245744+0000","last_change":"2024-09-19T05:02:24.422661+0000","last_active":"2024-09-19T05:04:38.245744+0000","last_peered":"2024-09-19T05:04:38.245744+0000","last_clean":"2024-09-19T05:04:38.245744+0000","last_became_active":"2024-09-19T05:02:24.422313+0000","last_became_peered":"2024-09-19T05:02:24.422313+0000","last_unstale":"2024-09-19T05:04:38.245744+0000","last_undegraded":"2024-09-19T05:04:38.245744+0000","last_fullsized":"2024-09-19T05:04:38.245744+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:07:39.729388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"86'8","reported_seq":159,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.684022+0000","last_change":"2024-09-19T05:02:22.028008+0000","last_active":"2024-09-19T05:04:38.684022+0000","last_peered":"2024-09-19T05:04:38.684022+0000","last_clean":"2024-09-19T05:04:38.684022+0000","last_became_active":"2024-09-19T05:02:22.027635+0000","last_became_peered":"2024-09-19T05:02:22.027635+0000","last_unstale":"2024-09-19T05:04:38.684022+0000","last_undegraded":"2024-09-19T05:04:38.684022+0000","last_fullsized":"2024-09-19T05:04:38.684022+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:41:36.902282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1515,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.356212+0000","last_change":"2024-09-19T05:02:24.435362+0000","last_active":"2024-09-19T05:04:38.356212+0000","last_peered":"2024-09-19T05:04:38.356212+0000","last_clean":"2024-09-19T05:04:38.356212+0000","last_became_active":"2024-09-19T05:02:24.435056+0000","last_became_peered":"2024-09-19T05:02:24.435056+0000","last_unstale":"2024-09-19T05:04:38.356212+0000","last_undegraded":"2024-09-19T05:04:38.356212+0000","last_fullsized":"2024-09-19T05:04:38.356212+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:58:40.092466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.340983+0000","last_change":"2024-09-19T05:02:22.077651+0000","last_active":"2024-09-19T05:04:38.340983+0000","last_peered":"2024-09-19T05:04:38.340983+0000","last_clean":"2024-09-19T05:04:38.340983+0000","last_became_active":"2024-09-19T05:02:22.076273+0000","last_became_peered":"2024-09-19T05:02:22.076273+0000","last_unstale":"2024-09-19T05:04:38.340983+0000","last_undegraded":"2024-09-19T05:04:38.340983+0000","last_fullsized":"2024-09-19T05:04:38.340983+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:28:42.064126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'5","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.356147+0000","last_change":"2024-09-19T05:02:24.438125+0000","last_active":"2024-09-19T05:04:38.356147+0000","last_peered":"2024-09-19T05:04:38.356147+0000","last_clean":"2024-09-19T05:04:38.356147+0000","last_became_active":"2024-09-19T05:02:24.437806+0000","last_became_peered":"2024-09-19T05:02:24.437806+0000","last_unstale":"2024-09-19T05:04:38.356147+0000","last_undegraded":"2024-09-19T05:04:38.356147+0000","last_fullsized":"2024-09-19T05:04:38.356147+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:46:08.338501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587986+0000","last_change":"2024-09-19T05:02:22.075920+0000","last_active":"2024-09-19T05:04:38.587986+0000","last_peered":"2024-09-19T05:04:38.587986+0000","last_clean":"2024-09-19T05:04:38.587986+0000","last_became_active":"2024-09-19T05:02:22.075609+0000","last_became_peered":"2024-09-19T05:02:22.075609+0000","last_unstale":"2024-09-19T05:04:38.587986+0000","last_undegraded":"2024-09-19T05:04:38.587986+0000","last_fullsized":"2024-09-19T05:04:38.587986+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:45.915625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.564327+0000","last_change":"2024-09-19T05:02:24.434681+0000","last_active":"2024-09-19T05:04:38.564327+0000","last_peered":"2024-09-19T05:04:38.564327+0000","last_clean":"2024-09-19T05:04:38.564327+0000","last_became_active":"2024-09-19T05:02:24.434257+0000","last_became_peered":"2024-09-19T05:02:24.434257+0000","last_unstale":"2024-09-19T05:04:38.564327+0000","last_undegraded":"2024-09-19T05:04:38.564327+0000","last_fullsized":"2024-09-19T05:04:38.564327+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:06:09.133112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'51","reported_seq":117,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.530942+0000","last_change":"2024-09-19T05:03:08.725679+0000","last_active":"2024-09-19T05:04:38.530942+0000","last_peered":"2024-09-19T05:04:38.530942+0000","last_clean":"2024-09-19T05:04:38.530942+0000","last_became_active":"2024-09-19T05:03:08.715651+0000","last_became_peered":"2024-09-19T05:03:08.715651+0000","last_unstale":"2024-09-19T05:04:38.530942+0000","last_undegraded":"2024-09-19T05:04:38.530942+0000","last_fullsized":"2024-09-19T05:04:38.530942+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:18:32.003686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":91,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.244565+0000","last_change":"2024-09-19T05:02:24.429807+0000","last_active":"2024-09-19T05:04:38.244565+0000","last_peered":"2024-09-19T05:04:38.244565+0000","last_clean":"2024-09-19T05:04:38.244565+0000","last_became_active":"2024-09-19T05:02:24.429477+0000","last_became_peered":"2024-09-19T05:02:24.429477+0000","last_unstale":"2024-09-19T05:04:38.244565+0000","last_undegraded":"2024-09-19T05:04:38.244565+0000","last_fullsized":"2024-09-19T05:04:38.244565+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:02.314851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.245566+0000","last_change":"2024-09-19T05:02:22.154534+0000","last_active":"2024-09-19T05:04:38.245566+0000","last_peered":"2024-09-19T05:04:38.245566+0000","last_clean":"2024-09-19T05:04:38.245566+0000","last_became_active":"2024-09-19T05:02:22.154238+0000","last_became_peered":"2024-09-19T05:02:22.154238+0000","last_unstale":"2024-09-19T05:04:38.245566+0000","last_undegraded":"2024-09-19T05:04:38.245566+0000","last_fullsized":"2024-09-19T05:04:38.245566+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:32:30.963503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.591417+0000","last_change":"2024-09-19T05:02:24.422817+0000","last_active":"2024-09-19T05:04:38.591417+0000","last_peered":"2024-09-19T05:04:38.591417+0000","last_clean":"2024-09-19T05:04:38.591417+0000","last_became_active":"2024-09-19T05:02:24.422478+0000","last_became_peered":"2024-09-19T05:02:24.422478+0000","last_unstale":"2024-09-19T05:04:38.591417+0000","last_undegraded":"2024-09-19T05:04:38.591417+0000","last_fullsized":"2024-09-19T05:04:38.591417+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:15:35.442149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.584586+0000","last_change":"2024-09-19T05:02:22.074396+0000","last_active":"2024-09-19T05:04:38.584586+0000","last_peered":"2024-09-19T05:04:38.584586+0000","last_clean":"2024-09-19T05:04:38.584586+0000","last_became_active":"2024-09-19T05:02:22.074108+0000","last_became_peered":"2024-09-19T05:02:22.074108+0000","last_unstale":"2024-09-19T05:04:38.584586+0000","last_undegraded":"2024-09-19T05:04:38.584586+0000","last_fullsized":"2024-09-19T05:04:38.584586+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:58:32.718702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587708+0000","last_change":"2024-09-19T05:02:24.429599+0000","last_active":"2024-09-19T05:04:38.587708+0000","last_peered":"2024-09-19T05:04:38.587708+0000","last_clean":"2024-09-19T05:04:38.587708+0000","last_became_active":"2024-09-19T05:02:24.429268+0000","last_became_peered":"2024-09-19T05:02:24.429268+0000","last_unstale":"2024-09-19T05:04:38.587708+0000","last_undegraded":"2024-09-19T05:04:38.587708+0000","last_fullsized":"2024-09-19T05:04:38.587708+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:57.123516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.244191+0000","last_change":"2024-09-19T05:02:22.021900+0000","last_active":"2024-09-19T05:04:38.244191+0000","last_peered":"2024-09-19T05:04:38.244191+0000","last_clean":"2024-09-19T05:04:38.244191+0000","last_became_active":"2024-09-19T05:02:22.020576+0000","last_became_peered":"2024-09-19T05:02:22.020576+0000","last_unstale":"2024-09-19T05:04:38.244191+0000","last_undegraded":"2024-09-19T05:04:38.244191+0000","last_fullsized":"2024-09-19T05:04:38.244191+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:44:14.466475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.343316+0000","last_change":"2024-09-19T05:02:24.420362+0000","last_active":"2024-09-19T05:04:38.343316+0000","last_peered":"2024-09-19T05:04:38.343316+0000","last_clean":"2024-09-19T05:04:38.343316+0000","last_became_active":"2024-09-19T05:02:24.420047+0000","last_became_peered":"2024-09-19T05:02:24.420047+0000","last_unstale":"2024-09-19T05:04:38.343316+0000","last_undegraded":"2024-09-19T05:04:38.343316+0000","last_fullsized":"2024-09-19T05:04:38.343316+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:28:56.977946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.455194+0000","last_change":"2024-09-19T05:02:22.024565+0000","last_active":"2024-09-19T05:04:38.455194+0000","last_peered":"2024-09-19T05:04:38.455194+0000","last_clean":"2024-09-19T05:04:38.455194+0000","last_became_active":"2024-09-19T05:02:22.024246+0000","last_became_peered":"2024-09-19T05:02:22.024246+0000","last_unstale":"2024-09-19T05:04:38.455194+0000","last_undegraded":"2024-09-19T05:04:38.455194+0000","last_fullsized":"2024-09-19T05:04:38.455194+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:58:27.491975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.244170+0000","last_change":"2024-09-19T05:02:24.428046+0000","last_active":"2024-09-19T05:04:38.244170+0000","last_peered":"2024-09-19T05:04:38.244170+0000","last_clean":"2024-09-19T05:04:38.244170+0000","last_became_active":"2024-09-19T05:02:24.427724+0000","last_became_peered":"2024-09-19T05:02:24.427724+0000","last_unstale":"2024-09-19T05:04:38.244170+0000","last_undegraded":"2024-09-19T05:04:38.244170+0000","last_fullsized":"2024-09-19T05:04:38.244170+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:11:54.252876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587778+0000","last_change":"2024-09-19T05:02:22.023054+0000","last_active":"2024-09-19T05:04:38.587778+0000","last_peered":"2024-09-19T05:04:38.587778+0000","last_clean":"2024-09-19T05:04:38.587778+0000","last_became_active":"2024-09-19T05:02:22.022700+0000","last_became_peered":"2024-09-19T05:02:22.022700+0000","last_unstale":"2024-09-19T05:04:38.587778+0000","last_undegraded":"2024-09-19T05:04:38.587778+0000","last_fullsized":"2024-09-19T05:04:38.587778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:44:19.714938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587808+0000","last_change":"2024-09-19T05:02:24.423045+0000","last_active":"2024-09-19T05:04:38.587808+0000","last_peered":"2024-09-19T05:04:38.587808+0000","last_clean":"2024-09-19T05:04:38.587808+0000","last_became_active":"2024-09-19T05:02:24.422719+0000","last_became_peered":"2024-09-19T05:02:24.422719+0000","last_unstale":"2024-09-19T05:04:38.587808+0000","last_undegraded":"2024-09-19T05:04:38.587808+0000","last_fullsized":"2024-09-19T05:04:38.587808+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:45.299603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.246181+0000","last_change":"2024-09-19T05:02:24.426942+0000","last_active":"2024-09-19T05:04:38.246181+0000","last_peered":"2024-09-19T05:04:38.246181+0000","last_clean":"2024-09-19T05:04:38.246181+0000","last_became_active":"2024-09-19T05:02:24.425896+0000","last_became_peered":"2024-09-19T05:02:24.425896+0000","last_unstale":"2024-09-19T05:04:38.246181+0000","last_undegraded":"2024-09-19T05:04:38.246181+0000","last_fullsized":"2024-09-19T05:04:38.246181+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:54:15.384309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587925+0000","last_change":"2024-09-19T05:02:22.021415+0000","last_active":"2024-09-19T05:04:38.587925+0000","last_peered":"2024-09-19T05:04:38.587925+0000","last_clean":"2024-09-19T05:04:38.587925+0000","last_became_active":"2024-09-19T05:02:22.020508+0000","last_became_peered":"2024-09-19T05:02:22.020508+0000","last_unstale":"2024-09-19T05:04:38.587925+0000","last_undegraded":"2024-09-19T05:04:38.587925+0000","last_fullsized":"2024-09-19T05:04:38.587925+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:35.912386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.584623+0000","last_change":"2024-09-19T05:02:24.435538+0000","last_active":"2024-09-19T05:04:38.584623+0000","last_peered":"2024-09-19T05:04:38.584623+0000","last_clean":"2024-09-19T05:04:38.584623+0000","last_became_active":"2024-09-19T05:02:24.435209+0000","last_became_peered":"2024-09-19T05:02:24.435209+0000","last_unstale":"2024-09-19T05:04:38.584623+0000","last_undegraded":"2024-09-19T05:04:38.584623+0000","last_fullsized":"2024-09-19T05:04:38.584623+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:55:30.075113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":61,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587897+0000","last_change":"2024-09-19T05:03:08.720605+0000","last_active":"2024-09-19T05:04:38.587897+0000","last_peered":"2024-09-19T05:04:38.587897+0000","last_clean":"2024-09-19T05:04:38.587897+0000","last_became_active":"2024-09-19T05:03:08.709207+0000","last_became_peered":"2024-09-19T05:03:08.709207+0000","last_unstale":"2024-09-19T05:04:38.587897+0000","last_undegraded":"2024-09-19T05:04:38.587897+0000","last_fullsized":"2024-09-19T05:04:38.587897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:52:26.691476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.591761+0000","last_change":"2024-09-19T05:02:24.428958+0000","last_active":"2024-09-19T05:04:38.591761+0000","last_peered":"2024-09-19T05:04:38.591761+0000","last_clean":"2024-09-19T05:04:38.591761+0000","last_became_active":"2024-09-19T05:02:24.428672+0000","last_became_peered":"2024-09-19T05:02:24.428672+0000","last_unstale":"2024-09-19T05:04:38.591761+0000","last_undegraded":"2024-09-19T05:04:38.591761+0000","last_fullsized":"2024-09-19T05:04:38.591761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:06:23.529706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.244328+0000","last_change":"2024-09-19T05:02:22.077034+0000","last_active":"2024-09-19T05:04:38.244328+0000","last_peered":"2024-09-19T05:04:38.244328+0000","last_clean":"2024-09-19T05:04:38.244328+0000","last_became_active":"2024-09-19T05:02:22.076182+0000","last_became_peered":"2024-09-19T05:02:22.076182+0000","last_unstale":"2024-09-19T05:04:38.244328+0000","last_undegraded":"2024-09-19T05:04:38.244328+0000","last_fullsized":"2024-09-19T05:04:38.244328+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:43:31.076464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.244089+0000","last_change":"2024-09-19T05:02:24.423765+0000","last_active":"2024-09-19T05:04:38.244089+0000","last_peered":"2024-09-19T05:04:38.244089+0000","last_clean":"2024-09-19T05:04:38.244089+0000","last_became_active":"2024-09-19T05:02:24.423445+0000","last_became_peered":"2024-09-19T05:02:24.423445+0000","last_unstale":"2024-09-19T05:04:38.244089+0000","last_undegraded":"2024-09-19T05:04:38.244089+0000","last_fullsized":"2024-09-19T05:04:38.244089+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:10:07.806697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"86'15","reported_seq":163,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587759+0000","last_change":"2024-09-19T05:02:22.023413+0000","last_active":"2024-09-19T05:04:38.587759+0000","last_peered":"2024-09-19T05:04:38.587759+0000","last_clean":"2024-09-19T05:04:38.587759+0000","last_became_active":"2024-09-19T05:02:22.023078+0000","last_became_peered":"2024-09-19T05:02:22.023078+0000","last_unstale":"2024-09-19T05:04:38.587759+0000","last_undegraded":"2024-09-19T05:04:38.587759+0000","last_fullsized":"2024-09-19T05:04:38.587759+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:11:09.185011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":603100,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1339,"num_read_kb":792,"num_write":381,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":981,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724992,"data_stored":1809300,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":308,"ondisk_log_size":308,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108990,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514537,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":857351,"data_compressed":27833,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743470,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32292,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712860,"statfs":{"total":95995035648,"available":95961968640,"internally_reserved":0,"allocated":1130496,"data_stored":1253796,"data_compressed":51173,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906995,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":995328,"data_stored":854839,"data_compressed":27833,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135928,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":453045,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397566,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1015808,"data_stored":1055001,"data_compressed":45017,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561091,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":954368,"data_stored":462267,"data_compressed":8687,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790024,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1052672,"data_stored":1185940,"data_compressed":51173,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986253,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":954368,"data_stored":461735,"data_compressed":8687,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215187,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1015808,"data_stored":985981,"data_compressed":33989,"data_compressed_allocated":45056,"data_compressed_original":540672,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1044480,"data_stored":532481,"data_compressed":8687,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574943,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1011712,"data_stored":521592,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":962560,"data_stored":454257,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36330,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36330,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36330,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":134954,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":202810,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":396850,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T05:04:43.733 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T05:04:43.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.937+0000 7f0096600640 1 -- 172.21.15.43:0/2306789973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0090103d30 msgr2=0x7f0090104130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:43.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.937+0000 7f0096600640 1 --2- 172.21.15.43:0/2306789973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0090103d30 0x7f0090104130 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0080009860 tx=0x7f008002a340 comp rx=0 tx=0).stop 2024-09-19T05:04:43.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.937+0000 7f0096600640 1 -- 172.21.15.43:0/2306789973 shutdown_connections 2024-09-19T05:04:43.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.937+0000 7f0096600640 1 --2- 172.21.15.43:0/2306789973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00901058e0 0x7f009010c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:43.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.937+0000 7f0096600640 1 --2- 172.21.15.43:0/2306789973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0090104f20 0x7f00901053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:43.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.937+0000 7f0096600640 1 --2- 172.21.15.43:0/2306789973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0090103d30 0x7f0090104130 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:43.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.937+0000 7f0096600640 1 -- 172.21.15.43:0/2306789973 >> 172.21.15.43:0/2306789973 conn(0x7f00900ff4b0 msgr2=0x7f00901018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:43.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.937+0000 7f0096600640 1 -- 172.21.15.43:0/2306789973 shutdown_connections 2024-09-19T05:04:43.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.937+0000 7f0096600640 1 -- 172.21.15.43:0/2306789973 wait complete. 2024-09-19T05:04:43.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.938+0000 7f0096600640 1 Processor -- start 2024-09-19T05:04:43.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.938+0000 7f0096600640 1 -- start start 2024-09-19T05:04:43.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.939+0000 7f0096600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0090103d30 0x7f009019a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:43.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.939+0000 7f0096600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0090104f20 0x7f009019a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:43.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.939+0000 7f0096600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00901058e0 0x7f00901a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:43.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.939+0000 7f0096600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f009010f990 con 0x7f0090103d30 2024-09-19T05:04:43.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.939+0000 7f0096600640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f009010f810 con 0x7f00901058e0 2024-09-19T05:04:43.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.939+0000 7f0096600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f009010fb10 con 0x7f0090104f20 2024-09-19T05:04:43.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.939+0000 7f008ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0090104f20 0x7f009019a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:43.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.939+0000 7f008f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0090103d30 0x7f009019a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:43.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.939+0000 7f008ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0090104f20 0x7f009019a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:39428/0 (socket says 172.21.15.43:39428) 2024-09-19T05:04:43.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.939+0000 7f008ea00640 1 -- 172.21.15.43:0/539673520 learned_addr learned my addr 172.21.15.43:0/539673520 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:43.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.939+0000 7f008fe00640 1 --2- 172.21.15.43:0/539673520 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00901058e0 0x7f00901a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:43.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.939+0000 7f008ea00640 1 -- 172.21.15.43:0/539673520 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00901058e0 msgr2=0x7f00901a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:43.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.939+0000 7f008ea00640 1 --2- 172.21.15.43:0/539673520 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00901058e0 0x7f00901a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:43.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.939+0000 7f008ea00640 1 -- 172.21.15.43:0/539673520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0090103d30 msgr2=0x7f009019a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:43.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.939+0000 7f008ea00640 1 --2- 172.21.15.43:0/539673520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0090103d30 0x7f009019a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:43.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.940+0000 7f008ea00640 1 -- 172.21.15.43:0/539673520 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00901a1c30 con 0x7f0090104f20 2024-09-19T05:04:43.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.940+0000 7f008ea00640 1 --2- 172.21.15.43:0/539673520 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0090104f20 0x7f009019a550 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f007c00e9f0 tx=0x7f007c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:43.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.940+0000 7f0087e00640 1 -- 172.21.15.43:0/539673520 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f007c00cd80 con 0x7f0090104f20 2024-09-19T05:04:43.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.940+0000 7f0087e00640 1 -- 172.21.15.43:0/539673520 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f007c010910 con 0x7f0090104f20 2024-09-19T05:04:43.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.940+0000 7f0096600640 1 -- 172.21.15.43:0/539673520 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00901a1f20 con 0x7f0090104f20 2024-09-19T05:04:43.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.940+0000 7f0087e00640 1 -- 172.21.15.43:0/539673520 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f007c018990 con 0x7f0090104f20 2024-09-19T05:04:43.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.940+0000 7f0096600640 1 -- 172.21.15.43:0/539673520 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00901a23e0 con 0x7f0090104f20 2024-09-19T05:04:43.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.941+0000 7f0096600640 1 -- 172.21.15.43:0/539673520 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f009010b1c0 con 0x7f0090104f20 2024-09-19T05:04:43.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.944+0000 7f0087e00640 1 -- 172.21.15.43:0/539673520 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f007c018b30 con 0x7f0090104f20 2024-09-19T05:04:43.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.945+0000 7f0087e00640 1 --2- 172.21.15.43:0/539673520 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f00740a34e0 0x7f00740a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:43.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.945+0000 7f0087e00640 1 -- 172.21.15.43:0/539673520 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f007c0d0340 con 0x7f0090104f20 2024-09-19T05:04:43.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.945+0000 7f008f400640 1 --2- 172.21.15.43:0/539673520 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f00740a34e0 0x7f00740a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:43.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.946+0000 7f008f400640 1 --2- 172.21.15.43:0/539673520 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f00740a34e0 0x7f00740a59a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0080030410 tx=0x7f0080030550 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:43.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:43.949+0000 7f0087e00640 1 -- 172.21.15.43:0/539673520 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f007c09c430 con 0x7f0090104f20 2024-09-19T05:04:44.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:43 smithi043 ceph-mon[30800]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:04:44.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:43 smithi043 ceph-mon[30800]: Adjusting osd_memory_target on smithi043 to 177.5M 2024-09-19T05:04:44.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:43 smithi043 ceph-mon[30800]: Unable to set osd_memory_target on smithi043 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-19T05:04:44.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:43 smithi043 ceph-mon[30800]: Updating smithi043:/etc/ceph/ceph.conf 2024-09-19T05:04:44.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:43 smithi043 ceph-mon[30800]: Updating smithi096:/etc/ceph/ceph.conf 2024-09-19T05:04:44.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:43 smithi043 ceph-mon[30800]: Updating smithi151:/etc/ceph/ceph.conf 2024-09-19T05:04:44.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:43 smithi043 ceph-mon[30800]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-09-19T05:04:44.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:43 smithi043 ceph-mon[30800]: Updating smithi096:/var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:04:44.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:43 smithi043 ceph-mon[30800]: Updating smithi151:/var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:04:44.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:43 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:44.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:43 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:44.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:43 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:44.032 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:43 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:44.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.150+0000 7f0096600640 1 -- 172.21.15.43:0/539673520 --> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0090062630 con 0x7f00740a34e0 2024-09-19T05:04:44.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:43 smithi151 ceph-mon[35425]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:04:44.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:43 smithi151 ceph-mon[35425]: Adjusting osd_memory_target on smithi043 to 177.5M 2024-09-19T05:04:44.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:43 smithi151 ceph-mon[35425]: Unable to set osd_memory_target on smithi043 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-19T05:04:44.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:43 smithi151 ceph-mon[35425]: Updating smithi043:/etc/ceph/ceph.conf 2024-09-19T05:04:44.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:43 smithi151 ceph-mon[35425]: Updating smithi096:/etc/ceph/ceph.conf 2024-09-19T05:04:44.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:43 smithi151 ceph-mon[35425]: Updating smithi151:/etc/ceph/ceph.conf 2024-09-19T05:04:44.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:43 smithi151 ceph-mon[35425]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-09-19T05:04:44.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:43 smithi151 ceph-mon[35425]: Updating smithi096:/var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:04:44.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:43 smithi151 ceph-mon[35425]: Updating smithi151:/var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:04:44.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:43 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:44.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:43 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:44.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:43 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:44.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:43 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:44.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.157+0000 7f0087e00640 1 -- 172.21.15.43:0/539673520 <== mgr.25136 v2:172.21.15.43:6800/2120444093 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258096 (secure 0 0 0) 0x7f0090062630 con 0x7f00740a34e0 2024-09-19T05:04:44.164 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:04:44.165 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-19T05:04:44.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.161+0000 7f0096600640 1 -- 172.21.15.43:0/539673520 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f00740a34e0 msgr2=0x7f00740a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:44.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.161+0000 7f0096600640 1 --2- 172.21.15.43:0/539673520 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f00740a34e0 0x7f00740a59a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0080030410 tx=0x7f0080030550 comp rx=0 tx=0).stop 2024-09-19T05:04:44.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.161+0000 7f0096600640 1 -- 172.21.15.43:0/539673520 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0090104f20 msgr2=0x7f009019a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:44.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.161+0000 7f0096600640 1 --2- 172.21.15.43:0/539673520 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0090104f20 0x7f009019a550 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f007c00e9f0 tx=0x7f007c00eeb0 comp rx=0 tx=0).stop 2024-09-19T05:04:44.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.161+0000 7f0096600640 1 -- 172.21.15.43:0/539673520 shutdown_connections 2024-09-19T05:04:44.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.161+0000 7f0096600640 1 --2- 172.21.15.43:0/539673520 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f00740a34e0 0x7f00740a59a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:44.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.161+0000 7f0096600640 1 --2- 172.21.15.43:0/539673520 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00901058e0 0x7f00901a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:44.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.161+0000 7f0096600640 1 --2- 172.21.15.43:0/539673520 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0090104f20 0x7f009019a550 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:44.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.161+0000 7f0096600640 1 --2- 172.21.15.43:0/539673520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0090103d30 0x7f009019a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:44.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.161+0000 7f0096600640 1 -- 172.21.15.43:0/539673520 >> 172.21.15.43:0/539673520 conn(0x7f00900ff4b0 msgr2=0x7f0090108390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:44.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.163+0000 7f0096600640 1 -- 172.21.15.43:0/539673520 shutdown_connections 2024-09-19T05:04:44.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.163+0000 7f0096600640 1 -- 172.21.15.43:0/539673520 wait complete. 2024-09-19T05:04:44.181 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-19T05:04:42.261564+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1193702,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1475,"num_read_kb":911,"num_write":564,"num_write_kb":4412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":1181717,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":947,"ondisk_log_size":947,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":377696,"kb_used_data":11784,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124564128,"statfs":{"total":1151940427776,"available":1151553667072,"internally_reserved":0,"allocated":12066816,"data_stored":9078285,"data_compressed":280452,"data_compressed_allocated":368640,"data_compressed_original":3588096,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.982555"},"pg_stats":[{"pgid":"3.3f","version":"85'8","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.343604+0000","last_change":"2024-09-19T05:02:24.426572+0000","last_active":"2024-09-19T05:04:38.343604+0000","last_peered":"2024-09-19T05:04:38.343604+0000","last_clean":"2024-09-19T05:04:38.343604+0000","last_became_active":"2024-09-19T05:02:24.426241+0000","last_became_peered":"2024-09-19T05:02:24.426241+0000","last_unstale":"2024-09-19T05:04:38.343604+0000","last_undegraded":"2024-09-19T05:04:38.343604+0000","last_fullsized":"2024-09-19T05:04:38.343604+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:42.511495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.591304+0000","last_change":"2024-09-19T05:02:24.427263+0000","last_active":"2024-09-19T05:04:38.591304+0000","last_peered":"2024-09-19T05:04:38.591304+0000","last_clean":"2024-09-19T05:04:38.591304+0000","last_became_active":"2024-09-19T05:02:24.426928+0000","last_became_peered":"2024-09-19T05:02:24.426928+0000","last_unstale":"2024-09-19T05:04:38.591304+0000","last_undegraded":"2024-09-19T05:04:38.591304+0000","last_fullsized":"2024-09-19T05:04:38.591304+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:03.177020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.591094+0000","last_change":"2024-09-19T05:02:24.432709+0000","last_active":"2024-09-19T05:04:38.591094+0000","last_peered":"2024-09-19T05:04:38.591094+0000","last_clean":"2024-09-19T05:04:38.591094+0000","last_became_active":"2024-09-19T05:02:24.431114+0000","last_became_peered":"2024-09-19T05:02:24.431114+0000","last_unstale":"2024-09-19T05:04:38.591094+0000","last_undegraded":"2024-09-19T05:04:38.591094+0000","last_fullsized":"2024-09-19T05:04:38.591094+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:57.244919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.340848+0000","last_change":"2024-09-19T05:03:08.713389+0000","last_active":"2024-09-19T05:04:38.340848+0000","last_peered":"2024-09-19T05:04:38.340848+0000","last_clean":"2024-09-19T05:04:38.340848+0000","last_became_active":"2024-09-19T05:03:08.713058+0000","last_became_peered":"2024-09-19T05:03:08.713058+0000","last_unstale":"2024-09-19T05:04:38.340848+0000","last_undegraded":"2024-09-19T05:04:38.340848+0000","last_fullsized":"2024-09-19T05:04:38.340848+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:52:10.993733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587640+0000","last_change":"2024-09-19T05:02:24.419992+0000","last_active":"2024-09-19T05:04:38.587640+0000","last_peered":"2024-09-19T05:04:38.587640+0000","last_clean":"2024-09-19T05:04:38.587640+0000","last_became_active":"2024-09-19T05:02:24.419643+0000","last_became_peered":"2024-09-19T05:02:24.419643+0000","last_unstale":"2024-09-19T05:04:38.587640+0000","last_undegraded":"2024-09-19T05:04:38.587640+0000","last_fullsized":"2024-09-19T05:04:38.587640+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:54:40.548216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.356087+0000","last_change":"2024-09-19T05:02:24.434106+0000","last_active":"2024-09-19T05:04:38.356087+0000","last_peered":"2024-09-19T05:04:38.356087+0000","last_clean":"2024-09-19T05:04:38.356087+0000","last_became_active":"2024-09-19T05:02:24.433124+0000","last_became_peered":"2024-09-19T05:02:24.433124+0000","last_unstale":"2024-09-19T05:04:38.356087+0000","last_undegraded":"2024-09-19T05:04:38.356087+0000","last_fullsized":"2024-09-19T05:04:38.356087+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:09:46.502506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587605+0000","last_change":"2024-09-19T05:02:24.429640+0000","last_active":"2024-09-19T05:04:38.587605+0000","last_peered":"2024-09-19T05:04:38.587605+0000","last_clean":"2024-09-19T05:04:38.587605+0000","last_became_active":"2024-09-19T05:02:24.429326+0000","last_became_peered":"2024-09-19T05:02:24.429326+0000","last_unstale":"2024-09-19T05:04:38.587605+0000","last_undegraded":"2024-09-19T05:04:38.587605+0000","last_fullsized":"2024-09-19T05:04:38.587605+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:27.330449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.243840+0000","last_change":"2024-09-19T05:02:24.430933+0000","last_active":"2024-09-19T05:04:38.243840+0000","last_peered":"2024-09-19T05:04:38.243840+0000","last_clean":"2024-09-19T05:04:38.243840+0000","last_became_active":"2024-09-19T05:02:24.430478+0000","last_became_peered":"2024-09-19T05:02:24.430478+0000","last_unstale":"2024-09-19T05:04:38.243840+0000","last_undegraded":"2024-09-19T05:04:38.243840+0000","last_fullsized":"2024-09-19T05:04:38.243840+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:45:22.171049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.585189+0000","last_change":"2024-09-19T05:02:24.433207+0000","last_active":"2024-09-19T05:04:38.585189+0000","last_peered":"2024-09-19T05:04:38.585189+0000","last_clean":"2024-09-19T05:04:38.585189+0000","last_became_active":"2024-09-19T05:02:24.432882+0000","last_became_peered":"2024-09-19T05:02:24.432882+0000","last_unstale":"2024-09-19T05:04:38.585189+0000","last_undegraded":"2024-09-19T05:04:38.585189+0000","last_fullsized":"2024-09-19T05:04:38.585189+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:06:42.748227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.356558+0000","last_change":"2024-09-19T05:02:24.433825+0000","last_active":"2024-09-19T05:04:38.356558+0000","last_peered":"2024-09-19T05:04:38.356558+0000","last_clean":"2024-09-19T05:04:38.356558+0000","last_became_active":"2024-09-19T05:02:24.433088+0000","last_became_peered":"2024-09-19T05:02:24.433088+0000","last_unstale":"2024-09-19T05:04:38.356558+0000","last_undegraded":"2024-09-19T05:04:38.356558+0000","last_fullsized":"2024-09-19T05:04:38.356558+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:12:53.039028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587972+0000","last_change":"2024-09-19T05:02:24.427661+0000","last_active":"2024-09-19T05:04:38.587972+0000","last_peered":"2024-09-19T05:04:38.587972+0000","last_clean":"2024-09-19T05:04:38.587972+0000","last_became_active":"2024-09-19T05:02:24.427093+0000","last_became_peered":"2024-09-19T05:02:24.427093+0000","last_unstale":"2024-09-19T05:04:38.587972+0000","last_undegraded":"2024-09-19T05:04:38.587972+0000","last_fullsized":"2024-09-19T05:04:38.587972+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:43.091041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.245390+0000","last_change":"2024-09-19T05:02:24.430820+0000","last_active":"2024-09-19T05:04:38.245390+0000","last_peered":"2024-09-19T05:04:38.245390+0000","last_clean":"2024-09-19T05:04:38.245390+0000","last_became_active":"2024-09-19T05:02:24.429945+0000","last_became_peered":"2024-09-19T05:02:24.429945+0000","last_unstale":"2024-09-19T05:04:38.245390+0000","last_undegraded":"2024-09-19T05:04:38.245390+0000","last_fullsized":"2024-09-19T05:04:38.245390+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:21:59.400725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.564024+0000","last_change":"2024-09-19T05:02:24.428695+0000","last_active":"2024-09-19T05:04:38.564024+0000","last_peered":"2024-09-19T05:04:38.564024+0000","last_clean":"2024-09-19T05:04:38.564024+0000","last_became_active":"2024-09-19T05:02:24.426385+0000","last_became_peered":"2024-09-19T05:02:24.426385+0000","last_unstale":"2024-09-19T05:04:38.564024+0000","last_undegraded":"2024-09-19T05:04:38.564024+0000","last_fullsized":"2024-09-19T05:04:38.564024+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:44:00.893313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.590202+0000","last_change":"2024-09-19T05:02:24.424935+0000","last_active":"2024-09-19T05:04:38.590202+0000","last_peered":"2024-09-19T05:04:38.590202+0000","last_clean":"2024-09-19T05:04:38.590202+0000","last_became_active":"2024-09-19T05:02:24.424219+0000","last_became_peered":"2024-09-19T05:02:24.424219+0000","last_unstale":"2024-09-19T05:04:38.590202+0000","last_undegraded":"2024-09-19T05:04:38.590202+0000","last_fullsized":"2024-09-19T05:04:38.590202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:00.890778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.589476+0000","last_change":"2024-09-19T05:02:24.425407+0000","last_active":"2024-09-19T05:04:38.589476+0000","last_peered":"2024-09-19T05:04:38.589476+0000","last_clean":"2024-09-19T05:04:38.589476+0000","last_became_active":"2024-09-19T05:02:24.425053+0000","last_became_peered":"2024-09-19T05:02:24.425053+0000","last_unstale":"2024-09-19T05:04:38.589476+0000","last_undegraded":"2024-09-19T05:04:38.589476+0000","last_fullsized":"2024-09-19T05:04:38.589476+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:54:10.024183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.584494+0000","last_change":"2024-09-19T05:02:24.427888+0000","last_active":"2024-09-19T05:04:38.584494+0000","last_peered":"2024-09-19T05:04:38.584494+0000","last_clean":"2024-09-19T05:04:38.584494+0000","last_became_active":"2024-09-19T05:02:24.427589+0000","last_became_peered":"2024-09-19T05:02:24.427589+0000","last_unstale":"2024-09-19T05:04:38.584494+0000","last_undegraded":"2024-09-19T05:04:38.584494+0000","last_fullsized":"2024-09-19T05:04:38.584494+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:02:02.654416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.343559+0000","last_change":"2024-09-19T05:02:24.426654+0000","last_active":"2024-09-19T05:04:38.343559+0000","last_peered":"2024-09-19T05:04:38.343559+0000","last_clean":"2024-09-19T05:04:38.343559+0000","last_became_active":"2024-09-19T05:02:24.426352+0000","last_became_peered":"2024-09-19T05:02:24.426352+0000","last_unstale":"2024-09-19T05:04:38.343559+0000","last_undegraded":"2024-09-19T05:04:38.343559+0000","last_fullsized":"2024-09-19T05:04:38.343559+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:20.055222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.340926+0000","last_change":"2024-09-19T05:02:24.432178+0000","last_active":"2024-09-19T05:04:38.340926+0000","last_peered":"2024-09-19T05:04:38.340926+0000","last_clean":"2024-09-19T05:04:38.340926+0000","last_became_active":"2024-09-19T05:02:24.429058+0000","last_became_peered":"2024-09-19T05:02:24.429058+0000","last_unstale":"2024-09-19T05:04:38.340926+0000","last_undegraded":"2024-09-19T05:04:38.340926+0000","last_fullsized":"2024-09-19T05:04:38.340926+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:08.001134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.455282+0000","last_change":"2024-09-19T05:02:24.428202+0000","last_active":"2024-09-19T05:04:38.455282+0000","last_peered":"2024-09-19T05:04:38.455282+0000","last_clean":"2024-09-19T05:04:38.455282+0000","last_became_active":"2024-09-19T05:02:24.427741+0000","last_became_peered":"2024-09-19T05:02:24.427741+0000","last_unstale":"2024-09-19T05:04:38.455282+0000","last_undegraded":"2024-09-19T05:04:38.455282+0000","last_fullsized":"2024-09-19T05:04:38.455282+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:42:51.076056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.343183+0000","last_change":"2024-09-19T05:02:24.425007+0000","last_active":"2024-09-19T05:04:38.343183+0000","last_peered":"2024-09-19T05:04:38.343183+0000","last_clean":"2024-09-19T05:04:38.343183+0000","last_became_active":"2024-09-19T05:02:24.424612+0000","last_became_peered":"2024-09-19T05:02:24.424612+0000","last_unstale":"2024-09-19T05:04:38.343183+0000","last_undegraded":"2024-09-19T05:04:38.343183+0000","last_fullsized":"2024-09-19T05:04:38.343183+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:36:06.115757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.564225+0000","last_change":"2024-09-19T05:02:24.428724+0000","last_active":"2024-09-19T05:04:38.564225+0000","last_peered":"2024-09-19T05:04:38.564225+0000","last_clean":"2024-09-19T05:04:38.564225+0000","last_became_active":"2024-09-19T05:02:24.427730+0000","last_became_peered":"2024-09-19T05:02:24.427730+0000","last_unstale":"2024-09-19T05:04:38.564225+0000","last_undegraded":"2024-09-19T05:04:38.564225+0000","last_fullsized":"2024-09-19T05:04:38.564225+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:15:25.466553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.584736+0000","last_change":"2024-09-19T05:02:24.426238+0000","last_active":"2024-09-19T05:04:38.584736+0000","last_peered":"2024-09-19T05:04:38.584736+0000","last_clean":"2024-09-19T05:04:38.584736+0000","last_became_active":"2024-09-19T05:02:24.425915+0000","last_became_peered":"2024-09-19T05:02:24.425915+0000","last_unstale":"2024-09-19T05:04:38.584736+0000","last_undegraded":"2024-09-19T05:04:38.584736+0000","last_fullsized":"2024-09-19T05:04:38.584736+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:37:18.444042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.243918+0000","last_change":"2024-09-19T05:02:24.425726+0000","last_active":"2024-09-19T05:04:38.243918+0000","last_peered":"2024-09-19T05:04:38.243918+0000","last_clean":"2024-09-19T05:04:38.243918+0000","last_became_active":"2024-09-19T05:02:24.425409+0000","last_became_peered":"2024-09-19T05:02:24.425409+0000","last_unstale":"2024-09-19T05:04:38.243918+0000","last_undegraded":"2024-09-19T05:04:38.243918+0000","last_fullsized":"2024-09-19T05:04:38.243918+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:12.068304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.245729+0000","last_change":"2024-09-19T05:02:24.425267+0000","last_active":"2024-09-19T05:04:38.245729+0000","last_peered":"2024-09-19T05:04:38.245729+0000","last_clean":"2024-09-19T05:04:38.245729+0000","last_became_active":"2024-09-19T05:02:24.424933+0000","last_became_peered":"2024-09-19T05:02:24.424933+0000","last_unstale":"2024-09-19T05:04:38.245729+0000","last_undegraded":"2024-09-19T05:04:38.245729+0000","last_fullsized":"2024-09-19T05:04:38.245729+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:51:55.274870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.244581+0000","last_change":"2024-09-19T05:02:24.427909+0000","last_active":"2024-09-19T05:04:38.244581+0000","last_peered":"2024-09-19T05:04:38.244581+0000","last_clean":"2024-09-19T05:04:38.244581+0000","last_became_active":"2024-09-19T05:02:24.425515+0000","last_became_peered":"2024-09-19T05:02:24.425515+0000","last_unstale":"2024-09-19T05:04:38.244581+0000","last_undegraded":"2024-09-19T05:04:38.244581+0000","last_fullsized":"2024-09-19T05:04:38.244581+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:12:22.390754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.455426+0000","last_change":"2024-09-19T05:02:24.426201+0000","last_active":"2024-09-19T05:04:38.455426+0000","last_peered":"2024-09-19T05:04:38.455426+0000","last_clean":"2024-09-19T05:04:38.455426+0000","last_became_active":"2024-09-19T05:02:24.425890+0000","last_became_peered":"2024-09-19T05:02:24.425890+0000","last_unstale":"2024-09-19T05:04:38.455426+0000","last_undegraded":"2024-09-19T05:04:38.455426+0000","last_fullsized":"2024-09-19T05:04:38.455426+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:36.680002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.343493+0000","last_change":"2024-09-19T05:02:24.416037+0000","last_active":"2024-09-19T05:04:38.343493+0000","last_peered":"2024-09-19T05:04:38.343493+0000","last_clean":"2024-09-19T05:04:38.343493+0000","last_became_active":"2024-09-19T05:02:24.415535+0000","last_became_peered":"2024-09-19T05:02:24.415535+0000","last_unstale":"2024-09-19T05:04:38.343493+0000","last_undegraded":"2024-09-19T05:04:38.343493+0000","last_fullsized":"2024-09-19T05:04:38.343493+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:14.756410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.564248+0000","last_change":"2024-09-19T05:03:08.708063+0000","last_active":"2024-09-19T05:04:38.564248+0000","last_peered":"2024-09-19T05:04:38.564248+0000","last_clean":"2024-09-19T05:04:38.564248+0000","last_became_active":"2024-09-19T05:03:08.707474+0000","last_became_peered":"2024-09-19T05:03:08.707474+0000","last_unstale":"2024-09-19T05:04:38.564248+0000","last_undegraded":"2024-09-19T05:04:38.564248+0000","last_fullsized":"2024-09-19T05:04:38.564248+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:20:55.604156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.589950+0000","last_change":"2024-09-19T05:02:24.423776+0000","last_active":"2024-09-19T05:04:38.589950+0000","last_peered":"2024-09-19T05:04:38.589950+0000","last_clean":"2024-09-19T05:04:38.589950+0000","last_became_active":"2024-09-19T05:02:24.423446+0000","last_became_peered":"2024-09-19T05:02:24.423446+0000","last_unstale":"2024-09-19T05:04:38.589950+0000","last_undegraded":"2024-09-19T05:04:38.589950+0000","last_fullsized":"2024-09-19T05:04:38.589950+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:47:02.747049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.455107+0000","last_change":"2024-09-19T05:02:24.420534+0000","last_active":"2024-09-19T05:04:38.455107+0000","last_peered":"2024-09-19T05:04:38.455107+0000","last_clean":"2024-09-19T05:04:38.455107+0000","last_became_active":"2024-09-19T05:02:24.419980+0000","last_became_peered":"2024-09-19T05:02:24.419980+0000","last_unstale":"2024-09-19T05:04:38.455107+0000","last_undegraded":"2024-09-19T05:04:38.455107+0000","last_fullsized":"2024-09-19T05:04:38.455107+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:33:49.537258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.455289+0000","last_change":"2024-09-19T05:02:24.424859+0000","last_active":"2024-09-19T05:04:38.455289+0000","last_peered":"2024-09-19T05:04:38.455289+0000","last_clean":"2024-09-19T05:04:38.455289+0000","last_became_active":"2024-09-19T05:02:24.424514+0000","last_became_peered":"2024-09-19T05:02:24.424514+0000","last_unstale":"2024-09-19T05:04:38.455289+0000","last_undegraded":"2024-09-19T05:04:38.455289+0000","last_fullsized":"2024-09-19T05:04:38.455289+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:41:41.929578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.343253+0000","last_change":"2024-09-19T05:02:24.421756+0000","last_active":"2024-09-19T05:04:38.343253+0000","last_peered":"2024-09-19T05:04:38.343253+0000","last_clean":"2024-09-19T05:04:38.343253+0000","last_became_active":"2024-09-19T05:02:24.420910+0000","last_became_peered":"2024-09-19T05:02:24.420910+0000","last_unstale":"2024-09-19T05:04:38.343253+0000","last_undegraded":"2024-09-19T05:04:38.343253+0000","last_fullsized":"2024-09-19T05:04:38.343253+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:33.759745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587628+0000","last_change":"2024-09-19T05:02:24.428514+0000","last_active":"2024-09-19T05:04:38.587628+0000","last_peered":"2024-09-19T05:04:38.587628+0000","last_clean":"2024-09-19T05:04:38.587628+0000","last_became_active":"2024-09-19T05:02:24.428206+0000","last_became_peered":"2024-09-19T05:02:24.428206+0000","last_unstale":"2024-09-19T05:04:38.587628+0000","last_undegraded":"2024-09-19T05:04:38.587628+0000","last_fullsized":"2024-09-19T05:04:38.587628+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:33.045739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"85'7","reported_seq":157,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.246167+0000","last_change":"2024-09-19T05:02:22.026683+0000","last_active":"2024-09-19T05:04:38.246167+0000","last_peered":"2024-09-19T05:04:38.246167+0000","last_clean":"2024-09-19T05:04:38.246167+0000","last_became_active":"2024-09-19T05:02:22.026342+0000","last_became_peered":"2024-09-19T05:02:22.026342+0000","last_unstale":"2024-09-19T05:04:38.246167+0000","last_undegraded":"2024-09-19T05:04:38.246167+0000","last_fullsized":"2024-09-19T05:04:38.246167+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:01:37.462838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.245923+0000","last_change":"2024-09-19T05:02:24.421753+0000","last_active":"2024-09-19T05:04:38.245923+0000","last_peered":"2024-09-19T05:04:38.245923+0000","last_clean":"2024-09-19T05:04:38.245923+0000","last_became_active":"2024-09-19T05:02:24.421455+0000","last_became_peered":"2024-09-19T05:02:24.421455+0000","last_unstale":"2024-09-19T05:04:38.245923+0000","last_undegraded":"2024-09-19T05:04:38.245923+0000","last_fullsized":"2024-09-19T05:04:38.245923+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:06.416550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.356217+0000","last_change":"2024-09-19T05:03:08.716866+0000","last_active":"2024-09-19T05:04:38.356217+0000","last_peered":"2024-09-19T05:04:38.356217+0000","last_clean":"2024-09-19T05:04:38.356217+0000","last_became_active":"2024-09-19T05:03:08.709783+0000","last_became_peered":"2024-09-19T05:03:08.709783+0000","last_unstale":"2024-09-19T05:04:38.356217+0000","last_undegraded":"2024-09-19T05:04:38.356217+0000","last_fullsized":"2024-09-19T05:04:38.356217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:26.103651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.563762+0000","last_change":"2024-09-19T05:02:22.026599+0000","last_active":"2024-09-19T05:04:38.563762+0000","last_peered":"2024-09-19T05:04:38.563762+0000","last_clean":"2024-09-19T05:04:38.563762+0000","last_became_active":"2024-09-19T05:02:22.024652+0000","last_became_peered":"2024-09-19T05:02:22.024652+0000","last_unstale":"2024-09-19T05:04:38.563762+0000","last_undegraded":"2024-09-19T05:04:38.563762+0000","last_fullsized":"2024-09-19T05:04:38.563762+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:14:32.508333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.243898+0000","last_change":"2024-09-19T05:02:24.419262+0000","last_active":"2024-09-19T05:04:38.243898+0000","last_peered":"2024-09-19T05:04:38.243898+0000","last_clean":"2024-09-19T05:04:38.243898+0000","last_became_active":"2024-09-19T05:02:24.418589+0000","last_became_peered":"2024-09-19T05:02:24.418589+0000","last_unstale":"2024-09-19T05:04:38.243898+0000","last_undegraded":"2024-09-19T05:04:38.243898+0000","last_fullsized":"2024-09-19T05:04:38.243898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:27.256676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.245917+0000","last_change":"2024-09-19T05:02:22.024225+0000","last_active":"2024-09-19T05:04:38.245917+0000","last_peered":"2024-09-19T05:04:38.245917+0000","last_clean":"2024-09-19T05:04:38.245917+0000","last_became_active":"2024-09-19T05:02:22.023882+0000","last_became_peered":"2024-09-19T05:02:22.023882+0000","last_unstale":"2024-09-19T05:04:38.245917+0000","last_undegraded":"2024-09-19T05:04:38.245917+0000","last_fullsized":"2024-09-19T05:04:38.245917+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:29.619634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.340536+0000","last_change":"2024-09-19T05:02:24.436511+0000","last_active":"2024-09-19T05:04:38.340536+0000","last_peered":"2024-09-19T05:04:38.340536+0000","last_clean":"2024-09-19T05:04:38.340536+0000","last_became_active":"2024-09-19T05:02:24.435565+0000","last_became_peered":"2024-09-19T05:02:24.435565+0000","last_unstale":"2024-09-19T05:04:38.340536+0000","last_undegraded":"2024-09-19T05:04:38.340536+0000","last_fullsized":"2024-09-19T05:04:38.340536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:16:43.779420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'7","reported_seq":273,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.683927+0000","last_change":"2024-09-19T05:02:22.024281+0000","last_active":"2024-09-19T05:04:38.683927+0000","last_peered":"2024-09-19T05:04:38.683927+0000","last_clean":"2024-09-19T05:04:38.683927+0000","last_became_active":"2024-09-19T05:02:22.023868+0000","last_became_peered":"2024-09-19T05:02:22.023868+0000","last_unstale":"2024-09-19T05:04:38.683927+0000","last_undegraded":"2024-09-19T05:04:38.683927+0000","last_fullsized":"2024-09-19T05:04:38.683927+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:29:47.597570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":240,"num_read_kb":240,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.584958+0000","last_change":"2024-09-19T05:02:24.436383+0000","last_active":"2024-09-19T05:04:38.584958+0000","last_peered":"2024-09-19T05:04:38.584958+0000","last_clean":"2024-09-19T05:04:38.584958+0000","last_became_active":"2024-09-19T05:02:24.436062+0000","last_became_peered":"2024-09-19T05:02:24.436062+0000","last_unstale":"2024-09-19T05:04:38.584958+0000","last_undegraded":"2024-09-19T05:04:38.584958+0000","last_fullsized":"2024-09-19T05:04:38.584958+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:48:42.574431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587817+0000","last_change":"2024-09-19T05:02:22.026950+0000","last_active":"2024-09-19T05:04:38.587817+0000","last_peered":"2024-09-19T05:04:38.587817+0000","last_clean":"2024-09-19T05:04:38.587817+0000","last_became_active":"2024-09-19T05:02:22.026498+0000","last_became_peered":"2024-09-19T05:02:22.026498+0000","last_unstale":"2024-09-19T05:04:38.587817+0000","last_undegraded":"2024-09-19T05:04:38.587817+0000","last_fullsized":"2024-09-19T05:04:38.587817+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:55:31.833714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.591864+0000","last_change":"2024-09-19T05:03:08.897727+0000","last_active":"2024-09-19T05:04:38.591864+0000","last_peered":"2024-09-19T05:04:38.591864+0000","last_clean":"2024-09-19T05:04:38.591864+0000","last_became_active":"2024-09-19T05:03:08.897382+0000","last_became_peered":"2024-09-19T05:03:08.897382+0000","last_unstale":"2024-09-19T05:04:38.591864+0000","last_undegraded":"2024-09-19T05:04:38.591864+0000","last_fullsized":"2024-09-19T05:04:38.591864+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:13.319291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.246219+0000","last_change":"2024-09-19T05:02:22.075901+0000","last_active":"2024-09-19T05:04:38.246219+0000","last_peered":"2024-09-19T05:04:38.246219+0000","last_clean":"2024-09-19T05:04:38.246219+0000","last_became_active":"2024-09-19T05:02:22.075512+0000","last_became_peered":"2024-09-19T05:02:22.075512+0000","last_unstale":"2024-09-19T05:04:38.246219+0000","last_undegraded":"2024-09-19T05:04:38.246219+0000","last_fullsized":"2024-09-19T05:04:38.246219+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:09:26.976888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.584706+0000","last_change":"2024-09-19T05:02:24.436797+0000","last_active":"2024-09-19T05:04:38.584706+0000","last_peered":"2024-09-19T05:04:38.584706+0000","last_clean":"2024-09-19T05:04:38.584706+0000","last_became_active":"2024-09-19T05:02:24.436457+0000","last_became_peered":"2024-09-19T05:02:24.436457+0000","last_unstale":"2024-09-19T05:04:38.584706+0000","last_undegraded":"2024-09-19T05:04:38.584706+0000","last_fullsized":"2024-09-19T05:04:38.584706+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:32:44.702601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'39","reported_seq":317,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.705059+0000","last_change":"2024-09-19T05:02:22.027415+0000","last_active":"2024-09-19T05:04:38.705059+0000","last_peered":"2024-09-19T05:04:38.705059+0000","last_clean":"2024-09-19T05:04:38.705059+0000","last_became_active":"2024-09-19T05:02:22.027065+0000","last_became_peered":"2024-09-19T05:02:22.027065+0000","last_unstale":"2024-09-19T05:04:38.705059+0000","last_undegraded":"2024-09-19T05:04:38.705059+0000","last_fullsized":"2024-09-19T05:04:38.705059+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:02.599310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68547,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":252,"num_read_kb":192,"num_write":41,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587855+0000","last_change":"2024-09-19T05:02:24.419718+0000","last_active":"2024-09-19T05:04:38.587855+0000","last_peered":"2024-09-19T05:04:38.587855+0000","last_clean":"2024-09-19T05:04:38.587855+0000","last_became_active":"2024-09-19T05:02:24.419388+0000","last_became_peered":"2024-09-19T05:02:24.419388+0000","last_unstale":"2024-09-19T05:04:38.587855+0000","last_undegraded":"2024-09-19T05:04:38.587855+0000","last_fullsized":"2024-09-19T05:04:38.587855+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:09.453551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.455176+0000","last_change":"2024-09-19T05:02:22.026220+0000","last_active":"2024-09-19T05:04:38.455176+0000","last_peered":"2024-09-19T05:04:38.455176+0000","last_clean":"2024-09-19T05:04:38.455176+0000","last_became_active":"2024-09-19T05:02:22.025838+0000","last_became_peered":"2024-09-19T05:02:22.025838+0000","last_unstale":"2024-09-19T05:04:38.455176+0000","last_undegraded":"2024-09-19T05:04:38.455176+0000","last_fullsized":"2024-09-19T05:04:38.455176+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:02.192799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.563968+0000","last_change":"2024-09-19T05:02:24.431872+0000","last_active":"2024-09-19T05:04:38.563968+0000","last_peered":"2024-09-19T05:04:38.563968+0000","last_clean":"2024-09-19T05:04:38.563968+0000","last_became_active":"2024-09-19T05:02:24.427227+0000","last_became_peered":"2024-09-19T05:02:24.427227+0000","last_unstale":"2024-09-19T05:04:38.563968+0000","last_undegraded":"2024-09-19T05:04:38.563968+0000","last_fullsized":"2024-09-19T05:04:38.563968+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:52.655108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.245966+0000","last_change":"2024-09-19T05:02:22.025136+0000","last_active":"2024-09-19T05:04:38.245966+0000","last_peered":"2024-09-19T05:04:38.245966+0000","last_clean":"2024-09-19T05:04:38.245966+0000","last_became_active":"2024-09-19T05:02:22.024813+0000","last_became_peered":"2024-09-19T05:02:22.024813+0000","last_unstale":"2024-09-19T05:04:38.245966+0000","last_undegraded":"2024-09-19T05:04:38.245966+0000","last_fullsized":"2024-09-19T05:04:38.245966+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:47:03.956661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587730+0000","last_change":"2024-09-19T05:02:24.428497+0000","last_active":"2024-09-19T05:04:38.587730+0000","last_peered":"2024-09-19T05:04:38.587730+0000","last_clean":"2024-09-19T05:04:38.587730+0000","last_became_active":"2024-09-19T05:02:24.427774+0000","last_became_peered":"2024-09-19T05:02:24.427774+0000","last_unstale":"2024-09-19T05:04:38.587730+0000","last_undegraded":"2024-09-19T05:04:38.587730+0000","last_fullsized":"2024-09-19T05:04:38.587730+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:32:02.160354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'8","reported_seq":174,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.845746+0000","last_change":"2024-09-19T05:03:08.718249+0000","last_active":"2024-09-19T05:04:38.845746+0000","last_peered":"2024-09-19T05:04:38.845746+0000","last_clean":"2024-09-19T05:04:38.845746+0000","last_became_active":"2024-09-19T05:03:08.713169+0000","last_became_peered":"2024-09-19T05:03:08.713169+0000","last_unstale":"2024-09-19T05:04:38.845746+0000","last_undegraded":"2024-09-19T05:04:38.845746+0000","last_fullsized":"2024-09-19T05:04:38.845746+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:34.640353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1515,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":981,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,10],"acting":[4,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587563+0000","last_change":"2024-09-19T05:02:24.433098+0000","last_active":"2024-09-19T05:04:38.587563+0000","last_peered":"2024-09-19T05:04:38.587563+0000","last_clean":"2024-09-19T05:04:38.587563+0000","last_became_active":"2024-09-19T05:02:24.432800+0000","last_became_peered":"2024-09-19T05:02:24.432800+0000","last_unstale":"2024-09-19T05:04:38.587563+0000","last_undegraded":"2024-09-19T05:04:38.587563+0000","last_fullsized":"2024-09-19T05:04:38.587563+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:50:30.163842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587677+0000","last_change":"2024-09-19T05:02:22.075865+0000","last_active":"2024-09-19T05:04:38.587677+0000","last_peered":"2024-09-19T05:04:38.587677+0000","last_clean":"2024-09-19T05:04:38.587677+0000","last_became_active":"2024-09-19T05:02:22.075185+0000","last_became_peered":"2024-09-19T05:02:22.075185+0000","last_unstale":"2024-09-19T05:04:38.587677+0000","last_undegraded":"2024-09-19T05:04:38.587677+0000","last_fullsized":"2024-09-19T05:04:38.587677+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:19.150155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.245715+0000","last_change":"2024-09-19T05:02:24.422759+0000","last_active":"2024-09-19T05:04:38.245715+0000","last_peered":"2024-09-19T05:04:38.245715+0000","last_clean":"2024-09-19T05:04:38.245715+0000","last_became_active":"2024-09-19T05:02:24.422421+0000","last_became_peered":"2024-09-19T05:02:24.422421+0000","last_unstale":"2024-09-19T05:04:38.245715+0000","last_undegraded":"2024-09-19T05:04:38.245715+0000","last_fullsized":"2024-09-19T05:04:38.245715+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:58:50.083017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.588067+0000","last_change":"2024-09-19T05:02:22.024305+0000","last_active":"2024-09-19T05:04:38.588067+0000","last_peered":"2024-09-19T05:04:38.588067+0000","last_clean":"2024-09-19T05:04:38.588067+0000","last_became_active":"2024-09-19T05:02:22.023616+0000","last_became_peered":"2024-09-19T05:02:22.023616+0000","last_unstale":"2024-09-19T05:04:38.588067+0000","last_undegraded":"2024-09-19T05:04:38.588067+0000","last_fullsized":"2024-09-19T05:04:38.588067+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:05.665219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.343423+0000","last_change":"2024-09-19T05:03:08.713219+0000","last_active":"2024-09-19T05:04:38.343423+0000","last_peered":"2024-09-19T05:04:38.343423+0000","last_clean":"2024-09-19T05:04:38.343423+0000","last_became_active":"2024-09-19T05:03:08.712908+0000","last_became_peered":"2024-09-19T05:03:08.712908+0000","last_unstale":"2024-09-19T05:04:38.343423+0000","last_undegraded":"2024-09-19T05:04:38.343423+0000","last_fullsized":"2024-09-19T05:04:38.343423+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:23.366098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.564341+0000","last_change":"2024-09-19T05:03:08.720385+0000","last_active":"2024-09-19T05:04:38.564341+0000","last_peered":"2024-09-19T05:04:38.564341+0000","last_clean":"2024-09-19T05:04:38.564341+0000","last_became_active":"2024-09-19T05:03:08.715635+0000","last_became_peered":"2024-09-19T05:03:08.715635+0000","last_unstale":"2024-09-19T05:04:38.564341+0000","last_undegraded":"2024-09-19T05:04:38.564341+0000","last_fullsized":"2024-09-19T05:04:38.564341+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:09:57.627667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587953+0000","last_change":"2024-09-19T05:02:24.419360+0000","last_active":"2024-09-19T05:04:38.587953+0000","last_peered":"2024-09-19T05:04:38.587953+0000","last_clean":"2024-09-19T05:04:38.587953+0000","last_became_active":"2024-09-19T05:02:24.419040+0000","last_became_peered":"2024-09-19T05:02:24.419040+0000","last_unstale":"2024-09-19T05:04:38.587953+0000","last_undegraded":"2024-09-19T05:04:38.587953+0000","last_fullsized":"2024-09-19T05:04:38.587953+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:48:12.922676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'41","reported_seq":76,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.532840+0000","last_change":"2024-09-19T05:02:22.023124+0000","last_active":"2024-09-19T05:04:38.532840+0000","last_peered":"2024-09-19T05:04:38.532840+0000","last_clean":"2024-09-19T05:04:38.532840+0000","last_became_active":"2024-09-19T05:02:22.022830+0000","last_became_peered":"2024-09-19T05:02:22.022830+0000","last_unstale":"2024-09-19T05:04:38.532840+0000","last_undegraded":"2024-09-19T05:04:38.532840+0000","last_fullsized":"2024-09-19T05:04:38.532840+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:22:13.208616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'7","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587815+0000","last_change":"2024-09-19T05:02:24.426169+0000","last_active":"2024-09-19T05:04:38.587815+0000","last_peered":"2024-09-19T05:04:38.587815+0000","last_clean":"2024-09-19T05:04:38.587815+0000","last_became_active":"2024-09-19T05:02:24.425816+0000","last_became_peered":"2024-09-19T05:02:24.425816+0000","last_unstale":"2024-09-19T05:04:38.587815+0000","last_undegraded":"2024-09-19T05:04:38.587815+0000","last_fullsized":"2024-09-19T05:04:38.587815+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:56:11.735351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'8","reported_seq":160,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.684817+0000","last_change":"2024-09-19T05:02:22.154391+0000","last_active":"2024-09-19T05:04:38.684817+0000","last_peered":"2024-09-19T05:04:38.684817+0000","last_clean":"2024-09-19T05:04:38.684817+0000","last_became_active":"2024-09-19T05:02:22.154008+0000","last_became_peered":"2024-09-19T05:02:22.154008+0000","last_unstale":"2024-09-19T05:04:38.684817+0000","last_undegraded":"2024-09-19T05:04:38.684817+0000","last_fullsized":"2024-09-19T05:04:38.684817+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:29.997649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":125,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.588194+0000","last_change":"2024-09-19T05:02:24.427416+0000","last_active":"2024-09-19T05:04:38.588194+0000","last_peered":"2024-09-19T05:04:38.588194+0000","last_clean":"2024-09-19T05:04:38.588194+0000","last_became_active":"2024-09-19T05:02:24.426645+0000","last_became_peered":"2024-09-19T05:02:24.426645+0000","last_unstale":"2024-09-19T05:04:38.588194+0000","last_undegraded":"2024-09-19T05:04:38.588194+0000","last_fullsized":"2024-09-19T05:04:38.588194+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:21.841013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":351,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587790+0000","last_change":"2024-09-19T05:01:04.407740+0000","last_active":"2024-09-19T05:04:38.587790+0000","last_peered":"2024-09-19T05:04:38.587790+0000","last_clean":"2024-09-19T05:04:38.587790+0000","last_became_active":"2024-09-19T05:01:04.407281+0000","last_became_peered":"2024-09-19T05:01:04.407281+0000","last_unstale":"2024-09-19T05:04:38.587790+0000","last_undegraded":"2024-09-19T05:04:38.587790+0000","last_fullsized":"2024-09-19T05:04:38.587790+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:57:27.643438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:57:27.643438+0000","last_clean_scrub_stamp":"2024-09-19T04:57:27.643438+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:54:50.242926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.245478+0000","last_change":"2024-09-19T05:02:22.023991+0000","last_active":"2024-09-19T05:04:38.245478+0000","last_peered":"2024-09-19T05:04:38.245478+0000","last_clean":"2024-09-19T05:04:38.245478+0000","last_became_active":"2024-09-19T05:02:22.023361+0000","last_became_peered":"2024-09-19T05:02:22.023361+0000","last_unstale":"2024-09-19T05:04:38.245478+0000","last_undegraded":"2024-09-19T05:04:38.245478+0000","last_fullsized":"2024-09-19T05:04:38.245478+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:37:05.551318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.244168+0000","last_change":"2024-09-19T05:02:24.424473+0000","last_active":"2024-09-19T05:04:38.244168+0000","last_peered":"2024-09-19T05:04:38.244168+0000","last_clean":"2024-09-19T05:04:38.244168+0000","last_became_active":"2024-09-19T05:02:24.423968+0000","last_became_peered":"2024-09-19T05:02:24.423968+0000","last_unstale":"2024-09-19T05:04:38.244168+0000","last_undegraded":"2024-09-19T05:04:38.244168+0000","last_fullsized":"2024-09-19T05:04:38.244168+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:29:53.305036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587734+0000","last_change":"2024-09-19T05:02:22.026278+0000","last_active":"2024-09-19T05:04:38.587734+0000","last_peered":"2024-09-19T05:04:38.587734+0000","last_clean":"2024-09-19T05:04:38.587734+0000","last_became_active":"2024-09-19T05:02:22.025759+0000","last_became_peered":"2024-09-19T05:02:22.025759+0000","last_unstale":"2024-09-19T05:04:38.587734+0000","last_undegraded":"2024-09-19T05:04:38.587734+0000","last_fullsized":"2024-09-19T05:04:38.587734+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:21:21.696382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.563673+0000","last_change":"2024-09-19T05:02:24.426880+0000","last_active":"2024-09-19T05:04:38.563673+0000","last_peered":"2024-09-19T05:04:38.563673+0000","last_clean":"2024-09-19T05:04:38.563673+0000","last_became_active":"2024-09-19T05:02:24.426336+0000","last_became_peered":"2024-09-19T05:02:24.426336+0000","last_unstale":"2024-09-19T05:04:38.563673+0000","last_undegraded":"2024-09-19T05:04:38.563673+0000","last_fullsized":"2024-09-19T05:04:38.563673+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:51:07.110738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"86'7","reported_seq":155,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.594472+0000","last_change":"2024-09-19T05:02:22.075789+0000","last_active":"2024-09-19T05:04:38.594472+0000","last_peered":"2024-09-19T05:04:38.594472+0000","last_clean":"2024-09-19T05:04:38.594472+0000","last_became_active":"2024-09-19T05:02:22.075415+0000","last_became_peered":"2024-09-19T05:02:22.075415+0000","last_unstale":"2024-09-19T05:04:38.594472+0000","last_undegraded":"2024-09-19T05:04:38.594472+0000","last_fullsized":"2024-09-19T05:04:38.594472+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:11:43.843627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.245744+0000","last_change":"2024-09-19T05:02:24.422661+0000","last_active":"2024-09-19T05:04:38.245744+0000","last_peered":"2024-09-19T05:04:38.245744+0000","last_clean":"2024-09-19T05:04:38.245744+0000","last_became_active":"2024-09-19T05:02:24.422313+0000","last_became_peered":"2024-09-19T05:02:24.422313+0000","last_unstale":"2024-09-19T05:04:38.245744+0000","last_undegraded":"2024-09-19T05:04:38.245744+0000","last_fullsized":"2024-09-19T05:04:38.245744+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:07:39.729388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"86'8","reported_seq":159,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.684022+0000","last_change":"2024-09-19T05:02:22.028008+0000","last_active":"2024-09-19T05:04:38.684022+0000","last_peered":"2024-09-19T05:04:38.684022+0000","last_clean":"2024-09-19T05:04:38.684022+0000","last_became_active":"2024-09-19T05:02:22.027635+0000","last_became_peered":"2024-09-19T05:02:22.027635+0000","last_unstale":"2024-09-19T05:04:38.684022+0000","last_undegraded":"2024-09-19T05:04:38.684022+0000","last_fullsized":"2024-09-19T05:04:38.684022+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:41:36.902282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1515,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.356212+0000","last_change":"2024-09-19T05:02:24.435362+0000","last_active":"2024-09-19T05:04:38.356212+0000","last_peered":"2024-09-19T05:04:38.356212+0000","last_clean":"2024-09-19T05:04:38.356212+0000","last_became_active":"2024-09-19T05:02:24.435056+0000","last_became_peered":"2024-09-19T05:02:24.435056+0000","last_unstale":"2024-09-19T05:04:38.356212+0000","last_undegraded":"2024-09-19T05:04:38.356212+0000","last_fullsized":"2024-09-19T05:04:38.356212+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:58:40.092466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.340983+0000","last_change":"2024-09-19T05:02:22.077651+0000","last_active":"2024-09-19T05:04:38.340983+0000","last_peered":"2024-09-19T05:04:38.340983+0000","last_clean":"2024-09-19T05:04:38.340983+0000","last_became_active":"2024-09-19T05:02:22.076273+0000","last_became_peered":"2024-09-19T05:02:22.076273+0000","last_unstale":"2024-09-19T05:04:38.340983+0000","last_undegraded":"2024-09-19T05:04:38.340983+0000","last_fullsized":"2024-09-19T05:04:38.340983+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:28:42.064126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'5","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.356147+0000","last_change":"2024-09-19T05:02:24.438125+0000","last_active":"2024-09-19T05:04:38.356147+0000","last_peered":"2024-09-19T05:04:38.356147+0000","last_clean":"2024-09-19T05:04:38.356147+0000","last_became_active":"2024-09-19T05:02:24.437806+0000","last_became_peered":"2024-09-19T05:02:24.437806+0000","last_unstale":"2024-09-19T05:04:38.356147+0000","last_undegraded":"2024-09-19T05:04:38.356147+0000","last_fullsized":"2024-09-19T05:04:38.356147+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:46:08.338501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587986+0000","last_change":"2024-09-19T05:02:22.075920+0000","last_active":"2024-09-19T05:04:38.587986+0000","last_peered":"2024-09-19T05:04:38.587986+0000","last_clean":"2024-09-19T05:04:38.587986+0000","last_became_active":"2024-09-19T05:02:22.075609+0000","last_became_peered":"2024-09-19T05:02:22.075609+0000","last_unstale":"2024-09-19T05:04:38.587986+0000","last_undegraded":"2024-09-19T05:04:38.587986+0000","last_fullsized":"2024-09-19T05:04:38.587986+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:45.915625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.564327+0000","last_change":"2024-09-19T05:02:24.434681+0000","last_active":"2024-09-19T05:04:38.564327+0000","last_peered":"2024-09-19T05:04:38.564327+0000","last_clean":"2024-09-19T05:04:38.564327+0000","last_became_active":"2024-09-19T05:02:24.434257+0000","last_became_peered":"2024-09-19T05:02:24.434257+0000","last_unstale":"2024-09-19T05:04:38.564327+0000","last_undegraded":"2024-09-19T05:04:38.564327+0000","last_fullsized":"2024-09-19T05:04:38.564327+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:06:09.133112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'51","reported_seq":117,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.530942+0000","last_change":"2024-09-19T05:03:08.725679+0000","last_active":"2024-09-19T05:04:38.530942+0000","last_peered":"2024-09-19T05:04:38.530942+0000","last_clean":"2024-09-19T05:04:38.530942+0000","last_became_active":"2024-09-19T05:03:08.715651+0000","last_became_peered":"2024-09-19T05:03:08.715651+0000","last_unstale":"2024-09-19T05:04:38.530942+0000","last_undegraded":"2024-09-19T05:04:38.530942+0000","last_fullsized":"2024-09-19T05:04:38.530942+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:18:32.003686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":91,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.244565+0000","last_change":"2024-09-19T05:02:24.429807+0000","last_active":"2024-09-19T05:04:38.244565+0000","last_peered":"2024-09-19T05:04:38.244565+0000","last_clean":"2024-09-19T05:04:38.244565+0000","last_became_active":"2024-09-19T05:02:24.429477+0000","last_became_peered":"2024-09-19T05:02:24.429477+0000","last_unstale":"2024-09-19T05:04:38.244565+0000","last_undegraded":"2024-09-19T05:04:38.244565+0000","last_fullsized":"2024-09-19T05:04:38.244565+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:02.314851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.245566+0000","last_change":"2024-09-19T05:02:22.154534+0000","last_active":"2024-09-19T05:04:38.245566+0000","last_peered":"2024-09-19T05:04:38.245566+0000","last_clean":"2024-09-19T05:04:38.245566+0000","last_became_active":"2024-09-19T05:02:22.154238+0000","last_became_peered":"2024-09-19T05:02:22.154238+0000","last_unstale":"2024-09-19T05:04:38.245566+0000","last_undegraded":"2024-09-19T05:04:38.245566+0000","last_fullsized":"2024-09-19T05:04:38.245566+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:32:30.963503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.591417+0000","last_change":"2024-09-19T05:02:24.422817+0000","last_active":"2024-09-19T05:04:38.591417+0000","last_peered":"2024-09-19T05:04:38.591417+0000","last_clean":"2024-09-19T05:04:38.591417+0000","last_became_active":"2024-09-19T05:02:24.422478+0000","last_became_peered":"2024-09-19T05:02:24.422478+0000","last_unstale":"2024-09-19T05:04:38.591417+0000","last_undegraded":"2024-09-19T05:04:38.591417+0000","last_fullsized":"2024-09-19T05:04:38.591417+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:15:35.442149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.584586+0000","last_change":"2024-09-19T05:02:22.074396+0000","last_active":"2024-09-19T05:04:38.584586+0000","last_peered":"2024-09-19T05:04:38.584586+0000","last_clean":"2024-09-19T05:04:38.584586+0000","last_became_active":"2024-09-19T05:02:22.074108+0000","last_became_peered":"2024-09-19T05:02:22.074108+0000","last_unstale":"2024-09-19T05:04:38.584586+0000","last_undegraded":"2024-09-19T05:04:38.584586+0000","last_fullsized":"2024-09-19T05:04:38.584586+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:58:32.718702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587708+0000","last_change":"2024-09-19T05:02:24.429599+0000","last_active":"2024-09-19T05:04:38.587708+0000","last_peered":"2024-09-19T05:04:38.587708+0000","last_clean":"2024-09-19T05:04:38.587708+0000","last_became_active":"2024-09-19T05:02:24.429268+0000","last_became_peered":"2024-09-19T05:02:24.429268+0000","last_unstale":"2024-09-19T05:04:38.587708+0000","last_undegraded":"2024-09-19T05:04:38.587708+0000","last_fullsized":"2024-09-19T05:04:38.587708+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:57.123516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.244191+0000","last_change":"2024-09-19T05:02:22.021900+0000","last_active":"2024-09-19T05:04:38.244191+0000","last_peered":"2024-09-19T05:04:38.244191+0000","last_clean":"2024-09-19T05:04:38.244191+0000","last_became_active":"2024-09-19T05:02:22.020576+0000","last_became_peered":"2024-09-19T05:02:22.020576+0000","last_unstale":"2024-09-19T05:04:38.244191+0000","last_undegraded":"2024-09-19T05:04:38.244191+0000","last_fullsized":"2024-09-19T05:04:38.244191+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:44:14.466475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.343316+0000","last_change":"2024-09-19T05:02:24.420362+0000","last_active":"2024-09-19T05:04:38.343316+0000","last_peered":"2024-09-19T05:04:38.343316+0000","last_clean":"2024-09-19T05:04:38.343316+0000","last_became_active":"2024-09-19T05:02:24.420047+0000","last_became_peered":"2024-09-19T05:02:24.420047+0000","last_unstale":"2024-09-19T05:04:38.343316+0000","last_undegraded":"2024-09-19T05:04:38.343316+0000","last_fullsized":"2024-09-19T05:04:38.343316+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:28:56.977946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.455194+0000","last_change":"2024-09-19T05:02:22.024565+0000","last_active":"2024-09-19T05:04:38.455194+0000","last_peered":"2024-09-19T05:04:38.455194+0000","last_clean":"2024-09-19T05:04:38.455194+0000","last_became_active":"2024-09-19T05:02:22.024246+0000","last_became_peered":"2024-09-19T05:02:22.024246+0000","last_unstale":"2024-09-19T05:04:38.455194+0000","last_undegraded":"2024-09-19T05:04:38.455194+0000","last_fullsized":"2024-09-19T05:04:38.455194+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:58:27.491975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.244170+0000","last_change":"2024-09-19T05:02:24.428046+0000","last_active":"2024-09-19T05:04:38.244170+0000","last_peered":"2024-09-19T05:04:38.244170+0000","last_clean":"2024-09-19T05:04:38.244170+0000","last_became_active":"2024-09-19T05:02:24.427724+0000","last_became_peered":"2024-09-19T05:02:24.427724+0000","last_unstale":"2024-09-19T05:04:38.244170+0000","last_undegraded":"2024-09-19T05:04:38.244170+0000","last_fullsized":"2024-09-19T05:04:38.244170+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:11:54.252876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587778+0000","last_change":"2024-09-19T05:02:22.023054+0000","last_active":"2024-09-19T05:04:38.587778+0000","last_peered":"2024-09-19T05:04:38.587778+0000","last_clean":"2024-09-19T05:04:38.587778+0000","last_became_active":"2024-09-19T05:02:22.022700+0000","last_became_peered":"2024-09-19T05:02:22.022700+0000","last_unstale":"2024-09-19T05:04:38.587778+0000","last_undegraded":"2024-09-19T05:04:38.587778+0000","last_fullsized":"2024-09-19T05:04:38.587778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:44:19.714938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587808+0000","last_change":"2024-09-19T05:02:24.423045+0000","last_active":"2024-09-19T05:04:38.587808+0000","last_peered":"2024-09-19T05:04:38.587808+0000","last_clean":"2024-09-19T05:04:38.587808+0000","last_became_active":"2024-09-19T05:02:24.422719+0000","last_became_peered":"2024-09-19T05:02:24.422719+0000","last_unstale":"2024-09-19T05:04:38.587808+0000","last_undegraded":"2024-09-19T05:04:38.587808+0000","last_fullsized":"2024-09-19T05:04:38.587808+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:45.299603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.246181+0000","last_change":"2024-09-19T05:02:24.426942+0000","last_active":"2024-09-19T05:04:38.246181+0000","last_peered":"2024-09-19T05:04:38.246181+0000","last_clean":"2024-09-19T05:04:38.246181+0000","last_became_active":"2024-09-19T05:02:24.425896+0000","last_became_peered":"2024-09-19T05:02:24.425896+0000","last_unstale":"2024-09-19T05:04:38.246181+0000","last_undegraded":"2024-09-19T05:04:38.246181+0000","last_fullsized":"2024-09-19T05:04:38.246181+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:54:15.384309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587925+0000","last_change":"2024-09-19T05:02:22.021415+0000","last_active":"2024-09-19T05:04:38.587925+0000","last_peered":"2024-09-19T05:04:38.587925+0000","last_clean":"2024-09-19T05:04:38.587925+0000","last_became_active":"2024-09-19T05:02:22.020508+0000","last_became_peered":"2024-09-19T05:02:22.020508+0000","last_unstale":"2024-09-19T05:04:38.587925+0000","last_undegraded":"2024-09-19T05:04:38.587925+0000","last_fullsized":"2024-09-19T05:04:38.587925+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:35.912386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.584623+0000","last_change":"2024-09-19T05:02:24.435538+0000","last_active":"2024-09-19T05:04:38.584623+0000","last_peered":"2024-09-19T05:04:38.584623+0000","last_clean":"2024-09-19T05:04:38.584623+0000","last_became_active":"2024-09-19T05:02:24.435209+0000","last_became_peered":"2024-09-19T05:02:24.435209+0000","last_unstale":"2024-09-19T05:04:38.584623+0000","last_undegraded":"2024-09-19T05:04:38.584623+0000","last_fullsized":"2024-09-19T05:04:38.584623+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:55:30.075113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":61,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587897+0000","last_change":"2024-09-19T05:03:08.720605+0000","last_active":"2024-09-19T05:04:38.587897+0000","last_peered":"2024-09-19T05:04:38.587897+0000","last_clean":"2024-09-19T05:04:38.587897+0000","last_became_active":"2024-09-19T05:03:08.709207+0000","last_became_peered":"2024-09-19T05:03:08.709207+0000","last_unstale":"2024-09-19T05:04:38.587897+0000","last_undegraded":"2024-09-19T05:04:38.587897+0000","last_fullsized":"2024-09-19T05:04:38.587897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:52:26.691476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.591761+0000","last_change":"2024-09-19T05:02:24.428958+0000","last_active":"2024-09-19T05:04:38.591761+0000","last_peered":"2024-09-19T05:04:38.591761+0000","last_clean":"2024-09-19T05:04:38.591761+0000","last_became_active":"2024-09-19T05:02:24.428672+0000","last_became_peered":"2024-09-19T05:02:24.428672+0000","last_unstale":"2024-09-19T05:04:38.591761+0000","last_undegraded":"2024-09-19T05:04:38.591761+0000","last_fullsized":"2024-09-19T05:04:38.591761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:06:23.529706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.244328+0000","last_change":"2024-09-19T05:02:22.077034+0000","last_active":"2024-09-19T05:04:38.244328+0000","last_peered":"2024-09-19T05:04:38.244328+0000","last_clean":"2024-09-19T05:04:38.244328+0000","last_became_active":"2024-09-19T05:02:22.076182+0000","last_became_peered":"2024-09-19T05:02:22.076182+0000","last_unstale":"2024-09-19T05:04:38.244328+0000","last_undegraded":"2024-09-19T05:04:38.244328+0000","last_fullsized":"2024-09-19T05:04:38.244328+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:43:31.076464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.244089+0000","last_change":"2024-09-19T05:02:24.423765+0000","last_active":"2024-09-19T05:04:38.244089+0000","last_peered":"2024-09-19T05:04:38.244089+0000","last_clean":"2024-09-19T05:04:38.244089+0000","last_became_active":"2024-09-19T05:02:24.423445+0000","last_became_peered":"2024-09-19T05:02:24.423445+0000","last_unstale":"2024-09-19T05:04:38.244089+0000","last_undegraded":"2024-09-19T05:04:38.244089+0000","last_fullsized":"2024-09-19T05:04:38.244089+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:10:07.806697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"86'15","reported_seq":163,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587759+0000","last_change":"2024-09-19T05:02:22.023413+0000","last_active":"2024-09-19T05:04:38.587759+0000","last_peered":"2024-09-19T05:04:38.587759+0000","last_clean":"2024-09-19T05:04:38.587759+0000","last_became_active":"2024-09-19T05:02:22.023078+0000","last_became_peered":"2024-09-19T05:02:22.023078+0000","last_unstale":"2024-09-19T05:04:38.587759+0000","last_undegraded":"2024-09-19T05:04:38.587759+0000","last_fullsized":"2024-09-19T05:04:38.587759+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:11:09.185011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":603100,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1339,"num_read_kb":792,"num_write":381,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":981,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724992,"data_stored":1809300,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":308,"ondisk_log_size":308,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108990,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514537,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":857351,"data_compressed":27833,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743470,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32292,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712860,"statfs":{"total":95995035648,"available":95961968640,"internally_reserved":0,"allocated":1130496,"data_stored":1253796,"data_compressed":51173,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906995,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":995328,"data_stored":854839,"data_compressed":27833,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135928,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":453045,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397566,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1015808,"data_stored":1055001,"data_compressed":45017,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561091,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":954368,"data_stored":462267,"data_compressed":8687,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790024,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1052672,"data_stored":1185940,"data_compressed":51173,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986253,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":954368,"data_stored":461735,"data_compressed":8687,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215187,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1015808,"data_stored":985981,"data_compressed":33989,"data_compressed_allocated":45056,"data_compressed_original":540672,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1044480,"data_stored":532481,"data_compressed":8687,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574943,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1011712,"data_stored":521592,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":962560,"data_stored":454257,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36330,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36330,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36330,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":134954,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":202810,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":396850,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T05:04:44.185 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-19T05:04:44.185 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T05:04:44.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:43 smithi096 ceph-mon[35085]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:04:44.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:43 smithi096 ceph-mon[35085]: Adjusting osd_memory_target on smithi043 to 177.5M 2024-09-19T05:04:44.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:43 smithi096 ceph-mon[35085]: Unable to set osd_memory_target on smithi043 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-19T05:04:44.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:43 smithi096 ceph-mon[35085]: Updating smithi043:/etc/ceph/ceph.conf 2024-09-19T05:04:44.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:43 smithi096 ceph-mon[35085]: Updating smithi096:/etc/ceph/ceph.conf 2024-09-19T05:04:44.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:43 smithi096 ceph-mon[35085]: Updating smithi151:/etc/ceph/ceph.conf 2024-09-19T05:04:44.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:43 smithi096 ceph-mon[35085]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-09-19T05:04:44.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:43 smithi096 ceph-mon[35085]: Updating smithi096:/var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:04:44.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:43 smithi096 ceph-mon[35085]: Updating smithi151:/var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:04:44.225 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:43 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:44.225 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:43 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:44.225 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:43 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:44.225 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:43 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:44.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.393+0000 7f17d0000640 1 -- 172.21.15.43:0/1201502288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17c8103f10 msgr2=0x7f17c81042f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:44.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.393+0000 7f17d0000640 1 --2- 172.21.15.43:0/1201502288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17c8103f10 0x7f17c81042f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f17bc009900 tx=0x7f17bc02a410 comp rx=0 tx=0).stop 2024-09-19T05:04:44.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.394+0000 7f17d0000640 1 -- 172.21.15.43:0/1201502288 shutdown_connections 2024-09-19T05:04:44.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.394+0000 7f17d0000640 1 --2- 172.21.15.43:0/1201502288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17c8109060 0x7f17c8110bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:44.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.394+0000 7f17d0000640 1 --2- 172.21.15.43:0/1201502288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17c81048c0 0x7f17c8108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:44.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.394+0000 7f17d0000640 1 --2- 172.21.15.43:0/1201502288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17c8103f10 0x7f17c81042f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:44.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.394+0000 7f17d0000640 1 -- 172.21.15.43:0/1201502288 >> 172.21.15.43:0/1201502288 conn(0x7f17c80ff980 msgr2=0x7f17c8101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:44.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.394+0000 7f17d0000640 1 -- 172.21.15.43:0/1201502288 shutdown_connections 2024-09-19T05:04:44.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.394+0000 7f17d0000640 1 -- 172.21.15.43:0/1201502288 wait complete. 2024-09-19T05:04:44.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.395+0000 7f17d0000640 1 Processor -- start 2024-09-19T05:04:44.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.395+0000 7f17d0000640 1 -- start start 2024-09-19T05:04:44.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.395+0000 7f17d0000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17c8103f10 0x7f17c819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:44.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.395+0000 7f17d0000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17c8109060 0x7f17c819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:44.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.395+0000 7f17d0000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17c819eec0 0x7f17c819f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:44.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.395+0000 7f17d0000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f17c8114100 con 0x7f17c8103f10 2024-09-19T05:04:44.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.395+0000 7f17d0000640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f17c8113f80 con 0x7f17c8109060 2024-09-19T05:04:44.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.395+0000 7f17d0000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f17c8114280 con 0x7f17c819eec0 2024-09-19T05:04:44.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.395+0000 7f17cd600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17c8103f10 0x7f17c819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:44.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.395+0000 7f17cd600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17c8103f10 0x7f17c819a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60154/0 (socket says 172.21.15.43:60154) 2024-09-19T05:04:44.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.395+0000 7f17cd600640 1 -- 172.21.15.43:0/4050704744 learned_addr learned my addr 172.21.15.43:0/4050704744 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:44.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.396+0000 7f17ccc00640 1 --2- 172.21.15.43:0/4050704744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17c8109060 0x7f17c819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:44.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.396+0000 7f17ce000640 1 --2- 172.21.15.43:0/4050704744 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17c819eec0 0x7f17c819f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:44.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.396+0000 7f17cd600640 1 -- 172.21.15.43:0/4050704744 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17c819eec0 msgr2=0x7f17c819f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:44.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.396+0000 7f17cd600640 1 --2- 172.21.15.43:0/4050704744 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17c819eec0 0x7f17c819f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:44.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.396+0000 7f17cd600640 1 -- 172.21.15.43:0/4050704744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17c8109060 msgr2=0x7f17c819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:44.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.396+0000 7f17cd600640 1 --2- 172.21.15.43:0/4050704744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17c8109060 0x7f17c819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:44.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.396+0000 7f17cd600640 1 -- 172.21.15.43:0/4050704744 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17c819f9b0 con 0x7f17c8103f10 2024-09-19T05:04:44.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.396+0000 7f17cd600640 1 --2- 172.21.15.43:0/4050704744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17c8103f10 0x7f17c819a600 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f17bc004770 tx=0x7f17bc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:44.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.397+0000 7f17c6000640 1 -- 172.21.15.43:0/4050704744 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc03f070 con 0x7f17c8103f10 2024-09-19T05:04:44.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.397+0000 7f17c6000640 1 -- 172.21.15.43:0/4050704744 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17bc031ca0 con 0x7f17c8103f10 2024-09-19T05:04:44.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.397+0000 7f17c6000640 1 -- 172.21.15.43:0/4050704744 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc035030 con 0x7f17c8103f10 2024-09-19T05:04:44.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.397+0000 7f17d0000640 1 -- 172.21.15.43:0/4050704744 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17c819fc40 con 0x7f17c8103f10 2024-09-19T05:04:44.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.397+0000 7f17d0000640 1 -- 172.21.15.43:0/4050704744 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17c81a7400 con 0x7f17c8103f10 2024-09-19T05:04:44.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.398+0000 7f17d0000640 1 -- 172.21.15.43:0/4050704744 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17c81042f0 con 0x7f17c8103f10 2024-09-19T05:04:44.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.400+0000 7f17c6000640 1 -- 172.21.15.43:0/4050704744 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f17bc042070 con 0x7f17c8103f10 2024-09-19T05:04:44.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.401+0000 7f17c6000640 1 --2- 172.21.15.43:0/4050704744 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f17b00a34e0 0x7f17b00a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:44.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.401+0000 7f17c6000640 1 -- 172.21.15.43:0/4050704744 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f17bc0f04d0 con 0x7f17c8103f10 2024-09-19T05:04:44.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.401+0000 7f17ccc00640 1 --2- 172.21.15.43:0/4050704744 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f17b00a34e0 0x7f17b00a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:44.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.402+0000 7f17ccc00640 1 --2- 172.21.15.43:0/4050704744 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f17b00a34e0 0x7f17b00a59a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f17b40095f0 tx=0x7f17b4005f90 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:44.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.405+0000 7f17c6000640 1 -- 172.21.15.43:0/4050704744 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f17bc0bc7b0 con 0x7f17c8103f10 2024-09-19T05:04:44.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.607+0000 7f17d0000640 1 -- 172.21.15.43:0/4050704744 --> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f17c81048c0 con 0x7f17b00a34e0 2024-09-19T05:04:44.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.614+0000 7f17c6000640 1 -- 172.21.15.43:0/4050704744 <== mgr.25136 v2:172.21.15.43:6800/2120444093 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258096 (secure 0 0 0) 0x7f17c81048c0 con 0x7f17b00a34e0 2024-09-19T05:04:44.619 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:04:44.621 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-19T05:04:44.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.618+0000 7f17d0000640 1 -- 172.21.15.43:0/4050704744 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f17b00a34e0 msgr2=0x7f17b00a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:44.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.618+0000 7f17d0000640 1 --2- 172.21.15.43:0/4050704744 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f17b00a34e0 0x7f17b00a59a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f17b40095f0 tx=0x7f17b4005f90 comp rx=0 tx=0).stop 2024-09-19T05:04:44.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.618+0000 7f17d0000640 1 -- 172.21.15.43:0/4050704744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17c8103f10 msgr2=0x7f17c819a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:44.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.618+0000 7f17d0000640 1 --2- 172.21.15.43:0/4050704744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17c8103f10 0x7f17c819a600 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f17bc004770 tx=0x7f17bc003c10 comp rx=0 tx=0).stop 2024-09-19T05:04:44.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.619+0000 7f17d0000640 1 -- 172.21.15.43:0/4050704744 shutdown_connections 2024-09-19T05:04:44.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.619+0000 7f17d0000640 1 --2- 172.21.15.43:0/4050704744 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17c819eec0 0x7f17c819f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:44.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.619+0000 7f17d0000640 1 --2- 172.21.15.43:0/4050704744 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f17b00a34e0 0x7f17b00a59a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:44.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.619+0000 7f17d0000640 1 --2- 172.21.15.43:0/4050704744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17c8109060 0x7f17c819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:44.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.619+0000 7f17d0000640 1 --2- 172.21.15.43:0/4050704744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17c8103f10 0x7f17c819a600 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:44.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.619+0000 7f17d0000640 1 -- 172.21.15.43:0/4050704744 >> 172.21.15.43:0/4050704744 conn(0x7f17c80ff980 msgr2=0x7f17c8100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:44.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.619+0000 7f17d0000640 1 -- 172.21.15.43:0/4050704744 shutdown_connections 2024-09-19T05:04:44.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:44.619+0000 7f17d0000640 1 -- 172.21.15.43:0/4050704744 wait complete. 2024-09-19T05:04:44.637 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-09-19T05:04:44.261931+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1193702,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1475,"num_read_kb":911,"num_write":564,"num_write_kb":4412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":1181717,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":947,"ondisk_log_size":947,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":377696,"kb_used_data":11784,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124564128,"statfs":{"total":1151940427776,"available":1151553667072,"internally_reserved":0,"allocated":12066816,"data_stored":9078285,"data_compressed":280452,"data_compressed_allocated":368640,"data_compressed_original":3588096,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.982921"},"pg_stats":[{"pgid":"3.3f","version":"85'8","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.343604+0000","last_change":"2024-09-19T05:02:24.426572+0000","last_active":"2024-09-19T05:04:38.343604+0000","last_peered":"2024-09-19T05:04:38.343604+0000","last_clean":"2024-09-19T05:04:38.343604+0000","last_became_active":"2024-09-19T05:02:24.426241+0000","last_became_peered":"2024-09-19T05:02:24.426241+0000","last_unstale":"2024-09-19T05:04:38.343604+0000","last_undegraded":"2024-09-19T05:04:38.343604+0000","last_fullsized":"2024-09-19T05:04:38.343604+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:42.511495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.591304+0000","last_change":"2024-09-19T05:02:24.427263+0000","last_active":"2024-09-19T05:04:38.591304+0000","last_peered":"2024-09-19T05:04:38.591304+0000","last_clean":"2024-09-19T05:04:38.591304+0000","last_became_active":"2024-09-19T05:02:24.426928+0000","last_became_peered":"2024-09-19T05:02:24.426928+0000","last_unstale":"2024-09-19T05:04:38.591304+0000","last_undegraded":"2024-09-19T05:04:38.591304+0000","last_fullsized":"2024-09-19T05:04:38.591304+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:03.177020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.591094+0000","last_change":"2024-09-19T05:02:24.432709+0000","last_active":"2024-09-19T05:04:38.591094+0000","last_peered":"2024-09-19T05:04:38.591094+0000","last_clean":"2024-09-19T05:04:38.591094+0000","last_became_active":"2024-09-19T05:02:24.431114+0000","last_became_peered":"2024-09-19T05:02:24.431114+0000","last_unstale":"2024-09-19T05:04:38.591094+0000","last_undegraded":"2024-09-19T05:04:38.591094+0000","last_fullsized":"2024-09-19T05:04:38.591094+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:57.244919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.340848+0000","last_change":"2024-09-19T05:03:08.713389+0000","last_active":"2024-09-19T05:04:38.340848+0000","last_peered":"2024-09-19T05:04:38.340848+0000","last_clean":"2024-09-19T05:04:38.340848+0000","last_became_active":"2024-09-19T05:03:08.713058+0000","last_became_peered":"2024-09-19T05:03:08.713058+0000","last_unstale":"2024-09-19T05:04:38.340848+0000","last_undegraded":"2024-09-19T05:04:38.340848+0000","last_fullsized":"2024-09-19T05:04:38.340848+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:52:10.993733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587640+0000","last_change":"2024-09-19T05:02:24.419992+0000","last_active":"2024-09-19T05:04:38.587640+0000","last_peered":"2024-09-19T05:04:38.587640+0000","last_clean":"2024-09-19T05:04:38.587640+0000","last_became_active":"2024-09-19T05:02:24.419643+0000","last_became_peered":"2024-09-19T05:02:24.419643+0000","last_unstale":"2024-09-19T05:04:38.587640+0000","last_undegraded":"2024-09-19T05:04:38.587640+0000","last_fullsized":"2024-09-19T05:04:38.587640+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:54:40.548216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.356087+0000","last_change":"2024-09-19T05:02:24.434106+0000","last_active":"2024-09-19T05:04:38.356087+0000","last_peered":"2024-09-19T05:04:38.356087+0000","last_clean":"2024-09-19T05:04:38.356087+0000","last_became_active":"2024-09-19T05:02:24.433124+0000","last_became_peered":"2024-09-19T05:02:24.433124+0000","last_unstale":"2024-09-19T05:04:38.356087+0000","last_undegraded":"2024-09-19T05:04:38.356087+0000","last_fullsized":"2024-09-19T05:04:38.356087+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:09:46.502506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587605+0000","last_change":"2024-09-19T05:02:24.429640+0000","last_active":"2024-09-19T05:04:38.587605+0000","last_peered":"2024-09-19T05:04:38.587605+0000","last_clean":"2024-09-19T05:04:38.587605+0000","last_became_active":"2024-09-19T05:02:24.429326+0000","last_became_peered":"2024-09-19T05:02:24.429326+0000","last_unstale":"2024-09-19T05:04:38.587605+0000","last_undegraded":"2024-09-19T05:04:38.587605+0000","last_fullsized":"2024-09-19T05:04:38.587605+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:27.330449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.243840+0000","last_change":"2024-09-19T05:02:24.430933+0000","last_active":"2024-09-19T05:04:38.243840+0000","last_peered":"2024-09-19T05:04:38.243840+0000","last_clean":"2024-09-19T05:04:38.243840+0000","last_became_active":"2024-09-19T05:02:24.430478+0000","last_became_peered":"2024-09-19T05:02:24.430478+0000","last_unstale":"2024-09-19T05:04:38.243840+0000","last_undegraded":"2024-09-19T05:04:38.243840+0000","last_fullsized":"2024-09-19T05:04:38.243840+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:45:22.171049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.585189+0000","last_change":"2024-09-19T05:02:24.433207+0000","last_active":"2024-09-19T05:04:38.585189+0000","last_peered":"2024-09-19T05:04:38.585189+0000","last_clean":"2024-09-19T05:04:38.585189+0000","last_became_active":"2024-09-19T05:02:24.432882+0000","last_became_peered":"2024-09-19T05:02:24.432882+0000","last_unstale":"2024-09-19T05:04:38.585189+0000","last_undegraded":"2024-09-19T05:04:38.585189+0000","last_fullsized":"2024-09-19T05:04:38.585189+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:06:42.748227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.356558+0000","last_change":"2024-09-19T05:02:24.433825+0000","last_active":"2024-09-19T05:04:38.356558+0000","last_peered":"2024-09-19T05:04:38.356558+0000","last_clean":"2024-09-19T05:04:38.356558+0000","last_became_active":"2024-09-19T05:02:24.433088+0000","last_became_peered":"2024-09-19T05:02:24.433088+0000","last_unstale":"2024-09-19T05:04:38.356558+0000","last_undegraded":"2024-09-19T05:04:38.356558+0000","last_fullsized":"2024-09-19T05:04:38.356558+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:12:53.039028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587972+0000","last_change":"2024-09-19T05:02:24.427661+0000","last_active":"2024-09-19T05:04:38.587972+0000","last_peered":"2024-09-19T05:04:38.587972+0000","last_clean":"2024-09-19T05:04:38.587972+0000","last_became_active":"2024-09-19T05:02:24.427093+0000","last_became_peered":"2024-09-19T05:02:24.427093+0000","last_unstale":"2024-09-19T05:04:38.587972+0000","last_undegraded":"2024-09-19T05:04:38.587972+0000","last_fullsized":"2024-09-19T05:04:38.587972+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:43.091041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.245390+0000","last_change":"2024-09-19T05:02:24.430820+0000","last_active":"2024-09-19T05:04:38.245390+0000","last_peered":"2024-09-19T05:04:38.245390+0000","last_clean":"2024-09-19T05:04:38.245390+0000","last_became_active":"2024-09-19T05:02:24.429945+0000","last_became_peered":"2024-09-19T05:02:24.429945+0000","last_unstale":"2024-09-19T05:04:38.245390+0000","last_undegraded":"2024-09-19T05:04:38.245390+0000","last_fullsized":"2024-09-19T05:04:38.245390+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:21:59.400725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.564024+0000","last_change":"2024-09-19T05:02:24.428695+0000","last_active":"2024-09-19T05:04:38.564024+0000","last_peered":"2024-09-19T05:04:38.564024+0000","last_clean":"2024-09-19T05:04:38.564024+0000","last_became_active":"2024-09-19T05:02:24.426385+0000","last_became_peered":"2024-09-19T05:02:24.426385+0000","last_unstale":"2024-09-19T05:04:38.564024+0000","last_undegraded":"2024-09-19T05:04:38.564024+0000","last_fullsized":"2024-09-19T05:04:38.564024+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:44:00.893313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.590202+0000","last_change":"2024-09-19T05:02:24.424935+0000","last_active":"2024-09-19T05:04:38.590202+0000","last_peered":"2024-09-19T05:04:38.590202+0000","last_clean":"2024-09-19T05:04:38.590202+0000","last_became_active":"2024-09-19T05:02:24.424219+0000","last_became_peered":"2024-09-19T05:02:24.424219+0000","last_unstale":"2024-09-19T05:04:38.590202+0000","last_undegraded":"2024-09-19T05:04:38.590202+0000","last_fullsized":"2024-09-19T05:04:38.590202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:00.890778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.589476+0000","last_change":"2024-09-19T05:02:24.425407+0000","last_active":"2024-09-19T05:04:38.589476+0000","last_peered":"2024-09-19T05:04:38.589476+0000","last_clean":"2024-09-19T05:04:38.589476+0000","last_became_active":"2024-09-19T05:02:24.425053+0000","last_became_peered":"2024-09-19T05:02:24.425053+0000","last_unstale":"2024-09-19T05:04:38.589476+0000","last_undegraded":"2024-09-19T05:04:38.589476+0000","last_fullsized":"2024-09-19T05:04:38.589476+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:54:10.024183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.584494+0000","last_change":"2024-09-19T05:02:24.427888+0000","last_active":"2024-09-19T05:04:38.584494+0000","last_peered":"2024-09-19T05:04:38.584494+0000","last_clean":"2024-09-19T05:04:38.584494+0000","last_became_active":"2024-09-19T05:02:24.427589+0000","last_became_peered":"2024-09-19T05:02:24.427589+0000","last_unstale":"2024-09-19T05:04:38.584494+0000","last_undegraded":"2024-09-19T05:04:38.584494+0000","last_fullsized":"2024-09-19T05:04:38.584494+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:02:02.654416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.343559+0000","last_change":"2024-09-19T05:02:24.426654+0000","last_active":"2024-09-19T05:04:38.343559+0000","last_peered":"2024-09-19T05:04:38.343559+0000","last_clean":"2024-09-19T05:04:38.343559+0000","last_became_active":"2024-09-19T05:02:24.426352+0000","last_became_peered":"2024-09-19T05:02:24.426352+0000","last_unstale":"2024-09-19T05:04:38.343559+0000","last_undegraded":"2024-09-19T05:04:38.343559+0000","last_fullsized":"2024-09-19T05:04:38.343559+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:20.055222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.340926+0000","last_change":"2024-09-19T05:02:24.432178+0000","last_active":"2024-09-19T05:04:38.340926+0000","last_peered":"2024-09-19T05:04:38.340926+0000","last_clean":"2024-09-19T05:04:38.340926+0000","last_became_active":"2024-09-19T05:02:24.429058+0000","last_became_peered":"2024-09-19T05:02:24.429058+0000","last_unstale":"2024-09-19T05:04:38.340926+0000","last_undegraded":"2024-09-19T05:04:38.340926+0000","last_fullsized":"2024-09-19T05:04:38.340926+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:08.001134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.455282+0000","last_change":"2024-09-19T05:02:24.428202+0000","last_active":"2024-09-19T05:04:38.455282+0000","last_peered":"2024-09-19T05:04:38.455282+0000","last_clean":"2024-09-19T05:04:38.455282+0000","last_became_active":"2024-09-19T05:02:24.427741+0000","last_became_peered":"2024-09-19T05:02:24.427741+0000","last_unstale":"2024-09-19T05:04:38.455282+0000","last_undegraded":"2024-09-19T05:04:38.455282+0000","last_fullsized":"2024-09-19T05:04:38.455282+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:42:51.076056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.343183+0000","last_change":"2024-09-19T05:02:24.425007+0000","last_active":"2024-09-19T05:04:38.343183+0000","last_peered":"2024-09-19T05:04:38.343183+0000","last_clean":"2024-09-19T05:04:38.343183+0000","last_became_active":"2024-09-19T05:02:24.424612+0000","last_became_peered":"2024-09-19T05:02:24.424612+0000","last_unstale":"2024-09-19T05:04:38.343183+0000","last_undegraded":"2024-09-19T05:04:38.343183+0000","last_fullsized":"2024-09-19T05:04:38.343183+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:36:06.115757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.564225+0000","last_change":"2024-09-19T05:02:24.428724+0000","last_active":"2024-09-19T05:04:38.564225+0000","last_peered":"2024-09-19T05:04:38.564225+0000","last_clean":"2024-09-19T05:04:38.564225+0000","last_became_active":"2024-09-19T05:02:24.427730+0000","last_became_peered":"2024-09-19T05:02:24.427730+0000","last_unstale":"2024-09-19T05:04:38.564225+0000","last_undegraded":"2024-09-19T05:04:38.564225+0000","last_fullsized":"2024-09-19T05:04:38.564225+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:15:25.466553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.584736+0000","last_change":"2024-09-19T05:02:24.426238+0000","last_active":"2024-09-19T05:04:38.584736+0000","last_peered":"2024-09-19T05:04:38.584736+0000","last_clean":"2024-09-19T05:04:38.584736+0000","last_became_active":"2024-09-19T05:02:24.425915+0000","last_became_peered":"2024-09-19T05:02:24.425915+0000","last_unstale":"2024-09-19T05:04:38.584736+0000","last_undegraded":"2024-09-19T05:04:38.584736+0000","last_fullsized":"2024-09-19T05:04:38.584736+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:37:18.444042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.243918+0000","last_change":"2024-09-19T05:02:24.425726+0000","last_active":"2024-09-19T05:04:38.243918+0000","last_peered":"2024-09-19T05:04:38.243918+0000","last_clean":"2024-09-19T05:04:38.243918+0000","last_became_active":"2024-09-19T05:02:24.425409+0000","last_became_peered":"2024-09-19T05:02:24.425409+0000","last_unstale":"2024-09-19T05:04:38.243918+0000","last_undegraded":"2024-09-19T05:04:38.243918+0000","last_fullsized":"2024-09-19T05:04:38.243918+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:12.068304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.245729+0000","last_change":"2024-09-19T05:02:24.425267+0000","last_active":"2024-09-19T05:04:38.245729+0000","last_peered":"2024-09-19T05:04:38.245729+0000","last_clean":"2024-09-19T05:04:38.245729+0000","last_became_active":"2024-09-19T05:02:24.424933+0000","last_became_peered":"2024-09-19T05:02:24.424933+0000","last_unstale":"2024-09-19T05:04:38.245729+0000","last_undegraded":"2024-09-19T05:04:38.245729+0000","last_fullsized":"2024-09-19T05:04:38.245729+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:51:55.274870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.244581+0000","last_change":"2024-09-19T05:02:24.427909+0000","last_active":"2024-09-19T05:04:38.244581+0000","last_peered":"2024-09-19T05:04:38.244581+0000","last_clean":"2024-09-19T05:04:38.244581+0000","last_became_active":"2024-09-19T05:02:24.425515+0000","last_became_peered":"2024-09-19T05:02:24.425515+0000","last_unstale":"2024-09-19T05:04:38.244581+0000","last_undegraded":"2024-09-19T05:04:38.244581+0000","last_fullsized":"2024-09-19T05:04:38.244581+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:12:22.390754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.455426+0000","last_change":"2024-09-19T05:02:24.426201+0000","last_active":"2024-09-19T05:04:38.455426+0000","last_peered":"2024-09-19T05:04:38.455426+0000","last_clean":"2024-09-19T05:04:38.455426+0000","last_became_active":"2024-09-19T05:02:24.425890+0000","last_became_peered":"2024-09-19T05:02:24.425890+0000","last_unstale":"2024-09-19T05:04:38.455426+0000","last_undegraded":"2024-09-19T05:04:38.455426+0000","last_fullsized":"2024-09-19T05:04:38.455426+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:36.680002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.343493+0000","last_change":"2024-09-19T05:02:24.416037+0000","last_active":"2024-09-19T05:04:38.343493+0000","last_peered":"2024-09-19T05:04:38.343493+0000","last_clean":"2024-09-19T05:04:38.343493+0000","last_became_active":"2024-09-19T05:02:24.415535+0000","last_became_peered":"2024-09-19T05:02:24.415535+0000","last_unstale":"2024-09-19T05:04:38.343493+0000","last_undegraded":"2024-09-19T05:04:38.343493+0000","last_fullsized":"2024-09-19T05:04:38.343493+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:14.756410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.564248+0000","last_change":"2024-09-19T05:03:08.708063+0000","last_active":"2024-09-19T05:04:38.564248+0000","last_peered":"2024-09-19T05:04:38.564248+0000","last_clean":"2024-09-19T05:04:38.564248+0000","last_became_active":"2024-09-19T05:03:08.707474+0000","last_became_peered":"2024-09-19T05:03:08.707474+0000","last_unstale":"2024-09-19T05:04:38.564248+0000","last_undegraded":"2024-09-19T05:04:38.564248+0000","last_fullsized":"2024-09-19T05:04:38.564248+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:20:55.604156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.589950+0000","last_change":"2024-09-19T05:02:24.423776+0000","last_active":"2024-09-19T05:04:38.589950+0000","last_peered":"2024-09-19T05:04:38.589950+0000","last_clean":"2024-09-19T05:04:38.589950+0000","last_became_active":"2024-09-19T05:02:24.423446+0000","last_became_peered":"2024-09-19T05:02:24.423446+0000","last_unstale":"2024-09-19T05:04:38.589950+0000","last_undegraded":"2024-09-19T05:04:38.589950+0000","last_fullsized":"2024-09-19T05:04:38.589950+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:47:02.747049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.455107+0000","last_change":"2024-09-19T05:02:24.420534+0000","last_active":"2024-09-19T05:04:38.455107+0000","last_peered":"2024-09-19T05:04:38.455107+0000","last_clean":"2024-09-19T05:04:38.455107+0000","last_became_active":"2024-09-19T05:02:24.419980+0000","last_became_peered":"2024-09-19T05:02:24.419980+0000","last_unstale":"2024-09-19T05:04:38.455107+0000","last_undegraded":"2024-09-19T05:04:38.455107+0000","last_fullsized":"2024-09-19T05:04:38.455107+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:33:49.537258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.455289+0000","last_change":"2024-09-19T05:02:24.424859+0000","last_active":"2024-09-19T05:04:38.455289+0000","last_peered":"2024-09-19T05:04:38.455289+0000","last_clean":"2024-09-19T05:04:38.455289+0000","last_became_active":"2024-09-19T05:02:24.424514+0000","last_became_peered":"2024-09-19T05:02:24.424514+0000","last_unstale":"2024-09-19T05:04:38.455289+0000","last_undegraded":"2024-09-19T05:04:38.455289+0000","last_fullsized":"2024-09-19T05:04:38.455289+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:41:41.929578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.343253+0000","last_change":"2024-09-19T05:02:24.421756+0000","last_active":"2024-09-19T05:04:38.343253+0000","last_peered":"2024-09-19T05:04:38.343253+0000","last_clean":"2024-09-19T05:04:38.343253+0000","last_became_active":"2024-09-19T05:02:24.420910+0000","last_became_peered":"2024-09-19T05:02:24.420910+0000","last_unstale":"2024-09-19T05:04:38.343253+0000","last_undegraded":"2024-09-19T05:04:38.343253+0000","last_fullsized":"2024-09-19T05:04:38.343253+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:33.759745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587628+0000","last_change":"2024-09-19T05:02:24.428514+0000","last_active":"2024-09-19T05:04:38.587628+0000","last_peered":"2024-09-19T05:04:38.587628+0000","last_clean":"2024-09-19T05:04:38.587628+0000","last_became_active":"2024-09-19T05:02:24.428206+0000","last_became_peered":"2024-09-19T05:02:24.428206+0000","last_unstale":"2024-09-19T05:04:38.587628+0000","last_undegraded":"2024-09-19T05:04:38.587628+0000","last_fullsized":"2024-09-19T05:04:38.587628+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:33.045739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"85'7","reported_seq":157,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.246167+0000","last_change":"2024-09-19T05:02:22.026683+0000","last_active":"2024-09-19T05:04:38.246167+0000","last_peered":"2024-09-19T05:04:38.246167+0000","last_clean":"2024-09-19T05:04:38.246167+0000","last_became_active":"2024-09-19T05:02:22.026342+0000","last_became_peered":"2024-09-19T05:02:22.026342+0000","last_unstale":"2024-09-19T05:04:38.246167+0000","last_undegraded":"2024-09-19T05:04:38.246167+0000","last_fullsized":"2024-09-19T05:04:38.246167+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:01:37.462838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.245923+0000","last_change":"2024-09-19T05:02:24.421753+0000","last_active":"2024-09-19T05:04:38.245923+0000","last_peered":"2024-09-19T05:04:38.245923+0000","last_clean":"2024-09-19T05:04:38.245923+0000","last_became_active":"2024-09-19T05:02:24.421455+0000","last_became_peered":"2024-09-19T05:02:24.421455+0000","last_unstale":"2024-09-19T05:04:38.245923+0000","last_undegraded":"2024-09-19T05:04:38.245923+0000","last_fullsized":"2024-09-19T05:04:38.245923+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:06.416550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.356217+0000","last_change":"2024-09-19T05:03:08.716866+0000","last_active":"2024-09-19T05:04:38.356217+0000","last_peered":"2024-09-19T05:04:38.356217+0000","last_clean":"2024-09-19T05:04:38.356217+0000","last_became_active":"2024-09-19T05:03:08.709783+0000","last_became_peered":"2024-09-19T05:03:08.709783+0000","last_unstale":"2024-09-19T05:04:38.356217+0000","last_undegraded":"2024-09-19T05:04:38.356217+0000","last_fullsized":"2024-09-19T05:04:38.356217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:26.103651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.563762+0000","last_change":"2024-09-19T05:02:22.026599+0000","last_active":"2024-09-19T05:04:38.563762+0000","last_peered":"2024-09-19T05:04:38.563762+0000","last_clean":"2024-09-19T05:04:38.563762+0000","last_became_active":"2024-09-19T05:02:22.024652+0000","last_became_peered":"2024-09-19T05:02:22.024652+0000","last_unstale":"2024-09-19T05:04:38.563762+0000","last_undegraded":"2024-09-19T05:04:38.563762+0000","last_fullsized":"2024-09-19T05:04:38.563762+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:14:32.508333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.243898+0000","last_change":"2024-09-19T05:02:24.419262+0000","last_active":"2024-09-19T05:04:38.243898+0000","last_peered":"2024-09-19T05:04:38.243898+0000","last_clean":"2024-09-19T05:04:38.243898+0000","last_became_active":"2024-09-19T05:02:24.418589+0000","last_became_peered":"2024-09-19T05:02:24.418589+0000","last_unstale":"2024-09-19T05:04:38.243898+0000","last_undegraded":"2024-09-19T05:04:38.243898+0000","last_fullsized":"2024-09-19T05:04:38.243898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:27.256676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.245917+0000","last_change":"2024-09-19T05:02:22.024225+0000","last_active":"2024-09-19T05:04:38.245917+0000","last_peered":"2024-09-19T05:04:38.245917+0000","last_clean":"2024-09-19T05:04:38.245917+0000","last_became_active":"2024-09-19T05:02:22.023882+0000","last_became_peered":"2024-09-19T05:02:22.023882+0000","last_unstale":"2024-09-19T05:04:38.245917+0000","last_undegraded":"2024-09-19T05:04:38.245917+0000","last_fullsized":"2024-09-19T05:04:38.245917+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:29.619634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.340536+0000","last_change":"2024-09-19T05:02:24.436511+0000","last_active":"2024-09-19T05:04:38.340536+0000","last_peered":"2024-09-19T05:04:38.340536+0000","last_clean":"2024-09-19T05:04:38.340536+0000","last_became_active":"2024-09-19T05:02:24.435565+0000","last_became_peered":"2024-09-19T05:02:24.435565+0000","last_unstale":"2024-09-19T05:04:38.340536+0000","last_undegraded":"2024-09-19T05:04:38.340536+0000","last_fullsized":"2024-09-19T05:04:38.340536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:16:43.779420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'7","reported_seq":273,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.683927+0000","last_change":"2024-09-19T05:02:22.024281+0000","last_active":"2024-09-19T05:04:38.683927+0000","last_peered":"2024-09-19T05:04:38.683927+0000","last_clean":"2024-09-19T05:04:38.683927+0000","last_became_active":"2024-09-19T05:02:22.023868+0000","last_became_peered":"2024-09-19T05:02:22.023868+0000","last_unstale":"2024-09-19T05:04:38.683927+0000","last_undegraded":"2024-09-19T05:04:38.683927+0000","last_fullsized":"2024-09-19T05:04:38.683927+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:29:47.597570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":240,"num_read_kb":240,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.584958+0000","last_change":"2024-09-19T05:02:24.436383+0000","last_active":"2024-09-19T05:04:38.584958+0000","last_peered":"2024-09-19T05:04:38.584958+0000","last_clean":"2024-09-19T05:04:38.584958+0000","last_became_active":"2024-09-19T05:02:24.436062+0000","last_became_peered":"2024-09-19T05:02:24.436062+0000","last_unstale":"2024-09-19T05:04:38.584958+0000","last_undegraded":"2024-09-19T05:04:38.584958+0000","last_fullsized":"2024-09-19T05:04:38.584958+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:48:42.574431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587817+0000","last_change":"2024-09-19T05:02:22.026950+0000","last_active":"2024-09-19T05:04:38.587817+0000","last_peered":"2024-09-19T05:04:38.587817+0000","last_clean":"2024-09-19T05:04:38.587817+0000","last_became_active":"2024-09-19T05:02:22.026498+0000","last_became_peered":"2024-09-19T05:02:22.026498+0000","last_unstale":"2024-09-19T05:04:38.587817+0000","last_undegraded":"2024-09-19T05:04:38.587817+0000","last_fullsized":"2024-09-19T05:04:38.587817+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:55:31.833714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.591864+0000","last_change":"2024-09-19T05:03:08.897727+0000","last_active":"2024-09-19T05:04:38.591864+0000","last_peered":"2024-09-19T05:04:38.591864+0000","last_clean":"2024-09-19T05:04:38.591864+0000","last_became_active":"2024-09-19T05:03:08.897382+0000","last_became_peered":"2024-09-19T05:03:08.897382+0000","last_unstale":"2024-09-19T05:04:38.591864+0000","last_undegraded":"2024-09-19T05:04:38.591864+0000","last_fullsized":"2024-09-19T05:04:38.591864+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:13.319291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.246219+0000","last_change":"2024-09-19T05:02:22.075901+0000","last_active":"2024-09-19T05:04:38.246219+0000","last_peered":"2024-09-19T05:04:38.246219+0000","last_clean":"2024-09-19T05:04:38.246219+0000","last_became_active":"2024-09-19T05:02:22.075512+0000","last_became_peered":"2024-09-19T05:02:22.075512+0000","last_unstale":"2024-09-19T05:04:38.246219+0000","last_undegraded":"2024-09-19T05:04:38.246219+0000","last_fullsized":"2024-09-19T05:04:38.246219+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:09:26.976888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.584706+0000","last_change":"2024-09-19T05:02:24.436797+0000","last_active":"2024-09-19T05:04:38.584706+0000","last_peered":"2024-09-19T05:04:38.584706+0000","last_clean":"2024-09-19T05:04:38.584706+0000","last_became_active":"2024-09-19T05:02:24.436457+0000","last_became_peered":"2024-09-19T05:02:24.436457+0000","last_unstale":"2024-09-19T05:04:38.584706+0000","last_undegraded":"2024-09-19T05:04:38.584706+0000","last_fullsized":"2024-09-19T05:04:38.584706+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:32:44.702601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'39","reported_seq":317,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.705059+0000","last_change":"2024-09-19T05:02:22.027415+0000","last_active":"2024-09-19T05:04:38.705059+0000","last_peered":"2024-09-19T05:04:38.705059+0000","last_clean":"2024-09-19T05:04:38.705059+0000","last_became_active":"2024-09-19T05:02:22.027065+0000","last_became_peered":"2024-09-19T05:02:22.027065+0000","last_unstale":"2024-09-19T05:04:38.705059+0000","last_undegraded":"2024-09-19T05:04:38.705059+0000","last_fullsized":"2024-09-19T05:04:38.705059+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:02.599310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68547,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":252,"num_read_kb":192,"num_write":41,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587855+0000","last_change":"2024-09-19T05:02:24.419718+0000","last_active":"2024-09-19T05:04:38.587855+0000","last_peered":"2024-09-19T05:04:38.587855+0000","last_clean":"2024-09-19T05:04:38.587855+0000","last_became_active":"2024-09-19T05:02:24.419388+0000","last_became_peered":"2024-09-19T05:02:24.419388+0000","last_unstale":"2024-09-19T05:04:38.587855+0000","last_undegraded":"2024-09-19T05:04:38.587855+0000","last_fullsized":"2024-09-19T05:04:38.587855+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:09.453551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.455176+0000","last_change":"2024-09-19T05:02:22.026220+0000","last_active":"2024-09-19T05:04:38.455176+0000","last_peered":"2024-09-19T05:04:38.455176+0000","last_clean":"2024-09-19T05:04:38.455176+0000","last_became_active":"2024-09-19T05:02:22.025838+0000","last_became_peered":"2024-09-19T05:02:22.025838+0000","last_unstale":"2024-09-19T05:04:38.455176+0000","last_undegraded":"2024-09-19T05:04:38.455176+0000","last_fullsized":"2024-09-19T05:04:38.455176+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:02.192799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.563968+0000","last_change":"2024-09-19T05:02:24.431872+0000","last_active":"2024-09-19T05:04:38.563968+0000","last_peered":"2024-09-19T05:04:38.563968+0000","last_clean":"2024-09-19T05:04:38.563968+0000","last_became_active":"2024-09-19T05:02:24.427227+0000","last_became_peered":"2024-09-19T05:02:24.427227+0000","last_unstale":"2024-09-19T05:04:38.563968+0000","last_undegraded":"2024-09-19T05:04:38.563968+0000","last_fullsized":"2024-09-19T05:04:38.563968+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:52.655108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.245966+0000","last_change":"2024-09-19T05:02:22.025136+0000","last_active":"2024-09-19T05:04:38.245966+0000","last_peered":"2024-09-19T05:04:38.245966+0000","last_clean":"2024-09-19T05:04:38.245966+0000","last_became_active":"2024-09-19T05:02:22.024813+0000","last_became_peered":"2024-09-19T05:02:22.024813+0000","last_unstale":"2024-09-19T05:04:38.245966+0000","last_undegraded":"2024-09-19T05:04:38.245966+0000","last_fullsized":"2024-09-19T05:04:38.245966+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:47:03.956661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587730+0000","last_change":"2024-09-19T05:02:24.428497+0000","last_active":"2024-09-19T05:04:38.587730+0000","last_peered":"2024-09-19T05:04:38.587730+0000","last_clean":"2024-09-19T05:04:38.587730+0000","last_became_active":"2024-09-19T05:02:24.427774+0000","last_became_peered":"2024-09-19T05:02:24.427774+0000","last_unstale":"2024-09-19T05:04:38.587730+0000","last_undegraded":"2024-09-19T05:04:38.587730+0000","last_fullsized":"2024-09-19T05:04:38.587730+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:32:02.160354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'8","reported_seq":174,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.845746+0000","last_change":"2024-09-19T05:03:08.718249+0000","last_active":"2024-09-19T05:04:38.845746+0000","last_peered":"2024-09-19T05:04:38.845746+0000","last_clean":"2024-09-19T05:04:38.845746+0000","last_became_active":"2024-09-19T05:03:08.713169+0000","last_became_peered":"2024-09-19T05:03:08.713169+0000","last_unstale":"2024-09-19T05:04:38.845746+0000","last_undegraded":"2024-09-19T05:04:38.845746+0000","last_fullsized":"2024-09-19T05:04:38.845746+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:34.640353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1515,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":981,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,10],"acting":[4,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587563+0000","last_change":"2024-09-19T05:02:24.433098+0000","last_active":"2024-09-19T05:04:38.587563+0000","last_peered":"2024-09-19T05:04:38.587563+0000","last_clean":"2024-09-19T05:04:38.587563+0000","last_became_active":"2024-09-19T05:02:24.432800+0000","last_became_peered":"2024-09-19T05:02:24.432800+0000","last_unstale":"2024-09-19T05:04:38.587563+0000","last_undegraded":"2024-09-19T05:04:38.587563+0000","last_fullsized":"2024-09-19T05:04:38.587563+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:50:30.163842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587677+0000","last_change":"2024-09-19T05:02:22.075865+0000","last_active":"2024-09-19T05:04:38.587677+0000","last_peered":"2024-09-19T05:04:38.587677+0000","last_clean":"2024-09-19T05:04:38.587677+0000","last_became_active":"2024-09-19T05:02:22.075185+0000","last_became_peered":"2024-09-19T05:02:22.075185+0000","last_unstale":"2024-09-19T05:04:38.587677+0000","last_undegraded":"2024-09-19T05:04:38.587677+0000","last_fullsized":"2024-09-19T05:04:38.587677+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:19.150155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.245715+0000","last_change":"2024-09-19T05:02:24.422759+0000","last_active":"2024-09-19T05:04:38.245715+0000","last_peered":"2024-09-19T05:04:38.245715+0000","last_clean":"2024-09-19T05:04:38.245715+0000","last_became_active":"2024-09-19T05:02:24.422421+0000","last_became_peered":"2024-09-19T05:02:24.422421+0000","last_unstale":"2024-09-19T05:04:38.245715+0000","last_undegraded":"2024-09-19T05:04:38.245715+0000","last_fullsized":"2024-09-19T05:04:38.245715+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:58:50.083017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.588067+0000","last_change":"2024-09-19T05:02:22.024305+0000","last_active":"2024-09-19T05:04:38.588067+0000","last_peered":"2024-09-19T05:04:38.588067+0000","last_clean":"2024-09-19T05:04:38.588067+0000","last_became_active":"2024-09-19T05:02:22.023616+0000","last_became_peered":"2024-09-19T05:02:22.023616+0000","last_unstale":"2024-09-19T05:04:38.588067+0000","last_undegraded":"2024-09-19T05:04:38.588067+0000","last_fullsized":"2024-09-19T05:04:38.588067+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:05.665219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.343423+0000","last_change":"2024-09-19T05:03:08.713219+0000","last_active":"2024-09-19T05:04:38.343423+0000","last_peered":"2024-09-19T05:04:38.343423+0000","last_clean":"2024-09-19T05:04:38.343423+0000","last_became_active":"2024-09-19T05:03:08.712908+0000","last_became_peered":"2024-09-19T05:03:08.712908+0000","last_unstale":"2024-09-19T05:04:38.343423+0000","last_undegraded":"2024-09-19T05:04:38.343423+0000","last_fullsized":"2024-09-19T05:04:38.343423+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:23.366098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.564341+0000","last_change":"2024-09-19T05:03:08.720385+0000","last_active":"2024-09-19T05:04:38.564341+0000","last_peered":"2024-09-19T05:04:38.564341+0000","last_clean":"2024-09-19T05:04:38.564341+0000","last_became_active":"2024-09-19T05:03:08.715635+0000","last_became_peered":"2024-09-19T05:03:08.715635+0000","last_unstale":"2024-09-19T05:04:38.564341+0000","last_undegraded":"2024-09-19T05:04:38.564341+0000","last_fullsized":"2024-09-19T05:04:38.564341+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:09:57.627667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587953+0000","last_change":"2024-09-19T05:02:24.419360+0000","last_active":"2024-09-19T05:04:38.587953+0000","last_peered":"2024-09-19T05:04:38.587953+0000","last_clean":"2024-09-19T05:04:38.587953+0000","last_became_active":"2024-09-19T05:02:24.419040+0000","last_became_peered":"2024-09-19T05:02:24.419040+0000","last_unstale":"2024-09-19T05:04:38.587953+0000","last_undegraded":"2024-09-19T05:04:38.587953+0000","last_fullsized":"2024-09-19T05:04:38.587953+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:48:12.922676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'41","reported_seq":76,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.532840+0000","last_change":"2024-09-19T05:02:22.023124+0000","last_active":"2024-09-19T05:04:38.532840+0000","last_peered":"2024-09-19T05:04:38.532840+0000","last_clean":"2024-09-19T05:04:38.532840+0000","last_became_active":"2024-09-19T05:02:22.022830+0000","last_became_peered":"2024-09-19T05:02:22.022830+0000","last_unstale":"2024-09-19T05:04:38.532840+0000","last_undegraded":"2024-09-19T05:04:38.532840+0000","last_fullsized":"2024-09-19T05:04:38.532840+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:22:13.208616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'7","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587815+0000","last_change":"2024-09-19T05:02:24.426169+0000","last_active":"2024-09-19T05:04:38.587815+0000","last_peered":"2024-09-19T05:04:38.587815+0000","last_clean":"2024-09-19T05:04:38.587815+0000","last_became_active":"2024-09-19T05:02:24.425816+0000","last_became_peered":"2024-09-19T05:02:24.425816+0000","last_unstale":"2024-09-19T05:04:38.587815+0000","last_undegraded":"2024-09-19T05:04:38.587815+0000","last_fullsized":"2024-09-19T05:04:38.587815+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:56:11.735351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'8","reported_seq":160,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.684817+0000","last_change":"2024-09-19T05:02:22.154391+0000","last_active":"2024-09-19T05:04:38.684817+0000","last_peered":"2024-09-19T05:04:38.684817+0000","last_clean":"2024-09-19T05:04:38.684817+0000","last_became_active":"2024-09-19T05:02:22.154008+0000","last_became_peered":"2024-09-19T05:02:22.154008+0000","last_unstale":"2024-09-19T05:04:38.684817+0000","last_undegraded":"2024-09-19T05:04:38.684817+0000","last_fullsized":"2024-09-19T05:04:38.684817+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:29.997649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":125,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.588194+0000","last_change":"2024-09-19T05:02:24.427416+0000","last_active":"2024-09-19T05:04:38.588194+0000","last_peered":"2024-09-19T05:04:38.588194+0000","last_clean":"2024-09-19T05:04:38.588194+0000","last_became_active":"2024-09-19T05:02:24.426645+0000","last_became_peered":"2024-09-19T05:02:24.426645+0000","last_unstale":"2024-09-19T05:04:38.588194+0000","last_undegraded":"2024-09-19T05:04:38.588194+0000","last_fullsized":"2024-09-19T05:04:38.588194+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:21.841013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":351,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587790+0000","last_change":"2024-09-19T05:01:04.407740+0000","last_active":"2024-09-19T05:04:38.587790+0000","last_peered":"2024-09-19T05:04:38.587790+0000","last_clean":"2024-09-19T05:04:38.587790+0000","last_became_active":"2024-09-19T05:01:04.407281+0000","last_became_peered":"2024-09-19T05:01:04.407281+0000","last_unstale":"2024-09-19T05:04:38.587790+0000","last_undegraded":"2024-09-19T05:04:38.587790+0000","last_fullsized":"2024-09-19T05:04:38.587790+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:57:27.643438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:57:27.643438+0000","last_clean_scrub_stamp":"2024-09-19T04:57:27.643438+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:54:50.242926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.245478+0000","last_change":"2024-09-19T05:02:22.023991+0000","last_active":"2024-09-19T05:04:38.245478+0000","last_peered":"2024-09-19T05:04:38.245478+0000","last_clean":"2024-09-19T05:04:38.245478+0000","last_became_active":"2024-09-19T05:02:22.023361+0000","last_became_peered":"2024-09-19T05:02:22.023361+0000","last_unstale":"2024-09-19T05:04:38.245478+0000","last_undegraded":"2024-09-19T05:04:38.245478+0000","last_fullsized":"2024-09-19T05:04:38.245478+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:37:05.551318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.244168+0000","last_change":"2024-09-19T05:02:24.424473+0000","last_active":"2024-09-19T05:04:38.244168+0000","last_peered":"2024-09-19T05:04:38.244168+0000","last_clean":"2024-09-19T05:04:38.244168+0000","last_became_active":"2024-09-19T05:02:24.423968+0000","last_became_peered":"2024-09-19T05:02:24.423968+0000","last_unstale":"2024-09-19T05:04:38.244168+0000","last_undegraded":"2024-09-19T05:04:38.244168+0000","last_fullsized":"2024-09-19T05:04:38.244168+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:29:53.305036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587734+0000","last_change":"2024-09-19T05:02:22.026278+0000","last_active":"2024-09-19T05:04:38.587734+0000","last_peered":"2024-09-19T05:04:38.587734+0000","last_clean":"2024-09-19T05:04:38.587734+0000","last_became_active":"2024-09-19T05:02:22.025759+0000","last_became_peered":"2024-09-19T05:02:22.025759+0000","last_unstale":"2024-09-19T05:04:38.587734+0000","last_undegraded":"2024-09-19T05:04:38.587734+0000","last_fullsized":"2024-09-19T05:04:38.587734+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:21:21.696382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.563673+0000","last_change":"2024-09-19T05:02:24.426880+0000","last_active":"2024-09-19T05:04:38.563673+0000","last_peered":"2024-09-19T05:04:38.563673+0000","last_clean":"2024-09-19T05:04:38.563673+0000","last_became_active":"2024-09-19T05:02:24.426336+0000","last_became_peered":"2024-09-19T05:02:24.426336+0000","last_unstale":"2024-09-19T05:04:38.563673+0000","last_undegraded":"2024-09-19T05:04:38.563673+0000","last_fullsized":"2024-09-19T05:04:38.563673+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:51:07.110738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"86'7","reported_seq":155,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.594472+0000","last_change":"2024-09-19T05:02:22.075789+0000","last_active":"2024-09-19T05:04:38.594472+0000","last_peered":"2024-09-19T05:04:38.594472+0000","last_clean":"2024-09-19T05:04:38.594472+0000","last_became_active":"2024-09-19T05:02:22.075415+0000","last_became_peered":"2024-09-19T05:02:22.075415+0000","last_unstale":"2024-09-19T05:04:38.594472+0000","last_undegraded":"2024-09-19T05:04:38.594472+0000","last_fullsized":"2024-09-19T05:04:38.594472+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:11:43.843627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.245744+0000","last_change":"2024-09-19T05:02:24.422661+0000","last_active":"2024-09-19T05:04:38.245744+0000","last_peered":"2024-09-19T05:04:38.245744+0000","last_clean":"2024-09-19T05:04:38.245744+0000","last_became_active":"2024-09-19T05:02:24.422313+0000","last_became_peered":"2024-09-19T05:02:24.422313+0000","last_unstale":"2024-09-19T05:04:38.245744+0000","last_undegraded":"2024-09-19T05:04:38.245744+0000","last_fullsized":"2024-09-19T05:04:38.245744+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:07:39.729388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"86'8","reported_seq":159,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.684022+0000","last_change":"2024-09-19T05:02:22.028008+0000","last_active":"2024-09-19T05:04:38.684022+0000","last_peered":"2024-09-19T05:04:38.684022+0000","last_clean":"2024-09-19T05:04:38.684022+0000","last_became_active":"2024-09-19T05:02:22.027635+0000","last_became_peered":"2024-09-19T05:02:22.027635+0000","last_unstale":"2024-09-19T05:04:38.684022+0000","last_undegraded":"2024-09-19T05:04:38.684022+0000","last_fullsized":"2024-09-19T05:04:38.684022+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:41:36.902282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1515,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.356212+0000","last_change":"2024-09-19T05:02:24.435362+0000","last_active":"2024-09-19T05:04:38.356212+0000","last_peered":"2024-09-19T05:04:38.356212+0000","last_clean":"2024-09-19T05:04:38.356212+0000","last_became_active":"2024-09-19T05:02:24.435056+0000","last_became_peered":"2024-09-19T05:02:24.435056+0000","last_unstale":"2024-09-19T05:04:38.356212+0000","last_undegraded":"2024-09-19T05:04:38.356212+0000","last_fullsized":"2024-09-19T05:04:38.356212+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:58:40.092466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.340983+0000","last_change":"2024-09-19T05:02:22.077651+0000","last_active":"2024-09-19T05:04:38.340983+0000","last_peered":"2024-09-19T05:04:38.340983+0000","last_clean":"2024-09-19T05:04:38.340983+0000","last_became_active":"2024-09-19T05:02:22.076273+0000","last_became_peered":"2024-09-19T05:02:22.076273+0000","last_unstale":"2024-09-19T05:04:38.340983+0000","last_undegraded":"2024-09-19T05:04:38.340983+0000","last_fullsized":"2024-09-19T05:04:38.340983+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:28:42.064126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'5","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.356147+0000","last_change":"2024-09-19T05:02:24.438125+0000","last_active":"2024-09-19T05:04:38.356147+0000","last_peered":"2024-09-19T05:04:38.356147+0000","last_clean":"2024-09-19T05:04:38.356147+0000","last_became_active":"2024-09-19T05:02:24.437806+0000","last_became_peered":"2024-09-19T05:02:24.437806+0000","last_unstale":"2024-09-19T05:04:38.356147+0000","last_undegraded":"2024-09-19T05:04:38.356147+0000","last_fullsized":"2024-09-19T05:04:38.356147+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:46:08.338501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587986+0000","last_change":"2024-09-19T05:02:22.075920+0000","last_active":"2024-09-19T05:04:38.587986+0000","last_peered":"2024-09-19T05:04:38.587986+0000","last_clean":"2024-09-19T05:04:38.587986+0000","last_became_active":"2024-09-19T05:02:22.075609+0000","last_became_peered":"2024-09-19T05:02:22.075609+0000","last_unstale":"2024-09-19T05:04:38.587986+0000","last_undegraded":"2024-09-19T05:04:38.587986+0000","last_fullsized":"2024-09-19T05:04:38.587986+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:45.915625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.564327+0000","last_change":"2024-09-19T05:02:24.434681+0000","last_active":"2024-09-19T05:04:38.564327+0000","last_peered":"2024-09-19T05:04:38.564327+0000","last_clean":"2024-09-19T05:04:38.564327+0000","last_became_active":"2024-09-19T05:02:24.434257+0000","last_became_peered":"2024-09-19T05:02:24.434257+0000","last_unstale":"2024-09-19T05:04:38.564327+0000","last_undegraded":"2024-09-19T05:04:38.564327+0000","last_fullsized":"2024-09-19T05:04:38.564327+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:06:09.133112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'51","reported_seq":117,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.530942+0000","last_change":"2024-09-19T05:03:08.725679+0000","last_active":"2024-09-19T05:04:38.530942+0000","last_peered":"2024-09-19T05:04:38.530942+0000","last_clean":"2024-09-19T05:04:38.530942+0000","last_became_active":"2024-09-19T05:03:08.715651+0000","last_became_peered":"2024-09-19T05:03:08.715651+0000","last_unstale":"2024-09-19T05:04:38.530942+0000","last_undegraded":"2024-09-19T05:04:38.530942+0000","last_fullsized":"2024-09-19T05:04:38.530942+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:18:32.003686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":91,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.244565+0000","last_change":"2024-09-19T05:02:24.429807+0000","last_active":"2024-09-19T05:04:38.244565+0000","last_peered":"2024-09-19T05:04:38.244565+0000","last_clean":"2024-09-19T05:04:38.244565+0000","last_became_active":"2024-09-19T05:02:24.429477+0000","last_became_peered":"2024-09-19T05:02:24.429477+0000","last_unstale":"2024-09-19T05:04:38.244565+0000","last_undegraded":"2024-09-19T05:04:38.244565+0000","last_fullsized":"2024-09-19T05:04:38.244565+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:02.314851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.245566+0000","last_change":"2024-09-19T05:02:22.154534+0000","last_active":"2024-09-19T05:04:38.245566+0000","last_peered":"2024-09-19T05:04:38.245566+0000","last_clean":"2024-09-19T05:04:38.245566+0000","last_became_active":"2024-09-19T05:02:22.154238+0000","last_became_peered":"2024-09-19T05:02:22.154238+0000","last_unstale":"2024-09-19T05:04:38.245566+0000","last_undegraded":"2024-09-19T05:04:38.245566+0000","last_fullsized":"2024-09-19T05:04:38.245566+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:32:30.963503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.591417+0000","last_change":"2024-09-19T05:02:24.422817+0000","last_active":"2024-09-19T05:04:38.591417+0000","last_peered":"2024-09-19T05:04:38.591417+0000","last_clean":"2024-09-19T05:04:38.591417+0000","last_became_active":"2024-09-19T05:02:24.422478+0000","last_became_peered":"2024-09-19T05:02:24.422478+0000","last_unstale":"2024-09-19T05:04:38.591417+0000","last_undegraded":"2024-09-19T05:04:38.591417+0000","last_fullsized":"2024-09-19T05:04:38.591417+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:15:35.442149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.584586+0000","last_change":"2024-09-19T05:02:22.074396+0000","last_active":"2024-09-19T05:04:38.584586+0000","last_peered":"2024-09-19T05:04:38.584586+0000","last_clean":"2024-09-19T05:04:38.584586+0000","last_became_active":"2024-09-19T05:02:22.074108+0000","last_became_peered":"2024-09-19T05:02:22.074108+0000","last_unstale":"2024-09-19T05:04:38.584586+0000","last_undegraded":"2024-09-19T05:04:38.584586+0000","last_fullsized":"2024-09-19T05:04:38.584586+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:58:32.718702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587708+0000","last_change":"2024-09-19T05:02:24.429599+0000","last_active":"2024-09-19T05:04:38.587708+0000","last_peered":"2024-09-19T05:04:38.587708+0000","last_clean":"2024-09-19T05:04:38.587708+0000","last_became_active":"2024-09-19T05:02:24.429268+0000","last_became_peered":"2024-09-19T05:02:24.429268+0000","last_unstale":"2024-09-19T05:04:38.587708+0000","last_undegraded":"2024-09-19T05:04:38.587708+0000","last_fullsized":"2024-09-19T05:04:38.587708+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:57.123516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.244191+0000","last_change":"2024-09-19T05:02:22.021900+0000","last_active":"2024-09-19T05:04:38.244191+0000","last_peered":"2024-09-19T05:04:38.244191+0000","last_clean":"2024-09-19T05:04:38.244191+0000","last_became_active":"2024-09-19T05:02:22.020576+0000","last_became_peered":"2024-09-19T05:02:22.020576+0000","last_unstale":"2024-09-19T05:04:38.244191+0000","last_undegraded":"2024-09-19T05:04:38.244191+0000","last_fullsized":"2024-09-19T05:04:38.244191+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:44:14.466475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.343316+0000","last_change":"2024-09-19T05:02:24.420362+0000","last_active":"2024-09-19T05:04:38.343316+0000","last_peered":"2024-09-19T05:04:38.343316+0000","last_clean":"2024-09-19T05:04:38.343316+0000","last_became_active":"2024-09-19T05:02:24.420047+0000","last_became_peered":"2024-09-19T05:02:24.420047+0000","last_unstale":"2024-09-19T05:04:38.343316+0000","last_undegraded":"2024-09-19T05:04:38.343316+0000","last_fullsized":"2024-09-19T05:04:38.343316+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:28:56.977946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.455194+0000","last_change":"2024-09-19T05:02:22.024565+0000","last_active":"2024-09-19T05:04:38.455194+0000","last_peered":"2024-09-19T05:04:38.455194+0000","last_clean":"2024-09-19T05:04:38.455194+0000","last_became_active":"2024-09-19T05:02:22.024246+0000","last_became_peered":"2024-09-19T05:02:22.024246+0000","last_unstale":"2024-09-19T05:04:38.455194+0000","last_undegraded":"2024-09-19T05:04:38.455194+0000","last_fullsized":"2024-09-19T05:04:38.455194+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:58:27.491975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.244170+0000","last_change":"2024-09-19T05:02:24.428046+0000","last_active":"2024-09-19T05:04:38.244170+0000","last_peered":"2024-09-19T05:04:38.244170+0000","last_clean":"2024-09-19T05:04:38.244170+0000","last_became_active":"2024-09-19T05:02:24.427724+0000","last_became_peered":"2024-09-19T05:02:24.427724+0000","last_unstale":"2024-09-19T05:04:38.244170+0000","last_undegraded":"2024-09-19T05:04:38.244170+0000","last_fullsized":"2024-09-19T05:04:38.244170+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:11:54.252876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587778+0000","last_change":"2024-09-19T05:02:22.023054+0000","last_active":"2024-09-19T05:04:38.587778+0000","last_peered":"2024-09-19T05:04:38.587778+0000","last_clean":"2024-09-19T05:04:38.587778+0000","last_became_active":"2024-09-19T05:02:22.022700+0000","last_became_peered":"2024-09-19T05:02:22.022700+0000","last_unstale":"2024-09-19T05:04:38.587778+0000","last_undegraded":"2024-09-19T05:04:38.587778+0000","last_fullsized":"2024-09-19T05:04:38.587778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:44:19.714938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587808+0000","last_change":"2024-09-19T05:02:24.423045+0000","last_active":"2024-09-19T05:04:38.587808+0000","last_peered":"2024-09-19T05:04:38.587808+0000","last_clean":"2024-09-19T05:04:38.587808+0000","last_became_active":"2024-09-19T05:02:24.422719+0000","last_became_peered":"2024-09-19T05:02:24.422719+0000","last_unstale":"2024-09-19T05:04:38.587808+0000","last_undegraded":"2024-09-19T05:04:38.587808+0000","last_fullsized":"2024-09-19T05:04:38.587808+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:45.299603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.246181+0000","last_change":"2024-09-19T05:02:24.426942+0000","last_active":"2024-09-19T05:04:38.246181+0000","last_peered":"2024-09-19T05:04:38.246181+0000","last_clean":"2024-09-19T05:04:38.246181+0000","last_became_active":"2024-09-19T05:02:24.425896+0000","last_became_peered":"2024-09-19T05:02:24.425896+0000","last_unstale":"2024-09-19T05:04:38.246181+0000","last_undegraded":"2024-09-19T05:04:38.246181+0000","last_fullsized":"2024-09-19T05:04:38.246181+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:54:15.384309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587925+0000","last_change":"2024-09-19T05:02:22.021415+0000","last_active":"2024-09-19T05:04:38.587925+0000","last_peered":"2024-09-19T05:04:38.587925+0000","last_clean":"2024-09-19T05:04:38.587925+0000","last_became_active":"2024-09-19T05:02:22.020508+0000","last_became_peered":"2024-09-19T05:02:22.020508+0000","last_unstale":"2024-09-19T05:04:38.587925+0000","last_undegraded":"2024-09-19T05:04:38.587925+0000","last_fullsized":"2024-09-19T05:04:38.587925+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:35.912386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.584623+0000","last_change":"2024-09-19T05:02:24.435538+0000","last_active":"2024-09-19T05:04:38.584623+0000","last_peered":"2024-09-19T05:04:38.584623+0000","last_clean":"2024-09-19T05:04:38.584623+0000","last_became_active":"2024-09-19T05:02:24.435209+0000","last_became_peered":"2024-09-19T05:02:24.435209+0000","last_unstale":"2024-09-19T05:04:38.584623+0000","last_undegraded":"2024-09-19T05:04:38.584623+0000","last_fullsized":"2024-09-19T05:04:38.584623+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:55:30.075113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":61,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587897+0000","last_change":"2024-09-19T05:03:08.720605+0000","last_active":"2024-09-19T05:04:38.587897+0000","last_peered":"2024-09-19T05:04:38.587897+0000","last_clean":"2024-09-19T05:04:38.587897+0000","last_became_active":"2024-09-19T05:03:08.709207+0000","last_became_peered":"2024-09-19T05:03:08.709207+0000","last_unstale":"2024-09-19T05:04:38.587897+0000","last_undegraded":"2024-09-19T05:04:38.587897+0000","last_fullsized":"2024-09-19T05:04:38.587897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:52:26.691476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.591761+0000","last_change":"2024-09-19T05:02:24.428958+0000","last_active":"2024-09-19T05:04:38.591761+0000","last_peered":"2024-09-19T05:04:38.591761+0000","last_clean":"2024-09-19T05:04:38.591761+0000","last_became_active":"2024-09-19T05:02:24.428672+0000","last_became_peered":"2024-09-19T05:02:24.428672+0000","last_unstale":"2024-09-19T05:04:38.591761+0000","last_undegraded":"2024-09-19T05:04:38.591761+0000","last_fullsized":"2024-09-19T05:04:38.591761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:06:23.529706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.244328+0000","last_change":"2024-09-19T05:02:22.077034+0000","last_active":"2024-09-19T05:04:38.244328+0000","last_peered":"2024-09-19T05:04:38.244328+0000","last_clean":"2024-09-19T05:04:38.244328+0000","last_became_active":"2024-09-19T05:02:22.076182+0000","last_became_peered":"2024-09-19T05:02:22.076182+0000","last_unstale":"2024-09-19T05:04:38.244328+0000","last_undegraded":"2024-09-19T05:04:38.244328+0000","last_fullsized":"2024-09-19T05:04:38.244328+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:43:31.076464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.244089+0000","last_change":"2024-09-19T05:02:24.423765+0000","last_active":"2024-09-19T05:04:38.244089+0000","last_peered":"2024-09-19T05:04:38.244089+0000","last_clean":"2024-09-19T05:04:38.244089+0000","last_became_active":"2024-09-19T05:02:24.423445+0000","last_became_peered":"2024-09-19T05:02:24.423445+0000","last_unstale":"2024-09-19T05:04:38.244089+0000","last_undegraded":"2024-09-19T05:04:38.244089+0000","last_fullsized":"2024-09-19T05:04:38.244089+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:23.394098+0000","last_clean_scrub_stamp":"2024-09-19T05:02:23.394098+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:10:07.806697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"86'15","reported_seq":163,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-19T05:04:38.587759+0000","last_change":"2024-09-19T05:02:22.023413+0000","last_active":"2024-09-19T05:04:38.587759+0000","last_peered":"2024-09-19T05:04:38.587759+0000","last_clean":"2024-09-19T05:04:38.587759+0000","last_became_active":"2024-09-19T05:02:22.023078+0000","last_became_peered":"2024-09-19T05:02:22.023078+0000","last_unstale":"2024-09-19T05:04:38.587759+0000","last_undegraded":"2024-09-19T05:04:38.587759+0000","last_fullsized":"2024-09-19T05:04:38.587759+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:02:21.002524+0000","last_clean_scrub_stamp":"2024-09-19T05:02:21.002524+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:11:09.185011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":603100,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1339,"num_read_kb":792,"num_write":381,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":981,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724992,"data_stored":1809300,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":308,"ondisk_log_size":308,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108990,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514537,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":857351,"data_compressed":27833,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743470,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32292,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712860,"statfs":{"total":95995035648,"available":95961968640,"internally_reserved":0,"allocated":1130496,"data_stored":1253796,"data_compressed":51173,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906995,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":995328,"data_stored":854839,"data_compressed":27833,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135928,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":453045,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397566,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1015808,"data_stored":1055001,"data_compressed":45017,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561091,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":954368,"data_stored":462267,"data_compressed":8687,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790024,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1052672,"data_stored":1185940,"data_compressed":51173,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986253,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":954368,"data_stored":461735,"data_compressed":8687,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215187,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1015808,"data_stored":985981,"data_compressed":33989,"data_compressed_allocated":45056,"data_compressed_original":540672,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1044480,"data_stored":532481,"data_compressed":8687,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574943,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1011712,"data_stored":521592,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":962560,"data_stored":454257,"data_compressed":5791,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36330,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36330,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36330,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":134954,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":202810,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":396850,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T05:04:44.642 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:04:45.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:44 smithi043 ceph-mon[30800]: Updating smithi043:/var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:04:45.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:44 smithi043 ceph-mon[30800]: Updating smithi096:/etc/ceph/ceph.client.admin.keyring 2024-09-19T05:04:45.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:44 smithi043 ceph-mon[30800]: Updating smithi151:/etc/ceph/ceph.client.admin.keyring 2024-09-19T05:04:45.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:44 smithi043 ceph-mon[30800]: Updating smithi096:/var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T05:04:45.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:44 smithi043 ceph-mon[30800]: Updating smithi151:/var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T05:04:45.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:44 smithi043 ceph-mon[30800]: from='client.25210 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:04:45.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:44 smithi043 ceph-mon[30800]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-09-19T05:04:45.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:44 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:45.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:44 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:45.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:44 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:45.077 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:45.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:44 smithi151 ceph-mon[35425]: Updating smithi043:/var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:04:45.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:44 smithi151 ceph-mon[35425]: Updating smithi096:/etc/ceph/ceph.client.admin.keyring 2024-09-19T05:04:45.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:44 smithi151 ceph-mon[35425]: Updating smithi151:/etc/ceph/ceph.client.admin.keyring 2024-09-19T05:04:45.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:44 smithi151 ceph-mon[35425]: Updating smithi096:/var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T05:04:45.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:44 smithi151 ceph-mon[35425]: Updating smithi151:/var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T05:04:45.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:44 smithi151 ceph-mon[35425]: from='client.25210 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:04:45.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:44 smithi151 ceph-mon[35425]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-09-19T05:04:45.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:44 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:45.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:44 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:45.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:44 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:45.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:44 smithi096 ceph-mon[35085]: Updating smithi043:/var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T05:04:45.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:44 smithi096 ceph-mon[35085]: Updating smithi096:/etc/ceph/ceph.client.admin.keyring 2024-09-19T05:04:45.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:44 smithi096 ceph-mon[35085]: Updating smithi151:/etc/ceph/ceph.client.admin.keyring 2024-09-19T05:04:45.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:44 smithi096 ceph-mon[35085]: Updating smithi096:/var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T05:04:45.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:44 smithi096 ceph-mon[35085]: Updating smithi151:/var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T05:04:45.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:44 smithi096 ceph-mon[35085]: from='client.25210 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:04:45.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:44 smithi096 ceph-mon[35085]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-09-19T05:04:45.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:44 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:45.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:44 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:45.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:44 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:45.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.672+0000 7f5e10600640 1 -- 172.21.15.43:0/3667993650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e08103ff0 msgr2=0x7f5e081043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:45.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.672+0000 7f5e10600640 1 --2- 172.21.15.43:0/3667993650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e08103ff0 0x7f5e081043d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc009860 tx=0x7f5dfc02a340 comp rx=0 tx=0).stop 2024-09-19T05:04:45.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.672+0000 7f5e10600640 1 -- 172.21.15.43:0/3667993650 shutdown_connections 2024-09-19T05:04:45.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.673+0000 7f5e10600640 1 --2- 172.21.15.43:0/3667993650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e08109120 0x7f5e08110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:45.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.673+0000 7f5e10600640 1 --2- 172.21.15.43:0/3667993650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e081049a0 0x7f5e081089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:45.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.673+0000 7f5e10600640 1 --2- 172.21.15.43:0/3667993650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e08103ff0 0x7f5e081043d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:45.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.673+0000 7f5e10600640 1 -- 172.21.15.43:0/3667993650 >> 172.21.15.43:0/3667993650 conn(0x7f5e080ffa90 msgr2=0x7f5e08101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:45.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.673+0000 7f5e10600640 1 -- 172.21.15.43:0/3667993650 shutdown_connections 2024-09-19T05:04:45.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.673+0000 7f5e10600640 1 -- 172.21.15.43:0/3667993650 wait complete. 2024-09-19T05:04:45.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.674+0000 7f5e10600640 1 Processor -- start 2024-09-19T05:04:45.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.674+0000 7f5e10600640 1 -- start start 2024-09-19T05:04:45.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.674+0000 7f5e10600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e08103ff0 0x7f5e0819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:45.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.674+0000 7f5e10600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e081049a0 0x7f5e0819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:45.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.674+0000 7f5e10600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e08109120 0x7f5e0819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:45.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.674+0000 7f5e10600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5e08114120 con 0x7f5e08109120 2024-09-19T05:04:45.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.674+0000 7f5e10600640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5e08113fa0 con 0x7f5e08103ff0 2024-09-19T05:04:45.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.674+0000 7f5e10600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5e081142a0 con 0x7f5e081049a0 2024-09-19T05:04:45.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.675+0000 7f5e0e600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e08109120 0x7f5e0819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:45.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.675+0000 7f5e0e600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e08109120 0x7f5e0819eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60172/0 (socket says 172.21.15.43:60172) 2024-09-19T05:04:45.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.675+0000 7f5e0e600640 1 -- 172.21.15.43:0/4070425386 learned_addr learned my addr 172.21.15.43:0/4070425386 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:45.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.675+0000 7f5e0dc00640 1 --2- 172.21.15.43:0/4070425386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e08103ff0 0x7f5e0819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:45.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.675+0000 7f5e0d200640 1 --2- 172.21.15.43:0/4070425386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e081049a0 0x7f5e0819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:45.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.675+0000 7f5e0dc00640 1 -- 172.21.15.43:0/4070425386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e081049a0 msgr2=0x7f5e0819ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:45.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.675+0000 7f5e0dc00640 1 --2- 172.21.15.43:0/4070425386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e081049a0 0x7f5e0819ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:45.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.675+0000 7f5e0dc00640 1 -- 172.21.15.43:0/4070425386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e08109120 msgr2=0x7f5e0819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:45.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.675+0000 7f5e0dc00640 1 --2- 172.21.15.43:0/4070425386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e08109120 0x7f5e0819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:45.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.675+0000 7f5e0dc00640 1 -- 172.21.15.43:0/4070425386 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e0819f5d0 con 0x7f5e08103ff0 2024-09-19T05:04:45.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.675+0000 7f5e0e600640 1 --2- 172.21.15.43:0/4070425386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e08109120 0x7f5e0819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:04:45.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.675+0000 7f5e0dc00640 1 --2- 172.21.15.43:0/4070425386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e08103ff0 0x7f5e0819a620 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc02a860 tx=0x7f5dfc033b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:45.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.676+0000 7f5e06000640 1 -- 172.21.15.43:0/4070425386 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc037040 con 0x7f5e08103ff0 2024-09-19T05:04:45.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.676+0000 7f5e06000640 1 -- 172.21.15.43:0/4070425386 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dfc033d50 con 0x7f5e08103ff0 2024-09-19T05:04:45.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.676+0000 7f5e06000640 1 -- 172.21.15.43:0/4070425386 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc003960 con 0x7f5e08103ff0 2024-09-19T05:04:45.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.676+0000 7f5e10600640 1 -- 172.21.15.43:0/4070425386 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e0819f860 con 0x7f5e08103ff0 2024-09-19T05:04:45.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.676+0000 7f5e10600640 1 -- 172.21.15.43:0/4070425386 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e081a71a0 con 0x7f5e08103ff0 2024-09-19T05:04:45.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.677+0000 7f5e10600640 1 -- 172.21.15.43:0/4070425386 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e081054d0 con 0x7f5e08103ff0 2024-09-19T05:04:45.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.679+0000 7f5e06000640 1 -- 172.21.15.43:0/4070425386 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f5dfc042020 con 0x7f5e08103ff0 2024-09-19T05:04:45.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.680+0000 7f5e06000640 1 --2- 172.21.15.43:0/4070425386 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f5df00a34d0 0x7f5df00a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:45.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.681+0000 7f5e06000640 1 -- 172.21.15.43:0/4070425386 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f5dfc0ef9e0 con 0x7f5e08103ff0 2024-09-19T05:04:45.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.681+0000 7f5e0d200640 1 --2- 172.21.15.43:0/4070425386 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f5df00a34d0 0x7f5df00a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:45.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.681+0000 7f5e0d200640 1 --2- 172.21.15.43:0/4070425386 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f5df00a34d0 0x7f5df00a5990 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5df40096c0 tx=0x7f5df4009260 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:45.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.683+0000 7f5e06000640 1 -- 172.21.15.43:0/4070425386 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5dfc049020 con 0x7f5e08103ff0 2024-09-19T05:04:45.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.905+0000 7f5e10600640 1 -- 172.21.15.43:0/4070425386 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e081043d0 con 0x7f5e08103ff0 2024-09-19T05:04:45.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.907+0000 7f5e06000640 1 -- 172.21.15.43:0/4070425386 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11965 (secure 0 0 0) 0x7f5e081043d0 con 0x7f5e08103ff0 2024-09-19T05:04:45.908 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:04:45.908 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":29,"btime":"2024-09-19T05:04:38:785407+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/773801169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":773801169},{"type":"v1","addr":"172.21.15.43:6835","nonce":773801169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1799137902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1799137902},{"type":"v1","addr":"172.21.15.43:6837","nonce":1799137902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:02:25.463041+0000","modified":"2024-09-19T05:04:38.039809+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24518,"mds_2":24511,"mds_3":24506,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14541":{"gid":14541,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/794881135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":794881135},{"type":"v1","addr":"172.21.15.43:6841","nonce":794881135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/2836453158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2836453158},{"type":"v1","addr":"172.21.15.151:6833","nonce":2836453158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/178785812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":178785812},{"type":"v1","addr":"172.21.15.96:6833","nonce":178785812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1671043820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1671043820},{"type":"v1","addr":"172.21.15.96:6835","nonce":1671043820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2972514328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2972514328},{"type":"v1","addr":"172.21.15.151:6835","nonce":2972514328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.151:6837/3960378010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3960378010},{"type":"v1","addr":"172.21.15.151:6837","nonce":3960378010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6837/3978388286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3978388286},{"type":"v1","addr":"172.21.15.96:6837","nonce":3978388286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.151:6839/3794104313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3794104313},{"type":"v1","addr":"172.21.15.151:6839","nonce":3794104313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.43:6839/3404555030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3404555030},{"type":"v1","addr":"172.21.15.43:6839","nonce":3404555030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/605527621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":605527621},{"type":"v1","addr":"172.21.15.96:6839","nonce":605527621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24518,24511,24506,24493]},"id":1}]} 2024-09-19T05:04:45.909 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 29 2024-09-19T05:04:45.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.909+0000 7f5e10600640 1 -- 172.21.15.43:0/4070425386 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f5df00a34d0 msgr2=0x7f5df00a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:45.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.909+0000 7f5e10600640 1 --2- 172.21.15.43:0/4070425386 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f5df00a34d0 0x7f5df00a5990 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5df40096c0 tx=0x7f5df4009260 comp rx=0 tx=0).stop 2024-09-19T05:04:45.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.909+0000 7f5e10600640 1 -- 172.21.15.43:0/4070425386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e08103ff0 msgr2=0x7f5e0819a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:45.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.909+0000 7f5e10600640 1 --2- 172.21.15.43:0/4070425386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e08103ff0 0x7f5e0819a620 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc02a860 tx=0x7f5dfc033b60 comp rx=0 tx=0).stop 2024-09-19T05:04:45.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.910+0000 7f5e10600640 1 -- 172.21.15.43:0/4070425386 shutdown_connections 2024-09-19T05:04:45.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.910+0000 7f5e10600640 1 --2- 172.21.15.43:0/4070425386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e08109120 0x7f5e0819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:45.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.910+0000 7f5e10600640 1 --2- 172.21.15.43:0/4070425386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e081049a0 0x7f5e0819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:45.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.910+0000 7f5e10600640 1 --2- 172.21.15.43:0/4070425386 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f5df00a34d0 0x7f5df00a5990 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:45.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.910+0000 7f5e10600640 1 --2- 172.21.15.43:0/4070425386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e08103ff0 0x7f5e0819a620 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:45.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.910+0000 7f5e10600640 1 -- 172.21.15.43:0/4070425386 >> 172.21.15.43:0/4070425386 conn(0x7f5e080ffa90 msgr2=0x7f5e08100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:45.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.910+0000 7f5e10600640 1 -- 172.21.15.43:0/4070425386 shutdown_connections 2024-09-19T05:04:45.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:45.910+0000 7f5e10600640 1 -- 172.21.15.43:0/4070425386 wait complete. 2024-09-19T05:04:46.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:45 smithi151 ceph-mon[35425]: from='client.25216 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:04:46.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:45 smithi151 ceph-mon[35425]: Updating smithi043:/var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T05:04:46.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:45 smithi151 ceph-mon[35425]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:04:46.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:45 smithi151 ceph-mon[35425]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:04:46.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:45 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:04:46.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:45 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:04:46.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:45 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:04:46.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:45 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:46.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:45 smithi096 ceph-mon[35085]: from='client.25216 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:04:46.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:45 smithi096 ceph-mon[35085]: Updating smithi043:/var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T05:04:46.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:45 smithi096 ceph-mon[35085]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:04:46.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:45 smithi096 ceph-mon[35085]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:04:46.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:45 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:04:46.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:45 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:04:46.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:45 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:04:46.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:45 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:46.253 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:45 smithi043 ceph-mon[30800]: from='client.25216 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:04:46.253 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:45 smithi043 ceph-mon[30800]: Updating smithi043:/var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T05:04:46.253 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:45 smithi043 ceph-mon[30800]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:04:46.253 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:45 smithi043 ceph-mon[30800]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T05:04:46.253 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:45 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:04:46.253 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:45 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:04:46.254 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:45 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:04:46.254 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:45 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:46.673 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T05:04:47.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:46 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/4070425386' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:04:47.057 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:47.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:46 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/4070425386' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:04:47.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:46 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/4070425386' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:04:47.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.630+0000 7fb44ce00640 1 -- 172.21.15.43:0/3375589092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb448102790 msgr2=0x7fb4481067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:47.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.630+0000 7fb44ce00640 1 --2- 172.21.15.43:0/3375589092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb448102790 0x7fb4481067e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb430009900 tx=0x7fb43002a3f0 comp rx=0 tx=0).stop 2024-09-19T05:04:47.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.630+0000 7fb44ce00640 1 -- 172.21.15.43:0/3375589092 shutdown_connections 2024-09-19T05:04:47.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.630+0000 7fb44ce00640 1 --2- 172.21.15.43:0/3375589092 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb448106f10 0x7fb44810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:47.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.630+0000 7fb44ce00640 1 --2- 172.21.15.43:0/3375589092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb448102790 0x7fb4481067e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:47.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.630+0000 7fb44ce00640 1 --2- 172.21.15.43:0/3375589092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb448101de0 0x7fb4481021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:47.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.630+0000 7fb44ce00640 1 -- 172.21.15.43:0/3375589092 >> 172.21.15.43:0/3375589092 conn(0x7fb4480fd8c0 msgr2=0x7fb4480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:47.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.630+0000 7fb44ce00640 1 -- 172.21.15.43:0/3375589092 shutdown_connections 2024-09-19T05:04:47.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.630+0000 7fb44ce00640 1 -- 172.21.15.43:0/3375589092 wait complete. 2024-09-19T05:04:47.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.631+0000 7fb44ce00640 1 Processor -- start 2024-09-19T05:04:47.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.631+0000 7fb44ce00640 1 -- start start 2024-09-19T05:04:47.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.632+0000 7fb44ce00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb448101de0 0x7fb44819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:47.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.632+0000 7fb44ce00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb448102790 0x7fb44819ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:47.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.632+0000 7fb44ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb448106f10 0x7fb44819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:47.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.632+0000 7fb44ce00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb448111fc0 con 0x7fb448106f10 2024-09-19T05:04:47.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.632+0000 7fb44ce00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb448111e40 con 0x7fb448101de0 2024-09-19T05:04:47.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.632+0000 7fb44ce00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb448112140 con 0x7fb448102790 2024-09-19T05:04:47.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.632+0000 7fb445e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb448101de0 0x7fb44819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:47.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.632+0000 7fb446800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb448106f10 0x7fb44819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:47.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.632+0000 7fb445e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb448101de0 0x7fb44819a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:46778/0 (socket says 172.21.15.43:46778) 2024-09-19T05:04:47.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.632+0000 7fb445e00640 1 -- 172.21.15.43:0/2463252050 learned_addr learned my addr 172.21.15.43:0/2463252050 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:47.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.632+0000 7fb43d400640 1 --2- 172.21.15.43:0/2463252050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb448102790 0x7fb44819ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:47.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.632+0000 7fb446800640 1 -- 172.21.15.43:0/2463252050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb448102790 msgr2=0x7fb44819ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:47.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.632+0000 7fb446800640 1 --2- 172.21.15.43:0/2463252050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb448102790 0x7fb44819ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:47.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.632+0000 7fb446800640 1 -- 172.21.15.43:0/2463252050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb448101de0 msgr2=0x7fb44819a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:47.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.632+0000 7fb446800640 1 --2- 172.21.15.43:0/2463252050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb448101de0 0x7fb44819a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:47.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.632+0000 7fb446800640 1 -- 172.21.15.43:0/2463252050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb44819f680 con 0x7fb448106f10 2024-09-19T05:04:47.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.633+0000 7fb446800640 1 --2- 172.21.15.43:0/2463252050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb448106f10 0x7fb44819efa0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb440009fd0 tx=0x7fb44000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:47.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.633+0000 7fb43f400640 1 -- 172.21.15.43:0/2463252050 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44000ce20 con 0x7fb448106f10 2024-09-19T05:04:47.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.633+0000 7fb43f400640 1 -- 172.21.15.43:0/2463252050 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb440005020 con 0x7fb448106f10 2024-09-19T05:04:47.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.633+0000 7fb44ce00640 1 -- 172.21.15.43:0/2463252050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb44819f970 con 0x7fb448106f10 2024-09-19T05:04:47.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.633+0000 7fb43f400640 1 -- 172.21.15.43:0/2463252050 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb440010720 con 0x7fb448106f10 2024-09-19T05:04:47.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.633+0000 7fb44ce00640 1 -- 172.21.15.43:0/2463252050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4481a7250 con 0x7fb448106f10 2024-09-19T05:04:47.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.634+0000 7fb44ce00640 1 -- 172.21.15.43:0/2463252050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4481021c0 con 0x7fb448106f10 2024-09-19T05:04:47.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.636+0000 7fb43f400640 1 -- 172.21.15.43:0/2463252050 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7fb440005540 con 0x7fb448106f10 2024-09-19T05:04:47.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.638+0000 7fb43f400640 1 --2- 172.21.15.43:0/2463252050 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb42c0a3550 0x7fb42c0a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:47.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.638+0000 7fb43f400640 1 -- 172.21.15.43:0/2463252050 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fb4400d0510 con 0x7fb448106f10 2024-09-19T05:04:47.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.638+0000 7fb445e00640 1 --2- 172.21.15.43:0/2463252050 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb42c0a3550 0x7fb42c0a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:47.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.639+0000 7fb445e00640 1 --2- 172.21.15.43:0/2463252050 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb42c0a3550 0x7fb42c0a5a10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb438009c90 tx=0x7fb438005f90 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:47.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.642+0000 7fb43f400640 1 -- 172.21.15.43:0/2463252050 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb44009cc50 con 0x7fb448106f10 2024-09-19T05:04:47.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.828+0000 7fb44ce00640 1 -- 172.21.15.43:0/2463252050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb4481a74e0 con 0x7fb448106f10 2024-09-19T05:04:47.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.832+0000 7fb43f400640 1 -- 172.21.15.43:0/2463252050 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+23038 (secure 0 0 0) 0x7fb44009c550 con 0x7fb448106f10 2024-09-19T05:04:47.833 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:04:47.834 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":86,"fsid":"1456a810-7643-11ef-bceb-c7b262605968","created":"2024-09-19T04:54:14.706644+0000","modified":"2024-09-19T05:04:38.237868+0000","last_up_change":"2024-09-19T05:01:27.348740+0000","last_in_change":"2024-09-19T05:01:03.396477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:57:27.642705+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:02:20.685123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:02:23.393596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"81a317a8-7ec8-45c3-bc10-7da023283635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6803","nonce":825351486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6805","nonce":825351486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6809","nonce":825351486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6807","nonce":825351486}]},"public_addr":"172.21.15.43:6803/825351486","cluster_addr":"172.21.15.43:6805/825351486","heartbeat_back_addr":"172.21.15.43:6809/825351486","heartbeat_front_addr":"172.21.15.43:6807/825351486","state":["exists","up"]},{"osd":1,"uuid":"e96fcfac-4fff-4a36-b38b-aebf57842740","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6801","nonce":2140095130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6803","nonce":2140095130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6807","nonce":2140095130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6805","nonce":2140095130}]},"public_addr":"172.21.15.96:6801/2140095130","cluster_addr":"172.21.15.96:6803/2140095130","heartbeat_back_addr":"172.21.15.96:6807/2140095130","heartbeat_front_addr":"172.21.15.96:6805/2140095130","state":["exists","up"]},{"osd":2,"uuid":"b659869e-114a-4f56-8aac-a2ed602b4058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6801","nonce":2047147996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6803","nonce":2047147996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6807","nonce":2047147996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6805","nonce":2047147996}]},"public_addr":"172.21.15.151:6801/2047147996","cluster_addr":"172.21.15.151:6803/2047147996","heartbeat_back_addr":"172.21.15.151:6807/2047147996","heartbeat_front_addr":"172.21.15.151:6805/2047147996","state":["exists","up"]},{"osd":3,"uuid":"7be64b2c-e45c-493d-a610-4ce41196061f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6811","nonce":1734673762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6813","nonce":1734673762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6817","nonce":1734673762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6815","nonce":1734673762}]},"public_addr":"172.21.15.43:6811/1734673762","cluster_addr":"172.21.15.43:6813/1734673762","heartbeat_back_addr":"172.21.15.43:6817/1734673762","heartbeat_front_addr":"172.21.15.43:6815/1734673762","state":["exists","up"]},{"osd":4,"uuid":"196d5af1-a6e5-496c-8933-4e58892f4360","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6809","nonce":3184785362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6811","nonce":3184785362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6815","nonce":3184785362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6813","nonce":3184785362}]},"public_addr":"172.21.15.96:6809/3184785362","cluster_addr":"172.21.15.96:6811/3184785362","heartbeat_back_addr":"172.21.15.96:6815/3184785362","heartbeat_front_addr":"172.21.15.96:6813/3184785362","state":["exists","up"]},{"osd":5,"uuid":"5492d474-8509-4865-bc2e-d08e31558a75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6809","nonce":2878451924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6811","nonce":2878451924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6815","nonce":2878451924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6813","nonce":2878451924}]},"public_addr":"172.21.15.151:6809/2878451924","cluster_addr":"172.21.15.151:6811/2878451924","heartbeat_back_addr":"172.21.15.151:6815/2878451924","heartbeat_front_addr":"172.21.15.151:6813/2878451924","state":["exists","up"]},{"osd":6,"uuid":"fb8457ed-7bb9-4435-9d15-c1cbfbddfcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6819","nonce":2008630290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6821","nonce":2008630290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6825","nonce":2008630290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6823","nonce":2008630290}]},"public_addr":"172.21.15.43:6819/2008630290","cluster_addr":"172.21.15.43:6821/2008630290","heartbeat_back_addr":"172.21.15.43:6825/2008630290","heartbeat_front_addr":"172.21.15.43:6823/2008630290","state":["exists","up"]},{"osd":7,"uuid":"02046db8-30de-4e92-b25a-1f07d6190e91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6817","nonce":3588713275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6819","nonce":3588713275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6823","nonce":3588713275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6821","nonce":3588713275}]},"public_addr":"172.21.15.96:6817/3588713275","cluster_addr":"172.21.15.96:6819/3588713275","heartbeat_back_addr":"172.21.15.96:6823/3588713275","heartbeat_front_addr":"172.21.15.96:6821/3588713275","state":["exists","up"]},{"osd":8,"uuid":"a5bb4218-51f3-4044-b43e-ebfc764f49ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6817","nonce":725316393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6819","nonce":725316393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6823","nonce":725316393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6821","nonce":725316393}]},"public_addr":"172.21.15.151:6817/725316393","cluster_addr":"172.21.15.151:6819/725316393","heartbeat_back_addr":"172.21.15.151:6823/725316393","heartbeat_front_addr":"172.21.15.151:6821/725316393","state":["exists","up"]},{"osd":9,"uuid":"6c19f65c-9750-4baa-9d63-7b8bd03e6955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6827","nonce":4053438621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6829","nonce":4053438621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6833","nonce":4053438621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6831","nonce":4053438621}]},"public_addr":"172.21.15.43:6827/4053438621","cluster_addr":"172.21.15.43:6829/4053438621","heartbeat_back_addr":"172.21.15.43:6833/4053438621","heartbeat_front_addr":"172.21.15.43:6831/4053438621","state":["exists","up"]},{"osd":10,"uuid":"a7aa7da3-8eab-47fe-a351-298b6ff75f14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6825","nonce":2403210086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6827","nonce":2403210086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6831","nonce":2403210086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6829","nonce":2403210086}]},"public_addr":"172.21.15.96:6825/2403210086","cluster_addr":"172.21.15.96:6827/2403210086","heartbeat_back_addr":"172.21.15.96:6831/2403210086","heartbeat_front_addr":"172.21.15.96:6829/2403210086","state":["exists","up"]},{"osd":11,"uuid":"760b6afc-253d-44dd-88e9-dcc97f67c9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6825","nonce":292202881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6827","nonce":292202881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6831","nonce":292202881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6829","nonce":292202881}]},"public_addr":"172.21.15.151:6825/292202881","cluster_addr":"172.21.15.151:6827/292202881","heartbeat_back_addr":"172.21.15.151:6831/292202881","heartbeat_front_addr":"172.21.15.151:6829/292202881","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:56:34.716991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:56:59.321174+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:57:24.281696+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:57:53.443880+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:58:19.205873+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:58:44.504873+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:13.650731+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:39.137876+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:04.579893+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:34.337312+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:59.434658+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:01:24.930529+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/3819455723":"2024-09-20T05:04:38.237695+0000","172.21.15.43:0/3712369735":"2024-09-20T05:04:38.237695+0000","172.21.15.43:0/330854185":"2024-09-20T04:54:39.548065+0000","172.21.15.43:0/6132665":"2024-09-20T04:54:39.548065+0000","172.21.15.43:0/3730884138":"2024-09-20T05:04:38.237695+0000","172.21.15.43:0/632163478":"2024-09-20T04:54:39.548065+0000","172.21.15.43:0/4271788315":"2024-09-20T04:55:07.037117+0000","172.21.15.43:6800/587463881":"2024-09-20T04:54:39.548065+0000","172.21.15.43:6800/3241799377":"2024-09-20T04:55:07.037117+0000","172.21.15.43:6801/3241799377":"2024-09-20T04:55:07.037117+0000","172.21.15.43:0/898247657":"2024-09-20T04:55:07.037117+0000","172.21.15.43:6801/3167108925":"2024-09-20T05:04:38.237695+0000","172.21.15.43:0/515912755":"2024-09-20T05:04:38.237695+0000","172.21.15.43:6801/587463881":"2024-09-20T04:54:39.548065+0000","172.21.15.43:0/3256350596":"2024-09-20T04:55:07.037117+0000","172.21.15.43:6800/3167108925":"2024-09-20T05:04:38.237695+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:04:47.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.834+0000 7fb44ce00640 1 -- 172.21.15.43:0/2463252050 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb42c0a3550 msgr2=0x7fb42c0a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:47.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.834+0000 7fb44ce00640 1 --2- 172.21.15.43:0/2463252050 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb42c0a3550 0x7fb42c0a5a10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb438009c90 tx=0x7fb438005f90 comp rx=0 tx=0).stop 2024-09-19T05:04:47.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.834+0000 7fb44ce00640 1 -- 172.21.15.43:0/2463252050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb448106f10 msgr2=0x7fb44819efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:47.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.834+0000 7fb44ce00640 1 --2- 172.21.15.43:0/2463252050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb448106f10 0x7fb44819efa0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb440009fd0 tx=0x7fb44000ef10 comp rx=0 tx=0).stop 2024-09-19T05:04:47.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.834+0000 7fb44ce00640 1 -- 172.21.15.43:0/2463252050 shutdown_connections 2024-09-19T05:04:47.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.834+0000 7fb44ce00640 1 --2- 172.21.15.43:0/2463252050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb448106f10 0x7fb44819efa0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:47.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.834+0000 7fb44ce00640 1 --2- 172.21.15.43:0/2463252050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb448102790 0x7fb44819ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:47.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.834+0000 7fb44ce00640 1 --2- 172.21.15.43:0/2463252050 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb42c0a3550 0x7fb42c0a5a10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:47.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.834+0000 7fb44ce00640 1 --2- 172.21.15.43:0/2463252050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb448101de0 0x7fb44819a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:47.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.834+0000 7fb44ce00640 1 -- 172.21.15.43:0/2463252050 >> 172.21.15.43:0/2463252050 conn(0x7fb4480fd8c0 msgr2=0x7fb4480ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:47.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.835+0000 7fb44ce00640 1 -- 172.21.15.43:0/2463252050 shutdown_connections 2024-09-19T05:04:47.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:47.835+0000 7fb44ce00640 1 -- 172.21.15.43:0/2463252050 wait complete. 2024-09-19T05:04:48.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:47 smithi151 ceph-mon[35425]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 1.4 KiB/s wr, 26 op/s 2024-09-19T05:04:48.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:47 smithi096 ceph-mon[35085]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 1.4 KiB/s wr, 26 op/s 2024-09-19T05:04:48.236 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:47 smithi043 ceph-mon[30800]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 1.4 KiB/s wr, 26 op/s 2024-09-19T05:04:48.596 DEBUG:tasks.fwd_scrub:Yielding 2024-09-19T05:04:48.596 INFO:teuthology.run_tasks:Running task exec... 2024-09-19T05:04:48.605 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T05:04:48.606 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi043.front.sepia.ceph.com 2024-09-19T05:04:48.606 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-09-19T05:04:48.606 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-19T05:04:48.606 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T05:04:48.606 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T05:04:48.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.766+0000 7f3f51800640 1 -- 172.21.15.43:0/1007311305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f4c103f50 msgr2=0x7f3f4c104330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:48.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.766+0000 7f3f51800640 1 --2- 172.21.15.43:0/1007311305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f4c103f50 0x7f3f4c104330 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c009900 tx=0x7f3f3c02a410 comp rx=0 tx=0).stop 2024-09-19T05:04:48.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.766+0000 7f3f51800640 1 -- 172.21.15.43:0/1007311305 shutdown_connections 2024-09-19T05:04:48.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.766+0000 7f3f51800640 1 --2- 172.21.15.43:0/1007311305 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f4c1090a0 0x7f3f4c110bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:48.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.766+0000 7f3f51800640 1 --2- 172.21.15.43:0/1007311305 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f4c104900 0x7f3f4c108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:48.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.766+0000 7f3f51800640 1 --2- 172.21.15.43:0/1007311305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f4c103f50 0x7f3f4c104330 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:48.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.766+0000 7f3f51800640 1 -- 172.21.15.43:0/1007311305 >> 172.21.15.43:0/1007311305 conn(0x7f3f4c0ff9e0 msgr2=0x7f3f4c101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:48.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.766+0000 7f3f51800640 1 -- 172.21.15.43:0/1007311305 shutdown_connections 2024-09-19T05:04:48.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.766+0000 7f3f51800640 1 -- 172.21.15.43:0/1007311305 wait complete. 2024-09-19T05:04:48.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.767+0000 7f3f51800640 1 Processor -- start 2024-09-19T05:04:48.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.767+0000 7f3f51800640 1 -- start start 2024-09-19T05:04:48.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.767+0000 7f3f51800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f4c103f50 0x7f3f4c19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:48.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.767+0000 7f3f51800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f4c1090a0 0x7f3f4c19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:48.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.767+0000 7f3f51800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f4c19eea0 0x7f3f4c19f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:48.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.767+0000 7f3f51800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4c114130 con 0x7f3f4c19eea0 2024-09-19T05:04:48.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.767+0000 7f3f51800640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4c113fb0 con 0x7f3f4c1090a0 2024-09-19T05:04:48.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.767+0000 7f3f51800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4c1142b0 con 0x7f3f4c103f50 2024-09-19T05:04:48.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.768+0000 7f3f4b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f4c19eea0 0x7f3f4c19f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:48.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.768+0000 7f3f4b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f4c19eea0 0x7f3f4c19f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38730/0 (socket says 172.21.15.43:38730) 2024-09-19T05:04:48.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.768+0000 7f3f4b400640 1 -- 172.21.15.43:0/879650239 learned_addr learned my addr 172.21.15.43:0/879650239 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:48.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.768+0000 7f3f4aa00640 1 --2- 172.21.15.43:0/879650239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f4c103f50 0x7f3f4c19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:48.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.768+0000 7f3f4a000640 1 --2- 172.21.15.43:0/879650239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f4c1090a0 0x7f3f4c19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:48.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.768+0000 7f3f4b400640 1 -- 172.21.15.43:0/879650239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f4c103f50 msgr2=0x7f3f4c19a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:48.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.768+0000 7f3f4b400640 1 --2- 172.21.15.43:0/879650239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f4c103f50 0x7f3f4c19a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:48.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.768+0000 7f3f4b400640 1 -- 172.21.15.43:0/879650239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f4c1090a0 msgr2=0x7f3f4c19ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:48.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.768+0000 7f3f4b400640 1 --2- 172.21.15.43:0/879650239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f4c1090a0 0x7f3f4c19ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:48.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.768+0000 7f3f4b400640 1 -- 172.21.15.43:0/879650239 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f4c19f990 con 0x7f3f4c19eea0 2024-09-19T05:04:48.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.769+0000 7f3f4b400640 1 --2- 172.21.15.43:0/879650239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f4c19eea0 0x7f3f4c19f350 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3f4400eb10 tx=0x7f3f4400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:48.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.769+0000 7f3f43400640 1 -- 172.21.15.43:0/879650239 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f44004b20 con 0x7f3f4c19eea0 2024-09-19T05:04:48.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.769+0000 7f3f43400640 1 -- 172.21.15.43:0/879650239 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f44004cc0 con 0x7f3f4c19eea0 2024-09-19T05:04:48.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.769+0000 7f3f51800640 1 -- 172.21.15.43:0/879650239 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f4c19fc80 con 0x7f3f4c19eea0 2024-09-19T05:04:48.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.769+0000 7f3f43400640 1 -- 172.21.15.43:0/879650239 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f44010760 con 0x7f3f4c19eea0 2024-09-19T05:04:48.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.769+0000 7f3f51800640 1 -- 172.21.15.43:0/879650239 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f4c1a7420 con 0x7f3f4c19eea0 2024-09-19T05:04:48.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.770+0000 7f3f51800640 1 -- 172.21.15.43:0/879650239 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f4c104330 con 0x7f3f4c19eea0 2024-09-19T05:04:48.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.772+0000 7f3f43400640 1 -- 172.21.15.43:0/879650239 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f3f44005000 con 0x7f3f4c19eea0 2024-09-19T05:04:48.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.773+0000 7f3f43400640 1 --2- 172.21.15.43:0/879650239 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f3f300a3580 0x7f3f300a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:48.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.774+0000 7f3f43400640 1 -- 172.21.15.43:0/879650239 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f3f440d0730 con 0x7f3f4c19eea0 2024-09-19T05:04:48.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.774+0000 7f3f4aa00640 1 --2- 172.21.15.43:0/879650239 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f3f300a3580 0x7f3f300a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:48.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.775+0000 7f3f4aa00640 1 --2- 172.21.15.43:0/879650239 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f3f300a3580 0x7f3f300a5a40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c000c00 tx=0x7f3f3c005b50 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:48.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.776+0000 7f3f43400640 1 -- 172.21.15.43:0/879650239 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3f44054210 con 0x7f3f4c19eea0 2024-09-19T05:04:48.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.985+0000 7f3f51800640 1 -- 172.21.15.43:0/879650239 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f3f4c19b930 con 0x7f3f4c19eea0 2024-09-19T05:04:48.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.990+0000 7f3f43400640 1 -- 172.21.15.43:0/879650239 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v30)=0 v30) ==== 129+0+0 (secure 0 0 0) 0x7f3f4c19b930 con 0x7f3f4c19eea0 2024-09-19T05:04:48.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.992+0000 7f3f51800640 1 -- 172.21.15.43:0/879650239 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f3f300a3580 msgr2=0x7f3f300a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:48.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.992+0000 7f3f51800640 1 --2- 172.21.15.43:0/879650239 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f3f300a3580 0x7f3f300a5a40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c000c00 tx=0x7f3f3c005b50 comp rx=0 tx=0).stop 2024-09-19T05:04:48.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.992+0000 7f3f51800640 1 -- 172.21.15.43:0/879650239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f4c19eea0 msgr2=0x7f3f4c19f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:48.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.992+0000 7f3f51800640 1 --2- 172.21.15.43:0/879650239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f4c19eea0 0x7f3f4c19f350 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3f4400eb10 tx=0x7f3f4400c2d0 comp rx=0 tx=0).stop 2024-09-19T05:04:48.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.993+0000 7f3f51800640 1 -- 172.21.15.43:0/879650239 shutdown_connections 2024-09-19T05:04:48.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.993+0000 7f3f51800640 1 --2- 172.21.15.43:0/879650239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f4c19eea0 0x7f3f4c19f350 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:48.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.993+0000 7f3f51800640 1 --2- 172.21.15.43:0/879650239 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f3f300a3580 0x7f3f300a5a40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:48.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.993+0000 7f3f51800640 1 --2- 172.21.15.43:0/879650239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f4c1090a0 0x7f3f4c19ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:48.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.993+0000 7f3f51800640 1 --2- 172.21.15.43:0/879650239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f4c103f50 0x7f3f4c19a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:48.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.993+0000 7f3f51800640 1 -- 172.21.15.43:0/879650239 >> 172.21.15.43:0/879650239 conn(0x7f3f4c0ff9e0 msgr2=0x7f3f4c100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:48.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.993+0000 7f3f51800640 1 -- 172.21.15.43:0/879650239 shutdown_connections 2024-09-19T05:04:48.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:48.993+0000 7f3f51800640 1 -- 172.21.15.43:0/879650239 wait complete. 2024-09-19T05:04:49.011 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:49.013 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-09-19T05:04:49.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:48 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/2463252050' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:04:49.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:48 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/2463252050' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:04:49.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.176+0000 7f4c62e00640 1 -- 172.21.15.43:0/2498495106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c5c107ad0 msgr2=0x7f4c5c107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:49.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.176+0000 7f4c62e00640 1 --2- 172.21.15.43:0/2498495106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c5c107ad0 0x7f4c5c107eb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009930 tx=0x7f4c4c02a420 comp rx=0 tx=0).stop 2024-09-19T05:04:49.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.176+0000 7f4c62e00640 1 -- 172.21.15.43:0/2498495106 shutdown_connections 2024-09-19T05:04:49.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.176+0000 7f4c62e00640 1 --2- 172.21.15.43:0/2498495106 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c5c102460 0x7f4c5c10e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.176+0000 7f4c62e00640 1 --2- 172.21.15.43:0/2498495106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c5c101ac0 0x7f4c5c101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.176+0000 7f4c62e00640 1 --2- 172.21.15.43:0/2498495106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c5c107ad0 0x7f4c5c107eb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.176+0000 7f4c62e00640 1 -- 172.21.15.43:0/2498495106 >> 172.21.15.43:0/2498495106 conn(0x7f4c5c0fd7e0 msgr2=0x7f4c5c0ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:49.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.176+0000 7f4c62e00640 1 -- 172.21.15.43:0/2498495106 shutdown_connections 2024-09-19T05:04:49.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.176+0000 7f4c62e00640 1 -- 172.21.15.43:0/2498495106 wait complete. 2024-09-19T05:04:49.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.177+0000 7f4c62e00640 1 Processor -- start 2024-09-19T05:04:49.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.177+0000 7f4c62e00640 1 -- start start 2024-09-19T05:04:49.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.178+0000 7f4c62e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c5c101ac0 0x7f4c5c19a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:49.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.178+0000 7f4c62e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c5c102460 0x7f4c5c19aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:49.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.178+0000 7f4c62e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c5c107ad0 0x7f4c5c19ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:49.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.178+0000 7f4c62e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5c111f60 con 0x7f4c5c102460 2024-09-19T05:04:49.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.178+0000 7f4c62e00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5c111de0 con 0x7f4c5c101ac0 2024-09-19T05:04:49.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.178+0000 7f4c62e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5c1120e0 con 0x7f4c5c107ad0 2024-09-19T05:04:49.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.178+0000 7f4c5be00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c5c101ac0 0x7f4c5c19a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:49.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.178+0000 7f4c60e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c5c107ad0 0x7f4c5c19ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:49.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.178+0000 7f4c5be00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c5c101ac0 0x7f4c5c19a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:46816/0 (socket says 172.21.15.43:46816) 2024-09-19T05:04:49.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.178+0000 7f4c5be00640 1 -- 172.21.15.43:0/2181393445 learned_addr learned my addr 172.21.15.43:0/2181393445 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:49.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.178+0000 7f4c5b400640 1 --2- 172.21.15.43:0/2181393445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c5c102460 0x7f4c5c19aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:49.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.178+0000 7f4c5be00640 1 -- 172.21.15.43:0/2181393445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c5c107ad0 msgr2=0x7f4c5c19ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:49.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.178+0000 7f4c5be00640 1 --2- 172.21.15.43:0/2181393445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c5c107ad0 0x7f4c5c19ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.178+0000 7f4c5be00640 1 -- 172.21.15.43:0/2181393445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c5c102460 msgr2=0x7f4c5c19aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:49.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.178+0000 7f4c5be00640 1 --2- 172.21.15.43:0/2181393445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c5c102460 0x7f4c5c19aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.179+0000 7f4c5be00640 1 -- 172.21.15.43:0/2181393445 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c5c19f570 con 0x7f4c5c101ac0 2024-09-19T05:04:49.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.179+0000 7f4c5be00640 1 --2- 172.21.15.43:0/2181393445 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c5c101ac0 0x7f4c5c19a580 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c02a940 tx=0x7f4c4c003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:49.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.179+0000 7f4c58c00640 1 -- 172.21.15.43:0/2181393445 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c03f070 con 0x7f4c5c101ac0 2024-09-19T05:04:49.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.179+0000 7f4c62e00640 1 -- 172.21.15.43:0/2181393445 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c5c19f800 con 0x7f4c5c101ac0 2024-09-19T05:04:49.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.179+0000 7f4c58c00640 1 -- 172.21.15.43:0/2181393445 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c4c031c90 con 0x7f4c5c101ac0 2024-09-19T05:04:49.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.179+0000 7f4c58c00640 1 -- 172.21.15.43:0/2181393445 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c035030 con 0x7f4c5c101ac0 2024-09-19T05:04:49.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.179+0000 7f4c62e00640 1 -- 172.21.15.43:0/2181393445 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c5c1a70f0 con 0x7f4c5c101ac0 2024-09-19T05:04:49.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.180+0000 7f4c62e00640 1 -- 172.21.15.43:0/2181393445 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c5c103200 con 0x7f4c5c101ac0 2024-09-19T05:04:49.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.182+0000 7f4c58c00640 1 -- 172.21.15.43:0/2181393445 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f4c4c042070 con 0x7f4c5c101ac0 2024-09-19T05:04:49.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.184+0000 7f4c58c00640 1 --2- 172.21.15.43:0/2181393445 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f4c400a3580 0x7f4c400a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:49.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.184+0000 7f4c58c00640 1 -- 172.21.15.43:0/2181393445 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f4c4c0f0cb0 con 0x7f4c5c101ac0 2024-09-19T05:04:49.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.184+0000 7f4c5b400640 1 --2- 172.21.15.43:0/2181393445 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f4c400a3580 0x7f4c400a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:49.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.185+0000 7f4c5b400640 1 --2- 172.21.15.43:0/2181393445 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f4c400a3580 0x7f4c400a5a40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c10ee20 tx=0x7f4c48005f90 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:49.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.186+0000 7f4c58c00640 1 -- 172.21.15.43:0/2181393445 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4c4c0bd4c0 con 0x7f4c5c101ac0 2024-09-19T05:04:49.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:48 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/2463252050' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:04:49.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.395+0000 7f4c62e00640 1 -- 172.21.15.43:0/2181393445 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f4c5c101f20 con 0x7f4c5c101ac0 2024-09-19T05:04:49.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.402+0000 7f4c58c00640 1 -- 172.21.15.43:0/2181393445 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v31)=0 v31) ==== 138+0+0 (secure 0 0 0) 0x7f4c5c101f20 con 0x7f4c5c101ac0 2024-09-19T05:04:49.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.405+0000 7f4c62e00640 1 -- 172.21.15.43:0/2181393445 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f4c400a3580 msgr2=0x7f4c400a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:49.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.405+0000 7f4c62e00640 1 --2- 172.21.15.43:0/2181393445 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f4c400a3580 0x7f4c400a5a40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c10ee20 tx=0x7f4c48005f90 comp rx=0 tx=0).stop 2024-09-19T05:04:49.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.405+0000 7f4c62e00640 1 -- 172.21.15.43:0/2181393445 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c5c101ac0 msgr2=0x7f4c5c19a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:49.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.405+0000 7f4c62e00640 1 --2- 172.21.15.43:0/2181393445 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c5c101ac0 0x7f4c5c19a580 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c02a940 tx=0x7f4c4c003b20 comp rx=0 tx=0).stop 2024-09-19T05:04:49.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.405+0000 7f4c62e00640 1 -- 172.21.15.43:0/2181393445 shutdown_connections 2024-09-19T05:04:49.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.405+0000 7f4c62e00640 1 --2- 172.21.15.43:0/2181393445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c5c107ad0 0x7f4c5c19ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.405+0000 7f4c62e00640 1 --2- 172.21.15.43:0/2181393445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c5c102460 0x7f4c5c19aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.405+0000 7f4c62e00640 1 --2- 172.21.15.43:0/2181393445 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f4c400a3580 0x7f4c400a5a40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.405+0000 7f4c62e00640 1 --2- 172.21.15.43:0/2181393445 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c5c101ac0 0x7f4c5c19a580 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.405+0000 7f4c62e00640 1 -- 172.21.15.43:0/2181393445 >> 172.21.15.43:0/2181393445 conn(0x7f4c5c0fd7e0 msgr2=0x7f4c5c0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:49.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.405+0000 7f4c62e00640 1 -- 172.21.15.43:0/2181393445 shutdown_connections 2024-09-19T05:04:49.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.405+0000 7f4c62e00640 1 -- 172.21.15.43:0/2181393445 wait complete. 2024-09-19T05:04:49.426 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-09-19T05:04:49.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.588+0000 7fc6ad800640 1 -- 172.21.15.43:0/823444907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6a8101ae0 msgr2=0x7fc6a8101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:49.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.588+0000 7fc6ad800640 1 --2- 172.21.15.43:0/823444907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6a8101ae0 0x7fc6a8101f40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc694009930 tx=0x7fc69402a420 comp rx=0 tx=0).stop 2024-09-19T05:04:49.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.588+0000 7fc6ad800640 1 -- 172.21.15.43:0/823444907 shutdown_connections 2024-09-19T05:04:49.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.588+0000 7fc6ad800640 1 --2- 172.21.15.43:0/823444907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc6a8102480 0x7fc6a810e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.588+0000 7fc6ad800640 1 --2- 172.21.15.43:0/823444907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6a8101ae0 0x7fc6a8101f40 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.588+0000 7fc6ad800640 1 --2- 172.21.15.43:0/823444907 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6a8107b80 0x7fc6a8107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.588+0000 7fc6ad800640 1 -- 172.21.15.43:0/823444907 >> 172.21.15.43:0/823444907 conn(0x7fc6a80fd7e0 msgr2=0x7fc6a80ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:49.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.588+0000 7fc6ad800640 1 -- 172.21.15.43:0/823444907 shutdown_connections 2024-09-19T05:04:49.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.588+0000 7fc6ad800640 1 -- 172.21.15.43:0/823444907 wait complete. 2024-09-19T05:04:49.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.589+0000 7fc6ad800640 1 Processor -- start 2024-09-19T05:04:49.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.589+0000 7fc6ad800640 1 -- start start 2024-09-19T05:04:49.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.590+0000 7fc6ad800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc6a8101ae0 0x7fc6a80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:49.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.590+0000 7fc6ad800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6a8102480 0x7fc6a8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:49.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.590+0000 7fc6ad800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6a8107b80 0x7fc6a8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:49.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.590+0000 7fc6ad800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a8111f80 con 0x7fc6a8102480 2024-09-19T05:04:49.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.590+0000 7fc6ad800640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a8111e00 con 0x7fc6a8101ae0 2024-09-19T05:04:49.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.590+0000 7fc6ad800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a8112100 con 0x7fc6a8107b80 2024-09-19T05:04:49.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.590+0000 7fc6a6000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6a8102480 0x7fc6a8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:49.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.590+0000 7fc6a6000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6a8102480 0x7fc6a8074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38758/0 (socket says 172.21.15.43:38758) 2024-09-19T05:04:49.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.590+0000 7fc6a6000640 1 -- 172.21.15.43:0/1567581976 learned_addr learned my addr 172.21.15.43:0/1567581976 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:49.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.590+0000 7fc6a7400640 1 --2- 172.21.15.43:0/1567581976 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6a8107b80 0x7fc6a8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:49.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.590+0000 7fc6a6a00640 1 --2- 172.21.15.43:0/1567581976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc6a8101ae0 0x7fc6a80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:49.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.590+0000 7fc6a6000640 1 -- 172.21.15.43:0/1567581976 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6a8107b80 msgr2=0x7fc6a8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:49.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.590+0000 7fc6a6000640 1 --2- 172.21.15.43:0/1567581976 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6a8107b80 0x7fc6a8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.590+0000 7fc6a6000640 1 -- 172.21.15.43:0/1567581976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc6a8101ae0 msgr2=0x7fc6a80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:49.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.590+0000 7fc6a6000640 1 --2- 172.21.15.43:0/1567581976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc6a8101ae0 0x7fc6a80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.590+0000 7fc6a6000640 1 -- 172.21.15.43:0/1567581976 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6a81a67f0 con 0x7fc6a8102480 2024-09-19T05:04:49.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.590+0000 7fc6a6a00640 1 --2- 172.21.15.43:0/1567581976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc6a8101ae0 0x7fc6a80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:04:49.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.591+0000 7fc6a6000640 1 --2- 172.21.15.43:0/1567581976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6a8102480 0x7fc6a8074a30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc69402a940 tx=0x7fc694003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:49.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.591+0000 7fc69f400640 1 -- 172.21.15.43:0/1567581976 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69403f070 con 0x7fc6a8102480 2024-09-19T05:04:49.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.591+0000 7fc6ad800640 1 -- 172.21.15.43:0/1567581976 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6a81a6a80 con 0x7fc6a8102480 2024-09-19T05:04:49.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.591+0000 7fc69f400640 1 -- 172.21.15.43:0/1567581976 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc694031c90 con 0x7fc6a8102480 2024-09-19T05:04:49.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.591+0000 7fc69f400640 1 -- 172.21.15.43:0/1567581976 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc694035030 con 0x7fc6a8102480 2024-09-19T05:04:49.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.591+0000 7fc6ad800640 1 -- 172.21.15.43:0/1567581976 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6a81a6fc0 con 0x7fc6a8102480 2024-09-19T05:04:49.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.593+0000 7fc6ad800640 1 -- 172.21.15.43:0/1567581976 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6a8101f40 con 0x7fc6a8102480 2024-09-19T05:04:49.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.595+0000 7fc69f400640 1 -- 172.21.15.43:0/1567581976 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7fc694042070 con 0x7fc6a8102480 2024-09-19T05:04:49.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.596+0000 7fc69f400640 1 --2- 172.21.15.43:0/1567581976 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fc68c0a3580 0x7fc68c0a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:49.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.596+0000 7fc69f400640 1 -- 172.21.15.43:0/1567581976 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fc6940f0d70 con 0x7fc6a8102480 2024-09-19T05:04:49.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.596+0000 7fc6a6a00640 1 --2- 172.21.15.43:0/1567581976 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fc68c0a3580 0x7fc68c0a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:49.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.597+0000 7fc6a6a00640 1 --2- 172.21.15.43:0/1567581976 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fc68c0a3580 0x7fc68c0a5a40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc698009ba0 tx=0x7fc698005f90 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:49.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.598+0000 7fc69f400640 1 -- 172.21.15.43:0/1567581976 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc6940bd5e0 con 0x7fc6a8102480 2024-09-19T05:04:49.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.621+0000 7f724f400640 1 -- 172.21.15.43:0/2345888426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7248101df0 msgr2=0x7f72481021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:49.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.621+0000 7f724f400640 1 --2- 172.21.15.43:0/2345888426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7248101df0 0x7f72481021d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f723c009900 tx=0x7f723c02a3f0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.621+0000 7f724f400640 1 -- 172.21.15.43:0/2345888426 shutdown_connections 2024-09-19T05:04:49.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.621+0000 7f724f400640 1 --2- 172.21.15.43:0/2345888426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7248106f20 0x7f724810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.621+0000 7f724f400640 1 --2- 172.21.15.43:0/2345888426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f72481027a0 0x7f72481067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.621+0000 7f724f400640 1 --2- 172.21.15.43:0/2345888426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7248101df0 0x7f72481021d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.621+0000 7f724f400640 1 -- 172.21.15.43:0/2345888426 >> 172.21.15.43:0/2345888426 conn(0x7f72480fd8b0 msgr2=0x7f72480ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:49.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.621+0000 7f724f400640 1 -- 172.21.15.43:0/2345888426 shutdown_connections 2024-09-19T05:04:49.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.621+0000 7f724f400640 1 -- 172.21.15.43:0/2345888426 wait complete. 2024-09-19T05:04:49.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.622+0000 7f724f400640 1 Processor -- start 2024-09-19T05:04:49.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.622+0000 7f724f400640 1 -- start start 2024-09-19T05:04:49.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.623+0000 7f724f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7248101df0 0x7f724819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:49.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.623+0000 7f724f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72481027a0 0x7f724819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:49.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.623+0000 7f724f400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7248106f20 0x7f724819eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:49.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.623+0000 7f724f400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7248111f80 con 0x7f72481027a0 2024-09-19T05:04:49.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.623+0000 7f724f400640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7248111e00 con 0x7f7248106f20 2024-09-19T05:04:49.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.623+0000 7f724f400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7248112100 con 0x7f7248101df0 2024-09-19T05:04:49.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.623+0000 7f724ca00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7248101df0 0x7f724819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:49.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.623+0000 7f724ca00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7248101df0 0x7f724819a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:36178/0 (socket says 172.21.15.43:36178) 2024-09-19T05:04:49.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.623+0000 7f724ca00640 1 -- 172.21.15.43:0/3494491756 learned_addr learned my addr 172.21.15.43:0/3494491756 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:49.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.623+0000 7f7247e00640 1 --2- 172.21.15.43:0/3494491756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72481027a0 0x7f724819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:49.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.623+0000 7f724d400640 1 --2- 172.21.15.43:0/3494491756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7248106f20 0x7f724819eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:49.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.624+0000 7f724ca00640 1 -- 172.21.15.43:0/3494491756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7248106f20 msgr2=0x7f724819eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:49.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.624+0000 7f724ca00640 1 --2- 172.21.15.43:0/3494491756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7248106f20 0x7f724819eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.624+0000 7f724ca00640 1 -- 172.21.15.43:0/3494491756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72481027a0 msgr2=0x7f724819ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:49.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.624+0000 7f724ca00640 1 --2- 172.21.15.43:0/3494491756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72481027a0 0x7f724819ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.624+0000 7f724ca00640 1 -- 172.21.15.43:0/3494491756 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f724819f580 con 0x7f7248101df0 2024-09-19T05:04:49.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.624+0000 7f724d400640 1 --2- 172.21.15.43:0/3494491756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7248106f20 0x7f724819eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:04:49.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.624+0000 7f724ca00640 1 --2- 172.21.15.43:0/3494491756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7248101df0 0x7f724819a5d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f723c006010 tx=0x7f723c004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:49.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.624+0000 7f7245600640 1 -- 172.21.15.43:0/3494491756 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f723c03f030 con 0x7f7248101df0 2024-09-19T05:04:49.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.624+0000 7f724f400640 1 -- 172.21.15.43:0/3494491756 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f724819f810 con 0x7f7248101df0 2024-09-19T05:04:49.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.624+0000 7f7245600640 1 -- 172.21.15.43:0/3494491756 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f723c004440 con 0x7f7248101df0 2024-09-19T05:04:49.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.625+0000 7f724f400640 1 -- 172.21.15.43:0/3494491756 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72481a7150 con 0x7f7248101df0 2024-09-19T05:04:49.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.625+0000 7f7245600640 1 -- 172.21.15.43:0/3494491756 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f723c035070 con 0x7f7248101df0 2024-09-19T05:04:49.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.627+0000 7f7251ebe740 1 Processor -- start 2024-09-19T05:04:49.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.628+0000 7f7251ebe740 1 -- start start 2024-09-19T05:04:49.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.628+0000 7f7245600640 1 -- 172.21.15.43:0/3494491756 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f723c042030 con 0x7f7248101df0 2024-09-19T05:04:49.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.628+0000 7f7251ebe740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558987ec8ac0 0x558987ec8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:49.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.628+0000 7f7251ebe740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558987ec93e0 0x558987ed3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:49.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.628+0000 7f7251ebe740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558987ed3ca0 0x558987ed6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:49.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.628+0000 7f7251ebe740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558987e2f290 con 0x558987ec93e0 2024-09-19T05:04:49.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.628+0000 7f7251ebe740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558987dd6b40 con 0x558987ed3ca0 2024-09-19T05:04:49.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.628+0000 7f7251ebe740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558987e2b250 con 0x558987ec8ac0 2024-09-19T05:04:49.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.628+0000 7f724ca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558987ec93e0 0x558987ed3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:49.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.629+0000 7f724ca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558987ec93e0 0x558987ed3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38776/0 (socket says 172.21.15.43:38776) 2024-09-19T05:04:49.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.629+0000 7f724ca00640 1 -- 172.21.15.43:0/2139370862 learned_addr learned my addr 172.21.15.43:0/2139370862 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:49.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.629+0000 7f724d400640 1 --2- 172.21.15.43:0/2139370862 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558987ec8ac0 0x558987ec8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:49.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.629+0000 7f7247e00640 1 --2- 172.21.15.43:0/2139370862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558987ed3ca0 0x558987ed6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:49.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.629+0000 7f724ca00640 1 -- 172.21.15.43:0/2139370862 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558987ec8ac0 msgr2=0x558987ec8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:49.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.629+0000 7f724ca00640 1 --2- 172.21.15.43:0/2139370862 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558987ec8ac0 0x558987ec8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.629+0000 7f724ca00640 1 -- 172.21.15.43:0/2139370862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558987ed3ca0 msgr2=0x558987ed6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:49.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.629+0000 7f724ca00640 1 --2- 172.21.15.43:0/2139370862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558987ed3ca0 0x558987ed6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.629+0000 7f724ca00640 1 -- 172.21.15.43:0/2139370862 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558987ba06a0 con 0x558987ec93e0 2024-09-19T05:04:49.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.630+0000 7f724ca00640 1 --2- 172.21.15.43:0/2139370862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558987ec93e0 0x558987ed3760 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f723c0eda00 tx=0x7f723c0f8620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c28bd947cca28f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:49.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.630+0000 7f7239c00640 1 -- 172.21.15.43:0/2139370862 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f723c0f24e0 con 0x558987ec93e0 2024-09-19T05:04:49.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.630+0000 7f7239c00640 1 -- 172.21.15.43:0/2139370862 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f723c0fd030 con 0x558987ec93e0 2024-09-19T05:04:49.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.630+0000 7f7239c00640 1 -- 172.21.15.43:0/2139370862 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f723c102a60 con 0x558987ec93e0 2024-09-19T05:04:49.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.631+0000 7f7251ebe740 1 -- 172.21.15.43:0/2139370862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558987ec93e0 msgr2=0x558987ed3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:49.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.631+0000 7f7251ebe740 1 --2- 172.21.15.43:0/2139370862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558987ec93e0 0x558987ed3760 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f723c0eda00 tx=0x7f723c0f8620 comp rx=0 tx=0).stop 2024-09-19T05:04:49.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.631+0000 7f7245600640 1 --2- 172.21.15.43:0/3494491756 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f72300a34d0 0x7f72300a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:49.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.631+0000 7f7245600640 1 -- 172.21.15.43:0/3494491756 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f723c0f03f0 con 0x7f7248101df0 2024-09-19T05:04:49.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.631+0000 7f7251ebe740 1 -- 172.21.15.43:0/2139370862 shutdown_connections 2024-09-19T05:04:49.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.631+0000 7f7251ebe740 1 --2- 172.21.15.43:0/2139370862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558987ed3ca0 0x558987ed6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.631+0000 7f7251ebe740 1 --2- 172.21.15.43:0/2139370862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558987ec93e0 0x558987ed3760 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.631+0000 7f7251ebe740 1 --2- 172.21.15.43:0/2139370862 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558987ec8ac0 0x558987ec8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.631+0000 7f7251ebe740 1 -- 172.21.15.43:0/2139370862 >> 172.21.15.43:0/2139370862 conn(0x558987ec5050 msgr2=0x558987ec5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:49.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.631+0000 7f724d400640 1 --2- 172.21.15.43:0/3494491756 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f72300a34d0 0x7f72300a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:49.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.631+0000 7f7251ebe740 1 -- 172.21.15.43:0/2139370862 shutdown_connections 2024-09-19T05:04:49.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.632+0000 7f7251ebe740 1 -- 172.21.15.43:0/2139370862 wait complete. 2024-09-19T05:04:49.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.632+0000 7f724d400640 1 --2- 172.21.15.43:0/3494491756 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f72300a34d0 0x7f72300a5990 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7240005fd0 tx=0x7f7240009500 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:49.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.632+0000 7f7251ebe740 1 -- start start 2024-09-19T05:04:49.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.633+0000 7f7251ebe740 1 Processor -- start 2024-09-19T05:04:49.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.633+0000 7f7251ebe740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558987ec8ac0 0x558987ed3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:49.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.633+0000 7f7251ebe740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558987ec93e0 0x558987ed0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:49.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.633+0000 7f7251ebe740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558987ed3ca0 0x558987ed0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:49.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.633+0000 7f7251ebe740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558987ed6630 con 0x558987ec8ac0 2024-09-19T05:04:49.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.633+0000 7f7251ebe740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558987de9fa0 con 0x558987ed3ca0 2024-09-19T05:04:49.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.633+0000 7f7251ebe740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558987ed67b0 con 0x558987ec93e0 2024-09-19T05:04:49.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.633+0000 7f7251ebe740 10 client.0 _ll_register_callbacks cb 0x558987b3dba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:04:49.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.634+0000 7f724ca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558987ec8ac0 0x558987ed3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:49.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.634+0000 7f724d400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558987ed3ca0 0x558987ed0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:49.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.634+0000 7f724d400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558987ed3ca0 0x558987ed0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:46878/0 (socket says 172.21.15.43:46878) 2024-09-19T05:04:49.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.634+0000 7f724d400640 1 -- 172.21.15.43:0/2548931484 learned_addr learned my addr 172.21.15.43:0/2548931484 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:49.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.634+0000 7f7247e00640 1 --2- 172.21.15.43:0/2548931484 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558987ec93e0 0x558987ed0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:49.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.634+0000 7f724d400640 1 -- 172.21.15.43:0/2548931484 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558987ec93e0 msgr2=0x558987ed0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:49.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.634+0000 7f724d400640 1 --2- 172.21.15.43:0/2548931484 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558987ec93e0 0x558987ed0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.634+0000 7f724d400640 1 -- 172.21.15.43:0/2548931484 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558987ec8ac0 msgr2=0x558987ed3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:49.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.634+0000 7f724d400640 1 --2- 172.21.15.43:0/2548931484 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558987ec8ac0 0x558987ed3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.634+0000 7f724d400640 1 -- 172.21.15.43:0/2548931484 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558987ed1550 con 0x558987ed3ca0 2024-09-19T05:04:49.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.634+0000 7f724d400640 1 --2- 172.21.15.43:0/2548931484 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558987ed3ca0 0x558987ed0d70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7240010b10 tx=0x7f7240004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e71aa651f0d3c8e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:49.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.634+0000 7f722e000640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:04:49.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.635+0000 7f722e000640 1 -- 172.21.15.43:0/2548931484 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724002c060 con 0x558987ed3ca0 2024-09-19T05:04:49.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.635+0000 7f7251ebe740 1 -- 172.21.15.43:0/2548931484 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f723c0fd030 con 0x558987ed3ca0 2024-09-19T05:04:49.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.635+0000 7f722e000640 1 -- 172.21.15.43:0/2548931484 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f7240017030 con 0x558987ed3ca0 2024-09-19T05:04:49.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.635+0000 7f722e000640 1 -- 172.21.15.43:0/2548931484 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f724001fc80 con 0x558987ed3ca0 2024-09-19T05:04:49.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.635+0000 7f7251ebe740 10 client.25286 fetch_fsmap learned FSMap version 29 2024-09-19T05:04:49.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.635+0000 7f7251ebe740 1 -- 172.21.15.43:0/2548931484 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558987ee25a0 con 0x558987ed3ca0 2024-09-19T05:04:49.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.635+0000 7f722e000640 1 -- 172.21.15.43:0/2548931484 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f723c0fd030 con 0x558987ed3ca0 2024-09-19T05:04:49.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.636+0000 7f722e000640 1 -- 172.21.15.43:0/2548931484 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7240027410 con 0x558987ed3ca0 2024-09-19T05:04:49.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.636+0000 7f7251ebe740 10 client.25286 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:04:49.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.636+0000 7f7251ebe740 10 client.25286 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:04:49.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.636+0000 7f7251ebe740 20 client.25286 populate_metadata read hostname 'smithi043' 2024-09-19T05:04:49.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.636+0000 7f7251ebe740 1 --2- 172.21.15.43:0/2548931484 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558987ee39b0 0x558987ee5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:49.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.636+0000 7f7251ebe740 4 client.25286 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:04:49.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.636+0000 7f7251ebe740 1 -- 172.21.15.43:0/2548931484 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558987e3c800 con 0x558987ee39b0 2024-09-19T05:04:49.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.637+0000 7f724ca00640 1 --2- 172.21.15.43:0/2548931484 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558987ee39b0 0x558987ee5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:49.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.637+0000 7f724ca00640 1 --2- 172.21.15.43:0/2548931484 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558987ee39b0 0x558987ee5e00 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74ed69e749d3d39a server_cookie=62f6489b2259fb8f in_seq=0 out_seq=0 2024-09-19T05:04:49.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.637+0000 7f722e000640 10 client.25286 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:04:49.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.638+0000 7f722e000640 1 -- 172.21.15.43:0/2548931484 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558987e3c800 con 0x558987ee39b0 2024-09-19T05:04:49.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.638+0000 7f722e000640 10 client.25286 handle_command_reply: tid=0 2024-09-19T05:04:49.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.638+0000 7f7251ebe740 1 client.25286 shutdown 2024-09-19T05:04:49.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.638+0000 7f7251ebe740 1 -- 172.21.15.43:0/2548931484 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558987ed3ca0 msgr2=0x558987ed0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:49.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.638+0000 7f7251ebe740 1 --2- 172.21.15.43:0/2548931484 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558987ed3ca0 0x558987ed0d70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7240010b10 tx=0x7f7240004ab0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.638+0000 7f7251ebe740 1 -- 172.21.15.43:0/2548931484 shutdown_connections 2024-09-19T05:04:49.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.638+0000 7f7251ebe740 1 --2- 172.21.15.43:0/2548931484 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558987ee39b0 0x558987ee5e00 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.638+0000 7f7251ebe740 1 --2- 172.21.15.43:0/2548931484 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558987ed3ca0 0x558987ed0d70 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.638+0000 7f7251ebe740 1 --2- 172.21.15.43:0/2548931484 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558987ec93e0 0x558987ed0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.638+0000 7f7251ebe740 1 --2- 172.21.15.43:0/2548931484 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558987ec8ac0 0x558987ed3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.638+0000 7f7251ebe740 1 -- 172.21.15.43:0/2548931484 >> 172.21.15.43:0/2548931484 conn(0x558987ec5050 msgr2=0x558987ec41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:49.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.638+0000 7f722e000640 0 client.25286 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:04:49.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.638+0000 7f7251ebe740 1 -- 172.21.15.43:0/2548931484 shutdown_connections 2024-09-19T05:04:49.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.639+0000 7f7251ebe740 1 -- 172.21.15.43:0/2548931484 wait complete. 2024-09-19T05:04:49.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.639+0000 7f7251ebe740 20 client.25286 trim_cache size 0 max 16384 2024-09-19T05:04:49.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.657+0000 7f7251ebe740 1 Processor -- start 2024-09-19T05:04:49.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.657+0000 7f7251ebe740 1 -- start start 2024-09-19T05:04:49.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.658+0000 7f7251ebe740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558987ec8ac0 0x558987ed2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:49.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.658+0000 7f7251ebe740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558987ec93e0 0x558987ec40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:49.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.658+0000 7f7251ebe740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558987ed3ca0 0x558987eeb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:49.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.658+0000 7f7251ebe740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f72481afa80 con 0x558987ed3ca0 2024-09-19T05:04:49.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.658+0000 7f7251ebe740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f72481b15c0 con 0x558987ec93e0 2024-09-19T05:04:49.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.658+0000 7f7251ebe740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f72481b0150 con 0x558987ec8ac0 2024-09-19T05:04:49.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.658+0000 7f724d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558987ed3ca0 0x558987eeb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:49.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.658+0000 7f724d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558987ed3ca0 0x558987eeb6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38794/0 (socket says 172.21.15.43:38794) 2024-09-19T05:04:49.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.658+0000 7f724d400640 1 -- 172.21.15.43:0/3575197275 learned_addr learned my addr 172.21.15.43:0/3575197275 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:49.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.658+0000 7f7247e00640 1 --2- 172.21.15.43:0/3575197275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558987ec93e0 0x558987ec40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:49.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.658+0000 7f724ca00640 1 --2- 172.21.15.43:0/3575197275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558987ec8ac0 0x558987ed2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:49.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.658+0000 7f7247e00640 1 -- 172.21.15.43:0/3575197275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558987ec8ac0 msgr2=0x558987ed2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:49.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.658+0000 7f7247e00640 1 --2- 172.21.15.43:0/3575197275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558987ec8ac0 0x558987ed2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.658+0000 7f7247e00640 1 -- 172.21.15.43:0/3575197275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558987ed3ca0 msgr2=0x558987eeb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:49.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.658+0000 7f7247e00640 1 --2- 172.21.15.43:0/3575197275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558987ed3ca0 0x558987eeb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.659+0000 7f7247e00640 1 -- 172.21.15.43:0/3575197275 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558987ec88f0 con 0x558987ec93e0 2024-09-19T05:04:49.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.659+0000 7f7247e00640 1 --2- 172.21.15.43:0/3575197275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558987ec93e0 0x558987ec40b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x558987ed1e70 tx=0x7f7234012820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ea03d4dd9b310cd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:49.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.659+0000 7f7239200640 1 -- 172.21.15.43:0/3575197275 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7234021070 con 0x558987ec93e0 2024-09-19T05:04:49.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.659+0000 7f7239200640 1 -- 172.21.15.43:0/3575197275 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72340035f0 con 0x558987ec93e0 2024-09-19T05:04:49.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.659+0000 7f7239200640 1 -- 172.21.15.43:0/3575197275 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f723400f6b0 con 0x558987ec93e0 2024-09-19T05:04:49.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.660+0000 7f7251ebe740 1 -- 172.21.15.43:0/3575197275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558987ec93e0 msgr2=0x558987ec40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:49.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.660+0000 7f7251ebe740 1 --2- 172.21.15.43:0/3575197275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558987ec93e0 0x558987ec40b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x558987ed1e70 tx=0x7f7234012820 comp rx=0 tx=0).stop 2024-09-19T05:04:49.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.660+0000 7f7251ebe740 1 -- 172.21.15.43:0/3575197275 shutdown_connections 2024-09-19T05:04:49.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.660+0000 7f7251ebe740 1 --2- 172.21.15.43:0/3575197275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558987ed3ca0 0x558987eeb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.660+0000 7f7251ebe740 1 --2- 172.21.15.43:0/3575197275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558987ec93e0 0x558987ec40b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.660+0000 7f7251ebe740 1 --2- 172.21.15.43:0/3575197275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558987ec8ac0 0x558987ed2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.660+0000 7f7251ebe740 1 -- 172.21.15.43:0/3575197275 >> 172.21.15.43:0/3575197275 conn(0x558987ec5050 msgr2=0x558987ec39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:49.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.660+0000 7f7251ebe740 1 -- 172.21.15.43:0/3575197275 shutdown_connections 2024-09-19T05:04:49.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.660+0000 7f7251ebe740 1 -- 172.21.15.43:0/3575197275 wait complete. 2024-09-19T05:04:49.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.660+0000 7f7251ebe740 1 -- start start 2024-09-19T05:04:49.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.661+0000 7f7251ebe740 1 Processor -- start 2024-09-19T05:04:49.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.661+0000 7f7251ebe740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558987ec8ac0 0x558987edc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:49.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.661+0000 7f7251ebe740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558987ec93e0 0x558987ed9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:49.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.661+0000 7f7251ebe740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558987ed3ca0 0x558987ed9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:49.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.661+0000 7f7251ebe740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f72481b0ef0 con 0x558987ed3ca0 2024-09-19T05:04:49.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.661+0000 7f7251ebe740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f72481b0820 con 0x558987ec93e0 2024-09-19T05:04:49.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.661+0000 7f7251ebe740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f72481ada60 con 0x558987ec8ac0 2024-09-19T05:04:49.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.661+0000 7f7251ebe740 10 client.0 _ll_register_callbacks cb 0x7f7240009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:04:49.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.662+0000 7f724d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558987ed3ca0 0x558987ed9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:49.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.662+0000 7f7247e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558987ec93e0 0x558987ed9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:49.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.662+0000 7f724ca00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558987ec8ac0 0x558987edc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:49.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.662+0000 7f7247e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558987ec93e0 0x558987ed9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:46904/0 (socket says 172.21.15.43:46904) 2024-09-19T05:04:49.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.662+0000 7f724ca00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558987ec8ac0 0x558987edc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:36192/0 (socket says 172.21.15.43:36192) 2024-09-19T05:04:49.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.662+0000 7f7247e00640 1 -- 172.21.15.43:0/3335256810 learned_addr learned my addr 172.21.15.43:0/3335256810 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:49.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.662+0000 7f7247e00640 1 -- 172.21.15.43:0/3335256810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558987ec8ac0 msgr2=0x558987edc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:49.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.662+0000 7f7247e00640 1 --2- 172.21.15.43:0/3335256810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558987ec8ac0 0x558987edc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.662+0000 7f7247e00640 1 -- 172.21.15.43:0/3335256810 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558987ed3ca0 msgr2=0x558987ed9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:49.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.662+0000 7f7247e00640 1 --2- 172.21.15.43:0/3335256810 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558987ed3ca0 0x558987ed9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.662+0000 7f7247e00640 1 -- 172.21.15.43:0/3335256810 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558987f55ec0 con 0x558987ec93e0 2024-09-19T05:04:49.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.662+0000 7f7247e00640 1 --2- 172.21.15.43:0/3335256810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558987ec93e0 0x558987ed9610 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7234003f90 tx=0x7f72340056c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e0bdab6c05a44f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:49.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.662+0000 7f722e000640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:04:49.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.663+0000 7f722e000640 1 -- 172.21.15.43:0/3335256810 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f723402b020 con 0x558987ec93e0 2024-09-19T05:04:49.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.663+0000 7f722e000640 1 -- 172.21.15.43:0/3335256810 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f7234003bb0 con 0x558987ec93e0 2024-09-19T05:04:49.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.663+0000 7f7251ebe740 1 -- 172.21.15.43:0/3335256810 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72340035f0 con 0x558987ec93e0 2024-09-19T05:04:49.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.663+0000 7f722e000640 1 -- 172.21.15.43:0/3335256810 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7234036050 con 0x558987ec93e0 2024-09-19T05:04:49.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.663+0000 7f7251ebe740 10 client.25292 fetch_fsmap learned FSMap version 29 2024-09-19T05:04:49.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.663+0000 7f7251ebe740 1 -- 172.21.15.43:0/3335256810 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558987f57080 con 0x558987ec93e0 2024-09-19T05:04:49.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.664+0000 7f722e000640 1 -- 172.21.15.43:0/3335256810 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72340035f0 con 0x558987ec93e0 2024-09-19T05:04:49.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.664+0000 7f722e000640 1 -- 172.21.15.43:0/3335256810 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f723401b030 con 0x558987ec93e0 2024-09-19T05:04:49.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.664+0000 7f7251ebe740 10 client.25292 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:04:49.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.664+0000 7f7251ebe740 10 client.25292 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:04:49.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.664+0000 7f7251ebe740 20 client.25292 populate_metadata read hostname 'smithi043' 2024-09-19T05:04:49.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.664+0000 7f7251ebe740 1 --2- 172.21.15.43:0/3335256810 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558987f57720 0x558987f59b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:49.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.664+0000 7f7251ebe740 4 client.25292 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T05:04:49.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.664+0000 7f7251ebe740 1 -- 172.21.15.43:0/3335256810 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558987ee6190 con 0x558987f57720 2024-09-19T05:04:49.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.664+0000 7f724ca00640 1 --2- 172.21.15.43:0/3335256810 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558987f57720 0x558987f59b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:49.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.665+0000 7f724ca00640 1 --2- 172.21.15.43:0/3335256810 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558987f57720 0x558987f59b40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2520574a4804c2c5 server_cookie=5eea97b3fb725dfd in_seq=0 out_seq=0 2024-09-19T05:04:49.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.665+0000 7f722e000640 10 client.25292 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:04:49.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.666+0000 7f722e000640 1 -- 172.21.15.43:0/3335256810 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558987ee6190 con 0x558987f57720 2024-09-19T05:04:49.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.666+0000 7f722e000640 10 client.25292 handle_command_reply: tid=0 2024-09-19T05:04:49.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.666+0000 7f7251ebe740 1 client.25292 shutdown 2024-09-19T05:04:49.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.666+0000 7f7251ebe740 1 -- 172.21.15.43:0/3335256810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558987ec93e0 msgr2=0x558987ed9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:49.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.666+0000 7f7251ebe740 1 --2- 172.21.15.43:0/3335256810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558987ec93e0 0x558987ed9610 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7234003f90 tx=0x7f72340056c0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.666+0000 7f7251ebe740 1 -- 172.21.15.43:0/3335256810 shutdown_connections 2024-09-19T05:04:49.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.666+0000 7f7251ebe740 1 --2- 172.21.15.43:0/3335256810 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558987f57720 0x558987f59b40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.666+0000 7f7251ebe740 1 --2- 172.21.15.43:0/3335256810 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558987ed3ca0 0x558987ed9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.666+0000 7f7251ebe740 1 --2- 172.21.15.43:0/3335256810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558987ec93e0 0x558987ed9610 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.666+0000 7f7251ebe740 1 --2- 172.21.15.43:0/3335256810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558987ec8ac0 0x558987edc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.666+0000 7f722e000640 0 client.25292 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:04:49.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.666+0000 7f7251ebe740 1 -- 172.21.15.43:0/3335256810 >> 172.21.15.43:0/3335256810 conn(0x558987ec5050 msgr2=0x558987ec39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:49.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.666+0000 7f7251ebe740 1 -- 172.21.15.43:0/3335256810 shutdown_connections 2024-09-19T05:04:49.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.666+0000 7f7251ebe740 1 -- 172.21.15.43:0/3335256810 wait complete. 2024-09-19T05:04:49.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.666+0000 7f7251ebe740 20 client.25292 trim_cache size 0 max 16384 2024-09-19T05:04:49.670 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:04:49.670 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-09-19T05:04:49.670 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "bd603c11-f503-4f2e-bf95-3ef8de152522", 2024-09-19T05:04:49.670 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-09-19T05:04:49.670 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:04:49.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.667+0000 7f724f400640 1 -- 172.21.15.43:0/3494491756 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f72300a34d0 msgr2=0x7f72300a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:49.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.667+0000 7f724f400640 1 --2- 172.21.15.43:0/3494491756 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f72300a34d0 0x7f72300a5990 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7240005fd0 tx=0x7f7240009500 comp rx=0 tx=0).stop 2024-09-19T05:04:49.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.667+0000 7f724f400640 1 -- 172.21.15.43:0/3494491756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7248101df0 msgr2=0x7f724819a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:49.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.667+0000 7f724f400640 1 --2- 172.21.15.43:0/3494491756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7248101df0 0x7f724819a5d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f723c006010 tx=0x7f723c004040 comp rx=0 tx=0).stop 2024-09-19T05:04:49.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.667+0000 7f724f400640 1 -- 172.21.15.43:0/3494491756 shutdown_connections 2024-09-19T05:04:49.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.667+0000 7f724f400640 1 --2- 172.21.15.43:0/3494491756 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f72300a34d0 0x7f72300a5990 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.667+0000 7f724f400640 1 --2- 172.21.15.43:0/3494491756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7248106f20 0x7f724819eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.667+0000 7f724f400640 1 --2- 172.21.15.43:0/3494491756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72481027a0 0x7f724819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.667+0000 7f724f400640 1 --2- 172.21.15.43:0/3494491756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7248101df0 0x7f724819a5d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.667+0000 7f724f400640 1 -- 172.21.15.43:0/3494491756 >> 172.21.15.43:0/3494491756 conn(0x7f72480fd8b0 msgr2=0x7f72480fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:49.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.668+0000 7f724f400640 1 -- 172.21.15.43:0/3494491756 shutdown_connections 2024-09-19T05:04:49.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.668+0000 7f724f400640 1 -- 172.21.15.43:0/3494491756 wait complete. 2024-09-19T05:04:49.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.798+0000 7fc6ad800640 1 -- 172.21.15.43:0/1567581976 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fc6a8075a20 con 0x7fc6a8102480 2024-09-19T05:04:49.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.802+0000 7fc69f400640 1 -- 172.21.15.43:0/1567581976 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v32)=0 v32) ==== 133+0+0 (secure 0 0 0) 0x7fc6a8075a20 con 0x7fc6a8102480 2024-09-19T05:04:49.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.805+0000 7fc6ad800640 1 -- 172.21.15.43:0/1567581976 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fc68c0a3580 msgr2=0x7fc68c0a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:49.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.805+0000 7fc6ad800640 1 --2- 172.21.15.43:0/1567581976 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fc68c0a3580 0x7fc68c0a5a40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc698009ba0 tx=0x7fc698005f90 comp rx=0 tx=0).stop 2024-09-19T05:04:49.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.805+0000 7fc6ad800640 1 -- 172.21.15.43:0/1567581976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6a8102480 msgr2=0x7fc6a8074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:49.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.805+0000 7fc6ad800640 1 --2- 172.21.15.43:0/1567581976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6a8102480 0x7fc6a8074a30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc69402a940 tx=0x7fc694003b90 comp rx=0 tx=0).stop 2024-09-19T05:04:49.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.805+0000 7fc6ad800640 1 -- 172.21.15.43:0/1567581976 shutdown_connections 2024-09-19T05:04:49.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.805+0000 7fc6ad800640 1 --2- 172.21.15.43:0/1567581976 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6a8107b80 0x7fc6a8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.805+0000 7fc6ad800640 1 --2- 172.21.15.43:0/1567581976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6a8102480 0x7fc6a8074a30 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.805+0000 7fc6ad800640 1 --2- 172.21.15.43:0/1567581976 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fc68c0a3580 0x7fc68c0a5a40 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.805+0000 7fc6ad800640 1 --2- 172.21.15.43:0/1567581976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc6a8101ae0 0x7fc6a80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.805+0000 7fc6ad800640 1 -- 172.21.15.43:0/1567581976 >> 172.21.15.43:0/1567581976 conn(0x7fc6a80fd7e0 msgr2=0x7fc6a80fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:49.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.805+0000 7fc6ad800640 1 -- 172.21.15.43:0/1567581976 shutdown_connections 2024-09-19T05:04:49.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.805+0000 7fc6ad800640 1 -- 172.21.15.43:0/1567581976 wait complete. 2024-09-19T05:04:49.824 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-19T05:04:49.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.986+0000 7f2130e00640 1 -- 172.21.15.43:0/4279615922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f212c103f30 msgr2=0x7f212c104310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:49.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.986+0000 7f2130e00640 1 --2- 172.21.15.43:0/4279615922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f212c103f30 0x7f212c104310 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f211c009900 tx=0x7f211c02a3f0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.986+0000 7f2130e00640 1 -- 172.21.15.43:0/4279615922 shutdown_connections 2024-09-19T05:04:49.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.986+0000 7f2130e00640 1 --2- 172.21.15.43:0/4279615922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f212c109080 0x7f212c110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.986+0000 7f2130e00640 1 --2- 172.21.15.43:0/4279615922 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f212c1048e0 0x7f212c108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.986+0000 7f2130e00640 1 --2- 172.21.15.43:0/4279615922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f212c103f30 0x7f212c104310 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.986+0000 7f2130e00640 1 -- 172.21.15.43:0/4279615922 >> 172.21.15.43:0/4279615922 conn(0x7f212c0ff980 msgr2=0x7f212c101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:49.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.986+0000 7f2130e00640 1 -- 172.21.15.43:0/4279615922 shutdown_connections 2024-09-19T05:04:49.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.987+0000 7f2130e00640 1 -- 172.21.15.43:0/4279615922 wait complete. 2024-09-19T05:04:49.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.988+0000 7f2130e00640 1 Processor -- start 2024-09-19T05:04:49.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.988+0000 7f2130e00640 1 -- start start 2024-09-19T05:04:49.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.988+0000 7f2130e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f212c103f30 0x7f212c19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:49.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.988+0000 7f2130e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f212c109080 0x7f212c19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:49.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.988+0000 7f2130e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f212c19ee80 0x7f212c19f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:49.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.988+0000 7f2130e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f212c114110 con 0x7f212c103f30 2024-09-19T05:04:49.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.988+0000 7f2130e00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f212c113f90 con 0x7f212c109080 2024-09-19T05:04:49.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.988+0000 7f2130e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f212c114290 con 0x7f212c19ee80 2024-09-19T05:04:49.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.988+0000 7f2129e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f212c103f30 0x7f212c19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:49.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.989+0000 7f2123e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f212c109080 0x7f212c19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:49.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.989+0000 7f2129e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f212c103f30 0x7f212c19a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38820/0 (socket says 172.21.15.43:38820) 2024-09-19T05:04:49.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.989+0000 7f2123e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f212c109080 0x7f212c19ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:46930/0 (socket says 172.21.15.43:46930) 2024-09-19T05:04:49.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.989+0000 7f2129e00640 1 -- 172.21.15.43:0/745151533 learned_addr learned my addr 172.21.15.43:0/745151533 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:49.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.989+0000 7f212a800640 1 --2- 172.21.15.43:0/745151533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f212c19ee80 0x7f212c19f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:49.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.989+0000 7f2123e00640 1 -- 172.21.15.43:0/745151533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f212c19ee80 msgr2=0x7f212c19f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:49.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.989+0000 7f2123e00640 1 --2- 172.21.15.43:0/745151533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f212c19ee80 0x7f212c19f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.989+0000 7f2123e00640 1 -- 172.21.15.43:0/745151533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f212c103f30 msgr2=0x7f212c19a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:49.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.989+0000 7f2123e00640 1 --2- 172.21.15.43:0/745151533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f212c103f30 0x7f212c19a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:49.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.989+0000 7f2123e00640 1 -- 172.21.15.43:0/745151533 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f212c19f970 con 0x7f212c109080 2024-09-19T05:04:49.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.989+0000 7f2123e00640 1 --2- 172.21.15.43:0/745151533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f212c109080 0x7f212c19ab00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f212c06b470 tx=0x7f211400dbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:49.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.994+0000 7f2122a00640 1 -- 172.21.15.43:0/745151533 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2114017070 con 0x7f212c109080 2024-09-19T05:04:49.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.994+0000 7f2122a00640 1 -- 172.21.15.43:0/745151533 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21140099d0 con 0x7f212c109080 2024-09-19T05:04:49.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.994+0000 7f2122a00640 1 -- 172.21.15.43:0/745151533 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21140047a0 con 0x7f212c109080 2024-09-19T05:04:49.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.994+0000 7f2130e00640 1 -- 172.21.15.43:0/745151533 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f212c19fc60 con 0x7f212c109080 2024-09-19T05:04:49.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.995+0000 7f2130e00640 1 -- 172.21.15.43:0/745151533 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f212c1a7400 con 0x7f212c109080 2024-09-19T05:04:49.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.996+0000 7f2130e00640 1 -- 172.21.15.43:0/745151533 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f212c104310 con 0x7f212c109080 2024-09-19T05:04:49.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.997+0000 7f2122a00640 1 -- 172.21.15.43:0/745151533 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f2114009b80 con 0x7f212c109080 2024-09-19T05:04:50.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.999+0000 7f2122a00640 1 --2- 172.21.15.43:0/745151533 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f21100a3600 0x7f21100a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:50.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.999+0000 7f2122a00640 1 -- 172.21.15.43:0/745151533 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f21140cfbf0 con 0x7f212c109080 2024-09-19T05:04:50.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.999+0000 7f2129e00640 1 --2- 172.21.15.43:0/745151533 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f21100a3600 0x7f21100a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:50.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:49.999+0000 7f2129e00640 1 --2- 172.21.15.43:0/745151533 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f21100a3600 0x7f21100a5ac0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f211c0096c0 tx=0x7f211c0046c0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:50.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.003+0000 7f2122a00640 1 -- 172.21.15.43:0/745151533 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2114020020 con 0x7f212c109080 2024-09-19T05:04:50.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.229+0000 7f2130e00640 1 -- 172.21.15.43:0/745151533 --> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f212c1048e0 con 0x7f21100a3600 2024-09-19T05:04:50.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:49 smithi043 ceph-mon[30800]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.1 KiB/s wr, 20 op/s 2024-09-19T05:04:50.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:49 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/879650239' entity='client.admin' 2024-09-19T05:04:50.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:49 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:04:50.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:49 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:04:50.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:49 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:04:50.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:49 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:50.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:49 smithi043 ceph-mon[30800]: from='client.? ' entity='client.admin' 2024-09-19T05:04:50.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:49 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:04:50.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:49 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:04:50.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:49 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:04:50.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:49 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:50.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:49 smithi043 ceph-mon[30800]: scrub queued for path: / 2024-09-19T05:04:50.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:49 smithi043 ceph-mon[30800]: scrub summary: idle+waiting paths [/] 2024-09-19T05:04:50.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:49 smithi043 ceph-mon[30800]: scrub summary: active paths [/] 2024-09-19T05:04:50.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:49 smithi043 ceph-mon[30800]: scrub summary: idle+waiting paths [/] 2024-09-19T05:04:50.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:49 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/1567581976' entity='client.admin' 2024-09-19T05:04:50.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:49 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:04:50.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:49 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:04:50.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:49 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:04:50.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:49 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:50.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.302+0000 7f2122a00640 1 -- 172.21.15.43:0/745151533 <== mgr.25136 v2:172.21.15.43:6800/2120444093 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f212c1048e0 con 0x7f21100a3600 2024-09-19T05:04:50.303 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule set for path / 2024-09-19T05:04:50.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.304+0000 7f2130e00640 1 -- 172.21.15.43:0/745151533 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f21100a3600 msgr2=0x7f21100a5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:50.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.304+0000 7f2130e00640 1 --2- 172.21.15.43:0/745151533 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f21100a3600 0x7f21100a5ac0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f211c0096c0 tx=0x7f211c0046c0 comp rx=0 tx=0).stop 2024-09-19T05:04:50.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.304+0000 7f2130e00640 1 -- 172.21.15.43:0/745151533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f212c109080 msgr2=0x7f212c19ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:50.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.304+0000 7f2130e00640 1 --2- 172.21.15.43:0/745151533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f212c109080 0x7f212c19ab00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f212c06b470 tx=0x7f211400dbb0 comp rx=0 tx=0).stop 2024-09-19T05:04:50.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.304+0000 7f2130e00640 1 -- 172.21.15.43:0/745151533 shutdown_connections 2024-09-19T05:04:50.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.304+0000 7f2130e00640 1 --2- 172.21.15.43:0/745151533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f212c19ee80 0x7f212c19f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:50.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.304+0000 7f2130e00640 1 --2- 172.21.15.43:0/745151533 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f21100a3600 0x7f21100a5ac0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:50.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.304+0000 7f2130e00640 1 --2- 172.21.15.43:0/745151533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f212c109080 0x7f212c19ab00 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:50.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.304+0000 7f2130e00640 1 --2- 172.21.15.43:0/745151533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f212c103f30 0x7f212c19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:50.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.304+0000 7f2130e00640 1 -- 172.21.15.43:0/745151533 >> 172.21.15.43:0/745151533 conn(0x7f212c0ff980 msgr2=0x7f212c100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:50.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.304+0000 7f2130e00640 1 -- 172.21.15.43:0/745151533 shutdown_connections 2024-09-19T05:04:50.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.304+0000 7f2130e00640 1 -- 172.21.15.43:0/745151533 wait complete. 2024-09-19T05:04:50.320 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-19T05:04:50.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:49 smithi151 ceph-mon[35425]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.1 KiB/s wr, 20 op/s 2024-09-19T05:04:50.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:49 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/879650239' entity='client.admin' 2024-09-19T05:04:50.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:49 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:04:50.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:49 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:04:50.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:49 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:04:50.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:49 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:50.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:49 smithi151 ceph-mon[35425]: from='client.? ' entity='client.admin' 2024-09-19T05:04:50.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:49 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:04:50.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:49 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:04:50.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:49 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:04:50.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:49 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:50.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:49 smithi151 ceph-mon[35425]: scrub queued for path: / 2024-09-19T05:04:50.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:49 smithi151 ceph-mon[35425]: scrub summary: idle+waiting paths [/] 2024-09-19T05:04:50.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:49 smithi151 ceph-mon[35425]: scrub summary: active paths [/] 2024-09-19T05:04:50.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:49 smithi151 ceph-mon[35425]: scrub summary: idle+waiting paths [/] 2024-09-19T05:04:50.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:49 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/1567581976' entity='client.admin' 2024-09-19T05:04:50.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:49 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:04:50.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:49 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:04:50.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:49 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:04:50.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:49 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:50.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:49 smithi096 ceph-mon[35085]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.1 KiB/s wr, 20 op/s 2024-09-19T05:04:50.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:49 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/879650239' entity='client.admin' 2024-09-19T05:04:50.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:49 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:04:50.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:49 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:04:50.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:49 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:04:50.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:49 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:50.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:49 smithi096 ceph-mon[35085]: from='client.? ' entity='client.admin' 2024-09-19T05:04:50.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:49 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:04:50.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:49 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:04:50.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:49 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:04:50.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:49 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:50.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:49 smithi096 ceph-mon[35085]: scrub queued for path: / 2024-09-19T05:04:50.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:49 smithi096 ceph-mon[35085]: scrub summary: idle+waiting paths [/] 2024-09-19T05:04:50.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:49 smithi096 ceph-mon[35085]: scrub summary: active paths [/] 2024-09-19T05:04:50.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:49 smithi096 ceph-mon[35085]: scrub summary: idle+waiting paths [/] 2024-09-19T05:04:50.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:49 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/1567581976' entity='client.admin' 2024-09-19T05:04:50.475 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:49 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:04:50.475 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:49 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:04:50.475 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:49 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:04:50.475 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:49 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:04:50.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.480+0000 7f7815000640 1 -- 172.21.15.43:0/786606276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78101090a0 msgr2=0x7f7810110bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:50.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.480+0000 7f7815000640 1 --2- 172.21.15.43:0/786606276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78101090a0 0x7f7810110bf0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7808009ca0 tx=0x7f780802a8c0 comp rx=0 tx=0).stop 2024-09-19T05:04:50.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.480+0000 7f7815000640 1 -- 172.21.15.43:0/786606276 shutdown_connections 2024-09-19T05:04:50.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.480+0000 7f7815000640 1 --2- 172.21.15.43:0/786606276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78101090a0 0x7f7810110bf0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:50.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.480+0000 7f7815000640 1 --2- 172.21.15.43:0/786606276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7810104900 0x7f7810108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:50.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.480+0000 7f7815000640 1 --2- 172.21.15.43:0/786606276 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7810103f50 0x7f7810104330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:50.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.480+0000 7f7815000640 1 -- 172.21.15.43:0/786606276 >> 172.21.15.43:0/786606276 conn(0x7f78100ff9e0 msgr2=0x7f7810101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:50.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.480+0000 7f7815000640 1 -- 172.21.15.43:0/786606276 shutdown_connections 2024-09-19T05:04:50.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.480+0000 7f7815000640 1 -- 172.21.15.43:0/786606276 wait complete. 2024-09-19T05:04:50.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.481+0000 7f7815000640 1 Processor -- start 2024-09-19T05:04:50.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.482+0000 7f7815000640 1 -- start start 2024-09-19T05:04:50.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.482+0000 7f7815000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7810103f50 0x7f781019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:50.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.482+0000 7f7815000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78101090a0 0x7f781019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:50.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.482+0000 7f7815000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f781019ed50 0x7f781019f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:50.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.482+0000 7f7815000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7810114120 con 0x7f78101090a0 2024-09-19T05:04:50.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.482+0000 7f7815000640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7810113fa0 con 0x7f781019ed50 2024-09-19T05:04:50.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.482+0000 7f7815000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f78101142a0 con 0x7f7810103f50 2024-09-19T05:04:50.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.482+0000 7f780d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78101090a0 0x7f781019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:50.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.482+0000 7f780d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78101090a0 0x7f781019a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38834/0 (socket says 172.21.15.43:38834) 2024-09-19T05:04:50.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.482+0000 7f780d400640 1 -- 172.21.15.43:0/1803002837 learned_addr learned my addr 172.21.15.43:0/1803002837 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:50.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.482+0000 7f780e800640 1 --2- 172.21.15.43:0/1803002837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f781019ed50 0x7f781019f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:50.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.482+0000 7f780de00640 1 --2- 172.21.15.43:0/1803002837 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7810103f50 0x7f781019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:50.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.483+0000 7f780d400640 1 -- 172.21.15.43:0/1803002837 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7810103f50 msgr2=0x7f781019a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:50.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.483+0000 7f780d400640 1 --2- 172.21.15.43:0/1803002837 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7810103f50 0x7f781019a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:50.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.483+0000 7f780d400640 1 -- 172.21.15.43:0/1803002837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f781019ed50 msgr2=0x7f781019f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:50.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.483+0000 7f780d400640 1 --2- 172.21.15.43:0/1803002837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f781019ed50 0x7f781019f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:50.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.483+0000 7f780d400640 1 -- 172.21.15.43:0/1803002837 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f781019f840 con 0x7f78101090a0 2024-09-19T05:04:50.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.483+0000 7f780d400640 1 --2- 172.21.15.43:0/1803002837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78101090a0 0x7f781019a9d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f77fc00e9c0 tx=0x7f77fc00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:50.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.483+0000 7f7806a00640 1 -- 172.21.15.43:0/1803002837 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc00ccc0 con 0x7f78101090a0 2024-09-19T05:04:50.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.483+0000 7f7815000640 1 -- 172.21.15.43:0/1803002837 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f781019fb30 con 0x7f78101090a0 2024-09-19T05:04:50.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.483+0000 7f7806a00640 1 -- 172.21.15.43:0/1803002837 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77fc005500 con 0x7f78101090a0 2024-09-19T05:04:50.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.483+0000 7f7806a00640 1 -- 172.21.15.43:0/1803002837 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc010430 con 0x7f78101090a0 2024-09-19T05:04:50.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.483+0000 7f7815000640 1 -- 172.21.15.43:0/1803002837 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78101a7320 con 0x7f78101090a0 2024-09-19T05:04:50.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.484+0000 7f7815000640 1 -- 172.21.15.43:0/1803002837 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7810104330 con 0x7f78101090a0 2024-09-19T05:04:50.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.487+0000 7f7806a00640 1 -- 172.21.15.43:0/1803002837 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f77fc005000 con 0x7f78101090a0 2024-09-19T05:04:50.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.488+0000 7f7806a00640 1 --2- 172.21.15.43:0/1803002837 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f77f40a3580 0x7f77f40a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:50.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.489+0000 7f7806a00640 1 -- 172.21.15.43:0/1803002837 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f77fc0d03f0 con 0x7f78101090a0 2024-09-19T05:04:50.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.489+0000 7f780de00640 1 --2- 172.21.15.43:0/1803002837 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f77f40a3580 0x7f77f40a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:50.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.489+0000 7f780de00640 1 --2- 172.21.15.43:0/1803002837 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f77f40a3580 0x7f77f40a5a40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f780000abe0 tx=0x7f780000a2d0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:50.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.492+0000 7f7806a00640 1 -- 172.21.15.43:0/1803002837 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f77fc09cb30 con 0x7f78101090a0 2024-09-19T05:04:50.648 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:04:50.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.719+0000 7f7815000640 1 -- 172.21.15.43:0/1803002837 --> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f7810104ae0 con 0x7f77f40a3580 2024-09-19T05:04:50.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.791+0000 7f7806a00640 1 -- 172.21.15.43:0/1803002837 <== mgr.25136 v2:172.21.15.43:6800/2120444093 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f7810104ae0 con 0x7f77f40a3580 2024-09-19T05:04:50.792 INFO:teuthology.orchestra.run.smithi043.stdout:Retention added to path / 2024-09-19T05:04:50.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.793+0000 7f7815000640 1 -- 172.21.15.43:0/1803002837 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f77f40a3580 msgr2=0x7f77f40a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:50.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.793+0000 7f7815000640 1 --2- 172.21.15.43:0/1803002837 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f77f40a3580 0x7f77f40a5a40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f780000abe0 tx=0x7f780000a2d0 comp rx=0 tx=0).stop 2024-09-19T05:04:50.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.793+0000 7f7815000640 1 -- 172.21.15.43:0/1803002837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78101090a0 msgr2=0x7f781019a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:50.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.793+0000 7f7815000640 1 --2- 172.21.15.43:0/1803002837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78101090a0 0x7f781019a9d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f77fc00e9c0 tx=0x7f77fc00edc0 comp rx=0 tx=0).stop 2024-09-19T05:04:50.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.793+0000 7f7815000640 1 -- 172.21.15.43:0/1803002837 shutdown_connections 2024-09-19T05:04:50.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.793+0000 7f7815000640 1 --2- 172.21.15.43:0/1803002837 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f77f40a3580 0x7f77f40a5a40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:50.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.793+0000 7f7815000640 1 --2- 172.21.15.43:0/1803002837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f781019ed50 0x7f781019f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:50.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.793+0000 7f7815000640 1 --2- 172.21.15.43:0/1803002837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78101090a0 0x7f781019a9d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:50.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.793+0000 7f7815000640 1 --2- 172.21.15.43:0/1803002837 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7810103f50 0x7f781019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:50.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.793+0000 7f7815000640 1 -- 172.21.15.43:0/1803002837 >> 172.21.15.43:0/1803002837 conn(0x7f78100ff9e0 msgr2=0x7f78101003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:50.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.793+0000 7f7815000640 1 -- 172.21.15.43:0/1803002837 shutdown_connections 2024-09-19T05:04:50.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.793+0000 7f7815000640 1 -- 172.21.15.43:0/1803002837 wait complete. 2024-09-19T05:04:50.811 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-19T05:04:50.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.973+0000 7f2777e00640 1 -- 172.21.15.43:0/1352613194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2778109b90 msgr2=0x7f2778109f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:50.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.973+0000 7f2777e00640 1 --2- 172.21.15.43:0/1352613194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2778109b90 0x7f2778109f70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2768009900 tx=0x7f276802a3f0 comp rx=0 tx=0).stop 2024-09-19T05:04:50.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.973+0000 7f2777e00640 1 -- 172.21.15.43:0/1352613194 shutdown_connections 2024-09-19T05:04:50.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.973+0000 7f2777e00640 1 --2- 172.21.15.43:0/1352613194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2778104490 0x7f27781109c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:50.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.973+0000 7f2777e00640 1 --2- 172.21.15.43:0/1352613194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2778103af0 0x7f2778103f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:50.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.973+0000 7f2777e00640 1 --2- 172.21.15.43:0/1352613194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2778109b90 0x7f2778109f70 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:50.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.973+0000 7f2777e00640 1 -- 172.21.15.43:0/1352613194 >> 172.21.15.43:0/1352613194 conn(0x7f27780ff7d0 msgr2=0x7f2778101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:50.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.973+0000 7f2777e00640 1 -- 172.21.15.43:0/1352613194 shutdown_connections 2024-09-19T05:04:50.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.973+0000 7f2777e00640 1 -- 172.21.15.43:0/1352613194 wait complete. 2024-09-19T05:04:50.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.974+0000 7f2777e00640 1 Processor -- start 2024-09-19T05:04:50.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.974+0000 7f2777e00640 1 -- start start 2024-09-19T05:04:50.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.974+0000 7f2777e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2778103af0 0x7f277819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:50.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.974+0000 7f2777e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2778104490 0x7f277819a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:50.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.975+0000 7f2777e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2778109b90 0x7f277819ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:50.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.975+0000 7f2777e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2778113f50 con 0x7f2778109b90 2024-09-19T05:04:50.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.975+0000 7f2777e00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2778113dd0 con 0x7f2778104490 2024-09-19T05:04:50.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.975+0000 7f2777e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f27781140d0 con 0x7f2778103af0 2024-09-19T05:04:50.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.975+0000 7f2777400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2778109b90 0x7f277819ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:50.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.975+0000 7f2776000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2778104490 0x7f277819a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:50.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.975+0000 7f2777400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2778109b90 0x7f277819ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38846/0 (socket says 172.21.15.43:38846) 2024-09-19T05:04:50.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.975+0000 7f2777400640 1 -- 172.21.15.43:0/2031443173 learned_addr learned my addr 172.21.15.43:0/2031443173 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:50.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.975+0000 7f2776a00640 1 --2- 172.21.15.43:0/2031443173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2778103af0 0x7f277819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:50.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.975+0000 7f2776000640 1 -- 172.21.15.43:0/2031443173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2778103af0 msgr2=0x7f277819a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:50.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.975+0000 7f2776000640 1 --2- 172.21.15.43:0/2031443173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2778103af0 0x7f277819a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:50.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.975+0000 7f2776000640 1 -- 172.21.15.43:0/2031443173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2778109b90 msgr2=0x7f277819ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:50.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.975+0000 7f2776000640 1 --2- 172.21.15.43:0/2031443173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2778109b90 0x7f277819ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:50.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.975+0000 7f2776000640 1 -- 172.21.15.43:0/2031443173 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f277819f3f0 con 0x7f2778104490 2024-09-19T05:04:50.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.975+0000 7f2777400640 1 --2- 172.21.15.43:0/2031443173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2778109b90 0x7f277819ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:04:50.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.976+0000 7f2776000640 1 --2- 172.21.15.43:0/2031443173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2778104490 0x7f277819a940 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f276400e9f0 tx=0x7f276400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:50.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.976+0000 7f276f400640 1 -- 172.21.15.43:0/2031443173 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276400cd80 con 0x7f2778104490 2024-09-19T05:04:50.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.976+0000 7f2777e00640 1 -- 172.21.15.43:0/2031443173 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f277819f6e0 con 0x7f2778104490 2024-09-19T05:04:50.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.976+0000 7f276f400640 1 -- 172.21.15.43:0/2031443173 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27640035f0 con 0x7f2778104490 2024-09-19T05:04:50.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.976+0000 7f276f400640 1 -- 172.21.15.43:0/2031443173 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2764010670 con 0x7f2778104490 2024-09-19T05:04:50.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.976+0000 7f2777e00640 1 -- 172.21.15.43:0/2031443173 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27781a6f70 con 0x7f2778104490 2024-09-19T05:04:50.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.977+0000 7f2777e00640 1 -- 172.21.15.43:0/2031443173 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2778103f50 con 0x7f2778104490 2024-09-19T05:04:50.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.980+0000 7f276f400640 1 -- 172.21.15.43:0/2031443173 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f27640037a0 con 0x7f2778104490 2024-09-19T05:04:50.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.982+0000 7f276f400640 1 --2- 172.21.15.43:0/2031443173 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f275c0a34d0 0x7f275c0a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:50.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.982+0000 7f276f400640 1 -- 172.21.15.43:0/2031443173 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f27640d04a0 con 0x7f2778104490 2024-09-19T05:04:50.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.982+0000 7f2776a00640 1 --2- 172.21.15.43:0/2031443173 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f275c0a34d0 0x7f275c0a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:50.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.983+0000 7f2776a00640 1 --2- 172.21.15.43:0/2031443173 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f275c0a34d0 0x7f275c0a5990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2768000c00 tx=0x7f2768005b50 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:50.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:50.983+0000 7f276f400640 1 -- 172.21.15.43:0/2031443173 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f276409cbe0 con 0x7f2778104490 2024-09-19T05:04:51.054 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:04:51.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.200+0000 7f2777e00640 1 -- 172.21.15.43:0/2031443173 --> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f2778062630 con 0x7f275c0a34d0 2024-09-19T05:04:51.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.207+0000 7f276f400640 1 -- 172.21.15.43:0/2031443173 <== mgr.25136 v2:172.21.15.43:6800/2120444093 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f2778062630 con 0x7f275c0a34d0 2024-09-19T05:04:51.208 INFO:teuthology.orchestra.run.smithi043.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-19T00:00:00", "created": "2024-09-19T05:04:50", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-19T05:04:51.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.210+0000 7f2777e00640 1 -- 172.21.15.43:0/2031443173 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f275c0a34d0 msgr2=0x7f275c0a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:51.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.210+0000 7f2777e00640 1 --2- 172.21.15.43:0/2031443173 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f275c0a34d0 0x7f275c0a5990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2768000c00 tx=0x7f2768005b50 comp rx=0 tx=0).stop 2024-09-19T05:04:51.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.210+0000 7f2777e00640 1 -- 172.21.15.43:0/2031443173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2778104490 msgr2=0x7f277819a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:51.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.210+0000 7f2777e00640 1 --2- 172.21.15.43:0/2031443173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2778104490 0x7f277819a940 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f276400e9f0 tx=0x7f276400eeb0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.210+0000 7f2777e00640 1 -- 172.21.15.43:0/2031443173 shutdown_connections 2024-09-19T05:04:51.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.210+0000 7f2777e00640 1 --2- 172.21.15.43:0/2031443173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2778109b90 0x7f277819ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.210+0000 7f2777e00640 1 --2- 172.21.15.43:0/2031443173 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f275c0a34d0 0x7f275c0a5990 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.210+0000 7f2777e00640 1 --2- 172.21.15.43:0/2031443173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2778104490 0x7f277819a940 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.210+0000 7f2777e00640 1 --2- 172.21.15.43:0/2031443173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2778103af0 0x7f277819a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.210+0000 7f2777e00640 1 -- 172.21.15.43:0/2031443173 >> 172.21.15.43:0/2031443173 conn(0x7f27780ff7d0 msgr2=0x7f27780fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:51.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.210+0000 7f2777e00640 1 -- 172.21.15.43:0/2031443173 shutdown_connections 2024-09-19T05:04:51.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.210+0000 7f2777e00640 1 -- 172.21.15.43:0/2031443173 wait complete. 2024-09-19T05:04:51.226 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-19T05:04:51.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.392+0000 7ff011800640 1 -- 172.21.15.43:0/2246965852 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff00c107b80 msgr2=0x7ff00c107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:51.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.392+0000 7ff011800640 1 --2- 172.21.15.43:0/2246965852 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff00c107b80 0x7ff00c107f60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7feffc009930 tx=0x7feffc02a420 comp rx=0 tx=0).stop 2024-09-19T05:04:51.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.392+0000 7ff011800640 1 -- 172.21.15.43:0/2246965852 shutdown_connections 2024-09-19T05:04:51.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.392+0000 7ff011800640 1 --2- 172.21.15.43:0/2246965852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff00c102480 0x7ff00c10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.392+0000 7ff011800640 1 --2- 172.21.15.43:0/2246965852 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff00c101ae0 0x7ff00c101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.392+0000 7ff011800640 1 --2- 172.21.15.43:0/2246965852 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff00c107b80 0x7ff00c107f60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.392+0000 7ff011800640 1 -- 172.21.15.43:0/2246965852 >> 172.21.15.43:0/2246965852 conn(0x7ff00c0fd7e0 msgr2=0x7ff00c0ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:51.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.392+0000 7ff011800640 1 -- 172.21.15.43:0/2246965852 shutdown_connections 2024-09-19T05:04:51.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.392+0000 7ff011800640 1 -- 172.21.15.43:0/2246965852 wait complete. 2024-09-19T05:04:51.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.393+0000 7ff011800640 1 Processor -- start 2024-09-19T05:04:51.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.393+0000 7ff011800640 1 -- start start 2024-09-19T05:04:51.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.394+0000 7ff011800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff00c101ae0 0x7ff00c19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:51.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.394+0000 7ff011800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff00c102480 0x7ff00c19aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:51.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.394+0000 7ff011800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff00c107b80 0x7ff00c19ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:51.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.394+0000 7ff011800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff00c111f80 con 0x7ff00c107b80 2024-09-19T05:04:51.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.394+0000 7ff011800640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff00c111e00 con 0x7ff00c101ae0 2024-09-19T05:04:51.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.394+0000 7ff011800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff00c112100 con 0x7ff00c102480 2024-09-19T05:04:51.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.394+0000 7ff00b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff00c107b80 0x7ff00c19ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:51.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.394+0000 7ff00b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff00c107b80 0x7ff00c19ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38866/0 (socket says 172.21.15.43:38866) 2024-09-19T05:04:51.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.394+0000 7ff00b400640 1 -- 172.21.15.43:0/585103590 learned_addr learned my addr 172.21.15.43:0/585103590 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:51.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.394+0000 7ff00aa00640 1 --2- 172.21.15.43:0/585103590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff00c101ae0 0x7ff00c19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:51.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.394+0000 7ff003e00640 1 --2- 172.21.15.43:0/585103590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff00c102480 0x7ff00c19aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:51.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.394+0000 7ff00aa00640 1 -- 172.21.15.43:0/585103590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff00c102480 msgr2=0x7ff00c19aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:51.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.394+0000 7ff00aa00640 1 --2- 172.21.15.43:0/585103590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff00c102480 0x7ff00c19aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.394+0000 7ff00aa00640 1 -- 172.21.15.43:0/585103590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff00c107b80 msgr2=0x7ff00c19ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:51.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.394+0000 7ff00aa00640 1 --2- 172.21.15.43:0/585103590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff00c107b80 0x7ff00c19ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.394+0000 7ff00aa00640 1 -- 172.21.15.43:0/585103590 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff00c19f590 con 0x7ff00c101ae0 2024-09-19T05:04:51.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.394+0000 7ff00b400640 1 --2- 172.21.15.43:0/585103590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff00c107b80 0x7ff00c19ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:04:51.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.395+0000 7ff00aa00640 1 --2- 172.21.15.43:0/585103590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff00c101ae0 0x7ff00c19a5a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7feffc02a940 tx=0x7feffc003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:51.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.395+0000 7ff003400640 1 -- 172.21.15.43:0/585103590 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feffc03f070 con 0x7ff00c101ae0 2024-09-19T05:04:51.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.395+0000 7ff011800640 1 -- 172.21.15.43:0/585103590 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff00c19f820 con 0x7ff00c101ae0 2024-09-19T05:04:51.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.395+0000 7ff003400640 1 -- 172.21.15.43:0/585103590 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feffc031c90 con 0x7ff00c101ae0 2024-09-19T05:04:51.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.395+0000 7ff003400640 1 -- 172.21.15.43:0/585103590 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feffc035030 con 0x7ff00c101ae0 2024-09-19T05:04:51.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.395+0000 7ff011800640 1 -- 172.21.15.43:0/585103590 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff00c1a7110 con 0x7ff00c101ae0 2024-09-19T05:04:51.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.396+0000 7ff011800640 1 -- 172.21.15.43:0/585103590 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff00c101f40 con 0x7ff00c101ae0 2024-09-19T05:04:51.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.398+0000 7ff003400640 1 -- 172.21.15.43:0/585103590 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7feffc042070 con 0x7ff00c101ae0 2024-09-19T05:04:51.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.399+0000 7ff003400640 1 --2- 172.21.15.43:0/585103590 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7feff00a3580 0x7feff00a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:51.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.399+0000 7ff003400640 1 -- 172.21.15.43:0/585103590 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7feffc0f0dd0 con 0x7ff00c101ae0 2024-09-19T05:04:51.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.400+0000 7ff003e00640 1 --2- 172.21.15.43:0/585103590 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7feff00a3580 0x7feff00a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:51.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.400+0000 7ff003e00640 1 --2- 172.21.15.43:0/585103590 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7feff00a3580 0x7feff00a5a40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7feff40095f0 tx=0x7feff4005f90 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:51.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.401+0000 7ff003400640 1 -- 172.21.15.43:0/585103590 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7feffc0bcf90 con 0x7ff00c101ae0 2024-09-19T05:04:51.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.615+0000 7ff011800640 1 -- 172.21.15.43:0/585103590 --> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7ff00c0ff4f0 con 0x7feff00a3580 2024-09-19T05:04:51.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.622+0000 7ff003400640 1 -- 172.21.15.43:0/585103590 <== mgr.25136 v2:172.21.15.43:6800/2120444093 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7ff00c0ff4f0 con 0x7feff00a3580 2024-09-19T05:04:51.623 INFO:teuthology.orchestra.run.smithi043.stdout:/ 1m 6m3h 2024-09-19T05:04:51.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.625+0000 7ff011800640 1 -- 172.21.15.43:0/585103590 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7feff00a3580 msgr2=0x7feff00a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:51.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.625+0000 7ff011800640 1 --2- 172.21.15.43:0/585103590 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7feff00a3580 0x7feff00a5a40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7feff40095f0 tx=0x7feff4005f90 comp rx=0 tx=0).stop 2024-09-19T05:04:51.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.625+0000 7ff011800640 1 -- 172.21.15.43:0/585103590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff00c101ae0 msgr2=0x7ff00c19a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:51.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.625+0000 7ff011800640 1 --2- 172.21.15.43:0/585103590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff00c101ae0 0x7ff00c19a5a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7feffc02a940 tx=0x7feffc003b20 comp rx=0 tx=0).stop 2024-09-19T05:04:51.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.625+0000 7ff011800640 1 -- 172.21.15.43:0/585103590 shutdown_connections 2024-09-19T05:04:51.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.625+0000 7ff011800640 1 --2- 172.21.15.43:0/585103590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff00c107b80 0x7ff00c19ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.625+0000 7ff011800640 1 --2- 172.21.15.43:0/585103590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff00c102480 0x7ff00c19aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.625+0000 7ff011800640 1 --2- 172.21.15.43:0/585103590 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7feff00a3580 0x7feff00a5a40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.625+0000 7ff011800640 1 --2- 172.21.15.43:0/585103590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff00c101ae0 0x7ff00c19a5a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.625+0000 7ff011800640 1 -- 172.21.15.43:0/585103590 >> 172.21.15.43:0/585103590 conn(0x7ff00c0fd7e0 msgr2=0x7ff00c0fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:51.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.625+0000 7ff011800640 1 -- 172.21.15.43:0/585103590 shutdown_connections 2024-09-19T05:04:51.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.625+0000 7ff011800640 1 -- 172.21.15.43:0/585103590 wait complete. 2024-09-19T05:04:51.643 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-19T05:04:51.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.646+0000 7f1858800640 1 -- 172.21.15.43:0/1448977011 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18501049c0 msgr2=0x7f1850108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:51.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.646+0000 7f1858800640 1 --2- 172.21.15.43:0/1448977011 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18501049c0 0x7f1850108a10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1840009860 tx=0x7f184002a340 comp rx=0 tx=0).stop 2024-09-19T05:04:51.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.647+0000 7f1858800640 1 -- 172.21.15.43:0/1448977011 shutdown_connections 2024-09-19T05:04:51.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.647+0000 7f1858800640 1 --2- 172.21.15.43:0/1448977011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1850109140 0x7f1850110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.647+0000 7f1858800640 1 --2- 172.21.15.43:0/1448977011 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18501049c0 0x7f1850108a10 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.647+0000 7f1858800640 1 --2- 172.21.15.43:0/1448977011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1850104010 0x7f18501043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.647+0000 7f1858800640 1 -- 172.21.15.43:0/1448977011 >> 172.21.15.43:0/1448977011 conn(0x7f18500ffa50 msgr2=0x7f1850101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:51.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.647+0000 7f1858800640 1 -- 172.21.15.43:0/1448977011 shutdown_connections 2024-09-19T05:04:51.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.647+0000 7f1858800640 1 -- 172.21.15.43:0/1448977011 wait complete. 2024-09-19T05:04:51.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.648+0000 7f1858800640 1 Processor -- start 2024-09-19T05:04:51.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.648+0000 7f1858800640 1 -- start start 2024-09-19T05:04:51.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.648+0000 7f1858800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1850104010 0x7f185019a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:51.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.648+0000 7f1858800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18501049c0 0x7f185019ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:51.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.648+0000 7f1858800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1850109140 0x7f185019eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:51.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.648+0000 7f1858800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1850114160 con 0x7f1850104010 2024-09-19T05:04:51.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.648+0000 7f1858800640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1850113fe0 con 0x7f1850109140 2024-09-19T05:04:51.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.648+0000 7f1858800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f18501142e0 con 0x7f18501049c0 2024-09-19T05:04:51.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.649+0000 7f1855e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1850104010 0x7f185019a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:51.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.649+0000 7f1855e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1850104010 0x7f185019a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38888/0 (socket says 172.21.15.43:38888) 2024-09-19T05:04:51.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.649+0000 7f1855e00640 1 -- 172.21.15.43:0/4114087695 learned_addr learned my addr 172.21.15.43:0/4114087695 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:51.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.649+0000 7f1855400640 1 --2- 172.21.15.43:0/4114087695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18501049c0 0x7f185019ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:51.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.649+0000 7f1856800640 1 --2- 172.21.15.43:0/4114087695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1850109140 0x7f185019eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:51.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.649+0000 7f1855e00640 1 -- 172.21.15.43:0/4114087695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18501049c0 msgr2=0x7f185019ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:51.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.649+0000 7f1855e00640 1 --2- 172.21.15.43:0/4114087695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18501049c0 0x7f185019ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.649+0000 7f1855e00640 1 -- 172.21.15.43:0/4114087695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1850109140 msgr2=0x7f185019eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:51.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.649+0000 7f1855e00640 1 --2- 172.21.15.43:0/4114087695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1850109140 0x7f185019eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.649+0000 7f1855e00640 1 -- 172.21.15.43:0/4114087695 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f185019f6d0 con 0x7f1850104010 2024-09-19T05:04:51.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.649+0000 7f1855e00640 1 --2- 172.21.15.43:0/4114087695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1850104010 0x7f185019a720 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f184400e9f0 tx=0x7f184400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:51.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.650+0000 7f184aa00640 1 -- 172.21.15.43:0/4114087695 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184400cd80 con 0x7f1850104010 2024-09-19T05:04:51.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.650+0000 7f184aa00640 1 -- 172.21.15.43:0/4114087695 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1844005500 con 0x7f1850104010 2024-09-19T05:04:51.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.650+0000 7f1858800640 1 -- 172.21.15.43:0/4114087695 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f185019f9c0 con 0x7f1850104010 2024-09-19T05:04:51.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.650+0000 7f184aa00640 1 -- 172.21.15.43:0/4114087695 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1844010690 con 0x7f1850104010 2024-09-19T05:04:51.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.650+0000 7f1858800640 1 -- 172.21.15.43:0/4114087695 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18501a72a0 con 0x7f1850104010 2024-09-19T05:04:51.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.653+0000 7f185b2a6740 1 Processor -- start 2024-09-19T05:04:51.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.653+0000 7f184aa00640 1 -- 172.21.15.43:0/4114087695 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f18440046c0 con 0x7f1850104010 2024-09-19T05:04:51.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.653+0000 7f185b2a6740 1 -- start start 2024-09-19T05:04:51.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.654+0000 7f185b2a6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be0d701b90 0x55be0d701f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:51.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.654+0000 7f185b2a6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be0d7024b0 0x55be0d70c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:51.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.654+0000 7f185b2a6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be0d70cd70 0x55be0d70f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:51.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.654+0000 7f185b2a6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55be0d6683e0 con 0x55be0d70cd70 2024-09-19T05:04:51.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.654+0000 7f185b2a6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55be0d60fd80 con 0x55be0d7024b0 2024-09-19T05:04:51.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.654+0000 7f185b2a6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55be0d6643a0 con 0x55be0d701b90 2024-09-19T05:04:51.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.654+0000 7f1855e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be0d7024b0 0x55be0d70c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:51.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.654+0000 7f1856800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be0d701b90 0x55be0d701f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:51.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.654+0000 7f1855400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be0d70cd70 0x55be0d70f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:51.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.654+0000 7f1856800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be0d701b90 0x55be0d701f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:36316/0 (socket says 172.21.15.43:36316) 2024-09-19T05:04:51.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.654+0000 7f1856800640 1 -- 172.21.15.43:0/3381724255 learned_addr learned my addr 172.21.15.43:0/3381724255 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:51.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.654+0000 7f1855400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be0d70cd70 0x55be0d70f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38900/0 (socket says 172.21.15.43:38900) 2024-09-19T05:04:51.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.654+0000 7f1855e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be0d7024b0 0x55be0d70c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:47020/0 (socket says 172.21.15.43:47020) 2024-09-19T05:04:51.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.655+0000 7f1856800640 1 -- 172.21.15.43:0/3381724255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be0d7024b0 msgr2=0x55be0d70c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:51.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.655+0000 7f1856800640 1 --2- 172.21.15.43:0/3381724255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be0d7024b0 0x55be0d70c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.655+0000 7f1856800640 1 -- 172.21.15.43:0/3381724255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be0d70cd70 msgr2=0x55be0d70f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:51.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.655+0000 7f1856800640 1 --2- 172.21.15.43:0/3381724255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be0d70cd70 0x55be0d70f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.655+0000 7f1856800640 1 -- 172.21.15.43:0/3381724255 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be0d62eae0 con 0x55be0d701b90 2024-09-19T05:04:51.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.655+0000 7f1855400640 1 --2- 172.21.15.43:0/3381724255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be0d70cd70 0x55be0d70f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:04:51.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.655+0000 7f1856800640 1 --2- 172.21.15.43:0/3381724255 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be0d701b90 0x55be0d701f70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f184c0099f0 tx=0x7f184c0095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e9f0487794a6fcb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:51.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.655+0000 7f184aa00640 1 --2- 172.21.15.43:0/4114087695 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f18380a34d0 0x7f18380a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:51.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.655+0000 7f183f000640 1 -- 172.21.15.43:0/3381724255 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184c01d070 con 0x55be0d701b90 2024-09-19T05:04:51.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.656+0000 7f183f000640 1 -- 172.21.15.43:0/3381724255 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f184c003e40 con 0x55be0d701b90 2024-09-19T05:04:51.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.656+0000 7f183f000640 1 -- 172.21.15.43:0/3381724255 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184c00dba0 con 0x55be0d701b90 2024-09-19T05:04:51.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.656+0000 7f184aa00640 1 -- 172.21.15.43:0/4114087695 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f18440d03d0 con 0x7f1850104010 2024-09-19T05:04:51.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.656+0000 7f1855e00640 1 --2- 172.21.15.43:0/4114087695 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f18380a34d0 0x7f18380a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:51.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.656+0000 7f185b2a6740 1 -- 172.21.15.43:0/3381724255 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be0d701b90 msgr2=0x55be0d701f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:51.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.656+0000 7f185b2a6740 1 --2- 172.21.15.43:0/3381724255 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be0d701b90 0x55be0d701f70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f184c0099f0 tx=0x7f184c0095c0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.656+0000 7f1855e00640 1 --2- 172.21.15.43:0/4114087695 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f18380a34d0 0x7f18380a5990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f18440cda10 tx=0x7f1844098040 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:51.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.656+0000 7f185b2a6740 1 -- 172.21.15.43:0/3381724255 shutdown_connections 2024-09-19T05:04:51.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.657+0000 7f185b2a6740 1 --2- 172.21.15.43:0/3381724255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be0d70cd70 0x55be0d70f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.657+0000 7f185b2a6740 1 --2- 172.21.15.43:0/3381724255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be0d7024b0 0x55be0d70c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.657+0000 7f185b2a6740 1 --2- 172.21.15.43:0/3381724255 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be0d701b90 0x55be0d701f70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.657+0000 7f185b2a6740 1 -- 172.21.15.43:0/3381724255 >> 172.21.15.43:0/3381724255 conn(0x55be0d6fe120 msgr2=0x55be0d6fe500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:51.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.657+0000 7f185b2a6740 1 -- 172.21.15.43:0/3381724255 shutdown_connections 2024-09-19T05:04:51.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.657+0000 7f185b2a6740 1 -- 172.21.15.43:0/3381724255 wait complete. 2024-09-19T05:04:51.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.657+0000 7f185b2a6740 1 -- start start 2024-09-19T05:04:51.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.657+0000 7f185b2a6740 1 Processor -- start 2024-09-19T05:04:51.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.658+0000 7f185b2a6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be0d701b90 0x55be0d709d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:51.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.658+0000 7f185b2a6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be0d7024b0 0x55be0d70a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:51.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.658+0000 7f185b2a6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be0d70cd70 0x55be0d70a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:51.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.658+0000 7f185b2a6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55be0d70f9e0 con 0x55be0d701b90 2024-09-19T05:04:51.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.658+0000 7f185b2a6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55be0d6230f0 con 0x55be0d7024b0 2024-09-19T05:04:51.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.658+0000 7f185b2a6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55be0d70fb60 con 0x55be0d70cd70 2024-09-19T05:04:51.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.658+0000 7f185b2a6740 10 client.0 _ll_register_callbacks cb 0x55be0d6149e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:04:51.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.658+0000 7f1855400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be0d7024b0 0x55be0d70a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:51.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.658+0000 7f1856800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be0d701b90 0x55be0d709d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:51.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.658+0000 7f1855400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be0d7024b0 0x55be0d70a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:47026/0 (socket says 172.21.15.43:47026) 2024-09-19T05:04:51.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.658+0000 7f1855400640 1 -- 172.21.15.43:0/2687817210 learned_addr learned my addr 172.21.15.43:0/2687817210 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:51.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.658+0000 7f1856800640 1 --2- 172.21.15.43:0/2687817210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be0d70cd70 0x55be0d70a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:51.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.659+0000 7f1855400640 1 -- 172.21.15.43:0/2687817210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be0d70cd70 msgr2=0x55be0d70a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:51.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.659+0000 7f1855400640 1 --2- 172.21.15.43:0/2687817210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be0d70cd70 0x55be0d70a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.659+0000 7f1855400640 1 -- 172.21.15.43:0/2687817210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be0d701b90 msgr2=0x55be0d709d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:51.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.659+0000 7f1855400640 1 --2- 172.21.15.43:0/2687817210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be0d701b90 0x55be0d709d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.659+0000 7f1855400640 1 -- 172.21.15.43:0/2687817210 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be0d3d93c0 con 0x55be0d7024b0 2024-09-19T05:04:51.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.659+0000 7f1856800640 1 --2- 172.21.15.43:0/2687817210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be0d70cd70 0x55be0d70a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:04:51.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.659+0000 7f1855400640 1 --2- 172.21.15.43:0/2687817210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be0d7024b0 0x55be0d70a270 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f18400057c0 tx=0x7f1840030340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbb4d93c8d7432d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:51.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.659+0000 7f1833400640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:04:51.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.659+0000 7f1833400640 1 -- 172.21.15.43:0/2687817210 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1840037040 con 0x55be0d7024b0 2024-09-19T05:04:51.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.659+0000 7f1833400640 1 -- 172.21.15.43:0/2687817210 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f1840030550 con 0x55be0d7024b0 2024-09-19T05:04:51.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.659+0000 7f185b2a6740 1 -- 172.21.15.43:0/2687817210 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55be0d675950 con 0x55be0d7024b0 2024-09-19T05:04:51.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.660+0000 7f1833400640 1 -- 172.21.15.43:0/2687817210 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f184004c050 con 0x55be0d7024b0 2024-09-19T05:04:51.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.660+0000 7f185b2a6740 10 client.25328 fetch_fsmap learned FSMap version 29 2024-09-19T05:04:51.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.660+0000 7f185b2a6740 1 -- 172.21.15.43:0/2687817210 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55be0d71b750 con 0x55be0d7024b0 2024-09-19T05:04:51.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.660+0000 7f1833400640 1 -- 172.21.15.43:0/2687817210 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55be0d675950 con 0x55be0d7024b0 2024-09-19T05:04:51.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.660+0000 7f1833400640 1 -- 172.21.15.43:0/2687817210 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f184003d450 con 0x55be0d7024b0 2024-09-19T05:04:51.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.660+0000 7f185b2a6740 10 client.25328 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:04:51.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.660+0000 7f185b2a6740 10 client.25328 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:04:51.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.660+0000 7f185b2a6740 20 client.25328 populate_metadata read hostname 'smithi043' 2024-09-19T05:04:51.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.660+0000 7f185b2a6740 1 --2- 172.21.15.43:0/2687817210 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55be0d71cc10 0x55be0d71f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:51.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.660+0000 7f185b2a6740 4 client.25328 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:04:51.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.660+0000 7f185b2a6740 1 -- 172.21.15.43:0/2687817210 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be0d66fe60 con 0x55be0d71cc10 2024-09-19T05:04:51.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.661+0000 7f1856800640 1 --2- 172.21.15.43:0/2687817210 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55be0d71cc10 0x55be0d71f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:51.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.661+0000 7f1856800640 1 --2- 172.21.15.43:0/2687817210 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55be0d71cc10 0x55be0d71f060 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2de9cdf3ea2cdd6 server_cookie=f7c10f7f68c2139 in_seq=0 out_seq=0 2024-09-19T05:04:51.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.661+0000 7f1833400640 10 client.25328 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:04:51.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.662+0000 7f1833400640 1 -- 172.21.15.43:0/2687817210 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55be0d66fe60 con 0x55be0d71cc10 2024-09-19T05:04:51.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.662+0000 7f1833400640 10 client.25328 handle_command_reply: tid=0 2024-09-19T05:04:51.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.662+0000 7f185b2a6740 1 client.25328 shutdown 2024-09-19T05:04:51.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.662+0000 7f185b2a6740 1 -- 172.21.15.43:0/2687817210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be0d7024b0 msgr2=0x55be0d70a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:51.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.662+0000 7f185b2a6740 1 --2- 172.21.15.43:0/2687817210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be0d7024b0 0x55be0d70a270 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f18400057c0 tx=0x7f1840030340 comp rx=0 tx=0).stop 2024-09-19T05:04:51.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.662+0000 7f185b2a6740 1 -- 172.21.15.43:0/2687817210 shutdown_connections 2024-09-19T05:04:51.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.662+0000 7f185b2a6740 1 --2- 172.21.15.43:0/2687817210 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55be0d71cc10 0x55be0d71f060 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.662+0000 7f1833400640 0 client.25328 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:04:51.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.662+0000 7f185b2a6740 1 --2- 172.21.15.43:0/2687817210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be0d70cd70 0x55be0d70a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.662+0000 7f185b2a6740 1 --2- 172.21.15.43:0/2687817210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be0d7024b0 0x55be0d70a270 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.662+0000 7f185b2a6740 1 --2- 172.21.15.43:0/2687817210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be0d701b90 0x55be0d709d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.663+0000 7f185b2a6740 1 -- 172.21.15.43:0/2687817210 >> 172.21.15.43:0/2687817210 conn(0x55be0d6fe120 msgr2=0x55be0d703e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:51.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.663+0000 7f185b2a6740 1 -- 172.21.15.43:0/2687817210 shutdown_connections 2024-09-19T05:04:51.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.663+0000 7f185b2a6740 1 -- 172.21.15.43:0/2687817210 wait complete. 2024-09-19T05:04:51.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.663+0000 7f185b2a6740 20 client.25328 trim_cache size 0 max 16384 2024-09-19T05:04:51.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.681+0000 7f185b2a6740 1 Processor -- start 2024-09-19T05:04:51.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.681+0000 7f185b2a6740 1 -- start start 2024-09-19T05:04:51.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.681+0000 7f185b2a6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be0d701b90 0x55be0d7045f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:51.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.681+0000 7f185b2a6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be0d7024b0 0x55be0d6fd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:51.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.682+0000 7f185b2a6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be0d70cd70 0x55be0d724930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:51.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.682+0000 7f185b2a6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f18501afa80 con 0x55be0d7024b0 2024-09-19T05:04:51.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.682+0000 7f185b2a6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f18501b15c0 con 0x55be0d701b90 2024-09-19T05:04:51.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.682+0000 7f185b2a6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f18501b0150 con 0x55be0d70cd70 2024-09-19T05:04:51.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.682+0000 7f1856800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be0d70cd70 0x55be0d724930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:51.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.682+0000 7f1855400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be0d7024b0 0x55be0d6fd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:51.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.682+0000 7f1856800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be0d70cd70 0x55be0d724930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:36330/0 (socket says 172.21.15.43:36330) 2024-09-19T05:04:51.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.682+0000 7f1855400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be0d7024b0 0x55be0d6fd0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38914/0 (socket says 172.21.15.43:38914) 2024-09-19T05:04:51.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.682+0000 7f1856800640 1 -- 172.21.15.43:0/2974116203 learned_addr learned my addr 172.21.15.43:0/2974116203 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:51.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.682+0000 7f1856800640 1 --2- 172.21.15.43:0/2974116203 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be0d701b90 0x55be0d7045f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:51.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.682+0000 7f1855400640 1 -- 172.21.15.43:0/2974116203 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be0d70cd70 msgr2=0x55be0d724930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:51.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.682+0000 7f1855400640 1 --2- 172.21.15.43:0/2974116203 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be0d70cd70 0x55be0d724930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.682+0000 7f1855400640 1 -- 172.21.15.43:0/2974116203 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be0d701b90 msgr2=0x55be0d7045f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:51.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.682+0000 7f1855400640 1 --2- 172.21.15.43:0/2974116203 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be0d701b90 0x55be0d7045f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.682+0000 7f1855400640 1 -- 172.21.15.43:0/2974116203 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be0d71f6e0 con 0x55be0d7024b0 2024-09-19T05:04:51.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.683+0000 7f1855400640 1 --2- 172.21.15.43:0/2974116203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be0d7024b0 0x55be0d6fd0d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f184003d870 tx=0x7f184002a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17efd5b325aefa98 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:51.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.683+0000 7f183e600640 1 -- 172.21.15.43:0/2974116203 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1840055070 con 0x55be0d7024b0 2024-09-19T05:04:51.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.683+0000 7f183e600640 1 -- 172.21.15.43:0/2974116203 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f184003dae0 con 0x55be0d7024b0 2024-09-19T05:04:51.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.683+0000 7f183e600640 1 -- 172.21.15.43:0/2974116203 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1840037070 con 0x55be0d7024b0 2024-09-19T05:04:51.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.684+0000 7f185b2a6740 1 -- 172.21.15.43:0/2974116203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be0d7024b0 msgr2=0x55be0d6fd0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:51.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.684+0000 7f185b2a6740 1 --2- 172.21.15.43:0/2974116203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be0d7024b0 0x55be0d6fd0d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f184003d870 tx=0x7f184002a340 comp rx=0 tx=0).stop 2024-09-19T05:04:51.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.684+0000 7f185b2a6740 1 -- 172.21.15.43:0/2974116203 shutdown_connections 2024-09-19T05:04:51.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.684+0000 7f185b2a6740 1 --2- 172.21.15.43:0/2974116203 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be0d70cd70 0x55be0d724930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.684+0000 7f185b2a6740 1 --2- 172.21.15.43:0/2974116203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be0d7024b0 0x55be0d6fd0d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.684+0000 7f185b2a6740 1 --2- 172.21.15.43:0/2974116203 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be0d701b90 0x55be0d7045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.684+0000 7f185b2a6740 1 -- 172.21.15.43:0/2974116203 >> 172.21.15.43:0/2974116203 conn(0x55be0d6fe120 msgr2=0x55be0d6fc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:51.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.684+0000 7f185b2a6740 1 -- 172.21.15.43:0/2974116203 shutdown_connections 2024-09-19T05:04:51.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.684+0000 7f185b2a6740 1 -- 172.21.15.43:0/2974116203 wait complete. 2024-09-19T05:04:51.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.685+0000 7f185b2a6740 1 -- start start 2024-09-19T05:04:51.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.685+0000 7f185b2a6740 1 Processor -- start 2024-09-19T05:04:51.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.686+0000 7f185b2a6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be0d701b90 0x55be0d710a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:51.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.686+0000 7f185b2a6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be0d7024b0 0x55be0d710f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:51.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.686+0000 7f185b2a6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be0d70cd70 0x55be0d7114a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:51.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.686+0000 7f185b2a6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f18501b0ef0 con 0x55be0d701b90 2024-09-19T05:04:51.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.686+0000 7f185b2a6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f18501b0820 con 0x55be0d70cd70 2024-09-19T05:04:51.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.686+0000 7f185b2a6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f18501ada60 con 0x55be0d7024b0 2024-09-19T05:04:51.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.686+0000 7f185b2a6740 10 client.0 _ll_register_callbacks cb 0x7f1840009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:04:51.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.686+0000 7f1855400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be0d7024b0 0x55be0d710f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:51.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.686+0000 7f1855400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be0d7024b0 0x55be0d710f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:36336/0 (socket says 172.21.15.43:36336) 2024-09-19T05:04:51.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.686+0000 7f1856800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be0d701b90 0x55be0d710a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:51.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.686+0000 7f1855400640 1 -- 172.21.15.43:0/504249219 learned_addr learned my addr 172.21.15.43:0/504249219 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:04:51.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.686+0000 7f1856800640 1 --2- 172.21.15.43:0/504249219 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be0d70cd70 0x55be0d7114a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:51.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.686+0000 7f1856800640 1 -- 172.21.15.43:0/504249219 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be0d7024b0 msgr2=0x55be0d710f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:51.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.686+0000 7f1856800640 1 --2- 172.21.15.43:0/504249219 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be0d7024b0 0x55be0d710f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.686+0000 7f1856800640 1 -- 172.21.15.43:0/504249219 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be0d70cd70 msgr2=0x55be0d7114a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:04:51.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.686+0000 7f1856800640 1 --2- 172.21.15.43:0/504249219 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be0d70cd70 0x55be0d7114a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.686+0000 7f1856800640 1 -- 172.21.15.43:0/504249219 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be0d786900 con 0x55be0d701b90 2024-09-19T05:04:51.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.686+0000 7f1855400640 1 --2- 172.21.15.43:0/504249219 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be0d7024b0 0x55be0d710f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:04:51.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.687+0000 7f1856800640 1 --2- 172.21.15.43:0/504249219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be0d701b90 0x55be0d710a20 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f184c009fd0 tx=0x7f184c005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c57741b51b1f0c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:04:51.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.687+0000 7f1833400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:04:51.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.687+0000 7f1833400640 1 -- 172.21.15.43:0/504249219 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184c039060 con 0x55be0d701b90 2024-09-19T05:04:51.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.687+0000 7f185b2a6740 1 -- 172.21.15.43:0/504249219 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f184003dae0 con 0x55be0d701b90 2024-09-19T05:04:51.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.687+0000 7f1833400640 1 -- 172.21.15.43:0/504249219 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f184c00de00 con 0x55be0d701b90 2024-09-19T05:04:51.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.687+0000 7f1833400640 1 -- 172.21.15.43:0/504249219 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f184c01d040 con 0x55be0d701b90 2024-09-19T05:04:51.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.687+0000 7f185b2a6740 10 client.15345 fetch_fsmap learned FSMap version 29 2024-09-19T05:04:51.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.687+0000 7f185b2a6740 1 -- 172.21.15.43:0/504249219 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55be0d787bc0 con 0x55be0d701b90 2024-09-19T05:04:51.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.688+0000 7f1833400640 1 -- 172.21.15.43:0/504249219 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f184003dae0 con 0x55be0d701b90 2024-09-19T05:04:51.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.688+0000 7f1833400640 1 -- 172.21.15.43:0/504249219 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f184c00b040 con 0x55be0d701b90 2024-09-19T05:04:51.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.688+0000 7f185b2a6740 10 client.15345 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:04:51.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.688+0000 7f185b2a6740 10 client.15345 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:04:51.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.688+0000 7f185b2a6740 20 client.15345 populate_metadata read hostname 'smithi043' 2024-09-19T05:04:51.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.688+0000 7f185b2a6740 1 --2- 172.21.15.43:0/504249219 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55be0d787fc0 0x55be0d78a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:04:51.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.688+0000 7f185b2a6740 4 client.15345 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:04:51.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.688+0000 7f185b2a6740 1 -- 172.21.15.43:0/504249219 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55be0d5ec0c0 con 0x55be0d787fc0 2024-09-19T05:04:51.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.688+0000 7f1855400640 1 --2- 172.21.15.43:0/504249219 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55be0d787fc0 0x55be0d78a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:04:51.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.689+0000 7f1855400640 1 --2- 172.21.15.43:0/504249219 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55be0d787fc0 0x55be0d78a3b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6cffb8225d4f1e0 server_cookie=d8e81da17a45a42a in_seq=0 out_seq=0 2024-09-19T05:04:51.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.689+0000 7f1833400640 10 client.15345 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:04:51.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.689+0000 7f1833400640 1 -- 172.21.15.43:0/504249219 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55be0d5ec0c0 con 0x55be0d787fc0 2024-09-19T05:04:51.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.689+0000 7f1833400640 10 client.15345 handle_command_reply: tid=0 2024-09-19T05:04:51.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.689+0000 7f185b2a6740 1 client.15345 shutdown 2024-09-19T05:04:51.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.689+0000 7f185b2a6740 1 -- 172.21.15.43:0/504249219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be0d701b90 msgr2=0x55be0d710a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:51.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.689+0000 7f185b2a6740 1 --2- 172.21.15.43:0/504249219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be0d701b90 0x55be0d710a20 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f184c009fd0 tx=0x7f184c005ad0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.689+0000 7f185b2a6740 1 -- 172.21.15.43:0/504249219 shutdown_connections 2024-09-19T05:04:51.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.689+0000 7f185b2a6740 1 --2- 172.21.15.43:0/504249219 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55be0d787fc0 0x55be0d78a3b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.689+0000 7f185b2a6740 1 --2- 172.21.15.43:0/504249219 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be0d70cd70 0x55be0d7114a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.689+0000 7f185b2a6740 1 --2- 172.21.15.43:0/504249219 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be0d7024b0 0x55be0d710f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.689+0000 7f185b2a6740 1 --2- 172.21.15.43:0/504249219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be0d701b90 0x55be0d710a20 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.689+0000 7f1833400640 0 client.15345 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:04:51.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.689+0000 7f185b2a6740 1 -- 172.21.15.43:0/504249219 >> 172.21.15.43:0/504249219 conn(0x55be0d6fe120 msgr2=0x55be0d7081d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:51.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.689+0000 7f185b2a6740 1 -- 172.21.15.43:0/504249219 shutdown_connections 2024-09-19T05:04:51.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.690+0000 7f185b2a6740 1 -- 172.21.15.43:0/504249219 wait complete. 2024-09-19T05:04:51.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.690+0000 7f185b2a6740 20 client.15345 trim_cache size 0 max 16384 2024-09-19T05:04:51.694 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:04:51.694 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T05:04:51.694 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-09-19T05:04:51.694 INFO:teuthology.orchestra.run.smithi043.stdout: "bd603c11-f503-4f2e-bf95-3ef8de152522": { 2024-09-19T05:04:51.694 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-09-19T05:04:51.694 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "bd603c11-f503-4f2e-bf95-3ef8de152522", 2024-09-19T05:04:51.694 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-09-19T05:04:51.695 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:04:51.695 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:04:51.695 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:04:51.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.690+0000 7f1858800640 1 -- 172.21.15.43:0/4114087695 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f18380a34d0 msgr2=0x7f18380a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:51.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.690+0000 7f1858800640 1 --2- 172.21.15.43:0/4114087695 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f18380a34d0 0x7f18380a5990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f18440cda10 tx=0x7f1844098040 comp rx=0 tx=0).stop 2024-09-19T05:04:51.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.690+0000 7f1858800640 1 -- 172.21.15.43:0/4114087695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1850104010 msgr2=0x7f185019a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:04:51.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.690+0000 7f1858800640 1 --2- 172.21.15.43:0/4114087695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1850104010 0x7f185019a720 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f184400e9f0 tx=0x7f184400edf0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.691+0000 7f1858800640 1 -- 172.21.15.43:0/4114087695 shutdown_connections 2024-09-19T05:04:51.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.691+0000 7f1858800640 1 --2- 172.21.15.43:0/4114087695 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f18380a34d0 0x7f18380a5990 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.691+0000 7f1858800640 1 --2- 172.21.15.43:0/4114087695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1850109140 0x7f185019eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.691+0000 7f1858800640 1 --2- 172.21.15.43:0/4114087695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18501049c0 0x7f185019ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.691+0000 7f1858800640 1 --2- 172.21.15.43:0/4114087695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1850104010 0x7f185019a720 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:04:51.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.691+0000 7f1858800640 1 -- 172.21.15.43:0/4114087695 >> 172.21.15.43:0/4114087695 conn(0x7f18500ffa50 msgr2=0x7f1850100df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:04:51.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.691+0000 7f1858800640 1 -- 172.21.15.43:0/4114087695 shutdown_connections 2024-09-19T05:04:51.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:04:51.691+0000 7f1858800640 1 -- 172.21.15.43:0/4114087695 wait complete. 2024-09-19T05:04:51.714 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-19T05:04:51.726 INFO:teuthology.run_tasks:Running task workunit... 2024-09-19T05:04:51.773 INFO:tasks.workunit:Pulling workunits from ref 6fad5dd73c93545e52f57f681e140a9c3529c11e 2024-09-19T05:04:51.774 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-19T05:04:51.774 INFO:tasks.workunit:timeout=3h 2024-09-19T05:04:51.774 INFO:tasks.workunit:cleanup=True 2024-09-19T05:04:51.774 DEBUG:teuthology.orchestra.run.smithi043:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-19T05:04:51.811 INFO:teuthology.orchestra.run.smithi043.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-19T05:04:51.811 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-19T05:04:51.811 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-19T05:04:51.811 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T05:04:51.811 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-19T05:04:51.811 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-19 05:02:48.775204199 +0000 2024-09-19T05:04:51.811 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-19 05:02:48.775204199 +0000 2024-09-19T05:04:51.811 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-19 05:04:37.121510465 +0000 2024-09-19T05:04:51.812 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-09-19 05:02:48.775204199 +0000 2024-09-19T05:04:51.812 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-19T05:04:51.812 DEBUG:teuthology.orchestra.run.smithi043:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-19T05:04:51.919 DEBUG:teuthology.orchestra.run.smithi043:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 6fad5dd73c93545e52f57f681e140a9c3529c11e 2024-09-19T05:04:51.957 INFO:tasks.workunit.client.0.smithi043.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-19T05:04:52.225 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:51 smithi043 ceph-mon[30800]: from='client.25298 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:52.225 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:51 smithi043 ceph-mon[30800]: pgmap v9: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 932 B/s wr, 21 op/s 2024-09-19T05:04:52.225 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:51 smithi043 ceph-mon[30800]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:52.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:51 smithi096 ceph-mon[35085]: from='client.25298 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:52.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:51 smithi096 ceph-mon[35085]: pgmap v9: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 932 B/s wr, 21 op/s 2024-09-19T05:04:52.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:51 smithi096 ceph-mon[35085]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:52.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:51 smithi151 ceph-mon[35425]: from='client.25298 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:52.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:51 smithi151 ceph-mon[35425]: pgmap v9: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 932 B/s wr, 21 op/s 2024-09-19T05:04:52.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:51 smithi151 ceph-mon[35425]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:52.585 INFO:tasks.cephfs.filesystem:scrub status for tag:bd603c11-f503-4f2e-bf95-3ef8de152522 - {'path': '/', 'tag': 'bd603c11-f503-4f2e-bf95-3ef8de152522', 'options': 'recursive,force'} 2024-09-19T05:04:53.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:52 smithi043 ceph-mon[30800]: from='client.25310 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:53.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:52 smithi043 ceph-mon[30800]: from='client.25316 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:53.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:52 smithi151 ceph-mon[35425]: from='client.25310 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:53.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:52 smithi151 ceph-mon[35425]: from='client.25316 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:53.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:53 smithi096 ceph-mon[35085]: from='client.25310 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:53.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:53 smithi096 ceph-mon[35085]: from='client.25316 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:04:54.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:54 smithi043 ceph-mon[30800]: pgmap v10: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 853 B/s wr, 19 op/s 2024-09-19T05:04:54.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:54 smithi151 ceph-mon[35425]: pgmap v10: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 853 B/s wr, 19 op/s 2024-09-19T05:04:54.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:54 smithi096 ceph-mon[35085]: pgmap v10: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 853 B/s wr, 19 op/s 2024-09-19T05:04:56.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:56 smithi043 ceph-mon[30800]: pgmap v11: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 853 B/s wr, 19 op/s 2024-09-19T05:04:56.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:56 smithi151 ceph-mon[35425]: pgmap v11: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 853 B/s wr, 19 op/s 2024-09-19T05:04:56.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:56 smithi096 ceph-mon[35085]: pgmap v11: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 853 B/s wr, 19 op/s 2024-09-19T05:04:58.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:04:58 smithi043 ceph-mon[30800]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 78 KiB/s wr, 56 op/s 2024-09-19T05:04:58.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:04:58 smithi151 ceph-mon[35425]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 78 KiB/s wr, 56 op/s 2024-09-19T05:04:58.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:04:58 smithi096 ceph-mon[35085]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 78 KiB/s wr, 56 op/s 2024-09-19T05:05:00.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:00 smithi043 ceph-mon[30800]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-09-19T05:05:00.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:00 smithi151 ceph-mon[35425]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-09-19T05:05:00.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:00 smithi096 ceph-mon[35085]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-09-19T05:05:01.031 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:05:00 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:05:00.803+0000 7f81c4400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:05:01.031 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:05:00 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:05:00.803+0000 7f81c4400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:05:01.031 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:05:00 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:05:00.803+0000 7f81c4400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:05:01.031 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:05:00 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:05:00.803+0000 7f81c4400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:05:01.031 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:05:00 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:05:00.803+0000 7f81c4400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T05:05:02.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:02 smithi043 ceph-mon[30800]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-09-19T05:05:02.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:02 smithi151 ceph-mon[35425]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-09-19T05:05:02.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:02 smithi096 ceph-mon[35085]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-09-19T05:05:03.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:03 smithi043 ceph-mon[30800]: scrub summary: idle 2024-09-19T05:05:03.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:03 smithi151 ceph-mon[35425]: scrub summary: idle 2024-09-19T05:05:03.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:03 smithi096 ceph-mon[35085]: scrub summary: idle 2024-09-19T05:05:04.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:04 smithi043 ceph-mon[30800]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-09-19T05:05:04.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:04 smithi043 ceph-mon[30800]: mgrmap e25: x(active, since 24s), standbys: z, y 2024-09-19T05:05:04.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:04 smithi151 ceph-mon[35425]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-09-19T05:05:04.455 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:04 smithi151 ceph-mon[35425]: mgrmap e25: x(active, since 24s), standbys: z, y 2024-09-19T05:05:04.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:04 smithi096 ceph-mon[35085]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-09-19T05:05:04.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:04 smithi096 ceph-mon[35085]: mgrmap e25: x(active, since 24s), standbys: z, y 2024-09-19T05:05:06.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:06 smithi043 ceph-mon[30800]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 88 KiB/s wr, 43 op/s 2024-09-19T05:05:06.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:06 smithi151 ceph-mon[35425]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 88 KiB/s wr, 43 op/s 2024-09-19T05:05:06.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:06 smithi096 ceph-mon[35085]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 88 KiB/s wr, 43 op/s 2024-09-19T05:05:08.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:08 smithi151 ceph-mon[35425]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 100 KiB/s wr, 60 op/s 2024-09-19T05:05:08.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:08 smithi096 ceph-mon[35085]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 100 KiB/s wr, 60 op/s 2024-09-19T05:05:08.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:08 smithi043 ceph-mon[30800]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 100 KiB/s wr, 60 op/s 2024-09-19T05:05:10.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:10 smithi151 ceph-mon[35425]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-19T05:05:10.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:10 smithi096 ceph-mon[35085]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-19T05:05:10.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:10 smithi043 ceph-mon[30800]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-19T05:05:12.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:12 smithi151 ceph-mon[35425]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-19T05:05:12.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:12 smithi096 ceph-mon[35085]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-19T05:05:12.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:12 smithi043 ceph-mon[30800]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-19T05:05:14.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:14 smithi151 ceph-mon[35425]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-19T05:05:14.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:14 smithi096 ceph-mon[35085]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-19T05:05:14.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:14 smithi043 ceph-mon[30800]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-19T05:05:16.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:16 smithi151 ceph-mon[35425]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-09-19T05:05:16.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:16 smithi096 ceph-mon[35085]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-09-19T05:05:16.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:16 smithi043 ceph-mon[30800]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-09-19T05:05:18.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:18 smithi151 ceph-mon[35425]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 25 op/s 2024-09-19T05:05:18.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:18 smithi096 ceph-mon[35085]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 25 op/s 2024-09-19T05:05:18.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:18 smithi043 ceph-mon[30800]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 25 op/s 2024-09-19T05:05:20.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:20 smithi151 ceph-mon[35425]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-19T05:05:20.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:20 smithi096 ceph-mon[35085]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-19T05:05:20.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:20 smithi043 ceph-mon[30800]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-19T05:05:22.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:22 smithi151 ceph-mon[35425]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-19T05:05:22.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:22 smithi096 ceph-mon[35085]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-19T05:05:22.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:22 smithi043 ceph-mon[30800]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-19T05:05:22.586 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:05:22.998 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:05:24.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.629+0000 7f85af600640 1 -- 172.21.15.43:0/355855992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a8104010 msgr2=0x7f85a81043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:24.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.629+0000 7f85af600640 1 --2- 172.21.15.43:0/355855992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a8104010 0x7f85a81043f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f859c009900 tx=0x7f859c02a3f0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.629+0000 7f85af600640 1 -- 172.21.15.43:0/355855992 shutdown_connections 2024-09-19T05:05:24.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.629+0000 7f85af600640 1 --2- 172.21.15.43:0/355855992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85a8109140 0x7f85a8110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.629+0000 7f85af600640 1 --2- 172.21.15.43:0/355855992 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85a81049c0 0x7f85a8108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.629+0000 7f85af600640 1 --2- 172.21.15.43:0/355855992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a8104010 0x7f85a81043f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.629+0000 7f85af600640 1 -- 172.21.15.43:0/355855992 >> 172.21.15.43:0/355855992 conn(0x7f85a80ffa50 msgr2=0x7f85a8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:24.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.629+0000 7f85af600640 1 -- 172.21.15.43:0/355855992 shutdown_connections 2024-09-19T05:05:24.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.629+0000 7f85af600640 1 -- 172.21.15.43:0/355855992 wait complete. 2024-09-19T05:05:24.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.630+0000 7f85af600640 1 Processor -- start 2024-09-19T05:05:24.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.630+0000 7f85af600640 1 -- start start 2024-09-19T05:05:24.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.630+0000 7f85af600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85a8104010 0x7f85a819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:24.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.630+0000 7f85af600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a81049c0 0x7f85a819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:24.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.630+0000 7f85af600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85a8109140 0x7f85a819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:24.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.630+0000 7f85af600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f85a8114160 con 0x7f85a81049c0 2024-09-19T05:05:24.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.630+0000 7f85af600640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f85a8113fe0 con 0x7f85a8109140 2024-09-19T05:05:24.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.630+0000 7f85af600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f85a81142e0 con 0x7f85a8104010 2024-09-19T05:05:24.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.631+0000 7f85a7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a81049c0 0x7f85a819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:24.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.631+0000 7f85a7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a81049c0 0x7f85a819ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37300/0 (socket says 172.21.15.43:37300) 2024-09-19T05:05:24.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.631+0000 7f85a7e00640 1 -- 172.21.15.43:0/1957900906 learned_addr learned my addr 172.21.15.43:0/1957900906 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:05:24.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.631+0000 7f85acc00640 1 --2- 172.21.15.43:0/1957900906 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85a8104010 0x7f85a819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:24.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.631+0000 7f85ad600640 1 --2- 172.21.15.43:0/1957900906 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85a8109140 0x7f85a819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:24.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.631+0000 7f85a7e00640 1 -- 172.21.15.43:0/1957900906 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85a8104010 msgr2=0x7f85a819a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:24.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.631+0000 7f85a7e00640 1 --2- 172.21.15.43:0/1957900906 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85a8104010 0x7f85a819a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.631+0000 7f85a7e00640 1 -- 172.21.15.43:0/1957900906 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85a8109140 msgr2=0x7f85a819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:24.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.631+0000 7f85a7e00640 1 --2- 172.21.15.43:0/1957900906 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85a8109140 0x7f85a819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.631+0000 7f85a7e00640 1 -- 172.21.15.43:0/1957900906 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a819f5d0 con 0x7f85a81049c0 2024-09-19T05:05:24.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.632+0000 7f85a7e00640 1 --2- 172.21.15.43:0/1957900906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a81049c0 0x7f85a819ab60 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f859400e9f0 tx=0x7f859400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:24.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.632+0000 7f85a5600640 1 -- 172.21.15.43:0/1957900906 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859400cd80 con 0x7f85a81049c0 2024-09-19T05:05:24.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.632+0000 7f85a5600640 1 -- 172.21.15.43:0/1957900906 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8594005500 con 0x7f85a81049c0 2024-09-19T05:05:24.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.632+0000 7f85af600640 1 -- 172.21.15.43:0/1957900906 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85a819f8c0 con 0x7f85a81049c0 2024-09-19T05:05:24.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.632+0000 7f85a5600640 1 -- 172.21.15.43:0/1957900906 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8594010690 con 0x7f85a81049c0 2024-09-19T05:05:24.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.632+0000 7f85af600640 1 -- 172.21.15.43:0/1957900906 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85a81a71a0 con 0x7f85a81049c0 2024-09-19T05:05:24.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.635+0000 7f85b20ce740 1 Processor -- start 2024-09-19T05:05:24.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.636+0000 7f85b20ce740 1 -- start start 2024-09-19T05:05:24.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.636+0000 7f85a5600640 1 -- 172.21.15.43:0/1957900906 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f85940046c0 con 0x7f85a81049c0 2024-09-19T05:05:24.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.636+0000 7f85b20ce740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bd900db90 0x558bd900df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:24.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.636+0000 7f85b20ce740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bd900e4b0 0x558bd9018830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:24.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.636+0000 7f85b20ce740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bd9018d70 0x558bd901b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:24.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.636+0000 7f85b20ce740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558bd8f743e0 con 0x558bd900db90 2024-09-19T05:05:24.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.636+0000 7f85b20ce740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558bd8f1bd80 con 0x558bd900e4b0 2024-09-19T05:05:24.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.636+0000 7f85b20ce740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558bd8f703a0 con 0x558bd9018d70 2024-09-19T05:05:24.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.636+0000 7f85ad600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bd900db90 0x558bd900df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:24.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.636+0000 7f85ad600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bd900db90 0x558bd900df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37302/0 (socket says 172.21.15.43:37302) 2024-09-19T05:05:24.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.636+0000 7f85ad600640 1 -- 172.21.15.43:0/85374434 learned_addr learned my addr 172.21.15.43:0/85374434 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:05:24.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.636+0000 7f85acc00640 1 --2- 172.21.15.43:0/85374434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bd900e4b0 0x558bd9018830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:24.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.637+0000 7f85a7e00640 1 --2- 172.21.15.43:0/85374434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bd9018d70 0x558bd901b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:24.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.637+0000 7f85ad600640 1 -- 172.21.15.43:0/85374434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bd9018d70 msgr2=0x558bd901b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:24.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.637+0000 7f85ad600640 1 --2- 172.21.15.43:0/85374434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bd9018d70 0x558bd901b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.637+0000 7f85ad600640 1 -- 172.21.15.43:0/85374434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bd900e4b0 msgr2=0x558bd9018830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:24.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.637+0000 7f85ad600640 1 --2- 172.21.15.43:0/85374434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bd900e4b0 0x558bd9018830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.637+0000 7f85ad600640 1 -- 172.21.15.43:0/85374434 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bd8f3aae0 con 0x558bd900db90 2024-09-19T05:05:24.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.637+0000 7f85ad600640 1 --2- 172.21.15.43:0/85374434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bd900db90 0x558bd900df70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f85a0009f50 tx=0x7f85a00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c711959f8aa93b4a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:24.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.637+0000 7f8599c00640 1 -- 172.21.15.43:0/85374434 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a0004d20 con 0x558bd900db90 2024-09-19T05:05:24.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.638+0000 7f8599c00640 1 -- 172.21.15.43:0/85374434 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85a0004050 con 0x558bd900db90 2024-09-19T05:05:24.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.638+0000 7f8599c00640 1 -- 172.21.15.43:0/85374434 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a0005220 con 0x558bd900db90 2024-09-19T05:05:24.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.638+0000 7f85a5600640 1 --2- 172.21.15.43:0/1957900906 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f85900a35e0 0x7f85900a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:24.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.638+0000 7f85a5600640 1 -- 172.21.15.43:0/1957900906 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f85940d0360 con 0x7f85a81049c0 2024-09-19T05:05:24.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.638+0000 7f85b20ce740 1 -- 172.21.15.43:0/85374434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bd900db90 msgr2=0x558bd900df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:24.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.638+0000 7f85b20ce740 1 --2- 172.21.15.43:0/85374434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bd900db90 0x558bd900df70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f85a0009f50 tx=0x7f85a00118d0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.638+0000 7f85acc00640 1 --2- 172.21.15.43:0/1957900906 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f85900a35e0 0x7f85900a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:24.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.639+0000 7f85b20ce740 1 -- 172.21.15.43:0/85374434 shutdown_connections 2024-09-19T05:05:24.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.639+0000 7f85b20ce740 1 --2- 172.21.15.43:0/85374434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bd9018d70 0x558bd901b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.639+0000 7f85b20ce740 1 --2- 172.21.15.43:0/85374434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bd900e4b0 0x558bd9018830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.639+0000 7f85b20ce740 1 --2- 172.21.15.43:0/85374434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bd900db90 0x558bd900df70 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.639+0000 7f85b20ce740 1 -- 172.21.15.43:0/85374434 >> 172.21.15.43:0/85374434 conn(0x558bd900a120 msgr2=0x558bd900a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:24.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.639+0000 7f85b20ce740 1 -- 172.21.15.43:0/85374434 shutdown_connections 2024-09-19T05:05:24.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.639+0000 7f85b20ce740 1 -- 172.21.15.43:0/85374434 wait complete. 2024-09-19T05:05:24.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.639+0000 7f85acc00640 1 --2- 172.21.15.43:0/1957900906 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f85900a35e0 0x7f85900a5aa0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f859c009230 tx=0x7f859c0058d0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:24.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.639+0000 7f85b20ce740 1 -- start start 2024-09-19T05:05:24.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.640+0000 7f85b20ce740 1 Processor -- start 2024-09-19T05:05:24.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.640+0000 7f85b20ce740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bd900db90 0x558bd9015bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:24.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.640+0000 7f85b20ce740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bd900e4b0 0x558bd9016100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:24.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.640+0000 7f85b20ce740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bd9018d70 0x558bd9016640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:24.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.640+0000 7f85b20ce740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558bd901b9e0 con 0x558bd900e4b0 2024-09-19T05:05:24.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.640+0000 7f85b20ce740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558bd8f2f0f0 con 0x558bd9018d70 2024-09-19T05:05:24.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.640+0000 7f85b20ce740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558bd901bb60 con 0x558bd900db90 2024-09-19T05:05:24.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.640+0000 7f85b20ce740 10 client.0 _ll_register_callbacks cb 0x558bd8f209e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:05:24.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.640+0000 7f85a7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bd900e4b0 0x558bd9016100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:24.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.640+0000 7f85a7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bd900e4b0 0x558bd9016100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37304/0 (socket says 172.21.15.43:37304) 2024-09-19T05:05:24.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.640+0000 7f85a7e00640 1 -- 172.21.15.43:0/2049092994 learned_addr learned my addr 172.21.15.43:0/2049092994 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:05:24.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.641+0000 7f85ad600640 1 --2- 172.21.15.43:0/2049092994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bd9018d70 0x558bd9016640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:24.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.641+0000 7f85acc00640 1 --2- 172.21.15.43:0/2049092994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bd900db90 0x558bd9015bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:24.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.641+0000 7f85a7e00640 1 -- 172.21.15.43:0/2049092994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bd900db90 msgr2=0x558bd9015bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:24.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.641+0000 7f85a7e00640 1 --2- 172.21.15.43:0/2049092994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bd900db90 0x558bd9015bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.641+0000 7f85a7e00640 1 -- 172.21.15.43:0/2049092994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bd9018d70 msgr2=0x558bd9016640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:24.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.641+0000 7f85a7e00640 1 --2- 172.21.15.43:0/2049092994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bd9018d70 0x558bd9016640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.641+0000 7f85a7e00640 1 -- 172.21.15.43:0/2049092994 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bd8ce53c0 con 0x558bd900e4b0 2024-09-19T05:05:24.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.641+0000 7f85ad600640 1 --2- 172.21.15.43:0/2049092994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bd9018d70 0x558bd9016640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:05:24.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.641+0000 7f85a7e00640 1 --2- 172.21.15.43:0/2049092994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bd900e4b0 0x558bd9016100 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f859400c6a0 tx=0x7f85940986b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fceeca1e4e77dde6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:24.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.641+0000 7f858e000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:05:24.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.641+0000 7f858e000640 1 -- 172.21.15.43:0/2049092994 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859409bd70 con 0x558bd900e4b0 2024-09-19T05:05:24.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.641+0000 7f858e000640 1 -- 172.21.15.43:0/2049092994 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f85940a4df0 con 0x558bd900e4b0 2024-09-19T05:05:24.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.641+0000 7f85b20ce740 1 -- 172.21.15.43:0/2049092994 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85a0004050 con 0x558bd900e4b0 2024-09-19T05:05:24.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.642+0000 7f858e000640 1 -- 172.21.15.43:0/2049092994 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f859409cb10 con 0x558bd900e4b0 2024-09-19T05:05:24.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.642+0000 7f85b20ce740 10 client.15363 fetch_fsmap learned FSMap version 29 2024-09-19T05:05:24.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.642+0000 7f85b20ce740 1 -- 172.21.15.43:0/2049092994 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558bd90275f0 con 0x558bd900e4b0 2024-09-19T05:05:24.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.642+0000 7f858e000640 1 -- 172.21.15.43:0/2049092994 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85a0004050 con 0x558bd900e4b0 2024-09-19T05:05:24.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.642+0000 7f858e000640 1 -- 172.21.15.43:0/2049092994 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f85940b5240 con 0x558bd900e4b0 2024-09-19T05:05:24.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.642+0000 7f85b20ce740 10 client.15363 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:05:24.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.642+0000 7f85b20ce740 10 client.15363 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:05:24.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.642+0000 7f85b20ce740 20 client.15363 populate_metadata read hostname 'smithi043' 2024-09-19T05:05:24.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.642+0000 7f85b20ce740 1 --2- 172.21.15.43:0/2049092994 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558bd9028a00 0x558bd902adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:24.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.642+0000 7f85b20ce740 4 client.15363 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:05:24.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.642+0000 7f85b20ce740 1 -- 172.21.15.43:0/2049092994 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558bd8f81950 con 0x558bd9028a00 2024-09-19T05:05:24.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.643+0000 7f85acc00640 1 --2- 172.21.15.43:0/2049092994 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558bd9028a00 0x558bd902adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:24.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.643+0000 7f85acc00640 1 --2- 172.21.15.43:0/2049092994 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558bd9028a00 0x558bd902adf0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f372b50411d1871 server_cookie=2873f21706666c8 in_seq=0 out_seq=0 2024-09-19T05:05:24.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.643+0000 7f858e000640 10 client.15363 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:05:24.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.644+0000 7f858e000640 1 -- 172.21.15.43:0/2049092994 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558bd8f81950 con 0x558bd9028a00 2024-09-19T05:05:24.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.644+0000 7f858e000640 10 client.15363 handle_command_reply: tid=0 2024-09-19T05:05:24.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.644+0000 7f85b20ce740 1 client.15363 shutdown 2024-09-19T05:05:24.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.645+0000 7f85b20ce740 1 -- 172.21.15.43:0/2049092994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bd900e4b0 msgr2=0x558bd9016100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:24.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.645+0000 7f85b20ce740 1 --2- 172.21.15.43:0/2049092994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bd900e4b0 0x558bd9016100 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f859400c6a0 tx=0x7f85940986b0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.645+0000 7f85b20ce740 1 -- 172.21.15.43:0/2049092994 shutdown_connections 2024-09-19T05:05:24.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.645+0000 7f85b20ce740 1 --2- 172.21.15.43:0/2049092994 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558bd9028a00 0x558bd902adf0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.645+0000 7f85b20ce740 1 --2- 172.21.15.43:0/2049092994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bd9018d70 0x558bd9016640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.645+0000 7f85b20ce740 1 --2- 172.21.15.43:0/2049092994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bd900e4b0 0x558bd9016100 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.645+0000 7f85b20ce740 1 --2- 172.21.15.43:0/2049092994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bd900db90 0x558bd9015bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.645+0000 7f85b20ce740 1 -- 172.21.15.43:0/2049092994 >> 172.21.15.43:0/2049092994 conn(0x558bd900a120 msgr2=0x558bd900fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:24.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.645+0000 7f858e000640 0 client.15363 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:05:24.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.645+0000 7f85b20ce740 1 -- 172.21.15.43:0/2049092994 shutdown_connections 2024-09-19T05:05:24.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.645+0000 7f85b20ce740 1 -- 172.21.15.43:0/2049092994 wait complete. 2024-09-19T05:05:24.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.645+0000 7f85b20ce740 20 client.15363 trim_cache size 0 max 16384 2024-09-19T05:05:24.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.673+0000 7f85b20ce740 1 Processor -- start 2024-09-19T05:05:24.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.673+0000 7f85b20ce740 1 -- start start 2024-09-19T05:05:24.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.674+0000 7f85b20ce740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bd900db90 0x558bd90105f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:24.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.674+0000 7f85b20ce740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bd900e4b0 0x558bd90306c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:24.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.674+0000 7f85b20ce740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bd9018d70 0x558bd9030c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:24.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.674+0000 7f85b20ce740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f85a81afa10 con 0x558bd900db90 2024-09-19T05:05:24.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.674+0000 7f85b20ce740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f85a81b1550 con 0x558bd900e4b0 2024-09-19T05:05:24.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.674+0000 7f85b20ce740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f85a81b00e0 con 0x558bd9018d70 2024-09-19T05:05:24.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.674+0000 7f85acc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bd900db90 0x558bd90105f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:24.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.674+0000 7f85acc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bd900db90 0x558bd90105f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37306/0 (socket says 172.21.15.43:37306) 2024-09-19T05:05:24.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.674+0000 7f85acc00640 1 -- 172.21.15.43:0/2762390397 learned_addr learned my addr 172.21.15.43:0/2762390397 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:05:24.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.674+0000 7f85ad600640 1 --2- 172.21.15.43:0/2762390397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bd9018d70 0x558bd9030c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:24.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.674+0000 7f85a7e00640 1 --2- 172.21.15.43:0/2762390397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bd900e4b0 0x558bd90306c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:24.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.675+0000 7f85acc00640 1 -- 172.21.15.43:0/2762390397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bd9018d70 msgr2=0x558bd9030c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:24.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.675+0000 7f85acc00640 1 --2- 172.21.15.43:0/2762390397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bd9018d70 0x558bd9030c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.675+0000 7f85acc00640 1 -- 172.21.15.43:0/2762390397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bd900e4b0 msgr2=0x558bd90306c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:24.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.675+0000 7f85acc00640 1 --2- 172.21.15.43:0/2762390397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bd900e4b0 0x558bd90306c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.675+0000 7f85acc00640 1 -- 172.21.15.43:0/2762390397 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bd9009f50 con 0x558bd900db90 2024-09-19T05:05:24.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.675+0000 7f85a7e00640 1 --2- 172.21.15.43:0/2762390397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bd900e4b0 0x558bd90306c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:05:24.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.675+0000 7f85acc00640 1 --2- 172.21.15.43:0/2762390397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bd900db90 0x558bd90105f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f859c054fd0 tx=0x7f859c03f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b88eac4e5e9bfe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:24.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.675+0000 7f8599200640 1 -- 172.21.15.43:0/2762390397 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c035040 con 0x558bd900db90 2024-09-19T05:05:24.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.676+0000 7f8599200640 1 -- 172.21.15.43:0/2762390397 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f859c05a070 con 0x558bd900db90 2024-09-19T05:05:24.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.676+0000 7f8599200640 1 -- 172.21.15.43:0/2762390397 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c03d450 con 0x558bd900db90 2024-09-19T05:05:24.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.676+0000 7f85b20ce740 1 -- 172.21.15.43:0/2762390397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bd900db90 msgr2=0x558bd90105f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:24.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.676+0000 7f85b20ce740 1 --2- 172.21.15.43:0/2762390397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bd900db90 0x558bd90105f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f859c054fd0 tx=0x7f859c03f000 comp rx=0 tx=0).stop 2024-09-19T05:05:24.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.677+0000 7f85b20ce740 1 -- 172.21.15.43:0/2762390397 shutdown_connections 2024-09-19T05:05:24.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.677+0000 7f85b20ce740 1 --2- 172.21.15.43:0/2762390397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bd9018d70 0x558bd9030c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.677+0000 7f85b20ce740 1 --2- 172.21.15.43:0/2762390397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bd900e4b0 0x558bd90306c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.677+0000 7f85b20ce740 1 --2- 172.21.15.43:0/2762390397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bd900db90 0x558bd90105f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.677+0000 7f85b20ce740 1 -- 172.21.15.43:0/2762390397 >> 172.21.15.43:0/2762390397 conn(0x558bd900a120 msgr2=0x558bd9014490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:24.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.677+0000 7f85b20ce740 1 -- 172.21.15.43:0/2762390397 shutdown_connections 2024-09-19T05:05:24.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.677+0000 7f85b20ce740 1 -- 172.21.15.43:0/2762390397 wait complete. 2024-09-19T05:05:24.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.677+0000 7f85b20ce740 1 -- start start 2024-09-19T05:05:24.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.678+0000 7f85b20ce740 1 Processor -- start 2024-09-19T05:05:24.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.678+0000 7f85b20ce740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bd900db90 0x558bd901ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:24.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.678+0000 7f85b20ce740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bd900e4b0 0x558bd901d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:24.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.678+0000 7f85b20ce740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bd9018d70 0x558bd9092690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:24.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.678+0000 7f85b20ce740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f85a81b0e80 con 0x558bd9018d70 2024-09-19T05:05:24.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.678+0000 7f85b20ce740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f85a81b07b0 con 0x558bd900e4b0 2024-09-19T05:05:24.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.678+0000 7f85b20ce740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f85a81ad9f0 con 0x558bd900db90 2024-09-19T05:05:24.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.678+0000 7f85b20ce740 10 client.0 _ll_register_callbacks cb 0x7f85940d3940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:05:24.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.678+0000 7f85acc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bd900db90 0x558bd901ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:24.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.678+0000 7f85acc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bd900db90 0x558bd901ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:46306/0 (socket says 172.21.15.43:46306) 2024-09-19T05:05:24.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.678+0000 7f85acc00640 1 -- 172.21.15.43:0/901983537 learned_addr learned my addr 172.21.15.43:0/901983537 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:05:24.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.679+0000 7f85a7e00640 1 --2- 172.21.15.43:0/901983537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bd900e4b0 0x558bd901d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:24.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.679+0000 7f85ad600640 1 --2- 172.21.15.43:0/901983537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bd9018d70 0x558bd9092690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:24.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.679+0000 7f85acc00640 1 -- 172.21.15.43:0/901983537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bd900e4b0 msgr2=0x558bd901d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:24.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.679+0000 7f85acc00640 1 --2- 172.21.15.43:0/901983537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bd900e4b0 0x558bd901d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.679+0000 7f85acc00640 1 -- 172.21.15.43:0/901983537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bd9018d70 msgr2=0x558bd9092690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:24.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.679+0000 7f85acc00640 1 --2- 172.21.15.43:0/901983537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bd9018d70 0x558bd9092690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.679+0000 7f85acc00640 1 -- 172.21.15.43:0/901983537 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bd900b1d0 con 0x558bd900db90 2024-09-19T05:05:24.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.679+0000 7f85acc00640 1 --2- 172.21.15.43:0/901983537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bd900db90 0x558bd901ce70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f859c02a910 tx=0x7f859c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c02ce05e83df44d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:24.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.679+0000 7f858e000640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:05:24.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.680+0000 7f858e000640 1 -- 172.21.15.43:0/901983537 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c063020 con 0x558bd900db90 2024-09-19T05:05:24.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.680+0000 7f858e000640 1 -- 172.21.15.43:0/901983537 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f859c0545f0 con 0x558bd900db90 2024-09-19T05:05:24.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.680+0000 7f85b20ce740 1 -- 172.21.15.43:0/901983537 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f859c05a070 con 0x558bd900db90 2024-09-19T05:05:24.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.680+0000 7f858e000640 1 -- 172.21.15.43:0/901983537 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f859c0547f0 con 0x558bd900db90 2024-09-19T05:05:24.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.680+0000 7f85b20ce740 10 client.25327 fetch_fsmap learned FSMap version 29 2024-09-19T05:05:24.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.680+0000 7f85b20ce740 1 -- 172.21.15.43:0/901983537 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558bd9093b00 con 0x558bd900db90 2024-09-19T05:05:24.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.681+0000 7f858e000640 1 -- 172.21.15.43:0/901983537 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f859c05a070 con 0x558bd900db90 2024-09-19T05:05:24.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.681+0000 7f858e000640 1 -- 172.21.15.43:0/901983537 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f859c03d8b0 con 0x558bd900db90 2024-09-19T05:05:24.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.681+0000 7f85b20ce740 10 client.25327 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:05:24.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.681+0000 7f85b20ce740 10 client.25327 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:05:24.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.681+0000 7f85b20ce740 20 client.25327 populate_metadata read hostname 'smithi043' 2024-09-19T05:05:24.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.681+0000 7f85b20ce740 1 --2- 172.21.15.43:0/901983537 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558bd90941f0 0x558bd9096610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:24.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.681+0000 7f85b20ce740 4 client.25327 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:05:24.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.681+0000 7f85b20ce740 1 -- 172.21.15.43:0/901983537 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558bd8f7be60 con 0x558bd90941f0 2024-09-19T05:05:24.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.681+0000 7f85a7e00640 1 --2- 172.21.15.43:0/901983537 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558bd90941f0 0x558bd9096610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:24.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.682+0000 7f85a7e00640 1 --2- 172.21.15.43:0/901983537 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558bd90941f0 0x558bd9096610 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6ea5bd50cbff782 server_cookie=b5ea957fa0523d25 in_seq=0 out_seq=0 2024-09-19T05:05:24.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.682+0000 7f858e000640 10 client.25327 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:05:24.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.682+0000 7f858e000640 1 -- 172.21.15.43:0/901983537 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558bd8f7be60 con 0x558bd90941f0 2024-09-19T05:05:24.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.682+0000 7f858e000640 10 client.25327 handle_command_reply: tid=0 2024-09-19T05:05:24.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.682+0000 7f85b20ce740 1 client.25327 shutdown 2024-09-19T05:05:24.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.682+0000 7f85b20ce740 1 -- 172.21.15.43:0/901983537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bd900db90 msgr2=0x558bd901ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:24.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.682+0000 7f85b20ce740 1 --2- 172.21.15.43:0/901983537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bd900db90 0x558bd901ce70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f859c02a910 tx=0x7f859c009260 comp rx=0 tx=0).stop 2024-09-19T05:05:24.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.683+0000 7f85b20ce740 1 -- 172.21.15.43:0/901983537 shutdown_connections 2024-09-19T05:05:24.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.683+0000 7f85b20ce740 1 --2- 172.21.15.43:0/901983537 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558bd90941f0 0x558bd9096610 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.683+0000 7f85b20ce740 1 --2- 172.21.15.43:0/901983537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bd9018d70 0x558bd9092690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.683+0000 7f85b20ce740 1 --2- 172.21.15.43:0/901983537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558bd900e4b0 0x558bd901d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.683+0000 7f85b20ce740 1 --2- 172.21.15.43:0/901983537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558bd900db90 0x558bd901ce70 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.683+0000 7f85b20ce740 1 -- 172.21.15.43:0/901983537 >> 172.21.15.43:0/901983537 conn(0x558bd900a120 msgr2=0x558bd9015b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:24.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.683+0000 7f858e000640 0 client.25327 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:05:24.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.683+0000 7f85b20ce740 1 -- 172.21.15.43:0/901983537 shutdown_connections 2024-09-19T05:05:24.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.683+0000 7f85b20ce740 1 -- 172.21.15.43:0/901983537 wait complete. 2024-09-19T05:05:24.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.683+0000 7f85b20ce740 20 client.25327 trim_cache size 0 max 16384 2024-09-19T05:05:24.034 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:05:24.034 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-09-19T05:05:24.034 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-09-19T05:05:24.034 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:05:24.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.684+0000 7f85af600640 1 -- 172.21.15.43:0/1957900906 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f85900a35e0 msgr2=0x7f85900a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:24.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.684+0000 7f85af600640 1 --2- 172.21.15.43:0/1957900906 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f85900a35e0 0x7f85900a5aa0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f859c009230 tx=0x7f859c0058d0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.684+0000 7f85af600640 1 -- 172.21.15.43:0/1957900906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a81049c0 msgr2=0x7f85a819ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:24.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.684+0000 7f85af600640 1 --2- 172.21.15.43:0/1957900906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a81049c0 0x7f85a819ab60 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f859400e9f0 tx=0x7f859400edf0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.684+0000 7f85af600640 1 -- 172.21.15.43:0/1957900906 shutdown_connections 2024-09-19T05:05:24.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.684+0000 7f85af600640 1 --2- 172.21.15.43:0/1957900906 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f85900a35e0 0x7f85900a5aa0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.684+0000 7f85af600640 1 --2- 172.21.15.43:0/1957900906 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85a8109140 0x7f85a819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.684+0000 7f85af600640 1 --2- 172.21.15.43:0/1957900906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a81049c0 0x7f85a819ab60 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.684+0000 7f85af600640 1 --2- 172.21.15.43:0/1957900906 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85a8104010 0x7f85a819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:24.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.684+0000 7f85af600640 1 -- 172.21.15.43:0/1957900906 >> 172.21.15.43:0/1957900906 conn(0x7f85a80ffa50 msgr2=0x7f85a8100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:24.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.684+0000 7f85af600640 1 -- 172.21.15.43:0/1957900906 shutdown_connections 2024-09-19T05:05:24.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:23.684+0000 7f85af600640 1 -- 172.21.15.43:0/1957900906 wait complete. 2024-09-19T05:05:24.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:24 smithi043 ceph-mon[30800]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:05:24.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:24 smithi151 ceph-mon[35425]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:05:24.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:24 smithi096 ceph-mon[35085]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:05:25.033 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T05:05:25.034 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:05:25.519 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:05:26.191 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:26 smithi043 ceph-mon[30800]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:05:26.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.252+0000 7f110be00640 1 -- 172.21.15.43:0/3890112821 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f110c1079c0 msgr2=0x7f110c107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:26.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.252+0000 7f110be00640 1 --2- 172.21.15.43:0/3890112821 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f110c1079c0 0x7f110c107da0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f10fc009900 tx=0x7f10fc02a3f0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.252+0000 7f110be00640 1 -- 172.21.15.43:0/3890112821 shutdown_connections 2024-09-19T05:05:26.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.252+0000 7f110be00640 1 --2- 172.21.15.43:0/3890112821 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f110c102360 0x7f110c10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.253+0000 7f110be00640 1 --2- 172.21.15.43:0/3890112821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f110c1019c0 0x7f110c101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.253+0000 7f110be00640 1 --2- 172.21.15.43:0/3890112821 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f110c1079c0 0x7f110c107da0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.253+0000 7f110be00640 1 -- 172.21.15.43:0/3890112821 >> 172.21.15.43:0/3890112821 conn(0x7f110c0fd740 msgr2=0x7f110c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:26.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.253+0000 7f110be00640 1 -- 172.21.15.43:0/3890112821 shutdown_connections 2024-09-19T05:05:26.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.253+0000 7f110be00640 1 -- 172.21.15.43:0/3890112821 wait complete. 2024-09-19T05:05:26.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.255+0000 7f110be00640 1 Processor -- start 2024-09-19T05:05:26.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.255+0000 7f110be00640 1 -- start start 2024-09-19T05:05:26.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.255+0000 7f110be00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f110c1019c0 0x7f110c1088f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:26.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.255+0000 7f110be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f110c102360 0x7f110c108e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:26.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.255+0000 7f110be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f110c1079c0 0x7f110c109370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:26.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.255+0000 7f110be00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f110c111e10 con 0x7f110c102360 2024-09-19T05:05:26.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.255+0000 7f110be00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f110c111c90 con 0x7f110c1019c0 2024-09-19T05:05:26.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.255+0000 7f110be00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f110c111f90 con 0x7f110c1079c0 2024-09-19T05:05:26.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.256+0000 7f110aa00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f110c1019c0 0x7f110c1088f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:26.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.256+0000 7f110aa00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f110c1019c0 0x7f110c1088f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:41038/0 (socket says 172.21.15.43:41038) 2024-09-19T05:05:26.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.256+0000 7f110aa00640 1 -- 172.21.15.43:0/1950473738 learned_addr learned my addr 172.21.15.43:0/1950473738 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:05:26.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.256+0000 7f110a000640 1 --2- 172.21.15.43:0/1950473738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f110c102360 0x7f110c108e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:26.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.256+0000 7f110aa00640 1 -- 172.21.15.43:0/1950473738 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f110c1079c0 msgr2=0x7f110c109370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:26.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.256+0000 7f110aa00640 1 --2- 172.21.15.43:0/1950473738 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f110c1079c0 0x7f110c109370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.256+0000 7f110aa00640 1 -- 172.21.15.43:0/1950473738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f110c102360 msgr2=0x7f110c108e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:26.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.257+0000 7f110aa00640 1 --2- 172.21.15.43:0/1950473738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f110c102360 0x7f110c108e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.257+0000 7f110aa00640 1 -- 172.21.15.43:0/1950473738 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f110c1aab10 con 0x7f110c1019c0 2024-09-19T05:05:26.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.257+0000 7f110a000640 1 --2- 172.21.15.43:0/1950473738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f110c102360 0x7f110c108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:05:26.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.257+0000 7f110aa00640 1 --2- 172.21.15.43:0/1950473738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f110c1019c0 0x7f110c1088f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f10fc003c10 tx=0x7f10fc003c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:26.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.258+0000 7f1103400640 1 -- 172.21.15.43:0/1950473738 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10fc03f070 con 0x7f110c1019c0 2024-09-19T05:05:26.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.258+0000 7f110be00640 1 -- 172.21.15.43:0/1950473738 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f110c1aada0 con 0x7f110c1019c0 2024-09-19T05:05:26.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.258+0000 7f110be00640 1 -- 172.21.15.43:0/1950473738 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f110c1ab270 con 0x7f110c1019c0 2024-09-19T05:05:26.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.258+0000 7f1103400640 1 -- 172.21.15.43:0/1950473738 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10fc031e30 con 0x7f110c1019c0 2024-09-19T05:05:26.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.258+0000 7f1103400640 1 -- 172.21.15.43:0/1950473738 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10fc035030 con 0x7f110c1019c0 2024-09-19T05:05:26.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.260+0000 7f110be00640 1 -- 172.21.15.43:0/1950473738 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10ec005290 con 0x7f110c1019c0 2024-09-19T05:05:26.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.261+0000 7f1103400640 1 -- 172.21.15.43:0/1950473738 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f10fc042070 con 0x7f110c1019c0 2024-09-19T05:05:26.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.262+0000 7f1103400640 1 --2- 172.21.15.43:0/1950473738 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f10f00a35e0 0x7f10f00a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:26.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.263+0000 7f1103400640 1 -- 172.21.15.43:0/1950473738 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f10fc0f0790 con 0x7f110c1019c0 2024-09-19T05:05:26.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.263+0000 7f110a000640 1 --2- 172.21.15.43:0/1950473738 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f10f00a35e0 0x7f10f00a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:26.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.264+0000 7f110a000640 1 --2- 172.21.15.43:0/1950473738 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f10f00a35e0 0x7f10f00a5aa0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f110c10a0b0 tx=0x7f10f8005f90 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:26.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.267+0000 7f1103400640 1 -- 172.21.15.43:0/1950473738 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f10fc0bcf40 con 0x7f110c1019c0 2024-09-19T05:05:26.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:26 smithi151 ceph-mon[35425]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:05:26.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:26 smithi096 ceph-mon[35085]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T05:05:26.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.546+0000 7f110be00640 1 -- 172.21.15.43:0/1950473738 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10ec005850 con 0x7f110c1019c0 2024-09-19T05:05:26.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.547+0000 7f1103400640 1 -- 172.21.15.43:0/1950473738 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11965 (secure 0 0 0) 0x7f10fc003e60 con 0x7f110c1019c0 2024-09-19T05:05:26.549 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:05:26.550 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":29,"btime":"2024-09-19T05:04:38:785407+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/773801169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":773801169},{"type":"v1","addr":"172.21.15.43:6835","nonce":773801169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1799137902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1799137902},{"type":"v1","addr":"172.21.15.43:6837","nonce":1799137902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:02:25.463041+0000","modified":"2024-09-19T05:04:38.039809+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24518,"mds_2":24511,"mds_3":24506,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14541":{"gid":14541,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/794881135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":794881135},{"type":"v1","addr":"172.21.15.43:6841","nonce":794881135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/2836453158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2836453158},{"type":"v1","addr":"172.21.15.151:6833","nonce":2836453158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/178785812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":178785812},{"type":"v1","addr":"172.21.15.96:6833","nonce":178785812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1671043820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1671043820},{"type":"v1","addr":"172.21.15.96:6835","nonce":1671043820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2972514328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2972514328},{"type":"v1","addr":"172.21.15.151:6835","nonce":2972514328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.151:6837/3960378010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3960378010},{"type":"v1","addr":"172.21.15.151:6837","nonce":3960378010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6837/3978388286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3978388286},{"type":"v1","addr":"172.21.15.96:6837","nonce":3978388286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.151:6839/3794104313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3794104313},{"type":"v1","addr":"172.21.15.151:6839","nonce":3794104313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.43:6839/3404555030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3404555030},{"type":"v1","addr":"172.21.15.43:6839","nonce":3404555030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/605527621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":605527621},{"type":"v1","addr":"172.21.15.96:6839","nonce":605527621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24518,24511,24506,24493]},"id":1}]} 2024-09-19T05:05:26.550 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 29 2024-09-19T05:05:26.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.549+0000 7f110be00640 1 -- 172.21.15.43:0/1950473738 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f10f00a35e0 msgr2=0x7f10f00a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:26.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.549+0000 7f110be00640 1 --2- 172.21.15.43:0/1950473738 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f10f00a35e0 0x7f10f00a5aa0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f110c10a0b0 tx=0x7f10f8005f90 comp rx=0 tx=0).stop 2024-09-19T05:05:26.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.549+0000 7f110be00640 1 -- 172.21.15.43:0/1950473738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f110c1019c0 msgr2=0x7f110c1088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:26.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.549+0000 7f110be00640 1 --2- 172.21.15.43:0/1950473738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f110c1019c0 0x7f110c1088f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f10fc003c10 tx=0x7f10fc003c40 comp rx=0 tx=0).stop 2024-09-19T05:05:26.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.550+0000 7f110be00640 1 -- 172.21.15.43:0/1950473738 shutdown_connections 2024-09-19T05:05:26.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.550+0000 7f110be00640 1 --2- 172.21.15.43:0/1950473738 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f110c1079c0 0x7f110c109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.550+0000 7f110be00640 1 --2- 172.21.15.43:0/1950473738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f110c102360 0x7f110c108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.550+0000 7f110be00640 1 --2- 172.21.15.43:0/1950473738 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f10f00a35e0 0x7f10f00a5aa0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.550+0000 7f110be00640 1 --2- 172.21.15.43:0/1950473738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f110c1019c0 0x7f110c1088f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:26.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.550+0000 7f110be00640 1 -- 172.21.15.43:0/1950473738 >> 172.21.15.43:0/1950473738 conn(0x7f110c0fd740 msgr2=0x7f110c10cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:26.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.550+0000 7f110be00640 1 -- 172.21.15.43:0/1950473738 shutdown_connections 2024-09-19T05:05:26.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:26.550+0000 7f110be00640 1 -- 172.21.15.43:0/1950473738 wait complete. 2024-09-19T05:05:27.155 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:27 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/1950473738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:05:27.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:27 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/1950473738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:05:27.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:27 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/1950473738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:05:27.507 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T05:05:27.940 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:05:28.341 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:28 smithi043 ceph-mon[30800]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T05:05:28.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:28 smithi151 ceph-mon[35425]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T05:05:28.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:28 smithi096 ceph-mon[35085]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-19T05:05:28.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.635+0000 7f6f86800640 1 -- 172.21.15.43:0/3035305804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f80103ff0 msgr2=0x7f6f801043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:28.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.635+0000 7f6f86800640 1 --2- 172.21.15.43:0/3035305804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f80103ff0 0x7f6f801043d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6f70009900 tx=0x7f6f7002a410 comp rx=0 tx=0).stop 2024-09-19T05:05:28.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.635+0000 7f6f86800640 1 -- 172.21.15.43:0/3035305804 shutdown_connections 2024-09-19T05:05:28.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.635+0000 7f6f86800640 1 --2- 172.21.15.43:0/3035305804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f80109120 0x7f6f80110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:28.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.635+0000 7f6f86800640 1 --2- 172.21.15.43:0/3035305804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f801049a0 0x7f6f801089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:28.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.635+0000 7f6f86800640 1 --2- 172.21.15.43:0/3035305804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f80103ff0 0x7f6f801043d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:28.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.635+0000 7f6f86800640 1 -- 172.21.15.43:0/3035305804 >> 172.21.15.43:0/3035305804 conn(0x7f6f800ffa90 msgr2=0x7f6f80101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:28.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.635+0000 7f6f86800640 1 -- 172.21.15.43:0/3035305804 shutdown_connections 2024-09-19T05:05:28.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.635+0000 7f6f86800640 1 -- 172.21.15.43:0/3035305804 wait complete. 2024-09-19T05:05:28.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.636+0000 7f6f86800640 1 Processor -- start 2024-09-19T05:05:28.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.637+0000 7f6f86800640 1 -- start start 2024-09-19T05:05:28.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.637+0000 7f6f86800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f801049a0 0x7f6f8010ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:28.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.637+0000 7f6f86800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f80109120 0x7f6f8010e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:28.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.637+0000 7f6f86800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f8010e830 0x7f6f801aaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:28.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.637+0000 7f6f86800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6f80114170 con 0x7f6f801049a0 2024-09-19T05:05:28.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.637+0000 7f6f86800640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6f80113ff0 con 0x7f6f8010e830 2024-09-19T05:05:28.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.637+0000 7f6f86800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6f801142f0 con 0x7f6f80109120 2024-09-19T05:05:28.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.638+0000 7f6f7f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f801049a0 0x7f6f8010ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:28.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.638+0000 7f6f7fe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f8010e830 0x7f6f801aaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:28.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.638+0000 7f6f7f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f801049a0 0x7f6f8010ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36570/0 (socket says 172.21.15.43:36570) 2024-09-19T05:05:28.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.638+0000 7f6f7f400640 1 -- 172.21.15.43:0/1508901729 learned_addr learned my addr 172.21.15.43:0/1508901729 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:05:28.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.639+0000 7f6f7ea00640 1 --2- 172.21.15.43:0/1508901729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f80109120 0x7f6f8010e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:28.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.639+0000 7f6f7f400640 1 -- 172.21.15.43:0/1508901729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f80109120 msgr2=0x7f6f8010e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:28.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.639+0000 7f6f7f400640 1 --2- 172.21.15.43:0/1508901729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f80109120 0x7f6f8010e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:28.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.639+0000 7f6f7f400640 1 -- 172.21.15.43:0/1508901729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f8010e830 msgr2=0x7f6f801aaeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:28.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.639+0000 7f6f7f400640 1 --2- 172.21.15.43:0/1508901729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f8010e830 0x7f6f801aaeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:28.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.639+0000 7f6f7f400640 1 -- 172.21.15.43:0/1508901729 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f801ab3f0 con 0x7f6f801049a0 2024-09-19T05:05:28.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.639+0000 7f6f7f400640 1 --2- 172.21.15.43:0/1508901729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f801049a0 0x7f6f8010ddb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6f7002aa00 tx=0x7f6f70003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:28.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.640+0000 7f6f77e00640 1 -- 172.21.15.43:0/1508901729 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7003f020 con 0x7f6f801049a0 2024-09-19T05:05:28.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.640+0000 7f6f77e00640 1 -- 172.21.15.43:0/1508901729 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f70031c30 con 0x7f6f801049a0 2024-09-19T05:05:28.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.640+0000 7f6f77e00640 1 -- 172.21.15.43:0/1508901729 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f70035070 con 0x7f6f801049a0 2024-09-19T05:05:28.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.640+0000 7f6f86800640 1 -- 172.21.15.43:0/1508901729 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f801ab5c0 con 0x7f6f801049a0 2024-09-19T05:05:28.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.640+0000 7f6f86800640 1 -- 172.21.15.43:0/1508901729 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f801ab9c0 con 0x7f6f801049a0 2024-09-19T05:05:28.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.645+0000 7f6f89241740 1 Processor -- start 2024-09-19T05:05:28.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.645+0000 7f6f89241740 1 -- start start 2024-09-19T05:05:28.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.645+0000 7f6f77e00640 1 -- 172.21.15.43:0/1508901729 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f6f70042020 con 0x7f6f801049a0 2024-09-19T05:05:28.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.645+0000 7f6f89241740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf610c1c70 0x55bf610c2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:28.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.645+0000 7f6f89241740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf610c2590 0x55bf610cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:28.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.645+0000 7f6f89241740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf610cce50 0x55bf610cf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:28.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.645+0000 7f6f89241740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bf610283e0 con 0x55bf610c1c70 2024-09-19T05:05:28.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.645+0000 7f6f89241740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55bf60fcfd80 con 0x55bf610cce50 2024-09-19T05:05:28.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.645+0000 7f6f89241740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55bf610243a0 con 0x55bf610c2590 2024-09-19T05:05:28.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.646+0000 7f6f7f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf610c2590 0x55bf610cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:28.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.646+0000 7f6f7f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf610c2590 0x55bf610cc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:49904/0 (socket says 172.21.15.43:49904) 2024-09-19T05:05:28.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.646+0000 7f6f7f400640 1 -- 172.21.15.43:0/1160463509 learned_addr learned my addr 172.21.15.43:0/1160463509 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:05:28.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.646+0000 7f6f7ea00640 1 --2- 172.21.15.43:0/1160463509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf610cce50 0x55bf610cf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:28.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.647+0000 7f6f7fe00640 1 --2- 172.21.15.43:0/1160463509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf610c1c70 0x55bf610c2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:28.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.647+0000 7f6f7ea00640 1 -- 172.21.15.43:0/1160463509 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf610c2590 msgr2=0x55bf610cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:28.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.647+0000 7f6f7ea00640 1 --2- 172.21.15.43:0/1160463509 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf610c2590 0x55bf610cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:28.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.647+0000 7f6f7ea00640 1 -- 172.21.15.43:0/1160463509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf610c1c70 msgr2=0x55bf610c2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:28.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.647+0000 7f6f7ea00640 1 --2- 172.21.15.43:0/1160463509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf610c1c70 0x55bf610c2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:28.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.647+0000 7f6f7ea00640 1 -- 172.21.15.43:0/1160463509 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf60feeae0 con 0x55bf610cce50 2024-09-19T05:05:28.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.647+0000 7f6f7f400640 1 --2- 172.21.15.43:0/1160463509 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf610c2590 0x55bf610cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:05:28.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.647+0000 7f6f77e00640 1 --2- 172.21.15.43:0/1508901729 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f6f640a35e0 0x7f6f640a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:28.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.648+0000 7f6f77e00640 1 -- 172.21.15.43:0/1508901729 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f6f700effe0 con 0x7f6f801049a0 2024-09-19T05:05:28.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.648+0000 7f6f7ea00640 1 --2- 172.21.15.43:0/1160463509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf610cce50 0x55bf610cf240 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6f8010f600 tx=0x7f6f6c010370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e7793097ed0cb56 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:28.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.648+0000 7f6f74c00640 1 -- 172.21.15.43:0/1160463509 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c010da0 con 0x55bf610cce50 2024-09-19T05:05:28.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.648+0000 7f6f7fe00640 1 --2- 172.21.15.43:0/1508901729 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f6f640a35e0 0x7f6f640a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:28.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.648+0000 7f6f74c00640 1 -- 172.21.15.43:0/1160463509 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f6c0035f0 con 0x55bf610cce50 2024-09-19T05:05:28.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.648+0000 7f6f74c00640 1 -- 172.21.15.43:0/1160463509 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c010da0 con 0x55bf610cce50 2024-09-19T05:05:28.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.649+0000 7f6f89241740 1 -- 172.21.15.43:0/1160463509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf610cce50 msgr2=0x55bf610cf240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:28.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.649+0000 7f6f89241740 1 --2- 172.21.15.43:0/1160463509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf610cce50 0x55bf610cf240 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6f8010f600 tx=0x7f6f6c010370 comp rx=0 tx=0).stop 2024-09-19T05:05:28.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.650+0000 7f6f89241740 1 -- 172.21.15.43:0/1160463509 shutdown_connections 2024-09-19T05:05:28.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.650+0000 7f6f89241740 1 --2- 172.21.15.43:0/1160463509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf610cce50 0x55bf610cf240 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:28.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.650+0000 7f6f89241740 1 --2- 172.21.15.43:0/1160463509 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf610c2590 0x55bf610cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:28.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.650+0000 7f6f89241740 1 --2- 172.21.15.43:0/1160463509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf610c1c70 0x55bf610c2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:28.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.650+0000 7f6f89241740 1 -- 172.21.15.43:0/1160463509 >> 172.21.15.43:0/1160463509 conn(0x55bf610be200 msgr2=0x55bf610be5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:28.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.650+0000 7f6f7fe00640 1 --2- 172.21.15.43:0/1508901729 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f6f640a35e0 0x7f6f640a5aa0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6f78005fd0 tx=0x7f6f78009500 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:28.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.650+0000 7f6f89241740 1 -- 172.21.15.43:0/1160463509 shutdown_connections 2024-09-19T05:05:28.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.650+0000 7f6f89241740 1 -- 172.21.15.43:0/1160463509 wait complete. 2024-09-19T05:05:28.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.651+0000 7f6f89241740 1 -- start start 2024-09-19T05:05:28.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.651+0000 7f6f89241740 1 Processor -- start 2024-09-19T05:05:28.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.652+0000 7f6f89241740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf610c1c70 0x55bf610c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:28.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.652+0000 7f6f89241740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf610c2590 0x55bf610ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:28.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.652+0000 7f6f89241740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf610cce50 0x55bf610ca710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:28.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.652+0000 7f6f89241740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bf610cfa70 con 0x55bf610cce50 2024-09-19T05:05:28.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.652+0000 7f6f89241740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55bf60fe30f0 con 0x55bf610c2590 2024-09-19T05:05:28.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.652+0000 7f6f89241740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55bf610cfbf0 con 0x55bf610c1c70 2024-09-19T05:05:28.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.652+0000 7f6f89241740 10 client.0 _ll_register_callbacks cb 0x55bf60fcabd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:05:28.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.652+0000 7f6f7ea00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf610c2590 0x55bf610ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:28.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.652+0000 7f6f7f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf610c1c70 0x55bf610c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:28.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.652+0000 7f6f7ea00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf610c2590 0x55bf610ca1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:41076/0 (socket says 172.21.15.43:41076) 2024-09-19T05:05:28.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.652+0000 7f6f7ea00640 1 -- 172.21.15.43:0/3845401587 learned_addr learned my addr 172.21.15.43:0/3845401587 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:05:28.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.653+0000 7f6f7fe00640 1 --2- 172.21.15.43:0/3845401587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf610cce50 0x55bf610ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:28.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.653+0000 7f6f7ea00640 1 -- 172.21.15.43:0/3845401587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf610c1c70 msgr2=0x55bf610c9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:28.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.653+0000 7f6f7ea00640 1 --2- 172.21.15.43:0/3845401587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf610c1c70 0x55bf610c9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:28.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.653+0000 7f6f7ea00640 1 -- 172.21.15.43:0/3845401587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf610cce50 msgr2=0x55bf610ca710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:28.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.653+0000 7f6f7ea00640 1 --2- 172.21.15.43:0/3845401587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf610cce50 0x55bf610ca710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:28.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.653+0000 7f6f7ea00640 1 -- 172.21.15.43:0/3845401587 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf60d993c0 con 0x55bf610c2590 2024-09-19T05:05:28.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.653+0000 7f6f7fe00640 1 --2- 172.21.15.43:0/3845401587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf610cce50 0x55bf610ca710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:05:28.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.653+0000 7f6f7ea00640 1 --2- 172.21.15.43:0/3845401587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf610c2590 0x55bf610ca1d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c0041f0 tx=0x7f6f6c004220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1a498c473c327a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:28.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.653+0000 7f6f69600640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:05:28.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.654+0000 7f6f69600640 1 -- 172.21.15.43:0/3845401587 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c003b10 con 0x55bf610c2590 2024-09-19T05:05:28.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.654+0000 7f6f89241740 1 -- 172.21.15.43:0/3845401587 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f6c0035f0 con 0x55bf610c2590 2024-09-19T05:05:28.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.654+0000 7f6f69600640 1 -- 172.21.15.43:0/3845401587 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f6f6c009750 con 0x55bf610c2590 2024-09-19T05:05:28.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.654+0000 7f6f69600640 1 -- 172.21.15.43:0/3845401587 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6f6c02d050 con 0x55bf610c2590 2024-09-19T05:05:28.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.654+0000 7f6f89241740 10 client.25376 fetch_fsmap learned FSMap version 29 2024-09-19T05:05:28.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.654+0000 7f6f89241740 1 -- 172.21.15.43:0/3845401587 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bf610dba50 con 0x55bf610c2590 2024-09-19T05:05:28.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.655+0000 7f6f69600640 1 -- 172.21.15.43:0/3845401587 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f6c0035f0 con 0x55bf610c2590 2024-09-19T05:05:28.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.655+0000 7f6f69600640 1 -- 172.21.15.43:0/3845401587 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6f6c00f040 con 0x55bf610c2590 2024-09-19T05:05:28.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.655+0000 7f6f89241740 10 client.25376 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:05:28.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.655+0000 7f6f89241740 10 client.25376 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-19T05:05:28.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.655+0000 7f6f89241740 20 client.25376 populate_metadata read hostname 'smithi043' 2024-09-19T05:05:28.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.655+0000 7f6f89241740 1 --2- 172.21.15.43:0/3845401587 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55bf610dce60 0x55bf610df250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:28.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.655+0000 7f6f89241740 4 client.25376 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:05:28.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.655+0000 7f6f89241740 1 -- 172.21.15.43:0/3845401587 --> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf61035950 con 0x55bf610dce60 2024-09-19T05:05:28.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.656+0000 7f6f7f400640 1 --2- 172.21.15.43:0/3845401587 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55bf610dce60 0x55bf610df250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:28.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.657+0000 7f6f7f400640 1 --2- 172.21.15.43:0/3845401587 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55bf610dce60 0x55bf610df250 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d38fb2b4afff6f2c server_cookie=e9b1d0660b0e3155 in_seq=0 out_seq=0 2024-09-19T05:05:28.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.657+0000 7f6f69600640 10 client.25376 ms_handle_connect on v2:172.21.15.151:6836/3960378010 2024-09-19T05:05:28.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.658+0000 7f6f69600640 1 -- 172.21.15.43:0/3845401587 <== mds.4 v2:172.21.15.151:6836/3960378010 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bf61035950 con 0x55bf610dce60 2024-09-19T05:05:28.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.658+0000 7f6f69600640 10 client.25376 handle_command_reply: tid=0 2024-09-19T05:05:28.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.658+0000 7f6f89241740 1 client.25376 shutdown 2024-09-19T05:05:28.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.659+0000 7f6f89241740 1 -- 172.21.15.43:0/3845401587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf610c2590 msgr2=0x55bf610ca1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:28.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.659+0000 7f6f89241740 1 --2- 172.21.15.43:0/3845401587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf610c2590 0x55bf610ca1d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c0041f0 tx=0x7f6f6c004220 comp rx=0 tx=0).stop 2024-09-19T05:05:28.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.659+0000 7f6f89241740 1 -- 172.21.15.43:0/3845401587 shutdown_connections 2024-09-19T05:05:28.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.659+0000 7f6f89241740 1 --2- 172.21.15.43:0/3845401587 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55bf610dce60 0x55bf610df250 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:28.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.659+0000 7f6f89241740 1 --2- 172.21.15.43:0/3845401587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf610cce50 0x55bf610ca710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:28.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.659+0000 7f6f89241740 1 --2- 172.21.15.43:0/3845401587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf610c2590 0x55bf610ca1d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:28.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.659+0000 7f6f89241740 1 --2- 172.21.15.43:0/3845401587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf610c1c70 0x55bf610c9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:28.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.659+0000 7f6f89241740 1 -- 172.21.15.43:0/3845401587 >> 172.21.15.43:0/3845401587 conn(0x55bf610be200 msgr2=0x55bf610c3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:28.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.659+0000 7f6f69600640 0 client.25376 ms_handle_reset on v2:172.21.15.151:6836/3960378010 2024-09-19T05:05:28.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.659+0000 7f6f89241740 1 -- 172.21.15.43:0/3845401587 shutdown_connections 2024-09-19T05:05:28.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.659+0000 7f6f89241740 1 -- 172.21.15.43:0/3845401587 wait complete. 2024-09-19T05:05:28.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.659+0000 7f6f89241740 20 client.25376 trim_cache size 0 max 16384 2024-09-19T05:05:28.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.673+0000 7f6f89241740 1 Processor -- start 2024-09-19T05:05:28.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.673+0000 7f6f89241740 1 -- start start 2024-09-19T05:05:28.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.674+0000 7f6f89241740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf610c1c70 0x55bf610cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:28.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.674+0000 7f6f89241740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf610c2590 0x55bf610ca560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:28.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.674+0000 7f6f89241740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf610cce50 0x55bf610e4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:28.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.674+0000 7f6f89241740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6f801b4480 con 0x55bf610c1c70 2024-09-19T05:05:28.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.674+0000 7f6f89241740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6f801b5fc0 con 0x55bf610cce50 2024-09-19T05:05:28.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.674+0000 7f6f89241740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6f801b4b50 con 0x55bf610c2590 2024-09-19T05:05:28.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.674+0000 7f6f7f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf610c1c70 0x55bf610cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:28.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.674+0000 7f6f7ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf610c2590 0x55bf610ca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:28.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.674+0000 7f6f7ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf610c2590 0x55bf610ca560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:49918/0 (socket says 172.21.15.43:49918) 2024-09-19T05:05:28.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.674+0000 7f6f7ea00640 1 -- 172.21.15.43:0/1820231049 learned_addr learned my addr 172.21.15.43:0/1820231049 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:05:28.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.674+0000 7f6f7fe00640 1 --2- 172.21.15.43:0/1820231049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf610cce50 0x55bf610e4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:28.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.675+0000 7f6f7f400640 1 -- 172.21.15.43:0/1820231049 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf610c2590 msgr2=0x55bf610ca560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:28.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.675+0000 7f6f7f400640 1 --2- 172.21.15.43:0/1820231049 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf610c2590 0x55bf610ca560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:28.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.675+0000 7f6f7f400640 1 -- 172.21.15.43:0/1820231049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf610cce50 msgr2=0x55bf610e4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:28.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.675+0000 7f6f7f400640 1 --2- 172.21.15.43:0/1820231049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf610cce50 0x55bf610e4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:28.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.675+0000 7f6f7f400640 1 -- 172.21.15.43:0/1820231049 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf610be030 con 0x55bf610c1c70 2024-09-19T05:05:28.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.675+0000 7f6f7f400640 1 --2- 172.21.15.43:0/1820231049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf610c1c70 0x55bf610cbfc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x55bf610bf060 tx=0x7f6f700f2430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f215b59f164cdc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:28.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.675+0000 7f6f6be00640 1 -- 172.21.15.43:0/1820231049 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f70031de0 con 0x55bf610c1c70 2024-09-19T05:05:28.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.676+0000 7f6f6be00640 1 -- 172.21.15.43:0/1820231049 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f70009510 con 0x55bf610c1c70 2024-09-19T05:05:28.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.676+0000 7f6f6be00640 1 -- 172.21.15.43:0/1820231049 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f700ba390 con 0x55bf610c1c70 2024-09-19T05:05:28.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.676+0000 7f6f89241740 1 -- 172.21.15.43:0/1820231049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf610c1c70 msgr2=0x55bf610cbfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:28.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.676+0000 7f6f89241740 1 --2- 172.21.15.43:0/1820231049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf610c1c70 0x55bf610cbfc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x55bf610bf060 tx=0x7f6f700f2430 comp rx=0 tx=0).stop 2024-09-19T05:05:28.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.676+0000 7f6f89241740 1 -- 172.21.15.43:0/1820231049 shutdown_connections 2024-09-19T05:05:28.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.676+0000 7f6f89241740 1 --2- 172.21.15.43:0/1820231049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf610cce50 0x55bf610e4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:28.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.676+0000 7f6f89241740 1 --2- 172.21.15.43:0/1820231049 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf610c2590 0x55bf610ca560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:28.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.676+0000 7f6f89241740 1 --2- 172.21.15.43:0/1820231049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf610c1c70 0x55bf610cbfc0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:28.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.676+0000 7f6f89241740 1 -- 172.21.15.43:0/1820231049 >> 172.21.15.43:0/1820231049 conn(0x55bf610be200 msgr2=0x55bf610bd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:28.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.677+0000 7f6f89241740 1 -- 172.21.15.43:0/1820231049 shutdown_connections 2024-09-19T05:05:28.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.677+0000 7f6f89241740 1 -- 172.21.15.43:0/1820231049 wait complete. 2024-09-19T05:05:28.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.677+0000 7f6f89241740 1 -- start start 2024-09-19T05:05:28.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.677+0000 7f6f89241740 1 Processor -- start 2024-09-19T05:05:28.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.678+0000 7f6f89241740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf610c1c70 0x55bf610d5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:28.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.678+0000 7f6f89241740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf610c2590 0x55bf610d0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:28.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.678+0000 7f6f89241740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf610cce50 0x55bf610d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:28.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.678+0000 7f6f89241740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6f801b58f0 con 0x55bf610c1c70 2024-09-19T05:05:28.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.678+0000 7f6f89241740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6f801b5220 con 0x55bf610cce50 2024-09-19T05:05:28.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.678+0000 7f6f89241740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6f801b2460 con 0x55bf610c2590 2024-09-19T05:05:28.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.678+0000 7f6f89241740 10 client.0 _ll_register_callbacks cb 0x7f6f700f3960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:05:28.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.679+0000 7f6f7ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf610c2590 0x55bf610d0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:28.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.679+0000 7f6f7f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf610c1c70 0x55bf610d5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:28.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.679+0000 7f6f7fe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf610cce50 0x55bf610d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:28.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.679+0000 7f6f7ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf610c2590 0x55bf610d0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:49920/0 (socket says 172.21.15.43:49920) 2024-09-19T05:05:28.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.679+0000 7f6f7ea00640 1 -- 172.21.15.43:0/3365418732 learned_addr learned my addr 172.21.15.43:0/3365418732 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:05:28.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.679+0000 7f6f7fe00640 1 -- 172.21.15.43:0/3365418732 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf610c2590 msgr2=0x55bf610d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:28.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.679+0000 7f6f7fe00640 1 --2- 172.21.15.43:0/3365418732 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf610c2590 0x55bf610d0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:28.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.679+0000 7f6f7fe00640 1 -- 172.21.15.43:0/3365418732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf610c1c70 msgr2=0x55bf610d5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:28.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.679+0000 7f6f7fe00640 1 --2- 172.21.15.43:0/3365418732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf610c1c70 0x55bf610d5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:28.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.679+0000 7f6f7fe00640 1 -- 172.21.15.43:0/3365418732 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf610b9770 con 0x55bf610cce50 2024-09-19T05:05:28.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.679+0000 7f6f7fe00640 1 --2- 172.21.15.43:0/3365418732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf610cce50 0x55bf610d0eb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6f78009b20 tx=0x7f6f78004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ebb72686dbc4589 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:28.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.679+0000 7f6f69600640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:05:28.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.680+0000 7f6f69600640 1 -- 172.21.15.43:0/3365418732 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7802f060 con 0x55bf610cce50 2024-09-19T05:05:28.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.680+0000 7f6f69600640 1 -- 172.21.15.43:0/3365418732 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f6f7801f850 con 0x55bf610cce50 2024-09-19T05:05:28.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.680+0000 7f6f89241740 1 -- 172.21.15.43:0/3365418732 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f70009510 con 0x55bf610cce50 2024-09-19T05:05:28.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.680+0000 7f6f69600640 1 -- 172.21.15.43:0/3365418732 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6f7801fa50 con 0x55bf610cce50 2024-09-19T05:05:28.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.680+0000 7f6f89241740 10 client.25382 fetch_fsmap learned FSMap version 29 2024-09-19T05:05:28.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.680+0000 7f6f89241740 1 -- 172.21.15.43:0/3365418732 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bf61147740 con 0x55bf610cce50 2024-09-19T05:05:28.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.681+0000 7f6f69600640 1 -- 172.21.15.43:0/3365418732 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f70009510 con 0x55bf610cce50 2024-09-19T05:05:28.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.681+0000 7f6f69600640 1 -- 172.21.15.43:0/3365418732 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6f78029070 con 0x55bf610cce50 2024-09-19T05:05:28.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.681+0000 7f6f89241740 10 client.25382 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:05:28.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.681+0000 7f6f89241740 10 client.25382 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-19T05:05:28.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.681+0000 7f6f89241740 20 client.25382 populate_metadata read hostname 'smithi043' 2024-09-19T05:05:28.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.681+0000 7f6f89241740 1 --2- 172.21.15.43:0/3365418732 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55bf61147f20 0x55bf6114a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:28.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.681+0000 7f6f89241740 4 client.25382 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:05:28.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.681+0000 7f6f89241740 1 -- 172.21.15.43:0/3365418732 --> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bf6102fe60 con 0x55bf61147f20 2024-09-19T05:05:28.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.682+0000 7f6f7f400640 1 --2- 172.21.15.43:0/3365418732 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55bf61147f20 0x55bf6114a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:28.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.682+0000 7f6f7f400640 1 --2- 172.21.15.43:0/3365418732 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55bf61147f20 0x55bf6114a340 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e75c52fa9f29eb5d server_cookie=8d2f1937e9874e8d in_seq=0 out_seq=0 2024-09-19T05:05:28.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.682+0000 7f6f69600640 10 client.25382 ms_handle_connect on v2:172.21.15.151:6836/3960378010 2024-09-19T05:05:28.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.682+0000 7f6f69600640 1 -- 172.21.15.43:0/3365418732 <== mds.4 v2:172.21.15.151:6836/3960378010 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bf6102fe60 con 0x55bf61147f20 2024-09-19T05:05:28.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.682+0000 7f6f69600640 10 client.25382 handle_command_reply: tid=0 2024-09-19T05:05:28.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.682+0000 7f6f89241740 1 client.25382 shutdown 2024-09-19T05:05:28.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.683+0000 7f6f89241740 1 -- 172.21.15.43:0/3365418732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf610cce50 msgr2=0x55bf610d0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:28.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.683+0000 7f6f89241740 1 --2- 172.21.15.43:0/3365418732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf610cce50 0x55bf610d0eb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6f78009b20 tx=0x7f6f78004ab0 comp rx=0 tx=0).stop 2024-09-19T05:05:28.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.683+0000 7f6f89241740 1 -- 172.21.15.43:0/3365418732 shutdown_connections 2024-09-19T05:05:28.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.683+0000 7f6f89241740 1 --2- 172.21.15.43:0/3365418732 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55bf61147f20 0x55bf6114a340 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:28.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.683+0000 7f6f69600640 0 client.25382 ms_handle_reset on v2:172.21.15.151:6836/3960378010 2024-09-19T05:05:28.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.683+0000 7f6f89241740 1 --2- 172.21.15.43:0/3365418732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf610cce50 0x55bf610d0eb0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:28.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.683+0000 7f6f89241740 1 --2- 172.21.15.43:0/3365418732 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf610c2590 0x55bf610d0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:28.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.683+0000 7f6f89241740 1 --2- 172.21.15.43:0/3365418732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf610c1c70 0x55bf610d5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:28.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.683+0000 7f6f89241740 1 -- 172.21.15.43:0/3365418732 >> 172.21.15.43:0/3365418732 conn(0x55bf610be200 msgr2=0x55bf610bd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:28.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.683+0000 7f6f89241740 1 -- 172.21.15.43:0/3365418732 shutdown_connections 2024-09-19T05:05:28.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.683+0000 7f6f89241740 1 -- 172.21.15.43:0/3365418732 wait complete. 2024-09-19T05:05:28.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.683+0000 7f6f89241740 20 client.25382 trim_cache size 0 max 16384 2024-09-19T05:05:28.687 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:05:28.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.684+0000 7f6f86800640 1 -- 172.21.15.43:0/1508901729 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f6f640a35e0 msgr2=0x7f6f640a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:28.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.684+0000 7f6f86800640 1 --2- 172.21.15.43:0/1508901729 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f6f640a35e0 0x7f6f640a5aa0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6f78005fd0 tx=0x7f6f78009500 comp rx=0 tx=0).stop 2024-09-19T05:05:28.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.684+0000 7f6f86800640 1 -- 172.21.15.43:0/1508901729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f801049a0 msgr2=0x7f6f8010ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:28.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.684+0000 7f6f86800640 1 --2- 172.21.15.43:0/1508901729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f801049a0 0x7f6f8010ddb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6f7002aa00 tx=0x7f6f70003c10 comp rx=0 tx=0).stop 2024-09-19T05:05:28.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.685+0000 7f6f86800640 1 -- 172.21.15.43:0/1508901729 shutdown_connections 2024-09-19T05:05:28.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.685+0000 7f6f86800640 1 --2- 172.21.15.43:0/1508901729 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f6f640a35e0 0x7f6f640a5aa0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:28.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.685+0000 7f6f86800640 1 --2- 172.21.15.43:0/1508901729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f8010e830 0x7f6f801aaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:28.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.685+0000 7f6f86800640 1 --2- 172.21.15.43:0/1508901729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f80109120 0x7f6f8010e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:28.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.685+0000 7f6f86800640 1 --2- 172.21.15.43:0/1508901729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f801049a0 0x7f6f8010ddb0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:28.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.685+0000 7f6f86800640 1 -- 172.21.15.43:0/1508901729 >> 172.21.15.43:0/1508901729 conn(0x7f6f800ffa90 msgr2=0x7f6f80100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:28.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.685+0000 7f6f86800640 1 -- 172.21.15.43:0/1508901729 shutdown_connections 2024-09-19T05:05:28.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:28.685+0000 7f6f86800640 1 -- 172.21.15.43:0/1508901729 wait complete. 2024-09-19T05:05:29.590 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T05:05:30.033 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:05:30.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:30 smithi043 ceph-mon[30800]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:05:30.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:30 smithi151 ceph-mon[35425]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:05:30.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:30 smithi096 ceph-mon[35085]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:05:30.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.666+0000 7f8ed9800640 1 -- 172.21.15.43:0/208866727 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ed41047e0 msgr2=0x7f8ed4108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:30.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.666+0000 7f8ed9800640 1 --2- 172.21.15.43:0/208866727 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ed41047e0 0x7f8ed4108830 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0009860 tx=0x7f8ec002a340 comp rx=0 tx=0).stop 2024-09-19T05:05:30.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.666+0000 7f8ed9800640 1 -- 172.21.15.43:0/208866727 shutdown_connections 2024-09-19T05:05:30.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.666+0000 7f8ed9800640 1 --2- 172.21.15.43:0/208866727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ed4108f60 0x7f8ed4110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:30.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.666+0000 7f8ed9800640 1 --2- 172.21.15.43:0/208866727 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ed41047e0 0x7f8ed4108830 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:30.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.666+0000 7f8ed9800640 1 --2- 172.21.15.43:0/208866727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ed4103e30 0x7f8ed4104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:30.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.666+0000 7f8ed9800640 1 -- 172.21.15.43:0/208866727 >> 172.21.15.43:0/208866727 conn(0x7f8ed40ff910 msgr2=0x7f8ed4101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:30.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.666+0000 7f8ed9800640 1 -- 172.21.15.43:0/208866727 shutdown_connections 2024-09-19T05:05:30.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.667+0000 7f8ed9800640 1 -- 172.21.15.43:0/208866727 wait complete. 2024-09-19T05:05:30.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.668+0000 7f8ed9800640 1 Processor -- start 2024-09-19T05:05:30.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.668+0000 7f8ed9800640 1 -- start start 2024-09-19T05:05:30.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.669+0000 7f8ed9800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ed4103e30 0x7f8ed41109f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:30.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.669+0000 7f8ed9800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ed41047e0 0x7f8ed410daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:30.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.669+0000 7f8ed9800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ed4108f60 0x7f8ed410e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:30.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.669+0000 7f8ed9800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed4113fc0 con 0x7f8ed4108f60 2024-09-19T05:05:30.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.669+0000 7f8ed9800640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed4113e40 con 0x7f8ed4103e30 2024-09-19T05:05:30.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.669+0000 7f8ed9800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed4114140 con 0x7f8ed41047e0 2024-09-19T05:05:30.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.669+0000 7f8ed3e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ed4103e30 0x7f8ed41109f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:30.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.669+0000 7f8ed3e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ed4103e30 0x7f8ed41109f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:41122/0 (socket says 172.21.15.43:41122) 2024-09-19T05:05:30.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.669+0000 7f8ed3e00640 1 -- 172.21.15.43:0/2087746098 learned_addr learned my addr 172.21.15.43:0/2087746098 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:05:30.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.669+0000 7f8ed3400640 1 --2- 172.21.15.43:0/2087746098 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ed41047e0 0x7f8ed410daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:30.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.669+0000 7f8ed8e00640 1 --2- 172.21.15.43:0/2087746098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ed4108f60 0x7f8ed410e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:30.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.670+0000 7f8ed3e00640 1 -- 172.21.15.43:0/2087746098 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ed41047e0 msgr2=0x7f8ed410daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:30.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.670+0000 7f8ed3e00640 1 --2- 172.21.15.43:0/2087746098 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ed41047e0 0x7f8ed410daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:30.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.670+0000 7f8ed3e00640 1 -- 172.21.15.43:0/2087746098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ed4108f60 msgr2=0x7f8ed410e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:30.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.670+0000 7f8ed3e00640 1 --2- 172.21.15.43:0/2087746098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ed4108f60 0x7f8ed410e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:30.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.670+0000 7f8ed3e00640 1 -- 172.21.15.43:0/2087746098 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ed41aacc0 con 0x7f8ed4103e30 2024-09-19T05:05:30.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.670+0000 7f8ed3e00640 1 --2- 172.21.15.43:0/2087746098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ed4103e30 0x7f8ed41109f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400d770 tx=0x7f8ec400dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:30.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.671+0000 7f8ed0c00640 1 -- 172.21.15.43:0/2087746098 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4017070 con 0x7f8ed4103e30 2024-09-19T05:05:30.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.671+0000 7f8ed0c00640 1 -- 172.21.15.43:0/2087746098 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ec4005500 con 0x7f8ed4103e30 2024-09-19T05:05:30.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.671+0000 7f8ed0c00640 1 -- 172.21.15.43:0/2087746098 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4004e00 con 0x7f8ed4103e30 2024-09-19T05:05:30.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.671+0000 7f8ed9800640 1 -- 172.21.15.43:0/2087746098 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ed41aae90 con 0x7f8ed4103e30 2024-09-19T05:05:30.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.671+0000 7f8ed9800640 1 -- 172.21.15.43:0/2087746098 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ed41ab290 con 0x7f8ed4103e30 2024-09-19T05:05:30.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.675+0000 7f8edd9e1740 1 Processor -- start 2024-09-19T05:05:30.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.676+0000 7f8edd9e1740 1 -- start start 2024-09-19T05:05:30.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.676+0000 7f8edd9e1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b9cf14e40 0x561b9cf15220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:30.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.676+0000 7f8edd9e1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b9cf15760 0x561b9cf1fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:30.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.676+0000 7f8edd9e1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b9cf20020 0x561b9cf22410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:30.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.676+0000 7f8ed0c00640 1 -- 172.21.15.43:0/2087746098 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f8ec4004fa0 con 0x7f8ed4103e30 2024-09-19T05:05:30.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.676+0000 7f8edd9e1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561b9ce7b3e0 con 0x561b9cf14e40 2024-09-19T05:05:30.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.676+0000 7f8edd9e1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561b9ce22d80 con 0x561b9cf15760 2024-09-19T05:05:30.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.676+0000 7f8edd9e1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561b9ce773a0 con 0x561b9cf20020 2024-09-19T05:05:30.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.677+0000 7f8ed3e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b9cf15760 0x561b9cf1fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:30.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.677+0000 7f8ed8e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b9cf14e40 0x561b9cf15220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:30.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.677+0000 7f8ed8e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b9cf14e40 0x561b9cf15220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36634/0 (socket says 172.21.15.43:36634) 2024-09-19T05:05:30.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.677+0000 7f8ed8e00640 1 -- 172.21.15.43:0/1638489114 learned_addr learned my addr 172.21.15.43:0/1638489114 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:05:30.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.677+0000 7f8ed3400640 1 --2- 172.21.15.43:0/1638489114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b9cf20020 0x561b9cf22410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:30.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.677+0000 7f8ed3400640 1 -- 172.21.15.43:0/1638489114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b9cf15760 msgr2=0x561b9cf1fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:30.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.677+0000 7f8ed3400640 1 --2- 172.21.15.43:0/1638489114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b9cf15760 0x561b9cf1fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:30.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.677+0000 7f8ed3400640 1 -- 172.21.15.43:0/1638489114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b9cf14e40 msgr2=0x561b9cf15220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:30.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.678+0000 7f8ed3400640 1 --2- 172.21.15.43:0/1638489114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b9cf14e40 0x561b9cf15220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:30.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.678+0000 7f8ed3400640 1 -- 172.21.15.43:0/1638489114 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b9ce41ae0 con 0x561b9cf20020 2024-09-19T05:05:30.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.678+0000 7f8ed8e00640 1 --2- 172.21.15.43:0/1638489114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b9cf14e40 0x561b9cf15220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:05:30.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.678+0000 7f8ed3400640 1 --2- 172.21.15.43:0/1638489114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b9cf20020 0x561b9cf22410 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0035040 tx=0x7f8ec0009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be152a4546c298c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:30.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.679+0000 7f8ed0c00640 1 --2- 172.21.15.43:0/2087746098 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f8eb80a35e0 0x7f8eb80a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:30.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.679+0000 7f8ed0c00640 1 -- 172.21.15.43:0/2087746098 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f8ec40cf930 con 0x7f8ed4103e30 2024-09-19T05:05:30.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.679+0000 7f8ec9600640 1 -- 172.21.15.43:0/1638489114 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0041020 con 0x561b9cf20020 2024-09-19T05:05:30.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.679+0000 7f8ec9600640 1 -- 172.21.15.43:0/1638489114 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ec0030e50 con 0x561b9cf20020 2024-09-19T05:05:30.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.679+0000 7f8ec9600640 1 -- 172.21.15.43:0/1638489114 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0037040 con 0x561b9cf20020 2024-09-19T05:05:30.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.680+0000 7f8ed8e00640 1 --2- 172.21.15.43:0/2087746098 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f8eb80a35e0 0x7f8eb80a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:30.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.680+0000 7f8edd9e1740 1 -- 172.21.15.43:0/1638489114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b9cf20020 msgr2=0x561b9cf22410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:30.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.680+0000 7f8edd9e1740 1 --2- 172.21.15.43:0/1638489114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b9cf20020 0x561b9cf22410 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0035040 tx=0x7f8ec0009510 comp rx=0 tx=0).stop 2024-09-19T05:05:30.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.681+0000 7f8edd9e1740 1 -- 172.21.15.43:0/1638489114 shutdown_connections 2024-09-19T05:05:30.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.681+0000 7f8edd9e1740 1 --2- 172.21.15.43:0/1638489114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b9cf20020 0x561b9cf22410 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:30.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.681+0000 7f8edd9e1740 1 --2- 172.21.15.43:0/1638489114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b9cf15760 0x561b9cf1fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:30.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.681+0000 7f8edd9e1740 1 --2- 172.21.15.43:0/1638489114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b9cf14e40 0x561b9cf15220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:30.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.681+0000 7f8edd9e1740 1 -- 172.21.15.43:0/1638489114 >> 172.21.15.43:0/1638489114 conn(0x561b9cf113d0 msgr2=0x561b9cf117b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:30.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.681+0000 7f8ed8e00640 1 --2- 172.21.15.43:0/2087746098 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f8eb80a35e0 0x7f8eb80a5aa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc005c40 tx=0x7f8ecc010040 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:30.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.681+0000 7f8edd9e1740 1 -- 172.21.15.43:0/1638489114 shutdown_connections 2024-09-19T05:05:30.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.681+0000 7f8edd9e1740 1 -- 172.21.15.43:0/1638489114 wait complete. 2024-09-19T05:05:30.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.682+0000 7f8edd9e1740 1 -- start start 2024-09-19T05:05:30.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.682+0000 7f8edd9e1740 1 Processor -- start 2024-09-19T05:05:30.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.683+0000 7f8edd9e1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b9cf14e40 0x561b9cf1cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:30.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.683+0000 7f8edd9e1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b9cf15760 0x561b9cf1d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:30.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.683+0000 7f8edd9e1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b9cf20020 0x561b9cf1da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:30.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.683+0000 7f8edd9e1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561b9cf22bf0 con 0x561b9cf15760 2024-09-19T05:05:30.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.683+0000 7f8edd9e1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561b9ce360f0 con 0x561b9cf20020 2024-09-19T05:05:30.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.683+0000 7f8edd9e1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561b9cf22d70 con 0x561b9cf14e40 2024-09-19T05:05:30.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.683+0000 7f8edd9e1740 10 client.0 _ll_register_callbacks cb 0x561b9ce1dbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:05:30.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.683+0000 7f8ed3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b9cf14e40 0x561b9cf1cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:30.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.683+0000 7f8ed3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b9cf14e40 0x561b9cf1cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:49964/0 (socket says 172.21.15.43:49964) 2024-09-19T05:05:30.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.683+0000 7f8ed3e00640 1 -- 172.21.15.43:0/3500846240 learned_addr learned my addr 172.21.15.43:0/3500846240 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:05:30.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.683+0000 7f8ed8e00640 1 --2- 172.21.15.43:0/3500846240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b9cf20020 0x561b9cf1da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:30.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.684+0000 7f8ed3400640 1 --2- 172.21.15.43:0/3500846240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b9cf15760 0x561b9cf1d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:30.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.684+0000 7f8ed3e00640 1 -- 172.21.15.43:0/3500846240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b9cf20020 msgr2=0x561b9cf1da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:30.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.684+0000 7f8ed3e00640 1 --2- 172.21.15.43:0/3500846240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b9cf20020 0x561b9cf1da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:30.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.684+0000 7f8ed3e00640 1 -- 172.21.15.43:0/3500846240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b9cf15760 msgr2=0x561b9cf1d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:30.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.684+0000 7f8ed3e00640 1 --2- 172.21.15.43:0/3500846240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b9cf15760 0x561b9cf1d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:30.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.684+0000 7f8ed3e00640 1 -- 172.21.15.43:0/3500846240 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b9cbec3c0 con 0x561b9cf14e40 2024-09-19T05:05:30.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.684+0000 7f8ed3400640 1 --2- 172.21.15.43:0/3500846240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b9cf15760 0x561b9cf1d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:05:30.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.684+0000 7f8ed3e00640 1 --2- 172.21.15.43:0/3500846240 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b9cf14e40 0x561b9cf1cfd0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40ccf40 tx=0x7f8ec409b3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f00b2da91bb8ed3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:30.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.684+0000 7f8ebe000640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:05:30.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.685+0000 7f8ebe000640 1 -- 172.21.15.43:0/3500846240 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4020070 con 0x561b9cf14e40 2024-09-19T05:05:30.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.685+0000 7f8edd9e1740 1 -- 172.21.15.43:0/3500846240 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ec0030e50 con 0x561b9cf14e40 2024-09-19T05:05:30.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.685+0000 7f8ebe000640 1 -- 172.21.15.43:0/3500846240 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f8ec4098100 con 0x561b9cf14e40 2024-09-19T05:05:30.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.685+0000 7f8ebe000640 1 -- 172.21.15.43:0/3500846240 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8ec40d1e60 con 0x561b9cf14e40 2024-09-19T05:05:30.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.685+0000 7f8edd9e1740 10 client.25357 fetch_fsmap learned FSMap version 29 2024-09-19T05:05:30.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.685+0000 7f8edd9e1740 1 -- 172.21.15.43:0/3500846240 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561b9cf2ed40 con 0x561b9cf14e40 2024-09-19T05:05:30.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.686+0000 7f8ebe000640 1 -- 172.21.15.43:0/3500846240 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ec0030e50 con 0x561b9cf14e40 2024-09-19T05:05:30.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.686+0000 7f8ebe000640 1 -- 172.21.15.43:0/3500846240 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8ec409aee0 con 0x561b9cf14e40 2024-09-19T05:05:30.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.686+0000 7f8edd9e1740 10 client.25357 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:05:30.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.686+0000 7f8edd9e1740 10 client.25357 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T05:05:30.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.686+0000 7f8edd9e1740 20 client.25357 populate_metadata read hostname 'smithi043' 2024-09-19T05:05:30.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.686+0000 7f8edd9e1740 1 --2- 172.21.15.43:0/3500846240 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x561b9cf30150 0x561b9cf32540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:30.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.686+0000 7f8edd9e1740 4 client.25357 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:05:30.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.686+0000 7f8edd9e1740 1 -- 172.21.15.43:0/3500846240 --> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b9ce88950 con 0x561b9cf30150 2024-09-19T05:05:30.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.687+0000 7f8ed3400640 1 --2- 172.21.15.43:0/3500846240 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x561b9cf30150 0x561b9cf32540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:30.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.688+0000 7f8ed3400640 1 --2- 172.21.15.43:0/3500846240 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x561b9cf30150 0x561b9cf32540 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=32855cf5591f3696 server_cookie=75bce655b27a57e0 in_seq=0 out_seq=0 2024-09-19T05:05:30.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.688+0000 7f8ebe000640 10 client.25357 ms_handle_connect on v2:172.21.15.96:6836/3978388286 2024-09-19T05:05:30.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.689+0000 7f8ebe000640 1 -- 172.21.15.43:0/3500846240 <== mds.3 v2:172.21.15.96:6836/3978388286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561b9ce88950 con 0x561b9cf30150 2024-09-19T05:05:30.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.689+0000 7f8ebe000640 10 client.25357 handle_command_reply: tid=0 2024-09-19T05:05:30.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.689+0000 7f8edd9e1740 1 client.25357 shutdown 2024-09-19T05:05:30.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.689+0000 7f8edd9e1740 1 -- 172.21.15.43:0/3500846240 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b9cf14e40 msgr2=0x561b9cf1cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:30.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.689+0000 7f8edd9e1740 1 --2- 172.21.15.43:0/3500846240 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b9cf14e40 0x561b9cf1cfd0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40ccf40 tx=0x7f8ec409b3f0 comp rx=0 tx=0).stop 2024-09-19T05:05:30.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.689+0000 7f8edd9e1740 1 -- 172.21.15.43:0/3500846240 shutdown_connections 2024-09-19T05:05:30.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.689+0000 7f8edd9e1740 1 --2- 172.21.15.43:0/3500846240 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x561b9cf30150 0x561b9cf32540 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:30.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.689+0000 7f8edd9e1740 1 --2- 172.21.15.43:0/3500846240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b9cf20020 0x561b9cf1da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:30.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.689+0000 7f8edd9e1740 1 --2- 172.21.15.43:0/3500846240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b9cf15760 0x561b9cf1d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:30.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.689+0000 7f8edd9e1740 1 --2- 172.21.15.43:0/3500846240 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b9cf14e40 0x561b9cf1cfd0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:30.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.689+0000 7f8edd9e1740 1 -- 172.21.15.43:0/3500846240 >> 172.21.15.43:0/3500846240 conn(0x561b9cf113d0 msgr2=0x561b9cf17100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:30.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.690+0000 7f8ebe000640 0 client.25357 ms_handle_reset on v2:172.21.15.96:6836/3978388286 2024-09-19T05:05:30.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.690+0000 7f8edd9e1740 1 -- 172.21.15.43:0/3500846240 shutdown_connections 2024-09-19T05:05:30.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.690+0000 7f8edd9e1740 1 -- 172.21.15.43:0/3500846240 wait complete. 2024-09-19T05:05:30.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.690+0000 7f8edd9e1740 20 client.25357 trim_cache size 0 max 16384 2024-09-19T05:05:30.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.718+0000 7f8edd9e1740 1 Processor -- start 2024-09-19T05:05:30.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.718+0000 7f8edd9e1740 1 -- start start 2024-09-19T05:05:30.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.718+0000 7f8edd9e1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b9cf14e40 0x561b9cf1c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:30.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.718+0000 7f8edd9e1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b9cf15760 0x561b9cf1d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:30.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.718+0000 7f8edd9e1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b9cf20020 0x561b9cf37e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:30.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.718+0000 7f8edd9e1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed41b3cb0 con 0x561b9cf14e40 2024-09-19T05:05:30.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.718+0000 7f8edd9e1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed41b57f0 con 0x561b9cf20020 2024-09-19T05:05:30.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.718+0000 7f8edd9e1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed41b4380 con 0x561b9cf15760 2024-09-19T05:05:30.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.718+0000 7f8ed3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b9cf14e40 0x561b9cf1c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:30.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.719+0000 7f8ed3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b9cf14e40 0x561b9cf1c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36660/0 (socket says 172.21.15.43:36660) 2024-09-19T05:05:30.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.719+0000 7f8ed3e00640 1 -- 172.21.15.43:0/3332886281 learned_addr learned my addr 172.21.15.43:0/3332886281 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:05:30.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.719+0000 7f8ed3400640 1 --2- 172.21.15.43:0/3332886281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b9cf15760 0x561b9cf1d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:30.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.719+0000 7f8ed8e00640 1 --2- 172.21.15.43:0/3332886281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b9cf20020 0x561b9cf37e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:30.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.719+0000 7f8ed3e00640 1 -- 172.21.15.43:0/3332886281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b9cf15760 msgr2=0x561b9cf1d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:30.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.719+0000 7f8ed3e00640 1 --2- 172.21.15.43:0/3332886281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b9cf15760 0x561b9cf1d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:30.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.719+0000 7f8ed3e00640 1 -- 172.21.15.43:0/3332886281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b9cf20020 msgr2=0x561b9cf37e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:30.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.719+0000 7f8ed3e00640 1 --2- 172.21.15.43:0/3332886281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b9cf20020 0x561b9cf37e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:30.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.719+0000 7f8ed3e00640 1 -- 172.21.15.43:0/3332886281 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b9cf18bb0 con 0x561b9cf14e40 2024-09-19T05:05:30.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.720+0000 7f8ed3400640 1 --2- 172.21.15.43:0/3332886281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b9cf15760 0x561b9cf1d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:05:30.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.720+0000 7f8ed3e00640 1 --2- 172.21.15.43:0/3332886281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b9cf14e40 0x561b9cf1c890 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8ec409b790 tx=0x7f8ec409b520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57dc5045a2c466f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:30.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.720+0000 7f8ec8c00640 1 -- 172.21.15.43:0/3332886281 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec40b3070 con 0x561b9cf14e40 2024-09-19T05:05:30.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.720+0000 7f8ec8c00640 1 -- 172.21.15.43:0/3332886281 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ec40046c0 con 0x561b9cf14e40 2024-09-19T05:05:30.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.720+0000 7f8ec8c00640 1 -- 172.21.15.43:0/3332886281 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4020070 con 0x561b9cf14e40 2024-09-19T05:05:30.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.722+0000 7f8edd9e1740 1 -- 172.21.15.43:0/3332886281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b9cf14e40 msgr2=0x561b9cf1c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:30.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.722+0000 7f8edd9e1740 1 --2- 172.21.15.43:0/3332886281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b9cf14e40 0x561b9cf1c890 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8ec409b790 tx=0x7f8ec409b520 comp rx=0 tx=0).stop 2024-09-19T05:05:30.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.722+0000 7f8edd9e1740 1 -- 172.21.15.43:0/3332886281 shutdown_connections 2024-09-19T05:05:30.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.722+0000 7f8edd9e1740 1 --2- 172.21.15.43:0/3332886281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b9cf20020 0x561b9cf37e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:30.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.722+0000 7f8edd9e1740 1 --2- 172.21.15.43:0/3332886281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b9cf15760 0x561b9cf1d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:30.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.722+0000 7f8edd9e1740 1 --2- 172.21.15.43:0/3332886281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b9cf14e40 0x561b9cf1c890 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:30.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.722+0000 7f8edd9e1740 1 -- 172.21.15.43:0/3332886281 >> 172.21.15.43:0/3332886281 conn(0x561b9cf113d0 msgr2=0x561b9cf1cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:30.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.722+0000 7f8edd9e1740 1 -- 172.21.15.43:0/3332886281 shutdown_connections 2024-09-19T05:05:30.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.722+0000 7f8edd9e1740 1 -- 172.21.15.43:0/3332886281 wait complete. 2024-09-19T05:05:30.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.723+0000 7f8edd9e1740 1 -- start start 2024-09-19T05:05:30.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.723+0000 7f8edd9e1740 1 Processor -- start 2024-09-19T05:05:30.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.724+0000 7f8edd9e1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b9cf14e40 0x561b9cf28850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:30.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.724+0000 7f8edd9e1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b9cf15760 0x561b9cf23b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:30.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.724+0000 7f8edd9e1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b9cf20020 0x561b9cf24190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:30.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.724+0000 7f8edd9e1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed41b5120 con 0x561b9cf14e40 2024-09-19T05:05:30.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.724+0000 7f8edd9e1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed41b4a50 con 0x561b9cf15760 2024-09-19T05:05:30.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.724+0000 7f8edd9e1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed41b1c90 con 0x561b9cf20020 2024-09-19T05:05:30.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.724+0000 7f8edd9e1740 10 client.0 _ll_register_callbacks cb 0x7f8ec40056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:05:30.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.724+0000 7f8ed3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b9cf14e40 0x561b9cf28850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:30.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.724+0000 7f8ed3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b9cf14e40 0x561b9cf28850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36668/0 (socket says 172.21.15.43:36668) 2024-09-19T05:05:30.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.724+0000 7f8ed3e00640 1 -- 172.21.15.43:0/2372380807 learned_addr learned my addr 172.21.15.43:0/2372380807 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:05:30.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.725+0000 7f8ed8e00640 1 --2- 172.21.15.43:0/2372380807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b9cf20020 0x561b9cf24190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:30.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.725+0000 7f8ed3400640 1 --2- 172.21.15.43:0/2372380807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b9cf15760 0x561b9cf23b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:30.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.725+0000 7f8ed3e00640 1 -- 172.21.15.43:0/2372380807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b9cf20020 msgr2=0x561b9cf24190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:30.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.725+0000 7f8ed3e00640 1 --2- 172.21.15.43:0/2372380807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b9cf20020 0x561b9cf24190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:30.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.725+0000 7f8ed3e00640 1 -- 172.21.15.43:0/2372380807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b9cf15760 msgr2=0x561b9cf23b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:30.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.725+0000 7f8ed3e00640 1 --2- 172.21.15.43:0/2372380807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b9cf15760 0x561b9cf23b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:30.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.725+0000 7f8ed3e00640 1 -- 172.21.15.43:0/2372380807 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b9cf249c0 con 0x561b9cf14e40 2024-09-19T05:05:30.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.725+0000 7f8ed3400640 1 --2- 172.21.15.43:0/2372380807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b9cf15760 0x561b9cf23b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:05:30.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.725+0000 7f8ed3e00640 1 --2- 172.21.15.43:0/2372380807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b9cf14e40 0x561b9cf28850 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4097ef0 tx=0x7f8ec409b3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1568ee4f6247e918 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:30.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.726+0000 7f8ebe000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:05:30.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.726+0000 7f8ebe000640 1 -- 172.21.15.43:0/2372380807 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec40b3050 con 0x561b9cf14e40 2024-09-19T05:05:30.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.726+0000 7f8ebe000640 1 -- 172.21.15.43:0/2372380807 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f8ec40ba2b0 con 0x561b9cf14e40 2024-09-19T05:05:30.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.726+0000 7f8edd9e1740 1 -- 172.21.15.43:0/2372380807 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561b9ce82e60 con 0x561b9cf14e40 2024-09-19T05:05:30.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.726+0000 7f8ebe000640 1 -- 172.21.15.43:0/2372380807 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8ec4020070 con 0x561b9cf14e40 2024-09-19T05:05:30.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.727+0000 7f8edd9e1740 10 client.15411 fetch_fsmap learned FSMap version 29 2024-09-19T05:05:30.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.727+0000 7f8edd9e1740 1 -- 172.21.15.43:0/2372380807 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561b9cf249c0 con 0x561b9cf14e40 2024-09-19T05:05:30.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.727+0000 7f8ebe000640 1 -- 172.21.15.43:0/2372380807 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x561b9ce82e60 con 0x561b9cf14e40 2024-09-19T05:05:30.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.727+0000 7f8ebe000640 1 -- 172.21.15.43:0/2372380807 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8ec40b4be0 con 0x561b9cf14e40 2024-09-19T05:05:30.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.728+0000 7f8edd9e1740 10 client.15411 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:05:30.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.728+0000 7f8edd9e1740 10 client.15411 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T05:05:30.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.728+0000 7f8edd9e1740 20 client.15411 populate_metadata read hostname 'smithi043' 2024-09-19T05:05:30.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.728+0000 7f8edd9e1740 1 --2- 172.21.15.43:0/2372380807 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x561b9cf9b110 0x561b9cf9d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:30.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.728+0000 7f8edd9e1740 4 client.15411 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:05:30.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.728+0000 7f8edd9e1740 1 -- 172.21.15.43:0/2372380807 --> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561b9cdff010 con 0x561b9cf9b110 2024-09-19T05:05:30.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.728+0000 7f8ed3400640 1 --2- 172.21.15.43:0/2372380807 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x561b9cf9b110 0x561b9cf9d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:30.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.729+0000 7f8ed3400640 1 --2- 172.21.15.43:0/2372380807 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x561b9cf9b110 0x561b9cf9d500 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b8316a7f07ca8fd6 server_cookie=2a1cd6d4ee45c711 in_seq=0 out_seq=0 2024-09-19T05:05:30.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.729+0000 7f8ebe000640 10 client.15411 ms_handle_connect on v2:172.21.15.96:6836/3978388286 2024-09-19T05:05:30.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.729+0000 7f8ebe000640 1 -- 172.21.15.43:0/2372380807 <== mds.3 v2:172.21.15.96:6836/3978388286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561b9cdff010 con 0x561b9cf9b110 2024-09-19T05:05:30.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.729+0000 7f8ebe000640 10 client.15411 handle_command_reply: tid=0 2024-09-19T05:05:30.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.729+0000 7f8edd9e1740 1 client.15411 shutdown 2024-09-19T05:05:30.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.729+0000 7f8edd9e1740 1 -- 172.21.15.43:0/2372380807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b9cf14e40 msgr2=0x561b9cf28850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:30.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.729+0000 7f8edd9e1740 1 --2- 172.21.15.43:0/2372380807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b9cf14e40 0x561b9cf28850 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4097ef0 tx=0x7f8ec409b3f0 comp rx=0 tx=0).stop 2024-09-19T05:05:30.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.729+0000 7f8edd9e1740 1 -- 172.21.15.43:0/2372380807 shutdown_connections 2024-09-19T05:05:30.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.729+0000 7f8edd9e1740 1 --2- 172.21.15.43:0/2372380807 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x561b9cf9b110 0x561b9cf9d500 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:30.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.729+0000 7f8edd9e1740 1 --2- 172.21.15.43:0/2372380807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b9cf20020 0x561b9cf24190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:30.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.729+0000 7f8edd9e1740 1 --2- 172.21.15.43:0/2372380807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b9cf15760 0x561b9cf23b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:30.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.729+0000 7f8edd9e1740 1 --2- 172.21.15.43:0/2372380807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561b9cf14e40 0x561b9cf28850 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:30.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.729+0000 7f8edd9e1740 1 -- 172.21.15.43:0/2372380807 >> 172.21.15.43:0/2372380807 conn(0x561b9cf113d0 msgr2=0x561b9cf17100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:30.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.729+0000 7f8ebe000640 0 client.15411 ms_handle_reset on v2:172.21.15.96:6836/3978388286 2024-09-19T05:05:30.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.730+0000 7f8edd9e1740 1 -- 172.21.15.43:0/2372380807 shutdown_connections 2024-09-19T05:05:30.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.730+0000 7f8edd9e1740 1 -- 172.21.15.43:0/2372380807 wait complete. 2024-09-19T05:05:30.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.731+0000 7f8edd9e1740 20 client.15411 trim_cache size 0 max 16384 2024-09-19T05:05:30.734 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:05:30.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.732+0000 7f8ed9800640 1 -- 172.21.15.43:0/2087746098 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f8eb80a35e0 msgr2=0x7f8eb80a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:30.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.732+0000 7f8ed9800640 1 --2- 172.21.15.43:0/2087746098 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f8eb80a35e0 0x7f8eb80a5aa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc005c40 tx=0x7f8ecc010040 comp rx=0 tx=0).stop 2024-09-19T05:05:30.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.732+0000 7f8ed9800640 1 -- 172.21.15.43:0/2087746098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ed4103e30 msgr2=0x7f8ed41109f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:30.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.732+0000 7f8ed9800640 1 --2- 172.21.15.43:0/2087746098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ed4103e30 0x7f8ed41109f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400d770 tx=0x7f8ec400dc30 comp rx=0 tx=0).stop 2024-09-19T05:05:30.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.732+0000 7f8ed9800640 1 -- 172.21.15.43:0/2087746098 shutdown_connections 2024-09-19T05:05:30.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.732+0000 7f8ed9800640 1 --2- 172.21.15.43:0/2087746098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ed4108f60 0x7f8ed410e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:30.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.732+0000 7f8ed9800640 1 --2- 172.21.15.43:0/2087746098 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ed41047e0 0x7f8ed410daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:30.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.732+0000 7f8ed9800640 1 --2- 172.21.15.43:0/2087746098 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f8eb80a35e0 0x7f8eb80a5aa0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:30.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.732+0000 7f8ed9800640 1 --2- 172.21.15.43:0/2087746098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ed4103e30 0x7f8ed41109f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:30.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.732+0000 7f8ed9800640 1 -- 172.21.15.43:0/2087746098 >> 172.21.15.43:0/2087746098 conn(0x7f8ed40ff910 msgr2=0x7f8ed4100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:30.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.733+0000 7f8ed9800640 1 -- 172.21.15.43:0/2087746098 shutdown_connections 2024-09-19T05:05:30.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:30.733+0000 7f8ed9800640 1 -- 172.21.15.43:0/2087746098 wait complete. 2024-09-19T05:05:31.725 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T05:05:32.178 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:05:32.385 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:32 smithi043 ceph-mon[30800]: pgmap v29: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-19T05:05:32.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:32 smithi151 ceph-mon[35425]: pgmap v29: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-19T05:05:32.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:32 smithi096 ceph-mon[35085]: pgmap v29: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-19T05:05:32.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.871+0000 7f3b63200640 1 -- 172.21.15.43:0/885075985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b64102360 msgr2=0x7f3b6410e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:32.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.871+0000 7f3b63200640 1 --2- 172.21.15.43:0/885075985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b64102360 0x7f3b6410e890 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c009ca0 tx=0x7f3b5c02a8c0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.871+0000 7f3b63200640 1 -- 172.21.15.43:0/885075985 shutdown_connections 2024-09-19T05:05:32.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.871+0000 7f3b63200640 1 --2- 172.21.15.43:0/885075985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b64102360 0x7f3b6410e890 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.871+0000 7f3b63200640 1 --2- 172.21.15.43:0/885075985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b641019c0 0x7f3b64101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.871+0000 7f3b63200640 1 --2- 172.21.15.43:0/885075985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b641079c0 0x7f3b64107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.871+0000 7f3b63200640 1 -- 172.21.15.43:0/885075985 >> 172.21.15.43:0/885075985 conn(0x7f3b640fd740 msgr2=0x7f3b640ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:32.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.872+0000 7f3b63200640 1 -- 172.21.15.43:0/885075985 shutdown_connections 2024-09-19T05:05:32.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.872+0000 7f3b63200640 1 -- 172.21.15.43:0/885075985 wait complete. 2024-09-19T05:05:32.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.874+0000 7f3b63200640 1 Processor -- start 2024-09-19T05:05:32.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.874+0000 7f3b63200640 1 -- start start 2024-09-19T05:05:32.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.874+0000 7f3b63200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b641019c0 0x7f3b6419a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:32.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.875+0000 7f3b63200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b64102360 0x7f3b6419a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:32.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.875+0000 7f3b63200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b641079c0 0x7f3b6419ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:32.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.875+0000 7f3b63200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3b64111e10 con 0x7f3b641079c0 2024-09-19T05:05:32.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.875+0000 7f3b63200640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3b64111c90 con 0x7f3b64102360 2024-09-19T05:05:32.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.875+0000 7f3b63200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3b64111f90 con 0x7f3b641019c0 2024-09-19T05:05:32.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.875+0000 7f3b61e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b641019c0 0x7f3b6419a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:32.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.875+0000 7f3b61e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b641019c0 0x7f3b6419a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:50006/0 (socket says 172.21.15.43:50006) 2024-09-19T05:05:32.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.875+0000 7f3b61e00640 1 -- 172.21.15.43:0/648112187 learned_addr learned my addr 172.21.15.43:0/648112187 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:05:32.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.875+0000 7f3b62800640 1 --2- 172.21.15.43:0/648112187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b641079c0 0x7f3b6419ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:32.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.875+0000 7f3b61400640 1 --2- 172.21.15.43:0/648112187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b64102360 0x7f3b6419a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:32.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.875+0000 7f3b61e00640 1 -- 172.21.15.43:0/648112187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b64102360 msgr2=0x7f3b6419a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:32.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.875+0000 7f3b61e00640 1 --2- 172.21.15.43:0/648112187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b64102360 0x7f3b6419a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.875+0000 7f3b61e00640 1 -- 172.21.15.43:0/648112187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b641079c0 msgr2=0x7f3b6419ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:32.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.875+0000 7f3b61e00640 1 --2- 172.21.15.43:0/648112187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b641079c0 0x7f3b6419ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.876+0000 7f3b61e00640 1 -- 172.21.15.43:0/648112187 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b6419f320 con 0x7f3b641019c0 2024-09-19T05:05:32.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.876+0000 7f3b61e00640 1 --2- 172.21.15.43:0/648112187 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b641019c0 0x7f3b6419a370 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3b54009f20 tx=0x7f3b5400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:32.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.876+0000 7f3b5aa00640 1 -- 172.21.15.43:0/648112187 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b54024070 con 0x7f3b641019c0 2024-09-19T05:05:32.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.876+0000 7f3b63200640 1 -- 172.21.15.43:0/648112187 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b6419f610 con 0x7f3b641019c0 2024-09-19T05:05:32.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.877+0000 7f3b63200640 1 -- 172.21.15.43:0/648112187 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b64108840 con 0x7f3b641019c0 2024-09-19T05:05:32.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.877+0000 7f3b5aa00640 1 -- 172.21.15.43:0/648112187 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b5401f440 con 0x7f3b641019c0 2024-09-19T05:05:32.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.877+0000 7f3b5aa00640 1 -- 172.21.15.43:0/648112187 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5401d5a0 con 0x7f3b641019c0 2024-09-19T05:05:32.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.881+0000 7f3b6b64a740 1 Processor -- start 2024-09-19T05:05:32.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.881+0000 7f3b6b64a740 1 -- start start 2024-09-19T05:05:32.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.882+0000 7f3b6b64a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561eb1f4be40 0x561eb1f4c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:32.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.882+0000 7f3b6b64a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561eb1f4c760 0x561eb1f56ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:32.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.882+0000 7f3b6b64a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb1f57020 0x561eb1f59410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:32.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.882+0000 7f3b6b64a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561eb1eb23e0 con 0x561eb1f4c760 2024-09-19T05:05:32.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.882+0000 7f3b6b64a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561eb1e59d80 con 0x561eb1f57020 2024-09-19T05:05:32.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.882+0000 7f3b6b64a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561eb1eae3a0 con 0x561eb1f4be40 2024-09-19T05:05:32.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.882+0000 7f3b5aa00640 1 -- 172.21.15.43:0/648112187 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f3b54027070 con 0x7f3b641019c0 2024-09-19T05:05:32.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.883+0000 7f3b61400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb1f57020 0x561eb1f59410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:32.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.883+0000 7f3b61400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb1f57020 0x561eb1f59410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:41184/0 (socket says 172.21.15.43:41184) 2024-09-19T05:05:32.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.883+0000 7f3b61400640 1 -- 172.21.15.43:0/1562161878 learned_addr learned my addr 172.21.15.43:0/1562161878 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:05:32.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.883+0000 7f3b61e00640 1 --2- 172.21.15.43:0/1562161878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561eb1f4c760 0x561eb1f56ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:32.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.883+0000 7f3b62800640 1 --2- 172.21.15.43:0/1562161878 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561eb1f4be40 0x561eb1f4c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:32.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.884+0000 7f3b62800640 1 -- 172.21.15.43:0/1562161878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb1f57020 msgr2=0x561eb1f59410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:32.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.884+0000 7f3b62800640 1 --2- 172.21.15.43:0/1562161878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb1f57020 0x561eb1f59410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.884+0000 7f3b62800640 1 -- 172.21.15.43:0/1562161878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561eb1f4c760 msgr2=0x561eb1f56ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:32.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.884+0000 7f3b62800640 1 --2- 172.21.15.43:0/1562161878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561eb1f4c760 0x561eb1f56ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.884+0000 7f3b62800640 1 -- 172.21.15.43:0/1562161878 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561eb1e78ae0 con 0x561eb1f4be40 2024-09-19T05:05:32.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.884+0000 7f3b61e00640 1 --2- 172.21.15.43:0/1562161878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561eb1f4c760 0x561eb1f56ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:05:32.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.884+0000 7f3b62800640 1 --2- 172.21.15.43:0/1562161878 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561eb1f4be40 0x561eb1f4c220 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c02ade0 tx=0x7f3b5c001140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c29423970d0be6ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:32.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.885+0000 7f3b5aa00640 1 --2- 172.21.15.43:0/648112187 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f3b480a35e0 0x7f3b480a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:32.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.885+0000 7f3b4f000640 1 -- 172.21.15.43:0/1562161878 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c031ba0 con 0x561eb1f4be40 2024-09-19T05:05:32.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.885+0000 7f3b5aa00640 1 -- 172.21.15.43:0/648112187 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f3b540d83c0 con 0x7f3b641019c0 2024-09-19T05:05:32.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.885+0000 7f3b4f000640 1 -- 172.21.15.43:0/1562161878 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b5c036040 con 0x561eb1f4be40 2024-09-19T05:05:32.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.885+0000 7f3b4f000640 1 -- 172.21.15.43:0/1562161878 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c004040 con 0x561eb1f4be40 2024-09-19T05:05:32.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.885+0000 7f3b61e00640 1 --2- 172.21.15.43:0/648112187 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f3b480a35e0 0x7f3b480a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:32.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.886+0000 7f3b6b64a740 1 -- 172.21.15.43:0/1562161878 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561eb1f4be40 msgr2=0x561eb1f4c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:32.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.886+0000 7f3b6b64a740 1 --2- 172.21.15.43:0/1562161878 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561eb1f4be40 0x561eb1f4c220 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c02ade0 tx=0x7f3b5c001140 comp rx=0 tx=0).stop 2024-09-19T05:05:32.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.886+0000 7f3b6b64a740 1 -- 172.21.15.43:0/1562161878 shutdown_connections 2024-09-19T05:05:32.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.886+0000 7f3b6b64a740 1 --2- 172.21.15.43:0/1562161878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb1f57020 0x561eb1f59410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.886+0000 7f3b6b64a740 1 --2- 172.21.15.43:0/1562161878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561eb1f4c760 0x561eb1f56ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.886+0000 7f3b6b64a740 1 --2- 172.21.15.43:0/1562161878 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561eb1f4be40 0x561eb1f4c220 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.886+0000 7f3b6b64a740 1 -- 172.21.15.43:0/1562161878 >> 172.21.15.43:0/1562161878 conn(0x561eb1f483d0 msgr2=0x561eb1f487b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:32.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.887+0000 7f3b61e00640 1 --2- 172.21.15.43:0/648112187 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f3b480a35e0 0x7f3b480a5aa0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3b540d5a50 tx=0x7f3b540dbdf0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:32.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.887+0000 7f3b6b64a740 1 -- 172.21.15.43:0/1562161878 shutdown_connections 2024-09-19T05:05:32.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.887+0000 7f3b6b64a740 1 -- 172.21.15.43:0/1562161878 wait complete. 2024-09-19T05:05:32.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.887+0000 7f3b6b64a740 1 -- start start 2024-09-19T05:05:32.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.888+0000 7f3b6b64a740 1 Processor -- start 2024-09-19T05:05:32.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.888+0000 7f3b6b64a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561eb1f4be40 0x561eb1f53f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:32.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.888+0000 7f3b6b64a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb1f4c760 0x561eb1f54440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:32.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.889+0000 7f3b6b64a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561eb1f57020 0x561eb1f54980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:32.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.889+0000 7f3b6b64a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561eb1f59bf0 con 0x561eb1f4be40 2024-09-19T05:05:32.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.889+0000 7f3b6b64a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561eb1e6d0f0 con 0x561eb1f4c760 2024-09-19T05:05:32.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.889+0000 7f3b6b64a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561eb1f59d70 con 0x561eb1f57020 2024-09-19T05:05:32.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.889+0000 7f3b6b64a740 10 client.0 _ll_register_callbacks cb 0x561eb1e54bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:05:32.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.889+0000 7f3b61400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb1f4c760 0x561eb1f54440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:32.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.889+0000 7f3b61400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb1f4c760 0x561eb1f54440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:41190/0 (socket says 172.21.15.43:41190) 2024-09-19T05:05:32.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.889+0000 7f3b61400640 1 -- 172.21.15.43:0/1317172538 learned_addr learned my addr 172.21.15.43:0/1317172538 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:05:32.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.889+0000 7f3b62800640 1 --2- 172.21.15.43:0/1317172538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561eb1f4be40 0x561eb1f53f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:32.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.890+0000 7f3b61400640 1 -- 172.21.15.43:0/1317172538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561eb1f57020 msgr2=0x561eb1f54980 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:05:32.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.890+0000 7f3b61400640 1 --2- 172.21.15.43:0/1317172538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561eb1f57020 0x561eb1f54980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.890+0000 7f3b61400640 1 -- 172.21.15.43:0/1317172538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561eb1f4be40 msgr2=0x561eb1f53f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:32.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.890+0000 7f3b61400640 1 --2- 172.21.15.43:0/1317172538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561eb1f4be40 0x561eb1f53f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.890+0000 7f3b61400640 1 -- 172.21.15.43:0/1317172538 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561eb1c233c0 con 0x561eb1f4c760 2024-09-19T05:05:32.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.890+0000 7f3b62800640 1 --2- 172.21.15.43:0/1317172538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561eb1f4be40 0x561eb1f53f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:05:32.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.890+0000 7f3b61400640 1 --2- 172.21.15.43:0/1317172538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb1f4c760 0x561eb1f54440 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3b500136e0 tx=0x7f3b50013ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=221f782ff799f42d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:32.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.890+0000 7f3b43400640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:05:32.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.890+0000 7f3b43400640 1 -- 172.21.15.43:0/1317172538 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b50011dd0 con 0x561eb1f4c760 2024-09-19T05:05:32.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.891+0000 7f3b6b64a740 1 -- 172.21.15.43:0/1317172538 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b5c036040 con 0x561eb1f4c760 2024-09-19T05:05:32.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.891+0000 7f3b43400640 1 -- 172.21.15.43:0/1317172538 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f3b50005020 con 0x561eb1f4c760 2024-09-19T05:05:32.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.891+0000 7f3b43400640 1 -- 172.21.15.43:0/1317172538 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3b50005220 con 0x561eb1f4c760 2024-09-19T05:05:32.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.891+0000 7f3b6b64a740 10 client.25406 fetch_fsmap learned FSMap version 29 2024-09-19T05:05:32.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.891+0000 7f3b6b64a740 1 -- 172.21.15.43:0/1317172538 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561eb1f65d70 con 0x561eb1f4c760 2024-09-19T05:05:32.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.891+0000 7f3b43400640 1 -- 172.21.15.43:0/1317172538 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b5c036040 con 0x561eb1f4c760 2024-09-19T05:05:32.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.892+0000 7f3b43400640 1 -- 172.21.15.43:0/1317172538 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3b50021020 con 0x561eb1f4c760 2024-09-19T05:05:32.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.893+0000 7f3b6b64a740 10 client.25406 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:05:32.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.893+0000 7f3b6b64a740 10 client.25406 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T05:05:32.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.893+0000 7f3b6b64a740 20 client.25406 populate_metadata read hostname 'smithi043' 2024-09-19T05:05:32.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.893+0000 7f3b6b64a740 1 --2- 172.21.15.43:0/1317172538 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x561eb1f67220 0x561eb1f69630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:32.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.893+0000 7f3b6b64a740 4 client.25406 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:05:32.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.893+0000 7f3b6b64a740 1 -- 172.21.15.43:0/1317172538 --> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561eb1ebf950 con 0x561eb1f67220 2024-09-19T05:05:32.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.893+0000 7f3b62800640 1 --2- 172.21.15.43:0/1317172538 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x561eb1f67220 0x561eb1f69630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:32.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.894+0000 7f3b62800640 1 --2- 172.21.15.43:0/1317172538 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x561eb1f67220 0x561eb1f69630 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bf0950787ce1bd7a server_cookie=ef74344bae5d8814 in_seq=0 out_seq=0 2024-09-19T05:05:32.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.894+0000 7f3b43400640 10 client.25406 ms_handle_connect on v2:172.21.15.151:6838/3794104313 2024-09-19T05:05:32.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.895+0000 7f3b43400640 1 -- 172.21.15.43:0/1317172538 <== mds.2 v2:172.21.15.151:6838/3794104313 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561eb1ebf950 con 0x561eb1f67220 2024-09-19T05:05:32.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.895+0000 7f3b43400640 10 client.25406 handle_command_reply: tid=0 2024-09-19T05:05:32.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.895+0000 7f3b6b64a740 1 client.25406 shutdown 2024-09-19T05:05:32.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.895+0000 7f3b6b64a740 1 -- 172.21.15.43:0/1317172538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb1f4c760 msgr2=0x561eb1f54440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:32.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.895+0000 7f3b6b64a740 1 --2- 172.21.15.43:0/1317172538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb1f4c760 0x561eb1f54440 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3b500136e0 tx=0x7f3b50013ba0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.895+0000 7f3b6b64a740 1 -- 172.21.15.43:0/1317172538 shutdown_connections 2024-09-19T05:05:32.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.895+0000 7f3b6b64a740 1 --2- 172.21.15.43:0/1317172538 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x561eb1f67220 0x561eb1f69630 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.895+0000 7f3b43400640 0 client.25406 ms_handle_reset on v2:172.21.15.151:6838/3794104313 2024-09-19T05:05:32.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.896+0000 7f3b6b64a740 1 --2- 172.21.15.43:0/1317172538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561eb1f57020 0x561eb1f54980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.896+0000 7f3b6b64a740 1 --2- 172.21.15.43:0/1317172538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb1f4c760 0x561eb1f54440 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.896+0000 7f3b6b64a740 1 --2- 172.21.15.43:0/1317172538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561eb1f4be40 0x561eb1f53f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.896+0000 7f3b6b64a740 1 -- 172.21.15.43:0/1317172538 >> 172.21.15.43:0/1317172538 conn(0x561eb1f483d0 msgr2=0x561eb1f4e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:32.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.896+0000 7f3b6b64a740 1 -- 172.21.15.43:0/1317172538 shutdown_connections 2024-09-19T05:05:32.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.896+0000 7f3b6b64a740 1 -- 172.21.15.43:0/1317172538 wait complete. 2024-09-19T05:05:32.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.896+0000 7f3b6b64a740 20 client.25406 trim_cache size 0 max 16384 2024-09-19T05:05:32.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.924+0000 7f3b6b64a740 1 Processor -- start 2024-09-19T05:05:32.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.924+0000 7f3b6b64a740 1 -- start start 2024-09-19T05:05:32.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.924+0000 7f3b6b64a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb1f4be40 0x561eb1f635a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:32.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.924+0000 7f3b6b64a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561eb1f4c760 0x561eb1f6ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:32.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.924+0000 7f3b6b64a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561eb1f57020 0x561eb1f6f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:32.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.924+0000 7f3b6b64a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4400e770 con 0x561eb1f4c760 2024-09-19T05:05:32.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.924+0000 7f3b6b64a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3b440102b0 con 0x561eb1f4be40 2024-09-19T05:05:32.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.924+0000 7f3b6b64a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4400ee40 con 0x561eb1f57020 2024-09-19T05:05:32.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.925+0000 7f3b62800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb1f4be40 0x561eb1f635a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:32.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.925+0000 7f3b62800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb1f4be40 0x561eb1f635a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:41206/0 (socket says 172.21.15.43:41206) 2024-09-19T05:05:32.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.925+0000 7f3b62800640 1 -- 172.21.15.43:0/3019118121 learned_addr learned my addr 172.21.15.43:0/3019118121 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:05:32.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.925+0000 7f3b62800640 1 --2- 172.21.15.43:0/3019118121 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561eb1f57020 0x561eb1f6f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:32.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.925+0000 7f3b61400640 1 --2- 172.21.15.43:0/3019118121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561eb1f4c760 0x561eb1f6ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:32.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.925+0000 7f3b62800640 1 -- 172.21.15.43:0/3019118121 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561eb1f57020 msgr2=0x561eb1f6f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:32.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.925+0000 7f3b62800640 1 --2- 172.21.15.43:0/3019118121 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561eb1f57020 0x561eb1f6f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.925+0000 7f3b62800640 1 -- 172.21.15.43:0/3019118121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561eb1f4c760 msgr2=0x561eb1f6ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:32.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.925+0000 7f3b62800640 1 --2- 172.21.15.43:0/3019118121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561eb1f4c760 0x561eb1f6ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.925+0000 7f3b62800640 1 -- 172.21.15.43:0/3019118121 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561eb1f62eb0 con 0x561eb1f4be40 2024-09-19T05:05:32.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.926+0000 7f3b61400640 1 --2- 172.21.15.43:0/3019118121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561eb1f4c760 0x561eb1f6ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:05:32.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.926+0000 7f3b62800640 1 --2- 172.21.15.43:0/3019118121 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb1f4be40 0x561eb1f635a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c009ca0 tx=0x7f3b5c031590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18902ad4df97bb6b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:32.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.927+0000 7f3b4e600640 1 -- 172.21.15.43:0/3019118121 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c031e60 con 0x561eb1f4be40 2024-09-19T05:05:32.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.927+0000 7f3b4e600640 1 -- 172.21.15.43:0/3019118121 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b5c036070 con 0x561eb1f4be40 2024-09-19T05:05:32.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.927+0000 7f3b4e600640 1 -- 172.21.15.43:0/3019118121 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c0042a0 con 0x561eb1f4be40 2024-09-19T05:05:32.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.928+0000 7f3b6b64a740 1 -- 172.21.15.43:0/3019118121 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb1f4be40 msgr2=0x561eb1f635a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:32.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.928+0000 7f3b6b64a740 1 --2- 172.21.15.43:0/3019118121 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb1f4be40 0x561eb1f635a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c009ca0 tx=0x7f3b5c031590 comp rx=0 tx=0).stop 2024-09-19T05:05:32.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.928+0000 7f3b6b64a740 1 -- 172.21.15.43:0/3019118121 shutdown_connections 2024-09-19T05:05:32.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.928+0000 7f3b6b64a740 1 --2- 172.21.15.43:0/3019118121 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561eb1f57020 0x561eb1f6f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.928+0000 7f3b6b64a740 1 --2- 172.21.15.43:0/3019118121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561eb1f4c760 0x561eb1f6ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.928+0000 7f3b6b64a740 1 --2- 172.21.15.43:0/3019118121 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb1f4be40 0x561eb1f635a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.928+0000 7f3b6b64a740 1 -- 172.21.15.43:0/3019118121 >> 172.21.15.43:0/3019118121 conn(0x561eb1f483d0 msgr2=0x561eb1f527d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:32.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.928+0000 7f3b6b64a740 1 -- 172.21.15.43:0/3019118121 shutdown_connections 2024-09-19T05:05:32.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.928+0000 7f3b6b64a740 1 -- 172.21.15.43:0/3019118121 wait complete. 2024-09-19T05:05:32.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.929+0000 7f3b6b64a740 1 -- start start 2024-09-19T05:05:32.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.929+0000 7f3b6b64a740 1 Processor -- start 2024-09-19T05:05:32.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.930+0000 7f3b6b64a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561eb1f4be40 0x561eb1f5f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:32.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.930+0000 7f3b6b64a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561eb1f4c760 0x561eb1f5a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:32.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.930+0000 7f3b6b64a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb1f57020 0x561eb1f5ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:32.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.930+0000 7f3b6b64a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4400fbe0 con 0x561eb1f4c760 2024-09-19T05:05:32.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.930+0000 7f3b6b64a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4400f510 con 0x561eb1f57020 2024-09-19T05:05:32.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.930+0000 7f3b6b64a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4400c750 con 0x561eb1f4be40 2024-09-19T05:05:32.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.930+0000 7f3b6b64a740 10 client.0 _ll_register_callbacks cb 0x7f3b50009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:05:32.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.930+0000 7f3b62800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb1f57020 0x561eb1f5ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:32.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.930+0000 7f3b62800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb1f57020 0x561eb1f5ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:41210/0 (socket says 172.21.15.43:41210) 2024-09-19T05:05:32.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.931+0000 7f3b62800640 1 -- 172.21.15.43:0/4261882742 learned_addr learned my addr 172.21.15.43:0/4261882742 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:05:32.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.931+0000 7f3b62800640 1 --2- 172.21.15.43:0/4261882742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561eb1f4be40 0x561eb1f5f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:32.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.931+0000 7f3b61400640 1 --2- 172.21.15.43:0/4261882742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561eb1f4c760 0x561eb1f5a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:32.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.931+0000 7f3b62800640 1 -- 172.21.15.43:0/4261882742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561eb1f4be40 msgr2=0x561eb1f5f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:32.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.931+0000 7f3b62800640 1 --2- 172.21.15.43:0/4261882742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561eb1f4be40 0x561eb1f5f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.931+0000 7f3b62800640 1 -- 172.21.15.43:0/4261882742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561eb1f4c760 msgr2=0x561eb1f5a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:32.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.931+0000 7f3b62800640 1 --2- 172.21.15.43:0/4261882742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561eb1f4c760 0x561eb1f5a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.931+0000 7f3b62800640 1 -- 172.21.15.43:0/4261882742 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561eb1f5b440 con 0x561eb1f57020 2024-09-19T05:05:32.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.931+0000 7f3b61400640 1 --2- 172.21.15.43:0/4261882742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561eb1f4c760 0x561eb1f5a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:05:32.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.932+0000 7f3b62800640 1 --2- 172.21.15.43:0/4261882742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb1f57020 0x561eb1f5ae00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c02c010 tx=0x7f3b5c0318b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c44cb93300f5fb96 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:32.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.932+0000 7f3b43400640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:05:32.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.933+0000 7f3b43400640 1 -- 172.21.15.43:0/4261882742 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c031600 con 0x561eb1f57020 2024-09-19T05:05:32.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.933+0000 7f3b43400640 1 -- 172.21.15.43:0/4261882742 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f3b5c009800 con 0x561eb1f57020 2024-09-19T05:05:32.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.933+0000 7f3b6b64a740 1 -- 172.21.15.43:0/4261882742 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b5c036070 con 0x561eb1f57020 2024-09-19T05:05:32.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.933+0000 7f3b43400640 1 -- 172.21.15.43:0/4261882742 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3b5c004b40 con 0x561eb1f57020 2024-09-19T05:05:32.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.933+0000 7f3b6b64a740 10 client.25412 fetch_fsmap learned FSMap version 29 2024-09-19T05:05:32.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.934+0000 7f3b6b64a740 1 -- 172.21.15.43:0/4261882742 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561eb1fd1d20 con 0x561eb1f57020 2024-09-19T05:05:32.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.934+0000 7f3b43400640 1 -- 172.21.15.43:0/4261882742 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b5c036070 con 0x561eb1f57020 2024-09-19T05:05:32.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.934+0000 7f3b43400640 1 -- 172.21.15.43:0/4261882742 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3b5c004320 con 0x561eb1f57020 2024-09-19T05:05:32.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.934+0000 7f3b6b64a740 10 client.25412 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:05:32.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.934+0000 7f3b6b64a740 10 client.25412 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T05:05:32.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.934+0000 7f3b6b64a740 20 client.25412 populate_metadata read hostname 'smithi043' 2024-09-19T05:05:32.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.935+0000 7f3b6b64a740 1 --2- 172.21.15.43:0/4261882742 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x561eb1f67220 0x561eb1fd3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:32.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.935+0000 7f3b6b64a740 4 client.25412 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:05:32.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.935+0000 7f3b6b64a740 1 -- 172.21.15.43:0/4261882742 --> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561eb1eb9e60 con 0x561eb1f67220 2024-09-19T05:05:32.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.935+0000 7f3b61400640 1 --2- 172.21.15.43:0/4261882742 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x561eb1f67220 0x561eb1fd3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:32.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.936+0000 7f3b61400640 1 --2- 172.21.15.43:0/4261882742 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x561eb1f67220 0x561eb1fd3f00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=763d5ebc95399984 server_cookie=c77c875d634756a4 in_seq=0 out_seq=0 2024-09-19T05:05:32.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.936+0000 7f3b43400640 10 client.25412 ms_handle_connect on v2:172.21.15.151:6838/3794104313 2024-09-19T05:05:32.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.936+0000 7f3b43400640 1 -- 172.21.15.43:0/4261882742 <== mds.2 v2:172.21.15.151:6838/3794104313 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561eb1eb9e60 con 0x561eb1f67220 2024-09-19T05:05:32.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.936+0000 7f3b43400640 10 client.25412 handle_command_reply: tid=0 2024-09-19T05:05:32.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.936+0000 7f3b6b64a740 1 client.25412 shutdown 2024-09-19T05:05:32.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.937+0000 7f3b6b64a740 1 -- 172.21.15.43:0/4261882742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb1f57020 msgr2=0x561eb1f5ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:32.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.937+0000 7f3b6b64a740 1 --2- 172.21.15.43:0/4261882742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb1f57020 0x561eb1f5ae00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c02c010 tx=0x7f3b5c0318b0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.937+0000 7f3b6b64a740 1 -- 172.21.15.43:0/4261882742 shutdown_connections 2024-09-19T05:05:32.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.937+0000 7f3b6b64a740 1 --2- 172.21.15.43:0/4261882742 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x561eb1f67220 0x561eb1fd3f00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.937+0000 7f3b6b64a740 1 --2- 172.21.15.43:0/4261882742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561eb1f57020 0x561eb1f5ae00 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.937+0000 7f3b43400640 0 client.25412 ms_handle_reset on v2:172.21.15.151:6838/3794104313 2024-09-19T05:05:32.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.937+0000 7f3b6b64a740 1 --2- 172.21.15.43:0/4261882742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561eb1f4c760 0x561eb1f5a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.937+0000 7f3b6b64a740 1 --2- 172.21.15.43:0/4261882742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561eb1f4be40 0x561eb1f5f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.937+0000 7f3b6b64a740 1 -- 172.21.15.43:0/4261882742 >> 172.21.15.43:0/4261882742 conn(0x561eb1f483d0 msgr2=0x561eb1f4d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:32.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.938+0000 7f3b6b64a740 1 -- 172.21.15.43:0/4261882742 shutdown_connections 2024-09-19T05:05:32.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.938+0000 7f3b6b64a740 1 -- 172.21.15.43:0/4261882742 wait complete. 2024-09-19T05:05:32.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.938+0000 7f3b6b64a740 20 client.25412 trim_cache size 0 max 16384 2024-09-19T05:05:32.941 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:05:32.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.939+0000 7f3b63200640 1 -- 172.21.15.43:0/648112187 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f3b480a35e0 msgr2=0x7f3b480a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:32.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.939+0000 7f3b63200640 1 --2- 172.21.15.43:0/648112187 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f3b480a35e0 0x7f3b480a5aa0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3b540d5a50 tx=0x7f3b540dbdf0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.939+0000 7f3b63200640 1 -- 172.21.15.43:0/648112187 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b641019c0 msgr2=0x7f3b6419a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:32.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.939+0000 7f3b63200640 1 --2- 172.21.15.43:0/648112187 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b641019c0 0x7f3b6419a370 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3b54009f20 tx=0x7f3b5400ee80 comp rx=0 tx=0).stop 2024-09-19T05:05:32.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.939+0000 7f3b63200640 1 -- 172.21.15.43:0/648112187 shutdown_connections 2024-09-19T05:05:32.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.939+0000 7f3b63200640 1 --2- 172.21.15.43:0/648112187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b641079c0 0x7f3b6419ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.939+0000 7f3b63200640 1 --2- 172.21.15.43:0/648112187 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f3b480a35e0 0x7f3b480a5aa0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.939+0000 7f3b63200640 1 --2- 172.21.15.43:0/648112187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b64102360 0x7f3b6419a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.939+0000 7f3b63200640 1 --2- 172.21.15.43:0/648112187 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b641019c0 0x7f3b6419a370 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:32.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.939+0000 7f3b63200640 1 -- 172.21.15.43:0/648112187 >> 172.21.15.43:0/648112187 conn(0x7f3b640fd740 msgr2=0x7f3b640fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:32.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.940+0000 7f3b63200640 1 -- 172.21.15.43:0/648112187 shutdown_connections 2024-09-19T05:05:32.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:32.940+0000 7f3b63200640 1 -- 172.21.15.43:0/648112187 wait complete. 2024-09-19T05:05:33.869 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T05:05:34.085 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:34 smithi043 ceph-mon[30800]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-19T05:05:34.334 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:05:34.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:34 smithi151 ceph-mon[35425]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-19T05:05:34.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:34 smithi096 ceph-mon[35085]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-19T05:05:34.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.963+0000 7f8c56200640 1 -- 172.21.15.43:0/3937159258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c50074620 msgr2=0x7f8c50074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:34.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.963+0000 7f8c56200640 1 --2- 172.21.15.43:0/3937159258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c50074620 0x7f8c50074a00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8c40009900 tx=0x7f8c4002a480 comp rx=0 tx=0).stop 2024-09-19T05:05:34.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.963+0000 7f8c56200640 1 -- 172.21.15.43:0/3937159258 shutdown_connections 2024-09-19T05:05:34.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.963+0000 7f8c56200640 1 --2- 172.21.15.43:0/3937159258 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c5010e3c0 0x7f8c50110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:34.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.963+0000 7f8c56200640 1 --2- 172.21.15.43:0/3937159258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c50074f40 0x7f8c5010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:34.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.963+0000 7f8c56200640 1 --2- 172.21.15.43:0/3937159258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c50074620 0x7f8c50074a00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:34.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.963+0000 7f8c56200640 1 -- 172.21.15.43:0/3937159258 >> 172.21.15.43:0/3937159258 conn(0x7f8c500fd8c0 msgr2=0x7f8c500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:34.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.963+0000 7f8c56200640 1 -- 172.21.15.43:0/3937159258 shutdown_connections 2024-09-19T05:05:34.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.963+0000 7f8c56200640 1 -- 172.21.15.43:0/3937159258 wait complete. 2024-09-19T05:05:34.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.965+0000 7f8c56200640 1 Processor -- start 2024-09-19T05:05:34.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.965+0000 7f8c56200640 1 -- start start 2024-09-19T05:05:34.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.965+0000 7f8c56200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c50074620 0x7f8c5019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:34.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.965+0000 7f8c56200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c50074f40 0x7f8c5019abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:34.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.965+0000 7f8c56200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c5010e3c0 0x7f8c5019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:34.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.965+0000 7f8c56200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8c50114210 con 0x7f8c50074620 2024-09-19T05:05:34.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.965+0000 7f8c56200640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8c50114090 con 0x7f8c5010e3c0 2024-09-19T05:05:34.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.965+0000 7f8c56200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8c50114390 con 0x7f8c50074f40 2024-09-19T05:05:34.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.966+0000 7f8c4fe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c5010e3c0 0x7f8c5019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:34.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.966+0000 7f8c4fe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c5010e3c0 0x7f8c5019ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:41232/0 (socket says 172.21.15.43:41232) 2024-09-19T05:05:34.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.966+0000 7f8c4fe00640 1 -- 172.21.15.43:0/1351637763 learned_addr learned my addr 172.21.15.43:0/1351637763 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:05:34.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.966+0000 7f8c4f400640 1 --2- 172.21.15.43:0/1351637763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c50074620 0x7f8c5019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:34.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.966+0000 7f8c4ea00640 1 --2- 172.21.15.43:0/1351637763 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c50074f40 0x7f8c5019abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:34.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.967+0000 7f8c4fe00640 1 -- 172.21.15.43:0/1351637763 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c50074f40 msgr2=0x7f8c5019abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:34.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.967+0000 7f8c4fe00640 1 --2- 172.21.15.43:0/1351637763 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c50074f40 0x7f8c5019abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:34.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.967+0000 7f8c4fe00640 1 -- 172.21.15.43:0/1351637763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c50074620 msgr2=0x7f8c5019a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:34.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.967+0000 7f8c4fe00640 1 --2- 172.21.15.43:0/1351637763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c50074620 0x7f8c5019a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:34.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.967+0000 7f8c4fe00640 1 -- 172.21.15.43:0/1351637763 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c5019f660 con 0x7f8c5010e3c0 2024-09-19T05:05:34.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.967+0000 7f8c4fe00640 1 --2- 172.21.15.43:0/1351637763 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c5010e3c0 0x7f8c5019ef80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8c4800b970 tx=0x7f8c4800be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:34.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.967+0000 7f8c47e00640 1 -- 172.21.15.43:0/1351637763 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c48019070 con 0x7f8c5010e3c0 2024-09-19T05:05:34.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.968+0000 7f8c47e00640 1 -- 172.21.15.43:0/1351637763 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c4800cb40 con 0x7f8c5010e3c0 2024-09-19T05:05:34.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.968+0000 7f8c56200640 1 -- 172.21.15.43:0/1351637763 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c5019f950 con 0x7f8c5010e3c0 2024-09-19T05:05:34.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.968+0000 7f8c47e00640 1 -- 172.21.15.43:0/1351637763 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c48007650 con 0x7f8c5010e3c0 2024-09-19T05:05:34.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.968+0000 7f8c56200640 1 -- 172.21.15.43:0/1351637763 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c501a7230 con 0x7f8c5010e3c0 2024-09-19T05:05:34.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.972+0000 7f8c58e1b740 1 Processor -- start 2024-09-19T05:05:34.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.973+0000 7f8c47e00640 1 -- 172.21.15.43:0/1351637763 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f8c4801c070 con 0x7f8c5010e3c0 2024-09-19T05:05:34.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.974+0000 7f8c58e1b740 1 -- start start 2024-09-19T05:05:34.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.974+0000 7f8c58e1b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590b16f7c70 0x5590b16f8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:34.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.974+0000 7f8c58e1b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b16f8590 0x5590b1702910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:34.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.974+0000 7f8c58e1b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590b1702e50 0x5590b1705240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:34.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.974+0000 7f8c58e1b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5590b165e3e0 con 0x5590b1702e50 2024-09-19T05:05:34.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.974+0000 7f8c58e1b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5590b1605d80 con 0x5590b16f7c70 2024-09-19T05:05:34.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.974+0000 7f8c58e1b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5590b165a3a0 con 0x5590b16f8590 2024-09-19T05:05:34.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.975+0000 7f8c4ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590b1702e50 0x5590b1705240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:34.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.975+0000 7f8c4f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b16f8590 0x5590b1702910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:34.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.975+0000 7f8c4ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590b1702e50 0x5590b1705240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36778/0 (socket says 172.21.15.43:36778) 2024-09-19T05:05:34.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.975+0000 7f8c4ea00640 1 -- 172.21.15.43:0/3652641160 learned_addr learned my addr 172.21.15.43:0/3652641160 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:05:34.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.975+0000 7f8c4ea00640 1 --2- 172.21.15.43:0/3652641160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590b16f7c70 0x5590b16f8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:34.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.976+0000 7f8c4ea00640 1 -- 172.21.15.43:0/3652641160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b16f8590 msgr2=0x5590b1702910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:34.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.976+0000 7f8c4ea00640 1 --2- 172.21.15.43:0/3652641160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b16f8590 0x5590b1702910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:34.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.976+0000 7f8c4ea00640 1 -- 172.21.15.43:0/3652641160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590b16f7c70 msgr2=0x5590b16f8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:34.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.976+0000 7f8c4ea00640 1 --2- 172.21.15.43:0/3652641160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590b16f7c70 0x5590b16f8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:34.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.976+0000 7f8c4ea00640 1 -- 172.21.15.43:0/3652641160 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590b1624ae0 con 0x5590b1702e50 2024-09-19T05:05:34.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.976+0000 7f8c47e00640 1 --2- 172.21.15.43:0/1351637763 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f8c340a35e0 0x7f8c340a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:34.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.976+0000 7f8c47e00640 1 -- 172.21.15.43:0/1351637763 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f8c480cf3b0 con 0x7f8c5010e3c0 2024-09-19T05:05:34.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.976+0000 7f8c4f400640 1 --2- 172.21.15.43:0/1351637763 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f8c340a35e0 0x7f8c340a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:34.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.977+0000 7f8c4ea00640 1 --2- 172.21.15.43:0/3652641160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590b1702e50 0x5590b1705240 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0097e0 tx=0x7f8c3c009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d400176b55bdfd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:34.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.977+0000 7f8c3be00640 1 -- 172.21.15.43:0/3652641160 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c013db0 con 0x5590b1702e50 2024-09-19T05:05:34.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.977+0000 7f8c3be00640 1 -- 172.21.15.43:0/3652641160 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c3c013120 con 0x5590b1702e50 2024-09-19T05:05:34.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.977+0000 7f8c3be00640 1 -- 172.21.15.43:0/3652641160 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c0035f0 con 0x5590b1702e50 2024-09-19T05:05:34.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.978+0000 7f8c58e1b740 1 -- 172.21.15.43:0/3652641160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590b1702e50 msgr2=0x5590b1705240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:34.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.978+0000 7f8c58e1b740 1 --2- 172.21.15.43:0/3652641160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590b1702e50 0x5590b1705240 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0097e0 tx=0x7f8c3c009770 comp rx=0 tx=0).stop 2024-09-19T05:05:34.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.978+0000 7f8c4f400640 1 --2- 172.21.15.43:0/1351637763 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f8c340a35e0 0x7f8c340a5aa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8c40000c00 tx=0x7f8c40009560 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:34.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.978+0000 7f8c58e1b740 1 -- 172.21.15.43:0/3652641160 shutdown_connections 2024-09-19T05:05:34.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.978+0000 7f8c58e1b740 1 --2- 172.21.15.43:0/3652641160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590b1702e50 0x5590b1705240 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:34.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.978+0000 7f8c58e1b740 1 --2- 172.21.15.43:0/3652641160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b16f8590 0x5590b1702910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:34.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.978+0000 7f8c58e1b740 1 --2- 172.21.15.43:0/3652641160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590b16f7c70 0x5590b16f8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:34.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.978+0000 7f8c58e1b740 1 -- 172.21.15.43:0/3652641160 >> 172.21.15.43:0/3652641160 conn(0x5590b16f4200 msgr2=0x5590b16f45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:34.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.979+0000 7f8c58e1b740 1 -- 172.21.15.43:0/3652641160 shutdown_connections 2024-09-19T05:05:34.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.979+0000 7f8c58e1b740 1 -- 172.21.15.43:0/3652641160 wait complete. 2024-09-19T05:05:34.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.980+0000 7f8c58e1b740 1 -- start start 2024-09-19T05:05:34.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.980+0000 7f8c58e1b740 1 Processor -- start 2024-09-19T05:05:34.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.981+0000 7f8c58e1b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590b16f7c70 0x5590b16ffdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:34.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.981+0000 7f8c58e1b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b16f8590 0x5590b1700300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:34.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.981+0000 7f8c58e1b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590b1702e50 0x5590b1700840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:34.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.981+0000 7f8c58e1b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5590b1705a20 con 0x5590b1702e50 2024-09-19T05:05:34.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.981+0000 7f8c58e1b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5590b16190f0 con 0x5590b16f7c70 2024-09-19T05:05:34.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.981+0000 7f8c58e1b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5590b1705ba0 con 0x5590b16f8590 2024-09-19T05:05:34.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.981+0000 7f8c58e1b740 10 client.0 _ll_register_callbacks cb 0x5590b1600bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:05:34.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.981+0000 7f8c4ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b16f8590 0x5590b1700300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:34.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.981+0000 7f8c4ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b16f8590 0x5590b1700300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:50056/0 (socket says 172.21.15.43:50056) 2024-09-19T05:05:34.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.981+0000 7f8c4ea00640 1 -- 172.21.15.43:0/3263739940 learned_addr learned my addr 172.21.15.43:0/3263739940 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:05:34.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.982+0000 7f8c4fe00640 1 --2- 172.21.15.43:0/3263739940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590b1702e50 0x5590b1700840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:34.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.982+0000 7f8c4f400640 1 --2- 172.21.15.43:0/3263739940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590b16f7c70 0x5590b16ffdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:34.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.982+0000 7f8c4ea00640 1 -- 172.21.15.43:0/3263739940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590b16f7c70 msgr2=0x5590b16ffdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:34.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.982+0000 7f8c4ea00640 1 --2- 172.21.15.43:0/3263739940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590b16f7c70 0x5590b16ffdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:34.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.982+0000 7f8c4ea00640 1 -- 172.21.15.43:0/3263739940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590b1702e50 msgr2=0x5590b1700840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:34.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.982+0000 7f8c4ea00640 1 --2- 172.21.15.43:0/3263739940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590b1702e50 0x5590b1700840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:34.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.982+0000 7f8c4ea00640 1 -- 172.21.15.43:0/3263739940 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590b13cf3c0 con 0x5590b16f8590 2024-09-19T05:05:34.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.982+0000 7f8c4f400640 1 --2- 172.21.15.43:0/3263739940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590b16f7c70 0x5590b16ffdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:05:34.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.982+0000 7f8c4fe00640 1 --2- 172.21.15.43:0/3263739940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590b1702e50 0x5590b1700840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:05:34.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.982+0000 7f8c4ea00640 1 --2- 172.21.15.43:0/3263739940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b16f8590 0x5590b1700300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c01d040 tx=0x7f8c3c005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=849e0c4db5fd8a7a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:34.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.982+0000 7f8c38c00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:05:34.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.983+0000 7f8c38c00640 1 -- 172.21.15.43:0/3263739940 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c004230 con 0x5590b16f8590 2024-09-19T05:05:34.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.983+0000 7f8c38c00640 1 -- 172.21.15.43:0/3263739940 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f8c3c0043d0 con 0x5590b16f8590 2024-09-19T05:05:34.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.983+0000 7f8c58e1b740 1 -- 172.21.15.43:0/3263739940 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c3c013120 con 0x5590b16f8590 2024-09-19T05:05:34.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.983+0000 7f8c38c00640 1 -- 172.21.15.43:0/3263739940 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8c3c0046c0 con 0x5590b16f8590 2024-09-19T05:05:34.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.983+0000 7f8c58e1b740 10 client.25384 fetch_fsmap learned FSMap version 29 2024-09-19T05:05:34.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.983+0000 7f8c58e1b740 1 -- 172.21.15.43:0/3263739940 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5590b1711ae0 con 0x5590b16f8590 2024-09-19T05:05:34.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.984+0000 7f8c38c00640 1 -- 172.21.15.43:0/3263739940 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c3c013120 con 0x5590b16f8590 2024-09-19T05:05:34.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.984+0000 7f8c38c00640 1 -- 172.21.15.43:0/3263739940 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8c3c003710 con 0x5590b16f8590 2024-09-19T05:05:34.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.984+0000 7f8c58e1b740 10 client.25384 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:05:34.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.984+0000 7f8c58e1b740 10 client.25384 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-19T05:05:34.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.984+0000 7f8c58e1b740 20 client.25384 populate_metadata read hostname 'smithi043' 2024-09-19T05:05:34.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.984+0000 7f8c58e1b740 1 --2- 172.21.15.43:0/3263739940 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5590b1712e30 0x5590b1715220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:34.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.984+0000 7f8c58e1b740 4 client.25384 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:05:34.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.984+0000 7f8c58e1b740 1 -- 172.21.15.43:0/3263739940 --> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590b166b950 con 0x5590b1712e30 2024-09-19T05:05:34.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.985+0000 7f8c4f400640 1 --2- 172.21.15.43:0/3263739940 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5590b1712e30 0x5590b1715220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:34.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.985+0000 7f8c4f400640 1 --2- 172.21.15.43:0/3263739940 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5590b1712e30 0x5590b1715220 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a328f4299eca850c server_cookie=83931ccad044448a in_seq=0 out_seq=0 2024-09-19T05:05:34.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.985+0000 7f8c38c00640 10 client.25384 ms_handle_connect on v2:172.21.15.43:6838/3404555030 2024-09-19T05:05:34.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.986+0000 7f8c38c00640 1 -- 172.21.15.43:0/3263739940 <== mds.1 v2:172.21.15.43:6838/3404555030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5590b166b950 con 0x5590b1712e30 2024-09-19T05:05:34.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.986+0000 7f8c38c00640 10 client.25384 handle_command_reply: tid=0 2024-09-19T05:05:34.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.986+0000 7f8c58e1b740 1 client.25384 shutdown 2024-09-19T05:05:34.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.987+0000 7f8c58e1b740 1 -- 172.21.15.43:0/3263739940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b16f8590 msgr2=0x5590b1700300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:34.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.987+0000 7f8c58e1b740 1 --2- 172.21.15.43:0/3263739940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b16f8590 0x5590b1700300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c01d040 tx=0x7f8c3c005270 comp rx=0 tx=0).stop 2024-09-19T05:05:34.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.987+0000 7f8c58e1b740 1 -- 172.21.15.43:0/3263739940 shutdown_connections 2024-09-19T05:05:34.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.987+0000 7f8c58e1b740 1 --2- 172.21.15.43:0/3263739940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590b1702e50 0x5590b1700840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:34.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.987+0000 7f8c58e1b740 1 --2- 172.21.15.43:0/3263739940 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5590b1712e30 0x5590b1715220 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:34.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.987+0000 7f8c58e1b740 1 --2- 172.21.15.43:0/3263739940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b16f8590 0x5590b1700300 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:34.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.987+0000 7f8c58e1b740 1 --2- 172.21.15.43:0/3263739940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590b16f7c70 0x5590b16ffdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:34.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.987+0000 7f8c58e1b740 1 -- 172.21.15.43:0/3263739940 >> 172.21.15.43:0/3263739940 conn(0x5590b16f4200 msgr2=0x5590b16f9ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:34.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.987+0000 7f8c38c00640 0 client.25384 ms_handle_reset on v2:172.21.15.43:6838/3404555030 2024-09-19T05:05:34.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.988+0000 7f8c58e1b740 1 -- 172.21.15.43:0/3263739940 shutdown_connections 2024-09-19T05:05:34.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.988+0000 7f8c58e1b740 1 -- 172.21.15.43:0/3263739940 wait complete. 2024-09-19T05:05:34.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:34.988+0000 7f8c58e1b740 20 client.25384 trim_cache size 0 max 16384 2024-09-19T05:05:35.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.002+0000 7f8c58e1b740 1 Processor -- start 2024-09-19T05:05:35.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.002+0000 7f8c58e1b740 1 -- start start 2024-09-19T05:05:35.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.003+0000 7f8c58e1b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590b16f7c70 0x5590b16f70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:35.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.003+0000 7f8c58e1b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590b16f8590 0x5590b1700690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:35.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.003+0000 7f8c58e1b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b1702e50 0x5590b171a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:35.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.003+0000 7f8c58e1b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8c501afb60 con 0x5590b16f7c70 2024-09-19T05:05:35.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.003+0000 7f8c58e1b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8c501b16a0 con 0x5590b16f8590 2024-09-19T05:05:35.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.003+0000 7f8c58e1b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8c501b0230 con 0x5590b1702e50 2024-09-19T05:05:35.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.003+0000 7f8c4fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b1702e50 0x5590b171a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:35.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.003+0000 7f8c4ea00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590b16f8590 0x5590b1700690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:35.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.003+0000 7f8c4fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b1702e50 0x5590b171a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:50058/0 (socket says 172.21.15.43:50058) 2024-09-19T05:05:35.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.003+0000 7f8c4fe00640 1 -- 172.21.15.43:0/1018015423 learned_addr learned my addr 172.21.15.43:0/1018015423 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:05:35.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.004+0000 7f8c4f400640 1 --2- 172.21.15.43:0/1018015423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590b16f7c70 0x5590b16f70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:35.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.004+0000 7f8c4ea00640 1 -- 172.21.15.43:0/1018015423 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b1702e50 msgr2=0x5590b171a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:35.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.004+0000 7f8c4ea00640 1 --2- 172.21.15.43:0/1018015423 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b1702e50 0x5590b171a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:35.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.004+0000 7f8c4ea00640 1 -- 172.21.15.43:0/1018015423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590b16f7c70 msgr2=0x5590b16f70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:35.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.004+0000 7f8c4ea00640 1 --2- 172.21.15.43:0/1018015423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590b16f7c70 0x5590b16f70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:35.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.004+0000 7f8c4ea00640 1 -- 172.21.15.43:0/1018015423 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590b16f83c0 con 0x5590b16f8590 2024-09-19T05:05:35.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.004+0000 7f8c4f400640 1 --2- 172.21.15.43:0/1018015423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590b16f7c70 0x5590b16f70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:05:35.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.005+0000 7f8c4ea00640 1 --2- 172.21.15.43:0/1018015423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590b16f8590 0x5590b1700690 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c00a010 tx=0x7f8c3c003ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=717b48d9e3ec59aa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:35.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.005+0000 7f8c3b400640 1 -- 172.21.15.43:0/1018015423 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c005670 con 0x5590b16f8590 2024-09-19T05:05:35.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.005+0000 7f8c3b400640 1 -- 172.21.15.43:0/1018015423 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c3c024030 con 0x5590b16f8590 2024-09-19T05:05:35.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.005+0000 7f8c3b400640 1 -- 172.21.15.43:0/1018015423 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c004860 con 0x5590b16f8590 2024-09-19T05:05:35.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.006+0000 7f8c58e1b740 1 -- 172.21.15.43:0/1018015423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590b16f8590 msgr2=0x5590b1700690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:35.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.006+0000 7f8c58e1b740 1 --2- 172.21.15.43:0/1018015423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590b16f8590 0x5590b1700690 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c00a010 tx=0x7f8c3c003ce0 comp rx=0 tx=0).stop 2024-09-19T05:05:35.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.006+0000 7f8c58e1b740 1 -- 172.21.15.43:0/1018015423 shutdown_connections 2024-09-19T05:05:35.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.006+0000 7f8c58e1b740 1 --2- 172.21.15.43:0/1018015423 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b1702e50 0x5590b171a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:35.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.006+0000 7f8c58e1b740 1 --2- 172.21.15.43:0/1018015423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590b16f8590 0x5590b1700690 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:35.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.006+0000 7f8c58e1b740 1 --2- 172.21.15.43:0/1018015423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590b16f7c70 0x5590b16f70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:35.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.006+0000 7f8c58e1b740 1 -- 172.21.15.43:0/1018015423 >> 172.21.15.43:0/1018015423 conn(0x5590b16f4200 msgr2=0x5590b16fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:35.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.006+0000 7f8c58e1b740 1 -- 172.21.15.43:0/1018015423 shutdown_connections 2024-09-19T05:05:35.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.006+0000 7f8c58e1b740 1 -- 172.21.15.43:0/1018015423 wait complete. 2024-09-19T05:05:35.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.007+0000 7f8c58e1b740 1 -- start start 2024-09-19T05:05:35.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.007+0000 7f8c58e1b740 1 Processor -- start 2024-09-19T05:05:35.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.008+0000 7f8c58e1b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b16f7c70 0x5590b1706550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:35.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.008+0000 7f8c58e1b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590b16f8590 0x5590b1706a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:35.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.008+0000 7f8c58e1b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590b1702e50 0x5590b177e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:35.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.008+0000 7f8c58e1b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8c501b0fd0 con 0x5590b16f8590 2024-09-19T05:05:35.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.008+0000 7f8c58e1b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8c501b0900 con 0x5590b1702e50 2024-09-19T05:05:35.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.008+0000 7f8c58e1b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8c501adb40 con 0x5590b16f7c70 2024-09-19T05:05:35.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.008+0000 7f8c58e1b740 10 client.0 _ll_register_callbacks cb 0x7f8c3c009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:05:35.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.008+0000 7f8c4f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b16f7c70 0x5590b1706550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:35.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.008+0000 7f8c4f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b16f7c70 0x5590b1706550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:50064/0 (socket says 172.21.15.43:50064) 2024-09-19T05:05:35.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.008+0000 7f8c4f400640 1 -- 172.21.15.43:0/2398639439 learned_addr learned my addr 172.21.15.43:0/2398639439 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:05:35.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.009+0000 7f8c4ea00640 1 --2- 172.21.15.43:0/2398639439 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590b16f8590 0x5590b1706a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:35.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.009+0000 7f8c4f400640 1 -- 172.21.15.43:0/2398639439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590b1702e50 msgr2=0x5590b177e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:35.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.009+0000 7f8c4f400640 1 --2- 172.21.15.43:0/2398639439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590b1702e50 0x5590b177e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:35.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.009+0000 7f8c4f400640 1 -- 172.21.15.43:0/2398639439 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590b16f8590 msgr2=0x5590b1706a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:35.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.009+0000 7f8c4f400640 1 --2- 172.21.15.43:0/2398639439 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590b16f8590 0x5590b1706a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:35.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.009+0000 7f8c4f400640 1 -- 172.21.15.43:0/2398639439 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590b177e7f0 con 0x5590b16f7c70 2024-09-19T05:05:35.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.009+0000 7f8c4ea00640 1 --2- 172.21.15.43:0/2398639439 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590b16f8590 0x5590b1706a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:05:35.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.009+0000 7f8c4f400640 1 --2- 172.21.15.43:0/2398639439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b16f7c70 0x5590b1706550 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8c400051b0 tx=0x7f8c40009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2eb834c1443eb602 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:35.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.009+0000 7f8c38c00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:05:35.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.010+0000 7f8c38c00640 1 -- 172.21.15.43:0/2398639439 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c40035040 con 0x5590b16f7c70 2024-09-19T05:05:35.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.010+0000 7f8c38c00640 1 -- 172.21.15.43:0/2398639439 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f8c40004d30 con 0x5590b16f7c70 2024-09-19T05:05:35.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.010+0000 7f8c58e1b740 1 -- 172.21.15.43:0/2398639439 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c3c024030 con 0x5590b16f7c70 2024-09-19T05:05:35.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.010+0000 7f8c38c00640 1 -- 172.21.15.43:0/2398639439 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8c400325c0 con 0x5590b16f7c70 2024-09-19T05:05:35.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.010+0000 7f8c58e1b740 10 client.25390 fetch_fsmap learned FSMap version 29 2024-09-19T05:05:35.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.010+0000 7f8c58e1b740 1 -- 172.21.15.43:0/2398639439 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5590b1785bc0 con 0x5590b16f7c70 2024-09-19T05:05:35.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.011+0000 7f8c38c00640 1 -- 172.21.15.43:0/2398639439 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c3c024030 con 0x5590b16f7c70 2024-09-19T05:05:35.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.011+0000 7f8c38c00640 1 -- 172.21.15.43:0/2398639439 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8c40003960 con 0x5590b16f7c70 2024-09-19T05:05:35.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.011+0000 7f8c58e1b740 10 client.25390 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:05:35.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.011+0000 7f8c58e1b740 10 client.25390 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-19T05:05:35.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.011+0000 7f8c58e1b740 20 client.25390 populate_metadata read hostname 'smithi043' 2024-09-19T05:05:35.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.011+0000 7f8c58e1b740 1 --2- 172.21.15.43:0/2398639439 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5590b17862f0 0x5590b1788750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:35.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.011+0000 7f8c58e1b740 4 client.25390 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:05:35.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.011+0000 7f8c58e1b740 1 -- 172.21.15.43:0/2398639439 --> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5590b1665e60 con 0x5590b17862f0 2024-09-19T05:05:35.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.012+0000 7f8c4ea00640 1 --2- 172.21.15.43:0/2398639439 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5590b17862f0 0x5590b1788750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:35.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.012+0000 7f8c4ea00640 1 --2- 172.21.15.43:0/2398639439 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5590b17862f0 0x5590b1788750 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=21d3b70d9ddf7866 server_cookie=3bd324db56e5ba9d in_seq=0 out_seq=0 2024-09-19T05:05:35.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.012+0000 7f8c38c00640 10 client.25390 ms_handle_connect on v2:172.21.15.43:6838/3404555030 2024-09-19T05:05:35.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.013+0000 7f8c38c00640 1 -- 172.21.15.43:0/2398639439 <== mds.1 v2:172.21.15.43:6838/3404555030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5590b1665e60 con 0x5590b17862f0 2024-09-19T05:05:35.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.013+0000 7f8c38c00640 10 client.25390 handle_command_reply: tid=0 2024-09-19T05:05:35.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.013+0000 7f8c58e1b740 1 client.25390 shutdown 2024-09-19T05:05:35.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.013+0000 7f8c58e1b740 1 -- 172.21.15.43:0/2398639439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b16f7c70 msgr2=0x5590b1706550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:35.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.013+0000 7f8c58e1b740 1 --2- 172.21.15.43:0/2398639439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b16f7c70 0x5590b1706550 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8c400051b0 tx=0x7f8c40009260 comp rx=0 tx=0).stop 2024-09-19T05:05:35.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.013+0000 7f8c58e1b740 1 -- 172.21.15.43:0/2398639439 shutdown_connections 2024-09-19T05:05:35.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.013+0000 7f8c58e1b740 1 --2- 172.21.15.43:0/2398639439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590b1702e50 0x5590b177e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:35.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.013+0000 7f8c58e1b740 1 --2- 172.21.15.43:0/2398639439 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590b16f8590 0x5590b1706a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:35.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.013+0000 7f8c58e1b740 1 --2- 172.21.15.43:0/2398639439 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5590b17862f0 0x5590b1788750 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:35.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.013+0000 7f8c58e1b740 1 --2- 172.21.15.43:0/2398639439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590b16f7c70 0x5590b1706550 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:35.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.013+0000 7f8c38c00640 0 client.25390 ms_handle_reset on v2:172.21.15.43:6838/3404555030 2024-09-19T05:05:35.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.013+0000 7f8c58e1b740 1 -- 172.21.15.43:0/2398639439 >> 172.21.15.43:0/2398639439 conn(0x5590b16f4200 msgr2=0x5590b16fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:35.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.013+0000 7f8c58e1b740 1 -- 172.21.15.43:0/2398639439 shutdown_connections 2024-09-19T05:05:35.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.014+0000 7f8c58e1b740 1 -- 172.21.15.43:0/2398639439 wait complete. 2024-09-19T05:05:35.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.014+0000 7f8c58e1b740 20 client.25390 trim_cache size 0 max 16384 2024-09-19T05:05:35.017 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:05:35.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.014+0000 7f8c56200640 1 -- 172.21.15.43:0/1351637763 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f8c340a35e0 msgr2=0x7f8c340a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:35.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.014+0000 7f8c56200640 1 --2- 172.21.15.43:0/1351637763 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f8c340a35e0 0x7f8c340a5aa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8c40000c00 tx=0x7f8c40009560 comp rx=0 tx=0).stop 2024-09-19T05:05:35.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.014+0000 7f8c56200640 1 -- 172.21.15.43:0/1351637763 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c5010e3c0 msgr2=0x7f8c5019ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:35.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.014+0000 7f8c56200640 1 --2- 172.21.15.43:0/1351637763 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c5010e3c0 0x7f8c5019ef80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8c4800b970 tx=0x7f8c4800be30 comp rx=0 tx=0).stop 2024-09-19T05:05:35.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.015+0000 7f8c56200640 1 -- 172.21.15.43:0/1351637763 shutdown_connections 2024-09-19T05:05:35.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.015+0000 7f8c56200640 1 --2- 172.21.15.43:0/1351637763 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f8c340a35e0 0x7f8c340a5aa0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:35.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.015+0000 7f8c56200640 1 --2- 172.21.15.43:0/1351637763 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c5010e3c0 0x7f8c5019ef80 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:35.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.015+0000 7f8c56200640 1 --2- 172.21.15.43:0/1351637763 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c50074f40 0x7f8c5019abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:35.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.015+0000 7f8c56200640 1 --2- 172.21.15.43:0/1351637763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c50074620 0x7f8c5019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:35.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.015+0000 7f8c56200640 1 -- 172.21.15.43:0/1351637763 >> 172.21.15.43:0/1351637763 conn(0x7f8c500fd8c0 msgr2=0x7f8c500fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:35.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.015+0000 7f8c56200640 1 -- 172.21.15.43:0/1351637763 shutdown_connections 2024-09-19T05:05:35.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:35.015+0000 7f8c56200640 1 -- 172.21.15.43:0/1351637763 wait complete. 2024-09-19T05:05:35.416 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:35 smithi151 ceph-mon[35425]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-19T05:05:35.455 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:35 smithi043 ceph-mon[30800]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-19T05:05:35.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:35 smithi096 ceph-mon[35085]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-19T05:05:36.012 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T05:05:36.487 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:05:37.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.148+0000 7f6c59800640 1 -- 172.21.15.43:0/2614687422 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c54102b90 msgr2=0x7f6c54102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:37.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.148+0000 7f6c59800640 1 --2- 172.21.15.43:0/2614687422 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c54102b90 0x7f6c54102f70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6c44009900 tx=0x7f6c4402a480 comp rx=0 tx=0).stop 2024-09-19T05:05:37.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.148+0000 7f6c59800640 1 -- 172.21.15.43:0/2614687422 shutdown_connections 2024-09-19T05:05:37.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.148+0000 7f6c59800640 1 --2- 172.21.15.43:0/2614687422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c5410bfc0 0x7f6c5410e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:37.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.148+0000 7f6c59800640 1 --2- 172.21.15.43:0/2614687422 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c541034b0 0x7f6c5410b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:37.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.148+0000 7f6c59800640 1 --2- 172.21.15.43:0/2614687422 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c54102b90 0x7f6c54102f70 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:37.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.148+0000 7f6c59800640 1 -- 172.21.15.43:0/2614687422 >> 172.21.15.43:0/2614687422 conn(0x7f6c540fc610 msgr2=0x7f6c540fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:37.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.148+0000 7f6c59800640 1 -- 172.21.15.43:0/2614687422 shutdown_connections 2024-09-19T05:05:37.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.148+0000 7f6c59800640 1 -- 172.21.15.43:0/2614687422 wait complete. 2024-09-19T05:05:37.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.149+0000 7f6c59800640 1 Processor -- start 2024-09-19T05:05:37.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.149+0000 7f6c59800640 1 -- start start 2024-09-19T05:05:37.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.150+0000 7f6c59800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c54102b90 0x7f6c5419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:37.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.150+0000 7f6c59800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c541034b0 0x7f6c5419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:37.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.150+0000 7f6c59800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c5410bfc0 0x7f6c5419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:37.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.150+0000 7f6c59800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6c54111dc0 con 0x7f6c541034b0 2024-09-19T05:05:37.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.150+0000 7f6c59800640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6c54111c40 con 0x7f6c5410bfc0 2024-09-19T05:05:37.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.150+0000 7f6c59800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6c54111f40 con 0x7f6c54102b90 2024-09-19T05:05:37.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.150+0000 7f6c53400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c541034b0 0x7f6c5419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:37.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.150+0000 7f6c53400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c541034b0 0x7f6c5419a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51720/0 (socket says 172.21.15.43:51720) 2024-09-19T05:05:37.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.150+0000 7f6c53400640 1 -- 172.21.15.43:0/3972685103 learned_addr learned my addr 172.21.15.43:0/3972685103 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:05:37.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.150+0000 7f6c58e00640 1 --2- 172.21.15.43:0/3972685103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c5410bfc0 0x7f6c5419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:37.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.151+0000 7f6c53400640 1 -- 172.21.15.43:0/3972685103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c54102b90 msgr2=0x7f6c5419a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:37.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.151+0000 7f6c53e00640 1 --2- 172.21.15.43:0/3972685103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c54102b90 0x7f6c5419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:37.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.151+0000 7f6c53400640 1 --2- 172.21.15.43:0/3972685103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c54102b90 0x7f6c5419a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:37.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.151+0000 7f6c53400640 1 -- 172.21.15.43:0/3972685103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c5410bfc0 msgr2=0x7f6c5419ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:37.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.151+0000 7f6c53400640 1 --2- 172.21.15.43:0/3972685103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c5410bfc0 0x7f6c5419ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:37.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.151+0000 7f6c53400640 1 -- 172.21.15.43:0/3972685103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c5419f410 con 0x7f6c541034b0 2024-09-19T05:05:37.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.151+0000 7f6c58e00640 1 --2- 172.21.15.43:0/3972685103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c5410bfc0 0x7f6c5419ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:05:37.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.151+0000 7f6c53400640 1 --2- 172.21.15.43:0/3972685103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c541034b0 0x7f6c5419a9a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6c4000e9f0 tx=0x7f6c4000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:37.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.152+0000 7f6c50c00640 1 -- 172.21.15.43:0/3972685103 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4000ccc0 con 0x7f6c541034b0 2024-09-19T05:05:37.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.152+0000 7f6c59800640 1 -- 172.21.15.43:0/3972685103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c5419f700 con 0x7f6c541034b0 2024-09-19T05:05:37.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.152+0000 7f6c50c00640 1 -- 172.21.15.43:0/3972685103 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c40010910 con 0x7f6c541034b0 2024-09-19T05:05:37.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.152+0000 7f6c50c00640 1 -- 172.21.15.43:0/3972685103 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c40018920 con 0x7f6c541034b0 2024-09-19T05:05:37.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.152+0000 7f6c59800640 1 -- 172.21.15.43:0/3972685103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c541a6fe0 con 0x7f6c541034b0 2024-09-19T05:05:37.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.157+0000 7f6c5d82e740 1 Processor -- start 2024-09-19T05:05:37.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.157+0000 7f6c5d82e740 1 -- start start 2024-09-19T05:05:37.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.158+0000 7f6c50c00640 1 -- 172.21.15.43:0/3972685103 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f6c40010430 con 0x7f6c541034b0 2024-09-19T05:05:37.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.158+0000 7f6c5d82e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa62afce40 0x55fa62afd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:37.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.158+0000 7f6c5d82e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa62afd760 0x55fa62b07ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:37.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.158+0000 7f6c5d82e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa62b08020 0x55fa62b0a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:37.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.158+0000 7f6c5d82e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fa62a633e0 con 0x55fa62afd760 2024-09-19T05:05:37.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.158+0000 7f6c5d82e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fa62a0ad80 con 0x55fa62b08020 2024-09-19T05:05:37.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.158+0000 7f6c5d82e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55fa62a5f3a0 con 0x55fa62afce40 2024-09-19T05:05:37.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.158+0000 7f6c53e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa62afd760 0x55fa62b07ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:37.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.158+0000 7f6c53e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa62afd760 0x55fa62b07ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51732/0 (socket says 172.21.15.43:51732) 2024-09-19T05:05:37.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.158+0000 7f6c53e00640 1 -- 172.21.15.43:0/3131977694 learned_addr learned my addr 172.21.15.43:0/3131977694 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:05:37.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.158+0000 7f6c53400640 1 --2- 172.21.15.43:0/3131977694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa62b08020 0x55fa62b0a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:37.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.159+0000 7f6c58e00640 1 --2- 172.21.15.43:0/3131977694 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa62afce40 0x55fa62afd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:37.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.159+0000 7f6c53400640 1 -- 172.21.15.43:0/3131977694 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa62afce40 msgr2=0x55fa62afd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:37.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.159+0000 7f6c53400640 1 --2- 172.21.15.43:0/3131977694 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa62afce40 0x55fa62afd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:37.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.159+0000 7f6c53400640 1 -- 172.21.15.43:0/3131977694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa62afd760 msgr2=0x55fa62b07ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:37.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.159+0000 7f6c53400640 1 --2- 172.21.15.43:0/3131977694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa62afd760 0x55fa62b07ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:37.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.159+0000 7f6c53400640 1 -- 172.21.15.43:0/3131977694 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa62a29ae0 con 0x55fa62b08020 2024-09-19T05:05:37.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.160+0000 7f6c53400640 1 --2- 172.21.15.43:0/3131977694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa62b08020 0x55fa62b0a410 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6c400cd9a0 tx=0x7f6c400d6a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c69ea3749782748 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:37.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.160+0000 7f6c50c00640 1 --2- 172.21.15.43:0/3972685103 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f6c380a35e0 0x7f6c380a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:37.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.161+0000 7f6c50c00640 1 -- 172.21.15.43:0/3972685103 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f6c400d0480 con 0x7f6c541034b0 2024-09-19T05:05:37.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.161+0000 7f6c49600640 1 -- 172.21.15.43:0/3131977694 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c400e3070 con 0x55fa62b08020 2024-09-19T05:05:37.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.161+0000 7f6c49600640 1 -- 172.21.15.43:0/3131977694 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c4009cc30 con 0x55fa62b08020 2024-09-19T05:05:37.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.161+0000 7f6c49600640 1 -- 172.21.15.43:0/3131977694 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4009c530 con 0x55fa62b08020 2024-09-19T05:05:37.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.161+0000 7f6c58e00640 1 --2- 172.21.15.43:0/3972685103 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f6c380a35e0 0x7f6c380a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:37.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.162+0000 7f6c5d82e740 1 -- 172.21.15.43:0/3131977694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa62b08020 msgr2=0x55fa62b0a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:37.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.162+0000 7f6c5d82e740 1 --2- 172.21.15.43:0/3131977694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa62b08020 0x55fa62b0a410 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6c400cd9a0 tx=0x7f6c400d6a50 comp rx=0 tx=0).stop 2024-09-19T05:05:37.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.162+0000 7f6c58e00640 1 --2- 172.21.15.43:0/3972685103 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f6c380a35e0 0x7f6c380a5aa0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c005fd0 tx=0x7f6c4c009500 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:37.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.162+0000 7f6c5d82e740 1 -- 172.21.15.43:0/3131977694 shutdown_connections 2024-09-19T05:05:37.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.162+0000 7f6c5d82e740 1 --2- 172.21.15.43:0/3131977694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa62b08020 0x55fa62b0a410 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:37.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.162+0000 7f6c5d82e740 1 --2- 172.21.15.43:0/3131977694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa62afd760 0x55fa62b07ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:37.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.162+0000 7f6c5d82e740 1 --2- 172.21.15.43:0/3131977694 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa62afce40 0x55fa62afd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:37.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.162+0000 7f6c5d82e740 1 -- 172.21.15.43:0/3131977694 >> 172.21.15.43:0/3131977694 conn(0x55fa62af93d0 msgr2=0x55fa62af97b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:37.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.162+0000 7f6c5d82e740 1 -- 172.21.15.43:0/3131977694 shutdown_connections 2024-09-19T05:05:37.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.162+0000 7f6c5d82e740 1 -- 172.21.15.43:0/3131977694 wait complete. 2024-09-19T05:05:37.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.163+0000 7f6c5d82e740 1 -- start start 2024-09-19T05:05:37.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.163+0000 7f6c5d82e740 1 Processor -- start 2024-09-19T05:05:37.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.164+0000 7f6c5d82e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa62afce40 0x55fa62b05000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:37.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.164+0000 7f6c5d82e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa62afd760 0x55fa62b05540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:37.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.164+0000 7f6c5d82e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa62b08020 0x55fa62b05a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:37.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.164+0000 7f6c5d82e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fa62b0abf0 con 0x55fa62b08020 2024-09-19T05:05:37.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.164+0000 7f6c5d82e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fa62a1e0f0 con 0x55fa62afce40 2024-09-19T05:05:37.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.164+0000 7f6c5d82e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55fa62b0ad70 con 0x55fa62afd760 2024-09-19T05:05:37.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.164+0000 7f6c5d82e740 10 client.0 _ll_register_callbacks cb 0x55fa62a05bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:05:37.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.164+0000 7f6c53e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa62afce40 0x55fa62b05000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:37.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.164+0000 7f6c58e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa62b08020 0x55fa62b05a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:37.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.164+0000 7f6c53e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa62afce40 0x55fa62b05000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:55616/0 (socket says 172.21.15.43:55616) 2024-09-19T05:05:37.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.164+0000 7f6c53e00640 1 -- 172.21.15.43:0/1154668916 learned_addr learned my addr 172.21.15.43:0/1154668916 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:05:37.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.165+0000 7f6c53400640 1 --2- 172.21.15.43:0/1154668916 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa62afd760 0x55fa62b05540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:37.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.165+0000 7f6c53e00640 1 -- 172.21.15.43:0/1154668916 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa62afd760 msgr2=0x55fa62b05540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:37.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.165+0000 7f6c53e00640 1 --2- 172.21.15.43:0/1154668916 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa62afd760 0x55fa62b05540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:37.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.165+0000 7f6c53e00640 1 -- 172.21.15.43:0/1154668916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa62b08020 msgr2=0x55fa62b05a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:37.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.165+0000 7f6c53e00640 1 --2- 172.21.15.43:0/1154668916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa62b08020 0x55fa62b05a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:37.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.165+0000 7f6c53e00640 1 -- 172.21.15.43:0/1154668916 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa627d43c0 con 0x55fa62afce40 2024-09-19T05:05:37.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.165+0000 7f6c58e00640 1 --2- 172.21.15.43:0/1154668916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa62b08020 0x55fa62b05a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:05:37.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.165+0000 7f6c53e00640 1 --2- 172.21.15.43:0/1154668916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa62afce40 0x55fa62b05000 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6c44000c00 tx=0x7f6c44032e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7db4b7a550388f87 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:37.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.165+0000 7f6c3e000640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:05:37.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.165+0000 7f6c3e000640 1 -- 172.21.15.43:0/1154668916 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c44040070 con 0x55fa62afce40 2024-09-19T05:05:37.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.166+0000 7f6c3e000640 1 -- 172.21.15.43:0/1154668916 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f6c44004040 con 0x55fa62afce40 2024-09-19T05:05:37.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.166+0000 7f6c5d82e740 1 -- 172.21.15.43:0/1154668916 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c4009cc30 con 0x55fa62afce40 2024-09-19T05:05:37.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.166+0000 7f6c3e000640 1 -- 172.21.15.43:0/1154668916 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6c44035040 con 0x55fa62afce40 2024-09-19T05:05:37.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.166+0000 7f6c5d82e740 10 client.25436 fetch_fsmap learned FSMap version 29 2024-09-19T05:05:37.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.166+0000 7f6c5d82e740 1 -- 172.21.15.43:0/1154668916 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fa62b16d10 con 0x55fa62afce40 2024-09-19T05:05:37.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.167+0000 7f6c3e000640 1 -- 172.21.15.43:0/1154668916 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c4009cc30 con 0x55fa62afce40 2024-09-19T05:05:37.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.167+0000 7f6c3e000640 1 -- 172.21.15.43:0/1154668916 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6c440046c0 con 0x55fa62afce40 2024-09-19T05:05:37.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.167+0000 7f6c5d82e740 10 client.25436 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:05:37.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.167+0000 7f6c5d82e740 10 client.25436 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:05:37.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.167+0000 7f6c5d82e740 20 client.25436 populate_metadata read hostname 'smithi043' 2024-09-19T05:05:37.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.167+0000 7f6c5d82e740 1 --2- 172.21.15.43:0/1154668916 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55fa62b181d0 0x55fa62b1a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:37.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.167+0000 7f6c5d82e740 4 client.25436 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:05:37.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.167+0000 7f6c5d82e740 1 -- 172.21.15.43:0/1154668916 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa62a70950 con 0x55fa62b181d0 2024-09-19T05:05:37.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.168+0000 7f6c53400640 1 --2- 172.21.15.43:0/1154668916 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55fa62b181d0 0x55fa62b1a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:37.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.168+0000 7f6c53400640 1 --2- 172.21.15.43:0/1154668916 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55fa62b181d0 0x55fa62b1a5c0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b63713c15b0a8288 server_cookie=ff17b5495ea2db8b in_seq=0 out_seq=0 2024-09-19T05:05:37.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.168+0000 7f6c3e000640 10 client.25436 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:05:37.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.169+0000 7f6c3e000640 1 -- 172.21.15.43:0/1154668916 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fa62a70950 con 0x55fa62b181d0 2024-09-19T05:05:37.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.169+0000 7f6c3e000640 10 client.25436 handle_command_reply: tid=0 2024-09-19T05:05:37.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.169+0000 7f6c5d82e740 1 client.25436 shutdown 2024-09-19T05:05:37.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.170+0000 7f6c5d82e740 1 -- 172.21.15.43:0/1154668916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa62afce40 msgr2=0x55fa62b05000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:37.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.170+0000 7f6c5d82e740 1 --2- 172.21.15.43:0/1154668916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa62afce40 0x55fa62b05000 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6c44000c00 tx=0x7f6c44032e60 comp rx=0 tx=0).stop 2024-09-19T05:05:37.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.170+0000 7f6c5d82e740 1 -- 172.21.15.43:0/1154668916 shutdown_connections 2024-09-19T05:05:37.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.170+0000 7f6c5d82e740 1 --2- 172.21.15.43:0/1154668916 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55fa62b181d0 0x55fa62b1a5c0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:37.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.170+0000 7f6c5d82e740 1 --2- 172.21.15.43:0/1154668916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa62b08020 0x55fa62b05a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:37.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.170+0000 7f6c5d82e740 1 --2- 172.21.15.43:0/1154668916 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa62afd760 0x55fa62b05540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:37.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.170+0000 7f6c5d82e740 1 --2- 172.21.15.43:0/1154668916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa62afce40 0x55fa62b05000 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:37.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.170+0000 7f6c3e000640 0 client.25436 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:05:37.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.170+0000 7f6c5d82e740 1 -- 172.21.15.43:0/1154668916 >> 172.21.15.43:0/1154668916 conn(0x55fa62af93d0 msgr2=0x55fa62aff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:37.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.170+0000 7f6c5d82e740 1 -- 172.21.15.43:0/1154668916 shutdown_connections 2024-09-19T05:05:37.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.171+0000 7f6c5d82e740 1 -- 172.21.15.43:0/1154668916 wait complete. 2024-09-19T05:05:37.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.171+0000 7f6c5d82e740 20 client.25436 trim_cache size 0 max 16384 2024-09-19T05:05:37.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.185+0000 7f6c5d82e740 1 Processor -- start 2024-09-19T05:05:37.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.185+0000 7f6c5d82e740 1 -- start start 2024-09-19T05:05:37.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.185+0000 7f6c5d82e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa62afce40 0x55fa62b1fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:37.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.185+0000 7f6c5d82e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa62afd760 0x55fa62b20380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:37.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.185+0000 7f6c5d82e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa62b08020 0x55fa62b022c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:37.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.185+0000 7f6c5d82e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6c541af8f0 con 0x55fa62b08020 2024-09-19T05:05:37.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.185+0000 7f6c5d82e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6c541b1430 con 0x55fa62afce40 2024-09-19T05:05:37.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.185+0000 7f6c5d82e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6c541affc0 con 0x55fa62afd760 2024-09-19T05:05:37.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.186+0000 7f6c53e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa62afce40 0x55fa62b1fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:37.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.186+0000 7f6c53e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa62afce40 0x55fa62b1fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:55626/0 (socket says 172.21.15.43:55626) 2024-09-19T05:05:37.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.186+0000 7f6c53e00640 1 -- 172.21.15.43:0/1555412204 learned_addr learned my addr 172.21.15.43:0/1555412204 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:05:37.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.186+0000 7f6c58e00640 1 --2- 172.21.15.43:0/1555412204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa62b08020 0x55fa62b022c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:37.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.186+0000 7f6c53e00640 1 -- 172.21.15.43:0/1555412204 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa62afd760 msgr2=0x55fa62b20380 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:05:37.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.187+0000 7f6c53e00640 1 --2- 172.21.15.43:0/1555412204 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa62afd760 0x55fa62b20380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:37.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.187+0000 7f6c53e00640 1 -- 172.21.15.43:0/1555412204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa62b08020 msgr2=0x55fa62b022c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:37.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.187+0000 7f6c53e00640 1 --2- 172.21.15.43:0/1555412204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa62b08020 0x55fa62b022c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:37.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.187+0000 7f6c53e00640 1 -- 172.21.15.43:0/1555412204 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa62b03590 con 0x55fa62afce40 2024-09-19T05:05:37.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.187+0000 7f6c53e00640 1 --2- 172.21.15.43:0/1555412204 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa62afce40 0x55fa62b1fe40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6c4402a4c0 tx=0x7f6c44009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=630acebea8ecec67 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:37.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.187+0000 7f6c48c00640 1 -- 172.21.15.43:0/1555412204 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c44051040 con 0x55fa62afce40 2024-09-19T05:05:37.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.187+0000 7f6c48c00640 1 -- 172.21.15.43:0/1555412204 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c44031d60 con 0x55fa62afce40 2024-09-19T05:05:37.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.187+0000 7f6c48c00640 1 -- 172.21.15.43:0/1555412204 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c44040040 con 0x55fa62afce40 2024-09-19T05:05:37.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.188+0000 7f6c5d82e740 1 -- 172.21.15.43:0/1555412204 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa62afce40 msgr2=0x55fa62b1fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:37.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.188+0000 7f6c5d82e740 1 --2- 172.21.15.43:0/1555412204 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa62afce40 0x55fa62b1fe40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6c4402a4c0 tx=0x7f6c44009650 comp rx=0 tx=0).stop 2024-09-19T05:05:37.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.188+0000 7f6c5d82e740 1 -- 172.21.15.43:0/1555412204 shutdown_connections 2024-09-19T05:05:37.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.188+0000 7f6c5d82e740 1 --2- 172.21.15.43:0/1555412204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa62b08020 0x55fa62b022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:37.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.188+0000 7f6c5d82e740 1 --2- 172.21.15.43:0/1555412204 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa62afd760 0x55fa62b20380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:37.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.188+0000 7f6c5d82e740 1 --2- 172.21.15.43:0/1555412204 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa62afce40 0x55fa62b1fe40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:37.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.188+0000 7f6c5d82e740 1 -- 172.21.15.43:0/1555412204 >> 172.21.15.43:0/1555412204 conn(0x55fa62af93d0 msgr2=0x55fa62b00300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:37.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.188+0000 7f6c5d82e740 1 -- 172.21.15.43:0/1555412204 shutdown_connections 2024-09-19T05:05:37.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.188+0000 7f6c5d82e740 1 -- 172.21.15.43:0/1555412204 wait complete. 2024-09-19T05:05:37.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.189+0000 7f6c5d82e740 1 -- start start 2024-09-19T05:05:37.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.189+0000 7f6c5d82e740 1 Processor -- start 2024-09-19T05:05:37.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.190+0000 7f6c5d82e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa62afce40 0x55fa62b01ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:37.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.190+0000 7f6c5d82e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa62afd760 0x55fa62b02530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:37.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.190+0000 7f6c5d82e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa62b08020 0x55fa62b02a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:37.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.190+0000 7f6c5d82e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6c541b0d60 con 0x55fa62b08020 2024-09-19T05:05:37.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.190+0000 7f6c5d82e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6c541b0690 con 0x55fa62afd760 2024-09-19T05:05:37.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.190+0000 7f6c5d82e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6c541ad8d0 con 0x55fa62afce40 2024-09-19T05:05:37.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.190+0000 7f6c5d82e740 10 client.0 _ll_register_callbacks cb 0x7f6c40005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:05:37.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.190+0000 7f6c53400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa62afd760 0x55fa62b02530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:37.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.190+0000 7f6c58e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa62b08020 0x55fa62b02a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:37.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.190+0000 7f6c53400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa62afd760 0x55fa62b02530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:55630/0 (socket says 172.21.15.43:55630) 2024-09-19T05:05:37.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.190+0000 7f6c53400640 1 -- 172.21.15.43:0/500111844 learned_addr learned my addr 172.21.15.43:0/500111844 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:05:37.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.191+0000 7f6c58e00640 1 -- 172.21.15.43:0/500111844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa62afce40 msgr2=0x55fa62b01ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:37.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.191+0000 7f6c58e00640 1 --2- 172.21.15.43:0/500111844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa62afce40 0x55fa62b01ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:37.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.191+0000 7f6c58e00640 1 -- 172.21.15.43:0/500111844 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa62afd760 msgr2=0x55fa62b02530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:37.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.191+0000 7f6c58e00640 1 --2- 172.21.15.43:0/500111844 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa62afd760 0x55fa62b02530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:37.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.191+0000 7f6c58e00640 1 -- 172.21.15.43:0/500111844 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa62b81e10 con 0x55fa62b08020 2024-09-19T05:05:37.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.191+0000 7f6c53400640 1 --2- 172.21.15.43:0/500111844 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa62afd760 0x55fa62b02530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:05:37.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.192+0000 7f6c58e00640 1 --2- 172.21.15.43:0/500111844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa62b08020 0x55fa62b02a70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x55fa62af49e0 tx=0x7f6c4c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe01978c3d068acc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:37.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.192+0000 7f6c3e000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:05:37.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.192+0000 7f6c3e000640 1 -- 172.21.15.43:0/500111844 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c02e060 con 0x55fa62b08020 2024-09-19T05:05:37.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.192+0000 7f6c3e000640 1 -- 172.21.15.43:0/500111844 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f6c4c01c400 con 0x55fa62b08020 2024-09-19T05:05:37.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.192+0000 7f6c5d82e740 1 -- 172.21.15.43:0/500111844 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c44031d60 con 0x55fa62b08020 2024-09-19T05:05:37.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.192+0000 7f6c3e000640 1 -- 172.21.15.43:0/500111844 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6c4c01c600 con 0x55fa62b08020 2024-09-19T05:05:37.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.192+0000 7f6c5d82e740 10 client.15459 fetch_fsmap learned FSMap version 29 2024-09-19T05:05:37.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.192+0000 7f6c5d82e740 1 -- 172.21.15.43:0/500111844 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fa62b830b0 con 0x55fa62b08020 2024-09-19T05:05:37.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.193+0000 7f6c3e000640 1 -- 172.21.15.43:0/500111844 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c44031d60 con 0x55fa62b08020 2024-09-19T05:05:37.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.193+0000 7f6c3e000640 1 -- 172.21.15.43:0/500111844 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6c4c028070 con 0x55fa62b08020 2024-09-19T05:05:37.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.193+0000 7f6c5d82e740 10 client.15459 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:05:37.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.193+0000 7f6c5d82e740 10 client.15459 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:05:37.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.193+0000 7f6c5d82e740 20 client.15459 populate_metadata read hostname 'smithi043' 2024-09-19T05:05:37.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.193+0000 7f6c5d82e740 1 --2- 172.21.15.43:0/500111844 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55fa62b83800 0x55fa62b85c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:37.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.193+0000 7f6c5d82e740 4 client.15459 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:05:37.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.193+0000 7f6c5d82e740 1 -- 172.21.15.43:0/500111844 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fa62a6ae60 con 0x55fa62b83800 2024-09-19T05:05:37.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.194+0000 7f6c53e00640 1 --2- 172.21.15.43:0/500111844 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55fa62b83800 0x55fa62b85c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:37.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.194+0000 7f6c53e00640 1 --2- 172.21.15.43:0/500111844 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55fa62b83800 0x55fa62b85c60 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ade027d6f73148b1 server_cookie=a522984beff1a08c in_seq=0 out_seq=0 2024-09-19T05:05:37.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.194+0000 7f6c3e000640 10 client.15459 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:05:37.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.195+0000 7f6c3e000640 1 -- 172.21.15.43:0/500111844 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fa62a6ae60 con 0x55fa62b83800 2024-09-19T05:05:37.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.195+0000 7f6c3e000640 10 client.15459 handle_command_reply: tid=0 2024-09-19T05:05:37.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.195+0000 7f6c5d82e740 1 client.15459 shutdown 2024-09-19T05:05:37.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.195+0000 7f6c5d82e740 1 -- 172.21.15.43:0/500111844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa62b08020 msgr2=0x55fa62b02a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:37.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.195+0000 7f6c5d82e740 1 --2- 172.21.15.43:0/500111844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa62b08020 0x55fa62b02a70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x55fa62af49e0 tx=0x7f6c4c004ab0 comp rx=0 tx=0).stop 2024-09-19T05:05:37.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.195+0000 7f6c5d82e740 1 -- 172.21.15.43:0/500111844 shutdown_connections 2024-09-19T05:05:37.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.195+0000 7f6c5d82e740 1 --2- 172.21.15.43:0/500111844 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55fa62b83800 0x55fa62b85c60 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:37.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.195+0000 7f6c5d82e740 1 --2- 172.21.15.43:0/500111844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa62b08020 0x55fa62b02a70 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:37.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.195+0000 7f6c5d82e740 1 --2- 172.21.15.43:0/500111844 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fa62afd760 0x55fa62b02530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:37.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.195+0000 7f6c3e000640 0 client.15459 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:05:37.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.195+0000 7f6c5d82e740 1 --2- 172.21.15.43:0/500111844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa62afce40 0x55fa62b01ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:37.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.195+0000 7f6c5d82e740 1 -- 172.21.15.43:0/500111844 >> 172.21.15.43:0/500111844 conn(0x55fa62af93d0 msgr2=0x55fa62af7c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:37.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.195+0000 7f6c5d82e740 1 -- 172.21.15.43:0/500111844 shutdown_connections 2024-09-19T05:05:37.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.195+0000 7f6c5d82e740 1 -- 172.21.15.43:0/500111844 wait complete. 2024-09-19T05:05:37.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.196+0000 7f6c5d82e740 20 client.15459 trim_cache size 0 max 16384 2024-09-19T05:05:37.200 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:05:37.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.196+0000 7f6c59800640 1 -- 172.21.15.43:0/3972685103 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f6c380a35e0 msgr2=0x7f6c380a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:37.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.196+0000 7f6c59800640 1 --2- 172.21.15.43:0/3972685103 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f6c380a35e0 0x7f6c380a5aa0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c005fd0 tx=0x7f6c4c009500 comp rx=0 tx=0).stop 2024-09-19T05:05:37.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.196+0000 7f6c59800640 1 -- 172.21.15.43:0/3972685103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c541034b0 msgr2=0x7f6c5419a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:37.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.196+0000 7f6c59800640 1 --2- 172.21.15.43:0/3972685103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c541034b0 0x7f6c5419a9a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6c4000e9f0 tx=0x7f6c4000eeb0 comp rx=0 tx=0).stop 2024-09-19T05:05:37.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.197+0000 7f6c59800640 1 -- 172.21.15.43:0/3972685103 shutdown_connections 2024-09-19T05:05:37.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.197+0000 7f6c59800640 1 --2- 172.21.15.43:0/3972685103 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f6c380a35e0 0x7f6c380a5aa0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:37.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.197+0000 7f6c59800640 1 --2- 172.21.15.43:0/3972685103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c5410bfc0 0x7f6c5419ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:37.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.197+0000 7f6c59800640 1 --2- 172.21.15.43:0/3972685103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c541034b0 0x7f6c5419a9a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:37.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.197+0000 7f6c59800640 1 --2- 172.21.15.43:0/3972685103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c54102b90 0x7f6c5419a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:37.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.197+0000 7f6c59800640 1 -- 172.21.15.43:0/3972685103 >> 172.21.15.43:0/3972685103 conn(0x7f6c540fc610 msgr2=0x7f6c5410d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:37.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.197+0000 7f6c59800640 1 -- 172.21.15.43:0/3972685103 shutdown_connections 2024-09-19T05:05:37.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:37.197+0000 7f6c59800640 1 -- 172.21.15.43:0/3972685103 wait complete. 2024-09-19T05:05:37.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:37 smithi151 ceph-mon[35425]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-19T05:05:37.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:37 smithi096 ceph-mon[35085]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-19T05:05:37.727 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:37 smithi043 ceph-mon[30800]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-19T05:05:39.153 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T05:05:39.153 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T05:05:39.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:39 smithi043 ceph-mon[30800]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-19T05:05:39.571 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:05:39.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:39 smithi151 ceph-mon[35425]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-19T05:05:39.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:39 smithi096 ceph-mon[35085]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-19T05:05:40.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.263+0000 7f644d000640 1 -- 172.21.15.43:0/1473136879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6448103e30 msgr2=0x7f6448104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:40.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.263+0000 7f644d000640 1 --2- 172.21.15.43:0/1473136879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6448103e30 0x7f6448104210 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6438009900 tx=0x7f643802a410 comp rx=0 tx=0).stop 2024-09-19T05:05:40.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.263+0000 7f644d000640 1 -- 172.21.15.43:0/1473136879 shutdown_connections 2024-09-19T05:05:40.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.263+0000 7f644d000640 1 --2- 172.21.15.43:0/1473136879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6448108f60 0x7f6448110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:40.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.263+0000 7f644d000640 1 --2- 172.21.15.43:0/1473136879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64481047e0 0x7f6448108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:40.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.263+0000 7f644d000640 1 --2- 172.21.15.43:0/1473136879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6448103e30 0x7f6448104210 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:40.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.263+0000 7f644d000640 1 -- 172.21.15.43:0/1473136879 >> 172.21.15.43:0/1473136879 conn(0x7f64480ff910 msgr2=0x7f6448101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:40.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.264+0000 7f644d000640 1 -- 172.21.15.43:0/1473136879 shutdown_connections 2024-09-19T05:05:40.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.264+0000 7f644d000640 1 -- 172.21.15.43:0/1473136879 wait complete. 2024-09-19T05:05:40.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.265+0000 7f644d000640 1 Processor -- start 2024-09-19T05:05:40.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.266+0000 7f644d000640 1 -- start start 2024-09-19T05:05:40.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.266+0000 7f644d000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6448103e30 0x7f644819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:40.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.266+0000 7f644d000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64481047e0 0x7f644819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:40.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.266+0000 7f644d000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6448108f60 0x7f644819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:40.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.266+0000 7f644d000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6448114010 con 0x7f6448108f60 2024-09-19T05:05:40.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.266+0000 7f644d000640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6448113e90 con 0x7f64481047e0 2024-09-19T05:05:40.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.266+0000 7f644d000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6448114190 con 0x7f6448103e30 2024-09-19T05:05:40.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.267+0000 7f6447400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6448103e30 0x7f644819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:40.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.267+0000 7f6447400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6448103e30 0x7f644819a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:58548/0 (socket says 172.21.15.43:58548) 2024-09-19T05:05:40.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.267+0000 7f6447400640 1 -- 172.21.15.43:0/1107470563 learned_addr learned my addr 172.21.15.43:0/1107470563 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:05:40.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.267+0000 7f6446a00640 1 --2- 172.21.15.43:0/1107470563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64481047e0 0x7f644819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:40.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.267+0000 7f6447400640 1 -- 172.21.15.43:0/1107470563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64481047e0 msgr2=0x7f644819a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:40.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.267+0000 7f6447400640 1 --2- 172.21.15.43:0/1107470563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64481047e0 0x7f644819a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:40.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.267+0000 7f6447400640 1 -- 172.21.15.43:0/1107470563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6448108f60 msgr2=0x7f644819ed40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T05:05:40.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.267+0000 7f6447e00640 1 --2- 172.21.15.43:0/1107470563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6448108f60 0x7f644819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:40.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.267+0000 7f6447400640 1 --2- 172.21.15.43:0/1107470563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6448108f60 0x7f644819ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:40.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.267+0000 7f6447400640 1 -- 172.21.15.43:0/1107470563 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644819f420 con 0x7f6448103e30 2024-09-19T05:05:40.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.268+0000 7f6447400640 1 --2- 172.21.15.43:0/1107470563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6448103e30 0x7f644819a470 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6438004770 tx=0x7f6438003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:40.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.268+0000 7f643fe00640 1 -- 172.21.15.43:0/1107470563 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643803f070 con 0x7f6448103e30 2024-09-19T05:05:40.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.268+0000 7f643fe00640 1 -- 172.21.15.43:0/1107470563 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6438031ca0 con 0x7f6448103e30 2024-09-19T05:05:40.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.268+0000 7f643fe00640 1 -- 172.21.15.43:0/1107470563 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6438035030 con 0x7f6448103e30 2024-09-19T05:05:40.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.268+0000 7f644d000640 1 -- 172.21.15.43:0/1107470563 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f644819f6b0 con 0x7f6448103e30 2024-09-19T05:05:40.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.268+0000 7f644d000640 1 -- 172.21.15.43:0/1107470563 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f644819fb40 con 0x7f6448103e30 2024-09-19T05:05:40.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.273+0000 7f6451032740 1 Processor -- start 2024-09-19T05:05:40.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.273+0000 7f6451032740 1 -- start start 2024-09-19T05:05:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.274+0000 7f643fe00640 1 -- 172.21.15.43:0/1107470563 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f6438042070 con 0x7f6448103e30 2024-09-19T05:05:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.274+0000 7f6451032740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bb02cfbd0 0x564bb02cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.274+0000 7f6451032740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bb02d04f0 0x564bb02e3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.274+0000 7f6451032740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bb02e36a0 0x564bb02e5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.274+0000 7f6451032740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564bb0236290 con 0x564bb02d04f0 2024-09-19T05:05:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.274+0000 7f6451032740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564bb01ddb40 con 0x564bb02e36a0 2024-09-19T05:05:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.274+0000 7f6451032740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564bb0232250 con 0x564bb02cfbd0 2024-09-19T05:05:40.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.274+0000 7f6447e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bb02cfbd0 0x564bb02cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:40.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.274+0000 7f6447e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bb02cfbd0 0x564bb02cffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:58556/0 (socket says 172.21.15.43:58556) 2024-09-19T05:05:40.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.274+0000 7f6447e00640 1 -- 172.21.15.43:0/3967487650 learned_addr learned my addr 172.21.15.43:0/3967487650 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:05:40.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.274+0000 7f6446a00640 1 --2- 172.21.15.43:0/3967487650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bb02e36a0 0x564bb02e5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:40.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.275+0000 7f6446a00640 1 -- 172.21.15.43:0/3967487650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bb02cfbd0 msgr2=0x564bb02cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:40.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.275+0000 7f6446a00640 1 --2- 172.21.15.43:0/3967487650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bb02cfbd0 0x564bb02cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:40.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.275+0000 7f6446a00640 1 -- 172.21.15.43:0/3967487650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bb02d04f0 msgr2=0x564bb02e3160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T05:05:40.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.275+0000 7f6446a00640 1 --2- 172.21.15.43:0/3967487650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bb02d04f0 0x564bb02e3160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:40.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.275+0000 7f6446a00640 1 -- 172.21.15.43:0/3967487650 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564baffa76a0 con 0x564bb02e36a0 2024-09-19T05:05:40.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.276+0000 7f6446a00640 1 --2- 172.21.15.43:0/3967487650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bb02e36a0 0x564bb02e5a90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6434009b60 tx=0x7f6434010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3f41a4cee08cda1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:40.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.276+0000 7f643fe00640 1 --2- 172.21.15.43:0/1107470563 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f642c0a3660 0x7f642c0a5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:40.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.276+0000 7f643fe00640 1 -- 172.21.15.43:0/1107470563 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f64380f02f0 con 0x7f6448103e30 2024-09-19T05:05:40.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.276+0000 7f643cc00640 1 -- 172.21.15.43:0/3967487650 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6434009e50 con 0x564bb02e36a0 2024-09-19T05:05:40.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.276+0000 7f6447e00640 1 --2- 172.21.15.43:0/1107470563 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f642c0a3660 0x7f642c0a5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:40.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.277+0000 7f643cc00640 1 -- 172.21.15.43:0/3967487650 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64340035f0 con 0x564bb02e36a0 2024-09-19T05:05:40.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.277+0000 7f643cc00640 1 -- 172.21.15.43:0/3967487650 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64340126b0 con 0x564bb02e36a0 2024-09-19T05:05:40.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.278+0000 7f6451032740 1 -- 172.21.15.43:0/3967487650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bb02e36a0 msgr2=0x564bb02e5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:40.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.278+0000 7f6451032740 1 --2- 172.21.15.43:0/3967487650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bb02e36a0 0x564bb02e5a90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6434009b60 tx=0x7f6434010740 comp rx=0 tx=0).stop 2024-09-19T05:05:40.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.278+0000 7f6447e00640 1 --2- 172.21.15.43:0/1107470563 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f642c0a3660 0x7f642c0a5b20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f64400094d0 tx=0x7f6440009500 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:40.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.278+0000 7f6451032740 1 -- 172.21.15.43:0/3967487650 shutdown_connections 2024-09-19T05:05:40.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.278+0000 7f6451032740 1 --2- 172.21.15.43:0/3967487650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bb02e36a0 0x564bb02e5a90 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:40.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.278+0000 7f6451032740 1 --2- 172.21.15.43:0/3967487650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bb02d04f0 0x564bb02e3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:40.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.278+0000 7f6451032740 1 --2- 172.21.15.43:0/3967487650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bb02cfbd0 0x564bb02cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:40.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.278+0000 7f6451032740 1 -- 172.21.15.43:0/3967487650 >> 172.21.15.43:0/3967487650 conn(0x564bb02cc240 msgr2=0x564bb02cc620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:40.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.279+0000 7f6451032740 1 -- 172.21.15.43:0/3967487650 shutdown_connections 2024-09-19T05:05:40.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.279+0000 7f6451032740 1 -- 172.21.15.43:0/3967487650 wait complete. 2024-09-19T05:05:40.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.279+0000 7f6451032740 1 -- start start 2024-09-19T05:05:40.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.280+0000 7f6451032740 1 Processor -- start 2024-09-19T05:05:40.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.280+0000 7f6451032740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bb02cfbd0 0x564bb02e30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:40.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.280+0000 7f6451032740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bb02d04f0 0x564bb02e03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:40.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.280+0000 7f6451032740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bb02e36a0 0x564bb02e0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:40.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.280+0000 7f6451032740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564bb02e62b0 con 0x564bb02d04f0 2024-09-19T05:05:40.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.280+0000 7f6451032740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564bb01f0fa0 con 0x564bb02cfbd0 2024-09-19T05:05:40.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.280+0000 7f6451032740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564bb02e6430 con 0x564bb02e36a0 2024-09-19T05:05:40.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.281+0000 7f6451032740 10 client.0 _ll_register_callbacks cb 0x564baff44ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:05:40.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.281+0000 7f6447400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bb02cfbd0 0x564bb02e30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:40.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.281+0000 7f6447400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bb02cfbd0 0x564bb02e30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:55672/0 (socket says 172.21.15.43:55672) 2024-09-19T05:05:40.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.281+0000 7f6447400640 1 -- 172.21.15.43:0/1656583232 learned_addr learned my addr 172.21.15.43:0/1656583232 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:05:40.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.281+0000 7f6447e00640 1 --2- 172.21.15.43:0/1656583232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bb02e36a0 0x564bb02e0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:40.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.282+0000 7f6446a00640 1 --2- 172.21.15.43:0/1656583232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bb02d04f0 0x564bb02e03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:40.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.282+0000 7f6447400640 1 -- 172.21.15.43:0/1656583232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bb02e36a0 msgr2=0x564bb02e0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:40.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.282+0000 7f6447400640 1 --2- 172.21.15.43:0/1656583232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bb02e36a0 0x564bb02e0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:40.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.282+0000 7f6447400640 1 -- 172.21.15.43:0/1656583232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bb02d04f0 msgr2=0x564bb02e03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:40.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.282+0000 7f6447400640 1 --2- 172.21.15.43:0/1656583232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bb02d04f0 0x564bb02e03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:40.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.282+0000 7f6447400640 1 -- 172.21.15.43:0/1656583232 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bb02ef790 con 0x564bb02cfbd0 2024-09-19T05:05:40.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.282+0000 7f6447400640 1 --2- 172.21.15.43:0/1656583232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bb02cfbd0 0x564bb02e30f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6438000c00 tx=0x7f64380bc2f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c0b205cccc856e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:40.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.282+0000 7f6431600640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:05:40.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.282+0000 7f6431600640 1 -- 172.21.15.43:0/1656583232 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64380da060 con 0x564bb02cfbd0 2024-09-19T05:05:40.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.282+0000 7f6431600640 1 -- 172.21.15.43:0/1656583232 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f643803b4f0 con 0x564bb02cfbd0 2024-09-19T05:05:40.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.283+0000 7f6451032740 1 -- 172.21.15.43:0/1656583232 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64340035f0 con 0x564bb02cfbd0 2024-09-19T05:05:40.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.283+0000 7f6431600640 1 -- 172.21.15.43:0/1656583232 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f64380ab050 con 0x564bb02cfbd0 2024-09-19T05:05:40.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.283+0000 7f6451032740 10 client.25451 fetch_fsmap learned FSMap version 29 2024-09-19T05:05:40.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.283+0000 7f6451032740 1 -- 172.21.15.43:0/1656583232 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564bb02f2350 con 0x564bb02cfbd0 2024-09-19T05:05:40.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.283+0000 7f6431600640 1 -- 172.21.15.43:0/1656583232 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64340035f0 con 0x564bb02cfbd0 2024-09-19T05:05:40.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.284+0000 7f6431600640 1 -- 172.21.15.43:0/1656583232 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f643804c060 con 0x564bb02cfbd0 2024-09-19T05:05:40.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.284+0000 7f6451032740 10 client.25451 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:05:40.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.284+0000 7f6451032740 10 client.25451 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:05:40.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.284+0000 7f6451032740 20 client.25451 populate_metadata read hostname 'smithi043' 2024-09-19T05:05:40.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.284+0000 7f6451032740 1 --2- 172.21.15.43:0/1656583232 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x564bb02f37b0 0x564bb02f5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:40.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.284+0000 7f6451032740 4 client.25451 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:05:40.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.284+0000 7f6451032740 1 -- 172.21.15.43:0/1656583232 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bb0243800 con 0x564bb02f37b0 2024-09-19T05:05:40.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.285+0000 7f6446a00640 1 --2- 172.21.15.43:0/1656583232 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x564bb02f37b0 0x564bb02f5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:40.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.286+0000 7f6446a00640 1 --2- 172.21.15.43:0/1656583232 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x564bb02f37b0 0x564bb02f5c00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7aa6819ba8761126 server_cookie=55bd0d860692d947 in_seq=0 out_seq=0 2024-09-19T05:05:40.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.286+0000 7f6431600640 10 client.25451 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:05:40.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.287+0000 7f6431600640 1 -- 172.21.15.43:0/1656583232 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564bb0243800 con 0x564bb02f37b0 2024-09-19T05:05:40.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.287+0000 7f6431600640 10 client.25451 handle_command_reply: tid=0 2024-09-19T05:05:40.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.287+0000 7f6451032740 1 client.25451 shutdown 2024-09-19T05:05:40.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.287+0000 7f6451032740 1 -- 172.21.15.43:0/1656583232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bb02cfbd0 msgr2=0x564bb02e30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:40.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.287+0000 7f6451032740 1 --2- 172.21.15.43:0/1656583232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bb02cfbd0 0x564bb02e30f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6438000c00 tx=0x7f64380bc2f0 comp rx=0 tx=0).stop 2024-09-19T05:05:40.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.287+0000 7f6451032740 1 -- 172.21.15.43:0/1656583232 shutdown_connections 2024-09-19T05:05:40.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.287+0000 7f6451032740 1 --2- 172.21.15.43:0/1656583232 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x564bb02f37b0 0x564bb02f5c00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:40.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.287+0000 7f6431600640 0 client.25451 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:05:40.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.287+0000 7f6451032740 1 --2- 172.21.15.43:0/1656583232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bb02e36a0 0x564bb02e0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:40.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.287+0000 7f6451032740 1 --2- 172.21.15.43:0/1656583232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bb02d04f0 0x564bb02e03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:40.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.287+0000 7f6451032740 1 --2- 172.21.15.43:0/1656583232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bb02cfbd0 0x564bb02e30f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:40.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.287+0000 7f6451032740 1 -- 172.21.15.43:0/1656583232 >> 172.21.15.43:0/1656583232 conn(0x564bb02cc240 msgr2=0x564bb02d1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:40.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.288+0000 7f6451032740 1 -- 172.21.15.43:0/1656583232 shutdown_connections 2024-09-19T05:05:40.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.288+0000 7f6451032740 1 -- 172.21.15.43:0/1656583232 wait complete. 2024-09-19T05:05:40.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.288+0000 7f6451032740 20 client.25451 trim_cache size 0 max 16384 2024-09-19T05:05:40.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.324+0000 7f6451032740 1 Processor -- start 2024-09-19T05:05:40.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.324+0000 7f6451032740 1 -- start start 2024-09-19T05:05:40.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.325+0000 7f6451032740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bb02cfbd0 0x564bb02e2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:40.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.325+0000 7f6451032740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bb02d04f0 0x564bb02fb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:40.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.325+0000 7f6451032740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bb02e36a0 0x564bb02fba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:40.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.325+0000 7f6451032740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f64481ab490 con 0x564bb02cfbd0 2024-09-19T05:05:40.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.325+0000 7f6451032740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f64481acfd0 con 0x564bb02d04f0 2024-09-19T05:05:40.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.325+0000 7f6451032740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f64481abb60 con 0x564bb02e36a0 2024-09-19T05:05:40.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.325+0000 7f6447400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bb02cfbd0 0x564bb02e2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:40.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.325+0000 7f6447400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bb02cfbd0 0x564bb02e2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51778/0 (socket says 172.21.15.43:51778) 2024-09-19T05:05:40.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.325+0000 7f6447400640 1 -- 172.21.15.43:0/1563991368 learned_addr learned my addr 172.21.15.43:0/1563991368 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:05:40.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.325+0000 7f6447e00640 1 --2- 172.21.15.43:0/1563991368 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bb02e36a0 0x564bb02fba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:40.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.326+0000 7f6446a00640 1 --2- 172.21.15.43:0/1563991368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bb02d04f0 0x564bb02fb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:40.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.326+0000 7f6447400640 1 -- 172.21.15.43:0/1563991368 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bb02e36a0 msgr2=0x564bb02fba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:40.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.326+0000 7f6447400640 1 --2- 172.21.15.43:0/1563991368 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bb02e36a0 0x564bb02fba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:40.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.326+0000 7f6447400640 1 -- 172.21.15.43:0/1563991368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bb02d04f0 msgr2=0x564bb02fb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:40.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.326+0000 7f6447400640 1 --2- 172.21.15.43:0/1563991368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bb02d04f0 0x564bb02fb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:40.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.326+0000 7f6447400640 1 -- 172.21.15.43:0/1563991368 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bb02e5dc0 con 0x564bb02cfbd0 2024-09-19T05:05:40.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.326+0000 7f6447400640 1 --2- 172.21.15.43:0/1563991368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bb02cfbd0 0x564bb02e2c10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f64380c5290 tx=0x7f6438005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17be8204599a3ccf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:40.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.327+0000 7f6433e00640 1 -- 172.21.15.43:0/1563991368 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64380b5070 con 0x564bb02cfbd0 2024-09-19T05:05:40.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.327+0000 7f6433e00640 1 -- 172.21.15.43:0/1563991368 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64380bcb60 con 0x564bb02cfbd0 2024-09-19T05:05:40.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.328+0000 7f6451032740 1 -- 172.21.15.43:0/1563991368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bb02cfbd0 msgr2=0x564bb02e2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:40.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.328+0000 7f6451032740 1 --2- 172.21.15.43:0/1563991368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bb02cfbd0 0x564bb02e2c10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f64380c5290 tx=0x7f6438005e70 comp rx=0 tx=0).stop 2024-09-19T05:05:40.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.328+0000 7f6451032740 1 -- 172.21.15.43:0/1563991368 shutdown_connections 2024-09-19T05:05:40.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.328+0000 7f6451032740 1 --2- 172.21.15.43:0/1563991368 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bb02e36a0 0x564bb02fba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:40.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.328+0000 7f6451032740 1 --2- 172.21.15.43:0/1563991368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bb02d04f0 0x564bb02fb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:40.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.328+0000 7f6451032740 1 --2- 172.21.15.43:0/1563991368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bb02cfbd0 0x564bb02e2c10 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:40.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.328+0000 7f6451032740 1 -- 172.21.15.43:0/1563991368 >> 172.21.15.43:0/1563991368 conn(0x564bb02cc240 msgr2=0x564bb02cb270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:40.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.329+0000 7f6451032740 1 -- 172.21.15.43:0/1563991368 shutdown_connections 2024-09-19T05:05:40.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.329+0000 7f6451032740 1 -- 172.21.15.43:0/1563991368 wait complete. 2024-09-19T05:05:40.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.329+0000 7f6451032740 1 -- start start 2024-09-19T05:05:40.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.330+0000 7f6451032740 1 Processor -- start 2024-09-19T05:05:40.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.330+0000 7f6451032740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bb02cfbd0 0x564bb02e8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:40.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.330+0000 7f6451032740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bb02d04f0 0x564bb02e8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:40.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.330+0000 7f6451032740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bb02e36a0 0x564bb02da8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:40.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.330+0000 7f6451032740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f64481ac900 con 0x564bb02d04f0 2024-09-19T05:05:40.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.330+0000 7f6451032740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f64481ac230 con 0x564bb02cfbd0 2024-09-19T05:05:40.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.330+0000 7f6451032740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f64481a9470 con 0x564bb02e36a0 2024-09-19T05:05:40.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.330+0000 7f6451032740 10 client.0 _ll_register_callbacks cb 0x7f6438005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:05:40.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.331+0000 7f6447e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bb02e36a0 0x564bb02da8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:40.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.331+0000 7f6447e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bb02e36a0 0x564bb02da8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:58584/0 (socket says 172.21.15.43:58584) 2024-09-19T05:05:40.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.331+0000 7f6447e00640 1 -- 172.21.15.43:0/2444194916 learned_addr learned my addr 172.21.15.43:0/2444194916 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:05:40.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.331+0000 7f6446a00640 1 --2- 172.21.15.43:0/2444194916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bb02d04f0 0x564bb02e8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:40.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.331+0000 7f6447400640 1 --2- 172.21.15.43:0/2444194916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bb02cfbd0 0x564bb02e8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:40.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.332+0000 7f6447e00640 1 -- 172.21.15.43:0/2444194916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bb02cfbd0 msgr2=0x564bb02e8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:40.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.332+0000 7f6447e00640 1 --2- 172.21.15.43:0/2444194916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bb02cfbd0 0x564bb02e8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:40.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.332+0000 7f6447e00640 1 -- 172.21.15.43:0/2444194916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bb02d04f0 msgr2=0x564bb02e8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:40.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.332+0000 7f6447e00640 1 --2- 172.21.15.43:0/2444194916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bb02d04f0 0x564bb02e8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:40.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.332+0000 7f6447e00640 1 -- 172.21.15.43:0/2444194916 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bb02dae10 con 0x564bb02e36a0 2024-09-19T05:05:40.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.332+0000 7f6447400640 1 --2- 172.21.15.43:0/2444194916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bb02cfbd0 0x564bb02e8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:05:40.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.333+0000 7f6446a00640 1 --2- 172.21.15.43:0/2444194916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bb02d04f0 0x564bb02e8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:05:40.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.333+0000 7f6447e00640 1 --2- 172.21.15.43:0/2444194916 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bb02e36a0 0x564bb02da8d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6440010a10 tx=0x7f6440004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74491e9f07cff67c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:40.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.333+0000 7f6431600640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:05:40.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.333+0000 7f6431600640 1 -- 172.21.15.43:0/2444194916 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644002e060 con 0x564bb02e36a0 2024-09-19T05:05:40.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.333+0000 7f6431600640 1 -- 172.21.15.43:0/2444194916 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f6440003d80 con 0x564bb02e36a0 2024-09-19T05:05:40.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.333+0000 7f6451032740 1 -- 172.21.15.43:0/2444194916 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64380bcb60 con 0x564bb02e36a0 2024-09-19T05:05:40.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.338+0000 7f6431600640 1 -- 172.21.15.43:0/2444194916 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6440004050 con 0x564bb02e36a0 2024-09-19T05:05:40.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.338+0000 7f6451032740 10 client.25417 fetch_fsmap learned FSMap version 29 2024-09-19T05:05:40.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.338+0000 7f6451032740 1 -- 172.21.15.43:0/2444194916 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564bb03564a0 con 0x564bb02e36a0 2024-09-19T05:05:40.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.339+0000 7f6431600640 1 -- 172.21.15.43:0/2444194916 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64380bcb60 con 0x564bb02e36a0 2024-09-19T05:05:40.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.339+0000 7f6431600640 1 -- 172.21.15.43:0/2444194916 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6440028030 con 0x564bb02e36a0 2024-09-19T05:05:40.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.340+0000 7f6451032740 10 client.25417 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:05:40.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.340+0000 7f6451032740 10 client.25417 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:05:40.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.340+0000 7f6451032740 20 client.25417 populate_metadata read hostname 'smithi043' 2024-09-19T05:05:40.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.340+0000 7f6451032740 1 --2- 172.21.15.43:0/2444194916 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x564bb0356b40 0x564bb0358f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:40.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.340+0000 7f6451032740 4 client.25417 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T05:05:40.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.340+0000 7f6451032740 1 -- 172.21.15.43:0/2444194916 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564bb023dd10 con 0x564bb0356b40 2024-09-19T05:05:40.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.340+0000 7f6447400640 1 --2- 172.21.15.43:0/2444194916 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x564bb0356b40 0x564bb0358f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:40.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.341+0000 7f6447400640 1 --2- 172.21.15.43:0/2444194916 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x564bb0356b40 0x564bb0358f60 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35b862ece1eda09 server_cookie=df58aa786ee065eb in_seq=0 out_seq=0 2024-09-19T05:05:40.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.341+0000 7f6431600640 10 client.25417 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:05:40.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.342+0000 7f6431600640 1 -- 172.21.15.43:0/2444194916 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564bb023dd10 con 0x564bb0356b40 2024-09-19T05:05:40.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.342+0000 7f6431600640 10 client.25417 handle_command_reply: tid=0 2024-09-19T05:05:40.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.342+0000 7f6451032740 1 client.25417 shutdown 2024-09-19T05:05:40.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.342+0000 7f6451032740 1 -- 172.21.15.43:0/2444194916 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bb02e36a0 msgr2=0x564bb02da8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:40.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.342+0000 7f6451032740 1 --2- 172.21.15.43:0/2444194916 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bb02e36a0 0x564bb02da8d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6440010a10 tx=0x7f6440004a40 comp rx=0 tx=0).stop 2024-09-19T05:05:40.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.342+0000 7f6451032740 1 -- 172.21.15.43:0/2444194916 shutdown_connections 2024-09-19T05:05:40.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.342+0000 7f6451032740 1 --2- 172.21.15.43:0/2444194916 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x564bb0356b40 0x564bb0358f60 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:40.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.342+0000 7f6451032740 1 --2- 172.21.15.43:0/2444194916 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bb02e36a0 0x564bb02da8d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:40.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.342+0000 7f6451032740 1 --2- 172.21.15.43:0/2444194916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bb02d04f0 0x564bb02e8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:40.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.342+0000 7f6451032740 1 --2- 172.21.15.43:0/2444194916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bb02cfbd0 0x564bb02e8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:40.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.342+0000 7f6451032740 1 -- 172.21.15.43:0/2444194916 >> 172.21.15.43:0/2444194916 conn(0x564bb02cc240 msgr2=0x564bb02ee570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:40.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.342+0000 7f6431600640 0 client.25417 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:05:40.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.343+0000 7f6451032740 1 -- 172.21.15.43:0/2444194916 shutdown_connections 2024-09-19T05:05:40.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.343+0000 7f6451032740 1 -- 172.21.15.43:0/2444194916 wait complete. 2024-09-19T05:05:40.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.343+0000 7f6451032740 20 client.25417 trim_cache size 0 max 16384 2024-09-19T05:05:40.349 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:05:40.349 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-09-19T05:05:40.349 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "f237750e-13ff-4229-94a4-59db242c6c08", 2024-09-19T05:05:40.349 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-09-19T05:05:40.349 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:05:40.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.345+0000 7f644d000640 1 -- 172.21.15.43:0/1107470563 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f642c0a3660 msgr2=0x7f642c0a5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:40.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.345+0000 7f644d000640 1 --2- 172.21.15.43:0/1107470563 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f642c0a3660 0x7f642c0a5b20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f64400094d0 tx=0x7f6440009500 comp rx=0 tx=0).stop 2024-09-19T05:05:40.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.345+0000 7f644d000640 1 -- 172.21.15.43:0/1107470563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6448103e30 msgr2=0x7f644819a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:40.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.345+0000 7f644d000640 1 --2- 172.21.15.43:0/1107470563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6448103e30 0x7f644819a470 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6438004770 tx=0x7f6438003c10 comp rx=0 tx=0).stop 2024-09-19T05:05:40.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.345+0000 7f644d000640 1 -- 172.21.15.43:0/1107470563 shutdown_connections 2024-09-19T05:05:40.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.345+0000 7f644d000640 1 --2- 172.21.15.43:0/1107470563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6448108f60 0x7f644819ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:40.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.345+0000 7f644d000640 1 --2- 172.21.15.43:0/1107470563 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f642c0a3660 0x7f642c0a5b20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:40.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.345+0000 7f644d000640 1 --2- 172.21.15.43:0/1107470563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64481047e0 0x7f644819a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:40.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.345+0000 7f644d000640 1 --2- 172.21.15.43:0/1107470563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6448103e30 0x7f644819a470 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:40.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.345+0000 7f644d000640 1 -- 172.21.15.43:0/1107470563 >> 172.21.15.43:0/1107470563 conn(0x7f64480ff910 msgr2=0x7f6448100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:40.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.347+0000 7f644d000640 1 -- 172.21.15.43:0/1107470563 shutdown_connections 2024-09-19T05:05:40.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:40.347+0000 7f644d000640 1 -- 172.21.15.43:0/1107470563 wait complete. 2024-09-19T05:05:41.163 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:05:41.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:41 smithi043 ceph-mon[30800]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-19T05:05:41.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:41 smithi043 ceph-mon[30800]: scrub queued for path: / 2024-09-19T05:05:41.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:41 smithi043 ceph-mon[30800]: scrub summary: idle+waiting paths [/] 2024-09-19T05:05:41.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:41 smithi043 ceph-mon[30800]: scrub summary: active paths [/] 2024-09-19T05:05:41.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:41 smithi043 ceph-mon[30800]: scrub summary: idle+waiting paths [/] 2024-09-19T05:05:41.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:41 smithi151 ceph-mon[35425]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-19T05:05:41.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:41 smithi151 ceph-mon[35425]: scrub queued for path: / 2024-09-19T05:05:41.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:41 smithi151 ceph-mon[35425]: scrub summary: idle+waiting paths [/] 2024-09-19T05:05:41.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:41 smithi151 ceph-mon[35425]: scrub summary: active paths [/] 2024-09-19T05:05:41.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:41 smithi151 ceph-mon[35425]: scrub summary: idle+waiting paths [/] 2024-09-19T05:05:41.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:41 smithi096 ceph-mon[35085]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-19T05:05:41.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:41 smithi096 ceph-mon[35085]: scrub queued for path: / 2024-09-19T05:05:41.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:41 smithi096 ceph-mon[35085]: scrub summary: idle+waiting paths [/] 2024-09-19T05:05:41.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:41 smithi096 ceph-mon[35085]: scrub summary: active paths [/] 2024-09-19T05:05:41.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:41 smithi096 ceph-mon[35085]: scrub summary: idle+waiting paths [/] 2024-09-19T05:05:41.728 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:05:42.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.420+0000 7ff0cb200640 1 -- 172.21.15.43:0/814184096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0cc10e280 msgr2=0x7ff0cc110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:42.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.420+0000 7ff0cb200640 1 --2- 172.21.15.43:0/814184096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0cc10e280 0x7ff0cc110670 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4009ca0 tx=0x7ff0c402a950 comp rx=0 tx=0).stop 2024-09-19T05:05:42.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.420+0000 7ff0cb200640 1 -- 172.21.15.43:0/814184096 shutdown_connections 2024-09-19T05:05:42.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.420+0000 7ff0cb200640 1 --2- 172.21.15.43:0/814184096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0cc10e280 0x7ff0cc110670 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:42.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.420+0000 7ff0cb200640 1 --2- 172.21.15.43:0/814184096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0cc074e30 0x7ff0cc10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:42.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.420+0000 7ff0cb200640 1 --2- 172.21.15.43:0/814184096 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0cc074510 0x7ff0cc0748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:42.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.420+0000 7ff0cb200640 1 -- 172.21.15.43:0/814184096 >> 172.21.15.43:0/814184096 conn(0x7ff0cc0fd740 msgr2=0x7ff0cc0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:42.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.420+0000 7ff0cb200640 1 -- 172.21.15.43:0/814184096 shutdown_connections 2024-09-19T05:05:42.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.421+0000 7ff0cb200640 1 -- 172.21.15.43:0/814184096 wait complete. 2024-09-19T05:05:42.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.422+0000 7ff0cb200640 1 Processor -- start 2024-09-19T05:05:42.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.422+0000 7ff0cb200640 1 -- start start 2024-09-19T05:05:42.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.423+0000 7ff0cb200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0cc074510 0x7ff0cc19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:42.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.423+0000 7ff0cb200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0cc074e30 0x7ff0cc19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:42.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.423+0000 7ff0cb200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0cc10e280 0x7ff0cc19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:42.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.423+0000 7ff0cb200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc114000 con 0x7ff0cc10e280 2024-09-19T05:05:42.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.423+0000 7ff0cb200640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc113e80 con 0x7ff0cc074510 2024-09-19T05:05:42.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.423+0000 7ff0cb200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc114180 con 0x7ff0cc074e30 2024-09-19T05:05:42.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.423+0000 7ff0c9e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0cc074510 0x7ff0cc19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:42.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.423+0000 7ff0c9e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0cc074510 0x7ff0cc19a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:55706/0 (socket says 172.21.15.43:55706) 2024-09-19T05:05:42.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.423+0000 7ff0c9e00640 1 -- 172.21.15.43:0/3496315221 learned_addr learned my addr 172.21.15.43:0/3496315221 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:05:42.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.424+0000 7ff0c9400640 1 --2- 172.21.15.43:0/3496315221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0cc074e30 0x7ff0cc19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:42.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.424+0000 7ff0c9e00640 1 -- 172.21.15.43:0/3496315221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0cc074e30 msgr2=0x7ff0cc19a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:42.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.424+0000 7ff0c9e00640 1 --2- 172.21.15.43:0/3496315221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0cc074e30 0x7ff0cc19a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:42.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.424+0000 7ff0c9e00640 1 -- 172.21.15.43:0/3496315221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0cc10e280 msgr2=0x7ff0cc19ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:42.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.424+0000 7ff0c9e00640 1 --2- 172.21.15.43:0/3496315221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0cc10e280 0x7ff0cc19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:42.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.424+0000 7ff0c9e00640 1 -- 172.21.15.43:0/3496315221 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0cc19f430 con 0x7ff0cc074510 2024-09-19T05:05:42.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.424+0000 7ff0c9e00640 1 --2- 172.21.15.43:0/3496315221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0cc074510 0x7ff0cc19a480 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc00e9f0 tx=0x7ff0bc00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:42.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.425+0000 7ff0c2a00640 1 -- 172.21.15.43:0/3496315221 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc00cd80 con 0x7ff0cc074510 2024-09-19T05:05:42.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.425+0000 7ff0c2a00640 1 -- 172.21.15.43:0/3496315221 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0bc005500 con 0x7ff0cc074510 2024-09-19T05:05:42.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.425+0000 7ff0cb200640 1 -- 172.21.15.43:0/3496315221 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0cc19f720 con 0x7ff0cc074510 2024-09-19T05:05:42.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.425+0000 7ff0c2a00640 1 -- 172.21.15.43:0/3496315221 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc010690 con 0x7ff0cc074510 2024-09-19T05:05:42.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.425+0000 7ff0cb200640 1 -- 172.21.15.43:0/3496315221 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0cc1a7000 con 0x7ff0cc074510 2024-09-19T05:05:42.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.430+0000 7ff0d3e5a740 1 Processor -- start 2024-09-19T05:05:42.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.431+0000 7ff0c2a00640 1 -- 172.21.15.43:0/3496315221 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7ff0bc0046c0 con 0x7ff0cc074510 2024-09-19T05:05:42.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.431+0000 7ff0d3e5a740 1 -- start start 2024-09-19T05:05:42.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.431+0000 7ff0d3e5a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f65d24c50 0x564f65d25030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:42.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.431+0000 7ff0d3e5a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f65d25570 0x564f65d38200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:42.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.431+0000 7ff0d3e5a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f65d38740 0x564f65d3ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:42.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.431+0000 7ff0d3e5a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564f65c8b3e0 con 0x564f65d38740 2024-09-19T05:05:42.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.431+0000 7ff0d3e5a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564f65c32d80 con 0x564f65d25570 2024-09-19T05:05:42.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.431+0000 7ff0d3e5a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564f65c873a0 con 0x564f65d24c50 2024-09-19T05:05:42.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.432+0000 7ff0ca800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f65d24c50 0x564f65d25030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:42.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.432+0000 7ff0ca800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f65d24c50 0x564f65d25030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:58618/0 (socket says 172.21.15.43:58618) 2024-09-19T05:05:42.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.432+0000 7ff0ca800640 1 -- 172.21.15.43:0/4067764556 learned_addr learned my addr 172.21.15.43:0/4067764556 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:05:42.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.432+0000 7ff0c9e00640 1 --2- 172.21.15.43:0/4067764556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f65d25570 0x564f65d38200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:42.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.432+0000 7ff0ca800640 1 -- 172.21.15.43:0/4067764556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f65d25570 msgr2=0x564f65d38200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:42.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.432+0000 7ff0ca800640 1 --2- 172.21.15.43:0/4067764556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f65d25570 0x564f65d38200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:42.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.432+0000 7ff0ca800640 1 -- 172.21.15.43:0/4067764556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f65d38740 msgr2=0x564f65d3ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:42.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.432+0000 7ff0ca800640 1 --2- 172.21.15.43:0/4067764556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f65d38740 0x564f65d3ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:42.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.432+0000 7ff0ca800640 1 -- 172.21.15.43:0/4067764556 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f65c51ae0 con 0x564f65d24c50 2024-09-19T05:05:42.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.433+0000 7ff0ca800640 1 --2- 172.21.15.43:0/4067764556 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f65d24c50 0x564f65d25030 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4004af0 tx=0x7ff0c4005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76306bfd7143b141 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:42.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.433+0000 7ff0b7400640 1 -- 172.21.15.43:0/4067764556 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4034040 con 0x564f65d24c50 2024-09-19T05:05:42.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.433+0000 7ff0b7400640 1 -- 172.21.15.43:0/4067764556 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0c4003e50 con 0x564f65d24c50 2024-09-19T05:05:42.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.433+0000 7ff0b7400640 1 -- 172.21.15.43:0/4067764556 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c40042a0 con 0x564f65d24c50 2024-09-19T05:05:42.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.433+0000 7ff0c2a00640 1 --2- 172.21.15.43:0/3496315221 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff0b00a35e0 0x7ff0b00a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:42.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.434+0000 7ff0c2a00640 1 -- 172.21.15.43:0/3496315221 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7ff0bc0d0450 con 0x7ff0cc074510 2024-09-19T05:05:42.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.434+0000 7ff0d3e5a740 1 -- 172.21.15.43:0/4067764556 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f65d24c50 msgr2=0x564f65d25030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:42.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.434+0000 7ff0d3e5a740 1 --2- 172.21.15.43:0/4067764556 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f65d24c50 0x564f65d25030 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4004af0 tx=0x7ff0c4005fd0 comp rx=0 tx=0).stop 2024-09-19T05:05:42.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.434+0000 7ff0c9e00640 1 --2- 172.21.15.43:0/3496315221 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff0b00a35e0 0x7ff0b00a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:42.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.435+0000 7ff0d3e5a740 1 -- 172.21.15.43:0/4067764556 shutdown_connections 2024-09-19T05:05:42.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.435+0000 7ff0d3e5a740 1 --2- 172.21.15.43:0/4067764556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f65d38740 0x564f65d3ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:42.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.435+0000 7ff0d3e5a740 1 --2- 172.21.15.43:0/4067764556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f65d25570 0x564f65d38200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:42.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.435+0000 7ff0d3e5a740 1 --2- 172.21.15.43:0/4067764556 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f65d24c50 0x564f65d25030 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:42.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.435+0000 7ff0d3e5a740 1 -- 172.21.15.43:0/4067764556 >> 172.21.15.43:0/4067764556 conn(0x564f65d212e0 msgr2=0x564f65d216c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:42.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.435+0000 7ff0d3e5a740 1 -- 172.21.15.43:0/4067764556 shutdown_connections 2024-09-19T05:05:42.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.435+0000 7ff0d3e5a740 1 -- 172.21.15.43:0/4067764556 wait complete. 2024-09-19T05:05:42.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.435+0000 7ff0c9e00640 1 --2- 172.21.15.43:0/3496315221 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff0b00a35e0 0x7ff0b00a5aa0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc0cd9d0 tx=0x7ff0bc0d3f00 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:42.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.436+0000 7ff0d3e5a740 1 -- start start 2024-09-19T05:05:42.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.436+0000 7ff0d3e5a740 1 Processor -- start 2024-09-19T05:05:42.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.437+0000 7ff0d3e5a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f65d24c50 0x564f65d356b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:42.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.437+0000 7ff0d3e5a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f65d25570 0x564f65d35bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:42.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.437+0000 7ff0d3e5a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f65d38740 0x564f65d36130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:42.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.437+0000 7ff0d3e5a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564f65d3b3b0 con 0x564f65d25570 2024-09-19T05:05:42.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.437+0000 7ff0d3e5a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564f65c460f0 con 0x564f65d38740 2024-09-19T05:05:42.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.437+0000 7ff0d3e5a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564f65d3b530 con 0x564f65d24c50 2024-09-19T05:05:42.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.437+0000 7ff0d3e5a740 10 client.0 _ll_register_callbacks cb 0x564f65c379e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:05:42.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.438+0000 7ff0ca800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f65d24c50 0x564f65d356b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:42.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.438+0000 7ff0ca800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f65d24c50 0x564f65d356b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:58626/0 (socket says 172.21.15.43:58626) 2024-09-19T05:05:42.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.438+0000 7ff0ca800640 1 -- 172.21.15.43:0/1370648054 learned_addr learned my addr 172.21.15.43:0/1370648054 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:05:42.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.438+0000 7ff0ca800640 1 --2- 172.21.15.43:0/1370648054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f65d38740 0x564f65d36130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:42.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.438+0000 7ff0c9400640 1 --2- 172.21.15.43:0/1370648054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f65d25570 0x564f65d35bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:42.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.438+0000 7ff0ca800640 1 -- 172.21.15.43:0/1370648054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f65d38740 msgr2=0x564f65d36130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:42.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.438+0000 7ff0ca800640 1 --2- 172.21.15.43:0/1370648054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f65d38740 0x564f65d36130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:42.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.438+0000 7ff0ca800640 1 -- 172.21.15.43:0/1370648054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f65d25570 msgr2=0x564f65d35bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:42.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.438+0000 7ff0ca800640 1 --2- 172.21.15.43:0/1370648054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f65d25570 0x564f65d35bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:42.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.438+0000 7ff0ca800640 1 -- 172.21.15.43:0/1370648054 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f659fc3c0 con 0x564f65d24c50 2024-09-19T05:05:42.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.439+0000 7ff0ca800640 1 --2- 172.21.15.43:0/1370648054 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f65d24c50 0x564f65d356b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff0c402ae50 tx=0x7ff0c402fdf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33ad4be71defde14 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:42.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.439+0000 7ff0abe00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:05:42.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.439+0000 7ff0abe00640 1 -- 172.21.15.43:0/1370648054 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4056060 con 0x564f65d24c50 2024-09-19T05:05:42.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.439+0000 7ff0abe00640 1 -- 172.21.15.43:0/1370648054 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7ff0c402f420 con 0x564f65d24c50 2024-09-19T05:05:42.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.439+0000 7ff0d3e5a740 1 -- 172.21.15.43:0/1370648054 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0c4003e50 con 0x564f65d24c50 2024-09-19T05:05:42.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.439+0000 7ff0abe00640 1 -- 172.21.15.43:0/1370648054 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff0c402f620 con 0x564f65d24c50 2024-09-19T05:05:42.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.439+0000 7ff0d3e5a740 10 client.25429 fetch_fsmap learned FSMap version 29 2024-09-19T05:05:42.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.439+0000 7ff0d3e5a740 1 -- 172.21.15.43:0/1370648054 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564f65d3b3b0 con 0x564f65d24c50 2024-09-19T05:05:42.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.440+0000 7ff0abe00640 1 -- 172.21.15.43:0/1370648054 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0c40044e0 con 0x564f65d24c50 2024-09-19T05:05:42.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.441+0000 7ff0abe00640 1 -- 172.21.15.43:0/1370648054 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff0c4004000 con 0x564f65d24c50 2024-09-19T05:05:42.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.441+0000 7ff0d3e5a740 10 client.25429 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:05:42.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.441+0000 7ff0d3e5a740 10 client.25429 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:05:42.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.441+0000 7ff0d3e5a740 20 client.25429 populate_metadata read hostname 'smithi043' 2024-09-19T05:05:42.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.441+0000 7ff0d3e5a740 1 --2- 172.21.15.43:0/1370648054 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x564f65d482a0 0x564f65d4a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:42.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.441+0000 7ff0d3e5a740 4 client.25429 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:05:42.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.441+0000 7ff0d3e5a740 1 -- 172.21.15.43:0/1370648054 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff0c4003e50 con 0x564f65d482a0 2024-09-19T05:05:42.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.442+0000 7ff0c9400640 1 --2- 172.21.15.43:0/1370648054 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x564f65d482a0 0x564f65d4a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:42.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.443+0000 7ff0c9400640 1 --2- 172.21.15.43:0/1370648054 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x564f65d482a0 0x564f65d4a690 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a86f9c85cd782f9 server_cookie=7d2e549ae4b18e74 in_seq=0 out_seq=0 2024-09-19T05:05:42.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.443+0000 7ff0abe00640 10 client.25429 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:05:42.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.443+0000 7ff0abe00640 1 -- 172.21.15.43:0/1370648054 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7ff0c4003e50 con 0x564f65d482a0 2024-09-19T05:05:42.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.443+0000 7ff0abe00640 10 client.25429 handle_command_reply: tid=0 2024-09-19T05:05:42.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.444+0000 7ff0d3e5a740 1 client.25429 shutdown 2024-09-19T05:05:42.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.444+0000 7ff0d3e5a740 1 -- 172.21.15.43:0/1370648054 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f65d24c50 msgr2=0x564f65d356b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:42.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.444+0000 7ff0d3e5a740 1 --2- 172.21.15.43:0/1370648054 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f65d24c50 0x564f65d356b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff0c402ae50 tx=0x7ff0c402fdf0 comp rx=0 tx=0).stop 2024-09-19T05:05:42.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.444+0000 7ff0d3e5a740 1 -- 172.21.15.43:0/1370648054 shutdown_connections 2024-09-19T05:05:42.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.444+0000 7ff0d3e5a740 1 --2- 172.21.15.43:0/1370648054 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x564f65d482a0 0x564f65d4a690 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:42.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.444+0000 7ff0abe00640 0 client.25429 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:05:42.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.444+0000 7ff0d3e5a740 1 --2- 172.21.15.43:0/1370648054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f65d38740 0x564f65d36130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:42.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.444+0000 7ff0d3e5a740 1 --2- 172.21.15.43:0/1370648054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f65d25570 0x564f65d35bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:42.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.444+0000 7ff0d3e5a740 1 --2- 172.21.15.43:0/1370648054 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f65d24c50 0x564f65d356b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:42.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.444+0000 7ff0d3e5a740 1 -- 172.21.15.43:0/1370648054 >> 172.21.15.43:0/1370648054 conn(0x564f65d212e0 msgr2=0x564f65d27040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:42.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.445+0000 7ff0d3e5a740 1 -- 172.21.15.43:0/1370648054 shutdown_connections 2024-09-19T05:05:42.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.445+0000 7ff0d3e5a740 1 -- 172.21.15.43:0/1370648054 wait complete. 2024-09-19T05:05:42.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.445+0000 7ff0d3e5a740 20 client.25429 trim_cache size 0 max 16384 2024-09-19T05:05:42.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.480+0000 7ff0d3e5a740 1 Processor -- start 2024-09-19T05:05:42.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.480+0000 7ff0d3e5a740 1 -- start start 2024-09-19T05:05:42.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.481+0000 7ff0d3e5a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f65d24c50 0x564f65d26820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:42.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.481+0000 7ff0d3e5a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f65d25570 0x564f65d35f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:42.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.481+0000 7ff0d3e5a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f65d38740 0x564f65d4fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:42.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.481+0000 7ff0d3e5a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc1ab470 con 0x564f65d24c50 2024-09-19T05:05:42.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.481+0000 7ff0d3e5a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc1acfb0 con 0x564f65d25570 2024-09-19T05:05:42.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.481+0000 7ff0d3e5a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc1abb40 con 0x564f65d38740 2024-09-19T05:05:42.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.481+0000 7ff0ca800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f65d24c50 0x564f65d26820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:42.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.481+0000 7ff0c9400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f65d25570 0x564f65d35f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:42.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.481+0000 7ff0ca800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f65d24c50 0x564f65d26820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51856/0 (socket says 172.21.15.43:51856) 2024-09-19T05:05:42.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.481+0000 7ff0c9400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f65d25570 0x564f65d35f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:55718/0 (socket says 172.21.15.43:55718) 2024-09-19T05:05:42.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.481+0000 7ff0ca800640 1 -- 172.21.15.43:0/2034643389 learned_addr learned my addr 172.21.15.43:0/2034643389 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:05:42.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.482+0000 7ff0c9400640 1 -- 172.21.15.43:0/2034643389 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f65d38740 msgr2=0x564f65d4fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:42.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.482+0000 7ff0c9400640 1 --2- 172.21.15.43:0/2034643389 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f65d38740 0x564f65d4fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:42.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.482+0000 7ff0c9400640 1 -- 172.21.15.43:0/2034643389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f65d24c50 msgr2=0x564f65d26820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:42.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.482+0000 7ff0c9400640 1 --2- 172.21.15.43:0/2034643389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f65d24c50 0x564f65d26820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:42.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.482+0000 7ff0c9400640 1 -- 172.21.15.43:0/2034643389 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f65d31290 con 0x564f65d25570 2024-09-19T05:05:42.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.482+0000 7ff0ca800640 1 --2- 172.21.15.43:0/2034643389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f65d24c50 0x564f65d26820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:05:42.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.483+0000 7ff0c9400640 1 --2- 172.21.15.43:0/2034643389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f65d25570 0x564f65d35f80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x564f65d36cf0 tx=0x7ff0b8009730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=103f3fa40fd9a3b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:42.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.483+0000 7ff0b6a00640 1 -- 172.21.15.43:0/2034643389 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b80140d0 con 0x564f65d25570 2024-09-19T05:05:42.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.483+0000 7ff0b6a00640 1 -- 172.21.15.43:0/2034643389 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0b8014270 con 0x564f65d25570 2024-09-19T05:05:42.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.484+0000 7ff0b6a00640 1 -- 172.21.15.43:0/2034643389 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b8004990 con 0x564f65d25570 2024-09-19T05:05:42.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.485+0000 7ff0d3e5a740 1 -- 172.21.15.43:0/2034643389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f65d25570 msgr2=0x564f65d35f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:42.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.485+0000 7ff0d3e5a740 1 --2- 172.21.15.43:0/2034643389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f65d25570 0x564f65d35f80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x564f65d36cf0 tx=0x7ff0b8009730 comp rx=0 tx=0).stop 2024-09-19T05:05:42.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.485+0000 7ff0d3e5a740 1 -- 172.21.15.43:0/2034643389 shutdown_connections 2024-09-19T05:05:42.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.485+0000 7ff0d3e5a740 1 --2- 172.21.15.43:0/2034643389 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f65d38740 0x564f65d4fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:42.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.485+0000 7ff0d3e5a740 1 --2- 172.21.15.43:0/2034643389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f65d25570 0x564f65d35f80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:42.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.485+0000 7ff0d3e5a740 1 --2- 172.21.15.43:0/2034643389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f65d24c50 0x564f65d26820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:42.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.485+0000 7ff0d3e5a740 1 -- 172.21.15.43:0/2034643389 >> 172.21.15.43:0/2034643389 conn(0x564f65d212e0 msgr2=0x564f65d34670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:42.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.486+0000 7ff0d3e5a740 1 -- 172.21.15.43:0/2034643389 shutdown_connections 2024-09-19T05:05:42.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.486+0000 7ff0d3e5a740 1 -- 172.21.15.43:0/2034643389 wait complete. 2024-09-19T05:05:42.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.487+0000 7ff0d3e5a740 1 -- start start 2024-09-19T05:05:42.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.487+0000 7ff0d3e5a740 1 Processor -- start 2024-09-19T05:05:42.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.488+0000 7ff0d3e5a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f65d24c50 0x564f65d3c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:42.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.488+0000 7ff0d3e5a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f65d25570 0x564f65d3cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:42.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.488+0000 7ff0d3e5a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f65d38740 0x564f65d3d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:42.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.488+0000 7ff0d3e5a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc1ac8e0 con 0x564f65d24c50 2024-09-19T05:05:42.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.488+0000 7ff0d3e5a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc1ac210 con 0x564f65d38740 2024-09-19T05:05:42.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.488+0000 7ff0d3e5a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc1a9450 con 0x564f65d25570 2024-09-19T05:05:42.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.488+0000 7ff0d3e5a740 10 client.0 _ll_register_callbacks cb 0x7ff0c4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:05:42.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.488+0000 7ff0c9400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f65d25570 0x564f65d3cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:42.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.488+0000 7ff0c9400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f65d25570 0x564f65d3cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:58640/0 (socket says 172.21.15.43:58640) 2024-09-19T05:05:42.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.488+0000 7ff0c9400640 1 -- 172.21.15.43:0/2594379452 learned_addr learned my addr 172.21.15.43:0/2594379452 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:05:42.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.489+0000 7ff0ca800640 1 --2- 172.21.15.43:0/2594379452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f65d24c50 0x564f65d3c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:42.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.490+0000 7ff0c9400640 1 -- 172.21.15.43:0/2594379452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f65d38740 msgr2=0x564f65d3d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:42.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.490+0000 7ff0c9400640 1 --2- 172.21.15.43:0/2594379452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f65d38740 0x564f65d3d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:42.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.490+0000 7ff0c9400640 1 -- 172.21.15.43:0/2594379452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f65d24c50 msgr2=0x564f65d3c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:05:42.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.490+0000 7ff0c9400640 1 --2- 172.21.15.43:0/2594379452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f65d24c50 0x564f65d3c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:42.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.490+0000 7ff0c9400640 1 -- 172.21.15.43:0/2594379452 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f65db1e70 con 0x564f65d25570 2024-09-19T05:05:42.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.490+0000 7ff0ca800640 1 --2- 172.21.15.43:0/2594379452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f65d24c50 0x564f65d3c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:05:42.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.491+0000 7ff0c9400640 1 --2- 172.21.15.43:0/2594379452 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f65d25570 0x564f65d3cbb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8005640 tx=0x7ff0b8015de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d0582279534321d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:05:42.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.491+0000 7ff0abe00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:05:42.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.491+0000 7ff0abe00640 1 -- 172.21.15.43:0/2594379452 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b8004260 con 0x564f65d25570 2024-09-19T05:05:42.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.491+0000 7ff0abe00640 1 -- 172.21.15.43:0/2594379452 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7ff0b8004400 con 0x564f65d25570 2024-09-19T05:05:42.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.491+0000 7ff0d3e5a740 1 -- 172.21.15.43:0/2594379452 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0b8014270 con 0x564f65d25570 2024-09-19T05:05:42.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.492+0000 7ff0abe00640 1 -- 172.21.15.43:0/2594379452 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff0b80035f0 con 0x564f65d25570 2024-09-19T05:05:42.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.492+0000 7ff0d3e5a740 10 client.25432 fetch_fsmap learned FSMap version 29 2024-09-19T05:05:42.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.492+0000 7ff0d3e5a740 1 -- 172.21.15.43:0/2594379452 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564f65db3140 con 0x564f65d25570 2024-09-19T05:05:42.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.493+0000 7ff0abe00640 1 -- 172.21.15.43:0/2594379452 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0b8014270 con 0x564f65d25570 2024-09-19T05:05:42.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.494+0000 7ff0abe00640 1 -- 172.21.15.43:0/2594379452 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff0b8012940 con 0x564f65d25570 2024-09-19T05:05:42.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.494+0000 7ff0d3e5a740 10 client.25432 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:05:42.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.494+0000 7ff0d3e5a740 10 client.25432 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:05:42.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.494+0000 7ff0d3e5a740 20 client.25432 populate_metadata read hostname 'smithi043' 2024-09-19T05:05:42.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.494+0000 7ff0d3e5a740 1 --2- 172.21.15.43:0/2594379452 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x564f65db3800 0x564f65db5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:05:42.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.494+0000 7ff0d3e5a740 4 client.25432 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:05:42.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.494+0000 7ff0d3e5a740 1 -- 172.21.15.43:0/2594379452 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564f65c98950 con 0x564f65db3800 2024-09-19T05:05:42.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.494+0000 7ff0ca800640 1 --2- 172.21.15.43:0/2594379452 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x564f65db3800 0x564f65db5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:05:42.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.495+0000 7ff0ca800640 1 --2- 172.21.15.43:0/2594379452 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x564f65db3800 0x564f65db5c20 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4c78a493d740a50 server_cookie=fada4a9b374af6e6 in_seq=0 out_seq=0 2024-09-19T05:05:42.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.495+0000 7ff0abe00640 10 client.25432 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:05:42.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.495+0000 7ff0abe00640 1 -- 172.21.15.43:0/2594379452 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564f65c98950 con 0x564f65db3800 2024-09-19T05:05:42.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.495+0000 7ff0abe00640 10 client.25432 handle_command_reply: tid=0 2024-09-19T05:05:42.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.495+0000 7ff0d3e5a740 1 client.25432 shutdown 2024-09-19T05:05:42.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.496+0000 7ff0d3e5a740 1 -- 172.21.15.43:0/2594379452 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f65d25570 msgr2=0x564f65d3cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:42.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.496+0000 7ff0d3e5a740 1 --2- 172.21.15.43:0/2594379452 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f65d25570 0x564f65d3cbb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8005640 tx=0x7ff0b8015de0 comp rx=0 tx=0).stop 2024-09-19T05:05:42.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.496+0000 7ff0d3e5a740 1 -- 172.21.15.43:0/2594379452 shutdown_connections 2024-09-19T05:05:42.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.496+0000 7ff0d3e5a740 1 --2- 172.21.15.43:0/2594379452 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x564f65db3800 0x564f65db5c20 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:42.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.496+0000 7ff0d3e5a740 1 --2- 172.21.15.43:0/2594379452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f65d38740 0x564f65d3d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:42.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.496+0000 7ff0d3e5a740 1 --2- 172.21.15.43:0/2594379452 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f65d25570 0x564f65d3cbb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:42.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.496+0000 7ff0d3e5a740 1 --2- 172.21.15.43:0/2594379452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f65d24c50 0x564f65d3c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:42.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.496+0000 7ff0d3e5a740 1 -- 172.21.15.43:0/2594379452 >> 172.21.15.43:0/2594379452 conn(0x564f65d212e0 msgr2=0x564f65d36f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:42.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.496+0000 7ff0abe00640 0 client.25432 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:05:42.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.496+0000 7ff0d3e5a740 1 -- 172.21.15.43:0/2594379452 shutdown_connections 2024-09-19T05:05:42.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.497+0000 7ff0d3e5a740 1 -- 172.21.15.43:0/2594379452 wait complete. 2024-09-19T05:05:42.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.497+0000 7ff0d3e5a740 20 client.25432 trim_cache size 0 max 16384 2024-09-19T05:05:42.500 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:05:42.500 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T05:05:42.500 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-09-19T05:05:42.500 INFO:teuthology.orchestra.run.smithi043.stdout: "f237750e-13ff-4229-94a4-59db242c6c08": { 2024-09-19T05:05:42.500 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-09-19T05:05:42.500 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "f237750e-13ff-4229-94a4-59db242c6c08", 2024-09-19T05:05:42.500 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-09-19T05:05:42.500 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:05:42.500 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:05:42.500 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:05:42.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.498+0000 7ff0cb200640 1 -- 172.21.15.43:0/3496315221 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff0b00a35e0 msgr2=0x7ff0b00a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:42.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.498+0000 7ff0cb200640 1 --2- 172.21.15.43:0/3496315221 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff0b00a35e0 0x7ff0b00a5aa0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc0cd9d0 tx=0x7ff0bc0d3f00 comp rx=0 tx=0).stop 2024-09-19T05:05:42.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.498+0000 7ff0cb200640 1 -- 172.21.15.43:0/3496315221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0cc074510 msgr2=0x7ff0cc19a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:05:42.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.498+0000 7ff0cb200640 1 --2- 172.21.15.43:0/3496315221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0cc074510 0x7ff0cc19a480 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc00e9f0 tx=0x7ff0bc00edf0 comp rx=0 tx=0).stop 2024-09-19T05:05:42.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.499+0000 7ff0cb200640 1 -- 172.21.15.43:0/3496315221 shutdown_connections 2024-09-19T05:05:42.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.499+0000 7ff0cb200640 1 --2- 172.21.15.43:0/3496315221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0cc10e280 0x7ff0cc19ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:42.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.499+0000 7ff0cb200640 1 --2- 172.21.15.43:0/3496315221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0cc074e30 0x7ff0cc19a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:42.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.499+0000 7ff0cb200640 1 --2- 172.21.15.43:0/3496315221 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff0b00a35e0 0x7ff0b00a5aa0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:42.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.499+0000 7ff0cb200640 1 --2- 172.21.15.43:0/3496315221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0cc074510 0x7ff0cc19a480 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:05:42.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.499+0000 7ff0cb200640 1 -- 172.21.15.43:0/3496315221 >> 172.21.15.43:0/3496315221 conn(0x7ff0cc0fd740 msgr2=0x7ff0cc0fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:05:42.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.499+0000 7ff0cb200640 1 -- 172.21.15.43:0/3496315221 shutdown_connections 2024-09-19T05:05:42.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:05:42.499+0000 7ff0cb200640 1 -- 172.21.15.43:0/3496315221 wait complete. 2024-09-19T05:05:43.295 INFO:tasks.cephfs.filesystem:scrub status for tag:f237750e-13ff-4229-94a4-59db242c6c08 - {'path': '/', 'tag': 'f237750e-13ff-4229-94a4-59db242c6c08', 'options': 'recursive,force'} 2024-09-19T05:05:43.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:43 smithi043 ceph-mon[30800]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:05:43.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:43 smithi043 ceph-mon[30800]: scrub summary: idle 2024-09-19T05:05:43.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:43 smithi151 ceph-mon[35425]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:05:43.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:43 smithi151 ceph-mon[35425]: scrub summary: idle 2024-09-19T05:05:43.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:43 smithi096 ceph-mon[35085]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:05:43.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:43 smithi096 ceph-mon[35085]: scrub summary: idle 2024-09-19T05:05:45.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:45 smithi151 ceph-mon[35425]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-09-19T05:05:45.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:45 smithi096 ceph-mon[35085]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-09-19T05:05:45.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:45 smithi043 ceph-mon[30800]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-09-19T05:05:47.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:47 smithi151 ceph-mon[35425]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-19T05:05:47.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:47 smithi096 ceph-mon[35085]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-19T05:05:47.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:47 smithi043 ceph-mon[30800]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-19T05:05:49.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:49 smithi151 ceph-mon[35425]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:05:49.725 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:49 smithi096 ceph-mon[35085]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:05:49.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:49 smithi043 ceph-mon[30800]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:05:50.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:50 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:05:50.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:50 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:05:50.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:50 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:05:50.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:50 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:05:50.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:50 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:05:50.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:50 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:05:50.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:50 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:05:50.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:50 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:05:50.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:50 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:05:50.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:50 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:05:50.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:50 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:05:50.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:50 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:05:50.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:50 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:05:50.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:50 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:05:50.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:50 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:05:51.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:51 smithi043 ceph-mon[30800]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:05:51.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:51 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:05:51.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:51 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:05:51.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:51 smithi151 ceph-mon[35425]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:05:51.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:51 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:05:51.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:51 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:05:51.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:51 smithi096 ceph-mon[35085]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:05:51.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:51 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:05:51.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:51 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:05:52.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:52 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:05:52.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:52 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:05:52.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:52 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:05:52.806 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:52 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:05:52.806 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:52 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:05:52.806 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:52 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:05:52.874 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:52 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:05:52.875 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:52 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:05:52.875 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:52 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:05:53.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:53 smithi043 ceph-mon[30800]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:05:53.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:53 smithi151 ceph-mon[35425]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:05:53.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:53 smithi096 ceph-mon[35085]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:05:55.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:55 smithi043 ceph-mon[30800]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 12 op/s 2024-09-19T05:05:55.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:55 smithi151 ceph-mon[35425]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 12 op/s 2024-09-19T05:05:55.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:55 smithi096 ceph-mon[35085]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 12 op/s 2024-09-19T05:05:57.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:57 smithi043 ceph-mon[30800]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-19T05:05:57.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:57 smithi151 ceph-mon[35425]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-19T05:05:57.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:57 smithi096 ceph-mon[35085]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-19T05:05:59.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:05:59 smithi043 ceph-mon[30800]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:05:59.904 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:05:59 smithi151 ceph-mon[35425]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:05:59.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:05:59 smithi096 ceph-mon[35085]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:06:00.574 INFO:tasks.workunit.client.0.smithi043.stderr:Updating files: 62% (8076/12896) Updating files: 63% (8125/12896) Updating files: 64% (8254/12896) Updating files: 65% (8383/12896) Updating files: 66% (8512/12896) Updating files: 67% (8641/12896) Updating files: 68% (8770/12896) Updating files: 69% (8899/12896) Updating files: 70% (9028/12896) Updating files: 71% (9157/12896) Updating files: 72% (9286/12896) Updating files: 73% (9415/12896) Updating files: 74% (9544/12896) Updating files: 75% (9672/12896) Updating files: 76% (9801/12896) Updating files: 77% (9930/12896) Updating files: 78% (10059/12896) Updating files: 79% (10188/12896) Updating files: 80% (10317/12896) Updating files: 81% (10446/12896) Updating files: 82% (10575/12896) Updating files: 83% (10704/12896) Updating files: 84% (10833/12896) Updating files: 85% (10962/12896) Updating files: 86% (11091/12896) Updating files: 87% (11220/12896) Updating files: 88% (11349/12896) Updating files: 89% (11478/12896) Updating files: 90% (11607/12896) Updating files: 91% (11736/12896) Updating files: 92% (11865/12896) Updating files: 93% (11994/12896) Updating files: 94% (12123/12896) Updating files: 95% (12252/12896) Updating files: 96% (12381/12896) Updating files: 97% (12510/12896) Updating files: 97% (12625/12896) Updating files: 98% (12639/12896) Updating files: 99% (12768/12896) Updating files: 100% (12896/12896) Updating files: 100% (12896/12896), done. 2024-09-19T05:06:01.553 INFO:tasks.workunit.client.0.smithi043.stderr:Note: switching to '6fad5dd73c93545e52f57f681e140a9c3529c11e'. 2024-09-19T05:06:01.553 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-09-19T05:06:01.553 INFO:tasks.workunit.client.0.smithi043.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-19T05:06:01.553 INFO:tasks.workunit.client.0.smithi043.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-19T05:06:01.553 INFO:tasks.workunit.client.0.smithi043.stderr:state without impacting any branches by switching back to a branch. 2024-09-19T05:06:01.553 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-09-19T05:06:01.553 INFO:tasks.workunit.client.0.smithi043.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-19T05:06:01.554 INFO:tasks.workunit.client.0.smithi043.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-19T05:06:01.554 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-09-19T05:06:01.554 INFO:tasks.workunit.client.0.smithi043.stderr: git switch -c 2024-09-19T05:06:01.554 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-09-19T05:06:01.554 INFO:tasks.workunit.client.0.smithi043.stderr:Or undo this operation with: 2024-09-19T05:06:01.554 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-09-19T05:06:01.554 INFO:tasks.workunit.client.0.smithi043.stderr: git switch - 2024-09-19T05:06:01.554 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-09-19T05:06:01.554 INFO:tasks.workunit.client.0.smithi043.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-19T05:06:01.554 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-09-19T05:06:01.554 INFO:tasks.workunit.client.0.smithi043.stderr:HEAD is now at 6fad5dd73c9 Merge pull request #59812 from neha-ojha/wip-rtd-squid 2024-09-19T05:06:01.567 DEBUG:teuthology.orchestra.run.smithi043:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-19T05:06:01.613 INFO:tasks.workunit.client.0.smithi043.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-19T05:06:01.618 INFO:tasks.workunit.client.0.smithi043.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-19T05:06:01.618 INFO:tasks.workunit.client.0.smithi043.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-19T05:06:01.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:01 smithi151 ceph-mon[35425]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:06:01.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:01 smithi096 ceph-mon[35085]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:06:02.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:01 smithi043 ceph-mon[30800]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:06:02.350 INFO:tasks.workunit.client.0.smithi043.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-19T05:06:02.447 INFO:tasks.workunit.client.0.smithi043.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-19T05:06:02.514 INFO:tasks.workunit.client.0.smithi043.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-19T05:06:02.517 INFO:tasks.workunit.client.0.smithi043.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-19T05:06:02.517 INFO:tasks.workunit.client.0.smithi043.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-19T05:06:02.583 INFO:tasks.workunit.client.0.smithi043.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-19T05:06:02.590 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-19T05:06:02.591 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-19T05:06:02.627 INFO:tasks.workunit:Running workunits matching suites/iozone.sh on client.0... 2024-09-19T05:06:02.628 INFO:tasks.workunit:Running workunit suites/iozone.sh... 2024-09-19T05:06:02.628 DEBUG:teuthology.orchestra.run.smithi043:workunit test suites/iozone.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fad5dd73c93545e52f57f681e140a9c3529c11e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh 2024-09-19T05:06:02.717 INFO:tasks.workunit.client.0.smithi043.stderr:+ iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-09-19T05:06:02.803 INFO:tasks.workunit.client.0.smithi043.stdout: Iozone: Performance Test of File I/O 2024-09-19T05:06:02.803 INFO:tasks.workunit.client.0.smithi043.stdout: Version $Revision: 3.494 $ 2024-09-19T05:06:02.803 INFO:tasks.workunit.client.0.smithi043.stdout: Compiled for 64 bit mode. 2024-09-19T05:06:02.803 INFO:tasks.workunit.client.0.smithi043.stdout: Build: linux-AMD64 2024-09-19T05:06:02.803 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-19T05:06:02.803 INFO:tasks.workunit.client.0.smithi043.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-09-19T05:06:02.803 INFO:tasks.workunit.client.0.smithi043.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-09-19T05:06:02.803 INFO:tasks.workunit.client.0.smithi043.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-09-19T05:06:02.804 INFO:tasks.workunit.client.0.smithi043.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-09-19T05:06:02.804 INFO:tasks.workunit.client.0.smithi043.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-09-19T05:06:02.804 INFO:tasks.workunit.client.0.smithi043.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-09-19T05:06:02.804 INFO:tasks.workunit.client.0.smithi043.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-09-19T05:06:02.804 INFO:tasks.workunit.client.0.smithi043.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-09-19T05:06:02.804 INFO:tasks.workunit.client.0.smithi043.stdout: Alexey Skidanov, Sudhir Kumar. 2024-09-19T05:06:02.804 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-19T05:06:02.804 INFO:tasks.workunit.client.0.smithi043.stdout: Run began: Thu Sep 19 05:06:02 2024 2024-09-19T05:06:02.804 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-19T05:06:02.804 INFO:tasks.workunit.client.0.smithi043.stdout: Include close in write timing 2024-09-19T05:06:02.804 INFO:tasks.workunit.client.0.smithi043.stdout: Include fsync in write timing 2024-09-19T05:06:02.804 INFO:tasks.workunit.client.0.smithi043.stdout: File size set to 1048576 kB 2024-09-19T05:06:02.804 INFO:tasks.workunit.client.0.smithi043.stdout: Record Size 16 kB 2024-09-19T05:06:02.805 INFO:tasks.workunit.client.0.smithi043.stdout: Command line used: iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-09-19T05:06:02.805 INFO:tasks.workunit.client.0.smithi043.stdout: Output is in kBytes/sec 2024-09-19T05:06:02.805 INFO:tasks.workunit.client.0.smithi043.stdout: Time Resolution = 0.000001 seconds. 2024-09-19T05:06:02.805 INFO:tasks.workunit.client.0.smithi043.stdout: Processor cache size set to 1024 kBytes. 2024-09-19T05:06:02.805 INFO:tasks.workunit.client.0.smithi043.stdout: Processor cache line size set to 32 bytes. 2024-09-19T05:06:02.805 INFO:tasks.workunit.client.0.smithi043.stdout: File stride size set to 17 * record size. 2024-09-19T05:06:02.805 INFO:tasks.workunit.client.0.smithi043.stdout: Throughput test with 1 process 2024-09-19T05:06:02.805 INFO:tasks.workunit.client.0.smithi043.stdout: Each process writes a 1048576 kByte file in 16 kByte records 2024-09-19T05:06:03.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:03 smithi151 ceph-mon[35425]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:06:03.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:03 smithi096 ceph-mon[35085]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:06:04.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:03 smithi043 ceph-mon[30800]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:06:05.904 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:05 smithi151 ceph-mon[35425]: pgmap v46: 97 pgs: 97 active+clean; 57 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.7 MiB/s wr, 30 op/s 2024-09-19T05:06:05.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:05 smithi096 ceph-mon[35085]: pgmap v46: 97 pgs: 97 active+clean; 57 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.7 MiB/s wr, 30 op/s 2024-09-19T05:06:06.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:05 smithi043 ceph-mon[30800]: pgmap v46: 97 pgs: 97 active+clean; 57 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.7 MiB/s wr, 30 op/s 2024-09-19T05:06:06.884 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-19T05:06:06.884 INFO:tasks.workunit.client.0.smithi043.stdout: Children see throughput for 1 initial writers = 257188.06 kB/sec 2024-09-19T05:06:06.884 INFO:tasks.workunit.client.0.smithi043.stdout: Parent sees throughput for 1 initial writers = 257149.07 kB/sec 2024-09-19T05:06:06.885 INFO:tasks.workunit.client.0.smithi043.stdout: Min throughput per process = 257188.06 kB/sec 2024-09-19T05:06:06.885 INFO:tasks.workunit.client.0.smithi043.stdout: Max throughput per process = 257188.06 kB/sec 2024-09-19T05:06:06.885 INFO:tasks.workunit.client.0.smithi043.stdout: Avg throughput per process = 257188.06 kB/sec 2024-09-19T05:06:06.885 INFO:tasks.workunit.client.0.smithi043.stdout: Min xfer = 1048576.00 kB 2024-09-19T05:06:07.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:07 smithi151 ceph-mon[35425]: pgmap v47: 97 pgs: 97 active+clean; 57 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.7 MiB/s wr, 26 op/s 2024-09-19T05:06:07.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:07 smithi096 ceph-mon[35085]: pgmap v47: 97 pgs: 97 active+clean; 57 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.7 MiB/s wr, 26 op/s 2024-09-19T05:06:08.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:07 smithi043 ceph-mon[30800]: pgmap v47: 97 pgs: 97 active+clean; 57 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.7 MiB/s wr, 26 op/s 2024-09-19T05:06:09.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:09 smithi151 ceph-mon[35425]: pgmap v48: 97 pgs: 97 active+clean; 57 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.7 MiB/s wr, 25 op/s 2024-09-19T05:06:09.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:09 smithi096 ceph-mon[35085]: pgmap v48: 97 pgs: 97 active+clean; 57 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.7 MiB/s wr, 25 op/s 2024-09-19T05:06:10.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:09 smithi043 ceph-mon[30800]: pgmap v48: 97 pgs: 97 active+clean; 57 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.7 MiB/s wr, 25 op/s 2024-09-19T05:06:11.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:11 smithi151 ceph-mon[35425]: pgmap v49: 97 pgs: 97 active+clean; 1.0 GiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 85 MiB/s wr, 51 op/s 2024-09-19T05:06:11.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:11 smithi151 ceph-mon[35425]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-09-19T05:06:11.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:11 smithi096 ceph-mon[35085]: pgmap v49: 97 pgs: 97 active+clean; 1.0 GiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 85 MiB/s wr, 51 op/s 2024-09-19T05:06:11.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:11 smithi096 ceph-mon[35085]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-09-19T05:06:12.034 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:11 smithi043 ceph-mon[30800]: pgmap v49: 97 pgs: 97 active+clean; 1.0 GiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 85 MiB/s wr, 51 op/s 2024-09-19T05:06:12.034 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:11 smithi043 ceph-mon[30800]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-09-19T05:06:13.296 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:06:13.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:13 smithi043 ceph-mon[30800]: pgmap v50: 97 pgs: 97 active+clean; 1.0 GiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 85 MiB/s wr, 47 op/s 2024-09-19T05:06:13.904 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:13 smithi151 ceph-mon[35425]: pgmap v50: 97 pgs: 97 active+clean; 1.0 GiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 85 MiB/s wr, 47 op/s 2024-09-19T05:06:13.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:13 smithi096 ceph-mon[35085]: pgmap v50: 97 pgs: 97 active+clean; 1.0 GiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 85 MiB/s wr, 47 op/s 2024-09-19T05:06:14.048 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:06:14.480 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-19T05:06:14.480 INFO:tasks.workunit.client.0.smithi043.stdout: Children see throughput for 1 rewriters = 301916.94 kB/sec 2024-09-19T05:06:14.481 INFO:tasks.workunit.client.0.smithi043.stdout: Parent sees throughput for 1 rewriters = 301865.85 kB/sec 2024-09-19T05:06:14.481 INFO:tasks.workunit.client.0.smithi043.stdout: Min throughput per process = 301916.94 kB/sec 2024-09-19T05:06:14.481 INFO:tasks.workunit.client.0.smithi043.stdout: Max throughput per process = 301916.94 kB/sec 2024-09-19T05:06:14.481 INFO:tasks.workunit.client.0.smithi043.stdout: Avg throughput per process = 301916.94 kB/sec 2024-09-19T05:06:14.481 INFO:tasks.workunit.client.0.smithi043.stdout: Min xfer = 1048576.00 kB 2024-09-19T05:06:15.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.415+0000 7efefcc00640 1 -- 172.21.15.43:0/692433752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efef810e3c0 msgr2=0x7efef8110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:15.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.415+0000 7efefcc00640 1 --2- 172.21.15.43:0/692433752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efef810e3c0 0x7efef8110810 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efef0009ca0 tx=0x7efef002a950 comp rx=0 tx=0).stop 2024-09-19T05:06:15.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.415+0000 7efefcc00640 1 -- 172.21.15.43:0/692433752 shutdown_connections 2024-09-19T05:06:15.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.415+0000 7efefcc00640 1 --2- 172.21.15.43:0/692433752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efef810e3c0 0x7efef8110810 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:15.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.415+0000 7efefcc00640 1 --2- 172.21.15.43:0/692433752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efef8074f40 0x7efef810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:15.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.415+0000 7efefcc00640 1 --2- 172.21.15.43:0/692433752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef8074620 0x7efef8074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:15.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.415+0000 7efefcc00640 1 -- 172.21.15.43:0/692433752 >> 172.21.15.43:0/692433752 conn(0x7efef80fd8c0 msgr2=0x7efef80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:15.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.415+0000 7efefcc00640 1 -- 172.21.15.43:0/692433752 shutdown_connections 2024-09-19T05:06:15.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.415+0000 7efefcc00640 1 -- 172.21.15.43:0/692433752 wait complete. 2024-09-19T05:06:15.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.416+0000 7efefcc00640 1 Processor -- start 2024-09-19T05:06:15.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.416+0000 7efefcc00640 1 -- start start 2024-09-19T05:06:15.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.416+0000 7efefcc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efef8074620 0x7efef819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:15.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.416+0000 7efefcc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efef8074f40 0x7efef819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:15.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.416+0000 7efefcc00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef810e3c0 0x7efef819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:15.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.417+0000 7efefcc00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efef8114150 con 0x7efef8074f40 2024-09-19T05:06:15.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.417+0000 7efefcc00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efef8113fd0 con 0x7efef810e3c0 2024-09-19T05:06:15.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.417+0000 7efefcc00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efef81142d0 con 0x7efef8074620 2024-09-19T05:06:15.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.417+0000 7efef5400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efef8074f40 0x7efef819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:15.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.417+0000 7efef5e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efef8074620 0x7efef819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:15.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.417+0000 7efef5e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efef8074620 0x7efef819a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:36666/0 (socket says 172.21.15.43:36666) 2024-09-19T05:06:15.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.417+0000 7efef5400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efef8074f40 0x7efef819ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57526/0 (socket says 172.21.15.43:57526) 2024-09-19T05:06:15.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.417+0000 7efef5e00640 1 -- 172.21.15.43:0/978837111 learned_addr learned my addr 172.21.15.43:0/978837111 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:15.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.417+0000 7efef6800640 1 --2- 172.21.15.43:0/978837111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef810e3c0 0x7efef819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:15.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.417+0000 7efef5e00640 1 -- 172.21.15.43:0/978837111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef810e3c0 msgr2=0x7efef819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:15.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.417+0000 7efef5e00640 1 --2- 172.21.15.43:0/978837111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef810e3c0 0x7efef819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:15.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.417+0000 7efef5e00640 1 -- 172.21.15.43:0/978837111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efef8074f40 msgr2=0x7efef819ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:15.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.417+0000 7efef5e00640 1 --2- 172.21.15.43:0/978837111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efef8074f40 0x7efef819ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:15.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.417+0000 7efef5e00640 1 -- 172.21.15.43:0/978837111 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef819f5c0 con 0x7efef8074620 2024-09-19T05:06:15.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.417+0000 7efef5400640 1 --2- 172.21.15.43:0/978837111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efef8074f40 0x7efef819ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:15.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.418+0000 7efef5e00640 1 --2- 172.21.15.43:0/978837111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efef8074620 0x7efef819a610 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efee800e9f0 tx=0x7efee800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:15.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.418+0000 7efeeea00640 1 -- 172.21.15.43:0/978837111 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee800cd80 con 0x7efef8074620 2024-09-19T05:06:15.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.418+0000 7efefcc00640 1 -- 172.21.15.43:0/978837111 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efef819f8b0 con 0x7efef8074620 2024-09-19T05:06:15.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.418+0000 7efeeea00640 1 -- 172.21.15.43:0/978837111 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efee8010910 con 0x7efef8074620 2024-09-19T05:06:15.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.419+0000 7efeeea00640 1 -- 172.21.15.43:0/978837111 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee80188f0 con 0x7efef8074620 2024-09-19T05:06:15.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.419+0000 7efefcc00640 1 -- 172.21.15.43:0/978837111 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efef81a7190 con 0x7efef8074620 2024-09-19T05:06:15.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.422+0000 7efeff60e740 1 Processor -- start 2024-09-19T05:06:15.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.422+0000 7efeff60e740 1 -- start start 2024-09-19T05:06:15.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.422+0000 7efeeea00640 1 -- 172.21.15.43:0/978837111 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7efee8010ac0 con 0x7efef8074620 2024-09-19T05:06:15.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.422+0000 7efeff60e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606d056db90 0x5606d056df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:15.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.422+0000 7efeff60e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606d056e4b0 0x5606d0578830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:15.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.422+0000 7efeff60e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606d0578d70 0x5606d057b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:15.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.422+0000 7efeff60e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5606d04d43e0 con 0x5606d0578d70 2024-09-19T05:06:15.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.422+0000 7efeff60e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5606d047bd80 con 0x5606d056db90 2024-09-19T05:06:15.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.422+0000 7efeff60e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5606d04d03a0 con 0x5606d056e4b0 2024-09-19T05:06:15.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.423+0000 7efef6800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606d056db90 0x5606d056df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:15.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.423+0000 7efef5400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606d0578d70 0x5606d057b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:15.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.423+0000 7efef5e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606d056e4b0 0x5606d0578830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:15.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.423+0000 7efef6800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606d056db90 0x5606d056df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:59642/0 (socket says 172.21.15.43:59642) 2024-09-19T05:06:15.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.423+0000 7efef6800640 1 -- 172.21.15.43:0/2226819936 learned_addr learned my addr 172.21.15.43:0/2226819936 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:15.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.423+0000 7efef5400640 1 -- 172.21.15.43:0/2226819936 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606d056e4b0 msgr2=0x5606d0578830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:15.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.423+0000 7efef5400640 1 --2- 172.21.15.43:0/2226819936 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606d056e4b0 0x5606d0578830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:15.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.423+0000 7efef5400640 1 -- 172.21.15.43:0/2226819936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606d056db90 msgr2=0x5606d056df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:15.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.423+0000 7efef5400640 1 --2- 172.21.15.43:0/2226819936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606d056db90 0x5606d056df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:15.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.423+0000 7efef5400640 1 -- 172.21.15.43:0/2226819936 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606d049aae0 con 0x5606d0578d70 2024-09-19T05:06:15.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.423+0000 7efef6800640 1 --2- 172.21.15.43:0/2226819936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606d056db90 0x5606d056df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:15.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.423+0000 7efef5400640 1 --2- 172.21.15.43:0/2226819936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606d0578d70 0x5606d057b160 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7efee4009f20 tx=0x7efee40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=330f88a7b56c303d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:15.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.424+0000 7efee3000640 1 -- 172.21.15.43:0/2226819936 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee4004030 con 0x5606d0578d70 2024-09-19T05:06:15.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.424+0000 7efee3000640 1 -- 172.21.15.43:0/2226819936 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efee4005500 con 0x5606d0578d70 2024-09-19T05:06:15.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.424+0000 7efee3000640 1 -- 172.21.15.43:0/2226819936 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee4004e20 con 0x5606d0578d70 2024-09-19T05:06:15.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.424+0000 7efeeea00640 1 --2- 172.21.15.43:0/978837111 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7efedc0a3430 0x7efedc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:15.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.424+0000 7efeeea00640 1 -- 172.21.15.43:0/978837111 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7efee80d00b0 con 0x7efef8074620 2024-09-19T05:06:15.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.424+0000 7efef6800640 1 --2- 172.21.15.43:0/978837111 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7efedc0a3430 0x7efedc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:15.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.425+0000 7efeff60e740 1 -- 172.21.15.43:0/2226819936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606d0578d70 msgr2=0x5606d057b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:15.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.425+0000 7efeff60e740 1 --2- 172.21.15.43:0/2226819936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606d0578d70 0x5606d057b160 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7efee4009f20 tx=0x7efee40118d0 comp rx=0 tx=0).stop 2024-09-19T05:06:15.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.425+0000 7efeff60e740 1 -- 172.21.15.43:0/2226819936 shutdown_connections 2024-09-19T05:06:15.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.425+0000 7efeff60e740 1 --2- 172.21.15.43:0/2226819936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606d0578d70 0x5606d057b160 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:15.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.425+0000 7efeff60e740 1 --2- 172.21.15.43:0/2226819936 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606d056e4b0 0x5606d0578830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:15.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.425+0000 7efeff60e740 1 --2- 172.21.15.43:0/2226819936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606d056db90 0x5606d056df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:15.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.425+0000 7efeff60e740 1 -- 172.21.15.43:0/2226819936 >> 172.21.15.43:0/2226819936 conn(0x5606d056a120 msgr2=0x5606d056a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:15.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.425+0000 7efeff60e740 1 -- 172.21.15.43:0/2226819936 shutdown_connections 2024-09-19T05:06:15.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.425+0000 7efeff60e740 1 -- 172.21.15.43:0/2226819936 wait complete. 2024-09-19T05:06:15.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.425+0000 7efef6800640 1 --2- 172.21.15.43:0/978837111 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7efedc0a3430 0x7efedc0a58f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efef0030040 tx=0x7efef0005c50 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:15.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.426+0000 7efeff60e740 1 -- start start 2024-09-19T05:06:15.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.426+0000 7efeff60e740 1 Processor -- start 2024-09-19T05:06:15.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.426+0000 7efeff60e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606d056db90 0x5606d0575c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:15.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.426+0000 7efeff60e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606d056e4b0 0x5606d05761c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:15.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.426+0000 7efeff60e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606d0578d70 0x5606d0576700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:15.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.426+0000 7efeff60e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5606d057b9e0 con 0x5606d056db90 2024-09-19T05:06:15.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.426+0000 7efeff60e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5606d048f0f0 con 0x5606d056e4b0 2024-09-19T05:06:15.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.426+0000 7efeff60e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5606d057bb60 con 0x5606d0578d70 2024-09-19T05:06:15.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.426+0000 7efeff60e740 10 client.0 _ll_register_callbacks cb 0x5606d04809e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:15.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.427+0000 7efef5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606d056db90 0x5606d0575c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:15.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.427+0000 7efef5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606d056db90 0x5606d0575c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57542/0 (socket says 172.21.15.43:57542) 2024-09-19T05:06:15.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.427+0000 7efef5400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606d056e4b0 0x5606d05761c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:15.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.427+0000 7efef5e00640 1 -- 172.21.15.43:0/1407783015 learned_addr learned my addr 172.21.15.43:0/1407783015 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:15.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.427+0000 7efef6800640 1 --2- 172.21.15.43:0/1407783015 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606d0578d70 0x5606d0576700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:15.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.427+0000 7efef5e00640 1 -- 172.21.15.43:0/1407783015 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606d0578d70 msgr2=0x5606d0576700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:15.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.427+0000 7efef5e00640 1 --2- 172.21.15.43:0/1407783015 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606d0578d70 0x5606d0576700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:15.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.427+0000 7efef5e00640 1 -- 172.21.15.43:0/1407783015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606d056e4b0 msgr2=0x5606d05761c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:15.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.427+0000 7efef5e00640 1 --2- 172.21.15.43:0/1407783015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606d056e4b0 0x5606d05761c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:15.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.427+0000 7efef5e00640 1 -- 172.21.15.43:0/1407783015 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606d02453c0 con 0x5606d056db90 2024-09-19T05:06:15.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.427+0000 7efef5e00640 1 --2- 172.21.15.43:0/1407783015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606d056db90 0x5606d0575c80 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efee80cd810 tx=0x7efee8098780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce9635c8952e3cc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:15.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.427+0000 7efed7400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:06:15.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.428+0000 7efed7400640 1 -- 172.21.15.43:0/1407783015 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee80cdc30 con 0x5606d056db90 2024-09-19T05:06:15.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.428+0000 7efed7400640 1 -- 172.21.15.43:0/1407783015 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7efee80d20d0 con 0x5606d056db90 2024-09-19T05:06:15.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.428+0000 7efeff60e740 1 -- 172.21.15.43:0/1407783015 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efee4005500 con 0x5606d056db90 2024-09-19T05:06:15.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.428+0000 7efed7400640 1 -- 172.21.15.43:0/1407783015 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efee809c3f0 con 0x5606d056db90 2024-09-19T05:06:15.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.428+0000 7efeff60e740 10 client.15495 fetch_fsmap learned FSMap version 29 2024-09-19T05:06:15.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.428+0000 7efeff60e740 1 -- 172.21.15.43:0/1407783015 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5606d0587690 con 0x5606d056db90 2024-09-19T05:06:15.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.428+0000 7efed7400640 1 -- 172.21.15.43:0/1407783015 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efee4005500 con 0x5606d056db90 2024-09-19T05:06:15.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.428+0000 7efed7400640 1 -- 172.21.15.43:0/1407783015 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7efee809b960 con 0x5606d056db90 2024-09-19T05:06:15.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.429+0000 7efeff60e740 10 client.15495 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:06:15.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.429+0000 7efeff60e740 10 client.15495 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:06:15.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.429+0000 7efeff60e740 20 client.15495 populate_metadata read hostname 'smithi043' 2024-09-19T05:06:15.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.429+0000 7efeff60e740 1 --2- 172.21.15.43:0/1407783015 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5606d0588aa0 0x5606d058ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:15.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.429+0000 7efeff60e740 4 client.15495 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:06:15.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.429+0000 7efeff60e740 1 -- 172.21.15.43:0/1407783015 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606d04e1950 con 0x5606d0588aa0 2024-09-19T05:06:15.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.429+0000 7efef5400640 1 --2- 172.21.15.43:0/1407783015 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5606d0588aa0 0x5606d058ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:15.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.430+0000 7efef5400640 1 --2- 172.21.15.43:0/1407783015 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5606d0588aa0 0x5606d058ae90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9a6cfe6c69ebd85 server_cookie=c3d01d421924946c in_seq=0 out_seq=0 2024-09-19T05:06:15.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.430+0000 7efed7400640 10 client.15495 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:06:15.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.430+0000 7efed7400640 1 -- 172.21.15.43:0/1407783015 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5606d04e1950 con 0x5606d0588aa0 2024-09-19T05:06:15.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.430+0000 7efed7400640 10 client.15495 handle_command_reply: tid=0 2024-09-19T05:06:15.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.431+0000 7efeff60e740 1 client.15495 shutdown 2024-09-19T05:06:15.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.431+0000 7efeff60e740 1 -- 172.21.15.43:0/1407783015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606d056db90 msgr2=0x5606d0575c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:15.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.431+0000 7efeff60e740 1 --2- 172.21.15.43:0/1407783015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606d056db90 0x5606d0575c80 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efee80cd810 tx=0x7efee8098780 comp rx=0 tx=0).stop 2024-09-19T05:06:15.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.431+0000 7efeff60e740 1 -- 172.21.15.43:0/1407783015 shutdown_connections 2024-09-19T05:06:15.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.431+0000 7efeff60e740 1 --2- 172.21.15.43:0/1407783015 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5606d0588aa0 0x5606d058ae90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:15.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.431+0000 7efeff60e740 1 --2- 172.21.15.43:0/1407783015 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606d0578d70 0x5606d0576700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:15.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.431+0000 7efeff60e740 1 --2- 172.21.15.43:0/1407783015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606d056e4b0 0x5606d05761c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:15.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.431+0000 7efeff60e740 1 --2- 172.21.15.43:0/1407783015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606d056db90 0x5606d0575c80 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:15.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.431+0000 7efeff60e740 1 -- 172.21.15.43:0/1407783015 >> 172.21.15.43:0/1407783015 conn(0x5606d056a120 msgr2=0x5606d056fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:15.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.431+0000 7efed7400640 0 client.15495 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:06:15.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.431+0000 7efeff60e740 1 -- 172.21.15.43:0/1407783015 shutdown_connections 2024-09-19T05:06:15.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.431+0000 7efeff60e740 1 -- 172.21.15.43:0/1407783015 wait complete. 2024-09-19T05:06:15.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.431+0000 7efeff60e740 20 client.15495 trim_cache size 0 max 16384 2024-09-19T05:06:15.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.449+0000 7efeff60e740 1 Processor -- start 2024-09-19T05:06:15.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.450+0000 7efeff60e740 1 -- start start 2024-09-19T05:06:15.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.450+0000 7efeff60e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606d056db90 0x5606d05705f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:15.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.450+0000 7efeff60e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606d056e4b0 0x5606d0577bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:15.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.450+0000 7efeff60e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606d0578d70 0x5606d0590730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:15.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.450+0000 7efeff60e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efef81afb60 con 0x5606d0578d70 2024-09-19T05:06:15.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.450+0000 7efeff60e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efef81b16a0 con 0x5606d056db90 2024-09-19T05:06:15.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.450+0000 7efeff60e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efef81b0230 con 0x5606d056e4b0 2024-09-19T05:06:15.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.450+0000 7efef6800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606d0578d70 0x5606d0590730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:15.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.450+0000 7efef6800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606d0578d70 0x5606d0590730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57544/0 (socket says 172.21.15.43:57544) 2024-09-19T05:06:15.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.450+0000 7efef6800640 1 -- 172.21.15.43:0/3394572413 learned_addr learned my addr 172.21.15.43:0/3394572413 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:15.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.450+0000 7efef5e00640 1 --2- 172.21.15.43:0/3394572413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606d056db90 0x5606d05705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:15.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.450+0000 7efef5400640 1 --2- 172.21.15.43:0/3394572413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606d056e4b0 0x5606d0577bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:15.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.451+0000 7efef6800640 1 -- 172.21.15.43:0/3394572413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606d056e4b0 msgr2=0x5606d0577bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:15.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.451+0000 7efef6800640 1 --2- 172.21.15.43:0/3394572413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606d056e4b0 0x5606d0577bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:15.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.451+0000 7efef6800640 1 -- 172.21.15.43:0/3394572413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606d056db90 msgr2=0x5606d05705f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:15.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.451+0000 7efef6800640 1 --2- 172.21.15.43:0/3394572413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606d056db90 0x5606d05705f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:15.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.451+0000 7efef6800640 1 -- 172.21.15.43:0/3394572413 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606d0574c30 con 0x5606d0578d70 2024-09-19T05:06:15.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.451+0000 7efef5400640 1 --2- 172.21.15.43:0/3394572413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606d056e4b0 0x5606d0577bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:06:15.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.451+0000 7efef6800640 1 --2- 172.21.15.43:0/3394572413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606d0578d70 0x5606d0590730 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7efef0009ca0 tx=0x7efef0042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=297c4a2e17e3a926 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:15.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.451+0000 7efee2600640 1 -- 172.21.15.43:0/3394572413 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef0034040 con 0x5606d0578d70 2024-09-19T05:06:15.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.452+0000 7efee2600640 1 -- 172.21.15.43:0/3394572413 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efef0003c30 con 0x5606d0578d70 2024-09-19T05:06:15.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.452+0000 7efee2600640 1 -- 172.21.15.43:0/3394572413 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef0030860 con 0x5606d0578d70 2024-09-19T05:06:15.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.452+0000 7efeff60e740 1 -- 172.21.15.43:0/3394572413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606d0578d70 msgr2=0x5606d0590730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:15.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.452+0000 7efeff60e740 1 --2- 172.21.15.43:0/3394572413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606d0578d70 0x5606d0590730 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7efef0009ca0 tx=0x7efef0042000 comp rx=0 tx=0).stop 2024-09-19T05:06:15.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.452+0000 7efeff60e740 1 -- 172.21.15.43:0/3394572413 shutdown_connections 2024-09-19T05:06:15.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.452+0000 7efeff60e740 1 --2- 172.21.15.43:0/3394572413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606d0578d70 0x5606d0590730 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:15.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.452+0000 7efeff60e740 1 --2- 172.21.15.43:0/3394572413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606d056e4b0 0x5606d0577bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:15.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.452+0000 7efeff60e740 1 --2- 172.21.15.43:0/3394572413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606d056db90 0x5606d05705f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:15.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.452+0000 7efeff60e740 1 -- 172.21.15.43:0/3394572413 >> 172.21.15.43:0/3394572413 conn(0x5606d056a120 msgr2=0x5606d05774e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:15.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.452+0000 7efeff60e740 1 -- 172.21.15.43:0/3394572413 shutdown_connections 2024-09-19T05:06:15.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.452+0000 7efeff60e740 1 -- 172.21.15.43:0/3394572413 wait complete. 2024-09-19T05:06:15.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.453+0000 7efeff60e740 1 -- start start 2024-09-19T05:06:15.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.453+0000 7efeff60e740 1 Processor -- start 2024-09-19T05:06:15.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.453+0000 7efeff60e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606d056db90 0x5606d057ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:15.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.454+0000 7efeff60e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606d056e4b0 0x5606d057d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:15.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.454+0000 7efeff60e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606d0578d70 0x5606d05f2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:15.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.454+0000 7efeff60e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efef81b0fd0 con 0x5606d056e4b0 2024-09-19T05:06:15.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.454+0000 7efeff60e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efef81b0900 con 0x5606d0578d70 2024-09-19T05:06:15.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.454+0000 7efeff60e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efef81adb40 con 0x5606d056db90 2024-09-19T05:06:15.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.454+0000 7efeff60e740 10 client.0 _ll_register_callbacks cb 0x7efee8009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:15.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.454+0000 7efef5400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606d056e4b0 0x5606d057d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:15.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.454+0000 7efef5400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606d056e4b0 0x5606d057d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57554/0 (socket says 172.21.15.43:57554) 2024-09-19T05:06:15.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.454+0000 7efef5e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606d056db90 0x5606d057ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:15.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.454+0000 7efef5400640 1 -- 172.21.15.43:0/1754097101 learned_addr learned my addr 172.21.15.43:0/1754097101 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:15.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.454+0000 7efef6800640 1 --2- 172.21.15.43:0/1754097101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606d0578d70 0x5606d05f2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:15.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.454+0000 7efef5e00640 1 -- 172.21.15.43:0/1754097101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606d0578d70 msgr2=0x5606d05f2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:15.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.454+0000 7efef5e00640 1 --2- 172.21.15.43:0/1754097101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606d0578d70 0x5606d05f2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:15.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.454+0000 7efef5e00640 1 -- 172.21.15.43:0/1754097101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606d056e4b0 msgr2=0x5606d057d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:15.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.454+0000 7efef5e00640 1 --2- 172.21.15.43:0/1754097101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606d056e4b0 0x5606d057d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:15.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.454+0000 7efef5e00640 1 -- 172.21.15.43:0/1754097101 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606d057dca0 con 0x5606d056db90 2024-09-19T05:06:15.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.454+0000 7efef5400640 1 --2- 172.21.15.43:0/1754097101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606d056e4b0 0x5606d057d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:15.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.455+0000 7efef5e00640 1 --2- 172.21.15.43:0/1754097101 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606d056db90 0x5606d057ce80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efee800cfd0 tx=0x7efee8009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8274fdc593db553 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:15.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.455+0000 7efed7400640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:06:15.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.455+0000 7efed7400640 1 -- 172.21.15.43:0/1754097101 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee809bf30 con 0x5606d056db90 2024-09-19T05:06:15.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.455+0000 7efed7400640 1 -- 172.21.15.43:0/1754097101 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7efee809b840 con 0x5606d056db90 2024-09-19T05:06:15.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.455+0000 7efeff60e740 1 -- 172.21.15.43:0/1754097101 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efef0003c30 con 0x5606d056db90 2024-09-19T05:06:15.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.455+0000 7efed7400640 1 -- 172.21.15.43:0/1754097101 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efee80a4fb0 con 0x5606d056db90 2024-09-19T05:06:15.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.455+0000 7efeff60e740 10 client.25450 fetch_fsmap learned FSMap version 29 2024-09-19T05:06:15.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.455+0000 7efeff60e740 1 -- 172.21.15.43:0/1754097101 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5606d05f3d60 con 0x5606d056db90 2024-09-19T05:06:15.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.455+0000 7efed7400640 1 -- 172.21.15.43:0/1754097101 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efef0003c30 con 0x5606d056db90 2024-09-19T05:06:15.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.456+0000 7efed7400640 1 -- 172.21.15.43:0/1754097101 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7efee80d3170 con 0x5606d056db90 2024-09-19T05:06:15.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.456+0000 7efeff60e740 10 client.25450 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:06:15.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.456+0000 7efeff60e740 10 client.25450 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:06:15.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.456+0000 7efeff60e740 20 client.25450 populate_metadata read hostname 'smithi043' 2024-09-19T05:06:15.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.456+0000 7efeff60e740 1 --2- 172.21.15.43:0/1754097101 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5606d05f4480 0x5606d05f6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:15.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.456+0000 7efeff60e740 4 client.25450 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:06:15.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.456+0000 7efeff60e740 1 -- 172.21.15.43:0/1754097101 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5606d056acd0 con 0x5606d05f4480 2024-09-19T05:06:15.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.456+0000 7efef5400640 1 --2- 172.21.15.43:0/1754097101 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5606d05f4480 0x5606d05f6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:15.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.457+0000 7efef5400640 1 --2- 172.21.15.43:0/1754097101 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5606d05f4480 0x5606d05f6870 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6aaf2145f4a825e9 server_cookie=30ca971945e26f38 in_seq=0 out_seq=0 2024-09-19T05:06:15.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.457+0000 7efed7400640 10 client.25450 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:06:15.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.457+0000 7efed7400640 1 -- 172.21.15.43:0/1754097101 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5606d056acd0 con 0x5606d05f4480 2024-09-19T05:06:15.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.457+0000 7efed7400640 10 client.25450 handle_command_reply: tid=0 2024-09-19T05:06:15.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.457+0000 7efeff60e740 1 client.25450 shutdown 2024-09-19T05:06:15.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.457+0000 7efeff60e740 1 -- 172.21.15.43:0/1754097101 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606d056db90 msgr2=0x5606d057ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:15.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.457+0000 7efeff60e740 1 --2- 172.21.15.43:0/1754097101 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606d056db90 0x5606d057ce80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efee800cfd0 tx=0x7efee8009310 comp rx=0 tx=0).stop 2024-09-19T05:06:15.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.457+0000 7efeff60e740 1 -- 172.21.15.43:0/1754097101 shutdown_connections 2024-09-19T05:06:15.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.457+0000 7efeff60e740 1 --2- 172.21.15.43:0/1754097101 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5606d05f4480 0x5606d05f6870 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:15.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.457+0000 7efeff60e740 1 --2- 172.21.15.43:0/1754097101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606d0578d70 0x5606d05f2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:15.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.457+0000 7efeff60e740 1 --2- 172.21.15.43:0/1754097101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5606d056e4b0 0x5606d057d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:15.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.457+0000 7efeff60e740 1 --2- 172.21.15.43:0/1754097101 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606d056db90 0x5606d057ce80 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:15.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.457+0000 7efeff60e740 1 -- 172.21.15.43:0/1754097101 >> 172.21.15.43:0/1754097101 conn(0x5606d056a120 msgr2=0x5606d0576460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:15.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.457+0000 7efed7400640 0 client.25450 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:06:15.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.457+0000 7efeff60e740 1 -- 172.21.15.43:0/1754097101 shutdown_connections 2024-09-19T05:06:15.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.457+0000 7efeff60e740 1 -- 172.21.15.43:0/1754097101 wait complete. 2024-09-19T05:06:15.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.457+0000 7efeff60e740 20 client.25450 trim_cache size 0 max 16384 2024-09-19T05:06:15.463 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:06:15.463 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-09-19T05:06:15.463 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-09-19T05:06:15.463 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:06:15.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.458+0000 7efefcc00640 1 -- 172.21.15.43:0/978837111 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7efedc0a3430 msgr2=0x7efedc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:15.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.458+0000 7efefcc00640 1 --2- 172.21.15.43:0/978837111 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7efedc0a3430 0x7efedc0a58f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efef0030040 tx=0x7efef0005c50 comp rx=0 tx=0).stop 2024-09-19T05:06:15.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.458+0000 7efefcc00640 1 -- 172.21.15.43:0/978837111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efef8074620 msgr2=0x7efef819a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:15.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.458+0000 7efefcc00640 1 --2- 172.21.15.43:0/978837111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efef8074620 0x7efef819a610 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efee800e9f0 tx=0x7efee800eeb0 comp rx=0 tx=0).stop 2024-09-19T05:06:15.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.458+0000 7efefcc00640 1 -- 172.21.15.43:0/978837111 shutdown_connections 2024-09-19T05:06:15.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.458+0000 7efefcc00640 1 --2- 172.21.15.43:0/978837111 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7efedc0a3430 0x7efedc0a58f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:15.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.458+0000 7efefcc00640 1 --2- 172.21.15.43:0/978837111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef810e3c0 0x7efef819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:15.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.458+0000 7efefcc00640 1 --2- 172.21.15.43:0/978837111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efef8074f40 0x7efef819ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:15.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.458+0000 7efefcc00640 1 --2- 172.21.15.43:0/978837111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efef8074620 0x7efef819a610 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:15.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.458+0000 7efefcc00640 1 -- 172.21.15.43:0/978837111 >> 172.21.15.43:0/978837111 conn(0x7efef80fd8c0 msgr2=0x7efef80fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:15.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.458+0000 7efefcc00640 1 -- 172.21.15.43:0/978837111 shutdown_connections 2024-09-19T05:06:15.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:15.459+0000 7efefcc00640 1 -- 172.21.15.43:0/978837111 wait complete. 2024-09-19T05:06:15.843 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:15 smithi043 ceph-mon[30800]: pgmap v51: 97 pgs: 97 active+clean; 1.0 GiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 159 MiB/s wr, 70 op/s 2024-09-19T05:06:15.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:15 smithi151 ceph-mon[35425]: pgmap v51: 97 pgs: 97 active+clean; 1.0 GiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 159 MiB/s wr, 70 op/s 2024-09-19T05:06:15.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:15 smithi096 ceph-mon[35085]: pgmap v51: 97 pgs: 97 active+clean; 1.0 GiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 159 MiB/s wr, 70 op/s 2024-09-19T05:06:16.256 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T05:06:16.257 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:06:16.639 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:06:17.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.261+0000 7f0ea2e00640 1 -- 172.21.15.43:0/1620164695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e9c074620 msgr2=0x7f0e9c074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:17.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.261+0000 7f0ea2e00640 1 --2- 172.21.15.43:0/1620164695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e9c074620 0x7f0e9c074a00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c009900 tx=0x7f0e8c02a480 comp rx=0 tx=0).stop 2024-09-19T05:06:17.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.261+0000 7f0ea2e00640 1 -- 172.21.15.43:0/1620164695 shutdown_connections 2024-09-19T05:06:17.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.261+0000 7f0ea2e00640 1 --2- 172.21.15.43:0/1620164695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e9c10e3b0 0x7f0e9c110800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:17.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.261+0000 7f0ea2e00640 1 --2- 172.21.15.43:0/1620164695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e9c074f40 0x7f0e9c10de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:17.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.261+0000 7f0ea2e00640 1 --2- 172.21.15.43:0/1620164695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e9c074620 0x7f0e9c074a00 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:17.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.261+0000 7f0ea2e00640 1 -- 172.21.15.43:0/1620164695 >> 172.21.15.43:0/1620164695 conn(0x7f0e9c0fd890 msgr2=0x7f0e9c0ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:17.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.261+0000 7f0ea2e00640 1 -- 172.21.15.43:0/1620164695 shutdown_connections 2024-09-19T05:06:17.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.261+0000 7f0ea2e00640 1 -- 172.21.15.43:0/1620164695 wait complete. 2024-09-19T05:06:17.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.263+0000 7f0ea2e00640 1 Processor -- start 2024-09-19T05:06:17.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.263+0000 7f0ea2e00640 1 -- start start 2024-09-19T05:06:17.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.263+0000 7f0ea2e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e9c074620 0x7f0e9c19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:17.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.263+0000 7f0ea2e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e9c074f40 0x7f0e9c19ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:17.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.263+0000 7f0ea2e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e9c10e3b0 0x7f0e9c19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:17.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.263+0000 7f0ea2e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9c1141b0 con 0x7f0e9c10e3b0 2024-09-19T05:06:17.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.263+0000 7f0ea2e00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9c114030 con 0x7f0e9c074f40 2024-09-19T05:06:17.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.263+0000 7f0ea2e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9c114330 con 0x7f0e9c074620 2024-09-19T05:06:17.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.264+0000 7f0ea0e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e9c10e3b0 0x7f0e9c19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:17.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.264+0000 7f0ea0e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e9c10e3b0 0x7f0e9c19efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41166/0 (socket says 172.21.15.43:41166) 2024-09-19T05:06:17.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.264+0000 7f0ea0e00640 1 -- 172.21.15.43:0/1638624935 learned_addr learned my addr 172.21.15.43:0/1638624935 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:17.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.264+0000 7f0e9be00640 1 --2- 172.21.15.43:0/1638624935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e9c074620 0x7f0e9c19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:17.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.264+0000 7f0e9b400640 1 --2- 172.21.15.43:0/1638624935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e9c074f40 0x7f0e9c19ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:17.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.264+0000 7f0ea0e00640 1 -- 172.21.15.43:0/1638624935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e9c074620 msgr2=0x7f0e9c19a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:17.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.264+0000 7f0ea0e00640 1 --2- 172.21.15.43:0/1638624935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e9c074620 0x7f0e9c19a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:17.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.264+0000 7f0ea0e00640 1 -- 172.21.15.43:0/1638624935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e9c074f40 msgr2=0x7f0e9c19ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:17.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.264+0000 7f0ea0e00640 1 --2- 172.21.15.43:0/1638624935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e9c074f40 0x7f0e9c19ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:17.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.264+0000 7f0ea0e00640 1 -- 172.21.15.43:0/1638624935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e9c19f6a0 con 0x7f0e9c10e3b0 2024-09-19T05:06:17.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.264+0000 7f0e9b400640 1 --2- 172.21.15.43:0/1638624935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e9c074f40 0x7f0e9c19ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:17.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.265+0000 7f0ea0e00640 1 --2- 172.21.15.43:0/1638624935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e9c10e3b0 0x7f0e9c19efc0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0e9400eb10 tx=0x7f0e9400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:17.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.265+0000 7f0e98c00640 1 -- 172.21.15.43:0/1638624935 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e94004b20 con 0x7f0e9c10e3b0 2024-09-19T05:06:17.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.265+0000 7f0e98c00640 1 -- 172.21.15.43:0/1638624935 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e94004cc0 con 0x7f0e9c10e3b0 2024-09-19T05:06:17.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.265+0000 7f0e98c00640 1 -- 172.21.15.43:0/1638624935 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e94010760 con 0x7f0e9c10e3b0 2024-09-19T05:06:17.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.265+0000 7f0ea2e00640 1 -- 172.21.15.43:0/1638624935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e9c19f990 con 0x7f0e9c10e3b0 2024-09-19T05:06:17.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.266+0000 7f0ea2e00640 1 -- 172.21.15.43:0/1638624935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e9c1a7270 con 0x7f0e9c10e3b0 2024-09-19T05:06:17.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.267+0000 7f0ea2e00640 1 -- 172.21.15.43:0/1638624935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e9c074a00 con 0x7f0e9c10e3b0 2024-09-19T05:06:17.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.269+0000 7f0e98c00640 1 -- 172.21.15.43:0/1638624935 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0e94005020 con 0x7f0e9c10e3b0 2024-09-19T05:06:17.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.271+0000 7f0e98c00640 1 --2- 172.21.15.43:0/1638624935 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f0e800a33b0 0x7f0e800a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:17.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.271+0000 7f0e98c00640 1 -- 172.21.15.43:0/1638624935 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f0e940d0260 con 0x7f0e9c10e3b0 2024-09-19T05:06:17.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.272+0000 7f0e9be00640 1 --2- 172.21.15.43:0/1638624935 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f0e800a33b0 0x7f0e800a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:17.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.272+0000 7f0e9be00640 1 --2- 172.21.15.43:0/1638624935 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f0e800a33b0 0x7f0e800a5870 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c000c00 tx=0x7f0e8c005b50 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:17.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.279+0000 7f0e98c00640 1 -- 172.21.15.43:0/1638624935 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0e9409cbf0 con 0x7f0e9c10e3b0 2024-09-19T05:06:17.399 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-19T05:06:17.399 INFO:tasks.workunit.client.0.smithi043.stdout: Children see throughput for 1 readers = 3547977.00 kB/sec 2024-09-19T05:06:17.399 INFO:tasks.workunit.client.0.smithi043.stdout: Parent sees throughput for 1 readers = 3541277.58 kB/sec 2024-09-19T05:06:17.399 INFO:tasks.workunit.client.0.smithi043.stdout: Min throughput per process = 3547977.00 kB/sec 2024-09-19T05:06:17.399 INFO:tasks.workunit.client.0.smithi043.stdout: Max throughput per process = 3547977.00 kB/sec 2024-09-19T05:06:17.400 INFO:tasks.workunit.client.0.smithi043.stdout: Avg throughput per process = 3547977.00 kB/sec 2024-09-19T05:06:17.400 INFO:tasks.workunit.client.0.smithi043.stdout: Min xfer = 1048576.00 kB 2024-09-19T05:06:17.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.507+0000 7f0ea2e00640 1 -- 172.21.15.43:0/1638624935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e9c1a7550 con 0x7f0e9c10e3b0 2024-09-19T05:06:17.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.508+0000 7f0e98c00640 1 -- 172.21.15.43:0/1638624935 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11965 (secure 0 0 0) 0x7f0e9409c4f0 con 0x7f0e9c10e3b0 2024-09-19T05:06:17.509 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:06:17.509 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":29,"btime":"2024-09-19T05:04:38:785407+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/773801169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":773801169},{"type":"v1","addr":"172.21.15.43:6835","nonce":773801169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1799137902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1799137902},{"type":"v1","addr":"172.21.15.43:6837","nonce":1799137902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:02:25.463041+0000","modified":"2024-09-19T05:04:38.039809+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24518,"mds_2":24511,"mds_3":24506,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14541":{"gid":14541,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/794881135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":794881135},{"type":"v1","addr":"172.21.15.43:6841","nonce":794881135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/2836453158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2836453158},{"type":"v1","addr":"172.21.15.151:6833","nonce":2836453158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/178785812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":178785812},{"type":"v1","addr":"172.21.15.96:6833","nonce":178785812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1671043820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1671043820},{"type":"v1","addr":"172.21.15.96:6835","nonce":1671043820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2972514328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2972514328},{"type":"v1","addr":"172.21.15.151:6835","nonce":2972514328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.151:6837/3960378010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3960378010},{"type":"v1","addr":"172.21.15.151:6837","nonce":3960378010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6837/3978388286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3978388286},{"type":"v1","addr":"172.21.15.96:6837","nonce":3978388286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.151:6839/3794104313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3794104313},{"type":"v1","addr":"172.21.15.151:6839","nonce":3794104313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.43:6839/3404555030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3404555030},{"type":"v1","addr":"172.21.15.43:6839","nonce":3404555030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/605527621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":605527621},{"type":"v1","addr":"172.21.15.96:6839","nonce":605527621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24518,24511,24506,24493]},"id":1}]} 2024-09-19T05:06:17.510 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 29 2024-09-19T05:06:17.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.510+0000 7f0ea2e00640 1 -- 172.21.15.43:0/1638624935 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f0e800a33b0 msgr2=0x7f0e800a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:17.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.510+0000 7f0ea2e00640 1 --2- 172.21.15.43:0/1638624935 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f0e800a33b0 0x7f0e800a5870 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c000c00 tx=0x7f0e8c005b50 comp rx=0 tx=0).stop 2024-09-19T05:06:17.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.510+0000 7f0ea2e00640 1 -- 172.21.15.43:0/1638624935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e9c10e3b0 msgr2=0x7f0e9c19efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:17.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.510+0000 7f0ea2e00640 1 --2- 172.21.15.43:0/1638624935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e9c10e3b0 0x7f0e9c19efc0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0e9400eb10 tx=0x7f0e9400c2d0 comp rx=0 tx=0).stop 2024-09-19T05:06:17.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.510+0000 7f0ea2e00640 1 -- 172.21.15.43:0/1638624935 shutdown_connections 2024-09-19T05:06:17.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.510+0000 7f0ea2e00640 1 --2- 172.21.15.43:0/1638624935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e9c10e3b0 0x7f0e9c19efc0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:17.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.510+0000 7f0ea2e00640 1 --2- 172.21.15.43:0/1638624935 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f0e800a33b0 0x7f0e800a5870 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:17.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.510+0000 7f0ea2e00640 1 --2- 172.21.15.43:0/1638624935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e9c074f40 0x7f0e9c19ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:17.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.510+0000 7f0ea2e00640 1 --2- 172.21.15.43:0/1638624935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e9c074620 0x7f0e9c19a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:17.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.510+0000 7f0ea2e00640 1 -- 172.21.15.43:0/1638624935 >> 172.21.15.43:0/1638624935 conn(0x7f0e9c0fd890 msgr2=0x7f0e9c0fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:17.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.511+0000 7f0ea2e00640 1 -- 172.21.15.43:0/1638624935 shutdown_connections 2024-09-19T05:06:17.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:17.511+0000 7f0ea2e00640 1 -- 172.21.15.43:0/1638624935 wait complete. 2024-09-19T05:06:17.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:17 smithi043 ceph-mon[30800]: pgmap v52: 97 pgs: 97 active+clean; 1.0 GiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 160 MiB/s wr, 50 op/s 2024-09-19T05:06:17.782 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:17 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/1638624935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:06:18.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:17 smithi151 ceph-mon[35425]: pgmap v52: 97 pgs: 97 active+clean; 1.0 GiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 160 MiB/s wr, 50 op/s 2024-09-19T05:06:18.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:17 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/1638624935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:06:18.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:17 smithi096 ceph-mon[35085]: pgmap v52: 97 pgs: 97 active+clean; 1.0 GiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 160 MiB/s wr, 50 op/s 2024-09-19T05:06:18.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:17 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/1638624935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:06:18.492 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T05:06:18.876 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:06:19.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.458+0000 7f86e2a00640 1 -- 172.21.15.43:0/3982602631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86dc103ff0 msgr2=0x7f86dc1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:19.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.458+0000 7f86e2a00640 1 --2- 172.21.15.43:0/3982602631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86dc103ff0 0x7f86dc1043d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f86cc009900 tx=0x7f86cc02a3f0 comp rx=0 tx=0).stop 2024-09-19T05:06:19.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.458+0000 7f86e2a00640 1 -- 172.21.15.43:0/3982602631 shutdown_connections 2024-09-19T05:06:19.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.458+0000 7f86e2a00640 1 --2- 172.21.15.43:0/3982602631 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86dc109120 0x7f86dc110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:19.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.458+0000 7f86e2a00640 1 --2- 172.21.15.43:0/3982602631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86dc1049a0 0x7f86dc1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:19.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.458+0000 7f86e2a00640 1 --2- 172.21.15.43:0/3982602631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86dc103ff0 0x7f86dc1043d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:19.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.458+0000 7f86e2a00640 1 -- 172.21.15.43:0/3982602631 >> 172.21.15.43:0/3982602631 conn(0x7f86dc0ffa90 msgr2=0x7f86dc101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:19.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.458+0000 7f86e2a00640 1 -- 172.21.15.43:0/3982602631 shutdown_connections 2024-09-19T05:06:19.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.458+0000 7f86e2a00640 1 -- 172.21.15.43:0/3982602631 wait complete. 2024-09-19T05:06:19.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.459+0000 7f86e2a00640 1 Processor -- start 2024-09-19T05:06:19.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.459+0000 7f86e2a00640 1 -- start start 2024-09-19T05:06:19.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.459+0000 7f86e2a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86dc103ff0 0x7f86dc076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:19.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.459+0000 7f86e2a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86dc1049a0 0x7f86dc076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:19.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.459+0000 7f86e2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86dc109120 0x7f86dc077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:19.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.459+0000 7f86e2a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f86dc114170 con 0x7f86dc109120 2024-09-19T05:06:19.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.459+0000 7f86e2a00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f86dc113ff0 con 0x7f86dc103ff0 2024-09-19T05:06:19.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.459+0000 7f86e2a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f86dc1142f0 con 0x7f86dc1049a0 2024-09-19T05:06:19.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.460+0000 7f86e0a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86dc109120 0x7f86dc077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:19.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.460+0000 7f86e0a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86dc109120 0x7f86dc077180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41180/0 (socket says 172.21.15.43:41180) 2024-09-19T05:06:19.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.460+0000 7f86e0a00640 1 -- 172.21.15.43:0/3994069374 learned_addr learned my addr 172.21.15.43:0/3994069374 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:19.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.460+0000 7f86dbe00640 1 --2- 172.21.15.43:0/3994069374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86dc103ff0 0x7f86dc076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:19.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.460+0000 7f86d3400640 1 --2- 172.21.15.43:0/3994069374 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86dc1049a0 0x7f86dc076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:19.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.460+0000 7f86e0a00640 1 -- 172.21.15.43:0/3994069374 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86dc1049a0 msgr2=0x7f86dc076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:19.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.460+0000 7f86e0a00640 1 --2- 172.21.15.43:0/3994069374 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86dc1049a0 0x7f86dc076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:19.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.460+0000 7f86e0a00640 1 -- 172.21.15.43:0/3994069374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86dc103ff0 msgr2=0x7f86dc076700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:19.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.460+0000 7f86e0a00640 1 --2- 172.21.15.43:0/3994069374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86dc103ff0 0x7f86dc076700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:19.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.460+0000 7f86e0a00640 1 -- 172.21.15.43:0/3994069374 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86dc1aad30 con 0x7f86dc109120 2024-09-19T05:06:19.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.460+0000 7f86e0a00640 1 --2- 172.21.15.43:0/3994069374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86dc109120 0x7f86dc077180 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f86d400db20 tx=0x7f86d400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:19.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.461+0000 7f86d9600640 1 -- 172.21.15.43:0/3994069374 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d4017070 con 0x7f86dc109120 2024-09-19T05:06:19.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.461+0000 7f86d9600640 1 -- 172.21.15.43:0/3994069374 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86d4003e50 con 0x7f86dc109120 2024-09-19T05:06:19.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.461+0000 7f86d9600640 1 -- 172.21.15.43:0/3994069374 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d4004820 con 0x7f86dc109120 2024-09-19T05:06:19.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.461+0000 7f86e2a00640 1 -- 172.21.15.43:0/3994069374 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86dc1aafc0 con 0x7f86dc109120 2024-09-19T05:06:19.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.461+0000 7f86e2a00640 1 -- 172.21.15.43:0/3994069374 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86dc1ab480 con 0x7f86dc109120 2024-09-19T05:06:19.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.464+0000 7f86e5417740 1 Processor -- start 2024-09-19T05:06:19.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.464+0000 7f86d9600640 1 -- 172.21.15.43:0/3994069374 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f86d40040b0 con 0x7f86dc109120 2024-09-19T05:06:19.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.464+0000 7f86e5417740 1 -- start start 2024-09-19T05:06:19.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.464+0000 7f86e5417740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ac0226c70 0x562ac0227050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:19.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.464+0000 7f86e5417740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac0227590 0x562ac0231910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:19.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.464+0000 7f86e5417740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ac0231e50 0x562ac0234240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:19.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.464+0000 7f86e5417740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562ac018d3e0 con 0x562ac0231e50 2024-09-19T05:06:19.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.464+0000 7f86e5417740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562ac0134d80 con 0x562ac0227590 2024-09-19T05:06:19.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.464+0000 7f86e5417740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562ac01893a0 con 0x562ac0226c70 2024-09-19T05:06:19.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.465+0000 7f86d3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ac0231e50 0x562ac0234240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:19.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.465+0000 7f86d3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ac0231e50 0x562ac0234240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41186/0 (socket says 172.21.15.43:41186) 2024-09-19T05:06:19.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.465+0000 7f86d3400640 1 -- 172.21.15.43:0/4179302142 learned_addr learned my addr 172.21.15.43:0/4179302142 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:19.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.465+0000 7f86dbe00640 1 --2- 172.21.15.43:0/4179302142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac0227590 0x562ac0231910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:19.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.465+0000 7f86d3400640 1 --2- 172.21.15.43:0/4179302142 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ac0226c70 0x562ac0227050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:19.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.465+0000 7f86d3400640 1 -- 172.21.15.43:0/4179302142 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ac0226c70 msgr2=0x562ac0227050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:19.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.465+0000 7f86d3400640 1 --2- 172.21.15.43:0/4179302142 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ac0226c70 0x562ac0227050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:19.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.465+0000 7f86d3400640 1 -- 172.21.15.43:0/4179302142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac0227590 msgr2=0x562ac0231910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:19.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.465+0000 7f86d3400640 1 --2- 172.21.15.43:0/4179302142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac0227590 0x562ac0231910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:19.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.465+0000 7f86d3400640 1 -- 172.21.15.43:0/4179302142 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ac0153ae0 con 0x562ac0231e50 2024-09-19T05:06:19.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.465+0000 7f86dbe00640 1 --2- 172.21.15.43:0/4179302142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac0227590 0x562ac0231910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:19.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.466+0000 7f86d3400640 1 --2- 172.21.15.43:0/4179302142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ac0231e50 0x562ac0234240 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f86c4009fd0 tx=0x7f86c40098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=711dbc0b69d3fa19 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:19.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.466+0000 7f86d1200640 1 -- 172.21.15.43:0/4179302142 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c401a070 con 0x562ac0231e50 2024-09-19T05:06:19.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.466+0000 7f86d1200640 1 -- 172.21.15.43:0/4179302142 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86c4004250 con 0x562ac0231e50 2024-09-19T05:06:19.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.466+0000 7f86d1200640 1 -- 172.21.15.43:0/4179302142 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c400f960 con 0x562ac0231e50 2024-09-19T05:06:19.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.466+0000 7f86d9600640 1 --2- 172.21.15.43:0/3994069374 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f86c00a3430 0x7f86c00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:19.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.467+0000 7f86d9600640 1 -- 172.21.15.43:0/3994069374 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f86d40cf930 con 0x7f86dc109120 2024-09-19T05:06:19.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.467+0000 7f86dbe00640 1 --2- 172.21.15.43:0/3994069374 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f86c00a3430 0x7f86c00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:19.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.467+0000 7f86e5417740 1 -- 172.21.15.43:0/4179302142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ac0231e50 msgr2=0x562ac0234240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:19.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.467+0000 7f86e5417740 1 --2- 172.21.15.43:0/4179302142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ac0231e50 0x562ac0234240 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f86c4009fd0 tx=0x7f86c40098e0 comp rx=0 tx=0).stop 2024-09-19T05:06:19.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.467+0000 7f86e5417740 1 -- 172.21.15.43:0/4179302142 shutdown_connections 2024-09-19T05:06:19.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.467+0000 7f86e5417740 1 --2- 172.21.15.43:0/4179302142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ac0231e50 0x562ac0234240 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:19.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.467+0000 7f86e5417740 1 --2- 172.21.15.43:0/4179302142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac0227590 0x562ac0231910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:19.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.467+0000 7f86e5417740 1 --2- 172.21.15.43:0/4179302142 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ac0226c70 0x562ac0227050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:19.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.467+0000 7f86e5417740 1 -- 172.21.15.43:0/4179302142 >> 172.21.15.43:0/4179302142 conn(0x562ac0223200 msgr2=0x562ac02235e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:19.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.467+0000 7f86e5417740 1 -- 172.21.15.43:0/4179302142 shutdown_connections 2024-09-19T05:06:19.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.467+0000 7f86e5417740 1 -- 172.21.15.43:0/4179302142 wait complete. 2024-09-19T05:06:19.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.468+0000 7f86dbe00640 1 --2- 172.21.15.43:0/3994069374 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f86c00a3430 0x7f86c00a58f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f86cc000c00 tx=0x7f86cc005b50 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:19.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.468+0000 7f86e5417740 1 -- start start 2024-09-19T05:06:19.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.468+0000 7f86e5417740 1 Processor -- start 2024-09-19T05:06:19.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.468+0000 7f86e5417740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ac0226c70 0x562ac022ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:19.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.468+0000 7f86e5417740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ac0227590 0x562ac022f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:19.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.468+0000 7f86e5417740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac0231e50 0x562ac022f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:19.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.468+0000 7f86e5417740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562ac0234a20 con 0x562ac0227590 2024-09-19T05:06:19.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.469+0000 7f86e5417740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562ac01480f0 con 0x562ac0231e50 2024-09-19T05:06:19.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.469+0000 7f86e5417740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562ac0234ba0 con 0x562ac0226c70 2024-09-19T05:06:19.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.469+0000 7f86e5417740 10 client.0 _ll_register_callbacks cb 0x562ac012fbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:19.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.469+0000 7f86d3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ac0227590 0x562ac022f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:19.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.469+0000 7f86e0a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac0231e50 0x562ac022f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:19.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.469+0000 7f86d3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ac0227590 0x562ac022f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41196/0 (socket says 172.21.15.43:41196) 2024-09-19T05:06:19.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.469+0000 7f86d3400640 1 -- 172.21.15.43:0/477219566 learned_addr learned my addr 172.21.15.43:0/477219566 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:19.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.469+0000 7f86dbe00640 1 --2- 172.21.15.43:0/477219566 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ac0226c70 0x562ac022ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:19.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.469+0000 7f86e0a00640 1 -- 172.21.15.43:0/477219566 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ac0226c70 msgr2=0x562ac022ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:19.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.469+0000 7f86e0a00640 1 --2- 172.21.15.43:0/477219566 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ac0226c70 0x562ac022ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:19.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.469+0000 7f86e0a00640 1 -- 172.21.15.43:0/477219566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ac0227590 msgr2=0x562ac022f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:19.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.469+0000 7f86e0a00640 1 --2- 172.21.15.43:0/477219566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ac0227590 0x562ac022f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:19.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.469+0000 7f86e0a00640 1 -- 172.21.15.43:0/477219566 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562abfefe3c0 con 0x562ac0231e50 2024-09-19T05:06:19.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.470+0000 7f86e0a00640 1 --2- 172.21.15.43:0/477219566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac0231e50 0x562ac022f7c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f86d40cd030 tx=0x7f86d409b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9e40d12ce9efa4d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:19.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.470+0000 7f86c9600640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:06:19.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.470+0000 7f86c9600640 1 -- 172.21.15.43:0/477219566 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d4020040 con 0x562ac0231e50 2024-09-19T05:06:19.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.470+0000 7f86c9600640 1 -- 172.21.15.43:0/477219566 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f86d409b0c0 con 0x562ac0231e50 2024-09-19T05:06:19.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.470+0000 7f86e5417740 1 -- 172.21.15.43:0/477219566 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86c4004250 con 0x562ac0231e50 2024-09-19T05:06:19.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.470+0000 7f86c9600640 1 -- 172.21.15.43:0/477219566 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f86d409ff70 con 0x562ac0231e50 2024-09-19T05:06:19.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.470+0000 7f86e5417740 10 client.25508 fetch_fsmap learned FSMap version 29 2024-09-19T05:06:19.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.470+0000 7f86e5417740 1 -- 172.21.15.43:0/477219566 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562ac0240a90 con 0x562ac0231e50 2024-09-19T05:06:19.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.470+0000 7f86c9600640 1 -- 172.21.15.43:0/477219566 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86c4004250 con 0x562ac0231e50 2024-09-19T05:06:19.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.471+0000 7f86c9600640 1 -- 172.21.15.43:0/477219566 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f86d40979c0 con 0x562ac0231e50 2024-09-19T05:06:19.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.471+0000 7f86e5417740 10 client.25508 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:06:19.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.471+0000 7f86e5417740 10 client.25508 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-19T05:06:19.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.471+0000 7f86e5417740 20 client.25508 populate_metadata read hostname 'smithi043' 2024-09-19T05:06:19.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.471+0000 7f86e5417740 1 --2- 172.21.15.43:0/477219566 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x562ac0241ef0 0x562ac0244340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:19.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.471+0000 7f86e5417740 4 client.25508 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:06:19.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.471+0000 7f86e5417740 1 -- 172.21.15.43:0/477219566 --> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ac019a950 con 0x562ac0241ef0 2024-09-19T05:06:19.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.471+0000 7f86dbe00640 1 --2- 172.21.15.43:0/477219566 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x562ac0241ef0 0x562ac0244340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:19.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.472+0000 7f86dbe00640 1 --2- 172.21.15.43:0/477219566 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x562ac0241ef0 0x562ac0244340 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f0482bd6e34f79c1 server_cookie=d22c30abddb17dcb in_seq=0 out_seq=0 2024-09-19T05:06:19.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.472+0000 7f86c9600640 10 client.25508 ms_handle_connect on v2:172.21.15.151:6836/3960378010 2024-09-19T05:06:19.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.473+0000 7f86c9600640 1 -- 172.21.15.43:0/477219566 <== mds.4 v2:172.21.15.151:6836/3960378010 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562ac019a950 con 0x562ac0241ef0 2024-09-19T05:06:19.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.473+0000 7f86c9600640 10 client.25508 handle_command_reply: tid=0 2024-09-19T05:06:19.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.473+0000 7f86e5417740 1 client.25508 shutdown 2024-09-19T05:06:19.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.473+0000 7f86e5417740 1 -- 172.21.15.43:0/477219566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac0231e50 msgr2=0x562ac022f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:19.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.473+0000 7f86e5417740 1 --2- 172.21.15.43:0/477219566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac0231e50 0x562ac022f7c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f86d40cd030 tx=0x7f86d409b8a0 comp rx=0 tx=0).stop 2024-09-19T05:06:19.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.473+0000 7f86e5417740 1 -- 172.21.15.43:0/477219566 shutdown_connections 2024-09-19T05:06:19.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.473+0000 7f86e5417740 1 --2- 172.21.15.43:0/477219566 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x562ac0241ef0 0x562ac0244340 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:19.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.473+0000 7f86e5417740 1 --2- 172.21.15.43:0/477219566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac0231e50 0x562ac022f7c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:19.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.473+0000 7f86e5417740 1 --2- 172.21.15.43:0/477219566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ac0227590 0x562ac022f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:19.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.473+0000 7f86e5417740 1 --2- 172.21.15.43:0/477219566 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ac0226c70 0x562ac022ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:19.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.473+0000 7f86c9600640 0 client.25508 ms_handle_reset on v2:172.21.15.151:6836/3960378010 2024-09-19T05:06:19.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.473+0000 7f86e5417740 1 -- 172.21.15.43:0/477219566 >> 172.21.15.43:0/477219566 conn(0x562ac0223200 msgr2=0x562ac0228ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:19.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.473+0000 7f86e5417740 1 -- 172.21.15.43:0/477219566 shutdown_connections 2024-09-19T05:06:19.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.473+0000 7f86e5417740 1 -- 172.21.15.43:0/477219566 wait complete. 2024-09-19T05:06:19.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.474+0000 7f86e5417740 20 client.25508 trim_cache size 0 max 16384 2024-09-19T05:06:19.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.487+0000 7f86e5417740 1 Processor -- start 2024-09-19T05:06:19.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.487+0000 7f86e5417740 1 -- start start 2024-09-19T05:06:19.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.488+0000 7f86e5417740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ac0226c70 0x562ac022e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:19.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.488+0000 7f86e5417740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ac0227590 0x562ac0249c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:19.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.488+0000 7f86e5417740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac0231e50 0x562ac024a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:19.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.488+0000 7f86e5417740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f86dc1b3e90 con 0x562ac0227590 2024-09-19T05:06:19.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.488+0000 7f86e5417740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f86dc1b59d0 con 0x562ac0231e50 2024-09-19T05:06:19.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.488+0000 7f86e5417740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f86dc1b4560 con 0x562ac0226c70 2024-09-19T05:06:19.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.488+0000 7f86d3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ac0227590 0x562ac0249c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:19.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.488+0000 7f86d3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ac0227590 0x562ac0249c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41198/0 (socket says 172.21.15.43:41198) 2024-09-19T05:06:19.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.488+0000 7f86d3400640 1 -- 172.21.15.43:0/316060878 learned_addr learned my addr 172.21.15.43:0/316060878 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:19.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.488+0000 7f86dbe00640 1 --2- 172.21.15.43:0/316060878 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ac0226c70 0x562ac022e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:19.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.488+0000 7f86e0a00640 1 --2- 172.21.15.43:0/316060878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac0231e50 0x562ac024a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:19.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.488+0000 7f86d3400640 1 -- 172.21.15.43:0/316060878 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ac0226c70 msgr2=0x562ac022e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:19.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.488+0000 7f86d3400640 1 --2- 172.21.15.43:0/316060878 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ac0226c70 0x562ac022e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:19.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.488+0000 7f86d3400640 1 -- 172.21.15.43:0/316060878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac0231e50 msgr2=0x562ac024a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:19.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.488+0000 7f86d3400640 1 --2- 172.21.15.43:0/316060878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac0231e50 0x562ac024a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:19.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.488+0000 7f86d3400640 1 -- 172.21.15.43:0/316060878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ac022dd30 con 0x562ac0227590 2024-09-19T05:06:19.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.488+0000 7f86dbe00640 1 --2- 172.21.15.43:0/316060878 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ac0226c70 0x562ac022e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:19.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.489+0000 7f86d3400640 1 --2- 172.21.15.43:0/316060878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ac0227590 0x562ac0249c10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f86c4012fd0 tx=0x7f86c4009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=126fac3a796a0cf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:19.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.489+0000 7f86cbe00640 1 -- 172.21.15.43:0/316060878 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c401a070 con 0x562ac0227590 2024-09-19T05:06:19.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.489+0000 7f86cbe00640 1 -- 172.21.15.43:0/316060878 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86c400f690 con 0x562ac0227590 2024-09-19T05:06:19.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.489+0000 7f86cbe00640 1 -- 172.21.15.43:0/316060878 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c40038e0 con 0x562ac0227590 2024-09-19T05:06:19.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.490+0000 7f86e5417740 1 -- 172.21.15.43:0/316060878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ac0227590 msgr2=0x562ac0249c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:19.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.490+0000 7f86e5417740 1 --2- 172.21.15.43:0/316060878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ac0227590 0x562ac0249c10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f86c4012fd0 tx=0x7f86c4009e90 comp rx=0 tx=0).stop 2024-09-19T05:06:19.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.490+0000 7f86e5417740 1 -- 172.21.15.43:0/316060878 shutdown_connections 2024-09-19T05:06:19.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.490+0000 7f86e5417740 1 --2- 172.21.15.43:0/316060878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac0231e50 0x562ac024a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:19.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.490+0000 7f86e5417740 1 --2- 172.21.15.43:0/316060878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ac0227590 0x562ac0249c10 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:19.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.490+0000 7f86e5417740 1 --2- 172.21.15.43:0/316060878 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ac0226c70 0x562ac022e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:19.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.490+0000 7f86e5417740 1 -- 172.21.15.43:0/316060878 >> 172.21.15.43:0/316060878 conn(0x562ac0223200 msgr2=0x562ac0230dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:19.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.490+0000 7f86e5417740 1 -- 172.21.15.43:0/316060878 shutdown_connections 2024-09-19T05:06:19.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.490+0000 7f86e5417740 1 -- 172.21.15.43:0/316060878 wait complete. 2024-09-19T05:06:19.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.490+0000 7f86e5417740 1 -- start start 2024-09-19T05:06:19.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.490+0000 7f86e5417740 1 Processor -- start 2024-09-19T05:06:19.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.491+0000 7f86e5417740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ac0226c70 0x562ac022bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:19.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.491+0000 7f86e5417740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ac0227590 0x562ac022c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:19.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.491+0000 7f86e5417740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac0231e50 0x562ac022c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:19.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.491+0000 7f86e5417740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f86dc1b5300 con 0x562ac0226c70 2024-09-19T05:06:19.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.491+0000 7f86e5417740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f86dc1b4c30 con 0x562ac0231e50 2024-09-19T05:06:19.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.491+0000 7f86e5417740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f86dc1b1e70 con 0x562ac0227590 2024-09-19T05:06:19.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.491+0000 7f86e5417740 10 client.0 _ll_register_callbacks cb 0x7f86d40092a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:19.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.491+0000 7f86d3400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ac0227590 0x562ac022c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:19.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.491+0000 7f86dbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ac0226c70 0x562ac022bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:19.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.491+0000 7f86d3400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ac0227590 0x562ac022c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:37780/0 (socket says 172.21.15.43:37780) 2024-09-19T05:06:19.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.491+0000 7f86d3400640 1 -- 172.21.15.43:0/1667324524 learned_addr learned my addr 172.21.15.43:0/1667324524 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:19.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.491+0000 7f86e0a00640 1 --2- 172.21.15.43:0/1667324524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac0231e50 0x562ac022c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:19.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.492+0000 7f86d3400640 1 -- 172.21.15.43:0/1667324524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac0231e50 msgr2=0x562ac022c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:19.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.492+0000 7f86d3400640 1 --2- 172.21.15.43:0/1667324524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac0231e50 0x562ac022c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:19.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.492+0000 7f86d3400640 1 -- 172.21.15.43:0/1667324524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ac0226c70 msgr2=0x562ac022bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:19.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.492+0000 7f86d3400640 1 --2- 172.21.15.43:0/1667324524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ac0226c70 0x562ac022bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:19.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.492+0000 7f86d3400640 1 -- 172.21.15.43:0/1667324524 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ac02abd40 con 0x562ac0227590 2024-09-19T05:06:19.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.492+0000 7f86d3400640 1 --2- 172.21.15.43:0/1667324524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ac0227590 0x562ac022c480 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f86c4009230 tx=0x7f86c4005760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8154367ffa3d4f8e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:19.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.492+0000 7f86c9600640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:06:19.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.492+0000 7f86c9600640 1 -- 172.21.15.43:0/1667324524 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c401a040 con 0x562ac0227590 2024-09-19T05:06:19.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.492+0000 7f86c9600640 1 -- 172.21.15.43:0/1667324524 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f86c400f840 con 0x562ac0227590 2024-09-19T05:06:19.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.492+0000 7f86e5417740 1 -- 172.21.15.43:0/1667324524 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86c400f690 con 0x562ac0227590 2024-09-19T05:06:19.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.493+0000 7f86c9600640 1 -- 172.21.15.43:0/1667324524 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f86c400fa80 con 0x562ac0227590 2024-09-19T05:06:19.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.493+0000 7f86e5417740 10 client.25474 fetch_fsmap learned FSMap version 29 2024-09-19T05:06:19.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.493+0000 7f86e5417740 1 -- 172.21.15.43:0/1667324524 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562ac02ad020 con 0x562ac0227590 2024-09-19T05:06:19.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.493+0000 7f86c9600640 1 -- 172.21.15.43:0/1667324524 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86c400f690 con 0x562ac0227590 2024-09-19T05:06:19.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.493+0000 7f86c9600640 1 -- 172.21.15.43:0/1667324524 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f86c403e450 con 0x562ac0227590 2024-09-19T05:06:19.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.493+0000 7f86e5417740 10 client.25474 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:06:19.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.493+0000 7f86e5417740 10 client.25474 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-19T05:06:19.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.493+0000 7f86e5417740 20 client.25474 populate_metadata read hostname 'smithi043' 2024-09-19T05:06:19.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.493+0000 7f86e5417740 1 --2- 172.21.15.43:0/1667324524 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x562ac02ad500 0x562ac02af920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:19.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.493+0000 7f86e5417740 4 client.25474 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:06:19.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.493+0000 7f86e5417740 1 -- 172.21.15.43:0/1667324524 --> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562ac0194e60 con 0x562ac02ad500 2024-09-19T05:06:19.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.494+0000 7f86dbe00640 1 --2- 172.21.15.43:0/1667324524 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x562ac02ad500 0x562ac02af920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:19.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.494+0000 7f86dbe00640 1 --2- 172.21.15.43:0/1667324524 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x562ac02ad500 0x562ac02af920 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bb777564d19d6bae server_cookie=3ba375e46294c206 in_seq=0 out_seq=0 2024-09-19T05:06:19.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.494+0000 7f86c9600640 10 client.25474 ms_handle_connect on v2:172.21.15.151:6836/3960378010 2024-09-19T05:06:19.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.495+0000 7f86c9600640 1 -- 172.21.15.43:0/1667324524 <== mds.4 v2:172.21.15.151:6836/3960378010 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562ac0194e60 con 0x562ac02ad500 2024-09-19T05:06:19.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.495+0000 7f86c9600640 10 client.25474 handle_command_reply: tid=0 2024-09-19T05:06:19.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.495+0000 7f86e5417740 1 client.25474 shutdown 2024-09-19T05:06:19.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.495+0000 7f86e5417740 1 -- 172.21.15.43:0/1667324524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ac0227590 msgr2=0x562ac022c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:19.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.495+0000 7f86e5417740 1 --2- 172.21.15.43:0/1667324524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ac0227590 0x562ac022c480 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f86c4009230 tx=0x7f86c4005760 comp rx=0 tx=0).stop 2024-09-19T05:06:19.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.495+0000 7f86e5417740 1 -- 172.21.15.43:0/1667324524 shutdown_connections 2024-09-19T05:06:19.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.495+0000 7f86e5417740 1 --2- 172.21.15.43:0/1667324524 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x562ac02ad500 0x562ac02af920 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:19.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.495+0000 7f86e5417740 1 --2- 172.21.15.43:0/1667324524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ac0231e50 0x562ac022c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:19.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.495+0000 7f86e5417740 1 --2- 172.21.15.43:0/1667324524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ac0227590 0x562ac022c480 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:19.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.495+0000 7f86e5417740 1 --2- 172.21.15.43:0/1667324524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ac0226c70 0x562ac022bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:19.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.495+0000 7f86e5417740 1 -- 172.21.15.43:0/1667324524 >> 172.21.15.43:0/1667324524 conn(0x562ac0223200 msgr2=0x562ac0221b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:19.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.495+0000 7f86c9600640 0 client.25474 ms_handle_reset on v2:172.21.15.151:6836/3960378010 2024-09-19T05:06:19.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.495+0000 7f86e5417740 1 -- 172.21.15.43:0/1667324524 shutdown_connections 2024-09-19T05:06:19.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.495+0000 7f86e5417740 1 -- 172.21.15.43:0/1667324524 wait complete. 2024-09-19T05:06:19.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.495+0000 7f86e5417740 20 client.25474 trim_cache size 0 max 16384 2024-09-19T05:06:19.501 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:06:19.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.496+0000 7f86e2a00640 1 -- 172.21.15.43:0/3994069374 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f86c00a3430 msgr2=0x7f86c00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:19.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.496+0000 7f86e2a00640 1 --2- 172.21.15.43:0/3994069374 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f86c00a3430 0x7f86c00a58f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f86cc000c00 tx=0x7f86cc005b50 comp rx=0 tx=0).stop 2024-09-19T05:06:19.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.496+0000 7f86e2a00640 1 -- 172.21.15.43:0/3994069374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86dc109120 msgr2=0x7f86dc077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:19.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.496+0000 7f86e2a00640 1 --2- 172.21.15.43:0/3994069374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86dc109120 0x7f86dc077180 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f86d400db20 tx=0x7f86d400b260 comp rx=0 tx=0).stop 2024-09-19T05:06:19.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.496+0000 7f86e2a00640 1 -- 172.21.15.43:0/3994069374 shutdown_connections 2024-09-19T05:06:19.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.496+0000 7f86e2a00640 1 --2- 172.21.15.43:0/3994069374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86dc109120 0x7f86dc077180 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:19.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.496+0000 7f86e2a00640 1 --2- 172.21.15.43:0/3994069374 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86dc1049a0 0x7f86dc076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:19.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.496+0000 7f86e2a00640 1 --2- 172.21.15.43:0/3994069374 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f86c00a3430 0x7f86c00a58f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:19.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.496+0000 7f86e2a00640 1 --2- 172.21.15.43:0/3994069374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86dc103ff0 0x7f86dc076700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:19.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.496+0000 7f86e2a00640 1 -- 172.21.15.43:0/3994069374 >> 172.21.15.43:0/3994069374 conn(0x7f86dc0ffa90 msgr2=0x7f86dc100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:19.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.496+0000 7f86e2a00640 1 -- 172.21.15.43:0/3994069374 shutdown_connections 2024-09-19T05:06:19.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:19.496+0000 7f86e2a00640 1 -- 172.21.15.43:0/3994069374 wait complete. 2024-09-19T05:06:20.033 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:19 smithi043 ceph-mon[30800]: pgmap v53: 97 pgs: 97 active+clean; 1.0 GiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 160 MiB/s wr, 50 op/s 2024-09-19T05:06:20.042 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-19T05:06:20.042 INFO:tasks.workunit.client.0.smithi043.stdout: Children see throughput for 1 re-readers = 2662705.75 kB/sec 2024-09-19T05:06:20.042 INFO:tasks.workunit.client.0.smithi043.stdout: Parent sees throughput for 1 re-readers = 2658884.18 kB/sec 2024-09-19T05:06:20.042 INFO:tasks.workunit.client.0.smithi043.stdout: Min throughput per process = 2662705.75 kB/sec 2024-09-19T05:06:20.042 INFO:tasks.workunit.client.0.smithi043.stdout: Max throughput per process = 2662705.75 kB/sec 2024-09-19T05:06:20.042 INFO:tasks.workunit.client.0.smithi043.stdout: Avg throughput per process = 2662705.75 kB/sec 2024-09-19T05:06:20.042 INFO:tasks.workunit.client.0.smithi043.stdout: Min xfer = 1048576.00 kB 2024-09-19T05:06:20.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:19 smithi151 ceph-mon[35425]: pgmap v53: 97 pgs: 97 active+clean; 1.0 GiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 160 MiB/s wr, 50 op/s 2024-09-19T05:06:20.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:19 smithi096 ceph-mon[35085]: pgmap v53: 97 pgs: 97 active+clean; 1.0 GiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 160 MiB/s wr, 50 op/s 2024-09-19T05:06:20.545 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T05:06:20.925 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:06:21.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.499+0000 7f3e93000640 1 -- 172.21.15.43:0/1982966836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e8c103ff0 msgr2=0x7f3e8c1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:21.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.499+0000 7f3e93000640 1 --2- 172.21.15.43:0/1982966836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e8c103ff0 0x7f3e8c1043d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c009900 tx=0x7f3e7c02a410 comp rx=0 tx=0).stop 2024-09-19T05:06:21.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.500+0000 7f3e93000640 1 -- 172.21.15.43:0/1982966836 shutdown_connections 2024-09-19T05:06:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.500+0000 7f3e93000640 1 --2- 172.21.15.43:0/1982966836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e8c109120 0x7f3e8c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.500+0000 7f3e93000640 1 --2- 172.21.15.43:0/1982966836 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e8c1049a0 0x7f3e8c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.500+0000 7f3e93000640 1 --2- 172.21.15.43:0/1982966836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e8c103ff0 0x7f3e8c1043d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.500+0000 7f3e93000640 1 -- 172.21.15.43:0/1982966836 >> 172.21.15.43:0/1982966836 conn(0x7f3e8c0ffa90 msgr2=0x7f3e8c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.500+0000 7f3e93000640 1 -- 172.21.15.43:0/1982966836 shutdown_connections 2024-09-19T05:06:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.500+0000 7f3e93000640 1 -- 172.21.15.43:0/1982966836 wait complete. 2024-09-19T05:06:21.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.501+0000 7f3e93000640 1 Processor -- start 2024-09-19T05:06:21.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.501+0000 7f3e93000640 1 -- start start 2024-09-19T05:06:21.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.501+0000 7f3e93000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e8c103ff0 0x7f3e8c19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:21.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.501+0000 7f3e93000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e8c1049a0 0x7f3e8c19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:21.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.501+0000 7f3e93000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e8c109120 0x7f3e8c19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:21.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.501+0000 7f3e93000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8c114190 con 0x7f3e8c1049a0 2024-09-19T05:06:21.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.501+0000 7f3e93000640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8c114010 con 0x7f3e8c103ff0 2024-09-19T05:06:21.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.501+0000 7f3e93000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8c114310 con 0x7f3e8c109120 2024-09-19T05:06:21.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.501+0000 7f3e83400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e8c1049a0 0x7f3e8c19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:21.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.501+0000 7f3e83400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e8c1049a0 0x7f3e8c19ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41218/0 (socket says 172.21.15.43:41218) 2024-09-19T05:06:21.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.502+0000 7f3e83400640 1 -- 172.21.15.43:0/1039840961 learned_addr learned my addr 172.21.15.43:0/1039840961 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:21.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.502+0000 7f3e8be00640 1 --2- 172.21.15.43:0/1039840961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e8c103ff0 0x7f3e8c19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:21.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.502+0000 7f3e91000640 1 --2- 172.21.15.43:0/1039840961 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e8c109120 0x7f3e8c19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:21.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.502+0000 7f3e83400640 1 -- 172.21.15.43:0/1039840961 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e8c109120 msgr2=0x7f3e8c19eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:21.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.502+0000 7f3e83400640 1 --2- 172.21.15.43:0/1039840961 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e8c109120 0x7f3e8c19eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:21.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.502+0000 7f3e83400640 1 -- 172.21.15.43:0/1039840961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e8c103ff0 msgr2=0x7f3e8c19a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:21.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.502+0000 7f3e83400640 1 --2- 172.21.15.43:0/1039840961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e8c103ff0 0x7f3e8c19a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:21.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.502+0000 7f3e83400640 1 -- 172.21.15.43:0/1039840961 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e8c19f5c0 con 0x7f3e8c1049a0 2024-09-19T05:06:21.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.502+0000 7f3e83400640 1 --2- 172.21.15.43:0/1039840961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e8c1049a0 0x7f3e8c19ab50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3e7400e9f0 tx=0x7f3e7400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:21.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.503+0000 7f3e89600640 1 -- 172.21.15.43:0/1039840961 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7400cd80 con 0x7f3e8c1049a0 2024-09-19T05:06:21.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.503+0000 7f3e89600640 1 -- 172.21.15.43:0/1039840961 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e74005500 con 0x7f3e8c1049a0 2024-09-19T05:06:21.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.503+0000 7f3e93000640 1 -- 172.21.15.43:0/1039840961 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e8c19f8b0 con 0x7f3e8c1049a0 2024-09-19T05:06:21.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.503+0000 7f3e89600640 1 -- 172.21.15.43:0/1039840961 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e74010690 con 0x7f3e8c1049a0 2024-09-19T05:06:21.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.503+0000 7f3e93000640 1 -- 172.21.15.43:0/1039840961 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e8c1a7190 con 0x7f3e8c1049a0 2024-09-19T05:06:21.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.506+0000 7f3e95a39740 1 Processor -- start 2024-09-19T05:06:21.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.506+0000 7f3e89600640 1 -- 172.21.15.43:0/1039840961 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f3e740046c0 con 0x7f3e8c1049a0 2024-09-19T05:06:21.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.506+0000 7f3e95a39740 1 -- start start 2024-09-19T05:06:21.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.506+0000 7f3e95a39740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c24bdac70 0x562c24bdb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:21.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.506+0000 7f3e95a39740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562c24bdb590 0x562c24be5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:21.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.507+0000 7f3e95a39740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c24be5e50 0x562c24be8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:21.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.507+0000 7f3e95a39740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562c24b413e0 con 0x562c24bdac70 2024-09-19T05:06:21.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.507+0000 7f3e95a39740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562c24ae8d80 con 0x562c24be5e50 2024-09-19T05:06:21.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.507+0000 7f3e95a39740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562c24b3d3a0 con 0x562c24bdb590 2024-09-19T05:06:21.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.507+0000 7f3e91000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c24bdac70 0x562c24bdb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:21.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.507+0000 7f3e91000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c24bdac70 0x562c24bdb050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41228/0 (socket says 172.21.15.43:41228) 2024-09-19T05:06:21.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.507+0000 7f3e91000640 1 -- 172.21.15.43:0/1368623361 learned_addr learned my addr 172.21.15.43:0/1368623361 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:21.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.507+0000 7f3e8be00640 1 --2- 172.21.15.43:0/1368623361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562c24bdb590 0x562c24be5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:21.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.507+0000 7f3e83400640 1 --2- 172.21.15.43:0/1368623361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c24be5e50 0x562c24be8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:21.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.507+0000 7f3e91000640 1 -- 172.21.15.43:0/1368623361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562c24bdb590 msgr2=0x562c24be5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:21.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.507+0000 7f3e91000640 1 --2- 172.21.15.43:0/1368623361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562c24bdb590 0x562c24be5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:21.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.507+0000 7f3e91000640 1 -- 172.21.15.43:0/1368623361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c24be5e50 msgr2=0x562c24be8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:21.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.507+0000 7f3e91000640 1 --2- 172.21.15.43:0/1368623361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c24be5e50 0x562c24be8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:21.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.507+0000 7f3e91000640 1 -- 172.21.15.43:0/1368623361 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c24b07ae0 con 0x562c24bdac70 2024-09-19T05:06:21.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.507+0000 7f3e8be00640 1 --2- 172.21.15.43:0/1368623361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562c24bdb590 0x562c24be5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:06:21.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.508+0000 7f3e91000640 1 --2- 172.21.15.43:0/1368623361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c24bdac70 0x562c24bdb050 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3e84009f20 tx=0x7f3e84010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baa35401073597ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:21.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.508+0000 7f3e89600640 1 --2- 172.21.15.43:0/1039840961 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f3e700a33b0 0x7f3e700a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:21.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.508+0000 7f3e81600640 1 -- 172.21.15.43:0/1368623361 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e84011700 con 0x562c24bdac70 2024-09-19T05:06:21.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.508+0000 7f3e81600640 1 -- 172.21.15.43:0/1368623361 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e84011d80 con 0x562c24bdac70 2024-09-19T05:06:21.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.508+0000 7f3e89600640 1 -- 172.21.15.43:0/1039840961 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f3e740d01a0 con 0x7f3e8c1049a0 2024-09-19T05:06:21.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.508+0000 7f3e81600640 1 -- 172.21.15.43:0/1368623361 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8401a380 con 0x562c24bdac70 2024-09-19T05:06:21.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.508+0000 7f3e8be00640 1 --2- 172.21.15.43:0/1039840961 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f3e700a33b0 0x7f3e700a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:21.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.509+0000 7f3e95a39740 1 -- 172.21.15.43:0/1368623361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c24bdac70 msgr2=0x562c24bdb050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:21.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.509+0000 7f3e95a39740 1 --2- 172.21.15.43:0/1368623361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c24bdac70 0x562c24bdb050 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3e84009f20 tx=0x7f3e84010860 comp rx=0 tx=0).stop 2024-09-19T05:06:21.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.509+0000 7f3e95a39740 1 -- 172.21.15.43:0/1368623361 shutdown_connections 2024-09-19T05:06:21.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.509+0000 7f3e95a39740 1 --2- 172.21.15.43:0/1368623361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c24be5e50 0x562c24be8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:21.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.509+0000 7f3e95a39740 1 --2- 172.21.15.43:0/1368623361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562c24bdb590 0x562c24be5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:21.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.509+0000 7f3e95a39740 1 --2- 172.21.15.43:0/1368623361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c24bdac70 0x562c24bdb050 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:21.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.509+0000 7f3e95a39740 1 -- 172.21.15.43:0/1368623361 >> 172.21.15.43:0/1368623361 conn(0x562c24bd7200 msgr2=0x562c24bd75e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:21.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.509+0000 7f3e8be00640 1 --2- 172.21.15.43:0/1039840961 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f3e700a33b0 0x7f3e700a5870 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c000c00 tx=0x7f3e7c005b50 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:21.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.509+0000 7f3e95a39740 1 -- 172.21.15.43:0/1368623361 shutdown_connections 2024-09-19T05:06:21.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.509+0000 7f3e95a39740 1 -- 172.21.15.43:0/1368623361 wait complete. 2024-09-19T05:06:21.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.510+0000 7f3e95a39740 1 -- start start 2024-09-19T05:06:21.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.510+0000 7f3e95a39740 1 Processor -- start 2024-09-19T05:06:21.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.510+0000 7f3e95a39740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562c24bdac70 0x562c24be2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:21.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.510+0000 7f3e95a39740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c24bdb590 0x562c24be32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:21.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.510+0000 7f3e95a39740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c24be5e50 0x562c24be3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:21.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.510+0000 7f3e95a39740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562c24be8a70 con 0x562c24bdb590 2024-09-19T05:06:21.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.510+0000 7f3e95a39740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562c24afc0f0 con 0x562c24be5e50 2024-09-19T05:06:21.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.510+0000 7f3e95a39740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562c24be8bf0 con 0x562c24bdac70 2024-09-19T05:06:21.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.511+0000 7f3e95a39740 10 client.0 _ll_register_callbacks cb 0x562c24ae3bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:21.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.511+0000 7f3e83400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c24bdb590 0x562c24be32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:21.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.511+0000 7f3e83400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c24bdb590 0x562c24be32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41240/0 (socket says 172.21.15.43:41240) 2024-09-19T05:06:21.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.511+0000 7f3e83400640 1 -- 172.21.15.43:0/1430945386 learned_addr learned my addr 172.21.15.43:0/1430945386 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:21.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.511+0000 7f3e8be00640 1 --2- 172.21.15.43:0/1430945386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562c24bdac70 0x562c24be2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:21.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.511+0000 7f3e91000640 1 --2- 172.21.15.43:0/1430945386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c24be5e50 0x562c24be3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:21.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.511+0000 7f3e83400640 1 -- 172.21.15.43:0/1430945386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562c24bdac70 msgr2=0x562c24be2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:21.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.511+0000 7f3e83400640 1 --2- 172.21.15.43:0/1430945386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562c24bdac70 0x562c24be2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:21.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.511+0000 7f3e83400640 1 -- 172.21.15.43:0/1430945386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c24be5e50 msgr2=0x562c24be3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:21.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.511+0000 7f3e83400640 1 --2- 172.21.15.43:0/1430945386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c24be5e50 0x562c24be3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:21.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.511+0000 7f3e83400640 1 -- 172.21.15.43:0/1430945386 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c248b23c0 con 0x562c24bdb590 2024-09-19T05:06:21.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.512+0000 7f3e83400640 1 --2- 172.21.15.43:0/1430945386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c24bdb590 0x562c24be32d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3e7400c6a0 tx=0x7f3e74098730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c00e9b88fe715e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:21.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.512+0000 7f3e7a000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:06:21.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.512+0000 7f3e7a000640 1 -- 172.21.15.43:0/1430945386 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7409bdf0 con 0x562c24bdb590 2024-09-19T05:06:21.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.512+0000 7f3e7a000640 1 -- 172.21.15.43:0/1430945386 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f3e740a4e70 con 0x562c24bdb590 2024-09-19T05:06:21.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.512+0000 7f3e95a39740 1 -- 172.21.15.43:0/1430945386 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e84011d80 con 0x562c24bdb590 2024-09-19T05:06:21.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.512+0000 7f3e7a000640 1 -- 172.21.15.43:0/1430945386 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3e7409cb90 con 0x562c24bdb590 2024-09-19T05:06:21.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.512+0000 7f3e95a39740 10 client.15537 fetch_fsmap learned FSMap version 29 2024-09-19T05:06:21.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.512+0000 7f3e95a39740 1 -- 172.21.15.43:0/1430945386 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562c24bf4bb0 con 0x562c24bdb590 2024-09-19T05:06:21.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.512+0000 7f3e7a000640 1 -- 172.21.15.43:0/1430945386 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e84011d80 con 0x562c24bdb590 2024-09-19T05:06:21.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.513+0000 7f3e7a000640 1 -- 172.21.15.43:0/1430945386 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3e7409b8e0 con 0x562c24bdb590 2024-09-19T05:06:21.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.513+0000 7f3e95a39740 10 client.15537 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:06:21.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.513+0000 7f3e95a39740 10 client.15537 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T05:06:21.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.513+0000 7f3e95a39740 20 client.15537 populate_metadata read hostname 'smithi043' 2024-09-19T05:06:21.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.513+0000 7f3e95a39740 1 --2- 172.21.15.43:0/1430945386 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x562c24bf6070 0x562c24bf8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:21.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.513+0000 7f3e95a39740 4 client.15537 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:06:21.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.513+0000 7f3e95a39740 1 -- 172.21.15.43:0/1430945386 --> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c24b4e950 con 0x562c24bf6070 2024-09-19T05:06:21.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.513+0000 7f3e8be00640 1 --2- 172.21.15.43:0/1430945386 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x562c24bf6070 0x562c24bf8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:21.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.514+0000 7f3e8be00640 1 --2- 172.21.15.43:0/1430945386 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x562c24bf6070 0x562c24bf8460 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7126cefe11258bbb server_cookie=c595f299d8800808 in_seq=0 out_seq=0 2024-09-19T05:06:21.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.514+0000 7f3e7a000640 10 client.15537 ms_handle_connect on v2:172.21.15.96:6836/3978388286 2024-09-19T05:06:21.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.515+0000 7f3e7a000640 1 -- 172.21.15.43:0/1430945386 <== mds.3 v2:172.21.15.96:6836/3978388286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562c24b4e950 con 0x562c24bf6070 2024-09-19T05:06:21.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.515+0000 7f3e7a000640 10 client.15537 handle_command_reply: tid=0 2024-09-19T05:06:21.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.515+0000 7f3e95a39740 1 client.15537 shutdown 2024-09-19T05:06:21.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.515+0000 7f3e95a39740 1 -- 172.21.15.43:0/1430945386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c24bdb590 msgr2=0x562c24be32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:21.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.515+0000 7f3e95a39740 1 --2- 172.21.15.43:0/1430945386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c24bdb590 0x562c24be32d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3e7400c6a0 tx=0x7f3e74098730 comp rx=0 tx=0).stop 2024-09-19T05:06:21.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.515+0000 7f3e95a39740 1 -- 172.21.15.43:0/1430945386 shutdown_connections 2024-09-19T05:06:21.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.515+0000 7f3e95a39740 1 --2- 172.21.15.43:0/1430945386 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x562c24bf6070 0x562c24bf8460 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:21.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.515+0000 7f3e95a39740 1 --2- 172.21.15.43:0/1430945386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c24be5e50 0x562c24be3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:21.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.515+0000 7f3e95a39740 1 --2- 172.21.15.43:0/1430945386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c24bdb590 0x562c24be32d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:21.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.515+0000 7f3e95a39740 1 --2- 172.21.15.43:0/1430945386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562c24bdac70 0x562c24be2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:21.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.515+0000 7f3e7a000640 0 client.15537 ms_handle_reset on v2:172.21.15.96:6836/3978388286 2024-09-19T05:06:21.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.515+0000 7f3e95a39740 1 -- 172.21.15.43:0/1430945386 >> 172.21.15.43:0/1430945386 conn(0x562c24bd7200 msgr2=0x562c24bd6380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:21.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.515+0000 7f3e95a39740 1 -- 172.21.15.43:0/1430945386 shutdown_connections 2024-09-19T05:06:21.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.515+0000 7f3e95a39740 1 -- 172.21.15.43:0/1430945386 wait complete. 2024-09-19T05:06:21.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.515+0000 7f3e95a39740 20 client.15537 trim_cache size 0 max 16384 2024-09-19T05:06:21.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.529+0000 7f3e95a39740 1 Processor -- start 2024-09-19T05:06:21.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.529+0000 7f3e95a39740 1 -- start start 2024-09-19T05:06:21.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.529+0000 7f3e95a39740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c24bdac70 0x562c24bdd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:21.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.529+0000 7f3e95a39740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c24bdb590 0x562c24bd75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:21.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.530+0000 7f3e95a39740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562c24be5e50 0x562c24bfdc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:21.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.530+0000 7f3e95a39740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8c1afa60 con 0x562c24bdac70 2024-09-19T05:06:21.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.530+0000 7f3e95a39740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8c1b15a0 con 0x562c24bdb590 2024-09-19T05:06:21.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.530+0000 7f3e95a39740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8c1b0130 con 0x562c24be5e50 2024-09-19T05:06:21.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.530+0000 7f3e8be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c24bdac70 0x562c24bdd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:21.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.530+0000 7f3e8be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c24bdac70 0x562c24bdd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41250/0 (socket says 172.21.15.43:41250) 2024-09-19T05:06:21.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.530+0000 7f3e8be00640 1 -- 172.21.15.43:0/1881137627 learned_addr learned my addr 172.21.15.43:0/1881137627 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:21.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.530+0000 7f3e83400640 1 --2- 172.21.15.43:0/1881137627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c24bdb590 0x562c24bd75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:21.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.530+0000 7f3e91000640 1 --2- 172.21.15.43:0/1881137627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562c24be5e50 0x562c24bfdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:21.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.530+0000 7f3e8be00640 1 -- 172.21.15.43:0/1881137627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562c24be5e50 msgr2=0x562c24bfdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:21.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.530+0000 7f3e8be00640 1 --2- 172.21.15.43:0/1881137627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562c24be5e50 0x562c24bfdc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:21.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.530+0000 7f3e8be00640 1 -- 172.21.15.43:0/1881137627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c24bdb590 msgr2=0x562c24bd75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:21.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.530+0000 7f3e8be00640 1 --2- 172.21.15.43:0/1881137627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c24bdb590 0x562c24bd75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:21.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.530+0000 7f3e8be00640 1 -- 172.21.15.43:0/1881137627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c24be1d40 con 0x562c24bdac70 2024-09-19T05:06:21.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.531+0000 7f3e8be00640 1 --2- 172.21.15.43:0/1881137627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c24bdac70 0x562c24bdd6d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c009900 tx=0x7f3e7c03e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f361b1dd58458e50 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:21.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.531+0000 7f3e80c00640 1 -- 172.21.15.43:0/1881137627 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c057070 con 0x562c24bdac70 2024-09-19T05:06:21.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.531+0000 7f3e80c00640 1 -- 172.21.15.43:0/1881137627 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e7c004c00 con 0x562c24bdac70 2024-09-19T05:06:21.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.531+0000 7f3e80c00640 1 -- 172.21.15.43:0/1881137627 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c035040 con 0x562c24bdac70 2024-09-19T05:06:21.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.532+0000 7f3e95a39740 1 -- 172.21.15.43:0/1881137627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c24bdac70 msgr2=0x562c24bdd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:21.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.532+0000 7f3e95a39740 1 --2- 172.21.15.43:0/1881137627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c24bdac70 0x562c24bdd6d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c009900 tx=0x7f3e7c03e000 comp rx=0 tx=0).stop 2024-09-19T05:06:21.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.532+0000 7f3e95a39740 1 -- 172.21.15.43:0/1881137627 shutdown_connections 2024-09-19T05:06:21.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.532+0000 7f3e95a39740 1 --2- 172.21.15.43:0/1881137627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562c24be5e50 0x562c24bfdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:21.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.532+0000 7f3e95a39740 1 --2- 172.21.15.43:0/1881137627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c24bdb590 0x562c24bd75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:21.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.532+0000 7f3e95a39740 1 --2- 172.21.15.43:0/1881137627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c24bdac70 0x562c24bdd6d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:21.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.532+0000 7f3e95a39740 1 -- 172.21.15.43:0/1881137627 >> 172.21.15.43:0/1881137627 conn(0x562c24bd7200 msgr2=0x562c24be4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:21.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.532+0000 7f3e95a39740 1 -- 172.21.15.43:0/1881137627 shutdown_connections 2024-09-19T05:06:21.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.532+0000 7f3e95a39740 1 -- 172.21.15.43:0/1881137627 wait complete. 2024-09-19T05:06:21.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.532+0000 7f3e95a39740 1 -- start start 2024-09-19T05:06:21.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.532+0000 7f3e95a39740 1 Processor -- start 2024-09-19T05:06:21.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.533+0000 7f3e95a39740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562c24bdac70 0x562c24bea2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:21.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.533+0000 7f3e95a39740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c24bdb590 0x562c24bea820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:21.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.533+0000 7f3e95a39740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c24be5e50 0x562c24c5fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:21.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.533+0000 7f3e95a39740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8c1b0ed0 con 0x562c24be5e50 2024-09-19T05:06:21.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.533+0000 7f3e95a39740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8c1b0800 con 0x562c24bdb590 2024-09-19T05:06:21.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.533+0000 7f3e95a39740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8c1ada20 con 0x562c24bdac70 2024-09-19T05:06:21.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.533+0000 7f3e95a39740 10 client.0 _ll_register_callbacks cb 0x7f3e74009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:21.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.533+0000 7f3e91000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c24be5e50 0x562c24c5fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:21.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.533+0000 7f3e8be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562c24bdac70 0x562c24bea2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:21.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.533+0000 7f3e83400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c24bdb590 0x562c24bea820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:21.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.533+0000 7f3e8be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562c24bdac70 0x562c24bea2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:37826/0 (socket says 172.21.15.43:37826) 2024-09-19T05:06:21.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.533+0000 7f3e8be00640 1 -- 172.21.15.43:0/3063549616 learned_addr learned my addr 172.21.15.43:0/3063549616 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:21.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.534+0000 7f3e8be00640 1 -- 172.21.15.43:0/3063549616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c24bdb590 msgr2=0x562c24bea820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:21.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.534+0000 7f3e8be00640 1 --2- 172.21.15.43:0/3063549616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c24bdb590 0x562c24bea820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:21.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.534+0000 7f3e8be00640 1 -- 172.21.15.43:0/3063549616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c24be5e50 msgr2=0x562c24c5fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:21.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.534+0000 7f3e8be00640 1 --2- 172.21.15.43:0/3063549616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c24be5e50 0x562c24c5fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:21.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.534+0000 7f3e8be00640 1 -- 172.21.15.43:0/3063549616 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c24c601a0 con 0x562c24bdac70 2024-09-19T05:06:21.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.534+0000 7f3e83400640 1 --2- 172.21.15.43:0/3063549616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c24bdb590 0x562c24bea820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:21.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.534+0000 7f3e8be00640 1 --2- 172.21.15.43:0/3063549616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562c24bdac70 0x562c24bea2e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c055040 tx=0x7f3e7c009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6aaed49a4d9343 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:21.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.534+0000 7f3e7a000640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:06:21.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.534+0000 7f3e7a000640 1 -- 172.21.15.43:0/3063549616 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c057040 con 0x562c24bdac70 2024-09-19T05:06:21.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.534+0000 7f3e95a39740 1 -- 172.21.15.43:0/3063549616 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e7c004c00 con 0x562c24bdac70 2024-09-19T05:06:21.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.534+0000 7f3e7a000640 1 -- 172.21.15.43:0/3063549616 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f3e7c0054a0 con 0x562c24bdac70 2024-09-19T05:06:21.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.535+0000 7f3e7a000640 1 -- 172.21.15.43:0/3063549616 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3e7c068050 con 0x562c24bdac70 2024-09-19T05:06:21.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.535+0000 7f3e95a39740 10 client.25492 fetch_fsmap learned FSMap version 29 2024-09-19T05:06:21.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.535+0000 7f3e95a39740 1 -- 172.21.15.43:0/3063549616 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562c24c614a0 con 0x562c24bdac70 2024-09-19T05:06:21.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.535+0000 7f3e7a000640 1 -- 172.21.15.43:0/3063549616 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e7c004c00 con 0x562c24bdac70 2024-09-19T05:06:21.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.535+0000 7f3e7a000640 1 -- 172.21.15.43:0/3063549616 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3e7c063030 con 0x562c24bdac70 2024-09-19T05:06:21.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.535+0000 7f3e95a39740 10 client.25492 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:06:21.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.535+0000 7f3e95a39740 10 client.25492 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T05:06:21.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.535+0000 7f3e95a39740 20 client.25492 populate_metadata read hostname 'smithi043' 2024-09-19T05:06:21.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.535+0000 7f3e95a39740 1 --2- 172.21.15.43:0/3063549616 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x562c24c61b30 0x562c24c63f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:21.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.535+0000 7f3e95a39740 4 client.25492 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:06:21.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.535+0000 7f3e95a39740 1 -- 172.21.15.43:0/3063549616 --> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562c24b48e60 con 0x562c24c61b30 2024-09-19T05:06:21.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.536+0000 7f3e83400640 1 --2- 172.21.15.43:0/3063549616 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x562c24c61b30 0x562c24c63f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:21.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.536+0000 7f3e83400640 1 --2- 172.21.15.43:0/3063549616 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x562c24c61b30 0x562c24c63f50 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=959a13be16a653eb server_cookie=6fb6a380c077bf7a in_seq=0 out_seq=0 2024-09-19T05:06:21.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.536+0000 7f3e7a000640 10 client.25492 ms_handle_connect on v2:172.21.15.96:6836/3978388286 2024-09-19T05:06:21.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.537+0000 7f3e7a000640 1 -- 172.21.15.43:0/3063549616 <== mds.3 v2:172.21.15.96:6836/3978388286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562c24b48e60 con 0x562c24c61b30 2024-09-19T05:06:21.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.537+0000 7f3e7a000640 10 client.25492 handle_command_reply: tid=0 2024-09-19T05:06:21.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.537+0000 7f3e95a39740 1 client.25492 shutdown 2024-09-19T05:06:21.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.537+0000 7f3e95a39740 1 -- 172.21.15.43:0/3063549616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562c24bdac70 msgr2=0x562c24bea2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:21.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.537+0000 7f3e95a39740 1 --2- 172.21.15.43:0/3063549616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562c24bdac70 0x562c24bea2e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c055040 tx=0x7f3e7c009620 comp rx=0 tx=0).stop 2024-09-19T05:06:21.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.537+0000 7f3e95a39740 1 -- 172.21.15.43:0/3063549616 shutdown_connections 2024-09-19T05:06:21.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.537+0000 7f3e95a39740 1 --2- 172.21.15.43:0/3063549616 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x562c24c61b30 0x562c24c63f50 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:21.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.537+0000 7f3e95a39740 1 --2- 172.21.15.43:0/3063549616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c24be5e50 0x562c24c5fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:21.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.537+0000 7f3e95a39740 1 --2- 172.21.15.43:0/3063549616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562c24bdb590 0x562c24bea820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:21.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.537+0000 7f3e95a39740 1 --2- 172.21.15.43:0/3063549616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562c24bdac70 0x562c24bea2e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:21.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.537+0000 7f3e7a000640 0 client.25492 ms_handle_reset on v2:172.21.15.96:6836/3978388286 2024-09-19T05:06:21.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.537+0000 7f3e95a39740 1 -- 172.21.15.43:0/3063549616 >> 172.21.15.43:0/3063549616 conn(0x562c24bd7200 msgr2=0x562c24be2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:21.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.537+0000 7f3e95a39740 1 -- 172.21.15.43:0/3063549616 shutdown_connections 2024-09-19T05:06:21.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.537+0000 7f3e95a39740 1 -- 172.21.15.43:0/3063549616 wait complete. 2024-09-19T05:06:21.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.537+0000 7f3e95a39740 20 client.25492 trim_cache size 0 max 16384 2024-09-19T05:06:21.542 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:06:21.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.538+0000 7f3e93000640 1 -- 172.21.15.43:0/1039840961 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f3e700a33b0 msgr2=0x7f3e700a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:21.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.538+0000 7f3e93000640 1 --2- 172.21.15.43:0/1039840961 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f3e700a33b0 0x7f3e700a5870 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c000c00 tx=0x7f3e7c005b50 comp rx=0 tx=0).stop 2024-09-19T05:06:21.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.538+0000 7f3e93000640 1 -- 172.21.15.43:0/1039840961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e8c1049a0 msgr2=0x7f3e8c19ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:21.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.538+0000 7f3e93000640 1 --2- 172.21.15.43:0/1039840961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e8c1049a0 0x7f3e8c19ab50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3e7400e9f0 tx=0x7f3e7400edf0 comp rx=0 tx=0).stop 2024-09-19T05:06:21.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.538+0000 7f3e93000640 1 -- 172.21.15.43:0/1039840961 shutdown_connections 2024-09-19T05:06:21.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.538+0000 7f3e93000640 1 --2- 172.21.15.43:0/1039840961 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e8c109120 0x7f3e8c19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:21.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.538+0000 7f3e93000640 1 --2- 172.21.15.43:0/1039840961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e8c1049a0 0x7f3e8c19ab50 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:21.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.538+0000 7f3e93000640 1 --2- 172.21.15.43:0/1039840961 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f3e700a33b0 0x7f3e700a5870 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:21.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.538+0000 7f3e93000640 1 --2- 172.21.15.43:0/1039840961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e8c103ff0 0x7f3e8c19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:21.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.538+0000 7f3e93000640 1 -- 172.21.15.43:0/1039840961 >> 172.21.15.43:0/1039840961 conn(0x7f3e8c0ffa90 msgr2=0x7f3e8c100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:21.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.538+0000 7f3e93000640 1 -- 172.21.15.43:0/1039840961 shutdown_connections 2024-09-19T05:06:21.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:21.538+0000 7f3e93000640 1 -- 172.21.15.43:0/1039840961 wait complete. 2024-09-19T05:06:22.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:21 smithi096 ceph-mon[35085]: pgmap v54: 97 pgs: 97 active+clean; 1.0 GiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 166 MiB/s wr, 56 op/s 2024-09-19T05:06:22.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:21 smithi043 ceph-mon[30800]: pgmap v54: 97 pgs: 97 active+clean; 1.0 GiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 166 MiB/s wr, 56 op/s 2024-09-19T05:06:22.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:21 smithi151 ceph-mon[35425]: pgmap v54: 97 pgs: 97 active+clean; 1.0 GiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 166 MiB/s wr, 56 op/s 2024-09-19T05:06:22.306 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T05:06:22.689 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:06:23.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.292+0000 7f198d000640 1 -- 172.21.15.43:0/2910900006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19881079c0 msgr2=0x7f1988107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:23.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.292+0000 7f198d000640 1 --2- 172.21.15.43:0/2910900006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19881079c0 0x7f1988107da0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1978009cd0 tx=0x7f197802a8e0 comp rx=0 tx=0).stop 2024-09-19T05:06:23.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.293+0000 7f198d000640 1 -- 172.21.15.43:0/2910900006 shutdown_connections 2024-09-19T05:06:23.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.293+0000 7f198d000640 1 --2- 172.21.15.43:0/2910900006 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1988102360 0x7f198810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:23.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.293+0000 7f198d000640 1 --2- 172.21.15.43:0/2910900006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19881019c0 0x7f1988101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:23.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.293+0000 7f198d000640 1 --2- 172.21.15.43:0/2910900006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19881079c0 0x7f1988107da0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:23.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.293+0000 7f198d000640 1 -- 172.21.15.43:0/2910900006 >> 172.21.15.43:0/2910900006 conn(0x7f19880fd740 msgr2=0x7f19880ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:23.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.293+0000 7f198d000640 1 -- 172.21.15.43:0/2910900006 shutdown_connections 2024-09-19T05:06:23.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.293+0000 7f198d000640 1 -- 172.21.15.43:0/2910900006 wait complete. 2024-09-19T05:06:23.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.294+0000 7f198d000640 1 Processor -- start 2024-09-19T05:06:23.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.294+0000 7f198d000640 1 -- start start 2024-09-19T05:06:23.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.294+0000 7f198d000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19881019c0 0x7f198819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:23.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.294+0000 7f198d000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1988102360 0x7f198819a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:23.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.294+0000 7f198d000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19881079c0 0x7f198819ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:23.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.294+0000 7f198d000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1988111e10 con 0x7f19881019c0 2024-09-19T05:06:23.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.294+0000 7f198d000640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1988111c90 con 0x7f19881079c0 2024-09-19T05:06:23.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.294+0000 7f198d000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1988111f90 con 0x7f1988102360 2024-09-19T05:06:23.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.295+0000 7f1987400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19881019c0 0x7f198819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:23.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.295+0000 7f1987400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19881019c0 0x7f198819a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41268/0 (socket says 172.21.15.43:41268) 2024-09-19T05:06:23.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.295+0000 7f197ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1988102360 0x7f198819a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:23.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.295+0000 7f1987400640 1 -- 172.21.15.43:0/1383705735 learned_addr learned my addr 172.21.15.43:0/1383705735 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:23.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.295+0000 7f1987e00640 1 --2- 172.21.15.43:0/1383705735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19881079c0 0x7f198819ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:23.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.295+0000 7f197ea00640 1 -- 172.21.15.43:0/1383705735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19881079c0 msgr2=0x7f198819ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:23.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.295+0000 7f197ea00640 1 --2- 172.21.15.43:0/1383705735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19881079c0 0x7f198819ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:23.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.295+0000 7f197ea00640 1 -- 172.21.15.43:0/1383705735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19881019c0 msgr2=0x7f198819a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:23.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.295+0000 7f197ea00640 1 --2- 172.21.15.43:0/1383705735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19881019c0 0x7f198819a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:23.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.295+0000 7f197ea00640 1 -- 172.21.15.43:0/1383705735 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f198819f3f0 con 0x7f1988102360 2024-09-19T05:06:23.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.295+0000 7f197ea00640 1 --2- 172.21.15.43:0/1383705735 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1988102360 0x7f198819a980 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f197000e9f0 tx=0x7f197000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:23.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.296+0000 7f1984c00640 1 -- 172.21.15.43:0/1383705735 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197000cd80 con 0x7f1988102360 2024-09-19T05:06:23.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.296+0000 7f198d000640 1 -- 172.21.15.43:0/1383705735 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f198819f6e0 con 0x7f1988102360 2024-09-19T05:06:23.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.296+0000 7f1984c00640 1 -- 172.21.15.43:0/1383705735 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19700035f0 con 0x7f1988102360 2024-09-19T05:06:23.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.296+0000 7f1984c00640 1 -- 172.21.15.43:0/1383705735 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1970010670 con 0x7f1988102360 2024-09-19T05:06:23.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.296+0000 7f198d000640 1 -- 172.21.15.43:0/1383705735 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19881a6fc0 con 0x7f1988102360 2024-09-19T05:06:23.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.299+0000 7f199124a740 1 Processor -- start 2024-09-19T05:06:23.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.299+0000 7f199124a740 1 -- start start 2024-09-19T05:06:23.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.299+0000 7f199124a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0854a4e40 0x55d0854a5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:23.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.299+0000 7f1984c00640 1 -- 172.21.15.43:0/1383705735 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f19700037a0 con 0x7f1988102360 2024-09-19T05:06:23.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.299+0000 7f199124a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0854a5760 0x55d0854afae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:23.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.299+0000 7f199124a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0854b0020 0x55d0854b2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:23.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.299+0000 7f199124a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d08540b3e0 con 0x55d0854a4e40 2024-09-19T05:06:23.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.299+0000 7f199124a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d0853b2d80 con 0x55d0854b0020 2024-09-19T05:06:23.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.299+0000 7f199124a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d0854073a0 con 0x55d0854a5760 2024-09-19T05:06:23.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.299+0000 7f1987e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0854a4e40 0x55d0854a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:23.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.299+0000 7f1987400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0854a5760 0x55d0854afae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:23.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.299+0000 7f1987e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0854a4e40 0x55d0854a5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41278/0 (socket says 172.21.15.43:41278) 2024-09-19T05:06:23.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.300+0000 7f1987e00640 1 -- 172.21.15.43:0/1172972561 learned_addr learned my addr 172.21.15.43:0/1172972561 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:23.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.300+0000 7f197ea00640 1 --2- 172.21.15.43:0/1172972561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0854b0020 0x55d0854b2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:23.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.300+0000 7f1987e00640 1 -- 172.21.15.43:0/1172972561 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0854a5760 msgr2=0x55d0854afae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:23.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.300+0000 7f1987e00640 1 --2- 172.21.15.43:0/1172972561 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0854a5760 0x55d0854afae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:23.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.300+0000 7f1987e00640 1 -- 172.21.15.43:0/1172972561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0854b0020 msgr2=0x55d0854b2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:23.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.300+0000 7f1987e00640 1 --2- 172.21.15.43:0/1172972561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0854b0020 0x55d0854b2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:23.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.300+0000 7f1987e00640 1 -- 172.21.15.43:0/1172972561 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0853d1ae0 con 0x55d0854a4e40 2024-09-19T05:06:23.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.300+0000 7f197ea00640 1 --2- 172.21.15.43:0/1172972561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0854b0020 0x55d0854b2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:06:23.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.300+0000 7f1987e00640 1 --2- 172.21.15.43:0/1172972561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0854a4e40 0x55d0854a5220 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1980009c20 tx=0x7f1980011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2087b9c4db7948c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:23.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.301+0000 7f197cc00640 1 -- 172.21.15.43:0/1172972561 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198001e070 con 0x55d0854a4e40 2024-09-19T05:06:23.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.301+0000 7f197cc00640 1 -- 172.21.15.43:0/1172972561 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1980019b10 con 0x55d0854a4e40 2024-09-19T05:06:23.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.301+0000 7f197cc00640 1 -- 172.21.15.43:0/1172972561 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198000e9e0 con 0x55d0854a4e40 2024-09-19T05:06:23.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.301+0000 7f1984c00640 1 --2- 172.21.15.43:0/1383705735 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f196c0a33b0 0x7f196c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:23.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.301+0000 7f199124a740 1 -- 172.21.15.43:0/1172972561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0854a4e40 msgr2=0x55d0854a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:23.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.301+0000 7f199124a740 1 --2- 172.21.15.43:0/1172972561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0854a4e40 0x55d0854a5220 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1980009c20 tx=0x7f1980011810 comp rx=0 tx=0).stop 2024-09-19T05:06:23.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.301+0000 7f1984c00640 1 -- 172.21.15.43:0/1383705735 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f19700cfff0 con 0x7f1988102360 2024-09-19T05:06:23.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.301+0000 7f1987400640 1 --2- 172.21.15.43:0/1383705735 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f196c0a33b0 0x7f196c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:23.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.302+0000 7f199124a740 1 -- 172.21.15.43:0/1172972561 shutdown_connections 2024-09-19T05:06:23.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.302+0000 7f199124a740 1 --2- 172.21.15.43:0/1172972561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0854b0020 0x55d0854b2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:23.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.302+0000 7f199124a740 1 --2- 172.21.15.43:0/1172972561 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0854a5760 0x55d0854afae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:23.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.302+0000 7f199124a740 1 --2- 172.21.15.43:0/1172972561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0854a4e40 0x55d0854a5220 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:23.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.302+0000 7f199124a740 1 -- 172.21.15.43:0/1172972561 >> 172.21.15.43:0/1172972561 conn(0x55d0854a13d0 msgr2=0x55d0854a17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:23.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.302+0000 7f199124a740 1 -- 172.21.15.43:0/1172972561 shutdown_connections 2024-09-19T05:06:23.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.302+0000 7f199124a740 1 -- 172.21.15.43:0/1172972561 wait complete. 2024-09-19T05:06:23.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.302+0000 7f1987400640 1 --2- 172.21.15.43:0/1383705735 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f196c0a33b0 0x7f196c0a5870 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1978000f80 tx=0x7f197803d040 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:23.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.303+0000 7f199124a740 1 -- start start 2024-09-19T05:06:23.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.303+0000 7f199124a740 1 Processor -- start 2024-09-19T05:06:23.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.303+0000 7f199124a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0854a4e40 0x55d0854acf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:23.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.303+0000 7f199124a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0854a5760 0x55d0854ad450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:23.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.303+0000 7f199124a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0854b0020 0x55d0854ad990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:23.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.303+0000 7f199124a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d0854b2bf0 con 0x55d0854b0020 2024-09-19T05:06:23.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.303+0000 7f199124a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d0853c60f0 con 0x55d0854a5760 2024-09-19T05:06:23.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.303+0000 7f199124a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d0854b2d70 con 0x55d0854a4e40 2024-09-19T05:06:23.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.303+0000 7f199124a740 10 client.0 _ll_register_callbacks cb 0x55d0853adbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:23.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.304+0000 7f1987e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0854b0020 0x55d0854ad990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:23.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.304+0000 7f1987e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0854b0020 0x55d0854ad990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41288/0 (socket says 172.21.15.43:41288) 2024-09-19T05:06:23.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.304+0000 7f1987e00640 1 -- 172.21.15.43:0/2795731128 learned_addr learned my addr 172.21.15.43:0/2795731128 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:23.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.304+0000 7f197ea00640 1 --2- 172.21.15.43:0/2795731128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0854a5760 0x55d0854ad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:23.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.304+0000 7f1987400640 1 --2- 172.21.15.43:0/2795731128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0854a4e40 0x55d0854acf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:23.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.304+0000 7f1987e00640 1 -- 172.21.15.43:0/2795731128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0854a4e40 msgr2=0x55d0854acf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:23.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.304+0000 7f1987e00640 1 --2- 172.21.15.43:0/2795731128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0854a4e40 0x55d0854acf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:23.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.304+0000 7f1987e00640 1 -- 172.21.15.43:0/2795731128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0854a5760 msgr2=0x55d0854ad450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:23.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.304+0000 7f1987e00640 1 --2- 172.21.15.43:0/2795731128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0854a5760 0x55d0854ad450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:23.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.304+0000 7f1987e00640 1 -- 172.21.15.43:0/2795731128 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d08517c3c0 con 0x55d0854b0020 2024-09-19T05:06:23.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.304+0000 7f197ea00640 1 --2- 172.21.15.43:0/2795731128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0854a5760 0x55d0854ad450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:23.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.304+0000 7f1987e00640 1 --2- 172.21.15.43:0/2795731128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0854b0020 0x55d0854ad990 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f198000e500 tx=0x7f1980019910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab3211dd9ba03f52 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:23.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.304+0000 7f1975600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:06:23.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.305+0000 7f1975600640 1 -- 172.21.15.43:0/2795731128 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1980027070 con 0x55d0854b0020 2024-09-19T05:06:23.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.305+0000 7f199124a740 1 -- 172.21.15.43:0/2795731128 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1980019b10 con 0x55d0854b0020 2024-09-19T05:06:23.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.305+0000 7f1975600640 1 -- 172.21.15.43:0/2795731128 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f1980004020 con 0x55d0854b0020 2024-09-19T05:06:23.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.305+0000 7f1975600640 1 -- 172.21.15.43:0/2795731128 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1980032050 con 0x55d0854b0020 2024-09-19T05:06:23.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.305+0000 7f199124a740 10 client.15555 fetch_fsmap learned FSMap version 29 2024-09-19T05:06:23.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.305+0000 7f199124a740 1 -- 172.21.15.43:0/2795731128 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d0854bec90 con 0x55d0854b0020 2024-09-19T05:06:23.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.305+0000 7f1975600640 1 -- 172.21.15.43:0/2795731128 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1980019b10 con 0x55d0854b0020 2024-09-19T05:06:23.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.306+0000 7f1975600640 1 -- 172.21.15.43:0/2795731128 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f19800047a0 con 0x55d0854b0020 2024-09-19T05:06:23.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.306+0000 7f199124a740 10 client.15555 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:06:23.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.306+0000 7f199124a740 10 client.15555 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T05:06:23.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.306+0000 7f199124a740 20 client.15555 populate_metadata read hostname 'smithi043' 2024-09-19T05:06:23.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.306+0000 7f199124a740 1 --2- 172.21.15.43:0/2795731128 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55d0854c00a0 0x55d0854c2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:23.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.306+0000 7f199124a740 4 client.15555 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:06:23.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.306+0000 7f199124a740 1 -- 172.21.15.43:0/2795731128 --> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d085418950 con 0x55d0854c00a0 2024-09-19T05:06:23.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.306+0000 7f1987400640 1 --2- 172.21.15.43:0/2795731128 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55d0854c00a0 0x55d0854c2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:23.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.307+0000 7f1987400640 1 --2- 172.21.15.43:0/2795731128 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55d0854c00a0 0x55d0854c2490 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9bbcd9ca280bff54 server_cookie=ff312c54edea2778 in_seq=0 out_seq=0 2024-09-19T05:06:23.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.307+0000 7f1975600640 10 client.15555 ms_handle_connect on v2:172.21.15.151:6838/3794104313 2024-09-19T05:06:23.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.307+0000 7f1975600640 1 -- 172.21.15.43:0/2795731128 <== mds.2 v2:172.21.15.151:6838/3794104313 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d085418950 con 0x55d0854c00a0 2024-09-19T05:06:23.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.307+0000 7f1975600640 10 client.15555 handle_command_reply: tid=0 2024-09-19T05:06:23.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.308+0000 7f199124a740 1 client.15555 shutdown 2024-09-19T05:06:23.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.308+0000 7f199124a740 1 -- 172.21.15.43:0/2795731128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0854b0020 msgr2=0x55d0854ad990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:23.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.308+0000 7f199124a740 1 --2- 172.21.15.43:0/2795731128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0854b0020 0x55d0854ad990 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f198000e500 tx=0x7f1980019910 comp rx=0 tx=0).stop 2024-09-19T05:06:23.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.308+0000 7f199124a740 1 -- 172.21.15.43:0/2795731128 shutdown_connections 2024-09-19T05:06:23.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.308+0000 7f199124a740 1 --2- 172.21.15.43:0/2795731128 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55d0854c00a0 0x55d0854c2490 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:23.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.308+0000 7f199124a740 1 --2- 172.21.15.43:0/2795731128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0854b0020 0x55d0854ad990 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:23.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.308+0000 7f199124a740 1 --2- 172.21.15.43:0/2795731128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0854a5760 0x55d0854ad450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:23.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.308+0000 7f199124a740 1 --2- 172.21.15.43:0/2795731128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0854a4e40 0x55d0854acf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:23.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.308+0000 7f199124a740 1 -- 172.21.15.43:0/2795731128 >> 172.21.15.43:0/2795731128 conn(0x55d0854a13d0 msgr2=0x55d0854a04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:23.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.308+0000 7f1975600640 0 client.15555 ms_handle_reset on v2:172.21.15.151:6838/3794104313 2024-09-19T05:06:23.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.308+0000 7f199124a740 1 -- 172.21.15.43:0/2795731128 shutdown_connections 2024-09-19T05:06:23.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.308+0000 7f199124a740 1 -- 172.21.15.43:0/2795731128 wait complete. 2024-09-19T05:06:23.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.308+0000 7f199124a740 20 client.15555 trim_cache size 0 max 16384 2024-09-19T05:06:23.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.322+0000 7f199124a740 1 Processor -- start 2024-09-19T05:06:23.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.322+0000 7f199124a740 1 -- start start 2024-09-19T05:06:23.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.322+0000 7f199124a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0854a4e40 0x55d0854a6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:23.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.322+0000 7f199124a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0854a5760 0x55d0854c7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:23.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.322+0000 7f199124a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0854b0020 0x55d0854c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:23.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.322+0000 7f199124a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f19881af8b0 con 0x55d0854b0020 2024-09-19T05:06:23.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.322+0000 7f199124a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f19881b13f0 con 0x55d0854a4e40 2024-09-19T05:06:23.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.322+0000 7f199124a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f19881aff80 con 0x55d0854a5760 2024-09-19T05:06:23.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.323+0000 7f1987e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0854b0020 0x55d0854c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:23.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.323+0000 7f1987e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0854b0020 0x55d0854c80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41302/0 (socket says 172.21.15.43:41302) 2024-09-19T05:06:23.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.323+0000 7f1987e00640 1 -- 172.21.15.43:0/28917895 learned_addr learned my addr 172.21.15.43:0/28917895 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:23.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.323+0000 7f197ea00640 1 --2- 172.21.15.43:0/28917895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0854a5760 0x55d0854c7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:23.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.323+0000 7f1987400640 1 --2- 172.21.15.43:0/28917895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0854a4e40 0x55d0854a6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:23.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.323+0000 7f1987e00640 1 -- 172.21.15.43:0/28917895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0854a5760 msgr2=0x55d0854c7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:23.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.323+0000 7f1987e00640 1 --2- 172.21.15.43:0/28917895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0854a5760 0x55d0854c7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:23.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.323+0000 7f1987e00640 1 -- 172.21.15.43:0/28917895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0854a4e40 msgr2=0x55d0854a6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:23.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.323+0000 7f1987e00640 1 --2- 172.21.15.43:0/28917895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0854a4e40 0x55d0854a6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:23.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.323+0000 7f1987e00640 1 -- 172.21.15.43:0/28917895 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0854a5fe0 con 0x55d0854b0020 2024-09-19T05:06:23.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.324+0000 7f1987e00640 1 --2- 172.21.15.43:0/28917895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0854b0020 0x55d0854c80f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1980004ce0 tx=0x7f19800096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd1d1e0450ca6e19 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:23.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.324+0000 7f1977e00640 1 -- 172.21.15.43:0/28917895 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1980027070 con 0x55d0854b0020 2024-09-19T05:06:23.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.324+0000 7f1977e00640 1 -- 172.21.15.43:0/28917895 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1980005850 con 0x55d0854b0020 2024-09-19T05:06:23.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.324+0000 7f1977e00640 1 -- 172.21.15.43:0/28917895 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198001e040 con 0x55d0854b0020 2024-09-19T05:06:23.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.324+0000 7f199124a740 1 -- 172.21.15.43:0/28917895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0854b0020 msgr2=0x55d0854c80f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:23.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.324+0000 7f199124a740 1 --2- 172.21.15.43:0/28917895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0854b0020 0x55d0854c80f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1980004ce0 tx=0x7f19800096a0 comp rx=0 tx=0).stop 2024-09-19T05:06:23.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.325+0000 7f199124a740 1 -- 172.21.15.43:0/28917895 shutdown_connections 2024-09-19T05:06:23.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.325+0000 7f199124a740 1 --2- 172.21.15.43:0/28917895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0854b0020 0x55d0854c80f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:23.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.325+0000 7f199124a740 1 --2- 172.21.15.43:0/28917895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0854a5760 0x55d0854c7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:23.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.325+0000 7f199124a740 1 --2- 172.21.15.43:0/28917895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0854a4e40 0x55d0854a6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:23.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.325+0000 7f199124a740 1 -- 172.21.15.43:0/28917895 >> 172.21.15.43:0/28917895 conn(0x55d0854a13d0 msgr2=0x55d0854a6890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:23.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.325+0000 7f199124a740 1 -- 172.21.15.43:0/28917895 shutdown_connections 2024-09-19T05:06:23.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.325+0000 7f199124a740 1 -- 172.21.15.43:0/28917895 wait complete. 2024-09-19T05:06:23.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.325+0000 7f199124a740 1 -- start start 2024-09-19T05:06:23.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.325+0000 7f199124a740 1 Processor -- start 2024-09-19T05:06:23.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.326+0000 7f199124a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0854a4e40 0x55d0854b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:23.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.326+0000 7f199124a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0854a5760 0x55d0854b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:23.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.326+0000 7f199124a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0854b0020 0x55d0854b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:23.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.326+0000 7f199124a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f19881b0d20 con 0x55d0854a4e40 2024-09-19T05:06:23.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.326+0000 7f199124a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f19881b0650 con 0x55d0854b0020 2024-09-19T05:06:23.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.326+0000 7f199124a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f19881ad890 con 0x55d0854a5760 2024-09-19T05:06:23.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.326+0000 7f199124a740 10 client.0 _ll_register_callbacks cb 0x7f198001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:23.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.326+0000 7f1987400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0854a4e40 0x55d0854b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:23.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.326+0000 7f1987e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0854b0020 0x55d0854b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:23.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.326+0000 7f1987400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0854a4e40 0x55d0854b2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41316/0 (socket says 172.21.15.43:41316) 2024-09-19T05:06:23.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.326+0000 7f1987400640 1 -- 172.21.15.43:0/3464045353 learned_addr learned my addr 172.21.15.43:0/3464045353 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:23.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.326+0000 7f197ea00640 1 --2- 172.21.15.43:0/3464045353 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0854a5760 0x55d0854b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:23.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.326+0000 7f1987e00640 1 -- 172.21.15.43:0/3464045353 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0854a5760 msgr2=0x55d0854b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:23.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.326+0000 7f1987e00640 1 --2- 172.21.15.43:0/3464045353 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0854a5760 0x55d0854b34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:23.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.326+0000 7f1987e00640 1 -- 172.21.15.43:0/3464045353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0854a4e40 msgr2=0x55d0854b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:23.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.326+0000 7f1987e00640 1 --2- 172.21.15.43:0/3464045353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0854a4e40 0x55d0854b2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:23.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.326+0000 7f1987e00640 1 -- 172.21.15.43:0/3464045353 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d085528bc0 con 0x55d0854b0020 2024-09-19T05:06:23.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.327+0000 7f1987e00640 1 --2- 172.21.15.43:0/3464045353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0854b0020 0x55d0854b3a00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1980009250 tx=0x7f1980009420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78f6c4473dc7177e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:23.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.327+0000 7f1975600640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:06:23.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.327+0000 7f1975600640 1 -- 172.21.15.43:0/3464045353 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198001e070 con 0x55d0854b0020 2024-09-19T05:06:23.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.327+0000 7f199124a740 1 -- 172.21.15.43:0/3464045353 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1980005850 con 0x55d0854b0020 2024-09-19T05:06:23.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.327+0000 7f1975600640 1 -- 172.21.15.43:0/3464045353 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f198000e940 con 0x55d0854b0020 2024-09-19T05:06:23.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.327+0000 7f1975600640 1 -- 172.21.15.43:0/3464045353 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1980045050 con 0x55d0854b0020 2024-09-19T05:06:23.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.327+0000 7f199124a740 10 client.25550 fetch_fsmap learned FSMap version 29 2024-09-19T05:06:23.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.327+0000 7f199124a740 1 -- 172.21.15.43:0/3464045353 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d085529ec0 con 0x55d0854b0020 2024-09-19T05:06:23.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.328+0000 7f1975600640 1 -- 172.21.15.43:0/3464045353 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1980005850 con 0x55d0854b0020 2024-09-19T05:06:23.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.328+0000 7f1975600640 1 -- 172.21.15.43:0/3464045353 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1980003b90 con 0x55d0854b0020 2024-09-19T05:06:23.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.328+0000 7f199124a740 10 client.25550 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:06:23.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.328+0000 7f199124a740 10 client.25550 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T05:06:23.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.328+0000 7f199124a740 20 client.25550 populate_metadata read hostname 'smithi043' 2024-09-19T05:06:23.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.328+0000 7f199124a740 1 --2- 172.21.15.43:0/3464045353 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55d08552a4f0 0x55d08552c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:23.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.328+0000 7f199124a740 4 client.25550 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:06:23.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.328+0000 7f199124a740 1 -- 172.21.15.43:0/3464045353 --> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d085412e60 con 0x55d08552a4f0 2024-09-19T05:06:23.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.328+0000 7f1987400640 1 --2- 172.21.15.43:0/3464045353 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55d08552a4f0 0x55d08552c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:23.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.329+0000 7f1987400640 1 --2- 172.21.15.43:0/3464045353 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55d08552a4f0 0x55d08552c910 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8abe974322815c3a server_cookie=a0dd822059a0af72 in_seq=0 out_seq=0 2024-09-19T05:06:23.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.329+0000 7f1975600640 10 client.25550 ms_handle_connect on v2:172.21.15.151:6838/3794104313 2024-09-19T05:06:23.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.329+0000 7f1975600640 1 -- 172.21.15.43:0/3464045353 <== mds.2 v2:172.21.15.151:6838/3794104313 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d085412e60 con 0x55d08552a4f0 2024-09-19T05:06:23.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.329+0000 7f1975600640 10 client.25550 handle_command_reply: tid=0 2024-09-19T05:06:23.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.329+0000 7f199124a740 1 client.25550 shutdown 2024-09-19T05:06:23.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.329+0000 7f199124a740 1 -- 172.21.15.43:0/3464045353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0854b0020 msgr2=0x55d0854b3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:23.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.329+0000 7f199124a740 1 --2- 172.21.15.43:0/3464045353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0854b0020 0x55d0854b3a00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1980009250 tx=0x7f1980009420 comp rx=0 tx=0).stop 2024-09-19T05:06:23.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.329+0000 7f199124a740 1 -- 172.21.15.43:0/3464045353 shutdown_connections 2024-09-19T05:06:23.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.329+0000 7f199124a740 1 --2- 172.21.15.43:0/3464045353 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55d08552a4f0 0x55d08552c910 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:23.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.329+0000 7f199124a740 1 --2- 172.21.15.43:0/3464045353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0854b0020 0x55d0854b3a00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:23.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.329+0000 7f199124a740 1 --2- 172.21.15.43:0/3464045353 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0854a5760 0x55d0854b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:23.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.329+0000 7f199124a740 1 --2- 172.21.15.43:0/3464045353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0854a4e40 0x55d0854b2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:23.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.329+0000 7f199124a740 1 -- 172.21.15.43:0/3464045353 >> 172.21.15.43:0/3464045353 conn(0x55d0854a13d0 msgr2=0x55d0854ae770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:23.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.329+0000 7f1975600640 0 client.25550 ms_handle_reset on v2:172.21.15.151:6838/3794104313 2024-09-19T05:06:23.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.330+0000 7f199124a740 1 -- 172.21.15.43:0/3464045353 shutdown_connections 2024-09-19T05:06:23.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.330+0000 7f199124a740 1 -- 172.21.15.43:0/3464045353 wait complete. 2024-09-19T05:06:23.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.330+0000 7f199124a740 20 client.25550 trim_cache size 0 max 16384 2024-09-19T05:06:23.334 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:06:23.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.330+0000 7f198d000640 1 -- 172.21.15.43:0/1383705735 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f196c0a33b0 msgr2=0x7f196c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:23.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.330+0000 7f198d000640 1 --2- 172.21.15.43:0/1383705735 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f196c0a33b0 0x7f196c0a5870 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1978000f80 tx=0x7f197803d040 comp rx=0 tx=0).stop 2024-09-19T05:06:23.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.330+0000 7f198d000640 1 -- 172.21.15.43:0/1383705735 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1988102360 msgr2=0x7f198819a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:23.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.330+0000 7f198d000640 1 --2- 172.21.15.43:0/1383705735 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1988102360 0x7f198819a980 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f197000e9f0 tx=0x7f197000eeb0 comp rx=0 tx=0).stop 2024-09-19T05:06:23.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.330+0000 7f198d000640 1 -- 172.21.15.43:0/1383705735 shutdown_connections 2024-09-19T05:06:23.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.330+0000 7f198d000640 1 --2- 172.21.15.43:0/1383705735 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f196c0a33b0 0x7f196c0a5870 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:23.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.330+0000 7f198d000640 1 --2- 172.21.15.43:0/1383705735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19881079c0 0x7f198819ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:23.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.330+0000 7f198d000640 1 --2- 172.21.15.43:0/1383705735 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1988102360 0x7f198819a980 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:23.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.330+0000 7f198d000640 1 --2- 172.21.15.43:0/1383705735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19881019c0 0x7f198819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:23.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.330+0000 7f198d000640 1 -- 172.21.15.43:0/1383705735 >> 172.21.15.43:0/1383705735 conn(0x7f19880fd740 msgr2=0x7f198810d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:23.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.331+0000 7f198d000640 1 -- 172.21.15.43:0/1383705735 shutdown_connections 2024-09-19T05:06:23.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:23.331+0000 7f198d000640 1 -- 172.21.15.43:0/1383705735 wait complete. 2024-09-19T05:06:24.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:23 smithi043 ceph-mon[30800]: pgmap v55: 97 pgs: 97 active+clean; 1.0 GiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 MiB/s wr, 30 op/s 2024-09-19T05:06:24.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:23 smithi151 ceph-mon[35425]: pgmap v55: 97 pgs: 97 active+clean; 1.0 GiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 MiB/s wr, 30 op/s 2024-09-19T05:06:24.198 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T05:06:24.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:23 smithi096 ceph-mon[35085]: pgmap v55: 97 pgs: 97 active+clean; 1.0 GiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 MiB/s wr, 30 op/s 2024-09-19T05:06:24.585 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:06:25.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.236+0000 7f610fc00640 1 -- 172.21.15.43:0/16091360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6108109120 msgr2=0x7f6108110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:25.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.236+0000 7f610fc00640 1 --2- 172.21.15.43:0/16091360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6108109120 0x7f6108110ca0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6100009ca0 tx=0x7f610002a8b0 comp rx=0 tx=0).stop 2024-09-19T05:06:25.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.236+0000 7f610fc00640 1 -- 172.21.15.43:0/16091360 shutdown_connections 2024-09-19T05:06:25.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.236+0000 7f610fc00640 1 --2- 172.21.15.43:0/16091360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6108109120 0x7f6108110ca0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:25.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.236+0000 7f610fc00640 1 --2- 172.21.15.43:0/16091360 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61081049a0 0x7f61081089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:25.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.236+0000 7f610fc00640 1 --2- 172.21.15.43:0/16091360 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6108103ff0 0x7f61081043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:25.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.236+0000 7f610fc00640 1 -- 172.21.15.43:0/16091360 >> 172.21.15.43:0/16091360 conn(0x7f61080ffa90 msgr2=0x7f6108101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:25.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.236+0000 7f610fc00640 1 -- 172.21.15.43:0/16091360 shutdown_connections 2024-09-19T05:06:25.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.236+0000 7f610fc00640 1 -- 172.21.15.43:0/16091360 wait complete. 2024-09-19T05:06:25.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.238+0000 7f610fc00640 1 Processor -- start 2024-09-19T05:06:25.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.238+0000 7f610fc00640 1 -- start start 2024-09-19T05:06:25.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.239+0000 7f610fc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6108103ff0 0x7f610819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:25.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.239+0000 7f610fc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61081049a0 0x7f610819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:25.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.239+0000 7f610fc00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6108109120 0x7f610819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:25.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.239+0000 7f610fc00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6108114180 con 0x7f6108103ff0 2024-09-19T05:06:25.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.239+0000 7f610fc00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6108114000 con 0x7f6108109120 2024-09-19T05:06:25.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.239+0000 7f610fc00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6108114300 con 0x7f61081049a0 2024-09-19T05:06:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.239+0000 7f610d200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6108103ff0 0x7f610819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.239+0000 7f610dc00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6108109120 0x7f610819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.239+0000 7f610dc00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6108109120 0x7f610819eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:38816/0 (socket says 172.21.15.43:38816) 2024-09-19T05:06:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.239+0000 7f610d200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6108103ff0 0x7f610819a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41332/0 (socket says 172.21.15.43:41332) 2024-09-19T05:06:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.239+0000 7f610dc00640 1 -- 172.21.15.43:0/1602335532 learned_addr learned my addr 172.21.15.43:0/1602335532 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.239+0000 7f6107e00640 1 --2- 172.21.15.43:0/1602335532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61081049a0 0x7f610819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.240+0000 7f610d200640 1 -- 172.21.15.43:0/1602335532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61081049a0 msgr2=0x7f610819ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.240+0000 7f610d200640 1 --2- 172.21.15.43:0/1602335532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61081049a0 0x7f610819ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.240+0000 7f610d200640 1 -- 172.21.15.43:0/1602335532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6108109120 msgr2=0x7f610819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.240+0000 7f610d200640 1 --2- 172.21.15.43:0/1602335532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6108109120 0x7f610819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.240+0000 7f610d200640 1 -- 172.21.15.43:0/1602335532 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f610819f5d0 con 0x7f6108103ff0 2024-09-19T05:06:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.240+0000 7f610dc00640 1 --2- 172.21.15.43:0/1602335532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6108109120 0x7f610819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:25.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.240+0000 7f6107e00640 1 --2- 172.21.15.43:0/1602335532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61081049a0 0x7f610819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:25.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.240+0000 7f610d200640 1 --2- 172.21.15.43:0/1602335532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6108103ff0 0x7f610819a620 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f60fc00e9f0 tx=0x7f60fc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:25.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.240+0000 7f6105600640 1 -- 172.21.15.43:0/1602335532 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60fc00ccc0 con 0x7f6108103ff0 2024-09-19T05:06:25.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.240+0000 7f6105600640 1 -- 172.21.15.43:0/1602335532 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60fc010910 con 0x7f6108103ff0 2024-09-19T05:06:25.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.240+0000 7f6105600640 1 -- 172.21.15.43:0/1602335532 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60fc018960 con 0x7f6108103ff0 2024-09-19T05:06:25.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.240+0000 7f610fc00640 1 -- 172.21.15.43:0/1602335532 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f610819f8c0 con 0x7f6108103ff0 2024-09-19T05:06:25.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.241+0000 7f610fc00640 1 -- 172.21.15.43:0/1602335532 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61081a71a0 con 0x7f6108103ff0 2024-09-19T05:06:25.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.244+0000 7f61126c2740 1 Processor -- start 2024-09-19T05:06:25.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.244+0000 7f61126c2740 1 -- start start 2024-09-19T05:06:25.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.244+0000 7f61126c2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5559ca307c70 0x5559ca308050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:25.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.244+0000 7f61126c2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559ca308590 0x5559ca312910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:25.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.244+0000 7f6105600640 1 -- 172.21.15.43:0/1602335532 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f60fc010ac0 con 0x7f6108103ff0 2024-09-19T05:06:25.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.244+0000 7f61126c2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559ca312e50 0x5559ca315240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:25.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.244+0000 7f61126c2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5559ca26e3e0 con 0x5559ca307c70 2024-09-19T05:06:25.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.244+0000 7f61126c2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5559ca215d80 con 0x5559ca312e50 2024-09-19T05:06:25.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.244+0000 7f61126c2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5559ca26a3a0 con 0x5559ca308590 2024-09-19T05:06:25.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.245+0000 7f610dc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5559ca307c70 0x5559ca308050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:25.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.245+0000 7f610d200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559ca308590 0x5559ca312910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:25.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.245+0000 7f610dc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5559ca307c70 0x5559ca308050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41348/0 (socket says 172.21.15.43:41348) 2024-09-19T05:06:25.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.245+0000 7f610dc00640 1 -- 172.21.15.43:0/3211700690 learned_addr learned my addr 172.21.15.43:0/3211700690 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:25.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.245+0000 7f6107e00640 1 --2- 172.21.15.43:0/3211700690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559ca312e50 0x5559ca315240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:25.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.245+0000 7f6107e00640 1 -- 172.21.15.43:0/3211700690 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559ca308590 msgr2=0x5559ca312910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:25.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.245+0000 7f6107e00640 1 --2- 172.21.15.43:0/3211700690 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559ca308590 0x5559ca312910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:25.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.245+0000 7f6107e00640 1 -- 172.21.15.43:0/3211700690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5559ca307c70 msgr2=0x5559ca308050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:25.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.245+0000 7f6107e00640 1 --2- 172.21.15.43:0/3211700690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5559ca307c70 0x5559ca308050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:25.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.245+0000 7f6107e00640 1 -- 172.21.15.43:0/3211700690 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559ca234ae0 con 0x5559ca312e50 2024-09-19T05:06:25.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.246+0000 7f6107e00640 1 --2- 172.21.15.43:0/3211700690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559ca312e50 0x5559ca315240 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f60f400bf30 tx=0x7f60f4010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbdaa3ee4f198e63 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:25.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.246+0000 7f60fa000640 1 -- 172.21.15.43:0/3211700690 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f40117c0 con 0x5559ca312e50 2024-09-19T05:06:25.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.246+0000 7f60fa000640 1 -- 172.21.15.43:0/3211700690 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60f4011e40 con 0x5559ca312e50 2024-09-19T05:06:25.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.246+0000 7f60fa000640 1 -- 172.21.15.43:0/3211700690 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f401a380 con 0x5559ca312e50 2024-09-19T05:06:25.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.246+0000 7f6105600640 1 --2- 172.21.15.43:0/1602335532 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f60f00a33b0 0x7f60f00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:25.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.246+0000 7f6105600640 1 -- 172.21.15.43:0/1602335532 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f60fc0d00f0 con 0x7f6108103ff0 2024-09-19T05:06:25.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.246+0000 7f610dc00640 1 --2- 172.21.15.43:0/1602335532 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f60f00a33b0 0x7f60f00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:25.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.247+0000 7f61126c2740 1 -- 172.21.15.43:0/3211700690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559ca312e50 msgr2=0x5559ca315240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:25.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.247+0000 7f61126c2740 1 --2- 172.21.15.43:0/3211700690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559ca312e50 0x5559ca315240 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f60f400bf30 tx=0x7f60f4010860 comp rx=0 tx=0).stop 2024-09-19T05:06:25.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.247+0000 7f61126c2740 1 -- 172.21.15.43:0/3211700690 shutdown_connections 2024-09-19T05:06:25.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.247+0000 7f61126c2740 1 --2- 172.21.15.43:0/3211700690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559ca312e50 0x5559ca315240 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:25.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.247+0000 7f61126c2740 1 --2- 172.21.15.43:0/3211700690 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559ca308590 0x5559ca312910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:25.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.247+0000 7f61126c2740 1 --2- 172.21.15.43:0/3211700690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5559ca307c70 0x5559ca308050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:25.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.247+0000 7f61126c2740 1 -- 172.21.15.43:0/3211700690 >> 172.21.15.43:0/3211700690 conn(0x5559ca304200 msgr2=0x5559ca3045e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:25.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.247+0000 7f610dc00640 1 --2- 172.21.15.43:0/1602335532 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f60f00a33b0 0x7f60f00a5870 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f61000308d0 tx=0x7f6100005d80 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:25.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.247+0000 7f61126c2740 1 -- 172.21.15.43:0/3211700690 shutdown_connections 2024-09-19T05:06:25.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.247+0000 7f61126c2740 1 -- 172.21.15.43:0/3211700690 wait complete. 2024-09-19T05:06:25.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.248+0000 7f61126c2740 1 -- start start 2024-09-19T05:06:25.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.248+0000 7f61126c2740 1 Processor -- start 2024-09-19T05:06:25.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.248+0000 7f61126c2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5559ca307c70 0x5559ca30fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:25.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.248+0000 7f61126c2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559ca308590 0x5559ca310250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:25.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.248+0000 7f61126c2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559ca312e50 0x5559ca310790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:25.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.248+0000 7f61126c2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5559ca315a20 con 0x5559ca307c70 2024-09-19T05:06:25.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.248+0000 7f61126c2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5559ca2290f0 con 0x5559ca308590 2024-09-19T05:06:25.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.248+0000 7f61126c2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5559ca315ba0 con 0x5559ca312e50 2024-09-19T05:06:25.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.249+0000 7f61126c2740 10 client.0 _ll_register_callbacks cb 0x5559ca210bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:25.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.249+0000 7f610d200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5559ca307c70 0x5559ca30fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:25.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.249+0000 7f6107e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559ca308590 0x5559ca310250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:25.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.249+0000 7f610d200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5559ca307c70 0x5559ca30fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41360/0 (socket says 172.21.15.43:41360) 2024-09-19T05:06:25.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.249+0000 7f610d200640 1 -- 172.21.15.43:0/178211134 learned_addr learned my addr 172.21.15.43:0/178211134 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:25.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.249+0000 7f610dc00640 1 --2- 172.21.15.43:0/178211134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559ca312e50 0x5559ca310790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:25.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.249+0000 7f610d200640 1 -- 172.21.15.43:0/178211134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559ca312e50 msgr2=0x5559ca310790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:25.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.249+0000 7f610d200640 1 --2- 172.21.15.43:0/178211134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559ca312e50 0x5559ca310790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:25.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.249+0000 7f610d200640 1 -- 172.21.15.43:0/178211134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559ca308590 msgr2=0x5559ca310250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:25.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.249+0000 7f610d200640 1 --2- 172.21.15.43:0/178211134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559ca308590 0x5559ca310250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:25.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.249+0000 7f610d200640 1 -- 172.21.15.43:0/178211134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559c9fdf3c0 con 0x5559ca307c70 2024-09-19T05:06:25.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.249+0000 7f610d200640 1 --2- 172.21.15.43:0/178211134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5559ca307c70 0x5559ca30fd10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f60fc00e9c0 tx=0x7f60fc00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a80ba17606fec7ac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:25.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.249+0000 7f60eea00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:06:25.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.250+0000 7f60eea00640 1 -- 172.21.15.43:0/178211134 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60fc09bf70 con 0x5559ca307c70 2024-09-19T05:06:25.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.250+0000 7f60eea00640 1 -- 172.21.15.43:0/178211134 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f60fc0d20d0 con 0x5559ca307c70 2024-09-19T05:06:25.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.250+0000 7f61126c2740 1 -- 172.21.15.43:0/178211134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5559ca27b950 con 0x5559ca307c70 2024-09-19T05:06:25.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.250+0000 7f60eea00640 1 -- 172.21.15.43:0/178211134 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f60fc09bd90 con 0x5559ca307c70 2024-09-19T05:06:25.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.250+0000 7f61126c2740 10 client.15573 fetch_fsmap learned FSMap version 29 2024-09-19T05:06:25.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.250+0000 7f61126c2740 1 -- 172.21.15.43:0/178211134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5559ca321a30 con 0x5559ca307c70 2024-09-19T05:06:25.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.250+0000 7f60eea00640 1 -- 172.21.15.43:0/178211134 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5559ca27b950 con 0x5559ca307c70 2024-09-19T05:06:25.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.251+0000 7f60eea00640 1 -- 172.21.15.43:0/178211134 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f60fc09b880 con 0x5559ca307c70 2024-09-19T05:06:25.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.251+0000 7f61126c2740 10 client.15573 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:06:25.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.251+0000 7f61126c2740 10 client.15573 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-19T05:06:25.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.251+0000 7f61126c2740 20 client.15573 populate_metadata read hostname 'smithi043' 2024-09-19T05:06:25.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.251+0000 7f61126c2740 1 --2- 172.21.15.43:0/178211134 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5559ca322ef0 0x5559ca325300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:25.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.251+0000 7f61126c2740 4 client.15573 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:06:25.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.251+0000 7f61126c2740 1 -- 172.21.15.43:0/178211134 --> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559ca275e60 con 0x5559ca322ef0 2024-09-19T05:06:25.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.251+0000 7f6107e00640 1 --2- 172.21.15.43:0/178211134 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5559ca322ef0 0x5559ca325300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:25.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.252+0000 7f6107e00640 1 --2- 172.21.15.43:0/178211134 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5559ca322ef0 0x5559ca325300 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cac1c25dc27b67fe server_cookie=9fef9ae9d78af53d in_seq=0 out_seq=0 2024-09-19T05:06:25.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.252+0000 7f60eea00640 10 client.15573 ms_handle_connect on v2:172.21.15.43:6838/3404555030 2024-09-19T05:06:25.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.252+0000 7f60eea00640 1 -- 172.21.15.43:0/178211134 <== mds.1 v2:172.21.15.43:6838/3404555030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5559ca275e60 con 0x5559ca322ef0 2024-09-19T05:06:25.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.252+0000 7f60eea00640 10 client.15573 handle_command_reply: tid=0 2024-09-19T05:06:25.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.253+0000 7f61126c2740 1 client.15573 shutdown 2024-09-19T05:06:25.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.253+0000 7f61126c2740 1 -- 172.21.15.43:0/178211134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5559ca307c70 msgr2=0x5559ca30fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:25.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.253+0000 7f61126c2740 1 --2- 172.21.15.43:0/178211134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5559ca307c70 0x5559ca30fd10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f60fc00e9c0 tx=0x7f60fc00cf60 comp rx=0 tx=0).stop 2024-09-19T05:06:25.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.253+0000 7f61126c2740 1 -- 172.21.15.43:0/178211134 shutdown_connections 2024-09-19T05:06:25.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.253+0000 7f61126c2740 1 --2- 172.21.15.43:0/178211134 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5559ca322ef0 0x5559ca325300 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:25.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.253+0000 7f61126c2740 1 --2- 172.21.15.43:0/178211134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559ca312e50 0x5559ca310790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:25.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.253+0000 7f61126c2740 1 --2- 172.21.15.43:0/178211134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559ca308590 0x5559ca310250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:25.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.253+0000 7f61126c2740 1 --2- 172.21.15.43:0/178211134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5559ca307c70 0x5559ca30fd10 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:25.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.253+0000 7f61126c2740 1 -- 172.21.15.43:0/178211134 >> 172.21.15.43:0/178211134 conn(0x5559ca304200 msgr2=0x5559ca309f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:25.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.253+0000 7f60eea00640 0 client.15573 ms_handle_reset on v2:172.21.15.43:6838/3404555030 2024-09-19T05:06:25.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.253+0000 7f61126c2740 1 -- 172.21.15.43:0/178211134 shutdown_connections 2024-09-19T05:06:25.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.253+0000 7f61126c2740 1 -- 172.21.15.43:0/178211134 wait complete. 2024-09-19T05:06:25.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.253+0000 7f61126c2740 20 client.15573 trim_cache size 0 max 16384 2024-09-19T05:06:25.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.267+0000 7f61126c2740 1 Processor -- start 2024-09-19T05:06:25.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.267+0000 7f61126c2740 1 -- start start 2024-09-19T05:06:25.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.268+0000 7f61126c2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559ca307c70 0x5559ca32abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:25.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.268+0000 7f61126c2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5559ca308590 0x5559ca32b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:25.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.268+0000 7f61126c2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559ca312e50 0x5559ca30cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:25.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.268+0000 7f61126c2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f61081afb40 con 0x5559ca308590 2024-09-19T05:06:25.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.268+0000 7f61126c2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f61081b1680 con 0x5559ca307c70 2024-09-19T05:06:25.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.268+0000 7f61126c2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f61081b0210 con 0x5559ca312e50 2024-09-19T05:06:25.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.268+0000 7f6107e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5559ca308590 0x5559ca32b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:25.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.268+0000 7f610d200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559ca307c70 0x5559ca32abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:25.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.268+0000 7f6107e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5559ca308590 0x5559ca32b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41374/0 (socket says 172.21.15.43:41374) 2024-09-19T05:06:25.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.268+0000 7f6107e00640 1 -- 172.21.15.43:0/168638852 learned_addr learned my addr 172.21.15.43:0/168638852 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:25.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.268+0000 7f610dc00640 1 --2- 172.21.15.43:0/168638852 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559ca312e50 0x5559ca30cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:25.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.268+0000 7f6107e00640 1 -- 172.21.15.43:0/168638852 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559ca312e50 msgr2=0x5559ca30cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:25.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.268+0000 7f6107e00640 1 --2- 172.21.15.43:0/168638852 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559ca312e50 0x5559ca30cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:25.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.268+0000 7f6107e00640 1 -- 172.21.15.43:0/168638852 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559ca307c70 msgr2=0x5559ca32abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:25.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.268+0000 7f6107e00640 1 --2- 172.21.15.43:0/168638852 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559ca307c70 0x5559ca32abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:25.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.268+0000 7f6107e00640 1 -- 172.21.15.43:0/168638852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559ca304030 con 0x5559ca308590 2024-09-19T05:06:25.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.268+0000 7f610dc00640 1 --2- 172.21.15.43:0/168638852 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559ca312e50 0x5559ca30cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:06:25.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.268+0000 7f610d200640 1 --2- 172.21.15.43:0/168638852 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559ca307c70 0x5559ca32abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:25.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.269+0000 7f6107e00640 1 --2- 172.21.15.43:0/168638852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5559ca308590 0x5559ca32b110 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f60f400bf30 tx=0x7f60f401e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279bfc1cc51a21aa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:25.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.269+0000 7f60f9600640 1 -- 172.21.15.43:0/168638852 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f401c820 con 0x5559ca308590 2024-09-19T05:06:25.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.269+0000 7f60f9600640 1 -- 172.21.15.43:0/168638852 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60f4005040 con 0x5559ca308590 2024-09-19T05:06:25.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.269+0000 7f60f9600640 1 -- 172.21.15.43:0/168638852 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f4011800 con 0x5559ca308590 2024-09-19T05:06:25.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.270+0000 7f61126c2740 1 -- 172.21.15.43:0/168638852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5559ca308590 msgr2=0x5559ca32b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:25.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.270+0000 7f61126c2740 1 --2- 172.21.15.43:0/168638852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5559ca308590 0x5559ca32b110 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f60f400bf30 tx=0x7f60f401e040 comp rx=0 tx=0).stop 2024-09-19T05:06:25.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.270+0000 7f61126c2740 1 -- 172.21.15.43:0/168638852 shutdown_connections 2024-09-19T05:06:25.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.270+0000 7f61126c2740 1 --2- 172.21.15.43:0/168638852 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559ca312e50 0x5559ca30cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:25.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.270+0000 7f61126c2740 1 --2- 172.21.15.43:0/168638852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5559ca308590 0x5559ca32b110 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:25.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.270+0000 7f61126c2740 1 --2- 172.21.15.43:0/168638852 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559ca307c70 0x5559ca32abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:25.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.270+0000 7f61126c2740 1 -- 172.21.15.43:0/168638852 >> 172.21.15.43:0/168638852 conn(0x5559ca304200 msgr2=0x5559ca309f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:25.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.270+0000 7f61126c2740 1 -- 172.21.15.43:0/168638852 shutdown_connections 2024-09-19T05:06:25.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.270+0000 7f61126c2740 1 -- 172.21.15.43:0/168638852 wait complete. 2024-09-19T05:06:25.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.270+0000 7f61126c2740 1 -- start start 2024-09-19T05:06:25.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.270+0000 7f61126c2740 1 Processor -- start 2024-09-19T05:06:25.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.271+0000 7f61126c2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5559ca307c70 0x5559ca31afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:25.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.271+0000 7f61126c2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559ca308590 0x5559ca31b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:25.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.271+0000 7f61126c2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559ca312e50 0x5559ca317dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:25.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.271+0000 7f61126c2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f61081b0fb0 con 0x5559ca307c70 2024-09-19T05:06:25.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.271+0000 7f61126c2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f61081b08e0 con 0x5559ca312e50 2024-09-19T05:06:25.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.271+0000 7f61126c2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f61081adb20 con 0x5559ca308590 2024-09-19T05:06:25.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.271+0000 7f61126c2740 10 client.0 _ll_register_callbacks cb 0x7f60fc0d3890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:25.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.271+0000 7f610d200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5559ca307c70 0x5559ca31afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:25.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.271+0000 7f610d200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5559ca307c70 0x5559ca31afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41378/0 (socket says 172.21.15.43:41378) 2024-09-19T05:06:25.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.271+0000 7f610d200640 1 -- 172.21.15.43:0/1120097446 learned_addr learned my addr 172.21.15.43:0/1120097446 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:25.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.271+0000 7f6107e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559ca308590 0x5559ca31b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:25.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.271+0000 7f610dc00640 1 --2- 172.21.15.43:0/1120097446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559ca312e50 0x5559ca317dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:25.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.272+0000 7f610d200640 1 -- 172.21.15.43:0/1120097446 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559ca308590 msgr2=0x5559ca31b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:25.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.272+0000 7f610d200640 1 --2- 172.21.15.43:0/1120097446 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559ca308590 0x5559ca31b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:25.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.272+0000 7f610d200640 1 -- 172.21.15.43:0/1120097446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559ca312e50 msgr2=0x5559ca317dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:25.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.272+0000 7f610d200640 1 --2- 172.21.15.43:0/1120097446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559ca312e50 0x5559ca317dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:25.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.272+0000 7f610d200640 1 -- 172.21.15.43:0/1120097446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559ca30aae0 con 0x5559ca307c70 2024-09-19T05:06:25.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.272+0000 7f610d200640 1 --2- 172.21.15.43:0/1120097446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5559ca307c70 0x5559ca31afd0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f60fc09bd00 tx=0x7f60fc0cd8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a5bd8d33e84689 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:25.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.272+0000 7f60eea00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:06:25.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.272+0000 7f60eea00640 1 -- 172.21.15.43:0/1120097446 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60fc0cdc80 con 0x5559ca307c70 2024-09-19T05:06:25.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.272+0000 7f61126c2740 1 -- 172.21.15.43:0/1120097446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60f4005040 con 0x5559ca307c70 2024-09-19T05:06:25.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.272+0000 7f60eea00640 1 -- 172.21.15.43:0/1120097446 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f60fc01ada0 con 0x5559ca307c70 2024-09-19T05:06:25.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.273+0000 7f60eea00640 1 -- 172.21.15.43:0/1120097446 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f60fc0d36d0 con 0x5559ca307c70 2024-09-19T05:06:25.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.273+0000 7f61126c2740 10 client.15579 fetch_fsmap learned FSMap version 29 2024-09-19T05:06:25.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.273+0000 7f61126c2740 1 -- 172.21.15.43:0/1120097446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5559ca2ff770 con 0x5559ca307c70 2024-09-19T05:06:25.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.273+0000 7f60eea00640 1 -- 172.21.15.43:0/1120097446 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60f4005040 con 0x5559ca307c70 2024-09-19T05:06:25.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.273+0000 7f60eea00640 1 -- 172.21.15.43:0/1120097446 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f60fc0982a0 con 0x5559ca307c70 2024-09-19T05:06:25.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.273+0000 7f61126c2740 10 client.15579 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:06:25.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.273+0000 7f61126c2740 10 client.15579 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-19T05:06:25.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.273+0000 7f61126c2740 20 client.15579 populate_metadata read hostname 'smithi043' 2024-09-19T05:06:25.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.273+0000 7f61126c2740 1 --2- 172.21.15.43:0/1120097446 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5559ca38e960 0x5559ca390d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:25.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.273+0000 7f61126c2740 4 client.15579 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:06:25.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.273+0000 7f61126c2740 1 -- 172.21.15.43:0/1120097446 --> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5559ca1f2010 con 0x5559ca38e960 2024-09-19T05:06:25.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.274+0000 7f6107e00640 1 --2- 172.21.15.43:0/1120097446 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5559ca38e960 0x5559ca390d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:25.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.274+0000 7f6107e00640 1 --2- 172.21.15.43:0/1120097446 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5559ca38e960 0x5559ca390d50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=52e85c4ede68777a server_cookie=16d19a0c5a6b462c in_seq=0 out_seq=0 2024-09-19T05:06:25.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.274+0000 7f60eea00640 10 client.15579 ms_handle_connect on v2:172.21.15.43:6838/3404555030 2024-09-19T05:06:25.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.274+0000 7f60eea00640 1 -- 172.21.15.43:0/1120097446 <== mds.1 v2:172.21.15.43:6838/3404555030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5559ca1f2010 con 0x5559ca38e960 2024-09-19T05:06:25.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.274+0000 7f60eea00640 10 client.15579 handle_command_reply: tid=0 2024-09-19T05:06:25.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.274+0000 7f61126c2740 1 client.15579 shutdown 2024-09-19T05:06:25.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.275+0000 7f61126c2740 1 -- 172.21.15.43:0/1120097446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5559ca307c70 msgr2=0x5559ca31afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:25.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.275+0000 7f61126c2740 1 --2- 172.21.15.43:0/1120097446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5559ca307c70 0x5559ca31afd0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f60fc09bd00 tx=0x7f60fc0cd8d0 comp rx=0 tx=0).stop 2024-09-19T05:06:25.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.275+0000 7f61126c2740 1 -- 172.21.15.43:0/1120097446 shutdown_connections 2024-09-19T05:06:25.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.275+0000 7f61126c2740 1 --2- 172.21.15.43:0/1120097446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5559ca312e50 0x5559ca317dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:25.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.275+0000 7f61126c2740 1 --2- 172.21.15.43:0/1120097446 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5559ca38e960 0x5559ca390d50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:25.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.275+0000 7f61126c2740 1 --2- 172.21.15.43:0/1120097446 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559ca308590 0x5559ca31b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:25.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.275+0000 7f61126c2740 1 --2- 172.21.15.43:0/1120097446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5559ca307c70 0x5559ca31afd0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:25.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.275+0000 7f61126c2740 1 -- 172.21.15.43:0/1120097446 >> 172.21.15.43:0/1120097446 conn(0x5559ca304200 msgr2=0x5559ca302b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:25.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.275+0000 7f60eea00640 0 client.15579 ms_handle_reset on v2:172.21.15.43:6838/3404555030 2024-09-19T05:06:25.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.275+0000 7f61126c2740 1 -- 172.21.15.43:0/1120097446 shutdown_connections 2024-09-19T05:06:25.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.275+0000 7f61126c2740 1 -- 172.21.15.43:0/1120097446 wait complete. 2024-09-19T05:06:25.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.275+0000 7f61126c2740 20 client.15579 trim_cache size 0 max 16384 2024-09-19T05:06:25.280 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:06:25.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.276+0000 7f610fc00640 1 -- 172.21.15.43:0/1602335532 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f60f00a33b0 msgr2=0x7f60f00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:25.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.276+0000 7f610fc00640 1 --2- 172.21.15.43:0/1602335532 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f60f00a33b0 0x7f60f00a5870 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f61000308d0 tx=0x7f6100005d80 comp rx=0 tx=0).stop 2024-09-19T05:06:25.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.276+0000 7f610fc00640 1 -- 172.21.15.43:0/1602335532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6108103ff0 msgr2=0x7f610819a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:25.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.276+0000 7f610fc00640 1 --2- 172.21.15.43:0/1602335532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6108103ff0 0x7f610819a620 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f60fc00e9f0 tx=0x7f60fc00eeb0 comp rx=0 tx=0).stop 2024-09-19T05:06:25.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.276+0000 7f610fc00640 1 -- 172.21.15.43:0/1602335532 shutdown_connections 2024-09-19T05:06:25.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.276+0000 7f610fc00640 1 --2- 172.21.15.43:0/1602335532 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f60f00a33b0 0x7f60f00a5870 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:25.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.276+0000 7f610fc00640 1 --2- 172.21.15.43:0/1602335532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6108109120 0x7f610819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:25.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.276+0000 7f610fc00640 1 --2- 172.21.15.43:0/1602335532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61081049a0 0x7f610819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:25.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.276+0000 7f610fc00640 1 --2- 172.21.15.43:0/1602335532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6108103ff0 0x7f610819a620 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:25.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.276+0000 7f610fc00640 1 -- 172.21.15.43:0/1602335532 >> 172.21.15.43:0/1602335532 conn(0x7f61080ffa90 msgr2=0x7f6108100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:25.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.276+0000 7f610fc00640 1 -- 172.21.15.43:0/1602335532 shutdown_connections 2024-09-19T05:06:25.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:25.276+0000 7f610fc00640 1 -- 172.21.15.43:0/1602335532 wait complete. 2024-09-19T05:06:26.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:25 smithi043 ceph-mon[30800]: pgmap v56: 97 pgs: 97 active+clean; 1.0 GiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 85 MiB/s wr, 34 op/s 2024-09-19T05:06:26.092 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T05:06:26.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:25 smithi151 ceph-mon[35425]: pgmap v56: 97 pgs: 97 active+clean; 1.0 GiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 85 MiB/s wr, 34 op/s 2024-09-19T05:06:26.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:25 smithi096 ceph-mon[35085]: pgmap v56: 97 pgs: 97 active+clean; 1.0 GiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 85 MiB/s wr, 34 op/s 2024-09-19T05:06:26.469 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:06:27.091 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-19T05:06:27.091 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-19T05:06:27.091 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-19T05:06:27.091 INFO:tasks.workunit.client.0.smithi043.stdout:iozone test complete. 2024-09-19T05:06:27.094 INFO:tasks.workunit.client.0.smithi043.stderr:+ iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-09-19T05:06:27.174 INFO:tasks.workunit.client.0.smithi043.stdout: Iozone: Performance Test of File I/O 2024-09-19T05:06:27.174 INFO:tasks.workunit.client.0.smithi043.stdout: Version $Revision: 3.494 $ 2024-09-19T05:06:27.174 INFO:tasks.workunit.client.0.smithi043.stdout: Compiled for 64 bit mode. 2024-09-19T05:06:27.174 INFO:tasks.workunit.client.0.smithi043.stdout: Build: linux-AMD64 2024-09-19T05:06:27.174 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-19T05:06:27.174 INFO:tasks.workunit.client.0.smithi043.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-09-19T05:06:27.174 INFO:tasks.workunit.client.0.smithi043.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-09-19T05:06:27.174 INFO:tasks.workunit.client.0.smithi043.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-09-19T05:06:27.174 INFO:tasks.workunit.client.0.smithi043.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-09-19T05:06:27.175 INFO:tasks.workunit.client.0.smithi043.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-09-19T05:06:27.175 INFO:tasks.workunit.client.0.smithi043.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-09-19T05:06:27.175 INFO:tasks.workunit.client.0.smithi043.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-09-19T05:06:27.175 INFO:tasks.workunit.client.0.smithi043.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-09-19T05:06:27.175 INFO:tasks.workunit.client.0.smithi043.stdout: Alexey Skidanov, Sudhir Kumar. 2024-09-19T05:06:27.175 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-19T05:06:27.175 INFO:tasks.workunit.client.0.smithi043.stdout: Run began: Thu Sep 19 05:06:27 2024 2024-09-19T05:06:27.175 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-19T05:06:27.175 INFO:tasks.workunit.client.0.smithi043.stdout: Include close in write timing 2024-09-19T05:06:27.175 INFO:tasks.workunit.client.0.smithi043.stdout: Include fsync in write timing 2024-09-19T05:06:27.175 INFO:tasks.workunit.client.0.smithi043.stdout: File size set to 1048576 kB 2024-09-19T05:06:27.175 INFO:tasks.workunit.client.0.smithi043.stdout: Record Size 1024 kB 2024-09-19T05:06:27.175 INFO:tasks.workunit.client.0.smithi043.stdout: Command line used: iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-09-19T05:06:27.175 INFO:tasks.workunit.client.0.smithi043.stdout: Output is in kBytes/sec 2024-09-19T05:06:27.175 INFO:tasks.workunit.client.0.smithi043.stdout: Time Resolution = 0.000001 seconds. 2024-09-19T05:06:27.176 INFO:tasks.workunit.client.0.smithi043.stdout: Processor cache size set to 1024 kBytes. 2024-09-19T05:06:27.176 INFO:tasks.workunit.client.0.smithi043.stdout: Processor cache line size set to 32 bytes. 2024-09-19T05:06:27.176 INFO:tasks.workunit.client.0.smithi043.stdout: File stride size set to 17 * record size. 2024-09-19T05:06:27.176 INFO:tasks.workunit.client.0.smithi043.stdout: Throughput test with 1 process 2024-09-19T05:06:27.176 INFO:tasks.workunit.client.0.smithi043.stdout: Each process writes a 1048576 kByte file in 1024 kByte records 2024-09-19T05:06:27.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.333+0000 7f5fd1e00640 1 -- 172.21.15.43:0/3749981233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fcc102d30 msgr2=0x7f5fcc103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:27.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.333+0000 7f5fd1e00640 1 --2- 172.21.15.43:0/3749981233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fcc102d30 0x7f5fcc103110 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc009900 tx=0x7f5fbc02a480 comp rx=0 tx=0).stop 2024-09-19T05:06:27.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.333+0000 7f5fd1e00640 1 -- 172.21.15.43:0/3749981233 shutdown_connections 2024-09-19T05:06:27.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.333+0000 7f5fd1e00640 1 --2- 172.21.15.43:0/3749981233 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5fcc10c160 0x7f5fcc10e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:27.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.333+0000 7f5fd1e00640 1 --2- 172.21.15.43:0/3749981233 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5fcc103650 0x7f5fcc10bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:27.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.333+0000 7f5fd1e00640 1 --2- 172.21.15.43:0/3749981233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fcc102d30 0x7f5fcc103110 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:27.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.333+0000 7f5fd1e00640 1 -- 172.21.15.43:0/3749981233 >> 172.21.15.43:0/3749981233 conn(0x7f5fcc0fc790 msgr2=0x7f5fcc0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:27.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.334+0000 7f5fd1e00640 1 -- 172.21.15.43:0/3749981233 shutdown_connections 2024-09-19T05:06:27.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.334+0000 7f5fd1e00640 1 -- 172.21.15.43:0/3749981233 wait complete. 2024-09-19T05:06:27.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.335+0000 7f5fd1e00640 1 Processor -- start 2024-09-19T05:06:27.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.335+0000 7f5fd1e00640 1 -- start start 2024-09-19T05:06:27.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.335+0000 7f5fd1e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5fcc102d30 0x7f5fcc19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:27.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.335+0000 7f5fd1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fcc103650 0x7f5fcc19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:27.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.335+0000 7f5fd1e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5fcc10c160 0x7f5fcc19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:27.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.335+0000 7f5fd1e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5fcc111f60 con 0x7f5fcc103650 2024-09-19T05:06:27.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.335+0000 7f5fd1e00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5fcc111de0 con 0x7f5fcc102d30 2024-09-19T05:06:27.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.335+0000 7f5fd1e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5fcc1120e0 con 0x7f5fcc10c160 2024-09-19T05:06:27.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.335+0000 7f5fc3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fcc103650 0x7f5fcc19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:27.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.335+0000 7f5fcb400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5fcc10c160 0x7f5fcc19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:27.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.335+0000 7f5fc3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fcc103650 0x7f5fcc19aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51440/0 (socket says 172.21.15.43:51440) 2024-09-19T05:06:27.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.335+0000 7f5fcb400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5fcc10c160 0x7f5fcc19ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:57674/0 (socket says 172.21.15.43:57674) 2024-09-19T05:06:27.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.335+0000 7f5fc3e00640 1 -- 172.21.15.43:0/1495953889 learned_addr learned my addr 172.21.15.43:0/1495953889 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:27.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.336+0000 7f5fcaa00640 1 --2- 172.21.15.43:0/1495953889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5fcc102d30 0x7f5fcc19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:27.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.336+0000 7f5fc3e00640 1 -- 172.21.15.43:0/1495953889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5fcc10c160 msgr2=0x7f5fcc19ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:27.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.336+0000 7f5fc3e00640 1 --2- 172.21.15.43:0/1495953889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5fcc10c160 0x7f5fcc19ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:27.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.336+0000 7f5fc3e00640 1 -- 172.21.15.43:0/1495953889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5fcc102d30 msgr2=0x7f5fcc19a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:27.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.336+0000 7f5fc3e00640 1 --2- 172.21.15.43:0/1495953889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5fcc102d30 0x7f5fcc19a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:27.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.336+0000 7f5fc3e00640 1 -- 172.21.15.43:0/1495953889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fcc19f610 con 0x7f5fcc103650 2024-09-19T05:06:27.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.336+0000 7f5fcb400640 1 --2- 172.21.15.43:0/1495953889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5fcc10c160 0x7f5fcc19ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:27.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.336+0000 7f5fcaa00640 1 --2- 172.21.15.43:0/1495953889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5fcc102d30 0x7f5fcc19a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:27.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.336+0000 7f5fc3e00640 1 --2- 172.21.15.43:0/1495953889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fcc103650 0x7f5fcc19aba0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5fb400d950 tx=0x7f5fb400dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:27.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.337+0000 7f5fd0a00640 1 -- 172.21.15.43:0/1495953889 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb4017030 con 0x7f5fcc103650 2024-09-19T05:06:27.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.337+0000 7f5fd1e00640 1 -- 172.21.15.43:0/1495953889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fcc19f900 con 0x7f5fcc103650 2024-09-19T05:06:27.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.337+0000 7f5fd0a00640 1 -- 172.21.15.43:0/1495953889 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fb400be10 con 0x7f5fcc103650 2024-09-19T05:06:27.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.337+0000 7f5fd0a00640 1 -- 172.21.15.43:0/1495953889 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb40055f0 con 0x7f5fcc103650 2024-09-19T05:06:27.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.337+0000 7f5fd1e00640 1 -- 172.21.15.43:0/1495953889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fcc1a71e0 con 0x7f5fcc103650 2024-09-19T05:06:27.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.340+0000 7f5fd0a00640 1 -- 172.21.15.43:0/1495953889 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f5fb4009a30 con 0x7f5fcc103650 2024-09-19T05:06:27.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.341+0000 7f5fd0a00640 1 --2- 172.21.15.43:0/1495953889 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f5fb00a3430 0x7f5fb00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:27.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.341+0000 7f5fd47f7740 1 Processor -- start 2024-09-19T05:06:27.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.342+0000 7f5fd0a00640 1 -- 172.21.15.43:0/1495953889 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f5fb40cf560 con 0x7f5fcc103650 2024-09-19T05:06:27.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.342+0000 7f5fd47f7740 1 -- start start 2024-09-19T05:06:27.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.342+0000 7f5fcaa00640 1 --2- 172.21.15.43:0/1495953889 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f5fb00a3430 0x7f5fb00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:27.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.342+0000 7f5fd47f7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590ddb0ac70 0x5590ddb0b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:27.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.342+0000 7f5fd47f7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ddb0b590 0x5590ddb15910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:27.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.342+0000 7f5fd47f7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590ddb15e50 0x5590ddb18240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:27.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.342+0000 7f5fd47f7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5590dda713e0 con 0x5590ddb15e50 2024-09-19T05:06:27.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.342+0000 7f5fd47f7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5590dda18d80 con 0x5590ddb0b590 2024-09-19T05:06:27.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.342+0000 7f5fd47f7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5590dda6d3a0 con 0x5590ddb0ac70 2024-09-19T05:06:27.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.343+0000 7f5fc3e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ddb0b590 0x5590ddb15910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:27.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.343+0000 7f5fc3e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ddb0b590 0x5590ddb15910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:46424/0 (socket says 172.21.15.43:46424) 2024-09-19T05:06:27.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.343+0000 7f5fc3e00640 1 -- 172.21.15.43:0/3829507544 learned_addr learned my addr 172.21.15.43:0/3829507544 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:27.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.343+0000 7f5fcb400640 1 --2- 172.21.15.43:0/3829507544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590ddb15e50 0x5590ddb18240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:27.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.343+0000 7f5fcaa00640 1 --2- 172.21.15.43:0/3829507544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590ddb0ac70 0x5590ddb0b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:27.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.343+0000 7f5fc3e00640 1 -- 172.21.15.43:0/3829507544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590ddb0ac70 msgr2=0x5590ddb0b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:27.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.343+0000 7f5fc3e00640 1 --2- 172.21.15.43:0/3829507544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590ddb0ac70 0x5590ddb0b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:27.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.343+0000 7f5fc3e00640 1 -- 172.21.15.43:0/3829507544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590ddb15e50 msgr2=0x5590ddb18240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:27.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.343+0000 7f5fc3e00640 1 --2- 172.21.15.43:0/3829507544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590ddb15e50 0x5590ddb18240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:27.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.343+0000 7f5fc3e00640 1 -- 172.21.15.43:0/3829507544 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590dda37ae0 con 0x5590ddb0b590 2024-09-19T05:06:27.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.343+0000 7f5fcaa00640 1 --2- 172.21.15.43:0/3829507544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590ddb0ac70 0x5590ddb0b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:06:27.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.343+0000 7f5fcb400640 1 --2- 172.21.15.43:0/3829507544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590ddb15e50 0x5590ddb18240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:06:27.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.343+0000 7f5fc3e00640 1 --2- 172.21.15.43:0/3829507544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ddb0b590 0x5590ddb15910 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5fb40981c0 tx=0x7f5fb409bef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cb08c6822e26eca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:27.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.343+0000 7f5fcaa00640 1 --2- 172.21.15.43:0/1495953889 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f5fb00a3430 0x7f5fb00a58f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc009230 tx=0x7f5fbc005db0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:27.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.343+0000 7f5fbbe00640 1 -- 172.21.15.43:0/3829507544 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb40b4070 con 0x5590ddb0b590 2024-09-19T05:06:27.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.344+0000 7f5fbbe00640 1 -- 172.21.15.43:0/3829507544 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fb409fba0 con 0x5590ddb0b590 2024-09-19T05:06:27.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.344+0000 7f5fbbe00640 1 -- 172.21.15.43:0/3829507544 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb4097e60 con 0x5590ddb0b590 2024-09-19T05:06:27.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.344+0000 7f5fd47f7740 1 -- 172.21.15.43:0/3829507544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ddb0b590 msgr2=0x5590ddb15910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:27.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.344+0000 7f5fd47f7740 1 --2- 172.21.15.43:0/3829507544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ddb0b590 0x5590ddb15910 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5fb40981c0 tx=0x7f5fb409bef0 comp rx=0 tx=0).stop 2024-09-19T05:06:27.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.344+0000 7f5fd47f7740 1 -- 172.21.15.43:0/3829507544 shutdown_connections 2024-09-19T05:06:27.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.344+0000 7f5fd47f7740 1 --2- 172.21.15.43:0/3829507544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590ddb15e50 0x5590ddb18240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:27.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.344+0000 7f5fd47f7740 1 --2- 172.21.15.43:0/3829507544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ddb0b590 0x5590ddb15910 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:27.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.344+0000 7f5fd47f7740 1 --2- 172.21.15.43:0/3829507544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590ddb0ac70 0x5590ddb0b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:27.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.344+0000 7f5fd47f7740 1 -- 172.21.15.43:0/3829507544 >> 172.21.15.43:0/3829507544 conn(0x5590ddb07200 msgr2=0x5590ddb075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:27.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.345+0000 7f5fd47f7740 1 -- 172.21.15.43:0/3829507544 shutdown_connections 2024-09-19T05:06:27.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.345+0000 7f5fd47f7740 1 -- 172.21.15.43:0/3829507544 wait complete. 2024-09-19T05:06:27.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.345+0000 7f5fd47f7740 1 -- start start 2024-09-19T05:06:27.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.345+0000 7f5fd47f7740 1 Processor -- start 2024-09-19T05:06:27.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.346+0000 7f5fd47f7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ddb0ac70 0x5590ddb12e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:27.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.346+0000 7f5fd47f7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590ddb0b590 0x5590ddb133a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:27.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.346+0000 7f5fd47f7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590ddb15e50 0x5590ddb138e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:27.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.346+0000 7f5fd47f7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5590ddb18a20 con 0x5590ddb15e50 2024-09-19T05:06:27.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.346+0000 7f5fd47f7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5590dda2c0f0 con 0x5590ddb0ac70 2024-09-19T05:06:27.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.346+0000 7f5fd47f7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5590ddb18ba0 con 0x5590ddb0b590 2024-09-19T05:06:27.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.346+0000 7f5fd47f7740 10 client.0 _ll_register_callbacks cb 0x5590dda13bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:27.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.346+0000 7f5fc3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590ddb0b590 0x5590ddb133a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:27.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.346+0000 7f5fcaa00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ddb0ac70 0x5590ddb12e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:27.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.346+0000 7f5fc3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590ddb0b590 0x5590ddb133a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:57694/0 (socket says 172.21.15.43:57694) 2024-09-19T05:06:27.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.346+0000 7f5fc3e00640 1 -- 172.21.15.43:0/688852288 learned_addr learned my addr 172.21.15.43:0/688852288 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:27.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.346+0000 7f5fcb400640 1 --2- 172.21.15.43:0/688852288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590ddb15e50 0x5590ddb138e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:27.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.347+0000 7f5fcaa00640 1 -- 172.21.15.43:0/688852288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590ddb0b590 msgr2=0x5590ddb133a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:27.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.347+0000 7f5fcaa00640 1 --2- 172.21.15.43:0/688852288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590ddb0b590 0x5590ddb133a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:27.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.347+0000 7f5fcaa00640 1 -- 172.21.15.43:0/688852288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590ddb15e50 msgr2=0x5590ddb138e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:27.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.347+0000 7f5fcaa00640 1 --2- 172.21.15.43:0/688852288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590ddb15e50 0x5590ddb138e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:27.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.347+0000 7f5fcaa00640 1 -- 172.21.15.43:0/688852288 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590dd7e23c0 con 0x5590ddb0ac70 2024-09-19T05:06:27.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.347+0000 7f5fcb400640 1 --2- 172.21.15.43:0/688852288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590ddb15e50 0x5590ddb138e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:06:27.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.347+0000 7f5fcaa00640 1 --2- 172.21.15.43:0/688852288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ddb0ac70 0x5590ddb12e60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc006010 tx=0x7f5fbc02a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f13fa588ce1ebe33 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:27.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.347+0000 7f5fb8c00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:06:27.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.347+0000 7f5fb8c00640 1 -- 172.21.15.43:0/688852288 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc04c070 con 0x5590ddb0ac70 2024-09-19T05:06:27.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.347+0000 7f5fd47f7740 1 -- 172.21.15.43:0/688852288 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fb409fba0 con 0x5590ddb0ac70 2024-09-19T05:06:27.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.347+0000 7f5fb8c00640 1 -- 172.21.15.43:0/688852288 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f5fbc003ac0 con 0x5590ddb0ac70 2024-09-19T05:06:27.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.348+0000 7f5fb8c00640 1 -- 172.21.15.43:0/688852288 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5fbc058050 con 0x5590ddb0ac70 2024-09-19T05:06:27.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.348+0000 7f5fd47f7740 10 client.25580 fetch_fsmap learned FSMap version 29 2024-09-19T05:06:27.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.348+0000 7f5fd47f7740 1 -- 172.21.15.43:0/688852288 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5590ddb24b40 con 0x5590ddb0ac70 2024-09-19T05:06:27.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.348+0000 7f5fb8c00640 1 -- 172.21.15.43:0/688852288 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fb409fba0 con 0x5590ddb0ac70 2024-09-19T05:06:27.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.348+0000 7f5fb8c00640 1 -- 172.21.15.43:0/688852288 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5fbc048410 con 0x5590ddb0ac70 2024-09-19T05:06:27.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.348+0000 7f5fd47f7740 10 client.25580 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:06:27.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.348+0000 7f5fd47f7740 10 client.25580 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:06:27.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.348+0000 7f5fd47f7740 20 client.25580 populate_metadata read hostname 'smithi043' 2024-09-19T05:06:27.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.348+0000 7f5fd47f7740 1 --2- 172.21.15.43:0/688852288 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5590ddb25f50 0x5590ddb28340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:27.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.348+0000 7f5fd47f7740 4 client.25580 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:06:27.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.348+0000 7f5fd47f7740 1 -- 172.21.15.43:0/688852288 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590dda7e950 con 0x5590ddb25f50 2024-09-19T05:06:27.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.349+0000 7f5fc3e00640 1 --2- 172.21.15.43:0/688852288 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5590ddb25f50 0x5590ddb28340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:27.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.349+0000 7f5fc3e00640 1 --2- 172.21.15.43:0/688852288 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5590ddb25f50 0x5590ddb28340 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f81f59a1c4076ba server_cookie=520f584f12a14e9b in_seq=0 out_seq=0 2024-09-19T05:06:27.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.349+0000 7f5fb8c00640 10 client.25580 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:06:27.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.350+0000 7f5fb8c00640 1 -- 172.21.15.43:0/688852288 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5590dda7e950 con 0x5590ddb25f50 2024-09-19T05:06:27.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.350+0000 7f5fb8c00640 10 client.25580 handle_command_reply: tid=0 2024-09-19T05:06:27.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.350+0000 7f5fd47f7740 1 client.25580 shutdown 2024-09-19T05:06:27.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.350+0000 7f5fd47f7740 1 -- 172.21.15.43:0/688852288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ddb0ac70 msgr2=0x5590ddb12e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:27.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.350+0000 7f5fd47f7740 1 --2- 172.21.15.43:0/688852288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ddb0ac70 0x5590ddb12e60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc006010 tx=0x7f5fbc02a480 comp rx=0 tx=0).stop 2024-09-19T05:06:27.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.350+0000 7f5fd47f7740 1 -- 172.21.15.43:0/688852288 shutdown_connections 2024-09-19T05:06:27.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.351+0000 7f5fd47f7740 1 --2- 172.21.15.43:0/688852288 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5590ddb25f50 0x5590ddb28340 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:27.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.351+0000 7f5fd47f7740 1 --2- 172.21.15.43:0/688852288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590ddb15e50 0x5590ddb138e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:27.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.351+0000 7f5fb8c00640 0 client.25580 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:06:27.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.351+0000 7f5fd47f7740 1 --2- 172.21.15.43:0/688852288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590ddb0b590 0x5590ddb133a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:27.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.351+0000 7f5fd47f7740 1 --2- 172.21.15.43:0/688852288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ddb0ac70 0x5590ddb12e60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:27.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.351+0000 7f5fd47f7740 1 -- 172.21.15.43:0/688852288 >> 172.21.15.43:0/688852288 conn(0x5590ddb07200 msgr2=0x5590ddb0cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:27.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.351+0000 7f5fd47f7740 1 -- 172.21.15.43:0/688852288 shutdown_connections 2024-09-19T05:06:27.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.351+0000 7f5fd47f7740 1 -- 172.21.15.43:0/688852288 wait complete. 2024-09-19T05:06:27.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.351+0000 7f5fd47f7740 20 client.25580 trim_cache size 0 max 16384 2024-09-19T05:06:27.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.365+0000 7f5fd47f7740 1 Processor -- start 2024-09-19T05:06:27.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.365+0000 7f5fd47f7740 1 -- start start 2024-09-19T05:06:27.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.365+0000 7f5fd47f7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590ddb07200 0x5590ddb0ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:27.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.366+0000 7f5fd47f7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ddb0ac70 0x5590ddb2db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:27.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.366+0000 7f5fd47f7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590ddb0b590 0x5590ddb2e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:27.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.366+0000 7f5fd47f7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5fcc1afba0 con 0x5590ddb07200 2024-09-19T05:06:27.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.366+0000 7f5fd47f7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5fcc1b16e0 con 0x5590ddb0ac70 2024-09-19T05:06:27.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.366+0000 7f5fd47f7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5fcc1b0270 con 0x5590ddb0b590 2024-09-19T05:06:27.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.366+0000 7f5fcaa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590ddb07200 0x5590ddb0ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:27.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.366+0000 7f5fcaa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590ddb07200 0x5590ddb0ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51472/0 (socket says 172.21.15.43:51472) 2024-09-19T05:06:27.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.366+0000 7f5fcaa00640 1 -- 172.21.15.43:0/994226099 learned_addr learned my addr 172.21.15.43:0/994226099 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:27.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.366+0000 7f5fcb400640 1 --2- 172.21.15.43:0/994226099 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590ddb0b590 0x5590ddb2e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:27.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.366+0000 7f5fc3e00640 1 --2- 172.21.15.43:0/994226099 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ddb0ac70 0x5590ddb2db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:27.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.366+0000 7f5fcaa00640 1 -- 172.21.15.43:0/994226099 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590ddb0b590 msgr2=0x5590ddb2e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:27.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.366+0000 7f5fcaa00640 1 --2- 172.21.15.43:0/994226099 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590ddb0b590 0x5590ddb2e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:27.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.366+0000 7f5fcaa00640 1 -- 172.21.15.43:0/994226099 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ddb0ac70 msgr2=0x5590ddb2db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:27.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.366+0000 7f5fcaa00640 1 --2- 172.21.15.43:0/994226099 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ddb0ac70 0x5590ddb2db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:27.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.366+0000 7f5fcaa00640 1 -- 172.21.15.43:0/994226099 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590ddb28920 con 0x5590ddb07200 2024-09-19T05:06:27.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.366+0000 7f5fc3e00640 1 --2- 172.21.15.43:0/994226099 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ddb0ac70 0x5590ddb2db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:06:27.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.367+0000 7f5fcaa00640 1 --2- 172.21.15.43:0/994226099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590ddb07200 0x5590ddb0ce20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc02a9a0 tx=0x7f5fbc005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0445707ec4d024f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:27.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.367+0000 7f5fbb400640 1 -- 172.21.15.43:0/994226099 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc04c040 con 0x5590ddb07200 2024-09-19T05:06:27.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.367+0000 7f5fbb400640 1 -- 172.21.15.43:0/994226099 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fbc0485e0 con 0x5590ddb07200 2024-09-19T05:06:27.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.367+0000 7f5fbb400640 1 -- 172.21.15.43:0/994226099 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc043070 con 0x5590ddb07200 2024-09-19T05:06:27.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.368+0000 7f5fd47f7740 1 -- 172.21.15.43:0/994226099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590ddb07200 msgr2=0x5590ddb0ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:27.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.368+0000 7f5fd47f7740 1 --2- 172.21.15.43:0/994226099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590ddb07200 0x5590ddb0ce20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc02a9a0 tx=0x7f5fbc005bd0 comp rx=0 tx=0).stop 2024-09-19T05:06:27.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.368+0000 7f5fd47f7740 1 -- 172.21.15.43:0/994226099 shutdown_connections 2024-09-19T05:06:27.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.368+0000 7f5fd47f7740 1 --2- 172.21.15.43:0/994226099 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590ddb0b590 0x5590ddb2e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:27.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.368+0000 7f5fd47f7740 1 --2- 172.21.15.43:0/994226099 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ddb0ac70 0x5590ddb2db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:27.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.368+0000 7f5fd47f7740 1 --2- 172.21.15.43:0/994226099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590ddb07200 0x5590ddb0ce20 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:27.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.368+0000 7f5fd47f7740 1 -- 172.21.15.43:0/994226099 >> 172.21.15.43:0/994226099 conn(0x5590ddb0e850 msgr2=0x5590ddb0c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:27.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.368+0000 7f5fd47f7740 1 -- 172.21.15.43:0/994226099 shutdown_connections 2024-09-19T05:06:27.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.368+0000 7f5fd47f7740 1 -- 172.21.15.43:0/994226099 wait complete. 2024-09-19T05:06:27.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.369+0000 7f5fd47f7740 1 -- start start 2024-09-19T05:06:27.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.369+0000 7f5fd47f7740 1 Processor -- start 2024-09-19T05:06:27.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.369+0000 7f5fd47f7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590ddb07200 0x5590ddb1a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:27.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.369+0000 7f5fd47f7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590ddb0ac70 0x5590ddb1a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:27.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.369+0000 7f5fd47f7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ddb0b590 0x5590ddb8fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:27.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.369+0000 7f5fd47f7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5fcc1b1010 con 0x5590ddb07200 2024-09-19T05:06:27.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.369+0000 7f5fd47f7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5fcc1b0940 con 0x5590ddb0b590 2024-09-19T05:06:27.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.369+0000 7f5fd47f7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5fcc1adb80 con 0x5590ddb0ac70 2024-09-19T05:06:27.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.370+0000 7f5fd47f7740 10 client.0 _ll_register_callbacks cb 0x7f5fbc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:27.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.370+0000 7f5fc3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590ddb0ac70 0x5590ddb1a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:27.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.370+0000 7f5fc3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590ddb0ac70 0x5590ddb1a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:57702/0 (socket says 172.21.15.43:57702) 2024-09-19T05:06:27.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.370+0000 7f5fc3e00640 1 -- 172.21.15.43:0/2772683775 learned_addr learned my addr 172.21.15.43:0/2772683775 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:27.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.370+0000 7f5fcaa00640 1 --2- 172.21.15.43:0/2772683775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590ddb07200 0x5590ddb1a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:27.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.370+0000 7f5fcb400640 1 --2- 172.21.15.43:0/2772683775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ddb0b590 0x5590ddb8fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:27.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.370+0000 7f5fc3e00640 1 -- 172.21.15.43:0/2772683775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ddb0b590 msgr2=0x5590ddb8fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:27.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.370+0000 7f5fc3e00640 1 --2- 172.21.15.43:0/2772683775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ddb0b590 0x5590ddb8fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:27.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.370+0000 7f5fc3e00640 1 -- 172.21.15.43:0/2772683775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590ddb07200 msgr2=0x5590ddb1a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:27.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.370+0000 7f5fc3e00640 1 --2- 172.21.15.43:0/2772683775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590ddb07200 0x5590ddb1a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:27.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.370+0000 7f5fc3e00640 1 -- 172.21.15.43:0/2772683775 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590ddb02770 con 0x5590ddb0ac70 2024-09-19T05:06:27.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.370+0000 7f5fcaa00640 1 --2- 172.21.15.43:0/2772683775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590ddb07200 0x5590ddb1a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:06:27.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.371+0000 7f5fc3e00640 1 --2- 172.21.15.43:0/2772683775 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590ddb0ac70 0x5590ddb1a720 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4099aa0 tx=0x7f5fb40045b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd43b24cd9d2c72e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:27.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.371+0000 7f5fb8c00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:06:27.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.371+0000 7f5fb8c00640 1 -- 172.21.15.43:0/2772683775 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb40b4040 con 0x5590ddb0ac70 2024-09-19T05:06:27.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.371+0000 7f5fb8c00640 1 -- 172.21.15.43:0/2772683775 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f5fb40ccee0 con 0x5590ddb0ac70 2024-09-19T05:06:27.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.371+0000 7f5fd47f7740 1 -- 172.21.15.43:0/2772683775 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fbc0485e0 con 0x5590ddb0ac70 2024-09-19T05:06:27.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.371+0000 7f5fb8c00640 1 -- 172.21.15.43:0/2772683775 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5fb40a4460 con 0x5590ddb0ac70 2024-09-19T05:06:27.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.371+0000 7f5fd47f7740 10 client.25543 fetch_fsmap learned FSMap version 29 2024-09-19T05:06:27.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.371+0000 7f5fd47f7740 1 -- 172.21.15.43:0/2772683775 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5590ddb91060 con 0x5590ddb0ac70 2024-09-19T05:06:27.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.371+0000 7f5fb8c00640 1 -- 172.21.15.43:0/2772683775 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fbc0485e0 con 0x5590ddb0ac70 2024-09-19T05:06:27.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.371+0000 7f5fb8c00640 1 -- 172.21.15.43:0/2772683775 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5fb40cf040 con 0x5590ddb0ac70 2024-09-19T05:06:27.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.372+0000 7f5fd47f7740 10 client.25543 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:06:27.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.372+0000 7f5fd47f7740 10 client.25543 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:06:27.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.372+0000 7f5fd47f7740 20 client.25543 populate_metadata read hostname 'smithi043' 2024-09-19T05:06:27.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.372+0000 7f5fd47f7740 1 --2- 172.21.15.43:0/2772683775 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5590ddb15e50 0x5590ddb93410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:27.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.372+0000 7f5fd47f7740 4 client.25543 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:06:27.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.372+0000 7f5fd47f7740 1 -- 172.21.15.43:0/2772683775 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5590dda78e60 con 0x5590ddb15e50 2024-09-19T05:06:27.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.372+0000 7f5fcaa00640 1 --2- 172.21.15.43:0/2772683775 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5590ddb15e50 0x5590ddb93410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:27.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.372+0000 7f5fcaa00640 1 --2- 172.21.15.43:0/2772683775 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5590ddb15e50 0x5590ddb93410 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=132e0a309a8618d8 server_cookie=de408b9f461e6f12 in_seq=0 out_seq=0 2024-09-19T05:06:27.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.372+0000 7f5fb8c00640 10 client.25543 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:06:27.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.373+0000 7f5fb8c00640 1 -- 172.21.15.43:0/2772683775 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5590dda78e60 con 0x5590ddb15e50 2024-09-19T05:06:27.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.373+0000 7f5fb8c00640 10 client.25543 handle_command_reply: tid=0 2024-09-19T05:06:27.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.373+0000 7f5fd47f7740 1 client.25543 shutdown 2024-09-19T05:06:27.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.373+0000 7f5fd47f7740 1 -- 172.21.15.43:0/2772683775 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590ddb0ac70 msgr2=0x5590ddb1a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:27.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.373+0000 7f5fd47f7740 1 --2- 172.21.15.43:0/2772683775 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590ddb0ac70 0x5590ddb1a720 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4099aa0 tx=0x7f5fb40045b0 comp rx=0 tx=0).stop 2024-09-19T05:06:27.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.373+0000 7f5fd47f7740 1 -- 172.21.15.43:0/2772683775 shutdown_connections 2024-09-19T05:06:27.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.373+0000 7f5fd47f7740 1 --2- 172.21.15.43:0/2772683775 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5590ddb15e50 0x5590ddb93410 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:27.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.373+0000 7f5fd47f7740 1 --2- 172.21.15.43:0/2772683775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ddb0b590 0x5590ddb8fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:27.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.373+0000 7f5fd47f7740 1 --2- 172.21.15.43:0/2772683775 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5590ddb0ac70 0x5590ddb1a720 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:27.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.373+0000 7f5fd47f7740 1 --2- 172.21.15.43:0/2772683775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590ddb07200 0x5590ddb1a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:27.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.373+0000 7f5fd47f7740 1 -- 172.21.15.43:0/2772683775 >> 172.21.15.43:0/2772683775 conn(0x5590ddb0e850 msgr2=0x5590ddb06380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:27.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.373+0000 7f5fb8c00640 0 client.25543 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:06:27.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.373+0000 7f5fd47f7740 1 -- 172.21.15.43:0/2772683775 shutdown_connections 2024-09-19T05:06:27.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.373+0000 7f5fd47f7740 1 -- 172.21.15.43:0/2772683775 wait complete. 2024-09-19T05:06:27.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.373+0000 7f5fd47f7740 20 client.25543 trim_cache size 0 max 16384 2024-09-19T05:06:27.379 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:06:27.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.374+0000 7f5fd1e00640 1 -- 172.21.15.43:0/1495953889 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f5fb00a3430 msgr2=0x7f5fb00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:27.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.374+0000 7f5fd1e00640 1 --2- 172.21.15.43:0/1495953889 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f5fb00a3430 0x7f5fb00a58f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc009230 tx=0x7f5fbc005db0 comp rx=0 tx=0).stop 2024-09-19T05:06:27.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.374+0000 7f5fd1e00640 1 -- 172.21.15.43:0/1495953889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fcc103650 msgr2=0x7f5fcc19aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:27.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.374+0000 7f5fd1e00640 1 --2- 172.21.15.43:0/1495953889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fcc103650 0x7f5fcc19aba0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5fb400d950 tx=0x7f5fb400dd50 comp rx=0 tx=0).stop 2024-09-19T05:06:27.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.374+0000 7f5fd1e00640 1 -- 172.21.15.43:0/1495953889 shutdown_connections 2024-09-19T05:06:27.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.374+0000 7f5fd1e00640 1 --2- 172.21.15.43:0/1495953889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5fcc10c160 0x7f5fcc19ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:27.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.374+0000 7f5fd1e00640 1 --2- 172.21.15.43:0/1495953889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fcc103650 0x7f5fcc19aba0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:27.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.374+0000 7f5fd1e00640 1 --2- 172.21.15.43:0/1495953889 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f5fb00a3430 0x7f5fb00a58f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:27.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.374+0000 7f5fd1e00640 1 --2- 172.21.15.43:0/1495953889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5fcc102d30 0x7f5fcc19a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:27.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.374+0000 7f5fd1e00640 1 -- 172.21.15.43:0/1495953889 >> 172.21.15.43:0/1495953889 conn(0x7f5fcc0fc790 msgr2=0x7f5fcc101b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:27.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.374+0000 7f5fd1e00640 1 -- 172.21.15.43:0/1495953889 shutdown_connections 2024-09-19T05:06:27.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:27.374+0000 7f5fd1e00640 1 -- 172.21.15.43:0/1495953889 wait complete. 2024-09-19T05:06:28.023 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:27 smithi043 ceph-mon[30800]: pgmap v57: 97 pgs: 97 active+clean; 1.0 GiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 MiB/s wr, 11 op/s 2024-09-19T05:06:28.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:27 smithi151 ceph-mon[35425]: pgmap v57: 97 pgs: 97 active+clean; 1.0 GiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 MiB/s wr, 11 op/s 2024-09-19T05:06:28.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:27 smithi096 ceph-mon[35085]: pgmap v57: 97 pgs: 97 active+clean; 1.0 GiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 MiB/s wr, 11 op/s 2024-09-19T05:06:29.394 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T05:06:29.395 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T05:06:30.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:29 smithi043 ceph-mon[30800]: pgmap v58: 97 pgs: 97 active+clean; 1.0 GiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.3 MiB/s wr, 10 op/s 2024-09-19T05:06:30.105 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:06:30.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:29 smithi151 ceph-mon[35425]: pgmap v58: 97 pgs: 97 active+clean; 1.0 GiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.3 MiB/s wr, 10 op/s 2024-09-19T05:06:30.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:29 smithi096 ceph-mon[35085]: pgmap v58: 97 pgs: 97 active+clean; 1.0 GiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.3 MiB/s wr, 10 op/s 2024-09-19T05:06:30.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.889+0000 7f0eae400640 1 -- 172.21.15.43:0/2523625076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ea8101de0 msgr2=0x7f0ea81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:30.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.889+0000 7f0eae400640 1 --2- 172.21.15.43:0/2523625076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ea8101de0 0x7f0ea81021c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0e98009900 tx=0x7f0e9802a3f0 comp rx=0 tx=0).stop 2024-09-19T05:06:30.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.890+0000 7f0eae400640 1 -- 172.21.15.43:0/2523625076 shutdown_connections 2024-09-19T05:06:30.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.890+0000 7f0eae400640 1 --2- 172.21.15.43:0/2523625076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ea8106f10 0x7f0ea810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:30.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.890+0000 7f0eae400640 1 --2- 172.21.15.43:0/2523625076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ea8102790 0x7f0ea81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:30.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.890+0000 7f0eae400640 1 --2- 172.21.15.43:0/2523625076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ea8101de0 0x7f0ea81021c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:30.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.890+0000 7f0eae400640 1 -- 172.21.15.43:0/2523625076 >> 172.21.15.43:0/2523625076 conn(0x7f0ea80fd8c0 msgr2=0x7f0ea80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:30.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.890+0000 7f0eae400640 1 -- 172.21.15.43:0/2523625076 shutdown_connections 2024-09-19T05:06:30.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.890+0000 7f0eae400640 1 -- 172.21.15.43:0/2523625076 wait complete. 2024-09-19T05:06:30.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.891+0000 7f0eae400640 1 Processor -- start 2024-09-19T05:06:30.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.891+0000 7f0eae400640 1 -- start start 2024-09-19T05:06:30.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.892+0000 7f0eae400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ea8101de0 0x7f0ea819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:30.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.892+0000 7f0eae400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ea8102790 0x7f0ea819f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:30.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.892+0000 7f0eae400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ea8106f10 0x7f0ea810db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:30.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.892+0000 7f0eae400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea8111f80 con 0x7f0ea8106f10 2024-09-19T05:06:30.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.892+0000 7f0eae400640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea8111e00 con 0x7f0ea8101de0 2024-09-19T05:06:30.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.892+0000 7f0eae400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea8112100 con 0x7f0ea8102790 2024-09-19T05:06:30.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.893+0000 7f0ea7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ea8106f10 0x7f0ea810db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:30.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.893+0000 7f0ea7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ea8106f10 0x7f0ea810db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51508/0 (socket says 172.21.15.43:51508) 2024-09-19T05:06:30.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.893+0000 7f0ea7e00640 1 -- 172.21.15.43:0/2826873134 learned_addr learned my addr 172.21.15.43:0/2826873134 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:30.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.893+0000 7f0ea6a00640 1 --2- 172.21.15.43:0/2826873134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ea8102790 0x7f0ea819f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:30.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.893+0000 7f0ea7e00640 1 -- 172.21.15.43:0/2826873134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ea8102790 msgr2=0x7f0ea819f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:30.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.893+0000 7f0ea7e00640 1 --2- 172.21.15.43:0/2826873134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ea8102790 0x7f0ea819f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:30.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.893+0000 7f0ea7400640 1 --2- 172.21.15.43:0/2826873134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ea8101de0 0x7f0ea819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:30.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.893+0000 7f0ea7e00640 1 -- 172.21.15.43:0/2826873134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ea8101de0 msgr2=0x7f0ea819f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:30.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.894+0000 7f0ea7e00640 1 --2- 172.21.15.43:0/2826873134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ea8101de0 0x7f0ea819f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:30.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.894+0000 7f0ea7e00640 1 -- 172.21.15.43:0/2826873134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea810e420 con 0x7f0ea8106f10 2024-09-19T05:06:30.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.894+0000 7f0ea7400640 1 --2- 172.21.15.43:0/2826873134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ea8101de0 0x7f0ea819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:06:30.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.894+0000 7f0ea7e00640 1 --2- 172.21.15.43:0/2826873134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ea8106f10 0x7f0ea810db90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0ea000eb10 tx=0x7f0ea000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:30.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.895+0000 7f0e9fe00640 1 -- 172.21.15.43:0/2826873134 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0004b20 con 0x7f0ea8106f10 2024-09-19T05:06:30.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.895+0000 7f0e9fe00640 1 -- 172.21.15.43:0/2826873134 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ea0004cc0 con 0x7f0ea8106f10 2024-09-19T05:06:30.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.895+0000 7f0e9fe00640 1 -- 172.21.15.43:0/2826873134 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0010760 con 0x7f0ea8106f10 2024-09-19T05:06:30.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.895+0000 7f0eae400640 1 -- 172.21.15.43:0/2826873134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ea810e710 con 0x7f0ea8106f10 2024-09-19T05:06:30.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.895+0000 7f0eae400640 1 -- 172.21.15.43:0/2826873134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ea819ad10 con 0x7f0ea8106f10 2024-09-19T05:06:30.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.900+0000 7f0eb0dee740 1 Processor -- start 2024-09-19T05:06:30.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.901+0000 7f0e9fe00640 1 -- 172.21.15.43:0/2826873134 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0ea0010900 con 0x7f0ea8106f10 2024-09-19T05:06:30.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.901+0000 7f0eb0dee740 1 -- start start 2024-09-19T05:06:30.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.902+0000 7f0eb0dee740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f864d9ac0 0x555f864d9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:30.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.902+0000 7f0eb0dee740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f864da3e0 0x555f864e4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:30.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.902+0000 7f0eb0dee740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f864e4ca0 0x555f864e7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:30.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.902+0000 7f0eb0dee740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555f86440290 con 0x555f864e4ca0 2024-09-19T05:06:30.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.902+0000 7f0eb0dee740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555f863e7b40 con 0x555f864da3e0 2024-09-19T05:06:30.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.902+0000 7f0eb0dee740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555f8643c250 con 0x555f864d9ac0 2024-09-19T05:06:30.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.902+0000 7f0ea6a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f864d9ac0 0x555f864d9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:30.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.903+0000 7f0ea6a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f864d9ac0 0x555f864d9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:57734/0 (socket says 172.21.15.43:57734) 2024-09-19T05:06:30.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.903+0000 7f0ea6a00640 1 -- 172.21.15.43:0/30093842 learned_addr learned my addr 172.21.15.43:0/30093842 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:30.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.903+0000 7f0ea7400640 1 --2- 172.21.15.43:0/30093842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f864da3e0 0x555f864e4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:30.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.903+0000 7f0ea6a00640 1 -- 172.21.15.43:0/30093842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f864da3e0 msgr2=0x555f864e4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:30.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.903+0000 7f0ea6a00640 1 --2- 172.21.15.43:0/30093842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f864da3e0 0x555f864e4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:30.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.903+0000 7f0ea6a00640 1 -- 172.21.15.43:0/30093842 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f864e4ca0 msgr2=0x555f864e7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:30.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.903+0000 7f0ea6a00640 1 --2- 172.21.15.43:0/30093842 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f864e4ca0 0x555f864e7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:30.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.903+0000 7f0ea6a00640 1 -- 172.21.15.43:0/30093842 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f861b16a0 con 0x555f864d9ac0 2024-09-19T05:06:30.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.903+0000 7f0e9fe00640 1 --2- 172.21.15.43:0/2826873134 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f0e8c0a3500 0x7f0e8c0a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:30.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.904+0000 7f0ea6a00640 1 --2- 172.21.15.43:0/30093842 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f864d9ac0 0x555f864d9ea0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0e940093e0 tx=0x7f0e94012820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64a21a358a46a36f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:30.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.904+0000 7f0e9fe00640 1 -- 172.21.15.43:0/2826873134 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f0ea00d0340 con 0x7f0ea8106f10 2024-09-19T05:06:30.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.904+0000 7f0ea7400640 1 --2- 172.21.15.43:0/2826873134 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f0e8c0a3500 0x7f0e8c0a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:30.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.904+0000 7f0e9cc00640 1 -- 172.21.15.43:0/30093842 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9400f630 con 0x555f864d9ac0 2024-09-19T05:06:30.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.904+0000 7f0e9cc00640 1 -- 172.21.15.43:0/30093842 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e94009e20 con 0x555f864d9ac0 2024-09-19T05:06:30.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.904+0000 7f0e9cc00640 1 -- 172.21.15.43:0/30093842 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9403cb30 con 0x555f864d9ac0 2024-09-19T05:06:30.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.906+0000 7f0eb0dee740 1 -- 172.21.15.43:0/30093842 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f864d9ac0 msgr2=0x555f864d9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:30.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.906+0000 7f0eb0dee740 1 --2- 172.21.15.43:0/30093842 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f864d9ac0 0x555f864d9ea0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0e940093e0 tx=0x7f0e94012820 comp rx=0 tx=0).stop 2024-09-19T05:06:30.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.906+0000 7f0eb0dee740 1 -- 172.21.15.43:0/30093842 shutdown_connections 2024-09-19T05:06:30.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.906+0000 7f0eb0dee740 1 --2- 172.21.15.43:0/30093842 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f864e4ca0 0x555f864e7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:30.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.906+0000 7f0eb0dee740 1 --2- 172.21.15.43:0/30093842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f864da3e0 0x555f864e4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:30.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.906+0000 7f0eb0dee740 1 --2- 172.21.15.43:0/30093842 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f864d9ac0 0x555f864d9ea0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:30.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.906+0000 7f0eb0dee740 1 -- 172.21.15.43:0/30093842 >> 172.21.15.43:0/30093842 conn(0x555f864d6050 msgr2=0x555f864d6430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:30.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.906+0000 7f0ea7400640 1 --2- 172.21.15.43:0/2826873134 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f0e8c0a3500 0x7f0e8c0a59c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x555f864d6eb0 tx=0x7f0e98005bd0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:30.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.906+0000 7f0eb0dee740 1 -- 172.21.15.43:0/30093842 shutdown_connections 2024-09-19T05:06:30.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.907+0000 7f0eb0dee740 1 -- 172.21.15.43:0/30093842 wait complete. 2024-09-19T05:06:30.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.907+0000 7f0eb0dee740 1 -- start start 2024-09-19T05:06:30.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.908+0000 7f0eb0dee740 1 Processor -- start 2024-09-19T05:06:30.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.908+0000 7f0eb0dee740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f864d9ac0 0x555f864e46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:30.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.908+0000 7f0eb0dee740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f864da3e0 0x555f864e1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:30.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.908+0000 7f0ea7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f864d9ac0 0x555f864e46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:30.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.909+0000 7f0ea7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f864d9ac0 0x555f864e46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:57738/0 (socket says 172.21.15.43:57738) 2024-09-19T05:06:30.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.909+0000 7f0ea7400640 1 -- 172.21.15.43:0/4013335316 learned_addr learned my addr 172.21.15.43:0/4013335316 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:30.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.909+0000 7f0eb0dee740 1 --2- 172.21.15.43:0/4013335316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f864e4ca0 0x555f864e1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:30.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.909+0000 7f0eb0dee740 1 -- 172.21.15.43:0/4013335316 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555f864e7850 con 0x555f864da3e0 2024-09-19T05:06:30.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.909+0000 7f0eb0dee740 1 -- 172.21.15.43:0/4013335316 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555f863fafa0 con 0x555f864e4ca0 2024-09-19T05:06:30.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.909+0000 7f0eb0dee740 1 -- 172.21.15.43:0/4013335316 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555f864e79d0 con 0x555f864d9ac0 2024-09-19T05:06:30.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.909+0000 7f0eb0dee740 10 client.0 _ll_register_callbacks cb 0x555f8614eba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:30.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.909+0000 7f0ea7e00640 1 --2- 172.21.15.43:0/4013335316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f864e4ca0 0x555f864e1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:30.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.910+0000 7f0ea7e00640 1 -- 172.21.15.43:0/4013335316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f864d9ac0 msgr2=0x555f864e46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:30.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.910+0000 7f0ea7e00640 1 --2- 172.21.15.43:0/4013335316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f864d9ac0 0x555f864e46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:30.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.910+0000 7f0ea7e00640 1 -- 172.21.15.43:0/4013335316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f864da3e0 msgr2=0x555f864e1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:30.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.910+0000 7f0ea7e00640 1 --2- 172.21.15.43:0/4013335316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f864da3e0 0x555f864e1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:30.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.910+0000 7f0ea7e00640 1 -- 172.21.15.43:0/4013335316 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f864e27b0 con 0x555f864e4ca0 2024-09-19T05:06:30.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.911+0000 7f0ea7e00640 1 --2- 172.21.15.43:0/4013335316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f864e4ca0 0x555f864e1f80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0ea00cdaa0 tx=0x7f0ea009c230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8956ed462e8dc0a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:30.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.911+0000 7f0e91600640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:06:30.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.911+0000 7f0e91600640 1 -- 172.21.15.43:0/4013335316 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea00b4070 con 0x555f864e4ca0 2024-09-19T05:06:30.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.911+0000 7f0e91600640 1 -- 172.21.15.43:0/4013335316 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f0ea00a4e10 con 0x555f864e4ca0 2024-09-19T05:06:30.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.911+0000 7f0eb0dee740 1 -- 172.21.15.43:0/4013335316 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e94009e20 con 0x555f864e4ca0 2024-09-19T05:06:30.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.912+0000 7f0e91600640 1 -- 172.21.15.43:0/4013335316 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0ea00a5090 con 0x555f864e4ca0 2024-09-19T05:06:30.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.912+0000 7f0eb0dee740 10 client.25592 fetch_fsmap learned FSMap version 29 2024-09-19T05:06:30.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.912+0000 7f0eb0dee740 1 -- 172.21.15.43:0/4013335316 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555f864f3890 con 0x555f864e4ca0 2024-09-19T05:06:30.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.912+0000 7f0e91600640 1 -- 172.21.15.43:0/4013335316 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e94009e20 con 0x555f864e4ca0 2024-09-19T05:06:30.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.913+0000 7f0e91600640 1 -- 172.21.15.43:0/4013335316 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0ea00b0c70 con 0x555f864e4ca0 2024-09-19T05:06:30.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.913+0000 7f0eb0dee740 10 client.25592 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:06:30.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.913+0000 7f0eb0dee740 10 client.25592 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:06:30.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.913+0000 7f0eb0dee740 20 client.25592 populate_metadata read hostname 'smithi043' 2024-09-19T05:06:30.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.913+0000 7f0eb0dee740 1 --2- 172.21.15.43:0/4013335316 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x555f864f4cf0 0x555f864f7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:30.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.913+0000 7f0eb0dee740 4 client.25592 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:06:30.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.913+0000 7f0eb0dee740 1 -- 172.21.15.43:0/4013335316 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f8644d800 con 0x555f864f4cf0 2024-09-19T05:06:30.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.913+0000 7f0ea7400640 1 --2- 172.21.15.43:0/4013335316 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x555f864f4cf0 0x555f864f7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:30.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.914+0000 7f0e91600640 10 client.25592 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:06:30.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.914+0000 7f0ea7400640 1 --2- 172.21.15.43:0/4013335316 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x555f864f4cf0 0x555f864f7180 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a649ec39e569aa server_cookie=a37e9e8b8dbdebb8 in_seq=0 out_seq=0 2024-09-19T05:06:30.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.914+0000 7f0e91600640 1 -- 172.21.15.43:0/4013335316 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555f8644d800 con 0x555f864f4cf0 2024-09-19T05:06:30.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.914+0000 7f0e91600640 10 client.25592 handle_command_reply: tid=0 2024-09-19T05:06:30.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.916+0000 7f0eb0dee740 1 client.25592 shutdown 2024-09-19T05:06:30.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.916+0000 7f0eb0dee740 1 -- 172.21.15.43:0/4013335316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f864e4ca0 msgr2=0x555f864e1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:30.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.916+0000 7f0eb0dee740 1 --2- 172.21.15.43:0/4013335316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f864e4ca0 0x555f864e1f80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0ea00cdaa0 tx=0x7f0ea009c230 comp rx=0 tx=0).stop 2024-09-19T05:06:30.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.916+0000 7f0eb0dee740 1 -- 172.21.15.43:0/4013335316 shutdown_connections 2024-09-19T05:06:30.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.916+0000 7f0eb0dee740 1 --2- 172.21.15.43:0/4013335316 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x555f864f4cf0 0x555f864f7180 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:30.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.916+0000 7f0eb0dee740 1 --2- 172.21.15.43:0/4013335316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f864e4ca0 0x555f864e1f80 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:30.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.917+0000 7f0e91600640 0 client.25592 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:06:30.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.917+0000 7f0eb0dee740 1 --2- 172.21.15.43:0/4013335316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f864da3e0 0x555f864e1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:30.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.917+0000 7f0eb0dee740 1 --2- 172.21.15.43:0/4013335316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f864d9ac0 0x555f864e46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:30.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.917+0000 7f0eb0dee740 1 -- 172.21.15.43:0/4013335316 >> 172.21.15.43:0/4013335316 conn(0x555f864d6050 msgr2=0x555f864dbd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:30.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.917+0000 7f0eb0dee740 1 -- 172.21.15.43:0/4013335316 shutdown_connections 2024-09-19T05:06:30.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.917+0000 7f0eb0dee740 1 -- 172.21.15.43:0/4013335316 wait complete. 2024-09-19T05:06:30.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.918+0000 7f0eb0dee740 20 client.25592 trim_cache size 0 max 16384 2024-09-19T05:06:30.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.949+0000 7f0eb0dee740 1 Processor -- start 2024-09-19T05:06:30.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.949+0000 7f0eb0dee740 1 -- start start 2024-09-19T05:06:30.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.950+0000 7f0eb0dee740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f864d9ac0 0x555f864fd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:30.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.950+0000 7f0eb0dee740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f864da3e0 0x555f864fdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:30.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.950+0000 7f0eb0dee740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f864e4ca0 0x555f864debd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:30.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.950+0000 7f0eb0dee740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea8073790 con 0x555f864da3e0 2024-09-19T05:06:30.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.950+0000 7f0eb0dee740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea81b5b30 con 0x555f864e4ca0 2024-09-19T05:06:30.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.950+0000 7f0eb0dee740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea8073e60 con 0x555f864d9ac0 2024-09-19T05:06:30.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.950+0000 7f0ea7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f864d9ac0 0x555f864fd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:30.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.951+0000 7f0ea7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f864d9ac0 0x555f864fd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:57752/0 (socket says 172.21.15.43:57752) 2024-09-19T05:06:30.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.951+0000 7f0ea7400640 1 -- 172.21.15.43:0/3125923641 learned_addr learned my addr 172.21.15.43:0/3125923641 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:30.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.951+0000 7f0ea7e00640 1 --2- 172.21.15.43:0/3125923641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f864e4ca0 0x555f864debd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:30.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.951+0000 7f0ea7400640 1 -- 172.21.15.43:0/3125923641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f864e4ca0 msgr2=0x555f864debd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:30.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.951+0000 7f0ea7400640 1 --2- 172.21.15.43:0/3125923641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f864e4ca0 0x555f864debd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:30.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.951+0000 7f0ea7400640 1 -- 172.21.15.43:0/3125923641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f864da3e0 msgr2=0x555f864fdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:30.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.951+0000 7f0ea7400640 1 --2- 172.21.15.43:0/3125923641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f864da3e0 0x555f864fdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:30.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.951+0000 7f0ea7400640 1 -- 172.21.15.43:0/3125923641 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f864dfec0 con 0x555f864d9ac0 2024-09-19T05:06:30.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.954+0000 7f0ea7400640 1 --2- 172.21.15.43:0/3125923641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f864d9ac0 0x555f864fd670 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0e9802a910 tx=0x7f0e98040000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c75c416fd64dfae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:30.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.955+0000 7f0e93e00640 1 -- 172.21.15.43:0/3125923641 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e98035040 con 0x555f864d9ac0 2024-09-19T05:06:30.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.955+0000 7f0e93e00640 1 -- 172.21.15.43:0/3125923641 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e98032aa0 con 0x555f864d9ac0 2024-09-19T05:06:30.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.955+0000 7f0e93e00640 1 -- 172.21.15.43:0/3125923641 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e98004c90 con 0x555f864d9ac0 2024-09-19T05:06:30.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.957+0000 7f0eb0dee740 1 -- 172.21.15.43:0/3125923641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f864d9ac0 msgr2=0x555f864fd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:30.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.957+0000 7f0eb0dee740 1 --2- 172.21.15.43:0/3125923641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f864d9ac0 0x555f864fd670 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0e9802a910 tx=0x7f0e98040000 comp rx=0 tx=0).stop 2024-09-19T05:06:30.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.959+0000 7f0eb0dee740 1 -- 172.21.15.43:0/3125923641 shutdown_connections 2024-09-19T05:06:30.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.959+0000 7f0eb0dee740 1 --2- 172.21.15.43:0/3125923641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f864e4ca0 0x555f864debd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:30.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.959+0000 7f0eb0dee740 1 --2- 172.21.15.43:0/3125923641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f864da3e0 0x555f864fdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:30.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.959+0000 7f0eb0dee740 1 --2- 172.21.15.43:0/3125923641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f864d9ac0 0x555f864fd670 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:30.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.959+0000 7f0eb0dee740 1 -- 172.21.15.43:0/3125923641 >> 172.21.15.43:0/3125923641 conn(0x555f864d6050 msgr2=0x555f864dcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:30.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.967+0000 7f0eb0dee740 1 -- 172.21.15.43:0/3125923641 shutdown_connections 2024-09-19T05:06:30.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.972+0000 7f0eb0dee740 1 -- 172.21.15.43:0/3125923641 wait complete. 2024-09-19T05:06:30.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.973+0000 7f0eb0dee740 1 -- start start 2024-09-19T05:06:30.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.973+0000 7f0eb0dee740 1 Processor -- start 2024-09-19T05:06:30.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.975+0000 7f0eb0dee740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f864d9ac0 0x555f864e8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:30.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.975+0000 7f0eb0dee740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f864da3e0 0x555f864e9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:30.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.976+0000 7f0eb0dee740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f864e4ca0 0x555f8655f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:30.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.976+0000 7f0ea6a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f864da3e0 0x555f864e9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:30.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.976+0000 7f0ea6a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f864da3e0 0x555f864e9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:46522/0 (socket says 172.21.15.43:46522) 2024-09-19T05:06:30.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.976+0000 7f0eb0dee740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea81b5460 con 0x555f864d9ac0 2024-09-19T05:06:30.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.976+0000 7f0eb0dee740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea81b4d90 con 0x555f864da3e0 2024-09-19T05:06:30.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.976+0000 7f0eb0dee740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea8071770 con 0x555f864e4ca0 2024-09-19T05:06:30.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.976+0000 7f0ea7e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f864e4ca0 0x555f8655f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:30.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.976+0000 7f0ea7e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f864e4ca0 0x555f8655f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:57758/0 (socket says 172.21.15.43:57758) 2024-09-19T05:06:30.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.976+0000 7f0eb0dee740 10 client.0 _ll_register_callbacks cb 0x7f0ea009c400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:30.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.976+0000 7f0ea7e00640 1 -- 172.21.15.43:0/3491033438 learned_addr learned my addr 172.21.15.43:0/3491033438 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:30.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.977+0000 7f0ea7e00640 1 -- 172.21.15.43:0/3491033438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f864da3e0 msgr2=0x555f864e9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:30.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.977+0000 7f0ea7e00640 1 --2- 172.21.15.43:0/3491033438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f864da3e0 0x555f864e9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:30.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.977+0000 7f0ea7e00640 1 -- 172.21.15.43:0/3491033438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f864d9ac0 msgr2=0x555f864e8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:30.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.977+0000 7f0ea7e00640 1 --2- 172.21.15.43:0/3491033438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f864d9ac0 0x555f864e8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:30.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.978+0000 7f0ea7e00640 1 -- 172.21.15.43:0/3491033438 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f8655fb80 con 0x555f864e4ca0 2024-09-19T05:06:30.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.982+0000 7f0e91600640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:06:30.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.982+0000 7f0ea7e00640 1 --2- 172.21.15.43:0/3491033438 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f864e4ca0 0x555f8655f640 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0ea009c5c0 tx=0x7f0ea00d02d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89c55721cea3ad0b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:30.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.985+0000 7f0e91600640 1 -- 172.21.15.43:0/3491033438 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea00a52a0 con 0x555f864e4ca0 2024-09-19T05:06:30.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.985+0000 7f0e91600640 1 -- 172.21.15.43:0/3491033438 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f0ea009b540 con 0x555f864e4ca0 2024-09-19T05:06:30.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.985+0000 7f0eb0dee740 1 -- 172.21.15.43:0/3491033438 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e98032aa0 con 0x555f864e4ca0 2024-09-19T05:06:30.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.987+0000 7f0e91600640 1 -- 172.21.15.43:0/3491033438 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0ea0084050 con 0x555f864e4ca0 2024-09-19T05:06:30.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.987+0000 7f0eb0dee740 10 client.25561 fetch_fsmap learned FSMap version 29 2024-09-19T05:06:30.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.987+0000 7f0eb0dee740 1 -- 172.21.15.43:0/3491033438 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555f86560d40 con 0x555f864e4ca0 2024-09-19T05:06:30.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.988+0000 7f0e91600640 1 -- 172.21.15.43:0/3491033438 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e98032aa0 con 0x555f864e4ca0 2024-09-19T05:06:30.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.988+0000 7f0e91600640 1 -- 172.21.15.43:0/3491033438 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0ea007d070 con 0x555f864e4ca0 2024-09-19T05:06:30.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.990+0000 7f0eb0dee740 10 client.25561 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:06:30.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.990+0000 7f0eb0dee740 10 client.25561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:06:30.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.990+0000 7f0eb0dee740 20 client.25561 populate_metadata read hostname 'smithi043' 2024-09-19T05:06:30.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.990+0000 7f0eb0dee740 1 --2- 172.21.15.43:0/3491033438 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x555f865612d0 0x555f865636c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:30.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.990+0000 7f0eb0dee740 4 client.25561 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T05:06:30.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.990+0000 7f0eb0dee740 1 -- 172.21.15.43:0/3491033438 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555f86447d10 con 0x555f865612d0 2024-09-19T05:06:30.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.991+0000 7f0ea7400640 1 --2- 172.21.15.43:0/3491033438 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x555f865612d0 0x555f865636c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:30.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.993+0000 7f0ea7400640 1 --2- 172.21.15.43:0/3491033438 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x555f865612d0 0x555f865636c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94bf5f2c6d07ac7c server_cookie=d8375e72d5599484 in_seq=0 out_seq=0 2024-09-19T05:06:30.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.993+0000 7f0e91600640 10 client.25561 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:06:30.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.995+0000 7f0e91600640 1 -- 172.21.15.43:0/3491033438 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555f86447d10 con 0x555f865612d0 2024-09-19T05:06:30.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.995+0000 7f0e91600640 10 client.25561 handle_command_reply: tid=0 2024-09-19T05:06:30.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.995+0000 7f0eb0dee740 1 client.25561 shutdown 2024-09-19T05:06:30.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.996+0000 7f0eb0dee740 1 -- 172.21.15.43:0/3491033438 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f864e4ca0 msgr2=0x555f8655f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:30.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.996+0000 7f0eb0dee740 1 --2- 172.21.15.43:0/3491033438 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f864e4ca0 0x555f8655f640 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0ea009c5c0 tx=0x7f0ea00d02d0 comp rx=0 tx=0).stop 2024-09-19T05:06:30.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.997+0000 7f0eb0dee740 1 -- 172.21.15.43:0/3491033438 shutdown_connections 2024-09-19T05:06:30.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.997+0000 7f0eb0dee740 1 --2- 172.21.15.43:0/3491033438 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x555f865612d0 0x555f865636c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:30.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.997+0000 7f0eb0dee740 1 --2- 172.21.15.43:0/3491033438 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f864e4ca0 0x555f8655f640 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:30.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.997+0000 7f0eb0dee740 1 --2- 172.21.15.43:0/3491033438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f864da3e0 0x555f864e9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:30.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.997+0000 7f0eb0dee740 1 --2- 172.21.15.43:0/3491033438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f864d9ac0 0x555f864e8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:30.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.997+0000 7f0eb0dee740 1 -- 172.21.15.43:0/3491033438 >> 172.21.15.43:0/3491033438 conn(0x555f864d6050 msgr2=0x555f864e31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:30.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.997+0000 7f0e91600640 0 client.25561 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:06:31.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.998+0000 7f0eb0dee740 1 -- 172.21.15.43:0/3491033438 shutdown_connections 2024-09-19T05:06:31.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.998+0000 7f0eb0dee740 1 -- 172.21.15.43:0/3491033438 wait complete. 2024-09-19T05:06:31.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:30.999+0000 7f0eb0dee740 20 client.25561 trim_cache size 0 max 16384 2024-09-19T05:06:31.001 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:06:31.001 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-09-19T05:06:31.001 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "43fc6c01-396f-4045-a20b-7290becbadf8", 2024-09-19T05:06:31.001 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-09-19T05:06:31.001 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:06:31.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:31.004+0000 7f0eae400640 1 -- 172.21.15.43:0/2826873134 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f0e8c0a3500 msgr2=0x7f0e8c0a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:31.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:31.004+0000 7f0eae400640 1 --2- 172.21.15.43:0/2826873134 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f0e8c0a3500 0x7f0e8c0a59c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x555f864d6eb0 tx=0x7f0e98005bd0 comp rx=0 tx=0).stop 2024-09-19T05:06:31.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:31.004+0000 7f0eae400640 1 -- 172.21.15.43:0/2826873134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ea8106f10 msgr2=0x7f0ea810db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:31.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:31.004+0000 7f0eae400640 1 --2- 172.21.15.43:0/2826873134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ea8106f10 0x7f0ea810db90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0ea000eb10 tx=0x7f0ea000c2d0 comp rx=0 tx=0).stop 2024-09-19T05:06:31.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:31.004+0000 7f0eae400640 1 -- 172.21.15.43:0/2826873134 shutdown_connections 2024-09-19T05:06:31.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:31.004+0000 7f0eae400640 1 --2- 172.21.15.43:0/2826873134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ea8106f10 0x7f0ea810db90 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:31.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:31.004+0000 7f0eae400640 1 --2- 172.21.15.43:0/2826873134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ea8102790 0x7f0ea819f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:31.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:31.004+0000 7f0eae400640 1 --2- 172.21.15.43:0/2826873134 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f0e8c0a3500 0x7f0e8c0a59c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:31.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:31.004+0000 7f0eae400640 1 --2- 172.21.15.43:0/2826873134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ea8101de0 0x7f0ea819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:31.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:31.004+0000 7f0eae400640 1 -- 172.21.15.43:0/2826873134 >> 172.21.15.43:0/2826873134 conn(0x7f0ea80fd8c0 msgr2=0x7f0ea80feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:31.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:31.004+0000 7f0eae400640 1 -- 172.21.15.43:0/2826873134 shutdown_connections 2024-09-19T05:06:31.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:31.004+0000 7f0eae400640 1 -- 172.21.15.43:0/2826873134 wait complete. 2024-09-19T05:06:31.823 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-19T05:06:31.823 INFO:tasks.workunit.client.0.smithi043.stdout: Children see throughput for 1 initial writers = 227359.91 kB/sec 2024-09-19T05:06:31.824 INFO:tasks.workunit.client.0.smithi043.stdout: Parent sees throughput for 1 initial writers = 227323.30 kB/sec 2024-09-19T05:06:31.824 INFO:tasks.workunit.client.0.smithi043.stdout: Min throughput per process = 227359.91 kB/sec 2024-09-19T05:06:31.824 INFO:tasks.workunit.client.0.smithi043.stdout: Max throughput per process = 227359.91 kB/sec 2024-09-19T05:06:31.824 INFO:tasks.workunit.client.0.smithi043.stdout: Avg throughput per process = 227359.91 kB/sec 2024-09-19T05:06:31.824 INFO:tasks.workunit.client.0.smithi043.stdout: Min xfer = 1048576.00 kB 2024-09-19T05:06:32.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:31 smithi043 ceph-mon[30800]: pgmap v59: 97 pgs: 97 active+clean; 290 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 30 MiB/s wr, 43 op/s 2024-09-19T05:06:32.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:31 smithi151 ceph-mon[35425]: pgmap v59: 97 pgs: 97 active+clean; 290 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 30 MiB/s wr, 43 op/s 2024-09-19T05:06:32.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:31 smithi096 ceph-mon[35085]: pgmap v59: 97 pgs: 97 active+clean; 290 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 30 MiB/s wr, 43 op/s 2024-09-19T05:06:32.708 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:06:33.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:32 smithi043 ceph-mon[30800]: scrub queued for path: / 2024-09-19T05:06:33.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:32 smithi043 ceph-mon[30800]: scrub summary: idle+waiting paths [/] 2024-09-19T05:06:33.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:32 smithi043 ceph-mon[30800]: scrub summary: active paths [/] 2024-09-19T05:06:33.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:32 smithi043 ceph-mon[30800]: scrub summary: idle+waiting paths [/] 2024-09-19T05:06:33.094 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:06:33.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:32 smithi151 ceph-mon[35425]: scrub queued for path: / 2024-09-19T05:06:33.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:32 smithi151 ceph-mon[35425]: scrub summary: idle+waiting paths [/] 2024-09-19T05:06:33.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:32 smithi151 ceph-mon[35425]: scrub summary: active paths [/] 2024-09-19T05:06:33.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:32 smithi151 ceph-mon[35425]: scrub summary: idle+waiting paths [/] 2024-09-19T05:06:33.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:32 smithi096 ceph-mon[35085]: scrub queued for path: / 2024-09-19T05:06:33.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:32 smithi096 ceph-mon[35085]: scrub summary: idle+waiting paths [/] 2024-09-19T05:06:33.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:32 smithi096 ceph-mon[35085]: scrub summary: active paths [/] 2024-09-19T05:06:33.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:32 smithi096 ceph-mon[35085]: scrub summary: idle+waiting paths [/] 2024-09-19T05:06:33.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.636+0000 7f4e04400640 1 -- 172.21.15.43:0/1878191956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4dfc112b90 msgr2=0x7f4dfc114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:33.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.636+0000 7f4e04400640 1 --2- 172.21.15.43:0/1878191956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4dfc112b90 0x7f4dfc114f80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4df4009900 tx=0x7f4df402a480 comp rx=0 tx=0).stop 2024-09-19T05:06:33.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.636+0000 7f4e04400640 1 -- 172.21.15.43:0/1878191956 shutdown_connections 2024-09-19T05:06:33.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.636+0000 7f4e04400640 1 --2- 172.21.15.43:0/1878191956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4dfc112b90 0x7f4dfc114f80 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:33.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.636+0000 7f4e04400640 1 --2- 172.21.15.43:0/1878191956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4dfc074eb0 0x7f4dfc112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:33.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.636+0000 7f4e04400640 1 --2- 172.21.15.43:0/1878191956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4dfc074590 0x7f4dfc074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:33.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.636+0000 7f4e04400640 1 -- 172.21.15.43:0/1878191956 >> 172.21.15.43:0/1878191956 conn(0x7f4dfc0fd8c0 msgr2=0x7f4dfc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:33.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.636+0000 7f4e04400640 1 -- 172.21.15.43:0/1878191956 shutdown_connections 2024-09-19T05:06:33.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.636+0000 7f4e04400640 1 -- 172.21.15.43:0/1878191956 wait complete. 2024-09-19T05:06:33.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.637+0000 7f4e04400640 1 Processor -- start 2024-09-19T05:06:33.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.637+0000 7f4e04400640 1 -- start start 2024-09-19T05:06:33.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.638+0000 7f4e04400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4dfc074590 0x7f4dfc1a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:33.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.638+0000 7f4e04400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4dfc074eb0 0x7f4dfc1a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:33.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.638+0000 7f4e04400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4dfc112b90 0x7f4dfc19e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:33.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.638+0000 7f4e04400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc118560 con 0x7f4dfc074590 2024-09-19T05:06:33.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.638+0000 7f4e04400640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc1183e0 con 0x7f4dfc074eb0 2024-09-19T05:06:33.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.638+0000 7f4e04400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc1186e0 con 0x7f4dfc112b90 2024-09-19T05:06:33.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.638+0000 7f4e01a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4dfc074590 0x7f4dfc1a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:33.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.638+0000 7f4e01a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4dfc074590 0x7f4dfc1a4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51550/0 (socket says 172.21.15.43:51550) 2024-09-19T05:06:33.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.638+0000 7f4e01a00640 1 -- 172.21.15.43:0/1553532272 learned_addr learned my addr 172.21.15.43:0/1553532272 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:33.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.638+0000 7f4e01000640 1 --2- 172.21.15.43:0/1553532272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4dfc074eb0 0x7f4dfc1a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:33.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.638+0000 7f4e02400640 1 --2- 172.21.15.43:0/1553532272 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4dfc112b90 0x7f4dfc19e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:33.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.639+0000 7f4e01a00640 1 -- 172.21.15.43:0/1553532272 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4dfc112b90 msgr2=0x7f4dfc19e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:33.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.639+0000 7f4e01a00640 1 --2- 172.21.15.43:0/1553532272 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4dfc112b90 0x7f4dfc19e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:33.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.639+0000 7f4e01a00640 1 -- 172.21.15.43:0/1553532272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4dfc074eb0 msgr2=0x7f4dfc1a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:33.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.639+0000 7f4e01a00640 1 --2- 172.21.15.43:0/1553532272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4dfc074eb0 0x7f4dfc1a4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:33.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.639+0000 7f4e01a00640 1 -- 172.21.15.43:0/1553532272 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dfc19eff0 con 0x7f4dfc074590 2024-09-19T05:06:33.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.639+0000 7f4e01000640 1 --2- 172.21.15.43:0/1553532272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4dfc074eb0 0x7f4dfc1a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:33.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.639+0000 7f4e01a00640 1 --2- 172.21.15.43:0/1553532272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4dfc074590 0x7f4dfc1a4700 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4df000eb90 tx=0x7f4df000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:33.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.639+0000 7f4dfa000640 1 -- 172.21.15.43:0/1553532272 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df0004d20 con 0x7f4dfc074590 2024-09-19T05:06:33.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.639+0000 7f4dfa000640 1 -- 172.21.15.43:0/1553532272 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4df000ee90 con 0x7f4dfc074590 2024-09-19T05:06:33.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.639+0000 7f4e04400640 1 -- 172.21.15.43:0/1553532272 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dfc19f2e0 con 0x7f4dfc074590 2024-09-19T05:06:33.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.639+0000 7f4dfa000640 1 -- 172.21.15.43:0/1553532272 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df0010830 con 0x7f4dfc074590 2024-09-19T05:06:33.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.639+0000 7f4e04400640 1 -- 172.21.15.43:0/1553532272 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dfc1ab5d0 con 0x7f4dfc074590 2024-09-19T05:06:33.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.642+0000 7f4dfa000640 1 -- 172.21.15.43:0/1553532272 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4df0005020 con 0x7f4dfc074590 2024-09-19T05:06:33.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.643+0000 7f4e069fd740 1 Processor -- start 2024-09-19T05:06:33.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.643+0000 7f4e069fd740 1 -- start start 2024-09-19T05:06:33.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.643+0000 7f4e069fd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a833c09b90 0x55a833c09f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:33.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.643+0000 7f4e069fd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a833c0a4b0 0x55a833c14830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:33.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.643+0000 7f4e069fd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a833c14d70 0x55a833c17160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:33.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.643+0000 7f4e069fd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a833b703e0 con 0x55a833c09b90 2024-09-19T05:06:33.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.643+0000 7f4e069fd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a833b17d80 con 0x55a833c0a4b0 2024-09-19T05:06:33.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.643+0000 7f4e069fd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a833b6c3a0 con 0x55a833c14d70 2024-09-19T05:06:33.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.643+0000 7f4e02400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a833c09b90 0x55a833c09f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:33.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.643+0000 7f4e02400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a833c09b90 0x55a833c09f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51560/0 (socket says 172.21.15.43:51560) 2024-09-19T05:06:33.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.643+0000 7f4e02400640 1 -- 172.21.15.43:0/1412782324 learned_addr learned my addr 172.21.15.43:0/1412782324 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:33.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.644+0000 7f4e01a00640 1 --2- 172.21.15.43:0/1412782324 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a833c0a4b0 0x55a833c14830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:33.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.644+0000 7f4e01000640 1 --2- 172.21.15.43:0/1412782324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a833c14d70 0x55a833c17160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:33.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.644+0000 7f4e01000640 1 -- 172.21.15.43:0/1412782324 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a833c0a4b0 msgr2=0x55a833c14830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:33.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.644+0000 7f4e01000640 1 --2- 172.21.15.43:0/1412782324 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a833c0a4b0 0x55a833c14830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:33.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.644+0000 7f4e01000640 1 -- 172.21.15.43:0/1412782324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a833c09b90 msgr2=0x55a833c09f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:33.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.644+0000 7f4e01000640 1 --2- 172.21.15.43:0/1412782324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a833c09b90 0x55a833c09f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:33.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.644+0000 7f4e01000640 1 -- 172.21.15.43:0/1412782324 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a833b36ae0 con 0x55a833c14d70 2024-09-19T05:06:33.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.644+0000 7f4e02400640 1 --2- 172.21.15.43:0/1412782324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a833c09b90 0x55a833c09f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:33.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.644+0000 7f4dfa000640 1 --2- 172.21.15.43:0/1553532272 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f4de40a3430 0x7f4de40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:33.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.644+0000 7f4dfa000640 1 -- 172.21.15.43:0/1553532272 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f4df00d0310 con 0x7f4dfc074590 2024-09-19T05:06:33.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.644+0000 7f4e01000640 1 --2- 172.21.15.43:0/1412782324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a833c14d70 0x55a833c17160 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc1a00b0 tx=0x7f4de8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1c00b2e273f8f68 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:33.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.645+0000 7f4e02400640 1 --2- 172.21.15.43:0/1553532272 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f4de40a3430 0x7f4de40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:33.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.645+0000 7f4deea00640 1 -- 172.21.15.43:0/1412782324 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de8004230 con 0x55a833c14d70 2024-09-19T05:06:33.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.645+0000 7f4deea00640 1 -- 172.21.15.43:0/1412782324 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4de80043d0 con 0x55a833c14d70 2024-09-19T05:06:33.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.645+0000 7f4deea00640 1 -- 172.21.15.43:0/1412782324 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de80035f0 con 0x55a833c14d70 2024-09-19T05:06:33.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.646+0000 7f4e069fd740 1 -- 172.21.15.43:0/1412782324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a833c14d70 msgr2=0x55a833c17160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:33.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.646+0000 7f4e069fd740 1 --2- 172.21.15.43:0/1412782324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a833c14d70 0x55a833c17160 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc1a00b0 tx=0x7f4de8011810 comp rx=0 tx=0).stop 2024-09-19T05:06:33.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.646+0000 7f4e02400640 1 --2- 172.21.15.43:0/1553532272 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f4de40a3430 0x7f4de40a58f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4df4005ec0 tx=0x7f4df4005d70 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:33.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.646+0000 7f4e069fd740 1 -- 172.21.15.43:0/1412782324 shutdown_connections 2024-09-19T05:06:33.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.646+0000 7f4e069fd740 1 --2- 172.21.15.43:0/1412782324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a833c14d70 0x55a833c17160 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:33.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.646+0000 7f4e069fd740 1 --2- 172.21.15.43:0/1412782324 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a833c0a4b0 0x55a833c14830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:33.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.646+0000 7f4e069fd740 1 --2- 172.21.15.43:0/1412782324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a833c09b90 0x55a833c09f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:33.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.646+0000 7f4e069fd740 1 -- 172.21.15.43:0/1412782324 >> 172.21.15.43:0/1412782324 conn(0x55a833c06120 msgr2=0x55a833c06500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:33.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.646+0000 7f4e069fd740 1 -- 172.21.15.43:0/1412782324 shutdown_connections 2024-09-19T05:06:33.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.646+0000 7f4e069fd740 1 -- 172.21.15.43:0/1412782324 wait complete. 2024-09-19T05:06:33.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.646+0000 7f4e069fd740 1 -- start start 2024-09-19T05:06:33.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.647+0000 7f4e069fd740 1 Processor -- start 2024-09-19T05:06:33.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.647+0000 7f4e069fd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a833c09b90 0x55a833c11c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:33.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.647+0000 7f4e069fd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a833c0a4b0 0x55a833c12150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:33.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.647+0000 7f4e069fd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a833c14d70 0x55a833c12690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:33.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.647+0000 7f4e069fd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a833c179e0 con 0x55a833c0a4b0 2024-09-19T05:06:33.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.647+0000 7f4e069fd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a833b2b0f0 con 0x55a833c09b90 2024-09-19T05:06:33.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.647+0000 7f4e069fd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a833c17b60 con 0x55a833c14d70 2024-09-19T05:06:33.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.647+0000 7f4e069fd740 10 client.0 _ll_register_callbacks cb 0x55a833b1c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:33.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.647+0000 7f4e01000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a833c0a4b0 0x55a833c12150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:33.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.647+0000 7f4e01000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a833c0a4b0 0x55a833c12150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51568/0 (socket says 172.21.15.43:51568) 2024-09-19T05:06:33.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.647+0000 7f4e01a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a833c09b90 0x55a833c11c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:33.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.647+0000 7f4e01000640 1 -- 172.21.15.43:0/1205628688 learned_addr learned my addr 172.21.15.43:0/1205628688 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:33.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.648+0000 7f4e02400640 1 --2- 172.21.15.43:0/1205628688 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a833c14d70 0x55a833c12690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:33.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.648+0000 7f4e01a00640 1 -- 172.21.15.43:0/1205628688 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a833c14d70 msgr2=0x55a833c12690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:33.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.648+0000 7f4e01a00640 1 --2- 172.21.15.43:0/1205628688 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a833c14d70 0x55a833c12690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:33.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.648+0000 7f4e01a00640 1 -- 172.21.15.43:0/1205628688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a833c0a4b0 msgr2=0x55a833c12150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:33.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.648+0000 7f4e01a00640 1 --2- 172.21.15.43:0/1205628688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a833c0a4b0 0x55a833c12150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:33.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.648+0000 7f4e01a00640 1 -- 172.21.15.43:0/1205628688 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8338e13c0 con 0x55a833c09b90 2024-09-19T05:06:33.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.648+0000 7f4e01000640 1 --2- 172.21.15.43:0/1205628688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a833c0a4b0 0x55a833c12150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:33.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.648+0000 7f4e01a00640 1 --2- 172.21.15.43:0/1205628688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a833c09b90 0x55a833c11c10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4df00cda10 tx=0x7f4df0009200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=914f7859ab90f6f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:33.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.648+0000 7f4de3400640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:06:33.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.648+0000 7f4de3400640 1 -- 172.21.15.43:0/1205628688 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df009c370 con 0x55a833c09b90 2024-09-19T05:06:33.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.648+0000 7f4de3400640 1 -- 172.21.15.43:0/1205628688 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f4df0005590 con 0x55a833c09b90 2024-09-19T05:06:33.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.648+0000 7f4e069fd740 1 -- 172.21.15.43:0/1205628688 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4de80043d0 con 0x55a833c09b90 2024-09-19T05:06:33.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.649+0000 7f4de3400640 1 -- 172.21.15.43:0/1205628688 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4df009bf00 con 0x55a833c09b90 2024-09-19T05:06:33.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.649+0000 7f4e069fd740 10 client.25610 fetch_fsmap learned FSMap version 29 2024-09-19T05:06:33.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.649+0000 7f4e069fd740 1 -- 172.21.15.43:0/1205628688 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a833c236a0 con 0x55a833c09b90 2024-09-19T05:06:33.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.649+0000 7f4de3400640 1 -- 172.21.15.43:0/1205628688 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4de80043d0 con 0x55a833c09b90 2024-09-19T05:06:33.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.649+0000 7f4de3400640 1 -- 172.21.15.43:0/1205628688 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4df00cdc90 con 0x55a833c09b90 2024-09-19T05:06:33.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.649+0000 7f4e069fd740 10 client.25610 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:06:33.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.649+0000 7f4e069fd740 10 client.25610 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:06:33.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.649+0000 7f4e069fd740 20 client.25610 populate_metadata read hostname 'smithi043' 2024-09-19T05:06:33.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.649+0000 7f4e069fd740 1 --2- 172.21.15.43:0/1205628688 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55a833c24ab0 0x55a833c26f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:33.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.649+0000 7f4e069fd740 4 client.25610 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:06:33.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.649+0000 7f4e069fd740 1 -- 172.21.15.43:0/1205628688 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a833b7d950 con 0x55a833c24ab0 2024-09-19T05:06:33.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.650+0000 7f4e01000640 1 --2- 172.21.15.43:0/1205628688 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55a833c24ab0 0x55a833c26f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:33.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.650+0000 7f4e01000640 1 --2- 172.21.15.43:0/1205628688 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55a833c24ab0 0x55a833c26f20 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81b68fe0ea67c8fd server_cookie=e58f7dae48ce1be7 in_seq=0 out_seq=0 2024-09-19T05:06:33.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.650+0000 7f4de3400640 10 client.25610 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:06:33.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.651+0000 7f4de3400640 1 -- 172.21.15.43:0/1205628688 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a833b7d950 con 0x55a833c24ab0 2024-09-19T05:06:33.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.651+0000 7f4de3400640 10 client.25610 handle_command_reply: tid=0 2024-09-19T05:06:33.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.651+0000 7f4e069fd740 1 client.25610 shutdown 2024-09-19T05:06:33.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.652+0000 7f4e069fd740 1 -- 172.21.15.43:0/1205628688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a833c09b90 msgr2=0x55a833c11c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:33.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.652+0000 7f4e069fd740 1 --2- 172.21.15.43:0/1205628688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a833c09b90 0x55a833c11c10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4df00cda10 tx=0x7f4df0009200 comp rx=0 tx=0).stop 2024-09-19T05:06:33.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.652+0000 7f4e069fd740 1 -- 172.21.15.43:0/1205628688 shutdown_connections 2024-09-19T05:06:33.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.652+0000 7f4e069fd740 1 --2- 172.21.15.43:0/1205628688 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55a833c24ab0 0x55a833c26f20 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:33.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.652+0000 7f4e069fd740 1 --2- 172.21.15.43:0/1205628688 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a833c14d70 0x55a833c12690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:33.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.652+0000 7f4e069fd740 1 --2- 172.21.15.43:0/1205628688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a833c0a4b0 0x55a833c12150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:33.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.652+0000 7f4e069fd740 1 --2- 172.21.15.43:0/1205628688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a833c09b90 0x55a833c11c10 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:33.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.652+0000 7f4de3400640 0 client.25610 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:06:33.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.652+0000 7f4e069fd740 1 -- 172.21.15.43:0/1205628688 >> 172.21.15.43:0/1205628688 conn(0x55a833c06120 msgr2=0x55a833c0be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:33.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.652+0000 7f4e069fd740 1 -- 172.21.15.43:0/1205628688 shutdown_connections 2024-09-19T05:06:33.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.652+0000 7f4e069fd740 1 -- 172.21.15.43:0/1205628688 wait complete. 2024-09-19T05:06:33.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.652+0000 7f4e069fd740 20 client.25610 trim_cache size 0 max 16384 2024-09-19T05:06:33.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.670+0000 7f4e069fd740 1 Processor -- start 2024-09-19T05:06:33.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.670+0000 7f4e069fd740 1 -- start start 2024-09-19T05:06:33.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.670+0000 7f4e069fd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a833c0a4b0 0x55a833c13b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:33.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.671+0000 7f4e069fd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a833c14d70 0x55a833c2c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:33.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.671+0000 7f4e069fd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a833c10d00 0x55a833c2cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:33.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.671+0000 7f4e069fd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc1b3f70 con 0x55a833c10d00 2024-09-19T05:06:33.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.671+0000 7f4e069fd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc1b5ab0 con 0x55a833c0a4b0 2024-09-19T05:06:33.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.671+0000 7f4e069fd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc1b4640 con 0x55a833c14d70 2024-09-19T05:06:33.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.671+0000 7f4e02400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a833c10d00 0x55a833c2cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:33.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.671+0000 7f4e01000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a833c14d70 0x55a833c2c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:33.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.671+0000 7f4e01a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a833c0a4b0 0x55a833c13b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:33.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.671+0000 7f4e01000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a833c14d70 0x55a833c2c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:57800/0 (socket says 172.21.15.43:57800) 2024-09-19T05:06:33.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.671+0000 7f4e01000640 1 -- 172.21.15.43:0/2001791168 learned_addr learned my addr 172.21.15.43:0/2001791168 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:33.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.671+0000 7f4e01000640 1 -- 172.21.15.43:0/2001791168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a833c0a4b0 msgr2=0x55a833c13b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:33.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.671+0000 7f4e01000640 1 --2- 172.21.15.43:0/2001791168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a833c0a4b0 0x55a833c13b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:33.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.671+0000 7f4e01000640 1 -- 172.21.15.43:0/2001791168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a833c10d00 msgr2=0x55a833c2cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:33.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.671+0000 7f4e01000640 1 --2- 172.21.15.43:0/2001791168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a833c10d00 0x55a833c2cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:33.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.671+0000 7f4e01000640 1 -- 172.21.15.43:0/2001791168 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a833c05f50 con 0x55a833c14d70 2024-09-19T05:06:33.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.672+0000 7f4e01000640 1 --2- 172.21.15.43:0/2001791168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a833c14d70 0x55a833c2c7f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4de8004b10 tx=0x7f4de8004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85348b9f64a50d23 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:33.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.672+0000 7f4dee000640 1 -- 172.21.15.43:0/2001791168 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de801e040 con 0x55a833c14d70 2024-09-19T05:06:33.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.672+0000 7f4dee000640 1 -- 172.21.15.43:0/2001791168 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4de802a070 con 0x55a833c14d70 2024-09-19T05:06:33.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.672+0000 7f4dee000640 1 -- 172.21.15.43:0/2001791168 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de8003850 con 0x55a833c14d70 2024-09-19T05:06:33.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.673+0000 7f4e069fd740 1 -- 172.21.15.43:0/2001791168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a833c14d70 msgr2=0x55a833c2c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:33.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.673+0000 7f4e069fd740 1 --2- 172.21.15.43:0/2001791168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a833c14d70 0x55a833c2c7f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4de8004b10 tx=0x7f4de8004b40 comp rx=0 tx=0).stop 2024-09-19T05:06:33.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.673+0000 7f4e069fd740 1 -- 172.21.15.43:0/2001791168 shutdown_connections 2024-09-19T05:06:33.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.673+0000 7f4e069fd740 1 --2- 172.21.15.43:0/2001791168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a833c10d00 0x55a833c2cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:33.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.673+0000 7f4e069fd740 1 --2- 172.21.15.43:0/2001791168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a833c14d70 0x55a833c2c7f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:33.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.673+0000 7f4e069fd740 1 --2- 172.21.15.43:0/2001791168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a833c0a4b0 0x55a833c13b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:33.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.673+0000 7f4e069fd740 1 -- 172.21.15.43:0/2001791168 >> 172.21.15.43:0/2001791168 conn(0x55a833c06120 msgr2=0x55a833c13470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:33.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.673+0000 7f4e069fd740 1 -- 172.21.15.43:0/2001791168 shutdown_connections 2024-09-19T05:06:33.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.673+0000 7f4e069fd740 1 -- 172.21.15.43:0/2001791168 wait complete. 2024-09-19T05:06:33.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.673+0000 7f4e069fd740 1 -- start start 2024-09-19T05:06:33.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.673+0000 7f4e069fd740 1 Processor -- start 2024-09-19T05:06:33.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.674+0000 7f4e069fd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a833c0a4b0 0x55a833c18c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:33.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.674+0000 7f4e069fd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a833c10d00 0x55a833c19180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:33.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.674+0000 7f4e069fd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a833c14d70 0x55a833c196c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:33.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.674+0000 7f4e069fd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc1b53e0 con 0x55a833c0a4b0 2024-09-19T05:06:33.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.674+0000 7f4e069fd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc1b4d10 con 0x55a833c14d70 2024-09-19T05:06:33.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.674+0000 7f4e069fd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc1b1f50 con 0x55a833c10d00 2024-09-19T05:06:33.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.674+0000 7f4e069fd740 10 client.0 _ll_register_callbacks cb 0x7f4df00d3ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:33.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.674+0000 7f4e01a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a833c0a4b0 0x55a833c18c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:33.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.674+0000 7f4e01000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a833c10d00 0x55a833c19180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:33.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.674+0000 7f4e01000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a833c10d00 0x55a833c19180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:57808/0 (socket says 172.21.15.43:57808) 2024-09-19T05:06:33.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.674+0000 7f4e01000640 1 -- 172.21.15.43:0/278597476 learned_addr learned my addr 172.21.15.43:0/278597476 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:33.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.674+0000 7f4e01a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a833c0a4b0 0x55a833c18c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51582/0 (socket says 172.21.15.43:51582) 2024-09-19T05:06:33.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.674+0000 7f4e02400640 1 --2- 172.21.15.43:0/278597476 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a833c14d70 0x55a833c196c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:33.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.675+0000 7f4e01000640 1 -- 172.21.15.43:0/278597476 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a833c14d70 msgr2=0x55a833c196c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:33.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.675+0000 7f4e01000640 1 --2- 172.21.15.43:0/278597476 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a833c14d70 0x55a833c196c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:33.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.675+0000 7f4e01000640 1 -- 172.21.15.43:0/278597476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a833c0a4b0 msgr2=0x55a833c18c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:33.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.675+0000 7f4e01000640 1 --2- 172.21.15.43:0/278597476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a833c0a4b0 0x55a833c18c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:33.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.675+0000 7f4e01000640 1 -- 172.21.15.43:0/278597476 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a833c8e920 con 0x55a833c10d00 2024-09-19T05:06:33.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.675+0000 7f4e01000640 1 --2- 172.21.15.43:0/278597476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a833c10d00 0x55a833c19180 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4de8009250 tx=0x7f4de8005ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8ef189e4eb90434 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:33.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.675+0000 7f4de3400640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:06:33.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.675+0000 7f4de3400640 1 -- 172.21.15.43:0/278597476 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de803a060 con 0x55a833c10d00 2024-09-19T05:06:33.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.675+0000 7f4e069fd740 1 -- 172.21.15.43:0/278597476 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4de802a070 con 0x55a833c10d00 2024-09-19T05:06:33.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.675+0000 7f4de3400640 1 -- 172.21.15.43:0/278597476 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f4de80042e0 con 0x55a833c10d00 2024-09-19T05:06:33.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.676+0000 7f4de3400640 1 -- 172.21.15.43:0/278597476 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4de8004520 con 0x55a833c10d00 2024-09-19T05:06:33.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.676+0000 7f4e069fd740 10 client.25579 fetch_fsmap learned FSMap version 29 2024-09-19T05:06:33.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.676+0000 7f4e069fd740 1 -- 172.21.15.43:0/278597476 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a833c8fbd0 con 0x55a833c10d00 2024-09-19T05:06:33.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.676+0000 7f4de3400640 1 -- 172.21.15.43:0/278597476 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4de802a070 con 0x55a833c10d00 2024-09-19T05:06:33.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.676+0000 7f4de3400640 1 -- 172.21.15.43:0/278597476 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4de8024040 con 0x55a833c10d00 2024-09-19T05:06:33.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.676+0000 7f4e069fd740 10 client.25579 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:06:33.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.676+0000 7f4e069fd740 10 client.25579 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:06:33.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.676+0000 7f4e069fd740 20 client.25579 populate_metadata read hostname 'smithi043' 2024-09-19T05:06:33.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.676+0000 7f4e069fd740 1 --2- 172.21.15.43:0/278597476 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55a833c90130 0x55a833c92550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:33.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.676+0000 7f4e069fd740 4 client.25579 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:06:33.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.676+0000 7f4e069fd740 1 -- 172.21.15.43:0/278597476 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a833b77e60 con 0x55a833c90130 2024-09-19T05:06:33.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.677+0000 7f4e01a00640 1 --2- 172.21.15.43:0/278597476 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55a833c90130 0x55a833c92550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:33.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.677+0000 7f4e01a00640 1 --2- 172.21.15.43:0/278597476 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55a833c90130 0x55a833c92550 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fe20aba22d54e3d server_cookie=17dbae924fd5c7f9 in_seq=0 out_seq=0 2024-09-19T05:06:33.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.677+0000 7f4de3400640 10 client.25579 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:06:33.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.677+0000 7f4de3400640 1 -- 172.21.15.43:0/278597476 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a833b77e60 con 0x55a833c90130 2024-09-19T05:06:33.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.677+0000 7f4de3400640 10 client.25579 handle_command_reply: tid=0 2024-09-19T05:06:33.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.677+0000 7f4e069fd740 1 client.25579 shutdown 2024-09-19T05:06:33.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.678+0000 7f4e069fd740 1 -- 172.21.15.43:0/278597476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a833c10d00 msgr2=0x55a833c19180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:33.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.678+0000 7f4e069fd740 1 --2- 172.21.15.43:0/278597476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a833c10d00 0x55a833c19180 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4de8009250 tx=0x7f4de8005ab0 comp rx=0 tx=0).stop 2024-09-19T05:06:33.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.678+0000 7f4e069fd740 1 -- 172.21.15.43:0/278597476 shutdown_connections 2024-09-19T05:06:33.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.678+0000 7f4e069fd740 1 --2- 172.21.15.43:0/278597476 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55a833c90130 0x55a833c92550 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:33.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.678+0000 7f4e069fd740 1 --2- 172.21.15.43:0/278597476 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a833c14d70 0x55a833c196c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:33.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.678+0000 7f4e069fd740 1 --2- 172.21.15.43:0/278597476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a833c10d00 0x55a833c19180 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:33.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.678+0000 7f4de3400640 0 client.25579 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:06:33.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.678+0000 7f4e069fd740 1 --2- 172.21.15.43:0/278597476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a833c0a4b0 0x55a833c18c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:33.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.678+0000 7f4e069fd740 1 -- 172.21.15.43:0/278597476 >> 172.21.15.43:0/278597476 conn(0x55a833c06120 msgr2=0x55a833c12370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:33.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.678+0000 7f4e069fd740 1 -- 172.21.15.43:0/278597476 shutdown_connections 2024-09-19T05:06:33.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.678+0000 7f4e069fd740 1 -- 172.21.15.43:0/278597476 wait complete. 2024-09-19T05:06:33.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.678+0000 7f4e069fd740 20 client.25579 trim_cache size 0 max 16384 2024-09-19T05:06:33.684 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:06:33.684 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-09-19T05:06:33.684 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-09-19T05:06:33.684 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:06:33.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.679+0000 7f4e04400640 1 -- 172.21.15.43:0/1553532272 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f4de40a3430 msgr2=0x7f4de40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:33.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.679+0000 7f4e04400640 1 --2- 172.21.15.43:0/1553532272 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f4de40a3430 0x7f4de40a58f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4df4005ec0 tx=0x7f4df4005d70 comp rx=0 tx=0).stop 2024-09-19T05:06:33.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.679+0000 7f4e04400640 1 -- 172.21.15.43:0/1553532272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4dfc074590 msgr2=0x7f4dfc1a4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:33.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.679+0000 7f4e04400640 1 --2- 172.21.15.43:0/1553532272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4dfc074590 0x7f4dfc1a4700 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4df000eb90 tx=0x7f4df000c6a0 comp rx=0 tx=0).stop 2024-09-19T05:06:33.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.679+0000 7f4e04400640 1 -- 172.21.15.43:0/1553532272 shutdown_connections 2024-09-19T05:06:33.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.679+0000 7f4e04400640 1 --2- 172.21.15.43:0/1553532272 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4dfc112b90 0x7f4dfc19e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:33.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.679+0000 7f4e04400640 1 --2- 172.21.15.43:0/1553532272 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f4de40a3430 0x7f4de40a58f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:33.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.679+0000 7f4e04400640 1 --2- 172.21.15.43:0/1553532272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4dfc074eb0 0x7f4dfc1a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:33.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.679+0000 7f4e04400640 1 --2- 172.21.15.43:0/1553532272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4dfc074590 0x7f4dfc1a4700 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:33.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.679+0000 7f4e04400640 1 -- 172.21.15.43:0/1553532272 >> 172.21.15.43:0/1553532272 conn(0x7f4dfc0fd8c0 msgr2=0x7f4dfc0ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:33.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.679+0000 7f4e04400640 1 -- 172.21.15.43:0/1553532272 shutdown_connections 2024-09-19T05:06:33.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:33.679+0000 7f4e04400640 1 -- 172.21.15.43:0/1553532272 wait complete. 2024-09-19T05:06:34.116 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:33 smithi043 ceph-mon[30800]: pgmap v60: 97 pgs: 97 active+clean; 290 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 24 MiB/s wr, 38 op/s 2024-09-19T05:06:34.116 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:33 smithi043 ceph-mon[30800]: scrub summary: idle 2024-09-19T05:06:34.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:33 smithi151 ceph-mon[35425]: pgmap v60: 97 pgs: 97 active+clean; 290 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 24 MiB/s wr, 38 op/s 2024-09-19T05:06:34.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:33 smithi151 ceph-mon[35425]: scrub summary: idle 2024-09-19T05:06:34.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:33 smithi096 ceph-mon[35085]: pgmap v60: 97 pgs: 97 active+clean; 290 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 24 MiB/s wr, 38 op/s 2024-09-19T05:06:34.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:33 smithi096 ceph-mon[35085]: scrub summary: idle 2024-09-19T05:06:34.580 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T05:06:34.581 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:06:35.020 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:06:35.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:35.830+0000 7f1c55800640 1 -- 172.21.15.43:0/1561568139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c50076720 msgr2=0x7f1c50076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:35.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:35.830+0000 7f1c55800640 1 --2- 172.21.15.43:0/1561568139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c50076720 0x7f1c50076b00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1c40009900 tx=0x7f1c4002a400 comp rx=0 tx=0).stop 2024-09-19T05:06:35.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:35.830+0000 7f1c55800640 1 -- 172.21.15.43:0/1561568139 shutdown_connections 2024-09-19T05:06:35.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:35.830+0000 7f1c55800640 1 --2- 172.21.15.43:0/1561568139 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c50112a50 0x7f1c50114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:35.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:35.830+0000 7f1c55800640 1 --2- 172.21.15.43:0/1561568139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c50077040 0x7f1c50112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:35.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:35.830+0000 7f1c55800640 1 --2- 172.21.15.43:0/1561568139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c50076720 0x7f1c50076b00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:35.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:35.830+0000 7f1c55800640 1 -- 172.21.15.43:0/1561568139 >> 172.21.15.43:0/1561568139 conn(0x7f1c500ffa50 msgr2=0x7f1c50101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:35.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:35.830+0000 7f1c55800640 1 -- 172.21.15.43:0/1561568139 shutdown_connections 2024-09-19T05:06:35.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:35.830+0000 7f1c55800640 1 -- 172.21.15.43:0/1561568139 wait complete. 2024-09-19T05:06:35.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:35.832+0000 7f1c55800640 1 Processor -- start 2024-09-19T05:06:35.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:35.832+0000 7f1c55800640 1 -- start start 2024-09-19T05:06:35.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:35.833+0000 7f1c55800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c50076720 0x7f1c50112150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:35.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:35.833+0000 7f1c55800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c50077040 0x7f1c5010b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:35.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:35.833+0000 7f1c55800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c50112a50 0x7f1c5010b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:35.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:35.833+0000 7f1c55800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1c50118680 con 0x7f1c50112a50 2024-09-19T05:06:35.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:35.833+0000 7f1c55800640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1c50118500 con 0x7f1c50077040 2024-09-19T05:06:35.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:35.833+0000 7f1c55800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1c50118800 con 0x7f1c50076720 2024-09-19T05:06:35.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:35.834+0000 7f1c4e000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c50077040 0x7f1c5010b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:35.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:35.834+0000 7f1c4e000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c50077040 0x7f1c5010b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:45734/0 (socket says 172.21.15.43:45734) 2024-09-19T05:06:35.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:35.834+0000 7f1c4e000640 1 -- 172.21.15.43:0/617149839 learned_addr learned my addr 172.21.15.43:0/617149839 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:35.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:35.834+0000 7f1c4ea00640 1 --2- 172.21.15.43:0/617149839 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c50076720 0x7f1c50112150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:35.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:35.834+0000 7f1c4f400640 1 --2- 172.21.15.43:0/617149839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c50112a50 0x7f1c5010b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:35.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:35.834+0000 7f1c4e000640 1 -- 172.21.15.43:0/617149839 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c50076720 msgr2=0x7f1c50112150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:35.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:35.834+0000 7f1c4e000640 1 --2- 172.21.15.43:0/617149839 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c50076720 0x7f1c50112150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:35.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:35.834+0000 7f1c4e000640 1 -- 172.21.15.43:0/617149839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c50112a50 msgr2=0x7f1c5010b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:35.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:35.835+0000 7f1c4e000640 1 --2- 172.21.15.43:0/617149839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c50112a50 0x7f1c5010b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:35.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:35.835+0000 7f1c4e000640 1 -- 172.21.15.43:0/617149839 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c5010be00 con 0x7f1c50077040 2024-09-19T05:06:35.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:35.835+0000 7f1c4f400640 1 --2- 172.21.15.43:0/617149839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c50112a50 0x7f1c5010b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:06:35.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:35.835+0000 7f1c4e000640 1 --2- 172.21.15.43:0/617149839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c50077040 0x7f1c5010b1e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c00b4a0 tx=0x7f1c3c00b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:35.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:35.835+0000 7f1c47400640 1 -- 172.21.15.43:0/617149839 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c014070 con 0x7f1c50077040 2024-09-19T05:06:35.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:35.836+0000 7f1c55800640 1 -- 172.21.15.43:0/617149839 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c5010fc80 con 0x7f1c50077040 2024-09-19T05:06:35.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:35.836+0000 7f1c47400640 1 -- 172.21.15.43:0/617149839 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c3c010af0 con 0x7f1c50077040 2024-09-19T05:06:35.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:35.836+0000 7f1c47400640 1 -- 172.21.15.43:0/617149839 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c017d10 con 0x7f1c50077040 2024-09-19T05:06:35.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:35.836+0000 7f1c55800640 1 -- 172.21.15.43:0/617149839 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c50110210 con 0x7f1c50077040 2024-09-19T05:06:35.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:35.837+0000 7f1c55800640 1 -- 172.21.15.43:0/617149839 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c50077e60 con 0x7f1c50077040 2024-09-19T05:06:35.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:35.839+0000 7f1c47400640 1 -- 172.21.15.43:0/617149839 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f1c3c017070 con 0x7f1c50077040 2024-09-19T05:06:35.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:35.841+0000 7f1c47400640 1 --2- 172.21.15.43:0/617149839 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f1c340a33b0 0x7f1c340a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:35.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:35.842+0000 7f1c47400640 1 -- 172.21.15.43:0/617149839 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f1c3c0cf0c0 con 0x7f1c50077040 2024-09-19T05:06:35.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:35.842+0000 7f1c4ea00640 1 --2- 172.21.15.43:0/617149839 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f1c340a33b0 0x7f1c340a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:35.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:35.843+0000 7f1c4ea00640 1 --2- 172.21.15.43:0/617149839 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f1c340a33b0 0x7f1c340a5870 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1c40000c00 tx=0x7f1c40005a20 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:35.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:35.845+0000 7f1c47400640 1 -- 172.21.15.43:0/617149839 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1c3c09bb60 con 0x7f1c50077040 2024-09-19T05:06:36.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:35 smithi043 ceph-mon[30800]: pgmap v61: 97 pgs: 97 active+clean; 1.0 GiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 85 MiB/s wr, 58 op/s 2024-09-19T05:06:36.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:35 smithi151 ceph-mon[35425]: pgmap v61: 97 pgs: 97 active+clean; 1.0 GiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 85 MiB/s wr, 58 op/s 2024-09-19T05:06:36.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:35 smithi096 ceph-mon[35085]: pgmap v61: 97 pgs: 97 active+clean; 1.0 GiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 85 MiB/s wr, 58 op/s 2024-09-19T05:06:36.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:36.302+0000 7f1c55800640 1 -- 172.21.15.43:0/617149839 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c5010bfd0 con 0x7f1c50077040 2024-09-19T05:06:36.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:36.303+0000 7f1c47400640 1 -- 172.21.15.43:0/617149839 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11965 (secure 0 0 0) 0x7f1c3c09b460 con 0x7f1c50077040 2024-09-19T05:06:36.304 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:06:36.305 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":29,"btime":"2024-09-19T05:04:38:785407+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/773801169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":773801169},{"type":"v1","addr":"172.21.15.43:6835","nonce":773801169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1799137902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1799137902},{"type":"v1","addr":"172.21.15.43:6837","nonce":1799137902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:02:25.463041+0000","modified":"2024-09-19T05:04:38.039809+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24518,"mds_2":24511,"mds_3":24506,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14541":{"gid":14541,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/794881135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":794881135},{"type":"v1","addr":"172.21.15.43:6841","nonce":794881135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/2836453158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2836453158},{"type":"v1","addr":"172.21.15.151:6833","nonce":2836453158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/178785812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":178785812},{"type":"v1","addr":"172.21.15.96:6833","nonce":178785812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1671043820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1671043820},{"type":"v1","addr":"172.21.15.96:6835","nonce":1671043820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2972514328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2972514328},{"type":"v1","addr":"172.21.15.151:6835","nonce":2972514328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.151:6837/3960378010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3960378010},{"type":"v1","addr":"172.21.15.151:6837","nonce":3960378010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6837/3978388286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3978388286},{"type":"v1","addr":"172.21.15.96:6837","nonce":3978388286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.151:6839/3794104313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3794104313},{"type":"v1","addr":"172.21.15.151:6839","nonce":3794104313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.43:6839/3404555030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3404555030},{"type":"v1","addr":"172.21.15.43:6839","nonce":3404555030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/605527621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":605527621},{"type":"v1","addr":"172.21.15.96:6839","nonce":605527621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24518,24511,24506,24493]},"id":1}]} 2024-09-19T05:06:36.305 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 29 2024-09-19T05:06:36.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:36.305+0000 7f1c55800640 1 -- 172.21.15.43:0/617149839 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f1c340a33b0 msgr2=0x7f1c340a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:36.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:36.305+0000 7f1c55800640 1 --2- 172.21.15.43:0/617149839 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f1c340a33b0 0x7f1c340a5870 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1c40000c00 tx=0x7f1c40005a20 comp rx=0 tx=0).stop 2024-09-19T05:06:36.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:36.305+0000 7f1c55800640 1 -- 172.21.15.43:0/617149839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c50077040 msgr2=0x7f1c5010b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:36.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:36.305+0000 7f1c55800640 1 --2- 172.21.15.43:0/617149839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c50077040 0x7f1c5010b1e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c00b4a0 tx=0x7f1c3c00b960 comp rx=0 tx=0).stop 2024-09-19T05:06:36.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:36.306+0000 7f1c55800640 1 -- 172.21.15.43:0/617149839 shutdown_connections 2024-09-19T05:06:36.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:36.306+0000 7f1c55800640 1 --2- 172.21.15.43:0/617149839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c50112a50 0x7f1c5010b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:36.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:36.306+0000 7f1c55800640 1 --2- 172.21.15.43:0/617149839 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f1c340a33b0 0x7f1c340a5870 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:36.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:36.306+0000 7f1c55800640 1 --2- 172.21.15.43:0/617149839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c50077040 0x7f1c5010b1e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:36.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:36.306+0000 7f1c55800640 1 --2- 172.21.15.43:0/617149839 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c50076720 0x7f1c50112150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:36.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:36.306+0000 7f1c55800640 1 -- 172.21.15.43:0/617149839 >> 172.21.15.43:0/617149839 conn(0x7f1c500ffa50 msgr2=0x7f1c50101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:36.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:36.306+0000 7f1c55800640 1 -- 172.21.15.43:0/617149839 shutdown_connections 2024-09-19T05:06:36.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:36.306+0000 7f1c55800640 1 -- 172.21.15.43:0/617149839 wait complete. 2024-09-19T05:06:37.037 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:36 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/617149839' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:06:37.109 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T05:06:37.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:36 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/617149839' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:06:37.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:36 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/617149839' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:06:37.774 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:06:38.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:37 smithi151 ceph-mon[35425]: pgmap v62: 97 pgs: 97 active+clean; 1.0 GiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 85 MiB/s wr, 53 op/s 2024-09-19T05:06:38.165 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:37 smithi043 ceph-mon[30800]: pgmap v62: 97 pgs: 97 active+clean; 1.0 GiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 85 MiB/s wr, 53 op/s 2024-09-19T05:06:38.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:37 smithi096 ceph-mon[35085]: pgmap v62: 97 pgs: 97 active+clean; 1.0 GiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 85 MiB/s wr, 53 op/s 2024-09-19T05:06:38.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.498+0000 7f0b85800640 1 -- 172.21.15.43:0/2898766471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b80108f40 msgr2=0x7f0b80110ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:38.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.498+0000 7f0b85800640 1 --2- 172.21.15.43:0/2898766471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b80108f40 0x7f0b80110ac0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0b78009860 tx=0x7f0b7802a340 comp rx=0 tx=0).stop 2024-09-19T05:06:38.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.498+0000 7f0b85800640 1 -- 172.21.15.43:0/2898766471 shutdown_connections 2024-09-19T05:06:38.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.498+0000 7f0b85800640 1 --2- 172.21.15.43:0/2898766471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b80108f40 0x7f0b80110ac0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:38.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.498+0000 7f0b85800640 1 --2- 172.21.15.43:0/2898766471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b801047c0 0x7f0b80108810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:38.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.498+0000 7f0b85800640 1 --2- 172.21.15.43:0/2898766471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b80103e10 0x7f0b801041f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:38.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.498+0000 7f0b85800640 1 -- 172.21.15.43:0/2898766471 >> 172.21.15.43:0/2898766471 conn(0x7f0b800ff870 msgr2=0x7f0b80101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:38.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.498+0000 7f0b85800640 1 -- 172.21.15.43:0/2898766471 shutdown_connections 2024-09-19T05:06:38.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.499+0000 7f0b85800640 1 -- 172.21.15.43:0/2898766471 wait complete. 2024-09-19T05:06:38.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.500+0000 7f0b85800640 1 Processor -- start 2024-09-19T05:06:38.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.500+0000 7f0b85800640 1 -- start start 2024-09-19T05:06:38.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.501+0000 7f0b85800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b80103e10 0x7f0b80073a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:38.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.501+0000 7f0b85800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b801047c0 0x7f0b8006cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:38.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.501+0000 7f0b85800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b80108f40 0x7f0b8006cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:38.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.501+0000 7f0b85800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0b80113fc0 con 0x7f0b80103e10 2024-09-19T05:06:38.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.501+0000 7f0b85800640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0b80113e40 con 0x7f0b801047c0 2024-09-19T05:06:38.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.501+0000 7f0b85800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0b80114140 con 0x7f0b80108f40 2024-09-19T05:06:38.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.501+0000 7f0b84e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b80108f40 0x7f0b8006cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:38.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.501+0000 7f0b7f400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b801047c0 0x7f0b8006cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:38.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.501+0000 7f0b84e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b80108f40 0x7f0b8006cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:49962/0 (socket says 172.21.15.43:49962) 2024-09-19T05:06:38.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.501+0000 7f0b84e00640 1 -- 172.21.15.43:0/1668388214 learned_addr learned my addr 172.21.15.43:0/1668388214 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:38.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.501+0000 7f0b7fe00640 1 --2- 172.21.15.43:0/1668388214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b80103e10 0x7f0b80073a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:38.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.501+0000 7f0b7f400640 1 -- 172.21.15.43:0/1668388214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b80108f40 msgr2=0x7f0b8006cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:38.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.501+0000 7f0b7f400640 1 --2- 172.21.15.43:0/1668388214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b80108f40 0x7f0b8006cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:38.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.501+0000 7f0b7f400640 1 -- 172.21.15.43:0/1668388214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b80103e10 msgr2=0x7f0b80073a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:38.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.501+0000 7f0b7f400640 1 --2- 172.21.15.43:0/1668388214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b80103e10 0x7f0b80073a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:38.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.501+0000 7f0b7f400640 1 -- 172.21.15.43:0/1668388214 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b8006d5e0 con 0x7f0b801047c0 2024-09-19T05:06:38.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.502+0000 7f0b7f400640 1 --2- 172.21.15.43:0/1668388214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b801047c0 0x7f0b8006cab0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00b3f0 tx=0x7f0b6c00b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:38.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.502+0000 7f0b7cc00640 1 -- 172.21.15.43:0/1668388214 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c015020 con 0x7f0b801047c0 2024-09-19T05:06:38.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.502+0000 7f0b7cc00640 1 -- 172.21.15.43:0/1668388214 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b6c011d10 con 0x7f0b801047c0 2024-09-19T05:06:38.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.502+0000 7f0b7cc00640 1 -- 172.21.15.43:0/1668388214 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c00f2e0 con 0x7f0b801047c0 2024-09-19T05:06:38.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.502+0000 7f0b85800640 1 -- 172.21.15.43:0/1668388214 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b8006d870 con 0x7f0b801047c0 2024-09-19T05:06:38.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.502+0000 7f0b85800640 1 -- 172.21.15.43:0/1668388214 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b80071810 con 0x7f0b801047c0 2024-09-19T05:06:38.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.506+0000 7f0b7cc00640 1 -- 172.21.15.43:0/1668388214 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0b6c018020 con 0x7f0b801047c0 2024-09-19T05:06:38.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.507+0000 7f0b89a2d740 1 Processor -- start 2024-09-19T05:06:38.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.507+0000 7f0b89a2d740 1 -- start start 2024-09-19T05:06:38.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.507+0000 7f0b89a2d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612afe7ee40 0x5612afe7f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:38.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.507+0000 7f0b89a2d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612afe7f760 0x5612afe89ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:38.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.507+0000 7f0b89a2d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612afe8a020 0x5612afe8c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:38.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.507+0000 7f0b89a2d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5612afde53e0 con 0x5612afe7f760 2024-09-19T05:06:38.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.507+0000 7f0b89a2d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5612afd8cd80 con 0x5612afe7ee40 2024-09-19T05:06:38.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.507+0000 7f0b89a2d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5612afde13a0 con 0x5612afe8a020 2024-09-19T05:06:38.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.508+0000 7f0b84e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612afe7ee40 0x5612afe7f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:38.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.508+0000 7f0b84e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612afe7ee40 0x5612afe7f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:45760/0 (socket says 172.21.15.43:45760) 2024-09-19T05:06:38.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.508+0000 7f0b84e00640 1 -- 172.21.15.43:0/130225612 learned_addr learned my addr 172.21.15.43:0/130225612 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:38.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.508+0000 7f0b7fe00640 1 --2- 172.21.15.43:0/130225612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612afe7f760 0x5612afe89ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:38.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.508+0000 7f0b7f400640 1 --2- 172.21.15.43:0/130225612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612afe8a020 0x5612afe8c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:38.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.508+0000 7f0b7fe00640 1 -- 172.21.15.43:0/130225612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612afe8a020 msgr2=0x5612afe8c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:38.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.509+0000 7f0b7fe00640 1 --2- 172.21.15.43:0/130225612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612afe8a020 0x5612afe8c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:38.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.509+0000 7f0b7fe00640 1 -- 172.21.15.43:0/130225612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612afe7ee40 msgr2=0x5612afe7f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:38.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.509+0000 7f0b7fe00640 1 --2- 172.21.15.43:0/130225612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612afe7ee40 0x5612afe7f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:38.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.509+0000 7f0b7fe00640 1 -- 172.21.15.43:0/130225612 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612afdabae0 con 0x5612afe7f760 2024-09-19T05:06:38.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.509+0000 7f0b7f400640 1 --2- 172.21.15.43:0/130225612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612afe8a020 0x5612afe8c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:06:38.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.509+0000 7f0b7fe00640 1 --2- 172.21.15.43:0/130225612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612afe7f760 0x5612afe89ae0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0b7000e4d0 tx=0x7f0b700095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fba1cbe2881dab38 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:38.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.509+0000 7f0b75600640 1 -- 172.21.15.43:0/130225612 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7001d070 con 0x5612afe7f760 2024-09-19T05:06:38.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.509+0000 7f0b75600640 1 -- 172.21.15.43:0/130225612 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b70003e40 con 0x5612afe7f760 2024-09-19T05:06:38.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.509+0000 7f0b75600640 1 -- 172.21.15.43:0/130225612 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7000dbd0 con 0x5612afe7f760 2024-09-19T05:06:38.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.510+0000 7f0b7cc00640 1 --2- 172.21.15.43:0/1668388214 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f0b640abbc0 0x7f0b640ae080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:38.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.510+0000 7f0b89a2d740 1 -- 172.21.15.43:0/130225612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612afe7f760 msgr2=0x5612afe89ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:38.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.510+0000 7f0b89a2d740 1 --2- 172.21.15.43:0/130225612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612afe7f760 0x5612afe89ae0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0b7000e4d0 tx=0x7f0b700095c0 comp rx=0 tx=0).stop 2024-09-19T05:06:38.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.510+0000 7f0b84e00640 1 --2- 172.21.15.43:0/1668388214 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f0b640abbc0 0x7f0b640ae080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:38.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.510+0000 7f0b89a2d740 1 -- 172.21.15.43:0/130225612 shutdown_connections 2024-09-19T05:06:38.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.510+0000 7f0b89a2d740 1 --2- 172.21.15.43:0/130225612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612afe8a020 0x5612afe8c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:38.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.510+0000 7f0b89a2d740 1 --2- 172.21.15.43:0/130225612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612afe7f760 0x5612afe89ae0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:38.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.510+0000 7f0b89a2d740 1 --2- 172.21.15.43:0/130225612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612afe7ee40 0x5612afe7f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:38.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.510+0000 7f0b89a2d740 1 -- 172.21.15.43:0/130225612 >> 172.21.15.43:0/130225612 conn(0x5612afe7b3d0 msgr2=0x5612afe7b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:38.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.510+0000 7f0b7cc00640 1 -- 172.21.15.43:0/1668388214 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f0b6c0cf170 con 0x7f0b801047c0 2024-09-19T05:06:38.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.511+0000 7f0b89a2d740 1 -- 172.21.15.43:0/130225612 shutdown_connections 2024-09-19T05:06:38.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.511+0000 7f0b89a2d740 1 -- 172.21.15.43:0/130225612 wait complete. 2024-09-19T05:06:38.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.511+0000 7f0b84e00640 1 --2- 172.21.15.43:0/1668388214 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f0b640abbc0 0x7f0b640ae080 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0b780304f0 tx=0x7f0b780094b0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:38.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.511+0000 7f0b89a2d740 1 -- start start 2024-09-19T05:06:38.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.512+0000 7f0b89a2d740 1 Processor -- start 2024-09-19T05:06:38.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.512+0000 7f0b89a2d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612afe7ee40 0x5612afe86ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:38.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.512+0000 7f0b89a2d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612afe7f760 0x5612afe87420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:38.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.512+0000 7f0b89a2d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612afe8a020 0x5612afe87960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:38.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.512+0000 7f0b89a2d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5612afe8cbf0 con 0x5612afe7ee40 2024-09-19T05:06:38.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.512+0000 7f0b89a2d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5612afda00f0 con 0x5612afe8a020 2024-09-19T05:06:38.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.512+0000 7f0b89a2d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5612afe8cd70 con 0x5612afe7f760 2024-09-19T05:06:38.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.512+0000 7f0b89a2d740 10 client.0 _ll_register_callbacks cb 0x5612afd87bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:38.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.512+0000 7f0b7f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612afe7f760 0x5612afe87420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:38.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.512+0000 7f0b84e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612afe8a020 0x5612afe87960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:38.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.512+0000 7f0b7f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612afe7f760 0x5612afe87420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:49976/0 (socket says 172.21.15.43:49976) 2024-09-19T05:06:38.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.512+0000 7f0b7f400640 1 -- 172.21.15.43:0/1578999151 learned_addr learned my addr 172.21.15.43:0/1578999151 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:38.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.513+0000 7f0b7f400640 1 -- 172.21.15.43:0/1578999151 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612afe8a020 msgr2=0x5612afe87960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:38.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.513+0000 7f0b7f400640 1 --2- 172.21.15.43:0/1578999151 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612afe8a020 0x5612afe87960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:38.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.513+0000 7f0b7f400640 1 -- 172.21.15.43:0/1578999151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612afe7ee40 msgr2=0x5612afe86ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:38.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.513+0000 7f0b7f400640 1 --2- 172.21.15.43:0/1578999151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612afe7ee40 0x5612afe86ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:38.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.513+0000 7f0b7f400640 1 -- 172.21.15.43:0/1578999151 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612afb563c0 con 0x5612afe7f760 2024-09-19T05:06:38.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.513+0000 7f0b84e00640 1 --2- 172.21.15.43:0/1578999151 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612afe8a020 0x5612afe87960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:38.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.513+0000 7f0b7f400640 1 --2- 172.21.15.43:0/1578999151 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612afe7f760 0x5612afe87420 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c09b270 tx=0x7f0b6c097910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=960524f5ff68f689 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:38.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.513+0000 7f0b6a000640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:06:38.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.513+0000 7f0b6a000640 1 -- 172.21.15.43:0/1578999151 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c009200 con 0x5612afe7f760 2024-09-19T05:06:38.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.513+0000 7f0b6a000640 1 -- 172.21.15.43:0/1578999151 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f0b6c097be0 con 0x5612afe7f760 2024-09-19T05:06:38.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.513+0000 7f0b89a2d740 1 -- 172.21.15.43:0/1578999151 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5612afdf2950 con 0x5612afe7f760 2024-09-19T05:06:38.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.513+0000 7f0b6a000640 1 -- 172.21.15.43:0/1578999151 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0b6c010eb0 con 0x5612afe7f760 2024-09-19T05:06:38.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.513+0000 7f0b89a2d740 10 client.25597 fetch_fsmap learned FSMap version 29 2024-09-19T05:06:38.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.513+0000 7f0b89a2d740 1 -- 172.21.15.43:0/1578999151 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5612afe98c00 con 0x5612afe7f760 2024-09-19T05:06:38.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.514+0000 7f0b6a000640 1 -- 172.21.15.43:0/1578999151 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5612afdf2950 con 0x5612afe7f760 2024-09-19T05:06:38.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.514+0000 7f0b6a000640 1 -- 172.21.15.43:0/1578999151 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0b6c0b42b0 con 0x5612afe7f760 2024-09-19T05:06:38.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.514+0000 7f0b89a2d740 10 client.25597 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:06:38.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.514+0000 7f0b89a2d740 10 client.25597 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-19T05:06:38.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.514+0000 7f0b89a2d740 20 client.25597 populate_metadata read hostname 'smithi043' 2024-09-19T05:06:38.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.514+0000 7f0b89a2d740 1 --2- 172.21.15.43:0/1578999151 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x5612afe9a0c0 0x5612afe9c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:38.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.514+0000 7f0b89a2d740 4 client.25597 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:06:38.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.514+0000 7f0b89a2d740 1 -- 172.21.15.43:0/1578999151 --> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612afdece60 con 0x5612afe9a0c0 2024-09-19T05:06:38.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.514+0000 7f0b7fe00640 1 --2- 172.21.15.43:0/1578999151 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x5612afe9a0c0 0x5612afe9c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:38.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.515+0000 7f0b7fe00640 1 --2- 172.21.15.43:0/1578999151 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x5612afe9a0c0 0x5612afe9c530 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=822025b09096878a server_cookie=a4e3ce2c3c5cd30b in_seq=0 out_seq=0 2024-09-19T05:06:38.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.515+0000 7f0b6a000640 10 client.25597 ms_handle_connect on v2:172.21.15.151:6836/3960378010 2024-09-19T05:06:38.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.516+0000 7f0b6a000640 1 -- 172.21.15.43:0/1578999151 <== mds.4 v2:172.21.15.151:6836/3960378010 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5612afdece60 con 0x5612afe9a0c0 2024-09-19T05:06:38.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.516+0000 7f0b6a000640 10 client.25597 handle_command_reply: tid=0 2024-09-19T05:06:38.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.516+0000 7f0b89a2d740 1 client.25597 shutdown 2024-09-19T05:06:38.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.516+0000 7f0b89a2d740 1 -- 172.21.15.43:0/1578999151 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612afe7f760 msgr2=0x5612afe87420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:38.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.516+0000 7f0b89a2d740 1 --2- 172.21.15.43:0/1578999151 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612afe7f760 0x5612afe87420 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c09b270 tx=0x7f0b6c097910 comp rx=0 tx=0).stop 2024-09-19T05:06:38.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.516+0000 7f0b89a2d740 1 -- 172.21.15.43:0/1578999151 shutdown_connections 2024-09-19T05:06:38.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.516+0000 7f0b89a2d740 1 --2- 172.21.15.43:0/1578999151 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x5612afe9a0c0 0x5612afe9c530 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:38.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.516+0000 7f0b89a2d740 1 --2- 172.21.15.43:0/1578999151 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612afe8a020 0x5612afe87960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:38.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.516+0000 7f0b89a2d740 1 --2- 172.21.15.43:0/1578999151 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612afe7f760 0x5612afe87420 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:38.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.516+0000 7f0b89a2d740 1 --2- 172.21.15.43:0/1578999151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612afe7ee40 0x5612afe86ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:38.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.516+0000 7f0b6a000640 0 client.25597 ms_handle_reset on v2:172.21.15.151:6836/3960378010 2024-09-19T05:06:38.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.516+0000 7f0b89a2d740 1 -- 172.21.15.43:0/1578999151 >> 172.21.15.43:0/1578999151 conn(0x5612afe7b3d0 msgr2=0x5612afe81100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:38.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.516+0000 7f0b89a2d740 1 -- 172.21.15.43:0/1578999151 shutdown_connections 2024-09-19T05:06:38.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.516+0000 7f0b89a2d740 1 -- 172.21.15.43:0/1578999151 wait complete. 2024-09-19T05:06:38.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.516+0000 7f0b89a2d740 20 client.25597 trim_cache size 0 max 16384 2024-09-19T05:06:38.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.530+0000 7f0b89a2d740 1 Processor -- start 2024-09-19T05:06:38.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.530+0000 7f0b89a2d740 1 -- start start 2024-09-19T05:06:38.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.531+0000 7f0b89a2d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612afe7ee40 0x5612afe818a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:38.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.531+0000 7f0b89a2d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612afe7f760 0x5612afe7b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:38.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.531+0000 7f0b89a2d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612afe8a020 0x5612afea1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:38.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.531+0000 7f0b89a2d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0b801a2cf0 con 0x5612afe7ee40 2024-09-19T05:06:38.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.531+0000 7f0b89a2d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0b801a4830 con 0x5612afe8a020 2024-09-19T05:06:38.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.531+0000 7f0b89a2d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0b801a33c0 con 0x5612afe7f760 2024-09-19T05:06:38.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.531+0000 7f0b84e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612afe8a020 0x5612afea1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:38.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.531+0000 7f0b84e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612afe8a020 0x5612afea1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:45768/0 (socket says 172.21.15.43:45768) 2024-09-19T05:06:38.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.531+0000 7f0b84e00640 1 -- 172.21.15.43:0/4182872231 learned_addr learned my addr 172.21.15.43:0/4182872231 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:38.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.532+0000 7f0b7fe00640 1 --2- 172.21.15.43:0/4182872231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612afe7ee40 0x5612afe818a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:38.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.532+0000 7f0b84e00640 1 -- 172.21.15.43:0/4182872231 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612afe7f760 msgr2=0x5612afe7b7b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:06:38.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.532+0000 7f0b84e00640 1 --2- 172.21.15.43:0/4182872231 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612afe7f760 0x5612afe7b7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:38.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.532+0000 7f0b84e00640 1 -- 172.21.15.43:0/4182872231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612afe7ee40 msgr2=0x5612afe818a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:38.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.532+0000 7f0b84e00640 1 --2- 172.21.15.43:0/4182872231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612afe7ee40 0x5612afe818a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:38.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.532+0000 7f0b84e00640 1 -- 172.21.15.43:0/4182872231 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612afe85e90 con 0x5612afe8a020 2024-09-19T05:06:38.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.532+0000 7f0b84e00640 1 --2- 172.21.15.43:0/4182872231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612afe8a020 0x5612afea1e00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0b78004400 tx=0x7f0b7803c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56ce24f29480253c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:38.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.532+0000 7f0b74c00640 1 -- 172.21.15.43:0/4182872231 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7804d070 con 0x5612afe8a020 2024-09-19T05:06:38.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.532+0000 7f0b74c00640 1 -- 172.21.15.43:0/4182872231 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b780046c0 con 0x5612afe8a020 2024-09-19T05:06:38.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.532+0000 7f0b74c00640 1 -- 172.21.15.43:0/4182872231 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b78041070 con 0x5612afe8a020 2024-09-19T05:06:38.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.533+0000 7f0b89a2d740 1 -- 172.21.15.43:0/4182872231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612afe8a020 msgr2=0x5612afea1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:38.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.533+0000 7f0b89a2d740 1 --2- 172.21.15.43:0/4182872231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612afe8a020 0x5612afea1e00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0b78004400 tx=0x7f0b7803c000 comp rx=0 tx=0).stop 2024-09-19T05:06:38.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.533+0000 7f0b89a2d740 1 -- 172.21.15.43:0/4182872231 shutdown_connections 2024-09-19T05:06:38.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.533+0000 7f0b89a2d740 1 --2- 172.21.15.43:0/4182872231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612afe8a020 0x5612afea1e00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:38.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.533+0000 7f0b89a2d740 1 --2- 172.21.15.43:0/4182872231 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612afe7f760 0x5612afe7b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:38.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.533+0000 7f0b89a2d740 1 --2- 172.21.15.43:0/4182872231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612afe7ee40 0x5612afe818a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:38.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.533+0000 7f0b89a2d740 1 -- 172.21.15.43:0/4182872231 >> 172.21.15.43:0/4182872231 conn(0x5612afe7b3d0 msgr2=0x5612afe7a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:38.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.534+0000 7f0b89a2d740 1 -- 172.21.15.43:0/4182872231 shutdown_connections 2024-09-19T05:06:38.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.535+0000 7f0b89a2d740 1 -- 172.21.15.43:0/4182872231 wait complete. 2024-09-19T05:06:38.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.535+0000 7f0b89a2d740 1 -- start start 2024-09-19T05:06:38.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.536+0000 7f0b89a2d740 1 Processor -- start 2024-09-19T05:06:38.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.537+0000 7f0b89a2d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612afe7ee40 0x5612afe8e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:38.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.537+0000 7f0b89a2d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612afe8a020 0x5612afe8e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:38.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.537+0000 7f0b89a2d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612afe8eb80 0x5612aff03dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:38.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.537+0000 7f0b89a2d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0b801a4160 con 0x5612afe8eb80 2024-09-19T05:06:38.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.537+0000 7f0b89a2d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0b801a3a90 con 0x5612afe8a020 2024-09-19T05:06:38.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.537+0000 7f0b89a2d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8007a730 con 0x5612afe7ee40 2024-09-19T05:06:38.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.537+0000 7f0b89a2d740 10 client.0 _ll_register_callbacks cb 0x7f0b6c011ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:38.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.540+0000 7f0b84e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612afe8eb80 0x5612aff03dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:38.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.540+0000 7f0b7f400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612afe8a020 0x5612afe8e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:38.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.540+0000 7f0b7fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612afe7ee40 0x5612afe8e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:38.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.540+0000 7f0b7fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612afe7ee40 0x5612afe8e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:49998/0 (socket says 172.21.15.43:49998) 2024-09-19T05:06:38.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.540+0000 7f0b7fe00640 1 -- 172.21.15.43:0/3828944378 learned_addr learned my addr 172.21.15.43:0/3828944378 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:38.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.541+0000 7f0b7fe00640 1 -- 172.21.15.43:0/3828944378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612afe8a020 msgr2=0x5612afe8e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:38.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.541+0000 7f0b7fe00640 1 --2- 172.21.15.43:0/3828944378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612afe8a020 0x5612afe8e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:38.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.541+0000 7f0b7fe00640 1 -- 172.21.15.43:0/3828944378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612afe8eb80 msgr2=0x5612aff03dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:38.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.541+0000 7f0b7fe00640 1 --2- 172.21.15.43:0/3828944378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612afe8eb80 0x5612aff03dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:38.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.541+0000 7f0b7fe00640 1 -- 172.21.15.43:0/3828944378 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612aff04540 con 0x5612afe7ee40 2024-09-19T05:06:38.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.541+0000 7f0b84e00640 1 --2- 172.21.15.43:0/3828944378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612afe8eb80 0x5612aff03dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:06:38.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.543+0000 7f0b7fe00640 1 --2- 172.21.15.43:0/3828944378 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612afe7ee40 0x5612afe8e0b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0b7000d800 tx=0x7f0b70005eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7eebcc5d59ead18 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:38.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.543+0000 7f0b6a000640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:06:38.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.543+0000 7f0b6a000640 1 -- 172.21.15.43:0/3828944378 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7001d030 con 0x5612afe7ee40 2024-09-19T05:06:38.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.544+0000 7f0b6a000640 1 -- 172.21.15.43:0/3828944378 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f0b7000dab0 con 0x5612afe7ee40 2024-09-19T05:06:38.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.544+0000 7f0b89a2d740 1 -- 172.21.15.43:0/3828944378 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b780046c0 con 0x5612afe7ee40 2024-09-19T05:06:38.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.545+0000 7f0b6a000640 1 -- 172.21.15.43:0/3828944378 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0b70009540 con 0x5612afe7ee40 2024-09-19T05:06:38.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.545+0000 7f0b89a2d740 10 client.25600 fetch_fsmap learned FSMap version 29 2024-09-19T05:06:38.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.545+0000 7f0b89a2d740 1 -- 172.21.15.43:0/3828944378 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5612aff05880 con 0x5612afe7ee40 2024-09-19T05:06:38.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.546+0000 7f0b6a000640 1 -- 172.21.15.43:0/3828944378 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b780046c0 con 0x5612afe7ee40 2024-09-19T05:06:38.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.546+0000 7f0b6a000640 1 -- 172.21.15.43:0/3828944378 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0b70023040 con 0x5612afe7ee40 2024-09-19T05:06:38.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.547+0000 7f0b89a2d740 10 client.25600 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:06:38.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.547+0000 7f0b89a2d740 10 client.25600 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-19T05:06:38.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.547+0000 7f0b89a2d740 20 client.25600 populate_metadata read hostname 'smithi043' 2024-09-19T05:06:38.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.547+0000 7f0b89a2d740 1 --2- 172.21.15.43:0/3828944378 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x5612aff05dc0 0x5612aff081b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:38.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.547+0000 7f0b89a2d740 4 client.25600 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:06:38.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.547+0000 7f0b89a2d740 1 -- 172.21.15.43:0/3828944378 --> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5612afd69010 con 0x5612aff05dc0 2024-09-19T05:06:38.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.547+0000 7f0b7f400640 1 --2- 172.21.15.43:0/3828944378 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x5612aff05dc0 0x5612aff081b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:38.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.548+0000 7f0b7f400640 1 --2- 172.21.15.43:0/3828944378 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x5612aff05dc0 0x5612aff081b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=116edacec403805f server_cookie=34897dae0b0f177c in_seq=0 out_seq=0 2024-09-19T05:06:38.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.548+0000 7f0b6a000640 10 client.25600 ms_handle_connect on v2:172.21.15.151:6836/3960378010 2024-09-19T05:06:38.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.548+0000 7f0b6a000640 1 -- 172.21.15.43:0/3828944378 <== mds.4 v2:172.21.15.151:6836/3960378010 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5612afd69010 con 0x5612aff05dc0 2024-09-19T05:06:38.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.548+0000 7f0b6a000640 10 client.25600 handle_command_reply: tid=0 2024-09-19T05:06:38.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.548+0000 7f0b89a2d740 1 client.25600 shutdown 2024-09-19T05:06:38.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.549+0000 7f0b89a2d740 1 -- 172.21.15.43:0/3828944378 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612afe7ee40 msgr2=0x5612afe8e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:38.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.549+0000 7f0b89a2d740 1 --2- 172.21.15.43:0/3828944378 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612afe7ee40 0x5612afe8e0b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0b7000d800 tx=0x7f0b70005eb0 comp rx=0 tx=0).stop 2024-09-19T05:06:38.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.549+0000 7f0b89a2d740 1 -- 172.21.15.43:0/3828944378 shutdown_connections 2024-09-19T05:06:38.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.549+0000 7f0b89a2d740 1 --2- 172.21.15.43:0/3828944378 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x5612aff05dc0 0x5612aff081b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:38.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.549+0000 7f0b89a2d740 1 --2- 172.21.15.43:0/3828944378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5612afe8eb80 0x5612aff03dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:38.551 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:06:38.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.549+0000 7f0b89a2d740 1 --2- 172.21.15.43:0/3828944378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612afe8a020 0x5612afe8e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:38.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.549+0000 7f0b89a2d740 1 --2- 172.21.15.43:0/3828944378 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612afe7ee40 0x5612afe8e0b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:38.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.549+0000 7f0b89a2d740 1 -- 172.21.15.43:0/3828944378 >> 172.21.15.43:0/3828944378 conn(0x5612afe7b3d0 msgr2=0x5612afe88740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:38.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.549+0000 7f0b6a000640 0 client.25600 ms_handle_reset on v2:172.21.15.151:6836/3960378010 2024-09-19T05:06:38.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.549+0000 7f0b89a2d740 1 -- 172.21.15.43:0/3828944378 shutdown_connections 2024-09-19T05:06:38.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.549+0000 7f0b89a2d740 1 -- 172.21.15.43:0/3828944378 wait complete. 2024-09-19T05:06:38.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.550+0000 7f0b89a2d740 20 client.25600 trim_cache size 0 max 16384 2024-09-19T05:06:38.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.550+0000 7f0b85800640 1 -- 172.21.15.43:0/1668388214 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f0b640abbc0 msgr2=0x7f0b640ae080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:38.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.550+0000 7f0b85800640 1 --2- 172.21.15.43:0/1668388214 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f0b640abbc0 0x7f0b640ae080 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0b780304f0 tx=0x7f0b780094b0 comp rx=0 tx=0).stop 2024-09-19T05:06:38.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.550+0000 7f0b85800640 1 -- 172.21.15.43:0/1668388214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b801047c0 msgr2=0x7f0b8006cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:38.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.550+0000 7f0b85800640 1 --2- 172.21.15.43:0/1668388214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b801047c0 0x7f0b8006cab0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00b3f0 tx=0x7f0b6c00b8b0 comp rx=0 tx=0).stop 2024-09-19T05:06:38.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.551+0000 7f0b85800640 1 -- 172.21.15.43:0/1668388214 shutdown_connections 2024-09-19T05:06:38.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.551+0000 7f0b85800640 1 --2- 172.21.15.43:0/1668388214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b80108f40 0x7f0b8006cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:38.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.551+0000 7f0b85800640 1 --2- 172.21.15.43:0/1668388214 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f0b640abbc0 0x7f0b640ae080 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:38.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.551+0000 7f0b85800640 1 --2- 172.21.15.43:0/1668388214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b801047c0 0x7f0b8006cab0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:38.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.551+0000 7f0b85800640 1 --2- 172.21.15.43:0/1668388214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b80103e10 0x7f0b80073a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:38.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.551+0000 7f0b85800640 1 -- 172.21.15.43:0/1668388214 >> 172.21.15.43:0/1668388214 conn(0x7f0b800ff870 msgr2=0x7f0b80100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:38.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.551+0000 7f0b85800640 1 -- 172.21.15.43:0/1668388214 shutdown_connections 2024-09-19T05:06:38.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:38.551+0000 7f0b85800640 1 -- 172.21.15.43:0/1668388214 wait complete. 2024-09-19T05:06:38.888 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-19T05:06:38.888 INFO:tasks.workunit.client.0.smithi043.stdout: Children see throughput for 1 rewriters = 238184.17 kB/sec 2024-09-19T05:06:38.888 INFO:tasks.workunit.client.0.smithi043.stdout: Parent sees throughput for 1 rewriters = 238145.71 kB/sec 2024-09-19T05:06:38.888 INFO:tasks.workunit.client.0.smithi043.stdout: Min throughput per process = 238184.17 kB/sec 2024-09-19T05:06:38.888 INFO:tasks.workunit.client.0.smithi043.stdout: Max throughput per process = 238184.17 kB/sec 2024-09-19T05:06:38.888 INFO:tasks.workunit.client.0.smithi043.stdout: Avg throughput per process = 238184.17 kB/sec 2024-09-19T05:06:38.888 INFO:tasks.workunit.client.0.smithi043.stdout: Min xfer = 1048576.00 kB 2024-09-19T05:06:40.052 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:39 smithi043 ceph-mon[30800]: pgmap v63: 97 pgs: 97 active+clean; 1.0 GiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 85 MiB/s wr, 53 op/s 2024-09-19T05:06:40.054 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T05:06:40.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:39 smithi151 ceph-mon[35425]: pgmap v63: 97 pgs: 97 active+clean; 1.0 GiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 85 MiB/s wr, 53 op/s 2024-09-19T05:06:40.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:39 smithi096 ceph-mon[35085]: pgmap v63: 97 pgs: 97 active+clean; 1.0 GiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 85 MiB/s wr, 53 op/s 2024-09-19T05:06:40.441 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:06:40.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.991+0000 7fdf6ee00640 1 -- 172.21.15.43:0/1348157674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf68112a50 msgr2=0x7fdf68114e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:40.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.991+0000 7fdf6ee00640 1 --2- 172.21.15.43:0/1348157674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf68112a50 0x7fdf68114e40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fdf60009ca0 tx=0x7fdf6002a8c0 comp rx=0 tx=0).stop 2024-09-19T05:06:40.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.992+0000 7fdf6ee00640 1 -- 172.21.15.43:0/1348157674 shutdown_connections 2024-09-19T05:06:40.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.992+0000 7fdf6ee00640 1 --2- 172.21.15.43:0/1348157674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf68112a50 0x7fdf68114e40 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:40.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.992+0000 7fdf6ee00640 1 --2- 172.21.15.43:0/1348157674 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf68077040 0x7fdf68112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:40.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.992+0000 7fdf6ee00640 1 --2- 172.21.15.43:0/1348157674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf68076720 0x7fdf68076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:40.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.992+0000 7fdf6ee00640 1 -- 172.21.15.43:0/1348157674 >> 172.21.15.43:0/1348157674 conn(0x7fdf680ffa50 msgr2=0x7fdf68101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:40.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.992+0000 7fdf6ee00640 1 -- 172.21.15.43:0/1348157674 shutdown_connections 2024-09-19T05:06:40.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.992+0000 7fdf6ee00640 1 -- 172.21.15.43:0/1348157674 wait complete. 2024-09-19T05:06:40.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.993+0000 7fdf6ee00640 1 Processor -- start 2024-09-19T05:06:40.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.993+0000 7fdf6ee00640 1 -- start start 2024-09-19T05:06:40.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.993+0000 7fdf6ee00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf68076720 0x7fdf6819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:40.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.993+0000 7fdf6ee00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf68077040 0x7fdf6819f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:40.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.993+0000 7fdf6ee00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf68112a50 0x7fdf681a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:40.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.993+0000 7fdf6ee00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdf68118600 con 0x7fdf68076720 2024-09-19T05:06:40.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.993+0000 7fdf6ee00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdf68118480 con 0x7fdf68112a50 2024-09-19T05:06:40.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.993+0000 7fdf6ee00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdf68118780 con 0x7fdf68077040 2024-09-19T05:06:40.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.993+0000 7fdf67e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf68076720 0x7fdf6819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:40.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.993+0000 7fdf67e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf68076720 0x7fdf6819eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46838/0 (socket says 172.21.15.43:46838) 2024-09-19T05:06:40.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.993+0000 7fdf67e00640 1 -- 172.21.15.43:0/2311601544 learned_addr learned my addr 172.21.15.43:0/2311601544 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:40.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.993+0000 7fdf6ce00640 1 --2- 172.21.15.43:0/2311601544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf68112a50 0x7fdf681a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:40.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.994+0000 7fdf67e00640 1 -- 172.21.15.43:0/2311601544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf68077040 msgr2=0x7fdf6819f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:40.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.994+0000 7fdf67400640 1 --2- 172.21.15.43:0/2311601544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf68077040 0x7fdf6819f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:40.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.994+0000 7fdf67e00640 1 --2- 172.21.15.43:0/2311601544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf68077040 0x7fdf6819f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:40.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.994+0000 7fdf67e00640 1 -- 172.21.15.43:0/2311601544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf68112a50 msgr2=0x7fdf681a3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:40.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.994+0000 7fdf67e00640 1 --2- 172.21.15.43:0/2311601544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf68112a50 0x7fdf681a3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:40.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.994+0000 7fdf67e00640 1 -- 172.21.15.43:0/2311601544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf681a3b90 con 0x7fdf68076720 2024-09-19T05:06:40.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.994+0000 7fdf6ce00640 1 --2- 172.21.15.43:0/2311601544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf68112a50 0x7fdf681a3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:40.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.994+0000 7fdf67400640 1 --2- 172.21.15.43:0/2311601544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf68077040 0x7fdf6819f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:06:40.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.994+0000 7fdf67e00640 1 --2- 172.21.15.43:0/2311601544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf68076720 0x7fdf6819eb60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fdf5800e9f0 tx=0x7fdf5800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:40.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.995+0000 7fdf64c00640 1 -- 172.21.15.43:0/2311601544 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5800cd80 con 0x7fdf68076720 2024-09-19T05:06:40.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.995+0000 7fdf64c00640 1 -- 172.21.15.43:0/2311601544 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf58010910 con 0x7fdf68076720 2024-09-19T05:06:40.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.995+0000 7fdf6ee00640 1 -- 172.21.15.43:0/2311601544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf681a3e80 con 0x7fdf68076720 2024-09-19T05:06:40.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.995+0000 7fdf64c00640 1 -- 172.21.15.43:0/2311601544 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf58018990 con 0x7fdf68076720 2024-09-19T05:06:40.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.995+0000 7fdf6ee00640 1 -- 172.21.15.43:0/2311601544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf681ab710 con 0x7fdf68076720 2024-09-19T05:06:41.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.998+0000 7fdf71a4c740 1 Processor -- start 2024-09-19T05:06:41.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.998+0000 7fdf64c00640 1 -- 172.21.15.43:0/2311601544 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fdf58010ac0 con 0x7fdf68076720 2024-09-19T05:06:41.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.998+0000 7fdf71a4c740 1 -- start start 2024-09-19T05:06:41.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.998+0000 7fdf71a4c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596abddbc70 0x5596abddc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:41.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.998+0000 7fdf71a4c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5596abddc590 0x5596abde6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:41.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.998+0000 7fdf71a4c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596abde6e50 0x5596abde9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:41.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.998+0000 7fdf71a4c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5596abd423e0 con 0x5596abddc590 2024-09-19T05:06:41.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.998+0000 7fdf71a4c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5596abce9d80 con 0x5596abddbc70 2024-09-19T05:06:41.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.998+0000 7fdf71a4c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5596abd3e3a0 con 0x5596abde6e50 2024-09-19T05:06:41.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.999+0000 7fdf67e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5596abddc590 0x5596abde6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:41.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.999+0000 7fdf67400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596abde6e50 0x5596abde9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:41.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.999+0000 7fdf67e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5596abddc590 0x5596abde6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46846/0 (socket says 172.21.15.43:46846) 2024-09-19T05:06:41.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.999+0000 7fdf6ce00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596abddbc70 0x5596abddc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:41.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.999+0000 7fdf67e00640 1 -- 172.21.15.43:0/715203326 learned_addr learned my addr 172.21.15.43:0/715203326 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:41.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.999+0000 7fdf67e00640 1 -- 172.21.15.43:0/715203326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596abde6e50 msgr2=0x5596abde9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:41.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.999+0000 7fdf67e00640 1 --2- 172.21.15.43:0/715203326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596abde6e50 0x5596abde9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.999+0000 7fdf67e00640 1 -- 172.21.15.43:0/715203326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596abddbc70 msgr2=0x5596abddc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:41.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.999+0000 7fdf67e00640 1 --2- 172.21.15.43:0/715203326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596abddbc70 0x5596abddc050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:40.999+0000 7fdf67e00640 1 -- 172.21.15.43:0/715203326 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596abd08ae0 con 0x5596abddc590 2024-09-19T05:06:41.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.000+0000 7fdf64c00640 1 --2- 172.21.15.43:0/2311601544 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fdf4c0a33b0 0x7fdf4c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:41.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.000+0000 7fdf67e00640 1 --2- 172.21.15.43:0/715203326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5596abddc590 0x5596abde6910 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fdf580cd770 tx=0x7fdf580d8780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b2fd963d825d73 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:41.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.000+0000 7fdf64c00640 1 -- 172.21.15.43:0/2311601544 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fdf580d0130 con 0x7fdf68076720 2024-09-19T05:06:41.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.000+0000 7fdf5d600640 1 -- 172.21.15.43:0/715203326 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf58098660 con 0x5596abddc590 2024-09-19T05:06:41.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.000+0000 7fdf6ce00640 1 --2- 172.21.15.43:0/2311601544 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fdf4c0a33b0 0x7fdf4c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:41.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.000+0000 7fdf5d600640 1 -- 172.21.15.43:0/715203326 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf5809cac0 con 0x5596abddc590 2024-09-19T05:06:41.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.000+0000 7fdf5d600640 1 -- 172.21.15.43:0/715203326 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf580d3cd0 con 0x5596abddc590 2024-09-19T05:06:41.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.001+0000 7fdf71a4c740 1 -- 172.21.15.43:0/715203326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5596abddc590 msgr2=0x5596abde6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:41.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.001+0000 7fdf71a4c740 1 --2- 172.21.15.43:0/715203326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5596abddc590 0x5596abde6910 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fdf580cd770 tx=0x7fdf580d8780 comp rx=0 tx=0).stop 2024-09-19T05:06:41.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.001+0000 7fdf6ce00640 1 --2- 172.21.15.43:0/2311601544 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fdf4c0a33b0 0x7fdf4c0a5870 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdf60005e30 tx=0x7fdf60005d80 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:41.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.001+0000 7fdf71a4c740 1 -- 172.21.15.43:0/715203326 shutdown_connections 2024-09-19T05:06:41.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.001+0000 7fdf71a4c740 1 --2- 172.21.15.43:0/715203326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596abde6e50 0x5596abde9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.001+0000 7fdf71a4c740 1 --2- 172.21.15.43:0/715203326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5596abddc590 0x5596abde6910 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.001+0000 7fdf71a4c740 1 --2- 172.21.15.43:0/715203326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596abddbc70 0x5596abddc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.001+0000 7fdf71a4c740 1 -- 172.21.15.43:0/715203326 >> 172.21.15.43:0/715203326 conn(0x5596abdd8200 msgr2=0x5596abdd85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:41.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.001+0000 7fdf71a4c740 1 -- 172.21.15.43:0/715203326 shutdown_connections 2024-09-19T05:06:41.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.002+0000 7fdf71a4c740 1 -- 172.21.15.43:0/715203326 wait complete. 2024-09-19T05:06:41.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.002+0000 7fdf71a4c740 1 -- start start 2024-09-19T05:06:41.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.002+0000 7fdf71a4c740 1 Processor -- start 2024-09-19T05:06:41.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.003+0000 7fdf71a4c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596abddbc70 0x5596abde3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:41.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.003+0000 7fdf71a4c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596abddc590 0x5596abde4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:41.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.003+0000 7fdf71a4c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5596abde6e50 0x5596abde4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:41.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.003+0000 7fdf71a4c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5596abde99c0 con 0x5596abde6e50 2024-09-19T05:06:41.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.003+0000 7fdf71a4c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5596abcfd0f0 con 0x5596abddbc70 2024-09-19T05:06:41.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.003+0000 7fdf71a4c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5596abde9b40 con 0x5596abddc590 2024-09-19T05:06:41.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.003+0000 7fdf71a4c740 10 client.0 _ll_register_callbacks cb 0x5596abce4bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:41.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.003+0000 7fdf67e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596abddbc70 0x5596abde3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:41.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.003+0000 7fdf67e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596abddbc70 0x5596abde3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:45802/0 (socket says 172.21.15.43:45802) 2024-09-19T05:06:41.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.003+0000 7fdf67e00640 1 -- 172.21.15.43:0/2289171923 learned_addr learned my addr 172.21.15.43:0/2289171923 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:41.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.003+0000 7fdf6ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5596abde6e50 0x5596abde4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:41.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.003+0000 7fdf67400640 1 --2- 172.21.15.43:0/2289171923 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596abddc590 0x5596abde4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:41.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.003+0000 7fdf67e00640 1 -- 172.21.15.43:0/2289171923 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596abddc590 msgr2=0x5596abde4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:41.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.003+0000 7fdf67e00640 1 --2- 172.21.15.43:0/2289171923 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596abddc590 0x5596abde4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.003+0000 7fdf67e00640 1 -- 172.21.15.43:0/2289171923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5596abde6e50 msgr2=0x5596abde4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:41.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.003+0000 7fdf67e00640 1 --2- 172.21.15.43:0/2289171923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5596abde6e50 0x5596abde4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.003+0000 7fdf67e00640 1 -- 172.21.15.43:0/2289171923 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596abab33c0 con 0x5596abddbc70 2024-09-19T05:06:41.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.004+0000 7fdf6ce00640 1 --2- 172.21.15.43:0/2289171923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5596abde6e50 0x5596abde4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:06:41.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.004+0000 7fdf67e00640 1 --2- 172.21.15.43:0/2289171923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596abddbc70 0x5596abde3d00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdf5809c350 tx=0x7fdf580d3750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c09c71b8bb6d9ee4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:41.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.004+0000 7fdf52000640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:06:41.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.004+0000 7fdf52000640 1 -- 172.21.15.43:0/2289171923 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5809bd20 con 0x5596abddbc70 2024-09-19T05:06:41.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.004+0000 7fdf71a4c740 1 -- 172.21.15.43:0/2289171923 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf5809cac0 con 0x5596abddbc70 2024-09-19T05:06:41.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.004+0000 7fdf52000640 1 -- 172.21.15.43:0/2289171923 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fdf5809c620 con 0x5596abddbc70 2024-09-19T05:06:41.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.004+0000 7fdf52000640 1 -- 172.21.15.43:0/2289171923 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdf5809bf00 con 0x5596abddbc70 2024-09-19T05:06:41.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.004+0000 7fdf71a4c740 10 client.25649 fetch_fsmap learned FSMap version 29 2024-09-19T05:06:41.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.004+0000 7fdf71a4c740 1 -- 172.21.15.43:0/2289171923 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5596abde99c0 con 0x5596abddbc70 2024-09-19T05:06:41.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.005+0000 7fdf52000640 1 -- 172.21.15.43:0/2289171923 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf5809cac0 con 0x5596abddbc70 2024-09-19T05:06:41.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.005+0000 7fdf52000640 1 -- 172.21.15.43:0/2289171923 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdf580a4ba0 con 0x5596abddbc70 2024-09-19T05:06:41.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.005+0000 7fdf71a4c740 10 client.25649 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:06:41.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.005+0000 7fdf71a4c740 10 client.25649 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T05:06:41.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.005+0000 7fdf71a4c740 20 client.25649 populate_metadata read hostname 'smithi043' 2024-09-19T05:06:41.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.005+0000 7fdf71a4c740 1 --2- 172.21.15.43:0/2289171923 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x5596abdf6b80 0x5596abdf8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:41.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.005+0000 7fdf71a4c740 4 client.25649 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:06:41.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.005+0000 7fdf71a4c740 1 -- 172.21.15.43:0/2289171923 --> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596abd4f950 con 0x5596abdf6b80 2024-09-19T05:06:41.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.005+0000 7fdf67400640 1 --2- 172.21.15.43:0/2289171923 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x5596abdf6b80 0x5596abdf8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:41.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.006+0000 7fdf67400640 1 --2- 172.21.15.43:0/2289171923 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x5596abdf6b80 0x5596abdf8f70 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f85b8aad2270af8b server_cookie=9d3918722a06156a in_seq=0 out_seq=0 2024-09-19T05:06:41.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.006+0000 7fdf52000640 10 client.25649 ms_handle_connect on v2:172.21.15.96:6836/3978388286 2024-09-19T05:06:41.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.006+0000 7fdf52000640 1 -- 172.21.15.43:0/2289171923 <== mds.3 v2:172.21.15.96:6836/3978388286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5596abd4f950 con 0x5596abdf6b80 2024-09-19T05:06:41.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.006+0000 7fdf52000640 10 client.25649 handle_command_reply: tid=0 2024-09-19T05:06:41.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.007+0000 7fdf71a4c740 1 client.25649 shutdown 2024-09-19T05:06:41.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.007+0000 7fdf71a4c740 1 -- 172.21.15.43:0/2289171923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596abddbc70 msgr2=0x5596abde3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:41.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.007+0000 7fdf71a4c740 1 --2- 172.21.15.43:0/2289171923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596abddbc70 0x5596abde3d00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdf5809c350 tx=0x7fdf580d3750 comp rx=0 tx=0).stop 2024-09-19T05:06:41.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.007+0000 7fdf71a4c740 1 -- 172.21.15.43:0/2289171923 shutdown_connections 2024-09-19T05:06:41.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.007+0000 7fdf71a4c740 1 --2- 172.21.15.43:0/2289171923 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x5596abdf6b80 0x5596abdf8f70 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.007+0000 7fdf71a4c740 1 --2- 172.21.15.43:0/2289171923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5596abde6e50 0x5596abde4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.007+0000 7fdf71a4c740 1 --2- 172.21.15.43:0/2289171923 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596abddc590 0x5596abde4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.007+0000 7fdf52000640 0 client.25649 ms_handle_reset on v2:172.21.15.96:6836/3978388286 2024-09-19T05:06:41.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.007+0000 7fdf71a4c740 1 --2- 172.21.15.43:0/2289171923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596abddbc70 0x5596abde3d00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.007+0000 7fdf71a4c740 1 -- 172.21.15.43:0/2289171923 >> 172.21.15.43:0/2289171923 conn(0x5596abdd8200 msgr2=0x5596abdddf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:41.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.007+0000 7fdf71a4c740 1 -- 172.21.15.43:0/2289171923 shutdown_connections 2024-09-19T05:06:41.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.007+0000 7fdf71a4c740 1 -- 172.21.15.43:0/2289171923 wait complete. 2024-09-19T05:06:41.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.007+0000 7fdf71a4c740 20 client.25649 trim_cache size 0 max 16384 2024-09-19T05:06:41.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.021+0000 7fdf71a4c740 1 Processor -- start 2024-09-19T05:06:41.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.021+0000 7fdf71a4c740 1 -- start start 2024-09-19T05:06:41.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.021+0000 7fdf71a4c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596abddbc70 0x5596abde25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:41.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.021+0000 7fdf71a4c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5596abddc590 0x5596abdfe6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:41.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.021+0000 7fdf71a4c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596abde6e50 0x5596abdfec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:41.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.021+0000 7fdf71a4c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdf681b3f80 con 0x5596abddc590 2024-09-19T05:06:41.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.021+0000 7fdf71a4c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdf681b5ac0 con 0x5596abde6e50 2024-09-19T05:06:41.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.021+0000 7fdf71a4c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdf681b4650 con 0x5596abddbc70 2024-09-19T05:06:41.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.022+0000 7fdf67400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5596abddc590 0x5596abdfe6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:41.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.022+0000 7fdf67400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5596abddc590 0x5596abdfe6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46864/0 (socket says 172.21.15.43:46864) 2024-09-19T05:06:41.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.022+0000 7fdf67400640 1 -- 172.21.15.43:0/4041962739 learned_addr learned my addr 172.21.15.43:0/4041962739 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:41.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.022+0000 7fdf6ce00640 1 --2- 172.21.15.43:0/4041962739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596abde6e50 0x5596abdfec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:41.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.022+0000 7fdf67e00640 1 --2- 172.21.15.43:0/4041962739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596abddbc70 0x5596abde25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:41.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.022+0000 7fdf67400640 1 -- 172.21.15.43:0/4041962739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596abddbc70 msgr2=0x5596abde25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:41.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.022+0000 7fdf67400640 1 --2- 172.21.15.43:0/4041962739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596abddbc70 0x5596abde25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.022+0000 7fdf67400640 1 -- 172.21.15.43:0/4041962739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596abde6e50 msgr2=0x5596abdfec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:41.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.022+0000 7fdf67400640 1 --2- 172.21.15.43:0/4041962739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596abde6e50 0x5596abdfec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.022+0000 7fdf67400640 1 -- 172.21.15.43:0/4041962739 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596abdd8030 con 0x5596abddc590 2024-09-19T05:06:41.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.022+0000 7fdf6ce00640 1 --2- 172.21.15.43:0/4041962739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596abde6e50 0x5596abdfec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:06:41.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.022+0000 7fdf67e00640 1 --2- 172.21.15.43:0/4041962739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596abddbc70 0x5596abde25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:06:41.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.022+0000 7fdf67400640 1 --2- 172.21.15.43:0/4041962739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5596abddc590 0x5596abdfe6e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fdf54009cf0 tx=0x7fdf54005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6341cf375e454af server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:41.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.023+0000 7fdf5cc00640 1 -- 172.21.15.43:0/4041962739 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf540142d0 con 0x5596abddc590 2024-09-19T05:06:41.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.023+0000 7fdf5cc00640 1 -- 172.21.15.43:0/4041962739 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf5401d940 con 0x5596abddc590 2024-09-19T05:06:41.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.023+0000 7fdf5cc00640 1 -- 172.21.15.43:0/4041962739 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf540129b0 con 0x5596abddc590 2024-09-19T05:06:41.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.023+0000 7fdf71a4c740 1 -- 172.21.15.43:0/4041962739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5596abddc590 msgr2=0x5596abdfe6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:41.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.023+0000 7fdf71a4c740 1 --2- 172.21.15.43:0/4041962739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5596abddc590 0x5596abdfe6e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fdf54009cf0 tx=0x7fdf54005dc0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.023+0000 7fdf71a4c740 1 -- 172.21.15.43:0/4041962739 shutdown_connections 2024-09-19T05:06:41.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.023+0000 7fdf71a4c740 1 --2- 172.21.15.43:0/4041962739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596abde6e50 0x5596abdfec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.023+0000 7fdf71a4c740 1 --2- 172.21.15.43:0/4041962739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5596abddc590 0x5596abdfe6e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.023+0000 7fdf71a4c740 1 --2- 172.21.15.43:0/4041962739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596abddbc70 0x5596abde25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.023+0000 7fdf71a4c740 1 -- 172.21.15.43:0/4041962739 >> 172.21.15.43:0/4041962739 conn(0x5596abdd8200 msgr2=0x5596abdddee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:41.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.023+0000 7fdf71a4c740 1 -- 172.21.15.43:0/4041962739 shutdown_connections 2024-09-19T05:06:41.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.023+0000 7fdf71a4c740 1 -- 172.21.15.43:0/4041962739 wait complete. 2024-09-19T05:06:41.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.024+0000 7fdf71a4c740 1 -- start start 2024-09-19T05:06:41.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.024+0000 7fdf71a4c740 1 Processor -- start 2024-09-19T05:06:41.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.024+0000 7fdf71a4c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5596abddbc70 0x5596abdeabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:41.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.024+0000 7fdf71a4c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596abddc590 0x5596abdeb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:41.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.024+0000 7fdf71a4c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596abde6e50 0x5596abdeb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:41.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.025+0000 7fdf71a4c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdf681b53f0 con 0x5596abddbc70 2024-09-19T05:06:41.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.025+0000 7fdf71a4c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdf681b4d20 con 0x5596abde6e50 2024-09-19T05:06:41.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.025+0000 7fdf71a4c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdf681b1f60 con 0x5596abddc590 2024-09-19T05:06:41.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.025+0000 7fdf71a4c740 10 client.0 _ll_register_callbacks cb 0x7fdf54005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:41.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.025+0000 7fdf67e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5596abddbc70 0x5596abdeabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:41.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.025+0000 7fdf67e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5596abddbc70 0x5596abdeabd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46874/0 (socket says 172.21.15.43:46874) 2024-09-19T05:06:41.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.025+0000 7fdf67e00640 1 -- 172.21.15.43:0/3082923298 learned_addr learned my addr 172.21.15.43:0/3082923298 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:41.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.025+0000 7fdf6ce00640 1 --2- 172.21.15.43:0/3082923298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596abde6e50 0x5596abdeb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:41.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.025+0000 7fdf67400640 1 --2- 172.21.15.43:0/3082923298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596abddc590 0x5596abdeb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:41.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.025+0000 7fdf67e00640 1 -- 172.21.15.43:0/3082923298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596abddc590 msgr2=0x5596abdeb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:41.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.025+0000 7fdf67e00640 1 --2- 172.21.15.43:0/3082923298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596abddc590 0x5596abdeb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.025+0000 7fdf67e00640 1 -- 172.21.15.43:0/3082923298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596abde6e50 msgr2=0x5596abdeb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:41.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.025+0000 7fdf67e00640 1 --2- 172.21.15.43:0/3082923298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596abde6e50 0x5596abdeb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.025+0000 7fdf67e00640 1 -- 172.21.15.43:0/3082923298 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596abdd3770 con 0x5596abddbc70 2024-09-19T05:06:41.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.026+0000 7fdf67e00640 1 --2- 172.21.15.43:0/3082923298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5596abddbc70 0x5596abdeabd0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fdf5809c580 tx=0x7fdf58009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b1c47474ed7cb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:41.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.026+0000 7fdf52000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:06:41.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.026+0000 7fdf52000640 1 -- 172.21.15.43:0/3082923298 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5809c190 con 0x5596abddbc70 2024-09-19T05:06:41.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.026+0000 7fdf52000640 1 -- 172.21.15.43:0/3082923298 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fdf5809b810 con 0x5596abddbc70 2024-09-19T05:06:41.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.026+0000 7fdf71a4c740 1 -- 172.21.15.43:0/3082923298 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf5401d940 con 0x5596abddbc70 2024-09-19T05:06:41.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.026+0000 7fdf52000640 1 -- 172.21.15.43:0/3082923298 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdf5809bd20 con 0x5596abddbc70 2024-09-19T05:06:41.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.026+0000 7fdf71a4c740 10 client.15651 fetch_fsmap learned FSMap version 29 2024-09-19T05:06:41.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.026+0000 7fdf71a4c740 1 -- 172.21.15.43:0/3082923298 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5596abe61870 con 0x5596abddbc70 2024-09-19T05:06:41.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.026+0000 7fdf52000640 1 -- 172.21.15.43:0/3082923298 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf5401d940 con 0x5596abddbc70 2024-09-19T05:06:41.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.027+0000 7fdf52000640 1 -- 172.21.15.43:0/3082923298 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdf58098110 con 0x5596abddbc70 2024-09-19T05:06:41.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.027+0000 7fdf71a4c740 10 client.15651 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:06:41.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.027+0000 7fdf71a4c740 10 client.15651 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T05:06:41.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.027+0000 7fdf71a4c740 20 client.15651 populate_metadata read hostname 'smithi043' 2024-09-19T05:06:41.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.027+0000 7fdf71a4c740 1 --2- 172.21.15.43:0/3082923298 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x5596abe61da0 0x5596abe64190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:41.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.027+0000 7fdf71a4c740 4 client.15651 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:06:41.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.027+0000 7fdf71a4c740 1 -- 172.21.15.43:0/3082923298 --> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5596abd49e60 con 0x5596abe61da0 2024-09-19T05:06:41.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.027+0000 7fdf67400640 1 --2- 172.21.15.43:0/3082923298 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x5596abe61da0 0x5596abe64190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:41.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.028+0000 7fdf67400640 1 --2- 172.21.15.43:0/3082923298 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x5596abe61da0 0x5596abe64190 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6891a738c2adf356 server_cookie=94fb724abc2dcf0b in_seq=0 out_seq=0 2024-09-19T05:06:41.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.028+0000 7fdf52000640 10 client.15651 ms_handle_connect on v2:172.21.15.96:6836/3978388286 2024-09-19T05:06:41.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.028+0000 7fdf52000640 1 -- 172.21.15.43:0/3082923298 <== mds.3 v2:172.21.15.96:6836/3978388286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5596abd49e60 con 0x5596abe61da0 2024-09-19T05:06:41.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.028+0000 7fdf52000640 10 client.15651 handle_command_reply: tid=0 2024-09-19T05:06:41.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.028+0000 7fdf71a4c740 1 client.15651 shutdown 2024-09-19T05:06:41.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.028+0000 7fdf71a4c740 1 -- 172.21.15.43:0/3082923298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5596abddbc70 msgr2=0x5596abdeabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:41.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.028+0000 7fdf71a4c740 1 --2- 172.21.15.43:0/3082923298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5596abddbc70 0x5596abdeabd0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fdf5809c580 tx=0x7fdf58009310 comp rx=0 tx=0).stop 2024-09-19T05:06:41.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.028+0000 7fdf71a4c740 1 -- 172.21.15.43:0/3082923298 shutdown_connections 2024-09-19T05:06:41.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.028+0000 7fdf71a4c740 1 --2- 172.21.15.43:0/3082923298 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x5596abe61da0 0x5596abe64190 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.028+0000 7fdf71a4c740 1 --2- 172.21.15.43:0/3082923298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5596abde6e50 0x5596abdeb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.028+0000 7fdf71a4c740 1 --2- 172.21.15.43:0/3082923298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596abddc590 0x5596abdeb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.028+0000 7fdf71a4c740 1 --2- 172.21.15.43:0/3082923298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5596abddbc70 0x5596abdeabd0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.028+0000 7fdf71a4c740 1 -- 172.21.15.43:0/3082923298 >> 172.21.15.43:0/3082923298 conn(0x5596abdd8200 msgr2=0x5596abddd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:41.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.028+0000 7fdf52000640 0 client.15651 ms_handle_reset on v2:172.21.15.96:6836/3978388286 2024-09-19T05:06:41.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.028+0000 7fdf71a4c740 1 -- 172.21.15.43:0/3082923298 shutdown_connections 2024-09-19T05:06:41.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.028+0000 7fdf71a4c740 1 -- 172.21.15.43:0/3082923298 wait complete. 2024-09-19T05:06:41.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.028+0000 7fdf71a4c740 20 client.15651 trim_cache size 0 max 16384 2024-09-19T05:06:41.036 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:06:41.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.029+0000 7fdf6ee00640 1 -- 172.21.15.43:0/2311601544 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fdf4c0a33b0 msgr2=0x7fdf4c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:41.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.029+0000 7fdf6ee00640 1 --2- 172.21.15.43:0/2311601544 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fdf4c0a33b0 0x7fdf4c0a5870 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdf60005e30 tx=0x7fdf60005d80 comp rx=0 tx=0).stop 2024-09-19T05:06:41.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.029+0000 7fdf6ee00640 1 -- 172.21.15.43:0/2311601544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf68076720 msgr2=0x7fdf6819eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:41.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.029+0000 7fdf6ee00640 1 --2- 172.21.15.43:0/2311601544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf68076720 0x7fdf6819eb60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fdf5800e9f0 tx=0x7fdf5800eeb0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.029+0000 7fdf6ee00640 1 -- 172.21.15.43:0/2311601544 shutdown_connections 2024-09-19T05:06:41.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.029+0000 7fdf6ee00640 1 --2- 172.21.15.43:0/2311601544 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fdf4c0a33b0 0x7fdf4c0a5870 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.029+0000 7fdf6ee00640 1 --2- 172.21.15.43:0/2311601544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf68112a50 0x7fdf681a3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.029+0000 7fdf6ee00640 1 --2- 172.21.15.43:0/2311601544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf68077040 0x7fdf6819f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.029+0000 7fdf6ee00640 1 --2- 172.21.15.43:0/2311601544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf68076720 0x7fdf6819eb60 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:41.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.029+0000 7fdf6ee00640 1 -- 172.21.15.43:0/2311601544 >> 172.21.15.43:0/2311601544 conn(0x7fdf680ffa50 msgr2=0x7fdf68101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:41.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.030+0000 7fdf6ee00640 1 -- 172.21.15.43:0/2311601544 shutdown_connections 2024-09-19T05:06:41.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:41.030+0000 7fdf6ee00640 1 -- 172.21.15.43:0/2311601544 wait complete. 2024-09-19T05:06:41.756 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-19T05:06:41.756 INFO:tasks.workunit.client.0.smithi043.stdout: Children see throughput for 1 readers = 5193929.00 kB/sec 2024-09-19T05:06:41.756 INFO:tasks.workunit.client.0.smithi043.stdout: Parent sees throughput for 1 readers = 5173990.94 kB/sec 2024-09-19T05:06:41.757 INFO:tasks.workunit.client.0.smithi043.stdout: Min throughput per process = 5193929.00 kB/sec 2024-09-19T05:06:41.757 INFO:tasks.workunit.client.0.smithi043.stdout: Max throughput per process = 5193929.00 kB/sec 2024-09-19T05:06:41.757 INFO:tasks.workunit.client.0.smithi043.stdout: Avg throughput per process = 5193929.00 kB/sec 2024-09-19T05:06:41.757 INFO:tasks.workunit.client.0.smithi043.stdout: Min xfer = 1048576.00 kB 2024-09-19T05:06:41.805 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T05:06:42.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:41 smithi043 ceph-mon[30800]: pgmap v64: 97 pgs: 97 active+clean; 1.0 GiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 171 MiB/s wr, 79 op/s 2024-09-19T05:06:42.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:41 smithi151 ceph-mon[35425]: pgmap v64: 97 pgs: 97 active+clean; 1.0 GiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 171 MiB/s wr, 79 op/s 2024-09-19T05:06:42.188 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:06:42.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:41 smithi096 ceph-mon[35085]: pgmap v64: 97 pgs: 97 active+clean; 1.0 GiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 171 MiB/s wr, 79 op/s 2024-09-19T05:06:42.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.725+0000 7f275b600640 1 -- 172.21.15.43:0/2838136060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2754104010 msgr2=0x7f27541043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:42.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.725+0000 7f275b600640 1 --2- 172.21.15.43:0/2838136060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2754104010 0x7f27541043f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2744009900 tx=0x7f274402a3f0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.725+0000 7f275b600640 1 -- 172.21.15.43:0/2838136060 shutdown_connections 2024-09-19T05:06:42.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.725+0000 7f275b600640 1 --2- 172.21.15.43:0/2838136060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2754109140 0x7f2754110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.725+0000 7f275b600640 1 --2- 172.21.15.43:0/2838136060 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27541049c0 0x7f2754108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.725+0000 7f275b600640 1 --2- 172.21.15.43:0/2838136060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2754104010 0x7f27541043f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.725+0000 7f275b600640 1 -- 172.21.15.43:0/2838136060 >> 172.21.15.43:0/2838136060 conn(0x7f27540ffa50 msgr2=0x7f2754101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:42.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.725+0000 7f275b600640 1 -- 172.21.15.43:0/2838136060 shutdown_connections 2024-09-19T05:06:42.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.725+0000 7f275b600640 1 -- 172.21.15.43:0/2838136060 wait complete. 2024-09-19T05:06:42.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.726+0000 7f275b600640 1 Processor -- start 2024-09-19T05:06:42.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.726+0000 7f275b600640 1 -- start start 2024-09-19T05:06:42.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.726+0000 7f275b600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2754104010 0x7f275419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:42.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.726+0000 7f275b600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27541049c0 0x7f275419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:42.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.726+0000 7f275b600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2754109140 0x7f275419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:42.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.726+0000 7f275b600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2754114160 con 0x7f27541049c0 2024-09-19T05:06:42.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.726+0000 7f275b600640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2754113fe0 con 0x7f2754104010 2024-09-19T05:06:42.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.726+0000 7f275b600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f27541142e0 con 0x7f2754109140 2024-09-19T05:06:42.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.727+0000 7f274be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27541049c0 0x7f275419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:42.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.727+0000 7f274be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27541049c0 0x7f275419ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46892/0 (socket says 172.21.15.43:46892) 2024-09-19T05:06:42.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.727+0000 7f274be00640 1 -- 172.21.15.43:0/1108123514 learned_addr learned my addr 172.21.15.43:0/1108123514 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:42.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.727+0000 7f2758c00640 1 --2- 172.21.15.43:0/1108123514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2754104010 0x7f275419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:42.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.727+0000 7f2759600640 1 --2- 172.21.15.43:0/1108123514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2754109140 0x7f275419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:42.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.727+0000 7f274be00640 1 -- 172.21.15.43:0/1108123514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2754109140 msgr2=0x7f275419ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:42.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.727+0000 7f274be00640 1 --2- 172.21.15.43:0/1108123514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2754109140 0x7f275419ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.727+0000 7f274be00640 1 -- 172.21.15.43:0/1108123514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2754104010 msgr2=0x7f275419a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:42.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.727+0000 7f274be00640 1 --2- 172.21.15.43:0/1108123514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2754104010 0x7f275419a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.727+0000 7f274be00640 1 -- 172.21.15.43:0/1108123514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f275419f5f0 con 0x7f27541049c0 2024-09-19T05:06:42.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.727+0000 7f2758c00640 1 --2- 172.21.15.43:0/1108123514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2754104010 0x7f275419a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:06:42.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.728+0000 7f274be00640 1 --2- 172.21.15.43:0/1108123514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27541049c0 0x7f275419ab80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f273c00e9f0 tx=0x7f273c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:42.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.728+0000 7f2752000640 1 -- 172.21.15.43:0/1108123514 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c00cd80 con 0x7f27541049c0 2024-09-19T05:06:42.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.728+0000 7f275b600640 1 -- 172.21.15.43:0/1108123514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f275419f8e0 con 0x7f27541049c0 2024-09-19T05:06:42.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.728+0000 7f2752000640 1 -- 172.21.15.43:0/1108123514 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f273c005500 con 0x7f27541049c0 2024-09-19T05:06:42.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.728+0000 7f2752000640 1 -- 172.21.15.43:0/1108123514 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c018630 con 0x7f27541049c0 2024-09-19T05:06:42.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.728+0000 7f275b600640 1 -- 172.21.15.43:0/1108123514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27541a71c0 con 0x7f27541049c0 2024-09-19T05:06:42.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.731+0000 7f275dfc6740 1 Processor -- start 2024-09-19T05:06:42.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.731+0000 7f2752000640 1 -- 172.21.15.43:0/1108123514 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f273c0046c0 con 0x7f27541049c0 2024-09-19T05:06:42.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.731+0000 7f275dfc6740 1 -- start start 2024-09-19T05:06:42.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.732+0000 7f275dfc6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d079d12c70 0x55d079d13050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:42.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.732+0000 7f275dfc6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d079d13590 0x55d079d1d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:42.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.732+0000 7f275dfc6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d079d1de50 0x55d079d20240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:42.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.732+0000 7f275dfc6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d079c793e0 con 0x55d079d1de50 2024-09-19T05:06:42.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.732+0000 7f275dfc6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d079c20d80 con 0x55d079d13590 2024-09-19T05:06:42.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.732+0000 7f275dfc6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d079c753a0 con 0x55d079d12c70 2024-09-19T05:06:42.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.732+0000 7f2758c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d079d13590 0x55d079d1d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:42.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.732+0000 7f2758c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d079d13590 0x55d079d1d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:45840/0 (socket says 172.21.15.43:45840) 2024-09-19T05:06:42.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.732+0000 7f2758c00640 1 -- 172.21.15.43:0/153061616 learned_addr learned my addr 172.21.15.43:0/153061616 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:42.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.732+0000 7f274be00640 1 --2- 172.21.15.43:0/153061616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d079d1de50 0x55d079d20240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:42.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.732+0000 7f2759600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d079d12c70 0x55d079d13050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:42.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.732+0000 7f2758c00640 1 -- 172.21.15.43:0/153061616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d079d12c70 msgr2=0x55d079d13050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:42.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.732+0000 7f2758c00640 1 --2- 172.21.15.43:0/153061616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d079d12c70 0x55d079d13050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.732+0000 7f2758c00640 1 -- 172.21.15.43:0/153061616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d079d1de50 msgr2=0x55d079d20240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:42.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.732+0000 7f2758c00640 1 --2- 172.21.15.43:0/153061616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d079d1de50 0x55d079d20240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.732+0000 7f2758c00640 1 -- 172.21.15.43:0/153061616 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d079c3fae0 con 0x55d079d13590 2024-09-19T05:06:42.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.733+0000 7f2758c00640 1 --2- 172.21.15.43:0/153061616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d079d13590 0x55d079d1d910 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2744033000 tx=0x7f2744005e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1e10c7e1474da12 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:42.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.733+0000 7f2752000640 1 --2- 172.21.15.43:0/1108123514 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f27380a33b0 0x7f27380a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:42.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.733+0000 7f2749c00640 1 -- 172.21.15.43:0/153061616 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274403f070 con 0x55d079d13590 2024-09-19T05:06:42.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.733+0000 7f2752000640 1 -- 172.21.15.43:0/1108123514 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f273c0d0300 con 0x7f27541049c0 2024-09-19T05:06:42.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.733+0000 7f2749c00640 1 -- 172.21.15.43:0/153061616 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2744004030 con 0x55d079d13590 2024-09-19T05:06:42.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.733+0000 7f2749c00640 1 -- 172.21.15.43:0/153061616 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2744035070 con 0x55d079d13590 2024-09-19T05:06:42.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.733+0000 7f2759600640 1 --2- 172.21.15.43:0/1108123514 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f27380a33b0 0x7f27380a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:42.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.734+0000 7f275dfc6740 1 -- 172.21.15.43:0/153061616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d079d13590 msgr2=0x55d079d1d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:42.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.734+0000 7f275dfc6740 1 --2- 172.21.15.43:0/153061616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d079d13590 0x55d079d1d910 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2744033000 tx=0x7f2744005e60 comp rx=0 tx=0).stop 2024-09-19T05:06:42.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.734+0000 7f2759600640 1 --2- 172.21.15.43:0/1108123514 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f27380a33b0 0x7f27380a5870 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f274c005fd0 tx=0x7f274c009500 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:42.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.734+0000 7f275dfc6740 1 -- 172.21.15.43:0/153061616 shutdown_connections 2024-09-19T05:06:42.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.734+0000 7f275dfc6740 1 --2- 172.21.15.43:0/153061616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d079d1de50 0x55d079d20240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.734+0000 7f275dfc6740 1 --2- 172.21.15.43:0/153061616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d079d13590 0x55d079d1d910 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.734+0000 7f275dfc6740 1 --2- 172.21.15.43:0/153061616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d079d12c70 0x55d079d13050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.734+0000 7f275dfc6740 1 -- 172.21.15.43:0/153061616 >> 172.21.15.43:0/153061616 conn(0x55d079d0f200 msgr2=0x55d079d0f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:42.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.734+0000 7f275dfc6740 1 -- 172.21.15.43:0/153061616 shutdown_connections 2024-09-19T05:06:42.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.734+0000 7f275dfc6740 1 -- 172.21.15.43:0/153061616 wait complete. 2024-09-19T05:06:42.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.735+0000 7f275dfc6740 1 -- start start 2024-09-19T05:06:42.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.735+0000 7f275dfc6740 1 Processor -- start 2024-09-19T05:06:42.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.735+0000 7f275dfc6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d079d12c70 0x55d079d1adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:42.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.735+0000 7f275dfc6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d079d13590 0x55d079d1b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:42.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.735+0000 7f275dfc6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d079d1de50 0x55d079d1b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:42.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.735+0000 7f275dfc6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d079d20a20 con 0x55d079d12c70 2024-09-19T05:06:42.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.735+0000 7f275dfc6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d079c340f0 con 0x55d079d1de50 2024-09-19T05:06:42.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.735+0000 7f275dfc6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d079d20ba0 con 0x55d079d13590 2024-09-19T05:06:42.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.735+0000 7f275dfc6740 10 client.0 _ll_register_callbacks cb 0x55d079c1bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:42.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.736+0000 7f2758c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d079d12c70 0x55d079d1adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:42.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.736+0000 7f2758c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d079d12c70 0x55d079d1adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46920/0 (socket says 172.21.15.43:46920) 2024-09-19T05:06:42.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.736+0000 7f2758c00640 1 -- 172.21.15.43:0/2447574134 learned_addr learned my addr 172.21.15.43:0/2447574134 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:42.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.736+0000 7f274be00640 1 --2- 172.21.15.43:0/2447574134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d079d13590 0x55d079d1b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:42.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.736+0000 7f2759600640 1 --2- 172.21.15.43:0/2447574134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d079d1de50 0x55d079d1b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:42.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.736+0000 7f2758c00640 1 -- 172.21.15.43:0/2447574134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d079d13590 msgr2=0x55d079d1b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:42.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.736+0000 7f2758c00640 1 --2- 172.21.15.43:0/2447574134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d079d13590 0x55d079d1b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.736+0000 7f2758c00640 1 -- 172.21.15.43:0/2447574134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d079d1de50 msgr2=0x55d079d1b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:42.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.736+0000 7f2758c00640 1 --2- 172.21.15.43:0/2447574134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d079d1de50 0x55d079d1b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.736+0000 7f2758c00640 1 -- 172.21.15.43:0/2447574134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0799ea3c0 con 0x55d079d12c70 2024-09-19T05:06:42.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.737+0000 7f2758c00640 1 --2- 172.21.15.43:0/2447574134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d079d12c70 0x55d079d1adc0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f274402a910 tx=0x7f2744031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c6df3097b28893f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:42.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.737+0000 7f2742000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:06:42.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.737+0000 7f2742000640 1 -- 172.21.15.43:0/2447574134 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2744035070 con 0x55d079d12c70 2024-09-19T05:06:42.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.737+0000 7f2742000640 1 -- 172.21.15.43:0/2447574134 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f27440317b0 con 0x55d079d12c70 2024-09-19T05:06:42.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.737+0000 7f275dfc6740 1 -- 172.21.15.43:0/2447574134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2744004030 con 0x55d079d12c70 2024-09-19T05:06:42.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.737+0000 7f2742000640 1 -- 172.21.15.43:0/2447574134 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2744055050 con 0x55d079d12c70 2024-09-19T05:06:42.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.737+0000 7f275dfc6740 10 client.15663 fetch_fsmap learned FSMap version 29 2024-09-19T05:06:42.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.737+0000 7f275dfc6740 1 -- 172.21.15.43:0/2447574134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d079d2cae0 con 0x55d079d12c70 2024-09-19T05:06:42.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.737+0000 7f2742000640 1 -- 172.21.15.43:0/2447574134 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2744004030 con 0x55d079d12c70 2024-09-19T05:06:42.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.738+0000 7f2742000640 1 -- 172.21.15.43:0/2447574134 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2744004750 con 0x55d079d12c70 2024-09-19T05:06:42.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.738+0000 7f275dfc6740 10 client.15663 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:06:42.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.738+0000 7f275dfc6740 10 client.15663 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T05:06:42.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.738+0000 7f275dfc6740 20 client.15663 populate_metadata read hostname 'smithi043' 2024-09-19T05:06:42.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.738+0000 7f275dfc6740 1 --2- 172.21.15.43:0/2447574134 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55d079d2de30 0x55d079d30220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:42.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.738+0000 7f275dfc6740 4 client.15663 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:06:42.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.738+0000 7f275dfc6740 1 -- 172.21.15.43:0/2447574134 --> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d079c86950 con 0x55d079d2de30 2024-09-19T05:06:42.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.738+0000 7f274be00640 1 --2- 172.21.15.43:0/2447574134 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55d079d2de30 0x55d079d30220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:42.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.739+0000 7f274be00640 1 --2- 172.21.15.43:0/2447574134 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55d079d2de30 0x55d079d30220 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=afa127f83c0c3707 server_cookie=c35b249f9c198d7b in_seq=0 out_seq=0 2024-09-19T05:06:42.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.739+0000 7f2742000640 10 client.15663 ms_handle_connect on v2:172.21.15.151:6838/3794104313 2024-09-19T05:06:42.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.740+0000 7f2742000640 1 -- 172.21.15.43:0/2447574134 <== mds.2 v2:172.21.15.151:6838/3794104313 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d079c86950 con 0x55d079d2de30 2024-09-19T05:06:42.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.740+0000 7f2742000640 10 client.15663 handle_command_reply: tid=0 2024-09-19T05:06:42.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.740+0000 7f275dfc6740 1 client.15663 shutdown 2024-09-19T05:06:42.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.740+0000 7f275dfc6740 1 -- 172.21.15.43:0/2447574134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d079d12c70 msgr2=0x55d079d1adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:42.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.740+0000 7f275dfc6740 1 --2- 172.21.15.43:0/2447574134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d079d12c70 0x55d079d1adc0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f274402a910 tx=0x7f2744031400 comp rx=0 tx=0).stop 2024-09-19T05:06:42.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.740+0000 7f275dfc6740 1 -- 172.21.15.43:0/2447574134 shutdown_connections 2024-09-19T05:06:42.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.740+0000 7f275dfc6740 1 --2- 172.21.15.43:0/2447574134 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55d079d2de30 0x55d079d30220 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.740+0000 7f275dfc6740 1 --2- 172.21.15.43:0/2447574134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d079d1de50 0x55d079d1b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.740+0000 7f275dfc6740 1 --2- 172.21.15.43:0/2447574134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d079d13590 0x55d079d1b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.740+0000 7f275dfc6740 1 --2- 172.21.15.43:0/2447574134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d079d12c70 0x55d079d1adc0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.740+0000 7f2742000640 0 client.15663 ms_handle_reset on v2:172.21.15.151:6838/3794104313 2024-09-19T05:06:42.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.740+0000 7f275dfc6740 1 -- 172.21.15.43:0/2447574134 >> 172.21.15.43:0/2447574134 conn(0x55d079d0f200 msgr2=0x55d079d14ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:42.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.740+0000 7f275dfc6740 1 -- 172.21.15.43:0/2447574134 shutdown_connections 2024-09-19T05:06:42.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.740+0000 7f275dfc6740 1 -- 172.21.15.43:0/2447574134 wait complete. 2024-09-19T05:06:42.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.740+0000 7f275dfc6740 20 client.15663 trim_cache size 0 max 16384 2024-09-19T05:06:42.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.754+0000 7f275dfc6740 1 Processor -- start 2024-09-19T05:06:42.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.754+0000 7f275dfc6740 1 -- start start 2024-09-19T05:06:42.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.754+0000 7f275dfc6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d079d12c70 0x55d079d18020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:42.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.754+0000 7f275dfc6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d079d13590 0x55d079d18560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:42.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.754+0000 7f275dfc6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d079d1de50 0x55d079d1c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:42.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.754+0000 7f275dfc6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f27541afa80 con 0x55d079d12c70 2024-09-19T05:06:42.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.754+0000 7f275dfc6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f27541b15c0 con 0x55d079d13590 2024-09-19T05:06:42.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.754+0000 7f275dfc6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f27541b0150 con 0x55d079d1de50 2024-09-19T05:06:42.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.755+0000 7f2758c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d079d12c70 0x55d079d18020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:42.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.755+0000 7f2758c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d079d12c70 0x55d079d18020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46928/0 (socket says 172.21.15.43:46928) 2024-09-19T05:06:42.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.755+0000 7f2758c00640 1 -- 172.21.15.43:0/3298661709 learned_addr learned my addr 172.21.15.43:0/3298661709 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:42.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.755+0000 7f274be00640 1 --2- 172.21.15.43:0/3298661709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d079d13590 0x55d079d18560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:42.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.755+0000 7f2759600640 1 --2- 172.21.15.43:0/3298661709 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d079d1de50 0x55d079d1c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:42.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.755+0000 7f2758c00640 1 -- 172.21.15.43:0/3298661709 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d079d1de50 msgr2=0x55d079d1c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:42.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.755+0000 7f2758c00640 1 --2- 172.21.15.43:0/3298661709 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d079d1de50 0x55d079d1c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.755+0000 7f2758c00640 1 -- 172.21.15.43:0/3298661709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d079d13590 msgr2=0x55d079d18560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:42.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.755+0000 7f2758c00640 1 --2- 172.21.15.43:0/3298661709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d079d13590 0x55d079d18560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.755+0000 7f2758c00640 1 -- 172.21.15.43:0/3298661709 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d079d133c0 con 0x55d079d12c70 2024-09-19T05:06:42.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.755+0000 7f274be00640 1 --2- 172.21.15.43:0/3298661709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d079d13590 0x55d079d18560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:42.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.756+0000 7f2758c00640 1 --2- 172.21.15.43:0/3298661709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d079d12c70 0x55d079d18020 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2744004af0 tx=0x7f27440056c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4575641c262a91f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:42.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.756+0000 7f2749200640 1 -- 172.21.15.43:0/3298661709 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2744035070 con 0x55d079d12c70 2024-09-19T05:06:42.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.756+0000 7f2749200640 1 -- 172.21.15.43:0/3298661709 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f274404c070 con 0x55d079d12c70 2024-09-19T05:06:42.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.756+0000 7f2749200640 1 -- 172.21.15.43:0/3298661709 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274403f040 con 0x55d079d12c70 2024-09-19T05:06:42.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.757+0000 7f275dfc6740 1 -- 172.21.15.43:0/3298661709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d079d12c70 msgr2=0x55d079d18020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:42.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.757+0000 7f275dfc6740 1 --2- 172.21.15.43:0/3298661709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d079d12c70 0x55d079d18020 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2744004af0 tx=0x7f27440056c0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.757+0000 7f275dfc6740 1 -- 172.21.15.43:0/3298661709 shutdown_connections 2024-09-19T05:06:42.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.757+0000 7f275dfc6740 1 --2- 172.21.15.43:0/3298661709 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d079d1de50 0x55d079d1c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.757+0000 7f275dfc6740 1 --2- 172.21.15.43:0/3298661709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d079d13590 0x55d079d18560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.757+0000 7f275dfc6740 1 --2- 172.21.15.43:0/3298661709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d079d12c70 0x55d079d18020 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.757+0000 7f275dfc6740 1 -- 172.21.15.43:0/3298661709 >> 172.21.15.43:0/3298661709 conn(0x55d079d0f200 msgr2=0x55d079d19eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:42.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.757+0000 7f275dfc6740 1 -- 172.21.15.43:0/3298661709 shutdown_connections 2024-09-19T05:06:42.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.757+0000 7f275dfc6740 1 -- 172.21.15.43:0/3298661709 wait complete. 2024-09-19T05:06:42.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.757+0000 7f275dfc6740 1 -- start start 2024-09-19T05:06:42.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.757+0000 7f275dfc6740 1 Processor -- start 2024-09-19T05:06:42.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.758+0000 7f275dfc6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d079d12c70 0x55d079d33850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:42.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.758+0000 7f275dfc6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d079d13590 0x55d079d30ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:42.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.758+0000 7f275dfc6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d079d1de50 0x55d079d310e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:42.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.758+0000 7f275dfc6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f27541b0ef0 con 0x55d079d1de50 2024-09-19T05:06:42.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.758+0000 7f275dfc6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f27541b0820 con 0x55d079d13590 2024-09-19T05:06:42.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.758+0000 7f275dfc6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f27541ada60 con 0x55d079d12c70 2024-09-19T05:06:42.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.758+0000 7f275dfc6740 10 client.0 _ll_register_callbacks cb 0x7f273c0d3c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:42.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.758+0000 7f2758c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d079d12c70 0x55d079d33850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:42.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.758+0000 7f2759600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d079d1de50 0x55d079d310e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:42.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.758+0000 7f2758c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d079d12c70 0x55d079d33850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:50126/0 (socket says 172.21.15.43:50126) 2024-09-19T05:06:42.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.758+0000 7f2758c00640 1 -- 172.21.15.43:0/3047156148 learned_addr learned my addr 172.21.15.43:0/3047156148 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:42.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.758+0000 7f274be00640 1 --2- 172.21.15.43:0/3047156148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d079d13590 0x55d079d30ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:42.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.759+0000 7f2758c00640 1 -- 172.21.15.43:0/3047156148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d079d13590 msgr2=0x55d079d30ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:42.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.759+0000 7f2758c00640 1 --2- 172.21.15.43:0/3047156148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d079d13590 0x55d079d30ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.759+0000 7f2758c00640 1 -- 172.21.15.43:0/3047156148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d079d1de50 msgr2=0x55d079d310e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:42.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.759+0000 7f2758c00640 1 --2- 172.21.15.43:0/3047156148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d079d1de50 0x55d079d310e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.759+0000 7f2758c00640 1 -- 172.21.15.43:0/3047156148 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d079d317d0 con 0x55d079d12c70 2024-09-19T05:06:42.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.759+0000 7f2758c00640 1 --2- 172.21.15.43:0/3047156148 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d079d12c70 0x55d079d33850 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f27440059d0 tx=0x7f2744004f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cfb8c749998e564 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:42.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.759+0000 7f2742000640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:06:42.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.759+0000 7f2742000640 1 -- 172.21.15.43:0/3047156148 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274403f040 con 0x55d079d12c70 2024-09-19T05:06:42.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.759+0000 7f275dfc6740 1 -- 172.21.15.43:0/3047156148 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f274404c070 con 0x55d079d12c70 2024-09-19T05:06:42.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.759+0000 7f2742000640 1 -- 172.21.15.43:0/3047156148 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f27440319c0 con 0x55d079d12c70 2024-09-19T05:06:42.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.760+0000 7f2742000640 1 -- 172.21.15.43:0/3047156148 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2744035040 con 0x55d079d12c70 2024-09-19T05:06:42.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.760+0000 7f275dfc6740 10 client.25633 fetch_fsmap learned FSMap version 29 2024-09-19T05:06:42.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.760+0000 7f275dfc6740 1 -- 172.21.15.43:0/3047156148 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d079d98610 con 0x55d079d12c70 2024-09-19T05:06:42.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.760+0000 7f2742000640 1 -- 172.21.15.43:0/3047156148 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f274404c070 con 0x55d079d12c70 2024-09-19T05:06:42.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.760+0000 7f2742000640 1 -- 172.21.15.43:0/3047156148 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f274406b950 con 0x55d079d12c70 2024-09-19T05:06:42.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.760+0000 7f275dfc6740 10 client.25633 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:06:42.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.760+0000 7f275dfc6740 10 client.25633 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T05:06:42.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.760+0000 7f275dfc6740 20 client.25633 populate_metadata read hostname 'smithi043' 2024-09-19T05:06:42.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.760+0000 7f275dfc6740 1 --2- 172.21.15.43:0/3047156148 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55d079d98c20 0x55d079d9b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:42.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.760+0000 7f275dfc6740 4 client.25633 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:06:42.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.760+0000 7f275dfc6740 1 -- 172.21.15.43:0/3047156148 --> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d079c80e60 con 0x55d079d98c20 2024-09-19T05:06:42.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.761+0000 7f274be00640 1 --2- 172.21.15.43:0/3047156148 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55d079d98c20 0x55d079d9b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:42.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.761+0000 7f274be00640 1 --2- 172.21.15.43:0/3047156148 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55d079d98c20 0x55d079d9b010 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6106ae9a7748f248 server_cookie=4a14c5068ad499f7 in_seq=0 out_seq=0 2024-09-19T05:06:42.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.761+0000 7f2742000640 10 client.25633 ms_handle_connect on v2:172.21.15.151:6838/3794104313 2024-09-19T05:06:42.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.761+0000 7f2742000640 1 -- 172.21.15.43:0/3047156148 <== mds.2 v2:172.21.15.151:6838/3794104313 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d079c80e60 con 0x55d079d98c20 2024-09-19T05:06:42.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.761+0000 7f2742000640 10 client.25633 handle_command_reply: tid=0 2024-09-19T05:06:42.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.761+0000 7f275dfc6740 1 client.25633 shutdown 2024-09-19T05:06:42.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.762+0000 7f275dfc6740 1 -- 172.21.15.43:0/3047156148 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d079d12c70 msgr2=0x55d079d33850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:42.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.762+0000 7f275dfc6740 1 --2- 172.21.15.43:0/3047156148 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d079d12c70 0x55d079d33850 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f27440059d0 tx=0x7f2744004f90 comp rx=0 tx=0).stop 2024-09-19T05:06:42.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.762+0000 7f275dfc6740 1 -- 172.21.15.43:0/3047156148 shutdown_connections 2024-09-19T05:06:42.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.762+0000 7f275dfc6740 1 --2- 172.21.15.43:0/3047156148 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55d079d98c20 0x55d079d9b010 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.762+0000 7f275dfc6740 1 --2- 172.21.15.43:0/3047156148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d079d1de50 0x55d079d310e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.762+0000 7f275dfc6740 1 --2- 172.21.15.43:0/3047156148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d079d13590 0x55d079d30ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.762+0000 7f275dfc6740 1 --2- 172.21.15.43:0/3047156148 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d079d12c70 0x55d079d33850 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.762+0000 7f2742000640 0 client.25633 ms_handle_reset on v2:172.21.15.151:6838/3794104313 2024-09-19T05:06:42.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.762+0000 7f275dfc6740 1 -- 172.21.15.43:0/3047156148 >> 172.21.15.43:0/3047156148 conn(0x55d079d0f200 msgr2=0x55d079d1c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:42.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.762+0000 7f275dfc6740 1 -- 172.21.15.43:0/3047156148 shutdown_connections 2024-09-19T05:06:42.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.762+0000 7f275dfc6740 1 -- 172.21.15.43:0/3047156148 wait complete. 2024-09-19T05:06:42.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.762+0000 7f275dfc6740 20 client.25633 trim_cache size 0 max 16384 2024-09-19T05:06:42.769 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:06:42.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.763+0000 7f275b600640 1 -- 172.21.15.43:0/1108123514 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f27380a33b0 msgr2=0x7f27380a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:42.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.763+0000 7f275b600640 1 --2- 172.21.15.43:0/1108123514 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f27380a33b0 0x7f27380a5870 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f274c005fd0 tx=0x7f274c009500 comp rx=0 tx=0).stop 2024-09-19T05:06:42.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.763+0000 7f275b600640 1 -- 172.21.15.43:0/1108123514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27541049c0 msgr2=0x7f275419ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:42.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.763+0000 7f275b600640 1 --2- 172.21.15.43:0/1108123514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27541049c0 0x7f275419ab80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f273c00e9f0 tx=0x7f273c00edf0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.763+0000 7f275b600640 1 -- 172.21.15.43:0/1108123514 shutdown_connections 2024-09-19T05:06:42.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.763+0000 7f275b600640 1 --2- 172.21.15.43:0/1108123514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2754109140 0x7f275419ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.763+0000 7f275b600640 1 --2- 172.21.15.43:0/1108123514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27541049c0 0x7f275419ab80 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.763+0000 7f275b600640 1 --2- 172.21.15.43:0/1108123514 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f27380a33b0 0x7f27380a5870 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.763+0000 7f275b600640 1 --2- 172.21.15.43:0/1108123514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2754104010 0x7f275419a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:42.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.763+0000 7f275b600640 1 -- 172.21.15.43:0/1108123514 >> 172.21.15.43:0/1108123514 conn(0x7f27540ffa50 msgr2=0x7f2754100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:42.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.763+0000 7f275b600640 1 -- 172.21.15.43:0/1108123514 shutdown_connections 2024-09-19T05:06:42.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:42.763+0000 7f275b600640 1 -- 172.21.15.43:0/1108123514 wait complete. 2024-09-19T05:06:43.649 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T05:06:44.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:43 smithi043 ceph-mon[30800]: pgmap v65: 97 pgs: 97 active+clean; 1.0 GiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 147 MiB/s wr, 45 op/s 2024-09-19T05:06:44.035 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:06:44.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:43 smithi151 ceph-mon[35425]: pgmap v65: 97 pgs: 97 active+clean; 1.0 GiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 147 MiB/s wr, 45 op/s 2024-09-19T05:06:44.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:43 smithi096 ceph-mon[35085]: pgmap v65: 97 pgs: 97 active+clean; 1.0 GiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 147 MiB/s wr, 45 op/s 2024-09-19T05:06:44.265 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-19T05:06:44.265 INFO:tasks.workunit.client.0.smithi043.stdout: Children see throughput for 1 re-readers = 3487859.25 kB/sec 2024-09-19T05:06:44.265 INFO:tasks.workunit.client.0.smithi043.stdout: Parent sees throughput for 1 re-readers = 3477148.69 kB/sec 2024-09-19T05:06:44.265 INFO:tasks.workunit.client.0.smithi043.stdout: Min throughput per process = 3487859.25 kB/sec 2024-09-19T05:06:44.265 INFO:tasks.workunit.client.0.smithi043.stdout: Max throughput per process = 3487859.25 kB/sec 2024-09-19T05:06:44.265 INFO:tasks.workunit.client.0.smithi043.stdout: Avg throughput per process = 3487859.25 kB/sec 2024-09-19T05:06:44.265 INFO:tasks.workunit.client.0.smithi043.stdout: Min xfer = 1048576.00 kB 2024-09-19T05:06:44.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.795+0000 7f1340200640 1 -- 172.21.15.43:0/2664102774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1338104010 msgr2=0x7f13381043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:44.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.795+0000 7f1340200640 1 --2- 172.21.15.43:0/2664102774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1338104010 0x7f13381043f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f132c009860 tx=0x7f132c02a340 comp rx=0 tx=0).stop 2024-09-19T05:06:44.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.795+0000 7f1340200640 1 -- 172.21.15.43:0/2664102774 shutdown_connections 2024-09-19T05:06:44.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.795+0000 7f1340200640 1 --2- 172.21.15.43:0/2664102774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1338109140 0x7f1338110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.795+0000 7f1340200640 1 --2- 172.21.15.43:0/2664102774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13381049c0 0x7f1338108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.795+0000 7f1340200640 1 --2- 172.21.15.43:0/2664102774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1338104010 0x7f13381043f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.795+0000 7f1340200640 1 -- 172.21.15.43:0/2664102774 >> 172.21.15.43:0/2664102774 conn(0x7f13380ffa50 msgr2=0x7f1338101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:44.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.795+0000 7f1340200640 1 -- 172.21.15.43:0/2664102774 shutdown_connections 2024-09-19T05:06:44.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.795+0000 7f1340200640 1 -- 172.21.15.43:0/2664102774 wait complete. 2024-09-19T05:06:44.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.796+0000 7f1340200640 1 Processor -- start 2024-09-19T05:06:44.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.797+0000 7f1340200640 1 -- start start 2024-09-19T05:06:44.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.797+0000 7f1340200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1338104010 0x7f133819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:44.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.797+0000 7f1340200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13381049c0 0x7f133819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:44.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.797+0000 7f1340200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1338109140 0x7f133819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:44.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.797+0000 7f1340200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1338114170 con 0x7f1338104010 2024-09-19T05:06:44.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.797+0000 7f1340200640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1338113ff0 con 0x7f1338109140 2024-09-19T05:06:44.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.797+0000 7f1340200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f13381142f0 con 0x7f13381049c0 2024-09-19T05:06:44.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.797+0000 7f133d800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1338104010 0x7f133819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:44.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.797+0000 7f133d800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1338104010 0x7f133819a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46962/0 (socket says 172.21.15.43:46962) 2024-09-19T05:06:44.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.797+0000 7f133d800640 1 -- 172.21.15.43:0/854142618 learned_addr learned my addr 172.21.15.43:0/854142618 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:44.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.797+0000 7f133ce00640 1 --2- 172.21.15.43:0/854142618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13381049c0 0x7f133819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:44.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.798+0000 7f133e200640 1 --2- 172.21.15.43:0/854142618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1338109140 0x7f133819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:44.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.798+0000 7f133d800640 1 -- 172.21.15.43:0/854142618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13381049c0 msgr2=0x7f133819ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:44.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.798+0000 7f133d800640 1 --2- 172.21.15.43:0/854142618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13381049c0 0x7f133819ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.798+0000 7f133d800640 1 -- 172.21.15.43:0/854142618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1338109140 msgr2=0x7f133819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:44.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.798+0000 7f133d800640 1 --2- 172.21.15.43:0/854142618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1338109140 0x7f133819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.798+0000 7f133d800640 1 -- 172.21.15.43:0/854142618 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f133819f5e0 con 0x7f1338104010 2024-09-19T05:06:44.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.798+0000 7f133ce00640 1 --2- 172.21.15.43:0/854142618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13381049c0 0x7f133819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:44.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.798+0000 7f133e200640 1 --2- 172.21.15.43:0/854142618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1338109140 0x7f133819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:44.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.798+0000 7f133d800640 1 --2- 172.21.15.43:0/854142618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1338104010 0x7f133819a630 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f132c035040 tx=0x7f132c033b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:44.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.799+0000 7f1336000640 1 -- 172.21.15.43:0/854142618 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132c037040 con 0x7f1338104010 2024-09-19T05:06:44.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.799+0000 7f1340200640 1 -- 172.21.15.43:0/854142618 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f133819f870 con 0x7f1338104010 2024-09-19T05:06:44.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.799+0000 7f1336000640 1 -- 172.21.15.43:0/854142618 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f132c040070 con 0x7f1338104010 2024-09-19T05:06:44.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.799+0000 7f1336000640 1 -- 172.21.15.43:0/854142618 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132c0039b0 con 0x7f1338104010 2024-09-19T05:06:44.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.799+0000 7f1340200640 1 -- 172.21.15.43:0/854142618 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13381a71b0 con 0x7f1338104010 2024-09-19T05:06:44.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.802+0000 7f1342df4740 1 Processor -- start 2024-09-19T05:06:44.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.802+0000 7f1342df4740 1 -- start start 2024-09-19T05:06:44.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.803+0000 7f1342df4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55806eb85c70 0x55806eb86050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:44.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.803+0000 7f1342df4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55806eb86590 0x55806eb90910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:44.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.803+0000 7f1342df4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55806eb90e50 0x55806eb93240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:44.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.803+0000 7f1342df4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55806eaec3e0 con 0x55806eb85c70 2024-09-19T05:06:44.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.803+0000 7f1342df4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55806ea93d80 con 0x55806eb86590 2024-09-19T05:06:44.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.803+0000 7f1342df4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55806eae83a0 con 0x55806eb90e50 2024-09-19T05:06:44.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.803+0000 7f133e200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55806eb85c70 0x55806eb86050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:44.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.803+0000 7f133ce00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55806eb90e50 0x55806eb93240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:44.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.803+0000 7f133ce00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55806eb90e50 0x55806eb93240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:50168/0 (socket says 172.21.15.43:50168) 2024-09-19T05:06:44.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.803+0000 7f133ce00640 1 -- 172.21.15.43:0/453840683 learned_addr learned my addr 172.21.15.43:0/453840683 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:44.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.803+0000 7f133e200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55806eb85c70 0x55806eb86050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46964/0 (socket says 172.21.15.43:46964) 2024-09-19T05:06:44.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.804+0000 7f133e200640 1 -- 172.21.15.43:0/453840683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55806eb90e50 msgr2=0x55806eb93240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:44.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.804+0000 7f133e200640 1 --2- 172.21.15.43:0/453840683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55806eb90e50 0x55806eb93240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.804+0000 7f133e200640 1 -- 172.21.15.43:0/453840683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55806eb86590 msgr2=0x55806eb90910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T05:06:44.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.804+0000 7f133e200640 1 --2- 172.21.15.43:0/453840683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55806eb86590 0x55806eb90910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.804+0000 7f133e200640 1 -- 172.21.15.43:0/453840683 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55806eab2ae0 con 0x55806eb85c70 2024-09-19T05:06:44.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.804+0000 7f133ce00640 1 --2- 172.21.15.43:0/453840683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55806eb90e50 0x55806eb93240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:44.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.804+0000 7f133e200640 1 --2- 172.21.15.43:0/453840683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55806eb85c70 0x55806eb86050 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1330005fd0 tx=0x7f1330011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d63ab4b4777405e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:44.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.804+0000 7f132aa00640 1 -- 172.21.15.43:0/453840683 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1330004d20 con 0x55806eb85c70 2024-09-19T05:06:44.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.804+0000 7f132aa00640 1 -- 172.21.15.43:0/453840683 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f133000f9a0 con 0x55806eb85c70 2024-09-19T05:06:44.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.804+0000 7f132aa00640 1 -- 172.21.15.43:0/453840683 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1330004550 con 0x55806eb85c70 2024-09-19T05:06:44.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.805+0000 7f1336000640 1 -- 172.21.15.43:0/854142618 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f132c0305e0 con 0x7f1338104010 2024-09-19T05:06:44.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.805+0000 7f1342df4740 1 -- 172.21.15.43:0/453840683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55806eb85c70 msgr2=0x55806eb86050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:44.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.805+0000 7f1342df4740 1 --2- 172.21.15.43:0/453840683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55806eb85c70 0x55806eb86050 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1330005fd0 tx=0x7f1330011bf0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.805+0000 7f1342df4740 1 -- 172.21.15.43:0/453840683 shutdown_connections 2024-09-19T05:06:44.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.805+0000 7f1342df4740 1 --2- 172.21.15.43:0/453840683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55806eb90e50 0x55806eb93240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.806+0000 7f1342df4740 1 --2- 172.21.15.43:0/453840683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55806eb86590 0x55806eb90910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.806+0000 7f1342df4740 1 --2- 172.21.15.43:0/453840683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55806eb85c70 0x55806eb86050 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.806+0000 7f1342df4740 1 -- 172.21.15.43:0/453840683 >> 172.21.15.43:0/453840683 conn(0x55806eb82200 msgr2=0x55806eb825e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:44.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.806+0000 7f1342df4740 1 -- 172.21.15.43:0/453840683 shutdown_connections 2024-09-19T05:06:44.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.806+0000 7f1342df4740 1 -- 172.21.15.43:0/453840683 wait complete. 2024-09-19T05:06:44.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.807+0000 7f1342df4740 1 -- start start 2024-09-19T05:06:44.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.807+0000 7f1336000640 1 --2- 172.21.15.43:0/854142618 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f13200a32d0 0x7f13200a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:44.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.807+0000 7f1342df4740 1 Processor -- start 2024-09-19T05:06:44.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.807+0000 7f1336000640 1 -- 172.21.15.43:0/854142618 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f132c0ef9c0 con 0x7f1338104010 2024-09-19T05:06:44.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.807+0000 7f133e200640 1 --2- 172.21.15.43:0/854142618 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f13200a32d0 0x7f13200a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:44.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.808+0000 7f1342df4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55806eb85c70 0x55806eb8f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:44.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.808+0000 7f1342df4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55806eb86590 0x55806eb8faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:44.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.808+0000 7f1342df4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55806eb90e50 0x55806eb8ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:44.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.808+0000 7f1342df4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55806eb93a70 con 0x55806eb86590 2024-09-19T05:06:44.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.808+0000 7f1342df4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55806eaa70f0 con 0x55806eb85c70 2024-09-19T05:06:44.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.808+0000 7f1342df4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55806eb93bf0 con 0x55806eb90e50 2024-09-19T05:06:44.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.808+0000 7f1342df4740 10 client.0 _ll_register_callbacks cb 0x55806ea8ebd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:44.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.808+0000 7f133d800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55806eb85c70 0x55806eb8f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:44.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.808+0000 7f133ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55806eb86590 0x55806eb8faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:44.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.808+0000 7f133d800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55806eb85c70 0x55806eb8f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:45882/0 (socket says 172.21.15.43:45882) 2024-09-19T05:06:44.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.808+0000 7f133d800640 1 -- 172.21.15.43:0/4133808163 learned_addr learned my addr 172.21.15.43:0/4133808163 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:44.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.808+0000 7f133e200640 1 --2- 172.21.15.43:0/854142618 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f13200a32d0 0x7f13200a5790 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f13300115a0 tx=0x7f1330009fd0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:44.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.808+0000 7f133e200640 1 --2- 172.21.15.43:0/4133808163 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55806eb90e50 0x55806eb8ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:44.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.808+0000 7f133d800640 1 -- 172.21.15.43:0/4133808163 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55806eb90e50 msgr2=0x55806eb8ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:44.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.808+0000 7f133d800640 1 --2- 172.21.15.43:0/4133808163 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55806eb90e50 0x55806eb8ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.808+0000 7f133d800640 1 -- 172.21.15.43:0/4133808163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55806eb86590 msgr2=0x55806eb8faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:44.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.808+0000 7f133d800640 1 --2- 172.21.15.43:0/4133808163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55806eb86590 0x55806eb8faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.808+0000 7f133d800640 1 -- 172.21.15.43:0/4133808163 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55806e85d3c0 con 0x55806eb85c70 2024-09-19T05:06:44.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.809+0000 7f133d800640 1 --2- 172.21.15.43:0/4133808163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55806eb85c70 0x55806eb8f560 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f132c02a930 tx=0x7f132c0bb920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16ab6b9dc59f4e94 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:44.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.809+0000 7f131f400640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:06:44.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.809+0000 7f131f400640 1 -- 172.21.15.43:0/4133808163 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132c0c4740 con 0x55806eb85c70 2024-09-19T05:06:44.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.809+0000 7f131f400640 1 -- 172.21.15.43:0/4133808163 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f132c003b50 con 0x55806eb85c70 2024-09-19T05:06:44.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.809+0000 7f1342df4740 1 -- 172.21.15.43:0/4133808163 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f133000f9a0 con 0x55806eb85c70 2024-09-19T05:06:44.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.809+0000 7f131f400640 1 -- 172.21.15.43:0/4133808163 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f132c0bbd60 con 0x55806eb85c70 2024-09-19T05:06:44.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.809+0000 7f1342df4740 10 client.25682 fetch_fsmap learned FSMap version 29 2024-09-19T05:06:44.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.809+0000 7f1342df4740 1 -- 172.21.15.43:0/4133808163 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55806eb9fa50 con 0x55806eb85c70 2024-09-19T05:06:44.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.810+0000 7f131f400640 1 -- 172.21.15.43:0/4133808163 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f133000f9a0 con 0x55806eb85c70 2024-09-19T05:06:44.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.810+0000 7f131f400640 1 -- 172.21.15.43:0/4133808163 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f132c03a070 con 0x55806eb85c70 2024-09-19T05:06:44.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.810+0000 7f1342df4740 10 client.25682 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:06:44.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.810+0000 7f1342df4740 10 client.25682 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-19T05:06:44.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.810+0000 7f1342df4740 20 client.25682 populate_metadata read hostname 'smithi043' 2024-09-19T05:06:44.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.810+0000 7f1342df4740 1 --2- 172.21.15.43:0/4133808163 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55806eba0df0 0x55806eba3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:44.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.810+0000 7f1342df4740 4 client.25682 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:06:44.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.810+0000 7f1342df4740 1 -- 172.21.15.43:0/4133808163 --> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55806eaf9950 con 0x55806eba0df0 2024-09-19T05:06:44.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.810+0000 7f133ce00640 1 --2- 172.21.15.43:0/4133808163 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55806eba0df0 0x55806eba3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:44.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.811+0000 7f133ce00640 1 --2- 172.21.15.43:0/4133808163 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55806eba0df0 0x55806eba3220 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5ea2353e5c23c73c server_cookie=7bb5430603b338df in_seq=0 out_seq=0 2024-09-19T05:06:44.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.811+0000 7f131f400640 10 client.25682 ms_handle_connect on v2:172.21.15.43:6838/3404555030 2024-09-19T05:06:44.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.811+0000 7f131f400640 1 -- 172.21.15.43:0/4133808163 <== mds.1 v2:172.21.15.43:6838/3404555030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55806eaf9950 con 0x55806eba0df0 2024-09-19T05:06:44.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.811+0000 7f131f400640 10 client.25682 handle_command_reply: tid=0 2024-09-19T05:06:44.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.812+0000 7f1342df4740 1 client.25682 shutdown 2024-09-19T05:06:44.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.812+0000 7f1342df4740 1 -- 172.21.15.43:0/4133808163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55806eb85c70 msgr2=0x55806eb8f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:44.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.812+0000 7f1342df4740 1 --2- 172.21.15.43:0/4133808163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55806eb85c70 0x55806eb8f560 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f132c02a930 tx=0x7f132c0bb920 comp rx=0 tx=0).stop 2024-09-19T05:06:44.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.812+0000 7f1342df4740 1 -- 172.21.15.43:0/4133808163 shutdown_connections 2024-09-19T05:06:44.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.812+0000 7f1342df4740 1 --2- 172.21.15.43:0/4133808163 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55806eba0df0 0x55806eba3220 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.812+0000 7f1342df4740 1 --2- 172.21.15.43:0/4133808163 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55806eb90e50 0x55806eb8ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.812+0000 7f1342df4740 1 --2- 172.21.15.43:0/4133808163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55806eb86590 0x55806eb8faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.812+0000 7f131f400640 0 client.25682 ms_handle_reset on v2:172.21.15.43:6838/3404555030 2024-09-19T05:06:44.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.812+0000 7f1342df4740 1 --2- 172.21.15.43:0/4133808163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55806eb85c70 0x55806eb8f560 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.812+0000 7f1342df4740 1 -- 172.21.15.43:0/4133808163 >> 172.21.15.43:0/4133808163 conn(0x55806eb82200 msgr2=0x55806eb87f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:44.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.812+0000 7f1342df4740 1 -- 172.21.15.43:0/4133808163 shutdown_connections 2024-09-19T05:06:44.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.812+0000 7f1342df4740 1 -- 172.21.15.43:0/4133808163 wait complete. 2024-09-19T05:06:44.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.812+0000 7f1342df4740 20 client.25682 trim_cache size 0 max 16384 2024-09-19T05:06:44.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.826+0000 7f1342df4740 1 Processor -- start 2024-09-19T05:06:44.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.826+0000 7f1342df4740 1 -- start start 2024-09-19T05:06:44.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.827+0000 7f1342df4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55806eb85c70 0x55806eb886d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:44.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.827+0000 7f1342df4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55806eb86590 0x55806eb8e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:44.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.827+0000 7f1342df4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55806eb90e50 0x55806eba8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:44.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.827+0000 7f1342df4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f13381afa20 con 0x55806eb85c70 2024-09-19T05:06:44.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.827+0000 7f1342df4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f13381b1560 con 0x55806eb90e50 2024-09-19T05:06:44.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.827+0000 7f1342df4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f13381b00f0 con 0x55806eb86590 2024-09-19T05:06:44.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.827+0000 7f133d800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55806eb85c70 0x55806eb886d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:44.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.827+0000 7f133d800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55806eb85c70 0x55806eb886d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46990/0 (socket says 172.21.15.43:46990) 2024-09-19T05:06:44.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.827+0000 7f133d800640 1 -- 172.21.15.43:0/1303170777 learned_addr learned my addr 172.21.15.43:0/1303170777 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:44.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.827+0000 7f133ce00640 1 --2- 172.21.15.43:0/1303170777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55806eb86590 0x55806eb8e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:44.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.827+0000 7f133e200640 1 --2- 172.21.15.43:0/1303170777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55806eb90e50 0x55806eba8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:44.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.827+0000 7f133ce00640 1 -- 172.21.15.43:0/1303170777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55806eb90e50 msgr2=0x55806eba8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:44.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.827+0000 7f133ce00640 1 --2- 172.21.15.43:0/1303170777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55806eb90e50 0x55806eba8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.827+0000 7f133ce00640 1 -- 172.21.15.43:0/1303170777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55806eb85c70 msgr2=0x55806eb886d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:44.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.827+0000 7f133ce00640 1 --2- 172.21.15.43:0/1303170777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55806eb85c70 0x55806eb886d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.827+0000 7f133ce00640 1 -- 172.21.15.43:0/1303170777 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55806eb82030 con 0x55806eb86590 2024-09-19T05:06:44.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.828+0000 7f133d800640 1 --2- 172.21.15.43:0/1303170777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55806eb85c70 0x55806eb886d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:44.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.828+0000 7f133ce00640 1 --2- 172.21.15.43:0/1303170777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55806eb86590 0x55806eb8e5d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x55806eba3a60 tx=0x7f13240046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=324e14637f018629 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:44.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.828+0000 7f132a000640 1 -- 172.21.15.43:0/1303170777 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1324004220 con 0x55806eb86590 2024-09-19T05:06:44.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.828+0000 7f132a000640 1 -- 172.21.15.43:0/1303170777 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13240043c0 con 0x55806eb86590 2024-09-19T05:06:44.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.828+0000 7f132a000640 1 -- 172.21.15.43:0/1303170777 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1324004a90 con 0x55806eb86590 2024-09-19T05:06:44.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.829+0000 7f1342df4740 1 -- 172.21.15.43:0/1303170777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55806eb86590 msgr2=0x55806eb8e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:44.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.829+0000 7f1342df4740 1 --2- 172.21.15.43:0/1303170777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55806eb86590 0x55806eb8e5d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x55806eba3a60 tx=0x7f13240046c0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.829+0000 7f1342df4740 1 -- 172.21.15.43:0/1303170777 shutdown_connections 2024-09-19T05:06:44.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.829+0000 7f1342df4740 1 --2- 172.21.15.43:0/1303170777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55806eb90e50 0x55806eba8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.829+0000 7f1342df4740 1 --2- 172.21.15.43:0/1303170777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55806eb86590 0x55806eb8e5d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.829+0000 7f1342df4740 1 --2- 172.21.15.43:0/1303170777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55806eb85c70 0x55806eb886d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.829+0000 7f1342df4740 1 -- 172.21.15.43:0/1303170777 >> 172.21.15.43:0/1303170777 conn(0x55806eb82200 msgr2=0x55806eb8fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:44.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.829+0000 7f1342df4740 1 -- 172.21.15.43:0/1303170777 shutdown_connections 2024-09-19T05:06:44.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.829+0000 7f1342df4740 1 -- 172.21.15.43:0/1303170777 wait complete. 2024-09-19T05:06:44.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.829+0000 7f1342df4740 1 -- start start 2024-09-19T05:06:44.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.830+0000 7f1342df4740 1 Processor -- start 2024-09-19T05:06:44.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.830+0000 7f1342df4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55806eb85c70 0x55806eb99750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:44.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.830+0000 7f1342df4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55806eb86590 0x55806eb8abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:44.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.830+0000 7f1342df4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55806eb90e50 0x55806eb8b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:44.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.830+0000 7f1342df4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f13381b0e90 con 0x55806eb85c70 2024-09-19T05:06:44.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.830+0000 7f1342df4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f13381b07c0 con 0x55806eb86590 2024-09-19T05:06:44.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.830+0000 7f1342df4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f13381ada00 con 0x55806eb90e50 2024-09-19T05:06:44.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.830+0000 7f1342df4740 10 client.0 _ll_register_callbacks cb 0x7f132c009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:44.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.830+0000 7f133d800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55806eb85c70 0x55806eb99750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:44.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.830+0000 7f133e200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55806eb90e50 0x55806eb8b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:44.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.830+0000 7f133d800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55806eb85c70 0x55806eb99750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47002/0 (socket says 172.21.15.43:47002) 2024-09-19T05:06:44.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.830+0000 7f133d800640 1 -- 172.21.15.43:0/588804756 learned_addr learned my addr 172.21.15.43:0/588804756 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:44.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.831+0000 7f133ce00640 1 --2- 172.21.15.43:0/588804756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55806eb86590 0x55806eb8abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:44.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.831+0000 7f133d800640 1 -- 172.21.15.43:0/588804756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55806eb90e50 msgr2=0x55806eb8b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:44.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.831+0000 7f133d800640 1 --2- 172.21.15.43:0/588804756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55806eb90e50 0x55806eb8b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.831+0000 7f133d800640 1 -- 172.21.15.43:0/588804756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55806eb86590 msgr2=0x55806eb8abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:44.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.831+0000 7f133d800640 1 --2- 172.21.15.43:0/588804756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55806eb86590 0x55806eb8abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.831+0000 7f133d800640 1 -- 172.21.15.43:0/588804756 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55806eb7d770 con 0x55806eb85c70 2024-09-19T05:06:44.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.831+0000 7f133d800640 1 --2- 172.21.15.43:0/588804756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55806eb85c70 0x55806eb99750 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f132c0bbcf0 tx=0x7f132c030550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d790214710c7de7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:44.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.831+0000 7f131f400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:06:44.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.831+0000 7f131f400640 1 -- 172.21.15.43:0/588804756 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132c0c49d0 con 0x55806eb85c70 2024-09-19T05:06:44.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.831+0000 7f131f400640 1 -- 172.21.15.43:0/588804756 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f132c0c4540 con 0x55806eb85c70 2024-09-19T05:06:44.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.831+0000 7f1342df4740 1 -- 172.21.15.43:0/588804756 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13240043c0 con 0x55806eb85c70 2024-09-19T05:06:44.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.832+0000 7f131f400640 1 -- 172.21.15.43:0/588804756 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f132c0c0110 con 0x55806eb85c70 2024-09-19T05:06:44.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.832+0000 7f1342df4740 10 client.15687 fetch_fsmap learned FSMap version 29 2024-09-19T05:06:44.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.832+0000 7f1342df4740 1 -- 172.21.15.43:0/588804756 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55806eb7d770 con 0x55806eb85c70 2024-09-19T05:06:44.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.832+0000 7f131f400640 1 -- 172.21.15.43:0/588804756 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13240043c0 con 0x55806eb85c70 2024-09-19T05:06:44.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.832+0000 7f131f400640 1 -- 172.21.15.43:0/588804756 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f132c03a070 con 0x55806eb85c70 2024-09-19T05:06:44.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.832+0000 7f1342df4740 10 client.15687 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:06:44.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.832+0000 7f1342df4740 10 client.15687 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-19T05:06:44.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.832+0000 7f1342df4740 20 client.15687 populate_metadata read hostname 'smithi043' 2024-09-19T05:06:44.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.832+0000 7f1342df4740 1 --2- 172.21.15.43:0/588804756 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55806ec0be00 0x55806ec0e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:44.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.832+0000 7f1342df4740 4 client.15687 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:06:44.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.832+0000 7f1342df4740 1 -- 172.21.15.43:0/588804756 --> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55806eaf3e60 con 0x55806ec0be00 2024-09-19T05:06:44.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.833+0000 7f133ce00640 1 --2- 172.21.15.43:0/588804756 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55806ec0be00 0x55806ec0e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:44.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.833+0000 7f133ce00640 1 --2- 172.21.15.43:0/588804756 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55806ec0be00 0x55806ec0e1f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4dbc499bd219716e server_cookie=de3c7ea7b2500903 in_seq=0 out_seq=0 2024-09-19T05:06:44.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.833+0000 7f131f400640 10 client.15687 ms_handle_connect on v2:172.21.15.43:6838/3404555030 2024-09-19T05:06:44.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.833+0000 7f131f400640 1 -- 172.21.15.43:0/588804756 <== mds.1 v2:172.21.15.43:6838/3404555030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55806eaf3e60 con 0x55806ec0be00 2024-09-19T05:06:44.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.833+0000 7f131f400640 10 client.15687 handle_command_reply: tid=0 2024-09-19T05:06:44.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.833+0000 7f1342df4740 1 client.15687 shutdown 2024-09-19T05:06:44.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.834+0000 7f1342df4740 1 -- 172.21.15.43:0/588804756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55806eb85c70 msgr2=0x55806eb99750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:44.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.834+0000 7f1342df4740 1 --2- 172.21.15.43:0/588804756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55806eb85c70 0x55806eb99750 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f132c0bbcf0 tx=0x7f132c030550 comp rx=0 tx=0).stop 2024-09-19T05:06:44.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.834+0000 7f1342df4740 1 -- 172.21.15.43:0/588804756 shutdown_connections 2024-09-19T05:06:44.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.834+0000 7f1342df4740 1 --2- 172.21.15.43:0/588804756 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55806ec0be00 0x55806ec0e1f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.834+0000 7f1342df4740 1 --2- 172.21.15.43:0/588804756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55806eb90e50 0x55806eb8b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.834+0000 7f1342df4740 1 --2- 172.21.15.43:0/588804756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55806eb86590 0x55806eb8abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.834+0000 7f131f400640 0 client.15687 ms_handle_reset on v2:172.21.15.43:6838/3404555030 2024-09-19T05:06:44.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.834+0000 7f1342df4740 1 --2- 172.21.15.43:0/588804756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55806eb85c70 0x55806eb99750 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.834+0000 7f1342df4740 1 -- 172.21.15.43:0/588804756 >> 172.21.15.43:0/588804756 conn(0x55806eb82200 msgr2=0x55806eb80b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:44.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.834+0000 7f1342df4740 1 -- 172.21.15.43:0/588804756 shutdown_connections 2024-09-19T05:06:44.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.834+0000 7f1342df4740 1 -- 172.21.15.43:0/588804756 wait complete. 2024-09-19T05:06:44.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.834+0000 7f1342df4740 20 client.15687 trim_cache size 0 max 16384 2024-09-19T05:06:44.840 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:06:44.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.835+0000 7f1340200640 1 -- 172.21.15.43:0/854142618 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f13200a32d0 msgr2=0x7f13200a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:44.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.835+0000 7f1340200640 1 --2- 172.21.15.43:0/854142618 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f13200a32d0 0x7f13200a5790 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f13300115a0 tx=0x7f1330009fd0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.835+0000 7f1340200640 1 -- 172.21.15.43:0/854142618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1338104010 msgr2=0x7f133819a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:44.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.835+0000 7f1340200640 1 --2- 172.21.15.43:0/854142618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1338104010 0x7f133819a630 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f132c035040 tx=0x7f132c033b60 comp rx=0 tx=0).stop 2024-09-19T05:06:44.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.835+0000 7f1340200640 1 -- 172.21.15.43:0/854142618 shutdown_connections 2024-09-19T05:06:44.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.835+0000 7f1340200640 1 --2- 172.21.15.43:0/854142618 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f13200a32d0 0x7f13200a5790 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.835+0000 7f1340200640 1 --2- 172.21.15.43:0/854142618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1338109140 0x7f133819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.835+0000 7f1340200640 1 --2- 172.21.15.43:0/854142618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13381049c0 0x7f133819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.835+0000 7f1340200640 1 --2- 172.21.15.43:0/854142618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1338104010 0x7f133819a630 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:44.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.835+0000 7f1340200640 1 -- 172.21.15.43:0/854142618 >> 172.21.15.43:0/854142618 conn(0x7f13380ffa50 msgr2=0x7f1338100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:44.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.835+0000 7f1340200640 1 -- 172.21.15.43:0/854142618 shutdown_connections 2024-09-19T05:06:44.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:44.835+0000 7f1340200640 1 -- 172.21.15.43:0/854142618 wait complete. 2024-09-19T05:06:45.626 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T05:06:46.009 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:06:46.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:45 smithi043 ceph-mon[30800]: pgmap v66: 97 pgs: 97 active+clean; 1.0 GiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 147 MiB/s wr, 50 op/s 2024-09-19T05:06:46.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:45 smithi151 ceph-mon[35425]: pgmap v66: 97 pgs: 97 active+clean; 1.0 GiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 147 MiB/s wr, 50 op/s 2024-09-19T05:06:46.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:45 smithi096 ceph-mon[35085]: pgmap v66: 97 pgs: 97 active+clean; 1.0 GiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 147 MiB/s wr, 50 op/s 2024-09-19T05:06:46.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.580+0000 7fb60fe00640 1 -- 172.21.15.43:0/3663922259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6101034b0 msgr2=0x7fb61010b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:46.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.580+0000 7fb60fe00640 1 --2- 172.21.15.43:0/3663922259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6101034b0 0x7fb61010b9a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8009900 tx=0x7fb5f802a480 comp rx=0 tx=0).stop 2024-09-19T05:06:46.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.580+0000 7fb60fe00640 1 -- 172.21.15.43:0/3663922259 shutdown_connections 2024-09-19T05:06:46.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.580+0000 7fb60fe00640 1 --2- 172.21.15.43:0/3663922259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb61010bfc0 0x7fb61010e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:46.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.580+0000 7fb60fe00640 1 --2- 172.21.15.43:0/3663922259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6101034b0 0x7fb61010b9a0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:46.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.580+0000 7fb60fe00640 1 --2- 172.21.15.43:0/3663922259 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb610102b90 0x7fb610102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:46.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.580+0000 7fb60fe00640 1 -- 172.21.15.43:0/3663922259 >> 172.21.15.43:0/3663922259 conn(0x7fb6100fc610 msgr2=0x7fb6100fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:46.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.580+0000 7fb60fe00640 1 -- 172.21.15.43:0/3663922259 shutdown_connections 2024-09-19T05:06:46.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.580+0000 7fb60fe00640 1 -- 172.21.15.43:0/3663922259 wait complete. 2024-09-19T05:06:46.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.581+0000 7fb60fe00640 1 Processor -- start 2024-09-19T05:06:46.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.581+0000 7fb60fe00640 1 -- start start 2024-09-19T05:06:46.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.582+0000 7fb60fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb610102b90 0x7fb61019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:46.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.582+0000 7fb60fe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6101034b0 0x7fb61019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:46.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.582+0000 7fb60fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb61010bfc0 0x7fb61019ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:46.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.582+0000 7fb60fe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb610111e10 con 0x7fb61010bfc0 2024-09-19T05:06:46.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.582+0000 7fb60fe00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb610111c90 con 0x7fb6101034b0 2024-09-19T05:06:46.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.582+0000 7fb60fe00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb610111f90 con 0x7fb610102b90 2024-09-19T05:06:46.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.582+0000 7fb60ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb610102b90 0x7fb61019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:46.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.582+0000 7fb60f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb61010bfc0 0x7fb61019ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:46.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.582+0000 7fb607e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6101034b0 0x7fb61019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:46.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.582+0000 7fb60ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb610102b90 0x7fb61019a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:39010/0 (socket says 172.21.15.43:39010) 2024-09-19T05:06:46.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.582+0000 7fb60ea00640 1 -- 172.21.15.43:0/973230483 learned_addr learned my addr 172.21.15.43:0/973230483 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:46.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.583+0000 7fb60f400640 1 -- 172.21.15.43:0/973230483 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb610102b90 msgr2=0x7fb61019a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:46.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.583+0000 7fb60f400640 1 --2- 172.21.15.43:0/973230483 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb610102b90 0x7fb61019a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:46.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.583+0000 7fb60f400640 1 -- 172.21.15.43:0/973230483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6101034b0 msgr2=0x7fb61019aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:46.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.583+0000 7fb60f400640 1 --2- 172.21.15.43:0/973230483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6101034b0 0x7fb61019aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:46.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.583+0000 7fb60f400640 1 -- 172.21.15.43:0/973230483 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb61019f4f0 con 0x7fb61010bfc0 2024-09-19T05:06:46.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.583+0000 7fb60f400640 1 --2- 172.21.15.43:0/973230483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb61010bfc0 0x7fb61019ee10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb608009fd0 tx=0x7fb60800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:46.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.583+0000 7fb607400640 1 -- 172.21.15.43:0/973230483 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb60800ce20 con 0x7fb61010bfc0 2024-09-19T05:06:46.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.583+0000 7fb607400640 1 -- 172.21.15.43:0/973230483 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb608005020 con 0x7fb61010bfc0 2024-09-19T05:06:46.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.583+0000 7fb60fe00640 1 -- 172.21.15.43:0/973230483 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb61019f7e0 con 0x7fb61010bfc0 2024-09-19T05:06:46.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.583+0000 7fb607400640 1 -- 172.21.15.43:0/973230483 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb608010720 con 0x7fb61010bfc0 2024-09-19T05:06:46.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.583+0000 7fb60fe00640 1 -- 172.21.15.43:0/973230483 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6101a70c0 con 0x7fb61010bfc0 2024-09-19T05:06:46.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.588+0000 7fb6181c1740 1 Processor -- start 2024-09-19T05:06:46.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.588+0000 7fb607400640 1 -- 172.21.15.43:0/973230483 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb608005540 con 0x7fb61010bfc0 2024-09-19T05:06:46.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.588+0000 7fb6181c1740 1 -- start start 2024-09-19T05:06:46.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.588+0000 7fb6181c1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0cfc2ce40 0x55d0cfc2d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:46.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.588+0000 7fb6181c1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0cfc2d760 0x55d0cfc37ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:46.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.589+0000 7fb6181c1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0cfc38020 0x55d0cfc3a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:46.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.589+0000 7fb6181c1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d0cfb933e0 con 0x55d0cfc38020 2024-09-19T05:06:46.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.589+0000 7fb6181c1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d0cfb3ad80 con 0x55d0cfc2ce40 2024-09-19T05:06:46.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.589+0000 7fb6181c1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d0cfb8f3a0 con 0x55d0cfc2d760 2024-09-19T05:06:46.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.589+0000 7fb607e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0cfc38020 0x55d0cfc3a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:46.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.589+0000 7fb60ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0cfc2d760 0x55d0cfc37ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:46.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.589+0000 7fb607e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0cfc38020 0x55d0cfc3a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51748/0 (socket says 172.21.15.43:51748) 2024-09-19T05:06:46.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.589+0000 7fb607e00640 1 -- 172.21.15.43:0/3174924703 learned_addr learned my addr 172.21.15.43:0/3174924703 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:46.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.589+0000 7fb607e00640 1 --2- 172.21.15.43:0/3174924703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0cfc2ce40 0x55d0cfc2d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:46.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.589+0000 7fb607400640 1 --2- 172.21.15.43:0/973230483 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb5f40a33b0 0x7fb5f40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:46.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.589+0000 7fb607e00640 1 -- 172.21.15.43:0/3174924703 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0cfc2d760 msgr2=0x55d0cfc37ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:46.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.589+0000 7fb607e00640 1 --2- 172.21.15.43:0/3174924703 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0cfc2d760 0x55d0cfc37ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:46.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.589+0000 7fb607e00640 1 -- 172.21.15.43:0/3174924703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0cfc2ce40 msgr2=0x55d0cfc2d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:46.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.589+0000 7fb607e00640 1 --2- 172.21.15.43:0/3174924703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0cfc2ce40 0x55d0cfc2d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:46.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.589+0000 7fb607e00640 1 -- 172.21.15.43:0/3174924703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0cfb59ae0 con 0x55d0cfc38020 2024-09-19T05:06:46.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.590+0000 7fb607400640 1 -- 172.21.15.43:0/973230483 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fb6080d03f0 con 0x7fb61010bfc0 2024-09-19T05:06:46.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.590+0000 7fb607e00640 1 --2- 172.21.15.43:0/3174924703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0cfc38020 0x55d0cfc3a410 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb5f80096c0 tx=0x7fb5f8005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3331cda3c8a11538 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:46.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.590+0000 7fb60f400640 1 --2- 172.21.15.43:0/973230483 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb5f40a33b0 0x7fb5f40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:46.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.590+0000 7fb5ff400640 1 -- 172.21.15.43:0/3174924703 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f8035030 con 0x55d0cfc38020 2024-09-19T05:06:46.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.590+0000 7fb5ff400640 1 -- 172.21.15.43:0/3174924703 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5f80046c0 con 0x55d0cfc38020 2024-09-19T05:06:46.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.590+0000 7fb5ff400640 1 -- 172.21.15.43:0/3174924703 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f8031870 con 0x55d0cfc38020 2024-09-19T05:06:46.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.591+0000 7fb6181c1740 1 -- 172.21.15.43:0/3174924703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0cfc38020 msgr2=0x55d0cfc3a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:46.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.591+0000 7fb6181c1740 1 --2- 172.21.15.43:0/3174924703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0cfc38020 0x55d0cfc3a410 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb5f80096c0 tx=0x7fb5f8005b50 comp rx=0 tx=0).stop 2024-09-19T05:06:46.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.591+0000 7fb60f400640 1 --2- 172.21.15.43:0/973230483 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb5f40a33b0 0x7fb5f40a5870 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb6080a4dc0 tx=0x7fb608005f70 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:46.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.591+0000 7fb6181c1740 1 -- 172.21.15.43:0/3174924703 shutdown_connections 2024-09-19T05:06:46.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.591+0000 7fb6181c1740 1 --2- 172.21.15.43:0/3174924703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0cfc38020 0x55d0cfc3a410 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:46.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.591+0000 7fb6181c1740 1 --2- 172.21.15.43:0/3174924703 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0cfc2d760 0x55d0cfc37ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:46.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.591+0000 7fb6181c1740 1 --2- 172.21.15.43:0/3174924703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0cfc2ce40 0x55d0cfc2d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:46.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.591+0000 7fb6181c1740 1 -- 172.21.15.43:0/3174924703 >> 172.21.15.43:0/3174924703 conn(0x55d0cfc293d0 msgr2=0x55d0cfc297b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:46.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.591+0000 7fb6181c1740 1 -- 172.21.15.43:0/3174924703 shutdown_connections 2024-09-19T05:06:46.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.591+0000 7fb6181c1740 1 -- 172.21.15.43:0/3174924703 wait complete. 2024-09-19T05:06:46.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.592+0000 7fb6181c1740 1 -- start start 2024-09-19T05:06:46.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.592+0000 7fb6181c1740 1 Processor -- start 2024-09-19T05:06:46.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.592+0000 7fb6181c1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0cfc2ce40 0x55d0cfc34ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:46.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.592+0000 7fb6181c1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0cfc2d760 0x55d0cfc35420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:46.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.592+0000 7fb6181c1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0cfc38020 0x55d0cfc35960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:46.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.592+0000 7fb6181c1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d0cfc3abf0 con 0x55d0cfc2d760 2024-09-19T05:06:46.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.592+0000 7fb6181c1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d0cfb4e0f0 con 0x55d0cfc38020 2024-09-19T05:06:46.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.592+0000 7fb6181c1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d0cfc3ad70 con 0x55d0cfc2ce40 2024-09-19T05:06:46.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.592+0000 7fb6181c1740 10 client.0 _ll_register_callbacks cb 0x55d0cfb35bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:46.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.593+0000 7fb60ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0cfc2d760 0x55d0cfc35420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:46.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.593+0000 7fb60ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0cfc2d760 0x55d0cfc35420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51756/0 (socket says 172.21.15.43:51756) 2024-09-19T05:06:46.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.593+0000 7fb60ea00640 1 -- 172.21.15.43:0/2025994874 learned_addr learned my addr 172.21.15.43:0/2025994874 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:46.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.593+0000 7fb607e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0cfc2ce40 0x55d0cfc34ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:46.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.593+0000 7fb607e00640 1 --2- 172.21.15.43:0/2025994874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0cfc38020 0x55d0cfc35960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:46.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.593+0000 7fb60ea00640 1 -- 172.21.15.43:0/2025994874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0cfc2ce40 msgr2=0x55d0cfc34ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:46.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.593+0000 7fb60ea00640 1 --2- 172.21.15.43:0/2025994874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0cfc2ce40 0x55d0cfc34ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.593+0000 7fb60ea00640 1 -- 172.21.15.43:0/2025994874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0cfc38020 msgr2=0x55d0cfc35960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.593+0000 7fb60ea00640 1 --2- 172.21.15.43:0/2025994874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0cfc38020 0x55d0cfc35960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.593+0000 7fb60ea00640 1 -- 172.21.15.43:0/2025994874 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0cf9043c0 con 0x55d0cfc2d760 2024-09-19T05:06:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.593+0000 7fb607e00640 1 --2- 172.21.15.43:0/2025994874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0cfc38020 0x55d0cfc35960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.594+0000 7fb60ea00640 1 --2- 172.21.15.43:0/2025994874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0cfc2d760 0x55d0cfc35420 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb60000bc50 tx=0x7fb6000039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856440c2dc209417 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.594+0000 7fb5f3e00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:06:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.594+0000 7fb5f3e00640 1 -- 172.21.15.43:0/2025994874 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb600011650 con 0x55d0cfc2d760 2024-09-19T05:06:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.594+0000 7fb6181c1740 1 -- 172.21.15.43:0/2025994874 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5f80046c0 con 0x55d0cfc2d760 2024-09-19T05:06:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.594+0000 7fb5f3e00640 1 -- 172.21.15.43:0/2025994874 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fb6000117f0 con 0x55d0cfc2d760 2024-09-19T05:06:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.594+0000 7fb5f3e00640 1 -- 172.21.15.43:0/2025994874 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb6000119f0 con 0x55d0cfc2d760 2024-09-19T05:06:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.594+0000 7fb6181c1740 10 client.15699 fetch_fsmap learned FSMap version 29 2024-09-19T05:06:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.594+0000 7fb6181c1740 1 -- 172.21.15.43:0/2025994874 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d0cfc46c60 con 0x55d0cfc2d760 2024-09-19T05:06:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.595+0000 7fb5f3e00640 1 -- 172.21.15.43:0/2025994874 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5f80046c0 con 0x55d0cfc2d760 2024-09-19T05:06:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.595+0000 7fb5f3e00640 1 -- 172.21.15.43:0/2025994874 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb60001bc20 con 0x55d0cfc2d760 2024-09-19T05:06:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.595+0000 7fb6181c1740 10 client.15699 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:06:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.595+0000 7fb6181c1740 10 client.15699 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:06:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.595+0000 7fb6181c1740 20 client.15699 populate_metadata read hostname 'smithi043' 2024-09-19T05:06:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.595+0000 7fb6181c1740 1 --2- 172.21.15.43:0/2025994874 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55d0cfc48170 0x55d0cfc4a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.595+0000 7fb6181c1740 4 client.15699 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:06:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.595+0000 7fb6181c1740 1 -- 172.21.15.43:0/2025994874 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0cfba0950 con 0x55d0cfc48170 2024-09-19T05:06:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.595+0000 7fb607e00640 1 --2- 172.21.15.43:0/2025994874 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55d0cfc48170 0x55d0cfc4a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.596+0000 7fb607e00640 1 --2- 172.21.15.43:0/2025994874 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55d0cfc48170 0x55d0cfc4a5b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f12d881c03d3759e server_cookie=8f9dbdd03bbcacfb in_seq=0 out_seq=0 2024-09-19T05:06:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.596+0000 7fb5f3e00640 10 client.15699 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:06:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.597+0000 7fb5f3e00640 1 -- 172.21.15.43:0/2025994874 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d0cfba0950 con 0x55d0cfc48170 2024-09-19T05:06:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.597+0000 7fb5f3e00640 10 client.15699 handle_command_reply: tid=0 2024-09-19T05:06:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.597+0000 7fb6181c1740 1 client.15699 shutdown 2024-09-19T05:06:46.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.597+0000 7fb6181c1740 1 -- 172.21.15.43:0/2025994874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0cfc2d760 msgr2=0x55d0cfc35420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:46.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.597+0000 7fb6181c1740 1 --2- 172.21.15.43:0/2025994874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0cfc2d760 0x55d0cfc35420 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb60000bc50 tx=0x7fb6000039c0 comp rx=0 tx=0).stop 2024-09-19T05:06:46.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.597+0000 7fb6181c1740 1 -- 172.21.15.43:0/2025994874 shutdown_connections 2024-09-19T05:06:46.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.597+0000 7fb6181c1740 1 --2- 172.21.15.43:0/2025994874 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55d0cfc48170 0x55d0cfc4a5b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:46.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.597+0000 7fb6181c1740 1 --2- 172.21.15.43:0/2025994874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0cfc38020 0x55d0cfc35960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:46.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.597+0000 7fb6181c1740 1 --2- 172.21.15.43:0/2025994874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0cfc2d760 0x55d0cfc35420 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:46.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.597+0000 7fb6181c1740 1 --2- 172.21.15.43:0/2025994874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0cfc2ce40 0x55d0cfc34ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:46.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.597+0000 7fb6181c1740 1 -- 172.21.15.43:0/2025994874 >> 172.21.15.43:0/2025994874 conn(0x55d0cfc293d0 msgr2=0x55d0cfc2f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:46.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.597+0000 7fb5f3e00640 0 client.15699 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:06:46.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.597+0000 7fb6181c1740 1 -- 172.21.15.43:0/2025994874 shutdown_connections 2024-09-19T05:06:46.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.597+0000 7fb6181c1740 1 -- 172.21.15.43:0/2025994874 wait complete. 2024-09-19T05:06:46.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.597+0000 7fb6181c1740 20 client.15699 trim_cache size 0 max 16384 2024-09-19T05:06:46.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.611+0000 7fb6181c1740 1 Processor -- start 2024-09-19T05:06:46.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.611+0000 7fb6181c1740 1 -- start start 2024-09-19T05:06:46.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.611+0000 7fb6181c1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0cfc2ce40 0x55d0cfc33680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:46.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.611+0000 7fb6181c1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0cfc2d760 0x55d0cfc33bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:46.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.611+0000 7fb6181c1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0cfc38020 0x55d0cfc34100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:46.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.611+0000 7fb6181c1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb6101af960 con 0x55d0cfc38020 2024-09-19T05:06:46.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.611+0000 7fb6181c1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb6101b14a0 con 0x55d0cfc2d760 2024-09-19T05:06:46.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.611+0000 7fb6181c1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb6101b0030 con 0x55d0cfc2ce40 2024-09-19T05:06:46.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.612+0000 7fb607e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0cfc38020 0x55d0cfc34100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:46.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.612+0000 7fb607e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0cfc38020 0x55d0cfc34100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51760/0 (socket says 172.21.15.43:51760) 2024-09-19T05:06:46.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.612+0000 7fb607e00640 1 -- 172.21.15.43:0/2625507514 learned_addr learned my addr 172.21.15.43:0/2625507514 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:46.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.612+0000 7fb60ea00640 1 --2- 172.21.15.43:0/2625507514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0cfc2d760 0x55d0cfc33bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:46.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.612+0000 7fb607e00640 1 --2- 172.21.15.43:0/2625507514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0cfc2ce40 0x55d0cfc33680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:46.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.612+0000 7fb607e00640 1 -- 172.21.15.43:0/2625507514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0cfc2ce40 msgr2=0x55d0cfc33680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:46.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.612+0000 7fb607e00640 1 --2- 172.21.15.43:0/2625507514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0cfc2ce40 0x55d0cfc33680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:46.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.612+0000 7fb607e00640 1 -- 172.21.15.43:0/2625507514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0cfc2d760 msgr2=0x55d0cfc33bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:46.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.612+0000 7fb607e00640 1 --2- 172.21.15.43:0/2625507514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0cfc2d760 0x55d0cfc33bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:46.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.612+0000 7fb607e00640 1 -- 172.21.15.43:0/2625507514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0cfc28c80 con 0x55d0cfc38020 2024-09-19T05:06:46.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.612+0000 7fb60ea00640 1 --2- 172.21.15.43:0/2625507514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0cfc2d760 0x55d0cfc33bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:46.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.613+0000 7fb607e00640 1 --2- 172.21.15.43:0/2625507514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0cfc38020 0x55d0cfc34100 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb5f80096c0 tx=0x7fb5f8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=740d6012e4093dc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:46.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.613+0000 7fb5fea00640 1 -- 172.21.15.43:0/2625507514 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f8035040 con 0x55d0cfc38020 2024-09-19T05:06:46.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.613+0000 7fb5fea00640 1 -- 172.21.15.43:0/2625507514 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5f8032df0 con 0x55d0cfc38020 2024-09-19T05:06:46.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.613+0000 7fb5fea00640 1 -- 172.21.15.43:0/2625507514 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f803dc70 con 0x55d0cfc38020 2024-09-19T05:06:46.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.614+0000 7fb6181c1740 1 -- 172.21.15.43:0/2625507514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0cfc38020 msgr2=0x55d0cfc34100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:46.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.614+0000 7fb6181c1740 1 --2- 172.21.15.43:0/2625507514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0cfc38020 0x55d0cfc34100 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb5f80096c0 tx=0x7fb5f8005fd0 comp rx=0 tx=0).stop 2024-09-19T05:06:46.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.614+0000 7fb6181c1740 1 -- 172.21.15.43:0/2625507514 shutdown_connections 2024-09-19T05:06:46.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.614+0000 7fb6181c1740 1 --2- 172.21.15.43:0/2625507514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0cfc38020 0x55d0cfc34100 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:46.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.614+0000 7fb6181c1740 1 --2- 172.21.15.43:0/2625507514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0cfc2d760 0x55d0cfc33bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:46.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.614+0000 7fb6181c1740 1 --2- 172.21.15.43:0/2625507514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0cfc2ce40 0x55d0cfc33680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:46.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.614+0000 7fb6181c1740 1 -- 172.21.15.43:0/2625507514 >> 172.21.15.43:0/2625507514 conn(0x55d0cfc293d0 msgr2=0x55d0cfc30200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:46.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.614+0000 7fb6181c1740 1 -- 172.21.15.43:0/2625507514 shutdown_connections 2024-09-19T05:06:46.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.614+0000 7fb6181c1740 1 -- 172.21.15.43:0/2625507514 wait complete. 2024-09-19T05:06:46.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.614+0000 7fb6181c1740 1 -- start start 2024-09-19T05:06:46.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.614+0000 7fb6181c1740 1 Processor -- start 2024-09-19T05:06:46.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.615+0000 7fb6181c1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0cfc2ce40 0x55d0cfc32b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:46.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.615+0000 7fb6181c1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0cfc2d760 0x55d0cfc3ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:46.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.615+0000 7fb6181c1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0cfc38020 0x55d0cfc40640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:46.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.615+0000 7fb6181c1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb6101b0dd0 con 0x55d0cfc38020 2024-09-19T05:06:46.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.615+0000 7fb6181c1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb6101b0700 con 0x55d0cfc2d760 2024-09-19T05:06:46.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.615+0000 7fb6181c1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb6101ad940 con 0x55d0cfc2ce40 2024-09-19T05:06:46.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.615+0000 7fb6181c1740 10 client.0 _ll_register_callbacks cb 0x7fb600004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:46.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.615+0000 7fb607e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0cfc2ce40 0x55d0cfc32b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:46.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.615+0000 7fb607e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0cfc2ce40 0x55d0cfc32b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:39056/0 (socket says 172.21.15.43:39056) 2024-09-19T05:06:46.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.615+0000 7fb607e00640 1 -- 172.21.15.43:0/101283156 learned_addr learned my addr 172.21.15.43:0/101283156 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:46.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.615+0000 7fb60ea00640 1 --2- 172.21.15.43:0/101283156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0cfc2d760 0x55d0cfc3ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:46.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.615+0000 7fb607e00640 1 --2- 172.21.15.43:0/101283156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0cfc38020 0x55d0cfc40640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:46.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.616+0000 7fb60ea00640 1 -- 172.21.15.43:0/101283156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0cfc2ce40 msgr2=0x55d0cfc32b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:46.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.616+0000 7fb60ea00640 1 --2- 172.21.15.43:0/101283156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0cfc2ce40 0x55d0cfc32b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:46.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.616+0000 7fb60ea00640 1 -- 172.21.15.43:0/101283156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0cfc38020 msgr2=0x55d0cfc40640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:46.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.616+0000 7fb60ea00640 1 --2- 172.21.15.43:0/101283156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0cfc38020 0x55d0cfc40640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:46.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.616+0000 7fb60ea00640 1 -- 172.21.15.43:0/101283156 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0cfc31150 con 0x55d0cfc2d760 2024-09-19T05:06:46.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.616+0000 7fb607e00640 1 --2- 172.21.15.43:0/101283156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0cfc2ce40 0x55d0cfc32b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:06:46.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.616+0000 7fb60ea00640 1 --2- 172.21.15.43:0/101283156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0cfc2d760 0x55d0cfc3ffc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb600003f40 tx=0x7fb60000b720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebb016e43ca10dcc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:46.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.616+0000 7fb5f3e00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:06:46.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.616+0000 7fb5f3e00640 1 -- 172.21.15.43:0/101283156 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb600028040 con 0x55d0cfc2d760 2024-09-19T05:06:46.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.616+0000 7fb6181c1740 1 -- 172.21.15.43:0/101283156 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5f8032df0 con 0x55d0cfc2d760 2024-09-19T05:06:46.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.616+0000 7fb5f3e00640 1 -- 172.21.15.43:0/101283156 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fb600011b90 con 0x55d0cfc2d760 2024-09-19T05:06:46.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.616+0000 7fb5f3e00640 1 -- 172.21.15.43:0/101283156 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb600011db0 con 0x55d0cfc2d760 2024-09-19T05:06:46.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.616+0000 7fb6181c1740 10 client.25706 fetch_fsmap learned FSMap version 29 2024-09-19T05:06:46.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.616+0000 7fb6181c1740 1 -- 172.21.15.43:0/101283156 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d0cfcb2060 con 0x55d0cfc2d760 2024-09-19T05:06:46.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.617+0000 7fb5f3e00640 1 -- 172.21.15.43:0/101283156 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5f8032df0 con 0x55d0cfc2d760 2024-09-19T05:06:46.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.617+0000 7fb5f3e00640 1 -- 172.21.15.43:0/101283156 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb60001b6d0 con 0x55d0cfc2d760 2024-09-19T05:06:46.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.617+0000 7fb6181c1740 10 client.25706 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:06:46.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.617+0000 7fb6181c1740 10 client.25706 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:06:46.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.617+0000 7fb6181c1740 20 client.25706 populate_metadata read hostname 'smithi043' 2024-09-19T05:06:46.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.617+0000 7fb6181c1740 1 --2- 172.21.15.43:0/101283156 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55d0cfcb2830 0x55d0cfcb4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:46.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.617+0000 7fb6181c1740 4 client.25706 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:06:46.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.617+0000 7fb6181c1740 1 -- 172.21.15.43:0/101283156 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d0cfb9ae60 con 0x55d0cfcb2830 2024-09-19T05:06:46.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.617+0000 7fb607e00640 1 --2- 172.21.15.43:0/101283156 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55d0cfcb2830 0x55d0cfcb4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:46.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.618+0000 7fb607e00640 1 --2- 172.21.15.43:0/101283156 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55d0cfcb2830 0x55d0cfcb4c50 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=506210eaeb027dfb server_cookie=e6f4e614948abe77 in_seq=0 out_seq=0 2024-09-19T05:06:46.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.618+0000 7fb5f3e00640 10 client.25706 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:06:46.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.618+0000 7fb5f3e00640 1 -- 172.21.15.43:0/101283156 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d0cfb9ae60 con 0x55d0cfcb2830 2024-09-19T05:06:46.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.618+0000 7fb5f3e00640 10 client.25706 handle_command_reply: tid=0 2024-09-19T05:06:46.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.618+0000 7fb6181c1740 1 client.25706 shutdown 2024-09-19T05:06:46.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.618+0000 7fb6181c1740 1 -- 172.21.15.43:0/101283156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0cfc2d760 msgr2=0x55d0cfc3ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:46.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.618+0000 7fb6181c1740 1 --2- 172.21.15.43:0/101283156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0cfc2d760 0x55d0cfc3ffc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb600003f40 tx=0x7fb60000b720 comp rx=0 tx=0).stop 2024-09-19T05:06:46.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.618+0000 7fb6181c1740 1 -- 172.21.15.43:0/101283156 shutdown_connections 2024-09-19T05:06:46.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.618+0000 7fb6181c1740 1 --2- 172.21.15.43:0/101283156 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55d0cfcb2830 0x55d0cfcb4c50 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:46.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.618+0000 7fb6181c1740 1 --2- 172.21.15.43:0/101283156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d0cfc38020 0x55d0cfc40640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:46.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.618+0000 7fb6181c1740 1 --2- 172.21.15.43:0/101283156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0cfc2d760 0x55d0cfc3ffc0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:46.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.618+0000 7fb5f3e00640 0 client.25706 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:06:46.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.618+0000 7fb6181c1740 1 --2- 172.21.15.43:0/101283156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0cfc2ce40 0x55d0cfc32b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:46.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.618+0000 7fb6181c1740 1 -- 172.21.15.43:0/101283156 >> 172.21.15.43:0/101283156 conn(0x55d0cfc293d0 msgr2=0x55d0cfc33680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:46.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.618+0000 7fb6181c1740 1 -- 172.21.15.43:0/101283156 shutdown_connections 2024-09-19T05:06:46.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.618+0000 7fb6181c1740 1 -- 172.21.15.43:0/101283156 wait complete. 2024-09-19T05:06:46.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.619+0000 7fb6181c1740 20 client.25706 trim_cache size 0 max 16384 2024-09-19T05:06:46.626 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:06:46.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.619+0000 7fb60fe00640 1 -- 172.21.15.43:0/973230483 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb5f40a33b0 msgr2=0x7fb5f40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:46.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.619+0000 7fb60fe00640 1 --2- 172.21.15.43:0/973230483 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb5f40a33b0 0x7fb5f40a5870 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb6080a4dc0 tx=0x7fb608005f70 comp rx=0 tx=0).stop 2024-09-19T05:06:46.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.619+0000 7fb60fe00640 1 -- 172.21.15.43:0/973230483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb61010bfc0 msgr2=0x7fb61019ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:46.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.619+0000 7fb60fe00640 1 --2- 172.21.15.43:0/973230483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb61010bfc0 0x7fb61019ee10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb608009fd0 tx=0x7fb60800ef10 comp rx=0 tx=0).stop 2024-09-19T05:06:46.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.619+0000 7fb60fe00640 1 -- 172.21.15.43:0/973230483 shutdown_connections 2024-09-19T05:06:46.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.619+0000 7fb60fe00640 1 --2- 172.21.15.43:0/973230483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb61010bfc0 0x7fb61019ee10 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:46.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.619+0000 7fb60fe00640 1 --2- 172.21.15.43:0/973230483 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb5f40a33b0 0x7fb5f40a5870 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:46.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.619+0000 7fb60fe00640 1 --2- 172.21.15.43:0/973230483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6101034b0 0x7fb61019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:46.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.619+0000 7fb60fe00640 1 --2- 172.21.15.43:0/973230483 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb610102b90 0x7fb61019a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:46.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.619+0000 7fb60fe00640 1 -- 172.21.15.43:0/973230483 >> 172.21.15.43:0/973230483 conn(0x7fb6100fc610 msgr2=0x7fb61010d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:46.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.620+0000 7fb60fe00640 1 -- 172.21.15.43:0/973230483 shutdown_connections 2024-09-19T05:06:46.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:46.620+0000 7fb60fe00640 1 -- 172.21.15.43:0/973230483 wait complete. 2024-09-19T05:06:48.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:47 smithi151 ceph-mon[35425]: pgmap v67: 97 pgs: 97 active+clean; 1.0 GiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 MiB/s wr, 29 op/s 2024-09-19T05:06:48.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:47 smithi096 ceph-mon[35085]: pgmap v67: 97 pgs: 97 active+clean; 1.0 GiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 MiB/s wr, 29 op/s 2024-09-19T05:06:48.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:47 smithi043 ceph-mon[30800]: pgmap v67: 97 pgs: 97 active+clean; 1.0 GiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 MiB/s wr, 29 op/s 2024-09-19T05:06:48.476 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T05:06:48.476 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T05:06:48.879 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:06:49.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.494+0000 7f3ba1200640 1 -- 172.21.15.43:0/978988333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b9c103ff0 msgr2=0x7f3b9c1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:49.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.494+0000 7f3ba1200640 1 --2- 172.21.15.43:0/978988333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b9c103ff0 0x7f3b9c1043d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c009900 tx=0x7f3b8c02a410 comp rx=0 tx=0).stop 2024-09-19T05:06:49.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.494+0000 7f3ba1200640 1 -- 172.21.15.43:0/978988333 shutdown_connections 2024-09-19T05:06:49.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.494+0000 7f3ba1200640 1 --2- 172.21.15.43:0/978988333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b9c109120 0x7f3b9c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:49.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.494+0000 7f3ba1200640 1 --2- 172.21.15.43:0/978988333 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b9c1049a0 0x7f3b9c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:49.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.494+0000 7f3ba1200640 1 --2- 172.21.15.43:0/978988333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b9c103ff0 0x7f3b9c1043d0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:49.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.494+0000 7f3ba1200640 1 -- 172.21.15.43:0/978988333 >> 172.21.15.43:0/978988333 conn(0x7f3b9c0ffa90 msgr2=0x7f3b9c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:49.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.494+0000 7f3ba1200640 1 -- 172.21.15.43:0/978988333 shutdown_connections 2024-09-19T05:06:49.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.494+0000 7f3ba1200640 1 -- 172.21.15.43:0/978988333 wait complete. 2024-09-19T05:06:49.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.495+0000 7f3ba1200640 1 Processor -- start 2024-09-19T05:06:49.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.496+0000 7f3ba1200640 1 -- start start 2024-09-19T05:06:49.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.496+0000 7f3ba1200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b9c103ff0 0x7f3b9c19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:49.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.496+0000 7f3ba1200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b9c1049a0 0x7f3b9c19abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:49.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.496+0000 7f3ba1200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b9c109120 0x7f3b9c19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:49.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.496+0000 7f3ba1200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9c114190 con 0x7f3b9c103ff0 2024-09-19T05:06:49.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.496+0000 7f3ba1200640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9c114010 con 0x7f3b9c1049a0 2024-09-19T05:06:49.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.496+0000 7f3ba1200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9c114310 con 0x7f3b9c109120 2024-09-19T05:06:49.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.496+0000 7f3b99e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b9c103ff0 0x7f3b9c19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:49.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.496+0000 7f3b99e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b9c103ff0 0x7f3b9c19a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51802/0 (socket says 172.21.15.43:51802) 2024-09-19T05:06:49.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.496+0000 7f3b99e00640 1 -- 172.21.15.43:0/3156768622 learned_addr learned my addr 172.21.15.43:0/3156768622 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:49.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.496+0000 7f3b99400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b9c1049a0 0x7f3b9c19abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:49.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.497+0000 7f3b9a800640 1 --2- 172.21.15.43:0/3156768622 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b9c109120 0x7f3b9c19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:49.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.497+0000 7f3b99400640 1 -- 172.21.15.43:0/3156768622 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b9c109120 msgr2=0x7f3b9c19ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:49.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.497+0000 7f3b99400640 1 --2- 172.21.15.43:0/3156768622 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b9c109120 0x7f3b9c19ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:49.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.497+0000 7f3b99400640 1 -- 172.21.15.43:0/3156768622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b9c103ff0 msgr2=0x7f3b9c19a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:49.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.497+0000 7f3b99400640 1 --2- 172.21.15.43:0/3156768622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b9c103ff0 0x7f3b9c19a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:49.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.497+0000 7f3b99400640 1 -- 172.21.15.43:0/3156768622 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b9c19f640 con 0x7f3b9c1049a0 2024-09-19T05:06:49.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.497+0000 7f3b9a800640 1 --2- 172.21.15.43:0/3156768622 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b9c109120 0x7f3b9c19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:06:49.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.497+0000 7f3b99400640 1 --2- 172.21.15.43:0/3156768622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b9c1049a0 0x7f3b9c19abd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3b8800e9f0 tx=0x7f3b8800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:49.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.497+0000 7f3b92a00640 1 -- 172.21.15.43:0/3156768622 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8800ccc0 con 0x7f3b9c1049a0 2024-09-19T05:06:49.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.497+0000 7f3ba1200640 1 -- 172.21.15.43:0/3156768622 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b9c19f930 con 0x7f3b9c1049a0 2024-09-19T05:06:49.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.497+0000 7f3b92a00640 1 -- 172.21.15.43:0/3156768622 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b88005020 con 0x7f3b9c1049a0 2024-09-19T05:06:49.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.498+0000 7f3b92a00640 1 -- 172.21.15.43:0/3156768622 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b880049e0 con 0x7f3b9c1049a0 2024-09-19T05:06:49.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.498+0000 7f3ba1200640 1 -- 172.21.15.43:0/3156768622 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b9c1a7210 con 0x7f3b9c1049a0 2024-09-19T05:06:49.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.501+0000 7f3b92a00640 1 -- 172.21.15.43:0/3156768622 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f3b88005560 con 0x7f3b9c1049a0 2024-09-19T05:06:49.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.502+0000 7f3ba3c5a740 1 Processor -- start 2024-09-19T05:06:49.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.502+0000 7f3ba3c5a740 1 -- start start 2024-09-19T05:06:49.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.503+0000 7f3b92a00640 1 --2- 172.21.15.43:0/3156768622 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f3b800a33b0 0x7f3b800a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:49.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.503+0000 7f3ba3c5a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f23005ac0 0x555f23005ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:49.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.503+0000 7f3ba3c5a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f230063e0 0x555f23010760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:49.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.503+0000 7f3ba3c5a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f23010ca0 0x555f23013090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:49.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.503+0000 7f3b92a00640 1 -- 172.21.15.43:0/3156768622 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f3b880cfd70 con 0x7f3b9c1049a0 2024-09-19T05:06:49.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.503+0000 7f3ba3c5a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555f22f6c290 con 0x555f23010ca0 2024-09-19T05:06:49.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.503+0000 7f3ba3c5a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555f22f13b40 con 0x555f23005ac0 2024-09-19T05:06:49.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.503+0000 7f3ba3c5a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555f22f68250 con 0x555f230063e0 2024-09-19T05:06:49.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.503+0000 7f3b9a800640 1 --2- 172.21.15.43:0/3156768622 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f3b800a33b0 0x7f3b800a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:49.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.503+0000 7f3b99400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f230063e0 0x555f23010760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:49.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.504+0000 7f3b99400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f230063e0 0x555f23010760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:39082/0 (socket says 172.21.15.43:39082) 2024-09-19T05:06:49.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.504+0000 7f3b99400640 1 -- 172.21.15.43:0/921188591 learned_addr learned my addr 172.21.15.43:0/921188591 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:49.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.504+0000 7f3b99e00640 1 --2- 172.21.15.43:0/921188591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f23005ac0 0x555f23005ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:49.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.504+0000 7f3b9a800640 1 --2- 172.21.15.43:0/921188591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f23010ca0 0x555f23013090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:49.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.504+0000 7f3b99400640 1 -- 172.21.15.43:0/921188591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f23005ac0 msgr2=0x555f23005ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:49.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.504+0000 7f3b99400640 1 --2- 172.21.15.43:0/921188591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f23005ac0 0x555f23005ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:49.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.504+0000 7f3b99400640 1 -- 172.21.15.43:0/921188591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f23010ca0 msgr2=0x555f23013090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:49.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.504+0000 7f3b99400640 1 --2- 172.21.15.43:0/921188591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f23010ca0 0x555f23013090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:49.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.504+0000 7f3b99400640 1 -- 172.21.15.43:0/921188591 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f22cdd6a0 con 0x555f230063e0 2024-09-19T05:06:49.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.504+0000 7f3b9a800640 1 --2- 172.21.15.43:0/3156768622 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f3b800a33b0 0x7f3b800a5870 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3b94009270 tx=0x7f3b94012b70 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:49.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.504+0000 7f3b99400640 1 --2- 172.21.15.43:0/921188591 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f230063e0 0x555f23010760 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3b880a4740 tx=0x7f3b8809c700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ec79a25e9955864 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:49.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.505+0000 7f3b87400640 1 -- 172.21.15.43:0/921188591 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8801c040 con 0x555f230063e0 2024-09-19T05:06:49.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.505+0000 7f3b87400640 1 -- 172.21.15.43:0/921188591 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b880bc0d0 con 0x555f230063e0 2024-09-19T05:06:49.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.505+0000 7f3b87400640 1 -- 172.21.15.43:0/921188591 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b88098810 con 0x555f230063e0 2024-09-19T05:06:49.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.505+0000 7f3ba3c5a740 1 -- 172.21.15.43:0/921188591 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f230063e0 msgr2=0x555f23010760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:49.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.505+0000 7f3ba3c5a740 1 --2- 172.21.15.43:0/921188591 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f230063e0 0x555f23010760 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3b880a4740 tx=0x7f3b8809c700 comp rx=0 tx=0).stop 2024-09-19T05:06:49.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.505+0000 7f3ba3c5a740 1 -- 172.21.15.43:0/921188591 shutdown_connections 2024-09-19T05:06:49.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.505+0000 7f3ba3c5a740 1 --2- 172.21.15.43:0/921188591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f23010ca0 0x555f23013090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:49.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.505+0000 7f3ba3c5a740 1 --2- 172.21.15.43:0/921188591 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f230063e0 0x555f23010760 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:49.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.505+0000 7f3ba3c5a740 1 --2- 172.21.15.43:0/921188591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f23005ac0 0x555f23005ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:49.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.505+0000 7f3ba3c5a740 1 -- 172.21.15.43:0/921188591 >> 172.21.15.43:0/921188591 conn(0x555f23002050 msgr2=0x555f23002430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:49.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.506+0000 7f3ba3c5a740 1 -- 172.21.15.43:0/921188591 shutdown_connections 2024-09-19T05:06:49.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.506+0000 7f3ba3c5a740 1 -- 172.21.15.43:0/921188591 wait complete. 2024-09-19T05:06:49.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.506+0000 7f3ba3c5a740 1 -- start start 2024-09-19T05:06:49.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.507+0000 7f3ba3c5a740 1 Processor -- start 2024-09-19T05:06:49.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.507+0000 7f3ba3c5a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f23005ac0 0x555f23010630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:49.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.507+0000 7f3ba3c5a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f230063e0 0x555f2300d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:49.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.507+0000 7f3ba3c5a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f23010ca0 0x555f2300de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:49.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.507+0000 7f3ba3c5a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555f23013910 con 0x555f230063e0 2024-09-19T05:06:49.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.507+0000 7f3ba3c5a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555f22f26fa0 con 0x555f23005ac0 2024-09-19T05:06:49.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.507+0000 7f3ba3c5a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555f23013a90 con 0x555f23010ca0 2024-09-19T05:06:49.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.507+0000 7f3ba3c5a740 10 client.0 _ll_register_callbacks cb 0x555f22c7aba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:49.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.508+0000 7f3b99e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f23005ac0 0x555f23010630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:49.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.508+0000 7f3b99e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f23005ac0 0x555f23010630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:58386/0 (socket says 172.21.15.43:58386) 2024-09-19T05:06:49.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.508+0000 7f3b99e00640 1 -- 172.21.15.43:0/3602862558 learned_addr learned my addr 172.21.15.43:0/3602862558 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:49.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.508+0000 7f3b9a800640 1 --2- 172.21.15.43:0/3602862558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f23010ca0 0x555f2300de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:49.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.508+0000 7f3b99400640 1 --2- 172.21.15.43:0/3602862558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f230063e0 0x555f2300d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:49.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.508+0000 7f3b99e00640 1 -- 172.21.15.43:0/3602862558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f23010ca0 msgr2=0x555f2300de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:49.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.508+0000 7f3b99e00640 1 --2- 172.21.15.43:0/3602862558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f23010ca0 0x555f2300de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:49.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.508+0000 7f3b99e00640 1 -- 172.21.15.43:0/3602862558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f230063e0 msgr2=0x555f2300d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:49.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.508+0000 7f3b99e00640 1 --2- 172.21.15.43:0/3602862558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f230063e0 0x555f2300d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:49.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.508+0000 7f3b99e00640 1 -- 172.21.15.43:0/3602862558 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f2300e690 con 0x555f23005ac0 2024-09-19T05:06:49.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.508+0000 7f3b99400640 1 --2- 172.21.15.43:0/3602862558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f230063e0 0x555f2300d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:06:49.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.509+0000 7f3b99e00640 1 --2- 172.21.15.43:0/3602862558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f23005ac0 0x555f23010630 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c0042e0 tx=0x7f3b8c004310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d91ebfbf7f1f3e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:49.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.509+0000 7f3b7be00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:06:49.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.509+0000 7f3b7be00640 1 -- 172.21.15.43:0/3602862558 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c035040 con 0x555f23005ac0 2024-09-19T05:06:49.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.509+0000 7f3ba3c5a740 1 -- 172.21.15.43:0/3602862558 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b880bc0d0 con 0x555f23005ac0 2024-09-19T05:06:49.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.509+0000 7f3b7be00640 1 -- 172.21.15.43:0/3602862558 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f3b8c005220 con 0x555f23005ac0 2024-09-19T05:06:49.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.509+0000 7f3b7be00640 1 -- 172.21.15.43:0/3602862558 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3b8c005420 con 0x555f23005ac0 2024-09-19T05:06:49.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.509+0000 7f3ba3c5a740 10 client.25718 fetch_fsmap learned FSMap version 29 2024-09-19T05:06:49.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.509+0000 7f3ba3c5a740 1 -- 172.21.15.43:0/3602862558 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555f2301f860 con 0x555f23005ac0 2024-09-19T05:06:49.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.510+0000 7f3b7be00640 1 -- 172.21.15.43:0/3602862558 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b880bc0d0 con 0x555f23005ac0 2024-09-19T05:06:49.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.510+0000 7f3b7be00640 1 -- 172.21.15.43:0/3602862558 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3b8c003920 con 0x555f23005ac0 2024-09-19T05:06:49.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.510+0000 7f3ba3c5a740 10 client.25718 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:06:49.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.510+0000 7f3ba3c5a740 10 client.25718 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:06:49.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.510+0000 7f3ba3c5a740 20 client.25718 populate_metadata read hostname 'smithi043' 2024-09-19T05:06:49.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.510+0000 7f3ba3c5a740 1 --2- 172.21.15.43:0/3602862558 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x555f23020d20 0x555f230231a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:49.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.510+0000 7f3ba3c5a740 4 client.25718 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:06:49.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.510+0000 7f3ba3c5a740 1 -- 172.21.15.43:0/3602862558 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f22f79800 con 0x555f23020d20 2024-09-19T05:06:49.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.511+0000 7f3b99400640 1 --2- 172.21.15.43:0/3602862558 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x555f23020d20 0x555f230231a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:49.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.511+0000 7f3b99400640 1 --2- 172.21.15.43:0/3602862558 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x555f23020d20 0x555f230231a0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10e34b463138e735 server_cookie=3b499a9ecd66cf13 in_seq=0 out_seq=0 2024-09-19T05:06:49.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.511+0000 7f3b7be00640 10 client.25718 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:06:49.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.512+0000 7f3b7be00640 1 -- 172.21.15.43:0/3602862558 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555f22f79800 con 0x555f23020d20 2024-09-19T05:06:49.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.512+0000 7f3b7be00640 10 client.25718 handle_command_reply: tid=0 2024-09-19T05:06:49.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.512+0000 7f3ba3c5a740 1 client.25718 shutdown 2024-09-19T05:06:49.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.512+0000 7f3ba3c5a740 1 -- 172.21.15.43:0/3602862558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f23005ac0 msgr2=0x555f23010630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:49.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.512+0000 7f3ba3c5a740 1 --2- 172.21.15.43:0/3602862558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f23005ac0 0x555f23010630 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c0042e0 tx=0x7f3b8c004310 comp rx=0 tx=0).stop 2024-09-19T05:06:49.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.512+0000 7f3ba3c5a740 1 -- 172.21.15.43:0/3602862558 shutdown_connections 2024-09-19T05:06:49.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.512+0000 7f3ba3c5a740 1 --2- 172.21.15.43:0/3602862558 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x555f23020d20 0x555f230231a0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:49.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.512+0000 7f3ba3c5a740 1 --2- 172.21.15.43:0/3602862558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f23010ca0 0x555f2300de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:49.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.512+0000 7f3ba3c5a740 1 --2- 172.21.15.43:0/3602862558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f230063e0 0x555f2300d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:49.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.512+0000 7f3b7be00640 0 client.25718 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:06:49.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.512+0000 7f3ba3c5a740 1 --2- 172.21.15.43:0/3602862558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f23005ac0 0x555f23010630 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:49.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.512+0000 7f3ba3c5a740 1 -- 172.21.15.43:0/3602862558 >> 172.21.15.43:0/3602862558 conn(0x555f23002050 msgr2=0x555f23007d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:49.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.512+0000 7f3ba3c5a740 1 -- 172.21.15.43:0/3602862558 shutdown_connections 2024-09-19T05:06:49.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.512+0000 7f3ba3c5a740 1 -- 172.21.15.43:0/3602862558 wait complete. 2024-09-19T05:06:49.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.512+0000 7f3ba3c5a740 20 client.25718 trim_cache size 0 max 16384 2024-09-19T05:06:49.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.531+0000 7f3ba3c5a740 1 Processor -- start 2024-09-19T05:06:49.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.531+0000 7f3ba3c5a740 1 -- start start 2024-09-19T05:06:49.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.531+0000 7f3ba3c5a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f23005ac0 0x555f2300f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:49.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.531+0000 7f3ba3c5a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f230063e0 0x555f23028a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:49.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.531+0000 7f3ba3c5a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f23010ca0 0x555f23028fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:49.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.531+0000 7f3ba3c5a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9c1afa20 con 0x555f23010ca0 2024-09-19T05:06:49.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.531+0000 7f3ba3c5a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9c1b1560 con 0x555f23005ac0 2024-09-19T05:06:49.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.531+0000 7f3ba3c5a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9c1b00f0 con 0x555f230063e0 2024-09-19T05:06:49.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.532+0000 7f3b99400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f230063e0 0x555f23028a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:49.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.532+0000 7f3b99e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f23005ac0 0x555f2300f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:49.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.532+0000 7f3b99e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f23005ac0 0x555f2300f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:58402/0 (socket says 172.21.15.43:58402) 2024-09-19T05:06:49.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.532+0000 7f3b99400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f230063e0 0x555f23028a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:39108/0 (socket says 172.21.15.43:39108) 2024-09-19T05:06:49.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.532+0000 7f3b99e00640 1 -- 172.21.15.43:0/4068939758 learned_addr learned my addr 172.21.15.43:0/4068939758 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:49.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.532+0000 7f3b9a800640 1 --2- 172.21.15.43:0/4068939758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f23010ca0 0x555f23028fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:49.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.532+0000 7f3b99e00640 1 -- 172.21.15.43:0/4068939758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f230063e0 msgr2=0x555f23028a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:49.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.532+0000 7f3b99e00640 1 --2- 172.21.15.43:0/4068939758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f230063e0 0x555f23028a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:49.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.532+0000 7f3b99e00640 1 -- 172.21.15.43:0/4068939758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f23010ca0 msgr2=0x555f23028fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:49.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.532+0000 7f3b99e00640 1 --2- 172.21.15.43:0/4068939758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f23010ca0 0x555f23028fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:49.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.532+0000 7f3b99e00640 1 -- 172.21.15.43:0/4068939758 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f2300c890 con 0x555f23005ac0 2024-09-19T05:06:49.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.532+0000 7f3b99e00640 1 --2- 172.21.15.43:0/4068939758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f23005ac0 0x555f2300f850 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c009850 tx=0x7f3b8c005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=966cbce7201b359d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:49.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.533+0000 7f3b86a00640 1 -- 172.21.15.43:0/4068939758 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c035030 con 0x555f23005ac0 2024-09-19T05:06:49.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.533+0000 7f3b86a00640 1 -- 172.21.15.43:0/4068939758 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b8c0315b0 con 0x555f23005ac0 2024-09-19T05:06:49.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.533+0000 7f3b86a00640 1 -- 172.21.15.43:0/4068939758 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c052040 con 0x555f23005ac0 2024-09-19T05:06:49.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.533+0000 7f3ba3c5a740 1 -- 172.21.15.43:0/4068939758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f23005ac0 msgr2=0x555f2300f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:49.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.533+0000 7f3ba3c5a740 1 --2- 172.21.15.43:0/4068939758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f23005ac0 0x555f2300f850 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c009850 tx=0x7f3b8c005a20 comp rx=0 tx=0).stop 2024-09-19T05:06:49.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.533+0000 7f3ba3c5a740 1 -- 172.21.15.43:0/4068939758 shutdown_connections 2024-09-19T05:06:49.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.533+0000 7f3ba3c5a740 1 --2- 172.21.15.43:0/4068939758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f23010ca0 0x555f23028fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:49.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.533+0000 7f3ba3c5a740 1 --2- 172.21.15.43:0/4068939758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f230063e0 0x555f23028a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:49.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.533+0000 7f3ba3c5a740 1 --2- 172.21.15.43:0/4068939758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f23005ac0 0x555f2300f850 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:49.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.533+0000 7f3ba3c5a740 1 -- 172.21.15.43:0/4068939758 >> 172.21.15.43:0/4068939758 conn(0x555f23002050 msgr2=0x555f2300f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:49.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.534+0000 7f3ba3c5a740 1 -- 172.21.15.43:0/4068939758 shutdown_connections 2024-09-19T05:06:49.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.534+0000 7f3ba3c5a740 1 -- 172.21.15.43:0/4068939758 wait complete. 2024-09-19T05:06:49.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.534+0000 7f3ba3c5a740 1 -- start start 2024-09-19T05:06:49.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.534+0000 7f3ba3c5a740 1 Processor -- start 2024-09-19T05:06:49.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.535+0000 7f3ba3c5a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f23005ac0 0x555f23016d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:49.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.535+0000 7f3ba3c5a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f230063e0 0x555f230172c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:49.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.535+0000 7f3ba3c5a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f23010ca0 0x555f2308aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:49.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.535+0000 7f3ba3c5a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9c1b0e90 con 0x555f230063e0 2024-09-19T05:06:49.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.535+0000 7f3ba3c5a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9c1b07c0 con 0x555f23005ac0 2024-09-19T05:06:49.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.535+0000 7f3ba3c5a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9c1ada00 con 0x555f23010ca0 2024-09-19T05:06:49.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.535+0000 7f3ba3c5a740 10 client.0 _ll_register_callbacks cb 0x7f3b8c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:49.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.535+0000 7f3b99e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f23005ac0 0x555f23016d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:49.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.535+0000 7f3b99400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f230063e0 0x555f230172c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:49.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.535+0000 7f3b9a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f23010ca0 0x555f2308aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:49.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.535+0000 7f3b99e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f23005ac0 0x555f23016d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:58416/0 (socket says 172.21.15.43:58416) 2024-09-19T05:06:49.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.535+0000 7f3b99e00640 1 -- 172.21.15.43:0/3273005568 learned_addr learned my addr 172.21.15.43:0/3273005568 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:49.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.535+0000 7f3b99e00640 1 -- 172.21.15.43:0/3273005568 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f23010ca0 msgr2=0x555f2308aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:49.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.535+0000 7f3b99e00640 1 --2- 172.21.15.43:0/3273005568 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f23010ca0 0x555f2308aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:49.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.535+0000 7f3b99e00640 1 -- 172.21.15.43:0/3273005568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f230063e0 msgr2=0x555f230172c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:49.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.535+0000 7f3b99e00640 1 --2- 172.21.15.43:0/3273005568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f230063e0 0x555f230172c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:49.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.535+0000 7f3b99e00640 1 -- 172.21.15.43:0/3273005568 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f23017b50 con 0x555f23005ac0 2024-09-19T05:06:49.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.536+0000 7f3b99e00640 1 --2- 172.21.15.43:0/3273005568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f23005ac0 0x555f23016d80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c003e30 tx=0x7f3b8c005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92421fb81d5ab4c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:49.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.536+0000 7f3b7be00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:06:49.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.536+0000 7f3b7be00640 1 -- 172.21.15.43:0/3273005568 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c052040 con 0x555f23005ac0 2024-09-19T05:06:49.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.536+0000 7f3b7be00640 1 -- 172.21.15.43:0/3273005568 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f3b8c0035f0 con 0x555f23005ac0 2024-09-19T05:06:49.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.536+0000 7f3ba3c5a740 1 -- 172.21.15.43:0/3273005568 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b8c0315b0 con 0x555f23005ac0 2024-09-19T05:06:49.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.537+0000 7f3b7be00640 1 -- 172.21.15.43:0/3273005568 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3b8c000e60 con 0x555f23005ac0 2024-09-19T05:06:49.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.537+0000 7f3ba3c5a740 10 client.25724 fetch_fsmap learned FSMap version 29 2024-09-19T05:06:49.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.537+0000 7f3ba3c5a740 1 -- 172.21.15.43:0/3273005568 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555f2308c0a0 con 0x555f23005ac0 2024-09-19T05:06:49.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.537+0000 7f3b7be00640 1 -- 172.21.15.43:0/3273005568 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b8c0315b0 con 0x555f23005ac0 2024-09-19T05:06:49.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.537+0000 7f3b7be00640 1 -- 172.21.15.43:0/3273005568 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3b8c03d450 con 0x555f23005ac0 2024-09-19T05:06:49.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.537+0000 7f3ba3c5a740 10 client.25724 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:06:49.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.537+0000 7f3ba3c5a740 10 client.25724 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:06:49.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.537+0000 7f3ba3c5a740 20 client.25724 populate_metadata read hostname 'smithi043' 2024-09-19T05:06:49.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.537+0000 7f3ba3c5a740 1 --2- 172.21.15.43:0/3273005568 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x555f2308c6a0 0x555f2308ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:49.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.537+0000 7f3ba3c5a740 4 client.25724 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T05:06:49.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.537+0000 7f3ba3c5a740 1 -- 172.21.15.43:0/3273005568 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555f22f73d10 con 0x555f2308c6a0 2024-09-19T05:06:49.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.538+0000 7f3b99400640 1 --2- 172.21.15.43:0/3273005568 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x555f2308c6a0 0x555f2308ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:49.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.538+0000 7f3b99400640 1 --2- 172.21.15.43:0/3273005568 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x555f2308c6a0 0x555f2308ea90 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f3d21792fcdefda server_cookie=2a73d8e18d78c487 in_seq=0 out_seq=0 2024-09-19T05:06:49.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.538+0000 7f3b7be00640 10 client.25724 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:06:49.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.539+0000 7f3b7be00640 1 -- 172.21.15.43:0/3273005568 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555f22f73d10 con 0x555f2308c6a0 2024-09-19T05:06:49.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.539+0000 7f3b7be00640 10 client.25724 handle_command_reply: tid=0 2024-09-19T05:06:49.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.539+0000 7f3ba3c5a740 1 client.25724 shutdown 2024-09-19T05:06:49.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.539+0000 7f3ba3c5a740 1 -- 172.21.15.43:0/3273005568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f23005ac0 msgr2=0x555f23016d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:49.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.539+0000 7f3ba3c5a740 1 --2- 172.21.15.43:0/3273005568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f23005ac0 0x555f23016d80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c003e30 tx=0x7f3b8c005d50 comp rx=0 tx=0).stop 2024-09-19T05:06:49.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.539+0000 7f3ba3c5a740 1 -- 172.21.15.43:0/3273005568 shutdown_connections 2024-09-19T05:06:49.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.539+0000 7f3ba3c5a740 1 --2- 172.21.15.43:0/3273005568 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x555f2308c6a0 0x555f2308ea90 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:49.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.539+0000 7f3ba3c5a740 1 --2- 172.21.15.43:0/3273005568 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f23010ca0 0x555f2308aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:49.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.539+0000 7f3ba3c5a740 1 --2- 172.21.15.43:0/3273005568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f230063e0 0x555f230172c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:49.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.539+0000 7f3ba3c5a740 1 --2- 172.21.15.43:0/3273005568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f23005ac0 0x555f23016d80 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:49.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.539+0000 7f3b7be00640 0 client.25724 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:06:49.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.540+0000 7f3ba3c5a740 1 -- 172.21.15.43:0/3273005568 >> 172.21.15.43:0/3273005568 conn(0x555f23002050 msgr2=0x555f230011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:49.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.540+0000 7f3ba3c5a740 1 -- 172.21.15.43:0/3273005568 shutdown_connections 2024-09-19T05:06:49.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.540+0000 7f3ba3c5a740 1 -- 172.21.15.43:0/3273005568 wait complete. 2024-09-19T05:06:49.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.540+0000 7f3ba3c5a740 20 client.25724 trim_cache size 0 max 16384 2024-09-19T05:06:49.546 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:06:49.546 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-09-19T05:06:49.546 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "ae47fa73-f4ad-48ab-8ae3-67ee82d2443e", 2024-09-19T05:06:49.546 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-09-19T05:06:49.546 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:06:49.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.541+0000 7f3ba1200640 1 -- 172.21.15.43:0/3156768622 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f3b800a33b0 msgr2=0x7f3b800a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:49.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.541+0000 7f3ba1200640 1 --2- 172.21.15.43:0/3156768622 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f3b800a33b0 0x7f3b800a5870 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3b94009270 tx=0x7f3b94012b70 comp rx=0 tx=0).stop 2024-09-19T05:06:49.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.541+0000 7f3ba1200640 1 -- 172.21.15.43:0/3156768622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b9c1049a0 msgr2=0x7f3b9c19abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:49.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.541+0000 7f3ba1200640 1 --2- 172.21.15.43:0/3156768622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b9c1049a0 0x7f3b9c19abd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3b8800e9f0 tx=0x7f3b8800eeb0 comp rx=0 tx=0).stop 2024-09-19T05:06:49.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.541+0000 7f3ba1200640 1 -- 172.21.15.43:0/3156768622 shutdown_connections 2024-09-19T05:06:49.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.541+0000 7f3ba1200640 1 --2- 172.21.15.43:0/3156768622 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b9c109120 0x7f3b9c19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:49.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.541+0000 7f3ba1200640 1 --2- 172.21.15.43:0/3156768622 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f3b800a33b0 0x7f3b800a5870 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:49.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.541+0000 7f3ba1200640 1 --2- 172.21.15.43:0/3156768622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b9c1049a0 0x7f3b9c19abd0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:49.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.541+0000 7f3ba1200640 1 --2- 172.21.15.43:0/3156768622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b9c103ff0 0x7f3b9c19a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:49.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.541+0000 7f3ba1200640 1 -- 172.21.15.43:0/3156768622 >> 172.21.15.43:0/3156768622 conn(0x7f3b9c0ffa90 msgr2=0x7f3b9c100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:49.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.542+0000 7f3ba1200640 1 -- 172.21.15.43:0/3156768622 shutdown_connections 2024-09-19T05:06:49.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:49.542+0000 7f3ba1200640 1 -- 172.21.15.43:0/3156768622 wait complete. 2024-09-19T05:06:50.074 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:49 smithi043 ceph-mon[30800]: pgmap v68: 97 pgs: 97 active+clean; 1.0 GiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 MiB/s wr, 29 op/s 2024-09-19T05:06:50.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:49 smithi151 ceph-mon[35425]: pgmap v68: 97 pgs: 97 active+clean; 1.0 GiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 MiB/s wr, 29 op/s 2024-09-19T05:06:50.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:49 smithi096 ceph-mon[35085]: pgmap v68: 97 pgs: 97 active+clean; 1.0 GiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 MiB/s wr, 29 op/s 2024-09-19T05:06:50.436 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:06:50.894 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:06:51.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:50 smithi151 ceph-mon[35425]: scrub queued for path: / 2024-09-19T05:06:51.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:50 smithi151 ceph-mon[35425]: scrub summary: idle+waiting paths [/] 2024-09-19T05:06:51.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:50 smithi151 ceph-mon[35425]: scrub summary: active paths [/] 2024-09-19T05:06:51.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:50 smithi151 ceph-mon[35425]: scrub summary: idle+waiting paths [/] 2024-09-19T05:06:51.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:50 smithi096 ceph-mon[35085]: scrub queued for path: / 2024-09-19T05:06:51.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:50 smithi096 ceph-mon[35085]: scrub summary: idle+waiting paths [/] 2024-09-19T05:06:51.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:50 smithi096 ceph-mon[35085]: scrub summary: active paths [/] 2024-09-19T05:06:51.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:50 smithi096 ceph-mon[35085]: scrub summary: idle+waiting paths [/] 2024-09-19T05:06:51.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:50 smithi043 ceph-mon[30800]: scrub queued for path: / 2024-09-19T05:06:51.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:50 smithi043 ceph-mon[30800]: scrub summary: idle+waiting paths [/] 2024-09-19T05:06:51.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:50 smithi043 ceph-mon[30800]: scrub summary: active paths [/] 2024-09-19T05:06:51.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:50 smithi043 ceph-mon[30800]: scrub summary: idle+waiting paths [/] 2024-09-19T05:06:51.440 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-19T05:06:51.440 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-19T05:06:51.440 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-19T05:06:51.440 INFO:tasks.workunit.client.0.smithi043.stdout:iozone test complete. 2024-09-19T05:06:51.442 INFO:tasks.workunit.client.0.smithi043.stderr:+ iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-09-19T05:06:51.522 INFO:tasks.workunit.client.0.smithi043.stdout: Iozone: Performance Test of File I/O 2024-09-19T05:06:51.523 INFO:tasks.workunit.client.0.smithi043.stdout: Version $Revision: 3.494 $ 2024-09-19T05:06:51.523 INFO:tasks.workunit.client.0.smithi043.stdout: Compiled for 64 bit mode. 2024-09-19T05:06:51.523 INFO:tasks.workunit.client.0.smithi043.stdout: Build: linux-AMD64 2024-09-19T05:06:51.523 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-19T05:06:51.523 INFO:tasks.workunit.client.0.smithi043.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-09-19T05:06:51.523 INFO:tasks.workunit.client.0.smithi043.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-09-19T05:06:51.523 INFO:tasks.workunit.client.0.smithi043.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-09-19T05:06:51.523 INFO:tasks.workunit.client.0.smithi043.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-09-19T05:06:51.523 INFO:tasks.workunit.client.0.smithi043.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-09-19T05:06:51.523 INFO:tasks.workunit.client.0.smithi043.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-09-19T05:06:51.523 INFO:tasks.workunit.client.0.smithi043.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-09-19T05:06:51.523 INFO:tasks.workunit.client.0.smithi043.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-09-19T05:06:51.523 INFO:tasks.workunit.client.0.smithi043.stdout: Alexey Skidanov, Sudhir Kumar. 2024-09-19T05:06:51.524 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-19T05:06:51.524 INFO:tasks.workunit.client.0.smithi043.stdout: Run began: Thu Sep 19 05:06:51 2024 2024-09-19T05:06:51.524 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-19T05:06:51.524 INFO:tasks.workunit.client.0.smithi043.stdout: Include close in write timing 2024-09-19T05:06:51.524 INFO:tasks.workunit.client.0.smithi043.stdout: Include fsync in write timing 2024-09-19T05:06:51.524 INFO:tasks.workunit.client.0.smithi043.stdout: File size set to 10485760 kB 2024-09-19T05:06:51.524 INFO:tasks.workunit.client.0.smithi043.stdout: Record Size 1024 kB 2024-09-19T05:06:51.524 INFO:tasks.workunit.client.0.smithi043.stdout: Command line used: iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-09-19T05:06:51.524 INFO:tasks.workunit.client.0.smithi043.stdout: Output is in kBytes/sec 2024-09-19T05:06:51.524 INFO:tasks.workunit.client.0.smithi043.stdout: Time Resolution = 0.000001 seconds. 2024-09-19T05:06:51.524 INFO:tasks.workunit.client.0.smithi043.stdout: Processor cache size set to 1024 kBytes. 2024-09-19T05:06:51.524 INFO:tasks.workunit.client.0.smithi043.stdout: Processor cache line size set to 32 bytes. 2024-09-19T05:06:51.524 INFO:tasks.workunit.client.0.smithi043.stdout: File stride size set to 17 * record size. 2024-09-19T05:06:51.525 INFO:tasks.workunit.client.0.smithi043.stdout: Throughput test with 1 process 2024-09-19T05:06:51.525 INFO:tasks.workunit.client.0.smithi043.stdout: Each process writes a 10485760 kByte file in 1024 kByte records 2024-09-19T05:06:51.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.910+0000 7f1d1be00640 1 -- 172.21.15.43:0/4206667330 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d14103ff0 msgr2=0x7f1d141043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:51.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.910+0000 7f1d1be00640 1 --2- 172.21.15.43:0/4206667330 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d14103ff0 0x7f1d141043d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1d08009900 tx=0x7f1d0802a410 comp rx=0 tx=0).stop 2024-09-19T05:06:51.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.911+0000 7f1d1be00640 1 -- 172.21.15.43:0/4206667330 shutdown_connections 2024-09-19T05:06:51.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.911+0000 7f1d1be00640 1 --2- 172.21.15.43:0/4206667330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d14109120 0x7f1d14110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:51.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.911+0000 7f1d1be00640 1 --2- 172.21.15.43:0/4206667330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d141049a0 0x7f1d141089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:51.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.911+0000 7f1d1be00640 1 --2- 172.21.15.43:0/4206667330 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d14103ff0 0x7f1d141043d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:51.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.911+0000 7f1d1be00640 1 -- 172.21.15.43:0/4206667330 >> 172.21.15.43:0/4206667330 conn(0x7f1d140ffa90 msgr2=0x7f1d14101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:51.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.911+0000 7f1d1be00640 1 -- 172.21.15.43:0/4206667330 shutdown_connections 2024-09-19T05:06:51.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.911+0000 7f1d1be00640 1 -- 172.21.15.43:0/4206667330 wait complete. 2024-09-19T05:06:51.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.912+0000 7f1d1be00640 1 Processor -- start 2024-09-19T05:06:51.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.912+0000 7f1d1be00640 1 -- start start 2024-09-19T05:06:51.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.912+0000 7f1d1be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d14103ff0 0x7f1d1419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:51.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.912+0000 7f1d1be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d141049a0 0x7f1d1419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:51.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.912+0000 7f1d1be00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d14109120 0x7f1d1419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:51.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.912+0000 7f1d1be00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1d14114170 con 0x7f1d14103ff0 2024-09-19T05:06:51.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.912+0000 7f1d1be00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1d14113ff0 con 0x7f1d14109120 2024-09-19T05:06:51.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.912+0000 7f1d1be00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1d141142f0 con 0x7f1d141049a0 2024-09-19T05:06:51.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.912+0000 7f1d18a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d141049a0 0x7f1d1419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:51.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.913+0000 7f1d19e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d14109120 0x7f1d1419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:51.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.913+0000 7f1d18a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d141049a0 0x7f1d1419abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:39130/0 (socket says 172.21.15.43:39130) 2024-09-19T05:06:51.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.913+0000 7f1d18a00640 1 -- 172.21.15.43:0/4214936838 learned_addr learned my addr 172.21.15.43:0/4214936838 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:51.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.913+0000 7f1d19400640 1 --2- 172.21.15.43:0/4214936838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d14103ff0 0x7f1d1419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:51.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.913+0000 7f1d19e00640 1 -- 172.21.15.43:0/4214936838 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d141049a0 msgr2=0x7f1d1419abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:51.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.913+0000 7f1d19e00640 1 --2- 172.21.15.43:0/4214936838 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d141049a0 0x7f1d1419abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:51.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.913+0000 7f1d19e00640 1 -- 172.21.15.43:0/4214936838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d14103ff0 msgr2=0x7f1d1419a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:51.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.913+0000 7f1d19e00640 1 --2- 172.21.15.43:0/4214936838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d14103ff0 0x7f1d1419a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:51.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.913+0000 7f1d19e00640 1 -- 172.21.15.43:0/4214936838 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d1419f620 con 0x7f1d14109120 2024-09-19T05:06:51.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.913+0000 7f1d19400640 1 --2- 172.21.15.43:0/4214936838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d14103ff0 0x7f1d1419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:06:51.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.913+0000 7f1d19e00640 1 --2- 172.21.15.43:0/4214936838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d14109120 0x7f1d1419ef40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c009fd0 tx=0x7f1d0c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:51.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.914+0000 7f1d12000640 1 -- 172.21.15.43:0/4214936838 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c00ce20 con 0x7f1d14109120 2024-09-19T05:06:51.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.914+0000 7f1d12000640 1 -- 172.21.15.43:0/4214936838 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d0c005020 con 0x7f1d14109120 2024-09-19T05:06:51.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.914+0000 7f1d1be00640 1 -- 172.21.15.43:0/4214936838 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d1419f910 con 0x7f1d14109120 2024-09-19T05:06:51.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.914+0000 7f1d12000640 1 -- 172.21.15.43:0/4214936838 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c010720 con 0x7f1d14109120 2024-09-19T05:06:51.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.914+0000 7f1d1be00640 1 -- 172.21.15.43:0/4214936838 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d141a71f0 con 0x7f1d14109120 2024-09-19T05:06:51.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.917+0000 7f1d1e84c740 1 Processor -- start 2024-09-19T05:06:51.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.917+0000 7f1d12000640 1 -- 172.21.15.43:0/4214936838 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f1d0c005540 con 0x7f1d14109120 2024-09-19T05:06:51.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.918+0000 7f1d1e84c740 1 -- start start 2024-09-19T05:06:51.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.918+0000 7f1d1e84c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b31b7db90 0x558b31b7df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:51.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.918+0000 7f1d1e84c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b31b7e4b0 0x558b31b88830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:51.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.918+0000 7f1d1e84c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b31b88d70 0x558b31b8b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:51.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.918+0000 7f1d1e84c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558b31ae43e0 con 0x558b31b88d70 2024-09-19T05:06:51.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.918+0000 7f1d1e84c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558b31a8bd80 con 0x558b31b7db90 2024-09-19T05:06:51.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.918+0000 7f1d1e84c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558b31ae03a0 con 0x558b31b7e4b0 2024-09-19T05:06:51.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.918+0000 7f1d19400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b31b7e4b0 0x558b31b88830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:51.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.918+0000 7f1d18a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b31b7db90 0x558b31b7df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:51.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.918+0000 7f1d19400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b31b7e4b0 0x558b31b88830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:39144/0 (socket says 172.21.15.43:39144) 2024-09-19T05:06:51.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.918+0000 7f1d19400640 1 -- 172.21.15.43:0/2071917132 learned_addr learned my addr 172.21.15.43:0/2071917132 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:51.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.919+0000 7f1d18a00640 1 --2- 172.21.15.43:0/2071917132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b31b88d70 0x558b31b8b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:51.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.919+0000 7f1d18a00640 1 -- 172.21.15.43:0/2071917132 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b31b7e4b0 msgr2=0x558b31b88830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:51.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.919+0000 7f1d18a00640 1 --2- 172.21.15.43:0/2071917132 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b31b7e4b0 0x558b31b88830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:51.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.919+0000 7f1d18a00640 1 -- 172.21.15.43:0/2071917132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b31b88d70 msgr2=0x558b31b8b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:51.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.919+0000 7f1d18a00640 1 --2- 172.21.15.43:0/2071917132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b31b88d70 0x558b31b8b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:51.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.919+0000 7f1d18a00640 1 -- 172.21.15.43:0/2071917132 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b31aaaae0 con 0x558b31b7db90 2024-09-19T05:06:51.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.919+0000 7f1d18a00640 1 --2- 172.21.15.43:0/2071917132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b31b7db90 0x558b31b7df70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x558b31b7b4a0 tx=0x7f1d000094e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1a430f6de874fce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:51.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.919+0000 7f1d06600640 1 -- 172.21.15.43:0/2071917132 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d00004030 con 0x558b31b7db90 2024-09-19T05:06:51.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.920+0000 7f1d06600640 1 -- 172.21.15.43:0/2071917132 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d00005500 con 0x558b31b7db90 2024-09-19T05:06:51.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.920+0000 7f1d06600640 1 -- 172.21.15.43:0/2071917132 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d00004d00 con 0x558b31b7db90 2024-09-19T05:06:51.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.920+0000 7f1d12000640 1 --2- 172.21.15.43:0/4214936838 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f1cfc0a3430 0x7f1cfc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:51.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.920+0000 7f1d1e84c740 1 -- 172.21.15.43:0/2071917132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b31b7db90 msgr2=0x558b31b7df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:51.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.920+0000 7f1d1e84c740 1 --2- 172.21.15.43:0/2071917132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b31b7db90 0x558b31b7df70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x558b31b7b4a0 tx=0x7f1d000094e0 comp rx=0 tx=0).stop 2024-09-19T05:06:51.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.920+0000 7f1d12000640 1 -- 172.21.15.43:0/4214936838 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f1d0c0d0430 con 0x7f1d14109120 2024-09-19T05:06:51.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.920+0000 7f1d1e84c740 1 -- 172.21.15.43:0/2071917132 shutdown_connections 2024-09-19T05:06:51.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.920+0000 7f1d1e84c740 1 --2- 172.21.15.43:0/2071917132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b31b88d70 0x558b31b8b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:51.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.920+0000 7f1d1e84c740 1 --2- 172.21.15.43:0/2071917132 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b31b7e4b0 0x558b31b88830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:51.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.920+0000 7f1d1e84c740 1 --2- 172.21.15.43:0/2071917132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b31b7db90 0x558b31b7df70 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:51.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.920+0000 7f1d1e84c740 1 -- 172.21.15.43:0/2071917132 >> 172.21.15.43:0/2071917132 conn(0x558b31b7a120 msgr2=0x558b31b7a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:51.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.921+0000 7f1d19400640 1 --2- 172.21.15.43:0/4214936838 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f1cfc0a3430 0x7f1cfc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:51.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.921+0000 7f1d1e84c740 1 -- 172.21.15.43:0/2071917132 shutdown_connections 2024-09-19T05:06:51.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.922+0000 7f1d1e84c740 1 -- 172.21.15.43:0/2071917132 wait complete. 2024-09-19T05:06:51.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.922+0000 7f1d19400640 1 --2- 172.21.15.43:0/4214936838 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f1cfc0a3430 0x7f1cfc0a58f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1d08031550 tx=0x7f1d08005b90 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:51.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.922+0000 7f1d1e84c740 1 -- start start 2024-09-19T05:06:51.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.922+0000 7f1d1e84c740 1 Processor -- start 2024-09-19T05:06:51.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.923+0000 7f1d1e84c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b31b7db90 0x558b31b85cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:51.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.923+0000 7f1d1e84c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b31b7e4b0 0x558b31b86230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:51.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.923+0000 7f1d1e84c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b31b88d70 0x558b31b86770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:51.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.923+0000 7f1d1e84c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558b31b8b9e0 con 0x558b31b7db90 2024-09-19T05:06:51.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.923+0000 7f1d1e84c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558b31a9f0f0 con 0x558b31b7e4b0 2024-09-19T05:06:51.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.923+0000 7f1d1e84c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558b31b8bb60 con 0x558b31b88d70 2024-09-19T05:06:51.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.923+0000 7f1d1e84c740 10 client.0 _ll_register_callbacks cb 0x558b31a909e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:51.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.923+0000 7f1d18a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b31b7e4b0 0x558b31b86230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:51.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.923+0000 7f1d18a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b31b7e4b0 0x558b31b86230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:58450/0 (socket says 172.21.15.43:58450) 2024-09-19T05:06:51.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.923+0000 7f1d18a00640 1 -- 172.21.15.43:0/3907246044 learned_addr learned my addr 172.21.15.43:0/3907246044 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:51.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.923+0000 7f1d19400640 1 --2- 172.21.15.43:0/3907246044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b31b7db90 0x558b31b85cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:51.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.923+0000 7f1d19e00640 1 --2- 172.21.15.43:0/3907246044 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b31b88d70 0x558b31b86770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:51.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.924+0000 7f1d18a00640 1 -- 172.21.15.43:0/3907246044 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b31b88d70 msgr2=0x558b31b86770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:51.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.924+0000 7f1d18a00640 1 --2- 172.21.15.43:0/3907246044 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b31b88d70 0x558b31b86770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:51.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.924+0000 7f1d18a00640 1 -- 172.21.15.43:0/3907246044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b31b7db90 msgr2=0x558b31b85cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:51.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.924+0000 7f1d18a00640 1 --2- 172.21.15.43:0/3907246044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b31b7db90 0x558b31b85cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:51.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.924+0000 7f1d18a00640 1 -- 172.21.15.43:0/3907246044 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b318553c0 con 0x558b31b7e4b0 2024-09-19T05:06:51.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.924+0000 7f1d19e00640 1 --2- 172.21.15.43:0/3907246044 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b31b88d70 0x558b31b86770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:06:51.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.924+0000 7f1d18a00640 1 --2- 172.21.15.43:0/3907246044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b31b7e4b0 0x558b31b86230 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1d000094b0 tx=0x7f1d00009cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61c09a8e50c5038f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:51.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.924+0000 7f1cfaa00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:06:51.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.924+0000 7f1cfaa00640 1 -- 172.21.15.43:0/3907246044 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0000f790 con 0x558b31b7e4b0 2024-09-19T05:06:51.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.924+0000 7f1cfaa00640 1 -- 172.21.15.43:0/3907246044 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f1d0000f930 con 0x558b31b7e4b0 2024-09-19T05:06:51.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.924+0000 7f1d1e84c740 1 -- 172.21.15.43:0/3907246044 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d00005500 con 0x558b31b7e4b0 2024-09-19T05:06:51.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.925+0000 7f1cfaa00640 1 -- 172.21.15.43:0/3907246044 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1d0000fb30 con 0x558b31b7e4b0 2024-09-19T05:06:51.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.925+0000 7f1d1e84c740 10 client.25736 fetch_fsmap learned FSMap version 29 2024-09-19T05:06:51.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.925+0000 7f1d1e84c740 1 -- 172.21.15.43:0/3907246044 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558b31b97740 con 0x558b31b7e4b0 2024-09-19T05:06:51.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.925+0000 7f1cfaa00640 1 -- 172.21.15.43:0/3907246044 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d00005500 con 0x558b31b7e4b0 2024-09-19T05:06:51.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.925+0000 7f1cfaa00640 1 -- 172.21.15.43:0/3907246044 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1d00027020 con 0x558b31b7e4b0 2024-09-19T05:06:51.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.925+0000 7f1d1e84c740 10 client.25736 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:06:51.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.925+0000 7f1d1e84c740 10 client.25736 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:06:51.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.925+0000 7f1d1e84c740 20 client.25736 populate_metadata read hostname 'smithi043' 2024-09-19T05:06:51.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.925+0000 7f1d1e84c740 1 --2- 172.21.15.43:0/3907246044 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558b31b98a90 0x558b31b9ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:51.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.925+0000 7f1d1e84c740 4 client.25736 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:06:51.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.925+0000 7f1d1e84c740 1 -- 172.21.15.43:0/3907246044 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b31af1950 con 0x558b31b98a90 2024-09-19T05:06:51.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.926+0000 7f1d19400640 1 --2- 172.21.15.43:0/3907246044 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558b31b98a90 0x558b31b9ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:51.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.927+0000 7f1cfaa00640 10 client.25736 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:06:51.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.927+0000 7f1d19400640 1 --2- 172.21.15.43:0/3907246044 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558b31b98a90 0x558b31b9ae80 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=399d2e1c9608a822 server_cookie=ebe8539513f12c30 in_seq=0 out_seq=0 2024-09-19T05:06:51.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.927+0000 7f1cfaa00640 1 -- 172.21.15.43:0/3907246044 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558b31af1950 con 0x558b31b98a90 2024-09-19T05:06:51.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.927+0000 7f1cfaa00640 10 client.25736 handle_command_reply: tid=0 2024-09-19T05:06:51.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.927+0000 7f1d1e84c740 1 client.25736 shutdown 2024-09-19T05:06:51.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.928+0000 7f1d1e84c740 1 -- 172.21.15.43:0/3907246044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b31b7e4b0 msgr2=0x558b31b86230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:51.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.928+0000 7f1d1e84c740 1 --2- 172.21.15.43:0/3907246044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b31b7e4b0 0x558b31b86230 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1d000094b0 tx=0x7f1d00009cd0 comp rx=0 tx=0).stop 2024-09-19T05:06:51.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.928+0000 7f1d1e84c740 1 -- 172.21.15.43:0/3907246044 shutdown_connections 2024-09-19T05:06:51.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.928+0000 7f1d1e84c740 1 --2- 172.21.15.43:0/3907246044 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558b31b98a90 0x558b31b9ae80 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:51.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.928+0000 7f1d1e84c740 1 --2- 172.21.15.43:0/3907246044 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b31b88d70 0x558b31b86770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:51.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.928+0000 7f1d1e84c740 1 --2- 172.21.15.43:0/3907246044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b31b7e4b0 0x558b31b86230 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:51.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.928+0000 7f1d1e84c740 1 --2- 172.21.15.43:0/3907246044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b31b7db90 0x558b31b85cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:51.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.928+0000 7f1d1e84c740 1 -- 172.21.15.43:0/3907246044 >> 172.21.15.43:0/3907246044 conn(0x558b31b7a120 msgr2=0x558b31b7fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:51.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.928+0000 7f1cfaa00640 0 client.25736 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:06:51.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.928+0000 7f1d1e84c740 1 -- 172.21.15.43:0/3907246044 shutdown_connections 2024-09-19T05:06:51.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.928+0000 7f1d1e84c740 1 -- 172.21.15.43:0/3907246044 wait complete. 2024-09-19T05:06:51.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.928+0000 7f1d1e84c740 20 client.25736 trim_cache size 0 max 16384 2024-09-19T05:06:51.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.947+0000 7f1d1e84c740 1 Processor -- start 2024-09-19T05:06:51.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.947+0000 7f1d1e84c740 1 -- start start 2024-09-19T05:06:51.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.947+0000 7f1d1e84c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b31b7db90 0x558b31b805f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:51.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.947+0000 7f1d1e84c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b31b7e4b0 0x558b31ba0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:51.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.947+0000 7f1d1e84c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b31b88d70 0x558b31ba0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:51.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.947+0000 7f1d1e84c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1d141afae0 con 0x558b31b7db90 2024-09-19T05:06:51.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.947+0000 7f1d1e84c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1d141b1620 con 0x558b31b88d70 2024-09-19T05:06:51.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.947+0000 7f1d1e84c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1d141b01b0 con 0x558b31b7e4b0 2024-09-19T05:06:51.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.948+0000 7f1d18a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b31b7e4b0 0x558b31ba0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:51.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.948+0000 7f1d18a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b31b7e4b0 0x558b31ba0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:39166/0 (socket says 172.21.15.43:39166) 2024-09-19T05:06:51.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.948+0000 7f1d18a00640 1 -- 172.21.15.43:0/3661616402 learned_addr learned my addr 172.21.15.43:0/3661616402 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:51.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.948+0000 7f1d19400640 1 --2- 172.21.15.43:0/3661616402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b31b7db90 0x558b31b805f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:51.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.948+0000 7f1d19e00640 1 --2- 172.21.15.43:0/3661616402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b31b88d70 0x558b31ba0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:51.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.948+0000 7f1d18a00640 1 -- 172.21.15.43:0/3661616402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b31b88d70 msgr2=0x558b31ba0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:51.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.948+0000 7f1d18a00640 1 --2- 172.21.15.43:0/3661616402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b31b88d70 0x558b31ba0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:51.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.948+0000 7f1d18a00640 1 -- 172.21.15.43:0/3661616402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b31b7db90 msgr2=0x558b31b805f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:51.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.948+0000 7f1d18a00640 1 --2- 172.21.15.43:0/3661616402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b31b7db90 0x558b31b805f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:51.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.948+0000 7f1d18a00640 1 -- 172.21.15.43:0/3661616402 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b31b7adb0 con 0x558b31b7e4b0 2024-09-19T05:06:51.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.949+0000 7f1d18a00640 1 --2- 172.21.15.43:0/3661616402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b31b7e4b0 0x558b31ba0610 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1d00011830 tx=0x7f1d00005db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a993851781d4f2d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:51.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.949+0000 7f1d05c00640 1 -- 172.21.15.43:0/3661616402 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d00036020 con 0x558b31b7e4b0 2024-09-19T05:06:51.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.949+0000 7f1d05c00640 1 -- 172.21.15.43:0/3661616402 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d00004c90 con 0x558b31b7e4b0 2024-09-19T05:06:51.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.949+0000 7f1d05c00640 1 -- 172.21.15.43:0/3661616402 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d000035f0 con 0x558b31b7e4b0 2024-09-19T05:06:51.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.950+0000 7f1d1e84c740 1 -- 172.21.15.43:0/3661616402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b31b7e4b0 msgr2=0x558b31ba0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:51.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.950+0000 7f1d1e84c740 1 --2- 172.21.15.43:0/3661616402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b31b7e4b0 0x558b31ba0610 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1d00011830 tx=0x7f1d00005db0 comp rx=0 tx=0).stop 2024-09-19T05:06:51.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.950+0000 7f1d1e84c740 1 -- 172.21.15.43:0/3661616402 shutdown_connections 2024-09-19T05:06:51.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.950+0000 7f1d1e84c740 1 --2- 172.21.15.43:0/3661616402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b31b88d70 0x558b31ba0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:51.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.950+0000 7f1d1e84c740 1 --2- 172.21.15.43:0/3661616402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b31b7e4b0 0x558b31ba0610 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:51.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.950+0000 7f1d1e84c740 1 --2- 172.21.15.43:0/3661616402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b31b7db90 0x558b31b805f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:51.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.950+0000 7f1d1e84c740 1 -- 172.21.15.43:0/3661616402 >> 172.21.15.43:0/3661616402 conn(0x558b31b7a120 msgr2=0x558b31b87d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:51.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.950+0000 7f1d1e84c740 1 -- 172.21.15.43:0/3661616402 shutdown_connections 2024-09-19T05:06:51.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.950+0000 7f1d1e84c740 1 -- 172.21.15.43:0/3661616402 wait complete. 2024-09-19T05:06:51.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.951+0000 7f1d1e84c740 1 -- start start 2024-09-19T05:06:51.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.951+0000 7f1d1e84c740 1 Processor -- start 2024-09-19T05:06:51.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.951+0000 7f1d1e84c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b31b7db90 0x558b31b8ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:51.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.951+0000 7f1d1e84c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b31b7e4b0 0x558b31b8cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:51.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.951+0000 7f1d1e84c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b31b88d70 0x558b31b8d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:51.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.951+0000 7f1d1e84c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1d141b0f50 con 0x558b31b7e4b0 2024-09-19T05:06:51.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.951+0000 7f1d1e84c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1d141b0880 con 0x558b31b7db90 2024-09-19T05:06:51.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.951+0000 7f1d1e84c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1d141adac0 con 0x558b31b88d70 2024-09-19T05:06:51.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.951+0000 7f1d1e84c740 10 client.0 _ll_register_callbacks cb 0x7f1d00011760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:06:51.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.952+0000 7f1d19400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b31b7db90 0x558b31b8ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:51.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.952+0000 7f1d19400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b31b7db90 0x558b31b8ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:58464/0 (socket says 172.21.15.43:58464) 2024-09-19T05:06:51.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.952+0000 7f1d19400640 1 -- 172.21.15.43:0/4125342604 learned_addr learned my addr 172.21.15.43:0/4125342604 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:06:51.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.952+0000 7f1d18a00640 1 --2- 172.21.15.43:0/4125342604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b31b7e4b0 0x558b31b8cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:51.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.952+0000 7f1d19e00640 1 --2- 172.21.15.43:0/4125342604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b31b88d70 0x558b31b8d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:51.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.952+0000 7f1d19400640 1 -- 172.21.15.43:0/4125342604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b31b88d70 msgr2=0x558b31b8d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:51.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.952+0000 7f1d19400640 1 --2- 172.21.15.43:0/4125342604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b31b88d70 0x558b31b8d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:51.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.952+0000 7f1d19400640 1 -- 172.21.15.43:0/4125342604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b31b7e4b0 msgr2=0x558b31b8cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:06:51.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.952+0000 7f1d19400640 1 --2- 172.21.15.43:0/4125342604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b31b7e4b0 0x558b31b8cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:51.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.952+0000 7f1d19400640 1 -- 172.21.15.43:0/4125342604 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b31b9b3e0 con 0x558b31b7db90 2024-09-19T05:06:51.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.952+0000 7f1d19400640 1 --2- 172.21.15.43:0/4125342604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b31b7db90 0x558b31b8ca80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1d0802a930 tx=0x7f1d08009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5be0cdf69865ea6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:06:51.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.952+0000 7f1cfaa00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:06:51.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.953+0000 7f1cfaa00640 1 -- 172.21.15.43:0/4125342604 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d08035040 con 0x558b31b7db90 2024-09-19T05:06:51.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.953+0000 7f1cfaa00640 1 -- 172.21.15.43:0/4125342604 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f1d080048c0 con 0x558b31b7db90 2024-09-19T05:06:51.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.953+0000 7f1d1e84c740 1 -- 172.21.15.43:0/4125342604 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d00004c90 con 0x558b31b7db90 2024-09-19T05:06:51.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.953+0000 7f1cfaa00640 1 -- 172.21.15.43:0/4125342604 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1d08004ac0 con 0x558b31b7db90 2024-09-19T05:06:51.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.953+0000 7f1d1e84c740 10 client.25742 fetch_fsmap learned FSMap version 29 2024-09-19T05:06:51.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.953+0000 7f1d1e84c740 1 -- 172.21.15.43:0/4125342604 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558b31c036c0 con 0x558b31b7db90 2024-09-19T05:06:51.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.953+0000 7f1cfaa00640 1 -- 172.21.15.43:0/4125342604 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d00004c90 con 0x558b31b7db90 2024-09-19T05:06:51.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.954+0000 7f1cfaa00640 1 -- 172.21.15.43:0/4125342604 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1d08058070 con 0x558b31b7db90 2024-09-19T05:06:51.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.954+0000 7f1d1e84c740 10 client.25742 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:06:51.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.954+0000 7f1d1e84c740 10 client.25742 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:06:51.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.954+0000 7f1d1e84c740 20 client.25742 populate_metadata read hostname 'smithi043' 2024-09-19T05:06:51.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.954+0000 7f1d1e84c740 1 --2- 172.21.15.43:0/4125342604 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558b31c03dc0 0x558b31c06200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:06:51.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.954+0000 7f1d1e84c740 4 client.25742 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:06:51.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.954+0000 7f1d1e84c740 1 -- 172.21.15.43:0/4125342604 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558b31aebe60 con 0x558b31c03dc0 2024-09-19T05:06:51.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.954+0000 7f1d18a00640 1 --2- 172.21.15.43:0/4125342604 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558b31c03dc0 0x558b31c06200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:06:51.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.955+0000 7f1d18a00640 1 --2- 172.21.15.43:0/4125342604 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558b31c03dc0 0x558b31c06200 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39c74ecfb0f1f6b1 server_cookie=b5ccef5debbd12d8 in_seq=0 out_seq=0 2024-09-19T05:06:51.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.955+0000 7f1cfaa00640 10 client.25742 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:06:51.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.955+0000 7f1cfaa00640 1 -- 172.21.15.43:0/4125342604 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558b31aebe60 con 0x558b31c03dc0 2024-09-19T05:06:51.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.955+0000 7f1cfaa00640 10 client.25742 handle_command_reply: tid=0 2024-09-19T05:06:51.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.955+0000 7f1d1e84c740 1 client.25742 shutdown 2024-09-19T05:06:51.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.955+0000 7f1d1e84c740 1 -- 172.21.15.43:0/4125342604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b31b7db90 msgr2=0x558b31b8ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:51.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.955+0000 7f1d1e84c740 1 --2- 172.21.15.43:0/4125342604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b31b7db90 0x558b31b8ca80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1d0802a930 tx=0x7f1d08009260 comp rx=0 tx=0).stop 2024-09-19T05:06:51.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.955+0000 7f1d1e84c740 1 -- 172.21.15.43:0/4125342604 shutdown_connections 2024-09-19T05:06:51.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.955+0000 7f1d1e84c740 1 --2- 172.21.15.43:0/4125342604 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558b31c03dc0 0x558b31c06200 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:51.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.955+0000 7f1d1e84c740 1 --2- 172.21.15.43:0/4125342604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b31b88d70 0x558b31b8d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:51.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.955+0000 7f1d1e84c740 1 --2- 172.21.15.43:0/4125342604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b31b7e4b0 0x558b31b8cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:51.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.955+0000 7f1d1e84c740 1 --2- 172.21.15.43:0/4125342604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b31b7db90 0x558b31b8ca80 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:51.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.955+0000 7f1d1e84c740 1 -- 172.21.15.43:0/4125342604 >> 172.21.15.43:0/4125342604 conn(0x558b31b7a120 msgr2=0x558b31b789c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:51.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.955+0000 7f1cfaa00640 0 client.25742 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:06:51.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.956+0000 7f1d1e84c740 1 -- 172.21.15.43:0/4125342604 shutdown_connections 2024-09-19T05:06:51.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.956+0000 7f1d1e84c740 1 -- 172.21.15.43:0/4125342604 wait complete. 2024-09-19T05:06:51.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.956+0000 7f1d1e84c740 20 client.25742 trim_cache size 0 max 16384 2024-09-19T05:06:51.962 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:06:51.962 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T05:06:51.962 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-09-19T05:06:51.962 INFO:teuthology.orchestra.run.smithi043.stdout: "ae47fa73-f4ad-48ab-8ae3-67ee82d2443e": { 2024-09-19T05:06:51.962 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-09-19T05:06:51.962 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "ae47fa73-f4ad-48ab-8ae3-67ee82d2443e", 2024-09-19T05:06:51.963 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-09-19T05:06:51.963 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:06:51.963 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:06:51.963 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:06:51.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.957+0000 7f1d1be00640 1 -- 172.21.15.43:0/4214936838 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f1cfc0a3430 msgr2=0x7f1cfc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:51.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.957+0000 7f1d1be00640 1 --2- 172.21.15.43:0/4214936838 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f1cfc0a3430 0x7f1cfc0a58f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1d08031550 tx=0x7f1d08005b90 comp rx=0 tx=0).stop 2024-09-19T05:06:51.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.957+0000 7f1d1be00640 1 -- 172.21.15.43:0/4214936838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d14109120 msgr2=0x7f1d1419ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:06:51.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.957+0000 7f1d1be00640 1 --2- 172.21.15.43:0/4214936838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d14109120 0x7f1d1419ef40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c009fd0 tx=0x7f1d0c00ef10 comp rx=0 tx=0).stop 2024-09-19T05:06:51.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.957+0000 7f1d1be00640 1 -- 172.21.15.43:0/4214936838 shutdown_connections 2024-09-19T05:06:51.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.957+0000 7f1d1be00640 1 --2- 172.21.15.43:0/4214936838 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f1cfc0a3430 0x7f1cfc0a58f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:51.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.957+0000 7f1d1be00640 1 --2- 172.21.15.43:0/4214936838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d14109120 0x7f1d1419ef40 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:51.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.957+0000 7f1d1be00640 1 --2- 172.21.15.43:0/4214936838 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d141049a0 0x7f1d1419abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:51.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.957+0000 7f1d1be00640 1 --2- 172.21.15.43:0/4214936838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d14103ff0 0x7f1d1419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:06:51.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.957+0000 7f1d1be00640 1 -- 172.21.15.43:0/4214936838 >> 172.21.15.43:0/4214936838 conn(0x7f1d140ffa90 msgr2=0x7f1d14100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:06:51.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.957+0000 7f1d1be00640 1 -- 172.21.15.43:0/4214936838 shutdown_connections 2024-09-19T05:06:51.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:06:51.957+0000 7f1d1be00640 1 -- 172.21.15.43:0/4214936838 wait complete. 2024-09-19T05:06:52.124 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:51 smithi096 ceph-mon[35085]: pgmap v69: 97 pgs: 97 active+clean; 1.0 GiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 85 MiB/s wr, 34 op/s 2024-09-19T05:06:52.124 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:51 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:06:52.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:51 smithi151 ceph-mon[35425]: pgmap v69: 97 pgs: 97 active+clean; 1.0 GiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 85 MiB/s wr, 34 op/s 2024-09-19T05:06:52.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:51 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:06:52.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:51 smithi043 ceph-mon[30800]: pgmap v69: 97 pgs: 97 active+clean; 1.0 GiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 85 MiB/s wr, 34 op/s 2024-09-19T05:06:52.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:51 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:06:52.821 INFO:tasks.cephfs.filesystem:scrub status for tag:ae47fa73-f4ad-48ab-8ae3-67ee82d2443e - {'path': '/', 'tag': 'ae47fa73-f4ad-48ab-8ae3-67ee82d2443e', 'options': 'recursive,force'} 2024-09-19T05:06:53.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:52 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:06:53.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:52 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:06:53.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:52 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:06:53.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:52 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:06:53.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:52 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:06:53.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:52 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:06:53.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:52 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:06:53.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:52 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:06:53.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:52 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:06:54.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:53 smithi151 ceph-mon[35425]: pgmap v70: 97 pgs: 97 active+clean; 1.0 GiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-09-19T05:06:54.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:53 smithi151 ceph-mon[35425]: scrub summary: idle 2024-09-19T05:06:54.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:53 smithi096 ceph-mon[35085]: pgmap v70: 97 pgs: 97 active+clean; 1.0 GiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-09-19T05:06:54.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:53 smithi096 ceph-mon[35085]: scrub summary: idle 2024-09-19T05:06:54.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:53 smithi043 ceph-mon[30800]: pgmap v70: 97 pgs: 97 active+clean; 1.0 GiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-09-19T05:06:54.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:53 smithi043 ceph-mon[30800]: scrub summary: idle 2024-09-19T05:06:56.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:55 smithi151 ceph-mon[35425]: pgmap v71: 97 pgs: 97 active+clean; 1.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.0 KiB/s wr, 36 op/s 2024-09-19T05:06:56.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:55 smithi096 ceph-mon[35085]: pgmap v71: 97 pgs: 97 active+clean; 1.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.0 KiB/s wr, 36 op/s 2024-09-19T05:06:56.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:55 smithi043 ceph-mon[30800]: pgmap v71: 97 pgs: 97 active+clean; 1.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.0 KiB/s wr, 36 op/s 2024-09-19T05:06:58.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:57 smithi151 ceph-mon[35425]: pgmap v72: 97 pgs: 97 active+clean; 1.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 KiB/s wr, 32 op/s 2024-09-19T05:06:58.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:57 smithi096 ceph-mon[35085]: pgmap v72: 97 pgs: 97 active+clean; 1.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 KiB/s wr, 32 op/s 2024-09-19T05:06:58.283 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:57 smithi043 ceph-mon[30800]: pgmap v72: 97 pgs: 97 active+clean; 1.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 KiB/s wr, 32 op/s 2024-09-19T05:06:59.862 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:06:59 smithi043 ceph-mon[30800]: pgmap v73: 97 pgs: 97 active+clean; 1.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 KiB/s wr, 32 op/s 2024-09-19T05:07:00.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:06:59 smithi151 ceph-mon[35425]: pgmap v73: 97 pgs: 97 active+clean; 1.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 KiB/s wr, 32 op/s 2024-09-19T05:07:00.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:06:59 smithi096 ceph-mon[35085]: pgmap v73: 97 pgs: 97 active+clean; 1.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 KiB/s wr, 32 op/s 2024-09-19T05:07:02.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:07:01 smithi151 ceph-mon[35425]: pgmap v74: 97 pgs: 97 active+clean; 1.4 GiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 117 MiB/s wr, 67 op/s 2024-09-19T05:07:02.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:07:01 smithi096 ceph-mon[35085]: pgmap v74: 97 pgs: 97 active+clean; 1.4 GiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 117 MiB/s wr, 67 op/s 2024-09-19T05:07:02.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:07:01 smithi043 ceph-mon[30800]: pgmap v74: 97 pgs: 97 active+clean; 1.4 GiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 117 MiB/s wr, 67 op/s 2024-09-19T05:07:04.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:07:03 smithi151 ceph-mon[35425]: pgmap v75: 97 pgs: 97 active+clean; 1.4 GiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 117 MiB/s wr, 63 op/s 2024-09-19T05:07:04.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:07:03 smithi096 ceph-mon[35085]: pgmap v75: 97 pgs: 97 active+clean; 1.4 GiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 117 MiB/s wr, 63 op/s 2024-09-19T05:07:04.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:07:03 smithi043 ceph-mon[30800]: pgmap v75: 97 pgs: 97 active+clean; 1.4 GiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 117 MiB/s wr, 63 op/s 2024-09-19T05:07:06.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:07:05 smithi151 ceph-mon[35425]: pgmap v76: 97 pgs: 97 active+clean; 2.7 GiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 231 MiB/s wr, 112 op/s 2024-09-19T05:07:06.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:07:05 smithi096 ceph-mon[35085]: pgmap v76: 97 pgs: 97 active+clean; 2.7 GiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 231 MiB/s wr, 112 op/s 2024-09-19T05:07:06.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:07:05 smithi043 ceph-mon[30800]: pgmap v76: 97 pgs: 97 active+clean; 2.7 GiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 231 MiB/s wr, 112 op/s 2024-09-19T05:07:08.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:07:07 smithi151 ceph-mon[35425]: pgmap v77: 97 pgs: 97 active+clean; 2.7 GiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 231 MiB/s wr, 84 op/s 2024-09-19T05:07:08.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:07:07 smithi096 ceph-mon[35085]: pgmap v77: 97 pgs: 97 active+clean; 2.7 GiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 231 MiB/s wr, 84 op/s 2024-09-19T05:07:08.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:07:07 smithi043 ceph-mon[30800]: pgmap v77: 97 pgs: 97 active+clean; 2.7 GiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 231 MiB/s wr, 84 op/s 2024-09-19T05:07:10.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:07:09 smithi151 ceph-mon[35425]: pgmap v78: 97 pgs: 97 active+clean; 2.7 GiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 231 MiB/s wr, 84 op/s 2024-09-19T05:07:10.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:07:09 smithi096 ceph-mon[35085]: pgmap v78: 97 pgs: 97 active+clean; 2.7 GiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 231 MiB/s wr, 84 op/s 2024-09-19T05:07:10.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:07:09 smithi043 ceph-mon[30800]: pgmap v78: 97 pgs: 97 active+clean; 2.7 GiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 231 MiB/s wr, 84 op/s 2024-09-19T05:07:11.933 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:07:11 smithi043 ceph-mon[30800]: pgmap v79: 97 pgs: 97 active+clean; 4.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 365 MiB/s wr, 123 op/s 2024-09-19T05:07:12.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:07:11 smithi096 ceph-mon[35085]: pgmap v79: 97 pgs: 97 active+clean; 4.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 365 MiB/s wr, 123 op/s 2024-09-19T05:07:12.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:07:11 smithi151 ceph-mon[35425]: pgmap v79: 97 pgs: 97 active+clean; 4.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 365 MiB/s wr, 123 op/s 2024-09-19T05:07:14.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:07:13 smithi096 ceph-mon[35085]: pgmap v80: 97 pgs: 97 active+clean; 4.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 249 MiB/s wr, 88 op/s 2024-09-19T05:07:14.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:07:13 smithi043 ceph-mon[30800]: pgmap v80: 97 pgs: 97 active+clean; 4.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 249 MiB/s wr, 88 op/s 2024-09-19T05:07:14.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:07:13 smithi151 ceph-mon[35425]: pgmap v80: 97 pgs: 97 active+clean; 4.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 249 MiB/s wr, 88 op/s 2024-09-19T05:07:16.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:07:15 smithi096 ceph-mon[35085]: pgmap v81: 97 pgs: 97 active+clean; 5.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 380 MiB/s wr, 126 op/s 2024-09-19T05:07:16.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:07:15 smithi043 ceph-mon[30800]: pgmap v81: 97 pgs: 97 active+clean; 5.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 380 MiB/s wr, 126 op/s 2024-09-19T05:07:16.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:07:15 smithi151 ceph-mon[35425]: pgmap v81: 97 pgs: 97 active+clean; 5.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 380 MiB/s wr, 126 op/s 2024-09-19T05:07:18.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:07:17 smithi096 ceph-mon[35085]: pgmap v82: 97 pgs: 97 active+clean; 5.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 266 MiB/s wr, 77 op/s 2024-09-19T05:07:18.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:07:17 smithi043 ceph-mon[30800]: pgmap v82: 97 pgs: 97 active+clean; 5.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 266 MiB/s wr, 77 op/s 2024-09-19T05:07:18.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:07:17 smithi151 ceph-mon[35425]: pgmap v82: 97 pgs: 97 active+clean; 5.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 266 MiB/s wr, 77 op/s 2024-09-19T05:07:20.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:07:19 smithi096 ceph-mon[35085]: pgmap v83: 97 pgs: 97 active+clean; 5.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 266 MiB/s wr, 77 op/s 2024-09-19T05:07:20.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:07:19 smithi043 ceph-mon[30800]: pgmap v83: 97 pgs: 97 active+clean; 5.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 266 MiB/s wr, 77 op/s 2024-09-19T05:07:20.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:07:19 smithi151 ceph-mon[35425]: pgmap v83: 97 pgs: 97 active+clean; 5.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 266 MiB/s wr, 77 op/s 2024-09-19T05:07:22.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:07:21 smithi043 ceph-mon[30800]: pgmap v84: 97 pgs: 97 active+clean; 7.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 410 MiB/s wr, 118 op/s 2024-09-19T05:07:22.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:07:21 smithi151 ceph-mon[35425]: pgmap v84: 97 pgs: 97 active+clean; 7.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 410 MiB/s wr, 118 op/s 2024-09-19T05:07:22.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:07:21 smithi096 ceph-mon[35085]: pgmap v84: 97 pgs: 97 active+clean; 7.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 410 MiB/s wr, 118 op/s 2024-09-19T05:07:22.823 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:07:23.599 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:07:24.138 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:07:23 smithi043 ceph-mon[30800]: pgmap v85: 97 pgs: 97 active+clean; 7.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 276 MiB/s wr, 78 op/s 2024-09-19T05:07:24.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:07:23 smithi151 ceph-mon[35425]: pgmap v85: 97 pgs: 97 active+clean; 7.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 276 MiB/s wr, 78 op/s 2024-09-19T05:07:24.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:07:23 smithi096 ceph-mon[35085]: pgmap v85: 97 pgs: 97 active+clean; 7.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 276 MiB/s wr, 78 op/s 2024-09-19T05:07:24.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.517+0000 7f2b8f200640 1 -- 172.21.15.43:0/1254827831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b90112870 msgr2=0x7f2b90114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:24.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.517+0000 7f2b8f200640 1 --2- 172.21.15.43:0/1254827831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b90112870 0x7f2b90114c60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2b88009ca0 tx=0x7f2b8802a8d0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.518+0000 7f2b8f200640 1 -- 172.21.15.43:0/1254827831 shutdown_connections 2024-09-19T05:07:24.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.518+0000 7f2b8f200640 1 --2- 172.21.15.43:0/1254827831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b90112870 0x7f2b90114c60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.518+0000 7f2b8f200640 1 --2- 172.21.15.43:0/1254827831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b90077040 0x7f2b90112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.518+0000 7f2b8f200640 1 --2- 172.21.15.43:0/1254827831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b90076720 0x7f2b90076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.518+0000 7f2b8f200640 1 -- 172.21.15.43:0/1254827831 >> 172.21.15.43:0/1254827831 conn(0x7f2b900ff910 msgr2=0x7f2b90101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:24.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.519+0000 7f2b8f200640 1 -- 172.21.15.43:0/1254827831 shutdown_connections 2024-09-19T05:07:24.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.519+0000 7f2b8f200640 1 -- 172.21.15.43:0/1254827831 wait complete. 2024-09-19T05:07:24.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.522+0000 7f2b8f200640 1 Processor -- start 2024-09-19T05:07:24.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.523+0000 7f2b8f200640 1 -- start start 2024-09-19T05:07:24.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.524+0000 7f2b8f200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b90076720 0x7f2b901a4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:24.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.524+0000 7f2b8f200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b90077040 0x7f2b901a4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:24.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.524+0000 7f2b8f200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b90112870 0x7f2b9019e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:24.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.525+0000 7f2b8f200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2b90118440 con 0x7f2b90112870 2024-09-19T05:07:24.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.525+0000 7f2b8f200640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2b901182c0 con 0x7f2b90077040 2024-09-19T05:07:24.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.525+0000 7f2b8f200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2b901185c0 con 0x7f2b90076720 2024-09-19T05:07:24.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.524+0000 7f2b8d400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b90077040 0x7f2b901a4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:24.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.525+0000 7f2b8d400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b90077040 0x7f2b901a4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:50342/0 (socket says 172.21.15.43:50342) 2024-09-19T05:07:24.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.525+0000 7f2b8d400640 1 -- 172.21.15.43:0/2165560674 learned_addr learned my addr 172.21.15.43:0/2165560674 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:24.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.524+0000 7f2b8de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b90076720 0x7f2b901a4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:24.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.525+0000 7f2b8e800640 1 --2- 172.21.15.43:0/2165560674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b90112870 0x7f2b9019e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:24.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.526+0000 7f2b8d400640 1 -- 172.21.15.43:0/2165560674 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b90076720 msgr2=0x7f2b901a4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:24.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.526+0000 7f2b8d400640 1 --2- 172.21.15.43:0/2165560674 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b90076720 0x7f2b901a4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.526+0000 7f2b8d400640 1 -- 172.21.15.43:0/2165560674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b90112870 msgr2=0x7f2b9019e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:24.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.526+0000 7f2b8d400640 1 --2- 172.21.15.43:0/2165560674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b90112870 0x7f2b9019e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.526+0000 7f2b8d400640 1 -- 172.21.15.43:0/2165560674 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b9019eea0 con 0x7f2b90077040 2024-09-19T05:07:24.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.527+0000 7f2b8d400640 1 --2- 172.21.15.43:0/2165560674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b90077040 0x7f2b901a4aa0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c00e9f0 tx=0x7f2b7c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:24.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.527+0000 7f2b86a00640 1 -- 172.21.15.43:0/2165560674 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c00ccc0 con 0x7f2b90077040 2024-09-19T05:07:24.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.527+0000 7f2b8f200640 1 -- 172.21.15.43:0/2165560674 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b9019f190 con 0x7f2b90077040 2024-09-19T05:07:24.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.527+0000 7f2b8f200640 1 -- 172.21.15.43:0/2165560674 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b901ab4e0 con 0x7f2b90077040 2024-09-19T05:07:24.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.528+0000 7f2b86a00640 1 -- 172.21.15.43:0/2165560674 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b7c010910 con 0x7f2b90077040 2024-09-19T05:07:24.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.528+0000 7f2b86a00640 1 -- 172.21.15.43:0/2165560674 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c0188a0 con 0x7f2b90077040 2024-09-19T05:07:24.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.532+0000 7f2b86a00640 1 -- 172.21.15.43:0/2165560674 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f2b7c010ac0 con 0x7f2b90077040 2024-09-19T05:07:24.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.536+0000 7f2b97489740 1 Processor -- start 2024-09-19T05:07:24.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.537+0000 7f2b97489740 1 -- start start 2024-09-19T05:07:24.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.538+0000 7f2b86a00640 1 --2- 172.21.15.43:0/2165560674 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f2b740a3500 0x7f2b740a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:24.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.539+0000 7f2b97489740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf8b52d50 0x564bf8b53130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:24.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.539+0000 7f2b97489740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bf8b53670 0x564bf8b5d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:24.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.539+0000 7f2b97489740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf8b5df30 0x564bf8b60320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:24.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.539+0000 7f2b97489740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564bf8ab93e0 con 0x564bf8b53670 2024-09-19T05:07:24.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.539+0000 7f2b97489740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564bf8a60d80 con 0x564bf8b52d50 2024-09-19T05:07:24.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.539+0000 7f2b97489740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564bf8ab53a0 con 0x564bf8b5df30 2024-09-19T05:07:24.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.541+0000 7f2b8de00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf8b52d50 0x564bf8b53130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:24.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.541+0000 7f2b86a00640 1 -- 172.21.15.43:0/2165560674 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f2b7c0d0040 con 0x7f2b90077040 2024-09-19T05:07:24.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.541+0000 7f2b8e800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf8b5df30 0x564bf8b60320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:24.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.541+0000 7f2b8e800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf8b5df30 0x564bf8b60320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:40294/0 (socket says 172.21.15.43:40294) 2024-09-19T05:07:24.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.541+0000 7f2b8e800640 1 -- 172.21.15.43:0/4012793303 learned_addr learned my addr 172.21.15.43:0/4012793303 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:24.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.541+0000 7f2b8e800640 1 --2- 172.21.15.43:0/2165560674 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f2b740a3500 0x7f2b740a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:24.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.542+0000 7f2b8d400640 1 --2- 172.21.15.43:0/4012793303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bf8b53670 0x564bf8b5d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:24.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.544+0000 7f2b8de00640 1 -- 172.21.15.43:0/4012793303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf8b5df30 msgr2=0x564bf8b60320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:24.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.544+0000 7f2b8de00640 1 --2- 172.21.15.43:0/4012793303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf8b5df30 0x564bf8b60320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.544+0000 7f2b8de00640 1 -- 172.21.15.43:0/4012793303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bf8b53670 msgr2=0x564bf8b5d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:24.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.544+0000 7f2b8de00640 1 --2- 172.21.15.43:0/4012793303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bf8b53670 0x564bf8b5d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.544+0000 7f2b8de00640 1 -- 172.21.15.43:0/4012793303 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bf8a7fae0 con 0x564bf8b52d50 2024-09-19T05:07:24.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.547+0000 7f2b8e800640 1 --2- 172.21.15.43:0/2165560674 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f2b740a3500 0x7f2b740a59c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x564bf8b50660 tx=0x7f2b8802a8d0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:24.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.547+0000 7f2b8de00640 1 --2- 172.21.15.43:0/4012793303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf8b52d50 0x564bf8b53130 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2b8000abf0 tx=0x7f2b8000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d057b65b1bb3ce8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:24.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.555+0000 7f2b7b000640 1 -- 172.21.15.43:0/4012793303 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b80005220 con 0x564bf8b52d50 2024-09-19T05:07:24.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.555+0000 7f2b7b000640 1 -- 172.21.15.43:0/4012793303 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b80004c80 con 0x564bf8b52d50 2024-09-19T05:07:24.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.556+0000 7f2b7b000640 1 -- 172.21.15.43:0/4012793303 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b80018940 con 0x564bf8b52d50 2024-09-19T05:07:24.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.562+0000 7f2b97489740 1 -- 172.21.15.43:0/4012793303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf8b52d50 msgr2=0x564bf8b53130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:24.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.562+0000 7f2b97489740 1 --2- 172.21.15.43:0/4012793303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf8b52d50 0x564bf8b53130 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2b8000abf0 tx=0x7f2b8000f790 comp rx=0 tx=0).stop 2024-09-19T05:07:24.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.566+0000 7f2b97489740 1 -- 172.21.15.43:0/4012793303 shutdown_connections 2024-09-19T05:07:24.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.566+0000 7f2b97489740 1 --2- 172.21.15.43:0/4012793303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf8b5df30 0x564bf8b60320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.566+0000 7f2b97489740 1 --2- 172.21.15.43:0/4012793303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bf8b53670 0x564bf8b5d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.566+0000 7f2b97489740 1 --2- 172.21.15.43:0/4012793303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf8b52d50 0x564bf8b53130 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.566+0000 7f2b97489740 1 -- 172.21.15.43:0/4012793303 >> 172.21.15.43:0/4012793303 conn(0x564bf8b4f2e0 msgr2=0x564bf8b4f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:24.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.567+0000 7f2b97489740 1 -- 172.21.15.43:0/4012793303 shutdown_connections 2024-09-19T05:07:24.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.567+0000 7f2b97489740 1 -- 172.21.15.43:0/4012793303 wait complete. 2024-09-19T05:07:24.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.568+0000 7f2b97489740 1 -- start start 2024-09-19T05:07:24.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.568+0000 7f2b97489740 1 Processor -- start 2024-09-19T05:07:24.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.576+0000 7f2b97489740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bf8b52d50 0x564bf8b5ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:24.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.576+0000 7f2b97489740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf8b53670 0x564bf8b5b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:24.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.578+0000 7f2b97489740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf8b5df30 0x564bf8b5b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:24.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.578+0000 7f2b97489740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564bf8b60ba0 con 0x564bf8b52d50 2024-09-19T05:07:24.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.578+0000 7f2b97489740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564bf8a740f0 con 0x564bf8b53670 2024-09-19T05:07:24.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.578+0000 7f2b97489740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564bf8b60d20 con 0x564bf8b5df30 2024-09-19T05:07:24.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.578+0000 7f2b97489740 10 client.0 _ll_register_callbacks cb 0x564bf8a659e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:24.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.580+0000 7f2b8d400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf8b53670 0x564bf8b5b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:24.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.580+0000 7f2b8d400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf8b53670 0x564bf8b5b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:50368/0 (socket says 172.21.15.43:50368) 2024-09-19T05:07:24.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.580+0000 7f2b8d400640 1 -- 172.21.15.43:0/604101676 learned_addr learned my addr 172.21.15.43:0/604101676 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:24.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.581+0000 7f2b8e800640 1 --2- 172.21.15.43:0/604101676 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf8b5df30 0x564bf8b5b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:24.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.581+0000 7f2b8d400640 1 -- 172.21.15.43:0/604101676 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf8b5df30 msgr2=0x564bf8b5b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:24.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.581+0000 7f2b8d400640 1 --2- 172.21.15.43:0/604101676 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf8b5df30 0x564bf8b5b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.581+0000 7f2b8d400640 1 -- 172.21.15.43:0/604101676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bf8b52d50 msgr2=0x564bf8b5ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:24.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.581+0000 7f2b8d400640 1 --2- 172.21.15.43:0/604101676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bf8b52d50 0x564bf8b5ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.581+0000 7f2b8d400640 1 -- 172.21.15.43:0/604101676 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bf882a3c0 con 0x564bf8b53670 2024-09-19T05:07:24.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.585+0000 7f2b8d400640 1 --2- 172.21.15.43:0/604101676 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf8b53670 0x564bf8b5b380 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c09c260 tx=0x7f2b7c010e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3007c77c5d55140e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:24.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.585+0000 7f2b6f400640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:07:24.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.585+0000 7f2b6f400640 1 -- 172.21.15.43:0/604101676 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c0a0680 con 0x564bf8b53670 2024-09-19T05:07:24.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.586+0000 7f2b6f400640 1 -- 172.21.15.43:0/604101676 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f2b7c0cd9c0 con 0x564bf8b53670 2024-09-19T05:07:24.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.586+0000 7f2b97489740 1 -- 172.21.15.43:0/604101676 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b80004c80 con 0x564bf8b53670 2024-09-19T05:07:24.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.586+0000 7f2b6f400640 1 -- 172.21.15.43:0/604101676 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2b7c0cdbc0 con 0x564bf8b53670 2024-09-19T05:07:24.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.587+0000 7f2b97489740 10 client.25754 fetch_fsmap learned FSMap version 29 2024-09-19T05:07:24.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.587+0000 7f2b97489740 1 -- 172.21.15.43:0/604101676 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564bf8b6c8e0 con 0x564bf8b53670 2024-09-19T05:07:24.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.587+0000 7f2b6f400640 1 -- 172.21.15.43:0/604101676 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b80004c80 con 0x564bf8b53670 2024-09-19T05:07:24.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.587+0000 7f2b6f400640 1 -- 172.21.15.43:0/604101676 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2b7c09b840 con 0x564bf8b53670 2024-09-19T05:07:24.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.587+0000 7f2b97489740 10 client.25754 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:07:24.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.587+0000 7f2b97489740 10 client.25754 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:07:24.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.587+0000 7f2b97489740 20 client.25754 populate_metadata read hostname 'smithi043' 2024-09-19T05:07:24.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.588+0000 7f2b97489740 1 --2- 172.21.15.43:0/604101676 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x564bf8b6dd90 0x564bf8b701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:24.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.588+0000 7f2b97489740 4 client.25754 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:07:24.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.588+0000 7f2b97489740 1 -- 172.21.15.43:0/604101676 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bf8ac6950 con 0x564bf8b6dd90 2024-09-19T05:07:24.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.588+0000 7f2b8de00640 1 --2- 172.21.15.43:0/604101676 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x564bf8b6dd90 0x564bf8b701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:24.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.589+0000 7f2b6f400640 10 client.25754 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:07:24.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.591+0000 7f2b8de00640 1 --2- 172.21.15.43:0/604101676 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x564bf8b6dd90 0x564bf8b701c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84c1b97201e744ac server_cookie=f3f956131d2266e2 in_seq=0 out_seq=0 2024-09-19T05:07:24.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.592+0000 7f2b6f400640 1 -- 172.21.15.43:0/604101676 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564bf8ac6950 con 0x564bf8b6dd90 2024-09-19T05:07:24.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.592+0000 7f2b6f400640 10 client.25754 handle_command_reply: tid=0 2024-09-19T05:07:24.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.593+0000 7f2b97489740 1 client.25754 shutdown 2024-09-19T05:07:24.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.594+0000 7f2b97489740 1 -- 172.21.15.43:0/604101676 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf8b53670 msgr2=0x564bf8b5b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:24.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.594+0000 7f2b97489740 1 --2- 172.21.15.43:0/604101676 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf8b53670 0x564bf8b5b380 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c09c260 tx=0x7f2b7c010e70 comp rx=0 tx=0).stop 2024-09-19T05:07:24.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.594+0000 7f2b97489740 1 -- 172.21.15.43:0/604101676 shutdown_connections 2024-09-19T05:07:24.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.594+0000 7f2b97489740 1 --2- 172.21.15.43:0/604101676 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x564bf8b6dd90 0x564bf8b701c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.594+0000 7f2b97489740 1 --2- 172.21.15.43:0/604101676 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf8b5df30 0x564bf8b5b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.594+0000 7f2b97489740 1 --2- 172.21.15.43:0/604101676 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf8b53670 0x564bf8b5b380 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.594+0000 7f2b97489740 1 --2- 172.21.15.43:0/604101676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bf8b52d50 0x564bf8b5ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.595+0000 7f2b6f400640 0 client.25754 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:07:24.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.595+0000 7f2b97489740 1 -- 172.21.15.43:0/604101676 >> 172.21.15.43:0/604101676 conn(0x564bf8b4f2e0 msgr2=0x564bf8b55010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:24.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.596+0000 7f2b97489740 1 -- 172.21.15.43:0/604101676 shutdown_connections 2024-09-19T05:07:24.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.597+0000 7f2b97489740 1 -- 172.21.15.43:0/604101676 wait complete. 2024-09-19T05:07:24.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.597+0000 7f2b97489740 20 client.25754 trim_cache size 0 max 16384 2024-09-19T05:07:24.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.636+0000 7f2b97489740 1 Processor -- start 2024-09-19T05:07:24.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.636+0000 7f2b97489740 1 -- start start 2024-09-19T05:07:24.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.637+0000 7f2b97489740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bf8b52d50 0x564bf8b557b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:24.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.637+0000 7f2b97489740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf8b53670 0x564bf8b5b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:24.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.637+0000 7f2b97489740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf8b5df30 0x564bf8b75ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:24.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.637+0000 7f2b97489740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7000e700 con 0x564bf8b52d50 2024-09-19T05:07:24.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.637+0000 7f2b97489740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2b70010240 con 0x564bf8b5df30 2024-09-19T05:07:24.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.637+0000 7f2b97489740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7000edd0 con 0x564bf8b53670 2024-09-19T05:07:24.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.638+0000 7f2b8e800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf8b5df30 0x564bf8b75ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:24.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.638+0000 7f2b8d400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf8b53670 0x564bf8b5b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:24.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.638+0000 7f2b8e800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf8b5df30 0x564bf8b75ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:50384/0 (socket says 172.21.15.43:50384) 2024-09-19T05:07:24.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.639+0000 7f2b8e800640 1 -- 172.21.15.43:0/522487751 learned_addr learned my addr 172.21.15.43:0/522487751 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:24.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.639+0000 7f2b8de00640 1 --2- 172.21.15.43:0/522487751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bf8b52d50 0x564bf8b557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:24.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.639+0000 7f2b8e800640 1 -- 172.21.15.43:0/522487751 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf8b53670 msgr2=0x564bf8b5b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:24.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.639+0000 7f2b8e800640 1 --2- 172.21.15.43:0/522487751 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf8b53670 0x564bf8b5b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.639+0000 7f2b8e800640 1 -- 172.21.15.43:0/522487751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bf8b52d50 msgr2=0x564bf8b557b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:24.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.639+0000 7f2b8e800640 1 --2- 172.21.15.43:0/522487751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bf8b52d50 0x564bf8b557b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.639+0000 7f2b8e800640 1 -- 172.21.15.43:0/522487751 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bf8b59390 con 0x564bf8b5df30 2024-09-19T05:07:24.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.639+0000 7f2b8d400640 1 --2- 172.21.15.43:0/522487751 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf8b53670 0x564bf8b5b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:07:24.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.640+0000 7f2b8de00640 1 --2- 172.21.15.43:0/522487751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bf8b52d50 0x564bf8b557b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:07:24.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.640+0000 7f2b8e800640 1 --2- 172.21.15.43:0/522487751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf8b5df30 0x564bf8b75ad0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2b88035000 tx=0x7f2b88043000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9dcbaa67d514e777 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:24.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.641+0000 7f2b7a600640 1 -- 172.21.15.43:0/522487751 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b88003a30 con 0x564bf8b5df30 2024-09-19T05:07:24.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.641+0000 7f2b7a600640 1 -- 172.21.15.43:0/522487751 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b88051030 con 0x564bf8b5df30 2024-09-19T05:07:24.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.641+0000 7f2b7a600640 1 -- 172.21.15.43:0/522487751 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b88004460 con 0x564bf8b5df30 2024-09-19T05:07:24.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.643+0000 7f2b97489740 1 -- 172.21.15.43:0/522487751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf8b5df30 msgr2=0x564bf8b75ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:24.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.643+0000 7f2b97489740 1 --2- 172.21.15.43:0/522487751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf8b5df30 0x564bf8b75ad0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2b88035000 tx=0x7f2b88043000 comp rx=0 tx=0).stop 2024-09-19T05:07:24.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.643+0000 7f2b97489740 1 -- 172.21.15.43:0/522487751 shutdown_connections 2024-09-19T05:07:24.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.643+0000 7f2b97489740 1 --2- 172.21.15.43:0/522487751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf8b5df30 0x564bf8b75ad0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.643+0000 7f2b97489740 1 --2- 172.21.15.43:0/522487751 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf8b53670 0x564bf8b5b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.643+0000 7f2b97489740 1 --2- 172.21.15.43:0/522487751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bf8b52d50 0x564bf8b557b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.643+0000 7f2b97489740 1 -- 172.21.15.43:0/522487751 >> 172.21.15.43:0/522487751 conn(0x564bf8b4f2e0 msgr2=0x564bf8b4e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:24.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.644+0000 7f2b97489740 1 -- 172.21.15.43:0/522487751 shutdown_connections 2024-09-19T05:07:24.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.644+0000 7f2b97489740 1 -- 172.21.15.43:0/522487751 wait complete. 2024-09-19T05:07:24.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.646+0000 7f2b97489740 1 -- start start 2024-09-19T05:07:24.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.648+0000 7f2b97489740 1 Processor -- start 2024-09-19T05:07:24.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.649+0000 7f2b97489740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf8b52d50 0x564bf8bd4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:24.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.649+0000 7f2b97489740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf8b53670 0x564bf8b65fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:24.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.649+0000 7f2b97489740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bf8b5df30 0x564bf8b665c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:24.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.649+0000 7f2b97489740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7000fb70 con 0x564bf8b5df30 2024-09-19T05:07:24.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.649+0000 7f2b97489740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7000f4a0 con 0x564bf8b53670 2024-09-19T05:07:24.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.649+0000 7f2b97489740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7000c6e0 con 0x564bf8b52d50 2024-09-19T05:07:24.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.649+0000 7f2b97489740 10 client.0 _ll_register_callbacks cb 0x7f2b7c09bc30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:24.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.651+0000 7f2b8de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf8b52d50 0x564bf8bd4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:24.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.651+0000 7f2b8e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bf8b5df30 0x564bf8b665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:24.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.651+0000 7f2b8e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bf8b5df30 0x564bf8b665c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56128/0 (socket says 172.21.15.43:56128) 2024-09-19T05:07:24.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.651+0000 7f2b8e800640 1 -- 172.21.15.43:0/4088124608 learned_addr learned my addr 172.21.15.43:0/4088124608 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:24.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.651+0000 7f2b8d400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf8b53670 0x564bf8b65fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:24.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.652+0000 7f2b8e800640 1 -- 172.21.15.43:0/4088124608 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf8b52d50 msgr2=0x564bf8bd4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:24.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.652+0000 7f2b8e800640 1 --2- 172.21.15.43:0/4088124608 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf8b52d50 0x564bf8bd4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.652+0000 7f2b8e800640 1 -- 172.21.15.43:0/4088124608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf8b53670 msgr2=0x564bf8b65fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:24.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.652+0000 7f2b8e800640 1 --2- 172.21.15.43:0/4088124608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf8b53670 0x564bf8b65fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.652+0000 7f2b8e800640 1 -- 172.21.15.43:0/4088124608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bf8bd17d0 con 0x564bf8b5df30 2024-09-19T05:07:24.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.653+0000 7f2b8d400640 1 --2- 172.21.15.43:0/4088124608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf8b53670 0x564bf8b65fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:07:24.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.653+0000 7f2b8e800640 1 --2- 172.21.15.43:0/4088124608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bf8b5df30 0x564bf8b665c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2b88006010 tx=0x7f2b88003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7598f33b42af8292 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:24.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.654+0000 7f2b6f400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:07:24.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.654+0000 7f2b6f400640 1 -- 172.21.15.43:0/4088124608 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b880317b0 con 0x564bf8b5df30 2024-09-19T05:07:24.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.655+0000 7f2b97489740 1 -- 172.21.15.43:0/4088124608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b88051030 con 0x564bf8b5df30 2024-09-19T05:07:24.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.655+0000 7f2b6f400640 1 -- 172.21.15.43:0/4088124608 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f2b88047040 con 0x564bf8b5df30 2024-09-19T05:07:24.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.655+0000 7f2b6f400640 1 -- 172.21.15.43:0/4088124608 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2b880317b0 con 0x564bf8b5df30 2024-09-19T05:07:24.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.656+0000 7f2b97489740 10 client.15747 fetch_fsmap learned FSMap version 29 2024-09-19T05:07:24.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.656+0000 7f2b97489740 1 -- 172.21.15.43:0/4088124608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564bf8be9cf0 con 0x564bf8b5df30 2024-09-19T05:07:24.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.656+0000 7f2b6f400640 1 -- 172.21.15.43:0/4088124608 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b88051030 con 0x564bf8b5df30 2024-09-19T05:07:24.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.657+0000 7f2b6f400640 1 -- 172.21.15.43:0/4088124608 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2b8804a030 con 0x564bf8b5df30 2024-09-19T05:07:24.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.657+0000 7f2b97489740 10 client.15747 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:07:24.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.657+0000 7f2b97489740 10 client.15747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:07:24.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.657+0000 7f2b97489740 20 client.15747 populate_metadata read hostname 'smithi043' 2024-09-19T05:07:24.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.657+0000 7f2b97489740 1 --2- 172.21.15.43:0/4088124608 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x564bf8b6dd90 0x564bf8bec370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:24.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.658+0000 7f2b97489740 4 client.15747 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:07:24.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.658+0000 7f2b97489740 1 -- 172.21.15.43:0/4088124608 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564bf8ac0e60 con 0x564bf8b6dd90 2024-09-19T05:07:24.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.659+0000 7f2b8de00640 1 --2- 172.21.15.43:0/4088124608 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x564bf8b6dd90 0x564bf8bec370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:24.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.659+0000 7f2b8de00640 1 --2- 172.21.15.43:0/4088124608 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x564bf8b6dd90 0x564bf8bec370 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf196f56415643b7 server_cookie=80f094586178882 in_seq=0 out_seq=0 2024-09-19T05:07:24.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.659+0000 7f2b6f400640 10 client.15747 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:07:24.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.659+0000 7f2b6f400640 1 -- 172.21.15.43:0/4088124608 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564bf8ac0e60 con 0x564bf8b6dd90 2024-09-19T05:07:24.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.659+0000 7f2b6f400640 10 client.15747 handle_command_reply: tid=0 2024-09-19T05:07:24.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.659+0000 7f2b97489740 1 client.15747 shutdown 2024-09-19T05:07:24.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.660+0000 7f2b97489740 1 -- 172.21.15.43:0/4088124608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bf8b5df30 msgr2=0x564bf8b665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:24.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.660+0000 7f2b97489740 1 --2- 172.21.15.43:0/4088124608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bf8b5df30 0x564bf8b665c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2b88006010 tx=0x7f2b88003dc0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.661+0000 7f2b97489740 1 -- 172.21.15.43:0/4088124608 shutdown_connections 2024-09-19T05:07:24.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.661+0000 7f2b97489740 1 --2- 172.21.15.43:0/4088124608 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x564bf8b6dd90 0x564bf8bec370 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.661+0000 7f2b97489740 1 --2- 172.21.15.43:0/4088124608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564bf8b5df30 0x564bf8b665c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.661+0000 7f2b97489740 1 --2- 172.21.15.43:0/4088124608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf8b53670 0x564bf8b65fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.661+0000 7f2b97489740 1 --2- 172.21.15.43:0/4088124608 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf8b52d50 0x564bf8bd4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.661+0000 7f2b97489740 1 -- 172.21.15.43:0/4088124608 >> 172.21.15.43:0/4088124608 conn(0x564bf8b4f2e0 msgr2=0x564bf8b4db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:24.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.661+0000 7f2b6f400640 0 client.15747 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:07:24.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.662+0000 7f2b97489740 1 -- 172.21.15.43:0/4088124608 shutdown_connections 2024-09-19T05:07:24.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.662+0000 7f2b97489740 1 -- 172.21.15.43:0/4088124608 wait complete. 2024-09-19T05:07:24.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.662+0000 7f2b97489740 20 client.15747 trim_cache size 0 max 16384 2024-09-19T05:07:24.664 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:07:24.664 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-09-19T05:07:24.664 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-09-19T05:07:24.664 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:07:24.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.663+0000 7f2b8f200640 1 -- 172.21.15.43:0/2165560674 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f2b740a3500 msgr2=0x7f2b740a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:24.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.663+0000 7f2b8f200640 1 --2- 172.21.15.43:0/2165560674 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f2b740a3500 0x7f2b740a59c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x564bf8b50660 tx=0x7f2b8802a8d0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.663+0000 7f2b8f200640 1 -- 172.21.15.43:0/2165560674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b90077040 msgr2=0x7f2b901a4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:24.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.663+0000 7f2b8f200640 1 --2- 172.21.15.43:0/2165560674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b90077040 0x7f2b901a4aa0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c00e9f0 tx=0x7f2b7c00edf0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.663+0000 7f2b8f200640 1 -- 172.21.15.43:0/2165560674 shutdown_connections 2024-09-19T05:07:24.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.663+0000 7f2b8f200640 1 --2- 172.21.15.43:0/2165560674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b90112870 0x7f2b9019e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.663+0000 7f2b8f200640 1 --2- 172.21.15.43:0/2165560674 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f2b740a3500 0x7f2b740a59c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.663+0000 7f2b8f200640 1 --2- 172.21.15.43:0/2165560674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b90077040 0x7f2b901a4aa0 secure :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c00e9f0 tx=0x7f2b7c00edf0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.663+0000 7f2b8f200640 1 --2- 172.21.15.43:0/2165560674 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b90076720 0x7f2b901a4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:24.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.663+0000 7f2b8f200640 1 -- 172.21.15.43:0/2165560674 >> 172.21.15.43:0/2165560674 conn(0x7f2b900ff910 msgr2=0x7f2b90101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:24.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.663+0000 7f2b8f200640 1 -- 172.21.15.43:0/2165560674 shutdown_connections 2024-09-19T05:07:24.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:24.664+0000 7f2b8f200640 1 -- 172.21.15.43:0/2165560674 wait complete. 2024-09-19T05:07:25.956 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T05:07:25.956 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:07:26.104 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:07:26 smithi043 ceph-mon[30800]: pgmap v86: 97 pgs: 97 active+clean; 9.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 428 MiB/s wr, 121 op/s 2024-09-19T05:07:26.346 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-19T05:07:26.346 INFO:tasks.workunit.client.0.smithi043.stdout: Children see throughput for 1 initial writers = 301437.66 kB/sec 2024-09-19T05:07:26.347 INFO:tasks.workunit.client.0.smithi043.stdout: Parent sees throughput for 1 initial writers = 301428.72 kB/sec 2024-09-19T05:07:26.347 INFO:tasks.workunit.client.0.smithi043.stdout: Min throughput per process = 301437.66 kB/sec 2024-09-19T05:07:26.347 INFO:tasks.workunit.client.0.smithi043.stdout: Max throughput per process = 301437.66 kB/sec 2024-09-19T05:07:26.347 INFO:tasks.workunit.client.0.smithi043.stdout: Avg throughput per process = 301437.66 kB/sec 2024-09-19T05:07:26.347 INFO:tasks.workunit.client.0.smithi043.stdout: Min xfer = 10485760.00 kB 2024-09-19T05:07:26.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:07:26 smithi151 ceph-mon[35425]: pgmap v86: 97 pgs: 97 active+clean; 9.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 428 MiB/s wr, 121 op/s 2024-09-19T05:07:26.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:07:26 smithi096 ceph-mon[35085]: pgmap v86: 97 pgs: 97 active+clean; 9.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 428 MiB/s wr, 121 op/s 2024-09-19T05:07:26.524 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:07:27.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.740+0000 7f7426600640 1 -- 172.21.15.43:0/1254923563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7420102360 msgr2=0x7f742010e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:27.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.740+0000 7f7426600640 1 --2- 172.21.15.43:0/1254923563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7420102360 0x7f742010e890 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7418009be0 tx=0x7f741802a6c0 comp rx=0 tx=0).stop 2024-09-19T05:07:27.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.741+0000 7f7426600640 1 -- 172.21.15.43:0/1254923563 shutdown_connections 2024-09-19T05:07:27.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.741+0000 7f7426600640 1 --2- 172.21.15.43:0/1254923563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7420102360 0x7f742010e890 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:27.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.741+0000 7f7426600640 1 --2- 172.21.15.43:0/1254923563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74201019c0 0x7f7420101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:27.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.741+0000 7f7426600640 1 --2- 172.21.15.43:0/1254923563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74201079c0 0x7f7420107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:27.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.741+0000 7f7426600640 1 -- 172.21.15.43:0/1254923563 >> 172.21.15.43:0/1254923563 conn(0x7f74200fd740 msgr2=0x7f74200ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:27.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.741+0000 7f7426600640 1 -- 172.21.15.43:0/1254923563 shutdown_connections 2024-09-19T05:07:27.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.741+0000 7f7426600640 1 -- 172.21.15.43:0/1254923563 wait complete. 2024-09-19T05:07:27.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.742+0000 7f7426600640 1 Processor -- start 2024-09-19T05:07:27.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.742+0000 7f7426600640 1 -- start start 2024-09-19T05:07:27.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.742+0000 7f7426600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74201019c0 0x7f742019a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:27.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.742+0000 7f7426600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7420102360 0x7f742019a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:27.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.742+0000 7f7426600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74201079c0 0x7f742019ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:27.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.742+0000 7f7426600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7420111d70 con 0x7f7420102360 2024-09-19T05:07:27.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.742+0000 7f7426600640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7420111bf0 con 0x7f74201079c0 2024-09-19T05:07:27.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.742+0000 7f7426600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7420111ef0 con 0x7f74201019c0 2024-09-19T05:07:27.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.743+0000 7f7425c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74201079c0 0x7f742019ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:27.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.743+0000 7f741fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7420102360 0x7f742019a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:27.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.743+0000 7f741fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7420102360 0x7f742019a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60854/0 (socket says 172.21.15.43:60854) 2024-09-19T05:07:27.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.743+0000 7f741fe00640 1 -- 172.21.15.43:0/3949427592 learned_addr learned my addr 172.21.15.43:0/3949427592 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:27.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.743+0000 7f7425200640 1 --2- 172.21.15.43:0/3949427592 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74201019c0 0x7f742019a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:27.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.744+0000 7f741fe00640 1 -- 172.21.15.43:0/3949427592 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74201019c0 msgr2=0x7f742019a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:27.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.744+0000 7f741fe00640 1 --2- 172.21.15.43:0/3949427592 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74201019c0 0x7f742019a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:27.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.744+0000 7f741fe00640 1 -- 172.21.15.43:0/3949427592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74201079c0 msgr2=0x7f742019ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:27.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.744+0000 7f741fe00640 1 --2- 172.21.15.43:0/3949427592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74201079c0 0x7f742019ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:27.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.744+0000 7f741fe00640 1 -- 172.21.15.43:0/3949427592 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f742019f3c0 con 0x7f7420102360 2024-09-19T05:07:27.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.744+0000 7f7425c00640 1 --2- 172.21.15.43:0/3949427592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74201079c0 0x7f742019ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:07:27.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.744+0000 7f741fe00640 1 --2- 172.21.15.43:0/3949427592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7420102360 0x7f742019a950 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f740c00e9c0 tx=0x7f740c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:27.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.744+0000 7f741d600640 1 -- 172.21.15.43:0/3949427592 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740c00ccc0 con 0x7f7420102360 2024-09-19T05:07:27.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.745+0000 7f741d600640 1 -- 172.21.15.43:0/3949427592 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f740c005500 con 0x7f7420102360 2024-09-19T05:07:27.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.745+0000 7f741d600640 1 -- 172.21.15.43:0/3949427592 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740c010430 con 0x7f7420102360 2024-09-19T05:07:27.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.745+0000 7f7426600640 1 -- 172.21.15.43:0/3949427592 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f742019f6b0 con 0x7f7420102360 2024-09-19T05:07:27.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.745+0000 7f7426600640 1 -- 172.21.15.43:0/3949427592 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74201a6f90 con 0x7f7420102360 2024-09-19T05:07:27.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.746+0000 7f7426600640 1 -- 172.21.15.43:0/3949427592 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7420101e20 con 0x7f7420102360 2024-09-19T05:07:27.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.748+0000 7f741d600640 1 -- 172.21.15.43:0/3949427592 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f740c0046c0 con 0x7f7420102360 2024-09-19T05:07:27.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.749+0000 7f741d600640 1 --2- 172.21.15.43:0/3949427592 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f74080a33b0 0x7f74080a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:27.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.749+0000 7f741d600640 1 -- 172.21.15.43:0/3949427592 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f740c0d00f0 con 0x7f7420102360 2024-09-19T05:07:27.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.750+0000 7f7425200640 1 --2- 172.21.15.43:0/3949427592 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f74080a33b0 0x7f74080a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:27.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.750+0000 7f7425200640 1 --2- 172.21.15.43:0/3949427592 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f74080a33b0 0x7f74080a5870 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7414009ae0 tx=0x7f7414005f90 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:27.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.754+0000 7f741d600640 1 -- 172.21.15.43:0/3949427592 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f740c09ca80 con 0x7f7420102360 2024-09-19T05:07:27.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.979+0000 7f7426600640 1 -- 172.21.15.43:0/3949427592 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74201a7280 con 0x7f7420102360 2024-09-19T05:07:27.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.980+0000 7f741d600640 1 -- 172.21.15.43:0/3949427592 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11965 (secure 0 0 0) 0x7f740c09c380 con 0x7f7420102360 2024-09-19T05:07:27.982 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:07:27.982 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":29,"btime":"2024-09-19T05:04:38:785407+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/773801169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":773801169},{"type":"v1","addr":"172.21.15.43:6835","nonce":773801169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1799137902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1799137902},{"type":"v1","addr":"172.21.15.43:6837","nonce":1799137902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:02:25.463041+0000","modified":"2024-09-19T05:04:38.039809+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24518,"mds_2":24511,"mds_3":24506,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14541":{"gid":14541,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/794881135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":794881135},{"type":"v1","addr":"172.21.15.43:6841","nonce":794881135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/2836453158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2836453158},{"type":"v1","addr":"172.21.15.151:6833","nonce":2836453158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/178785812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":178785812},{"type":"v1","addr":"172.21.15.96:6833","nonce":178785812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1671043820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1671043820},{"type":"v1","addr":"172.21.15.96:6835","nonce":1671043820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2972514328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2972514328},{"type":"v1","addr":"172.21.15.151:6835","nonce":2972514328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.151:6837/3960378010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3960378010},{"type":"v1","addr":"172.21.15.151:6837","nonce":3960378010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6837/3978388286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3978388286},{"type":"v1","addr":"172.21.15.96:6837","nonce":3978388286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.151:6839/3794104313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3794104313},{"type":"v1","addr":"172.21.15.151:6839","nonce":3794104313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.43:6839/3404555030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3404555030},{"type":"v1","addr":"172.21.15.43:6839","nonce":3404555030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/605527621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":605527621},{"type":"v1","addr":"172.21.15.96:6839","nonce":605527621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24518,24511,24506,24493]},"id":1}]} 2024-09-19T05:07:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 29 2024-09-19T05:07:27.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.983+0000 7f7426600640 1 -- 172.21.15.43:0/3949427592 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f74080a33b0 msgr2=0x7f74080a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:27.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.983+0000 7f7426600640 1 --2- 172.21.15.43:0/3949427592 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f74080a33b0 0x7f74080a5870 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7414009ae0 tx=0x7f7414005f90 comp rx=0 tx=0).stop 2024-09-19T05:07:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.983+0000 7f7426600640 1 -- 172.21.15.43:0/3949427592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7420102360 msgr2=0x7f742019a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.983+0000 7f7426600640 1 --2- 172.21.15.43:0/3949427592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7420102360 0x7f742019a950 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f740c00e9c0 tx=0x7f740c00ee80 comp rx=0 tx=0).stop 2024-09-19T05:07:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.983+0000 7f7426600640 1 -- 172.21.15.43:0/3949427592 shutdown_connections 2024-09-19T05:07:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.983+0000 7f7426600640 1 --2- 172.21.15.43:0/3949427592 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f74080a33b0 0x7f74080a5870 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.983+0000 7f7426600640 1 --2- 172.21.15.43:0/3949427592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74201079c0 0x7f742019ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.983+0000 7f7426600640 1 --2- 172.21.15.43:0/3949427592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7420102360 0x7f742019a950 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:27.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.983+0000 7f7426600640 1 --2- 172.21.15.43:0/3949427592 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74201019c0 0x7f742019a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:27.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.983+0000 7f7426600640 1 -- 172.21.15.43:0/3949427592 >> 172.21.15.43:0/3949427592 conn(0x7f74200fd740 msgr2=0x7f742010d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:27.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.984+0000 7f7426600640 1 -- 172.21.15.43:0/3949427592 shutdown_connections 2024-09-19T05:07:27.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:27.984+0000 7f7426600640 1 -- 172.21.15.43:0/3949427592 wait complete. 2024-09-19T05:07:28.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:07:27 smithi043 ceph-mon[30800]: pgmap v87: 97 pgs: 97 active+clean; 9.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 296 MiB/s wr, 83 op/s 2024-09-19T05:07:28.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:07:27 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/3949427592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:07:28.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:07:27 smithi151 ceph-mon[35425]: pgmap v87: 97 pgs: 97 active+clean; 9.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 296 MiB/s wr, 83 op/s 2024-09-19T05:07:28.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:07:27 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/3949427592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:07:28.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:07:27 smithi096 ceph-mon[35085]: pgmap v87: 97 pgs: 97 active+clean; 9.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 296 MiB/s wr, 83 op/s 2024-09-19T05:07:28.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:07:27 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/3949427592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:07:28.864 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T05:07:29.277 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:07:29.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.891+0000 7fcf63600640 1 -- 172.21.15.43:0/809172098 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf5c101de0 msgr2=0x7fcf5c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:29.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.891+0000 7fcf63600640 1 --2- 172.21.15.43:0/809172098 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf5c101de0 0x7fcf5c1021c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcf50009860 tx=0x7fcf5002a340 comp rx=0 tx=0).stop 2024-09-19T05:07:29.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.892+0000 7fcf63600640 1 -- 172.21.15.43:0/809172098 shutdown_connections 2024-09-19T05:07:29.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.892+0000 7fcf63600640 1 --2- 172.21.15.43:0/809172098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf5c106f10 0x7fcf5c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:29.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.892+0000 7fcf63600640 1 --2- 172.21.15.43:0/809172098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf5c102790 0x7fcf5c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:29.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.892+0000 7fcf63600640 1 --2- 172.21.15.43:0/809172098 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf5c101de0 0x7fcf5c1021c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:29.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.892+0000 7fcf63600640 1 -- 172.21.15.43:0/809172098 >> 172.21.15.43:0/809172098 conn(0x7fcf5c0fd8e0 msgr2=0x7fcf5c0ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:29.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.892+0000 7fcf63600640 1 -- 172.21.15.43:0/809172098 shutdown_connections 2024-09-19T05:07:29.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.892+0000 7fcf63600640 1 -- 172.21.15.43:0/809172098 wait complete. 2024-09-19T05:07:29.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.893+0000 7fcf63600640 1 Processor -- start 2024-09-19T05:07:29.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.893+0000 7fcf63600640 1 -- start start 2024-09-19T05:07:29.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.894+0000 7fcf63600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf5c101de0 0x7fcf5c1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:29.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.894+0000 7fcf63600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf5c102790 0x7fcf5c1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:29.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.894+0000 7fcf63600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf5c106f10 0x7fcf5c19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:29.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.894+0000 7fcf63600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5c111f70 con 0x7fcf5c102790 2024-09-19T05:07:29.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.894+0000 7fcf63600640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5c111df0 con 0x7fcf5c106f10 2024-09-19T05:07:29.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.894+0000 7fcf63600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5c1120f0 con 0x7fcf5c101de0 2024-09-19T05:07:29.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.894+0000 7fcf5be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf5c102790 0x7fcf5c1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:29.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.894+0000 7fcf60c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf5c101de0 0x7fcf5c1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:29.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.894+0000 7fcf5be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf5c102790 0x7fcf5c1989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60866/0 (socket says 172.21.15.43:60866) 2024-09-19T05:07:29.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.894+0000 7fcf60c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf5c101de0 0x7fcf5c1984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:59866/0 (socket says 172.21.15.43:59866) 2024-09-19T05:07:29.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.894+0000 7fcf5be00640 1 -- 172.21.15.43:0/2286889021 learned_addr learned my addr 172.21.15.43:0/2286889021 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:29.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.894+0000 7fcf61600640 1 --2- 172.21.15.43:0/2286889021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf5c106f10 0x7fcf5c19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:29.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.895+0000 7fcf60c00640 1 -- 172.21.15.43:0/2286889021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf5c106f10 msgr2=0x7fcf5c19cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:29.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.895+0000 7fcf60c00640 1 --2- 172.21.15.43:0/2286889021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf5c106f10 0x7fcf5c19cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:29.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.895+0000 7fcf60c00640 1 -- 172.21.15.43:0/2286889021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf5c102790 msgr2=0x7fcf5c1989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:29.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.895+0000 7fcf60c00640 1 --2- 172.21.15.43:0/2286889021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf5c102790 0x7fcf5c1989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:29.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.895+0000 7fcf60c00640 1 -- 172.21.15.43:0/2286889021 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf5c19d460 con 0x7fcf5c101de0 2024-09-19T05:07:29.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.895+0000 7fcf61600640 1 --2- 172.21.15.43:0/2286889021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf5c106f10 0x7fcf5c19cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:07:29.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.895+0000 7fcf5be00640 1 --2- 172.21.15.43:0/2286889021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf5c102790 0x7fcf5c1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:07:29.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.895+0000 7fcf60c00640 1 --2- 172.21.15.43:0/2286889021 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf5c101de0 0x7fcf5c1984b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcf50035040 tx=0x7fcf50033b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:29.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.895+0000 7fcf59600640 1 -- 172.21.15.43:0/2286889021 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5003f020 con 0x7fcf5c101de0 2024-09-19T05:07:29.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.895+0000 7fcf59600640 1 -- 172.21.15.43:0/2286889021 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf50030550 con 0x7fcf5c101de0 2024-09-19T05:07:29.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.895+0000 7fcf59600640 1 -- 172.21.15.43:0/2286889021 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf50037040 con 0x7fcf5c101de0 2024-09-19T05:07:29.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.895+0000 7fcf63600640 1 -- 172.21.15.43:0/2286889021 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf5c19d6f0 con 0x7fcf5c101de0 2024-09-19T05:07:29.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.896+0000 7fcf63600640 1 -- 172.21.15.43:0/2286889021 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf5c1a5030 con 0x7fcf5c101de0 2024-09-19T05:07:29.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.899+0000 7fcf6625f740 1 Processor -- start 2024-09-19T05:07:29.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.899+0000 7fcf59600640 1 -- 172.21.15.43:0/2286889021 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fcf50042020 con 0x7fcf5c101de0 2024-09-19T05:07:29.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.899+0000 7fcf6625f740 1 -- start start 2024-09-19T05:07:29.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.899+0000 7fcf6625f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560935ee6080 0x560935ee6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:29.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.900+0000 7fcf6625f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560935ee69a0 0x560935ef0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:29.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.900+0000 7fcf6625f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560935ef1260 0x560935ef3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:29.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.900+0000 7fcf6625f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560935e483e0 con 0x560935ee69a0 2024-09-19T05:07:29.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.900+0000 7fcf6625f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560935defd80 con 0x560935ee6080 2024-09-19T05:07:29.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.900+0000 7fcf6625f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560935e443a0 con 0x560935ef1260 2024-09-19T05:07:29.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.900+0000 7fcf60c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560935ee69a0 0x560935ef0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:29.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.900+0000 7fcf61600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560935ee6080 0x560935ee6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:29.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.900+0000 7fcf61600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560935ee6080 0x560935ee6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:49132/0 (socket says 172.21.15.43:49132) 2024-09-19T05:07:29.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.900+0000 7fcf61600640 1 -- 172.21.15.43:0/876179058 learned_addr learned my addr 172.21.15.43:0/876179058 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:29.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.900+0000 7fcf61600640 1 -- 172.21.15.43:0/876179058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560935ef1260 msgr2=0x560935ef3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:29.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.900+0000 7fcf61600640 1 --2- 172.21.15.43:0/876179058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560935ef1260 0x560935ef3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:29.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.900+0000 7fcf61600640 1 -- 172.21.15.43:0/876179058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560935ee69a0 msgr2=0x560935ef0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:29.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.900+0000 7fcf61600640 1 --2- 172.21.15.43:0/876179058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560935ee69a0 0x560935ef0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:29.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.900+0000 7fcf61600640 1 -- 172.21.15.43:0/876179058 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560935e0eae0 con 0x560935ee6080 2024-09-19T05:07:29.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.901+0000 7fcf61600640 1 --2- 172.21.15.43:0/876179058 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560935ee6080 0x560935ee6460 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcf54009f10 tx=0x7fcf5400f7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b387e7c8f0e7a072 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:29.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.901+0000 7fcf4dc00640 1 -- 172.21.15.43:0/876179058 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5401d070 con 0x560935ee6080 2024-09-19T05:07:29.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.901+0000 7fcf4dc00640 1 -- 172.21.15.43:0/876179058 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf5400dad0 con 0x560935ee6080 2024-09-19T05:07:29.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.901+0000 7fcf4dc00640 1 -- 172.21.15.43:0/876179058 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf54018dd0 con 0x560935ee6080 2024-09-19T05:07:29.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.901+0000 7fcf59600640 1 --2- 172.21.15.43:0/2286889021 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fcf440a3430 0x7fcf440a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:29.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.902+0000 7fcf60c00640 1 --2- 172.21.15.43:0/2286889021 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fcf440a3430 0x7fcf440a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:29.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.902+0000 7fcf59600640 1 -- 172.21.15.43:0/2286889021 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fcf500efd80 con 0x7fcf5c101de0 2024-09-19T05:07:29.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.902+0000 7fcf6625f740 1 -- 172.21.15.43:0/876179058 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560935ee6080 msgr2=0x560935ee6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:29.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.902+0000 7fcf6625f740 1 --2- 172.21.15.43:0/876179058 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560935ee6080 0x560935ee6460 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcf54009f10 tx=0x7fcf5400f7d0 comp rx=0 tx=0).stop 2024-09-19T05:07:29.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.902+0000 7fcf6625f740 1 -- 172.21.15.43:0/876179058 shutdown_connections 2024-09-19T05:07:29.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.902+0000 7fcf6625f740 1 --2- 172.21.15.43:0/876179058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560935ef1260 0x560935ef3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:29.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.902+0000 7fcf6625f740 1 --2- 172.21.15.43:0/876179058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560935ee69a0 0x560935ef0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:29.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.902+0000 7fcf6625f740 1 --2- 172.21.15.43:0/876179058 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560935ee6080 0x560935ee6460 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:29.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.902+0000 7fcf6625f740 1 -- 172.21.15.43:0/876179058 >> 172.21.15.43:0/876179058 conn(0x560935ee2610 msgr2=0x560935ee29f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:29.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.902+0000 7fcf60c00640 1 --2- 172.21.15.43:0/2286889021 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fcf440a3430 0x7fcf440a58f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x560935ee3470 tx=0x7fcf500f3830 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:29.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.902+0000 7fcf6625f740 1 -- 172.21.15.43:0/876179058 shutdown_connections 2024-09-19T05:07:29.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.903+0000 7fcf6625f740 1 -- 172.21.15.43:0/876179058 wait complete. 2024-09-19T05:07:29.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.903+0000 7fcf6625f740 1 -- start start 2024-09-19T05:07:29.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.903+0000 7fcf6625f740 1 Processor -- start 2024-09-19T05:07:29.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.904+0000 7fcf6625f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560935ee6080 0x560935eee220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:29.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.904+0000 7fcf6625f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560935ee69a0 0x560935eee760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:29.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.904+0000 7fcf6625f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560935ef1260 0x560935eeeca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:29.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.904+0000 7fcf6625f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560935ef3e30 con 0x560935ee6080 2024-09-19T05:07:29.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.904+0000 7fcf6625f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560935e030f0 con 0x560935ee69a0 2024-09-19T05:07:29.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.904+0000 7fcf6625f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560935ef3fb0 con 0x560935ef1260 2024-09-19T05:07:29.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.904+0000 7fcf6625f740 10 client.0 _ll_register_callbacks cb 0x560935deabd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:29.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.904+0000 7fcf61600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560935ee6080 0x560935eee220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:29.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.904+0000 7fcf5be00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560935ee69a0 0x560935eee760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:29.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.904+0000 7fcf5be00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560935ee69a0 0x560935eee760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:49142/0 (socket says 172.21.15.43:49142) 2024-09-19T05:07:29.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.904+0000 7fcf5be00640 1 -- 172.21.15.43:0/2174227042 learned_addr learned my addr 172.21.15.43:0/2174227042 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:29.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.904+0000 7fcf61600640 1 --2- 172.21.15.43:0/2174227042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560935ef1260 0x560935eeeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:29.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.905+0000 7fcf5be00640 1 -- 172.21.15.43:0/2174227042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560935ef1260 msgr2=0x560935eeeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:29.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.905+0000 7fcf5be00640 1 --2- 172.21.15.43:0/2174227042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560935ef1260 0x560935eeeca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:29.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.905+0000 7fcf5be00640 1 -- 172.21.15.43:0/2174227042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560935ee6080 msgr2=0x560935eee220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:29.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.905+0000 7fcf5be00640 1 --2- 172.21.15.43:0/2174227042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560935ee6080 0x560935eee220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:29.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.905+0000 7fcf5be00640 1 -- 172.21.15.43:0/2174227042 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560935bb93c0 con 0x560935ee69a0 2024-09-19T05:07:29.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.905+0000 7fcf61600640 1 --2- 172.21.15.43:0/2174227042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560935ee6080 0x560935eee220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:07:29.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.905+0000 7fcf5be00640 1 --2- 172.21.15.43:0/2174227042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560935ee69a0 0x560935eee760 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcf480119f0 tx=0x7fcf48011eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee62746fca0ebf09 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:29.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.905+0000 7fcf42000640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:07:29.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.905+0000 7fcf42000640 1 -- 172.21.15.43:0/2174227042 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4801e060 con 0x560935ee69a0 2024-09-19T05:07:29.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.905+0000 7fcf6625f740 1 -- 172.21.15.43:0/2174227042 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf5400dad0 con 0x560935ee69a0 2024-09-19T05:07:29.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.905+0000 7fcf42000640 1 -- 172.21.15.43:0/2174227042 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fcf4800fcb0 con 0x560935ee69a0 2024-09-19T05:07:29.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.906+0000 7fcf42000640 1 -- 172.21.15.43:0/2174227042 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcf48019410 con 0x560935ee69a0 2024-09-19T05:07:29.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.906+0000 7fcf6625f740 10 client.25778 fetch_fsmap learned FSMap version 29 2024-09-19T05:07:29.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.906+0000 7fcf6625f740 1 -- 172.21.15.43:0/2174227042 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560935f00010 con 0x560935ee69a0 2024-09-19T05:07:29.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.906+0000 7fcf42000640 1 -- 172.21.15.43:0/2174227042 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf5400dad0 con 0x560935ee69a0 2024-09-19T05:07:29.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.906+0000 7fcf42000640 1 -- 172.21.15.43:0/2174227042 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcf48021430 con 0x560935ee69a0 2024-09-19T05:07:29.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.907+0000 7fcf6625f740 10 client.25778 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:07:29.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.907+0000 7fcf6625f740 10 client.25778 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-19T05:07:29.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.907+0000 7fcf6625f740 20 client.25778 populate_metadata read hostname 'smithi043' 2024-09-19T05:07:29.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.907+0000 7fcf6625f740 1 --2- 172.21.15.43:0/2174227042 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x560935f014d0 0x560935f038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:29.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.907+0000 7fcf6625f740 4 client.25778 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:07:29.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.907+0000 7fcf6625f740 1 -- 172.21.15.43:0/2174227042 --> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560935e55950 con 0x560935f014d0 2024-09-19T05:07:29.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.907+0000 7fcf61600640 1 --2- 172.21.15.43:0/2174227042 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x560935f014d0 0x560935f038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:29.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.908+0000 7fcf42000640 10 client.25778 ms_handle_connect on v2:172.21.15.151:6836/3960378010 2024-09-19T05:07:29.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.908+0000 7fcf61600640 1 --2- 172.21.15.43:0/2174227042 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x560935f014d0 0x560935f038e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c5f7f04519a21e80 server_cookie=88c89f19b36a61b8 in_seq=0 out_seq=0 2024-09-19T05:07:29.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.908+0000 7fcf42000640 1 -- 172.21.15.43:0/2174227042 <== mds.4 v2:172.21.15.151:6836/3960378010 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560935e55950 con 0x560935f014d0 2024-09-19T05:07:29.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.908+0000 7fcf42000640 10 client.25778 handle_command_reply: tid=0 2024-09-19T05:07:29.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.908+0000 7fcf6625f740 1 client.25778 shutdown 2024-09-19T05:07:29.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.909+0000 7fcf6625f740 1 -- 172.21.15.43:0/2174227042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560935ee69a0 msgr2=0x560935eee760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:29.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.909+0000 7fcf6625f740 1 --2- 172.21.15.43:0/2174227042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560935ee69a0 0x560935eee760 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcf480119f0 tx=0x7fcf48011eb0 comp rx=0 tx=0).stop 2024-09-19T05:07:29.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.909+0000 7fcf6625f740 1 -- 172.21.15.43:0/2174227042 shutdown_connections 2024-09-19T05:07:29.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.909+0000 7fcf6625f740 1 --2- 172.21.15.43:0/2174227042 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x560935f014d0 0x560935f038e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:29.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.909+0000 7fcf6625f740 1 --2- 172.21.15.43:0/2174227042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560935ef1260 0x560935eeeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:29.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.909+0000 7fcf42000640 0 client.25778 ms_handle_reset on v2:172.21.15.151:6836/3960378010 2024-09-19T05:07:29.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.909+0000 7fcf6625f740 1 --2- 172.21.15.43:0/2174227042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560935ee69a0 0x560935eee760 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:29.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.909+0000 7fcf6625f740 1 --2- 172.21.15.43:0/2174227042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560935ee6080 0x560935eee220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:29.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.909+0000 7fcf6625f740 1 -- 172.21.15.43:0/2174227042 >> 172.21.15.43:0/2174227042 conn(0x560935ee2610 msgr2=0x560935edd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:29.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.909+0000 7fcf6625f740 1 -- 172.21.15.43:0/2174227042 shutdown_connections 2024-09-19T05:07:29.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.909+0000 7fcf6625f740 1 -- 172.21.15.43:0/2174227042 wait complete. 2024-09-19T05:07:29.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.909+0000 7fcf6625f740 20 client.25778 trim_cache size 0 max 16384 2024-09-19T05:07:29.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.938+0000 7fcf6625f740 1 Processor -- start 2024-09-19T05:07:29.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.938+0000 7fcf6625f740 1 -- start start 2024-09-19T05:07:29.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.938+0000 7fcf6625f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560935ee6080 0x560935edd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:29.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.938+0000 7fcf6625f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560935ee69a0 0x560935f091f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:29.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.938+0000 7fcf6625f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560935ef1260 0x560935f09730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:29.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.938+0000 7fcf6625f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5c1ab670 con 0x560935ee69a0 2024-09-19T05:07:29.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.938+0000 7fcf6625f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5c1ad1b0 con 0x560935ef1260 2024-09-19T05:07:29.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.938+0000 7fcf6625f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5c1abd40 con 0x560935ee6080 2024-09-19T05:07:29.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.939+0000 7fcf61600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560935ee6080 0x560935edd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:29.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.939+0000 7fcf61600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560935ee6080 0x560935edd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:59904/0 (socket says 172.21.15.43:59904) 2024-09-19T05:07:29.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.939+0000 7fcf61600640 1 -- 172.21.15.43:0/132998189 learned_addr learned my addr 172.21.15.43:0/132998189 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:29.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.939+0000 7fcf5be00640 1 --2- 172.21.15.43:0/132998189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560935ee69a0 0x560935f091f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:29.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.939+0000 7fcf61600640 1 --2- 172.21.15.43:0/132998189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560935ef1260 0x560935f09730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:29.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.939+0000 7fcf61600640 1 -- 172.21.15.43:0/132998189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560935ef1260 msgr2=0x560935f09730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:29.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.939+0000 7fcf61600640 1 --2- 172.21.15.43:0/132998189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560935ef1260 0x560935f09730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:29.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.939+0000 7fcf61600640 1 -- 172.21.15.43:0/132998189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560935ee69a0 msgr2=0x560935f091f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:29.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.939+0000 7fcf61600640 1 --2- 172.21.15.43:0/132998189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560935ee69a0 0x560935f091f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:29.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.939+0000 7fcf61600640 1 -- 172.21.15.43:0/132998189 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560935ed9770 con 0x560935ee6080 2024-09-19T05:07:29.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.939+0000 7fcf61600640 1 --2- 172.21.15.43:0/132998189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560935ee6080 0x560935edd270 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcf54018860 tx=0x7fcf54009980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7167945bc8ddb32a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:29.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.940+0000 7fcf4d200640 1 -- 172.21.15.43:0/132998189 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5401d040 con 0x560935ee6080 2024-09-19T05:07:29.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.940+0000 7fcf4d200640 1 -- 172.21.15.43:0/132998189 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf54018a40 con 0x560935ee6080 2024-09-19T05:07:29.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.940+0000 7fcf4d200640 1 -- 172.21.15.43:0/132998189 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf54032a10 con 0x560935ee6080 2024-09-19T05:07:29.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.941+0000 7fcf6625f740 1 -- 172.21.15.43:0/132998189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560935ee6080 msgr2=0x560935edd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:29.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.941+0000 7fcf6625f740 1 --2- 172.21.15.43:0/132998189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560935ee6080 0x560935edd270 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcf54018860 tx=0x7fcf54009980 comp rx=0 tx=0).stop 2024-09-19T05:07:29.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.941+0000 7fcf6625f740 1 -- 172.21.15.43:0/132998189 shutdown_connections 2024-09-19T05:07:29.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.941+0000 7fcf6625f740 1 --2- 172.21.15.43:0/132998189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560935ef1260 0x560935f09730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:29.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.941+0000 7fcf6625f740 1 --2- 172.21.15.43:0/132998189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560935ee69a0 0x560935f091f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:29.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.941+0000 7fcf6625f740 1 --2- 172.21.15.43:0/132998189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560935ee6080 0x560935edd270 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:29.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.941+0000 7fcf6625f740 1 -- 172.21.15.43:0/132998189 >> 172.21.15.43:0/132998189 conn(0x560935ee2610 msgr2=0x560935edcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:29.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.941+0000 7fcf6625f740 1 -- 172.21.15.43:0/132998189 shutdown_connections 2024-09-19T05:07:29.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.941+0000 7fcf6625f740 1 -- 172.21.15.43:0/132998189 wait complete. 2024-09-19T05:07:29.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.942+0000 7fcf6625f740 1 -- start start 2024-09-19T05:07:29.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.942+0000 7fcf6625f740 1 Processor -- start 2024-09-19T05:07:29.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.943+0000 7fcf6625f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560935ee6080 0x560935eeb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:29.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.943+0000 7fcf6625f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560935ee69a0 0x560935eeb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:29.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.943+0000 7fcf6625f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560935ef1260 0x560935eebc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:29.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.943+0000 7fcf6625f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5c1acae0 con 0x560935ee6080 2024-09-19T05:07:29.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.943+0000 7fcf6625f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5c1ac410 con 0x560935ee69a0 2024-09-19T05:07:29.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.943+0000 7fcf6625f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5c1a9650 con 0x560935ef1260 2024-09-19T05:07:29.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.943+0000 7fcf6625f740 10 client.0 _ll_register_callbacks cb 0x7fcf48009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:29.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.943+0000 7fcf61600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560935ee6080 0x560935eeb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:29.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.943+0000 7fcf61600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560935ee6080 0x560935eeb1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60918/0 (socket says 172.21.15.43:60918) 2024-09-19T05:07:29.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.943+0000 7fcf61600640 1 -- 172.21.15.43:0/1492706448 learned_addr learned my addr 172.21.15.43:0/1492706448 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:29.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.943+0000 7fcf5be00640 1 --2- 172.21.15.43:0/1492706448 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560935ee69a0 0x560935eeb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:29.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.943+0000 7fcf61600640 1 --2- 172.21.15.43:0/1492706448 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560935ef1260 0x560935eebc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:29.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.943+0000 7fcf5be00640 1 -- 172.21.15.43:0/1492706448 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560935ef1260 msgr2=0x560935eebc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:29.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.943+0000 7fcf5be00640 1 --2- 172.21.15.43:0/1492706448 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560935ef1260 0x560935eebc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:29.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.943+0000 7fcf5be00640 1 -- 172.21.15.43:0/1492706448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560935ee6080 msgr2=0x560935eeb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:29.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.943+0000 7fcf5be00640 1 --2- 172.21.15.43:0/1492706448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560935ee6080 0x560935eeb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:29.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.944+0000 7fcf5be00640 1 -- 172.21.15.43:0/1492706448 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560935ee32a0 con 0x560935ee69a0 2024-09-19T05:07:29.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.944+0000 7fcf5be00640 1 --2- 172.21.15.43:0/1492706448 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560935ee69a0 0x560935eeb710 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcf480218a0 tx=0x7fcf480116e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d9d9a6db5954dd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:29.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.944+0000 7fcf42000640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:07:29.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.944+0000 7fcf42000640 1 -- 172.21.15.43:0/1492706448 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf48021c90 con 0x560935ee69a0 2024-09-19T05:07:29.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.944+0000 7fcf42000640 1 -- 172.21.15.43:0/1492706448 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fcf48021e30 con 0x560935ee69a0 2024-09-19T05:07:29.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.944+0000 7fcf6625f740 1 -- 172.21.15.43:0/1492706448 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf54018a40 con 0x560935ee69a0 2024-09-19T05:07:29.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.945+0000 7fcf42000640 1 -- 172.21.15.43:0/1492706448 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcf48005070 con 0x560935ee69a0 2024-09-19T05:07:29.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.945+0000 7fcf6625f740 10 client.25784 fetch_fsmap learned FSMap version 29 2024-09-19T05:07:29.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.945+0000 7fcf6625f740 1 -- 172.21.15.43:0/1492706448 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560935f6c280 con 0x560935ee69a0 2024-09-19T05:07:29.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.945+0000 7fcf42000640 1 -- 172.21.15.43:0/1492706448 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf54018a40 con 0x560935ee69a0 2024-09-19T05:07:29.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.945+0000 7fcf42000640 1 -- 172.21.15.43:0/1492706448 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcf4802a020 con 0x560935ee69a0 2024-09-19T05:07:29.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.946+0000 7fcf6625f740 10 client.25784 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:07:29.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.946+0000 7fcf6625f740 10 client.25784 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-19T05:07:29.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.946+0000 7fcf6625f740 20 client.25784 populate_metadata read hostname 'smithi043' 2024-09-19T05:07:29.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.946+0000 7fcf6625f740 1 --2- 172.21.15.43:0/1492706448 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x560935f014d0 0x560935f6e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:29.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.946+0000 7fcf6625f740 4 client.25784 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:07:29.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.946+0000 7fcf6625f740 1 -- 172.21.15.43:0/1492706448 --> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560935e4fe60 con 0x560935f014d0 2024-09-19T05:07:29.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.946+0000 7fcf61600640 1 --2- 172.21.15.43:0/1492706448 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x560935f014d0 0x560935f6e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:29.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.947+0000 7fcf61600640 1 --2- 172.21.15.43:0/1492706448 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x560935f014d0 0x560935f6e980 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6ab893724ccfaaaa server_cookie=820c1a7049094550 in_seq=0 out_seq=0 2024-09-19T05:07:29.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.947+0000 7fcf42000640 10 client.25784 ms_handle_connect on v2:172.21.15.151:6836/3960378010 2024-09-19T05:07:29.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.947+0000 7fcf42000640 1 -- 172.21.15.43:0/1492706448 <== mds.4 v2:172.21.15.151:6836/3960378010 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560935e4fe60 con 0x560935f014d0 2024-09-19T05:07:29.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.947+0000 7fcf42000640 10 client.25784 handle_command_reply: tid=0 2024-09-19T05:07:29.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.947+0000 7fcf6625f740 1 client.25784 shutdown 2024-09-19T05:07:29.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.947+0000 7fcf6625f740 1 -- 172.21.15.43:0/1492706448 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560935ee69a0 msgr2=0x560935eeb710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:29.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.947+0000 7fcf6625f740 1 --2- 172.21.15.43:0/1492706448 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560935ee69a0 0x560935eeb710 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcf480218a0 tx=0x7fcf480116e0 comp rx=0 tx=0).stop 2024-09-19T05:07:29.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.947+0000 7fcf6625f740 1 -- 172.21.15.43:0/1492706448 shutdown_connections 2024-09-19T05:07:29.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.947+0000 7fcf6625f740 1 --2- 172.21.15.43:0/1492706448 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x560935f014d0 0x560935f6e980 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:29.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.947+0000 7fcf6625f740 1 --2- 172.21.15.43:0/1492706448 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560935ef1260 0x560935eebc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:29.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.947+0000 7fcf6625f740 1 --2- 172.21.15.43:0/1492706448 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560935ee69a0 0x560935eeb710 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:29.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.947+0000 7fcf6625f740 1 --2- 172.21.15.43:0/1492706448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560935ee6080 0x560935eeb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:29.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.947+0000 7fcf42000640 0 client.25784 ms_handle_reset on v2:172.21.15.151:6836/3960378010 2024-09-19T05:07:29.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.947+0000 7fcf6625f740 1 -- 172.21.15.43:0/1492706448 >> 172.21.15.43:0/1492706448 conn(0x560935ee2610 msgr2=0x560935ef02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:29.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.947+0000 7fcf6625f740 1 -- 172.21.15.43:0/1492706448 shutdown_connections 2024-09-19T05:07:29.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.947+0000 7fcf6625f740 1 -- 172.21.15.43:0/1492706448 wait complete. 2024-09-19T05:07:29.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.948+0000 7fcf6625f740 20 client.25784 trim_cache size 0 max 16384 2024-09-19T05:07:29.950 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:07:29.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.948+0000 7fcf63600640 1 -- 172.21.15.43:0/2286889021 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fcf440a3430 msgr2=0x7fcf440a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:29.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.948+0000 7fcf63600640 1 --2- 172.21.15.43:0/2286889021 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fcf440a3430 0x7fcf440a58f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x560935ee3470 tx=0x7fcf500f3830 comp rx=0 tx=0).stop 2024-09-19T05:07:29.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.948+0000 7fcf63600640 1 -- 172.21.15.43:0/2286889021 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf5c101de0 msgr2=0x7fcf5c1984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:29.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.948+0000 7fcf63600640 1 --2- 172.21.15.43:0/2286889021 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf5c101de0 0x7fcf5c1984b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcf50035040 tx=0x7fcf50033b60 comp rx=0 tx=0).stop 2024-09-19T05:07:29.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.949+0000 7fcf63600640 1 -- 172.21.15.43:0/2286889021 shutdown_connections 2024-09-19T05:07:29.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.949+0000 7fcf63600640 1 --2- 172.21.15.43:0/2286889021 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fcf440a3430 0x7fcf440a58f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:29.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.949+0000 7fcf63600640 1 --2- 172.21.15.43:0/2286889021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf5c106f10 0x7fcf5c19cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:29.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.949+0000 7fcf63600640 1 --2- 172.21.15.43:0/2286889021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf5c102790 0x7fcf5c1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:29.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.949+0000 7fcf63600640 1 --2- 172.21.15.43:0/2286889021 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf5c101de0 0x7fcf5c1984b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:29.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.949+0000 7fcf63600640 1 -- 172.21.15.43:0/2286889021 >> 172.21.15.43:0/2286889021 conn(0x7fcf5c0fd8e0 msgr2=0x7fcf5c0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:29.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.949+0000 7fcf63600640 1 -- 172.21.15.43:0/2286889021 shutdown_connections 2024-09-19T05:07:29.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:29.949+0000 7fcf63600640 1 -- 172.21.15.43:0/2286889021 wait complete. 2024-09-19T05:07:30.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:07:30 smithi043 ceph-mon[30800]: pgmap v88: 97 pgs: 97 active+clean; 9.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 296 MiB/s wr, 83 op/s 2024-09-19T05:07:30.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:07:30 smithi151 ceph-mon[35425]: pgmap v88: 97 pgs: 97 active+clean; 9.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 296 MiB/s wr, 83 op/s 2024-09-19T05:07:30.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:07:30 smithi096 ceph-mon[35085]: pgmap v88: 97 pgs: 97 active+clean; 9.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 296 MiB/s wr, 83 op/s 2024-09-19T05:07:31.121 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T05:07:31.855 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:07:32.290 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:07:32 smithi043 ceph-mon[30800]: pgmap v89: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 356 MiB/s wr, 102 op/s 2024-09-19T05:07:32.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:07:32 smithi151 ceph-mon[35425]: pgmap v89: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 356 MiB/s wr, 102 op/s 2024-09-19T05:07:32.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:07:32 smithi096 ceph-mon[35085]: pgmap v89: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 356 MiB/s wr, 102 op/s 2024-09-19T05:07:32.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.872+0000 7fc766800640 1 -- 172.21.15.43:0/565377573 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7600ffb60 msgr2=0x7fc7600fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:32.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.872+0000 7fc766800640 1 --2- 172.21.15.43:0/565377573 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7600ffb60 0x7fc7600fffa0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc748009930 tx=0x7fc74802a460 comp rx=0 tx=0).stop 2024-09-19T05:07:32.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.873+0000 7fc766800640 1 -- 172.21.15.43:0/565377573 shutdown_connections 2024-09-19T05:07:32.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.874+0000 7fc766800640 1 --2- 172.21.15.43:0/565377573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7601004e0 0x7fc760110b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:32.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.874+0000 7fc766800640 1 --2- 172.21.15.43:0/565377573 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7600ffb60 0x7fc7600fffa0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:32.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.874+0000 7fc766800640 1 --2- 172.21.15.43:0/565377573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc760102990 0x7fc760102d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:32.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.874+0000 7fc766800640 1 -- 172.21.15.43:0/565377573 >> 172.21.15.43:0/565377573 conn(0x7fc7600fba90 msgr2=0x7fc7600fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:32.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.874+0000 7fc766800640 1 -- 172.21.15.43:0/565377573 shutdown_connections 2024-09-19T05:07:32.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.876+0000 7fc766800640 1 -- 172.21.15.43:0/565377573 wait complete. 2024-09-19T05:07:32.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.877+0000 7fc766800640 1 Processor -- start 2024-09-19T05:07:32.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.877+0000 7fc766800640 1 -- start start 2024-09-19T05:07:32.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.878+0000 7fc766800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7600ffb60 0x7fc7601a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:32.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.878+0000 7fc766800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7601004e0 0x7fc7601a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:32.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.878+0000 7fc766800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc760102990 0x7fc76019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:32.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.878+0000 7fc766800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc7601141e0 con 0x7fc7601004e0 2024-09-19T05:07:32.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.878+0000 7fc766800640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc760114060 con 0x7fc7600ffb60 2024-09-19T05:07:32.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.878+0000 7fc766800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc760114360 con 0x7fc760102990 2024-09-19T05:07:32.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.879+0000 7fc757e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7601004e0 0x7fc7601a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:32.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.879+0000 7fc757e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7601004e0 0x7fc7601a4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60930/0 (socket says 172.21.15.43:60930) 2024-09-19T05:07:32.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.879+0000 7fc757e00640 1 -- 172.21.15.43:0/1401142248 learned_addr learned my addr 172.21.15.43:0/1401142248 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:32.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.880+0000 7fc75fe00640 1 --2- 172.21.15.43:0/1401142248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc760102990 0x7fc76019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:32.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.881+0000 7fc757e00640 1 -- 172.21.15.43:0/1401142248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc760102990 msgr2=0x7fc76019e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:32.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.881+0000 7fc757e00640 1 --2- 172.21.15.43:0/1401142248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc760102990 0x7fc76019e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:32.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.881+0000 7fc757e00640 1 -- 172.21.15.43:0/1401142248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7600ffb60 msgr2=0x7fc7601a4760 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T05:07:32.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.881+0000 7fc757e00640 1 --2- 172.21.15.43:0/1401142248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7600ffb60 0x7fc7601a4760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:32.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.881+0000 7fc757e00640 1 -- 172.21.15.43:0/1401142248 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc76019f050 con 0x7fc7601004e0 2024-09-19T05:07:32.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.882+0000 7fc757e00640 1 --2- 172.21.15.43:0/1401142248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7601004e0 0x7fc7601a4ca0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc74802a980 tx=0x7fc748003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:32.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.883+0000 7fc75cc00640 1 -- 172.21.15.43:0/1401142248 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74803f070 con 0x7fc7601004e0 2024-09-19T05:07:32.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.883+0000 7fc75cc00640 1 -- 172.21.15.43:0/1401142248 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc74803b460 con 0x7fc7601004e0 2024-09-19T05:07:32.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.883+0000 7fc75cc00640 1 -- 172.21.15.43:0/1401142248 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74803f070 con 0x7fc7601004e0 2024-09-19T05:07:32.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.884+0000 7fc766800640 1 -- 172.21.15.43:0/1401142248 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc76019f2e0 con 0x7fc7601004e0 2024-09-19T05:07:32.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.885+0000 7fc766800640 1 -- 172.21.15.43:0/1401142248 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc760107b20 con 0x7fc7601004e0 2024-09-19T05:07:32.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.891+0000 7fc75cc00640 1 -- 172.21.15.43:0/1401142248 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fc748042070 con 0x7fc7601004e0 2024-09-19T05:07:32.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.895+0000 7fc769284740 1 Processor -- start 2024-09-19T05:07:32.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.896+0000 7fc769284740 1 -- start start 2024-09-19T05:07:32.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.897+0000 7fc769284740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba4d516c70 0x55ba4d517050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:32.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.897+0000 7fc769284740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba4d517590 0x55ba4d521910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:32.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.897+0000 7fc769284740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba4d521e50 0x55ba4d524240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:32.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.897+0000 7fc769284740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ba4d47d3e0 con 0x55ba4d516c70 2024-09-19T05:07:32.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.897+0000 7fc769284740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ba4d424d80 con 0x55ba4d521e50 2024-09-19T05:07:32.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.897+0000 7fc769284740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ba4d4793a0 con 0x55ba4d517590 2024-09-19T05:07:32.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.897+0000 7fc75cc00640 1 --2- 172.21.15.43:0/1401142248 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fc7440a3630 0x7fc7440a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:32.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.897+0000 7fc75cc00640 1 -- 172.21.15.43:0/1401142248 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fc7480ef9f0 con 0x7fc7601004e0 2024-09-19T05:07:32.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.898+0000 7fc75fe00640 1 --2- 172.21.15.43:0/1401142248 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fc7440a3630 0x7fc7440a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:32.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.898+0000 7fc75f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba4d517590 0x55ba4d521910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:32.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.898+0000 7fc75f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba4d517590 0x55ba4d521910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:59942/0 (socket says 172.21.15.43:59942) 2024-09-19T05:07:32.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.898+0000 7fc75f400640 1 -- 172.21.15.43:0/363916480 learned_addr learned my addr 172.21.15.43:0/363916480 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:32.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.898+0000 7fc75fe00640 1 --2- 172.21.15.43:0/363916480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba4d516c70 0x55ba4d517050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:32.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.898+0000 7fc757e00640 1 --2- 172.21.15.43:0/363916480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba4d521e50 0x55ba4d524240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:32.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.899+0000 7fc75f400640 1 -- 172.21.15.43:0/363916480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba4d521e50 msgr2=0x55ba4d524240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:32.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.899+0000 7fc75f400640 1 --2- 172.21.15.43:0/363916480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba4d521e50 0x55ba4d524240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:32.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.899+0000 7fc75f400640 1 -- 172.21.15.43:0/363916480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba4d516c70 msgr2=0x55ba4d517050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:32.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.899+0000 7fc75f400640 1 --2- 172.21.15.43:0/363916480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba4d516c70 0x55ba4d517050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:32.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.899+0000 7fc75f400640 1 -- 172.21.15.43:0/363916480 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba4d443ae0 con 0x55ba4d517590 2024-09-19T05:07:32.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.899+0000 7fc757e00640 1 --2- 172.21.15.43:0/363916480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba4d521e50 0x55ba4d524240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:07:32.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.900+0000 7fc75fe00640 1 --2- 172.21.15.43:0/1401142248 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fc7440a3630 0x7fc7440a5af0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc758007310 tx=0x7fc758007ac0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:32.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.901+0000 7fc75f400640 1 --2- 172.21.15.43:0/363916480 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba4d517590 0x55ba4d521910 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc750009740 tx=0x7fc750009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b8a6304c6ae9989 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:32.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.901+0000 7fc754c00640 1 -- 172.21.15.43:0/363916480 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc750015020 con 0x55ba4d517590 2024-09-19T05:07:32.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.902+0000 7fc754c00640 1 -- 172.21.15.43:0/363916480 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc750009e50 con 0x55ba4d517590 2024-09-19T05:07:32.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.902+0000 7fc754c00640 1 -- 172.21.15.43:0/363916480 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7500105e0 con 0x55ba4d517590 2024-09-19T05:07:32.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.903+0000 7fc769284740 1 -- 172.21.15.43:0/363916480 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba4d517590 msgr2=0x55ba4d521910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:32.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.903+0000 7fc769284740 1 --2- 172.21.15.43:0/363916480 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba4d517590 0x55ba4d521910 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc750009740 tx=0x7fc750009c00 comp rx=0 tx=0).stop 2024-09-19T05:07:32.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.904+0000 7fc769284740 1 -- 172.21.15.43:0/363916480 shutdown_connections 2024-09-19T05:07:32.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.904+0000 7fc769284740 1 --2- 172.21.15.43:0/363916480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba4d521e50 0x55ba4d524240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:32.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.904+0000 7fc769284740 1 --2- 172.21.15.43:0/363916480 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba4d517590 0x55ba4d521910 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:32.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.904+0000 7fc769284740 1 --2- 172.21.15.43:0/363916480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba4d516c70 0x55ba4d517050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:32.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.904+0000 7fc769284740 1 -- 172.21.15.43:0/363916480 >> 172.21.15.43:0/363916480 conn(0x55ba4d513200 msgr2=0x55ba4d5135e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:32.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.907+0000 7fc769284740 1 -- 172.21.15.43:0/363916480 shutdown_connections 2024-09-19T05:07:32.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.911+0000 7fc769284740 1 -- 172.21.15.43:0/363916480 wait complete. 2024-09-19T05:07:32.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.912+0000 7fc769284740 1 -- start start 2024-09-19T05:07:32.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.912+0000 7fc769284740 1 Processor -- start 2024-09-19T05:07:32.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.913+0000 7fc769284740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba4d516c70 0x55ba4d51ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:32.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.913+0000 7fc769284740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba4d517590 0x55ba4d51f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:32.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.913+0000 7fc769284740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba4d521e50 0x55ba4d51f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:32.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.913+0000 7fc769284740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ba4d535a30 con 0x55ba4d516c70 2024-09-19T05:07:32.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.913+0000 7fc769284740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ba4d4380f0 con 0x55ba4d517590 2024-09-19T05:07:32.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.913+0000 7fc769284740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ba4d535bb0 con 0x55ba4d521e50 2024-09-19T05:07:32.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.913+0000 7fc769284740 10 client.0 _ll_register_callbacks cb 0x55ba4d41fbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:32.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.914+0000 7fc757e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba4d517590 0x55ba4d51f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:32.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.914+0000 7fc757e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba4d517590 0x55ba4d51f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:49206/0 (socket says 172.21.15.43:49206) 2024-09-19T05:07:32.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.914+0000 7fc757e00640 1 -- 172.21.15.43:0/3460179683 learned_addr learned my addr 172.21.15.43:0/3460179683 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:32.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.915+0000 7fc75f400640 1 --2- 172.21.15.43:0/3460179683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba4d516c70 0x55ba4d51ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:32.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.915+0000 7fc757e00640 1 -- 172.21.15.43:0/3460179683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba4d521e50 msgr2=0x55ba4d51f730 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:07:32.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.915+0000 7fc757e00640 1 --2- 172.21.15.43:0/3460179683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba4d521e50 0x55ba4d51f730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:32.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.915+0000 7fc757e00640 1 -- 172.21.15.43:0/3460179683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba4d516c70 msgr2=0x55ba4d51ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:32.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.915+0000 7fc757e00640 1 --2- 172.21.15.43:0/3460179683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba4d516c70 0x55ba4d51ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:32.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.915+0000 7fc757e00640 1 -- 172.21.15.43:0/3460179683 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba4d1ee3c0 con 0x55ba4d517590 2024-09-19T05:07:32.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.916+0000 7fc757e00640 1 --2- 172.21.15.43:0/3460179683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba4d517590 0x55ba4d51f1f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc7480b86b0 tx=0x7fc7480bc3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=919572d10ef6844a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:32.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.916+0000 7fc74d600640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:07:32.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.917+0000 7fc74d600640 1 -- 172.21.15.43:0/3460179683 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7480c0090 con 0x55ba4d517590 2024-09-19T05:07:32.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.917+0000 7fc74d600640 1 -- 172.21.15.43:0/3460179683 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fc74803b610 con 0x55ba4d517590 2024-09-19T05:07:32.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.917+0000 7fc769284740 1 -- 172.21.15.43:0/3460179683 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc750009e50 con 0x55ba4d517590 2024-09-19T05:07:32.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.919+0000 7fc74d600640 1 -- 172.21.15.43:0/3460179683 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc7480c4950 con 0x55ba4d517590 2024-09-19T05:07:32.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.919+0000 7fc769284740 10 client.25787 fetch_fsmap learned FSMap version 29 2024-09-19T05:07:32.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.919+0000 7fc769284740 1 -- 172.21.15.43:0/3460179683 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ba4d5419c0 con 0x55ba4d517590 2024-09-19T05:07:32.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.919+0000 7fc74d600640 1 -- 172.21.15.43:0/3460179683 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc750009e50 con 0x55ba4d517590 2024-09-19T05:07:32.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.919+0000 7fc74d600640 1 -- 172.21.15.43:0/3460179683 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc7480bb130 con 0x55ba4d517590 2024-09-19T05:07:32.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.919+0000 7fc769284740 10 client.25787 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:07:32.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.919+0000 7fc769284740 10 client.25787 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T05:07:32.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.919+0000 7fc769284740 20 client.25787 populate_metadata read hostname 'smithi043' 2024-09-19T05:07:32.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.919+0000 7fc769284740 1 --2- 172.21.15.43:0/3460179683 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55ba4d542d60 0x55ba4d5451f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:32.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.919+0000 7fc769284740 4 client.25787 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:07:32.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.919+0000 7fc769284740 1 -- 172.21.15.43:0/3460179683 --> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba4d48a950 con 0x55ba4d542d60 2024-09-19T05:07:32.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.920+0000 7fc75f400640 1 --2- 172.21.15.43:0/3460179683 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55ba4d542d60 0x55ba4d5451f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:32.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.922+0000 7fc75f400640 1 --2- 172.21.15.43:0/3460179683 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55ba4d542d60 0x55ba4d5451f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=feb4359dd78b3112 server_cookie=10a080f170612d04 in_seq=0 out_seq=0 2024-09-19T05:07:32.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.922+0000 7fc74d600640 10 client.25787 ms_handle_connect on v2:172.21.15.96:6836/3978388286 2024-09-19T05:07:32.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.923+0000 7fc74d600640 1 -- 172.21.15.43:0/3460179683 <== mds.3 v2:172.21.15.96:6836/3978388286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ba4d48a950 con 0x55ba4d542d60 2024-09-19T05:07:32.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.923+0000 7fc74d600640 10 client.25787 handle_command_reply: tid=0 2024-09-19T05:07:32.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.923+0000 7fc769284740 1 client.25787 shutdown 2024-09-19T05:07:32.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.924+0000 7fc769284740 1 -- 172.21.15.43:0/3460179683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba4d517590 msgr2=0x55ba4d51f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:32.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.924+0000 7fc769284740 1 --2- 172.21.15.43:0/3460179683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba4d517590 0x55ba4d51f1f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc7480b86b0 tx=0x7fc7480bc3e0 comp rx=0 tx=0).stop 2024-09-19T05:07:32.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.924+0000 7fc769284740 1 -- 172.21.15.43:0/3460179683 shutdown_connections 2024-09-19T05:07:32.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.924+0000 7fc769284740 1 --2- 172.21.15.43:0/3460179683 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55ba4d542d60 0x55ba4d5451f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:32.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.924+0000 7fc769284740 1 --2- 172.21.15.43:0/3460179683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba4d521e50 0x55ba4d51f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:32.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.924+0000 7fc769284740 1 --2- 172.21.15.43:0/3460179683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba4d517590 0x55ba4d51f1f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:32.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.924+0000 7fc769284740 1 --2- 172.21.15.43:0/3460179683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba4d516c70 0x55ba4d51ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:32.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.924+0000 7fc769284740 1 -- 172.21.15.43:0/3460179683 >> 172.21.15.43:0/3460179683 conn(0x55ba4d513200 msgr2=0x55ba4d518f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:32.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.924+0000 7fc74d600640 0 client.25787 ms_handle_reset on v2:172.21.15.96:6836/3978388286 2024-09-19T05:07:32.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.924+0000 7fc769284740 1 -- 172.21.15.43:0/3460179683 shutdown_connections 2024-09-19T05:07:32.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.924+0000 7fc769284740 1 -- 172.21.15.43:0/3460179683 wait complete. 2024-09-19T05:07:32.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.925+0000 7fc769284740 20 client.25787 trim_cache size 0 max 16384 2024-09-19T05:07:32.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.942+0000 7fc769284740 1 Processor -- start 2024-09-19T05:07:32.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.942+0000 7fc769284740 1 -- start start 2024-09-19T05:07:32.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.942+0000 7fc769284740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba4d516c70 0x55ba4d51f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:32.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.942+0000 7fc769284740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba4d517590 0x55ba4d54aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:32.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.942+0000 7fc769284740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba4d521e50 0x55ba4d54b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:32.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.942+0000 7fc769284740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc76006bc60 con 0x55ba4d516c70 2024-09-19T05:07:32.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.942+0000 7fc769284740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc76006d770 con 0x55ba4d517590 2024-09-19T05:07:32.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.942+0000 7fc769284740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc76006c300 con 0x55ba4d521e50 2024-09-19T05:07:32.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.943+0000 7fc757e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba4d517590 0x55ba4d54aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:32.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.943+0000 7fc757e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba4d517590 0x55ba4d54aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:49210/0 (socket says 172.21.15.43:49210) 2024-09-19T05:07:32.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.943+0000 7fc757e00640 1 -- 172.21.15.43:0/1963367842 learned_addr learned my addr 172.21.15.43:0/1963367842 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:32.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.943+0000 7fc75fe00640 1 --2- 172.21.15.43:0/1963367842 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba4d521e50 0x55ba4d54b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:32.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.943+0000 7fc75f400640 1 --2- 172.21.15.43:0/1963367842 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba4d516c70 0x55ba4d51f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:32.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.944+0000 7fc75fe00640 1 -- 172.21.15.43:0/1963367842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba4d517590 msgr2=0x55ba4d54aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:32.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.944+0000 7fc75fe00640 1 --2- 172.21.15.43:0/1963367842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba4d517590 0x55ba4d54aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:32.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.944+0000 7fc75fe00640 1 -- 172.21.15.43:0/1963367842 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba4d516c70 msgr2=0x55ba4d51f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:32.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.944+0000 7fc75fe00640 1 --2- 172.21.15.43:0/1963367842 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba4d516c70 0x55ba4d51f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:32.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.944+0000 7fc75fe00640 1 -- 172.21.15.43:0/1963367842 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba4d50e770 con 0x55ba4d521e50 2024-09-19T05:07:32.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.944+0000 7fc757e00640 1 --2- 172.21.15.43:0/1963367842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba4d517590 0x55ba4d54aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:07:32.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.944+0000 7fc75fe00640 1 --2- 172.21.15.43:0/1963367842 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba4d521e50 0x55ba4d54b000 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc7580117b0 tx=0x7fc75800c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=444e48becddbcf3a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:32.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.944+0000 7fc74fe00640 1 -- 172.21.15.43:0/1963367842 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc758022070 con 0x55ba4d521e50 2024-09-19T05:07:32.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.944+0000 7fc74fe00640 1 -- 172.21.15.43:0/1963367842 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc75800fc20 con 0x55ba4d521e50 2024-09-19T05:07:32.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.945+0000 7fc74fe00640 1 -- 172.21.15.43:0/1963367842 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7580056a0 con 0x55ba4d521e50 2024-09-19T05:07:32.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.945+0000 7fc769284740 1 -- 172.21.15.43:0/1963367842 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba4d521e50 msgr2=0x55ba4d54b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:32.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.945+0000 7fc769284740 1 --2- 172.21.15.43:0/1963367842 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba4d521e50 0x55ba4d54b000 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc7580117b0 tx=0x7fc75800c000 comp rx=0 tx=0).stop 2024-09-19T05:07:32.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.945+0000 7fc769284740 1 -- 172.21.15.43:0/1963367842 shutdown_connections 2024-09-19T05:07:32.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.945+0000 7fc769284740 1 --2- 172.21.15.43:0/1963367842 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba4d521e50 0x55ba4d54b000 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:32.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.945+0000 7fc769284740 1 --2- 172.21.15.43:0/1963367842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba4d517590 0x55ba4d54aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:32.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.945+0000 7fc769284740 1 --2- 172.21.15.43:0/1963367842 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba4d516c70 0x55ba4d51f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:32.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.946+0000 7fc769284740 1 -- 172.21.15.43:0/1963367842 >> 172.21.15.43:0/1963367842 conn(0x55ba4d513200 msgr2=0x55ba4d520d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:32.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.946+0000 7fc769284740 1 -- 172.21.15.43:0/1963367842 shutdown_connections 2024-09-19T05:07:32.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.946+0000 7fc769284740 1 -- 172.21.15.43:0/1963367842 wait complete. 2024-09-19T05:07:32.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.946+0000 7fc769284740 1 -- start start 2024-09-19T05:07:32.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.947+0000 7fc769284740 1 Processor -- start 2024-09-19T05:07:32.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.948+0000 7fc769284740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba4d516c70 0x55ba4d53ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:32.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.948+0000 7fc769284740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba4d517590 0x55ba4d53b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:32.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.948+0000 7fc769284740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba4d521e50 0x55ba4d537e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:32.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.948+0000 7fc769284740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc76006d0a0 con 0x55ba4d521e50 2024-09-19T05:07:32.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.948+0000 7fc769284740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc76006c9d0 con 0x55ba4d517590 2024-09-19T05:07:32.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.948+0000 7fc769284740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc76010e330 con 0x55ba4d516c70 2024-09-19T05:07:32.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.948+0000 7fc769284740 10 client.0 _ll_register_callbacks cb 0x7fc748009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:32.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.949+0000 7fc75f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba4d516c70 0x55ba4d53ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:32.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.949+0000 7fc75f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba4d516c70 0x55ba4d53ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:59978/0 (socket says 172.21.15.43:59978) 2024-09-19T05:07:32.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.949+0000 7fc757e00640 1 --2- 172.21.15.43:0/3285904563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba4d517590 0x55ba4d53b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:32.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.949+0000 7fc75fe00640 1 --2- 172.21.15.43:0/3285904563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba4d521e50 0x55ba4d537e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:32.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.949+0000 7fc75f400640 1 -- 172.21.15.43:0/3285904563 learned_addr learned my addr 172.21.15.43:0/3285904563 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:32.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.949+0000 7fc757e00640 1 -- 172.21.15.43:0/3285904563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba4d516c70 msgr2=0x55ba4d53ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:32.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.949+0000 7fc757e00640 1 --2- 172.21.15.43:0/3285904563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba4d516c70 0x55ba4d53ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:32.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.949+0000 7fc757e00640 1 -- 172.21.15.43:0/3285904563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba4d521e50 msgr2=0x55ba4d537e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:32.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.949+0000 7fc757e00640 1 --2- 172.21.15.43:0/3285904563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba4d521e50 0x55ba4d537e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:32.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.949+0000 7fc757e00640 1 -- 172.21.15.43:0/3285904563 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba4d519ae0 con 0x55ba4d517590 2024-09-19T05:07:32.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.950+0000 7fc757e00640 1 --2- 172.21.15.43:0/3285904563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba4d517590 0x55ba4d53b3a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc7480bb9e0 tx=0x7fc7480092e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cf9dbfa1bcbd9d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:32.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.950+0000 7fc74d600640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:07:32.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.950+0000 7fc74d600640 1 -- 172.21.15.43:0/3285904563 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7480c44c0 con 0x55ba4d517590 2024-09-19T05:07:32.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.950+0000 7fc74d600640 1 -- 172.21.15.43:0/3285904563 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fc7480c4660 con 0x55ba4d517590 2024-09-19T05:07:32.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.951+0000 7fc769284740 1 -- 172.21.15.43:0/3285904563 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc75800fc20 con 0x55ba4d517590 2024-09-19T05:07:32.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.951+0000 7fc74d600640 1 -- 172.21.15.43:0/3285904563 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc7480c4ba0 con 0x55ba4d517590 2024-09-19T05:07:32.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.951+0000 7fc769284740 10 client.25793 fetch_fsmap learned FSMap version 29 2024-09-19T05:07:32.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.951+0000 7fc769284740 1 -- 172.21.15.43:0/3285904563 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ba4d519ae0 con 0x55ba4d517590 2024-09-19T05:07:32.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.952+0000 7fc74d600640 1 -- 172.21.15.43:0/3285904563 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc75800fc20 con 0x55ba4d517590 2024-09-19T05:07:32.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.952+0000 7fc74d600640 1 -- 172.21.15.43:0/3285904563 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc7480c1220 con 0x55ba4d517590 2024-09-19T05:07:32.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.952+0000 7fc769284740 10 client.25793 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:07:32.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.952+0000 7fc769284740 10 client.25793 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T05:07:32.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.952+0000 7fc769284740 20 client.25793 populate_metadata read hostname 'smithi043' 2024-09-19T05:07:32.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.952+0000 7fc769284740 1 --2- 172.21.15.43:0/3285904563 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55ba4d5ae530 0x55ba4d5b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:32.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.952+0000 7fc769284740 4 client.25793 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:07:32.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.952+0000 7fc769284740 1 -- 172.21.15.43:0/3285904563 --> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ba4d484e60 con 0x55ba4d5ae530 2024-09-19T05:07:32.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.953+0000 7fc75f400640 1 --2- 172.21.15.43:0/3285904563 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55ba4d5ae530 0x55ba4d5b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:32.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.954+0000 7fc75f400640 1 --2- 172.21.15.43:0/3285904563 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55ba4d5ae530 0x55ba4d5b0950 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=23207769447de6ae server_cookie=e83e21bc6d3cf659 in_seq=0 out_seq=0 2024-09-19T05:07:32.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.954+0000 7fc74d600640 10 client.25793 ms_handle_connect on v2:172.21.15.96:6836/3978388286 2024-09-19T05:07:32.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.954+0000 7fc74d600640 1 -- 172.21.15.43:0/3285904563 <== mds.3 v2:172.21.15.96:6836/3978388286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ba4d484e60 con 0x55ba4d5ae530 2024-09-19T05:07:32.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.954+0000 7fc74d600640 10 client.25793 handle_command_reply: tid=0 2024-09-19T05:07:32.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.954+0000 7fc769284740 1 client.25793 shutdown 2024-09-19T05:07:32.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.955+0000 7fc769284740 1 -- 172.21.15.43:0/3285904563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba4d517590 msgr2=0x55ba4d53b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:32.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.955+0000 7fc769284740 1 --2- 172.21.15.43:0/3285904563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba4d517590 0x55ba4d53b3a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc7480bb9e0 tx=0x7fc7480092e0 comp rx=0 tx=0).stop 2024-09-19T05:07:32.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.956+0000 7fc769284740 1 -- 172.21.15.43:0/3285904563 shutdown_connections 2024-09-19T05:07:32.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.956+0000 7fc769284740 1 --2- 172.21.15.43:0/3285904563 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55ba4d5ae530 0x55ba4d5b0950 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:32.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.956+0000 7fc769284740 1 --2- 172.21.15.43:0/3285904563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba4d521e50 0x55ba4d537e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:32.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.956+0000 7fc769284740 1 --2- 172.21.15.43:0/3285904563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba4d517590 0x55ba4d53b3a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:32.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.956+0000 7fc769284740 1 --2- 172.21.15.43:0/3285904563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba4d516c70 0x55ba4d53ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:32.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.956+0000 7fc769284740 1 -- 172.21.15.43:0/3285904563 >> 172.21.15.43:0/3285904563 conn(0x55ba4d513200 msgr2=0x55ba4d511b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:32.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.956+0000 7fc74d600640 0 client.25793 ms_handle_reset on v2:172.21.15.96:6836/3978388286 2024-09-19T05:07:32.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.956+0000 7fc769284740 1 -- 172.21.15.43:0/3285904563 shutdown_connections 2024-09-19T05:07:32.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.957+0000 7fc769284740 1 -- 172.21.15.43:0/3285904563 wait complete. 2024-09-19T05:07:32.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.957+0000 7fc769284740 20 client.25793 trim_cache size 0 max 16384 2024-09-19T05:07:32.960 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:07:32.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.959+0000 7fc766800640 1 -- 172.21.15.43:0/1401142248 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fc7440a3630 msgr2=0x7fc7440a5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:32.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.959+0000 7fc766800640 1 --2- 172.21.15.43:0/1401142248 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fc7440a3630 0x7fc7440a5af0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc758007310 tx=0x7fc758007ac0 comp rx=0 tx=0).stop 2024-09-19T05:07:32.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.959+0000 7fc766800640 1 -- 172.21.15.43:0/1401142248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7601004e0 msgr2=0x7fc7601a4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:32.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.959+0000 7fc766800640 1 --2- 172.21.15.43:0/1401142248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7601004e0 0x7fc7601a4ca0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc74802a980 tx=0x7fc748003c00 comp rx=0 tx=0).stop 2024-09-19T05:07:32.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.960+0000 7fc766800640 1 -- 172.21.15.43:0/1401142248 shutdown_connections 2024-09-19T05:07:32.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.960+0000 7fc766800640 1 --2- 172.21.15.43:0/1401142248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc760102990 0x7fc76019e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:32.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.960+0000 7fc766800640 1 --2- 172.21.15.43:0/1401142248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7601004e0 0x7fc7601a4ca0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:32.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.960+0000 7fc766800640 1 --2- 172.21.15.43:0/1401142248 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fc7440a3630 0x7fc7440a5af0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:32.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.960+0000 7fc766800640 1 --2- 172.21.15.43:0/1401142248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7600ffb60 0x7fc7601a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:32.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.960+0000 7fc766800640 1 -- 172.21.15.43:0/1401142248 >> 172.21.15.43:0/1401142248 conn(0x7fc7600fba90 msgr2=0x7fc7600fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:32.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.961+0000 7fc766800640 1 -- 172.21.15.43:0/1401142248 shutdown_connections 2024-09-19T05:07:32.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:32.961+0000 7fc766800640 1 -- 172.21.15.43:0/1401142248 wait complete. 2024-09-19T05:07:34.323 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:07:34 smithi043 ceph-mon[30800]: pgmap v90: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 212 MiB/s wr, 61 op/s 2024-09-19T05:07:34.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:07:34 smithi151 ceph-mon[35425]: pgmap v90: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 212 MiB/s wr, 61 op/s 2024-09-19T05:07:34.415 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T05:07:34.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:07:34 smithi096 ceph-mon[35085]: pgmap v90: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 212 MiB/s wr, 61 op/s 2024-09-19T05:07:35.286 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:07:36.295 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:07:36 smithi043 ceph-mon[30800]: pgmap v91: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 307 MiB/s wr, 90 op/s 2024-09-19T05:07:36.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.291+0000 7f3800e00640 1 -- 172.21.15.43:0/2058561278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37fc1047e0 msgr2=0x7f37fc108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:36.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.291+0000 7f3800e00640 1 --2- 172.21.15.43:0/2058561278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37fc1047e0 0x7f37fc108830 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f37e8009900 tx=0x7f37e802a410 comp rx=0 tx=0).stop 2024-09-19T05:07:36.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.292+0000 7f3800e00640 1 -- 172.21.15.43:0/2058561278 shutdown_connections 2024-09-19T05:07:36.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.292+0000 7f3800e00640 1 --2- 172.21.15.43:0/2058561278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37fc108f60 0x7f37fc110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.292+0000 7f3800e00640 1 --2- 172.21.15.43:0/2058561278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37fc1047e0 0x7f37fc108830 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.292+0000 7f3800e00640 1 --2- 172.21.15.43:0/2058561278 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f37fc103e30 0x7f37fc104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.292+0000 7f3800e00640 1 -- 172.21.15.43:0/2058561278 >> 172.21.15.43:0/2058561278 conn(0x7f37fc0ff910 msgr2=0x7f37fc101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:36.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.294+0000 7f3800e00640 1 -- 172.21.15.43:0/2058561278 shutdown_connections 2024-09-19T05:07:36.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.294+0000 7f3800e00640 1 -- 172.21.15.43:0/2058561278 wait complete. 2024-09-19T05:07:36.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.296+0000 7f3800e00640 1 Processor -- start 2024-09-19T05:07:36.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.296+0000 7f3800e00640 1 -- start start 2024-09-19T05:07:36.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.297+0000 7f3800e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37fc103e30 0x7f37fc110950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:36.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.297+0000 7f3800e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37fc1047e0 0x7f37fc10da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:36.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.297+0000 7f3800e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f37fc108f60 0x7f37fc10df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:36.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.297+0000 7f3800e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f37fc114050 con 0x7f37fc103e30 2024-09-19T05:07:36.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.297+0000 7f3800e00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f37fc113ed0 con 0x7f37fc108f60 2024-09-19T05:07:36.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.297+0000 7f3800e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f37fc1141d0 con 0x7f37fc1047e0 2024-09-19T05:07:36.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.298+0000 7f37fb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37fc103e30 0x7f37fc110950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:36.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.298+0000 7f37fb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37fc103e30 0x7f37fc110950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50622/0 (socket says 172.21.15.43:50622) 2024-09-19T05:07:36.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.298+0000 7f37fb400640 1 -- 172.21.15.43:0/2454433926 learned_addr learned my addr 172.21.15.43:0/2454433926 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:36.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.299+0000 7f37faa00640 1 --2- 172.21.15.43:0/2454433926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37fc1047e0 0x7f37fc10da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:36.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.299+0000 7f37fbe00640 1 --2- 172.21.15.43:0/2454433926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f37fc108f60 0x7f37fc10df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:36.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.299+0000 7f37fb400640 1 -- 172.21.15.43:0/2454433926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37fc1047e0 msgr2=0x7f37fc10da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:36.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.299+0000 7f37fb400640 1 --2- 172.21.15.43:0/2454433926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37fc1047e0 0x7f37fc10da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.299+0000 7f37fb400640 1 -- 172.21.15.43:0/2454433926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f37fc108f60 msgr2=0x7f37fc10df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:36.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.299+0000 7f37fb400640 1 --2- 172.21.15.43:0/2454433926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f37fc108f60 0x7f37fc10df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.299+0000 7f37fb400640 1 -- 172.21.15.43:0/2454433926 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37fc10e800 con 0x7f37fc103e30 2024-09-19T05:07:36.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.300+0000 7f37fb400640 1 --2- 172.21.15.43:0/2454433926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37fc103e30 0x7f37fc110950 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f37ec00d950 tx=0x7f37ec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:36.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.300+0000 7f37f3e00640 1 -- 172.21.15.43:0/2454433926 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ec017070 con 0x7f37fc103e30 2024-09-19T05:07:36.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.300+0000 7f3800e00640 1 -- 172.21.15.43:0/2454433926 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37fc1a0690 con 0x7f37fc103e30 2024-09-19T05:07:36.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.300+0000 7f37f3e00640 1 -- 172.21.15.43:0/2454433926 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37ec009c30 con 0x7f37fc103e30 2024-09-19T05:07:36.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.300+0000 7f37f3e00640 1 -- 172.21.15.43:0/2454433926 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ec017070 con 0x7f37fc103e30 2024-09-19T05:07:36.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.302+0000 7f3800e00640 1 -- 172.21.15.43:0/2454433926 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37fc1a0c20 con 0x7f37fc103e30 2024-09-19T05:07:36.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.305+0000 7f37f3e00640 1 -- 172.21.15.43:0/2454433926 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f37ec0109d0 con 0x7f37fc103e30 2024-09-19T05:07:36.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.306+0000 7f3804e25740 1 Processor -- start 2024-09-19T05:07:36.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.307+0000 7f3804e25740 1 -- start start 2024-09-19T05:07:36.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.307+0000 7f3804e25740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632f5405e40 0x5632f5406220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:36.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.307+0000 7f3804e25740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632f5406760 0x5632f5410ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:36.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.307+0000 7f3804e25740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5632f5411020 0x5632f5413410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:36.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.307+0000 7f3804e25740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5632f536c3e0 con 0x5632f5405e40 2024-09-19T05:07:36.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.307+0000 7f3804e25740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5632f5313d80 con 0x5632f5406760 2024-09-19T05:07:36.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.307+0000 7f3804e25740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5632f53683a0 con 0x5632f5411020 2024-09-19T05:07:36.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.307+0000 7f37f3e00640 1 --2- 172.21.15.43:0/2454433926 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f37e00a36b0 0x7f37e00a5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:36.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.308+0000 7f37fbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5632f5411020 0x5632f5413410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:36.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.308+0000 7f37fbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632f5405e40 0x5632f5406220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:36.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.308+0000 7f37fbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632f5405e40 0x5632f5406220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50636/0 (socket says 172.21.15.43:50636) 2024-09-19T05:07:36.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.308+0000 7f37fbe00640 1 -- 172.21.15.43:0/3735119276 learned_addr learned my addr 172.21.15.43:0/3735119276 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:36.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.308+0000 7f37fb400640 1 --2- 172.21.15.43:0/3735119276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632f5406760 0x5632f5410ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:36.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.308+0000 7f37faa00640 1 --2- 172.21.15.43:0/2454433926 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f37e00a36b0 0x7f37e00a5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:36.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.309+0000 7f37f3e00640 1 -- 172.21.15.43:0/2454433926 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f37ec0cef20 con 0x7f37fc103e30 2024-09-19T05:07:36.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.310+0000 7f37fbe00640 1 -- 172.21.15.43:0/3735119276 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5632f5411020 msgr2=0x5632f5413410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:36.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.310+0000 7f37fbe00640 1 --2- 172.21.15.43:0/3735119276 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5632f5411020 0x5632f5413410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.310+0000 7f37fbe00640 1 -- 172.21.15.43:0/3735119276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632f5406760 msgr2=0x5632f5410ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:36.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.310+0000 7f37fbe00640 1 --2- 172.21.15.43:0/3735119276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632f5406760 0x5632f5410ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.310+0000 7f37fbe00640 1 -- 172.21.15.43:0/3735119276 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632f5332ae0 con 0x5632f5405e40 2024-09-19T05:07:36.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.310+0000 7f37fb400640 1 --2- 172.21.15.43:0/3735119276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632f5406760 0x5632f5410ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:07:36.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.311+0000 7f37fbe00640 1 --2- 172.21.15.43:0/3735119276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632f5405e40 0x5632f5406220 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f37f4006bc0 tx=0x7f37f4005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ae2359ca6daf58 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:36.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.311+0000 7f37faa00640 1 --2- 172.21.15.43:0/2454433926 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f37e00a36b0 0x7f37e00a5b70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f37e80096c0 tx=0x7f37e8005fd0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:36.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.311+0000 7f37f0c00640 1 -- 172.21.15.43:0/3735119276 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f4003c10 con 0x5632f5405e40 2024-09-19T05:07:36.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.312+0000 7f37f0c00640 1 -- 172.21.15.43:0/3735119276 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37f4003db0 con 0x5632f5405e40 2024-09-19T05:07:36.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.312+0000 7f37f0c00640 1 -- 172.21.15.43:0/3735119276 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f4011a50 con 0x5632f5405e40 2024-09-19T05:07:36.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.313+0000 7f3804e25740 1 -- 172.21.15.43:0/3735119276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632f5405e40 msgr2=0x5632f5406220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:36.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.313+0000 7f3804e25740 1 --2- 172.21.15.43:0/3735119276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632f5405e40 0x5632f5406220 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f37f4006bc0 tx=0x7f37f4005da0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.313+0000 7f3804e25740 1 -- 172.21.15.43:0/3735119276 shutdown_connections 2024-09-19T05:07:36.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.313+0000 7f3804e25740 1 --2- 172.21.15.43:0/3735119276 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5632f5411020 0x5632f5413410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.313+0000 7f3804e25740 1 --2- 172.21.15.43:0/3735119276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632f5406760 0x5632f5410ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.313+0000 7f3804e25740 1 --2- 172.21.15.43:0/3735119276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632f5405e40 0x5632f5406220 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.313+0000 7f3804e25740 1 -- 172.21.15.43:0/3735119276 >> 172.21.15.43:0/3735119276 conn(0x5632f54023d0 msgr2=0x5632f54027b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:36.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.314+0000 7f3804e25740 1 -- 172.21.15.43:0/3735119276 shutdown_connections 2024-09-19T05:07:36.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.315+0000 7f3804e25740 1 -- 172.21.15.43:0/3735119276 wait complete. 2024-09-19T05:07:36.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.315+0000 7f3804e25740 1 -- start start 2024-09-19T05:07:36.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.316+0000 7f3804e25740 1 Processor -- start 2024-09-19T05:07:36.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.316+0000 7f3804e25740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632f5405e40 0x5632f540deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:36.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.316+0000 7f3804e25740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632f5406760 0x5632f540e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:36.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.316+0000 7f3804e25740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5632f5411020 0x5632f540e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:36.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.316+0000 7f3804e25740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5632f5413bf0 con 0x5632f5406760 2024-09-19T05:07:36.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.316+0000 7f3804e25740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5632f53270f0 con 0x5632f5405e40 2024-09-19T05:07:36.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.316+0000 7f3804e25740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5632f5413d70 con 0x5632f5411020 2024-09-19T05:07:36.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.316+0000 7f3804e25740 10 client.0 _ll_register_callbacks cb 0x5632f530ebd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:36.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.317+0000 7f37fbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5632f5411020 0x5632f540e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:36.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.317+0000 7f37faa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632f5406760 0x5632f540e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:36.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.317+0000 7f37fbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5632f5411020 0x5632f540e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:59470/0 (socket says 172.21.15.43:59470) 2024-09-19T05:07:36.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.317+0000 7f37fbe00640 1 -- 172.21.15.43:0/3628435589 learned_addr learned my addr 172.21.15.43:0/3628435589 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:36.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.317+0000 7f37fb400640 1 --2- 172.21.15.43:0/3628435589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632f5405e40 0x5632f540deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:36.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.317+0000 7f37fbe00640 1 -- 172.21.15.43:0/3628435589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632f5405e40 msgr2=0x5632f540deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:36.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.317+0000 7f37fbe00640 1 --2- 172.21.15.43:0/3628435589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632f5405e40 0x5632f540deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.317+0000 7f37fbe00640 1 -- 172.21.15.43:0/3628435589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632f5406760 msgr2=0x5632f540e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:36.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.317+0000 7f37fbe00640 1 --2- 172.21.15.43:0/3628435589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632f5406760 0x5632f540e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.317+0000 7f37fbe00640 1 -- 172.21.15.43:0/3628435589 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632f50dd3c0 con 0x5632f5411020 2024-09-19T05:07:36.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.317+0000 7f37fbe00640 1 --2- 172.21.15.43:0/3628435589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5632f5411020 0x5632f540e930 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f37f4003fa0 tx=0x7f37f4004a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bdd9b605495720 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:36.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.317+0000 7f37e5600640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:07:36.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.318+0000 7f37e5600640 1 -- 172.21.15.43:0/3628435589 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f4027020 con 0x5632f5411020 2024-09-19T05:07:36.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.318+0000 7f37e5600640 1 -- 172.21.15.43:0/3628435589 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f37f4004430 con 0x5632f5411020 2024-09-19T05:07:36.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.318+0000 7f3804e25740 1 -- 172.21.15.43:0/3628435589 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37f4003db0 con 0x5632f5411020 2024-09-19T05:07:36.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.318+0000 7f37e5600640 1 -- 172.21.15.43:0/3628435589 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f37f4032050 con 0x5632f5411020 2024-09-19T05:07:36.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.318+0000 7f3804e25740 10 client.25762 fetch_fsmap learned FSMap version 29 2024-09-19T05:07:36.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.318+0000 7f3804e25740 1 -- 172.21.15.43:0/3628435589 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5632f541fb30 con 0x5632f5411020 2024-09-19T05:07:36.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.318+0000 7f37e5600640 1 -- 172.21.15.43:0/3628435589 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37f40062d0 con 0x5632f5411020 2024-09-19T05:07:36.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.319+0000 7f37e5600640 1 -- 172.21.15.43:0/3628435589 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f37f40235a0 con 0x5632f5411020 2024-09-19T05:07:36.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.319+0000 7f3804e25740 10 client.25762 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:07:36.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.319+0000 7f3804e25740 10 client.25762 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T05:07:36.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.319+0000 7f3804e25740 20 client.25762 populate_metadata read hostname 'smithi043' 2024-09-19T05:07:36.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.319+0000 7f3804e25740 1 --2- 172.21.15.43:0/3628435589 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x5632f5420f90 0x5632f5423380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:36.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.319+0000 7f3804e25740 4 client.25762 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:07:36.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.319+0000 7f3804e25740 1 -- 172.21.15.43:0/3628435589 --> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f37f4003db0 con 0x5632f5420f90 2024-09-19T05:07:36.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.320+0000 7f37fb400640 1 --2- 172.21.15.43:0/3628435589 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x5632f5420f90 0x5632f5423380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:36.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.320+0000 7f37fb400640 1 --2- 172.21.15.43:0/3628435589 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x5632f5420f90 0x5632f5423380 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f78dbccc1698c708 server_cookie=d677a6dd38fd0102 in_seq=0 out_seq=0 2024-09-19T05:07:36.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.320+0000 7f37e5600640 10 client.25762 ms_handle_connect on v2:172.21.15.151:6838/3794104313 2024-09-19T05:07:36.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.321+0000 7f37e5600640 1 -- 172.21.15.43:0/3628435589 <== mds.2 v2:172.21.15.151:6838/3794104313 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f37f4003db0 con 0x5632f5420f90 2024-09-19T05:07:36.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.321+0000 7f37e5600640 10 client.25762 handle_command_reply: tid=0 2024-09-19T05:07:36.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.321+0000 7f3804e25740 1 client.25762 shutdown 2024-09-19T05:07:36.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.321+0000 7f3804e25740 1 -- 172.21.15.43:0/3628435589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5632f5411020 msgr2=0x5632f540e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:36.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.321+0000 7f3804e25740 1 --2- 172.21.15.43:0/3628435589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5632f5411020 0x5632f540e930 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f37f4003fa0 tx=0x7f37f4004a00 comp rx=0 tx=0).stop 2024-09-19T05:07:36.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.321+0000 7f3804e25740 1 -- 172.21.15.43:0/3628435589 shutdown_connections 2024-09-19T05:07:36.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.321+0000 7f3804e25740 1 --2- 172.21.15.43:0/3628435589 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x5632f5420f90 0x5632f5423380 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.321+0000 7f3804e25740 1 --2- 172.21.15.43:0/3628435589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5632f5411020 0x5632f540e930 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.321+0000 7f3804e25740 1 --2- 172.21.15.43:0/3628435589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632f5406760 0x5632f540e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.321+0000 7f3804e25740 1 --2- 172.21.15.43:0/3628435589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632f5405e40 0x5632f540deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.321+0000 7f3804e25740 1 -- 172.21.15.43:0/3628435589 >> 172.21.15.43:0/3628435589 conn(0x5632f54023d0 msgr2=0x5632f5408100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:36.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.321+0000 7f37e5600640 0 client.25762 ms_handle_reset on v2:172.21.15.151:6838/3794104313 2024-09-19T05:07:36.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.322+0000 7f3804e25740 1 -- 172.21.15.43:0/3628435589 shutdown_connections 2024-09-19T05:07:36.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.322+0000 7f3804e25740 1 -- 172.21.15.43:0/3628435589 wait complete. 2024-09-19T05:07:36.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.323+0000 7f3804e25740 20 client.25762 trim_cache size 0 max 16384 2024-09-19T05:07:36.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.337+0000 7f3804e25740 1 Processor -- start 2024-09-19T05:07:36.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.337+0000 7f3804e25740 1 -- start start 2024-09-19T05:07:36.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.337+0000 7f3804e25740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632f5405e40 0x5632f54088f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:36.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.337+0000 7f3804e25740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5632f5406760 0x5632f540ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:36.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.337+0000 7f3804e25740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632f5411020 0x5632f5428920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:36.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.337+0000 7f3804e25740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f37fc1c4db0 con 0x5632f5411020 2024-09-19T05:07:36.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.337+0000 7f3804e25740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f37fc1c68f0 con 0x5632f5405e40 2024-09-19T05:07:36.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.337+0000 7f3804e25740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f37fc1c5480 con 0x5632f5406760 2024-09-19T05:07:36.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.338+0000 7f37fb400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632f5405e40 0x5632f54088f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:36.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.338+0000 7f37fbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632f5411020 0x5632f5428920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:36.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.338+0000 7f37fb400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632f5405e40 0x5632f54088f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:60642/0 (socket says 172.21.15.43:60642) 2024-09-19T05:07:36.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.338+0000 7f37fb400640 1 -- 172.21.15.43:0/40989992 learned_addr learned my addr 172.21.15.43:0/40989992 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:36.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.338+0000 7f37faa00640 1 --2- 172.21.15.43:0/40989992 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5632f5406760 0x5632f540ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:36.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.338+0000 7f37fb400640 1 -- 172.21.15.43:0/40989992 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5632f5406760 msgr2=0x5632f540ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:36.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.338+0000 7f37fb400640 1 --2- 172.21.15.43:0/40989992 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5632f5406760 0x5632f540ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.338+0000 7f37fb400640 1 -- 172.21.15.43:0/40989992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632f5411020 msgr2=0x5632f5428920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:36.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.338+0000 7f37fb400640 1 --2- 172.21.15.43:0/40989992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632f5411020 0x5632f5428920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.338+0000 7f37fb400640 1 -- 172.21.15.43:0/40989992 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632f540c420 con 0x5632f5405e40 2024-09-19T05:07:36.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.338+0000 7f37fb400640 1 --2- 172.21.15.43:0/40989992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632f5405e40 0x5632f54088f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f37ec097610 tx=0x7f37ec00bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8aba671d7ca840df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:36.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.339+0000 7f37e7e00640 1 -- 172.21.15.43:0/40989992 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ec020040 con 0x5632f5405e40 2024-09-19T05:07:36.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.339+0000 7f37e7e00640 1 -- 172.21.15.43:0/40989992 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37ec099250 con 0x5632f5405e40 2024-09-19T05:07:36.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.339+0000 7f37e7e00640 1 -- 172.21.15.43:0/40989992 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ec0bb0b0 con 0x5632f5405e40 2024-09-19T05:07:36.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.339+0000 7f3804e25740 1 -- 172.21.15.43:0/40989992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632f5405e40 msgr2=0x5632f54088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:36.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.339+0000 7f3804e25740 1 --2- 172.21.15.43:0/40989992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632f5405e40 0x5632f54088f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f37ec097610 tx=0x7f37ec00bf60 comp rx=0 tx=0).stop 2024-09-19T05:07:36.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.339+0000 7f3804e25740 1 -- 172.21.15.43:0/40989992 shutdown_connections 2024-09-19T05:07:36.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.339+0000 7f3804e25740 1 --2- 172.21.15.43:0/40989992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632f5411020 0x5632f5428920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.339+0000 7f3804e25740 1 --2- 172.21.15.43:0/40989992 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5632f5406760 0x5632f540ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.339+0000 7f3804e25740 1 --2- 172.21.15.43:0/40989992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632f5405e40 0x5632f54088f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.339+0000 7f3804e25740 1 -- 172.21.15.43:0/40989992 >> 172.21.15.43:0/40989992 conn(0x5632f54023d0 msgr2=0x5632f540c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:36.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.340+0000 7f3804e25740 1 -- 172.21.15.43:0/40989992 shutdown_connections 2024-09-19T05:07:36.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.340+0000 7f3804e25740 1 -- 172.21.15.43:0/40989992 wait complete. 2024-09-19T05:07:36.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.340+0000 7f3804e25740 1 -- start start 2024-09-19T05:07:36.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.340+0000 7f3804e25740 1 Processor -- start 2024-09-19T05:07:36.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.341+0000 7f3804e25740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5632f5405e40 0x5632f5418fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:36.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.341+0000 7f3804e25740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632f5406760 0x5632f5419500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:36.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.341+0000 7f3804e25740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632f5411020 0x5632f540a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:36.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.341+0000 7f37fb400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5632f5405e40 0x5632f5418fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:36.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.341+0000 7f37faa00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632f5406760 0x5632f5419500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:36.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.341+0000 7f3804e25740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f37fc1c6220 con 0x5632f5411020 2024-09-19T05:07:36.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.341+0000 7f3804e25740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f37fc1c5b50 con 0x5632f5406760 2024-09-19T05:07:36.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.341+0000 7f3804e25740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f37fc073460 con 0x5632f5405e40 2024-09-19T05:07:36.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.341+0000 7f37faa00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632f5406760 0x5632f5419500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:60656/0 (socket says 172.21.15.43:60656) 2024-09-19T05:07:36.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.341+0000 7f37faa00640 1 -- 172.21.15.43:0/2438576634 learned_addr learned my addr 172.21.15.43:0/2438576634 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:36.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.341+0000 7f3804e25740 10 client.0 _ll_register_callbacks cb 0x7f37f4008070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:36.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.342+0000 7f37fbe00640 1 --2- 172.21.15.43:0/2438576634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632f5411020 0x5632f540a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:36.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.342+0000 7f37faa00640 1 -- 172.21.15.43:0/2438576634 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5632f5405e40 msgr2=0x5632f5418fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:36.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.342+0000 7f37faa00640 1 --2- 172.21.15.43:0/2438576634 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5632f5405e40 0x5632f5418fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.342+0000 7f37faa00640 1 -- 172.21.15.43:0/2438576634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632f5411020 msgr2=0x5632f540a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:36.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.342+0000 7f37faa00640 1 --2- 172.21.15.43:0/2438576634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632f5411020 0x5632f540a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.342+0000 7f37faa00640 1 -- 172.21.15.43:0/2438576634 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632f540be30 con 0x5632f5406760 2024-09-19T05:07:36.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.342+0000 7f37fbe00640 1 --2- 172.21.15.43:0/2438576634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632f5411020 0x5632f540a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:07:36.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.342+0000 7f37faa00640 1 --2- 172.21.15.43:0/2438576634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632f5406760 0x5632f5419500 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f37e8009900 tx=0x7f37e8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f5449fd303a7d91 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:36.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.342+0000 7f37e5600640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:07:36.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.343+0000 7f37e5600640 1 -- 172.21.15.43:0/2438576634 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e8035050 con 0x5632f5406760 2024-09-19T05:07:36.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.343+0000 7f37e5600640 1 -- 172.21.15.43:0/2438576634 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f37e8004ad0 con 0x5632f5406760 2024-09-19T05:07:36.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.343+0000 7f3804e25740 1 -- 172.21.15.43:0/2438576634 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37ec099250 con 0x5632f5406760 2024-09-19T05:07:36.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.343+0000 7f37e5600640 1 -- 172.21.15.43:0/2438576634 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f37e8004cd0 con 0x5632f5406760 2024-09-19T05:07:36.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.343+0000 7f3804e25740 10 client.25808 fetch_fsmap learned FSMap version 29 2024-09-19T05:07:36.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.343+0000 7f3804e25740 1 -- 172.21.15.43:0/2438576634 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5632f548ae20 con 0x5632f5406760 2024-09-19T05:07:36.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.343+0000 7f37e5600640 1 -- 172.21.15.43:0/2438576634 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37ec099250 con 0x5632f5406760 2024-09-19T05:07:36.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.344+0000 7f37e5600640 1 -- 172.21.15.43:0/2438576634 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f37e8003710 con 0x5632f5406760 2024-09-19T05:07:36.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.344+0000 7f3804e25740 10 client.25808 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:07:36.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.344+0000 7f3804e25740 10 client.25808 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T05:07:36.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.344+0000 7f3804e25740 20 client.25808 populate_metadata read hostname 'smithi043' 2024-09-19T05:07:36.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.344+0000 7f3804e25740 1 --2- 172.21.15.43:0/2438576634 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x5632f548b5b0 0x5632f548d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:36.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.344+0000 7f3804e25740 4 client.25808 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:07:36.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.344+0000 7f3804e25740 1 -- 172.21.15.43:0/2438576634 --> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5632f5379950 con 0x5632f548b5b0 2024-09-19T05:07:36.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.344+0000 7f37fb400640 1 --2- 172.21.15.43:0/2438576634 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x5632f548b5b0 0x5632f548d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:36.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.345+0000 7f37fb400640 1 --2- 172.21.15.43:0/2438576634 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x5632f548b5b0 0x5632f548d9d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d6b57620ebea7544 server_cookie=d83a1d643dd995f in_seq=0 out_seq=0 2024-09-19T05:07:36.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.345+0000 7f37e5600640 10 client.25808 ms_handle_connect on v2:172.21.15.151:6838/3794104313 2024-09-19T05:07:36.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.345+0000 7f37e5600640 1 -- 172.21.15.43:0/2438576634 <== mds.2 v2:172.21.15.151:6838/3794104313 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5632f5379950 con 0x5632f548b5b0 2024-09-19T05:07:36.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.345+0000 7f37e5600640 10 client.25808 handle_command_reply: tid=0 2024-09-19T05:07:36.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.345+0000 7f3804e25740 1 client.25808 shutdown 2024-09-19T05:07:36.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.345+0000 7f3804e25740 1 -- 172.21.15.43:0/2438576634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632f5406760 msgr2=0x5632f5419500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:36.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.345+0000 7f3804e25740 1 --2- 172.21.15.43:0/2438576634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632f5406760 0x5632f5419500 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f37e8009900 tx=0x7f37e8009260 comp rx=0 tx=0).stop 2024-09-19T05:07:36.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.345+0000 7f3804e25740 1 -- 172.21.15.43:0/2438576634 shutdown_connections 2024-09-19T05:07:36.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.345+0000 7f3804e25740 1 --2- 172.21.15.43:0/2438576634 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x5632f548b5b0 0x5632f548d9d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.345+0000 7f3804e25740 1 --2- 172.21.15.43:0/2438576634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5632f5411020 0x5632f540a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.345+0000 7f3804e25740 1 --2- 172.21.15.43:0/2438576634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632f5406760 0x5632f5419500 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.345+0000 7f3804e25740 1 --2- 172.21.15.43:0/2438576634 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5632f5405e40 0x5632f5418fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.345+0000 7f3804e25740 1 -- 172.21.15.43:0/2438576634 >> 172.21.15.43:0/2438576634 conn(0x5632f54023d0 msgr2=0x5632f540cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:36.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.345+0000 7f37e5600640 0 client.25808 ms_handle_reset on v2:172.21.15.151:6838/3794104313 2024-09-19T05:07:36.352 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:07:36.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.346+0000 7f3804e25740 1 -- 172.21.15.43:0/2438576634 shutdown_connections 2024-09-19T05:07:36.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.346+0000 7f3804e25740 1 -- 172.21.15.43:0/2438576634 wait complete. 2024-09-19T05:07:36.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.346+0000 7f3804e25740 20 client.25808 trim_cache size 0 max 16384 2024-09-19T05:07:36.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.346+0000 7f3800e00640 1 -- 172.21.15.43:0/2454433926 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f37e00a36b0 msgr2=0x7f37e00a5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:36.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.346+0000 7f3800e00640 1 --2- 172.21.15.43:0/2454433926 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f37e00a36b0 0x7f37e00a5b70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f37e80096c0 tx=0x7f37e8005fd0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.347+0000 7f3800e00640 1 -- 172.21.15.43:0/2454433926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37fc103e30 msgr2=0x7f37fc110950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:36.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.347+0000 7f3800e00640 1 --2- 172.21.15.43:0/2454433926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37fc103e30 0x7f37fc110950 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f37ec00d950 tx=0x7f37ec00de10 comp rx=0 tx=0).stop 2024-09-19T05:07:36.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.347+0000 7f3800e00640 1 -- 172.21.15.43:0/2454433926 shutdown_connections 2024-09-19T05:07:36.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.347+0000 7f3800e00640 1 --2- 172.21.15.43:0/2454433926 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f37e00a36b0 0x7f37e00a5b70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.347+0000 7f3800e00640 1 --2- 172.21.15.43:0/2454433926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f37fc108f60 0x7f37fc10df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.347+0000 7f3800e00640 1 --2- 172.21.15.43:0/2454433926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37fc1047e0 0x7f37fc10da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.347+0000 7f3800e00640 1 --2- 172.21.15.43:0/2454433926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37fc103e30 0x7f37fc110950 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:36.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.347+0000 7f3800e00640 1 -- 172.21.15.43:0/2454433926 >> 172.21.15.43:0/2454433926 conn(0x7f37fc0ff910 msgr2=0x7f37fc100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:36.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.347+0000 7f3800e00640 1 -- 172.21.15.43:0/2454433926 shutdown_connections 2024-09-19T05:07:36.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:36.347+0000 7f3800e00640 1 -- 172.21.15.43:0/2454433926 wait complete. 2024-09-19T05:07:36.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:07:36 smithi151 ceph-mon[35425]: pgmap v91: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 307 MiB/s wr, 90 op/s 2024-09-19T05:07:36.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:07:36 smithi096 ceph-mon[35085]: pgmap v91: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 307 MiB/s wr, 90 op/s 2024-09-19T05:07:38.035 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T05:07:38.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:07:38 smithi043 ceph-mon[30800]: pgmap v92: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 155 MiB/s wr, 48 op/s 2024-09-19T05:07:38.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:07:38 smithi151 ceph-mon[35425]: pgmap v92: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 155 MiB/s wr, 48 op/s 2024-09-19T05:07:38.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:07:38 smithi096 ceph-mon[35085]: pgmap v92: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 155 MiB/s wr, 48 op/s 2024-09-19T05:07:38.713 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:07:39.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.488+0000 7f6689c00640 1 -- 172.21.15.43:0/105504162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6684109830 msgr2=0x7f6684109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:39.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.488+0000 7f6689c00640 1 --2- 172.21.15.43:0/105504162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6684109830 0x7f6684109c90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6670009860 tx=0x7f667002a340 comp rx=0 tx=0).stop 2024-09-19T05:07:39.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.491+0000 7f6689c00640 1 -- 172.21.15.43:0/105504162 shutdown_connections 2024-09-19T05:07:39.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.491+0000 7f6689c00640 1 --2- 172.21.15.43:0/105504162 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f668411b140 0x7f668411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.491+0000 7f6689c00640 1 --2- 172.21.15.43:0/105504162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6684109830 0x7f6684109c90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.491+0000 7f6689c00640 1 --2- 172.21.15.43:0/105504162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6684109450 0x7f6684110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.491+0000 7f6689c00640 1 -- 172.21.15.43:0/105504162 >> 172.21.15.43:0/105504162 conn(0x7f668406e140 msgr2=0x7f668406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:39.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.492+0000 7f6689c00640 1 -- 172.21.15.43:0/105504162 shutdown_connections 2024-09-19T05:07:39.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.493+0000 7f6689c00640 1 -- 172.21.15.43:0/105504162 wait complete. 2024-09-19T05:07:39.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.495+0000 7f6689c00640 1 Processor -- start 2024-09-19T05:07:39.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.495+0000 7f6689c00640 1 -- start start 2024-09-19T05:07:39.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.496+0000 7f6689c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6684109450 0x7f66841a7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:39.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.496+0000 7f6689c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6684109830 0x7f66841a77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:39.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.496+0000 7f6689c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f668411b140 0x7f66841abb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:39.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.496+0000 7f6689c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6684120da0 con 0x7f6684109830 2024-09-19T05:07:39.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.496+0000 7f6689c00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6684120c20 con 0x7f6684109450 2024-09-19T05:07:39.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.496+0000 7f6689c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6684120f20 con 0x7f668411b140 2024-09-19T05:07:39.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.496+0000 7f6683400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f668411b140 0x7f66841abb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:39.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.496+0000 7f6682a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6684109450 0x7f66841a7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:39.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.496+0000 7f6683400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f668411b140 0x7f66841abb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:59510/0 (socket says 172.21.15.43:59510) 2024-09-19T05:07:39.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.496+0000 7f6683400640 1 -- 172.21.15.43:0/3217054615 learned_addr learned my addr 172.21.15.43:0/3217054615 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:39.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.497+0000 7f6682a00640 1 -- 172.21.15.43:0/3217054615 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f668411b140 msgr2=0x7f66841abb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:39.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.497+0000 7f6682a00640 1 --2- 172.21.15.43:0/3217054615 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f668411b140 0x7f66841abb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.497+0000 7f6682a00640 1 -- 172.21.15.43:0/3217054615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6684109830 msgr2=0x7f66841a77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:39.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.497+0000 7f6682a00640 1 --2- 172.21.15.43:0/3217054615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6684109830 0x7f66841a77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.497+0000 7f6682a00640 1 -- 172.21.15.43:0/3217054615 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66841ac230 con 0x7f6684109450 2024-09-19T05:07:39.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.498+0000 7f6682a00640 1 --2- 172.21.15.43:0/3217054615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6684109450 0x7f66841a7280 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f667400c970 tx=0x7f667400ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:39.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.498+0000 7f667b400640 1 -- 172.21.15.43:0/3217054615 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6674007bb0 con 0x7f6684109450 2024-09-19T05:07:39.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.498+0000 7f667b400640 1 -- 172.21.15.43:0/3217054615 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6674007d50 con 0x7f6684109450 2024-09-19T05:07:39.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.499+0000 7f667b400640 1 -- 172.21.15.43:0/3217054615 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667400f450 con 0x7f6684109450 2024-09-19T05:07:39.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.499+0000 7f6689c00640 1 -- 172.21.15.43:0/3217054615 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66841ac520 con 0x7f6684109450 2024-09-19T05:07:39.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.499+0000 7f6689c00640 1 -- 172.21.15.43:0/3217054615 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66841b3e00 con 0x7f6684109450 2024-09-19T05:07:39.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.504+0000 7f668c813740 1 Processor -- start 2024-09-19T05:07:39.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.505+0000 7f668c813740 1 -- start start 2024-09-19T05:07:39.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.509+0000 7f668c813740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56337afb0c70 0x56337afb1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:39.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.509+0000 7f667b400640 1 -- 172.21.15.43:0/3217054615 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6674004eb0 con 0x7f6684109450 2024-09-19T05:07:39.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.510+0000 7f6683400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56337afb0c70 0x56337afb1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:39.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.510+0000 7f668c813740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56337afb1590 0x56337afbb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:39.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.510+0000 7f668c813740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56337afbbe50 0x56337afbe240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:39.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.510+0000 7f668c813740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56337af173e0 con 0x56337afb1590 2024-09-19T05:07:39.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.510+0000 7f668c813740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56337aebed80 con 0x56337afb0c70 2024-09-19T05:07:39.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.510+0000 7f668c813740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56337af133a0 con 0x56337afbbe50 2024-09-19T05:07:39.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.510+0000 7f6683400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56337afb0c70 0x56337afb1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:60680/0 (socket says 172.21.15.43:60680) 2024-09-19T05:07:39.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.510+0000 7f6683400640 1 -- 172.21.15.43:0/1294986380 learned_addr learned my addr 172.21.15.43:0/1294986380 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:39.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.510+0000 7f6683400640 1 -- 172.21.15.43:0/1294986380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56337afbbe50 msgr2=0x56337afbe240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:39.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.510+0000 7f6683400640 1 --2- 172.21.15.43:0/1294986380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56337afbbe50 0x56337afbe240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.510+0000 7f6683400640 1 -- 172.21.15.43:0/1294986380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56337afb1590 msgr2=0x56337afbb910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T05:07:39.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.510+0000 7f6683400640 1 --2- 172.21.15.43:0/1294986380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56337afb1590 0x56337afbb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.510+0000 7f6683400640 1 -- 172.21.15.43:0/1294986380 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56337aeddae0 con 0x56337afb0c70 2024-09-19T05:07:39.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.511+0000 7f6683400640 1 --2- 172.21.15.43:0/1294986380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56337afb0c70 0x56337afb1050 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f667c00e590 tx=0x7f667c0095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a75b2dbe08eb193 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:39.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.511+0000 7f667b400640 1 --2- 172.21.15.43:0/3217054615 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f66680a34b0 0x7f66680a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:39.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.511+0000 7f666fe00640 1 -- 172.21.15.43:0/1294986380 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c003970 con 0x56337afb0c70 2024-09-19T05:07:39.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.512+0000 7f667b400640 1 -- 172.21.15.43:0/3217054615 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f66740cfc50 con 0x7f6684109450 2024-09-19T05:07:39.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.512+0000 7f666fe00640 1 -- 172.21.15.43:0/1294986380 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f667c00d810 con 0x56337afb0c70 2024-09-19T05:07:39.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.513+0000 7f6682a00640 1 --2- 172.21.15.43:0/3217054615 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f66680a34b0 0x7f66680a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:39.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.514+0000 7f668c813740 1 -- 172.21.15.43:0/1294986380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56337afb0c70 msgr2=0x56337afb1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:39.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.515+0000 7f666fe00640 1 -- 172.21.15.43:0/1294986380 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c003970 con 0x56337afb0c70 2024-09-19T05:07:39.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.515+0000 7f668c813740 1 --2- 172.21.15.43:0/1294986380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56337afb0c70 0x56337afb1050 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f667c00e590 tx=0x7f667c0095c0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.515+0000 7f668c813740 1 -- 172.21.15.43:0/1294986380 shutdown_connections 2024-09-19T05:07:39.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.515+0000 7f668c813740 1 --2- 172.21.15.43:0/1294986380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56337afbbe50 0x56337afbe240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.515+0000 7f668c813740 1 --2- 172.21.15.43:0/1294986380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56337afb1590 0x56337afbb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.515+0000 7f668c813740 1 --2- 172.21.15.43:0/1294986380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56337afb0c70 0x56337afb1050 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.515+0000 7f668c813740 1 -- 172.21.15.43:0/1294986380 >> 172.21.15.43:0/1294986380 conn(0x56337afad200 msgr2=0x56337afad5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:39.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.516+0000 7f6682a00640 1 --2- 172.21.15.43:0/3217054615 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f66680a34b0 0x7f66680a5970 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x56337afae060 tx=0x7f6674098180 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:39.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.516+0000 7f668c813740 1 -- 172.21.15.43:0/1294986380 shutdown_connections 2024-09-19T05:07:39.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.517+0000 7f668c813740 1 -- 172.21.15.43:0/1294986380 wait complete. 2024-09-19T05:07:39.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.518+0000 7f668c813740 1 -- start start 2024-09-19T05:07:39.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.518+0000 7f668c813740 1 Processor -- start 2024-09-19T05:07:39.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.519+0000 7f668c813740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56337afb0c70 0x56337afbb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:39.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.520+0000 7f668c813740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56337afb1590 0x56337afb8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:39.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.520+0000 7f6683400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56337afb0c70 0x56337afbb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:39.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.520+0000 7f6683400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56337afb0c70 0x56337afbb880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:60688/0 (socket says 172.21.15.43:60688) 2024-09-19T05:07:39.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.520+0000 7f6683400640 1 -- 172.21.15.43:0/2881979993 learned_addr learned my addr 172.21.15.43:0/2881979993 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:39.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.520+0000 7f668c813740 1 --2- 172.21.15.43:0/2881979993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56337afbbe50 0x56337afb92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:39.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.520+0000 7f668c813740 1 -- 172.21.15.43:0/2881979993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56337afbea70 con 0x56337afb1590 2024-09-19T05:07:39.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.520+0000 7f668c813740 1 -- 172.21.15.43:0/2881979993 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56337aed20f0 con 0x56337afb0c70 2024-09-19T05:07:39.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.520+0000 7f668c813740 1 -- 172.21.15.43:0/2881979993 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56337afbebf0 con 0x56337afbbe50 2024-09-19T05:07:39.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.521+0000 7f6683400640 1 --2- 172.21.15.43:0/2881979993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56337afbbe50 0x56337afb92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:39.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.521+0000 7f6682000640 1 --2- 172.21.15.43:0/2881979993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56337afb1590 0x56337afb8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:39.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.522+0000 7f6683400640 1 -- 172.21.15.43:0/2881979993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56337afbbe50 msgr2=0x56337afb92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:39.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.522+0000 7f6683400640 1 --2- 172.21.15.43:0/2881979993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56337afbbe50 0x56337afb92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.522+0000 7f6683400640 1 -- 172.21.15.43:0/2881979993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56337afb1590 msgr2=0x56337afb8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:39.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.522+0000 7f6683400640 1 --2- 172.21.15.43:0/2881979993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56337afb1590 0x56337afb8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.522+0000 7f6683400640 1 -- 172.21.15.43:0/2881979993 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56337ac883c0 con 0x56337afb0c70 2024-09-19T05:07:39.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.522+0000 7f668c813740 10 client.0 _ll_register_callbacks cb 0x56337aeb9bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:39.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.522+0000 7f668c813740 1 -- 172.21.15.43:0/2881979993 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f667c00d810 con 0x56337afb0c70 2024-09-19T05:07:39.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.522+0000 7f6683400640 1 --2- 172.21.15.43:0/2881979993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56337afb0c70 0x56337afbb880 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f667c00eee0 tx=0x7f667c009e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3da4c0b92069316 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:39.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.522+0000 7f666cc00640 10 client.25820 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:07:39.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.523+0000 7f666cc00640 1 -- 172.21.15.43:0/2881979993 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c03c020 con 0x56337afb0c70 2024-09-19T05:07:39.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.523+0000 7f666cc00640 1 -- 172.21.15.43:0/2881979993 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f667c004780 con 0x56337afb0c70 2024-09-19T05:07:39.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.523+0000 7f666cc00640 1 -- 172.21.15.43:0/2881979993 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f667c003970 con 0x56337afb0c70 2024-09-19T05:07:39.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.523+0000 7f668c813740 10 client.25820 fetch_fsmap learned FSMap version 29 2024-09-19T05:07:39.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.523+0000 7f668c813740 1 -- 172.21.15.43:0/2881979993 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({fsmap=29}) -- 0x56337afca860 con 0x56337afb0c70 2024-09-19T05:07:39.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.524+0000 7f666cc00640 1 -- 172.21.15.43:0/2881979993 <== mon.2 v2:172.21.15.96:3300/0 4 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f667c00b040 con 0x56337afb0c70 2024-09-19T05:07:39.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.524+0000 7f668c813740 10 client.25820 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:07:39.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.524+0000 7f668c813740 10 client.25820 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-19T05:07:39.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.524+0000 7f668c813740 20 client.25820 populate_metadata read hostname 'smithi043' 2024-09-19T05:07:39.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.524+0000 7f668c813740 1 --2- 172.21.15.43:0/2881979993 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x56337afcbbb0 0x56337afcdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:39.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.525+0000 7f668c813740 4 client.25820 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:07:39.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.525+0000 7f668c813740 1 -- 172.21.15.43:0/2881979993 --> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56337af24950 con 0x56337afcbbb0 2024-09-19T05:07:39.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.526+0000 7f6682000640 1 --2- 172.21.15.43:0/2881979993 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x56337afcbbb0 0x56337afcdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:39.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.528+0000 7f6682000640 1 --2- 172.21.15.43:0/2881979993 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x56337afcbbb0 0x56337afcdfa0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4c5fd02d9b3aca97 server_cookie=b844c845f6608070 in_seq=0 out_seq=0 2024-09-19T05:07:39.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.531+0000 7f666cc00640 10 client.25820 ms_handle_connect on v2:172.21.15.43:6838/3404555030 2024-09-19T05:07:39.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.535+0000 7f666cc00640 1 -- 172.21.15.43:0/2881979993 <== mds.1 v2:172.21.15.43:6838/3404555030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56337af24950 con 0x56337afcbbb0 2024-09-19T05:07:39.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.535+0000 7f666cc00640 10 client.25820 handle_command_reply: tid=0 2024-09-19T05:07:39.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.536+0000 7f668c813740 1 client.25820 shutdown 2024-09-19T05:07:39.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.539+0000 7f668c813740 1 -- 172.21.15.43:0/2881979993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56337afb0c70 msgr2=0x56337afbb880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:39.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.539+0000 7f668c813740 1 --2- 172.21.15.43:0/2881979993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56337afb0c70 0x56337afbb880 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f667c00eee0 tx=0x7f667c009e90 comp rx=0 tx=0).stop 2024-09-19T05:07:39.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.543+0000 7f668c813740 1 -- 172.21.15.43:0/2881979993 shutdown_connections 2024-09-19T05:07:39.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.543+0000 7f668c813740 1 --2- 172.21.15.43:0/2881979993 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x56337afcbbb0 0x56337afcdfa0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.543+0000 7f668c813740 1 --2- 172.21.15.43:0/2881979993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56337afbbe50 0x56337afb92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.543+0000 7f668c813740 1 --2- 172.21.15.43:0/2881979993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56337afb1590 0x56337afb8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.543+0000 7f668c813740 1 --2- 172.21.15.43:0/2881979993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56337afb0c70 0x56337afbb880 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.543+0000 7f668c813740 1 -- 172.21.15.43:0/2881979993 >> 172.21.15.43:0/2881979993 conn(0x56337afad200 msgr2=0x56337afb2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:39.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.543+0000 7f666cc00640 0 client.25820 ms_handle_reset on v2:172.21.15.43:6838/3404555030 2024-09-19T05:07:39.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.544+0000 7f668c813740 1 -- 172.21.15.43:0/2881979993 shutdown_connections 2024-09-19T05:07:39.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.545+0000 7f668c813740 1 -- 172.21.15.43:0/2881979993 wait complete. 2024-09-19T05:07:39.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.548+0000 7f668c813740 20 client.25820 trim_cache size 0 max 16384 2024-09-19T05:07:39.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.581+0000 7f668c813740 1 Processor -- start 2024-09-19T05:07:39.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.581+0000 7f668c813740 1 -- start start 2024-09-19T05:07:39.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.581+0000 7f668c813740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56337afb0c70 0x56337afb8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:39.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.581+0000 7f668c813740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56337afb1590 0x56337afad5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:39.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.581+0000 7f668c813740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56337afbbe50 0x56337afd3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:39.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.581+0000 7f668c813740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f66841b40c0 con 0x56337afb1590 2024-09-19T05:07:39.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.581+0000 7f668c813740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f66841b5c00 con 0x56337afbbe50 2024-09-19T05:07:39.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.581+0000 7f668c813740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f66841b4790 con 0x56337afb0c70 2024-09-19T05:07:39.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.582+0000 7f6683400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56337afb0c70 0x56337afb8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:39.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.582+0000 7f6683400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56337afb0c70 0x56337afb8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:59536/0 (socket says 172.21.15.43:59536) 2024-09-19T05:07:39.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.582+0000 7f6683400640 1 -- 172.21.15.43:0/3045952426 learned_addr learned my addr 172.21.15.43:0/3045952426 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:39.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.582+0000 7f6683400640 1 --2- 172.21.15.43:0/3045952426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56337afbbe50 0x56337afd3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:39.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.583+0000 7f6683400640 1 -- 172.21.15.43:0/3045952426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56337afbbe50 msgr2=0x56337afd3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:39.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.583+0000 7f6683400640 1 --2- 172.21.15.43:0/3045952426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56337afbbe50 0x56337afd3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.583+0000 7f6683400640 1 -- 172.21.15.43:0/3045952426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56337afb1590 msgr2=0x56337afad5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:39.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.583+0000 7f6683400640 1 --2- 172.21.15.43:0/3045952426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56337afb1590 0x56337afad5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.583+0000 7f6683400640 1 -- 172.21.15.43:0/3045952426 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56337afad030 con 0x56337afb0c70 2024-09-19T05:07:39.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.583+0000 7f6683400640 1 --2- 172.21.15.43:0/3045952426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56337afb0c70 0x56337afb8410 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f667c00d430 tx=0x7f667c038040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd1d388fbe672add server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:39.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.584+0000 7f666f400640 1 -- 172.21.15.43:0/3045952426 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c03c030 con 0x56337afb0c70 2024-09-19T05:07:39.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.584+0000 7f666f400640 1 -- 172.21.15.43:0/3045952426 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f667c00d810 con 0x56337afb0c70 2024-09-19T05:07:39.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.584+0000 7f666f400640 1 -- 172.21.15.43:0/3045952426 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c040600 con 0x56337afb0c70 2024-09-19T05:07:39.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.585+0000 7f668c813740 1 -- 172.21.15.43:0/3045952426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56337afb0c70 msgr2=0x56337afb8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:39.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.585+0000 7f668c813740 1 --2- 172.21.15.43:0/3045952426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56337afb0c70 0x56337afb8410 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f667c00d430 tx=0x7f667c038040 comp rx=0 tx=0).stop 2024-09-19T05:07:39.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.585+0000 7f668c813740 1 -- 172.21.15.43:0/3045952426 shutdown_connections 2024-09-19T05:07:39.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.585+0000 7f668c813740 1 --2- 172.21.15.43:0/3045952426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56337afbbe50 0x56337afd3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.585+0000 7f668c813740 1 --2- 172.21.15.43:0/3045952426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56337afb1590 0x56337afad5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.585+0000 7f668c813740 1 --2- 172.21.15.43:0/3045952426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56337afb0c70 0x56337afb8410 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.585+0000 7f668c813740 1 -- 172.21.15.43:0/3045952426 >> 172.21.15.43:0/3045952426 conn(0x56337afad200 msgr2=0x56337afc7590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:39.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.585+0000 7f668c813740 1 -- 172.21.15.43:0/3045952426 shutdown_connections 2024-09-19T05:07:39.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.585+0000 7f668c813740 1 -- 172.21.15.43:0/3045952426 wait complete. 2024-09-19T05:07:39.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.586+0000 7f668c813740 1 -- start start 2024-09-19T05:07:39.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.586+0000 7f668c813740 1 Processor -- start 2024-09-19T05:07:39.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.587+0000 7f668c813740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56337afb0c70 0x56337afc3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:39.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.587+0000 7f668c813740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56337afb1590 0x56337afc4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:39.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.587+0000 7f668c813740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56337afbbe50 0x56337afbeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:39.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.587+0000 7f668c813740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f66841b5530 con 0x56337afbbe50 2024-09-19T05:07:39.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.587+0000 7f668c813740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f66841b4e60 con 0x56337afb0c70 2024-09-19T05:07:39.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.587+0000 7f668c813740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f66841171f0 con 0x56337afb1590 2024-09-19T05:07:39.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.587+0000 7f668c813740 10 client.0 _ll_register_callbacks cb 0x7f667c00dcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:39.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.587+0000 7f6683400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56337afbbe50 0x56337afbeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:39.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.587+0000 7f6683400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56337afbbe50 0x56337afbeeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50732/0 (socket says 172.21.15.43:50732) 2024-09-19T05:07:39.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.587+0000 7f6683400640 1 -- 172.21.15.43:0/338971390 learned_addr learned my addr 172.21.15.43:0/338971390 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:39.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.588+0000 7f6683400640 1 -- 172.21.15.43:0/338971390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56337afb1590 msgr2=0x56337afc4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:39.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.588+0000 7f6683400640 1 --2- 172.21.15.43:0/338971390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56337afb1590 0x56337afc4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.588+0000 7f6683400640 1 -- 172.21.15.43:0/338971390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56337afb0c70 msgr2=0x56337afc3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:39.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.588+0000 7f6683400640 1 --2- 172.21.15.43:0/338971390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56337afb0c70 0x56337afc3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.588+0000 7f6683400640 1 -- 172.21.15.43:0/338971390 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56337afa8770 con 0x56337afbbe50 2024-09-19T05:07:39.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.589+0000 7f6683400640 1 --2- 172.21.15.43:0/338971390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56337afbbe50 0x56337afbeeb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f667c00de00 tx=0x7f667c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=951084218a65baa1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:39.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.589+0000 7f666cc00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:07:39.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.589+0000 7f666cc00640 1 -- 172.21.15.43:0/338971390 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c03c040 con 0x56337afbbe50 2024-09-19T05:07:39.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.589+0000 7f666cc00640 1 -- 172.21.15.43:0/338971390 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f667c004a00 con 0x56337afbbe50 2024-09-19T05:07:39.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.589+0000 7f668c813740 1 -- 172.21.15.43:0/338971390 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f667c00d810 con 0x56337afbbe50 2024-09-19T05:07:39.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.589+0000 7f666cc00640 1 -- 172.21.15.43:0/338971390 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f667c05b050 con 0x56337afbbe50 2024-09-19T05:07:39.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.589+0000 7f668c813740 10 client.15807 fetch_fsmap learned FSMap version 29 2024-09-19T05:07:39.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.589+0000 7f668c813740 1 -- 172.21.15.43:0/338971390 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56337b036890 con 0x56337afbbe50 2024-09-19T05:07:39.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.590+0000 7f666cc00640 1 -- 172.21.15.43:0/338971390 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f667c00d810 con 0x56337afbbe50 2024-09-19T05:07:39.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.591+0000 7f666cc00640 1 -- 172.21.15.43:0/338971390 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f667c056030 con 0x56337afbbe50 2024-09-19T05:07:39.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.591+0000 7f668c813740 10 client.15807 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:07:39.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.591+0000 7f668c813740 10 client.15807 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-19T05:07:39.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.591+0000 7f668c813740 20 client.15807 populate_metadata read hostname 'smithi043' 2024-09-19T05:07:39.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.591+0000 7f668c813740 1 --2- 172.21.15.43:0/338971390 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x56337b036fa0 0x56337b039390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:39.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.591+0000 7f668c813740 4 client.15807 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:07:39.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.591+0000 7f668c813740 1 -- 172.21.15.43:0/338971390 --> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56337af1ee60 con 0x56337b036fa0 2024-09-19T05:07:39.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.591+0000 7f6682000640 1 --2- 172.21.15.43:0/338971390 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x56337b036fa0 0x56337b039390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:39.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.592+0000 7f6682000640 1 --2- 172.21.15.43:0/338971390 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x56337b036fa0 0x56337b039390 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b8c1567fd7294bb4 server_cookie=de70b8777afdedb3 in_seq=0 out_seq=0 2024-09-19T05:07:39.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.592+0000 7f666cc00640 10 client.15807 ms_handle_connect on v2:172.21.15.43:6838/3404555030 2024-09-19T05:07:39.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.592+0000 7f666cc00640 1 -- 172.21.15.43:0/338971390 <== mds.1 v2:172.21.15.43:6838/3404555030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56337af1ee60 con 0x56337b036fa0 2024-09-19T05:07:39.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.592+0000 7f666cc00640 10 client.15807 handle_command_reply: tid=0 2024-09-19T05:07:39.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.592+0000 7f668c813740 1 client.15807 shutdown 2024-09-19T05:07:39.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.592+0000 7f668c813740 1 -- 172.21.15.43:0/338971390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56337afbbe50 msgr2=0x56337afbeeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:39.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.592+0000 7f668c813740 1 --2- 172.21.15.43:0/338971390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56337afbbe50 0x56337afbeeb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f667c00de00 tx=0x7f667c003c20 comp rx=0 tx=0).stop 2024-09-19T05:07:39.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.593+0000 7f668c813740 1 -- 172.21.15.43:0/338971390 shutdown_connections 2024-09-19T05:07:39.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.593+0000 7f668c813740 1 --2- 172.21.15.43:0/338971390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56337afbbe50 0x56337afbeeb0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.593+0000 7f668c813740 1 --2- 172.21.15.43:0/338971390 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x56337b036fa0 0x56337b039390 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.593+0000 7f668c813740 1 --2- 172.21.15.43:0/338971390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56337afb1590 0x56337afc4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.593+0000 7f666cc00640 0 client.15807 ms_handle_reset on v2:172.21.15.43:6838/3404555030 2024-09-19T05:07:39.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.593+0000 7f668c813740 1 --2- 172.21.15.43:0/338971390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56337afb0c70 0x56337afc3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.593+0000 7f668c813740 1 -- 172.21.15.43:0/338971390 >> 172.21.15.43:0/338971390 conn(0x56337afad200 msgr2=0x56337afabb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:39.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.593+0000 7f668c813740 1 -- 172.21.15.43:0/338971390 shutdown_connections 2024-09-19T05:07:39.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.593+0000 7f668c813740 1 -- 172.21.15.43:0/338971390 wait complete. 2024-09-19T05:07:39.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.593+0000 7f668c813740 20 client.15807 trim_cache size 0 max 16384 2024-09-19T05:07:39.597 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:07:39.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.594+0000 7f6689c00640 1 -- 172.21.15.43:0/3217054615 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f66680a34b0 msgr2=0x7f66680a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:39.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.594+0000 7f6689c00640 1 --2- 172.21.15.43:0/3217054615 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f66680a34b0 0x7f66680a5970 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x56337afae060 tx=0x7f6674098180 comp rx=0 tx=0).stop 2024-09-19T05:07:39.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.594+0000 7f6689c00640 1 -- 172.21.15.43:0/3217054615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6684109450 msgr2=0x7f66841a7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:39.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.594+0000 7f6689c00640 1 --2- 172.21.15.43:0/3217054615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6684109450 0x7f66841a7280 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f667400c970 tx=0x7f667400ce30 comp rx=0 tx=0).stop 2024-09-19T05:07:39.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.594+0000 7f6689c00640 1 -- 172.21.15.43:0/3217054615 shutdown_connections 2024-09-19T05:07:39.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.594+0000 7f6689c00640 1 --2- 172.21.15.43:0/3217054615 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f668411b140 0x7f66841abb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.594+0000 7f6689c00640 1 --2- 172.21.15.43:0/3217054615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6684109830 0x7f66841a77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.594+0000 7f6689c00640 1 --2- 172.21.15.43:0/3217054615 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f66680a34b0 0x7f66680a5970 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.594+0000 7f6689c00640 1 --2- 172.21.15.43:0/3217054615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6684109450 0x7f66841a7280 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:39.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.594+0000 7f6689c00640 1 -- 172.21.15.43:0/3217054615 >> 172.21.15.43:0/3217054615 conn(0x7f668406e140 msgr2=0x7f668411c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:39.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.594+0000 7f6689c00640 1 -- 172.21.15.43:0/3217054615 shutdown_connections 2024-09-19T05:07:39.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:39.595+0000 7f6689c00640 1 -- 172.21.15.43:0/3217054615 wait complete. 2024-09-19T05:07:40.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:07:40 smithi151 ceph-mon[35425]: pgmap v93: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 155 MiB/s wr, 48 op/s 2024-09-19T05:07:40.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:07:40 smithi096 ceph-mon[35085]: pgmap v93: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 155 MiB/s wr, 48 op/s 2024-09-19T05:07:40.500 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:07:40 smithi043 ceph-mon[30800]: pgmap v93: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 155 MiB/s wr, 48 op/s 2024-09-19T05:07:41.274 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T05:07:42.000 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:07:42.370 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:07:42 smithi043 ceph-mon[30800]: pgmap v94: 97 pgs: 97 active+clean; 13 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 281 MiB/s wr, 85 op/s 2024-09-19T05:07:42.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:07:42 smithi151 ceph-mon[35425]: pgmap v94: 97 pgs: 97 active+clean; 13 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 281 MiB/s wr, 85 op/s 2024-09-19T05:07:42.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:07:42 smithi096 ceph-mon[35085]: pgmap v94: 97 pgs: 97 active+clean; 13 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 281 MiB/s wr, 85 op/s 2024-09-19T05:07:42.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.787+0000 7f5d35200640 1 -- 172.21.15.43:0/2034599658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d30106f10 msgr2=0x7f5d3010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:42.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.787+0000 7f5d35200640 1 --2- 172.21.15.43:0/2034599658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d30106f10 0x7f5d3010ea90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5d28009ca0 tx=0x7f5d2802a8b0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.788+0000 7f5d35200640 1 -- 172.21.15.43:0/2034599658 shutdown_connections 2024-09-19T05:07:42.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.788+0000 7f5d35200640 1 --2- 172.21.15.43:0/2034599658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d30106f10 0x7f5d3010ea90 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.788+0000 7f5d35200640 1 --2- 172.21.15.43:0/2034599658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d30102790 0x7f5d301067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.788+0000 7f5d35200640 1 --2- 172.21.15.43:0/2034599658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d30101de0 0x7f5d301021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.788+0000 7f5d35200640 1 -- 172.21.15.43:0/2034599658 >> 172.21.15.43:0/2034599658 conn(0x7f5d300fd8c0 msgr2=0x7f5d300ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:42.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.788+0000 7f5d35200640 1 -- 172.21.15.43:0/2034599658 shutdown_connections 2024-09-19T05:07:42.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.788+0000 7f5d35200640 1 -- 172.21.15.43:0/2034599658 wait complete. 2024-09-19T05:07:42.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.789+0000 7f5d35200640 1 Processor -- start 2024-09-19T05:07:42.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.789+0000 7f5d35200640 1 -- start start 2024-09-19T05:07:42.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.790+0000 7f5d35200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d30101de0 0x7f5d30198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:42.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.790+0000 7f5d35200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d30102790 0x7f5d301989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:42.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.790+0000 7f5d35200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d30106f10 0x7f5d3019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:42.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.790+0000 7f5d35200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5d30111fc0 con 0x7f5d30102790 2024-09-19T05:07:42.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.790+0000 7f5d35200640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5d30111e40 con 0x7f5d30106f10 2024-09-19T05:07:42.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.790+0000 7f5d35200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5d30112140 con 0x7f5d30101de0 2024-09-19T05:07:42.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.790+0000 7f5d2de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d30101de0 0x7f5d30198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:42.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.790+0000 7f5d2de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d30101de0 0x7f5d30198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:59564/0 (socket says 172.21.15.43:59564) 2024-09-19T05:07:42.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.790+0000 7f5d2de00640 1 -- 172.21.15.43:0/1689886199 learned_addr learned my addr 172.21.15.43:0/1689886199 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:42.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.790+0000 7f5d2d400640 1 --2- 172.21.15.43:0/1689886199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d30102790 0x7f5d301989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:42.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.790+0000 7f5d2e800640 1 --2- 172.21.15.43:0/1689886199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d30106f10 0x7f5d3019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:42.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.791+0000 7f5d2de00640 1 -- 172.21.15.43:0/1689886199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d30106f10 msgr2=0x7f5d3019cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:42.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.791+0000 7f5d2de00640 1 --2- 172.21.15.43:0/1689886199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d30106f10 0x7f5d3019cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.791+0000 7f5d2de00640 1 -- 172.21.15.43:0/1689886199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d30102790 msgr2=0x7f5d301989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:42.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.791+0000 7f5d2de00640 1 --2- 172.21.15.43:0/1689886199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d30102790 0x7f5d301989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.791+0000 7f5d2de00640 1 -- 172.21.15.43:0/1689886199 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d3019d430 con 0x7f5d30101de0 2024-09-19T05:07:42.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.791+0000 7f5d2e800640 1 --2- 172.21.15.43:0/1689886199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d30106f10 0x7f5d3019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:07:42.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.791+0000 7f5d2de00640 1 --2- 172.21.15.43:0/1689886199 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d30101de0 0x7f5d30198480 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5d2000e9f0 tx=0x7f5d2000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:42.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.791+0000 7f5d26a00640 1 -- 172.21.15.43:0/1689886199 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2000cd80 con 0x7f5d30101de0 2024-09-19T05:07:42.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.791+0000 7f5d26a00640 1 -- 172.21.15.43:0/1689886199 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d20005500 con 0x7f5d30101de0 2024-09-19T05:07:42.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.791+0000 7f5d35200640 1 -- 172.21.15.43:0/1689886199 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d3019d720 con 0x7f5d30101de0 2024-09-19T05:07:42.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.791+0000 7f5d26a00640 1 -- 172.21.15.43:0/1689886199 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d20010690 con 0x7f5d30101de0 2024-09-19T05:07:42.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.791+0000 7f5d35200640 1 -- 172.21.15.43:0/1689886199 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d301a5000 con 0x7f5d30101de0 2024-09-19T05:07:42.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.795+0000 7f5d26a00640 1 -- 172.21.15.43:0/1689886199 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f5d200046c0 con 0x7f5d30101de0 2024-09-19T05:07:42.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.796+0000 7f5d26a00640 1 --2- 172.21.15.43:0/1689886199 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f5d140a3430 0x7f5d140a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:42.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.796+0000 7f5d26a00640 1 -- 172.21.15.43:0/1689886199 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f5d200d0140 con 0x7f5d30101de0 2024-09-19T05:07:42.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.797+0000 7f5d2d400640 1 --2- 172.21.15.43:0/1689886199 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f5d140a3430 0x7f5d140a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:42.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.797+0000 7f5d37e31740 1 Processor -- start 2024-09-19T05:07:42.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.797+0000 7f5d37e31740 1 -- start start 2024-09-19T05:07:42.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.797+0000 7f5d2d400640 1 --2- 172.21.15.43:0/1689886199 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f5d140a3430 0x7f5d140a58f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5d301087b0 tx=0x7f5d18005f90 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:42.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.798+0000 7f5d37e31740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621442ffc70 0x562144300050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:42.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.798+0000 7f5d37e31740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562144300590 0x56214430a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:42.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.798+0000 7f5d37e31740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56214430ae50 0x56214430d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:42.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.798+0000 7f5d37e31740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5621442663e0 con 0x562144300590 2024-09-19T05:07:42.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.798+0000 7f5d37e31740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56214420dd80 con 0x56214430ae50 2024-09-19T05:07:42.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.798+0000 7f5d37e31740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5621442623a0 con 0x5621442ffc70 2024-09-19T05:07:42.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.798+0000 7f5d2de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621442ffc70 0x562144300050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:42.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.798+0000 7f5d2de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621442ffc70 0x562144300050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:59570/0 (socket says 172.21.15.43:59570) 2024-09-19T05:07:42.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.798+0000 7f5d2d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562144300590 0x56214430a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:42.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.798+0000 7f5d2de00640 1 -- 172.21.15.43:0/167668517 learned_addr learned my addr 172.21.15.43:0/167668517 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:42.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.798+0000 7f5d2e800640 1 --2- 172.21.15.43:0/167668517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56214430ae50 0x56214430d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:42.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.798+0000 7f5d2de00640 1 -- 172.21.15.43:0/167668517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56214430ae50 msgr2=0x56214430d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:42.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.798+0000 7f5d2de00640 1 --2- 172.21.15.43:0/167668517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56214430ae50 0x56214430d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.798+0000 7f5d2de00640 1 -- 172.21.15.43:0/167668517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562144300590 msgr2=0x56214430a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:42.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.798+0000 7f5d2de00640 1 --2- 172.21.15.43:0/167668517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562144300590 0x56214430a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.799+0000 7f5d2de00640 1 -- 172.21.15.43:0/167668517 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56214422cae0 con 0x5621442ffc70 2024-09-19T05:07:42.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.799+0000 7f5d2d400640 1 --2- 172.21.15.43:0/167668517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562144300590 0x56214430a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:07:42.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.799+0000 7f5d2de00640 1 --2- 172.21.15.43:0/167668517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621442ffc70 0x562144300050 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5d20009380 tx=0x7f5d2009c030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81774cab78b9fac5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:42.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.799+0000 7f5d1f400640 1 -- 172.21.15.43:0/167668517 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2009c430 con 0x5621442ffc70 2024-09-19T05:07:42.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.799+0000 7f5d1f400640 1 -- 172.21.15.43:0/167668517 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d200a07e0 con 0x5621442ffc70 2024-09-19T05:07:42.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.799+0000 7f5d1f400640 1 -- 172.21.15.43:0/167668517 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d200b5220 con 0x5621442ffc70 2024-09-19T05:07:42.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.800+0000 7f5d37e31740 1 -- 172.21.15.43:0/167668517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621442ffc70 msgr2=0x562144300050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:42.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.800+0000 7f5d37e31740 1 --2- 172.21.15.43:0/167668517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621442ffc70 0x562144300050 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5d20009380 tx=0x7f5d2009c030 comp rx=0 tx=0).stop 2024-09-19T05:07:42.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.800+0000 7f5d37e31740 1 -- 172.21.15.43:0/167668517 shutdown_connections 2024-09-19T05:07:42.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.800+0000 7f5d37e31740 1 --2- 172.21.15.43:0/167668517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56214430ae50 0x56214430d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.800+0000 7f5d37e31740 1 --2- 172.21.15.43:0/167668517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562144300590 0x56214430a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.800+0000 7f5d37e31740 1 --2- 172.21.15.43:0/167668517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621442ffc70 0x562144300050 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.800+0000 7f5d37e31740 1 -- 172.21.15.43:0/167668517 >> 172.21.15.43:0/167668517 conn(0x5621442fc200 msgr2=0x5621442fc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:42.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.800+0000 7f5d37e31740 1 -- 172.21.15.43:0/167668517 shutdown_connections 2024-09-19T05:07:42.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.800+0000 7f5d37e31740 1 -- 172.21.15.43:0/167668517 wait complete. 2024-09-19T05:07:42.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.801+0000 7f5d37e31740 1 -- start start 2024-09-19T05:07:42.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.801+0000 7f5d37e31740 1 Processor -- start 2024-09-19T05:07:42.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.802+0000 7f5d37e31740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621442ffc70 0x562144307d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:42.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.802+0000 7f5d37e31740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562144300590 0x5621443082d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:42.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.802+0000 7f5d37e31740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56214430ae50 0x562144308810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:42.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.802+0000 7f5d37e31740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56214430da20 con 0x56214430ae50 2024-09-19T05:07:42.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.802+0000 7f5d37e31740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5621442210f0 con 0x562144300590 2024-09-19T05:07:42.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.802+0000 7f5d37e31740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56214430dba0 con 0x5621442ffc70 2024-09-19T05:07:42.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.802+0000 7f5d2de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621442ffc70 0x562144307d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:42.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.802+0000 7f5d2de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621442ffc70 0x562144307d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:59576/0 (socket says 172.21.15.43:59576) 2024-09-19T05:07:42.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.802+0000 7f5d2de00640 1 -- 172.21.15.43:0/3301865331 learned_addr learned my addr 172.21.15.43:0/3301865331 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:42.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.802+0000 7f5d2d400640 1 --2- 172.21.15.43:0/3301865331 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562144300590 0x5621443082d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:42.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.802+0000 7f5d37e31740 10 client.0 _ll_register_callbacks cb 0x562144208bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:42.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.802+0000 7f5d2de00640 1 -- 172.21.15.43:0/3301865331 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562144300590 msgr2=0x5621443082d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:42.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.802+0000 7f5d2de00640 1 --2- 172.21.15.43:0/3301865331 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562144300590 0x5621443082d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.802+0000 7f5d2de00640 1 -- 172.21.15.43:0/3301865331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56214430ae50 msgr2=0x562144308810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T05:07:42.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.802+0000 7f5d2de00640 1 --2- 172.21.15.43:0/3301865331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56214430ae50 0x562144308810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.802+0000 7f5d2de00640 1 -- 172.21.15.43:0/3301865331 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562143fd73c0 con 0x5621442ffc70 2024-09-19T05:07:42.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.802+0000 7f5d2de00640 1 --2- 172.21.15.43:0/3301865331 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621442ffc70 0x562144307d90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5d200cd950 tx=0x7f5d2009c4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=350c88b1bf93532b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:42.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.802+0000 7f5d13e00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:07:42.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.803+0000 7f5d13e00640 1 -- 172.21.15.43:0/3301865331 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d200b5240 con 0x5621442ffc70 2024-09-19T05:07:42.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.803+0000 7f5d13e00640 1 -- 172.21.15.43:0/3301865331 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f5d200a4c10 con 0x5621442ffc70 2024-09-19T05:07:42.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.803+0000 7f5d37e31740 1 -- 172.21.15.43:0/3301865331 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d200a07e0 con 0x5621442ffc70 2024-09-19T05:07:42.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.803+0000 7f5d13e00640 1 -- 172.21.15.43:0/3301865331 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5d200b53e0 con 0x5621442ffc70 2024-09-19T05:07:42.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.803+0000 7f5d37e31740 10 client.25792 fetch_fsmap learned FSMap version 29 2024-09-19T05:07:42.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.803+0000 7f5d37e31740 1 -- 172.21.15.43:0/3301865331 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56214430da20 con 0x5621442ffc70 2024-09-19T05:07:42.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.803+0000 7f5d13e00640 1 -- 172.21.15.43:0/3301865331 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d200a07e0 con 0x5621442ffc70 2024-09-19T05:07:42.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.803+0000 7f5d13e00640 1 -- 172.21.15.43:0/3301865331 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5d200a6070 con 0x5621442ffc70 2024-09-19T05:07:42.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.803+0000 7f5d37e31740 10 client.25792 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:07:42.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.803+0000 7f5d37e31740 10 client.25792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:07:42.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.804+0000 7f5d37e31740 20 client.25792 populate_metadata read hostname 'smithi043' 2024-09-19T05:07:42.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.804+0000 7f5d37e31740 1 --2- 172.21.15.43:0/3301865331 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x56214432bc40 0x56214432e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:42.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.804+0000 7f5d37e31740 4 client.25792 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:07:42.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.804+0000 7f5d37e31740 1 -- 172.21.15.43:0/3301865331 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562144273950 con 0x56214432bc40 2024-09-19T05:07:42.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.804+0000 7f5d2d400640 1 --2- 172.21.15.43:0/3301865331 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x56214432bc40 0x56214432e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:42.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.804+0000 7f5d2d400640 1 --2- 172.21.15.43:0/3301865331 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x56214432bc40 0x56214432e030 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52a001d1c330580e server_cookie=961abba711c318c9 in_seq=0 out_seq=0 2024-09-19T05:07:42.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.804+0000 7f5d13e00640 10 client.25792 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:07:42.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.805+0000 7f5d13e00640 1 -- 172.21.15.43:0/3301865331 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562144273950 con 0x56214432bc40 2024-09-19T05:07:42.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.805+0000 7f5d13e00640 10 client.25792 handle_command_reply: tid=0 2024-09-19T05:07:42.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.805+0000 7f5d37e31740 1 client.25792 shutdown 2024-09-19T05:07:42.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.805+0000 7f5d37e31740 1 -- 172.21.15.43:0/3301865331 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621442ffc70 msgr2=0x562144307d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:42.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.805+0000 7f5d37e31740 1 --2- 172.21.15.43:0/3301865331 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621442ffc70 0x562144307d90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5d200cd950 tx=0x7f5d2009c4b0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.806+0000 7f5d37e31740 1 -- 172.21.15.43:0/3301865331 shutdown_connections 2024-09-19T05:07:42.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.806+0000 7f5d37e31740 1 --2- 172.21.15.43:0/3301865331 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x56214432bc40 0x56214432e030 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.806+0000 7f5d37e31740 1 --2- 172.21.15.43:0/3301865331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56214430ae50 0x562144308810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.806+0000 7f5d37e31740 1 --2- 172.21.15.43:0/3301865331 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562144300590 0x5621443082d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.806+0000 7f5d37e31740 1 --2- 172.21.15.43:0/3301865331 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621442ffc70 0x562144307d90 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.806+0000 7f5d37e31740 1 -- 172.21.15.43:0/3301865331 >> 172.21.15.43:0/3301865331 conn(0x5621442fc200 msgr2=0x562144301f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:42.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.806+0000 7f5d13e00640 0 client.25792 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:07:42.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.806+0000 7f5d37e31740 1 -- 172.21.15.43:0/3301865331 shutdown_connections 2024-09-19T05:07:42.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.806+0000 7f5d37e31740 1 -- 172.21.15.43:0/3301865331 wait complete. 2024-09-19T05:07:42.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.806+0000 7f5d37e31740 20 client.25792 trim_cache size 0 max 16384 2024-09-19T05:07:42.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.820+0000 7f5d37e31740 1 Processor -- start 2024-09-19T05:07:42.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.820+0000 7f5d37e31740 1 -- start start 2024-09-19T05:07:42.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.820+0000 7f5d37e31740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621442ffc70 0x5621443026d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:42.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.820+0000 7f5d37e31740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562144300590 0x5621442fb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:42.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.820+0000 7f5d37e31740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56214430ae50 0x5621443337a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:42.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.820+0000 7f5d37e31740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5d300742f0 con 0x562144300590 2024-09-19T05:07:42.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.820+0000 7f5d37e31740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5d30075e30 con 0x56214430ae50 2024-09-19T05:07:42.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.820+0000 7f5d37e31740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5d300749c0 con 0x5621442ffc70 2024-09-19T05:07:42.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.821+0000 7f5d2d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562144300590 0x5621442fb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:42.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.821+0000 7f5d2de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621442ffc70 0x5621443026d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:42.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.821+0000 7f5d2d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562144300590 0x5621442fb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50768/0 (socket says 172.21.15.43:50768) 2024-09-19T05:07:42.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.821+0000 7f5d2d400640 1 -- 172.21.15.43:0/2068470195 learned_addr learned my addr 172.21.15.43:0/2068470195 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:42.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.821+0000 7f5d2e800640 1 --2- 172.21.15.43:0/2068470195 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56214430ae50 0x5621443337a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:42.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.821+0000 7f5d2e800640 1 -- 172.21.15.43:0/2068470195 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621442ffc70 msgr2=0x5621443026d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:42.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.821+0000 7f5d2e800640 1 --2- 172.21.15.43:0/2068470195 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621442ffc70 0x5621443026d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.821+0000 7f5d2e800640 1 -- 172.21.15.43:0/2068470195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562144300590 msgr2=0x5621442fb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:42.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.821+0000 7f5d2e800640 1 --2- 172.21.15.43:0/2068470195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562144300590 0x5621442fb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.821+0000 7f5d2e800640 1 -- 172.21.15.43:0/2068470195 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621442fce90 con 0x56214430ae50 2024-09-19T05:07:42.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.821+0000 7f5d2e800640 1 --2- 172.21.15.43:0/2068470195 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56214430ae50 0x5621443337a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5d28006010 tx=0x7f5d280303a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=857d6f3cc6307175 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:42.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.822+0000 7f5d1ea00640 1 -- 172.21.15.43:0/2068470195 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d28003970 con 0x56214430ae50 2024-09-19T05:07:42.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.822+0000 7f5d1ea00640 1 -- 172.21.15.43:0/2068470195 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d28036040 con 0x56214430ae50 2024-09-19T05:07:42.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.822+0000 7f5d1ea00640 1 -- 172.21.15.43:0/2068470195 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d28004330 con 0x56214430ae50 2024-09-19T05:07:42.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.823+0000 7f5d37e31740 1 -- 172.21.15.43:0/2068470195 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56214430ae50 msgr2=0x5621443337a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:42.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.823+0000 7f5d37e31740 1 --2- 172.21.15.43:0/2068470195 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56214430ae50 0x5621443337a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5d28006010 tx=0x7f5d280303a0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.823+0000 7f5d37e31740 1 -- 172.21.15.43:0/2068470195 shutdown_connections 2024-09-19T05:07:42.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.823+0000 7f5d37e31740 1 --2- 172.21.15.43:0/2068470195 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56214430ae50 0x5621443337a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.823+0000 7f5d37e31740 1 --2- 172.21.15.43:0/2068470195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562144300590 0x5621442fb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.823+0000 7f5d37e31740 1 --2- 172.21.15.43:0/2068470195 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621442ffc70 0x5621443026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.823+0000 7f5d37e31740 1 -- 172.21.15.43:0/2068470195 >> 172.21.15.43:0/2068470195 conn(0x5621442fc200 msgr2=0x5621442fab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:42.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.823+0000 7f5d37e31740 1 -- 172.21.15.43:0/2068470195 shutdown_connections 2024-09-19T05:07:42.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.823+0000 7f5d37e31740 1 -- 172.21.15.43:0/2068470195 wait complete. 2024-09-19T05:07:42.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.823+0000 7f5d37e31740 1 -- start start 2024-09-19T05:07:42.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.824+0000 7f5d37e31740 1 Processor -- start 2024-09-19T05:07:42.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.824+0000 7f5d37e31740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621442ffc70 0x562144313120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:42.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.824+0000 7f5d37e31740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562144300590 0x562144313660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:42.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.824+0000 7f5d37e31740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56214430ae50 0x56214430e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:42.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.824+0000 7f5d37e31740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5d30075760 con 0x562144300590 2024-09-19T05:07:42.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.824+0000 7f5d37e31740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5d30075090 con 0x5621442ffc70 2024-09-19T05:07:42.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.824+0000 7f5d37e31740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5d300722d0 con 0x56214430ae50 2024-09-19T05:07:42.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.824+0000 7f5d37e31740 10 client.0 _ll_register_callbacks cb 0x7f5d18005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:42.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.825+0000 7f5d2d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562144300590 0x562144313660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:42.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.825+0000 7f5d2de00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621442ffc70 0x562144313120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:42.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.825+0000 7f5d2d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562144300590 0x562144313660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50780/0 (socket says 172.21.15.43:50780) 2024-09-19T05:07:42.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.825+0000 7f5d2d400640 1 -- 172.21.15.43:0/3434086956 learned_addr learned my addr 172.21.15.43:0/3434086956 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:42.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.825+0000 7f5d2e800640 1 --2- 172.21.15.43:0/3434086956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56214430ae50 0x56214430e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:42.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.825+0000 7f5d2de00640 1 -- 172.21.15.43:0/3434086956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56214430ae50 msgr2=0x56214430e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:42.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.825+0000 7f5d2de00640 1 --2- 172.21.15.43:0/3434086956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56214430ae50 0x56214430e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.825+0000 7f5d2de00640 1 -- 172.21.15.43:0/3434086956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562144300590 msgr2=0x562144313660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:42.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.825+0000 7f5d2de00640 1 --2- 172.21.15.43:0/3434086956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562144300590 0x562144313660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.825+0000 7f5d2de00640 1 -- 172.21.15.43:0/3434086956 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56214430edd0 con 0x5621442ffc70 2024-09-19T05:07:42.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.825+0000 7f5d2e800640 1 --2- 172.21.15.43:0/3434086956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56214430ae50 0x56214430e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:07:42.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.825+0000 7f5d2d400640 1 --2- 172.21.15.43:0/3434086956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562144300590 0x562144313660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:07:42.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.825+0000 7f5d2de00640 1 --2- 172.21.15.43:0/3434086956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621442ffc70 0x562144313120 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5d20098e00 tx=0x7f5d20004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87559bfd6d942624 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:42.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.825+0000 7f5d13e00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:07:42.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.826+0000 7f5d13e00640 1 -- 172.21.15.43:0/3434086956 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d200ad080 con 0x5621442ffc70 2024-09-19T05:07:42.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.826+0000 7f5d13e00640 1 -- 172.21.15.43:0/3434086956 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f5d200a4e20 con 0x5621442ffc70 2024-09-19T05:07:42.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.826+0000 7f5d37e31740 1 -- 172.21.15.43:0/3434086956 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d28036040 con 0x5621442ffc70 2024-09-19T05:07:42.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.826+0000 7f5d13e00640 1 -- 172.21.15.43:0/3434086956 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5d2008b050 con 0x5621442ffc70 2024-09-19T05:07:42.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.826+0000 7f5d37e31740 10 client.25841 fetch_fsmap learned FSMap version 29 2024-09-19T05:07:42.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.826+0000 7f5d37e31740 1 -- 172.21.15.43:0/3434086956 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56214439eb60 con 0x5621442ffc70 2024-09-19T05:07:42.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.826+0000 7f5d13e00640 1 -- 172.21.15.43:0/3434086956 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d28036040 con 0x5621442ffc70 2024-09-19T05:07:42.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.826+0000 7f5d13e00640 1 -- 172.21.15.43:0/3434086956 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5d200a6070 con 0x5621442ffc70 2024-09-19T05:07:42.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.826+0000 7f5d37e31740 10 client.25841 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:07:42.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.826+0000 7f5d37e31740 10 client.25841 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:07:42.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.826+0000 7f5d37e31740 20 client.25841 populate_metadata read hostname 'smithi043' 2024-09-19T05:07:42.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.826+0000 7f5d37e31740 1 --2- 172.21.15.43:0/3434086956 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x56214439f0c0 0x5621443a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:42.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.827+0000 7f5d37e31740 4 client.25841 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:07:42.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.827+0000 7f5d37e31740 1 -- 172.21.15.43:0/3434086956 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56214426de60 con 0x56214439f0c0 2024-09-19T05:07:42.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.828+0000 7f5d2d400640 1 --2- 172.21.15.43:0/3434086956 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x56214439f0c0 0x5621443a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:42.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.828+0000 7f5d2d400640 1 --2- 172.21.15.43:0/3434086956 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x56214439f0c0 0x5621443a14e0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db9f62b98b9ae7e3 server_cookie=c1420f5d2af85e9b in_seq=0 out_seq=0 2024-09-19T05:07:42.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.828+0000 7f5d13e00640 10 client.25841 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:07:42.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.828+0000 7f5d13e00640 1 -- 172.21.15.43:0/3434086956 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56214426de60 con 0x56214439f0c0 2024-09-19T05:07:42.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.828+0000 7f5d13e00640 10 client.25841 handle_command_reply: tid=0 2024-09-19T05:07:42.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.828+0000 7f5d37e31740 1 client.25841 shutdown 2024-09-19T05:07:42.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.828+0000 7f5d37e31740 1 -- 172.21.15.43:0/3434086956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621442ffc70 msgr2=0x562144313120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:42.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.829+0000 7f5d37e31740 1 --2- 172.21.15.43:0/3434086956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621442ffc70 0x562144313120 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5d20098e00 tx=0x7f5d20004a40 comp rx=0 tx=0).stop 2024-09-19T05:07:42.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.829+0000 7f5d37e31740 1 -- 172.21.15.43:0/3434086956 shutdown_connections 2024-09-19T05:07:42.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.829+0000 7f5d37e31740 1 --2- 172.21.15.43:0/3434086956 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x56214439f0c0 0x5621443a14e0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.829+0000 7f5d37e31740 1 --2- 172.21.15.43:0/3434086956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56214430ae50 0x56214430e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.829+0000 7f5d37e31740 1 --2- 172.21.15.43:0/3434086956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562144300590 0x562144313660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.829+0000 7f5d37e31740 1 --2- 172.21.15.43:0/3434086956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621442ffc70 0x562144313120 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.829+0000 7f5d13e00640 0 client.25841 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:07:42.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.829+0000 7f5d37e31740 1 -- 172.21.15.43:0/3434086956 >> 172.21.15.43:0/3434086956 conn(0x5621442fc200 msgr2=0x5621442fb380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:42.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.829+0000 7f5d37e31740 1 -- 172.21.15.43:0/3434086956 shutdown_connections 2024-09-19T05:07:42.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.829+0000 7f5d37e31740 1 -- 172.21.15.43:0/3434086956 wait complete. 2024-09-19T05:07:42.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.829+0000 7f5d37e31740 20 client.25841 trim_cache size 0 max 16384 2024-09-19T05:07:42.833 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:07:42.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.830+0000 7f5d35200640 1 -- 172.21.15.43:0/1689886199 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f5d140a3430 msgr2=0x7f5d140a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:42.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.830+0000 7f5d35200640 1 --2- 172.21.15.43:0/1689886199 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f5d140a3430 0x7f5d140a58f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5d301087b0 tx=0x7f5d18005f90 comp rx=0 tx=0).stop 2024-09-19T05:07:42.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.830+0000 7f5d35200640 1 -- 172.21.15.43:0/1689886199 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d30101de0 msgr2=0x7f5d30198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:42.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.830+0000 7f5d35200640 1 --2- 172.21.15.43:0/1689886199 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d30101de0 0x7f5d30198480 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5d2000e9f0 tx=0x7f5d2000edf0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.830+0000 7f5d35200640 1 -- 172.21.15.43:0/1689886199 shutdown_connections 2024-09-19T05:07:42.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.830+0000 7f5d35200640 1 --2- 172.21.15.43:0/1689886199 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f5d140a3430 0x7f5d140a58f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.830+0000 7f5d35200640 1 --2- 172.21.15.43:0/1689886199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d30106f10 0x7f5d3019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.830+0000 7f5d35200640 1 --2- 172.21.15.43:0/1689886199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d30102790 0x7f5d301989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.830+0000 7f5d35200640 1 --2- 172.21.15.43:0/1689886199 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d30101de0 0x7f5d30198480 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:42.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.830+0000 7f5d35200640 1 -- 172.21.15.43:0/1689886199 >> 172.21.15.43:0/1689886199 conn(0x7f5d300fd8c0 msgr2=0x7f5d300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:42.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.830+0000 7f5d35200640 1 -- 172.21.15.43:0/1689886199 shutdown_connections 2024-09-19T05:07:42.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:42.830+0000 7f5d35200640 1 -- 172.21.15.43:0/1689886199 wait complete. 2024-09-19T05:07:44.358 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:07:44 smithi043 ceph-mon[30800]: pgmap v95: 97 pgs: 97 active+clean; 13 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 221 MiB/s wr, 65 op/s 2024-09-19T05:07:44.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:07:44 smithi151 ceph-mon[35425]: pgmap v95: 97 pgs: 97 active+clean; 13 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 221 MiB/s wr, 65 op/s 2024-09-19T05:07:44.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:07:44 smithi096 ceph-mon[35085]: pgmap v95: 97 pgs: 97 active+clean; 13 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 221 MiB/s wr, 65 op/s 2024-09-19T05:07:45.361 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T05:07:45.362 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T05:07:46.093 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:07:46.334 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:07:46 smithi043 ceph-mon[30800]: pgmap v96: 97 pgs: 97 active+clean; 14 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 350 MiB/s wr, 103 op/s 2024-09-19T05:07:46.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:07:46 smithi151 ceph-mon[35425]: pgmap v96: 97 pgs: 97 active+clean; 14 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 350 MiB/s wr, 103 op/s 2024-09-19T05:07:46.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:07:46 smithi096 ceph-mon[35085]: pgmap v96: 97 pgs: 97 active+clean; 14 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 350 MiB/s wr, 103 op/s 2024-09-19T05:07:46.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.935+0000 7fc064a00640 1 -- 172.21.15.43:0/2199989041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc060077040 msgr2=0x7fc060112330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:46.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.935+0000 7fc064a00640 1 --2- 172.21.15.43:0/2199989041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc060077040 0x7fc060112330 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc04c009900 tx=0x7fc04c02a3f0 comp rx=0 tx=0).stop 2024-09-19T05:07:46.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.936+0000 7fc064a00640 1 -- 172.21.15.43:0/2199989041 shutdown_connections 2024-09-19T05:07:46.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.936+0000 7fc064a00640 1 --2- 172.21.15.43:0/2199989041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc060112870 0x7fc060114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:46.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.936+0000 7fc064a00640 1 --2- 172.21.15.43:0/2199989041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc060077040 0x7fc060112330 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:46.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.936+0000 7fc064a00640 1 --2- 172.21.15.43:0/2199989041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc060076720 0x7fc060076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:46.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.936+0000 7fc064a00640 1 -- 172.21.15.43:0/2199989041 >> 172.21.15.43:0/2199989041 conn(0x7fc0600ff8d0 msgr2=0x7fc060101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:46.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.937+0000 7fc064a00640 1 -- 172.21.15.43:0/2199989041 shutdown_connections 2024-09-19T05:07:46.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.937+0000 7fc064a00640 1 -- 172.21.15.43:0/2199989041 wait complete. 2024-09-19T05:07:46.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.938+0000 7fc064a00640 1 Processor -- start 2024-09-19T05:07:46.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.939+0000 7fc064a00640 1 -- start start 2024-09-19T05:07:46.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.940+0000 7fc064a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc060076720 0x7fc060108020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:46.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.940+0000 7fc064a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc060077040 0x7fc060108560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:46.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.940+0000 7fc064a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc060112870 0x7fc060072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:46.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.940+0000 7fc064a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc060103e70 con 0x7fc060076720 2024-09-19T05:07:46.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.940+0000 7fc064a00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc060103cf0 con 0x7fc060112870 2024-09-19T05:07:46.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.940+0000 7fc064a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc060103ff0 con 0x7fc060077040 2024-09-19T05:07:46.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.941+0000 7fc05f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc060076720 0x7fc060108020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:46.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.941+0000 7fc05fe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc060112870 0x7fc060072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:46.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.941+0000 7fc05fe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc060112870 0x7fc060072f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:37756/0 (socket says 172.21.15.43:37756) 2024-09-19T05:07:46.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.941+0000 7fc05f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc060076720 0x7fc060108020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52600/0 (socket says 172.21.15.43:52600) 2024-09-19T05:07:46.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.941+0000 7fc05fe00640 1 -- 172.21.15.43:0/1324385889 learned_addr learned my addr 172.21.15.43:0/1324385889 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:46.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.941+0000 7fc05ea00640 1 --2- 172.21.15.43:0/1324385889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc060077040 0x7fc060108560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:46.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.942+0000 7fc05f400640 1 -- 172.21.15.43:0/1324385889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc060077040 msgr2=0x7fc060108560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:46.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.942+0000 7fc05f400640 1 --2- 172.21.15.43:0/1324385889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc060077040 0x7fc060108560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:46.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.942+0000 7fc05f400640 1 -- 172.21.15.43:0/1324385889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc060112870 msgr2=0x7fc060072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:46.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.942+0000 7fc05f400640 1 --2- 172.21.15.43:0/1324385889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc060112870 0x7fc060072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:46.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.942+0000 7fc05f400640 1 -- 172.21.15.43:0/1324385889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0600737f0 con 0x7fc060076720 2024-09-19T05:07:46.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.943+0000 7fc05f400640 1 --2- 172.21.15.43:0/1324385889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc060076720 0x7fc060108020 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc05000c970 tx=0x7fc05000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:46.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.944+0000 7fc057e00640 1 -- 172.21.15.43:0/1324385889 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc050007bb0 con 0x7fc060076720 2024-09-19T05:07:46.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.944+0000 7fc057e00640 1 -- 172.21.15.43:0/1324385889 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc050007d50 con 0x7fc060076720 2024-09-19T05:07:46.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.944+0000 7fc057e00640 1 -- 172.21.15.43:0/1324385889 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05000e870 con 0x7fc060076720 2024-09-19T05:07:46.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.944+0000 7fc064a00640 1 -- 172.21.15.43:0/1324385889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc060073ae0 con 0x7fc060076720 2024-09-19T05:07:46.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.944+0000 7fc064a00640 1 -- 172.21.15.43:0/1324385889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc06010ff20 con 0x7fc060076720 2024-09-19T05:07:46.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.954+0000 7fc068c49740 1 Processor -- start 2024-09-19T05:07:46.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.957+0000 7fc068c49740 1 -- start start 2024-09-19T05:07:46.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.957+0000 7fc068c49740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ddc101cb0 0x558ddc102090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:46.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.957+0000 7fc068c49740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ddc1025d0 0x558ddc10c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:46.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.957+0000 7fc068c49740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558ddc10ce90 0x558ddc10f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:46.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.957+0000 7fc068c49740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558ddc068290 con 0x558ddc1025d0 2024-09-19T05:07:46.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.957+0000 7fc068c49740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558ddc00fb40 con 0x558ddc101cb0 2024-09-19T05:07:46.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.957+0000 7fc068c49740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558ddc064250 con 0x558ddc10ce90 2024-09-19T05:07:46.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.958+0000 7fc05ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558ddc10ce90 0x558ddc10f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:46.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.958+0000 7fc05ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558ddc10ce90 0x558ddc10f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:36340/0 (socket says 172.21.15.43:36340) 2024-09-19T05:07:46.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.958+0000 7fc05ea00640 1 -- 172.21.15.43:0/1596060635 learned_addr learned my addr 172.21.15.43:0/1596060635 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:46.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.959+0000 7fc05fe00640 1 --2- 172.21.15.43:0/1596060635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ddc101cb0 0x558ddc102090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:46.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.959+0000 7fc05ea00640 1 -- 172.21.15.43:0/1596060635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ddc101cb0 msgr2=0x558ddc102090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:46.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.959+0000 7fc05ea00640 1 --2- 172.21.15.43:0/1596060635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ddc101cb0 0x558ddc102090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:46.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.959+0000 7fc05ea00640 1 -- 172.21.15.43:0/1596060635 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ddc1025d0 msgr2=0x558ddc10c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:46.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.959+0000 7fc05ea00640 1 --2- 172.21.15.43:0/1596060635 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ddc1025d0 0x558ddc10c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:46.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.959+0000 7fc05ea00640 1 -- 172.21.15.43:0/1596060635 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ddbdd96a0 con 0x558ddc10ce90 2024-09-19T05:07:46.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.960+0000 7fc05fe00640 1 --2- 172.21.15.43:0/1596060635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ddc101cb0 0x558ddc102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:07:46.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.961+0000 7fc05ea00640 1 --2- 172.21.15.43:0/1596060635 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558ddc10ce90 0x558ddc10f280 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc04c02a960 tx=0x7fc04c0040f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=710bb83dacac767a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:46.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.962+0000 7fc04be00640 1 -- 172.21.15.43:0/1596060635 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04c035030 con 0x558ddc10ce90 2024-09-19T05:07:46.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.962+0000 7fc04be00640 1 -- 172.21.15.43:0/1596060635 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc04c005520 con 0x558ddc10ce90 2024-09-19T05:07:46.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.964+0000 7fc068c49740 1 -- 172.21.15.43:0/1596060635 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558ddc10ce90 msgr2=0x558ddc10f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:46.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.964+0000 7fc068c49740 1 --2- 172.21.15.43:0/1596060635 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558ddc10ce90 0x558ddc10f280 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc04c02a960 tx=0x7fc04c0040f0 comp rx=0 tx=0).stop 2024-09-19T05:07:46.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.964+0000 7fc068c49740 1 -- 172.21.15.43:0/1596060635 shutdown_connections 2024-09-19T05:07:46.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.964+0000 7fc068c49740 1 --2- 172.21.15.43:0/1596060635 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558ddc10ce90 0x558ddc10f280 secure :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc04c02a960 tx=0x7fc04c0040f0 comp rx=0 tx=0).stop 2024-09-19T05:07:46.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.964+0000 7fc068c49740 1 --2- 172.21.15.43:0/1596060635 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ddc1025d0 0x558ddc10c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:46.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.964+0000 7fc068c49740 1 --2- 172.21.15.43:0/1596060635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ddc101cb0 0x558ddc102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:46.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.964+0000 7fc068c49740 1 -- 172.21.15.43:0/1596060635 >> 172.21.15.43:0/1596060635 conn(0x558ddc0fe240 msgr2=0x558ddc0fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:46.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.967+0000 7fc068c49740 1 -- 172.21.15.43:0/1596060635 shutdown_connections 2024-09-19T05:07:46.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.967+0000 7fc068c49740 1 -- 172.21.15.43:0/1596060635 wait complete. 2024-09-19T05:07:46.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.968+0000 7fc068c49740 1 -- start start 2024-09-19T05:07:46.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.968+0000 7fc068c49740 1 Processor -- start 2024-09-19T05:07:46.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.969+0000 7fc068c49740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558ddc101cb0 0x558ddc109c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:46.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.969+0000 7fc068c49740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ddc1025d0 0x558ddc10a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:46.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.969+0000 7fc068c49740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ddc10a710 0x558ddc1190f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:46.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.969+0000 7fc068c49740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558ddc10fba0 con 0x558ddc10a710 2024-09-19T05:07:46.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.969+0000 7fc068c49740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558ddc022fa0 con 0x558ddc1025d0 2024-09-19T05:07:46.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.969+0000 7fc068c49740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558ddc10fd20 con 0x558ddc101cb0 2024-09-19T05:07:46.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.969+0000 7fc05fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558ddc101cb0 0x558ddc109c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:46.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.969+0000 7fc05fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558ddc101cb0 0x558ddc109c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:36342/0 (socket says 172.21.15.43:36342) 2024-09-19T05:07:46.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.969+0000 7fc05fe00640 1 -- 172.21.15.43:0/38449620 learned_addr learned my addr 172.21.15.43:0/38449620 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:46.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.969+0000 7fc068c49740 10 client.0 _ll_register_callbacks cb 0x558ddbd76ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:46.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.969+0000 7fc05f400640 1 --2- 172.21.15.43:0/38449620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ddc1025d0 0x558ddc10a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:46.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.953+0000 7fc057e00640 1 -- 172.21.15.43:0/1324385889 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fc05000ea10 con 0x7fc060076720 2024-09-19T05:07:46.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.970+0000 7fc05fe00640 1 -- 172.21.15.43:0/38449620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ddc1025d0 msgr2=0x558ddc10a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:46.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.970+0000 7fc05fe00640 1 --2- 172.21.15.43:0/38449620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ddc1025d0 0x558ddc10a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:46.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.970+0000 7fc05fe00640 1 -- 172.21.15.43:0/38449620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ddc10a710 msgr2=0x558ddc1190f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:46.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.970+0000 7fc05fe00640 1 --2- 172.21.15.43:0/38449620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ddc10a710 0x558ddc1190f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:46.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.970+0000 7fc05fe00640 1 -- 172.21.15.43:0/38449620 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ddc119630 con 0x558ddc101cb0 2024-09-19T05:07:46.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.971+0000 7fc05fe00640 1 --2- 172.21.15.43:0/38449620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558ddc101cb0 0x558ddc109c90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc058009f20 tx=0x7fc058011b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6872d1d47292184f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:46.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.971+0000 7fc048c00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:07:46.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.972+0000 7fc057e00640 1 --2- 172.21.15.43:0/1324385889 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fc0440a32c0 0x7fc0440a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:46.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.972+0000 7fc057e00640 1 -- 172.21.15.43:0/1324385889 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fc0500cfdd0 con 0x7fc060076720 2024-09-19T05:07:46.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.972+0000 7fc048c00640 1 -- 172.21.15.43:0/38449620 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05803a070 con 0x558ddc101cb0 2024-09-19T05:07:46.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.973+0000 7fc048c00640 1 -- 172.21.15.43:0/38449620 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fc058004b20 con 0x558ddc101cb0 2024-09-19T05:07:46.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.973+0000 7fc068c49740 1 -- 172.21.15.43:0/38449620 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558ddc075800 con 0x558ddc101cb0 2024-09-19T05:07:46.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.974+0000 7fc05f400640 1 --2- 172.21.15.43:0/1324385889 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fc0440a32c0 0x7fc0440a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:46.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.974+0000 7fc048c00640 1 -- 172.21.15.43:0/38449620 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc05803a070 con 0x558ddc101cb0 2024-09-19T05:07:46.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.975+0000 7fc068c49740 10 client.25810 fetch_fsmap learned FSMap version 29 2024-09-19T05:07:46.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.975+0000 7fc068c49740 1 -- 172.21.15.43:0/38449620 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558ddc11c240 con 0x558ddc101cb0 2024-09-19T05:07:46.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.975+0000 7fc05f400640 1 --2- 172.21.15.43:0/1324385889 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fc0440a32c0 0x7fc0440a5780 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x558ddc0ff0a0 tx=0x7fc05000cf60 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:46.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.981+0000 7fc048c00640 1 -- 172.21.15.43:0/38449620 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558ddc075800 con 0x558ddc101cb0 2024-09-19T05:07:46.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.981+0000 7fc048c00640 1 -- 172.21.15.43:0/38449620 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc058005480 con 0x558ddc101cb0 2024-09-19T05:07:46.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.981+0000 7fc068c49740 10 client.25810 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:07:46.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.981+0000 7fc068c49740 10 client.25810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:07:46.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.981+0000 7fc068c49740 20 client.25810 populate_metadata read hostname 'smithi043' 2024-09-19T05:07:46.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.981+0000 7fc068c49740 1 --2- 172.21.15.43:0/38449620 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558ddc11dbc0 0x558ddc120020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:46.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.981+0000 7fc068c49740 4 client.25810 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:07:46.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.981+0000 7fc068c49740 1 -- 172.21.15.43:0/38449620 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ddc06fd10 con 0x558ddc11dbc0 2024-09-19T05:07:46.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.982+0000 7fc05ea00640 1 --2- 172.21.15.43:0/38449620 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558ddc11dbc0 0x558ddc120020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:46.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.984+0000 7fc05ea00640 1 --2- 172.21.15.43:0/38449620 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558ddc11dbc0 0x558ddc120020 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fde52fd1abbc1f46 server_cookie=3e6a91bf936424d3 in_seq=0 out_seq=0 2024-09-19T05:07:46.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.984+0000 7fc048c00640 10 client.25810 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:07:46.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.985+0000 7fc048c00640 1 -- 172.21.15.43:0/38449620 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558ddc06fd10 con 0x558ddc11dbc0 2024-09-19T05:07:46.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.985+0000 7fc048c00640 10 client.25810 handle_command_reply: tid=0 2024-09-19T05:07:46.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.985+0000 7fc068c49740 1 client.25810 shutdown 2024-09-19T05:07:46.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.987+0000 7fc068c49740 1 -- 172.21.15.43:0/38449620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558ddc101cb0 msgr2=0x558ddc109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:46.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.987+0000 7fc068c49740 1 --2- 172.21.15.43:0/38449620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558ddc101cb0 0x558ddc109c90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc058009f20 tx=0x7fc058011b70 comp rx=0 tx=0).stop 2024-09-19T05:07:46.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.987+0000 7fc068c49740 1 -- 172.21.15.43:0/38449620 shutdown_connections 2024-09-19T05:07:46.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.987+0000 7fc068c49740 1 --2- 172.21.15.43:0/38449620 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558ddc11dbc0 0x558ddc120020 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:46.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.987+0000 7fc068c49740 1 --2- 172.21.15.43:0/38449620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ddc10a710 0x558ddc1190f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:46.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.987+0000 7fc068c49740 1 --2- 172.21.15.43:0/38449620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ddc1025d0 0x558ddc10a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:46.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.987+0000 7fc068c49740 1 --2- 172.21.15.43:0/38449620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558ddc101cb0 0x558ddc109c90 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:46.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.987+0000 7fc068c49740 1 -- 172.21.15.43:0/38449620 >> 172.21.15.43:0/38449620 conn(0x558ddc0fe240 msgr2=0x558ddc103fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:46.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.987+0000 7fc048c00640 0 client.25810 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:07:46.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.988+0000 7fc068c49740 1 -- 172.21.15.43:0/38449620 shutdown_connections 2024-09-19T05:07:46.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.988+0000 7fc068c49740 1 -- 172.21.15.43:0/38449620 wait complete. 2024-09-19T05:07:46.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:46.988+0000 7fc068c49740 20 client.25810 trim_cache size 0 max 16384 2024-09-19T05:07:47.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.022+0000 7fc068c49740 1 Processor -- start 2024-09-19T05:07:47.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.022+0000 7fc068c49740 1 -- start start 2024-09-19T05:07:47.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.023+0000 7fc068c49740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ddc101cb0 0x558ddc118ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:47.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.023+0000 7fc068c49740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ddc1025d0 0x558ddc105000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:47.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.023+0000 7fc068c49740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558ddc10a710 0x558ddc125930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:47.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.023+0000 7fc068c49740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc04000e7f0 con 0x558ddc101cb0 2024-09-19T05:07:47.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.023+0000 7fc068c49740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc040010330 con 0x558ddc1025d0 2024-09-19T05:07:47.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.023+0000 7fc068c49740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc04000eec0 con 0x558ddc10a710 2024-09-19T05:07:47.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.023+0000 7fc05ea00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ddc1025d0 0x558ddc105000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:47.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.023+0000 7fc05ea00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ddc1025d0 0x558ddc105000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:37776/0 (socket says 172.21.15.43:37776) 2024-09-19T05:07:47.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.023+0000 7fc05ea00640 1 -- 172.21.15.43:0/3619599592 learned_addr learned my addr 172.21.15.43:0/3619599592 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:47.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.023+0000 7fc05fe00640 1 --2- 172.21.15.43:0/3619599592 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558ddc10a710 0x558ddc125930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:47.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.023+0000 7fc05fe00640 1 --2- 172.21.15.43:0/3619599592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ddc101cb0 0x558ddc118ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:47.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.024+0000 7fc05ea00640 1 -- 172.21.15.43:0/3619599592 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558ddc10a710 msgr2=0x558ddc125930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:47.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.024+0000 7fc05ea00640 1 --2- 172.21.15.43:0/3619599592 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558ddc10a710 0x558ddc125930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.024+0000 7fc05ea00640 1 -- 172.21.15.43:0/3619599592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ddc101cb0 msgr2=0x558ddc118ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:47.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.024+0000 7fc05ea00640 1 --2- 172.21.15.43:0/3619599592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ddc101cb0 0x558ddc118ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.024+0000 7fc05ea00640 1 -- 172.21.15.43:0/3619599592 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ddc101a30 con 0x558ddc1025d0 2024-09-19T05:07:47.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.025+0000 7fc05ea00640 1 --2- 172.21.15.43:0/3619599592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ddc1025d0 0x558ddc105000 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc04c02a9e0 tx=0x7fc04c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee6f9e818a63dca5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:47.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.026+0000 7fc04b400640 1 -- 172.21.15.43:0/3619599592 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04c005270 con 0x558ddc1025d0 2024-09-19T05:07:47.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.026+0000 7fc04b400640 1 -- 172.21.15.43:0/3619599592 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc04c005410 con 0x558ddc1025d0 2024-09-19T05:07:47.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.026+0000 7fc04b400640 1 -- 172.21.15.43:0/3619599592 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04c0035f0 con 0x558ddc1025d0 2024-09-19T05:07:47.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.027+0000 7fc068c49740 1 -- 172.21.15.43:0/3619599592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ddc1025d0 msgr2=0x558ddc105000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:47.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.027+0000 7fc068c49740 1 --2- 172.21.15.43:0/3619599592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ddc1025d0 0x558ddc105000 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc04c02a9e0 tx=0x7fc04c000bc0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.027+0000 7fc068c49740 1 -- 172.21.15.43:0/3619599592 shutdown_connections 2024-09-19T05:07:47.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.027+0000 7fc068c49740 1 --2- 172.21.15.43:0/3619599592 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558ddc10a710 0x558ddc125930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.027+0000 7fc068c49740 1 --2- 172.21.15.43:0/3619599592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ddc1025d0 0x558ddc105000 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.028+0000 7fc068c49740 1 --2- 172.21.15.43:0/3619599592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ddc101cb0 0x558ddc118ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.028+0000 7fc068c49740 1 -- 172.21.15.43:0/3619599592 >> 172.21.15.43:0/3619599592 conn(0x558ddc0fe240 msgr2=0x558ddc1048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:47.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.028+0000 7fc068c49740 1 -- 172.21.15.43:0/3619599592 shutdown_connections 2024-09-19T05:07:47.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.029+0000 7fc068c49740 1 -- 172.21.15.43:0/3619599592 wait complete. 2024-09-19T05:07:47.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.029+0000 7fc068c49740 1 -- start start 2024-09-19T05:07:47.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.030+0000 7fc068c49740 1 Processor -- start 2024-09-19T05:07:47.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.031+0000 7fc068c49740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ddc101cb0 0x558ddc1152a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:47.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.031+0000 7fc068c49740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558ddc1025d0 0x558ddc1157e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:47.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.031+0000 7fc068c49740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ddc10a710 0x558ddc1064c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:47.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.031+0000 7fc068c49740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc04000fc60 con 0x558ddc101cb0 2024-09-19T05:07:47.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.031+0000 7fc068c49740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc04000f590 con 0x558ddc10a710 2024-09-19T05:07:47.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.031+0000 7fc068c49740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc04000c7d0 con 0x558ddc1025d0 2024-09-19T05:07:47.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.031+0000 7fc068c49740 10 client.0 _ll_register_callbacks cb 0x7fc058009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:47.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.033+0000 7fc05ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558ddc1025d0 0x558ddc1157e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:47.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.034+0000 7fc05fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ddc101cb0 0x558ddc1152a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:47.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.034+0000 7fc05fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ddc101cb0 0x558ddc1152a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52640/0 (socket says 172.21.15.43:52640) 2024-09-19T05:07:47.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.034+0000 7fc05fe00640 1 -- 172.21.15.43:0/2953603132 learned_addr learned my addr 172.21.15.43:0/2953603132 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:47.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.035+0000 7fc05fe00640 1 --2- 172.21.15.43:0/2953603132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ddc10a710 0x558ddc1064c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:47.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.035+0000 7fc05ea00640 1 -- 172.21.15.43:0/2953603132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ddc10a710 msgr2=0x558ddc1064c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:47.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.035+0000 7fc05ea00640 1 --2- 172.21.15.43:0/2953603132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ddc10a710 0x558ddc1064c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.035+0000 7fc05ea00640 1 -- 172.21.15.43:0/2953603132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ddc101cb0 msgr2=0x558ddc1152a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:47.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.035+0000 7fc05ea00640 1 --2- 172.21.15.43:0/2953603132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ddc101cb0 0x558ddc1152a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.036+0000 7fc05ea00640 1 -- 172.21.15.43:0/2953603132 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ddc106ca0 con 0x558ddc1025d0 2024-09-19T05:07:47.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.036+0000 7fc05ea00640 1 --2- 172.21.15.43:0/2953603132 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558ddc1025d0 0x558ddc1157e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc04c009900 tx=0x7fc04c000ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4391fee9bda677e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:47.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.037+0000 7fc05fe00640 1 --2- 172.21.15.43:0/2953603132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ddc101cb0 0x558ddc1152a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:07:47.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.037+0000 7fc048c00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:07:47.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.037+0000 7fc048c00640 1 -- 172.21.15.43:0/2953603132 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04c009520 con 0x558ddc1025d0 2024-09-19T05:07:47.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.037+0000 7fc048c00640 1 -- 172.21.15.43:0/2953603132 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fc04c004d20 con 0x558ddc1025d0 2024-09-19T05:07:47.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.040+0000 7fc068c49740 1 -- 172.21.15.43:0/2953603132 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc04c005410 con 0x558ddc1025d0 2024-09-19T05:07:47.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.040+0000 7fc048c00640 1 -- 172.21.15.43:0/2953603132 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc04c0055c0 con 0x558ddc1025d0 2024-09-19T05:07:47.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.040+0000 7fc068c49740 10 client.25816 fetch_fsmap learned FSMap version 29 2024-09-19T05:07:47.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.040+0000 7fc068c49740 1 -- 172.21.15.43:0/2953603132 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558ddc188420 con 0x558ddc1025d0 2024-09-19T05:07:47.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.040+0000 7fc048c00640 1 -- 172.21.15.43:0/2953603132 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc04c005410 con 0x558ddc1025d0 2024-09-19T05:07:47.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.041+0000 7fc048c00640 1 -- 172.21.15.43:0/2953603132 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc04c003d00 con 0x558ddc1025d0 2024-09-19T05:07:47.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.041+0000 7fc068c49740 10 client.25816 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:07:47.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.041+0000 7fc068c49740 10 client.25816 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:07:47.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.041+0000 7fc068c49740 20 client.25816 populate_metadata read hostname 'smithi043' 2024-09-19T05:07:47.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.041+0000 7fc068c49740 1 --2- 172.21.15.43:0/2953603132 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558ddc188650 0x558ddc18aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:47.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.041+0000 7fc068c49740 4 client.25816 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T05:07:47.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.041+0000 7fc068c49740 1 -- 172.21.15.43:0/2953603132 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558ddbfebfb0 con 0x558ddc188650 2024-09-19T05:07:47.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.041+0000 7fc05fe00640 1 --2- 172.21.15.43:0/2953603132 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558ddc188650 0x558ddc18aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:47.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.042+0000 7fc048c00640 10 client.25816 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:07:47.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.042+0000 7fc05fe00640 1 --2- 172.21.15.43:0/2953603132 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558ddc188650 0x558ddc18aa40 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc04719bdf6f1d44 server_cookie=4263b68d373e3674 in_seq=0 out_seq=0 2024-09-19T05:07:47.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.043+0000 7fc048c00640 1 -- 172.21.15.43:0/2953603132 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558ddbfebfb0 con 0x558ddc188650 2024-09-19T05:07:47.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.043+0000 7fc048c00640 10 client.25816 handle_command_reply: tid=0 2024-09-19T05:07:47.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.043+0000 7fc068c49740 1 client.25816 shutdown 2024-09-19T05:07:47.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.044+0000 7fc068c49740 1 -- 172.21.15.43:0/2953603132 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558ddc1025d0 msgr2=0x558ddc1157e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:47.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.044+0000 7fc068c49740 1 --2- 172.21.15.43:0/2953603132 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558ddc1025d0 0x558ddc1157e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc04c009900 tx=0x7fc04c000ed0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.044+0000 7fc068c49740 1 -- 172.21.15.43:0/2953603132 shutdown_connections 2024-09-19T05:07:47.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.044+0000 7fc068c49740 1 --2- 172.21.15.43:0/2953603132 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558ddc188650 0x558ddc18aa40 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.044+0000 7fc068c49740 1 --2- 172.21.15.43:0/2953603132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558ddc10a710 0x558ddc1064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.044+0000 7fc068c49740 1 --2- 172.21.15.43:0/2953603132 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558ddc1025d0 0x558ddc1157e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.044+0000 7fc068c49740 1 --2- 172.21.15.43:0/2953603132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ddc101cb0 0x558ddc1152a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.044+0000 7fc068c49740 1 -- 172.21.15.43:0/2953603132 >> 172.21.15.43:0/2953603132 conn(0x558ddc0fe240 msgr2=0x558ddc105110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:47.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.045+0000 7fc048c00640 0 client.25816 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:07:47.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.046+0000 7fc068c49740 1 -- 172.21.15.43:0/2953603132 shutdown_connections 2024-09-19T05:07:47.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.046+0000 7fc068c49740 1 -- 172.21.15.43:0/2953603132 wait complete. 2024-09-19T05:07:47.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.046+0000 7fc068c49740 20 client.25816 trim_cache size 0 max 16384 2024-09-19T05:07:47.050 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:07:47.050 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-09-19T05:07:47.050 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "073df51f-a293-48c7-8cc7-9a8b04610df4", 2024-09-19T05:07:47.050 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-09-19T05:07:47.050 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:07:47.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.049+0000 7fc064a00640 1 -- 172.21.15.43:0/1324385889 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fc0440a32c0 msgr2=0x7fc0440a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:47.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.049+0000 7fc064a00640 1 --2- 172.21.15.43:0/1324385889 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fc0440a32c0 0x7fc0440a5780 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x558ddc0ff0a0 tx=0x7fc05000cf60 comp rx=0 tx=0).stop 2024-09-19T05:07:47.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.049+0000 7fc064a00640 1 -- 172.21.15.43:0/1324385889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc060076720 msgr2=0x7fc060108020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:47.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.049+0000 7fc064a00640 1 --2- 172.21.15.43:0/1324385889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc060076720 0x7fc060108020 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc05000c970 tx=0x7fc05000ce30 comp rx=0 tx=0).stop 2024-09-19T05:07:47.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.050+0000 7fc064a00640 1 -- 172.21.15.43:0/1324385889 shutdown_connections 2024-09-19T05:07:47.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.050+0000 7fc064a00640 1 --2- 172.21.15.43:0/1324385889 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fc0440a32c0 0x7fc0440a5780 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.050+0000 7fc064a00640 1 --2- 172.21.15.43:0/1324385889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc060112870 0x7fc060072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.050+0000 7fc064a00640 1 --2- 172.21.15.43:0/1324385889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc060077040 0x7fc060108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.050+0000 7fc064a00640 1 --2- 172.21.15.43:0/1324385889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc060076720 0x7fc060108020 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:47.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.050+0000 7fc064a00640 1 -- 172.21.15.43:0/1324385889 >> 172.21.15.43:0/1324385889 conn(0x7fc0600ff8d0 msgr2=0x7fc060101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:47.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.051+0000 7fc064a00640 1 -- 172.21.15.43:0/1324385889 shutdown_connections 2024-09-19T05:07:47.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:47.051+0000 7fc064a00640 1 -- 172.21.15.43:0/1324385889 wait complete. 2024-09-19T05:07:48.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:07:48 smithi043 ceph-mon[30800]: pgmap v97: 97 pgs: 97 active+clean; 14 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 255 MiB/s wr, 74 op/s 2024-09-19T05:07:48.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:07:48 smithi043 ceph-mon[30800]: scrub queued for path: / 2024-09-19T05:07:48.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:07:48 smithi043 ceph-mon[30800]: scrub summary: idle+waiting paths [/] 2024-09-19T05:07:48.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:07:48 smithi043 ceph-mon[30800]: scrub summary: active paths [/] 2024-09-19T05:07:48.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:07:48 smithi043 ceph-mon[30800]: scrub summary: idle+waiting paths [/] 2024-09-19T05:07:48.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:07:48 smithi043 ceph-mon[30800]: scrub summary: idle 2024-09-19T05:07:48.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:07:48 smithi151 ceph-mon[35425]: pgmap v97: 97 pgs: 97 active+clean; 14 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 255 MiB/s wr, 74 op/s 2024-09-19T05:07:48.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:07:48 smithi151 ceph-mon[35425]: scrub queued for path: / 2024-09-19T05:07:48.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:07:48 smithi151 ceph-mon[35425]: scrub summary: idle+waiting paths [/] 2024-09-19T05:07:48.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:07:48 smithi151 ceph-mon[35425]: scrub summary: active paths [/] 2024-09-19T05:07:48.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:07:48 smithi151 ceph-mon[35425]: scrub summary: idle+waiting paths [/] 2024-09-19T05:07:48.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:07:48 smithi151 ceph-mon[35425]: scrub summary: idle 2024-09-19T05:07:48.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:07:48 smithi096 ceph-mon[35085]: pgmap v97: 97 pgs: 97 active+clean; 14 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 255 MiB/s wr, 74 op/s 2024-09-19T05:07:48.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:07:48 smithi096 ceph-mon[35085]: scrub queued for path: / 2024-09-19T05:07:48.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:07:48 smithi096 ceph-mon[35085]: scrub summary: idle+waiting paths [/] 2024-09-19T05:07:48.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:07:48 smithi096 ceph-mon[35085]: scrub summary: active paths [/] 2024-09-19T05:07:48.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:07:48 smithi096 ceph-mon[35085]: scrub summary: idle+waiting paths [/] 2024-09-19T05:07:48.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:07:48 smithi096 ceph-mon[35085]: scrub summary: idle 2024-09-19T05:07:48.610 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:07:49.397 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:07:50.284 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:07:50 smithi043 ceph-mon[30800]: pgmap v98: 97 pgs: 97 active+clean; 14 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 255 MiB/s wr, 74 op/s 2024-09-19T05:07:50.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.318+0000 7f7c5ea00640 1 -- 172.21.15.43:0/1362743017 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c58108450 msgr2=0x7f7c58108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:50.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.318+0000 7f7c5ea00640 1 --2- 172.21.15.43:0/1362743017 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c58108450 0x7f7c58108830 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009900 tx=0x7f7c4802a440 comp rx=0 tx=0).stop 2024-09-19T05:07:50.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.318+0000 7f7c5ea00640 1 -- 172.21.15.43:0/1362743017 shutdown_connections 2024-09-19T05:07:50.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.318+0000 7f7c5ea00640 1 --2- 172.21.15.43:0/1362743017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c5810d530 0x7f7c581150b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.318+0000 7f7c5ea00640 1 --2- 172.21.15.43:0/1362743017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c58108e00 0x7f7c5810ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.318+0000 7f7c5ea00640 1 --2- 172.21.15.43:0/1362743017 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c58108450 0x7f7c58108830 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.318+0000 7f7c5ea00640 1 -- 172.21.15.43:0/1362743017 >> 172.21.15.43:0/1362743017 conn(0x7f7c58077ba0 msgr2=0x7f7c58079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:50.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.319+0000 7f7c5ea00640 1 -- 172.21.15.43:0/1362743017 shutdown_connections 2024-09-19T05:07:50.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.319+0000 7f7c5ea00640 1 -- 172.21.15.43:0/1362743017 wait complete. 2024-09-19T05:07:50.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.320+0000 7f7c5ea00640 1 Processor -- start 2024-09-19T05:07:50.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.321+0000 7f7c5ea00640 1 -- start start 2024-09-19T05:07:50.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.321+0000 7f7c5ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c58108450 0x7f7c58073940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:50.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.321+0000 7f7c5ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c58108e00 0x7f7c58073e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:50.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.321+0000 7f7c5ea00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c5810d530 0x7f7c5806cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:50.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.321+0000 7f7c5ea00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7c58118580 con 0x7f7c58108e00 2024-09-19T05:07:50.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.321+0000 7f7c5ea00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7c58118400 con 0x7f7c5810d530 2024-09-19T05:07:50.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.321+0000 7f7c5ea00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7c58118700 con 0x7f7c58108450 2024-09-19T05:07:50.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.322+0000 7f7c5ca00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c5810d530 0x7f7c5806cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:50.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.322+0000 7f7c5ca00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c5810d530 0x7f7c5806cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:37796/0 (socket says 172.21.15.43:37796) 2024-09-19T05:07:50.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.322+0000 7f7c5ca00640 1 -- 172.21.15.43:0/3497968991 learned_addr learned my addr 172.21.15.43:0/3497968991 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:50.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.323+0000 7f7c5ca00640 1 -- 172.21.15.43:0/3497968991 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c58108450 msgr2=0x7f7c58073940 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T05:07:50.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.323+0000 7f7c5ca00640 1 --2- 172.21.15.43:0/3497968991 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c58108450 0x7f7c58073940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.323+0000 7f7c5ca00640 1 -- 172.21.15.43:0/3497968991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c58108e00 msgr2=0x7f7c58073e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:50.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.323+0000 7f7c5ca00640 1 --2- 172.21.15.43:0/3497968991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c58108e00 0x7f7c58073e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.323+0000 7f7c5ca00640 1 -- 172.21.15.43:0/3497968991 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c5806d470 con 0x7f7c5810d530 2024-09-19T05:07:50.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.323+0000 7f7c5ca00640 1 --2- 172.21.15.43:0/3497968991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c5810d530 0x7f7c5806cbb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000e960 tx=0x7f7c5000ee20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:50.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.324+0000 7f7c54c00640 1 -- 172.21.15.43:0/3497968991 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c50004d20 con 0x7f7c5810d530 2024-09-19T05:07:50.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.324+0000 7f7c54c00640 1 -- 172.21.15.43:0/3497968991 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c50005020 con 0x7f7c5810d530 2024-09-19T05:07:50.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.324+0000 7f7c54c00640 1 -- 172.21.15.43:0/3497968991 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c50004d20 con 0x7f7c5810d530 2024-09-19T05:07:50.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.324+0000 7f7c5ea00640 1 -- 172.21.15.43:0/3497968991 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c5806d700 con 0x7f7c5810d530 2024-09-19T05:07:50.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.325+0000 7f7c5ea00640 1 -- 172.21.15.43:0/3497968991 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c58071640 con 0x7f7c5810d530 2024-09-19T05:07:50.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.330+0000 7f7c613d1740 1 Processor -- start 2024-09-19T05:07:50.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.332+0000 7f7c54c00640 1 -- 172.21.15.43:0/3497968991 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f7c5000cb20 con 0x7f7c5810d530 2024-09-19T05:07:50.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.336+0000 7f7c613d1740 1 -- start start 2024-09-19T05:07:50.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.342+0000 7f7c613d1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb3d613b90 0x55fb3d613f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:50.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.342+0000 7f7c613d1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb3d6144b0 0x55fb3d61e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:50.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.342+0000 7f7c613d1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb3d61ed70 0x55fb3d621160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:50.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.342+0000 7f7c613d1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fb3d57a3e0 con 0x55fb3d613b90 2024-09-19T05:07:50.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.342+0000 7f7c613d1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fb3d521d80 con 0x55fb3d6144b0 2024-09-19T05:07:50.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.342+0000 7f7c613d1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55fb3d5763a0 con 0x55fb3d61ed70 2024-09-19T05:07:50.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.344+0000 7f7c57400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb3d61ed70 0x55fb3d621160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:50.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.345+0000 7f7c57e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb3d6144b0 0x55fb3d61e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:50.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.345+0000 7f7c57e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb3d6144b0 0x55fb3d61e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:37808/0 (socket says 172.21.15.43:37808) 2024-09-19T05:07:50.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.345+0000 7f7c57e00640 1 -- 172.21.15.43:0/4159106980 learned_addr learned my addr 172.21.15.43:0/4159106980 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:50.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.346+0000 7f7c57400640 1 --2- 172.21.15.43:0/4159106980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb3d613b90 0x55fb3d613f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:50.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.347+0000 7f7c54c00640 1 --2- 172.21.15.43:0/3497968991 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f7c3c0a3630 0x7f7c3c0a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:50.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.347+0000 7f7c54c00640 1 -- 172.21.15.43:0/3497968991 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f7c500cfac0 con 0x7f7c5810d530 2024-09-19T05:07:50.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.347+0000 7f7c57400640 1 -- 172.21.15.43:0/4159106980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb3d6144b0 msgr2=0x55fb3d61e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:50.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.347+0000 7f7c57400640 1 --2- 172.21.15.43:0/4159106980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb3d6144b0 0x55fb3d61e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.347+0000 7f7c57400640 1 -- 172.21.15.43:0/4159106980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb3d613b90 msgr2=0x55fb3d613f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:50.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.347+0000 7f7c57400640 1 --2- 172.21.15.43:0/4159106980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb3d613b90 0x55fb3d613f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.347+0000 7f7c57400640 1 -- 172.21.15.43:0/4159106980 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb3d540ae0 con 0x55fb3d61ed70 2024-09-19T05:07:50.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.348+0000 7f7c57400640 1 --2- 172.21.15.43:0/4159106980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb3d61ed70 0x55fb3d621160 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7c5806e190 tx=0x7f7c44009530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0ffa3f67ad05218 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:50.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.349+0000 7f7c5ca00640 1 --2- 172.21.15.43:0/3497968991 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f7c3c0a3630 0x7f7c3c0a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:50.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.349+0000 7f7c4d200640 1 -- 172.21.15.43:0/4159106980 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c44005020 con 0x55fb3d61ed70 2024-09-19T05:07:50.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.349+0000 7f7c4d200640 1 -- 172.21.15.43:0/4159106980 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c440047a0 con 0x55fb3d61ed70 2024-09-19T05:07:50.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.349+0000 7f7c4d200640 1 -- 172.21.15.43:0/4159106980 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c440116a0 con 0x55fb3d61ed70 2024-09-19T05:07:50.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.350+0000 7f7c5ca00640 1 --2- 172.21.15.43:0/3497968991 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f7c3c0a3630 0x7f7c3c0a5af0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7c50098780 tx=0x7f7c50004a40 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:50.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.352+0000 7f7c613d1740 1 -- 172.21.15.43:0/4159106980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb3d61ed70 msgr2=0x55fb3d621160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:50.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.352+0000 7f7c613d1740 1 --2- 172.21.15.43:0/4159106980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb3d61ed70 0x55fb3d621160 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7c5806e190 tx=0x7f7c44009530 comp rx=0 tx=0).stop 2024-09-19T05:07:50.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.352+0000 7f7c613d1740 1 -- 172.21.15.43:0/4159106980 shutdown_connections 2024-09-19T05:07:50.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.352+0000 7f7c613d1740 1 --2- 172.21.15.43:0/4159106980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb3d61ed70 0x55fb3d621160 secure :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7c5806e190 tx=0x7f7c44009530 comp rx=0 tx=0).stop 2024-09-19T05:07:50.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.352+0000 7f7c613d1740 1 --2- 172.21.15.43:0/4159106980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb3d6144b0 0x55fb3d61e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.352+0000 7f7c613d1740 1 --2- 172.21.15.43:0/4159106980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb3d613b90 0x55fb3d613f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.352+0000 7f7c613d1740 1 -- 172.21.15.43:0/4159106980 >> 172.21.15.43:0/4159106980 conn(0x55fb3d610120 msgr2=0x55fb3d610500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:50.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.352+0000 7f7c613d1740 1 -- 172.21.15.43:0/4159106980 shutdown_connections 2024-09-19T05:07:50.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.352+0000 7f7c613d1740 1 -- 172.21.15.43:0/4159106980 wait complete. 2024-09-19T05:07:50.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.353+0000 7f7c613d1740 1 -- start start 2024-09-19T05:07:50.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.354+0000 7f7c613d1740 1 Processor -- start 2024-09-19T05:07:50.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.354+0000 7f7c613d1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb3d613b90 0x55fb3d61bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:50.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.354+0000 7f7c613d1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb3d6144b0 0x55fb3d61c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:50.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.354+0000 7f7c613d1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb3d61cab0 0x55fb3d62ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:50.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.354+0000 7f7c613d1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fb3d6219e0 con 0x55fb3d613b90 2024-09-19T05:07:50.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.354+0000 7f7c613d1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fb3d5350f0 con 0x55fb3d6144b0 2024-09-19T05:07:50.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.354+0000 7f7c613d1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55fb3d621b60 con 0x55fb3d61cab0 2024-09-19T05:07:50.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.355+0000 7f7c613d1740 10 client.0 _ll_register_callbacks cb 0x55fb3d5269e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:50.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.356+0000 7f7c57e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb3d6144b0 0x55fb3d61c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:50.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.356+0000 7f7c57e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb3d6144b0 0x55fb3d61c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:37812/0 (socket says 172.21.15.43:37812) 2024-09-19T05:07:50.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.356+0000 7f7c57e00640 1 -- 172.21.15.43:0/3005823535 learned_addr learned my addr 172.21.15.43:0/3005823535 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:50.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.357+0000 7f7c57e00640 1 -- 172.21.15.43:0/3005823535 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb3d61cab0 msgr2=0x55fb3d62ae50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:07:50.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.357+0000 7f7c57e00640 1 --2- 172.21.15.43:0/3005823535 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb3d61cab0 0x55fb3d62ae50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.357+0000 7f7c57400640 1 --2- 172.21.15.43:0/3005823535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb3d613b90 0x55fb3d61bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:50.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.357+0000 7f7c57e00640 1 -- 172.21.15.43:0/3005823535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb3d613b90 msgr2=0x55fb3d61bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:50.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.359+0000 7f7c57e00640 1 --2- 172.21.15.43:0/3005823535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb3d613b90 0x55fb3d61bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.359+0000 7f7c57e00640 1 -- 172.21.15.43:0/3005823535 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb3d2eb3c0 con 0x55fb3d6144b0 2024-09-19T05:07:50.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.359+0000 7f7c57400640 1 --2- 172.21.15.43:0/3005823535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb3d613b90 0x55fb3d61bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:07:50.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.360+0000 7f7c57e00640 1 --2- 172.21.15.43:0/3005823535 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb3d6144b0 0x55fb3d61c4d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009900 tx=0x7f7c480048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=262dedd389faa419 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:50.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.360+0000 7f7c41600640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:07:50.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.362+0000 7f7c41600640 1 -- 172.21.15.43:0/3005823535 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c48003f60 con 0x55fb3d6144b0 2024-09-19T05:07:50.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.363+0000 7f7c41600640 1 -- 172.21.15.43:0/3005823535 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f7c48004100 con 0x55fb3d6144b0 2024-09-19T05:07:50.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.363+0000 7f7c613d1740 1 -- 172.21.15.43:0/3005823535 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c440047a0 con 0x55fb3d6144b0 2024-09-19T05:07:50.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.365+0000 7f7c41600640 1 -- 172.21.15.43:0/3005823535 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7c48004a60 con 0x55fb3d6144b0 2024-09-19T05:07:50.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.365+0000 7f7c613d1740 10 client.25868 fetch_fsmap learned FSMap version 29 2024-09-19T05:07:50.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.366+0000 7f7c613d1740 1 -- 172.21.15.43:0/3005823535 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fb3d62dda0 con 0x55fb3d6144b0 2024-09-19T05:07:50.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.368+0000 7f7c41600640 1 -- 172.21.15.43:0/3005823535 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c440047a0 con 0x55fb3d6144b0 2024-09-19T05:07:50.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.368+0000 7f7c41600640 1 -- 172.21.15.43:0/3005823535 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7c48046470 con 0x55fb3d6144b0 2024-09-19T05:07:50.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.368+0000 7f7c613d1740 10 client.25868 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:07:50.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.368+0000 7f7c613d1740 10 client.25868 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:07:50.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.368+0000 7f7c613d1740 20 client.25868 populate_metadata read hostname 'smithi043' 2024-09-19T05:07:50.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.368+0000 7f7c613d1740 1 --2- 172.21.15.43:0/3005823535 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55fb3d62f260 0x55fb3d631690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:50.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.368+0000 7f7c613d1740 4 client.25868 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:07:50.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.368+0000 7f7c613d1740 1 -- 172.21.15.43:0/3005823535 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb3d587950 con 0x55fb3d62f260 2024-09-19T05:07:50.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.370+0000 7f7c57400640 1 --2- 172.21.15.43:0/3005823535 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55fb3d62f260 0x55fb3d631690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:50.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.372+0000 7f7c57400640 1 --2- 172.21.15.43:0/3005823535 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55fb3d62f260 0x55fb3d631690 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a1cde85923761ff server_cookie=d790e05172adf9aa in_seq=0 out_seq=0 2024-09-19T05:07:50.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.372+0000 7f7c41600640 10 client.25868 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:07:50.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.374+0000 7f7c41600640 1 -- 172.21.15.43:0/3005823535 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fb3d587950 con 0x55fb3d62f260 2024-09-19T05:07:50.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.374+0000 7f7c41600640 10 client.25868 handle_command_reply: tid=0 2024-09-19T05:07:50.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.375+0000 7f7c613d1740 1 client.25868 shutdown 2024-09-19T05:07:50.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.375+0000 7f7c613d1740 1 -- 172.21.15.43:0/3005823535 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb3d6144b0 msgr2=0x55fb3d61c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:50.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.375+0000 7f7c613d1740 1 --2- 172.21.15.43:0/3005823535 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb3d6144b0 0x55fb3d61c4d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009900 tx=0x7f7c480048f0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.376+0000 7f7c613d1740 1 -- 172.21.15.43:0/3005823535 shutdown_connections 2024-09-19T05:07:50.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.376+0000 7f7c613d1740 1 --2- 172.21.15.43:0/3005823535 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55fb3d62f260 0x55fb3d631690 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.376+0000 7f7c41600640 0 client.25868 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:07:50.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.376+0000 7f7c613d1740 1 --2- 172.21.15.43:0/3005823535 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb3d61cab0 0x55fb3d62ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.377+0000 7f7c613d1740 1 --2- 172.21.15.43:0/3005823535 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb3d6144b0 0x55fb3d61c4d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.377+0000 7f7c613d1740 1 --2- 172.21.15.43:0/3005823535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb3d613b90 0x55fb3d61bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.377+0000 7f7c613d1740 1 -- 172.21.15.43:0/3005823535 >> 172.21.15.43:0/3005823535 conn(0x55fb3d610120 msgr2=0x55fb3d615e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:50.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.377+0000 7f7c613d1740 1 -- 172.21.15.43:0/3005823535 shutdown_connections 2024-09-19T05:07:50.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.378+0000 7f7c613d1740 1 -- 172.21.15.43:0/3005823535 wait complete. 2024-09-19T05:07:50.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.378+0000 7f7c613d1740 20 client.25868 trim_cache size 0 max 16384 2024-09-19T05:07:50.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:07:50 smithi151 ceph-mon[35425]: pgmap v98: 97 pgs: 97 active+clean; 14 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 255 MiB/s wr, 74 op/s 2024-09-19T05:07:50.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.408+0000 7f7c613d1740 1 Processor -- start 2024-09-19T05:07:50.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.408+0000 7f7c613d1740 1 -- start start 2024-09-19T05:07:50.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.409+0000 7f7c613d1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb3d613b90 0x55fb3d61bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:50.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.409+0000 7f7c613d1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb3d6144b0 0x55fb3d62ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:50.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.409+0000 7f7c613d1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb3d61cab0 0x55fb3d615d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:50.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.409+0000 7f7c613d1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7c581bc700 con 0x55fb3d61cab0 2024-09-19T05:07:50.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.409+0000 7f7c613d1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7c581be240 con 0x55fb3d6144b0 2024-09-19T05:07:50.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.409+0000 7f7c613d1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7c581bcdd0 con 0x55fb3d613b90 2024-09-19T05:07:50.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.409+0000 7f7c57e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb3d6144b0 0x55fb3d62ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:50.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.409+0000 7f7c57e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb3d6144b0 0x55fb3d62ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:37826/0 (socket says 172.21.15.43:37826) 2024-09-19T05:07:50.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.409+0000 7f7c57e00640 1 -- 172.21.15.43:0/1149770437 learned_addr learned my addr 172.21.15.43:0/1149770437 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:50.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.410+0000 7f7c57400640 1 --2- 172.21.15.43:0/1149770437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb3d61cab0 0x55fb3d615d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:50.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.411+0000 7f7c57400640 1 -- 172.21.15.43:0/1149770437 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb3d613b90 msgr2=0x55fb3d61bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:50.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.411+0000 7f7c57400640 1 --2- 172.21.15.43:0/1149770437 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb3d613b90 0x55fb3d61bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.411+0000 7f7c57400640 1 -- 172.21.15.43:0/1149770437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb3d6144b0 msgr2=0x55fb3d62ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:50.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.411+0000 7f7c57400640 1 --2- 172.21.15.43:0/1149770437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb3d6144b0 0x55fb3d62ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.411+0000 7f7c57400640 1 -- 172.21.15.43:0/1149770437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb3d61af40 con 0x55fb3d61cab0 2024-09-19T05:07:50.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.411+0000 7f7c57400640 1 --2- 172.21.15.43:0/1149770437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb3d61cab0 0x55fb3d615d70 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x55fb3d6111d0 tx=0x7f7c44015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513b65a2602ee265 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:50.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.412+0000 7f7c43e00640 1 -- 172.21.15.43:0/1149770437 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c440132f0 con 0x55fb3d61cab0 2024-09-19T05:07:50.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.412+0000 7f7c43e00640 1 -- 172.21.15.43:0/1149770437 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55fb3d587950 con 0x55fb3d61cab0 2024-09-19T05:07:50.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.412+0000 7f7c43e00640 1 -- 172.21.15.43:0/1149770437 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c44005130 con 0x55fb3d61cab0 2024-09-19T05:07:50.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.413+0000 7f7c613d1740 1 -- 172.21.15.43:0/1149770437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb3d61cab0 msgr2=0x55fb3d615d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:50.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.413+0000 7f7c613d1740 1 --2- 172.21.15.43:0/1149770437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb3d61cab0 0x55fb3d615d70 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x55fb3d6111d0 tx=0x7f7c44015040 comp rx=0 tx=0).stop 2024-09-19T05:07:50.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.413+0000 7f7c613d1740 1 -- 172.21.15.43:0/1149770437 shutdown_connections 2024-09-19T05:07:50.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.413+0000 7f7c613d1740 1 --2- 172.21.15.43:0/1149770437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb3d61cab0 0x55fb3d615d70 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.413+0000 7f7c613d1740 1 --2- 172.21.15.43:0/1149770437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb3d6144b0 0x55fb3d62ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.413+0000 7f7c613d1740 1 --2- 172.21.15.43:0/1149770437 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb3d613b90 0x55fb3d61bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.413+0000 7f7c613d1740 1 -- 172.21.15.43:0/1149770437 >> 172.21.15.43:0/1149770437 conn(0x55fb3d610120 msgr2=0x55fb3d615660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:50.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.414+0000 7f7c613d1740 1 -- 172.21.15.43:0/1149770437 shutdown_connections 2024-09-19T05:07:50.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.414+0000 7f7c613d1740 1 -- 172.21.15.43:0/1149770437 wait complete. 2024-09-19T05:07:50.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.415+0000 7f7c613d1740 1 -- start start 2024-09-19T05:07:50.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.415+0000 7f7c613d1740 1 Processor -- start 2024-09-19T05:07:50.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.416+0000 7f7c613d1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb3d613b90 0x55fb3d627200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:50.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.416+0000 7f7c613d1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb3d6144b0 0x55fb3d6234a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:50.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.416+0000 7f7c613d1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb3d62f260 0x55fb3d6239e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:50.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.416+0000 7f7c613d1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7c581bdb70 con 0x55fb3d613b90 2024-09-19T05:07:50.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.416+0000 7f7c613d1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7c581bd4a0 con 0x55fb3d62f260 2024-09-19T05:07:50.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.416+0000 7f7c613d1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7c581ba6e0 con 0x55fb3d6144b0 2024-09-19T05:07:50.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.416+0000 7f7c613d1740 10 client.0 _ll_register_callbacks cb 0x7f7c48005d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:50.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.416+0000 7f7c57e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb3d6144b0 0x55fb3d6234a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:50.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.416+0000 7f7c57e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb3d6144b0 0x55fb3d6234a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:36398/0 (socket says 172.21.15.43:36398) 2024-09-19T05:07:50.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.416+0000 7f7c57e00640 1 -- 172.21.15.43:0/4222118685 learned_addr learned my addr 172.21.15.43:0/4222118685 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:50.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.416+0000 7f7c57400640 1 --2- 172.21.15.43:0/4222118685 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb3d62f260 0x55fb3d6239e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:50.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.417+0000 7f7c57400640 1 --2- 172.21.15.43:0/4222118685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb3d613b90 0x55fb3d627200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:50.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.417+0000 7f7c57e00640 1 -- 172.21.15.43:0/4222118685 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb3d62f260 msgr2=0x55fb3d6239e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:50.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.417+0000 7f7c57e00640 1 --2- 172.21.15.43:0/4222118685 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb3d62f260 0x55fb3d6239e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.417+0000 7f7c57e00640 1 -- 172.21.15.43:0/4222118685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb3d613b90 msgr2=0x55fb3d627200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:50.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.417+0000 7f7c57e00640 1 --2- 172.21.15.43:0/4222118685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb3d613b90 0x55fb3d627200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.417+0000 7f7c57e00640 1 -- 172.21.15.43:0/4222118685 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb3d6214d0 con 0x55fb3d6144b0 2024-09-19T05:07:50.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.417+0000 7f7c57400640 1 --2- 172.21.15.43:0/4222118685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb3d613b90 0x55fb3d627200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:07:50.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.418+0000 7f7c57e00640 1 --2- 172.21.15.43:0/4222118685 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb3d6144b0 0x55fb3d6234a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7c48004c20 tx=0x7f7c48046c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ec34d1054e4197a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:50.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.418+0000 7f7c41600640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:07:50.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.418+0000 7f7c41600640 1 -- 172.21.15.43:0/4222118685 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c48055070 con 0x55fb3d6144b0 2024-09-19T05:07:50.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.418+0000 7f7c41600640 1 -- 172.21.15.43:0/4222118685 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f7c48004170 con 0x55fb3d6144b0 2024-09-19T05:07:50.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.418+0000 7f7c613d1740 1 -- 172.21.15.43:0/4222118685 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fb3d587950 con 0x55fb3d6144b0 2024-09-19T05:07:50.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.418+0000 7f7c41600640 1 -- 172.21.15.43:0/4222118685 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7c4802aac0 con 0x55fb3d6144b0 2024-09-19T05:07:50.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.421+0000 7f7c613d1740 10 client.25825 fetch_fsmap learned FSMap version 29 2024-09-19T05:07:50.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.421+0000 7f7c613d1740 1 -- 172.21.15.43:0/4222118685 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fb3d69aaa0 con 0x55fb3d6144b0 2024-09-19T05:07:50.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.421+0000 7f7c41600640 1 -- 172.21.15.43:0/4222118685 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55fb3d587950 con 0x55fb3d6144b0 2024-09-19T05:07:50.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.421+0000 7f7c41600640 1 -- 172.21.15.43:0/4222118685 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7c480054c0 con 0x55fb3d6144b0 2024-09-19T05:07:50.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.422+0000 7f7c613d1740 10 client.25825 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:07:50.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.422+0000 7f7c613d1740 10 client.25825 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:07:50.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.422+0000 7f7c613d1740 20 client.25825 populate_metadata read hostname 'smithi043' 2024-09-19T05:07:50.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.422+0000 7f7c613d1740 1 --2- 172.21.15.43:0/4222118685 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55fb3d69b210 0x55fb3d69d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:50.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.422+0000 7f7c613d1740 4 client.25825 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:07:50.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.422+0000 7f7c613d1740 1 -- 172.21.15.43:0/4222118685 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fb3d581e60 con 0x55fb3d69b210 2024-09-19T05:07:50.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.422+0000 7f7c57400640 1 --2- 172.21.15.43:0/4222118685 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55fb3d69b210 0x55fb3d69d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:50.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.423+0000 7f7c57400640 1 --2- 172.21.15.43:0/4222118685 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55fb3d69b210 0x55fb3d69d630 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f81093a2b05ae6b5 server_cookie=413fc3cafda0c335 in_seq=0 out_seq=0 2024-09-19T05:07:50.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.423+0000 7f7c41600640 10 client.25825 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:07:50.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.423+0000 7f7c41600640 1 -- 172.21.15.43:0/4222118685 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fb3d581e60 con 0x55fb3d69b210 2024-09-19T05:07:50.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.423+0000 7f7c41600640 10 client.25825 handle_command_reply: tid=0 2024-09-19T05:07:50.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.423+0000 7f7c613d1740 1 client.25825 shutdown 2024-09-19T05:07:50.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.423+0000 7f7c613d1740 1 -- 172.21.15.43:0/4222118685 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb3d6144b0 msgr2=0x55fb3d6234a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:50.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.423+0000 7f7c613d1740 1 --2- 172.21.15.43:0/4222118685 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb3d6144b0 0x55fb3d6234a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7c48004c20 tx=0x7f7c48046c90 comp rx=0 tx=0).stop 2024-09-19T05:07:50.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.424+0000 7f7c613d1740 1 -- 172.21.15.43:0/4222118685 shutdown_connections 2024-09-19T05:07:50.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.424+0000 7f7c613d1740 1 --2- 172.21.15.43:0/4222118685 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55fb3d69b210 0x55fb3d69d630 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.424+0000 7f7c613d1740 1 --2- 172.21.15.43:0/4222118685 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb3d62f260 0x55fb3d6239e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.424+0000 7f7c613d1740 1 --2- 172.21.15.43:0/4222118685 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb3d6144b0 0x55fb3d6234a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.424+0000 7f7c613d1740 1 --2- 172.21.15.43:0/4222118685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb3d613b90 0x55fb3d627200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.424+0000 7f7c41600640 0 client.25825 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:07:50.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.424+0000 7f7c613d1740 1 -- 172.21.15.43:0/4222118685 >> 172.21.15.43:0/4222118685 conn(0x55fb3d610120 msgr2=0x55fb3d61e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:50.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.424+0000 7f7c613d1740 1 -- 172.21.15.43:0/4222118685 shutdown_connections 2024-09-19T05:07:50.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.424+0000 7f7c613d1740 1 -- 172.21.15.43:0/4222118685 wait complete. 2024-09-19T05:07:50.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.424+0000 7f7c613d1740 20 client.25825 trim_cache size 0 max 16384 2024-09-19T05:07:50.427 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:07:50.427 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-09-19T05:07:50.427 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-09-19T05:07:50.427 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:07:50.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.425+0000 7f7c5ea00640 1 -- 172.21.15.43:0/3497968991 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f7c3c0a3630 msgr2=0x7f7c3c0a5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:50.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.425+0000 7f7c5ea00640 1 --2- 172.21.15.43:0/3497968991 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f7c3c0a3630 0x7f7c3c0a5af0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7c50098780 tx=0x7f7c50004a40 comp rx=0 tx=0).stop 2024-09-19T05:07:50.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.425+0000 7f7c5ea00640 1 -- 172.21.15.43:0/3497968991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c5810d530 msgr2=0x7f7c5806cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:50.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.425+0000 7f7c5ea00640 1 --2- 172.21.15.43:0/3497968991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c5810d530 0x7f7c5806cbb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000e960 tx=0x7f7c5000ee20 comp rx=0 tx=0).stop 2024-09-19T05:07:50.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.425+0000 7f7c5ea00640 1 -- 172.21.15.43:0/3497968991 shutdown_connections 2024-09-19T05:07:50.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.425+0000 7f7c5ea00640 1 --2- 172.21.15.43:0/3497968991 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f7c3c0a3630 0x7f7c3c0a5af0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.425+0000 7f7c5ea00640 1 --2- 172.21.15.43:0/3497968991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c5810d530 0x7f7c5806cbb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.425+0000 7f7c5ea00640 1 --2- 172.21.15.43:0/3497968991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c58108e00 0x7f7c58073e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.425+0000 7f7c5ea00640 1 --2- 172.21.15.43:0/3497968991 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c58108450 0x7f7c58073940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:50.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.425+0000 7f7c5ea00640 1 -- 172.21.15.43:0/3497968991 >> 172.21.15.43:0/3497968991 conn(0x7f7c58077ba0 msgr2=0x7f7c58078e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:50.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.425+0000 7f7c5ea00640 1 -- 172.21.15.43:0/3497968991 shutdown_connections 2024-09-19T05:07:50.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:50.426+0000 7f7c5ea00640 1 -- 172.21.15.43:0/3497968991 wait complete. 2024-09-19T05:07:50.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:07:50 smithi096 ceph-mon[35085]: pgmap v98: 97 pgs: 97 active+clean; 14 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 255 MiB/s wr, 74 op/s 2024-09-19T05:07:51.844 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T05:07:51.845 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:07:52.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:07:52 smithi151 ceph-mon[35425]: pgmap v99: 97 pgs: 97 active+clean; 16 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 383 MiB/s wr, 112 op/s 2024-09-19T05:07:52.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:07:52 smithi096 ceph-mon[35085]: pgmap v99: 97 pgs: 97 active+clean; 16 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 383 MiB/s wr, 112 op/s 2024-09-19T05:07:52.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:07:52 smithi043 ceph-mon[30800]: pgmap v99: 97 pgs: 97 active+clean; 16 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 383 MiB/s wr, 112 op/s 2024-09-19T05:07:52.556 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:07:53.353 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:07:53 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:07:53.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:07:53 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:07:53.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:07:53 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:07:53.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:53.557+0000 7fb5f1800640 1 -- 172.21.15.43:0/1251231048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5e40956b0 msgr2=0x7fb5e4095b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:53.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:53.557+0000 7fb5f1800640 1 --2- 172.21.15.43:0/1251231048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5e40956b0 0x7fb5e4095b10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8009930 tx=0x7fb5d802a420 comp rx=0 tx=0).stop 2024-09-19T05:07:53.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:53.558+0000 7fb5f1800640 1 -- 172.21.15.43:0/1251231048 shutdown_connections 2024-09-19T05:07:53.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:53.558+0000 7fb5f1800640 1 --2- 172.21.15.43:0/1251231048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5e4096050 0x7fb5e40a2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:53.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:53.558+0000 7fb5f1800640 1 --2- 172.21.15.43:0/1251231048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5e40956b0 0x7fb5e4095b10 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:53.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:53.558+0000 7fb5f1800640 1 --2- 172.21.15.43:0/1251231048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e409b670 0x7fb5e409ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:53.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:53.558+0000 7fb5f1800640 1 -- 172.21.15.43:0/1251231048 >> 172.21.15.43:0/1251231048 conn(0x7fb5e400c970 msgr2=0x7fb5e400cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:53.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:53.558+0000 7fb5f1800640 1 -- 172.21.15.43:0/1251231048 shutdown_connections 2024-09-19T05:07:53.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:53.558+0000 7fb5f1800640 1 -- 172.21.15.43:0/1251231048 wait complete. 2024-09-19T05:07:53.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:53.560+0000 7fb5f1800640 1 Processor -- start 2024-09-19T05:07:53.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:53.561+0000 7fb5f1800640 1 -- start start 2024-09-19T05:07:53.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:53.561+0000 7fb5f1800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5e40956b0 0x7fb5e40a9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:53.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:53.561+0000 7fb5f1800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e4096050 0x7fb5e40aa280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:53.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:53.561+0000 7fb5f1800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5e409b670 0x7fb5e40afa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:53.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:53.562+0000 7fb5ebe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5e40956b0 0x7fb5e40a9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:53.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:53.562+0000 7fb5ebe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5e40956b0 0x7fb5e40a9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:36406/0 (socket says 172.21.15.43:36406) 2024-09-19T05:07:53.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:53.562+0000 7fb5f1800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e40085a0 con 0x7fb5e4096050 2024-09-19T05:07:53.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:53.562+0000 7fb5f1800640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e4008420 con 0x7fb5e409b670 2024-09-19T05:07:53.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:53.562+0000 7fb5f1800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e4008720 con 0x7fb5e40956b0 2024-09-19T05:07:53.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:53.563+0000 7fb5f0e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5e409b670 0x7fb5e40afa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:53.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:53.563+0000 7fb5f0e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5e409b670 0x7fb5e40afa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:37852/0 (socket says 172.21.15.43:37852) 2024-09-19T05:07:53.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:53.563+0000 7fb5f0e00640 1 -- 172.21.15.43:0/781729545 learned_addr learned my addr 172.21.15.43:0/781729545 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:53.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:53.563+0000 7fb5eb400640 1 --2- 172.21.15.43:0/781729545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e4096050 0x7fb5e40aa280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:53.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:53.564+0000 7fb5f0e00640 1 -- 172.21.15.43:0/781729545 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5e40956b0 msgr2=0x7fb5e40a9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:53.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:53.564+0000 7fb5f0e00640 1 --2- 172.21.15.43:0/781729545 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5e40956b0 0x7fb5e40a9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:53.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:53.564+0000 7fb5f0e00640 1 -- 172.21.15.43:0/781729545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e4096050 msgr2=0x7fb5e40aa280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:53.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:53.565+0000 7fb5f0e00640 1 --2- 172.21.15.43:0/781729545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e4096050 0x7fb5e40aa280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:53.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:53.565+0000 7fb5f0e00640 1 -- 172.21.15.43:0/781729545 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e40b0320 con 0x7fb5e409b670 2024-09-19T05:07:53.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:53.565+0000 7fb5eb400640 1 --2- 172.21.15.43:0/781729545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e4096050 0x7fb5e40aa280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:07:53.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:53.565+0000 7fb5f0e00640 1 --2- 172.21.15.43:0/781729545 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5e409b670 0x7fb5e40afa60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec0518b0 tx=0x7fb5ec06b9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:53.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:53.566+0000 7fb5e8c00640 1 -- 172.21.15.43:0/781729545 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec074ca0 con 0x7fb5e409b670 2024-09-19T05:07:53.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:53.566+0000 7fb5e8c00640 1 -- 172.21.15.43:0/781729545 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5ec084020 con 0x7fb5e409b670 2024-09-19T05:07:53.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:53.566+0000 7fb5e8c00640 1 -- 172.21.15.43:0/781729545 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec07e600 con 0x7fb5e409b670 2024-09-19T05:07:53.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:53.567+0000 7fb5f1800640 1 -- 172.21.15.43:0/781729545 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5e40b0610 con 0x7fb5e409b670 2024-09-19T05:07:53.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:53.567+0000 7fb5f1800640 1 -- 172.21.15.43:0/781729545 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5e40a5030 con 0x7fb5e409b670 2024-09-19T05:07:53.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:53.572+0000 7fb5f1800640 1 -- 172.21.15.43:0/781729545 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5e4096d40 con 0x7fb5e409b670 2024-09-19T05:07:53.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:53.578+0000 7fb5e8c00640 1 -- 172.21.15.43:0/781729545 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb5ec07f9d0 con 0x7fb5e409b670 2024-09-19T05:07:53.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:53.581+0000 7fb5e8c00640 1 --2- 172.21.15.43:0/781729545 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb5d40a3500 0x7fb5d40a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:53.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:53.584+0000 7fb5ebe00640 1 --2- 172.21.15.43:0/781729545 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb5d40a3500 0x7fb5d40a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:53.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:53.586+0000 7fb5e8c00640 1 -- 172.21.15.43:0/781729545 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fb5ec13cce0 con 0x7fb5e409b670 2024-09-19T05:07:53.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:53.589+0000 7fb5e8c00640 1 -- 172.21.15.43:0/781729545 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb5ec143080 con 0x7fb5e409b670 2024-09-19T05:07:53.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:53.594+0000 7fb5ebe00640 1 --2- 172.21.15.43:0/781729545 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb5d40a3500 0x7fb5d40a59c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0005d00 tx=0x7fb5e000b410 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:54.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:54.020+0000 7fb5f1800640 1 -- 172.21.15.43:0/781729545 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5e40a5ae0 con 0x7fb5e409b670 2024-09-19T05:07:54.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:54.021+0000 7fb5e8c00640 1 -- 172.21.15.43:0/781729545 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11965 (secure 0 0 0) 0x7fb5ec08e020 con 0x7fb5e409b670 2024-09-19T05:07:54.024 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:07:54.024 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 29 2024-09-19T05:07:54.025 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":29,"btime":"2024-09-19T05:04:38:785407+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/773801169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":773801169},{"type":"v1","addr":"172.21.15.43:6835","nonce":773801169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1799137902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1799137902},{"type":"v1","addr":"172.21.15.43:6837","nonce":1799137902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:02:25.463041+0000","modified":"2024-09-19T05:04:38.039809+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24518,"mds_2":24511,"mds_3":24506,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14541":{"gid":14541,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/794881135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":794881135},{"type":"v1","addr":"172.21.15.43:6841","nonce":794881135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/2836453158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2836453158},{"type":"v1","addr":"172.21.15.151:6833","nonce":2836453158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/178785812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":178785812},{"type":"v1","addr":"172.21.15.96:6833","nonce":178785812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1671043820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1671043820},{"type":"v1","addr":"172.21.15.96:6835","nonce":1671043820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2972514328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2972514328},{"type":"v1","addr":"172.21.15.151:6835","nonce":2972514328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.151:6837/3960378010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3960378010},{"type":"v1","addr":"172.21.15.151:6837","nonce":3960378010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6837/3978388286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3978388286},{"type":"v1","addr":"172.21.15.96:6837","nonce":3978388286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.151:6839/3794104313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3794104313},{"type":"v1","addr":"172.21.15.151:6839","nonce":3794104313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.43:6839/3404555030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3404555030},{"type":"v1","addr":"172.21.15.43:6839","nonce":3404555030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/605527621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":605527621},{"type":"v1","addr":"172.21.15.96:6839","nonce":605527621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24518,24511,24506,24493]},"id":1}]} 2024-09-19T05:07:54.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:54.024+0000 7fb5f1800640 1 -- 172.21.15.43:0/781729545 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb5d40a3500 msgr2=0x7fb5d40a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:54.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:54.024+0000 7fb5f1800640 1 --2- 172.21.15.43:0/781729545 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb5d40a3500 0x7fb5d40a59c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0005d00 tx=0x7fb5e000b410 comp rx=0 tx=0).stop 2024-09-19T05:07:54.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:54.024+0000 7fb5f1800640 1 -- 172.21.15.43:0/781729545 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5e409b670 msgr2=0x7fb5e40afa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:54.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:54.024+0000 7fb5f1800640 1 --2- 172.21.15.43:0/781729545 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5e409b670 0x7fb5e40afa60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec0518b0 tx=0x7fb5ec06b9f0 comp rx=0 tx=0).stop 2024-09-19T05:07:54.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:54.025+0000 7fb5f1800640 1 -- 172.21.15.43:0/781729545 shutdown_connections 2024-09-19T05:07:54.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:54.025+0000 7fb5f1800640 1 --2- 172.21.15.43:0/781729545 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb5d40a3500 0x7fb5d40a59c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:54.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:54.025+0000 7fb5f1800640 1 --2- 172.21.15.43:0/781729545 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5e409b670 0x7fb5e40afa60 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:54.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:54.025+0000 7fb5f1800640 1 --2- 172.21.15.43:0/781729545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e4096050 0x7fb5e40aa280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:54.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:54.025+0000 7fb5f1800640 1 --2- 172.21.15.43:0/781729545 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5e40956b0 0x7fb5e40a9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:54.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:54.025+0000 7fb5f1800640 1 -- 172.21.15.43:0/781729545 >> 172.21.15.43:0/781729545 conn(0x7fb5e400c970 msgr2=0x7fb5e4094310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:54.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:54.026+0000 7fb5f1800640 1 -- 172.21.15.43:0/781729545 shutdown_connections 2024-09-19T05:07:54.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:54.026+0000 7fb5f1800640 1 -- 172.21.15.43:0/781729545 wait complete. 2024-09-19T05:07:54.285 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:07:54 smithi043 ceph-mon[30800]: pgmap v100: 97 pgs: 97 active+clean; 16 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 257 MiB/s wr, 75 op/s 2024-09-19T05:07:54.285 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:07:54 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/781729545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:07:54.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:07:54 smithi151 ceph-mon[35425]: pgmap v100: 97 pgs: 97 active+clean; 16 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 257 MiB/s wr, 75 op/s 2024-09-19T05:07:54.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:07:54 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/781729545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:07:54.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:07:54 smithi096 ceph-mon[35085]: pgmap v100: 97 pgs: 97 active+clean; 16 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 257 MiB/s wr, 75 op/s 2024-09-19T05:07:54.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:07:54 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/781729545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:07:55.059 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T05:07:55.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:07:55 smithi043 ceph-mon[30800]: pgmap v101: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 381 MiB/s wr, 110 op/s 2024-09-19T05:07:55.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:07:55 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:07:55.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:07:55 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:07:55.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:07:55 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:07:55.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:07:55 smithi096 ceph-mon[35085]: pgmap v101: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 381 MiB/s wr, 110 op/s 2024-09-19T05:07:55.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:07:55 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:07:55.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:07:55 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:07:55.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:07:55 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:07:55.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:07:55 smithi151 ceph-mon[35425]: pgmap v101: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 381 MiB/s wr, 110 op/s 2024-09-19T05:07:55.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:07:55 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:07:55.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:07:55 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:07:55.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:07:55 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:07:55.735 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:07:56.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.493+0000 7f15e6c00640 1 -- 172.21.15.43:0/2019430881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15e006e4f0 msgr2=0x7f15e006e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:56.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.493+0000 7f15e6c00640 1 --2- 172.21.15.43:0/2019430881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15e006e4f0 0x7f15e006e8d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f15d4009930 tx=0x7f15d402a460 comp rx=0 tx=0).stop 2024-09-19T05:07:56.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.497+0000 7f15e6c00640 1 -- 172.21.15.43:0/2019430881 shutdown_connections 2024-09-19T05:07:56.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.497+0000 7f15e6c00640 1 --2- 172.21.15.43:0/2019430881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15e0114ab0 0x7f15e0116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:56.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.497+0000 7f15e6c00640 1 --2- 172.21.15.43:0/2019430881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15e006ee10 0x7f15e0114570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:56.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.497+0000 7f15e6c00640 1 --2- 172.21.15.43:0/2019430881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15e006e4f0 0x7f15e006e8d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:56.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.497+0000 7f15e6c00640 1 -- 172.21.15.43:0/2019430881 >> 172.21.15.43:0/2019430881 conn(0x7f15e0073180 msgr2=0x7f15e0073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:56.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.499+0000 7f15e6c00640 1 -- 172.21.15.43:0/2019430881 shutdown_connections 2024-09-19T05:07:56.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.500+0000 7f15e6c00640 1 -- 172.21.15.43:0/2019430881 wait complete. 2024-09-19T05:07:56.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.502+0000 7f15e6c00640 1 Processor -- start 2024-09-19T05:07:56.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.504+0000 7f15e6c00640 1 -- start start 2024-09-19T05:07:56.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.504+0000 7f15e6c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15e006e4f0 0x7f15e0114010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:56.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.504+0000 7f15e6c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15e006ee10 0x7f15e0114550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:56.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.504+0000 7f15e6c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15e0114ab0 0x7f15e010b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:56.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.504+0000 7f15e6c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f15e011a590 con 0x7f15e006e4f0 2024-09-19T05:07:56.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.504+0000 7f15e6c00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f15e011a410 con 0x7f15e006ee10 2024-09-19T05:07:56.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.504+0000 7f15e6c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f15e011a710 con 0x7f15e0114ab0 2024-09-19T05:07:56.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.505+0000 7f15e6200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15e0114ab0 0x7f15e010b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:56.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.506+0000 7f15e5800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15e006e4f0 0x7f15e0114010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:56.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.506+0000 7f15e5800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15e006e4f0 0x7f15e0114010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45666/0 (socket says 172.21.15.43:45666) 2024-09-19T05:07:56.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.506+0000 7f15e5800640 1 -- 172.21.15.43:0/1954489932 learned_addr learned my addr 172.21.15.43:0/1954489932 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:56.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.506+0000 7f15e4e00640 1 --2- 172.21.15.43:0/1954489932 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15e006ee10 0x7f15e0114550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:56.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.506+0000 7f15e5800640 1 -- 172.21.15.43:0/1954489932 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15e0114ab0 msgr2=0x7f15e010b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:56.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.507+0000 7f15e5800640 1 --2- 172.21.15.43:0/1954489932 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15e0114ab0 0x7f15e010b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:56.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.507+0000 7f15e5800640 1 -- 172.21.15.43:0/1954489932 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15e006ee10 msgr2=0x7f15e0114550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:56.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.507+0000 7f15e5800640 1 --2- 172.21.15.43:0/1954489932 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15e006ee10 0x7f15e0114550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:56.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.507+0000 7f15e5800640 1 -- 172.21.15.43:0/1954489932 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e010baa0 con 0x7f15e006e4f0 2024-09-19T05:07:56.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.507+0000 7f15e4e00640 1 --2- 172.21.15.43:0/1954489932 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15e006ee10 0x7f15e0114550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:07:56.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.507+0000 7f15e6200640 1 --2- 172.21.15.43:0/1954489932 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15e0114ab0 0x7f15e010b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:07:56.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.507+0000 7f15e5800640 1 --2- 172.21.15.43:0/1954489932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15e006e4f0 0x7f15e0114010 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f15d4009900 tx=0x7f15d4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:56.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.508+0000 7f15de000640 1 -- 172.21.15.43:0/1954489932 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d403f070 con 0x7f15e006e4f0 2024-09-19T05:07:56.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.509+0000 7f15de000640 1 -- 172.21.15.43:0/1954489932 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15d4031ca0 con 0x7f15e006e4f0 2024-09-19T05:07:56.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.509+0000 7f15de000640 1 -- 172.21.15.43:0/1954489932 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d4035030 con 0x7f15e006e4f0 2024-09-19T05:07:56.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.509+0000 7f15e6c00640 1 -- 172.21.15.43:0/1954489932 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15e010bd30 con 0x7f15e006e4f0 2024-09-19T05:07:56.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.510+0000 7f15e6c00640 1 -- 172.21.15.43:0/1954489932 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15e010fc30 con 0x7f15e006e4f0 2024-09-19T05:07:56.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.516+0000 7f15de000640 1 -- 172.21.15.43:0/1954489932 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f15d4042070 con 0x7f15e006e4f0 2024-09-19T05:07:56.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.517+0000 7f15eac74740 1 Processor -- start 2024-09-19T05:07:56.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.517+0000 7f15eac74740 1 -- start start 2024-09-19T05:07:56.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.517+0000 7f15eac74740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddb0e2ae40 0x55ddb0e2b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:56.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.518+0000 7f15eac74740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddb0e2b760 0x55ddb0e35ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:56.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.518+0000 7f15eac74740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddb0e36020 0x55ddb0e38410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:56.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.518+0000 7f15eac74740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ddb0d913e0 con 0x55ddb0e2b760 2024-09-19T05:07:56.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.518+0000 7f15eac74740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ddb0d38d80 con 0x55ddb0e2ae40 2024-09-19T05:07:56.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.518+0000 7f15eac74740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ddb0d8d3a0 con 0x55ddb0e36020 2024-09-19T05:07:56.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.518+0000 7f15de000640 1 --2- 172.21.15.43:0/1954489932 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f15c80a3580 0x7f15c80a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:56.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.518+0000 7f15de000640 1 -- 172.21.15.43:0/1954489932 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f15d40f0360 con 0x7f15e006e4f0 2024-09-19T05:07:56.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.519+0000 7f15e6200640 1 --2- 172.21.15.43:0/1954489932 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f15c80a3580 0x7f15c80a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:56.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.519+0000 7f15e4e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddb0e36020 0x55ddb0e38410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:56.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.519+0000 7f15e4e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddb0e36020 0x55ddb0e38410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:58804/0 (socket says 172.21.15.43:58804) 2024-09-19T05:07:56.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.519+0000 7f15e4e00640 1 -- 172.21.15.43:0/1724887066 learned_addr learned my addr 172.21.15.43:0/1724887066 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:56.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.520+0000 7f15e4e00640 1 -- 172.21.15.43:0/1724887066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddb0e2ae40 msgr2=0x55ddb0e2b220 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:07:56.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.520+0000 7f15e4e00640 1 --2- 172.21.15.43:0/1724887066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddb0e2ae40 0x55ddb0e2b220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:56.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.520+0000 7f15e4e00640 1 -- 172.21.15.43:0/1724887066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddb0e2b760 msgr2=0x55ddb0e35ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:56.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.520+0000 7f15e4e00640 1 --2- 172.21.15.43:0/1724887066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddb0e2b760 0x55ddb0e35ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:56.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.520+0000 7f15e4e00640 1 -- 172.21.15.43:0/1724887066 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddb0d57ae0 con 0x55ddb0e36020 2024-09-19T05:07:56.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.522+0000 7f15e4e00640 1 --2- 172.21.15.43:0/1724887066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddb0e36020 0x55ddb0e38410 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f15cc00bce0 tx=0x7f15cc00f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0fd5882ad8a1624 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:56.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.522+0000 7f15e6200640 1 --2- 172.21.15.43:0/1954489932 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f15c80a3580 0x7f15c80a5a40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f15d8009810 tx=0x7f15d8009500 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:56.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.523+0000 7f15d2a00640 1 -- 172.21.15.43:0/1724887066 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc015070 con 0x55ddb0e36020 2024-09-19T05:07:56.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.523+0000 7f15d2a00640 1 -- 172.21.15.43:0/1724887066 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15cc01a940 con 0x55ddb0e36020 2024-09-19T05:07:56.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.523+0000 7f15d2a00640 1 -- 172.21.15.43:0/1724887066 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc011aa0 con 0x55ddb0e36020 2024-09-19T05:07:56.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.525+0000 7f15eac74740 1 -- 172.21.15.43:0/1724887066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddb0e36020 msgr2=0x55ddb0e38410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:56.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.526+0000 7f15eac74740 1 --2- 172.21.15.43:0/1724887066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddb0e36020 0x55ddb0e38410 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f15cc00bce0 tx=0x7f15cc00f730 comp rx=0 tx=0).stop 2024-09-19T05:07:56.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.526+0000 7f15eac74740 1 -- 172.21.15.43:0/1724887066 shutdown_connections 2024-09-19T05:07:56.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.526+0000 7f15eac74740 1 --2- 172.21.15.43:0/1724887066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddb0e36020 0x55ddb0e38410 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:56.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.526+0000 7f15eac74740 1 --2- 172.21.15.43:0/1724887066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddb0e2b760 0x55ddb0e35ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:56.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.526+0000 7f15eac74740 1 --2- 172.21.15.43:0/1724887066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddb0e2ae40 0x55ddb0e2b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:56.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.526+0000 7f15eac74740 1 -- 172.21.15.43:0/1724887066 >> 172.21.15.43:0/1724887066 conn(0x55ddb0e273d0 msgr2=0x55ddb0e277b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:56.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.528+0000 7f15eac74740 1 -- 172.21.15.43:0/1724887066 shutdown_connections 2024-09-19T05:07:56.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.528+0000 7f15eac74740 1 -- 172.21.15.43:0/1724887066 wait complete. 2024-09-19T05:07:56.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.530+0000 7f15eac74740 1 -- start start 2024-09-19T05:07:56.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.531+0000 7f15eac74740 1 Processor -- start 2024-09-19T05:07:56.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.533+0000 7f15eac74740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddb0e2ae40 0x55ddb0e33030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:56.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.533+0000 7f15eac74740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddb0e2b760 0x55ddb0e33570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:56.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.533+0000 7f15eac74740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddb0e36020 0x55ddb0e33ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:56.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.533+0000 7f15eac74740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ddb0e38c40 con 0x55ddb0e36020 2024-09-19T05:07:56.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.533+0000 7f15eac74740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ddb0d4c0f0 con 0x55ddb0e2b760 2024-09-19T05:07:56.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.533+0000 7f15eac74740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ddb0e38dc0 con 0x55ddb0e2ae40 2024-09-19T05:07:56.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.533+0000 7f15eac74740 10 client.0 _ll_register_callbacks cb 0x55ddb0d33bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:56.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.535+0000 7f15e6200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddb0e36020 0x55ddb0e33ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:56.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.535+0000 7f15e5800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddb0e2ae40 0x55ddb0e33030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:56.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.535+0000 7f15e5800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddb0e2ae40 0x55ddb0e33030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:58820/0 (socket says 172.21.15.43:58820) 2024-09-19T05:07:56.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.535+0000 7f15e5800640 1 -- 172.21.15.43:0/3311550099 learned_addr learned my addr 172.21.15.43:0/3311550099 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:56.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.536+0000 7f15e4e00640 1 --2- 172.21.15.43:0/3311550099 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddb0e2b760 0x55ddb0e33570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:56.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.536+0000 7f15e6200640 1 -- 172.21.15.43:0/3311550099 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddb0e2ae40 msgr2=0x55ddb0e33030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:56.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.536+0000 7f15e6200640 1 --2- 172.21.15.43:0/3311550099 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddb0e2ae40 0x55ddb0e33030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:56.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.536+0000 7f15e6200640 1 -- 172.21.15.43:0/3311550099 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddb0e2b760 msgr2=0x55ddb0e33570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:56.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.536+0000 7f15e6200640 1 --2- 172.21.15.43:0/3311550099 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddb0e2b760 0x55ddb0e33570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:56.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.536+0000 7f15e6200640 1 -- 172.21.15.43:0/3311550099 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddb0b023c0 con 0x55ddb0e36020 2024-09-19T05:07:56.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.538+0000 7f15e6200640 1 --2- 172.21.15.43:0/3311550099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddb0e36020 0x55ddb0e33ab0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f15d80116c0 tx=0x7f15d8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=387618b140c6b506 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:56.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.538+0000 7f15c7400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:07:56.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.538+0000 7f15c7400640 1 -- 172.21.15.43:0/3311550099 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d8022070 con 0x55ddb0e36020 2024-09-19T05:07:56.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.538+0000 7f15eac74740 1 -- 172.21.15.43:0/3311550099 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15cc01a940 con 0x55ddb0e36020 2024-09-19T05:07:56.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.538+0000 7f15c7400640 1 -- 172.21.15.43:0/3311550099 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f15d80109a0 con 0x55ddb0e36020 2024-09-19T05:07:56.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.543+0000 7f15c7400640 1 -- 172.21.15.43:0/3311550099 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f15d802e050 con 0x55ddb0e36020 2024-09-19T05:07:56.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.543+0000 7f15eac74740 10 client.15861 fetch_fsmap learned FSMap version 29 2024-09-19T05:07:56.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.543+0000 7f15eac74740 1 -- 172.21.15.43:0/3311550099 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ddb0e44e50 con 0x55ddb0e36020 2024-09-19T05:07:56.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.543+0000 7f15c7400640 1 -- 172.21.15.43:0/3311550099 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15cc01a940 con 0x55ddb0e36020 2024-09-19T05:07:56.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.544+0000 7f15c7400640 1 -- 172.21.15.43:0/3311550099 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f15d80263b0 con 0x55ddb0e36020 2024-09-19T05:07:56.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.545+0000 7f15eac74740 10 client.15861 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:07:56.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.545+0000 7f15eac74740 10 client.15861 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-19T05:07:56.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.545+0000 7f15eac74740 20 client.15861 populate_metadata read hostname 'smithi043' 2024-09-19T05:07:56.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.545+0000 7f15eac74740 1 --2- 172.21.15.43:0/3311550099 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55ddb0e46260 0x55ddb0e48650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:56.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.545+0000 7f15eac74740 4 client.15861 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:07:56.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.545+0000 7f15eac74740 1 -- 172.21.15.43:0/3311550099 --> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddb0d9e950 con 0x55ddb0e46260 2024-09-19T05:07:56.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.546+0000 7f15e5800640 1 --2- 172.21.15.43:0/3311550099 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55ddb0e46260 0x55ddb0e48650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:56.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.548+0000 7f15e5800640 1 --2- 172.21.15.43:0/3311550099 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55ddb0e46260 0x55ddb0e48650 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8d3c0e670ebd3b30 server_cookie=78dc36a59fa08234 in_seq=0 out_seq=0 2024-09-19T05:07:56.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.548+0000 7f15c7400640 10 client.15861 ms_handle_connect on v2:172.21.15.151:6836/3960378010 2024-09-19T05:07:56.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.549+0000 7f15c7400640 1 -- 172.21.15.43:0/3311550099 <== mds.4 v2:172.21.15.151:6836/3960378010 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ddb0d9e950 con 0x55ddb0e46260 2024-09-19T05:07:56.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.549+0000 7f15c7400640 10 client.15861 handle_command_reply: tid=0 2024-09-19T05:07:56.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.550+0000 7f15eac74740 1 client.15861 shutdown 2024-09-19T05:07:56.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.551+0000 7f15eac74740 1 -- 172.21.15.43:0/3311550099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddb0e36020 msgr2=0x55ddb0e33ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:56.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.551+0000 7f15eac74740 1 --2- 172.21.15.43:0/3311550099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddb0e36020 0x55ddb0e33ab0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f15d80116c0 tx=0x7f15d8004ab0 comp rx=0 tx=0).stop 2024-09-19T05:07:56.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.552+0000 7f15eac74740 1 -- 172.21.15.43:0/3311550099 shutdown_connections 2024-09-19T05:07:56.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.552+0000 7f15eac74740 1 --2- 172.21.15.43:0/3311550099 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55ddb0e46260 0x55ddb0e48650 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:56.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.552+0000 7f15eac74740 1 --2- 172.21.15.43:0/3311550099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddb0e36020 0x55ddb0e33ab0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:56.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.552+0000 7f15c7400640 0 client.15861 ms_handle_reset on v2:172.21.15.151:6836/3960378010 2024-09-19T05:07:56.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.552+0000 7f15eac74740 1 --2- 172.21.15.43:0/3311550099 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddb0e2b760 0x55ddb0e33570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:56.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.552+0000 7f15eac74740 1 --2- 172.21.15.43:0/3311550099 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddb0e2ae40 0x55ddb0e33030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:56.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.552+0000 7f15eac74740 1 -- 172.21.15.43:0/3311550099 >> 172.21.15.43:0/3311550099 conn(0x55ddb0e273d0 msgr2=0x55ddb0e2d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:56.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.553+0000 7f15eac74740 1 -- 172.21.15.43:0/3311550099 shutdown_connections 2024-09-19T05:07:56.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.554+0000 7f15eac74740 1 -- 172.21.15.43:0/3311550099 wait complete. 2024-09-19T05:07:56.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.554+0000 7f15eac74740 20 client.15861 trim_cache size 0 max 16384 2024-09-19T05:07:56.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.584+0000 7f15eac74740 1 Processor -- start 2024-09-19T05:07:56.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.584+0000 7f15eac74740 1 -- start start 2024-09-19T05:07:56.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.585+0000 7f15eac74740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddb0e2ae40 0x55ddb0e4ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:56.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.585+0000 7f15eac74740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddb0e2b760 0x55ddb0e4e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:56.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.585+0000 7f15eac74740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddb0e36020 0x55ddb0e30290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:56.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.585+0000 7f15eac74740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f15e01c4d60 con 0x55ddb0e2b760 2024-09-19T05:07:56.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.585+0000 7f15eac74740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f15e01c68a0 con 0x55ddb0e2ae40 2024-09-19T05:07:56.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.585+0000 7f15eac74740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f15e01c5430 con 0x55ddb0e36020 2024-09-19T05:07:56.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.585+0000 7f15e5800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddb0e2ae40 0x55ddb0e4ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:56.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.585+0000 7f15e4e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddb0e2b760 0x55ddb0e4e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:56.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.585+0000 7f15e4e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddb0e2b760 0x55ddb0e4e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45690/0 (socket says 172.21.15.43:45690) 2024-09-19T05:07:56.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.585+0000 7f15e4e00640 1 -- 172.21.15.43:0/2546506186 learned_addr learned my addr 172.21.15.43:0/2546506186 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:56.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.586+0000 7f15e4e00640 1 -- 172.21.15.43:0/2546506186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddb0e36020 msgr2=0x55ddb0e30290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:56.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.586+0000 7f15e4e00640 1 --2- 172.21.15.43:0/2546506186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddb0e36020 0x55ddb0e30290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:56.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.586+0000 7f15e4e00640 1 -- 172.21.15.43:0/2546506186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddb0e2ae40 msgr2=0x55ddb0e4ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:56.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.586+0000 7f15e4e00640 1 --2- 172.21.15.43:0/2546506186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddb0e2ae40 0x55ddb0e4ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:56.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.586+0000 7f15e4e00640 1 -- 172.21.15.43:0/2546506186 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddb0e26c80 con 0x55ddb0e2b760 2024-09-19T05:07:56.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.586+0000 7f15e5800640 1 --2- 172.21.15.43:0/2546506186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddb0e2ae40 0x55ddb0e4ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:07:56.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.587+0000 7f15e4e00640 1 --2- 172.21.15.43:0/2546506186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddb0e2b760 0x55ddb0e4e410 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f15cc00bc30 tx=0x7f15cc011550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7c472080525c797 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:56.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.587+0000 7f15d2000640 1 -- 172.21.15.43:0/2546506186 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc015040 con 0x55ddb0e2b760 2024-09-19T05:07:56.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.587+0000 7f15d2000640 1 -- 172.21.15.43:0/2546506186 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15cc00be50 con 0x55ddb0e2b760 2024-09-19T05:07:56.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.587+0000 7f15d2000640 1 -- 172.21.15.43:0/2546506186 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc011a50 con 0x55ddb0e2b760 2024-09-19T05:07:56.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.588+0000 7f15eac74740 1 -- 172.21.15.43:0/2546506186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddb0e2b760 msgr2=0x55ddb0e4e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:56.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.588+0000 7f15eac74740 1 --2- 172.21.15.43:0/2546506186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddb0e2b760 0x55ddb0e4e410 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f15cc00bc30 tx=0x7f15cc011550 comp rx=0 tx=0).stop 2024-09-19T05:07:56.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.589+0000 7f15eac74740 1 -- 172.21.15.43:0/2546506186 shutdown_connections 2024-09-19T05:07:56.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.589+0000 7f15eac74740 1 --2- 172.21.15.43:0/2546506186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddb0e36020 0x55ddb0e30290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:56.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.589+0000 7f15eac74740 1 --2- 172.21.15.43:0/2546506186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddb0e2b760 0x55ddb0e4e410 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:56.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.589+0000 7f15eac74740 1 --2- 172.21.15.43:0/2546506186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddb0e2ae40 0x55ddb0e4ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:56.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.589+0000 7f15eac74740 1 -- 172.21.15.43:0/2546506186 >> 172.21.15.43:0/2546506186 conn(0x55ddb0e273d0 msgr2=0x55ddb0e350b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:56.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.589+0000 7f15eac74740 1 -- 172.21.15.43:0/2546506186 shutdown_connections 2024-09-19T05:07:56.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.589+0000 7f15eac74740 1 -- 172.21.15.43:0/2546506186 wait complete. 2024-09-19T05:07:56.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.590+0000 7f15eac74740 1 -- start start 2024-09-19T05:07:56.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.590+0000 7f15eac74740 1 Processor -- start 2024-09-19T05:07:56.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.591+0000 7f15eac74740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddb0e2ae40 0x55ddb0e3e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:56.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.591+0000 7f15eac74740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddb0e2b760 0x55ddb0e3e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:56.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.591+0000 7f15eac74740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddb0e36020 0x55ddb0e3a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:56.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.591+0000 7f15eac74740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f15e01c61d0 con 0x55ddb0e36020 2024-09-19T05:07:56.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.591+0000 7f15eac74740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f15e01c5b00 con 0x55ddb0e2b760 2024-09-19T05:07:56.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.591+0000 7f15eac74740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f15e01c2d40 con 0x55ddb0e2ae40 2024-09-19T05:07:56.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.591+0000 7f15eac74740 10 client.0 _ll_register_callbacks cb 0x7f15d8015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:56.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.592+0000 7f15e6200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddb0e36020 0x55ddb0e3a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:56.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.592+0000 7f15e4e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddb0e2b760 0x55ddb0e3e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:56.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.592+0000 7f15e6200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddb0e36020 0x55ddb0e3a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45700/0 (socket says 172.21.15.43:45700) 2024-09-19T05:07:56.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.592+0000 7f15e6200640 1 -- 172.21.15.43:0/364895783 learned_addr learned my addr 172.21.15.43:0/364895783 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:56.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.592+0000 7f15e5800640 1 --2- 172.21.15.43:0/364895783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddb0e2ae40 0x55ddb0e3e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:56.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.593+0000 7f15e6200640 1 -- 172.21.15.43:0/364895783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddb0e2ae40 msgr2=0x55ddb0e3e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:56.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.593+0000 7f15e6200640 1 --2- 172.21.15.43:0/364895783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddb0e2ae40 0x55ddb0e3e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:56.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.593+0000 7f15e6200640 1 -- 172.21.15.43:0/364895783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddb0e2b760 msgr2=0x55ddb0e3e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:56.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.593+0000 7f15e6200640 1 --2- 172.21.15.43:0/364895783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddb0e2b760 0x55ddb0e3e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:56.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.593+0000 7f15e6200640 1 -- 172.21.15.43:0/364895783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddb0e3ad90 con 0x55ddb0e36020 2024-09-19T05:07:56.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.593+0000 7f15e5800640 1 --2- 172.21.15.43:0/364895783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddb0e2ae40 0x55ddb0e3e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:07:56.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.594+0000 7f15e6200640 1 --2- 172.21.15.43:0/364895783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddb0e36020 0x55ddb0e3a560 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f15d800fd70 tx=0x7f15d8009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=610ba7e08448142f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:56.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.594+0000 7f15c7400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:07:56.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.594+0000 7f15c7400640 1 -- 172.21.15.43:0/364895783 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d8022030 con 0x55ddb0e36020 2024-09-19T05:07:56.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.594+0000 7f15c7400640 1 -- 172.21.15.43:0/364895783 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f15d80049e0 con 0x55ddb0e36020 2024-09-19T05:07:56.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.595+0000 7f15eac74740 1 -- 172.21.15.43:0/364895783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15cc00be50 con 0x55ddb0e36020 2024-09-19T05:07:56.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.595+0000 7f15c7400640 1 -- 172.21.15.43:0/364895783 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f15d8026bf0 con 0x55ddb0e36020 2024-09-19T05:07:56.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.595+0000 7f15eac74740 10 client.15867 fetch_fsmap learned FSMap version 29 2024-09-19T05:07:56.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.595+0000 7f15eac74740 1 -- 172.21.15.43:0/364895783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ddb0eb1990 con 0x55ddb0e36020 2024-09-19T05:07:56.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.596+0000 7f15c7400640 1 -- 172.21.15.43:0/364895783 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15cc00be50 con 0x55ddb0e36020 2024-09-19T05:07:56.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.596+0000 7f15c7400640 1 -- 172.21.15.43:0/364895783 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f15d803a030 con 0x55ddb0e36020 2024-09-19T05:07:56.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.596+0000 7f15eac74740 10 client.15867 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:07:56.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.596+0000 7f15eac74740 10 client.15867 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-19T05:07:56.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.596+0000 7f15eac74740 20 client.15867 populate_metadata read hostname 'smithi043' 2024-09-19T05:07:56.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.596+0000 7f15eac74740 1 --2- 172.21.15.43:0/364895783 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55ddb0e46260 0x55ddb0eb4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:56.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.596+0000 7f15eac74740 4 client.15867 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:07:56.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.596+0000 7f15eac74740 1 -- 172.21.15.43:0/364895783 --> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ddb0d98e60 con 0x55ddb0e46260 2024-09-19T05:07:56.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.598+0000 7f15e5800640 1 --2- 172.21.15.43:0/364895783 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55ddb0e46260 0x55ddb0eb4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:56.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.598+0000 7f15e5800640 1 --2- 172.21.15.43:0/364895783 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55ddb0e46260 0x55ddb0eb4120 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a3d9e52fdd48926 server_cookie=c38aa3a6f7d51c57 in_seq=0 out_seq=0 2024-09-19T05:07:56.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.598+0000 7f15c7400640 10 client.15867 ms_handle_connect on v2:172.21.15.151:6836/3960378010 2024-09-19T05:07:56.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.599+0000 7f15c7400640 1 -- 172.21.15.43:0/364895783 <== mds.4 v2:172.21.15.151:6836/3960378010 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ddb0d98e60 con 0x55ddb0e46260 2024-09-19T05:07:56.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.599+0000 7f15c7400640 10 client.15867 handle_command_reply: tid=0 2024-09-19T05:07:56.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.599+0000 7f15eac74740 1 client.15867 shutdown 2024-09-19T05:07:56.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.599+0000 7f15eac74740 1 -- 172.21.15.43:0/364895783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddb0e36020 msgr2=0x55ddb0e3a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:56.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.599+0000 7f15eac74740 1 --2- 172.21.15.43:0/364895783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddb0e36020 0x55ddb0e3a560 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f15d800fd70 tx=0x7f15d8009f10 comp rx=0 tx=0).stop 2024-09-19T05:07:56.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.600+0000 7f15eac74740 1 -- 172.21.15.43:0/364895783 shutdown_connections 2024-09-19T05:07:56.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.600+0000 7f15eac74740 1 --2- 172.21.15.43:0/364895783 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55ddb0e46260 0x55ddb0eb4120 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:56.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.600+0000 7f15eac74740 1 --2- 172.21.15.43:0/364895783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ddb0e36020 0x55ddb0e3a560 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:56.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.600+0000 7f15c7400640 0 client.15867 ms_handle_reset on v2:172.21.15.151:6836/3960378010 2024-09-19T05:07:56.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.600+0000 7f15eac74740 1 --2- 172.21.15.43:0/364895783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ddb0e2b760 0x55ddb0e3e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:56.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.600+0000 7f15eac74740 1 --2- 172.21.15.43:0/364895783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddb0e2ae40 0x55ddb0e3e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:56.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.600+0000 7f15eac74740 1 -- 172.21.15.43:0/364895783 >> 172.21.15.43:0/364895783 conn(0x55ddb0e273d0 msgr2=0x55ddb0e25c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:56.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.601+0000 7f15eac74740 1 -- 172.21.15.43:0/364895783 shutdown_connections 2024-09-19T05:07:56.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.601+0000 7f15eac74740 1 -- 172.21.15.43:0/364895783 wait complete. 2024-09-19T05:07:56.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.602+0000 7f15eac74740 20 client.15867 trim_cache size 0 max 16384 2024-09-19T05:07:56.605 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:07:56.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.606+0000 7f15e6c00640 1 -- 172.21.15.43:0/1954489932 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f15c80a3580 msgr2=0x7f15c80a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:56.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.606+0000 7f15e6c00640 1 --2- 172.21.15.43:0/1954489932 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f15c80a3580 0x7f15c80a5a40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f15d8009810 tx=0x7f15d8009500 comp rx=0 tx=0).stop 2024-09-19T05:07:56.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.606+0000 7f15e6c00640 1 -- 172.21.15.43:0/1954489932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15e006e4f0 msgr2=0x7f15e0114010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:56.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.606+0000 7f15e6c00640 1 --2- 172.21.15.43:0/1954489932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15e006e4f0 0x7f15e0114010 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f15d4009900 tx=0x7f15d4003c10 comp rx=0 tx=0).stop 2024-09-19T05:07:56.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.606+0000 7f15e6c00640 1 -- 172.21.15.43:0/1954489932 shutdown_connections 2024-09-19T05:07:56.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.606+0000 7f15e6c00640 1 --2- 172.21.15.43:0/1954489932 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15e0114ab0 0x7f15e010b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:56.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.606+0000 7f15e6c00640 1 --2- 172.21.15.43:0/1954489932 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f15c80a3580 0x7f15c80a5a40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:56.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.606+0000 7f15e6c00640 1 --2- 172.21.15.43:0/1954489932 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15e006ee10 0x7f15e0114550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:56.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.606+0000 7f15e6c00640 1 --2- 172.21.15.43:0/1954489932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15e006e4f0 0x7f15e0114010 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:56.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.606+0000 7f15e6c00640 1 -- 172.21.15.43:0/1954489932 >> 172.21.15.43:0/1954489932 conn(0x7f15e0073180 msgr2=0x7f15e006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:56.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.607+0000 7f15e6c00640 1 -- 172.21.15.43:0/1954489932 shutdown_connections 2024-09-19T05:07:56.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:56.607+0000 7f15e6c00640 1 -- 172.21.15.43:0/1954489932 wait complete. 2024-09-19T05:07:57.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:07:57 smithi043 ceph-mon[30800]: pgmap v102: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 252 MiB/s wr, 73 op/s 2024-09-19T05:07:57.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:07:57 smithi151 ceph-mon[35425]: pgmap v102: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 252 MiB/s wr, 73 op/s 2024-09-19T05:07:57.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:07:57 smithi096 ceph-mon[35085]: pgmap v102: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 252 MiB/s wr, 73 op/s 2024-09-19T05:07:57.756 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T05:07:58.469 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:07:59.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.329+0000 7fa1c2800640 1 -- 172.21.15.43:0/1402975420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1bc10ef30 msgr2=0x7fa1bc10d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:59.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.329+0000 7fa1c2800640 1 --2- 172.21.15.43:0/1402975420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1bc10ef30 0x7fa1bc10d2a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac009930 tx=0x7fa1ac02a440 comp rx=0 tx=0).stop 2024-09-19T05:07:59.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.331+0000 7fa1c2800640 1 -- 172.21.15.43:0/1402975420 shutdown_connections 2024-09-19T05:07:59.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.331+0000 7fa1c2800640 1 --2- 172.21.15.43:0/1402975420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1bc10d9b0 0x7fa1bc115510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:59.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.331+0000 7fa1c2800640 1 --2- 172.21.15.43:0/1402975420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1bc10ef30 0x7fa1bc10d2a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:59.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.331+0000 7fa1c2800640 1 --2- 172.21.15.43:0/1402975420 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1bc10e580 0x7fa1bc10e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:59.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.331+0000 7fa1c2800640 1 -- 172.21.15.43:0/1402975420 >> 172.21.15.43:0/1402975420 conn(0x7fa1bc06e140 msgr2=0x7fa1bc06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:59.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.331+0000 7fa1c2800640 1 -- 172.21.15.43:0/1402975420 shutdown_connections 2024-09-19T05:07:59.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.332+0000 7fa1c2800640 1 -- 172.21.15.43:0/1402975420 wait complete. 2024-09-19T05:07:59.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.334+0000 7fa1c2800640 1 Processor -- start 2024-09-19T05:07:59.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.335+0000 7fa1c2800640 1 -- start start 2024-09-19T05:07:59.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.337+0000 7fa1c2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1bc10d9b0 0x7fa1bc114050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:59.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.337+0000 7fa1c2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1bc10e580 0x7fa1bc114590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:59.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.337+0000 7fa1c2800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1bc10ef30 0x7fa1bc1acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:59.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.337+0000 7fa1c2800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa1bc118600 con 0x7fa1bc10e580 2024-09-19T05:07:59.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.337+0000 7fa1c2800640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa1bc118480 con 0x7fa1bc10ef30 2024-09-19T05:07:59.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.337+0000 7fa1c2800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa1bc118780 con 0x7fa1bc10d9b0 2024-09-19T05:07:59.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.338+0000 7fa1baa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1bc10e580 0x7fa1bc114590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:59.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.339+0000 7fa1bbe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1bc10ef30 0x7fa1bc1acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:59.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.339+0000 7fa1baa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1bc10e580 0x7fa1bc114590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45720/0 (socket says 172.21.15.43:45720) 2024-09-19T05:07:59.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.339+0000 7fa1baa00640 1 -- 172.21.15.43:0/3709091093 learned_addr learned my addr 172.21.15.43:0/3709091093 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:59.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.340+0000 7fa1bbe00640 1 -- 172.21.15.43:0/3709091093 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1bc10d9b0 msgr2=0x7fa1bc114050 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T05:07:59.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.340+0000 7fa1bbe00640 1 --2- 172.21.15.43:0/3709091093 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1bc10d9b0 0x7fa1bc114050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:59.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.340+0000 7fa1bbe00640 1 -- 172.21.15.43:0/3709091093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1bc10e580 msgr2=0x7fa1bc114590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:59.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.340+0000 7fa1bbe00640 1 --2- 172.21.15.43:0/3709091093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1bc10e580 0x7fa1bc114590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:59.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.340+0000 7fa1bbe00640 1 -- 172.21.15.43:0/3709091093 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1bc1ad4d0 con 0x7fa1bc10ef30 2024-09-19T05:07:59.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.340+0000 7fa1bbe00640 1 --2- 172.21.15.43:0/3709091093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1bc10ef30 0x7fa1bc1acdf0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa1b400be20 tx=0x7fa1b400bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:59.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.341+0000 7fa1b3e00640 1 -- 172.21.15.43:0/3709091093 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b4019070 con 0x7fa1bc10ef30 2024-09-19T05:07:59.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.341+0000 7fa1c2800640 1 -- 172.21.15.43:0/3709091093 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1bc1ad7c0 con 0x7fa1bc10ef30 2024-09-19T05:07:59.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.341+0000 7fa1c2800640 1 -- 172.21.15.43:0/3709091093 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1bc1b3ec0 con 0x7fa1bc10ef30 2024-09-19T05:07:59.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.343+0000 7fa1b3e00640 1 -- 172.21.15.43:0/3709091093 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1b4014920 con 0x7fa1bc10ef30 2024-09-19T05:07:59.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.344+0000 7fa1b3e00640 1 -- 172.21.15.43:0/3709091093 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b4012930 con 0x7fa1bc10ef30 2024-09-19T05:07:59.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.350+0000 7fa1b3e00640 1 -- 172.21.15.43:0/3709091093 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa1b401c070 con 0x7fa1bc10ef30 2024-09-19T05:07:59.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.353+0000 7fa1b3e00640 1 --2- 172.21.15.43:0/3709091093 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fa1a00a3430 0x7fa1a00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:59.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.354+0000 7fa1b3e00640 1 -- 172.21.15.43:0/3709091093 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fa1b40cf490 con 0x7fa1bc10ef30 2024-09-19T05:07:59.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.359+0000 7fa1c541c740 1 Processor -- start 2024-09-19T05:07:59.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.359+0000 7fa1c541c740 1 -- start start 2024-09-19T05:07:59.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.360+0000 7fa1c541c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f481a6fc70 0x55f481a70050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:59.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.360+0000 7fa1c541c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f481a70590 0x55f481a7a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:59.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.360+0000 7fa1c541c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f481a7ae50 0x55f481a7d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:59.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.360+0000 7fa1c541c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f4819d63e0 con 0x55f481a6fc70 2024-09-19T05:07:59.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.360+0000 7fa1bbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f481a7ae50 0x55f481a7d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:59.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.360+0000 7fa1bbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f481a7ae50 0x55f481a7d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:58858/0 (socket says 172.21.15.43:58858) 2024-09-19T05:07:59.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.360+0000 7fa1bbe00640 1 -- 172.21.15.43:0/684888515 learned_addr learned my addr 172.21.15.43:0/684888515 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:59.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.361+0000 7fa1c541c740 1 -- 172.21.15.43:0/684888515 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f48197dd80 con 0x55f481a70590 2024-09-19T05:07:59.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.361+0000 7fa1c541c740 1 -- 172.21.15.43:0/684888515 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f4819d23a0 con 0x55f481a7ae50 2024-09-19T05:07:59.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.361+0000 7fa1baa00640 1 --2- 172.21.15.43:0/684888515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f481a70590 0x55f481a7a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:59.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.363+0000 7fa1bbe00640 1 -- 172.21.15.43:0/684888515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f481a70590 msgr2=0x55f481a7a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:59.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.363+0000 7fa1bbe00640 1 --2- 172.21.15.43:0/684888515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f481a70590 0x55f481a7a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:59.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.363+0000 7fa1bbe00640 1 -- 172.21.15.43:0/684888515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f481a6fc70 msgr2=0x55f481a70050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T05:07:59.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.363+0000 7fa1bbe00640 1 --2- 172.21.15.43:0/684888515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f481a6fc70 0x55f481a70050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:59.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.363+0000 7fa1bbe00640 1 -- 172.21.15.43:0/684888515 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f48199cae0 con 0x55f481a7ae50 2024-09-19T05:07:59.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.363+0000 7fa1bbe00640 1 --2- 172.21.15.43:0/684888515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f481a7ae50 0x55f481a7d240 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4097c90 tx=0x7fa1b409b190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14aa096bfbf95411 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:59.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.363+0000 7fa1b0c00640 1 -- 172.21.15.43:0/684888515 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b409fbe0 con 0x55f481a7ae50 2024-09-19T05:07:59.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.364+0000 7fa1b0c00640 1 -- 172.21.15.43:0/684888515 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1b40a44a0 con 0x55f481a7ae50 2024-09-19T05:07:59.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.364+0000 7fa1b0c00640 1 -- 172.21.15.43:0/684888515 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b40a4010 con 0x55f481a7ae50 2024-09-19T05:07:59.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.365+0000 7fa1c541c740 1 -- 172.21.15.43:0/684888515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f481a7ae50 msgr2=0x55f481a7d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:59.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.365+0000 7fa1c541c740 1 --2- 172.21.15.43:0/684888515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f481a7ae50 0x55f481a7d240 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4097c90 tx=0x7fa1b409b190 comp rx=0 tx=0).stop 2024-09-19T05:07:59.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.365+0000 7fa1c541c740 1 -- 172.21.15.43:0/684888515 shutdown_connections 2024-09-19T05:07:59.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.365+0000 7fa1c541c740 1 --2- 172.21.15.43:0/684888515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f481a7ae50 0x55f481a7d240 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:59.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.365+0000 7fa1c541c740 1 --2- 172.21.15.43:0/684888515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f481a70590 0x55f481a7a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:59.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.365+0000 7fa1c541c740 1 --2- 172.21.15.43:0/684888515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f481a6fc70 0x55f481a70050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:59.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.365+0000 7fa1c541c740 1 -- 172.21.15.43:0/684888515 >> 172.21.15.43:0/684888515 conn(0x55f481a6c200 msgr2=0x55f481a6c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:59.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.367+0000 7fa1c541c740 1 -- 172.21.15.43:0/684888515 shutdown_connections 2024-09-19T05:07:59.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.367+0000 7fa1c541c740 1 -- 172.21.15.43:0/684888515 wait complete. 2024-09-19T05:07:59.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.368+0000 7fa1c541c740 1 -- start start 2024-09-19T05:07:59.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.368+0000 7fa1c541c740 1 Processor -- start 2024-09-19T05:07:59.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.369+0000 7fa1bb400640 1 --2- 172.21.15.43:0/3709091093 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fa1a00a3430 0x7fa1a00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:59.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.371+0000 7fa1c541c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f481a70590 0x55f481a77ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:59.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.371+0000 7fa1c541c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f481a7ae50 0x55f481a78430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:59.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.371+0000 7fa1c541c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f481a78970 0x55f481a86f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:59.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.371+0000 7fa1c541c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f481a7da20 con 0x55f481a70590 2024-09-19T05:07:59.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.371+0000 7fa1c541c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f4819910f0 con 0x55f481a7ae50 2024-09-19T05:07:59.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.371+0000 7fa1c541c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f481a7dba0 con 0x55f481a78970 2024-09-19T05:07:59.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.371+0000 7fa1c541c740 10 client.0 _ll_register_callbacks cb 0x55f481978bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:59.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.372+0000 7fa1bbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f481a78970 0x55f481a86f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:59.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.372+0000 7fa1bbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f481a78970 0x55f481a86f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:58866/0 (socket says 172.21.15.43:58866) 2024-09-19T05:07:59.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.372+0000 7fa1bbe00640 1 -- 172.21.15.43:0/1207951426 learned_addr learned my addr 172.21.15.43:0/1207951426 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:59.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.372+0000 7fa1baa00640 1 --2- 172.21.15.43:0/1207951426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f481a7ae50 0x55f481a78430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:59.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.374+0000 7fa1bbe00640 1 -- 172.21.15.43:0/1207951426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f481a7ae50 msgr2=0x55f481a78430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:59.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.374+0000 7fa1bbe00640 1 --2- 172.21.15.43:0/1207951426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f481a7ae50 0x55f481a78430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:59.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.374+0000 7fa1bbe00640 1 -- 172.21.15.43:0/1207951426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f481a70590 msgr2=0x55f481a77ef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T05:07:59.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.374+0000 7fa1bbe00640 1 --2- 172.21.15.43:0/1207951426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f481a70590 0x55f481a77ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:59.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.374+0000 7fa1bbe00640 1 -- 172.21.15.43:0/1207951426 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4817473c0 con 0x55f481a78970 2024-09-19T05:07:59.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.374+0000 7fa1baa00640 1 --2- 172.21.15.43:0/1207951426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f481a7ae50 0x55f481a78430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:07:59.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.375+0000 7fa1bbe00640 1 --2- 172.21.15.43:0/1207951426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f481a78970 0x55f481a86f30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4009f90 tx=0x7fa1b409b2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b9d16effe272801 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:59.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.375+0000 7fa1a5600640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:07:59.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.375+0000 7fa1a5600640 1 -- 172.21.15.43:0/1207951426 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b40b3070 con 0x55f481a78970 2024-09-19T05:07:59.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.376+0000 7fa1c541c740 1 -- 172.21.15.43:0/1207951426 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1b40a44a0 con 0x55f481a78970 2024-09-19T05:07:59.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.376+0000 7fa1a5600640 1 -- 172.21.15.43:0/1207951426 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fa1b4014440 con 0x55f481a78970 2024-09-19T05:07:59.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.377+0000 7fa1a5600640 1 -- 172.21.15.43:0/1207951426 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa1b40b3070 con 0x55f481a78970 2024-09-19T05:07:59.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.378+0000 7fa1c541c740 10 client.25852 fetch_fsmap learned FSMap version 29 2024-09-19T05:07:59.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.378+0000 7fa1c541c740 1 -- 172.21.15.43:0/1207951426 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f481a7da20 con 0x55f481a78970 2024-09-19T05:07:59.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.378+0000 7fa1a5600640 1 -- 172.21.15.43:0/1207951426 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1b40a44a0 con 0x55f481a78970 2024-09-19T05:07:59.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.378+0000 7fa1a5600640 1 -- 172.21.15.43:0/1207951426 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa1b4084b30 con 0x55f481a78970 2024-09-19T05:07:59.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.379+0000 7fa1c541c740 10 client.25852 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:07:59.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.379+0000 7fa1c541c740 10 client.25852 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T05:07:59.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.379+0000 7fa1c541c740 20 client.25852 populate_metadata read hostname 'smithi043' 2024-09-19T05:07:59.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.379+0000 7fa1c541c740 1 --2- 172.21.15.43:0/1207951426 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55f481a8b220 0x55f481a8d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:59.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.379+0000 7fa1c541c740 4 client.25852 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:07:59.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.379+0000 7fa1c541c740 1 -- 172.21.15.43:0/1207951426 --> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4819e3950 con 0x55f481a8b220 2024-09-19T05:07:59.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.381+0000 7fa1bb400640 1 --2- 172.21.15.43:0/3709091093 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fa1a00a3430 0x7fa1a00a58f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa1a80095f0 tx=0x7fa1a8005f90 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:59.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.382+0000 7fa1bb400640 1 --2- 172.21.15.43:0/1207951426 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55f481a8b220 0x55f481a8d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:59.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.384+0000 7fa1bb400640 1 --2- 172.21.15.43:0/1207951426 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55f481a8b220 0x55f481a8d610 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4c491d387717b0ae server_cookie=ec82d12ecedf4953 in_seq=0 out_seq=0 2024-09-19T05:07:59.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.384+0000 7fa1a5600640 10 client.25852 ms_handle_connect on v2:172.21.15.96:6836/3978388286 2024-09-19T05:07:59.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.385+0000 7fa1a5600640 1 -- 172.21.15.43:0/1207951426 <== mds.3 v2:172.21.15.96:6836/3978388286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f4819e3950 con 0x55f481a8b220 2024-09-19T05:07:59.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.385+0000 7fa1a5600640 10 client.25852 handle_command_reply: tid=0 2024-09-19T05:07:59.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.385+0000 7fa1c541c740 1 client.25852 shutdown 2024-09-19T05:07:59.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.385+0000 7fa1c541c740 1 -- 172.21.15.43:0/1207951426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f481a78970 msgr2=0x55f481a86f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:59.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.385+0000 7fa1c541c740 1 --2- 172.21.15.43:0/1207951426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f481a78970 0x55f481a86f30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4009f90 tx=0x7fa1b409b2c0 comp rx=0 tx=0).stop 2024-09-19T05:07:59.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.386+0000 7fa1c541c740 1 -- 172.21.15.43:0/1207951426 shutdown_connections 2024-09-19T05:07:59.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.386+0000 7fa1c541c740 1 --2- 172.21.15.43:0/1207951426 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55f481a8b220 0x55f481a8d610 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:59.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.386+0000 7fa1c541c740 1 --2- 172.21.15.43:0/1207951426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f481a78970 0x55f481a86f30 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:59.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.386+0000 7fa1a5600640 0 client.25852 ms_handle_reset on v2:172.21.15.96:6836/3978388286 2024-09-19T05:07:59.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.386+0000 7fa1c541c740 1 --2- 172.21.15.43:0/1207951426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f481a7ae50 0x55f481a78430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:59.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.386+0000 7fa1c541c740 1 --2- 172.21.15.43:0/1207951426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f481a70590 0x55f481a77ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:59.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.386+0000 7fa1c541c740 1 -- 172.21.15.43:0/1207951426 >> 172.21.15.43:0/1207951426 conn(0x55f481a6c200 msgr2=0x55f481a71f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:59.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.387+0000 7fa1c541c740 1 -- 172.21.15.43:0/1207951426 shutdown_connections 2024-09-19T05:07:59.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.387+0000 7fa1c541c740 1 -- 172.21.15.43:0/1207951426 wait complete. 2024-09-19T05:07:59.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.388+0000 7fa1c541c740 20 client.25852 trim_cache size 0 max 16384 2024-09-19T05:07:59.425 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:07:59 smithi043 ceph-mon[30800]: pgmap v103: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 252 MiB/s wr, 73 op/s 2024-09-19T05:07:59.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.423+0000 7fa1c541c740 1 Processor -- start 2024-09-19T05:07:59.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.423+0000 7fa1c541c740 1 -- start start 2024-09-19T05:07:59.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.424+0000 7fa1c541c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f481a70590 0x55f481a86d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:59.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.424+0000 7fa1c541c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f481a78970 0x55f481a71e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:59.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.425+0000 7fa1c541c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f481a7ae50 0x55f481a92cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:59.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.425+0000 7fa1c541c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa19c00e7f0 con 0x55f481a7ae50 2024-09-19T05:07:59.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.425+0000 7fa1c541c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa19c010330 con 0x55f481a78970 2024-09-19T05:07:59.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.425+0000 7fa1c541c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa19c00eec0 con 0x55f481a70590 2024-09-19T05:07:59.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.426+0000 7fa1bbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f481a7ae50 0x55f481a92cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:59.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.427+0000 7fa1bbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f481a7ae50 0x55f481a92cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45732/0 (socket says 172.21.15.43:45732) 2024-09-19T05:07:59.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.427+0000 7fa1bbe00640 1 -- 172.21.15.43:0/3469926946 learned_addr learned my addr 172.21.15.43:0/3469926946 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:59.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.427+0000 7fa1bb400640 1 --2- 172.21.15.43:0/3469926946 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f481a70590 0x55f481a86d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:59.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.428+0000 7fa1baa00640 1 --2- 172.21.15.43:0/3469926946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f481a78970 0x55f481a71e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:59.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.433+0000 7fa1bb400640 1 -- 172.21.15.43:0/3469926946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f481a78970 msgr2=0x55f481a71e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:59.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.434+0000 7fa1bb400640 1 --2- 172.21.15.43:0/3469926946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f481a78970 0x55f481a71e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:59.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.434+0000 7fa1bb400640 1 -- 172.21.15.43:0/3469926946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f481a7ae50 msgr2=0x55f481a92cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:59.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.434+0000 7fa1bb400640 1 --2- 172.21.15.43:0/3469926946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f481a7ae50 0x55f481a92cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:59.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.434+0000 7fa1bb400640 1 -- 172.21.15.43:0/3469926946 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f481a67770 con 0x55f481a70590 2024-09-19T05:07:59.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.434+0000 7fa1baa00640 1 --2- 172.21.15.43:0/3469926946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f481a78970 0x55f481a71e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:07:59.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.434+0000 7fa1bbe00640 1 --2- 172.21.15.43:0/3469926946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f481a7ae50 0x55f481a92cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:07:59.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.434+0000 7fa1bb400640 1 --2- 172.21.15.43:0/3469926946 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f481a70590 0x55f481a86d20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x55f481a8dc10 tx=0x7fa1a8015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=215bae9b8edc7c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:59.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.435+0000 7fa1a7e00640 1 -- 172.21.15.43:0/3469926946 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8029070 con 0x55f481a70590 2024-09-19T05:07:59.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.435+0000 7fa1a7e00640 1 -- 172.21.15.43:0/3469926946 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1a80242d0 con 0x55f481a70590 2024-09-19T05:07:59.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.435+0000 7fa1a7e00640 1 -- 172.21.15.43:0/3469926946 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a801c680 con 0x55f481a70590 2024-09-19T05:07:59.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.436+0000 7fa1c541c740 1 -- 172.21.15.43:0/3469926946 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f481a70590 msgr2=0x55f481a86d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:59.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.436+0000 7fa1c541c740 1 --2- 172.21.15.43:0/3469926946 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f481a70590 0x55f481a86d20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x55f481a8dc10 tx=0x7fa1a8015000 comp rx=0 tx=0).stop 2024-09-19T05:07:59.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.436+0000 7fa1c541c740 1 -- 172.21.15.43:0/3469926946 shutdown_connections 2024-09-19T05:07:59.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.436+0000 7fa1c541c740 1 --2- 172.21.15.43:0/3469926946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f481a7ae50 0x55f481a92cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:59.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.436+0000 7fa1c541c740 1 --2- 172.21.15.43:0/3469926946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f481a78970 0x55f481a71e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:59.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.436+0000 7fa1c541c740 1 --2- 172.21.15.43:0/3469926946 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f481a70590 0x55f481a86d20 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:59.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.436+0000 7fa1c541c740 1 -- 172.21.15.43:0/3469926946 >> 172.21.15.43:0/3469926946 conn(0x55f481a6c200 msgr2=0x55f481a71710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:59.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.438+0000 7fa1c541c740 1 -- 172.21.15.43:0/3469926946 shutdown_connections 2024-09-19T05:07:59.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.439+0000 7fa1c541c740 1 -- 172.21.15.43:0/3469926946 wait complete. 2024-09-19T05:07:59.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.440+0000 7fa1c541c740 1 -- start start 2024-09-19T05:07:59.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.442+0000 7fa1c541c740 1 Processor -- start 2024-09-19T05:07:59.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.445+0000 7fa1c541c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f481a70590 0x55f481a7eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:59.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.445+0000 7fa1c541c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f481a78970 0x55f481a7f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:59.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.445+0000 7fa1c541c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f481a7ae50 0x55f481a7f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:59.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.445+0000 7fa1c541c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa19c00fc60 con 0x55f481a78970 2024-09-19T05:07:59.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.445+0000 7fa1c541c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa19c00f590 con 0x55f481a7ae50 2024-09-19T05:07:59.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.445+0000 7fa1c541c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa19c00c7d0 con 0x55f481a70590 2024-09-19T05:07:59.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.446+0000 7fa1c541c740 10 client.0 _ll_register_callbacks cb 0x7fa1a8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:07:59.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.446+0000 7fa1bbe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f481a7ae50 0x55f481a7f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:59.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.446+0000 7fa1bbe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f481a7ae50 0x55f481a7f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:44102/0 (socket says 172.21.15.43:44102) 2024-09-19T05:07:59.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.446+0000 7fa1bbe00640 1 -- 172.21.15.43:0/3252035467 learned_addr learned my addr 172.21.15.43:0/3252035467 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:07:59.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.446+0000 7fa1bbe00640 1 -- 172.21.15.43:0/3252035467 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f481a70590 msgr2=0x55f481a7eed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T05:07:59.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.446+0000 7fa1bbe00640 1 --2- 172.21.15.43:0/3252035467 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f481a70590 0x55f481a7eed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:59.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.446+0000 7fa1bbe00640 1 -- 172.21.15.43:0/3252035467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f481a78970 msgr2=0x55f481a7f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:07:59.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.446+0000 7fa1bbe00640 1 --2- 172.21.15.43:0/3252035467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f481a78970 0x55f481a7f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:59.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.446+0000 7fa1bbe00640 1 -- 172.21.15.43:0/3252035467 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f481af4f70 con 0x55f481a7ae50 2024-09-19T05:07:59.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.447+0000 7fa1bbe00640 1 --2- 172.21.15.43:0/3252035467 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f481a7ae50 0x55f481a7f970 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa1b400bdf0 tx=0x7fa1b4097410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b781be09efc2d650 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:07:59.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.449+0000 7fa1a5600640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:07:59.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.449+0000 7fa1a5600640 1 -- 172.21.15.43:0/3252035467 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b40a44a0 con 0x55f481a7ae50 2024-09-19T05:07:59.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.450+0000 7fa1a5600640 1 -- 172.21.15.43:0/3252035467 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fa1b40a4010 con 0x55f481a7ae50 2024-09-19T05:07:59.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.450+0000 7fa1c541c740 1 -- 172.21.15.43:0/3252035467 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1a80242d0 con 0x55f481a7ae50 2024-09-19T05:07:59.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.451+0000 7fa1a5600640 1 -- 172.21.15.43:0/3252035467 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa1b407f070 con 0x55f481a7ae50 2024-09-19T05:07:59.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.452+0000 7fa1c541c740 10 client.25910 fetch_fsmap learned FSMap version 29 2024-09-19T05:07:59.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.452+0000 7fa1c541c740 1 -- 172.21.15.43:0/3252035467 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f481af6270 con 0x55f481a7ae50 2024-09-19T05:07:59.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.454+0000 7fa1a5600640 1 -- 172.21.15.43:0/3252035467 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1a80242d0 con 0x55f481a7ae50 2024-09-19T05:07:59.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.458+0000 7fa1a5600640 1 -- 172.21.15.43:0/3252035467 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa1b40976a0 con 0x55f481a7ae50 2024-09-19T05:07:59.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.458+0000 7fa1c541c740 10 client.25910 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:07:59.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.458+0000 7fa1c541c740 10 client.25910 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T05:07:59.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.458+0000 7fa1c541c740 20 client.25910 populate_metadata read hostname 'smithi043' 2024-09-19T05:07:59.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.458+0000 7fa1c541c740 1 --2- 172.21.15.43:0/3252035467 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55f481af6750 0x55f481af8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:07:59.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.458+0000 7fa1c541c740 4 client.25910 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:07:59.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.458+0000 7fa1c541c740 1 -- 172.21.15.43:0/3252035467 --> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f4819dde60 con 0x55f481af6750 2024-09-19T05:07:59.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.464+0000 7fa1bb400640 1 --2- 172.21.15.43:0/3252035467 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55f481af6750 0x55f481af8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:07:59.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.466+0000 7fa1bb400640 1 --2- 172.21.15.43:0/3252035467 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55f481af6750 0x55f481af8b40 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3cd9f2699e0d125c server_cookie=4eaf9ace9afbc280 in_seq=0 out_seq=0 2024-09-19T05:07:59.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.466+0000 7fa1a5600640 10 client.25910 ms_handle_connect on v2:172.21.15.96:6836/3978388286 2024-09-19T05:07:59.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.466+0000 7fa1a5600640 1 -- 172.21.15.43:0/3252035467 <== mds.3 v2:172.21.15.96:6836/3978388286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f4819dde60 con 0x55f481af6750 2024-09-19T05:07:59.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.466+0000 7fa1a5600640 10 client.25910 handle_command_reply: tid=0 2024-09-19T05:07:59.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.466+0000 7fa1c541c740 1 client.25910 shutdown 2024-09-19T05:07:59.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.468+0000 7fa1c541c740 1 -- 172.21.15.43:0/3252035467 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f481a7ae50 msgr2=0x55f481a7f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:59.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.468+0000 7fa1c541c740 1 --2- 172.21.15.43:0/3252035467 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f481a7ae50 0x55f481a7f970 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa1b400bdf0 tx=0x7fa1b4097410 comp rx=0 tx=0).stop 2024-09-19T05:07:59.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.469+0000 7fa1c541c740 1 -- 172.21.15.43:0/3252035467 shutdown_connections 2024-09-19T05:07:59.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.469+0000 7fa1c541c740 1 --2- 172.21.15.43:0/3252035467 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55f481af6750 0x55f481af8b40 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:59.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.469+0000 7fa1c541c740 1 --2- 172.21.15.43:0/3252035467 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f481a7ae50 0x55f481a7f970 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:59.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.469+0000 7fa1c541c740 1 --2- 172.21.15.43:0/3252035467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f481a78970 0x55f481a7f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:59.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.469+0000 7fa1c541c740 1 --2- 172.21.15.43:0/3252035467 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f481a70590 0x55f481a7eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:59.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.469+0000 7fa1c541c740 1 -- 172.21.15.43:0/3252035467 >> 172.21.15.43:0/3252035467 conn(0x55f481a6c200 msgr2=0x55f481a71ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:59.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.469+0000 7fa1a5600640 0 client.25910 ms_handle_reset on v2:172.21.15.96:6836/3978388286 2024-09-19T05:07:59.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.469+0000 7fa1c541c740 1 -- 172.21.15.43:0/3252035467 shutdown_connections 2024-09-19T05:07:59.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.470+0000 7fa1c541c740 1 -- 172.21.15.43:0/3252035467 wait complete. 2024-09-19T05:07:59.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.470+0000 7fa1c541c740 20 client.25910 trim_cache size 0 max 16384 2024-09-19T05:07:59.473 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:07:59.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.471+0000 7fa1c2800640 1 -- 172.21.15.43:0/3709091093 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fa1a00a3430 msgr2=0x7fa1a00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:59.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.471+0000 7fa1c2800640 1 --2- 172.21.15.43:0/3709091093 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fa1a00a3430 0x7fa1a00a58f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa1a80095f0 tx=0x7fa1a8005f90 comp rx=0 tx=0).stop 2024-09-19T05:07:59.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.471+0000 7fa1c2800640 1 -- 172.21.15.43:0/3709091093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1bc10ef30 msgr2=0x7fa1bc1acdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:07:59.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.471+0000 7fa1c2800640 1 --2- 172.21.15.43:0/3709091093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1bc10ef30 0x7fa1bc1acdf0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa1b400be20 tx=0x7fa1b400bf10 comp rx=0 tx=0).stop 2024-09-19T05:07:59.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.471+0000 7fa1c2800640 1 -- 172.21.15.43:0/3709091093 shutdown_connections 2024-09-19T05:07:59.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.471+0000 7fa1c2800640 1 --2- 172.21.15.43:0/3709091093 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fa1a00a3430 0x7fa1a00a58f0 secure :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa1a80095f0 tx=0x7fa1a8005f90 comp rx=0 tx=0).stop 2024-09-19T05:07:59.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.471+0000 7fa1c2800640 1 --2- 172.21.15.43:0/3709091093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1bc10ef30 0x7fa1bc1acdf0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:59.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.471+0000 7fa1c2800640 1 --2- 172.21.15.43:0/3709091093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1bc10e580 0x7fa1bc114590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:59.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.471+0000 7fa1c2800640 1 --2- 172.21.15.43:0/3709091093 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1bc10d9b0 0x7fa1bc114050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:07:59.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.471+0000 7fa1c2800640 1 -- 172.21.15.43:0/3709091093 >> 172.21.15.43:0/3709091093 conn(0x7fa1bc06e140 msgr2=0x7fa1bc11b030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:07:59.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.472+0000 7fa1c2800640 1 -- 172.21.15.43:0/3709091093 shutdown_connections 2024-09-19T05:07:59.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:07:59.472+0000 7fa1c2800640 1 -- 172.21.15.43:0/3709091093 wait complete. 2024-09-19T05:07:59.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:07:59 smithi096 ceph-mon[35085]: pgmap v103: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 252 MiB/s wr, 73 op/s 2024-09-19T05:07:59.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:07:59 smithi151 ceph-mon[35425]: pgmap v103: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 252 MiB/s wr, 73 op/s 2024-09-19T05:08:00.705 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T05:08:01.435 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:08:01.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:08:01 smithi096 ceph-mon[35085]: pgmap v104: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 383 MiB/s wr, 110 op/s 2024-09-19T05:08:01.784 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:08:01 smithi043 ceph-mon[30800]: pgmap v104: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 383 MiB/s wr, 110 op/s 2024-09-19T05:08:01.904 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:08:01 smithi151 ceph-mon[35425]: pgmap v104: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 383 MiB/s wr, 110 op/s 2024-09-19T05:08:02.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.286+0000 7fb1b9000640 1 -- 172.21.15.43:0/3999876202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1b4109120 msgr2=0x7fb1b4110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:02.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.286+0000 7fb1b9000640 1 --2- 172.21.15.43:0/3999876202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1b4109120 0x7fb1b4110ca0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac009be0 tx=0x7fb1ac02a6c0 comp rx=0 tx=0).stop 2024-09-19T05:08:02.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.286+0000 7fb1b9000640 1 -- 172.21.15.43:0/3999876202 shutdown_connections 2024-09-19T05:08:02.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.286+0000 7fb1b9000640 1 --2- 172.21.15.43:0/3999876202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1b4109120 0x7fb1b4110ca0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:02.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.286+0000 7fb1b9000640 1 --2- 172.21.15.43:0/3999876202 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1b41049a0 0x7fb1b41089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:02.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.286+0000 7fb1b9000640 1 --2- 172.21.15.43:0/3999876202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1b4103ff0 0x7fb1b41043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:02.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.286+0000 7fb1b9000640 1 -- 172.21.15.43:0/3999876202 >> 172.21.15.43:0/3999876202 conn(0x7fb1b40ffa90 msgr2=0x7fb1b4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:02.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.287+0000 7fb1b9000640 1 -- 172.21.15.43:0/3999876202 shutdown_connections 2024-09-19T05:08:02.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.287+0000 7fb1b9000640 1 -- 172.21.15.43:0/3999876202 wait complete. 2024-09-19T05:08:02.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.288+0000 7fb1b9000640 1 Processor -- start 2024-09-19T05:08:02.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.288+0000 7fb1b9000640 1 -- start start 2024-09-19T05:08:02.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.289+0000 7fb1b9000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1b4103ff0 0x7fb1b419a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:02.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.289+0000 7fb1b9000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1b41049a0 0x7fb1b419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:02.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.289+0000 7fb1b9000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1b4109120 0x7fb1b419ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:02.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.289+0000 7fb1b9000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b4114130 con 0x7fb1b4103ff0 2024-09-19T05:08:02.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.289+0000 7fb1b9000640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b4113fb0 con 0x7fb1b41049a0 2024-09-19T05:08:02.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.289+0000 7fb1b9000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b41142b0 con 0x7fb1b4109120 2024-09-19T05:08:02.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.289+0000 7fb1b1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1b4103ff0 0x7fb1b419a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:02.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.289+0000 7fb1b2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1b4109120 0x7fb1b419ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:02.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.290+0000 7fb1b2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1b4109120 0x7fb1b419ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:58900/0 (socket says 172.21.15.43:58900) 2024-09-19T05:08:02.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.290+0000 7fb1b2800640 1 -- 172.21.15.43:0/2331393816 learned_addr learned my addr 172.21.15.43:0/2331393816 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:02.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.290+0000 7fb1b1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1b4103ff0 0x7fb1b419a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45746/0 (socket says 172.21.15.43:45746) 2024-09-19T05:08:02.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.290+0000 7fb1b1400640 1 --2- 172.21.15.43:0/2331393816 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1b41049a0 0x7fb1b419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:02.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.290+0000 7fb1b1e00640 1 -- 172.21.15.43:0/2331393816 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1b4109120 msgr2=0x7fb1b419ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:02.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.290+0000 7fb1b1e00640 1 --2- 172.21.15.43:0/2331393816 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1b4109120 0x7fb1b419ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:02.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.290+0000 7fb1b1e00640 1 -- 172.21.15.43:0/2331393816 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1b41049a0 msgr2=0x7fb1b419aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:02.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.290+0000 7fb1b1e00640 1 --2- 172.21.15.43:0/2331393816 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1b41049a0 0x7fb1b419aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:02.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.290+0000 7fb1b1e00640 1 -- 172.21.15.43:0/2331393816 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b419f4e0 con 0x7fb1b4103ff0 2024-09-19T05:08:02.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.290+0000 7fb1b2800640 1 --2- 172.21.15.43:0/2331393816 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1b4109120 0x7fb1b419ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:02.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.291+0000 7fb1b1e00640 1 --2- 172.21.15.43:0/2331393816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1b4103ff0 0x7fb1b419a530 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb1a400c970 tx=0x7fb1a400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:02.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.291+0000 7fb1aaa00640 1 -- 172.21.15.43:0/2331393816 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a4007bb0 con 0x7fb1b4103ff0 2024-09-19T05:08:02.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.291+0000 7fb1aaa00640 1 -- 172.21.15.43:0/2331393816 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1a4007d50 con 0x7fb1b4103ff0 2024-09-19T05:08:02.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.291+0000 7fb1aaa00640 1 -- 172.21.15.43:0/2331393816 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a4018820 con 0x7fb1b4103ff0 2024-09-19T05:08:02.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.291+0000 7fb1b9000640 1 -- 172.21.15.43:0/2331393816 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1b419f7d0 con 0x7fb1b4103ff0 2024-09-19T05:08:02.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.292+0000 7fb1b9000640 1 -- 172.21.15.43:0/2331393816 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1b41a70b0 con 0x7fb1b4103ff0 2024-09-19T05:08:02.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.297+0000 7fb1bbbe2740 1 Processor -- start 2024-09-19T05:08:02.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.298+0000 7fb1bbbe2740 1 -- start start 2024-09-19T05:08:02.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.298+0000 7fb1aaa00640 1 -- 172.21.15.43:0/2331393816 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb1a40189c0 con 0x7fb1b4103ff0 2024-09-19T05:08:02.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.300+0000 7fb1bbbe2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cc122bbc70 0x55cc122bc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:02.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.300+0000 7fb1aaa00640 1 --2- 172.21.15.43:0/2331393816 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb1980a3430 0x7fb1980a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:02.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.300+0000 7fb1aaa00640 1 -- 172.21.15.43:0/2331393816 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fb1a40cfb80 con 0x7fb1b4103ff0 2024-09-19T05:08:02.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.301+0000 7fb1bbbe2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc122bc590 0x55cc122c6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:02.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.301+0000 7fb1bbbe2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc122c6e50 0x55cc122c9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:02.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.301+0000 7fb1bbbe2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cc122223e0 con 0x55cc122c6e50 2024-09-19T05:08:02.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.301+0000 7fb1bbbe2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55cc121c9d80 con 0x55cc122bc590 2024-09-19T05:08:02.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.301+0000 7fb1bbbe2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55cc1221e3a0 con 0x55cc122bbc70 2024-09-19T05:08:02.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.301+0000 7fb1b1e00640 1 --2- 172.21.15.43:0/2331393816 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb1980a3430 0x7fb1980a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:02.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.301+0000 7fb1b1400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc122bc590 0x55cc122c6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:02.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.301+0000 7fb1b1400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc122bc590 0x55cc122c6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:44142/0 (socket says 172.21.15.43:44142) 2024-09-19T05:08:02.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.301+0000 7fb1b1400640 1 -- 172.21.15.43:0/4217380464 learned_addr learned my addr 172.21.15.43:0/4217380464 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:02.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.302+0000 7fb1b1400640 1 -- 172.21.15.43:0/4217380464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cc122bbc70 msgr2=0x55cc122bc050 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:08:02.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.302+0000 7fb1b1400640 1 --2- 172.21.15.43:0/4217380464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cc122bbc70 0x55cc122bc050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:02.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.302+0000 7fb1b1e00640 1 --2- 172.21.15.43:0/2331393816 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb1980a3430 0x7fb1980a58f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb1a40982d0 tx=0x7fb1a4007f00 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:02.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.302+0000 7fb1b1400640 1 -- 172.21.15.43:0/4217380464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc122c6e50 msgr2=0x55cc122c9240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T05:08:02.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.302+0000 7fb1b2800640 1 --2- 172.21.15.43:0/4217380464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc122c6e50 0x55cc122c9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:02.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.302+0000 7fb1b1400640 1 --2- 172.21.15.43:0/4217380464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc122c6e50 0x55cc122c9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:02.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.302+0000 7fb1b1400640 1 -- 172.21.15.43:0/4217380464 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc121e8ae0 con 0x55cc122bc590 2024-09-19T05:08:02.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.302+0000 7fb1b2800640 1 --2- 172.21.15.43:0/4217380464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc122c6e50 0x55cc122c9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:08:02.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.303+0000 7fb1b1400640 1 --2- 172.21.15.43:0/4217380464 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc122bc590 0x55cc122c6910 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0004580 tx=0x7fb1a000bbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc7b6f5c65adf8e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:02.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.303+0000 7fb19f400640 1 -- 172.21.15.43:0/4217380464 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a0003bd0 con 0x55cc122bc590 2024-09-19T05:08:02.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.303+0000 7fb19f400640 1 -- 172.21.15.43:0/4217380464 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1a0015070 con 0x55cc122bc590 2024-09-19T05:08:02.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.303+0000 7fb19f400640 1 -- 172.21.15.43:0/4217380464 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a0011730 con 0x55cc122bc590 2024-09-19T05:08:02.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.304+0000 7fb1bbbe2740 1 -- 172.21.15.43:0/4217380464 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc122bc590 msgr2=0x55cc122c6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:02.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.304+0000 7fb1bbbe2740 1 --2- 172.21.15.43:0/4217380464 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc122bc590 0x55cc122c6910 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0004580 tx=0x7fb1a000bbc0 comp rx=0 tx=0).stop 2024-09-19T05:08:02.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.305+0000 7fb1bbbe2740 1 -- 172.21.15.43:0/4217380464 shutdown_connections 2024-09-19T05:08:02.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.305+0000 7fb1bbbe2740 1 --2- 172.21.15.43:0/4217380464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc122c6e50 0x55cc122c9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:02.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.305+0000 7fb1bbbe2740 1 --2- 172.21.15.43:0/4217380464 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc122bc590 0x55cc122c6910 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:02.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.305+0000 7fb1bbbe2740 1 --2- 172.21.15.43:0/4217380464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cc122bbc70 0x55cc122bc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:02.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.305+0000 7fb1bbbe2740 1 -- 172.21.15.43:0/4217380464 >> 172.21.15.43:0/4217380464 conn(0x55cc122b8200 msgr2=0x55cc122b85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:02.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.305+0000 7fb1bbbe2740 1 -- 172.21.15.43:0/4217380464 shutdown_connections 2024-09-19T05:08:02.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.305+0000 7fb1bbbe2740 1 -- 172.21.15.43:0/4217380464 wait complete. 2024-09-19T05:08:02.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.306+0000 7fb1bbbe2740 1 -- start start 2024-09-19T05:08:02.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.306+0000 7fb1bbbe2740 1 Processor -- start 2024-09-19T05:08:02.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.307+0000 7fb1bbbe2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cc122bbc70 0x55cc122c3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:02.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.307+0000 7fb1bbbe2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc122bc590 0x55cc122c43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:02.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.307+0000 7fb1bbbe2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc122c6e50 0x55cc122c48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:02.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.307+0000 7fb1bbbe2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cc122c9a20 con 0x55cc122bc590 2024-09-19T05:08:02.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.307+0000 7fb1bbbe2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55cc121dd0f0 con 0x55cc122c6e50 2024-09-19T05:08:02.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.307+0000 7fb1bbbe2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55cc122c9ba0 con 0x55cc122bbc70 2024-09-19T05:08:02.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.308+0000 7fb1b1400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc122bc590 0x55cc122c43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:02.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.308+0000 7fb1b2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cc122bbc70 0x55cc122c3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:02.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.308+0000 7fb1b1400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc122bc590 0x55cc122c43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45772/0 (socket says 172.21.15.43:45772) 2024-09-19T05:08:02.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.308+0000 7fb1b2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cc122bbc70 0x55cc122c3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:58914/0 (socket says 172.21.15.43:58914) 2024-09-19T05:08:02.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.308+0000 7fb1b1400640 1 -- 172.21.15.43:0/2641562889 learned_addr learned my addr 172.21.15.43:0/2641562889 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:02.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.308+0000 7fb1b2800640 1 --2- 172.21.15.43:0/2641562889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc122c6e50 0x55cc122c48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:02.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.309+0000 7fb1b1400640 1 -- 172.21.15.43:0/2641562889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cc122bbc70 msgr2=0x55cc122c3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:02.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.309+0000 7fb1b1400640 1 --2- 172.21.15.43:0/2641562889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cc122bbc70 0x55cc122c3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:02.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.309+0000 7fb1b1400640 1 -- 172.21.15.43:0/2641562889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc122c6e50 msgr2=0x55cc122c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:02.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.309+0000 7fb1b1400640 1 --2- 172.21.15.43:0/2641562889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc122c6e50 0x55cc122c48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:02.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.309+0000 7fb1b1400640 1 -- 172.21.15.43:0/2641562889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc11f933c0 con 0x55cc122bc590 2024-09-19T05:08:02.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.309+0000 7fb1b2800640 1 --2- 172.21.15.43:0/2641562889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc122c6e50 0x55cc122c48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:02.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.309+0000 7fb1bbbe2740 10 client.0 _ll_register_callbacks cb 0x55cc121c4bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:02.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.309+0000 7fb1bbbe2740 1 -- 172.21.15.43:0/2641562889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1a0015070 con 0x55cc122bc590 2024-09-19T05:08:02.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.310+0000 7fb1b1400640 1 --2- 172.21.15.43:0/2641562889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc122bc590 0x55cc122c43a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0003e00 tx=0x7fb1a0005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d7da28b10a6e0c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:02.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.310+0000 7fb193e00640 10 client.15879 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:08:02.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.311+0000 7fb193e00640 1 -- 172.21.15.43:0/2641562889 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a001a680 con 0x55cc122bc590 2024-09-19T05:08:02.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.312+0000 7fb193e00640 1 -- 172.21.15.43:0/2641562889 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fb1a001a820 con 0x55cc122bc590 2024-09-19T05:08:02.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.312+0000 7fb193e00640 1 -- 172.21.15.43:0/2641562889 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb1a0005070 con 0x55cc122bc590 2024-09-19T05:08:02.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.312+0000 7fb1bbbe2740 10 client.15879 fetch_fsmap learned FSMap version 29 2024-09-19T05:08:02.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.312+0000 7fb1bbbe2740 1 -- 172.21.15.43:0/2641562889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({fsmap=29}) -- 0x55cc122d59e0 con 0x55cc122bc590 2024-09-19T05:08:02.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.313+0000 7fb193e00640 1 -- 172.21.15.43:0/2641562889 <== mon.0 v2:172.21.15.43:3300/0 4 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb1a0005210 con 0x55cc122bc590 2024-09-19T05:08:02.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.313+0000 7fb1bbbe2740 10 client.15879 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:08:02.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.313+0000 7fb1bbbe2740 10 client.15879 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T05:08:02.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.313+0000 7fb1bbbe2740 20 client.15879 populate_metadata read hostname 'smithi043' 2024-09-19T05:08:02.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.313+0000 7fb1bbbe2740 1 --2- 172.21.15.43:0/2641562889 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55cc122d6d90 0x55cc122d9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:02.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.313+0000 7fb1bbbe2740 4 client.15879 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:08:02.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.313+0000 7fb1bbbe2740 1 -- 172.21.15.43:0/2641562889 --> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc1222f950 con 0x55cc122d6d90 2024-09-19T05:08:02.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.314+0000 7fb1b2800640 1 --2- 172.21.15.43:0/2641562889 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55cc122d6d90 0x55cc122d9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:02.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.316+0000 7fb1b2800640 1 --2- 172.21.15.43:0/2641562889 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55cc122d6d90 0x55cc122d9180 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b72aff03f2ef8c3b server_cookie=f0ca3223921e71e3 in_seq=0 out_seq=0 2024-09-19T05:08:02.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.316+0000 7fb193e00640 10 client.15879 ms_handle_connect on v2:172.21.15.151:6838/3794104313 2024-09-19T05:08:02.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.319+0000 7fb193e00640 1 -- 172.21.15.43:0/2641562889 <== mds.2 v2:172.21.15.151:6838/3794104313 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cc1222f950 con 0x55cc122d6d90 2024-09-19T05:08:02.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.319+0000 7fb193e00640 10 client.15879 handle_command_reply: tid=0 2024-09-19T05:08:02.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.320+0000 7fb1bbbe2740 1 client.15879 shutdown 2024-09-19T05:08:02.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.320+0000 7fb1bbbe2740 1 -- 172.21.15.43:0/2641562889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc122bc590 msgr2=0x55cc122c43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:02.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.320+0000 7fb1bbbe2740 1 --2- 172.21.15.43:0/2641562889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc122bc590 0x55cc122c43a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0003e00 tx=0x7fb1a0005900 comp rx=0 tx=0).stop 2024-09-19T05:08:02.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.321+0000 7fb1bbbe2740 1 -- 172.21.15.43:0/2641562889 shutdown_connections 2024-09-19T05:08:02.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.321+0000 7fb1bbbe2740 1 --2- 172.21.15.43:0/2641562889 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55cc122d6d90 0x55cc122d9180 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:02.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.321+0000 7fb193e00640 0 client.15879 ms_handle_reset on v2:172.21.15.151:6838/3794104313 2024-09-19T05:08:02.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.321+0000 7fb1bbbe2740 1 --2- 172.21.15.43:0/2641562889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc122c6e50 0x55cc122c48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:02.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.321+0000 7fb1bbbe2740 1 --2- 172.21.15.43:0/2641562889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc122bc590 0x55cc122c43a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:02.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.321+0000 7fb1bbbe2740 1 --2- 172.21.15.43:0/2641562889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cc122bbc70 0x55cc122c3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:02.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.321+0000 7fb1bbbe2740 1 -- 172.21.15.43:0/2641562889 >> 172.21.15.43:0/2641562889 conn(0x55cc122b8200 msgr2=0x55cc122b7380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:02.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.323+0000 7fb1bbbe2740 1 -- 172.21.15.43:0/2641562889 shutdown_connections 2024-09-19T05:08:02.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.323+0000 7fb1bbbe2740 1 -- 172.21.15.43:0/2641562889 wait complete. 2024-09-19T05:08:02.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.323+0000 7fb1bbbe2740 20 client.15879 trim_cache size 0 max 16384 2024-09-19T05:08:02.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.346+0000 7fb1bbbe2740 1 Processor -- start 2024-09-19T05:08:02.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.347+0000 7fb1bbbe2740 1 -- start start 2024-09-19T05:08:02.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.347+0000 7fb1bbbe2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc122bbc70 0x55cc122b85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:02.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.347+0000 7fb1bbbe2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc122bc590 0x55cc122dc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:02.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.347+0000 7fb1bbbe2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cc122c6e50 0x55cc122dc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:02.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.347+0000 7fb1bbbe2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b41ab550 con 0x55cc122bbc70 2024-09-19T05:08:02.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.347+0000 7fb1bbbe2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b41ad090 con 0x55cc122bc590 2024-09-19T05:08:02.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.347+0000 7fb1bbbe2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b41abc20 con 0x55cc122c6e50 2024-09-19T05:08:02.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.347+0000 7fb1b2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc122bbc70 0x55cc122b85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:02.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.347+0000 7fb1b2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc122bbc70 0x55cc122b85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45784/0 (socket says 172.21.15.43:45784) 2024-09-19T05:08:02.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.347+0000 7fb1b2800640 1 -- 172.21.15.43:0/708835281 learned_addr learned my addr 172.21.15.43:0/708835281 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:02.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.348+0000 7fb1b1400640 1 --2- 172.21.15.43:0/708835281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc122bc590 0x55cc122dc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:02.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.348+0000 7fb1b2800640 1 --2- 172.21.15.43:0/708835281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cc122c6e50 0x55cc122dc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:02.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.348+0000 7fb1b1400640 1 -- 172.21.15.43:0/708835281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cc122c6e50 msgr2=0x55cc122dc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:02.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.348+0000 7fb1b1400640 1 --2- 172.21.15.43:0/708835281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cc122c6e50 0x55cc122dc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:02.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.348+0000 7fb1b1400640 1 -- 172.21.15.43:0/708835281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc122bbc70 msgr2=0x55cc122b85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:02.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.348+0000 7fb1b1400640 1 --2- 172.21.15.43:0/708835281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc122bbc70 0x55cc122b85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:02.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.348+0000 7fb1b1400640 1 -- 172.21.15.43:0/708835281 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc122c2e10 con 0x55cc122bc590 2024-09-19T05:08:02.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.348+0000 7fb1b1400640 1 --2- 172.21.15.43:0/708835281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc122bc590 0x55cc122dc1a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb1a00041f0 tx=0x7fb1a0005000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bd67a4d6c95b32b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:02.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.349+0000 7fb19ea00640 1 -- 172.21.15.43:0/708835281 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a0005850 con 0x55cc122bc590 2024-09-19T05:08:02.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.349+0000 7fb19ea00640 1 -- 172.21.15.43:0/708835281 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1a0015070 con 0x55cc122bc590 2024-09-19T05:08:02.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.349+0000 7fb19ea00640 1 -- 172.21.15.43:0/708835281 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a0004a00 con 0x55cc122bc590 2024-09-19T05:08:02.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.351+0000 7fb1bbbe2740 1 -- 172.21.15.43:0/708835281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc122bc590 msgr2=0x55cc122dc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:02.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.351+0000 7fb1bbbe2740 1 --2- 172.21.15.43:0/708835281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc122bc590 0x55cc122dc1a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb1a00041f0 tx=0x7fb1a0005000 comp rx=0 tx=0).stop 2024-09-19T05:08:02.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.351+0000 7fb1bbbe2740 1 -- 172.21.15.43:0/708835281 shutdown_connections 2024-09-19T05:08:02.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.351+0000 7fb1bbbe2740 1 --2- 172.21.15.43:0/708835281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cc122c6e50 0x55cc122dc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:02.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.351+0000 7fb1bbbe2740 1 --2- 172.21.15.43:0/708835281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc122bc590 0x55cc122dc1a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:02.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.351+0000 7fb1bbbe2740 1 --2- 172.21.15.43:0/708835281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc122bbc70 0x55cc122b85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:02.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.351+0000 7fb1bbbe2740 1 -- 172.21.15.43:0/708835281 >> 172.21.15.43:0/708835281 conn(0x55cc122b8200 msgr2=0x55cc122c5ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:02.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.351+0000 7fb1bbbe2740 1 -- 172.21.15.43:0/708835281 shutdown_connections 2024-09-19T05:08:02.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.351+0000 7fb1bbbe2740 1 -- 172.21.15.43:0/708835281 wait complete. 2024-09-19T05:08:02.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.351+0000 7fb1bbbe2740 1 -- start start 2024-09-19T05:08:02.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.352+0000 7fb1bbbe2740 1 Processor -- start 2024-09-19T05:08:02.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.352+0000 7fb1bbbe2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc122bbc70 0x55cc122dca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:02.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.352+0000 7fb1bbbe2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cc122bc590 0x55cc122d9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:02.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.352+0000 7fb1bbbe2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc122c6e50 0x55cc122da4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:02.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.352+0000 7fb1bbbe2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b41ac9c0 con 0x55cc122c6e50 2024-09-19T05:08:02.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.352+0000 7fb1bbbe2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b41ac2f0 con 0x55cc122bbc70 2024-09-19T05:08:02.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.352+0000 7fb1bbbe2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b41a9550 con 0x55cc122bc590 2024-09-19T05:08:02.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.352+0000 7fb1bbbe2740 10 client.0 _ll_register_callbacks cb 0x7fb1a00043d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:02.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.353+0000 7fb1b2800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc122bbc70 0x55cc122dca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:02.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.353+0000 7fb1b2800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc122bbc70 0x55cc122dca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:44172/0 (socket says 172.21.15.43:44172) 2024-09-19T05:08:02.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.353+0000 7fb1b2800640 1 -- 172.21.15.43:0/798866657 learned_addr learned my addr 172.21.15.43:0/798866657 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:02.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.353+0000 7fb1b2800640 1 --2- 172.21.15.43:0/798866657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc122c6e50 0x55cc122da4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:02.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.353+0000 7fb1b1400640 1 --2- 172.21.15.43:0/798866657 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cc122bc590 0x55cc122d9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:02.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.354+0000 7fb1b2800640 1 -- 172.21.15.43:0/798866657 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cc122bc590 msgr2=0x55cc122d9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:02.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.354+0000 7fb1b2800640 1 --2- 172.21.15.43:0/798866657 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cc122bc590 0x55cc122d9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:02.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.354+0000 7fb1b2800640 1 -- 172.21.15.43:0/798866657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc122c6e50 msgr2=0x55cc122da4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:02.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.354+0000 7fb1b2800640 1 --2- 172.21.15.43:0/798866657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc122c6e50 0x55cc122da4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:02.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.354+0000 7fb1b2800640 1 -- 172.21.15.43:0/798866657 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc123406d0 con 0x55cc122bbc70 2024-09-19T05:08:02.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.354+0000 7fb1b2800640 1 --2- 172.21.15.43:0/798866657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc122bbc70 0x55cc122dca70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac043010 tx=0x7fb1ac043040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c259ea9a312a2bb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:02.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.354+0000 7fb193e00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:08:02.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.354+0000 7fb193e00640 1 -- 172.21.15.43:0/798866657 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac061060 con 0x55cc122bbc70 2024-09-19T05:08:02.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.355+0000 7fb1bbbe2740 1 -- 172.21.15.43:0/798866657 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1a0015070 con 0x55cc122bbc70 2024-09-19T05:08:02.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.355+0000 7fb193e00640 1 -- 172.21.15.43:0/798866657 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fb1ac053600 con 0x55cc122bbc70 2024-09-19T05:08:02.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.355+0000 7fb193e00640 1 -- 172.21.15.43:0/798866657 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb1ac04a040 con 0x55cc122bbc70 2024-09-19T05:08:02.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.355+0000 7fb1bbbe2740 10 client.25928 fetch_fsmap learned FSMap version 29 2024-09-19T05:08:02.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.355+0000 7fb1bbbe2740 1 -- 172.21.15.43:0/798866657 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cc123418e0 con 0x55cc122bbc70 2024-09-19T05:08:02.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.355+0000 7fb193e00640 1 -- 172.21.15.43:0/798866657 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1a0015070 con 0x55cc122bbc70 2024-09-19T05:08:02.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.356+0000 7fb193e00640 1 -- 172.21.15.43:0/798866657 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb1ac041480 con 0x55cc122bbc70 2024-09-19T05:08:02.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.356+0000 7fb1bbbe2740 10 client.25928 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:08:02.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.356+0000 7fb1bbbe2740 10 client.25928 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T05:08:02.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.356+0000 7fb1bbbe2740 20 client.25928 populate_metadata read hostname 'smithi043' 2024-09-19T05:08:02.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.356+0000 7fb1bbbe2740 1 --2- 172.21.15.43:0/798866657 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55cc122d6d90 0x55cc12343f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:02.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.356+0000 7fb1bbbe2740 4 client.25928 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:08:02.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.356+0000 7fb1bbbe2740 1 -- 172.21.15.43:0/798866657 --> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cc12229e60 con 0x55cc122d6d90 2024-09-19T05:08:02.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.356+0000 7fb1b1400640 1 --2- 172.21.15.43:0/798866657 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55cc122d6d90 0x55cc12343f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:02.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.357+0000 7fb1b1400640 1 --2- 172.21.15.43:0/798866657 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55cc122d6d90 0x55cc12343f20 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=237ae1aa2302716 server_cookie=a6583c97bc85c21c in_seq=0 out_seq=0 2024-09-19T05:08:02.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.357+0000 7fb193e00640 10 client.25928 ms_handle_connect on v2:172.21.15.151:6838/3794104313 2024-09-19T05:08:02.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.357+0000 7fb193e00640 1 -- 172.21.15.43:0/798866657 <== mds.2 v2:172.21.15.151:6838/3794104313 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cc12229e60 con 0x55cc122d6d90 2024-09-19T05:08:02.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.357+0000 7fb193e00640 10 client.25928 handle_command_reply: tid=0 2024-09-19T05:08:02.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.357+0000 7fb1bbbe2740 1 client.25928 shutdown 2024-09-19T05:08:02.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.358+0000 7fb1bbbe2740 1 -- 172.21.15.43:0/798866657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc122bbc70 msgr2=0x55cc122dca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:02.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.358+0000 7fb1bbbe2740 1 --2- 172.21.15.43:0/798866657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc122bbc70 0x55cc122dca70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac043010 tx=0x7fb1ac043040 comp rx=0 tx=0).stop 2024-09-19T05:08:02.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.358+0000 7fb1bbbe2740 1 -- 172.21.15.43:0/798866657 shutdown_connections 2024-09-19T05:08:02.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.358+0000 7fb1bbbe2740 1 --2- 172.21.15.43:0/798866657 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55cc122d6d90 0x55cc12343f20 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:02.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.358+0000 7fb1bbbe2740 1 --2- 172.21.15.43:0/798866657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc122c6e50 0x55cc122da4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:02.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.358+0000 7fb1bbbe2740 1 --2- 172.21.15.43:0/798866657 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cc122bc590 0x55cc122d9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:02.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.358+0000 7fb1bbbe2740 1 --2- 172.21.15.43:0/798866657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cc122bbc70 0x55cc122dca70 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:02.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.358+0000 7fb1bbbe2740 1 -- 172.21.15.43:0/798866657 >> 172.21.15.43:0/798866657 conn(0x55cc122b8200 msgr2=0x55cc122d1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:02.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.359+0000 7fb193e00640 0 client.25928 ms_handle_reset on v2:172.21.15.151:6838/3794104313 2024-09-19T05:08:02.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.359+0000 7fb1bbbe2740 1 -- 172.21.15.43:0/798866657 shutdown_connections 2024-09-19T05:08:02.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.359+0000 7fb1bbbe2740 1 -- 172.21.15.43:0/798866657 wait complete. 2024-09-19T05:08:02.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.363+0000 7fb1bbbe2740 20 client.25928 trim_cache size 0 max 16384 2024-09-19T05:08:02.365 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:08:02.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.365+0000 7fb1b9000640 1 -- 172.21.15.43:0/2331393816 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb1980a3430 msgr2=0x7fb1980a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:02.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.365+0000 7fb1b9000640 1 --2- 172.21.15.43:0/2331393816 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb1980a3430 0x7fb1980a58f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb1a40982d0 tx=0x7fb1a4007f00 comp rx=0 tx=0).stop 2024-09-19T05:08:02.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.365+0000 7fb1b9000640 1 -- 172.21.15.43:0/2331393816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1b4103ff0 msgr2=0x7fb1b419a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:02.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.365+0000 7fb1b9000640 1 --2- 172.21.15.43:0/2331393816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1b4103ff0 0x7fb1b419a530 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb1a400c970 tx=0x7fb1a400ce30 comp rx=0 tx=0).stop 2024-09-19T05:08:02.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.365+0000 7fb1b9000640 1 -- 172.21.15.43:0/2331393816 shutdown_connections 2024-09-19T05:08:02.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.365+0000 7fb1b9000640 1 --2- 172.21.15.43:0/2331393816 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1b4109120 0x7fb1b419ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:02.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.365+0000 7fb1b9000640 1 --2- 172.21.15.43:0/2331393816 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb1980a3430 0x7fb1980a58f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:02.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.365+0000 7fb1b9000640 1 --2- 172.21.15.43:0/2331393816 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1b41049a0 0x7fb1b419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:02.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.365+0000 7fb1b9000640 1 --2- 172.21.15.43:0/2331393816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1b4103ff0 0x7fb1b419a530 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:02.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.365+0000 7fb1b9000640 1 -- 172.21.15.43:0/2331393816 >> 172.21.15.43:0/2331393816 conn(0x7fb1b40ffa90 msgr2=0x7fb1b4100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:02.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.366+0000 7fb1b9000640 1 -- 172.21.15.43:0/2331393816 shutdown_connections 2024-09-19T05:08:02.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:02.366+0000 7fb1b9000640 1 -- 172.21.15.43:0/2331393816 wait complete. 2024-09-19T05:08:03.377 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T05:08:03.449 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-19T05:08:03.449 INFO:tasks.workunit.client.0.smithi043.stdout: Children see throughput for 1 rewriters = 305763.91 kB/sec 2024-09-19T05:08:03.449 INFO:tasks.workunit.client.0.smithi043.stdout: Parent sees throughput for 1 rewriters = 305757.80 kB/sec 2024-09-19T05:08:03.449 INFO:tasks.workunit.client.0.smithi043.stdout: Min throughput per process = 305763.91 kB/sec 2024-09-19T05:08:03.449 INFO:tasks.workunit.client.0.smithi043.stdout: Max throughput per process = 305763.91 kB/sec 2024-09-19T05:08:03.449 INFO:tasks.workunit.client.0.smithi043.stdout: Avg throughput per process = 305763.91 kB/sec 2024-09-19T05:08:03.449 INFO:tasks.workunit.client.0.smithi043.stdout: Min xfer = 10485760.00 kB 2024-09-19T05:08:03.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:08:03 smithi096 ceph-mon[35085]: pgmap v105: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 254 MiB/s wr, 72 op/s 2024-09-19T05:08:03.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:08:03 smithi043 ceph-mon[30800]: pgmap v105: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 254 MiB/s wr, 72 op/s 2024-09-19T05:08:03.787 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:08:03.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:08:03 smithi151 ceph-mon[35425]: pgmap v105: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 254 MiB/s wr, 72 op/s 2024-09-19T05:08:05.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.112+0000 7fee2ee00640 1 -- 172.21.15.43:0/363278297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee28109140 msgr2=0x7fee28110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:05.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.112+0000 7fee2ee00640 1 --2- 172.21.15.43:0/363278297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee28109140 0x7fee28110cc0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fee20009ca0 tx=0x7fee2002a8c0 comp rx=0 tx=0).stop 2024-09-19T05:08:05.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.112+0000 7fee2ee00640 1 -- 172.21.15.43:0/363278297 shutdown_connections 2024-09-19T05:08:05.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.112+0000 7fee2ee00640 1 --2- 172.21.15.43:0/363278297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee28109140 0x7fee28110cc0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:05.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.112+0000 7fee2ee00640 1 --2- 172.21.15.43:0/363278297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee281049c0 0x7fee28108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:05.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.112+0000 7fee2ee00640 1 --2- 172.21.15.43:0/363278297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee28104010 0x7fee281043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:05.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.112+0000 7fee2ee00640 1 -- 172.21.15.43:0/363278297 >> 172.21.15.43:0/363278297 conn(0x7fee280ffa50 msgr2=0x7fee28101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:05.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.112+0000 7fee2ee00640 1 -- 172.21.15.43:0/363278297 shutdown_connections 2024-09-19T05:08:05.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.112+0000 7fee2ee00640 1 -- 172.21.15.43:0/363278297 wait complete. 2024-09-19T05:08:05.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.113+0000 7fee2ee00640 1 Processor -- start 2024-09-19T05:08:05.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.114+0000 7fee2ee00640 1 -- start start 2024-09-19T05:08:05.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.114+0000 7fee2ee00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee28104010 0x7fee28076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:05.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.114+0000 7fee2ee00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee281049c0 0x7fee28076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:05.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.114+0000 7fee2ee00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee28109140 0x7fee28077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:05.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.114+0000 7fee2ee00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fee28114170 con 0x7fee28109140 2024-09-19T05:08:05.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.114+0000 7fee2ee00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fee28113ff0 con 0x7fee28104010 2024-09-19T05:08:05.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.114+0000 7fee2ee00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fee281142f0 con 0x7fee281049c0 2024-09-19T05:08:05.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.114+0000 7fee27e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee28104010 0x7fee28076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:05.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.114+0000 7fee27e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee28104010 0x7fee28076700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:44200/0 (socket says 172.21.15.43:44200) 2024-09-19T05:08:05.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.114+0000 7fee27e00640 1 -- 172.21.15.43:0/729048005 learned_addr learned my addr 172.21.15.43:0/729048005 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:05.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.114+0000 7fee2ce00640 1 --2- 172.21.15.43:0/729048005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee28109140 0x7fee28077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:05.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.114+0000 7fee27400640 1 --2- 172.21.15.43:0/729048005 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee281049c0 0x7fee28076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:05.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.115+0000 7fee27e00640 1 -- 172.21.15.43:0/729048005 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee281049c0 msgr2=0x7fee28076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:05.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.115+0000 7fee27e00640 1 --2- 172.21.15.43:0/729048005 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee281049c0 0x7fee28076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:05.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.115+0000 7fee27e00640 1 -- 172.21.15.43:0/729048005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee28109140 msgr2=0x7fee28077180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:05.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.115+0000 7fee27e00640 1 --2- 172.21.15.43:0/729048005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee28109140 0x7fee28077180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:05.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.115+0000 7fee27e00640 1 -- 172.21.15.43:0/729048005 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee281aacb0 con 0x7fee28104010 2024-09-19T05:08:05.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.115+0000 7fee27e00640 1 --2- 172.21.15.43:0/729048005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee28104010 0x7fee28076700 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fee1800ea50 tx=0x7fee1800ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:05.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.116+0000 7fee24c00640 1 -- 172.21.15.43:0/729048005 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1800cd60 con 0x7fee28104010 2024-09-19T05:08:05.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.116+0000 7fee24c00640 1 -- 172.21.15.43:0/729048005 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee18005500 con 0x7fee28104010 2024-09-19T05:08:05.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.116+0000 7fee2ee00640 1 -- 172.21.15.43:0/729048005 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee281aafa0 con 0x7fee28104010 2024-09-19T05:08:05.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.116+0000 7fee24c00640 1 -- 172.21.15.43:0/729048005 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee18010690 con 0x7fee28104010 2024-09-19T05:08:05.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.116+0000 7fee2ee00640 1 -- 172.21.15.43:0/729048005 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee281ab450 con 0x7fee28104010 2024-09-19T05:08:05.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.119+0000 7fee319dd740 1 Processor -- start 2024-09-19T05:08:05.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.119+0000 7fee319dd740 1 -- start start 2024-09-19T05:08:05.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.119+0000 7fee24c00640 1 -- 172.21.15.43:0/729048005 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fee18004f30 con 0x7fee28104010 2024-09-19T05:08:05.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.119+0000 7fee319dd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637e4d8cc70 0x5637e4d8d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:05.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.119+0000 7fee319dd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637e4d8d590 0x5637e4d97910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:05.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.119+0000 7fee319dd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e4d97e50 0x5637e4d9a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:05.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.119+0000 7fee319dd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5637e4cf33e0 con 0x5637e4d8d590 2024-09-19T05:08:05.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.119+0000 7fee319dd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5637e4c9ad80 con 0x5637e4d97e50 2024-09-19T05:08:05.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.119+0000 7fee319dd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5637e4cef3a0 con 0x5637e4d8cc70 2024-09-19T05:08:05.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.120+0000 7fee27e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637e4d8d590 0x5637e4d97910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:05.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.120+0000 7fee27e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637e4d8d590 0x5637e4d97910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45820/0 (socket says 172.21.15.43:45820) 2024-09-19T05:08:05.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.120+0000 7fee2ce00640 1 --2- 172.21.15.43:0/839169022 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637e4d8cc70 0x5637e4d8d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:05.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.120+0000 7fee27e00640 1 -- 172.21.15.43:0/839169022 learned_addr learned my addr 172.21.15.43:0/839169022 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:05.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.120+0000 7fee2ce00640 1 -- 172.21.15.43:0/839169022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e4d97e50 msgr2=0x5637e4d9a240 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:08:05.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.120+0000 7fee2ce00640 1 --2- 172.21.15.43:0/839169022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e4d97e50 0x5637e4d9a240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:05.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.120+0000 7fee2ce00640 1 -- 172.21.15.43:0/839169022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637e4d8d590 msgr2=0x5637e4d97910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:05.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.120+0000 7fee2ce00640 1 --2- 172.21.15.43:0/839169022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637e4d8d590 0x5637e4d97910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:05.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.120+0000 7fee2ce00640 1 -- 172.21.15.43:0/839169022 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637e4cb9ae0 con 0x5637e4d8cc70 2024-09-19T05:08:05.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.120+0000 7fee27e00640 1 --2- 172.21.15.43:0/839169022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637e4d8d590 0x5637e4d97910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:05.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.121+0000 7fee2ce00640 1 --2- 172.21.15.43:0/839169022 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637e4d8cc70 0x5637e4d8d050 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fee20006010 tx=0x7fee20005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bdaf9d6fb2a28b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:05.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.121+0000 7fee24c00640 1 --2- 172.21.15.43:0/729048005 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fee0c0a3430 0x7fee0c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:05.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.121+0000 7fee1d600640 1 -- 172.21.15.43:0/839169022 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee20031db0 con 0x5637e4d8cc70 2024-09-19T05:08:05.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.121+0000 7fee24c00640 1 -- 172.21.15.43:0/729048005 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fee180cffe0 con 0x7fee28104010 2024-09-19T05:08:05.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.121+0000 7fee1d600640 1 -- 172.21.15.43:0/839169022 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee20036040 con 0x5637e4d8cc70 2024-09-19T05:08:05.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.121+0000 7fee1d600640 1 -- 172.21.15.43:0/839169022 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee200042a0 con 0x5637e4d8cc70 2024-09-19T05:08:05.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.121+0000 7fee27e00640 1 --2- 172.21.15.43:0/729048005 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fee0c0a3430 0x7fee0c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:05.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.122+0000 7fee319dd740 1 -- 172.21.15.43:0/839169022 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637e4d8cc70 msgr2=0x5637e4d8d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:05.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.122+0000 7fee27e00640 1 --2- 172.21.15.43:0/729048005 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fee0c0a3430 0x7fee0c0a58f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fee1800ea20 tx=0x7fee180d3ab0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:05.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.122+0000 7fee319dd740 1 --2- 172.21.15.43:0/839169022 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637e4d8cc70 0x5637e4d8d050 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fee20006010 tx=0x7fee20005f10 comp rx=0 tx=0).stop 2024-09-19T05:08:05.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.122+0000 7fee319dd740 1 -- 172.21.15.43:0/839169022 shutdown_connections 2024-09-19T05:08:05.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.122+0000 7fee319dd740 1 --2- 172.21.15.43:0/839169022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e4d97e50 0x5637e4d9a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:05.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.122+0000 7fee319dd740 1 --2- 172.21.15.43:0/839169022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637e4d8d590 0x5637e4d97910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:05.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.122+0000 7fee319dd740 1 --2- 172.21.15.43:0/839169022 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637e4d8cc70 0x5637e4d8d050 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:05.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.122+0000 7fee319dd740 1 -- 172.21.15.43:0/839169022 >> 172.21.15.43:0/839169022 conn(0x5637e4d89200 msgr2=0x5637e4d895e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:05.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.122+0000 7fee319dd740 1 -- 172.21.15.43:0/839169022 shutdown_connections 2024-09-19T05:08:05.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.122+0000 7fee319dd740 1 -- 172.21.15.43:0/839169022 wait complete. 2024-09-19T05:08:05.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.123+0000 7fee319dd740 1 -- start start 2024-09-19T05:08:05.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.123+0000 7fee319dd740 1 Processor -- start 2024-09-19T05:08:05.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.124+0000 7fee319dd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e4d8cc70 0x5637e4d94db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:05.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.124+0000 7fee319dd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637e4d8d590 0x5637e4d952f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:05.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.124+0000 7fee319dd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637e4d97e50 0x5637e4d95830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:05.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.124+0000 7fee319dd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5637e4d9a8a0 con 0x5637e4d97e50 2024-09-19T05:08:05.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.124+0000 7fee319dd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5637e4cae0f0 con 0x5637e4d8cc70 2024-09-19T05:08:05.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.124+0000 7fee319dd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5637e4d9aa20 con 0x5637e4d8d590 2024-09-19T05:08:05.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.124+0000 7fee319dd740 10 client.0 _ll_register_callbacks cb 0x5637e4c95bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:05.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.124+0000 7fee2ce00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e4d8cc70 0x5637e4d94db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:05.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.124+0000 7fee27400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637e4d8d590 0x5637e4d952f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:05.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.124+0000 7fee2ce00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e4d8cc70 0x5637e4d94db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:44226/0 (socket says 172.21.15.43:44226) 2024-09-19T05:08:05.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.124+0000 7fee2ce00640 1 -- 172.21.15.43:0/3577549738 learned_addr learned my addr 172.21.15.43:0/3577549738 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:05.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.124+0000 7fee2ce00640 1 --2- 172.21.15.43:0/3577549738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637e4d97e50 0x5637e4d95830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:05.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.124+0000 7fee27400640 1 -- 172.21.15.43:0/3577549738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e4d8cc70 msgr2=0x5637e4d94db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:05.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.124+0000 7fee27400640 1 --2- 172.21.15.43:0/3577549738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e4d8cc70 0x5637e4d94db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:05.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.125+0000 7fee27400640 1 -- 172.21.15.43:0/3577549738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637e4d97e50 msgr2=0x5637e4d95830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:05.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.125+0000 7fee27400640 1 --2- 172.21.15.43:0/3577549738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637e4d97e50 0x5637e4d95830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:05.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.125+0000 7fee27400640 1 -- 172.21.15.43:0/3577549738 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637e4a643c0 con 0x5637e4d8d590 2024-09-19T05:08:05.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.125+0000 7fee2ce00640 1 --2- 172.21.15.43:0/3577549738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e4d8cc70 0x5637e4d94db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:05.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.125+0000 7fee27400640 1 --2- 172.21.15.43:0/3577549738 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637e4d8d590 0x5637e4d952f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fee28077f50 tx=0x7fee14011b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=766e88274ba153a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:05.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.125+0000 7fee12000640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:08:05.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.125+0000 7fee12000640 1 -- 172.21.15.43:0/3577549738 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee14004460 con 0x5637e4d8d590 2024-09-19T05:08:05.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.125+0000 7fee12000640 1 -- 172.21.15.43:0/3577549738 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fee1400f920 con 0x5637e4d8d590 2024-09-19T05:08:05.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.125+0000 7fee319dd740 1 -- 172.21.15.43:0/3577549738 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee20036040 con 0x5637e4d8d590 2024-09-19T05:08:05.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.126+0000 7fee12000640 1 -- 172.21.15.43:0/3577549738 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fee1400fb20 con 0x5637e4d8d590 2024-09-19T05:08:05.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.126+0000 7fee319dd740 10 client.25882 fetch_fsmap learned FSMap version 29 2024-09-19T05:08:05.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.126+0000 7fee319dd740 1 -- 172.21.15.43:0/3577549738 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5637e4da6a10 con 0x5637e4d8d590 2024-09-19T05:08:05.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.126+0000 7fee12000640 1 -- 172.21.15.43:0/3577549738 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee20036040 con 0x5637e4d8d590 2024-09-19T05:08:05.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.126+0000 7fee12000640 1 -- 172.21.15.43:0/3577549738 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fee14018070 con 0x5637e4d8d590 2024-09-19T05:08:05.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.126+0000 7fee319dd740 10 client.25882 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:08:05.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.126+0000 7fee319dd740 10 client.25882 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-19T05:08:05.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.126+0000 7fee319dd740 20 client.25882 populate_metadata read hostname 'smithi043' 2024-09-19T05:08:05.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.126+0000 7fee319dd740 1 --2- 172.21.15.43:0/3577549738 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5637e4da7f70 0x5637e4daa390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:05.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.126+0000 7fee319dd740 4 client.25882 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:08:05.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.126+0000 7fee319dd740 1 -- 172.21.15.43:0/3577549738 --> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637e4d00950 con 0x5637e4da7f70 2024-09-19T05:08:05.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.127+0000 7fee2ce00640 1 --2- 172.21.15.43:0/3577549738 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5637e4da7f70 0x5637e4daa390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:05.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.127+0000 7fee2ce00640 1 --2- 172.21.15.43:0/3577549738 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5637e4da7f70 0x5637e4daa390 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5b156c30ca8fbd31 server_cookie=4293436d1a138f65 in_seq=0 out_seq=0 2024-09-19T05:08:05.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.127+0000 7fee12000640 10 client.25882 ms_handle_connect on v2:172.21.15.43:6838/3404555030 2024-09-19T05:08:05.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.128+0000 7fee12000640 1 -- 172.21.15.43:0/3577549738 <== mds.1 v2:172.21.15.43:6838/3404555030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5637e4d00950 con 0x5637e4da7f70 2024-09-19T05:08:05.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.128+0000 7fee12000640 10 client.25882 handle_command_reply: tid=0 2024-09-19T05:08:05.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.128+0000 7fee319dd740 1 client.25882 shutdown 2024-09-19T05:08:05.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.128+0000 7fee319dd740 1 -- 172.21.15.43:0/3577549738 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637e4d8d590 msgr2=0x5637e4d952f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:05.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.128+0000 7fee319dd740 1 --2- 172.21.15.43:0/3577549738 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637e4d8d590 0x5637e4d952f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fee28077f50 tx=0x7fee14011b50 comp rx=0 tx=0).stop 2024-09-19T05:08:05.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.128+0000 7fee319dd740 1 -- 172.21.15.43:0/3577549738 shutdown_connections 2024-09-19T05:08:05.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.128+0000 7fee319dd740 1 --2- 172.21.15.43:0/3577549738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637e4d97e50 0x5637e4d95830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:05.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.128+0000 7fee319dd740 1 --2- 172.21.15.43:0/3577549738 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5637e4da7f70 0x5637e4daa390 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:05.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.128+0000 7fee319dd740 1 --2- 172.21.15.43:0/3577549738 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637e4d8d590 0x5637e4d952f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:05.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.128+0000 7fee319dd740 1 --2- 172.21.15.43:0/3577549738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e4d8cc70 0x5637e4d94db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:05.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.128+0000 7fee319dd740 1 -- 172.21.15.43:0/3577549738 >> 172.21.15.43:0/3577549738 conn(0x5637e4d89200 msgr2=0x5637e4d8ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:05.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.128+0000 7fee12000640 0 client.25882 ms_handle_reset on v2:172.21.15.43:6838/3404555030 2024-09-19T05:08:05.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.129+0000 7fee319dd740 1 -- 172.21.15.43:0/3577549738 shutdown_connections 2024-09-19T05:08:05.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.129+0000 7fee319dd740 1 -- 172.21.15.43:0/3577549738 wait complete. 2024-09-19T05:08:05.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.129+0000 7fee319dd740 20 client.25882 trim_cache size 0 max 16384 2024-09-19T05:08:05.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.142+0000 7fee319dd740 1 Processor -- start 2024-09-19T05:08:05.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.143+0000 7fee319dd740 1 -- start start 2024-09-19T05:08:05.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.143+0000 7fee319dd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637e4d8cc70 0x5637e4d901d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:05.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.143+0000 7fee319dd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637e4d8d590 0x5637e4dafc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:05.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.143+0000 7fee319dd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e4d97e50 0x5637e4db0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:05.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.143+0000 7fee319dd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fee281b3dd0 con 0x5637e4d8d590 2024-09-19T05:08:05.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.143+0000 7fee319dd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fee281b5910 con 0x5637e4d97e50 2024-09-19T05:08:05.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.143+0000 7fee319dd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fee281b44a0 con 0x5637e4d8cc70 2024-09-19T05:08:05.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.143+0000 7fee27400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637e4d8d590 0x5637e4dafc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:05.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.143+0000 7fee2ce00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637e4d8cc70 0x5637e4d901d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:05.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.143+0000 7fee2ce00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637e4d8cc70 0x5637e4d901d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:58988/0 (socket says 172.21.15.43:58988) 2024-09-19T05:08:05.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.143+0000 7fee2ce00640 1 -- 172.21.15.43:0/2541398177 learned_addr learned my addr 172.21.15.43:0/2541398177 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:05.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.143+0000 7fee2ce00640 1 --2- 172.21.15.43:0/2541398177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e4d97e50 0x5637e4db0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:05.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.144+0000 7fee2ce00640 1 -- 172.21.15.43:0/2541398177 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637e4d8cc70 msgr2=0x5637e4d901d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:05.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.144+0000 7fee2ce00640 1 --2- 172.21.15.43:0/2541398177 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637e4d8cc70 0x5637e4d901d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:05.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.144+0000 7fee2ce00640 1 -- 172.21.15.43:0/2541398177 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637e4d8d590 msgr2=0x5637e4dafc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:05.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.144+0000 7fee2ce00640 1 --2- 172.21.15.43:0/2541398177 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637e4d8d590 0x5637e4dafc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:05.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.144+0000 7fee2ce00640 1 -- 172.21.15.43:0/2541398177 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637e4d8caa0 con 0x5637e4d97e50 2024-09-19T05:08:05.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.144+0000 7fee27400640 1 --2- 172.21.15.43:0/2541398177 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637e4d8d590 0x5637e4dafc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:05.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.144+0000 7fee2ce00640 1 --2- 172.21.15.43:0/2541398177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e4d97e50 0x5637e4db0180 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fee20031950 tx=0x7fee2002a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19c11f57c4dd7382 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:05.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.144+0000 7fee1cc00640 1 -- 172.21.15.43:0/2541398177 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2002f610 con 0x5637e4d97e50 2024-09-19T05:08:05.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.144+0000 7fee1cc00640 1 -- 172.21.15.43:0/2541398177 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee20036040 con 0x5637e4d97e50 2024-09-19T05:08:05.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.144+0000 7fee1cc00640 1 -- 172.21.15.43:0/2541398177 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee20030bc0 con 0x5637e4d97e50 2024-09-19T05:08:05.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.145+0000 7fee319dd740 1 -- 172.21.15.43:0/2541398177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e4d97e50 msgr2=0x5637e4db0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:05.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.145+0000 7fee319dd740 1 --2- 172.21.15.43:0/2541398177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e4d97e50 0x5637e4db0180 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fee20031950 tx=0x7fee2002a8c0 comp rx=0 tx=0).stop 2024-09-19T05:08:05.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.145+0000 7fee319dd740 1 -- 172.21.15.43:0/2541398177 shutdown_connections 2024-09-19T05:08:05.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.145+0000 7fee319dd740 1 --2- 172.21.15.43:0/2541398177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e4d97e50 0x5637e4db0180 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:05.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.145+0000 7fee319dd740 1 --2- 172.21.15.43:0/2541398177 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637e4d8d590 0x5637e4dafc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:05.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.145+0000 7fee319dd740 1 --2- 172.21.15.43:0/2541398177 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637e4d8cc70 0x5637e4d901d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:05.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.145+0000 7fee319dd740 1 -- 172.21.15.43:0/2541398177 >> 172.21.15.43:0/2541398177 conn(0x5637e4d89200 msgr2=0x5637e4d93ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:05.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.145+0000 7fee319dd740 1 -- 172.21.15.43:0/2541398177 shutdown_connections 2024-09-19T05:08:05.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.145+0000 7fee319dd740 1 -- 172.21.15.43:0/2541398177 wait complete. 2024-09-19T05:08:05.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.146+0000 7fee319dd740 1 -- start start 2024-09-19T05:08:05.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.146+0000 7fee319dd740 1 Processor -- start 2024-09-19T05:08:05.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.147+0000 7fee319dd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637e4d8cc70 0x5637e4da02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:05.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.147+0000 7fee319dd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e4d8d590 0x5637e4d9b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:05.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.147+0000 7fee319dd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637e4d97e50 0x5637e4d9bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:05.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.147+0000 7fee319dd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fee281b5240 con 0x5637e4d8cc70 2024-09-19T05:08:05.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.147+0000 7fee319dd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fee281b4b70 con 0x5637e4d8d590 2024-09-19T05:08:05.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.147+0000 7fee319dd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fee281b1db0 con 0x5637e4d97e50 2024-09-19T05:08:05.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.147+0000 7fee319dd740 10 client.0 _ll_register_callbacks cb 0x7fee140097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:05.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.147+0000 7fee2ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637e4d8cc70 0x5637e4da02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:05.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.147+0000 7fee2ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637e4d8cc70 0x5637e4da02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45842/0 (socket says 172.21.15.43:45842) 2024-09-19T05:08:05.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.147+0000 7fee2ce00640 1 -- 172.21.15.43:0/2198755361 learned_addr learned my addr 172.21.15.43:0/2198755361 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:05.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.147+0000 7fee27400640 1 --2- 172.21.15.43:0/2198755361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e4d8d590 0x5637e4d9b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:05.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.147+0000 7fee2ce00640 1 --2- 172.21.15.43:0/2198755361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637e4d97e50 0x5637e4d9bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:05.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.148+0000 7fee2ce00640 1 -- 172.21.15.43:0/2198755361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637e4d97e50 msgr2=0x5637e4d9bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:05.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.148+0000 7fee2ce00640 1 --2- 172.21.15.43:0/2198755361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637e4d97e50 0x5637e4d9bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:05.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.148+0000 7fee2ce00640 1 -- 172.21.15.43:0/2198755361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e4d8d590 msgr2=0x5637e4d9b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:05.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.148+0000 7fee2ce00640 1 --2- 172.21.15.43:0/2198755361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e4d8d590 0x5637e4d9b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:05.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.148+0000 7fee2ce00640 1 -- 172.21.15.43:0/2198755361 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637e4d84770 con 0x5637e4d8cc70 2024-09-19T05:08:05.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.148+0000 7fee2ce00640 1 --2- 172.21.15.43:0/2198755361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637e4d8cc70 0x5637e4da02a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fee2002ade0 tx=0x7fee20031bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e5d8ce24623122f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:05.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.148+0000 7fee12000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:08:05.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.148+0000 7fee12000640 1 -- 172.21.15.43:0/2198755361 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee20003c30 con 0x5637e4d8cc70 2024-09-19T05:08:05.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.148+0000 7fee12000640 1 -- 172.21.15.43:0/2198755361 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fee20003dd0 con 0x5637e4d8cc70 2024-09-19T05:08:05.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.148+0000 7fee319dd740 1 -- 172.21.15.43:0/2198755361 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee20036040 con 0x5637e4d8cc70 2024-09-19T05:08:05.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.148+0000 7fee12000640 1 -- 172.21.15.43:0/2198755361 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fee20004b40 con 0x5637e4d8cc70 2024-09-19T05:08:05.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.148+0000 7fee319dd740 10 client.15903 fetch_fsmap learned FSMap version 29 2024-09-19T05:08:05.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.148+0000 7fee319dd740 1 -- 172.21.15.43:0/2198755361 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5637e4e12920 con 0x5637e4d8cc70 2024-09-19T05:08:05.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.149+0000 7fee12000640 1 -- 172.21.15.43:0/2198755361 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee20036040 con 0x5637e4d8cc70 2024-09-19T05:08:05.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.149+0000 7fee12000640 1 -- 172.21.15.43:0/2198755361 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fee20050040 con 0x5637e4d8cc70 2024-09-19T05:08:05.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.149+0000 7fee319dd740 10 client.15903 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:08:05.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.149+0000 7fee319dd740 10 client.15903 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-19T05:08:05.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.149+0000 7fee319dd740 20 client.15903 populate_metadata read hostname 'smithi043' 2024-09-19T05:08:05.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.149+0000 7fee319dd740 1 --2- 172.21.15.43:0/2198755361 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5637e4e12cb0 0x5637e4e150a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:05.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.149+0000 7fee319dd740 4 client.15903 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:08:05.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.149+0000 7fee319dd740 1 -- 172.21.15.43:0/2198755361 --> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5637e4cfae60 con 0x5637e4e12cb0 2024-09-19T05:08:05.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.149+0000 7fee27400640 1 --2- 172.21.15.43:0/2198755361 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5637e4e12cb0 0x5637e4e150a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:05.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.150+0000 7fee27400640 1 --2- 172.21.15.43:0/2198755361 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5637e4e12cb0 0x5637e4e150a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b63e9e0638f9c982 server_cookie=a6c9b8166f857247 in_seq=0 out_seq=0 2024-09-19T05:08:05.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.150+0000 7fee12000640 10 client.15903 ms_handle_connect on v2:172.21.15.43:6838/3404555030 2024-09-19T05:08:05.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.150+0000 7fee12000640 1 -- 172.21.15.43:0/2198755361 <== mds.1 v2:172.21.15.43:6838/3404555030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5637e4cfae60 con 0x5637e4e12cb0 2024-09-19T05:08:05.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.150+0000 7fee12000640 10 client.15903 handle_command_reply: tid=0 2024-09-19T05:08:05.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.150+0000 7fee319dd740 1 client.15903 shutdown 2024-09-19T05:08:05.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.150+0000 7fee319dd740 1 -- 172.21.15.43:0/2198755361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637e4d8cc70 msgr2=0x5637e4da02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:05.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.150+0000 7fee319dd740 1 --2- 172.21.15.43:0/2198755361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637e4d8cc70 0x5637e4da02a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fee2002ade0 tx=0x7fee20031bb0 comp rx=0 tx=0).stop 2024-09-19T05:08:05.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.150+0000 7fee319dd740 1 -- 172.21.15.43:0/2198755361 shutdown_connections 2024-09-19T05:08:05.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.150+0000 7fee319dd740 1 --2- 172.21.15.43:0/2198755361 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5637e4e12cb0 0x5637e4e150a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:05.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.151+0000 7fee319dd740 1 --2- 172.21.15.43:0/2198755361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637e4d97e50 0x5637e4d9bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:05.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.151+0000 7fee319dd740 1 --2- 172.21.15.43:0/2198755361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637e4d8d590 0x5637e4d9b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:05.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.151+0000 7fee319dd740 1 --2- 172.21.15.43:0/2198755361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5637e4d8cc70 0x5637e4da02a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:05.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.151+0000 7fee319dd740 1 -- 172.21.15.43:0/2198755361 >> 172.21.15.43:0/2198755361 conn(0x5637e4d89200 msgr2=0x5637e4d87b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:05.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.151+0000 7fee12000640 0 client.15903 ms_handle_reset on v2:172.21.15.43:6838/3404555030 2024-09-19T05:08:05.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.151+0000 7fee319dd740 1 -- 172.21.15.43:0/2198755361 shutdown_connections 2024-09-19T05:08:05.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.151+0000 7fee319dd740 1 -- 172.21.15.43:0/2198755361 wait complete. 2024-09-19T05:08:05.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.151+0000 7fee319dd740 20 client.15903 trim_cache size 0 max 16384 2024-09-19T05:08:05.157 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:08:05.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.151+0000 7fee2ee00640 1 -- 172.21.15.43:0/729048005 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fee0c0a3430 msgr2=0x7fee0c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:05.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.151+0000 7fee2ee00640 1 --2- 172.21.15.43:0/729048005 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fee0c0a3430 0x7fee0c0a58f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fee1800ea20 tx=0x7fee180d3ab0 comp rx=0 tx=0).stop 2024-09-19T05:08:05.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.151+0000 7fee2ee00640 1 -- 172.21.15.43:0/729048005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee28104010 msgr2=0x7fee28076700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:05.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.151+0000 7fee2ee00640 1 --2- 172.21.15.43:0/729048005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee28104010 0x7fee28076700 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fee1800ea50 tx=0x7fee1800ee50 comp rx=0 tx=0).stop 2024-09-19T05:08:05.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.152+0000 7fee2ee00640 1 -- 172.21.15.43:0/729048005 shutdown_connections 2024-09-19T05:08:05.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.152+0000 7fee2ee00640 1 --2- 172.21.15.43:0/729048005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee28109140 0x7fee28077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:05.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.152+0000 7fee2ee00640 1 --2- 172.21.15.43:0/729048005 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee281049c0 0x7fee28076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:05.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.152+0000 7fee2ee00640 1 --2- 172.21.15.43:0/729048005 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fee0c0a3430 0x7fee0c0a58f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:05.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.152+0000 7fee2ee00640 1 --2- 172.21.15.43:0/729048005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee28104010 0x7fee28076700 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:05.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.152+0000 7fee2ee00640 1 -- 172.21.15.43:0/729048005 >> 172.21.15.43:0/729048005 conn(0x7fee280ffa50 msgr2=0x7fee28100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:05.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.152+0000 7fee2ee00640 1 -- 172.21.15.43:0/729048005 shutdown_connections 2024-09-19T05:08:05.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:05.152+0000 7fee2ee00640 1 -- 172.21.15.43:0/729048005 wait complete. 2024-09-19T05:08:05.684 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:08:05 smithi043 ceph-mon[30800]: pgmap v106: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 375 MiB/s wr, 121 op/s 2024-09-19T05:08:05.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:08:05 smithi096 ceph-mon[35085]: pgmap v106: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 375 MiB/s wr, 121 op/s 2024-09-19T05:08:05.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:08:05 smithi151 ceph-mon[35425]: pgmap v106: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 375 MiB/s wr, 121 op/s 2024-09-19T05:08:06.071 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T05:08:06.452 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:08:07.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.064+0000 7f97b2200640 1 -- 172.21.15.43:0/4013972783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97ac10e420 msgr2=0x7f97ac110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:07.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.064+0000 7f97b2200640 1 --2- 172.21.15.43:0/4013972783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97ac10e420 0x7f97ac110810 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f97a4009ca0 tx=0x7f97a402a950 comp rx=0 tx=0).stop 2024-09-19T05:08:07.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.064+0000 7f97b2200640 1 -- 172.21.15.43:0/4013972783 shutdown_connections 2024-09-19T05:08:07.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.064+0000 7f97b2200640 1 --2- 172.21.15.43:0/4013972783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97ac10e420 0x7f97ac110810 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:07.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.064+0000 7f97b2200640 1 --2- 172.21.15.43:0/4013972783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97ac074ee0 0x7f97ac10dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:07.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.064+0000 7f97b2200640 1 --2- 172.21.15.43:0/4013972783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97ac0745c0 0x7f97ac0749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:07.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.064+0000 7f97b2200640 1 -- 172.21.15.43:0/4013972783 >> 172.21.15.43:0/4013972783 conn(0x7f97ac0fd8e0 msgr2=0x7f97ac0ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:07.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.064+0000 7f97b2200640 1 -- 172.21.15.43:0/4013972783 shutdown_connections 2024-09-19T05:08:07.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.065+0000 7f97b2200640 1 -- 172.21.15.43:0/4013972783 wait complete. 2024-09-19T05:08:07.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.066+0000 7f97b2200640 1 Processor -- start 2024-09-19T05:08:07.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.066+0000 7f97b2200640 1 -- start start 2024-09-19T05:08:07.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.067+0000 7f97b2200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97ac0745c0 0x7f97ac19a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:07.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.067+0000 7f97b2200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97ac074ee0 0x7f97ac19abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:07.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.067+0000 7f97b2200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97ac10e420 0x7f97ac19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:07.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.067+0000 7f97b2200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f97ac1141c0 con 0x7f97ac10e420 2024-09-19T05:08:07.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.067+0000 7f97b2200640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f97ac114040 con 0x7f97ac0745c0 2024-09-19T05:08:07.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.067+0000 7f97b2200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f97ac114340 con 0x7f97ac074ee0 2024-09-19T05:08:07.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.067+0000 7f97ab400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97ac0745c0 0x7f97ac19a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:07.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.067+0000 7f97ab400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97ac0745c0 0x7f97ac19a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:52342/0 (socket says 172.21.15.43:52342) 2024-09-19T05:08:07.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.067+0000 7f97ab400640 1 -- 172.21.15.43:0/2009973008 learned_addr learned my addr 172.21.15.43:0/2009973008 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:07.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.067+0000 7f97aaa00640 1 --2- 172.21.15.43:0/2009973008 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97ac074ee0 0x7f97ac19abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:07.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.067+0000 7f97abe00640 1 --2- 172.21.15.43:0/2009973008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97ac10e420 0x7f97ac19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:07.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.067+0000 7f97ab400640 1 -- 172.21.15.43:0/2009973008 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97ac074ee0 msgr2=0x7f97ac19abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:07.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.067+0000 7f97ab400640 1 --2- 172.21.15.43:0/2009973008 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97ac074ee0 0x7f97ac19abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:07.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.067+0000 7f97ab400640 1 -- 172.21.15.43:0/2009973008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97ac10e420 msgr2=0x7f97ac19ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:07.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.067+0000 7f97ab400640 1 --2- 172.21.15.43:0/2009973008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97ac10e420 0x7f97ac19ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:07.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.067+0000 7f97ab400640 1 -- 172.21.15.43:0/2009973008 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97ac19f630 con 0x7f97ac0745c0 2024-09-19T05:08:07.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.068+0000 7f97ab400640 1 --2- 172.21.15.43:0/2009973008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97ac0745c0 0x7f97ac19a680 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f979c00e9f0 tx=0x7f979c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:07.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.068+0000 7f97a3e00640 1 -- 172.21.15.43:0/2009973008 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c00cd80 con 0x7f97ac0745c0 2024-09-19T05:08:07.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.068+0000 7f97a3e00640 1 -- 172.21.15.43:0/2009973008 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f979c010910 con 0x7f97ac0745c0 2024-09-19T05:08:07.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.068+0000 7f97a3e00640 1 -- 172.21.15.43:0/2009973008 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c018990 con 0x7f97ac0745c0 2024-09-19T05:08:07.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.068+0000 7f97b2200640 1 -- 172.21.15.43:0/2009973008 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97ac19f920 con 0x7f97ac0745c0 2024-09-19T05:08:07.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.068+0000 7f97b2200640 1 -- 172.21.15.43:0/2009973008 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97ac1a7200 con 0x7f97ac0745c0 2024-09-19T05:08:07.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.072+0000 7f97b4dfe740 1 Processor -- start 2024-09-19T05:08:07.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.072+0000 7f97b4dfe740 1 -- start start 2024-09-19T05:08:07.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.072+0000 7f97a3e00640 1 -- 172.21.15.43:0/2009973008 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f979c010ac0 con 0x7f97ac0745c0 2024-09-19T05:08:07.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.072+0000 7f97b4dfe740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569fd1bbc70 0x5569fd1bc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:07.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.072+0000 7f97b4dfe740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569fd1bc590 0x5569fd1c6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:07.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.072+0000 7f97b4dfe740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569fd1c6e50 0x5569fd1c9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:07.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.072+0000 7f97b4dfe740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5569fd1223e0 con 0x5569fd1bc590 2024-09-19T05:08:07.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.072+0000 7f97b4dfe740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5569fd0c9d80 con 0x5569fd1bbc70 2024-09-19T05:08:07.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.072+0000 7f97b4dfe740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5569fd11e3a0 con 0x5569fd1c6e50 2024-09-19T05:08:07.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.073+0000 7f97ab400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569fd1bc590 0x5569fd1c6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:07.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.073+0000 7f97abe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569fd1bbc70 0x5569fd1bc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:07.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.073+0000 7f97aaa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569fd1c6e50 0x5569fd1c9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:07.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.073+0000 7f97abe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569fd1bbc70 0x5569fd1bc050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:52346/0 (socket says 172.21.15.43:52346) 2024-09-19T05:08:07.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.073+0000 7f97abe00640 1 -- 172.21.15.43:0/2119118150 learned_addr learned my addr 172.21.15.43:0/2119118150 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:07.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.073+0000 7f97abe00640 1 -- 172.21.15.43:0/2119118150 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569fd1c6e50 msgr2=0x5569fd1c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:07.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.073+0000 7f97abe00640 1 --2- 172.21.15.43:0/2119118150 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569fd1c6e50 0x5569fd1c9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:07.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.073+0000 7f97abe00640 1 -- 172.21.15.43:0/2119118150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569fd1bc590 msgr2=0x5569fd1c6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:07.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.073+0000 7f97abe00640 1 --2- 172.21.15.43:0/2119118150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569fd1bc590 0x5569fd1c6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:07.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.073+0000 7f97abe00640 1 -- 172.21.15.43:0/2119118150 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569fd0e8ae0 con 0x5569fd1bbc70 2024-09-19T05:08:07.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.074+0000 7f97abe00640 1 --2- 172.21.15.43:0/2119118150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569fd1bbc70 0x5569fd1bc050 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f97a4009ca0 tx=0x7f97a4009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef4863f036147af4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:07.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.074+0000 7f97a3e00640 1 --2- 172.21.15.43:0/2009973008 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f97900a33b0 0x7f97900a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:07.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.074+0000 7f9797e00640 1 -- 172.21.15.43:0/2119118150 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a4034040 con 0x5569fd1bbc70 2024-09-19T05:08:07.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.074+0000 7f9797e00640 1 -- 172.21.15.43:0/2119118150 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97a4003e50 con 0x5569fd1bbc70 2024-09-19T05:08:07.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.074+0000 7f9797e00640 1 -- 172.21.15.43:0/2119118150 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a4004330 con 0x5569fd1bbc70 2024-09-19T05:08:07.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.075+0000 7f97a3e00640 1 -- 172.21.15.43:0/2009973008 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f979c0d0170 con 0x7f97ac0745c0 2024-09-19T05:08:07.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.075+0000 7f97ab400640 1 --2- 172.21.15.43:0/2009973008 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f97900a33b0 0x7f97900a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:07.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.075+0000 7f97b4dfe740 1 -- 172.21.15.43:0/2119118150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569fd1bbc70 msgr2=0x5569fd1bc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:07.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.075+0000 7f97b4dfe740 1 --2- 172.21.15.43:0/2119118150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569fd1bbc70 0x5569fd1bc050 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f97a4009ca0 tx=0x7f97a4009a20 comp rx=0 tx=0).stop 2024-09-19T05:08:07.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.075+0000 7f97ab400640 1 --2- 172.21.15.43:0/2009973008 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f97900a33b0 0x7f97900a5870 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f979c0cd980 tx=0x7f979c0d3cc0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:07.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.075+0000 7f97b4dfe740 1 -- 172.21.15.43:0/2119118150 shutdown_connections 2024-09-19T05:08:07.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.075+0000 7f97b4dfe740 1 --2- 172.21.15.43:0/2119118150 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569fd1c6e50 0x5569fd1c9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:07.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.075+0000 7f97b4dfe740 1 --2- 172.21.15.43:0/2119118150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569fd1bc590 0x5569fd1c6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:07.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.075+0000 7f97b4dfe740 1 --2- 172.21.15.43:0/2119118150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569fd1bbc70 0x5569fd1bc050 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:07.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.075+0000 7f97b4dfe740 1 -- 172.21.15.43:0/2119118150 >> 172.21.15.43:0/2119118150 conn(0x5569fd1b8200 msgr2=0x5569fd1b85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:07.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.076+0000 7f97b4dfe740 1 -- 172.21.15.43:0/2119118150 shutdown_connections 2024-09-19T05:08:07.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.076+0000 7f97b4dfe740 1 -- 172.21.15.43:0/2119118150 wait complete. 2024-09-19T05:08:07.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.076+0000 7f97b4dfe740 1 -- start start 2024-09-19T05:08:07.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.076+0000 7f97b4dfe740 1 Processor -- start 2024-09-19T05:08:07.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.077+0000 7f97b4dfe740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569fd1bbc70 0x5569fd1c3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:07.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.077+0000 7f97b4dfe740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569fd1bc590 0x5569fd1c42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:07.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.077+0000 7f97b4dfe740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569fd1c6e50 0x5569fd1c47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:07.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.077+0000 7f97b4dfe740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5569fd1c9a20 con 0x5569fd1bbc70 2024-09-19T05:08:07.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.077+0000 7f97b4dfe740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5569fd0dd0f0 con 0x5569fd1bc590 2024-09-19T05:08:07.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.077+0000 7f97b4dfe740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5569fd1c9ba0 con 0x5569fd1c6e50 2024-09-19T05:08:07.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.077+0000 7f97b4dfe740 10 client.0 _ll_register_callbacks cb 0x5569fd0c4bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:07.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.077+0000 7f97abe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569fd1bbc70 0x5569fd1c3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:07.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.077+0000 7f97aaa00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569fd1bc590 0x5569fd1c42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:07.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.077+0000 7f97aaa00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569fd1bc590 0x5569fd1c42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:52356/0 (socket says 172.21.15.43:52356) 2024-09-19T05:08:07.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.077+0000 7f97aaa00640 1 -- 172.21.15.43:0/2648441742 learned_addr learned my addr 172.21.15.43:0/2648441742 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:07.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.077+0000 7f97abe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569fd1bbc70 0x5569fd1c3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56850/0 (socket says 172.21.15.43:56850) 2024-09-19T05:08:07.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.078+0000 7f97aaa00640 1 -- 172.21.15.43:0/2648441742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569fd1c6e50 msgr2=0x5569fd1c47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:07.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.078+0000 7f97aaa00640 1 --2- 172.21.15.43:0/2648441742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569fd1c6e50 0x5569fd1c47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:07.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.078+0000 7f97aaa00640 1 -- 172.21.15.43:0/2648441742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569fd1bbc70 msgr2=0x5569fd1c3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:07.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.078+0000 7f97aaa00640 1 --2- 172.21.15.43:0/2648441742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569fd1bbc70 0x5569fd1c3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:07.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.078+0000 7f97aaa00640 1 -- 172.21.15.43:0/2648441742 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569fce933c0 con 0x5569fd1bc590 2024-09-19T05:08:07.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.078+0000 7f97aaa00640 1 --2- 172.21.15.43:0/2648441742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569fd1bc590 0x5569fd1c42b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9798012730 tx=0x7f9798012bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=280826693e0381b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:07.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.078+0000 7f9794c00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:08:07.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.078+0000 7f9794c00640 1 -- 172.21.15.43:0/2648441742 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9798004240 con 0x5569fd1bc590 2024-09-19T05:08:07.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.078+0000 7f9794c00640 1 -- 172.21.15.43:0/2648441742 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f97980043e0 con 0x5569fd1bc590 2024-09-19T05:08:07.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.078+0000 7f97b4dfe740 1 -- 172.21.15.43:0/2648441742 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97a4003e50 con 0x5569fd1bc590 2024-09-19T05:08:07.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.079+0000 7f9794c00640 1 -- 172.21.15.43:0/2648441742 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f97980119a0 con 0x5569fd1bc590 2024-09-19T05:08:07.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.079+0000 7f97b4dfe740 10 client.25955 fetch_fsmap learned FSMap version 29 2024-09-19T05:08:07.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.079+0000 7f97b4dfe740 1 -- 172.21.15.43:0/2648441742 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5569fd1d5af0 con 0x5569fd1bc590 2024-09-19T05:08:07.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.079+0000 7f9794c00640 1 -- 172.21.15.43:0/2648441742 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97a4003e50 con 0x5569fd1bc590 2024-09-19T05:08:07.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.079+0000 7f9794c00640 1 -- 172.21.15.43:0/2648441742 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9798025470 con 0x5569fd1bc590 2024-09-19T05:08:07.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.080+0000 7f97b4dfe740 10 client.25955 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:08:07.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.080+0000 7f97b4dfe740 10 client.25955 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:08:07.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.080+0000 7f97b4dfe740 20 client.25955 populate_metadata read hostname 'smithi043' 2024-09-19T05:08:07.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.080+0000 7f97b4dfe740 1 --2- 172.21.15.43:0/2648441742 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5569fd1d6f00 0x5569fd1d9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:07.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.080+0000 7f97b4dfe740 4 client.25955 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:08:07.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.080+0000 7f97b4dfe740 1 -- 172.21.15.43:0/2648441742 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569fd12f950 con 0x5569fd1d6f00 2024-09-19T05:08:07.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.080+0000 7f97abe00640 1 --2- 172.21.15.43:0/2648441742 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5569fd1d6f00 0x5569fd1d9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:07.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.081+0000 7f97abe00640 1 --2- 172.21.15.43:0/2648441742 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5569fd1d6f00 0x5569fd1d9310 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbd2cd0629dc653b server_cookie=af7328e0253731e8 in_seq=0 out_seq=0 2024-09-19T05:08:07.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.081+0000 7f9794c00640 10 client.25955 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:08:07.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.082+0000 7f9794c00640 1 -- 172.21.15.43:0/2648441742 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5569fd12f950 con 0x5569fd1d6f00 2024-09-19T05:08:07.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.082+0000 7f9794c00640 10 client.25955 handle_command_reply: tid=0 2024-09-19T05:08:07.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.082+0000 7f97b4dfe740 1 client.25955 shutdown 2024-09-19T05:08:07.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.082+0000 7f97b4dfe740 1 -- 172.21.15.43:0/2648441742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569fd1bc590 msgr2=0x5569fd1c42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:07.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.082+0000 7f97b4dfe740 1 --2- 172.21.15.43:0/2648441742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569fd1bc590 0x5569fd1c42b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9798012730 tx=0x7f9798012bf0 comp rx=0 tx=0).stop 2024-09-19T05:08:07.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.082+0000 7f97b4dfe740 1 -- 172.21.15.43:0/2648441742 shutdown_connections 2024-09-19T05:08:07.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.082+0000 7f97b4dfe740 1 --2- 172.21.15.43:0/2648441742 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5569fd1d6f00 0x5569fd1d9310 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:07.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.082+0000 7f97b4dfe740 1 --2- 172.21.15.43:0/2648441742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569fd1c6e50 0x5569fd1c47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:07.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.082+0000 7f97b4dfe740 1 --2- 172.21.15.43:0/2648441742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569fd1bc590 0x5569fd1c42b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:07.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.082+0000 7f97b4dfe740 1 --2- 172.21.15.43:0/2648441742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569fd1bbc70 0x5569fd1c3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:07.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.082+0000 7f9794c00640 0 client.25955 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:08:07.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.082+0000 7f97b4dfe740 1 -- 172.21.15.43:0/2648441742 >> 172.21.15.43:0/2648441742 conn(0x5569fd1b8200 msgr2=0x5569fd1bdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:07.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.083+0000 7f97b4dfe740 1 -- 172.21.15.43:0/2648441742 shutdown_connections 2024-09-19T05:08:07.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.083+0000 7f97b4dfe740 1 -- 172.21.15.43:0/2648441742 wait complete. 2024-09-19T05:08:07.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.083+0000 7f97b4dfe740 20 client.25955 trim_cache size 0 max 16384 2024-09-19T05:08:07.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.097+0000 7f97b4dfe740 1 Processor -- start 2024-09-19T05:08:07.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.097+0000 7f97b4dfe740 1 -- start start 2024-09-19T05:08:07.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.098+0000 7f97b4dfe740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569fd1bbc70 0x5569fd1be720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:07.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.098+0000 7f97b4dfe740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569fd1bc590 0x5569fd1dec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:07.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.098+0000 7f97b4dfe740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569fd1c6e50 0x5569fd1df160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:07.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.098+0000 7f97b4dfe740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f97ac1afba0 con 0x5569fd1c6e50 2024-09-19T05:08:07.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.098+0000 7f97b4dfe740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f97ac1b16e0 con 0x5569fd1bbc70 2024-09-19T05:08:07.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.098+0000 7f97b4dfe740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f97ac1b0270 con 0x5569fd1bc590 2024-09-19T05:08:07.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.098+0000 7f97aaa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569fd1bc590 0x5569fd1dec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:07.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.098+0000 7f97abe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569fd1bbc70 0x5569fd1be720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:07.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.098+0000 7f97aaa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569fd1bc590 0x5569fd1dec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:59746/0 (socket says 172.21.15.43:59746) 2024-09-19T05:08:07.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.098+0000 7f97aaa00640 1 -- 172.21.15.43:0/2577432546 learned_addr learned my addr 172.21.15.43:0/2577432546 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:07.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.098+0000 7f97abe00640 1 --2- 172.21.15.43:0/2577432546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569fd1c6e50 0x5569fd1df160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:07.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.099+0000 7f97aaa00640 1 -- 172.21.15.43:0/2577432546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569fd1bbc70 msgr2=0x5569fd1be720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:07.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.099+0000 7f97aaa00640 1 --2- 172.21.15.43:0/2577432546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569fd1bbc70 0x5569fd1be720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:07.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.099+0000 7f97aaa00640 1 -- 172.21.15.43:0/2577432546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569fd1c6e50 msgr2=0x5569fd1df160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:07.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.099+0000 7f97aaa00640 1 --2- 172.21.15.43:0/2577432546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569fd1c6e50 0x5569fd1df160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:07.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.099+0000 7f97aaa00640 1 -- 172.21.15.43:0/2577432546 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569fd1b3770 con 0x5569fd1bc590 2024-09-19T05:08:07.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.099+0000 7f97aaa00640 1 --2- 172.21.15.43:0/2577432546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569fd1bc590 0x5569fd1dec20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9798009cf0 tx=0x7f9798025600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=156ef2af78424cde server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:07.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.099+0000 7f9797400640 1 -- 172.21.15.43:0/2577432546 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9798011b40 con 0x5569fd1bc590 2024-09-19T05:08:07.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.099+0000 7f9797400640 1 -- 172.21.15.43:0/2577432546 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9798011ce0 con 0x5569fd1bc590 2024-09-19T05:08:07.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.099+0000 7f9797400640 1 -- 172.21.15.43:0/2577432546 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9798003850 con 0x5569fd1bc590 2024-09-19T05:08:07.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.100+0000 7f97b4dfe740 1 -- 172.21.15.43:0/2577432546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569fd1bc590 msgr2=0x5569fd1dec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:07.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.100+0000 7f97b4dfe740 1 --2- 172.21.15.43:0/2577432546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569fd1bc590 0x5569fd1dec20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9798009cf0 tx=0x7f9798025600 comp rx=0 tx=0).stop 2024-09-19T05:08:07.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.101+0000 7f97b4dfe740 1 -- 172.21.15.43:0/2577432546 shutdown_connections 2024-09-19T05:08:07.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.101+0000 7f97b4dfe740 1 --2- 172.21.15.43:0/2577432546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569fd1c6e50 0x5569fd1df160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:07.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.101+0000 7f97b4dfe740 1 --2- 172.21.15.43:0/2577432546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569fd1bc590 0x5569fd1dec20 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:07.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.101+0000 7f97b4dfe740 1 --2- 172.21.15.43:0/2577432546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569fd1bbc70 0x5569fd1be720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:07.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.101+0000 7f97b4dfe740 1 -- 172.21.15.43:0/2577432546 >> 172.21.15.43:0/2577432546 conn(0x5569fd1b8200 msgr2=0x5569fd1c5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:07.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.101+0000 7f97b4dfe740 1 -- 172.21.15.43:0/2577432546 shutdown_connections 2024-09-19T05:08:07.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.101+0000 7f97b4dfe740 1 -- 172.21.15.43:0/2577432546 wait complete. 2024-09-19T05:08:07.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.101+0000 7f97b4dfe740 1 -- start start 2024-09-19T05:08:07.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.101+0000 7f97b4dfe740 1 Processor -- start 2024-09-19T05:08:07.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.102+0000 7f97b4dfe740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569fd1bbc70 0x5569fd1cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:07.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.102+0000 7f97b4dfe740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569fd1bc590 0x5569fd1cb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:07.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.102+0000 7f97b4dfe740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569fd1c6e50 0x5569fd240bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:07.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.102+0000 7f97b4dfe740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f97ac1b1010 con 0x5569fd1bc590 2024-09-19T05:08:07.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.102+0000 7f97b4dfe740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f97ac1b0940 con 0x5569fd1c6e50 2024-09-19T05:08:07.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.102+0000 7f97b4dfe740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f97ac1adb80 con 0x5569fd1bbc70 2024-09-19T05:08:07.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.102+0000 7f97b4dfe740 10 client.0 _ll_register_callbacks cb 0x7f9798009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:07.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.102+0000 7f97aaa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569fd1bc590 0x5569fd1cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:07.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.102+0000 7f97aaa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569fd1bc590 0x5569fd1cb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56854/0 (socket says 172.21.15.43:56854) 2024-09-19T05:08:07.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.102+0000 7f97aaa00640 1 -- 172.21.15.43:0/3865989358 learned_addr learned my addr 172.21.15.43:0/3865989358 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:07.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.102+0000 7f97abe00640 1 --2- 172.21.15.43:0/3865989358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569fd1bbc70 0x5569fd1cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:07.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.102+0000 7f97abe00640 1 --2- 172.21.15.43:0/3865989358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569fd1c6e50 0x5569fd240bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:07.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.103+0000 7f97abe00640 1 -- 172.21.15.43:0/3865989358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569fd1bbc70 msgr2=0x5569fd1cb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:07.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.103+0000 7f97abe00640 1 --2- 172.21.15.43:0/3865989358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569fd1bbc70 0x5569fd1cb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:07.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.103+0000 7f97abe00640 1 -- 172.21.15.43:0/3865989358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569fd1bc590 msgr2=0x5569fd1cb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:07.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.103+0000 7f97abe00640 1 --2- 172.21.15.43:0/3865989358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569fd1bc590 0x5569fd1cb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:07.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.103+0000 7f97abe00640 1 -- 172.21.15.43:0/3865989358 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569fd1b92b0 con 0x5569fd1c6e50 2024-09-19T05:08:07.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.103+0000 7f97aaa00640 1 --2- 172.21.15.43:0/3865989358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569fd1bc590 0x5569fd1cb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:07.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.103+0000 7f97abe00640 1 --2- 172.21.15.43:0/3865989358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569fd1c6e50 0x5569fd240bf0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f97a402af40 tx=0x7f97a402fee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59a3f0844d78b659 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:07.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.103+0000 7f9794c00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:08:07.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.104+0000 7f9794c00640 1 -- 172.21.15.43:0/3865989358 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a4060060 con 0x5569fd1c6e50 2024-09-19T05:08:07.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.104+0000 7f97b4dfe740 1 -- 172.21.15.43:0/3865989358 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9798011ce0 con 0x5569fd1c6e50 2024-09-19T05:08:07.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.104+0000 7f9794c00640 1 -- 172.21.15.43:0/3865989358 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f97a402f560 con 0x5569fd1c6e50 2024-09-19T05:08:07.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.104+0000 7f9794c00640 1 -- 172.21.15.43:0/3865989358 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f97a4034040 con 0x5569fd1c6e50 2024-09-19T05:08:07.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.104+0000 7f97b4dfe740 10 client.25961 fetch_fsmap learned FSMap version 29 2024-09-19T05:08:07.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.104+0000 7f97b4dfe740 1 -- 172.21.15.43:0/3865989358 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5569fd242160 con 0x5569fd1c6e50 2024-09-19T05:08:07.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.104+0000 7f9794c00640 1 -- 172.21.15.43:0/3865989358 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9798011ce0 con 0x5569fd1c6e50 2024-09-19T05:08:07.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.104+0000 7f9794c00640 1 -- 172.21.15.43:0/3865989358 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f97a4042070 con 0x5569fd1c6e50 2024-09-19T05:08:07.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.105+0000 7f97b4dfe740 10 client.25961 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:08:07.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.105+0000 7f97b4dfe740 10 client.25961 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:08:07.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.105+0000 7f97b4dfe740 20 client.25961 populate_metadata read hostname 'smithi043' 2024-09-19T05:08:07.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.105+0000 7f97b4dfe740 1 --2- 172.21.15.43:0/3865989358 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5569fd2424a0 0x5569fd244890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:07.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.105+0000 7f97b4dfe740 4 client.25961 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:08:07.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.105+0000 7f97b4dfe740 1 -- 172.21.15.43:0/3865989358 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5569fd129e60 con 0x5569fd2424a0 2024-09-19T05:08:07.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.105+0000 7f97aaa00640 1 --2- 172.21.15.43:0/3865989358 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5569fd2424a0 0x5569fd244890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:07.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.105+0000 7f97aaa00640 1 --2- 172.21.15.43:0/3865989358 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5569fd2424a0 0x5569fd244890 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee28e46544b57a9f server_cookie=17a4fb7fc2638ce8 in_seq=0 out_seq=0 2024-09-19T05:08:07.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.105+0000 7f9794c00640 10 client.25961 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:08:07.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.106+0000 7f9794c00640 1 -- 172.21.15.43:0/3865989358 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5569fd129e60 con 0x5569fd2424a0 2024-09-19T05:08:07.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.106+0000 7f9794c00640 10 client.25961 handle_command_reply: tid=0 2024-09-19T05:08:07.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.106+0000 7f97b4dfe740 1 client.25961 shutdown 2024-09-19T05:08:07.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.106+0000 7f97b4dfe740 1 -- 172.21.15.43:0/3865989358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569fd1c6e50 msgr2=0x5569fd240bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:07.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.106+0000 7f97b4dfe740 1 --2- 172.21.15.43:0/3865989358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569fd1c6e50 0x5569fd240bf0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f97a402af40 tx=0x7f97a402fee0 comp rx=0 tx=0).stop 2024-09-19T05:08:07.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.106+0000 7f97b4dfe740 1 -- 172.21.15.43:0/3865989358 shutdown_connections 2024-09-19T05:08:07.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.106+0000 7f97b4dfe740 1 --2- 172.21.15.43:0/3865989358 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5569fd2424a0 0x5569fd244890 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:07.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.106+0000 7f97b4dfe740 1 --2- 172.21.15.43:0/3865989358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569fd1c6e50 0x5569fd240bf0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:07.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.106+0000 7f97b4dfe740 1 --2- 172.21.15.43:0/3865989358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569fd1bc590 0x5569fd1cb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:07.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.106+0000 7f97b4dfe740 1 --2- 172.21.15.43:0/3865989358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569fd1bbc70 0x5569fd1cb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:07.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.106+0000 7f9794c00640 0 client.25961 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:08:07.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.106+0000 7f97b4dfe740 1 -- 172.21.15.43:0/3865989358 >> 172.21.15.43:0/3865989358 conn(0x5569fd1b8200 msgr2=0x5569fd1c5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:07.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.106+0000 7f97b4dfe740 1 -- 172.21.15.43:0/3865989358 shutdown_connections 2024-09-19T05:08:07.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.106+0000 7f97b4dfe740 1 -- 172.21.15.43:0/3865989358 wait complete. 2024-09-19T05:08:07.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.107+0000 7f97b4dfe740 20 client.25961 trim_cache size 0 max 16384 2024-09-19T05:08:07.112 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:08:07.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.107+0000 7f97b2200640 1 -- 172.21.15.43:0/2009973008 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f97900a33b0 msgr2=0x7f97900a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:07.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.107+0000 7f97b2200640 1 --2- 172.21.15.43:0/2009973008 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f97900a33b0 0x7f97900a5870 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f979c0cd980 tx=0x7f979c0d3cc0 comp rx=0 tx=0).stop 2024-09-19T05:08:07.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.107+0000 7f97b2200640 1 -- 172.21.15.43:0/2009973008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97ac0745c0 msgr2=0x7f97ac19a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:07.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.107+0000 7f97b2200640 1 --2- 172.21.15.43:0/2009973008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97ac0745c0 0x7f97ac19a680 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f979c00e9f0 tx=0x7f979c00eeb0 comp rx=0 tx=0).stop 2024-09-19T05:08:07.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.107+0000 7f97b2200640 1 -- 172.21.15.43:0/2009973008 shutdown_connections 2024-09-19T05:08:07.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.107+0000 7f97b2200640 1 --2- 172.21.15.43:0/2009973008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97ac10e420 0x7f97ac19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:07.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.107+0000 7f97b2200640 1 --2- 172.21.15.43:0/2009973008 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97ac074ee0 0x7f97ac19abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:07.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.107+0000 7f97b2200640 1 --2- 172.21.15.43:0/2009973008 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f97900a33b0 0x7f97900a5870 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:07.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.107+0000 7f97b2200640 1 --2- 172.21.15.43:0/2009973008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97ac0745c0 0x7f97ac19a680 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:07.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.107+0000 7f97b2200640 1 -- 172.21.15.43:0/2009973008 >> 172.21.15.43:0/2009973008 conn(0x7f97ac0fd8e0 msgr2=0x7f97ac0fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:07.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.108+0000 7f97b2200640 1 -- 172.21.15.43:0/2009973008 shutdown_connections 2024-09-19T05:08:07.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:07.108+0000 7f97b2200640 1 -- 172.21.15.43:0/2009973008 wait complete. 2024-09-19T05:08:07.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:08:07 smithi096 ceph-mon[35085]: pgmap v107: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 251 MiB/s wr, 86 op/s 2024-09-19T05:08:07.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:08:07 smithi043 ceph-mon[30800]: pgmap v107: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 251 MiB/s wr, 86 op/s 2024-09-19T05:08:07.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:08:07 smithi151 ceph-mon[35425]: pgmap v107: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 251 MiB/s wr, 86 op/s 2024-09-19T05:08:08.493 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-19T05:08:08.494 INFO:tasks.workunit.client.0.smithi043.stdout: Children see throughput for 1 readers = 5248448.50 kB/sec 2024-09-19T05:08:08.494 INFO:tasks.workunit.client.0.smithi043.stdout: Parent sees throughput for 1 readers = 5246505.13 kB/sec 2024-09-19T05:08:08.494 INFO:tasks.workunit.client.0.smithi043.stdout: Min throughput per process = 5248448.50 kB/sec 2024-09-19T05:08:08.494 INFO:tasks.workunit.client.0.smithi043.stdout: Max throughput per process = 5248448.50 kB/sec 2024-09-19T05:08:08.494 INFO:tasks.workunit.client.0.smithi043.stdout: Avg throughput per process = 5248448.50 kB/sec 2024-09-19T05:08:08.494 INFO:tasks.workunit.client.0.smithi043.stdout: Min xfer = 10485760.00 kB 2024-09-19T05:08:09.057 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T05:08:09.058 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T05:08:09.438 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:08:09.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:08:09 smithi096 ceph-mon[35085]: pgmap v108: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 251 MiB/s wr, 86 op/s 2024-09-19T05:08:09.757 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:08:09 smithi043 ceph-mon[30800]: pgmap v108: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 251 MiB/s wr, 86 op/s 2024-09-19T05:08:09.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:08:09 smithi151 ceph-mon[35425]: pgmap v108: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 251 MiB/s wr, 86 op/s 2024-09-19T05:08:10.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.034+0000 7f24f6600640 1 -- 172.21.15.43:0/1368693619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24f0077040 msgr2=0x7f24f0112350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:10.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.034+0000 7f24f6600640 1 --2- 172.21.15.43:0/1368693619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24f0077040 0x7f24f0112350 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f24dc009900 tx=0x7f24dc02a3f0 comp rx=0 tx=0).stop 2024-09-19T05:08:10.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.035+0000 7f24f6600640 1 -- 172.21.15.43:0/1368693619 shutdown_connections 2024-09-19T05:08:10.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.035+0000 7f24f6600640 1 --2- 172.21.15.43:0/1368693619 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f24f0112890 0x7f24f0114c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:10.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.035+0000 7f24f6600640 1 --2- 172.21.15.43:0/1368693619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24f0077040 0x7f24f0112350 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:10.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.035+0000 7f24f6600640 1 --2- 172.21.15.43:0/1368693619 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24f0076720 0x7f24f0076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:10.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.035+0000 7f24f6600640 1 -- 172.21.15.43:0/1368693619 >> 172.21.15.43:0/1368693619 conn(0x7f24f00ff8b0 msgr2=0x7f24f0101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:10.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.035+0000 7f24f6600640 1 -- 172.21.15.43:0/1368693619 shutdown_connections 2024-09-19T05:08:10.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.035+0000 7f24f6600640 1 -- 172.21.15.43:0/1368693619 wait complete. 2024-09-19T05:08:10.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.036+0000 7f24f6600640 1 Processor -- start 2024-09-19T05:08:10.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.036+0000 7f24f6600640 1 -- start start 2024-09-19T05:08:10.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.036+0000 7f24f6600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f24f0076720 0x7f24f01a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:10.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.036+0000 7f24f6600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24f0077040 0x7f24f01a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:10.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.036+0000 7f24f6600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24f0112890 0x7f24f019e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:10.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.036+0000 7f24f6600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f24f0118570 con 0x7f24f0077040 2024-09-19T05:08:10.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.036+0000 7f24f6600640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f24f01183f0 con 0x7f24f0112890 2024-09-19T05:08:10.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.036+0000 7f24f6600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f24f01186f0 con 0x7f24f0076720 2024-09-19T05:08:10.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.037+0000 7f24efe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24f0077040 0x7f24f01a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:10.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.037+0000 7f24efe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24f0077040 0x7f24f01a4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56878/0 (socket says 172.21.15.43:56878) 2024-09-19T05:08:10.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.037+0000 7f24efe00640 1 -- 172.21.15.43:0/986919940 learned_addr learned my addr 172.21.15.43:0/986919940 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:10.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.037+0000 7f24f5c00640 1 --2- 172.21.15.43:0/986919940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24f0112890 0x7f24f019e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:10.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.037+0000 7f24f5200640 1 --2- 172.21.15.43:0/986919940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f24f0076720 0x7f24f01a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:10.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.037+0000 7f24efe00640 1 -- 172.21.15.43:0/986919940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f24f0076720 msgr2=0x7f24f01a46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:10.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.037+0000 7f24efe00640 1 --2- 172.21.15.43:0/986919940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f24f0076720 0x7f24f01a46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:10.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.037+0000 7f24efe00640 1 -- 172.21.15.43:0/986919940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24f0112890 msgr2=0x7f24f019e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:10.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.037+0000 7f24efe00640 1 --2- 172.21.15.43:0/986919940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24f0112890 0x7f24f019e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:10.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.037+0000 7f24efe00640 1 -- 172.21.15.43:0/986919940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24f019efc0 con 0x7f24f0077040 2024-09-19T05:08:10.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.037+0000 7f24f5c00640 1 --2- 172.21.15.43:0/986919940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24f0112890 0x7f24f019e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:10.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.037+0000 7f24efe00640 1 --2- 172.21.15.43:0/986919940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24f0077040 0x7f24f01a4be0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f24dc004770 tx=0x7f24dc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:10.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.038+0000 7f24ed600640 1 -- 172.21.15.43:0/986919940 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24dc03f070 con 0x7f24f0077040 2024-09-19T05:08:10.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.038+0000 7f24ed600640 1 -- 172.21.15.43:0/986919940 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24dc031ca0 con 0x7f24f0077040 2024-09-19T05:08:10.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.038+0000 7f24f6600640 1 -- 172.21.15.43:0/986919940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24f019f250 con 0x7f24f0077040 2024-09-19T05:08:10.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.038+0000 7f24ed600640 1 -- 172.21.15.43:0/986919940 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24dc035030 con 0x7f24f0077040 2024-09-19T05:08:10.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.038+0000 7f24f6600640 1 -- 172.21.15.43:0/986919940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24f01ab500 con 0x7f24f0077040 2024-09-19T05:08:10.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.042+0000 7f24fa863740 1 Processor -- start 2024-09-19T05:08:10.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.042+0000 7f24fa863740 1 -- start start 2024-09-19T05:08:10.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.043+0000 7f24ed600640 1 -- 172.21.15.43:0/986919940 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f24dc042070 con 0x7f24f0077040 2024-09-19T05:08:10.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.043+0000 7f24fa863740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a78a5ecb0 0x559a78a5f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:10.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.043+0000 7f24fa863740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a78a5f5d0 0x559a78a69950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:10.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.043+0000 7f24fa863740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a78a69e90 0x559a78a6c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:10.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.043+0000 7f24fa863740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559a789c5290 con 0x559a78a69e90 2024-09-19T05:08:10.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.043+0000 7f24fa863740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559a7896cb40 con 0x559a78a5ecb0 2024-09-19T05:08:10.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.043+0000 7f24fa863740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559a789c1250 con 0x559a78a5f5d0 2024-09-19T05:08:10.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.043+0000 7f24f5c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a78a5ecb0 0x559a78a5f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:10.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.043+0000 7f24f5c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a78a5ecb0 0x559a78a5f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:52390/0 (socket says 172.21.15.43:52390) 2024-09-19T05:08:10.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.043+0000 7f24f5c00640 1 -- 172.21.15.43:0/231570294 learned_addr learned my addr 172.21.15.43:0/231570294 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:10.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.043+0000 7f24efe00640 1 --2- 172.21.15.43:0/231570294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a78a69e90 0x559a78a6c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:10.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.043+0000 7f24f5200640 1 --2- 172.21.15.43:0/231570294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a78a5f5d0 0x559a78a69950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:10.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.044+0000 7f24f5c00640 1 -- 172.21.15.43:0/231570294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a78a5f5d0 msgr2=0x559a78a69950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:10.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.044+0000 7f24f5c00640 1 --2- 172.21.15.43:0/231570294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a78a5f5d0 0x559a78a69950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:10.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.044+0000 7f24f5c00640 1 -- 172.21.15.43:0/231570294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a78a69e90 msgr2=0x559a78a6c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:10.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.044+0000 7f24f5c00640 1 --2- 172.21.15.43:0/231570294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a78a69e90 0x559a78a6c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:10.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.044+0000 7f24f5c00640 1 -- 172.21.15.43:0/231570294 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a787366a0 con 0x559a78a5ecb0 2024-09-19T05:08:10.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.044+0000 7f24f5c00640 1 --2- 172.21.15.43:0/231570294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a78a5ecb0 0x559a78a5f090 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f24e8011790 tx=0x7f24e8011c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e79fd595be82fd70 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:10.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.044+0000 7f24e2000640 1 -- 172.21.15.43:0/231570294 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e8003c30 con 0x559a78a5ecb0 2024-09-19T05:08:10.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.045+0000 7f24e2000640 1 -- 172.21.15.43:0/231570294 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24e8003dd0 con 0x559a78a5ecb0 2024-09-19T05:08:10.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.045+0000 7f24e2000640 1 -- 172.21.15.43:0/231570294 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e80193e0 con 0x559a78a5ecb0 2024-09-19T05:08:10.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.045+0000 7f24ed600640 1 --2- 172.21.15.43:0/986919940 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f24d80a3430 0x7f24d80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:10.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.045+0000 7f24ed600640 1 -- 172.21.15.43:0/986919940 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f24dc0f0220 con 0x7f24f0077040 2024-09-19T05:08:10.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.045+0000 7f24fa863740 1 -- 172.21.15.43:0/231570294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a78a5ecb0 msgr2=0x559a78a5f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:10.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.045+0000 7f24fa863740 1 --2- 172.21.15.43:0/231570294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a78a5ecb0 0x559a78a5f090 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f24e8011790 tx=0x7f24e8011c50 comp rx=0 tx=0).stop 2024-09-19T05:08:10.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.045+0000 7f24f5200640 1 --2- 172.21.15.43:0/986919940 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f24d80a3430 0x7f24d80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:10.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.045+0000 7f24fa863740 1 -- 172.21.15.43:0/231570294 shutdown_connections 2024-09-19T05:08:10.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.045+0000 7f24fa863740 1 --2- 172.21.15.43:0/231570294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a78a69e90 0x559a78a6c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:10.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.045+0000 7f24fa863740 1 --2- 172.21.15.43:0/231570294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a78a5f5d0 0x559a78a69950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:10.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.045+0000 7f24fa863740 1 --2- 172.21.15.43:0/231570294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a78a5ecb0 0x559a78a5f090 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:10.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.045+0000 7f24fa863740 1 -- 172.21.15.43:0/231570294 >> 172.21.15.43:0/231570294 conn(0x559a78a5b240 msgr2=0x559a78a5b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:10.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.045+0000 7f24fa863740 1 -- 172.21.15.43:0/231570294 shutdown_connections 2024-09-19T05:08:10.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.046+0000 7f24fa863740 1 -- 172.21.15.43:0/231570294 wait complete. 2024-09-19T05:08:10.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.046+0000 7f24fa863740 1 -- start start 2024-09-19T05:08:10.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.046+0000 7f24f5200640 1 --2- 172.21.15.43:0/986919940 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f24d80a3430 0x7f24d80a58f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x559a78a5c0a0 tx=0x7f24e4005f90 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:10.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.046+0000 7f24fa863740 1 Processor -- start 2024-09-19T05:08:10.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.047+0000 7f24fa863740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a78a5ecb0 0x559a78a69760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:10.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.047+0000 7f24fa863740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a78a5f5d0 0x559a78a66a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:10.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.047+0000 7f24fa863740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a78a69e90 0x559a78a66f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:10.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.047+0000 7f24fa863740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559a78a6cb00 con 0x559a78a69e90 2024-09-19T05:08:10.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.047+0000 7f24fa863740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559a7897ffa0 con 0x559a78a5f5d0 2024-09-19T05:08:10.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.047+0000 7f24fa863740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559a78a6cc80 con 0x559a78a5ecb0 2024-09-19T05:08:10.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.047+0000 7f24fa863740 10 client.0 _ll_register_callbacks cb 0x559a786d3ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:10.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.047+0000 7f24f5200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a78a5ecb0 0x559a78a69760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:10.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.047+0000 7f24efe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a78a5f5d0 0x559a78a66a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:10.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.047+0000 7f24f5200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a78a5ecb0 0x559a78a69760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:59802/0 (socket says 172.21.15.43:59802) 2024-09-19T05:08:10.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.047+0000 7f24f5200640 1 -- 172.21.15.43:0/294780019 learned_addr learned my addr 172.21.15.43:0/294780019 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:10.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.047+0000 7f24f5c00640 1 --2- 172.21.15.43:0/294780019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a78a69e90 0x559a78a66f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:10.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.047+0000 7f24efe00640 1 -- 172.21.15.43:0/294780019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a78a5ecb0 msgr2=0x559a78a69760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:10.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.047+0000 7f24efe00640 1 --2- 172.21.15.43:0/294780019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a78a5ecb0 0x559a78a69760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:10.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.047+0000 7f24efe00640 1 -- 172.21.15.43:0/294780019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a78a69e90 msgr2=0x559a78a66f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:10.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.047+0000 7f24efe00640 1 --2- 172.21.15.43:0/294780019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a78a69e90 0x559a78a66f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:10.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.047+0000 7f24efe00640 1 -- 172.21.15.43:0/294780019 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a78a67780 con 0x559a78a5f5d0 2024-09-19T05:08:10.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.047+0000 7f24f5200640 1 --2- 172.21.15.43:0/294780019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a78a5ecb0 0x559a78a69760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:10.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.048+0000 7f24efe00640 1 --2- 172.21.15.43:0/294780019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a78a5f5d0 0x559a78a66a10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f24dc0ed980 tx=0x7f24dc02a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e95c61efbad59f3c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:10.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.048+0000 7f24d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:08:10.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.048+0000 7f24d6a00640 1 -- 172.21.15.43:0/294780019 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24dc049070 con 0x559a78a5f5d0 2024-09-19T05:08:10.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.048+0000 7f24fa863740 1 -- 172.21.15.43:0/294780019 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24e8003dd0 con 0x559a78a5f5d0 2024-09-19T05:08:10.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.048+0000 7f24d6a00640 1 -- 172.21.15.43:0/294780019 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f24dc0b8cf0 con 0x559a78a5f5d0 2024-09-19T05:08:10.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.048+0000 7f24d6a00640 1 -- 172.21.15.43:0/294780019 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f24dc0bbfe0 con 0x559a78a5f5d0 2024-09-19T05:08:10.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.048+0000 7f24fa863740 10 client.25973 fetch_fsmap learned FSMap version 29 2024-09-19T05:08:10.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.048+0000 7f24fa863740 1 -- 172.21.15.43:0/294780019 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559a78a6cb00 con 0x559a78a5f5d0 2024-09-19T05:08:10.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.048+0000 7f24d6a00640 1 -- 172.21.15.43:0/294780019 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24e8003dd0 con 0x559a78a5f5d0 2024-09-19T05:08:10.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.049+0000 7f24d6a00640 1 -- 172.21.15.43:0/294780019 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f24dc03b4f0 con 0x559a78a5f5d0 2024-09-19T05:08:10.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.049+0000 7f24fa863740 10 client.25973 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:08:10.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.049+0000 7f24fa863740 10 client.25973 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:08:10.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.049+0000 7f24fa863740 20 client.25973 populate_metadata read hostname 'smithi043' 2024-09-19T05:08:10.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.049+0000 7f24fa863740 1 --2- 172.21.15.43:0/294780019 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x559a78a79da0 0x559a78a7c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:10.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.049+0000 7f24fa863740 4 client.25973 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:08:10.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.049+0000 7f24fa863740 1 -- 172.21.15.43:0/294780019 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a789d2800 con 0x559a78a79da0 2024-09-19T05:08:10.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.049+0000 7f24f5200640 1 --2- 172.21.15.43:0/294780019 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x559a78a79da0 0x559a78a7c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:10.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.050+0000 7f24f5200640 1 --2- 172.21.15.43:0/294780019 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x559a78a79da0 0x559a78a7c1c0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6aab7b0895b22ab server_cookie=ee8cb052f2e937d2 in_seq=0 out_seq=0 2024-09-19T05:08:10.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.050+0000 7f24d6a00640 10 client.25973 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:08:10.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.051+0000 7f24d6a00640 1 -- 172.21.15.43:0/294780019 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559a789d2800 con 0x559a78a79da0 2024-09-19T05:08:10.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.051+0000 7f24d6a00640 10 client.25973 handle_command_reply: tid=0 2024-09-19T05:08:10.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.051+0000 7f24fa863740 1 client.25973 shutdown 2024-09-19T05:08:10.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.051+0000 7f24fa863740 1 -- 172.21.15.43:0/294780019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a78a5f5d0 msgr2=0x559a78a66a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:10.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.051+0000 7f24fa863740 1 --2- 172.21.15.43:0/294780019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a78a5f5d0 0x559a78a66a10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f24dc0ed980 tx=0x7f24dc02a3f0 comp rx=0 tx=0).stop 2024-09-19T05:08:10.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.051+0000 7f24fa863740 1 -- 172.21.15.43:0/294780019 shutdown_connections 2024-09-19T05:08:10.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.051+0000 7f24fa863740 1 --2- 172.21.15.43:0/294780019 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x559a78a79da0 0x559a78a7c1c0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:10.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.051+0000 7f24fa863740 1 --2- 172.21.15.43:0/294780019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a78a69e90 0x559a78a66f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:10.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.051+0000 7f24fa863740 1 --2- 172.21.15.43:0/294780019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a78a5f5d0 0x559a78a66a10 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:10.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.051+0000 7f24fa863740 1 --2- 172.21.15.43:0/294780019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a78a5ecb0 0x559a78a69760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:10.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.051+0000 7f24d6a00640 0 client.25973 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:08:10.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.051+0000 7f24fa863740 1 -- 172.21.15.43:0/294780019 >> 172.21.15.43:0/294780019 conn(0x559a78a5b240 msgr2=0x559a78a60f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:10.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.051+0000 7f24fa863740 1 -- 172.21.15.43:0/294780019 shutdown_connections 2024-09-19T05:08:10.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.051+0000 7f24fa863740 1 -- 172.21.15.43:0/294780019 wait complete. 2024-09-19T05:08:10.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.051+0000 7f24fa863740 20 client.25973 trim_cache size 0 max 16384 2024-09-19T05:08:10.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.069+0000 7f24fa863740 1 Processor -- start 2024-09-19T05:08:10.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.069+0000 7f24fa863740 1 -- start start 2024-09-19T05:08:10.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.069+0000 7f24fa863740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a78a5ecb0 0x559a78a68980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:10.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.069+0000 7f24fa863740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a78a5f5d0 0x559a78a81ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:10.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.070+0000 7f24fa863740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a78a69e90 0x559a78a82010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:10.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.070+0000 7f24fa863740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f24f01b3d00 con 0x559a78a69e90 2024-09-19T05:08:10.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.070+0000 7f24fa863740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f24f01b5840 con 0x559a78a5f5d0 2024-09-19T05:08:10.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.070+0000 7f24fa863740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f24f01b43d0 con 0x559a78a5ecb0 2024-09-19T05:08:10.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.070+0000 7f24f5c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a78a69e90 0x559a78a82010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:10.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.070+0000 7f24f5200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a78a5ecb0 0x559a78a68980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:10.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.070+0000 7f24f5200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a78a5ecb0 0x559a78a68980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:59810/0 (socket says 172.21.15.43:59810) 2024-09-19T05:08:10.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.070+0000 7f24f5200640 1 -- 172.21.15.43:0/84319626 learned_addr learned my addr 172.21.15.43:0/84319626 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:10.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.070+0000 7f24f5c00640 1 --2- 172.21.15.43:0/84319626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a78a69e90 0x559a78a82010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56906/0 (socket says 172.21.15.43:56906) 2024-09-19T05:08:10.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.070+0000 7f24efe00640 1 --2- 172.21.15.43:0/84319626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a78a5f5d0 0x559a78a81ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:10.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.070+0000 7f24f5200640 1 -- 172.21.15.43:0/84319626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a78a5f5d0 msgr2=0x559a78a81ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:10.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.070+0000 7f24f5200640 1 --2- 172.21.15.43:0/84319626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a78a5f5d0 0x559a78a81ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:10.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.070+0000 7f24f5200640 1 -- 172.21.15.43:0/84319626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a78a69e90 msgr2=0x559a78a82010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:10.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.070+0000 7f24f5200640 1 --2- 172.21.15.43:0/84319626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a78a69e90 0x559a78a82010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:10.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.070+0000 7f24f5200640 1 -- 172.21.15.43:0/84319626 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a78a659c0 con 0x559a78a5ecb0 2024-09-19T05:08:10.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.071+0000 7f24f5200640 1 --2- 172.21.15.43:0/84319626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a78a5ecb0 0x559a78a68980 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f24e4009460 tx=0x7f24e401b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86887f0f3c7bbce0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:10.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.071+0000 7f24e1600640 1 -- 172.21.15.43:0/84319626 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e402b530 con 0x559a78a5ecb0 2024-09-19T05:08:10.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.071+0000 7f24e1600640 1 -- 172.21.15.43:0/84319626 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24e402bbb0 con 0x559a78a5ecb0 2024-09-19T05:08:10.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.071+0000 7f24e1600640 1 -- 172.21.15.43:0/84319626 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e401e420 con 0x559a78a5ecb0 2024-09-19T05:08:10.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.072+0000 7f24fa863740 1 -- 172.21.15.43:0/84319626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a78a5ecb0 msgr2=0x559a78a68980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:10.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.072+0000 7f24fa863740 1 --2- 172.21.15.43:0/84319626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a78a5ecb0 0x559a78a68980 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f24e4009460 tx=0x7f24e401b000 comp rx=0 tx=0).stop 2024-09-19T05:08:10.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.072+0000 7f24fa863740 1 -- 172.21.15.43:0/84319626 shutdown_connections 2024-09-19T05:08:10.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.072+0000 7f24fa863740 1 --2- 172.21.15.43:0/84319626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a78a69e90 0x559a78a82010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:10.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.072+0000 7f24fa863740 1 --2- 172.21.15.43:0/84319626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a78a5f5d0 0x559a78a81ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:10.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.072+0000 7f24fa863740 1 --2- 172.21.15.43:0/84319626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a78a5ecb0 0x559a78a68980 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:10.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.072+0000 7f24fa863740 1 -- 172.21.15.43:0/84319626 >> 172.21.15.43:0/84319626 conn(0x559a78a5b240 msgr2=0x559a78a68270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:10.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.072+0000 7f24fa863740 1 -- 172.21.15.43:0/84319626 shutdown_connections 2024-09-19T05:08:10.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.072+0000 7f24fa863740 1 -- 172.21.15.43:0/84319626 wait complete. 2024-09-19T05:08:10.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.072+0000 7f24fa863740 1 -- start start 2024-09-19T05:08:10.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.072+0000 7f24fa863740 1 Processor -- start 2024-09-19T05:08:10.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.073+0000 7f24fa863740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a78a5ecb0 0x559a78a70000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:10.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.073+0000 7f24fa863740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a78a5f5d0 0x559a78a70540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:10.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.073+0000 7f24fa863740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a78a69e90 0x559a78ae3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:10.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.073+0000 7f24fa863740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f24f01b5170 con 0x559a78a69e90 2024-09-19T05:08:10.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.073+0000 7f24fa863740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f24f01b4aa0 con 0x559a78a5f5d0 2024-09-19T05:08:10.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.073+0000 7f24fa863740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f24f01b1ce0 con 0x559a78a5ecb0 2024-09-19T05:08:10.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.073+0000 7f24fa863740 10 client.0 _ll_register_callbacks cb 0x7f24dc009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:10.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.073+0000 7f24f5200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a78a5ecb0 0x559a78a70000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:10.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.073+0000 7f24f5200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a78a5ecb0 0x559a78a70000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:59820/0 (socket says 172.21.15.43:59820) 2024-09-19T05:08:10.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.073+0000 7f24f5200640 1 -- 172.21.15.43:0/747808595 learned_addr learned my addr 172.21.15.43:0/747808595 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:10.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.073+0000 7f24efe00640 1 --2- 172.21.15.43:0/747808595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a78a5f5d0 0x559a78a70540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:10.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.073+0000 7f24f5c00640 1 --2- 172.21.15.43:0/747808595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a78a69e90 0x559a78ae3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:10.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.074+0000 7f24f5200640 1 -- 172.21.15.43:0/747808595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a78a5f5d0 msgr2=0x559a78a70540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:10.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.074+0000 7f24f5200640 1 --2- 172.21.15.43:0/747808595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a78a5f5d0 0x559a78a70540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:10.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.074+0000 7f24f5200640 1 -- 172.21.15.43:0/747808595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a78a69e90 msgr2=0x559a78ae3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:10.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.074+0000 7f24f5200640 1 --2- 172.21.15.43:0/747808595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a78a69e90 0x559a78ae3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:10.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.074+0000 7f24f5200640 1 -- 172.21.15.43:0/747808595 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a78ae3ff0 con 0x559a78a5ecb0 2024-09-19T05:08:10.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.074+0000 7f24efe00640 1 --2- 172.21.15.43:0/747808595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a78a5f5d0 0x559a78a70540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:10.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.074+0000 7f24f5200640 1 --2- 172.21.15.43:0/747808595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a78a5ecb0 0x559a78a70000 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f24e4005f30 tx=0x7f24e4004350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4bd179181bc11bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:10.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.074+0000 7f24d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:08:10.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.074+0000 7f24d6a00640 1 -- 172.21.15.43:0/747808595 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e401fde0 con 0x559a78a5ecb0 2024-09-19T05:08:10.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.074+0000 7f24d6a00640 1 -- 172.21.15.43:0/747808595 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f24e401f4f0 con 0x559a78a5ecb0 2024-09-19T05:08:10.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.074+0000 7f24fa863740 1 -- 172.21.15.43:0/747808595 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24e402bbb0 con 0x559a78a5ecb0 2024-09-19T05:08:10.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.075+0000 7f24d6a00640 1 -- 172.21.15.43:0/747808595 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f24e401f730 con 0x559a78a5ecb0 2024-09-19T05:08:10.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.075+0000 7f24fa863740 10 client.25921 fetch_fsmap learned FSMap version 29 2024-09-19T05:08:10.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.075+0000 7f24fa863740 1 -- 172.21.15.43:0/747808595 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559a78ae5180 con 0x559a78a5ecb0 2024-09-19T05:08:10.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.075+0000 7f24d6a00640 1 -- 172.21.15.43:0/747808595 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24e402bbb0 con 0x559a78a5ecb0 2024-09-19T05:08:10.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.075+0000 7f24d6a00640 1 -- 172.21.15.43:0/747808595 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f24e403c070 con 0x559a78a5ecb0 2024-09-19T05:08:10.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.075+0000 7f24fa863740 10 client.25921 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:08:10.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.075+0000 7f24fa863740 10 client.25921 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:08:10.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.075+0000 7f24fa863740 20 client.25921 populate_metadata read hostname 'smithi043' 2024-09-19T05:08:10.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.075+0000 7f24fa863740 1 --2- 172.21.15.43:0/747808595 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x559a78ae57e0 0x559a78ae7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:10.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.075+0000 7f24fa863740 4 client.25921 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T05:08:10.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.075+0000 7f24fa863740 1 -- 172.21.15.43:0/747808595 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559a789ccd10 con 0x559a78ae57e0 2024-09-19T05:08:10.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.075+0000 7f24efe00640 1 --2- 172.21.15.43:0/747808595 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x559a78ae57e0 0x559a78ae7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:10.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.076+0000 7f24efe00640 1 --2- 172.21.15.43:0/747808595 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x559a78ae57e0 0x559a78ae7c00 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=713e0b797bd03a06 server_cookie=2f40448c8ba869e1 in_seq=0 out_seq=0 2024-09-19T05:08:10.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.076+0000 7f24d6a00640 10 client.25921 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:08:10.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.077+0000 7f24d6a00640 1 -- 172.21.15.43:0/747808595 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559a789ccd10 con 0x559a78ae57e0 2024-09-19T05:08:10.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.077+0000 7f24d6a00640 10 client.25921 handle_command_reply: tid=0 2024-09-19T05:08:10.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.077+0000 7f24fa863740 1 client.25921 shutdown 2024-09-19T05:08:10.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.077+0000 7f24fa863740 1 -- 172.21.15.43:0/747808595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a78a5ecb0 msgr2=0x559a78a70000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:10.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.077+0000 7f24fa863740 1 --2- 172.21.15.43:0/747808595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a78a5ecb0 0x559a78a70000 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f24e4005f30 tx=0x7f24e4004350 comp rx=0 tx=0).stop 2024-09-19T05:08:10.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.077+0000 7f24fa863740 1 -- 172.21.15.43:0/747808595 shutdown_connections 2024-09-19T05:08:10.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.077+0000 7f24fa863740 1 --2- 172.21.15.43:0/747808595 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x559a78ae57e0 0x559a78ae7c00 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:10.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.077+0000 7f24fa863740 1 --2- 172.21.15.43:0/747808595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a78a69e90 0x559a78ae3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:10.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.077+0000 7f24fa863740 1 --2- 172.21.15.43:0/747808595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a78a5f5d0 0x559a78a70540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:10.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.077+0000 7f24fa863740 1 --2- 172.21.15.43:0/747808595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559a78a5ecb0 0x559a78a70000 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:10.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.077+0000 7f24fa863740 1 -- 172.21.15.43:0/747808595 >> 172.21.15.43:0/747808595 conn(0x559a78a5b240 msgr2=0x559a78a5a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:10.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.077+0000 7f24d6a00640 0 client.25921 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:08:10.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.078+0000 7f24fa863740 1 -- 172.21.15.43:0/747808595 shutdown_connections 2024-09-19T05:08:10.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.078+0000 7f24fa863740 1 -- 172.21.15.43:0/747808595 wait complete. 2024-09-19T05:08:10.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.078+0000 7f24fa863740 20 client.25921 trim_cache size 0 max 16384 2024-09-19T05:08:10.085 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:08:10.085 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-09-19T05:08:10.085 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "086b4d5b-2d76-4a69-8f1f-a6fbeed795cf", 2024-09-19T05:08:10.085 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-09-19T05:08:10.085 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:08:10.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.078+0000 7f24f6600640 1 -- 172.21.15.43:0/986919940 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f24d80a3430 msgr2=0x7f24d80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:10.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.079+0000 7f24f6600640 1 --2- 172.21.15.43:0/986919940 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f24d80a3430 0x7f24d80a58f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x559a78a5c0a0 tx=0x7f24e4005f90 comp rx=0 tx=0).stop 2024-09-19T05:08:10.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.079+0000 7f24f6600640 1 -- 172.21.15.43:0/986919940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24f0077040 msgr2=0x7f24f01a4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:10.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.079+0000 7f24f6600640 1 --2- 172.21.15.43:0/986919940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24f0077040 0x7f24f01a4be0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f24dc004770 tx=0x7f24dc003c10 comp rx=0 tx=0).stop 2024-09-19T05:08:10.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.079+0000 7f24f6600640 1 -- 172.21.15.43:0/986919940 shutdown_connections 2024-09-19T05:08:10.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.079+0000 7f24f6600640 1 --2- 172.21.15.43:0/986919940 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f24d80a3430 0x7f24d80a58f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:10.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.079+0000 7f24f6600640 1 --2- 172.21.15.43:0/986919940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24f0112890 0x7f24f019e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:10.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.079+0000 7f24f6600640 1 --2- 172.21.15.43:0/986919940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24f0077040 0x7f24f01a4be0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:10.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.079+0000 7f24f6600640 1 --2- 172.21.15.43:0/986919940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f24f0076720 0x7f24f01a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:10.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.079+0000 7f24f6600640 1 -- 172.21.15.43:0/986919940 >> 172.21.15.43:0/986919940 conn(0x7f24f00ff8b0 msgr2=0x7f24f0101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:10.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.079+0000 7f24f6600640 1 -- 172.21.15.43:0/986919940 shutdown_connections 2024-09-19T05:08:10.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:10.079+0000 7f24f6600640 1 -- 172.21.15.43:0/986919940 wait complete. 2024-09-19T05:08:10.929 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:08:11.335 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:08:11.699 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:08:11 smithi043 ceph-mon[30800]: scrub queued for path: / 2024-09-19T05:08:11.699 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:08:11 smithi043 ceph-mon[30800]: scrub summary: idle+waiting paths [/] 2024-09-19T05:08:11.699 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:08:11 smithi043 ceph-mon[30800]: scrub summary: active paths [/] 2024-09-19T05:08:11.699 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:08:11 smithi043 ceph-mon[30800]: scrub summary: idle+waiting paths [/] 2024-09-19T05:08:11.699 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:08:11 smithi043 ceph-mon[30800]: pgmap v109: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 251 MiB/s wr, 90 op/s 2024-09-19T05:08:11.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:08:11 smithi096 ceph-mon[35085]: scrub queued for path: / 2024-09-19T05:08:11.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:08:11 smithi096 ceph-mon[35085]: scrub summary: idle+waiting paths [/] 2024-09-19T05:08:11.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:08:11 smithi096 ceph-mon[35085]: scrub summary: active paths [/] 2024-09-19T05:08:11.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:08:11 smithi096 ceph-mon[35085]: scrub summary: idle+waiting paths [/] 2024-09-19T05:08:11.724 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:08:11 smithi096 ceph-mon[35085]: pgmap v109: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 251 MiB/s wr, 90 op/s 2024-09-19T05:08:11.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:08:11 smithi151 ceph-mon[35425]: scrub queued for path: / 2024-09-19T05:08:11.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:08:11 smithi151 ceph-mon[35425]: scrub summary: idle+waiting paths [/] 2024-09-19T05:08:11.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:08:11 smithi151 ceph-mon[35425]: scrub summary: active paths [/] 2024-09-19T05:08:11.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:08:11 smithi151 ceph-mon[35425]: scrub summary: idle+waiting paths [/] 2024-09-19T05:08:11.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:08:11 smithi151 ceph-mon[35425]: pgmap v109: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 251 MiB/s wr, 90 op/s 2024-09-19T05:08:11.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.949+0000 7f79f3400640 1 -- 172.21.15.43:0/2557549070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79ec101de0 msgr2=0x7f79ec1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:11.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.949+0000 7f79f3400640 1 --2- 172.21.15.43:0/2557549070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79ec101de0 0x7f79ec1021c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f79e0009900 tx=0x7f79e002a410 comp rx=0 tx=0).stop 2024-09-19T05:08:11.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.949+0000 7f79f3400640 1 -- 172.21.15.43:0/2557549070 shutdown_connections 2024-09-19T05:08:11.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.949+0000 7f79f3400640 1 --2- 172.21.15.43:0/2557549070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79ec106f10 0x7f79ec10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:11.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.949+0000 7f79f3400640 1 --2- 172.21.15.43:0/2557549070 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79ec102790 0x7f79ec1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:11.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.949+0000 7f79f3400640 1 --2- 172.21.15.43:0/2557549070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79ec101de0 0x7f79ec1021c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:11.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.950+0000 7f79f3400640 1 -- 172.21.15.43:0/2557549070 >> 172.21.15.43:0/2557549070 conn(0x7f79ec0fd8e0 msgr2=0x7f79ec0ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:11.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.950+0000 7f79f3400640 1 -- 172.21.15.43:0/2557549070 shutdown_connections 2024-09-19T05:08:11.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.950+0000 7f79f3400640 1 -- 172.21.15.43:0/2557549070 wait complete. 2024-09-19T05:08:11.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.951+0000 7f79f3400640 1 Processor -- start 2024-09-19T05:08:11.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.951+0000 7f79f3400640 1 -- start start 2024-09-19T05:08:11.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.952+0000 7f79f3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79ec101de0 0x7f79ec19a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:11.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.952+0000 7f79f3400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79ec102790 0x7f79ec19abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:11.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.952+0000 7f79f3400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79ec106f10 0x7f79ec19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:11.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.952+0000 7f79f3400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f79ec111f70 con 0x7f79ec101de0 2024-09-19T05:08:11.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.952+0000 7f79f3400640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f79ec111df0 con 0x7f79ec106f10 2024-09-19T05:08:11.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.952+0000 7f79f3400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f79ec1120f0 con 0x7f79ec102790 2024-09-19T05:08:11.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.952+0000 7f79f1400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79ec106f10 0x7f79ec19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:11.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.952+0000 7f79f1400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79ec106f10 0x7f79ec19ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:52446/0 (socket says 172.21.15.43:52446) 2024-09-19T05:08:11.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.952+0000 7f79ebe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79ec102790 0x7f79ec19abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:11.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.952+0000 7f79f1400640 1 -- 172.21.15.43:0/3340941125 learned_addr learned my addr 172.21.15.43:0/3340941125 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:11.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.952+0000 7f79f0a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79ec101de0 0x7f79ec19a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:11.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.953+0000 7f79f1400640 1 -- 172.21.15.43:0/3340941125 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79ec102790 msgr2=0x7f79ec19abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:11.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.953+0000 7f79f1400640 1 --2- 172.21.15.43:0/3340941125 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79ec102790 0x7f79ec19abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:11.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.953+0000 7f79f1400640 1 -- 172.21.15.43:0/3340941125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79ec101de0 msgr2=0x7f79ec19a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:11.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.953+0000 7f79f1400640 1 --2- 172.21.15.43:0/3340941125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79ec101de0 0x7f79ec19a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:11.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.953+0000 7f79f1400640 1 -- 172.21.15.43:0/3340941125 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79ec19f660 con 0x7f79ec106f10 2024-09-19T05:08:11.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.953+0000 7f79ebe00640 1 --2- 172.21.15.43:0/3340941125 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79ec102790 0x7f79ec19abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:08:11.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.953+0000 7f79f1400640 1 --2- 172.21.15.43:0/3340941125 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79ec106f10 0x7f79ec19ef80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f79e400eb10 tx=0x7f79e400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:11.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.953+0000 7f79e9600640 1 -- 172.21.15.43:0/3340941125 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e4004b20 con 0x7f79ec106f10 2024-09-19T05:08:11.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.953+0000 7f79e9600640 1 -- 172.21.15.43:0/3340941125 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79e4004cc0 con 0x7f79ec106f10 2024-09-19T05:08:11.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.953+0000 7f79e9600640 1 -- 172.21.15.43:0/3340941125 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e4010760 con 0x7f79ec106f10 2024-09-19T05:08:11.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.953+0000 7f79f3400640 1 -- 172.21.15.43:0/3340941125 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79ec19f950 con 0x7f79ec106f10 2024-09-19T05:08:11.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.954+0000 7f79f3400640 1 -- 172.21.15.43:0/3340941125 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79ec1a7230 con 0x7f79ec106f10 2024-09-19T05:08:11.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.957+0000 7f79e9600640 1 -- 172.21.15.43:0/3340941125 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f79e4005020 con 0x7f79ec106f10 2024-09-19T05:08:11.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.957+0000 7f79f602c740 1 Processor -- start 2024-09-19T05:08:11.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.957+0000 7f79f602c740 1 -- start start 2024-09-19T05:08:11.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.958+0000 7f79f602c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7ab855b90 0x55d7ab855f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:11.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.958+0000 7f79f602c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7ab8564b0 0x55d7ab860830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:11.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.958+0000 7f79f602c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7ab860d70 0x55d7ab863160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:11.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.958+0000 7f79f602c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d7ab7bc3e0 con 0x55d7ab855b90 2024-09-19T05:08:11.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.958+0000 7f79f602c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d7ab763d80 con 0x55d7ab860d70 2024-09-19T05:08:11.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.958+0000 7f79f602c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d7ab7b83a0 con 0x55d7ab8564b0 2024-09-19T05:08:11.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.958+0000 7f79ebe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7ab855b90 0x55d7ab855f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:11.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.958+0000 7f79f0a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7ab8564b0 0x55d7ab860830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:11.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.958+0000 7f79f0a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7ab8564b0 0x55d7ab860830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:59840/0 (socket says 172.21.15.43:59840) 2024-09-19T05:08:11.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.958+0000 7f79ebe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7ab855b90 0x55d7ab855f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56926/0 (socket says 172.21.15.43:56926) 2024-09-19T05:08:11.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.958+0000 7f79f0a00640 1 -- 172.21.15.43:0/2828204150 learned_addr learned my addr 172.21.15.43:0/2828204150 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:11.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.958+0000 7f79ebe00640 1 --2- 172.21.15.43:0/2828204150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7ab860d70 0x55d7ab863160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:11.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.959+0000 7f79e9600640 1 --2- 172.21.15.43:0/3340941125 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f79d40a3430 0x7f79d40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:11.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.959+0000 7f79f0a00640 1 -- 172.21.15.43:0/2828204150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7ab860d70 msgr2=0x55d7ab863160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:11.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.959+0000 7f79f0a00640 1 --2- 172.21.15.43:0/2828204150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7ab860d70 0x55d7ab863160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:11.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.959+0000 7f79f0a00640 1 -- 172.21.15.43:0/2828204150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7ab855b90 msgr2=0x55d7ab855f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:11.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.959+0000 7f79f0a00640 1 --2- 172.21.15.43:0/2828204150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7ab855b90 0x55d7ab855f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:11.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.959+0000 7f79f0a00640 1 -- 172.21.15.43:0/2828204150 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7ab782ae0 con 0x55d7ab8564b0 2024-09-19T05:08:11.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.959+0000 7f79ebe00640 1 --2- 172.21.15.43:0/2828204150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7ab860d70 0x55d7ab863160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:08:11.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.959+0000 7f79e9600640 1 -- 172.21.15.43:0/3340941125 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f79e40d02b0 con 0x7f79ec106f10 2024-09-19T05:08:11.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.959+0000 7f79f0a00640 1 --2- 172.21.15.43:0/2828204150 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7ab8564b0 0x55d7ab860830 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f79e0009900 tx=0x7f79e0005be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=221cae9a09a3ecce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:11.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.959+0000 7f79f1400640 1 --2- 172.21.15.43:0/3340941125 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f79d40a3430 0x7f79d40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:11.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.960+0000 7f79ddc00640 1 -- 172.21.15.43:0/2828204150 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e0040070 con 0x55d7ab8564b0 2024-09-19T05:08:11.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.960+0000 7f79ddc00640 1 -- 172.21.15.43:0/2828204150 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79e0031870 con 0x55d7ab8564b0 2024-09-19T05:08:11.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.960+0000 7f79ddc00640 1 -- 172.21.15.43:0/2828204150 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e0035040 con 0x55d7ab8564b0 2024-09-19T05:08:11.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.960+0000 7f79f602c740 1 -- 172.21.15.43:0/2828204150 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7ab8564b0 msgr2=0x55d7ab860830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:11.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.960+0000 7f79f1400640 1 --2- 172.21.15.43:0/3340941125 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f79d40a3430 0x7f79d40a58f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f79e40cda10 tx=0x7f79e4005b40 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:11.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.960+0000 7f79f602c740 1 --2- 172.21.15.43:0/2828204150 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7ab8564b0 0x55d7ab860830 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f79e0009900 tx=0x7f79e0005be0 comp rx=0 tx=0).stop 2024-09-19T05:08:11.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.961+0000 7f79f602c740 1 -- 172.21.15.43:0/2828204150 shutdown_connections 2024-09-19T05:08:11.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.961+0000 7f79f602c740 1 --2- 172.21.15.43:0/2828204150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7ab860d70 0x55d7ab863160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:11.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.961+0000 7f79f602c740 1 --2- 172.21.15.43:0/2828204150 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7ab8564b0 0x55d7ab860830 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:11.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.961+0000 7f79f602c740 1 --2- 172.21.15.43:0/2828204150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7ab855b90 0x55d7ab855f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:11.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.961+0000 7f79f602c740 1 -- 172.21.15.43:0/2828204150 >> 172.21.15.43:0/2828204150 conn(0x55d7ab852120 msgr2=0x55d7ab852500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:11.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.961+0000 7f79f602c740 1 -- 172.21.15.43:0/2828204150 shutdown_connections 2024-09-19T05:08:11.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.961+0000 7f79f602c740 1 -- 172.21.15.43:0/2828204150 wait complete. 2024-09-19T05:08:11.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.961+0000 7f79f602c740 1 -- start start 2024-09-19T05:08:11.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.962+0000 7f79f602c740 1 Processor -- start 2024-09-19T05:08:11.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.962+0000 7f79f602c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7ab855b90 0x55d7ab85dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:11.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.962+0000 7f79f602c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7ab8564b0 0x55d7ab85e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:11.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.962+0000 7f79f602c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7ab860d70 0x55d7ab85e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:11.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.962+0000 7f79f602c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d7ab8639e0 con 0x55d7ab860d70 2024-09-19T05:08:11.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.962+0000 7f79f602c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d7ab7770f0 con 0x55d7ab8564b0 2024-09-19T05:08:11.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.962+0000 7f79f602c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d7ab863b60 con 0x55d7ab855b90 2024-09-19T05:08:11.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.962+0000 7f79f602c740 10 client.0 _ll_register_callbacks cb 0x55d7ab7689e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:11.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.962+0000 7f79ebe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7ab855b90 0x55d7ab85dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:11.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.962+0000 7f79ebe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7ab860d70 0x55d7ab85e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:11.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.962+0000 7f79ebe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7ab855b90 0x55d7ab85dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:59850/0 (socket says 172.21.15.43:59850) 2024-09-19T05:08:11.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.962+0000 7f79ebe00640 1 -- 172.21.15.43:0/3373684689 learned_addr learned my addr 172.21.15.43:0/3373684689 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:11.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.963+0000 7f79f0a00640 1 --2- 172.21.15.43:0/3373684689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7ab8564b0 0x55d7ab85e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:11.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.963+0000 7f79ebe00640 1 -- 172.21.15.43:0/3373684689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7ab8564b0 msgr2=0x55d7ab85e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:11.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.963+0000 7f79ebe00640 1 --2- 172.21.15.43:0/3373684689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7ab8564b0 0x55d7ab85e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:11.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.963+0000 7f79ebe00640 1 -- 172.21.15.43:0/3373684689 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7ab860d70 msgr2=0x55d7ab85e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:11.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.963+0000 7f79ebe00640 1 --2- 172.21.15.43:0/3373684689 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7ab860d70 0x55d7ab85e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:11.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.963+0000 7f79ebe00640 1 -- 172.21.15.43:0/3373684689 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7ab52d3c0 con 0x55d7ab855b90 2024-09-19T05:08:11.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.963+0000 7f79f0a00640 1 --2- 172.21.15.43:0/3373684689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7ab8564b0 0x55d7ab85e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:11.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.963+0000 7f79ebe00640 1 --2- 172.21.15.43:0/3373684689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7ab855b90 0x55d7ab85dca0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f79d8013f50 tx=0x7f79d8005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87ae95dc8f297df5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:11.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.963+0000 7f79d2000640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:08:11.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.964+0000 7f79d2000640 1 -- 172.21.15.43:0/3373684689 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d8013050 con 0x55d7ab855b90 2024-09-19T05:08:11.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.964+0000 7f79d2000640 1 -- 172.21.15.43:0/3373684689 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f79d80131f0 con 0x55d7ab855b90 2024-09-19T05:08:11.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.964+0000 7f79f602c740 1 -- 172.21.15.43:0/3373684689 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79e0031870 con 0x55d7ab855b90 2024-09-19T05:08:11.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.964+0000 7f79d2000640 1 -- 172.21.15.43:0/3373684689 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f79d8022470 con 0x55d7ab855b90 2024-09-19T05:08:11.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.964+0000 7f79f602c740 10 client.25933 fetch_fsmap learned FSMap version 29 2024-09-19T05:08:11.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.964+0000 7f79f602c740 1 -- 172.21.15.43:0/3373684689 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d7ab86f740 con 0x55d7ab855b90 2024-09-19T05:08:11.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.964+0000 7f79d2000640 1 -- 172.21.15.43:0/3373684689 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79e0031870 con 0x55d7ab855b90 2024-09-19T05:08:11.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.965+0000 7f79d2000640 1 -- 172.21.15.43:0/3373684689 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f79d8011440 con 0x55d7ab855b90 2024-09-19T05:08:11.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.965+0000 7f79f602c740 10 client.25933 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:08:11.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.965+0000 7f79f602c740 10 client.25933 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:08:11.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.965+0000 7f79f602c740 20 client.25933 populate_metadata read hostname 'smithi043' 2024-09-19T05:08:11.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.965+0000 7f79f602c740 1 --2- 172.21.15.43:0/3373684689 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55d7ab870c00 0x55d7ab872ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:11.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.965+0000 7f79f602c740 4 client.25933 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:08:11.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.965+0000 7f79f602c740 1 -- 172.21.15.43:0/3373684689 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7ab7c9950 con 0x55d7ab870c00 2024-09-19T05:08:11.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.965+0000 7f79f0a00640 1 --2- 172.21.15.43:0/3373684689 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55d7ab870c00 0x55d7ab872ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:11.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.966+0000 7f79f0a00640 1 --2- 172.21.15.43:0/3373684689 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55d7ab870c00 0x55d7ab872ff0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d8950d48e102596 server_cookie=bde5f51890e77435 in_seq=0 out_seq=0 2024-09-19T05:08:11.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.966+0000 7f79d2000640 10 client.25933 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:08:11.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.966+0000 7f79d2000640 1 -- 172.21.15.43:0/3373684689 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d7ab7c9950 con 0x55d7ab870c00 2024-09-19T05:08:11.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.966+0000 7f79d2000640 10 client.25933 handle_command_reply: tid=0 2024-09-19T05:08:11.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.966+0000 7f79f602c740 1 client.25933 shutdown 2024-09-19T05:08:11.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.967+0000 7f79f602c740 1 -- 172.21.15.43:0/3373684689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7ab855b90 msgr2=0x55d7ab85dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:11.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.967+0000 7f79f602c740 1 --2- 172.21.15.43:0/3373684689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7ab855b90 0x55d7ab85dca0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f79d8013f50 tx=0x7f79d8005f90 comp rx=0 tx=0).stop 2024-09-19T05:08:11.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.967+0000 7f79f602c740 1 -- 172.21.15.43:0/3373684689 shutdown_connections 2024-09-19T05:08:11.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.967+0000 7f79f602c740 1 --2- 172.21.15.43:0/3373684689 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55d7ab870c00 0x55d7ab872ff0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:11.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.967+0000 7f79f602c740 1 --2- 172.21.15.43:0/3373684689 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7ab860d70 0x55d7ab85e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:11.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.967+0000 7f79f602c740 1 --2- 172.21.15.43:0/3373684689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7ab8564b0 0x55d7ab85e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:11.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.967+0000 7f79f602c740 1 --2- 172.21.15.43:0/3373684689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7ab855b90 0x55d7ab85dca0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:11.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.967+0000 7f79d2000640 0 client.25933 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:08:11.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.967+0000 7f79f602c740 1 -- 172.21.15.43:0/3373684689 >> 172.21.15.43:0/3373684689 conn(0x55d7ab852120 msgr2=0x55d7ab857e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:11.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.967+0000 7f79f602c740 1 -- 172.21.15.43:0/3373684689 shutdown_connections 2024-09-19T05:08:11.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.967+0000 7f79f602c740 1 -- 172.21.15.43:0/3373684689 wait complete. 2024-09-19T05:08:11.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.967+0000 7f79f602c740 20 client.25933 trim_cache size 0 max 16384 2024-09-19T05:08:11.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.986+0000 7f79f602c740 1 Processor -- start 2024-09-19T05:08:11.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.986+0000 7f79f602c740 1 -- start start 2024-09-19T05:08:11.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.987+0000 7f79f602c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7ab855b90 0x55d7ab8788c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:11.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.987+0000 7f79f602c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7ab8564b0 0x55d7ab878e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:11.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.987+0000 7f79f602c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7ab860d70 0x55d7ab85af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:11.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.987+0000 7f79f602c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f79ec1afad0 con 0x55d7ab860d70 2024-09-19T05:08:11.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.987+0000 7f79f602c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f79ec1b1610 con 0x55d7ab855b90 2024-09-19T05:08:11.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.987+0000 7f79f602c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f79ec1b01a0 con 0x55d7ab8564b0 2024-09-19T05:08:11.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.987+0000 7f79ebe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7ab860d70 0x55d7ab85af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:11.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.987+0000 7f79f0a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7ab8564b0 0x55d7ab878e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:11.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.987+0000 7f79ebe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7ab860d70 0x55d7ab85af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56946/0 (socket says 172.21.15.43:56946) 2024-09-19T05:08:11.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.987+0000 7f79ebe00640 1 -- 172.21.15.43:0/2372189750 learned_addr learned my addr 172.21.15.43:0/2372189750 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:11.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.987+0000 7f79f0a00640 1 --2- 172.21.15.43:0/2372189750 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7ab8564b0 0x55d7ab878e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:59854/0 (socket says 172.21.15.43:59854) 2024-09-19T05:08:11.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.987+0000 7f79ebe00640 1 --2- 172.21.15.43:0/2372189750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7ab855b90 0x55d7ab8788c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:11.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.988+0000 7f79f0a00640 1 -- 172.21.15.43:0/2372189750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7ab855b90 msgr2=0x55d7ab8788c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:11.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.988+0000 7f79f0a00640 1 --2- 172.21.15.43:0/2372189750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7ab855b90 0x55d7ab8788c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:11.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.988+0000 7f79f0a00640 1 -- 172.21.15.43:0/2372189750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7ab860d70 msgr2=0x55d7ab85af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:11.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.988+0000 7f79f0a00640 1 --2- 172.21.15.43:0/2372189750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7ab860d70 0x55d7ab85af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:11.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.988+0000 7f79f0a00640 1 -- 172.21.15.43:0/2372189750 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7ab852db0 con 0x55d7ab8564b0 2024-09-19T05:08:11.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.988+0000 7f79ebe00640 1 --2- 172.21.15.43:0/2372189750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7ab860d70 0x55d7ab85af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:11.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.988+0000 7f79f0a00640 1 --2- 172.21.15.43:0/2372189750 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7ab8564b0 0x55d7ab878e00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f79e0006010 tx=0x7f79e0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94a86e08a9008bfc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:11.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.988+0000 7f79dd200640 1 -- 172.21.15.43:0/2372189750 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e0040040 con 0x55d7ab8564b0 2024-09-19T05:08:11.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.988+0000 7f79dd200640 1 -- 172.21.15.43:0/2372189750 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79e0031670 con 0x55d7ab8564b0 2024-09-19T05:08:11.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.988+0000 7f79dd200640 1 -- 172.21.15.43:0/2372189750 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e0035040 con 0x55d7ab8564b0 2024-09-19T05:08:11.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.989+0000 7f79f602c740 1 -- 172.21.15.43:0/2372189750 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7ab8564b0 msgr2=0x55d7ab878e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:11.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.989+0000 7f79f602c740 1 --2- 172.21.15.43:0/2372189750 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7ab8564b0 0x55d7ab878e00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f79e0006010 tx=0x7f79e0000bc0 comp rx=0 tx=0).stop 2024-09-19T05:08:11.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.989+0000 7f79f602c740 1 -- 172.21.15.43:0/2372189750 shutdown_connections 2024-09-19T05:08:11.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.989+0000 7f79f602c740 1 --2- 172.21.15.43:0/2372189750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7ab860d70 0x55d7ab85af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:11.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.989+0000 7f79f602c740 1 --2- 172.21.15.43:0/2372189750 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7ab8564b0 0x55d7ab878e00 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:11.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.989+0000 7f79f602c740 1 --2- 172.21.15.43:0/2372189750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7ab855b90 0x55d7ab8788c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:11.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.989+0000 7f79f602c740 1 -- 172.21.15.43:0/2372189750 >> 172.21.15.43:0/2372189750 conn(0x55d7ab852120 msgr2=0x55d7ab85c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:11.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.989+0000 7f79f602c740 1 -- 172.21.15.43:0/2372189750 shutdown_connections 2024-09-19T05:08:11.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.989+0000 7f79f602c740 1 -- 172.21.15.43:0/2372189750 wait complete. 2024-09-19T05:08:11.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.990+0000 7f79f602c740 1 -- start start 2024-09-19T05:08:11.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.990+0000 7f79f602c740 1 Processor -- start 2024-09-19T05:08:11.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.990+0000 7f79f602c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7ab855b90 0x55d7ab85ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:11.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.990+0000 7f79f602c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7ab8564b0 0x55d7ab85b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:11.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.991+0000 7f79f602c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7ab860d70 0x55d7ab85b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:11.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.991+0000 7f79f602c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f79ec1b0f40 con 0x55d7ab860d70 2024-09-19T05:08:11.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.991+0000 7f79f602c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f79ec1b0870 con 0x55d7ab8564b0 2024-09-19T05:08:11.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.991+0000 7f79f602c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f79ec1adab0 con 0x55d7ab855b90 2024-09-19T05:08:11.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.991+0000 7f79f602c740 10 client.0 _ll_register_callbacks cb 0x7f79d8013d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:11.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.991+0000 7f79ebe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7ab860d70 0x55d7ab85b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:11.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.991+0000 7f79f0a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7ab8564b0 0x55d7ab85b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:11.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.991+0000 7f79ebe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7ab860d70 0x55d7ab85b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56950/0 (socket says 172.21.15.43:56950) 2024-09-19T05:08:11.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.991+0000 7f79f0a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7ab8564b0 0x55d7ab85b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:52488/0 (socket says 172.21.15.43:52488) 2024-09-19T05:08:11.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.991+0000 7f79ebe00640 1 -- 172.21.15.43:0/48158382 learned_addr learned my addr 172.21.15.43:0/48158382 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:11.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.991+0000 7f79ebe00640 1 --2- 172.21.15.43:0/48158382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7ab855b90 0x55d7ab85ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:11.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.991+0000 7f79f0a00640 1 -- 172.21.15.43:0/48158382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7ab855b90 msgr2=0x55d7ab85ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:11.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.991+0000 7f79f0a00640 1 --2- 172.21.15.43:0/48158382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7ab855b90 0x55d7ab85ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:11.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.992+0000 7f79f0a00640 1 -- 172.21.15.43:0/48158382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7ab860d70 msgr2=0x55d7ab85b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:11.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.992+0000 7f79f0a00640 1 --2- 172.21.15.43:0/48158382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7ab860d70 0x55d7ab85b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:11.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.992+0000 7f79f0a00640 1 -- 172.21.15.43:0/48158382 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7ab8e3170 con 0x55d7ab8564b0 2024-09-19T05:08:11.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.992+0000 7f79ebe00640 1 --2- 172.21.15.43:0/48158382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7ab855b90 0x55d7ab85ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:11.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.992+0000 7f79f0a00640 1 --2- 172.21.15.43:0/48158382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7ab8564b0 0x55d7ab85b350 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x55d7ab86d960 tx=0x7f79e0005df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48a2c6c64ac61856 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:11.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.992+0000 7f79d2000640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:08:11.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.992+0000 7f79d2000640 1 -- 172.21.15.43:0/48158382 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e0035050 con 0x55d7ab8564b0 2024-09-19T05:08:11.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.992+0000 7f79d2000640 1 -- 172.21.15.43:0/48158382 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f79e00055b0 con 0x55d7ab8564b0 2024-09-19T05:08:11.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.992+0000 7f79f602c740 1 -- 172.21.15.43:0/48158382 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79e0031670 con 0x55d7ab8564b0 2024-09-19T05:08:11.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.993+0000 7f79d2000640 1 -- 172.21.15.43:0/48158382 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f79e0066050 con 0x55d7ab8564b0 2024-09-19T05:08:11.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.993+0000 7f79f602c740 10 client.25997 fetch_fsmap learned FSMap version 29 2024-09-19T05:08:11.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.993+0000 7f79f602c740 1 -- 172.21.15.43:0/48158382 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d7ab8e4480 con 0x55d7ab8564b0 2024-09-19T05:08:11.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.993+0000 7f79d2000640 1 -- 172.21.15.43:0/48158382 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79e0031670 con 0x55d7ab8564b0 2024-09-19T05:08:11.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.993+0000 7f79d2000640 1 -- 172.21.15.43:0/48158382 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f79e004f4d0 con 0x55d7ab8564b0 2024-09-19T05:08:12.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.993+0000 7f79f602c740 10 client.25997 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:08:12.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.993+0000 7f79f602c740 10 client.25997 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:08:12.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.993+0000 7f79f602c740 20 client.25997 populate_metadata read hostname 'smithi043' 2024-09-19T05:08:12.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.993+0000 7f79f602c740 1 --2- 172.21.15.43:0/48158382 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55d7ab8e47b0 0x55d7ab8e6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:12.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.993+0000 7f79f602c740 4 client.25997 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:08:12.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.993+0000 7f79f602c740 1 -- 172.21.15.43:0/48158382 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d7ab7c3e60 con 0x55d7ab8e47b0 2024-09-19T05:08:12.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.994+0000 7f79ebe00640 1 --2- 172.21.15.43:0/48158382 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55d7ab8e47b0 0x55d7ab8e6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:12.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.994+0000 7f79d2000640 10 client.25997 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:08:12.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.994+0000 7f79ebe00640 1 --2- 172.21.15.43:0/48158382 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55d7ab8e47b0 0x55d7ab8e6ba0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9991435839762a26 server_cookie=dfd438984c51dbaa in_seq=0 out_seq=0 2024-09-19T05:08:12.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.994+0000 7f79d2000640 1 -- 172.21.15.43:0/48158382 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d7ab7c3e60 con 0x55d7ab8e47b0 2024-09-19T05:08:12.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.994+0000 7f79d2000640 10 client.25997 handle_command_reply: tid=0 2024-09-19T05:08:12.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.994+0000 7f79f602c740 1 client.25997 shutdown 2024-09-19T05:08:12.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.995+0000 7f79f602c740 1 -- 172.21.15.43:0/48158382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7ab8564b0 msgr2=0x55d7ab85b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:12.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.995+0000 7f79f602c740 1 --2- 172.21.15.43:0/48158382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7ab8564b0 0x55d7ab85b350 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x55d7ab86d960 tx=0x7f79e0005df0 comp rx=0 tx=0).stop 2024-09-19T05:08:12.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.995+0000 7f79f602c740 1 -- 172.21.15.43:0/48158382 shutdown_connections 2024-09-19T05:08:12.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.995+0000 7f79f602c740 1 --2- 172.21.15.43:0/48158382 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55d7ab8e47b0 0x55d7ab8e6ba0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:12.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.995+0000 7f79f602c740 1 --2- 172.21.15.43:0/48158382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7ab860d70 0x55d7ab85b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:12.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.995+0000 7f79f602c740 1 --2- 172.21.15.43:0/48158382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7ab8564b0 0x55d7ab85b350 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:12.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.995+0000 7f79f602c740 1 --2- 172.21.15.43:0/48158382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7ab855b90 0x55d7ab85ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:12.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.995+0000 7f79d2000640 0 client.25997 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:08:12.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.995+0000 7f79f602c740 1 -- 172.21.15.43:0/48158382 >> 172.21.15.43:0/48158382 conn(0x55d7ab852120 msgr2=0x55d7ab85c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:12.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.995+0000 7f79f602c740 1 -- 172.21.15.43:0/48158382 shutdown_connections 2024-09-19T05:08:12.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.995+0000 7f79f602c740 1 -- 172.21.15.43:0/48158382 wait complete. 2024-09-19T05:08:12.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.995+0000 7f79f602c740 20 client.25997 trim_cache size 0 max 16384 2024-09-19T05:08:12.002 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:08:12.002 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T05:08:12.002 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-09-19T05:08:12.002 INFO:teuthology.orchestra.run.smithi043.stdout: "086b4d5b-2d76-4a69-8f1f-a6fbeed795cf": { 2024-09-19T05:08:12.002 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-09-19T05:08:12.002 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "086b4d5b-2d76-4a69-8f1f-a6fbeed795cf", 2024-09-19T05:08:12.002 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-09-19T05:08:12.002 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:08:12.002 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:08:12.002 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:08:12.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.996+0000 7f79f3400640 1 -- 172.21.15.43:0/3340941125 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f79d40a3430 msgr2=0x7f79d40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:12.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.996+0000 7f79f3400640 1 --2- 172.21.15.43:0/3340941125 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f79d40a3430 0x7f79d40a58f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f79e40cda10 tx=0x7f79e4005b40 comp rx=0 tx=0).stop 2024-09-19T05:08:12.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.996+0000 7f79f3400640 1 -- 172.21.15.43:0/3340941125 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79ec106f10 msgr2=0x7f79ec19ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:12.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.996+0000 7f79f3400640 1 --2- 172.21.15.43:0/3340941125 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79ec106f10 0x7f79ec19ef80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f79e400eb10 tx=0x7f79e400c2d0 comp rx=0 tx=0).stop 2024-09-19T05:08:12.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.996+0000 7f79f3400640 1 -- 172.21.15.43:0/3340941125 shutdown_connections 2024-09-19T05:08:12.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.996+0000 7f79f3400640 1 --2- 172.21.15.43:0/3340941125 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f79d40a3430 0x7f79d40a58f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:12.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.996+0000 7f79f3400640 1 --2- 172.21.15.43:0/3340941125 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79ec106f10 0x7f79ec19ef80 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:12.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.996+0000 7f79f3400640 1 --2- 172.21.15.43:0/3340941125 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79ec102790 0x7f79ec19abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:12.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.996+0000 7f79f3400640 1 --2- 172.21.15.43:0/3340941125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79ec101de0 0x7f79ec19a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:12.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.996+0000 7f79f3400640 1 -- 172.21.15.43:0/3340941125 >> 172.21.15.43:0/3340941125 conn(0x7f79ec0fd8e0 msgr2=0x7f79ec0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:12.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.996+0000 7f79f3400640 1 -- 172.21.15.43:0/3340941125 shutdown_connections 2024-09-19T05:08:12.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:11.996+0000 7f79f3400640 1 -- 172.21.15.43:0/3340941125 wait complete. 2024-09-19T05:08:12.857 INFO:tasks.cephfs.filesystem:scrub status for tag:086b4d5b-2d76-4a69-8f1f-a6fbeed795cf - {'path': '/', 'tag': '086b4d5b-2d76-4a69-8f1f-a6fbeed795cf', 'options': 'recursive,force'} 2024-09-19T05:08:13.649 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-19T05:08:13.649 INFO:tasks.workunit.client.0.smithi043.stdout: Children see throughput for 1 re-readers = 3512807.00 kB/sec 2024-09-19T05:08:13.649 INFO:tasks.workunit.client.0.smithi043.stdout: Parent sees throughput for 1 re-readers = 3511933.72 kB/sec 2024-09-19T05:08:13.649 INFO:tasks.workunit.client.0.smithi043.stdout: Min throughput per process = 3512807.00 kB/sec 2024-09-19T05:08:13.649 INFO:tasks.workunit.client.0.smithi043.stdout: Max throughput per process = 3512807.00 kB/sec 2024-09-19T05:08:13.649 INFO:tasks.workunit.client.0.smithi043.stdout: Avg throughput per process = 3512807.00 kB/sec 2024-09-19T05:08:13.649 INFO:tasks.workunit.client.0.smithi043.stdout: Min xfer = 10485760.00 kB 2024-09-19T05:08:13.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:08:13 smithi096 ceph-mon[35085]: pgmap v110: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 121 MiB/s wr, 53 op/s 2024-09-19T05:08:13.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:08:13 smithi096 ceph-mon[35085]: scrub summary: idle 2024-09-19T05:08:13.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:08:13 smithi043 ceph-mon[30800]: pgmap v110: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 121 MiB/s wr, 53 op/s 2024-09-19T05:08:13.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:08:13 smithi043 ceph-mon[30800]: scrub summary: idle 2024-09-19T05:08:13.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:08:13 smithi151 ceph-mon[35425]: pgmap v110: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 121 MiB/s wr, 53 op/s 2024-09-19T05:08:13.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:08:13 smithi151 ceph-mon[35425]: scrub summary: idle 2024-09-19T05:08:15.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:08:15 smithi096 ceph-mon[35085]: pgmap v111: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 121 MiB/s wr, 58 op/s 2024-09-19T05:08:15.785 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:08:15 smithi043 ceph-mon[30800]: pgmap v111: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 121 MiB/s wr, 58 op/s 2024-09-19T05:08:15.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:08:15 smithi151 ceph-mon[35425]: pgmap v111: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 121 MiB/s wr, 58 op/s 2024-09-19T05:08:17.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:08:17 smithi096 ceph-mon[35085]: pgmap v112: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-19T05:08:17.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:08:17 smithi043 ceph-mon[30800]: pgmap v112: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-19T05:08:17.904 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:08:17 smithi151 ceph-mon[35425]: pgmap v112: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-19T05:08:19.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:08:19 smithi096 ceph-mon[35085]: pgmap v113: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-19T05:08:19.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:08:19 smithi043 ceph-mon[30800]: pgmap v113: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-19T05:08:19.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:08:19 smithi151 ceph-mon[35425]: pgmap v113: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-19T05:08:21.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:08:21 smithi096 ceph-mon[35085]: pgmap v114: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 3.0 KiB/s wr, 13 op/s 2024-09-19T05:08:21.782 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:08:21 smithi043 ceph-mon[30800]: pgmap v114: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 3.0 KiB/s wr, 13 op/s 2024-09-19T05:08:21.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:08:21 smithi151 ceph-mon[35425]: pgmap v114: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 3.0 KiB/s wr, 13 op/s 2024-09-19T05:08:23.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:08:23 smithi096 ceph-mon[35085]: pgmap v115: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-09-19T05:08:23.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:08:23 smithi043 ceph-mon[30800]: pgmap v115: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-09-19T05:08:23.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:08:23 smithi151 ceph-mon[35425]: pgmap v115: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-09-19T05:08:24.539 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-19T05:08:24.539 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-19T05:08:24.539 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-19T05:08:24.539 INFO:tasks.workunit.client.0.smithi043.stdout:iozone test complete. 2024-09-19T05:08:24.542 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-19T05:08:24.542 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-19T05:08:24.609 INFO:tasks.workunit:Stopping ['suites/iozone.sh'] on client.0... 2024-09-19T05:08:24.609 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-19T05:08:25.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:08:25 smithi096 ceph-mon[35085]: pgmap v116: 97 pgs: 97 active+clean; 16 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.6 KiB/s wr, 227 op/s 2024-09-19T05:08:25.732 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:08:25 smithi043 ceph-mon[30800]: pgmap v116: 97 pgs: 97 active+clean; 16 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.6 KiB/s wr, 227 op/s 2024-09-19T05:08:25.734 DEBUG:teuthology.parallel:result is None 2024-09-19T05:08:25.735 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-19T05:08:25.802 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-19T05:08:25.803 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-19T05:08:25.813 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-19T05:08:25.813 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T05:08:25.813 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi043.front.sepia.ceph.com 2024-09-19T05:08:25.813 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-19T05:08:25.872 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-19T05:08:25.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:08:25 smithi151 ceph-mon[35425]: pgmap v116: 97 pgs: 97 active+clean; 16 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.6 KiB/s wr, 227 op/s 2024-09-19T05:08:27.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:08:27 smithi096 ceph-mon[35085]: pgmap v117: 97 pgs: 97 active+clean; 16 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.6 KiB/s wr, 222 op/s 2024-09-19T05:08:27.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:08:27 smithi043 ceph-mon[30800]: pgmap v117: 97 pgs: 97 active+clean; 16 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.6 KiB/s wr, 222 op/s 2024-09-19T05:08:27.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:08:27 smithi151 ceph-mon[35425]: pgmap v117: 97 pgs: 97 active+clean; 16 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.6 KiB/s wr, 222 op/s 2024-09-19T05:08:29.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:08:29 smithi096 ceph-mon[35085]: pgmap v118: 97 pgs: 97 active+clean; 16 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.6 KiB/s wr, 222 op/s 2024-09-19T05:08:29.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:08:29 smithi043 ceph-mon[30800]: pgmap v118: 97 pgs: 97 active+clean; 16 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.6 KiB/s wr, 222 op/s 2024-09-19T05:08:29.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:08:29 smithi151 ceph-mon[35425]: pgmap v118: 97 pgs: 97 active+clean; 16 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.6 KiB/s wr, 222 op/s 2024-09-19T05:08:31.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:08:31 smithi096 ceph-mon[35085]: pgmap v119: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 KiB/s wr, 227 op/s 2024-09-19T05:08:31.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:08:31 smithi043 ceph-mon[30800]: pgmap v119: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 KiB/s wr, 227 op/s 2024-09-19T05:08:31.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:08:31 smithi151 ceph-mon[35425]: pgmap v119: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 KiB/s wr, 227 op/s 2024-09-19T05:08:33.723 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:08:33 smithi096 ceph-mon[35085]: pgmap v120: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 4.2 KiB/s wr, 223 op/s 2024-09-19T05:08:33.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:08:33 smithi043 ceph-mon[30800]: pgmap v120: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 4.2 KiB/s wr, 223 op/s 2024-09-19T05:08:33.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:08:33 smithi151 ceph-mon[35425]: pgmap v120: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 4.2 KiB/s wr, 223 op/s 2024-09-19T05:08:35.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:08:35 smithi043 ceph-mon[30800]: pgmap v121: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 KiB/s wr, 227 op/s 2024-09-19T05:08:35.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:08:35 smithi151 ceph-mon[35425]: pgmap v121: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 KiB/s wr, 227 op/s 2024-09-19T05:08:35.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:08:35 smithi096 ceph-mon[35085]: pgmap v121: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 KiB/s wr, 227 op/s 2024-09-19T05:08:37.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:08:37 smithi043 ceph-mon[30800]: pgmap v122: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2024-09-19T05:08:37.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:08:37 smithi151 ceph-mon[35425]: pgmap v122: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2024-09-19T05:08:37.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:08:37 smithi096 ceph-mon[35085]: pgmap v122: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2024-09-19T05:08:39.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:08:39 smithi043 ceph-mon[30800]: pgmap v123: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2024-09-19T05:08:39.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:08:39 smithi151 ceph-mon[35425]: pgmap v123: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2024-09-19T05:08:39.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:08:39 smithi096 ceph-mon[35085]: pgmap v123: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2024-09-19T05:08:41.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:08:41 smithi043 ceph-mon[30800]: pgmap v124: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 3.8 KiB/s wr, 13 op/s 2024-09-19T05:08:41.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:08:41 smithi151 ceph-mon[35425]: pgmap v124: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 3.8 KiB/s wr, 13 op/s 2024-09-19T05:08:41.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:08:41 smithi096 ceph-mon[35085]: pgmap v124: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 3.8 KiB/s wr, 13 op/s 2024-09-19T05:08:42.858 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:08:43.251 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:08:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:08:43 smithi043 ceph-mon[30800]: pgmap v125: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-19T05:08:43.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.850+0000 7f6264800640 1 -- 172.21.15.43:0/4223452329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f625c102790 msgr2=0x7f625c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:43.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.850+0000 7f6264800640 1 --2- 172.21.15.43:0/4223452329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f625c102790 0x7f625c1067e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f624c009900 tx=0x7f624c02a410 comp rx=0 tx=0).stop 2024-09-19T05:08:43.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.850+0000 7f6264800640 1 -- 172.21.15.43:0/4223452329 shutdown_connections 2024-09-19T05:08:43.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.850+0000 7f6264800640 1 --2- 172.21.15.43:0/4223452329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f625c106f10 0x7f625c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:43.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.850+0000 7f6264800640 1 --2- 172.21.15.43:0/4223452329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f625c102790 0x7f625c1067e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:43.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.850+0000 7f6264800640 1 --2- 172.21.15.43:0/4223452329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f625c101de0 0x7f625c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:43.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.850+0000 7f6264800640 1 -- 172.21.15.43:0/4223452329 >> 172.21.15.43:0/4223452329 conn(0x7f625c0fd8c0 msgr2=0x7f625c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:43.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.850+0000 7f6264800640 1 -- 172.21.15.43:0/4223452329 shutdown_connections 2024-09-19T05:08:43.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.850+0000 7f6264800640 1 -- 172.21.15.43:0/4223452329 wait complete. 2024-09-19T05:08:43.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.851+0000 7f6264800640 1 Processor -- start 2024-09-19T05:08:43.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.851+0000 7f6264800640 1 -- start start 2024-09-19T05:08:43.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.852+0000 7f6264800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f625c101de0 0x7f625c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:43.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.852+0000 7f6264800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f625c102790 0x7f625c1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:43.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.852+0000 7f6264800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f625c106f10 0x7f625c19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:43.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.852+0000 7f6264800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f625c111fc0 con 0x7f625c101de0 2024-09-19T05:08:43.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.852+0000 7f6264800640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f625c111e40 con 0x7f625c106f10 2024-09-19T05:08:43.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.852+0000 7f6264800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f625c112140 con 0x7f625c102790 2024-09-19T05:08:43.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.852+0000 7f6261e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f625c101de0 0x7f625c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:43.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.852+0000 7f6261400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f625c102790 0x7f625c1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:43.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.852+0000 7f6261e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f625c101de0 0x7f625c198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33750/0 (socket says 172.21.15.43:33750) 2024-09-19T05:08:43.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.852+0000 7f6261e00640 1 -- 172.21.15.43:0/1160008873 learned_addr learned my addr 172.21.15.43:0/1160008873 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:43.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.852+0000 7f6262800640 1 --2- 172.21.15.43:0/1160008873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f625c106f10 0x7f625c19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:43.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.853+0000 7f6261400640 1 -- 172.21.15.43:0/1160008873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f625c106f10 msgr2=0x7f625c19cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:43.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.853+0000 7f6261400640 1 --2- 172.21.15.43:0/1160008873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f625c106f10 0x7f625c19cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:43.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.853+0000 7f6261400640 1 -- 172.21.15.43:0/1160008873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f625c101de0 msgr2=0x7f625c198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:43.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.853+0000 7f6261400640 1 --2- 172.21.15.43:0/1160008873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f625c101de0 0x7f625c198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:43.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.853+0000 7f6261400640 1 -- 172.21.15.43:0/1160008873 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f625c19d410 con 0x7f625c102790 2024-09-19T05:08:43.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.853+0000 7f6261e00640 1 --2- 172.21.15.43:0/1160008873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f625c101de0 0x7f625c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:43.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.853+0000 7f6261400640 1 --2- 172.21.15.43:0/1160008873 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f625c102790 0x7f625c1989a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f624c02a930 tx=0x7f624c003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:43.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.853+0000 7f6256a00640 1 -- 172.21.15.43:0/1160008873 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c03f070 con 0x7f625c102790 2024-09-19T05:08:43.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.853+0000 7f6264800640 1 -- 172.21.15.43:0/1160008873 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f625c19d6a0 con 0x7f625c102790 2024-09-19T05:08:43.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.853+0000 7f6256a00640 1 -- 172.21.15.43:0/1160008873 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f624c031c90 con 0x7f625c102790 2024-09-19T05:08:43.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.853+0000 7f6256a00640 1 -- 172.21.15.43:0/1160008873 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c035030 con 0x7f625c102790 2024-09-19T05:08:43.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.854+0000 7f6264800640 1 -- 172.21.15.43:0/1160008873 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f625c1a4fe0 con 0x7f625c102790 2024-09-19T05:08:43.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.857+0000 7f6267202740 1 Processor -- start 2024-09-19T05:08:43.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.857+0000 7f6256a00640 1 -- 172.21.15.43:0/1160008873 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f624c042070 con 0x7f625c102790 2024-09-19T05:08:43.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.857+0000 7f6267202740 1 -- start start 2024-09-19T05:08:43.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.857+0000 7f6267202740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630605a7fa0 0x5630605a8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:43.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.857+0000 7f6267202740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630605a88c0 0x5630605b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:43.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.857+0000 7f6267202740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630605b3180 0x5630605b5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:43.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.857+0000 7f6267202740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56306050a3e0 con 0x5630605a7fa0 2024-09-19T05:08:43.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.857+0000 7f6267202740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5630604b1d80 con 0x5630605a88c0 2024-09-19T05:08:43.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.857+0000 7f6267202740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5630605063a0 con 0x5630605b3180 2024-09-19T05:08:43.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.857+0000 7f6262800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630605a7fa0 0x5630605a8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:43.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.857+0000 7f6262800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630605a7fa0 0x5630605a8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33764/0 (socket says 172.21.15.43:33764) 2024-09-19T05:08:43.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.857+0000 7f6262800640 1 -- 172.21.15.43:0/2182992317 learned_addr learned my addr 172.21.15.43:0/2182992317 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:43.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.858+0000 7f6261e00640 1 --2- 172.21.15.43:0/2182992317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630605a88c0 0x5630605b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:43.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.858+0000 7f6261400640 1 --2- 172.21.15.43:0/2182992317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630605b3180 0x5630605b5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:43.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.858+0000 7f6262800640 1 -- 172.21.15.43:0/2182992317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630605b3180 msgr2=0x5630605b5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:43.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.858+0000 7f6262800640 1 --2- 172.21.15.43:0/2182992317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630605b3180 0x5630605b5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:43.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.858+0000 7f6262800640 1 -- 172.21.15.43:0/2182992317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630605a88c0 msgr2=0x5630605b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:43.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.858+0000 7f6262800640 1 --2- 172.21.15.43:0/2182992317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630605a88c0 0x5630605b2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:43.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.858+0000 7f6262800640 1 -- 172.21.15.43:0/2182992317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630604d0ae0 con 0x5630605a7fa0 2024-09-19T05:08:43.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.859+0000 7f6262800640 1 --2- 172.21.15.43:0/2182992317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630605a7fa0 0x5630605a8380 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6258009f50 tx=0x7f62580118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec678bf3d2901529 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:43.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.859+0000 7f6256a00640 1 --2- 172.21.15.43:0/1160008873 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f62440a33b0 0x7f62440a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:43.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.859+0000 7f6256a00640 1 -- 172.21.15.43:0/1160008873 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f624c0f06a0 con 0x7f625c102790 2024-09-19T05:08:43.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.859+0000 7f624b000640 1 -- 172.21.15.43:0/2182992317 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6258004b20 con 0x5630605a7fa0 2024-09-19T05:08:43.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.859+0000 7f6261e00640 1 --2- 172.21.15.43:0/1160008873 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f62440a33b0 0x7f62440a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:43.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.859+0000 7f624b000640 1 -- 172.21.15.43:0/2182992317 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6258004cc0 con 0x5630605a7fa0 2024-09-19T05:08:43.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.859+0000 7f624b000640 1 -- 172.21.15.43:0/2182992317 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6258005020 con 0x5630605a7fa0 2024-09-19T05:08:43.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.860+0000 7f6267202740 1 -- 172.21.15.43:0/2182992317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630605a7fa0 msgr2=0x5630605a8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:43.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.860+0000 7f6267202740 1 --2- 172.21.15.43:0/2182992317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630605a7fa0 0x5630605a8380 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6258009f50 tx=0x7f62580118d0 comp rx=0 tx=0).stop 2024-09-19T05:08:43.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.860+0000 7f6261e00640 1 --2- 172.21.15.43:0/1160008873 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f62440a33b0 0x7f62440a5870 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6250009620 tx=0x7f6250005f90 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:43.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.860+0000 7f6267202740 1 -- 172.21.15.43:0/2182992317 shutdown_connections 2024-09-19T05:08:43.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.860+0000 7f6267202740 1 --2- 172.21.15.43:0/2182992317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630605b3180 0x5630605b5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:43.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.860+0000 7f6267202740 1 --2- 172.21.15.43:0/2182992317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630605a88c0 0x5630605b2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:43.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.860+0000 7f6267202740 1 --2- 172.21.15.43:0/2182992317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630605a7fa0 0x5630605a8380 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:43.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.860+0000 7f6267202740 1 -- 172.21.15.43:0/2182992317 >> 172.21.15.43:0/2182992317 conn(0x5630605a4530 msgr2=0x5630605a4910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:43.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.860+0000 7f6267202740 1 -- 172.21.15.43:0/2182992317 shutdown_connections 2024-09-19T05:08:43.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.860+0000 7f6267202740 1 -- 172.21.15.43:0/2182992317 wait complete. 2024-09-19T05:08:43.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.860+0000 7f6267202740 1 -- start start 2024-09-19T05:08:43.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.861+0000 7f6267202740 1 Processor -- start 2024-09-19T05:08:43.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.861+0000 7f6267202740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630605a7fa0 0x5630605b00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:43.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.861+0000 7f6267202740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630605a88c0 0x5630605b0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:43.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.861+0000 7f6267202740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630605b3180 0x5630605b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:43.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.861+0000 7f6267202740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5630605b5df0 con 0x5630605a7fa0 2024-09-19T05:08:43.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.861+0000 7f6267202740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5630604c50f0 con 0x5630605a88c0 2024-09-19T05:08:43.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.861+0000 7f6267202740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5630605b5f70 con 0x5630605b3180 2024-09-19T05:08:43.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.861+0000 7f6267202740 10 client.0 _ll_register_callbacks cb 0x5630604b69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:43.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.861+0000 7f6261400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630605a88c0 0x5630605b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:43.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.861+0000 7f6261e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630605a7fa0 0x5630605b00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:43.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.862+0000 7f6261400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630605a88c0 0x5630605b0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:41238/0 (socket says 172.21.15.43:41238) 2024-09-19T05:08:43.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.862+0000 7f6261400640 1 -- 172.21.15.43:0/3226454984 learned_addr learned my addr 172.21.15.43:0/3226454984 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:43.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.862+0000 7f6262800640 1 --2- 172.21.15.43:0/3226454984 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630605b3180 0x5630605b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:43.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.862+0000 7f6261400640 1 -- 172.21.15.43:0/3226454984 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630605b3180 msgr2=0x5630605b0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:43.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.862+0000 7f6261400640 1 --2- 172.21.15.43:0/3226454984 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630605b3180 0x5630605b0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:43.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.862+0000 7f6261400640 1 -- 172.21.15.43:0/3226454984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630605a7fa0 msgr2=0x5630605b00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:43.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.862+0000 7f6261400640 1 --2- 172.21.15.43:0/3226454984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630605a7fa0 0x5630605b00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:43.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.862+0000 7f6261400640 1 -- 172.21.15.43:0/3226454984 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56306027b3c0 con 0x5630605a88c0 2024-09-19T05:08:43.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.862+0000 7f6261e00640 1 --2- 172.21.15.43:0/3226454984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630605a7fa0 0x5630605b00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:43.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.862+0000 7f6261400640 1 --2- 172.21.15.43:0/3226454984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630605a88c0 0x5630605b0610 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f624c0bc970 tx=0x7f624c0f4040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53f5192eb0bdb14e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:43.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.862+0000 7f623f400640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:08:43.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.862+0000 7f623f400640 1 -- 172.21.15.43:0/3226454984 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c049070 con 0x5630605a88c0 2024-09-19T05:08:43.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.863+0000 7f6267202740 1 -- 172.21.15.43:0/3226454984 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6258004cc0 con 0x5630605a88c0 2024-09-19T05:08:43.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.863+0000 7f623f400640 1 -- 172.21.15.43:0/3226454984 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f624c0b9050 con 0x5630605a88c0 2024-09-19T05:08:43.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.863+0000 7f623f400640 1 -- 172.21.15.43:0/3226454984 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f624c0b8c80 con 0x5630605a88c0 2024-09-19T05:08:43.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.863+0000 7f6267202740 10 client.26009 fetch_fsmap learned FSMap version 29 2024-09-19T05:08:43.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.863+0000 7f6267202740 1 -- 172.21.15.43:0/3226454984 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5630605c1b00 con 0x5630605a88c0 2024-09-19T05:08:43.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.863+0000 7f623f400640 1 -- 172.21.15.43:0/3226454984 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6258004cc0 con 0x5630605a88c0 2024-09-19T05:08:43.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.863+0000 7f623f400640 1 -- 172.21.15.43:0/3226454984 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f624c0a5ce0 con 0x5630605a88c0 2024-09-19T05:08:43.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.863+0000 7f6267202740 10 client.26009 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:08:43.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.863+0000 7f6267202740 10 client.26009 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:08:43.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.863+0000 7f6267202740 20 client.26009 populate_metadata read hostname 'smithi043' 2024-09-19T05:08:43.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.863+0000 7f6267202740 1 --2- 172.21.15.43:0/3226454984 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5630605c2ef0 0x5630605c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:43.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.863+0000 7f6267202740 4 client.26009 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:08:43.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.863+0000 7f6267202740 1 -- 172.21.15.43:0/3226454984 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563060517950 con 0x5630605c2ef0 2024-09-19T05:08:43.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.864+0000 7f6261e00640 1 --2- 172.21.15.43:0/3226454984 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5630605c2ef0 0x5630605c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:43.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.864+0000 7f6261e00640 1 --2- 172.21.15.43:0/3226454984 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5630605c2ef0 0x5630605c5370 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=926c7283d1f507e5 server_cookie=42a4a9a006673335 in_seq=0 out_seq=0 2024-09-19T05:08:43.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.864+0000 7f623f400640 10 client.26009 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:08:43.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.865+0000 7f623f400640 1 -- 172.21.15.43:0/3226454984 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563060517950 con 0x5630605c2ef0 2024-09-19T05:08:43.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.865+0000 7f623f400640 10 client.26009 handle_command_reply: tid=0 2024-09-19T05:08:43.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.865+0000 7f6267202740 1 client.26009 shutdown 2024-09-19T05:08:43.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.865+0000 7f6267202740 1 -- 172.21.15.43:0/3226454984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630605a88c0 msgr2=0x5630605b0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:43.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.865+0000 7f6267202740 1 --2- 172.21.15.43:0/3226454984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630605a88c0 0x5630605b0610 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f624c0bc970 tx=0x7f624c0f4040 comp rx=0 tx=0).stop 2024-09-19T05:08:43.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.865+0000 7f6267202740 1 -- 172.21.15.43:0/3226454984 shutdown_connections 2024-09-19T05:08:43.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.865+0000 7f6267202740 1 --2- 172.21.15.43:0/3226454984 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5630605c2ef0 0x5630605c5370 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:43.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.865+0000 7f6267202740 1 --2- 172.21.15.43:0/3226454984 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630605b3180 0x5630605b0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:43.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.865+0000 7f6267202740 1 --2- 172.21.15.43:0/3226454984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630605a88c0 0x5630605b0610 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:43.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.865+0000 7f6267202740 1 --2- 172.21.15.43:0/3226454984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630605a7fa0 0x5630605b00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:43.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.865+0000 7f6267202740 1 -- 172.21.15.43:0/3226454984 >> 172.21.15.43:0/3226454984 conn(0x5630605a4530 msgr2=0x5630605aa260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:43.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.865+0000 7f623f400640 0 client.26009 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:08:43.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.866+0000 7f6267202740 1 -- 172.21.15.43:0/3226454984 shutdown_connections 2024-09-19T05:08:43.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.866+0000 7f6267202740 1 -- 172.21.15.43:0/3226454984 wait complete. 2024-09-19T05:08:43.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.866+0000 7f6267202740 20 client.26009 trim_cache size 0 max 16384 2024-09-19T05:08:43.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.884+0000 7f6267202740 1 Processor -- start 2024-09-19T05:08:43.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.884+0000 7f6267202740 1 -- start start 2024-09-19T05:08:43.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.884+0000 7f6267202740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630605a7fa0 0x5630605cac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:43.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.884+0000 7f6267202740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630605a88c0 0x5630605cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:43.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.884+0000 7f6267202740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630605b3180 0x5630605ad390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:43.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.884+0000 7f6267202740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f625c1ab5e0 con 0x5630605a88c0 2024-09-19T05:08:43.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.884+0000 7f6267202740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f625c1ad120 con 0x5630605b3180 2024-09-19T05:08:43.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.884+0000 7f6267202740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f625c1abcb0 con 0x5630605a7fa0 2024-09-19T05:08:43.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.884+0000 7f6261400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630605a88c0 0x5630605cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:43.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.884+0000 7f6261400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630605a88c0 0x5630605cb1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33780/0 (socket says 172.21.15.43:33780) 2024-09-19T05:08:43.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.884+0000 7f6261400640 1 -- 172.21.15.43:0/380185257 learned_addr learned my addr 172.21.15.43:0/380185257 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:43.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.884+0000 7f6261e00640 1 --2- 172.21.15.43:0/380185257 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630605a7fa0 0x5630605cac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:43.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.884+0000 7f6262800640 1 --2- 172.21.15.43:0/380185257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630605b3180 0x5630605ad390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:43.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.885+0000 7f6261400640 1 -- 172.21.15.43:0/380185257 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630605a7fa0 msgr2=0x5630605cac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:43.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.885+0000 7f6261400640 1 --2- 172.21.15.43:0/380185257 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630605a7fa0 0x5630605cac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:43.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.885+0000 7f6261400640 1 -- 172.21.15.43:0/380185257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630605b3180 msgr2=0x5630605ad390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:43.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.885+0000 7f6261400640 1 --2- 172.21.15.43:0/380185257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630605b3180 0x5630605ad390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:43.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.885+0000 7f6261400640 1 -- 172.21.15.43:0/380185257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630605c59f0 con 0x5630605a88c0 2024-09-19T05:08:43.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.885+0000 7f6262800640 1 --2- 172.21.15.43:0/380185257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630605b3180 0x5630605ad390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:43.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.885+0000 7f6261400640 1 --2- 172.21.15.43:0/380185257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630605a88c0 0x5630605cb1c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f624c0bc6e0 tx=0x7f624c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e59f91170b0dca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:43.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.885+0000 7f624a600640 1 -- 172.21.15.43:0/380185257 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c049070 con 0x5630605a88c0 2024-09-19T05:08:43.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.885+0000 7f624a600640 1 -- 172.21.15.43:0/380185257 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f624c0f6220 con 0x5630605a88c0 2024-09-19T05:08:43.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.885+0000 7f624a600640 1 -- 172.21.15.43:0/380185257 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c0bc340 con 0x5630605a88c0 2024-09-19T05:08:43.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.886+0000 7f6267202740 1 -- 172.21.15.43:0/380185257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630605a88c0 msgr2=0x5630605cb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:43.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.886+0000 7f6267202740 1 --2- 172.21.15.43:0/380185257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630605a88c0 0x5630605cb1c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f624c0bc6e0 tx=0x7f624c005d50 comp rx=0 tx=0).stop 2024-09-19T05:08:43.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.886+0000 7f6267202740 1 -- 172.21.15.43:0/380185257 shutdown_connections 2024-09-19T05:08:43.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.886+0000 7f6267202740 1 --2- 172.21.15.43:0/380185257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630605b3180 0x5630605ad390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:43.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.886+0000 7f6267202740 1 --2- 172.21.15.43:0/380185257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630605a88c0 0x5630605cb1c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:43.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.886+0000 7f6267202740 1 --2- 172.21.15.43:0/380185257 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630605a7fa0 0x5630605cac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:43.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.886+0000 7f6267202740 1 -- 172.21.15.43:0/380185257 >> 172.21.15.43:0/380185257 conn(0x5630605a4530 msgr2=0x5630605b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:43.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.886+0000 7f6267202740 1 -- 172.21.15.43:0/380185257 shutdown_connections 2024-09-19T05:08:43.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.886+0000 7f6267202740 1 -- 172.21.15.43:0/380185257 wait complete. 2024-09-19T05:08:43.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.887+0000 7f6267202740 1 -- start start 2024-09-19T05:08:43.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.887+0000 7f6267202740 1 Processor -- start 2024-09-19T05:08:43.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.887+0000 7f6267202740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630605a7fa0 0x5630605bb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:43.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.887+0000 7f6267202740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630605a88c0 0x5630605bb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:43.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.887+0000 7f6267202740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630605b3180 0x5630605b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:43.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.887+0000 7f6267202740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f625c1aca50 con 0x5630605b3180 2024-09-19T05:08:43.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.887+0000 7f6267202740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f625c1ac380 con 0x5630605a88c0 2024-09-19T05:08:43.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.887+0000 7f6267202740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f625c1a95c0 con 0x5630605a7fa0 2024-09-19T05:08:43.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.887+0000 7f6267202740 10 client.0 _ll_register_callbacks cb 0x7f624c005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:43.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.888+0000 7f6262800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630605b3180 0x5630605b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:43.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.888+0000 7f6261e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630605a7fa0 0x5630605bb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:43.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.888+0000 7f6261e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630605a7fa0 0x5630605bb1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:37696/0 (socket says 172.21.15.43:37696) 2024-09-19T05:08:43.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.888+0000 7f6261e00640 1 -- 172.21.15.43:0/886673208 learned_addr learned my addr 172.21.15.43:0/886673208 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:43.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.888+0000 7f6262800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630605b3180 0x5630605b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33786/0 (socket says 172.21.15.43:33786) 2024-09-19T05:08:43.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.888+0000 7f6261400640 1 --2- 172.21.15.43:0/886673208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630605a88c0 0x5630605bb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:43.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.888+0000 7f6261e00640 1 -- 172.21.15.43:0/886673208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630605a88c0 msgr2=0x5630605bb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:43.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.888+0000 7f6261e00640 1 --2- 172.21.15.43:0/886673208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630605a88c0 0x5630605bb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:43.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.888+0000 7f6261e00640 1 -- 172.21.15.43:0/886673208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630605b3180 msgr2=0x5630605b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:43.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.888+0000 7f6261e00640 1 --2- 172.21.15.43:0/886673208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630605b3180 0x5630605b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:43.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.888+0000 7f6261e00640 1 -- 172.21.15.43:0/886673208 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630605a7db0 con 0x5630605a7fa0 2024-09-19T05:08:43.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.888+0000 7f6261e00640 1 --2- 172.21.15.43:0/886673208 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630605a7fa0 0x5630605bb1c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f625000f8d0 tx=0x7f62500046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93b0d17a2c505e4d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:43.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.888+0000 7f623f400640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:08:43.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.889+0000 7f623f400640 1 -- 172.21.15.43:0/886673208 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625002b7d0 con 0x5630605a7fa0 2024-09-19T05:08:43.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.889+0000 7f6267202740 1 -- 172.21.15.43:0/886673208 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f624c0f6220 con 0x5630605a7fa0 2024-09-19T05:08:43.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.889+0000 7f623f400640 1 -- 172.21.15.43:0/886673208 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f625002b970 con 0x5630605a7fa0 2024-09-19T05:08:43.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.889+0000 7f623f400640 1 -- 172.21.15.43:0/886673208 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f625002bb70 con 0x5630605a7fa0 2024-09-19T05:08:43.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.889+0000 7f6267202740 10 client.25957 fetch_fsmap learned FSMap version 29 2024-09-19T05:08:43.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.889+0000 7f6267202740 1 -- 172.21.15.43:0/886673208 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56306062e010 con 0x5630605a7fa0 2024-09-19T05:08:43.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.889+0000 7f623f400640 1 -- 172.21.15.43:0/886673208 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f624c0f6220 con 0x5630605a7fa0 2024-09-19T05:08:43.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.890+0000 7f623f400640 1 -- 172.21.15.43:0/886673208 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6250004c10 con 0x5630605a7fa0 2024-09-19T05:08:43.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.890+0000 7f6267202740 10 client.25957 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:08:43.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.890+0000 7f6267202740 10 client.25957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:08:43.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.890+0000 7f6267202740 20 client.25957 populate_metadata read hostname 'smithi043' 2024-09-19T05:08:43.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.890+0000 7f6267202740 1 --2- 172.21.15.43:0/886673208 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x56306062e790 0x563060630bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:43.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.890+0000 7f6267202740 4 client.25957 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:08:43.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.890+0000 7f6267202740 1 -- 172.21.15.43:0/886673208 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563060511e60 con 0x56306062e790 2024-09-19T05:08:43.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.890+0000 7f6261400640 1 --2- 172.21.15.43:0/886673208 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x56306062e790 0x563060630bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:43.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.891+0000 7f6261400640 1 --2- 172.21.15.43:0/886673208 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x56306062e790 0x563060630bb0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b280380d35c19f0b server_cookie=14b5c2f3ff4d8d6d in_seq=0 out_seq=0 2024-09-19T05:08:43.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.891+0000 7f623f400640 10 client.25957 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:08:43.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.891+0000 7f623f400640 1 -- 172.21.15.43:0/886673208 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563060511e60 con 0x56306062e790 2024-09-19T05:08:43.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.891+0000 7f623f400640 10 client.25957 handle_command_reply: tid=0 2024-09-19T05:08:43.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.891+0000 7f6267202740 1 client.25957 shutdown 2024-09-19T05:08:43.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.891+0000 7f6267202740 1 -- 172.21.15.43:0/886673208 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630605a7fa0 msgr2=0x5630605bb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:43.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.891+0000 7f6267202740 1 --2- 172.21.15.43:0/886673208 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630605a7fa0 0x5630605bb1c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f625000f8d0 tx=0x7f62500046c0 comp rx=0 tx=0).stop 2024-09-19T05:08:43.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.891+0000 7f6267202740 1 -- 172.21.15.43:0/886673208 shutdown_connections 2024-09-19T05:08:43.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.891+0000 7f6267202740 1 --2- 172.21.15.43:0/886673208 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x56306062e790 0x563060630bb0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:43.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.891+0000 7f6267202740 1 --2- 172.21.15.43:0/886673208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630605b3180 0x5630605b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:43.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.891+0000 7f6267202740 1 --2- 172.21.15.43:0/886673208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5630605a88c0 0x5630605bb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:43.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.891+0000 7f6267202740 1 --2- 172.21.15.43:0/886673208 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630605a7fa0 0x5630605bb1c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:43.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.891+0000 7f6267202740 1 -- 172.21.15.43:0/886673208 >> 172.21.15.43:0/886673208 conn(0x5630605a4530 msgr2=0x5630605a99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:43.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.891+0000 7f623f400640 0 client.25957 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:08:43.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.891+0000 7f6267202740 1 -- 172.21.15.43:0/886673208 shutdown_connections 2024-09-19T05:08:43.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.891+0000 7f6267202740 1 -- 172.21.15.43:0/886673208 wait complete. 2024-09-19T05:08:43.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.891+0000 7f6267202740 20 client.25957 trim_cache size 0 max 16384 2024-09-19T05:08:43.898 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:08:43.898 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-09-19T05:08:43.899 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-09-19T05:08:43.899 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:08:43.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.892+0000 7f6264800640 1 -- 172.21.15.43:0/1160008873 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f62440a33b0 msgr2=0x7f62440a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:43.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.892+0000 7f6264800640 1 --2- 172.21.15.43:0/1160008873 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f62440a33b0 0x7f62440a5870 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6250009620 tx=0x7f6250005f90 comp rx=0 tx=0).stop 2024-09-19T05:08:43.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.892+0000 7f6264800640 1 -- 172.21.15.43:0/1160008873 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f625c102790 msgr2=0x7f625c1989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:43.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.892+0000 7f6264800640 1 --2- 172.21.15.43:0/1160008873 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f625c102790 0x7f625c1989a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f624c02a930 tx=0x7f624c003b20 comp rx=0 tx=0).stop 2024-09-19T05:08:43.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.892+0000 7f6264800640 1 -- 172.21.15.43:0/1160008873 shutdown_connections 2024-09-19T05:08:43.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.892+0000 7f6264800640 1 --2- 172.21.15.43:0/1160008873 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f62440a33b0 0x7f62440a5870 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:43.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.892+0000 7f6264800640 1 --2- 172.21.15.43:0/1160008873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f625c106f10 0x7f625c19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:43.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.892+0000 7f6264800640 1 --2- 172.21.15.43:0/1160008873 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f625c102790 0x7f625c1989a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:43.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.892+0000 7f6264800640 1 --2- 172.21.15.43:0/1160008873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f625c101de0 0x7f625c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:43.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.892+0000 7f6264800640 1 -- 172.21.15.43:0/1160008873 >> 172.21.15.43:0/1160008873 conn(0x7f625c0fd8c0 msgr2=0x7f625c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:43.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.892+0000 7f6264800640 1 -- 172.21.15.43:0/1160008873 shutdown_connections 2024-09-19T05:08:43.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:43.892+0000 7f6264800640 1 -- 172.21.15.43:0/1160008873 wait complete. 2024-09-19T05:08:43.904 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:08:43 smithi151 ceph-mon[35425]: pgmap v125: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-19T05:08:43.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:08:43 smithi096 ceph-mon[35085]: pgmap v125: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-19T05:08:44.757 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T05:08:44.757 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:08:45.140 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:08:45.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:08:45 smithi043 ceph-mon[30800]: pgmap v126: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-19T05:08:45.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.726+0000 7f7a20000640 1 -- 172.21.15.43:0/3374039324 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a18077040 msgr2=0x7f7a181124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:45.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.726+0000 7f7a20000640 1 --2- 172.21.15.43:0/3374039324 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a18077040 0x7f7a181124f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7a04009900 tx=0x7f7a0402a3f0 comp rx=0 tx=0).stop 2024-09-19T05:08:45.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.726+0000 7f7a20000640 1 -- 172.21.15.43:0/3374039324 shutdown_connections 2024-09-19T05:08:45.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.726+0000 7f7a20000640 1 --2- 172.21.15.43:0/3374039324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a18112a30 0x7f7a18114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:45.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.726+0000 7f7a20000640 1 --2- 172.21.15.43:0/3374039324 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a18077040 0x7f7a181124f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:45.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.726+0000 7f7a20000640 1 --2- 172.21.15.43:0/3374039324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a18076720 0x7f7a18076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:45.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.726+0000 7f7a20000640 1 -- 172.21.15.43:0/3374039324 >> 172.21.15.43:0/3374039324 conn(0x7f7a180ffa90 msgr2=0x7f7a18101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:45.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.726+0000 7f7a20000640 1 -- 172.21.15.43:0/3374039324 shutdown_connections 2024-09-19T05:08:45.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.726+0000 7f7a20000640 1 -- 172.21.15.43:0/3374039324 wait complete. 2024-09-19T05:08:45.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.727+0000 7f7a20000640 1 Processor -- start 2024-09-19T05:08:45.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.727+0000 7f7a20000640 1 -- start start 2024-09-19T05:08:45.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.728+0000 7f7a20000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a18076720 0x7f7a181a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:45.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.728+0000 7f7a20000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a18077040 0x7f7a181a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:45.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.728+0000 7f7a20000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a18112a30 0x7f7a1819e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:45.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.728+0000 7f7a20000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7a181185b0 con 0x7f7a18077040 2024-09-19T05:08:45.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.728+0000 7f7a20000640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7a18118430 con 0x7f7a18112a30 2024-09-19T05:08:45.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.728+0000 7f7a20000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7a18118730 con 0x7f7a18076720 2024-09-19T05:08:45.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.728+0000 7f7a1cc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a18077040 0x7f7a181a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:45.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.728+0000 7f7a1d600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a18076720 0x7f7a181a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:45.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.728+0000 7f7a1cc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a18077040 0x7f7a181a4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33806/0 (socket says 172.21.15.43:33806) 2024-09-19T05:08:45.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.728+0000 7f7a1cc00640 1 -- 172.21.15.43:0/2620992739 learned_addr learned my addr 172.21.15.43:0/2620992739 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:45.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.729+0000 7f7a1e000640 1 --2- 172.21.15.43:0/2620992739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a18112a30 0x7f7a1819e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:45.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.729+0000 7f7a1d600640 1 -- 172.21.15.43:0/2620992739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a18112a30 msgr2=0x7f7a1819e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:45.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.729+0000 7f7a1d600640 1 --2- 172.21.15.43:0/2620992739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a18112a30 0x7f7a1819e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:45.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.729+0000 7f7a1d600640 1 -- 172.21.15.43:0/2620992739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a18077040 msgr2=0x7f7a181a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:45.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.729+0000 7f7a1d600640 1 --2- 172.21.15.43:0/2620992739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a18077040 0x7f7a181a4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:45.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.729+0000 7f7a1d600640 1 -- 172.21.15.43:0/2620992739 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a1819eff0 con 0x7f7a18076720 2024-09-19T05:08:45.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.729+0000 7f7a1cc00640 1 --2- 172.21.15.43:0/2620992739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a18077040 0x7f7a181a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:45.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.729+0000 7f7a1d600640 1 --2- 172.21.15.43:0/2620992739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a18076720 0x7f7a181a4700 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c00e9f0 tx=0x7f7a0c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:45.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.729+0000 7f7a16000640 1 -- 172.21.15.43:0/2620992739 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c00ccc0 con 0x7f7a18076720 2024-09-19T05:08:45.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.730+0000 7f7a16000640 1 -- 172.21.15.43:0/2620992739 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a0c010910 con 0x7f7a18076720 2024-09-19T05:08:45.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.730+0000 7f7a16000640 1 -- 172.21.15.43:0/2620992739 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c018960 con 0x7f7a18076720 2024-09-19T05:08:45.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.730+0000 7f7a20000640 1 -- 172.21.15.43:0/2620992739 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a1819f2e0 con 0x7f7a18076720 2024-09-19T05:08:45.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.730+0000 7f7a20000640 1 -- 172.21.15.43:0/2620992739 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a181ab5d0 con 0x7f7a18076720 2024-09-19T05:08:45.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.731+0000 7f7a20000640 1 -- 172.21.15.43:0/2620992739 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a18076b00 con 0x7f7a18076720 2024-09-19T05:08:45.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.733+0000 7f7a16000640 1 -- 172.21.15.43:0/2620992739 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f7a0c010ac0 con 0x7f7a18076720 2024-09-19T05:08:45.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.734+0000 7f7a16000640 1 --2- 172.21.15.43:0/2620992739 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f7a000a33b0 0x7f7a000a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:45.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.735+0000 7f7a16000640 1 -- 172.21.15.43:0/2620992739 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f7a0c0d0190 con 0x7f7a18076720 2024-09-19T05:08:45.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.735+0000 7f7a1cc00640 1 --2- 172.21.15.43:0/2620992739 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f7a000a33b0 0x7f7a000a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:45.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.735+0000 7f7a1cc00640 1 --2- 172.21.15.43:0/2620992739 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f7a000a33b0 0x7f7a000a5870 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7a180792b0 tx=0x7f7a04005a20 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:45.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.736+0000 7f7a16000640 1 -- 172.21.15.43:0/2620992739 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7a0c09c530 con 0x7f7a18076720 2024-09-19T05:08:45.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:08:45 smithi151 ceph-mon[35425]: pgmap v126: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-19T05:08:45.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.961+0000 7f7a20000640 1 -- 172.21.15.43:0/2620992739 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7a1810dbf0 con 0x7f7a18076720 2024-09-19T05:08:45.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.962+0000 7f7a16000640 1 -- 172.21.15.43:0/2620992739 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11965 (secure 0 0 0) 0x7f7a0c0a0830 con 0x7f7a18076720 2024-09-19T05:08:45.965 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:08:45.965 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":29,"btime":"2024-09-19T05:04:38:785407+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/773801169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":773801169},{"type":"v1","addr":"172.21.15.43:6835","nonce":773801169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1799137902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1799137902},{"type":"v1","addr":"172.21.15.43:6837","nonce":1799137902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:02:25.463041+0000","modified":"2024-09-19T05:04:38.039809+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24518,"mds_2":24511,"mds_3":24506,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14541":{"gid":14541,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/794881135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":794881135},{"type":"v1","addr":"172.21.15.43:6841","nonce":794881135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/2836453158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2836453158},{"type":"v1","addr":"172.21.15.151:6833","nonce":2836453158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/178785812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":178785812},{"type":"v1","addr":"172.21.15.96:6833","nonce":178785812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1671043820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1671043820},{"type":"v1","addr":"172.21.15.96:6835","nonce":1671043820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2972514328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2972514328},{"type":"v1","addr":"172.21.15.151:6835","nonce":2972514328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.151:6837/3960378010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3960378010},{"type":"v1","addr":"172.21.15.151:6837","nonce":3960378010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6837/3978388286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3978388286},{"type":"v1","addr":"172.21.15.96:6837","nonce":3978388286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.151:6839/3794104313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3794104313},{"type":"v1","addr":"172.21.15.151:6839","nonce":3794104313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.43:6839/3404555030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3404555030},{"type":"v1","addr":"172.21.15.43:6839","nonce":3404555030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/605527621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":605527621},{"type":"v1","addr":"172.21.15.96:6839","nonce":605527621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24518,24511,24506,24493]},"id":1}]} 2024-09-19T05:08:45.965 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 29 2024-09-19T05:08:45.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.964+0000 7f7a20000640 1 -- 172.21.15.43:0/2620992739 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f7a000a33b0 msgr2=0x7f7a000a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:45.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.964+0000 7f7a20000640 1 --2- 172.21.15.43:0/2620992739 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f7a000a33b0 0x7f7a000a5870 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7a180792b0 tx=0x7f7a04005a20 comp rx=0 tx=0).stop 2024-09-19T05:08:45.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.964+0000 7f7a20000640 1 -- 172.21.15.43:0/2620992739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a18076720 msgr2=0x7f7a181a4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:45.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.964+0000 7f7a20000640 1 --2- 172.21.15.43:0/2620992739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a18076720 0x7f7a181a4700 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c00e9f0 tx=0x7f7a0c00eeb0 comp rx=0 tx=0).stop 2024-09-19T05:08:45.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.965+0000 7f7a20000640 1 -- 172.21.15.43:0/2620992739 shutdown_connections 2024-09-19T05:08:45.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.965+0000 7f7a20000640 1 --2- 172.21.15.43:0/2620992739 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f7a000a33b0 0x7f7a000a5870 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:45.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.965+0000 7f7a20000640 1 --2- 172.21.15.43:0/2620992739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a18112a30 0x7f7a1819e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:45.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.965+0000 7f7a20000640 1 --2- 172.21.15.43:0/2620992739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a18077040 0x7f7a181a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:45.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.965+0000 7f7a20000640 1 --2- 172.21.15.43:0/2620992739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a18076720 0x7f7a181a4700 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:45.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.965+0000 7f7a20000640 1 -- 172.21.15.43:0/2620992739 >> 172.21.15.43:0/2620992739 conn(0x7f7a180ffa90 msgr2=0x7f7a18101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:45.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.965+0000 7f7a20000640 1 -- 172.21.15.43:0/2620992739 shutdown_connections 2024-09-19T05:08:45.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:45.965+0000 7f7a20000640 1 -- 172.21.15.43:0/2620992739 wait complete. 2024-09-19T05:08:45.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:08:45 smithi096 ceph-mon[35085]: pgmap v126: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-19T05:08:46.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:08:46 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/2620992739' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:08:46.874 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T05:08:46.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:08:46 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/2620992739' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:08:46.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:08:46 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/2620992739' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:08:47.258 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:08:47.650 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:08:47 smithi043 ceph-mon[30800]: pgmap v127: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:08:47.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.856+0000 7fd6be400640 1 -- 172.21.15.43:0/2423524402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6b810e450 msgr2=0x7fd6b8110840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:47.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.856+0000 7fd6be400640 1 --2- 172.21.15.43:0/2423524402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6b810e450 0x7fd6b8110840 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0009ca0 tx=0x7fd6b002a950 comp rx=0 tx=0).stop 2024-09-19T05:08:47.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.856+0000 7fd6be400640 1 -- 172.21.15.43:0/2423524402 shutdown_connections 2024-09-19T05:08:47.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.856+0000 7fd6be400640 1 --2- 172.21.15.43:0/2423524402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6b810e450 0x7fd6b8110840 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:47.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.856+0000 7fd6be400640 1 --2- 172.21.15.43:0/2423524402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6b8074e30 0x7fd6b810df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:47.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.856+0000 7fd6be400640 1 --2- 172.21.15.43:0/2423524402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6b8074510 0x7fd6b80748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:47.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.856+0000 7fd6be400640 1 -- 172.21.15.43:0/2423524402 >> 172.21.15.43:0/2423524402 conn(0x7fd6b80fd8b0 msgr2=0x7fd6b80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:47.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.856+0000 7fd6be400640 1 -- 172.21.15.43:0/2423524402 shutdown_connections 2024-09-19T05:08:47.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.856+0000 7fd6be400640 1 -- 172.21.15.43:0/2423524402 wait complete. 2024-09-19T05:08:47.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.857+0000 7fd6be400640 1 Processor -- start 2024-09-19T05:08:47.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.857+0000 7fd6be400640 1 -- start start 2024-09-19T05:08:47.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.858+0000 7fd6be400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6b8074510 0x7fd6b819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:47.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.858+0000 7fd6be400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6b8074e30 0x7fd6b819ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:47.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.858+0000 7fd6be400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6b810e450 0x7fd6b819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:47.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.858+0000 7fd6be400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b8114170 con 0x7fd6b8074e30 2024-09-19T05:08:47.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.858+0000 7fd6be400640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b8113ff0 con 0x7fd6b810e450 2024-09-19T05:08:47.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.858+0000 7fd6be400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b81142f0 con 0x7fd6b8074510 2024-09-19T05:08:47.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.858+0000 7fd6b7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6b8074510 0x7fd6b819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:47.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.858+0000 7fd6b7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6b8074510 0x7fd6b819a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:39544/0 (socket says 172.21.15.43:39544) 2024-09-19T05:08:47.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.858+0000 7fd6b7400640 1 -- 172.21.15.43:0/138903559 learned_addr learned my addr 172.21.15.43:0/138903559 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:47.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.858+0000 7fd6afe00640 1 --2- 172.21.15.43:0/138903559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6b8074e30 0x7fd6b819ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:47.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.858+0000 7fd6b7e00640 1 --2- 172.21.15.43:0/138903559 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6b810e450 0x7fd6b819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:47.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.858+0000 7fd6b7400640 1 -- 172.21.15.43:0/138903559 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6b810e450 msgr2=0x7fd6b819efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:47.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.858+0000 7fd6b7400640 1 --2- 172.21.15.43:0/138903559 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6b810e450 0x7fd6b819efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:47.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.858+0000 7fd6b7400640 1 -- 172.21.15.43:0/138903559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6b8074e30 msgr2=0x7fd6b819ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:47.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.858+0000 7fd6b7400640 1 --2- 172.21.15.43:0/138903559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6b8074e30 0x7fd6b819ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:47.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.858+0000 7fd6b7400640 1 -- 172.21.15.43:0/138903559 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b819f680 con 0x7fd6b8074510 2024-09-19T05:08:47.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.859+0000 7fd6b7e00640 1 --2- 172.21.15.43:0/138903559 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6b810e450 0x7fd6b819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:08:47.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.859+0000 7fd6b7400640 1 --2- 172.21.15.43:0/138903559 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6b8074510 0x7fd6b819a6d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd6a800e9f0 tx=0x7fd6a800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:47.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.859+0000 7fd6b4c00640 1 -- 172.21.15.43:0/138903559 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a800cd80 con 0x7fd6b8074510 2024-09-19T05:08:47.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.859+0000 7fd6b4c00640 1 -- 172.21.15.43:0/138903559 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6a8005500 con 0x7fd6b8074510 2024-09-19T05:08:47.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.859+0000 7fd6b4c00640 1 -- 172.21.15.43:0/138903559 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8010690 con 0x7fd6b8074510 2024-09-19T05:08:47.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.859+0000 7fd6be400640 1 -- 172.21.15.43:0/138903559 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6b819f970 con 0x7fd6b8074510 2024-09-19T05:08:47.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.859+0000 7fd6be400640 1 -- 172.21.15.43:0/138903559 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6b81a7250 con 0x7fd6b8074510 2024-09-19T05:08:47.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.862+0000 7fd6c0dfa740 1 Processor -- start 2024-09-19T05:08:47.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.862+0000 7fd6b4c00640 1 -- 172.21.15.43:0/138903559 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fd6a80046c0 con 0x7fd6b8074510 2024-09-19T05:08:47.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.862+0000 7fd6c0dfa740 1 -- start start 2024-09-19T05:08:47.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.863+0000 7fd6c0dfa740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ad42f0c70 0x561ad42f1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:47.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.863+0000 7fd6c0dfa740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ad42f1590 0x561ad42fb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:47.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.863+0000 7fd6c0dfa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ad42fbe50 0x561ad42fe240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:47.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.863+0000 7fd6c0dfa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561ad42573e0 con 0x561ad42fbe50 2024-09-19T05:08:47.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.863+0000 7fd6c0dfa740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561ad41fed80 con 0x561ad42f1590 2024-09-19T05:08:47.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.863+0000 7fd6c0dfa740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561ad42533a0 con 0x561ad42f0c70 2024-09-19T05:08:47.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.863+0000 7fd6b7400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ad42f1590 0x561ad42fb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:47.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.863+0000 7fd6b7400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ad42f1590 0x561ad42fb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:41850/0 (socket says 172.21.15.43:41850) 2024-09-19T05:08:47.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.863+0000 7fd6b7400640 1 -- 172.21.15.43:0/1474583641 learned_addr learned my addr 172.21.15.43:0/1474583641 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:47.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.863+0000 7fd6b7e00640 1 --2- 172.21.15.43:0/1474583641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ad42f0c70 0x561ad42f1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:47.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.863+0000 7fd6afe00640 1 --2- 172.21.15.43:0/1474583641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ad42fbe50 0x561ad42fe240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:47.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.864+0000 7fd6b7400640 1 -- 172.21.15.43:0/1474583641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ad42f0c70 msgr2=0x561ad42f1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:47.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.864+0000 7fd6b7400640 1 --2- 172.21.15.43:0/1474583641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ad42f0c70 0x561ad42f1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:47.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.864+0000 7fd6b7400640 1 -- 172.21.15.43:0/1474583641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ad42fbe50 msgr2=0x561ad42fe240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:47.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.864+0000 7fd6b7400640 1 --2- 172.21.15.43:0/1474583641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ad42fbe50 0x561ad42fe240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:47.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.864+0000 7fd6b7400640 1 -- 172.21.15.43:0/1474583641 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ad421dae0 con 0x561ad42f1590 2024-09-19T05:08:47.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.864+0000 7fd6b7e00640 1 --2- 172.21.15.43:0/1474583641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ad42f0c70 0x561ad42f1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:08:47.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.864+0000 7fd6afe00640 1 --2- 172.21.15.43:0/1474583641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ad42fbe50 0x561ad42fe240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:08:47.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.864+0000 7fd6b4c00640 1 --2- 172.21.15.43:0/138903559 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fd69c0a3430 0x7fd69c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:47.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.864+0000 7fd6b7400640 1 --2- 172.21.15.43:0/1474583641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ad42f1590 0x561ad42fb910 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd6a80cd7e0 tx=0x7fd6a80d8320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d307278ba956192 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:47.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.864+0000 7fd6b4c00640 1 -- 172.21.15.43:0/138903559 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fd6a80d01a0 con 0x7fd6b8074510 2024-09-19T05:08:47.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.864+0000 7fd6b7e00640 1 --2- 172.21.15.43:0/138903559 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fd69c0a3430 0x7fd69c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:47.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.864+0000 7fd6acc00640 1 -- 172.21.15.43:0/1474583641 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a80986d0 con 0x561ad42f1590 2024-09-19T05:08:47.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.865+0000 7fd6acc00640 1 -- 172.21.15.43:0/1474583641 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6a809cb30 con 0x561ad42f1590 2024-09-19T05:08:47.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.865+0000 7fd6acc00640 1 -- 172.21.15.43:0/1474583641 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a809bd90 con 0x561ad42f1590 2024-09-19T05:08:47.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.865+0000 7fd6c0dfa740 1 -- 172.21.15.43:0/1474583641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ad42f1590 msgr2=0x561ad42fb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:47.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.865+0000 7fd6b7e00640 1 --2- 172.21.15.43:0/138903559 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fd69c0a3430 0x7fd69c0a58f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0005cc0 tx=0x7fd6b0005c50 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:47.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.865+0000 7fd6c0dfa740 1 --2- 172.21.15.43:0/1474583641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ad42f1590 0x561ad42fb910 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd6a80cd7e0 tx=0x7fd6a80d8320 comp rx=0 tx=0).stop 2024-09-19T05:08:47.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.866+0000 7fd6c0dfa740 1 -- 172.21.15.43:0/1474583641 shutdown_connections 2024-09-19T05:08:47.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.866+0000 7fd6c0dfa740 1 --2- 172.21.15.43:0/1474583641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ad42fbe50 0x561ad42fe240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:47.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.866+0000 7fd6c0dfa740 1 --2- 172.21.15.43:0/1474583641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ad42f1590 0x561ad42fb910 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:47.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.866+0000 7fd6c0dfa740 1 --2- 172.21.15.43:0/1474583641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ad42f0c70 0x561ad42f1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:47.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.866+0000 7fd6c0dfa740 1 -- 172.21.15.43:0/1474583641 >> 172.21.15.43:0/1474583641 conn(0x561ad42ed200 msgr2=0x561ad42ed5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:47.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.866+0000 7fd6c0dfa740 1 -- 172.21.15.43:0/1474583641 shutdown_connections 2024-09-19T05:08:47.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.866+0000 7fd6c0dfa740 1 -- 172.21.15.43:0/1474583641 wait complete. 2024-09-19T05:08:47.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.866+0000 7fd6c0dfa740 1 -- start start 2024-09-19T05:08:47.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.866+0000 7fd6c0dfa740 1 Processor -- start 2024-09-19T05:08:47.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.867+0000 7fd6c0dfa740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ad42f0c70 0x561ad42f8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:47.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.867+0000 7fd6c0dfa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ad42f1590 0x561ad42f9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:47.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.867+0000 7fd6c0dfa740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ad42fbe50 0x561ad42f9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:47.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.867+0000 7fd6c0dfa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561ad42fea20 con 0x561ad42f1590 2024-09-19T05:08:47.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.867+0000 7fd6c0dfa740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561ad42120f0 con 0x561ad42fbe50 2024-09-19T05:08:47.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.867+0000 7fd6c0dfa740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561ad42feba0 con 0x561ad42f0c70 2024-09-19T05:08:47.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.867+0000 7fd6c0dfa740 10 client.0 _ll_register_callbacks cb 0x561ad41f9bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:47.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.867+0000 7fd6b7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ad42f0c70 0x561ad42f8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:47.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.867+0000 7fd6afe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ad42f1590 0x561ad42f9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:47.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.867+0000 7fd6b7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ad42f0c70 0x561ad42f8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:39564/0 (socket says 172.21.15.43:39564) 2024-09-19T05:08:47.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.867+0000 7fd6b7400640 1 -- 172.21.15.43:0/2970621856 learned_addr learned my addr 172.21.15.43:0/2970621856 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:47.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.867+0000 7fd6b7e00640 1 --2- 172.21.15.43:0/2970621856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ad42fbe50 0x561ad42f9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:47.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.868+0000 7fd6b7400640 1 -- 172.21.15.43:0/2970621856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ad42fbe50 msgr2=0x561ad42f9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:47.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.868+0000 7fd6b7400640 1 --2- 172.21.15.43:0/2970621856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ad42fbe50 0x561ad42f9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:47.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.868+0000 7fd6b7400640 1 -- 172.21.15.43:0/2970621856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ad42f1590 msgr2=0x561ad42f9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:47.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.868+0000 7fd6b7400640 1 --2- 172.21.15.43:0/2970621856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ad42f1590 0x561ad42f9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:47.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.868+0000 7fd6b7400640 1 -- 172.21.15.43:0/2970621856 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ad3fc83c0 con 0x561ad42f0c70 2024-09-19T05:08:47.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.868+0000 7fd6afe00640 1 --2- 172.21.15.43:0/2970621856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ad42f1590 0x561ad42f9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:47.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.868+0000 7fd6b7e00640 1 --2- 172.21.15.43:0/2970621856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ad42fbe50 0x561ad42f9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:08:47.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.868+0000 7fd6b7400640 1 --2- 172.21.15.43:0/2970621856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ad42f0c70 0x561ad42f8e00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd6a80a4b70 tx=0x7fd6a80d0420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=759821e757550449 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:47.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.868+0000 7fd6a5600640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:08:47.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.868+0000 7fd6a5600640 1 -- 172.21.15.43:0/2970621856 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a809bf70 con 0x561ad42f0c70 2024-09-19T05:08:47.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.868+0000 7fd6a5600640 1 -- 172.21.15.43:0/2970621856 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fd6a809c890 con 0x561ad42f0c70 2024-09-19T05:08:47.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.868+0000 7fd6c0dfa740 1 -- 172.21.15.43:0/2970621856 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6a809cb30 con 0x561ad42f0c70 2024-09-19T05:08:47.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.868+0000 7fd6a5600640 1 -- 172.21.15.43:0/2970621856 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd6a809c200 con 0x561ad42f0c70 2024-09-19T05:08:47.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.868+0000 7fd6c0dfa740 10 client.25975 fetch_fsmap learned FSMap version 29 2024-09-19T05:08:47.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.868+0000 7fd6c0dfa740 1 -- 172.21.15.43:0/2970621856 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561ad42fea20 con 0x561ad42f0c70 2024-09-19T05:08:47.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.869+0000 7fd6a5600640 1 -- 172.21.15.43:0/2970621856 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6a809cb30 con 0x561ad42f0c70 2024-09-19T05:08:47.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.869+0000 7fd6a5600640 1 -- 172.21.15.43:0/2970621856 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd6a80a4c10 con 0x561ad42f0c70 2024-09-19T05:08:47.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.869+0000 7fd6c0dfa740 10 client.25975 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:08:47.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.869+0000 7fd6c0dfa740 10 client.25975 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-19T05:08:47.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.870+0000 7fd6c0dfa740 20 client.25975 populate_metadata read hostname 'smithi043' 2024-09-19T05:08:47.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.870+0000 7fd6c0dfa740 1 --2- 172.21.15.43:0/2970621856 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x561ad430bcf0 0x561ad430e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:47.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.870+0000 7fd6c0dfa740 4 client.25975 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:08:47.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.870+0000 7fd6c0dfa740 1 -- 172.21.15.43:0/2970621856 --> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ad4264950 con 0x561ad430bcf0 2024-09-19T05:08:47.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.870+0000 7fd6afe00640 1 --2- 172.21.15.43:0/2970621856 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x561ad430bcf0 0x561ad430e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:47.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.871+0000 7fd6afe00640 1 --2- 172.21.15.43:0/2970621856 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x561ad430bcf0 0x561ad430e0e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=71f1347f5d9b3c5 server_cookie=fa30dfca54a922ce in_seq=0 out_seq=0 2024-09-19T05:08:47.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.871+0000 7fd6a5600640 10 client.25975 ms_handle_connect on v2:172.21.15.151:6836/3960378010 2024-09-19T05:08:47.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.872+0000 7fd6a5600640 1 -- 172.21.15.43:0/2970621856 <== mds.4 v2:172.21.15.151:6836/3960378010 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561ad4264950 con 0x561ad430bcf0 2024-09-19T05:08:47.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.872+0000 7fd6a5600640 10 client.25975 handle_command_reply: tid=0 2024-09-19T05:08:47.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.872+0000 7fd6c0dfa740 1 client.25975 shutdown 2024-09-19T05:08:47.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.872+0000 7fd6c0dfa740 1 -- 172.21.15.43:0/2970621856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ad42f0c70 msgr2=0x561ad42f8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:47.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.872+0000 7fd6c0dfa740 1 --2- 172.21.15.43:0/2970621856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ad42f0c70 0x561ad42f8e00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd6a80a4b70 tx=0x7fd6a80d0420 comp rx=0 tx=0).stop 2024-09-19T05:08:47.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.872+0000 7fd6c0dfa740 1 -- 172.21.15.43:0/2970621856 shutdown_connections 2024-09-19T05:08:47.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.872+0000 7fd6c0dfa740 1 --2- 172.21.15.43:0/2970621856 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x561ad430bcf0 0x561ad430e0e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:47.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.872+0000 7fd6c0dfa740 1 --2- 172.21.15.43:0/2970621856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ad42fbe50 0x561ad42f9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:47.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.872+0000 7fd6c0dfa740 1 --2- 172.21.15.43:0/2970621856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ad42f1590 0x561ad42f9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:47.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.872+0000 7fd6c0dfa740 1 --2- 172.21.15.43:0/2970621856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ad42f0c70 0x561ad42f8e00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:47.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.872+0000 7fd6c0dfa740 1 -- 172.21.15.43:0/2970621856 >> 172.21.15.43:0/2970621856 conn(0x561ad42ed200 msgr2=0x561ad42f2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:47.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.872+0000 7fd6a5600640 0 client.25975 ms_handle_reset on v2:172.21.15.151:6836/3960378010 2024-09-19T05:08:47.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.872+0000 7fd6c0dfa740 1 -- 172.21.15.43:0/2970621856 shutdown_connections 2024-09-19T05:08:47.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.872+0000 7fd6c0dfa740 1 -- 172.21.15.43:0/2970621856 wait complete. 2024-09-19T05:08:47.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.872+0000 7fd6c0dfa740 20 client.25975 trim_cache size 0 max 16384 2024-09-19T05:08:47.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.886+0000 7fd6c0dfa740 1 Processor -- start 2024-09-19T05:08:47.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.886+0000 7fd6c0dfa740 1 -- start start 2024-09-19T05:08:47.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.887+0000 7fd6c0dfa740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ad42f0c70 0x561ad4313710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:47.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.887+0000 7fd6c0dfa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ad42f1590 0x561ad4313c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:47.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.887+0000 7fd6c0dfa740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ad42fbe50 0x561ad42f6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:47.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.887+0000 7fd6c0dfa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b81afac0 con 0x561ad42f1590 2024-09-19T05:08:47.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.887+0000 7fd6c0dfa740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b81b1600 con 0x561ad42fbe50 2024-09-19T05:08:47.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.887+0000 7fd6c0dfa740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b81b0190 con 0x561ad42f0c70 2024-09-19T05:08:47.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.887+0000 7fd6afe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ad42f1590 0x561ad4313c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:47.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.887+0000 7fd6afe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ad42f1590 0x561ad4313c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49358/0 (socket says 172.21.15.43:49358) 2024-09-19T05:08:47.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.887+0000 7fd6afe00640 1 -- 172.21.15.43:0/238965454 learned_addr learned my addr 172.21.15.43:0/238965454 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:47.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.887+0000 7fd6b7e00640 1 --2- 172.21.15.43:0/238965454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ad42fbe50 0x561ad42f6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:47.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.887+0000 7fd6b7400640 1 --2- 172.21.15.43:0/238965454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ad42f0c70 0x561ad4313710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:47.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.887+0000 7fd6afe00640 1 -- 172.21.15.43:0/238965454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ad42f0c70 msgr2=0x561ad4313710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:47.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.887+0000 7fd6afe00640 1 --2- 172.21.15.43:0/238965454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ad42f0c70 0x561ad4313710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:47.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.887+0000 7fd6afe00640 1 -- 172.21.15.43:0/238965454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ad42fbe50 msgr2=0x561ad42f6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:47.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.887+0000 7fd6afe00640 1 --2- 172.21.15.43:0/238965454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ad42fbe50 0x561ad42f6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:47.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.887+0000 7fd6afe00640 1 -- 172.21.15.43:0/238965454 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ad42f13c0 con 0x561ad42f1590 2024-09-19T05:08:47.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.888+0000 7fd6afe00640 1 --2- 172.21.15.43:0/238965454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ad42f1590 0x561ad4313c50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x561ad42ee580 tx=0x7fd6a0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f4fb7b7d819289 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:47.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.888+0000 7fd6a7e00640 1 -- 172.21.15.43:0/238965454 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a00122d0 con 0x561ad42f1590 2024-09-19T05:08:47.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.888+0000 7fd6a7e00640 1 -- 172.21.15.43:0/238965454 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6a0005500 con 0x561ad42f1590 2024-09-19T05:08:47.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.888+0000 7fd6a7e00640 1 -- 172.21.15.43:0/238965454 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0004a00 con 0x561ad42f1590 2024-09-19T05:08:47.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.889+0000 7fd6c0dfa740 1 -- 172.21.15.43:0/238965454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ad42f1590 msgr2=0x561ad4313c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:47.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.889+0000 7fd6c0dfa740 1 --2- 172.21.15.43:0/238965454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ad42f1590 0x561ad4313c50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x561ad42ee580 tx=0x7fd6a0005f90 comp rx=0 tx=0).stop 2024-09-19T05:08:47.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.889+0000 7fd6c0dfa740 1 -- 172.21.15.43:0/238965454 shutdown_connections 2024-09-19T05:08:47.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.889+0000 7fd6c0dfa740 1 --2- 172.21.15.43:0/238965454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ad42fbe50 0x561ad42f6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:47.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.889+0000 7fd6c0dfa740 1 --2- 172.21.15.43:0/238965454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ad42f1590 0x561ad4313c50 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:47.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.889+0000 7fd6c0dfa740 1 --2- 172.21.15.43:0/238965454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ad42f0c70 0x561ad4313710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:47.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.889+0000 7fd6c0dfa740 1 -- 172.21.15.43:0/238965454 >> 172.21.15.43:0/238965454 conn(0x561ad42ed200 msgr2=0x561ad42f40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:47.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.889+0000 7fd6c0dfa740 1 -- 172.21.15.43:0/238965454 shutdown_connections 2024-09-19T05:08:47.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.889+0000 7fd6c0dfa740 1 -- 172.21.15.43:0/238965454 wait complete. 2024-09-19T05:08:47.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.889+0000 7fd6c0dfa740 1 -- start start 2024-09-19T05:08:47.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.889+0000 7fd6c0dfa740 1 Processor -- start 2024-09-19T05:08:47.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.890+0000 7fd6c0dfa740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ad42f0c70 0x561ad42ff580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:47.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.890+0000 7fd6c0dfa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ad42f1590 0x561ad42ffac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:47.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.890+0000 7fd6c0dfa740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ad42fbe50 0x561ad436e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:47.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.890+0000 7fd6c0dfa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b81b0f30 con 0x561ad42f1590 2024-09-19T05:08:47.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.890+0000 7fd6c0dfa740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b81b0860 con 0x561ad42f0c70 2024-09-19T05:08:47.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.890+0000 7fd6c0dfa740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b81adaa0 con 0x561ad42fbe50 2024-09-19T05:08:47.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.890+0000 7fd6c0dfa740 10 client.0 _ll_register_callbacks cb 0x7fd6a80bf070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:47.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.890+0000 7fd6b7400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ad42f0c70 0x561ad42ff580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:47.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.890+0000 7fd6afe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ad42f1590 0x561ad42ffac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:47.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.890+0000 7fd6b7400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ad42f0c70 0x561ad42ff580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:41876/0 (socket says 172.21.15.43:41876) 2024-09-19T05:08:47.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.890+0000 7fd6b7400640 1 -- 172.21.15.43:0/1800773965 learned_addr learned my addr 172.21.15.43:0/1800773965 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:47.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.890+0000 7fd6b7e00640 1 --2- 172.21.15.43:0/1800773965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ad42fbe50 0x561ad436e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:47.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.891+0000 7fd6b7400640 1 -- 172.21.15.43:0/1800773965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ad42fbe50 msgr2=0x561ad436e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:47.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.891+0000 7fd6b7400640 1 --2- 172.21.15.43:0/1800773965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ad42fbe50 0x561ad436e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:47.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.891+0000 7fd6b7400640 1 -- 172.21.15.43:0/1800773965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ad42f1590 msgr2=0x561ad42ffac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:47.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.891+0000 7fd6b7400640 1 --2- 172.21.15.43:0/1800773965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ad42f1590 0x561ad42ffac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:47.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.891+0000 7fd6b7400640 1 -- 172.21.15.43:0/1800773965 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ad436ed50 con 0x561ad42f0c70 2024-09-19T05:08:47.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.891+0000 7fd6b7400640 1 --2- 172.21.15.43:0/1800773965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ad42f0c70 0x561ad42ff580 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8098890 tx=0x7fd6a8004b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7ac123bc5151500 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:47.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.891+0000 7fd6a5600640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:08:47.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.891+0000 7fd6a5600640 1 -- 172.21.15.43:0/1800773965 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a809bf70 con 0x561ad42f0c70 2024-09-19T05:08:47.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.891+0000 7fd6a5600640 1 -- 172.21.15.43:0/1800773965 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fd6a809c890 con 0x561ad42f0c70 2024-09-19T05:08:47.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.891+0000 7fd6c0dfa740 1 -- 172.21.15.43:0/1800773965 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6a0005500 con 0x561ad42f0c70 2024-09-19T05:08:47.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.892+0000 7fd6a5600640 1 -- 172.21.15.43:0/1800773965 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd6a809bd90 con 0x561ad42f0c70 2024-09-19T05:08:47.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.892+0000 7fd6c0dfa740 10 client.26039 fetch_fsmap learned FSMap version 29 2024-09-19T05:08:47.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.892+0000 7fd6c0dfa740 1 -- 172.21.15.43:0/1800773965 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561ad43760e0 con 0x561ad42f0c70 2024-09-19T05:08:47.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.892+0000 7fd6a5600640 1 -- 172.21.15.43:0/1800773965 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6a0005500 con 0x561ad42f0c70 2024-09-19T05:08:47.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.892+0000 7fd6a5600640 1 -- 172.21.15.43:0/1800773965 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd6a809b880 con 0x561ad42f0c70 2024-09-19T05:08:47.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.892+0000 7fd6c0dfa740 10 client.26039 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:08:47.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.892+0000 7fd6c0dfa740 10 client.26039 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-19T05:08:47.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.892+0000 7fd6c0dfa740 20 client.26039 populate_metadata read hostname 'smithi043' 2024-09-19T05:08:47.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.892+0000 7fd6c0dfa740 1 --2- 172.21.15.43:0/1800773965 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x561ad43765b0 0x561ad43789a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:47.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.892+0000 7fd6c0dfa740 4 client.26039 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:08:47.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.892+0000 7fd6c0dfa740 1 -- 172.21.15.43:0/1800773965 --> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561ad425ee60 con 0x561ad43765b0 2024-09-19T05:08:47.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.892+0000 7fd6afe00640 1 --2- 172.21.15.43:0/1800773965 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x561ad43765b0 0x561ad43789a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:47.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.893+0000 7fd6afe00640 1 --2- 172.21.15.43:0/1800773965 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x561ad43765b0 0x561ad43789a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a3c3128a28e2af0f server_cookie=648922e43c807146 in_seq=0 out_seq=0 2024-09-19T05:08:47.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.893+0000 7fd6a5600640 10 client.26039 ms_handle_connect on v2:172.21.15.151:6836/3960378010 2024-09-19T05:08:47.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.893+0000 7fd6a5600640 1 -- 172.21.15.43:0/1800773965 <== mds.4 v2:172.21.15.151:6836/3960378010 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561ad425ee60 con 0x561ad43765b0 2024-09-19T05:08:47.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.893+0000 7fd6a5600640 10 client.26039 handle_command_reply: tid=0 2024-09-19T05:08:47.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.893+0000 7fd6c0dfa740 1 client.26039 shutdown 2024-09-19T05:08:47.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.893+0000 7fd6c0dfa740 1 -- 172.21.15.43:0/1800773965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ad42f0c70 msgr2=0x561ad42ff580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:47.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.893+0000 7fd6c0dfa740 1 --2- 172.21.15.43:0/1800773965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ad42f0c70 0x561ad42ff580 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8098890 tx=0x7fd6a8004b30 comp rx=0 tx=0).stop 2024-09-19T05:08:47.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.893+0000 7fd6c0dfa740 1 -- 172.21.15.43:0/1800773965 shutdown_connections 2024-09-19T05:08:47.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.893+0000 7fd6c0dfa740 1 --2- 172.21.15.43:0/1800773965 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x561ad43765b0 0x561ad43789a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:47.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.893+0000 7fd6c0dfa740 1 --2- 172.21.15.43:0/1800773965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ad42fbe50 0x561ad436e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:47.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.893+0000 7fd6c0dfa740 1 --2- 172.21.15.43:0/1800773965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ad42f1590 0x561ad42ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:47.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.893+0000 7fd6a5600640 0 client.26039 ms_handle_reset on v2:172.21.15.151:6836/3960378010 2024-09-19T05:08:47.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.893+0000 7fd6c0dfa740 1 --2- 172.21.15.43:0/1800773965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ad42f0c70 0x561ad42ff580 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:47.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.894+0000 7fd6c0dfa740 1 -- 172.21.15.43:0/1800773965 >> 172.21.15.43:0/1800773965 conn(0x561ad42ed200 msgr2=0x561ad42f8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:47.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.894+0000 7fd6c0dfa740 1 -- 172.21.15.43:0/1800773965 shutdown_connections 2024-09-19T05:08:47.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.894+0000 7fd6c0dfa740 1 -- 172.21.15.43:0/1800773965 wait complete. 2024-09-19T05:08:47.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.894+0000 7fd6c0dfa740 20 client.26039 trim_cache size 0 max 16384 2024-09-19T05:08:47.899 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:08:47.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.894+0000 7fd6be400640 1 -- 172.21.15.43:0/138903559 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fd69c0a3430 msgr2=0x7fd69c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:47.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.894+0000 7fd6be400640 1 --2- 172.21.15.43:0/138903559 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fd69c0a3430 0x7fd69c0a58f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0005cc0 tx=0x7fd6b0005c50 comp rx=0 tx=0).stop 2024-09-19T05:08:47.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.894+0000 7fd6be400640 1 -- 172.21.15.43:0/138903559 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6b8074510 msgr2=0x7fd6b819a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:47.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.894+0000 7fd6be400640 1 --2- 172.21.15.43:0/138903559 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6b8074510 0x7fd6b819a6d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd6a800e9f0 tx=0x7fd6a800edf0 comp rx=0 tx=0).stop 2024-09-19T05:08:47.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.895+0000 7fd6be400640 1 -- 172.21.15.43:0/138903559 shutdown_connections 2024-09-19T05:08:47.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.895+0000 7fd6be400640 1 --2- 172.21.15.43:0/138903559 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fd69c0a3430 0x7fd69c0a58f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:47.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.895+0000 7fd6be400640 1 --2- 172.21.15.43:0/138903559 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6b810e450 0x7fd6b819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:47.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.895+0000 7fd6be400640 1 --2- 172.21.15.43:0/138903559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6b8074e30 0x7fd6b819ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:47.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.895+0000 7fd6be400640 1 --2- 172.21.15.43:0/138903559 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6b8074510 0x7fd6b819a6d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:47.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.895+0000 7fd6be400640 1 -- 172.21.15.43:0/138903559 >> 172.21.15.43:0/138903559 conn(0x7fd6b80fd8b0 msgr2=0x7fd6b80fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:47.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.895+0000 7fd6be400640 1 -- 172.21.15.43:0/138903559 shutdown_connections 2024-09-19T05:08:47.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:47.895+0000 7fd6be400640 1 -- 172.21.15.43:0/138903559 wait complete. 2024-09-19T05:08:47.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:08:47 smithi151 ceph-mon[35425]: pgmap v127: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:08:47.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:08:47 smithi096 ceph-mon[35085]: pgmap v127: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:08:48.734 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T05:08:49.113 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:08:49.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.698+0000 7fa422c00640 1 -- 172.21.15.43:0/996826127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa41c103ff0 msgr2=0x7fa41c1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:49.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.698+0000 7fa422c00640 1 --2- 172.21.15.43:0/996826127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa41c103ff0 0x7fa41c1043d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa40c009900 tx=0x7fa40c02a410 comp rx=0 tx=0).stop 2024-09-19T05:08:49.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.698+0000 7fa422c00640 1 -- 172.21.15.43:0/996826127 shutdown_connections 2024-09-19T05:08:49.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.698+0000 7fa422c00640 1 --2- 172.21.15.43:0/996826127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa41c109120 0x7fa41c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:49.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.698+0000 7fa422c00640 1 --2- 172.21.15.43:0/996826127 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa41c1049a0 0x7fa41c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:49.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.698+0000 7fa422c00640 1 --2- 172.21.15.43:0/996826127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa41c103ff0 0x7fa41c1043d0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:49.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.698+0000 7fa422c00640 1 -- 172.21.15.43:0/996826127 >> 172.21.15.43:0/996826127 conn(0x7fa41c0ffa90 msgr2=0x7fa41c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:49.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.698+0000 7fa422c00640 1 -- 172.21.15.43:0/996826127 shutdown_connections 2024-09-19T05:08:49.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.698+0000 7fa422c00640 1 -- 172.21.15.43:0/996826127 wait complete. 2024-09-19T05:08:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.699+0000 7fa422c00640 1 Processor -- start 2024-09-19T05:08:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.699+0000 7fa422c00640 1 -- start start 2024-09-19T05:08:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.700+0000 7fa422c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa41c103ff0 0x7fa41c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.700+0000 7fa422c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa41c1049a0 0x7fa41c19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.700+0000 7fa422c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa41c109120 0x7fa41c19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.700+0000 7fa422c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa41c114170 con 0x7fa41c103ff0 2024-09-19T05:08:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.700+0000 7fa422c00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa41c113ff0 con 0x7fa41c109120 2024-09-19T05:08:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.700+0000 7fa422c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa41c1142f0 con 0x7fa41c1049a0 2024-09-19T05:08:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.700+0000 7fa41be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa41c103ff0 0x7fa41c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.700+0000 7fa41be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa41c103ff0 0x7fa41c19a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49382/0 (socket says 172.21.15.43:49382) 2024-09-19T05:08:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.700+0000 7fa41be00640 1 -- 172.21.15.43:0/88890632 learned_addr learned my addr 172.21.15.43:0/88890632 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.700+0000 7fa413e00640 1 --2- 172.21.15.43:0/88890632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa41c1049a0 0x7fa41c19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.700+0000 7fa420c00640 1 --2- 172.21.15.43:0/88890632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa41c109120 0x7fa41c19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:49.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.700+0000 7fa41be00640 1 -- 172.21.15.43:0/88890632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa41c1049a0 msgr2=0x7fa41c19abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:49.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.700+0000 7fa41be00640 1 --2- 172.21.15.43:0/88890632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa41c1049a0 0x7fa41c19abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:49.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.700+0000 7fa41be00640 1 -- 172.21.15.43:0/88890632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa41c109120 msgr2=0x7fa41c19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:49.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.700+0000 7fa41be00640 1 --2- 172.21.15.43:0/88890632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa41c109120 0x7fa41c19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:49.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.700+0000 7fa41be00640 1 -- 172.21.15.43:0/88890632 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa41c19f620 con 0x7fa41c103ff0 2024-09-19T05:08:49.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.700+0000 7fa413e00640 1 --2- 172.21.15.43:0/88890632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa41c1049a0 0x7fa41c19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:49.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.701+0000 7fa41be00640 1 --2- 172.21.15.43:0/88890632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa41c103ff0 0x7fa41c19a670 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa40c004770 tx=0x7fa40c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:49.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.701+0000 7fa419600640 1 -- 172.21.15.43:0/88890632 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c03f070 con 0x7fa41c103ff0 2024-09-19T05:08:49.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.701+0000 7fa419600640 1 -- 172.21.15.43:0/88890632 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa40c031ca0 con 0x7fa41c103ff0 2024-09-19T05:08:49.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.701+0000 7fa422c00640 1 -- 172.21.15.43:0/88890632 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa41c19f8b0 con 0x7fa41c103ff0 2024-09-19T05:08:49.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.701+0000 7fa419600640 1 -- 172.21.15.43:0/88890632 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c035030 con 0x7fa41c103ff0 2024-09-19T05:08:49.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.701+0000 7fa422c00640 1 -- 172.21.15.43:0/88890632 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa41c1a71f0 con 0x7fa41c103ff0 2024-09-19T05:08:49.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.704+0000 7fa419600640 1 -- 172.21.15.43:0/88890632 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa40c042070 con 0x7fa41c103ff0 2024-09-19T05:08:49.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.705+0000 7fa425626740 1 Processor -- start 2024-09-19T05:08:49.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.705+0000 7fa425626740 1 -- start start 2024-09-19T05:08:49.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.705+0000 7fa425626740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0e84ec70 0x561c0e84f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:49.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.705+0000 7fa425626740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0e84f590 0x561c0e859910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:49.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.706+0000 7fa419600640 1 --2- 172.21.15.43:0/88890632 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fa4000a33b0 0x7fa4000a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:49.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.706+0000 7fa425626740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c0e859e50 0x561c0e85c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:49.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.706+0000 7fa425626740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561c0e7b53e0 con 0x561c0e84ec70 2024-09-19T05:08:49.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.706+0000 7fa425626740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561c0e75cd80 con 0x561c0e84f590 2024-09-19T05:08:49.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.706+0000 7fa425626740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561c0e7b13a0 con 0x561c0e859e50 2024-09-19T05:08:49.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.706+0000 7fa419600640 1 -- 172.21.15.43:0/88890632 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fa40c0f0350 con 0x7fa41c103ff0 2024-09-19T05:08:49.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.706+0000 7fa41be00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0e84f590 0x561c0e859910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:49.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.706+0000 7fa41be00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0e84f590 0x561c0e859910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:41910/0 (socket says 172.21.15.43:41910) 2024-09-19T05:08:49.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.706+0000 7fa41be00640 1 -- 172.21.15.43:0/54712388 learned_addr learned my addr 172.21.15.43:0/54712388 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:49.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.706+0000 7fa413e00640 1 --2- 172.21.15.43:0/54712388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c0e859e50 0x561c0e85c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:49.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.706+0000 7fa420c00640 1 --2- 172.21.15.43:0/54712388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0e84ec70 0x561c0e84f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:49.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.706+0000 7fa41be00640 1 -- 172.21.15.43:0/54712388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c0e859e50 msgr2=0x561c0e85c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:49.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.706+0000 7fa41be00640 1 --2- 172.21.15.43:0/54712388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c0e859e50 0x561c0e85c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:49.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.706+0000 7fa41be00640 1 -- 172.21.15.43:0/54712388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0e84ec70 msgr2=0x561c0e84f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:49.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.706+0000 7fa41be00640 1 --2- 172.21.15.43:0/54712388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0e84ec70 0x561c0e84f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:49.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.706+0000 7fa41be00640 1 -- 172.21.15.43:0/54712388 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c0e77bae0 con 0x561c0e84f590 2024-09-19T05:08:49.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.706+0000 7fa420c00640 1 --2- 172.21.15.43:0/54712388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0e84ec70 0x561c0e84f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:08:49.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.706+0000 7fa420c00640 1 --2- 172.21.15.43:0/88890632 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fa4000a33b0 0x7fa4000a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:49.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.707+0000 7fa41be00640 1 --2- 172.21.15.43:0/54712388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0e84f590 0x561c0e859910 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa40c0edab0 tx=0x7fa40c03b890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ce9f3e853713c7d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:49.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.707+0000 7fa411200640 1 -- 172.21.15.43:0/54712388 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c049050 con 0x561c0e84f590 2024-09-19T05:08:49.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.707+0000 7fa411200640 1 -- 172.21.15.43:0/54712388 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa40c0c5310 con 0x561c0e84f590 2024-09-19T05:08:49.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.707+0000 7fa411200640 1 -- 172.21.15.43:0/54712388 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c0eded0 con 0x561c0e84f590 2024-09-19T05:08:49.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.708+0000 7fa425626740 1 -- 172.21.15.43:0/54712388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0e84f590 msgr2=0x561c0e859910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:49.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.708+0000 7fa420c00640 1 --2- 172.21.15.43:0/88890632 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fa4000a33b0 0x7fa4000a5870 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa414005fd0 tx=0x7fa414004a40 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:49.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.708+0000 7fa425626740 1 --2- 172.21.15.43:0/54712388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0e84f590 0x561c0e859910 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa40c0edab0 tx=0x7fa40c03b890 comp rx=0 tx=0).stop 2024-09-19T05:08:49.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.708+0000 7fa425626740 1 -- 172.21.15.43:0/54712388 shutdown_connections 2024-09-19T05:08:49.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.708+0000 7fa425626740 1 --2- 172.21.15.43:0/54712388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c0e859e50 0x561c0e85c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:49.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.708+0000 7fa425626740 1 --2- 172.21.15.43:0/54712388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0e84f590 0x561c0e859910 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:49.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.708+0000 7fa425626740 1 --2- 172.21.15.43:0/54712388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0e84ec70 0x561c0e84f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:49.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.708+0000 7fa425626740 1 -- 172.21.15.43:0/54712388 >> 172.21.15.43:0/54712388 conn(0x561c0e84b200 msgr2=0x561c0e84b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:49.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.708+0000 7fa425626740 1 -- 172.21.15.43:0/54712388 shutdown_connections 2024-09-19T05:08:49.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.708+0000 7fa425626740 1 -- 172.21.15.43:0/54712388 wait complete. 2024-09-19T05:08:49.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.708+0000 7fa425626740 1 -- start start 2024-09-19T05:08:49.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.709+0000 7fa425626740 1 Processor -- start 2024-09-19T05:08:49.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.709+0000 7fa425626740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0e84ec70 0x561c0e856db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:49.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.709+0000 7fa425626740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c0e84f590 0x561c0e8572f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:49.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.709+0000 7fa425626740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0e859e50 0x561c0e857830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:49.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.709+0000 7fa425626740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561c0e85c7e0 con 0x561c0e84ec70 2024-09-19T05:08:49.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.709+0000 7fa425626740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561c0e7700f0 con 0x561c0e859e50 2024-09-19T05:08:49.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.709+0000 7fa425626740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561c0e85c960 con 0x561c0e84f590 2024-09-19T05:08:49.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.709+0000 7fa425626740 10 client.0 _ll_register_callbacks cb 0x561c0e757bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:49.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.709+0000 7fa41be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0e84ec70 0x561c0e856db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:49.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.709+0000 7fa420c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0e859e50 0x561c0e857830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:49.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.709+0000 7fa413e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c0e84f590 0x561c0e8572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:49.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.709+0000 7fa41be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0e84ec70 0x561c0e856db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49400/0 (socket says 172.21.15.43:49400) 2024-09-19T05:08:49.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.709+0000 7fa420c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0e859e50 0x561c0e857830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:41920/0 (socket says 172.21.15.43:41920) 2024-09-19T05:08:49.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.710+0000 7fa41be00640 1 -- 172.21.15.43:0/129740172 learned_addr learned my addr 172.21.15.43:0/129740172 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:49.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.710+0000 7fa41be00640 1 -- 172.21.15.43:0/129740172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c0e84f590 msgr2=0x561c0e8572f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:49.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.710+0000 7fa41be00640 1 --2- 172.21.15.43:0/129740172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c0e84f590 0x561c0e8572f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:49.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.710+0000 7fa41be00640 1 -- 172.21.15.43:0/129740172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0e859e50 msgr2=0x561c0e857830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:49.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.710+0000 7fa41be00640 1 --2- 172.21.15.43:0/129740172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0e859e50 0x561c0e857830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:49.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.710+0000 7fa41be00640 1 -- 172.21.15.43:0/129740172 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c0e5263c0 con 0x561c0e84ec70 2024-09-19T05:08:49.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.710+0000 7fa413e00640 1 --2- 172.21.15.43:0/129740172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c0e84f590 0x561c0e8572f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:49.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.710+0000 7fa420c00640 1 --2- 172.21.15.43:0/129740172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0e859e50 0x561c0e857830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:49.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.710+0000 7fa41be00640 1 --2- 172.21.15.43:0/129740172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0e84ec70 0x561c0e856db0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa40c032fd0 tx=0x7fa40c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f4ad5db571185b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:49.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.710+0000 7fa409600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:08:49.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.710+0000 7fa409600640 1 -- 172.21.15.43:0/129740172 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c049070 con 0x561c0e84ec70 2024-09-19T05:08:49.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.710+0000 7fa409600640 1 -- 172.21.15.43:0/129740172 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fa40c0baa50 con 0x561c0e84ec70 2024-09-19T05:08:49.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.710+0000 7fa425626740 1 -- 172.21.15.43:0/129740172 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa40c0c5310 con 0x561c0e84ec70 2024-09-19T05:08:49.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.711+0000 7fa409600640 1 -- 172.21.15.43:0/129740172 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa40c0b8dc0 con 0x561c0e84ec70 2024-09-19T05:08:49.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.711+0000 7fa425626740 10 client.16008 fetch_fsmap learned FSMap version 29 2024-09-19T05:08:49.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.711+0000 7fa425626740 1 -- 172.21.15.43:0/129740172 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561c0e85c7e0 con 0x561c0e84ec70 2024-09-19T05:08:49.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.711+0000 7fa409600640 1 -- 172.21.15.43:0/129740172 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa40c0c5310 con 0x561c0e84ec70 2024-09-19T05:08:49.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.711+0000 7fa409600640 1 -- 172.21.15.43:0/129740172 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa40c0c4e80 con 0x561c0e84ec70 2024-09-19T05:08:49.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.711+0000 7fa425626740 10 client.16008 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:08:49.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.711+0000 7fa425626740 10 client.16008 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T05:08:49.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.711+0000 7fa425626740 20 client.16008 populate_metadata read hostname 'smithi043' 2024-09-19T05:08:49.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.711+0000 7fa425626740 1 --2- 172.21.15.43:0/129740172 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x561c0e8698f0 0x561c0e86bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:49.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.711+0000 7fa425626740 4 client.16008 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:08:49.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.711+0000 7fa425626740 1 -- 172.21.15.43:0/129740172 --> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c0e7c2950 con 0x561c0e8698f0 2024-09-19T05:08:49.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.712+0000 7fa413e00640 1 --2- 172.21.15.43:0/129740172 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x561c0e8698f0 0x561c0e86bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:49.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.712+0000 7fa413e00640 1 --2- 172.21.15.43:0/129740172 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x561c0e8698f0 0x561c0e86bce0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cd04110457e59b96 server_cookie=455e6e27dfab5bab in_seq=0 out_seq=0 2024-09-19T05:08:49.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.712+0000 7fa409600640 10 client.16008 ms_handle_connect on v2:172.21.15.96:6836/3978388286 2024-09-19T05:08:49.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.713+0000 7fa409600640 1 -- 172.21.15.43:0/129740172 <== mds.3 v2:172.21.15.96:6836/3978388286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561c0e7c2950 con 0x561c0e8698f0 2024-09-19T05:08:49.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.713+0000 7fa409600640 10 client.16008 handle_command_reply: tid=0 2024-09-19T05:08:49.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.713+0000 7fa425626740 1 client.16008 shutdown 2024-09-19T05:08:49.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.713+0000 7fa425626740 1 -- 172.21.15.43:0/129740172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0e84ec70 msgr2=0x561c0e856db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:49.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.713+0000 7fa425626740 1 --2- 172.21.15.43:0/129740172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0e84ec70 0x561c0e856db0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa40c032fd0 tx=0x7fa40c02a410 comp rx=0 tx=0).stop 2024-09-19T05:08:49.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.713+0000 7fa425626740 1 -- 172.21.15.43:0/129740172 shutdown_connections 2024-09-19T05:08:49.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.713+0000 7fa425626740 1 --2- 172.21.15.43:0/129740172 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x561c0e8698f0 0x561c0e86bce0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:49.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.713+0000 7fa425626740 1 --2- 172.21.15.43:0/129740172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0e859e50 0x561c0e857830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:49.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.713+0000 7fa425626740 1 --2- 172.21.15.43:0/129740172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c0e84f590 0x561c0e8572f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:49.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.713+0000 7fa425626740 1 --2- 172.21.15.43:0/129740172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0e84ec70 0x561c0e856db0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:49.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.713+0000 7fa409600640 0 client.16008 ms_handle_reset on v2:172.21.15.96:6836/3978388286 2024-09-19T05:08:49.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.713+0000 7fa425626740 1 -- 172.21.15.43:0/129740172 >> 172.21.15.43:0/129740172 conn(0x561c0e84b200 msgr2=0x561c0e84a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:49.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.714+0000 7fa425626740 1 -- 172.21.15.43:0/129740172 shutdown_connections 2024-09-19T05:08:49.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.714+0000 7fa425626740 1 -- 172.21.15.43:0/129740172 wait complete. 2024-09-19T05:08:49.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.714+0000 7fa425626740 20 client.16008 trim_cache size 0 max 16384 2024-09-19T05:08:49.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.728+0000 7fa425626740 1 Processor -- start 2024-09-19T05:08:49.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.728+0000 7fa425626740 1 -- start start 2024-09-19T05:08:49.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.728+0000 7fa425626740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0e84ec70 0x561c0e856670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:49.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.728+0000 7fa425626740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c0e84f590 0x561c0e871450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:49.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.728+0000 7fa425626740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0e859e50 0x561c0e871990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:49.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.728+0000 7fa425626740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa41c1afa90 con 0x561c0e859e50 2024-09-19T05:08:49.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.728+0000 7fa425626740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa41c1b15d0 con 0x561c0e84ec70 2024-09-19T05:08:49.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.728+0000 7fa425626740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa41c1b0160 con 0x561c0e84f590 2024-09-19T05:08:49.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.728+0000 7fa420c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0e859e50 0x561c0e871990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:49.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.728+0000 7fa420c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0e859e50 0x561c0e871990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49404/0 (socket says 172.21.15.43:49404) 2024-09-19T05:08:49.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.728+0000 7fa420c00640 1 -- 172.21.15.43:0/1524914624 learned_addr learned my addr 172.21.15.43:0/1524914624 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:49.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.728+0000 7fa41be00640 1 --2- 172.21.15.43:0/1524914624 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0e84ec70 0x561c0e856670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:49.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.728+0000 7fa413e00640 1 --2- 172.21.15.43:0/1524914624 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c0e84f590 0x561c0e871450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:49.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.729+0000 7fa420c00640 1 -- 172.21.15.43:0/1524914624 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c0e84f590 msgr2=0x561c0e871450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:49.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.729+0000 7fa420c00640 1 --2- 172.21.15.43:0/1524914624 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c0e84f590 0x561c0e871450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:49.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.729+0000 7fa420c00640 1 -- 172.21.15.43:0/1524914624 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0e84ec70 msgr2=0x561c0e856670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:49.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.729+0000 7fa420c00640 1 --2- 172.21.15.43:0/1524914624 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0e84ec70 0x561c0e856670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:49.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.729+0000 7fa420c00640 1 -- 172.21.15.43:0/1524914624 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c0e846770 con 0x561c0e859e50 2024-09-19T05:08:49.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.729+0000 7fa41be00640 1 --2- 172.21.15.43:0/1524914624 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0e84ec70 0x561c0e856670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:49.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.729+0000 7fa420c00640 1 --2- 172.21.15.43:0/1524914624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0e859e50 0x561c0e871990 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa41401fc90 tx=0x7fa414016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f848ca4d0493b212 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:49.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.729+0000 7fa40be00640 1 -- 172.21.15.43:0/1524914624 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa414004b20 con 0x561c0e859e50 2024-09-19T05:08:49.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.729+0000 7fa40be00640 1 -- 172.21.15.43:0/1524914624 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa41401ce50 con 0x561c0e859e50 2024-09-19T05:08:49.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.729+0000 7fa40be00640 1 -- 172.21.15.43:0/1524914624 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa414028540 con 0x561c0e859e50 2024-09-19T05:08:49.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.730+0000 7fa425626740 1 -- 172.21.15.43:0/1524914624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0e859e50 msgr2=0x561c0e871990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:49.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.730+0000 7fa425626740 1 --2- 172.21.15.43:0/1524914624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0e859e50 0x561c0e871990 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa41401fc90 tx=0x7fa414016000 comp rx=0 tx=0).stop 2024-09-19T05:08:49.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.730+0000 7fa425626740 1 -- 172.21.15.43:0/1524914624 shutdown_connections 2024-09-19T05:08:49.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.730+0000 7fa425626740 1 --2- 172.21.15.43:0/1524914624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0e859e50 0x561c0e871990 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:49.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.730+0000 7fa425626740 1 --2- 172.21.15.43:0/1524914624 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c0e84f590 0x561c0e871450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:49.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.730+0000 7fa425626740 1 --2- 172.21.15.43:0/1524914624 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0e84ec70 0x561c0e856670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:49.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.730+0000 7fa425626740 1 -- 172.21.15.43:0/1524914624 >> 172.21.15.43:0/1524914624 conn(0x561c0e84b200 msgr2=0x561c0e858e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:49.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.730+0000 7fa425626740 1 -- 172.21.15.43:0/1524914624 shutdown_connections 2024-09-19T05:08:49.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.730+0000 7fa425626740 1 -- 172.21.15.43:0/1524914624 wait complete. 2024-09-19T05:08:49.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.731+0000 7fa425626740 1 -- start start 2024-09-19T05:08:49.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.731+0000 7fa425626740 1 Processor -- start 2024-09-19T05:08:49.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.731+0000 7fa425626740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c0e84ec70 0x561c0e861cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:49.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.731+0000 7fa425626740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0e84f590 0x561c0e862200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:49.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.731+0000 7fa425626740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0e859e50 0x561c0e85d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:49.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.731+0000 7fa425626740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa41c1b0f00 con 0x561c0e84f590 2024-09-19T05:08:49.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.731+0000 7fa425626740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa41c1b0830 con 0x561c0e859e50 2024-09-19T05:08:49.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.731+0000 7fa425626740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa41c1ada70 con 0x561c0e84ec70 2024-09-19T05:08:49.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.731+0000 7fa425626740 10 client.0 _ll_register_callbacks cb 0x7fa404009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:49.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.732+0000 7fa413e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0e84f590 0x561c0e862200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:49.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.732+0000 7fa41be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c0e84ec70 0x561c0e861cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:49.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.732+0000 7fa413e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0e84f590 0x561c0e862200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49420/0 (socket says 172.21.15.43:49420) 2024-09-19T05:08:49.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.732+0000 7fa413e00640 1 -- 172.21.15.43:0/2740677279 learned_addr learned my addr 172.21.15.43:0/2740677279 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:49.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.732+0000 7fa420c00640 1 --2- 172.21.15.43:0/2740677279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0e859e50 0x561c0e85d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:49.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.732+0000 7fa413e00640 1 -- 172.21.15.43:0/2740677279 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c0e84ec70 msgr2=0x561c0e861cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:49.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.732+0000 7fa413e00640 1 --2- 172.21.15.43:0/2740677279 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c0e84ec70 0x561c0e861cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:49.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.732+0000 7fa413e00640 1 -- 172.21.15.43:0/2740677279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0e859e50 msgr2=0x561c0e85d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:49.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.732+0000 7fa413e00640 1 --2- 172.21.15.43:0/2740677279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0e859e50 0x561c0e85d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:49.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.732+0000 7fa413e00640 1 -- 172.21.15.43:0/2740677279 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c0e857d80 con 0x561c0e84f590 2024-09-19T05:08:49.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.732+0000 7fa413e00640 1 --2- 172.21.15.43:0/2740677279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0e84f590 0x561c0e862200 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa4040129f0 tx=0x7fa40401f7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b788141dafbe9f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:49.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.732+0000 7fa409600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:08:49.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.733+0000 7fa409600640 1 -- 172.21.15.43:0/2740677279 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa404004220 con 0x561c0e84f590 2024-09-19T05:08:49.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.733+0000 7fa409600640 1 -- 172.21.15.43:0/2740677279 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fa4040043c0 con 0x561c0e84f590 2024-09-19T05:08:49.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.733+0000 7fa425626740 1 -- 172.21.15.43:0/2740677279 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa41401ce50 con 0x561c0e84f590 2024-09-19T05:08:49.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.733+0000 7fa409600640 1 -- 172.21.15.43:0/2740677279 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa4040035f0 con 0x561c0e84f590 2024-09-19T05:08:49.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.733+0000 7fa425626740 10 client.16014 fetch_fsmap learned FSMap version 29 2024-09-19T05:08:49.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.733+0000 7fa425626740 1 -- 172.21.15.43:0/2740677279 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561c0e8d3db0 con 0x561c0e84f590 2024-09-19T05:08:49.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.733+0000 7fa409600640 1 -- 172.21.15.43:0/2740677279 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa41401ce50 con 0x561c0e84f590 2024-09-19T05:08:49.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.734+0000 7fa409600640 1 -- 172.21.15.43:0/2740677279 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa40400f780 con 0x561c0e84f590 2024-09-19T05:08:49.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.734+0000 7fa425626740 10 client.16014 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:08:49.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.734+0000 7fa425626740 10 client.16014 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T05:08:49.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.734+0000 7fa425626740 20 client.16014 populate_metadata read hostname 'smithi043' 2024-09-19T05:08:49.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.734+0000 7fa425626740 1 --2- 172.21.15.43:0/2740677279 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x561c0e8d4530 0x561c0e8d6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:49.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.734+0000 7fa425626740 4 client.16014 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:08:49.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.734+0000 7fa425626740 1 -- 172.21.15.43:0/2740677279 --> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561c0e7bce60 con 0x561c0e8d4530 2024-09-19T05:08:49.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.734+0000 7fa41be00640 1 --2- 172.21.15.43:0/2740677279 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x561c0e8d4530 0x561c0e8d6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:49.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.735+0000 7fa41be00640 1 --2- 172.21.15.43:0/2740677279 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x561c0e8d4530 0x561c0e8d6950 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2e3265212454e51 server_cookie=c3242074e51057d4 in_seq=0 out_seq=0 2024-09-19T05:08:49.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.735+0000 7fa409600640 10 client.16014 ms_handle_connect on v2:172.21.15.96:6836/3978388286 2024-09-19T05:08:49.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.735+0000 7fa409600640 1 -- 172.21.15.43:0/2740677279 <== mds.3 v2:172.21.15.96:6836/3978388286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561c0e7bce60 con 0x561c0e8d4530 2024-09-19T05:08:49.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.735+0000 7fa409600640 10 client.16014 handle_command_reply: tid=0 2024-09-19T05:08:49.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.735+0000 7fa425626740 1 client.16014 shutdown 2024-09-19T05:08:49.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.735+0000 7fa425626740 1 -- 172.21.15.43:0/2740677279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0e84f590 msgr2=0x561c0e862200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:49.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.735+0000 7fa425626740 1 --2- 172.21.15.43:0/2740677279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0e84f590 0x561c0e862200 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa4040129f0 tx=0x7fa40401f7c0 comp rx=0 tx=0).stop 2024-09-19T05:08:49.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.735+0000 7fa425626740 1 -- 172.21.15.43:0/2740677279 shutdown_connections 2024-09-19T05:08:49.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.735+0000 7fa425626740 1 --2- 172.21.15.43:0/2740677279 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x561c0e8d4530 0x561c0e8d6950 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:49.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.735+0000 7fa425626740 1 --2- 172.21.15.43:0/2740677279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0e859e50 0x561c0e85d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:49.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.735+0000 7fa425626740 1 --2- 172.21.15.43:0/2740677279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561c0e84f590 0x561c0e862200 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:49.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.735+0000 7fa425626740 1 --2- 172.21.15.43:0/2740677279 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c0e84ec70 0x561c0e861cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:49.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.735+0000 7fa425626740 1 -- 172.21.15.43:0/2740677279 >> 172.21.15.43:0/2740677279 conn(0x561c0e84b200 msgr2=0x561c0e8506c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:49.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.735+0000 7fa409600640 0 client.16014 ms_handle_reset on v2:172.21.15.96:6836/3978388286 2024-09-19T05:08:49.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.735+0000 7fa425626740 1 -- 172.21.15.43:0/2740677279 shutdown_connections 2024-09-19T05:08:49.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.735+0000 7fa425626740 1 -- 172.21.15.43:0/2740677279 wait complete. 2024-09-19T05:08:49.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.736+0000 7fa425626740 20 client.16014 trim_cache size 0 max 16384 2024-09-19T05:08:49.741 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:08:49.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.736+0000 7fa422c00640 1 -- 172.21.15.43:0/88890632 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fa4000a33b0 msgr2=0x7fa4000a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:49.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.736+0000 7fa422c00640 1 --2- 172.21.15.43:0/88890632 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fa4000a33b0 0x7fa4000a5870 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa414005fd0 tx=0x7fa414004a40 comp rx=0 tx=0).stop 2024-09-19T05:08:49.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.736+0000 7fa422c00640 1 -- 172.21.15.43:0/88890632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa41c103ff0 msgr2=0x7fa41c19a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:49.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.736+0000 7fa422c00640 1 --2- 172.21.15.43:0/88890632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa41c103ff0 0x7fa41c19a670 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa40c004770 tx=0x7fa40c003c10 comp rx=0 tx=0).stop 2024-09-19T05:08:49.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.736+0000 7fa422c00640 1 -- 172.21.15.43:0/88890632 shutdown_connections 2024-09-19T05:08:49.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.736+0000 7fa422c00640 1 --2- 172.21.15.43:0/88890632 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fa4000a33b0 0x7fa4000a5870 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:49.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.736+0000 7fa422c00640 1 --2- 172.21.15.43:0/88890632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa41c109120 0x7fa41c19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:49.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.736+0000 7fa422c00640 1 --2- 172.21.15.43:0/88890632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa41c1049a0 0x7fa41c19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:49.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.736+0000 7fa422c00640 1 --2- 172.21.15.43:0/88890632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa41c103ff0 0x7fa41c19a670 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:49.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.736+0000 7fa422c00640 1 -- 172.21.15.43:0/88890632 >> 172.21.15.43:0/88890632 conn(0x7fa41c0ffa90 msgr2=0x7fa41c100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:49.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.737+0000 7fa422c00640 1 -- 172.21.15.43:0/88890632 shutdown_connections 2024-09-19T05:08:49.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:49.737+0000 7fa422c00640 1 -- 172.21.15.43:0/88890632 wait complete. 2024-09-19T05:08:49.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:08:49 smithi043 ceph-mon[30800]: pgmap v128: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:08:49.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:08:49 smithi151 ceph-mon[35425]: pgmap v128: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:08:49.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:08:49 smithi096 ceph-mon[35085]: pgmap v128: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:08:50.687 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T05:08:51.070 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:08:51.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.674+0000 7fcbb9400640 1 -- 172.21.15.43:0/1573132414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbb4077040 msgr2=0x7fcbb4112510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:51.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.674+0000 7fcbb9400640 1 --2- 172.21.15.43:0/1573132414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbb4077040 0x7fcbb4112510 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009900 tx=0x7fcba002a410 comp rx=0 tx=0).stop 2024-09-19T05:08:51.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.674+0000 7fcbb9400640 1 -- 172.21.15.43:0/1573132414 shutdown_connections 2024-09-19T05:08:51.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.674+0000 7fcbb9400640 1 --2- 172.21.15.43:0/1573132414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcbb4112a50 0x7fcbb4114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:51.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.674+0000 7fcbb9400640 1 --2- 172.21.15.43:0/1573132414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbb4077040 0x7fcbb4112510 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:51.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.674+0000 7fcbb9400640 1 --2- 172.21.15.43:0/1573132414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcbb4076720 0x7fcbb4076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:51.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.674+0000 7fcbb9400640 1 -- 172.21.15.43:0/1573132414 >> 172.21.15.43:0/1573132414 conn(0x7fcbb40ffa50 msgr2=0x7fcbb4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:51.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.674+0000 7fcbb9400640 1 -- 172.21.15.43:0/1573132414 shutdown_connections 2024-09-19T05:08:51.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.674+0000 7fcbb9400640 1 -- 172.21.15.43:0/1573132414 wait complete. 2024-09-19T05:08:51.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.675+0000 7fcbb9400640 1 Processor -- start 2024-09-19T05:08:51.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.675+0000 7fcbb9400640 1 -- start start 2024-09-19T05:08:51.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.676+0000 7fcbb9400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcbb4076720 0x7fcbb41a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:51.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.676+0000 7fcbb9400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbb4077040 0x7fcbb41a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:51.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.676+0000 7fcbb9400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcbb4112a50 0x7fcbb419e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:51.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.676+0000 7fcbb9400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb41185c0 con 0x7fcbb4077040 2024-09-19T05:08:51.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.676+0000 7fcbb9400640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb4118440 con 0x7fcbb4112a50 2024-09-19T05:08:51.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.676+0000 7fcbb9400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb4118740 con 0x7fcbb4076720 2024-09-19T05:08:51.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.676+0000 7fcbb2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcbb4076720 0x7fcbb41a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:51.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.676+0000 7fcbb2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcbb4076720 0x7fcbb41a4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:39672/0 (socket says 172.21.15.43:39672) 2024-09-19T05:08:51.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.676+0000 7fcbb2800640 1 -- 172.21.15.43:0/529748020 learned_addr learned my addr 172.21.15.43:0/529748020 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:51.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.676+0000 7fcbb1e00640 1 --2- 172.21.15.43:0/529748020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbb4077040 0x7fcbb41a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:51.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.676+0000 7fcbb3200640 1 --2- 172.21.15.43:0/529748020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcbb4112a50 0x7fcbb419e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:51.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.677+0000 7fcbb3200640 1 -- 172.21.15.43:0/529748020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcbb4076720 msgr2=0x7fcbb41a4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:51.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.677+0000 7fcbb3200640 1 --2- 172.21.15.43:0/529748020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcbb4076720 0x7fcbb41a4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:51.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.677+0000 7fcbb3200640 1 -- 172.21.15.43:0/529748020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbb4077040 msgr2=0x7fcbb41a4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:51.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.677+0000 7fcbb3200640 1 --2- 172.21.15.43:0/529748020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbb4077040 0x7fcbb41a4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:51.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.677+0000 7fcbb3200640 1 -- 172.21.15.43:0/529748020 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb419f020 con 0x7fcbb4112a50 2024-09-19T05:08:51.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.677+0000 7fcbb2800640 1 --2- 172.21.15.43:0/529748020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcbb4076720 0x7fcbb41a4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:51.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.677+0000 7fcbb3200640 1 --2- 172.21.15.43:0/529748020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcbb4112a50 0x7fcbb419e8f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcbb407aa10 tx=0x7fcbac00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:51.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.677+0000 7fcbab400640 1 -- 172.21.15.43:0/529748020 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbac00cdf0 con 0x7fcbb4112a50 2024-09-19T05:08:51.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.677+0000 7fcbab400640 1 -- 172.21.15.43:0/529748020 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbac005020 con 0x7fcbb4112a50 2024-09-19T05:08:51.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.677+0000 7fcbb9400640 1 -- 172.21.15.43:0/529748020 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbb419f310 con 0x7fcbb4112a50 2024-09-19T05:08:51.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.677+0000 7fcbab400640 1 -- 172.21.15.43:0/529748020 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbac010670 con 0x7fcbb4112a50 2024-09-19T05:08:51.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.677+0000 7fcbb9400640 1 -- 172.21.15.43:0/529748020 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbb41ab600 con 0x7fcbb4112a50 2024-09-19T05:08:51.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.680+0000 7fcbbbdc3740 1 Processor -- start 2024-09-19T05:08:51.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.680+0000 7fcbab400640 1 -- 172.21.15.43:0/529748020 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fcbac0054d0 con 0x7fcbb4112a50 2024-09-19T05:08:51.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.680+0000 7fcbbbdc3740 1 -- start start 2024-09-19T05:08:51.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.681+0000 7fcbbbdc3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594ea53cc70 0x5594ea53d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:51.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.681+0000 7fcbbbdc3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594ea53d590 0x5594ea547910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:51.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.681+0000 7fcbbbdc3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594ea547e50 0x5594ea54a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:51.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.681+0000 7fcbbbdc3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5594ea4a33e0 con 0x5594ea53d590 2024-09-19T05:08:51.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.681+0000 7fcbbbdc3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5594ea44ad80 con 0x5594ea547e50 2024-09-19T05:08:51.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.681+0000 7fcbbbdc3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5594ea49f3a0 con 0x5594ea53cc70 2024-09-19T05:08:51.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.681+0000 7fcbb2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594ea53d590 0x5594ea547910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:51.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.681+0000 7fcbb1e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594ea53cc70 0x5594ea53d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:51.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.681+0000 7fcbb1e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594ea53cc70 0x5594ea53d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:39680/0 (socket says 172.21.15.43:39680) 2024-09-19T05:08:51.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.681+0000 7fcbb1e00640 1 -- 172.21.15.43:0/4187188626 learned_addr learned my addr 172.21.15.43:0/4187188626 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:51.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.681+0000 7fcbb1e00640 1 --2- 172.21.15.43:0/4187188626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594ea547e50 0x5594ea54a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:51.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.681+0000 7fcbb2800640 1 -- 172.21.15.43:0/4187188626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594ea53cc70 msgr2=0x5594ea53d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:51.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.681+0000 7fcbb2800640 1 --2- 172.21.15.43:0/4187188626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594ea53cc70 0x5594ea53d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:51.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.681+0000 7fcbb2800640 1 -- 172.21.15.43:0/4187188626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594ea547e50 msgr2=0x5594ea54a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:51.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.681+0000 7fcbb2800640 1 --2- 172.21.15.43:0/4187188626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594ea547e50 0x5594ea54a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:51.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.681+0000 7fcbb2800640 1 -- 172.21.15.43:0/4187188626 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594ea469ae0 con 0x5594ea53d590 2024-09-19T05:08:51.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.682+0000 7fcbb2800640 1 --2- 172.21.15.43:0/4187188626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594ea53d590 0x5594ea547910 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcba4005fd0 tx=0x7fcba4010960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d53bc17d37506ac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:51.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.682+0000 7fcb9f400640 1 -- 172.21.15.43:0/4187188626 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba4004230 con 0x5594ea53d590 2024-09-19T05:08:51.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.682+0000 7fcbab400640 1 --2- 172.21.15.43:0/529748020 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fcb980a33b0 0x7fcb980a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:51.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.682+0000 7fcb9f400640 1 -- 172.21.15.43:0/4187188626 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcba40043d0 con 0x5594ea53d590 2024-09-19T05:08:51.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.682+0000 7fcb9f400640 1 -- 172.21.15.43:0/4187188626 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba40035f0 con 0x5594ea53d590 2024-09-19T05:08:51.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.682+0000 7fcbab400640 1 -- 172.21.15.43:0/529748020 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fcbac0d03b0 con 0x7fcbb4112a50 2024-09-19T05:08:51.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.682+0000 7fcbb1e00640 1 --2- 172.21.15.43:0/529748020 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fcb980a33b0 0x7fcb980a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:51.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.683+0000 7fcbbbdc3740 1 -- 172.21.15.43:0/4187188626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594ea53d590 msgr2=0x5594ea547910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:51.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.683+0000 7fcbbbdc3740 1 --2- 172.21.15.43:0/4187188626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594ea53d590 0x5594ea547910 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcba4005fd0 tx=0x7fcba4010960 comp rx=0 tx=0).stop 2024-09-19T05:08:51.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.683+0000 7fcbbbdc3740 1 -- 172.21.15.43:0/4187188626 shutdown_connections 2024-09-19T05:08:51.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.683+0000 7fcbbbdc3740 1 --2- 172.21.15.43:0/4187188626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594ea547e50 0x5594ea54a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:51.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.683+0000 7fcbbbdc3740 1 --2- 172.21.15.43:0/4187188626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594ea53d590 0x5594ea547910 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:51.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.683+0000 7fcbbbdc3740 1 --2- 172.21.15.43:0/4187188626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594ea53cc70 0x5594ea53d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:51.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.683+0000 7fcbbbdc3740 1 -- 172.21.15.43:0/4187188626 >> 172.21.15.43:0/4187188626 conn(0x5594ea539200 msgr2=0x5594ea5395e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:51.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.683+0000 7fcbbbdc3740 1 -- 172.21.15.43:0/4187188626 shutdown_connections 2024-09-19T05:08:51.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.683+0000 7fcbbbdc3740 1 -- 172.21.15.43:0/4187188626 wait complete. 2024-09-19T05:08:51.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.683+0000 7fcbb1e00640 1 --2- 172.21.15.43:0/529748020 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fcb980a33b0 0x7fcb980a5870 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x5594ea53a580 tx=0x7fcba00058d0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:51.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.684+0000 7fcbbbdc3740 1 -- start start 2024-09-19T05:08:51.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.684+0000 7fcbbbdc3740 1 Processor -- start 2024-09-19T05:08:51.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.684+0000 7fcbbbdc3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594ea53cc70 0x5594ea544ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:51.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.684+0000 7fcbbbdc3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594ea53d590 0x5594ea545220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:51.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.684+0000 7fcbbbdc3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594ea547e50 0x5594ea545760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:51.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.684+0000 7fcbbbdc3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5594ea54aa20 con 0x5594ea53d590 2024-09-19T05:08:51.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.684+0000 7fcbbbdc3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5594ea45e0f0 con 0x5594ea547e50 2024-09-19T05:08:51.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.684+0000 7fcbbbdc3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5594ea54aba0 con 0x5594ea53cc70 2024-09-19T05:08:51.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.684+0000 7fcbbbdc3740 10 client.0 _ll_register_callbacks cb 0x5594ea445bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:51.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.685+0000 7fcbb1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594ea53d590 0x5594ea545220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:51.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.685+0000 7fcbb3200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594ea547e50 0x5594ea545760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:51.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.685+0000 7fcbb2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594ea53cc70 0x5594ea544ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:51.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.685+0000 7fcbb1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594ea53d590 0x5594ea545220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49462/0 (socket says 172.21.15.43:49462) 2024-09-19T05:08:51.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.685+0000 7fcbb3200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594ea547e50 0x5594ea545760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:41968/0 (socket says 172.21.15.43:41968) 2024-09-19T05:08:51.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.685+0000 7fcbb1e00640 1 -- 172.21.15.43:0/1182986957 learned_addr learned my addr 172.21.15.43:0/1182986957 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:51.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.685+0000 7fcbb3200640 1 -- 172.21.15.43:0/1182986957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594ea53cc70 msgr2=0x5594ea544ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:51.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.685+0000 7fcbb3200640 1 --2- 172.21.15.43:0/1182986957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594ea53cc70 0x5594ea544ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:51.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.685+0000 7fcbb3200640 1 -- 172.21.15.43:0/1182986957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594ea53d590 msgr2=0x5594ea545220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:51.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.685+0000 7fcbb3200640 1 --2- 172.21.15.43:0/1182986957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594ea53d590 0x5594ea545220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:51.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.685+0000 7fcbb3200640 1 -- 172.21.15.43:0/1182986957 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594ea2143c0 con 0x5594ea547e50 2024-09-19T05:08:51.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.685+0000 7fcbb3200640 1 --2- 172.21.15.43:0/1182986957 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594ea547e50 0x5594ea545760 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcbac0cdab0 tx=0x7fcbac09ce00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf54741c85d738d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:51.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.685+0000 7fcb93e00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:08:51.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.685+0000 7fcb93e00640 1 -- 172.21.15.43:0/1182986957 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbac0b4020 con 0x5594ea547e50 2024-09-19T05:08:51.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.685+0000 7fcb93e00640 1 -- 172.21.15.43:0/1182986957 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fcbac09c8a0 con 0x5594ea547e50 2024-09-19T05:08:51.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.685+0000 7fcbbbdc3740 1 -- 172.21.15.43:0/1182986957 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcba40043d0 con 0x5594ea547e50 2024-09-19T05:08:51.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.686+0000 7fcb93e00640 1 -- 172.21.15.43:0/1182986957 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcbac0a52b0 con 0x5594ea547e50 2024-09-19T05:08:51.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.686+0000 7fcbbbdc3740 10 client.26069 fetch_fsmap learned FSMap version 29 2024-09-19T05:08:51.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.686+0000 7fcbbbdc3740 1 -- 172.21.15.43:0/1182986957 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5594ea556a50 con 0x5594ea547e50 2024-09-19T05:08:51.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.686+0000 7fcb93e00640 1 -- 172.21.15.43:0/1182986957 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcba40043d0 con 0x5594ea547e50 2024-09-19T05:08:51.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.686+0000 7fcb93e00640 1 -- 172.21.15.43:0/1182986957 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcbac0b0b60 con 0x5594ea547e50 2024-09-19T05:08:51.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.686+0000 7fcbbbdc3740 10 client.26069 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:08:51.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.686+0000 7fcbbbdc3740 10 client.26069 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T05:08:51.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.686+0000 7fcbbbdc3740 20 client.26069 populate_metadata read hostname 'smithi043' 2024-09-19T05:08:51.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.686+0000 7fcbbbdc3740 1 --2- 172.21.15.43:0/1182986957 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x5594ea557e60 0x5594ea55a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:51.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.686+0000 7fcbbbdc3740 4 client.26069 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:08:51.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.686+0000 7fcbbbdc3740 1 -- 172.21.15.43:0/1182986957 --> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594ea4b0950 con 0x5594ea557e60 2024-09-19T05:08:51.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.687+0000 7fcbb2800640 1 --2- 172.21.15.43:0/1182986957 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x5594ea557e60 0x5594ea55a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:51.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.687+0000 7fcbb2800640 1 --2- 172.21.15.43:0/1182986957 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x5594ea557e60 0x5594ea55a2d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5496b35411552fa5 server_cookie=6a5d3a08f3a716f3 in_seq=0 out_seq=0 2024-09-19T05:08:51.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.687+0000 7fcb93e00640 10 client.26069 ms_handle_connect on v2:172.21.15.151:6838/3794104313 2024-09-19T05:08:51.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.688+0000 7fcb93e00640 1 -- 172.21.15.43:0/1182986957 <== mds.2 v2:172.21.15.151:6838/3794104313 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5594ea4b0950 con 0x5594ea557e60 2024-09-19T05:08:51.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.688+0000 7fcb93e00640 10 client.26069 handle_command_reply: tid=0 2024-09-19T05:08:51.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.689+0000 7fcbbbdc3740 1 client.26069 shutdown 2024-09-19T05:08:51.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.689+0000 7fcbbbdc3740 1 -- 172.21.15.43:0/1182986957 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594ea547e50 msgr2=0x5594ea545760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:51.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.689+0000 7fcbbbdc3740 1 --2- 172.21.15.43:0/1182986957 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594ea547e50 0x5594ea545760 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcbac0cdab0 tx=0x7fcbac09ce00 comp rx=0 tx=0).stop 2024-09-19T05:08:51.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.689+0000 7fcbbbdc3740 1 -- 172.21.15.43:0/1182986957 shutdown_connections 2024-09-19T05:08:51.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.689+0000 7fcbbbdc3740 1 --2- 172.21.15.43:0/1182986957 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x5594ea557e60 0x5594ea55a2d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:51.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.689+0000 7fcbbbdc3740 1 --2- 172.21.15.43:0/1182986957 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594ea547e50 0x5594ea545760 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:51.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.689+0000 7fcbbbdc3740 1 --2- 172.21.15.43:0/1182986957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594ea53d590 0x5594ea545220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:51.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.689+0000 7fcbbbdc3740 1 --2- 172.21.15.43:0/1182986957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594ea53cc70 0x5594ea544ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:51.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.689+0000 7fcb93e00640 0 client.26069 ms_handle_reset on v2:172.21.15.151:6838/3794104313 2024-09-19T05:08:51.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.689+0000 7fcbbbdc3740 1 -- 172.21.15.43:0/1182986957 >> 172.21.15.43:0/1182986957 conn(0x5594ea539200 msgr2=0x5594ea538380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:51.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.689+0000 7fcbbbdc3740 1 -- 172.21.15.43:0/1182986957 shutdown_connections 2024-09-19T05:08:51.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.689+0000 7fcbbbdc3740 1 -- 172.21.15.43:0/1182986957 wait complete. 2024-09-19T05:08:51.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.689+0000 7fcbbbdc3740 20 client.26069 trim_cache size 0 max 16384 2024-09-19T05:08:51.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.703+0000 7fcbbbdc3740 1 Processor -- start 2024-09-19T05:08:51.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.703+0000 7fcbbbdc3740 1 -- start start 2024-09-19T05:08:51.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.703+0000 7fcbbbdc3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594ea53cc70 0x5594ea53f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:51.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.703+0000 7fcbbbdc3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594ea53d590 0x5594ea55fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:51.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.703+0000 7fcbbbdc3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594ea547e50 0x5594ea560100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:51.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.703+0000 7fcbbbdc3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb41b3f30 con 0x5594ea53cc70 2024-09-19T05:08:51.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.703+0000 7fcbbbdc3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb41b5a70 con 0x5594ea547e50 2024-09-19T05:08:51.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.703+0000 7fcbbbdc3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb41b4600 con 0x5594ea53d590 2024-09-19T05:08:51.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.703+0000 7fcbb2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594ea53cc70 0x5594ea53f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:51.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.703+0000 7fcbb2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594ea53cc70 0x5594ea53f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49476/0 (socket says 172.21.15.43:49476) 2024-09-19T05:08:51.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.703+0000 7fcbb2800640 1 -- 172.21.15.43:0/39686158 learned_addr learned my addr 172.21.15.43:0/39686158 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:51.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.703+0000 7fcbb1e00640 1 --2- 172.21.15.43:0/39686158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594ea53d590 0x5594ea55fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:51.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.704+0000 7fcbb3200640 1 --2- 172.21.15.43:0/39686158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594ea547e50 0x5594ea560100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:51.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.704+0000 7fcbb2800640 1 -- 172.21.15.43:0/39686158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594ea53d590 msgr2=0x5594ea55fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:51.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.704+0000 7fcbb2800640 1 --2- 172.21.15.43:0/39686158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594ea53d590 0x5594ea55fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:51.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.704+0000 7fcbb2800640 1 -- 172.21.15.43:0/39686158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594ea547e50 msgr2=0x5594ea560100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:51.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.704+0000 7fcbb2800640 1 --2- 172.21.15.43:0/39686158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594ea547e50 0x5594ea560100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:51.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.704+0000 7fcbb2800640 1 -- 172.21.15.43:0/39686158 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594ea53db00 con 0x5594ea53cc70 2024-09-19T05:08:51.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.704+0000 7fcbb3200640 1 --2- 172.21.15.43:0/39686158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594ea547e50 0x5594ea560100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:08:51.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.704+0000 7fcbb2800640 1 --2- 172.21.15.43:0/39686158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594ea53cc70 0x5594ea53f6d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fcba400b770 tx=0x7fcba4004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18fcd60f9aa02a43 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:51.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.704+0000 7fcb9ea00640 1 -- 172.21.15.43:0/39686158 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba4004050 con 0x5594ea53cc70 2024-09-19T05:08:51.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.704+0000 7fcb9ea00640 1 -- 172.21.15.43:0/39686158 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcba40035f0 con 0x5594ea53cc70 2024-09-19T05:08:51.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.704+0000 7fcb9ea00640 1 -- 172.21.15.43:0/39686158 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba40115c0 con 0x5594ea53cc70 2024-09-19T05:08:51.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.705+0000 7fcbbbdc3740 1 -- 172.21.15.43:0/39686158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594ea53cc70 msgr2=0x5594ea53f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:51.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.705+0000 7fcbbbdc3740 1 --2- 172.21.15.43:0/39686158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594ea53cc70 0x5594ea53f6d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fcba400b770 tx=0x7fcba4004ac0 comp rx=0 tx=0).stop 2024-09-19T05:08:51.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.705+0000 7fcbbbdc3740 1 -- 172.21.15.43:0/39686158 shutdown_connections 2024-09-19T05:08:51.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.705+0000 7fcbbbdc3740 1 --2- 172.21.15.43:0/39686158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594ea547e50 0x5594ea560100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:51.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.705+0000 7fcbbbdc3740 1 --2- 172.21.15.43:0/39686158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594ea53d590 0x5594ea55fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:51.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.705+0000 7fcbbbdc3740 1 --2- 172.21.15.43:0/39686158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594ea53cc70 0x5594ea53f6d0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:51.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.705+0000 7fcbbbdc3740 1 -- 172.21.15.43:0/39686158 >> 172.21.15.43:0/39686158 conn(0x5594ea539200 msgr2=0x5594ea546d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:51.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.705+0000 7fcbbbdc3740 1 -- 172.21.15.43:0/39686158 shutdown_connections 2024-09-19T05:08:51.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.705+0000 7fcbbbdc3740 1 -- 172.21.15.43:0/39686158 wait complete. 2024-09-19T05:08:51.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.706+0000 7fcbbbdc3740 1 -- start start 2024-09-19T05:08:51.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.706+0000 7fcbbbdc3740 1 Processor -- start 2024-09-19T05:08:51.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.706+0000 7fcbbbdc3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594ea53cc70 0x5594ea54c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:51.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.706+0000 7fcbbbdc3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594ea53d590 0x5594ea54c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:51.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.706+0000 7fcbbbdc3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594ea547e50 0x5594ea5c1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:51.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.706+0000 7fcbbbdc3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb41b53a0 con 0x5594ea53d590 2024-09-19T05:08:51.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.706+0000 7fcbbbdc3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb41b4cd0 con 0x5594ea53cc70 2024-09-19T05:08:51.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.706+0000 7fcbbbdc3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb41b1f10 con 0x5594ea547e50 2024-09-19T05:08:51.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.706+0000 7fcbbbdc3740 10 client.0 _ll_register_callbacks cb 0x7fcbac09c1f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:51.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.707+0000 7fcbb1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594ea53d590 0x5594ea54c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:51.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.707+0000 7fcbb1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594ea53d590 0x5594ea54c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49484/0 (socket says 172.21.15.43:49484) 2024-09-19T05:08:51.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.707+0000 7fcbb1e00640 1 -- 172.21.15.43:0/3890311599 learned_addr learned my addr 172.21.15.43:0/3890311599 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:51.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.707+0000 7fcbb2800640 1 --2- 172.21.15.43:0/3890311599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594ea53cc70 0x5594ea54c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:51.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.707+0000 7fcbb3200640 1 --2- 172.21.15.43:0/3890311599 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594ea547e50 0x5594ea5c1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:51.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.707+0000 7fcbb2800640 1 -- 172.21.15.43:0/3890311599 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594ea547e50 msgr2=0x5594ea5c1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:51.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.707+0000 7fcbb2800640 1 --2- 172.21.15.43:0/3890311599 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594ea547e50 0x5594ea5c1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:51.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.707+0000 7fcbb2800640 1 -- 172.21.15.43:0/3890311599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594ea53d590 msgr2=0x5594ea54c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:51.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.707+0000 7fcbb2800640 1 --2- 172.21.15.43:0/3890311599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594ea53d590 0x5594ea54c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:51.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.707+0000 7fcbb2800640 1 -- 172.21.15.43:0/3890311599 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594ea534770 con 0x5594ea53cc70 2024-09-19T05:08:51.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.707+0000 7fcbb2800640 1 --2- 172.21.15.43:0/3890311599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594ea53cc70 0x5594ea54c200 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcba40104a0 tx=0x7fcba400bcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ba989f8409e768c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:51.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.707+0000 7fcb93e00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:08:51.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.708+0000 7fcb93e00640 1 -- 172.21.15.43:0/3890311599 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba40037a0 con 0x5594ea53cc70 2024-09-19T05:08:51.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.708+0000 7fcb93e00640 1 -- 172.21.15.43:0/3890311599 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fcba4022040 con 0x5594ea53cc70 2024-09-19T05:08:51.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.708+0000 7fcbbbdc3740 1 -- 172.21.15.43:0/3890311599 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcba40035f0 con 0x5594ea53cc70 2024-09-19T05:08:51.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.708+0000 7fcb93e00640 1 -- 172.21.15.43:0/3890311599 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcba403c050 con 0x5594ea53cc70 2024-09-19T05:08:51.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.708+0000 7fcbbbdc3740 10 client.26075 fetch_fsmap learned FSMap version 29 2024-09-19T05:08:51.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.708+0000 7fcbbbdc3740 1 -- 172.21.15.43:0/3890311599 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5594ea5c3160 con 0x5594ea53cc70 2024-09-19T05:08:51.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.708+0000 7fcb93e00640 1 -- 172.21.15.43:0/3890311599 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcba40035f0 con 0x5594ea53cc70 2024-09-19T05:08:51.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.708+0000 7fcb93e00640 1 -- 172.21.15.43:0/3890311599 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcba4011650 con 0x5594ea53cc70 2024-09-19T05:08:51.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.708+0000 7fcbbbdc3740 10 client.26075 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:08:51.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.708+0000 7fcbbbdc3740 10 client.26075 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T05:08:51.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.708+0000 7fcbbbdc3740 20 client.26075 populate_metadata read hostname 'smithi043' 2024-09-19T05:08:51.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.708+0000 7fcbbbdc3740 1 --2- 172.21.15.43:0/3890311599 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x5594ea5c35d0 0x5594ea5c59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:51.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.709+0000 7fcbbbdc3740 4 client.26075 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:08:51.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.709+0000 7fcbbbdc3740 1 -- 172.21.15.43:0/3890311599 --> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5594ea4aae60 con 0x5594ea5c35d0 2024-09-19T05:08:51.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.709+0000 7fcbb1e00640 1 --2- 172.21.15.43:0/3890311599 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x5594ea5c35d0 0x5594ea5c59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:51.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.710+0000 7fcbb1e00640 1 --2- 172.21.15.43:0/3890311599 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x5594ea5c35d0 0x5594ea5c59c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bf4275834b5b13e3 server_cookie=2a9cfa3225689776 in_seq=0 out_seq=0 2024-09-19T05:08:51.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.710+0000 7fcb93e00640 10 client.26075 ms_handle_connect on v2:172.21.15.151:6838/3794104313 2024-09-19T05:08:51.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.710+0000 7fcb93e00640 1 -- 172.21.15.43:0/3890311599 <== mds.2 v2:172.21.15.151:6838/3794104313 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5594ea4aae60 con 0x5594ea5c35d0 2024-09-19T05:08:51.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.710+0000 7fcb93e00640 10 client.26075 handle_command_reply: tid=0 2024-09-19T05:08:51.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.710+0000 7fcbbbdc3740 1 client.26075 shutdown 2024-09-19T05:08:51.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.710+0000 7fcbbbdc3740 1 -- 172.21.15.43:0/3890311599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594ea53cc70 msgr2=0x5594ea54c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:51.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.710+0000 7fcbbbdc3740 1 --2- 172.21.15.43:0/3890311599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594ea53cc70 0x5594ea54c200 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcba40104a0 tx=0x7fcba400bcf0 comp rx=0 tx=0).stop 2024-09-19T05:08:51.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.710+0000 7fcbbbdc3740 1 -- 172.21.15.43:0/3890311599 shutdown_connections 2024-09-19T05:08:51.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.710+0000 7fcbbbdc3740 1 --2- 172.21.15.43:0/3890311599 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x5594ea5c35d0 0x5594ea5c59c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:51.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.710+0000 7fcbbbdc3740 1 --2- 172.21.15.43:0/3890311599 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5594ea547e50 0x5594ea5c1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:51.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.710+0000 7fcbbbdc3740 1 --2- 172.21.15.43:0/3890311599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5594ea53d590 0x5594ea54c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:51.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.710+0000 7fcbbbdc3740 1 --2- 172.21.15.43:0/3890311599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594ea53cc70 0x5594ea54c200 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:51.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.710+0000 7fcb93e00640 0 client.26075 ms_handle_reset on v2:172.21.15.151:6838/3794104313 2024-09-19T05:08:51.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.710+0000 7fcbbbdc3740 1 -- 172.21.15.43:0/3890311599 >> 172.21.15.43:0/3890311599 conn(0x5594ea539200 msgr2=0x5594ea537b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:51.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.710+0000 7fcbbbdc3740 1 -- 172.21.15.43:0/3890311599 shutdown_connections 2024-09-19T05:08:51.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.710+0000 7fcbbbdc3740 1 -- 172.21.15.43:0/3890311599 wait complete. 2024-09-19T05:08:51.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.710+0000 7fcbbbdc3740 20 client.26075 trim_cache size 0 max 16384 2024-09-19T05:08:51.719 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:08:51.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.711+0000 7fcbb9400640 1 -- 172.21.15.43:0/529748020 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fcb980a33b0 msgr2=0x7fcb980a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:51.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.711+0000 7fcbb9400640 1 --2- 172.21.15.43:0/529748020 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fcb980a33b0 0x7fcb980a5870 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x5594ea53a580 tx=0x7fcba00058d0 comp rx=0 tx=0).stop 2024-09-19T05:08:51.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.711+0000 7fcbb9400640 1 -- 172.21.15.43:0/529748020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcbb4112a50 msgr2=0x7fcbb419e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:51.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.711+0000 7fcbb9400640 1 --2- 172.21.15.43:0/529748020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcbb4112a50 0x7fcbb419e8f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcbb407aa10 tx=0x7fcbac00ef10 comp rx=0 tx=0).stop 2024-09-19T05:08:51.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.711+0000 7fcbb9400640 1 -- 172.21.15.43:0/529748020 shutdown_connections 2024-09-19T05:08:51.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.711+0000 7fcbb9400640 1 --2- 172.21.15.43:0/529748020 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fcb980a33b0 0x7fcb980a5870 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:51.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.711+0000 7fcbb9400640 1 --2- 172.21.15.43:0/529748020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcbb4112a50 0x7fcbb419e8f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:51.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.711+0000 7fcbb9400640 1 --2- 172.21.15.43:0/529748020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbb4077040 0x7fcbb41a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:51.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.711+0000 7fcbb9400640 1 --2- 172.21.15.43:0/529748020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcbb4076720 0x7fcbb41a4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:51.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.711+0000 7fcbb9400640 1 -- 172.21.15.43:0/529748020 >> 172.21.15.43:0/529748020 conn(0x7fcbb40ffa50 msgr2=0x7fcbb4101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:51.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.711+0000 7fcbb9400640 1 -- 172.21.15.43:0/529748020 shutdown_connections 2024-09-19T05:08:51.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:51.711+0000 7fcbb9400640 1 -- 172.21.15.43:0/529748020 wait complete. 2024-09-19T05:08:51.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:08:51 smithi043 ceph-mon[30800]: pgmap v129: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:08:51.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:08:51 smithi151 ceph-mon[35425]: pgmap v129: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:08:51.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:08:51 smithi096 ceph-mon[35085]: pgmap v129: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:08:52.716 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T05:08:53.097 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:08:53.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.719+0000 7fccf6200640 1 -- 172.21.15.43:0/808418186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccf0106f20 msgr2=0x7fccf010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:53.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.719+0000 7fccf6200640 1 --2- 172.21.15.43:0/808418186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccf0106f20 0x7fccf010eaa0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fcce8009ca0 tx=0x7fcce802a8c0 comp rx=0 tx=0).stop 2024-09-19T05:08:53.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.719+0000 7fccf6200640 1 -- 172.21.15.43:0/808418186 shutdown_connections 2024-09-19T05:08:53.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.719+0000 7fccf6200640 1 --2- 172.21.15.43:0/808418186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccf0106f20 0x7fccf010eaa0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:53.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.719+0000 7fccf6200640 1 --2- 172.21.15.43:0/808418186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fccf01027a0 0x7fccf01067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:53.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.719+0000 7fccf6200640 1 --2- 172.21.15.43:0/808418186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fccf0101df0 0x7fccf01021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:53.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.719+0000 7fccf6200640 1 -- 172.21.15.43:0/808418186 >> 172.21.15.43:0/808418186 conn(0x7fccf00fd8b0 msgr2=0x7fccf00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:53.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.719+0000 7fccf6200640 1 -- 172.21.15.43:0/808418186 shutdown_connections 2024-09-19T05:08:53.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.719+0000 7fccf6200640 1 -- 172.21.15.43:0/808418186 wait complete. 2024-09-19T05:08:53.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.720+0000 7fccf6200640 1 Processor -- start 2024-09-19T05:08:53.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.720+0000 7fccf6200640 1 -- start start 2024-09-19T05:08:53.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.720+0000 7fccf6200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fccf0101df0 0x7fccf019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:53.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.720+0000 7fccf6200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccf01027a0 0x7fccf019aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:53.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.720+0000 7fccf6200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fccf0106f20 0x7fccf019ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:53.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.720+0000 7fccf6200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fccf0111fd0 con 0x7fccf01027a0 2024-09-19T05:08:53.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.720+0000 7fccf6200640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fccf0111e50 con 0x7fccf0106f20 2024-09-19T05:08:53.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.720+0000 7fccf6200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fccf0112150 con 0x7fccf0101df0 2024-09-19T05:08:53.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.721+0000 7fcceea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccf01027a0 0x7fccf019aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:53.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.721+0000 7fcceea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccf01027a0 0x7fccf019aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49494/0 (socket says 172.21.15.43:49494) 2024-09-19T05:08:53.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.721+0000 7fcceea00640 1 -- 172.21.15.43:0/1791156070 learned_addr learned my addr 172.21.15.43:0/1791156070 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:53.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.721+0000 7fccefe00640 1 --2- 172.21.15.43:0/1791156070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fccf0106f20 0x7fccf019ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:53.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.721+0000 7fccef400640 1 --2- 172.21.15.43:0/1791156070 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fccf0101df0 0x7fccf019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:53.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.721+0000 7fcceea00640 1 -- 172.21.15.43:0/1791156070 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fccf0101df0 msgr2=0x7fccf019a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:53.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.721+0000 7fcceea00640 1 --2- 172.21.15.43:0/1791156070 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fccf0101df0 0x7fccf019a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:53.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.721+0000 7fcceea00640 1 -- 172.21.15.43:0/1791156070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fccf0106f20 msgr2=0x7fccf019ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:53.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.721+0000 7fcceea00640 1 --2- 172.21.15.43:0/1791156070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fccf0106f20 0x7fccf019ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:53.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.721+0000 7fcceea00640 1 -- 172.21.15.43:0/1791156070 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf019f550 con 0x7fccf01027a0 2024-09-19T05:08:53.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.722+0000 7fcceea00640 1 --2- 172.21.15.43:0/1791156070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccf01027a0 0x7fccf019aae0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fccdc00e9f0 tx=0x7fccdc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:53.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.722+0000 7fcce7e00640 1 -- 172.21.15.43:0/1791156070 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccdc00cd80 con 0x7fccf01027a0 2024-09-19T05:08:53.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.722+0000 7fcce7e00640 1 -- 172.21.15.43:0/1791156070 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccdc010910 con 0x7fccf01027a0 2024-09-19T05:08:53.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.722+0000 7fccf6200640 1 -- 172.21.15.43:0/1791156070 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccf019f840 con 0x7fccf01027a0 2024-09-19T05:08:53.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.722+0000 7fcce7e00640 1 -- 172.21.15.43:0/1791156070 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccdc018990 con 0x7fccf01027a0 2024-09-19T05:08:53.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.722+0000 7fccf6200640 1 -- 172.21.15.43:0/1791156070 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccf01a7120 con 0x7fccf01027a0 2024-09-19T05:08:53.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.725+0000 7fccf8e22740 1 Processor -- start 2024-09-19T05:08:53.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.725+0000 7fccf8e22740 1 -- start start 2024-09-19T05:08:53.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.725+0000 7fccf8e22740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba88328b90 0x55ba88328f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:53.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.726+0000 7fccf8e22740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba883294b0 0x55ba8833c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:53.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.726+0000 7fccf8e22740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba8833c660 0x55ba8833ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:53.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.726+0000 7fccf8e22740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ba8828f3e0 con 0x55ba8833c660 2024-09-19T05:08:53.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.726+0000 7fccf8e22740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ba88236d80 con 0x55ba88328b90 2024-09-19T05:08:53.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.726+0000 7fccf8e22740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ba8828b3a0 con 0x55ba883294b0 2024-09-19T05:08:53.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.726+0000 7fcce7e00640 1 -- 172.21.15.43:0/1791156070 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fccdc010ac0 con 0x7fccf01027a0 2024-09-19T05:08:53.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.726+0000 7fccefe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba88328b90 0x55ba88328f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:53.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.726+0000 7fccef400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba883294b0 0x55ba8833c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:53.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.726+0000 7fccefe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba88328b90 0x55ba88328f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:42006/0 (socket says 172.21.15.43:42006) 2024-09-19T05:08:53.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.726+0000 7fccefe00640 1 -- 172.21.15.43:0/2934297059 learned_addr learned my addr 172.21.15.43:0/2934297059 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:53.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.726+0000 7fcceea00640 1 --2- 172.21.15.43:0/2934297059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba8833c660 0x55ba8833ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:53.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.726+0000 7fccefe00640 1 -- 172.21.15.43:0/2934297059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba883294b0 msgr2=0x55ba8833c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:53.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.726+0000 7fccefe00640 1 --2- 172.21.15.43:0/2934297059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba883294b0 0x55ba8833c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:53.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.726+0000 7fccefe00640 1 -- 172.21.15.43:0/2934297059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba8833c660 msgr2=0x55ba8833ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:53.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.726+0000 7fccefe00640 1 --2- 172.21.15.43:0/2934297059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba8833c660 0x55ba8833ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:53.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.726+0000 7fccefe00640 1 -- 172.21.15.43:0/2934297059 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba88255ae0 con 0x55ba88328b90 2024-09-19T05:08:53.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.726+0000 7fcceea00640 1 --2- 172.21.15.43:0/2934297059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba8833c660 0x55ba8833ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:08:53.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.727+0000 7fccefe00640 1 --2- 172.21.15.43:0/2934297059 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba88328b90 0x55ba88328f70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcce8009ca0 tx=0x7fcce80093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3532bd1457a58ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:53.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.727+0000 7fccdbe00640 1 -- 172.21.15.43:0/2934297059 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce8031db0 con 0x55ba88328b90 2024-09-19T05:08:53.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.727+0000 7fccdbe00640 1 -- 172.21.15.43:0/2934297059 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcce8036030 con 0x55ba88328b90 2024-09-19T05:08:53.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.727+0000 7fccdbe00640 1 -- 172.21.15.43:0/2934297059 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce80042a0 con 0x55ba88328b90 2024-09-19T05:08:53.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.728+0000 7fcce7e00640 1 --2- 172.21.15.43:0/1791156070 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fccd40a33b0 0x7fccd40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:53.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.728+0000 7fcce7e00640 1 -- 172.21.15.43:0/1791156070 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fccdc0cffe0 con 0x7fccf01027a0 2024-09-19T05:08:53.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.728+0000 7fccef400640 1 --2- 172.21.15.43:0/1791156070 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fccd40a33b0 0x7fccd40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:53.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.728+0000 7fccf8e22740 1 -- 172.21.15.43:0/2934297059 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba88328b90 msgr2=0x55ba88328f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:53.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.728+0000 7fccf8e22740 1 --2- 172.21.15.43:0/2934297059 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba88328b90 0x55ba88328f70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcce8009ca0 tx=0x7fcce80093e0 comp rx=0 tx=0).stop 2024-09-19T05:08:53.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.728+0000 7fccf8e22740 1 -- 172.21.15.43:0/2934297059 shutdown_connections 2024-09-19T05:08:53.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.728+0000 7fccf8e22740 1 --2- 172.21.15.43:0/2934297059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba8833c660 0x55ba8833ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:53.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.728+0000 7fccf8e22740 1 --2- 172.21.15.43:0/2934297059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba883294b0 0x55ba8833c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:53.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.728+0000 7fccf8e22740 1 --2- 172.21.15.43:0/2934297059 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba88328b90 0x55ba88328f70 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:53.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.728+0000 7fccf8e22740 1 -- 172.21.15.43:0/2934297059 >> 172.21.15.43:0/2934297059 conn(0x55ba88325200 msgr2=0x55ba883255e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:53.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.728+0000 7fccf8e22740 1 -- 172.21.15.43:0/2934297059 shutdown_connections 2024-09-19T05:08:53.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.729+0000 7fccf8e22740 1 -- 172.21.15.43:0/2934297059 wait complete. 2024-09-19T05:08:53.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.729+0000 7fccef400640 1 --2- 172.21.15.43:0/1791156070 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fccd40a33b0 0x7fccd40a5870 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcce0009e00 tx=0x7fcce0005f90 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:53.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.729+0000 7fccf8e22740 1 -- start start 2024-09-19T05:08:53.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.729+0000 7fccf8e22740 1 Processor -- start 2024-09-19T05:08:53.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.730+0000 7fccf8e22740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba88328b90 0x55ba883395e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:53.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.730+0000 7fccf8e22740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba883294b0 0x55ba88339b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:53.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.730+0000 7fccf8e22740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba8833c660 0x55ba8833a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:53.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.730+0000 7fccf8e22740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ba8833eff0 con 0x55ba88328b90 2024-09-19T05:08:53.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.730+0000 7fccf8e22740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ba8824a0f0 con 0x55ba8833c660 2024-09-19T05:08:53.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.730+0000 7fccf8e22740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ba8833f170 con 0x55ba883294b0 2024-09-19T05:08:53.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.730+0000 7fccf8e22740 10 client.0 _ll_register_callbacks cb 0x55ba88231bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:53.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.730+0000 7fccef400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba88328b90 0x55ba883395e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:53.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.730+0000 7fccef400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba88328b90 0x55ba883395e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49520/0 (socket says 172.21.15.43:49520) 2024-09-19T05:08:53.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.730+0000 7fccef400640 1 -- 172.21.15.43:0/3030833001 learned_addr learned my addr 172.21.15.43:0/3030833001 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:53.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.730+0000 7fccefe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba8833c660 0x55ba8833a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:53.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.730+0000 7fcceea00640 1 --2- 172.21.15.43:0/3030833001 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba883294b0 0x55ba88339b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:53.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.731+0000 7fccefe00640 1 -- 172.21.15.43:0/3030833001 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba883294b0 msgr2=0x55ba88339b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:53.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.731+0000 7fccefe00640 1 --2- 172.21.15.43:0/3030833001 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba883294b0 0x55ba88339b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:53.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.731+0000 7fccefe00640 1 -- 172.21.15.43:0/3030833001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba88328b90 msgr2=0x55ba883395e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:53.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.731+0000 7fccefe00640 1 --2- 172.21.15.43:0/3030833001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba88328b90 0x55ba883395e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:53.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.731+0000 7fccefe00640 1 -- 172.21.15.43:0/3030833001 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba880003c0 con 0x55ba8833c660 2024-09-19T05:08:53.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.731+0000 7fcceea00640 1 --2- 172.21.15.43:0/3030833001 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba883294b0 0x55ba88339b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:53.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.731+0000 7fccefe00640 1 --2- 172.21.15.43:0/3030833001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba8833c660 0x55ba8833a060 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcce8009bf0 tx=0x7fcce8031810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99432f4eaea38297 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:53.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.731+0000 7fccd8c00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:08:53.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.731+0000 7fccd8c00640 1 -- 172.21.15.43:0/3030833001 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce804a040 con 0x55ba8833c660 2024-09-19T05:08:53.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.731+0000 7fccd8c00640 1 -- 172.21.15.43:0/3030833001 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fcce8003970 con 0x55ba8833c660 2024-09-19T05:08:53.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.731+0000 7fccf8e22740 1 -- 172.21.15.43:0/3030833001 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcce8036030 con 0x55ba8833c660 2024-09-19T05:08:53.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.732+0000 7fccd8c00640 1 -- 172.21.15.43:0/3030833001 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcce805e050 con 0x55ba8833c660 2024-09-19T05:08:53.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.732+0000 7fccf8e22740 10 client.26087 fetch_fsmap learned FSMap version 29 2024-09-19T05:08:53.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.732+0000 7fccf8e22740 1 -- 172.21.15.43:0/3030833001 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ba8833eff0 con 0x55ba8833c660 2024-09-19T05:08:53.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.732+0000 7fccd8c00640 1 -- 172.21.15.43:0/3030833001 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcce8036030 con 0x55ba8833c660 2024-09-19T05:08:53.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.732+0000 7fccd8c00640 1 -- 172.21.15.43:0/3030833001 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcce804d030 con 0x55ba8833c660 2024-09-19T05:08:53.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.732+0000 7fccf8e22740 10 client.26087 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:08:53.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.732+0000 7fccf8e22740 10 client.26087 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-19T05:08:53.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.733+0000 7fccf8e22740 20 client.26087 populate_metadata read hostname 'smithi043' 2024-09-19T05:08:53.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.733+0000 7fccf8e22740 1 --2- 172.21.15.43:0/3030833001 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55ba8834c230 0x55ba8834e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:53.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.733+0000 7fccf8e22740 4 client.26087 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:08:53.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.733+0000 7fccf8e22740 1 -- 172.21.15.43:0/3030833001 --> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba8829c950 con 0x55ba8834c230 2024-09-19T05:08:53.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.733+0000 7fccef400640 1 --2- 172.21.15.43:0/3030833001 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55ba8834c230 0x55ba8834e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:53.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.733+0000 7fccef400640 1 --2- 172.21.15.43:0/3030833001 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55ba8834c230 0x55ba8834e620 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9b44b0ff8870532a server_cookie=b1a6fc24cd4e951d in_seq=0 out_seq=0 2024-09-19T05:08:53.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.733+0000 7fccd8c00640 10 client.26087 ms_handle_connect on v2:172.21.15.43:6838/3404555030 2024-09-19T05:08:53.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.734+0000 7fccd8c00640 1 -- 172.21.15.43:0/3030833001 <== mds.1 v2:172.21.15.43:6838/3404555030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ba8829c950 con 0x55ba8834c230 2024-09-19T05:08:53.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.734+0000 7fccd8c00640 10 client.26087 handle_command_reply: tid=0 2024-09-19T05:08:53.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.734+0000 7fccf8e22740 1 client.26087 shutdown 2024-09-19T05:08:53.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.734+0000 7fccf8e22740 1 -- 172.21.15.43:0/3030833001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba8833c660 msgr2=0x55ba8833a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:53.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.734+0000 7fccf8e22740 1 --2- 172.21.15.43:0/3030833001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba8833c660 0x55ba8833a060 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcce8009bf0 tx=0x7fcce8031810 comp rx=0 tx=0).stop 2024-09-19T05:08:53.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.734+0000 7fccf8e22740 1 -- 172.21.15.43:0/3030833001 shutdown_connections 2024-09-19T05:08:53.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.734+0000 7fccf8e22740 1 --2- 172.21.15.43:0/3030833001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba8833c660 0x55ba8833a060 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:53.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.734+0000 7fccf8e22740 1 --2- 172.21.15.43:0/3030833001 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55ba8834c230 0x55ba8834e620 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:53.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.734+0000 7fccf8e22740 1 --2- 172.21.15.43:0/3030833001 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba883294b0 0x55ba88339b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:53.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.734+0000 7fccf8e22740 1 --2- 172.21.15.43:0/3030833001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba88328b90 0x55ba883395e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:53.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.734+0000 7fccd8c00640 0 client.26087 ms_handle_reset on v2:172.21.15.43:6838/3404555030 2024-09-19T05:08:53.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.734+0000 7fccf8e22740 1 -- 172.21.15.43:0/3030833001 >> 172.21.15.43:0/3030833001 conn(0x55ba88325200 msgr2=0x55ba88324380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:53.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.735+0000 7fccf8e22740 1 -- 172.21.15.43:0/3030833001 shutdown_connections 2024-09-19T05:08:53.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.735+0000 7fccf8e22740 1 -- 172.21.15.43:0/3030833001 wait complete. 2024-09-19T05:08:53.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.735+0000 7fccf8e22740 20 client.26087 trim_cache size 0 max 16384 2024-09-19T05:08:53.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.749+0000 7fccf8e22740 1 Processor -- start 2024-09-19T05:08:53.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.749+0000 7fccf8e22740 1 -- start start 2024-09-19T05:08:53.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.749+0000 7fccf8e22740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba88328b90 0x55ba88353ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:53.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.749+0000 7fccf8e22740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba883294b0 0x55ba88354400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:53.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.749+0000 7fccf8e22740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba8833c660 0x55ba88336840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:53.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.749+0000 7fccf8e22740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fccf01ab550 con 0x55ba8833c660 2024-09-19T05:08:53.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.749+0000 7fccf8e22740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fccf01ad090 con 0x55ba88328b90 2024-09-19T05:08:53.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.749+0000 7fccf8e22740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fccf01abc20 con 0x55ba883294b0 2024-09-19T05:08:53.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.749+0000 7fccefe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba8833c660 0x55ba88336840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:53.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.749+0000 7fccefe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba8833c660 0x55ba88336840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49534/0 (socket says 172.21.15.43:49534) 2024-09-19T05:08:53.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.749+0000 7fccefe00640 1 -- 172.21.15.43:0/1420495921 learned_addr learned my addr 172.21.15.43:0/1420495921 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:53.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.749+0000 7fccef400640 1 --2- 172.21.15.43:0/1420495921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba88328b90 0x55ba88353ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:53.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.749+0000 7fcceea00640 1 --2- 172.21.15.43:0/1420495921 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba883294b0 0x55ba88354400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:53.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.750+0000 7fccefe00640 1 -- 172.21.15.43:0/1420495921 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba883294b0 msgr2=0x55ba88354400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:53.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.750+0000 7fccefe00640 1 --2- 172.21.15.43:0/1420495921 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba883294b0 0x55ba88354400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:53.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.750+0000 7fccefe00640 1 -- 172.21.15.43:0/1420495921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba88328b90 msgr2=0x55ba88353ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:53.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.750+0000 7fccefe00640 1 --2- 172.21.15.43:0/1420495921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba88328b90 0x55ba88353ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:53.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.750+0000 7fccefe00640 1 -- 172.21.15.43:0/1420495921 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba883292e0 con 0x55ba8833c660 2024-09-19T05:08:53.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.750+0000 7fccef400640 1 --2- 172.21.15.43:0/1420495921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba88328b90 0x55ba88353ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:53.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.750+0000 7fccefe00640 1 --2- 172.21.15.43:0/1420495921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba8833c660 0x55ba88336840 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcce8000f80 tx=0x7fcce8031760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b64720e51667795b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:53.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.750+0000 7fccdb400640 1 -- 172.21.15.43:0/1420495921 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce8009540 con 0x55ba8833c660 2024-09-19T05:08:53.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.750+0000 7fccdb400640 1 -- 172.21.15.43:0/1420495921 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcce8036040 con 0x55ba8833c660 2024-09-19T05:08:53.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.750+0000 7fccdb400640 1 -- 172.21.15.43:0/1420495921 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce802fba0 con 0x55ba8833c660 2024-09-19T05:08:53.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.751+0000 7fccf8e22740 1 -- 172.21.15.43:0/1420495921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba8833c660 msgr2=0x55ba88336840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:53.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.751+0000 7fccf8e22740 1 --2- 172.21.15.43:0/1420495921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba8833c660 0x55ba88336840 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcce8000f80 tx=0x7fcce8031760 comp rx=0 tx=0).stop 2024-09-19T05:08:53.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.751+0000 7fccf8e22740 1 -- 172.21.15.43:0/1420495921 shutdown_connections 2024-09-19T05:08:53.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.751+0000 7fccf8e22740 1 --2- 172.21.15.43:0/1420495921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba8833c660 0x55ba88336840 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:53.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.751+0000 7fccf8e22740 1 --2- 172.21.15.43:0/1420495921 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba883294b0 0x55ba88354400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:53.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.751+0000 7fccf8e22740 1 --2- 172.21.15.43:0/1420495921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba88328b90 0x55ba88353ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:53.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.751+0000 7fccf8e22740 1 -- 172.21.15.43:0/1420495921 >> 172.21.15.43:0/1420495921 conn(0x55ba88325200 msgr2=0x55ba8833b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:53.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.751+0000 7fccf8e22740 1 -- 172.21.15.43:0/1420495921 shutdown_connections 2024-09-19T05:08:53.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.751+0000 7fccf8e22740 1 -- 172.21.15.43:0/1420495921 wait complete. 2024-09-19T05:08:53.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.752+0000 7fccf8e22740 1 -- start start 2024-09-19T05:08:53.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.752+0000 7fccf8e22740 1 Processor -- start 2024-09-19T05:08:53.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.752+0000 7fccf8e22740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba88328b90 0x55ba88344b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:53.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.752+0000 7fccf8e22740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba883294b0 0x55ba8833fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:53.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.752+0000 7fccf8e22740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba8833c660 0x55ba88340430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:53.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.752+0000 7fccf8e22740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fccf01ac9c0 con 0x55ba88328b90 2024-09-19T05:08:53.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.752+0000 7fccf8e22740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fccf01ac2f0 con 0x55ba8833c660 2024-09-19T05:08:53.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.752+0000 7fccf8e22740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fccf01a9550 con 0x55ba883294b0 2024-09-19T05:08:53.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.752+0000 7fccf8e22740 10 client.0 _ll_register_callbacks cb 0x7fcce0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:53.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.753+0000 7fccef400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba88328b90 0x55ba88344b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:53.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.753+0000 7fccef400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba88328b90 0x55ba88344b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49544/0 (socket says 172.21.15.43:49544) 2024-09-19T05:08:53.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.753+0000 7fccef400640 1 -- 172.21.15.43:0/1567353046 learned_addr learned my addr 172.21.15.43:0/1567353046 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:53.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.753+0000 7fcceea00640 1 --2- 172.21.15.43:0/1567353046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba883294b0 0x55ba8833fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:53.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.753+0000 7fccefe00640 1 --2- 172.21.15.43:0/1567353046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba8833c660 0x55ba88340430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:53.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.753+0000 7fccef400640 1 -- 172.21.15.43:0/1567353046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba883294b0 msgr2=0x55ba8833fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:53.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.753+0000 7fccef400640 1 --2- 172.21.15.43:0/1567353046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba883294b0 0x55ba8833fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:53.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.753+0000 7fccef400640 1 -- 172.21.15.43:0/1567353046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba8833c660 msgr2=0x55ba88340430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:53.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.753+0000 7fccef400640 1 --2- 172.21.15.43:0/1567353046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba8833c660 0x55ba88340430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:53.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.753+0000 7fccef400640 1 -- 172.21.15.43:0/1567353046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba88340c30 con 0x55ba88328b90 2024-09-19T05:08:53.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.753+0000 7fccef400640 1 --2- 172.21.15.43:0/1567353046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba88328b90 0x55ba88344b20 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcce00096e0 tx=0x7fcce00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb0966e7c08d83b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:53.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.753+0000 7fccd8c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:08:53.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.754+0000 7fccd8c00640 1 -- 172.21.15.43:0/1567353046 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce001dd70 con 0x55ba88328b90 2024-09-19T05:08:53.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.754+0000 7fccd8c00640 1 -- 172.21.15.43:0/1567353046 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fcce0004020 con 0x55ba88328b90 2024-09-19T05:08:53.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.754+0000 7fccf8e22740 1 -- 172.21.15.43:0/1567353046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcce8036040 con 0x55ba88328b90 2024-09-19T05:08:53.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.754+0000 7fccd8c00640 1 -- 172.21.15.43:0/1567353046 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcce0004220 con 0x55ba88328b90 2024-09-19T05:08:53.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.754+0000 7fccf8e22740 10 client.16047 fetch_fsmap learned FSMap version 29 2024-09-19T05:08:53.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.754+0000 7fccf8e22740 1 -- 172.21.15.43:0/1567353046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ba883b6ec0 con 0x55ba88328b90 2024-09-19T05:08:53.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.754+0000 7fccd8c00640 1 -- 172.21.15.43:0/1567353046 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcce8036040 con 0x55ba88328b90 2024-09-19T05:08:53.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.755+0000 7fccd8c00640 1 -- 172.21.15.43:0/1567353046 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcce0004b80 con 0x55ba88328b90 2024-09-19T05:08:53.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.755+0000 7fccf8e22740 10 client.16047 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:08:53.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.755+0000 7fccf8e22740 10 client.16047 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-19T05:08:53.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.755+0000 7fccf8e22740 20 client.16047 populate_metadata read hostname 'smithi043' 2024-09-19T05:08:53.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.755+0000 7fccf8e22740 1 --2- 172.21.15.43:0/1567353046 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55ba883b75c0 0x55ba883b99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:53.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.755+0000 7fccf8e22740 4 client.16047 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:08:53.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.755+0000 7fccf8e22740 1 -- 172.21.15.43:0/1567353046 --> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ba88296e60 con 0x55ba883b75c0 2024-09-19T05:08:53.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.755+0000 7fcceea00640 1 --2- 172.21.15.43:0/1567353046 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55ba883b75c0 0x55ba883b99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:53.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.755+0000 7fcceea00640 1 --2- 172.21.15.43:0/1567353046 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55ba883b75c0 0x55ba883b99e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=86e2bc5983ee9446 server_cookie=2f8274817d54a5f7 in_seq=0 out_seq=0 2024-09-19T05:08:53.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.755+0000 7fccd8c00640 10 client.16047 ms_handle_connect on v2:172.21.15.43:6838/3404555030 2024-09-19T05:08:53.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.756+0000 7fccd8c00640 1 -- 172.21.15.43:0/1567353046 <== mds.1 v2:172.21.15.43:6838/3404555030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ba88296e60 con 0x55ba883b75c0 2024-09-19T05:08:53.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.756+0000 7fccd8c00640 10 client.16047 handle_command_reply: tid=0 2024-09-19T05:08:53.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.756+0000 7fccf8e22740 1 client.16047 shutdown 2024-09-19T05:08:53.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.756+0000 7fccf8e22740 1 -- 172.21.15.43:0/1567353046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba88328b90 msgr2=0x55ba88344b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:53.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.756+0000 7fccf8e22740 1 --2- 172.21.15.43:0/1567353046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba88328b90 0x55ba88344b20 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcce00096e0 tx=0x7fcce00046c0 comp rx=0 tx=0).stop 2024-09-19T05:08:53.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.756+0000 7fccf8e22740 1 -- 172.21.15.43:0/1567353046 shutdown_connections 2024-09-19T05:08:53.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.756+0000 7fccf8e22740 1 --2- 172.21.15.43:0/1567353046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba8833c660 0x55ba88340430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:53.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.756+0000 7fccf8e22740 1 --2- 172.21.15.43:0/1567353046 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55ba883b75c0 0x55ba883b99e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:53.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.756+0000 7fccf8e22740 1 --2- 172.21.15.43:0/1567353046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba883294b0 0x55ba8833fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:53.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.756+0000 7fccd8c00640 0 client.16047 ms_handle_reset on v2:172.21.15.43:6838/3404555030 2024-09-19T05:08:53.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.756+0000 7fccf8e22740 1 --2- 172.21.15.43:0/1567353046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ba88328b90 0x55ba88344b20 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:53.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.756+0000 7fccf8e22740 1 -- 172.21.15.43:0/1567353046 >> 172.21.15.43:0/1567353046 conn(0x55ba88325200 msgr2=0x55ba8832a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:53.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.756+0000 7fccf8e22740 1 -- 172.21.15.43:0/1567353046 shutdown_connections 2024-09-19T05:08:53.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.756+0000 7fccf8e22740 1 -- 172.21.15.43:0/1567353046 wait complete. 2024-09-19T05:08:53.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.756+0000 7fccf8e22740 20 client.16047 trim_cache size 0 max 16384 2024-09-19T05:08:53.761 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:08:53.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.757+0000 7fccf6200640 1 -- 172.21.15.43:0/1791156070 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fccd40a33b0 msgr2=0x7fccd40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:53.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.757+0000 7fccf6200640 1 --2- 172.21.15.43:0/1791156070 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fccd40a33b0 0x7fccd40a5870 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcce0009e00 tx=0x7fcce0005f90 comp rx=0 tx=0).stop 2024-09-19T05:08:53.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.757+0000 7fccf6200640 1 -- 172.21.15.43:0/1791156070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccf01027a0 msgr2=0x7fccf019aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:53.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.757+0000 7fccf6200640 1 --2- 172.21.15.43:0/1791156070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccf01027a0 0x7fccf019aae0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fccdc00e9f0 tx=0x7fccdc00eeb0 comp rx=0 tx=0).stop 2024-09-19T05:08:53.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.757+0000 7fccf6200640 1 -- 172.21.15.43:0/1791156070 shutdown_connections 2024-09-19T05:08:53.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.757+0000 7fccf6200640 1 --2- 172.21.15.43:0/1791156070 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fccd40a33b0 0x7fccd40a5870 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:53.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.757+0000 7fccf6200640 1 --2- 172.21.15.43:0/1791156070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fccf0106f20 0x7fccf019ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:53.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.757+0000 7fccf6200640 1 --2- 172.21.15.43:0/1791156070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccf01027a0 0x7fccf019aae0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:53.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.757+0000 7fccf6200640 1 --2- 172.21.15.43:0/1791156070 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fccf0101df0 0x7fccf019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:53.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.757+0000 7fccf6200640 1 -- 172.21.15.43:0/1791156070 >> 172.21.15.43:0/1791156070 conn(0x7fccf00fd8b0 msgr2=0x7fccf00fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:53.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.757+0000 7fccf6200640 1 -- 172.21.15.43:0/1791156070 shutdown_connections 2024-09-19T05:08:53.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:53.757+0000 7fccf6200640 1 -- 172.21.15.43:0/1791156070 wait complete. 2024-09-19T05:08:53.772 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:08:53 smithi043 ceph-mon[30800]: pgmap v130: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:08:53.775 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:08:53 smithi096 ceph-mon[35085]: pgmap v130: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:08:53.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:08:53 smithi151 ceph-mon[35425]: pgmap v130: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:08:55.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:08:55 smithi043 ceph-mon[30800]: pgmap v131: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:08:55.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:08:55 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:08:55.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:08:55 smithi151 ceph-mon[35425]: pgmap v131: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:08:55.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:08:55 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:08:55.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:08:55 smithi096 ceph-mon[35085]: pgmap v131: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:08:55.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:08:55 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:08:56.442 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T05:08:56.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:08:56 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:08:56.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:08:56 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:08:56.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:08:56 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:08:56.829 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:08:56.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:08:56 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:08:56.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:08:56 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:08:56.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:08:56 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:08:56.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:08:56 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:08:56.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:08:56 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:08:56.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:08:56 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:08:57.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.480+0000 7f954d400640 1 -- 172.21.15.43:0/3298515477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95481079c0 msgr2=0x7f9548107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:57.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.480+0000 7f954d400640 1 --2- 172.21.15.43:0/3298515477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95481079c0 0x7f9548107da0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9538009cd0 tx=0x7f953802a8e0 comp rx=0 tx=0).stop 2024-09-19T05:08:57.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.480+0000 7f954d400640 1 -- 172.21.15.43:0/3298515477 shutdown_connections 2024-09-19T05:08:57.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.480+0000 7f954d400640 1 --2- 172.21.15.43:0/3298515477 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9548102360 0x7f954810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:57.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.480+0000 7f954d400640 1 --2- 172.21.15.43:0/3298515477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95481019c0 0x7f9548101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:57.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.480+0000 7f954d400640 1 --2- 172.21.15.43:0/3298515477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95481079c0 0x7f9548107da0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:57.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.480+0000 7f954d400640 1 -- 172.21.15.43:0/3298515477 >> 172.21.15.43:0/3298515477 conn(0x7f95480fd740 msgr2=0x7f95480ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:57.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.480+0000 7f954d400640 1 -- 172.21.15.43:0/3298515477 shutdown_connections 2024-09-19T05:08:57.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.480+0000 7f954d400640 1 -- 172.21.15.43:0/3298515477 wait complete. 2024-09-19T05:08:57.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.482+0000 7f954d400640 1 Processor -- start 2024-09-19T05:08:57.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.482+0000 7f954d400640 1 -- start start 2024-09-19T05:08:57.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.482+0000 7f954d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95481019c0 0x7f954819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:57.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.482+0000 7f954d400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9548102360 0x7f954819a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:57.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.482+0000 7f954d400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95481079c0 0x7f954819ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:57.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.482+0000 7f954d400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9548111e10 con 0x7f95481019c0 2024-09-19T05:08:57.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.482+0000 7f954d400640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9548111c90 con 0x7f95481079c0 2024-09-19T05:08:57.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.482+0000 7f954d400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9548111f90 con 0x7f9548102360 2024-09-19T05:08:57.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.483+0000 7f9547e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95481019c0 0x7f954819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:57.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.483+0000 7f9547e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95481019c0 0x7f954819a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48792/0 (socket says 172.21.15.43:48792) 2024-09-19T05:08:57.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.483+0000 7f9547e00640 1 -- 172.21.15.43:0/923007546 learned_addr learned my addr 172.21.15.43:0/923007546 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:57.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.483+0000 7f954ca00640 1 --2- 172.21.15.43:0/923007546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95481079c0 0x7f954819ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:57.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.483+0000 7f953f400640 1 --2- 172.21.15.43:0/923007546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9548102360 0x7f954819a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:57.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.483+0000 7f9547e00640 1 -- 172.21.15.43:0/923007546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9548102360 msgr2=0x7f954819a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:57.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.483+0000 7f9547e00640 1 --2- 172.21.15.43:0/923007546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9548102360 0x7f954819a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:57.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.483+0000 7f9547e00640 1 -- 172.21.15.43:0/923007546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95481079c0 msgr2=0x7f954819ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:57.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.483+0000 7f9547e00640 1 --2- 172.21.15.43:0/923007546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95481079c0 0x7f954819ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:57.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.483+0000 7f9547e00640 1 -- 172.21.15.43:0/923007546 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f954819f3f0 con 0x7f95481019c0 2024-09-19T05:08:57.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.484+0000 7f9547e00640 1 --2- 172.21.15.43:0/923007546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95481019c0 0x7f954819a440 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9538033040 tx=0x7f9538003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:57.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.484+0000 7f9545600640 1 -- 172.21.15.43:0/923007546 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9538004040 con 0x7f95481019c0 2024-09-19T05:08:57.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.484+0000 7f9545600640 1 -- 172.21.15.43:0/923007546 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95380046c0 con 0x7f95481019c0 2024-09-19T05:08:57.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.484+0000 7f954d400640 1 -- 172.21.15.43:0/923007546 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f954819f680 con 0x7f95481019c0 2024-09-19T05:08:57.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.484+0000 7f9545600640 1 -- 172.21.15.43:0/923007546 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9538046480 con 0x7f95481019c0 2024-09-19T05:08:57.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.484+0000 7f954d400640 1 -- 172.21.15.43:0/923007546 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95481a6fc0 con 0x7f95481019c0 2024-09-19T05:08:57.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.487+0000 7f9545600640 1 -- 172.21.15.43:0/923007546 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f95380305a0 con 0x7f95481019c0 2024-09-19T05:08:57.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.487+0000 7f9551481740 1 Processor -- start 2024-09-19T05:08:57.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.487+0000 7f9551481740 1 -- start start 2024-09-19T05:08:57.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.488+0000 7f9551481740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578370e8e40 0x5578370e9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:57.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.488+0000 7f9551481740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578370e9760 0x5578370f3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:57.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.488+0000 7f9551481740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5578370f4020 0x5578370f6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:57.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.488+0000 7f9551481740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55783704f3e0 con 0x5578370e9760 2024-09-19T05:08:57.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.488+0000 7f9551481740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557836ff6d80 con 0x5578370f4020 2024-09-19T05:08:57.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.488+0000 7f9551481740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55783704b3a0 con 0x5578370e8e40 2024-09-19T05:08:57.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.488+0000 7f954ca00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578370e8e40 0x5578370e9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:57.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.488+0000 7f9547e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578370e9760 0x5578370f3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:57.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.488+0000 7f954ca00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578370e8e40 0x5578370e9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:38472/0 (socket says 172.21.15.43:38472) 2024-09-19T05:08:57.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.488+0000 7f954ca00640 1 -- 172.21.15.43:0/3359498576 learned_addr learned my addr 172.21.15.43:0/3359498576 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:57.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.488+0000 7f953f400640 1 --2- 172.21.15.43:0/3359498576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5578370f4020 0x5578370f6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:57.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.488+0000 7f954ca00640 1 -- 172.21.15.43:0/3359498576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5578370f4020 msgr2=0x5578370f6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:57.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.488+0000 7f954ca00640 1 --2- 172.21.15.43:0/3359498576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5578370f4020 0x5578370f6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:57.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.488+0000 7f954ca00640 1 -- 172.21.15.43:0/3359498576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578370e9760 msgr2=0x5578370f3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:57.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.488+0000 7f954ca00640 1 --2- 172.21.15.43:0/3359498576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578370e9760 0x5578370f3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:57.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.488+0000 7f954ca00640 1 -- 172.21.15.43:0/3359498576 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557837015ae0 con 0x5578370e8e40 2024-09-19T05:08:57.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.489+0000 7f954ca00640 1 --2- 172.21.15.43:0/3359498576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578370e8e40 0x5578370e9220 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9540009c20 tx=0x7f9540011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8410b0a0b771fac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:57.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.489+0000 7f9545600640 1 --2- 172.21.15.43:0/923007546 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f952c0a33b0 0x7f952c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:57.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.489+0000 7f953d200640 1 -- 172.21.15.43:0/3359498576 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954001e070 con 0x5578370e8e40 2024-09-19T05:08:57.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.489+0000 7f953d200640 1 -- 172.21.15.43:0/3359498576 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9540019b10 con 0x5578370e8e40 2024-09-19T05:08:57.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.489+0000 7f953d200640 1 -- 172.21.15.43:0/3359498576 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954000e9e0 con 0x5578370e8e40 2024-09-19T05:08:57.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.489+0000 7f9547e00640 1 --2- 172.21.15.43:0/923007546 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f952c0a33b0 0x7f952c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:57.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.489+0000 7f9545600640 1 -- 172.21.15.43:0/923007546 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f953802e030 con 0x7f95481019c0 2024-09-19T05:08:57.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.490+0000 7f9551481740 1 -- 172.21.15.43:0/3359498576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578370e8e40 msgr2=0x5578370e9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:57.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.490+0000 7f9547e00640 1 --2- 172.21.15.43:0/923007546 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f952c0a33b0 0x7f952c0a5870 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x5578370e6230 tx=0x7f95380f4b80 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:57.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.490+0000 7f9551481740 1 --2- 172.21.15.43:0/3359498576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578370e8e40 0x5578370e9220 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9540009c20 tx=0x7f9540011810 comp rx=0 tx=0).stop 2024-09-19T05:08:57.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.490+0000 7f9551481740 1 -- 172.21.15.43:0/3359498576 shutdown_connections 2024-09-19T05:08:57.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.490+0000 7f9551481740 1 --2- 172.21.15.43:0/3359498576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5578370f4020 0x5578370f6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:57.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.490+0000 7f9551481740 1 --2- 172.21.15.43:0/3359498576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578370e9760 0x5578370f3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:57.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.490+0000 7f9551481740 1 --2- 172.21.15.43:0/3359498576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578370e8e40 0x5578370e9220 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:57.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.490+0000 7f9551481740 1 -- 172.21.15.43:0/3359498576 >> 172.21.15.43:0/3359498576 conn(0x5578370e53d0 msgr2=0x5578370e57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:57.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.490+0000 7f9551481740 1 -- 172.21.15.43:0/3359498576 shutdown_connections 2024-09-19T05:08:57.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.490+0000 7f9551481740 1 -- 172.21.15.43:0/3359498576 wait complete. 2024-09-19T05:08:57.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.491+0000 7f9551481740 1 -- start start 2024-09-19T05:08:57.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.491+0000 7f9551481740 1 Processor -- start 2024-09-19T05:08:57.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.491+0000 7f9551481740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578370e8e40 0x5578370f0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:57.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.491+0000 7f9551481740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5578370e9760 0x5578370f1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:57.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.491+0000 7f9551481740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578370f4020 0x5578370f1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:57.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.491+0000 7f9551481740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5578370f6bf0 con 0x5578370e8e40 2024-09-19T05:08:57.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.491+0000 7f9551481740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55783700a0f0 con 0x5578370e9760 2024-09-19T05:08:57.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.491+0000 7f9551481740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5578370f6d70 con 0x5578370f4020 2024-09-19T05:08:57.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.491+0000 7f9551481740 10 client.0 _ll_register_callbacks cb 0x557836ff1bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:57.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.492+0000 7f954ca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578370e8e40 0x5578370f0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:57.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.492+0000 7f954ca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578370e8e40 0x5578370f0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48820/0 (socket says 172.21.15.43:48820) 2024-09-19T05:08:57.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.492+0000 7f954ca00640 1 -- 172.21.15.43:0/3642702982 learned_addr learned my addr 172.21.15.43:0/3642702982 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:57.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.492+0000 7f953f400640 1 --2- 172.21.15.43:0/3642702982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5578370e9760 0x5578370f1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:57.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.492+0000 7f954ca00640 1 --2- 172.21.15.43:0/3642702982 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578370f4020 0x5578370f1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:57.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.492+0000 7f953f400640 1 -- 172.21.15.43:0/3642702982 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578370f4020 msgr2=0x5578370f1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:57.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.492+0000 7f953f400640 1 --2- 172.21.15.43:0/3642702982 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578370f4020 0x5578370f1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:57.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.492+0000 7f953f400640 1 -- 172.21.15.43:0/3642702982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578370e8e40 msgr2=0x5578370f0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:57.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.492+0000 7f953f400640 1 --2- 172.21.15.43:0/3642702982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578370e8e40 0x5578370f0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:57.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.492+0000 7f953f400640 1 -- 172.21.15.43:0/3642702982 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557836dc03c0 con 0x5578370e9760 2024-09-19T05:08:57.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.492+0000 7f954ca00640 1 --2- 172.21.15.43:0/3642702982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578370e8e40 0x5578370f0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:08:57.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.492+0000 7f953f400640 1 --2- 172.21.15.43:0/3642702982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5578370e9760 0x5578370f1440 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f954810ec20 tx=0x7f953000bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb3a9087f6e722bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:57.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.492+0000 7f9535600640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:08:57.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.493+0000 7f9535600640 1 -- 172.21.15.43:0/3642702982 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9530004020 con 0x5578370e9760 2024-09-19T05:08:57.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.493+0000 7f9551481740 1 -- 172.21.15.43:0/3642702982 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9540019b10 con 0x5578370e9760 2024-09-19T05:08:57.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.493+0000 7f9535600640 1 -- 172.21.15.43:0/3642702982 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f95300041c0 con 0x5578370e9760 2024-09-19T05:08:57.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.493+0000 7f9535600640 1 -- 172.21.15.43:0/3642702982 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f95300043c0 con 0x5578370e9760 2024-09-19T05:08:57.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.493+0000 7f9551481740 10 client.26105 fetch_fsmap learned FSMap version 29 2024-09-19T05:08:57.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.493+0000 7f9551481740 1 -- 172.21.15.43:0/3642702982 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557837102cb0 con 0x5578370e9760 2024-09-19T05:08:57.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.493+0000 7f9535600640 1 -- 172.21.15.43:0/3642702982 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9540019b10 con 0x5578370e9760 2024-09-19T05:08:57.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.493+0000 7f9535600640 1 -- 172.21.15.43:0/3642702982 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9530010700 con 0x5578370e9760 2024-09-19T05:08:57.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.493+0000 7f9551481740 10 client.26105 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:08:57.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.493+0000 7f9551481740 10 client.26105 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:08:57.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.493+0000 7f9551481740 20 client.26105 populate_metadata read hostname 'smithi043' 2024-09-19T05:08:57.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.494+0000 7f9551481740 1 --2- 172.21.15.43:0/3642702982 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5578371041c0 0x557837106620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:57.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.494+0000 7f9551481740 4 client.26105 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:08:57.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.494+0000 7f9551481740 1 -- 172.21.15.43:0/3642702982 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55783705c950 con 0x5578371041c0 2024-09-19T05:08:57.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.494+0000 7f954ca00640 1 --2- 172.21.15.43:0/3642702982 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5578371041c0 0x557837106620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:57.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.494+0000 7f954ca00640 1 --2- 172.21.15.43:0/3642702982 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5578371041c0 0x557837106620 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40f45ce4d22d10c6 server_cookie=c97c1145bab6edc0 in_seq=0 out_seq=0 2024-09-19T05:08:57.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.494+0000 7f9535600640 10 client.26105 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:08:57.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.495+0000 7f9535600640 1 -- 172.21.15.43:0/3642702982 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55783705c950 con 0x5578371041c0 2024-09-19T05:08:57.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.495+0000 7f9535600640 10 client.26105 handle_command_reply: tid=0 2024-09-19T05:08:57.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.495+0000 7f9551481740 1 client.26105 shutdown 2024-09-19T05:08:57.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.495+0000 7f9551481740 1 -- 172.21.15.43:0/3642702982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5578370e9760 msgr2=0x5578370f1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:57.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.495+0000 7f9551481740 1 --2- 172.21.15.43:0/3642702982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5578370e9760 0x5578370f1440 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f954810ec20 tx=0x7f953000bf90 comp rx=0 tx=0).stop 2024-09-19T05:08:57.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.495+0000 7f9551481740 1 -- 172.21.15.43:0/3642702982 shutdown_connections 2024-09-19T05:08:57.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.495+0000 7f9551481740 1 --2- 172.21.15.43:0/3642702982 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5578371041c0 0x557837106620 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:57.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.495+0000 7f9551481740 1 --2- 172.21.15.43:0/3642702982 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578370f4020 0x5578370f1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:57.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.495+0000 7f9551481740 1 --2- 172.21.15.43:0/3642702982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5578370e9760 0x5578370f1440 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:57.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.495+0000 7f9551481740 1 --2- 172.21.15.43:0/3642702982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578370e8e40 0x5578370f0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:57.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.495+0000 7f9535600640 0 client.26105 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:08:57.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.495+0000 7f9551481740 1 -- 172.21.15.43:0/3642702982 >> 172.21.15.43:0/3642702982 conn(0x5578370e53d0 msgr2=0x5578370eb100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:57.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.495+0000 7f9551481740 1 -- 172.21.15.43:0/3642702982 shutdown_connections 2024-09-19T05:08:57.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.495+0000 7f9551481740 1 -- 172.21.15.43:0/3642702982 wait complete. 2024-09-19T05:08:57.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.496+0000 7f9551481740 20 client.26105 trim_cache size 0 max 16384 2024-09-19T05:08:57.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.509+0000 7f9551481740 1 Processor -- start 2024-09-19T05:08:57.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.509+0000 7f9551481740 1 -- start start 2024-09-19T05:08:57.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.510+0000 7f9551481740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578370e8e40 0x5578371065b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:57.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.510+0000 7f9551481740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5578370e9760 0x5578370ef7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:57.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.510+0000 7f9551481740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578370f4020 0x5578370efd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.510+0000 7f9551481740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f95481af7b0 con 0x5578370e8e40 2024-09-19T05:08:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.510+0000 7f9551481740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f95481b12f0 con 0x5578370e9760 2024-09-19T05:08:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.510+0000 7f9551481740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f95481afe80 con 0x5578370f4020 2024-09-19T05:08:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.510+0000 7f954ca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578370e8e40 0x5578371065b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.510+0000 7f954ca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578370e8e40 0x5578371065b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48828/0 (socket says 172.21.15.43:48828) 2024-09-19T05:08:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.510+0000 7f954ca00640 1 -- 172.21.15.43:0/1854798115 learned_addr learned my addr 172.21.15.43:0/1854798115 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.510+0000 7f953f400640 1 --2- 172.21.15.43:0/1854798115 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5578370e9760 0x5578370ef7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.510+0000 7f954ca00640 1 --2- 172.21.15.43:0/1854798115 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578370f4020 0x5578370efd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.510+0000 7f954ca00640 1 -- 172.21.15.43:0/1854798115 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578370f4020 msgr2=0x5578370efd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.510+0000 7f954ca00640 1 --2- 172.21.15.43:0/1854798115 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578370f4020 0x5578370efd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.510+0000 7f954ca00640 1 -- 172.21.15.43:0/1854798115 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5578370e9760 msgr2=0x5578370ef7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.510+0000 7f954ca00640 1 --2- 172.21.15.43:0/1854798115 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5578370e9760 0x5578370ef7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.510+0000 7f954ca00640 1 -- 172.21.15.43:0/1854798115 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578370ef450 con 0x5578370e8e40 2024-09-19T05:08:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.511+0000 7f954ca00640 1 --2- 172.21.15.43:0/1854798115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578370e8e40 0x5578371065b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f954000e5d0 tx=0x7f9540009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47a4cbde09f6d842 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:57.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.511+0000 7f9537e00640 1 -- 172.21.15.43:0/1854798115 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9540034070 con 0x5578370e8e40 2024-09-19T05:08:57.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.511+0000 7f9537e00640 1 -- 172.21.15.43:0/1854798115 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f954000e3b0 con 0x5578370e8e40 2024-09-19T05:08:57.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.511+0000 7f9537e00640 1 -- 172.21.15.43:0/1854798115 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954001e040 con 0x5578370e8e40 2024-09-19T05:08:57.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.512+0000 7f9551481740 1 -- 172.21.15.43:0/1854798115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578370e8e40 msgr2=0x5578371065b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:57.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.512+0000 7f9551481740 1 --2- 172.21.15.43:0/1854798115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578370e8e40 0x5578371065b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f954000e5d0 tx=0x7f9540009910 comp rx=0 tx=0).stop 2024-09-19T05:08:57.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.512+0000 7f9551481740 1 -- 172.21.15.43:0/1854798115 shutdown_connections 2024-09-19T05:08:57.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.512+0000 7f9551481740 1 --2- 172.21.15.43:0/1854798115 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578370f4020 0x5578370efd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:57.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.512+0000 7f9551481740 1 --2- 172.21.15.43:0/1854798115 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5578370e9760 0x5578370ef7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:57.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.512+0000 7f9551481740 1 --2- 172.21.15.43:0/1854798115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578370e8e40 0x5578371065b0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:57.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.512+0000 7f9551481740 1 -- 172.21.15.43:0/1854798115 >> 172.21.15.43:0/1854798115 conn(0x5578370e53d0 msgr2=0x5578370e57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:57.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.512+0000 7f9551481740 1 -- 172.21.15.43:0/1854798115 shutdown_connections 2024-09-19T05:08:57.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.512+0000 7f9551481740 1 -- 172.21.15.43:0/1854798115 wait complete. 2024-09-19T05:08:57.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.512+0000 7f9551481740 1 -- start start 2024-09-19T05:08:57.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.512+0000 7f9551481740 1 Processor -- start 2024-09-19T05:08:57.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.513+0000 7f9551481740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578370e8e40 0x5578370f7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:57.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.513+0000 7f9551481740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578370e9760 0x5578370f7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:57.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.513+0000 7f9551481740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5578370f4020 0x55783716cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:57.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.513+0000 7f9551481740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f95481b0c20 con 0x5578370e8e40 2024-09-19T05:08:57.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.513+0000 7f9551481740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f95481b0550 con 0x5578370f4020 2024-09-19T05:08:57.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.513+0000 7f9551481740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f95481ad790 con 0x5578370e9760 2024-09-19T05:08:57.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.513+0000 7f9551481740 10 client.0 _ll_register_callbacks cb 0x7f953000bb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:08:57.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.513+0000 7f953f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578370e9760 0x5578370f7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:57.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.513+0000 7f954ca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578370e8e40 0x5578370f7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:57.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.513+0000 7f953f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578370e9760 0x5578370f7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:38478/0 (socket says 172.21.15.43:38478) 2024-09-19T05:08:57.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.513+0000 7f953f400640 1 -- 172.21.15.43:0/3297912050 learned_addr learned my addr 172.21.15.43:0/3297912050 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:08:57.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.513+0000 7f954ca00640 1 --2- 172.21.15.43:0/3297912050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5578370f4020 0x55783716cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:57.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.514+0000 7f953f400640 1 -- 172.21.15.43:0/3297912050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5578370f4020 msgr2=0x55783716cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:57.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.514+0000 7f953f400640 1 --2- 172.21.15.43:0/3297912050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5578370f4020 0x55783716cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:57.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.514+0000 7f953f400640 1 -- 172.21.15.43:0/3297912050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578370e8e40 msgr2=0x5578370f7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:08:57.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.514+0000 7f953f400640 1 --2- 172.21.15.43:0/3297912050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578370e8e40 0x5578370f7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:57.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.514+0000 7f953f400640 1 -- 172.21.15.43:0/3297912050 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55783716d430 con 0x5578370e9760 2024-09-19T05:08:57.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.514+0000 7f954ca00640 1 --2- 172.21.15.43:0/3297912050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5578370f4020 0x55783716cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:08:57.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.514+0000 7f953f400640 1 --2- 172.21.15.43:0/3297912050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578370e9760 0x5578370f7ab0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f95300058c0 tx=0x7f953000b9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eff890aa8fe4d912 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:08:57.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.514+0000 7f9535600640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:08:57.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.514+0000 7f9535600640 1 -- 172.21.15.43:0/3297912050 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9530005250 con 0x5578370e9760 2024-09-19T05:08:57.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.514+0000 7f9535600640 1 -- 172.21.15.43:0/3297912050 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f95300053f0 con 0x5578370e9760 2024-09-19T05:08:57.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.514+0000 7f9551481740 1 -- 172.21.15.43:0/3297912050 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f954000e3b0 con 0x5578370e9760 2024-09-19T05:08:57.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.515+0000 7f9535600640 1 -- 172.21.15.43:0/3297912050 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9530039050 con 0x5578370e9760 2024-09-19T05:08:57.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.515+0000 7f9551481740 10 client.26053 fetch_fsmap learned FSMap version 29 2024-09-19T05:08:57.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.515+0000 7f9551481740 1 -- 172.21.15.43:0/3297912050 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55783716e730 con 0x5578370e9760 2024-09-19T05:08:57.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.515+0000 7f9535600640 1 -- 172.21.15.43:0/3297912050 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f954000e3b0 con 0x5578370e9760 2024-09-19T05:08:57.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.515+0000 7f9535600640 1 -- 172.21.15.43:0/3297912050 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9530034030 con 0x5578370e9760 2024-09-19T05:08:57.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.515+0000 7f9551481740 10 client.26053 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:08:57.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.515+0000 7f9551481740 10 client.26053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:08:57.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.515+0000 7f9551481740 20 client.26053 populate_metadata read hostname 'smithi043' 2024-09-19T05:08:57.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.515+0000 7f9551481740 1 --2- 172.21.15.43:0/3297912050 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55783716ef00 0x557837171320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:08:57.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.515+0000 7f9551481740 4 client.26053 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:08:57.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.515+0000 7f9551481740 1 -- 172.21.15.43:0/3297912050 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557837056e60 con 0x55783716ef00 2024-09-19T05:08:57.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.516+0000 7f954ca00640 1 --2- 172.21.15.43:0/3297912050 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55783716ef00 0x557837171320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:08:57.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.516+0000 7f954ca00640 1 --2- 172.21.15.43:0/3297912050 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55783716ef00 0x557837171320 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0b013a22cf45541 server_cookie=a486294035e86fb in_seq=0 out_seq=0 2024-09-19T05:08:57.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.516+0000 7f9535600640 10 client.26053 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:08:57.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.516+0000 7f9535600640 1 -- 172.21.15.43:0/3297912050 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557837056e60 con 0x55783716ef00 2024-09-19T05:08:57.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.516+0000 7f9535600640 10 client.26053 handle_command_reply: tid=0 2024-09-19T05:08:57.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.516+0000 7f9551481740 1 client.26053 shutdown 2024-09-19T05:08:57.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.516+0000 7f9551481740 1 -- 172.21.15.43:0/3297912050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578370e9760 msgr2=0x5578370f7ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:57.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.516+0000 7f9551481740 1 --2- 172.21.15.43:0/3297912050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578370e9760 0x5578370f7ab0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f95300058c0 tx=0x7f953000b9d0 comp rx=0 tx=0).stop 2024-09-19T05:08:57.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.516+0000 7f9551481740 1 -- 172.21.15.43:0/3297912050 shutdown_connections 2024-09-19T05:08:57.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.517+0000 7f9551481740 1 --2- 172.21.15.43:0/3297912050 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55783716ef00 0x557837171320 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:57.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.517+0000 7f9551481740 1 --2- 172.21.15.43:0/3297912050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5578370f4020 0x55783716cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:57.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.517+0000 7f9551481740 1 --2- 172.21.15.43:0/3297912050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578370e9760 0x5578370f7ab0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:57.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.517+0000 7f9535600640 0 client.26053 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:08:57.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.517+0000 7f9551481740 1 --2- 172.21.15.43:0/3297912050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578370e8e40 0x5578370f7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:57.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.517+0000 7f9551481740 1 -- 172.21.15.43:0/3297912050 >> 172.21.15.43:0/3297912050 conn(0x5578370e53d0 msgr2=0x5578370ef710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:57.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.517+0000 7f9551481740 1 -- 172.21.15.43:0/3297912050 shutdown_connections 2024-09-19T05:08:57.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.517+0000 7f9551481740 1 -- 172.21.15.43:0/3297912050 wait complete. 2024-09-19T05:08:57.540 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:08:57.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.517+0000 7f9551481740 20 client.26053 trim_cache size 0 max 16384 2024-09-19T05:08:57.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.517+0000 7f954d400640 1 -- 172.21.15.43:0/923007546 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f952c0a33b0 msgr2=0x7f952c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:57.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.517+0000 7f954d400640 1 --2- 172.21.15.43:0/923007546 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f952c0a33b0 0x7f952c0a5870 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x5578370e6230 tx=0x7f95380f4b80 comp rx=0 tx=0).stop 2024-09-19T05:08:57.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.517+0000 7f954d400640 1 -- 172.21.15.43:0/923007546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95481019c0 msgr2=0x7f954819a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:08:57.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.517+0000 7f954d400640 1 --2- 172.21.15.43:0/923007546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95481019c0 0x7f954819a440 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9538033040 tx=0x7f9538003970 comp rx=0 tx=0).stop 2024-09-19T05:08:57.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.518+0000 7f954d400640 1 -- 172.21.15.43:0/923007546 shutdown_connections 2024-09-19T05:08:57.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.518+0000 7f954d400640 1 --2- 172.21.15.43:0/923007546 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f952c0a33b0 0x7f952c0a5870 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:57.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.518+0000 7f954d400640 1 --2- 172.21.15.43:0/923007546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95481079c0 0x7f954819ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:57.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.518+0000 7f954d400640 1 --2- 172.21.15.43:0/923007546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9548102360 0x7f954819a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:57.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.518+0000 7f954d400640 1 --2- 172.21.15.43:0/923007546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95481019c0 0x7f954819a440 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:08:57.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.518+0000 7f954d400640 1 -- 172.21.15.43:0/923007546 >> 172.21.15.43:0/923007546 conn(0x7f95480fd740 msgr2=0x7f954810d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:08:57.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.518+0000 7f954d400640 1 -- 172.21.15.43:0/923007546 shutdown_connections 2024-09-19T05:08:57.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:08:57.518+0000 7f954d400640 1 -- 172.21.15.43:0/923007546 wait complete. 2024-09-19T05:08:57.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:08:57 smithi043 ceph-mon[30800]: pgmap v132: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:08:57.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:08:57 smithi151 ceph-mon[35425]: pgmap v132: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:08:57.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:08:57 smithi096 ceph-mon[35085]: pgmap v132: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:08:59.414 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T05:08:59.414 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T05:08:59.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:08:59 smithi043 ceph-mon[30800]: pgmap v133: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:08:59.810 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:08:59.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:08:59 smithi151 ceph-mon[35425]: pgmap v133: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:08:59.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:08:59 smithi096 ceph-mon[35085]: pgmap v133: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:09:00.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.445+0000 7f73f4800640 1 -- 172.21.15.43:0/986251181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73ec109120 msgr2=0x7f73ec110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:00.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.445+0000 7f73f4800640 1 --2- 172.21.15.43:0/986251181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73ec109120 0x7f73ec110ca0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f73e8009ca0 tx=0x7f73e802a8c0 comp rx=0 tx=0).stop 2024-09-19T05:09:00.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.445+0000 7f73f4800640 1 -- 172.21.15.43:0/986251181 shutdown_connections 2024-09-19T05:09:00.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.445+0000 7f73f4800640 1 --2- 172.21.15.43:0/986251181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73ec109120 0x7f73ec110ca0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:00.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.445+0000 7f73f4800640 1 --2- 172.21.15.43:0/986251181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73ec1049a0 0x7f73ec1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:00.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.445+0000 7f73f4800640 1 --2- 172.21.15.43:0/986251181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73ec103ff0 0x7f73ec1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:00.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.445+0000 7f73f4800640 1 -- 172.21.15.43:0/986251181 >> 172.21.15.43:0/986251181 conn(0x7f73ec0ffa90 msgr2=0x7f73ec101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:00.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.445+0000 7f73f4800640 1 -- 172.21.15.43:0/986251181 shutdown_connections 2024-09-19T05:09:00.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.445+0000 7f73f4800640 1 -- 172.21.15.43:0/986251181 wait complete. 2024-09-19T05:09:00.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.446+0000 7f73f4800640 1 Processor -- start 2024-09-19T05:09:00.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.447+0000 7f73f4800640 1 -- start start 2024-09-19T05:09:00.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.447+0000 7f73f4800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73ec103ff0 0x7f73ec19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:00.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.447+0000 7f73f4800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73ec1049a0 0x7f73ec19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:00.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.447+0000 7f73f4800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73ec109120 0x7f73ec19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:00.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.447+0000 7f73f4800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec1141a0 con 0x7f73ec1049a0 2024-09-19T05:09:00.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.447+0000 7f73f4800640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec114020 con 0x7f73ec103ff0 2024-09-19T05:09:00.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.447+0000 7f73f4800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec114320 con 0x7f73ec109120 2024-09-19T05:09:00.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.447+0000 7f73f1e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73ec103ff0 0x7f73ec19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:00.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.447+0000 7f73f1400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73ec1049a0 0x7f73ec19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:00.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.447+0000 7f73f2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73ec109120 0x7f73ec19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:00.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.447+0000 7f73f1e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73ec103ff0 0x7f73ec19a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:38356/0 (socket says 172.21.15.43:38356) 2024-09-19T05:09:00.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.447+0000 7f73f1e00640 1 -- 172.21.15.43:0/364825672 learned_addr learned my addr 172.21.15.43:0/364825672 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:00.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.448+0000 7f73f1400640 1 -- 172.21.15.43:0/364825672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73ec109120 msgr2=0x7f73ec19eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:00.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.448+0000 7f73f1400640 1 --2- 172.21.15.43:0/364825672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73ec109120 0x7f73ec19eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:00.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.448+0000 7f73f1400640 1 -- 172.21.15.43:0/364825672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73ec103ff0 msgr2=0x7f73ec19a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:00.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.448+0000 7f73f1400640 1 --2- 172.21.15.43:0/364825672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73ec103ff0 0x7f73ec19a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:00.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.448+0000 7f73f1400640 1 -- 172.21.15.43:0/364825672 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73ec19f5b0 con 0x7f73ec1049a0 2024-09-19T05:09:00.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.448+0000 7f73f1e00640 1 --2- 172.21.15.43:0/364825672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73ec103ff0 0x7f73ec19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:00.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.448+0000 7f73f2800640 1 --2- 172.21.15.43:0/364825672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73ec109120 0x7f73ec19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:00.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.448+0000 7f73f1400640 1 --2- 172.21.15.43:0/364825672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73ec1049a0 0x7f73ec19ab40 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f73dc00b3f0 tx=0x7f73dc00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:00.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.449+0000 7f73e6a00640 1 -- 172.21.15.43:0/364825672 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc015020 con 0x7f73ec1049a0 2024-09-19T05:09:00.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.449+0000 7f73e6a00640 1 -- 172.21.15.43:0/364825672 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73dc011d10 con 0x7f73ec1049a0 2024-09-19T05:09:00.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.449+0000 7f73f4800640 1 -- 172.21.15.43:0/364825672 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73ec19f8a0 con 0x7f73ec1049a0 2024-09-19T05:09:00.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.449+0000 7f73e6a00640 1 -- 172.21.15.43:0/364825672 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc00f2e0 con 0x7f73ec1049a0 2024-09-19T05:09:00.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.449+0000 7f73f4800640 1 -- 172.21.15.43:0/364825672 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73ec1a7180 con 0x7f73ec1049a0 2024-09-19T05:09:00.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.452+0000 7f73e6a00640 1 -- 172.21.15.43:0/364825672 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f73dc018020 con 0x7f73ec1049a0 2024-09-19T05:09:00.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.452+0000 7f73f71d5740 1 Processor -- start 2024-09-19T05:09:00.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.452+0000 7f73f71d5740 1 -- start start 2024-09-19T05:09:00.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.453+0000 7f73f71d5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f03b76ac0 0x562f03b76ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:00.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.453+0000 7f73f71d5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f03b773e0 0x562f03b81760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:00.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.453+0000 7f73f71d5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562f03b81ca0 0x562f03b84090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:00.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.453+0000 7f73f71d5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562f03add290 con 0x562f03b773e0 2024-09-19T05:09:00.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.453+0000 7f73f71d5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562f03a84b40 con 0x562f03b76ac0 2024-09-19T05:09:00.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.453+0000 7f73f71d5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562f03ad9250 con 0x562f03b81ca0 2024-09-19T05:09:00.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.453+0000 7f73f1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f03b773e0 0x562f03b81760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:00.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.453+0000 7f73f1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f03b773e0 0x562f03b81760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48860/0 (socket says 172.21.15.43:48860) 2024-09-19T05:09:00.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.453+0000 7f73f2800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f03b76ac0 0x562f03b76ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:00.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.453+0000 7f73f1e00640 1 -- 172.21.15.43:0/2637213997 learned_addr learned my addr 172.21.15.43:0/2637213997 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:00.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.453+0000 7f73f1400640 1 --2- 172.21.15.43:0/2637213997 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562f03b81ca0 0x562f03b84090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:00.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.454+0000 7f73f2800640 1 -- 172.21.15.43:0/2637213997 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562f03b81ca0 msgr2=0x562f03b84090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:00.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.454+0000 7f73f2800640 1 --2- 172.21.15.43:0/2637213997 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562f03b81ca0 0x562f03b84090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:00.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.454+0000 7f73f2800640 1 -- 172.21.15.43:0/2637213997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f03b773e0 msgr2=0x562f03b81760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:00.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.454+0000 7f73f2800640 1 --2- 172.21.15.43:0/2637213997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f03b773e0 0x562f03b81760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:00.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.454+0000 7f73f2800640 1 -- 172.21.15.43:0/2637213997 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f0384e6a0 con 0x562f03b76ac0 2024-09-19T05:09:00.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.454+0000 7f73f1400640 1 --2- 172.21.15.43:0/2637213997 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562f03b81ca0 0x562f03b84090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:09:00.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.454+0000 7f73f2800640 1 --2- 172.21.15.43:0/2637213997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f03b76ac0 0x562f03b76ea0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f73e8006010 tx=0x7f73e80098b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4d98d721a46ffa6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:00.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.454+0000 7f73e6a00640 1 --2- 172.21.15.43:0/364825672 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f73d40a3430 0x7f73d40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:00.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.454+0000 7f73e6a00640 1 -- 172.21.15.43:0/364825672 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f73dc0cf320 con 0x7f73ec1049a0 2024-09-19T05:09:00.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.454+0000 7f73f1e00640 1 --2- 172.21.15.43:0/364825672 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f73d40a3430 0x7f73d40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:00.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.454+0000 7f73db000640 1 -- 172.21.15.43:0/2637213997 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e8003bf0 con 0x562f03b76ac0 2024-09-19T05:09:00.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.454+0000 7f73db000640 1 -- 172.21.15.43:0/2637213997 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73e8036040 con 0x562f03b76ac0 2024-09-19T05:09:00.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.455+0000 7f73db000640 1 -- 172.21.15.43:0/2637213997 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e80046f0 con 0x562f03b76ac0 2024-09-19T05:09:00.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.455+0000 7f73f71d5740 1 -- 172.21.15.43:0/2637213997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f03b76ac0 msgr2=0x562f03b76ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:00.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.455+0000 7f73f1e00640 1 --2- 172.21.15.43:0/364825672 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f73d40a3430 0x7f73d40a58f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x562f03b73eb0 tx=0x7f73e0005f90 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:00.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.455+0000 7f73f71d5740 1 --2- 172.21.15.43:0/2637213997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f03b76ac0 0x562f03b76ea0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f73e8006010 tx=0x7f73e80098b0 comp rx=0 tx=0).stop 2024-09-19T05:09:00.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.456+0000 7f73f71d5740 1 -- 172.21.15.43:0/2637213997 shutdown_connections 2024-09-19T05:09:00.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.456+0000 7f73f71d5740 1 --2- 172.21.15.43:0/2637213997 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562f03b81ca0 0x562f03b84090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:00.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.456+0000 7f73f71d5740 1 --2- 172.21.15.43:0/2637213997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f03b773e0 0x562f03b81760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:00.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.456+0000 7f73f71d5740 1 --2- 172.21.15.43:0/2637213997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f03b76ac0 0x562f03b76ea0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:00.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.456+0000 7f73f71d5740 1 -- 172.21.15.43:0/2637213997 >> 172.21.15.43:0/2637213997 conn(0x562f03b73050 msgr2=0x562f03b73430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:00.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.456+0000 7f73f71d5740 1 -- 172.21.15.43:0/2637213997 shutdown_connections 2024-09-19T05:09:00.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.456+0000 7f73f71d5740 1 -- 172.21.15.43:0/2637213997 wait complete. 2024-09-19T05:09:00.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.456+0000 7f73f71d5740 1 -- start start 2024-09-19T05:09:00.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.456+0000 7f73f71d5740 1 Processor -- start 2024-09-19T05:09:00.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.457+0000 7f73f71d5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f03b76ac0 0x562f03b81620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:00.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.457+0000 7f73f71d5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f03b773e0 0x562f03b7e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:00.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.457+0000 7f73f71d5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562f03b81ca0 0x562f03b7ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:00.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.457+0000 7f73f71d5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562f03b84790 con 0x562f03b773e0 2024-09-19T05:09:00.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.457+0000 7f73f71d5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562f03a97fa0 con 0x562f03b76ac0 2024-09-19T05:09:00.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.457+0000 7f73f71d5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562f03b84910 con 0x562f03b81ca0 2024-09-19T05:09:00.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.457+0000 7f73f71d5740 10 client.0 _ll_register_callbacks cb 0x562f037ebba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:00.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.457+0000 7f73f2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562f03b81ca0 0x562f03b7ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:00.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.457+0000 7f73f2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562f03b81ca0 0x562f03b7ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:38498/0 (socket says 172.21.15.43:38498) 2024-09-19T05:09:00.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.457+0000 7f73f1e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f03b76ac0 0x562f03b81620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:00.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.457+0000 7f73f2800640 1 -- 172.21.15.43:0/3460023805 learned_addr learned my addr 172.21.15.43:0/3460023805 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:00.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.457+0000 7f73f1400640 1 --2- 172.21.15.43:0/3460023805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f03b773e0 0x562f03b7e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:00.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.458+0000 7f73f2800640 1 -- 172.21.15.43:0/3460023805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f03b76ac0 msgr2=0x562f03b81620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:00.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.458+0000 7f73f2800640 1 --2- 172.21.15.43:0/3460023805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f03b76ac0 0x562f03b81620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:00.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.458+0000 7f73f2800640 1 -- 172.21.15.43:0/3460023805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f03b773e0 msgr2=0x562f03b7e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:00.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.458+0000 7f73f2800640 1 --2- 172.21.15.43:0/3460023805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f03b773e0 0x562f03b7e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:00.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.458+0000 7f73f2800640 1 -- 172.21.15.43:0/3460023805 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f03b7f680 con 0x562f03b81ca0 2024-09-19T05:09:00.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.458+0000 7f73f2800640 1 --2- 172.21.15.43:0/3460023805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562f03b81ca0 0x562f03b7ee50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f73e8031730 tx=0x7f73e8009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecca466a74849f4f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:00.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.458+0000 7f73cf400640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:09:00.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.458+0000 7f73cf400640 1 -- 172.21.15.43:0/3460023805 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e80039f0 con 0x562f03b81ca0 2024-09-19T05:09:00.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.458+0000 7f73f71d5740 1 -- 172.21.15.43:0/3460023805 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73e8036040 con 0x562f03b81ca0 2024-09-19T05:09:00.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.458+0000 7f73cf400640 1 -- 172.21.15.43:0/3460023805 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f73e8003b90 con 0x562f03b81ca0 2024-09-19T05:09:00.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.459+0000 7f73cf400640 1 -- 172.21.15.43:0/3460023805 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f73e805c050 con 0x562f03b81ca0 2024-09-19T05:09:00.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.459+0000 7f73f71d5740 10 client.26062 fetch_fsmap learned FSMap version 29 2024-09-19T05:09:00.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.459+0000 7f73f71d5740 1 -- 172.21.15.43:0/3460023805 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562f03b84790 con 0x562f03b81ca0 2024-09-19T05:09:00.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.459+0000 7f73cf400640 1 -- 172.21.15.43:0/3460023805 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73e8036040 con 0x562f03b81ca0 2024-09-19T05:09:00.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.460+0000 7f73cf400640 1 -- 172.21.15.43:0/3460023805 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f73e8057030 con 0x562f03b81ca0 2024-09-19T05:09:00.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.460+0000 7f73f71d5740 10 client.26062 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:09:00.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.460+0000 7f73f71d5740 10 client.26062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:09:00.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.460+0000 7f73f71d5740 20 client.26062 populate_metadata read hostname 'smithi043' 2024-09-19T05:09:00.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.460+0000 7f73f71d5740 1 --2- 172.21.15.43:0/3460023805 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x562f03b91860 0x562f03b93c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:00.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.460+0000 7f73f71d5740 4 client.26062 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:09:00.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.460+0000 7f73f71d5740 1 -- 172.21.15.43:0/3460023805 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f03aea800 con 0x562f03b91860 2024-09-19T05:09:00.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.460+0000 7f73f1e00640 1 --2- 172.21.15.43:0/3460023805 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x562f03b91860 0x562f03b93c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:00.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.461+0000 7f73f1e00640 1 --2- 172.21.15.43:0/3460023805 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x562f03b91860 0x562f03b93c50 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=748866c17c047836 server_cookie=983e12f3c5a6fff in_seq=0 out_seq=0 2024-09-19T05:09:00.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.461+0000 7f73cf400640 10 client.26062 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:09:00.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.462+0000 7f73cf400640 1 -- 172.21.15.43:0/3460023805 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562f03aea800 con 0x562f03b91860 2024-09-19T05:09:00.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.462+0000 7f73cf400640 10 client.26062 handle_command_reply: tid=0 2024-09-19T05:09:00.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.462+0000 7f73f71d5740 1 client.26062 shutdown 2024-09-19T05:09:00.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.462+0000 7f73f71d5740 1 -- 172.21.15.43:0/3460023805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562f03b81ca0 msgr2=0x562f03b7ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:00.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.462+0000 7f73f71d5740 1 --2- 172.21.15.43:0/3460023805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562f03b81ca0 0x562f03b7ee50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f73e8031730 tx=0x7f73e8009a20 comp rx=0 tx=0).stop 2024-09-19T05:09:00.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.462+0000 7f73f71d5740 1 -- 172.21.15.43:0/3460023805 shutdown_connections 2024-09-19T05:09:00.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.462+0000 7f73f71d5740 1 --2- 172.21.15.43:0/3460023805 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x562f03b91860 0x562f03b93c50 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:00.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.462+0000 7f73f71d5740 1 --2- 172.21.15.43:0/3460023805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562f03b81ca0 0x562f03b7ee50 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:00.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.462+0000 7f73f71d5740 1 --2- 172.21.15.43:0/3460023805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f03b773e0 0x562f03b7e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:00.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.462+0000 7f73f71d5740 1 --2- 172.21.15.43:0/3460023805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f03b76ac0 0x562f03b81620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:00.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.462+0000 7f73f71d5740 1 -- 172.21.15.43:0/3460023805 >> 172.21.15.43:0/3460023805 conn(0x562f03b73050 msgr2=0x562f03b78d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:00.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.462+0000 7f73cf400640 0 client.26062 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:09:00.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.462+0000 7f73f71d5740 1 -- 172.21.15.43:0/3460023805 shutdown_connections 2024-09-19T05:09:00.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.462+0000 7f73f71d5740 1 -- 172.21.15.43:0/3460023805 wait complete. 2024-09-19T05:09:00.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.463+0000 7f73f71d5740 20 client.26062 trim_cache size 0 max 16384 2024-09-19T05:09:00.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.481+0000 7f73f71d5740 1 Processor -- start 2024-09-19T05:09:00.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.481+0000 7f73f71d5740 1 -- start start 2024-09-19T05:09:00.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.481+0000 7f73f71d5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f03b73050 0x562f03b7d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:00.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.481+0000 7f73f71d5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f03b76ac0 0x562f03b994d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:00.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.481+0000 7f73f71d5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562f03b773e0 0x562f03b99a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:00.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.481+0000 7f73f71d5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec1af980 con 0x562f03b76ac0 2024-09-19T05:09:00.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.481+0000 7f73f71d5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec1b14c0 con 0x562f03b73050 2024-09-19T05:09:00.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.481+0000 7f73f71d5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec1b0050 con 0x562f03b773e0 2024-09-19T05:09:00.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.482+0000 7f73f1400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f03b76ac0 0x562f03b994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:00.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.482+0000 7f73f2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562f03b773e0 0x562f03b99a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:00.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.482+0000 7f73f1400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f03b76ac0 0x562f03b994d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48874/0 (socket says 172.21.15.43:48874) 2024-09-19T05:09:00.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.482+0000 7f73f1400640 1 -- 172.21.15.43:0/3056849779 learned_addr learned my addr 172.21.15.43:0/3056849779 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:00.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.482+0000 7f73f1e00640 1 --2- 172.21.15.43:0/3056849779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f03b73050 0x562f03b7d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:00.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.482+0000 7f73f2800640 1 -- 172.21.15.43:0/3056849779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f03b73050 msgr2=0x562f03b7d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:00.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.482+0000 7f73f2800640 1 --2- 172.21.15.43:0/3056849779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f03b73050 0x562f03b7d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:00.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.482+0000 7f73f2800640 1 -- 172.21.15.43:0/3056849779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f03b76ac0 msgr2=0x562f03b994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:00.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.482+0000 7f73f2800640 1 --2- 172.21.15.43:0/3056849779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f03b76ac0 0x562f03b994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:00.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.482+0000 7f73f2800640 1 -- 172.21.15.43:0/3056849779 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f03b77210 con 0x562f03b773e0 2024-09-19T05:09:00.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.482+0000 7f73f1e00640 1 --2- 172.21.15.43:0/3056849779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f03b73050 0x562f03b7d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:00.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.482+0000 7f73f1400640 1 --2- 172.21.15.43:0/3056849779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f03b76ac0 0x562f03b994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:00.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.482+0000 7f73f2800640 1 --2- 172.21.15.43:0/3056849779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562f03b773e0 0x562f03b99a10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f73e802adc0 tx=0x7f73e8004420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=101b6ed231a3d0e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:00.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.483+0000 7f73da600640 1 -- 172.21.15.43:0/3056849779 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e8009b20 con 0x562f03b773e0 2024-09-19T05:09:00.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.483+0000 7f73da600640 1 -- 172.21.15.43:0/3056849779 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73e8036040 con 0x562f03b773e0 2024-09-19T05:09:00.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.483+0000 7f73da600640 1 -- 172.21.15.43:0/3056849779 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e8004e00 con 0x562f03b773e0 2024-09-19T05:09:00.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.484+0000 7f73f71d5740 1 -- 172.21.15.43:0/3056849779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562f03b773e0 msgr2=0x562f03b99a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:00.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.484+0000 7f73f71d5740 1 --2- 172.21.15.43:0/3056849779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562f03b773e0 0x562f03b99a10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f73e802adc0 tx=0x7f73e8004420 comp rx=0 tx=0).stop 2024-09-19T05:09:00.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.484+0000 7f73f71d5740 1 -- 172.21.15.43:0/3056849779 shutdown_connections 2024-09-19T05:09:00.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.484+0000 7f73f71d5740 1 --2- 172.21.15.43:0/3056849779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562f03b773e0 0x562f03b99a10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:00.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.484+0000 7f73f71d5740 1 --2- 172.21.15.43:0/3056849779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f03b76ac0 0x562f03b994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:00.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.484+0000 7f73f71d5740 1 --2- 172.21.15.43:0/3056849779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f03b73050 0x562f03b7d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:00.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.484+0000 7f73f71d5740 1 -- 172.21.15.43:0/3056849779 >> 172.21.15.43:0/3056849779 conn(0x562f03b73ad0 msgr2=0x562f03b7d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:00.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.484+0000 7f73f71d5740 1 -- 172.21.15.43:0/3056849779 shutdown_connections 2024-09-19T05:09:00.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.484+0000 7f73f71d5740 1 -- 172.21.15.43:0/3056849779 wait complete. 2024-09-19T05:09:00.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.484+0000 7f73f71d5740 1 -- start start 2024-09-19T05:09:00.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.484+0000 7f73f71d5740 1 Processor -- start 2024-09-19T05:09:00.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.485+0000 7f73f71d5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f03b73050 0x562f03b8a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:00.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.485+0000 7f73f71d5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562f03b76ac0 0x562f03b7b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:00.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.485+0000 7f73f71d5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f03b773e0 0x562f03b7bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:00.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.485+0000 7f73f71d5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec1b0df0 con 0x562f03b73050 2024-09-19T05:09:00.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.485+0000 7f73f71d5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec1b0720 con 0x562f03b773e0 2024-09-19T05:09:00.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.485+0000 7f73f71d5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec1ad960 con 0x562f03b76ac0 2024-09-19T05:09:00.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.485+0000 7f73f71d5740 10 client.0 _ll_register_callbacks cb 0x7f73e0009550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:00.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.485+0000 7f73f1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f03b73050 0x562f03b8a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:00.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.485+0000 7f73f1400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562f03b76ac0 0x562f03b7b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:00.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.485+0000 7f73f1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f03b73050 0x562f03b8a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48882/0 (socket says 172.21.15.43:48882) 2024-09-19T05:09:00.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.485+0000 7f73f1400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562f03b76ac0 0x562f03b7b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:38514/0 (socket says 172.21.15.43:38514) 2024-09-19T05:09:00.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.485+0000 7f73f1e00640 1 -- 172.21.15.43:0/3927814776 learned_addr learned my addr 172.21.15.43:0/3927814776 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:00.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.485+0000 7f73f2800640 1 --2- 172.21.15.43:0/3927814776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f03b773e0 0x562f03b7bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:00.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.486+0000 7f73f1400640 1 -- 172.21.15.43:0/3927814776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f03b773e0 msgr2=0x562f03b7bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:00.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.486+0000 7f73f1400640 1 --2- 172.21.15.43:0/3927814776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f03b773e0 0x562f03b7bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:00.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.486+0000 7f73f1400640 1 -- 172.21.15.43:0/3927814776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f03b73050 msgr2=0x562f03b8a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:00.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.486+0000 7f73f1400640 1 --2- 172.21.15.43:0/3927814776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f03b73050 0x562f03b8a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:00.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.486+0000 7f73f1400640 1 -- 172.21.15.43:0/3927814776 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f03b7c550 con 0x562f03b76ac0 2024-09-19T05:09:00.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.486+0000 7f73f1400640 1 --2- 172.21.15.43:0/3927814776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562f03b76ac0 0x562f03b7b780 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f73dc09b420 tx=0x7f73dc09b140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35e3450be007d40b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:00.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.486+0000 7f73cf400640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:09:00.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.486+0000 7f73cf400640 1 -- 172.21.15.43:0/3927814776 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc09fad0 con 0x562f03b76ac0 2024-09-19T05:09:00.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.486+0000 7f73cf400640 1 -- 172.21.15.43:0/3927814776 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f73dc0d2d40 con 0x562f03b76ac0 2024-09-19T05:09:00.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.486+0000 7f73f71d5740 1 -- 172.21.15.43:0/3927814776 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73e8036040 con 0x562f03b76ac0 2024-09-19T05:09:00.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.486+0000 7f73cf400640 1 -- 172.21.15.43:0/3927814776 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f73dc0a4390 con 0x562f03b76ac0 2024-09-19T05:09:00.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.486+0000 7f73f71d5740 10 client.26068 fetch_fsmap learned FSMap version 29 2024-09-19T05:09:00.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.487+0000 7f73f71d5740 1 -- 172.21.15.43:0/3927814776 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562f03bfc5b0 con 0x562f03b76ac0 2024-09-19T05:09:00.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.487+0000 7f73cf400640 1 -- 172.21.15.43:0/3927814776 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73e8036040 con 0x562f03b76ac0 2024-09-19T05:09:00.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.487+0000 7f73cf400640 1 -- 172.21.15.43:0/3927814776 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f73dc097d90 con 0x562f03b76ac0 2024-09-19T05:09:00.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.487+0000 7f73f71d5740 10 client.26068 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:09:00.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.487+0000 7f73f71d5740 10 client.26068 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:09:00.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.487+0000 7f73f71d5740 20 client.26068 populate_metadata read hostname 'smithi043' 2024-09-19T05:09:00.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.487+0000 7f73f71d5740 1 --2- 172.21.15.43:0/3927814776 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x562f03b81ca0 0x562f03bfed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:00.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.487+0000 7f73f71d5740 4 client.26068 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T05:09:00.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.487+0000 7f73f71d5740 1 -- 172.21.15.43:0/3927814776 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562f03ae4d10 con 0x562f03b81ca0 2024-09-19T05:09:00.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.488+0000 7f73f1e00640 1 --2- 172.21.15.43:0/3927814776 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x562f03b81ca0 0x562f03bfed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:00.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.488+0000 7f73f1e00640 1 --2- 172.21.15.43:0/3927814776 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x562f03b81ca0 0x562f03bfed90 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=738a2e121b8645f6 server_cookie=7c3adb89075c5846 in_seq=0 out_seq=0 2024-09-19T05:09:00.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.488+0000 7f73cf400640 10 client.26068 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:09:00.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.489+0000 7f73cf400640 1 -- 172.21.15.43:0/3927814776 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562f03ae4d10 con 0x562f03b81ca0 2024-09-19T05:09:00.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.489+0000 7f73cf400640 10 client.26068 handle_command_reply: tid=0 2024-09-19T05:09:00.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.489+0000 7f73f71d5740 1 client.26068 shutdown 2024-09-19T05:09:00.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.489+0000 7f73f71d5740 1 -- 172.21.15.43:0/3927814776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562f03b76ac0 msgr2=0x562f03b7b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:00.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.489+0000 7f73f71d5740 1 --2- 172.21.15.43:0/3927814776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562f03b76ac0 0x562f03b7b780 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f73dc09b420 tx=0x7f73dc09b140 comp rx=0 tx=0).stop 2024-09-19T05:09:00.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.489+0000 7f73f71d5740 1 -- 172.21.15.43:0/3927814776 shutdown_connections 2024-09-19T05:09:00.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.489+0000 7f73f71d5740 1 --2- 172.21.15.43:0/3927814776 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x562f03b81ca0 0x562f03bfed90 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:00.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.490+0000 7f73f71d5740 1 --2- 172.21.15.43:0/3927814776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f03b773e0 0x562f03b7bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:00.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.490+0000 7f73f71d5740 1 --2- 172.21.15.43:0/3927814776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562f03b76ac0 0x562f03b7b780 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:00.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.490+0000 7f73f71d5740 1 --2- 172.21.15.43:0/3927814776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f03b73050 0x562f03b8a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:00.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.490+0000 7f73cf400640 0 client.26068 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:09:00.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.490+0000 7f73f71d5740 1 -- 172.21.15.43:0/3927814776 >> 172.21.15.43:0/3927814776 conn(0x562f03b73ad0 msgr2=0x562f03b7d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:00.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.490+0000 7f73f71d5740 1 -- 172.21.15.43:0/3927814776 shutdown_connections 2024-09-19T05:09:00.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.490+0000 7f73f71d5740 1 -- 172.21.15.43:0/3927814776 wait complete. 2024-09-19T05:09:00.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.490+0000 7f73f71d5740 20 client.26068 trim_cache size 0 max 16384 2024-09-19T05:09:00.493 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:09:00.493 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-09-19T05:09:00.493 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "e42c9d8c-0ded-4dd2-b7f6-e90dd339b657", 2024-09-19T05:09:00.493 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-09-19T05:09:00.493 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:09:00.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.491+0000 7f73f4800640 1 -- 172.21.15.43:0/364825672 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f73d40a3430 msgr2=0x7f73d40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:00.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.491+0000 7f73f4800640 1 --2- 172.21.15.43:0/364825672 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f73d40a3430 0x7f73d40a58f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x562f03b73eb0 tx=0x7f73e0005f90 comp rx=0 tx=0).stop 2024-09-19T05:09:00.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.491+0000 7f73f4800640 1 -- 172.21.15.43:0/364825672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73ec1049a0 msgr2=0x7f73ec19ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:00.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.491+0000 7f73f4800640 1 --2- 172.21.15.43:0/364825672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73ec1049a0 0x7f73ec19ab40 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f73dc00b3f0 tx=0x7f73dc00b8b0 comp rx=0 tx=0).stop 2024-09-19T05:09:00.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.491+0000 7f73f4800640 1 -- 172.21.15.43:0/364825672 shutdown_connections 2024-09-19T05:09:00.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.491+0000 7f73f4800640 1 --2- 172.21.15.43:0/364825672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73ec109120 0x7f73ec19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:00.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.491+0000 7f73f4800640 1 --2- 172.21.15.43:0/364825672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73ec1049a0 0x7f73ec19ab40 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:00.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.491+0000 7f73f4800640 1 --2- 172.21.15.43:0/364825672 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f73d40a3430 0x7f73d40a58f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:00.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.491+0000 7f73f4800640 1 --2- 172.21.15.43:0/364825672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73ec103ff0 0x7f73ec19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:00.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.491+0000 7f73f4800640 1 -- 172.21.15.43:0/364825672 >> 172.21.15.43:0/364825672 conn(0x7f73ec0ffa90 msgr2=0x7f73ec101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:00.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.491+0000 7f73f4800640 1 -- 172.21.15.43:0/364825672 shutdown_connections 2024-09-19T05:09:00.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:00.491+0000 7f73f4800640 1 -- 172.21.15.43:0/364825672 wait complete. 2024-09-19T05:09:01.349 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:09:01.734 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:09:01.759 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:01 smithi043 ceph-mon[30800]: pgmap v134: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:09:01.760 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:01 smithi043 ceph-mon[30800]: scrub queued for path: / 2024-09-19T05:09:01.760 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:01 smithi043 ceph-mon[30800]: scrub summary: idle+waiting paths [/] 2024-09-19T05:09:01.760 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:01 smithi043 ceph-mon[30800]: scrub summary: active paths [/] 2024-09-19T05:09:01.760 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:01 smithi043 ceph-mon[30800]: scrub summary: idle+waiting paths [/] 2024-09-19T05:09:01.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:01 smithi151 ceph-mon[35425]: pgmap v134: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:09:01.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:01 smithi151 ceph-mon[35425]: scrub queued for path: / 2024-09-19T05:09:01.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:01 smithi151 ceph-mon[35425]: scrub summary: idle+waiting paths [/] 2024-09-19T05:09:01.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:01 smithi151 ceph-mon[35425]: scrub summary: active paths [/] 2024-09-19T05:09:01.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:01 smithi151 ceph-mon[35425]: scrub summary: idle+waiting paths [/] 2024-09-19T05:09:01.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:01 smithi096 ceph-mon[35085]: pgmap v134: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:09:01.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:01 smithi096 ceph-mon[35085]: scrub queued for path: / 2024-09-19T05:09:01.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:01 smithi096 ceph-mon[35085]: scrub summary: idle+waiting paths [/] 2024-09-19T05:09:01.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:01 smithi096 ceph-mon[35085]: scrub summary: active paths [/] 2024-09-19T05:09:01.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:01 smithi096 ceph-mon[35085]: scrub summary: idle+waiting paths [/] 2024-09-19T05:09:02.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.412+0000 7fc1a1a00640 1 -- 172.21.15.43:0/894987417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc19c077040 msgr2=0x7fc19c112510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:02.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.412+0000 7fc1a1a00640 1 --2- 172.21.15.43:0/894987417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc19c077040 0x7fc19c112510 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc184009900 tx=0x7fc18402a410 comp rx=0 tx=0).stop 2024-09-19T05:09:02.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.412+0000 7fc1a1a00640 1 -- 172.21.15.43:0/894987417 shutdown_connections 2024-09-19T05:09:02.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.412+0000 7fc1a1a00640 1 --2- 172.21.15.43:0/894987417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc19c112a50 0x7fc19c114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:02.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.412+0000 7fc1a1a00640 1 --2- 172.21.15.43:0/894987417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc19c077040 0x7fc19c112510 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:02.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.412+0000 7fc1a1a00640 1 --2- 172.21.15.43:0/894987417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc19c076720 0x7fc19c076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:02.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.412+0000 7fc1a1a00640 1 -- 172.21.15.43:0/894987417 >> 172.21.15.43:0/894987417 conn(0x7fc19c0ffa50 msgr2=0x7fc19c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:02.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.412+0000 7fc1a1a00640 1 -- 172.21.15.43:0/894987417 shutdown_connections 2024-09-19T05:09:02.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.412+0000 7fc1a1a00640 1 -- 172.21.15.43:0/894987417 wait complete. 2024-09-19T05:09:02.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.413+0000 7fc1a1a00640 1 Processor -- start 2024-09-19T05:09:02.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.413+0000 7fc1a1a00640 1 -- start start 2024-09-19T05:09:02.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.414+0000 7fc1a1a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc19c076720 0x7fc19c1a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:02.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.414+0000 7fc1a1a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc19c077040 0x7fc19c1a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:02.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.414+0000 7fc1a1a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc19c112a50 0x7fc19c19e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:02.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.414+0000 7fc1a1a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c1185e0 con 0x7fc19c076720 2024-09-19T05:09:02.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.414+0000 7fc1a1a00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c118460 con 0x7fc19c112a50 2024-09-19T05:09:02.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.414+0000 7fc1a1a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c118760 con 0x7fc19c077040 2024-09-19T05:09:02.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.414+0000 7fc19aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc19c076720 0x7fc19c1a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:02.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.414+0000 7fc19aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc19c076720 0x7fc19c1a4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48892/0 (socket says 172.21.15.43:48892) 2024-09-19T05:09:02.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.414+0000 7fc19aa00640 1 -- 172.21.15.43:0/3612571167 learned_addr learned my addr 172.21.15.43:0/3612571167 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:02.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.414+0000 7fc192000640 1 --2- 172.21.15.43:0/3612571167 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc19c077040 0x7fc19c1a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:02.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.414+0000 7fc19b400640 1 --2- 172.21.15.43:0/3612571167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc19c112a50 0x7fc19c19e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:02.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.414+0000 7fc19aa00640 1 -- 172.21.15.43:0/3612571167 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc19c077040 msgr2=0x7fc19c1a4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:02.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.414+0000 7fc19aa00640 1 --2- 172.21.15.43:0/3612571167 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc19c077040 0x7fc19c1a4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:02.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.414+0000 7fc19aa00640 1 -- 172.21.15.43:0/3612571167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc19c112a50 msgr2=0x7fc19c19e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:02.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.414+0000 7fc19aa00640 1 --2- 172.21.15.43:0/3612571167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc19c112a50 0x7fc19c19e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:02.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.414+0000 7fc19aa00640 1 -- 172.21.15.43:0/3612571167 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc19c19f010 con 0x7fc19c076720 2024-09-19T05:09:02.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.414+0000 7fc192000640 1 --2- 172.21.15.43:0/3612571167 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc19c077040 0x7fc19c1a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:02.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.415+0000 7fc19aa00640 1 --2- 172.21.15.43:0/3612571167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc19c076720 0x7fc19c1a4720 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc18c00e9f0 tx=0x7fc18c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:02.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.415+0000 7fc193e00640 1 -- 172.21.15.43:0/3612571167 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c00ccc0 con 0x7fc19c076720 2024-09-19T05:09:02.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.415+0000 7fc193e00640 1 -- 172.21.15.43:0/3612571167 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc18c00ce60 con 0x7fc19c076720 2024-09-19T05:09:02.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.415+0000 7fc193e00640 1 -- 172.21.15.43:0/3612571167 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c010690 con 0x7fc19c076720 2024-09-19T05:09:02.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.415+0000 7fc1a1a00640 1 -- 172.21.15.43:0/3612571167 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc19c19f300 con 0x7fc19c076720 2024-09-19T05:09:02.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.415+0000 7fc1a1a00640 1 -- 172.21.15.43:0/3612571167 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc19c1ab5f0 con 0x7fc19c076720 2024-09-19T05:09:02.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.418+0000 7fc193e00640 1 -- 172.21.15.43:0/3612571167 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fc18c0046c0 con 0x7fc19c076720 2024-09-19T05:09:02.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.418+0000 7fc1a45cc740 1 Processor -- start 2024-09-19T05:09:02.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.418+0000 7fc1a45cc740 1 -- start start 2024-09-19T05:09:02.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.419+0000 7fc1a45cc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558dba162b90 0x558dba162f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:02.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.419+0000 7fc1a45cc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558dba1634b0 0x558dba16d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:02.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.419+0000 7fc1a45cc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dba16dd70 0x558dba170160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:02.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.419+0000 7fc1a45cc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558dba0c93e0 con 0x558dba16dd70 2024-09-19T05:09:02.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.419+0000 7fc1a45cc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558dba070d80 con 0x558dba162b90 2024-09-19T05:09:02.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.419+0000 7fc1a45cc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558dba0c53a0 con 0x558dba1634b0 2024-09-19T05:09:02.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.419+0000 7fc192000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dba16dd70 0x558dba170160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:02.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.419+0000 7fc192000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dba16dd70 0x558dba170160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48902/0 (socket says 172.21.15.43:48902) 2024-09-19T05:09:02.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.419+0000 7fc192000640 1 -- 172.21.15.43:0/221767729 learned_addr learned my addr 172.21.15.43:0/221767729 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:02.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.419+0000 7fc19b400640 1 --2- 172.21.15.43:0/221767729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558dba162b90 0x558dba162f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:02.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.419+0000 7fc19aa00640 1 --2- 172.21.15.43:0/221767729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558dba1634b0 0x558dba16d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:02.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.420+0000 7fc19b400640 1 -- 172.21.15.43:0/221767729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558dba1634b0 msgr2=0x558dba16d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:02.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.420+0000 7fc19b400640 1 --2- 172.21.15.43:0/221767729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558dba1634b0 0x558dba16d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:02.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.420+0000 7fc19b400640 1 -- 172.21.15.43:0/221767729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dba16dd70 msgr2=0x558dba170160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:02.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.420+0000 7fc19b400640 1 --2- 172.21.15.43:0/221767729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dba16dd70 0x558dba170160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:02.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.420+0000 7fc19b400640 1 -- 172.21.15.43:0/221767729 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dba08fae0 con 0x558dba162b90 2024-09-19T05:09:02.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.420+0000 7fc19b400640 1 --2- 172.21.15.43:0/221767729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558dba162b90 0x558dba162f70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc19c07aa10 tx=0x7fc194011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2be735190101d48f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:02.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.420+0000 7fc18be00640 1 -- 172.21.15.43:0/221767729 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194004d20 con 0x558dba162b90 2024-09-19T05:09:02.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.420+0000 7fc18be00640 1 -- 172.21.15.43:0/221767729 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc194004050 con 0x558dba162b90 2024-09-19T05:09:02.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.420+0000 7fc18be00640 1 -- 172.21.15.43:0/221767729 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194005220 con 0x558dba162b90 2024-09-19T05:09:02.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.420+0000 7fc193e00640 1 --2- 172.21.15.43:0/3612571167 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fc1800a3430 0x7fc1800a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:02.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.421+0000 7fc193e00640 1 -- 172.21.15.43:0/3612571167 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fc18c0d0070 con 0x7fc19c076720 2024-09-19T05:09:02.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.421+0000 7fc19aa00640 1 --2- 172.21.15.43:0/3612571167 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fc1800a3430 0x7fc1800a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:02.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.421+0000 7fc1a45cc740 1 -- 172.21.15.43:0/221767729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558dba162b90 msgr2=0x558dba162f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:02.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.421+0000 7fc1a45cc740 1 --2- 172.21.15.43:0/221767729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558dba162b90 0x558dba162f70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc19c07aa10 tx=0x7fc194011810 comp rx=0 tx=0).stop 2024-09-19T05:09:02.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.421+0000 7fc1a45cc740 1 -- 172.21.15.43:0/221767729 shutdown_connections 2024-09-19T05:09:02.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.421+0000 7fc1a45cc740 1 --2- 172.21.15.43:0/221767729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dba16dd70 0x558dba170160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:02.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.421+0000 7fc1a45cc740 1 --2- 172.21.15.43:0/221767729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558dba1634b0 0x558dba16d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:02.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.421+0000 7fc1a45cc740 1 --2- 172.21.15.43:0/221767729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558dba162b90 0x558dba162f70 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:02.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.421+0000 7fc1a45cc740 1 -- 172.21.15.43:0/221767729 >> 172.21.15.43:0/221767729 conn(0x558dba15f120 msgr2=0x558dba15f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:02.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.421+0000 7fc1a45cc740 1 -- 172.21.15.43:0/221767729 shutdown_connections 2024-09-19T05:09:02.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.421+0000 7fc1a45cc740 1 -- 172.21.15.43:0/221767729 wait complete. 2024-09-19T05:09:02.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.421+0000 7fc19aa00640 1 --2- 172.21.15.43:0/3612571167 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fc1800a3430 0x7fc1800a58f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc18c0cd880 tx=0x7fc18c0bc040 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:02.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.422+0000 7fc1a45cc740 1 -- start start 2024-09-19T05:09:02.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.422+0000 7fc1a45cc740 1 Processor -- start 2024-09-19T05:09:02.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.423+0000 7fc1a45cc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558dba162b90 0x558dba16acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:02.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.423+0000 7fc1a45cc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558dba1634b0 0x558dba16b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:02.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.423+0000 7fc1a45cc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dba16dd70 0x558dba16b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:02.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.423+0000 7fc1a45cc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558dba1709e0 con 0x558dba16dd70 2024-09-19T05:09:02.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.423+0000 7fc1a45cc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558dba0840f0 con 0x558dba1634b0 2024-09-19T05:09:02.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.423+0000 7fc1a45cc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558dba170b60 con 0x558dba162b90 2024-09-19T05:09:02.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.423+0000 7fc1a45cc740 10 client.0 _ll_register_callbacks cb 0x558dba0759e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:02.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.423+0000 7fc19b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558dba162b90 0x558dba16acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:02.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.423+0000 7fc19b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dba16dd70 0x558dba16b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:02.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.423+0000 7fc192000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558dba1634b0 0x558dba16b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:02.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.423+0000 7fc19b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558dba162b90 0x558dba16acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:38560/0 (socket says 172.21.15.43:38560) 2024-09-19T05:09:02.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.423+0000 7fc19b400640 1 -- 172.21.15.43:0/1744390607 learned_addr learned my addr 172.21.15.43:0/1744390607 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:02.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.423+0000 7fc192000640 1 -- 172.21.15.43:0/1744390607 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558dba162b90 msgr2=0x558dba16acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:02.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.423+0000 7fc192000640 1 --2- 172.21.15.43:0/1744390607 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558dba162b90 0x558dba16acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:02.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.423+0000 7fc192000640 1 -- 172.21.15.43:0/1744390607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dba16dd70 msgr2=0x558dba16b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:02.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.423+0000 7fc192000640 1 --2- 172.21.15.43:0/1744390607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dba16dd70 0x558dba16b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:02.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.423+0000 7fc192000640 1 -- 172.21.15.43:0/1744390607 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558db9e3a3c0 con 0x558dba1634b0 2024-09-19T05:09:02.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.424+0000 7fc192000640 1 --2- 172.21.15.43:0/1744390607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558dba1634b0 0x558dba16b1f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc184000c00 tx=0x7fc1840317e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cf6a648d6653487 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:02.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.424+0000 7fc188c00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:09:02.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.424+0000 7fc188c00640 1 -- 172.21.15.43:0/1744390607 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc184035040 con 0x558dba1634b0 2024-09-19T05:09:02.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.424+0000 7fc188c00640 1 -- 172.21.15.43:0/1744390607 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fc184004250 con 0x558dba1634b0 2024-09-19T05:09:02.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.424+0000 7fc1a45cc740 1 -- 172.21.15.43:0/1744390607 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc194004050 con 0x558dba1634b0 2024-09-19T05:09:02.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.424+0000 7fc188c00640 1 -- 172.21.15.43:0/1744390607 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc184004450 con 0x558dba1634b0 2024-09-19T05:09:02.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.424+0000 7fc1a45cc740 10 client.26138 fetch_fsmap learned FSMap version 29 2024-09-19T05:09:02.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.424+0000 7fc1a45cc740 1 -- 172.21.15.43:0/1744390607 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558dba17c6e0 con 0x558dba1634b0 2024-09-19T05:09:02.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.425+0000 7fc188c00640 1 -- 172.21.15.43:0/1744390607 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc194004050 con 0x558dba1634b0 2024-09-19T05:09:02.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.425+0000 7fc188c00640 1 -- 172.21.15.43:0/1744390607 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc184048720 con 0x558dba1634b0 2024-09-19T05:09:02.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.425+0000 7fc1a45cc740 10 client.26138 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:09:02.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.425+0000 7fc1a45cc740 10 client.26138 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:09:02.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.425+0000 7fc1a45cc740 20 client.26138 populate_metadata read hostname 'smithi043' 2024-09-19T05:09:02.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.425+0000 7fc1a45cc740 1 --2- 172.21.15.43:0/1744390607 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558dba17dba0 0x558dba17ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:02.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.425+0000 7fc1a45cc740 4 client.26138 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:09:02.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.425+0000 7fc1a45cc740 1 -- 172.21.15.43:0/1744390607 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558dba0d6950 con 0x558dba17dba0 2024-09-19T05:09:02.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.425+0000 7fc19b400640 1 --2- 172.21.15.43:0/1744390607 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558dba17dba0 0x558dba17ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:02.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.426+0000 7fc19b400640 1 --2- 172.21.15.43:0/1744390607 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558dba17dba0 0x558dba17ff90 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73e139d43fb43409 server_cookie=77fcbbcd0329f394 in_seq=0 out_seq=0 2024-09-19T05:09:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.426+0000 7fc188c00640 10 client.26138 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:09:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.427+0000 7fc188c00640 1 -- 172.21.15.43:0/1744390607 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558dba0d6950 con 0x558dba17dba0 2024-09-19T05:09:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.427+0000 7fc188c00640 10 client.26138 handle_command_reply: tid=0 2024-09-19T05:09:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.427+0000 7fc1a45cc740 1 client.26138 shutdown 2024-09-19T05:09:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.427+0000 7fc1a45cc740 1 -- 172.21.15.43:0/1744390607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558dba1634b0 msgr2=0x558dba16b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.427+0000 7fc1a45cc740 1 --2- 172.21.15.43:0/1744390607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558dba1634b0 0x558dba16b1f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc184000c00 tx=0x7fc1840317e0 comp rx=0 tx=0).stop 2024-09-19T05:09:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.427+0000 7fc1a45cc740 1 -- 172.21.15.43:0/1744390607 shutdown_connections 2024-09-19T05:09:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.427+0000 7fc1a45cc740 1 --2- 172.21.15.43:0/1744390607 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558dba17dba0 0x558dba17ff90 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.427+0000 7fc1a45cc740 1 --2- 172.21.15.43:0/1744390607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dba16dd70 0x558dba16b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.427+0000 7fc1a45cc740 1 --2- 172.21.15.43:0/1744390607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558dba1634b0 0x558dba16b1f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.427+0000 7fc1a45cc740 1 --2- 172.21.15.43:0/1744390607 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558dba162b90 0x558dba16acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.427+0000 7fc188c00640 0 client.26138 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:09:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.427+0000 7fc1a45cc740 1 -- 172.21.15.43:0/1744390607 >> 172.21.15.43:0/1744390607 conn(0x558dba15f120 msgr2=0x558dba164e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:02.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.427+0000 7fc1a45cc740 1 -- 172.21.15.43:0/1744390607 shutdown_connections 2024-09-19T05:09:02.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.427+0000 7fc1a45cc740 1 -- 172.21.15.43:0/1744390607 wait complete. 2024-09-19T05:09:02.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.427+0000 7fc1a45cc740 20 client.26138 trim_cache size 0 max 16384 2024-09-19T05:09:02.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.445+0000 7fc1a45cc740 1 Processor -- start 2024-09-19T05:09:02.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.446+0000 7fc1a45cc740 1 -- start start 2024-09-19T05:09:02.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.446+0000 7fc1a45cc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558dba162b90 0x558dba169580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:02.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.446+0000 7fc1a45cc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dba1634b0 0x558dba16b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:02.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.446+0000 7fc1a45cc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558dba16dd70 0x558dba15e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:02.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.446+0000 7fc1a45cc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c1b3e20 con 0x558dba1634b0 2024-09-19T05:09:02.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.446+0000 7fc1a45cc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c1b5960 con 0x558dba162b90 2024-09-19T05:09:02.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.446+0000 7fc1a45cc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c1b44f0 con 0x558dba16dd70 2024-09-19T05:09:02.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.446+0000 7fc192000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dba1634b0 0x558dba16b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:02.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.446+0000 7fc192000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dba1634b0 0x558dba16b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48932/0 (socket says 172.21.15.43:48932) 2024-09-19T05:09:02.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.446+0000 7fc192000640 1 -- 172.21.15.43:0/124017338 learned_addr learned my addr 172.21.15.43:0/124017338 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:02.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.446+0000 7fc19b400640 1 --2- 172.21.15.43:0/124017338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558dba162b90 0x558dba169580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:02.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.446+0000 7fc19b400640 1 --2- 172.21.15.43:0/124017338 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558dba16dd70 0x558dba15e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:02.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.447+0000 7fc192000640 1 -- 172.21.15.43:0/124017338 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558dba16dd70 msgr2=0x558dba15e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:02.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.447+0000 7fc192000640 1 --2- 172.21.15.43:0/124017338 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558dba16dd70 0x558dba15e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:02.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.447+0000 7fc192000640 1 -- 172.21.15.43:0/124017338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558dba162b90 msgr2=0x558dba169580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:02.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.447+0000 7fc192000640 1 --2- 172.21.15.43:0/124017338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558dba162b90 0x558dba169580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:02.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.447+0000 7fc192000640 1 -- 172.21.15.43:0/124017338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dba1632e0 con 0x558dba1634b0 2024-09-19T05:09:02.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.447+0000 7fc192000640 1 --2- 172.21.15.43:0/124017338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dba1634b0 0x558dba16b580 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc184048b70 tx=0x7fc184009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed6fdb7c5fa9ed2d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:02.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.447+0000 7fc18b400640 1 -- 172.21.15.43:0/124017338 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc184035070 con 0x558dba1634b0 2024-09-19T05:09:02.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.447+0000 7fc18b400640 1 -- 172.21.15.43:0/124017338 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc184005550 con 0x558dba1634b0 2024-09-19T05:09:02.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.447+0000 7fc18b400640 1 -- 172.21.15.43:0/124017338 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc184055040 con 0x558dba1634b0 2024-09-19T05:09:02.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.448+0000 7fc1a45cc740 1 -- 172.21.15.43:0/124017338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dba1634b0 msgr2=0x558dba16b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:02.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.448+0000 7fc1a45cc740 1 --2- 172.21.15.43:0/124017338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dba1634b0 0x558dba16b580 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc184048b70 tx=0x7fc184009230 comp rx=0 tx=0).stop 2024-09-19T05:09:02.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.448+0000 7fc1a45cc740 1 -- 172.21.15.43:0/124017338 shutdown_connections 2024-09-19T05:09:02.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.448+0000 7fc1a45cc740 1 --2- 172.21.15.43:0/124017338 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558dba16dd70 0x558dba15e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:02.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.448+0000 7fc1a45cc740 1 --2- 172.21.15.43:0/124017338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dba1634b0 0x558dba16b580 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:02.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.448+0000 7fc1a45cc740 1 --2- 172.21.15.43:0/124017338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558dba162b90 0x558dba169580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:02.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.448+0000 7fc1a45cc740 1 -- 172.21.15.43:0/124017338 >> 172.21.15.43:0/124017338 conn(0x558dba15f120 msgr2=0x558dba15d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:02.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.448+0000 7fc1a45cc740 1 -- 172.21.15.43:0/124017338 shutdown_connections 2024-09-19T05:09:02.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.448+0000 7fc1a45cc740 1 -- 172.21.15.43:0/124017338 wait complete. 2024-09-19T05:09:02.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.449+0000 7fc1a45cc740 1 -- start start 2024-09-19T05:09:02.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.449+0000 7fc1a45cc740 1 Processor -- start 2024-09-19T05:09:02.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.449+0000 7fc1a45cc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dba162b90 0x558dba171d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:02.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.449+0000 7fc1a45cc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558dba1634b0 0x558dba172250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:02.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.449+0000 7fc1a45cc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558dba16dd70 0x558dba172790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:02.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.449+0000 7fc1a45cc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c1b5290 con 0x558dba162b90 2024-09-19T05:09:02.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.449+0000 7fc1a45cc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c1b4bc0 con 0x558dba16dd70 2024-09-19T05:09:02.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.449+0000 7fc1a45cc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c1b1e00 con 0x558dba1634b0 2024-09-19T05:09:02.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.450+0000 7fc1a45cc740 10 client.0 _ll_register_callbacks cb 0x7fc184005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:02.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.450+0000 7fc19b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dba162b90 0x558dba171d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:02.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.450+0000 7fc19b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dba162b90 0x558dba171d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48942/0 (socket says 172.21.15.43:48942) 2024-09-19T05:09:02.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.450+0000 7fc19b400640 1 -- 172.21.15.43:0/53051445 learned_addr learned my addr 172.21.15.43:0/53051445 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:02.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.450+0000 7fc192000640 1 --2- 172.21.15.43:0/53051445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558dba1634b0 0x558dba172250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:02.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.450+0000 7fc19b400640 1 --2- 172.21.15.43:0/53051445 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558dba16dd70 0x558dba172790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:02.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.450+0000 7fc19b400640 1 -- 172.21.15.43:0/53051445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558dba1634b0 msgr2=0x558dba172250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:02.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.450+0000 7fc19b400640 1 --2- 172.21.15.43:0/53051445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558dba1634b0 0x558dba172250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:02.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.450+0000 7fc19b400640 1 -- 172.21.15.43:0/53051445 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558dba16dd70 msgr2=0x558dba172790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:02.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.450+0000 7fc19b400640 1 --2- 172.21.15.43:0/53051445 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558dba16dd70 0x558dba172790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:02.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.450+0000 7fc19b400640 1 -- 172.21.15.43:0/53051445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dba1e7a30 con 0x558dba162b90 2024-09-19T05:09:02.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.451+0000 7fc19b400640 1 --2- 172.21.15.43:0/53051445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dba162b90 0x558dba171d10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc194014040 tx=0x7fc194000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800593b15c329306 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:02.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.451+0000 7fc188c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:09:02.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.451+0000 7fc188c00640 1 -- 172.21.15.43:0/53051445 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19401d040 con 0x558dba162b90 2024-09-19T05:09:02.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.451+0000 7fc1a45cc740 1 -- 172.21.15.43:0/53051445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc184005550 con 0x558dba162b90 2024-09-19T05:09:02.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.451+0000 7fc188c00640 1 -- 172.21.15.43:0/53051445 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fc194002030 con 0x558dba162b90 2024-09-19T05:09:02.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.451+0000 7fc188c00640 1 -- 172.21.15.43:0/53051445 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc194004560 con 0x558dba162b90 2024-09-19T05:09:02.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.451+0000 7fc1a45cc740 10 client.16101 fetch_fsmap learned FSMap version 29 2024-09-19T05:09:02.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.451+0000 7fc1a45cc740 1 -- 172.21.15.43:0/53051445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558dba1e8d20 con 0x558dba162b90 2024-09-19T05:09:02.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.451+0000 7fc188c00640 1 -- 172.21.15.43:0/53051445 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc184005550 con 0x558dba162b90 2024-09-19T05:09:02.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.452+0000 7fc188c00640 1 -- 172.21.15.43:0/53051445 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc194033020 con 0x558dba162b90 2024-09-19T05:09:02.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.452+0000 7fc1a45cc740 10 client.16101 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:09:02.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.452+0000 7fc1a45cc740 10 client.16101 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:09:02.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.452+0000 7fc1a45cc740 20 client.16101 populate_metadata read hostname 'smithi043' 2024-09-19T05:09:02.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.452+0000 7fc1a45cc740 1 --2- 172.21.15.43:0/53051445 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558dba1e9200 0x558dba1eb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:02.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.452+0000 7fc1a45cc740 4 client.16101 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:09:02.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.452+0000 7fc1a45cc740 1 -- 172.21.15.43:0/53051445 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558dba0d0e60 con 0x558dba1e9200 2024-09-19T05:09:02.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.452+0000 7fc192000640 1 --2- 172.21.15.43:0/53051445 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558dba1e9200 0x558dba1eb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:02.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.452+0000 7fc192000640 1 --2- 172.21.15.43:0/53051445 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558dba1e9200 0x558dba1eb5f0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=515960a5c04c3c07 server_cookie=293d1269ac13fc98 in_seq=0 out_seq=0 2024-09-19T05:09:02.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.452+0000 7fc188c00640 10 client.16101 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:09:02.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.453+0000 7fc188c00640 1 -- 172.21.15.43:0/53051445 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558dba0d0e60 con 0x558dba1e9200 2024-09-19T05:09:02.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.453+0000 7fc188c00640 10 client.16101 handle_command_reply: tid=0 2024-09-19T05:09:02.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.453+0000 7fc1a45cc740 1 client.16101 shutdown 2024-09-19T05:09:02.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.453+0000 7fc1a45cc740 1 -- 172.21.15.43:0/53051445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dba162b90 msgr2=0x558dba171d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:02.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.453+0000 7fc1a45cc740 1 --2- 172.21.15.43:0/53051445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dba162b90 0x558dba171d10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc194014040 tx=0x7fc194000f40 comp rx=0 tx=0).stop 2024-09-19T05:09:02.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.453+0000 7fc1a45cc740 1 -- 172.21.15.43:0/53051445 shutdown_connections 2024-09-19T05:09:02.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.453+0000 7fc1a45cc740 1 --2- 172.21.15.43:0/53051445 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558dba1e9200 0x558dba1eb5f0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:02.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.453+0000 7fc1a45cc740 1 --2- 172.21.15.43:0/53051445 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558dba16dd70 0x558dba172790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:02.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.453+0000 7fc1a45cc740 1 --2- 172.21.15.43:0/53051445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558dba1634b0 0x558dba172250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:02.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.453+0000 7fc1a45cc740 1 --2- 172.21.15.43:0/53051445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dba162b90 0x558dba171d10 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:02.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.453+0000 7fc188c00640 0 client.16101 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:09:02.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.453+0000 7fc1a45cc740 1 -- 172.21.15.43:0/53051445 >> 172.21.15.43:0/53051445 conn(0x558dba15f120 msgr2=0x558dba164e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:02.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.453+0000 7fc1a45cc740 1 -- 172.21.15.43:0/53051445 shutdown_connections 2024-09-19T05:09:02.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.453+0000 7fc1a45cc740 1 -- 172.21.15.43:0/53051445 wait complete. 2024-09-19T05:09:02.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.453+0000 7fc1a45cc740 20 client.16101 trim_cache size 0 max 16384 2024-09-19T05:09:02.460 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:09:02.460 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T05:09:02.460 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-09-19T05:09:02.460 INFO:teuthology.orchestra.run.smithi043.stdout: "e42c9d8c-0ded-4dd2-b7f6-e90dd339b657": { 2024-09-19T05:09:02.460 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-09-19T05:09:02.460 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "e42c9d8c-0ded-4dd2-b7f6-e90dd339b657", 2024-09-19T05:09:02.460 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-09-19T05:09:02.460 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:09:02.460 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:09:02.460 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:09:02.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.454+0000 7fc1a1a00640 1 -- 172.21.15.43:0/3612571167 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fc1800a3430 msgr2=0x7fc1800a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:02.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.454+0000 7fc1a1a00640 1 --2- 172.21.15.43:0/3612571167 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fc1800a3430 0x7fc1800a58f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc18c0cd880 tx=0x7fc18c0bc040 comp rx=0 tx=0).stop 2024-09-19T05:09:02.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.454+0000 7fc1a1a00640 1 -- 172.21.15.43:0/3612571167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc19c076720 msgr2=0x7fc19c1a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:02.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.454+0000 7fc1a1a00640 1 --2- 172.21.15.43:0/3612571167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc19c076720 0x7fc19c1a4720 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc18c00e9f0 tx=0x7fc18c00edf0 comp rx=0 tx=0).stop 2024-09-19T05:09:02.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.454+0000 7fc1a1a00640 1 -- 172.21.15.43:0/3612571167 shutdown_connections 2024-09-19T05:09:02.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.454+0000 7fc1a1a00640 1 --2- 172.21.15.43:0/3612571167 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fc1800a3430 0x7fc1800a58f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:02.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.454+0000 7fc1a1a00640 1 --2- 172.21.15.43:0/3612571167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc19c112a50 0x7fc19c19e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:02.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.454+0000 7fc1a1a00640 1 --2- 172.21.15.43:0/3612571167 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc19c077040 0x7fc19c1a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:02.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.454+0000 7fc1a1a00640 1 --2- 172.21.15.43:0/3612571167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc19c076720 0x7fc19c1a4720 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:02.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.454+0000 7fc1a1a00640 1 -- 172.21.15.43:0/3612571167 >> 172.21.15.43:0/3612571167 conn(0x7fc19c0ffa50 msgr2=0x7fc19c101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:02.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.454+0000 7fc1a1a00640 1 -- 172.21.15.43:0/3612571167 shutdown_connections 2024-09-19T05:09:02.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:02.454+0000 7fc1a1a00640 1 -- 172.21.15.43:0/3612571167 wait complete. 2024-09-19T05:09:03.309 INFO:tasks.cephfs.filesystem:scrub status for tag:e42c9d8c-0ded-4dd2-b7f6-e90dd339b657 - {'path': '/', 'tag': 'e42c9d8c-0ded-4dd2-b7f6-e90dd339b657', 'options': 'recursive,force'} 2024-09-19T05:09:03.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:03 smithi151 ceph-mon[35425]: pgmap v135: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:09:03.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:03 smithi151 ceph-mon[35425]: scrub summary: idle 2024-09-19T05:09:03.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:03 smithi096 ceph-mon[35085]: pgmap v135: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:09:03.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:03 smithi096 ceph-mon[35085]: scrub summary: idle 2024-09-19T05:09:04.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:03 smithi043 ceph-mon[30800]: pgmap v135: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:09:04.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:03 smithi043 ceph-mon[30800]: scrub summary: idle 2024-09-19T05:09:05.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:05 smithi151 ceph-mon[35425]: pgmap v136: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-19T05:09:05.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:05 smithi096 ceph-mon[35085]: pgmap v136: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-19T05:09:06.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:05 smithi043 ceph-mon[30800]: pgmap v136: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-19T05:09:07.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:07 smithi151 ceph-mon[35425]: pgmap v137: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-19T05:09:07.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:07 smithi096 ceph-mon[35085]: pgmap v137: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-19T05:09:08.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:07 smithi043 ceph-mon[30800]: pgmap v137: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-19T05:09:09.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:09 smithi151 ceph-mon[35425]: pgmap v138: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-19T05:09:09.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:09 smithi096 ceph-mon[35085]: pgmap v138: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-19T05:09:10.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:09 smithi043 ceph-mon[30800]: pgmap v138: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-19T05:09:11.854 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:11 smithi043 ceph-mon[30800]: pgmap v139: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-19T05:09:11.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:11 smithi151 ceph-mon[35425]: pgmap v139: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-19T05:09:11.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:11 smithi096 ceph-mon[35085]: pgmap v139: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-19T05:09:13.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:13 smithi151 ceph-mon[35425]: pgmap v140: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-19T05:09:13.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:13 smithi096 ceph-mon[35085]: pgmap v140: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-19T05:09:14.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:13 smithi043 ceph-mon[30800]: pgmap v140: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-19T05:09:15.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:15 smithi151 ceph-mon[35425]: pgmap v141: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-19T05:09:15.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:15 smithi096 ceph-mon[35085]: pgmap v141: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-19T05:09:16.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:15 smithi043 ceph-mon[30800]: pgmap v141: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-19T05:09:17.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:17 smithi151 ceph-mon[35425]: pgmap v142: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-19T05:09:18.247 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:17 smithi043 ceph-mon[30800]: pgmap v142: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-19T05:09:18.247 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:17 smithi096 ceph-mon[35085]: pgmap v142: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-19T05:09:19.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:19 smithi151 ceph-mon[35425]: pgmap v143: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-19T05:09:19.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:19 smithi096 ceph-mon[35085]: pgmap v143: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-19T05:09:20.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:19 smithi043 ceph-mon[30800]: pgmap v143: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-19T05:09:21.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:21 smithi151 ceph-mon[35425]: pgmap v144: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-19T05:09:21.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:21 smithi096 ceph-mon[35085]: pgmap v144: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-19T05:09:22.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:21 smithi043 ceph-mon[30800]: pgmap v144: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-19T05:09:23.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:23 smithi151 ceph-mon[35425]: pgmap v145: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:09:23.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:23 smithi096 ceph-mon[35085]: pgmap v145: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:09:24.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:23 smithi043 ceph-mon[30800]: pgmap v145: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:09:25.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:25 smithi151 ceph-mon[35425]: pgmap v146: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:09:25.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:25 smithi096 ceph-mon[35085]: pgmap v146: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:09:26.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:25 smithi043 ceph-mon[30800]: pgmap v146: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:09:27.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:27 smithi151 ceph-mon[35425]: pgmap v147: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:09:27.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:27 smithi096 ceph-mon[35085]: pgmap v147: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:09:28.032 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:27 smithi043 ceph-mon[30800]: pgmap v147: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:09:29.833 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:29 smithi043 ceph-mon[30800]: pgmap v148: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:09:29.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:29 smithi151 ceph-mon[35425]: pgmap v148: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:09:29.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:29 smithi096 ceph-mon[35085]: pgmap v148: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:09:31.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:31 smithi043 ceph-mon[30800]: pgmap v149: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:09:31.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:31 smithi151 ceph-mon[35425]: pgmap v149: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:09:31.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:31 smithi096 ceph-mon[35085]: pgmap v149: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:09:33.311 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:09:33.709 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:09:33.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:33 smithi043 ceph-mon[30800]: pgmap v150: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:09:33.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:33 smithi151 ceph-mon[35425]: pgmap v150: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:09:33.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:33 smithi096 ceph-mon[35085]: pgmap v150: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:09:34.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.312+0000 7f295d000640 1 -- 172.21.15.43:0/1319360792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2958103ff0 msgr2=0x7f29581043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:34.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.312+0000 7f295d000640 1 --2- 172.21.15.43:0/1319360792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2958103ff0 0x7f29581043d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2948009ca0 tx=0x7f294802a8c0 comp rx=0 tx=0).stop 2024-09-19T05:09:34.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.312+0000 7f295d000640 1 -- 172.21.15.43:0/1319360792 shutdown_connections 2024-09-19T05:09:34.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.312+0000 7f295d000640 1 --2- 172.21.15.43:0/1319360792 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2958109120 0x7f2958110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:34.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.312+0000 7f295d000640 1 --2- 172.21.15.43:0/1319360792 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f29581049a0 0x7f29581089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:34.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.312+0000 7f295d000640 1 --2- 172.21.15.43:0/1319360792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2958103ff0 0x7f29581043d0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:34.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.312+0000 7f295d000640 1 -- 172.21.15.43:0/1319360792 >> 172.21.15.43:0/1319360792 conn(0x7f29580ffa90 msgr2=0x7f2958101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:34.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.312+0000 7f295d000640 1 -- 172.21.15.43:0/1319360792 shutdown_connections 2024-09-19T05:09:34.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.312+0000 7f295d000640 1 -- 172.21.15.43:0/1319360792 wait complete. 2024-09-19T05:09:34.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.313+0000 7f295d000640 1 Processor -- start 2024-09-19T05:09:34.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.313+0000 7f295d000640 1 -- start start 2024-09-19T05:09:34.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.314+0000 7f295d000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2958103ff0 0x7f295819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:34.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.314+0000 7f295d000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29581049a0 0x7f295819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:34.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.314+0000 7f295d000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2958109120 0x7f295819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:34.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.314+0000 7f295d000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2958114120 con 0x7f29581049a0 2024-09-19T05:09:34.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.314+0000 7f295d000640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2958113fa0 con 0x7f2958103ff0 2024-09-19T05:09:34.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.314+0000 7f295d000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f29581142a0 con 0x7f2958109120 2024-09-19T05:09:34.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.314+0000 7f2955e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2958103ff0 0x7f295819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:34.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.314+0000 7f294d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29581049a0 0x7f295819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:34.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.314+0000 7f2955e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2958103ff0 0x7f295819a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:47000/0 (socket says 172.21.15.43:47000) 2024-09-19T05:09:34.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.314+0000 7f2955e00640 1 -- 172.21.15.43:0/88942253 learned_addr learned my addr 172.21.15.43:0/88942253 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:34.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.314+0000 7f2956800640 1 --2- 172.21.15.43:0/88942253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2958109120 0x7f295819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:34.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.314+0000 7f2955e00640 1 -- 172.21.15.43:0/88942253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2958109120 msgr2=0x7f295819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:34.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.314+0000 7f2955e00640 1 --2- 172.21.15.43:0/88942253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2958109120 0x7f295819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:34.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.314+0000 7f2955e00640 1 -- 172.21.15.43:0/88942253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29581049a0 msgr2=0x7f295819ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:34.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.314+0000 7f2955e00640 1 --2- 172.21.15.43:0/88942253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29581049a0 0x7f295819ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:34.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.314+0000 7f2955e00640 1 -- 172.21.15.43:0/88942253 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f295819f5d0 con 0x7f2958103ff0 2024-09-19T05:09:34.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.315+0000 7f2955e00640 1 --2- 172.21.15.43:0/88942253 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2958103ff0 0x7f295819a620 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f294802ade0 tx=0x7f29480319f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:34.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.315+0000 7f294f400640 1 -- 172.21.15.43:0/88942253 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294802fa20 con 0x7f2958103ff0 2024-09-19T05:09:34.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.315+0000 7f295d000640 1 -- 172.21.15.43:0/88942253 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f295819f860 con 0x7f2958103ff0 2024-09-19T05:09:34.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.315+0000 7f294f400640 1 -- 172.21.15.43:0/88942253 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f294802fbc0 con 0x7f2958103ff0 2024-09-19T05:09:34.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.315+0000 7f294f400640 1 -- 172.21.15.43:0/88942253 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29480467b0 con 0x7f2958103ff0 2024-09-19T05:09:34.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.316+0000 7f295d000640 1 -- 172.21.15.43:0/88942253 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29581a71a0 con 0x7f2958103ff0 2024-09-19T05:09:34.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.318+0000 7f295fc00740 1 Processor -- start 2024-09-19T05:09:34.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.319+0000 7f295fc00740 1 -- start start 2024-09-19T05:09:34.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.319+0000 7f294f400640 1 -- 172.21.15.43:0/88942253 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f2948046950 con 0x7f2958103ff0 2024-09-19T05:09:34.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.319+0000 7f295fc00740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf0e579b90 0x55bf0e579f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:34.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.319+0000 7f295fc00740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0e57a4b0 0x55bf0e584830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:34.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.319+0000 7f295fc00740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf0e584d70 0x55bf0e587160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:34.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.319+0000 7f295fc00740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bf0e4e03e0 con 0x55bf0e57a4b0 2024-09-19T05:09:34.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.319+0000 7f295fc00740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55bf0e487d80 con 0x55bf0e584d70 2024-09-19T05:09:34.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.319+0000 7f295fc00740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55bf0e4dc3a0 con 0x55bf0e579b90 2024-09-19T05:09:34.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.319+0000 7f2955e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0e57a4b0 0x55bf0e584830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:34.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.319+0000 7f294d400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf0e584d70 0x55bf0e587160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:34.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.319+0000 7f2956800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf0e579b90 0x55bf0e579f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:34.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.319+0000 7f294d400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf0e584d70 0x55bf0e587160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:47016/0 (socket says 172.21.15.43:47016) 2024-09-19T05:09:34.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.319+0000 7f294d400640 1 -- 172.21.15.43:0/350081429 learned_addr learned my addr 172.21.15.43:0/350081429 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:34.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.320+0000 7f294d400640 1 -- 172.21.15.43:0/350081429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf0e579b90 msgr2=0x55bf0e579f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:34.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.320+0000 7f294d400640 1 --2- 172.21.15.43:0/350081429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf0e579b90 0x55bf0e579f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:34.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.320+0000 7f294d400640 1 -- 172.21.15.43:0/350081429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0e57a4b0 msgr2=0x55bf0e584830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:34.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.320+0000 7f294d400640 1 --2- 172.21.15.43:0/350081429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0e57a4b0 0x55bf0e584830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:34.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.320+0000 7f294d400640 1 -- 172.21.15.43:0/350081429 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf0e4a6ae0 con 0x55bf0e584d70 2024-09-19T05:09:34.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.320+0000 7f294d400640 1 --2- 172.21.15.43:0/350081429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf0e584d70 0x55bf0e587160 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f294000bc90 tx=0x7f29400107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba0901bd6281454b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:34.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.320+0000 7f2947400640 1 -- 172.21.15.43:0/350081429 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2940005220 con 0x55bf0e584d70 2024-09-19T05:09:34.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.320+0000 7f2947400640 1 -- 172.21.15.43:0/350081429 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f294000be40 con 0x55bf0e584d70 2024-09-19T05:09:34.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.320+0000 7f2947400640 1 -- 172.21.15.43:0/350081429 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2940011770 con 0x55bf0e584d70 2024-09-19T05:09:34.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.321+0000 7f294f400640 1 --2- 172.21.15.43:0/88942253 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f293c0a33b0 0x7f293c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:34.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.321+0000 7f294f400640 1 -- 172.21.15.43:0/88942253 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f29480f10b0 con 0x7f2958103ff0 2024-09-19T05:09:34.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.321+0000 7f2956800640 1 --2- 172.21.15.43:0/88942253 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f293c0a33b0 0x7f293c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:34.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.321+0000 7f295fc00740 1 -- 172.21.15.43:0/350081429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf0e584d70 msgr2=0x55bf0e587160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:34.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.321+0000 7f295fc00740 1 --2- 172.21.15.43:0/350081429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf0e584d70 0x55bf0e587160 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f294000bc90 tx=0x7f29400107a0 comp rx=0 tx=0).stop 2024-09-19T05:09:34.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.322+0000 7f295fc00740 1 -- 172.21.15.43:0/350081429 shutdown_connections 2024-09-19T05:09:34.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.322+0000 7f295fc00740 1 --2- 172.21.15.43:0/350081429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf0e584d70 0x55bf0e587160 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:34.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.322+0000 7f295fc00740 1 --2- 172.21.15.43:0/350081429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0e57a4b0 0x55bf0e584830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:34.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.322+0000 7f295fc00740 1 --2- 172.21.15.43:0/350081429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf0e579b90 0x55bf0e579f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:34.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.322+0000 7f295fc00740 1 -- 172.21.15.43:0/350081429 >> 172.21.15.43:0/350081429 conn(0x55bf0e576120 msgr2=0x55bf0e576500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:34.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.322+0000 7f295fc00740 1 -- 172.21.15.43:0/350081429 shutdown_connections 2024-09-19T05:09:34.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.322+0000 7f295fc00740 1 -- 172.21.15.43:0/350081429 wait complete. 2024-09-19T05:09:34.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.322+0000 7f2956800640 1 --2- 172.21.15.43:0/88942253 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f293c0a33b0 0x7f293c0a5870 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2950005fd0 tx=0x7f2950005cc0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:34.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.322+0000 7f295fc00740 1 -- start start 2024-09-19T05:09:34.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.323+0000 7f295fc00740 1 Processor -- start 2024-09-19T05:09:34.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.323+0000 7f295fc00740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf0e579b90 0x55bf0e581bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:34.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.323+0000 7f295fc00740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0e57a4b0 0x55bf0e582110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:34.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.323+0000 7f295fc00740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf0e584d70 0x55bf0e582650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:34.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.323+0000 7f295fc00740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bf0e5879e0 con 0x55bf0e57a4b0 2024-09-19T05:09:34.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.323+0000 7f295fc00740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55bf0e49b0f0 con 0x55bf0e579b90 2024-09-19T05:09:34.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.323+0000 7f295fc00740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55bf0e587b60 con 0x55bf0e584d70 2024-09-19T05:09:34.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.323+0000 7f295fc00740 10 client.0 _ll_register_callbacks cb 0x55bf0e48c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:34.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.323+0000 7f294d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0e57a4b0 0x55bf0e582110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:34.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.323+0000 7f294d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0e57a4b0 0x55bf0e582110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60642/0 (socket says 172.21.15.43:60642) 2024-09-19T05:09:34.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.323+0000 7f2955e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf0e579b90 0x55bf0e581bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:34.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.323+0000 7f294d400640 1 -- 172.21.15.43:0/2429176515 learned_addr learned my addr 172.21.15.43:0/2429176515 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:34.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.324+0000 7f2956800640 1 --2- 172.21.15.43:0/2429176515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf0e584d70 0x55bf0e582650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:34.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.324+0000 7f294d400640 1 -- 172.21.15.43:0/2429176515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf0e584d70 msgr2=0x55bf0e582650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:34.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.324+0000 7f294d400640 1 --2- 172.21.15.43:0/2429176515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf0e584d70 0x55bf0e582650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:34.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.324+0000 7f294d400640 1 -- 172.21.15.43:0/2429176515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf0e579b90 msgr2=0x55bf0e581bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:34.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.324+0000 7f294d400640 1 --2- 172.21.15.43:0/2429176515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf0e579b90 0x55bf0e581bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:34.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.324+0000 7f294d400640 1 -- 172.21.15.43:0/2429176515 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf0e2513c0 con 0x55bf0e57a4b0 2024-09-19T05:09:34.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.324+0000 7f2956800640 1 --2- 172.21.15.43:0/2429176515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf0e584d70 0x55bf0e582650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:09:34.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.324+0000 7f2955e00640 1 --2- 172.21.15.43:0/2429176515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf0e579b90 0x55bf0e581bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:34.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.324+0000 7f294d400640 1 --2- 172.21.15.43:0/2429176515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0e57a4b0 0x55bf0e582110 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2940010ca0 tx=0x7f2940005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fcd14f3a3014cce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:34.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.324+0000 7f293be00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:09:34.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.325+0000 7f293be00640 1 -- 172.21.15.43:0/2429176515 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2940005450 con 0x55bf0e57a4b0 2024-09-19T05:09:34.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.325+0000 7f293be00640 1 -- 172.21.15.43:0/2429176515 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f2940004020 con 0x55bf0e57a4b0 2024-09-19T05:09:34.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.325+0000 7f295fc00740 1 -- 172.21.15.43:0/2429176515 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bf0e4ed950 con 0x55bf0e57a4b0 2024-09-19T05:09:34.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.325+0000 7f293be00640 1 -- 172.21.15.43:0/2429176515 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2940004220 con 0x55bf0e57a4b0 2024-09-19T05:09:34.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.325+0000 7f295fc00740 10 client.16113 fetch_fsmap learned FSMap version 29 2024-09-19T05:09:34.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.325+0000 7f295fc00740 1 -- 172.21.15.43:0/2429176515 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bf0e593640 con 0x55bf0e57a4b0 2024-09-19T05:09:34.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.325+0000 7f293be00640 1 -- 172.21.15.43:0/2429176515 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55bf0e4ed950 con 0x55bf0e57a4b0 2024-09-19T05:09:34.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.325+0000 7f293be00640 1 -- 172.21.15.43:0/2429176515 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2940003a70 con 0x55bf0e57a4b0 2024-09-19T05:09:34.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.325+0000 7f295fc00740 10 client.16113 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:09:34.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.325+0000 7f295fc00740 10 client.16113 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:09:34.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.326+0000 7f295fc00740 20 client.16113 populate_metadata read hostname 'smithi043' 2024-09-19T05:09:34.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.326+0000 7f295fc00740 1 --2- 172.21.15.43:0/2429176515 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55bf0e5949f0 0x55bf0e596de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:34.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.326+0000 7f295fc00740 4 client.16113 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:09:34.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.326+0000 7f295fc00740 1 -- 172.21.15.43:0/2429176515 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf0e4e7e60 con 0x55bf0e5949f0 2024-09-19T05:09:34.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.326+0000 7f2955e00640 1 --2- 172.21.15.43:0/2429176515 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55bf0e5949f0 0x55bf0e596de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:34.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.327+0000 7f2955e00640 1 --2- 172.21.15.43:0/2429176515 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55bf0e5949f0 0x55bf0e596de0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8a166abb635206b server_cookie=bf8871d82fb583bd in_seq=0 out_seq=0 2024-09-19T05:09:34.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.327+0000 7f293be00640 10 client.16113 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:09:34.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.327+0000 7f293be00640 1 -- 172.21.15.43:0/2429176515 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bf0e4e7e60 con 0x55bf0e5949f0 2024-09-19T05:09:34.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.327+0000 7f293be00640 10 client.16113 handle_command_reply: tid=0 2024-09-19T05:09:34.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.327+0000 7f295fc00740 1 client.16113 shutdown 2024-09-19T05:09:34.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.328+0000 7f295fc00740 1 -- 172.21.15.43:0/2429176515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0e57a4b0 msgr2=0x55bf0e582110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:34.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.328+0000 7f295fc00740 1 --2- 172.21.15.43:0/2429176515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0e57a4b0 0x55bf0e582110 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2940010ca0 tx=0x7f2940005230 comp rx=0 tx=0).stop 2024-09-19T05:09:34.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.328+0000 7f295fc00740 1 -- 172.21.15.43:0/2429176515 shutdown_connections 2024-09-19T05:09:34.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.328+0000 7f295fc00740 1 --2- 172.21.15.43:0/2429176515 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55bf0e5949f0 0x55bf0e596de0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:34.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.328+0000 7f295fc00740 1 --2- 172.21.15.43:0/2429176515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf0e584d70 0x55bf0e582650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:34.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.328+0000 7f295fc00740 1 --2- 172.21.15.43:0/2429176515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0e57a4b0 0x55bf0e582110 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:34.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.328+0000 7f295fc00740 1 --2- 172.21.15.43:0/2429176515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf0e579b90 0x55bf0e581bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:34.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.328+0000 7f295fc00740 1 -- 172.21.15.43:0/2429176515 >> 172.21.15.43:0/2429176515 conn(0x55bf0e576120 msgr2=0x55bf0e57be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:34.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.328+0000 7f293be00640 0 client.16113 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:09:34.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.328+0000 7f295fc00740 1 -- 172.21.15.43:0/2429176515 shutdown_connections 2024-09-19T05:09:34.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.328+0000 7f295fc00740 1 -- 172.21.15.43:0/2429176515 wait complete. 2024-09-19T05:09:34.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.328+0000 7f295fc00740 20 client.16113 trim_cache size 0 max 16384 2024-09-19T05:09:34.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.346+0000 7f295fc00740 1 Processor -- start 2024-09-19T05:09:34.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.346+0000 7f295fc00740 1 -- start start 2024-09-19T05:09:34.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.347+0000 7f295fc00740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf0e579b90 0x55bf0e583b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:34.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.347+0000 7f295fc00740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0e57a4b0 0x55bf0e59c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:34.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.347+0000 7f295fc00740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf0e584d70 0x55bf0e59c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:34.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.347+0000 7f295fc00740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f29581afa40 con 0x55bf0e57a4b0 2024-09-19T05:09:34.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.347+0000 7f295fc00740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f29581b1580 con 0x55bf0e584d70 2024-09-19T05:09:34.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.347+0000 7f295fc00740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f29581b0110 con 0x55bf0e579b90 2024-09-19T05:09:34.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.347+0000 7f294d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0e57a4b0 0x55bf0e59c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:34.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.347+0000 7f294d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0e57a4b0 0x55bf0e59c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60656/0 (socket says 172.21.15.43:60656) 2024-09-19T05:09:34.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.347+0000 7f294d400640 1 -- 172.21.15.43:0/2907049014 learned_addr learned my addr 172.21.15.43:0/2907049014 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:34.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.347+0000 7f2956800640 1 --2- 172.21.15.43:0/2907049014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf0e584d70 0x55bf0e59c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:34.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.347+0000 7f294d400640 1 -- 172.21.15.43:0/2907049014 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf0e579b90 msgr2=0x55bf0e583b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:34.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.347+0000 7f294d400640 1 --2- 172.21.15.43:0/2907049014 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf0e579b90 0x55bf0e583b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:34.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.347+0000 7f294d400640 1 -- 172.21.15.43:0/2907049014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf0e584d70 msgr2=0x55bf0e59c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:34.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.347+0000 7f294d400640 1 --2- 172.21.15.43:0/2907049014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf0e584d70 0x55bf0e59c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:34.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.347+0000 7f294d400640 1 -- 172.21.15.43:0/2907049014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf0e575bc0 con 0x55bf0e57a4b0 2024-09-19T05:09:34.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.348+0000 7f294d400640 1 --2- 172.21.15.43:0/2907049014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0e57a4b0 0x55bf0e59c410 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f294000bbb0 tx=0x7f2940000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76de88d6bc5b5553 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:34.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.348+0000 7f2946a00640 1 -- 172.21.15.43:0/2907049014 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2940003fe0 con 0x55bf0e57a4b0 2024-09-19T05:09:34.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.348+0000 7f2946a00640 1 -- 172.21.15.43:0/2907049014 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2940011db0 con 0x55bf0e57a4b0 2024-09-19T05:09:34.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.348+0000 7f2946a00640 1 -- 172.21.15.43:0/2907049014 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2940004ad0 con 0x55bf0e57a4b0 2024-09-19T05:09:34.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.349+0000 7f295fc00740 1 -- 172.21.15.43:0/2907049014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0e57a4b0 msgr2=0x55bf0e59c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:34.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.349+0000 7f295fc00740 1 --2- 172.21.15.43:0/2907049014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0e57a4b0 0x55bf0e59c410 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f294000bbb0 tx=0x7f2940000bc0 comp rx=0 tx=0).stop 2024-09-19T05:09:34.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.349+0000 7f295fc00740 1 -- 172.21.15.43:0/2907049014 shutdown_connections 2024-09-19T05:09:34.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.349+0000 7f295fc00740 1 --2- 172.21.15.43:0/2907049014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf0e584d70 0x55bf0e59c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:34.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.349+0000 7f295fc00740 1 --2- 172.21.15.43:0/2907049014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0e57a4b0 0x55bf0e59c410 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:34.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.349+0000 7f295fc00740 1 --2- 172.21.15.43:0/2907049014 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf0e579b90 0x55bf0e583b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:34.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.349+0000 7f295fc00740 1 -- 172.21.15.43:0/2907049014 >> 172.21.15.43:0/2907049014 conn(0x55bf0e576120 msgr2=0x55bf0e583430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:34.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.349+0000 7f295fc00740 1 -- 172.21.15.43:0/2907049014 shutdown_connections 2024-09-19T05:09:34.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.349+0000 7f295fc00740 1 -- 172.21.15.43:0/2907049014 wait complete. 2024-09-19T05:09:34.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.349+0000 7f295fc00740 1 -- start start 2024-09-19T05:09:34.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.350+0000 7f295fc00740 1 Processor -- start 2024-09-19T05:09:34.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.350+0000 7f295fc00740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf0e579b90 0x55bf0e58cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:34.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.350+0000 7f295fc00740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0e57a4b0 0x55bf0e58d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:34.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.350+0000 7f295fc00740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf0e584d70 0x55bf0e57e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:34.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.350+0000 7f295fc00740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f29581b0eb0 con 0x55bf0e57a4b0 2024-09-19T05:09:34.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.350+0000 7f295fc00740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f29581b07e0 con 0x55bf0e579b90 2024-09-19T05:09:34.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.350+0000 7f295fc00740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f29581ada20 con 0x55bf0e584d70 2024-09-19T05:09:34.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.350+0000 7f295fc00740 10 client.0 _ll_register_callbacks cb 0x7f294000bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:34.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.350+0000 7f294d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0e57a4b0 0x55bf0e58d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:34.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.350+0000 7f2955e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf0e579b90 0x55bf0e58cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:34.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.350+0000 7f2955e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf0e579b90 0x55bf0e58cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:47040/0 (socket says 172.21.15.43:47040) 2024-09-19T05:09:34.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.350+0000 7f2955e00640 1 -- 172.21.15.43:0/2848834048 learned_addr learned my addr 172.21.15.43:0/2848834048 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:34.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.350+0000 7f2956800640 1 --2- 172.21.15.43:0/2848834048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf0e584d70 0x55bf0e57e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:34.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.351+0000 7f294d400640 1 -- 172.21.15.43:0/2848834048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf0e584d70 msgr2=0x55bf0e57e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:34.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.351+0000 7f294d400640 1 --2- 172.21.15.43:0/2848834048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf0e584d70 0x55bf0e57e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:34.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.351+0000 7f294d400640 1 -- 172.21.15.43:0/2848834048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf0e579b90 msgr2=0x55bf0e58cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:34.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.351+0000 7f294d400640 1 --2- 172.21.15.43:0/2848834048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf0e579b90 0x55bf0e58cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:34.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.351+0000 7f294d400640 1 -- 172.21.15.43:0/2848834048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf0e57ec00 con 0x55bf0e57a4b0 2024-09-19T05:09:34.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.351+0000 7f294d400640 1 --2- 172.21.15.43:0/2848834048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0e57a4b0 0x55bf0e58d500 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x55bf0e587890 tx=0x7f2940005f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7842c6f2ef5b6c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:34.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.351+0000 7f293be00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:09:34.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.351+0000 7f293be00640 1 -- 172.21.15.43:0/2848834048 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2940005620 con 0x55bf0e57a4b0 2024-09-19T05:09:34.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.352+0000 7f293be00640 1 -- 172.21.15.43:0/2848834048 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f2940011790 con 0x55bf0e57a4b0 2024-09-19T05:09:34.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.352+0000 7f295fc00740 1 -- 172.21.15.43:0/2848834048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2940011db0 con 0x55bf0e57a4b0 2024-09-19T05:09:34.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.352+0000 7f293be00640 1 -- 172.21.15.43:0/2848834048 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f29400119b0 con 0x55bf0e57a4b0 2024-09-19T05:09:34.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.352+0000 7f295fc00740 10 client.16119 fetch_fsmap learned FSMap version 29 2024-09-19T05:09:34.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.352+0000 7f295fc00740 1 -- 172.21.15.43:0/2848834048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bf0e5fee70 con 0x55bf0e57a4b0 2024-09-19T05:09:34.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.352+0000 7f293be00640 1 -- 172.21.15.43:0/2848834048 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2940011db0 con 0x55bf0e57a4b0 2024-09-19T05:09:34.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.352+0000 7f293be00640 1 -- 172.21.15.43:0/2848834048 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f294003d020 con 0x55bf0e57a4b0 2024-09-19T05:09:34.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.352+0000 7f295fc00740 10 client.16119 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:09:34.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.352+0000 7f295fc00740 10 client.16119 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:09:34.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.352+0000 7f295fc00740 20 client.16119 populate_metadata read hostname 'smithi043' 2024-09-19T05:09:34.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.352+0000 7f295fc00740 1 --2- 172.21.15.43:0/2848834048 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55bf0e5ff4d0 0x55bf0e6018c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:34.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.352+0000 7f295fc00740 4 client.16119 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:09:34.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.352+0000 7f295fc00740 1 -- 172.21.15.43:0/2848834048 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bf0e4640c0 con 0x55bf0e5ff4d0 2024-09-19T05:09:34.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.353+0000 7f2955e00640 1 --2- 172.21.15.43:0/2848834048 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55bf0e5ff4d0 0x55bf0e6018c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:34.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.353+0000 7f2955e00640 1 --2- 172.21.15.43:0/2848834048 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55bf0e5ff4d0 0x55bf0e6018c0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=231fdc93f516261c server_cookie=2c8157048ceb25c0 in_seq=0 out_seq=0 2024-09-19T05:09:34.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.353+0000 7f293be00640 10 client.16119 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:09:34.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.353+0000 7f293be00640 1 -- 172.21.15.43:0/2848834048 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bf0e4640c0 con 0x55bf0e5ff4d0 2024-09-19T05:09:34.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.353+0000 7f293be00640 10 client.16119 handle_command_reply: tid=0 2024-09-19T05:09:34.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.353+0000 7f295fc00740 1 client.16119 shutdown 2024-09-19T05:09:34.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.354+0000 7f295fc00740 1 -- 172.21.15.43:0/2848834048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0e57a4b0 msgr2=0x55bf0e58d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:34.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.354+0000 7f295fc00740 1 --2- 172.21.15.43:0/2848834048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0e57a4b0 0x55bf0e58d500 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x55bf0e587890 tx=0x7f2940005f50 comp rx=0 tx=0).stop 2024-09-19T05:09:34.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.354+0000 7f295fc00740 1 -- 172.21.15.43:0/2848834048 shutdown_connections 2024-09-19T05:09:34.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.354+0000 7f295fc00740 1 --2- 172.21.15.43:0/2848834048 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55bf0e5ff4d0 0x55bf0e6018c0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:34.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.354+0000 7f295fc00740 1 --2- 172.21.15.43:0/2848834048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bf0e584d70 0x55bf0e57e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:34.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.354+0000 7f295fc00740 1 --2- 172.21.15.43:0/2848834048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf0e57a4b0 0x55bf0e58d500 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:34.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.354+0000 7f295fc00740 1 --2- 172.21.15.43:0/2848834048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bf0e579b90 0x55bf0e58cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:34.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.354+0000 7f293be00640 0 client.16119 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:09:34.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.354+0000 7f295fc00740 1 -- 172.21.15.43:0/2848834048 >> 172.21.15.43:0/2848834048 conn(0x55bf0e576120 msgr2=0x55bf0e5751c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:34.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.354+0000 7f295fc00740 1 -- 172.21.15.43:0/2848834048 shutdown_connections 2024-09-19T05:09:34.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.354+0000 7f295fc00740 1 -- 172.21.15.43:0/2848834048 wait complete. 2024-09-19T05:09:34.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.354+0000 7f295fc00740 20 client.16119 trim_cache size 0 max 16384 2024-09-19T05:09:34.361 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:09:34.361 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-09-19T05:09:34.361 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-09-19T05:09:34.361 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:09:34.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.355+0000 7f295d000640 1 -- 172.21.15.43:0/88942253 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f293c0a33b0 msgr2=0x7f293c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:34.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.355+0000 7f295d000640 1 --2- 172.21.15.43:0/88942253 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f293c0a33b0 0x7f293c0a5870 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2950005fd0 tx=0x7f2950005cc0 comp rx=0 tx=0).stop 2024-09-19T05:09:34.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.355+0000 7f295d000640 1 -- 172.21.15.43:0/88942253 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2958103ff0 msgr2=0x7f295819a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:34.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.355+0000 7f295d000640 1 --2- 172.21.15.43:0/88942253 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2958103ff0 0x7f295819a620 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f294802ade0 tx=0x7f29480319f0 comp rx=0 tx=0).stop 2024-09-19T05:09:34.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.355+0000 7f295d000640 1 -- 172.21.15.43:0/88942253 shutdown_connections 2024-09-19T05:09:34.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.355+0000 7f295d000640 1 --2- 172.21.15.43:0/88942253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2958109120 0x7f295819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:34.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.355+0000 7f295d000640 1 --2- 172.21.15.43:0/88942253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29581049a0 0x7f295819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:34.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.355+0000 7f295d000640 1 --2- 172.21.15.43:0/88942253 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f293c0a33b0 0x7f293c0a5870 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:34.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.355+0000 7f295d000640 1 --2- 172.21.15.43:0/88942253 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2958103ff0 0x7f295819a620 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:34.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.355+0000 7f295d000640 1 -- 172.21.15.43:0/88942253 >> 172.21.15.43:0/88942253 conn(0x7f29580ffa90 msgr2=0x7f2958100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:34.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.356+0000 7f295d000640 1 -- 172.21.15.43:0/88942253 shutdown_connections 2024-09-19T05:09:34.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:34.356+0000 7f295d000640 1 -- 172.21.15.43:0/88942253 wait complete. 2024-09-19T05:09:35.233 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T05:09:35.233 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:09:35.615 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:09:35.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:35 smithi151 ceph-mon[35425]: pgmap v151: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:09:35.929 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:35 smithi043 ceph-mon[30800]: pgmap v151: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:09:35.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:35 smithi096 ceph-mon[35085]: pgmap v151: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:09:36.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.168+0000 7f46d2800640 1 -- 172.21.15.43:0/3105383784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46cc103ff0 msgr2=0x7f46cc1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:36.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.168+0000 7f46d2800640 1 --2- 172.21.15.43:0/3105383784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46cc103ff0 0x7f46cc1043d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f46bc009900 tx=0x7f46bc02a410 comp rx=0 tx=0).stop 2024-09-19T05:09:36.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.168+0000 7f46d2800640 1 -- 172.21.15.43:0/3105383784 shutdown_connections 2024-09-19T05:09:36.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.168+0000 7f46d2800640 1 --2- 172.21.15.43:0/3105383784 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46cc109120 0x7f46cc110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:36.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.168+0000 7f46d2800640 1 --2- 172.21.15.43:0/3105383784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46cc1049a0 0x7f46cc1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:36.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.168+0000 7f46d2800640 1 --2- 172.21.15.43:0/3105383784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46cc103ff0 0x7f46cc1043d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:36.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.168+0000 7f46d2800640 1 -- 172.21.15.43:0/3105383784 >> 172.21.15.43:0/3105383784 conn(0x7f46cc0ffa90 msgr2=0x7f46cc101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:36.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.168+0000 7f46d2800640 1 -- 172.21.15.43:0/3105383784 shutdown_connections 2024-09-19T05:09:36.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.168+0000 7f46d2800640 1 -- 172.21.15.43:0/3105383784 wait complete. 2024-09-19T05:09:36.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.169+0000 7f46d2800640 1 Processor -- start 2024-09-19T05:09:36.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.169+0000 7f46d2800640 1 -- start start 2024-09-19T05:09:36.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.169+0000 7f46d2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46cc103ff0 0x7f46cc19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:36.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.169+0000 7f46d2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46cc1049a0 0x7f46cc19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:36.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.169+0000 7f46d2800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46cc109120 0x7f46cc19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:36.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.169+0000 7f46d2800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc114120 con 0x7f46cc103ff0 2024-09-19T05:09:36.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.169+0000 7f46d2800640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc113fa0 con 0x7f46cc109120 2024-09-19T05:09:36.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.169+0000 7f46d2800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc1142a0 con 0x7f46cc1049a0 2024-09-19T05:09:36.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.170+0000 7f46cb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46cc103ff0 0x7f46cc19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:36.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.170+0000 7f46cb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46cc103ff0 0x7f46cc19a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40742/0 (socket says 172.21.15.43:40742) 2024-09-19T05:09:36.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.170+0000 7f46cb400640 1 -- 172.21.15.43:0/294937184 learned_addr learned my addr 172.21.15.43:0/294937184 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:36.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.170+0000 7f46cbe00640 1 --2- 172.21.15.43:0/294937184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46cc109120 0x7f46cc19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:36.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.170+0000 7f46cb400640 1 -- 172.21.15.43:0/294937184 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46cc1049a0 msgr2=0x7f46cc19ab20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T05:09:36.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.170+0000 7f46cb400640 1 --2- 172.21.15.43:0/294937184 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46cc1049a0 0x7f46cc19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:36.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.170+0000 7f46cb400640 1 -- 172.21.15.43:0/294937184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46cc109120 msgr2=0x7f46cc19eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:36.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.170+0000 7f46cb400640 1 --2- 172.21.15.43:0/294937184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46cc109120 0x7f46cc19eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:36.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.170+0000 7f46cb400640 1 -- 172.21.15.43:0/294937184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46cc19f590 con 0x7f46cc103ff0 2024-09-19T05:09:36.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.171+0000 7f46cb400640 1 --2- 172.21.15.43:0/294937184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46cc103ff0 0x7f46cc19a5e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f46bc006010 tx=0x7f46bc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:36.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.171+0000 7f46c8c00640 1 -- 172.21.15.43:0/294937184 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc03f030 con 0x7f46cc103ff0 2024-09-19T05:09:36.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.171+0000 7f46d2800640 1 -- 172.21.15.43:0/294937184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46cc19f820 con 0x7f46cc103ff0 2024-09-19T05:09:36.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.171+0000 7f46c8c00640 1 -- 172.21.15.43:0/294937184 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46bc003ce0 con 0x7f46cc103ff0 2024-09-19T05:09:36.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.171+0000 7f46d2800640 1 -- 172.21.15.43:0/294937184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46cc1a7160 con 0x7f46cc103ff0 2024-09-19T05:09:36.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.171+0000 7f46c8c00640 1 -- 172.21.15.43:0/294937184 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc049060 con 0x7f46cc103ff0 2024-09-19T05:09:36.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.172+0000 7f46d2800640 1 -- 172.21.15.43:0/294937184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46cc1043d0 con 0x7f46cc103ff0 2024-09-19T05:09:36.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.174+0000 7f46c8c00640 1 -- 172.21.15.43:0/294937184 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f46bc042030 con 0x7f46cc103ff0 2024-09-19T05:09:36.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.175+0000 7f46c8c00640 1 --2- 172.21.15.43:0/294937184 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f46b00a33b0 0x7f46b00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:36.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.176+0000 7f46c8c00640 1 -- 172.21.15.43:0/294937184 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f46bc0f01b0 con 0x7f46cc103ff0 2024-09-19T05:09:36.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.176+0000 7f46c2a00640 1 --2- 172.21.15.43:0/294937184 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f46b00a33b0 0x7f46b00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:36.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.176+0000 7f46c2a00640 1 --2- 172.21.15.43:0/294937184 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f46b00a33b0 0x7f46b00a5870 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f46b40058c0 tx=0x7f46b4005850 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:36.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.180+0000 7f46c8c00640 1 -- 172.21.15.43:0/294937184 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f46bc0bcb40 con 0x7f46cc103ff0 2024-09-19T05:09:36.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.401+0000 7f46d2800640 1 -- 172.21.15.43:0/294937184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46cc10c300 con 0x7f46cc103ff0 2024-09-19T05:09:36.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.402+0000 7f46c8c00640 1 -- 172.21.15.43:0/294937184 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11965 (secure 0 0 0) 0x7f46bc0bc440 con 0x7f46cc103ff0 2024-09-19T05:09:36.404 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:09:36.404 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":29,"btime":"2024-09-19T05:04:38:785407+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/773801169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":773801169},{"type":"v1","addr":"172.21.15.43:6835","nonce":773801169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1799137902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1799137902},{"type":"v1","addr":"172.21.15.43:6837","nonce":1799137902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:02:25.463041+0000","modified":"2024-09-19T05:04:38.039809+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24518,"mds_2":24511,"mds_3":24506,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14541":{"gid":14541,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/794881135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":794881135},{"type":"v1","addr":"172.21.15.43:6841","nonce":794881135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/2836453158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2836453158},{"type":"v1","addr":"172.21.15.151:6833","nonce":2836453158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/178785812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":178785812},{"type":"v1","addr":"172.21.15.96:6833","nonce":178785812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1671043820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1671043820},{"type":"v1","addr":"172.21.15.96:6835","nonce":1671043820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2972514328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2972514328},{"type":"v1","addr":"172.21.15.151:6835","nonce":2972514328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.151:6837/3960378010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3960378010},{"type":"v1","addr":"172.21.15.151:6837","nonce":3960378010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6837/3978388286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3978388286},{"type":"v1","addr":"172.21.15.96:6837","nonce":3978388286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.151:6839/3794104313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3794104313},{"type":"v1","addr":"172.21.15.151:6839","nonce":3794104313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.43:6839/3404555030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3404555030},{"type":"v1","addr":"172.21.15.43:6839","nonce":3404555030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/605527621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":605527621},{"type":"v1","addr":"172.21.15.96:6839","nonce":605527621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24518,24511,24506,24493]},"id":1}]} 2024-09-19T05:09:36.405 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 29 2024-09-19T05:09:36.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.404+0000 7f46d2800640 1 -- 172.21.15.43:0/294937184 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f46b00a33b0 msgr2=0x7f46b00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:36.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.404+0000 7f46d2800640 1 --2- 172.21.15.43:0/294937184 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f46b00a33b0 0x7f46b00a5870 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f46b40058c0 tx=0x7f46b4005850 comp rx=0 tx=0).stop 2024-09-19T05:09:36.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.404+0000 7f46d2800640 1 -- 172.21.15.43:0/294937184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46cc103ff0 msgr2=0x7f46cc19a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:36.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.404+0000 7f46d2800640 1 --2- 172.21.15.43:0/294937184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46cc103ff0 0x7f46cc19a5e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f46bc006010 tx=0x7f46bc0042f0 comp rx=0 tx=0).stop 2024-09-19T05:09:36.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.404+0000 7f46d2800640 1 -- 172.21.15.43:0/294937184 shutdown_connections 2024-09-19T05:09:36.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.404+0000 7f46d2800640 1 --2- 172.21.15.43:0/294937184 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f46b00a33b0 0x7f46b00a5870 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:36.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.404+0000 7f46d2800640 1 --2- 172.21.15.43:0/294937184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46cc109120 0x7f46cc19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:36.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.404+0000 7f46d2800640 1 --2- 172.21.15.43:0/294937184 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46cc1049a0 0x7f46cc19ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:36.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.404+0000 7f46d2800640 1 --2- 172.21.15.43:0/294937184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46cc103ff0 0x7f46cc19a5e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:36.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.404+0000 7f46d2800640 1 -- 172.21.15.43:0/294937184 >> 172.21.15.43:0/294937184 conn(0x7f46cc0ffa90 msgr2=0x7f46cc100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:36.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.404+0000 7f46d2800640 1 -- 172.21.15.43:0/294937184 shutdown_connections 2024-09-19T05:09:36.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:36.404+0000 7f46d2800640 1 -- 172.21.15.43:0/294937184 wait complete. 2024-09-19T05:09:36.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:36 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/294937184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:09:36.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:36 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/294937184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:09:36.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:36 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/294937184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:09:37.175 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T05:09:37.557 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:09:37.871 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:37 smithi043 ceph-mon[30800]: pgmap v152: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:09:37.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:37 smithi151 ceph-mon[35425]: pgmap v152: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:09:37.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:37 smithi096 ceph-mon[35085]: pgmap v152: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:09:38.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.121+0000 7f689ea00640 1 -- 172.21.15.43:0/3986171658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68981049a0 msgr2=0x7f68981089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:38.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.121+0000 7f689ea00640 1 --2- 172.21.15.43:0/3986171658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68981049a0 0x7f68981089f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6880009900 tx=0x7f688002a410 comp rx=0 tx=0).stop 2024-09-19T05:09:38.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.121+0000 7f689ea00640 1 -- 172.21.15.43:0/3986171658 shutdown_connections 2024-09-19T05:09:38.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.121+0000 7f689ea00640 1 --2- 172.21.15.43:0/3986171658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6898109120 0x7f6898110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:38.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.121+0000 7f689ea00640 1 --2- 172.21.15.43:0/3986171658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68981049a0 0x7f68981089f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:38.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.121+0000 7f689ea00640 1 --2- 172.21.15.43:0/3986171658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6898103ff0 0x7f68981043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:38.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.121+0000 7f689ea00640 1 -- 172.21.15.43:0/3986171658 >> 172.21.15.43:0/3986171658 conn(0x7f68980ffa90 msgr2=0x7f6898101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:38.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.121+0000 7f689ea00640 1 -- 172.21.15.43:0/3986171658 shutdown_connections 2024-09-19T05:09:38.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.121+0000 7f689ea00640 1 -- 172.21.15.43:0/3986171658 wait complete. 2024-09-19T05:09:38.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.122+0000 7f689ea00640 1 Processor -- start 2024-09-19T05:09:38.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.122+0000 7f689ea00640 1 -- start start 2024-09-19T05:09:38.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.123+0000 7f689ea00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6898103ff0 0x7f6898076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:38.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.123+0000 7f689ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68981049a0 0x7f6898076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:38.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.123+0000 7f689ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6898109120 0x7f6898077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:38.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.123+0000 7f689ea00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6898114190 con 0x7f68981049a0 2024-09-19T05:09:38.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.123+0000 7f689ea00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6898114010 con 0x7f6898103ff0 2024-09-19T05:09:38.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.123+0000 7f689ea00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6898114310 con 0x7f6898109120 2024-09-19T05:09:38.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.123+0000 7f6897e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6898103ff0 0x7f6898076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:38.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.123+0000 7f688f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68981049a0 0x7f6898076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:38.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.123+0000 7f6897e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6898103ff0 0x7f6898076700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:52422/0 (socket says 172.21.15.43:52422) 2024-09-19T05:09:38.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.123+0000 7f6897e00640 1 -- 172.21.15.43:0/1505351319 learned_addr learned my addr 172.21.15.43:0/1505351319 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:38.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.123+0000 7f689ca00640 1 --2- 172.21.15.43:0/1505351319 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6898109120 0x7f6898077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:38.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.124+0000 7f6897e00640 1 -- 172.21.15.43:0/1505351319 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6898109120 msgr2=0x7f6898077180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:38.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.124+0000 7f6897e00640 1 --2- 172.21.15.43:0/1505351319 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6898109120 0x7f6898077180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:38.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.124+0000 7f6897e00640 1 -- 172.21.15.43:0/1505351319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68981049a0 msgr2=0x7f6898076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:38.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.124+0000 7f6897e00640 1 --2- 172.21.15.43:0/1505351319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68981049a0 0x7f6898076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:38.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.124+0000 7f6897e00640 1 -- 172.21.15.43:0/1505351319 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68981aacd0 con 0x7f6898103ff0 2024-09-19T05:09:38.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.124+0000 7f6897e00640 1 --2- 172.21.15.43:0/1505351319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6898103ff0 0x7f6898076700 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f688800ea50 tx=0x7f688800ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:38.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.124+0000 7f6895600640 1 -- 172.21.15.43:0/1505351319 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688800cd60 con 0x7f6898103ff0 2024-09-19T05:09:38.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.124+0000 7f6895600640 1 -- 172.21.15.43:0/1505351319 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6888005500 con 0x7f6898103ff0 2024-09-19T05:09:38.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.124+0000 7f689ea00640 1 -- 172.21.15.43:0/1505351319 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68981aafc0 con 0x7f6898103ff0 2024-09-19T05:09:38.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.124+0000 7f6895600640 1 -- 172.21.15.43:0/1505351319 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6888010690 con 0x7f6898103ff0 2024-09-19T05:09:38.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.124+0000 7f689ea00640 1 -- 172.21.15.43:0/1505351319 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68981ab420 con 0x7f6898103ff0 2024-09-19T05:09:38.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.127+0000 7f68a1618740 1 Processor -- start 2024-09-19T05:09:38.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.128+0000 7f6895600640 1 -- 172.21.15.43:0/1505351319 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6888004f30 con 0x7f6898103ff0 2024-09-19T05:09:38.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.128+0000 7f68a1618740 1 -- start start 2024-09-19T05:09:38.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.128+0000 7f68a1618740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557fbe571c70 0x557fbe572050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:38.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.128+0000 7f68a1618740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fbe572590 0x557fbe57c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:38.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.128+0000 7f68a1618740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557fbe57ce50 0x557fbe57f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:38.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.128+0000 7f68a1618740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557fbe4d83e0 con 0x557fbe572590 2024-09-19T05:09:38.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.128+0000 7f68a1618740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557fbe47fd80 con 0x557fbe57ce50 2024-09-19T05:09:38.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.128+0000 7f68a1618740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557fbe4d43a0 con 0x557fbe571c70 2024-09-19T05:09:38.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.128+0000 7f6897e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fbe572590 0x557fbe57c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:38.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.128+0000 7f6897e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fbe572590 0x557fbe57c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40774/0 (socket says 172.21.15.43:40774) 2024-09-19T05:09:38.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.128+0000 7f6897e00640 1 -- 172.21.15.43:0/306767980 learned_addr learned my addr 172.21.15.43:0/306767980 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:38.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.129+0000 7f689ca00640 1 --2- 172.21.15.43:0/306767980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557fbe571c70 0x557fbe572050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:38.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.129+0000 7f688f400640 1 --2- 172.21.15.43:0/306767980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557fbe57ce50 0x557fbe57f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:38.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.129+0000 7f6897e00640 1 -- 172.21.15.43:0/306767980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557fbe571c70 msgr2=0x557fbe572050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:38.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.129+0000 7f6897e00640 1 --2- 172.21.15.43:0/306767980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557fbe571c70 0x557fbe572050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:38.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.129+0000 7f6897e00640 1 -- 172.21.15.43:0/306767980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557fbe57ce50 msgr2=0x557fbe57f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:38.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.129+0000 7f6897e00640 1 --2- 172.21.15.43:0/306767980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557fbe57ce50 0x557fbe57f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:38.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.129+0000 7f6897e00640 1 -- 172.21.15.43:0/306767980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fbe49eae0 con 0x557fbe572590 2024-09-19T05:09:38.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.129+0000 7f689ca00640 1 --2- 172.21.15.43:0/306767980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557fbe571c70 0x557fbe572050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:38.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.129+0000 7f6897e00640 1 --2- 172.21.15.43:0/306767980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fbe572590 0x557fbe57c910 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f68880cd630 tx=0x7f68880d80e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc72460572dfdc94 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:38.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.130+0000 7f6895600640 1 --2- 172.21.15.43:0/1505351319 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f687c0a3430 0x7f687c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:38.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.130+0000 7f688d200640 1 -- 172.21.15.43:0/306767980 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68880e3070 con 0x557fbe572590 2024-09-19T05:09:38.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.130+0000 7f688d200640 1 -- 172.21.15.43:0/306767980 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f688809c980 con 0x557fbe572590 2024-09-19T05:09:38.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.130+0000 7f688d200640 1 -- 172.21.15.43:0/306767980 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688806f360 con 0x557fbe572590 2024-09-19T05:09:38.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.130+0000 7f6895600640 1 -- 172.21.15.43:0/1505351319 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f68880cfff0 con 0x7f6898103ff0 2024-09-19T05:09:38.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.130+0000 7f689ca00640 1 --2- 172.21.15.43:0/1505351319 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f687c0a3430 0x7f687c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:38.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.130+0000 7f68a1618740 1 -- 172.21.15.43:0/306767980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fbe572590 msgr2=0x557fbe57c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:38.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.130+0000 7f68a1618740 1 --2- 172.21.15.43:0/306767980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fbe572590 0x557fbe57c910 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f68880cd630 tx=0x7f68880d80e0 comp rx=0 tx=0).stop 2024-09-19T05:09:38.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.131+0000 7f68a1618740 1 -- 172.21.15.43:0/306767980 shutdown_connections 2024-09-19T05:09:38.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.131+0000 7f68a1618740 1 --2- 172.21.15.43:0/306767980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557fbe57ce50 0x557fbe57f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:38.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.131+0000 7f68a1618740 1 --2- 172.21.15.43:0/306767980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fbe572590 0x557fbe57c910 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:38.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.131+0000 7f68a1618740 1 --2- 172.21.15.43:0/306767980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557fbe571c70 0x557fbe572050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:38.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.131+0000 7f68a1618740 1 -- 172.21.15.43:0/306767980 >> 172.21.15.43:0/306767980 conn(0x557fbe56e200 msgr2=0x557fbe56e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:38.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.131+0000 7f68a1618740 1 -- 172.21.15.43:0/306767980 shutdown_connections 2024-09-19T05:09:38.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.131+0000 7f689ca00640 1 --2- 172.21.15.43:0/1505351319 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f687c0a3430 0x7f687c0a58f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6890005fd0 tx=0x7f68900096a0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:38.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.131+0000 7f68a1618740 1 -- 172.21.15.43:0/306767980 wait complete. 2024-09-19T05:09:38.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.131+0000 7f68a1618740 1 -- start start 2024-09-19T05:09:38.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.132+0000 7f68a1618740 1 Processor -- start 2024-09-19T05:09:38.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.132+0000 7f68a1618740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557fbe571c70 0x557fbe579cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:38.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.132+0000 7f68a1618740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557fbe572590 0x557fbe57a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:38.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.132+0000 7f68a1618740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fbe57ce50 0x557fbe57a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:38.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.132+0000 7f68a1618740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557fbe57fa20 con 0x557fbe57ce50 2024-09-19T05:09:38.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.132+0000 7f68a1618740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557fbe4930f0 con 0x557fbe572590 2024-09-19T05:09:38.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.132+0000 7f68a1618740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557fbe57fba0 con 0x557fbe571c70 2024-09-19T05:09:38.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.132+0000 7f68a1618740 10 client.0 _ll_register_callbacks cb 0x557fbe47abd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:38.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.132+0000 7f689ca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fbe57ce50 0x557fbe57a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:38.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.132+0000 7f689ca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fbe57ce50 0x557fbe57a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40776/0 (socket says 172.21.15.43:40776) 2024-09-19T05:09:38.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.132+0000 7f689ca00640 1 -- 172.21.15.43:0/1658652502 learned_addr learned my addr 172.21.15.43:0/1658652502 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:38.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.132+0000 7f6897e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557fbe571c70 0x557fbe579cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:38.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.133+0000 7f688f400640 1 --2- 172.21.15.43:0/1658652502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557fbe572590 0x557fbe57a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:38.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.133+0000 7f6897e00640 1 -- 172.21.15.43:0/1658652502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557fbe572590 msgr2=0x557fbe57a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:38.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.133+0000 7f6897e00640 1 --2- 172.21.15.43:0/1658652502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557fbe572590 0x557fbe57a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:38.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.133+0000 7f6897e00640 1 -- 172.21.15.43:0/1658652502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fbe57ce50 msgr2=0x557fbe57a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:38.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.133+0000 7f6897e00640 1 --2- 172.21.15.43:0/1658652502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fbe57ce50 0x557fbe57a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:38.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.133+0000 7f6897e00640 1 -- 172.21.15.43:0/1658652502 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fbe2493c0 con 0x557fbe571c70 2024-09-19T05:09:38.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.133+0000 7f689ca00640 1 --2- 172.21.15.43:0/1658652502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fbe57ce50 0x557fbe57a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:38.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.133+0000 7f6897e00640 1 --2- 172.21.15.43:0/1658652502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557fbe571c70 0x557fbe579cf0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f688800ea20 tx=0x7f68880d0270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15454d231c818c96 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:38.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.133+0000 7f6885600640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:09:38.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.133+0000 7f6885600640 1 -- 172.21.15.43:0/1658652502 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688806e4a0 con 0x557fbe571c70 2024-09-19T05:09:38.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.133+0000 7f6885600640 1 -- 172.21.15.43:0/1658652502 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f688806f360 con 0x557fbe571c70 2024-09-19T05:09:38.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.133+0000 7f68a1618740 1 -- 172.21.15.43:0/1658652502 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f688809c980 con 0x557fbe571c70 2024-09-19T05:09:38.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.134+0000 7f6885600640 1 -- 172.21.15.43:0/1658652502 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f688806e6e0 con 0x557fbe571c70 2024-09-19T05:09:38.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.134+0000 7f68a1618740 10 client.26116 fetch_fsmap learned FSMap version 29 2024-09-19T05:09:38.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.134+0000 7f68a1618740 1 -- 172.21.15.43:0/1658652502 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557fbe57fa20 con 0x557fbe571c70 2024-09-19T05:09:38.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.134+0000 7f6885600640 1 -- 172.21.15.43:0/1658652502 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f688809c980 con 0x557fbe571c70 2024-09-19T05:09:38.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.134+0000 7f6885600640 1 -- 172.21.15.43:0/1658652502 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f68880cd970 con 0x557fbe571c70 2024-09-19T05:09:38.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.134+0000 7f68a1618740 10 client.26116 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:09:38.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.134+0000 7f68a1618740 10 client.26116 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-19T05:09:38.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.134+0000 7f68a1618740 20 client.26116 populate_metadata read hostname 'smithi043' 2024-09-19T05:09:38.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.134+0000 7f68a1618740 1 --2- 172.21.15.43:0/1658652502 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x557fbe58cb90 0x557fbe58ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:38.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.134+0000 7f68a1618740 4 client.26116 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:09:38.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.134+0000 7f68a1618740 1 -- 172.21.15.43:0/1658652502 --> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557fbe4e5950 con 0x557fbe58cb90 2024-09-19T05:09:38.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.135+0000 7f688f400640 1 --2- 172.21.15.43:0/1658652502 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x557fbe58cb90 0x557fbe58ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:38.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.135+0000 7f688f400640 1 --2- 172.21.15.43:0/1658652502 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x557fbe58cb90 0x557fbe58ef80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=684fe2f0083a1c9e server_cookie=4a6e115fd6b678b2 in_seq=0 out_seq=0 2024-09-19T05:09:38.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.135+0000 7f6885600640 10 client.26116 ms_handle_connect on v2:172.21.15.151:6836/3960378010 2024-09-19T05:09:38.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.136+0000 7f6885600640 1 -- 172.21.15.43:0/1658652502 <== mds.4 v2:172.21.15.151:6836/3960378010 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557fbe4e5950 con 0x557fbe58cb90 2024-09-19T05:09:38.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.136+0000 7f6885600640 10 client.26116 handle_command_reply: tid=0 2024-09-19T05:09:38.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.136+0000 7f68a1618740 1 client.26116 shutdown 2024-09-19T05:09:38.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.136+0000 7f68a1618740 1 -- 172.21.15.43:0/1658652502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557fbe571c70 msgr2=0x557fbe579cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:38.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.136+0000 7f68a1618740 1 --2- 172.21.15.43:0/1658652502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557fbe571c70 0x557fbe579cf0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f688800ea20 tx=0x7f68880d0270 comp rx=0 tx=0).stop 2024-09-19T05:09:38.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.136+0000 7f68a1618740 1 -- 172.21.15.43:0/1658652502 shutdown_connections 2024-09-19T05:09:38.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.136+0000 7f68a1618740 1 --2- 172.21.15.43:0/1658652502 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x557fbe58cb90 0x557fbe58ef80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:38.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.137+0000 7f68a1618740 1 --2- 172.21.15.43:0/1658652502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fbe57ce50 0x557fbe57a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:38.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.137+0000 7f6885600640 0 client.26116 ms_handle_reset on v2:172.21.15.151:6836/3960378010 2024-09-19T05:09:38.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.137+0000 7f68a1618740 1 --2- 172.21.15.43:0/1658652502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557fbe572590 0x557fbe57a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:38.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.137+0000 7f68a1618740 1 --2- 172.21.15.43:0/1658652502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557fbe571c70 0x557fbe579cf0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:38.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.137+0000 7f68a1618740 1 -- 172.21.15.43:0/1658652502 >> 172.21.15.43:0/1658652502 conn(0x557fbe56e200 msgr2=0x557fbe56d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:38.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.137+0000 7f68a1618740 1 -- 172.21.15.43:0/1658652502 shutdown_connections 2024-09-19T05:09:38.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.137+0000 7f68a1618740 1 -- 172.21.15.43:0/1658652502 wait complete. 2024-09-19T05:09:38.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.137+0000 7f68a1618740 20 client.26116 trim_cache size 0 max 16384 2024-09-19T05:09:38.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.151+0000 7f68a1618740 1 Processor -- start 2024-09-19T05:09:38.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.151+0000 7f68a1618740 1 -- start start 2024-09-19T05:09:38.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.151+0000 7f68a1618740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557fbe571c70 0x557fbe5945b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:38.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.151+0000 7f68a1618740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fbe572590 0x557fbe594af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:38.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.151+0000 7f68a1618740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557fbe57ce50 0x557fbe57b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:38.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.151+0000 7f68a1618740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f68981b3eb0 con 0x557fbe572590 2024-09-19T05:09:38.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.151+0000 7f68a1618740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f68981b59f0 con 0x557fbe571c70 2024-09-19T05:09:38.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.151+0000 7f68a1618740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f68981b4580 con 0x557fbe57ce50 2024-09-19T05:09:38.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.151+0000 7f688f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fbe572590 0x557fbe594af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:38.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.151+0000 7f6897e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557fbe571c70 0x557fbe5945b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:38.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.151+0000 7f688f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fbe572590 0x557fbe594af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40790/0 (socket says 172.21.15.43:40790) 2024-09-19T05:09:38.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.151+0000 7f688f400640 1 -- 172.21.15.43:0/2969669446 learned_addr learned my addr 172.21.15.43:0/2969669446 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:38.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.152+0000 7f689ca00640 1 --2- 172.21.15.43:0/2969669446 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557fbe57ce50 0x557fbe57b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:38.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.152+0000 7f688f400640 1 -- 172.21.15.43:0/2969669446 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557fbe57ce50 msgr2=0x557fbe57b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:38.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.152+0000 7f688f400640 1 --2- 172.21.15.43:0/2969669446 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557fbe57ce50 0x557fbe57b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:38.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.152+0000 7f688f400640 1 -- 172.21.15.43:0/2969669446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557fbe571c70 msgr2=0x557fbe5945b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:38.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.152+0000 7f688f400640 1 --2- 172.21.15.43:0/2969669446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557fbe571c70 0x557fbe5945b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:38.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.152+0000 7f688f400640 1 -- 172.21.15.43:0/2969669446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fbe569770 con 0x557fbe572590 2024-09-19T05:09:38.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.152+0000 7f6897e00640 1 --2- 172.21.15.43:0/2969669446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557fbe571c70 0x557fbe5945b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:38.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.152+0000 7f688f400640 1 --2- 172.21.15.43:0/2969669446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fbe572590 0x557fbe594af0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6880005c40 tx=0x7f6880005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9decfbc5eb5986c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:38.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.152+0000 7f6887e00640 1 -- 172.21.15.43:0/2969669446 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6880035040 con 0x557fbe572590 2024-09-19T05:09:38.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.152+0000 7f6887e00640 1 -- 172.21.15.43:0/2969669446 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f688004b070 con 0x557fbe572590 2024-09-19T05:09:38.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.153+0000 7f6887e00640 1 -- 172.21.15.43:0/2969669446 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6880031810 con 0x557fbe572590 2024-09-19T05:09:38.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.153+0000 7f68a1618740 1 -- 172.21.15.43:0/2969669446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fbe572590 msgr2=0x557fbe594af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:38.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.153+0000 7f68a1618740 1 --2- 172.21.15.43:0/2969669446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fbe572590 0x557fbe594af0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6880005c40 tx=0x7f6880005b50 comp rx=0 tx=0).stop 2024-09-19T05:09:38.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.153+0000 7f68a1618740 1 -- 172.21.15.43:0/2969669446 shutdown_connections 2024-09-19T05:09:38.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.153+0000 7f68a1618740 1 --2- 172.21.15.43:0/2969669446 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557fbe57ce50 0x557fbe57b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:38.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.153+0000 7f68a1618740 1 --2- 172.21.15.43:0/2969669446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fbe572590 0x557fbe594af0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:38.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.153+0000 7f68a1618740 1 --2- 172.21.15.43:0/2969669446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557fbe571c70 0x557fbe5945b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:38.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.153+0000 7f68a1618740 1 -- 172.21.15.43:0/2969669446 >> 172.21.15.43:0/2969669446 conn(0x557fbe56e200 msgr2=0x557fbe578de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:38.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.153+0000 7f68a1618740 1 -- 172.21.15.43:0/2969669446 shutdown_connections 2024-09-19T05:09:38.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.153+0000 7f68a1618740 1 -- 172.21.15.43:0/2969669446 wait complete. 2024-09-19T05:09:38.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.154+0000 7f68a1618740 1 -- start start 2024-09-19T05:09:38.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.154+0000 7f68a1618740 1 Processor -- start 2024-09-19T05:09:38.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.154+0000 7f68a1618740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fbe571c70 0x557fbe580530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:38.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.154+0000 7f68a1618740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557fbe572590 0x557fbe580a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:38.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.155+0000 7f68a1618740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557fbe57ce50 0x557fbe5ef540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:38.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.155+0000 7f68a1618740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f68981b5320 con 0x557fbe571c70 2024-09-19T05:09:38.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.155+0000 7f68a1618740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f68981b4c50 con 0x557fbe572590 2024-09-19T05:09:38.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.155+0000 7f68a1618740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f68981b1e90 con 0x557fbe57ce50 2024-09-19T05:09:38.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.155+0000 7f68a1618740 10 client.0 _ll_register_callbacks cb 0x7f68880e1070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:38.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.155+0000 7f6897e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fbe571c70 0x557fbe580530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:38.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.155+0000 7f6897e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fbe571c70 0x557fbe580530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40802/0 (socket says 172.21.15.43:40802) 2024-09-19T05:09:38.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.155+0000 7f6897e00640 1 -- 172.21.15.43:0/576252698 learned_addr learned my addr 172.21.15.43:0/576252698 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:38.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.155+0000 7f689ca00640 1 --2- 172.21.15.43:0/576252698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557fbe57ce50 0x557fbe5ef540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:38.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.155+0000 7f688f400640 1 --2- 172.21.15.43:0/576252698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557fbe572590 0x557fbe580a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:38.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.155+0000 7f6897e00640 1 -- 172.21.15.43:0/576252698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557fbe57ce50 msgr2=0x557fbe5ef540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:38.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.155+0000 7f6897e00640 1 --2- 172.21.15.43:0/576252698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557fbe57ce50 0x557fbe5ef540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:38.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.155+0000 7f6897e00640 1 -- 172.21.15.43:0/576252698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557fbe572590 msgr2=0x557fbe580a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:38.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.155+0000 7f6897e00640 1 --2- 172.21.15.43:0/576252698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557fbe572590 0x557fbe580a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:38.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.155+0000 7f6897e00640 1 -- 172.21.15.43:0/576252698 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fbe5efbe0 con 0x557fbe571c70 2024-09-19T05:09:38.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.156+0000 7f6897e00640 1 --2- 172.21.15.43:0/576252698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fbe571c70 0x557fbe580530 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f688809be30 tx=0x7f688806e880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e722e07fa9029aa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:38.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.156+0000 7f6885600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:09:38.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.156+0000 7f6885600640 1 -- 172.21.15.43:0/576252698 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688806e920 con 0x557fbe571c70 2024-09-19T05:09:38.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.156+0000 7f6885600640 1 -- 172.21.15.43:0/576252698 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f688806f2c0 con 0x557fbe571c70 2024-09-19T05:09:38.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.156+0000 7f68a1618740 1 -- 172.21.15.43:0/576252698 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f688004b070 con 0x557fbe571c70 2024-09-19T05:09:38.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.156+0000 7f6885600640 1 -- 172.21.15.43:0/576252698 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f688806eb40 con 0x557fbe571c70 2024-09-19T05:09:38.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.156+0000 7f68a1618740 10 client.16143 fetch_fsmap learned FSMap version 29 2024-09-19T05:09:38.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.156+0000 7f68a1618740 1 -- 172.21.15.43:0/576252698 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557fbe5f6f50 con 0x557fbe571c70 2024-09-19T05:09:38.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.156+0000 7f6885600640 1 -- 172.21.15.43:0/576252698 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f688004b070 con 0x557fbe571c70 2024-09-19T05:09:38.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.156+0000 7f6885600640 1 -- 172.21.15.43:0/576252698 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f688806e2c0 con 0x557fbe571c70 2024-09-19T05:09:38.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.157+0000 7f68a1618740 10 client.16143 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:09:38.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.157+0000 7f68a1618740 10 client.16143 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-19T05:09:38.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.157+0000 7f68a1618740 20 client.16143 populate_metadata read hostname 'smithi043' 2024-09-19T05:09:38.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.157+0000 7f68a1618740 1 --2- 172.21.15.43:0/576252698 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x557fbe58cb90 0x557fbe5f93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:38.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.157+0000 7f68a1618740 4 client.16143 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:09:38.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.157+0000 7f68a1618740 1 -- 172.21.15.43:0/576252698 --> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557fbe4dfe60 con 0x557fbe58cb90 2024-09-19T05:09:38.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.157+0000 7f688f400640 1 --2- 172.21.15.43:0/576252698 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x557fbe58cb90 0x557fbe5f93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:38.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.157+0000 7f688f400640 1 --2- 172.21.15.43:0/576252698 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x557fbe58cb90 0x557fbe5f93a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=82db98863288fa81 server_cookie=40ed6adac24ffbdb in_seq=0 out_seq=0 2024-09-19T05:09:38.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.157+0000 7f6885600640 10 client.16143 ms_handle_connect on v2:172.21.15.151:6836/3960378010 2024-09-19T05:09:38.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.158+0000 7f6885600640 1 -- 172.21.15.43:0/576252698 <== mds.4 v2:172.21.15.151:6836/3960378010 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557fbe4dfe60 con 0x557fbe58cb90 2024-09-19T05:09:38.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.158+0000 7f6885600640 10 client.16143 handle_command_reply: tid=0 2024-09-19T05:09:38.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.158+0000 7f68a1618740 1 client.16143 shutdown 2024-09-19T05:09:38.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.158+0000 7f68a1618740 1 -- 172.21.15.43:0/576252698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fbe571c70 msgr2=0x557fbe580530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:38.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.158+0000 7f68a1618740 1 --2- 172.21.15.43:0/576252698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fbe571c70 0x557fbe580530 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f688809be30 tx=0x7f688806e880 comp rx=0 tx=0).stop 2024-09-19T05:09:38.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.158+0000 7f68a1618740 1 -- 172.21.15.43:0/576252698 shutdown_connections 2024-09-19T05:09:38.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.158+0000 7f68a1618740 1 --2- 172.21.15.43:0/576252698 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x557fbe58cb90 0x557fbe5f93a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:38.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.158+0000 7f68a1618740 1 --2- 172.21.15.43:0/576252698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557fbe57ce50 0x557fbe5ef540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:38.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.158+0000 7f68a1618740 1 --2- 172.21.15.43:0/576252698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557fbe572590 0x557fbe580a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:38.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.158+0000 7f6885600640 0 client.16143 ms_handle_reset on v2:172.21.15.151:6836/3960378010 2024-09-19T05:09:38.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.158+0000 7f68a1618740 1 --2- 172.21.15.43:0/576252698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fbe571c70 0x557fbe580530 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:38.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.158+0000 7f68a1618740 1 -- 172.21.15.43:0/576252698 >> 172.21.15.43:0/576252698 conn(0x557fbe56e200 msgr2=0x557fbe56cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:38.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.158+0000 7f68a1618740 1 -- 172.21.15.43:0/576252698 shutdown_connections 2024-09-19T05:09:38.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.158+0000 7f68a1618740 1 -- 172.21.15.43:0/576252698 wait complete. 2024-09-19T05:09:38.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.158+0000 7f68a1618740 20 client.16143 trim_cache size 0 max 16384 2024-09-19T05:09:38.165 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:09:38.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.159+0000 7f689ea00640 1 -- 172.21.15.43:0/1505351319 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f687c0a3430 msgr2=0x7f687c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:38.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.159+0000 7f689ea00640 1 --2- 172.21.15.43:0/1505351319 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f687c0a3430 0x7f687c0a58f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6890005fd0 tx=0x7f68900096a0 comp rx=0 tx=0).stop 2024-09-19T05:09:38.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.159+0000 7f689ea00640 1 -- 172.21.15.43:0/1505351319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6898103ff0 msgr2=0x7f6898076700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:38.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.159+0000 7f689ea00640 1 --2- 172.21.15.43:0/1505351319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6898103ff0 0x7f6898076700 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f688800ea50 tx=0x7f688800ee50 comp rx=0 tx=0).stop 2024-09-19T05:09:38.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.159+0000 7f689ea00640 1 -- 172.21.15.43:0/1505351319 shutdown_connections 2024-09-19T05:09:38.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.159+0000 7f689ea00640 1 --2- 172.21.15.43:0/1505351319 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6898109120 0x7f6898077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:38.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.159+0000 7f689ea00640 1 --2- 172.21.15.43:0/1505351319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68981049a0 0x7f6898076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:38.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.159+0000 7f689ea00640 1 --2- 172.21.15.43:0/1505351319 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f687c0a3430 0x7f687c0a58f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:38.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.159+0000 7f689ea00640 1 --2- 172.21.15.43:0/1505351319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6898103ff0 0x7f6898076700 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:38.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.159+0000 7f689ea00640 1 -- 172.21.15.43:0/1505351319 >> 172.21.15.43:0/1505351319 conn(0x7f68980ffa90 msgr2=0x7f6898100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:38.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.159+0000 7f689ea00640 1 -- 172.21.15.43:0/1505351319 shutdown_connections 2024-09-19T05:09:38.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:38.159+0000 7f689ea00640 1 -- 172.21.15.43:0/1505351319 wait complete. 2024-09-19T05:09:38.968 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T05:09:39.354 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:09:39.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:39 smithi043 ceph-mon[30800]: pgmap v153: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:09:39.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:39 smithi151 ceph-mon[35425]: pgmap v153: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:09:39.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.922+0000 7f06e7800640 1 -- 172.21.15.43:0/3947864245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06e0109140 msgr2=0x7f06e0110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:39.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.922+0000 7f06e7800640 1 --2- 172.21.15.43:0/3947864245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06e0109140 0x7f06e0110cc0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f06d8009ca0 tx=0x7f06d802a8c0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.922+0000 7f06e7800640 1 -- 172.21.15.43:0/3947864245 shutdown_connections 2024-09-19T05:09:39.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.922+0000 7f06e7800640 1 --2- 172.21.15.43:0/3947864245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06e0109140 0x7f06e0110cc0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.922+0000 7f06e7800640 1 --2- 172.21.15.43:0/3947864245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06e01049c0 0x7f06e0108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.922+0000 7f06e7800640 1 --2- 172.21.15.43:0/3947864245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06e0104010 0x7f06e01043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.922+0000 7f06e7800640 1 -- 172.21.15.43:0/3947864245 >> 172.21.15.43:0/3947864245 conn(0x7f06e00ffa50 msgr2=0x7f06e0101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:39.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.922+0000 7f06e7800640 1 -- 172.21.15.43:0/3947864245 shutdown_connections 2024-09-19T05:09:39.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.922+0000 7f06e7800640 1 -- 172.21.15.43:0/3947864245 wait complete. 2024-09-19T05:09:39.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.923+0000 7f06e7800640 1 Processor -- start 2024-09-19T05:09:39.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.923+0000 7f06e7800640 1 -- start start 2024-09-19T05:09:39.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.924+0000 7f06e7800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06e0104010 0x7f06e019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:39.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.924+0000 7f06e7800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06e01049c0 0x7f06e019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:39.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.924+0000 7f06e7800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06e0109140 0x7f06e019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:39.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.924+0000 7f06e7800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f06e01141a0 con 0x7f06e0109140 2024-09-19T05:09:39.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.924+0000 7f06e7800640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f06e0114020 con 0x7f06e01049c0 2024-09-19T05:09:39.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.924+0000 7f06e7800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f06e0114320 con 0x7f06e0104010 2024-09-19T05:09:39.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.924+0000 7f06e5800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06e0109140 0x7f06e019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:39.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.924+0000 7f06e4e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06e0104010 0x7f06e019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:39.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.924+0000 7f06e5800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06e0109140 0x7f06e019ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40822/0 (socket says 172.21.15.43:40822) 2024-09-19T05:09:39.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.924+0000 7f06e5800640 1 -- 172.21.15.43:0/3775733096 learned_addr learned my addr 172.21.15.43:0/3775733096 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:39.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.924+0000 7f06d7e00640 1 --2- 172.21.15.43:0/3775733096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06e01049c0 0x7f06e019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:39.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.924+0000 7f06e5800640 1 -- 172.21.15.43:0/3775733096 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06e0104010 msgr2=0x7f06e019a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:39.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.924+0000 7f06e5800640 1 --2- 172.21.15.43:0/3775733096 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06e0104010 0x7f06e019a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.924+0000 7f06e5800640 1 -- 172.21.15.43:0/3775733096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06e01049c0 msgr2=0x7f06e019abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:39.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.924+0000 7f06e5800640 1 --2- 172.21.15.43:0/3775733096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06e01049c0 0x7f06e019abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.924+0000 7f06e5800640 1 -- 172.21.15.43:0/3775733096 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06e019f650 con 0x7f06e0109140 2024-09-19T05:09:39.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.925+0000 7f06e4e00640 1 --2- 172.21.15.43:0/3775733096 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06e0104010 0x7f06e019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:39.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.925+0000 7f06e5800640 1 --2- 172.21.15.43:0/3775733096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06e0109140 0x7f06e019ef70 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f06d8005e30 tx=0x7f06d8003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:39.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.925+0000 7f06de000640 1 -- 172.21.15.43:0/3775733096 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d8031650 con 0x7f06e0109140 2024-09-19T05:09:39.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.925+0000 7f06e7800640 1 -- 172.21.15.43:0/3775733096 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06e019f8e0 con 0x7f06e0109140 2024-09-19T05:09:39.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.925+0000 7f06de000640 1 -- 172.21.15.43:0/3775733096 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06d80317f0 con 0x7f06e0109140 2024-09-19T05:09:39.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.925+0000 7f06de000640 1 -- 172.21.15.43:0/3775733096 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d802f9d0 con 0x7f06e0109140 2024-09-19T05:09:39.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.925+0000 7f06e7800640 1 -- 172.21.15.43:0/3775733096 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06e01a7220 con 0x7f06e0109140 2024-09-19T05:09:39.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.928+0000 7f06ea3c4740 1 Processor -- start 2024-09-19T05:09:39.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.928+0000 7f06de000640 1 -- 172.21.15.43:0/3775733096 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f06d802fbf0 con 0x7f06e0109140 2024-09-19T05:09:39.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.929+0000 7f06ea3c4740 1 -- start start 2024-09-19T05:09:39.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.929+0000 7f06ea3c4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaa9ad7c70 0x55aaa9ad8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:39.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.929+0000 7f06ea3c4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaa9ad8590 0x55aaa9ae2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:39.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.929+0000 7f06ea3c4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaa9ae2e50 0x55aaa9ae5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:39.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.929+0000 7f06ea3c4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55aaa9a3e3e0 con 0x55aaa9ae2e50 2024-09-19T05:09:39.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.929+0000 7f06ea3c4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55aaa99e5d80 con 0x55aaa9ad8590 2024-09-19T05:09:39.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.929+0000 7f06ea3c4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55aaa9a3a3a0 con 0x55aaa9ad7c70 2024-09-19T05:09:39.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.929+0000 7f06d7e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaa9ad7c70 0x55aaa9ad8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:39.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.930+0000 7f06d7e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaa9ad7c70 0x55aaa9ad8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:45594/0 (socket says 172.21.15.43:45594) 2024-09-19T05:09:39.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.930+0000 7f06d7e00640 1 -- 172.21.15.43:0/2406871376 learned_addr learned my addr 172.21.15.43:0/2406871376 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:39.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.930+0000 7f06e4e00640 1 --2- 172.21.15.43:0/2406871376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaa9ad8590 0x55aaa9ae2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:39.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.930+0000 7f06d7e00640 1 --2- 172.21.15.43:0/2406871376 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaa9ae2e50 0x55aaa9ae5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:39.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.930+0000 7f06e4e00640 1 -- 172.21.15.43:0/2406871376 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaa9ad7c70 msgr2=0x55aaa9ad8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:39.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.930+0000 7f06e4e00640 1 --2- 172.21.15.43:0/2406871376 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaa9ad7c70 0x55aaa9ad8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.930+0000 7f06e4e00640 1 -- 172.21.15.43:0/2406871376 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaa9ae2e50 msgr2=0x55aaa9ae5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:39.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.930+0000 7f06e4e00640 1 --2- 172.21.15.43:0/2406871376 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaa9ae2e50 0x55aaa9ae5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.930+0000 7f06e4e00640 1 -- 172.21.15.43:0/2406871376 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaa9a04ae0 con 0x55aaa9ad8590 2024-09-19T05:09:39.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.930+0000 7f06d7e00640 1 --2- 172.21.15.43:0/2406871376 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaa9ad7c70 0x55aaa9ad8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:39.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.930+0000 7f06de000640 1 --2- 172.21.15.43:0/3775733096 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f06c40a33b0 0x7f06c40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:39.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.930+0000 7f06de000640 1 -- 172.21.15.43:0/3775733096 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f06d80f0f80 con 0x7f06e0109140 2024-09-19T05:09:39.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.930+0000 7f06e4e00640 1 --2- 172.21.15.43:0/2406871376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaa9ad8590 0x55aaa9ae2910 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f06d0009fd0 tx=0x7f06d00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad28822cc855252b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:39.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.931+0000 7f06d7e00640 1 --2- 172.21.15.43:0/3775733096 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f06c40a33b0 0x7f06c40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:39.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.931+0000 7f06d6000640 1 -- 172.21.15.43:0/2406871376 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d0004030 con 0x55aaa9ad8590 2024-09-19T05:09:39.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.931+0000 7f06d6000640 1 -- 172.21.15.43:0/2406871376 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06d0005500 con 0x55aaa9ad8590 2024-09-19T05:09:39.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.931+0000 7f06d6000640 1 -- 172.21.15.43:0/2406871376 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d0004d40 con 0x55aaa9ad8590 2024-09-19T05:09:39.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.932+0000 7f06ea3c4740 1 -- 172.21.15.43:0/2406871376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaa9ad8590 msgr2=0x55aaa9ae2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:39.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.932+0000 7f06d7e00640 1 --2- 172.21.15.43:0/3775733096 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f06c40a33b0 0x7f06c40a5870 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x55aaa9ad5580 tx=0x7f06c8004730 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:39.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.932+0000 7f06ea3c4740 1 --2- 172.21.15.43:0/2406871376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaa9ad8590 0x55aaa9ae2910 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f06d0009fd0 tx=0x7f06d00118d0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.932+0000 7f06ea3c4740 1 -- 172.21.15.43:0/2406871376 shutdown_connections 2024-09-19T05:09:39.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.932+0000 7f06ea3c4740 1 --2- 172.21.15.43:0/2406871376 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaa9ae2e50 0x55aaa9ae5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.932+0000 7f06ea3c4740 1 --2- 172.21.15.43:0/2406871376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaa9ad8590 0x55aaa9ae2910 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.932+0000 7f06ea3c4740 1 --2- 172.21.15.43:0/2406871376 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaa9ad7c70 0x55aaa9ad8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.932+0000 7f06ea3c4740 1 -- 172.21.15.43:0/2406871376 >> 172.21.15.43:0/2406871376 conn(0x55aaa9ad4200 msgr2=0x55aaa9ad45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:39.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.932+0000 7f06ea3c4740 1 -- 172.21.15.43:0/2406871376 shutdown_connections 2024-09-19T05:09:39.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.932+0000 7f06ea3c4740 1 -- 172.21.15.43:0/2406871376 wait complete. 2024-09-19T05:09:39.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.932+0000 7f06ea3c4740 1 -- start start 2024-09-19T05:09:39.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.933+0000 7f06ea3c4740 1 Processor -- start 2024-09-19T05:09:39.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.933+0000 7f06ea3c4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaa9ad7c70 0x55aaa9adfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:39.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.933+0000 7f06ea3c4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaa9ad8590 0x55aaa9ae02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:39.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.933+0000 7f06ea3c4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaa9ae2e50 0x55aaa9ae07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:39.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.933+0000 7f06ea3c4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55aaa9ae5a20 con 0x55aaa9ae2e50 2024-09-19T05:09:39.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.933+0000 7f06ea3c4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55aaa99f90f0 con 0x55aaa9ad7c70 2024-09-19T05:09:39.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.933+0000 7f06ea3c4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55aaa9ae5ba0 con 0x55aaa9ad8590 2024-09-19T05:09:39.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.933+0000 7f06ea3c4740 10 client.0 _ll_register_callbacks cb 0x55aaa99e0bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:39.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.933+0000 7f06e4e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaa9ad7c70 0x55aaa9adfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:39.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.933+0000 7f06e4e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaa9ad7c70 0x55aaa9adfd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:52480/0 (socket says 172.21.15.43:52480) 2024-09-19T05:09:39.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.933+0000 7f06e4e00640 1 -- 172.21.15.43:0/2695619642 learned_addr learned my addr 172.21.15.43:0/2695619642 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:39.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.933+0000 7f06e5800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaa9ae2e50 0x55aaa9ae07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:39.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.934+0000 7f06d7e00640 1 --2- 172.21.15.43:0/2695619642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaa9ad8590 0x55aaa9ae02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:39.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.934+0000 7f06e4e00640 1 -- 172.21.15.43:0/2695619642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaa9ad8590 msgr2=0x55aaa9ae02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:39.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.934+0000 7f06e4e00640 1 --2- 172.21.15.43:0/2695619642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaa9ad8590 0x55aaa9ae02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.934+0000 7f06e4e00640 1 -- 172.21.15.43:0/2695619642 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaa9ae2e50 msgr2=0x55aaa9ae07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:39.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.934+0000 7f06e4e00640 1 --2- 172.21.15.43:0/2695619642 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaa9ae2e50 0x55aaa9ae07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.934+0000 7f06e4e00640 1 -- 172.21.15.43:0/2695619642 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aaa97af3c0 con 0x55aaa9ad7c70 2024-09-19T05:09:39.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.934+0000 7f06e5800640 1 --2- 172.21.15.43:0/2695619642 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaa9ae2e50 0x55aaa9ae07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:39.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.934+0000 7f06e4e00640 1 --2- 172.21.15.43:0/2695619642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaa9ad7c70 0x55aaa9adfd60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f06d001b040 tx=0x7f06d0005100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47902539f2b8c0c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:39.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.934+0000 7f06cea00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:09:39.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.934+0000 7f06cea00640 1 -- 172.21.15.43:0/2695619642 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d0028070 con 0x55aaa9ad7c70 2024-09-19T05:09:39.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.935+0000 7f06cea00640 1 -- 172.21.15.43:0/2695619642 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f06d000f5a0 con 0x55aaa9ad7c70 2024-09-19T05:09:39.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.935+0000 7f06ea3c4740 1 -- 172.21.15.43:0/2695619642 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06d0005500 con 0x55aaa9ad7c70 2024-09-19T05:09:39.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.935+0000 7f06cea00640 1 -- 172.21.15.43:0/2695619642 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f06d0034050 con 0x55aaa9ad7c70 2024-09-19T05:09:39.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.935+0000 7f06ea3c4740 10 client.26198 fetch_fsmap learned FSMap version 29 2024-09-19T05:09:39.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.935+0000 7f06ea3c4740 1 -- 172.21.15.43:0/2695619642 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55aaa9af1a90 con 0x55aaa9ad7c70 2024-09-19T05:09:39.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.935+0000 7f06cea00640 1 -- 172.21.15.43:0/2695619642 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06d0005500 con 0x55aaa9ad7c70 2024-09-19T05:09:39.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.935+0000 7f06cea00640 1 -- 172.21.15.43:0/2695619642 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f06d0019690 con 0x55aaa9ad7c70 2024-09-19T05:09:39.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.935+0000 7f06ea3c4740 10 client.26198 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:09:39.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.935+0000 7f06ea3c4740 10 client.26198 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T05:09:39.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.935+0000 7f06ea3c4740 20 client.26198 populate_metadata read hostname 'smithi043' 2024-09-19T05:09:39.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.935+0000 7f06ea3c4740 1 --2- 172.21.15.43:0/2695619642 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55aaa9af2e40 0x55aaa9af5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:39.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.935+0000 7f06ea3c4740 4 client.26198 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:09:39.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.935+0000 7f06ea3c4740 1 -- 172.21.15.43:0/2695619642 --> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aaa9a4b950 con 0x55aaa9af2e40 2024-09-19T05:09:39.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.936+0000 7f06d7e00640 1 --2- 172.21.15.43:0/2695619642 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55aaa9af2e40 0x55aaa9af5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:39.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.936+0000 7f06d7e00640 1 --2- 172.21.15.43:0/2695619642 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55aaa9af2e40 0x55aaa9af5230 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cdbee5be28c20674 server_cookie=10c7d50c1bc90c55 in_seq=0 out_seq=0 2024-09-19T05:09:39.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.936+0000 7f06cea00640 10 client.26198 ms_handle_connect on v2:172.21.15.96:6836/3978388286 2024-09-19T05:09:39.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.937+0000 7f06cea00640 1 -- 172.21.15.43:0/2695619642 <== mds.3 v2:172.21.15.96:6836/3978388286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aaa9a4b950 con 0x55aaa9af2e40 2024-09-19T05:09:39.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.937+0000 7f06cea00640 10 client.26198 handle_command_reply: tid=0 2024-09-19T05:09:39.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.937+0000 7f06ea3c4740 1 client.26198 shutdown 2024-09-19T05:09:39.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.937+0000 7f06ea3c4740 1 -- 172.21.15.43:0/2695619642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaa9ad7c70 msgr2=0x55aaa9adfd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:39.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.937+0000 7f06ea3c4740 1 --2- 172.21.15.43:0/2695619642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaa9ad7c70 0x55aaa9adfd60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f06d001b040 tx=0x7f06d0005100 comp rx=0 tx=0).stop 2024-09-19T05:09:39.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.937+0000 7f06ea3c4740 1 -- 172.21.15.43:0/2695619642 shutdown_connections 2024-09-19T05:09:39.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.937+0000 7f06ea3c4740 1 --2- 172.21.15.43:0/2695619642 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55aaa9af2e40 0x55aaa9af5230 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.937+0000 7f06ea3c4740 1 --2- 172.21.15.43:0/2695619642 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaa9ae2e50 0x55aaa9ae07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.937+0000 7f06ea3c4740 1 --2- 172.21.15.43:0/2695619642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaa9ad8590 0x55aaa9ae02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.937+0000 7f06cea00640 0 client.26198 ms_handle_reset on v2:172.21.15.96:6836/3978388286 2024-09-19T05:09:39.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.937+0000 7f06ea3c4740 1 --2- 172.21.15.43:0/2695619642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaa9ad7c70 0x55aaa9adfd60 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.937+0000 7f06ea3c4740 1 -- 172.21.15.43:0/2695619642 >> 172.21.15.43:0/2695619642 conn(0x55aaa9ad4200 msgr2=0x55aaa9ad9ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:39.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.938+0000 7f06ea3c4740 1 -- 172.21.15.43:0/2695619642 shutdown_connections 2024-09-19T05:09:39.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.938+0000 7f06ea3c4740 1 -- 172.21.15.43:0/2695619642 wait complete. 2024-09-19T05:09:39.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.938+0000 7f06ea3c4740 20 client.26198 trim_cache size 0 max 16384 2024-09-19T05:09:39.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.951+0000 7f06ea3c4740 1 Processor -- start 2024-09-19T05:09:39.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.952+0000 7f06ea3c4740 1 -- start start 2024-09-19T05:09:39.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.952+0000 7f06ea3c4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaa9ad7c70 0x55aaa9afaa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:39.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.952+0000 7f06ea3c4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaa9ad8590 0x55aaa9afaf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:39.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.952+0000 7f06ea3c4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaa9ae2e50 0x55aaa9ad96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:39.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.952+0000 7f06ea3c4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f06e01afac0 con 0x55aaa9ae2e50 2024-09-19T05:09:39.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.952+0000 7f06ea3c4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f06e01b1600 con 0x55aaa9ad7c70 2024-09-19T05:09:39.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.952+0000 7f06ea3c4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f06e01b0190 con 0x55aaa9ad8590 2024-09-19T05:09:39.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.952+0000 7f06e5800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaa9ae2e50 0x55aaa9ad96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:39.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.952+0000 7f06e5800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaa9ae2e50 0x55aaa9ad96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40840/0 (socket says 172.21.15.43:40840) 2024-09-19T05:09:39.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.952+0000 7f06e5800640 1 -- 172.21.15.43:0/1186735765 learned_addr learned my addr 172.21.15.43:0/1186735765 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:39.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.952+0000 7f06e4e00640 1 --2- 172.21.15.43:0/1186735765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaa9ad7c70 0x55aaa9afaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:39.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.952+0000 7f06d7e00640 1 --2- 172.21.15.43:0/1186735765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaa9ad8590 0x55aaa9afaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:39.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.953+0000 7f06e5800640 1 -- 172.21.15.43:0/1186735765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaa9ad8590 msgr2=0x55aaa9afaf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:39.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.953+0000 7f06e5800640 1 --2- 172.21.15.43:0/1186735765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaa9ad8590 0x55aaa9afaf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.953+0000 7f06e5800640 1 -- 172.21.15.43:0/1186735765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaa9ad7c70 msgr2=0x55aaa9afaa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:39.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.953+0000 7f06e5800640 1 --2- 172.21.15.43:0/1186735765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaa9ad7c70 0x55aaa9afaa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.953+0000 7f06e5800640 1 -- 172.21.15.43:0/1186735765 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaa9acf770 con 0x55aaa9ae2e50 2024-09-19T05:09:39.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.953+0000 7f06e5800640 1 --2- 172.21.15.43:0/1186735765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaa9ae2e50 0x55aaa9ad96c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f06d80bd2b0 tx=0x7f06d80095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4252d7c8905f4416 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:39.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.953+0000 7f06d5600640 1 -- 172.21.15.43:0/1186735765 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d80db070 con 0x55aaa9ae2e50 2024-09-19T05:09:39.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.953+0000 7f06d5600640 1 -- 172.21.15.43:0/1186735765 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06d8036040 con 0x55aaa9ae2e50 2024-09-19T05:09:39.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.953+0000 7f06d5600640 1 -- 172.21.15.43:0/1186735765 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d80d5030 con 0x55aaa9ae2e50 2024-09-19T05:09:39.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.954+0000 7f06ea3c4740 1 -- 172.21.15.43:0/1186735765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaa9ae2e50 msgr2=0x55aaa9ad96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:39.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.954+0000 7f06ea3c4740 1 --2- 172.21.15.43:0/1186735765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaa9ae2e50 0x55aaa9ad96c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f06d80bd2b0 tx=0x7f06d80095c0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.954+0000 7f06ea3c4740 1 -- 172.21.15.43:0/1186735765 shutdown_connections 2024-09-19T05:09:39.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.954+0000 7f06ea3c4740 1 --2- 172.21.15.43:0/1186735765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaa9ae2e50 0x55aaa9ad96c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.954+0000 7f06ea3c4740 1 --2- 172.21.15.43:0/1186735765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaa9ad8590 0x55aaa9afaf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.954+0000 7f06ea3c4740 1 --2- 172.21.15.43:0/1186735765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaa9ad7c70 0x55aaa9afaa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.954+0000 7f06ea3c4740 1 -- 172.21.15.43:0/1186735765 >> 172.21.15.43:0/1186735765 conn(0x55aaa9ad4200 msgr2=0x55aaa9adee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:39.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.954+0000 7f06ea3c4740 1 -- 172.21.15.43:0/1186735765 shutdown_connections 2024-09-19T05:09:39.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.954+0000 7f06ea3c4740 1 -- 172.21.15.43:0/1186735765 wait complete. 2024-09-19T05:09:39.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.954+0000 7f06ea3c4740 1 -- start start 2024-09-19T05:09:39.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.955+0000 7f06ea3c4740 1 Processor -- start 2024-09-19T05:09:39.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.955+0000 7f06ea3c4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaa9ad7c70 0x55aaa9aeb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:39.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.955+0000 7f06ea3c4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaa9ad8590 0x55aaa9ae6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:39.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.955+0000 7f06ea3c4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaa9ae2e50 0x55aaa9ae6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:39.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.955+0000 7f06ea3c4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f06e01b0f30 con 0x55aaa9ae2e50 2024-09-19T05:09:39.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.955+0000 7f06ea3c4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f06e01b0860 con 0x55aaa9ad8590 2024-09-19T05:09:39.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.955+0000 7f06ea3c4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f06e01adaa0 con 0x55aaa9ad7c70 2024-09-19T05:09:39.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.955+0000 7f06ea3c4740 10 client.0 _ll_register_callbacks cb 0x7f06c8009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:39.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.955+0000 7f06e4e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaa9ad7c70 0x55aaa9aeb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:39.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.955+0000 7f06e5800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaa9ae2e50 0x55aaa9ae6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:39.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.955+0000 7f06e4e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaa9ad7c70 0x55aaa9aeb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:45620/0 (socket says 172.21.15.43:45620) 2024-09-19T05:09:39.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.955+0000 7f06e5800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaa9ae2e50 0x55aaa9ae6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40852/0 (socket says 172.21.15.43:40852) 2024-09-19T05:09:39.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.955+0000 7f06e4e00640 1 -- 172.21.15.43:0/2710902024 learned_addr learned my addr 172.21.15.43:0/2710902024 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:39.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.955+0000 7f06d7e00640 1 --2- 172.21.15.43:0/2710902024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaa9ad8590 0x55aaa9ae6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:39.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.956+0000 7f06e4e00640 1 -- 172.21.15.43:0/2710902024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaa9ad8590 msgr2=0x55aaa9ae6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:39.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.956+0000 7f06e4e00640 1 --2- 172.21.15.43:0/2710902024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaa9ad8590 0x55aaa9ae6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.956+0000 7f06e4e00640 1 -- 172.21.15.43:0/2710902024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaa9ae2e50 msgr2=0x55aaa9ae6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:39.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.956+0000 7f06e4e00640 1 --2- 172.21.15.43:0/2710902024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaa9ae2e50 0x55aaa9ae6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.956+0000 7f06e4e00640 1 -- 172.21.15.43:0/2710902024 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aaa9ae6ff0 con 0x55aaa9ad7c70 2024-09-19T05:09:39.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.956+0000 7f06e5800640 1 --2- 172.21.15.43:0/2710902024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaa9ae2e50 0x55aaa9ae6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:39.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.956+0000 7f06e4e00640 1 --2- 172.21.15.43:0/2710902024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaa9ad7c70 0x55aaa9aeb240 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f06d00059f0 tx=0x7f06d0004a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e4b8bf6ea763b20 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:39.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.956+0000 7f06cea00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:09:39.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.956+0000 7f06cea00640 1 -- 172.21.15.43:0/2710902024 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d0028040 con 0x55aaa9ad7c70 2024-09-19T05:09:39.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.956+0000 7f06ea3c4740 1 -- 172.21.15.43:0/2710902024 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06d8036040 con 0x55aaa9ad7c70 2024-09-19T05:09:39.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.956+0000 7f06cea00640 1 -- 172.21.15.43:0/2710902024 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f06d0005500 con 0x55aaa9ad7c70 2024-09-19T05:09:39.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.957+0000 7f06cea00640 1 -- 172.21.15.43:0/2710902024 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f06d0011dd0 con 0x55aaa9ad7c70 2024-09-19T05:09:39.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.957+0000 7f06ea3c4740 10 client.26140 fetch_fsmap learned FSMap version 29 2024-09-19T05:09:39.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.957+0000 7f06ea3c4740 1 -- 172.21.15.43:0/2710902024 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55aaa9b5d6e0 con 0x55aaa9ad7c70 2024-09-19T05:09:39.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.957+0000 7f06cea00640 1 -- 172.21.15.43:0/2710902024 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06d8036040 con 0x55aaa9ad7c70 2024-09-19T05:09:39.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.957+0000 7f06cea00640 1 -- 172.21.15.43:0/2710902024 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f06d00035f0 con 0x55aaa9ad7c70 2024-09-19T05:09:39.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.957+0000 7f06ea3c4740 10 client.26140 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:09:39.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.957+0000 7f06ea3c4740 10 client.26140 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T05:09:39.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.957+0000 7f06ea3c4740 20 client.26140 populate_metadata read hostname 'smithi043' 2024-09-19T05:09:39.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.957+0000 7f06ea3c4740 1 --2- 172.21.15.43:0/2710902024 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55aaa9b5dd50 0x55aaa9b60170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:39.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.957+0000 7f06ea3c4740 4 client.26140 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:09:39.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.957+0000 7f06ea3c4740 1 -- 172.21.15.43:0/2710902024 --> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aaa9a45e60 con 0x55aaa9b5dd50 2024-09-19T05:09:39.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.958+0000 7f06d7e00640 1 --2- 172.21.15.43:0/2710902024 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55aaa9b5dd50 0x55aaa9b60170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:39.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.958+0000 7f06d7e00640 1 --2- 172.21.15.43:0/2710902024 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55aaa9b5dd50 0x55aaa9b60170 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cb0115c2bc763271 server_cookie=811b4d91ca983088 in_seq=0 out_seq=0 2024-09-19T05:09:39.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.958+0000 7f06cea00640 10 client.26140 ms_handle_connect on v2:172.21.15.96:6836/3978388286 2024-09-19T05:09:39.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.958+0000 7f06cea00640 1 -- 172.21.15.43:0/2710902024 <== mds.3 v2:172.21.15.96:6836/3978388286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aaa9a45e60 con 0x55aaa9b5dd50 2024-09-19T05:09:39.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.958+0000 7f06cea00640 10 client.26140 handle_command_reply: tid=0 2024-09-19T05:09:39.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.958+0000 7f06ea3c4740 1 client.26140 shutdown 2024-09-19T05:09:39.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.959+0000 7f06ea3c4740 1 -- 172.21.15.43:0/2710902024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaa9ad7c70 msgr2=0x55aaa9aeb240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:39.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.959+0000 7f06ea3c4740 1 --2- 172.21.15.43:0/2710902024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaa9ad7c70 0x55aaa9aeb240 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f06d00059f0 tx=0x7f06d0004a10 comp rx=0 tx=0).stop 2024-09-19T05:09:39.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.959+0000 7f06ea3c4740 1 -- 172.21.15.43:0/2710902024 shutdown_connections 2024-09-19T05:09:39.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.959+0000 7f06ea3c4740 1 --2- 172.21.15.43:0/2710902024 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55aaa9b5dd50 0x55aaa9b60170 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.959+0000 7f06ea3c4740 1 --2- 172.21.15.43:0/2710902024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaa9ae2e50 0x55aaa9ae6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.959+0000 7f06ea3c4740 1 --2- 172.21.15.43:0/2710902024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaa9ad8590 0x55aaa9ae6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.959+0000 7f06cea00640 0 client.26140 ms_handle_reset on v2:172.21.15.96:6836/3978388286 2024-09-19T05:09:39.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.959+0000 7f06ea3c4740 1 --2- 172.21.15.43:0/2710902024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaa9ad7c70 0x55aaa9aeb240 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.959+0000 7f06ea3c4740 1 -- 172.21.15.43:0/2710902024 >> 172.21.15.43:0/2710902024 conn(0x55aaa9ad4200 msgr2=0x55aaa9ad2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:39.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.959+0000 7f06ea3c4740 1 -- 172.21.15.43:0/2710902024 shutdown_connections 2024-09-19T05:09:39.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.959+0000 7f06ea3c4740 1 -- 172.21.15.43:0/2710902024 wait complete. 2024-09-19T05:09:39.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.959+0000 7f06ea3c4740 20 client.26140 trim_cache size 0 max 16384 2024-09-19T05:09:39.966 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:09:39.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.960+0000 7f06e7800640 1 -- 172.21.15.43:0/3775733096 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f06c40a33b0 msgr2=0x7f06c40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:39.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.960+0000 7f06e7800640 1 --2- 172.21.15.43:0/3775733096 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f06c40a33b0 0x7f06c40a5870 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x55aaa9ad5580 tx=0x7f06c8004730 comp rx=0 tx=0).stop 2024-09-19T05:09:39.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.960+0000 7f06e7800640 1 -- 172.21.15.43:0/3775733096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06e0109140 msgr2=0x7f06e019ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:39.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.960+0000 7f06e7800640 1 --2- 172.21.15.43:0/3775733096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06e0109140 0x7f06e019ef70 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f06d8005e30 tx=0x7f06d8003d00 comp rx=0 tx=0).stop 2024-09-19T05:09:39.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.960+0000 7f06e7800640 1 -- 172.21.15.43:0/3775733096 shutdown_connections 2024-09-19T05:09:39.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.960+0000 7f06e7800640 1 --2- 172.21.15.43:0/3775733096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06e0109140 0x7f06e019ef70 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.960+0000 7f06e7800640 1 --2- 172.21.15.43:0/3775733096 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f06c40a33b0 0x7f06c40a5870 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.960+0000 7f06e7800640 1 --2- 172.21.15.43:0/3775733096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06e01049c0 0x7f06e019abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.960+0000 7f06e7800640 1 --2- 172.21.15.43:0/3775733096 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06e0104010 0x7f06e019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:39.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.960+0000 7f06e7800640 1 -- 172.21.15.43:0/3775733096 >> 172.21.15.43:0/3775733096 conn(0x7f06e00ffa50 msgr2=0x7f06e0100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:39.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.960+0000 7f06e7800640 1 -- 172.21.15.43:0/3775733096 shutdown_connections 2024-09-19T05:09:39.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:39.960+0000 7f06e7800640 1 -- 172.21.15.43:0/3775733096 wait complete. 2024-09-19T05:09:39.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:39 smithi096 ceph-mon[35085]: pgmap v153: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:09:40.692 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T05:09:41.071 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:09:41.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.598+0000 7f1bb0400640 1 -- 172.21.15.43:0/3995734590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ba81049a0 msgr2=0x7f1ba81089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:41.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.598+0000 7f1bb0400640 1 --2- 172.21.15.43:0/3995734590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ba81049a0 0x7f1ba81089f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1b94009860 tx=0x7f1b9402a340 comp rx=0 tx=0).stop 2024-09-19T05:09:41.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.598+0000 7f1bb0400640 1 -- 172.21.15.43:0/3995734590 shutdown_connections 2024-09-19T05:09:41.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.598+0000 7f1bb0400640 1 --2- 172.21.15.43:0/3995734590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ba8109120 0x7f1ba8110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:41.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.598+0000 7f1bb0400640 1 --2- 172.21.15.43:0/3995734590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ba81049a0 0x7f1ba81089f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:41.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.598+0000 7f1bb0400640 1 --2- 172.21.15.43:0/3995734590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ba8103ff0 0x7f1ba81043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:41.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.598+0000 7f1bb0400640 1 -- 172.21.15.43:0/3995734590 >> 172.21.15.43:0/3995734590 conn(0x7f1ba80ffa90 msgr2=0x7f1ba8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:41.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.598+0000 7f1bb0400640 1 -- 172.21.15.43:0/3995734590 shutdown_connections 2024-09-19T05:09:41.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.598+0000 7f1bb0400640 1 -- 172.21.15.43:0/3995734590 wait complete. 2024-09-19T05:09:41.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.599+0000 7f1bb0400640 1 Processor -- start 2024-09-19T05:09:41.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.599+0000 7f1bb0400640 1 -- start start 2024-09-19T05:09:41.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.599+0000 7f1bb0400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ba8103ff0 0x7f1ba8076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:41.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.599+0000 7f1bb0400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ba81049a0 0x7f1ba8076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:41.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.599+0000 7f1bb0400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ba8109120 0x7f1ba8077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:41.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.599+0000 7f1bb0400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba8114140 con 0x7f1ba8109120 2024-09-19T05:09:41.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.599+0000 7f1bb0400640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba8113fc0 con 0x7f1ba8103ff0 2024-09-19T05:09:41.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.599+0000 7f1bb0400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba81142c0 con 0x7f1ba81049a0 2024-09-19T05:09:41.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.600+0000 7f1bae400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ba8109120 0x7f1ba8077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:41.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.600+0000 7f1bae400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ba8109120 0x7f1ba8077180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40874/0 (socket says 172.21.15.43:40874) 2024-09-19T05:09:41.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.600+0000 7f1bae400640 1 -- 172.21.15.43:0/2037337412 learned_addr learned my addr 172.21.15.43:0/2037337412 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:41.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.600+0000 7f1bad000640 1 --2- 172.21.15.43:0/2037337412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ba81049a0 0x7f1ba8076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:41.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.600+0000 7f1bada00640 1 --2- 172.21.15.43:0/2037337412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ba8103ff0 0x7f1ba8076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:41.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.600+0000 7f1bae400640 1 -- 172.21.15.43:0/2037337412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ba81049a0 msgr2=0x7f1ba8076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:41.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.600+0000 7f1bae400640 1 --2- 172.21.15.43:0/2037337412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ba81049a0 0x7f1ba8076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:41.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.600+0000 7f1bae400640 1 -- 172.21.15.43:0/2037337412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ba8103ff0 msgr2=0x7f1ba8076700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:41.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.600+0000 7f1bae400640 1 --2- 172.21.15.43:0/2037337412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ba8103ff0 0x7f1ba8076700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:41.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.600+0000 7f1bae400640 1 -- 172.21.15.43:0/2037337412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba81aad60 con 0x7f1ba8109120 2024-09-19T05:09:41.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.600+0000 7f1bae400640 1 --2- 172.21.15.43:0/2037337412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ba8109120 0x7f1ba8077180 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1ba000c370 tx=0x7f1ba000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:41.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.601+0000 7f1ba6000640 1 -- 172.21.15.43:0/2037337412 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0003970 con 0x7f1ba8109120 2024-09-19T05:09:41.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.601+0000 7f1bb0400640 1 -- 172.21.15.43:0/2037337412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ba81ab050 con 0x7f1ba8109120 2024-09-19T05:09:41.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.601+0000 7f1ba6000640 1 -- 172.21.15.43:0/2037337412 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ba0004020 con 0x7f1ba8109120 2024-09-19T05:09:41.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.601+0000 7f1ba6000640 1 -- 172.21.15.43:0/2037337412 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba000bad0 con 0x7f1ba8109120 2024-09-19T05:09:41.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.601+0000 7f1bb0400640 1 -- 172.21.15.43:0/2037337412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ba81ab510 con 0x7f1ba8109120 2024-09-19T05:09:41.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.604+0000 7f1ba6000640 1 -- 172.21.15.43:0/2037337412 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f1ba001d020 con 0x7f1ba8109120 2024-09-19T05:09:41.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.604+0000 7f1bb2ec2740 1 Processor -- start 2024-09-19T05:09:41.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.605+0000 7f1bb2ec2740 1 -- start start 2024-09-19T05:09:41.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.605+0000 7f1bb2ec2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c3143b1c70 0x55c3143b2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:41.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.605+0000 7f1bb2ec2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3143b2590 0x55c3143bc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:41.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.605+0000 7f1bb2ec2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c3143bce50 0x55c3143bf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:41.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.605+0000 7f1bb2ec2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c3143183e0 con 0x55c3143bce50 2024-09-19T05:09:41.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.605+0000 7f1bb2ec2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c3142bfd80 con 0x55c3143b2590 2024-09-19T05:09:41.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.605+0000 7f1bb2ec2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c3143143a0 con 0x55c3143b1c70 2024-09-19T05:09:41.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.606+0000 7f1bada00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3143b2590 0x55c3143bc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:41.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.606+0000 7f1bad000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c3143b1c70 0x55c3143b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:41.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.606+0000 7f1bada00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3143b2590 0x55c3143bc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:52528/0 (socket says 172.21.15.43:52528) 2024-09-19T05:09:41.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.606+0000 7f1bad000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c3143b1c70 0x55c3143b2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:45652/0 (socket says 172.21.15.43:45652) 2024-09-19T05:09:41.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.606+0000 7f1bada00640 1 -- 172.21.15.43:0/1124956217 learned_addr learned my addr 172.21.15.43:0/1124956217 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:41.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.606+0000 7f1bad000640 1 --2- 172.21.15.43:0/1124956217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c3143bce50 0x55c3143bf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:41.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.606+0000 7f1bada00640 1 -- 172.21.15.43:0/1124956217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c3143b1c70 msgr2=0x55c3143b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:41.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.606+0000 7f1bada00640 1 --2- 172.21.15.43:0/1124956217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c3143b1c70 0x55c3143b2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:41.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.606+0000 7f1bada00640 1 -- 172.21.15.43:0/1124956217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c3143bce50 msgr2=0x55c3143bf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:41.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.606+0000 7f1bada00640 1 --2- 172.21.15.43:0/1124956217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c3143bce50 0x55c3143bf240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:41.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.606+0000 7f1bada00640 1 -- 172.21.15.43:0/1124956217 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3142deae0 con 0x55c3143b2590 2024-09-19T05:09:41.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.606+0000 7f1bad000640 1 --2- 172.21.15.43:0/1124956217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c3143bce50 0x55c3143bf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:09:41.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.606+0000 7f1ba6000640 1 --2- 172.21.15.43:0/2037337412 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f1b900a33b0 0x7f1b900a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:41.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.606+0000 7f1ba6000640 1 -- 172.21.15.43:0/2037337412 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f1ba00cfaf0 con 0x7f1ba8109120 2024-09-19T05:09:41.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.607+0000 7f1bada00640 1 --2- 172.21.15.43:0/1124956217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3143b2590 0x55c3143bc910 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c009d20 tx=0x7f1b9c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2317796f27ed8e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:41.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.607+0000 7f1bad000640 1 --2- 172.21.15.43:0/2037337412 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f1b900a33b0 0x7f1b900a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:41.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.607+0000 7f1b9aa00640 1 -- 172.21.15.43:0/1124956217 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c01e070 con 0x55c3143b2590 2024-09-19T05:09:41.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.607+0000 7f1b9aa00640 1 -- 172.21.15.43:0/1124956217 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b9c019b10 con 0x55c3143b2590 2024-09-19T05:09:41.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.607+0000 7f1b9aa00640 1 -- 172.21.15.43:0/1124956217 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c00ebf0 con 0x55c3143b2590 2024-09-19T05:09:41.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.608+0000 7f1bb2ec2740 1 -- 172.21.15.43:0/1124956217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3143b2590 msgr2=0x55c3143bc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:41.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.608+0000 7f1bb2ec2740 1 --2- 172.21.15.43:0/1124956217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3143b2590 0x55c3143bc910 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c009d20 tx=0x7f1b9c011810 comp rx=0 tx=0).stop 2024-09-19T05:09:41.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.608+0000 7f1bad000640 1 --2- 172.21.15.43:0/2037337412 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f1b900a33b0 0x7f1b900a5870 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1b9402a930 tx=0x7f1b940058d0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:41.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.608+0000 7f1bb2ec2740 1 -- 172.21.15.43:0/1124956217 shutdown_connections 2024-09-19T05:09:41.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.608+0000 7f1bb2ec2740 1 --2- 172.21.15.43:0/1124956217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c3143bce50 0x55c3143bf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:41.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.608+0000 7f1bb2ec2740 1 --2- 172.21.15.43:0/1124956217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3143b2590 0x55c3143bc910 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:41.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.608+0000 7f1bb2ec2740 1 --2- 172.21.15.43:0/1124956217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c3143b1c70 0x55c3143b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:41.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.608+0000 7f1bb2ec2740 1 -- 172.21.15.43:0/1124956217 >> 172.21.15.43:0/1124956217 conn(0x55c3143ae200 msgr2=0x55c3143ae5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:41.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.608+0000 7f1bb2ec2740 1 -- 172.21.15.43:0/1124956217 shutdown_connections 2024-09-19T05:09:41.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.608+0000 7f1bb2ec2740 1 -- 172.21.15.43:0/1124956217 wait complete. 2024-09-19T05:09:41.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.608+0000 7f1bb2ec2740 1 -- start start 2024-09-19T05:09:41.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.609+0000 7f1bb2ec2740 1 Processor -- start 2024-09-19T05:09:41.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.609+0000 7f1bb2ec2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3143b1c70 0x55c3143b9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:41.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.609+0000 7f1bb2ec2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c3143b2590 0x55c3143ba2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:41.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.609+0000 7f1bb2ec2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c3143bce50 0x55c3143ba830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:41.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.609+0000 7f1bb2ec2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c3143bfa20 con 0x55c3143bce50 2024-09-19T05:09:41.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.609+0000 7f1bb2ec2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c3142d30f0 con 0x55c3143b1c70 2024-09-19T05:09:41.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.609+0000 7f1bb2ec2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c3143bfba0 con 0x55c3143b2590 2024-09-19T05:09:41.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.609+0000 7f1bb2ec2740 10 client.0 _ll_register_callbacks cb 0x55c3142babd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:41.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.609+0000 7f1bae400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c3143bce50 0x55c3143ba830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:41.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.609+0000 7f1bae400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c3143bce50 0x55c3143ba830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40892/0 (socket says 172.21.15.43:40892) 2024-09-19T05:09:41.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.609+0000 7f1bae400640 1 -- 172.21.15.43:0/2155259022 learned_addr learned my addr 172.21.15.43:0/2155259022 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:41.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.609+0000 7f1bada00640 1 --2- 172.21.15.43:0/2155259022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3143b1c70 0x55c3143b9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:41.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.609+0000 7f1bad000640 1 --2- 172.21.15.43:0/2155259022 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c3143b2590 0x55c3143ba2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:41.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.610+0000 7f1bada00640 1 -- 172.21.15.43:0/2155259022 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c3143b2590 msgr2=0x55c3143ba2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:41.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.610+0000 7f1bada00640 1 --2- 172.21.15.43:0/2155259022 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c3143b2590 0x55c3143ba2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:41.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.610+0000 7f1bada00640 1 -- 172.21.15.43:0/2155259022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c3143bce50 msgr2=0x55c3143ba830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:41.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.610+0000 7f1bada00640 1 --2- 172.21.15.43:0/2155259022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c3143bce50 0x55c3143ba830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:41.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.610+0000 7f1bada00640 1 -- 172.21.15.43:0/2155259022 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3140893c0 con 0x55c3143b1c70 2024-09-19T05:09:41.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.610+0000 7f1bae400640 1 --2- 172.21.15.43:0/2155259022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c3143bce50 0x55c3143ba830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:41.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.610+0000 7f1bada00640 1 --2- 172.21.15.43:0/2155259022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3143b1c70 0x55c3143b9db0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c01c040 tx=0x7f1b9c019660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfd91fd0500e59df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:41.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.610+0000 7f1b8f400640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:09:41.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.610+0000 7f1b8f400640 1 -- 172.21.15.43:0/2155259022 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c01e070 con 0x55c3143b1c70 2024-09-19T05:09:41.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.610+0000 7f1bb2ec2740 1 -- 172.21.15.43:0/2155259022 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b9c019b10 con 0x55c3143b1c70 2024-09-19T05:09:41.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.610+0000 7f1b8f400640 1 -- 172.21.15.43:0/2155259022 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f1b9c00e8f0 con 0x55c3143b1c70 2024-09-19T05:09:41.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.611+0000 7f1b8f400640 1 -- 172.21.15.43:0/2155259022 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1b9c030050 con 0x55c3143b1c70 2024-09-19T05:09:41.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.611+0000 7f1bb2ec2740 10 client.26216 fetch_fsmap learned FSMap version 29 2024-09-19T05:09:41.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.611+0000 7f1bb2ec2740 1 -- 172.21.15.43:0/2155259022 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c3143cbae0 con 0x55c3143b1c70 2024-09-19T05:09:41.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.611+0000 7f1b8f400640 1 -- 172.21.15.43:0/2155259022 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b9c019b10 con 0x55c3143b1c70 2024-09-19T05:09:41.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.611+0000 7f1b8f400640 1 -- 172.21.15.43:0/2155259022 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1b9c0053a0 con 0x55c3143b1c70 2024-09-19T05:09:41.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.611+0000 7f1bb2ec2740 10 client.26216 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:09:41.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.611+0000 7f1bb2ec2740 10 client.26216 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T05:09:41.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.611+0000 7f1bb2ec2740 20 client.26216 populate_metadata read hostname 'smithi043' 2024-09-19T05:09:41.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.611+0000 7f1bb2ec2740 1 --2- 172.21.15.43:0/2155259022 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55c3143ccfa0 0x55c3143cf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:41.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.611+0000 7f1bb2ec2740 4 client.26216 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:09:41.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.611+0000 7f1bb2ec2740 1 -- 172.21.15.43:0/2155259022 --> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c314325950 con 0x55c3143ccfa0 2024-09-19T05:09:41.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.612+0000 7f1bad000640 1 --2- 172.21.15.43:0/2155259022 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55c3143ccfa0 0x55c3143cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:41.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.612+0000 7f1bad000640 1 --2- 172.21.15.43:0/2155259022 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55c3143ccfa0 0x55c3143cf390 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=65b66df6c1029d54 server_cookie=31d23ad8fae2c421 in_seq=0 out_seq=0 2024-09-19T05:09:41.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.612+0000 7f1b8f400640 10 client.26216 ms_handle_connect on v2:172.21.15.151:6838/3794104313 2024-09-19T05:09:41.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.613+0000 7f1b8f400640 1 -- 172.21.15.43:0/2155259022 <== mds.2 v2:172.21.15.151:6838/3794104313 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c314325950 con 0x55c3143ccfa0 2024-09-19T05:09:41.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.613+0000 7f1b8f400640 10 client.26216 handle_command_reply: tid=0 2024-09-19T05:09:41.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.613+0000 7f1bb2ec2740 1 client.26216 shutdown 2024-09-19T05:09:41.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.614+0000 7f1bb2ec2740 1 -- 172.21.15.43:0/2155259022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3143b1c70 msgr2=0x55c3143b9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:41.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.614+0000 7f1bb2ec2740 1 --2- 172.21.15.43:0/2155259022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3143b1c70 0x55c3143b9db0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c01c040 tx=0x7f1b9c019660 comp rx=0 tx=0).stop 2024-09-19T05:09:41.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.614+0000 7f1bb2ec2740 1 -- 172.21.15.43:0/2155259022 shutdown_connections 2024-09-19T05:09:41.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.614+0000 7f1bb2ec2740 1 --2- 172.21.15.43:0/2155259022 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55c3143ccfa0 0x55c3143cf390 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:41.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.614+0000 7f1bb2ec2740 1 --2- 172.21.15.43:0/2155259022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c3143bce50 0x55c3143ba830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:41.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.614+0000 7f1bb2ec2740 1 --2- 172.21.15.43:0/2155259022 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c3143b2590 0x55c3143ba2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:41.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.614+0000 7f1bb2ec2740 1 --2- 172.21.15.43:0/2155259022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3143b1c70 0x55c3143b9db0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:41.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.614+0000 7f1b8f400640 0 client.26216 ms_handle_reset on v2:172.21.15.151:6838/3794104313 2024-09-19T05:09:41.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.614+0000 7f1bb2ec2740 1 -- 172.21.15.43:0/2155259022 >> 172.21.15.43:0/2155259022 conn(0x55c3143ae200 msgr2=0x55c3143b3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:41.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.614+0000 7f1bb2ec2740 1 -- 172.21.15.43:0/2155259022 shutdown_connections 2024-09-19T05:09:41.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.614+0000 7f1bb2ec2740 1 -- 172.21.15.43:0/2155259022 wait complete. 2024-09-19T05:09:41.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.614+0000 7f1bb2ec2740 20 client.26216 trim_cache size 0 max 16384 2024-09-19T05:09:41.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.628+0000 7f1bb2ec2740 1 Processor -- start 2024-09-19T05:09:41.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.628+0000 7f1bb2ec2740 1 -- start start 2024-09-19T05:09:41.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.628+0000 7f1bb2ec2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c3143b1c70 0x55c3143bc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:41.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.628+0000 7f1bb2ec2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3143b2590 0x55c3143d4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:41.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.628+0000 7f1bb2ec2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c3143bce50 0x55c3143d4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:41.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.628+0000 7f1bb2ec2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba81b3dd0 con 0x55c3143b1c70 2024-09-19T05:09:41.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.628+0000 7f1bb2ec2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba81b5910 con 0x55c3143b2590 2024-09-19T05:09:41.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.628+0000 7f1bb2ec2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba81b44a0 con 0x55c3143bce50 2024-09-19T05:09:41.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.628+0000 7f1bada00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c3143b1c70 0x55c3143bc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:41.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.628+0000 7f1bada00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c3143b1c70 0x55c3143bc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40906/0 (socket says 172.21.15.43:40906) 2024-09-19T05:09:41.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.628+0000 7f1bada00640 1 -- 172.21.15.43:0/988469413 learned_addr learned my addr 172.21.15.43:0/988469413 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:41.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.628+0000 7f1bad000640 1 --2- 172.21.15.43:0/988469413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3143b2590 0x55c3143d4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:41.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.629+0000 7f1bada00640 1 -- 172.21.15.43:0/988469413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c3143bce50 msgr2=0x55c3143d4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:41.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.629+0000 7f1bada00640 1 --2- 172.21.15.43:0/988469413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c3143bce50 0x55c3143d4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:41.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.629+0000 7f1bada00640 1 -- 172.21.15.43:0/988469413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3143b2590 msgr2=0x55c3143d4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:41.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.629+0000 7f1bada00640 1 --2- 172.21.15.43:0/988469413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3143b2590 0x55c3143d4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:41.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.629+0000 7f1bada00640 1 -- 172.21.15.43:0/988469413 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3143a9770 con 0x55c3143b1c70 2024-09-19T05:09:41.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.629+0000 7f1bada00640 1 --2- 172.21.15.43:0/988469413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c3143b1c70 0x55c3143bc0e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c011850 tx=0x7f1b9c00e3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=213cd426bc25ac05 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:41.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.629+0000 7f1b9a000640 1 -- 172.21.15.43:0/988469413 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c015070 con 0x55c3143b1c70 2024-09-19T05:09:41.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.629+0000 7f1b9a000640 1 -- 172.21.15.43:0/988469413 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b9c01f040 con 0x55c3143b1c70 2024-09-19T05:09:41.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.629+0000 7f1b9a000640 1 -- 172.21.15.43:0/988469413 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c01e070 con 0x55c3143b1c70 2024-09-19T05:09:41.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.630+0000 7f1bb2ec2740 1 -- 172.21.15.43:0/988469413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c3143b1c70 msgr2=0x55c3143bc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:41.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.630+0000 7f1bb2ec2740 1 --2- 172.21.15.43:0/988469413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c3143b1c70 0x55c3143bc0e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c011850 tx=0x7f1b9c00e3b0 comp rx=0 tx=0).stop 2024-09-19T05:09:41.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.630+0000 7f1bb2ec2740 1 -- 172.21.15.43:0/988469413 shutdown_connections 2024-09-19T05:09:41.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.630+0000 7f1bb2ec2740 1 --2- 172.21.15.43:0/988469413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c3143bce50 0x55c3143d4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:41.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.630+0000 7f1bb2ec2740 1 --2- 172.21.15.43:0/988469413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3143b2590 0x55c3143d4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:41.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.630+0000 7f1bb2ec2740 1 --2- 172.21.15.43:0/988469413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c3143b1c70 0x55c3143bc0e0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:41.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.630+0000 7f1bb2ec2740 1 -- 172.21.15.43:0/988469413 >> 172.21.15.43:0/988469413 conn(0x55c3143ae200 msgr2=0x55c3143b3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:41.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.630+0000 7f1bb2ec2740 1 -- 172.21.15.43:0/988469413 shutdown_connections 2024-09-19T05:09:41.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.630+0000 7f1bb2ec2740 1 -- 172.21.15.43:0/988469413 wait complete. 2024-09-19T05:09:41.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.630+0000 7f1bb2ec2740 1 -- start start 2024-09-19T05:09:41.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.631+0000 7f1bb2ec2740 1 Processor -- start 2024-09-19T05:09:41.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.631+0000 7f1bb2ec2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c3143b1c70 0x55c3143c5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:41.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.631+0000 7f1bb2ec2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3143b2590 0x55c3143c07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:41.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.631+0000 7f1bb2ec2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c3143bce50 0x55c3143c0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:41.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.631+0000 7f1bb2ec2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba81b5240 con 0x55c3143b1c70 2024-09-19T05:09:41.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.631+0000 7f1bb2ec2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba81b4b70 con 0x55c3143b2590 2024-09-19T05:09:41.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.631+0000 7f1bb2ec2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba81b1db0 con 0x55c3143bce50 2024-09-19T05:09:41.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.631+0000 7f1bb2ec2740 10 client.0 _ll_register_callbacks cb 0x7f1b9c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:41.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.631+0000 7f1bada00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c3143b1c70 0x55c3143c5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:41.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.631+0000 7f1bada00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c3143b1c70 0x55c3143c5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40914/0 (socket says 172.21.15.43:40914) 2024-09-19T05:09:41.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.632+0000 7f1bada00640 1 -- 172.21.15.43:0/3675044291 learned_addr learned my addr 172.21.15.43:0/3675044291 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:41.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.632+0000 7f1bad000640 1 --2- 172.21.15.43:0/3675044291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3143b2590 0x55c3143c07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:41.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.632+0000 7f1bae400640 1 --2- 172.21.15.43:0/3675044291 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c3143bce50 0x55c3143c0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:41.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.632+0000 7f1bada00640 1 -- 172.21.15.43:0/3675044291 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c3143bce50 msgr2=0x55c3143c0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:41.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.632+0000 7f1bada00640 1 --2- 172.21.15.43:0/3675044291 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c3143bce50 0x55c3143c0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:41.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.632+0000 7f1bada00640 1 -- 172.21.15.43:0/3675044291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3143b2590 msgr2=0x55c3143c07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:41.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.632+0000 7f1bada00640 1 --2- 172.21.15.43:0/3675044291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3143b2590 0x55c3143c07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:41.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.632+0000 7f1bada00640 1 -- 172.21.15.43:0/3675044291 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3143c14d0 con 0x55c3143b1c70 2024-09-19T05:09:41.674 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:09:41.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.632+0000 7f1bada00640 1 --2- 172.21.15.43:0/3675044291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c3143b1c70 0x55c3143c5480 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c004730 tx=0x7f1b9c009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=166377ed5f1f8bc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:41.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.632+0000 7f1b8f400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:09:41.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.632+0000 7f1b8f400640 1 -- 172.21.15.43:0/3675044291 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c015040 con 0x55c3143b1c70 2024-09-19T05:09:41.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.632+0000 7f1b8f400640 1 -- 172.21.15.43:0/3675044291 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f1b9c019df0 con 0x55c3143b1c70 2024-09-19T05:09:41.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.633+0000 7f1bb2ec2740 1 -- 172.21.15.43:0/3675044291 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b9c01f040 con 0x55c3143b1c70 2024-09-19T05:09:41.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.633+0000 7f1b8f400640 1 -- 172.21.15.43:0/3675044291 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1b9c01e040 con 0x55c3143b1c70 2024-09-19T05:09:41.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.633+0000 7f1bb2ec2740 10 client.16179 fetch_fsmap learned FSMap version 29 2024-09-19T05:09:41.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.633+0000 7f1bb2ec2740 1 -- 172.21.15.43:0/3675044291 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c3144379d0 con 0x55c3143b1c70 2024-09-19T05:09:41.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.633+0000 7f1b8f400640 1 -- 172.21.15.43:0/3675044291 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b9c01f040 con 0x55c3143b1c70 2024-09-19T05:09:41.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.633+0000 7f1b8f400640 1 -- 172.21.15.43:0/3675044291 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1b9c03c710 con 0x55c3143b1c70 2024-09-19T05:09:41.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.633+0000 7f1bb2ec2740 10 client.16179 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:09:41.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.633+0000 7f1bb2ec2740 10 client.16179 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T05:09:41.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.633+0000 7f1bb2ec2740 20 client.16179 populate_metadata read hostname 'smithi043' 2024-09-19T05:09:41.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.633+0000 7f1bb2ec2740 1 --2- 172.21.15.43:0/3675044291 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55c314438020 0x55c31443a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:41.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.633+0000 7f1bb2ec2740 4 client.16179 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:09:41.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.633+0000 7f1bb2ec2740 1 -- 172.21.15.43:0/3675044291 --> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c31431fe60 con 0x55c314438020 2024-09-19T05:09:41.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.634+0000 7f1bad000640 1 --2- 172.21.15.43:0/3675044291 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55c314438020 0x55c31443a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:41.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.634+0000 7f1bad000640 1 --2- 172.21.15.43:0/3675044291 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55c314438020 0x55c31443a410 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3904ee8c817f97e6 server_cookie=b23984bf739121dc in_seq=0 out_seq=0 2024-09-19T05:09:41.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.634+0000 7f1b8f400640 10 client.16179 ms_handle_connect on v2:172.21.15.151:6838/3794104313 2024-09-19T05:09:41.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.635+0000 7f1b8f400640 1 -- 172.21.15.43:0/3675044291 <== mds.2 v2:172.21.15.151:6838/3794104313 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c31431fe60 con 0x55c314438020 2024-09-19T05:09:41.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.635+0000 7f1b8f400640 10 client.16179 handle_command_reply: tid=0 2024-09-19T05:09:41.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.635+0000 7f1bb2ec2740 1 client.16179 shutdown 2024-09-19T05:09:41.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.635+0000 7f1bb2ec2740 1 -- 172.21.15.43:0/3675044291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c3143b1c70 msgr2=0x55c3143c5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:41.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.635+0000 7f1bb2ec2740 1 --2- 172.21.15.43:0/3675044291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c3143b1c70 0x55c3143c5480 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c004730 tx=0x7f1b9c009ed0 comp rx=0 tx=0).stop 2024-09-19T05:09:41.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.635+0000 7f1bb2ec2740 1 -- 172.21.15.43:0/3675044291 shutdown_connections 2024-09-19T05:09:41.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.635+0000 7f1bb2ec2740 1 --2- 172.21.15.43:0/3675044291 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55c314438020 0x55c31443a410 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:41.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.635+0000 7f1bb2ec2740 1 --2- 172.21.15.43:0/3675044291 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c3143bce50 0x55c3143c0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:41.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.635+0000 7f1bb2ec2740 1 --2- 172.21.15.43:0/3675044291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3143b2590 0x55c3143c07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:41.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.635+0000 7f1bb2ec2740 1 --2- 172.21.15.43:0/3675044291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c3143b1c70 0x55c3143c5480 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:41.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.635+0000 7f1bb2ec2740 1 -- 172.21.15.43:0/3675044291 >> 172.21.15.43:0/3675044291 conn(0x55c3143ae200 msgr2=0x55c3143b8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:41.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.635+0000 7f1b8f400640 0 client.16179 ms_handle_reset on v2:172.21.15.151:6838/3794104313 2024-09-19T05:09:41.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.635+0000 7f1bb2ec2740 1 -- 172.21.15.43:0/3675044291 shutdown_connections 2024-09-19T05:09:41.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.635+0000 7f1bb2ec2740 1 -- 172.21.15.43:0/3675044291 wait complete. 2024-09-19T05:09:41.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.635+0000 7f1bb2ec2740 20 client.16179 trim_cache size 0 max 16384 2024-09-19T05:09:41.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.636+0000 7f1bb0400640 1 -- 172.21.15.43:0/2037337412 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f1b900a33b0 msgr2=0x7f1b900a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:41.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.636+0000 7f1bb0400640 1 --2- 172.21.15.43:0/2037337412 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f1b900a33b0 0x7f1b900a5870 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1b9402a930 tx=0x7f1b940058d0 comp rx=0 tx=0).stop 2024-09-19T05:09:41.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.636+0000 7f1bb0400640 1 -- 172.21.15.43:0/2037337412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ba8109120 msgr2=0x7f1ba8077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:41.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.636+0000 7f1bb0400640 1 --2- 172.21.15.43:0/2037337412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ba8109120 0x7f1ba8077180 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1ba000c370 tx=0x7f1ba000c830 comp rx=0 tx=0).stop 2024-09-19T05:09:41.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.636+0000 7f1bb0400640 1 -- 172.21.15.43:0/2037337412 shutdown_connections 2024-09-19T05:09:41.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.636+0000 7f1bb0400640 1 --2- 172.21.15.43:0/2037337412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ba8109120 0x7f1ba8077180 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:41.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.636+0000 7f1bb0400640 1 --2- 172.21.15.43:0/2037337412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ba81049a0 0x7f1ba8076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:41.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.636+0000 7f1bb0400640 1 --2- 172.21.15.43:0/2037337412 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f1b900a33b0 0x7f1b900a5870 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:41.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.636+0000 7f1bb0400640 1 --2- 172.21.15.43:0/2037337412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ba8103ff0 0x7f1ba8076700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:41.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.636+0000 7f1bb0400640 1 -- 172.21.15.43:0/2037337412 >> 172.21.15.43:0/2037337412 conn(0x7f1ba80ffa90 msgr2=0x7f1ba8100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:41.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.636+0000 7f1bb0400640 1 -- 172.21.15.43:0/2037337412 shutdown_connections 2024-09-19T05:09:41.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:41.636+0000 7f1bb0400640 1 -- 172.21.15.43:0/2037337412 wait complete. 2024-09-19T05:09:41.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:41 smithi043 ceph-mon[30800]: pgmap v154: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:09:41.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:41 smithi151 ceph-mon[35425]: pgmap v154: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:09:41.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:41 smithi096 ceph-mon[35085]: pgmap v154: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:09:42.647 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T05:09:43.025 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:09:43.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.610+0000 7f6e7d400640 1 -- 172.21.15.43:0/846521682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e781047e0 msgr2=0x7f6e78108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:43.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.610+0000 7f6e7d400640 1 --2- 172.21.15.43:0/846521682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e781047e0 0x7f6e78108830 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6e64009900 tx=0x7f6e6402a3f0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.610+0000 7f6e7d400640 1 -- 172.21.15.43:0/846521682 shutdown_connections 2024-09-19T05:09:43.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.610+0000 7f6e7d400640 1 --2- 172.21.15.43:0/846521682 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e78108f60 0x7f6e78110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.610+0000 7f6e7d400640 1 --2- 172.21.15.43:0/846521682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e781047e0 0x7f6e78108830 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.610+0000 7f6e7d400640 1 --2- 172.21.15.43:0/846521682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e78103e30 0x7f6e78104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.610+0000 7f6e7d400640 1 -- 172.21.15.43:0/846521682 >> 172.21.15.43:0/846521682 conn(0x7f6e780ff910 msgr2=0x7f6e78101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:43.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.610+0000 7f6e7d400640 1 -- 172.21.15.43:0/846521682 shutdown_connections 2024-09-19T05:09:43.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.610+0000 7f6e7d400640 1 -- 172.21.15.43:0/846521682 wait complete. 2024-09-19T05:09:43.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.611+0000 7f6e7d400640 1 Processor -- start 2024-09-19T05:09:43.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.612+0000 7f6e7d400640 1 -- start start 2024-09-19T05:09:43.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.612+0000 7f6e7d400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e78103e30 0x7f6e7819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:43.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.612+0000 7f6e7d400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e781047e0 0x7f6e7819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:43.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.612+0000 7f6e7d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e78108f60 0x7f6e7819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:43.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.612+0000 7f6e7d400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6e78113fd0 con 0x7f6e78108f60 2024-09-19T05:09:43.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.612+0000 7f6e7d400640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6e78113e50 con 0x7f6e78103e30 2024-09-19T05:09:43.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.612+0000 7f6e7d400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6e78114150 con 0x7f6e781047e0 2024-09-19T05:09:43.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.612+0000 7f6e7ca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e78108f60 0x7f6e7819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:43.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.612+0000 7f6e77e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e78103e30 0x7f6e7819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:43.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.612+0000 7f6e7ca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e78108f60 0x7f6e7819ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40944/0 (socket says 172.21.15.43:40944) 2024-09-19T05:09:43.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.612+0000 7f6e77400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e781047e0 0x7f6e7819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:43.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.612+0000 7f6e7ca00640 1 -- 172.21.15.43:0/3209768322 learned_addr learned my addr 172.21.15.43:0/3209768322 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:43.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.613+0000 7f6e77e00640 1 -- 172.21.15.43:0/3209768322 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e781047e0 msgr2=0x7f6e7819a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:43.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.613+0000 7f6e77e00640 1 --2- 172.21.15.43:0/3209768322 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e781047e0 0x7f6e7819a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.613+0000 7f6e77e00640 1 -- 172.21.15.43:0/3209768322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e78108f60 msgr2=0x7f6e7819ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:43.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.613+0000 7f6e77e00640 1 --2- 172.21.15.43:0/3209768322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e78108f60 0x7f6e7819ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.613+0000 7f6e77e00640 1 -- 172.21.15.43:0/3209768322 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e7819f440 con 0x7f6e78103e30 2024-09-19T05:09:43.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.613+0000 7f6e77e00640 1 --2- 172.21.15.43:0/3209768322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e78103e30 0x7f6e7819a490 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6e68005fd0 tx=0x7f6e6800dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:43.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.613+0000 7f6e74c00640 1 -- 172.21.15.43:0/3209768322 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e68009c30 con 0x7f6e78103e30 2024-09-19T05:09:43.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.613+0000 7f6e74c00640 1 -- 172.21.15.43:0/3209768322 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e68009dd0 con 0x7f6e78103e30 2024-09-19T05:09:43.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.613+0000 7f6e7d400640 1 -- 172.21.15.43:0/3209768322 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e7819f730 con 0x7f6e78103e30 2024-09-19T05:09:43.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.613+0000 7f6e74c00640 1 -- 172.21.15.43:0/3209768322 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e68010460 con 0x7f6e78103e30 2024-09-19T05:09:43.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.613+0000 7f6e7d400640 1 -- 172.21.15.43:0/3209768322 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e781a7010 con 0x7f6e78103e30 2024-09-19T05:09:43.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.618+0000 7f6e815f1740 1 Processor -- start 2024-09-19T05:09:43.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.618+0000 7f6e815f1740 1 -- start start 2024-09-19T05:09:43.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.618+0000 7f6e74c00640 1 -- 172.21.15.43:0/3209768322 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6e68003f50 con 0x7f6e78103e30 2024-09-19T05:09:43.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.618+0000 7f6e815f1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e136de7e40 0x55e136de8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:43.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.618+0000 7f6e815f1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e136de8760 0x55e136df2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:43.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.618+0000 7f6e815f1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e136df3020 0x55e136df5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:43.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.618+0000 7f6e815f1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e136d4e3e0 con 0x55e136de8760 2024-09-19T05:09:43.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.618+0000 7f6e815f1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e136cf5d80 con 0x55e136df3020 2024-09-19T05:09:43.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.618+0000 7f6e815f1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e136d4a3a0 con 0x55e136de7e40 2024-09-19T05:09:43.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.619+0000 7f6e7ca00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e136de7e40 0x55e136de8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:43.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.619+0000 7f6e77e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e136de8760 0x55e136df2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:43.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.619+0000 7f6e7ca00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e136de7e40 0x55e136de8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:45708/0 (socket says 172.21.15.43:45708) 2024-09-19T05:09:43.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.619+0000 7f6e7ca00640 1 -- 172.21.15.43:0/1087114484 learned_addr learned my addr 172.21.15.43:0/1087114484 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:43.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.619+0000 7f6e77400640 1 --2- 172.21.15.43:0/1087114484 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e136df3020 0x55e136df5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:43.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.619+0000 7f6e7ca00640 1 -- 172.21.15.43:0/1087114484 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e136df3020 msgr2=0x55e136df5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:43.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.619+0000 7f6e7ca00640 1 --2- 172.21.15.43:0/1087114484 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e136df3020 0x55e136df5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.619+0000 7f6e7ca00640 1 -- 172.21.15.43:0/1087114484 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e136de8760 msgr2=0x55e136df2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:43.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.619+0000 7f6e7ca00640 1 --2- 172.21.15.43:0/1087114484 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e136de8760 0x55e136df2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.619+0000 7f6e7ca00640 1 -- 172.21.15.43:0/1087114484 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e136d14ae0 con 0x55e136de7e40 2024-09-19T05:09:43.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.619+0000 7f6e7ca00640 1 --2- 172.21.15.43:0/1087114484 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e136de7e40 0x55e136de8220 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6e70011470 tx=0x7f6e70011930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae4eb4522908d5b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:43.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.620+0000 7f6e6d200640 1 -- 172.21.15.43:0/1087114484 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e70004b20 con 0x55e136de7e40 2024-09-19T05:09:43.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.620+0000 7f6e6d200640 1 -- 172.21.15.43:0/1087114484 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e70004cc0 con 0x55e136de7e40 2024-09-19T05:09:43.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.620+0000 7f6e6d200640 1 -- 172.21.15.43:0/1087114484 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e70005450 con 0x55e136de7e40 2024-09-19T05:09:43.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.620+0000 7f6e74c00640 1 --2- 172.21.15.43:0/3209768322 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f6e5c0a33b0 0x7f6e5c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:43.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.620+0000 7f6e74c00640 1 -- 172.21.15.43:0/3209768322 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f6e680cfba0 con 0x7f6e78103e30 2024-09-19T05:09:43.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.621+0000 7f6e815f1740 1 -- 172.21.15.43:0/1087114484 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e136de7e40 msgr2=0x55e136de8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:43.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.621+0000 7f6e815f1740 1 --2- 172.21.15.43:0/1087114484 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e136de7e40 0x55e136de8220 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6e70011470 tx=0x7f6e70011930 comp rx=0 tx=0).stop 2024-09-19T05:09:43.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.621+0000 7f6e77e00640 1 --2- 172.21.15.43:0/3209768322 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f6e5c0a33b0 0x7f6e5c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:43.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.621+0000 7f6e815f1740 1 -- 172.21.15.43:0/1087114484 shutdown_connections 2024-09-19T05:09:43.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.621+0000 7f6e815f1740 1 --2- 172.21.15.43:0/1087114484 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e136df3020 0x55e136df5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.621+0000 7f6e815f1740 1 --2- 172.21.15.43:0/1087114484 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e136de8760 0x55e136df2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.621+0000 7f6e815f1740 1 --2- 172.21.15.43:0/1087114484 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e136de7e40 0x55e136de8220 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.621+0000 7f6e815f1740 1 -- 172.21.15.43:0/1087114484 >> 172.21.15.43:0/1087114484 conn(0x55e136de43d0 msgr2=0x55e136de47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:43.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.621+0000 7f6e815f1740 1 -- 172.21.15.43:0/1087114484 shutdown_connections 2024-09-19T05:09:43.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.621+0000 7f6e815f1740 1 -- 172.21.15.43:0/1087114484 wait complete. 2024-09-19T05:09:43.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.621+0000 7f6e77e00640 1 --2- 172.21.15.43:0/3209768322 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f6e5c0a33b0 0x7f6e5c0a5870 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6e680cd3b0 tx=0x7f6e680d36f0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:43.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.621+0000 7f6e815f1740 1 -- start start 2024-09-19T05:09:43.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.622+0000 7f6e815f1740 1 Processor -- start 2024-09-19T05:09:43.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.622+0000 7f6e815f1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e136de7e40 0x55e136defec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:43.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.622+0000 7f6e815f1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e136de8760 0x55e136df0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:43.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.622+0000 7f6e815f1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e136df3020 0x55e136df0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:43.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.622+0000 7f6e815f1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e136df5bf0 con 0x55e136df3020 2024-09-19T05:09:43.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.622+0000 7f6e815f1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e136d090f0 con 0x55e136de8760 2024-09-19T05:09:43.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.622+0000 7f6e815f1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e136df5d70 con 0x55e136de7e40 2024-09-19T05:09:43.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.622+0000 7f6e815f1740 10 client.0 _ll_register_callbacks cb 0x55e136cf0bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:43.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.622+0000 7f6e77400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e136de8760 0x55e136df0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:43.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.622+0000 7f6e77400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e136de8760 0x55e136df0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:52580/0 (socket says 172.21.15.43:52580) 2024-09-19T05:09:43.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.622+0000 7f6e77400640 1 -- 172.21.15.43:0/311942183 learned_addr learned my addr 172.21.15.43:0/311942183 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:43.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.623+0000 7f6e7ca00640 1 --2- 172.21.15.43:0/311942183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e136de7e40 0x55e136defec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:43.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.623+0000 7f6e7ca00640 1 --2- 172.21.15.43:0/311942183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e136df3020 0x55e136df0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:43.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.623+0000 7f6e77400640 1 -- 172.21.15.43:0/311942183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e136de7e40 msgr2=0x55e136defec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:43.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.623+0000 7f6e77400640 1 --2- 172.21.15.43:0/311942183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e136de7e40 0x55e136defec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.623+0000 7f6e77400640 1 -- 172.21.15.43:0/311942183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e136df3020 msgr2=0x55e136df0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:43.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.623+0000 7f6e77400640 1 --2- 172.21.15.43:0/311942183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e136df3020 0x55e136df0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.623+0000 7f6e77400640 1 -- 172.21.15.43:0/311942183 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e136abf3c0 con 0x55e136de8760 2024-09-19T05:09:43.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.623+0000 7f6e77400640 1 --2- 172.21.15.43:0/311942183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e136de8760 0x55e136df0400 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6e64033000 tx=0x7f6e64004420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed07901262ece21e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:43.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.623+0000 7f6e61600640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:09:43.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.624+0000 7f6e61600640 1 -- 172.21.15.43:0/311942183 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e64035040 con 0x55e136de8760 2024-09-19T05:09:43.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.624+0000 7f6e61600640 1 -- 172.21.15.43:0/311942183 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f6e64005450 con 0x55e136de8760 2024-09-19T05:09:43.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.624+0000 7f6e815f1740 1 -- 172.21.15.43:0/311942183 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e70004cc0 con 0x55e136de8760 2024-09-19T05:09:43.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.624+0000 7f6e61600640 1 -- 172.21.15.43:0/311942183 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6e64005650 con 0x55e136de8760 2024-09-19T05:09:43.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.624+0000 7f6e815f1740 10 client.26234 fetch_fsmap learned FSMap version 29 2024-09-19T05:09:43.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.624+0000 7f6e815f1740 1 -- 172.21.15.43:0/311942183 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e136e01be0 con 0x55e136de8760 2024-09-19T05:09:43.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.624+0000 7f6e61600640 1 -- 172.21.15.43:0/311942183 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e70004cc0 con 0x55e136de8760 2024-09-19T05:09:43.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.624+0000 7f6e61600640 1 -- 172.21.15.43:0/311942183 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6e64031b40 con 0x55e136de8760 2024-09-19T05:09:43.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.624+0000 7f6e815f1740 10 client.26234 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:09:43.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.624+0000 7f6e815f1740 10 client.26234 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-19T05:09:43.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.624+0000 7f6e815f1740 20 client.26234 populate_metadata read hostname 'smithi043' 2024-09-19T05:09:43.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.624+0000 7f6e815f1740 1 --2- 172.21.15.43:0/311942183 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55e136e030a0 0x55e136e054c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:43.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.624+0000 7f6e815f1740 4 client.26234 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:09:43.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.624+0000 7f6e815f1740 1 -- 172.21.15.43:0/311942183 --> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e136d5b950 con 0x55e136e030a0 2024-09-19T05:09:43.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.625+0000 7f6e7ca00640 1 --2- 172.21.15.43:0/311942183 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55e136e030a0 0x55e136e054c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:43.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.625+0000 7f6e7ca00640 1 --2- 172.21.15.43:0/311942183 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55e136e030a0 0x55e136e054c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3b3cac402baf8c35 server_cookie=890bf4a6bba843bf in_seq=0 out_seq=0 2024-09-19T05:09:43.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.625+0000 7f6e61600640 10 client.26234 ms_handle_connect on v2:172.21.15.43:6838/3404555030 2024-09-19T05:09:43.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.626+0000 7f6e61600640 1 -- 172.21.15.43:0/311942183 <== mds.1 v2:172.21.15.43:6838/3404555030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e136d5b950 con 0x55e136e030a0 2024-09-19T05:09:43.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.626+0000 7f6e61600640 10 client.26234 handle_command_reply: tid=0 2024-09-19T05:09:43.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.626+0000 7f6e815f1740 1 client.26234 shutdown 2024-09-19T05:09:43.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.626+0000 7f6e815f1740 1 -- 172.21.15.43:0/311942183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e136de8760 msgr2=0x55e136df0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:43.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.626+0000 7f6e815f1740 1 --2- 172.21.15.43:0/311942183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e136de8760 0x55e136df0400 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6e64033000 tx=0x7f6e64004420 comp rx=0 tx=0).stop 2024-09-19T05:09:43.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.626+0000 7f6e815f1740 1 -- 172.21.15.43:0/311942183 shutdown_connections 2024-09-19T05:09:43.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.626+0000 7f6e815f1740 1 --2- 172.21.15.43:0/311942183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e136df3020 0x55e136df0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.626+0000 7f6e815f1740 1 --2- 172.21.15.43:0/311942183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e136de8760 0x55e136df0400 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.626+0000 7f6e815f1740 1 --2- 172.21.15.43:0/311942183 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55e136e030a0 0x55e136e054c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.626+0000 7f6e815f1740 1 --2- 172.21.15.43:0/311942183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e136de7e40 0x55e136defec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.626+0000 7f6e815f1740 1 -- 172.21.15.43:0/311942183 >> 172.21.15.43:0/311942183 conn(0x55e136de43d0 msgr2=0x55e136dea100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:43.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.626+0000 7f6e61600640 0 client.26234 ms_handle_reset on v2:172.21.15.43:6838/3404555030 2024-09-19T05:09:43.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.627+0000 7f6e815f1740 1 -- 172.21.15.43:0/311942183 shutdown_connections 2024-09-19T05:09:43.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.627+0000 7f6e815f1740 1 -- 172.21.15.43:0/311942183 wait complete. 2024-09-19T05:09:43.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.627+0000 7f6e815f1740 20 client.26234 trim_cache size 0 max 16384 2024-09-19T05:09:43.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.640+0000 7f6e815f1740 1 Processor -- start 2024-09-19T05:09:43.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.640+0000 7f6e815f1740 1 -- start start 2024-09-19T05:09:43.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.641+0000 7f6e815f1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e136de7e40 0x55e136df0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:43.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.641+0000 7f6e815f1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e136de8760 0x55e136df1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:43.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.641+0000 7f6e815f1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e136df3020 0x55e136e0ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:43.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.641+0000 7f6e815f1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6e781af960 con 0x55e136df3020 2024-09-19T05:09:43.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.641+0000 7f6e815f1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6e781b14a0 con 0x55e136de7e40 2024-09-19T05:09:43.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.641+0000 7f6e815f1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6e781b0030 con 0x55e136de8760 2024-09-19T05:09:43.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.641+0000 7f6e7ca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e136df3020 0x55e136e0ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:43.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.641+0000 7f6e7ca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e136df3020 0x55e136e0ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40954/0 (socket says 172.21.15.43:40954) 2024-09-19T05:09:43.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.641+0000 7f6e7ca00640 1 -- 172.21.15.43:0/1840269914 learned_addr learned my addr 172.21.15.43:0/1840269914 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:43.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.641+0000 7f6e7ca00640 1 --2- 172.21.15.43:0/1840269914 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e136de7e40 0x55e136df0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:43.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.641+0000 7f6e77400640 1 --2- 172.21.15.43:0/1840269914 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e136de8760 0x55e136df1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:43.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.641+0000 7f6e7ca00640 1 -- 172.21.15.43:0/1840269914 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e136de8760 msgr2=0x55e136df1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:43.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.641+0000 7f6e7ca00640 1 --2- 172.21.15.43:0/1840269914 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e136de8760 0x55e136df1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.641+0000 7f6e7ca00640 1 -- 172.21.15.43:0/1840269914 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e136de7e40 msgr2=0x55e136df0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:43.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.641+0000 7f6e7ca00640 1 --2- 172.21.15.43:0/1840269914 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e136de7e40 0x55e136df0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.641+0000 7f6e7ca00640 1 -- 172.21.15.43:0/1840269914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e136de7c70 con 0x55e136df3020 2024-09-19T05:09:43.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.642+0000 7f6e7ca00640 1 --2- 172.21.15.43:0/1840269914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e136df3020 0x55e136e0ad90 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6e70005060 tx=0x7f6e70009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfeaea684fdea178 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:43.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.642+0000 7f6e63e00640 1 -- 172.21.15.43:0/1840269914 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e7000f600 con 0x55e136df3020 2024-09-19T05:09:43.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.642+0000 7f6e63e00640 1 -- 172.21.15.43:0/1840269914 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e7000fc80 con 0x55e136df3020 2024-09-19T05:09:43.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.642+0000 7f6e63e00640 1 -- 172.21.15.43:0/1840269914 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e70005930 con 0x55e136df3020 2024-09-19T05:09:43.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.643+0000 7f6e815f1740 1 -- 172.21.15.43:0/1840269914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e136df3020 msgr2=0x55e136e0ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:43.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.643+0000 7f6e815f1740 1 --2- 172.21.15.43:0/1840269914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e136df3020 0x55e136e0ad90 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6e70005060 tx=0x7f6e70009500 comp rx=0 tx=0).stop 2024-09-19T05:09:43.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.643+0000 7f6e815f1740 1 -- 172.21.15.43:0/1840269914 shutdown_connections 2024-09-19T05:09:43.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.643+0000 7f6e815f1740 1 --2- 172.21.15.43:0/1840269914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e136df3020 0x55e136e0ad90 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.643+0000 7f6e815f1740 1 --2- 172.21.15.43:0/1840269914 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e136de8760 0x55e136df1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.643+0000 7f6e815f1740 1 --2- 172.21.15.43:0/1840269914 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e136de7e40 0x55e136df0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.643+0000 7f6e815f1740 1 -- 172.21.15.43:0/1840269914 >> 172.21.15.43:0/1840269914 conn(0x55e136de43d0 msgr2=0x55e136df1720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:43.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.643+0000 7f6e815f1740 1 -- 172.21.15.43:0/1840269914 shutdown_connections 2024-09-19T05:09:43.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.643+0000 7f6e815f1740 1 -- 172.21.15.43:0/1840269914 wait complete. 2024-09-19T05:09:43.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.643+0000 7f6e815f1740 1 -- start start 2024-09-19T05:09:43.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.643+0000 7f6e815f1740 1 Processor -- start 2024-09-19T05:09:43.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.644+0000 7f6e815f1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e136de7e40 0x55e136dfb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:43.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.644+0000 7f6e815f1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e136de8760 0x55e136df6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:43.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.644+0000 7f6e815f1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e136df3020 0x55e136df7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:43.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.644+0000 7f6e815f1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6e781b0dd0 con 0x55e136de7e40 2024-09-19T05:09:43.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.644+0000 7f6e815f1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6e781b0700 con 0x55e136df3020 2024-09-19T05:09:43.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.644+0000 7f6e815f1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6e781ad940 con 0x55e136de8760 2024-09-19T05:09:43.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.644+0000 7f6e815f1740 10 client.0 _ll_register_callbacks cb 0x7f6e640040b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:43.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.644+0000 7f6e7ca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e136de7e40 0x55e136dfb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:43.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.644+0000 7f6e7ca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e136de7e40 0x55e136dfb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40958/0 (socket says 172.21.15.43:40958) 2024-09-19T05:09:43.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.644+0000 7f6e7ca00640 1 -- 172.21.15.43:0/2109124318 learned_addr learned my addr 172.21.15.43:0/2109124318 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:43.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.644+0000 7f6e77400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e136de8760 0x55e136df6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:43.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.645+0000 7f6e7ca00640 1 --2- 172.21.15.43:0/2109124318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e136df3020 0x55e136df7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:43.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.645+0000 7f6e77400640 1 -- 172.21.15.43:0/2109124318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e136df3020 msgr2=0x55e136df7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:43.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.645+0000 7f6e77400640 1 --2- 172.21.15.43:0/2109124318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e136df3020 0x55e136df7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.645+0000 7f6e77400640 1 -- 172.21.15.43:0/2109124318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e136de7e40 msgr2=0x55e136dfb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:43.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.645+0000 7f6e77400640 1 --2- 172.21.15.43:0/2109124318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e136de7e40 0x55e136dfb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.645+0000 7f6e77400640 1 -- 172.21.15.43:0/2109124318 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e136deacb0 con 0x55e136de8760 2024-09-19T05:09:43.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.645+0000 7f6e7ca00640 1 --2- 172.21.15.43:0/2109124318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e136de7e40 0x55e136dfb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:43.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.645+0000 7f6e77400640 1 --2- 172.21.15.43:0/2109124318 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e136de8760 0x55e136df6bd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6e64031ee0 tx=0x7f6e64003d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87f306a5a8a8f355 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:43.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.645+0000 7f6e61600640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:09:43.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.645+0000 7f6e61600640 1 -- 172.21.15.43:0/2109124318 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e64035070 con 0x55e136de8760 2024-09-19T05:09:43.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.646+0000 7f6e815f1740 1 -- 172.21.15.43:0/2109124318 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e7000fc80 con 0x55e136de8760 2024-09-19T05:09:43.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.646+0000 7f6e61600640 1 -- 172.21.15.43:0/2109124318 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f6e640317b0 con 0x55e136de8760 2024-09-19T05:09:43.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.646+0000 7f6e61600640 1 -- 172.21.15.43:0/2109124318 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6e64053040 con 0x55e136de8760 2024-09-19T05:09:43.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.646+0000 7f6e815f1740 10 client.26173 fetch_fsmap learned FSMap version 29 2024-09-19T05:09:43.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.646+0000 7f6e815f1740 1 -- 172.21.15.43:0/2109124318 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e136de5480 con 0x55e136de8760 2024-09-19T05:09:43.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.646+0000 7f6e61600640 1 -- 172.21.15.43:0/2109124318 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e7000fc80 con 0x55e136de8760 2024-09-19T05:09:43.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.646+0000 7f6e61600640 1 -- 172.21.15.43:0/2109124318 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6e64004ba0 con 0x55e136de8760 2024-09-19T05:09:43.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.646+0000 7f6e815f1740 10 client.26173 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:09:43.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.646+0000 7f6e815f1740 10 client.26173 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-19T05:09:43.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.646+0000 7f6e815f1740 20 client.26173 populate_metadata read hostname 'smithi043' 2024-09-19T05:09:43.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.647+0000 7f6e815f1740 1 --2- 172.21.15.43:0/2109124318 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55e136e6e2e0 0x55e136e70700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:43.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.647+0000 7f6e815f1740 4 client.26173 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:09:43.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.647+0000 7f6e815f1740 1 -- 172.21.15.43:0/2109124318 --> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e136d55e60 con 0x55e136e6e2e0 2024-09-19T05:09:43.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.647+0000 7f6e7ca00640 1 --2- 172.21.15.43:0/2109124318 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55e136e6e2e0 0x55e136e70700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:43.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.647+0000 7f6e7ca00640 1 --2- 172.21.15.43:0/2109124318 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55e136e6e2e0 0x55e136e70700 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e13fea3151547e59 server_cookie=c0983bbc1295ce63 in_seq=0 out_seq=0 2024-09-19T05:09:43.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.647+0000 7f6e61600640 10 client.26173 ms_handle_connect on v2:172.21.15.43:6838/3404555030 2024-09-19T05:09:43.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.648+0000 7f6e61600640 1 -- 172.21.15.43:0/2109124318 <== mds.1 v2:172.21.15.43:6838/3404555030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e136d55e60 con 0x55e136e6e2e0 2024-09-19T05:09:43.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.648+0000 7f6e61600640 10 client.26173 handle_command_reply: tid=0 2024-09-19T05:09:43.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.648+0000 7f6e815f1740 1 client.26173 shutdown 2024-09-19T05:09:43.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.648+0000 7f6e815f1740 1 -- 172.21.15.43:0/2109124318 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e136de8760 msgr2=0x55e136df6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:43.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.648+0000 7f6e815f1740 1 --2- 172.21.15.43:0/2109124318 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e136de8760 0x55e136df6bd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6e64031ee0 tx=0x7f6e64003d90 comp rx=0 tx=0).stop 2024-09-19T05:09:43.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.648+0000 7f6e815f1740 1 -- 172.21.15.43:0/2109124318 shutdown_connections 2024-09-19T05:09:43.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.648+0000 7f6e815f1740 1 --2- 172.21.15.43:0/2109124318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e136df3020 0x55e136df7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.648+0000 7f6e815f1740 1 --2- 172.21.15.43:0/2109124318 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55e136e6e2e0 0x55e136e70700 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.648+0000 7f6e815f1740 1 --2- 172.21.15.43:0/2109124318 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e136de8760 0x55e136df6bd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.648+0000 7f6e815f1740 1 --2- 172.21.15.43:0/2109124318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e136de7e40 0x55e136dfb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.648+0000 7f6e61600640 0 client.26173 ms_handle_reset on v2:172.21.15.43:6838/3404555030 2024-09-19T05:09:43.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.648+0000 7f6e815f1740 1 -- 172.21.15.43:0/2109124318 >> 172.21.15.43:0/2109124318 conn(0x55e136de43d0 msgr2=0x55e136df1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:43.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.648+0000 7f6e815f1740 1 -- 172.21.15.43:0/2109124318 shutdown_connections 2024-09-19T05:09:43.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.648+0000 7f6e815f1740 1 -- 172.21.15.43:0/2109124318 wait complete. 2024-09-19T05:09:43.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.648+0000 7f6e815f1740 20 client.26173 trim_cache size 0 max 16384 2024-09-19T05:09:43.654 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:09:43.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.649+0000 7f6e7d400640 1 -- 172.21.15.43:0/3209768322 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f6e5c0a33b0 msgr2=0x7f6e5c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:43.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.649+0000 7f6e7d400640 1 --2- 172.21.15.43:0/3209768322 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f6e5c0a33b0 0x7f6e5c0a5870 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6e680cd3b0 tx=0x7f6e680d36f0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.649+0000 7f6e7d400640 1 -- 172.21.15.43:0/3209768322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e78103e30 msgr2=0x7f6e7819a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:43.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.649+0000 7f6e7d400640 1 --2- 172.21.15.43:0/3209768322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e78103e30 0x7f6e7819a490 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6e68005fd0 tx=0x7f6e6800dde0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.649+0000 7f6e7d400640 1 -- 172.21.15.43:0/3209768322 shutdown_connections 2024-09-19T05:09:43.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.649+0000 7f6e7d400640 1 --2- 172.21.15.43:0/3209768322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e78108f60 0x7f6e7819ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.649+0000 7f6e7d400640 1 --2- 172.21.15.43:0/3209768322 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e781047e0 0x7f6e7819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.649+0000 7f6e7d400640 1 --2- 172.21.15.43:0/3209768322 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f6e5c0a33b0 0x7f6e5c0a5870 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.649+0000 7f6e7d400640 1 --2- 172.21.15.43:0/3209768322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6e78103e30 0x7f6e7819a490 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:43.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.649+0000 7f6e7d400640 1 -- 172.21.15.43:0/3209768322 >> 172.21.15.43:0/3209768322 conn(0x7f6e780ff910 msgr2=0x7f6e78100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:43.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.649+0000 7f6e7d400640 1 -- 172.21.15.43:0/3209768322 shutdown_connections 2024-09-19T05:09:43.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:43.649+0000 7f6e7d400640 1 -- 172.21.15.43:0/3209768322 wait complete. 2024-09-19T05:09:43.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:43 smithi043 ceph-mon[30800]: pgmap v155: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:09:43.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:43 smithi151 ceph-mon[35425]: pgmap v155: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:09:43.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:43 smithi096 ceph-mon[35085]: pgmap v155: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:09:44.396 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T05:09:44.778 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:09:45.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.360+0000 7f759a800640 1 -- 172.21.15.43:0/913742210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7594103e30 msgr2=0x7f7594104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:45.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.360+0000 7f759a800640 1 --2- 172.21.15.43:0/913742210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7594103e30 0x7f7594104210 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7588009900 tx=0x7f758802a3f0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.360+0000 7f759a800640 1 -- 172.21.15.43:0/913742210 shutdown_connections 2024-09-19T05:09:45.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.360+0000 7f759a800640 1 --2- 172.21.15.43:0/913742210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7594108f60 0x7f7594110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.360+0000 7f759a800640 1 --2- 172.21.15.43:0/913742210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75941047e0 0x7f7594108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.360+0000 7f759a800640 1 --2- 172.21.15.43:0/913742210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7594103e30 0x7f7594104210 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.360+0000 7f759a800640 1 -- 172.21.15.43:0/913742210 >> 172.21.15.43:0/913742210 conn(0x7f75940ff8d0 msgr2=0x7f7594101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:45.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.360+0000 7f759a800640 1 -- 172.21.15.43:0/913742210 shutdown_connections 2024-09-19T05:09:45.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.361+0000 7f759a800640 1 -- 172.21.15.43:0/913742210 wait complete. 2024-09-19T05:09:45.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.361+0000 7f759a800640 1 Processor -- start 2024-09-19T05:09:45.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.362+0000 7f759a800640 1 -- start start 2024-09-19T05:09:45.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.362+0000 7f759a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7594103e30 0x7f759419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:45.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.362+0000 7f759a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75941047e0 0x7f759419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:45.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.362+0000 7f759a800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7594108f60 0x7f759419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:45.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.362+0000 7f759a800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7594114010 con 0x7f7594103e30 2024-09-19T05:09:45.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.362+0000 7f759a800640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7594113e90 con 0x7f7594108f60 2024-09-19T05:09:45.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.362+0000 7f759a800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7594114190 con 0x7f75941047e0 2024-09-19T05:09:45.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.362+0000 7f7599400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7594103e30 0x7f759419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:45.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.362+0000 7f7599400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7594103e30 0x7f759419a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40962/0 (socket says 172.21.15.43:40962) 2024-09-19T05:09:45.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.362+0000 7f7599400640 1 -- 172.21.15.43:0/1457267742 learned_addr learned my addr 172.21.15.43:0/1457267742 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:45.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.362+0000 7f7598a00640 1 --2- 172.21.15.43:0/1457267742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75941047e0 0x7f759419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:45.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.362+0000 7f7599e00640 1 --2- 172.21.15.43:0/1457267742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7594108f60 0x7f759419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:45.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.363+0000 7f7599400640 1 -- 172.21.15.43:0/1457267742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75941047e0 msgr2=0x7f759419a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:45.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.363+0000 7f7599400640 1 --2- 172.21.15.43:0/1457267742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75941047e0 0x7f759419a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.363+0000 7f7599400640 1 -- 172.21.15.43:0/1457267742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7594108f60 msgr2=0x7f759419ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:45.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.363+0000 7f7599400640 1 --2- 172.21.15.43:0/1457267742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7594108f60 0x7f759419ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.363+0000 7f7599400640 1 -- 172.21.15.43:0/1457267742 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f759419f400 con 0x7f7594103e30 2024-09-19T05:09:45.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.363+0000 7f7599e00640 1 --2- 172.21.15.43:0/1457267742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7594108f60 0x7f759419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:09:45.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.363+0000 7f7598a00640 1 --2- 172.21.15.43:0/1457267742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75941047e0 0x7f759419a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:09:45.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.363+0000 7f7599400640 1 --2- 172.21.15.43:0/1457267742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7594103e30 0x7f759419a450 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7588004770 tx=0x7f7588003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:45.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.363+0000 7f7592000640 1 -- 172.21.15.43:0/1457267742 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758803f070 con 0x7f7594103e30 2024-09-19T05:09:45.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.363+0000 7f7592000640 1 -- 172.21.15.43:0/1457267742 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7588031ca0 con 0x7f7594103e30 2024-09-19T05:09:45.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.363+0000 7f759a800640 1 -- 172.21.15.43:0/1457267742 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f759419f690 con 0x7f7594103e30 2024-09-19T05:09:45.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.363+0000 7f7592000640 1 -- 172.21.15.43:0/1457267742 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7588035030 con 0x7f7594103e30 2024-09-19T05:09:45.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.364+0000 7f759a800640 1 -- 172.21.15.43:0/1457267742 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75941a6fd0 con 0x7f7594103e30 2024-09-19T05:09:45.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.367+0000 7f7592000640 1 -- 172.21.15.43:0/1457267742 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f7588042070 con 0x7f7594103e30 2024-09-19T05:09:45.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.367+0000 7f759e823740 1 Processor -- start 2024-09-19T05:09:45.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.367+0000 7f759e823740 1 -- start start 2024-09-19T05:09:45.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.368+0000 7f759e823740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ee121ee40 0x559ee121f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:45.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.368+0000 7f759e823740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ee121f760 0x559ee1229ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:45.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.368+0000 7f759e823740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ee122a020 0x559ee122c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:45.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.368+0000 7f759e823740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559ee11853e0 con 0x559ee121ee40 2024-09-19T05:09:45.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.368+0000 7f759e823740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559ee112cd80 con 0x559ee122a020 2024-09-19T05:09:45.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.368+0000 7f759e823740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559ee11813a0 con 0x559ee121f760 2024-09-19T05:09:45.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.368+0000 7f7599e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ee121ee40 0x559ee121f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:45.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.368+0000 7f7599400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ee121f760 0x559ee1229ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:45.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.368+0000 7f7599e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ee121ee40 0x559ee121f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40976/0 (socket says 172.21.15.43:40976) 2024-09-19T05:09:45.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.368+0000 7f7599e00640 1 -- 172.21.15.43:0/3511405490 learned_addr learned my addr 172.21.15.43:0/3511405490 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:45.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.368+0000 7f7598a00640 1 --2- 172.21.15.43:0/3511405490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ee122a020 0x559ee122c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:45.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.369+0000 7f7599e00640 1 -- 172.21.15.43:0/3511405490 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ee121f760 msgr2=0x559ee1229ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:45.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.369+0000 7f7599e00640 1 --2- 172.21.15.43:0/3511405490 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ee121f760 0x559ee1229ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.369+0000 7f7599e00640 1 -- 172.21.15.43:0/3511405490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ee122a020 msgr2=0x559ee122c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:45.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.369+0000 7f7599e00640 1 --2- 172.21.15.43:0/3511405490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ee122a020 0x559ee122c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.369+0000 7f7599e00640 1 -- 172.21.15.43:0/3511405490 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ee114bae0 con 0x559ee121ee40 2024-09-19T05:09:45.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.369+0000 7f7599400640 1 --2- 172.21.15.43:0/3511405490 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ee121f760 0x559ee1229ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:45.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.369+0000 7f7592000640 1 --2- 172.21.15.43:0/1457267742 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f757c0a3430 0x7f757c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:45.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.369+0000 7f7592000640 1 -- 172.21.15.43:0/1457267742 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f75880f0410 con 0x7f7594103e30 2024-09-19T05:09:45.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.369+0000 7f7599400640 1 --2- 172.21.15.43:0/1457267742 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f757c0a3430 0x7f757c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:45.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.369+0000 7f7599e00640 1 --2- 172.21.15.43:0/3511405490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ee121ee40 0x559ee121f220 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f758c009fd0 tx=0x7f758c010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beabe11791e129b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:45.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.369+0000 7f7586600640 1 -- 172.21.15.43:0/3511405490 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758c011700 con 0x559ee121ee40 2024-09-19T05:09:45.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.370+0000 7f7586600640 1 -- 172.21.15.43:0/3511405490 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f758c011d80 con 0x559ee121ee40 2024-09-19T05:09:45.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.370+0000 7f7586600640 1 -- 172.21.15.43:0/3511405490 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758c00e3c0 con 0x559ee121ee40 2024-09-19T05:09:45.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.370+0000 7f759e823740 1 -- 172.21.15.43:0/3511405490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ee121ee40 msgr2=0x559ee121f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:45.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.370+0000 7f759e823740 1 --2- 172.21.15.43:0/3511405490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ee121ee40 0x559ee121f220 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f758c009fd0 tx=0x7f758c010860 comp rx=0 tx=0).stop 2024-09-19T05:09:45.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.370+0000 7f7599400640 1 --2- 172.21.15.43:0/1457267742 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f757c0a3430 0x7f757c0a58f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f75880f3fd0 tx=0x7f75880f3ec0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:45.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.370+0000 7f759e823740 1 -- 172.21.15.43:0/3511405490 shutdown_connections 2024-09-19T05:09:45.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.370+0000 7f759e823740 1 --2- 172.21.15.43:0/3511405490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ee122a020 0x559ee122c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.370+0000 7f759e823740 1 --2- 172.21.15.43:0/3511405490 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ee121f760 0x559ee1229ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.370+0000 7f759e823740 1 --2- 172.21.15.43:0/3511405490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ee121ee40 0x559ee121f220 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.370+0000 7f759e823740 1 -- 172.21.15.43:0/3511405490 >> 172.21.15.43:0/3511405490 conn(0x559ee121b3d0 msgr2=0x559ee121b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:45.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.371+0000 7f759e823740 1 -- 172.21.15.43:0/3511405490 shutdown_connections 2024-09-19T05:09:45.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.371+0000 7f759e823740 1 -- 172.21.15.43:0/3511405490 wait complete. 2024-09-19T05:09:45.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.371+0000 7f759e823740 1 -- start start 2024-09-19T05:09:45.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.371+0000 7f759e823740 1 Processor -- start 2024-09-19T05:09:45.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.372+0000 7f759e823740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ee121ee40 0x559ee1226f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:45.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.372+0000 7f759e823740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ee121f760 0x559ee1227470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:45.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.372+0000 7f759e823740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ee122a020 0x559ee12279b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:45.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.372+0000 7f759e823740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559ee122cbf0 con 0x559ee121f760 2024-09-19T05:09:45.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.372+0000 7f759e823740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559ee11400f0 con 0x559ee121ee40 2024-09-19T05:09:45.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.372+0000 7f759e823740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559ee122cd70 con 0x559ee122a020 2024-09-19T05:09:45.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.372+0000 7f759e823740 10 client.0 _ll_register_callbacks cb 0x559ee1127bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:45.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.372+0000 7f7598a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ee121f760 0x559ee1227470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:45.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.372+0000 7f7599e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ee121ee40 0x559ee1226f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:45.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.372+0000 7f7599e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ee121ee40 0x559ee1226f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:52624/0 (socket says 172.21.15.43:52624) 2024-09-19T05:09:45.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.372+0000 7f7598a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ee121f760 0x559ee1227470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40984/0 (socket says 172.21.15.43:40984) 2024-09-19T05:09:45.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.372+0000 7f7599e00640 1 -- 172.21.15.43:0/729331132 learned_addr learned my addr 172.21.15.43:0/729331132 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:45.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.372+0000 7f7599e00640 1 --2- 172.21.15.43:0/729331132 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ee122a020 0x559ee12279b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:45.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.373+0000 7f7598a00640 1 -- 172.21.15.43:0/729331132 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ee122a020 msgr2=0x559ee12279b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:45.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.373+0000 7f7598a00640 1 --2- 172.21.15.43:0/729331132 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ee122a020 0x559ee12279b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.373+0000 7f7598a00640 1 -- 172.21.15.43:0/729331132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ee121ee40 msgr2=0x559ee1226f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:45.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.373+0000 7f7598a00640 1 --2- 172.21.15.43:0/729331132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ee121ee40 0x559ee1226f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.373+0000 7f7598a00640 1 -- 172.21.15.43:0/729331132 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ee0ef63c0 con 0x559ee121f760 2024-09-19T05:09:45.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.373+0000 7f7599e00640 1 --2- 172.21.15.43:0/729331132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ee121ee40 0x559ee1226f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:45.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.373+0000 7f7598a00640 1 --2- 172.21.15.43:0/729331132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ee121f760 0x559ee1227470 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f75940ffcf0 tx=0x7f758000b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=522f5275f8e1931 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:45.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.373+0000 7f757aa00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:09:45.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.373+0000 7f757aa00640 1 -- 172.21.15.43:0/729331132 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7580015d50 con 0x559ee121f760 2024-09-19T05:09:45.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.374+0000 7f759e823740 1 -- 172.21.15.43:0/729331132 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f758c011d80 con 0x559ee121f760 2024-09-19T05:09:45.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.374+0000 7f757aa00640 1 -- 172.21.15.43:0/729331132 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f7580004050 con 0x559ee121f760 2024-09-19T05:09:45.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.374+0000 7f757aa00640 1 -- 172.21.15.43:0/729331132 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7580004250 con 0x559ee121f760 2024-09-19T05:09:45.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.374+0000 7f759e823740 10 client.16209 fetch_fsmap learned FSMap version 29 2024-09-19T05:09:45.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.374+0000 7f759e823740 1 -- 172.21.15.43:0/729331132 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559ee1238c60 con 0x559ee121f760 2024-09-19T05:09:45.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.374+0000 7f757aa00640 1 -- 172.21.15.43:0/729331132 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f758c011d80 con 0x559ee121f760 2024-09-19T05:09:45.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.374+0000 7f757aa00640 1 -- 172.21.15.43:0/729331132 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f75800170d0 con 0x559ee121f760 2024-09-19T05:09:45.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.374+0000 7f759e823740 10 client.16209 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:09:45.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.374+0000 7f759e823740 10 client.16209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:09:45.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.374+0000 7f759e823740 20 client.16209 populate_metadata read hostname 'smithi043' 2024-09-19T05:09:45.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.375+0000 7f759e823740 1 --2- 172.21.15.43:0/729331132 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x559ee123a120 0x559ee123c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:45.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.375+0000 7f759e823740 4 client.16209 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:09:45.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.375+0000 7f759e823740 1 -- 172.21.15.43:0/729331132 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ee1192950 con 0x559ee123a120 2024-09-19T05:09:45.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.375+0000 7f7599e00640 1 --2- 172.21.15.43:0/729331132 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x559ee123a120 0x559ee123c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:45.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.376+0000 7f7599e00640 1 --2- 172.21.15.43:0/729331132 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x559ee123a120 0x559ee123c580 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=294f26ab71ab9dbf server_cookie=46a9d3ef548bdf2a in_seq=0 out_seq=0 2024-09-19T05:09:45.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.376+0000 7f757aa00640 10 client.16209 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:09:45.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.376+0000 7f757aa00640 1 -- 172.21.15.43:0/729331132 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559ee1192950 con 0x559ee123a120 2024-09-19T05:09:45.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.376+0000 7f757aa00640 10 client.16209 handle_command_reply: tid=0 2024-09-19T05:09:45.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.376+0000 7f759e823740 1 client.16209 shutdown 2024-09-19T05:09:45.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.377+0000 7f759e823740 1 -- 172.21.15.43:0/729331132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ee121f760 msgr2=0x559ee1227470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:45.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.377+0000 7f759e823740 1 --2- 172.21.15.43:0/729331132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ee121f760 0x559ee1227470 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f75940ffcf0 tx=0x7f758000b690 comp rx=0 tx=0).stop 2024-09-19T05:09:45.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.377+0000 7f759e823740 1 -- 172.21.15.43:0/729331132 shutdown_connections 2024-09-19T05:09:45.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.377+0000 7f759e823740 1 --2- 172.21.15.43:0/729331132 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x559ee123a120 0x559ee123c580 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.377+0000 7f759e823740 1 --2- 172.21.15.43:0/729331132 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ee122a020 0x559ee12279b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.377+0000 7f757aa00640 0 client.16209 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:09:45.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.377+0000 7f759e823740 1 --2- 172.21.15.43:0/729331132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ee121f760 0x559ee1227470 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.377+0000 7f759e823740 1 --2- 172.21.15.43:0/729331132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ee121ee40 0x559ee1226f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.377+0000 7f759e823740 1 -- 172.21.15.43:0/729331132 >> 172.21.15.43:0/729331132 conn(0x559ee121b3d0 msgr2=0x559ee12210b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:45.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.377+0000 7f759e823740 1 -- 172.21.15.43:0/729331132 shutdown_connections 2024-09-19T05:09:45.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.377+0000 7f759e823740 1 -- 172.21.15.43:0/729331132 wait complete. 2024-09-19T05:09:45.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.377+0000 7f759e823740 20 client.16209 trim_cache size 0 max 16384 2024-09-19T05:09:45.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.391+0000 7f759e823740 1 Processor -- start 2024-09-19T05:09:45.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.391+0000 7f759e823740 1 -- start start 2024-09-19T05:09:45.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.391+0000 7f759e823740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ee121ee40 0x559ee12218a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:45.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.391+0000 7f759e823740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ee121f760 0x559ee1228ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:45.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.391+0000 7f759e823740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ee122a020 0x559ee1241e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:45.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.391+0000 7f759e823740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f75941ab4b0 con 0x559ee121ee40 2024-09-19T05:09:45.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.391+0000 7f759e823740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f75941acff0 con 0x559ee122a020 2024-09-19T05:09:45.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.391+0000 7f759e823740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f75941abb80 con 0x559ee121f760 2024-09-19T05:09:45.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.392+0000 7f7599e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ee121ee40 0x559ee12218a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:45.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.392+0000 7f7599e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ee121ee40 0x559ee12218a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40990/0 (socket says 172.21.15.43:40990) 2024-09-19T05:09:45.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.392+0000 7f7599e00640 1 -- 172.21.15.43:0/914130292 learned_addr learned my addr 172.21.15.43:0/914130292 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:45.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.392+0000 7f7598a00640 1 --2- 172.21.15.43:0/914130292 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ee121f760 0x559ee1228ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:45.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.392+0000 7f7599e00640 1 --2- 172.21.15.43:0/914130292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ee122a020 0x559ee1241e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:45.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.392+0000 7f7599e00640 1 -- 172.21.15.43:0/914130292 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ee121f760 msgr2=0x559ee1228ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:45.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.392+0000 7f7599e00640 1 --2- 172.21.15.43:0/914130292 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ee121f760 0x559ee1228ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.392+0000 7f7599e00640 1 -- 172.21.15.43:0/914130292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ee122a020 msgr2=0x559ee1241e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:45.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.392+0000 7f7599e00640 1 --2- 172.21.15.43:0/914130292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ee122a020 0x559ee1241e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.392+0000 7f7599e00640 1 -- 172.21.15.43:0/914130292 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ee1225480 con 0x559ee121ee40 2024-09-19T05:09:45.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.393+0000 7f7599e00640 1 --2- 172.21.15.43:0/914130292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ee121ee40 0x559ee12218a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f758c000f80 tx=0x7f758c01bb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce5b2d56ce92044f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:45.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.393+0000 7f7585c00640 1 -- 172.21.15.43:0/914130292 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758c024040 con 0x559ee121ee40 2024-09-19T05:09:45.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.393+0000 7f7585c00640 1 -- 172.21.15.43:0/914130292 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f758c004be0 con 0x559ee121ee40 2024-09-19T05:09:45.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.393+0000 7f7585c00640 1 -- 172.21.15.43:0/914130292 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758c0052f0 con 0x559ee121ee40 2024-09-19T05:09:45.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.393+0000 7f759e823740 1 -- 172.21.15.43:0/914130292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ee121ee40 msgr2=0x559ee12218a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:45.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.393+0000 7f759e823740 1 --2- 172.21.15.43:0/914130292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ee121ee40 0x559ee12218a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f758c000f80 tx=0x7f758c01bb20 comp rx=0 tx=0).stop 2024-09-19T05:09:45.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.393+0000 7f759e823740 1 -- 172.21.15.43:0/914130292 shutdown_connections 2024-09-19T05:09:45.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.393+0000 7f759e823740 1 --2- 172.21.15.43:0/914130292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ee122a020 0x559ee1241e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.393+0000 7f759e823740 1 --2- 172.21.15.43:0/914130292 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ee121f760 0x559ee1228ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.393+0000 7f759e823740 1 --2- 172.21.15.43:0/914130292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ee121ee40 0x559ee12218a0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.393+0000 7f759e823740 1 -- 172.21.15.43:0/914130292 >> 172.21.15.43:0/914130292 conn(0x559ee121b3d0 msgr2=0x559ee1228790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:45.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.394+0000 7f759e823740 1 -- 172.21.15.43:0/914130292 shutdown_connections 2024-09-19T05:09:45.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.394+0000 7f759e823740 1 -- 172.21.15.43:0/914130292 wait complete. 2024-09-19T05:09:45.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.394+0000 7f759e823740 1 -- start start 2024-09-19T05:09:45.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.394+0000 7f759e823740 1 Processor -- start 2024-09-19T05:09:45.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.395+0000 7f759e823740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ee121ee40 0x559ee12326d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:45.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.395+0000 7f759e823740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ee121f760 0x559ee122da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:45.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.395+0000 7f759e823740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ee122a020 0x559ee122df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:45.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.395+0000 7f759e823740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f75941ac920 con 0x559ee122a020 2024-09-19T05:09:45.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.395+0000 7f759e823740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f75941ac250 con 0x559ee121ee40 2024-09-19T05:09:45.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.395+0000 7f759e823740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f75941a94b0 con 0x559ee121f760 2024-09-19T05:09:45.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.395+0000 7f759e823740 10 client.0 _ll_register_callbacks cb 0x7f7580017ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:45.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.395+0000 7f7599e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ee121ee40 0x559ee12326d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:45.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.395+0000 7f7599e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ee121ee40 0x559ee12326d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:52646/0 (socket says 172.21.15.43:52646) 2024-09-19T05:09:45.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.395+0000 7f7599e00640 1 -- 172.21.15.43:0/79184582 learned_addr learned my addr 172.21.15.43:0/79184582 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:45.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.395+0000 7f7598a00640 1 --2- 172.21.15.43:0/79184582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ee121f760 0x559ee122da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:45.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.395+0000 7f7599e00640 1 --2- 172.21.15.43:0/79184582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ee122a020 0x559ee122df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:45.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.395+0000 7f7598a00640 1 -- 172.21.15.43:0/79184582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ee121ee40 msgr2=0x559ee12326d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:45.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.395+0000 7f7598a00640 1 --2- 172.21.15.43:0/79184582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ee121ee40 0x559ee12326d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.396+0000 7f7598a00640 1 -- 172.21.15.43:0/79184582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ee122a020 msgr2=0x559ee122df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:45.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.396+0000 7f7598a00640 1 --2- 172.21.15.43:0/79184582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ee122a020 0x559ee122df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.396+0000 7f7598a00640 1 -- 172.21.15.43:0/79184582 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ee122e740 con 0x559ee121f760 2024-09-19T05:09:45.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.396+0000 7f7598a00640 1 --2- 172.21.15.43:0/79184582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ee121f760 0x559ee122da00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7580017570 tx=0x7f7580017990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c02428c98b563c53 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:45.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.396+0000 7f757aa00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:09:45.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.396+0000 7f757aa00640 1 -- 172.21.15.43:0/79184582 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7580017cc0 con 0x559ee121f760 2024-09-19T05:09:45.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.396+0000 7f757aa00640 1 -- 172.21.15.43:0/79184582 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f7580005040 con 0x559ee121f760 2024-09-19T05:09:45.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.396+0000 7f759e823740 1 -- 172.21.15.43:0/79184582 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f758c004be0 con 0x559ee121f760 2024-09-19T05:09:45.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.396+0000 7f757aa00640 1 -- 172.21.15.43:0/79184582 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7580043050 con 0x559ee121f760 2024-09-19T05:09:45.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.396+0000 7f759e823740 10 client.26191 fetch_fsmap learned FSMap version 29 2024-09-19T05:09:45.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.397+0000 7f759e823740 1 -- 172.21.15.43:0/79184582 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559ee12a4df0 con 0x559ee121f760 2024-09-19T05:09:45.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.397+0000 7f757aa00640 1 -- 172.21.15.43:0/79184582 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f758c004be0 con 0x559ee121f760 2024-09-19T05:09:45.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.397+0000 7f757aa00640 1 -- 172.21.15.43:0/79184582 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f758003e030 con 0x559ee121f760 2024-09-19T05:09:45.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.397+0000 7f759e823740 10 client.26191 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:09:45.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.397+0000 7f759e823740 10 client.26191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:09:45.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.397+0000 7f759e823740 20 client.26191 populate_metadata read hostname 'smithi043' 2024-09-19T05:09:45.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.397+0000 7f759e823740 1 --2- 172.21.15.43:0/79184582 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x559ee123a120 0x559ee12a7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:45.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.397+0000 7f759e823740 4 client.26191 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:09:45.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.397+0000 7f759e823740 1 -- 172.21.15.43:0/79184582 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559ee118ce60 con 0x559ee123a120 2024-09-19T05:09:45.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.398+0000 7f7599e00640 1 --2- 172.21.15.43:0/79184582 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x559ee123a120 0x559ee12a7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:45.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.398+0000 7f7599e00640 1 --2- 172.21.15.43:0/79184582 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x559ee123a120 0x559ee12a7600 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a1ce3a794da4a6b server_cookie=17065876671a8f06 in_seq=0 out_seq=0 2024-09-19T05:09:45.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.398+0000 7f757aa00640 10 client.26191 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:09:45.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.398+0000 7f757aa00640 1 -- 172.21.15.43:0/79184582 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559ee118ce60 con 0x559ee123a120 2024-09-19T05:09:45.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.398+0000 7f757aa00640 10 client.26191 handle_command_reply: tid=0 2024-09-19T05:09:45.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.398+0000 7f759e823740 1 client.26191 shutdown 2024-09-19T05:09:45.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.399+0000 7f759e823740 1 -- 172.21.15.43:0/79184582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ee121f760 msgr2=0x559ee122da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:45.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.399+0000 7f759e823740 1 --2- 172.21.15.43:0/79184582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ee121f760 0x559ee122da00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7580017570 tx=0x7f7580017990 comp rx=0 tx=0).stop 2024-09-19T05:09:45.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.399+0000 7f759e823740 1 -- 172.21.15.43:0/79184582 shutdown_connections 2024-09-19T05:09:45.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.399+0000 7f759e823740 1 --2- 172.21.15.43:0/79184582 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x559ee123a120 0x559ee12a7600 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.399+0000 7f759e823740 1 --2- 172.21.15.43:0/79184582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559ee122a020 0x559ee122df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.399+0000 7f759e823740 1 --2- 172.21.15.43:0/79184582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ee121f760 0x559ee122da00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.399+0000 7f759e823740 1 --2- 172.21.15.43:0/79184582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559ee121ee40 0x559ee12326d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.399+0000 7f757aa00640 0 client.26191 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:09:45.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.399+0000 7f759e823740 1 -- 172.21.15.43:0/79184582 >> 172.21.15.43:0/79184582 conn(0x559ee121b3d0 msgr2=0x559ee1228fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:45.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.399+0000 7f759e823740 1 -- 172.21.15.43:0/79184582 shutdown_connections 2024-09-19T05:09:45.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.399+0000 7f759e823740 1 -- 172.21.15.43:0/79184582 wait complete. 2024-09-19T05:09:45.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.399+0000 7f759e823740 20 client.26191 trim_cache size 0 max 16384 2024-09-19T05:09:45.405 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:09:45.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.400+0000 7f759a800640 1 -- 172.21.15.43:0/1457267742 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f757c0a3430 msgr2=0x7f757c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:45.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.400+0000 7f759a800640 1 --2- 172.21.15.43:0/1457267742 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f757c0a3430 0x7f757c0a58f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f75880f3fd0 tx=0x7f75880f3ec0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.400+0000 7f759a800640 1 -- 172.21.15.43:0/1457267742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7594103e30 msgr2=0x7f759419a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:45.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.400+0000 7f759a800640 1 --2- 172.21.15.43:0/1457267742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7594103e30 0x7f759419a450 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7588004770 tx=0x7f7588003c10 comp rx=0 tx=0).stop 2024-09-19T05:09:45.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.400+0000 7f759a800640 1 -- 172.21.15.43:0/1457267742 shutdown_connections 2024-09-19T05:09:45.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.400+0000 7f759a800640 1 --2- 172.21.15.43:0/1457267742 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f757c0a3430 0x7f757c0a58f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.400+0000 7f759a800640 1 --2- 172.21.15.43:0/1457267742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7594108f60 0x7f759419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.400+0000 7f759a800640 1 --2- 172.21.15.43:0/1457267742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75941047e0 0x7f759419a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.400+0000 7f759a800640 1 --2- 172.21.15.43:0/1457267742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7594103e30 0x7f759419a450 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:45.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.400+0000 7f759a800640 1 -- 172.21.15.43:0/1457267742 >> 172.21.15.43:0/1457267742 conn(0x7f75940ff8d0 msgr2=0x7f7594100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:45.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.400+0000 7f759a800640 1 -- 172.21.15.43:0/1457267742 shutdown_connections 2024-09-19T05:09:45.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:45.400+0000 7f759a800640 1 -- 172.21.15.43:0/1457267742 wait complete. 2024-09-19T05:09:45.862 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:45 smithi043 ceph-mon[30800]: pgmap v156: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:09:45.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:45 smithi151 ceph-mon[35425]: pgmap v156: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:09:45.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:45 smithi096 ceph-mon[35085]: pgmap v156: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:09:47.308 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T05:09:47.309 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T05:09:47.701 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:09:47.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:47 smithi151 ceph-mon[35425]: pgmap v157: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:09:47.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:47 smithi096 ceph-mon[35085]: pgmap v157: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:09:48.001 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:47 smithi043 ceph-mon[30800]: pgmap v157: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:09:48.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.263+0000 7f4562600640 1 -- 172.21.15.43:0/2752494451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f455c1049a0 msgr2=0x7f455c1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:48.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.263+0000 7f4562600640 1 --2- 172.21.15.43:0/2752494451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f455c1049a0 0x7f455c1089f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4544009900 tx=0x7f454402a3f0 comp rx=0 tx=0).stop 2024-09-19T05:09:48.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.263+0000 7f4562600640 1 -- 172.21.15.43:0/2752494451 shutdown_connections 2024-09-19T05:09:48.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.263+0000 7f4562600640 1 --2- 172.21.15.43:0/2752494451 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f455c109120 0x7f455c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:48.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.263+0000 7f4562600640 1 --2- 172.21.15.43:0/2752494451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f455c1049a0 0x7f455c1089f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:48.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.263+0000 7f4562600640 1 --2- 172.21.15.43:0/2752494451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f455c103ff0 0x7f455c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:48.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.263+0000 7f4562600640 1 -- 172.21.15.43:0/2752494451 >> 172.21.15.43:0/2752494451 conn(0x7f455c0ffa90 msgr2=0x7f455c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:48.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.263+0000 7f4562600640 1 -- 172.21.15.43:0/2752494451 shutdown_connections 2024-09-19T05:09:48.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.263+0000 7f4562600640 1 -- 172.21.15.43:0/2752494451 wait complete. 2024-09-19T05:09:48.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.264+0000 7f4562600640 1 Processor -- start 2024-09-19T05:09:48.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.264+0000 7f4562600640 1 -- start start 2024-09-19T05:09:48.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.264+0000 7f4562600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f455c103ff0 0x7f455c19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:48.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.264+0000 7f4562600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f455c1049a0 0x7f455c19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:48.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.264+0000 7f4562600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f455c109120 0x7f455c19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:48.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.264+0000 7f4562600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f455c114140 con 0x7f455c1049a0 2024-09-19T05:09:48.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.264+0000 7f4562600640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f455c113fc0 con 0x7f455c103ff0 2024-09-19T05:09:48.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.264+0000 7f4562600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f455c1142c0 con 0x7f455c109120 2024-09-19T05:09:48.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.265+0000 7f4553e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f455c1049a0 0x7f455c19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:48.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.265+0000 7f4553e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f455c1049a0 0x7f455c19abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58648/0 (socket says 172.21.15.43:58648) 2024-09-19T05:09:48.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.265+0000 7f4553e00640 1 -- 172.21.15.43:0/2525950926 learned_addr learned my addr 172.21.15.43:0/2525950926 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:48.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.265+0000 7f455b400640 1 --2- 172.21.15.43:0/2525950926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f455c103ff0 0x7f455c19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:48.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.265+0000 7f455be00640 1 --2- 172.21.15.43:0/2525950926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f455c109120 0x7f455c19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:48.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.265+0000 7f4553e00640 1 -- 172.21.15.43:0/2525950926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f455c109120 msgr2=0x7f455c19ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:48.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.265+0000 7f4553e00640 1 --2- 172.21.15.43:0/2525950926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f455c109120 0x7f455c19ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:48.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.265+0000 7f4553e00640 1 -- 172.21.15.43:0/2525950926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f455c103ff0 msgr2=0x7f455c19a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:48.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.265+0000 7f4553e00640 1 --2- 172.21.15.43:0/2525950926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f455c103ff0 0x7f455c19a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:48.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.265+0000 7f4553e00640 1 -- 172.21.15.43:0/2525950926 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f455c19f650 con 0x7f455c1049a0 2024-09-19T05:09:48.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.265+0000 7f455be00640 1 --2- 172.21.15.43:0/2525950926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f455c109120 0x7f455c19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:09:48.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.265+0000 7f455b400640 1 --2- 172.21.15.43:0/2525950926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f455c103ff0 0x7f455c19a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:48.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.266+0000 7f4553e00640 1 --2- 172.21.15.43:0/2525950926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f455c1049a0 0x7f455c19abe0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4544006010 tx=0x7f4544004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:48.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.266+0000 7f4558c00640 1 -- 172.21.15.43:0/2525950926 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454403f030 con 0x7f455c1049a0 2024-09-19T05:09:48.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.266+0000 7f4562600640 1 -- 172.21.15.43:0/2525950926 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f455c19f8e0 con 0x7f455c1049a0 2024-09-19T05:09:48.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.266+0000 7f4558c00640 1 -- 172.21.15.43:0/2525950926 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4544004440 con 0x7f455c1049a0 2024-09-19T05:09:48.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.266+0000 7f4558c00640 1 -- 172.21.15.43:0/2525950926 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4544035070 con 0x7f455c1049a0 2024-09-19T05:09:48.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.266+0000 7f4562600640 1 -- 172.21.15.43:0/2525950926 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f455c1a7220 con 0x7f455c1049a0 2024-09-19T05:09:48.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.270+0000 7f45651fb740 1 Processor -- start 2024-09-19T05:09:48.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.270+0000 7f45651fb740 1 -- start start 2024-09-19T05:09:48.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.271+0000 7f4558c00640 1 -- 172.21.15.43:0/2525950926 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4544042030 con 0x7f455c1049a0 2024-09-19T05:09:48.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.271+0000 7f45651fb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55638348aac0 0x55638348aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:48.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.271+0000 7f45651fb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55638348b3e0 0x556383495760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:48.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.271+0000 7f45651fb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556383495ca0 0x556383498090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:48.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.271+0000 7f45651fb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5563833f1290 con 0x556383495ca0 2024-09-19T05:09:48.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.271+0000 7f45651fb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556383398b40 con 0x55638348b3e0 2024-09-19T05:09:48.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.271+0000 7f45651fb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5563833ed250 con 0x55638348aac0 2024-09-19T05:09:48.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.271+0000 7f455be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55638348aac0 0x55638348aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:48.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.271+0000 7f455be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55638348aac0 0x55638348aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:47810/0 (socket says 172.21.15.43:47810) 2024-09-19T05:09:48.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.271+0000 7f455b400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55638348b3e0 0x556383495760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:48.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.271+0000 7f455be00640 1 -- 172.21.15.43:0/291876214 learned_addr learned my addr 172.21.15.43:0/291876214 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:48.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.271+0000 7f4553e00640 1 --2- 172.21.15.43:0/291876214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556383495ca0 0x556383498090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:48.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.271+0000 7f455be00640 1 -- 172.21.15.43:0/291876214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55638348b3e0 msgr2=0x556383495760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:48.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.272+0000 7f455be00640 1 --2- 172.21.15.43:0/291876214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55638348b3e0 0x556383495760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:48.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.272+0000 7f455be00640 1 -- 172.21.15.43:0/291876214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556383495ca0 msgr2=0x556383498090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:48.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.272+0000 7f455be00640 1 --2- 172.21.15.43:0/291876214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556383495ca0 0x556383498090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:48.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.272+0000 7f455be00640 1 -- 172.21.15.43:0/291876214 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563831626a0 con 0x55638348aac0 2024-09-19T05:09:48.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.272+0000 7f455b400640 1 --2- 172.21.15.43:0/291876214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55638348b3e0 0x556383495760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:48.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.272+0000 7f455be00640 1 --2- 172.21.15.43:0/291876214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55638348aac0 0x55638348aea0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4554009fd0 tx=0x7f4554010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c6ae82d8bb71683 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:48.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.272+0000 7f4550c00640 1 -- 172.21.15.43:0/291876214 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4554011700 con 0x55638348aac0 2024-09-19T05:09:48.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.272+0000 7f4550c00640 1 -- 172.21.15.43:0/291876214 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4554011d80 con 0x55638348aac0 2024-09-19T05:09:48.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.272+0000 7f4550c00640 1 -- 172.21.15.43:0/291876214 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455400e3c0 con 0x55638348aac0 2024-09-19T05:09:48.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.273+0000 7f4558c00640 1 --2- 172.21.15.43:0/2525950926 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f45400a33b0 0x7f45400a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:48.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.273+0000 7f4558c00640 1 -- 172.21.15.43:0/2525950926 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f45440f0110 con 0x7f455c1049a0 2024-09-19T05:09:48.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.273+0000 7f455b400640 1 --2- 172.21.15.43:0/2525950926 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f45400a33b0 0x7f45400a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:48.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.273+0000 7f45651fb740 1 -- 172.21.15.43:0/291876214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55638348aac0 msgr2=0x55638348aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:48.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.273+0000 7f45651fb740 1 --2- 172.21.15.43:0/291876214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55638348aac0 0x55638348aea0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4554009fd0 tx=0x7f4554010860 comp rx=0 tx=0).stop 2024-09-19T05:09:48.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.273+0000 7f45651fb740 1 -- 172.21.15.43:0/291876214 shutdown_connections 2024-09-19T05:09:48.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.273+0000 7f45651fb740 1 --2- 172.21.15.43:0/291876214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556383495ca0 0x556383498090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:48.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.273+0000 7f45651fb740 1 --2- 172.21.15.43:0/291876214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55638348b3e0 0x556383495760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:48.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.273+0000 7f45651fb740 1 --2- 172.21.15.43:0/291876214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55638348aac0 0x55638348aea0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:48.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.273+0000 7f45651fb740 1 -- 172.21.15.43:0/291876214 >> 172.21.15.43:0/291876214 conn(0x556383487050 msgr2=0x556383487430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:48.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.273+0000 7f45651fb740 1 -- 172.21.15.43:0/291876214 shutdown_connections 2024-09-19T05:09:48.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.273+0000 7f45651fb740 1 -- 172.21.15.43:0/291876214 wait complete. 2024-09-19T05:09:48.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.274+0000 7f45651fb740 1 -- start start 2024-09-19T05:09:48.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.274+0000 7f455b400640 1 --2- 172.21.15.43:0/2525950926 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f45400a33b0 0x7f45400a5870 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f454c009fd0 tx=0x7f454c005f90 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:48.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.274+0000 7f45651fb740 1 Processor -- start 2024-09-19T05:09:48.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.274+0000 7f45651fb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55638348aac0 0x556383495630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:48.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.274+0000 7f45651fb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55638348b3e0 0x5563834928e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:48.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.274+0000 7f45651fb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556383495ca0 0x556383492e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:48.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.275+0000 7f45651fb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556383498910 con 0x55638348b3e0 2024-09-19T05:09:48.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.275+0000 7f45651fb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5563833abfa0 con 0x556383495ca0 2024-09-19T05:09:48.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.275+0000 7f45651fb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556383498a90 con 0x55638348aac0 2024-09-19T05:09:48.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.275+0000 7f45651fb740 10 client.0 _ll_register_callbacks cb 0x5563830ffba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:48.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.275+0000 7f4553e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55638348b3e0 0x5563834928e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:48.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.275+0000 7f455b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55638348aac0 0x556383495630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:48.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.275+0000 7f4553e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55638348b3e0 0x5563834928e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58672/0 (socket says 172.21.15.43:58672) 2024-09-19T05:09:48.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.275+0000 7f4553e00640 1 -- 172.21.15.43:0/2848494918 learned_addr learned my addr 172.21.15.43:0/2848494918 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:48.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.275+0000 7f455b400640 1 --2- 172.21.15.43:0/2848494918 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55638348aac0 0x556383495630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:47820/0 (socket says 172.21.15.43:47820) 2024-09-19T05:09:48.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.275+0000 7f455be00640 1 --2- 172.21.15.43:0/2848494918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556383495ca0 0x556383492e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:48.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.275+0000 7f4553e00640 1 -- 172.21.15.43:0/2848494918 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55638348aac0 msgr2=0x556383495630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:48.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.275+0000 7f4553e00640 1 --2- 172.21.15.43:0/2848494918 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55638348aac0 0x556383495630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:48.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.275+0000 7f4553e00640 1 -- 172.21.15.43:0/2848494918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556383495ca0 msgr2=0x556383492e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:48.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.275+0000 7f4553e00640 1 --2- 172.21.15.43:0/2848494918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556383495ca0 0x556383492e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:48.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.275+0000 7f4553e00640 1 -- 172.21.15.43:0/2848494918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556383493690 con 0x55638348b3e0 2024-09-19T05:09:48.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.276+0000 7f4553e00640 1 --2- 172.21.15.43:0/2848494918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55638348b3e0 0x5563834928e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f45440b8860 tx=0x7f45440f3850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9771b7d3f7f3062b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:48.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.276+0000 7f4549600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:09:48.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.276+0000 7f4549600640 1 -- 172.21.15.43:0/2848494918 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4544049070 con 0x55638348b3e0 2024-09-19T05:09:48.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.276+0000 7f4549600640 1 -- 172.21.15.43:0/2848494918 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f45440b8a70 con 0x55638348b3e0 2024-09-19T05:09:48.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.276+0000 7f45651fb740 1 -- 172.21.15.43:0/2848494918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4554011d80 con 0x55638348b3e0 2024-09-19T05:09:48.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.276+0000 7f4549600640 1 -- 172.21.15.43:0/2848494918 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f45440bc1d0 con 0x55638348b3e0 2024-09-19T05:09:48.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.276+0000 7f45651fb740 10 client.16227 fetch_fsmap learned FSMap version 29 2024-09-19T05:09:48.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.276+0000 7f45651fb740 1 -- 172.21.15.43:0/2848494918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5563834a4860 con 0x55638348b3e0 2024-09-19T05:09:48.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.276+0000 7f4549600640 1 -- 172.21.15.43:0/2848494918 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4554011d80 con 0x55638348b3e0 2024-09-19T05:09:48.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.277+0000 7f4549600640 1 -- 172.21.15.43:0/2848494918 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4544031bd0 con 0x55638348b3e0 2024-09-19T05:09:48.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.277+0000 7f45651fb740 10 client.16227 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:09:48.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.277+0000 7f45651fb740 10 client.16227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:09:48.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.277+0000 7f45651fb740 20 client.16227 populate_metadata read hostname 'smithi043' 2024-09-19T05:09:48.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.277+0000 7f45651fb740 1 --2- 172.21.15.43:0/2848494918 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5563834a5d60 0x5563834a8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:48.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.277+0000 7f45651fb740 4 client.16227 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:09:48.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.277+0000 7f45651fb740 1 -- 172.21.15.43:0/2848494918 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563833fe800 con 0x5563834a5d60 2024-09-19T05:09:48.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.277+0000 7f455b400640 1 --2- 172.21.15.43:0/2848494918 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5563834a5d60 0x5563834a8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:48.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.278+0000 7f455b400640 1 --2- 172.21.15.43:0/2848494918 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5563834a5d60 0x5563834a8190 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0ddcc127f8d270c server_cookie=3647bd60e6a61c04 in_seq=0 out_seq=0 2024-09-19T05:09:48.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.278+0000 7f4549600640 10 client.16227 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:09:48.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.278+0000 7f4549600640 1 -- 172.21.15.43:0/2848494918 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5563833fe800 con 0x5563834a5d60 2024-09-19T05:09:48.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.278+0000 7f4549600640 10 client.16227 handle_command_reply: tid=0 2024-09-19T05:09:48.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.279+0000 7f45651fb740 1 client.16227 shutdown 2024-09-19T05:09:48.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.279+0000 7f45651fb740 1 -- 172.21.15.43:0/2848494918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55638348b3e0 msgr2=0x5563834928e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:48.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.279+0000 7f45651fb740 1 --2- 172.21.15.43:0/2848494918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55638348b3e0 0x5563834928e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f45440b8860 tx=0x7f45440f3850 comp rx=0 tx=0).stop 2024-09-19T05:09:48.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.279+0000 7f45651fb740 1 -- 172.21.15.43:0/2848494918 shutdown_connections 2024-09-19T05:09:48.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.279+0000 7f45651fb740 1 --2- 172.21.15.43:0/2848494918 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5563834a5d60 0x5563834a8190 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:48.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.279+0000 7f45651fb740 1 --2- 172.21.15.43:0/2848494918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556383495ca0 0x556383492e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:48.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.279+0000 7f45651fb740 1 --2- 172.21.15.43:0/2848494918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55638348b3e0 0x5563834928e0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:48.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.279+0000 7f45651fb740 1 --2- 172.21.15.43:0/2848494918 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55638348aac0 0x556383495630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:48.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.279+0000 7f45651fb740 1 -- 172.21.15.43:0/2848494918 >> 172.21.15.43:0/2848494918 conn(0x556383487050 msgr2=0x5563834861c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:48.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.279+0000 7f4549600640 0 client.16227 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:09:48.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.279+0000 7f45651fb740 1 -- 172.21.15.43:0/2848494918 shutdown_connections 2024-09-19T05:09:48.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.279+0000 7f45651fb740 1 -- 172.21.15.43:0/2848494918 wait complete. 2024-09-19T05:09:48.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.279+0000 7f45651fb740 20 client.16227 trim_cache size 0 max 16384 2024-09-19T05:09:48.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.298+0000 7f45651fb740 1 Processor -- start 2024-09-19T05:09:48.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.298+0000 7f45651fb740 1 -- start start 2024-09-19T05:09:48.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.298+0000 7f45651fb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55638348aac0 0x55638348cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:48.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.298+0000 7f45651fb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55638348b3e0 0x5563834ada60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:48.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.298+0000 7f45651fb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556383495ca0 0x5563834adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:48.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.298+0000 7f45651fb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f455c1ab620 con 0x556383495ca0 2024-09-19T05:09:48.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.298+0000 7f45651fb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f455c1ad160 con 0x55638348b3e0 2024-09-19T05:09:48.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.298+0000 7f45651fb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f455c1abcf0 con 0x55638348aac0 2024-09-19T05:09:48.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.298+0000 7f455be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556383495ca0 0x5563834adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:48.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.298+0000 7f455be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556383495ca0 0x5563834adfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58674/0 (socket says 172.21.15.43:58674) 2024-09-19T05:09:48.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.298+0000 7f455be00640 1 -- 172.21.15.43:0/2093036805 learned_addr learned my addr 172.21.15.43:0/2093036805 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:48.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.298+0000 7f4553e00640 1 --2- 172.21.15.43:0/2093036805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55638348b3e0 0x5563834ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:48.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.298+0000 7f455b400640 1 --2- 172.21.15.43:0/2093036805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55638348aac0 0x55638348cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:48.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.299+0000 7f455be00640 1 -- 172.21.15.43:0/2093036805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55638348aac0 msgr2=0x55638348cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:48.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.299+0000 7f455be00640 1 --2- 172.21.15.43:0/2093036805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55638348aac0 0x55638348cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:48.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.299+0000 7f455be00640 1 -- 172.21.15.43:0/2093036805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55638348b3e0 msgr2=0x5563834ada60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:48.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.299+0000 7f455be00640 1 --2- 172.21.15.43:0/2093036805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55638348b3e0 0x5563834ada60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:48.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.299+0000 7f455be00640 1 -- 172.21.15.43:0/2093036805 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556383491890 con 0x556383495ca0 2024-09-19T05:09:48.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.299+0000 7f455b400640 1 --2- 172.21.15.43:0/2093036805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55638348aac0 0x55638348cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:48.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.299+0000 7f455be00640 1 --2- 172.21.15.43:0/2093036805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556383495ca0 0x5563834adfa0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f455401bc20 tx=0x7f455401bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21eb05051181a889 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:48.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.299+0000 7f454be00640 1 -- 172.21.15.43:0/2093036805 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45540049c0 con 0x556383495ca0 2024-09-19T05:09:48.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.299+0000 7f454be00640 1 -- 172.21.15.43:0/2093036805 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4554023040 con 0x556383495ca0 2024-09-19T05:09:48.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.299+0000 7f454be00640 1 -- 172.21.15.43:0/2093036805 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4554011760 con 0x556383495ca0 2024-09-19T05:09:48.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.300+0000 7f45651fb740 1 -- 172.21.15.43:0/2093036805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556383495ca0 msgr2=0x5563834adfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:48.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.300+0000 7f45651fb740 1 --2- 172.21.15.43:0/2093036805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556383495ca0 0x5563834adfa0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f455401bc20 tx=0x7f455401bb90 comp rx=0 tx=0).stop 2024-09-19T05:09:48.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.300+0000 7f45651fb740 1 -- 172.21.15.43:0/2093036805 shutdown_connections 2024-09-19T05:09:48.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.300+0000 7f45651fb740 1 --2- 172.21.15.43:0/2093036805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556383495ca0 0x5563834adfa0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:48.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.300+0000 7f45651fb740 1 --2- 172.21.15.43:0/2093036805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55638348b3e0 0x5563834ada60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:48.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.300+0000 7f45651fb740 1 --2- 172.21.15.43:0/2093036805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55638348aac0 0x55638348cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:48.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.300+0000 7f45651fb740 1 -- 172.21.15.43:0/2093036805 >> 172.21.15.43:0/2093036805 conn(0x556383487050 msgr2=0x55638348c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:48.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.300+0000 7f45651fb740 1 -- 172.21.15.43:0/2093036805 shutdown_connections 2024-09-19T05:09:48.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.300+0000 7f45651fb740 1 -- 172.21.15.43:0/2093036805 wait complete. 2024-09-19T05:09:48.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.301+0000 7f45651fb740 1 -- start start 2024-09-19T05:09:48.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.301+0000 7f45651fb740 1 Processor -- start 2024-09-19T05:09:48.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.301+0000 7f45651fb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55638348aac0 0x55638349e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:48.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.301+0000 7f45651fb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55638348b3e0 0x55638349acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:48.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.301+0000 7f45651fb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556383495ca0 0x55638349b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:48.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.301+0000 7f45651fb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f455c1aca90 con 0x556383495ca0 2024-09-19T05:09:48.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.301+0000 7f45651fb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f455c1ac3c0 con 0x55638348aac0 2024-09-19T05:09:48.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.301+0000 7f45651fb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f455c1a9600 con 0x55638348b3e0 2024-09-19T05:09:48.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.301+0000 7f45651fb740 10 client.0 _ll_register_callbacks cb 0x7f45440f3950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:48.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.302+0000 7f455be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556383495ca0 0x55638349b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:48.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.302+0000 7f455b400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55638348aac0 0x55638349e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:48.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.302+0000 7f455b400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55638348aac0 0x55638349e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:38862/0 (socket says 172.21.15.43:38862) 2024-09-19T05:09:48.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.302+0000 7f455b400640 1 -- 172.21.15.43:0/2354564190 learned_addr learned my addr 172.21.15.43:0/2354564190 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:48.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.302+0000 7f4553e00640 1 --2- 172.21.15.43:0/2354564190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55638348b3e0 0x55638349acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:48.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.302+0000 7f455b400640 1 -- 172.21.15.43:0/2354564190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55638348b3e0 msgr2=0x55638349acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:48.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.302+0000 7f455b400640 1 --2- 172.21.15.43:0/2354564190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55638348b3e0 0x55638349acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:48.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.302+0000 7f455b400640 1 -- 172.21.15.43:0/2354564190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556383495ca0 msgr2=0x55638349b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:48.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.302+0000 7f455b400640 1 --2- 172.21.15.43:0/2354564190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556383495ca0 0x55638349b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:48.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.302+0000 7f455b400640 1 -- 172.21.15.43:0/2354564190 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55638349ba60 con 0x55638348aac0 2024-09-19T05:09:48.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.302+0000 7f455b400640 1 --2- 172.21.15.43:0/2354564190 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55638348aac0 0x55638349e4e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x5563834a8a00 tx=0x7f454c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8afed75e15e1b54a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:48.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.302+0000 7f4549600640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:09:48.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.303+0000 7f4549600640 1 -- 172.21.15.43:0/2354564190 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c030030 con 0x55638348aac0 2024-09-19T05:09:48.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.303+0000 7f4549600640 1 -- 172.21.15.43:0/2354564190 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f454c018030 con 0x55638348aac0 2024-09-19T05:09:48.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.303+0000 7f45651fb740 1 -- 172.21.15.43:0/2354564190 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4554023040 con 0x55638348aac0 2024-09-19T05:09:48.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.303+0000 7f4549600640 1 -- 172.21.15.43:0/2354564190 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f454c03c050 con 0x55638348aac0 2024-09-19T05:09:48.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.303+0000 7f45651fb740 10 client.26276 fetch_fsmap learned FSMap version 29 2024-09-19T05:09:48.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.303+0000 7f45651fb740 1 -- 172.21.15.43:0/2354564190 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556383511b80 con 0x55638348aac0 2024-09-19T05:09:48.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.303+0000 7f4549600640 1 -- 172.21.15.43:0/2354564190 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4554023040 con 0x55638348aac0 2024-09-19T05:09:48.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.304+0000 7f4549600640 1 -- 172.21.15.43:0/2354564190 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f454c004c60 con 0x55638348aac0 2024-09-19T05:09:48.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.304+0000 7f45651fb740 10 client.26276 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:09:48.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.304+0000 7f45651fb740 10 client.26276 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:09:48.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.304+0000 7f45651fb740 20 client.26276 populate_metadata read hostname 'smithi043' 2024-09-19T05:09:48.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.304+0000 7f45651fb740 1 --2- 172.21.15.43:0/2354564190 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5563835122b0 0x5563835146d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:48.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.304+0000 7f45651fb740 4 client.26276 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T05:09:48.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.304+0000 7f45651fb740 1 -- 172.21.15.43:0/2354564190 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5563833f8d10 con 0x5563835122b0 2024-09-19T05:09:48.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.304+0000 7f4553e00640 1 --2- 172.21.15.43:0/2354564190 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5563835122b0 0x5563835146d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:48.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.305+0000 7f4553e00640 1 --2- 172.21.15.43:0/2354564190 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5563835122b0 0x5563835146d0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10f24f396495a628 server_cookie=401f11f6ef2d9d1f in_seq=0 out_seq=0 2024-09-19T05:09:48.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.305+0000 7f4549600640 10 client.26276 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:09:48.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.306+0000 7f4549600640 1 -- 172.21.15.43:0/2354564190 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5563833f8d10 con 0x5563835122b0 2024-09-19T05:09:48.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.306+0000 7f4549600640 10 client.26276 handle_command_reply: tid=0 2024-09-19T05:09:48.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.306+0000 7f45651fb740 1 client.26276 shutdown 2024-09-19T05:09:48.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.306+0000 7f45651fb740 1 -- 172.21.15.43:0/2354564190 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55638348aac0 msgr2=0x55638349e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:48.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.306+0000 7f45651fb740 1 --2- 172.21.15.43:0/2354564190 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55638348aac0 0x55638349e4e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x5563834a8a00 tx=0x7f454c0046c0 comp rx=0 tx=0).stop 2024-09-19T05:09:48.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.306+0000 7f45651fb740 1 -- 172.21.15.43:0/2354564190 shutdown_connections 2024-09-19T05:09:48.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.306+0000 7f45651fb740 1 --2- 172.21.15.43:0/2354564190 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5563835122b0 0x5563835146d0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:48.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.306+0000 7f45651fb740 1 --2- 172.21.15.43:0/2354564190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556383495ca0 0x55638349b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:48.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.306+0000 7f4549600640 0 client.26276 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:09:48.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.306+0000 7f45651fb740 1 --2- 172.21.15.43:0/2354564190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55638348b3e0 0x55638349acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:48.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.306+0000 7f45651fb740 1 --2- 172.21.15.43:0/2354564190 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55638348aac0 0x55638349e4e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:48.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.306+0000 7f45651fb740 1 -- 172.21.15.43:0/2354564190 >> 172.21.15.43:0/2354564190 conn(0x556383487050 msgr2=0x55638348cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:48.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.306+0000 7f45651fb740 1 -- 172.21.15.43:0/2354564190 shutdown_connections 2024-09-19T05:09:48.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.306+0000 7f45651fb740 1 -- 172.21.15.43:0/2354564190 wait complete. 2024-09-19T05:09:48.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.306+0000 7f45651fb740 20 client.26276 trim_cache size 0 max 16384 2024-09-19T05:09:48.312 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:09:48.312 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-09-19T05:09:48.312 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "4a3c194c-2ad8-47c0-8fcc-c3f5dbee3c41", 2024-09-19T05:09:48.312 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-09-19T05:09:48.312 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:09:48.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.307+0000 7f4562600640 1 -- 172.21.15.43:0/2525950926 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f45400a33b0 msgr2=0x7f45400a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:48.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.307+0000 7f4562600640 1 --2- 172.21.15.43:0/2525950926 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f45400a33b0 0x7f45400a5870 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f454c009fd0 tx=0x7f454c005f90 comp rx=0 tx=0).stop 2024-09-19T05:09:48.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.307+0000 7f4562600640 1 -- 172.21.15.43:0/2525950926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f455c1049a0 msgr2=0x7f455c19abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:48.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.307+0000 7f4562600640 1 --2- 172.21.15.43:0/2525950926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f455c1049a0 0x7f455c19abe0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4544006010 tx=0x7f4544004040 comp rx=0 tx=0).stop 2024-09-19T05:09:48.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.307+0000 7f4562600640 1 -- 172.21.15.43:0/2525950926 shutdown_connections 2024-09-19T05:09:48.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.307+0000 7f4562600640 1 --2- 172.21.15.43:0/2525950926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f455c109120 0x7f455c19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:48.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.307+0000 7f4562600640 1 --2- 172.21.15.43:0/2525950926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f455c1049a0 0x7f455c19abe0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:48.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.307+0000 7f4562600640 1 --2- 172.21.15.43:0/2525950926 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f45400a33b0 0x7f45400a5870 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:48.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.307+0000 7f4562600640 1 --2- 172.21.15.43:0/2525950926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f455c103ff0 0x7f455c19a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:48.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.307+0000 7f4562600640 1 -- 172.21.15.43:0/2525950926 >> 172.21.15.43:0/2525950926 conn(0x7f455c0ffa90 msgr2=0x7f455c100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:48.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.308+0000 7f4562600640 1 -- 172.21.15.43:0/2525950926 shutdown_connections 2024-09-19T05:09:48.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:48.308+0000 7f4562600640 1 -- 172.21.15.43:0/2525950926 wait complete. 2024-09-19T05:09:49.208 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:09:49.588 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:09:49.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:49 smithi043 ceph-mon[30800]: scrub queued for path: / 2024-09-19T05:09:49.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:49 smithi043 ceph-mon[30800]: scrub summary: idle+waiting paths [/] 2024-09-19T05:09:49.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:49 smithi043 ceph-mon[30800]: scrub summary: active paths [/] 2024-09-19T05:09:49.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:49 smithi043 ceph-mon[30800]: scrub summary: idle+waiting paths [/] 2024-09-19T05:09:49.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:49 smithi043 ceph-mon[30800]: pgmap v158: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:09:49.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:49 smithi151 ceph-mon[35425]: scrub queued for path: / 2024-09-19T05:09:49.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:49 smithi151 ceph-mon[35425]: scrub summary: idle+waiting paths [/] 2024-09-19T05:09:49.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:49 smithi151 ceph-mon[35425]: scrub summary: active paths [/] 2024-09-19T05:09:49.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:49 smithi151 ceph-mon[35425]: scrub summary: idle+waiting paths [/] 2024-09-19T05:09:49.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:49 smithi151 ceph-mon[35425]: pgmap v158: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:09:49.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:49 smithi096 ceph-mon[35085]: scrub queued for path: / 2024-09-19T05:09:49.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:49 smithi096 ceph-mon[35085]: scrub summary: idle+waiting paths [/] 2024-09-19T05:09:49.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:49 smithi096 ceph-mon[35085]: scrub summary: active paths [/] 2024-09-19T05:09:49.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:49 smithi096 ceph-mon[35085]: scrub summary: idle+waiting paths [/] 2024-09-19T05:09:49.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:49 smithi096 ceph-mon[35085]: pgmap v158: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:09:50.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.138+0000 7f9708a00640 1 -- 172.21.15.43:0/1078945550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9704103e30 msgr2=0x7f9704104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:50.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.138+0000 7f9708a00640 1 --2- 172.21.15.43:0/1078945550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9704103e30 0x7f9704104210 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f96f4009900 tx=0x7f96f402a3f0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.138+0000 7f9708a00640 1 -- 172.21.15.43:0/1078945550 shutdown_connections 2024-09-19T05:09:50.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.138+0000 7f9708a00640 1 --2- 172.21.15.43:0/1078945550 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9704108f60 0x7f9704110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.138+0000 7f9708a00640 1 --2- 172.21.15.43:0/1078945550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97041047e0 0x7f9704108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.138+0000 7f9708a00640 1 --2- 172.21.15.43:0/1078945550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9704103e30 0x7f9704104210 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.138+0000 7f9708a00640 1 -- 172.21.15.43:0/1078945550 >> 172.21.15.43:0/1078945550 conn(0x7f97040ff8d0 msgr2=0x7f9704101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:50.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.139+0000 7f9708a00640 1 -- 172.21.15.43:0/1078945550 shutdown_connections 2024-09-19T05:09:50.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.139+0000 7f9708a00640 1 -- 172.21.15.43:0/1078945550 wait complete. 2024-09-19T05:09:50.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.139+0000 7f9708a00640 1 Processor -- start 2024-09-19T05:09:50.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.140+0000 7f9708a00640 1 -- start start 2024-09-19T05:09:50.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.140+0000 7f9708a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9704103e30 0x7f970419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:50.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.140+0000 7f9708a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97041047e0 0x7f970419a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:50.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.140+0000 7f9708a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9704108f60 0x7f970419ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:50.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.140+0000 7f9708a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9704114010 con 0x7f97041047e0 2024-09-19T05:09:50.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.140+0000 7f9708a00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9704113e90 con 0x7f9704103e30 2024-09-19T05:09:50.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.140+0000 7f9708a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9704114190 con 0x7f9704108f60 2024-09-19T05:09:50.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.140+0000 7f96fbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97041047e0 0x7f970419a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:50.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.140+0000 7f96fbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97041047e0 0x7f970419a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58698/0 (socket says 172.21.15.43:58698) 2024-09-19T05:09:50.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.140+0000 7f96fbe00640 1 -- 172.21.15.43:0/773465259 learned_addr learned my addr 172.21.15.43:0/773465259 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:50.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.140+0000 7f9703400640 1 --2- 172.21.15.43:0/773465259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9704103e30 0x7f970419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:50.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.140+0000 7f9703e00640 1 --2- 172.21.15.43:0/773465259 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9704108f60 0x7f970419ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:50.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.141+0000 7f96fbe00640 1 -- 172.21.15.43:0/773465259 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9704108f60 msgr2=0x7f970419ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:50.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.141+0000 7f96fbe00640 1 --2- 172.21.15.43:0/773465259 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9704108f60 0x7f970419ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.141+0000 7f96fbe00640 1 -- 172.21.15.43:0/773465259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9704103e30 msgr2=0x7f970419a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:50.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.141+0000 7f96fbe00640 1 --2- 172.21.15.43:0/773465259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9704103e30 0x7f970419a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.141+0000 7f96fbe00640 1 -- 172.21.15.43:0/773465259 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f970419f3e0 con 0x7f97041047e0 2024-09-19T05:09:50.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.141+0000 7f9703400640 1 --2- 172.21.15.43:0/773465259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9704103e30 0x7f970419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:50.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.141+0000 7f96fbe00640 1 --2- 172.21.15.43:0/773465259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97041047e0 0x7f970419a970 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f96ec00e9f0 tx=0x7f96ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:50.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.141+0000 7f9700c00640 1 -- 172.21.15.43:0/773465259 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec00cd80 con 0x7f97041047e0 2024-09-19T05:09:50.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.141+0000 7f9700c00640 1 -- 172.21.15.43:0/773465259 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96ec010910 con 0x7f97041047e0 2024-09-19T05:09:50.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.141+0000 7f9700c00640 1 -- 172.21.15.43:0/773465259 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec018990 con 0x7f97041047e0 2024-09-19T05:09:50.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.142+0000 7f9708a00640 1 -- 172.21.15.43:0/773465259 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f970419f6d0 con 0x7f97041047e0 2024-09-19T05:09:50.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.142+0000 7f9708a00640 1 -- 172.21.15.43:0/773465259 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97041a6fb0 con 0x7f97041047e0 2024-09-19T05:09:50.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.146+0000 7f970cc43740 1 Processor -- start 2024-09-19T05:09:50.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.146+0000 7f970cc43740 1 -- start start 2024-09-19T05:09:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.146+0000 7f970cc43740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ae9dbfbd50 0x55ae9dbfc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.146+0000 7f970cc43740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae9dbfc670 0x55ae9dc069f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.146+0000 7f970cc43740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae9dc06f30 0x55ae9dc09320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.146+0000 7f9700c00640 1 -- 172.21.15.43:0/773465259 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f96ec010ac0 con 0x7f97041047e0 2024-09-19T05:09:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.146+0000 7f970cc43740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ae9db623e0 con 0x55ae9dbfbd50 2024-09-19T05:09:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.146+0000 7f970cc43740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ae9db09d80 con 0x55ae9dbfc670 2024-09-19T05:09:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.146+0000 7f970cc43740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ae9db5e3a0 con 0x55ae9dc06f30 2024-09-19T05:09:50.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.147+0000 7f9703400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae9dbfc670 0x55ae9dc069f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:50.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.147+0000 7f9703e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ae9dbfbd50 0x55ae9dbfc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:50.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.147+0000 7f9703400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae9dbfc670 0x55ae9dc069f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:38886/0 (socket says 172.21.15.43:38886) 2024-09-19T05:09:50.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.147+0000 7f9703400640 1 -- 172.21.15.43:0/4089628682 learned_addr learned my addr 172.21.15.43:0/4089628682 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:50.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.147+0000 7f96fbe00640 1 --2- 172.21.15.43:0/4089628682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae9dc06f30 0x55ae9dc09320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:50.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.147+0000 7f9703400640 1 -- 172.21.15.43:0/4089628682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae9dc06f30 msgr2=0x55ae9dc09320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:50.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.147+0000 7f9703400640 1 --2- 172.21.15.43:0/4089628682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae9dc06f30 0x55ae9dc09320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.147+0000 7f9703400640 1 -- 172.21.15.43:0/4089628682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ae9dbfbd50 msgr2=0x55ae9dbfc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:50.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.147+0000 7f9703400640 1 --2- 172.21.15.43:0/4089628682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ae9dbfbd50 0x55ae9dbfc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.147+0000 7f9703400640 1 -- 172.21.15.43:0/4089628682 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae9db28ae0 con 0x55ae9dbfc670 2024-09-19T05:09:50.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.147+0000 7f96fbe00640 1 --2- 172.21.15.43:0/4089628682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae9dc06f30 0x55ae9dc09320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:50.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.147+0000 7f9703e00640 1 --2- 172.21.15.43:0/4089628682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ae9dbfbd50 0x55ae9dbfc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:50.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.148+0000 7f9703400640 1 --2- 172.21.15.43:0/4089628682 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae9dbfc670 0x55ae9dc069f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f96f4009900 tx=0x7f96f4005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b95fb3d80eb44fc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:50.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.148+0000 7f96f3e00640 1 -- 172.21.15.43:0/4089628682 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f4035040 con 0x55ae9dbfc670 2024-09-19T05:09:50.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.148+0000 7f96f3e00640 1 -- 172.21.15.43:0/4089628682 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96f4004760 con 0x55ae9dbfc670 2024-09-19T05:09:50.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.148+0000 7f96f3e00640 1 -- 172.21.15.43:0/4089628682 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f40038e0 con 0x55ae9dbfc670 2024-09-19T05:09:50.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.149+0000 7f9700c00640 1 --2- 172.21.15.43:0/773465259 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f96e80a3430 0x7f96e80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:50.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.149+0000 7f9700c00640 1 -- 172.21.15.43:0/773465259 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f96ec0cffe0 con 0x7f97041047e0 2024-09-19T05:09:50.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.149+0000 7f9703e00640 1 --2- 172.21.15.43:0/773465259 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f96e80a3430 0x7f96e80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:50.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.149+0000 7f970cc43740 1 -- 172.21.15.43:0/4089628682 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae9dbfc670 msgr2=0x55ae9dc069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:50.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.149+0000 7f970cc43740 1 --2- 172.21.15.43:0/4089628682 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae9dbfc670 0x55ae9dc069f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f96f4009900 tx=0x7f96f4005de0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.150+0000 7f970cc43740 1 -- 172.21.15.43:0/4089628682 shutdown_connections 2024-09-19T05:09:50.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.150+0000 7f970cc43740 1 --2- 172.21.15.43:0/4089628682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae9dc06f30 0x55ae9dc09320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.150+0000 7f970cc43740 1 --2- 172.21.15.43:0/4089628682 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae9dbfc670 0x55ae9dc069f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.150+0000 7f970cc43740 1 --2- 172.21.15.43:0/4089628682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ae9dbfbd50 0x55ae9dbfc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.150+0000 7f970cc43740 1 -- 172.21.15.43:0/4089628682 >> 172.21.15.43:0/4089628682 conn(0x55ae9dbf82e0 msgr2=0x55ae9dbf86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:50.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.150+0000 7f970cc43740 1 -- 172.21.15.43:0/4089628682 shutdown_connections 2024-09-19T05:09:50.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.150+0000 7f970cc43740 1 -- 172.21.15.43:0/4089628682 wait complete. 2024-09-19T05:09:50.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.150+0000 7f9703e00640 1 --2- 172.21.15.43:0/773465259 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f96e80a3430 0x7f96e80a58f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f96fc009fd0 tx=0x7f96fc009500 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:50.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.150+0000 7f970cc43740 1 -- start start 2024-09-19T05:09:50.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.151+0000 7f970cc43740 1 Processor -- start 2024-09-19T05:09:50.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.151+0000 7f970cc43740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ae9dbfbd50 0x55ae9dc03d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:50.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.151+0000 7f970cc43740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae9dbfc670 0x55ae9dc042a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:50.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.151+0000 7f970cc43740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae9dc06f30 0x55ae9dc047e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:50.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.151+0000 7f970cc43740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ae9dc09ba0 con 0x55ae9dbfbd50 2024-09-19T05:09:50.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.151+0000 7f970cc43740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ae9db1d0f0 con 0x55ae9dc06f30 2024-09-19T05:09:50.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.151+0000 7f970cc43740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ae9dc09d20 con 0x55ae9dbfc670 2024-09-19T05:09:50.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.151+0000 7f970cc43740 10 client.0 _ll_register_callbacks cb 0x55ae9db0e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:50.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.151+0000 7f96fbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae9dbfc670 0x55ae9dc042a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:50.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.151+0000 7f9703400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ae9dbfbd50 0x55ae9dc03d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:50.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.151+0000 7f96fbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae9dbfc670 0x55ae9dc042a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:47886/0 (socket says 172.21.15.43:47886) 2024-09-19T05:09:50.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.151+0000 7f96fbe00640 1 -- 172.21.15.43:0/3503451708 learned_addr learned my addr 172.21.15.43:0/3503451708 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:50.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.152+0000 7f9703e00640 1 --2- 172.21.15.43:0/3503451708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae9dc06f30 0x55ae9dc047e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:50.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.152+0000 7f96fbe00640 1 -- 172.21.15.43:0/3503451708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae9dc06f30 msgr2=0x55ae9dc047e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:50.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.152+0000 7f96fbe00640 1 --2- 172.21.15.43:0/3503451708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae9dc06f30 0x55ae9dc047e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.152+0000 7f96fbe00640 1 -- 172.21.15.43:0/3503451708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ae9dbfbd50 msgr2=0x55ae9dc03d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:50.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.152+0000 7f96fbe00640 1 --2- 172.21.15.43:0/3503451708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ae9dbfbd50 0x55ae9dc03d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.152+0000 7f96fbe00640 1 -- 172.21.15.43:0/3503451708 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae9d8d33c0 con 0x55ae9dbfc670 2024-09-19T05:09:50.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.152+0000 7f9703e00640 1 --2- 172.21.15.43:0/3503451708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae9dc06f30 0x55ae9dc047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:09:50.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.152+0000 7f96fbe00640 1 --2- 172.21.15.43:0/3503451708 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae9dbfc670 0x55ae9dc042a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f96ec0cd740 tx=0x7f96ec098680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2dcd05b12757f8d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:50.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.152+0000 7f96f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:09:50.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.152+0000 7f96f0c00640 1 -- 172.21.15.43:0/3503451708 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec0cdb70 con 0x55ae9dbfc670 2024-09-19T05:09:50.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.152+0000 7f970cc43740 1 -- 172.21.15.43:0/3503451708 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96f4004760 con 0x55ae9dbfc670 2024-09-19T05:09:50.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.152+0000 7f96f0c00640 1 -- 172.21.15.43:0/3503451708 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f96ec0a4c50 con 0x55ae9dbfc670 2024-09-19T05:09:50.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.153+0000 7f96f0c00640 1 -- 172.21.15.43:0/3503451708 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f96ec09c320 con 0x55ae9dbfc670 2024-09-19T05:09:50.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.153+0000 7f970cc43740 10 client.26221 fetch_fsmap learned FSMap version 29 2024-09-19T05:09:50.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.153+0000 7f970cc43740 1 -- 172.21.15.43:0/3503451708 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ae9dc157d0 con 0x55ae9dbfc670 2024-09-19T05:09:50.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.153+0000 7f96f0c00640 1 -- 172.21.15.43:0/3503451708 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96f4004760 con 0x55ae9dbfc670 2024-09-19T05:09:50.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.153+0000 7f96f0c00640 1 -- 172.21.15.43:0/3503451708 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f96ec0bc140 con 0x55ae9dbfc670 2024-09-19T05:09:50.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.153+0000 7f970cc43740 10 client.26221 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:09:50.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.153+0000 7f970cc43740 10 client.26221 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:09:50.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.153+0000 7f970cc43740 20 client.26221 populate_metadata read hostname 'smithi043' 2024-09-19T05:09:50.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.153+0000 7f970cc43740 1 --2- 172.21.15.43:0/3503451708 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55ae9dc16c90 0x55ae9dc190a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:50.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.153+0000 7f970cc43740 4 client.26221 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:09:50.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.153+0000 7f970cc43740 1 -- 172.21.15.43:0/3503451708 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae9db6f950 con 0x55ae9dc16c90 2024-09-19T05:09:50.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.154+0000 7f9703400640 1 --2- 172.21.15.43:0/3503451708 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55ae9dc16c90 0x55ae9dc190a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:50.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.154+0000 7f9703400640 1 --2- 172.21.15.43:0/3503451708 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55ae9dc16c90 0x55ae9dc190a0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1acb88d5e5dc320 server_cookie=5a59ad0d017b1008 in_seq=0 out_seq=0 2024-09-19T05:09:50.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.154+0000 7f96f0c00640 10 client.26221 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:09:50.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.155+0000 7f96f0c00640 1 -- 172.21.15.43:0/3503451708 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ae9db6f950 con 0x55ae9dc16c90 2024-09-19T05:09:50.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.155+0000 7f96f0c00640 10 client.26221 handle_command_reply: tid=0 2024-09-19T05:09:50.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.155+0000 7f970cc43740 1 client.26221 shutdown 2024-09-19T05:09:50.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.156+0000 7f970cc43740 1 -- 172.21.15.43:0/3503451708 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae9dbfc670 msgr2=0x55ae9dc042a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:50.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.156+0000 7f970cc43740 1 --2- 172.21.15.43:0/3503451708 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae9dbfc670 0x55ae9dc042a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f96ec0cd740 tx=0x7f96ec098680 comp rx=0 tx=0).stop 2024-09-19T05:09:50.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.156+0000 7f970cc43740 1 -- 172.21.15.43:0/3503451708 shutdown_connections 2024-09-19T05:09:50.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.156+0000 7f970cc43740 1 --2- 172.21.15.43:0/3503451708 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55ae9dc16c90 0x55ae9dc190a0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.156+0000 7f970cc43740 1 --2- 172.21.15.43:0/3503451708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae9dc06f30 0x55ae9dc047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.156+0000 7f970cc43740 1 --2- 172.21.15.43:0/3503451708 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae9dbfc670 0x55ae9dc042a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.156+0000 7f970cc43740 1 --2- 172.21.15.43:0/3503451708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ae9dbfbd50 0x55ae9dc03d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.156+0000 7f970cc43740 1 -- 172.21.15.43:0/3503451708 >> 172.21.15.43:0/3503451708 conn(0x55ae9dbf82e0 msgr2=0x55ae9dbfe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:50.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.156+0000 7f96f0c00640 0 client.26221 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:09:50.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.156+0000 7f970cc43740 1 -- 172.21.15.43:0/3503451708 shutdown_connections 2024-09-19T05:09:50.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.156+0000 7f970cc43740 1 -- 172.21.15.43:0/3503451708 wait complete. 2024-09-19T05:09:50.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.156+0000 7f970cc43740 20 client.26221 trim_cache size 0 max 16384 2024-09-19T05:09:50.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.174+0000 7f970cc43740 1 Processor -- start 2024-09-19T05:09:50.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.174+0000 7f970cc43740 1 -- start start 2024-09-19T05:09:50.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.175+0000 7f970cc43740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae9dbfbd50 0x55ae9dc1e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:50.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.175+0000 7f970cc43740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ae9dbfc670 0x55ae9dc1eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:50.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.175+0000 7f970cc43740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae9dc06f30 0x55ae9dbf6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:50.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.175+0000 7f970cc43740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f97041af900 con 0x55ae9dbfc670 2024-09-19T05:09:50.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.175+0000 7f970cc43740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f97041b1440 con 0x55ae9dc06f30 2024-09-19T05:09:50.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.175+0000 7f970cc43740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f97041affd0 con 0x55ae9dbfbd50 2024-09-19T05:09:50.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.175+0000 7f96fbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ae9dbfc670 0x55ae9dc1eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:50.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.175+0000 7f9703e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae9dc06f30 0x55ae9dbf6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:50.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.175+0000 7f9703400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae9dbfbd50 0x55ae9dc1e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:50.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.175+0000 7f9703e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae9dc06f30 0x55ae9dbf6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:38902/0 (socket says 172.21.15.43:38902) 2024-09-19T05:09:50.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.175+0000 7f96fbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ae9dbfc670 0x55ae9dc1eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58722/0 (socket says 172.21.15.43:58722) 2024-09-19T05:09:50.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.175+0000 7f9703e00640 1 -- 172.21.15.43:0/2280684001 learned_addr learned my addr 172.21.15.43:0/2280684001 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:50.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.175+0000 7f9703e00640 1 -- 172.21.15.43:0/2280684001 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae9dbfbd50 msgr2=0x55ae9dc1e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:50.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.175+0000 7f9703e00640 1 --2- 172.21.15.43:0/2280684001 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae9dbfbd50 0x55ae9dc1e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.175+0000 7f9703e00640 1 -- 172.21.15.43:0/2280684001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ae9dbfc670 msgr2=0x55ae9dc1eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:50.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.175+0000 7f9703e00640 1 --2- 172.21.15.43:0/2280684001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ae9dbfc670 0x55ae9dc1eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.175+0000 7f9703e00640 1 -- 172.21.15.43:0/2280684001 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae9dbf8110 con 0x55ae9dc06f30 2024-09-19T05:09:50.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.176+0000 7f9703e00640 1 --2- 172.21.15.43:0/2280684001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae9dc06f30 0x55ae9dbf6b60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f96fc011450 tx=0x7f96fc016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95d8d55445da4d2d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:50.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.176+0000 7f96f3400640 1 -- 172.21.15.43:0/2280684001 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc004030 con 0x55ae9dc06f30 2024-09-19T05:09:50.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.176+0000 7f96f3400640 1 -- 172.21.15.43:0/2280684001 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96fc0046b0 con 0x55ae9dc06f30 2024-09-19T05:09:50.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.176+0000 7f96f3400640 1 -- 172.21.15.43:0/2280684001 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc010690 con 0x55ae9dc06f30 2024-09-19T05:09:50.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.177+0000 7f970cc43740 1 -- 172.21.15.43:0/2280684001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae9dc06f30 msgr2=0x55ae9dbf6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:50.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.177+0000 7f970cc43740 1 --2- 172.21.15.43:0/2280684001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae9dc06f30 0x55ae9dbf6b60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f96fc011450 tx=0x7f96fc016000 comp rx=0 tx=0).stop 2024-09-19T05:09:50.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.177+0000 7f970cc43740 1 -- 172.21.15.43:0/2280684001 shutdown_connections 2024-09-19T05:09:50.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.177+0000 7f970cc43740 1 --2- 172.21.15.43:0/2280684001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae9dc06f30 0x55ae9dbf6b60 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.177+0000 7f970cc43740 1 --2- 172.21.15.43:0/2280684001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ae9dbfc670 0x55ae9dc1eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.177+0000 7f970cc43740 1 --2- 172.21.15.43:0/2280684001 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae9dbfbd50 0x55ae9dc1e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.177+0000 7f970cc43740 1 -- 172.21.15.43:0/2280684001 >> 172.21.15.43:0/2280684001 conn(0x55ae9dbf82e0 msgr2=0x55ae9dc02e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:50.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.177+0000 7f970cc43740 1 -- 172.21.15.43:0/2280684001 shutdown_connections 2024-09-19T05:09:50.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.177+0000 7f970cc43740 1 -- 172.21.15.43:0/2280684001 wait complete. 2024-09-19T05:09:50.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.178+0000 7f970cc43740 1 -- start start 2024-09-19T05:09:50.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.178+0000 7f970cc43740 1 Processor -- start 2024-09-19T05:09:50.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.178+0000 7f970cc43740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ae9dbfbd50 0x55ae9dc0f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:50.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.178+0000 7f970cc43740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae9dbfc670 0x55ae9dc0aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:50.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.178+0000 7f970cc43740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae9dc06f30 0x55ae9dc0af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:50.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.178+0000 7f970cc43740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f97041b0d70 con 0x55ae9dbfbd50 2024-09-19T05:09:50.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.178+0000 7f970cc43740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f97041b06a0 con 0x55ae9dc06f30 2024-09-19T05:09:50.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.178+0000 7f970cc43740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f97041ad8e0 con 0x55ae9dbfc670 2024-09-19T05:09:50.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.178+0000 7f970cc43740 10 client.0 _ll_register_callbacks cb 0x7f96ec00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:09:50.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.179+0000 7f9703400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ae9dbfbd50 0x55ae9dc0f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:50.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.179+0000 7f9703400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ae9dbfbd50 0x55ae9dc0f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58728/0 (socket says 172.21.15.43:58728) 2024-09-19T05:09:50.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.179+0000 7f9703400640 1 -- 172.21.15.43:0/2434916866 learned_addr learned my addr 172.21.15.43:0/2434916866 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:09:50.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.179+0000 7f9703e00640 1 --2- 172.21.15.43:0/2434916866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae9dc06f30 0x55ae9dc0af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:50.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.179+0000 7f96fbe00640 1 --2- 172.21.15.43:0/2434916866 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae9dbfc670 0x55ae9dc0aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:50.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.179+0000 7f9703400640 1 -- 172.21.15.43:0/2434916866 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae9dbfc670 msgr2=0x55ae9dc0aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:50.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.179+0000 7f9703400640 1 --2- 172.21.15.43:0/2434916866 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae9dbfc670 0x55ae9dc0aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.179+0000 7f9703400640 1 -- 172.21.15.43:0/2434916866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae9dc06f30 msgr2=0x55ae9dc0af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:09:50.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.179+0000 7f9703400640 1 --2- 172.21.15.43:0/2434916866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae9dc06f30 0x55ae9dc0af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.179+0000 7f9703400640 1 -- 172.21.15.43:0/2434916866 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae9dc0b740 con 0x55ae9dbfbd50 2024-09-19T05:09:50.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.179+0000 7f9703e00640 1 --2- 172.21.15.43:0/2434916866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae9dc06f30 0x55ae9dc0af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:09:50.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.180+0000 7f9703400640 1 --2- 172.21.15.43:0/2434916866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ae9dbfbd50 0x55ae9dc0f6a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x55ae9dbf9140 tx=0x7f96f402a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f33aa0ebcf4f13d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:09:50.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.180+0000 7f96f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:09:50.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.180+0000 7f96f0c00640 1 -- 172.21.15.43:0/2434916866 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f4035050 con 0x55ae9dbfbd50 2024-09-19T05:09:50.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.180+0000 7f96f0c00640 1 -- 172.21.15.43:0/2434916866 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f96f40319c0 con 0x55ae9dbfbd50 2024-09-19T05:09:50.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.180+0000 7f970cc43740 1 -- 172.21.15.43:0/2434916866 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96fc0046b0 con 0x55ae9dbfbd50 2024-09-19T05:09:50.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.180+0000 7f96f0c00640 1 -- 172.21.15.43:0/2434916866 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f96f4060050 con 0x55ae9dbfbd50 2024-09-19T05:09:50.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.180+0000 7f970cc43740 10 client.16251 fetch_fsmap learned FSMap version 29 2024-09-19T05:09:50.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.180+0000 7f970cc43740 1 -- 172.21.15.43:0/2434916866 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ae9dc81900 con 0x55ae9dbfbd50 2024-09-19T05:09:50.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.180+0000 7f96f0c00640 1 -- 172.21.15.43:0/2434916866 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96fc0046b0 con 0x55ae9dbfbd50 2024-09-19T05:09:50.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.181+0000 7f96f0c00640 1 -- 172.21.15.43:0/2434916866 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f96f4050480 con 0x55ae9dbfbd50 2024-09-19T05:09:50.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.181+0000 7f970cc43740 10 client.16251 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:09:50.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.181+0000 7f970cc43740 10 client.16251 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:09:50.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.181+0000 7f970cc43740 20 client.16251 populate_metadata read hostname 'smithi043' 2024-09-19T05:09:50.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.181+0000 7f970cc43740 1 --2- 172.21.15.43:0/2434916866 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55ae9dc81d00 0x55ae9dc840f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:09:50.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.181+0000 7f970cc43740 4 client.16251 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:09:50.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.181+0000 7f970cc43740 1 -- 172.21.15.43:0/2434916866 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ae9db69e60 con 0x55ae9dc81d00 2024-09-19T05:09:50.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.181+0000 7f96fbe00640 1 --2- 172.21.15.43:0/2434916866 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55ae9dc81d00 0x55ae9dc840f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:09:50.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.182+0000 7f96fbe00640 1 --2- 172.21.15.43:0/2434916866 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55ae9dc81d00 0x55ae9dc840f0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cdac13128194098 server_cookie=3047adb8a29e55a7 in_seq=0 out_seq=0 2024-09-19T05:09:50.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.182+0000 7f96f0c00640 10 client.16251 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:09:50.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.182+0000 7f96f0c00640 1 -- 172.21.15.43:0/2434916866 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ae9db69e60 con 0x55ae9dc81d00 2024-09-19T05:09:50.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.182+0000 7f96f0c00640 10 client.16251 handle_command_reply: tid=0 2024-09-19T05:09:50.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.182+0000 7f970cc43740 1 client.16251 shutdown 2024-09-19T05:09:50.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.182+0000 7f970cc43740 1 -- 172.21.15.43:0/2434916866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ae9dbfbd50 msgr2=0x55ae9dc0f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:50.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.182+0000 7f970cc43740 1 --2- 172.21.15.43:0/2434916866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ae9dbfbd50 0x55ae9dc0f6a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x55ae9dbf9140 tx=0x7f96f402a3f0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.182+0000 7f970cc43740 1 -- 172.21.15.43:0/2434916866 shutdown_connections 2024-09-19T05:09:50.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.182+0000 7f970cc43740 1 --2- 172.21.15.43:0/2434916866 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55ae9dc81d00 0x55ae9dc840f0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.182+0000 7f970cc43740 1 --2- 172.21.15.43:0/2434916866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae9dc06f30 0x55ae9dc0af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.182+0000 7f970cc43740 1 --2- 172.21.15.43:0/2434916866 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae9dbfc670 0x55ae9dc0aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.182+0000 7f970cc43740 1 --2- 172.21.15.43:0/2434916866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ae9dbfbd50 0x55ae9dc0f6a0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.182+0000 7f970cc43740 1 -- 172.21.15.43:0/2434916866 >> 172.21.15.43:0/2434916866 conn(0x55ae9dbf82e0 msgr2=0x55ae9dbf6b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:50.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.182+0000 7f96f0c00640 0 client.16251 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:09:50.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.182+0000 7f970cc43740 1 -- 172.21.15.43:0/2434916866 shutdown_connections 2024-09-19T05:09:50.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.182+0000 7f970cc43740 1 -- 172.21.15.43:0/2434916866 wait complete. 2024-09-19T05:09:50.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.182+0000 7f970cc43740 20 client.16251 trim_cache size 0 max 16384 2024-09-19T05:09:50.188 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:09:50.188 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T05:09:50.188 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-09-19T05:09:50.189 INFO:teuthology.orchestra.run.smithi043.stdout: "4a3c194c-2ad8-47c0-8fcc-c3f5dbee3c41": { 2024-09-19T05:09:50.189 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-09-19T05:09:50.189 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "4a3c194c-2ad8-47c0-8fcc-c3f5dbee3c41", 2024-09-19T05:09:50.189 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-09-19T05:09:50.189 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:09:50.189 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:09:50.189 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:09:50.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.183+0000 7f9708a00640 1 -- 172.21.15.43:0/773465259 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f96e80a3430 msgr2=0x7f96e80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:50.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.183+0000 7f9708a00640 1 --2- 172.21.15.43:0/773465259 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f96e80a3430 0x7f96e80a58f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f96fc009fd0 tx=0x7f96fc009500 comp rx=0 tx=0).stop 2024-09-19T05:09:50.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.183+0000 7f9708a00640 1 -- 172.21.15.43:0/773465259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97041047e0 msgr2=0x7f970419a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:09:50.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.183+0000 7f9708a00640 1 --2- 172.21.15.43:0/773465259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97041047e0 0x7f970419a970 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f96ec00e9f0 tx=0x7f96ec00eeb0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.183+0000 7f9708a00640 1 -- 172.21.15.43:0/773465259 shutdown_connections 2024-09-19T05:09:50.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.183+0000 7f9708a00640 1 --2- 172.21.15.43:0/773465259 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9704108f60 0x7f970419ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.183+0000 7f9708a00640 1 --2- 172.21.15.43:0/773465259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97041047e0 0x7f970419a970 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.183+0000 7f9708a00640 1 --2- 172.21.15.43:0/773465259 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f96e80a3430 0x7f96e80a58f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.183+0000 7f9708a00640 1 --2- 172.21.15.43:0/773465259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9704103e30 0x7f970419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:09:50.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.183+0000 7f9708a00640 1 -- 172.21.15.43:0/773465259 >> 172.21.15.43:0/773465259 conn(0x7f97040ff8d0 msgr2=0x7f9704100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:09:50.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.183+0000 7f9708a00640 1 -- 172.21.15.43:0/773465259 shutdown_connections 2024-09-19T05:09:50.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:09:50.183+0000 7f9708a00640 1 -- 172.21.15.43:0/773465259 wait complete. 2024-09-19T05:09:51.052 INFO:tasks.cephfs.filesystem:scrub status for tag:4a3c194c-2ad8-47c0-8fcc-c3f5dbee3c41 - {'path': '/', 'tag': '4a3c194c-2ad8-47c0-8fcc-c3f5dbee3c41', 'options': 'recursive,force'} 2024-09-19T05:09:51.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:51 smithi151 ceph-mon[35425]: pgmap v159: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:09:51.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:51 smithi096 ceph-mon[35085]: pgmap v159: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:09:52.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:51 smithi043 ceph-mon[30800]: pgmap v159: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:09:53.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:53 smithi151 ceph-mon[35425]: pgmap v160: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:09:53.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:53 smithi151 ceph-mon[35425]: scrub summary: idle 2024-09-19T05:09:53.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:53 smithi096 ceph-mon[35085]: pgmap v160: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:09:53.974 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:53 smithi096 ceph-mon[35085]: scrub summary: idle 2024-09-19T05:09:54.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:53 smithi043 ceph-mon[30800]: pgmap v160: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:09:54.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:53 smithi043 ceph-mon[30800]: scrub summary: idle 2024-09-19T05:09:55.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:55 smithi151 ceph-mon[35425]: pgmap v161: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:09:55.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:55 smithi096 ceph-mon[35085]: pgmap v161: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:09:56.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:55 smithi043 ceph-mon[30800]: pgmap v161: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:09:56.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:56 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:09:56.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:56 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:09:57.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:56 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:09:57.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:57 smithi151 ceph-mon[35425]: pgmap v162: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:09:57.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:57 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:09:57.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:57 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:09:57.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:57 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:09:57.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:57 smithi096 ceph-mon[35085]: pgmap v162: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:09:57.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:57 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:09:57.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:57 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:09:57.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:57 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:09:58.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:57 smithi043 ceph-mon[30800]: pgmap v162: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:09:58.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:57 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:09:58.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:57 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:09:58.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:57 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:09:59.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:09:59 smithi151 ceph-mon[35425]: pgmap v163: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:09:59.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:09:59 smithi096 ceph-mon[35085]: pgmap v163: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:10:00.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:09:59 smithi043 ceph-mon[30800]: pgmap v163: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:10:00.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:00 smithi043 ceph-mon[30800]: overall HEALTH_OK 2024-09-19T05:10:00.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:00 smithi151 ceph-mon[35425]: overall HEALTH_OK 2024-09-19T05:10:00.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:00 smithi096 ceph-mon[35085]: overall HEALTH_OK 2024-09-19T05:10:01.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:01 smithi151 ceph-mon[35425]: pgmap v164: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:10:01.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:01 smithi096 ceph-mon[35085]: pgmap v164: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:10:02.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:01 smithi043 ceph-mon[30800]: pgmap v164: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:10:03.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:03 smithi151 ceph-mon[35425]: pgmap v165: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:10:03.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:03 smithi096 ceph-mon[35085]: pgmap v165: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:10:04.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:03 smithi043 ceph-mon[30800]: pgmap v165: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:10:05.905 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:05 smithi151 ceph-mon[35425]: pgmap v166: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-19T05:10:05.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:05 smithi096 ceph-mon[35085]: pgmap v166: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-19T05:10:06.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:05 smithi043 ceph-mon[30800]: pgmap v166: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-19T05:10:07.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:07 smithi096 ceph-mon[35085]: pgmap v167: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-19T05:10:08.032 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:07 smithi043 ceph-mon[30800]: pgmap v167: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-19T05:10:08.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:07 smithi151 ceph-mon[35425]: pgmap v167: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-19T05:10:09.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:09 smithi096 ceph-mon[35085]: pgmap v168: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-19T05:10:10.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:09 smithi043 ceph-mon[30800]: pgmap v168: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-19T05:10:10.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:09 smithi151 ceph-mon[35425]: pgmap v168: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-19T05:10:11.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:11 smithi043 ceph-mon[30800]: pgmap v169: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-19T05:10:11.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:11 smithi096 ceph-mon[35085]: pgmap v169: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-19T05:10:12.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:11 smithi151 ceph-mon[35425]: pgmap v169: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-19T05:10:13.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:13 smithi096 ceph-mon[35085]: pgmap v170: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-19T05:10:14.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:13 smithi043 ceph-mon[30800]: pgmap v170: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-19T05:10:14.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:13 smithi151 ceph-mon[35425]: pgmap v170: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-19T05:10:15.973 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:15 smithi096 ceph-mon[35085]: pgmap v171: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-19T05:10:16.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:15 smithi043 ceph-mon[30800]: pgmap v171: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-19T05:10:16.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:15 smithi151 ceph-mon[35425]: pgmap v171: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-19T05:10:18.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:17 smithi151 ceph-mon[35425]: pgmap v172: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-19T05:10:18.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:17 smithi096 ceph-mon[35085]: pgmap v172: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-19T05:10:18.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:17 smithi043 ceph-mon[30800]: pgmap v172: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-19T05:10:20.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:19 smithi151 ceph-mon[35425]: pgmap v173: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-19T05:10:20.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:19 smithi096 ceph-mon[35085]: pgmap v173: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-19T05:10:20.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:19 smithi043 ceph-mon[30800]: pgmap v173: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-19T05:10:21.053 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:10:21.447 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:10:21.843 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:21 smithi043 ceph-mon[30800]: pgmap v174: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-19T05:10:22.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.110+0000 7f7aba000640 1 -- 172.21.15.43:0/96724692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ab41079d0 msgr2=0x7f7ab4107db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:22.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.110+0000 7f7aba000640 1 --2- 172.21.15.43:0/96724692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ab41079d0 0x7f7ab4107db0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4009930 tx=0x7f7aa402a420 comp rx=0 tx=0).stop 2024-09-19T05:10:22.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.110+0000 7f7aba000640 1 -- 172.21.15.43:0/96724692 shutdown_connections 2024-09-19T05:10:22.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.110+0000 7f7aba000640 1 --2- 172.21.15.43:0/96724692 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ab4102370 0x7f7ab410e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:22.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.110+0000 7f7aba000640 1 --2- 172.21.15.43:0/96724692 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ab41019d0 0x7f7ab4101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:22.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.110+0000 7f7aba000640 1 --2- 172.21.15.43:0/96724692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ab41079d0 0x7f7ab4107db0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:22.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.110+0000 7f7aba000640 1 -- 172.21.15.43:0/96724692 >> 172.21.15.43:0/96724692 conn(0x7f7ab40fd710 msgr2=0x7f7ab40ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:22.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.110+0000 7f7aba000640 1 -- 172.21.15.43:0/96724692 shutdown_connections 2024-09-19T05:10:22.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.110+0000 7f7aba000640 1 -- 172.21.15.43:0/96724692 wait complete. 2024-09-19T05:10:22.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.111+0000 7f7aba000640 1 Processor -- start 2024-09-19T05:10:22.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.111+0000 7f7aba000640 1 -- start start 2024-09-19T05:10:22.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.112+0000 7f7aba000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ab41019d0 0x7f7ab40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:22.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.112+0000 7f7aba000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ab4102370 0x7f7ab4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:22.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.112+0000 7f7aba000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ab41079d0 0x7f7ab4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:22.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.112+0000 7f7aba000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab4111dd0 con 0x7f7ab4102370 2024-09-19T05:10:22.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.112+0000 7f7aba000640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab4111c50 con 0x7f7ab41019d0 2024-09-19T05:10:22.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.112+0000 7f7aba000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab4111f50 con 0x7f7ab41079d0 2024-09-19T05:10:22.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.112+0000 7f7ab8c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ab41019d0 0x7f7ab40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:22.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.112+0000 7f7aabe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ab4102370 0x7f7ab4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:22.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.112+0000 7f7ab8c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ab41019d0 0x7f7ab40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:47360/0 (socket says 172.21.15.43:47360) 2024-09-19T05:10:22.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.112+0000 7f7aabe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ab4102370 0x7f7ab4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43642/0 (socket says 172.21.15.43:43642) 2024-09-19T05:10:22.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.112+0000 7f7ab8c00640 1 -- 172.21.15.43:0/3035716630 learned_addr learned my addr 172.21.15.43:0/3035716630 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:22.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.112+0000 7f7ab9600640 1 --2- 172.21.15.43:0/3035716630 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ab41079d0 0x7f7ab4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:22.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.113+0000 7f7ab8c00640 1 -- 172.21.15.43:0/3035716630 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ab41079d0 msgr2=0x7f7ab4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:22.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.113+0000 7f7ab8c00640 1 --2- 172.21.15.43:0/3035716630 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ab41079d0 0x7f7ab4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:22.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.113+0000 7f7ab8c00640 1 -- 172.21.15.43:0/3035716630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ab4102370 msgr2=0x7f7ab4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:22.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.113+0000 7f7ab8c00640 1 --2- 172.21.15.43:0/3035716630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ab4102370 0x7f7ab4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:22.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.113+0000 7f7ab8c00640 1 -- 172.21.15.43:0/3035716630 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab41a67d0 con 0x7f7ab41019d0 2024-09-19T05:10:22.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.113+0000 7f7ab8c00640 1 --2- 172.21.15.43:0/3035716630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ab41019d0 0x7f7ab40744f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4009230 tx=0x7f7aa4004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:22.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.113+0000 7f7ab2000640 1 -- 172.21.15.43:0/3035716630 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa403f030 con 0x7f7ab41019d0 2024-09-19T05:10:22.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.113+0000 7f7aba000640 1 -- 172.21.15.43:0/3035716630 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ab41a6a60 con 0x7f7ab41019d0 2024-09-19T05:10:22.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.113+0000 7f7ab2000640 1 -- 172.21.15.43:0/3035716630 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7aa4004440 con 0x7f7ab41019d0 2024-09-19T05:10:22.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.114+0000 7f7aba000640 1 -- 172.21.15.43:0/3035716630 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ab41a6f90 con 0x7f7ab41019d0 2024-09-19T05:10:22.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.114+0000 7f7ab2000640 1 -- 172.21.15.43:0/3035716630 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4035070 con 0x7f7ab41019d0 2024-09-19T05:10:22.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.117+0000 7f7abe04b740 1 Processor -- start 2024-09-19T05:10:22.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.117+0000 7f7abe04b740 1 -- start start 2024-09-19T05:10:22.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.117+0000 7f7ab2000640 1 -- 172.21.15.43:0/3035716630 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f7aa4042030 con 0x7f7ab41019d0 2024-09-19T05:10:22.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.117+0000 7f7abe04b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ece108d50 0x561ece109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:22.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.117+0000 7f7abe04b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ece109670 0x561ece1139f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:22.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.117+0000 7f7abe04b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ece113f30 0x561ece116320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:22.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.117+0000 7f7abe04b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561ece06f3e0 con 0x561ece108d50 2024-09-19T05:10:22.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.117+0000 7f7abe04b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561ece016d80 con 0x561ece113f30 2024-09-19T05:10:22.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.117+0000 7f7abe04b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561ece06b3a0 con 0x561ece109670 2024-09-19T05:10:22.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.118+0000 7f7ab9600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ece108d50 0x561ece109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:22.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.118+0000 7f7ab8c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ece109670 0x561ece1139f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:22.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.118+0000 7f7ab9600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ece108d50 0x561ece109130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43650/0 (socket says 172.21.15.43:43650) 2024-09-19T05:10:22.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.118+0000 7f7ab9600640 1 -- 172.21.15.43:0/2826075180 learned_addr learned my addr 172.21.15.43:0/2826075180 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:22.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.118+0000 7f7aabe00640 1 --2- 172.21.15.43:0/2826075180 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ece113f30 0x561ece116320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:22.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.118+0000 7f7aabe00640 1 -- 172.21.15.43:0/2826075180 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ece109670 msgr2=0x561ece1139f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:22.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.118+0000 7f7aabe00640 1 --2- 172.21.15.43:0/2826075180 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ece109670 0x561ece1139f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:22.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.118+0000 7f7aabe00640 1 -- 172.21.15.43:0/2826075180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ece108d50 msgr2=0x561ece109130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:22.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.118+0000 7f7aabe00640 1 --2- 172.21.15.43:0/2826075180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ece108d50 0x561ece109130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:22.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.118+0000 7f7aabe00640 1 -- 172.21.15.43:0/2826075180 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ece035ae0 con 0x561ece113f30 2024-09-19T05:10:22.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.118+0000 7f7aabe00640 1 --2- 172.21.15.43:0/2826075180 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ece113f30 0x561ece116320 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009bd0 tx=0x7f7a9c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=896e54832f26946 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:22.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.119+0000 7f7aa9c00640 1 -- 172.21.15.43:0/2826075180 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c004230 con 0x561ece113f30 2024-09-19T05:10:22.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.119+0000 7f7aa9c00640 1 -- 172.21.15.43:0/2826075180 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a9c009e40 con 0x561ece113f30 2024-09-19T05:10:22.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.119+0000 7f7aa9c00640 1 -- 172.21.15.43:0/2826075180 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c004db0 con 0x561ece113f30 2024-09-19T05:10:22.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.119+0000 7f7ab2000640 1 --2- 172.21.15.43:0/3035716630 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f7a980a3430 0x7f7a980a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:22.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.119+0000 7f7ab2000640 1 -- 172.21.15.43:0/3035716630 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f7aa40effe0 con 0x7f7ab41019d0 2024-09-19T05:10:22.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.119+0000 7f7ab9600640 1 --2- 172.21.15.43:0/3035716630 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f7a980a3430 0x7f7a980a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:22.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.120+0000 7f7abe04b740 1 -- 172.21.15.43:0/2826075180 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ece113f30 msgr2=0x561ece116320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:22.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.120+0000 7f7abe04b740 1 --2- 172.21.15.43:0/2826075180 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ece113f30 0x561ece116320 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009bd0 tx=0x7f7a9c011810 comp rx=0 tx=0).stop 2024-09-19T05:10:22.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.120+0000 7f7abe04b740 1 -- 172.21.15.43:0/2826075180 shutdown_connections 2024-09-19T05:10:22.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.120+0000 7f7abe04b740 1 --2- 172.21.15.43:0/2826075180 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ece113f30 0x561ece116320 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:22.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.120+0000 7f7abe04b740 1 --2- 172.21.15.43:0/2826075180 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ece109670 0x561ece1139f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:22.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.120+0000 7f7abe04b740 1 --2- 172.21.15.43:0/2826075180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ece108d50 0x561ece109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:22.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.120+0000 7f7abe04b740 1 -- 172.21.15.43:0/2826075180 >> 172.21.15.43:0/2826075180 conn(0x561ece1052e0 msgr2=0x561ece1056c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:22.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.120+0000 7f7abe04b740 1 -- 172.21.15.43:0/2826075180 shutdown_connections 2024-09-19T05:10:22.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.120+0000 7f7abe04b740 1 -- 172.21.15.43:0/2826075180 wait complete. 2024-09-19T05:10:22.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.120+0000 7f7ab9600640 1 --2- 172.21.15.43:0/3035716630 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f7a980a3430 0x7f7a980a58f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7aac011630 tx=0x7f7aac009500 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:22.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.120+0000 7f7abe04b740 1 -- start start 2024-09-19T05:10:22.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.121+0000 7f7abe04b740 1 Processor -- start 2024-09-19T05:10:22.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.121+0000 7f7abe04b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ece108d50 0x561ece110d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:22.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.121+0000 7f7abe04b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ece109670 0x561ece1112b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:22.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.121+0000 7f7abe04b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ece113f30 0x561ece1117f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:22.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.121+0000 7f7abe04b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561ece116ba0 con 0x561ece109670 2024-09-19T05:10:22.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.121+0000 7f7abe04b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561ece02a0f0 con 0x561ece113f30 2024-09-19T05:10:22.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.121+0000 7f7abe04b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561ece116d20 con 0x561ece108d50 2024-09-19T05:10:22.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.121+0000 7f7abe04b740 10 client.0 _ll_register_callbacks cb 0x561ece01b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:22.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.121+0000 7f7ab8c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ece108d50 0x561ece110d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:22.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.122+0000 7f7ab8c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ece108d50 0x561ece110d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:54904/0 (socket says 172.21.15.43:54904) 2024-09-19T05:10:22.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.122+0000 7f7aabe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ece109670 0x561ece1112b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:22.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.122+0000 7f7ab8c00640 1 -- 172.21.15.43:0/3954808829 learned_addr learned my addr 172.21.15.43:0/3954808829 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:22.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.122+0000 7f7ab9600640 1 --2- 172.21.15.43:0/3954808829 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ece113f30 0x561ece1117f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:22.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.122+0000 7f7ab8c00640 1 -- 172.21.15.43:0/3954808829 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ece113f30 msgr2=0x561ece1117f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:22.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.122+0000 7f7ab8c00640 1 --2- 172.21.15.43:0/3954808829 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ece113f30 0x561ece1117f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:22.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.122+0000 7f7ab8c00640 1 -- 172.21.15.43:0/3954808829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ece109670 msgr2=0x561ece1112b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:22.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.122+0000 7f7ab8c00640 1 --2- 172.21.15.43:0/3954808829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ece109670 0x561ece1112b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:22.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.122+0000 7f7ab8c00640 1 -- 172.21.15.43:0/3954808829 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ecdde03c0 con 0x561ece108d50 2024-09-19T05:10:22.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.122+0000 7f7ab8c00640 1 --2- 172.21.15.43:0/3954808829 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ece108d50 0x561ece110d70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4009900 tx=0x7f7aa403b520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5630e05c6cf18f3f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:22.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.122+0000 7f7aa2000640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:10:22.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.122+0000 7f7aa2000640 1 -- 172.21.15.43:0/3954808829 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4049050 con 0x561ece108d50 2024-09-19T05:10:22.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.122+0000 7f7aa2000640 1 -- 172.21.15.43:0/3954808829 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f7aa40b89f0 con 0x561ece108d50 2024-09-19T05:10:22.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.122+0000 7f7abe04b740 1 -- 172.21.15.43:0/3954808829 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561ece07c950 con 0x561ece108d50 2024-09-19T05:10:22.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.123+0000 7f7aa2000640 1 -- 172.21.15.43:0/3954808829 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7aa40bbec0 con 0x561ece108d50 2024-09-19T05:10:22.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.123+0000 7f7abe04b740 10 client.26239 fetch_fsmap learned FSMap version 29 2024-09-19T05:10:22.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.123+0000 7f7abe04b740 1 -- 172.21.15.43:0/3954808829 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561ece116ba0 con 0x561ece108d50 2024-09-19T05:10:22.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.123+0000 7f7aa2000640 1 -- 172.21.15.43:0/3954808829 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x561ece07c950 con 0x561ece108d50 2024-09-19T05:10:22.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.123+0000 7f7aa2000640 1 -- 172.21.15.43:0/3954808829 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7aa40f20d0 con 0x561ece108d50 2024-09-19T05:10:22.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.123+0000 7f7abe04b740 10 client.26239 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:10:22.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.123+0000 7f7abe04b740 10 client.26239 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:10:22.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.123+0000 7f7abe04b740 20 client.26239 populate_metadata read hostname 'smithi043' 2024-09-19T05:10:22.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.123+0000 7f7abe04b740 1 --2- 172.21.15.43:0/3954808829 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x561ece123c40 0x561ece126090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:22.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.123+0000 7f7abe04b740 4 client.26239 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:10:22.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.123+0000 7f7abe04b740 1 -- 172.21.15.43:0/3954808829 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ece076e60 con 0x561ece123c40 2024-09-19T05:10:22.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.124+0000 7f7aabe00640 1 --2- 172.21.15.43:0/3954808829 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x561ece123c40 0x561ece126090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:22.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.124+0000 7f7aabe00640 1 --2- 172.21.15.43:0/3954808829 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x561ece123c40 0x561ece126090 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e764b1936a1dde6 server_cookie=39906e340c253560 in_seq=0 out_seq=0 2024-09-19T05:10:22.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.124+0000 7f7aa2000640 10 client.26239 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:10:22.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.125+0000 7f7aa2000640 1 -- 172.21.15.43:0/3954808829 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561ece076e60 con 0x561ece123c40 2024-09-19T05:10:22.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.125+0000 7f7aa2000640 10 client.26239 handle_command_reply: tid=0 2024-09-19T05:10:22.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.125+0000 7f7abe04b740 1 client.26239 shutdown 2024-09-19T05:10:22.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.125+0000 7f7abe04b740 1 -- 172.21.15.43:0/3954808829 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ece108d50 msgr2=0x561ece110d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:22.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.126+0000 7f7abe04b740 1 --2- 172.21.15.43:0/3954808829 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ece108d50 0x561ece110d70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4009900 tx=0x7f7aa403b520 comp rx=0 tx=0).stop 2024-09-19T05:10:22.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.126+0000 7f7abe04b740 1 -- 172.21.15.43:0/3954808829 shutdown_connections 2024-09-19T05:10:22.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.126+0000 7f7abe04b740 1 --2- 172.21.15.43:0/3954808829 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x561ece123c40 0x561ece126090 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:22.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.126+0000 7f7abe04b740 1 --2- 172.21.15.43:0/3954808829 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ece113f30 0x561ece1117f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:22.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.126+0000 7f7aa2000640 0 client.26239 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:10:22.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.126+0000 7f7abe04b740 1 --2- 172.21.15.43:0/3954808829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ece109670 0x561ece1112b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:22.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.126+0000 7f7abe04b740 1 --2- 172.21.15.43:0/3954808829 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ece108d50 0x561ece110d70 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:22.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.126+0000 7f7abe04b740 1 -- 172.21.15.43:0/3954808829 >> 172.21.15.43:0/3954808829 conn(0x561ece1052e0 msgr2=0x561ece10b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:22.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.126+0000 7f7abe04b740 1 -- 172.21.15.43:0/3954808829 shutdown_connections 2024-09-19T05:10:22.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.126+0000 7f7abe04b740 1 -- 172.21.15.43:0/3954808829 wait complete. 2024-09-19T05:10:22.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.126+0000 7f7abe04b740 20 client.26239 trim_cache size 0 max 16384 2024-09-19T05:10:22.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.144+0000 7f7abe04b740 1 Processor -- start 2024-09-19T05:10:22.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.144+0000 7f7abe04b740 1 -- start start 2024-09-19T05:10:22.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.144+0000 7f7abe04b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ece108d50 0x561ece12b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:22.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.144+0000 7f7abe04b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ece109670 0x561ece12be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:22.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.144+0000 7f7abe04b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ece113f30 0x561ece1125d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:22.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.144+0000 7f7abe04b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab41af930 con 0x561ece109670 2024-09-19T05:10:22.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.144+0000 7f7abe04b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab41b1470 con 0x561ece108d50 2024-09-19T05:10:22.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.144+0000 7f7abe04b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab41b0000 con 0x561ece113f30 2024-09-19T05:10:22.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.145+0000 7f7aabe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ece109670 0x561ece12be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:22.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.145+0000 7f7aabe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ece109670 0x561ece12be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43668/0 (socket says 172.21.15.43:43668) 2024-09-19T05:10:22.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.145+0000 7f7aabe00640 1 -- 172.21.15.43:0/3031096921 learned_addr learned my addr 172.21.15.43:0/3031096921 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:22.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.145+0000 7f7ab8c00640 1 --2- 172.21.15.43:0/3031096921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ece108d50 0x561ece12b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:22.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.145+0000 7f7ab9600640 1 --2- 172.21.15.43:0/3031096921 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ece113f30 0x561ece1125d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:22.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.145+0000 7f7aabe00640 1 -- 172.21.15.43:0/3031096921 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ece113f30 msgr2=0x561ece1125d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:22.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.145+0000 7f7aabe00640 1 --2- 172.21.15.43:0/3031096921 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ece113f30 0x561ece1125d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:22.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.145+0000 7f7aabe00640 1 -- 172.21.15.43:0/3031096921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ece108d50 msgr2=0x561ece12b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:22.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.145+0000 7f7aabe00640 1 --2- 172.21.15.43:0/3031096921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ece108d50 0x561ece12b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:22.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.145+0000 7f7aabe00640 1 -- 172.21.15.43:0/3031096921 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ece105110 con 0x561ece109670 2024-09-19T05:10:22.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.145+0000 7f7ab9600640 1 --2- 172.21.15.43:0/3031096921 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ece113f30 0x561ece1125d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:10:22.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.145+0000 7f7aabe00640 1 --2- 172.21.15.43:0/3031096921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ece109670 0x561ece12be40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c0058c0 tx=0x7f7a9c005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=883d74b9d4810169 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:22.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.146+0000 7f7aa9200640 1 -- 172.21.15.43:0/3031096921 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c030070 con 0x561ece109670 2024-09-19T05:10:22.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.146+0000 7f7aa9200640 1 -- 172.21.15.43:0/3031096921 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a9c004470 con 0x561ece109670 2024-09-19T05:10:22.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.146+0000 7f7aa9200640 1 -- 172.21.15.43:0/3031096921 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c01e040 con 0x561ece109670 2024-09-19T05:10:22.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.146+0000 7f7abe04b740 1 -- 172.21.15.43:0/3031096921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ece109670 msgr2=0x561ece12be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:22.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.146+0000 7f7abe04b740 1 --2- 172.21.15.43:0/3031096921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ece109670 0x561ece12be40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c0058c0 tx=0x7f7a9c005f10 comp rx=0 tx=0).stop 2024-09-19T05:10:22.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.146+0000 7f7abe04b740 1 -- 172.21.15.43:0/3031096921 shutdown_connections 2024-09-19T05:10:22.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.146+0000 7f7abe04b740 1 --2- 172.21.15.43:0/3031096921 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ece113f30 0x561ece1125d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:22.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.146+0000 7f7abe04b740 1 --2- 172.21.15.43:0/3031096921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ece109670 0x561ece12be40 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:22.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.146+0000 7f7abe04b740 1 --2- 172.21.15.43:0/3031096921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ece108d50 0x561ece12b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:22.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.146+0000 7f7abe04b740 1 -- 172.21.15.43:0/3031096921 >> 172.21.15.43:0/3031096921 conn(0x561ece1052e0 msgr2=0x561ece10fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:22.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.146+0000 7f7abe04b740 1 -- 172.21.15.43:0/3031096921 shutdown_connections 2024-09-19T05:10:22.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.146+0000 7f7abe04b740 1 -- 172.21.15.43:0/3031096921 wait complete. 2024-09-19T05:10:22.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.147+0000 7f7abe04b740 1 -- start start 2024-09-19T05:10:22.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.147+0000 7f7abe04b740 1 Processor -- start 2024-09-19T05:10:22.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.147+0000 7f7abe04b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ece108d50 0x561ece11c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:22.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.148+0000 7f7abe04b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ece109670 0x561ece117900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:22.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.148+0000 7f7abe04b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ece113f30 0x561ece117e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:22.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.148+0000 7f7abe04b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab41b0da0 con 0x561ece108d50 2024-09-19T05:10:22.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.148+0000 7f7abe04b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab41b06d0 con 0x561ece113f30 2024-09-19T05:10:22.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.148+0000 7f7abe04b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab41ad910 con 0x561ece109670 2024-09-19T05:10:22.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.148+0000 7f7abe04b740 10 client.0 _ll_register_callbacks cb 0x7f7aa4005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:22.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.148+0000 7f7ab9600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ece113f30 0x561ece117e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:22.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.148+0000 7f7ab9600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ece113f30 0x561ece117e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:47384/0 (socket says 172.21.15.43:47384) 2024-09-19T05:10:22.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.148+0000 7f7ab9600640 1 -- 172.21.15.43:0/3284177487 learned_addr learned my addr 172.21.15.43:0/3284177487 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:22.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.148+0000 7f7ab8c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ece108d50 0x561ece11c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:22.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.148+0000 7f7aabe00640 1 --2- 172.21.15.43:0/3284177487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ece109670 0x561ece117900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:22.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.148+0000 7f7ab9600640 1 -- 172.21.15.43:0/3284177487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ece109670 msgr2=0x561ece117900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:22.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.148+0000 7f7ab9600640 1 --2- 172.21.15.43:0/3284177487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ece109670 0x561ece117900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:22.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.148+0000 7f7ab9600640 1 -- 172.21.15.43:0/3284177487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ece108d50 msgr2=0x561ece11c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:22.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.148+0000 7f7ab9600640 1 --2- 172.21.15.43:0/3284177487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ece108d50 0x561ece11c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:22.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.148+0000 7f7ab9600640 1 -- 172.21.15.43:0/3284177487 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ece105f70 con 0x561ece113f30 2024-09-19T05:10:22.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.149+0000 7f7ab9600640 1 --2- 172.21.15.43:0/3284177487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ece113f30 0x561ece117e40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7aac004040 tx=0x7f7aac004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e820ff95fc5ab91 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:22.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.149+0000 7f7aa2000640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:10:22.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.149+0000 7f7aa2000640 1 -- 172.21.15.43:0/3284177487 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aac02d060 con 0x561ece113f30 2024-09-19T05:10:22.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.149+0000 7f7aa2000640 1 -- 172.21.15.43:0/3284177487 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f7aac00f9d0 con 0x561ece113f30 2024-09-19T05:10:22.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.149+0000 7f7abe04b740 1 -- 172.21.15.43:0/3284177487 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a9c004470 con 0x561ece113f30 2024-09-19T05:10:22.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.149+0000 7f7aa2000640 1 -- 172.21.15.43:0/3284177487 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7aac00fbd0 con 0x561ece113f30 2024-09-19T05:10:22.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.149+0000 7f7abe04b740 10 client.26312 fetch_fsmap learned FSMap version 29 2024-09-19T05:10:22.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.149+0000 7f7abe04b740 1 -- 172.21.15.43:0/3284177487 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561ece18e650 con 0x561ece113f30 2024-09-19T05:10:22.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.150+0000 7f7aa2000640 1 -- 172.21.15.43:0/3284177487 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a9c004470 con 0x561ece113f30 2024-09-19T05:10:22.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.150+0000 7f7aa2000640 1 -- 172.21.15.43:0/3284177487 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7aac028420 con 0x561ece113f30 2024-09-19T05:10:22.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.150+0000 7f7abe04b740 10 client.26312 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:10:22.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.150+0000 7f7abe04b740 10 client.26312 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:10:22.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.150+0000 7f7abe04b740 20 client.26312 populate_metadata read hostname 'smithi043' 2024-09-19T05:10:22.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.150+0000 7f7abe04b740 1 --2- 172.21.15.43:0/3284177487 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x561ece18edd0 0x561ece1911f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:22.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.150+0000 7f7abe04b740 4 client.26312 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:10:22.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.150+0000 7f7abe04b740 1 -- 172.21.15.43:0/3284177487 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561ecdff30c0 con 0x561ece18edd0 2024-09-19T05:10:22.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.150+0000 7f7ab8c00640 1 --2- 172.21.15.43:0/3284177487 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x561ece18edd0 0x561ece1911f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:22.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.151+0000 7f7ab8c00640 1 --2- 172.21.15.43:0/3284177487 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x561ece18edd0 0x561ece1911f0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1ea411fcd0dfd88 server_cookie=a8daf3c95a0d0d7e in_seq=0 out_seq=0 2024-09-19T05:10:22.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.151+0000 7f7aa2000640 10 client.26312 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:10:22.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.151+0000 7f7aa2000640 1 -- 172.21.15.43:0/3284177487 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561ecdff30c0 con 0x561ece18edd0 2024-09-19T05:10:22.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.151+0000 7f7aa2000640 10 client.26312 handle_command_reply: tid=0 2024-09-19T05:10:22.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.151+0000 7f7abe04b740 1 client.26312 shutdown 2024-09-19T05:10:22.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.151+0000 7f7abe04b740 1 -- 172.21.15.43:0/3284177487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ece113f30 msgr2=0x561ece117e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:22.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.151+0000 7f7abe04b740 1 --2- 172.21.15.43:0/3284177487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ece113f30 0x561ece117e40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7aac004040 tx=0x7f7aac004a40 comp rx=0 tx=0).stop 2024-09-19T05:10:22.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.151+0000 7f7abe04b740 1 -- 172.21.15.43:0/3284177487 shutdown_connections 2024-09-19T05:10:22.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.151+0000 7f7abe04b740 1 --2- 172.21.15.43:0/3284177487 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x561ece18edd0 0x561ece1911f0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:22.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.151+0000 7f7abe04b740 1 --2- 172.21.15.43:0/3284177487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ece113f30 0x561ece117e40 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:22.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.151+0000 7f7abe04b740 1 --2- 172.21.15.43:0/3284177487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ece109670 0x561ece117900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:22.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.151+0000 7f7abe04b740 1 --2- 172.21.15.43:0/3284177487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ece108d50 0x561ece11c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:22.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.151+0000 7f7aa2000640 0 client.26312 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:10:22.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.151+0000 7f7abe04b740 1 -- 172.21.15.43:0/3284177487 >> 172.21.15.43:0/3284177487 conn(0x561ece1052e0 msgr2=0x561ece10a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:22.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.151+0000 7f7abe04b740 1 -- 172.21.15.43:0/3284177487 shutdown_connections 2024-09-19T05:10:22.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.151+0000 7f7abe04b740 1 -- 172.21.15.43:0/3284177487 wait complete. 2024-09-19T05:10:22.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.151+0000 7f7abe04b740 20 client.26312 trim_cache size 0 max 16384 2024-09-19T05:10:22.156 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:10:22.157 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-09-19T05:10:22.157 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-09-19T05:10:22.157 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:10:22.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.152+0000 7f7aba000640 1 -- 172.21.15.43:0/3035716630 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f7a980a3430 msgr2=0x7f7a980a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:22.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.152+0000 7f7aba000640 1 --2- 172.21.15.43:0/3035716630 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f7a980a3430 0x7f7a980a58f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7aac011630 tx=0x7f7aac009500 comp rx=0 tx=0).stop 2024-09-19T05:10:22.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.152+0000 7f7aba000640 1 -- 172.21.15.43:0/3035716630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ab41019d0 msgr2=0x7f7ab40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:22.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.152+0000 7f7aba000640 1 --2- 172.21.15.43:0/3035716630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ab41019d0 0x7f7ab40744f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4009230 tx=0x7f7aa4004040 comp rx=0 tx=0).stop 2024-09-19T05:10:22.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.152+0000 7f7aba000640 1 -- 172.21.15.43:0/3035716630 shutdown_connections 2024-09-19T05:10:22.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.152+0000 7f7aba000640 1 --2- 172.21.15.43:0/3035716630 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ab41079d0 0x7f7ab4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:22.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.152+0000 7f7aba000640 1 --2- 172.21.15.43:0/3035716630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ab4102370 0x7f7ab4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:22.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.152+0000 7f7aba000640 1 --2- 172.21.15.43:0/3035716630 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f7a980a3430 0x7f7a980a58f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:22.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.152+0000 7f7aba000640 1 --2- 172.21.15.43:0/3035716630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ab41019d0 0x7f7ab40744f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:22.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.152+0000 7f7aba000640 1 -- 172.21.15.43:0/3035716630 >> 172.21.15.43:0/3035716630 conn(0x7f7ab40fd710 msgr2=0x7f7ab40fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:22.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.152+0000 7f7aba000640 1 -- 172.21.15.43:0/3035716630 shutdown_connections 2024-09-19T05:10:22.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:22.152+0000 7f7aba000640 1 -- 172.21.15.43:0/3035716630 wait complete. 2024-09-19T05:10:22.159 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:21 smithi151 ceph-mon[35425]: pgmap v174: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-19T05:10:22.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:21 smithi096 ceph-mon[35085]: pgmap v174: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-19T05:10:23.030 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T05:10:23.031 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:10:23.413 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:10:23.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:23.940+0000 7f5948000640 1 -- 172.21.15.43:0/3984256762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5940074620 msgr2=0x7f5940074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:23.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:23.940+0000 7f5948000640 1 --2- 172.21.15.43:0/3984256762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5940074620 0x7f5940074a00 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5934009900 tx=0x7f593402a480 comp rx=0 tx=0).stop 2024-09-19T05:10:23.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:23.940+0000 7f5948000640 1 -- 172.21.15.43:0/3984256762 shutdown_connections 2024-09-19T05:10:23.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:23.940+0000 7f5948000640 1 --2- 172.21.15.43:0/3984256762 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f594010e3c0 0x7f5940110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:23.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:23.940+0000 7f5948000640 1 --2- 172.21.15.43:0/3984256762 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5940074f40 0x7f594010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:23.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:23.940+0000 7f5948000640 1 --2- 172.21.15.43:0/3984256762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5940074620 0x7f5940074a00 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:23.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:23.940+0000 7f5948000640 1 -- 172.21.15.43:0/3984256762 >> 172.21.15.43:0/3984256762 conn(0x7f59400fd8c0 msgr2=0x7f59400ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:23.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:23.940+0000 7f5948000640 1 -- 172.21.15.43:0/3984256762 shutdown_connections 2024-09-19T05:10:23.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:23.940+0000 7f5948000640 1 -- 172.21.15.43:0/3984256762 wait complete. 2024-09-19T05:10:23.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:23.941+0000 7f5948000640 1 Processor -- start 2024-09-19T05:10:23.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:23.941+0000 7f5948000640 1 -- start start 2024-09-19T05:10:23.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:23.941+0000 7f5948000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5940074620 0x7f594019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:23.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:23.941+0000 7f5948000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5940074f40 0x7f594019ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:23.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:23.941+0000 7f5948000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f594010e3c0 0x7f594019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:23.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:23.941+0000 7f5948000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5940114150 con 0x7f5940074620 2024-09-19T05:10:23.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:23.941+0000 7f5948000640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5940113fd0 con 0x7f5940074f40 2024-09-19T05:10:23.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:23.941+0000 7f5948000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f59401142d0 con 0x7f594010e3c0 2024-09-19T05:10:23.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:23.942+0000 7f5945600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5940074620 0x7f594019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:23.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:23.942+0000 7f5945600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5940074620 0x7f594019a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43698/0 (socket says 172.21.15.43:43698) 2024-09-19T05:10:23.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:23.942+0000 7f5945600640 1 -- 172.21.15.43:0/2296760745 learned_addr learned my addr 172.21.15.43:0/2296760745 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:23.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:23.942+0000 7f5944c00640 1 --2- 172.21.15.43:0/2296760745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5940074f40 0x7f594019ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:23.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:23.942+0000 7f5946000640 1 --2- 172.21.15.43:0/2296760745 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f594010e3c0 0x7f594019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:23.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:23.942+0000 7f5945600640 1 -- 172.21.15.43:0/2296760745 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f594010e3c0 msgr2=0x7f594019efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:23.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:23.942+0000 7f5945600640 1 --2- 172.21.15.43:0/2296760745 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f594010e3c0 0x7f594019efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:23.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:23.942+0000 7f5945600640 1 -- 172.21.15.43:0/2296760745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5940074f40 msgr2=0x7f594019ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:23.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:23.942+0000 7f5945600640 1 --2- 172.21.15.43:0/2296760745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5940074f40 0x7f594019ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:23.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:23.942+0000 7f5945600640 1 -- 172.21.15.43:0/2296760745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f594019f6c0 con 0x7f5940074620 2024-09-19T05:10:23.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:23.943+0000 7f5945600640 1 --2- 172.21.15.43:0/2296760745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5940074620 0x7f594019a710 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5934006010 tx=0x7f5934004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:23.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:23.943+0000 7f593e000640 1 -- 172.21.15.43:0/2296760745 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593403f030 con 0x7f5940074620 2024-09-19T05:10:23.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:23.943+0000 7f5948000640 1 -- 172.21.15.43:0/2296760745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f594019f950 con 0x7f5940074620 2024-09-19T05:10:23.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:23.943+0000 7f593e000640 1 -- 172.21.15.43:0/2296760745 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5934004440 con 0x7f5940074620 2024-09-19T05:10:23.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:23.943+0000 7f593e000640 1 -- 172.21.15.43:0/2296760745 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5934035070 con 0x7f5940074620 2024-09-19T05:10:23.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:23.943+0000 7f5948000640 1 -- 172.21.15.43:0/2296760745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59401a7290 con 0x7f5940074620 2024-09-19T05:10:23.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:23.944+0000 7f5948000640 1 -- 172.21.15.43:0/2296760745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5940075cd0 con 0x7f5940074620 2024-09-19T05:10:23.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:23.946+0000 7f593e000640 1 -- 172.21.15.43:0/2296760745 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f5934042030 con 0x7f5940074620 2024-09-19T05:10:23.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:23.947+0000 7f593e000640 1 --2- 172.21.15.43:0/2296760745 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f59280a33b0 0x7f59280a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:23.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:23.947+0000 7f593e000640 1 -- 172.21.15.43:0/2296760745 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f59340f00b0 con 0x7f5940074620 2024-09-19T05:10:23.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:23.948+0000 7f5944c00640 1 --2- 172.21.15.43:0/2296760745 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f59280a33b0 0x7f59280a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:23.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:23.948+0000 7f5944c00640 1 --2- 172.21.15.43:0/2296760745 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f59280a33b0 0x7f59280a5870 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f594010f8b0 tx=0x7f592c005f90 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:23.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:23.952+0000 7f593e000640 1 -- 172.21.15.43:0/2296760745 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f59340bc3f0 con 0x7f5940074620 2024-09-19T05:10:24.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:23 smithi151 ceph-mon[35425]: pgmap v175: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:10:24.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:24.173+0000 7f5948000640 1 -- 172.21.15.43:0/2296760745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5940109590 con 0x7f5940074620 2024-09-19T05:10:24.175 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:23 smithi043 ceph-mon[30800]: pgmap v175: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:10:24.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:24.174+0000 7f593e000640 1 -- 172.21.15.43:0/2296760745 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11965 (secure 0 0 0) 0x7f5940109590 con 0x7f5940074620 2024-09-19T05:10:24.176 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:10:24.176 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":29,"btime":"2024-09-19T05:04:38:785407+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/773801169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":773801169},{"type":"v1","addr":"172.21.15.43:6835","nonce":773801169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1799137902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1799137902},{"type":"v1","addr":"172.21.15.43:6837","nonce":1799137902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:02:25.463041+0000","modified":"2024-09-19T05:04:38.039809+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24518,"mds_2":24511,"mds_3":24506,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14541":{"gid":14541,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/794881135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":794881135},{"type":"v1","addr":"172.21.15.43:6841","nonce":794881135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/2836453158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2836453158},{"type":"v1","addr":"172.21.15.151:6833","nonce":2836453158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/178785812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":178785812},{"type":"v1","addr":"172.21.15.96:6833","nonce":178785812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1671043820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1671043820},{"type":"v1","addr":"172.21.15.96:6835","nonce":1671043820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2972514328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2972514328},{"type":"v1","addr":"172.21.15.151:6835","nonce":2972514328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.151:6837/3960378010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3960378010},{"type":"v1","addr":"172.21.15.151:6837","nonce":3960378010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6837/3978388286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3978388286},{"type":"v1","addr":"172.21.15.96:6837","nonce":3978388286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.151:6839/3794104313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3794104313},{"type":"v1","addr":"172.21.15.151:6839","nonce":3794104313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.43:6839/3404555030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3404555030},{"type":"v1","addr":"172.21.15.43:6839","nonce":3404555030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/605527621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":605527621},{"type":"v1","addr":"172.21.15.96:6839","nonce":605527621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24518,24511,24506,24493]},"id":1}]} 2024-09-19T05:10:24.176 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 29 2024-09-19T05:10:24.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:24.177+0000 7f5948000640 1 -- 172.21.15.43:0/2296760745 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f59280a33b0 msgr2=0x7f59280a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:24.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:24.177+0000 7f5948000640 1 --2- 172.21.15.43:0/2296760745 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f59280a33b0 0x7f59280a5870 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f594010f8b0 tx=0x7f592c005f90 comp rx=0 tx=0).stop 2024-09-19T05:10:24.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:24.177+0000 7f5948000640 1 -- 172.21.15.43:0/2296760745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5940074620 msgr2=0x7f594019a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:24.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:24.177+0000 7f5948000640 1 --2- 172.21.15.43:0/2296760745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5940074620 0x7f594019a710 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5934006010 tx=0x7f5934004040 comp rx=0 tx=0).stop 2024-09-19T05:10:24.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:24.177+0000 7f5948000640 1 -- 172.21.15.43:0/2296760745 shutdown_connections 2024-09-19T05:10:24.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:24.177+0000 7f5948000640 1 --2- 172.21.15.43:0/2296760745 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f594010e3c0 0x7f594019efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:24.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:24.177+0000 7f5948000640 1 --2- 172.21.15.43:0/2296760745 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f59280a33b0 0x7f59280a5870 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:24.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:24.177+0000 7f5948000640 1 --2- 172.21.15.43:0/2296760745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5940074f40 0x7f594019ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:24.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:24.177+0000 7f5948000640 1 --2- 172.21.15.43:0/2296760745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5940074620 0x7f594019a710 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:24.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:24.177+0000 7f5948000640 1 -- 172.21.15.43:0/2296760745 >> 172.21.15.43:0/2296760745 conn(0x7f59400fd8c0 msgr2=0x7f59400fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:24.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:24.178+0000 7f5948000640 1 -- 172.21.15.43:0/2296760745 shutdown_connections 2024-09-19T05:10:24.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:24.178+0000 7f5948000640 1 -- 172.21.15.43:0/2296760745 wait complete. 2024-09-19T05:10:24.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:23 smithi096 ceph-mon[35085]: pgmap v175: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:10:25.051 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T05:10:25.101 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:24 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/2296760745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:10:25.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:24 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/2296760745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:10:25.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:24 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/2296760745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:10:25.440 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:10:26.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.062+0000 7f0c39e00640 1 -- 172.21.15.43:0/3644650584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c34077040 msgr2=0x7f0c341124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:26.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.062+0000 7f0c39e00640 1 --2- 172.21.15.43:0/3644650584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c34077040 0x7f0c341124f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c009900 tx=0x7f0c1c02a410 comp rx=0 tx=0).stop 2024-09-19T05:10:26.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.063+0000 7f0c39e00640 1 -- 172.21.15.43:0/3644650584 shutdown_connections 2024-09-19T05:10:26.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.063+0000 7f0c39e00640 1 --2- 172.21.15.43:0/3644650584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c34112a30 0x7f0c34114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:26.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.063+0000 7f0c39e00640 1 --2- 172.21.15.43:0/3644650584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c34077040 0x7f0c341124f0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:26.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.063+0000 7f0c39e00640 1 --2- 172.21.15.43:0/3644650584 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c34076720 0x7f0c34076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:26.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.063+0000 7f0c39e00640 1 -- 172.21.15.43:0/3644650584 >> 172.21.15.43:0/3644650584 conn(0x7f0c340ffa90 msgr2=0x7f0c34101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:26.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.063+0000 7f0c39e00640 1 -- 172.21.15.43:0/3644650584 shutdown_connections 2024-09-19T05:10:26.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.063+0000 7f0c39e00640 1 -- 172.21.15.43:0/3644650584 wait complete. 2024-09-19T05:10:26.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.064+0000 7f0c39e00640 1 Processor -- start 2024-09-19T05:10:26.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.064+0000 7f0c39e00640 1 -- start start 2024-09-19T05:10:26.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.064+0000 7f0c39e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c34076720 0x7f0c341a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:26.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.064+0000 7f0c39e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c34077040 0x7f0c341a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:26.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.064+0000 7f0c39e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c34112a30 0x7f0c3419e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:26.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.064+0000 7f0c39e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0c341185c0 con 0x7f0c34077040 2024-09-19T05:10:26.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.064+0000 7f0c39e00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0c34118440 con 0x7f0c34112a30 2024-09-19T05:10:26.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.064+0000 7f0c39e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0c34118740 con 0x7f0c34076720 2024-09-19T05:10:26.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.064+0000 7f0c2a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c34077040 0x7f0c341a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:26.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.064+0000 7f0c2a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c34077040 0x7f0c341a4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35996/0 (socket says 172.21.15.43:35996) 2024-09-19T05:10:26.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.064+0000 7f0c2a000640 1 -- 172.21.15.43:0/1098626606 learned_addr learned my addr 172.21.15.43:0/1098626606 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:26.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.065+0000 7f0c32a00640 1 --2- 172.21.15.43:0/1098626606 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c34076720 0x7f0c341a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:26.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.065+0000 7f0c33400640 1 --2- 172.21.15.43:0/1098626606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c34112a30 0x7f0c3419e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:26.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.065+0000 7f0c2a000640 1 -- 172.21.15.43:0/1098626606 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c34076720 msgr2=0x7f0c341a4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:26.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.065+0000 7f0c2a000640 1 --2- 172.21.15.43:0/1098626606 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c34076720 0x7f0c341a4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:26.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.065+0000 7f0c2a000640 1 -- 172.21.15.43:0/1098626606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c34112a30 msgr2=0x7f0c3419e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:26.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.065+0000 7f0c2a000640 1 --2- 172.21.15.43:0/1098626606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c34112a30 0x7f0c3419e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:26.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.065+0000 7f0c2a000640 1 -- 172.21.15.43:0/1098626606 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c3419f100 con 0x7f0c34077040 2024-09-19T05:10:26.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.065+0000 7f0c32a00640 1 --2- 172.21.15.43:0/1098626606 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c34076720 0x7f0c341a4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:10:26.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.065+0000 7f0c2a000640 1 --2- 172.21.15.43:0/1098626606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c34077040 0x7f0c341a4d50 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c006010 tx=0x7f0c1c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:26.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.066+0000 7f0c38a00640 1 -- 172.21.15.43:0/1098626606 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c03f030 con 0x7f0c34077040 2024-09-19T05:10:26.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.066+0000 7f0c38a00640 1 -- 172.21.15.43:0/1098626606 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c1c003ce0 con 0x7f0c34077040 2024-09-19T05:10:26.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.066+0000 7f0c39e00640 1 -- 172.21.15.43:0/1098626606 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c3419f390 con 0x7f0c34077040 2024-09-19T05:10:26.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.066+0000 7f0c38a00640 1 -- 172.21.15.43:0/1098626606 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c035070 con 0x7f0c34077040 2024-09-19T05:10:26.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.066+0000 7f0c39e00640 1 -- 172.21.15.43:0/1098626606 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c341ab6e0 con 0x7f0c34077040 2024-09-19T05:10:26.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.070+0000 7f0c3c8ae740 1 Processor -- start 2024-09-19T05:10:26.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.070+0000 7f0c3c8ae740 1 -- start start 2024-09-19T05:10:26.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.070+0000 7f0c38a00640 1 -- 172.21.15.43:0/1098626606 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0c1c042030 con 0x7f0c34077040 2024-09-19T05:10:26.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.071+0000 7f0c3c8ae740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56063d2b6c70 0x56063d2b7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:26.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.071+0000 7f0c3c8ae740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56063d2b7590 0x56063d2c1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:26.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.071+0000 7f0c3c8ae740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56063d2c1e50 0x56063d2c4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:26.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.071+0000 7f0c3c8ae740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56063d21d3e0 con 0x56063d2b6c70 2024-09-19T05:10:26.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.071+0000 7f0c3c8ae740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56063d1c4d80 con 0x56063d2c1e50 2024-09-19T05:10:26.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.071+0000 7f0c3c8ae740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56063d2193a0 con 0x56063d2b7590 2024-09-19T05:10:26.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.071+0000 7f0c32a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56063d2b7590 0x56063d2c1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:26.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.071+0000 7f0c33400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56063d2b6c70 0x56063d2b7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:26.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.071+0000 7f0c32a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56063d2b7590 0x56063d2c1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:41726/0 (socket says 172.21.15.43:41726) 2024-09-19T05:10:26.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.071+0000 7f0c32a00640 1 -- 172.21.15.43:0/696783642 learned_addr learned my addr 172.21.15.43:0/696783642 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:26.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.071+0000 7f0c2a000640 1 --2- 172.21.15.43:0/696783642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56063d2c1e50 0x56063d2c4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:26.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.071+0000 7f0c32a00640 1 -- 172.21.15.43:0/696783642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56063d2c1e50 msgr2=0x56063d2c4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:26.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.071+0000 7f0c32a00640 1 --2- 172.21.15.43:0/696783642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56063d2c1e50 0x56063d2c4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:26.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.071+0000 7f0c32a00640 1 -- 172.21.15.43:0/696783642 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56063d2b6c70 msgr2=0x56063d2b7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:26.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.071+0000 7f0c32a00640 1 --2- 172.21.15.43:0/696783642 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56063d2b6c70 0x56063d2b7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:26.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.071+0000 7f0c32a00640 1 -- 172.21.15.43:0/696783642 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56063d1e3ae0 con 0x56063d2b7590 2024-09-19T05:10:26.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.072+0000 7f0c2a000640 1 --2- 172.21.15.43:0/696783642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56063d2c1e50 0x56063d2c4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:10:26.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.072+0000 7f0c32a00640 1 --2- 172.21.15.43:0/696783642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56063d2b7590 0x56063d2c1910 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0c240114f0 tx=0x7f0c240119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98f513d7a001ace1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:26.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.072+0000 7f0c23e00640 1 -- 172.21.15.43:0/696783642 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c24004020 con 0x56063d2b7590 2024-09-19T05:10:26.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.072+0000 7f0c23e00640 1 -- 172.21.15.43:0/696783642 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c240041c0 con 0x56063d2b7590 2024-09-19T05:10:26.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.072+0000 7f0c23e00640 1 -- 172.21.15.43:0/696783642 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2400f940 con 0x56063d2b7590 2024-09-19T05:10:26.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.072+0000 7f0c38a00640 1 --2- 172.21.15.43:0/1098626606 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f0c180a33b0 0x7f0c180a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:26.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.073+0000 7f0c38a00640 1 -- 172.21.15.43:0/1098626606 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f0c1c0efd70 con 0x7f0c34077040 2024-09-19T05:10:26.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.073+0000 7f0c33400640 1 --2- 172.21.15.43:0/1098626606 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f0c180a33b0 0x7f0c180a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:26.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.073+0000 7f0c3c8ae740 1 -- 172.21.15.43:0/696783642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56063d2b7590 msgr2=0x56063d2c1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:26.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.073+0000 7f0c3c8ae740 1 --2- 172.21.15.43:0/696783642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56063d2b7590 0x56063d2c1910 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0c240114f0 tx=0x7f0c240119b0 comp rx=0 tx=0).stop 2024-09-19T05:10:26.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.073+0000 7f0c3c8ae740 1 -- 172.21.15.43:0/696783642 shutdown_connections 2024-09-19T05:10:26.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.073+0000 7f0c3c8ae740 1 --2- 172.21.15.43:0/696783642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56063d2c1e50 0x56063d2c4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:26.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.073+0000 7f0c3c8ae740 1 --2- 172.21.15.43:0/696783642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56063d2b7590 0x56063d2c1910 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:26.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.073+0000 7f0c3c8ae740 1 --2- 172.21.15.43:0/696783642 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56063d2b6c70 0x56063d2b7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:26.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.073+0000 7f0c3c8ae740 1 -- 172.21.15.43:0/696783642 >> 172.21.15.43:0/696783642 conn(0x56063d2b3200 msgr2=0x56063d2b35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:26.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.073+0000 7f0c33400640 1 --2- 172.21.15.43:0/1098626606 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f0c180a33b0 0x7f0c180a5870 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0c3406b460 tx=0x7f0c2c0096a0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:26.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.073+0000 7f0c3c8ae740 1 -- 172.21.15.43:0/696783642 shutdown_connections 2024-09-19T05:10:26.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.073+0000 7f0c3c8ae740 1 -- 172.21.15.43:0/696783642 wait complete. 2024-09-19T05:10:26.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.074+0000 7f0c3c8ae740 1 -- start start 2024-09-19T05:10:26.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.074+0000 7f0c3c8ae740 1 Processor -- start 2024-09-19T05:10:26.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.075+0000 7f0c3c8ae740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56063d2b6c70 0x56063d2bec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:26.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.075+0000 7f0c3c8ae740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56063d2b7590 0x56063d2bf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:26.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.075+0000 7f0c3c8ae740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56063d2c1e50 0x56063d2bf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:26.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.075+0000 7f0c3c8ae740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56063d2c4a20 con 0x56063d2c1e50 2024-09-19T05:10:26.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.075+0000 7f0c3c8ae740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56063d1d80f0 con 0x56063d2b6c70 2024-09-19T05:10:26.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.075+0000 7f0c3c8ae740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56063d2c4ba0 con 0x56063d2b7590 2024-09-19T05:10:26.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.075+0000 7f0c3c8ae740 10 client.0 _ll_register_callbacks cb 0x56063d1bfbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:26.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.075+0000 7f0c33400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56063d2c1e50 0x56063d2bf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:26.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.075+0000 7f0c2a000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56063d2b7590 0x56063d2bf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:26.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.075+0000 7f0c32a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56063d2b6c70 0x56063d2bec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:26.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.075+0000 7f0c33400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56063d2c1e50 0x56063d2bf6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36014/0 (socket says 172.21.15.43:36014) 2024-09-19T05:10:26.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.075+0000 7f0c33400640 1 -- 172.21.15.43:0/3639249384 learned_addr learned my addr 172.21.15.43:0/3639249384 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:26.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.075+0000 7f0c32a00640 1 -- 172.21.15.43:0/3639249384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56063d2b7590 msgr2=0x56063d2bf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:26.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.075+0000 7f0c32a00640 1 --2- 172.21.15.43:0/3639249384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56063d2b7590 0x56063d2bf170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:26.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.075+0000 7f0c32a00640 1 -- 172.21.15.43:0/3639249384 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56063d2c1e50 msgr2=0x56063d2bf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:26.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.075+0000 7f0c32a00640 1 --2- 172.21.15.43:0/3639249384 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56063d2c1e50 0x56063d2bf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:26.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.075+0000 7f0c32a00640 1 -- 172.21.15.43:0/3639249384 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56063cf8e3c0 con 0x56063d2b6c70 2024-09-19T05:10:26.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.076+0000 7f0c32a00640 1 --2- 172.21.15.43:0/3639249384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56063d2b6c70 0x56063d2bec30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0c24011eb0 tx=0x7f0c24009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d033bc0ea8357ca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:26.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.076+0000 7f0c20c00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:10:26.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.076+0000 7f0c20c00640 1 -- 172.21.15.43:0/3639249384 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c24028020 con 0x56063d2b6c70 2024-09-19T05:10:26.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.076+0000 7f0c3c8ae740 1 -- 172.21.15.43:0/3639249384 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c240041c0 con 0x56063d2b6c70 2024-09-19T05:10:26.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.076+0000 7f0c20c00640 1 -- 172.21.15.43:0/3639249384 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f0c24016040 con 0x56063d2b6c70 2024-09-19T05:10:26.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.076+0000 7f0c20c00640 1 -- 172.21.15.43:0/3639249384 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0c24033050 con 0x56063d2b6c70 2024-09-19T05:10:26.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.076+0000 7f0c3c8ae740 10 client.26330 fetch_fsmap learned FSMap version 29 2024-09-19T05:10:26.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.076+0000 7f0c3c8ae740 1 -- 172.21.15.43:0/3639249384 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56063d2d0990 con 0x56063d2b6c70 2024-09-19T05:10:26.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.077+0000 7f0c20c00640 1 -- 172.21.15.43:0/3639249384 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c240041c0 con 0x56063d2b6c70 2024-09-19T05:10:26.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.077+0000 7f0c20c00640 1 -- 172.21.15.43:0/3639249384 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0c24004ac0 con 0x56063d2b6c70 2024-09-19T05:10:26.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.077+0000 7f0c3c8ae740 10 client.26330 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:10:26.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.077+0000 7f0c3c8ae740 10 client.26330 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-19T05:10:26.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.077+0000 7f0c3c8ae740 20 client.26330 populate_metadata read hostname 'smithi043' 2024-09-19T05:10:26.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.077+0000 7f0c3c8ae740 1 --2- 172.21.15.43:0/3639249384 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x56063d2d1d40 0x56063d2d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:26.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.077+0000 7f0c3c8ae740 4 client.26330 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:10:26.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.077+0000 7f0c3c8ae740 1 -- 172.21.15.43:0/3639249384 --> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56063d22a950 con 0x56063d2d1d40 2024-09-19T05:10:26.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.078+0000 7f0c2a000640 1 --2- 172.21.15.43:0/3639249384 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x56063d2d1d40 0x56063d2d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:26.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.078+0000 7f0c2a000640 1 --2- 172.21.15.43:0/3639249384 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x56063d2d1d40 0x56063d2d4130 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d7eb9efc7c7a9875 server_cookie=51ba76278a5db859 in_seq=0 out_seq=0 2024-09-19T05:10:26.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.078+0000 7f0c20c00640 10 client.26330 ms_handle_connect on v2:172.21.15.151:6836/3960378010 2024-09-19T05:10:26.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.079+0000 7f0c20c00640 1 -- 172.21.15.43:0/3639249384 <== mds.4 v2:172.21.15.151:6836/3960378010 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56063d22a950 con 0x56063d2d1d40 2024-09-19T05:10:26.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.079+0000 7f0c20c00640 10 client.26330 handle_command_reply: tid=0 2024-09-19T05:10:26.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.079+0000 7f0c3c8ae740 1 client.26330 shutdown 2024-09-19T05:10:26.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.079+0000 7f0c3c8ae740 1 -- 172.21.15.43:0/3639249384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56063d2b6c70 msgr2=0x56063d2bec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:26.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.079+0000 7f0c3c8ae740 1 --2- 172.21.15.43:0/3639249384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56063d2b6c70 0x56063d2bec30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0c24011eb0 tx=0x7f0c24009ca0 comp rx=0 tx=0).stop 2024-09-19T05:10:26.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.079+0000 7f0c3c8ae740 1 -- 172.21.15.43:0/3639249384 shutdown_connections 2024-09-19T05:10:26.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.079+0000 7f0c3c8ae740 1 --2- 172.21.15.43:0/3639249384 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x56063d2d1d40 0x56063d2d4130 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:26.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.079+0000 7f0c3c8ae740 1 --2- 172.21.15.43:0/3639249384 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56063d2c1e50 0x56063d2bf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:26.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.079+0000 7f0c3c8ae740 1 --2- 172.21.15.43:0/3639249384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56063d2b7590 0x56063d2bf170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:26.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.079+0000 7f0c3c8ae740 1 --2- 172.21.15.43:0/3639249384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56063d2b6c70 0x56063d2bec30 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:26.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.079+0000 7f0c20c00640 0 client.26330 ms_handle_reset on v2:172.21.15.151:6836/3960378010 2024-09-19T05:10:26.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.079+0000 7f0c3c8ae740 1 -- 172.21.15.43:0/3639249384 >> 172.21.15.43:0/3639249384 conn(0x56063d2b3200 msgr2=0x56063d2b8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:26.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.079+0000 7f0c3c8ae740 1 -- 172.21.15.43:0/3639249384 shutdown_connections 2024-09-19T05:10:26.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.079+0000 7f0c3c8ae740 1 -- 172.21.15.43:0/3639249384 wait complete. 2024-09-19T05:10:26.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.080+0000 7f0c3c8ae740 20 client.26330 trim_cache size 0 max 16384 2024-09-19T05:10:26.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.093+0000 7f0c3c8ae740 1 Processor -- start 2024-09-19T05:10:26.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.093+0000 7f0c3c8ae740 1 -- start start 2024-09-19T05:10:26.095 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:25 smithi043 ceph-mon[30800]: pgmap v176: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:10:26.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.094+0000 7f0c3c8ae740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56063d2b6c70 0x56063d2d98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:26.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.094+0000 7f0c3c8ae740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56063d2b7590 0x56063d2d9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:26.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.094+0000 7f0c3c8ae740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56063d2c1e50 0x56063d2c0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:26.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.094+0000 7f0c3c8ae740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0c341b3ea0 con 0x56063d2b6c70 2024-09-19T05:10:26.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.094+0000 7f0c3c8ae740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0c341b59e0 con 0x56063d2c1e50 2024-09-19T05:10:26.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.094+0000 7f0c3c8ae740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0c341b4570 con 0x56063d2b7590 2024-09-19T05:10:26.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.094+0000 7f0c32a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56063d2b6c70 0x56063d2d98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:26.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.094+0000 7f0c32a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56063d2b6c70 0x56063d2d98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36016/0 (socket says 172.21.15.43:36016) 2024-09-19T05:10:26.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.094+0000 7f0c32a00640 1 -- 172.21.15.43:0/120600420 learned_addr learned my addr 172.21.15.43:0/120600420 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:26.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.094+0000 7f0c2a000640 1 --2- 172.21.15.43:0/120600420 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56063d2b7590 0x56063d2d9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:26.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.094+0000 7f0c33400640 1 --2- 172.21.15.43:0/120600420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56063d2c1e50 0x56063d2c0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:26.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.094+0000 7f0c2a000640 1 -- 172.21.15.43:0/120600420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56063d2c1e50 msgr2=0x56063d2c0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:26.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.094+0000 7f0c2a000640 1 --2- 172.21.15.43:0/120600420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56063d2c1e50 0x56063d2c0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:26.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.095+0000 7f0c2a000640 1 -- 172.21.15.43:0/120600420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56063d2b6c70 msgr2=0x56063d2d98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:26.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.095+0000 7f0c2a000640 1 --2- 172.21.15.43:0/120600420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56063d2b6c70 0x56063d2d98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:26.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.095+0000 7f0c2a000640 1 -- 172.21.15.43:0/120600420 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56063d2d46c0 con 0x56063d2b7590 2024-09-19T05:10:26.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.095+0000 7f0c33400640 1 --2- 172.21.15.43:0/120600420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56063d2c1e50 0x56063d2c0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:10:26.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.095+0000 7f0c32a00640 1 --2- 172.21.15.43:0/120600420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56063d2b6c70 0x56063d2d98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:10:26.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.095+0000 7f0c2a000640 1 --2- 172.21.15.43:0/120600420 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56063d2b7590 0x56063d2d9e30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c0ed470 tx=0x7f0c1c02a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8cb9b097b336794 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:26.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.095+0000 7f0c23400640 1 -- 172.21.15.43:0/120600420 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c049040 con 0x56063d2b7590 2024-09-19T05:10:26.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.095+0000 7f0c23400640 1 -- 172.21.15.43:0/120600420 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c1c0bc170 con 0x56063d2b7590 2024-09-19T05:10:26.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.095+0000 7f0c23400640 1 -- 172.21.15.43:0/120600420 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c0c0120 con 0x56063d2b7590 2024-09-19T05:10:26.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.096+0000 7f0c3c8ae740 1 -- 172.21.15.43:0/120600420 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56063d2b7590 msgr2=0x56063d2d9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:26.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.096+0000 7f0c3c8ae740 1 --2- 172.21.15.43:0/120600420 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56063d2b7590 0x56063d2d9e30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c0ed470 tx=0x7f0c1c02a410 comp rx=0 tx=0).stop 2024-09-19T05:10:26.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.096+0000 7f0c3c8ae740 1 -- 172.21.15.43:0/120600420 shutdown_connections 2024-09-19T05:10:26.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.096+0000 7f0c3c8ae740 1 --2- 172.21.15.43:0/120600420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56063d2c1e50 0x56063d2c0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:26.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.096+0000 7f0c3c8ae740 1 --2- 172.21.15.43:0/120600420 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56063d2b7590 0x56063d2d9e30 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:26.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.096+0000 7f0c3c8ae740 1 --2- 172.21.15.43:0/120600420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56063d2b6c70 0x56063d2d98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:26.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.096+0000 7f0c3c8ae740 1 -- 172.21.15.43:0/120600420 >> 172.21.15.43:0/120600420 conn(0x56063d2b3200 msgr2=0x56063d2b2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:26.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.096+0000 7f0c3c8ae740 1 -- 172.21.15.43:0/120600420 shutdown_connections 2024-09-19T05:10:26.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.096+0000 7f0c3c8ae740 1 -- 172.21.15.43:0/120600420 wait complete. 2024-09-19T05:10:26.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.096+0000 7f0c3c8ae740 1 -- start start 2024-09-19T05:10:26.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.097+0000 7f0c3c8ae740 1 Processor -- start 2024-09-19T05:10:26.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.097+0000 7f0c3c8ae740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56063d2b6c70 0x56063d2ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:26.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.097+0000 7f0c3c8ae740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56063d2b7590 0x56063d2c2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:26.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.097+0000 7f0c3c8ae740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56063d2c1e50 0x56063d2c3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:26.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.097+0000 7f0c3c8ae740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0c341b5310 con 0x56063d2c1e50 2024-09-19T05:10:26.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.097+0000 7f0c3c8ae740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0c341b4c40 con 0x56063d2b7590 2024-09-19T05:10:26.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.097+0000 7f0c3c8ae740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0c341b1e80 con 0x56063d2b6c70 2024-09-19T05:10:26.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.097+0000 7f0c3c8ae740 10 client.0 _ll_register_callbacks cb 0x7f0c2c009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:26.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.097+0000 7f0c33400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56063d2c1e50 0x56063d2c3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:26.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.097+0000 7f0c32a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56063d2b6c70 0x56063d2ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:26.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.098+0000 7f0c32a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56063d2b6c70 0x56063d2ca660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:41750/0 (socket says 172.21.15.43:41750) 2024-09-19T05:10:26.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.098+0000 7f0c32a00640 1 -- 172.21.15.43:0/929709495 learned_addr learned my addr 172.21.15.43:0/929709495 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:26.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.098+0000 7f0c2a000640 1 --2- 172.21.15.43:0/929709495 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56063d2b7590 0x56063d2c2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:26.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.098+0000 7f0c32a00640 1 -- 172.21.15.43:0/929709495 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56063d2b7590 msgr2=0x56063d2c2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:26.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.098+0000 7f0c32a00640 1 --2- 172.21.15.43:0/929709495 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56063d2b7590 0x56063d2c2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:26.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.098+0000 7f0c32a00640 1 -- 172.21.15.43:0/929709495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56063d2c1e50 msgr2=0x56063d2c3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:26.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.098+0000 7f0c32a00640 1 --2- 172.21.15.43:0/929709495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56063d2c1e50 0x56063d2c3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:26.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.098+0000 7f0c32a00640 1 -- 172.21.15.43:0/929709495 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56063d2ae770 con 0x56063d2b6c70 2024-09-19T05:10:26.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.098+0000 7f0c2a000640 1 --2- 172.21.15.43:0/929709495 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56063d2b7590 0x56063d2c2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:10:26.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.098+0000 7f0c32a00640 1 --2- 172.21.15.43:0/929709495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56063d2b6c70 0x56063d2ca660 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0c2400f6f0 tx=0x7f0c2400f9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2548f3bf1bd1dbd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:26.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.098+0000 7f0c20c00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:10:26.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.099+0000 7f0c20c00640 1 -- 172.21.15.43:0/929709495 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c24028040 con 0x56063d2b6c70 2024-09-19T05:10:26.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.099+0000 7f0c3c8ae740 1 -- 172.21.15.43:0/929709495 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c1c0bc170 con 0x56063d2b6c70 2024-09-19T05:10:26.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.099+0000 7f0c20c00640 1 -- 172.21.15.43:0/929709495 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f0c2400fa90 con 0x56063d2b6c70 2024-09-19T05:10:26.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.099+0000 7f0c20c00640 1 -- 172.21.15.43:0/929709495 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0c24045050 con 0x56063d2b6c70 2024-09-19T05:10:26.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.099+0000 7f0c3c8ae740 10 client.26269 fetch_fsmap learned FSMap version 29 2024-09-19T05:10:26.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.099+0000 7f0c3c8ae740 1 -- 172.21.15.43:0/929709495 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56063d2b7b00 con 0x56063d2b6c70 2024-09-19T05:10:26.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.099+0000 7f0c20c00640 1 -- 172.21.15.43:0/929709495 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c1c0bc170 con 0x56063d2b6c70 2024-09-19T05:10:26.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.100+0000 7f0c20c00640 1 -- 172.21.15.43:0/929709495 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0c24004580 con 0x56063d2b6c70 2024-09-19T05:10:26.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.100+0000 7f0c3c8ae740 10 client.26269 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:10:26.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.100+0000 7f0c3c8ae740 10 client.26269 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-19T05:10:26.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.100+0000 7f0c3c8ae740 20 client.26269 populate_metadata read hostname 'smithi043' 2024-09-19T05:10:26.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.100+0000 7f0c3c8ae740 1 --2- 172.21.15.43:0/929709495 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x56063d33cc50 0x56063d33f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:26.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.100+0000 7f0c3c8ae740 4 client.26269 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:10:26.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.100+0000 7f0c3c8ae740 1 -- 172.21.15.43:0/929709495 --> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56063d224e60 con 0x56063d33cc50 2024-09-19T05:10:26.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.100+0000 7f0c2a000640 1 --2- 172.21.15.43:0/929709495 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x56063d33cc50 0x56063d33f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:26.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.100+0000 7f0c2a000640 1 --2- 172.21.15.43:0/929709495 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x56063d33cc50 0x56063d33f040 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=82613580ee7c89ec server_cookie=53ddaabcac732c9c in_seq=0 out_seq=0 2024-09-19T05:10:26.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.100+0000 7f0c20c00640 10 client.26269 ms_handle_connect on v2:172.21.15.151:6836/3960378010 2024-09-19T05:10:26.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.101+0000 7f0c20c00640 1 -- 172.21.15.43:0/929709495 <== mds.4 v2:172.21.15.151:6836/3960378010 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56063d224e60 con 0x56063d33cc50 2024-09-19T05:10:26.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.101+0000 7f0c20c00640 10 client.26269 handle_command_reply: tid=0 2024-09-19T05:10:26.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.101+0000 7f0c3c8ae740 1 client.26269 shutdown 2024-09-19T05:10:26.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.101+0000 7f0c3c8ae740 1 -- 172.21.15.43:0/929709495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56063d2b6c70 msgr2=0x56063d2ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:26.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.101+0000 7f0c3c8ae740 1 --2- 172.21.15.43:0/929709495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56063d2b6c70 0x56063d2ca660 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0c2400f6f0 tx=0x7f0c2400f9a0 comp rx=0 tx=0).stop 2024-09-19T05:10:26.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.101+0000 7f0c3c8ae740 1 -- 172.21.15.43:0/929709495 shutdown_connections 2024-09-19T05:10:26.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.101+0000 7f0c3c8ae740 1 --2- 172.21.15.43:0/929709495 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x56063d33cc50 0x56063d33f040 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:26.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.101+0000 7f0c3c8ae740 1 --2- 172.21.15.43:0/929709495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56063d2c1e50 0x56063d2c3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:26.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.101+0000 7f0c3c8ae740 1 --2- 172.21.15.43:0/929709495 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56063d2b7590 0x56063d2c2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:26.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.101+0000 7f0c3c8ae740 1 --2- 172.21.15.43:0/929709495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56063d2b6c70 0x56063d2ca660 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:26.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.101+0000 7f0c3c8ae740 1 -- 172.21.15.43:0/929709495 >> 172.21.15.43:0/929709495 conn(0x56063d2b3200 msgr2=0x56063d2c0490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:26.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.101+0000 7f0c20c00640 0 client.26269 ms_handle_reset on v2:172.21.15.151:6836/3960378010 2024-09-19T05:10:26.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.101+0000 7f0c3c8ae740 1 -- 172.21.15.43:0/929709495 shutdown_connections 2024-09-19T05:10:26.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.101+0000 7f0c3c8ae740 1 -- 172.21.15.43:0/929709495 wait complete. 2024-09-19T05:10:26.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.102+0000 7f0c3c8ae740 20 client.26269 trim_cache size 0 max 16384 2024-09-19T05:10:26.108 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:10:26.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.102+0000 7f0c39e00640 1 -- 172.21.15.43:0/1098626606 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f0c180a33b0 msgr2=0x7f0c180a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:26.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.102+0000 7f0c39e00640 1 --2- 172.21.15.43:0/1098626606 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f0c180a33b0 0x7f0c180a5870 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0c3406b460 tx=0x7f0c2c0096a0 comp rx=0 tx=0).stop 2024-09-19T05:10:26.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.102+0000 7f0c39e00640 1 -- 172.21.15.43:0/1098626606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c34077040 msgr2=0x7f0c341a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:26.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.102+0000 7f0c39e00640 1 --2- 172.21.15.43:0/1098626606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c34077040 0x7f0c341a4d50 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c006010 tx=0x7f0c1c0042f0 comp rx=0 tx=0).stop 2024-09-19T05:10:26.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.103+0000 7f0c39e00640 1 -- 172.21.15.43:0/1098626606 shutdown_connections 2024-09-19T05:10:26.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.103+0000 7f0c39e00640 1 --2- 172.21.15.43:0/1098626606 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f0c180a33b0 0x7f0c180a5870 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:26.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.103+0000 7f0c39e00640 1 --2- 172.21.15.43:0/1098626606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c34112a30 0x7f0c3419e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:26.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.103+0000 7f0c39e00640 1 --2- 172.21.15.43:0/1098626606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c34077040 0x7f0c341a4d50 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:26.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.103+0000 7f0c39e00640 1 --2- 172.21.15.43:0/1098626606 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c34076720 0x7f0c341a4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:26.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.103+0000 7f0c39e00640 1 -- 172.21.15.43:0/1098626606 >> 172.21.15.43:0/1098626606 conn(0x7f0c340ffa90 msgr2=0x7f0c34101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:26.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.103+0000 7f0c39e00640 1 -- 172.21.15.43:0/1098626606 shutdown_connections 2024-09-19T05:10:26.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:26.103+0000 7f0c39e00640 1 -- 172.21.15.43:0/1098626606 wait complete. 2024-09-19T05:10:26.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:25 smithi151 ceph-mon[35425]: pgmap v176: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:10:26.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:25 smithi096 ceph-mon[35085]: pgmap v176: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:10:27.010 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T05:10:27.390 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:10:27.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.979+0000 7f2c7e000640 1 -- 172.21.15.43:0/3903497679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c78103ff0 msgr2=0x7f2c781043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.979+0000 7f2c7e000640 1 --2- 172.21.15.43:0/3903497679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c78103ff0 0x7f2c781043d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2c68009900 tx=0x7f2c6802a3f0 comp rx=0 tx=0).stop 2024-09-19T05:10:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.979+0000 7f2c7e000640 1 -- 172.21.15.43:0/3903497679 shutdown_connections 2024-09-19T05:10:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.979+0000 7f2c7e000640 1 --2- 172.21.15.43:0/3903497679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c78109120 0x7f2c78110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.979+0000 7f2c7e000640 1 --2- 172.21.15.43:0/3903497679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c781049a0 0x7f2c781089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.979+0000 7f2c7e000640 1 --2- 172.21.15.43:0/3903497679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c78103ff0 0x7f2c781043d0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.979+0000 7f2c7e000640 1 -- 172.21.15.43:0/3903497679 >> 172.21.15.43:0/3903497679 conn(0x7f2c780ffa90 msgr2=0x7f2c78101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.979+0000 7f2c7e000640 1 -- 172.21.15.43:0/3903497679 shutdown_connections 2024-09-19T05:10:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.979+0000 7f2c7e000640 1 -- 172.21.15.43:0/3903497679 wait complete. 2024-09-19T05:10:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.980+0000 7f2c7e000640 1 Processor -- start 2024-09-19T05:10:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.980+0000 7f2c7e000640 1 -- start start 2024-09-19T05:10:27.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.981+0000 7f2c7e000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c78103ff0 0x7f2c7819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:27.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.981+0000 7f2c7e000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c781049a0 0x7f2c7819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:27.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.981+0000 7f2c7e000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c78109120 0x7f2c7819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:27.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.981+0000 7f2c7e000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2c78114150 con 0x7f2c781049a0 2024-09-19T05:10:27.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.981+0000 7f2c7e000640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2c78113fd0 con 0x7f2c78109120 2024-09-19T05:10:27.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.981+0000 7f2c7e000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2c781142d0 con 0x7f2c78103ff0 2024-09-19T05:10:27.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.981+0000 7f2c76a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c781049a0 0x7f2c7819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:27.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.981+0000 7f2c76a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c781049a0 0x7f2c7819ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36050/0 (socket says 172.21.15.43:36050) 2024-09-19T05:10:27.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.981+0000 7f2c76a00640 1 -- 172.21.15.43:0/2922930382 learned_addr learned my addr 172.21.15.43:0/2922930382 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:27.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.981+0000 7f2c77400640 1 --2- 172.21.15.43:0/2922930382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c78103ff0 0x7f2c7819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:27.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.981+0000 7f2c77e00640 1 --2- 172.21.15.43:0/2922930382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c78109120 0x7f2c7819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:27.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.982+0000 7f2c76a00640 1 -- 172.21.15.43:0/2922930382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c78103ff0 msgr2=0x7f2c7819a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:27.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.982+0000 7f2c76a00640 1 --2- 172.21.15.43:0/2922930382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c78103ff0 0x7f2c7819a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.982+0000 7f2c76a00640 1 -- 172.21.15.43:0/2922930382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c78109120 msgr2=0x7f2c7819eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.982+0000 7f2c76a00640 1 --2- 172.21.15.43:0/2922930382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c78109120 0x7f2c7819eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.982+0000 7f2c76a00640 1 -- 172.21.15.43:0/2922930382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c7819f5a0 con 0x7f2c781049a0 2024-09-19T05:10:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.982+0000 7f2c76a00640 1 --2- 172.21.15.43:0/2922930382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c781049a0 0x7f2c7819ab30 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2c6400e9f0 tx=0x7f2c6400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.982+0000 7f2c6fe00640 1 -- 172.21.15.43:0/2922930382 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6400cd80 con 0x7f2c781049a0 2024-09-19T05:10:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.982+0000 7f2c6fe00640 1 -- 172.21.15.43:0/2922930382 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c64010910 con 0x7f2c781049a0 2024-09-19T05:10:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.982+0000 7f2c6fe00640 1 -- 172.21.15.43:0/2922930382 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c64018990 con 0x7f2c781049a0 2024-09-19T05:10:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.982+0000 7f2c7e000640 1 -- 172.21.15.43:0/2922930382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c7819f890 con 0x7f2c781049a0 2024-09-19T05:10:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.982+0000 7f2c7e000640 1 -- 172.21.15.43:0/2922930382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c781a7170 con 0x7f2c781049a0 2024-09-19T05:10:27.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.985+0000 7f2c80bf4740 1 Processor -- start 2024-09-19T05:10:27.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.986+0000 7f2c80bf4740 1 -- start start 2024-09-19T05:10:27.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.986+0000 7f2c6fe00640 1 -- 172.21.15.43:0/2922930382 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f2c64010ac0 con 0x7f2c781049a0 2024-09-19T05:10:27.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.986+0000 7f2c80bf4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d82c917c70 0x55d82c918050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:27.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.986+0000 7f2c80bf4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d82c918590 0x55d82c922910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:27.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.986+0000 7f2c80bf4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d82c922e50 0x55d82c925240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:27.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.986+0000 7f2c80bf4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d82c87e3e0 con 0x55d82c918590 2024-09-19T05:10:27.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.986+0000 7f2c80bf4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d82c825d80 con 0x55d82c917c70 2024-09-19T05:10:27.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.986+0000 7f2c80bf4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d82c87a3a0 con 0x55d82c922e50 2024-09-19T05:10:27.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.986+0000 7f2c77e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d82c917c70 0x55d82c918050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:27.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.986+0000 7f2c77e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d82c917c70 0x55d82c918050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:51576/0 (socket says 172.21.15.43:51576) 2024-09-19T05:10:27.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.986+0000 7f2c77e00640 1 -- 172.21.15.43:0/2114982125 learned_addr learned my addr 172.21.15.43:0/2114982125 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:27.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.986+0000 7f2c77400640 1 --2- 172.21.15.43:0/2114982125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d82c918590 0x55d82c922910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:27.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.986+0000 7f2c76a00640 1 --2- 172.21.15.43:0/2114982125 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d82c922e50 0x55d82c925240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:27.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.987+0000 7f2c77e00640 1 -- 172.21.15.43:0/2114982125 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d82c922e50 msgr2=0x55d82c925240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:27.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.987+0000 7f2c77e00640 1 --2- 172.21.15.43:0/2114982125 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d82c922e50 0x55d82c925240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:27.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.987+0000 7f2c77e00640 1 -- 172.21.15.43:0/2114982125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d82c918590 msgr2=0x55d82c922910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:27.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.987+0000 7f2c77e00640 1 --2- 172.21.15.43:0/2114982125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d82c918590 0x55d82c922910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:27.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.987+0000 7f2c77e00640 1 -- 172.21.15.43:0/2114982125 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d82c844ae0 con 0x55d82c917c70 2024-09-19T05:10:27.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.987+0000 7f2c77e00640 1 --2- 172.21.15.43:0/2114982125 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d82c917c70 0x55d82c918050 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2c70009f50 tx=0x7f2c700118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f29ac8abd571d0b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:27.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.987+0000 7f2c63e00640 1 -- 172.21.15.43:0/2114982125 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c70004d20 con 0x55d82c917c70 2024-09-19T05:10:27.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.987+0000 7f2c63e00640 1 -- 172.21.15.43:0/2114982125 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c70004050 con 0x55d82c917c70 2024-09-19T05:10:27.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.987+0000 7f2c63e00640 1 -- 172.21.15.43:0/2114982125 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c70005240 con 0x55d82c917c70 2024-09-19T05:10:28.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.988+0000 7f2c6fe00640 1 --2- 172.21.15.43:0/2922930382 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f2c5c0a33b0 0x7f2c5c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:28.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.988+0000 7f2c6fe00640 1 -- 172.21.15.43:0/2922930382 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f2c640d0080 con 0x7f2c781049a0 2024-09-19T05:10:28.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.988+0000 7f2c77400640 1 --2- 172.21.15.43:0/2922930382 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f2c5c0a33b0 0x7f2c5c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:28.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.988+0000 7f2c80bf4740 1 -- 172.21.15.43:0/2114982125 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d82c917c70 msgr2=0x55d82c918050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:28.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.988+0000 7f2c80bf4740 1 --2- 172.21.15.43:0/2114982125 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d82c917c70 0x55d82c918050 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2c70009f50 tx=0x7f2c700118d0 comp rx=0 tx=0).stop 2024-09-19T05:10:28.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.988+0000 7f2c80bf4740 1 -- 172.21.15.43:0/2114982125 shutdown_connections 2024-09-19T05:10:28.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.988+0000 7f2c80bf4740 1 --2- 172.21.15.43:0/2114982125 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d82c922e50 0x55d82c925240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:28.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.988+0000 7f2c80bf4740 1 --2- 172.21.15.43:0/2114982125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d82c918590 0x55d82c922910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:28.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.988+0000 7f2c80bf4740 1 --2- 172.21.15.43:0/2114982125 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d82c917c70 0x55d82c918050 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:28.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.988+0000 7f2c80bf4740 1 -- 172.21.15.43:0/2114982125 >> 172.21.15.43:0/2114982125 conn(0x55d82c914200 msgr2=0x55d82c9145e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:28.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.989+0000 7f2c80bf4740 1 -- 172.21.15.43:0/2114982125 shutdown_connections 2024-09-19T05:10:28.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.989+0000 7f2c80bf4740 1 -- 172.21.15.43:0/2114982125 wait complete. 2024-09-19T05:10:28.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.989+0000 7f2c77400640 1 --2- 172.21.15.43:0/2922930382 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f2c5c0a33b0 0x7f2c5c0a5870 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2c68009230 tx=0x7f2c680058d0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:28.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.989+0000 7f2c80bf4740 1 -- start start 2024-09-19T05:10:28.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.989+0000 7f2c80bf4740 1 Processor -- start 2024-09-19T05:10:28.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.990+0000 7f2c80bf4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d82c917c70 0x55d82c91fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:28.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.990+0000 7f2c80bf4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d82c918590 0x55d82c9201a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:28.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.990+0000 7f2c80bf4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d82c922e50 0x55d82c9206e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:28.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.990+0000 7f2c80bf4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d82c925a20 con 0x55d82c917c70 2024-09-19T05:10:28.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.990+0000 7f2c80bf4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d82c8390f0 con 0x55d82c918590 2024-09-19T05:10:28.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.990+0000 7f2c80bf4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d82c925ba0 con 0x55d82c922e50 2024-09-19T05:10:28.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.990+0000 7f2c80bf4740 10 client.0 _ll_register_callbacks cb 0x55d82c820bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:28.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.990+0000 7f2c77400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d82c917c70 0x55d82c91fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:28.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.990+0000 7f2c77e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d82c922e50 0x55d82c9206e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:28.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.990+0000 7f2c77400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d82c917c70 0x55d82c91fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36068/0 (socket says 172.21.15.43:36068) 2024-09-19T05:10:28.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.990+0000 7f2c77400640 1 -- 172.21.15.43:0/3047035619 learned_addr learned my addr 172.21.15.43:0/3047035619 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:28.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.990+0000 7f2c76a00640 1 --2- 172.21.15.43:0/3047035619 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d82c918590 0x55d82c9201a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:28.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.991+0000 7f2c76a00640 1 -- 172.21.15.43:0/3047035619 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d82c922e50 msgr2=0x55d82c9206e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:28.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.991+0000 7f2c76a00640 1 --2- 172.21.15.43:0/3047035619 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d82c922e50 0x55d82c9206e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:28.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.991+0000 7f2c76a00640 1 -- 172.21.15.43:0/3047035619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d82c917c70 msgr2=0x55d82c91fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:28.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.991+0000 7f2c76a00640 1 --2- 172.21.15.43:0/3047035619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d82c917c70 0x55d82c91fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:28.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.991+0000 7f2c76a00640 1 -- 172.21.15.43:0/3047035619 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d82c5ef3c0 con 0x55d82c918590 2024-09-19T05:10:28.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.991+0000 7f2c77400640 1 --2- 172.21.15.43:0/3047035619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d82c917c70 0x55d82c91fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:10:28.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.991+0000 7f2c76a00640 1 --2- 172.21.15.43:0/3047035619 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d82c918590 0x55d82c9201a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2c640cd780 tx=0x7f2c6409bf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2916e03d155cb614 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:28.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.991+0000 7f2c60c00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:10:28.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.991+0000 7f2c60c00640 1 -- 172.21.15.43:0/3047035619 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c640cdc00 con 0x55d82c918590 2024-09-19T05:10:28.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.991+0000 7f2c60c00640 1 -- 172.21.15.43:0/3047035619 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f2c640bc0e0 con 0x55d82c918590 2024-09-19T05:10:28.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.991+0000 7f2c80bf4740 1 -- 172.21.15.43:0/3047035619 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c70004050 con 0x55d82c918590 2024-09-19T05:10:28.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.992+0000 7f2c60c00640 1 -- 172.21.15.43:0/3047035619 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2c640a06c0 con 0x55d82c918590 2024-09-19T05:10:28.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.992+0000 7f2c80bf4740 10 client.26345 fetch_fsmap learned FSMap version 29 2024-09-19T05:10:28.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.992+0000 7f2c80bf4740 1 -- 172.21.15.43:0/3047035619 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d82c931980 con 0x55d82c918590 2024-09-19T05:10:28.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.992+0000 7f2c60c00640 1 -- 172.21.15.43:0/3047035619 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c70004050 con 0x55d82c918590 2024-09-19T05:10:28.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.992+0000 7f2c60c00640 1 -- 172.21.15.43:0/3047035619 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2c64018cc0 con 0x55d82c918590 2024-09-19T05:10:28.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.992+0000 7f2c80bf4740 10 client.26345 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:10:28.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.992+0000 7f2c80bf4740 10 client.26345 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T05:10:28.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.992+0000 7f2c80bf4740 20 client.26345 populate_metadata read hostname 'smithi043' 2024-09-19T05:10:28.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.992+0000 7f2c80bf4740 1 --2- 172.21.15.43:0/3047035619 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55d82c932e40 0x55d82c935230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:28.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.992+0000 7f2c80bf4740 4 client.26345 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:10:28.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.992+0000 7f2c80bf4740 1 -- 172.21.15.43:0/3047035619 --> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d82c88b950 con 0x55d82c932e40 2024-09-19T05:10:28.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.993+0000 7f2c77400640 1 --2- 172.21.15.43:0/3047035619 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55d82c932e40 0x55d82c935230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:28.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.993+0000 7f2c77400640 1 --2- 172.21.15.43:0/3047035619 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55d82c932e40 0x55d82c935230 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a15ab74a10068c17 server_cookie=46a08f6b13ce84f9 in_seq=0 out_seq=0 2024-09-19T05:10:28.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.993+0000 7f2c60c00640 10 client.26345 ms_handle_connect on v2:172.21.15.96:6836/3978388286 2024-09-19T05:10:28.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.994+0000 7f2c60c00640 1 -- 172.21.15.43:0/3047035619 <== mds.3 v2:172.21.15.96:6836/3978388286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d82c88b950 con 0x55d82c932e40 2024-09-19T05:10:28.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.994+0000 7f2c60c00640 10 client.26345 handle_command_reply: tid=0 2024-09-19T05:10:28.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.994+0000 7f2c80bf4740 1 client.26345 shutdown 2024-09-19T05:10:28.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.994+0000 7f2c80bf4740 1 -- 172.21.15.43:0/3047035619 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d82c918590 msgr2=0x55d82c9201a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:28.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.994+0000 7f2c80bf4740 1 --2- 172.21.15.43:0/3047035619 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d82c918590 0x55d82c9201a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2c640cd780 tx=0x7f2c6409bf00 comp rx=0 tx=0).stop 2024-09-19T05:10:28.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.994+0000 7f2c80bf4740 1 -- 172.21.15.43:0/3047035619 shutdown_connections 2024-09-19T05:10:28.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.994+0000 7f2c80bf4740 1 --2- 172.21.15.43:0/3047035619 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55d82c932e40 0x55d82c935230 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:28.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.994+0000 7f2c80bf4740 1 --2- 172.21.15.43:0/3047035619 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d82c922e50 0x55d82c9206e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:28.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.994+0000 7f2c80bf4740 1 --2- 172.21.15.43:0/3047035619 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d82c918590 0x55d82c9201a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:28.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.994+0000 7f2c80bf4740 1 --2- 172.21.15.43:0/3047035619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d82c917c70 0x55d82c91fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:28.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.994+0000 7f2c80bf4740 1 -- 172.21.15.43:0/3047035619 >> 172.21.15.43:0/3047035619 conn(0x55d82c914200 msgr2=0x55d82c919f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:28.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.994+0000 7f2c60c00640 0 client.26345 ms_handle_reset on v2:172.21.15.96:6836/3978388286 2024-09-19T05:10:28.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.995+0000 7f2c80bf4740 1 -- 172.21.15.43:0/3047035619 shutdown_connections 2024-09-19T05:10:28.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.995+0000 7f2c80bf4740 1 -- 172.21.15.43:0/3047035619 wait complete. 2024-09-19T05:10:28.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:27.995+0000 7f2c80bf4740 20 client.26345 trim_cache size 0 max 16384 2024-09-19T05:10:28.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.008+0000 7f2c80bf4740 1 Processor -- start 2024-09-19T05:10:28.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.008+0000 7f2c80bf4740 1 -- start start 2024-09-19T05:10:28.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.009+0000 7f2c80bf4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d82c917c70 0x55d82c91a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:28.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.009+0000 7f2c80bf4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d82c918590 0x55d82c93ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:28.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.009+0000 7f2c80bf4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d82c922e50 0x55d82c93b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:28.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.009+0000 7f2c80bf4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2c781afa10 con 0x55d82c918590 2024-09-19T05:10:28.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.009+0000 7f2c80bf4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2c781b1550 con 0x55d82c922e50 2024-09-19T05:10:28.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.009+0000 7f2c80bf4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2c781b00e0 con 0x55d82c917c70 2024-09-19T05:10:28.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.009+0000 7f2c76a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d82c918590 0x55d82c93ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:28.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.009+0000 7f2c76a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d82c918590 0x55d82c93ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36074/0 (socket says 172.21.15.43:36074) 2024-09-19T05:10:28.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.009+0000 7f2c76a00640 1 -- 172.21.15.43:0/2563311511 learned_addr learned my addr 172.21.15.43:0/2563311511 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:28.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.009+0000 7f2c77400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d82c917c70 0x55d82c91a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:28.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.009+0000 7f2c77e00640 1 --2- 172.21.15.43:0/2563311511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d82c922e50 0x55d82c93b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:28.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.010+0000 7f2c76a00640 1 -- 172.21.15.43:0/2563311511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d82c917c70 msgr2=0x55d82c91a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:28.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.010+0000 7f2c76a00640 1 --2- 172.21.15.43:0/2563311511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d82c917c70 0x55d82c91a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:28.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.010+0000 7f2c76a00640 1 -- 172.21.15.43:0/2563311511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d82c922e50 msgr2=0x55d82c93b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:28.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.010+0000 7f2c76a00640 1 --2- 172.21.15.43:0/2563311511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d82c922e50 0x55d82c93b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:28.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.010+0000 7f2c76a00640 1 -- 172.21.15.43:0/2563311511 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d82c9358b0 con 0x55d82c918590 2024-09-19T05:10:28.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.010+0000 7f2c77400640 1 --2- 172.21.15.43:0/2563311511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d82c917c70 0x55d82c91a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:10:28.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.010+0000 7f2c76a00640 1 --2- 172.21.15.43:0/2563311511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d82c918590 0x55d82c93ab40 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2c6409b830 tx=0x7f2c6409c030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f086a9b5efc8f977 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:28.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.010+0000 7f2c63400640 1 -- 172.21.15.43:0/2563311511 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6409bd20 con 0x55d82c918590 2024-09-19T05:10:28.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.010+0000 7f2c63400640 1 -- 172.21.15.43:0/2563311511 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c640a4f80 con 0x55d82c918590 2024-09-19T05:10:28.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.010+0000 7f2c63400640 1 -- 172.21.15.43:0/2563311511 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c640bc2b0 con 0x55d82c918590 2024-09-19T05:10:28.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.011+0000 7f2c80bf4740 1 -- 172.21.15.43:0/2563311511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d82c918590 msgr2=0x55d82c93ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:28.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.011+0000 7f2c80bf4740 1 --2- 172.21.15.43:0/2563311511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d82c918590 0x55d82c93ab40 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2c6409b830 tx=0x7f2c6409c030 comp rx=0 tx=0).stop 2024-09-19T05:10:28.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.011+0000 7f2c80bf4740 1 -- 172.21.15.43:0/2563311511 shutdown_connections 2024-09-19T05:10:28.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.011+0000 7f2c80bf4740 1 --2- 172.21.15.43:0/2563311511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d82c922e50 0x55d82c93b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:28.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.011+0000 7f2c80bf4740 1 --2- 172.21.15.43:0/2563311511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d82c918590 0x55d82c93ab40 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:28.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.011+0000 7f2c80bf4740 1 --2- 172.21.15.43:0/2563311511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d82c917c70 0x55d82c91a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:28.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.011+0000 7f2c80bf4740 1 -- 172.21.15.43:0/2563311511 >> 172.21.15.43:0/2563311511 conn(0x55d82c914200 msgr2=0x55d82c921ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:28.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.011+0000 7f2c80bf4740 1 -- 172.21.15.43:0/2563311511 shutdown_connections 2024-09-19T05:10:28.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.011+0000 7f2c80bf4740 1 -- 172.21.15.43:0/2563311511 wait complete. 2024-09-19T05:10:28.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.011+0000 7f2c80bf4740 1 -- start start 2024-09-19T05:10:28.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.012+0000 7f2c80bf4740 1 Processor -- start 2024-09-19T05:10:28.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.012+0000 7f2c80bf4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d82c917c70 0x55d82c926a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:28.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.012+0000 7f2c80bf4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d82c918590 0x55d82c926fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:28.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.012+0000 7f2c80bf4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d82c922e50 0x55d82c927500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:28.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.012+0000 7f2c80bf4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2c781b0e80 con 0x55d82c917c70 2024-09-19T05:10:28.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.012+0000 7f2c80bf4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2c781b07b0 con 0x55d82c922e50 2024-09-19T05:10:28.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.012+0000 7f2c80bf4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2c781ad9f0 con 0x55d82c918590 2024-09-19T05:10:28.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.012+0000 7f2c80bf4740 10 client.0 _ll_register_callbacks cb 0x7f2c6400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:28.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.012+0000 7f2c77400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d82c917c70 0x55d82c926a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:28.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.013+0000 7f2c77400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d82c917c70 0x55d82c926a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36084/0 (socket says 172.21.15.43:36084) 2024-09-19T05:10:28.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.013+0000 7f2c77400640 1 -- 172.21.15.43:0/4055687147 learned_addr learned my addr 172.21.15.43:0/4055687147 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:28.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.013+0000 7f2c76a00640 1 --2- 172.21.15.43:0/4055687147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d82c918590 0x55d82c926fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:28.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.013+0000 7f2c77e00640 1 --2- 172.21.15.43:0/4055687147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d82c922e50 0x55d82c927500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:28.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.013+0000 7f2c77400640 1 -- 172.21.15.43:0/4055687147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d82c918590 msgr2=0x55d82c926fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:28.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.013+0000 7f2c77400640 1 --2- 172.21.15.43:0/4055687147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d82c918590 0x55d82c926fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:28.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.013+0000 7f2c77400640 1 -- 172.21.15.43:0/4055687147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d82c922e50 msgr2=0x55d82c927500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:28.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.013+0000 7f2c77400640 1 --2- 172.21.15.43:0/4055687147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d82c922e50 0x55d82c927500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:28.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.013+0000 7f2c77400640 1 -- 172.21.15.43:0/4055687147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d82c90f770 con 0x55d82c917c70 2024-09-19T05:10:28.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.013+0000 7f2c77400640 1 --2- 172.21.15.43:0/4055687147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d82c917c70 0x55d82c926a80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2c68005940 tx=0x7f2c68009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=830acc5ae7b3b39e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:28.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.013+0000 7f2c60c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:10:28.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.014+0000 7f2c60c00640 1 -- 172.21.15.43:0/4055687147 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c68035040 con 0x55d82c917c70 2024-09-19T05:10:28.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.014+0000 7f2c60c00640 1 -- 172.21.15.43:0/4055687147 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f2c68003dd0 con 0x55d82c917c70 2024-09-19T05:10:28.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.014+0000 7f2c80bf4740 1 -- 172.21.15.43:0/4055687147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c640a4f80 con 0x55d82c917c70 2024-09-19T05:10:28.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.014+0000 7f2c60c00640 1 -- 172.21.15.43:0/4055687147 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2c68054300 con 0x55d82c917c70 2024-09-19T05:10:28.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.014+0000 7f2c80bf4740 10 client.16311 fetch_fsmap learned FSMap version 29 2024-09-19T05:10:28.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.014+0000 7f2c80bf4740 1 -- 172.21.15.43:0/4055687147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d82c9152b0 con 0x55d82c917c70 2024-09-19T05:10:28.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.014+0000 7f2c60c00640 1 -- 172.21.15.43:0/4055687147 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c640a4f80 con 0x55d82c917c70 2024-09-19T05:10:28.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.014+0000 7f2c60c00640 1 -- 172.21.15.43:0/4055687147 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2c68066630 con 0x55d82c917c70 2024-09-19T05:10:28.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.014+0000 7f2c80bf4740 10 client.16311 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:10:28.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.014+0000 7f2c80bf4740 10 client.16311 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T05:10:28.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.014+0000 7f2c80bf4740 20 client.16311 populate_metadata read hostname 'smithi043' 2024-09-19T05:10:28.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.015+0000 7f2c80bf4740 1 --2- 172.21.15.43:0/4055687147 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55d82c99e070 0x55d82c9a0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:28.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.015+0000 7f2c80bf4740 4 client.16311 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:10:28.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.015+0000 7f2c80bf4740 1 -- 172.21.15.43:0/4055687147 --> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d82c885e60 con 0x55d82c99e070 2024-09-19T05:10:28.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.015+0000 7f2c76a00640 1 --2- 172.21.15.43:0/4055687147 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55d82c99e070 0x55d82c9a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:28.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.015+0000 7f2c76a00640 1 --2- 172.21.15.43:0/4055687147 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55d82c99e070 0x55d82c9a0460 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c6cd9c042e488191 server_cookie=9abf93dec40e45bd in_seq=0 out_seq=0 2024-09-19T05:10:28.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.015+0000 7f2c60c00640 10 client.16311 ms_handle_connect on v2:172.21.15.96:6836/3978388286 2024-09-19T05:10:28.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.016+0000 7f2c60c00640 1 -- 172.21.15.43:0/4055687147 <== mds.3 v2:172.21.15.96:6836/3978388286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d82c885e60 con 0x55d82c99e070 2024-09-19T05:10:28.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.016+0000 7f2c60c00640 10 client.16311 handle_command_reply: tid=0 2024-09-19T05:10:28.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.016+0000 7f2c80bf4740 1 client.16311 shutdown 2024-09-19T05:10:28.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.016+0000 7f2c80bf4740 1 -- 172.21.15.43:0/4055687147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d82c917c70 msgr2=0x55d82c926a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:28.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.016+0000 7f2c80bf4740 1 --2- 172.21.15.43:0/4055687147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d82c917c70 0x55d82c926a80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2c68005940 tx=0x7f2c68009260 comp rx=0 tx=0).stop 2024-09-19T05:10:28.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.016+0000 7f2c80bf4740 1 -- 172.21.15.43:0/4055687147 shutdown_connections 2024-09-19T05:10:28.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.016+0000 7f2c80bf4740 1 --2- 172.21.15.43:0/4055687147 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55d82c99e070 0x55d82c9a0460 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:28.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.016+0000 7f2c80bf4740 1 --2- 172.21.15.43:0/4055687147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d82c922e50 0x55d82c927500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:28.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.016+0000 7f2c80bf4740 1 --2- 172.21.15.43:0/4055687147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d82c918590 0x55d82c926fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:28.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.016+0000 7f2c60c00640 0 client.16311 ms_handle_reset on v2:172.21.15.96:6836/3978388286 2024-09-19T05:10:28.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.016+0000 7f2c80bf4740 1 --2- 172.21.15.43:0/4055687147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d82c917c70 0x55d82c926a80 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:28.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.016+0000 7f2c80bf4740 1 -- 172.21.15.43:0/4055687147 >> 172.21.15.43:0/4055687147 conn(0x55d82c914200 msgr2=0x55d82c912b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:28.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.016+0000 7f2c80bf4740 1 -- 172.21.15.43:0/4055687147 shutdown_connections 2024-09-19T05:10:28.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.016+0000 7f2c80bf4740 1 -- 172.21.15.43:0/4055687147 wait complete. 2024-09-19T05:10:28.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.016+0000 7f2c80bf4740 20 client.16311 trim_cache size 0 max 16384 2024-09-19T05:10:28.023 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:10:28.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.017+0000 7f2c7e000640 1 -- 172.21.15.43:0/2922930382 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f2c5c0a33b0 msgr2=0x7f2c5c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:28.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.017+0000 7f2c7e000640 1 --2- 172.21.15.43:0/2922930382 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f2c5c0a33b0 0x7f2c5c0a5870 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2c68009230 tx=0x7f2c680058d0 comp rx=0 tx=0).stop 2024-09-19T05:10:28.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.017+0000 7f2c7e000640 1 -- 172.21.15.43:0/2922930382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c781049a0 msgr2=0x7f2c7819ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:28.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.017+0000 7f2c7e000640 1 --2- 172.21.15.43:0/2922930382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c781049a0 0x7f2c7819ab30 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2c6400e9f0 tx=0x7f2c6400eeb0 comp rx=0 tx=0).stop 2024-09-19T05:10:28.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.017+0000 7f2c7e000640 1 -- 172.21.15.43:0/2922930382 shutdown_connections 2024-09-19T05:10:28.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.017+0000 7f2c7e000640 1 --2- 172.21.15.43:0/2922930382 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f2c5c0a33b0 0x7f2c5c0a5870 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:28.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.017+0000 7f2c7e000640 1 --2- 172.21.15.43:0/2922930382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c78109120 0x7f2c7819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:28.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.017+0000 7f2c7e000640 1 --2- 172.21.15.43:0/2922930382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c781049a0 0x7f2c7819ab30 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:28.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.017+0000 7f2c7e000640 1 --2- 172.21.15.43:0/2922930382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c78103ff0 0x7f2c7819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:28.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.017+0000 7f2c7e000640 1 -- 172.21.15.43:0/2922930382 >> 172.21.15.43:0/2922930382 conn(0x7f2c780ffa90 msgr2=0x7f2c78100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:28.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.017+0000 7f2c7e000640 1 -- 172.21.15.43:0/2922930382 shutdown_connections 2024-09-19T05:10:28.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:28.017+0000 7f2c7e000640 1 -- 172.21.15.43:0/2922930382 wait complete. 2024-09-19T05:10:28.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:27 smithi151 ceph-mon[35425]: pgmap v177: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:10:28.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:27 smithi096 ceph-mon[35085]: pgmap v177: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:10:28.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:27 smithi043 ceph-mon[30800]: pgmap v177: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:10:28.887 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T05:10:29.267 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:10:29.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.802+0000 7fe9dd200640 1 -- 172.21.15.43:0/4173613452 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9d81049c0 msgr2=0x7fe9d8108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:29.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.802+0000 7fe9dd200640 1 --2- 172.21.15.43:0/4173613452 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9d81049c0 0x7fe9d8108a10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0009900 tx=0x7fe9c002a400 comp rx=0 tx=0).stop 2024-09-19T05:10:29.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.802+0000 7fe9dd200640 1 -- 172.21.15.43:0/4173613452 shutdown_connections 2024-09-19T05:10:29.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.802+0000 7fe9dd200640 1 --2- 172.21.15.43:0/4173613452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9d8109140 0x7fe9d8110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:29.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.802+0000 7fe9dd200640 1 --2- 172.21.15.43:0/4173613452 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9d81049c0 0x7fe9d8108a10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:29.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.802+0000 7fe9dd200640 1 --2- 172.21.15.43:0/4173613452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9d8104010 0x7fe9d81043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:29.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.802+0000 7fe9dd200640 1 -- 172.21.15.43:0/4173613452 >> 172.21.15.43:0/4173613452 conn(0x7fe9d80ffa50 msgr2=0x7fe9d8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:29.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.802+0000 7fe9dd200640 1 -- 172.21.15.43:0/4173613452 shutdown_connections 2024-09-19T05:10:29.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.802+0000 7fe9dd200640 1 -- 172.21.15.43:0/4173613452 wait complete. 2024-09-19T05:10:29.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.803+0000 7fe9dd200640 1 Processor -- start 2024-09-19T05:10:29.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.803+0000 7fe9dd200640 1 -- start start 2024-09-19T05:10:29.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.803+0000 7fe9dd200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9d8104010 0x7fe9d819a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:29.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.803+0000 7fe9dd200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9d81049c0 0x7fe9d819aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:29.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.803+0000 7fe9dd200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9d8109140 0x7fe9d819ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:29.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.803+0000 7fe9dd200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d8114140 con 0x7fe9d8109140 2024-09-19T05:10:29.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.803+0000 7fe9dd200640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d8113fc0 con 0x7fe9d8104010 2024-09-19T05:10:29.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.803+0000 7fe9dd200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d81142c0 con 0x7fe9d81049c0 2024-09-19T05:10:29.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.804+0000 7fe9cfe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9d81049c0 0x7fe9d819aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:29.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.804+0000 7fe9d6800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9d8109140 0x7fe9d819ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:29.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.804+0000 7fe9cfe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9d81049c0 0x7fe9d819aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:41824/0 (socket says 172.21.15.43:41824) 2024-09-19T05:10:29.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.804+0000 7fe9cfe00640 1 -- 172.21.15.43:0/3874761236 learned_addr learned my addr 172.21.15.43:0/3874761236 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:29.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.804+0000 7fe9d5e00640 1 --2- 172.21.15.43:0/3874761236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9d8104010 0x7fe9d819a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:29.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.804+0000 7fe9cfe00640 1 -- 172.21.15.43:0/3874761236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9d8104010 msgr2=0x7fe9d819a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:29.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.804+0000 7fe9cfe00640 1 --2- 172.21.15.43:0/3874761236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9d8104010 0x7fe9d819a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:29.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.804+0000 7fe9cfe00640 1 -- 172.21.15.43:0/3874761236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9d8109140 msgr2=0x7fe9d819ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:29.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.804+0000 7fe9cfe00640 1 --2- 172.21.15.43:0/3874761236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9d8109140 0x7fe9d819ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:29.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.804+0000 7fe9cfe00640 1 -- 172.21.15.43:0/3874761236 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d819f610 con 0x7fe9d81049c0 2024-09-19T05:10:29.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.804+0000 7fe9d6800640 1 --2- 172.21.15.43:0/3874761236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9d8109140 0x7fe9d819ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:10:29.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.805+0000 7fe9cfe00640 1 --2- 172.21.15.43:0/3874761236 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9d81049c0 0x7fe9d819aba0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0006010 tx=0x7fe9c0004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:29.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.805+0000 7fe9cea00640 1 -- 172.21.15.43:0/3874761236 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c003f030 con 0x7fe9d81049c0 2024-09-19T05:10:29.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.805+0000 7fe9cea00640 1 -- 172.21.15.43:0/3874761236 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9c0004440 con 0x7fe9d81049c0 2024-09-19T05:10:29.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.805+0000 7fe9cea00640 1 -- 172.21.15.43:0/3874761236 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0035070 con 0x7fe9d81049c0 2024-09-19T05:10:29.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.805+0000 7fe9dd200640 1 -- 172.21.15.43:0/3874761236 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9d819f8a0 con 0x7fe9d81049c0 2024-09-19T05:10:29.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.805+0000 7fe9dd200640 1 -- 172.21.15.43:0/3874761236 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9d819fd30 con 0x7fe9d81049c0 2024-09-19T05:10:29.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.808+0000 7fe9dfc80740 1 Processor -- start 2024-09-19T05:10:29.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.808+0000 7fe9dfc80740 1 -- start start 2024-09-19T05:10:29.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.809+0000 7fe9cea00640 1 -- 172.21.15.43:0/3874761236 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fe9c0042030 con 0x7fe9d81049c0 2024-09-19T05:10:29.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.809+0000 7fe9dfc80740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d1713c70 0x5624d1714050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:29.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.809+0000 7fe9dfc80740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d1714590 0x5624d171e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:29.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.809+0000 7fe9dfc80740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624d171ee50 0x5624d1721240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:29.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.809+0000 7fe9dfc80740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5624d167a3e0 con 0x5624d1714590 2024-09-19T05:10:29.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.809+0000 7fe9dfc80740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5624d1621d80 con 0x5624d1713c70 2024-09-19T05:10:29.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.809+0000 7fe9dfc80740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5624d16763a0 con 0x5624d171ee50 2024-09-19T05:10:29.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.809+0000 7fe9d5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d1714590 0x5624d171e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:29.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.809+0000 7fe9d5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d1714590 0x5624d171e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36122/0 (socket says 172.21.15.43:36122) 2024-09-19T05:10:29.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.809+0000 7fe9d5e00640 1 -- 172.21.15.43:0/2127267811 learned_addr learned my addr 172.21.15.43:0/2127267811 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:29.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.809+0000 7fe9d6800640 1 --2- 172.21.15.43:0/2127267811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d1713c70 0x5624d1714050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:29.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.810+0000 7fe9d6800640 1 -- 172.21.15.43:0/2127267811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624d171ee50 msgr2=0x5624d1721240 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:10:29.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.810+0000 7fe9d6800640 1 --2- 172.21.15.43:0/2127267811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624d171ee50 0x5624d1721240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:29.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.810+0000 7fe9d6800640 1 -- 172.21.15.43:0/2127267811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d1714590 msgr2=0x5624d171e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:29.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.810+0000 7fe9d6800640 1 --2- 172.21.15.43:0/2127267811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d1714590 0x5624d171e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:29.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.810+0000 7fe9d6800640 1 -- 172.21.15.43:0/2127267811 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624d1640ae0 con 0x5624d1713c70 2024-09-19T05:10:29.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.810+0000 7fe9d6800640 1 --2- 172.21.15.43:0/2127267811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d1713c70 0x5624d1714050 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0011760 tx=0x7fe9d0011c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9943aa316da8ef90 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:29.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.810+0000 7fe9c7400640 1 -- 172.21.15.43:0/2127267811 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d0004d20 con 0x5624d1713c70 2024-09-19T05:10:29.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.810+0000 7fe9c7400640 1 -- 172.21.15.43:0/2127267811 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9d0003970 con 0x5624d1713c70 2024-09-19T05:10:29.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.810+0000 7fe9c7400640 1 -- 172.21.15.43:0/2127267811 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d0004440 con 0x5624d1713c70 2024-09-19T05:10:29.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.810+0000 7fe9cea00640 1 --2- 172.21.15.43:0/3874761236 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fe9bc0a3430 0x7fe9bc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:29.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.811+0000 7fe9cea00640 1 -- 172.21.15.43:0/3874761236 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fe9c00efed0 con 0x7fe9d81049c0 2024-09-19T05:10:29.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.811+0000 7fe9d5e00640 1 --2- 172.21.15.43:0/3874761236 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fe9bc0a3430 0x7fe9bc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:29.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.811+0000 7fe9dfc80740 1 -- 172.21.15.43:0/2127267811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d1713c70 msgr2=0x5624d1714050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:29.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.811+0000 7fe9dfc80740 1 --2- 172.21.15.43:0/2127267811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d1713c70 0x5624d1714050 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0011760 tx=0x7fe9d0011c20 comp rx=0 tx=0).stop 2024-09-19T05:10:29.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.811+0000 7fe9dfc80740 1 -- 172.21.15.43:0/2127267811 shutdown_connections 2024-09-19T05:10:29.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.811+0000 7fe9dfc80740 1 --2- 172.21.15.43:0/2127267811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624d171ee50 0x5624d1721240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:29.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.811+0000 7fe9dfc80740 1 --2- 172.21.15.43:0/2127267811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d1714590 0x5624d171e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:29.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.811+0000 7fe9dfc80740 1 --2- 172.21.15.43:0/2127267811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d1713c70 0x5624d1714050 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:29.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.811+0000 7fe9dfc80740 1 -- 172.21.15.43:0/2127267811 >> 172.21.15.43:0/2127267811 conn(0x5624d1710200 msgr2=0x5624d17105e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:29.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.811+0000 7fe9dfc80740 1 -- 172.21.15.43:0/2127267811 shutdown_connections 2024-09-19T05:10:29.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.812+0000 7fe9dfc80740 1 -- 172.21.15.43:0/2127267811 wait complete. 2024-09-19T05:10:29.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.812+0000 7fe9d5e00640 1 --2- 172.21.15.43:0/3874761236 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fe9bc0a3430 0x7fe9bc0a58f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x5624d1711060 tx=0x7fe9c8005f90 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:29.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.812+0000 7fe9dfc80740 1 -- start start 2024-09-19T05:10:29.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.812+0000 7fe9dfc80740 1 Processor -- start 2024-09-19T05:10:29.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.813+0000 7fe9dfc80740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d1713c70 0x5624d171bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:29.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.813+0000 7fe9dfc80740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d1714590 0x5624d171c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:29.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.813+0000 7fe9dfc80740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624d171ee50 0x5624d171c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:29.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.813+0000 7fe9dfc80740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5624d1721a20 con 0x5624d1714590 2024-09-19T05:10:29.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.813+0000 7fe9dfc80740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5624d16350f0 con 0x5624d1713c70 2024-09-19T05:10:29.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.813+0000 7fe9dfc80740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5624d1721ba0 con 0x5624d171ee50 2024-09-19T05:10:29.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.813+0000 7fe9dfc80740 10 client.0 _ll_register_callbacks cb 0x5624d161cbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:29.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.813+0000 7fe9cfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d1714590 0x5624d171c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:29.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.813+0000 7fe9d5e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d1713c70 0x5624d171bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:29.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.813+0000 7fe9cfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d1714590 0x5624d171c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36130/0 (socket says 172.21.15.43:36130) 2024-09-19T05:10:29.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.813+0000 7fe9d5e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d1713c70 0x5624d171bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:51654/0 (socket says 172.21.15.43:51654) 2024-09-19T05:10:29.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.813+0000 7fe9cfe00640 1 -- 172.21.15.43:0/3036798517 learned_addr learned my addr 172.21.15.43:0/3036798517 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:29.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.813+0000 7fe9d6800640 1 --2- 172.21.15.43:0/3036798517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624d171ee50 0x5624d171c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:29.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.814+0000 7fe9cfe00640 1 -- 172.21.15.43:0/3036798517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624d171ee50 msgr2=0x5624d171c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:29.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.814+0000 7fe9cfe00640 1 --2- 172.21.15.43:0/3036798517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624d171ee50 0x5624d171c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:29.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.814+0000 7fe9cfe00640 1 -- 172.21.15.43:0/3036798517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d1713c70 msgr2=0x5624d171bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:29.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.814+0000 7fe9cfe00640 1 --2- 172.21.15.43:0/3036798517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d1713c70 0x5624d171bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:29.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.814+0000 7fe9cfe00640 1 -- 172.21.15.43:0/3036798517 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624d13eb3c0 con 0x5624d1714590 2024-09-19T05:10:29.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.814+0000 7fe9cfe00640 1 --2- 172.21.15.43:0/3036798517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d1714590 0x5624d171c2c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0000c00 tx=0x7fe9c00bbf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d319bceb3120460 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:29.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.814+0000 7fe9bbe00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:10:29.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.814+0000 7fe9bbe00640 1 -- 172.21.15.43:0/3036798517 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0049040 con 0x5624d1714590 2024-09-19T05:10:29.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.814+0000 7fe9bbe00640 1 -- 172.21.15.43:0/3036798517 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fe9c00b8830 con 0x5624d1714590 2024-09-19T05:10:29.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.814+0000 7fe9dfc80740 1 -- 172.21.15.43:0/3036798517 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9d0003970 con 0x5624d1714590 2024-09-19T05:10:29.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.815+0000 7fe9bbe00640 1 -- 172.21.15.43:0/3036798517 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe9c00bc8c0 con 0x5624d1714590 2024-09-19T05:10:29.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.815+0000 7fe9dfc80740 10 client.16323 fetch_fsmap learned FSMap version 29 2024-09-19T05:10:29.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.815+0000 7fe9dfc80740 1 -- 172.21.15.43:0/3036798517 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5624d172dab0 con 0x5624d1714590 2024-09-19T05:10:29.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.815+0000 7fe9bbe00640 1 -- 172.21.15.43:0/3036798517 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9d0003970 con 0x5624d1714590 2024-09-19T05:10:29.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.815+0000 7fe9bbe00640 1 -- 172.21.15.43:0/3036798517 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe9c0003c60 con 0x5624d1714590 2024-09-19T05:10:29.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.815+0000 7fe9dfc80740 10 client.16323 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:10:29.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.815+0000 7fe9dfc80740 10 client.16323 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T05:10:29.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.815+0000 7fe9dfc80740 20 client.16323 populate_metadata read hostname 'smithi043' 2024-09-19T05:10:29.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.815+0000 7fe9dfc80740 1 --2- 172.21.15.43:0/3036798517 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x5624d172eea0 0x5624d1731290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:29.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.815+0000 7fe9dfc80740 4 client.16323 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:10:29.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.815+0000 7fe9dfc80740 1 -- 172.21.15.43:0/3036798517 --> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624d1687950 con 0x5624d172eea0 2024-09-19T05:10:29.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.816+0000 7fe9d5e00640 1 --2- 172.21.15.43:0/3036798517 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x5624d172eea0 0x5624d1731290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:29.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.817+0000 7fe9d5e00640 1 --2- 172.21.15.43:0/3036798517 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x5624d172eea0 0x5624d1731290 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=85fbe2ad9fd0a3a1 server_cookie=b660462cfbc01845 in_seq=0 out_seq=0 2024-09-19T05:10:29.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.817+0000 7fe9bbe00640 10 client.16323 ms_handle_connect on v2:172.21.15.151:6838/3794104313 2024-09-19T05:10:29.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.817+0000 7fe9bbe00640 1 -- 172.21.15.43:0/3036798517 <== mds.2 v2:172.21.15.151:6838/3794104313 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5624d1687950 con 0x5624d172eea0 2024-09-19T05:10:29.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.817+0000 7fe9bbe00640 10 client.16323 handle_command_reply: tid=0 2024-09-19T05:10:29.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.817+0000 7fe9dfc80740 1 client.16323 shutdown 2024-09-19T05:10:29.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.818+0000 7fe9dfc80740 1 -- 172.21.15.43:0/3036798517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d1714590 msgr2=0x5624d171c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:29.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.818+0000 7fe9dfc80740 1 --2- 172.21.15.43:0/3036798517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d1714590 0x5624d171c2c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0000c00 tx=0x7fe9c00bbf90 comp rx=0 tx=0).stop 2024-09-19T05:10:29.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.818+0000 7fe9dfc80740 1 -- 172.21.15.43:0/3036798517 shutdown_connections 2024-09-19T05:10:29.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.818+0000 7fe9dfc80740 1 --2- 172.21.15.43:0/3036798517 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x5624d172eea0 0x5624d1731290 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:29.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.818+0000 7fe9dfc80740 1 --2- 172.21.15.43:0/3036798517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624d171ee50 0x5624d171c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:29.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.818+0000 7fe9dfc80740 1 --2- 172.21.15.43:0/3036798517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d1714590 0x5624d171c2c0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:29.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.818+0000 7fe9dfc80740 1 --2- 172.21.15.43:0/3036798517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d1713c70 0x5624d171bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:29.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.818+0000 7fe9bbe00640 0 client.16323 ms_handle_reset on v2:172.21.15.151:6838/3794104313 2024-09-19T05:10:29.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.818+0000 7fe9dfc80740 1 -- 172.21.15.43:0/3036798517 >> 172.21.15.43:0/3036798517 conn(0x5624d1710200 msgr2=0x5624d1715f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:29.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.818+0000 7fe9dfc80740 1 -- 172.21.15.43:0/3036798517 shutdown_connections 2024-09-19T05:10:29.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.818+0000 7fe9dfc80740 1 -- 172.21.15.43:0/3036798517 wait complete. 2024-09-19T05:10:29.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.818+0000 7fe9dfc80740 20 client.16323 trim_cache size 0 max 16384 2024-09-19T05:10:29.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.832+0000 7fe9dfc80740 1 Processor -- start 2024-09-19T05:10:29.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.832+0000 7fe9dfc80740 1 -- start start 2024-09-19T05:10:29.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.832+0000 7fe9dfc80740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d1713c70 0x5624d1715e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:29.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.833+0000 7fe9dfc80740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624d1714590 0x5624d1736ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:29.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.833+0000 7fe9dfc80740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d171ee50 0x5624d17370e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:29.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.833+0000 7fe9dfc80740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d81afa80 con 0x5624d171ee50 2024-09-19T05:10:29.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.833+0000 7fe9dfc80740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d81b15c0 con 0x5624d1713c70 2024-09-19T05:10:29.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.833+0000 7fe9dfc80740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d81b0150 con 0x5624d1714590 2024-09-19T05:10:29.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.833+0000 7fe9d6800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d171ee50 0x5624d17370e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:29.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.833+0000 7fe9d6800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d171ee50 0x5624d17370e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36142/0 (socket says 172.21.15.43:36142) 2024-09-19T05:10:29.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.833+0000 7fe9d6800640 1 -- 172.21.15.43:0/3758293413 learned_addr learned my addr 172.21.15.43:0/3758293413 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:29.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.833+0000 7fe9cfe00640 1 --2- 172.21.15.43:0/3758293413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624d1714590 0x5624d1736ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:29.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.833+0000 7fe9d5e00640 1 --2- 172.21.15.43:0/3758293413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d1713c70 0x5624d1715e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:29.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.833+0000 7fe9d6800640 1 -- 172.21.15.43:0/3758293413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624d1714590 msgr2=0x5624d1736ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:29.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.833+0000 7fe9d6800640 1 --2- 172.21.15.43:0/3758293413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624d1714590 0x5624d1736ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:29.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.833+0000 7fe9d6800640 1 -- 172.21.15.43:0/3758293413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d1713c70 msgr2=0x5624d1715e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:29.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.833+0000 7fe9d6800640 1 --2- 172.21.15.43:0/3758293413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d1713c70 0x5624d1715e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:29.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.833+0000 7fe9d6800640 1 -- 172.21.15.43:0/3758293413 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624d1731910 con 0x5624d171ee50 2024-09-19T05:10:29.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.833+0000 7fe9d5e00640 1 --2- 172.21.15.43:0/3758293413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d1713c70 0x5624d1715e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:10:29.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.834+0000 7fe9d6800640 1 --2- 172.21.15.43:0/3758293413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d171ee50 0x5624d17370e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe9d000f790 tx=0x7fe9d00111d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf60ec410aecf2ff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:29.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.834+0000 7fe9c6a00640 1 -- 172.21.15.43:0/3758293413 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d001d030 con 0x5624d171ee50 2024-09-19T05:10:29.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.834+0000 7fe9c6a00640 1 -- 172.21.15.43:0/3758293413 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9d00113c0 con 0x5624d171ee50 2024-09-19T05:10:29.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.834+0000 7fe9c6a00640 1 -- 172.21.15.43:0/3758293413 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d00266e0 con 0x5624d171ee50 2024-09-19T05:10:29.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.835+0000 7fe9dfc80740 1 -- 172.21.15.43:0/3758293413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d171ee50 msgr2=0x5624d17370e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:29.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.835+0000 7fe9dfc80740 1 --2- 172.21.15.43:0/3758293413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d171ee50 0x5624d17370e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe9d000f790 tx=0x7fe9d00111d0 comp rx=0 tx=0).stop 2024-09-19T05:10:29.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.835+0000 7fe9dfc80740 1 -- 172.21.15.43:0/3758293413 shutdown_connections 2024-09-19T05:10:29.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.835+0000 7fe9dfc80740 1 --2- 172.21.15.43:0/3758293413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d171ee50 0x5624d17370e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:29.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.835+0000 7fe9dfc80740 1 --2- 172.21.15.43:0/3758293413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624d1714590 0x5624d1736ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:29.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.835+0000 7fe9dfc80740 1 --2- 172.21.15.43:0/3758293413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d1713c70 0x5624d1715e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:29.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.835+0000 7fe9dfc80740 1 -- 172.21.15.43:0/3758293413 >> 172.21.15.43:0/3758293413 conn(0x5624d1710200 msgr2=0x5624d1715710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:29.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.835+0000 7fe9dfc80740 1 -- 172.21.15.43:0/3758293413 shutdown_connections 2024-09-19T05:10:29.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.835+0000 7fe9dfc80740 1 -- 172.21.15.43:0/3758293413 wait complete. 2024-09-19T05:10:29.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.836+0000 7fe9dfc80740 1 -- start start 2024-09-19T05:10:29.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.836+0000 7fe9dfc80740 1 Processor -- start 2024-09-19T05:10:29.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.836+0000 7fe9dfc80740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d1713c70 0x5624d1722ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:29.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.836+0000 7fe9dfc80740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d1714590 0x5624d1723010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:29.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.836+0000 7fe9dfc80740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624d171ee50 0x5624d1723550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:29.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.836+0000 7fe9dfc80740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d81b0ef0 con 0x5624d1714590 2024-09-19T05:10:29.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.836+0000 7fe9dfc80740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d81b0820 con 0x5624d1713c70 2024-09-19T05:10:29.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.836+0000 7fe9dfc80740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d81ada60 con 0x5624d171ee50 2024-09-19T05:10:29.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.836+0000 7fe9dfc80740 10 client.0 _ll_register_callbacks cb 0x7fe9c00314b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:29.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.837+0000 7fe9cfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d1714590 0x5624d1723010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:29.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.837+0000 7fe9d6800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624d171ee50 0x5624d1723550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:29.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.837+0000 7fe9cfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d1714590 0x5624d1723010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36158/0 (socket says 172.21.15.43:36158) 2024-09-19T05:10:29.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.837+0000 7fe9cfe00640 1 -- 172.21.15.43:0/3601937518 learned_addr learned my addr 172.21.15.43:0/3601937518 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:29.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.837+0000 7fe9d5e00640 1 --2- 172.21.15.43:0/3601937518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d1713c70 0x5624d1722ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:29.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.837+0000 7fe9d5e00640 1 -- 172.21.15.43:0/3601937518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624d171ee50 msgr2=0x5624d1723550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:29.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.837+0000 7fe9d5e00640 1 --2- 172.21.15.43:0/3601937518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624d171ee50 0x5624d1723550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:29.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.837+0000 7fe9d5e00640 1 -- 172.21.15.43:0/3601937518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d1714590 msgr2=0x5624d1723010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:29.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.837+0000 7fe9d5e00640 1 --2- 172.21.15.43:0/3601937518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d1714590 0x5624d1723010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:29.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.837+0000 7fe9d5e00640 1 -- 172.21.15.43:0/3601937518 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624d1710030 con 0x5624d1713c70 2024-09-19T05:10:29.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.838+0000 7fe9d5e00640 1 --2- 172.21.15.43:0/3601937518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d1713c70 0x5624d1722ad0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x5624d1731ae0 tx=0x7fe9c80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e919d473995294e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:29.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.838+0000 7fe9bbe00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:10:29.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.838+0000 7fe9bbe00640 1 -- 172.21.15.43:0/3601937518 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c801de50 con 0x5624d1713c70 2024-09-19T05:10:29.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.838+0000 7fe9dfc80740 1 -- 172.21.15.43:0/3601937518 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9d00113c0 con 0x5624d1713c70 2024-09-19T05:10:29.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.838+0000 7fe9bbe00640 1 -- 172.21.15.43:0/3601937518 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fe9c802a420 con 0x5624d1713c70 2024-09-19T05:10:29.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.838+0000 7fe9bbe00640 1 -- 172.21.15.43:0/3601937518 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe9c802a620 con 0x5624d1713c70 2024-09-19T05:10:29.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.838+0000 7fe9dfc80740 10 client.26369 fetch_fsmap learned FSMap version 29 2024-09-19T05:10:29.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.838+0000 7fe9dfc80740 1 -- 172.21.15.43:0/3601937518 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5624d1799c60 con 0x5624d1713c70 2024-09-19T05:10:29.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.838+0000 7fe9bbe00640 1 -- 172.21.15.43:0/3601937518 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9d00113c0 con 0x5624d1713c70 2024-09-19T05:10:29.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.839+0000 7fe9bbe00640 1 -- 172.21.15.43:0/3601937518 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe9c8033ae0 con 0x5624d1713c70 2024-09-19T05:10:29.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.839+0000 7fe9dfc80740 10 client.26369 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:10:29.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.839+0000 7fe9dfc80740 10 client.26369 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T05:10:29.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.839+0000 7fe9dfc80740 20 client.26369 populate_metadata read hostname 'smithi043' 2024-09-19T05:10:29.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.839+0000 7fe9dfc80740 1 --2- 172.21.15.43:0/3601937518 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x5624d179a380 0x5624d179c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:29.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.839+0000 7fe9dfc80740 4 client.26369 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:10:29.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.839+0000 7fe9dfc80740 1 -- 172.21.15.43:0/3601937518 --> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5624d1681e60 con 0x5624d179a380 2024-09-19T05:10:29.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.839+0000 7fe9cfe00640 1 --2- 172.21.15.43:0/3601937518 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x5624d179a380 0x5624d179c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:29.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.840+0000 7fe9cfe00640 1 --2- 172.21.15.43:0/3601937518 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x5624d179a380 0x5624d179c7a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=51d0bccbf6f4517d server_cookie=5d995417045b7a41 in_seq=0 out_seq=0 2024-09-19T05:10:29.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.840+0000 7fe9bbe00640 10 client.26369 ms_handle_connect on v2:172.21.15.151:6838/3794104313 2024-09-19T05:10:29.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.840+0000 7fe9bbe00640 1 -- 172.21.15.43:0/3601937518 <== mds.2 v2:172.21.15.151:6838/3794104313 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5624d1681e60 con 0x5624d179a380 2024-09-19T05:10:29.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.840+0000 7fe9bbe00640 10 client.26369 handle_command_reply: tid=0 2024-09-19T05:10:29.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.840+0000 7fe9dfc80740 1 client.26369 shutdown 2024-09-19T05:10:29.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.840+0000 7fe9dfc80740 1 -- 172.21.15.43:0/3601937518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d1713c70 msgr2=0x5624d1722ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:29.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.840+0000 7fe9dfc80740 1 --2- 172.21.15.43:0/3601937518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d1713c70 0x5624d1722ad0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x5624d1731ae0 tx=0x7fe9c80046c0 comp rx=0 tx=0).stop 2024-09-19T05:10:29.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.840+0000 7fe9dfc80740 1 -- 172.21.15.43:0/3601937518 shutdown_connections 2024-09-19T05:10:29.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.840+0000 7fe9dfc80740 1 --2- 172.21.15.43:0/3601937518 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x5624d179a380 0x5624d179c7a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:29.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.840+0000 7fe9dfc80740 1 --2- 172.21.15.43:0/3601937518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624d171ee50 0x5624d1723550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:29.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.840+0000 7fe9dfc80740 1 --2- 172.21.15.43:0/3601937518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d1714590 0x5624d1723010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:29.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.840+0000 7fe9dfc80740 1 --2- 172.21.15.43:0/3601937518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d1713c70 0x5624d1722ad0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:29.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.840+0000 7fe9bbe00640 0 client.26369 ms_handle_reset on v2:172.21.15.151:6838/3794104313 2024-09-19T05:10:29.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.840+0000 7fe9dfc80740 1 -- 172.21.15.43:0/3601937518 >> 172.21.15.43:0/3601937518 conn(0x5624d1710200 msgr2=0x5624d171a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:29.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.841+0000 7fe9dfc80740 1 -- 172.21.15.43:0/3601937518 shutdown_connections 2024-09-19T05:10:29.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.841+0000 7fe9dfc80740 1 -- 172.21.15.43:0/3601937518 wait complete. 2024-09-19T05:10:29.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.841+0000 7fe9dfc80740 20 client.26369 trim_cache size 0 max 16384 2024-09-19T05:10:29.846 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:10:29.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.842+0000 7fe9dd200640 1 -- 172.21.15.43:0/3874761236 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fe9bc0a3430 msgr2=0x7fe9bc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:29.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.842+0000 7fe9dd200640 1 --2- 172.21.15.43:0/3874761236 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fe9bc0a3430 0x7fe9bc0a58f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x5624d1711060 tx=0x7fe9c8005f90 comp rx=0 tx=0).stop 2024-09-19T05:10:29.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.842+0000 7fe9dd200640 1 -- 172.21.15.43:0/3874761236 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9d81049c0 msgr2=0x7fe9d819aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:29.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.842+0000 7fe9dd200640 1 --2- 172.21.15.43:0/3874761236 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9d81049c0 0x7fe9d819aba0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0006010 tx=0x7fe9c0004040 comp rx=0 tx=0).stop 2024-09-19T05:10:29.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.842+0000 7fe9dd200640 1 -- 172.21.15.43:0/3874761236 shutdown_connections 2024-09-19T05:10:29.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.842+0000 7fe9dd200640 1 --2- 172.21.15.43:0/3874761236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9d8109140 0x7fe9d819ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:29.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.842+0000 7fe9dd200640 1 --2- 172.21.15.43:0/3874761236 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9d81049c0 0x7fe9d819aba0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:29.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.842+0000 7fe9dd200640 1 --2- 172.21.15.43:0/3874761236 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fe9bc0a3430 0x7fe9bc0a58f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:29.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.842+0000 7fe9dd200640 1 --2- 172.21.15.43:0/3874761236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9d8104010 0x7fe9d819a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:29.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.842+0000 7fe9dd200640 1 -- 172.21.15.43:0/3874761236 >> 172.21.15.43:0/3874761236 conn(0x7fe9d80ffa50 msgr2=0x7fe9d8100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:29.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.842+0000 7fe9dd200640 1 -- 172.21.15.43:0/3874761236 shutdown_connections 2024-09-19T05:10:29.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:29.842+0000 7fe9dd200640 1 -- 172.21.15.43:0/3874761236 wait complete. 2024-09-19T05:10:30.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:29 smithi151 ceph-mon[35425]: pgmap v178: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:10:30.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:29 smithi096 ceph-mon[35085]: pgmap v178: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:10:30.261 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:29 smithi043 ceph-mon[30800]: pgmap v178: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:10:30.770 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T05:10:31.151 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:10:31.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.719+0000 7f7bb0a00640 1 -- 172.21.15.43:0/2645124534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bac108f60 msgr2=0x7f7bac110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:31.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.719+0000 7f7bb0a00640 1 --2- 172.21.15.43:0/2645124534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bac108f60 0x7f7bac110ae0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4009ca0 tx=0x7f7ba402a8c0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.719+0000 7f7bb0a00640 1 -- 172.21.15.43:0/2645124534 shutdown_connections 2024-09-19T05:10:31.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.719+0000 7f7bb0a00640 1 --2- 172.21.15.43:0/2645124534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bac108f60 0x7f7bac110ae0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.719+0000 7f7bb0a00640 1 --2- 172.21.15.43:0/2645124534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7bac1047e0 0x7f7bac108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.719+0000 7f7bb0a00640 1 --2- 172.21.15.43:0/2645124534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7bac103e30 0x7f7bac104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.719+0000 7f7bb0a00640 1 -- 172.21.15.43:0/2645124534 >> 172.21.15.43:0/2645124534 conn(0x7f7bac0ff8d0 msgr2=0x7f7bac101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:31.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.719+0000 7f7bb0a00640 1 -- 172.21.15.43:0/2645124534 shutdown_connections 2024-09-19T05:10:31.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.719+0000 7f7bb0a00640 1 -- 172.21.15.43:0/2645124534 wait complete. 2024-09-19T05:10:31.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.720+0000 7f7bb0a00640 1 Processor -- start 2024-09-19T05:10:31.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.720+0000 7f7bb0a00640 1 -- start start 2024-09-19T05:10:31.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.721+0000 7f7bb0a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7bac103e30 0x7f7bac19a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:31.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.721+0000 7f7bb0a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bac1047e0 0x7f7bac19a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:31.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.721+0000 7f7bb0a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7bac108f60 0x7f7bac19ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:31.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.721+0000 7f7bb0a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7bac114010 con 0x7f7bac1047e0 2024-09-19T05:10:31.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.721+0000 7f7bb0a00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7bac113e90 con 0x7f7bac108f60 2024-09-19T05:10:31.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.721+0000 7f7bb0a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7bac114190 con 0x7f7bac103e30 2024-09-19T05:10:31.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.721+0000 7f7ba1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bac1047e0 0x7f7bac19a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:31.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.721+0000 7f7ba1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bac1047e0 0x7f7bac19a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36176/0 (socket says 172.21.15.43:36176) 2024-09-19T05:10:31.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.721+0000 7f7ba1e00640 1 -- 172.21.15.43:0/2583431213 learned_addr learned my addr 172.21.15.43:0/2583431213 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:31.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.721+0000 7f7baa800640 1 --2- 172.21.15.43:0/2583431213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7bac103e30 0x7f7bac19a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:31.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.721+0000 7f7bab200640 1 --2- 172.21.15.43:0/2583431213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7bac108f60 0x7f7bac19ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:31.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.722+0000 7f7ba1e00640 1 -- 172.21.15.43:0/2583431213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7bac103e30 msgr2=0x7f7bac19a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:31.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.722+0000 7f7ba1e00640 1 --2- 172.21.15.43:0/2583431213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7bac103e30 0x7f7bac19a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.722+0000 7f7ba1e00640 1 -- 172.21.15.43:0/2583431213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7bac108f60 msgr2=0x7f7bac19ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:31.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.722+0000 7f7ba1e00640 1 --2- 172.21.15.43:0/2583431213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7bac108f60 0x7f7bac19ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.722+0000 7f7ba1e00640 1 -- 172.21.15.43:0/2583431213 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bac19f3a0 con 0x7f7bac1047e0 2024-09-19T05:10:31.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.722+0000 7f7baa800640 1 --2- 172.21.15.43:0/2583431213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7bac103e30 0x7f7bac19a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:10:31.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.722+0000 7f7ba1e00640 1 --2- 172.21.15.43:0/2583431213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bac1047e0 0x7f7bac19a930 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7b9400e9f0 tx=0x7f7b9400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:31.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.722+0000 7f7ba3e00640 1 -- 172.21.15.43:0/2583431213 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9400cd80 con 0x7f7bac1047e0 2024-09-19T05:10:31.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.722+0000 7f7ba3e00640 1 -- 172.21.15.43:0/2583431213 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b94005500 con 0x7f7bac1047e0 2024-09-19T05:10:31.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.722+0000 7f7bb0a00640 1 -- 172.21.15.43:0/2583431213 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bac19f690 con 0x7f7bac1047e0 2024-09-19T05:10:31.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.722+0000 7f7ba3e00640 1 -- 172.21.15.43:0/2583431213 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b94010690 con 0x7f7bac1047e0 2024-09-19T05:10:31.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.722+0000 7f7bb0a00640 1 -- 172.21.15.43:0/2583431213 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bac1a6f70 con 0x7f7bac1047e0 2024-09-19T05:10:31.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.726+0000 7f7bb3dcd740 1 Processor -- start 2024-09-19T05:10:31.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.727+0000 7f7bb3dcd740 1 -- start start 2024-09-19T05:10:31.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.727+0000 7f7ba3e00640 1 -- 172.21.15.43:0/2583431213 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f7b940046c0 con 0x7f7bac1047e0 2024-09-19T05:10:31.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.727+0000 7f7bb3dcd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648594e1e40 0x5648594e2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:31.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.727+0000 7f7bb3dcd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648594e2760 0x5648594ecae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:31.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.727+0000 7f7bb3dcd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648594ed020 0x5648594ef410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:31.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.727+0000 7f7bb3dcd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5648594483e0 con 0x5648594ed020 2024-09-19T05:10:31.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.727+0000 7f7bb3dcd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5648593efd80 con 0x5648594e1e40 2024-09-19T05:10:31.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.727+0000 7f7bb3dcd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5648594443a0 con 0x5648594e2760 2024-09-19T05:10:31.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.727+0000 7f7ba1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648594ed020 0x5648594ef410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:31.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.728+0000 7f7bab200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648594e1e40 0x5648594e2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:31.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.728+0000 7f7baa800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648594e2760 0x5648594ecae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:31.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.728+0000 7f7bab200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648594e1e40 0x5648594e2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:51716/0 (socket says 172.21.15.43:51716) 2024-09-19T05:10:31.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.728+0000 7f7bab200640 1 -- 172.21.15.43:0/1722989283 learned_addr learned my addr 172.21.15.43:0/1722989283 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.728+0000 7f7baa800640 1 -- 172.21.15.43:0/1722989283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648594e1e40 msgr2=0x5648594e2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.728+0000 7f7baa800640 1 --2- 172.21.15.43:0/1722989283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648594e1e40 0x5648594e2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.728+0000 7f7baa800640 1 -- 172.21.15.43:0/1722989283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648594ed020 msgr2=0x5648594ef410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.728+0000 7f7baa800640 1 --2- 172.21.15.43:0/1722989283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648594ed020 0x5648594ef410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.728+0000 7f7baa800640 1 -- 172.21.15.43:0/1722989283 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56485940eae0 con 0x5648594e2760 2024-09-19T05:10:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.728+0000 7f7baa800640 1 --2- 172.21.15.43:0/1722989283 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648594e2760 0x5648594ecae0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c009fd0 tx=0x7f7b9c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=febf8080a220ffbc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.729+0000 7f7ba3e00640 1 --2- 172.21.15.43:0/2583431213 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f7b900a3430 0x7f7b900a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:31.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.729+0000 7f7b9b400640 1 -- 172.21.15.43:0/1722989283 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c004030 con 0x5648594e2760 2024-09-19T05:10:31.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.729+0000 7f7b9b400640 1 -- 172.21.15.43:0/1722989283 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b9c005500 con 0x5648594e2760 2024-09-19T05:10:31.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.729+0000 7f7b9b400640 1 -- 172.21.15.43:0/1722989283 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c004e10 con 0x5648594e2760 2024-09-19T05:10:31.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.729+0000 7f7ba3e00640 1 -- 172.21.15.43:0/2583431213 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f7b940d0060 con 0x7f7bac1047e0 2024-09-19T05:10:31.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.729+0000 7f7bab200640 1 --2- 172.21.15.43:0/2583431213 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f7b900a3430 0x7f7b900a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:31.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.729+0000 7f7bb3dcd740 1 -- 172.21.15.43:0/1722989283 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648594e2760 msgr2=0x5648594ecae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:31.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.729+0000 7f7bb3dcd740 1 --2- 172.21.15.43:0/1722989283 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648594e2760 0x5648594ecae0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c009fd0 tx=0x7f7b9c0118d0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.730+0000 7f7bb3dcd740 1 -- 172.21.15.43:0/1722989283 shutdown_connections 2024-09-19T05:10:31.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.730+0000 7f7bb3dcd740 1 --2- 172.21.15.43:0/1722989283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648594ed020 0x5648594ef410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.730+0000 7f7bb3dcd740 1 --2- 172.21.15.43:0/1722989283 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648594e2760 0x5648594ecae0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.730+0000 7f7bb3dcd740 1 --2- 172.21.15.43:0/1722989283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648594e1e40 0x5648594e2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.730+0000 7f7bb3dcd740 1 -- 172.21.15.43:0/1722989283 >> 172.21.15.43:0/1722989283 conn(0x5648594de3d0 msgr2=0x5648594de7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:31.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.730+0000 7f7bb3dcd740 1 -- 172.21.15.43:0/1722989283 shutdown_connections 2024-09-19T05:10:31.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.730+0000 7f7bab200640 1 --2- 172.21.15.43:0/2583431213 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f7b900a3430 0x7f7b900a58f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4009a60 tx=0x7f7ba4005d80 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:31.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.730+0000 7f7bb3dcd740 1 -- 172.21.15.43:0/1722989283 wait complete. 2024-09-19T05:10:31.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.730+0000 7f7bb3dcd740 1 -- start start 2024-09-19T05:10:31.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.731+0000 7f7bb3dcd740 1 Processor -- start 2024-09-19T05:10:31.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.731+0000 7f7bb3dcd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648594e1e40 0x5648594e9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:31.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.731+0000 7f7bb3dcd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648594e2760 0x5648594ea430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:31.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.731+0000 7f7bb3dcd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648594ed020 0x5648594ea970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:31.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.731+0000 7f7bb3dcd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5648594efbf0 con 0x5648594e2760 2024-09-19T05:10:31.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.731+0000 7f7bb3dcd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5648594030f0 con 0x5648594ed020 2024-09-19T05:10:31.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.731+0000 7f7bb3dcd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5648594efd70 con 0x5648594e1e40 2024-09-19T05:10:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.731+0000 7f7bb3dcd740 10 client.0 _ll_register_callbacks cb 0x5648593eabd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.731+0000 7f7baa800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648594e1e40 0x5648594e9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.731+0000 7f7baa800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648594e1e40 0x5648594e9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:41896/0 (socket says 172.21.15.43:41896) 2024-09-19T05:10:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.731+0000 7f7ba1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648594e2760 0x5648594ea430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.731+0000 7f7baa800640 1 -- 172.21.15.43:0/4264310794 learned_addr learned my addr 172.21.15.43:0/4264310794 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.732+0000 7f7bab200640 1 --2- 172.21.15.43:0/4264310794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648594ed020 0x5648594ea970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.732+0000 7f7baa800640 1 -- 172.21.15.43:0/4264310794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648594ed020 msgr2=0x5648594ea970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.732+0000 7f7baa800640 1 --2- 172.21.15.43:0/4264310794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648594ed020 0x5648594ea970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.732+0000 7f7baa800640 1 -- 172.21.15.43:0/4264310794 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648594e2760 msgr2=0x5648594ea430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:31.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.732+0000 7f7baa800640 1 --2- 172.21.15.43:0/4264310794 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648594e2760 0x5648594ea430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.732+0000 7f7baa800640 1 -- 172.21.15.43:0/4264310794 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648591b93c0 con 0x5648594e1e40 2024-09-19T05:10:31.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.732+0000 7f7bab200640 1 --2- 172.21.15.43:0/4264310794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648594ed020 0x5648594ea970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:10:31.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.732+0000 7f7baa800640 1 --2- 172.21.15.43:0/4264310794 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648594e1e40 0x5648594e9ef0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c01b040 tx=0x7f7b9c0051b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89fd7a9399014239 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:31.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.732+0000 7f7b8fe00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:10:31.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.733+0000 7f7b8fe00640 1 -- 172.21.15.43:0/4264310794 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c028070 con 0x5648594e1e40 2024-09-19T05:10:31.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.733+0000 7f7bb3dcd740 1 -- 172.21.15.43:0/4264310794 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b9c005500 con 0x5648594e1e40 2024-09-19T05:10:31.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.733+0000 7f7b8fe00640 1 -- 172.21.15.43:0/4264310794 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f7b9c00f5a0 con 0x5648594e1e40 2024-09-19T05:10:31.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.733+0000 7f7b8fe00640 1 -- 172.21.15.43:0/4264310794 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7b9c034050 con 0x5648594e1e40 2024-09-19T05:10:31.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.733+0000 7f7bb3dcd740 10 client.26314 fetch_fsmap learned FSMap version 29 2024-09-19T05:10:31.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.733+0000 7f7bb3dcd740 1 -- 172.21.15.43:0/4264310794 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5648594fbc10 con 0x5648594e1e40 2024-09-19T05:10:31.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.734+0000 7f7b8fe00640 1 -- 172.21.15.43:0/4264310794 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b9c005500 con 0x5648594e1e40 2024-09-19T05:10:31.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.734+0000 7f7b8fe00640 1 -- 172.21.15.43:0/4264310794 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7b9c019690 con 0x5648594e1e40 2024-09-19T05:10:31.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.734+0000 7f7bb3dcd740 10 client.26314 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:10:31.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.734+0000 7f7bb3dcd740 10 client.26314 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-19T05:10:31.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.734+0000 7f7bb3dcd740 20 client.26314 populate_metadata read hostname 'smithi043' 2024-09-19T05:10:31.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.734+0000 7f7bb3dcd740 1 --2- 172.21.15.43:0/4264310794 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5648594fcfc0 0x5648594ff3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:31.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.734+0000 7f7bb3dcd740 4 client.26314 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:10:31.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.734+0000 7f7bb3dcd740 1 -- 172.21.15.43:0/4264310794 --> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564859455950 con 0x5648594fcfc0 2024-09-19T05:10:31.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.734+0000 7f7ba1e00640 1 --2- 172.21.15.43:0/4264310794 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5648594fcfc0 0x5648594ff3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:31.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.735+0000 7f7ba1e00640 1 --2- 172.21.15.43:0/4264310794 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5648594fcfc0 0x5648594ff3b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=91fea400818a0af1 server_cookie=f1addd1bd46b0822 in_seq=0 out_seq=0 2024-09-19T05:10:31.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.735+0000 7f7b8fe00640 10 client.26314 ms_handle_connect on v2:172.21.15.43:6838/3404555030 2024-09-19T05:10:31.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.735+0000 7f7b8fe00640 1 -- 172.21.15.43:0/4264310794 <== mds.1 v2:172.21.15.43:6838/3404555030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564859455950 con 0x5648594fcfc0 2024-09-19T05:10:31.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.735+0000 7f7b8fe00640 10 client.26314 handle_command_reply: tid=0 2024-09-19T05:10:31.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.736+0000 7f7bb3dcd740 1 client.26314 shutdown 2024-09-19T05:10:31.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.736+0000 7f7bb3dcd740 1 -- 172.21.15.43:0/4264310794 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648594e1e40 msgr2=0x5648594e9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:31.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.736+0000 7f7bb3dcd740 1 --2- 172.21.15.43:0/4264310794 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648594e1e40 0x5648594e9ef0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c01b040 tx=0x7f7b9c0051b0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.736+0000 7f7bb3dcd740 1 -- 172.21.15.43:0/4264310794 shutdown_connections 2024-09-19T05:10:31.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.736+0000 7f7bb3dcd740 1 --2- 172.21.15.43:0/4264310794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648594ed020 0x5648594ea970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.736+0000 7f7bb3dcd740 1 --2- 172.21.15.43:0/4264310794 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648594e2760 0x5648594ea430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.736+0000 7f7bb3dcd740 1 --2- 172.21.15.43:0/4264310794 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5648594fcfc0 0x5648594ff3b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.736+0000 7f7bb3dcd740 1 --2- 172.21.15.43:0/4264310794 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648594e1e40 0x5648594e9ef0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.736+0000 7f7bb3dcd740 1 -- 172.21.15.43:0/4264310794 >> 172.21.15.43:0/4264310794 conn(0x5648594de3d0 msgr2=0x5648594e40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:31.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.736+0000 7f7b8fe00640 0 client.26314 ms_handle_reset on v2:172.21.15.43:6838/3404555030 2024-09-19T05:10:31.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.736+0000 7f7bb3dcd740 1 -- 172.21.15.43:0/4264310794 shutdown_connections 2024-09-19T05:10:31.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.736+0000 7f7bb3dcd740 1 -- 172.21.15.43:0/4264310794 wait complete. 2024-09-19T05:10:31.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.736+0000 7f7bb3dcd740 20 client.26314 trim_cache size 0 max 16384 2024-09-19T05:10:31.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.750+0000 7f7bb3dcd740 1 Processor -- start 2024-09-19T05:10:31.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.750+0000 7f7bb3dcd740 1 -- start start 2024-09-19T05:10:31.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.750+0000 7f7bb3dcd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648594e1e40 0x564859504b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:31.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.750+0000 7f7bb3dcd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648594e2760 0x5648595050b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:31.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.750+0000 7f7bb3dcd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648594ed020 0x5648594dcc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:31.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.750+0000 7f7bb3dcd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7bac1af870 con 0x5648594e1e40 2024-09-19T05:10:31.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.751+0000 7f7bb3dcd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7bac1b13b0 con 0x5648594e2760 2024-09-19T05:10:31.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.751+0000 7f7bb3dcd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7bac1aff40 con 0x5648594ed020 2024-09-19T05:10:31.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.751+0000 7f7baa800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648594e1e40 0x564859504b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:31.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.751+0000 7f7bab200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648594ed020 0x5648594dcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:31.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.751+0000 7f7bab200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648594ed020 0x5648594dcc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:41906/0 (socket says 172.21.15.43:41906) 2024-09-19T05:10:31.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.751+0000 7f7baa800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648594e1e40 0x564859504b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36206/0 (socket says 172.21.15.43:36206) 2024-09-19T05:10:31.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.751+0000 7f7bab200640 1 -- 172.21.15.43:0/3397075946 learned_addr learned my addr 172.21.15.43:0/3397075946 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:31.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.751+0000 7f7ba1e00640 1 --2- 172.21.15.43:0/3397075946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648594e2760 0x5648595050b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:31.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.751+0000 7f7baa800640 1 -- 172.21.15.43:0/3397075946 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648594ed020 msgr2=0x5648594dcc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:31.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.751+0000 7f7baa800640 1 --2- 172.21.15.43:0/3397075946 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648594ed020 0x5648594dcc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.751+0000 7f7baa800640 1 -- 172.21.15.43:0/3397075946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648594e2760 msgr2=0x5648595050b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:31.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.751+0000 7f7baa800640 1 --2- 172.21.15.43:0/3397075946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648594e2760 0x5648595050b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.751+0000 7f7baa800640 1 -- 172.21.15.43:0/3397075946 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648594ff940 con 0x5648594e1e40 2024-09-19T05:10:31.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.751+0000 7f7ba1e00640 1 --2- 172.21.15.43:0/3397075946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648594e2760 0x5648595050b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:10:31.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.752+0000 7f7baa800640 1 --2- 172.21.15.43:0/3397075946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648594e1e40 0x564859504b70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c019a90 tx=0x7f7b9c009d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa2d328151eb5809 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:31.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.752+0000 7f7b9aa00640 1 -- 172.21.15.43:0/3397075946 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c031070 con 0x5648594e1e40 2024-09-19T05:10:31.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.752+0000 7f7b9aa00640 1 -- 172.21.15.43:0/3397075946 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b9c0048d0 con 0x5648594e1e40 2024-09-19T05:10:31.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.752+0000 7f7b9aa00640 1 -- 172.21.15.43:0/3397075946 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c028040 con 0x5648594e1e40 2024-09-19T05:10:31.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.752+0000 7f7bb3dcd740 1 -- 172.21.15.43:0/3397075946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648594e1e40 msgr2=0x564859504b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:31.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.752+0000 7f7bb3dcd740 1 --2- 172.21.15.43:0/3397075946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648594e1e40 0x564859504b70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c019a90 tx=0x7f7b9c009d00 comp rx=0 tx=0).stop 2024-09-19T05:10:31.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.752+0000 7f7bb3dcd740 1 -- 172.21.15.43:0/3397075946 shutdown_connections 2024-09-19T05:10:31.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.752+0000 7f7bb3dcd740 1 --2- 172.21.15.43:0/3397075946 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648594ed020 0x5648594dcc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.752+0000 7f7bb3dcd740 1 --2- 172.21.15.43:0/3397075946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648594e2760 0x5648595050b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.752+0000 7f7bb3dcd740 1 --2- 172.21.15.43:0/3397075946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648594e1e40 0x564859504b70 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.752+0000 7f7bb3dcd740 1 -- 172.21.15.43:0/3397075946 >> 172.21.15.43:0/3397075946 conn(0x5648594de3d0 msgr2=0x5648594ebf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:31.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.753+0000 7f7bb3dcd740 1 -- 172.21.15.43:0/3397075946 shutdown_connections 2024-09-19T05:10:31.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.753+0000 7f7bb3dcd740 1 -- 172.21.15.43:0/3397075946 wait complete. 2024-09-19T05:10:31.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.753+0000 7f7bb3dcd740 1 -- start start 2024-09-19T05:10:31.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.753+0000 7f7bb3dcd740 1 Processor -- start 2024-09-19T05:10:31.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.754+0000 7f7bb3dcd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648594e1e40 0x5648594f5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:31.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.754+0000 7f7bb3dcd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648594e2760 0x5648594f58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:31.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.754+0000 7f7bb3dcd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648594ed020 0x5648594e6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:31.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.754+0000 7f7bb3dcd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7bac1b0ce0 con 0x5648594e1e40 2024-09-19T05:10:31.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.754+0000 7f7bb3dcd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7bac1b0610 con 0x5648594e2760 2024-09-19T05:10:31.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.754+0000 7f7bb3dcd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7bac1ad850 con 0x5648594ed020 2024-09-19T05:10:31.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.754+0000 7f7bb3dcd740 10 client.0 _ll_register_callbacks cb 0x7f7b940d3a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:31.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.754+0000 7f7baa800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648594e1e40 0x5648594f5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:31.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.754+0000 7f7ba1e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648594e2760 0x5648594f58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:31.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.754+0000 7f7baa800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648594e1e40 0x5648594f5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36208/0 (socket says 172.21.15.43:36208) 2024-09-19T05:10:31.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.754+0000 7f7baa800640 1 -- 172.21.15.43:0/926531045 learned_addr learned my addr 172.21.15.43:0/926531045 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:31.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.754+0000 7f7bab200640 1 --2- 172.21.15.43:0/926531045 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648594ed020 0x5648594e6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:31.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.755+0000 7f7ba1e00640 1 -- 172.21.15.43:0/926531045 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648594ed020 msgr2=0x5648594e6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:31.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.755+0000 7f7ba1e00640 1 --2- 172.21.15.43:0/926531045 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648594ed020 0x5648594e6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.755+0000 7f7ba1e00640 1 -- 172.21.15.43:0/926531045 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648594e1e40 msgr2=0x5648594f5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:31.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.755+0000 7f7ba1e00640 1 --2- 172.21.15.43:0/926531045 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648594e1e40 0x5648594f5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.755+0000 7f7ba1e00640 1 -- 172.21.15.43:0/926531045 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648594e73c0 con 0x5648594e2760 2024-09-19T05:10:31.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.755+0000 7f7baa800640 1 --2- 172.21.15.43:0/926531045 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648594e1e40 0x5648594f5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:10:31.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.755+0000 7f7ba1e00640 1 --2- 172.21.15.43:0/926531045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648594e2760 0x5648594f58b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7b940cd7c0 tx=0x7f7b940baf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20ba1e91b371ef6d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:31.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.755+0000 7f7b8fe00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:10:31.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.755+0000 7f7b8fe00640 1 -- 172.21.15.43:0/926531045 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9409be90 con 0x5648594e2760 2024-09-19T05:10:31.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.755+0000 7f7b8fe00640 1 -- 172.21.15.43:0/926531045 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f7b940cda40 con 0x5648594e2760 2024-09-19T05:10:31.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.755+0000 7f7bb3dcd740 1 -- 172.21.15.43:0/926531045 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b9c0048d0 con 0x5648594e2760 2024-09-19T05:10:31.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.755+0000 7f7b8fe00640 1 -- 172.21.15.43:0/926531045 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7b9409bcb0 con 0x5648594e2760 2024-09-19T05:10:31.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.755+0000 7f7bb3dcd740 10 client.26387 fetch_fsmap learned FSMap version 29 2024-09-19T05:10:31.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.755+0000 7f7bb3dcd740 1 -- 172.21.15.43:0/926531045 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564859567890 con 0x5648594e2760 2024-09-19T05:10:31.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.756+0000 7f7b8fe00640 1 -- 172.21.15.43:0/926531045 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b9c0048d0 con 0x5648594e2760 2024-09-19T05:10:31.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.756+0000 7f7b8fe00640 1 -- 172.21.15.43:0/926531045 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7b940989c0 con 0x5648594e2760 2024-09-19T05:10:31.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.756+0000 7f7bb3dcd740 10 client.26387 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:10:31.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.756+0000 7f7bb3dcd740 10 client.26387 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-19T05:10:31.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.756+0000 7f7bb3dcd740 20 client.26387 populate_metadata read hostname 'smithi043' 2024-09-19T05:10:31.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.756+0000 7f7bb3dcd740 1 --2- 172.21.15.43:0/926531045 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x564859568060 0x56485956a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:31.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.756+0000 7f7bb3dcd740 4 client.26387 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:10:31.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.756+0000 7f7bb3dcd740 1 -- 172.21.15.43:0/926531045 --> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56485944fe60 con 0x564859568060 2024-09-19T05:10:31.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.756+0000 7f7baa800640 1 --2- 172.21.15.43:0/926531045 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x564859568060 0x56485956a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:31.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.757+0000 7f7baa800640 1 --2- 172.21.15.43:0/926531045 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x564859568060 0x56485956a480 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=50b5b9efd2aee8c3 server_cookie=fd88fa00e18a9d7d in_seq=0 out_seq=0 2024-09-19T05:10:31.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.757+0000 7f7b8fe00640 10 client.26387 ms_handle_connect on v2:172.21.15.43:6838/3404555030 2024-09-19T05:10:31.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.757+0000 7f7b8fe00640 1 -- 172.21.15.43:0/926531045 <== mds.1 v2:172.21.15.43:6838/3404555030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56485944fe60 con 0x564859568060 2024-09-19T05:10:31.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.757+0000 7f7b8fe00640 10 client.26387 handle_command_reply: tid=0 2024-09-19T05:10:31.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.757+0000 7f7bb3dcd740 1 client.26387 shutdown 2024-09-19T05:10:31.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.757+0000 7f7bb3dcd740 1 -- 172.21.15.43:0/926531045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648594e2760 msgr2=0x5648594f58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:31.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.757+0000 7f7bb3dcd740 1 --2- 172.21.15.43:0/926531045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648594e2760 0x5648594f58b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7b940cd7c0 tx=0x7f7b940baf90 comp rx=0 tx=0).stop 2024-09-19T05:10:31.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.757+0000 7f7bb3dcd740 1 -- 172.21.15.43:0/926531045 shutdown_connections 2024-09-19T05:10:31.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.757+0000 7f7bb3dcd740 1 --2- 172.21.15.43:0/926531045 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x564859568060 0x56485956a480 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.757+0000 7f7bb3dcd740 1 --2- 172.21.15.43:0/926531045 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648594ed020 0x5648594e6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.757+0000 7f7b8fe00640 0 client.26387 ms_handle_reset on v2:172.21.15.43:6838/3404555030 2024-09-19T05:10:31.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.757+0000 7f7bb3dcd740 1 --2- 172.21.15.43:0/926531045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648594e2760 0x5648594f58b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.757+0000 7f7bb3dcd740 1 --2- 172.21.15.43:0/926531045 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648594e1e40 0x5648594f5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.757+0000 7f7bb3dcd740 1 -- 172.21.15.43:0/926531045 >> 172.21.15.43:0/926531045 conn(0x5648594de3d0 msgr2=0x5648594e86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:31.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.758+0000 7f7bb3dcd740 1 -- 172.21.15.43:0/926531045 shutdown_connections 2024-09-19T05:10:31.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.758+0000 7f7bb3dcd740 1 -- 172.21.15.43:0/926531045 wait complete. 2024-09-19T05:10:31.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.758+0000 7f7bb3dcd740 20 client.26387 trim_cache size 0 max 16384 2024-09-19T05:10:31.763 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:10:31.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.758+0000 7f7bb0a00640 1 -- 172.21.15.43:0/2583431213 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f7b900a3430 msgr2=0x7f7b900a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:31.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.758+0000 7f7bb0a00640 1 --2- 172.21.15.43:0/2583431213 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f7b900a3430 0x7f7b900a58f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4009a60 tx=0x7f7ba4005d80 comp rx=0 tx=0).stop 2024-09-19T05:10:31.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.758+0000 7f7bb0a00640 1 -- 172.21.15.43:0/2583431213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bac1047e0 msgr2=0x7f7bac19a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:31.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.758+0000 7f7bb0a00640 1 --2- 172.21.15.43:0/2583431213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bac1047e0 0x7f7bac19a930 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7b9400e9f0 tx=0x7f7b9400edf0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.759+0000 7f7bb0a00640 1 -- 172.21.15.43:0/2583431213 shutdown_connections 2024-09-19T05:10:31.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.759+0000 7f7bb0a00640 1 --2- 172.21.15.43:0/2583431213 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f7b900a3430 0x7f7b900a58f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.759+0000 7f7bb0a00640 1 --2- 172.21.15.43:0/2583431213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7bac108f60 0x7f7bac19ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.759+0000 7f7bb0a00640 1 --2- 172.21.15.43:0/2583431213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bac1047e0 0x7f7bac19a930 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.759+0000 7f7bb0a00640 1 --2- 172.21.15.43:0/2583431213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7bac103e30 0x7f7bac19a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:31.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.759+0000 7f7bb0a00640 1 -- 172.21.15.43:0/2583431213 >> 172.21.15.43:0/2583431213 conn(0x7f7bac0ff8d0 msgr2=0x7f7bac101ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:31.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.759+0000 7f7bb0a00640 1 -- 172.21.15.43:0/2583431213 shutdown_connections 2024-09-19T05:10:31.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:31.759+0000 7f7bb0a00640 1 -- 172.21.15.43:0/2583431213 wait complete. 2024-09-19T05:10:32.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:31 smithi043 ceph-mon[30800]: pgmap v179: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:10:32.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:31 smithi151 ceph-mon[35425]: pgmap v179: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:10:32.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:31 smithi096 ceph-mon[35085]: pgmap v179: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:10:32.655 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T05:10:33.035 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:10:33.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.619+0000 7f7c25600640 1 -- 172.21.15.43:0/2403681271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c2010e3c0 msgr2=0x7f7c20110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:33.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.619+0000 7f7c25600640 1 --2- 172.21.15.43:0/2403681271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c2010e3c0 0x7f7c20110810 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7c18009ca0 tx=0x7f7c1802a940 comp rx=0 tx=0).stop 2024-09-19T05:10:33.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.619+0000 7f7c25600640 1 -- 172.21.15.43:0/2403681271 shutdown_connections 2024-09-19T05:10:33.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.619+0000 7f7c25600640 1 --2- 172.21.15.43:0/2403681271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c2010e3c0 0x7f7c20110810 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.619+0000 7f7c25600640 1 --2- 172.21.15.43:0/2403681271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c20074f40 0x7f7c2010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.619+0000 7f7c25600640 1 --2- 172.21.15.43:0/2403681271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c20074620 0x7f7c20074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.619+0000 7f7c25600640 1 -- 172.21.15.43:0/2403681271 >> 172.21.15.43:0/2403681271 conn(0x7f7c200fd8c0 msgr2=0x7f7c200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:33.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.619+0000 7f7c25600640 1 -- 172.21.15.43:0/2403681271 shutdown_connections 2024-09-19T05:10:33.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.620+0000 7f7c25600640 1 -- 172.21.15.43:0/2403681271 wait complete. 2024-09-19T05:10:33.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.620+0000 7f7c25600640 1 Processor -- start 2024-09-19T05:10:33.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.621+0000 7f7c25600640 1 -- start start 2024-09-19T05:10:33.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.621+0000 7f7c25600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c20074620 0x7f7c2019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:33.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.621+0000 7f7c25600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c20074f40 0x7f7c2019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:33.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.621+0000 7f7c25600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c2010e3c0 0x7f7c2019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:33.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.621+0000 7f7c25600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7c201141b0 con 0x7f7c20074620 2024-09-19T05:10:33.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.621+0000 7f7c25600640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7c20114030 con 0x7f7c2010e3c0 2024-09-19T05:10:33.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.621+0000 7f7c25600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7c20114330 con 0x7f7c20074f40 2024-09-19T05:10:33.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.621+0000 7f7c1ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c20074620 0x7f7c2019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:33.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.621+0000 7f7c1ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c20074620 0x7f7c2019a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36216/0 (socket says 172.21.15.43:36216) 2024-09-19T05:10:33.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.621+0000 7f7c1ea00640 1 -- 172.21.15.43:0/4115573891 learned_addr learned my addr 172.21.15.43:0/4115573891 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:33.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.621+0000 7f7c17e00640 1 --2- 172.21.15.43:0/4115573891 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c20074f40 0x7f7c2019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:33.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.621+0000 7f7c1f400640 1 --2- 172.21.15.43:0/4115573891 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c2010e3c0 0x7f7c2019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:33.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.622+0000 7f7c1ea00640 1 -- 172.21.15.43:0/4115573891 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c20074f40 msgr2=0x7f7c2019ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:33.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.622+0000 7f7c1ea00640 1 --2- 172.21.15.43:0/4115573891 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c20074f40 0x7f7c2019ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.622+0000 7f7c1ea00640 1 -- 172.21.15.43:0/4115573891 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c2010e3c0 msgr2=0x7f7c2019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:33.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.622+0000 7f7c1ea00640 1 --2- 172.21.15.43:0/4115573891 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c2010e3c0 0x7f7c2019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.622+0000 7f7c1ea00640 1 -- 172.21.15.43:0/4115573891 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c2019f5e0 con 0x7f7c20074620 2024-09-19T05:10:33.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.622+0000 7f7c1ea00640 1 --2- 172.21.15.43:0/4115573891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c20074620 0x7f7c2019a630 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7c1000d770 tx=0x7f7c1000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:33.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.622+0000 7f7c17400640 1 -- 172.21.15.43:0/4115573891 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c10017070 con 0x7f7c20074620 2024-09-19T05:10:33.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.622+0000 7f7c17400640 1 -- 172.21.15.43:0/4115573891 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c10005500 con 0x7f7c20074620 2024-09-19T05:10:33.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.622+0000 7f7c17400640 1 -- 172.21.15.43:0/4115573891 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c10004e00 con 0x7f7c20074620 2024-09-19T05:10:33.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.622+0000 7f7c25600640 1 -- 172.21.15.43:0/4115573891 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c2019f8d0 con 0x7f7c20074620 2024-09-19T05:10:33.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.623+0000 7f7c25600640 1 -- 172.21.15.43:0/4115573891 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c201a71b0 con 0x7f7c20074620 2024-09-19T05:10:33.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.626+0000 7f7c17400640 1 -- 172.21.15.43:0/4115573891 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f7c10004fa0 con 0x7f7c20074620 2024-09-19T05:10:33.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.626+0000 7f7c28272740 1 Processor -- start 2024-09-19T05:10:33.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.626+0000 7f7c28272740 1 -- start start 2024-09-19T05:10:33.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.626+0000 7f7c28272740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f1c34c70 0x5569f1c35050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:33.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.626+0000 7f7c28272740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f1c35590 0x5569f1c3f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:33.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.626+0000 7f7c28272740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f1c3fe50 0x5569f1c42240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:33.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.626+0000 7f7c28272740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5569f1b9b3e0 con 0x5569f1c3fe50 2024-09-19T05:10:33.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.626+0000 7f7c28272740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5569f1b42d80 con 0x5569f1c34c70 2024-09-19T05:10:33.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.626+0000 7f7c28272740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5569f1b973a0 con 0x5569f1c35590 2024-09-19T05:10:33.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.627+0000 7f7c17e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f1c3fe50 0x5569f1c42240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:33.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.627+0000 7f7c17e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f1c3fe50 0x5569f1c42240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36230/0 (socket says 172.21.15.43:36230) 2024-09-19T05:10:33.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.627+0000 7f7c17e00640 1 -- 172.21.15.43:0/2087641314 learned_addr learned my addr 172.21.15.43:0/2087641314 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:33.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.627+0000 7f7c1f400640 1 --2- 172.21.15.43:0/2087641314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f1c34c70 0x5569f1c35050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:33.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.627+0000 7f7c17e00640 1 -- 172.21.15.43:0/2087641314 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f1c35590 msgr2=0x5569f1c3f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:33.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.627+0000 7f7c17e00640 1 --2- 172.21.15.43:0/2087641314 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f1c35590 0x5569f1c3f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.627+0000 7f7c17e00640 1 -- 172.21.15.43:0/2087641314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f1c34c70 msgr2=0x5569f1c35050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:33.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.627+0000 7f7c17e00640 1 --2- 172.21.15.43:0/2087641314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f1c34c70 0x5569f1c35050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.627+0000 7f7c17e00640 1 -- 172.21.15.43:0/2087641314 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569f1b61ae0 con 0x5569f1c3fe50 2024-09-19T05:10:33.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.627+0000 7f7c1f400640 1 --2- 172.21.15.43:0/2087641314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f1c34c70 0x5569f1c35050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:10:33.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.627+0000 7f7c17400640 1 --2- 172.21.15.43:0/4115573891 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f7c040a3430 0x7f7c040a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:33.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.627+0000 7f7c17e00640 1 --2- 172.21.15.43:0/2087641314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f1c3fe50 0x5569f1c42240 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7c0800bc30 tx=0x7f7c080107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e961af02748ac7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:33.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.628+0000 7f7c17400640 1 -- 172.21.15.43:0/4115573891 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f7c100cf630 con 0x7f7c20074620 2024-09-19T05:10:33.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.628+0000 7f7c0f400640 1 -- 172.21.15.43:0/2087641314 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0801c480 con 0x5569f1c3fe50 2024-09-19T05:10:33.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.628+0000 7f7c1f400640 1 --2- 172.21.15.43:0/4115573891 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f7c040a3430 0x7f7c040a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:33.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.628+0000 7f7c0f400640 1 -- 172.21.15.43:0/2087641314 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c0801cb00 con 0x5569f1c3fe50 2024-09-19T05:10:33.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.628+0000 7f7c0f400640 1 -- 172.21.15.43:0/2087641314 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c08011c00 con 0x5569f1c3fe50 2024-09-19T05:10:33.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.629+0000 7f7c28272740 1 -- 172.21.15.43:0/2087641314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f1c3fe50 msgr2=0x5569f1c42240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:33.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.629+0000 7f7c1f400640 1 --2- 172.21.15.43:0/4115573891 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f7c040a3430 0x7f7c040a58f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7c18006010 tx=0x7f7c18005d80 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:33.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.629+0000 7f7c28272740 1 --2- 172.21.15.43:0/2087641314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f1c3fe50 0x5569f1c42240 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7c0800bc30 tx=0x7f7c080107a0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.629+0000 7f7c28272740 1 -- 172.21.15.43:0/2087641314 shutdown_connections 2024-09-19T05:10:33.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.629+0000 7f7c28272740 1 --2- 172.21.15.43:0/2087641314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f1c3fe50 0x5569f1c42240 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.629+0000 7f7c28272740 1 --2- 172.21.15.43:0/2087641314 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f1c35590 0x5569f1c3f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.629+0000 7f7c28272740 1 --2- 172.21.15.43:0/2087641314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f1c34c70 0x5569f1c35050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.629+0000 7f7c28272740 1 -- 172.21.15.43:0/2087641314 >> 172.21.15.43:0/2087641314 conn(0x5569f1c31200 msgr2=0x5569f1c315e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:33.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.629+0000 7f7c28272740 1 -- 172.21.15.43:0/2087641314 shutdown_connections 2024-09-19T05:10:33.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.629+0000 7f7c28272740 1 -- 172.21.15.43:0/2087641314 wait complete. 2024-09-19T05:10:33.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.630+0000 7f7c28272740 1 -- start start 2024-09-19T05:10:33.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.630+0000 7f7c28272740 1 Processor -- start 2024-09-19T05:10:33.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.630+0000 7f7c28272740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f1c34c70 0x5569f1c3cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:33.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.630+0000 7f7c28272740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f1c35590 0x5569f1c3d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:33.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.630+0000 7f7c28272740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f1c3fe50 0x5569f1c3d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:33.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.630+0000 7f7c28272740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5569f1c42a20 con 0x5569f1c35590 2024-09-19T05:10:33.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.630+0000 7f7c28272740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5569f1b560f0 con 0x5569f1c34c70 2024-09-19T05:10:33.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.630+0000 7f7c28272740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5569f1c42ba0 con 0x5569f1c3fe50 2024-09-19T05:10:33.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.630+0000 7f7c28272740 10 client.0 _ll_register_callbacks cb 0x5569f1b3dbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:33.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.631+0000 7f7c17e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f1c35590 0x5569f1c3d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:33.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.631+0000 7f7c17e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f1c35590 0x5569f1c3d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36246/0 (socket says 172.21.15.43:36246) 2024-09-19T05:10:33.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.631+0000 7f7c17e00640 1 -- 172.21.15.43:0/914434111 learned_addr learned my addr 172.21.15.43:0/914434111 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:33.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.631+0000 7f7c1ea00640 1 --2- 172.21.15.43:0/914434111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f1c34c70 0x5569f1c3cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:33.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.631+0000 7f7c1f400640 1 --2- 172.21.15.43:0/914434111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f1c3fe50 0x5569f1c3d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:33.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.631+0000 7f7c1ea00640 1 -- 172.21.15.43:0/914434111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f1c3fe50 msgr2=0x5569f1c3d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:33.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.631+0000 7f7c1ea00640 1 --2- 172.21.15.43:0/914434111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f1c3fe50 0x5569f1c3d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.631+0000 7f7c1ea00640 1 -- 172.21.15.43:0/914434111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f1c35590 msgr2=0x5569f1c3d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:33.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.631+0000 7f7c1ea00640 1 --2- 172.21.15.43:0/914434111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f1c35590 0x5569f1c3d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.631+0000 7f7c1ea00640 1 -- 172.21.15.43:0/914434111 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569f190c3c0 con 0x5569f1c34c70 2024-09-19T05:10:33.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.631+0000 7f7c1f400640 1 --2- 172.21.15.43:0/914434111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f1c3fe50 0x5569f1c3d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:10:33.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.631+0000 7f7c17e00640 1 --2- 172.21.15.43:0/914434111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f1c35590 0x5569f1c3d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:10:33.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.632+0000 7f7c1ea00640 1 --2- 172.21.15.43:0/914434111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f1c34c70 0x5569f1c3cdf0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7c100ccd90 tx=0x7f7c1009b3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=923337457818ec19 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:33.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.632+0000 7f7c03e00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:10:33.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.632+0000 7f7c03e00640 1 -- 172.21.15.43:0/914434111 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c10020070 con 0x5569f1c34c70 2024-09-19T05:10:33.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.632+0000 7f7c28272740 1 -- 172.21.15.43:0/914434111 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c0801cb00 con 0x5569f1c34c70 2024-09-19T05:10:33.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.632+0000 7f7c03e00640 1 -- 172.21.15.43:0/914434111 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f7c10009cc0 con 0x5569f1c34c70 2024-09-19T05:10:33.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.632+0000 7f7c03e00640 1 -- 172.21.15.43:0/914434111 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7c100a4590 con 0x5569f1c34c70 2024-09-19T05:10:33.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.632+0000 7f7c28272740 10 client.26399 fetch_fsmap learned FSMap version 29 2024-09-19T05:10:33.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.632+0000 7f7c28272740 1 -- 172.21.15.43:0/914434111 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5569f1c4eb50 con 0x5569f1c34c70 2024-09-19T05:10:33.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.632+0000 7f7c03e00640 1 -- 172.21.15.43:0/914434111 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c0801cb00 con 0x5569f1c34c70 2024-09-19T05:10:33.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.633+0000 7f7c03e00640 1 -- 172.21.15.43:0/914434111 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7c1009ae20 con 0x5569f1c34c70 2024-09-19T05:10:33.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.633+0000 7f7c28272740 10 client.26399 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:10:33.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.633+0000 7f7c28272740 10 client.26399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:10:33.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.633+0000 7f7c28272740 20 client.26399 populate_metadata read hostname 'smithi043' 2024-09-19T05:10:33.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.633+0000 7f7c28272740 1 --2- 172.21.15.43:0/914434111 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5569f1c4ff60 0x5569f1c52350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:33.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.633+0000 7f7c28272740 4 client.26399 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:10:33.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.633+0000 7f7c28272740 1 -- 172.21.15.43:0/914434111 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569f1ba8950 con 0x5569f1c4ff60 2024-09-19T05:10:33.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.633+0000 7f7c17e00640 1 --2- 172.21.15.43:0/914434111 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5569f1c4ff60 0x5569f1c52350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:33.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.634+0000 7f7c17e00640 1 --2- 172.21.15.43:0/914434111 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5569f1c4ff60 0x5569f1c52350 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=697fd781ea3d860a server_cookie=2487e8ab9b59acd8 in_seq=0 out_seq=0 2024-09-19T05:10:33.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.634+0000 7f7c03e00640 10 client.26399 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:10:33.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.634+0000 7f7c03e00640 1 -- 172.21.15.43:0/914434111 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5569f1ba8950 con 0x5569f1c4ff60 2024-09-19T05:10:33.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.634+0000 7f7c03e00640 10 client.26399 handle_command_reply: tid=0 2024-09-19T05:10:33.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.634+0000 7f7c28272740 1 client.26399 shutdown 2024-09-19T05:10:33.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.635+0000 7f7c28272740 1 -- 172.21.15.43:0/914434111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f1c34c70 msgr2=0x5569f1c3cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:33.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.635+0000 7f7c28272740 1 --2- 172.21.15.43:0/914434111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f1c34c70 0x5569f1c3cdf0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7c100ccd90 tx=0x7f7c1009b3f0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.635+0000 7f7c28272740 1 -- 172.21.15.43:0/914434111 shutdown_connections 2024-09-19T05:10:33.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.635+0000 7f7c28272740 1 --2- 172.21.15.43:0/914434111 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5569f1c4ff60 0x5569f1c52350 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.635+0000 7f7c28272740 1 --2- 172.21.15.43:0/914434111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f1c3fe50 0x5569f1c3d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.635+0000 7f7c28272740 1 --2- 172.21.15.43:0/914434111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f1c35590 0x5569f1c3d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.635+0000 7f7c28272740 1 --2- 172.21.15.43:0/914434111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f1c34c70 0x5569f1c3cdf0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.635+0000 7f7c28272740 1 -- 172.21.15.43:0/914434111 >> 172.21.15.43:0/914434111 conn(0x5569f1c31200 msgr2=0x5569f1c36ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:33.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.635+0000 7f7c03e00640 0 client.26399 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:10:33.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.635+0000 7f7c28272740 1 -- 172.21.15.43:0/914434111 shutdown_connections 2024-09-19T05:10:33.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.635+0000 7f7c28272740 1 -- 172.21.15.43:0/914434111 wait complete. 2024-09-19T05:10:33.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.635+0000 7f7c28272740 20 client.26399 trim_cache size 0 max 16384 2024-09-19T05:10:33.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.649+0000 7f7c28272740 1 Processor -- start 2024-09-19T05:10:33.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.649+0000 7f7c28272740 1 -- start start 2024-09-19T05:10:33.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.649+0000 7f7c28272740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f1c34c70 0x5569f1c57c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:33.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.649+0000 7f7c28272740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f1c35590 0x5569f1c58140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:33.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.649+0000 7f7c28272740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f1c3fe50 0x5569f1c3a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:33.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.649+0000 7f7c28272740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7c201af9c0 con 0x5569f1c35590 2024-09-19T05:10:33.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.649+0000 7f7c28272740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7c201b1500 con 0x5569f1c34c70 2024-09-19T05:10:33.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.649+0000 7f7c28272740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7c201b0090 con 0x5569f1c3fe50 2024-09-19T05:10:33.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.649+0000 7f7c17e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f1c35590 0x5569f1c58140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:33.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.650+0000 7f7c17e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f1c35590 0x5569f1c58140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36248/0 (socket says 172.21.15.43:36248) 2024-09-19T05:10:33.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.650+0000 7f7c17e00640 1 -- 172.21.15.43:0/3868622667 learned_addr learned my addr 172.21.15.43:0/3868622667 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:33.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.650+0000 7f7c1ea00640 1 --2- 172.21.15.43:0/3868622667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f1c34c70 0x5569f1c57c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:33.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.650+0000 7f7c1f400640 1 --2- 172.21.15.43:0/3868622667 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f1c3fe50 0x5569f1c3a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:33.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.650+0000 7f7c17e00640 1 -- 172.21.15.43:0/3868622667 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f1c3fe50 msgr2=0x5569f1c3a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:33.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.650+0000 7f7c17e00640 1 --2- 172.21.15.43:0/3868622667 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f1c3fe50 0x5569f1c3a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.650+0000 7f7c17e00640 1 -- 172.21.15.43:0/3868622667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f1c34c70 msgr2=0x5569f1c57c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:33.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.650+0000 7f7c17e00640 1 --2- 172.21.15.43:0/3868622667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f1c34c70 0x5569f1c57c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.650+0000 7f7c17e00640 1 -- 172.21.15.43:0/3868622667 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569f1c2c770 con 0x5569f1c35590 2024-09-19T05:10:33.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.650+0000 7f7c1ea00640 1 --2- 172.21.15.43:0/3868622667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f1c34c70 0x5569f1c57c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:10:33.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.650+0000 7f7c17e00640 1 --2- 172.21.15.43:0/3868622667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f1c35590 0x5569f1c58140 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7c08011010 tx=0x7f7c0800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab9bd1f4577d8b65 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:33.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.651+0000 7f7c0ea00640 1 -- 172.21.15.43:0/3868622667 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c08004220 con 0x5569f1c35590 2024-09-19T05:10:33.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.651+0000 7f7c0ea00640 1 -- 172.21.15.43:0/3868622667 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c080043c0 con 0x5569f1c35590 2024-09-19T05:10:33.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.651+0000 7f7c0ea00640 1 -- 172.21.15.43:0/3868622667 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c08011ce0 con 0x5569f1c35590 2024-09-19T05:10:33.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.651+0000 7f7c28272740 1 -- 172.21.15.43:0/3868622667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f1c35590 msgr2=0x5569f1c58140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:33.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.651+0000 7f7c28272740 1 --2- 172.21.15.43:0/3868622667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f1c35590 0x5569f1c58140 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7c08011010 tx=0x7f7c0800b390 comp rx=0 tx=0).stop 2024-09-19T05:10:33.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.651+0000 7f7c28272740 1 -- 172.21.15.43:0/3868622667 shutdown_connections 2024-09-19T05:10:33.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.651+0000 7f7c28272740 1 --2- 172.21.15.43:0/3868622667 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f1c3fe50 0x5569f1c3a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.651+0000 7f7c28272740 1 --2- 172.21.15.43:0/3868622667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f1c35590 0x5569f1c58140 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.651+0000 7f7c28272740 1 --2- 172.21.15.43:0/3868622667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f1c34c70 0x5569f1c57c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.651+0000 7f7c28272740 1 -- 172.21.15.43:0/3868622667 >> 172.21.15.43:0/3868622667 conn(0x5569f1c31200 msgr2=0x5569f1c3ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:33.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.651+0000 7f7c28272740 1 -- 172.21.15.43:0/3868622667 shutdown_connections 2024-09-19T05:10:33.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.651+0000 7f7c28272740 1 -- 172.21.15.43:0/3868622667 wait complete. 2024-09-19T05:10:33.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.652+0000 7f7c28272740 1 -- start start 2024-09-19T05:10:33.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.652+0000 7f7c28272740 1 Processor -- start 2024-09-19T05:10:33.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.652+0000 7f7c28272740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f1c34c70 0x5569f1c39e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:33.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.652+0000 7f7c28272740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f1c35590 0x5569f1c3a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:33.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.653+0000 7f7c28272740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f1c3fe50 0x5569f1c3a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:33.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.653+0000 7f7c28272740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7c201b0e30 con 0x5569f1c34c70 2024-09-19T05:10:33.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.653+0000 7f7c28272740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7c201b0760 con 0x5569f1c3fe50 2024-09-19T05:10:33.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.653+0000 7f7c28272740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7c201ad9a0 con 0x5569f1c35590 2024-09-19T05:10:33.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.653+0000 7f7c28272740 10 client.0 _ll_register_callbacks cb 0x7f7c100056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:33.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.653+0000 7f7c17e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f1c35590 0x5569f1c3a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:33.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.653+0000 7f7c1ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f1c34c70 0x5569f1c39e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:33.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.653+0000 7f7c17e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f1c35590 0x5569f1c3a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:41980/0 (socket says 172.21.15.43:41980) 2024-09-19T05:10:33.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.653+0000 7f7c17e00640 1 -- 172.21.15.43:0/2659300070 learned_addr learned my addr 172.21.15.43:0/2659300070 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:33.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.653+0000 7f7c1f400640 1 --2- 172.21.15.43:0/2659300070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f1c3fe50 0x5569f1c3a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:33.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.653+0000 7f7c1ea00640 1 -- 172.21.15.43:0/2659300070 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f1c35590 msgr2=0x5569f1c3a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:33.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.653+0000 7f7c1ea00640 1 --2- 172.21.15.43:0/2659300070 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f1c35590 0x5569f1c3a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.653+0000 7f7c1ea00640 1 -- 172.21.15.43:0/2659300070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f1c3fe50 msgr2=0x5569f1c3a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:33.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.653+0000 7f7c1ea00640 1 --2- 172.21.15.43:0/2659300070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f1c3fe50 0x5569f1c3a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.653+0000 7f7c1ea00640 1 -- 172.21.15.43:0/2659300070 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569f1cb9c20 con 0x5569f1c34c70 2024-09-19T05:10:33.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.653+0000 7f7c1f400640 1 --2- 172.21.15.43:0/2659300070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f1c3fe50 0x5569f1c3a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:10:33.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.654+0000 7f7c1ea00640 1 --2- 172.21.15.43:0/2659300070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f1c34c70 0x5569f1c39e60 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7c100cce40 tx=0x7f7c100048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e893959340373631 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:33.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.654+0000 7f7c03e00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:10:33.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.654+0000 7f7c03e00640 1 -- 172.21.15.43:0/2659300070 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c10020070 con 0x5569f1c34c70 2024-09-19T05:10:33.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.654+0000 7f7c03e00640 1 -- 172.21.15.43:0/2659300070 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f7c100cd010 con 0x5569f1c34c70 2024-09-19T05:10:33.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.654+0000 7f7c28272740 1 -- 172.21.15.43:0/2659300070 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c080043c0 con 0x5569f1c34c70 2024-09-19T05:10:33.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.654+0000 7f7c03e00640 1 -- 172.21.15.43:0/2659300070 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7c10098040 con 0x5569f1c34c70 2024-09-19T05:10:33.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.654+0000 7f7c28272740 10 client.16365 fetch_fsmap learned FSMap version 29 2024-09-19T05:10:33.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.654+0000 7f7c28272740 1 -- 172.21.15.43:0/2659300070 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5569f1cbaf20 con 0x5569f1c34c70 2024-09-19T05:10:33.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.655+0000 7f7c03e00640 1 -- 172.21.15.43:0/2659300070 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c080043c0 con 0x5569f1c34c70 2024-09-19T05:10:33.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.655+0000 7f7c03e00640 1 -- 172.21.15.43:0/2659300070 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7c100977b0 con 0x5569f1c34c70 2024-09-19T05:10:33.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.655+0000 7f7c28272740 10 client.16365 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:10:33.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.655+0000 7f7c28272740 10 client.16365 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:10:33.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.655+0000 7f7c28272740 20 client.16365 populate_metadata read hostname 'smithi043' 2024-09-19T05:10:33.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.655+0000 7f7c28272740 1 --2- 172.21.15.43:0/2659300070 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5569f1cbb640 0x5569f1cbda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:33.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.655+0000 7f7c28272740 4 client.16365 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:10:33.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.655+0000 7f7c28272740 1 -- 172.21.15.43:0/2659300070 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5569f1ba2e60 con 0x5569f1cbb640 2024-09-19T05:10:33.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.655+0000 7f7c17e00640 1 --2- 172.21.15.43:0/2659300070 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5569f1cbb640 0x5569f1cbda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:33.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.656+0000 7f7c17e00640 1 --2- 172.21.15.43:0/2659300070 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5569f1cbb640 0x5569f1cbda60 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1c6544780784a6c server_cookie=22d25b1a356d4a88 in_seq=0 out_seq=0 2024-09-19T05:10:33.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.656+0000 7f7c03e00640 10 client.16365 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:10:33.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.656+0000 7f7c03e00640 1 -- 172.21.15.43:0/2659300070 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5569f1ba2e60 con 0x5569f1cbb640 2024-09-19T05:10:33.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.656+0000 7f7c03e00640 10 client.16365 handle_command_reply: tid=0 2024-09-19T05:10:33.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.656+0000 7f7c28272740 1 client.16365 shutdown 2024-09-19T05:10:33.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.656+0000 7f7c28272740 1 -- 172.21.15.43:0/2659300070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f1c34c70 msgr2=0x5569f1c39e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:33.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.656+0000 7f7c28272740 1 --2- 172.21.15.43:0/2659300070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f1c34c70 0x5569f1c39e60 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7c100cce40 tx=0x7f7c100048a0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.656+0000 7f7c28272740 1 -- 172.21.15.43:0/2659300070 shutdown_connections 2024-09-19T05:10:33.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.656+0000 7f7c28272740 1 --2- 172.21.15.43:0/2659300070 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5569f1cbb640 0x5569f1cbda60 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.656+0000 7f7c28272740 1 --2- 172.21.15.43:0/2659300070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f1c3fe50 0x5569f1c3a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.656+0000 7f7c28272740 1 --2- 172.21.15.43:0/2659300070 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f1c35590 0x5569f1c3a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.656+0000 7f7c28272740 1 --2- 172.21.15.43:0/2659300070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f1c34c70 0x5569f1c39e60 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.656+0000 7f7c28272740 1 -- 172.21.15.43:0/2659300070 >> 172.21.15.43:0/2659300070 conn(0x5569f1c31200 msgr2=0x5569f1c366c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:33.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.656+0000 7f7c03e00640 0 client.16365 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:10:33.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.656+0000 7f7c28272740 1 -- 172.21.15.43:0/2659300070 shutdown_connections 2024-09-19T05:10:33.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.657+0000 7f7c28272740 1 -- 172.21.15.43:0/2659300070 wait complete. 2024-09-19T05:10:33.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.657+0000 7f7c28272740 20 client.16365 trim_cache size 0 max 16384 2024-09-19T05:10:33.666 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:10:33.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.657+0000 7f7c25600640 1 -- 172.21.15.43:0/4115573891 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f7c040a3430 msgr2=0x7f7c040a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:33.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.657+0000 7f7c25600640 1 --2- 172.21.15.43:0/4115573891 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f7c040a3430 0x7f7c040a58f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7c18006010 tx=0x7f7c18005d80 comp rx=0 tx=0).stop 2024-09-19T05:10:33.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.657+0000 7f7c25600640 1 -- 172.21.15.43:0/4115573891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c20074620 msgr2=0x7f7c2019a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:33.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.657+0000 7f7c25600640 1 --2- 172.21.15.43:0/4115573891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c20074620 0x7f7c2019a630 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7c1000d770 tx=0x7f7c1000dc30 comp rx=0 tx=0).stop 2024-09-19T05:10:33.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.657+0000 7f7c25600640 1 -- 172.21.15.43:0/4115573891 shutdown_connections 2024-09-19T05:10:33.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.657+0000 7f7c25600640 1 --2- 172.21.15.43:0/4115573891 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f7c040a3430 0x7f7c040a58f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.657+0000 7f7c25600640 1 --2- 172.21.15.43:0/4115573891 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c2010e3c0 0x7f7c2019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.657+0000 7f7c25600640 1 --2- 172.21.15.43:0/4115573891 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c20074f40 0x7f7c2019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.657+0000 7f7c25600640 1 --2- 172.21.15.43:0/4115573891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c20074620 0x7f7c2019a630 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:33.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.657+0000 7f7c25600640 1 -- 172.21.15.43:0/4115573891 >> 172.21.15.43:0/4115573891 conn(0x7f7c200fd8c0 msgr2=0x7f7c200fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:33.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.658+0000 7f7c25600640 1 -- 172.21.15.43:0/4115573891 shutdown_connections 2024-09-19T05:10:33.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:33.658+0000 7f7c25600640 1 -- 172.21.15.43:0/4115573891 wait complete. 2024-09-19T05:10:34.143 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:33 smithi043 ceph-mon[30800]: pgmap v180: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-19T05:10:34.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:33 smithi151 ceph-mon[35425]: pgmap v180: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-19T05:10:34.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:33 smithi096 ceph-mon[35085]: pgmap v180: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-19T05:10:35.533 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T05:10:35.534 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T05:10:35.852 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:35 smithi043 ceph-mon[30800]: pgmap v181: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:10:35.925 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:10:36.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:35 smithi151 ceph-mon[35425]: pgmap v181: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:10:36.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:35 smithi096 ceph-mon[35085]: pgmap v181: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:10:36.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.498+0000 7f5a5a400640 1 -- 172.21.15.43:0/1656811082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a54076720 msgr2=0x7f5a54076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:36.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.498+0000 7f5a5a400640 1 --2- 172.21.15.43:0/1656811082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a54076720 0x7f5a54076b00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5a44009860 tx=0x7f5a4402a340 comp rx=0 tx=0).stop 2024-09-19T05:10:36.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.498+0000 7f5a5a400640 1 -- 172.21.15.43:0/1656811082 shutdown_connections 2024-09-19T05:10:36.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.498+0000 7f5a5a400640 1 --2- 172.21.15.43:0/1656811082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a54112a30 0x7f5a54114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.498+0000 7f5a5a400640 1 --2- 172.21.15.43:0/1656811082 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a54077040 0x7f5a541124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.498+0000 7f5a5a400640 1 --2- 172.21.15.43:0/1656811082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a54076720 0x7f5a54076b00 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.498+0000 7f5a5a400640 1 -- 172.21.15.43:0/1656811082 >> 172.21.15.43:0/1656811082 conn(0x7f5a540ffa90 msgr2=0x7f5a54101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:36.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.498+0000 7f5a5a400640 1 -- 172.21.15.43:0/1656811082 shutdown_connections 2024-09-19T05:10:36.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.498+0000 7f5a5a400640 1 -- 172.21.15.43:0/1656811082 wait complete. 2024-09-19T05:10:36.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.500+0000 7f5a5a400640 1 Processor -- start 2024-09-19T05:10:36.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.500+0000 7f5a5a400640 1 -- start start 2024-09-19T05:10:36.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.500+0000 7f5a5a400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a54076720 0x7f5a541a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:36.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.500+0000 7f5a5a400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a54077040 0x7f5a541a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:36.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.500+0000 7f5a5a400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a54112a30 0x7f5a5419e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:36.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.500+0000 7f5a5a400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5a541185a0 con 0x7f5a54076720 2024-09-19T05:10:36.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.500+0000 7f5a5a400640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5a54118420 con 0x7f5a54077040 2024-09-19T05:10:36.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.500+0000 7f5a5a400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5a54118720 con 0x7f5a54112a30 2024-09-19T05:10:36.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.501+0000 7f5a53400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a54076720 0x7f5a541a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:36.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.501+0000 7f5a53400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a54076720 0x7f5a541a4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46890/0 (socket says 172.21.15.43:46890) 2024-09-19T05:10:36.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.501+0000 7f5a53400640 1 -- 172.21.15.43:0/256094024 learned_addr learned my addr 172.21.15.43:0/256094024 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:36.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.501+0000 7f5a4be00640 1 --2- 172.21.15.43:0/256094024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a54077040 0x7f5a541a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:36.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.501+0000 7f5a53e00640 1 --2- 172.21.15.43:0/256094024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a54112a30 0x7f5a5419e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:36.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.501+0000 7f5a53400640 1 -- 172.21.15.43:0/256094024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a54112a30 msgr2=0x7f5a5419e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:36.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.501+0000 7f5a53400640 1 --2- 172.21.15.43:0/256094024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a54112a30 0x7f5a5419e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.501+0000 7f5a53400640 1 -- 172.21.15.43:0/256094024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a54077040 msgr2=0x7f5a541a4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:36.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.501+0000 7f5a53400640 1 --2- 172.21.15.43:0/256094024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a54077040 0x7f5a541a4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.501+0000 7f5a53400640 1 -- 172.21.15.43:0/256094024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a5419f000 con 0x7f5a54076720 2024-09-19T05:10:36.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.501+0000 7f5a4be00640 1 --2- 172.21.15.43:0/256094024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a54077040 0x7f5a541a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:10:36.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.502+0000 7f5a53400640 1 --2- 172.21.15.43:0/256094024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a54076720 0x7f5a541a4710 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5a44035040 tx=0x7f5a44033b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:36.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.502+0000 7f5a50c00640 1 -- 172.21.15.43:0/256094024 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a44037040 con 0x7f5a54076720 2024-09-19T05:10:36.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.502+0000 7f5a50c00640 1 -- 172.21.15.43:0/256094024 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a44040070 con 0x7f5a54076720 2024-09-19T05:10:36.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.502+0000 7f5a50c00640 1 -- 172.21.15.43:0/256094024 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a440039b0 con 0x7f5a54076720 2024-09-19T05:10:36.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.502+0000 7f5a5a400640 1 -- 172.21.15.43:0/256094024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a5419f290 con 0x7f5a54076720 2024-09-19T05:10:36.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.502+0000 7f5a5a400640 1 -- 172.21.15.43:0/256094024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a541ab5e0 con 0x7f5a54076720 2024-09-19T05:10:36.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.505+0000 7f5a5ce0b740 1 Processor -- start 2024-09-19T05:10:36.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.505+0000 7f5a50c00640 1 -- 172.21.15.43:0/256094024 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f5a440305e0 con 0x7f5a54076720 2024-09-19T05:10:36.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.505+0000 7f5a5ce0b740 1 -- start start 2024-09-19T05:10:36.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.505+0000 7f5a5ce0b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a43658ac0 0x558a43658ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:36.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.506+0000 7f5a5ce0b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a436593e0 0x558a43663760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:36.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.506+0000 7f5a5ce0b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a43663ca0 0x558a43666090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:36.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.506+0000 7f5a5ce0b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558a435bf290 con 0x558a436593e0 2024-09-19T05:10:36.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.506+0000 7f5a5ce0b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558a43566b40 con 0x558a43663ca0 2024-09-19T05:10:36.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.506+0000 7f5a5ce0b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558a435bb250 con 0x558a43658ac0 2024-09-19T05:10:36.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.506+0000 7f5a53e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a43658ac0 0x558a43658ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:36.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.506+0000 7f5a53400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a436593e0 0x558a43663760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:36.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.506+0000 7f5a53e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a43658ac0 0x558a43658ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:40704/0 (socket says 172.21.15.43:40704) 2024-09-19T05:10:36.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.506+0000 7f5a53e00640 1 -- 172.21.15.43:0/2863897763 learned_addr learned my addr 172.21.15.43:0/2863897763 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:36.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.506+0000 7f5a4be00640 1 --2- 172.21.15.43:0/2863897763 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a43663ca0 0x558a43666090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:36.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.506+0000 7f5a53e00640 1 -- 172.21.15.43:0/2863897763 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a43663ca0 msgr2=0x558a43666090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:36.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.506+0000 7f5a53e00640 1 --2- 172.21.15.43:0/2863897763 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a43663ca0 0x558a43666090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.506+0000 7f5a53e00640 1 -- 172.21.15.43:0/2863897763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a436593e0 msgr2=0x558a43663760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:36.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.506+0000 7f5a53e00640 1 --2- 172.21.15.43:0/2863897763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a436593e0 0x558a43663760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.506+0000 7f5a53e00640 1 -- 172.21.15.43:0/2863897763 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a433306a0 con 0x558a43658ac0 2024-09-19T05:10:36.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.507+0000 7f5a53e00640 1 --2- 172.21.15.43:0/2863897763 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a43658ac0 0x558a43658ea0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c009fc0 tx=0x7f5a4c0095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d48e1133a3e1bde7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:36.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.507+0000 7f5a48c00640 1 -- 172.21.15.43:0/2863897763 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c004b20 con 0x558a43658ac0 2024-09-19T05:10:36.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.507+0000 7f5a50c00640 1 --2- 172.21.15.43:0/256094024 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f5a380a3430 0x7f5a380a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:36.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.507+0000 7f5a48c00640 1 -- 172.21.15.43:0/2863897763 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a4c004cc0 con 0x558a43658ac0 2024-09-19T05:10:36.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.507+0000 7f5a48c00640 1 -- 172.21.15.43:0/2863897763 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c0055e0 con 0x558a43658ac0 2024-09-19T05:10:36.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.507+0000 7f5a50c00640 1 -- 172.21.15.43:0/256094024 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f5a440efbf0 con 0x7f5a54076720 2024-09-19T05:10:36.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.507+0000 7f5a53400640 1 --2- 172.21.15.43:0/256094024 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f5a380a3430 0x7f5a380a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:36.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.508+0000 7f5a5ce0b740 1 -- 172.21.15.43:0/2863897763 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a43658ac0 msgr2=0x558a43658ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:36.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.508+0000 7f5a5ce0b740 1 --2- 172.21.15.43:0/2863897763 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a43658ac0 0x558a43658ea0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c009fc0 tx=0x7f5a4c0095c0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.508+0000 7f5a53400640 1 --2- 172.21.15.43:0/256094024 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f5a380a3430 0x7f5a380a58f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x558a43655eb0 tx=0x7f5a440f36a0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:36.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.508+0000 7f5a5ce0b740 1 -- 172.21.15.43:0/2863897763 shutdown_connections 2024-09-19T05:10:36.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.508+0000 7f5a5ce0b740 1 --2- 172.21.15.43:0/2863897763 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a43663ca0 0x558a43666090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.508+0000 7f5a5ce0b740 1 --2- 172.21.15.43:0/2863897763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a436593e0 0x558a43663760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.508+0000 7f5a5ce0b740 1 --2- 172.21.15.43:0/2863897763 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a43658ac0 0x558a43658ea0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.508+0000 7f5a5ce0b740 1 -- 172.21.15.43:0/2863897763 >> 172.21.15.43:0/2863897763 conn(0x558a43655050 msgr2=0x558a43655430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:36.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.508+0000 7f5a5ce0b740 1 -- 172.21.15.43:0/2863897763 shutdown_connections 2024-09-19T05:10:36.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.508+0000 7f5a5ce0b740 1 -- 172.21.15.43:0/2863897763 wait complete. 2024-09-19T05:10:36.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.509+0000 7f5a5ce0b740 1 -- start start 2024-09-19T05:10:36.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.509+0000 7f5a5ce0b740 1 Processor -- start 2024-09-19T05:10:36.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.509+0000 7f5a5ce0b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a43658ac0 0x558a43663590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:36.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.509+0000 7f5a5ce0b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a436593e0 0x558a43660840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:36.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.509+0000 7f5a5ce0b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a43663ca0 0x558a43660d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:36.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.509+0000 7f5a5ce0b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558a43666910 con 0x558a43658ac0 2024-09-19T05:10:36.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.509+0000 7f5a5ce0b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558a43579fa0 con 0x558a436593e0 2024-09-19T05:10:36.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.509+0000 7f5a5ce0b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558a43666a90 con 0x558a43663ca0 2024-09-19T05:10:36.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.509+0000 7f5a5ce0b740 10 client.0 _ll_register_callbacks cb 0x558a432cdba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:36.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.510+0000 7f5a53e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a43658ac0 0x558a43663590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:36.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.510+0000 7f5a53e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a43658ac0 0x558a43663590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46912/0 (socket says 172.21.15.43:46912) 2024-09-19T05:10:36.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.510+0000 7f5a53e00640 1 -- 172.21.15.43:0/1456925798 learned_addr learned my addr 172.21.15.43:0/1456925798 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:36.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.510+0000 7f5a4be00640 1 --2- 172.21.15.43:0/1456925798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a436593e0 0x558a43660840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:36.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.510+0000 7f5a53e00640 1 --2- 172.21.15.43:0/1456925798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a43663ca0 0x558a43660d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:36.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.510+0000 7f5a4be00640 1 -- 172.21.15.43:0/1456925798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a43663ca0 msgr2=0x558a43660d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:36.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.510+0000 7f5a4be00640 1 --2- 172.21.15.43:0/1456925798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a43663ca0 0x558a43660d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.510+0000 7f5a4be00640 1 -- 172.21.15.43:0/1456925798 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a43658ac0 msgr2=0x558a43663590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:36.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.510+0000 7f5a4be00640 1 --2- 172.21.15.43:0/1456925798 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a43658ac0 0x558a43663590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.510+0000 7f5a4be00640 1 -- 172.21.15.43:0/1456925798 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a43661560 con 0x558a436593e0 2024-09-19T05:10:36.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.511+0000 7f5a4be00640 1 --2- 172.21.15.43:0/1456925798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a436593e0 0x558a43660840 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5a54079370 tx=0x7f5a3c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67836676296b70f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:36.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.511+0000 7f5a41600640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:10:36.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.511+0000 7f5a41600640 1 -- 172.21.15.43:0/1456925798 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c012110 con 0x558a436593e0 2024-09-19T05:10:36.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.511+0000 7f5a5ce0b740 1 -- 172.21.15.43:0/1456925798 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a4c004cc0 con 0x558a436593e0 2024-09-19T05:10:36.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.511+0000 7f5a41600640 1 -- 172.21.15.43:0/1456925798 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f5a3c004050 con 0x558a436593e0 2024-09-19T05:10:36.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.511+0000 7f5a41600640 1 -- 172.21.15.43:0/1456925798 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5a3c0122b0 con 0x558a436593e0 2024-09-19T05:10:36.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.511+0000 7f5a5ce0b740 10 client.26417 fetch_fsmap learned FSMap version 29 2024-09-19T05:10:36.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.511+0000 7f5a5ce0b740 1 -- 172.21.15.43:0/1456925798 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558a436728b0 con 0x558a436593e0 2024-09-19T05:10:36.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.511+0000 7f5a41600640 1 -- 172.21.15.43:0/1456925798 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a4c004cc0 con 0x558a436593e0 2024-09-19T05:10:36.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.512+0000 7f5a41600640 1 -- 172.21.15.43:0/1456925798 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5a3c010430 con 0x558a436593e0 2024-09-19T05:10:36.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.512+0000 7f5a5ce0b740 10 client.26417 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:10:36.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.512+0000 7f5a5ce0b740 10 client.26417 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:10:36.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.512+0000 7f5a5ce0b740 20 client.26417 populate_metadata read hostname 'smithi043' 2024-09-19T05:10:36.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.512+0000 7f5a5ce0b740 1 --2- 172.21.15.43:0/1456925798 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558a43673d10 0x558a43676130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:36.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.512+0000 7f5a5ce0b740 4 client.26417 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:10:36.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.512+0000 7f5a5ce0b740 1 -- 172.21.15.43:0/1456925798 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a435cc800 con 0x558a43673d10 2024-09-19T05:10:36.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.512+0000 7f5a53e00640 1 --2- 172.21.15.43:0/1456925798 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558a43673d10 0x558a43676130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:36.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.513+0000 7f5a53e00640 1 --2- 172.21.15.43:0/1456925798 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558a43673d10 0x558a43676130 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56665ce1db1fde8d server_cookie=742219f80527290e in_seq=0 out_seq=0 2024-09-19T05:10:36.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.513+0000 7f5a41600640 10 client.26417 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:10:36.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.513+0000 7f5a41600640 1 -- 172.21.15.43:0/1456925798 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558a435cc800 con 0x558a43673d10 2024-09-19T05:10:36.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.513+0000 7f5a41600640 10 client.26417 handle_command_reply: tid=0 2024-09-19T05:10:36.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.513+0000 7f5a5ce0b740 1 client.26417 shutdown 2024-09-19T05:10:36.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.514+0000 7f5a5ce0b740 1 -- 172.21.15.43:0/1456925798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a436593e0 msgr2=0x558a43660840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:36.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.514+0000 7f5a5ce0b740 1 --2- 172.21.15.43:0/1456925798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a436593e0 0x558a43660840 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5a54079370 tx=0x7f5a3c00f790 comp rx=0 tx=0).stop 2024-09-19T05:10:36.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.514+0000 7f5a5ce0b740 1 -- 172.21.15.43:0/1456925798 shutdown_connections 2024-09-19T05:10:36.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.514+0000 7f5a5ce0b740 1 --2- 172.21.15.43:0/1456925798 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558a43673d10 0x558a43676130 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.514+0000 7f5a5ce0b740 1 --2- 172.21.15.43:0/1456925798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a43663ca0 0x558a43660d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.514+0000 7f5a5ce0b740 1 --2- 172.21.15.43:0/1456925798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a436593e0 0x558a43660840 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.514+0000 7f5a5ce0b740 1 --2- 172.21.15.43:0/1456925798 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a43658ac0 0x558a43663590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.514+0000 7f5a5ce0b740 1 -- 172.21.15.43:0/1456925798 >> 172.21.15.43:0/1456925798 conn(0x558a43655050 msgr2=0x558a436541c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:36.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.514+0000 7f5a41600640 0 client.26417 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:10:36.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.514+0000 7f5a5ce0b740 1 -- 172.21.15.43:0/1456925798 shutdown_connections 2024-09-19T05:10:36.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.514+0000 7f5a5ce0b740 1 -- 172.21.15.43:0/1456925798 wait complete. 2024-09-19T05:10:36.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.514+0000 7f5a5ce0b740 20 client.26417 trim_cache size 0 max 16384 2024-09-19T05:10:36.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.532+0000 7f5a5ce0b740 1 Processor -- start 2024-09-19T05:10:36.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.532+0000 7f5a5ce0b740 1 -- start start 2024-09-19T05:10:36.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.533+0000 7f5a5ce0b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a43658ac0 0x558a4367ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:36.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.533+0000 7f5a5ce0b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a43663ca0 0x558a4367bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:36.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.533+0000 7f5a5ce0b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a4365b5e0 0x558a4365ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:36.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.533+0000 7f5a5ce0b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5a541b3ef0 con 0x558a43663ca0 2024-09-19T05:10:36.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.533+0000 7f5a5ce0b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5a541b5a30 con 0x558a43658ac0 2024-09-19T05:10:36.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.533+0000 7f5a5ce0b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5a541b45c0 con 0x558a4365b5e0 2024-09-19T05:10:36.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.533+0000 7f5a4be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a43663ca0 0x558a4367bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:36.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.533+0000 7f5a4be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a43663ca0 0x558a4367bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46924/0 (socket says 172.21.15.43:46924) 2024-09-19T05:10:36.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.533+0000 7f5a4be00640 1 -- 172.21.15.43:0/2698185589 learned_addr learned my addr 172.21.15.43:0/2698185589 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:36.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.533+0000 7f5a53e00640 1 --2- 172.21.15.43:0/2698185589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a43658ac0 0x558a4367ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:36.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.533+0000 7f5a53e00640 1 --2- 172.21.15.43:0/2698185589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a4365b5e0 0x558a4365ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:36.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.533+0000 7f5a4be00640 1 -- 172.21.15.43:0/2698185589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a4365b5e0 msgr2=0x558a4365ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:36.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.533+0000 7f5a4be00640 1 --2- 172.21.15.43:0/2698185589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a4365b5e0 0x558a4365ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.533+0000 7f5a4be00640 1 -- 172.21.15.43:0/2698185589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a43658ac0 msgr2=0x558a4367ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:36.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.533+0000 7f5a4be00640 1 --2- 172.21.15.43:0/2698185589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a43658ac0 0x558a4367ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.534+0000 7f5a4be00640 1 -- 172.21.15.43:0/2698185589 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a4365edd0 con 0x558a43663ca0 2024-09-19T05:10:36.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.534+0000 7f5a53e00640 1 --2- 172.21.15.43:0/2698185589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a43658ac0 0x558a4367ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:10:36.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.534+0000 7f5a4be00640 1 --2- 172.21.15.43:0/2698185589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a43663ca0 0x558a4367bf40 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c010a40 tx=0x7f5a3c00fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8616486fa98453d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:36.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.534+0000 7f5a43e00640 1 -- 172.21.15.43:0/2698185589 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c029040 con 0x558a43663ca0 2024-09-19T05:10:36.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.534+0000 7f5a43e00640 1 -- 172.21.15.43:0/2698185589 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a3c0055f0 con 0x558a43663ca0 2024-09-19T05:10:36.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.534+0000 7f5a43e00640 1 -- 172.21.15.43:0/2698185589 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c003850 con 0x558a43663ca0 2024-09-19T05:10:36.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.535+0000 7f5a5ce0b740 1 -- 172.21.15.43:0/2698185589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a43663ca0 msgr2=0x558a4367bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:36.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.535+0000 7f5a5ce0b740 1 --2- 172.21.15.43:0/2698185589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a43663ca0 0x558a4367bf40 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c010a40 tx=0x7f5a3c00fc90 comp rx=0 tx=0).stop 2024-09-19T05:10:36.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.535+0000 7f5a5ce0b740 1 -- 172.21.15.43:0/2698185589 shutdown_connections 2024-09-19T05:10:36.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.535+0000 7f5a5ce0b740 1 --2- 172.21.15.43:0/2698185589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a4365b5e0 0x558a4365ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.535+0000 7f5a5ce0b740 1 --2- 172.21.15.43:0/2698185589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a43663ca0 0x558a4367bf40 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.535+0000 7f5a5ce0b740 1 --2- 172.21.15.43:0/2698185589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a43658ac0 0x558a4367ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.535+0000 7f5a5ce0b740 1 -- 172.21.15.43:0/2698185589 >> 172.21.15.43:0/2698185589 conn(0x558a43655050 msgr2=0x558a4365f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:36.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.535+0000 7f5a5ce0b740 1 -- 172.21.15.43:0/2698185589 shutdown_connections 2024-09-19T05:10:36.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.535+0000 7f5a5ce0b740 1 -- 172.21.15.43:0/2698185589 wait complete. 2024-09-19T05:10:36.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.535+0000 7f5a5ce0b740 1 -- start start 2024-09-19T05:10:36.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.536+0000 7f5a5ce0b740 1 Processor -- start 2024-09-19T05:10:36.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.536+0000 7f5a5ce0b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a43658ac0 0x558a4366bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:36.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.536+0000 7f5a5ce0b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a4365b5e0 0x558a4366c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:36.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.536+0000 7f5a5ce0b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a43663ca0 0x558a43668ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:36.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.536+0000 7f5a5ce0b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5a541b5360 con 0x558a4365b5e0 2024-09-19T05:10:36.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.536+0000 7f5a5ce0b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5a541b4c90 con 0x558a43663ca0 2024-09-19T05:10:36.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.536+0000 7f5a5ce0b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5a541b1ed0 con 0x558a43658ac0 2024-09-19T05:10:36.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.536+0000 7f5a5ce0b740 10 client.0 _ll_register_callbacks cb 0x7f5a3c009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:36.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.536+0000 7f5a4be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a4365b5e0 0x558a4366c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:36.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.537+0000 7f5a4be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a4365b5e0 0x558a4366c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46934/0 (socket says 172.21.15.43:46934) 2024-09-19T05:10:36.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.537+0000 7f5a4be00640 1 -- 172.21.15.43:0/1457185634 learned_addr learned my addr 172.21.15.43:0/1457185634 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:36.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.537+0000 7f5a53e00640 1 --2- 172.21.15.43:0/1457185634 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a43658ac0 0x558a4366bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:36.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.537+0000 7f5a53e00640 1 --2- 172.21.15.43:0/1457185634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a43663ca0 0x558a43668ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:36.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.537+0000 7f5a4be00640 1 -- 172.21.15.43:0/1457185634 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a43658ac0 msgr2=0x558a4366bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:36.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.537+0000 7f5a4be00640 1 --2- 172.21.15.43:0/1457185634 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a43658ac0 0x558a4366bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.537+0000 7f5a4be00640 1 -- 172.21.15.43:0/1457185634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a43663ca0 msgr2=0x558a43668ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:36.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.537+0000 7f5a4be00640 1 --2- 172.21.15.43:0/1457185634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a43663ca0 0x558a43668ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.537+0000 7f5a4be00640 1 -- 172.21.15.43:0/1457185634 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a43669220 con 0x558a4365b5e0 2024-09-19T05:10:36.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.537+0000 7f5a4be00640 1 --2- 172.21.15.43:0/1457185634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a4365b5e0 0x558a4366c250 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c000c00 tx=0x7f5a3c009fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e8dd418688a46c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:36.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.537+0000 7f5a41600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:10:36.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.538+0000 7f5a41600640 1 -- 172.21.15.43:0/1457185634 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c005320 con 0x558a4365b5e0 2024-09-19T05:10:36.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.538+0000 7f5a5ce0b740 1 -- 172.21.15.43:0/1457185634 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a3c0055f0 con 0x558a4365b5e0 2024-09-19T05:10:36.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.538+0000 7f5a41600640 1 -- 172.21.15.43:0/1457185634 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f5a3c0035f0 con 0x558a4365b5e0 2024-09-19T05:10:36.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.538+0000 7f5a41600640 1 -- 172.21.15.43:0/1457185634 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5a3c03d050 con 0x558a4365b5e0 2024-09-19T05:10:36.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.538+0000 7f5a5ce0b740 10 client.16383 fetch_fsmap learned FSMap version 29 2024-09-19T05:10:36.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.538+0000 7f5a5ce0b740 1 -- 172.21.15.43:0/1457185634 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558a436df220 con 0x558a4365b5e0 2024-09-19T05:10:36.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.538+0000 7f5a41600640 1 -- 172.21.15.43:0/1457185634 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a3c0055f0 con 0x558a4365b5e0 2024-09-19T05:10:36.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.538+0000 7f5a41600640 1 -- 172.21.15.43:0/1457185634 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5a3c025710 con 0x558a4365b5e0 2024-09-19T05:10:36.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.538+0000 7f5a5ce0b740 10 client.16383 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:10:36.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.538+0000 7f5a5ce0b740 10 client.16383 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:10:36.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.538+0000 7f5a5ce0b740 20 client.16383 populate_metadata read hostname 'smithi043' 2024-09-19T05:10:36.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.539+0000 7f5a5ce0b740 1 --2- 172.21.15.43:0/1457185634 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558a436df910 0x558a436e1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:36.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.539+0000 7f5a5ce0b740 4 client.16383 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T05:10:36.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.539+0000 7f5a5ce0b740 1 -- 172.21.15.43:0/1457185634 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558a435c6d10 con 0x558a436df910 2024-09-19T05:10:36.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.539+0000 7f5a53e00640 1 --2- 172.21.15.43:0/1457185634 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558a436df910 0x558a436e1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:36.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.539+0000 7f5a53e00640 1 --2- 172.21.15.43:0/1457185634 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558a436df910 0x558a436e1d30 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef955a20ae4e76c8 server_cookie=8d8a027e092d8f26 in_seq=0 out_seq=0 2024-09-19T05:10:36.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.539+0000 7f5a41600640 10 client.16383 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:10:36.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.540+0000 7f5a41600640 1 -- 172.21.15.43:0/1457185634 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558a435c6d10 con 0x558a436df910 2024-09-19T05:10:36.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.540+0000 7f5a41600640 10 client.16383 handle_command_reply: tid=0 2024-09-19T05:10:36.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.540+0000 7f5a5ce0b740 1 client.16383 shutdown 2024-09-19T05:10:36.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.541+0000 7f5a5ce0b740 1 -- 172.21.15.43:0/1457185634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a4365b5e0 msgr2=0x558a4366c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:36.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.541+0000 7f5a5ce0b740 1 --2- 172.21.15.43:0/1457185634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a4365b5e0 0x558a4366c250 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c000c00 tx=0x7f5a3c009fa0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.541+0000 7f5a5ce0b740 1 -- 172.21.15.43:0/1457185634 shutdown_connections 2024-09-19T05:10:36.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.541+0000 7f5a5ce0b740 1 --2- 172.21.15.43:0/1457185634 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x558a436df910 0x558a436e1d30 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.541+0000 7f5a5ce0b740 1 --2- 172.21.15.43:0/1457185634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a43663ca0 0x558a43668ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.541+0000 7f5a5ce0b740 1 --2- 172.21.15.43:0/1457185634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558a4365b5e0 0x558a4366c250 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.541+0000 7f5a5ce0b740 1 --2- 172.21.15.43:0/1457185634 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558a43658ac0 0x558a4366bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.541+0000 7f5a5ce0b740 1 -- 172.21.15.43:0/1457185634 >> 172.21.15.43:0/1457185634 conn(0x558a43655050 msgr2=0x558a4365f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:36.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.541+0000 7f5a41600640 0 client.16383 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:10:36.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.541+0000 7f5a5ce0b740 1 -- 172.21.15.43:0/1457185634 shutdown_connections 2024-09-19T05:10:36.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.541+0000 7f5a5ce0b740 1 -- 172.21.15.43:0/1457185634 wait complete. 2024-09-19T05:10:36.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.541+0000 7f5a5ce0b740 20 client.16383 trim_cache size 0 max 16384 2024-09-19T05:10:36.546 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:10:36.546 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-09-19T05:10:36.546 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "84e32b74-df41-4359-976f-0f474b0099a4", 2024-09-19T05:10:36.546 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-09-19T05:10:36.546 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:10:36.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.542+0000 7f5a5a400640 1 -- 172.21.15.43:0/256094024 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f5a380a3430 msgr2=0x7f5a380a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:36.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.542+0000 7f5a5a400640 1 --2- 172.21.15.43:0/256094024 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f5a380a3430 0x7f5a380a58f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x558a43655eb0 tx=0x7f5a440f36a0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.542+0000 7f5a5a400640 1 -- 172.21.15.43:0/256094024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a54076720 msgr2=0x7f5a541a4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:36.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.542+0000 7f5a5a400640 1 --2- 172.21.15.43:0/256094024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a54076720 0x7f5a541a4710 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5a44035040 tx=0x7f5a44033b60 comp rx=0 tx=0).stop 2024-09-19T05:10:36.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.542+0000 7f5a5a400640 1 -- 172.21.15.43:0/256094024 shutdown_connections 2024-09-19T05:10:36.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.542+0000 7f5a5a400640 1 --2- 172.21.15.43:0/256094024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a54112a30 0x7f5a5419e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.542+0000 7f5a5a400640 1 --2- 172.21.15.43:0/256094024 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f5a380a3430 0x7f5a380a58f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.542+0000 7f5a5a400640 1 --2- 172.21.15.43:0/256094024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a54077040 0x7f5a541a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.542+0000 7f5a5a400640 1 --2- 172.21.15.43:0/256094024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a54076720 0x7f5a541a4710 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:36.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.542+0000 7f5a5a400640 1 -- 172.21.15.43:0/256094024 >> 172.21.15.43:0/256094024 conn(0x7f5a540ffa90 msgr2=0x7f5a54101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:36.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.542+0000 7f5a5a400640 1 -- 172.21.15.43:0/256094024 shutdown_connections 2024-09-19T05:10:36.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:36.542+0000 7f5a5a400640 1 -- 172.21.15.43:0/256094024 wait complete. 2024-09-19T05:10:37.393 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:10:37.772 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:10:38.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:37 smithi043 ceph-mon[30800]: pgmap v182: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-19T05:10:38.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:37 smithi043 ceph-mon[30800]: scrub queued for path: / 2024-09-19T05:10:38.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:37 smithi043 ceph-mon[30800]: scrub summary: idle+waiting paths [/] 2024-09-19T05:10:38.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:37 smithi043 ceph-mon[30800]: scrub summary: active paths [/] 2024-09-19T05:10:38.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:37 smithi043 ceph-mon[30800]: scrub summary: idle+waiting paths [/] 2024-09-19T05:10:38.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:37 smithi151 ceph-mon[35425]: pgmap v182: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-19T05:10:38.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:37 smithi151 ceph-mon[35425]: scrub queued for path: / 2024-09-19T05:10:38.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:37 smithi151 ceph-mon[35425]: scrub summary: idle+waiting paths [/] 2024-09-19T05:10:38.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:37 smithi151 ceph-mon[35425]: scrub summary: active paths [/] 2024-09-19T05:10:38.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:37 smithi151 ceph-mon[35425]: scrub summary: idle+waiting paths [/] 2024-09-19T05:10:38.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:37 smithi096 ceph-mon[35085]: pgmap v182: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-19T05:10:38.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:37 smithi096 ceph-mon[35085]: scrub queued for path: / 2024-09-19T05:10:38.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:37 smithi096 ceph-mon[35085]: scrub summary: idle+waiting paths [/] 2024-09-19T05:10:38.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:37 smithi096 ceph-mon[35085]: scrub summary: active paths [/] 2024-09-19T05:10:38.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:37 smithi096 ceph-mon[35085]: scrub summary: idle+waiting paths [/] 2024-09-19T05:10:38.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.344+0000 7f2a53000640 1 -- 172.21.15.43:0/790930183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a4c10e3c0 msgr2=0x7f2a4c110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:38.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.344+0000 7f2a53000640 1 --2- 172.21.15.43:0/790930183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a4c10e3c0 0x7f2a4c110810 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2a44009ca0 tx=0x7f2a4402a950 comp rx=0 tx=0).stop 2024-09-19T05:10:38.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.344+0000 7f2a53000640 1 -- 172.21.15.43:0/790930183 shutdown_connections 2024-09-19T05:10:38.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.344+0000 7f2a53000640 1 --2- 172.21.15.43:0/790930183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a4c10e3c0 0x7f2a4c110810 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:38.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.344+0000 7f2a53000640 1 --2- 172.21.15.43:0/790930183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a4c074f40 0x7f2a4c10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:38.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.344+0000 7f2a53000640 1 --2- 172.21.15.43:0/790930183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a4c074620 0x7f2a4c074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:38.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.344+0000 7f2a53000640 1 -- 172.21.15.43:0/790930183 >> 172.21.15.43:0/790930183 conn(0x7f2a4c0fd8c0 msgr2=0x7f2a4c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:38.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.344+0000 7f2a53000640 1 -- 172.21.15.43:0/790930183 shutdown_connections 2024-09-19T05:10:38.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.344+0000 7f2a53000640 1 -- 172.21.15.43:0/790930183 wait complete. 2024-09-19T05:10:38.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.345+0000 7f2a53000640 1 Processor -- start 2024-09-19T05:10:38.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.345+0000 7f2a53000640 1 -- start start 2024-09-19T05:10:38.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.345+0000 7f2a53000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a4c074620 0x7f2a4c19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:38.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.345+0000 7f2a53000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a4c074f40 0x7f2a4c19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:38.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.345+0000 7f2a53000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a4c10e3c0 0x7f2a4c19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:38.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.345+0000 7f2a53000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4c114160 con 0x7f2a4c10e3c0 2024-09-19T05:10:38.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.345+0000 7f2a53000640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4c113fe0 con 0x7f2a4c074620 2024-09-19T05:10:38.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.345+0000 7f2a53000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4c1142e0 con 0x7f2a4c074f40 2024-09-19T05:10:38.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.346+0000 7f2a51000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a4c10e3c0 0x7f2a4c19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:38.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.346+0000 7f2a51000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a4c10e3c0 0x7f2a4c19ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46956/0 (socket says 172.21.15.43:46956) 2024-09-19T05:10:38.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.346+0000 7f2a51000640 1 -- 172.21.15.43:0/4207014305 learned_addr learned my addr 172.21.15.43:0/4207014305 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:38.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.346+0000 7f2a43400640 1 --2- 172.21.15.43:0/4207014305 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a4c074f40 0x7f2a4c19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:38.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.346+0000 7f2a4be00640 1 --2- 172.21.15.43:0/4207014305 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a4c074620 0x7f2a4c19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:38.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.346+0000 7f2a51000640 1 -- 172.21.15.43:0/4207014305 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a4c074f40 msgr2=0x7f2a4c19aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:38.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.346+0000 7f2a51000640 1 --2- 172.21.15.43:0/4207014305 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a4c074f40 0x7f2a4c19aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:38.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.346+0000 7f2a51000640 1 -- 172.21.15.43:0/4207014305 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a4c074620 msgr2=0x7f2a4c19a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:38.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.346+0000 7f2a51000640 1 --2- 172.21.15.43:0/4207014305 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a4c074620 0x7f2a4c19a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:38.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.346+0000 7f2a51000640 1 -- 172.21.15.43:0/4207014305 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a4c19f610 con 0x7f2a4c10e3c0 2024-09-19T05:10:38.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.347+0000 7f2a51000640 1 --2- 172.21.15.43:0/4207014305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a4c10e3c0 0x7f2a4c19ef30 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2a44004af0 tx=0x7f2a4402f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:38.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.347+0000 7f2a49600640 1 -- 172.21.15.43:0/4207014305 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a44034040 con 0x7f2a4c10e3c0 2024-09-19T05:10:38.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.347+0000 7f2a49600640 1 -- 172.21.15.43:0/4207014305 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a44030920 con 0x7f2a4c10e3c0 2024-09-19T05:10:38.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.347+0000 7f2a49600640 1 -- 172.21.15.43:0/4207014305 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a44004670 con 0x7f2a4c10e3c0 2024-09-19T05:10:38.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.347+0000 7f2a53000640 1 -- 172.21.15.43:0/4207014305 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a4c19f8a0 con 0x7f2a4c10e3c0 2024-09-19T05:10:38.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.347+0000 7f2a53000640 1 -- 172.21.15.43:0/4207014305 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a4c19fd30 con 0x7f2a4c10e3c0 2024-09-19T05:10:38.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.350+0000 7f2a49600640 1 -- 172.21.15.43:0/4207014305 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f2a44037020 con 0x7f2a4c10e3c0 2024-09-19T05:10:38.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.350+0000 7f2a55a46740 1 Processor -- start 2024-09-19T05:10:38.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.350+0000 7f2a55a46740 1 -- start start 2024-09-19T05:10:38.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.351+0000 7f2a55a46740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac0af18b90 0x55ac0af18f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:38.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.351+0000 7f2a55a46740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac0af194b0 0x55ac0af23830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:38.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.351+0000 7f2a55a46740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ac0af23d70 0x55ac0af26160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:38.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.351+0000 7f2a55a46740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ac0ae7f3e0 con 0x55ac0af23d70 2024-09-19T05:10:38.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.351+0000 7f2a55a46740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ac0ae26d80 con 0x55ac0af18b90 2024-09-19T05:10:38.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.351+0000 7f2a55a46740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ac0ae7b3a0 con 0x55ac0af194b0 2024-09-19T05:10:38.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.351+0000 7f2a4be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac0af194b0 0x55ac0af23830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:38.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.351+0000 7f2a43400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac0af18b90 0x55ac0af18f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:38.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.351+0000 7f2a4be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac0af194b0 0x55ac0af23830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:40752/0 (socket says 172.21.15.43:40752) 2024-09-19T05:10:38.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.351+0000 7f2a4be00640 1 -- 172.21.15.43:0/797534350 learned_addr learned my addr 172.21.15.43:0/797534350 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:38.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.351+0000 7f2a43400640 1 --2- 172.21.15.43:0/797534350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ac0af23d70 0x55ac0af26160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:38.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.352+0000 7f2a4be00640 1 -- 172.21.15.43:0/797534350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac0af18b90 msgr2=0x55ac0af18f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:38.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.352+0000 7f2a4be00640 1 --2- 172.21.15.43:0/797534350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac0af18b90 0x55ac0af18f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:38.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.352+0000 7f2a4be00640 1 -- 172.21.15.43:0/797534350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ac0af23d70 msgr2=0x55ac0af26160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:38.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.352+0000 7f2a4be00640 1 --2- 172.21.15.43:0/797534350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ac0af23d70 0x55ac0af26160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:38.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.352+0000 7f2a4be00640 1 -- 172.21.15.43:0/797534350 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac0ae45ae0 con 0x55ac0af194b0 2024-09-19T05:10:38.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.352+0000 7f2a4be00640 1 --2- 172.21.15.43:0/797534350 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac0af194b0 0x55ac0af23830 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c009d20 tx=0x7f2a3c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3999e27665967de server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:38.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.352+0000 7f2a49600640 1 --2- 172.21.15.43:0/4207014305 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f2a300a3430 0x7f2a300a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:38.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.352+0000 7f2a49600640 1 -- 172.21.15.43:0/4207014305 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f2a440f0900 con 0x7f2a4c10e3c0 2024-09-19T05:10:38.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.352+0000 7f2a41600640 1 -- 172.21.15.43:0/797534350 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c01e070 con 0x55ac0af194b0 2024-09-19T05:10:38.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.353+0000 7f2a41600640 1 -- 172.21.15.43:0/797534350 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a3c019b10 con 0x55ac0af194b0 2024-09-19T05:10:38.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.353+0000 7f2a41600640 1 -- 172.21.15.43:0/797534350 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c00eb80 con 0x55ac0af194b0 2024-09-19T05:10:38.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.353+0000 7f2a43400640 1 --2- 172.21.15.43:0/4207014305 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f2a300a3430 0x7f2a300a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:38.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.353+0000 7f2a55a46740 1 -- 172.21.15.43:0/797534350 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac0af194b0 msgr2=0x55ac0af23830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:38.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.353+0000 7f2a43400640 1 --2- 172.21.15.43:0/4207014305 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f2a300a3430 0x7f2a300a58f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x55ac0af164a0 tx=0x7f2a34005f90 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:38.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.353+0000 7f2a55a46740 1 --2- 172.21.15.43:0/797534350 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac0af194b0 0x55ac0af23830 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c009d20 tx=0x7f2a3c011810 comp rx=0 tx=0).stop 2024-09-19T05:10:38.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.354+0000 7f2a55a46740 1 -- 172.21.15.43:0/797534350 shutdown_connections 2024-09-19T05:10:38.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.354+0000 7f2a55a46740 1 --2- 172.21.15.43:0/797534350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ac0af23d70 0x55ac0af26160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:38.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.354+0000 7f2a55a46740 1 --2- 172.21.15.43:0/797534350 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac0af194b0 0x55ac0af23830 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:38.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.354+0000 7f2a55a46740 1 --2- 172.21.15.43:0/797534350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac0af18b90 0x55ac0af18f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:38.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.354+0000 7f2a55a46740 1 -- 172.21.15.43:0/797534350 >> 172.21.15.43:0/797534350 conn(0x55ac0af15120 msgr2=0x55ac0af15500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:38.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.354+0000 7f2a55a46740 1 -- 172.21.15.43:0/797534350 shutdown_connections 2024-09-19T05:10:38.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.354+0000 7f2a55a46740 1 -- 172.21.15.43:0/797534350 wait complete. 2024-09-19T05:10:38.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.354+0000 7f2a55a46740 1 -- start start 2024-09-19T05:10:38.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.354+0000 7f2a55a46740 1 Processor -- start 2024-09-19T05:10:38.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.355+0000 7f2a55a46740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac0af18b90 0x55ac0af20c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:38.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.355+0000 7f2a55a46740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac0af194b0 0x55ac0af21170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:38.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.355+0000 7f2a55a46740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ac0af23d70 0x55ac0af216b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:38.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.355+0000 7f2a55a46740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ac0af269e0 con 0x55ac0af23d70 2024-09-19T05:10:38.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.355+0000 7f2a55a46740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ac0ae3a0f0 con 0x55ac0af194b0 2024-09-19T05:10:38.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.355+0000 7f2a55a46740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ac0af26b60 con 0x55ac0af18b90 2024-09-19T05:10:38.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.355+0000 7f2a55a46740 10 client.0 _ll_register_callbacks cb 0x55ac0ae2b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:38.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.355+0000 7f2a4be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac0af18b90 0x55ac0af20c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:38.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.355+0000 7f2a4be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac0af18b90 0x55ac0af20c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:40768/0 (socket says 172.21.15.43:40768) 2024-09-19T05:10:38.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.355+0000 7f2a4be00640 1 -- 172.21.15.43:0/2537676829 learned_addr learned my addr 172.21.15.43:0/2537676829 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:38.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.355+0000 7f2a43400640 1 --2- 172.21.15.43:0/2537676829 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac0af194b0 0x55ac0af21170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:38.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.355+0000 7f2a51000640 1 --2- 172.21.15.43:0/2537676829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ac0af23d70 0x55ac0af216b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:38.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.355+0000 7f2a4be00640 1 -- 172.21.15.43:0/2537676829 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac0af194b0 msgr2=0x55ac0af21170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:38.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.355+0000 7f2a4be00640 1 --2- 172.21.15.43:0/2537676829 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac0af194b0 0x55ac0af21170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:38.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.355+0000 7f2a4be00640 1 -- 172.21.15.43:0/2537676829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ac0af23d70 msgr2=0x55ac0af216b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:38.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.355+0000 7f2a4be00640 1 --2- 172.21.15.43:0/2537676829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ac0af23d70 0x55ac0af216b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:38.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.356+0000 7f2a4be00640 1 -- 172.21.15.43:0/2537676829 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac0abf03c0 con 0x55ac0af18b90 2024-09-19T05:10:38.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.356+0000 7f2a4be00640 1 --2- 172.21.15.43:0/2537676829 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac0af18b90 0x55ac0af20c30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c00e720 tx=0x7f2a3c019660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c44b79815d017c20 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:38.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.356+0000 7f2a3a000640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:10:38.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.356+0000 7f2a3a000640 1 -- 172.21.15.43:0/2537676829 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c01e070 con 0x55ac0af18b90 2024-09-19T05:10:38.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.356+0000 7f2a3a000640 1 -- 172.21.15.43:0/2537676829 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f2a3c00ea80 con 0x55ac0af18b90 2024-09-19T05:10:38.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.356+0000 7f2a55a46740 1 -- 172.21.15.43:0/2537676829 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a3c019b10 con 0x55ac0af18b90 2024-09-19T05:10:38.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.356+0000 7f2a3a000640 1 -- 172.21.15.43:0/2537676829 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2a3c031050 con 0x55ac0af18b90 2024-09-19T05:10:38.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.356+0000 7f2a55a46740 10 client.26365 fetch_fsmap learned FSMap version 29 2024-09-19T05:10:38.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.356+0000 7f2a55a46740 1 -- 172.21.15.43:0/2537676829 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ac0af32660 con 0x55ac0af18b90 2024-09-19T05:10:38.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.357+0000 7f2a3a000640 1 -- 172.21.15.43:0/2537676829 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a3c019b10 con 0x55ac0af18b90 2024-09-19T05:10:38.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.357+0000 7f2a3a000640 1 -- 172.21.15.43:0/2537676829 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2a3c0046c0 con 0x55ac0af18b90 2024-09-19T05:10:38.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.357+0000 7f2a55a46740 10 client.26365 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:10:38.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.357+0000 7f2a55a46740 10 client.26365 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:10:38.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.357+0000 7f2a55a46740 20 client.26365 populate_metadata read hostname 'smithi043' 2024-09-19T05:10:38.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.357+0000 7f2a55a46740 1 --2- 172.21.15.43:0/2537676829 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55ac0af33ac0 0x55ac0af35eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:38.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.357+0000 7f2a55a46740 4 client.26365 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:10:38.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.357+0000 7f2a55a46740 1 -- 172.21.15.43:0/2537676829 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac0ae8c950 con 0x55ac0af33ac0 2024-09-19T05:10:38.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.357+0000 7f2a43400640 1 --2- 172.21.15.43:0/2537676829 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55ac0af33ac0 0x55ac0af35eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:38.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.358+0000 7f2a43400640 1 --2- 172.21.15.43:0/2537676829 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55ac0af33ac0 0x55ac0af35eb0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5925ed54ad6a7b9d server_cookie=39303e16b7b8759c in_seq=0 out_seq=0 2024-09-19T05:10:38.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.358+0000 7f2a3a000640 10 client.26365 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:10:38.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.359+0000 7f2a3a000640 1 -- 172.21.15.43:0/2537676829 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ac0ae8c950 con 0x55ac0af33ac0 2024-09-19T05:10:38.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.359+0000 7f2a3a000640 10 client.26365 handle_command_reply: tid=0 2024-09-19T05:10:38.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.359+0000 7f2a55a46740 1 client.26365 shutdown 2024-09-19T05:10:38.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.359+0000 7f2a55a46740 1 -- 172.21.15.43:0/2537676829 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac0af18b90 msgr2=0x55ac0af20c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:38.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.359+0000 7f2a55a46740 1 --2- 172.21.15.43:0/2537676829 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac0af18b90 0x55ac0af20c30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c00e720 tx=0x7f2a3c019660 comp rx=0 tx=0).stop 2024-09-19T05:10:38.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.359+0000 7f2a55a46740 1 -- 172.21.15.43:0/2537676829 shutdown_connections 2024-09-19T05:10:38.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.359+0000 7f2a55a46740 1 --2- 172.21.15.43:0/2537676829 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55ac0af33ac0 0x55ac0af35eb0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:38.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.359+0000 7f2a55a46740 1 --2- 172.21.15.43:0/2537676829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ac0af23d70 0x55ac0af216b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:38.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.359+0000 7f2a3a000640 0 client.26365 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:10:38.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.359+0000 7f2a55a46740 1 --2- 172.21.15.43:0/2537676829 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac0af194b0 0x55ac0af21170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:38.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.359+0000 7f2a55a46740 1 --2- 172.21.15.43:0/2537676829 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac0af18b90 0x55ac0af20c30 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:38.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.359+0000 7f2a55a46740 1 -- 172.21.15.43:0/2537676829 >> 172.21.15.43:0/2537676829 conn(0x55ac0af15120 msgr2=0x55ac0af1ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:38.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.359+0000 7f2a55a46740 1 -- 172.21.15.43:0/2537676829 shutdown_connections 2024-09-19T05:10:38.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.360+0000 7f2a55a46740 1 -- 172.21.15.43:0/2537676829 wait complete. 2024-09-19T05:10:38.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.360+0000 7f2a55a46740 20 client.26365 trim_cache size 0 max 16384 2024-09-19T05:10:38.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.378+0000 7f2a55a46740 1 Processor -- start 2024-09-19T05:10:38.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.378+0000 7f2a55a46740 1 -- start start 2024-09-19T05:10:38.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.378+0000 7f2a55a46740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ac0af18b90 0x55ac0af1b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:38.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.378+0000 7f2a55a46740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac0af194b0 0x55ac0af3b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:38.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.378+0000 7f2a55a46740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac0af23d70 0x55ac0af3bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:38.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.378+0000 7f2a55a46740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4c1afa70 con 0x55ac0af18b90 2024-09-19T05:10:38.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.378+0000 7f2a55a46740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4c1b15b0 con 0x55ac0af23d70 2024-09-19T05:10:38.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.378+0000 7f2a55a46740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4c1b0140 con 0x55ac0af194b0 2024-09-19T05:10:38.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.379+0000 7f2a4be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ac0af18b90 0x55ac0af1b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:38.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.379+0000 7f2a4be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ac0af18b90 0x55ac0af1b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46984/0 (socket says 172.21.15.43:46984) 2024-09-19T05:10:38.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.379+0000 7f2a4be00640 1 -- 172.21.15.43:0/2001806402 learned_addr learned my addr 172.21.15.43:0/2001806402 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:38.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.379+0000 7f2a43400640 1 --2- 172.21.15.43:0/2001806402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac0af194b0 0x55ac0af3b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:38.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.379+0000 7f2a51000640 1 --2- 172.21.15.43:0/2001806402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac0af23d70 0x55ac0af3bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:38.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.379+0000 7f2a4be00640 1 -- 172.21.15.43:0/2001806402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac0af194b0 msgr2=0x55ac0af3b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:38.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.379+0000 7f2a4be00640 1 --2- 172.21.15.43:0/2001806402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac0af194b0 0x55ac0af3b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:38.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.379+0000 7f2a4be00640 1 -- 172.21.15.43:0/2001806402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac0af23d70 msgr2=0x55ac0af3bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:38.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.379+0000 7f2a4be00640 1 --2- 172.21.15.43:0/2001806402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac0af23d70 0x55ac0af3bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:38.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.379+0000 7f2a4be00640 1 -- 172.21.15.43:0/2001806402 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac0af14bc0 con 0x55ac0af18b90 2024-09-19T05:10:38.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.379+0000 7f2a4be00640 1 --2- 172.21.15.43:0/2001806402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ac0af18b90 0x55ac0af1b5f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c005e00 tx=0x7f2a3c011e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28483cf41c132726 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:38.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.380+0000 7f2a40c00640 1 -- 172.21.15.43:0/2001806402 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c015020 con 0x55ac0af18b90 2024-09-19T05:10:38.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.380+0000 7f2a40c00640 1 -- 172.21.15.43:0/2001806402 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a3c028070 con 0x55ac0af18b90 2024-09-19T05:10:38.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.380+0000 7f2a40c00640 1 -- 172.21.15.43:0/2001806402 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c01e040 con 0x55ac0af18b90 2024-09-19T05:10:38.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.380+0000 7f2a55a46740 1 -- 172.21.15.43:0/2001806402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ac0af18b90 msgr2=0x55ac0af1b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:38.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.380+0000 7f2a55a46740 1 --2- 172.21.15.43:0/2001806402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ac0af18b90 0x55ac0af1b5f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c005e00 tx=0x7f2a3c011e40 comp rx=0 tx=0).stop 2024-09-19T05:10:38.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.380+0000 7f2a55a46740 1 -- 172.21.15.43:0/2001806402 shutdown_connections 2024-09-19T05:10:38.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.380+0000 7f2a55a46740 1 --2- 172.21.15.43:0/2001806402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac0af23d70 0x55ac0af3bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:38.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.380+0000 7f2a55a46740 1 --2- 172.21.15.43:0/2001806402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac0af194b0 0x55ac0af3b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:38.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.380+0000 7f2a55a46740 1 --2- 172.21.15.43:0/2001806402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ac0af18b90 0x55ac0af1b5f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:38.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.380+0000 7f2a55a46740 1 -- 172.21.15.43:0/2001806402 >> 172.21.15.43:0/2001806402 conn(0x55ac0af15120 msgr2=0x55ac0af22cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:38.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.380+0000 7f2a55a46740 1 -- 172.21.15.43:0/2001806402 shutdown_connections 2024-09-19T05:10:38.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.381+0000 7f2a55a46740 1 -- 172.21.15.43:0/2001806402 wait complete. 2024-09-19T05:10:38.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.381+0000 7f2a55a46740 1 -- start start 2024-09-19T05:10:38.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.381+0000 7f2a55a46740 1 Processor -- start 2024-09-19T05:10:38.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.381+0000 7f2a55a46740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac0af18b90 0x55ac0af2c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:38.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.381+0000 7f2a55a46740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac0af194b0 0x55ac0af27540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:38.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.381+0000 7f2a55a46740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ac0af23d70 0x55ac0af27a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:38.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.381+0000 7f2a55a46740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4c1b0ee0 con 0x55ac0af23d70 2024-09-19T05:10:38.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.381+0000 7f2a55a46740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4c1b0810 con 0x55ac0af18b90 2024-09-19T05:10:38.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.381+0000 7f2a55a46740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4c1ada50 con 0x55ac0af194b0 2024-09-19T05:10:38.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.382+0000 7f2a55a46740 10 client.0 _ll_register_callbacks cb 0x7f2a34009990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:38.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.382+0000 7f2a51000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ac0af23d70 0x55ac0af27a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:38.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.382+0000 7f2a51000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ac0af23d70 0x55ac0af27a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46998/0 (socket says 172.21.15.43:46998) 2024-09-19T05:10:38.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.382+0000 7f2a51000640 1 -- 172.21.15.43:0/93698292 learned_addr learned my addr 172.21.15.43:0/93698292 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:38.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.382+0000 7f2a43400640 1 --2- 172.21.15.43:0/93698292 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac0af194b0 0x55ac0af27540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:38.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.382+0000 7f2a4be00640 1 --2- 172.21.15.43:0/93698292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac0af18b90 0x55ac0af2c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:38.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.382+0000 7f2a51000640 1 -- 172.21.15.43:0/93698292 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac0af194b0 msgr2=0x55ac0af27540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:38.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.382+0000 7f2a51000640 1 --2- 172.21.15.43:0/93698292 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac0af194b0 0x55ac0af27540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:38.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.382+0000 7f2a51000640 1 -- 172.21.15.43:0/93698292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac0af18b90 msgr2=0x55ac0af2c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:38.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.382+0000 7f2a51000640 1 --2- 172.21.15.43:0/93698292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac0af18b90 0x55ac0af2c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:38.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.382+0000 7f2a51000640 1 -- 172.21.15.43:0/93698292 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac0af27fc0 con 0x55ac0af23d70 2024-09-19T05:10:38.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.382+0000 7f2a4be00640 1 --2- 172.21.15.43:0/93698292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac0af18b90 0x55ac0af2c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:10:38.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.382+0000 7f2a51000640 1 --2- 172.21.15.43:0/93698292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ac0af23d70 0x55ac0af27a80 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2a440bcc30 tx=0x7f2a440bcaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5acac485812e82d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:38.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.382+0000 7f2a3a000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:10:38.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.383+0000 7f2a3a000640 1 -- 172.21.15.43:0/93698292 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4402c040 con 0x55ac0af23d70 2024-09-19T05:10:38.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.383+0000 7f2a3a000640 1 -- 172.21.15.43:0/93698292 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f2a440c5480 con 0x55ac0af23d70 2024-09-19T05:10:38.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.383+0000 7f2a55a46740 1 -- 172.21.15.43:0/93698292 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a3c028070 con 0x55ac0af23d70 2024-09-19T05:10:38.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.383+0000 7f2a3a000640 1 -- 172.21.15.43:0/93698292 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2a440bd3a0 con 0x55ac0af23d70 2024-09-19T05:10:38.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.383+0000 7f2a55a46740 10 client.16401 fetch_fsmap learned FSMap version 29 2024-09-19T05:10:38.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.383+0000 7f2a55a46740 1 -- 172.21.15.43:0/93698292 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ac0af9e3f0 con 0x55ac0af23d70 2024-09-19T05:10:38.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.383+0000 7f2a3a000640 1 -- 172.21.15.43:0/93698292 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a3c028070 con 0x55ac0af23d70 2024-09-19T05:10:38.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.383+0000 7f2a3a000640 1 -- 172.21.15.43:0/93698292 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2a4404d030 con 0x55ac0af23d70 2024-09-19T05:10:38.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.384+0000 7f2a55a46740 10 client.16401 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:10:38.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.384+0000 7f2a55a46740 10 client.16401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:10:38.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.384+0000 7f2a55a46740 20 client.16401 populate_metadata read hostname 'smithi043' 2024-09-19T05:10:38.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.384+0000 7f2a55a46740 1 --2- 172.21.15.43:0/93698292 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55ac0af9eb10 0x55ac0afa0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:38.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.384+0000 7f2a55a46740 4 client.16401 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:10:38.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.384+0000 7f2a55a46740 1 -- 172.21.15.43:0/93698292 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ac0ae86e60 con 0x55ac0af9eb10 2024-09-19T05:10:38.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.384+0000 7f2a4be00640 1 --2- 172.21.15.43:0/93698292 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55ac0af9eb10 0x55ac0afa0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:38.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.384+0000 7f2a4be00640 1 --2- 172.21.15.43:0/93698292 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55ac0af9eb10 0x55ac0afa0f30 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fee836f3034212d9 server_cookie=67f2879ea783022f in_seq=0 out_seq=0 2024-09-19T05:10:38.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.384+0000 7f2a3a000640 10 client.16401 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:10:38.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.385+0000 7f2a3a000640 1 -- 172.21.15.43:0/93698292 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ac0ae86e60 con 0x55ac0af9eb10 2024-09-19T05:10:38.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.385+0000 7f2a3a000640 10 client.16401 handle_command_reply: tid=0 2024-09-19T05:10:38.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.385+0000 7f2a55a46740 1 client.16401 shutdown 2024-09-19T05:10:38.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.385+0000 7f2a55a46740 1 -- 172.21.15.43:0/93698292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ac0af23d70 msgr2=0x55ac0af27a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:38.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.385+0000 7f2a55a46740 1 --2- 172.21.15.43:0/93698292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ac0af23d70 0x55ac0af27a80 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2a440bcc30 tx=0x7f2a440bcaf0 comp rx=0 tx=0).stop 2024-09-19T05:10:38.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.385+0000 7f2a55a46740 1 -- 172.21.15.43:0/93698292 shutdown_connections 2024-09-19T05:10:38.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.385+0000 7f2a55a46740 1 --2- 172.21.15.43:0/93698292 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55ac0af9eb10 0x55ac0afa0f30 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:38.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.385+0000 7f2a55a46740 1 --2- 172.21.15.43:0/93698292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ac0af23d70 0x55ac0af27a80 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:38.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.385+0000 7f2a55a46740 1 --2- 172.21.15.43:0/93698292 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac0af194b0 0x55ac0af27540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:38.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.385+0000 7f2a55a46740 1 --2- 172.21.15.43:0/93698292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ac0af18b90 0x55ac0af2c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:38.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.385+0000 7f2a55a46740 1 -- 172.21.15.43:0/93698292 >> 172.21.15.43:0/93698292 conn(0x55ac0af15120 msgr2=0x55ac0af139a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:38.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.385+0000 7f2a3a000640 0 client.16401 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:10:38.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.385+0000 7f2a55a46740 1 -- 172.21.15.43:0/93698292 shutdown_connections 2024-09-19T05:10:38.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.385+0000 7f2a55a46740 1 -- 172.21.15.43:0/93698292 wait complete. 2024-09-19T05:10:38.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.385+0000 7f2a55a46740 20 client.16401 trim_cache size 0 max 16384 2024-09-19T05:10:38.391 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:10:38.392 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-09-19T05:10:38.392 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-09-19T05:10:38.392 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:10:38.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.386+0000 7f2a53000640 1 -- 172.21.15.43:0/4207014305 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f2a300a3430 msgr2=0x7f2a300a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:38.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.386+0000 7f2a53000640 1 --2- 172.21.15.43:0/4207014305 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f2a300a3430 0x7f2a300a58f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x55ac0af164a0 tx=0x7f2a34005f90 comp rx=0 tx=0).stop 2024-09-19T05:10:38.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.386+0000 7f2a53000640 1 -- 172.21.15.43:0/4207014305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a4c10e3c0 msgr2=0x7f2a4c19ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:38.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.386+0000 7f2a53000640 1 --2- 172.21.15.43:0/4207014305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a4c10e3c0 0x7f2a4c19ef30 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2a44004af0 tx=0x7f2a4402f3a0 comp rx=0 tx=0).stop 2024-09-19T05:10:38.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.386+0000 7f2a53000640 1 -- 172.21.15.43:0/4207014305 shutdown_connections 2024-09-19T05:10:38.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.386+0000 7f2a53000640 1 --2- 172.21.15.43:0/4207014305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a4c10e3c0 0x7f2a4c19ef30 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:38.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.386+0000 7f2a53000640 1 --2- 172.21.15.43:0/4207014305 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a4c074f40 0x7f2a4c19aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:38.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.386+0000 7f2a53000640 1 --2- 172.21.15.43:0/4207014305 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f2a300a3430 0x7f2a300a58f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:38.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.386+0000 7f2a53000640 1 --2- 172.21.15.43:0/4207014305 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a4c074620 0x7f2a4c19a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:38.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.386+0000 7f2a53000640 1 -- 172.21.15.43:0/4207014305 >> 172.21.15.43:0/4207014305 conn(0x7f2a4c0fd8c0 msgr2=0x7f2a4c0fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:38.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.386+0000 7f2a53000640 1 -- 172.21.15.43:0/4207014305 shutdown_connections 2024-09-19T05:10:38.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:38.386+0000 7f2a53000640 1 -- 172.21.15.43:0/4207014305 wait complete. 2024-09-19T05:10:39.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:38 smithi151 ceph-mon[35425]: scrub summary: idle 2024-09-19T05:10:39.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:38 smithi096 ceph-mon[35085]: scrub summary: idle 2024-09-19T05:10:39.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:38 smithi043 ceph-mon[30800]: scrub summary: idle 2024-09-19T05:10:39.379 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T05:10:39.379 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:10:39.760 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:10:40.118 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:39 smithi043 ceph-mon[30800]: pgmap v183: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-19T05:10:40.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:39 smithi096 ceph-mon[35085]: pgmap v183: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-19T05:10:40.273 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:39 smithi151 ceph-mon[35425]: pgmap v183: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-19T05:10:40.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.313+0000 7f033ce00640 1 -- 172.21.15.43:0/3977029367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0338074ee0 msgr2=0x7f033810dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:40.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.313+0000 7f033ce00640 1 --2- 172.21.15.43:0/3977029367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0338074ee0 0x7f033810dee0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f0324009900 tx=0x7f032402a480 comp rx=0 tx=0).stop 2024-09-19T05:10:40.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.313+0000 7f033ce00640 1 -- 172.21.15.43:0/3977029367 shutdown_connections 2024-09-19T05:10:40.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.313+0000 7f033ce00640 1 --2- 172.21.15.43:0/3977029367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f033810e420 0x7f0338110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:40.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.313+0000 7f033ce00640 1 --2- 172.21.15.43:0/3977029367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0338074ee0 0x7f033810dee0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:40.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.313+0000 7f033ce00640 1 --2- 172.21.15.43:0/3977029367 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03380745c0 0x7f03380749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:40.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.313+0000 7f033ce00640 1 -- 172.21.15.43:0/3977029367 >> 172.21.15.43:0/3977029367 conn(0x7f03380fd8c0 msgr2=0x7f03380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:40.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.313+0000 7f033ce00640 1 -- 172.21.15.43:0/3977029367 shutdown_connections 2024-09-19T05:10:40.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.313+0000 7f033ce00640 1 -- 172.21.15.43:0/3977029367 wait complete. 2024-09-19T05:10:40.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.314+0000 7f033ce00640 1 Processor -- start 2024-09-19T05:10:40.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.315+0000 7f033ce00640 1 -- start start 2024-09-19T05:10:40.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.315+0000 7f033ce00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03380745c0 0x7f033819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:40.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.315+0000 7f033ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0338074ee0 0x7f033819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:40.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.315+0000 7f033ce00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f033810e420 0x7f033819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:40.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.315+0000 7f033ce00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0338114160 con 0x7f0338074ee0 2024-09-19T05:10:40.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.315+0000 7f033ce00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0338113fe0 con 0x7f033810e420 2024-09-19T05:10:40.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.315+0000 7f033ce00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f03381142e0 con 0x7f03380745c0 2024-09-19T05:10:40.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.315+0000 7f0335400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0338074ee0 0x7f033819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:40.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.315+0000 7f0335e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03380745c0 0x7f033819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:40.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.315+0000 7f0335e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03380745c0 0x7f033819a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:40802/0 (socket says 172.21.15.43:40802) 2024-09-19T05:10:40.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.315+0000 7f0335e00640 1 -- 172.21.15.43:0/1787971530 learned_addr learned my addr 172.21.15.43:0/1787971530 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:40.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.316+0000 7f0336800640 1 --2- 172.21.15.43:0/1787971530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f033810e420 0x7f033819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:40.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.316+0000 7f0335e00640 1 -- 172.21.15.43:0/1787971530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f033810e420 msgr2=0x7f033819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:40.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.316+0000 7f0335e00640 1 --2- 172.21.15.43:0/1787971530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f033810e420 0x7f033819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:40.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.316+0000 7f0335e00640 1 -- 172.21.15.43:0/1787971530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0338074ee0 msgr2=0x7f033819abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:40.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.316+0000 7f0335e00640 1 --2- 172.21.15.43:0/1787971530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0338074ee0 0x7f033819abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:40.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.316+0000 7f0335e00640 1 -- 172.21.15.43:0/1787971530 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f033819f650 con 0x7f03380745c0 2024-09-19T05:10:40.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.316+0000 7f0335e00640 1 --2- 172.21.15.43:0/1787971530 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03380745c0 0x7f033819a6a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f032800e9f0 tx=0x7f032800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:40.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.317+0000 7f032ea00640 1 -- 172.21.15.43:0/1787971530 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032800cd80 con 0x7f03380745c0 2024-09-19T05:10:40.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.317+0000 7f033ce00640 1 -- 172.21.15.43:0/1787971530 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f033819f940 con 0x7f03380745c0 2024-09-19T05:10:40.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.317+0000 7f032ea00640 1 -- 172.21.15.43:0/1787971530 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0328010910 con 0x7f03380745c0 2024-09-19T05:10:40.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.317+0000 7f033ce00640 1 -- 172.21.15.43:0/1787971530 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03381a7220 con 0x7f03380745c0 2024-09-19T05:10:40.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.317+0000 7f032ea00640 1 -- 172.21.15.43:0/1787971530 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0328018980 con 0x7f03380745c0 2024-09-19T05:10:40.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.318+0000 7f033ce00640 1 -- 172.21.15.43:0/1787971530 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03380749a0 con 0x7f03380745c0 2024-09-19T05:10:40.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.320+0000 7f032ea00640 1 -- 172.21.15.43:0/1787971530 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0328010ac0 con 0x7f03380745c0 2024-09-19T05:10:40.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.321+0000 7f032ea00640 1 --2- 172.21.15.43:0/1787971530 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f031c0a3430 0x7f031c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:40.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.322+0000 7f032ea00640 1 -- 172.21.15.43:0/1787971530 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f03280d0270 con 0x7f03380745c0 2024-09-19T05:10:40.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.322+0000 7f0335400640 1 --2- 172.21.15.43:0/1787971530 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f031c0a3430 0x7f031c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:40.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.322+0000 7f0335400640 1 --2- 172.21.15.43:0/1787971530 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f031c0a3430 0x7f031c0a58f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0324009230 tx=0x7f03240058d0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:40.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.323+0000 7f032ea00640 1 -- 172.21.15.43:0/1787971530 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f032809c5b0 con 0x7f03380745c0 2024-09-19T05:10:40.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.553+0000 7f033ce00640 1 -- 172.21.15.43:0/1787971530 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03381a73f0 con 0x7f03380745c0 2024-09-19T05:10:40.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.554+0000 7f032ea00640 1 -- 172.21.15.43:0/1787971530 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11965 (secure 0 0 0) 0x7f03280a08b0 con 0x7f03380745c0 2024-09-19T05:10:40.556 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:10:40.556 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":29,"btime":"2024-09-19T05:04:38:785407+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/773801169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":773801169},{"type":"v1","addr":"172.21.15.43:6835","nonce":773801169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1799137902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1799137902},{"type":"v1","addr":"172.21.15.43:6837","nonce":1799137902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:02:25.463041+0000","modified":"2024-09-19T05:04:38.039809+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24518,"mds_2":24511,"mds_3":24506,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14541":{"gid":14541,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/794881135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":794881135},{"type":"v1","addr":"172.21.15.43:6841","nonce":794881135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/2836453158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2836453158},{"type":"v1","addr":"172.21.15.151:6833","nonce":2836453158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/178785812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":178785812},{"type":"v1","addr":"172.21.15.96:6833","nonce":178785812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1671043820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1671043820},{"type":"v1","addr":"172.21.15.96:6835","nonce":1671043820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2972514328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2972514328},{"type":"v1","addr":"172.21.15.151:6835","nonce":2972514328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.151:6837/3960378010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3960378010},{"type":"v1","addr":"172.21.15.151:6837","nonce":3960378010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6837/3978388286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3978388286},{"type":"v1","addr":"172.21.15.96:6837","nonce":3978388286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.151:6839/3794104313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3794104313},{"type":"v1","addr":"172.21.15.151:6839","nonce":3794104313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.43:6839/3404555030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3404555030},{"type":"v1","addr":"172.21.15.43:6839","nonce":3404555030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/605527621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":605527621},{"type":"v1","addr":"172.21.15.96:6839","nonce":605527621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24518,24511,24506,24493]},"id":1}]} 2024-09-19T05:10:40.556 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 29 2024-09-19T05:10:40.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.557+0000 7f033ce00640 1 -- 172.21.15.43:0/1787971530 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f031c0a3430 msgr2=0x7f031c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:40.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.557+0000 7f033ce00640 1 --2- 172.21.15.43:0/1787971530 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f031c0a3430 0x7f031c0a58f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0324009230 tx=0x7f03240058d0 comp rx=0 tx=0).stop 2024-09-19T05:10:40.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.557+0000 7f033ce00640 1 -- 172.21.15.43:0/1787971530 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03380745c0 msgr2=0x7f033819a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:40.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.557+0000 7f033ce00640 1 --2- 172.21.15.43:0/1787971530 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03380745c0 0x7f033819a6a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f032800e9f0 tx=0x7f032800eeb0 comp rx=0 tx=0).stop 2024-09-19T05:10:40.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.557+0000 7f033ce00640 1 -- 172.21.15.43:0/1787971530 shutdown_connections 2024-09-19T05:10:40.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.557+0000 7f033ce00640 1 --2- 172.21.15.43:0/1787971530 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f031c0a3430 0x7f031c0a58f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:40.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.557+0000 7f033ce00640 1 --2- 172.21.15.43:0/1787971530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f033810e420 0x7f033819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:40.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.557+0000 7f033ce00640 1 --2- 172.21.15.43:0/1787971530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0338074ee0 0x7f033819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:40.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.557+0000 7f033ce00640 1 --2- 172.21.15.43:0/1787971530 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03380745c0 0x7f033819a6a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:40.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.557+0000 7f033ce00640 1 -- 172.21.15.43:0/1787971530 >> 172.21.15.43:0/1787971530 conn(0x7f03380fd8c0 msgr2=0x7f03380fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:40.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.557+0000 7f033ce00640 1 -- 172.21.15.43:0/1787971530 shutdown_connections 2024-09-19T05:10:40.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:40.557+0000 7f033ce00640 1 -- 172.21.15.43:0/1787971530 wait complete. 2024-09-19T05:10:41.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:40 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/1787971530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:10:41.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:40 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/1787971530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:10:41.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:40 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/1787971530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:10:41.422 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T05:10:41.798 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:10:42.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:41 smithi043 ceph-mon[30800]: pgmap v184: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 op/s 2024-09-19T05:10:42.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:41 smithi151 ceph-mon[35425]: pgmap v184: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 op/s 2024-09-19T05:10:42.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:41 smithi096 ceph-mon[35085]: pgmap v184: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 op/s 2024-09-19T05:10:42.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.384+0000 7f7482400640 1 -- 172.21.15.43:0/1428848299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f747c1049c0 msgr2=0x7f747c108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:42.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.384+0000 7f7482400640 1 --2- 172.21.15.43:0/1428848299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f747c1049c0 0x7f747c108a10 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7464009900 tx=0x7f746402a410 comp rx=0 tx=0).stop 2024-09-19T05:10:42.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.385+0000 7f7482400640 1 -- 172.21.15.43:0/1428848299 shutdown_connections 2024-09-19T05:10:42.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.385+0000 7f7482400640 1 --2- 172.21.15.43:0/1428848299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f747c109140 0x7f747c110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:42.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.385+0000 7f7482400640 1 --2- 172.21.15.43:0/1428848299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f747c1049c0 0x7f747c108a10 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:42.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.385+0000 7f7482400640 1 --2- 172.21.15.43:0/1428848299 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f747c104010 0x7f747c1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:42.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.385+0000 7f7482400640 1 -- 172.21.15.43:0/1428848299 >> 172.21.15.43:0/1428848299 conn(0x7f747c0ffa50 msgr2=0x7f747c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:42.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.385+0000 7f7482400640 1 -- 172.21.15.43:0/1428848299 shutdown_connections 2024-09-19T05:10:42.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.385+0000 7f7482400640 1 -- 172.21.15.43:0/1428848299 wait complete. 2024-09-19T05:10:42.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.386+0000 7f7482400640 1 Processor -- start 2024-09-19T05:10:42.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.386+0000 7f7482400640 1 -- start start 2024-09-19T05:10:42.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.387+0000 7f7482400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f747c104010 0x7f747c19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:42.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.387+0000 7f7482400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f747c1049c0 0x7f747c19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:42.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.387+0000 7f7482400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f747c109140 0x7f747c19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:42.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.387+0000 7f7482400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f747c114140 con 0x7f747c104010 2024-09-19T05:10:42.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.387+0000 7f7482400640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f747c113fc0 con 0x7f747c1049c0 2024-09-19T05:10:42.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.387+0000 7f7482400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f747c1142c0 con 0x7f747c109140 2024-09-19T05:10:42.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.387+0000 7f747be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f747c109140 0x7f747c19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:42.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.387+0000 7f747b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f747c104010 0x7f747c19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:42.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.387+0000 7f747be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f747c109140 0x7f747c19ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:40826/0 (socket says 172.21.15.43:40826) 2024-09-19T05:10:42.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.387+0000 7f747be00640 1 -- 172.21.15.43:0/3913809421 learned_addr learned my addr 172.21.15.43:0/3913809421 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:42.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.387+0000 7f7472a00640 1 --2- 172.21.15.43:0/3913809421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f747c1049c0 0x7f747c19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:42.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.388+0000 7f747be00640 1 -- 172.21.15.43:0/3913809421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f747c1049c0 msgr2=0x7f747c19aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:42.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.388+0000 7f747be00640 1 --2- 172.21.15.43:0/3913809421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f747c1049c0 0x7f747c19aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:42.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.388+0000 7f747be00640 1 -- 172.21.15.43:0/3913809421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f747c104010 msgr2=0x7f747c19a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:42.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.388+0000 7f747be00640 1 --2- 172.21.15.43:0/3913809421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f747c104010 0x7f747c19a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:42.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.388+0000 7f747be00640 1 -- 172.21.15.43:0/3913809421 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f747c19f610 con 0x7f747c109140 2024-09-19T05:10:42.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.388+0000 7f7472a00640 1 --2- 172.21.15.43:0/3913809421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f747c1049c0 0x7f747c19aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:10:42.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.388+0000 7f747be00640 1 --2- 172.21.15.43:0/3913809421 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f747c109140 0x7f747c19ef30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f747400efd0 tx=0x7f747400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:42.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.388+0000 7f7478c00640 1 -- 172.21.15.43:0/3913809421 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7474003b70 con 0x7f747c109140 2024-09-19T05:10:42.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.388+0000 7f7478c00640 1 -- 172.21.15.43:0/3913809421 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7474003d10 con 0x7f747c109140 2024-09-19T05:10:42.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.388+0000 7f7478c00640 1 -- 172.21.15.43:0/3913809421 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74740109f0 con 0x7f747c109140 2024-09-19T05:10:42.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.388+0000 7f7482400640 1 -- 172.21.15.43:0/3913809421 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f747c19f900 con 0x7f747c109140 2024-09-19T05:10:42.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.388+0000 7f7482400640 1 -- 172.21.15.43:0/3913809421 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f747c1a71e0 con 0x7f747c109140 2024-09-19T05:10:42.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.392+0000 7f7484e3a740 1 Processor -- start 2024-09-19T05:10:42.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.393+0000 7f7478c00640 1 -- 172.21.15.43:0/3913809421 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f7474004020 con 0x7f747c109140 2024-09-19T05:10:42.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.393+0000 7f7484e3a740 1 -- start start 2024-09-19T05:10:42.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.393+0000 7f7484e3a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556795048c70 0x556795049050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:42.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.393+0000 7f7484e3a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556795049590 0x556795053910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:42.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.393+0000 7f7484e3a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556795053e50 0x556795056240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:42.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.393+0000 7f7484e3a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556794faf3e0 con 0x556795053e50 2024-09-19T05:10:42.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.393+0000 7f7484e3a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556794f56d80 con 0x556795048c70 2024-09-19T05:10:42.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.393+0000 7f7484e3a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556794fab3a0 con 0x556795049590 2024-09-19T05:10:42.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.394+0000 7f747b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556795049590 0x556795053910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:42.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.394+0000 7f7472a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556795053e50 0x556795056240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:42.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.394+0000 7f747b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556795049590 0x556795053910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:40830/0 (socket says 172.21.15.43:40830) 2024-09-19T05:10:42.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.394+0000 7f747b400640 1 -- 172.21.15.43:0/1654042042 learned_addr learned my addr 172.21.15.43:0/1654042042 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:42.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.394+0000 7f7472a00640 1 --2- 172.21.15.43:0/1654042042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556795048c70 0x556795049050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:42.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.394+0000 7f747b400640 1 -- 172.21.15.43:0/1654042042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556795048c70 msgr2=0x556795049050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:42.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.394+0000 7f747b400640 1 --2- 172.21.15.43:0/1654042042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556795048c70 0x556795049050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:42.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.394+0000 7f747b400640 1 -- 172.21.15.43:0/1654042042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556795053e50 msgr2=0x556795056240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:42.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.394+0000 7f747b400640 1 --2- 172.21.15.43:0/1654042042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556795053e50 0x556795056240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:42.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.394+0000 7f747b400640 1 -- 172.21.15.43:0/1654042042 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556794f75ae0 con 0x556795049590 2024-09-19T05:10:42.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.394+0000 7f7472a00640 1 --2- 172.21.15.43:0/1654042042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556795053e50 0x556795056240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:10:42.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.395+0000 7f7478c00640 1 --2- 172.21.15.43:0/3913809421 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f74600a33b0 0x7f74600a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:42.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.395+0000 7f747b400640 1 --2- 172.21.15.43:0/1654042042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556795049590 0x556795053910 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f746c009cf0 tx=0x7f746c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1a4c8492c230a1a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:42.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.395+0000 7f7478c00640 1 -- 172.21.15.43:0/3913809421 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f74740d0110 con 0x7f747c109140 2024-09-19T05:10:42.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.395+0000 7f7472a00640 1 --2- 172.21.15.43:0/3913809421 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f74600a33b0 0x7f74600a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:42.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.395+0000 7f7470c00640 1 -- 172.21.15.43:0/1654042042 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746c004230 con 0x556795049590 2024-09-19T05:10:42.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.395+0000 7f7470c00640 1 -- 172.21.15.43:0/1654042042 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f746c0043d0 con 0x556795049590 2024-09-19T05:10:42.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.395+0000 7f7470c00640 1 -- 172.21.15.43:0/1654042042 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746c019960 con 0x556795049590 2024-09-19T05:10:42.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.396+0000 7f7484e3a740 1 -- 172.21.15.43:0/1654042042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556795049590 msgr2=0x556795053910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:42.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.396+0000 7f7472a00640 1 --2- 172.21.15.43:0/3913809421 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f74600a33b0 0x7f74600a5870 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7464009230 tx=0x7f74640314e0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:42.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.396+0000 7f7484e3a740 1 --2- 172.21.15.43:0/1654042042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556795049590 0x556795053910 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f746c009cf0 tx=0x7f746c011810 comp rx=0 tx=0).stop 2024-09-19T05:10:42.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.396+0000 7f7484e3a740 1 -- 172.21.15.43:0/1654042042 shutdown_connections 2024-09-19T05:10:42.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.396+0000 7f7484e3a740 1 --2- 172.21.15.43:0/1654042042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556795053e50 0x556795056240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:42.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.396+0000 7f7484e3a740 1 --2- 172.21.15.43:0/1654042042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556795049590 0x556795053910 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:42.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.396+0000 7f7484e3a740 1 --2- 172.21.15.43:0/1654042042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556795048c70 0x556795049050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:42.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.396+0000 7f7484e3a740 1 -- 172.21.15.43:0/1654042042 >> 172.21.15.43:0/1654042042 conn(0x556795045200 msgr2=0x5567950455e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:42.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.396+0000 7f7484e3a740 1 -- 172.21.15.43:0/1654042042 shutdown_connections 2024-09-19T05:10:42.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.396+0000 7f7484e3a740 1 -- 172.21.15.43:0/1654042042 wait complete. 2024-09-19T05:10:42.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.396+0000 7f7484e3a740 1 -- start start 2024-09-19T05:10:42.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.397+0000 7f7484e3a740 1 Processor -- start 2024-09-19T05:10:42.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.397+0000 7f7484e3a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556795048c70 0x556795050d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:42.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.397+0000 7f7484e3a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556795049590 0x556795051280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:42.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.397+0000 7f7484e3a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556795053e50 0x5567950517c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:42.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.397+0000 7f7484e3a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556795056a20 con 0x556795049590 2024-09-19T05:10:42.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.397+0000 7f7484e3a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556794f6a0f0 con 0x556795048c70 2024-09-19T05:10:42.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.397+0000 7f7484e3a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556795056ba0 con 0x556795053e50 2024-09-19T05:10:42.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.397+0000 7f7484e3a740 10 client.0 _ll_register_callbacks cb 0x556794f51bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:42.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.397+0000 7f7472a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556795049590 0x556795051280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:42.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.397+0000 7f747be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556795053e50 0x5567950517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:42.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.397+0000 7f747b400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556795048c70 0x556795050d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:42.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.397+0000 7f7472a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556795049590 0x556795051280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47064/0 (socket says 172.21.15.43:47064) 2024-09-19T05:10:42.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.397+0000 7f747be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556795053e50 0x5567950517c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:40836/0 (socket says 172.21.15.43:40836) 2024-09-19T05:10:42.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.397+0000 7f7472a00640 1 -- 172.21.15.43:0/1627539503 learned_addr learned my addr 172.21.15.43:0/1627539503 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:42.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.398+0000 7f747b400640 1 -- 172.21.15.43:0/1627539503 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556795053e50 msgr2=0x5567950517c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:42.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.398+0000 7f747b400640 1 --2- 172.21.15.43:0/1627539503 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556795053e50 0x5567950517c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:42.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.398+0000 7f747b400640 1 -- 172.21.15.43:0/1627539503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556795049590 msgr2=0x556795051280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:42.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.398+0000 7f747b400640 1 --2- 172.21.15.43:0/1627539503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556795049590 0x556795051280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:42.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.398+0000 7f747b400640 1 -- 172.21.15.43:0/1627539503 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556794d203c0 con 0x556795048c70 2024-09-19T05:10:42.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.398+0000 7f747be00640 1 --2- 172.21.15.43:0/1627539503 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556795053e50 0x5567950517c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:10:42.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.398+0000 7f7472a00640 1 --2- 172.21.15.43:0/1627539503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556795049590 0x556795051280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:10:42.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.398+0000 7f747b400640 1 --2- 172.21.15.43:0/1627539503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556795048c70 0x556795050d40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f746c019500 tx=0x7f746c005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d80f52de4bd93dee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:42.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.398+0000 7f7469600640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:10:42.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.399+0000 7f7469600640 1 -- 172.21.15.43:0/1627539503 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746c009910 con 0x556795048c70 2024-09-19T05:10:42.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.399+0000 7f7469600640 1 -- 172.21.15.43:0/1627539503 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f746c0035f0 con 0x556795048c70 2024-09-19T05:10:42.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.399+0000 7f7484e3a740 1 -- 172.21.15.43:0/1627539503 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f746c0043d0 con 0x556795048c70 2024-09-19T05:10:42.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.399+0000 7f7469600640 1 -- 172.21.15.43:0/1627539503 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f746c032050 con 0x556795048c70 2024-09-19T05:10:42.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.399+0000 7f7484e3a740 10 client.26456 fetch_fsmap learned FSMap version 29 2024-09-19T05:10:42.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.399+0000 7f7484e3a740 1 -- 172.21.15.43:0/1627539503 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556795062a60 con 0x556795048c70 2024-09-19T05:10:42.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.399+0000 7f7469600640 1 -- 172.21.15.43:0/1627539503 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f746c0043d0 con 0x556795048c70 2024-09-19T05:10:42.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.400+0000 7f7469600640 1 -- 172.21.15.43:0/1627539503 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f746c004cd0 con 0x556795048c70 2024-09-19T05:10:42.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.400+0000 7f7484e3a740 10 client.26456 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:10:42.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.400+0000 7f7484e3a740 10 client.26456 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-19T05:10:42.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.400+0000 7f7484e3a740 20 client.26456 populate_metadata read hostname 'smithi043' 2024-09-19T05:10:42.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.400+0000 7f7484e3a740 1 --2- 172.21.15.43:0/1627539503 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x556795063ec0 0x5567950662b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:42.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.400+0000 7f7484e3a740 4 client.26456 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:10:42.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.400+0000 7f7484e3a740 1 -- 172.21.15.43:0/1627539503 --> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556794fbc950 con 0x556795063ec0 2024-09-19T05:10:42.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.400+0000 7f7472a00640 1 --2- 172.21.15.43:0/1627539503 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x556795063ec0 0x5567950662b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:42.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.401+0000 7f7472a00640 1 --2- 172.21.15.43:0/1627539503 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x556795063ec0 0x5567950662b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=12e3465042add9cc server_cookie=8329d1afd065bd50 in_seq=0 out_seq=0 2024-09-19T05:10:42.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.401+0000 7f7469600640 10 client.26456 ms_handle_connect on v2:172.21.15.151:6836/3960378010 2024-09-19T05:10:42.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.402+0000 7f7469600640 1 -- 172.21.15.43:0/1627539503 <== mds.4 v2:172.21.15.151:6836/3960378010 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556794fbc950 con 0x556795063ec0 2024-09-19T05:10:42.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.402+0000 7f7469600640 10 client.26456 handle_command_reply: tid=0 2024-09-19T05:10:42.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.402+0000 7f7484e3a740 1 client.26456 shutdown 2024-09-19T05:10:42.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.402+0000 7f7484e3a740 1 -- 172.21.15.43:0/1627539503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556795048c70 msgr2=0x556795050d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:42.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.402+0000 7f7484e3a740 1 --2- 172.21.15.43:0/1627539503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556795048c70 0x556795050d40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f746c019500 tx=0x7f746c005270 comp rx=0 tx=0).stop 2024-09-19T05:10:42.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.402+0000 7f7484e3a740 1 -- 172.21.15.43:0/1627539503 shutdown_connections 2024-09-19T05:10:42.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.402+0000 7f7484e3a740 1 --2- 172.21.15.43:0/1627539503 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x556795063ec0 0x5567950662b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:42.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.402+0000 7f7484e3a740 1 --2- 172.21.15.43:0/1627539503 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556795053e50 0x5567950517c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:42.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.402+0000 7f7484e3a740 1 --2- 172.21.15.43:0/1627539503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556795049590 0x556795051280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:42.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.402+0000 7f7484e3a740 1 --2- 172.21.15.43:0/1627539503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556795048c70 0x556795050d40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:42.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.402+0000 7f7469600640 0 client.26456 ms_handle_reset on v2:172.21.15.151:6836/3960378010 2024-09-19T05:10:42.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.402+0000 7f7484e3a740 1 -- 172.21.15.43:0/1627539503 >> 172.21.15.43:0/1627539503 conn(0x556795045200 msgr2=0x55679504aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:42.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.402+0000 7f7484e3a740 1 -- 172.21.15.43:0/1627539503 shutdown_connections 2024-09-19T05:10:42.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.402+0000 7f7484e3a740 1 -- 172.21.15.43:0/1627539503 wait complete. 2024-09-19T05:10:42.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.403+0000 7f7484e3a740 20 client.26456 trim_cache size 0 max 16384 2024-09-19T05:10:42.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.416+0000 7f7484e3a740 1 Processor -- start 2024-09-19T05:10:42.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.416+0000 7f7484e3a740 1 -- start start 2024-09-19T05:10:42.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.417+0000 7f7484e3a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556795048c70 0x55679506bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:42.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.417+0000 7f7484e3a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556795049590 0x55679506c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:42.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.417+0000 7f7484e3a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556795053e50 0x556795043b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:42.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.417+0000 7f7484e3a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f747c1afad0 con 0x556795053e50 2024-09-19T05:10:42.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.417+0000 7f7484e3a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f747c1b1610 con 0x556795049590 2024-09-19T05:10:42.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.417+0000 7f7484e3a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f747c1b01a0 con 0x556795048c70 2024-09-19T05:10:42.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.417+0000 7f747be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556795053e50 0x556795043b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:42.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.417+0000 7f7472a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556795049590 0x55679506c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:42.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.417+0000 7f7472a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556795049590 0x55679506c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:43540/0 (socket says 172.21.15.43:43540) 2024-09-19T05:10:42.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.417+0000 7f7472a00640 1 -- 172.21.15.43:0/135536802 learned_addr learned my addr 172.21.15.43:0/135536802 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:42.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.417+0000 7f747b400640 1 --2- 172.21.15.43:0/135536802 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556795048c70 0x55679506bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:42.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.418+0000 7f7472a00640 1 -- 172.21.15.43:0/135536802 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556795048c70 msgr2=0x55679506bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:42.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.418+0000 7f7472a00640 1 --2- 172.21.15.43:0/135536802 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556795048c70 0x55679506bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:42.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.418+0000 7f7472a00640 1 -- 172.21.15.43:0/135536802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556795053e50 msgr2=0x556795043b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:42.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.418+0000 7f7472a00640 1 --2- 172.21.15.43:0/135536802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556795053e50 0x556795043b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:42.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.418+0000 7f7472a00640 1 -- 172.21.15.43:0/135536802 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556795045030 con 0x556795049590 2024-09-19T05:10:42.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.418+0000 7f747be00640 1 --2- 172.21.15.43:0/135536802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556795053e50 0x556795043b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:10:42.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.418+0000 7f7472a00640 1 --2- 172.21.15.43:0/135536802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556795049590 0x55679506c020 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7464006010 tx=0x7f746403f040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5178a396c896e8d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:42.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.418+0000 7f746be00640 1 -- 172.21.15.43:0/135536802 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7464058070 con 0x556795049590 2024-09-19T05:10:42.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.418+0000 7f746be00640 1 -- 172.21.15.43:0/135536802 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74640035f0 con 0x556795049590 2024-09-19T05:10:42.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.418+0000 7f746be00640 1 -- 172.21.15.43:0/135536802 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7464035040 con 0x556795049590 2024-09-19T05:10:42.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.419+0000 7f7484e3a740 1 -- 172.21.15.43:0/135536802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556795049590 msgr2=0x55679506c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:42.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.419+0000 7f7484e3a740 1 --2- 172.21.15.43:0/135536802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556795049590 0x55679506c020 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7464006010 tx=0x7f746403f040 comp rx=0 tx=0).stop 2024-09-19T05:10:42.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.419+0000 7f7484e3a740 1 -- 172.21.15.43:0/135536802 shutdown_connections 2024-09-19T05:10:42.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.419+0000 7f7484e3a740 1 --2- 172.21.15.43:0/135536802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556795053e50 0x556795043b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:42.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.419+0000 7f7484e3a740 1 --2- 172.21.15.43:0/135536802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556795049590 0x55679506c020 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:42.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.419+0000 7f7484e3a740 1 --2- 172.21.15.43:0/135536802 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556795048c70 0x55679506bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:42.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.419+0000 7f7484e3a740 1 -- 172.21.15.43:0/135536802 >> 172.21.15.43:0/135536802 conn(0x556795045200 msgr2=0x556795052dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:42.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.419+0000 7f7484e3a740 1 -- 172.21.15.43:0/135536802 shutdown_connections 2024-09-19T05:10:42.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.419+0000 7f7484e3a740 1 -- 172.21.15.43:0/135536802 wait complete. 2024-09-19T05:10:42.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.419+0000 7f7484e3a740 1 -- start start 2024-09-19T05:10:42.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.420+0000 7f7484e3a740 1 Processor -- start 2024-09-19T05:10:42.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.420+0000 7f7484e3a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556795048c70 0x55679505bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:42.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.420+0000 7f7484e3a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556795049590 0x55679505c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:42.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.420+0000 7f7484e3a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556795053e50 0x55679504d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:42.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.420+0000 7f7484e3a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f747c1b0f40 con 0x556795049590 2024-09-19T05:10:42.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.420+0000 7f7484e3a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f747c1b0870 con 0x556795053e50 2024-09-19T05:10:42.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.420+0000 7f7484e3a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f747c1adab0 con 0x556795048c70 2024-09-19T05:10:42.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.420+0000 7f7484e3a740 10 client.0 _ll_register_callbacks cb 0x7f746c009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:42.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.420+0000 7f7472a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556795049590 0x55679505c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:42.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.420+0000 7f7472a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556795049590 0x55679505c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47076/0 (socket says 172.21.15.43:47076) 2024-09-19T05:10:42.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.420+0000 7f7472a00640 1 -- 172.21.15.43:0/2456821213 learned_addr learned my addr 172.21.15.43:0/2456821213 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:42.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.420+0000 7f747b400640 1 --2- 172.21.15.43:0/2456821213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556795048c70 0x55679505bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:42.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.421+0000 7f747be00640 1 --2- 172.21.15.43:0/2456821213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556795053e50 0x55679504d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:42.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.421+0000 7f7472a00640 1 -- 172.21.15.43:0/2456821213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556795048c70 msgr2=0x55679505bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:42.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.421+0000 7f7472a00640 1 --2- 172.21.15.43:0/2456821213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556795048c70 0x55679505bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:42.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.421+0000 7f7472a00640 1 -- 172.21.15.43:0/2456821213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556795053e50 msgr2=0x55679504d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:42.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.421+0000 7f7472a00640 1 --2- 172.21.15.43:0/2456821213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556795053e50 0x55679504d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:42.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.421+0000 7f7472a00640 1 -- 172.21.15.43:0/2456821213 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556795040770 con 0x556795049590 2024-09-19T05:10:42.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.421+0000 7f747b400640 1 --2- 172.21.15.43:0/2456821213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556795048c70 0x55679505bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:10:42.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.421+0000 7f7472a00640 1 --2- 172.21.15.43:0/2456821213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556795049590 0x55679505c330 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7464009900 tx=0x7f74640041e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d0226677c487d03 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:42.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.421+0000 7f7469600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:10:42.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.422+0000 7f7469600640 1 -- 172.21.15.43:0/2456821213 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7464058070 con 0x556795049590 2024-09-19T05:10:42.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.422+0000 7f7469600640 1 -- 172.21.15.43:0/2456821213 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f74640037a0 con 0x556795049590 2024-09-19T05:10:42.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.422+0000 7f7484e3a740 1 -- 172.21.15.43:0/2456821213 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74640035f0 con 0x556795049590 2024-09-19T05:10:42.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.422+0000 7f7469600640 1 -- 172.21.15.43:0/2456821213 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7464035040 con 0x556795049590 2024-09-19T05:10:42.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.422+0000 7f7484e3a740 10 client.16425 fetch_fsmap learned FSMap version 29 2024-09-19T05:10:42.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.422+0000 7f7484e3a740 1 -- 172.21.15.43:0/2456821213 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5567950cdf90 con 0x556795049590 2024-09-19T05:10:42.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.422+0000 7f7469600640 1 -- 172.21.15.43:0/2456821213 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74640035f0 con 0x556795049590 2024-09-19T05:10:42.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.422+0000 7f7469600640 1 -- 172.21.15.43:0/2456821213 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7464064050 con 0x556795049590 2024-09-19T05:10:42.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.422+0000 7f7484e3a740 10 client.16425 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:10:42.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.423+0000 7f7484e3a740 10 client.16425 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-19T05:10:42.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.423+0000 7f7484e3a740 20 client.16425 populate_metadata read hostname 'smithi043' 2024-09-19T05:10:42.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.423+0000 7f7484e3a740 1 --2- 172.21.15.43:0/2456821213 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x5567950ce660 0x5567950d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:42.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.423+0000 7f7484e3a740 4 client.16425 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:10:42.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.423+0000 7f7484e3a740 1 -- 172.21.15.43:0/2456821213 --> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556794fb6e60 con 0x5567950ce660 2024-09-19T05:10:42.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.423+0000 7f747b400640 1 --2- 172.21.15.43:0/2456821213 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x5567950ce660 0x5567950d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:42.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.423+0000 7f747b400640 1 --2- 172.21.15.43:0/2456821213 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x5567950ce660 0x5567950d0a80 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1e1896624810407f server_cookie=316798e8cb322454 in_seq=0 out_seq=0 2024-09-19T05:10:42.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.423+0000 7f7469600640 10 client.16425 ms_handle_connect on v2:172.21.15.151:6836/3960378010 2024-09-19T05:10:42.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.424+0000 7f7469600640 1 -- 172.21.15.43:0/2456821213 <== mds.4 v2:172.21.15.151:6836/3960378010 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556794fb6e60 con 0x5567950ce660 2024-09-19T05:10:42.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.424+0000 7f7469600640 10 client.16425 handle_command_reply: tid=0 2024-09-19T05:10:42.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.424+0000 7f7484e3a740 1 client.16425 shutdown 2024-09-19T05:10:42.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.424+0000 7f7484e3a740 1 -- 172.21.15.43:0/2456821213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556795049590 msgr2=0x55679505c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:42.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.424+0000 7f7484e3a740 1 --2- 172.21.15.43:0/2456821213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556795049590 0x55679505c330 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7464009900 tx=0x7f74640041e0 comp rx=0 tx=0).stop 2024-09-19T05:10:42.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.424+0000 7f7484e3a740 1 -- 172.21.15.43:0/2456821213 shutdown_connections 2024-09-19T05:10:42.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.424+0000 7f7484e3a740 1 --2- 172.21.15.43:0/2456821213 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x5567950ce660 0x5567950d0a80 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:42.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.424+0000 7f7484e3a740 1 --2- 172.21.15.43:0/2456821213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556795053e50 0x55679504d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:42.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.424+0000 7f7484e3a740 1 --2- 172.21.15.43:0/2456821213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556795049590 0x55679505c330 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:42.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.424+0000 7f7484e3a740 1 --2- 172.21.15.43:0/2456821213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556795048c70 0x55679505bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:42.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.424+0000 7f7484e3a740 1 -- 172.21.15.43:0/2456821213 >> 172.21.15.43:0/2456821213 conn(0x556795045200 msgr2=0x556795043b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:42.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.424+0000 7f7469600640 0 client.16425 ms_handle_reset on v2:172.21.15.151:6836/3960378010 2024-09-19T05:10:42.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.424+0000 7f7484e3a740 1 -- 172.21.15.43:0/2456821213 shutdown_connections 2024-09-19T05:10:42.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.424+0000 7f7484e3a740 1 -- 172.21.15.43:0/2456821213 wait complete. 2024-09-19T05:10:42.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.424+0000 7f7484e3a740 20 client.16425 trim_cache size 0 max 16384 2024-09-19T05:10:42.430 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:10:42.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.425+0000 7f7482400640 1 -- 172.21.15.43:0/3913809421 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f74600a33b0 msgr2=0x7f74600a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:42.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.425+0000 7f7482400640 1 --2- 172.21.15.43:0/3913809421 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f74600a33b0 0x7f74600a5870 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7464009230 tx=0x7f74640314e0 comp rx=0 tx=0).stop 2024-09-19T05:10:42.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.425+0000 7f7482400640 1 -- 172.21.15.43:0/3913809421 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f747c109140 msgr2=0x7f747c19ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:42.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.425+0000 7f7482400640 1 --2- 172.21.15.43:0/3913809421 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f747c109140 0x7f747c19ef30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f747400efd0 tx=0x7f747400c6a0 comp rx=0 tx=0).stop 2024-09-19T05:10:42.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.425+0000 7f7482400640 1 -- 172.21.15.43:0/3913809421 shutdown_connections 2024-09-19T05:10:42.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.425+0000 7f7482400640 1 --2- 172.21.15.43:0/3913809421 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f747c109140 0x7f747c19ef30 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:42.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.425+0000 7f7482400640 1 --2- 172.21.15.43:0/3913809421 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f74600a33b0 0x7f74600a5870 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:42.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.425+0000 7f7482400640 1 --2- 172.21.15.43:0/3913809421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f747c1049c0 0x7f747c19aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:42.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.425+0000 7f7482400640 1 --2- 172.21.15.43:0/3913809421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f747c104010 0x7f747c19a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:42.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.425+0000 7f7482400640 1 -- 172.21.15.43:0/3913809421 >> 172.21.15.43:0/3913809421 conn(0x7f747c0ffa50 msgr2=0x7f747c100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:42.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.425+0000 7f7482400640 1 -- 172.21.15.43:0/3913809421 shutdown_connections 2024-09-19T05:10:42.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:42.425+0000 7f7482400640 1 -- 172.21.15.43:0/3913809421 wait complete. 2024-09-19T05:10:43.258 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T05:10:43.655 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:10:44.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:43 smithi151 ceph-mon[35425]: pgmap v185: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:10:44.221 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:43 smithi043 ceph-mon[30800]: pgmap v185: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:10:44.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:43 smithi096 ceph-mon[35085]: pgmap v185: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:10:44.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.338+0000 7f0312c00640 1 -- 172.21.15.43:0/3728405880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f030c074620 msgr2=0x7f030c074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:44.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.338+0000 7f0312c00640 1 --2- 172.21.15.43:0/3728405880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f030c074620 0x7f030c074a00 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f02fc009900 tx=0x7f02fc02a480 comp rx=0 tx=0).stop 2024-09-19T05:10:44.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.338+0000 7f0312c00640 1 -- 172.21.15.43:0/3728405880 shutdown_connections 2024-09-19T05:10:44.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.338+0000 7f0312c00640 1 --2- 172.21.15.43:0/3728405880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f030c10e3c0 0x7f030c110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:44.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.338+0000 7f0312c00640 1 --2- 172.21.15.43:0/3728405880 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f030c074f40 0x7f030c10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:44.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.338+0000 7f0312c00640 1 --2- 172.21.15.43:0/3728405880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f030c074620 0x7f030c074a00 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:44.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.338+0000 7f0312c00640 1 -- 172.21.15.43:0/3728405880 >> 172.21.15.43:0/3728405880 conn(0x7f030c0fd8c0 msgr2=0x7f030c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:44.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.338+0000 7f0312c00640 1 -- 172.21.15.43:0/3728405880 shutdown_connections 2024-09-19T05:10:44.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.339+0000 7f0312c00640 1 -- 172.21.15.43:0/3728405880 wait complete. 2024-09-19T05:10:44.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.340+0000 7f0312c00640 1 Processor -- start 2024-09-19T05:10:44.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.340+0000 7f0312c00640 1 -- start start 2024-09-19T05:10:44.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.340+0000 7f0312c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f030c074620 0x7f030c19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:44.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.340+0000 7f0312c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f030c074f40 0x7f030c19abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:44.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.340+0000 7f0312c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f030c10e3c0 0x7f030c19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:44.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.340+0000 7f0312c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f030c1141c0 con 0x7f030c074620 2024-09-19T05:10:44.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.340+0000 7f0312c00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f030c114040 con 0x7f030c10e3c0 2024-09-19T05:10:44.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.340+0000 7f0312c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f030c114340 con 0x7f030c074f40 2024-09-19T05:10:44.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.341+0000 7f030be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f030c074620 0x7f030c19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:44.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.341+0000 7f030be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f030c074620 0x7f030c19a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47082/0 (socket says 172.21.15.43:47082) 2024-09-19T05:10:44.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.341+0000 7f030be00640 1 -- 172.21.15.43:0/1926671993 learned_addr learned my addr 172.21.15.43:0/1926671993 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:44.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.341+0000 7f0303e00640 1 --2- 172.21.15.43:0/1926671993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f030c074f40 0x7f030c19abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:44.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.341+0000 7f0310c00640 1 --2- 172.21.15.43:0/1926671993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f030c10e3c0 0x7f030c19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:44.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.341+0000 7f030be00640 1 -- 172.21.15.43:0/1926671993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f030c074f40 msgr2=0x7f030c19abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:44.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.341+0000 7f030be00640 1 --2- 172.21.15.43:0/1926671993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f030c074f40 0x7f030c19abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:44.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.341+0000 7f030be00640 1 -- 172.21.15.43:0/1926671993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f030c10e3c0 msgr2=0x7f030c19ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:44.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.341+0000 7f030be00640 1 --2- 172.21.15.43:0/1926671993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f030c10e3c0 0x7f030c19ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:44.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.341+0000 7f030be00640 1 -- 172.21.15.43:0/1926671993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f030c19f640 con 0x7f030c074620 2024-09-19T05:10:44.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.342+0000 7f030be00640 1 --2- 172.21.15.43:0/1926671993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f030c074620 0x7f030c19a690 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f02fc004770 tx=0x7f02fc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:44.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.342+0000 7f0309600640 1 -- 172.21.15.43:0/1926671993 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02fc03f070 con 0x7f030c074620 2024-09-19T05:10:44.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.342+0000 7f0309600640 1 -- 172.21.15.43:0/1926671993 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02fc031ca0 con 0x7f030c074620 2024-09-19T05:10:44.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.342+0000 7f0312c00640 1 -- 172.21.15.43:0/1926671993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f030c19f8d0 con 0x7f030c074620 2024-09-19T05:10:44.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.342+0000 7f0309600640 1 -- 172.21.15.43:0/1926671993 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02fc035030 con 0x7f030c074620 2024-09-19T05:10:44.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.342+0000 7f0312c00640 1 -- 172.21.15.43:0/1926671993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f030c1a7210 con 0x7f030c074620 2024-09-19T05:10:44.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.346+0000 7f03157c0740 1 Processor -- start 2024-09-19T05:10:44.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.347+0000 7f0309600640 1 -- 172.21.15.43:0/1926671993 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f02fc042070 con 0x7f030c074620 2024-09-19T05:10:44.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.347+0000 7f03157c0740 1 -- start start 2024-09-19T05:10:44.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.347+0000 7f03157c0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f5103c70 0x5569f5104050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:44.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.347+0000 7f03157c0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f5104590 0x5569f510e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:44.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.347+0000 7f03157c0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f510ee50 0x5569f5111240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:44.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.347+0000 7f03157c0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5569f506a3e0 con 0x5569f5103c70 2024-09-19T05:10:44.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.347+0000 7f03157c0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5569f5011d80 con 0x5569f5104590 2024-09-19T05:10:44.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.347+0000 7f03157c0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5569f50663a0 con 0x5569f510ee50 2024-09-19T05:10:44.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.347+0000 7f0310c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f5103c70 0x5569f5104050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:44.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.347+0000 7f030be00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f5104590 0x5569f510e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:44.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.347+0000 7f0310c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f5103c70 0x5569f5104050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47086/0 (socket says 172.21.15.43:47086) 2024-09-19T05:10:44.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.347+0000 7f0310c00640 1 -- 172.21.15.43:0/2132404358 learned_addr learned my addr 172.21.15.43:0/2132404358 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:44.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.348+0000 7f0303e00640 1 --2- 172.21.15.43:0/2132404358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f510ee50 0x5569f5111240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:44.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.348+0000 7f030be00640 1 -- 172.21.15.43:0/2132404358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f510ee50 msgr2=0x5569f5111240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:44.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.348+0000 7f030be00640 1 --2- 172.21.15.43:0/2132404358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f510ee50 0x5569f5111240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:44.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.348+0000 7f030be00640 1 -- 172.21.15.43:0/2132404358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f5103c70 msgr2=0x5569f5104050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:44.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.348+0000 7f030be00640 1 --2- 172.21.15.43:0/2132404358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f5103c70 0x5569f5104050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:44.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.348+0000 7f030be00640 1 -- 172.21.15.43:0/2132404358 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569f5030ae0 con 0x5569f5104590 2024-09-19T05:10:44.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.348+0000 7f030be00640 1 --2- 172.21.15.43:0/2132404358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f5104590 0x5569f510e910 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f02fc0eda30 tx=0x7f02fc0f85c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e418226c489f4f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:44.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.349+0000 7f0301200640 1 -- 172.21.15.43:0/2132404358 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02fc0f2310 con 0x5569f5104590 2024-09-19T05:10:44.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.349+0000 7f0301200640 1 -- 172.21.15.43:0/2132404358 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02fc0f24b0 con 0x5569f5104590 2024-09-19T05:10:44.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.349+0000 7f0301200640 1 -- 172.21.15.43:0/2132404358 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02fc101a30 con 0x5569f5104590 2024-09-19T05:10:44.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.349+0000 7f0309600640 1 --2- 172.21.15.43:0/1926671993 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f02f00a33b0 0x7f02f00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:44.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.349+0000 7f0309600640 1 -- 172.21.15.43:0/1926671993 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f02fc0f0330 con 0x7f030c074620 2024-09-19T05:10:44.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.349+0000 7f0310c00640 1 --2- 172.21.15.43:0/1926671993 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f02f00a33b0 0x7f02f00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:44.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.349+0000 7f03157c0740 1 -- 172.21.15.43:0/2132404358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f5104590 msgr2=0x5569f510e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:44.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.349+0000 7f03157c0740 1 --2- 172.21.15.43:0/2132404358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f5104590 0x5569f510e910 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f02fc0eda30 tx=0x7f02fc0f85c0 comp rx=0 tx=0).stop 2024-09-19T05:10:44.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.350+0000 7f03157c0740 1 -- 172.21.15.43:0/2132404358 shutdown_connections 2024-09-19T05:10:44.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.350+0000 7f03157c0740 1 --2- 172.21.15.43:0/2132404358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f510ee50 0x5569f5111240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:44.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.350+0000 7f03157c0740 1 --2- 172.21.15.43:0/2132404358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f5104590 0x5569f510e910 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:44.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.350+0000 7f03157c0740 1 --2- 172.21.15.43:0/2132404358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f5103c70 0x5569f5104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:44.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.350+0000 7f03157c0740 1 -- 172.21.15.43:0/2132404358 >> 172.21.15.43:0/2132404358 conn(0x5569f5100200 msgr2=0x5569f51005e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:44.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.350+0000 7f0310c00640 1 --2- 172.21.15.43:0/1926671993 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f02f00a33b0 0x7f02f00a5870 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0304011630 tx=0x7f0304009500 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:44.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.350+0000 7f03157c0740 1 -- 172.21.15.43:0/2132404358 shutdown_connections 2024-09-19T05:10:44.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.350+0000 7f03157c0740 1 -- 172.21.15.43:0/2132404358 wait complete. 2024-09-19T05:10:44.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.351+0000 7f03157c0740 1 -- start start 2024-09-19T05:10:44.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.351+0000 7f03157c0740 1 Processor -- start 2024-09-19T05:10:44.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.351+0000 7f03157c0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f5103c70 0x5569f510bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:44.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.351+0000 7f03157c0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f5104590 0x5569f510c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:44.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.351+0000 7f03157c0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f510ee50 0x5569f510c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:44.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.351+0000 7f03157c0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5569f51117e0 con 0x5569f5103c70 2024-09-19T05:10:44.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.351+0000 7f03157c0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5569f50250f0 con 0x5569f5104590 2024-09-19T05:10:44.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.351+0000 7f03157c0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5569f5111960 con 0x5569f510ee50 2024-09-19T05:10:44.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.351+0000 7f03157c0740 10 client.0 _ll_register_callbacks cb 0x5569f500cbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:44.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.352+0000 7f030be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f5103c70 0x5569f510bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:44.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.352+0000 7f030be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f5103c70 0x5569f510bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47098/0 (socket says 172.21.15.43:47098) 2024-09-19T05:10:44.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.352+0000 7f030be00640 1 -- 172.21.15.43:0/3575612755 learned_addr learned my addr 172.21.15.43:0/3575612755 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:44.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.352+0000 7f0310c00640 1 --2- 172.21.15.43:0/3575612755 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f510ee50 0x5569f510c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:44.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.352+0000 7f0303e00640 1 --2- 172.21.15.43:0/3575612755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f5104590 0x5569f510c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:44.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.352+0000 7f030be00640 1 -- 172.21.15.43:0/3575612755 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f510ee50 msgr2=0x5569f510c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:44.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.352+0000 7f030be00640 1 --2- 172.21.15.43:0/3575612755 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f510ee50 0x5569f510c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:44.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.352+0000 7f030be00640 1 -- 172.21.15.43:0/3575612755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f5104590 msgr2=0x5569f510c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:44.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.352+0000 7f030be00640 1 --2- 172.21.15.43:0/3575612755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f5104590 0x5569f510c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:44.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.352+0000 7f030be00640 1 -- 172.21.15.43:0/3575612755 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569f4ddb3c0 con 0x5569f5103c70 2024-09-19T05:10:44.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.352+0000 7f030be00640 1 --2- 172.21.15.43:0/3575612755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f5103c70 0x5569f510bcd0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f02fc0bc6c0 tx=0x7f02fc02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1904516e7122519 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:44.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.352+0000 7f02f9600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:10:44.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.353+0000 7f02f9600640 1 -- 172.21.15.43:0/3575612755 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02fc049070 con 0x5569f5103c70 2024-09-19T05:10:44.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.353+0000 7f02f9600640 1 -- 172.21.15.43:0/3575612755 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f02fc0baa30 con 0x5569f5103c70 2024-09-19T05:10:44.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.353+0000 7f03157c0740 1 -- 172.21.15.43:0/3575612755 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02fc0f24b0 con 0x5569f5103c70 2024-09-19T05:10:44.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.353+0000 7f02f9600640 1 -- 172.21.15.43:0/3575612755 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f02fc0b89d0 con 0x5569f5103c70 2024-09-19T05:10:44.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.353+0000 7f03157c0740 10 client.16437 fetch_fsmap learned FSMap version 29 2024-09-19T05:10:44.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.353+0000 7f03157c0740 1 -- 172.21.15.43:0/3575612755 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5569f51117e0 con 0x5569f5103c70 2024-09-19T05:10:44.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.353+0000 7f02f9600640 1 -- 172.21.15.43:0/3575612755 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02fc0f24b0 con 0x5569f5103c70 2024-09-19T05:10:44.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.353+0000 7f02f9600640 1 -- 172.21.15.43:0/3575612755 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f02fc0b8da0 con 0x5569f5103c70 2024-09-19T05:10:44.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.354+0000 7f03157c0740 10 client.16437 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:10:44.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.354+0000 7f03157c0740 10 client.16437 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T05:10:44.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.354+0000 7f03157c0740 20 client.16437 populate_metadata read hostname 'smithi043' 2024-09-19T05:10:44.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.354+0000 7f03157c0740 1 --2- 172.21.15.43:0/3575612755 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x5569f511e9b0 0x5569f5120dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:44.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.354+0000 7f03157c0740 4 client.16437 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:10:44.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.354+0000 7f03157c0740 1 -- 172.21.15.43:0/3575612755 --> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569f5077950 con 0x5569f511e9b0 2024-09-19T05:10:44.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.354+0000 7f0303e00640 1 --2- 172.21.15.43:0/3575612755 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x5569f511e9b0 0x5569f5120dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:44.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.355+0000 7f0303e00640 1 --2- 172.21.15.43:0/3575612755 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x5569f511e9b0 0x5569f5120dd0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a910c9b6f615ae82 server_cookie=ff97c6be8c970886 in_seq=0 out_seq=0 2024-09-19T05:10:44.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.355+0000 7f02f9600640 10 client.16437 ms_handle_connect on v2:172.21.15.96:6836/3978388286 2024-09-19T05:10:44.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.355+0000 7f02f9600640 1 -- 172.21.15.43:0/3575612755 <== mds.3 v2:172.21.15.96:6836/3978388286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5569f5077950 con 0x5569f511e9b0 2024-09-19T05:10:44.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.355+0000 7f02f9600640 10 client.16437 handle_command_reply: tid=0 2024-09-19T05:10:44.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.355+0000 7f03157c0740 1 client.16437 shutdown 2024-09-19T05:10:44.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.356+0000 7f03157c0740 1 -- 172.21.15.43:0/3575612755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f5103c70 msgr2=0x5569f510bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:44.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.356+0000 7f03157c0740 1 --2- 172.21.15.43:0/3575612755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f5103c70 0x5569f510bcd0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f02fc0bc6c0 tx=0x7f02fc02a480 comp rx=0 tx=0).stop 2024-09-19T05:10:44.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.356+0000 7f03157c0740 1 -- 172.21.15.43:0/3575612755 shutdown_connections 2024-09-19T05:10:44.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.356+0000 7f03157c0740 1 --2- 172.21.15.43:0/3575612755 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x5569f511e9b0 0x5569f5120dd0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:44.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.356+0000 7f03157c0740 1 --2- 172.21.15.43:0/3575612755 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f510ee50 0x5569f510c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:44.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.356+0000 7f03157c0740 1 --2- 172.21.15.43:0/3575612755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f5104590 0x5569f510c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:44.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.356+0000 7f03157c0740 1 --2- 172.21.15.43:0/3575612755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f5103c70 0x5569f510bcd0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:44.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.356+0000 7f03157c0740 1 -- 172.21.15.43:0/3575612755 >> 172.21.15.43:0/3575612755 conn(0x5569f5100200 msgr2=0x5569f50ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:44.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.356+0000 7f02f9600640 0 client.16437 ms_handle_reset on v2:172.21.15.96:6836/3978388286 2024-09-19T05:10:44.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.356+0000 7f03157c0740 1 -- 172.21.15.43:0/3575612755 shutdown_connections 2024-09-19T05:10:44.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.356+0000 7f03157c0740 1 -- 172.21.15.43:0/3575612755 wait complete. 2024-09-19T05:10:44.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.356+0000 7f03157c0740 20 client.16437 trim_cache size 0 max 16384 2024-09-19T05:10:44.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.370+0000 7f03157c0740 1 Processor -- start 2024-09-19T05:10:44.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.370+0000 7f03157c0740 1 -- start start 2024-09-19T05:10:44.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.370+0000 7f03157c0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f5103c70 0x5569f510a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:44.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.370+0000 7f03157c0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f5104590 0x5569f510c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:44.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.370+0000 7f03157c0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f510ee50 0x5569f51266a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:44.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.370+0000 7f03157c0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f030c1afb20 con 0x5569f5104590 2024-09-19T05:10:44.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.370+0000 7f03157c0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f030c1b1660 con 0x5569f5103c70 2024-09-19T05:10:44.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.370+0000 7f03157c0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f030c1b01f0 con 0x5569f510ee50 2024-09-19T05:10:44.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.371+0000 7f0303e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f5104590 0x5569f510c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:44.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.371+0000 7f0310c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f510ee50 0x5569f51266a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:44.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.371+0000 7f0303e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f5104590 0x5569f510c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47114/0 (socket says 172.21.15.43:47114) 2024-09-19T05:10:44.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.371+0000 7f0303e00640 1 -- 172.21.15.43:0/1592579955 learned_addr learned my addr 172.21.15.43:0/1592579955 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:44.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.371+0000 7f030be00640 1 --2- 172.21.15.43:0/1592579955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f5103c70 0x5569f510a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:44.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.371+0000 7f0303e00640 1 -- 172.21.15.43:0/1592579955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f510ee50 msgr2=0x5569f51266a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:44.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.371+0000 7f0303e00640 1 --2- 172.21.15.43:0/1592579955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f510ee50 0x5569f51266a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:44.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.371+0000 7f0303e00640 1 -- 172.21.15.43:0/1592579955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f5103c70 msgr2=0x5569f510a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:44.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.371+0000 7f0303e00640 1 --2- 172.21.15.43:0/1592579955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f5103c70 0x5569f510a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:44.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.371+0000 7f0303e00640 1 -- 172.21.15.43:0/1592579955 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569f5100030 con 0x5569f5104590 2024-09-19T05:10:44.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.371+0000 7f030be00640 1 --2- 172.21.15.43:0/1592579955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f5103c70 0x5569f510a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:10:44.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.371+0000 7f0310c00640 1 --2- 172.21.15.43:0/1592579955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f510ee50 0x5569f51266a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:10:44.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.371+0000 7f0303e00640 1 --2- 172.21.15.43:0/1592579955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f5104590 0x5569f510c5a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f02f4000c00 tx=0x7f02f4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ecc874d705d1c54 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:44.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.372+0000 7f02fbe00640 1 -- 172.21.15.43:0/1592579955 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f4022070 con 0x5569f5104590 2024-09-19T05:10:44.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.372+0000 7f02fbe00640 1 -- 172.21.15.43:0/1592579955 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02f4004250 con 0x5569f5104590 2024-09-19T05:10:44.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.372+0000 7f02fbe00640 1 -- 172.21.15.43:0/1592579955 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f401f7b0 con 0x5569f5104590 2024-09-19T05:10:44.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.372+0000 7f03157c0740 1 -- 172.21.15.43:0/1592579955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f5104590 msgr2=0x5569f510c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:44.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.372+0000 7f03157c0740 1 --2- 172.21.15.43:0/1592579955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f5104590 0x5569f510c5a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f02f4000c00 tx=0x7f02f4005f90 comp rx=0 tx=0).stop 2024-09-19T05:10:44.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.372+0000 7f03157c0740 1 -- 172.21.15.43:0/1592579955 shutdown_connections 2024-09-19T05:10:44.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.372+0000 7f03157c0740 1 --2- 172.21.15.43:0/1592579955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f510ee50 0x5569f51266a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:44.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.372+0000 7f03157c0740 1 --2- 172.21.15.43:0/1592579955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f5104590 0x5569f510c5a0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:44.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.372+0000 7f03157c0740 1 --2- 172.21.15.43:0/1592579955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f5103c70 0x5569f510a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:44.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.372+0000 7f03157c0740 1 -- 172.21.15.43:0/1592579955 >> 172.21.15.43:0/1592579955 conn(0x5569f5100200 msgr2=0x5569f510dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:44.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.373+0000 7f03157c0740 1 -- 172.21.15.43:0/1592579955 shutdown_connections 2024-09-19T05:10:44.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.373+0000 7f03157c0740 1 -- 172.21.15.43:0/1592579955 wait complete. 2024-09-19T05:10:44.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.373+0000 7f03157c0740 1 -- start start 2024-09-19T05:10:44.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.373+0000 7f03157c0740 1 Processor -- start 2024-09-19T05:10:44.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.374+0000 7f03157c0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f5103c70 0x5569f5108e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:44.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.374+0000 7f03157c0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f5104590 0x5569f5109350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:44.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.374+0000 7f03157c0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f510ee50 0x5569f5109890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:44.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.374+0000 7f03157c0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f030c1b0f90 con 0x5569f5103c70 2024-09-19T05:10:44.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.374+0000 7f03157c0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f030c1b08c0 con 0x5569f5104590 2024-09-19T05:10:44.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.374+0000 7f03157c0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f030c1adb00 con 0x5569f510ee50 2024-09-19T05:10:44.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.374+0000 7f03157c0740 10 client.0 _ll_register_callbacks cb 0x7f0304009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:44.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.374+0000 7f030be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f5103c70 0x5569f5108e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:44.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.374+0000 7f0303e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f5104590 0x5569f5109350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:44.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.374+0000 7f030be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f5103c70 0x5569f5108e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47128/0 (socket says 172.21.15.43:47128) 2024-09-19T05:10:44.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.374+0000 7f030be00640 1 -- 172.21.15.43:0/263941135 learned_addr learned my addr 172.21.15.43:0/263941135 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:44.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.374+0000 7f030be00640 1 -- 172.21.15.43:0/263941135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f510ee50 msgr2=0x5569f5109890 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:10:44.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.374+0000 7f030be00640 1 --2- 172.21.15.43:0/263941135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f510ee50 0x5569f5109890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:44.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.374+0000 7f030be00640 1 -- 172.21.15.43:0/263941135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f5104590 msgr2=0x5569f5109350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:44.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.374+0000 7f030be00640 1 --2- 172.21.15.43:0/263941135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f5104590 0x5569f5109350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:44.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.375+0000 7f030be00640 1 -- 172.21.15.43:0/263941135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569f5188720 con 0x5569f5103c70 2024-09-19T05:10:44.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.375+0000 7f030be00640 1 --2- 172.21.15.43:0/263941135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f5103c70 0x5569f5108e10 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f02fc0c4dc0 tx=0x7f02fc03b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25614e06990e8b05 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:44.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.375+0000 7f02f9600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:10:44.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.375+0000 7f02f9600640 1 -- 172.21.15.43:0/263941135 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02fc049070 con 0x5569f5103c70 2024-09-19T05:10:44.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.375+0000 7f02f9600640 1 -- 172.21.15.43:0/263941135 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f02fc03b4f0 con 0x5569f5103c70 2024-09-19T05:10:44.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.375+0000 7f03157c0740 1 -- 172.21.15.43:0/263941135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02f4004250 con 0x5569f5103c70 2024-09-19T05:10:44.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.375+0000 7f02f9600640 1 -- 172.21.15.43:0/263941135 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f02fc005d50 con 0x5569f5103c70 2024-09-19T05:10:44.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.375+0000 7f03157c0740 10 client.16443 fetch_fsmap learned FSMap version 29 2024-09-19T05:10:44.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.375+0000 7f03157c0740 1 -- 172.21.15.43:0/263941135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5569f5188720 con 0x5569f5103c70 2024-09-19T05:10:44.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.376+0000 7f02f9600640 1 -- 172.21.15.43:0/263941135 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02f4004250 con 0x5569f5103c70 2024-09-19T05:10:44.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.376+0000 7f02f9600640 1 -- 172.21.15.43:0/263941135 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f02fc0c4e60 con 0x5569f5103c70 2024-09-19T05:10:44.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.376+0000 7f03157c0740 10 client.16443 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:10:44.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.376+0000 7f03157c0740 10 client.16443 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T05:10:44.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.376+0000 7f03157c0740 20 client.16443 populate_metadata read hostname 'smithi043' 2024-09-19T05:10:44.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.376+0000 7f03157c0740 1 --2- 172.21.15.43:0/263941135 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x5569f5189d90 0x5569f518c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:44.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.376+0000 7f03157c0740 4 client.16443 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:10:44.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.376+0000 7f03157c0740 1 -- 172.21.15.43:0/263941135 --> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5569f5071e60 con 0x5569f5189d90 2024-09-19T05:10:44.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.376+0000 7f0303e00640 1 --2- 172.21.15.43:0/263941135 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x5569f5189d90 0x5569f518c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:44.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.377+0000 7f0303e00640 1 --2- 172.21.15.43:0/263941135 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x5569f5189d90 0x5569f518c1b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ea215ef4414010d3 server_cookie=68cb9bc57b92cb54 in_seq=0 out_seq=0 2024-09-19T05:10:44.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.377+0000 7f02f9600640 10 client.16443 ms_handle_connect on v2:172.21.15.96:6836/3978388286 2024-09-19T05:10:44.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.377+0000 7f02f9600640 1 -- 172.21.15.43:0/263941135 <== mds.3 v2:172.21.15.96:6836/3978388286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5569f5071e60 con 0x5569f5189d90 2024-09-19T05:10:44.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.377+0000 7f02f9600640 10 client.16443 handle_command_reply: tid=0 2024-09-19T05:10:44.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.377+0000 7f03157c0740 1 client.16443 shutdown 2024-09-19T05:10:44.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.377+0000 7f03157c0740 1 -- 172.21.15.43:0/263941135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f5103c70 msgr2=0x5569f5108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:44.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.377+0000 7f03157c0740 1 --2- 172.21.15.43:0/263941135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f5103c70 0x5569f5108e10 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f02fc0c4dc0 tx=0x7f02fc03b460 comp rx=0 tx=0).stop 2024-09-19T05:10:44.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.377+0000 7f03157c0740 1 -- 172.21.15.43:0/263941135 shutdown_connections 2024-09-19T05:10:44.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.377+0000 7f03157c0740 1 --2- 172.21.15.43:0/263941135 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x5569f5189d90 0x5569f518c1b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:44.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.377+0000 7f03157c0740 1 --2- 172.21.15.43:0/263941135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f510ee50 0x5569f5109890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:44.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.377+0000 7f03157c0740 1 --2- 172.21.15.43:0/263941135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569f5104590 0x5569f5109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:44.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.377+0000 7f03157c0740 1 --2- 172.21.15.43:0/263941135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569f5103c70 0x5569f5108e10 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:44.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.377+0000 7f02f9600640 0 client.16443 ms_handle_reset on v2:172.21.15.96:6836/3978388286 2024-09-19T05:10:44.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.377+0000 7f03157c0740 1 -- 172.21.15.43:0/263941135 >> 172.21.15.43:0/263941135 conn(0x5569f5100200 msgr2=0x5569f50feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:44.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.378+0000 7f03157c0740 1 -- 172.21.15.43:0/263941135 shutdown_connections 2024-09-19T05:10:44.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.378+0000 7f03157c0740 1 -- 172.21.15.43:0/263941135 wait complete. 2024-09-19T05:10:44.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.378+0000 7f03157c0740 20 client.16443 trim_cache size 0 max 16384 2024-09-19T05:10:44.383 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:10:44.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.378+0000 7f0312c00640 1 -- 172.21.15.43:0/1926671993 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f02f00a33b0 msgr2=0x7f02f00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:44.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.378+0000 7f0312c00640 1 --2- 172.21.15.43:0/1926671993 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f02f00a33b0 0x7f02f00a5870 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0304011630 tx=0x7f0304009500 comp rx=0 tx=0).stop 2024-09-19T05:10:44.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.379+0000 7f0312c00640 1 -- 172.21.15.43:0/1926671993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f030c074620 msgr2=0x7f030c19a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:44.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.379+0000 7f0312c00640 1 --2- 172.21.15.43:0/1926671993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f030c074620 0x7f030c19a690 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f02fc004770 tx=0x7f02fc003c10 comp rx=0 tx=0).stop 2024-09-19T05:10:44.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.379+0000 7f0312c00640 1 -- 172.21.15.43:0/1926671993 shutdown_connections 2024-09-19T05:10:44.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.379+0000 7f0312c00640 1 --2- 172.21.15.43:0/1926671993 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f02f00a33b0 0x7f02f00a5870 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:44.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.379+0000 7f0312c00640 1 --2- 172.21.15.43:0/1926671993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f030c10e3c0 0x7f030c19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:44.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.379+0000 7f0312c00640 1 --2- 172.21.15.43:0/1926671993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f030c074f40 0x7f030c19abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:44.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.379+0000 7f0312c00640 1 --2- 172.21.15.43:0/1926671993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f030c074620 0x7f030c19a690 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:44.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.379+0000 7f0312c00640 1 -- 172.21.15.43:0/1926671993 >> 172.21.15.43:0/1926671993 conn(0x7f030c0fd8c0 msgr2=0x7f030c0fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:44.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.379+0000 7f0312c00640 1 -- 172.21.15.43:0/1926671993 shutdown_connections 2024-09-19T05:10:44.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:44.379+0000 7f0312c00640 1 -- 172.21.15.43:0/1926671993 wait complete. 2024-09-19T05:10:45.274 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T05:10:45.653 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:10:46.098 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:45 smithi043 ceph-mon[30800]: pgmap v186: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:10:46.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:45 smithi151 ceph-mon[35425]: pgmap v186: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:10:46.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.211+0000 7f43b7200640 1 -- 172.21.15.43:0/850934697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43b8103e30 msgr2=0x7f43b8104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:46.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.211+0000 7f43b7200640 1 --2- 172.21.15.43:0/850934697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43b8103e30 0x7f43b8104210 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f43a8009900 tx=0x7f43a802a3f0 comp rx=0 tx=0).stop 2024-09-19T05:10:46.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.211+0000 7f43b7200640 1 -- 172.21.15.43:0/850934697 shutdown_connections 2024-09-19T05:10:46.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.211+0000 7f43b7200640 1 --2- 172.21.15.43:0/850934697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43b8108f60 0x7f43b8110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:46.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.211+0000 7f43b7200640 1 --2- 172.21.15.43:0/850934697 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43b81047e0 0x7f43b8108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:46.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.211+0000 7f43b7200640 1 --2- 172.21.15.43:0/850934697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43b8103e30 0x7f43b8104210 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:46.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.211+0000 7f43b7200640 1 -- 172.21.15.43:0/850934697 >> 172.21.15.43:0/850934697 conn(0x7f43b80ff8d0 msgr2=0x7f43b8101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:46.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.211+0000 7f43b7200640 1 -- 172.21.15.43:0/850934697 shutdown_connections 2024-09-19T05:10:46.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.211+0000 7f43b7200640 1 -- 172.21.15.43:0/850934697 wait complete. 2024-09-19T05:10:46.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.212+0000 7f43b7200640 1 Processor -- start 2024-09-19T05:10:46.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.212+0000 7f43b7200640 1 -- start start 2024-09-19T05:10:46.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.212+0000 7f43b7200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43b8103e30 0x7f43b819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:46.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.213+0000 7f43b7200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43b81047e0 0x7f43b819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:46.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.213+0000 7f43b7200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43b8108f60 0x7f43b819ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:46.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.213+0000 7f43b7200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f43b8113fc0 con 0x7f43b81047e0 2024-09-19T05:10:46.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.213+0000 7f43b7200640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f43b8113e40 con 0x7f43b8103e30 2024-09-19T05:10:46.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.213+0000 7f43b7200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f43b8114140 con 0x7f43b8108f60 2024-09-19T05:10:46.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.213+0000 7f43afe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43b81047e0 0x7f43b819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:46.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.213+0000 7f43afe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43b81047e0 0x7f43b819a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56472/0 (socket says 172.21.15.43:56472) 2024-09-19T05:10:46.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.213+0000 7f43afe00640 1 -- 172.21.15.43:0/2299415522 learned_addr learned my addr 172.21.15.43:0/2299415522 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:46.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.213+0000 7f43b5e00640 1 --2- 172.21.15.43:0/2299415522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43b8103e30 0x7f43b819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:46.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.213+0000 7f43b6800640 1 --2- 172.21.15.43:0/2299415522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43b8108f60 0x7f43b819ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:46.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.213+0000 7f43afe00640 1 -- 172.21.15.43:0/2299415522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43b8108f60 msgr2=0x7f43b819ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:46.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.213+0000 7f43afe00640 1 --2- 172.21.15.43:0/2299415522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43b8108f60 0x7f43b819ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:46.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.213+0000 7f43afe00640 1 -- 172.21.15.43:0/2299415522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43b8103e30 msgr2=0x7f43b819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:46.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.213+0000 7f43afe00640 1 --2- 172.21.15.43:0/2299415522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43b8103e30 0x7f43b819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:46.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.213+0000 7f43afe00640 1 -- 172.21.15.43:0/2299415522 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43b819f430 con 0x7f43b81047e0 2024-09-19T05:10:46.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.213+0000 7f43b5e00640 1 --2- 172.21.15.43:0/2299415522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43b8103e30 0x7f43b819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:10:46.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.214+0000 7f43afe00640 1 --2- 172.21.15.43:0/2299415522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43b81047e0 0x7f43b819a9c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f43a000e9f0 tx=0x7f43a000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:46.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.214+0000 7f43aea00640 1 -- 172.21.15.43:0/2299415522 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a000cd80 con 0x7f43b81047e0 2024-09-19T05:10:46.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.214+0000 7f43aea00640 1 -- 172.21.15.43:0/2299415522 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43a0005500 con 0x7f43b81047e0 2024-09-19T05:10:46.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.214+0000 7f43b7200640 1 -- 172.21.15.43:0/2299415522 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43b819f720 con 0x7f43b81047e0 2024-09-19T05:10:46.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.214+0000 7f43aea00640 1 -- 172.21.15.43:0/2299415522 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a0010690 con 0x7f43b81047e0 2024-09-19T05:10:46.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.214+0000 7f43b7200640 1 -- 172.21.15.43:0/2299415522 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43b81a7000 con 0x7f43b81047e0 2024-09-19T05:10:46.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.217+0000 7f43bfa03740 1 Processor -- start 2024-09-19T05:10:46.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.217+0000 7f43aea00640 1 -- 172.21.15.43:0/2299415522 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f43a00046c0 con 0x7f43b81047e0 2024-09-19T05:10:46.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.217+0000 7f43bfa03740 1 -- start start 2024-09-19T05:10:46.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.218+0000 7f43bfa03740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5d4202e40 0x55e5d4203220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:46.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.218+0000 7f43bfa03740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d4203760 0x55e5d420dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:46.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.218+0000 7f43bfa03740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d420e020 0x55e5d4210410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:46.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.218+0000 7f43bfa03740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e5d41693e0 con 0x55e5d4202e40 2024-09-19T05:10:46.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.218+0000 7f43bfa03740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e5d4110d80 con 0x55e5d420e020 2024-09-19T05:10:46.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.218+0000 7f43bfa03740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e5d41653a0 con 0x55e5d4203760 2024-09-19T05:10:46.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.218+0000 7f43b6800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5d4202e40 0x55e5d4203220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:46.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.218+0000 7f43b5e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d4203760 0x55e5d420dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:46.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.218+0000 7f43b5e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d4203760 0x55e5d420dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:49486/0 (socket says 172.21.15.43:49486) 2024-09-19T05:10:46.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.218+0000 7f43b5e00640 1 -- 172.21.15.43:0/1262134 learned_addr learned my addr 172.21.15.43:0/1262134 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:46.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.218+0000 7f43afe00640 1 --2- 172.21.15.43:0/1262134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d420e020 0x55e5d4210410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:46.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.218+0000 7f43b5e00640 1 -- 172.21.15.43:0/1262134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d420e020 msgr2=0x55e5d4210410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:46.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.218+0000 7f43b5e00640 1 --2- 172.21.15.43:0/1262134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d420e020 0x55e5d4210410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:46.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.219+0000 7f43b5e00640 1 -- 172.21.15.43:0/1262134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5d4202e40 msgr2=0x55e5d4203220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:46.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.219+0000 7f43b5e00640 1 --2- 172.21.15.43:0/1262134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5d4202e40 0x55e5d4203220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:46.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.219+0000 7f43b5e00640 1 -- 172.21.15.43:0/1262134 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5d412fae0 con 0x55e5d4203760 2024-09-19T05:10:46.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.219+0000 7f43afe00640 1 --2- 172.21.15.43:0/1262134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d420e020 0x55e5d4210410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:10:46.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.219+0000 7f43b5e00640 1 --2- 172.21.15.43:0/1262134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d4203760 0x55e5d420dae0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f43a8009900 tx=0x7f43a8005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d634770fc41b06d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:46.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.219+0000 7f43a7400640 1 -- 172.21.15.43:0/1262134 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a8035040 con 0x55e5d4203760 2024-09-19T05:10:46.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.219+0000 7f43a7400640 1 -- 172.21.15.43:0/1262134 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43a8031830 con 0x55e5d4203760 2024-09-19T05:10:46.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.219+0000 7f43a7400640 1 -- 172.21.15.43:0/1262134 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a80047a0 con 0x55e5d4203760 2024-09-19T05:10:46.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.220+0000 7f43aea00640 1 --2- 172.21.15.43:0/2299415522 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f439c0a33b0 0x7f439c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:46.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.220+0000 7f43aea00640 1 -- 172.21.15.43:0/2299415522 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f43a00d0150 con 0x7f43b81047e0 2024-09-19T05:10:46.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.220+0000 7f43b6800640 1 --2- 172.21.15.43:0/2299415522 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f439c0a33b0 0x7f439c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:46.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.220+0000 7f43bfa03740 1 -- 172.21.15.43:0/1262134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d4203760 msgr2=0x55e5d420dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:46.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.220+0000 7f43bfa03740 1 --2- 172.21.15.43:0/1262134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d4203760 0x55e5d420dae0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f43a8009900 tx=0x7f43a8005d50 comp rx=0 tx=0).stop 2024-09-19T05:10:46.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.220+0000 7f43bfa03740 1 -- 172.21.15.43:0/1262134 shutdown_connections 2024-09-19T05:10:46.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.220+0000 7f43bfa03740 1 --2- 172.21.15.43:0/1262134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d420e020 0x55e5d4210410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:46.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.220+0000 7f43bfa03740 1 --2- 172.21.15.43:0/1262134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d4203760 0x55e5d420dae0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:46.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.220+0000 7f43bfa03740 1 --2- 172.21.15.43:0/1262134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5d4202e40 0x55e5d4203220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:46.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.220+0000 7f43bfa03740 1 -- 172.21.15.43:0/1262134 >> 172.21.15.43:0/1262134 conn(0x55e5d41ff3d0 msgr2=0x55e5d41ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:46.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.221+0000 7f43bfa03740 1 -- 172.21.15.43:0/1262134 shutdown_connections 2024-09-19T05:10:46.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.221+0000 7f43b6800640 1 --2- 172.21.15.43:0/2299415522 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f439c0a33b0 0x7f439c0a5870 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f43b0005fd0 tx=0x7f43b0009500 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:46.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.221+0000 7f43bfa03740 1 -- 172.21.15.43:0/1262134 wait complete. 2024-09-19T05:10:46.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.221+0000 7f43bfa03740 1 -- start start 2024-09-19T05:10:46.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.221+0000 7f43bfa03740 1 Processor -- start 2024-09-19T05:10:46.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.222+0000 7f43bfa03740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5d4202e40 0x55e5d420aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:46.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.222+0000 7f43bfa03740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d4203760 0x55e5d420b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:46.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.222+0000 7f43bfa03740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d420e020 0x55e5d420b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:46.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.222+0000 7f43bfa03740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e5d4210bf0 con 0x55e5d4202e40 2024-09-19T05:10:46.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.222+0000 7f43bfa03740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e5d41240f0 con 0x55e5d4203760 2024-09-19T05:10:46.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.222+0000 7f43bfa03740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e5d4210d70 con 0x55e5d420e020 2024-09-19T05:10:46.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.222+0000 7f43bfa03740 10 client.0 _ll_register_callbacks cb 0x55e5d410bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:46.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.222+0000 7f43b5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5d4202e40 0x55e5d420aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:46.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.222+0000 7f43afe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d4203760 0x55e5d420b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:46.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.222+0000 7f43afe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d4203760 0x55e5d420b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:54128/0 (socket says 172.21.15.43:54128) 2024-09-19T05:10:46.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.222+0000 7f43afe00640 1 -- 172.21.15.43:0/3557333860 learned_addr learned my addr 172.21.15.43:0/3557333860 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:46.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.222+0000 7f43b6800640 1 --2- 172.21.15.43:0/3557333860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d420e020 0x55e5d420b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:46.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.223+0000 7f43afe00640 1 -- 172.21.15.43:0/3557333860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d420e020 msgr2=0x55e5d420b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:46.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.223+0000 7f43afe00640 1 --2- 172.21.15.43:0/3557333860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d420e020 0x55e5d420b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:46.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.223+0000 7f43afe00640 1 -- 172.21.15.43:0/3557333860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5d4202e40 msgr2=0x55e5d420aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:46.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.223+0000 7f43afe00640 1 --2- 172.21.15.43:0/3557333860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5d4202e40 0x55e5d420aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:46.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.223+0000 7f43afe00640 1 -- 172.21.15.43:0/3557333860 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5d3eda3c0 con 0x55e5d4203760 2024-09-19T05:10:46.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.223+0000 7f43afe00640 1 --2- 172.21.15.43:0/3557333860 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d4203760 0x55e5d420b420 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f43a000c6a0 tx=0x7f43a009c270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=beee8514bf0f081c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:46.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.223+0000 7f439be00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:10:46.230 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:45 smithi096 ceph-mon[35085]: pgmap v186: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:10:46.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.223+0000 7f439be00640 1 -- 172.21.15.43:0/3557333860 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a00cdce0 con 0x55e5d4203760 2024-09-19T05:10:46.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.223+0000 7f439be00640 1 -- 172.21.15.43:0/3557333860 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f43a00a4dc0 con 0x55e5d4203760 2024-09-19T05:10:46.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.223+0000 7f43bfa03740 1 -- 172.21.15.43:0/3557333860 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43a8031830 con 0x55e5d4203760 2024-09-19T05:10:46.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.223+0000 7f439be00640 1 -- 172.21.15.43:0/3557333860 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f43a009c3e0 con 0x55e5d4203760 2024-09-19T05:10:46.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.223+0000 7f43bfa03740 10 client.26489 fetch_fsmap learned FSMap version 29 2024-09-19T05:10:46.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.223+0000 7f43bfa03740 1 -- 172.21.15.43:0/3557333860 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e5d421cb70 con 0x55e5d4203760 2024-09-19T05:10:46.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.224+0000 7f439be00640 1 -- 172.21.15.43:0/3557333860 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43a8031830 con 0x55e5d4203760 2024-09-19T05:10:46.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.224+0000 7f439be00640 1 -- 172.21.15.43:0/3557333860 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f43a009b9c0 con 0x55e5d4203760 2024-09-19T05:10:46.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.224+0000 7f43bfa03740 10 client.26489 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:10:46.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.224+0000 7f43bfa03740 10 client.26489 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T05:10:46.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.224+0000 7f43bfa03740 20 client.26489 populate_metadata read hostname 'smithi043' 2024-09-19T05:10:46.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.224+0000 7f43bfa03740 1 --2- 172.21.15.43:0/3557333860 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55e5d421e030 0x55e5d4220420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:46.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.224+0000 7f43bfa03740 4 client.26489 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:10:46.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.224+0000 7f43bfa03740 1 -- 172.21.15.43:0/3557333860 --> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5d4176950 con 0x55e5d421e030 2024-09-19T05:10:46.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.225+0000 7f43b5e00640 1 --2- 172.21.15.43:0/3557333860 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55e5d421e030 0x55e5d4220420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:46.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.225+0000 7f43b5e00640 1 --2- 172.21.15.43:0/3557333860 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55e5d421e030 0x55e5d4220420 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4bc9dd08347650a6 server_cookie=201163de2b221c5d in_seq=0 out_seq=0 2024-09-19T05:10:46.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.225+0000 7f439be00640 10 client.26489 ms_handle_connect on v2:172.21.15.151:6838/3794104313 2024-09-19T05:10:46.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.226+0000 7f439be00640 1 -- 172.21.15.43:0/3557333860 <== mds.2 v2:172.21.15.151:6838/3794104313 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e5d4176950 con 0x55e5d421e030 2024-09-19T05:10:46.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.226+0000 7f439be00640 10 client.26489 handle_command_reply: tid=0 2024-09-19T05:10:46.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.226+0000 7f43bfa03740 1 client.26489 shutdown 2024-09-19T05:10:46.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.226+0000 7f43bfa03740 1 -- 172.21.15.43:0/3557333860 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d4203760 msgr2=0x55e5d420b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:46.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.226+0000 7f43bfa03740 1 --2- 172.21.15.43:0/3557333860 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d4203760 0x55e5d420b420 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f43a000c6a0 tx=0x7f43a009c270 comp rx=0 tx=0).stop 2024-09-19T05:10:46.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.226+0000 7f43bfa03740 1 -- 172.21.15.43:0/3557333860 shutdown_connections 2024-09-19T05:10:46.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.226+0000 7f43bfa03740 1 --2- 172.21.15.43:0/3557333860 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55e5d421e030 0x55e5d4220420 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:46.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.226+0000 7f43bfa03740 1 --2- 172.21.15.43:0/3557333860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d420e020 0x55e5d420b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:46.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.226+0000 7f43bfa03740 1 --2- 172.21.15.43:0/3557333860 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d4203760 0x55e5d420b420 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:46.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.226+0000 7f43bfa03740 1 --2- 172.21.15.43:0/3557333860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5d4202e40 0x55e5d420aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:46.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.226+0000 7f439be00640 0 client.26489 ms_handle_reset on v2:172.21.15.151:6838/3794104313 2024-09-19T05:10:46.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.226+0000 7f43bfa03740 1 -- 172.21.15.43:0/3557333860 >> 172.21.15.43:0/3557333860 conn(0x55e5d41ff3d0 msgr2=0x55e5d4205100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:46.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.226+0000 7f43bfa03740 1 -- 172.21.15.43:0/3557333860 shutdown_connections 2024-09-19T05:10:46.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.226+0000 7f43bfa03740 1 -- 172.21.15.43:0/3557333860 wait complete. 2024-09-19T05:10:46.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.227+0000 7f43bfa03740 20 client.26489 trim_cache size 0 max 16384 2024-09-19T05:10:46.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.240+0000 7f43bfa03740 1 Processor -- start 2024-09-19T05:10:46.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.240+0000 7f43bfa03740 1 -- start start 2024-09-19T05:10:46.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.241+0000 7f43bfa03740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d4202e40 0x55e5d4225cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:46.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.241+0000 7f43bfa03740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5d4203760 0x55e5d4226210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:46.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.241+0000 7f43bfa03740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d420e020 0x55e5d4208160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:46.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.241+0000 7f43bfa03740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f43b81af860 con 0x55e5d4203760 2024-09-19T05:10:46.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.241+0000 7f43bfa03740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f43b81b13a0 con 0x55e5d4202e40 2024-09-19T05:10:46.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.241+0000 7f43bfa03740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f43b81aff30 con 0x55e5d420e020 2024-09-19T05:10:46.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.241+0000 7f43b5e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d4202e40 0x55e5d4225cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:46.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.241+0000 7f43afe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5d4203760 0x55e5d4226210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:46.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.241+0000 7f43b5e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d4202e40 0x55e5d4225cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:54130/0 (socket says 172.21.15.43:54130) 2024-09-19T05:10:46.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.241+0000 7f43b5e00640 1 -- 172.21.15.43:0/2997368874 learned_addr learned my addr 172.21.15.43:0/2997368874 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:46.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.241+0000 7f43b6800640 1 --2- 172.21.15.43:0/2997368874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d420e020 0x55e5d4208160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:46.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.241+0000 7f43b5e00640 1 -- 172.21.15.43:0/2997368874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d420e020 msgr2=0x55e5d4208160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:46.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.241+0000 7f43b5e00640 1 --2- 172.21.15.43:0/2997368874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d420e020 0x55e5d4208160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:46.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.241+0000 7f43b5e00640 1 -- 172.21.15.43:0/2997368874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5d4203760 msgr2=0x55e5d4226210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:46.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.241+0000 7f43b5e00640 1 --2- 172.21.15.43:0/2997368874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5d4203760 0x55e5d4226210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:46.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.241+0000 7f43b5e00640 1 -- 172.21.15.43:0/2997368874 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5d4202c70 con 0x55e5d4202e40 2024-09-19T05:10:46.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.242+0000 7f43b5e00640 1 --2- 172.21.15.43:0/2997368874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d4202e40 0x55e5d4225cd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f43a80057c0 tx=0x7f43a802a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2c36b23ac53ede9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:46.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.242+0000 7f43a6a00640 1 -- 172.21.15.43:0/2997368874 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a8035040 con 0x55e5d4202e40 2024-09-19T05:10:46.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.242+0000 7f43a6a00640 1 -- 172.21.15.43:0/2997368874 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43a8031b80 con 0x55e5d4202e40 2024-09-19T05:10:46.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.242+0000 7f43a6a00640 1 -- 172.21.15.43:0/2997368874 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a8003cc0 con 0x55e5d4202e40 2024-09-19T05:10:46.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.243+0000 7f43bfa03740 1 -- 172.21.15.43:0/2997368874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d4202e40 msgr2=0x55e5d4225cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:46.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.243+0000 7f43bfa03740 1 --2- 172.21.15.43:0/2997368874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d4202e40 0x55e5d4225cd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f43a80057c0 tx=0x7f43a802a3f0 comp rx=0 tx=0).stop 2024-09-19T05:10:46.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.243+0000 7f43bfa03740 1 -- 172.21.15.43:0/2997368874 shutdown_connections 2024-09-19T05:10:46.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.243+0000 7f43bfa03740 1 --2- 172.21.15.43:0/2997368874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d420e020 0x55e5d4208160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:46.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.243+0000 7f43bfa03740 1 --2- 172.21.15.43:0/2997368874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5d4203760 0x55e5d4226210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:46.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.243+0000 7f43bfa03740 1 --2- 172.21.15.43:0/2997368874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d4202e40 0x55e5d4225cd0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:46.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.243+0000 7f43bfa03740 1 -- 172.21.15.43:0/2997368874 >> 172.21.15.43:0/2997368874 conn(0x55e5d41ff3d0 msgr2=0x55e5d4206200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:46.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.243+0000 7f43bfa03740 1 -- 172.21.15.43:0/2997368874 shutdown_connections 2024-09-19T05:10:46.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.243+0000 7f43bfa03740 1 -- 172.21.15.43:0/2997368874 wait complete. 2024-09-19T05:10:46.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.243+0000 7f43bfa03740 1 -- start start 2024-09-19T05:10:46.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.243+0000 7f43bfa03740 1 Processor -- start 2024-09-19T05:10:46.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.244+0000 7f43bfa03740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5d4202e40 0x55e5d4207ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:46.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.244+0000 7f43bfa03740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d4203760 0x55e5d4208430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:46.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.244+0000 7f43bfa03740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d420e020 0x55e5d4208970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:46.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.244+0000 7f43bfa03740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f43b81b0cd0 con 0x55e5d4202e40 2024-09-19T05:10:46.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.244+0000 7f43bfa03740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f43b81b0600 con 0x55e5d420e020 2024-09-19T05:10:46.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.244+0000 7f43bfa03740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f43b81ad840 con 0x55e5d4203760 2024-09-19T05:10:46.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.244+0000 7f43bfa03740 10 client.0 _ll_register_callbacks cb 0x7f43a0096f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:46.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.244+0000 7f43b5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5d4202e40 0x55e5d4207ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:46.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.244+0000 7f43afe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d4203760 0x55e5d4208430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:46.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.244+0000 7f43b5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5d4202e40 0x55e5d4207ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56520/0 (socket says 172.21.15.43:56520) 2024-09-19T05:10:46.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.244+0000 7f43b5e00640 1 -- 172.21.15.43:0/3717919923 learned_addr learned my addr 172.21.15.43:0/3717919923 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:46.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.244+0000 7f43b6800640 1 --2- 172.21.15.43:0/3717919923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d420e020 0x55e5d4208970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:46.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.245+0000 7f43afe00640 1 -- 172.21.15.43:0/3717919923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d420e020 msgr2=0x55e5d4208970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:46.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.245+0000 7f43afe00640 1 --2- 172.21.15.43:0/3717919923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d420e020 0x55e5d4208970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:46.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.245+0000 7f43afe00640 1 -- 172.21.15.43:0/3717919923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5d4202e40 msgr2=0x55e5d4207ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:46.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.245+0000 7f43afe00640 1 --2- 172.21.15.43:0/3717919923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5d4202e40 0x55e5d4207ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:46.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.245+0000 7f43afe00640 1 -- 172.21.15.43:0/3717919923 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5d4200060 con 0x55e5d4203760 2024-09-19T05:10:46.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.245+0000 7f43afe00640 1 --2- 172.21.15.43:0/3717919923 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d4203760 0x55e5d4208430 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f43a009a690 tx=0x7f43a00a4df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8289e6d8ccffd586 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:46.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.245+0000 7f439be00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:10:46.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.245+0000 7f439be00640 1 -- 172.21.15.43:0/3717919923 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a00cdce0 con 0x55e5d4203760 2024-09-19T05:10:46.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.245+0000 7f439be00640 1 -- 172.21.15.43:0/3717919923 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f43a009b2a0 con 0x55e5d4203760 2024-09-19T05:10:46.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.245+0000 7f43bfa03740 1 -- 172.21.15.43:0/3717919923 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43a8031b80 con 0x55e5d4203760 2024-09-19T05:10:46.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.245+0000 7f439be00640 1 -- 172.21.15.43:0/3717919923 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f43a009bd40 con 0x55e5d4203760 2024-09-19T05:10:46.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.245+0000 7f43bfa03740 10 client.26428 fetch_fsmap learned FSMap version 29 2024-09-19T05:10:46.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.245+0000 7f43bfa03740 1 -- 172.21.15.43:0/3717919923 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e5d4288dc0 con 0x55e5d4203760 2024-09-19T05:10:46.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.246+0000 7f439be00640 1 -- 172.21.15.43:0/3717919923 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43a8031b80 con 0x55e5d4203760 2024-09-19T05:10:46.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.246+0000 7f439be00640 1 -- 172.21.15.43:0/3717919923 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f43a0004f40 con 0x55e5d4203760 2024-09-19T05:10:46.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.246+0000 7f43bfa03740 10 client.26428 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:10:46.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.246+0000 7f43bfa03740 10 client.26428 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T05:10:46.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.246+0000 7f43bfa03740 20 client.26428 populate_metadata read hostname 'smithi043' 2024-09-19T05:10:46.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.246+0000 7f43bfa03740 1 --2- 172.21.15.43:0/3717919923 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55e5d42893b0 0x55e5d428b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:46.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.246+0000 7f43bfa03740 4 client.26428 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:10:46.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.246+0000 7f43bfa03740 1 -- 172.21.15.43:0/3717919923 --> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e5d4170e60 con 0x55e5d42893b0 2024-09-19T05:10:46.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.246+0000 7f43b5e00640 1 --2- 172.21.15.43:0/3717919923 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55e5d42893b0 0x55e5d428b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:46.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.247+0000 7f43b5e00640 1 --2- 172.21.15.43:0/3717919923 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55e5d42893b0 0x55e5d428b7d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=860a618e0eee3215 server_cookie=7a7b8b151fe3fa99 in_seq=0 out_seq=0 2024-09-19T05:10:46.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.247+0000 7f439be00640 10 client.26428 ms_handle_connect on v2:172.21.15.151:6838/3794104313 2024-09-19T05:10:46.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.247+0000 7f439be00640 1 -- 172.21.15.43:0/3717919923 <== mds.2 v2:172.21.15.151:6838/3794104313 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e5d4170e60 con 0x55e5d42893b0 2024-09-19T05:10:46.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.247+0000 7f439be00640 10 client.26428 handle_command_reply: tid=0 2024-09-19T05:10:46.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.247+0000 7f43bfa03740 1 client.26428 shutdown 2024-09-19T05:10:46.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.247+0000 7f43bfa03740 1 -- 172.21.15.43:0/3717919923 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d4203760 msgr2=0x55e5d4208430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:46.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.247+0000 7f43bfa03740 1 --2- 172.21.15.43:0/3717919923 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d4203760 0x55e5d4208430 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f43a009a690 tx=0x7f43a00a4df0 comp rx=0 tx=0).stop 2024-09-19T05:10:46.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.247+0000 7f43bfa03740 1 -- 172.21.15.43:0/3717919923 shutdown_connections 2024-09-19T05:10:46.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.247+0000 7f43bfa03740 1 --2- 172.21.15.43:0/3717919923 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55e5d42893b0 0x55e5d428b7d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:46.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.247+0000 7f43bfa03740 1 --2- 172.21.15.43:0/3717919923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d420e020 0x55e5d4208970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:46.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.247+0000 7f43bfa03740 1 --2- 172.21.15.43:0/3717919923 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d4203760 0x55e5d4208430 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:46.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.247+0000 7f43bfa03740 1 --2- 172.21.15.43:0/3717919923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5d4202e40 0x55e5d4207ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:46.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.247+0000 7f439be00640 0 client.26428 ms_handle_reset on v2:172.21.15.151:6838/3794104313 2024-09-19T05:10:46.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.247+0000 7f43bfa03740 1 -- 172.21.15.43:0/3717919923 >> 172.21.15.43:0/3717919923 conn(0x55e5d41ff3d0 msgr2=0x55e5d4204890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:46.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.248+0000 7f43bfa03740 1 -- 172.21.15.43:0/3717919923 shutdown_connections 2024-09-19T05:10:46.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.248+0000 7f43bfa03740 1 -- 172.21.15.43:0/3717919923 wait complete. 2024-09-19T05:10:46.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.248+0000 7f43bfa03740 20 client.26428 trim_cache size 0 max 16384 2024-09-19T05:10:46.254 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:10:46.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.248+0000 7f43b7200640 1 -- 172.21.15.43:0/2299415522 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f439c0a33b0 msgr2=0x7f439c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:46.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.248+0000 7f43b7200640 1 --2- 172.21.15.43:0/2299415522 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f439c0a33b0 0x7f439c0a5870 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f43b0005fd0 tx=0x7f43b0009500 comp rx=0 tx=0).stop 2024-09-19T05:10:46.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.248+0000 7f43b7200640 1 -- 172.21.15.43:0/2299415522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43b81047e0 msgr2=0x7f43b819a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:46.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.248+0000 7f43b7200640 1 --2- 172.21.15.43:0/2299415522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43b81047e0 0x7f43b819a9c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f43a000e9f0 tx=0x7f43a000edf0 comp rx=0 tx=0).stop 2024-09-19T05:10:46.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.248+0000 7f43b7200640 1 -- 172.21.15.43:0/2299415522 shutdown_connections 2024-09-19T05:10:46.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.248+0000 7f43b7200640 1 --2- 172.21.15.43:0/2299415522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43b8108f60 0x7f43b819ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:46.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.248+0000 7f43b7200640 1 --2- 172.21.15.43:0/2299415522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43b81047e0 0x7f43b819a9c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:46.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.248+0000 7f43b7200640 1 --2- 172.21.15.43:0/2299415522 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f439c0a33b0 0x7f439c0a5870 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:46.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.248+0000 7f43b7200640 1 --2- 172.21.15.43:0/2299415522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43b8103e30 0x7f43b819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:46.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.248+0000 7f43b7200640 1 -- 172.21.15.43:0/2299415522 >> 172.21.15.43:0/2299415522 conn(0x7f43b80ff8d0 msgr2=0x7f43b8100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:46.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.249+0000 7f43b7200640 1 -- 172.21.15.43:0/2299415522 shutdown_connections 2024-09-19T05:10:46.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:46.249+0000 7f43b7200640 1 -- 172.21.15.43:0/2299415522 wait complete. 2024-09-19T05:10:47.104 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T05:10:47.480 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:10:48.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.040+0000 7fba67200640 1 -- 172.21.15.43:0/1282410268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba681047e0 msgr2=0x7fba68108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:48.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.040+0000 7fba67200640 1 --2- 172.21.15.43:0/1282410268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba681047e0 0x7fba68108830 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fba50009900 tx=0x7fba5002a3f0 comp rx=0 tx=0).stop 2024-09-19T05:10:48.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.040+0000 7fba67200640 1 -- 172.21.15.43:0/1282410268 shutdown_connections 2024-09-19T05:10:48.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.040+0000 7fba67200640 1 --2- 172.21.15.43:0/1282410268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba68108f60 0x7fba68110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:48.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.040+0000 7fba67200640 1 --2- 172.21.15.43:0/1282410268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba681047e0 0x7fba68108830 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:48.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.040+0000 7fba67200640 1 --2- 172.21.15.43:0/1282410268 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba68103e30 0x7fba68104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:48.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.040+0000 7fba67200640 1 -- 172.21.15.43:0/1282410268 >> 172.21.15.43:0/1282410268 conn(0x7fba680ff910 msgr2=0x7fba68101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:48.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.040+0000 7fba67200640 1 -- 172.21.15.43:0/1282410268 shutdown_connections 2024-09-19T05:10:48.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.040+0000 7fba67200640 1 -- 172.21.15.43:0/1282410268 wait complete. 2024-09-19T05:10:48.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.041+0000 7fba67200640 1 Processor -- start 2024-09-19T05:10:48.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.041+0000 7fba67200640 1 -- start start 2024-09-19T05:10:48.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.042+0000 7fba67200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba68103e30 0x7fba6819a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:48.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.042+0000 7fba67200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba681047e0 0x7fba6819aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:48.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.042+0000 7fba67200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba68108f60 0x7fba6819ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:48.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.042+0000 7fba67200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fba68114010 con 0x7fba68103e30 2024-09-19T05:10:48.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.042+0000 7fba67200640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fba68113e90 con 0x7fba681047e0 2024-09-19T05:10:48.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.042+0000 7fba67200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fba68114190 con 0x7fba68108f60 2024-09-19T05:10:48.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.042+0000 7fba65e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba68103e30 0x7fba6819a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:48.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.042+0000 7fba65e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba68103e30 0x7fba6819a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56544/0 (socket says 172.21.15.43:56544) 2024-09-19T05:10:48.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.042+0000 7fba65e00640 1 -- 172.21.15.43:0/391498493 learned_addr learned my addr 172.21.15.43:0/391498493 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:48.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.042+0000 7fba5fe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba681047e0 0x7fba6819aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:48.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.042+0000 7fba66800640 1 --2- 172.21.15.43:0/391498493 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba68108f60 0x7fba6819ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:48.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.043+0000 7fba65e00640 1 -- 172.21.15.43:0/391498493 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba68108f60 msgr2=0x7fba6819ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:48.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.043+0000 7fba65e00640 1 --2- 172.21.15.43:0/391498493 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba68108f60 0x7fba6819ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:48.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.043+0000 7fba65e00640 1 -- 172.21.15.43:0/391498493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba681047e0 msgr2=0x7fba6819aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:48.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.043+0000 7fba65e00640 1 --2- 172.21.15.43:0/391498493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba681047e0 0x7fba6819aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:48.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.043+0000 7fba65e00640 1 -- 172.21.15.43:0/391498493 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba6819f4e0 con 0x7fba68103e30 2024-09-19T05:10:48.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.043+0000 7fba5fe00640 1 --2- 172.21.15.43:0/391498493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba681047e0 0x7fba6819aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:10:48.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.043+0000 7fba65e00640 1 --2- 172.21.15.43:0/391498493 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba68103e30 0x7fba6819a530 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fba5800e9f0 tx=0x7fba5800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:48.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.043+0000 7fba5ea00640 1 -- 172.21.15.43:0/391498493 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5800cd80 con 0x7fba68103e30 2024-09-19T05:10:48.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.043+0000 7fba5ea00640 1 -- 172.21.15.43:0/391498493 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba58005500 con 0x7fba68103e30 2024-09-19T05:10:48.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.043+0000 7fba67200640 1 -- 172.21.15.43:0/391498493 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba6819f7d0 con 0x7fba68103e30 2024-09-19T05:10:48.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.043+0000 7fba5ea00640 1 -- 172.21.15.43:0/391498493 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba58010690 con 0x7fba68103e30 2024-09-19T05:10:48.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.043+0000 7fba67200640 1 -- 172.21.15.43:0/391498493 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba681a70b0 con 0x7fba68103e30 2024-09-19T05:10:48.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.046+0000 7fba5ea00640 1 -- 172.21.15.43:0/391498493 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fba580046c0 con 0x7fba68103e30 2024-09-19T05:10:48.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.046+0000 7fba6fc71740 1 Processor -- start 2024-09-19T05:10:48.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.047+0000 7fba6fc71740 1 -- start start 2024-09-19T05:10:48.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.047+0000 7fba6fc71740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55961cb6be40 0x55961cb6c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:48.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.047+0000 7fba6fc71740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55961cb6c760 0x55961cb76ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:48.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.047+0000 7fba6fc71740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55961cb77020 0x55961cb79410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:48.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.047+0000 7fba6fc71740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55961cad23e0 con 0x55961cb6c760 2024-09-19T05:10:48.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.047+0000 7fba6fc71740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55961ca79d80 con 0x55961cb6be40 2024-09-19T05:10:48.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.047+0000 7fba6fc71740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55961cace3a0 con 0x55961cb77020 2024-09-19T05:10:48.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.047+0000 7fba65e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55961cb6c760 0x55961cb76ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:48.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.047+0000 7fba66800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55961cb6be40 0x55961cb6c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:48.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.047+0000 7fba65e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55961cb6c760 0x55961cb76ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56556/0 (socket says 172.21.15.43:56556) 2024-09-19T05:10:48.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.047+0000 7fba65e00640 1 -- 172.21.15.43:0/2591973190 learned_addr learned my addr 172.21.15.43:0/2591973190 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:48.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.048+0000 7fba5fe00640 1 --2- 172.21.15.43:0/2591973190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55961cb77020 0x55961cb79410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:48.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.048+0000 7fba66800640 1 -- 172.21.15.43:0/2591973190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55961cb77020 msgr2=0x55961cb79410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:48.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.048+0000 7fba66800640 1 --2- 172.21.15.43:0/2591973190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55961cb77020 0x55961cb79410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:48.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.048+0000 7fba66800640 1 -- 172.21.15.43:0/2591973190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55961cb6c760 msgr2=0x55961cb76ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:48.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.048+0000 7fba66800640 1 --2- 172.21.15.43:0/2591973190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55961cb6c760 0x55961cb76ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:48.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.048+0000 7fba66800640 1 -- 172.21.15.43:0/2591973190 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55961ca98ae0 con 0x55961cb6be40 2024-09-19T05:10:48.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.048+0000 7fba5fe00640 1 --2- 172.21.15.43:0/2591973190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55961cb77020 0x55961cb79410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:10:48.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.048+0000 7fba66800640 1 --2- 172.21.15.43:0/2591973190 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55961cb6be40 0x55961cb6c220 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fba60009f00 tx=0x7fba60011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=767071fa46d5e3a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:48.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.048+0000 7fba5ea00640 1 --2- 172.21.15.43:0/391498493 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fba4c0a33b0 0x7fba4c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:48.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.049+0000 7fba5ea00640 1 -- 172.21.15.43:0/391498493 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fba580d0150 con 0x7fba68103e30 2024-09-19T05:10:48.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.049+0000 7fba65e00640 1 --2- 172.21.15.43:0/391498493 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fba4c0a33b0 0x7fba4c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:48.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.049+0000 7fba57400640 1 -- 172.21.15.43:0/2591973190 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba60004d20 con 0x55961cb6be40 2024-09-19T05:10:48.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.049+0000 7fba57400640 1 -- 172.21.15.43:0/2591973190 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba60004050 con 0x55961cb6be40 2024-09-19T05:10:48.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.049+0000 7fba57400640 1 -- 172.21.15.43:0/2591973190 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba60005280 con 0x55961cb6be40 2024-09-19T05:10:48.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.050+0000 7fba6fc71740 1 -- 172.21.15.43:0/2591973190 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55961cb6be40 msgr2=0x55961cb6c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:48.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.050+0000 7fba6fc71740 1 --2- 172.21.15.43:0/2591973190 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55961cb6be40 0x55961cb6c220 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fba60009f00 tx=0x7fba60011810 comp rx=0 tx=0).stop 2024-09-19T05:10:48.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.050+0000 7fba65e00640 1 --2- 172.21.15.43:0/391498493 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fba4c0a33b0 0x7fba4c0a5870 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fba5809c370 tx=0x7fba58098040 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:48.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.050+0000 7fba6fc71740 1 -- 172.21.15.43:0/2591973190 shutdown_connections 2024-09-19T05:10:48.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.050+0000 7fba6fc71740 1 --2- 172.21.15.43:0/2591973190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55961cb77020 0x55961cb79410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:48.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.050+0000 7fba6fc71740 1 --2- 172.21.15.43:0/2591973190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55961cb6c760 0x55961cb76ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:48.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.050+0000 7fba6fc71740 1 --2- 172.21.15.43:0/2591973190 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55961cb6be40 0x55961cb6c220 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:48.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.050+0000 7fba6fc71740 1 -- 172.21.15.43:0/2591973190 >> 172.21.15.43:0/2591973190 conn(0x55961cb683d0 msgr2=0x55961cb687b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:48.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.050+0000 7fba6fc71740 1 -- 172.21.15.43:0/2591973190 shutdown_connections 2024-09-19T05:10:48.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.050+0000 7fba6fc71740 1 -- 172.21.15.43:0/2591973190 wait complete. 2024-09-19T05:10:48.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.050+0000 7fba6fc71740 1 -- start start 2024-09-19T05:10:48.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.050+0000 7fba6fc71740 1 Processor -- start 2024-09-19T05:10:48.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.051+0000 7fba6fc71740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55961cb6be40 0x55961cb73f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:48.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.051+0000 7fba6fc71740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55961cb6c760 0x55961cb74480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:48.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.051+0000 7fba6fc71740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55961cb77020 0x55961cb749c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:48.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.051+0000 7fba6fc71740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55961cb79bf0 con 0x55961cb6c760 2024-09-19T05:10:48.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.051+0000 7fba6fc71740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55961ca8d0f0 con 0x55961cb6be40 2024-09-19T05:10:48.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.051+0000 7fba6fc71740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55961cb79d70 con 0x55961cb77020 2024-09-19T05:10:48.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.051+0000 7fba6fc71740 10 client.0 _ll_register_callbacks cb 0x55961ca74bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:48.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.051+0000 7fba66800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55961cb6be40 0x55961cb73f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:48.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.051+0000 7fba5fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55961cb6c760 0x55961cb74480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:48.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.051+0000 7fba66800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55961cb6be40 0x55961cb73f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:54178/0 (socket says 172.21.15.43:54178) 2024-09-19T05:10:48.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.051+0000 7fba66800640 1 -- 172.21.15.43:0/2804603979 learned_addr learned my addr 172.21.15.43:0/2804603979 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:48.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.051+0000 7fba66800640 1 --2- 172.21.15.43:0/2804603979 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55961cb77020 0x55961cb749c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:48.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.052+0000 7fba5fe00640 1 -- 172.21.15.43:0/2804603979 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55961cb77020 msgr2=0x55961cb749c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:48.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.052+0000 7fba5fe00640 1 --2- 172.21.15.43:0/2804603979 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55961cb77020 0x55961cb749c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:48.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.052+0000 7fba5fe00640 1 -- 172.21.15.43:0/2804603979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55961cb6be40 msgr2=0x55961cb73f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:48.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.052+0000 7fba5fe00640 1 --2- 172.21.15.43:0/2804603979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55961cb6be40 0x55961cb73f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:48.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.052+0000 7fba5fe00640 1 -- 172.21.15.43:0/2804603979 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55961c8433c0 con 0x55961cb6c760 2024-09-19T05:10:48.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.052+0000 7fba5fe00640 1 --2- 172.21.15.43:0/2804603979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55961cb6c760 0x55961cb74480 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fba5002a9e0 tx=0x7fba50005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b0bb68c6fcd5991 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:48.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.052+0000 7fba4be00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:10:48.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.052+0000 7fba4be00640 1 -- 172.21.15.43:0/2804603979 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba50035040 con 0x55961cb6c760 2024-09-19T05:10:48.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.053+0000 7fba4be00640 1 -- 172.21.15.43:0/2804603979 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fba50004760 con 0x55961cb6c760 2024-09-19T05:10:48.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.053+0000 7fba6fc71740 1 -- 172.21.15.43:0/2804603979 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba60004050 con 0x55961cb6c760 2024-09-19T05:10:48.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.053+0000 7fba4be00640 1 -- 172.21.15.43:0/2804603979 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fba50004960 con 0x55961cb6c760 2024-09-19T05:10:48.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.053+0000 7fba6fc71740 10 client.16473 fetch_fsmap learned FSMap version 29 2024-09-19T05:10:48.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.053+0000 7fba6fc71740 1 -- 172.21.15.43:0/2804603979 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55961cb85c60 con 0x55961cb6c760 2024-09-19T05:10:48.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.053+0000 7fba4be00640 1 -- 172.21.15.43:0/2804603979 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba60004050 con 0x55961cb6c760 2024-09-19T05:10:48.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.053+0000 7fba4be00640 1 -- 172.21.15.43:0/2804603979 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fba50044020 con 0x55961cb6c760 2024-09-19T05:10:48.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.053+0000 7fba6fc71740 10 client.16473 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:10:48.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.053+0000 7fba6fc71740 10 client.16473 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-19T05:10:48.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.054+0000 7fba6fc71740 20 client.16473 populate_metadata read hostname 'smithi043' 2024-09-19T05:10:48.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.054+0000 7fba6fc71740 1 --2- 172.21.15.43:0/2804603979 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55961cb87070 0x55961cb89460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:48.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.054+0000 7fba6fc71740 4 client.16473 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:10:48.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.054+0000 7fba6fc71740 1 -- 172.21.15.43:0/2804603979 --> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55961cadf950 con 0x55961cb87070 2024-09-19T05:10:48.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.054+0000 7fba66800640 1 --2- 172.21.15.43:0/2804603979 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55961cb87070 0x55961cb89460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:48.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.054+0000 7fba66800640 1 --2- 172.21.15.43:0/2804603979 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55961cb87070 0x55961cb89460 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fc9bda5945ccbf9f server_cookie=1b50593b82e39b5e in_seq=0 out_seq=0 2024-09-19T05:10:48.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.054+0000 7fba4be00640 10 client.16473 ms_handle_connect on v2:172.21.15.43:6838/3404555030 2024-09-19T05:10:48.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.055+0000 7fba4be00640 1 -- 172.21.15.43:0/2804603979 <== mds.1 v2:172.21.15.43:6838/3404555030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55961cadf950 con 0x55961cb87070 2024-09-19T05:10:48.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.055+0000 7fba4be00640 10 client.16473 handle_command_reply: tid=0 2024-09-19T05:10:48.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.055+0000 7fba6fc71740 1 client.16473 shutdown 2024-09-19T05:10:48.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.055+0000 7fba6fc71740 1 -- 172.21.15.43:0/2804603979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55961cb6c760 msgr2=0x55961cb74480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:48.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.055+0000 7fba6fc71740 1 --2- 172.21.15.43:0/2804603979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55961cb6c760 0x55961cb74480 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fba5002a9e0 tx=0x7fba50005270 comp rx=0 tx=0).stop 2024-09-19T05:10:48.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.056+0000 7fba6fc71740 1 -- 172.21.15.43:0/2804603979 shutdown_connections 2024-09-19T05:10:48.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.056+0000 7fba6fc71740 1 --2- 172.21.15.43:0/2804603979 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55961cb87070 0x55961cb89460 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:48.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.056+0000 7fba6fc71740 1 --2- 172.21.15.43:0/2804603979 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55961cb77020 0x55961cb749c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:48.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.056+0000 7fba6fc71740 1 --2- 172.21.15.43:0/2804603979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55961cb6c760 0x55961cb74480 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:48.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.056+0000 7fba4be00640 0 client.16473 ms_handle_reset on v2:172.21.15.43:6838/3404555030 2024-09-19T05:10:48.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.056+0000 7fba6fc71740 1 --2- 172.21.15.43:0/2804603979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55961cb6be40 0x55961cb73f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:48.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.056+0000 7fba6fc71740 1 -- 172.21.15.43:0/2804603979 >> 172.21.15.43:0/2804603979 conn(0x55961cb683d0 msgr2=0x55961cb6e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:48.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.056+0000 7fba6fc71740 1 -- 172.21.15.43:0/2804603979 shutdown_connections 2024-09-19T05:10:48.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.056+0000 7fba6fc71740 1 -- 172.21.15.43:0/2804603979 wait complete. 2024-09-19T05:10:48.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.056+0000 7fba6fc71740 20 client.16473 trim_cache size 0 max 16384 2024-09-19T05:10:48.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.070+0000 7fba6fc71740 1 Processor -- start 2024-09-19T05:10:48.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.070+0000 7fba6fc71740 1 -- start start 2024-09-19T05:10:48.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.070+0000 7fba6fc71740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55961cb6be40 0x55961cb6e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:48.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.070+0000 7fba6fc71740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55961cb6c760 0x55961cb75eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:48.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.070+0000 7fba6fc71740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55961cb77020 0x55961cb8ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:48.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.070+0000 7fba6fc71740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fba681af890 con 0x55961cb77020 2024-09-19T05:10:48.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.070+0000 7fba6fc71740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fba681b13d0 con 0x55961cb6c760 2024-09-19T05:10:48.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.070+0000 7fba6fc71740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fba681aff60 con 0x55961cb6be40 2024-09-19T05:10:48.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.070+0000 7fba66800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55961cb77020 0x55961cb8ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:48.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.070+0000 7fba5fe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55961cb6c760 0x55961cb75eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:48.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.070+0000 7fba66800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55961cb77020 0x55961cb8ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56578/0 (socket says 172.21.15.43:56578) 2024-09-19T05:10:48.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.071+0000 7fba66800640 1 -- 172.21.15.43:0/1964842824 learned_addr learned my addr 172.21.15.43:0/1964842824 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:48.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.071+0000 7fba5fe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55961cb6c760 0x55961cb75eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:54180/0 (socket says 172.21.15.43:54180) 2024-09-19T05:10:48.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.071+0000 7fba66800640 1 --2- 172.21.15.43:0/1964842824 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55961cb6be40 0x55961cb6e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.071+0000 7fba5fe00640 1 -- 172.21.15.43:0/1964842824 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55961cb6be40 msgr2=0x55961cb6e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.071+0000 7fba5fe00640 1 --2- 172.21.15.43:0/1964842824 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55961cb6be40 0x55961cb6e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.071+0000 7fba5fe00640 1 -- 172.21.15.43:0/1964842824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55961cb77020 msgr2=0x55961cb8ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.071+0000 7fba5fe00640 1 --2- 172.21.15.43:0/1964842824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55961cb77020 0x55961cb8ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.071+0000 7fba5fe00640 1 -- 172.21.15.43:0/1964842824 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55961cb69480 con 0x55961cb6c760 2024-09-19T05:10:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.071+0000 7fba66800640 1 --2- 172.21.15.43:0/1964842824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55961cb77020 0x55961cb8ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:10:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.071+0000 7fba5fe00640 1 --2- 172.21.15.43:0/1964842824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55961cb6c760 0x55961cb75eb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fba50009900 tx=0x7fba50009830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6558fb97d478f07e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.071+0000 7fba56a00640 1 -- 172.21.15.43:0/1964842824 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba50054070 con 0x55961cb6c760 2024-09-19T05:10:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.071+0000 7fba56a00640 1 -- 172.21.15.43:0/1964842824 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba500043f0 con 0x55961cb6c760 2024-09-19T05:10:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.071+0000 7fba56a00640 1 -- 172.21.15.43:0/1964842824 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba50035070 con 0x55961cb6c760 2024-09-19T05:10:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.072+0000 7fba6fc71740 1 -- 172.21.15.43:0/1964842824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55961cb6c760 msgr2=0x55961cb75eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.072+0000 7fba6fc71740 1 --2- 172.21.15.43:0/1964842824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55961cb6c760 0x55961cb75eb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fba50009900 tx=0x7fba50009830 comp rx=0 tx=0).stop 2024-09-19T05:10:48.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.072+0000 7fba6fc71740 1 -- 172.21.15.43:0/1964842824 shutdown_connections 2024-09-19T05:10:48.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.072+0000 7fba6fc71740 1 --2- 172.21.15.43:0/1964842824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55961cb77020 0x55961cb8ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:48.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.072+0000 7fba6fc71740 1 --2- 172.21.15.43:0/1964842824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55961cb6c760 0x55961cb75eb0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:48.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.072+0000 7fba6fc71740 1 --2- 172.21.15.43:0/1964842824 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55961cb6be40 0x55961cb6e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:48.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.072+0000 7fba6fc71740 1 -- 172.21.15.43:0/1964842824 >> 172.21.15.43:0/1964842824 conn(0x55961cb683d0 msgr2=0x55961cb757a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:48.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.072+0000 7fba6fc71740 1 -- 172.21.15.43:0/1964842824 shutdown_connections 2024-09-19T05:10:48.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.072+0000 7fba6fc71740 1 -- 172.21.15.43:0/1964842824 wait complete. 2024-09-19T05:10:48.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.073+0000 7fba6fc71740 1 -- start start 2024-09-19T05:10:48.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.073+0000 7fba6fc71740 1 Processor -- start 2024-09-19T05:10:48.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.074+0000 7fba6fc71740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55961cb6be40 0x55961cb7f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:48.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.074+0000 7fba6fc71740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55961cb6c760 0x55961cb7f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:48.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.074+0000 7fba6fc71740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55961cb77020 0x55961cb7a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:48.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.074+0000 7fba6fc71740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fba681b0d00 con 0x55961cb77020 2024-09-19T05:10:48.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.074+0000 7fba6fc71740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fba681b0630 con 0x55961cb6c760 2024-09-19T05:10:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.074+0000 7fba6fc71740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fba681ad870 con 0x55961cb6be40 2024-09-19T05:10:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.074+0000 7fba6fc71740 10 client.0 _ll_register_callbacks cb 0x7fba50009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.074+0000 7fba66800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55961cb6be40 0x55961cb7f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.074+0000 7fba5fe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55961cb6c760 0x55961cb7f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.074+0000 7fba66800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55961cb6be40 0x55961cb7f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:49568/0 (socket says 172.21.15.43:49568) 2024-09-19T05:10:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.074+0000 7fba66800640 1 -- 172.21.15.43:0/3344677939 learned_addr learned my addr 172.21.15.43:0/3344677939 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.074+0000 7fba66800640 1 --2- 172.21.15.43:0/3344677939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55961cb77020 0x55961cb7a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.074+0000 7fba5fe00640 1 -- 172.21.15.43:0/3344677939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55961cb6be40 msgr2=0x55961cb7f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.075+0000 7fba5fe00640 1 --2- 172.21.15.43:0/3344677939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55961cb6be40 0x55961cb7f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.075+0000 7fba5fe00640 1 -- 172.21.15.43:0/3344677939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55961cb77020 msgr2=0x55961cb7a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.075+0000 7fba5fe00640 1 --2- 172.21.15.43:0/3344677939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55961cb77020 0x55961cb7a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.075+0000 7fba5fe00640 1 -- 172.21.15.43:0/3344677939 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55961cb7af40 con 0x55961cb6c760 2024-09-19T05:10:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.075+0000 7fba5fe00640 1 --2- 172.21.15.43:0/3344677939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55961cb6c760 0x55961cb7f850 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fba50000c00 tx=0x7fba50003c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=479298bd348a2da5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.075+0000 7fba4be00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:10:48.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.075+0000 7fba4be00640 1 -- 172.21.15.43:0/3344677939 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba50035070 con 0x55961cb6c760 2024-09-19T05:10:48.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.075+0000 7fba6fc71740 1 -- 172.21.15.43:0/3344677939 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba500043f0 con 0x55961cb6c760 2024-09-19T05:10:48.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.075+0000 7fba4be00640 1 -- 172.21.15.43:0/3344677939 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fba50004760 con 0x55961cb6c760 2024-09-19T05:10:48.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.076+0000 7fba4be00640 1 -- 172.21.15.43:0/3344677939 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fba50054070 con 0x55961cb6c760 2024-09-19T05:10:48.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.076+0000 7fba6fc71740 10 client.26513 fetch_fsmap learned FSMap version 29 2024-09-19T05:10:48.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.076+0000 7fba6fc71740 1 -- 172.21.15.43:0/3344677939 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55961cbf18c0 con 0x55961cb6c760 2024-09-19T05:10:48.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.076+0000 7fba4be00640 1 -- 172.21.15.43:0/3344677939 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba500043f0 con 0x55961cb6c760 2024-09-19T05:10:48.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.076+0000 7fba4be00640 1 -- 172.21.15.43:0/3344677939 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fba50044040 con 0x55961cb6c760 2024-09-19T05:10:48.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.076+0000 7fba6fc71740 10 client.26513 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:10:48.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.076+0000 7fba6fc71740 10 client.26513 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-19T05:10:48.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.076+0000 7fba6fc71740 20 client.26513 populate_metadata read hostname 'smithi043' 2024-09-19T05:10:48.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.076+0000 7fba6fc71740 1 --2- 172.21.15.43:0/3344677939 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55961cbf1f30 0x55961cbf4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:48.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.076+0000 7fba6fc71740 4 client.26513 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:10:48.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.076+0000 7fba6fc71740 1 -- 172.21.15.43:0/3344677939 --> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55961cad9e60 con 0x55961cbf1f30 2024-09-19T05:10:48.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.076+0000 7fba66800640 1 --2- 172.21.15.43:0/3344677939 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55961cbf1f30 0x55961cbf4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:48.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.077+0000 7fba66800640 1 --2- 172.21.15.43:0/3344677939 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55961cbf1f30 0x55961cbf4350 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=418fb39f7ca64d server_cookie=197688af0435ad1 in_seq=0 out_seq=0 2024-09-19T05:10:48.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.077+0000 7fba4be00640 10 client.26513 ms_handle_connect on v2:172.21.15.43:6838/3404555030 2024-09-19T05:10:48.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.077+0000 7fba4be00640 1 -- 172.21.15.43:0/3344677939 <== mds.1 v2:172.21.15.43:6838/3404555030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55961cad9e60 con 0x55961cbf1f30 2024-09-19T05:10:48.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.077+0000 7fba4be00640 10 client.26513 handle_command_reply: tid=0 2024-09-19T05:10:48.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.077+0000 7fba6fc71740 1 client.26513 shutdown 2024-09-19T05:10:48.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.077+0000 7fba6fc71740 1 -- 172.21.15.43:0/3344677939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55961cb6c760 msgr2=0x55961cb7f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:48.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.077+0000 7fba6fc71740 1 --2- 172.21.15.43:0/3344677939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55961cb6c760 0x55961cb7f850 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fba50000c00 tx=0x7fba50003c60 comp rx=0 tx=0).stop 2024-09-19T05:10:48.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.077+0000 7fba6fc71740 1 -- 172.21.15.43:0/3344677939 shutdown_connections 2024-09-19T05:10:48.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.077+0000 7fba6fc71740 1 --2- 172.21.15.43:0/3344677939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55961cb77020 0x55961cb7a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:48.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.077+0000 7fba6fc71740 1 --2- 172.21.15.43:0/3344677939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55961cb6c760 0x55961cb7f850 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:48.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.077+0000 7fba6fc71740 1 --2- 172.21.15.43:0/3344677939 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55961cbf1f30 0x55961cbf4350 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:48.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.077+0000 7fba6fc71740 1 --2- 172.21.15.43:0/3344677939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55961cb6be40 0x55961cb7f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:48.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.077+0000 7fba6fc71740 1 -- 172.21.15.43:0/3344677939 >> 172.21.15.43:0/3344677939 conn(0x55961cb683d0 msgr2=0x55961cb75fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:48.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.077+0000 7fba4be00640 0 client.26513 ms_handle_reset on v2:172.21.15.43:6838/3404555030 2024-09-19T05:10:48.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.078+0000 7fba6fc71740 1 -- 172.21.15.43:0/3344677939 shutdown_connections 2024-09-19T05:10:48.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.078+0000 7fba6fc71740 1 -- 172.21.15.43:0/3344677939 wait complete. 2024-09-19T05:10:48.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.078+0000 7fba6fc71740 20 client.26513 trim_cache size 0 max 16384 2024-09-19T05:10:48.083 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:10:48.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.078+0000 7fba67200640 1 -- 172.21.15.43:0/391498493 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fba4c0a33b0 msgr2=0x7fba4c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:48.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.078+0000 7fba67200640 1 --2- 172.21.15.43:0/391498493 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fba4c0a33b0 0x7fba4c0a5870 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fba5809c370 tx=0x7fba58098040 comp rx=0 tx=0).stop 2024-09-19T05:10:48.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.078+0000 7fba67200640 1 -- 172.21.15.43:0/391498493 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba68103e30 msgr2=0x7fba6819a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:48.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.078+0000 7fba67200640 1 --2- 172.21.15.43:0/391498493 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba68103e30 0x7fba6819a530 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fba5800e9f0 tx=0x7fba5800edf0 comp rx=0 tx=0).stop 2024-09-19T05:10:48.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.078+0000 7fba67200640 1 -- 172.21.15.43:0/391498493 shutdown_connections 2024-09-19T05:10:48.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.078+0000 7fba67200640 1 --2- 172.21.15.43:0/391498493 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba68108f60 0x7fba6819ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:48.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.078+0000 7fba67200640 1 --2- 172.21.15.43:0/391498493 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fba4c0a33b0 0x7fba4c0a5870 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:48.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.078+0000 7fba67200640 1 --2- 172.21.15.43:0/391498493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba681047e0 0x7fba6819aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:48.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.078+0000 7fba67200640 1 --2- 172.21.15.43:0/391498493 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba68103e30 0x7fba6819a530 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:48.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.079+0000 7fba67200640 1 -- 172.21.15.43:0/391498493 >> 172.21.15.43:0/391498493 conn(0x7fba680ff910 msgr2=0x7fba68100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:48.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.079+0000 7fba67200640 1 -- 172.21.15.43:0/391498493 shutdown_connections 2024-09-19T05:10:48.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:48.079+0000 7fba67200640 1 -- 172.21.15.43:0/391498493 wait complete. 2024-09-19T05:10:48.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:47 smithi151 ceph-mon[35425]: pgmap v187: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:10:48.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:47 smithi096 ceph-mon[35085]: pgmap v187: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:10:48.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:47 smithi043 ceph-mon[30800]: pgmap v187: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:10:48.870 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T05:10:49.250 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:10:49.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.841+0000 7f9601c00640 1 -- 172.21.15.43:0/96785474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95fc1047e0 msgr2=0x7f95fc108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:49.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.841+0000 7f9601c00640 1 --2- 172.21.15.43:0/96785474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95fc1047e0 0x7f95fc108830 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f95e8009900 tx=0x7f95e802a3f0 comp rx=0 tx=0).stop 2024-09-19T05:10:49.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.842+0000 7f9601c00640 1 -- 172.21.15.43:0/96785474 shutdown_connections 2024-09-19T05:10:49.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.842+0000 7f9601c00640 1 --2- 172.21.15.43:0/96785474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95fc108f60 0x7f95fc110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:49.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.842+0000 7f9601c00640 1 --2- 172.21.15.43:0/96785474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95fc1047e0 0x7f95fc108830 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:49.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.842+0000 7f9601c00640 1 --2- 172.21.15.43:0/96785474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95fc103e30 0x7f95fc104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:49.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.842+0000 7f9601c00640 1 -- 172.21.15.43:0/96785474 >> 172.21.15.43:0/96785474 conn(0x7f95fc0ff8d0 msgr2=0x7f95fc101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:49.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.842+0000 7f9601c00640 1 -- 172.21.15.43:0/96785474 shutdown_connections 2024-09-19T05:10:49.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.842+0000 7f9601c00640 1 -- 172.21.15.43:0/96785474 wait complete. 2024-09-19T05:10:49.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.843+0000 7f9601c00640 1 Processor -- start 2024-09-19T05:10:49.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.843+0000 7f9601c00640 1 -- start start 2024-09-19T05:10:49.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.843+0000 7f9601c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95fc103e30 0x7f95fc19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:49.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.843+0000 7f9601c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95fc1047e0 0x7f95fc19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:49.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.843+0000 7f9601c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95fc108f60 0x7f95fc19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:49.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.843+0000 7f9601c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f95fc113fc0 con 0x7f95fc108f60 2024-09-19T05:10:49.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.843+0000 7f9601c00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f95fc113e40 con 0x7f95fc1047e0 2024-09-19T05:10:49.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.843+0000 7f9601c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f95fc114140 con 0x7f95fc103e30 2024-09-19T05:10:49.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.843+0000 7f9601200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95fc108f60 0x7f95fc19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:49.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.843+0000 7f9601200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95fc108f60 0x7f95fc19ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56610/0 (socket says 172.21.15.43:56610) 2024-09-19T05:10:49.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.843+0000 7f9601200640 1 -- 172.21.15.43:0/3098174288 learned_addr learned my addr 172.21.15.43:0/3098174288 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:49.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.844+0000 7f95fb400640 1 --2- 172.21.15.43:0/3098174288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95fc1047e0 0x7f95fc19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:49.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.844+0000 7f95fbe00640 1 --2- 172.21.15.43:0/3098174288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95fc103e30 0x7f95fc19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:49.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.844+0000 7f9601200640 1 -- 172.21.15.43:0/3098174288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95fc103e30 msgr2=0x7f95fc19a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:49.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.844+0000 7f9601200640 1 --2- 172.21.15.43:0/3098174288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95fc103e30 0x7f95fc19a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:49.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.844+0000 7f9601200640 1 -- 172.21.15.43:0/3098174288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95fc1047e0 msgr2=0x7f95fc19a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:49.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.844+0000 7f9601200640 1 --2- 172.21.15.43:0/3098174288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95fc1047e0 0x7f95fc19a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:49.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.844+0000 7f9601200640 1 -- 172.21.15.43:0/3098174288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95fc19f410 con 0x7f95fc108f60 2024-09-19T05:10:49.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.844+0000 7f9601200640 1 --2- 172.21.15.43:0/3098174288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95fc108f60 0x7f95fc19ed30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f95f400eb10 tx=0x7f95f400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:49.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.844+0000 7f95f8c00640 1 -- 172.21.15.43:0/3098174288 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f4004b20 con 0x7f95fc108f60 2024-09-19T05:10:49.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.845+0000 7f95f8c00640 1 -- 172.21.15.43:0/3098174288 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95f4004cc0 con 0x7f95fc108f60 2024-09-19T05:10:49.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.845+0000 7f9601c00640 1 -- 172.21.15.43:0/3098174288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95fc19f700 con 0x7f95fc108f60 2024-09-19T05:10:49.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.845+0000 7f95f8c00640 1 -- 172.21.15.43:0/3098174288 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f4010760 con 0x7f95fc108f60 2024-09-19T05:10:49.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.845+0000 7f9601c00640 1 -- 172.21.15.43:0/3098174288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95fc1a6fe0 con 0x7f95fc108f60 2024-09-19T05:10:49.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.848+0000 7f9605c87740 1 Processor -- start 2024-09-19T05:10:49.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.848+0000 7f95f8c00640 1 -- 172.21.15.43:0/3098174288 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f95f4005020 con 0x7f95fc108f60 2024-09-19T05:10:49.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.848+0000 7f9605c87740 1 -- start start 2024-09-19T05:10:49.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.848+0000 7f9605c87740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c1f735e40 0x559c1f736220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:49.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.848+0000 7f9605c87740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c1f736760 0x559c1f740ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:49.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.848+0000 7f9605c87740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c1f741020 0x559c1f743410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:49.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.848+0000 7f9605c87740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559c1f69c3e0 con 0x559c1f741020 2024-09-19T05:10:49.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.848+0000 7f9605c87740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559c1f643d80 con 0x559c1f735e40 2024-09-19T05:10:49.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.848+0000 7f9605c87740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559c1f6983a0 con 0x559c1f736760 2024-09-19T05:10:49.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.849+0000 7f95fb400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c1f735e40 0x559c1f736220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:49.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.849+0000 7f95fb400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c1f735e40 0x559c1f736220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:54224/0 (socket says 172.21.15.43:54224) 2024-09-19T05:10:49.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.849+0000 7f95fb400640 1 -- 172.21.15.43:0/588030007 learned_addr learned my addr 172.21.15.43:0/588030007 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:49.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.849+0000 7f95fbe00640 1 --2- 172.21.15.43:0/588030007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c1f736760 0x559c1f740ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:49.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.849+0000 7f95fb400640 1 --2- 172.21.15.43:0/588030007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c1f741020 0x559c1f743410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:49.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.849+0000 7f95fbe00640 1 -- 172.21.15.43:0/588030007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c1f735e40 msgr2=0x559c1f736220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:49.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.849+0000 7f95fbe00640 1 --2- 172.21.15.43:0/588030007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c1f735e40 0x559c1f736220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:49.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.849+0000 7f95fbe00640 1 -- 172.21.15.43:0/588030007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c1f741020 msgr2=0x559c1f743410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:49.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.849+0000 7f95fbe00640 1 --2- 172.21.15.43:0/588030007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c1f741020 0x559c1f743410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:49.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.849+0000 7f95fbe00640 1 -- 172.21.15.43:0/588030007 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c1f662ae0 con 0x559c1f736760 2024-09-19T05:10:49.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.849+0000 7f95f8c00640 1 --2- 172.21.15.43:0/3098174288 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f95e00a33b0 0x7f95e00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:49.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.849+0000 7f95f8c00640 1 -- 172.21.15.43:0/3098174288 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f95f40d03a0 con 0x7f95fc108f60 2024-09-19T05:10:49.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.850+0000 7f95fbe00640 1 --2- 172.21.15.43:0/588030007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c1f736760 0x559c1f740ae0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f95ec009c20 tx=0x7f95ec011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38158113b745489c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:49.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.850+0000 7f95fb400640 1 --2- 172.21.15.43:0/3098174288 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f95e00a33b0 0x7f95e00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:49.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.850+0000 7f95f1600640 1 -- 172.21.15.43:0/588030007 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ec004050 con 0x559c1f736760 2024-09-19T05:10:49.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.850+0000 7f95f1600640 1 -- 172.21.15.43:0/588030007 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95ec005500 con 0x559c1f736760 2024-09-19T05:10:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.850+0000 7f95f1600640 1 -- 172.21.15.43:0/588030007 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ec004e00 con 0x559c1f736760 2024-09-19T05:10:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.851+0000 7f9605c87740 1 -- 172.21.15.43:0/588030007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c1f736760 msgr2=0x559c1f740ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.851+0000 7f95fb400640 1 --2- 172.21.15.43:0/3098174288 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f95e00a33b0 0x7f95e00a5870 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x559c1f733750 tx=0x7f95e80058d0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.851+0000 7f9605c87740 1 --2- 172.21.15.43:0/588030007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c1f736760 0x559c1f740ae0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f95ec009c20 tx=0x7f95ec011810 comp rx=0 tx=0).stop 2024-09-19T05:10:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.851+0000 7f9605c87740 1 -- 172.21.15.43:0/588030007 shutdown_connections 2024-09-19T05:10:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.851+0000 7f9605c87740 1 --2- 172.21.15.43:0/588030007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c1f741020 0x559c1f743410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.851+0000 7f9605c87740 1 --2- 172.21.15.43:0/588030007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c1f736760 0x559c1f740ae0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.851+0000 7f9605c87740 1 --2- 172.21.15.43:0/588030007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c1f735e40 0x559c1f736220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.851+0000 7f9605c87740 1 -- 172.21.15.43:0/588030007 >> 172.21.15.43:0/588030007 conn(0x559c1f7323d0 msgr2=0x559c1f7327b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.851+0000 7f9605c87740 1 -- 172.21.15.43:0/588030007 shutdown_connections 2024-09-19T05:10:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.851+0000 7f9605c87740 1 -- 172.21.15.43:0/588030007 wait complete. 2024-09-19T05:10:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.851+0000 7f9605c87740 1 -- start start 2024-09-19T05:10:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.852+0000 7f9605c87740 1 Processor -- start 2024-09-19T05:10:49.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.852+0000 7f9605c87740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c1f735e40 0x559c1f73de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:49.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.852+0000 7f9605c87740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c1f736760 0x559c1f73e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:49.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.852+0000 7f9605c87740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c1f741020 0x559c1f73e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:49.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.852+0000 7f9605c87740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559c1f743bf0 con 0x559c1f741020 2024-09-19T05:10:49.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.852+0000 7f9605c87740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559c1f6570f0 con 0x559c1f735e40 2024-09-19T05:10:49.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.852+0000 7f9605c87740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559c1f743d70 con 0x559c1f736760 2024-09-19T05:10:49.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.852+0000 7f9605c87740 10 client.0 _ll_register_callbacks cb 0x559c1f63ebd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:49.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.852+0000 7f9601200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c1f741020 0x559c1f73e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:49.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.852+0000 7f95fb400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c1f736760 0x559c1f73e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:49.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.852+0000 7f95fbe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c1f735e40 0x559c1f73de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:49.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.852+0000 7f9601200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c1f741020 0x559c1f73e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56630/0 (socket says 172.21.15.43:56630) 2024-09-19T05:10:49.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.852+0000 7f95fb400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c1f736760 0x559c1f73e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:49616/0 (socket says 172.21.15.43:49616) 2024-09-19T05:10:49.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.852+0000 7f9601200640 1 -- 172.21.15.43:0/2360373892 learned_addr learned my addr 172.21.15.43:0/2360373892 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:49.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.853+0000 7f9601200640 1 -- 172.21.15.43:0/2360373892 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c1f736760 msgr2=0x559c1f73e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:49.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.853+0000 7f9601200640 1 --2- 172.21.15.43:0/2360373892 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c1f736760 0x559c1f73e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:49.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.853+0000 7f9601200640 1 -- 172.21.15.43:0/2360373892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c1f735e40 msgr2=0x559c1f73de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:49.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.853+0000 7f9601200640 1 --2- 172.21.15.43:0/2360373892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c1f735e40 0x559c1f73de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:49.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.853+0000 7f9601200640 1 -- 172.21.15.43:0/2360373892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c1f40d3c0 con 0x559c1f741020 2024-09-19T05:10:49.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.853+0000 7f9601200640 1 --2- 172.21.15.43:0/2360373892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c1f741020 0x559c1f73e900 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f95f4000f80 tx=0x7f95f409c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42fe65118b4276e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:49.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.853+0000 7f95e6000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:10:49.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.853+0000 7f95e6000640 1 -- 172.21.15.43:0/2360373892 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f4005510 con 0x559c1f741020 2024-09-19T05:10:49.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.853+0000 7f95e6000640 1 -- 172.21.15.43:0/2360373892 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f95f409ba80 con 0x559c1f741020 2024-09-19T05:10:49.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.853+0000 7f9605c87740 1 -- 172.21.15.43:0/2360373892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95ec005500 con 0x559c1f741020 2024-09-19T05:10:49.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.853+0000 7f95e6000640 1 -- 172.21.15.43:0/2360373892 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f95f40a09e0 con 0x559c1f741020 2024-09-19T05:10:49.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.854+0000 7f9605c87740 10 client.16491 fetch_fsmap learned FSMap version 29 2024-09-19T05:10:49.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.854+0000 7f9605c87740 1 -- 172.21.15.43:0/2360373892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559c1f74fbe0 con 0x559c1f741020 2024-09-19T05:10:49.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.854+0000 7f95e6000640 1 -- 172.21.15.43:0/2360373892 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95ec005500 con 0x559c1f741020 2024-09-19T05:10:49.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.854+0000 7f95e6000640 1 -- 172.21.15.43:0/2360373892 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f95f409b610 con 0x559c1f741020 2024-09-19T05:10:49.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.854+0000 7f9605c87740 10 client.16491 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:10:49.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.854+0000 7f9605c87740 10 client.16491 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:10:49.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.854+0000 7f9605c87740 20 client.16491 populate_metadata read hostname 'smithi043' 2024-09-19T05:10:49.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.854+0000 7f9605c87740 1 --2- 172.21.15.43:0/2360373892 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x559c1f751040 0x559c1f7534e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:49.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.854+0000 7f9605c87740 4 client.16491 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:10:49.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.854+0000 7f9605c87740 1 -- 172.21.15.43:0/2360373892 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c1f6a9950 con 0x559c1f751040 2024-09-19T05:10:49.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.855+0000 7f95fbe00640 1 --2- 172.21.15.43:0/2360373892 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x559c1f751040 0x559c1f7534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:49.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.855+0000 7f95fbe00640 1 --2- 172.21.15.43:0/2360373892 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x559c1f751040 0x559c1f7534e0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f7e6f810d9c445d server_cookie=3257324d2bf2a831 in_seq=0 out_seq=0 2024-09-19T05:10:49.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.855+0000 7f95e6000640 10 client.16491 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:10:49.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.856+0000 7f95e6000640 1 -- 172.21.15.43:0/2360373892 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559c1f6a9950 con 0x559c1f751040 2024-09-19T05:10:49.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.856+0000 7f95e6000640 10 client.16491 handle_command_reply: tid=0 2024-09-19T05:10:49.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.856+0000 7f9605c87740 1 client.16491 shutdown 2024-09-19T05:10:49.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.856+0000 7f9605c87740 1 -- 172.21.15.43:0/2360373892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c1f741020 msgr2=0x559c1f73e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:49.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.856+0000 7f9605c87740 1 --2- 172.21.15.43:0/2360373892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c1f741020 0x559c1f73e900 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f95f4000f80 tx=0x7f95f409c230 comp rx=0 tx=0).stop 2024-09-19T05:10:49.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.856+0000 7f9605c87740 1 -- 172.21.15.43:0/2360373892 shutdown_connections 2024-09-19T05:10:49.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.856+0000 7f9605c87740 1 --2- 172.21.15.43:0/2360373892 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x559c1f751040 0x559c1f7534e0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:49.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.856+0000 7f9605c87740 1 --2- 172.21.15.43:0/2360373892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c1f741020 0x559c1f73e900 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:49.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.856+0000 7f9605c87740 1 --2- 172.21.15.43:0/2360373892 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c1f736760 0x559c1f73e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:49.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.856+0000 7f9605c87740 1 --2- 172.21.15.43:0/2360373892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c1f735e40 0x559c1f73de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:49.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.856+0000 7f9605c87740 1 -- 172.21.15.43:0/2360373892 >> 172.21.15.43:0/2360373892 conn(0x559c1f7323d0 msgr2=0x559c1f7380b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:49.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.856+0000 7f95e6000640 0 client.16491 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:10:49.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.856+0000 7f9605c87740 1 -- 172.21.15.43:0/2360373892 shutdown_connections 2024-09-19T05:10:49.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.856+0000 7f9605c87740 1 -- 172.21.15.43:0/2360373892 wait complete. 2024-09-19T05:10:49.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.857+0000 7f9605c87740 20 client.16491 trim_cache size 0 max 16384 2024-09-19T05:10:49.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.870+0000 7f9605c87740 1 Processor -- start 2024-09-19T05:10:49.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.870+0000 7f9605c87740 1 -- start start 2024-09-19T05:10:49.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.870+0000 7f9605c87740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c1f735e40 0x559c1f7388a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:49.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.870+0000 7f9605c87740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c1f736760 0x559c1f72d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:49.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.870+0000 7f9605c87740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c1f741020 0x559c1f73fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:49.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.870+0000 7f9605c87740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f95fc1af940 con 0x559c1f736760 2024-09-19T05:10:49.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.871+0000 7f9605c87740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f95fc1b1480 con 0x559c1f735e40 2024-09-19T05:10:49.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.871+0000 7f9605c87740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f95fc1b0010 con 0x559c1f741020 2024-09-19T05:10:49.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.871+0000 7f95fb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c1f736760 0x559c1f72d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:49.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.871+0000 7f95fb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c1f736760 0x559c1f72d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56636/0 (socket says 172.21.15.43:56636) 2024-09-19T05:10:49.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.871+0000 7f95fb400640 1 -- 172.21.15.43:0/1929704316 learned_addr learned my addr 172.21.15.43:0/1929704316 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:49.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.871+0000 7f9601200640 1 --2- 172.21.15.43:0/1929704316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c1f741020 0x559c1f73fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:49.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.871+0000 7f95fbe00640 1 --2- 172.21.15.43:0/1929704316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c1f735e40 0x559c1f7388a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:49.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.871+0000 7f95fb400640 1 -- 172.21.15.43:0/1929704316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c1f741020 msgr2=0x559c1f73fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:49.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.871+0000 7f95fb400640 1 --2- 172.21.15.43:0/1929704316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c1f741020 0x559c1f73fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:49.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.871+0000 7f95fb400640 1 -- 172.21.15.43:0/1929704316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c1f735e40 msgr2=0x559c1f7388a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:49.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.871+0000 7f95fb400640 1 --2- 172.21.15.43:0/1929704316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c1f735e40 0x559c1f7388a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:49.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.871+0000 7f95fb400640 1 -- 172.21.15.43:0/1929704316 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c1f73ce30 con 0x559c1f736760 2024-09-19T05:10:49.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.871+0000 7f95fbe00640 1 --2- 172.21.15.43:0/1929704316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c1f735e40 0x559c1f7388a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:10:49.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.872+0000 7f95fb400640 1 --2- 172.21.15.43:0/1929704316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c1f736760 0x559c1f72d300 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f95e8009230 tx=0x7f95e8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78fc5ec39d737501 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:49.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.872+0000 7f95f0c00640 1 -- 172.21.15.43:0/1929704316 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e8035040 con 0x559c1f736760 2024-09-19T05:10:49.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.872+0000 7f95f0c00640 1 -- 172.21.15.43:0/1929704316 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95e8003cd0 con 0x559c1f736760 2024-09-19T05:10:49.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.872+0000 7f95f0c00640 1 -- 172.21.15.43:0/1929704316 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e803de50 con 0x559c1f736760 2024-09-19T05:10:49.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.872+0000 7f9605c87740 1 -- 172.21.15.43:0/1929704316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c1f736760 msgr2=0x559c1f72d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:49.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.872+0000 7f9605c87740 1 --2- 172.21.15.43:0/1929704316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c1f736760 0x559c1f72d300 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f95e8009230 tx=0x7f95e8009260 comp rx=0 tx=0).stop 2024-09-19T05:10:49.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.873+0000 7f9605c87740 1 -- 172.21.15.43:0/1929704316 shutdown_connections 2024-09-19T05:10:49.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.873+0000 7f9605c87740 1 --2- 172.21.15.43:0/1929704316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c1f741020 0x559c1f73fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:49.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.873+0000 7f9605c87740 1 --2- 172.21.15.43:0/1929704316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c1f736760 0x559c1f72d300 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:49.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.873+0000 7f9605c87740 1 --2- 172.21.15.43:0/1929704316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c1f735e40 0x559c1f7388a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:49.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.873+0000 7f9605c87740 1 -- 172.21.15.43:0/1929704316 >> 172.21.15.43:0/1929704316 conn(0x559c1f7323d0 msgr2=0x559c1f73f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:49.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.873+0000 7f9605c87740 1 -- 172.21.15.43:0/1929704316 shutdown_connections 2024-09-19T05:10:49.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.873+0000 7f9605c87740 1 -- 172.21.15.43:0/1929704316 wait complete. 2024-09-19T05:10:49.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.873+0000 7f9605c87740 1 -- start start 2024-09-19T05:10:49.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.873+0000 7f9605c87740 1 Processor -- start 2024-09-19T05:10:49.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.874+0000 7f9605c87740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c1f735e40 0x559c1f7497f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:49.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.874+0000 7f9605c87740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c1f736760 0x559c1f744b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:49.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.874+0000 7f9605c87740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c1f741020 0x559c1f7450a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:49.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.874+0000 7f9605c87740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f95fc1b0db0 con 0x559c1f735e40 2024-09-19T05:10:49.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.874+0000 7f9605c87740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f95fc1b06e0 con 0x559c1f741020 2024-09-19T05:10:49.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.874+0000 7f9605c87740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f95fc1ad920 con 0x559c1f736760 2024-09-19T05:10:49.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.874+0000 7f9605c87740 10 client.0 _ll_register_callbacks cb 0x7f95f4098940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:49.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.874+0000 7f95fbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c1f735e40 0x559c1f7497f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:49.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.874+0000 7f95fbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c1f735e40 0x559c1f7497f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56650/0 (socket says 172.21.15.43:56650) 2024-09-19T05:10:49.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.874+0000 7f9601200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c1f741020 0x559c1f7450a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:49.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.874+0000 7f95fbe00640 1 -- 172.21.15.43:0/2502914195 learned_addr learned my addr 172.21.15.43:0/2502914195 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:49.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.874+0000 7f95fb400640 1 --2- 172.21.15.43:0/2502914195 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c1f736760 0x559c1f744b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:49.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.875+0000 7f95fbe00640 1 -- 172.21.15.43:0/2502914195 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c1f736760 msgr2=0x559c1f744b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:49.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.875+0000 7f95fbe00640 1 --2- 172.21.15.43:0/2502914195 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c1f736760 0x559c1f744b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:49.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.875+0000 7f95fbe00640 1 -- 172.21.15.43:0/2502914195 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c1f741020 msgr2=0x559c1f7450a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:49.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.875+0000 7f95fbe00640 1 --2- 172.21.15.43:0/2502914195 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c1f741020 0x559c1f7450a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:49.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.875+0000 7f95fbe00640 1 -- 172.21.15.43:0/2502914195 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c1f7458d0 con 0x559c1f735e40 2024-09-19T05:10:49.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.875+0000 7f95fbe00640 1 --2- 172.21.15.43:0/2502914195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c1f735e40 0x559c1f7497f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f95ec0099b0 tx=0x7f95ec005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36b24dcb0d5a1915 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:49.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.875+0000 7f95e6000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:10:49.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.875+0000 7f95e6000640 1 -- 172.21.15.43:0/2502914195 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ec004240 con 0x559c1f735e40 2024-09-19T05:10:49.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.875+0000 7f9605c87740 1 -- 172.21.15.43:0/2502914195 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95e8003cd0 con 0x559c1f735e40 2024-09-19T05:10:49.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.875+0000 7f95e6000640 1 -- 172.21.15.43:0/2502914195 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f95ec0043e0 con 0x559c1f735e40 2024-09-19T05:10:49.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.876+0000 7f95e6000640 1 -- 172.21.15.43:0/2502914195 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f95ec0035f0 con 0x559c1f735e40 2024-09-19T05:10:49.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.876+0000 7f9605c87740 10 client.16497 fetch_fsmap learned FSMap version 29 2024-09-19T05:10:49.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.876+0000 7f9605c87740 1 -- 172.21.15.43:0/2502914195 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559c1f7bbe20 con 0x559c1f735e40 2024-09-19T05:10:49.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.876+0000 7f95e6000640 1 -- 172.21.15.43:0/2502914195 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95e8003cd0 con 0x559c1f735e40 2024-09-19T05:10:49.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.876+0000 7f95e6000640 1 -- 172.21.15.43:0/2502914195 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f95ec0376b0 con 0x559c1f735e40 2024-09-19T05:10:49.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.876+0000 7f9605c87740 10 client.16497 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:10:49.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.876+0000 7f9605c87740 10 client.16497 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:10:49.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.876+0000 7f9605c87740 20 client.16497 populate_metadata read hostname 'smithi043' 2024-09-19T05:10:49.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.876+0000 7f9605c87740 1 --2- 172.21.15.43:0/2502914195 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x559c1f7bc360 0x559c1f7be750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:49.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.876+0000 7f9605c87740 4 client.16497 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:10:49.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.876+0000 7f9605c87740 1 -- 172.21.15.43:0/2502914195 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559c1f6a3e60 con 0x559c1f7bc360 2024-09-19T05:10:49.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.877+0000 7f95fb400640 1 --2- 172.21.15.43:0/2502914195 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x559c1f7bc360 0x559c1f7be750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:49.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.877+0000 7f95fb400640 1 --2- 172.21.15.43:0/2502914195 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x559c1f7bc360 0x559c1f7be750 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=726d901124661588 server_cookie=5d81ae7a63c6856c in_seq=0 out_seq=0 2024-09-19T05:10:49.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.877+0000 7f95e6000640 10 client.16497 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:10:49.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.877+0000 7f95e6000640 1 -- 172.21.15.43:0/2502914195 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559c1f6a3e60 con 0x559c1f7bc360 2024-09-19T05:10:49.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.877+0000 7f95e6000640 10 client.16497 handle_command_reply: tid=0 2024-09-19T05:10:49.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.877+0000 7f9605c87740 1 client.16497 shutdown 2024-09-19T05:10:49.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.878+0000 7f9605c87740 1 -- 172.21.15.43:0/2502914195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c1f735e40 msgr2=0x559c1f7497f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:49.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.878+0000 7f9605c87740 1 --2- 172.21.15.43:0/2502914195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c1f735e40 0x559c1f7497f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f95ec0099b0 tx=0x7f95ec005da0 comp rx=0 tx=0).stop 2024-09-19T05:10:49.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.878+0000 7f9605c87740 1 -- 172.21.15.43:0/2502914195 shutdown_connections 2024-09-19T05:10:49.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.878+0000 7f9605c87740 1 --2- 172.21.15.43:0/2502914195 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x559c1f7bc360 0x559c1f7be750 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:49.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.878+0000 7f9605c87740 1 --2- 172.21.15.43:0/2502914195 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c1f741020 0x559c1f7450a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:49.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.878+0000 7f9605c87740 1 --2- 172.21.15.43:0/2502914195 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c1f736760 0x559c1f744b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:49.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.878+0000 7f9605c87740 1 --2- 172.21.15.43:0/2502914195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c1f735e40 0x559c1f7497f0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:49.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.878+0000 7f9605c87740 1 -- 172.21.15.43:0/2502914195 >> 172.21.15.43:0/2502914195 conn(0x559c1f7323d0 msgr2=0x559c1f73e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:49.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.878+0000 7f95e6000640 0 client.16497 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:10:49.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.878+0000 7f9605c87740 1 -- 172.21.15.43:0/2502914195 shutdown_connections 2024-09-19T05:10:49.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.878+0000 7f9605c87740 1 -- 172.21.15.43:0/2502914195 wait complete. 2024-09-19T05:10:49.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.878+0000 7f9605c87740 20 client.16497 trim_cache size 0 max 16384 2024-09-19T05:10:49.896 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:10:49.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.879+0000 7f9601c00640 1 -- 172.21.15.43:0/3098174288 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f95e00a33b0 msgr2=0x7f95e00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:49.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.879+0000 7f9601c00640 1 --2- 172.21.15.43:0/3098174288 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f95e00a33b0 0x7f95e00a5870 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x559c1f733750 tx=0x7f95e80058d0 comp rx=0 tx=0).stop 2024-09-19T05:10:49.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.879+0000 7f9601c00640 1 -- 172.21.15.43:0/3098174288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95fc108f60 msgr2=0x7f95fc19ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:49.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.879+0000 7f9601c00640 1 --2- 172.21.15.43:0/3098174288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95fc108f60 0x7f95fc19ed30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f95f400eb10 tx=0x7f95f400c2d0 comp rx=0 tx=0).stop 2024-09-19T05:10:49.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.879+0000 7f9601c00640 1 -- 172.21.15.43:0/3098174288 shutdown_connections 2024-09-19T05:10:49.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.879+0000 7f9601c00640 1 --2- 172.21.15.43:0/3098174288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95fc108f60 0x7f95fc19ed30 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:49.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.879+0000 7f9601c00640 1 --2- 172.21.15.43:0/3098174288 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f95e00a33b0 0x7f95e00a5870 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:49.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.879+0000 7f9601c00640 1 --2- 172.21.15.43:0/3098174288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95fc1047e0 0x7f95fc19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:49.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.879+0000 7f9601c00640 1 --2- 172.21.15.43:0/3098174288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95fc103e30 0x7f95fc19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:49.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.879+0000 7f9601c00640 1 -- 172.21.15.43:0/3098174288 >> 172.21.15.43:0/3098174288 conn(0x7f95fc0ff8d0 msgr2=0x7f95fc100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:49.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.879+0000 7f9601c00640 1 -- 172.21.15.43:0/3098174288 shutdown_connections 2024-09-19T05:10:49.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:49.879+0000 7f9601c00640 1 -- 172.21.15.43:0/3098174288 wait complete. 2024-09-19T05:10:50.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:49 smithi043 ceph-mon[30800]: pgmap v188: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:10:50.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:49 smithi151 ceph-mon[35425]: pgmap v188: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:10:50.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:49 smithi096 ceph-mon[35085]: pgmap v188: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:10:51.663 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T05:10:51.663 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T05:10:52.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:51 smithi043 ceph-mon[30800]: pgmap v189: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:10:52.053 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:10:52.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:51 smithi151 ceph-mon[35425]: pgmap v189: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:10:52.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:51 smithi096 ceph-mon[35085]: pgmap v189: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:10:52.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.641+0000 7f991ce00640 1 -- 172.21.15.43:0/3459717660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99181079c0 msgr2=0x7f9918107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:52.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.641+0000 7f991ce00640 1 --2- 172.21.15.43:0/3459717660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99181079c0 0x7f9918107da0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9908009930 tx=0x7f990802a420 comp rx=0 tx=0).stop 2024-09-19T05:10:52.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.641+0000 7f991ce00640 1 -- 172.21.15.43:0/3459717660 shutdown_connections 2024-09-19T05:10:52.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.641+0000 7f991ce00640 1 --2- 172.21.15.43:0/3459717660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9918102360 0x7f991810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:52.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.641+0000 7f991ce00640 1 --2- 172.21.15.43:0/3459717660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99181019c0 0x7f9918101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:52.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.641+0000 7f991ce00640 1 --2- 172.21.15.43:0/3459717660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99181079c0 0x7f9918107da0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:52.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.641+0000 7f991ce00640 1 -- 172.21.15.43:0/3459717660 >> 172.21.15.43:0/3459717660 conn(0x7f99180fd6e0 msgr2=0x7f99180ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:52.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.641+0000 7f991ce00640 1 -- 172.21.15.43:0/3459717660 shutdown_connections 2024-09-19T05:10:52.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.642+0000 7f991ce00640 1 -- 172.21.15.43:0/3459717660 wait complete. 2024-09-19T05:10:52.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.642+0000 7f991ce00640 1 Processor -- start 2024-09-19T05:10:52.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.643+0000 7f991ce00640 1 -- start start 2024-09-19T05:10:52.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.643+0000 7f991ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99181019c0 0x7f99181982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:52.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.643+0000 7f991ce00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9918102360 0x7f9918198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:52.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.643+0000 7f991ce00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99181079c0 0x7f991819cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:52.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.643+0000 7f991ce00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9918111dc0 con 0x7f99181019c0 2024-09-19T05:10:52.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.643+0000 7f991ce00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9918111c40 con 0x7f9918102360 2024-09-19T05:10:52.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.643+0000 7f991ce00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9918111f40 con 0x7f99181079c0 2024-09-19T05:10:52.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.643+0000 7f9917400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99181019c0 0x7f99181982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:52.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.643+0000 7f9917400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99181019c0 0x7f99181982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56668/0 (socket says 172.21.15.43:56668) 2024-09-19T05:10:52.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.643+0000 7f9917400640 1 -- 172.21.15.43:0/307742669 learned_addr learned my addr 172.21.15.43:0/307742669 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:52.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.643+0000 7f9916a00640 1 --2- 172.21.15.43:0/307742669 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9918102360 0x7f9918198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:52.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.643+0000 7f9917e00640 1 --2- 172.21.15.43:0/307742669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99181079c0 0x7f991819cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:52.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.644+0000 7f9916a00640 1 -- 172.21.15.43:0/307742669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99181079c0 msgr2=0x7f991819cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:52.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.644+0000 7f9916a00640 1 --2- 172.21.15.43:0/307742669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99181079c0 0x7f991819cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:52.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.644+0000 7f9916a00640 1 -- 172.21.15.43:0/307742669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99181019c0 msgr2=0x7f99181982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:52.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.644+0000 7f9916a00640 1 --2- 172.21.15.43:0/307742669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99181019c0 0x7f99181982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:52.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.644+0000 7f9916a00640 1 -- 172.21.15.43:0/307742669 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f991819d270 con 0x7f9918102360 2024-09-19T05:10:52.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.644+0000 7f9917400640 1 --2- 172.21.15.43:0/307742669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99181019c0 0x7f99181982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:10:52.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.644+0000 7f9916a00640 1 --2- 172.21.15.43:0/307742669 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9918102360 0x7f9918198800 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f990400e9f0 tx=0x7f990400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:52.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.644+0000 7f990fe00640 1 -- 172.21.15.43:0/307742669 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990400cd80 con 0x7f9918102360 2024-09-19T05:10:52.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.644+0000 7f990fe00640 1 -- 172.21.15.43:0/307742669 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99040035f0 con 0x7f9918102360 2024-09-19T05:10:52.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.644+0000 7f991ce00640 1 -- 172.21.15.43:0/307742669 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f991819d560 con 0x7f9918102360 2024-09-19T05:10:52.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.644+0000 7f990fe00640 1 -- 172.21.15.43:0/307742669 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9904010670 con 0x7f9918102360 2024-09-19T05:10:52.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.645+0000 7f991ce00640 1 -- 172.21.15.43:0/307742669 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99181a4e40 con 0x7f9918102360 2024-09-19T05:10:52.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.648+0000 7f9920e80740 1 Processor -- start 2024-09-19T05:10:52.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.648+0000 7f9920e80740 1 -- start start 2024-09-19T05:10:52.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.648+0000 7f9920e80740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573bb6dbcb0 0x5573bb6dc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:52.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.648+0000 7f9920e80740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573bb6dc5d0 0x5573bb6e6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:52.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.648+0000 7f9920e80740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573bb6e6e90 0x5573bb6e9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:52.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.648+0000 7f9920e80740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5573bb642290 con 0x5573bb6dc5d0 2024-09-19T05:10:52.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.648+0000 7f9920e80740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5573bb5e9b40 con 0x5573bb6e6e90 2024-09-19T05:10:52.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.648+0000 7f9920e80740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5573bb63e250 con 0x5573bb6dbcb0 2024-09-19T05:10:52.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.648+0000 7f9917400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573bb6dc5d0 0x5573bb6e6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:52.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.648+0000 7f9917e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573bb6dbcb0 0x5573bb6dc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:52.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.648+0000 7f9917400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573bb6dc5d0 0x5573bb6e6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56680/0 (socket says 172.21.15.43:56680) 2024-09-19T05:10:52.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.649+0000 7f9917400640 1 -- 172.21.15.43:0/878770286 learned_addr learned my addr 172.21.15.43:0/878770286 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:52.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.649+0000 7f990fe00640 1 -- 172.21.15.43:0/307742669 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f99040037a0 con 0x7f9918102360 2024-09-19T05:10:52.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.649+0000 7f9917400640 1 -- 172.21.15.43:0/878770286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573bb6dbcb0 msgr2=0x5573bb6dc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:52.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.649+0000 7f9916a00640 1 --2- 172.21.15.43:0/878770286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573bb6e6e90 0x5573bb6e9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:52.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.649+0000 7f9917400640 1 --2- 172.21.15.43:0/878770286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573bb6dbcb0 0x5573bb6dc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:52.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.649+0000 7f9917400640 1 -- 172.21.15.43:0/878770286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573bb6e6e90 msgr2=0x5573bb6e9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:52.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.649+0000 7f9917400640 1 --2- 172.21.15.43:0/878770286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573bb6e6e90 0x5573bb6e9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:52.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.649+0000 7f9917400640 1 -- 172.21.15.43:0/878770286 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573bb3b36a0 con 0x5573bb6dc5d0 2024-09-19T05:10:52.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.649+0000 7f9916a00640 1 --2- 172.21.15.43:0/878770286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573bb6e6e90 0x5573bb6e9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:10:52.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.650+0000 7f9917400640 1 --2- 172.21.15.43:0/878770286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573bb6dc5d0 0x5573bb6e6950 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9908006010 tx=0x7f9908005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e316bc5446ffaaf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:52.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.650+0000 7f990cc00640 1 -- 172.21.15.43:0/878770286 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9908035040 con 0x5573bb6dc5d0 2024-09-19T05:10:52.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.650+0000 7f990cc00640 1 -- 172.21.15.43:0/878770286 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9908031830 con 0x5573bb6dc5d0 2024-09-19T05:10:52.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.650+0000 7f990cc00640 1 -- 172.21.15.43:0/878770286 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9908004a00 con 0x5573bb6dc5d0 2024-09-19T05:10:52.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.651+0000 7f9920e80740 1 -- 172.21.15.43:0/878770286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573bb6dc5d0 msgr2=0x5573bb6e6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:52.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.651+0000 7f9920e80740 1 --2- 172.21.15.43:0/878770286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573bb6dc5d0 0x5573bb6e6950 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9908006010 tx=0x7f9908005dc0 comp rx=0 tx=0).stop 2024-09-19T05:10:52.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.651+0000 7f990fe00640 1 --2- 172.21.15.43:0/307742669 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f98fc0a33b0 0x7f98fc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:52.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.651+0000 7f9920e80740 1 -- 172.21.15.43:0/878770286 shutdown_connections 2024-09-19T05:10:52.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.651+0000 7f9920e80740 1 --2- 172.21.15.43:0/878770286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573bb6e6e90 0x5573bb6e9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:52.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.651+0000 7f9920e80740 1 --2- 172.21.15.43:0/878770286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573bb6dc5d0 0x5573bb6e6950 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:52.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.651+0000 7f9920e80740 1 --2- 172.21.15.43:0/878770286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573bb6dbcb0 0x5573bb6dc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:52.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.651+0000 7f9920e80740 1 -- 172.21.15.43:0/878770286 >> 172.21.15.43:0/878770286 conn(0x5573bb6d8240 msgr2=0x5573bb6d8620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:52.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.651+0000 7f9920e80740 1 -- 172.21.15.43:0/878770286 shutdown_connections 2024-09-19T05:10:52.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.651+0000 7f990fe00640 1 -- 172.21.15.43:0/307742669 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f99040cfff0 con 0x7f9918102360 2024-09-19T05:10:52.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.651+0000 7f9920e80740 1 -- 172.21.15.43:0/878770286 wait complete. 2024-09-19T05:10:52.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.651+0000 7f9917e00640 1 --2- 172.21.15.43:0/307742669 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f98fc0a33b0 0x7f98fc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:52.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.652+0000 7f9920e80740 1 -- start start 2024-09-19T05:10:52.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.652+0000 7f9917e00640 1 --2- 172.21.15.43:0/307742669 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f98fc0a33b0 0x7f98fc0a5870 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9910005fd0 tx=0x7f9910009500 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:52.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.652+0000 7f9920e80740 1 Processor -- start 2024-09-19T05:10:52.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.653+0000 7f9920e80740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573bb6dbcb0 0x5573bb6eb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:52.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.653+0000 7f9920e80740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573bb6dc5d0 0x5573bb6eb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:52.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.653+0000 7f9920e80740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573bb6e6e90 0x5573bb6ef6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:52.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.653+0000 7f9920e80740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5573bb6e9b00 con 0x5573bb6dbcb0 2024-09-19T05:10:52.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.653+0000 7f9920e80740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5573bb5fcfa0 con 0x5573bb6dc5d0 2024-09-19T05:10:52.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.653+0000 7f9920e80740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5573bb6e9c80 con 0x5573bb6e6e90 2024-09-19T05:10:52.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.653+0000 7f9920e80740 10 client.0 _ll_register_callbacks cb 0x5573bb350ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:52.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.653+0000 7f9917400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573bb6dbcb0 0x5573bb6eb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:52.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.653+0000 7f9917e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573bb6e6e90 0x5573bb6ef6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:52.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.653+0000 7f9917400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573bb6dbcb0 0x5573bb6eb020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56694/0 (socket says 172.21.15.43:56694) 2024-09-19T05:10:52.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.653+0000 7f9917400640 1 -- 172.21.15.43:0/1239615301 learned_addr learned my addr 172.21.15.43:0/1239615301 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:52.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.653+0000 7f9916a00640 1 --2- 172.21.15.43:0/1239615301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573bb6dc5d0 0x5573bb6eb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:52.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.653+0000 7f9917400640 1 -- 172.21.15.43:0/1239615301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573bb6e6e90 msgr2=0x5573bb6ef6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:52.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.653+0000 7f9917400640 1 --2- 172.21.15.43:0/1239615301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573bb6e6e90 0x5573bb6ef6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:52.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.653+0000 7f9917400640 1 -- 172.21.15.43:0/1239615301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573bb6dc5d0 msgr2=0x5573bb6eb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:52.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.653+0000 7f9917400640 1 --2- 172.21.15.43:0/1239615301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573bb6dc5d0 0x5573bb6eb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:52.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.653+0000 7f9917400640 1 -- 172.21.15.43:0/1239615301 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573bb6efd70 con 0x5573bb6dbcb0 2024-09-19T05:10:52.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.653+0000 7f9917e00640 1 --2- 172.21.15.43:0/1239615301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573bb6e6e90 0x5573bb6ef6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:10:52.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.654+0000 7f9917400640 1 --2- 172.21.15.43:0/1239615301 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573bb6dbcb0 0x5573bb6eb020 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9908000c00 tx=0x7f9908009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6a8604075ef9cb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:52.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.654+0000 7f9901600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:10:52.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.654+0000 7f9901600640 1 -- 172.21.15.43:0/1239615301 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990804e030 con 0x5573bb6dbcb0 2024-09-19T05:10:52.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.654+0000 7f9901600640 1 -- 172.21.15.43:0/1239615301 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f9908031630 con 0x5573bb6dbcb0 2024-09-19T05:10:52.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.654+0000 7f9920e80740 1 -- 172.21.15.43:0/1239615301 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9908031830 con 0x5573bb6dbcb0 2024-09-19T05:10:52.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.654+0000 7f9901600640 1 -- 172.21.15.43:0/1239615301 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9908035040 con 0x5573bb6dbcb0 2024-09-19T05:10:52.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.654+0000 7f9920e80740 10 client.16509 fetch_fsmap learned FSMap version 29 2024-09-19T05:10:52.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.654+0000 7f9920e80740 1 -- 172.21.15.43:0/1239615301 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5573bb6f9cb0 con 0x5573bb6dbcb0 2024-09-19T05:10:52.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.655+0000 7f9901600640 1 -- 172.21.15.43:0/1239615301 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9908031830 con 0x5573bb6dbcb0 2024-09-19T05:10:52.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.655+0000 7f9901600640 1 -- 172.21.15.43:0/1239615301 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9908004770 con 0x5573bb6dbcb0 2024-09-19T05:10:52.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.655+0000 7f9920e80740 10 client.16509 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:10:52.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.655+0000 7f9920e80740 10 client.16509 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:10:52.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.655+0000 7f9920e80740 20 client.16509 populate_metadata read hostname 'smithi043' 2024-09-19T05:10:52.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.655+0000 7f9920e80740 1 --2- 172.21.15.43:0/1239615301 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5573bb6fb000 0x5573bb6fd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:52.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.655+0000 7f9920e80740 4 client.16509 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:10:52.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.655+0000 7f9920e80740 1 -- 172.21.15.43:0/1239615301 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573bb64f800 con 0x5573bb6fb000 2024-09-19T05:10:52.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.655+0000 7f9916a00640 1 --2- 172.21.15.43:0/1239615301 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5573bb6fb000 0x5573bb6fd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:52.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.656+0000 7f9916a00640 1 --2- 172.21.15.43:0/1239615301 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5573bb6fb000 0x5573bb6fd3f0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57a7b8f1aa20ac77 server_cookie=55cf7e0ffb7dff3e in_seq=0 out_seq=0 2024-09-19T05:10:52.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.656+0000 7f9901600640 10 client.16509 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:10:52.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.656+0000 7f9901600640 1 -- 172.21.15.43:0/1239615301 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5573bb64f800 con 0x5573bb6fb000 2024-09-19T05:10:52.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.656+0000 7f9901600640 10 client.16509 handle_command_reply: tid=0 2024-09-19T05:10:52.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.657+0000 7f9920e80740 1 client.16509 shutdown 2024-09-19T05:10:52.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.657+0000 7f9920e80740 1 -- 172.21.15.43:0/1239615301 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573bb6dbcb0 msgr2=0x5573bb6eb020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:52.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.657+0000 7f9920e80740 1 --2- 172.21.15.43:0/1239615301 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573bb6dbcb0 0x5573bb6eb020 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9908000c00 tx=0x7f9908009520 comp rx=0 tx=0).stop 2024-09-19T05:10:52.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.657+0000 7f9920e80740 1 -- 172.21.15.43:0/1239615301 shutdown_connections 2024-09-19T05:10:52.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.657+0000 7f9920e80740 1 --2- 172.21.15.43:0/1239615301 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5573bb6fb000 0x5573bb6fd3f0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:52.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.657+0000 7f9920e80740 1 --2- 172.21.15.43:0/1239615301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573bb6e6e90 0x5573bb6ef6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:52.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.657+0000 7f9920e80740 1 --2- 172.21.15.43:0/1239615301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573bb6dc5d0 0x5573bb6eb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:52.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.657+0000 7f9920e80740 1 --2- 172.21.15.43:0/1239615301 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573bb6dbcb0 0x5573bb6eb020 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:52.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.657+0000 7f9901600640 0 client.16509 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:10:52.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.657+0000 7f9920e80740 1 -- 172.21.15.43:0/1239615301 >> 172.21.15.43:0/1239615301 conn(0x5573bb6d8240 msgr2=0x5573bb6ddf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:52.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.657+0000 7f9920e80740 1 -- 172.21.15.43:0/1239615301 shutdown_connections 2024-09-19T05:10:52.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.657+0000 7f9920e80740 1 -- 172.21.15.43:0/1239615301 wait complete. 2024-09-19T05:10:52.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.657+0000 7f9920e80740 20 client.16509 trim_cache size 0 max 16384 2024-09-19T05:10:52.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.675+0000 7f9920e80740 1 Processor -- start 2024-09-19T05:10:52.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.676+0000 7f9920e80740 1 -- start start 2024-09-19T05:10:52.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.676+0000 7f9920e80740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573bb6dbcb0 0x5573bb6ef6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:52.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.676+0000 7f9920e80740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573bb6dc5d0 0x5573bb702990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:52.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.676+0000 7f9920e80740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573bb6e6e90 0x5573bb702ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:52.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.676+0000 7f9920e80740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f99181ab590 con 0x5573bb6dbcb0 2024-09-19T05:10:52.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.676+0000 7f9920e80740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f99181ad0d0 con 0x5573bb6e6e90 2024-09-19T05:10:52.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.676+0000 7f9920e80740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f99181abc60 con 0x5573bb6dc5d0 2024-09-19T05:10:52.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.676+0000 7f9917400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573bb6dbcb0 0x5573bb6ef6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:52.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.676+0000 7f9917400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573bb6dbcb0 0x5573bb6ef6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56706/0 (socket says 172.21.15.43:56706) 2024-09-19T05:10:52.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.676+0000 7f9917400640 1 -- 172.21.15.43:0/760090831 learned_addr learned my addr 172.21.15.43:0/760090831 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:52.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.676+0000 7f9916a00640 1 --2- 172.21.15.43:0/760090831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573bb6dc5d0 0x5573bb702990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:52.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.676+0000 7f9917e00640 1 --2- 172.21.15.43:0/760090831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573bb6e6e90 0x5573bb702ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:52.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.677+0000 7f9917400640 1 -- 172.21.15.43:0/760090831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573bb6dc5d0 msgr2=0x5573bb702990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:52.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.677+0000 7f9917400640 1 --2- 172.21.15.43:0/760090831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573bb6dc5d0 0x5573bb702990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:52.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.677+0000 7f9917400640 1 -- 172.21.15.43:0/760090831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573bb6e6e90 msgr2=0x5573bb702ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:52.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.677+0000 7f9917400640 1 --2- 172.21.15.43:0/760090831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573bb6e6e90 0x5573bb702ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:52.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.677+0000 7f9917400640 1 -- 172.21.15.43:0/760090831 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573bb6dc400 con 0x5573bb6dbcb0 2024-09-19T05:10:52.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.677+0000 7f9916a00640 1 --2- 172.21.15.43:0/760090831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573bb6dc5d0 0x5573bb702990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:10:52.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.677+0000 7f9917400640 1 --2- 172.21.15.43:0/760090831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573bb6dbcb0 0x5573bb6ef6c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9908004b40 tx=0x7f9908005540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b864a424cd47d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:52.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.677+0000 7f9903e00640 1 -- 172.21.15.43:0/760090831 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990804e040 con 0x5573bb6dbcb0 2024-09-19T05:10:52.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.677+0000 7f9903e00640 1 -- 172.21.15.43:0/760090831 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99080048e0 con 0x5573bb6dbcb0 2024-09-19T05:10:52.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.677+0000 7f9903e00640 1 -- 172.21.15.43:0/760090831 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9908031d70 con 0x5573bb6dbcb0 2024-09-19T05:10:52.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.678+0000 7f9920e80740 1 -- 172.21.15.43:0/760090831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573bb6dbcb0 msgr2=0x5573bb6ef6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:52.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.678+0000 7f9920e80740 1 --2- 172.21.15.43:0/760090831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573bb6dbcb0 0x5573bb6ef6c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9908004b40 tx=0x7f9908005540 comp rx=0 tx=0).stop 2024-09-19T05:10:52.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.678+0000 7f9920e80740 1 -- 172.21.15.43:0/760090831 shutdown_connections 2024-09-19T05:10:52.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.678+0000 7f9920e80740 1 --2- 172.21.15.43:0/760090831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573bb6e6e90 0x5573bb702ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:52.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.678+0000 7f9920e80740 1 --2- 172.21.15.43:0/760090831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573bb6dc5d0 0x5573bb702990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:52.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.678+0000 7f9920e80740 1 --2- 172.21.15.43:0/760090831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573bb6dbcb0 0x5573bb6ef6c0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:52.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.678+0000 7f9920e80740 1 -- 172.21.15.43:0/760090831 >> 172.21.15.43:0/760090831 conn(0x5573bb6d8240 msgr2=0x5573bb6d7290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:52.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.678+0000 7f9920e80740 1 -- 172.21.15.43:0/760090831 shutdown_connections 2024-09-19T05:10:52.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.678+0000 7f9920e80740 1 -- 172.21.15.43:0/760090831 wait complete. 2024-09-19T05:10:52.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.678+0000 7f9920e80740 1 -- start start 2024-09-19T05:10:52.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.679+0000 7f9920e80740 1 Processor -- start 2024-09-19T05:10:52.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.679+0000 7f9920e80740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573bb6dbcb0 0x5573bb6f3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:52.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.679+0000 7f9920e80740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573bb6dc5d0 0x5573bb6f38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:52.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.679+0000 7f9920e80740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573bb6e6e90 0x5573bb764960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:52.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.679+0000 7f9920e80740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f99181aca00 con 0x5573bb6dc5d0 2024-09-19T05:10:52.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.679+0000 7f9920e80740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f99181ac330 con 0x5573bb6e6e90 2024-09-19T05:10:52.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.679+0000 7f9920e80740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f99181a9570 con 0x5573bb6dbcb0 2024-09-19T05:10:52.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.679+0000 7f9920e80740 10 client.0 _ll_register_callbacks cb 0x7f9908040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:52.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.679+0000 7f9916a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573bb6dc5d0 0x5573bb6f38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:52.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.679+0000 7f9917400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573bb6dbcb0 0x5573bb6f3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:52.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.680+0000 7f9917400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573bb6dbcb0 0x5573bb6f3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:49694/0 (socket says 172.21.15.43:49694) 2024-09-19T05:10:52.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.680+0000 7f9917400640 1 -- 172.21.15.43:0/1024819873 learned_addr learned my addr 172.21.15.43:0/1024819873 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:52.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.680+0000 7f9917e00640 1 --2- 172.21.15.43:0/1024819873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573bb6e6e90 0x5573bb764960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:52.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.680+0000 7f9917400640 1 -- 172.21.15.43:0/1024819873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573bb6e6e90 msgr2=0x5573bb764960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:52.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.680+0000 7f9917400640 1 --2- 172.21.15.43:0/1024819873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573bb6e6e90 0x5573bb764960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:52.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.680+0000 7f9917400640 1 -- 172.21.15.43:0/1024819873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573bb6dc5d0 msgr2=0x5573bb6f38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:52.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.680+0000 7f9917400640 1 --2- 172.21.15.43:0/1024819873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573bb6dc5d0 0x5573bb6f38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:52.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.680+0000 7f9917400640 1 -- 172.21.15.43:0/1024819873 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573bb764ea0 con 0x5573bb6dbcb0 2024-09-19T05:10:52.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.680+0000 7f9917400640 1 --2- 172.21.15.43:0/1024819873 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573bb6dbcb0 0x5573bb6f3390 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9908004bc0 tx=0x7f9908009520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c03630cccaa61643 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:52.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.680+0000 7f9901600640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:10:52.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.681+0000 7f9901600640 1 -- 172.21.15.43:0/1024819873 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9908061040 con 0x5573bb6dbcb0 2024-09-19T05:10:52.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.681+0000 7f9901600640 1 -- 172.21.15.43:0/1024819873 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f9908004d30 con 0x5573bb6dbcb0 2024-09-19T05:10:52.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.681+0000 7f9920e80740 1 -- 172.21.15.43:0/1024819873 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99080048e0 con 0x5573bb6dbcb0 2024-09-19T05:10:52.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.681+0000 7f9901600640 1 -- 172.21.15.43:0/1024819873 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f990804e040 con 0x5573bb6dbcb0 2024-09-19T05:10:52.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.681+0000 7f9920e80740 10 client.26482 fetch_fsmap learned FSMap version 29 2024-09-19T05:10:52.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.681+0000 7f9920e80740 1 -- 172.21.15.43:0/1024819873 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5573bb7661b0 con 0x5573bb6dbcb0 2024-09-19T05:10:52.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.682+0000 7f9901600640 1 -- 172.21.15.43:0/1024819873 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99080048e0 con 0x5573bb6dbcb0 2024-09-19T05:10:52.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.682+0000 7f9901600640 1 -- 172.21.15.43:0/1024819873 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f99080039f0 con 0x5573bb6dbcb0 2024-09-19T05:10:52.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.682+0000 7f9920e80740 10 client.26482 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:10:52.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.682+0000 7f9920e80740 10 client.26482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:10:52.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.682+0000 7f9920e80740 20 client.26482 populate_metadata read hostname 'smithi043' 2024-09-19T05:10:52.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.682+0000 7f9920e80740 1 --2- 172.21.15.43:0/1024819873 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5573bb7667b0 0x5573bb768ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:52.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.682+0000 7f9920e80740 4 client.26482 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T05:10:52.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.682+0000 7f9920e80740 1 -- 172.21.15.43:0/1024819873 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5573bb649d10 con 0x5573bb7667b0 2024-09-19T05:10:52.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.682+0000 7f9916a00640 1 --2- 172.21.15.43:0/1024819873 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5573bb7667b0 0x5573bb768ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:52.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.683+0000 7f9916a00640 1 --2- 172.21.15.43:0/1024819873 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5573bb7667b0 0x5573bb768ba0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f016ce57123aec4 server_cookie=454ab65442e6fdf4 in_seq=0 out_seq=0 2024-09-19T05:10:52.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.683+0000 7f9901600640 10 client.26482 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:10:52.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.684+0000 7f9901600640 1 -- 172.21.15.43:0/1024819873 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5573bb649d10 con 0x5573bb7667b0 2024-09-19T05:10:52.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.684+0000 7f9901600640 10 client.26482 handle_command_reply: tid=0 2024-09-19T05:10:52.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.684+0000 7f9920e80740 1 client.26482 shutdown 2024-09-19T05:10:52.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.684+0000 7f9920e80740 1 -- 172.21.15.43:0/1024819873 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573bb6dbcb0 msgr2=0x5573bb6f3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:52.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.684+0000 7f9920e80740 1 --2- 172.21.15.43:0/1024819873 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573bb6dbcb0 0x5573bb6f3390 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9908004bc0 tx=0x7f9908009520 comp rx=0 tx=0).stop 2024-09-19T05:10:52.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.684+0000 7f9920e80740 1 -- 172.21.15.43:0/1024819873 shutdown_connections 2024-09-19T05:10:52.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.684+0000 7f9920e80740 1 --2- 172.21.15.43:0/1024819873 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5573bb7667b0 0x5573bb768ba0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:52.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.684+0000 7f9920e80740 1 --2- 172.21.15.43:0/1024819873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573bb6e6e90 0x5573bb764960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:52.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.684+0000 7f9920e80740 1 --2- 172.21.15.43:0/1024819873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573bb6dc5d0 0x5573bb6f38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:52.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.684+0000 7f9920e80740 1 --2- 172.21.15.43:0/1024819873 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573bb6dbcb0 0x5573bb6f3390 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:52.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.684+0000 7f9920e80740 1 -- 172.21.15.43:0/1024819873 >> 172.21.15.43:0/1024819873 conn(0x5573bb6d8240 msgr2=0x5573bb6d6a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:52.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.684+0000 7f9901600640 0 client.26482 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:10:52.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.684+0000 7f9920e80740 1 -- 172.21.15.43:0/1024819873 shutdown_connections 2024-09-19T05:10:52.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.684+0000 7f9920e80740 1 -- 172.21.15.43:0/1024819873 wait complete. 2024-09-19T05:10:52.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.685+0000 7f9920e80740 20 client.26482 trim_cache size 0 max 16384 2024-09-19T05:10:52.689 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:10:52.689 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-09-19T05:10:52.689 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "37e0244a-44fa-429d-a4b6-91bc84328226", 2024-09-19T05:10:52.689 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-09-19T05:10:52.689 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:10:52.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.685+0000 7f991ce00640 1 -- 172.21.15.43:0/307742669 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f98fc0a33b0 msgr2=0x7f98fc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:52.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.685+0000 7f991ce00640 1 --2- 172.21.15.43:0/307742669 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f98fc0a33b0 0x7f98fc0a5870 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9910005fd0 tx=0x7f9910009500 comp rx=0 tx=0).stop 2024-09-19T05:10:52.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.685+0000 7f991ce00640 1 -- 172.21.15.43:0/307742669 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9918102360 msgr2=0x7f9918198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:52.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.685+0000 7f991ce00640 1 --2- 172.21.15.43:0/307742669 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9918102360 0x7f9918198800 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f990400e9f0 tx=0x7f990400eeb0 comp rx=0 tx=0).stop 2024-09-19T05:10:52.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.685+0000 7f991ce00640 1 -- 172.21.15.43:0/307742669 shutdown_connections 2024-09-19T05:10:52.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.685+0000 7f991ce00640 1 --2- 172.21.15.43:0/307742669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99181079c0 0x7f991819cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:52.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.685+0000 7f991ce00640 1 --2- 172.21.15.43:0/307742669 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f98fc0a33b0 0x7f98fc0a5870 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:52.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.685+0000 7f991ce00640 1 --2- 172.21.15.43:0/307742669 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9918102360 0x7f9918198800 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:52.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.685+0000 7f991ce00640 1 --2- 172.21.15.43:0/307742669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99181019c0 0x7f99181982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:52.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.685+0000 7f991ce00640 1 -- 172.21.15.43:0/307742669 >> 172.21.15.43:0/307742669 conn(0x7f99180fd6e0 msgr2=0x7f99180fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:52.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.686+0000 7f991ce00640 1 -- 172.21.15.43:0/307742669 shutdown_connections 2024-09-19T05:10:52.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:52.686+0000 7f991ce00640 1 -- 172.21.15.43:0/307742669 wait complete. 2024-09-19T05:10:53.608 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:10:53.989 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:10:54.143 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:53 smithi043 ceph-mon[30800]: pgmap v190: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:10:54.143 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:53 smithi043 ceph-mon[30800]: scrub queued for path: / 2024-09-19T05:10:54.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:53 smithi043 ceph-mon[30800]: scrub summary: idle+waiting paths [/] 2024-09-19T05:10:54.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:53 smithi043 ceph-mon[30800]: scrub summary: active paths [/] 2024-09-19T05:10:54.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:53 smithi043 ceph-mon[30800]: scrub summary: idle+waiting paths [/] 2024-09-19T05:10:54.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:53 smithi043 ceph-mon[30800]: scrub summary: idle 2024-09-19T05:10:54.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:53 smithi151 ceph-mon[35425]: pgmap v190: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:10:54.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:53 smithi151 ceph-mon[35425]: scrub queued for path: / 2024-09-19T05:10:54.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:53 smithi151 ceph-mon[35425]: scrub summary: idle+waiting paths [/] 2024-09-19T05:10:54.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:53 smithi151 ceph-mon[35425]: scrub summary: active paths [/] 2024-09-19T05:10:54.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:53 smithi151 ceph-mon[35425]: scrub summary: idle+waiting paths [/] 2024-09-19T05:10:54.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:53 smithi151 ceph-mon[35425]: scrub summary: idle 2024-09-19T05:10:54.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:53 smithi096 ceph-mon[35085]: pgmap v190: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:10:54.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:53 smithi096 ceph-mon[35085]: scrub queued for path: / 2024-09-19T05:10:54.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:53 smithi096 ceph-mon[35085]: scrub summary: idle+waiting paths [/] 2024-09-19T05:10:54.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:53 smithi096 ceph-mon[35085]: scrub summary: active paths [/] 2024-09-19T05:10:54.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:53 smithi096 ceph-mon[35085]: scrub summary: idle+waiting paths [/] 2024-09-19T05:10:54.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:53 smithi096 ceph-mon[35085]: scrub summary: idle 2024-09-19T05:10:54.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.520+0000 7fb888a00640 1 -- 172.21.15.43:0/3550219116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8841079c0 msgr2=0x7fb884107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:54.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.520+0000 7fb888a00640 1 --2- 172.21.15.43:0/3550219116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8841079c0 0x7fb884107da0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb874009930 tx=0x7fb87402a420 comp rx=0 tx=0).stop 2024-09-19T05:10:54.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.520+0000 7fb888a00640 1 -- 172.21.15.43:0/3550219116 shutdown_connections 2024-09-19T05:10:54.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.520+0000 7fb888a00640 1 --2- 172.21.15.43:0/3550219116 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb884102360 0x7fb88410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:54.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.520+0000 7fb888a00640 1 --2- 172.21.15.43:0/3550219116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8841019c0 0x7fb884101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:54.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.520+0000 7fb888a00640 1 --2- 172.21.15.43:0/3550219116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8841079c0 0x7fb884107da0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:54.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.520+0000 7fb888a00640 1 -- 172.21.15.43:0/3550219116 >> 172.21.15.43:0/3550219116 conn(0x7fb8840fd740 msgr2=0x7fb8840ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:54.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.520+0000 7fb888a00640 1 -- 172.21.15.43:0/3550219116 shutdown_connections 2024-09-19T05:10:54.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.520+0000 7fb888a00640 1 -- 172.21.15.43:0/3550219116 wait complete. 2024-09-19T05:10:54.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.521+0000 7fb888a00640 1 Processor -- start 2024-09-19T05:10:54.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.521+0000 7fb888a00640 1 -- start start 2024-09-19T05:10:54.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.521+0000 7fb888a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8841019c0 0x7fb88419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:54.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.521+0000 7fb888a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb884102360 0x7fb88419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:54.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.522+0000 7fb888a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb8841079c0 0x7fb88419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:54.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.522+0000 7fb888a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb884111dd0 con 0x7fb884102360 2024-09-19T05:10:54.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.522+0000 7fb888a00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb884111c50 con 0x7fb8841019c0 2024-09-19T05:10:54.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.522+0000 7fb888a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb884111f50 con 0x7fb8841079c0 2024-09-19T05:10:54.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.522+0000 7fb879e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb884102360 0x7fb88419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:54.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.522+0000 7fb879e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb884102360 0x7fb88419aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56732/0 (socket says 172.21.15.43:56732) 2024-09-19T05:10:54.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.522+0000 7fb879e00640 1 -- 172.21.15.43:0/3070196239 learned_addr learned my addr 172.21.15.43:0/3070196239 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:54.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.522+0000 7fb882800640 1 --2- 172.21.15.43:0/3070196239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8841019c0 0x7fb88419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:54.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.522+0000 7fb883200640 1 --2- 172.21.15.43:0/3070196239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb8841079c0 0x7fb88419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:54.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.522+0000 7fb879e00640 1 -- 172.21.15.43:0/3070196239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb8841079c0 msgr2=0x7fb88419edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:54.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.522+0000 7fb879e00640 1 --2- 172.21.15.43:0/3070196239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb8841079c0 0x7fb88419edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:54.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.522+0000 7fb879e00640 1 -- 172.21.15.43:0/3070196239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8841019c0 msgr2=0x7fb88419a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:54.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.522+0000 7fb879e00640 1 --2- 172.21.15.43:0/3070196239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8841019c0 0x7fb88419a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:54.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.522+0000 7fb879e00640 1 -- 172.21.15.43:0/3070196239 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb88419f490 con 0x7fb884102360 2024-09-19T05:10:54.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.523+0000 7fb879e00640 1 --2- 172.21.15.43:0/3070196239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb884102360 0x7fb88419aa20 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb86c00e9f0 tx=0x7fb86c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:54.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.523+0000 7fb87be00640 1 -- 172.21.15.43:0/3070196239 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c00cd80 con 0x7fb884102360 2024-09-19T05:10:54.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.523+0000 7fb87be00640 1 -- 172.21.15.43:0/3070196239 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb86c010910 con 0x7fb884102360 2024-09-19T05:10:54.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.523+0000 7fb87be00640 1 -- 172.21.15.43:0/3070196239 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c018990 con 0x7fb884102360 2024-09-19T05:10:54.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.523+0000 7fb888a00640 1 -- 172.21.15.43:0/3070196239 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb88419f780 con 0x7fb884102360 2024-09-19T05:10:54.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.523+0000 7fb888a00640 1 -- 172.21.15.43:0/3070196239 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8841a7060 con 0x7fb884102360 2024-09-19T05:10:54.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.527+0000 7fb88bfff740 1 Processor -- start 2024-09-19T05:10:54.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.528+0000 7fb88bfff740 1 -- start start 2024-09-19T05:10:54.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.528+0000 7fb88bfff740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b975855d50 0x55b975856130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:54.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.528+0000 7fb88bfff740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b975856670 0x55b9758609f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:54.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.528+0000 7fb88bfff740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b975860f30 0x55b975863320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:54.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.528+0000 7fb88bfff740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b9757bc3e0 con 0x55b975856670 2024-09-19T05:10:54.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.528+0000 7fb88bfff740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b975763d80 con 0x55b975860f30 2024-09-19T05:10:54.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.528+0000 7fb88bfff740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b9757b83a0 con 0x55b975855d50 2024-09-19T05:10:54.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.528+0000 7fb87be00640 1 -- 172.21.15.43:0/3070196239 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb86c010ac0 con 0x7fb884102360 2024-09-19T05:10:54.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.529+0000 7fb883200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b975855d50 0x55b975856130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:54.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.529+0000 7fb883200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b975855d50 0x55b975856130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:49720/0 (socket says 172.21.15.43:49720) 2024-09-19T05:10:54.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.529+0000 7fb883200640 1 -- 172.21.15.43:0/1799625776 learned_addr learned my addr 172.21.15.43:0/1799625776 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:54.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.529+0000 7fb882800640 1 --2- 172.21.15.43:0/1799625776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b975856670 0x55b9758609f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:54.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.529+0000 7fb879e00640 1 --2- 172.21.15.43:0/1799625776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b975860f30 0x55b975863320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:54.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.529+0000 7fb883200640 1 -- 172.21.15.43:0/1799625776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b975860f30 msgr2=0x55b975863320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:54.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.529+0000 7fb883200640 1 --2- 172.21.15.43:0/1799625776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b975860f30 0x55b975863320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:54.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.529+0000 7fb883200640 1 -- 172.21.15.43:0/1799625776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b975856670 msgr2=0x55b9758609f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:54.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.529+0000 7fb883200640 1 --2- 172.21.15.43:0/1799625776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b975856670 0x55b9758609f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:54.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.529+0000 7fb883200640 1 -- 172.21.15.43:0/1799625776 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b975782ae0 con 0x55b975855d50 2024-09-19T05:10:54.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.529+0000 7fb879e00640 1 --2- 172.21.15.43:0/1799625776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b975860f30 0x55b975863320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:10:54.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.529+0000 7fb882800640 1 --2- 172.21.15.43:0/1799625776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b975856670 0x55b9758609f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:10:54.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.530+0000 7fb883200640 1 --2- 172.21.15.43:0/1799625776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b975855d50 0x55b975856130 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb87c009f50 tx=0x7fb87c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef6f4080a42e2e61 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:54.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.530+0000 7fb873400640 1 -- 172.21.15.43:0/1799625776 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87c004b20 con 0x55b975855d50 2024-09-19T05:10:54.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.530+0000 7fb873400640 1 -- 172.21.15.43:0/1799625776 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb87c004cc0 con 0x55b975855d50 2024-09-19T05:10:54.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.530+0000 7fb873400640 1 -- 172.21.15.43:0/1799625776 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87c005020 con 0x55b975855d50 2024-09-19T05:10:54.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.530+0000 7fb87be00640 1 --2- 172.21.15.43:0/3070196239 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb8680a33b0 0x7fb8680a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:54.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.531+0000 7fb87be00640 1 -- 172.21.15.43:0/3070196239 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fb86c0d0020 con 0x7fb884102360 2024-09-19T05:10:54.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.531+0000 7fb88bfff740 1 -- 172.21.15.43:0/1799625776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b975855d50 msgr2=0x55b975856130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:54.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.531+0000 7fb88bfff740 1 --2- 172.21.15.43:0/1799625776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b975855d50 0x55b975856130 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb87c009f50 tx=0x7fb87c0118d0 comp rx=0 tx=0).stop 2024-09-19T05:10:54.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.531+0000 7fb882800640 1 --2- 172.21.15.43:0/3070196239 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb8680a33b0 0x7fb8680a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:54.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.531+0000 7fb88bfff740 1 -- 172.21.15.43:0/1799625776 shutdown_connections 2024-09-19T05:10:54.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.531+0000 7fb88bfff740 1 --2- 172.21.15.43:0/1799625776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b975860f30 0x55b975863320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:54.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.531+0000 7fb88bfff740 1 --2- 172.21.15.43:0/1799625776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b975856670 0x55b9758609f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:54.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.531+0000 7fb88bfff740 1 --2- 172.21.15.43:0/1799625776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b975855d50 0x55b975856130 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:54.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.531+0000 7fb88bfff740 1 -- 172.21.15.43:0/1799625776 >> 172.21.15.43:0/1799625776 conn(0x55b9758522e0 msgr2=0x55b9758526c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:54.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.531+0000 7fb88bfff740 1 -- 172.21.15.43:0/1799625776 shutdown_connections 2024-09-19T05:10:54.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.531+0000 7fb88bfff740 1 -- 172.21.15.43:0/1799625776 wait complete. 2024-09-19T05:10:54.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.532+0000 7fb88bfff740 1 -- start start 2024-09-19T05:10:54.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.532+0000 7fb882800640 1 --2- 172.21.15.43:0/3070196239 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb8680a33b0 0x7fb8680a5870 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb874006010 tx=0x7fb874000bc0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:54.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.532+0000 7fb88bfff740 1 Processor -- start 2024-09-19T05:10:54.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.532+0000 7fb88bfff740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b975855d50 0x55b97585de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:54.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.533+0000 7fb88bfff740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b975856670 0x55b97585e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:54.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.533+0000 7fb88bfff740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b975860f30 0x55b97585e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:54.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.533+0000 7fb88bfff740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b975863ba0 con 0x55b975855d50 2024-09-19T05:10:54.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.533+0000 7fb88bfff740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b9757770f0 con 0x55b975856670 2024-09-19T05:10:54.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.533+0000 7fb88bfff740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b975863d20 con 0x55b975860f30 2024-09-19T05:10:54.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.533+0000 7fb88bfff740 10 client.0 _ll_register_callbacks cb 0x55b9757689e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:54.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.533+0000 7fb882800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b975855d50 0x55b97585de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:54.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.533+0000 7fb883200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b975860f30 0x55b97585e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:54.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.533+0000 7fb882800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b975855d50 0x55b97585de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56744/0 (socket says 172.21.15.43:56744) 2024-09-19T05:10:54.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.533+0000 7fb882800640 1 -- 172.21.15.43:0/1704578144 learned_addr learned my addr 172.21.15.43:0/1704578144 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:54.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.533+0000 7fb879e00640 1 --2- 172.21.15.43:0/1704578144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b975856670 0x55b97585e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:54.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.533+0000 7fb883200640 1 -- 172.21.15.43:0/1704578144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b975856670 msgr2=0x55b97585e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:54.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.533+0000 7fb883200640 1 --2- 172.21.15.43:0/1704578144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b975856670 0x55b97585e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:54.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.533+0000 7fb883200640 1 -- 172.21.15.43:0/1704578144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b975855d50 msgr2=0x55b97585de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:54.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.533+0000 7fb883200640 1 --2- 172.21.15.43:0/1704578144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b975855d50 0x55b97585de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:54.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.533+0000 7fb883200640 1 -- 172.21.15.43:0/1704578144 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b97552d3c0 con 0x55b975860f30 2024-09-19T05:10:54.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.533+0000 7fb882800640 1 --2- 172.21.15.43:0/1704578144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b975855d50 0x55b97585de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:10:54.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.534+0000 7fb883200640 1 --2- 172.21.15.43:0/1704578144 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b975860f30 0x55b97585e8a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb87c006010 tx=0x7fb87c004000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b6dad412d5e0e3a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:54.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.534+0000 7fb867e00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:10:54.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.534+0000 7fb867e00640 1 -- 172.21.15.43:0/1704578144 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87c028070 con 0x55b975860f30 2024-09-19T05:10:54.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.534+0000 7fb88bfff740 1 -- 172.21.15.43:0/1704578144 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb87c004cc0 con 0x55b975860f30 2024-09-19T05:10:54.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.534+0000 7fb867e00640 1 -- 172.21.15.43:0/1704578144 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fb87c003c40 con 0x55b975860f30 2024-09-19T05:10:54.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.534+0000 7fb867e00640 1 -- 172.21.15.43:0/1704578144 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb87c02f590 con 0x55b975860f30 2024-09-19T05:10:54.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.534+0000 7fb88bfff740 10 client.26494 fetch_fsmap learned FSMap version 29 2024-09-19T05:10:54.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.534+0000 7fb88bfff740 1 -- 172.21.15.43:0/1704578144 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b97586f810 con 0x55b975860f30 2024-09-19T05:10:54.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.535+0000 7fb867e00640 1 -- 172.21.15.43:0/1704578144 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb87c004cc0 con 0x55b975860f30 2024-09-19T05:10:54.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.535+0000 7fb867e00640 1 -- 172.21.15.43:0/1704578144 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb87c0056f0 con 0x55b975860f30 2024-09-19T05:10:54.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.535+0000 7fb88bfff740 10 client.26494 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:10:54.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.535+0000 7fb88bfff740 10 client.26494 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:10:54.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.535+0000 7fb88bfff740 20 client.26494 populate_metadata read hostname 'smithi043' 2024-09-19T05:10:54.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.535+0000 7fb88bfff740 1 --2- 172.21.15.43:0/1704578144 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55b975870b60 0x55b975872f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:54.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.535+0000 7fb88bfff740 4 client.26494 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:10:54.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.535+0000 7fb88bfff740 1 -- 172.21.15.43:0/1704578144 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9757c9950 con 0x55b975870b60 2024-09-19T05:10:54.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.535+0000 7fb882800640 1 --2- 172.21.15.43:0/1704578144 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55b975870b60 0x55b975872f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:54.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.536+0000 7fb882800640 1 --2- 172.21.15.43:0/1704578144 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55b975870b60 0x55b975872f50 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84785036dbd566ad server_cookie=bba8ac758f04c825 in_seq=0 out_seq=0 2024-09-19T05:10:54.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.536+0000 7fb867e00640 10 client.26494 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:10:54.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.537+0000 7fb867e00640 1 -- 172.21.15.43:0/1704578144 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b9757c9950 con 0x55b975870b60 2024-09-19T05:10:54.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.537+0000 7fb867e00640 10 client.26494 handle_command_reply: tid=0 2024-09-19T05:10:54.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.537+0000 7fb88bfff740 1 client.26494 shutdown 2024-09-19T05:10:54.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.537+0000 7fb88bfff740 1 -- 172.21.15.43:0/1704578144 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b975860f30 msgr2=0x55b97585e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:54.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.537+0000 7fb88bfff740 1 --2- 172.21.15.43:0/1704578144 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b975860f30 0x55b97585e8a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb87c006010 tx=0x7fb87c004000 comp rx=0 tx=0).stop 2024-09-19T05:10:54.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.537+0000 7fb88bfff740 1 -- 172.21.15.43:0/1704578144 shutdown_connections 2024-09-19T05:10:54.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.537+0000 7fb88bfff740 1 --2- 172.21.15.43:0/1704578144 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55b975870b60 0x55b975872f50 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:54.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.537+0000 7fb88bfff740 1 --2- 172.21.15.43:0/1704578144 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b975860f30 0x55b97585e8a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:54.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.537+0000 7fb88bfff740 1 --2- 172.21.15.43:0/1704578144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b975856670 0x55b97585e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:54.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.537+0000 7fb88bfff740 1 --2- 172.21.15.43:0/1704578144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b975855d50 0x55b97585de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:54.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.537+0000 7fb867e00640 0 client.26494 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:10:54.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.537+0000 7fb88bfff740 1 -- 172.21.15.43:0/1704578144 >> 172.21.15.43:0/1704578144 conn(0x55b9758522e0 msgr2=0x55b975858010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:54.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.537+0000 7fb88bfff740 1 -- 172.21.15.43:0/1704578144 shutdown_connections 2024-09-19T05:10:54.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.537+0000 7fb88bfff740 1 -- 172.21.15.43:0/1704578144 wait complete. 2024-09-19T05:10:54.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.538+0000 7fb88bfff740 20 client.26494 trim_cache size 0 max 16384 2024-09-19T05:10:54.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.556+0000 7fb88bfff740 1 Processor -- start 2024-09-19T05:10:54.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.556+0000 7fb88bfff740 1 -- start start 2024-09-19T05:10:54.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.556+0000 7fb88bfff740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b975855d50 0x55b975878690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:54.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.556+0000 7fb88bfff740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b975856670 0x55b975878bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:54.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.556+0000 7fb88bfff740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b975860f30 0x55b97585b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:54.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.556+0000 7fb88bfff740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb8841afa20 con 0x55b975856670 2024-09-19T05:10:54.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.556+0000 7fb88bfff740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb8841b1560 con 0x55b975860f30 2024-09-19T05:10:54.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.556+0000 7fb88bfff740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb8841b00f0 con 0x55b975855d50 2024-09-19T05:10:54.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.556+0000 7fb879e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b975856670 0x55b975878bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:54.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.556+0000 7fb879e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b975856670 0x55b975878bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56756/0 (socket says 172.21.15.43:56756) 2024-09-19T05:10:54.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.556+0000 7fb879e00640 1 -- 172.21.15.43:0/365286572 learned_addr learned my addr 172.21.15.43:0/365286572 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:54.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.556+0000 7fb883200640 1 --2- 172.21.15.43:0/365286572 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b975860f30 0x55b97585b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:54.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.556+0000 7fb882800640 1 --2- 172.21.15.43:0/365286572 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b975855d50 0x55b975878690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:54.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.557+0000 7fb879e00640 1 -- 172.21.15.43:0/365286572 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b975855d50 msgr2=0x55b975878690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:54.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.557+0000 7fb879e00640 1 --2- 172.21.15.43:0/365286572 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b975855d50 0x55b975878690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:54.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.557+0000 7fb879e00640 1 -- 172.21.15.43:0/365286572 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b975860f30 msgr2=0x55b97585b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:54.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.557+0000 7fb879e00640 1 --2- 172.21.15.43:0/365286572 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b975860f30 0x55b97585b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:54.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.557+0000 7fb879e00640 1 -- 172.21.15.43:0/365286572 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b975852110 con 0x55b975856670 2024-09-19T05:10:54.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.557+0000 7fb883200640 1 --2- 172.21.15.43:0/365286572 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b975860f30 0x55b97585b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:10:54.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.557+0000 7fb879e00640 1 --2- 172.21.15.43:0/365286572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b975856670 0x55b975878bd0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb86c09c0c0 tx=0x7fb86c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35703a15e02ac679 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:54.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.557+0000 7fb872a00640 1 -- 172.21.15.43:0/365286572 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c0a4f80 con 0x55b975856670 2024-09-19T05:10:54.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.557+0000 7fb872a00640 1 -- 172.21.15.43:0/365286572 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb86c0cda00 con 0x55b975856670 2024-09-19T05:10:54.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.557+0000 7fb872a00640 1 -- 172.21.15.43:0/365286572 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c0d2350 con 0x55b975856670 2024-09-19T05:10:54.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.558+0000 7fb88bfff740 1 -- 172.21.15.43:0/365286572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b975856670 msgr2=0x55b975878bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:54.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.558+0000 7fb88bfff740 1 --2- 172.21.15.43:0/365286572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b975856670 0x55b975878bd0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb86c09c0c0 tx=0x7fb86c000bc0 comp rx=0 tx=0).stop 2024-09-19T05:10:54.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.558+0000 7fb88bfff740 1 -- 172.21.15.43:0/365286572 shutdown_connections 2024-09-19T05:10:54.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.558+0000 7fb88bfff740 1 --2- 172.21.15.43:0/365286572 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b975860f30 0x55b97585b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:54.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.558+0000 7fb88bfff740 1 --2- 172.21.15.43:0/365286572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b975856670 0x55b975878bd0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:54.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.558+0000 7fb88bfff740 1 --2- 172.21.15.43:0/365286572 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b975855d50 0x55b975878690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:54.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.558+0000 7fb88bfff740 1 -- 172.21.15.43:0/365286572 >> 172.21.15.43:0/365286572 conn(0x55b9758522e0 msgr2=0x55b97585fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:54.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.558+0000 7fb88bfff740 1 -- 172.21.15.43:0/365286572 shutdown_connections 2024-09-19T05:10:54.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.559+0000 7fb88bfff740 1 -- 172.21.15.43:0/365286572 wait complete. 2024-09-19T05:10:54.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.559+0000 7fb88bfff740 1 -- start start 2024-09-19T05:10:54.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.559+0000 7fb88bfff740 1 Processor -- start 2024-09-19T05:10:54.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.560+0000 7fb88bfff740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b975855d50 0x55b9758654c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:54.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.560+0000 7fb88bfff740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b975856670 0x55b975865a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:54.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.560+0000 7fb88bfff740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b975860f30 0x55b9758da660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:54.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.560+0000 7fb88bfff740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb8841b0e90 con 0x55b975855d50 2024-09-19T05:10:54.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.560+0000 7fb88bfff740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb8841b07c0 con 0x55b975856670 2024-09-19T05:10:54.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.560+0000 7fb88bfff740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb8841ada00 con 0x55b975860f30 2024-09-19T05:10:54.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.560+0000 7fb88bfff740 10 client.0 _ll_register_callbacks cb 0x7fb87c004370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:54.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.560+0000 7fb882800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b975855d50 0x55b9758654c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:54.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.560+0000 7fb882800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b975855d50 0x55b9758654c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56766/0 (socket says 172.21.15.43:56766) 2024-09-19T05:10:54.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.560+0000 7fb882800640 1 -- 172.21.15.43:0/277307372 learned_addr learned my addr 172.21.15.43:0/277307372 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:54.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.560+0000 7fb883200640 1 --2- 172.21.15.43:0/277307372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b975860f30 0x55b9758da660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:54.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.560+0000 7fb879e00640 1 --2- 172.21.15.43:0/277307372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b975856670 0x55b975865a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:54.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.560+0000 7fb882800640 1 -- 172.21.15.43:0/277307372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b975860f30 msgr2=0x55b9758da660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:54.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.560+0000 7fb882800640 1 --2- 172.21.15.43:0/277307372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b975860f30 0x55b9758da660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:54.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.560+0000 7fb882800640 1 -- 172.21.15.43:0/277307372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b975856670 msgr2=0x55b975865a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:54.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.560+0000 7fb882800640 1 --2- 172.21.15.43:0/277307372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b975856670 0x55b975865a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:54.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.560+0000 7fb882800640 1 -- 172.21.15.43:0/277307372 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b975851b90 con 0x55b975855d50 2024-09-19T05:10:54.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.560+0000 7fb879e00640 1 --2- 172.21.15.43:0/277307372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b975856670 0x55b975865a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:10:54.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.560+0000 7fb883200640 1 --2- 172.21.15.43:0/277307372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b975860f30 0x55b9758da660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:10:54.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.561+0000 7fb882800640 1 --2- 172.21.15.43:0/277307372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b975855d50 0x55b9758654c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb8740045a0 tx=0x7fb87403e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=865b2283790399df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:54.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.561+0000 7fb867e00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:10:54.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.561+0000 7fb867e00640 1 -- 172.21.15.43:0/277307372 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb874035040 con 0x55b975855d50 2024-09-19T05:10:54.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.561+0000 7fb88bfff740 1 -- 172.21.15.43:0/277307372 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb86c0cda00 con 0x55b975855d50 2024-09-19T05:10:54.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.561+0000 7fb867e00640 1 -- 172.21.15.43:0/277307372 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fb874004c50 con 0x55b975855d50 2024-09-19T05:10:54.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.562+0000 7fb867e00640 1 -- 172.21.15.43:0/277307372 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb874004e70 con 0x55b975855d50 2024-09-19T05:10:54.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.562+0000 7fb88bfff740 10 client.16533 fetch_fsmap learned FSMap version 29 2024-09-19T05:10:54.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.562+0000 7fb88bfff740 1 -- 172.21.15.43:0/277307372 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b9758dbe00 con 0x55b975855d50 2024-09-19T05:10:54.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.562+0000 7fb867e00640 1 -- 172.21.15.43:0/277307372 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb86c0cda00 con 0x55b975855d50 2024-09-19T05:10:54.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.562+0000 7fb867e00640 1 -- 172.21.15.43:0/277307372 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb87c0046e0 con 0x55b975855d50 2024-09-19T05:10:54.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.562+0000 7fb88bfff740 10 client.16533 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:10:54.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.562+0000 7fb88bfff740 10 client.16533 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:10:54.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.562+0000 7fb88bfff740 20 client.16533 populate_metadata read hostname 'smithi043' 2024-09-19T05:10:54.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.562+0000 7fb88bfff740 1 --2- 172.21.15.43:0/277307372 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55b9758dc4d0 0x55b9758de8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:54.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.562+0000 7fb88bfff740 4 client.16533 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:10:54.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.562+0000 7fb88bfff740 1 -- 172.21.15.43:0/277307372 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b9757c3e60 con 0x55b9758dc4d0 2024-09-19T05:10:54.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.563+0000 7fb879e00640 1 --2- 172.21.15.43:0/277307372 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55b9758dc4d0 0x55b9758de8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:54.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.563+0000 7fb879e00640 1 --2- 172.21.15.43:0/277307372 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55b9758dc4d0 0x55b9758de8f0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3e77cd015cd678 server_cookie=c2e6f6fc87292e0 in_seq=0 out_seq=0 2024-09-19T05:10:54.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.563+0000 7fb867e00640 10 client.16533 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:10:54.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.563+0000 7fb867e00640 1 -- 172.21.15.43:0/277307372 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b9757c3e60 con 0x55b9758dc4d0 2024-09-19T05:10:54.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.563+0000 7fb867e00640 10 client.16533 handle_command_reply: tid=0 2024-09-19T05:10:54.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.563+0000 7fb88bfff740 1 client.16533 shutdown 2024-09-19T05:10:54.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.564+0000 7fb88bfff740 1 -- 172.21.15.43:0/277307372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b975855d50 msgr2=0x55b9758654c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:54.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.564+0000 7fb88bfff740 1 --2- 172.21.15.43:0/277307372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b975855d50 0x55b9758654c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb8740045a0 tx=0x7fb87403e070 comp rx=0 tx=0).stop 2024-09-19T05:10:54.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.564+0000 7fb88bfff740 1 -- 172.21.15.43:0/277307372 shutdown_connections 2024-09-19T05:10:54.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.564+0000 7fb88bfff740 1 --2- 172.21.15.43:0/277307372 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55b9758dc4d0 0x55b9758de8f0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:54.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.564+0000 7fb88bfff740 1 --2- 172.21.15.43:0/277307372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b975860f30 0x55b9758da660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:54.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.564+0000 7fb88bfff740 1 --2- 172.21.15.43:0/277307372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b975856670 0x55b975865a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:54.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.564+0000 7fb88bfff740 1 --2- 172.21.15.43:0/277307372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b975855d50 0x55b9758654c0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:54.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.564+0000 7fb88bfff740 1 -- 172.21.15.43:0/277307372 >> 172.21.15.43:0/277307372 conn(0x55b9758522e0 msgr2=0x55b97585ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:54.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.564+0000 7fb867e00640 0 client.16533 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:10:54.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.564+0000 7fb88bfff740 1 -- 172.21.15.43:0/277307372 shutdown_connections 2024-09-19T05:10:54.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.564+0000 7fb88bfff740 1 -- 172.21.15.43:0/277307372 wait complete. 2024-09-19T05:10:54.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.564+0000 7fb88bfff740 20 client.16533 trim_cache size 0 max 16384 2024-09-19T05:10:54.570 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:10:54.570 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-09-19T05:10:54.570 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-09-19T05:10:54.570 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:10:54.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.565+0000 7fb888a00640 1 -- 172.21.15.43:0/3070196239 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb8680a33b0 msgr2=0x7fb8680a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:54.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.565+0000 7fb888a00640 1 --2- 172.21.15.43:0/3070196239 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb8680a33b0 0x7fb8680a5870 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb874006010 tx=0x7fb874000bc0 comp rx=0 tx=0).stop 2024-09-19T05:10:54.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.565+0000 7fb888a00640 1 -- 172.21.15.43:0/3070196239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb884102360 msgr2=0x7fb88419aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:54.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.565+0000 7fb888a00640 1 --2- 172.21.15.43:0/3070196239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb884102360 0x7fb88419aa20 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb86c00e9f0 tx=0x7fb86c00eeb0 comp rx=0 tx=0).stop 2024-09-19T05:10:54.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.565+0000 7fb888a00640 1 -- 172.21.15.43:0/3070196239 shutdown_connections 2024-09-19T05:10:54.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.565+0000 7fb888a00640 1 --2- 172.21.15.43:0/3070196239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb8841079c0 0x7fb88419edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:54.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.565+0000 7fb888a00640 1 --2- 172.21.15.43:0/3070196239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb884102360 0x7fb88419aa20 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:54.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.565+0000 7fb888a00640 1 --2- 172.21.15.43:0/3070196239 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb8680a33b0 0x7fb8680a5870 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:54.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.565+0000 7fb888a00640 1 --2- 172.21.15.43:0/3070196239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8841019c0 0x7fb88419a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:54.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.565+0000 7fb888a00640 1 -- 172.21.15.43:0/3070196239 >> 172.21.15.43:0/3070196239 conn(0x7fb8840fd740 msgr2=0x7fb88410cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:54.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.566+0000 7fb888a00640 1 -- 172.21.15.43:0/3070196239 shutdown_connections 2024-09-19T05:10:54.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:54.566+0000 7fb888a00640 1 -- 172.21.15.43:0/3070196239 wait complete. 2024-09-19T05:10:55.371 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T05:10:55.371 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:10:55.755 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:10:56.078 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:55 smithi043 ceph-mon[30800]: pgmap v191: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:10:56.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:55 smithi151 ceph-mon[35425]: pgmap v191: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:10:56.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:55 smithi096 ceph-mon[35085]: pgmap v191: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:10:56.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.303+0000 7f438d400640 1 -- 172.21.15.43:0/1746086183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43881049a0 msgr2=0x7f43881089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:56.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.303+0000 7f438d400640 1 --2- 172.21.15.43:0/1746086183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43881049a0 0x7f43881089f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4370009900 tx=0x7f437002a3f0 comp rx=0 tx=0).stop 2024-09-19T05:10:56.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.303+0000 7f438d400640 1 -- 172.21.15.43:0/1746086183 shutdown_connections 2024-09-19T05:10:56.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.303+0000 7f438d400640 1 --2- 172.21.15.43:0/1746086183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4388109120 0x7f4388110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:56.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.303+0000 7f438d400640 1 --2- 172.21.15.43:0/1746086183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43881049a0 0x7f43881089f0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:56.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.303+0000 7f438d400640 1 --2- 172.21.15.43:0/1746086183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4388103ff0 0x7f43881043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:56.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.303+0000 7f438d400640 1 -- 172.21.15.43:0/1746086183 >> 172.21.15.43:0/1746086183 conn(0x7f43880ffa90 msgr2=0x7f4388101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:56.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.303+0000 7f438d400640 1 -- 172.21.15.43:0/1746086183 shutdown_connections 2024-09-19T05:10:56.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.303+0000 7f438d400640 1 -- 172.21.15.43:0/1746086183 wait complete. 2024-09-19T05:10:56.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.304+0000 7f438d400640 1 Processor -- start 2024-09-19T05:10:56.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.304+0000 7f438d400640 1 -- start start 2024-09-19T05:10:56.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.304+0000 7f438d400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4388103ff0 0x7f438819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:56.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.304+0000 7f438d400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43881049a0 0x7f438819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:56.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.304+0000 7f438d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4388109120 0x7f438819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:56.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.304+0000 7f438d400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4388114190 con 0x7f4388109120 2024-09-19T05:10:56.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.304+0000 7f438d400640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4388114010 con 0x7f4388103ff0 2024-09-19T05:10:56.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.304+0000 7f438d400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4388114310 con 0x7f43881049a0 2024-09-19T05:10:56.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.305+0000 7f4387200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4388109120 0x7f438819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:56.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.305+0000 7f4387200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4388109120 0x7f438819eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38214/0 (socket says 172.21.15.43:38214) 2024-09-19T05:10:56.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.305+0000 7f4386800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4388103ff0 0x7f438819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:56.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.305+0000 7f4387200640 1 -- 172.21.15.43:0/3477194325 learned_addr learned my addr 172.21.15.43:0/3477194325 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:56.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.305+0000 7f437de00640 1 --2- 172.21.15.43:0/3477194325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43881049a0 0x7f438819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:56.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.305+0000 7f4387200640 1 -- 172.21.15.43:0/3477194325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43881049a0 msgr2=0x7f438819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:56.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.305+0000 7f4387200640 1 --2- 172.21.15.43:0/3477194325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43881049a0 0x7f438819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:56.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.305+0000 7f4387200640 1 -- 172.21.15.43:0/3477194325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4388103ff0 msgr2=0x7f438819a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:56.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.305+0000 7f4387200640 1 --2- 172.21.15.43:0/3477194325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4388103ff0 0x7f438819a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:56.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.305+0000 7f4387200640 1 -- 172.21.15.43:0/3477194325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f438819f5a0 con 0x7f4388109120 2024-09-19T05:10:56.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.305+0000 7f4386800640 1 --2- 172.21.15.43:0/3477194325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4388103ff0 0x7f438819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:10:56.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.306+0000 7f4387200640 1 --2- 172.21.15.43:0/3477194325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4388109120 0x7f438819eec0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f438000e970 tx=0x7f438000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:56.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.306+0000 7f437fe00640 1 -- 172.21.15.43:0/3477194325 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438000ccc0 con 0x7f4388109120 2024-09-19T05:10:56.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.306+0000 7f437fe00640 1 -- 172.21.15.43:0/3477194325 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f438000ce60 con 0x7f4388109120 2024-09-19T05:10:56.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.306+0000 7f438d400640 1 -- 172.21.15.43:0/3477194325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f438819f890 con 0x7f4388109120 2024-09-19T05:10:56.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.306+0000 7f437fe00640 1 -- 172.21.15.43:0/3477194325 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4380010490 con 0x7f4388109120 2024-09-19T05:10:56.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.306+0000 7f438d400640 1 -- 172.21.15.43:0/3477194325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43881a7170 con 0x7f4388109120 2024-09-19T05:10:56.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.307+0000 7f438d400640 1 -- 172.21.15.43:0/3477194325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43881043d0 con 0x7f4388109120 2024-09-19T05:10:56.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.309+0000 7f437fe00640 1 -- 172.21.15.43:0/3477194325 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4380005020 con 0x7f4388109120 2024-09-19T05:10:56.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.311+0000 7f437fe00640 1 --2- 172.21.15.43:0/3477194325 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f436c0a3430 0x7f436c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:56.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.312+0000 7f437fe00640 1 -- 172.21.15.43:0/3477194325 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f43800d0090 con 0x7f4388109120 2024-09-19T05:10:56.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.312+0000 7f4386800640 1 --2- 172.21.15.43:0/3477194325 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f436c0a3430 0x7f436c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:56.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.312+0000 7f4386800640 1 --2- 172.21.15.43:0/3477194325 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f436c0a3430 0x7f436c0a58f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f43780099a0 tx=0x7f4378009500 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:56.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.316+0000 7f437fe00640 1 -- 172.21.15.43:0/3477194325 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f438009ca20 con 0x7f4388109120 2024-09-19T05:10:56.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.542+0000 7f438d400640 1 -- 172.21.15.43:0/3477194325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43881a7400 con 0x7f4388109120 2024-09-19T05:10:56.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.543+0000 7f437fe00640 1 -- 172.21.15.43:0/3477194325 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11965 (secure 0 0 0) 0x7f438009c320 con 0x7f4388109120 2024-09-19T05:10:56.545 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:10:56.545 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":29,"btime":"2024-09-19T05:04:38:785407+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/773801169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":773801169},{"type":"v1","addr":"172.21.15.43:6835","nonce":773801169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1799137902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1799137902},{"type":"v1","addr":"172.21.15.43:6837","nonce":1799137902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:02:25.463041+0000","modified":"2024-09-19T05:04:38.039809+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24518,"mds_2":24511,"mds_3":24506,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14541":{"gid":14541,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/794881135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":794881135},{"type":"v1","addr":"172.21.15.43:6841","nonce":794881135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/2836453158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2836453158},{"type":"v1","addr":"172.21.15.151:6833","nonce":2836453158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/178785812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":178785812},{"type":"v1","addr":"172.21.15.96:6833","nonce":178785812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1671043820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1671043820},{"type":"v1","addr":"172.21.15.96:6835","nonce":1671043820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2972514328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2972514328},{"type":"v1","addr":"172.21.15.151:6835","nonce":2972514328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.151:6837/3960378010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3960378010},{"type":"v1","addr":"172.21.15.151:6837","nonce":3960378010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6837/3978388286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3978388286},{"type":"v1","addr":"172.21.15.96:6837","nonce":3978388286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.151:6839/3794104313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3794104313},{"type":"v1","addr":"172.21.15.151:6839","nonce":3794104313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.43:6839/3404555030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3404555030},{"type":"v1","addr":"172.21.15.43:6839","nonce":3404555030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/605527621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":605527621},{"type":"v1","addr":"172.21.15.96:6839","nonce":605527621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24518,24511,24506,24493]},"id":1}]} 2024-09-19T05:10:56.546 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 29 2024-09-19T05:10:56.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.545+0000 7f438d400640 1 -- 172.21.15.43:0/3477194325 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f436c0a3430 msgr2=0x7f436c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:56.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.545+0000 7f438d400640 1 --2- 172.21.15.43:0/3477194325 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f436c0a3430 0x7f436c0a58f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f43780099a0 tx=0x7f4378009500 comp rx=0 tx=0).stop 2024-09-19T05:10:56.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.545+0000 7f438d400640 1 -- 172.21.15.43:0/3477194325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4388109120 msgr2=0x7f438819eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:56.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.545+0000 7f438d400640 1 --2- 172.21.15.43:0/3477194325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4388109120 0x7f438819eec0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f438000e970 tx=0x7f438000ed70 comp rx=0 tx=0).stop 2024-09-19T05:10:56.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.545+0000 7f438d400640 1 -- 172.21.15.43:0/3477194325 shutdown_connections 2024-09-19T05:10:56.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.545+0000 7f438d400640 1 --2- 172.21.15.43:0/3477194325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4388109120 0x7f438819eec0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:56.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.545+0000 7f438d400640 1 --2- 172.21.15.43:0/3477194325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43881049a0 0x7f438819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:56.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.545+0000 7f438d400640 1 --2- 172.21.15.43:0/3477194325 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f436c0a3430 0x7f436c0a58f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:56.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.545+0000 7f438d400640 1 --2- 172.21.15.43:0/3477194325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4388103ff0 0x7f438819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:56.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.545+0000 7f438d400640 1 -- 172.21.15.43:0/3477194325 >> 172.21.15.43:0/3477194325 conn(0x7f43880ffa90 msgr2=0x7f4388101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:56.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.545+0000 7f438d400640 1 -- 172.21.15.43:0/3477194325 shutdown_connections 2024-09-19T05:10:56.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:56.545+0000 7f438d400640 1 -- 172.21.15.43:0/3477194325 wait complete. 2024-09-19T05:10:57.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:56 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/3477194325' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:10:57.197 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:56 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/3477194325' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:10:57.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:56 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/3477194325' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:10:57.235 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T05:10:57.621 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:10:58.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:57 smithi043 ceph-mon[30800]: pgmap v192: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:10:58.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:57 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:10:58.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:57 smithi151 ceph-mon[35425]: pgmap v192: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:10:58.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:57 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:10:58.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:57 smithi096 ceph-mon[35085]: pgmap v192: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:10:58.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:57 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:10:58.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.216+0000 7fd9e2e00640 1 -- 172.21.15.43:0/1422534832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9dc102b90 msgr2=0x7fd9dc102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:58.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.216+0000 7fd9e2e00640 1 --2- 172.21.15.43:0/1422534832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9dc102b90 0x7fd9dc102f70 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0009900 tx=0x7fd9d002a480 comp rx=0 tx=0).stop 2024-09-19T05:10:58.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.216+0000 7fd9e2e00640 1 -- 172.21.15.43:0/1422534832 shutdown_connections 2024-09-19T05:10:58.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.216+0000 7fd9e2e00640 1 --2- 172.21.15.43:0/1422534832 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9dc10bfc0 0x7fd9dc10e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:58.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.216+0000 7fd9e2e00640 1 --2- 172.21.15.43:0/1422534832 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9dc1034b0 0x7fd9dc10b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:58.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.216+0000 7fd9e2e00640 1 --2- 172.21.15.43:0/1422534832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9dc102b90 0x7fd9dc102f70 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:58.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.216+0000 7fd9e2e00640 1 -- 172.21.15.43:0/1422534832 >> 172.21.15.43:0/1422534832 conn(0x7fd9dc0fc610 msgr2=0x7fd9dc0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:58.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.216+0000 7fd9e2e00640 1 -- 172.21.15.43:0/1422534832 shutdown_connections 2024-09-19T05:10:58.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.216+0000 7fd9e2e00640 1 -- 172.21.15.43:0/1422534832 wait complete. 2024-09-19T05:10:58.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.217+0000 7fd9e2e00640 1 Processor -- start 2024-09-19T05:10:58.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.218+0000 7fd9e2e00640 1 -- start start 2024-09-19T05:10:58.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.218+0000 7fd9e2e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9dc102b90 0x7fd9dc1982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:58.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.218+0000 7fd9e2e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9dc1034b0 0x7fd9dc198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:58.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.218+0000 7fd9e2e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9dc10bfc0 0x7fd9dc19cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:58.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.218+0000 7fd9e2e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd9dc111e10 con 0x7fd9dc1034b0 2024-09-19T05:10:58.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.218+0000 7fd9e2e00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd9dc111c90 con 0x7fd9dc10bfc0 2024-09-19T05:10:58.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.218+0000 7fd9e2e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd9dc111f90 con 0x7fd9dc102b90 2024-09-19T05:10:58.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.218+0000 7fd9e1a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9dc102b90 0x7fd9dc1982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:58.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.218+0000 7fd9e1a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9dc102b90 0x7fd9dc1982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:59926/0 (socket says 172.21.15.43:59926) 2024-09-19T05:10:58.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.218+0000 7fd9e1000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9dc1034b0 0x7fd9dc198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:58.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.218+0000 7fd9e1a00640 1 -- 172.21.15.43:0/3292920357 learned_addr learned my addr 172.21.15.43:0/3292920357 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:58.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.218+0000 7fd9e2400640 1 --2- 172.21.15.43:0/3292920357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9dc10bfc0 0x7fd9dc19cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:58.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.219+0000 7fd9e1a00640 1 -- 172.21.15.43:0/3292920357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9dc10bfc0 msgr2=0x7fd9dc19cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:58.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.219+0000 7fd9e1a00640 1 --2- 172.21.15.43:0/3292920357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9dc10bfc0 0x7fd9dc19cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:58.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.219+0000 7fd9e1a00640 1 -- 172.21.15.43:0/3292920357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9dc1034b0 msgr2=0x7fd9dc198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:58.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.219+0000 7fd9e1a00640 1 --2- 172.21.15.43:0/3292920357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9dc1034b0 0x7fd9dc198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:58.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.219+0000 7fd9e1a00640 1 -- 172.21.15.43:0/3292920357 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9dc19d280 con 0x7fd9dc102b90 2024-09-19T05:10:58.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.219+0000 7fd9e1a00640 1 --2- 172.21.15.43:0/3292920357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9dc102b90 0x7fd9dc1982d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0004770 tx=0x7fd9d0003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:58.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.219+0000 7fd9da000640 1 -- 172.21.15.43:0/3292920357 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d003f070 con 0x7fd9dc102b90 2024-09-19T05:10:58.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.219+0000 7fd9da000640 1 -- 172.21.15.43:0/3292920357 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9d0031ca0 con 0x7fd9dc102b90 2024-09-19T05:10:58.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.220+0000 7fd9e2e00640 1 -- 172.21.15.43:0/3292920357 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9dc19d510 con 0x7fd9dc102b90 2024-09-19T05:10:58.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.220+0000 7fd9da000640 1 -- 172.21.15.43:0/3292920357 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0035030 con 0x7fd9dc102b90 2024-09-19T05:10:58.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.220+0000 7fd9e2e00640 1 -- 172.21.15.43:0/3292920357 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9dc1a4e50 con 0x7fd9dc102b90 2024-09-19T05:10:58.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.223+0000 7fd9da000640 1 -- 172.21.15.43:0/3292920357 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fd9d0042070 con 0x7fd9dc102b90 2024-09-19T05:10:58.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.224+0000 7fd9e6eb8740 1 Processor -- start 2024-09-19T05:10:58.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.224+0000 7fd9e6eb8740 1 -- start start 2024-09-19T05:10:58.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.224+0000 7fd9da000640 1 --2- 172.21.15.43:0/3292920357 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fd9c40a33b0 0x7fd9c40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:58.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.224+0000 7fd9e6eb8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c14017de40 0x55c14017e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:58.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.224+0000 7fd9e6eb8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c14017e760 0x55c140188ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:58.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.225+0000 7fd9e6eb8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c140189020 0x55c14018b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:58.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.225+0000 7fd9e6eb8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c1400e43e0 con 0x55c14017e760 2024-09-19T05:10:58.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.225+0000 7fd9e6eb8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c14008bd80 con 0x55c140189020 2024-09-19T05:10:58.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.225+0000 7fd9e6eb8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c1400e03a0 con 0x55c14017de40 2024-09-19T05:10:58.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.225+0000 7fd9da000640 1 -- 172.21.15.43:0/3292920357 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fd9d00f0090 con 0x7fd9dc102b90 2024-09-19T05:10:58.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.225+0000 7fd9e2400640 1 --2- 172.21.15.43:0/3292920357 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fd9c40a33b0 0x7fd9c40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:58.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.225+0000 7fd9e1000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c14017e760 0x55c140188ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:58.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.225+0000 7fd9e1000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c14017e760 0x55c140188ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38248/0 (socket says 172.21.15.43:38248) 2024-09-19T05:10:58.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.225+0000 7fd9e1000640 1 -- 172.21.15.43:0/3580150761 learned_addr learned my addr 172.21.15.43:0/3580150761 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:58.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.225+0000 7fd9e1a00640 1 --2- 172.21.15.43:0/3580150761 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c14017de40 0x55c14017e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:58.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.225+0000 7fd9e2400640 1 --2- 172.21.15.43:0/3580150761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c140189020 0x55c14018b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:58.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.225+0000 7fd9e1a00640 1 -- 172.21.15.43:0/3580150761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c140189020 msgr2=0x55c14018b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:58.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.225+0000 7fd9e1a00640 1 --2- 172.21.15.43:0/3580150761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c140189020 0x55c14018b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:58.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.225+0000 7fd9e1a00640 1 -- 172.21.15.43:0/3580150761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c14017e760 msgr2=0x55c140188ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:58.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.225+0000 7fd9e1a00640 1 --2- 172.21.15.43:0/3580150761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c14017e760 0x55c140188ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:58.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.225+0000 7fd9e1a00640 1 -- 172.21.15.43:0/3580150761 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1400aaae0 con 0x55c14017de40 2024-09-19T05:10:58.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.226+0000 7fd9e1000640 1 --2- 172.21.15.43:0/3580150761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c14017e760 0x55c140188ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:10:58.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.226+0000 7fd9e2400640 1 --2- 172.21.15.43:0/3292920357 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fd9c40a33b0 0x7fd9c40a5870 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4005fd0 tx=0x7fd9d4009ad0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:58.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.226+0000 7fd9e1a00640 1 --2- 172.21.15.43:0/3580150761 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c14017de40 0x55c14017e220 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd9d00ed790 tx=0x7fd9d0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52f42070e72c8f5a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:58.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.226+0000 7fd9cea00640 1 -- 172.21.15.43:0/3580150761 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0049040 con 0x55c14017de40 2024-09-19T05:10:58.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.226+0000 7fd9cea00640 1 -- 172.21.15.43:0/3580150761 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9d002e5e0 con 0x55c14017de40 2024-09-19T05:10:58.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.226+0000 7fd9cea00640 1 -- 172.21.15.43:0/3580150761 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d00c06d0 con 0x55c14017de40 2024-09-19T05:10:58.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.227+0000 7fd9e6eb8740 1 -- 172.21.15.43:0/3580150761 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c14017de40 msgr2=0x55c14017e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:58.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.227+0000 7fd9e6eb8740 1 --2- 172.21.15.43:0/3580150761 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c14017de40 0x55c14017e220 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd9d00ed790 tx=0x7fd9d0009260 comp rx=0 tx=0).stop 2024-09-19T05:10:58.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.227+0000 7fd9e6eb8740 1 -- 172.21.15.43:0/3580150761 shutdown_connections 2024-09-19T05:10:58.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.227+0000 7fd9e6eb8740 1 --2- 172.21.15.43:0/3580150761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c140189020 0x55c14018b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:58.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.227+0000 7fd9e6eb8740 1 --2- 172.21.15.43:0/3580150761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c14017e760 0x55c140188ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:58.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.227+0000 7fd9e6eb8740 1 --2- 172.21.15.43:0/3580150761 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c14017de40 0x55c14017e220 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:58.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.227+0000 7fd9e6eb8740 1 -- 172.21.15.43:0/3580150761 >> 172.21.15.43:0/3580150761 conn(0x55c14017a3d0 msgr2=0x55c14017a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:58.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.227+0000 7fd9e6eb8740 1 -- 172.21.15.43:0/3580150761 shutdown_connections 2024-09-19T05:10:58.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.227+0000 7fd9e6eb8740 1 -- 172.21.15.43:0/3580150761 wait complete. 2024-09-19T05:10:58.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.228+0000 7fd9e6eb8740 1 -- start start 2024-09-19T05:10:58.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.228+0000 7fd9e6eb8740 1 Processor -- start 2024-09-19T05:10:58.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.228+0000 7fd9e6eb8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14017de40 0x55c14018d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:58.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.228+0000 7fd9e6eb8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c14017e760 0x55c14018d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:58.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.228+0000 7fd9e6eb8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c140189020 0x55c1401918d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:58.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.228+0000 7fd9e6eb8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c14018b9b0 con 0x55c14017e760 2024-09-19T05:10:58.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.228+0000 7fd9e6eb8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c14009f0f0 con 0x55c14017de40 2024-09-19T05:10:58.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.228+0000 7fd9e6eb8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c14018bb30 con 0x55c140189020 2024-09-19T05:10:58.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.228+0000 7fd9e6eb8740 10 client.0 _ll_register_callbacks cb 0x55c140086bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:58.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.229+0000 7fd9e1000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c14017e760 0x55c14018d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:58.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.229+0000 7fd9e1a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14017de40 0x55c14018d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:58.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.229+0000 7fd9e1000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c14017e760 0x55c14018d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38254/0 (socket says 172.21.15.43:38254) 2024-09-19T05:10:58.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.229+0000 7fd9e1a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14017de40 0x55c14018d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:39478/0 (socket says 172.21.15.43:39478) 2024-09-19T05:10:58.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.229+0000 7fd9e1000640 1 -- 172.21.15.43:0/2847512548 learned_addr learned my addr 172.21.15.43:0/2847512548 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:58.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.229+0000 7fd9e2400640 1 --2- 172.21.15.43:0/2847512548 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c140189020 0x55c1401918d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:58.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.229+0000 7fd9e1a00640 1 -- 172.21.15.43:0/2847512548 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c140189020 msgr2=0x55c1401918d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:58.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.229+0000 7fd9e1a00640 1 --2- 172.21.15.43:0/2847512548 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c140189020 0x55c1401918d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:58.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.229+0000 7fd9e1a00640 1 -- 172.21.15.43:0/2847512548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c14017e760 msgr2=0x55c14018d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:58.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.229+0000 7fd9e1a00640 1 --2- 172.21.15.43:0/2847512548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c14017e760 0x55c14018d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:58.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.229+0000 7fd9e1a00640 1 -- 172.21.15.43:0/2847512548 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c13fe553c0 con 0x55c14017de40 2024-09-19T05:10:58.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.229+0000 7fd9e1a00640 1 --2- 172.21.15.43:0/2847512548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14017de40 0x55c14018d220 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x55c14017ee00 tx=0x7fd9d00095d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b3720291365eca9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:58.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.229+0000 7fd9c3400640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:10:58.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.230+0000 7fd9c3400640 1 -- 172.21.15.43:0/2847512548 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0049050 con 0x55c14017de40 2024-09-19T05:10:58.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.230+0000 7fd9c3400640 1 -- 172.21.15.43:0/2847512548 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fd9d00b8a40 con 0x55c14017de40 2024-09-19T05:10:58.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.230+0000 7fd9e6eb8740 1 -- 172.21.15.43:0/2847512548 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9d002e5e0 con 0x55c14017de40 2024-09-19T05:10:58.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.230+0000 7fd9c3400640 1 -- 172.21.15.43:0/2847512548 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd9d0043780 con 0x55c14017de40 2024-09-19T05:10:58.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.230+0000 7fd9e6eb8740 10 client.26579 fetch_fsmap learned FSMap version 29 2024-09-19T05:10:58.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.230+0000 7fd9e6eb8740 1 -- 172.21.15.43:0/2847512548 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c14018b9b0 con 0x55c14017de40 2024-09-19T05:10:58.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.230+0000 7fd9c3400640 1 -- 172.21.15.43:0/2847512548 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9d002e5e0 con 0x55c14017de40 2024-09-19T05:10:58.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.230+0000 7fd9c3400640 1 -- 172.21.15.43:0/2847512548 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd9d004c060 con 0x55c14017de40 2024-09-19T05:10:58.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.230+0000 7fd9e6eb8740 10 client.26579 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:10:58.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.230+0000 7fd9e6eb8740 10 client.26579 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-19T05:10:58.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.230+0000 7fd9e6eb8740 20 client.26579 populate_metadata read hostname 'smithi043' 2024-09-19T05:10:58.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.230+0000 7fd9e6eb8740 1 --2- 172.21.15.43:0/2847512548 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55c14019cef0 0x55c14019f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:58.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.230+0000 7fd9e6eb8740 4 client.26579 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:10:58.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.230+0000 7fd9e6eb8740 1 -- 172.21.15.43:0/2847512548 --> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1400f1950 con 0x55c14019cef0 2024-09-19T05:10:58.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.231+0000 7fd9e1000640 1 --2- 172.21.15.43:0/2847512548 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55c14019cef0 0x55c14019f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:58.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.231+0000 7fd9e1000640 1 --2- 172.21.15.43:0/2847512548 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55c14019cef0 0x55c14019f2e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a6fb634ee87eda4f server_cookie=ec072685e2cb78fa in_seq=0 out_seq=0 2024-09-19T05:10:58.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.231+0000 7fd9c3400640 10 client.26579 ms_handle_connect on v2:172.21.15.151:6836/3960378010 2024-09-19T05:10:58.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.232+0000 7fd9c3400640 1 -- 172.21.15.43:0/2847512548 <== mds.4 v2:172.21.15.151:6836/3960378010 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c1400f1950 con 0x55c14019cef0 2024-09-19T05:10:58.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.232+0000 7fd9c3400640 10 client.26579 handle_command_reply: tid=0 2024-09-19T05:10:58.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.232+0000 7fd9e6eb8740 1 client.26579 shutdown 2024-09-19T05:10:58.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.232+0000 7fd9e6eb8740 1 -- 172.21.15.43:0/2847512548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14017de40 msgr2=0x55c14018d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:58.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.232+0000 7fd9e6eb8740 1 --2- 172.21.15.43:0/2847512548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14017de40 0x55c14018d220 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x55c14017ee00 tx=0x7fd9d00095d0 comp rx=0 tx=0).stop 2024-09-19T05:10:58.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.232+0000 7fd9e6eb8740 1 -- 172.21.15.43:0/2847512548 shutdown_connections 2024-09-19T05:10:58.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.232+0000 7fd9e6eb8740 1 --2- 172.21.15.43:0/2847512548 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55c14019cef0 0x55c14019f2e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:58.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.232+0000 7fd9e6eb8740 1 --2- 172.21.15.43:0/2847512548 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c140189020 0x55c1401918d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:58.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.232+0000 7fd9e6eb8740 1 --2- 172.21.15.43:0/2847512548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c14017e760 0x55c14018d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:58.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.232+0000 7fd9e6eb8740 1 --2- 172.21.15.43:0/2847512548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14017de40 0x55c14018d220 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:58.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.232+0000 7fd9c3400640 0 client.26579 ms_handle_reset on v2:172.21.15.151:6836/3960378010 2024-09-19T05:10:58.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.232+0000 7fd9e6eb8740 1 -- 172.21.15.43:0/2847512548 >> 172.21.15.43:0/2847512548 conn(0x55c14017a3d0 msgr2=0x55c140180100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:58.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.233+0000 7fd9e6eb8740 1 -- 172.21.15.43:0/2847512548 shutdown_connections 2024-09-19T05:10:58.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.233+0000 7fd9e6eb8740 1 -- 172.21.15.43:0/2847512548 wait complete. 2024-09-19T05:10:58.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.233+0000 7fd9e6eb8740 20 client.26579 trim_cache size 0 max 16384 2024-09-19T05:10:58.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.247+0000 7fd9e6eb8740 1 Processor -- start 2024-09-19T05:10:58.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.247+0000 7fd9e6eb8740 1 -- start start 2024-09-19T05:10:58.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.247+0000 7fd9e6eb8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c14017de40 0x55c1401918c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:58.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.247+0000 7fd9e6eb8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14017e760 0x55c1401a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:58.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.247+0000 7fd9e6eb8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c140189020 0x55c1401a5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:58.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.247+0000 7fd9e6eb8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd9dc1ab580 con 0x55c14017de40 2024-09-19T05:10:58.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.247+0000 7fd9e6eb8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd9dc1ad0c0 con 0x55c14017e760 2024-09-19T05:10:58.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.247+0000 7fd9e6eb8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd9dc1abc50 con 0x55c140189020 2024-09-19T05:10:58.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.247+0000 7fd9e1a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c14017de40 0x55c1401918c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:58.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.247+0000 7fd9e1000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14017e760 0x55c1401a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:58.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.247+0000 7fd9e1a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c14017de40 0x55c1401918c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38264/0 (socket says 172.21.15.43:38264) 2024-09-19T05:10:58.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.247+0000 7fd9e1000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14017e760 0x55c1401a4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:39490/0 (socket says 172.21.15.43:39490) 2024-09-19T05:10:58.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.247+0000 7fd9e1a00640 1 -- 172.21.15.43:0/2072485462 learned_addr learned my addr 172.21.15.43:0/2072485462 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:58.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.247+0000 7fd9e2400640 1 --2- 172.21.15.43:0/2072485462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c140189020 0x55c1401a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:58.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.248+0000 7fd9e1000640 1 -- 172.21.15.43:0/2072485462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c140189020 msgr2=0x55c1401a5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:58.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.248+0000 7fd9e1000640 1 --2- 172.21.15.43:0/2072485462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c140189020 0x55c1401a5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:58.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.248+0000 7fd9e1000640 1 -- 172.21.15.43:0/2072485462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c14017de40 msgr2=0x55c1401918c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:58.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.248+0000 7fd9e1000640 1 --2- 172.21.15.43:0/2072485462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c14017de40 0x55c1401918c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:58.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.248+0000 7fd9e1000640 1 -- 172.21.15.43:0/2072485462 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c14017e590 con 0x55c14017e760 2024-09-19T05:10:58.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.248+0000 7fd9e1a00640 1 --2- 172.21.15.43:0/2072485462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c14017de40 0x55c1401918c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:10:58.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.248+0000 7fd9e1000640 1 --2- 172.21.15.43:0/2072485462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14017e760 0x55c1401a4b40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x55c14019fa80 tx=0x7fd9c8005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75af2000220100f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:58.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.248+0000 7fd9ce000640 1 -- 172.21.15.43:0/2072485462 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c8003da0 con 0x55c14017e760 2024-09-19T05:10:58.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.248+0000 7fd9ce000640 1 -- 172.21.15.43:0/2072485462 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9c8010910 con 0x55c14017e760 2024-09-19T05:10:58.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.248+0000 7fd9ce000640 1 -- 172.21.15.43:0/2072485462 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c801e8c0 con 0x55c14017e760 2024-09-19T05:10:58.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.249+0000 7fd9e6eb8740 1 -- 172.21.15.43:0/2072485462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14017e760 msgr2=0x55c1401a4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:58.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.249+0000 7fd9e6eb8740 1 --2- 172.21.15.43:0/2072485462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14017e760 0x55c1401a4b40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x55c14019fa80 tx=0x7fd9c8005f90 comp rx=0 tx=0).stop 2024-09-19T05:10:58.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.249+0000 7fd9e6eb8740 1 -- 172.21.15.43:0/2072485462 shutdown_connections 2024-09-19T05:10:58.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.249+0000 7fd9e6eb8740 1 --2- 172.21.15.43:0/2072485462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c140189020 0x55c1401a5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:58.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.249+0000 7fd9e6eb8740 1 --2- 172.21.15.43:0/2072485462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14017e760 0x55c1401a4b40 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:58.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.249+0000 7fd9e6eb8740 1 --2- 172.21.15.43:0/2072485462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c14017de40 0x55c1401918c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:58.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.249+0000 7fd9e6eb8740 1 -- 172.21.15.43:0/2072485462 >> 172.21.15.43:0/2072485462 conn(0x55c14017a3d0 msgr2=0x55c14018bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:58.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.249+0000 7fd9e6eb8740 1 -- 172.21.15.43:0/2072485462 shutdown_connections 2024-09-19T05:10:58.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.249+0000 7fd9e6eb8740 1 -- 172.21.15.43:0/2072485462 wait complete. 2024-09-19T05:10:58.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.250+0000 7fd9e6eb8740 1 -- start start 2024-09-19T05:10:58.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.250+0000 7fd9e6eb8740 1 Processor -- start 2024-09-19T05:10:58.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.250+0000 7fd9e6eb8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c14017de40 0x55c1401995a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:58.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.250+0000 7fd9e6eb8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14017e760 0x55c140194980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:58.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.250+0000 7fd9e6eb8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c140189020 0x55c140194ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:58.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.250+0000 7fd9e6eb8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd9dc1ac9f0 con 0x55c14017de40 2024-09-19T05:10:58.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.250+0000 7fd9e6eb8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd9dc1ac320 con 0x55c14017e760 2024-09-19T05:10:58.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.250+0000 7fd9e6eb8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd9dc1a9560 con 0x55c140189020 2024-09-19T05:10:58.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.250+0000 7fd9e6eb8740 10 client.0 _ll_register_callbacks cb 0x7fd9c8009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:10:58.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.251+0000 7fd9e1000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14017e760 0x55c140194980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:58.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.251+0000 7fd9e1000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14017e760 0x55c140194980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:39500/0 (socket says 172.21.15.43:39500) 2024-09-19T05:10:58.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.251+0000 7fd9e1a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c14017de40 0x55c1401995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:58.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.251+0000 7fd9e1000640 1 -- 172.21.15.43:0/4133803934 learned_addr learned my addr 172.21.15.43:0/4133803934 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:10:58.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.251+0000 7fd9e2400640 1 --2- 172.21.15.43:0/4133803934 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c140189020 0x55c140194ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:58.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.251+0000 7fd9e1000640 1 -- 172.21.15.43:0/4133803934 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c140189020 msgr2=0x55c140194ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:58.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.251+0000 7fd9e1000640 1 --2- 172.21.15.43:0/4133803934 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c140189020 0x55c140194ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:58.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.251+0000 7fd9e1000640 1 -- 172.21.15.43:0/4133803934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c14017de40 msgr2=0x55c1401995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:10:58.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.251+0000 7fd9e1000640 1 --2- 172.21.15.43:0/4133803934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c14017de40 0x55c1401995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:58.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.251+0000 7fd9e1000640 1 -- 172.21.15.43:0/4133803934 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1401955b0 con 0x55c14017e760 2024-09-19T05:10:58.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.251+0000 7fd9e1000640 1 --2- 172.21.15.43:0/4133803934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14017e760 0x55c140194980 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8009700 tx=0x7fd9c8005760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b50296f72602cf42 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:10:58.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.251+0000 7fd9c3400640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:10:58.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.251+0000 7fd9c3400640 1 -- 172.21.15.43:0/4133803934 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c8036060 con 0x55c14017e760 2024-09-19T05:10:58.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.252+0000 7fd9c3400640 1 -- 172.21.15.43:0/4133803934 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fd9c8003c60 con 0x55c14017e760 2024-09-19T05:10:58.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.252+0000 7fd9e6eb8740 1 -- 172.21.15.43:0/4133803934 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9c8010910 con 0x55c14017e760 2024-09-19T05:10:58.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.252+0000 7fd9c3400640 1 -- 172.21.15.43:0/4133803934 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd9c8003e60 con 0x55c14017e760 2024-09-19T05:10:58.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.252+0000 7fd9e6eb8740 10 client.26585 fetch_fsmap learned FSMap version 29 2024-09-19T05:10:58.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.252+0000 7fd9e6eb8740 1 -- 172.21.15.43:0/4133803934 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c140207a10 con 0x55c14017e760 2024-09-19T05:10:58.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.252+0000 7fd9c3400640 1 -- 172.21.15.43:0/4133803934 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9c8010910 con 0x55c14017e760 2024-09-19T05:10:58.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.252+0000 7fd9c3400640 1 -- 172.21.15.43:0/4133803934 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd9c801ea40 con 0x55c14017e760 2024-09-19T05:10:58.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.253+0000 7fd9e6eb8740 10 client.26585 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:10:58.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.253+0000 7fd9e6eb8740 10 client.26585 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-19T05:10:58.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.253+0000 7fd9e6eb8740 20 client.26585 populate_metadata read hostname 'smithi043' 2024-09-19T05:10:58.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.253+0000 7fd9e6eb8740 1 --2- 172.21.15.43:0/4133803934 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55c140208080 0x55c14020a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:10:58.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.253+0000 7fd9e6eb8740 4 client.26585 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:10:58.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.253+0000 7fd9e6eb8740 1 -- 172.21.15.43:0/4133803934 --> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c1400ebe60 con 0x55c140208080 2024-09-19T05:10:58.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.253+0000 7fd9e1a00640 1 --2- 172.21.15.43:0/4133803934 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55c140208080 0x55c14020a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:10:58.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.253+0000 7fd9e1a00640 1 --2- 172.21.15.43:0/4133803934 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55c140208080 0x55c14020a4a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=62963dd261120212 server_cookie=b005f416eda31565 in_seq=0 out_seq=0 2024-09-19T05:10:58.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.253+0000 7fd9c3400640 10 client.26585 ms_handle_connect on v2:172.21.15.151:6836/3960378010 2024-09-19T05:10:58.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.254+0000 7fd9c3400640 1 -- 172.21.15.43:0/4133803934 <== mds.4 v2:172.21.15.151:6836/3960378010 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c1400ebe60 con 0x55c140208080 2024-09-19T05:10:58.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.254+0000 7fd9c3400640 10 client.26585 handle_command_reply: tid=0 2024-09-19T05:10:58.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.254+0000 7fd9e6eb8740 1 client.26585 shutdown 2024-09-19T05:10:58.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.254+0000 7fd9e6eb8740 1 -- 172.21.15.43:0/4133803934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14017e760 msgr2=0x55c140194980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:58.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.254+0000 7fd9e6eb8740 1 --2- 172.21.15.43:0/4133803934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14017e760 0x55c140194980 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8009700 tx=0x7fd9c8005760 comp rx=0 tx=0).stop 2024-09-19T05:10:58.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.254+0000 7fd9e6eb8740 1 -- 172.21.15.43:0/4133803934 shutdown_connections 2024-09-19T05:10:58.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.254+0000 7fd9e6eb8740 1 --2- 172.21.15.43:0/4133803934 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55c140208080 0x55c14020a4a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:58.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.254+0000 7fd9e6eb8740 1 --2- 172.21.15.43:0/4133803934 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c140189020 0x55c140194ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:58.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.254+0000 7fd9e6eb8740 1 --2- 172.21.15.43:0/4133803934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c14017e760 0x55c140194980 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:58.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.254+0000 7fd9e6eb8740 1 --2- 172.21.15.43:0/4133803934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c14017de40 0x55c1401995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:58.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.254+0000 7fd9c3400640 0 client.26585 ms_handle_reset on v2:172.21.15.151:6836/3960378010 2024-09-19T05:10:58.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.254+0000 7fd9e6eb8740 1 -- 172.21.15.43:0/4133803934 >> 172.21.15.43:0/4133803934 conn(0x55c14017a3d0 msgr2=0x55c140178c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:58.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.254+0000 7fd9e6eb8740 1 -- 172.21.15.43:0/4133803934 shutdown_connections 2024-09-19T05:10:58.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.254+0000 7fd9e6eb8740 1 -- 172.21.15.43:0/4133803934 wait complete. 2024-09-19T05:10:58.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.254+0000 7fd9e6eb8740 20 client.26585 trim_cache size 0 max 16384 2024-09-19T05:10:58.278 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:10:58.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.255+0000 7fd9e2e00640 1 -- 172.21.15.43:0/3292920357 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fd9c40a33b0 msgr2=0x7fd9c40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:58.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.255+0000 7fd9e2e00640 1 --2- 172.21.15.43:0/3292920357 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fd9c40a33b0 0x7fd9c40a5870 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4005fd0 tx=0x7fd9d4009ad0 comp rx=0 tx=0).stop 2024-09-19T05:10:58.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.255+0000 7fd9e2e00640 1 -- 172.21.15.43:0/3292920357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9dc102b90 msgr2=0x7fd9dc1982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:10:58.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.255+0000 7fd9e2e00640 1 --2- 172.21.15.43:0/3292920357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9dc102b90 0x7fd9dc1982d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0004770 tx=0x7fd9d0003c10 comp rx=0 tx=0).stop 2024-09-19T05:10:58.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.255+0000 7fd9e2e00640 1 -- 172.21.15.43:0/3292920357 shutdown_connections 2024-09-19T05:10:58.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.255+0000 7fd9e2e00640 1 --2- 172.21.15.43:0/3292920357 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fd9c40a33b0 0x7fd9c40a5870 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:58.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.255+0000 7fd9e2e00640 1 --2- 172.21.15.43:0/3292920357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9dc10bfc0 0x7fd9dc19cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:58.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.255+0000 7fd9e2e00640 1 --2- 172.21.15.43:0/3292920357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9dc1034b0 0x7fd9dc198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:58.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.255+0000 7fd9e2e00640 1 --2- 172.21.15.43:0/3292920357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9dc102b90 0x7fd9dc1982d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:10:58.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.255+0000 7fd9e2e00640 1 -- 172.21.15.43:0/3292920357 >> 172.21.15.43:0/3292920357 conn(0x7fd9dc0fc610 msgr2=0x7fd9dc0fd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:10:58.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.255+0000 7fd9e2e00640 1 -- 172.21.15.43:0/3292920357 shutdown_connections 2024-09-19T05:10:58.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:10:58.255+0000 7fd9e2e00640 1 -- 172.21.15.43:0/3292920357 wait complete. 2024-09-19T05:10:59.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:58 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:10:59.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:58 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:10:59.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:58 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:10:59.078 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T05:10:59.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:58 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:10:59.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:58 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:10:59.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:58 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:10:59.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:58 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:10:59.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:58 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:10:59.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:58 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:10:59.459 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:11:00.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.067+0000 7f1c27800640 1 -- 172.21.15.43:0/2938690763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c20109140 msgr2=0x7f1c20110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:00.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.067+0000 7f1c27800640 1 --2- 172.21.15.43:0/2938690763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c20109140 0x7f1c20110cc0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1c18009ca0 tx=0x7f1c1802a8c0 comp rx=0 tx=0).stop 2024-09-19T05:11:00.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.067+0000 7f1c27800640 1 -- 172.21.15.43:0/2938690763 shutdown_connections 2024-09-19T05:11:00.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.067+0000 7f1c27800640 1 --2- 172.21.15.43:0/2938690763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c20109140 0x7f1c20110cc0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:00.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.067+0000 7f1c27800640 1 --2- 172.21.15.43:0/2938690763 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c201049c0 0x7f1c20108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:00.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.067+0000 7f1c27800640 1 --2- 172.21.15.43:0/2938690763 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c20104010 0x7f1c201043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:00.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.067+0000 7f1c27800640 1 -- 172.21.15.43:0/2938690763 >> 172.21.15.43:0/2938690763 conn(0x7f1c200ffa50 msgr2=0x7f1c20101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:00.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.067+0000 7f1c27800640 1 -- 172.21.15.43:0/2938690763 shutdown_connections 2024-09-19T05:11:00.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.067+0000 7f1c27800640 1 -- 172.21.15.43:0/2938690763 wait complete. 2024-09-19T05:11:00.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.068+0000 7f1c27800640 1 Processor -- start 2024-09-19T05:11:00.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.068+0000 7f1c27800640 1 -- start start 2024-09-19T05:11:00.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.068+0000 7f1c27800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c20104010 0x7f1c2019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:00.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.068+0000 7f1c27800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c201049c0 0x7f1c2019ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:00.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.068+0000 7f1c27800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c20109140 0x7f1c2019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:00.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.068+0000 7f1c27800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1c201141c0 con 0x7f1c201049c0 2024-09-19T05:11:00.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.068+0000 7f1c27800640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1c20114040 con 0x7f1c20109140 2024-09-19T05:11:00.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.068+0000 7f1c27800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1c20114340 con 0x7f1c20104010 2024-09-19T05:11:00.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.069+0000 7f1c17e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c201049c0 0x7f1c2019ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:00.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.069+0000 7f1c25800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c20109140 0x7f1c2019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:00.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.069+0000 7f1c25800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c20109140 0x7f1c2019ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:39530/0 (socket says 172.21.15.43:39530) 2024-09-19T05:11:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.069+0000 7f1c24e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c20104010 0x7f1c2019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.069+0000 7f1c25800640 1 -- 172.21.15.43:0/2188206100 learned_addr learned my addr 172.21.15.43:0/2188206100 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.069+0000 7f1c24e00640 1 -- 172.21.15.43:0/2188206100 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c20109140 msgr2=0x7f1c2019ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.069+0000 7f1c24e00640 1 --2- 172.21.15.43:0/2188206100 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c20109140 0x7f1c2019ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.069+0000 7f1c24e00640 1 -- 172.21.15.43:0/2188206100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c201049c0 msgr2=0x7f1c2019ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.069+0000 7f1c24e00640 1 --2- 172.21.15.43:0/2188206100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c201049c0 0x7f1c2019ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.069+0000 7f1c24e00640 1 -- 172.21.15.43:0/2188206100 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c2019f670 con 0x7f1c20104010 2024-09-19T05:11:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.069+0000 7f1c17e00640 1 --2- 172.21.15.43:0/2188206100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c201049c0 0x7f1c2019ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:11:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.070+0000 7f1c24e00640 1 --2- 172.21.15.43:0/2188206100 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c20104010 0x7f1c2019a6c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1c1000e9f0 tx=0x7f1c1000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.070+0000 7f1c1e000640 1 -- 172.21.15.43:0/2188206100 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1000ccc0 con 0x7f1c20104010 2024-09-19T05:11:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.070+0000 7f1c27800640 1 -- 172.21.15.43:0/2188206100 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c2019f960 con 0x7f1c20104010 2024-09-19T05:11:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.070+0000 7f1c1e000640 1 -- 172.21.15.43:0/2188206100 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c10010910 con 0x7f1c20104010 2024-09-19T05:11:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.070+0000 7f1c1e000640 1 -- 172.21.15.43:0/2188206100 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c10018960 con 0x7f1c20104010 2024-09-19T05:11:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.070+0000 7f1c27800640 1 -- 172.21.15.43:0/2188206100 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c201a7240 con 0x7f1c20104010 2024-09-19T05:11:00.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.073+0000 7f1c2a3fb740 1 Processor -- start 2024-09-19T05:11:00.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.073+0000 7f1c1e000640 1 -- 172.21.15.43:0/2188206100 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f1c10010ac0 con 0x7f1c20104010 2024-09-19T05:11:00.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.073+0000 7f1c2a3fb740 1 -- start start 2024-09-19T05:11:00.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.074+0000 7f1c2a3fb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651a6495c70 0x5651a6496050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:00.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.074+0000 7f1c2a3fb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a6496590 0x5651a64a0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:00.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.074+0000 7f1c2a3fb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a64a0e50 0x5651a64a3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:00.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.074+0000 7f1c2a3fb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5651a63fc3e0 con 0x5651a64a0e50 2024-09-19T05:11:00.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.074+0000 7f1c2a3fb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5651a63a3d80 con 0x5651a6495c70 2024-09-19T05:11:00.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.074+0000 7f1c2a3fb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5651a63f83a0 con 0x5651a6496590 2024-09-19T05:11:00.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.074+0000 7f1c25800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651a6495c70 0x5651a6496050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:00.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.074+0000 7f1c17e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a64a0e50 0x5651a64a3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:00.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.074+0000 7f1c25800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651a6495c70 0x5651a6496050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:39536/0 (socket says 172.21.15.43:39536) 2024-09-19T05:11:00.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.074+0000 7f1c25800640 1 -- 172.21.15.43:0/3122257085 learned_addr learned my addr 172.21.15.43:0/3122257085 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:00.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.074+0000 7f1c24e00640 1 --2- 172.21.15.43:0/3122257085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a6496590 0x5651a64a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:00.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.074+0000 7f1c25800640 1 -- 172.21.15.43:0/3122257085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a6496590 msgr2=0x5651a64a0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:00.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.074+0000 7f1c25800640 1 --2- 172.21.15.43:0/3122257085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a6496590 0x5651a64a0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:00.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.074+0000 7f1c25800640 1 -- 172.21.15.43:0/3122257085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a64a0e50 msgr2=0x5651a64a3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:00.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.074+0000 7f1c25800640 1 --2- 172.21.15.43:0/3122257085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a64a0e50 0x5651a64a3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:00.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.074+0000 7f1c25800640 1 -- 172.21.15.43:0/3122257085 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651a63c2ae0 con 0x5651a6495c70 2024-09-19T05:11:00.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.075+0000 7f1c25800640 1 --2- 172.21.15.43:0/3122257085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651a6495c70 0x5651a6496050 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1c18005fb0 tx=0x7f1c18009a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e27a86b02a5d083 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:00.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.075+0000 7f1c1e000640 1 --2- 172.21.15.43:0/2188206100 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f1c040a3430 0x7f1c040a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:00.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.075+0000 7f1c16000640 1 -- 172.21.15.43:0/3122257085 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c18031db0 con 0x5651a6495c70 2024-09-19T05:11:00.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.075+0000 7f1c1e000640 1 -- 172.21.15.43:0/2188206100 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f1c100d0060 con 0x7f1c20104010 2024-09-19T05:11:00.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.075+0000 7f1c24e00640 1 --2- 172.21.15.43:0/2188206100 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f1c040a3430 0x7f1c040a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:00.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.075+0000 7f1c16000640 1 -- 172.21.15.43:0/3122257085 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c18036030 con 0x5651a6495c70 2024-09-19T05:11:00.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.075+0000 7f1c16000640 1 -- 172.21.15.43:0/3122257085 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c18004260 con 0x5651a6495c70 2024-09-19T05:11:00.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.076+0000 7f1c2a3fb740 1 -- 172.21.15.43:0/3122257085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651a6495c70 msgr2=0x5651a6496050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:00.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.076+0000 7f1c2a3fb740 1 --2- 172.21.15.43:0/3122257085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651a6495c70 0x5651a6496050 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1c18005fb0 tx=0x7f1c18009a10 comp rx=0 tx=0).stop 2024-09-19T05:11:00.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.076+0000 7f1c24e00640 1 --2- 172.21.15.43:0/2188206100 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f1c040a3430 0x7f1c040a58f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x5651a6493060 tx=0x7f1c100d3b10 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:00.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.076+0000 7f1c2a3fb740 1 -- 172.21.15.43:0/3122257085 shutdown_connections 2024-09-19T05:11:00.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.076+0000 7f1c2a3fb740 1 --2- 172.21.15.43:0/3122257085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a64a0e50 0x5651a64a3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:00.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.076+0000 7f1c2a3fb740 1 --2- 172.21.15.43:0/3122257085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a6496590 0x5651a64a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:00.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.076+0000 7f1c2a3fb740 1 --2- 172.21.15.43:0/3122257085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651a6495c70 0x5651a6496050 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:00.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.076+0000 7f1c2a3fb740 1 -- 172.21.15.43:0/3122257085 >> 172.21.15.43:0/3122257085 conn(0x5651a6492200 msgr2=0x5651a64925e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:00.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.076+0000 7f1c2a3fb740 1 -- 172.21.15.43:0/3122257085 shutdown_connections 2024-09-19T05:11:00.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.076+0000 7f1c2a3fb740 1 -- 172.21.15.43:0/3122257085 wait complete. 2024-09-19T05:11:00.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.077+0000 7f1c2a3fb740 1 -- start start 2024-09-19T05:11:00.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.077+0000 7f1c2a3fb740 1 Processor -- start 2024-09-19T05:11:00.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.077+0000 7f1c2a3fb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a6495c70 0x5651a649dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:00.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.077+0000 7f1c2a3fb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a6496590 0x5651a649e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:00.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.077+0000 7f1c2a3fb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651a64a0e50 0x5651a649e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:00.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.077+0000 7f1c2a3fb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5651a64a38a0 con 0x5651a6496590 2024-09-19T05:11:00.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.077+0000 7f1c2a3fb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5651a63b70f0 con 0x5651a64a0e50 2024-09-19T05:11:00.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.077+0000 7f1c2a3fb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5651a64a3a20 con 0x5651a6495c70 2024-09-19T05:11:00.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.077+0000 7f1c2a3fb740 10 client.0 _ll_register_callbacks cb 0x5651a639ebd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:00.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.078+0000 7f1c25800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a6495c70 0x5651a649dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:00.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.078+0000 7f1c17e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a6496590 0x5651a649e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:00.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.078+0000 7f1c17e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a6496590 0x5651a649e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38304/0 (socket says 172.21.15.43:38304) 2024-09-19T05:11:00.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.078+0000 7f1c17e00640 1 -- 172.21.15.43:0/2953845938 learned_addr learned my addr 172.21.15.43:0/2953845938 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:00.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.078+0000 7f1c25800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a6495c70 0x5651a649dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:59990/0 (socket says 172.21.15.43:59990) 2024-09-19T05:11:00.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.078+0000 7f1c25800640 1 --2- 172.21.15.43:0/2953845938 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651a64a0e50 0x5651a649e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:00.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.078+0000 7f1c17e00640 1 -- 172.21.15.43:0/2953845938 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a6495c70 msgr2=0x5651a649dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:00.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.078+0000 7f1c17e00640 1 --2- 172.21.15.43:0/2953845938 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a6495c70 0x5651a649dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:00.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.078+0000 7f1c17e00640 1 -- 172.21.15.43:0/2953845938 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651a64a0e50 msgr2=0x5651a649e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:00.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.078+0000 7f1c17e00640 1 --2- 172.21.15.43:0/2953845938 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651a64a0e50 0x5651a649e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:00.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.078+0000 7f1c17e00640 1 -- 172.21.15.43:0/2953845938 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651a616d3c0 con 0x5651a6496590 2024-09-19T05:11:00.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.078+0000 7f1c25800640 1 --2- 172.21.15.43:0/2953845938 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a6495c70 0x5651a649dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:00.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.078+0000 7f1c17e00640 1 --2- 172.21.15.43:0/2953845938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a6496590 0x5651a649e210 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1c08012d50 tx=0x7f1c0800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e490851409425c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:00.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.078+0000 7f1c0ea00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:11:00.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.079+0000 7f1c0ea00640 1 -- 172.21.15.43:0/2953845938 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c080120d0 con 0x5651a6496590 2024-09-19T05:11:00.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.079+0000 7f1c0ea00640 1 -- 172.21.15.43:0/2953845938 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f1c08010430 con 0x5651a6496590 2024-09-19T05:11:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.079+0000 7f1c2a3fb740 1 -- 172.21.15.43:0/2953845938 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c18036030 con 0x5651a6496590 2024-09-19T05:11:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.079+0000 7f1c0ea00640 1 -- 172.21.15.43:0/2953845938 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1c08012270 con 0x5651a6496590 2024-09-19T05:11:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.079+0000 7f1c2a3fb740 10 client.16569 fetch_fsmap learned FSMap version 29 2024-09-19T05:11:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.079+0000 7f1c2a3fb740 1 -- 172.21.15.43:0/2953845938 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5651a64af8d0 con 0x5651a6496590 2024-09-19T05:11:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.079+0000 7f1c0ea00640 1 -- 172.21.15.43:0/2953845938 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c18036030 con 0x5651a6496590 2024-09-19T05:11:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.080+0000 7f1c0ea00640 1 -- 172.21.15.43:0/2953845938 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1c0801c5d0 con 0x5651a6496590 2024-09-19T05:11:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.080+0000 7f1c2a3fb740 10 client.16569 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:11:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.080+0000 7f1c2a3fb740 10 client.16569 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T05:11:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.080+0000 7f1c2a3fb740 20 client.16569 populate_metadata read hostname 'smithi043' 2024-09-19T05:11:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.080+0000 7f1c2a3fb740 1 --2- 172.21.15.43:0/2953845938 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x5651a64b0e30 0x5651a64b3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.080+0000 7f1c2a3fb740 4 client.16569 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:11:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.080+0000 7f1c2a3fb740 1 -- 172.21.15.43:0/2953845938 --> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651a6409950 con 0x5651a64b0e30 2024-09-19T05:11:00.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.080+0000 7f1c25800640 1 --2- 172.21.15.43:0/2953845938 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x5651a64b0e30 0x5651a64b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:00.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.081+0000 7f1c25800640 1 --2- 172.21.15.43:0/2953845938 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x5651a64b0e30 0x5651a64b3270 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=806780d714db9267 server_cookie=4c6881f810b8cea9 in_seq=0 out_seq=0 2024-09-19T05:11:00.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.081+0000 7f1c0ea00640 10 client.16569 ms_handle_connect on v2:172.21.15.96:6836/3978388286 2024-09-19T05:11:00.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.082+0000 7f1c0ea00640 1 -- 172.21.15.43:0/2953845938 <== mds.3 v2:172.21.15.96:6836/3978388286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5651a6409950 con 0x5651a64b0e30 2024-09-19T05:11:00.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.082+0000 7f1c0ea00640 10 client.16569 handle_command_reply: tid=0 2024-09-19T05:11:00.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.082+0000 7f1c2a3fb740 1 client.16569 shutdown 2024-09-19T05:11:00.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.082+0000 7f1c2a3fb740 1 -- 172.21.15.43:0/2953845938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a6496590 msgr2=0x5651a649e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:00.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.082+0000 7f1c2a3fb740 1 --2- 172.21.15.43:0/2953845938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a6496590 0x5651a649e210 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1c08012d50 tx=0x7f1c0800f790 comp rx=0 tx=0).stop 2024-09-19T05:11:00.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.082+0000 7f1c2a3fb740 1 -- 172.21.15.43:0/2953845938 shutdown_connections 2024-09-19T05:11:00.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.082+0000 7f1c2a3fb740 1 --2- 172.21.15.43:0/2953845938 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x5651a64b0e30 0x5651a64b3270 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:00.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.082+0000 7f1c2a3fb740 1 --2- 172.21.15.43:0/2953845938 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651a64a0e50 0x5651a649e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:00.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.082+0000 7f1c2a3fb740 1 --2- 172.21.15.43:0/2953845938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a6496590 0x5651a649e210 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:00.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.082+0000 7f1c2a3fb740 1 --2- 172.21.15.43:0/2953845938 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a6495c70 0x5651a649dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:00.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.082+0000 7f1c2a3fb740 1 -- 172.21.15.43:0/2953845938 >> 172.21.15.43:0/2953845938 conn(0x5651a6492200 msgr2=0x5651a6497ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:00.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.082+0000 7f1c0ea00640 0 client.16569 ms_handle_reset on v2:172.21.15.96:6836/3978388286 2024-09-19T05:11:00.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.082+0000 7f1c2a3fb740 1 -- 172.21.15.43:0/2953845938 shutdown_connections 2024-09-19T05:11:00.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.082+0000 7f1c2a3fb740 1 -- 172.21.15.43:0/2953845938 wait complete. 2024-09-19T05:11:00.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.083+0000 7f1c2a3fb740 20 client.16569 trim_cache size 0 max 16384 2024-09-19T05:11:00.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.096+0000 7f1c2a3fb740 1 Processor -- start 2024-09-19T05:11:00.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.096+0000 7f1c2a3fb740 1 -- start start 2024-09-19T05:11:00.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.097+0000 7f1c2a3fb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a6495c70 0x5651a649fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:00.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.097+0000 7f1c2a3fb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651a6496590 0x5651a64b8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:00.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.097+0000 7f1c2a3fb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a64a0e50 0x5651a64b9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:00.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.097+0000 7f1c2a3fb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1c201afa90 con 0x5651a6495c70 2024-09-19T05:11:00.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.097+0000 7f1c2a3fb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1c201b15d0 con 0x5651a6496590 2024-09-19T05:11:00.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.097+0000 7f1c2a3fb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1c201b0160 con 0x5651a64a0e50 2024-09-19T05:11:00.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.097+0000 7f1c25800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a6495c70 0x5651a649fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:00.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.097+0000 7f1c25800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a6495c70 0x5651a649fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38308/0 (socket says 172.21.15.43:38308) 2024-09-19T05:11:00.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.097+0000 7f1c25800640 1 -- 172.21.15.43:0/1140562877 learned_addr learned my addr 172.21.15.43:0/1140562877 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:00.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.097+0000 7f1c17e00640 1 --2- 172.21.15.43:0/1140562877 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651a6496590 0x5651a64b8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:00.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.097+0000 7f1c25800640 1 -- 172.21.15.43:0/1140562877 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a64a0e50 msgr2=0x5651a64b9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:00.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.097+0000 7f1c25800640 1 --2- 172.21.15.43:0/1140562877 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a64a0e50 0x5651a64b9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:00.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.097+0000 7f1c25800640 1 -- 172.21.15.43:0/1140562877 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651a6496590 msgr2=0x5651a64b8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:00.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.097+0000 7f1c25800640 1 --2- 172.21.15.43:0/1140562877 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651a6496590 0x5651a64b8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:00.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.097+0000 7f1c25800640 1 -- 172.21.15.43:0/1140562877 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651a6492030 con 0x5651a6495c70 2024-09-19T05:11:00.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.098+0000 7f1c25800640 1 --2- 172.21.15.43:0/1140562877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a6495c70 0x5651a649fc40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1c18005fb0 tx=0x7f1c18031940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbd3402f64ecb0ad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:00.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.098+0000 7f1c15600640 1 -- 172.21.15.43:0/1140562877 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c18031550 con 0x5651a6495c70 2024-09-19T05:11:00.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.098+0000 7f1c15600640 1 -- 172.21.15.43:0/1140562877 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c18057070 con 0x5651a6495c70 2024-09-19T05:11:00.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.098+0000 7f1c15600640 1 -- 172.21.15.43:0/1140562877 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c180042a0 con 0x5651a6495c70 2024-09-19T05:11:00.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.098+0000 7f1c2a3fb740 1 -- 172.21.15.43:0/1140562877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a6495c70 msgr2=0x5651a649fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:00.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.098+0000 7f1c2a3fb740 1 --2- 172.21.15.43:0/1140562877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a6495c70 0x5651a649fc40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1c18005fb0 tx=0x7f1c18031940 comp rx=0 tx=0).stop 2024-09-19T05:11:00.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.099+0000 7f1c2a3fb740 1 -- 172.21.15.43:0/1140562877 shutdown_connections 2024-09-19T05:11:00.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.099+0000 7f1c2a3fb740 1 --2- 172.21.15.43:0/1140562877 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a64a0e50 0x5651a64b9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:00.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.099+0000 7f1c2a3fb740 1 --2- 172.21.15.43:0/1140562877 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651a6496590 0x5651a64b8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:00.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.099+0000 7f1c2a3fb740 1 --2- 172.21.15.43:0/1140562877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a6495c70 0x5651a649fc40 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:00.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.099+0000 7f1c2a3fb740 1 -- 172.21.15.43:0/1140562877 >> 172.21.15.43:0/1140562877 conn(0x5651a6492200 msgr2=0x5651a649f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:00.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.099+0000 7f1c2a3fb740 1 -- 172.21.15.43:0/1140562877 shutdown_connections 2024-09-19T05:11:00.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.099+0000 7f1c2a3fb740 1 -- 172.21.15.43:0/1140562877 wait complete. 2024-09-19T05:11:00.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.099+0000 7f1c2a3fb740 1 -- start start 2024-09-19T05:11:00.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.099+0000 7f1c2a3fb740 1 Processor -- start 2024-09-19T05:11:00.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.100+0000 7f1c2a3fb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a6495c70 0x5651a649ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:00.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.100+0000 7f1c2a3fb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a6496590 0x5651a649b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:00.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.100+0000 7f1c2a3fb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651a64a0e50 0x5651a649b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:00.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.100+0000 7f1c2a3fb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1c201b0f00 con 0x5651a6495c70 2024-09-19T05:11:00.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.100+0000 7f1c2a3fb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1c201b0830 con 0x5651a64a0e50 2024-09-19T05:11:00.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.100+0000 7f1c2a3fb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1c201ada70 con 0x5651a6496590 2024-09-19T05:11:00.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.100+0000 7f1c2a3fb740 10 client.0 _ll_register_callbacks cb 0x7f1c08009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:00.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.100+0000 7f1c25800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a6495c70 0x5651a649ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.100+0000 7f1c25800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a6495c70 0x5651a649ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38312/0 (socket says 172.21.15.43:38312) 2024-09-19T05:11:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.100+0000 7f1c25800640 1 -- 172.21.15.43:0/318239207 learned_addr learned my addr 172.21.15.43:0/318239207 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.100+0000 7f1c17e00640 1 --2- 172.21.15.43:0/318239207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a6496590 0x5651a649b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.100+0000 7f1c25800640 1 --2- 172.21.15.43:0/318239207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651a64a0e50 0x5651a649b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.101+0000 7f1c17e00640 1 -- 172.21.15.43:0/318239207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651a64a0e50 msgr2=0x5651a649b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.101+0000 7f1c17e00640 1 --2- 172.21.15.43:0/318239207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651a64a0e50 0x5651a649b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.101+0000 7f1c17e00640 1 -- 172.21.15.43:0/318239207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a6495c70 msgr2=0x5651a649ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.101+0000 7f1c17e00640 1 --2- 172.21.15.43:0/318239207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a6495c70 0x5651a649ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.101+0000 7f1c17e00640 1 -- 172.21.15.43:0/318239207 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651a648d770 con 0x5651a6496590 2024-09-19T05:11:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.101+0000 7f1c17e00640 1 --2- 172.21.15.43:0/318239207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a6496590 0x5651a649b220 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1c0801cac0 tx=0x7f1c08012a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2feff17123e95222 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.101+0000 7f1c0ea00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:11:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.101+0000 7f1c0ea00640 1 -- 172.21.15.43:0/318239207 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c08005d30 con 0x5651a6496590 2024-09-19T05:11:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.101+0000 7f1c0ea00640 1 -- 172.21.15.43:0/318239207 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f1c0801c630 con 0x5651a6496590 2024-09-19T05:11:00.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.101+0000 7f1c2a3fb740 1 -- 172.21.15.43:0/318239207 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c18057070 con 0x5651a6496590 2024-09-19T05:11:00.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.102+0000 7f1c0ea00640 1 -- 172.21.15.43:0/318239207 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1c08033050 con 0x5651a6496590 2024-09-19T05:11:00.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.102+0000 7f1c2a3fb740 10 client.26539 fetch_fsmap learned FSMap version 29 2024-09-19T05:11:00.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.102+0000 7f1c2a3fb740 1 -- 172.21.15.43:0/318239207 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5651a651bc00 con 0x5651a6496590 2024-09-19T05:11:00.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.102+0000 7f1c0ea00640 1 -- 172.21.15.43:0/318239207 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c18057070 con 0x5651a6496590 2024-09-19T05:11:00.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.102+0000 7f1c0ea00640 1 -- 172.21.15.43:0/318239207 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1c0802b020 con 0x5651a6496590 2024-09-19T05:11:00.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.102+0000 7f1c2a3fb740 10 client.26539 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:11:00.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.102+0000 7f1c2a3fb740 10 client.26539 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T05:11:00.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.102+0000 7f1c2a3fb740 20 client.26539 populate_metadata read hostname 'smithi043' 2024-09-19T05:11:00.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.102+0000 7f1c2a3fb740 1 --2- 172.21.15.43:0/318239207 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x5651a651c3d0 0x5651a651e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:00.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.102+0000 7f1c2a3fb740 4 client.26539 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:11:00.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.102+0000 7f1c2a3fb740 1 -- 172.21.15.43:0/318239207 --> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5651a6403e60 con 0x5651a651c3d0 2024-09-19T05:11:00.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.103+0000 7f1c25800640 1 --2- 172.21.15.43:0/318239207 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x5651a651c3d0 0x5651a651e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:00.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.103+0000 7f1c25800640 1 --2- 172.21.15.43:0/318239207 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x5651a651c3d0 0x5651a651e7f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dc72e80485f4d16a server_cookie=df5c2af6baa44600 in_seq=0 out_seq=0 2024-09-19T05:11:00.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.103+0000 7f1c0ea00640 10 client.26539 ms_handle_connect on v2:172.21.15.96:6836/3978388286 2024-09-19T05:11:00.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.103+0000 7f1c0ea00640 1 -- 172.21.15.43:0/318239207 <== mds.3 v2:172.21.15.96:6836/3978388286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5651a6403e60 con 0x5651a651c3d0 2024-09-19T05:11:00.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.103+0000 7f1c0ea00640 10 client.26539 handle_command_reply: tid=0 2024-09-19T05:11:00.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.103+0000 7f1c2a3fb740 1 client.26539 shutdown 2024-09-19T05:11:00.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.103+0000 7f1c2a3fb740 1 -- 172.21.15.43:0/318239207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a6496590 msgr2=0x5651a649b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:00.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.103+0000 7f1c2a3fb740 1 --2- 172.21.15.43:0/318239207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a6496590 0x5651a649b220 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1c0801cac0 tx=0x7f1c08012a00 comp rx=0 tx=0).stop 2024-09-19T05:11:00.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.104+0000 7f1c2a3fb740 1 -- 172.21.15.43:0/318239207 shutdown_connections 2024-09-19T05:11:00.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.104+0000 7f1c2a3fb740 1 --2- 172.21.15.43:0/318239207 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x5651a651c3d0 0x5651a651e7f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:00.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.104+0000 7f1c2a3fb740 1 --2- 172.21.15.43:0/318239207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5651a64a0e50 0x5651a649b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:00.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.104+0000 7f1c2a3fb740 1 --2- 172.21.15.43:0/318239207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a6496590 0x5651a649b220 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:00.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.104+0000 7f1c0ea00640 0 client.26539 ms_handle_reset on v2:172.21.15.96:6836/3978388286 2024-09-19T05:11:00.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.104+0000 7f1c2a3fb740 1 --2- 172.21.15.43:0/318239207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a6495c70 0x5651a649ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:00.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.104+0000 7f1c2a3fb740 1 -- 172.21.15.43:0/318239207 >> 172.21.15.43:0/318239207 conn(0x5651a6492200 msgr2=0x5651a6491380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:00.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.104+0000 7f1c2a3fb740 1 -- 172.21.15.43:0/318239207 shutdown_connections 2024-09-19T05:11:00.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.104+0000 7f1c2a3fb740 1 -- 172.21.15.43:0/318239207 wait complete. 2024-09-19T05:11:00.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.104+0000 7f1c2a3fb740 20 client.26539 trim_cache size 0 max 16384 2024-09-19T05:11:00.110 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:11:00.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.105+0000 7f1c27800640 1 -- 172.21.15.43:0/2188206100 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f1c040a3430 msgr2=0x7f1c040a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:00.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.105+0000 7f1c27800640 1 --2- 172.21.15.43:0/2188206100 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f1c040a3430 0x7f1c040a58f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x5651a6493060 tx=0x7f1c100d3b10 comp rx=0 tx=0).stop 2024-09-19T05:11:00.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.105+0000 7f1c27800640 1 -- 172.21.15.43:0/2188206100 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c20104010 msgr2=0x7f1c2019a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:00.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.105+0000 7f1c27800640 1 --2- 172.21.15.43:0/2188206100 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c20104010 0x7f1c2019a6c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1c1000e9f0 tx=0x7f1c1000eeb0 comp rx=0 tx=0).stop 2024-09-19T05:11:00.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.105+0000 7f1c27800640 1 -- 172.21.15.43:0/2188206100 shutdown_connections 2024-09-19T05:11:00.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.105+0000 7f1c27800640 1 --2- 172.21.15.43:0/2188206100 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f1c040a3430 0x7f1c040a58f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:00.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.105+0000 7f1c27800640 1 --2- 172.21.15.43:0/2188206100 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c20109140 0x7f1c2019ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:00.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.105+0000 7f1c27800640 1 --2- 172.21.15.43:0/2188206100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c201049c0 0x7f1c2019ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:00.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.105+0000 7f1c27800640 1 --2- 172.21.15.43:0/2188206100 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c20104010 0x7f1c2019a6c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:00.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.105+0000 7f1c27800640 1 -- 172.21.15.43:0/2188206100 >> 172.21.15.43:0/2188206100 conn(0x7f1c200ffa50 msgr2=0x7f1c20100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:00.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.105+0000 7f1c27800640 1 -- 172.21.15.43:0/2188206100 shutdown_connections 2024-09-19T05:11:00.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:00.105+0000 7f1c27800640 1 -- 172.21.15.43:0/2188206100 wait complete. 2024-09-19T05:11:00.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:10:59 smithi151 ceph-mon[35425]: pgmap v193: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:11:00.156 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:10:59 smithi043 ceph-mon[30800]: pgmap v193: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:11:00.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:10:59 smithi096 ceph-mon[35085]: pgmap v193: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:11:01.047 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T05:11:01.426 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:11:01.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:01.993+0000 7f7133000640 1 -- 172.21.15.43:0/701590347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f712c074ee0 msgr2=0x7f712c10dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:01.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:01.993+0000 7f7133000640 1 --2- 172.21.15.43:0/701590347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f712c074ee0 0x7f712c10dd20 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f7118009900 tx=0x7f711802a480 comp rx=0 tx=0).stop 2024-09-19T05:11:01.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:01.993+0000 7f7133000640 1 -- 172.21.15.43:0/701590347 shutdown_connections 2024-09-19T05:11:01.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:01.993+0000 7f7133000640 1 --2- 172.21.15.43:0/701590347 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f712c10e260 0x7f712c110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:01.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:01.993+0000 7f7133000640 1 --2- 172.21.15.43:0/701590347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f712c074ee0 0x7f712c10dd20 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:01.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:01.993+0000 7f7133000640 1 --2- 172.21.15.43:0/701590347 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f712c0745c0 0x7f712c0749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:01.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:01.993+0000 7f7133000640 1 -- 172.21.15.43:0/701590347 >> 172.21.15.43:0/701590347 conn(0x7f712c0fd740 msgr2=0x7f712c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:01.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:01.993+0000 7f7133000640 1 -- 172.21.15.43:0/701590347 shutdown_connections 2024-09-19T05:11:01.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:01.993+0000 7f7133000640 1 -- 172.21.15.43:0/701590347 wait complete. 2024-09-19T05:11:01.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:01.994+0000 7f7133000640 1 Processor -- start 2024-09-19T05:11:01.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:01.994+0000 7f7133000640 1 -- start start 2024-09-19T05:11:01.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:01.995+0000 7f7133000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f712c0745c0 0x7f712c19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:01.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:01.995+0000 7f7133000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f712c074ee0 0x7f712c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:01.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:01.995+0000 7f7133000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f712c10e260 0x7f712c19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:01.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:01.995+0000 7f7133000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f712c114000 con 0x7f712c074ee0 2024-09-19T05:11:01.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:01.995+0000 7f7133000640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f712c113e80 con 0x7f712c10e260 2024-09-19T05:11:01.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:01.995+0000 7f7133000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f712c114180 con 0x7f712c0745c0 2024-09-19T05:11:01.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:01.995+0000 7f7131200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f712c074ee0 0x7f712c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:01.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:01.995+0000 7f7132600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f712c10e260 0x7f712c19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:01.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:01.995+0000 7f7131200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f712c074ee0 0x7f712c19aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38328/0 (socket says 172.21.15.43:38328) 2024-09-19T05:11:01.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:01.995+0000 7f7131200640 1 -- 172.21.15.43:0/1321530636 learned_addr learned my addr 172.21.15.43:0/1321530636 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:01.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:01.995+0000 7f7132600640 1 -- 172.21.15.43:0/1321530636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f712c0745c0 msgr2=0x7f712c19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:01.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:01.995+0000 7f7132600640 1 --2- 172.21.15.43:0/1321530636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f712c0745c0 0x7f712c19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:02.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:01.995+0000 7f7132600640 1 -- 172.21.15.43:0/1321530636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f712c074ee0 msgr2=0x7f712c19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:02.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:01.995+0000 7f7132600640 1 --2- 172.21.15.43:0/1321530636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f712c074ee0 0x7f712c19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:02.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:01.995+0000 7f7132600640 1 -- 172.21.15.43:0/1321530636 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f712c19f470 con 0x7f712c10e260 2024-09-19T05:11:02.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:01.996+0000 7f7132600640 1 --2- 172.21.15.43:0/1321530636 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f712c10e260 0x7f712c19ed90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7124005fd0 tx=0x7f712400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:02.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:01.996+0000 7f712a000640 1 -- 172.21.15.43:0/1321530636 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712400cdf0 con 0x7f712c10e260 2024-09-19T05:11:02.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:01.996+0000 7f712a000640 1 -- 172.21.15.43:0/1321530636 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7124005020 con 0x7f712c10e260 2024-09-19T05:11:02.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:01.996+0000 7f712a000640 1 -- 172.21.15.43:0/1321530636 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71240106d0 con 0x7f712c10e260 2024-09-19T05:11:02.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:01.996+0000 7f7133000640 1 -- 172.21.15.43:0/1321530636 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f712c19f760 con 0x7f712c10e260 2024-09-19T05:11:02.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:01.996+0000 7f7133000640 1 -- 172.21.15.43:0/1321530636 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f712c1a7040 con 0x7f712c10e260 2024-09-19T05:11:02.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.000+0000 7f7137213740 1 Processor -- start 2024-09-19T05:11:02.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.000+0000 7f712a000640 1 -- 172.21.15.43:0/1321530636 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f71240054d0 con 0x7f712c10e260 2024-09-19T05:11:02.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.001+0000 7f7137213740 1 -- start start 2024-09-19T05:11:02.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.001+0000 7f7137213740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609e603be40 0x5609e603c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:02.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.001+0000 7f7137213740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609e603c760 0x5609e6046ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:02.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.001+0000 7f7137213740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609e6047020 0x5609e6049410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:02.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.001+0000 7f7137213740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5609e5fa23e0 con 0x5609e603c760 2024-09-19T05:11:02.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.001+0000 7f7137213740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5609e5f49d80 con 0x5609e603be40 2024-09-19T05:11:02.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.001+0000 7f7137213740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5609e5f9e3a0 con 0x5609e6047020 2024-09-19T05:11:02.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.001+0000 7f7131200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609e603be40 0x5609e603c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:02.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.001+0000 7f7131c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609e603c760 0x5609e6046ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:02.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.001+0000 7f7131200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609e603be40 0x5609e603c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:39602/0 (socket says 172.21.15.43:39602) 2024-09-19T05:11:02.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.001+0000 7f7131200640 1 -- 172.21.15.43:0/3395342502 learned_addr learned my addr 172.21.15.43:0/3395342502 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:02.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.002+0000 7f7131200640 1 --2- 172.21.15.43:0/3395342502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609e6047020 0x5609e6049410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:02.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.002+0000 7f7131200640 1 -- 172.21.15.43:0/3395342502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609e6047020 msgr2=0x5609e6049410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:02.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.002+0000 7f7131200640 1 --2- 172.21.15.43:0/3395342502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609e6047020 0x5609e6049410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:02.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.002+0000 7f7131200640 1 -- 172.21.15.43:0/3395342502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609e603c760 msgr2=0x5609e6046ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:02.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.002+0000 7f7131200640 1 --2- 172.21.15.43:0/3395342502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609e603c760 0x5609e6046ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:02.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.002+0000 7f7131200640 1 -- 172.21.15.43:0/3395342502 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609e5f68ae0 con 0x5609e603be40 2024-09-19T05:11:02.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.002+0000 7f7131c00640 1 --2- 172.21.15.43:0/3395342502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609e603c760 0x5609e6046ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:02.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.002+0000 7f7131200640 1 --2- 172.21.15.43:0/3395342502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609e603be40 0x5609e603c220 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f711802aa70 tx=0x7f711802a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e38b7a0a25d8cba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:02.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.002+0000 7f712a000640 1 --2- 172.21.15.43:0/1321530636 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f71140a3430 0x7f71140a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:02.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.003+0000 7f711ea00640 1 -- 172.21.15.43:0/3395342502 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7118035030 con 0x5609e603be40 2024-09-19T05:11:02.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.003+0000 7f711ea00640 1 -- 172.21.15.43:0/3395342502 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71180047a0 con 0x5609e603be40 2024-09-19T05:11:02.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.003+0000 7f711ea00640 1 -- 172.21.15.43:0/3395342502 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7118003850 con 0x5609e603be40 2024-09-19T05:11:02.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.003+0000 7f712a000640 1 -- 172.21.15.43:0/1321530636 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f71240d0360 con 0x7f712c10e260 2024-09-19T05:11:02.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.003+0000 7f7131c00640 1 --2- 172.21.15.43:0/1321530636 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f71140a3430 0x7f71140a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:02.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.003+0000 7f7137213740 1 -- 172.21.15.43:0/3395342502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609e603be40 msgr2=0x5609e603c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:02.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.003+0000 7f7137213740 1 --2- 172.21.15.43:0/3395342502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609e603be40 0x5609e603c220 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f711802aa70 tx=0x7f711802a480 comp rx=0 tx=0).stop 2024-09-19T05:11:02.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.004+0000 7f7137213740 1 -- 172.21.15.43:0/3395342502 shutdown_connections 2024-09-19T05:11:02.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.004+0000 7f7137213740 1 --2- 172.21.15.43:0/3395342502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609e6047020 0x5609e6049410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:02.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.004+0000 7f7131c00640 1 --2- 172.21.15.43:0/1321530636 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f71140a3430 0x7f71140a58f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x5609e6039230 tx=0x7f7120009530 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:02.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.004+0000 7f7137213740 1 --2- 172.21.15.43:0/3395342502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609e603c760 0x5609e6046ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:02.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.004+0000 7f7137213740 1 --2- 172.21.15.43:0/3395342502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609e603be40 0x5609e603c220 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:02.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.004+0000 7f7137213740 1 -- 172.21.15.43:0/3395342502 >> 172.21.15.43:0/3395342502 conn(0x5609e60383d0 msgr2=0x5609e60387b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:02.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.004+0000 7f7137213740 1 -- 172.21.15.43:0/3395342502 shutdown_connections 2024-09-19T05:11:02.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.004+0000 7f7137213740 1 -- 172.21.15.43:0/3395342502 wait complete. 2024-09-19T05:11:02.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.004+0000 7f7137213740 1 -- start start 2024-09-19T05:11:02.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.004+0000 7f7137213740 1 Processor -- start 2024-09-19T05:11:02.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.005+0000 7f7137213740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609e603be40 0x5609e6043f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:02.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.005+0000 7f7137213740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609e603c760 0x5609e60444b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:02.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.005+0000 7f7137213740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609e6047020 0x5609e60449f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:02.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.005+0000 7f7137213740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5609e6049bf0 con 0x5609e603c760 2024-09-19T05:11:02.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.005+0000 7f7137213740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5609e5f5d0f0 con 0x5609e6047020 2024-09-19T05:11:02.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.005+0000 7f7137213740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5609e6049d70 con 0x5609e603be40 2024-09-19T05:11:02.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.005+0000 7f7137213740 10 client.0 _ll_register_callbacks cb 0x5609e5f44bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:02.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.005+0000 7f7131200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609e603c760 0x5609e60444b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:02.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.005+0000 7f7131c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609e603be40 0x5609e6043f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:02.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.005+0000 7f7132600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609e6047020 0x5609e60449f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:02.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.005+0000 7f7131200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609e603c760 0x5609e60444b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38350/0 (socket says 172.21.15.43:38350) 2024-09-19T05:11:02.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.005+0000 7f7131200640 1 -- 172.21.15.43:0/1703229782 learned_addr learned my addr 172.21.15.43:0/1703229782 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:02.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.006+0000 7f7132600640 1 -- 172.21.15.43:0/1703229782 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609e603be40 msgr2=0x5609e6043f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:02.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.006+0000 7f7132600640 1 --2- 172.21.15.43:0/1703229782 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609e603be40 0x5609e6043f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:02.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.006+0000 7f7132600640 1 -- 172.21.15.43:0/1703229782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609e603c760 msgr2=0x5609e60444b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:02.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.006+0000 7f7132600640 1 --2- 172.21.15.43:0/1703229782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609e603c760 0x5609e60444b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:02.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.006+0000 7f7132600640 1 -- 172.21.15.43:0/1703229782 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609e5d133c0 con 0x5609e6047020 2024-09-19T05:11:02.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.006+0000 7f7132600640 1 --2- 172.21.15.43:0/1703229782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609e6047020 0x5609e60449f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f71240cda30 tx=0x7f712409cdb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f5276e2c0bbc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:02.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.006+0000 7f7113400640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:11:02.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.006+0000 7f7113400640 1 -- 172.21.15.43:0/1703229782 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71240a5260 con 0x5609e6047020 2024-09-19T05:11:02.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.006+0000 7f7113400640 1 -- 172.21.15.43:0/1703229782 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f712409c850 con 0x5609e6047020 2024-09-19T05:11:02.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.006+0000 7f7137213740 1 -- 172.21.15.43:0/1703229782 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71180047a0 con 0x5609e6047020 2024-09-19T05:11:02.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.006+0000 7f7113400640 1 -- 172.21.15.43:0/1703229782 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f712409ca50 con 0x5609e6047020 2024-09-19T05:11:02.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.006+0000 7f7137213740 10 client.26615 fetch_fsmap learned FSMap version 29 2024-09-19T05:11:02.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.006+0000 7f7137213740 1 -- 172.21.15.43:0/1703229782 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5609e6055cf0 con 0x5609e6047020 2024-09-19T05:11:02.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.007+0000 7f7113400640 1 -- 172.21.15.43:0/1703229782 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71180047a0 con 0x5609e6047020 2024-09-19T05:11:02.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.007+0000 7f7113400640 1 -- 172.21.15.43:0/1703229782 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f712409b4b0 con 0x5609e6047020 2024-09-19T05:11:02.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.007+0000 7f7137213740 10 client.26615 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:11:02.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.007+0000 7f7137213740 10 client.26615 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T05:11:02.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.007+0000 7f7137213740 20 client.26615 populate_metadata read hostname 'smithi043' 2024-09-19T05:11:02.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.007+0000 7f7137213740 1 --2- 172.21.15.43:0/1703229782 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x5609e60571b0 0x5609e60595e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:02.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.007+0000 7f7137213740 4 client.26615 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:11:02.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.007+0000 7f7137213740 1 -- 172.21.15.43:0/1703229782 --> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609e5faf950 con 0x5609e60571b0 2024-09-19T05:11:02.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.007+0000 7f7131c00640 1 --2- 172.21.15.43:0/1703229782 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x5609e60571b0 0x5609e60595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:02.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.008+0000 7f7131c00640 1 --2- 172.21.15.43:0/1703229782 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x5609e60571b0 0x5609e60595e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ad1f01e268f031be server_cookie=9ee171ac4a588fe1 in_seq=0 out_seq=0 2024-09-19T05:11:02.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.008+0000 7f7113400640 10 client.26615 ms_handle_connect on v2:172.21.15.151:6838/3794104313 2024-09-19T05:11:02.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.009+0000 7f7113400640 1 -- 172.21.15.43:0/1703229782 <== mds.2 v2:172.21.15.151:6838/3794104313 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5609e5faf950 con 0x5609e60571b0 2024-09-19T05:11:02.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.009+0000 7f7113400640 10 client.26615 handle_command_reply: tid=0 2024-09-19T05:11:02.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.009+0000 7f7137213740 1 client.26615 shutdown 2024-09-19T05:11:02.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.009+0000 7f7137213740 1 -- 172.21.15.43:0/1703229782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609e6047020 msgr2=0x5609e60449f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:02.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.009+0000 7f7137213740 1 --2- 172.21.15.43:0/1703229782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609e6047020 0x5609e60449f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f71240cda30 tx=0x7f712409cdb0 comp rx=0 tx=0).stop 2024-09-19T05:11:02.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.009+0000 7f7137213740 1 -- 172.21.15.43:0/1703229782 shutdown_connections 2024-09-19T05:11:02.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.009+0000 7f7137213740 1 --2- 172.21.15.43:0/1703229782 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x5609e60571b0 0x5609e60595e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:02.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.009+0000 7f7137213740 1 --2- 172.21.15.43:0/1703229782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609e6047020 0x5609e60449f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:02.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.009+0000 7f7137213740 1 --2- 172.21.15.43:0/1703229782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609e603c760 0x5609e60444b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:02.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.009+0000 7f7137213740 1 --2- 172.21.15.43:0/1703229782 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609e603be40 0x5609e6043f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:02.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.009+0000 7f7113400640 0 client.26615 ms_handle_reset on v2:172.21.15.151:6838/3794104313 2024-09-19T05:11:02.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.009+0000 7f7137213740 1 -- 172.21.15.43:0/1703229782 >> 172.21.15.43:0/1703229782 conn(0x5609e60383d0 msgr2=0x5609e603e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:02.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.009+0000 7f7137213740 1 -- 172.21.15.43:0/1703229782 shutdown_connections 2024-09-19T05:11:02.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.009+0000 7f7137213740 1 -- 172.21.15.43:0/1703229782 wait complete. 2024-09-19T05:11:02.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.009+0000 7f7137213740 20 client.26615 trim_cache size 0 max 16384 2024-09-19T05:11:02.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.023+0000 7f7137213740 1 Processor -- start 2024-09-19T05:11:02.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.023+0000 7f7137213740 1 -- start start 2024-09-19T05:11:02.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.024+0000 7f7137213740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609e603be40 0x5609e603e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:02.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.024+0000 7f7137213740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609e603c760 0x5609e6045ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:02.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.024+0000 7f7137213740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609e6047020 0x5609e605ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:02.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.024+0000 7f7137213740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f712c1af970 con 0x5609e603c760 2024-09-19T05:11:02.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.024+0000 7f7137213740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f712c1b14b0 con 0x5609e603be40 2024-09-19T05:11:02.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.024+0000 7f7137213740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f712c1b0040 con 0x5609e6047020 2024-09-19T05:11:02.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.024+0000 7f7131c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609e603be40 0x5609e603e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:02.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.024+0000 7f7131200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609e603c760 0x5609e6045ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:02.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.024+0000 7f7131c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609e603be40 0x5609e603e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:39620/0 (socket says 172.21.15.43:39620) 2024-09-19T05:11:02.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.024+0000 7f7131c00640 1 -- 172.21.15.43:0/2004612083 learned_addr learned my addr 172.21.15.43:0/2004612083 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:02.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.024+0000 7f7132600640 1 --2- 172.21.15.43:0/2004612083 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609e6047020 0x5609e605ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:02.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.024+0000 7f7131c00640 1 -- 172.21.15.43:0/2004612083 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609e6047020 msgr2=0x5609e605ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:02.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.024+0000 7f7131c00640 1 --2- 172.21.15.43:0/2004612083 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609e6047020 0x5609e605ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:02.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.024+0000 7f7131c00640 1 -- 172.21.15.43:0/2004612083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609e603c760 msgr2=0x5609e6045ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:02.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.024+0000 7f7131c00640 1 --2- 172.21.15.43:0/2004612083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609e603c760 0x5609e6045ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:02.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.025+0000 7f7131c00640 1 -- 172.21.15.43:0/2004612083 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609e603bc70 con 0x5609e603be40 2024-09-19T05:11:02.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.025+0000 7f7131c00640 1 --2- 172.21.15.43:0/2004612083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609e603be40 0x5609e603e8f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x5609e6059ea0 tx=0x7f7120014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f26d2d9bf1e9090 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:02.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.025+0000 7f711e000640 1 -- 172.21.15.43:0/2004612083 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7120004050 con 0x5609e603be40 2024-09-19T05:11:02.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.025+0000 7f711e000640 1 -- 172.21.15.43:0/2004612083 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71200041f0 con 0x5609e603be40 2024-09-19T05:11:02.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.025+0000 7f711e000640 1 -- 172.21.15.43:0/2004612083 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7120028510 con 0x5609e603be40 2024-09-19T05:11:02.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.026+0000 7f7137213740 1 -- 172.21.15.43:0/2004612083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609e603be40 msgr2=0x5609e603e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:02.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.026+0000 7f7137213740 1 --2- 172.21.15.43:0/2004612083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609e603be40 0x5609e603e8f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x5609e6059ea0 tx=0x7f7120014000 comp rx=0 tx=0).stop 2024-09-19T05:11:02.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.026+0000 7f7137213740 1 -- 172.21.15.43:0/2004612083 shutdown_connections 2024-09-19T05:11:02.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.026+0000 7f7137213740 1 --2- 172.21.15.43:0/2004612083 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609e6047020 0x5609e605ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:02.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.026+0000 7f7137213740 1 --2- 172.21.15.43:0/2004612083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609e603c760 0x5609e6045ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:02.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.026+0000 7f7137213740 1 --2- 172.21.15.43:0/2004612083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609e603be40 0x5609e603e8f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:02.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.026+0000 7f7137213740 1 -- 172.21.15.43:0/2004612083 >> 172.21.15.43:0/2004612083 conn(0x5609e60383d0 msgr2=0x5609e60457d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:02.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.026+0000 7f7137213740 1 -- 172.21.15.43:0/2004612083 shutdown_connections 2024-09-19T05:11:02.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.026+0000 7f7137213740 1 -- 172.21.15.43:0/2004612083 wait complete. 2024-09-19T05:11:02.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.026+0000 7f7137213740 1 -- start start 2024-09-19T05:11:02.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.027+0000 7f7137213740 1 Processor -- start 2024-09-19T05:11:02.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.027+0000 7f7137213740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609e603be40 0x5609e604f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:02.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.027+0000 7f7137213740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609e603c760 0x5609e604aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:02.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.027+0000 7f7137213740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609e6047020 0x5609e604af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:02.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.027+0000 7f7137213740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f712c1b0de0 con 0x5609e603be40 2024-09-19T05:11:02.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.027+0000 7f7137213740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f712c1b0710 con 0x5609e603c760 2024-09-19T05:11:02.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.027+0000 7f7137213740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f712c1ad950 con 0x5609e6047020 2024-09-19T05:11:02.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.027+0000 7f7137213740 10 client.0 _ll_register_callbacks cb 0x7f712409c1a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:02.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.028+0000 7f7131200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609e603c760 0x5609e604aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:02.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.028+0000 7f7131200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609e603c760 0x5609e604aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:39636/0 (socket says 172.21.15.43:39636) 2024-09-19T05:11:02.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.028+0000 7f7131200640 1 -- 172.21.15.43:0/1158810415 learned_addr learned my addr 172.21.15.43:0/1158810415 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:02.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.028+0000 7f7131c00640 1 --2- 172.21.15.43:0/1158810415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609e603be40 0x5609e604f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:02.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.028+0000 7f7132600640 1 --2- 172.21.15.43:0/1158810415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609e6047020 0x5609e604af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:02.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.028+0000 7f7131200640 1 -- 172.21.15.43:0/1158810415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609e6047020 msgr2=0x5609e604af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:02.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.028+0000 7f7131200640 1 --2- 172.21.15.43:0/1158810415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609e6047020 0x5609e604af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:02.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.028+0000 7f7131200640 1 -- 172.21.15.43:0/1158810415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609e603be40 msgr2=0x5609e604f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:02.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.028+0000 7f7131200640 1 --2- 172.21.15.43:0/1158810415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609e603be40 0x5609e604f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:02.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.028+0000 7f7131200640 1 -- 172.21.15.43:0/1158810415 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609e604b790 con 0x5609e603c760 2024-09-19T05:11:02.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.028+0000 7f7131c00640 1 --2- 172.21.15.43:0/1158810415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609e603be40 0x5609e604f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:02.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.028+0000 7f7132600640 1 --2- 172.21.15.43:0/1158810415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609e6047020 0x5609e604af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:11:02.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.029+0000 7f7131200640 1 --2- 172.21.15.43:0/1158810415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609e603c760 0x5609e604aa20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7118031550 tx=0x7f7118005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=186c6ceb8b145947 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:02.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.029+0000 7f7113400640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:11:02.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.029+0000 7f7113400640 1 -- 172.21.15.43:0/1158810415 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7118035040 con 0x5609e603c760 2024-09-19T05:11:02.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.029+0000 7f7137213740 1 -- 172.21.15.43:0/1158810415 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71200041f0 con 0x5609e603c760 2024-09-19T05:11:02.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.029+0000 7f7113400640 1 -- 172.21.15.43:0/1158810415 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f71180048c0 con 0x5609e603c760 2024-09-19T05:11:02.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.029+0000 7f7113400640 1 -- 172.21.15.43:0/1158810415 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7118004ac0 con 0x5609e603c760 2024-09-19T05:11:02.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.029+0000 7f7137213740 10 client.26621 fetch_fsmap learned FSMap version 29 2024-09-19T05:11:02.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.029+0000 7f7137213740 1 -- 172.21.15.43:0/1158810415 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5609e60ca6d0 con 0x5609e603c760 2024-09-19T05:11:02.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.029+0000 7f7113400640 1 -- 172.21.15.43:0/1158810415 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71200041f0 con 0x5609e603c760 2024-09-19T05:11:02.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.030+0000 7f7113400640 1 -- 172.21.15.43:0/1158810415 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f71180035f0 con 0x5609e603c760 2024-09-19T05:11:02.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.030+0000 7f7137213740 10 client.26621 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:11:02.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.030+0000 7f7137213740 10 client.26621 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T05:11:02.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.030+0000 7f7137213740 20 client.26621 populate_metadata read hostname 'smithi043' 2024-09-19T05:11:02.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.030+0000 7f7137213740 1 --2- 172.21.15.43:0/1158810415 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x5609e60ca900 0x5609e60cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:02.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.030+0000 7f7137213740 4 client.26621 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:11:02.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.030+0000 7f7137213740 1 -- 172.21.15.43:0/1158810415 --> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5609e5fa9e60 con 0x5609e60ca900 2024-09-19T05:11:02.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.030+0000 7f7131c00640 1 --2- 172.21.15.43:0/1158810415 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x5609e60ca900 0x5609e60cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:02.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.031+0000 7f7131c00640 1 --2- 172.21.15.43:0/1158810415 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x5609e60ca900 0x5609e60cccf0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7e3d6b49eba9ab20 server_cookie=2258b050cfdccf4c in_seq=0 out_seq=0 2024-09-19T05:11:02.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.031+0000 7f7113400640 10 client.26621 ms_handle_connect on v2:172.21.15.151:6838/3794104313 2024-09-19T05:11:02.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.031+0000 7f7113400640 1 -- 172.21.15.43:0/1158810415 <== mds.2 v2:172.21.15.151:6838/3794104313 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5609e5fa9e60 con 0x5609e60ca900 2024-09-19T05:11:02.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.031+0000 7f7113400640 10 client.26621 handle_command_reply: tid=0 2024-09-19T05:11:02.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.031+0000 7f7137213740 1 client.26621 shutdown 2024-09-19T05:11:02.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.031+0000 7f7137213740 1 -- 172.21.15.43:0/1158810415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609e603c760 msgr2=0x5609e604aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:02.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.031+0000 7f7137213740 1 --2- 172.21.15.43:0/1158810415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609e603c760 0x5609e604aa20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7118031550 tx=0x7f7118005270 comp rx=0 tx=0).stop 2024-09-19T05:11:02.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.031+0000 7f7137213740 1 -- 172.21.15.43:0/1158810415 shutdown_connections 2024-09-19T05:11:02.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.031+0000 7f7137213740 1 --2- 172.21.15.43:0/1158810415 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x5609e60ca900 0x5609e60cccf0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:02.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.031+0000 7f7137213740 1 --2- 172.21.15.43:0/1158810415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609e6047020 0x5609e604af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:02.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.031+0000 7f7137213740 1 --2- 172.21.15.43:0/1158810415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609e603c760 0x5609e604aa20 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:02.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.031+0000 7f7137213740 1 --2- 172.21.15.43:0/1158810415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5609e603be40 0x5609e604f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:02.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.031+0000 7f7137213740 1 -- 172.21.15.43:0/1158810415 >> 172.21.15.43:0/1158810415 conn(0x5609e60383d0 msgr2=0x5609e6045ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:02.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.031+0000 7f7113400640 0 client.26621 ms_handle_reset on v2:172.21.15.151:6838/3794104313 2024-09-19T05:11:02.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.031+0000 7f7137213740 1 -- 172.21.15.43:0/1158810415 shutdown_connections 2024-09-19T05:11:02.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.031+0000 7f7137213740 1 -- 172.21.15.43:0/1158810415 wait complete. 2024-09-19T05:11:02.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.032+0000 7f7137213740 20 client.26621 trim_cache size 0 max 16384 2024-09-19T05:11:02.039 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:11:02.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.032+0000 7f7133000640 1 -- 172.21.15.43:0/1321530636 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f71140a3430 msgr2=0x7f71140a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:02.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.032+0000 7f7133000640 1 --2- 172.21.15.43:0/1321530636 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f71140a3430 0x7f71140a58f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x5609e6039230 tx=0x7f7120009530 comp rx=0 tx=0).stop 2024-09-19T05:11:02.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.032+0000 7f7133000640 1 -- 172.21.15.43:0/1321530636 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f712c10e260 msgr2=0x7f712c19ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:02.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.032+0000 7f7133000640 1 --2- 172.21.15.43:0/1321530636 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f712c10e260 0x7f712c19ed90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7124005fd0 tx=0x7f712400ef10 comp rx=0 tx=0).stop 2024-09-19T05:11:02.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.032+0000 7f7133000640 1 -- 172.21.15.43:0/1321530636 shutdown_connections 2024-09-19T05:11:02.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.032+0000 7f7133000640 1 --2- 172.21.15.43:0/1321530636 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f71140a3430 0x7f71140a58f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:02.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.032+0000 7f7133000640 1 --2- 172.21.15.43:0/1321530636 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f712c10e260 0x7f712c19ed90 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:02.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.032+0000 7f7133000640 1 --2- 172.21.15.43:0/1321530636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f712c074ee0 0x7f712c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:02.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.032+0000 7f7133000640 1 --2- 172.21.15.43:0/1321530636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f712c0745c0 0x7f712c19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:02.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.032+0000 7f7133000640 1 -- 172.21.15.43:0/1321530636 >> 172.21.15.43:0/1321530636 conn(0x7f712c0fd740 msgr2=0x7f712c0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:02.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.033+0000 7f7133000640 1 -- 172.21.15.43:0/1321530636 shutdown_connections 2024-09-19T05:11:02.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:02.033+0000 7f7133000640 1 -- 172.21.15.43:0/1321530636 wait complete. 2024-09-19T05:11:02.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:11:01 smithi151 ceph-mon[35425]: pgmap v194: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:11:02.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:11:01 smithi096 ceph-mon[35085]: pgmap v194: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:11:02.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:11:01 smithi043 ceph-mon[30800]: pgmap v194: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:11:02.809 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T05:11:03.190 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:11:03.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.745+0000 7f8bde600640 1 -- 172.21.15.43:0/1090753601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bd8074ee0 msgr2=0x7f8bd810dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:03.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.745+0000 7f8bde600640 1 --2- 172.21.15.43:0/1090753601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bd8074ee0 0x7f8bd810dd20 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0009900 tx=0x7f8bc002a480 comp rx=0 tx=0).stop 2024-09-19T05:11:03.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.745+0000 7f8bde600640 1 -- 172.21.15.43:0/1090753601 shutdown_connections 2024-09-19T05:11:03.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.745+0000 7f8bde600640 1 --2- 172.21.15.43:0/1090753601 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8bd810e260 0x7f8bd8110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:03.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.745+0000 7f8bde600640 1 --2- 172.21.15.43:0/1090753601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bd8074ee0 0x7f8bd810dd20 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:03.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.745+0000 7f8bde600640 1 --2- 172.21.15.43:0/1090753601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8bd80745c0 0x7f8bd80749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:03.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.745+0000 7f8bde600640 1 -- 172.21.15.43:0/1090753601 >> 172.21.15.43:0/1090753601 conn(0x7f8bd80fd740 msgr2=0x7f8bd80ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:03.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.745+0000 7f8bde600640 1 -- 172.21.15.43:0/1090753601 shutdown_connections 2024-09-19T05:11:03.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.745+0000 7f8bde600640 1 -- 172.21.15.43:0/1090753601 wait complete. 2024-09-19T05:11:03.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.746+0000 7f8bde600640 1 Processor -- start 2024-09-19T05:11:03.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.746+0000 7f8bde600640 1 -- start start 2024-09-19T05:11:03.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.747+0000 7f8bde600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8bd80745c0 0x7f8bd819a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:03.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.747+0000 7f8bde600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8bd8074ee0 0x7f8bd819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:03.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.747+0000 7f8bde600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bd810e260 0x7f8bd819ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:03.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.747+0000 7f8bde600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd8113fb0 con 0x7f8bd810e260 2024-09-19T05:11:03.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.747+0000 7f8bde600640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd8113e30 con 0x7f8bd80745c0 2024-09-19T05:11:03.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.747+0000 7f8bde600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd8114130 con 0x7f8bd8074ee0 2024-09-19T05:11:03.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.747+0000 7f8bdd200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8bd80745c0 0x7f8bd819a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:03.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.747+0000 7f8bcfe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8bd8074ee0 0x7f8bd819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:03.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.747+0000 7f8bdd200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8bd80745c0 0x7f8bd819a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:39654/0 (socket says 172.21.15.43:39654) 2024-09-19T05:11:03.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.747+0000 7f8bdd200640 1 -- 172.21.15.43:0/1305212800 learned_addr learned my addr 172.21.15.43:0/1305212800 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:03.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.747+0000 7f8bddc00640 1 --2- 172.21.15.43:0/1305212800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bd810e260 0x7f8bd819ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:03.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.748+0000 7f8bdd200640 1 -- 172.21.15.43:0/1305212800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8bd8074ee0 msgr2=0x7f8bd819aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:03.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.748+0000 7f8bdd200640 1 --2- 172.21.15.43:0/1305212800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8bd8074ee0 0x7f8bd819aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:03.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.748+0000 7f8bdd200640 1 -- 172.21.15.43:0/1305212800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bd810e260 msgr2=0x7f8bd819ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:03.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.748+0000 7f8bdd200640 1 --2- 172.21.15.43:0/1305212800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bd810e260 0x7f8bd819ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:03.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.748+0000 7f8bdd200640 1 -- 172.21.15.43:0/1305212800 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd819f520 con 0x7f8bd80745c0 2024-09-19T05:11:03.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.748+0000 7f8bcfe00640 1 --2- 172.21.15.43:0/1305212800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8bd8074ee0 0x7f8bd819aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:11:03.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.748+0000 7f8bdd200640 1 --2- 172.21.15.43:0/1305212800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8bd80745c0 0x7f8bd819a570 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8bc800e9f0 tx=0x7f8bc800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:03.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.748+0000 7f8bd6000640 1 -- 172.21.15.43:0/1305212800 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc800cd80 con 0x7f8bd80745c0 2024-09-19T05:11:03.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.748+0000 7f8bde600640 1 -- 172.21.15.43:0/1305212800 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bd819f810 con 0x7f8bd80745c0 2024-09-19T05:11:03.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.749+0000 7f8bde600640 1 -- 172.21.15.43:0/1305212800 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bd81a70f0 con 0x7f8bd80745c0 2024-09-19T05:11:03.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.749+0000 7f8bd6000640 1 -- 172.21.15.43:0/1305212800 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bc8010910 con 0x7f8bd80745c0 2024-09-19T05:11:03.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.749+0000 7f8bd6000640 1 -- 172.21.15.43:0/1305212800 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc8018980 con 0x7f8bd80745c0 2024-09-19T05:11:03.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.752+0000 7f8be2603740 1 Processor -- start 2024-09-19T05:11:03.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.752+0000 7f8bd6000640 1 -- 172.21.15.43:0/1305212800 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f8bc8010ac0 con 0x7f8bd80745c0 2024-09-19T05:11:03.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.752+0000 7f8be2603740 1 -- start start 2024-09-19T05:11:03.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.752+0000 7f8be2603740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55720e267e40 0x55720e268220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:03.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.752+0000 7f8be2603740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55720e268760 0x55720e272ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:03.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.752+0000 7f8be2603740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55720e273020 0x55720e275410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:03.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.752+0000 7f8be2603740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55720e1ce3e0 con 0x55720e273020 2024-09-19T05:11:03.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.752+0000 7f8be2603740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55720e175d80 con 0x55720e268760 2024-09-19T05:11:03.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.752+0000 7f8be2603740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55720e1ca3a0 con 0x55720e267e40 2024-09-19T05:11:03.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.753+0000 7f8bddc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55720e267e40 0x55720e268220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:03.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.753+0000 7f8bcfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55720e273020 0x55720e275410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:03.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.753+0000 7f8bddc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55720e267e40 0x55720e268220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:60108/0 (socket says 172.21.15.43:60108) 2024-09-19T05:11:03.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.753+0000 7f8bddc00640 1 -- 172.21.15.43:0/287520804 learned_addr learned my addr 172.21.15.43:0/287520804 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:03.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.753+0000 7f8bdd200640 1 --2- 172.21.15.43:0/287520804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55720e268760 0x55720e272ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:03.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.753+0000 7f8bddc00640 1 -- 172.21.15.43:0/287520804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55720e268760 msgr2=0x55720e272ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:03.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.753+0000 7f8bddc00640 1 --2- 172.21.15.43:0/287520804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55720e268760 0x55720e272ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:03.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.753+0000 7f8bddc00640 1 -- 172.21.15.43:0/287520804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55720e273020 msgr2=0x55720e275410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:03.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.753+0000 7f8bddc00640 1 --2- 172.21.15.43:0/287520804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55720e273020 0x55720e275410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:03.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.753+0000 7f8bddc00640 1 -- 172.21.15.43:0/287520804 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55720e194ae0 con 0x55720e267e40 2024-09-19T05:11:03.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.753+0000 7f8bdd200640 1 --2- 172.21.15.43:0/287520804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55720e268760 0x55720e272ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:03.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.753+0000 7f8bcfe00640 1 --2- 172.21.15.43:0/287520804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55720e273020 0x55720e275410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:11:03.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.754+0000 7f8bd6000640 1 --2- 172.21.15.43:0/1305212800 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f8bbc0a33b0 0x7f8bbc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:03.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.754+0000 7f8bddc00640 1 --2- 172.21.15.43:0/287520804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55720e267e40 0x55720e268220 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0009f30 tx=0x7f8bd00118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e061b8225d7c581f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:03.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.754+0000 7f8bd6000640 1 -- 172.21.15.43:0/1305212800 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f8bc80d0140 con 0x7f8bd80745c0 2024-09-19T05:11:03.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.754+0000 7f8bce000640 1 -- 172.21.15.43:0/287520804 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0004d20 con 0x55720e267e40 2024-09-19T05:11:03.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.754+0000 7f8bdd200640 1 --2- 172.21.15.43:0/1305212800 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f8bbc0a33b0 0x7f8bbc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:03.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.754+0000 7f8bce000640 1 -- 172.21.15.43:0/287520804 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bd0004050 con 0x55720e267e40 2024-09-19T05:11:03.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.754+0000 7f8bce000640 1 -- 172.21.15.43:0/287520804 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0005260 con 0x55720e267e40 2024-09-19T05:11:03.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.755+0000 7f8be2603740 1 -- 172.21.15.43:0/287520804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55720e267e40 msgr2=0x55720e268220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:03.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.755+0000 7f8be2603740 1 --2- 172.21.15.43:0/287520804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55720e267e40 0x55720e268220 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0009f30 tx=0x7f8bd00118d0 comp rx=0 tx=0).stop 2024-09-19T05:11:03.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.755+0000 7f8bdd200640 1 --2- 172.21.15.43:0/1305212800 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f8bbc0a33b0 0x7f8bbc0a5870 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x55720e265230 tx=0x7f8bc8018420 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:03.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.755+0000 7f8be2603740 1 -- 172.21.15.43:0/287520804 shutdown_connections 2024-09-19T05:11:03.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.755+0000 7f8be2603740 1 --2- 172.21.15.43:0/287520804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55720e273020 0x55720e275410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:03.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.755+0000 7f8be2603740 1 --2- 172.21.15.43:0/287520804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55720e268760 0x55720e272ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:03.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.755+0000 7f8be2603740 1 --2- 172.21.15.43:0/287520804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55720e267e40 0x55720e268220 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:03.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.755+0000 7f8be2603740 1 -- 172.21.15.43:0/287520804 >> 172.21.15.43:0/287520804 conn(0x55720e2643d0 msgr2=0x55720e2647b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:03.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.755+0000 7f8be2603740 1 -- 172.21.15.43:0/287520804 shutdown_connections 2024-09-19T05:11:03.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.755+0000 7f8be2603740 1 -- 172.21.15.43:0/287520804 wait complete. 2024-09-19T05:11:03.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.756+0000 7f8be2603740 1 -- start start 2024-09-19T05:11:03.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.756+0000 7f8be2603740 1 Processor -- start 2024-09-19T05:11:03.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.756+0000 7f8be2603740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55720e267e40 0x55720e26ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:03.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.756+0000 7f8be2603740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55720e268760 0x55720e270450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:03.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.756+0000 7f8be2603740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55720e273020 0x55720e270990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:03.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.756+0000 7f8be2603740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55720e275bf0 con 0x55720e273020 2024-09-19T05:11:03.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.756+0000 7f8be2603740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55720e1890f0 con 0x55720e268760 2024-09-19T05:11:03.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.756+0000 7f8be2603740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55720e275d70 con 0x55720e267e40 2024-09-19T05:11:03.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.757+0000 7f8be2603740 10 client.0 _ll_register_callbacks cb 0x55720e170bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:03.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.757+0000 7f8bddc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55720e267e40 0x55720e26ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:03.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.757+0000 7f8bddc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55720e273020 0x55720e270990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:03.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.757+0000 7f8bcfe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55720e268760 0x55720e270450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:03.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.757+0000 7f8bddc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55720e273020 0x55720e270990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38414/0 (socket says 172.21.15.43:38414) 2024-09-19T05:11:03.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.757+0000 7f8bcfe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55720e268760 0x55720e270450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:39668/0 (socket says 172.21.15.43:39668) 2024-09-19T05:11:03.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.757+0000 7f8bddc00640 1 -- 172.21.15.43:0/2122980166 learned_addr learned my addr 172.21.15.43:0/2122980166 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:03.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.757+0000 7f8bcfe00640 1 -- 172.21.15.43:0/2122980166 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55720e267e40 msgr2=0x55720e26ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:03.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.757+0000 7f8bcfe00640 1 --2- 172.21.15.43:0/2122980166 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55720e267e40 0x55720e26ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:03.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.757+0000 7f8bcfe00640 1 -- 172.21.15.43:0/2122980166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55720e273020 msgr2=0x55720e270990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:03.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.757+0000 7f8bcfe00640 1 --2- 172.21.15.43:0/2122980166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55720e273020 0x55720e270990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:03.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.757+0000 7f8bcfe00640 1 -- 172.21.15.43:0/2122980166 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55720df3f3c0 con 0x55720e268760 2024-09-19T05:11:03.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.757+0000 7f8bddc00640 1 --2- 172.21.15.43:0/2122980166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55720e273020 0x55720e270990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:11:03.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.758+0000 7f8bcfe00640 1 --2- 172.21.15.43:0/2122980166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55720e268760 0x55720e270450 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0005940 tx=0x7f8bc00317e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=672ec7a3c3b011d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:03.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.758+0000 7f8bc6a00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:11:03.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.758+0000 7f8bc6a00640 1 -- 172.21.15.43:0/2122980166 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0035040 con 0x55720e268760 2024-09-19T05:11:03.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.758+0000 7f8bc6a00640 1 -- 172.21.15.43:0/2122980166 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f8bc0004450 con 0x55720e268760 2024-09-19T05:11:03.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.758+0000 7f8be2603740 1 -- 172.21.15.43:0/2122980166 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bd0004050 con 0x55720e268760 2024-09-19T05:11:03.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.758+0000 7f8bc6a00640 1 -- 172.21.15.43:0/2122980166 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8bc00035f0 con 0x55720e268760 2024-09-19T05:11:03.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.758+0000 7f8be2603740 10 client.26633 fetch_fsmap learned FSMap version 29 2024-09-19T05:11:03.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.758+0000 7f8be2603740 1 -- 172.21.15.43:0/2122980166 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55720e281c30 con 0x55720e268760 2024-09-19T05:11:03.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.759+0000 7f8bc6a00640 1 -- 172.21.15.43:0/2122980166 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bd0004050 con 0x55720e268760 2024-09-19T05:11:03.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.759+0000 7f8bc6a00640 1 -- 172.21.15.43:0/2122980166 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8bc0048780 con 0x55720e268760 2024-09-19T05:11:03.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.759+0000 7f8be2603740 10 client.26633 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:11:03.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.759+0000 7f8be2603740 10 client.26633 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-19T05:11:03.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.759+0000 7f8be2603740 20 client.26633 populate_metadata read hostname 'smithi043' 2024-09-19T05:11:03.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.759+0000 7f8be2603740 1 --2- 172.21.15.43:0/2122980166 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55720e283040 0x55720e285430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:03.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.759+0000 7f8be2603740 4 client.26633 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:11:03.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.759+0000 7f8be2603740 1 -- 172.21.15.43:0/2122980166 --> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55720e1db950 con 0x55720e283040 2024-09-19T05:11:03.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.759+0000 7f8bddc00640 1 --2- 172.21.15.43:0/2122980166 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55720e283040 0x55720e285430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:03.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.760+0000 7f8bddc00640 1 --2- 172.21.15.43:0/2122980166 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55720e283040 0x55720e285430 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d54ce6e8826723fd server_cookie=b025d8d348e7a983 in_seq=0 out_seq=0 2024-09-19T05:11:03.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.760+0000 7f8bc6a00640 10 client.26633 ms_handle_connect on v2:172.21.15.43:6838/3404555030 2024-09-19T05:11:03.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.761+0000 7f8bc6a00640 1 -- 172.21.15.43:0/2122980166 <== mds.1 v2:172.21.15.43:6838/3404555030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55720e1db950 con 0x55720e283040 2024-09-19T05:11:03.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.761+0000 7f8bc6a00640 10 client.26633 handle_command_reply: tid=0 2024-09-19T05:11:03.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.761+0000 7f8be2603740 1 client.26633 shutdown 2024-09-19T05:11:03.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.761+0000 7f8be2603740 1 -- 172.21.15.43:0/2122980166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55720e268760 msgr2=0x55720e270450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:03.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.761+0000 7f8be2603740 1 --2- 172.21.15.43:0/2122980166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55720e268760 0x55720e270450 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0005940 tx=0x7f8bc00317e0 comp rx=0 tx=0).stop 2024-09-19T05:11:03.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.761+0000 7f8be2603740 1 -- 172.21.15.43:0/2122980166 shutdown_connections 2024-09-19T05:11:03.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.761+0000 7f8be2603740 1 --2- 172.21.15.43:0/2122980166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55720e273020 0x55720e270990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:03.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.761+0000 7f8be2603740 1 --2- 172.21.15.43:0/2122980166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55720e268760 0x55720e270450 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:03.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.761+0000 7f8be2603740 1 --2- 172.21.15.43:0/2122980166 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55720e283040 0x55720e285430 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:03.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.761+0000 7f8be2603740 1 --2- 172.21.15.43:0/2122980166 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55720e267e40 0x55720e26ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:03.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.761+0000 7f8be2603740 1 -- 172.21.15.43:0/2122980166 >> 172.21.15.43:0/2122980166 conn(0x55720e2643d0 msgr2=0x55720e26a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:03.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.761+0000 7f8bc6a00640 0 client.26633 ms_handle_reset on v2:172.21.15.43:6838/3404555030 2024-09-19T05:11:03.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.761+0000 7f8be2603740 1 -- 172.21.15.43:0/2122980166 shutdown_connections 2024-09-19T05:11:03.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.761+0000 7f8be2603740 1 -- 172.21.15.43:0/2122980166 wait complete. 2024-09-19T05:11:03.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.761+0000 7f8be2603740 20 client.26633 trim_cache size 0 max 16384 2024-09-19T05:11:03.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.775+0000 7f8be2603740 1 Processor -- start 2024-09-19T05:11:03.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.775+0000 7f8be2603740 1 -- start start 2024-09-19T05:11:03.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.776+0000 7f8be2603740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55720e267e40 0x55720e26e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:03.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.776+0000 7f8be2603740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55720e268760 0x55720e2707e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:03.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.776+0000 7f8be2603740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55720e273020 0x55720e28ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:03.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.776+0000 7f8be2603740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd81af800 con 0x55720e268760 2024-09-19T05:11:03.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.776+0000 7f8be2603740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd81b1340 con 0x55720e273020 2024-09-19T05:11:03.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.776+0000 7f8be2603740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd81afed0 con 0x55720e267e40 2024-09-19T05:11:03.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.776+0000 7f8bcfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55720e268760 0x55720e2707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:03.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.776+0000 7f8bcfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55720e268760 0x55720e2707e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38416/0 (socket says 172.21.15.43:38416) 2024-09-19T05:11:03.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.776+0000 7f8bcfe00640 1 -- 172.21.15.43:0/165955742 learned_addr learned my addr 172.21.15.43:0/165955742 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:03.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.776+0000 7f8bddc00640 1 --2- 172.21.15.43:0/165955742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55720e273020 0x55720e28ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:03.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.776+0000 7f8bddc00640 1 --2- 172.21.15.43:0/165955742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55720e267e40 0x55720e26e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:03.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.776+0000 7f8bcfe00640 1 -- 172.21.15.43:0/165955742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55720e267e40 msgr2=0x55720e26e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:03.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.776+0000 7f8bcfe00640 1 --2- 172.21.15.43:0/165955742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55720e267e40 0x55720e26e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:03.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.776+0000 7f8bcfe00640 1 -- 172.21.15.43:0/165955742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55720e273020 msgr2=0x55720e28ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:03.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.776+0000 7f8bcfe00640 1 --2- 172.21.15.43:0/165955742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55720e273020 0x55720e28ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:03.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.776+0000 7f8bcfe00640 1 -- 172.21.15.43:0/165955742 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55720e265060 con 0x55720e268760 2024-09-19T05:11:03.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.777+0000 7f8bddc00640 1 --2- 172.21.15.43:0/165955742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55720e267e40 0x55720e26e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:03.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.777+0000 7f8bcfe00640 1 --2- 172.21.15.43:0/165955742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55720e268760 0x55720e2707e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0048b80 tx=0x7f8bc00095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f0b17e5a672092b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:03.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.777+0000 7f8bcd600640 1 -- 172.21.15.43:0/165955742 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0035030 con 0x55720e268760 2024-09-19T05:11:03.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.777+0000 7f8bcd600640 1 -- 172.21.15.43:0/165955742 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bc0048960 con 0x55720e268760 2024-09-19T05:11:03.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.777+0000 7f8bcd600640 1 -- 172.21.15.43:0/165955742 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0005290 con 0x55720e268760 2024-09-19T05:11:03.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.778+0000 7f8be2603740 1 -- 172.21.15.43:0/165955742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55720e268760 msgr2=0x55720e2707e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:03.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.778+0000 7f8be2603740 1 --2- 172.21.15.43:0/165955742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55720e268760 0x55720e2707e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0048b80 tx=0x7f8bc00095a0 comp rx=0 tx=0).stop 2024-09-19T05:11:03.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.778+0000 7f8be2603740 1 -- 172.21.15.43:0/165955742 shutdown_connections 2024-09-19T05:11:03.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.778+0000 7f8be2603740 1 --2- 172.21.15.43:0/165955742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55720e273020 0x55720e28ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:03.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.778+0000 7f8be2603740 1 --2- 172.21.15.43:0/165955742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55720e268760 0x55720e2707e0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:03.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.778+0000 7f8be2603740 1 --2- 172.21.15.43:0/165955742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55720e267e40 0x55720e26e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:03.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.778+0000 7f8be2603740 1 -- 172.21.15.43:0/165955742 >> 172.21.15.43:0/165955742 conn(0x55720e2643d0 msgr2=0x55720e271f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:03.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.778+0000 7f8be2603740 1 -- 172.21.15.43:0/165955742 shutdown_connections 2024-09-19T05:11:03.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.778+0000 7f8be2603740 1 -- 172.21.15.43:0/165955742 wait complete. 2024-09-19T05:11:03.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.779+0000 7f8be2603740 1 -- start start 2024-09-19T05:11:03.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.779+0000 7f8be2603740 1 Processor -- start 2024-09-19T05:11:03.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.779+0000 7f8be2603740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55720e267e40 0x55720e27b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:03.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.779+0000 7f8be2603740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55720e268760 0x55720e26cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:03.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.779+0000 7f8be2603740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55720e273020 0x55720e26cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:03.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.779+0000 7f8be2603740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd81b0c70 con 0x55720e268760 2024-09-19T05:11:03.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.779+0000 7f8be2603740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd81b05a0 con 0x55720e267e40 2024-09-19T05:11:03.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.779+0000 7f8be2603740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd81ad7e0 con 0x55720e273020 2024-09-19T05:11:03.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.779+0000 7f8be2603740 10 client.0 _ll_register_callbacks cb 0x7f8bc0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:03.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.780+0000 7f8bcfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55720e268760 0x55720e26cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:03.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.780+0000 7f8bddc00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55720e267e40 0x55720e27b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:03.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.780+0000 7f8bcfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55720e268760 0x55720e26cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38432/0 (socket says 172.21.15.43:38432) 2024-09-19T05:11:03.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.780+0000 7f8bcfe00640 1 -- 172.21.15.43:0/174282522 learned_addr learned my addr 172.21.15.43:0/174282522 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:03.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.780+0000 7f8bddc00640 1 --2- 172.21.15.43:0/174282522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55720e273020 0x55720e26cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:03.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.780+0000 7f8bcfe00640 1 -- 172.21.15.43:0/174282522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55720e273020 msgr2=0x55720e26cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:03.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.780+0000 7f8bcfe00640 1 --2- 172.21.15.43:0/174282522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55720e273020 0x55720e26cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:03.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.780+0000 7f8bcfe00640 1 -- 172.21.15.43:0/174282522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55720e267e40 msgr2=0x55720e27b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:03.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.780+0000 7f8bcfe00640 1 --2- 172.21.15.43:0/174282522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55720e267e40 0x55720e27b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:03.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.780+0000 7f8bcfe00640 1 -- 172.21.15.43:0/174282522 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55720e26d660 con 0x55720e268760 2024-09-19T05:11:03.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.780+0000 7f8bddc00640 1 --2- 172.21.15.43:0/174282522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55720e273020 0x55720e26cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:03.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.780+0000 7f8bcfe00640 1 --2- 172.21.15.43:0/174282522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55720e268760 0x55720e26cab0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0009850 tx=0x7f8bc00045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af5376fe64076219 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:03.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.780+0000 7f8bc6a00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:11:03.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.781+0000 7f8bc6a00640 1 -- 172.21.15.43:0/174282522 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0035070 con 0x55720e268760 2024-09-19T05:11:03.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.781+0000 7f8bc6a00640 1 -- 172.21.15.43:0/174282522 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f8bc0031a90 con 0x55720e268760 2024-09-19T05:11:03.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.781+0000 7f8be2603740 1 -- 172.21.15.43:0/174282522 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bc0048960 con 0x55720e268760 2024-09-19T05:11:03.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.781+0000 7f8bc6a00640 1 -- 172.21.15.43:0/174282522 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8bc00317c0 con 0x55720e268760 2024-09-19T05:11:03.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.781+0000 7f8be2603740 10 client.16611 fetch_fsmap learned FSMap version 29 2024-09-19T05:11:03.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.781+0000 7f8be2603740 1 -- 172.21.15.43:0/174282522 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55720e2edb40 con 0x55720e268760 2024-09-19T05:11:03.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.781+0000 7f8bc6a00640 1 -- 172.21.15.43:0/174282522 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bc0048960 con 0x55720e268760 2024-09-19T05:11:03.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.782+0000 7f8bc6a00640 1 -- 172.21.15.43:0/174282522 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8bc00038b0 con 0x55720e268760 2024-09-19T05:11:03.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.782+0000 7f8be2603740 10 client.16611 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:11:03.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.782+0000 7f8be2603740 10 client.16611 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-19T05:11:03.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.782+0000 7f8be2603740 20 client.16611 populate_metadata read hostname 'smithi043' 2024-09-19T05:11:03.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.782+0000 7f8be2603740 1 --2- 172.21.15.43:0/174282522 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55720e2ee190 0x55720e2f0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:03.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.782+0000 7f8be2603740 4 client.16611 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:11:03.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.782+0000 7f8be2603740 1 -- 172.21.15.43:0/174282522 --> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55720e1d5e60 con 0x55720e2ee190 2024-09-19T05:11:03.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.782+0000 7f8bddc00640 1 --2- 172.21.15.43:0/174282522 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55720e2ee190 0x55720e2f0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:03.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.783+0000 7f8bddc00640 1 --2- 172.21.15.43:0/174282522 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55720e2ee190 0x55720e2f0580 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=21dfea805fb9ef0b server_cookie=9e523023526682b4 in_seq=0 out_seq=0 2024-09-19T05:11:03.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.783+0000 7f8bc6a00640 10 client.16611 ms_handle_connect on v2:172.21.15.43:6838/3404555030 2024-09-19T05:11:03.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.783+0000 7f8bc6a00640 1 -- 172.21.15.43:0/174282522 <== mds.1 v2:172.21.15.43:6838/3404555030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55720e1d5e60 con 0x55720e2ee190 2024-09-19T05:11:03.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.783+0000 7f8bc6a00640 10 client.16611 handle_command_reply: tid=0 2024-09-19T05:11:03.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.783+0000 7f8be2603740 1 client.16611 shutdown 2024-09-19T05:11:03.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.783+0000 7f8be2603740 1 -- 172.21.15.43:0/174282522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55720e268760 msgr2=0x55720e26cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:03.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.783+0000 7f8be2603740 1 --2- 172.21.15.43:0/174282522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55720e268760 0x55720e26cab0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0009850 tx=0x7f8bc00045c0 comp rx=0 tx=0).stop 2024-09-19T05:11:03.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.783+0000 7f8be2603740 1 -- 172.21.15.43:0/174282522 shutdown_connections 2024-09-19T05:11:03.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.783+0000 7f8be2603740 1 --2- 172.21.15.43:0/174282522 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55720e2ee190 0x55720e2f0580 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:03.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.783+0000 7f8be2603740 1 --2- 172.21.15.43:0/174282522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55720e273020 0x55720e26cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:03.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.783+0000 7f8be2603740 1 --2- 172.21.15.43:0/174282522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55720e268760 0x55720e26cab0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:03.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.783+0000 7f8be2603740 1 --2- 172.21.15.43:0/174282522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55720e267e40 0x55720e27b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:03.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.783+0000 7f8be2603740 1 -- 172.21.15.43:0/174282522 >> 172.21.15.43:0/174282522 conn(0x55720e2643d0 msgr2=0x55720e262c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:03.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.784+0000 7f8bc6a00640 0 client.16611 ms_handle_reset on v2:172.21.15.43:6838/3404555030 2024-09-19T05:11:03.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.784+0000 7f8be2603740 1 -- 172.21.15.43:0/174282522 shutdown_connections 2024-09-19T05:11:03.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.784+0000 7f8be2603740 1 -- 172.21.15.43:0/174282522 wait complete. 2024-09-19T05:11:03.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.784+0000 7f8be2603740 20 client.16611 trim_cache size 0 max 16384 2024-09-19T05:11:03.792 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:11:03.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.785+0000 7f8bde600640 1 -- 172.21.15.43:0/1305212800 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f8bbc0a33b0 msgr2=0x7f8bbc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:03.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.785+0000 7f8bde600640 1 --2- 172.21.15.43:0/1305212800 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f8bbc0a33b0 0x7f8bbc0a5870 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x55720e265230 tx=0x7f8bc8018420 comp rx=0 tx=0).stop 2024-09-19T05:11:03.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.785+0000 7f8bde600640 1 -- 172.21.15.43:0/1305212800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8bd80745c0 msgr2=0x7f8bd819a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:03.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.785+0000 7f8bde600640 1 --2- 172.21.15.43:0/1305212800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8bd80745c0 0x7f8bd819a570 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8bc800e9f0 tx=0x7f8bc800eeb0 comp rx=0 tx=0).stop 2024-09-19T05:11:03.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.785+0000 7f8bde600640 1 -- 172.21.15.43:0/1305212800 shutdown_connections 2024-09-19T05:11:03.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.785+0000 7f8bde600640 1 --2- 172.21.15.43:0/1305212800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bd810e260 0x7f8bd819ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:03.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.785+0000 7f8bde600640 1 --2- 172.21.15.43:0/1305212800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8bd8074ee0 0x7f8bd819aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:03.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.785+0000 7f8bde600640 1 --2- 172.21.15.43:0/1305212800 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f8bbc0a33b0 0x7f8bbc0a5870 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:03.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.785+0000 7f8bde600640 1 --2- 172.21.15.43:0/1305212800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8bd80745c0 0x7f8bd819a570 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:03.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.785+0000 7f8bde600640 1 -- 172.21.15.43:0/1305212800 >> 172.21.15.43:0/1305212800 conn(0x7f8bd80fd740 msgr2=0x7f8bd80fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:03.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.785+0000 7f8bde600640 1 -- 172.21.15.43:0/1305212800 shutdown_connections 2024-09-19T05:11:03.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:03.785+0000 7f8bde600640 1 -- 172.21.15.43:0/1305212800 wait complete. 2024-09-19T05:11:04.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:11:03 smithi096 ceph-mon[35085]: pgmap v195: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:11:04.279 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:11:03 smithi043 ceph-mon[30800]: pgmap v195: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:11:04.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:11:03 smithi151 ceph-mon[35425]: pgmap v195: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:11:04.683 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T05:11:05.061 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:11:05.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.621+0000 7f2a87200640 1 -- 172.21.15.43:0/803445538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a8810e260 msgr2=0x7f2a88110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:05.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.621+0000 7f2a87200640 1 --2- 172.21.15.43:0/803445538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a8810e260 0x7f2a88110650 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f2a80009ca0 tx=0x7f2a8002a950 comp rx=0 tx=0).stop 2024-09-19T05:11:05.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.621+0000 7f2a87200640 1 -- 172.21.15.43:0/803445538 shutdown_connections 2024-09-19T05:11:05.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.621+0000 7f2a87200640 1 --2- 172.21.15.43:0/803445538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a8810e260 0x7f2a88110650 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:05.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.621+0000 7f2a87200640 1 --2- 172.21.15.43:0/803445538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a88074ee0 0x7f2a8810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:05.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.621+0000 7f2a87200640 1 --2- 172.21.15.43:0/803445538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a880745c0 0x7f2a880749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:05.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.621+0000 7f2a87200640 1 -- 172.21.15.43:0/803445538 >> 172.21.15.43:0/803445538 conn(0x7f2a880fd740 msgr2=0x7f2a880ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:05.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.621+0000 7f2a87200640 1 -- 172.21.15.43:0/803445538 shutdown_connections 2024-09-19T05:11:05.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.621+0000 7f2a87200640 1 -- 172.21.15.43:0/803445538 wait complete. 2024-09-19T05:11:05.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.622+0000 7f2a87200640 1 Processor -- start 2024-09-19T05:11:05.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.623+0000 7f2a87200640 1 -- start start 2024-09-19T05:11:05.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.623+0000 7f2a87200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a880745c0 0x7f2a8819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:05.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.623+0000 7f2a87200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a88074ee0 0x7f2a8819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:05.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.623+0000 7f2a87200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a8810e260 0x7f2a8819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:05.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.623+0000 7f2a87200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2a88113f90 con 0x7f2a880745c0 2024-09-19T05:11:05.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.623+0000 7f2a87200640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2a88113e10 con 0x7f2a8810e260 2024-09-19T05:11:05.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.623+0000 7f2a87200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2a88114110 con 0x7f2a88074ee0 2024-09-19T05:11:05.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.623+0000 7f2a85e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a880745c0 0x7f2a8819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:05.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.623+0000 7f2a85e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a880745c0 0x7f2a8819a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38458/0 (socket says 172.21.15.43:38458) 2024-09-19T05:11:05.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.623+0000 7f2a85e00640 1 -- 172.21.15.43:0/156809009 learned_addr learned my addr 172.21.15.43:0/156809009 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:05.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.623+0000 7f2a7fe00640 1 --2- 172.21.15.43:0/156809009 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a88074ee0 0x7f2a8819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:05.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.623+0000 7f2a86800640 1 --2- 172.21.15.43:0/156809009 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a8810e260 0x7f2a8819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:05.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.624+0000 7f2a85e00640 1 -- 172.21.15.43:0/156809009 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a88074ee0 msgr2=0x7f2a8819a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:05.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.624+0000 7f2a85e00640 1 --2- 172.21.15.43:0/156809009 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a88074ee0 0x7f2a8819a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:05.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.624+0000 7f2a85e00640 1 -- 172.21.15.43:0/156809009 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a8810e260 msgr2=0x7f2a8819ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:05.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.624+0000 7f2a85e00640 1 --2- 172.21.15.43:0/156809009 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a8810e260 0x7f2a8819ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:05.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.624+0000 7f2a85e00640 1 -- 172.21.15.43:0/156809009 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a8819f440 con 0x7f2a880745c0 2024-09-19T05:11:05.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.624+0000 7f2a85e00640 1 --2- 172.21.15.43:0/156809009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a880745c0 0x7f2a8819a490 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f2a7800e9f0 tx=0x7f2a7800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:05.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.624+0000 7f2a7ea00640 1 -- 172.21.15.43:0/156809009 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7800cd80 con 0x7f2a880745c0 2024-09-19T05:11:05.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.624+0000 7f2a7ea00640 1 -- 172.21.15.43:0/156809009 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a78010910 con 0x7f2a880745c0 2024-09-19T05:11:05.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.624+0000 7f2a7ea00640 1 -- 172.21.15.43:0/156809009 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a78018990 con 0x7f2a880745c0 2024-09-19T05:11:05.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.624+0000 7f2a87200640 1 -- 172.21.15.43:0/156809009 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a8819f730 con 0x7f2a880745c0 2024-09-19T05:11:05.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.625+0000 7f2a87200640 1 -- 172.21.15.43:0/156809009 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a881a7010 con 0x7f2a880745c0 2024-09-19T05:11:05.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.628+0000 7f2a8f49b740 1 Processor -- start 2024-09-19T05:11:05.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.628+0000 7f2a8f49b740 1 -- start start 2024-09-19T05:11:05.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.628+0000 7f2a8f49b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56231b198e40 0x56231b199220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:05.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.628+0000 7f2a8f49b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56231b199760 0x56231b1a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:05.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.628+0000 7f2a8f49b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56231b1a4020 0x56231b1a6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:05.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.628+0000 7f2a8f49b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56231b0ff3e0 con 0x56231b198e40 2024-09-19T05:11:05.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.628+0000 7f2a8f49b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56231b0a6d80 con 0x56231b1a4020 2024-09-19T05:11:05.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.628+0000 7f2a8f49b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56231b0fb3a0 con 0x56231b199760 2024-09-19T05:11:05.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.629+0000 7f2a86800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56231b198e40 0x56231b199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:05.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.629+0000 7f2a86800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56231b198e40 0x56231b199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38464/0 (socket says 172.21.15.43:38464) 2024-09-19T05:11:05.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.629+0000 7f2a86800640 1 -- 172.21.15.43:0/1230184172 learned_addr learned my addr 172.21.15.43:0/1230184172 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:05.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.629+0000 7f2a7fe00640 1 --2- 172.21.15.43:0/1230184172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56231b1a4020 0x56231b1a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:05.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.629+0000 7f2a86800640 1 -- 172.21.15.43:0/1230184172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56231b199760 msgr2=0x56231b1a3ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T05:11:05.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.629+0000 7f2a86800640 1 --2- 172.21.15.43:0/1230184172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56231b199760 0x56231b1a3ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:05.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.629+0000 7f2a86800640 1 -- 172.21.15.43:0/1230184172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56231b1a4020 msgr2=0x56231b1a6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:05.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.629+0000 7f2a86800640 1 --2- 172.21.15.43:0/1230184172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56231b1a4020 0x56231b1a6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:05.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.629+0000 7f2a86800640 1 -- 172.21.15.43:0/1230184172 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56231b0c5ae0 con 0x56231b198e40 2024-09-19T05:11:05.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.630+0000 7f2a7ea00640 1 -- 172.21.15.43:0/156809009 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f2a78010ac0 con 0x7f2a880745c0 2024-09-19T05:11:05.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.630+0000 7f2a86800640 1 --2- 172.21.15.43:0/1230184172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56231b198e40 0x56231b199220 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f2a80009ca0 tx=0x7f2a800093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=946cc8ab922427dd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:05.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.630+0000 7f2a77000640 1 -- 172.21.15.43:0/1230184172 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a80048070 con 0x56231b198e40 2024-09-19T05:11:05.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.630+0000 7f2a77000640 1 -- 172.21.15.43:0/1230184172 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a8002f940 con 0x56231b198e40 2024-09-19T05:11:05.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.630+0000 7f2a77000640 1 -- 172.21.15.43:0/1230184172 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a80034040 con 0x56231b198e40 2024-09-19T05:11:05.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.631+0000 7f2a8f49b740 1 -- 172.21.15.43:0/1230184172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56231b198e40 msgr2=0x56231b199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:05.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.631+0000 7f2a8f49b740 1 --2- 172.21.15.43:0/1230184172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56231b198e40 0x56231b199220 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f2a80009ca0 tx=0x7f2a800093e0 comp rx=0 tx=0).stop 2024-09-19T05:11:05.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.631+0000 7f2a8f49b740 1 -- 172.21.15.43:0/1230184172 shutdown_connections 2024-09-19T05:11:05.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.631+0000 7f2a8f49b740 1 --2- 172.21.15.43:0/1230184172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56231b1a4020 0x56231b1a6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:05.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.631+0000 7f2a8f49b740 1 --2- 172.21.15.43:0/1230184172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56231b199760 0x56231b1a3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:05.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.631+0000 7f2a8f49b740 1 --2- 172.21.15.43:0/1230184172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56231b198e40 0x56231b199220 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:05.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.631+0000 7f2a8f49b740 1 -- 172.21.15.43:0/1230184172 >> 172.21.15.43:0/1230184172 conn(0x56231b1953d0 msgr2=0x56231b1957b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:05.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.631+0000 7f2a8f49b740 1 -- 172.21.15.43:0/1230184172 shutdown_connections 2024-09-19T05:11:05.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.631+0000 7f2a8f49b740 1 -- 172.21.15.43:0/1230184172 wait complete. 2024-09-19T05:11:05.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.632+0000 7f2a7ea00640 1 --2- 172.21.15.43:0/156809009 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f2a6c0a3340 0x7f2a6c0a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:05.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.632+0000 7f2a7ea00640 1 -- 172.21.15.43:0/156809009 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f2a780cffe0 con 0x7f2a880745c0 2024-09-19T05:11:05.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.632+0000 7f2a86800640 1 --2- 172.21.15.43:0/156809009 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f2a6c0a3340 0x7f2a6c0a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:05.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.632+0000 7f2a8f49b740 1 -- start start 2024-09-19T05:11:05.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.633+0000 7f2a8f49b740 1 Processor -- start 2024-09-19T05:11:05.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.633+0000 7f2a86800640 1 --2- 172.21.15.43:0/156809009 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f2a6c0a3340 0x7f2a6c0a5800 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2a80003b30 tx=0x7f2a80003cd0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:05.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.633+0000 7f2a8f49b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56231b198e40 0x56231b1a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:05.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.633+0000 7f2a8f49b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56231b199760 0x56231b1a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:05.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.633+0000 7f2a8f49b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56231b1a4020 0x56231b1a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:05.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.633+0000 7f2a8f49b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56231b1a6bf0 con 0x56231b1a4020 2024-09-19T05:11:05.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.633+0000 7f2a8f49b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56231b0ba0f0 con 0x56231b199760 2024-09-19T05:11:05.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.633+0000 7f2a8f49b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56231b1a6d70 con 0x56231b198e40 2024-09-19T05:11:05.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.633+0000 7f2a8f49b740 10 client.0 _ll_register_callbacks cb 0x56231b0a1bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:05.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.634+0000 7f2a85e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56231b198e40 0x56231b1a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:05.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.634+0000 7f2a86800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56231b1a4020 0x56231b1a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:05.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.634+0000 7f2a85e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56231b198e40 0x56231b1a0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:60164/0 (socket says 172.21.15.43:60164) 2024-09-19T05:11:05.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.634+0000 7f2a85e00640 1 -- 172.21.15.43:0/2286674225 learned_addr learned my addr 172.21.15.43:0/2286674225 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:05.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.634+0000 7f2a7fe00640 1 --2- 172.21.15.43:0/2286674225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56231b199760 0x56231b1a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:05.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.634+0000 7f2a85e00640 1 -- 172.21.15.43:0/2286674225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56231b199760 msgr2=0x56231b1a14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:05.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.634+0000 7f2a85e00640 1 --2- 172.21.15.43:0/2286674225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56231b199760 0x56231b1a14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:05.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.634+0000 7f2a85e00640 1 -- 172.21.15.43:0/2286674225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56231b1a4020 msgr2=0x56231b1a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:05.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.634+0000 7f2a85e00640 1 --2- 172.21.15.43:0/2286674225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56231b1a4020 0x56231b1a1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:05.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.634+0000 7f2a85e00640 1 -- 172.21.15.43:0/2286674225 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56231ae703c0 con 0x56231b198e40 2024-09-19T05:11:05.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.635+0000 7f2a85e00640 1 --2- 172.21.15.43:0/2286674225 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56231b198e40 0x56231b1a0fa0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2a780cd740 tx=0x7f2a7809bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55ac1dbad6f8a788 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:05.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.635+0000 7f2a6b400640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:11:05.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.635+0000 7f2a6b400640 1 -- 172.21.15.43:0/2286674225 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a780a0320 con 0x56231b198e40 2024-09-19T05:11:05.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.635+0000 7f2a8f49b740 1 -- 172.21.15.43:0/2286674225 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a8002f940 con 0x56231b198e40 2024-09-19T05:11:05.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.635+0000 7f2a6b400640 1 -- 172.21.15.43:0/2286674225 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f2a780bc070 con 0x56231b198e40 2024-09-19T05:11:05.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.635+0000 7f2a6b400640 1 -- 172.21.15.43:0/2286674225 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2a780b5250 con 0x56231b198e40 2024-09-19T05:11:05.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.635+0000 7f2a8f49b740 10 client.26581 fetch_fsmap learned FSMap version 29 2024-09-19T05:11:05.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.635+0000 7f2a8f49b740 1 -- 172.21.15.43:0/2286674225 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56231b1bb510 con 0x56231b198e40 2024-09-19T05:11:05.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.635+0000 7f2a6b400640 1 -- 172.21.15.43:0/2286674225 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a8002f940 con 0x56231b198e40 2024-09-19T05:11:05.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.636+0000 7f2a6b400640 1 -- 172.21.15.43:0/2286674225 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2a7809b770 con 0x56231b198e40 2024-09-19T05:11:05.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.636+0000 7f2a8f49b740 10 client.26581 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:11:05.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.636+0000 7f2a8f49b740 10 client.26581 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:11:05.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.636+0000 7f2a8f49b740 20 client.26581 populate_metadata read hostname 'smithi043' 2024-09-19T05:11:05.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.636+0000 7f2a8f49b740 1 --2- 172.21.15.43:0/2286674225 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x56231b1bc9d0 0x56231b1bedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:05.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.636+0000 7f2a8f49b740 4 client.26581 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:11:05.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.636+0000 7f2a8f49b740 1 -- 172.21.15.43:0/2286674225 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56231b10c950 con 0x56231b1bc9d0 2024-09-19T05:11:05.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.636+0000 7f2a7fe00640 1 --2- 172.21.15.43:0/2286674225 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x56231b1bc9d0 0x56231b1bedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:05.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.637+0000 7f2a7fe00640 1 --2- 172.21.15.43:0/2286674225 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x56231b1bc9d0 0x56231b1bedc0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df05843a4a459daa server_cookie=b8800454d73b8a03 in_seq=0 out_seq=0 2024-09-19T05:11:05.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.637+0000 7f2a6b400640 10 client.26581 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:11:05.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.638+0000 7f2a6b400640 1 -- 172.21.15.43:0/2286674225 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56231b10c950 con 0x56231b1bc9d0 2024-09-19T05:11:05.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.638+0000 7f2a6b400640 10 client.26581 handle_command_reply: tid=0 2024-09-19T05:11:05.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.638+0000 7f2a8f49b740 1 client.26581 shutdown 2024-09-19T05:11:05.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.638+0000 7f2a8f49b740 1 -- 172.21.15.43:0/2286674225 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56231b198e40 msgr2=0x56231b1a0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:05.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.638+0000 7f2a8f49b740 1 --2- 172.21.15.43:0/2286674225 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56231b198e40 0x56231b1a0fa0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2a780cd740 tx=0x7f2a7809bd80 comp rx=0 tx=0).stop 2024-09-19T05:11:05.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.638+0000 7f2a8f49b740 1 -- 172.21.15.43:0/2286674225 shutdown_connections 2024-09-19T05:11:05.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.638+0000 7f2a8f49b740 1 --2- 172.21.15.43:0/2286674225 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x56231b1bc9d0 0x56231b1bedc0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:05.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.638+0000 7f2a8f49b740 1 --2- 172.21.15.43:0/2286674225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56231b1a4020 0x56231b1a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:05.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.638+0000 7f2a8f49b740 1 --2- 172.21.15.43:0/2286674225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56231b199760 0x56231b1a14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:05.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.638+0000 7f2a8f49b740 1 --2- 172.21.15.43:0/2286674225 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56231b198e40 0x56231b1a0fa0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:05.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.638+0000 7f2a8f49b740 1 -- 172.21.15.43:0/2286674225 >> 172.21.15.43:0/2286674225 conn(0x56231b1953d0 msgr2=0x56231b19b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:05.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.638+0000 7f2a6b400640 0 client.26581 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:11:05.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.638+0000 7f2a8f49b740 1 -- 172.21.15.43:0/2286674225 shutdown_connections 2024-09-19T05:11:05.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.638+0000 7f2a8f49b740 1 -- 172.21.15.43:0/2286674225 wait complete. 2024-09-19T05:11:05.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.639+0000 7f2a8f49b740 20 client.26581 trim_cache size 0 max 16384 2024-09-19T05:11:05.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.654+0000 7f2a8f49b740 1 Processor -- start 2024-09-19T05:11:05.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.654+0000 7f2a8f49b740 1 -- start start 2024-09-19T05:11:05.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.654+0000 7f2a8f49b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56231b198e40 0x56231b194c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:05.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.654+0000 7f2a8f49b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56231b199760 0x56231b1c4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:05.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.654+0000 7f2a8f49b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56231b1a4020 0x56231b1c4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:05.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.654+0000 7f2a8f49b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2a881ab4e0 con 0x56231b1a4020 2024-09-19T05:11:05.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.654+0000 7f2a8f49b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2a881ad020 con 0x56231b198e40 2024-09-19T05:11:05.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.654+0000 7f2a8f49b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2a881abbb0 con 0x56231b199760 2024-09-19T05:11:05.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.654+0000 7f2a7fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56231b199760 0x56231b1c4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:05.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.654+0000 7f2a86800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56231b1a4020 0x56231b1c4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:05.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.654+0000 7f2a7fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56231b199760 0x56231b1c4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:60176/0 (socket says 172.21.15.43:60176) 2024-09-19T05:11:05.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.654+0000 7f2a7fe00640 1 -- 172.21.15.43:0/578504056 learned_addr learned my addr 172.21.15.43:0/578504056 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:05.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.655+0000 7f2a85e00640 1 --2- 172.21.15.43:0/578504056 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56231b198e40 0x56231b194c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:05.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.655+0000 7f2a7fe00640 1 -- 172.21.15.43:0/578504056 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56231b198e40 msgr2=0x56231b194c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:05.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.655+0000 7f2a7fe00640 1 --2- 172.21.15.43:0/578504056 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56231b198e40 0x56231b194c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:05.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.655+0000 7f2a7fe00640 1 -- 172.21.15.43:0/578504056 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56231b1a4020 msgr2=0x56231b1c4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:05.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.655+0000 7f2a7fe00640 1 --2- 172.21.15.43:0/578504056 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56231b1a4020 0x56231b1c4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:05.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.655+0000 7f2a7fe00640 1 -- 172.21.15.43:0/578504056 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56231b1bf380 con 0x56231b199760 2024-09-19T05:11:05.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.655+0000 7f2a7fe00640 1 --2- 172.21.15.43:0/578504056 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56231b199760 0x56231b1c4610 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x56231b196750 tx=0x7f2a70005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b3f8afb047e53cf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:05.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.655+0000 7f2a76600640 1 -- 172.21.15.43:0/578504056 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a70005220 con 0x56231b199760 2024-09-19T05:11:05.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.655+0000 7f2a76600640 1 -- 172.21.15.43:0/578504056 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a70004c80 con 0x56231b199760 2024-09-19T05:11:05.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.655+0000 7f2a76600640 1 -- 172.21.15.43:0/578504056 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7001eb10 con 0x56231b199760 2024-09-19T05:11:05.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.656+0000 7f2a8f49b740 1 -- 172.21.15.43:0/578504056 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56231b199760 msgr2=0x56231b1c4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:05.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.656+0000 7f2a8f49b740 1 --2- 172.21.15.43:0/578504056 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56231b199760 0x56231b1c4610 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x56231b196750 tx=0x7f2a70005f90 comp rx=0 tx=0).stop 2024-09-19T05:11:05.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.656+0000 7f2a8f49b740 1 -- 172.21.15.43:0/578504056 shutdown_connections 2024-09-19T05:11:05.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.656+0000 7f2a8f49b740 1 --2- 172.21.15.43:0/578504056 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56231b1a4020 0x56231b1c4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:05.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.656+0000 7f2a8f49b740 1 --2- 172.21.15.43:0/578504056 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56231b199760 0x56231b1c4610 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:05.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.656+0000 7f2a8f49b740 1 --2- 172.21.15.43:0/578504056 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56231b198e40 0x56231b194c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:05.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.656+0000 7f2a8f49b740 1 -- 172.21.15.43:0/578504056 >> 172.21.15.43:0/578504056 conn(0x56231b1953d0 msgr2=0x56231b1a3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:05.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.656+0000 7f2a8f49b740 1 -- 172.21.15.43:0/578504056 shutdown_connections 2024-09-19T05:11:05.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.656+0000 7f2a8f49b740 1 -- 172.21.15.43:0/578504056 wait complete. 2024-09-19T05:11:05.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.657+0000 7f2a8f49b740 1 -- start start 2024-09-19T05:11:05.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.657+0000 7f2a8f49b740 1 Processor -- start 2024-09-19T05:11:05.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.657+0000 7f2a8f49b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56231b198e40 0x56231b1b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:05.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.657+0000 7f2a8f49b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56231b199760 0x56231b1b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:05.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.657+0000 7f2a8f49b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56231b1a4020 0x56231b1b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:05.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.657+0000 7f2a8f49b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2a881ac950 con 0x56231b199760 2024-09-19T05:11:05.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.657+0000 7f2a8f49b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2a881ac280 con 0x56231b198e40 2024-09-19T05:11:05.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.657+0000 7f2a8f49b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2a881a94c0 con 0x56231b1a4020 2024-09-19T05:11:05.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.657+0000 7f2a8f49b740 10 client.0 _ll_register_callbacks cb 0x7f2a7800cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:05.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.658+0000 7f2a7fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56231b199760 0x56231b1b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:05.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.658+0000 7f2a85e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56231b198e40 0x56231b1b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:05.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.658+0000 7f2a7fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56231b199760 0x56231b1b1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38494/0 (socket says 172.21.15.43:38494) 2024-09-19T05:11:05.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.658+0000 7f2a7fe00640 1 -- 172.21.15.43:0/2247425464 learned_addr learned my addr 172.21.15.43:0/2247425464 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:05.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.658+0000 7f2a86800640 1 --2- 172.21.15.43:0/2247425464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56231b1a4020 0x56231b1b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:05.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.658+0000 7f2a85e00640 1 -- 172.21.15.43:0/2247425464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56231b1a4020 msgr2=0x56231b1b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:05.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.658+0000 7f2a85e00640 1 --2- 172.21.15.43:0/2247425464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56231b1a4020 0x56231b1b1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:05.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.658+0000 7f2a85e00640 1 -- 172.21.15.43:0/2247425464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56231b199760 msgr2=0x56231b1b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:05.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.658+0000 7f2a85e00640 1 --2- 172.21.15.43:0/2247425464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56231b199760 0x56231b1b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:05.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.658+0000 7f2a85e00640 1 -- 172.21.15.43:0/2247425464 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56231b1b24a0 con 0x56231b198e40 2024-09-19T05:11:05.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.658+0000 7f2a85e00640 1 --2- 172.21.15.43:0/2247425464 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56231b198e40 0x56231b1b4e90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2a78098cf0 tx=0x7f2a780a07a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b81ae3cf7a5bd284 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:05.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.658+0000 7f2a6b400640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:11:05.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.659+0000 7f2a6b400640 1 -- 172.21.15.43:0/2247425464 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7809c0f0 con 0x56231b198e40 2024-09-19T05:11:05.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.659+0000 7f2a6b400640 1 -- 172.21.15.43:0/2247425464 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f2a780bc0c0 con 0x56231b198e40 2024-09-19T05:11:05.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.659+0000 7f2a8f49b740 1 -- 172.21.15.43:0/2247425464 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a70004c80 con 0x56231b198e40 2024-09-19T05:11:05.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.659+0000 7f2a6b400640 1 -- 172.21.15.43:0/2247425464 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2a78098070 con 0x56231b198e40 2024-09-19T05:11:05.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.659+0000 7f2a8f49b740 10 client.26657 fetch_fsmap learned FSMap version 29 2024-09-19T05:11:05.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.659+0000 7f2a8f49b740 1 -- 172.21.15.43:0/2247425464 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56231b228520 con 0x56231b198e40 2024-09-19T05:11:05.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.659+0000 7f2a6b400640 1 -- 172.21.15.43:0/2247425464 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a70004c80 con 0x56231b198e40 2024-09-19T05:11:05.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.659+0000 7f2a6b400640 1 -- 172.21.15.43:0/2247425464 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2a7809b1e0 con 0x56231b198e40 2024-09-19T05:11:05.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.659+0000 7f2a8f49b740 10 client.26657 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:11:05.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.659+0000 7f2a8f49b740 10 client.26657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:11:05.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.659+0000 7f2a8f49b740 20 client.26657 populate_metadata read hostname 'smithi043' 2024-09-19T05:11:05.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.659+0000 7f2a8f49b740 1 --2- 172.21.15.43:0/2247425464 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x56231b228aa0 0x56231b22ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:05.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.659+0000 7f2a8f49b740 4 client.26657 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:11:05.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.659+0000 7f2a8f49b740 1 -- 172.21.15.43:0/2247425464 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56231b106e60 con 0x56231b228aa0 2024-09-19T05:11:05.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.660+0000 7f2a7fe00640 1 --2- 172.21.15.43:0/2247425464 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x56231b228aa0 0x56231b22ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:05.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.660+0000 7f2a7fe00640 1 --2- 172.21.15.43:0/2247425464 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x56231b228aa0 0x56231b22ae90 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=396d19bc6db5a431 server_cookie=4f7280038e9c239a in_seq=0 out_seq=0 2024-09-19T05:11:05.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.660+0000 7f2a6b400640 10 client.26657 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:11:05.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.661+0000 7f2a6b400640 1 -- 172.21.15.43:0/2247425464 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56231b106e60 con 0x56231b228aa0 2024-09-19T05:11:05.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.661+0000 7f2a6b400640 10 client.26657 handle_command_reply: tid=0 2024-09-19T05:11:05.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.661+0000 7f2a8f49b740 1 client.26657 shutdown 2024-09-19T05:11:05.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.661+0000 7f2a8f49b740 1 -- 172.21.15.43:0/2247425464 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56231b198e40 msgr2=0x56231b1b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:05.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.661+0000 7f2a8f49b740 1 --2- 172.21.15.43:0/2247425464 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56231b198e40 0x56231b1b4e90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2a78098cf0 tx=0x7f2a780a07a0 comp rx=0 tx=0).stop 2024-09-19T05:11:05.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.661+0000 7f2a8f49b740 1 -- 172.21.15.43:0/2247425464 shutdown_connections 2024-09-19T05:11:05.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.661+0000 7f2a8f49b740 1 --2- 172.21.15.43:0/2247425464 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x56231b228aa0 0x56231b22ae90 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:05.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.661+0000 7f2a8f49b740 1 --2- 172.21.15.43:0/2247425464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56231b1a4020 0x56231b1b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:05.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.661+0000 7f2a8f49b740 1 --2- 172.21.15.43:0/2247425464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56231b199760 0x56231b1b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:05.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.661+0000 7f2a6b400640 0 client.26657 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:11:05.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.661+0000 7f2a8f49b740 1 --2- 172.21.15.43:0/2247425464 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56231b198e40 0x56231b1b4e90 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:05.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.661+0000 7f2a8f49b740 1 -- 172.21.15.43:0/2247425464 >> 172.21.15.43:0/2247425464 conn(0x56231b1953d0 msgr2=0x56231b1a0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:05.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.661+0000 7f2a8f49b740 1 -- 172.21.15.43:0/2247425464 shutdown_connections 2024-09-19T05:11:05.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.661+0000 7f2a8f49b740 1 -- 172.21.15.43:0/2247425464 wait complete. 2024-09-19T05:11:05.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.661+0000 7f2a8f49b740 20 client.26657 trim_cache size 0 max 16384 2024-09-19T05:11:05.668 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:11:05.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.662+0000 7f2a87200640 1 -- 172.21.15.43:0/156809009 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f2a6c0a3340 msgr2=0x7f2a6c0a5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:05.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.662+0000 7f2a87200640 1 --2- 172.21.15.43:0/156809009 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f2a6c0a3340 0x7f2a6c0a5800 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2a80003b30 tx=0x7f2a80003cd0 comp rx=0 tx=0).stop 2024-09-19T05:11:05.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.662+0000 7f2a87200640 1 -- 172.21.15.43:0/156809009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a880745c0 msgr2=0x7f2a8819a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:05.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.662+0000 7f2a87200640 1 --2- 172.21.15.43:0/156809009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a880745c0 0x7f2a8819a490 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f2a7800e9f0 tx=0x7f2a7800eeb0 comp rx=0 tx=0).stop 2024-09-19T05:11:05.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.662+0000 7f2a87200640 1 -- 172.21.15.43:0/156809009 shutdown_connections 2024-09-19T05:11:05.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.662+0000 7f2a87200640 1 --2- 172.21.15.43:0/156809009 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f2a6c0a3340 0x7f2a6c0a5800 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:05.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.662+0000 7f2a87200640 1 --2- 172.21.15.43:0/156809009 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a8810e260 0x7f2a8819ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:05.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.662+0000 7f2a87200640 1 --2- 172.21.15.43:0/156809009 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a88074ee0 0x7f2a8819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:05.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.662+0000 7f2a87200640 1 --2- 172.21.15.43:0/156809009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a880745c0 0x7f2a8819a490 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:05.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.662+0000 7f2a87200640 1 -- 172.21.15.43:0/156809009 >> 172.21.15.43:0/156809009 conn(0x7f2a880fd740 msgr2=0x7f2a880fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:05.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.662+0000 7f2a87200640 1 -- 172.21.15.43:0/156809009 shutdown_connections 2024-09-19T05:11:05.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:05.662+0000 7f2a87200640 1 -- 172.21.15.43:0/156809009 wait complete. 2024-09-19T05:11:06.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:11:05 smithi096 ceph-mon[35085]: pgmap v196: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-09-19T05:11:06.236 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:11:05 smithi043 ceph-mon[30800]: pgmap v196: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-09-19T05:11:06.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:11:05 smithi151 ceph-mon[35425]: pgmap v196: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-09-19T05:11:07.462 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T05:11:07.462 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T05:11:07.855 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:11:08.166 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:11:07 smithi043 ceph-mon[30800]: pgmap v197: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-09-19T05:11:08.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:11:07 smithi096 ceph-mon[35085]: pgmap v197: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-09-19T05:11:08.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:11:07 smithi151 ceph-mon[35425]: pgmap v197: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-09-19T05:11:08.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.467+0000 7fcd92e00640 1 -- 172.21.15.43:0/460199251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd8c102790 msgr2=0x7fcd8c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:08.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.467+0000 7fcd92e00640 1 --2- 172.21.15.43:0/460199251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd8c102790 0x7fcd8c1067e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fcd74009900 tx=0x7fcd7402a3f0 comp rx=0 tx=0).stop 2024-09-19T05:11:08.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.467+0000 7fcd92e00640 1 -- 172.21.15.43:0/460199251 shutdown_connections 2024-09-19T05:11:08.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.467+0000 7fcd92e00640 1 --2- 172.21.15.43:0/460199251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd8c106f10 0x7fcd8c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:08.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.467+0000 7fcd92e00640 1 --2- 172.21.15.43:0/460199251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd8c102790 0x7fcd8c1067e0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:08.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.467+0000 7fcd92e00640 1 --2- 172.21.15.43:0/460199251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd8c101de0 0x7fcd8c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:08.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.467+0000 7fcd92e00640 1 -- 172.21.15.43:0/460199251 >> 172.21.15.43:0/460199251 conn(0x7fcd8c0fd8c0 msgr2=0x7fcd8c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:08.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.467+0000 7fcd92e00640 1 -- 172.21.15.43:0/460199251 shutdown_connections 2024-09-19T05:11:08.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.467+0000 7fcd92e00640 1 -- 172.21.15.43:0/460199251 wait complete. 2024-09-19T05:11:08.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.468+0000 7fcd92e00640 1 Processor -- start 2024-09-19T05:11:08.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.468+0000 7fcd92e00640 1 -- start start 2024-09-19T05:11:08.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.468+0000 7fcd92e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd8c101de0 0x7fcd8c19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:08.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.468+0000 7fcd92e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd8c102790 0x7fcd8c19abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:08.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.468+0000 7fcd92e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd8c106f10 0x7fcd8c19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:08.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.468+0000 7fcd92e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8c111fc0 con 0x7fcd8c102790 2024-09-19T05:11:08.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.468+0000 7fcd92e00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8c111e40 con 0x7fcd8c101de0 2024-09-19T05:11:08.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.468+0000 7fcd92e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8c112140 con 0x7fcd8c106f10 2024-09-19T05:11:08.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.469+0000 7fcd83400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd8c102790 0x7fcd8c19abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:08.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.469+0000 7fcd8be00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd8c101de0 0x7fcd8c19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:08.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.469+0000 7fcd8be00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd8c101de0 0x7fcd8c19a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:45886/0 (socket says 172.21.15.43:45886) 2024-09-19T05:11:08.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.469+0000 7fcd8be00640 1 -- 172.21.15.43:0/3800433478 learned_addr learned my addr 172.21.15.43:0/3800433478 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:08.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.469+0000 7fcd90e00640 1 --2- 172.21.15.43:0/3800433478 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd8c106f10 0x7fcd8c19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:08.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.469+0000 7fcd8be00640 1 -- 172.21.15.43:0/3800433478 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd8c106f10 msgr2=0x7fcd8c19ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:08.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.469+0000 7fcd8be00640 1 --2- 172.21.15.43:0/3800433478 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd8c106f10 0x7fcd8c19ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:08.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.469+0000 7fcd8be00640 1 -- 172.21.15.43:0/3800433478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd8c102790 msgr2=0x7fcd8c19abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:08.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.469+0000 7fcd8be00640 1 --2- 172.21.15.43:0/3800433478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd8c102790 0x7fcd8c19abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:08.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.469+0000 7fcd8be00640 1 -- 172.21.15.43:0/3800433478 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd8c19f640 con 0x7fcd8c101de0 2024-09-19T05:11:08.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.470+0000 7fcd8be00640 1 --2- 172.21.15.43:0/3800433478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd8c101de0 0x7fcd8c19a690 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c00ea60 tx=0x7fcd7c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:08.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.470+0000 7fcd89600640 1 -- 172.21.15.43:0/3800433478 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c00cde0 con 0x7fcd8c101de0 2024-09-19T05:11:08.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.470+0000 7fcd92e00640 1 -- 172.21.15.43:0/3800433478 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd8c19f930 con 0x7fcd8c101de0 2024-09-19T05:11:08.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.470+0000 7fcd89600640 1 -- 172.21.15.43:0/3800433478 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd7c0035f0 con 0x7fcd8c101de0 2024-09-19T05:11:08.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.470+0000 7fcd89600640 1 -- 172.21.15.43:0/3800433478 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c010670 con 0x7fcd8c101de0 2024-09-19T05:11:08.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.470+0000 7fcd92e00640 1 -- 172.21.15.43:0/3800433478 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd8c1a7210 con 0x7fcd8c101de0 2024-09-19T05:11:08.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.473+0000 7fcd958a1740 1 Processor -- start 2024-09-19T05:11:08.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.473+0000 7fcd958a1740 1 -- start start 2024-09-19T05:11:08.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.473+0000 7fcd89600640 1 -- 172.21.15.43:0/3800433478 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fcd7c0037a0 con 0x7fcd8c101de0 2024-09-19T05:11:08.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.474+0000 7fcd958a1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d58641aac0 0x55d58641aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:08.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.474+0000 7fcd958a1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d58641b3e0 0x55d586425760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:08.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.474+0000 7fcd958a1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d586425ca0 0x55d586428090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:08.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.474+0000 7fcd958a1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d586381290 con 0x55d586425ca0 2024-09-19T05:11:08.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.474+0000 7fcd958a1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d586328b40 con 0x55d58641b3e0 2024-09-19T05:11:08.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.474+0000 7fcd958a1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d58637d250 con 0x55d58641aac0 2024-09-19T05:11:08.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.474+0000 7fcd83400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d586425ca0 0x55d586428090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:08.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.474+0000 7fcd8be00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d58641b3e0 0x55d586425760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:08.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.474+0000 7fcd90e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d58641aac0 0x55d58641aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:08.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.474+0000 7fcd83400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d586425ca0 0x55d586428090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53070/0 (socket says 172.21.15.43:53070) 2024-09-19T05:11:08.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.474+0000 7fcd83400640 1 -- 172.21.15.43:0/618493822 learned_addr learned my addr 172.21.15.43:0/618493822 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:08.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.474+0000 7fcd83400640 1 -- 172.21.15.43:0/618493822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d58641aac0 msgr2=0x55d58641aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:08.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.474+0000 7fcd83400640 1 --2- 172.21.15.43:0/618493822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d58641aac0 0x55d58641aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:08.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.474+0000 7fcd83400640 1 -- 172.21.15.43:0/618493822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d58641b3e0 msgr2=0x55d586425760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:08.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.474+0000 7fcd83400640 1 --2- 172.21.15.43:0/618493822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d58641b3e0 0x55d586425760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:08.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.474+0000 7fcd83400640 1 -- 172.21.15.43:0/618493822 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5860f26a0 con 0x55d586425ca0 2024-09-19T05:11:08.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.475+0000 7fcd83400640 1 --2- 172.21.15.43:0/618493822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d586425ca0 0x55d586428090 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fcd74009900 tx=0x7fcd74005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=567f697a33cf8c3c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:08.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.475+0000 7fcd81600640 1 -- 172.21.15.43:0/618493822 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd74035040 con 0x55d586425ca0 2024-09-19T05:11:08.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.475+0000 7fcd81600640 1 -- 172.21.15.43:0/618493822 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd74004400 con 0x55d586425ca0 2024-09-19T05:11:08.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.475+0000 7fcd89600640 1 --2- 172.21.15.43:0/3800433478 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fcd700a33b0 0x7fcd700a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:08.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.476+0000 7fcd81600640 1 -- 172.21.15.43:0/618493822 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd740035f0 con 0x55d586425ca0 2024-09-19T05:11:08.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.476+0000 7fcd89600640 1 -- 172.21.15.43:0/3800433478 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fcd7c0cffe0 con 0x7fcd8c101de0 2024-09-19T05:11:08.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.476+0000 7fcd90e00640 1 --2- 172.21.15.43:0/3800433478 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fcd700a33b0 0x7fcd700a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:08.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.476+0000 7fcd958a1740 1 -- 172.21.15.43:0/618493822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d586425ca0 msgr2=0x55d586428090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:08.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.476+0000 7fcd958a1740 1 --2- 172.21.15.43:0/618493822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d586425ca0 0x55d586428090 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fcd74009900 tx=0x7fcd74005de0 comp rx=0 tx=0).stop 2024-09-19T05:11:08.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.476+0000 7fcd958a1740 1 -- 172.21.15.43:0/618493822 shutdown_connections 2024-09-19T05:11:08.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.476+0000 7fcd958a1740 1 --2- 172.21.15.43:0/618493822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d586425ca0 0x55d586428090 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:08.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.476+0000 7fcd958a1740 1 --2- 172.21.15.43:0/618493822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d58641b3e0 0x55d586425760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:08.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.476+0000 7fcd958a1740 1 --2- 172.21.15.43:0/618493822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d58641aac0 0x55d58641aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:08.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.476+0000 7fcd958a1740 1 -- 172.21.15.43:0/618493822 >> 172.21.15.43:0/618493822 conn(0x55d586417050 msgr2=0x55d586417430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:08.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.476+0000 7fcd958a1740 1 -- 172.21.15.43:0/618493822 shutdown_connections 2024-09-19T05:11:08.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.476+0000 7fcd90e00640 1 --2- 172.21.15.43:0/3800433478 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fcd700a33b0 0x7fcd700a5870 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcd84005fd0 tx=0x7fcd84009500 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:08.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.477+0000 7fcd958a1740 1 -- 172.21.15.43:0/618493822 wait complete. 2024-09-19T05:11:08.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.477+0000 7fcd958a1740 1 -- start start 2024-09-19T05:11:08.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.477+0000 7fcd958a1740 1 Processor -- start 2024-09-19T05:11:08.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.478+0000 7fcd958a1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d58641aac0 0x55d5864255e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:08.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.478+0000 7fcd958a1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d58641b3e0 0x55d586422890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:08.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.478+0000 7fcd958a1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d586425ca0 0x55d586422dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:08.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.478+0000 7fcd958a1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d586428960 con 0x55d586425ca0 2024-09-19T05:11:08.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.478+0000 7fcd958a1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d58633bfa0 con 0x55d58641b3e0 2024-09-19T05:11:08.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.478+0000 7fcd958a1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d586428ae0 con 0x55d58641aac0 2024-09-19T05:11:08.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.478+0000 7fcd958a1740 10 client.0 _ll_register_callbacks cb 0x55d58608fba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:08.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.478+0000 7fcd90e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d586425ca0 0x55d586422dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:08.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.478+0000 7fcd8be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d58641aac0 0x55d5864255e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:08.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.478+0000 7fcd90e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d586425ca0 0x55d586422dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53086/0 (socket says 172.21.15.43:53086) 2024-09-19T05:11:08.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.478+0000 7fcd90e00640 1 -- 172.21.15.43:0/293640227 learned_addr learned my addr 172.21.15.43:0/293640227 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:08.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.478+0000 7fcd83400640 1 --2- 172.21.15.43:0/293640227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d58641b3e0 0x55d586422890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:08.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.478+0000 7fcd8be00640 1 -- 172.21.15.43:0/293640227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d58641b3e0 msgr2=0x55d586422890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:08.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.479+0000 7fcd8be00640 1 --2- 172.21.15.43:0/293640227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d58641b3e0 0x55d586422890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:08.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.479+0000 7fcd8be00640 1 -- 172.21.15.43:0/293640227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d586425ca0 msgr2=0x55d586422dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:08.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.479+0000 7fcd8be00640 1 --2- 172.21.15.43:0/293640227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d586425ca0 0x55d586422dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:08.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.479+0000 7fcd8be00640 1 -- 172.21.15.43:0/293640227 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d586423600 con 0x55d58641aac0 2024-09-19T05:11:08.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.479+0000 7fcd83400640 1 --2- 172.21.15.43:0/293640227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d58641b3e0 0x55d586422890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:11:08.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.479+0000 7fcd90e00640 1 --2- 172.21.15.43:0/293640227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d586425ca0 0x55d586422dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:11:08.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.479+0000 7fcd8be00640 1 --2- 172.21.15.43:0/293640227 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d58641aac0 0x55d5864255e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c0cd720 tx=0x7fcd7c0d3600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bb07ef1b1fb9a8b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:08.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.479+0000 7fcd7a000640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:11:08.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.479+0000 7fcd7a000640 1 -- 172.21.15.43:0/293640227 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c09c080 con 0x55d58641aac0 2024-09-19T05:11:08.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.479+0000 7fcd958a1740 1 -- 172.21.15.43:0/293640227 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd74004400 con 0x55d58641aac0 2024-09-19T05:11:08.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.479+0000 7fcd7a000640 1 -- 172.21.15.43:0/293640227 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fcd7c098920 con 0x55d58641aac0 2024-09-19T05:11:08.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.480+0000 7fcd7a000640 1 -- 172.21.15.43:0/293640227 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcd7c09bc10 con 0x55d58641aac0 2024-09-19T05:11:08.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.480+0000 7fcd958a1740 10 client.26599 fetch_fsmap learned FSMap version 29 2024-09-19T05:11:08.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.480+0000 7fcd958a1740 1 -- 172.21.15.43:0/293640227 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d586434870 con 0x55d58641aac0 2024-09-19T05:11:08.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.480+0000 7fcd7a000640 1 -- 172.21.15.43:0/293640227 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd74004400 con 0x55d58641aac0 2024-09-19T05:11:08.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.480+0000 7fcd7a000640 1 -- 172.21.15.43:0/293640227 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcd7c0cd9a0 con 0x55d58641aac0 2024-09-19T05:11:08.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.480+0000 7fcd958a1740 10 client.26599 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:11:08.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.480+0000 7fcd958a1740 10 client.26599 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:11:08.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.480+0000 7fcd958a1740 20 client.26599 populate_metadata read hostname 'smithi043' 2024-09-19T05:11:08.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.480+0000 7fcd958a1740 1 --2- 172.21.15.43:0/293640227 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55d586435d30 0x55d5864381a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:08.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.480+0000 7fcd958a1740 4 client.26599 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:11:08.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.480+0000 7fcd958a1740 1 -- 172.21.15.43:0/293640227 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d58638e800 con 0x55d586435d30 2024-09-19T05:11:08.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.481+0000 7fcd83400640 1 --2- 172.21.15.43:0/293640227 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55d586435d30 0x55d5864381a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:08.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.481+0000 7fcd83400640 1 --2- 172.21.15.43:0/293640227 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55d586435d30 0x55d5864381a0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd39ef20d820d86c server_cookie=960897a4c417aa23 in_seq=0 out_seq=0 2024-09-19T05:11:08.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.481+0000 7fcd7a000640 10 client.26599 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:11:08.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.482+0000 7fcd7a000640 1 -- 172.21.15.43:0/293640227 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d58638e800 con 0x55d586435d30 2024-09-19T05:11:08.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.482+0000 7fcd7a000640 10 client.26599 handle_command_reply: tid=0 2024-09-19T05:11:08.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.482+0000 7fcd958a1740 1 client.26599 shutdown 2024-09-19T05:11:08.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.482+0000 7fcd958a1740 1 -- 172.21.15.43:0/293640227 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d58641aac0 msgr2=0x55d5864255e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:08.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.482+0000 7fcd958a1740 1 --2- 172.21.15.43:0/293640227 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d58641aac0 0x55d5864255e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c0cd720 tx=0x7fcd7c0d3600 comp rx=0 tx=0).stop 2024-09-19T05:11:08.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.482+0000 7fcd958a1740 1 -- 172.21.15.43:0/293640227 shutdown_connections 2024-09-19T05:11:08.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.482+0000 7fcd958a1740 1 --2- 172.21.15.43:0/293640227 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55d586435d30 0x55d5864381a0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:08.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.482+0000 7fcd958a1740 1 --2- 172.21.15.43:0/293640227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d586425ca0 0x55d586422dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:08.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.482+0000 7fcd958a1740 1 --2- 172.21.15.43:0/293640227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d58641b3e0 0x55d586422890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:08.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.482+0000 7fcd958a1740 1 --2- 172.21.15.43:0/293640227 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d58641aac0 0x55d5864255e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:08.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.482+0000 7fcd958a1740 1 -- 172.21.15.43:0/293640227 >> 172.21.15.43:0/293640227 conn(0x55d586417050 msgr2=0x55d5864161c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:08.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.482+0000 7fcd7a000640 0 client.26599 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:11:08.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.482+0000 7fcd958a1740 1 -- 172.21.15.43:0/293640227 shutdown_connections 2024-09-19T05:11:08.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.482+0000 7fcd958a1740 1 -- 172.21.15.43:0/293640227 wait complete. 2024-09-19T05:11:08.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.483+0000 7fcd958a1740 20 client.26599 trim_cache size 0 max 16384 2024-09-19T05:11:08.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.500+0000 7fcd958a1740 1 Processor -- start 2024-09-19T05:11:08.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.501+0000 7fcd958a1740 1 -- start start 2024-09-19T05:11:08.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.501+0000 7fcd958a1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d58641aac0 0x55d58641de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:08.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.501+0000 7fcd958a1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d58641b3e0 0x55d58643da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:08.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.501+0000 7fcd958a1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d586425ca0 0x55d58643dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:08.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.501+0000 7fcd958a1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8c1af9d0 con 0x55d58641b3e0 2024-09-19T05:11:08.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.501+0000 7fcd958a1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8c1b1510 con 0x55d586425ca0 2024-09-19T05:11:08.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.501+0000 7fcd958a1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8c1b00a0 con 0x55d58641aac0 2024-09-19T05:11:08.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.501+0000 7fcd83400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d58641b3e0 0x55d58643da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:08.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.501+0000 7fcd83400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d58641b3e0 0x55d58643da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53098/0 (socket says 172.21.15.43:53098) 2024-09-19T05:11:08.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.501+0000 7fcd83400640 1 -- 172.21.15.43:0/4293045354 learned_addr learned my addr 172.21.15.43:0/4293045354 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:08.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.501+0000 7fcd8be00640 1 --2- 172.21.15.43:0/4293045354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d58641aac0 0x55d58641de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:08.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.501+0000 7fcd90e00640 1 --2- 172.21.15.43:0/4293045354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d586425ca0 0x55d58643dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:08.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.502+0000 7fcd8be00640 1 -- 172.21.15.43:0/4293045354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d586425ca0 msgr2=0x55d58643dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:08.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.502+0000 7fcd8be00640 1 --2- 172.21.15.43:0/4293045354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d586425ca0 0x55d58643dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:08.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.502+0000 7fcd8be00640 1 -- 172.21.15.43:0/4293045354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d58641b3e0 msgr2=0x55d58643da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:08.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.502+0000 7fcd8be00640 1 --2- 172.21.15.43:0/4293045354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d58641b3e0 0x55d58643da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:08.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.502+0000 7fcd8be00640 1 -- 172.21.15.43:0/4293045354 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d586421840 con 0x55d58641aac0 2024-09-19T05:11:08.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.502+0000 7fcd83400640 1 --2- 172.21.15.43:0/4293045354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d58641b3e0 0x55d58643da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:11:08.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.502+0000 7fcd8be00640 1 --2- 172.21.15.43:0/4293045354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d58641aac0 0x55d58641de10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c00c760 tx=0x7fcd7c003d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab9db0f24d23b92d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:08.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.502+0000 7fcd80c00640 1 -- 172.21.15.43:0/4293045354 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c09bdf0 con 0x55d58641aac0 2024-09-19T05:11:08.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.502+0000 7fcd80c00640 1 -- 172.21.15.43:0/4293045354 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd7c0a0660 con 0x55d58641aac0 2024-09-19T05:11:08.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.502+0000 7fcd80c00640 1 -- 172.21.15.43:0/4293045354 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c0d3190 con 0x55d58641aac0 2024-09-19T05:11:08.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.503+0000 7fcd958a1740 1 -- 172.21.15.43:0/4293045354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d58641aac0 msgr2=0x55d58641de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:08.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.503+0000 7fcd958a1740 1 --2- 172.21.15.43:0/4293045354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d58641aac0 0x55d58641de10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c00c760 tx=0x7fcd7c003d40 comp rx=0 tx=0).stop 2024-09-19T05:11:08.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.503+0000 7fcd958a1740 1 -- 172.21.15.43:0/4293045354 shutdown_connections 2024-09-19T05:11:08.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.503+0000 7fcd958a1740 1 --2- 172.21.15.43:0/4293045354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d586425ca0 0x55d58643dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:08.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.503+0000 7fcd958a1740 1 --2- 172.21.15.43:0/4293045354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d58641b3e0 0x55d58643da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:08.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.503+0000 7fcd958a1740 1 --2- 172.21.15.43:0/4293045354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d58641aac0 0x55d58641de10 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:08.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.503+0000 7fcd958a1740 1 -- 172.21.15.43:0/4293045354 >> 172.21.15.43:0/4293045354 conn(0x55d586417050 msgr2=0x55d58641d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:08.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.503+0000 7fcd958a1740 1 -- 172.21.15.43:0/4293045354 shutdown_connections 2024-09-19T05:11:08.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.503+0000 7fcd958a1740 1 -- 172.21.15.43:0/4293045354 wait complete. 2024-09-19T05:11:08.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.503+0000 7fcd958a1740 1 -- start start 2024-09-19T05:11:08.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.504+0000 7fcd958a1740 1 Processor -- start 2024-09-19T05:11:08.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.504+0000 7fcd958a1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d58641aac0 0x55d586429c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:08.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.504+0000 7fcd958a1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d58641b3e0 0x55d58642a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:08.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.504+0000 7fcd958a1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d586425ca0 0x55d58642a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:08.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.504+0000 7fcd958a1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8c1b0e40 con 0x55d58641b3e0 2024-09-19T05:11:08.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.504+0000 7fcd958a1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8c1b0770 con 0x55d586425ca0 2024-09-19T05:11:08.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.504+0000 7fcd958a1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8c1ad9b0 con 0x55d58641aac0 2024-09-19T05:11:08.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.504+0000 7fcd958a1740 10 client.0 _ll_register_callbacks cb 0x7fcd7c00e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:08.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.504+0000 7fcd8be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d58641aac0 0x55d586429c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:08.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.504+0000 7fcd83400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d58641b3e0 0x55d58642a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:08.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.504+0000 7fcd8be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d58641aac0 0x55d586429c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:55406/0 (socket says 172.21.15.43:55406) 2024-09-19T05:11:08.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.504+0000 7fcd8be00640 1 -- 172.21.15.43:0/1899748198 learned_addr learned my addr 172.21.15.43:0/1899748198 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:08.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.504+0000 7fcd90e00640 1 --2- 172.21.15.43:0/1899748198 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d586425ca0 0x55d58642a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:08.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.505+0000 7fcd83400640 1 -- 172.21.15.43:0/1899748198 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d58641aac0 msgr2=0x55d586429c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:08.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.505+0000 7fcd83400640 1 --2- 172.21.15.43:0/1899748198 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d58641aac0 0x55d586429c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:08.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.505+0000 7fcd83400640 1 -- 172.21.15.43:0/1899748198 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d586425ca0 msgr2=0x55d58642a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:08.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.505+0000 7fcd83400640 1 --2- 172.21.15.43:0/1899748198 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d586425ca0 0x55d58642a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:08.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.505+0000 7fcd83400640 1 -- 172.21.15.43:0/1899748198 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d58649fbe0 con 0x55d58641b3e0 2024-09-19T05:11:08.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.505+0000 7fcd83400640 1 --2- 172.21.15.43:0/1899748198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d58641b3e0 0x55d58642a150 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fcd74004b50 tx=0x7fcd7402a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d50bc3da9ab382e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:08.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.505+0000 7fcd7a000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:11:08.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.505+0000 7fcd7a000640 1 -- 172.21.15.43:0/1899748198 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd74035040 con 0x55d58641b3e0 2024-09-19T05:11:08.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.505+0000 7fcd7a000640 1 -- 172.21.15.43:0/1899748198 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fcd740315b0 con 0x55d58641b3e0 2024-09-19T05:11:08.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.505+0000 7fcd958a1740 1 -- 172.21.15.43:0/1899748198 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d586388d10 con 0x55d58641b3e0 2024-09-19T05:11:08.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.506+0000 7fcd7a000640 1 -- 172.21.15.43:0/1899748198 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcd74004920 con 0x55d58641b3e0 2024-09-19T05:11:08.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.506+0000 7fcd958a1740 10 client.16647 fetch_fsmap learned FSMap version 29 2024-09-19T05:11:08.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.506+0000 7fcd958a1740 1 -- 172.21.15.43:0/1899748198 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d5864a0f40 con 0x55d58641b3e0 2024-09-19T05:11:08.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.506+0000 7fcd7a000640 1 -- 172.21.15.43:0/1899748198 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d586388d10 con 0x55d58641b3e0 2024-09-19T05:11:08.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.506+0000 7fcd7a000640 1 -- 172.21.15.43:0/1899748198 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcd74003b90 con 0x55d58641b3e0 2024-09-19T05:11:08.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.506+0000 7fcd958a1740 10 client.16647 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:11:08.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.506+0000 7fcd958a1740 10 client.16647 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:11:08.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.506+0000 7fcd958a1740 20 client.16647 populate_metadata read hostname 'smithi043' 2024-09-19T05:11:08.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.506+0000 7fcd958a1740 1 --2- 172.21.15.43:0/1899748198 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55d5864a1340 0x55d5864a3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:08.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.506+0000 7fcd958a1740 4 client.16647 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T05:11:08.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.506+0000 7fcd958a1740 1 -- 172.21.15.43:0/1899748198 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d586304fb0 con 0x55d5864a1340 2024-09-19T05:11:08.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.507+0000 7fcd8be00640 1 --2- 172.21.15.43:0/1899748198 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55d5864a1340 0x55d5864a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:08.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.507+0000 7fcd8be00640 1 --2- 172.21.15.43:0/1899748198 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55d5864a1340 0x55d5864a3730 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf3b6b984f3f88a4 server_cookie=7dff0072e8f2c3ba in_seq=0 out_seq=0 2024-09-19T05:11:08.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.507+0000 7fcd7a000640 10 client.16647 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:11:08.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.508+0000 7fcd7a000640 1 -- 172.21.15.43:0/1899748198 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d586304fb0 con 0x55d5864a1340 2024-09-19T05:11:08.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.508+0000 7fcd7a000640 10 client.16647 handle_command_reply: tid=0 2024-09-19T05:11:08.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.508+0000 7fcd958a1740 1 client.16647 shutdown 2024-09-19T05:11:08.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.508+0000 7fcd958a1740 1 -- 172.21.15.43:0/1899748198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d58641b3e0 msgr2=0x55d58642a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:08.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.508+0000 7fcd958a1740 1 --2- 172.21.15.43:0/1899748198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d58641b3e0 0x55d58642a150 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fcd74004b50 tx=0x7fcd7402a3f0 comp rx=0 tx=0).stop 2024-09-19T05:11:08.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.509+0000 7fcd958a1740 1 -- 172.21.15.43:0/1899748198 shutdown_connections 2024-09-19T05:11:08.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.509+0000 7fcd958a1740 1 --2- 172.21.15.43:0/1899748198 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55d5864a1340 0x55d5864a3730 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:08.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.509+0000 7fcd958a1740 1 --2- 172.21.15.43:0/1899748198 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d586425ca0 0x55d58642a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:08.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.509+0000 7fcd958a1740 1 --2- 172.21.15.43:0/1899748198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d58641b3e0 0x55d58642a150 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:08.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.509+0000 7fcd958a1740 1 --2- 172.21.15.43:0/1899748198 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d58641aac0 0x55d586429c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:08.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.509+0000 7fcd958a1740 1 -- 172.21.15.43:0/1899748198 >> 172.21.15.43:0/1899748198 conn(0x55d586417050 msgr2=0x55d5864161c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:08.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.509+0000 7fcd7a000640 0 client.16647 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:11:08.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.509+0000 7fcd958a1740 1 -- 172.21.15.43:0/1899748198 shutdown_connections 2024-09-19T05:11:08.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.509+0000 7fcd958a1740 1 -- 172.21.15.43:0/1899748198 wait complete. 2024-09-19T05:11:08.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.509+0000 7fcd958a1740 20 client.16647 trim_cache size 0 max 16384 2024-09-19T05:11:08.515 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:11:08.515 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-09-19T05:11:08.515 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "5ef37815-b0da-4d8a-8070-96de311b047a", 2024-09-19T05:11:08.515 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-09-19T05:11:08.515 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:11:08.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.510+0000 7fcd92e00640 1 -- 172.21.15.43:0/3800433478 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fcd700a33b0 msgr2=0x7fcd700a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:08.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.510+0000 7fcd92e00640 1 --2- 172.21.15.43:0/3800433478 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fcd700a33b0 0x7fcd700a5870 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcd84005fd0 tx=0x7fcd84009500 comp rx=0 tx=0).stop 2024-09-19T05:11:08.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.510+0000 7fcd92e00640 1 -- 172.21.15.43:0/3800433478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd8c101de0 msgr2=0x7fcd8c19a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:08.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.510+0000 7fcd92e00640 1 --2- 172.21.15.43:0/3800433478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd8c101de0 0x7fcd8c19a690 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c00ea60 tx=0x7fcd7c00ef20 comp rx=0 tx=0).stop 2024-09-19T05:11:08.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.510+0000 7fcd92e00640 1 -- 172.21.15.43:0/3800433478 shutdown_connections 2024-09-19T05:11:08.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.510+0000 7fcd92e00640 1 --2- 172.21.15.43:0/3800433478 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd8c106f10 0x7fcd8c19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:08.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.510+0000 7fcd92e00640 1 --2- 172.21.15.43:0/3800433478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd8c102790 0x7fcd8c19abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:08.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.510+0000 7fcd92e00640 1 --2- 172.21.15.43:0/3800433478 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fcd700a33b0 0x7fcd700a5870 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:08.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.510+0000 7fcd92e00640 1 --2- 172.21.15.43:0/3800433478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd8c101de0 0x7fcd8c19a690 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:08.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.510+0000 7fcd92e00640 1 -- 172.21.15.43:0/3800433478 >> 172.21.15.43:0/3800433478 conn(0x7fcd8c0fd8c0 msgr2=0x7fcd8c0ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:08.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.510+0000 7fcd92e00640 1 -- 172.21.15.43:0/3800433478 shutdown_connections 2024-09-19T05:11:08.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:08.510+0000 7fcd92e00640 1 -- 172.21.15.43:0/3800433478 wait complete. 2024-09-19T05:11:09.354 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:11:09.734 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:11:10.185 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:11:09 smithi043 ceph-mon[30800]: pgmap v198: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-09-19T05:11:10.185 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:11:09 smithi043 ceph-mon[30800]: scrub queued for path: / 2024-09-19T05:11:10.185 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:11:09 smithi043 ceph-mon[30800]: scrub summary: idle+waiting paths [/] 2024-09-19T05:11:10.185 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:11:09 smithi043 ceph-mon[30800]: scrub summary: active paths [/] 2024-09-19T05:11:10.185 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:11:09 smithi043 ceph-mon[30800]: scrub summary: idle+waiting paths [/] 2024-09-19T05:11:10.185 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:11:09 smithi043 ceph-mon[30800]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T05:11:10.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:11:09 smithi096 ceph-mon[35085]: pgmap v198: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-09-19T05:11:10.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:11:09 smithi096 ceph-mon[35085]: scrub queued for path: / 2024-09-19T05:11:10.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:11:09 smithi096 ceph-mon[35085]: scrub summary: idle+waiting paths [/] 2024-09-19T05:11:10.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:11:09 smithi096 ceph-mon[35085]: scrub summary: active paths [/] 2024-09-19T05:11:10.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:11:09 smithi096 ceph-mon[35085]: scrub summary: idle+waiting paths [/] 2024-09-19T05:11:10.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:11:09 smithi096 ceph-mon[35085]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T05:11:10.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.329+0000 7fdb87800640 1 -- 172.21.15.43:0/3740951658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb80076720 msgr2=0x7fdb80076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:10.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.329+0000 7fdb87800640 1 --2- 172.21.15.43:0/3740951658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb80076720 0x7fdb80076b00 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fdb74009900 tx=0x7fdb7402a410 comp rx=0 tx=0).stop 2024-09-19T05:11:10.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.330+0000 7fdb87800640 1 -- 172.21.15.43:0/3740951658 shutdown_connections 2024-09-19T05:11:10.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.330+0000 7fdb87800640 1 --2- 172.21.15.43:0/3740951658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb80112a50 0x7fdb80114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:10.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.330+0000 7fdb87800640 1 --2- 172.21.15.43:0/3740951658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb80077040 0x7fdb80112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:10.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.330+0000 7fdb87800640 1 --2- 172.21.15.43:0/3740951658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb80076720 0x7fdb80076b00 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:10.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.330+0000 7fdb87800640 1 -- 172.21.15.43:0/3740951658 >> 172.21.15.43:0/3740951658 conn(0x7fdb800ffa50 msgr2=0x7fdb80101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:10.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.330+0000 7fdb87800640 1 -- 172.21.15.43:0/3740951658 shutdown_connections 2024-09-19T05:11:10.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.330+0000 7fdb87800640 1 -- 172.21.15.43:0/3740951658 wait complete. 2024-09-19T05:11:10.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.330+0000 7fdb87800640 1 Processor -- start 2024-09-19T05:11:10.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.331+0000 7fdb87800640 1 -- start start 2024-09-19T05:11:10.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.331+0000 7fdb87800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb80076720 0x7fdb801a4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:10.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.331+0000 7fdb87800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb80077040 0x7fdb801a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:10.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.331+0000 7fdb87800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb80112a50 0x7fdb8019e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:10.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.331+0000 7fdb87800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdb80118630 con 0x7fdb80076720 2024-09-19T05:11:10.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.331+0000 7fdb87800640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdb801184b0 con 0x7fdb80077040 2024-09-19T05:11:10.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.331+0000 7fdb87800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdb801187b0 con 0x7fdb80112a50 2024-09-19T05:11:10.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.331+0000 7fdb84e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb80076720 0x7fdb801a4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:10.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.331+0000 7fdb84e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb80076720 0x7fdb801a4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53128/0 (socket says 172.21.15.43:53128) 2024-09-19T05:11:10.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.331+0000 7fdb84e00640 1 -- 172.21.15.43:0/2846493492 learned_addr learned my addr 172.21.15.43:0/2846493492 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:10.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.331+0000 7fdb85800640 1 --2- 172.21.15.43:0/2846493492 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb80112a50 0x7fdb8019e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:10.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.331+0000 7fdb7fe00640 1 --2- 172.21.15.43:0/2846493492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb80077040 0x7fdb801a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:10.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.332+0000 7fdb84e00640 1 -- 172.21.15.43:0/2846493492 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb80112a50 msgr2=0x7fdb8019e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:10.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.332+0000 7fdb84e00640 1 --2- 172.21.15.43:0/2846493492 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb80112a50 0x7fdb8019e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:10.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.332+0000 7fdb84e00640 1 -- 172.21.15.43:0/2846493492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb80077040 msgr2=0x7fdb801a4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:10.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.332+0000 7fdb84e00640 1 --2- 172.21.15.43:0/2846493492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb80077040 0x7fdb801a4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:10.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.332+0000 7fdb84e00640 1 -- 172.21.15.43:0/2846493492 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb8019f110 con 0x7fdb80076720 2024-09-19T05:11:10.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.332+0000 7fdb84e00640 1 --2- 172.21.15.43:0/2846493492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb80076720 0x7fdb801a4820 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fdb74004770 tx=0x7fdb74003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:10.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.332+0000 7fdb7d600640 1 -- 172.21.15.43:0/2846493492 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7403f070 con 0x7fdb80076720 2024-09-19T05:11:10.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.332+0000 7fdb7d600640 1 -- 172.21.15.43:0/2846493492 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb74031ca0 con 0x7fdb80076720 2024-09-19T05:11:10.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.332+0000 7fdb87800640 1 -- 172.21.15.43:0/2846493492 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb8019f3a0 con 0x7fdb80076720 2024-09-19T05:11:10.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.332+0000 7fdb7d600640 1 -- 172.21.15.43:0/2846493492 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb74035030 con 0x7fdb80076720 2024-09-19T05:11:10.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.333+0000 7fdb87800640 1 -- 172.21.15.43:0/2846493492 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb801ab6f0 con 0x7fdb80076720 2024-09-19T05:11:10.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.336+0000 7fdb8a3ef740 1 Processor -- start 2024-09-19T05:11:10.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.336+0000 7fdb7d600640 1 -- 172.21.15.43:0/2846493492 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fdb74042070 con 0x7fdb80076720 2024-09-19T05:11:10.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.336+0000 7fdb8a3ef740 1 -- start start 2024-09-19T05:11:10.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.336+0000 7fdb8a3ef740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cab8a5cb90 0x55cab8a5cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:10.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.336+0000 7fdb8a3ef740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cab8a5d4b0 0x55cab8a67830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:10.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.336+0000 7fdb8a3ef740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cab8a67d70 0x55cab8a6a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:10.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.336+0000 7fdb8a3ef740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cab89c33e0 con 0x55cab8a5d4b0 2024-09-19T05:11:10.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.336+0000 7fdb8a3ef740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55cab896ad80 con 0x55cab8a67d70 2024-09-19T05:11:10.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.336+0000 7fdb8a3ef740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55cab89bf3a0 con 0x55cab8a5cb90 2024-09-19T05:11:10.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.337+0000 7fdb84e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cab8a5d4b0 0x55cab8a67830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:10.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.337+0000 7fdb85800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cab8a5cb90 0x55cab8a5cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:10.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.337+0000 7fdb84e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cab8a5d4b0 0x55cab8a67830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53134/0 (socket says 172.21.15.43:53134) 2024-09-19T05:11:10.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.337+0000 7fdb84e00640 1 -- 172.21.15.43:0/3283244474 learned_addr learned my addr 172.21.15.43:0/3283244474 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:10.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.337+0000 7fdb7fe00640 1 --2- 172.21.15.43:0/3283244474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cab8a67d70 0x55cab8a6a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:10.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.337+0000 7fdb85800640 1 -- 172.21.15.43:0/3283244474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cab8a67d70 msgr2=0x55cab8a6a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:10.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.337+0000 7fdb85800640 1 --2- 172.21.15.43:0/3283244474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cab8a67d70 0x55cab8a6a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:10.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.337+0000 7fdb85800640 1 -- 172.21.15.43:0/3283244474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cab8a5d4b0 msgr2=0x55cab8a67830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:10.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.337+0000 7fdb85800640 1 --2- 172.21.15.43:0/3283244474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cab8a5d4b0 0x55cab8a67830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:10.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.337+0000 7fdb85800640 1 -- 172.21.15.43:0/3283244474 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cab8989ae0 con 0x55cab8a5cb90 2024-09-19T05:11:10.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.338+0000 7fdb85800640 1 --2- 172.21.15.43:0/3283244474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cab8a5cb90 0x55cab8a5cf70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdb8007aa10 tx=0x7fdb78011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c4e81b39b115b31 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:10.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.338+0000 7fdb7d600640 1 --2- 172.21.15.43:0/2846493492 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fdb680a3430 0x7fdb680a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:10.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.338+0000 7fdb72000640 1 -- 172.21.15.43:0/3283244474 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb78004d20 con 0x55cab8a5cb90 2024-09-19T05:11:10.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.338+0000 7fdb72000640 1 -- 172.21.15.43:0/3283244474 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb78004050 con 0x55cab8a5cb90 2024-09-19T05:11:10.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.338+0000 7fdb72000640 1 -- 172.21.15.43:0/3283244474 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb78005280 con 0x55cab8a5cb90 2024-09-19T05:11:10.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.338+0000 7fdb7d600640 1 -- 172.21.15.43:0/2846493492 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8586+0+0 (secure 0 0 0) 0x7fdb740f02d0 con 0x7fdb80076720 2024-09-19T05:11:10.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.338+0000 7fdb84e00640 1 --2- 172.21.15.43:0/2846493492 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fdb680a3430 0x7fdb680a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:10.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.339+0000 7fdb8a3ef740 1 -- 172.21.15.43:0/3283244474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cab8a5cb90 msgr2=0x55cab8a5cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:10.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.339+0000 7fdb84e00640 1 --2- 172.21.15.43:0/2846493492 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fdb680a3430 0x7fdb680a58f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdb740f0570 tx=0x7fdb740f3d80 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:10.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.339+0000 7fdb8a3ef740 1 --2- 172.21.15.43:0/3283244474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cab8a5cb90 0x55cab8a5cf70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdb8007aa10 tx=0x7fdb78011810 comp rx=0 tx=0).stop 2024-09-19T05:11:10.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.339+0000 7fdb8a3ef740 1 -- 172.21.15.43:0/3283244474 shutdown_connections 2024-09-19T05:11:10.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.339+0000 7fdb8a3ef740 1 --2- 172.21.15.43:0/3283244474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cab8a67d70 0x55cab8a6a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:10.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.339+0000 7fdb8a3ef740 1 --2- 172.21.15.43:0/3283244474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cab8a5d4b0 0x55cab8a67830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:10.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.339+0000 7fdb8a3ef740 1 --2- 172.21.15.43:0/3283244474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cab8a5cb90 0x55cab8a5cf70 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:10.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.339+0000 7fdb8a3ef740 1 -- 172.21.15.43:0/3283244474 >> 172.21.15.43:0/3283244474 conn(0x55cab8a59120 msgr2=0x55cab8a59500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:10.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.339+0000 7fdb8a3ef740 1 -- 172.21.15.43:0/3283244474 shutdown_connections 2024-09-19T05:11:10.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.339+0000 7fdb8a3ef740 1 -- 172.21.15.43:0/3283244474 wait complete. 2024-09-19T05:11:10.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.340+0000 7fdb8a3ef740 1 -- start start 2024-09-19T05:11:10.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.340+0000 7fdb8a3ef740 1 Processor -- start 2024-09-19T05:11:10.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.340+0000 7fdb8a3ef740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cab8a5cb90 0x55cab8a64d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:10.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.340+0000 7fdb8a3ef740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cab8a5d4b0 0x55cab8a65280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:10.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.340+0000 7fdb8a3ef740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cab8a67d70 0x55cab8a657c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:10.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.340+0000 7fdb8a3ef740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cab8a6a9e0 con 0x55cab8a5cb90 2024-09-19T05:11:10.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.340+0000 7fdb8a3ef740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55cab897e0f0 con 0x55cab8a5d4b0 2024-09-19T05:11:10.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.340+0000 7fdb8a3ef740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55cab8a6ab60 con 0x55cab8a67d70 2024-09-19T05:11:10.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.340+0000 7fdb8a3ef740 10 client.0 _ll_register_callbacks cb 0x55cab896f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:10.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.341+0000 7fdb85800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cab8a5cb90 0x55cab8a64d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:10.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.341+0000 7fdb85800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cab8a5cb90 0x55cab8a64d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53144/0 (socket says 172.21.15.43:53144) 2024-09-19T05:11:10.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.341+0000 7fdb85800640 1 -- 172.21.15.43:0/2861726269 learned_addr learned my addr 172.21.15.43:0/2861726269 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:10.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.341+0000 7fdb7fe00640 1 --2- 172.21.15.43:0/2861726269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cab8a5d4b0 0x55cab8a65280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:10.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.341+0000 7fdb85800640 1 --2- 172.21.15.43:0/2861726269 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cab8a67d70 0x55cab8a657c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:10.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.341+0000 7fdb7fe00640 1 -- 172.21.15.43:0/2861726269 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cab8a67d70 msgr2=0x55cab8a657c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:10.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.341+0000 7fdb7fe00640 1 --2- 172.21.15.43:0/2861726269 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cab8a67d70 0x55cab8a657c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:10.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.341+0000 7fdb7fe00640 1 -- 172.21.15.43:0/2861726269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cab8a5cb90 msgr2=0x55cab8a64d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:10.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.341+0000 7fdb7fe00640 1 --2- 172.21.15.43:0/2861726269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cab8a5cb90 0x55cab8a64d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:10.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.341+0000 7fdb7fe00640 1 -- 172.21.15.43:0/2861726269 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cab87343c0 con 0x55cab8a5d4b0 2024-09-19T05:11:10.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.342+0000 7fdb7fe00640 1 --2- 172.21.15.43:0/2861726269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cab8a5d4b0 0x55cab8a65280 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdb800792b0 tx=0x7fdb6c012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c145c826939ae55e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:10.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.342+0000 7fdb66a00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:11:10.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.342+0000 7fdb66a00640 1 -- 172.21.15.43:0/2861726269 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c004020 con 0x55cab8a5d4b0 2024-09-19T05:11:10.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.342+0000 7fdb66a00640 1 -- 172.21.15.43:0/2861726269 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8586+0+0 (secure 0 0 0) 0x7fdb6c0041c0 con 0x55cab8a5d4b0 2024-09-19T05:11:10.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.342+0000 7fdb8a3ef740 1 -- 172.21.15.43:0/2861726269 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb78004050 con 0x55cab8a5d4b0 2024-09-19T05:11:10.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.342+0000 7fdb66a00640 1 -- 172.21.15.43:0/2861726269 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdb6c0043c0 con 0x55cab8a5d4b0 2024-09-19T05:11:10.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.342+0000 7fdb8a3ef740 10 client.26687 fetch_fsmap learned FSMap version 29 2024-09-19T05:11:10.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.342+0000 7fdb8a3ef740 1 -- 172.21.15.43:0/2861726269 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cab8a767c0 con 0x55cab8a5d4b0 2024-09-19T05:11:10.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.343+0000 7fdb66a00640 1 -- 172.21.15.43:0/2861726269 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb78004050 con 0x55cab8a5d4b0 2024-09-19T05:11:10.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.343+0000 7fdb66a00640 1 -- 172.21.15.43:0/2861726269 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdb6c003b80 con 0x55cab8a5d4b0 2024-09-19T05:11:10.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.343+0000 7fdb8a3ef740 10 client.26687 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:11:10.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.343+0000 7fdb8a3ef740 10 client.26687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:11:10.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.343+0000 7fdb8a3ef740 20 client.26687 populate_metadata read hostname 'smithi043' 2024-09-19T05:11:10.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.343+0000 7fdb8a3ef740 1 --2- 172.21.15.43:0/2861726269 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55cab8a77c80 0x55cab8a7a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:10.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.343+0000 7fdb8a3ef740 4 client.26687 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:11:10.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.343+0000 7fdb8a3ef740 1 -- 172.21.15.43:0/2861726269 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cab89d0950 con 0x55cab8a77c80 2024-09-19T05:11:10.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.343+0000 7fdb85800640 1 --2- 172.21.15.43:0/2861726269 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55cab8a77c80 0x55cab8a7a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:10.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.344+0000 7fdb85800640 1 --2- 172.21.15.43:0/2861726269 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55cab8a77c80 0x55cab8a7a090 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b3bde89022ecc30 server_cookie=e4a8e2a204ce2583 in_seq=0 out_seq=0 2024-09-19T05:11:10.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.344+0000 7fdb66a00640 10 client.26687 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:11:10.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.344+0000 7fdb66a00640 1 -- 172.21.15.43:0/2861726269 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cab89d0950 con 0x55cab8a77c80 2024-09-19T05:11:10.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.344+0000 7fdb66a00640 10 client.26687 handle_command_reply: tid=0 2024-09-19T05:11:10.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.345+0000 7fdb8a3ef740 1 client.26687 shutdown 2024-09-19T05:11:10.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.345+0000 7fdb8a3ef740 1 -- 172.21.15.43:0/2861726269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cab8a5d4b0 msgr2=0x55cab8a65280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:10.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.345+0000 7fdb8a3ef740 1 --2- 172.21.15.43:0/2861726269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cab8a5d4b0 0x55cab8a65280 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdb800792b0 tx=0x7fdb6c012b60 comp rx=0 tx=0).stop 2024-09-19T05:11:10.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.345+0000 7fdb8a3ef740 1 -- 172.21.15.43:0/2861726269 shutdown_connections 2024-09-19T05:11:10.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.345+0000 7fdb8a3ef740 1 --2- 172.21.15.43:0/2861726269 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55cab8a77c80 0x55cab8a7a090 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:10.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.345+0000 7fdb8a3ef740 1 --2- 172.21.15.43:0/2861726269 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cab8a67d70 0x55cab8a657c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:10.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.345+0000 7fdb8a3ef740 1 --2- 172.21.15.43:0/2861726269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cab8a5d4b0 0x55cab8a65280 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:10.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.345+0000 7fdb8a3ef740 1 --2- 172.21.15.43:0/2861726269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cab8a5cb90 0x55cab8a64d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:10.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.345+0000 7fdb66a00640 0 client.26687 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:11:10.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.345+0000 7fdb8a3ef740 1 -- 172.21.15.43:0/2861726269 >> 172.21.15.43:0/2861726269 conn(0x55cab8a59120 msgr2=0x55cab8a5ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:10.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.345+0000 7fdb8a3ef740 1 -- 172.21.15.43:0/2861726269 shutdown_connections 2024-09-19T05:11:10.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.345+0000 7fdb8a3ef740 1 -- 172.21.15.43:0/2861726269 wait complete. 2024-09-19T05:11:10.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.345+0000 7fdb8a3ef740 20 client.26687 trim_cache size 0 max 16384 2024-09-19T05:11:10.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.363+0000 7fdb8a3ef740 1 Processor -- start 2024-09-19T05:11:10.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.363+0000 7fdb8a3ef740 1 -- start start 2024-09-19T05:11:10.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.364+0000 7fdb8a3ef740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cab8a5cb90 0x55cab8a7f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:10.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.364+0000 7fdb8a3ef740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cab8a5d4b0 0x55cab8a7fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:10.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.364+0000 7fdb8a3ef740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cab8a67d70 0x55cab8a61fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:10.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.364+0000 7fdb8a3ef740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdb801b4030 con 0x55cab8a67d70 2024-09-19T05:11:10.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.364+0000 7fdb8a3ef740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdb801b5b70 con 0x55cab8a5cb90 2024-09-19T05:11:10.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.364+0000 7fdb8a3ef740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdb801b4700 con 0x55cab8a5d4b0 2024-09-19T05:11:10.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.364+0000 7fdb85800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cab8a5cb90 0x55cab8a7f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:10.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.364+0000 7fdb85800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cab8a5cb90 0x55cab8a7f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:45964/0 (socket says 172.21.15.43:45964) 2024-09-19T05:11:10.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.364+0000 7fdb85800640 1 -- 172.21.15.43:0/2170085243 learned_addr learned my addr 172.21.15.43:0/2170085243 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:10.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.364+0000 7fdb7fe00640 1 --2- 172.21.15.43:0/2170085243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cab8a5d4b0 0x55cab8a7fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:10.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.364+0000 7fdb85800640 1 --2- 172.21.15.43:0/2170085243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cab8a67d70 0x55cab8a61fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:10.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.364+0000 7fdb85800640 1 -- 172.21.15.43:0/2170085243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cab8a5d4b0 msgr2=0x55cab8a7fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:10.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.364+0000 7fdb85800640 1 --2- 172.21.15.43:0/2170085243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cab8a5d4b0 0x55cab8a7fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:10.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.364+0000 7fdb85800640 1 -- 172.21.15.43:0/2170085243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cab8a67d70 msgr2=0x55cab8a61fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:10.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.364+0000 7fdb85800640 1 --2- 172.21.15.43:0/2170085243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cab8a67d70 0x55cab8a61fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:10.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.364+0000 7fdb85800640 1 -- 172.21.15.43:0/2170085243 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cab8a59db0 con 0x55cab8a5cb90 2024-09-19T05:11:10.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.365+0000 7fdb85800640 1 --2- 172.21.15.43:0/2170085243 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cab8a5cb90 0x55cab8a7f9a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fdb78009320 tx=0x7fdb78004780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=497680725ffd0a8d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:10.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.365+0000 7fdb71600640 1 -- 172.21.15.43:0/2170085243 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7801d040 con 0x55cab8a5cb90 2024-09-19T05:11:10.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.365+0000 7fdb71600640 1 -- 172.21.15.43:0/2170085243 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb78019910 con 0x55cab8a5cb90 2024-09-19T05:11:10.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.365+0000 7fdb71600640 1 -- 172.21.15.43:0/2170085243 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb78027820 con 0x55cab8a5cb90 2024-09-19T05:11:10.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.366+0000 7fdb8a3ef740 1 -- 172.21.15.43:0/2170085243 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cab8a5cb90 msgr2=0x55cab8a7f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:10.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.366+0000 7fdb8a3ef740 1 --2- 172.21.15.43:0/2170085243 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cab8a5cb90 0x55cab8a7f9a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fdb78009320 tx=0x7fdb78004780 comp rx=0 tx=0).stop 2024-09-19T05:11:10.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.366+0000 7fdb8a3ef740 1 -- 172.21.15.43:0/2170085243 shutdown_connections 2024-09-19T05:11:10.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.366+0000 7fdb8a3ef740 1 --2- 172.21.15.43:0/2170085243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cab8a67d70 0x55cab8a61fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:10.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.366+0000 7fdb8a3ef740 1 --2- 172.21.15.43:0/2170085243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cab8a5d4b0 0x55cab8a7fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:10.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.366+0000 7fdb8a3ef740 1 --2- 172.21.15.43:0/2170085243 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cab8a5cb90 0x55cab8a7f9a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:10.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.366+0000 7fdb8a3ef740 1 -- 172.21.15.43:0/2170085243 >> 172.21.15.43:0/2170085243 conn(0x55cab8a59120 msgr2=0x55cab8a5ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:10.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.366+0000 7fdb8a3ef740 1 -- 172.21.15.43:0/2170085243 shutdown_connections 2024-09-19T05:11:10.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.366+0000 7fdb8a3ef740 1 -- 172.21.15.43:0/2170085243 wait complete. 2024-09-19T05:11:10.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.366+0000 7fdb8a3ef740 1 -- start start 2024-09-19T05:11:10.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.366+0000 7fdb8a3ef740 1 Processor -- start 2024-09-19T05:11:10.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.367+0000 7fdb8a3ef740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cab8a5cb90 0x55cab8a6ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:10.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.367+0000 7fdb8a3ef740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cab8a5d4b0 0x55cab8a704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:10.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.367+0000 7fdb8a3ef740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cab8a67d70 0x55cab8a6cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:10.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.367+0000 7fdb8a3ef740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdb801b54a0 con 0x55cab8a5d4b0 2024-09-19T05:11:10.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.367+0000 7fdb8a3ef740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdb801b4dd0 con 0x55cab8a67d70 2024-09-19T05:11:10.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.367+0000 7fdb8a3ef740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdb801b2030 con 0x55cab8a5cb90 2024-09-19T05:11:10.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.367+0000 7fdb8a3ef740 10 client.0 _ll_register_callbacks cb 0x7fdb6c009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:10.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.367+0000 7fdb85800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cab8a5cb90 0x55cab8a6ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:10.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.367+0000 7fdb85800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cab8a5cb90 0x55cab8a6ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:55450/0 (socket says 172.21.15.43:55450) 2024-09-19T05:11:10.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.367+0000 7fdb85800640 1 -- 172.21.15.43:0/3406611 learned_addr learned my addr 172.21.15.43:0/3406611 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:10.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.367+0000 7fdb85800640 1 --2- 172.21.15.43:0/3406611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cab8a67d70 0x55cab8a6cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:10.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.367+0000 7fdb7fe00640 1 --2- 172.21.15.43:0/3406611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cab8a5d4b0 0x55cab8a704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:10.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.368+0000 7fdb85800640 1 -- 172.21.15.43:0/3406611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cab8a67d70 msgr2=0x55cab8a6cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:10.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.368+0000 7fdb85800640 1 --2- 172.21.15.43:0/3406611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cab8a67d70 0x55cab8a6cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:10.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.368+0000 7fdb85800640 1 -- 172.21.15.43:0/3406611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cab8a5d4b0 msgr2=0x55cab8a704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:10.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.368+0000 7fdb85800640 1 --2- 172.21.15.43:0/3406611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cab8a5d4b0 0x55cab8a704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:10.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.368+0000 7fdb85800640 1 -- 172.21.15.43:0/3406611 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cab8a6d4c0 con 0x55cab8a5cb90 2024-09-19T05:11:10.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.368+0000 7fdb85800640 1 --2- 172.21.15.43:0/3406611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cab8a5cb90 0x55cab8a6ff90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdb78009c90 tx=0x7fdb78027400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21013898aa510abc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:10.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.368+0000 7fdb66a00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:11:10.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.368+0000 7fdb66a00640 1 -- 172.21.15.43:0/3406611 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7801d040 con 0x55cab8a5cb90 2024-09-19T05:11:10.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.368+0000 7fdb66a00640 1 -- 172.21.15.43:0/3406611 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8586+0+0 (secure 0 0 0) 0x7fdb78019430 con 0x55cab8a5cb90 2024-09-19T05:11:10.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.368+0000 7fdb8a3ef740 1 -- 172.21.15.43:0/3406611 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb78019910 con 0x55cab8a5cb90 2024-09-19T05:11:10.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.369+0000 7fdb66a00640 1 -- 172.21.15.43:0/3406611 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdb78019630 con 0x55cab8a5cb90 2024-09-19T05:11:10.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.369+0000 7fdb8a3ef740 10 client.26623 fetch_fsmap learned FSMap version 29 2024-09-19T05:11:10.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.369+0000 7fdb8a3ef740 1 -- 172.21.15.43:0/3406611 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cab8ae33d0 con 0x55cab8a5cb90 2024-09-19T05:11:10.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.369+0000 7fdb66a00640 1 -- 172.21.15.43:0/3406611 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb78019910 con 0x55cab8a5cb90 2024-09-19T05:11:10.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.369+0000 7fdb66a00640 1 -- 172.21.15.43:0/3406611 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdb780434e0 con 0x55cab8a5cb90 2024-09-19T05:11:10.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.369+0000 7fdb8a3ef740 10 client.26623 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:11:10.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.369+0000 7fdb8a3ef740 10 client.26623 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:11:10.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.369+0000 7fdb8a3ef740 20 client.26623 populate_metadata read hostname 'smithi043' 2024-09-19T05:11:10.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.369+0000 7fdb8a3ef740 1 --2- 172.21.15.43:0/3406611 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55cab8ae38f0 0x55cab8ae5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:10.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.369+0000 7fdb8a3ef740 4 client.26623 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:11:10.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.369+0000 7fdb8a3ef740 1 -- 172.21.15.43:0/3406611 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cab89cae60 con 0x55cab8ae38f0 2024-09-19T05:11:10.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.369+0000 7fdb7fe00640 1 --2- 172.21.15.43:0/3406611 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55cab8ae38f0 0x55cab8ae5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:10.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.370+0000 7fdb7fe00640 1 --2- 172.21.15.43:0/3406611 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55cab8ae38f0 0x55cab8ae5ce0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8697f43329e50558 server_cookie=4df3f2325d7530a8 in_seq=0 out_seq=0 2024-09-19T05:11:10.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.370+0000 7fdb66a00640 10 client.26623 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:11:10.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.370+0000 7fdb66a00640 1 -- 172.21.15.43:0/3406611 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55cab89cae60 con 0x55cab8ae38f0 2024-09-19T05:11:10.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.370+0000 7fdb66a00640 10 client.26623 handle_command_reply: tid=0 2024-09-19T05:11:10.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.370+0000 7fdb8a3ef740 1 client.26623 shutdown 2024-09-19T05:11:10.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.370+0000 7fdb8a3ef740 1 -- 172.21.15.43:0/3406611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cab8a5cb90 msgr2=0x55cab8a6ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:10.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.370+0000 7fdb8a3ef740 1 --2- 172.21.15.43:0/3406611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cab8a5cb90 0x55cab8a6ff90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdb78009c90 tx=0x7fdb78027400 comp rx=0 tx=0).stop 2024-09-19T05:11:10.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.370+0000 7fdb8a3ef740 1 -- 172.21.15.43:0/3406611 shutdown_connections 2024-09-19T05:11:10.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.370+0000 7fdb8a3ef740 1 --2- 172.21.15.43:0/3406611 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55cab8ae38f0 0x55cab8ae5ce0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:10.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.371+0000 7fdb8a3ef740 1 --2- 172.21.15.43:0/3406611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cab8a67d70 0x55cab8a6cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:10.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.371+0000 7fdb8a3ef740 1 --2- 172.21.15.43:0/3406611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cab8a5d4b0 0x55cab8a704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:10.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.371+0000 7fdb66a00640 0 client.26623 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:11:10.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.371+0000 7fdb8a3ef740 1 --2- 172.21.15.43:0/3406611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cab8a5cb90 0x55cab8a6ff90 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:10.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.371+0000 7fdb8a3ef740 1 -- 172.21.15.43:0/3406611 >> 172.21.15.43:0/3406611 conn(0x55cab8a59120 msgr2=0x55cab8a665a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:10.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.371+0000 7fdb8a3ef740 1 -- 172.21.15.43:0/3406611 shutdown_connections 2024-09-19T05:11:10.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.371+0000 7fdb8a3ef740 1 -- 172.21.15.43:0/3406611 wait complete. 2024-09-19T05:11:10.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.371+0000 7fdb8a3ef740 20 client.26623 trim_cache size 0 max 16384 2024-09-19T05:11:10.378 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:11:10.378 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T05:11:10.379 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-09-19T05:11:10.379 INFO:teuthology.orchestra.run.smithi043.stdout: "5ef37815-b0da-4d8a-8070-96de311b047a": { 2024-09-19T05:11:10.379 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-09-19T05:11:10.379 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "5ef37815-b0da-4d8a-8070-96de311b047a", 2024-09-19T05:11:10.379 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-09-19T05:11:10.379 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:11:10.379 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:11:10.379 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:11:10.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.372+0000 7fdb87800640 1 -- 172.21.15.43:0/2846493492 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fdb680a3430 msgr2=0x7fdb680a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:10.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.372+0000 7fdb87800640 1 --2- 172.21.15.43:0/2846493492 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fdb680a3430 0x7fdb680a58f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdb740f0570 tx=0x7fdb740f3d80 comp rx=0 tx=0).stop 2024-09-19T05:11:10.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.372+0000 7fdb87800640 1 -- 172.21.15.43:0/2846493492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb80076720 msgr2=0x7fdb801a4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:10.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.372+0000 7fdb87800640 1 --2- 172.21.15.43:0/2846493492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb80076720 0x7fdb801a4820 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fdb74004770 tx=0x7fdb74003c10 comp rx=0 tx=0).stop 2024-09-19T05:11:10.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.372+0000 7fdb87800640 1 -- 172.21.15.43:0/2846493492 shutdown_connections 2024-09-19T05:11:10.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.372+0000 7fdb87800640 1 --2- 172.21.15.43:0/2846493492 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb80112a50 0x7fdb8019e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:10.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.372+0000 7fdb87800640 1 --2- 172.21.15.43:0/2846493492 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fdb680a3430 0x7fdb680a58f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:10.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.372+0000 7fdb87800640 1 --2- 172.21.15.43:0/2846493492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb80077040 0x7fdb801a4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:10.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.372+0000 7fdb87800640 1 --2- 172.21.15.43:0/2846493492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb80076720 0x7fdb801a4820 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:10.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.372+0000 7fdb87800640 1 -- 172.21.15.43:0/2846493492 >> 172.21.15.43:0/2846493492 conn(0x7fdb800ffa50 msgr2=0x7fdb80101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:10.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.372+0000 7fdb87800640 1 -- 172.21.15.43:0/2846493492 shutdown_connections 2024-09-19T05:11:10.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:10.372+0000 7fdb87800640 1 -- 172.21.15.43:0/2846493492 wait complete. 2024-09-19T05:11:10.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:11:09 smithi151 ceph-mon[35425]: pgmap v198: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-09-19T05:11:10.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:11:09 smithi151 ceph-mon[35425]: scrub queued for path: / 2024-09-19T05:11:10.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:11:09 smithi151 ceph-mon[35425]: scrub summary: idle+waiting paths [/] 2024-09-19T05:11:10.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:11:09 smithi151 ceph-mon[35425]: scrub summary: active paths [/] 2024-09-19T05:11:10.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:11:09 smithi151 ceph-mon[35425]: scrub summary: idle+waiting paths [/] 2024-09-19T05:11:10.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:11:09 smithi151 ceph-mon[35425]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T05:11:11.179 INFO:tasks.cephfs.filesystem:scrub status for tag:5ef37815-b0da-4d8a-8070-96de311b047a - {'path': '/', 'tag': '5ef37815-b0da-4d8a-8070-96de311b047a', 'options': 'recursive,force'} 2024-09-19T05:11:12.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:11:11 smithi096 ceph-mon[35085]: pgmap v200: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-09-19T05:11:12.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:11:11 smithi043 ceph-mon[30800]: pgmap v200: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-09-19T05:11:12.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:11:11 smithi151 ceph-mon[35425]: pgmap v200: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-09-19T05:11:14.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:11:13 smithi096 ceph-mon[35085]: pgmap v201: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-09-19T05:11:14.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:11:13 smithi096 ceph-mon[35085]: scrub summary: idle 2024-09-19T05:11:14.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:11:13 smithi043 ceph-mon[30800]: pgmap v201: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-09-19T05:11:14.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:11:13 smithi043 ceph-mon[30800]: scrub summary: idle 2024-09-19T05:11:14.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:11:13 smithi151 ceph-mon[35425]: pgmap v201: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-09-19T05:11:14.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:11:13 smithi151 ceph-mon[35425]: scrub summary: idle 2024-09-19T05:11:16.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:11:15 smithi096 ceph-mon[35085]: pgmap v202: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T05:11:16.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:11:15 smithi096 ceph-mon[35085]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T05:11:16.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:11:15 smithi043 ceph-mon[30800]: pgmap v202: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T05:11:16.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:11:15 smithi043 ceph-mon[30800]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T05:11:16.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:11:15 smithi151 ceph-mon[35425]: pgmap v202: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T05:11:16.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:11:15 smithi151 ceph-mon[35425]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T05:11:18.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:11:17 smithi096 ceph-mon[35085]: pgmap v204: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 13 op/s 2024-09-19T05:11:18.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:11:17 smithi043 ceph-mon[30800]: pgmap v204: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 13 op/s 2024-09-19T05:11:18.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:11:17 smithi151 ceph-mon[35425]: pgmap v204: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 13 op/s 2024-09-19T05:11:20.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:11:19 smithi096 ceph-mon[35085]: pgmap v205: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 216 B/s wr, 11 op/s 2024-09-19T05:11:20.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:11:19 smithi043 ceph-mon[30800]: pgmap v205: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 216 B/s wr, 11 op/s 2024-09-19T05:11:20.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:11:19 smithi151 ceph-mon[35425]: pgmap v205: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 216 B/s wr, 11 op/s 2024-09-19T05:11:22.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:11:21 smithi096 ceph-mon[35085]: pgmap v206: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T05:11:22.291 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:11:21 smithi043 ceph-mon[30800]: pgmap v206: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T05:11:22.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:11:21 smithi151 ceph-mon[35425]: pgmap v206: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T05:11:24.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:11:23 smithi096 ceph-mon[35085]: pgmap v207: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T05:11:24.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:11:23 smithi043 ceph-mon[30800]: pgmap v207: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T05:11:24.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:11:23 smithi151 ceph-mon[35425]: pgmap v207: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T05:11:26.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:11:25 smithi096 ceph-mon[35085]: pgmap v208: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T05:11:26.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:11:25 smithi043 ceph-mon[30800]: pgmap v208: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T05:11:26.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:11:25 smithi151 ceph-mon[35425]: pgmap v208: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T05:11:28.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:11:27 smithi096 ceph-mon[35085]: pgmap v209: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-19T05:11:28.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:11:27 smithi043 ceph-mon[30800]: pgmap v209: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-19T05:11:28.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:11:27 smithi151 ceph-mon[35425]: pgmap v209: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-19T05:11:30.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:11:29 smithi096 ceph-mon[35085]: pgmap v210: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:11:30.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:11:29 smithi043 ceph-mon[30800]: pgmap v210: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:11:30.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:11:29 smithi151 ceph-mon[35425]: pgmap v210: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:11:32.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:11:31 smithi096 ceph-mon[35085]: pgmap v211: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:11:32.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:11:31 smithi043 ceph-mon[30800]: pgmap v211: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:11:32.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:11:31 smithi151 ceph-mon[35425]: pgmap v211: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:11:34.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:11:33 smithi043 ceph-mon[30800]: pgmap v212: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:11:34.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:11:33 smithi151 ceph-mon[35425]: pgmap v212: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:11:34.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:11:33 smithi096 ceph-mon[35085]: pgmap v212: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:11:36.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:11:35 smithi043 ceph-mon[30800]: pgmap v213: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-09-19T05:11:36.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:11:35 smithi151 ceph-mon[35425]: pgmap v213: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-09-19T05:11:36.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:11:35 smithi096 ceph-mon[35085]: pgmap v213: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-09-19T05:11:38.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:11:37 smithi043 ceph-mon[30800]: pgmap v214: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-09-19T05:11:38.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:11:37 smithi151 ceph-mon[35425]: pgmap v214: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-09-19T05:11:38.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:11:37 smithi096 ceph-mon[35085]: pgmap v214: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-09-19T05:11:40.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:11:39 smithi043 ceph-mon[30800]: pgmap v215: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-09-19T05:11:40.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:11:39 smithi151 ceph-mon[35425]: pgmap v215: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-09-19T05:11:40.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:11:39 smithi096 ceph-mon[35085]: pgmap v215: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-09-19T05:11:41.180 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:11:41.575 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:11:42.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.170+0000 7fb802200640 1 -- 172.21.15.43:0/2209927257 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7fc109140 msgr2=0x7fb7fc110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:42.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.170+0000 7fb802200640 1 --2- 172.21.15.43:0/2209927257 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7fc109140 0x7fb7fc110cc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4009be0 tx=0x7fb7f402a6d0 comp rx=0 tx=0).stop 2024-09-19T05:11:42.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.170+0000 7fb802200640 1 -- 172.21.15.43:0/2209927257 shutdown_connections 2024-09-19T05:11:42.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.170+0000 7fb802200640 1 --2- 172.21.15.43:0/2209927257 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7fc109140 0x7fb7fc110cc0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:42.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.170+0000 7fb802200640 1 --2- 172.21.15.43:0/2209927257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7fc1049c0 0x7fb7fc108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:42.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.170+0000 7fb802200640 1 --2- 172.21.15.43:0/2209927257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7fc104010 0x7fb7fc1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:42.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.170+0000 7fb802200640 1 -- 172.21.15.43:0/2209927257 >> 172.21.15.43:0/2209927257 conn(0x7fb7fc0ffa50 msgr2=0x7fb7fc101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:42.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.170+0000 7fb802200640 1 -- 172.21.15.43:0/2209927257 shutdown_connections 2024-09-19T05:11:42.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.170+0000 7fb802200640 1 -- 172.21.15.43:0/2209927257 wait complete. 2024-09-19T05:11:42.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.171+0000 7fb802200640 1 Processor -- start 2024-09-19T05:11:42.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.171+0000 7fb802200640 1 -- start start 2024-09-19T05:11:42.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.171+0000 7fb802200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7fc104010 0x7fb7fc19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:42.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.171+0000 7fb802200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7fc1049c0 0x7fb7fc19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:42.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.172+0000 7fb802200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7fc109140 0x7fb7fc19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:42.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.172+0000 7fb802200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc114170 con 0x7fb7fc104010 2024-09-19T05:11:42.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.172+0000 7fb802200640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc113ff0 con 0x7fb7fc1049c0 2024-09-19T05:11:42.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.172+0000 7fb802200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc1142f0 con 0x7fb7fc109140 2024-09-19T05:11:42.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.172+0000 7fb7fb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7fc104010 0x7fb7fc19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:42.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.172+0000 7fb7fb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7fc104010 0x7fb7fc19a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52012/0 (socket says 172.21.15.43:52012) 2024-09-19T05:11:42.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.172+0000 7fb7fb400640 1 -- 172.21.15.43:0/1134318975 learned_addr learned my addr 172.21.15.43:0/1134318975 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:42.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.172+0000 7fb7fbe00640 1 --2- 172.21.15.43:0/1134318975 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7fc109140 0x7fb7fc19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:42.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.172+0000 7fb7faa00640 1 --2- 172.21.15.43:0/1134318975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7fc1049c0 0x7fb7fc19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:42.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.172+0000 7fb7fb400640 1 -- 172.21.15.43:0/1134318975 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7fc109140 msgr2=0x7fb7fc19ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:42.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.172+0000 7fb7fb400640 1 --2- 172.21.15.43:0/1134318975 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7fc109140 0x7fb7fc19ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:42.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.172+0000 7fb7fb400640 1 -- 172.21.15.43:0/1134318975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7fc1049c0 msgr2=0x7fb7fc19ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:42.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.172+0000 7fb7fb400640 1 --2- 172.21.15.43:0/1134318975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7fc1049c0 0x7fb7fc19ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:42.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.172+0000 7fb7fb400640 1 -- 172.21.15.43:0/1134318975 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7fc19f5e0 con 0x7fb7fc104010 2024-09-19T05:11:42.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.173+0000 7fb7fb400640 1 --2- 172.21.15.43:0/1134318975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7fc104010 0x7fb7fc19a630 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec00ea60 tx=0x7fb7ec00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:42.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.173+0000 7fb7f3e00640 1 -- 172.21.15.43:0/1134318975 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec00cd20 con 0x7fb7fc104010 2024-09-19T05:11:42.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.173+0000 7fb7f3e00640 1 -- 172.21.15.43:0/1134318975 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7ec0035f0 con 0x7fb7fc104010 2024-09-19T05:11:42.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.173+0000 7fb802200640 1 -- 172.21.15.43:0/1134318975 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7fc19f8d0 con 0x7fb7fc104010 2024-09-19T05:11:42.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.173+0000 7fb7f3e00640 1 -- 172.21.15.43:0/1134318975 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec010690 con 0x7fb7fc104010 2024-09-19T05:11:42.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.173+0000 7fb802200640 1 -- 172.21.15.43:0/1134318975 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7fc1a71b0 con 0x7fb7fc104010 2024-09-19T05:11:42.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.177+0000 7fb804c8b740 1 Processor -- start 2024-09-19T05:11:42.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.177+0000 7fb7f3e00640 1 -- 172.21.15.43:0/1134318975 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb7ec0037a0 con 0x7fb7fc104010 2024-09-19T05:11:42.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.177+0000 7fb804c8b740 1 -- start start 2024-09-19T05:11:42.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.177+0000 7fb804c8b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55daf10cfb90 0x55daf10cff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:42.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.177+0000 7fb804c8b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daf10d04b0 0x55daf10da830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:42.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.177+0000 7fb804c8b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55daf10dad70 0x55daf10dd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:42.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.177+0000 7fb804c8b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55daf10363e0 con 0x55daf10d04b0 2024-09-19T05:11:42.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.177+0000 7fb804c8b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55daf0fddd80 con 0x55daf10dad70 2024-09-19T05:11:42.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.177+0000 7fb804c8b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55daf10323a0 con 0x55daf10cfb90 2024-09-19T05:11:42.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.177+0000 7fb7fb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daf10d04b0 0x55daf10da830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:42.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.177+0000 7fb7fbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55daf10cfb90 0x55daf10cff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:42.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.178+0000 7fb7faa00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55daf10dad70 0x55daf10dd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:42.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.178+0000 7fb7fb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daf10d04b0 0x55daf10da830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52026/0 (socket says 172.21.15.43:52026) 2024-09-19T05:11:42.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.178+0000 7fb7fbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55daf10cfb90 0x55daf10cff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:49768/0 (socket says 172.21.15.43:49768) 2024-09-19T05:11:42.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.178+0000 7fb7fb400640 1 -- 172.21.15.43:0/3431098251 learned_addr learned my addr 172.21.15.43:0/3431098251 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:42.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.178+0000 7fb7fb400640 1 -- 172.21.15.43:0/3431098251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55daf10cfb90 msgr2=0x55daf10cff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:42.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.178+0000 7fb7fb400640 1 --2- 172.21.15.43:0/3431098251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55daf10cfb90 0x55daf10cff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:42.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.178+0000 7fb7fb400640 1 -- 172.21.15.43:0/3431098251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55daf10dad70 msgr2=0x55daf10dd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:42.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.178+0000 7fb7fb400640 1 --2- 172.21.15.43:0/3431098251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55daf10dad70 0x55daf10dd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:42.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.178+0000 7fb7fb400640 1 -- 172.21.15.43:0/3431098251 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55daf0ffcae0 con 0x55daf10d04b0 2024-09-19T05:11:42.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.178+0000 7fb7faa00640 1 --2- 172.21.15.43:0/3431098251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55daf10dad70 0x55daf10dd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:11:42.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.178+0000 7fb7fb400640 1 --2- 172.21.15.43:0/3431098251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daf10d04b0 0x55daf10da830 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec0cd610 tx=0x7fb7ec0d84b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=944bb53d9ba2bcb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:42.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.179+0000 7fb7f3e00640 1 --2- 172.21.15.43:0/1134318975 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb7e00a3430 0x7fb7e00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:42.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.179+0000 7fb7e7e00640 1 -- 172.21.15.43:0/3431098251 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec0e3070 con 0x55daf10d04b0 2024-09-19T05:11:42.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.179+0000 7fb7f3e00640 1 -- 172.21.15.43:0/1134318975 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7fb7ec0cfff0 con 0x7fb7fc104010 2024-09-19T05:11:42.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.179+0000 7fb7e7e00640 1 -- 172.21.15.43:0/3431098251 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7ec0bdb80 con 0x55daf10d04b0 2024-09-19T05:11:42.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.179+0000 7fb7e7e00640 1 -- 172.21.15.43:0/3431098251 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec063e30 con 0x55daf10d04b0 2024-09-19T05:11:42.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.179+0000 7fb7fbe00640 1 --2- 172.21.15.43:0/1134318975 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb7e00a3430 0x7fb7e00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:42.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.180+0000 7fb804c8b740 1 -- 172.21.15.43:0/3431098251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daf10d04b0 msgr2=0x55daf10da830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:42.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.180+0000 7fb7fbe00640 1 --2- 172.21.15.43:0/1134318975 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb7e00a3430 0x7fb7e00a58f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4004af0 tx=0x7fb7f4005fd0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:42.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.180+0000 7fb804c8b740 1 --2- 172.21.15.43:0/3431098251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daf10d04b0 0x55daf10da830 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec0cd610 tx=0x7fb7ec0d84b0 comp rx=0 tx=0).stop 2024-09-19T05:11:42.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.180+0000 7fb804c8b740 1 -- 172.21.15.43:0/3431098251 shutdown_connections 2024-09-19T05:11:42.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.180+0000 7fb804c8b740 1 --2- 172.21.15.43:0/3431098251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55daf10dad70 0x55daf10dd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:42.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.180+0000 7fb804c8b740 1 --2- 172.21.15.43:0/3431098251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daf10d04b0 0x55daf10da830 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:42.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.180+0000 7fb804c8b740 1 --2- 172.21.15.43:0/3431098251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55daf10cfb90 0x55daf10cff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:42.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.180+0000 7fb804c8b740 1 -- 172.21.15.43:0/3431098251 >> 172.21.15.43:0/3431098251 conn(0x55daf10cc120 msgr2=0x55daf10cc500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:42.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.180+0000 7fb804c8b740 1 -- 172.21.15.43:0/3431098251 shutdown_connections 2024-09-19T05:11:42.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.180+0000 7fb804c8b740 1 -- 172.21.15.43:0/3431098251 wait complete. 2024-09-19T05:11:42.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.181+0000 7fb804c8b740 1 -- start start 2024-09-19T05:11:42.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.181+0000 7fb804c8b740 1 Processor -- start 2024-09-19T05:11:42.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.182+0000 7fb804c8b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55daf10cfb90 0x55daf10d7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:42.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.182+0000 7fb804c8b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55daf10d04b0 0x55daf10d82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:42.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.182+0000 7fb804c8b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daf10dad70 0x55daf10d87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:42.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.182+0000 7fb804c8b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55daf10dd9e0 con 0x55daf10dad70 2024-09-19T05:11:42.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.182+0000 7fb804c8b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55daf0ff10f0 con 0x55daf10d04b0 2024-09-19T05:11:42.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.182+0000 7fb804c8b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55daf10ddb60 con 0x55daf10cfb90 2024-09-19T05:11:42.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.182+0000 7fb804c8b740 10 client.0 _ll_register_callbacks cb 0x55daf0fe29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:42.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.182+0000 7fb7fbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daf10dad70 0x55daf10d87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:42.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.182+0000 7fb7fb400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55daf10cfb90 0x55daf10d7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:42.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.182+0000 7fb7fbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daf10dad70 0x55daf10d87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52038/0 (socket says 172.21.15.43:52038) 2024-09-19T05:11:42.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.182+0000 7fb7fbe00640 1 -- 172.21.15.43:0/301577000 learned_addr learned my addr 172.21.15.43:0/301577000 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:42.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.182+0000 7fb7faa00640 1 --2- 172.21.15.43:0/301577000 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55daf10d04b0 0x55daf10d82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:42.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.182+0000 7fb7fb400640 1 -- 172.21.15.43:0/301577000 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55daf10d04b0 msgr2=0x55daf10d82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:42.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.182+0000 7fb7fb400640 1 --2- 172.21.15.43:0/301577000 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55daf10d04b0 0x55daf10d82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:42.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.182+0000 7fb7fb400640 1 -- 172.21.15.43:0/301577000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daf10dad70 msgr2=0x55daf10d87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:42.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.182+0000 7fb7fb400640 1 --2- 172.21.15.43:0/301577000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daf10dad70 0x55daf10d87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:42.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.182+0000 7fb7fb400640 1 -- 172.21.15.43:0/301577000 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55daf0da73c0 con 0x55daf10cfb90 2024-09-19T05:11:42.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.183+0000 7fb7fb400640 1 --2- 172.21.15.43:0/301577000 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55daf10cfb90 0x55daf10d7d70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x55daf10ccf80 tx=0x7fb7ec0d38f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b3769a9b916e6b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:42.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.183+0000 7fb7e4c00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:11:42.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.183+0000 7fb7e4c00640 1 -- 172.21.15.43:0/301577000 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec065e00 con 0x55daf10cfb90 2024-09-19T05:11:42.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.183+0000 7fb7e4c00640 1 -- 172.21.15.43:0/301577000 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7fb7ec003b30 con 0x55daf10cfb90 2024-09-19T05:11:42.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.183+0000 7fb804c8b740 1 -- 172.21.15.43:0/301577000 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7ec0bdb80 con 0x55daf10cfb90 2024-09-19T05:11:42.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.183+0000 7fb7e4c00640 1 -- 172.21.15.43:0/301577000 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb7ec066130 con 0x55daf10cfb90 2024-09-19T05:11:42.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.183+0000 7fb804c8b740 10 client.26635 fetch_fsmap learned FSMap version 29 2024-09-19T05:11:42.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.183+0000 7fb804c8b740 1 -- 172.21.15.43:0/301577000 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55daf10dd9e0 con 0x55daf10cfb90 2024-09-19T05:11:42.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.183+0000 7fb7e4c00640 1 -- 172.21.15.43:0/301577000 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7ec0bdb80 con 0x55daf10cfb90 2024-09-19T05:11:42.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.184+0000 7fb7e4c00640 1 -- 172.21.15.43:0/301577000 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb7ec0988d0 con 0x55daf10cfb90 2024-09-19T05:11:42.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.184+0000 7fb804c8b740 10 client.26635 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:11:42.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.184+0000 7fb804c8b740 10 client.26635 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:11:42.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.184+0000 7fb804c8b740 20 client.26635 populate_metadata read hostname 'smithi043' 2024-09-19T05:11:42.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.184+0000 7fb804c8b740 1 --2- 172.21.15.43:0/301577000 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55daf10ea930 0x55daf10ecd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:42.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.184+0000 7fb804c8b740 4 client.26635 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:11:42.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.184+0000 7fb804c8b740 1 -- 172.21.15.43:0/301577000 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55daf1043950 con 0x55daf10ea930 2024-09-19T05:11:42.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.184+0000 7fb7faa00640 1 --2- 172.21.15.43:0/301577000 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55daf10ea930 0x55daf10ecd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:42.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.185+0000 7fb7faa00640 1 --2- 172.21.15.43:0/301577000 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55daf10ea930 0x55daf10ecd20 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b844636a42cdbf7 server_cookie=c8f1e629deb764c0 in_seq=0 out_seq=0 2024-09-19T05:11:42.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.185+0000 7fb7e4c00640 10 client.26635 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:11:42.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.186+0000 7fb7e4c00640 1 -- 172.21.15.43:0/301577000 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55daf1043950 con 0x55daf10ea930 2024-09-19T05:11:42.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.186+0000 7fb7e4c00640 10 client.26635 handle_command_reply: tid=0 2024-09-19T05:11:42.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.186+0000 7fb804c8b740 1 client.26635 shutdown 2024-09-19T05:11:42.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.186+0000 7fb804c8b740 1 -- 172.21.15.43:0/301577000 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55daf10cfb90 msgr2=0x55daf10d7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:42.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.186+0000 7fb804c8b740 1 --2- 172.21.15.43:0/301577000 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55daf10cfb90 0x55daf10d7d70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x55daf10ccf80 tx=0x7fb7ec0d38f0 comp rx=0 tx=0).stop 2024-09-19T05:11:42.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.186+0000 7fb804c8b740 1 -- 172.21.15.43:0/301577000 shutdown_connections 2024-09-19T05:11:42.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.186+0000 7fb804c8b740 1 --2- 172.21.15.43:0/301577000 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55daf10ea930 0x55daf10ecd20 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:42.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.186+0000 7fb804c8b740 1 --2- 172.21.15.43:0/301577000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daf10dad70 0x55daf10d87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:42.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.186+0000 7fb804c8b740 1 --2- 172.21.15.43:0/301577000 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55daf10d04b0 0x55daf10d82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:42.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.186+0000 7fb804c8b740 1 --2- 172.21.15.43:0/301577000 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55daf10cfb90 0x55daf10d7d70 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:42.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.186+0000 7fb7e4c00640 0 client.26635 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:11:42.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.186+0000 7fb804c8b740 1 -- 172.21.15.43:0/301577000 >> 172.21.15.43:0/301577000 conn(0x55daf10cc120 msgr2=0x55daf10d1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:42.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.186+0000 7fb804c8b740 1 -- 172.21.15.43:0/301577000 shutdown_connections 2024-09-19T05:11:42.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.186+0000 7fb804c8b740 1 -- 172.21.15.43:0/301577000 wait complete. 2024-09-19T05:11:42.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.186+0000 7fb804c8b740 20 client.26635 trim_cache size 0 max 16384 2024-09-19T05:11:42.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.204+0000 7fb804c8b740 1 Processor -- start 2024-09-19T05:11:42.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.205+0000 7fb804c8b740 1 -- start start 2024-09-19T05:11:42.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.205+0000 7fb804c8b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55daf10cfb90 0x55daf10f2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:42.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.205+0000 7fb804c8b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55daf10d04b0 0x55daf10f29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:42.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.205+0000 7fb804c8b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daf10dad70 0x55daf10d4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:42.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.205+0000 7fb804c8b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc1afc20 con 0x55daf10dad70 2024-09-19T05:11:42.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.205+0000 7fb804c8b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc1b1760 con 0x55daf10cfb90 2024-09-19T05:11:42.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.205+0000 7fb804c8b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc1b02f0 con 0x55daf10d04b0 2024-09-19T05:11:42.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.205+0000 7fb7fbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daf10dad70 0x55daf10d4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:42.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.205+0000 7fb7fb400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55daf10cfb90 0x55daf10f2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:42.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.205+0000 7fb7fbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daf10dad70 0x55daf10d4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52048/0 (socket says 172.21.15.43:52048) 2024-09-19T05:11:42.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.205+0000 7fb7fbe00640 1 -- 172.21.15.43:0/1541849466 learned_addr learned my addr 172.21.15.43:0/1541849466 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:42.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.205+0000 7fb7faa00640 1 --2- 172.21.15.43:0/1541849466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55daf10d04b0 0x55daf10f29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:42.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.206+0000 7fb7fb400640 1 -- 172.21.15.43:0/1541849466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55daf10d04b0 msgr2=0x55daf10f29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:42.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.206+0000 7fb7fb400640 1 --2- 172.21.15.43:0/1541849466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55daf10d04b0 0x55daf10f29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:42.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.206+0000 7fb7fb400640 1 -- 172.21.15.43:0/1541849466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daf10dad70 msgr2=0x55daf10d4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:42.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.206+0000 7fb7fb400640 1 --2- 172.21.15.43:0/1541849466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daf10dad70 0x55daf10d4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:42.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.206+0000 7fb7fb400640 1 -- 172.21.15.43:0/1541849466 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55daf10ccdb0 con 0x55daf10cfb90 2024-09-19T05:11:42.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.206+0000 7fb7faa00640 1 --2- 172.21.15.43:0/1541849466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55daf10d04b0 0x55daf10f29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:42.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.206+0000 7fb7fb400640 1 --2- 172.21.15.43:0/1541849466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55daf10cfb90 0x55daf10f2460 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec0986c0 tx=0x7fb7ec0d0270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=193a526e230abaf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:42.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.206+0000 7fb7e7400640 1 -- 172.21.15.43:0/1541849466 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec066310 con 0x55daf10cfb90 2024-09-19T05:11:42.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.206+0000 7fb7e7400640 1 -- 172.21.15.43:0/1541849466 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7ec09c960 con 0x55daf10cfb90 2024-09-19T05:11:42.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.206+0000 7fb7e7400640 1 -- 172.21.15.43:0/1541849466 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec0664f0 con 0x55daf10cfb90 2024-09-19T05:11:42.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.207+0000 7fb804c8b740 1 -- 172.21.15.43:0/1541849466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55daf10cfb90 msgr2=0x55daf10f2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:42.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.207+0000 7fb804c8b740 1 --2- 172.21.15.43:0/1541849466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55daf10cfb90 0x55daf10f2460 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec0986c0 tx=0x7fb7ec0d0270 comp rx=0 tx=0).stop 2024-09-19T05:11:42.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.207+0000 7fb804c8b740 1 -- 172.21.15.43:0/1541849466 shutdown_connections 2024-09-19T05:11:42.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.207+0000 7fb804c8b740 1 --2- 172.21.15.43:0/1541849466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daf10dad70 0x55daf10d4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:42.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.207+0000 7fb804c8b740 1 --2- 172.21.15.43:0/1541849466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55daf10d04b0 0x55daf10f29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:42.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.207+0000 7fb804c8b740 1 --2- 172.21.15.43:0/1541849466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55daf10cfb90 0x55daf10f2460 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:42.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.207+0000 7fb804c8b740 1 -- 172.21.15.43:0/1541849466 >> 172.21.15.43:0/1541849466 conn(0x55daf10cc120 msgr2=0x55daf10d9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:42.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.207+0000 7fb804c8b740 1 -- 172.21.15.43:0/1541849466 shutdown_connections 2024-09-19T05:11:42.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.207+0000 7fb804c8b740 1 -- 172.21.15.43:0/1541849466 wait complete. 2024-09-19T05:11:42.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.207+0000 7fb804c8b740 1 -- start start 2024-09-19T05:11:42.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.208+0000 7fb804c8b740 1 Processor -- start 2024-09-19T05:11:42.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.208+0000 7fb804c8b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daf10cfb90 0x55daf10dedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:42.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.208+0000 7fb804c8b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55daf10d04b0 0x55daf10df310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:42.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.208+0000 7fb804c8b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55daf10dad70 0x55daf10df850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:42.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.208+0000 7fb804c8b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc1b1090 con 0x55daf10cfb90 2024-09-19T05:11:42.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.208+0000 7fb804c8b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc1b09c0 con 0x55daf10dad70 2024-09-19T05:11:42.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.208+0000 7fb804c8b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc1adc20 con 0x55daf10d04b0 2024-09-19T05:11:42.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.208+0000 7fb804c8b740 10 client.0 _ll_register_callbacks cb 0x7fb7ec005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:42.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.208+0000 7fb7fb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daf10cfb90 0x55daf10dedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:42.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.208+0000 7fb7fb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daf10cfb90 0x55daf10dedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52052/0 (socket says 172.21.15.43:52052) 2024-09-19T05:11:42.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.208+0000 7fb7fb400640 1 -- 172.21.15.43:0/3534347334 learned_addr learned my addr 172.21.15.43:0/3534347334 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:42.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.208+0000 7fb7faa00640 1 --2- 172.21.15.43:0/3534347334 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55daf10d04b0 0x55daf10df310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:42.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.209+0000 7fb7fbe00640 1 --2- 172.21.15.43:0/3534347334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55daf10dad70 0x55daf10df850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:42.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.209+0000 7fb7fb400640 1 -- 172.21.15.43:0/3534347334 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55daf10d04b0 msgr2=0x55daf10df310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:42.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.209+0000 7fb7fb400640 1 --2- 172.21.15.43:0/3534347334 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55daf10d04b0 0x55daf10df310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:42.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.209+0000 7fb7fb400640 1 -- 172.21.15.43:0/3534347334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55daf10dad70 msgr2=0x55daf10df850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:42.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.209+0000 7fb7fb400640 1 --2- 172.21.15.43:0/3534347334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55daf10dad70 0x55daf10df850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:42.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.209+0000 7fb7fb400640 1 -- 172.21.15.43:0/3534347334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55daf11546d0 con 0x55daf10cfb90 2024-09-19T05:11:42.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.209+0000 7fb7faa00640 1 --2- 172.21.15.43:0/3534347334 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55daf10d04b0 0x55daf10df310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:11:42.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.209+0000 7fb7fbe00640 1 --2- 172.21.15.43:0/3534347334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55daf10dad70 0x55daf10df850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:42.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.209+0000 7fb7fb400640 1 --2- 172.21.15.43:0/3534347334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daf10cfb90 0x55daf10dedd0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec0d89b0 tx=0x7fb7ec09bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e308975507c583 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:42.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.209+0000 7fb7e4c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:11:42.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.209+0000 7fb7e4c00640 1 -- 172.21.15.43:0/3534347334 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec066780 con 0x55daf10cfb90 2024-09-19T05:11:42.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.209+0000 7fb7e4c00640 1 -- 172.21.15.43:0/3534347334 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7fb7ec003b30 con 0x55daf10cfb90 2024-09-19T05:11:42.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.209+0000 7fb804c8b740 1 -- 172.21.15.43:0/3534347334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7ec09c960 con 0x55daf10cfb90 2024-09-19T05:11:42.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.210+0000 7fb7e4c00640 1 -- 172.21.15.43:0/3534347334 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb7ec061520 con 0x55daf10cfb90 2024-09-19T05:11:42.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.210+0000 7fb804c8b740 10 client.16683 fetch_fsmap learned FSMap version 29 2024-09-19T05:11:42.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.210+0000 7fb804c8b740 1 -- 172.21.15.43:0/3534347334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55daf11546d0 con 0x55daf10cfb90 2024-09-19T05:11:42.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.210+0000 7fb7e4c00640 1 -- 172.21.15.43:0/3534347334 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7ec09c960 con 0x55daf10cfb90 2024-09-19T05:11:42.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.210+0000 7fb7e4c00640 1 -- 172.21.15.43:0/3534347334 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb7ec0cd950 con 0x55daf10cfb90 2024-09-19T05:11:42.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.210+0000 7fb804c8b740 10 client.16683 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:11:42.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.210+0000 7fb804c8b740 10 client.16683 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:11:42.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.210+0000 7fb804c8b740 20 client.16683 populate_metadata read hostname 'smithi043' 2024-09-19T05:11:42.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.210+0000 7fb804c8b740 1 --2- 172.21.15.43:0/3534347334 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55daf1155c80 0x55daf1158070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:42.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.210+0000 7fb804c8b740 4 client.16683 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:11:42.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.210+0000 7fb804c8b740 1 -- 172.21.15.43:0/3534347334 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55daf103de60 con 0x55daf1155c80 2024-09-19T05:11:42.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.211+0000 7fb7faa00640 1 --2- 172.21.15.43:0/3534347334 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55daf1155c80 0x55daf1158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:42.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.211+0000 7fb7faa00640 1 --2- 172.21.15.43:0/3534347334 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55daf1155c80 0x55daf1158070 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5090f211f5c68871 server_cookie=df2277a52333bbb6 in_seq=0 out_seq=0 2024-09-19T05:11:42.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.211+0000 7fb7e4c00640 10 client.16683 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:11:42.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.211+0000 7fb7e4c00640 1 -- 172.21.15.43:0/3534347334 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55daf103de60 con 0x55daf1155c80 2024-09-19T05:11:42.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.211+0000 7fb7e4c00640 10 client.16683 handle_command_reply: tid=0 2024-09-19T05:11:42.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.211+0000 7fb804c8b740 1 client.16683 shutdown 2024-09-19T05:11:42.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.212+0000 7fb804c8b740 1 -- 172.21.15.43:0/3534347334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daf10cfb90 msgr2=0x55daf10dedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:42.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.212+0000 7fb804c8b740 1 --2- 172.21.15.43:0/3534347334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daf10cfb90 0x55daf10dedd0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec0d89b0 tx=0x7fb7ec09bda0 comp rx=0 tx=0).stop 2024-09-19T05:11:42.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.212+0000 7fb804c8b740 1 -- 172.21.15.43:0/3534347334 shutdown_connections 2024-09-19T05:11:42.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.212+0000 7fb804c8b740 1 --2- 172.21.15.43:0/3534347334 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55daf1155c80 0x55daf1158070 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:42.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.212+0000 7fb804c8b740 1 --2- 172.21.15.43:0/3534347334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55daf10dad70 0x55daf10df850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:42.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.212+0000 7fb804c8b740 1 --2- 172.21.15.43:0/3534347334 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55daf10d04b0 0x55daf10df310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:42.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.212+0000 7fb804c8b740 1 --2- 172.21.15.43:0/3534347334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daf10cfb90 0x55daf10dedd0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:42.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.212+0000 7fb804c8b740 1 -- 172.21.15.43:0/3534347334 >> 172.21.15.43:0/3534347334 conn(0x55daf10cc120 msgr2=0x55daf10d95d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:42.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.212+0000 7fb7e4c00640 0 client.16683 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:11:42.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.212+0000 7fb804c8b740 1 -- 172.21.15.43:0/3534347334 shutdown_connections 2024-09-19T05:11:42.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.212+0000 7fb804c8b740 1 -- 172.21.15.43:0/3534347334 wait complete. 2024-09-19T05:11:42.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.212+0000 7fb804c8b740 20 client.16683 trim_cache size 0 max 16384 2024-09-19T05:11:42.221 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:11:42.221 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-09-19T05:11:42.221 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-09-19T05:11:42.221 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:11:42.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.213+0000 7fb802200640 1 -- 172.21.15.43:0/1134318975 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb7e00a3430 msgr2=0x7fb7e00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:42.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.213+0000 7fb802200640 1 --2- 172.21.15.43:0/1134318975 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb7e00a3430 0x7fb7e00a58f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4004af0 tx=0x7fb7f4005fd0 comp rx=0 tx=0).stop 2024-09-19T05:11:42.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.213+0000 7fb802200640 1 -- 172.21.15.43:0/1134318975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7fc104010 msgr2=0x7fb7fc19a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:42.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.213+0000 7fb802200640 1 --2- 172.21.15.43:0/1134318975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7fc104010 0x7fb7fc19a630 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec00ea60 tx=0x7fb7ec00ee60 comp rx=0 tx=0).stop 2024-09-19T05:11:42.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.213+0000 7fb802200640 1 -- 172.21.15.43:0/1134318975 shutdown_connections 2024-09-19T05:11:42.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.213+0000 7fb802200640 1 --2- 172.21.15.43:0/1134318975 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7fc109140 0x7fb7fc19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:42.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.213+0000 7fb802200640 1 --2- 172.21.15.43:0/1134318975 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb7e00a3430 0x7fb7e00a58f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:42.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.213+0000 7fb802200640 1 --2- 172.21.15.43:0/1134318975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7fc1049c0 0x7fb7fc19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:42.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.213+0000 7fb802200640 1 --2- 172.21.15.43:0/1134318975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7fc104010 0x7fb7fc19a630 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:42.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.213+0000 7fb802200640 1 -- 172.21.15.43:0/1134318975 >> 172.21.15.43:0/1134318975 conn(0x7fb7fc0ffa50 msgr2=0x7fb7fc100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:42.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.213+0000 7fb802200640 1 -- 172.21.15.43:0/1134318975 shutdown_connections 2024-09-19T05:11:42.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:42.213+0000 7fb802200640 1 -- 172.21.15.43:0/1134318975 wait complete. 2024-09-19T05:11:42.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:11:41 smithi043 ceph-mon[30800]: pgmap v216: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-09-19T05:11:42.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:11:41 smithi151 ceph-mon[35425]: pgmap v216: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-09-19T05:11:42.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:11:41 smithi096 ceph-mon[35085]: pgmap v216: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-09-19T05:11:43.184 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T05:11:43.397 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:11:43.777 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:11:44.256 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:11:43 smithi043 ceph-mon[30800]: pgmap v217: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-09-19T05:11:44.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.368+0000 7fb97c800640 1 -- 172.21.15.43:0/750279086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb974074f40 msgr2=0x7fb97410de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:44.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.368+0000 7fb97c800640 1 --2- 172.21.15.43:0/750279086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb974074f40 0x7fb97410de80 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fb964009900 tx=0x7fb96402a480 comp rx=0 tx=0).stop 2024-09-19T05:11:44.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.368+0000 7fb97c800640 1 -- 172.21.15.43:0/750279086 shutdown_connections 2024-09-19T05:11:44.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.368+0000 7fb97c800640 1 --2- 172.21.15.43:0/750279086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb97410e3c0 0x7fb974110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:44.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.368+0000 7fb97c800640 1 --2- 172.21.15.43:0/750279086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb974074f40 0x7fb97410de80 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:44.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.368+0000 7fb97c800640 1 --2- 172.21.15.43:0/750279086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb974074620 0x7fb974074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:44.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.368+0000 7fb97c800640 1 -- 172.21.15.43:0/750279086 >> 172.21.15.43:0/750279086 conn(0x7fb9740fd8c0 msgr2=0x7fb9740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:44.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.369+0000 7fb97c800640 1 -- 172.21.15.43:0/750279086 shutdown_connections 2024-09-19T05:11:44.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.369+0000 7fb97c800640 1 -- 172.21.15.43:0/750279086 wait complete. 2024-09-19T05:11:44.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.370+0000 7fb97c800640 1 Processor -- start 2024-09-19T05:11:44.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.370+0000 7fb97c800640 1 -- start start 2024-09-19T05:11:44.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.370+0000 7fb97c800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb974074620 0x7fb97419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:44.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.370+0000 7fb97c800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb974074f40 0x7fb97419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:44.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.370+0000 7fb97c800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb97410e3c0 0x7fb97419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:44.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.371+0000 7fb97c800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb9741141b0 con 0x7fb974074620 2024-09-19T05:11:44.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.371+0000 7fb97c800640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb974114030 con 0x7fb974074f40 2024-09-19T05:11:44.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.371+0000 7fb97c800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb974114330 con 0x7fb97410e3c0 2024-09-19T05:11:44.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.371+0000 7fb979e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb974074620 0x7fb97419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:44.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.371+0000 7fb979e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb974074620 0x7fb97419a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52066/0 (socket says 172.21.15.43:52066) 2024-09-19T05:11:44.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.371+0000 7fb979e00640 1 -- 172.21.15.43:0/3678160923 learned_addr learned my addr 172.21.15.43:0/3678160923 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:44.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.371+0000 7fb979400640 1 --2- 172.21.15.43:0/3678160923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb974074f40 0x7fb97419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:44.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.371+0000 7fb97a800640 1 --2- 172.21.15.43:0/3678160923 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb97410e3c0 0x7fb97419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:44.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.371+0000 7fb979400640 1 -- 172.21.15.43:0/3678160923 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb97410e3c0 msgr2=0x7fb97419ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:44.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.371+0000 7fb979400640 1 --2- 172.21.15.43:0/3678160923 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb97410e3c0 0x7fb97419ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:44.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.371+0000 7fb979400640 1 -- 172.21.15.43:0/3678160923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb974074620 msgr2=0x7fb97419a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:44.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.371+0000 7fb979400640 1 --2- 172.21.15.43:0/3678160923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb974074620 0x7fb97419a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:44.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.371+0000 7fb979400640 1 -- 172.21.15.43:0/3678160923 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb97419f620 con 0x7fb974074f40 2024-09-19T05:11:44.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.372+0000 7fb979400640 1 --2- 172.21.15.43:0/3678160923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb974074f40 0x7fb97419abb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb96402a9a0 tx=0x7fb964003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:44.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.372+0000 7fb96ea00640 1 -- 172.21.15.43:0/3678160923 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96403f070 con 0x7fb974074f40 2024-09-19T05:11:44.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.372+0000 7fb97c800640 1 -- 172.21.15.43:0/3678160923 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb97419f8b0 con 0x7fb974074f40 2024-09-19T05:11:44.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.372+0000 7fb96ea00640 1 -- 172.21.15.43:0/3678160923 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb964031c90 con 0x7fb974074f40 2024-09-19T05:11:44.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.372+0000 7fb96ea00640 1 -- 172.21.15.43:0/3678160923 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb964035030 con 0x7fb974074f40 2024-09-19T05:11:44.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.372+0000 7fb97c800640 1 -- 172.21.15.43:0/3678160923 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9741a71f0 con 0x7fb974074f40 2024-09-19T05:11:44.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.373+0000 7fb97c800640 1 -- 172.21.15.43:0/3678160923 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb974075cd0 con 0x7fb974074f40 2024-09-19T05:11:44.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.375+0000 7fb96ea00640 1 -- 172.21.15.43:0/3678160923 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb964042070 con 0x7fb974074f40 2024-09-19T05:11:44.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.376+0000 7fb96ea00640 1 --2- 172.21.15.43:0/3678160923 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb95c0a33b0 0x7fb95c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:44.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.377+0000 7fb96ea00640 1 -- 172.21.15.43:0/3678160923 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7fb9640f0c70 con 0x7fb974074f40 2024-09-19T05:11:44.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.377+0000 7fb979e00640 1 --2- 172.21.15.43:0/3678160923 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb95c0a33b0 0x7fb95c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:44.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.378+0000 7fb979e00640 1 --2- 172.21.15.43:0/3678160923 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb95c0a33b0 0x7fb95c0a5870 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb968009c90 tx=0x7fb968005f90 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:44.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.379+0000 7fb96ea00640 1 -- 172.21.15.43:0/3678160923 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb9640bcf90 con 0x7fb974074f40 2024-09-19T05:11:44.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:11:43 smithi151 ceph-mon[35425]: pgmap v217: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-09-19T05:11:44.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:11:43 smithi096 ceph-mon[35085]: pgmap v217: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-09-19T05:11:44.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.606+0000 7fb97c800640 1 -- 172.21.15.43:0/3678160923 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb974109540 con 0x7fb974074f40 2024-09-19T05:11:44.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.607+0000 7fb96ea00640 1 -- 172.21.15.43:0/3678160923 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11965 (secure 0 0 0) 0x7fb9640c1290 con 0x7fb974074f40 2024-09-19T05:11:44.608 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:11:44.609 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":29,"btime":"2024-09-19T05:04:38:785407+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/773801169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":773801169},{"type":"v1","addr":"172.21.15.43:6835","nonce":773801169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1799137902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1799137902},{"type":"v1","addr":"172.21.15.43:6837","nonce":1799137902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:02:25.463041+0000","modified":"2024-09-19T05:04:38.039809+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24518,"mds_2":24511,"mds_3":24506,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14541":{"gid":14541,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/794881135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":794881135},{"type":"v1","addr":"172.21.15.43:6841","nonce":794881135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/2836453158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2836453158},{"type":"v1","addr":"172.21.15.151:6833","nonce":2836453158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/178785812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":178785812},{"type":"v1","addr":"172.21.15.96:6833","nonce":178785812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1671043820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1671043820},{"type":"v1","addr":"172.21.15.96:6835","nonce":1671043820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2972514328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2972514328},{"type":"v1","addr":"172.21.15.151:6835","nonce":2972514328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.151:6837/3960378010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3960378010},{"type":"v1","addr":"172.21.15.151:6837","nonce":3960378010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6837/3978388286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3978388286},{"type":"v1","addr":"172.21.15.96:6837","nonce":3978388286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.151:6839/3794104313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3794104313},{"type":"v1","addr":"172.21.15.151:6839","nonce":3794104313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.43:6839/3404555030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3404555030},{"type":"v1","addr":"172.21.15.43:6839","nonce":3404555030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/605527621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":605527621},{"type":"v1","addr":"172.21.15.96:6839","nonce":605527621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24518,24511,24506,24493]},"id":1}]} 2024-09-19T05:11:44.609 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 29 2024-09-19T05:11:44.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.609+0000 7fb97c800640 1 -- 172.21.15.43:0/3678160923 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb95c0a33b0 msgr2=0x7fb95c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:44.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.609+0000 7fb97c800640 1 --2- 172.21.15.43:0/3678160923 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb95c0a33b0 0x7fb95c0a5870 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb968009c90 tx=0x7fb968005f90 comp rx=0 tx=0).stop 2024-09-19T05:11:44.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.609+0000 7fb97c800640 1 -- 172.21.15.43:0/3678160923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb974074f40 msgr2=0x7fb97419abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:44.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.609+0000 7fb97c800640 1 --2- 172.21.15.43:0/3678160923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb974074f40 0x7fb97419abb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb96402a9a0 tx=0x7fb964003b20 comp rx=0 tx=0).stop 2024-09-19T05:11:44.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.609+0000 7fb97c800640 1 -- 172.21.15.43:0/3678160923 shutdown_connections 2024-09-19T05:11:44.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.609+0000 7fb97c800640 1 --2- 172.21.15.43:0/3678160923 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb97410e3c0 0x7fb97419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:44.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.609+0000 7fb97c800640 1 --2- 172.21.15.43:0/3678160923 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb95c0a33b0 0x7fb95c0a5870 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:44.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.609+0000 7fb97c800640 1 --2- 172.21.15.43:0/3678160923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb974074f40 0x7fb97419abb0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:44.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.609+0000 7fb97c800640 1 --2- 172.21.15.43:0/3678160923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb974074620 0x7fb97419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:44.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.609+0000 7fb97c800640 1 -- 172.21.15.43:0/3678160923 >> 172.21.15.43:0/3678160923 conn(0x7fb9740fd8c0 msgr2=0x7fb9740fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:44.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.609+0000 7fb97c800640 1 -- 172.21.15.43:0/3678160923 shutdown_connections 2024-09-19T05:11:44.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:44.610+0000 7fb97c800640 1 -- 172.21.15.43:0/3678160923 wait complete. 2024-09-19T05:11:45.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:11:44 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/3678160923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:11:45.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:11:44 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/3678160923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:11:45.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:11:44 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/3678160923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:11:45.494 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T05:11:45.876 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:11:46.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:11:45 smithi043 ceph-mon[30800]: pgmap v218: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-09-19T05:11:46.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:11:45 smithi151 ceph-mon[35425]: pgmap v218: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-09-19T05:11:46.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.445+0000 7fb893200640 1 -- 172.21.15.43:0/2250294593 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb894074ee0 msgr2=0x7fb89410dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:46.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.445+0000 7fb893200640 1 --2- 172.21.15.43:0/2250294593 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb894074ee0 0x7fb89410dd00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb880009900 tx=0x7fb88002a480 comp rx=0 tx=0).stop 2024-09-19T05:11:46.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.445+0000 7fb893200640 1 -- 172.21.15.43:0/2250294593 shutdown_connections 2024-09-19T05:11:46.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.445+0000 7fb893200640 1 --2- 172.21.15.43:0/2250294593 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb89410e240 0x7fb894110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.445+0000 7fb893200640 1 --2- 172.21.15.43:0/2250294593 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb894074ee0 0x7fb89410dd00 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.445+0000 7fb893200640 1 --2- 172.21.15.43:0/2250294593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8940745c0 0x7fb8940749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.445+0000 7fb893200640 1 -- 172.21.15.43:0/2250294593 >> 172.21.15.43:0/2250294593 conn(0x7fb8940fd700 msgr2=0x7fb8940ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:46.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.445+0000 7fb893200640 1 -- 172.21.15.43:0/2250294593 shutdown_connections 2024-09-19T05:11:46.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.445+0000 7fb893200640 1 -- 172.21.15.43:0/2250294593 wait complete. 2024-09-19T05:11:46.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.446+0000 7fb893200640 1 Processor -- start 2024-09-19T05:11:46.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.446+0000 7fb893200640 1 -- start start 2024-09-19T05:11:46.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.447+0000 7fb893200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8940745c0 0x7fb89419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:46.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.447+0000 7fb893200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb894074ee0 0x7fb89419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:46.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.447+0000 7fb893200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb89410e240 0x7fb89419edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:46.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.447+0000 7fb893200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb894113fe0 con 0x7fb89410e240 2024-09-19T05:11:46.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.447+0000 7fb893200640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb894113e60 con 0x7fb8940745c0 2024-09-19T05:11:46.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.447+0000 7fb893200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb894114160 con 0x7fb894074ee0 2024-09-19T05:11:46.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.447+0000 7fb891e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8940745c0 0x7fb89419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:46.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.447+0000 7fb891e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8940745c0 0x7fb89419a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:50480/0 (socket says 172.21.15.43:50480) 2024-09-19T05:11:46.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.447+0000 7fb891e00640 1 -- 172.21.15.43:0/3177305750 learned_addr learned my addr 172.21.15.43:0/3177305750 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:46.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.447+0000 7fb891400640 1 --2- 172.21.15.43:0/3177305750 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb894074ee0 0x7fb89419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:46.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.447+0000 7fb892800640 1 --2- 172.21.15.43:0/3177305750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb89410e240 0x7fb89419edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:46.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.448+0000 7fb891e00640 1 -- 172.21.15.43:0/3177305750 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb894074ee0 msgr2=0x7fb89419aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:46.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.448+0000 7fb891e00640 1 --2- 172.21.15.43:0/3177305750 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb894074ee0 0x7fb89419aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.448+0000 7fb891e00640 1 -- 172.21.15.43:0/3177305750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb89410e240 msgr2=0x7fb89419edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:46.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.448+0000 7fb891e00640 1 --2- 172.21.15.43:0/3177305750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb89410e240 0x7fb89419edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.448+0000 7fb891e00640 1 -- 172.21.15.43:0/3177305750 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb89419f4d0 con 0x7fb8940745c0 2024-09-19T05:11:46.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.448+0000 7fb891e00640 1 --2- 172.21.15.43:0/3177305750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8940745c0 0x7fb89419a520 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb88400e9f0 tx=0x7fb88400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:46.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.448+0000 7fb88aa00640 1 -- 172.21.15.43:0/3177305750 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88400cd80 con 0x7fb8940745c0 2024-09-19T05:11:46.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.448+0000 7fb893200640 1 -- 172.21.15.43:0/3177305750 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb89419f7c0 con 0x7fb8940745c0 2024-09-19T05:11:46.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.448+0000 7fb88aa00640 1 -- 172.21.15.43:0/3177305750 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb884010910 con 0x7fb8940745c0 2024-09-19T05:11:46.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.448+0000 7fb88aa00640 1 -- 172.21.15.43:0/3177305750 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb884018980 con 0x7fb8940745c0 2024-09-19T05:11:46.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.448+0000 7fb893200640 1 -- 172.21.15.43:0/3177305750 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8941a70a0 con 0x7fb8940745c0 2024-09-19T05:11:46.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.451+0000 7fb89b3d5740 1 Processor -- start 2024-09-19T05:11:46.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.451+0000 7fb88aa00640 1 -- 172.21.15.43:0/3177305750 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb884010ac0 con 0x7fb8940745c0 2024-09-19T05:11:46.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.451+0000 7fb89b3d5740 1 -- start start 2024-09-19T05:11:46.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.452+0000 7fb89b3d5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558efabc7e40 0x558efabc8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:46.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.452+0000 7fb89b3d5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558efabc8760 0x558efabd2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:46.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.452+0000 7fb89b3d5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558efabd3020 0x558efabd5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:46.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.452+0000 7fb89b3d5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558efab2e3e0 con 0x558efabc7e40 2024-09-19T05:11:46.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.452+0000 7fb89b3d5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558efaad5d80 con 0x558efabd3020 2024-09-19T05:11:46.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.452+0000 7fb89b3d5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558efab2a3a0 con 0x558efabc8760 2024-09-19T05:11:46.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.452+0000 7fb892800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558efabc7e40 0x558efabc8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:46.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.452+0000 7fb892800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558efabc7e40 0x558efabc8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42886/0 (socket says 172.21.15.43:42886) 2024-09-19T05:11:46.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.452+0000 7fb892800640 1 -- 172.21.15.43:0/490603092 learned_addr learned my addr 172.21.15.43:0/490603092 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:46.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.452+0000 7fb891e00640 1 --2- 172.21.15.43:0/490603092 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558efabc8760 0x558efabd2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:46.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.452+0000 7fb891400640 1 --2- 172.21.15.43:0/490603092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558efabd3020 0x558efabd5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:46.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.453+0000 7fb892800640 1 -- 172.21.15.43:0/490603092 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558efabc8760 msgr2=0x558efabd2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:46.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.453+0000 7fb892800640 1 --2- 172.21.15.43:0/490603092 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558efabc8760 0x558efabd2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.453+0000 7fb892800640 1 -- 172.21.15.43:0/490603092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558efabd3020 msgr2=0x558efabd5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:46.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.453+0000 7fb892800640 1 --2- 172.21.15.43:0/490603092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558efabd3020 0x558efabd5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.453+0000 7fb892800640 1 -- 172.21.15.43:0/490603092 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558efaaf4ae0 con 0x558efabc7e40 2024-09-19T05:11:46.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.453+0000 7fb88aa00640 1 --2- 172.21.15.43:0/3177305750 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb8780a33b0 0x7fb8780a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:46.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.453+0000 7fb892800640 1 --2- 172.21.15.43:0/490603092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558efabc7e40 0x558efabc8220 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb88c009f00 tx=0x7fb88c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceef9cb1ee75c635 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:46.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.453+0000 7fb88aa00640 1 -- 172.21.15.43:0/3177305750 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7fb8840d0120 con 0x7fb8940745c0 2024-09-19T05:11:46.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.453+0000 7fb891e00640 1 --2- 172.21.15.43:0/3177305750 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb8780a33b0 0x7fb8780a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:46.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.454+0000 7fb87f000640 1 -- 172.21.15.43:0/490603092 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88c003b70 con 0x558efabc7e40 2024-09-19T05:11:46.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.454+0000 7fb87f000640 1 -- 172.21.15.43:0/490603092 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb88c003d10 con 0x558efabc7e40 2024-09-19T05:11:46.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.454+0000 7fb87f000640 1 -- 172.21.15.43:0/490603092 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88c011a40 con 0x558efabc7e40 2024-09-19T05:11:46.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.454+0000 7fb89b3d5740 1 -- 172.21.15.43:0/490603092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558efabc7e40 msgr2=0x558efabc8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:46.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.454+0000 7fb891e00640 1 --2- 172.21.15.43:0/3177305750 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb8780a33b0 0x7fb8780a5870 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x558efabc5230 tx=0x7fb8840d3bd0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:46.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.454+0000 7fb89b3d5740 1 --2- 172.21.15.43:0/490603092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558efabc7e40 0x558efabc8220 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb88c009f00 tx=0x7fb88c0107a0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.454+0000 7fb89b3d5740 1 -- 172.21.15.43:0/490603092 shutdown_connections 2024-09-19T05:11:46.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.454+0000 7fb89b3d5740 1 --2- 172.21.15.43:0/490603092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558efabd3020 0x558efabd5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.454+0000 7fb89b3d5740 1 --2- 172.21.15.43:0/490603092 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558efabc8760 0x558efabd2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.454+0000 7fb89b3d5740 1 --2- 172.21.15.43:0/490603092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558efabc7e40 0x558efabc8220 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.454+0000 7fb89b3d5740 1 -- 172.21.15.43:0/490603092 >> 172.21.15.43:0/490603092 conn(0x558efabc43d0 msgr2=0x558efabc47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:46.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.455+0000 7fb89b3d5740 1 -- 172.21.15.43:0/490603092 shutdown_connections 2024-09-19T05:11:46.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.455+0000 7fb89b3d5740 1 -- 172.21.15.43:0/490603092 wait complete. 2024-09-19T05:11:46.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.455+0000 7fb89b3d5740 1 -- start start 2024-09-19T05:11:46.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.455+0000 7fb89b3d5740 1 Processor -- start 2024-09-19T05:11:46.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.456+0000 7fb89b3d5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558efabc7e40 0x558efabcff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:46.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.456+0000 7fb89b3d5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558efabc8760 0x558efabd0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:46.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.456+0000 7fb89b3d5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558efabd3020 0x558efabd09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:46.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.456+0000 7fb89b3d5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558efabd5bf0 con 0x558efabd3020 2024-09-19T05:11:46.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.456+0000 7fb89b3d5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558efaae90f0 con 0x558efabc7e40 2024-09-19T05:11:46.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.456+0000 7fb89b3d5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558efabd5d70 con 0x558efabc8760 2024-09-19T05:11:46.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.456+0000 7fb89b3d5740 10 client.0 _ll_register_callbacks cb 0x558efaad0bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:46.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.456+0000 7fb892800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558efabd3020 0x558efabd09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:46.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.456+0000 7fb891400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558efabc8760 0x558efabd0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:46.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.456+0000 7fb892800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558efabd3020 0x558efabd09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42900/0 (socket says 172.21.15.43:42900) 2024-09-19T05:11:46.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.456+0000 7fb892800640 1 -- 172.21.15.43:0/2290204770 learned_addr learned my addr 172.21.15.43:0/2290204770 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:46.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.456+0000 7fb892800640 1 --2- 172.21.15.43:0/2290204770 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558efabc7e40 0x558efabcff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:46.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.457+0000 7fb891400640 1 -- 172.21.15.43:0/2290204770 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558efabc7e40 msgr2=0x558efabcff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:46.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.457+0000 7fb891400640 1 --2- 172.21.15.43:0/2290204770 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558efabc7e40 0x558efabcff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.457+0000 7fb891400640 1 -- 172.21.15.43:0/2290204770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558efabd3020 msgr2=0x558efabd09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:46.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.457+0000 7fb891400640 1 --2- 172.21.15.43:0/2290204770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558efabd3020 0x558efabd09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.457+0000 7fb891400640 1 -- 172.21.15.43:0/2290204770 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558efa89f3c0 con 0x558efabc8760 2024-09-19T05:11:46.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.457+0000 7fb891400640 1 --2- 172.21.15.43:0/2290204770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558efabc8760 0x558efabd0460 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb88002a9a0 tx=0x7fb880004440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=920ca1f4ae894f87 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:46.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.457+0000 7fb873400640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:11:46.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.457+0000 7fb873400640 1 -- 172.21.15.43:0/2290204770 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb880035040 con 0x558efabc8760 2024-09-19T05:11:46.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.457+0000 7fb89b3d5740 1 -- 172.21.15.43:0/2290204770 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb88c003d10 con 0x558efabc8760 2024-09-19T05:11:46.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.457+0000 7fb873400640 1 -- 172.21.15.43:0/2290204770 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7fb880005450 con 0x558efabc8760 2024-09-19T05:11:46.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.458+0000 7fb873400640 1 -- 172.21.15.43:0/2290204770 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb880005650 con 0x558efabc8760 2024-09-19T05:11:46.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.458+0000 7fb89b3d5740 10 client.26659 fetch_fsmap learned FSMap version 29 2024-09-19T05:11:46.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.458+0000 7fb89b3d5740 1 -- 172.21.15.43:0/2290204770 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558efabe1c90 con 0x558efabc8760 2024-09-19T05:11:46.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.458+0000 7fb873400640 1 -- 172.21.15.43:0/2290204770 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb88c003d10 con 0x558efabc8760 2024-09-19T05:11:46.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.458+0000 7fb873400640 1 -- 172.21.15.43:0/2290204770 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb880031aa0 con 0x558efabc8760 2024-09-19T05:11:46.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.459+0000 7fb89b3d5740 10 client.26659 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:11:46.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.459+0000 7fb89b3d5740 10 client.26659 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-19T05:11:46.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.459+0000 7fb89b3d5740 20 client.26659 populate_metadata read hostname 'smithi043' 2024-09-19T05:11:46.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.459+0000 7fb89b3d5740 1 --2- 172.21.15.43:0/2290204770 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x558efabe30a0 0x558efabe5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:46.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.459+0000 7fb89b3d5740 4 client.26659 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:11:46.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.459+0000 7fb89b3d5740 1 -- 172.21.15.43:0/2290204770 --> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558efab3b950 con 0x558efabe30a0 2024-09-19T05:11:46.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.459+0000 7fb892800640 1 --2- 172.21.15.43:0/2290204770 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x558efabe30a0 0x558efabe5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:46.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.459+0000 7fb892800640 1 --2- 172.21.15.43:0/2290204770 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x558efabe30a0 0x558efabe5490 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a47e77ebc3eee27e server_cookie=995c1c28f70b87b4 in_seq=0 out_seq=0 2024-09-19T05:11:46.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.459+0000 7fb873400640 10 client.26659 ms_handle_connect on v2:172.21.15.151:6836/3960378010 2024-09-19T05:11:46.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.460+0000 7fb873400640 1 -- 172.21.15.43:0/2290204770 <== mds.4 v2:172.21.15.151:6836/3960378010 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558efab3b950 con 0x558efabe30a0 2024-09-19T05:11:46.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.460+0000 7fb873400640 10 client.26659 handle_command_reply: tid=0 2024-09-19T05:11:46.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.460+0000 7fb89b3d5740 1 client.26659 shutdown 2024-09-19T05:11:46.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.461+0000 7fb89b3d5740 1 -- 172.21.15.43:0/2290204770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558efabc8760 msgr2=0x558efabd0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:46.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.461+0000 7fb89b3d5740 1 --2- 172.21.15.43:0/2290204770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558efabc8760 0x558efabd0460 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb88002a9a0 tx=0x7fb880004440 comp rx=0 tx=0).stop 2024-09-19T05:11:46.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.461+0000 7fb89b3d5740 1 -- 172.21.15.43:0/2290204770 shutdown_connections 2024-09-19T05:11:46.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.461+0000 7fb89b3d5740 1 --2- 172.21.15.43:0/2290204770 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x558efabe30a0 0x558efabe5490 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.461+0000 7fb89b3d5740 1 --2- 172.21.15.43:0/2290204770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558efabd3020 0x558efabd09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.461+0000 7fb89b3d5740 1 --2- 172.21.15.43:0/2290204770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558efabc8760 0x558efabd0460 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.461+0000 7fb89b3d5740 1 --2- 172.21.15.43:0/2290204770 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558efabc7e40 0x558efabcff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.461+0000 7fb873400640 0 client.26659 ms_handle_reset on v2:172.21.15.151:6836/3960378010 2024-09-19T05:11:46.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.461+0000 7fb89b3d5740 1 -- 172.21.15.43:0/2290204770 >> 172.21.15.43:0/2290204770 conn(0x558efabc43d0 msgr2=0x558efabca0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:46.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.461+0000 7fb89b3d5740 1 -- 172.21.15.43:0/2290204770 shutdown_connections 2024-09-19T05:11:46.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.461+0000 7fb89b3d5740 1 -- 172.21.15.43:0/2290204770 wait complete. 2024-09-19T05:11:46.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.461+0000 7fb89b3d5740 20 client.26659 trim_cache size 0 max 16384 2024-09-19T05:11:46.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:11:45 smithi096 ceph-mon[35085]: pgmap v218: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-09-19T05:11:46.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.475+0000 7fb89b3d5740 1 Processor -- start 2024-09-19T05:11:46.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.475+0000 7fb89b3d5740 1 -- start start 2024-09-19T05:11:46.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.475+0000 7fb89b3d5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558efabc7e40 0x558efabead60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:46.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.475+0000 7fb89b3d5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558efabc8760 0x558efabeb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:46.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.475+0000 7fb89b3d5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558efabd3020 0x558efabcfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:46.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.475+0000 7fb89b3d5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb8941af900 con 0x558efabd3020 2024-09-19T05:11:46.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.475+0000 7fb89b3d5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb8941b1440 con 0x558efabc8760 2024-09-19T05:11:46.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.475+0000 7fb89b3d5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb8941affd0 con 0x558efabc7e40 2024-09-19T05:11:46.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.476+0000 7fb892800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558efabd3020 0x558efabcfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:46.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.476+0000 7fb892800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558efabc7e40 0x558efabead60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:46.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.476+0000 7fb892800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558efabc7e40 0x558efabead60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:51662/0 (socket says 172.21.15.43:51662) 2024-09-19T05:11:46.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.476+0000 7fb892800640 1 -- 172.21.15.43:0/1181067596 learned_addr learned my addr 172.21.15.43:0/1181067596 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:46.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.476+0000 7fb891400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558efabc8760 0x558efabeb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:46.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.476+0000 7fb892800640 1 -- 172.21.15.43:0/1181067596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558efabc8760 msgr2=0x558efabeb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:46.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.476+0000 7fb892800640 1 --2- 172.21.15.43:0/1181067596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558efabc8760 0x558efabeb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.476+0000 7fb892800640 1 -- 172.21.15.43:0/1181067596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558efabd3020 msgr2=0x558efabcfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:46.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.476+0000 7fb892800640 1 --2- 172.21.15.43:0/1181067596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558efabd3020 0x558efabcfee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.476+0000 7fb892800640 1 -- 172.21.15.43:0/1181067596 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558efabc3c80 con 0x558efabc7e40 2024-09-19T05:11:46.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.476+0000 7fb892800640 1 --2- 172.21.15.43:0/1181067596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558efabc7e40 0x558efabead60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb88c011630 tx=0x7fb88c0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=423259966666f1db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:46.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.477+0000 7fb87e600640 1 -- 172.21.15.43:0/1181067596 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88c025040 con 0x558efabc7e40 2024-09-19T05:11:46.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.477+0000 7fb87e600640 1 -- 172.21.15.43:0/1181067596 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb88c004ca0 con 0x558efabc7e40 2024-09-19T05:11:46.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.477+0000 7fb87e600640 1 -- 172.21.15.43:0/1181067596 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88c005510 con 0x558efabc7e40 2024-09-19T05:11:46.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.477+0000 7fb89b3d5740 1 -- 172.21.15.43:0/1181067596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558efabc7e40 msgr2=0x558efabead60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:46.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.477+0000 7fb89b3d5740 1 --2- 172.21.15.43:0/1181067596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558efabc7e40 0x558efabead60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb88c011630 tx=0x7fb88c0107a0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.477+0000 7fb89b3d5740 1 -- 172.21.15.43:0/1181067596 shutdown_connections 2024-09-19T05:11:46.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.477+0000 7fb89b3d5740 1 --2- 172.21.15.43:0/1181067596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558efabd3020 0x558efabcfee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.477+0000 7fb89b3d5740 1 --2- 172.21.15.43:0/1181067596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558efabc8760 0x558efabeb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.477+0000 7fb89b3d5740 1 --2- 172.21.15.43:0/1181067596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558efabc7e40 0x558efabead60 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.477+0000 7fb89b3d5740 1 -- 172.21.15.43:0/1181067596 >> 172.21.15.43:0/1181067596 conn(0x558efabc43d0 msgr2=0x558efabcf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:46.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.477+0000 7fb89b3d5740 1 -- 172.21.15.43:0/1181067596 shutdown_connections 2024-09-19T05:11:46.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.477+0000 7fb89b3d5740 1 -- 172.21.15.43:0/1181067596 wait complete. 2024-09-19T05:11:46.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.478+0000 7fb89b3d5740 1 -- start start 2024-09-19T05:11:46.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.478+0000 7fb89b3d5740 1 Processor -- start 2024-09-19T05:11:46.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.478+0000 7fb89b3d5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558efabc7e40 0x558efabdb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:46.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.478+0000 7fb89b3d5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558efabc8760 0x558efabd68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:46.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.478+0000 7fb89b3d5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558efabd3020 0x558efabd6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:46.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.478+0000 7fb89b3d5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb8941b0d70 con 0x558efabc8760 2024-09-19T05:11:46.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.478+0000 7fb89b3d5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb8941b06a0 con 0x558efabc7e40 2024-09-19T05:11:46.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.478+0000 7fb89b3d5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb8941ad900 con 0x558efabd3020 2024-09-19T05:11:46.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.478+0000 7fb89b3d5740 10 client.0 _ll_register_callbacks cb 0x7fb880009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:46.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.479+0000 7fb892800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558efabc7e40 0x558efabdb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:46.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.479+0000 7fb892800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558efabc7e40 0x558efabdb550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:50522/0 (socket says 172.21.15.43:50522) 2024-09-19T05:11:46.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.479+0000 7fb892800640 1 -- 172.21.15.43:0/4076960539 learned_addr learned my addr 172.21.15.43:0/4076960539 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:46.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.479+0000 7fb892800640 1 --2- 172.21.15.43:0/4076960539 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558efabd3020 0x558efabd6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:46.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.479+0000 7fb891400640 1 --2- 172.21.15.43:0/4076960539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558efabc8760 0x558efabd68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:46.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.479+0000 7fb892800640 1 -- 172.21.15.43:0/4076960539 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558efabd3020 msgr2=0x558efabd6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:46.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.479+0000 7fb892800640 1 --2- 172.21.15.43:0/4076960539 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558efabd3020 0x558efabd6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.479+0000 7fb892800640 1 -- 172.21.15.43:0/4076960539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558efabc8760 msgr2=0x558efabd68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:46.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.479+0000 7fb892800640 1 --2- 172.21.15.43:0/4076960539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558efabc8760 0x558efabd68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.479+0000 7fb892800640 1 -- 172.21.15.43:0/4076960539 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558efabd7450 con 0x558efabc7e40 2024-09-19T05:11:46.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.479+0000 7fb892800640 1 --2- 172.21.15.43:0/4076960539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558efabc7e40 0x558efabdb550 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb88c011010 tx=0x7fb88c010df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1faffdd9b7c6512c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:46.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.479+0000 7fb873400640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:11:46.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.480+0000 7fb873400640 1 -- 172.21.15.43:0/4076960539 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88c0117a0 con 0x558efabc7e40 2024-09-19T05:11:46.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.480+0000 7fb873400640 1 -- 172.21.15.43:0/4076960539 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7fb88c011940 con 0x558efabc7e40 2024-09-19T05:11:46.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.480+0000 7fb89b3d5740 1 -- 172.21.15.43:0/4076960539 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb88c004ca0 con 0x558efabc7e40 2024-09-19T05:11:46.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.480+0000 7fb873400640 1 -- 172.21.15.43:0/4076960539 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb88c011b60 con 0x558efabc7e40 2024-09-19T05:11:46.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.480+0000 7fb89b3d5740 10 client.26735 fetch_fsmap learned FSMap version 29 2024-09-19T05:11:46.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.480+0000 7fb89b3d5740 1 -- 172.21.15.43:0/4076960539 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558efac4db80 con 0x558efabc7e40 2024-09-19T05:11:46.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.480+0000 7fb873400640 1 -- 172.21.15.43:0/4076960539 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb88c004ca0 con 0x558efabc7e40 2024-09-19T05:11:46.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.480+0000 7fb873400640 1 -- 172.21.15.43:0/4076960539 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb88c02b040 con 0x558efabc7e40 2024-09-19T05:11:46.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.480+0000 7fb89b3d5740 10 client.26735 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:11:46.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.480+0000 7fb89b3d5740 10 client.26735 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-19T05:11:46.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.481+0000 7fb89b3d5740 20 client.26735 populate_metadata read hostname 'smithi043' 2024-09-19T05:11:46.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.481+0000 7fb89b3d5740 1 --2- 172.21.15.43:0/4076960539 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x558efac4e1a0 0x558efac50590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:46.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.481+0000 7fb89b3d5740 4 client.26735 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:11:46.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.481+0000 7fb89b3d5740 1 -- 172.21.15.43:0/4076960539 --> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558efab35e60 con 0x558efac4e1a0 2024-09-19T05:11:46.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.481+0000 7fb891400640 1 --2- 172.21.15.43:0/4076960539 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x558efac4e1a0 0x558efac50590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:46.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.481+0000 7fb891400640 1 --2- 172.21.15.43:0/4076960539 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x558efac4e1a0 0x558efac50590 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a7964f62dc151310 server_cookie=e9b26d9b5bae034d in_seq=0 out_seq=0 2024-09-19T05:11:46.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.481+0000 7fb873400640 10 client.26735 ms_handle_connect on v2:172.21.15.151:6836/3960378010 2024-09-19T05:11:46.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.482+0000 7fb873400640 1 -- 172.21.15.43:0/4076960539 <== mds.4 v2:172.21.15.151:6836/3960378010 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558efab35e60 con 0x558efac4e1a0 2024-09-19T05:11:46.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.482+0000 7fb873400640 10 client.26735 handle_command_reply: tid=0 2024-09-19T05:11:46.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.482+0000 7fb89b3d5740 1 client.26735 shutdown 2024-09-19T05:11:46.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.482+0000 7fb89b3d5740 1 -- 172.21.15.43:0/4076960539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558efabc7e40 msgr2=0x558efabdb550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:46.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.482+0000 7fb89b3d5740 1 --2- 172.21.15.43:0/4076960539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558efabc7e40 0x558efabdb550 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb88c011010 tx=0x7fb88c010df0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.482+0000 7fb89b3d5740 1 -- 172.21.15.43:0/4076960539 shutdown_connections 2024-09-19T05:11:46.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.482+0000 7fb89b3d5740 1 --2- 172.21.15.43:0/4076960539 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x558efac4e1a0 0x558efac50590 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.482+0000 7fb89b3d5740 1 --2- 172.21.15.43:0/4076960539 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558efabd3020 0x558efabd6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.482+0000 7fb89b3d5740 1 --2- 172.21.15.43:0/4076960539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558efabc8760 0x558efabd68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.482+0000 7fb89b3d5740 1 --2- 172.21.15.43:0/4076960539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558efabc7e40 0x558efabdb550 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.482+0000 7fb873400640 0 client.26735 ms_handle_reset on v2:172.21.15.151:6836/3960378010 2024-09-19T05:11:46.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.482+0000 7fb89b3d5740 1 -- 172.21.15.43:0/4076960539 >> 172.21.15.43:0/4076960539 conn(0x558efabc43d0 msgr2=0x558efabc2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:46.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.482+0000 7fb89b3d5740 1 -- 172.21.15.43:0/4076960539 shutdown_connections 2024-09-19T05:11:46.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.482+0000 7fb89b3d5740 1 -- 172.21.15.43:0/4076960539 wait complete. 2024-09-19T05:11:46.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.482+0000 7fb89b3d5740 20 client.26735 trim_cache size 0 max 16384 2024-09-19T05:11:46.498 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:11:46.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.483+0000 7fb893200640 1 -- 172.21.15.43:0/3177305750 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb8780a33b0 msgr2=0x7fb8780a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:46.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.483+0000 7fb893200640 1 --2- 172.21.15.43:0/3177305750 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb8780a33b0 0x7fb8780a5870 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x558efabc5230 tx=0x7fb8840d3bd0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.483+0000 7fb893200640 1 -- 172.21.15.43:0/3177305750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8940745c0 msgr2=0x7fb89419a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:46.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.483+0000 7fb893200640 1 --2- 172.21.15.43:0/3177305750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8940745c0 0x7fb89419a520 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb88400e9f0 tx=0x7fb88400eeb0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.483+0000 7fb893200640 1 -- 172.21.15.43:0/3177305750 shutdown_connections 2024-09-19T05:11:46.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.483+0000 7fb893200640 1 --2- 172.21.15.43:0/3177305750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb89410e240 0x7fb89419edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.483+0000 7fb893200640 1 --2- 172.21.15.43:0/3177305750 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb894074ee0 0x7fb89419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.483+0000 7fb893200640 1 --2- 172.21.15.43:0/3177305750 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb8780a33b0 0x7fb8780a5870 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.483+0000 7fb893200640 1 --2- 172.21.15.43:0/3177305750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8940745c0 0x7fb89419a520 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:46.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.483+0000 7fb893200640 1 -- 172.21.15.43:0/3177305750 >> 172.21.15.43:0/3177305750 conn(0x7fb8940fd700 msgr2=0x7fb8940fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:46.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.483+0000 7fb893200640 1 -- 172.21.15.43:0/3177305750 shutdown_connections 2024-09-19T05:11:46.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:46.483+0000 7fb893200640 1 -- 172.21.15.43:0/3177305750 wait complete. 2024-09-19T05:11:47.440 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T05:11:47.820 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:11:48.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:11:48 smithi043 ceph-mon[30800]: pgmap v219: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:11:48.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.397+0000 7f6cf8a00640 1 -- 172.21.15.43:0/1905801492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cf4074ee0 msgr2=0x7f6cf410def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:48.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.397+0000 7f6cf8a00640 1 --2- 172.21.15.43:0/1905801492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cf4074ee0 0x7f6cf410def0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc009900 tx=0x7f6cdc02a480 comp rx=0 tx=0).stop 2024-09-19T05:11:48.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.397+0000 7f6cf8a00640 1 -- 172.21.15.43:0/1905801492 shutdown_connections 2024-09-19T05:11:48.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.397+0000 7f6cf8a00640 1 --2- 172.21.15.43:0/1905801492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6cf410e430 0x7f6cf4110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:48.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.397+0000 7f6cf8a00640 1 --2- 172.21.15.43:0/1905801492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cf4074ee0 0x7f6cf410def0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:48.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.397+0000 7f6cf8a00640 1 --2- 172.21.15.43:0/1905801492 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6cf40745c0 0x7f6cf40749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:48.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.397+0000 7f6cf8a00640 1 -- 172.21.15.43:0/1905801492 >> 172.21.15.43:0/1905801492 conn(0x7f6cf40fd8b0 msgr2=0x7f6cf40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:48.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.398+0000 7f6cf8a00640 1 -- 172.21.15.43:0/1905801492 shutdown_connections 2024-09-19T05:11:48.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.398+0000 7f6cf8a00640 1 -- 172.21.15.43:0/1905801492 wait complete. 2024-09-19T05:11:48.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.398+0000 7f6cf8a00640 1 Processor -- start 2024-09-19T05:11:48.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.399+0000 7f6cf8a00640 1 -- start start 2024-09-19T05:11:48.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.399+0000 7f6cf8a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6cf40745c0 0x7f6cf419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:48.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.399+0000 7f6cf8a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cf4074ee0 0x7f6cf419ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:48.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.399+0000 7f6cf8a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6cf410e430 0x7f6cf419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:48.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.399+0000 7f6cf8a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf4114170 con 0x7f6cf4074ee0 2024-09-19T05:11:48.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.399+0000 7f6cf8a00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf4113ff0 con 0x7f6cf410e430 2024-09-19T05:11:48.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.399+0000 7f6cf8a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf41142f0 con 0x7f6cf40745c0 2024-09-19T05:11:48.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.399+0000 7f6cebe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cf4074ee0 0x7f6cf419ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:48.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.399+0000 7f6cf1e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6cf40745c0 0x7f6cf419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:48.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.399+0000 7f6cf1e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6cf40745c0 0x7f6cf419a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:51682/0 (socket says 172.21.15.43:51682) 2024-09-19T05:11:48.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.399+0000 7f6cf1e00640 1 -- 172.21.15.43:0/1817591122 learned_addr learned my addr 172.21.15.43:0/1817591122 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:48.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.399+0000 7f6cf2800640 1 --2- 172.21.15.43:0/1817591122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6cf410e430 0x7f6cf419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:48.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.400+0000 7f6cebe00640 1 -- 172.21.15.43:0/1817591122 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6cf40745c0 msgr2=0x7f6cf419a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:48.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.400+0000 7f6cebe00640 1 --2- 172.21.15.43:0/1817591122 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6cf40745c0 0x7f6cf419a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:48.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.400+0000 7f6cebe00640 1 -- 172.21.15.43:0/1817591122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6cf410e430 msgr2=0x7f6cf419efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:48.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.400+0000 7f6cebe00640 1 --2- 172.21.15.43:0/1817591122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6cf410e430 0x7f6cf419efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:48.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.400+0000 7f6cebe00640 1 -- 172.21.15.43:0/1817591122 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf419f680 con 0x7f6cf4074ee0 2024-09-19T05:11:48.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.400+0000 7f6cf1e00640 1 --2- 172.21.15.43:0/1817591122 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6cf40745c0 0x7f6cf419a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:11:48.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.400+0000 7f6cf2800640 1 --2- 172.21.15.43:0/1817591122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6cf410e430 0x7f6cf419efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:11:48.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.400+0000 7f6cebe00640 1 --2- 172.21.15.43:0/1817591122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cf4074ee0 0x7f6cf419ac10 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc005940 tx=0x7f6cdc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:48.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.401+0000 7f6ceaa00640 1 -- 172.21.15.43:0/1817591122 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc03f030 con 0x7f6cf4074ee0 2024-09-19T05:11:48.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.401+0000 7f6ceaa00640 1 -- 172.21.15.43:0/1817591122 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cdc003ad0 con 0x7f6cf4074ee0 2024-09-19T05:11:48.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.401+0000 7f6cf8a00640 1 -- 172.21.15.43:0/1817591122 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cf419f910 con 0x7f6cf4074ee0 2024-09-19T05:11:48.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.401+0000 7f6ceaa00640 1 -- 172.21.15.43:0/1817591122 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc035070 con 0x7f6cf4074ee0 2024-09-19T05:11:48.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.401+0000 7f6cf8a00640 1 -- 172.21.15.43:0/1817591122 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cf41a7250 con 0x7f6cf4074ee0 2024-09-19T05:11:48.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:11:48 smithi151 ceph-mon[35425]: pgmap v219: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:11:48.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.404+0000 7f6ceaa00640 1 -- 172.21.15.43:0/1817591122 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6cdc042030 con 0x7f6cf4074ee0 2024-09-19T05:11:48.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.404+0000 7f6cfb643740 1 Processor -- start 2024-09-19T05:11:48.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.404+0000 7f6cfb643740 1 -- start start 2024-09-19T05:11:48.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.404+0000 7f6cfb643740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563010615c70 0x563010616050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:48.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.404+0000 7f6cfb643740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563010616590 0x563010620910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:48.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.404+0000 7f6cfb643740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563010620e50 0x563010623240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:48.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.404+0000 7f6cfb643740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56301057c3e0 con 0x563010615c70 2024-09-19T05:11:48.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.404+0000 7f6cfb643740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563010523d80 con 0x563010616590 2024-09-19T05:11:48.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.404+0000 7f6cfb643740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5630105783a0 con 0x563010620e50 2024-09-19T05:11:48.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.405+0000 7f6cf2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563010615c70 0x563010616050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:48.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.405+0000 7f6cf2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563010615c70 0x563010616050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42950/0 (socket says 172.21.15.43:42950) 2024-09-19T05:11:48.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.405+0000 7f6cf2800640 1 -- 172.21.15.43:0/4066922278 learned_addr learned my addr 172.21.15.43:0/4066922278 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:48.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.405+0000 7f6cf1e00640 1 --2- 172.21.15.43:0/4066922278 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563010616590 0x563010620910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:48.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.405+0000 7f6cebe00640 1 --2- 172.21.15.43:0/4066922278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563010620e50 0x563010623240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:48.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.405+0000 7f6cf1e00640 1 -- 172.21.15.43:0/4066922278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563010620e50 msgr2=0x563010623240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:48.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.405+0000 7f6cf1e00640 1 --2- 172.21.15.43:0/4066922278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563010620e50 0x563010623240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:48.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.405+0000 7f6cf1e00640 1 -- 172.21.15.43:0/4066922278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563010615c70 msgr2=0x563010616050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:48.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.405+0000 7f6cf1e00640 1 --2- 172.21.15.43:0/4066922278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563010615c70 0x563010616050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:48.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.405+0000 7f6cf1e00640 1 -- 172.21.15.43:0/4066922278 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563010542ae0 con 0x563010616590 2024-09-19T05:11:48.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.406+0000 7f6cf1e00640 1 --2- 172.21.15.43:0/4066922278 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563010616590 0x563010620910 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6ce40058c0 tx=0x7f6ce40109e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ec937401d70bdd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:48.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.406+0000 7f6ceaa00640 1 --2- 172.21.15.43:0/1817591122 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f6cd80a33b0 0x7f6cd80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:48.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.406+0000 7f6ce3000640 1 -- 172.21.15.43:0/4066922278 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce40117c0 con 0x563010616590 2024-09-19T05:11:48.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.406+0000 7f6ce3000640 1 -- 172.21.15.43:0/4066922278 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ce4011e40 con 0x563010616590 2024-09-19T05:11:48.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.406+0000 7f6ce3000640 1 -- 172.21.15.43:0/4066922278 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce400d340 con 0x563010616590 2024-09-19T05:11:48.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.406+0000 7f6ceaa00640 1 -- 172.21.15.43:0/1817591122 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f6cdc0efff0 con 0x7f6cf4074ee0 2024-09-19T05:11:48.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.406+0000 7f6cf2800640 1 --2- 172.21.15.43:0/1817591122 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f6cd80a33b0 0x7f6cd80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:48.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.407+0000 7f6cfb643740 1 -- 172.21.15.43:0/4066922278 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563010616590 msgr2=0x563010620910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:48.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.407+0000 7f6cfb643740 1 --2- 172.21.15.43:0/4066922278 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563010616590 0x563010620910 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6ce40058c0 tx=0x7f6ce40109e0 comp rx=0 tx=0).stop 2024-09-19T05:11:48.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.407+0000 7f6cf2800640 1 --2- 172.21.15.43:0/1817591122 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f6cd80a33b0 0x7f6cd80a5870 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6cec009fd0 tx=0x7f6cec009500 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:48.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.407+0000 7f6cfb643740 1 -- 172.21.15.43:0/4066922278 shutdown_connections 2024-09-19T05:11:48.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.407+0000 7f6cfb643740 1 --2- 172.21.15.43:0/4066922278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563010620e50 0x563010623240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:48.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.407+0000 7f6cfb643740 1 --2- 172.21.15.43:0/4066922278 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563010616590 0x563010620910 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:48.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.407+0000 7f6cfb643740 1 --2- 172.21.15.43:0/4066922278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563010615c70 0x563010616050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:48.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.407+0000 7f6cfb643740 1 -- 172.21.15.43:0/4066922278 >> 172.21.15.43:0/4066922278 conn(0x563010612200 msgr2=0x5630106125e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:48.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.407+0000 7f6cfb643740 1 -- 172.21.15.43:0/4066922278 shutdown_connections 2024-09-19T05:11:48.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.407+0000 7f6cfb643740 1 -- 172.21.15.43:0/4066922278 wait complete. 2024-09-19T05:11:48.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.407+0000 7f6cfb643740 1 -- start start 2024-09-19T05:11:48.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.408+0000 7f6cfb643740 1 Processor -- start 2024-09-19T05:11:48.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.408+0000 7f6cfb643740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563010615c70 0x56301061dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:48.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.408+0000 7f6cfb643740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563010616590 0x56301061e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:48.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.408+0000 7f6cfb643740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563010620e50 0x56301061e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:48.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.408+0000 7f6cfb643740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563010623a20 con 0x563010616590 2024-09-19T05:11:48.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.408+0000 7f6cfb643740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5630105370f0 con 0x563010615c70 2024-09-19T05:11:48.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.408+0000 7f6cfb643740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563010623ba0 con 0x563010620e50 2024-09-19T05:11:48.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.408+0000 7f6cfb643740 10 client.0 _ll_register_callbacks cb 0x56301051ebd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:48.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.408+0000 7f6cebe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563010616590 0x56301061e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:48.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.409+0000 7f6cf1e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563010615c70 0x56301061dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:48.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.409+0000 7f6cebe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563010616590 0x56301061e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42964/0 (socket says 172.21.15.43:42964) 2024-09-19T05:11:48.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.409+0000 7f6cf1e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563010615c70 0x56301061dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:50556/0 (socket says 172.21.15.43:50556) 2024-09-19T05:11:48.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.409+0000 7f6cebe00640 1 -- 172.21.15.43:0/1881772619 learned_addr learned my addr 172.21.15.43:0/1881772619 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:48.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.409+0000 7f6cf2800640 1 --2- 172.21.15.43:0/1881772619 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563010620e50 0x56301061e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:48.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.409+0000 7f6cf1e00640 1 -- 172.21.15.43:0/1881772619 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563010620e50 msgr2=0x56301061e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:48.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.409+0000 7f6cf1e00640 1 --2- 172.21.15.43:0/1881772619 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563010620e50 0x56301061e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:48.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.409+0000 7f6cf1e00640 1 -- 172.21.15.43:0/1881772619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563010616590 msgr2=0x56301061e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:48.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.409+0000 7f6cf1e00640 1 --2- 172.21.15.43:0/1881772619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563010616590 0x56301061e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:48.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.409+0000 7f6cf1e00640 1 -- 172.21.15.43:0/1881772619 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630102ed3c0 con 0x563010615c70 2024-09-19T05:11:48.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.409+0000 7f6cf1e00640 1 --2- 172.21.15.43:0/1881772619 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563010615c70 0x56301061dcb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4000c00 tx=0x7f6ce400eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=698952fec72bed90 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:48.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.409+0000 7f6cd7400640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:11:48.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.410+0000 7f6cd7400640 1 -- 172.21.15.43:0/1881772619 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce400e700 con 0x563010615c70 2024-09-19T05:11:48.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.410+0000 7f6cd7400640 1 -- 172.21.15.43:0/1881772619 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f6ce4004050 con 0x563010615c70 2024-09-19T05:11:48.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.410+0000 7f6cfb643740 1 -- 172.21.15.43:0/1881772619 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563010589950 con 0x563010615c70 2024-09-19T05:11:48.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.410+0000 7f6cd7400640 1 -- 172.21.15.43:0/1881772619 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6ce400e8a0 con 0x563010615c70 2024-09-19T05:11:48.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.410+0000 7f6cfb643740 10 client.26747 fetch_fsmap learned FSMap version 29 2024-09-19T05:11:48.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.410+0000 7f6cfb643740 1 -- 172.21.15.43:0/1881772619 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56301062fa10 con 0x563010615c70 2024-09-19T05:11:48.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.410+0000 7f6cd7400640 1 -- 172.21.15.43:0/1881772619 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x563010589950 con 0x563010615c70 2024-09-19T05:11:48.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.411+0000 7f6cd7400640 1 -- 172.21.15.43:0/1881772619 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6ce4011800 con 0x563010615c70 2024-09-19T05:11:48.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.411+0000 7f6cfb643740 10 client.26747 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:11:48.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.411+0000 7f6cfb643740 10 client.26747 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T05:11:48.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.411+0000 7f6cfb643740 20 client.26747 populate_metadata read hostname 'smithi043' 2024-09-19T05:11:48.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.411+0000 7f6cfb643740 1 --2- 172.21.15.43:0/1881772619 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x563010630e10 0x563010633200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:48.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.411+0000 7f6cfb643740 4 client.26747 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:11:48.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.411+0000 7f6cfb643740 1 -- 172.21.15.43:0/1881772619 --> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563010583e60 con 0x563010630e10 2024-09-19T05:11:48.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.411+0000 7f6cebe00640 1 --2- 172.21.15.43:0/1881772619 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x563010630e10 0x563010633200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:48.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.412+0000 7f6cebe00640 1 --2- 172.21.15.43:0/1881772619 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x563010630e10 0x563010633200 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e906c8e154c4f533 server_cookie=b0c3b04abbc9d78a in_seq=0 out_seq=0 2024-09-19T05:11:48.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.412+0000 7f6cd7400640 10 client.26747 ms_handle_connect on v2:172.21.15.96:6836/3978388286 2024-09-19T05:11:48.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.413+0000 7f6cd7400640 1 -- 172.21.15.43:0/1881772619 <== mds.3 v2:172.21.15.96:6836/3978388286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563010583e60 con 0x563010630e10 2024-09-19T05:11:48.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.413+0000 7f6cd7400640 10 client.26747 handle_command_reply: tid=0 2024-09-19T05:11:48.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.413+0000 7f6cfb643740 1 client.26747 shutdown 2024-09-19T05:11:48.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.413+0000 7f6cfb643740 1 -- 172.21.15.43:0/1881772619 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563010615c70 msgr2=0x56301061dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:48.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.413+0000 7f6cfb643740 1 --2- 172.21.15.43:0/1881772619 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563010615c70 0x56301061dcb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4000c00 tx=0x7f6ce400eef0 comp rx=0 tx=0).stop 2024-09-19T05:11:48.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.413+0000 7f6cfb643740 1 -- 172.21.15.43:0/1881772619 shutdown_connections 2024-09-19T05:11:48.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.413+0000 7f6cfb643740 1 --2- 172.21.15.43:0/1881772619 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x563010630e10 0x563010633200 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:48.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.413+0000 7f6cfb643740 1 --2- 172.21.15.43:0/1881772619 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563010620e50 0x56301061e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:48.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.413+0000 7f6cfb643740 1 --2- 172.21.15.43:0/1881772619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563010616590 0x56301061e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:48.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.413+0000 7f6cfb643740 1 --2- 172.21.15.43:0/1881772619 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563010615c70 0x56301061dcb0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:48.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.413+0000 7f6cfb643740 1 -- 172.21.15.43:0/1881772619 >> 172.21.15.43:0/1881772619 conn(0x563010612200 msgr2=0x563010611380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:48.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.413+0000 7f6cd7400640 0 client.26747 ms_handle_reset on v2:172.21.15.96:6836/3978388286 2024-09-19T05:11:48.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.413+0000 7f6cfb643740 1 -- 172.21.15.43:0/1881772619 shutdown_connections 2024-09-19T05:11:48.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.413+0000 7f6cfb643740 1 -- 172.21.15.43:0/1881772619 wait complete. 2024-09-19T05:11:48.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.413+0000 7f6cfb643740 20 client.26747 trim_cache size 0 max 16384 2024-09-19T05:11:48.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.427+0000 7f6cfb643740 1 Processor -- start 2024-09-19T05:11:48.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.428+0000 7f6cfb643740 1 -- start start 2024-09-19T05:11:48.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.428+0000 7f6cfb643740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563010615c70 0x56301061fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:48.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.428+0000 7f6cfb643740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563010616590 0x5630106389d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:48.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.428+0000 7f6cfb643740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563010620e50 0x563010638f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:48.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.428+0000 7f6cfb643740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf41afb50 con 0x563010620e50 2024-09-19T05:11:48.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.428+0000 7f6cfb643740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf41b1690 con 0x563010616590 2024-09-19T05:11:48.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.428+0000 7f6cfb643740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf41b0220 con 0x563010615c70 2024-09-19T05:11:48.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.428+0000 7f6cf1e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563010615c70 0x56301061fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:48.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.428+0000 7f6cf2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563010620e50 0x563010638f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:48.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.428+0000 7f6cf1e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563010615c70 0x56301061fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:51700/0 (socket says 172.21.15.43:51700) 2024-09-19T05:11:48.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.428+0000 7f6cebe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563010616590 0x5630106389d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:48.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.428+0000 7f6cf1e00640 1 -- 172.21.15.43:0/1215638664 learned_addr learned my addr 172.21.15.43:0/1215638664 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:48.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.429+0000 7f6cf1e00640 1 -- 172.21.15.43:0/1215638664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563010616590 msgr2=0x5630106389d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:48.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.429+0000 7f6cf1e00640 1 --2- 172.21.15.43:0/1215638664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563010616590 0x5630106389d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:48.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.429+0000 7f6cf1e00640 1 -- 172.21.15.43:0/1215638664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563010620e50 msgr2=0x563010638f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:48.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.429+0000 7f6cf1e00640 1 --2- 172.21.15.43:0/1215638664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563010620e50 0x563010638f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:48.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.429+0000 7f6cf1e00640 1 -- 172.21.15.43:0/1215638664 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563010612030 con 0x563010615c70 2024-09-19T05:11:48.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.429+0000 7f6cf1e00640 1 --2- 172.21.15.43:0/1215638664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563010615c70 0x56301061fc20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4011bf0 tx=0x7f6ce400b700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7b57fa9dacd5cb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:48.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.429+0000 7f6ce2600640 1 -- 172.21.15.43:0/1215638664 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce400e8a0 con 0x563010615c70 2024-09-19T05:11:48.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.429+0000 7f6ce2600640 1 -- 172.21.15.43:0/1215638664 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ce4004070 con 0x563010615c70 2024-09-19T05:11:48.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.429+0000 7f6ce2600640 1 -- 172.21.15.43:0/1215638664 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4003850 con 0x563010615c70 2024-09-19T05:11:48.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.430+0000 7f6cfb643740 1 -- 172.21.15.43:0/1215638664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563010615c70 msgr2=0x56301061fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:48.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.430+0000 7f6cfb643740 1 --2- 172.21.15.43:0/1215638664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563010615c70 0x56301061fc20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4011bf0 tx=0x7f6ce400b700 comp rx=0 tx=0).stop 2024-09-19T05:11:48.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.430+0000 7f6cfb643740 1 -- 172.21.15.43:0/1215638664 shutdown_connections 2024-09-19T05:11:48.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.430+0000 7f6cfb643740 1 --2- 172.21.15.43:0/1215638664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563010620e50 0x563010638f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:48.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.430+0000 7f6cfb643740 1 --2- 172.21.15.43:0/1215638664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563010616590 0x5630106389d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:48.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.430+0000 7f6cfb643740 1 --2- 172.21.15.43:0/1215638664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563010615c70 0x56301061fc20 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:48.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.430+0000 7f6cfb643740 1 -- 172.21.15.43:0/1215638664 >> 172.21.15.43:0/1215638664 conn(0x563010612200 msgr2=0x56301061f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:48.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.430+0000 7f6cfb643740 1 -- 172.21.15.43:0/1215638664 shutdown_connections 2024-09-19T05:11:48.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.430+0000 7f6cfb643740 1 -- 172.21.15.43:0/1215638664 wait complete. 2024-09-19T05:11:48.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.431+0000 7f6cfb643740 1 -- start start 2024-09-19T05:11:48.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.431+0000 7f6cfb643740 1 Processor -- start 2024-09-19T05:11:48.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.431+0000 7f6cfb643740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563010615c70 0x563010623e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:48.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.431+0000 7f6cfb643740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563010616590 0x563010624390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:48.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.431+0000 7f6cfb643740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563010620e50 0x5630106248d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:48.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.431+0000 7f6cfb643740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf41b0fc0 con 0x563010615c70 2024-09-19T05:11:48.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.431+0000 7f6cfb643740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf41b08f0 con 0x563010616590 2024-09-19T05:11:48.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.431+0000 7f6cfb643740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf41adb50 con 0x563010620e50 2024-09-19T05:11:48.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.431+0000 7f6cfb643740 10 client.0 _ll_register_callbacks cb 0x7f6cec009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:48.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.432+0000 7f6cf1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563010615c70 0x563010623e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:48.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.432+0000 7f6cebe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563010616590 0x563010624390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:48.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.432+0000 7f6cf2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563010620e50 0x5630106248d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:48.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.432+0000 7f6cebe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563010616590 0x563010624390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:50572/0 (socket says 172.21.15.43:50572) 2024-09-19T05:11:48.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.432+0000 7f6cebe00640 1 -- 172.21.15.43:0/2303823038 learned_addr learned my addr 172.21.15.43:0/2303823038 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:48.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.432+0000 7f6cebe00640 1 -- 172.21.15.43:0/2303823038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563010620e50 msgr2=0x5630106248d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:48.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.432+0000 7f6cebe00640 1 --2- 172.21.15.43:0/2303823038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563010620e50 0x5630106248d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:48.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.432+0000 7f6cebe00640 1 -- 172.21.15.43:0/2303823038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563010615c70 msgr2=0x563010623e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:48.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.432+0000 7f6cebe00640 1 --2- 172.21.15.43:0/2303823038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563010615c70 0x563010623e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:48.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.432+0000 7f6cebe00640 1 -- 172.21.15.43:0/2303823038 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56301060d770 con 0x563010616590 2024-09-19T05:11:48.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.432+0000 7f6cebe00640 1 --2- 172.21.15.43:0/2303823038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563010616590 0x563010624390 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc031b80 tx=0x7f6cdc0f3ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9efce0b75c194a2b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:48.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.432+0000 7f6cd7400640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:11:48.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.432+0000 7f6cd7400640 1 -- 172.21.15.43:0/2303823038 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc0ede60 con 0x563010616590 2024-09-19T05:11:48.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.432+0000 7f6cd7400640 1 -- 172.21.15.43:0/2303823038 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f6cdc0ba650 con 0x563010616590 2024-09-19T05:11:48.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.432+0000 7f6cfb643740 1 -- 172.21.15.43:0/2303823038 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ce4004070 con 0x563010616590 2024-09-19T05:11:48.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.433+0000 7f6cd7400640 1 -- 172.21.15.43:0/2303823038 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6cdc049040 con 0x563010616590 2024-09-19T05:11:48.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.433+0000 7f6cfb643740 10 client.26753 fetch_fsmap learned FSMap version 29 2024-09-19T05:11:48.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.433+0000 7f6cfb643740 1 -- 172.21.15.43:0/2303823038 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56301069adc0 con 0x563010616590 2024-09-19T05:11:48.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.433+0000 7f6cd7400640 1 -- 172.21.15.43:0/2303823038 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ce4004070 con 0x563010616590 2024-09-19T05:11:48.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.433+0000 7f6cd7400640 1 -- 172.21.15.43:0/2303823038 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6cdc04c060 con 0x563010616590 2024-09-19T05:11:48.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.433+0000 7f6cfb643740 10 client.26753 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:11:48.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.433+0000 7f6cfb643740 10 client.26753 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T05:11:48.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.433+0000 7f6cfb643740 20 client.26753 populate_metadata read hostname 'smithi043' 2024-09-19T05:11:48.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.433+0000 7f6cfb643740 1 --2- 172.21.15.43:0/2303823038 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x56301069b570 0x56301069d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:48.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.433+0000 7f6cfb643740 4 client.26753 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:11:48.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.433+0000 7f6cfb643740 1 -- 172.21.15.43:0/2303823038 --> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563010500010 con 0x56301069b570 2024-09-19T05:11:48.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.434+0000 7f6cf1e00640 1 --2- 172.21.15.43:0/2303823038 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x56301069b570 0x56301069d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:48.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.434+0000 7f6cf1e00640 1 --2- 172.21.15.43:0/2303823038 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x56301069b570 0x56301069d990 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=314f2da1c53dccb5 server_cookie=8e031b3aa3d72b9f in_seq=0 out_seq=0 2024-09-19T05:11:48.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.434+0000 7f6cd7400640 10 client.26753 ms_handle_connect on v2:172.21.15.96:6836/3978388286 2024-09-19T05:11:48.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.434+0000 7f6cd7400640 1 -- 172.21.15.43:0/2303823038 <== mds.3 v2:172.21.15.96:6836/3978388286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563010500010 con 0x56301069b570 2024-09-19T05:11:48.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.434+0000 7f6cd7400640 10 client.26753 handle_command_reply: tid=0 2024-09-19T05:11:48.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.434+0000 7f6cfb643740 1 client.26753 shutdown 2024-09-19T05:11:48.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.434+0000 7f6cfb643740 1 -- 172.21.15.43:0/2303823038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563010616590 msgr2=0x563010624390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:48.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.434+0000 7f6cfb643740 1 --2- 172.21.15.43:0/2303823038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563010616590 0x563010624390 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc031b80 tx=0x7f6cdc0f3ae0 comp rx=0 tx=0).stop 2024-09-19T05:11:48.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.435+0000 7f6cfb643740 1 -- 172.21.15.43:0/2303823038 shutdown_connections 2024-09-19T05:11:48.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.435+0000 7f6cfb643740 1 --2- 172.21.15.43:0/2303823038 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x56301069b570 0x56301069d990 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:48.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.435+0000 7f6cfb643740 1 --2- 172.21.15.43:0/2303823038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563010620e50 0x5630106248d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:48.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.435+0000 7f6cfb643740 1 --2- 172.21.15.43:0/2303823038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563010616590 0x563010624390 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:48.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.435+0000 7f6cfb643740 1 --2- 172.21.15.43:0/2303823038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563010615c70 0x563010623e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:48.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.435+0000 7f6cfb643740 1 -- 172.21.15.43:0/2303823038 >> 172.21.15.43:0/2303823038 conn(0x563010612200 msgr2=0x563010611380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:48.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.435+0000 7f6cd7400640 0 client.26753 ms_handle_reset on v2:172.21.15.96:6836/3978388286 2024-09-19T05:11:48.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.435+0000 7f6cfb643740 1 -- 172.21.15.43:0/2303823038 shutdown_connections 2024-09-19T05:11:48.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.435+0000 7f6cfb643740 1 -- 172.21.15.43:0/2303823038 wait complete. 2024-09-19T05:11:48.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.435+0000 7f6cfb643740 20 client.26753 trim_cache size 0 max 16384 2024-09-19T05:11:48.440 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:11:48.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.435+0000 7f6cf8a00640 1 -- 172.21.15.43:0/1817591122 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f6cd80a33b0 msgr2=0x7f6cd80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:48.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.435+0000 7f6cf8a00640 1 --2- 172.21.15.43:0/1817591122 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f6cd80a33b0 0x7f6cd80a5870 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6cec009fd0 tx=0x7f6cec009500 comp rx=0 tx=0).stop 2024-09-19T05:11:48.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.435+0000 7f6cf8a00640 1 -- 172.21.15.43:0/1817591122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cf4074ee0 msgr2=0x7f6cf419ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:48.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.435+0000 7f6cf8a00640 1 --2- 172.21.15.43:0/1817591122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cf4074ee0 0x7f6cf419ac10 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc005940 tx=0x7f6cdc004270 comp rx=0 tx=0).stop 2024-09-19T05:11:48.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.436+0000 7f6cf8a00640 1 -- 172.21.15.43:0/1817591122 shutdown_connections 2024-09-19T05:11:48.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.436+0000 7f6cf8a00640 1 --2- 172.21.15.43:0/1817591122 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f6cd80a33b0 0x7f6cd80a5870 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:48.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.436+0000 7f6cf8a00640 1 --2- 172.21.15.43:0/1817591122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6cf410e430 0x7f6cf419efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:48.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.436+0000 7f6cf8a00640 1 --2- 172.21.15.43:0/1817591122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cf4074ee0 0x7f6cf419ac10 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:48.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.436+0000 7f6cf8a00640 1 --2- 172.21.15.43:0/1817591122 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6cf40745c0 0x7f6cf419a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:48.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.436+0000 7f6cf8a00640 1 -- 172.21.15.43:0/1817591122 >> 172.21.15.43:0/1817591122 conn(0x7f6cf40fd8b0 msgr2=0x7f6cf40fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:48.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.436+0000 7f6cf8a00640 1 -- 172.21.15.43:0/1817591122 shutdown_connections 2024-09-19T05:11:48.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:48.436+0000 7f6cf8a00640 1 -- 172.21.15.43:0/1817591122 wait complete. 2024-09-19T05:11:48.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:11:48 smithi096 ceph-mon[35085]: pgmap v219: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:11:49.197 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T05:11:49.575 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:11:50.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.135+0000 7f616bc00640 1 -- 172.21.15.43:0/4131108196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6164103ff0 msgr2=0x7f61641043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:50.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.135+0000 7f616bc00640 1 --2- 172.21.15.43:0/4131108196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6164103ff0 0x7f61641043d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6158009900 tx=0x7f615802a3f0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.135+0000 7f616bc00640 1 -- 172.21.15.43:0/4131108196 shutdown_connections 2024-09-19T05:11:50.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.135+0000 7f616bc00640 1 --2- 172.21.15.43:0/4131108196 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6164109120 0x7f6164110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.135+0000 7f616bc00640 1 --2- 172.21.15.43:0/4131108196 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61641049a0 0x7f61641089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.135+0000 7f616bc00640 1 --2- 172.21.15.43:0/4131108196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6164103ff0 0x7f61641043d0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.135+0000 7f616bc00640 1 -- 172.21.15.43:0/4131108196 >> 172.21.15.43:0/4131108196 conn(0x7f61640ffa90 msgr2=0x7f6164101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:50.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.135+0000 7f616bc00640 1 -- 172.21.15.43:0/4131108196 shutdown_connections 2024-09-19T05:11:50.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.135+0000 7f616bc00640 1 -- 172.21.15.43:0/4131108196 wait complete. 2024-09-19T05:11:50.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.136+0000 7f616bc00640 1 Processor -- start 2024-09-19T05:11:50.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.137+0000 7f616bc00640 1 -- start start 2024-09-19T05:11:50.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.137+0000 7f616bc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6164103ff0 0x7f616419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:50.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.137+0000 7f616bc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61641049a0 0x7f616419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:50.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.137+0000 7f616bc00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6164109120 0x7f616419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:50.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.137+0000 7f616bc00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6164114190 con 0x7f61641049a0 2024-09-19T05:11:50.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.137+0000 7f616bc00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6164114010 con 0x7f6164109120 2024-09-19T05:11:50.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.137+0000 7f616bc00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6164114310 con 0x7f6164103ff0 2024-09-19T05:11:50.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.137+0000 7f6163e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61641049a0 0x7f616419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:50.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.137+0000 7f6169200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6164103ff0 0x7f616419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:50.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.137+0000 7f6163e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61641049a0 0x7f616419abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42988/0 (socket says 172.21.15.43:42988) 2024-09-19T05:11:50.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.137+0000 7f6169200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6164103ff0 0x7f616419a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:51716/0 (socket says 172.21.15.43:51716) 2024-09-19T05:11:50.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.137+0000 7f6163e00640 1 -- 172.21.15.43:0/3041437923 learned_addr learned my addr 172.21.15.43:0/3041437923 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:50.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.138+0000 7f6169c00640 1 --2- 172.21.15.43:0/3041437923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6164109120 0x7f616419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:50.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.138+0000 7f6163e00640 1 -- 172.21.15.43:0/3041437923 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6164103ff0 msgr2=0x7f616419a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:50.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.138+0000 7f6163e00640 1 --2- 172.21.15.43:0/3041437923 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6164103ff0 0x7f616419a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.138+0000 7f6163e00640 1 -- 172.21.15.43:0/3041437923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6164109120 msgr2=0x7f616419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:50.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.138+0000 7f6163e00640 1 --2- 172.21.15.43:0/3041437923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6164109120 0x7f616419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.138+0000 7f6163e00640 1 -- 172.21.15.43:0/3041437923 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f616419f640 con 0x7f61641049a0 2024-09-19T05:11:50.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.138+0000 7f6163e00640 1 --2- 172.21.15.43:0/3041437923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61641049a0 0x7f616419abd0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f615000e9f0 tx=0x7f615000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:50.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.138+0000 7f6161600640 1 -- 172.21.15.43:0/3041437923 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615000cd80 con 0x7f61641049a0 2024-09-19T05:11:50.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.139+0000 7f6161600640 1 -- 172.21.15.43:0/3041437923 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6150005500 con 0x7f61641049a0 2024-09-19T05:11:50.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.139+0000 7f616bc00640 1 -- 172.21.15.43:0/3041437923 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f616419f930 con 0x7f61641049a0 2024-09-19T05:11:50.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.139+0000 7f6161600640 1 -- 172.21.15.43:0/3041437923 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6150010690 con 0x7f61641049a0 2024-09-19T05:11:50.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.139+0000 7f616bc00640 1 -- 172.21.15.43:0/3041437923 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61641a7210 con 0x7f61641049a0 2024-09-19T05:11:50.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.143+0000 7f616e5c9740 1 Processor -- start 2024-09-19T05:11:50.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.143+0000 7f616e5c9740 1 -- start start 2024-09-19T05:11:50.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.144+0000 7f616e5c9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7e531ac70 0x55a7e531b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:50.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.144+0000 7f616e5c9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e531b590 0x55a7e5325910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:50.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.144+0000 7f616e5c9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7e5325e50 0x55a7e5328240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:50.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.144+0000 7f616e5c9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a7e52813e0 con 0x55a7e531b590 2024-09-19T05:11:50.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.144+0000 7f616e5c9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a7e5228d80 con 0x55a7e531ac70 2024-09-19T05:11:50.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.144+0000 7f616e5c9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a7e527d3a0 con 0x55a7e5325e50 2024-09-19T05:11:50.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.144+0000 7f6169200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e531b590 0x55a7e5325910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:50.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.144+0000 7f6161600640 1 -- 172.21.15.43:0/3041437923 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f61500046c0 con 0x7f61641049a0 2024-09-19T05:11:50.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.144+0000 7f6169200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e531b590 0x55a7e5325910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42998/0 (socket says 172.21.15.43:42998) 2024-09-19T05:11:50.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.144+0000 7f6169200640 1 -- 172.21.15.43:0/3678483399 learned_addr learned my addr 172.21.15.43:0/3678483399 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:50.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.144+0000 7f6169c00640 1 --2- 172.21.15.43:0/3678483399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7e531ac70 0x55a7e531b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:50.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.144+0000 7f6163e00640 1 --2- 172.21.15.43:0/3678483399 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7e5325e50 0x55a7e5328240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:50.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.145+0000 7f6169200640 1 -- 172.21.15.43:0/3678483399 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7e5325e50 msgr2=0x55a7e5328240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:50.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.145+0000 7f6169200640 1 --2- 172.21.15.43:0/3678483399 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7e5325e50 0x55a7e5328240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.145+0000 7f6169200640 1 -- 172.21.15.43:0/3678483399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7e531ac70 msgr2=0x55a7e531b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:50.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.145+0000 7f6169200640 1 --2- 172.21.15.43:0/3678483399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7e531ac70 0x55a7e531b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.145+0000 7f6169200640 1 -- 172.21.15.43:0/3678483399 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7e5247ae0 con 0x55a7e531b590 2024-09-19T05:11:50.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.145+0000 7f6169c00640 1 --2- 172.21.15.43:0/3678483399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7e531ac70 0x55a7e531b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:11:50.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.145+0000 7f6169200640 1 --2- 172.21.15.43:0/3678483399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e531b590 0x55a7e5325910 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f6158009900 tx=0x7f6158005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511d59ec77815daa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:50.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.145+0000 7f6156000640 1 -- 172.21.15.43:0/3678483399 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6158035040 con 0x55a7e531b590 2024-09-19T05:11:50.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.145+0000 7f6156000640 1 -- 172.21.15.43:0/3678483399 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6158004400 con 0x55a7e531b590 2024-09-19T05:11:50.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.145+0000 7f6156000640 1 -- 172.21.15.43:0/3678483399 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61580035f0 con 0x55a7e531b590 2024-09-19T05:11:50.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.146+0000 7f6161600640 1 --2- 172.21.15.43:0/3041437923 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f614c0a3430 0x7f614c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:50.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.146+0000 7f6161600640 1 -- 172.21.15.43:0/3041437923 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f61500d0110 con 0x7f61641049a0 2024-09-19T05:11:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.146+0000 7f616e5c9740 1 -- 172.21.15.43:0/3678483399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e531b590 msgr2=0x55a7e5325910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.146+0000 7f616e5c9740 1 --2- 172.21.15.43:0/3678483399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e531b590 0x55a7e5325910 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f6158009900 tx=0x7f6158005de0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.146+0000 7f6169c00640 1 --2- 172.21.15.43:0/3041437923 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f614c0a3430 0x7f614c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.147+0000 7f616e5c9740 1 -- 172.21.15.43:0/3678483399 shutdown_connections 2024-09-19T05:11:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.147+0000 7f616e5c9740 1 --2- 172.21.15.43:0/3678483399 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7e5325e50 0x55a7e5328240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.147+0000 7f616e5c9740 1 --2- 172.21.15.43:0/3678483399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e531b590 0x55a7e5325910 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.147+0000 7f616e5c9740 1 --2- 172.21.15.43:0/3678483399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7e531ac70 0x55a7e531b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.147+0000 7f616e5c9740 1 -- 172.21.15.43:0/3678483399 >> 172.21.15.43:0/3678483399 conn(0x55a7e5317200 msgr2=0x55a7e53175e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.147+0000 7f616e5c9740 1 -- 172.21.15.43:0/3678483399 shutdown_connections 2024-09-19T05:11:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.147+0000 7f616e5c9740 1 -- 172.21.15.43:0/3678483399 wait complete. 2024-09-19T05:11:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.147+0000 7f6169c00640 1 --2- 172.21.15.43:0/3041437923 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f614c0a3430 0x7f614c0a58f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f615c009d40 tx=0x7f615c009500 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:50.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.147+0000 7f616e5c9740 1 -- start start 2024-09-19T05:11:50.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.148+0000 7f616e5c9740 1 Processor -- start 2024-09-19T05:11:50.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.148+0000 7f616e5c9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7e531ac70 0x55a7e5322cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:50.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.148+0000 7f616e5c9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7e531b590 0x55a7e53231f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:50.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.148+0000 7f616e5c9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e5325e50 0x55a7e5323730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:50.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.148+0000 7f616e5c9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a7e5328a20 con 0x55a7e5325e50 2024-09-19T05:11:50.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.148+0000 7f616e5c9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a7e523c0f0 con 0x55a7e531ac70 2024-09-19T05:11:50.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.148+0000 7f616e5c9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a7e5328ba0 con 0x55a7e531b590 2024-09-19T05:11:50.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.148+0000 7f616e5c9740 10 client.0 _ll_register_callbacks cb 0x55a7e5223bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:50.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.148+0000 7f6169200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7e531ac70 0x55a7e5322cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:50.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.148+0000 7f6169c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e5325e50 0x55a7e5323730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:50.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.148+0000 7f6163e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7e531b590 0x55a7e53231f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:50.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.148+0000 7f6169200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7e531ac70 0x55a7e5322cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:50598/0 (socket says 172.21.15.43:50598) 2024-09-19T05:11:50.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.148+0000 7f6169200640 1 -- 172.21.15.43:0/1339299654 learned_addr learned my addr 172.21.15.43:0/1339299654 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:50.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.149+0000 7f6169200640 1 -- 172.21.15.43:0/1339299654 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7e531b590 msgr2=0x55a7e53231f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:50.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.149+0000 7f6169200640 1 --2- 172.21.15.43:0/1339299654 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7e531b590 0x55a7e53231f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.149+0000 7f6169200640 1 -- 172.21.15.43:0/1339299654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e5325e50 msgr2=0x55a7e5323730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:50.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.149+0000 7f6169200640 1 --2- 172.21.15.43:0/1339299654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e5325e50 0x55a7e5323730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.149+0000 7f6169200640 1 -- 172.21.15.43:0/1339299654 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7e4ff23c0 con 0x55a7e531ac70 2024-09-19T05:11:50.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.149+0000 7f6169200640 1 --2- 172.21.15.43:0/1339299654 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7e531ac70 0x55a7e5322cb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6158004600 tx=0x7f615802a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0dd1efe26caa836 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:50.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.149+0000 7f614aa00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:11:50.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.150+0000 7f614aa00640 1 -- 172.21.15.43:0/1339299654 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6158035070 con 0x55a7e531ac70 2024-09-19T05:11:50.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.150+0000 7f616e5c9740 1 -- 172.21.15.43:0/1339299654 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6158004400 con 0x55a7e531ac70 2024-09-19T05:11:50.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.150+0000 7f614aa00640 1 -- 172.21.15.43:0/1339299654 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f6158004200 con 0x55a7e531ac70 2024-09-19T05:11:50.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.150+0000 7f614aa00640 1 -- 172.21.15.43:0/1339299654 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6158009520 con 0x55a7e531ac70 2024-09-19T05:11:50.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.150+0000 7f616e5c9740 10 client.26765 fetch_fsmap learned FSMap version 29 2024-09-19T05:11:50.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.150+0000 7f616e5c9740 1 -- 172.21.15.43:0/1339299654 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a7e5334a10 con 0x55a7e531ac70 2024-09-19T05:11:50.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.150+0000 7f614aa00640 1 -- 172.21.15.43:0/1339299654 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6158004400 con 0x55a7e531ac70 2024-09-19T05:11:50.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.151+0000 7f614aa00640 1 -- 172.21.15.43:0/1339299654 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6158031bf0 con 0x55a7e531ac70 2024-09-19T05:11:50.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.151+0000 7f616e5c9740 10 client.26765 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:11:50.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.151+0000 7f616e5c9740 10 client.26765 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T05:11:50.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.151+0000 7f616e5c9740 20 client.26765 populate_metadata read hostname 'smithi043' 2024-09-19T05:11:50.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.151+0000 7f616e5c9740 1 --2- 172.21.15.43:0/1339299654 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55a7e5335d60 0x55a7e5338150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:50.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.151+0000 7f616e5c9740 4 client.26765 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:11:50.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.151+0000 7f616e5c9740 1 -- 172.21.15.43:0/1339299654 --> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7e528e950 con 0x55a7e5335d60 2024-09-19T05:11:50.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.151+0000 7f6163e00640 1 --2- 172.21.15.43:0/1339299654 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55a7e5335d60 0x55a7e5338150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:50.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.152+0000 7f6163e00640 1 --2- 172.21.15.43:0/1339299654 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55a7e5335d60 0x55a7e5338150 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f271b61bc512c2eb server_cookie=e7da847956f7f5ef in_seq=0 out_seq=0 2024-09-19T05:11:50.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.152+0000 7f614aa00640 10 client.26765 ms_handle_connect on v2:172.21.15.151:6838/3794104313 2024-09-19T05:11:50.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.153+0000 7f614aa00640 1 -- 172.21.15.43:0/1339299654 <== mds.2 v2:172.21.15.151:6838/3794104313 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a7e528e950 con 0x55a7e5335d60 2024-09-19T05:11:50.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.153+0000 7f614aa00640 10 client.26765 handle_command_reply: tid=0 2024-09-19T05:11:50.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.153+0000 7f616e5c9740 1 client.26765 shutdown 2024-09-19T05:11:50.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.153+0000 7f616e5c9740 1 -- 172.21.15.43:0/1339299654 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7e531ac70 msgr2=0x55a7e5322cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:50.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.153+0000 7f616e5c9740 1 --2- 172.21.15.43:0/1339299654 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7e531ac70 0x55a7e5322cb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6158004600 tx=0x7f615802a3f0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.153+0000 7f616e5c9740 1 -- 172.21.15.43:0/1339299654 shutdown_connections 2024-09-19T05:11:50.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.153+0000 7f616e5c9740 1 --2- 172.21.15.43:0/1339299654 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55a7e5335d60 0x55a7e5338150 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.153+0000 7f616e5c9740 1 --2- 172.21.15.43:0/1339299654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e5325e50 0x55a7e5323730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.153+0000 7f616e5c9740 1 --2- 172.21.15.43:0/1339299654 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7e531b590 0x55a7e53231f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.153+0000 7f616e5c9740 1 --2- 172.21.15.43:0/1339299654 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7e531ac70 0x55a7e5322cb0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.153+0000 7f616e5c9740 1 -- 172.21.15.43:0/1339299654 >> 172.21.15.43:0/1339299654 conn(0x55a7e5317200 msgr2=0x55a7e5316380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:50.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.153+0000 7f614aa00640 0 client.26765 ms_handle_reset on v2:172.21.15.151:6838/3794104313 2024-09-19T05:11:50.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.153+0000 7f616e5c9740 1 -- 172.21.15.43:0/1339299654 shutdown_connections 2024-09-19T05:11:50.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.153+0000 7f616e5c9740 1 -- 172.21.15.43:0/1339299654 wait complete. 2024-09-19T05:11:50.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.153+0000 7f616e5c9740 20 client.26765 trim_cache size 0 max 16384 2024-09-19T05:11:50.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.167+0000 7f616e5c9740 1 Processor -- start 2024-09-19T05:11:50.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.167+0000 7f616e5c9740 1 -- start start 2024-09-19T05:11:50.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.168+0000 7f616e5c9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e531ac70 0x55a7e531d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:50.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.168+0000 7f616e5c9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7e531b590 0x55a7e533d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:50.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.168+0000 7f616e5c9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7e5325e50 0x55a7e533dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:50.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.168+0000 7f616e5c9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f61641afaf0 con 0x55a7e531ac70 2024-09-19T05:11:50.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.168+0000 7f616e5c9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f61641b1630 con 0x55a7e5325e50 2024-09-19T05:11:50.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.168+0000 7f616e5c9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f61641b01c0 con 0x55a7e531b590 2024-09-19T05:11:50.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.168+0000 7f6169200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e531ac70 0x55a7e531d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:50.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.168+0000 7f6169200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e531ac70 0x55a7e531d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43016/0 (socket says 172.21.15.43:43016) 2024-09-19T05:11:50.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.168+0000 7f6169200640 1 -- 172.21.15.43:0/2812238413 learned_addr learned my addr 172.21.15.43:0/2812238413 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:50.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.168+0000 7f6163e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7e531b590 0x55a7e533d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:50.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.168+0000 7f6169c00640 1 --2- 172.21.15.43:0/2812238413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7e5325e50 0x55a7e533dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:50.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.168+0000 7f6169200640 1 -- 172.21.15.43:0/2812238413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7e531b590 msgr2=0x55a7e533d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:50.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.168+0000 7f6169200640 1 --2- 172.21.15.43:0/2812238413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7e531b590 0x55a7e533d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.168+0000 7f6169200640 1 -- 172.21.15.43:0/2812238413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7e5325e50 msgr2=0x55a7e533dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:50.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.168+0000 7f6169200640 1 --2- 172.21.15.43:0/2812238413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7e5325e50 0x55a7e533dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.168+0000 7f6169200640 1 -- 172.21.15.43:0/2812238413 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7e5312770 con 0x55a7e531ac70 2024-09-19T05:11:50.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.168+0000 7f6169c00640 1 --2- 172.21.15.43:0/2812238413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7e5325e50 0x55a7e533dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:50.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.169+0000 7f6169200640 1 --2- 172.21.15.43:0/2812238413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e531ac70 0x55a7e531d640 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f61580035f0 tx=0x7f6158003af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=467a9f381eecfd14 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:50.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.169+0000 7f6155600640 1 -- 172.21.15.43:0/2812238413 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6158035070 con 0x55a7e531ac70 2024-09-19T05:11:50.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.169+0000 7f6155600640 1 -- 172.21.15.43:0/2812238413 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61580057f0 con 0x55a7e531ac70 2024-09-19T05:11:50.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.169+0000 7f6155600640 1 -- 172.21.15.43:0/2812238413 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6158005380 con 0x55a7e531ac70 2024-09-19T05:11:50.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.170+0000 7f616e5c9740 1 -- 172.21.15.43:0/2812238413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e531ac70 msgr2=0x55a7e531d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:50.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.170+0000 7f616e5c9740 1 --2- 172.21.15.43:0/2812238413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e531ac70 0x55a7e531d640 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f61580035f0 tx=0x7f6158003af0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.170+0000 7f616e5c9740 1 -- 172.21.15.43:0/2812238413 shutdown_connections 2024-09-19T05:11:50.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.170+0000 7f616e5c9740 1 --2- 172.21.15.43:0/2812238413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7e5325e50 0x55a7e533dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.170+0000 7f616e5c9740 1 --2- 172.21.15.43:0/2812238413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7e531b590 0x55a7e533d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.170+0000 7f616e5c9740 1 --2- 172.21.15.43:0/2812238413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e531ac70 0x55a7e531d640 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.170+0000 7f616e5c9740 1 -- 172.21.15.43:0/2812238413 >> 172.21.15.43:0/2812238413 conn(0x55a7e5317200 msgr2=0x55a7e5324d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:50.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.170+0000 7f616e5c9740 1 -- 172.21.15.43:0/2812238413 shutdown_connections 2024-09-19T05:11:50.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.170+0000 7f616e5c9740 1 -- 172.21.15.43:0/2812238413 wait complete. 2024-09-19T05:11:50.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.170+0000 7f616e5c9740 1 -- start start 2024-09-19T05:11:50.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.170+0000 7f616e5c9740 1 Processor -- start 2024-09-19T05:11:50.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.171+0000 7f616e5c9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7e531ac70 0x55a7e532e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:50.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.171+0000 7f616e5c9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7e531b590 0x55a7e532e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:50.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.171+0000 7f616e5c9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e5325e50 0x55a7e53294f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:50.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.171+0000 7f616e5c9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f61641b0f60 con 0x55a7e5325e50 2024-09-19T05:11:50.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.171+0000 7f616e5c9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f61641b0890 con 0x55a7e531b590 2024-09-19T05:11:50.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.171+0000 7f616e5c9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f61641adaf0 con 0x55a7e531ac70 2024-09-19T05:11:50.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.171+0000 7f616e5c9740 10 client.0 _ll_register_callbacks cb 0x7f615c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:50.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.171+0000 7f6169200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7e531ac70 0x55a7e532e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:50.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.171+0000 7f6169200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7e531ac70 0x55a7e532e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:51738/0 (socket says 172.21.15.43:51738) 2024-09-19T05:11:50.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.171+0000 7f6169200640 1 -- 172.21.15.43:0/1787684319 learned_addr learned my addr 172.21.15.43:0/1787684319 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:50.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.171+0000 7f6163e00640 1 --2- 172.21.15.43:0/1787684319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7e531b590 0x55a7e532e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:50.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.171+0000 7f6169c00640 1 --2- 172.21.15.43:0/1787684319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e5325e50 0x55a7e53294f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:50.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.172+0000 7f6163e00640 1 -- 172.21.15.43:0/1787684319 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7e531ac70 msgr2=0x55a7e532e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:50.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.172+0000 7f6163e00640 1 --2- 172.21.15.43:0/1787684319 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7e531ac70 0x55a7e532e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.172+0000 7f6163e00640 1 -- 172.21.15.43:0/1787684319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e5325e50 msgr2=0x55a7e53294f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:50.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.172+0000 7f6163e00640 1 --2- 172.21.15.43:0/1787684319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e5325e50 0x55a7e53294f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.172+0000 7f6163e00640 1 -- 172.21.15.43:0/1787684319 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7e5329d20 con 0x55a7e531b590 2024-09-19T05:11:50.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.172+0000 7f6169200640 1 --2- 172.21.15.43:0/1787684319 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7e531ac70 0x55a7e532e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:11:50.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.172+0000 7f6163e00640 1 --2- 172.21.15.43:0/1787684319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7e531b590 0x55a7e532e650 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f61500cd7f0 tx=0x7f61500bc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65035c2ceb251a38 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:50.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.172+0000 7f614aa00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:11:50.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.172+0000 7f614aa00640 1 -- 172.21.15.43:0/1787684319 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615009c380 con 0x55a7e531b590 2024-09-19T05:11:50.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.172+0000 7f614aa00640 1 -- 172.21.15.43:0/1787684319 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f61500cda70 con 0x55a7e531b590 2024-09-19T05:11:50.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.172+0000 7f616e5c9740 1 -- 172.21.15.43:0/1787684319 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61580057f0 con 0x55a7e531b590 2024-09-19T05:11:50.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.172+0000 7f614aa00640 1 -- 172.21.15.43:0/1787684319 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f61500cdc70 con 0x55a7e531b590 2024-09-19T05:11:50.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.172+0000 7f616e5c9740 10 client.26771 fetch_fsmap learned FSMap version 29 2024-09-19T05:11:50.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.172+0000 7f616e5c9740 1 -- 172.21.15.43:0/1787684319 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a7e53a0380 con 0x55a7e531b590 2024-09-19T05:11:50.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.173+0000 7f614aa00640 1 -- 172.21.15.43:0/1787684319 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61500ae030 con 0x55a7e531b590 2024-09-19T05:11:50.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.173+0000 7f614aa00640 1 -- 172.21.15.43:0/1787684319 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f61500b0030 con 0x55a7e531b590 2024-09-19T05:11:50.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.173+0000 7f616e5c9740 10 client.26771 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:11:50.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.173+0000 7f616e5c9740 10 client.26771 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T05:11:50.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.173+0000 7f616e5c9740 20 client.26771 populate_metadata read hostname 'smithi043' 2024-09-19T05:11:50.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.173+0000 7f616e5c9740 1 --2- 172.21.15.43:0/1787684319 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55a7e53a0b00 0x55a7e53a2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:50.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.173+0000 7f616e5c9740 4 client.26771 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:11:50.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.173+0000 7f616e5c9740 1 -- 172.21.15.43:0/1787684319 --> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a7e5288e60 con 0x55a7e53a0b00 2024-09-19T05:11:50.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.173+0000 7f6169200640 1 --2- 172.21.15.43:0/1787684319 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55a7e53a0b00 0x55a7e53a2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:50.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.174+0000 7f6169200640 1 --2- 172.21.15.43:0/1787684319 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55a7e53a0b00 0x55a7e53a2f20 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=16671bf3bd1c492a server_cookie=6edd64175638af42 in_seq=0 out_seq=0 2024-09-19T05:11:50.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.174+0000 7f614aa00640 10 client.26771 ms_handle_connect on v2:172.21.15.151:6838/3794104313 2024-09-19T05:11:50.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.174+0000 7f614aa00640 1 -- 172.21.15.43:0/1787684319 <== mds.2 v2:172.21.15.151:6838/3794104313 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a7e5288e60 con 0x55a7e53a0b00 2024-09-19T05:11:50.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.174+0000 7f614aa00640 10 client.26771 handle_command_reply: tid=0 2024-09-19T05:11:50.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.174+0000 7f616e5c9740 1 client.26771 shutdown 2024-09-19T05:11:50.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.174+0000 7f616e5c9740 1 -- 172.21.15.43:0/1787684319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7e531b590 msgr2=0x55a7e532e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:50.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.174+0000 7f616e5c9740 1 --2- 172.21.15.43:0/1787684319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7e531b590 0x55a7e532e650 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f61500cd7f0 tx=0x7f61500bc040 comp rx=0 tx=0).stop 2024-09-19T05:11:50.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.174+0000 7f616e5c9740 1 -- 172.21.15.43:0/1787684319 shutdown_connections 2024-09-19T05:11:50.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.174+0000 7f616e5c9740 1 --2- 172.21.15.43:0/1787684319 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55a7e53a0b00 0x55a7e53a2f20 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.174+0000 7f616e5c9740 1 --2- 172.21.15.43:0/1787684319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e5325e50 0x55a7e53294f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.174+0000 7f616e5c9740 1 --2- 172.21.15.43:0/1787684319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7e531b590 0x55a7e532e650 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.174+0000 7f614aa00640 0 client.26771 ms_handle_reset on v2:172.21.15.151:6838/3794104313 2024-09-19T05:11:50.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.174+0000 7f616e5c9740 1 --2- 172.21.15.43:0/1787684319 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7e531ac70 0x55a7e532e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.174+0000 7f616e5c9740 1 -- 172.21.15.43:0/1787684319 >> 172.21.15.43:0/1787684319 conn(0x55a7e5317200 msgr2=0x55a7e5315b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:50.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.175+0000 7f616e5c9740 1 -- 172.21.15.43:0/1787684319 shutdown_connections 2024-09-19T05:11:50.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.175+0000 7f616e5c9740 1 -- 172.21.15.43:0/1787684319 wait complete. 2024-09-19T05:11:50.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.175+0000 7f616e5c9740 20 client.26771 trim_cache size 0 max 16384 2024-09-19T05:11:50.180 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:11:50.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.175+0000 7f616bc00640 1 -- 172.21.15.43:0/3041437923 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f614c0a3430 msgr2=0x7f614c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:50.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.175+0000 7f616bc00640 1 --2- 172.21.15.43:0/3041437923 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f614c0a3430 0x7f614c0a58f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f615c009d40 tx=0x7f615c009500 comp rx=0 tx=0).stop 2024-09-19T05:11:50.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.175+0000 7f616bc00640 1 -- 172.21.15.43:0/3041437923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61641049a0 msgr2=0x7f616419abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:50.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.175+0000 7f616bc00640 1 --2- 172.21.15.43:0/3041437923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61641049a0 0x7f616419abd0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f615000e9f0 tx=0x7f615000edf0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.175+0000 7f616bc00640 1 -- 172.21.15.43:0/3041437923 shutdown_connections 2024-09-19T05:11:50.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.175+0000 7f616bc00640 1 --2- 172.21.15.43:0/3041437923 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f614c0a3430 0x7f614c0a58f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.175+0000 7f616bc00640 1 --2- 172.21.15.43:0/3041437923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6164109120 0x7f616419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.175+0000 7f616bc00640 1 --2- 172.21.15.43:0/3041437923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61641049a0 0x7f616419abd0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.175+0000 7f616bc00640 1 --2- 172.21.15.43:0/3041437923 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6164103ff0 0x7f616419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:50.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.175+0000 7f616bc00640 1 -- 172.21.15.43:0/3041437923 >> 172.21.15.43:0/3041437923 conn(0x7f61640ffa90 msgr2=0x7f6164100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:50.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.176+0000 7f616bc00640 1 -- 172.21.15.43:0/3041437923 shutdown_connections 2024-09-19T05:11:50.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:50.176+0000 7f616bc00640 1 -- 172.21.15.43:0/3041437923 wait complete. 2024-09-19T05:11:50.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:11:50 smithi043 ceph-mon[30800]: pgmap v220: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:11:50.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:11:50 smithi151 ceph-mon[35425]: pgmap v220: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:11:50.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:11:50 smithi096 ceph-mon[35085]: pgmap v220: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:11:51.216 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T05:11:51.595 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:11:52.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.135+0000 7f2c8ba00640 1 -- 172.21.15.43:0/2059985572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c84074eb0 msgr2=0x7f2c84112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:52.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.135+0000 7f2c8ba00640 1 --2- 172.21.15.43:0/2059985572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c84074eb0 0x7f2c84112650 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f2c70009900 tx=0x7f2c7002a480 comp rx=0 tx=0).stop 2024-09-19T05:11:52.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.135+0000 7f2c8ba00640 1 -- 172.21.15.43:0/2059985572 shutdown_connections 2024-09-19T05:11:52.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.135+0000 7f2c8ba00640 1 --2- 172.21.15.43:0/2059985572 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c84112b90 0x7f2c84114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:52.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.135+0000 7f2c8ba00640 1 --2- 172.21.15.43:0/2059985572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c84074eb0 0x7f2c84112650 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:52.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.135+0000 7f2c8ba00640 1 --2- 172.21.15.43:0/2059985572 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c84074590 0x7f2c84074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:52.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.135+0000 7f2c8ba00640 1 -- 172.21.15.43:0/2059985572 >> 172.21.15.43:0/2059985572 conn(0x7f2c840fd8c0 msgr2=0x7f2c840ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:52.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.136+0000 7f2c8ba00640 1 -- 172.21.15.43:0/2059985572 shutdown_connections 2024-09-19T05:11:52.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.136+0000 7f2c8ba00640 1 -- 172.21.15.43:0/2059985572 wait complete. 2024-09-19T05:11:52.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.136+0000 7f2c8ba00640 1 Processor -- start 2024-09-19T05:11:52.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.137+0000 7f2c8ba00640 1 -- start start 2024-09-19T05:11:52.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.137+0000 7f2c8ba00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c84074590 0x7f2c8419eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:52.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.137+0000 7f2c8ba00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c84074eb0 0x7f2c8419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:52.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.137+0000 7f2c8ba00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c84112b90 0x7f2c841a3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:52.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.137+0000 7f2c8ba00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2c84118560 con 0x7f2c84074eb0 2024-09-19T05:11:52.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.137+0000 7f2c8ba00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2c841183e0 con 0x7f2c84074590 2024-09-19T05:11:52.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.137+0000 7f2c8ba00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2c841186e0 con 0x7f2c84112b90 2024-09-19T05:11:52.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.137+0000 7f2c83e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c84074eb0 0x7f2c8419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:52.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.137+0000 7f2c83e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c84074eb0 0x7f2c8419f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43052/0 (socket says 172.21.15.43:43052) 2024-09-19T05:11:52.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.137+0000 7f2c83e00640 1 -- 172.21.15.43:0/579187746 learned_addr learned my addr 172.21.15.43:0/579187746 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:52.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.137+0000 7f2c89000640 1 --2- 172.21.15.43:0/579187746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c84074590 0x7f2c8419eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:52.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.137+0000 7f2c89a00640 1 --2- 172.21.15.43:0/579187746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c84112b90 0x7f2c841a3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:52.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.138+0000 7f2c89000640 1 -- 172.21.15.43:0/579187746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c84112b90 msgr2=0x7f2c841a3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:52.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.138+0000 7f2c89000640 1 --2- 172.21.15.43:0/579187746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c84112b90 0x7f2c841a3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:52.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.138+0000 7f2c89000640 1 -- 172.21.15.43:0/579187746 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c84074eb0 msgr2=0x7f2c8419f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:52.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.138+0000 7f2c89000640 1 --2- 172.21.15.43:0/579187746 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c84074eb0 0x7f2c8419f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:52.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.138+0000 7f2c89000640 1 -- 172.21.15.43:0/579187746 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c841a3b50 con 0x7f2c84074590 2024-09-19T05:11:52.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.138+0000 7f2c89a00640 1 --2- 172.21.15.43:0/579187746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c84112b90 0x7f2c841a3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:52.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.138+0000 7f2c83e00640 1 --2- 172.21.15.43:0/579187746 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c84074eb0 0x7f2c8419f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:11:52.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.138+0000 7f2c89000640 1 --2- 172.21.15.43:0/579187746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c84074590 0x7f2c8419eb20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2c7800e9f0 tx=0x7f2c7800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:52.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.138+0000 7f2c81600640 1 -- 172.21.15.43:0/579187746 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7800ccc0 con 0x7f2c84074590 2024-09-19T05:11:52.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.138+0000 7f2c81600640 1 -- 172.21.15.43:0/579187746 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c78010910 con 0x7f2c84074590 2024-09-19T05:11:52.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.138+0000 7f2c8ba00640 1 -- 172.21.15.43:0/579187746 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c841a3e40 con 0x7f2c84074590 2024-09-19T05:11:52.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.138+0000 7f2c81600640 1 -- 172.21.15.43:0/579187746 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c78018960 con 0x7f2c84074590 2024-09-19T05:11:52.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.138+0000 7f2c8ba00640 1 -- 172.21.15.43:0/579187746 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c841ab6d0 con 0x7f2c84074590 2024-09-19T05:11:52.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.141+0000 7f2c8e48c740 1 Processor -- start 2024-09-19T05:11:52.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.141+0000 7f2c8e48c740 1 -- start start 2024-09-19T05:11:52.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.142+0000 7f2c81600640 1 -- 172.21.15.43:0/579187746 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f2c78010ac0 con 0x7f2c84074590 2024-09-19T05:11:52.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.142+0000 7f2c8e48c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e0f5dd9c70 0x55e0f5dda050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:52.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.142+0000 7f2c8e48c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0f5dda590 0x55e0f5de4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:52.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.142+0000 7f2c8e48c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f5de4e50 0x55e0f5de7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:52.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.142+0000 7f2c8e48c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e0f5d403e0 con 0x55e0f5de4e50 2024-09-19T05:11:52.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.142+0000 7f2c8e48c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e0f5ce7d80 con 0x55e0f5dda590 2024-09-19T05:11:52.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.142+0000 7f2c8e48c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e0f5d3c3a0 con 0x55e0f5dd9c70 2024-09-19T05:11:52.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.142+0000 7f2c89000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0f5dda590 0x55e0f5de4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:52.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.142+0000 7f2c89000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0f5dda590 0x55e0f5de4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:50664/0 (socket says 172.21.15.43:50664) 2024-09-19T05:11:52.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.142+0000 7f2c89000640 1 -- 172.21.15.43:0/3118262978 learned_addr learned my addr 172.21.15.43:0/3118262978 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:52.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.142+0000 7f2c83e00640 1 --2- 172.21.15.43:0/3118262978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f5de4e50 0x55e0f5de7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:52.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.142+0000 7f2c89a00640 1 --2- 172.21.15.43:0/3118262978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e0f5dd9c70 0x55e0f5dda050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:52.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.143+0000 7f2c89000640 1 -- 172.21.15.43:0/3118262978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e0f5dd9c70 msgr2=0x55e0f5dda050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:52.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.143+0000 7f2c89000640 1 --2- 172.21.15.43:0/3118262978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e0f5dd9c70 0x55e0f5dda050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:52.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.143+0000 7f2c89000640 1 -- 172.21.15.43:0/3118262978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f5de4e50 msgr2=0x55e0f5de7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:52.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.143+0000 7f2c89000640 1 --2- 172.21.15.43:0/3118262978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f5de4e50 0x55e0f5de7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:52.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.143+0000 7f2c89000640 1 -- 172.21.15.43:0/3118262978 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0f5d06ae0 con 0x55e0f5dda590 2024-09-19T05:11:52.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.143+0000 7f2c83e00640 1 --2- 172.21.15.43:0/3118262978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f5de4e50 0x55e0f5de7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:11:52.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.143+0000 7f2c89000640 1 --2- 172.21.15.43:0/3118262978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0f5dda590 0x55e0f5de4910 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2c780cd820 tx=0x7f2c780d8860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d091827abfde369 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:52.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.143+0000 7f2c76000640 1 -- 172.21.15.43:0/3118262978 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c780e3070 con 0x55e0f5dda590 2024-09-19T05:11:52.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.143+0000 7f2c76000640 1 -- 172.21.15.43:0/3118262978 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c780d0b00 con 0x55e0f5dda590 2024-09-19T05:11:52.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.143+0000 7f2c76000640 1 -- 172.21.15.43:0/3118262978 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c780deae0 con 0x55e0f5dda590 2024-09-19T05:11:52.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.143+0000 7f2c81600640 1 --2- 172.21.15.43:0/579187746 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f2c6c0a33b0 0x7f2c6c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:52.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.144+0000 7f2c81600640 1 -- 172.21.15.43:0/579187746 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f2c780d0170 con 0x7f2c84074590 2024-09-19T05:11:52.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.144+0000 7f2c89a00640 1 --2- 172.21.15.43:0/579187746 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f2c6c0a33b0 0x7f2c6c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:52.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.144+0000 7f2c8e48c740 1 -- 172.21.15.43:0/3118262978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0f5dda590 msgr2=0x55e0f5de4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:52.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.144+0000 7f2c8e48c740 1 --2- 172.21.15.43:0/3118262978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0f5dda590 0x55e0f5de4910 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2c780cd820 tx=0x7f2c780d8860 comp rx=0 tx=0).stop 2024-09-19T05:11:52.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.144+0000 7f2c8e48c740 1 -- 172.21.15.43:0/3118262978 shutdown_connections 2024-09-19T05:11:52.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.144+0000 7f2c8e48c740 1 --2- 172.21.15.43:0/3118262978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f5de4e50 0x55e0f5de7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:52.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.144+0000 7f2c8e48c740 1 --2- 172.21.15.43:0/3118262978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0f5dda590 0x55e0f5de4910 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:52.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.144+0000 7f2c8e48c740 1 --2- 172.21.15.43:0/3118262978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e0f5dd9c70 0x55e0f5dda050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:52.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.144+0000 7f2c8e48c740 1 -- 172.21.15.43:0/3118262978 >> 172.21.15.43:0/3118262978 conn(0x55e0f5dd6200 msgr2=0x55e0f5dd65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:52.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.144+0000 7f2c8e48c740 1 -- 172.21.15.43:0/3118262978 shutdown_connections 2024-09-19T05:11:52.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.144+0000 7f2c89a00640 1 --2- 172.21.15.43:0/579187746 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f2c6c0a33b0 0x7f2c6c0a5870 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2c84076600 tx=0x7f2c7c009500 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:52.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.144+0000 7f2c8e48c740 1 -- 172.21.15.43:0/3118262978 wait complete. 2024-09-19T05:11:52.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.145+0000 7f2c8e48c740 1 -- start start 2024-09-19T05:11:52.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.145+0000 7f2c8e48c740 1 Processor -- start 2024-09-19T05:11:52.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.146+0000 7f2c8e48c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e0f5dd9c70 0x55e0f5de1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:52.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.146+0000 7f2c8e48c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f5dda590 0x55e0f5de22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:52.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.146+0000 7f2c8e48c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0f5de4e50 0x55e0f5de2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:52.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.146+0000 7f2c8e48c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e0f5de7a20 con 0x55e0f5dda590 2024-09-19T05:11:52.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.146+0000 7f2c8e48c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e0f5cfb0f0 con 0x55e0f5de4e50 2024-09-19T05:11:52.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.146+0000 7f2c8e48c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e0f5de7ba0 con 0x55e0f5dd9c70 2024-09-19T05:11:52.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.146+0000 7f2c8e48c740 10 client.0 _ll_register_callbacks cb 0x55e0f5ce2bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:52.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.146+0000 7f2c83e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f5dda590 0x55e0f5de22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:52.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.146+0000 7f2c83e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f5dda590 0x55e0f5de22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43078/0 (socket says 172.21.15.43:43078) 2024-09-19T05:11:52.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.146+0000 7f2c83e00640 1 -- 172.21.15.43:0/4061528434 learned_addr learned my addr 172.21.15.43:0/4061528434 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:52.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.146+0000 7f2c89000640 1 --2- 172.21.15.43:0/4061528434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e0f5dd9c70 0x55e0f5de1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:52.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.146+0000 7f2c89a00640 1 --2- 172.21.15.43:0/4061528434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0f5de4e50 0x55e0f5de2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:52.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.146+0000 7f2c83e00640 1 -- 172.21.15.43:0/4061528434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e0f5dd9c70 msgr2=0x55e0f5de1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:52.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.146+0000 7f2c83e00640 1 --2- 172.21.15.43:0/4061528434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e0f5dd9c70 0x55e0f5de1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:52.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.146+0000 7f2c83e00640 1 -- 172.21.15.43:0/4061528434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0f5de4e50 msgr2=0x55e0f5de2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:52.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.146+0000 7f2c83e00640 1 --2- 172.21.15.43:0/4061528434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0f5de4e50 0x55e0f5de2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:52.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.146+0000 7f2c83e00640 1 -- 172.21.15.43:0/4061528434 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0f5ab13c0 con 0x55e0f5dda590 2024-09-19T05:11:52.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.147+0000 7f2c83e00640 1 --2- 172.21.15.43:0/4061528434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f5dda590 0x55e0f5de22d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f2c70009a20 tx=0x7f2c70031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5762662a9ceeb6b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:52.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.147+0000 7f2c6aa00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:11:52.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.147+0000 7f2c6aa00640 1 -- 172.21.15.43:0/4061528434 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c70035040 con 0x55e0f5dda590 2024-09-19T05:11:52.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.147+0000 7f2c6aa00640 1 -- 172.21.15.43:0/4061528434 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f2c70004460 con 0x55e0f5dda590 2024-09-19T05:11:52.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.147+0000 7f2c8e48c740 1 -- 172.21.15.43:0/4061528434 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c780d0b00 con 0x55e0f5dda590 2024-09-19T05:11:52.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.147+0000 7f2c6aa00640 1 -- 172.21.15.43:0/4061528434 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2c70005030 con 0x55e0f5dda590 2024-09-19T05:11:52.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.147+0000 7f2c8e48c740 10 client.16755 fetch_fsmap learned FSMap version 29 2024-09-19T05:11:52.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.147+0000 7f2c8e48c740 1 -- 172.21.15.43:0/4061528434 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e0f5df3ab0 con 0x55e0f5dda590 2024-09-19T05:11:52.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.148+0000 7f2c6aa00640 1 -- 172.21.15.43:0/4061528434 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c780d0b00 con 0x55e0f5dda590 2024-09-19T05:11:52.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.148+0000 7f2c6aa00640 1 -- 172.21.15.43:0/4061528434 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2c700035f0 con 0x55e0f5dda590 2024-09-19T05:11:52.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.148+0000 7f2c8e48c740 10 client.16755 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:11:52.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.148+0000 7f2c8e48c740 10 client.16755 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-19T05:11:52.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.148+0000 7f2c8e48c740 20 client.16755 populate_metadata read hostname 'smithi043' 2024-09-19T05:11:52.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.148+0000 7f2c8e48c740 1 --2- 172.21.15.43:0/4061528434 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55e0f5df4f70 0x55e0f5df73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:52.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.148+0000 7f2c8e48c740 4 client.16755 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:11:52.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.148+0000 7f2c8e48c740 1 -- 172.21.15.43:0/4061528434 --> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0f5d4d950 con 0x55e0f5df4f70 2024-09-19T05:11:52.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.148+0000 7f2c89000640 1 --2- 172.21.15.43:0/4061528434 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55e0f5df4f70 0x55e0f5df73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:52.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.149+0000 7f2c89000640 1 --2- 172.21.15.43:0/4061528434 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55e0f5df4f70 0x55e0f5df73a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a7a70fc9736165e8 server_cookie=ef86d871bdc06d23 in_seq=0 out_seq=0 2024-09-19T05:11:52.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.149+0000 7f2c6aa00640 10 client.16755 ms_handle_connect on v2:172.21.15.43:6838/3404555030 2024-09-19T05:11:52.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.149+0000 7f2c6aa00640 1 -- 172.21.15.43:0/4061528434 <== mds.1 v2:172.21.15.43:6838/3404555030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e0f5d4d950 con 0x55e0f5df4f70 2024-09-19T05:11:52.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.149+0000 7f2c6aa00640 10 client.16755 handle_command_reply: tid=0 2024-09-19T05:11:52.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.150+0000 7f2c8e48c740 1 client.16755 shutdown 2024-09-19T05:11:52.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.150+0000 7f2c8e48c740 1 -- 172.21.15.43:0/4061528434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f5dda590 msgr2=0x55e0f5de22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:52.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.150+0000 7f2c8e48c740 1 --2- 172.21.15.43:0/4061528434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f5dda590 0x55e0f5de22d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f2c70009a20 tx=0x7f2c70031920 comp rx=0 tx=0).stop 2024-09-19T05:11:52.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.150+0000 7f2c8e48c740 1 -- 172.21.15.43:0/4061528434 shutdown_connections 2024-09-19T05:11:52.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.150+0000 7f2c8e48c740 1 --2- 172.21.15.43:0/4061528434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0f5de4e50 0x55e0f5de2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:52.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.150+0000 7f2c8e48c740 1 --2- 172.21.15.43:0/4061528434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f5dda590 0x55e0f5de22d0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:52.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.150+0000 7f2c8e48c740 1 --2- 172.21.15.43:0/4061528434 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55e0f5df4f70 0x55e0f5df73a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:52.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.150+0000 7f2c8e48c740 1 --2- 172.21.15.43:0/4061528434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e0f5dd9c70 0x55e0f5de1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:52.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.150+0000 7f2c8e48c740 1 -- 172.21.15.43:0/4061528434 >> 172.21.15.43:0/4061528434 conn(0x55e0f5dd6200 msgr2=0x55e0f5dd5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:52.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.150+0000 7f2c6aa00640 0 client.16755 ms_handle_reset on v2:172.21.15.43:6838/3404555030 2024-09-19T05:11:52.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.150+0000 7f2c8e48c740 1 -- 172.21.15.43:0/4061528434 shutdown_connections 2024-09-19T05:11:52.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.150+0000 7f2c8e48c740 1 -- 172.21.15.43:0/4061528434 wait complete. 2024-09-19T05:11:52.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.150+0000 7f2c8e48c740 20 client.16755 trim_cache size 0 max 16384 2024-09-19T05:11:52.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.164+0000 7f2c8e48c740 1 Processor -- start 2024-09-19T05:11:52.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.164+0000 7f2c8e48c740 1 -- start start 2024-09-19T05:11:52.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.164+0000 7f2c8e48c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e0f5dd9c70 0x55e0f5de3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:52.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.164+0000 7f2c8e48c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0f5dda590 0x55e0f5dfcc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:52.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.164+0000 7f2c8e48c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f5de4e50 0x55e0f5dfd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:52.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.164+0000 7f2c8e48c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2c841b4040 con 0x55e0f5de4e50 2024-09-19T05:11:52.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.164+0000 7f2c8e48c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2c841b5b80 con 0x55e0f5dda590 2024-09-19T05:11:52.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.164+0000 7f2c8e48c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2c841b4710 con 0x55e0f5dd9c70 2024-09-19T05:11:52.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.165+0000 7f2c89a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f5de4e50 0x55e0f5dfd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:52.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.165+0000 7f2c89a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f5de4e50 0x55e0f5dfd1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43094/0 (socket says 172.21.15.43:43094) 2024-09-19T05:11:52.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.165+0000 7f2c89a00640 1 -- 172.21.15.43:0/4052674432 learned_addr learned my addr 172.21.15.43:0/4052674432 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:52.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.165+0000 7f2c83e00640 1 --2- 172.21.15.43:0/4052674432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0f5dda590 0x55e0f5dfcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:52.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.165+0000 7f2c89000640 1 --2- 172.21.15.43:0/4052674432 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e0f5dd9c70 0x55e0f5de3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:52.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.165+0000 7f2c89a00640 1 -- 172.21.15.43:0/4052674432 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e0f5dd9c70 msgr2=0x55e0f5de3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:52.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.165+0000 7f2c89a00640 1 --2- 172.21.15.43:0/4052674432 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e0f5dd9c70 0x55e0f5de3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:52.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.165+0000 7f2c89a00640 1 -- 172.21.15.43:0/4052674432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0f5dda590 msgr2=0x55e0f5dfcc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:52.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.165+0000 7f2c89a00640 1 --2- 172.21.15.43:0/4052674432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0f5dda590 0x55e0f5dfcc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:52.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.165+0000 7f2c89a00640 1 -- 172.21.15.43:0/4052674432 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0f5dd9aa0 con 0x55e0f5de4e50 2024-09-19T05:11:52.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.166+0000 7f2c89a00640 1 --2- 172.21.15.43:0/4052674432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f5de4e50 0x55e0f5dfd1b0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c010a00 tx=0x7f2c7c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9fe760a99f8de38 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:52.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.166+0000 7f2c75600640 1 -- 172.21.15.43:0/4052674432 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c01e9a0 con 0x55e0f5de4e50 2024-09-19T05:11:52.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.166+0000 7f2c75600640 1 -- 172.21.15.43:0/4052674432 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c7c01eb40 con 0x55e0f5de4e50 2024-09-19T05:11:52.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.166+0000 7f2c75600640 1 -- 172.21.15.43:0/4052674432 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c01c610 con 0x55e0f5de4e50 2024-09-19T05:11:52.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.167+0000 7f2c8e48c740 1 -- 172.21.15.43:0/4052674432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f5de4e50 msgr2=0x55e0f5dfd1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:52.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.167+0000 7f2c8e48c740 1 --2- 172.21.15.43:0/4052674432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f5de4e50 0x55e0f5dfd1b0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c010a00 tx=0x7f2c7c019000 comp rx=0 tx=0).stop 2024-09-19T05:11:52.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.167+0000 7f2c8e48c740 1 -- 172.21.15.43:0/4052674432 shutdown_connections 2024-09-19T05:11:52.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.167+0000 7f2c8e48c740 1 --2- 172.21.15.43:0/4052674432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f5de4e50 0x55e0f5dfd1b0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:52.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.167+0000 7f2c8e48c740 1 --2- 172.21.15.43:0/4052674432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0f5dda590 0x55e0f5dfcc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:52.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.167+0000 7f2c8e48c740 1 --2- 172.21.15.43:0/4052674432 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e0f5dd9c70 0x55e0f5de3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:52.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.167+0000 7f2c8e48c740 1 -- 172.21.15.43:0/4052674432 >> 172.21.15.43:0/4052674432 conn(0x55e0f5dd6200 msgr2=0x55e0f5de35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:52.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.167+0000 7f2c8e48c740 1 -- 172.21.15.43:0/4052674432 shutdown_connections 2024-09-19T05:11:52.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.167+0000 7f2c8e48c740 1 -- 172.21.15.43:0/4052674432 wait complete. 2024-09-19T05:11:52.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.167+0000 7f2c8e48c740 1 -- start start 2024-09-19T05:11:52.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.167+0000 7f2c8e48c740 1 Processor -- start 2024-09-19T05:11:52.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.168+0000 7f2c8e48c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0f5dd9c70 0x55e0f5de8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:52.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.168+0000 7f2c8e48c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e0f5dda590 0x55e0f5de9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:52.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.168+0000 7f2c8e48c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f5de4e50 0x55e0f5de97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:52.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.168+0000 7f2c8e48c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2c841b54b0 con 0x55e0f5de4e50 2024-09-19T05:11:52.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.168+0000 7f2c8e48c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2c841b4de0 con 0x55e0f5dd9c70 2024-09-19T05:11:52.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.168+0000 7f2c8e48c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2c841b2040 con 0x55e0f5dda590 2024-09-19T05:11:52.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.168+0000 7f2c8e48c740 10 client.0 _ll_register_callbacks cb 0x7f2c70009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:52.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.168+0000 7f2c83e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e0f5dda590 0x55e0f5de9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:52.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.168+0000 7f2c83e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e0f5dda590 0x55e0f5de9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:51800/0 (socket says 172.21.15.43:51800) 2024-09-19T05:11:52.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.168+0000 7f2c83e00640 1 -- 172.21.15.43:0/1403801531 learned_addr learned my addr 172.21.15.43:0/1403801531 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:52.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.168+0000 7f2c89a00640 1 --2- 172.21.15.43:0/1403801531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f5de4e50 0x55e0f5de97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:52.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.168+0000 7f2c89000640 1 --2- 172.21.15.43:0/1403801531 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0f5dd9c70 0x55e0f5de8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:52.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.169+0000 7f2c83e00640 1 -- 172.21.15.43:0/1403801531 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0f5dd9c70 msgr2=0x55e0f5de8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:52.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.169+0000 7f2c83e00640 1 --2- 172.21.15.43:0/1403801531 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0f5dd9c70 0x55e0f5de8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:52.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.169+0000 7f2c83e00640 1 -- 172.21.15.43:0/1403801531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f5de4e50 msgr2=0x55e0f5de97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:52.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.169+0000 7f2c83e00640 1 --2- 172.21.15.43:0/1403801531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f5de4e50 0x55e0f5de97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:52.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.169+0000 7f2c83e00640 1 -- 172.21.15.43:0/1403801531 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0f5dd6e90 con 0x55e0f5dda590 2024-09-19T05:11:52.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.169+0000 7f2c89a00640 1 --2- 172.21.15.43:0/1403801531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f5de4e50 0x55e0f5de97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:52.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.169+0000 7f2c83e00640 1 --2- 172.21.15.43:0/1403801531 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e0f5dda590 0x55e0f5de9270 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2c70009850 tx=0x7f2c70005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99387c7e320b20c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:52.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.169+0000 7f2c6aa00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:11:52.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.169+0000 7f2c6aa00640 1 -- 172.21.15.43:0/1403801531 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c70051040 con 0x55e0f5dda590 2024-09-19T05:11:52.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.169+0000 7f2c8e48c740 1 -- 172.21.15.43:0/1403801531 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c7c01eb40 con 0x55e0f5dda590 2024-09-19T05:11:52.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.169+0000 7f2c6aa00640 1 -- 172.21.15.43:0/1403801531 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f2c70003de0 con 0x55e0f5dda590 2024-09-19T05:11:52.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.169+0000 7f2c6aa00640 1 -- 172.21.15.43:0/1403801531 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2c70035040 con 0x55e0f5dda590 2024-09-19T05:11:52.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.170+0000 7f2c8e48c740 10 client.26719 fetch_fsmap learned FSMap version 29 2024-09-19T05:11:52.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.170+0000 7f2c8e48c740 1 -- 172.21.15.43:0/1403801531 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e0f5dd1770 con 0x55e0f5dda590 2024-09-19T05:11:52.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.170+0000 7f2c6aa00640 1 -- 172.21.15.43:0/1403801531 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c7c01eb40 con 0x55e0f5dda590 2024-09-19T05:11:52.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.170+0000 7f2c6aa00640 1 -- 172.21.15.43:0/1403801531 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2c70004760 con 0x55e0f5dda590 2024-09-19T05:11:52.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.170+0000 7f2c8e48c740 10 client.26719 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:11:52.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.170+0000 7f2c8e48c740 10 client.26719 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-19T05:11:52.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.170+0000 7f2c8e48c740 20 client.26719 populate_metadata read hostname 'smithi043' 2024-09-19T05:11:52.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.170+0000 7f2c8e48c740 1 --2- 172.21.15.43:0/1403801531 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55e0f5e60510 0x55e0f5e62930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:52.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.170+0000 7f2c8e48c740 4 client.26719 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:11:52.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.170+0000 7f2c8e48c740 1 -- 172.21.15.43:0/1403801531 --> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e0f5d47e60 con 0x55e0f5e60510 2024-09-19T05:11:52.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.171+0000 7f2c89000640 1 --2- 172.21.15.43:0/1403801531 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55e0f5e60510 0x55e0f5e62930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:52.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.171+0000 7f2c89000640 1 --2- 172.21.15.43:0/1403801531 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55e0f5e60510 0x55e0f5e62930 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=479320f3fa65b28c server_cookie=74cb4972c95ad44a in_seq=0 out_seq=0 2024-09-19T05:11:52.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.171+0000 7f2c6aa00640 10 client.26719 ms_handle_connect on v2:172.21.15.43:6838/3404555030 2024-09-19T05:11:52.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.172+0000 7f2c6aa00640 1 -- 172.21.15.43:0/1403801531 <== mds.1 v2:172.21.15.43:6838/3404555030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e0f5d47e60 con 0x55e0f5e60510 2024-09-19T05:11:52.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.172+0000 7f2c6aa00640 10 client.26719 handle_command_reply: tid=0 2024-09-19T05:11:52.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.172+0000 7f2c8e48c740 1 client.26719 shutdown 2024-09-19T05:11:52.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.172+0000 7f2c8e48c740 1 -- 172.21.15.43:0/1403801531 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e0f5dda590 msgr2=0x55e0f5de9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:52.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.172+0000 7f2c8e48c740 1 --2- 172.21.15.43:0/1403801531 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e0f5dda590 0x55e0f5de9270 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2c70009850 tx=0x7f2c70005c00 comp rx=0 tx=0).stop 2024-09-19T05:11:52.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.172+0000 7f2c8e48c740 1 -- 172.21.15.43:0/1403801531 shutdown_connections 2024-09-19T05:11:52.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.172+0000 7f2c8e48c740 1 --2- 172.21.15.43:0/1403801531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f5de4e50 0x55e0f5de97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:52.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.172+0000 7f2c8e48c740 1 --2- 172.21.15.43:0/1403801531 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55e0f5e60510 0x55e0f5e62930 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:52.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.172+0000 7f2c8e48c740 1 --2- 172.21.15.43:0/1403801531 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e0f5dda590 0x55e0f5de9270 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:52.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.172+0000 7f2c8e48c740 1 --2- 172.21.15.43:0/1403801531 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0f5dd9c70 0x55e0f5de8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:52.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.172+0000 7f2c6aa00640 0 client.26719 ms_handle_reset on v2:172.21.15.43:6838/3404555030 2024-09-19T05:11:52.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.172+0000 7f2c8e48c740 1 -- 172.21.15.43:0/1403801531 >> 172.21.15.43:0/1403801531 conn(0x55e0f5dd6200 msgr2=0x55e0f5de1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:52.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.172+0000 7f2c8e48c740 1 -- 172.21.15.43:0/1403801531 shutdown_connections 2024-09-19T05:11:52.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.172+0000 7f2c8e48c740 1 -- 172.21.15.43:0/1403801531 wait complete. 2024-09-19T05:11:52.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.172+0000 7f2c8e48c740 20 client.26719 trim_cache size 0 max 16384 2024-09-19T05:11:52.188 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:11:52.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.173+0000 7f2c8ba00640 1 -- 172.21.15.43:0/579187746 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f2c6c0a33b0 msgr2=0x7f2c6c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:52.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.173+0000 7f2c8ba00640 1 --2- 172.21.15.43:0/579187746 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f2c6c0a33b0 0x7f2c6c0a5870 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2c84076600 tx=0x7f2c7c009500 comp rx=0 tx=0).stop 2024-09-19T05:11:52.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.173+0000 7f2c8ba00640 1 -- 172.21.15.43:0/579187746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c84074590 msgr2=0x7f2c8419eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:52.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.173+0000 7f2c8ba00640 1 --2- 172.21.15.43:0/579187746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c84074590 0x7f2c8419eb20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2c7800e9f0 tx=0x7f2c7800eeb0 comp rx=0 tx=0).stop 2024-09-19T05:11:52.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.173+0000 7f2c8ba00640 1 -- 172.21.15.43:0/579187746 shutdown_connections 2024-09-19T05:11:52.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.173+0000 7f2c8ba00640 1 --2- 172.21.15.43:0/579187746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c84112b90 0x7f2c841a3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:52.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.173+0000 7f2c8ba00640 1 --2- 172.21.15.43:0/579187746 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c84074eb0 0x7f2c8419f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:52.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.173+0000 7f2c8ba00640 1 --2- 172.21.15.43:0/579187746 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f2c6c0a33b0 0x7f2c6c0a5870 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:52.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.173+0000 7f2c8ba00640 1 --2- 172.21.15.43:0/579187746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c84074590 0x7f2c8419eb20 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:52.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.173+0000 7f2c8ba00640 1 -- 172.21.15.43:0/579187746 >> 172.21.15.43:0/579187746 conn(0x7f2c840fd8c0 msgr2=0x7f2c840ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:52.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.173+0000 7f2c8ba00640 1 -- 172.21.15.43:0/579187746 shutdown_connections 2024-09-19T05:11:52.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:52.173+0000 7f2c8ba00640 1 -- 172.21.15.43:0/579187746 wait complete. 2024-09-19T05:11:52.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:11:52 smithi043 ceph-mon[30800]: pgmap v221: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:11:52.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:11:52 smithi151 ceph-mon[35425]: pgmap v221: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:11:52.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:11:52 smithi096 ceph-mon[35085]: pgmap v221: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:11:53.133 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T05:11:53.512 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:11:54.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.053+0000 7f8e39600640 1 -- 172.21.15.43:0/1240173543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e34102790 msgr2=0x7f8e341067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:54.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.053+0000 7f8e39600640 1 --2- 172.21.15.43:0/1240173543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e34102790 0x7f8e341067e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c009900 tx=0x7f8e1c02a3f0 comp rx=0 tx=0).stop 2024-09-19T05:11:54.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.053+0000 7f8e39600640 1 -- 172.21.15.43:0/1240173543 shutdown_connections 2024-09-19T05:11:54.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.053+0000 7f8e39600640 1 --2- 172.21.15.43:0/1240173543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e34106f10 0x7f8e3410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:54.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.053+0000 7f8e39600640 1 --2- 172.21.15.43:0/1240173543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e34102790 0x7f8e341067e0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:54.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.053+0000 7f8e39600640 1 --2- 172.21.15.43:0/1240173543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e34101de0 0x7f8e341021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:54.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.053+0000 7f8e39600640 1 -- 172.21.15.43:0/1240173543 >> 172.21.15.43:0/1240173543 conn(0x7f8e340fd8c0 msgr2=0x7f8e340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:54.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.053+0000 7f8e39600640 1 -- 172.21.15.43:0/1240173543 shutdown_connections 2024-09-19T05:11:54.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.053+0000 7f8e39600640 1 -- 172.21.15.43:0/1240173543 wait complete. 2024-09-19T05:11:54.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.054+0000 7f8e39600640 1 Processor -- start 2024-09-19T05:11:54.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.054+0000 7f8e39600640 1 -- start start 2024-09-19T05:11:54.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.055+0000 7f8e39600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e34101de0 0x7f8e340744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:54.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.055+0000 7f8e39600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e34102790 0x7f8e34074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:54.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.055+0000 7f8e39600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e34106f10 0x7f8e34074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:54.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.055+0000 7f8e39600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8e34111f70 con 0x7f8e34102790 2024-09-19T05:11:54.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.055+0000 7f8e39600640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8e34111df0 con 0x7f8e34101de0 2024-09-19T05:11:54.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.055+0000 7f8e39600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8e341120f0 con 0x7f8e34106f10 2024-09-19T05:11:54.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.055+0000 7f8e2be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e34102790 0x7f8e34074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:54.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.055+0000 7f8e32a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e34101de0 0x7f8e340744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:54.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.055+0000 7f8e2be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e34102790 0x7f8e34074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43118/0 (socket says 172.21.15.43:43118) 2024-09-19T05:11:54.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.055+0000 7f8e2be00640 1 -- 172.21.15.43:0/4252485076 learned_addr learned my addr 172.21.15.43:0/4252485076 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:54.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.055+0000 7f8e33400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e34106f10 0x7f8e34074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:54.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.055+0000 7f8e32a00640 1 -- 172.21.15.43:0/4252485076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e34106f10 msgr2=0x7f8e34074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:54.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.056+0000 7f8e32a00640 1 --2- 172.21.15.43:0/4252485076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e34106f10 0x7f8e34074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:54.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.056+0000 7f8e32a00640 1 -- 172.21.15.43:0/4252485076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e34102790 msgr2=0x7f8e34074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:54.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.056+0000 7f8e32a00640 1 --2- 172.21.15.43:0/4252485076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e34102790 0x7f8e34074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:54.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.056+0000 7f8e32a00640 1 -- 172.21.15.43:0/4252485076 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e341a68f0 con 0x7f8e34101de0 2024-09-19T05:11:54.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.056+0000 7f8e33400640 1 --2- 172.21.15.43:0/4252485076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e34106f10 0x7f8e34074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:11:54.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.056+0000 7f8e2be00640 1 --2- 172.21.15.43:0/4252485076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e34102790 0x7f8e34074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:11:54.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.056+0000 7f8e32a00640 1 --2- 172.21.15.43:0/4252485076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e34101de0 0x7f8e340744f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8e2400d720 tx=0x7f8e2400dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:54.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.056+0000 7f8e2b400640 1 -- 172.21.15.43:0/4252485076 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e24017070 con 0x7f8e34101de0 2024-09-19T05:11:54.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.056+0000 7f8e39600640 1 -- 172.21.15.43:0/4252485076 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e341a6be0 con 0x7f8e34101de0 2024-09-19T05:11:54.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.056+0000 7f8e2b400640 1 -- 172.21.15.43:0/4252485076 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e2400de30 con 0x7f8e34101de0 2024-09-19T05:11:54.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.056+0000 7f8e2b400640 1 -- 172.21.15.43:0/4252485076 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e240047a0 con 0x7f8e34101de0 2024-09-19T05:11:54.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.056+0000 7f8e39600640 1 -- 172.21.15.43:0/4252485076 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e341a7120 con 0x7f8e34101de0 2024-09-19T05:11:54.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.059+0000 7f8e3c058740 1 Processor -- start 2024-09-19T05:11:54.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.059+0000 7f8e2b400640 1 -- 172.21.15.43:0/4252485076 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f8e240099d0 con 0x7f8e34101de0 2024-09-19T05:11:54.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.059+0000 7f8e3c058740 1 -- start start 2024-09-19T05:11:54.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.060+0000 7f8e3c058740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20d7bc70 0x55cf20d7c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:54.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.060+0000 7f8e3c058740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf20d7c590 0x55cf20d86910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:54.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.060+0000 7f8e3c058740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf20d86e50 0x55cf20d89240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:54.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.060+0000 7f8e3c058740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cf20ce23e0 con 0x55cf20d86e50 2024-09-19T05:11:54.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.060+0000 7f8e3c058740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55cf20c89d80 con 0x55cf20d7c590 2024-09-19T05:11:54.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.060+0000 7f8e3c058740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55cf20cde3a0 con 0x55cf20d7bc70 2024-09-19T05:11:54.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.060+0000 7f8e2be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf20d86e50 0x55cf20d89240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:54.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.060+0000 7f8e2be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf20d86e50 0x55cf20d89240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43128/0 (socket says 172.21.15.43:43128) 2024-09-19T05:11:54.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.060+0000 7f8e33400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20d7bc70 0x55cf20d7c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:54.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.060+0000 7f8e2be00640 1 -- 172.21.15.43:0/2761302652 learned_addr learned my addr 172.21.15.43:0/2761302652 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:54.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.060+0000 7f8e32a00640 1 --2- 172.21.15.43:0/2761302652 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf20d7c590 0x55cf20d86910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:54.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.060+0000 7f8e33400640 1 -- 172.21.15.43:0/2761302652 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf20d7c590 msgr2=0x55cf20d86910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:54.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.060+0000 7f8e33400640 1 --2- 172.21.15.43:0/2761302652 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf20d7c590 0x55cf20d86910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:54.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.060+0000 7f8e33400640 1 -- 172.21.15.43:0/2761302652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf20d86e50 msgr2=0x55cf20d89240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:54.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.060+0000 7f8e33400640 1 --2- 172.21.15.43:0/2761302652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf20d86e50 0x55cf20d89240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:54.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.060+0000 7f8e33400640 1 -- 172.21.15.43:0/2761302652 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf20ca8ae0 con 0x55cf20d7bc70 2024-09-19T05:11:54.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.061+0000 7f8e33400640 1 --2- 172.21.15.43:0/2761302652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20d7bc70 0x55cf20d7c050 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c010710 tx=0x7f8e2c010bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12c4985986ca2b46 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:54.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.061+0000 7f8e23400640 1 -- 172.21.15.43:0/2761302652 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c004b20 con 0x55cf20d7bc70 2024-09-19T05:11:54.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.061+0000 7f8e23400640 1 -- 172.21.15.43:0/2761302652 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e2c00edb0 con 0x55cf20d7bc70 2024-09-19T05:11:54.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.061+0000 7f8e23400640 1 -- 172.21.15.43:0/2761302652 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c005690 con 0x55cf20d7bc70 2024-09-19T05:11:54.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.062+0000 7f8e2b400640 1 --2- 172.21.15.43:0/4252485076 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f8e180a3430 0x7f8e180a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:54.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.062+0000 7f8e3c058740 1 -- 172.21.15.43:0/2761302652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20d7bc70 msgr2=0x55cf20d7c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:54.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.062+0000 7f8e3c058740 1 --2- 172.21.15.43:0/2761302652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20d7bc70 0x55cf20d7c050 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c010710 tx=0x7f8e2c010bd0 comp rx=0 tx=0).stop 2024-09-19T05:11:54.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.062+0000 7f8e2b400640 1 -- 172.21.15.43:0/4252485076 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f8e240cf910 con 0x7f8e34101de0 2024-09-19T05:11:54.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.062+0000 7f8e32a00640 1 --2- 172.21.15.43:0/4252485076 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f8e180a3430 0x7f8e180a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:54.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.062+0000 7f8e3c058740 1 -- 172.21.15.43:0/2761302652 shutdown_connections 2024-09-19T05:11:54.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.062+0000 7f8e3c058740 1 --2- 172.21.15.43:0/2761302652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf20d86e50 0x55cf20d89240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:54.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.062+0000 7f8e3c058740 1 --2- 172.21.15.43:0/2761302652 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf20d7c590 0x55cf20d86910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:54.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.062+0000 7f8e3c058740 1 --2- 172.21.15.43:0/2761302652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20d7bc70 0x55cf20d7c050 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:54.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.062+0000 7f8e3c058740 1 -- 172.21.15.43:0/2761302652 >> 172.21.15.43:0/2761302652 conn(0x55cf20d78200 msgr2=0x55cf20d785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:54.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.062+0000 7f8e3c058740 1 -- 172.21.15.43:0/2761302652 shutdown_connections 2024-09-19T05:11:54.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.062+0000 7f8e3c058740 1 -- 172.21.15.43:0/2761302652 wait complete. 2024-09-19T05:11:54.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.062+0000 7f8e32a00640 1 --2- 172.21.15.43:0/4252485076 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f8e180a3430 0x7f8e180a58f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x55cf20d79060 tx=0x7f8e240d5530 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:54.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.063+0000 7f8e3c058740 1 -- start start 2024-09-19T05:11:54.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.063+0000 7f8e3c058740 1 Processor -- start 2024-09-19T05:11:54.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.063+0000 7f8e3c058740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf20d7bc70 0x55cf20d83c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:54.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.063+0000 7f8e3c058740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20d7c590 0x55cf20d841d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:54.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.063+0000 7f8e3c058740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf20d86e50 0x55cf20d84710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:54.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.063+0000 7f8e3c058740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cf20d89a20 con 0x55cf20d86e50 2024-09-19T05:11:54.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.063+0000 7f8e3c058740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55cf20c9d0f0 con 0x55cf20d7bc70 2024-09-19T05:11:54.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.063+0000 7f8e3c058740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55cf20d89ba0 con 0x55cf20d7c590 2024-09-19T05:11:54.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.063+0000 7f8e3c058740 10 client.0 _ll_register_callbacks cb 0x55cf20c84bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:54.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.064+0000 7f8e33400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf20d86e50 0x55cf20d84710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:54.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.064+0000 7f8e2be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20d7c590 0x55cf20d841d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:54.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.064+0000 7f8e33400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf20d86e50 0x55cf20d84710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43132/0 (socket says 172.21.15.43:43132) 2024-09-19T05:11:54.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.064+0000 7f8e2be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20d7c590 0x55cf20d841d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:51852/0 (socket says 172.21.15.43:51852) 2024-09-19T05:11:54.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.064+0000 7f8e33400640 1 -- 172.21.15.43:0/368786593 learned_addr learned my addr 172.21.15.43:0/368786593 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:54.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.064+0000 7f8e33400640 1 --2- 172.21.15.43:0/368786593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf20d7bc70 0x55cf20d83c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:54.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.064+0000 7f8e2be00640 1 -- 172.21.15.43:0/368786593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf20d7bc70 msgr2=0x55cf20d83c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:54.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.064+0000 7f8e2be00640 1 --2- 172.21.15.43:0/368786593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf20d7bc70 0x55cf20d83c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:54.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.064+0000 7f8e2be00640 1 -- 172.21.15.43:0/368786593 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf20d86e50 msgr2=0x55cf20d84710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:54.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.064+0000 7f8e2be00640 1 --2- 172.21.15.43:0/368786593 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf20d86e50 0x55cf20d84710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:54.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.064+0000 7f8e2be00640 1 -- 172.21.15.43:0/368786593 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf20a533c0 con 0x55cf20d7c590 2024-09-19T05:11:54.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.064+0000 7f8e2be00640 1 --2- 172.21.15.43:0/368786593 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20d7c590 0x55cf20d841d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c005c40 tx=0x7f8e1c0319b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e646e3f7d7294a62 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:54.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.064+0000 7f8e17e00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:11:54.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.065+0000 7f8e17e00640 1 -- 172.21.15.43:0/368786593 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c035040 con 0x55cf20d7c590 2024-09-19T05:11:54.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.065+0000 7f8e3c058740 1 -- 172.21.15.43:0/368786593 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e2c00edb0 con 0x55cf20d7c590 2024-09-19T05:11:54.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.065+0000 7f8e17e00640 1 -- 172.21.15.43:0/368786593 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f8e1c004240 con 0x55cf20d7c590 2024-09-19T05:11:54.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.065+0000 7f8e17e00640 1 -- 172.21.15.43:0/368786593 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8e1c004440 con 0x55cf20d7c590 2024-09-19T05:11:54.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.065+0000 7f8e3c058740 10 client.26731 fetch_fsmap learned FSMap version 29 2024-09-19T05:11:54.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.065+0000 7f8e3c058740 1 -- 172.21.15.43:0/368786593 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cf20d959e0 con 0x55cf20d7c590 2024-09-19T05:11:54.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.065+0000 7f8e17e00640 1 -- 172.21.15.43:0/368786593 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e2c00edb0 con 0x55cf20d7c590 2024-09-19T05:11:54.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.065+0000 7f8e17e00640 1 -- 172.21.15.43:0/368786593 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8e1c004c90 con 0x55cf20d7c590 2024-09-19T05:11:54.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.066+0000 7f8e3c058740 10 client.26731 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:11:54.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.066+0000 7f8e3c058740 10 client.26731 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:11:54.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.066+0000 7f8e3c058740 20 client.26731 populate_metadata read hostname 'smithi043' 2024-09-19T05:11:54.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.066+0000 7f8e3c058740 1 --2- 172.21.15.43:0/368786593 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55cf20d96ea0 0x55cf20d99290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:54.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.066+0000 7f8e3c058740 4 client.26731 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:11:54.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.066+0000 7f8e3c058740 1 -- 172.21.15.43:0/368786593 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf20cef950 con 0x55cf20d96ea0 2024-09-19T05:11:54.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.066+0000 7f8e33400640 1 --2- 172.21.15.43:0/368786593 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55cf20d96ea0 0x55cf20d99290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:54.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.066+0000 7f8e33400640 1 --2- 172.21.15.43:0/368786593 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55cf20d96ea0 0x55cf20d99290 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=428418ef52f2cce9 server_cookie=f01987d7f7e894db in_seq=0 out_seq=0 2024-09-19T05:11:54.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.066+0000 7f8e17e00640 10 client.26731 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:11:54.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.067+0000 7f8e17e00640 1 -- 172.21.15.43:0/368786593 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cf20cef950 con 0x55cf20d96ea0 2024-09-19T05:11:54.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.067+0000 7f8e17e00640 10 client.26731 handle_command_reply: tid=0 2024-09-19T05:11:54.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.067+0000 7f8e3c058740 1 client.26731 shutdown 2024-09-19T05:11:54.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.067+0000 7f8e3c058740 1 -- 172.21.15.43:0/368786593 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20d7c590 msgr2=0x55cf20d841d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:54.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.067+0000 7f8e3c058740 1 --2- 172.21.15.43:0/368786593 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20d7c590 0x55cf20d841d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c005c40 tx=0x7f8e1c0319b0 comp rx=0 tx=0).stop 2024-09-19T05:11:54.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.067+0000 7f8e3c058740 1 -- 172.21.15.43:0/368786593 shutdown_connections 2024-09-19T05:11:54.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.067+0000 7f8e3c058740 1 --2- 172.21.15.43:0/368786593 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55cf20d96ea0 0x55cf20d99290 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:54.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.067+0000 7f8e3c058740 1 --2- 172.21.15.43:0/368786593 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf20d86e50 0x55cf20d84710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:54.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.067+0000 7f8e3c058740 1 --2- 172.21.15.43:0/368786593 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20d7c590 0x55cf20d841d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:54.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.067+0000 7f8e3c058740 1 --2- 172.21.15.43:0/368786593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf20d7bc70 0x55cf20d83c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:54.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.067+0000 7f8e17e00640 0 client.26731 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:11:54.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.067+0000 7f8e3c058740 1 -- 172.21.15.43:0/368786593 >> 172.21.15.43:0/368786593 conn(0x55cf20d78200 msgr2=0x55cf20d77380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:54.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.068+0000 7f8e3c058740 1 -- 172.21.15.43:0/368786593 shutdown_connections 2024-09-19T05:11:54.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.068+0000 7f8e3c058740 1 -- 172.21.15.43:0/368786593 wait complete. 2024-09-19T05:11:54.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.068+0000 7f8e3c058740 20 client.26731 trim_cache size 0 max 16384 2024-09-19T05:11:54.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.081+0000 7f8e3c058740 1 Processor -- start 2024-09-19T05:11:54.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.081+0000 7f8e3c058740 1 -- start start 2024-09-19T05:11:54.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.082+0000 7f8e3c058740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf20d7bc70 0x55cf20d9eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:54.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.082+0000 7f8e3c058740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20d7c590 0x55cf20d9f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:54.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.082+0000 7f8e3c058740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf20d86e50 0x55cf20d82430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:54.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.082+0000 7f8e3c058740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8e341afa30 con 0x55cf20d7bc70 2024-09-19T05:11:54.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.082+0000 7f8e3c058740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8e341b1570 con 0x55cf20d86e50 2024-09-19T05:11:54.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.082+0000 7f8e3c058740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8e341b0100 con 0x55cf20d7c590 2024-09-19T05:11:54.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.082+0000 7f8e33400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf20d7bc70 0x55cf20d9eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:54.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.082+0000 7f8e33400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf20d7bc70 0x55cf20d9eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43146/0 (socket says 172.21.15.43:43146) 2024-09-19T05:11:54.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.082+0000 7f8e33400640 1 -- 172.21.15.43:0/323523995 learned_addr learned my addr 172.21.15.43:0/323523995 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:54.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.082+0000 7f8e2be00640 1 --2- 172.21.15.43:0/323523995 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20d7c590 0x55cf20d9f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:54.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.082+0000 7f8e33400640 1 -- 172.21.15.43:0/323523995 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20d7c590 msgr2=0x55cf20d9f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:54.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.082+0000 7f8e33400640 1 --2- 172.21.15.43:0/323523995 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20d7c590 0x55cf20d9f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:54.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.082+0000 7f8e33400640 1 -- 172.21.15.43:0/323523995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf20d86e50 msgr2=0x55cf20d82430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:54.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.082+0000 7f8e33400640 1 --2- 172.21.15.43:0/323523995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf20d86e50 0x55cf20d82430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:54.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.082+0000 7f8e33400640 1 -- 172.21.15.43:0/323523995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf20d78030 con 0x55cf20d7bc70 2024-09-19T05:11:54.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.082+0000 7f8e2be00640 1 --2- 172.21.15.43:0/323523995 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20d7c590 0x55cf20d9f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:54.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.083+0000 7f8e33400640 1 --2- 172.21.15.43:0/323523995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf20d7bc70 0x55cf20d9eb60 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c000f60 tx=0x7f8e2c009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24fd7032e1acef56 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:54.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.083+0000 7f8e22a00640 1 -- 172.21.15.43:0/323523995 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c025040 con 0x55cf20d7bc70 2024-09-19T05:11:54.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.083+0000 7f8e22a00640 1 -- 172.21.15.43:0/323523995 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e2c00edb0 con 0x55cf20d7bc70 2024-09-19T05:11:54.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.083+0000 7f8e22a00640 1 -- 172.21.15.43:0/323523995 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c012450 con 0x55cf20d7bc70 2024-09-19T05:11:54.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.083+0000 7f8e3c058740 1 -- 172.21.15.43:0/323523995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf20d7bc70 msgr2=0x55cf20d9eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:54.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.083+0000 7f8e3c058740 1 --2- 172.21.15.43:0/323523995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf20d7bc70 0x55cf20d9eb60 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c000f60 tx=0x7f8e2c009600 comp rx=0 tx=0).stop 2024-09-19T05:11:54.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.084+0000 7f8e3c058740 1 -- 172.21.15.43:0/323523995 shutdown_connections 2024-09-19T05:11:54.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.084+0000 7f8e3c058740 1 --2- 172.21.15.43:0/323523995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf20d86e50 0x55cf20d82430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:54.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.084+0000 7f8e3c058740 1 --2- 172.21.15.43:0/323523995 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20d7c590 0x55cf20d9f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:54.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.084+0000 7f8e3c058740 1 --2- 172.21.15.43:0/323523995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf20d7bc70 0x55cf20d9eb60 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:54.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.084+0000 7f8e3c058740 1 -- 172.21.15.43:0/323523995 >> 172.21.15.43:0/323523995 conn(0x55cf20d78200 msgr2=0x55cf20d85d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:54.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.084+0000 7f8e3c058740 1 -- 172.21.15.43:0/323523995 shutdown_connections 2024-09-19T05:11:54.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.084+0000 7f8e3c058740 1 -- 172.21.15.43:0/323523995 wait complete. 2024-09-19T05:11:54.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.084+0000 7f8e3c058740 1 -- start start 2024-09-19T05:11:54.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.084+0000 7f8e3c058740 1 Processor -- start 2024-09-19T05:11:54.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.085+0000 7f8e3c058740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf20d7bc70 0x55cf20d8aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:54.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.085+0000 7f8e3c058740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20d7c590 0x55cf20d8b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:54.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.085+0000 7f8e3c058740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf20d86e50 0x55cf20d8b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:54.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.085+0000 7f8e3c058740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8e341b0ea0 con 0x55cf20d86e50 2024-09-19T05:11:54.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.085+0000 7f8e3c058740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8e341b07d0 con 0x55cf20d7bc70 2024-09-19T05:11:54.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.085+0000 7f8e3c058740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8e341ada30 con 0x55cf20d7c590 2024-09-19T05:11:54.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.085+0000 7f8e3c058740 10 client.0 _ll_register_callbacks cb 0x7f8e1c009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:54.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.085+0000 7f8e2be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20d7c590 0x55cf20d8b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:54.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.085+0000 7f8e33400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf20d7bc70 0x55cf20d8aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:54.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.085+0000 7f8e2be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20d7c590 0x55cf20d8b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:51870/0 (socket says 172.21.15.43:51870) 2024-09-19T05:11:54.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.085+0000 7f8e2be00640 1 -- 172.21.15.43:0/4227512002 learned_addr learned my addr 172.21.15.43:0/4227512002 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:54.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.085+0000 7f8e33400640 1 --2- 172.21.15.43:0/4227512002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf20d86e50 0x55cf20d8b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:54.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.085+0000 7f8e2be00640 1 -- 172.21.15.43:0/4227512002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf20d7bc70 msgr2=0x55cf20d8aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:54.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.085+0000 7f8e2be00640 1 --2- 172.21.15.43:0/4227512002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf20d7bc70 0x55cf20d8aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:54.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.085+0000 7f8e2be00640 1 -- 172.21.15.43:0/4227512002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf20d86e50 msgr2=0x55cf20d8b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:54.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.085+0000 7f8e2be00640 1 --2- 172.21.15.43:0/4227512002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf20d86e50 0x55cf20d8b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:54.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.085+0000 7f8e2be00640 1 -- 172.21.15.43:0/4227512002 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf20d7eae0 con 0x55cf20d7c590 2024-09-19T05:11:54.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.085+0000 7f8e33400640 1 --2- 172.21.15.43:0/4227512002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf20d86e50 0x55cf20d8b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:54.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.086+0000 7f8e2be00640 1 --2- 172.21.15.43:0/4227512002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20d7c590 0x55cf20d8b020 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c0036c0 tx=0x7f8e1c031c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbe1666b26947214 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:54.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.086+0000 7f8e17e00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:11:54.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.086+0000 7f8e17e00640 1 -- 172.21.15.43:0/4227512002 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c035070 con 0x55cf20d7c590 2024-09-19T05:11:54.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.086+0000 7f8e3c058740 1 -- 172.21.15.43:0/4227512002 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e2c00edb0 con 0x55cf20d7c590 2024-09-19T05:11:54.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.086+0000 7f8e17e00640 1 -- 172.21.15.43:0/4227512002 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f8e1c003f30 con 0x55cf20d7c590 2024-09-19T05:11:54.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.086+0000 7f8e17e00640 1 -- 172.21.15.43:0/4227512002 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8e1c053070 con 0x55cf20d7c590 2024-09-19T05:11:54.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.086+0000 7f8e3c058740 10 client.26737 fetch_fsmap learned FSMap version 29 2024-09-19T05:11:54.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.086+0000 7f8e3c058740 1 -- 172.21.15.43:0/4227512002 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cf20d792b0 con 0x55cf20d7c590 2024-09-19T05:11:54.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.086+0000 7f8e17e00640 1 -- 172.21.15.43:0/4227512002 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e2c00edb0 con 0x55cf20d7c590 2024-09-19T05:11:54.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.087+0000 7f8e17e00640 1 -- 172.21.15.43:0/4227512002 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8e1c046c80 con 0x55cf20d7c590 2024-09-19T05:11:54.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.087+0000 7f8e3c058740 10 client.26737 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:11:54.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.087+0000 7f8e3c058740 10 client.26737 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:11:54.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.087+0000 7f8e3c058740 20 client.26737 populate_metadata read hostname 'smithi043' 2024-09-19T05:11:54.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.087+0000 7f8e3c058740 1 --2- 172.21.15.43:0/4227512002 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55cf20e0a930 0x55cf20e0cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:54.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.087+0000 7f8e3c058740 4 client.26737 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:11:54.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.087+0000 7f8e3c058740 1 -- 172.21.15.43:0/4227512002 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cf20ce9e60 con 0x55cf20e0a930 2024-09-19T05:11:54.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.087+0000 7f8e33400640 1 --2- 172.21.15.43:0/4227512002 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55cf20e0a930 0x55cf20e0cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:54.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.087+0000 7f8e33400640 1 --2- 172.21.15.43:0/4227512002 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55cf20e0a930 0x55cf20e0cd50 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2d9eedbd83cb64e server_cookie=91af2b5dbfd60427 in_seq=0 out_seq=0 2024-09-19T05:11:54.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.087+0000 7f8e17e00640 10 client.26737 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:11:54.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.088+0000 7f8e17e00640 1 -- 172.21.15.43:0/4227512002 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cf20ce9e60 con 0x55cf20e0a930 2024-09-19T05:11:54.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.088+0000 7f8e17e00640 10 client.26737 handle_command_reply: tid=0 2024-09-19T05:11:54.107 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:11:54.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.088+0000 7f8e3c058740 1 client.26737 shutdown 2024-09-19T05:11:54.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.088+0000 7f8e3c058740 1 -- 172.21.15.43:0/4227512002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20d7c590 msgr2=0x55cf20d8b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:54.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.088+0000 7f8e3c058740 1 --2- 172.21.15.43:0/4227512002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20d7c590 0x55cf20d8b020 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c0036c0 tx=0x7f8e1c031c50 comp rx=0 tx=0).stop 2024-09-19T05:11:54.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.088+0000 7f8e3c058740 1 -- 172.21.15.43:0/4227512002 shutdown_connections 2024-09-19T05:11:54.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.088+0000 7f8e3c058740 1 --2- 172.21.15.43:0/4227512002 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55cf20e0a930 0x55cf20e0cd50 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:54.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.088+0000 7f8e3c058740 1 --2- 172.21.15.43:0/4227512002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cf20d86e50 0x55cf20d8b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:54.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.088+0000 7f8e3c058740 1 --2- 172.21.15.43:0/4227512002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20d7c590 0x55cf20d8b020 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:54.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.088+0000 7f8e3c058740 1 --2- 172.21.15.43:0/4227512002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf20d7bc70 0x55cf20d8aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:54.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.088+0000 7f8e17e00640 0 client.26737 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:11:54.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.088+0000 7f8e3c058740 1 -- 172.21.15.43:0/4227512002 >> 172.21.15.43:0/4227512002 conn(0x55cf20d78200 msgr2=0x55cf20d83c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:54.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.088+0000 7f8e3c058740 1 -- 172.21.15.43:0/4227512002 shutdown_connections 2024-09-19T05:11:54.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.088+0000 7f8e3c058740 1 -- 172.21.15.43:0/4227512002 wait complete. 2024-09-19T05:11:54.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.088+0000 7f8e3c058740 20 client.26737 trim_cache size 0 max 16384 2024-09-19T05:11:54.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.089+0000 7f8e39600640 1 -- 172.21.15.43:0/4252485076 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f8e180a3430 msgr2=0x7f8e180a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:54.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.089+0000 7f8e39600640 1 --2- 172.21.15.43:0/4252485076 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f8e180a3430 0x7f8e180a58f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x55cf20d79060 tx=0x7f8e240d5530 comp rx=0 tx=0).stop 2024-09-19T05:11:54.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.089+0000 7f8e39600640 1 -- 172.21.15.43:0/4252485076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e34101de0 msgr2=0x7f8e340744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:54.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.089+0000 7f8e39600640 1 --2- 172.21.15.43:0/4252485076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e34101de0 0x7f8e340744f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8e2400d720 tx=0x7f8e2400dbe0 comp rx=0 tx=0).stop 2024-09-19T05:11:54.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.089+0000 7f8e39600640 1 -- 172.21.15.43:0/4252485076 shutdown_connections 2024-09-19T05:11:54.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.089+0000 7f8e39600640 1 --2- 172.21.15.43:0/4252485076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e34106f10 0x7f8e34074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:54.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.089+0000 7f8e39600640 1 --2- 172.21.15.43:0/4252485076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e34102790 0x7f8e34074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:54.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.089+0000 7f8e39600640 1 --2- 172.21.15.43:0/4252485076 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f8e180a3430 0x7f8e180a58f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:54.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.089+0000 7f8e39600640 1 --2- 172.21.15.43:0/4252485076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e34101de0 0x7f8e340744f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:54.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.089+0000 7f8e39600640 1 -- 172.21.15.43:0/4252485076 >> 172.21.15.43:0/4252485076 conn(0x7f8e340fd8c0 msgr2=0x7f8e340fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:54.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.089+0000 7f8e39600640 1 -- 172.21.15.43:0/4252485076 shutdown_connections 2024-09-19T05:11:54.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:54.089+0000 7f8e39600640 1 -- 172.21.15.43:0/4252485076 wait complete. 2024-09-19T05:11:54.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:11:54 smithi096 ceph-mon[35085]: pgmap v222: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:11:54.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:11:54 smithi043 ceph-mon[30800]: pgmap v222: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:11:54.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:11:54 smithi151 ceph-mon[35425]: pgmap v222: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:11:55.876 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T05:11:55.877 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T05:11:56.266 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:11:56.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:11:56 smithi043 ceph-mon[30800]: pgmap v223: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:11:56.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:11:56 smithi151 ceph-mon[35425]: pgmap v223: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:11:56.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:11:56 smithi096 ceph-mon[35085]: pgmap v223: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:11:56.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.807+0000 7f17e2800640 1 -- 172.21.15.43:0/2962213737 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17dc103ff0 msgr2=0x7f17dc1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:56.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.808+0000 7f17e2800640 1 --2- 172.21.15.43:0/2962213737 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17dc103ff0 0x7f17dc1043d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f17cc009860 tx=0x7f17cc02a340 comp rx=0 tx=0).stop 2024-09-19T05:11:56.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.808+0000 7f17e2800640 1 -- 172.21.15.43:0/2962213737 shutdown_connections 2024-09-19T05:11:56.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.808+0000 7f17e2800640 1 --2- 172.21.15.43:0/2962213737 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17dc109120 0x7f17dc110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:56.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.808+0000 7f17e2800640 1 --2- 172.21.15.43:0/2962213737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17dc1049a0 0x7f17dc1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:56.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.808+0000 7f17e2800640 1 --2- 172.21.15.43:0/2962213737 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17dc103ff0 0x7f17dc1043d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:56.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.808+0000 7f17e2800640 1 -- 172.21.15.43:0/2962213737 >> 172.21.15.43:0/2962213737 conn(0x7f17dc0ffa90 msgr2=0x7f17dc101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:56.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.808+0000 7f17e2800640 1 -- 172.21.15.43:0/2962213737 shutdown_connections 2024-09-19T05:11:56.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.808+0000 7f17e2800640 1 -- 172.21.15.43:0/2962213737 wait complete. 2024-09-19T05:11:56.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.809+0000 7f17e2800640 1 Processor -- start 2024-09-19T05:11:56.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.809+0000 7f17e2800640 1 -- start start 2024-09-19T05:11:56.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.809+0000 7f17e2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17dc103ff0 0x7f17dc19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:56.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.809+0000 7f17e2800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17dc1049a0 0x7f17dc19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:56.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.809+0000 7f17e2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17dc109120 0x7f17dc19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:56.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.809+0000 7f17e2800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc114150 con 0x7f17dc103ff0 2024-09-19T05:11:56.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.809+0000 7f17e2800640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc113fd0 con 0x7f17dc1049a0 2024-09-19T05:11:56.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.809+0000 7f17e2800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1142d0 con 0x7f17dc109120 2024-09-19T05:11:56.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.810+0000 7f17db400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17dc103ff0 0x7f17dc19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:56.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.810+0000 7f17db400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17dc103ff0 0x7f17dc19a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35086/0 (socket says 172.21.15.43:35086) 2024-09-19T05:11:56.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.810+0000 7f17db400640 1 -- 172.21.15.43:0/4009783972 learned_addr learned my addr 172.21.15.43:0/4009783972 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:56.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.810+0000 7f17d3e00640 1 --2- 172.21.15.43:0/4009783972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17dc1049a0 0x7f17dc19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:56.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.810+0000 7f17dbe00640 1 --2- 172.21.15.43:0/4009783972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17dc109120 0x7f17dc19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:56.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.810+0000 7f17db400640 1 -- 172.21.15.43:0/4009783972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17dc109120 msgr2=0x7f17dc19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:56.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.810+0000 7f17db400640 1 --2- 172.21.15.43:0/4009783972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17dc109120 0x7f17dc19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:56.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.810+0000 7f17db400640 1 -- 172.21.15.43:0/4009783972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17dc1049a0 msgr2=0x7f17dc19abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:56.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.810+0000 7f17db400640 1 --2- 172.21.15.43:0/4009783972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17dc1049a0 0x7f17dc19abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:56.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.810+0000 7f17db400640 1 -- 172.21.15.43:0/4009783972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17dc19f620 con 0x7f17dc103ff0 2024-09-19T05:11:56.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.810+0000 7f17d3e00640 1 --2- 172.21.15.43:0/4009783972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17dc1049a0 0x7f17dc19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:11:56.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.810+0000 7f17db400640 1 --2- 172.21.15.43:0/4009783972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17dc103ff0 0x7f17dc19a670 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f17cc035040 tx=0x7f17cc033b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:56.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.811+0000 7f17d8c00640 1 -- 172.21.15.43:0/4009783972 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc037040 con 0x7f17dc103ff0 2024-09-19T05:11:56.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.811+0000 7f17d8c00640 1 -- 172.21.15.43:0/4009783972 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17cc040070 con 0x7f17dc103ff0 2024-09-19T05:11:56.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.811+0000 7f17e2800640 1 -- 172.21.15.43:0/4009783972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17dc19f8b0 con 0x7f17dc103ff0 2024-09-19T05:11:56.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.811+0000 7f17d8c00640 1 -- 172.21.15.43:0/4009783972 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc0039b0 con 0x7f17dc103ff0 2024-09-19T05:11:56.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.811+0000 7f17e2800640 1 -- 172.21.15.43:0/4009783972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17dc1a71f0 con 0x7f17dc103ff0 2024-09-19T05:11:56.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.814+0000 7f17e5471740 1 Processor -- start 2024-09-19T05:11:56.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.814+0000 7f17d8c00640 1 -- 172.21.15.43:0/4009783972 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f17cc0305e0 con 0x7f17dc103ff0 2024-09-19T05:11:56.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.814+0000 7f17e5471740 1 -- start start 2024-09-19T05:11:56.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.814+0000 7f17e5471740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555d3a43aac0 0x555d3a43aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:56.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.814+0000 7f17e5471740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d3a43b3e0 0x555d3a445760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:56.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.814+0000 7f17e5471740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d3a445ca0 0x555d3a448090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:56.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.814+0000 7f17e5471740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555d3a3a1290 con 0x555d3a43aac0 2024-09-19T05:11:56.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.814+0000 7f17e5471740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555d3a348b40 con 0x555d3a445ca0 2024-09-19T05:11:56.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.814+0000 7f17e5471740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555d3a39d250 con 0x555d3a43b3e0 2024-09-19T05:11:56.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.815+0000 7f17dbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555d3a43aac0 0x555d3a43aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:56.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.815+0000 7f17dbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555d3a43aac0 0x555d3a43aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35090/0 (socket says 172.21.15.43:35090) 2024-09-19T05:11:56.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.815+0000 7f17dbe00640 1 -- 172.21.15.43:0/1965774039 learned_addr learned my addr 172.21.15.43:0/1965774039 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:56.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.815+0000 7f17d3e00640 1 --2- 172.21.15.43:0/1965774039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d3a445ca0 0x555d3a448090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:56.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.815+0000 7f17db400640 1 --2- 172.21.15.43:0/1965774039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d3a43b3e0 0x555d3a445760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:56.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.815+0000 7f17dbe00640 1 -- 172.21.15.43:0/1965774039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d3a43b3e0 msgr2=0x555d3a445760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:56.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.815+0000 7f17dbe00640 1 --2- 172.21.15.43:0/1965774039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d3a43b3e0 0x555d3a445760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:56.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.815+0000 7f17dbe00640 1 -- 172.21.15.43:0/1965774039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d3a445ca0 msgr2=0x555d3a448090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:56.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.815+0000 7f17dbe00640 1 --2- 172.21.15.43:0/1965774039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d3a445ca0 0x555d3a448090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:56.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.815+0000 7f17dbe00640 1 -- 172.21.15.43:0/1965774039 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d3a1126a0 con 0x555d3a43aac0 2024-09-19T05:11:56.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.815+0000 7f17d3e00640 1 --2- 172.21.15.43:0/1965774039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d3a445ca0 0x555d3a448090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:11:56.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.816+0000 7f17d8c00640 1 --2- 172.21.15.43:0/4009783972 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f17c00a33b0 0x7f17c00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:56.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.816+0000 7f17dbe00640 1 --2- 172.21.15.43:0/1965774039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555d3a43aac0 0x555d3a43aea0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f17d4009f50 tx=0x7f17d40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5fd4804b7142850 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:56.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.816+0000 7f17db400640 1 --2- 172.21.15.43:0/4009783972 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f17c00a33b0 0x7f17c00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:56.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.816+0000 7f17d8c00640 1 -- 172.21.15.43:0/4009783972 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f17cc0efc40 con 0x7f17dc103ff0 2024-09-19T05:11:56.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.816+0000 7f17d0c00640 1 -- 172.21.15.43:0/1965774039 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d4004d20 con 0x555d3a43aac0 2024-09-19T05:11:56.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.816+0000 7f17d0c00640 1 -- 172.21.15.43:0/1965774039 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17d4004050 con 0x555d3a43aac0 2024-09-19T05:11:56.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.816+0000 7f17d0c00640 1 -- 172.21.15.43:0/1965774039 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d4005280 con 0x555d3a43aac0 2024-09-19T05:11:56.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.817+0000 7f17e5471740 1 -- 172.21.15.43:0/1965774039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555d3a43aac0 msgr2=0x555d3a43aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:56.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.817+0000 7f17db400640 1 --2- 172.21.15.43:0/4009783972 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f17c00a33b0 0x7f17c00a5870 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x555d3a437eb0 tx=0x7f17cc02a340 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:56.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.817+0000 7f17e5471740 1 --2- 172.21.15.43:0/1965774039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555d3a43aac0 0x555d3a43aea0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f17d4009f50 tx=0x7f17d40118d0 comp rx=0 tx=0).stop 2024-09-19T05:11:56.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.817+0000 7f17e5471740 1 -- 172.21.15.43:0/1965774039 shutdown_connections 2024-09-19T05:11:56.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.817+0000 7f17e5471740 1 --2- 172.21.15.43:0/1965774039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d3a445ca0 0x555d3a448090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:56.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.817+0000 7f17e5471740 1 --2- 172.21.15.43:0/1965774039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d3a43b3e0 0x555d3a445760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:56.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.817+0000 7f17e5471740 1 --2- 172.21.15.43:0/1965774039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555d3a43aac0 0x555d3a43aea0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:56.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.817+0000 7f17e5471740 1 -- 172.21.15.43:0/1965774039 >> 172.21.15.43:0/1965774039 conn(0x555d3a437050 msgr2=0x555d3a437430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:56.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.817+0000 7f17e5471740 1 -- 172.21.15.43:0/1965774039 shutdown_connections 2024-09-19T05:11:56.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.817+0000 7f17e5471740 1 -- 172.21.15.43:0/1965774039 wait complete. 2024-09-19T05:11:56.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.817+0000 7f17e5471740 1 -- start start 2024-09-19T05:11:56.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.818+0000 7f17e5471740 1 Processor -- start 2024-09-19T05:11:56.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.818+0000 7f17e5471740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555d3a43aac0 0x555d3a4455c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:56.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.818+0000 7f17e5471740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d3a43b3e0 0x555d3a442870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:56.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.818+0000 7f17e5471740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d3a445ca0 0x555d3a442db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:56.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.818+0000 7f17e5471740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555d3a448910 con 0x555d3a43aac0 2024-09-19T05:11:56.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.818+0000 7f17e5471740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555d3a35bfa0 con 0x555d3a445ca0 2024-09-19T05:11:56.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.818+0000 7f17e5471740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555d3a448a90 con 0x555d3a43b3e0 2024-09-19T05:11:56.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.818+0000 7f17e5471740 10 client.0 _ll_register_callbacks cb 0x555d3a0afba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:56.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.818+0000 7f17dbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555d3a43aac0 0x555d3a4455c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:56.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.818+0000 7f17dbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555d3a43aac0 0x555d3a4455c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35104/0 (socket says 172.21.15.43:35104) 2024-09-19T05:11:56.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.818+0000 7f17dbe00640 1 -- 172.21.15.43:0/3891217958 learned_addr learned my addr 172.21.15.43:0/3891217958 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:56.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.819+0000 7f17d3e00640 1 --2- 172.21.15.43:0/3891217958 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d3a43b3e0 0x555d3a442870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:56.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.819+0000 7f17dbe00640 1 --2- 172.21.15.43:0/3891217958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d3a445ca0 0x555d3a442db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:56.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.819+0000 7f17dbe00640 1 -- 172.21.15.43:0/3891217958 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d3a43b3e0 msgr2=0x555d3a442870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:56.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.819+0000 7f17dbe00640 1 --2- 172.21.15.43:0/3891217958 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d3a43b3e0 0x555d3a442870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:56.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.819+0000 7f17dbe00640 1 -- 172.21.15.43:0/3891217958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d3a445ca0 msgr2=0x555d3a442db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:56.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.819+0000 7f17dbe00640 1 --2- 172.21.15.43:0/3891217958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d3a445ca0 0x555d3a442db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:56.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.819+0000 7f17dbe00640 1 -- 172.21.15.43:0/3891217958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d3a4435b0 con 0x555d3a43aac0 2024-09-19T05:11:56.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.819+0000 7f17dbe00640 1 --2- 172.21.15.43:0/3891217958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555d3a43aac0 0x555d3a4455c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f17d4005c40 tx=0x7f17d4004870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=314717b81fb0b908 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:56.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.819+0000 7f17c9600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:11:56.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.820+0000 7f17c9600640 1 -- 172.21.15.43:0/3891217958 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d402b070 con 0x555d3a43aac0 2024-09-19T05:11:56.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.820+0000 7f17e5471740 1 -- 172.21.15.43:0/3891217958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17d4004050 con 0x555d3a43aac0 2024-09-19T05:11:56.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.820+0000 7f17c9600640 1 -- 172.21.15.43:0/3891217958 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f17d4025650 con 0x555d3a43aac0 2024-09-19T05:11:56.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.820+0000 7f17c9600640 1 -- 172.21.15.43:0/3891217958 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f17d4003e10 con 0x555d3a43aac0 2024-09-19T05:11:56.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.820+0000 7f17e5471740 10 client.16791 fetch_fsmap learned FSMap version 29 2024-09-19T05:11:56.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.820+0000 7f17e5471740 1 -- 172.21.15.43:0/3891217958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555d3a4547f0 con 0x555d3a43aac0 2024-09-19T05:11:56.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.820+0000 7f17c9600640 1 -- 172.21.15.43:0/3891217958 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17d4004050 con 0x555d3a43aac0 2024-09-19T05:11:56.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.820+0000 7f17c9600640 1 -- 172.21.15.43:0/3891217958 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f17d40279d0 con 0x555d3a43aac0 2024-09-19T05:11:56.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.820+0000 7f17e5471740 10 client.16791 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:11:56.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.820+0000 7f17e5471740 10 client.16791 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:11:56.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.820+0000 7f17e5471740 20 client.16791 populate_metadata read hostname 'smithi043' 2024-09-19T05:11:56.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.820+0000 7f17e5471740 1 --2- 172.21.15.43:0/3891217958 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x555d3a455ba0 0x555d3a457f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:56.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.821+0000 7f17e5471740 4 client.16791 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:11:56.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.821+0000 7f17e5471740 1 -- 172.21.15.43:0/3891217958 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d3a3ae800 con 0x555d3a455ba0 2024-09-19T05:11:56.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.821+0000 7f17d3e00640 1 --2- 172.21.15.43:0/3891217958 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x555d3a455ba0 0x555d3a457f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:56.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.821+0000 7f17d3e00640 1 --2- 172.21.15.43:0/3891217958 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x555d3a455ba0 0x555d3a457f90 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe25f299e220c558 server_cookie=2f683558876c6902 in_seq=0 out_seq=0 2024-09-19T05:11:56.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.821+0000 7f17c9600640 10 client.16791 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:11:56.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.822+0000 7f17c9600640 1 -- 172.21.15.43:0/3891217958 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555d3a3ae800 con 0x555d3a455ba0 2024-09-19T05:11:56.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.822+0000 7f17c9600640 10 client.16791 handle_command_reply: tid=0 2024-09-19T05:11:56.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.822+0000 7f17e5471740 1 client.16791 shutdown 2024-09-19T05:11:56.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.822+0000 7f17e5471740 1 -- 172.21.15.43:0/3891217958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555d3a43aac0 msgr2=0x555d3a4455c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:56.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.822+0000 7f17e5471740 1 --2- 172.21.15.43:0/3891217958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555d3a43aac0 0x555d3a4455c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f17d4005c40 tx=0x7f17d4004870 comp rx=0 tx=0).stop 2024-09-19T05:11:56.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.823+0000 7f17e5471740 1 -- 172.21.15.43:0/3891217958 shutdown_connections 2024-09-19T05:11:56.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.823+0000 7f17e5471740 1 --2- 172.21.15.43:0/3891217958 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x555d3a455ba0 0x555d3a457f90 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:56.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.823+0000 7f17e5471740 1 --2- 172.21.15.43:0/3891217958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d3a445ca0 0x555d3a442db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:56.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.823+0000 7f17e5471740 1 --2- 172.21.15.43:0/3891217958 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d3a43b3e0 0x555d3a442870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:56.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.823+0000 7f17e5471740 1 --2- 172.21.15.43:0/3891217958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555d3a43aac0 0x555d3a4455c0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:56.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.823+0000 7f17e5471740 1 -- 172.21.15.43:0/3891217958 >> 172.21.15.43:0/3891217958 conn(0x555d3a437050 msgr2=0x555d3a4361c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:56.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.823+0000 7f17c9600640 0 client.16791 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:11:56.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.823+0000 7f17e5471740 1 -- 172.21.15.43:0/3891217958 shutdown_connections 2024-09-19T05:11:56.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.823+0000 7f17e5471740 1 -- 172.21.15.43:0/3891217958 wait complete. 2024-09-19T05:11:56.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.823+0000 7f17e5471740 20 client.16791 trim_cache size 0 max 16384 2024-09-19T05:11:56.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.841+0000 7f17e5471740 1 Processor -- start 2024-09-19T05:11:56.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.841+0000 7f17e5471740 1 -- start start 2024-09-19T05:11:56.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.841+0000 7f17e5471740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d3a43aac0 0x555d3a441850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:56.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.841+0000 7f17e5471740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555d3a43b3e0 0x555d3a45d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:56.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.841+0000 7f17e5471740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d3a445ca0 0x555d3a45dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:56.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.842+0000 7f17e5471740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1afa90 con 0x555d3a43b3e0 2024-09-19T05:11:56.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.842+0000 7f17e5471740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1b15d0 con 0x555d3a445ca0 2024-09-19T05:11:56.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.842+0000 7f17e5471740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1b0160 con 0x555d3a43aac0 2024-09-19T05:11:56.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.842+0000 7f17d3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555d3a43b3e0 0x555d3a45d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:56.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.842+0000 7f17d3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555d3a43b3e0 0x555d3a45d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35108/0 (socket says 172.21.15.43:35108) 2024-09-19T05:11:56.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.842+0000 7f17d3e00640 1 -- 172.21.15.43:0/3566484490 learned_addr learned my addr 172.21.15.43:0/3566484490 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:56.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.842+0000 7f17dbe00640 1 --2- 172.21.15.43:0/3566484490 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d3a43aac0 0x555d3a441850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:56.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.842+0000 7f17dbe00640 1 --2- 172.21.15.43:0/3566484490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d3a445ca0 0x555d3a45dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:56.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.842+0000 7f17d3e00640 1 -- 172.21.15.43:0/3566484490 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d3a43aac0 msgr2=0x555d3a441850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:56.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.842+0000 7f17d3e00640 1 --2- 172.21.15.43:0/3566484490 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d3a43aac0 0x555d3a441850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:56.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.842+0000 7f17d3e00640 1 -- 172.21.15.43:0/3566484490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d3a445ca0 msgr2=0x555d3a45dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:56.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.842+0000 7f17d3e00640 1 --2- 172.21.15.43:0/3566484490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d3a445ca0 0x555d3a45dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:56.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.842+0000 7f17d3e00640 1 -- 172.21.15.43:0/3566484490 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d3a43b210 con 0x555d3a43b3e0 2024-09-19T05:11:56.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.842+0000 7f17dbe00640 1 --2- 172.21.15.43:0/3566484490 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d3a43aac0 0x555d3a441850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:11:56.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.843+0000 7f17d3e00640 1 --2- 172.21.15.43:0/3566484490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555d3a43b3e0 0x555d3a45d6d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f17c4005fd0 tx=0x7f17c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37a448daa6c5dcae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:56.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.843+0000 7f17cbe00640 1 -- 172.21.15.43:0/3566484490 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c400faa0 con 0x555d3a43b3e0 2024-09-19T05:11:56.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.843+0000 7f17cbe00640 1 -- 172.21.15.43:0/3566484490 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17c400fc40 con 0x555d3a43b3e0 2024-09-19T05:11:56.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.843+0000 7f17cbe00640 1 -- 172.21.15.43:0/3566484490 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c4025630 con 0x555d3a43b3e0 2024-09-19T05:11:56.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.844+0000 7f17e5471740 1 -- 172.21.15.43:0/3566484490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555d3a43b3e0 msgr2=0x555d3a45d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:56.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.844+0000 7f17e5471740 1 --2- 172.21.15.43:0/3566484490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555d3a43b3e0 0x555d3a45d6d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f17c4005fd0 tx=0x7f17c40046c0 comp rx=0 tx=0).stop 2024-09-19T05:11:56.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.844+0000 7f17e5471740 1 -- 172.21.15.43:0/3566484490 shutdown_connections 2024-09-19T05:11:56.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.844+0000 7f17e5471740 1 --2- 172.21.15.43:0/3566484490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d3a445ca0 0x555d3a45dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:56.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.844+0000 7f17e5471740 1 --2- 172.21.15.43:0/3566484490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555d3a43b3e0 0x555d3a45d6d0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:56.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.844+0000 7f17e5471740 1 --2- 172.21.15.43:0/3566484490 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d3a43aac0 0x555d3a441850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:56.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.844+0000 7f17e5471740 1 -- 172.21.15.43:0/3566484490 >> 172.21.15.43:0/3566484490 conn(0x555d3a437050 msgr2=0x555d3a441140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:56.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.844+0000 7f17e5471740 1 -- 172.21.15.43:0/3566484490 shutdown_connections 2024-09-19T05:11:56.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.844+0000 7f17e5471740 1 -- 172.21.15.43:0/3566484490 wait complete. 2024-09-19T05:11:56.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.844+0000 7f17e5471740 1 -- start start 2024-09-19T05:11:56.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.845+0000 7f17e5471740 1 Processor -- start 2024-09-19T05:11:56.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.845+0000 7f17e5471740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555d3a43aac0 0x555d3a44dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:56.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.845+0000 7f17e5471740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d3a43b3e0 0x555d3a44e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:56.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.845+0000 7f17e5471740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d3a445ca0 0x555d3a449400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:56.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.845+0000 7f17e5471740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1b0f00 con 0x555d3a43aac0 2024-09-19T05:11:56.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.845+0000 7f17e5471740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1b0830 con 0x555d3a43b3e0 2024-09-19T05:11:56.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.845+0000 7f17e5471740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1ada90 con 0x555d3a445ca0 2024-09-19T05:11:56.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.845+0000 7f17e5471740 10 client.0 _ll_register_callbacks cb 0x7f17c4009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:56.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.845+0000 7f17dbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555d3a43aac0 0x555d3a44dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:56.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.845+0000 7f17dbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555d3a43aac0 0x555d3a44dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35110/0 (socket says 172.21.15.43:35110) 2024-09-19T05:11:56.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.845+0000 7f17dbe00640 1 -- 172.21.15.43:0/2464489766 learned_addr learned my addr 172.21.15.43:0/2464489766 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:56.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.845+0000 7f17d3e00640 1 --2- 172.21.15.43:0/2464489766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d3a43b3e0 0x555d3a44e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:56.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.846+0000 7f17dbe00640 1 --2- 172.21.15.43:0/2464489766 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d3a445ca0 0x555d3a449400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:56.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.846+0000 7f17dbe00640 1 -- 172.21.15.43:0/2464489766 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d3a445ca0 msgr2=0x555d3a449400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:56.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.846+0000 7f17dbe00640 1 --2- 172.21.15.43:0/2464489766 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d3a445ca0 0x555d3a449400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:56.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.846+0000 7f17dbe00640 1 -- 172.21.15.43:0/2464489766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d3a43b3e0 msgr2=0x555d3a44e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:56.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.846+0000 7f17dbe00640 1 --2- 172.21.15.43:0/2464489766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d3a43b3e0 0x555d3a44e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:56.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.846+0000 7f17dbe00640 1 -- 172.21.15.43:0/2464489766 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d3a449c30 con 0x555d3a43aac0 2024-09-19T05:11:56.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.846+0000 7f17d3e00640 1 --2- 172.21.15.43:0/2464489766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d3a43b3e0 0x555d3a44e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:11:56.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.846+0000 7f17dbe00640 1 --2- 172.21.15.43:0/2464489766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555d3a43aac0 0x555d3a44dfb0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f17d4027dd0 tx=0x7f17d4004870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ede4342a8ad28df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:56.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.846+0000 7f17c9600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:11:56.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.846+0000 7f17c9600640 1 -- 172.21.15.43:0/2464489766 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d4041020 con 0x555d3a43aac0 2024-09-19T05:11:56.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.846+0000 7f17c9600640 1 -- 172.21.15.43:0/2464489766 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f17d40048e0 con 0x555d3a43aac0 2024-09-19T05:11:56.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.846+0000 7f17e5471740 1 -- 172.21.15.43:0/2464489766 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17c400fc40 con 0x555d3a43aac0 2024-09-19T05:11:56.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.847+0000 7f17c9600640 1 -- 172.21.15.43:0/2464489766 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f17d402b040 con 0x555d3a43aac0 2024-09-19T05:11:56.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.847+0000 7f17e5471740 10 client.16797 fetch_fsmap learned FSMap version 29 2024-09-19T05:11:56.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.847+0000 7f17e5471740 1 -- 172.21.15.43:0/2464489766 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555d3a4c0280 con 0x555d3a43aac0 2024-09-19T05:11:56.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.847+0000 7f17c9600640 1 -- 172.21.15.43:0/2464489766 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17c400fc40 con 0x555d3a43aac0 2024-09-19T05:11:56.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.847+0000 7f17c9600640 1 -- 172.21.15.43:0/2464489766 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f17d4025c00 con 0x555d3a43aac0 2024-09-19T05:11:56.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.847+0000 7f17e5471740 10 client.16797 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:11:56.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.847+0000 7f17e5471740 10 client.16797 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:11:56.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.847+0000 7f17e5471740 20 client.16797 populate_metadata read hostname 'smithi043' 2024-09-19T05:11:56.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.847+0000 7f17e5471740 1 --2- 172.21.15.43:0/2464489766 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x555d3a4c08f0 0x555d3a4c2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:56.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.847+0000 7f17e5471740 4 client.16797 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T05:11:56.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.847+0000 7f17e5471740 1 -- 172.21.15.43:0/2464489766 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555d3a3a8d10 con 0x555d3a4c08f0 2024-09-19T05:11:56.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.848+0000 7f17d3e00640 1 --2- 172.21.15.43:0/2464489766 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x555d3a4c08f0 0x555d3a4c2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:56.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.848+0000 7f17d3e00640 1 --2- 172.21.15.43:0/2464489766 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x555d3a4c08f0 0x555d3a4c2d10 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efbb2e9714f684b0 server_cookie=b37f19f86d0a70a4 in_seq=0 out_seq=0 2024-09-19T05:11:56.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.848+0000 7f17c9600640 10 client.16797 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:11:56.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.850+0000 7f17c9600640 1 -- 172.21.15.43:0/2464489766 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555d3a3a8d10 con 0x555d3a4c08f0 2024-09-19T05:11:56.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.850+0000 7f17c9600640 10 client.16797 handle_command_reply: tid=0 2024-09-19T05:11:56.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.850+0000 7f17e5471740 1 client.16797 shutdown 2024-09-19T05:11:56.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.850+0000 7f17e5471740 1 -- 172.21.15.43:0/2464489766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555d3a43aac0 msgr2=0x555d3a44dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:56.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.850+0000 7f17e5471740 1 --2- 172.21.15.43:0/2464489766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555d3a43aac0 0x555d3a44dfb0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f17d4027dd0 tx=0x7f17d4004870 comp rx=0 tx=0).stop 2024-09-19T05:11:56.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.850+0000 7f17e5471740 1 -- 172.21.15.43:0/2464489766 shutdown_connections 2024-09-19T05:11:56.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.850+0000 7f17e5471740 1 --2- 172.21.15.43:0/2464489766 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x555d3a4c08f0 0x555d3a4c2d10 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:56.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.850+0000 7f17e5471740 1 --2- 172.21.15.43:0/2464489766 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d3a445ca0 0x555d3a449400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:56.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.850+0000 7f17e5471740 1 --2- 172.21.15.43:0/2464489766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d3a43b3e0 0x555d3a44e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:56.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.850+0000 7f17e5471740 1 --2- 172.21.15.43:0/2464489766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555d3a43aac0 0x555d3a44dfb0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:56.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.850+0000 7f17c9600640 0 client.16797 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:11:56.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.850+0000 7f17e5471740 1 -- 172.21.15.43:0/2464489766 >> 172.21.15.43:0/2464489766 conn(0x555d3a437050 msgr2=0x555d3a441960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:56.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.850+0000 7f17e5471740 1 -- 172.21.15.43:0/2464489766 shutdown_connections 2024-09-19T05:11:56.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.850+0000 7f17e5471740 1 -- 172.21.15.43:0/2464489766 wait complete. 2024-09-19T05:11:56.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.850+0000 7f17e5471740 20 client.16797 trim_cache size 0 max 16384 2024-09-19T05:11:56.857 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:11:56.857 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-09-19T05:11:56.857 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "16b261ba-3030-4747-904a-a5cb51625011", 2024-09-19T05:11:56.857 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-09-19T05:11:56.857 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:11:56.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.851+0000 7f17e2800640 1 -- 172.21.15.43:0/4009783972 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f17c00a33b0 msgr2=0x7f17c00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:56.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.851+0000 7f17e2800640 1 --2- 172.21.15.43:0/4009783972 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f17c00a33b0 0x7f17c00a5870 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x555d3a437eb0 tx=0x7f17cc02a340 comp rx=0 tx=0).stop 2024-09-19T05:11:56.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.851+0000 7f17e2800640 1 -- 172.21.15.43:0/4009783972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17dc103ff0 msgr2=0x7f17dc19a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:56.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.851+0000 7f17e2800640 1 --2- 172.21.15.43:0/4009783972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17dc103ff0 0x7f17dc19a670 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f17cc035040 tx=0x7f17cc033b60 comp rx=0 tx=0).stop 2024-09-19T05:11:56.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.851+0000 7f17e2800640 1 -- 172.21.15.43:0/4009783972 shutdown_connections 2024-09-19T05:11:56.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.851+0000 7f17e2800640 1 --2- 172.21.15.43:0/4009783972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17dc109120 0x7f17dc19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:56.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.851+0000 7f17e2800640 1 --2- 172.21.15.43:0/4009783972 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f17c00a33b0 0x7f17c00a5870 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:56.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.851+0000 7f17e2800640 1 --2- 172.21.15.43:0/4009783972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17dc1049a0 0x7f17dc19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:56.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.851+0000 7f17e2800640 1 --2- 172.21.15.43:0/4009783972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17dc103ff0 0x7f17dc19a670 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:56.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.851+0000 7f17e2800640 1 -- 172.21.15.43:0/4009783972 >> 172.21.15.43:0/4009783972 conn(0x7f17dc0ffa90 msgr2=0x7f17dc100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:56.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.851+0000 7f17e2800640 1 -- 172.21.15.43:0/4009783972 shutdown_connections 2024-09-19T05:11:56.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:56.851+0000 7f17e2800640 1 -- 172.21.15.43:0/4009783972 wait complete. 2024-09-19T05:11:57.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:11:57 smithi043 ceph-mon[30800]: scrub queued for path: / 2024-09-19T05:11:57.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:11:57 smithi043 ceph-mon[30800]: scrub summary: idle+waiting paths [/] 2024-09-19T05:11:57.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:11:57 smithi043 ceph-mon[30800]: scrub summary: active paths [/] 2024-09-19T05:11:57.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:11:57 smithi043 ceph-mon[30800]: scrub summary: idle+waiting paths [/] 2024-09-19T05:11:57.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:11:57 smithi151 ceph-mon[35425]: scrub queued for path: / 2024-09-19T05:11:57.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:11:57 smithi151 ceph-mon[35425]: scrub summary: idle+waiting paths [/] 2024-09-19T05:11:57.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:11:57 smithi151 ceph-mon[35425]: scrub summary: active paths [/] 2024-09-19T05:11:57.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:11:57 smithi151 ceph-mon[35425]: scrub summary: idle+waiting paths [/] 2024-09-19T05:11:57.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:11:57 smithi096 ceph-mon[35085]: scrub queued for path: / 2024-09-19T05:11:57.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:11:57 smithi096 ceph-mon[35085]: scrub summary: idle+waiting paths [/] 2024-09-19T05:11:57.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:11:57 smithi096 ceph-mon[35085]: scrub summary: active paths [/] 2024-09-19T05:11:57.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:11:57 smithi096 ceph-mon[35085]: scrub summary: idle+waiting paths [/] 2024-09-19T05:11:57.738 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:11:58.119 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:11:58.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:11:58 smithi043 ceph-mon[30800]: pgmap v224: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:11:58.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:11:58 smithi043 ceph-mon[30800]: scrub summary: idle 2024-09-19T05:11:58.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:11:58 smithi151 ceph-mon[35425]: pgmap v224: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:11:58.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:11:58 smithi151 ceph-mon[35425]: scrub summary: idle 2024-09-19T05:11:58.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:11:58 smithi096 ceph-mon[35085]: pgmap v224: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:11:58.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:11:58 smithi096 ceph-mon[35085]: scrub summary: idle 2024-09-19T05:11:58.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.725+0000 7f49e5000640 1 -- 172.21.15.43:0/1345380783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49e01049a0 msgr2=0x7f49e01089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:58.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.725+0000 7f49e5000640 1 --2- 172.21.15.43:0/1345380783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49e01049a0 0x7f49e01089f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f49cc009900 tx=0x7f49cc02a410 comp rx=0 tx=0).stop 2024-09-19T05:11:58.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.725+0000 7f49e5000640 1 -- 172.21.15.43:0/1345380783 shutdown_connections 2024-09-19T05:11:58.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.725+0000 7f49e5000640 1 --2- 172.21.15.43:0/1345380783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e0109120 0x7f49e0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:58.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.725+0000 7f49e5000640 1 --2- 172.21.15.43:0/1345380783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49e01049a0 0x7f49e01089f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:58.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.725+0000 7f49e5000640 1 --2- 172.21.15.43:0/1345380783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49e0103ff0 0x7f49e01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:58.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.725+0000 7f49e5000640 1 -- 172.21.15.43:0/1345380783 >> 172.21.15.43:0/1345380783 conn(0x7f49e00ffa90 msgr2=0x7f49e0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:58.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.725+0000 7f49e5000640 1 -- 172.21.15.43:0/1345380783 shutdown_connections 2024-09-19T05:11:58.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.726+0000 7f49e5000640 1 -- 172.21.15.43:0/1345380783 wait complete. 2024-09-19T05:11:58.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.726+0000 7f49e5000640 1 Processor -- start 2024-09-19T05:11:58.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.727+0000 7f49e5000640 1 -- start start 2024-09-19T05:11:58.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.727+0000 7f49e5000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49e0103ff0 0x7f49e019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:58.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.727+0000 7f49e5000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49e01049a0 0x7f49e019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:58.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.727+0000 7f49e5000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e0109120 0x7f49e019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:58.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.727+0000 7f49e5000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f49e0114140 con 0x7f49e01049a0 2024-09-19T05:11:58.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.727+0000 7f49e5000640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f49e0113fc0 con 0x7f49e0109120 2024-09-19T05:11:58.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.727+0000 7f49e5000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f49e01142c0 con 0x7f49e0103ff0 2024-09-19T05:11:58.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.727+0000 7f49dd400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49e01049a0 0x7f49e019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:58.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.727+0000 7f49dde00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49e0103ff0 0x7f49e019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:58.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.727+0000 7f49de800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e0109120 0x7f49e019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:58.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.727+0000 7f49dde00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49e0103ff0 0x7f49e019a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:59788/0 (socket says 172.21.15.43:59788) 2024-09-19T05:11:58.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.727+0000 7f49dde00640 1 -- 172.21.15.43:0/1030983598 learned_addr learned my addr 172.21.15.43:0/1030983598 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:58.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.728+0000 7f49de800640 1 -- 172.21.15.43:0/1030983598 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49e0103ff0 msgr2=0x7f49e019a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:58.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.728+0000 7f49de800640 1 --2- 172.21.15.43:0/1030983598 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49e0103ff0 0x7f49e019a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:58.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.728+0000 7f49de800640 1 -- 172.21.15.43:0/1030983598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49e01049a0 msgr2=0x7f49e019abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:58.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.728+0000 7f49de800640 1 --2- 172.21.15.43:0/1030983598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49e01049a0 0x7f49e019abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:58.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.728+0000 7f49de800640 1 -- 172.21.15.43:0/1030983598 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49e019f650 con 0x7f49e0109120 2024-09-19T05:11:58.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.728+0000 7f49de800640 1 --2- 172.21.15.43:0/1030983598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e0109120 0x7f49e019ef70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f49d8009fd0 tx=0x7f49d800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:58.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.728+0000 7f49d6a00640 1 -- 172.21.15.43:0/1030983598 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d800ce20 con 0x7f49e0109120 2024-09-19T05:11:58.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.728+0000 7f49d6a00640 1 -- 172.21.15.43:0/1030983598 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49d8005020 con 0x7f49e0109120 2024-09-19T05:11:58.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.728+0000 7f49e5000640 1 -- 172.21.15.43:0/1030983598 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49e019f940 con 0x7f49e0109120 2024-09-19T05:11:58.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.728+0000 7f49d6a00640 1 -- 172.21.15.43:0/1030983598 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d8010720 con 0x7f49e0109120 2024-09-19T05:11:58.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.729+0000 7f49e5000640 1 -- 172.21.15.43:0/1030983598 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49e01a7220 con 0x7f49e0109120 2024-09-19T05:11:58.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.732+0000 7f49e7a52740 1 Processor -- start 2024-09-19T05:11:58.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.732+0000 7f49d6a00640 1 -- 172.21.15.43:0/1030983598 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f49d8005540 con 0x7f49e0109120 2024-09-19T05:11:58.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.732+0000 7f49e7a52740 1 -- start start 2024-09-19T05:11:58.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.733+0000 7f49e7a52740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d56961b90 0x563d56961f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:58.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.733+0000 7f49e7a52740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d569624b0 0x563d5696c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:58.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.733+0000 7f49e7a52740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d5696cd70 0x563d5696f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:58.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.733+0000 7f49e7a52740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563d568c83e0 con 0x563d5696cd70 2024-09-19T05:11:58.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.733+0000 7f49e7a52740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563d5686fd80 con 0x563d56961b90 2024-09-19T05:11:58.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.733+0000 7f49e7a52740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563d568c43a0 con 0x563d569624b0 2024-09-19T05:11:58.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.733+0000 7f49dde00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d569624b0 0x563d5696c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:58.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.733+0000 7f49dde00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d569624b0 0x563d5696c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:59802/0 (socket says 172.21.15.43:59802) 2024-09-19T05:11:58.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.733+0000 7f49dde00640 1 -- 172.21.15.43:0/2096936453 learned_addr learned my addr 172.21.15.43:0/2096936453 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:58.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.733+0000 7f49dd400640 1 --2- 172.21.15.43:0/2096936453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d5696cd70 0x563d5696f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:58.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.733+0000 7f49dd400640 1 --2- 172.21.15.43:0/2096936453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d56961b90 0x563d56961f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:58.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.734+0000 7f49dde00640 1 -- 172.21.15.43:0/2096936453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d56961b90 msgr2=0x563d56961f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:58.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.734+0000 7f49dde00640 1 --2- 172.21.15.43:0/2096936453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d56961b90 0x563d56961f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:58.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.734+0000 7f49dde00640 1 -- 172.21.15.43:0/2096936453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d5696cd70 msgr2=0x563d5696f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:58.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.734+0000 7f49dde00640 1 --2- 172.21.15.43:0/2096936453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d5696cd70 0x563d5696f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:58.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.734+0000 7f49dde00640 1 -- 172.21.15.43:0/2096936453 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d5688eae0 con 0x563d569624b0 2024-09-19T05:11:58.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.734+0000 7f49dde00640 1 --2- 172.21.15.43:0/2096936453 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d569624b0 0x563d5696c830 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009dc0 tx=0x7f49d0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a0f34eed07e03a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:58.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.734+0000 7f49d6a00640 1 --2- 172.21.15.43:0/1030983598 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f49c40a33b0 0x7f49c40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:58.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.734+0000 7f49d6a00640 1 -- 172.21.15.43:0/1030983598 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f49d80d0320 con 0x7f49e0109120 2024-09-19T05:11:58.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.734+0000 7f49cb400640 1 -- 172.21.15.43:0/2096936453 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d0005220 con 0x563d569624b0 2024-09-19T05:11:58.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.734+0000 7f49cb400640 1 -- 172.21.15.43:0/2096936453 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49d0004c80 con 0x563d569624b0 2024-09-19T05:11:58.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.734+0000 7f49cb400640 1 -- 172.21.15.43:0/2096936453 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d0019930 con 0x563d569624b0 2024-09-19T05:11:58.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.735+0000 7f49dd400640 1 --2- 172.21.15.43:0/1030983598 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f49c40a33b0 0x7f49c40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:58.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.735+0000 7f49e7a52740 1 -- 172.21.15.43:0/2096936453 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d569624b0 msgr2=0x563d5696c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:58.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.735+0000 7f49e7a52740 1 --2- 172.21.15.43:0/2096936453 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d569624b0 0x563d5696c830 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009dc0 tx=0x7f49d0011810 comp rx=0 tx=0).stop 2024-09-19T05:11:58.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.735+0000 7f49dd400640 1 --2- 172.21.15.43:0/1030983598 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f49c40a33b0 0x7f49c40a5870 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f49cc009230 tx=0x7f49cc000bc0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:58.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.735+0000 7f49e7a52740 1 -- 172.21.15.43:0/2096936453 shutdown_connections 2024-09-19T05:11:58.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.735+0000 7f49e7a52740 1 --2- 172.21.15.43:0/2096936453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d5696cd70 0x563d5696f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:58.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.735+0000 7f49e7a52740 1 --2- 172.21.15.43:0/2096936453 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d569624b0 0x563d5696c830 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:58.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.735+0000 7f49e7a52740 1 --2- 172.21.15.43:0/2096936453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d56961b90 0x563d56961f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:58.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.735+0000 7f49e7a52740 1 -- 172.21.15.43:0/2096936453 >> 172.21.15.43:0/2096936453 conn(0x563d5695e120 msgr2=0x563d5695e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:58.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.736+0000 7f49e7a52740 1 -- 172.21.15.43:0/2096936453 shutdown_connections 2024-09-19T05:11:58.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.736+0000 7f49e7a52740 1 -- 172.21.15.43:0/2096936453 wait complete. 2024-09-19T05:11:58.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.736+0000 7f49e7a52740 1 -- start start 2024-09-19T05:11:58.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.736+0000 7f49e7a52740 1 Processor -- start 2024-09-19T05:11:58.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.737+0000 7f49e7a52740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d56961b90 0x563d56969b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:58.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.737+0000 7f49e7a52740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d569624b0 0x563d5696a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:58.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.737+0000 7f49e7a52740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d5696cd70 0x563d5696a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:58.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.737+0000 7f49e7a52740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563d5696f9e0 con 0x563d5696cd70 2024-09-19T05:11:58.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.737+0000 7f49e7a52740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563d568830f0 con 0x563d56961b90 2024-09-19T05:11:58.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.737+0000 7f49e7a52740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563d5696fb60 con 0x563d569624b0 2024-09-19T05:11:58.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.737+0000 7f49e7a52740 10 client.0 _ll_register_callbacks cb 0x563d568749e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:58.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.737+0000 7f49dd400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d569624b0 0x563d5696a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:58.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.737+0000 7f49dde00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d56961b90 0x563d56969b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:58.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.737+0000 7f49dde00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d56961b90 0x563d56969b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:41250/0 (socket says 172.21.15.43:41250) 2024-09-19T05:11:58.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.737+0000 7f49dde00640 1 -- 172.21.15.43:0/1364772962 learned_addr learned my addr 172.21.15.43:0/1364772962 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:58.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.737+0000 7f49dd400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d569624b0 0x563d5696a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:59808/0 (socket says 172.21.15.43:59808) 2024-09-19T05:11:58.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.737+0000 7f49de800640 1 --2- 172.21.15.43:0/1364772962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d5696cd70 0x563d5696a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:58.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.738+0000 7f49dde00640 1 -- 172.21.15.43:0/1364772962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d569624b0 msgr2=0x563d5696a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:58.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.738+0000 7f49dde00640 1 --2- 172.21.15.43:0/1364772962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d569624b0 0x563d5696a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:58.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.738+0000 7f49dde00640 1 -- 172.21.15.43:0/1364772962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d5696cd70 msgr2=0x563d5696a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:58.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.738+0000 7f49dde00640 1 --2- 172.21.15.43:0/1364772962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d5696cd70 0x563d5696a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:58.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.738+0000 7f49dde00640 1 -- 172.21.15.43:0/1364772962 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d566393c0 con 0x563d56961b90 2024-09-19T05:11:58.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.738+0000 7f49dd400640 1 --2- 172.21.15.43:0/1364772962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d569624b0 0x563d5696a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:11:58.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.738+0000 7f49dde00640 1 --2- 172.21.15.43:0/1364772962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d56961b90 0x563d56969b60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f49d0019510 tx=0x7f49d00055b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e07571e37b3dccfd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:58.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.738+0000 7f49bfe00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:11:58.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.738+0000 7f49bfe00640 1 -- 172.21.15.43:0/1364772962 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d0027070 con 0x563d56961b90 2024-09-19T05:11:58.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.738+0000 7f49bfe00640 1 -- 172.21.15.43:0/1364772962 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f49d0004450 con 0x563d56961b90 2024-09-19T05:11:58.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.738+0000 7f49e7a52740 1 -- 172.21.15.43:0/1364772962 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49d0004c80 con 0x563d56961b90 2024-09-19T05:11:58.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.739+0000 7f49bfe00640 1 -- 172.21.15.43:0/1364772962 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f49d0033050 con 0x563d56961b90 2024-09-19T05:11:58.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.739+0000 7f49e7a52740 10 client.26834 fetch_fsmap learned FSMap version 29 2024-09-19T05:11:58.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.739+0000 7f49e7a52740 1 -- 172.21.15.43:0/1364772962 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563d5697b590 con 0x563d56961b90 2024-09-19T05:11:58.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.739+0000 7f49bfe00640 1 -- 172.21.15.43:0/1364772962 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49d0004c80 con 0x563d56961b90 2024-09-19T05:11:58.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.739+0000 7f49bfe00640 1 -- 172.21.15.43:0/1364772962 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f49d00035f0 con 0x563d56961b90 2024-09-19T05:11:58.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.739+0000 7f49e7a52740 10 client.26834 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:11:58.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.739+0000 7f49e7a52740 10 client.26834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:11:58.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.739+0000 7f49e7a52740 20 client.26834 populate_metadata read hostname 'smithi043' 2024-09-19T05:11:58.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.739+0000 7f49e7a52740 1 --2- 172.21.15.43:0/1364772962 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x563d5697c940 0x563d5697ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:58.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.739+0000 7f49e7a52740 4 client.26834 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:11:58.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.739+0000 7f49e7a52740 1 -- 172.21.15.43:0/1364772962 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d568d5950 con 0x563d5697c940 2024-09-19T05:11:58.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.740+0000 7f49dd400640 1 --2- 172.21.15.43:0/1364772962 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x563d5697c940 0x563d5697ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:58.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.740+0000 7f49bfe00640 10 client.26834 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:11:58.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.740+0000 7f49dd400640 1 --2- 172.21.15.43:0/1364772962 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x563d5697c940 0x563d5697ed30 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ed0fd60fa12945c server_cookie=9753129c408bdd81 in_seq=0 out_seq=0 2024-09-19T05:11:58.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.741+0000 7f49bfe00640 1 -- 172.21.15.43:0/1364772962 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563d568d5950 con 0x563d5697c940 2024-09-19T05:11:58.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.741+0000 7f49bfe00640 10 client.26834 handle_command_reply: tid=0 2024-09-19T05:11:58.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.741+0000 7f49e7a52740 1 client.26834 shutdown 2024-09-19T05:11:58.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.742+0000 7f49e7a52740 1 -- 172.21.15.43:0/1364772962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d56961b90 msgr2=0x563d56969b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:58.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.742+0000 7f49e7a52740 1 --2- 172.21.15.43:0/1364772962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d56961b90 0x563d56969b60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f49d0019510 tx=0x7f49d00055b0 comp rx=0 tx=0).stop 2024-09-19T05:11:58.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.742+0000 7f49e7a52740 1 -- 172.21.15.43:0/1364772962 shutdown_connections 2024-09-19T05:11:58.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.742+0000 7f49e7a52740 1 --2- 172.21.15.43:0/1364772962 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x563d5697c940 0x563d5697ed30 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:58.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.742+0000 7f49e7a52740 1 --2- 172.21.15.43:0/1364772962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d5696cd70 0x563d5696a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:58.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.742+0000 7f49e7a52740 1 --2- 172.21.15.43:0/1364772962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d569624b0 0x563d5696a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:58.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.742+0000 7f49e7a52740 1 --2- 172.21.15.43:0/1364772962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d56961b90 0x563d56969b60 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:58.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.742+0000 7f49bfe00640 0 client.26834 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:11:58.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.742+0000 7f49e7a52740 1 -- 172.21.15.43:0/1364772962 >> 172.21.15.43:0/1364772962 conn(0x563d5695e120 msgr2=0x563d56963e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:58.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.742+0000 7f49e7a52740 1 -- 172.21.15.43:0/1364772962 shutdown_connections 2024-09-19T05:11:58.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.742+0000 7f49e7a52740 1 -- 172.21.15.43:0/1364772962 wait complete. 2024-09-19T05:11:58.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.742+0000 7f49e7a52740 20 client.26834 trim_cache size 0 max 16384 2024-09-19T05:11:58.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.760+0000 7f49e7a52740 1 Processor -- start 2024-09-19T05:11:58.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.760+0000 7f49e7a52740 1 -- start start 2024-09-19T05:11:58.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.761+0000 7f49e7a52740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d56961b90 0x563d5696b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:58.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.761+0000 7f49e7a52740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d569624b0 0x563d56969310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:58.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.761+0000 7f49e7a52740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d5696cd70 0x563d56984590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:58.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.761+0000 7f49e7a52740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f49e01afb60 con 0x563d5696cd70 2024-09-19T05:11:58.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.761+0000 7f49e7a52740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f49e01b16a0 con 0x563d569624b0 2024-09-19T05:11:58.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.761+0000 7f49e7a52740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f49e01b0230 con 0x563d56961b90 2024-09-19T05:11:58.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.761+0000 7f49dd400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d569624b0 0x563d56969310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:58.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.761+0000 7f49dde00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d56961b90 0x563d5696b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:58.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.761+0000 7f49de800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d5696cd70 0x563d56984590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:58.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.761+0000 7f49dd400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d569624b0 0x563d56969310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:41252/0 (socket says 172.21.15.43:41252) 2024-09-19T05:11:58.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.761+0000 7f49dde00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d56961b90 0x563d5696b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:59814/0 (socket says 172.21.15.43:59814) 2024-09-19T05:11:58.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.761+0000 7f49dde00640 1 -- 172.21.15.43:0/732291950 learned_addr learned my addr 172.21.15.43:0/732291950 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:58.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.761+0000 7f49dde00640 1 -- 172.21.15.43:0/732291950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d569624b0 msgr2=0x563d56969310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:58.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.761+0000 7f49dde00640 1 --2- 172.21.15.43:0/732291950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d569624b0 0x563d56969310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:58.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.761+0000 7f49dde00640 1 -- 172.21.15.43:0/732291950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d5696cd70 msgr2=0x563d56984590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:58.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.761+0000 7f49dde00640 1 --2- 172.21.15.43:0/732291950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d5696cd70 0x563d56984590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:58.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.761+0000 7f49dde00640 1 -- 172.21.15.43:0/732291950 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d5697f360 con 0x563d56961b90 2024-09-19T05:11:58.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.762+0000 7f49dde00640 1 --2- 172.21.15.43:0/732291950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d56961b90 0x563d5696b3c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f49d0004010 tx=0x7f49d00059b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b759b2d56673bb7a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:58.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.762+0000 7f49caa00640 1 -- 172.21.15.43:0/732291950 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d0030020 con 0x563d56961b90 2024-09-19T05:11:58.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.762+0000 7f49caa00640 1 -- 172.21.15.43:0/732291950 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49d0005620 con 0x563d56961b90 2024-09-19T05:11:58.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.762+0000 7f49caa00640 1 -- 172.21.15.43:0/732291950 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d0027070 con 0x563d56961b90 2024-09-19T05:11:58.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.763+0000 7f49e7a52740 1 -- 172.21.15.43:0/732291950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d56961b90 msgr2=0x563d5696b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:58.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.763+0000 7f49e7a52740 1 --2- 172.21.15.43:0/732291950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d56961b90 0x563d5696b3c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f49d0004010 tx=0x7f49d00059b0 comp rx=0 tx=0).stop 2024-09-19T05:11:58.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.763+0000 7f49e7a52740 1 -- 172.21.15.43:0/732291950 shutdown_connections 2024-09-19T05:11:58.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.763+0000 7f49e7a52740 1 --2- 172.21.15.43:0/732291950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d5696cd70 0x563d56984590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:58.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.763+0000 7f49e7a52740 1 --2- 172.21.15.43:0/732291950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d569624b0 0x563d56969310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:58.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.763+0000 7f49e7a52740 1 --2- 172.21.15.43:0/732291950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d56961b90 0x563d5696b3c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:58.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.763+0000 7f49e7a52740 1 -- 172.21.15.43:0/732291950 >> 172.21.15.43:0/732291950 conn(0x563d5695e120 msgr2=0x563d5695d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:58.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.763+0000 7f49e7a52740 1 -- 172.21.15.43:0/732291950 shutdown_connections 2024-09-19T05:11:58.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.763+0000 7f49e7a52740 1 -- 172.21.15.43:0/732291950 wait complete. 2024-09-19T05:11:58.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.764+0000 7f49e7a52740 1 -- start start 2024-09-19T05:11:58.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.764+0000 7f49e7a52740 1 Processor -- start 2024-09-19T05:11:58.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.764+0000 7f49e7a52740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d56961b90 0x563d569751c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:58.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.764+0000 7f49e7a52740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d569624b0 0x563d569695a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:58.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.764+0000 7f49e7a52740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d5696cd70 0x563d56969ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:58.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.764+0000 7f49e7a52740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f49e01b0fd0 con 0x563d569624b0 2024-09-19T05:11:58.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.764+0000 7f49e7a52740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f49e01b0900 con 0x563d56961b90 2024-09-19T05:11:58.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.764+0000 7f49e7a52740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f49e01adb40 con 0x563d5696cd70 2024-09-19T05:11:58.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.764+0000 7f49e7a52740 10 client.0 _ll_register_callbacks cb 0x7f49d0005310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:11:58.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.765+0000 7f49dde00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d56961b90 0x563d569751c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:58.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.765+0000 7f49dd400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d569624b0 0x563d569695a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:58.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.765+0000 7f49dde00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d56961b90 0x563d569751c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:41268/0 (socket says 172.21.15.43:41268) 2024-09-19T05:11:58.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.765+0000 7f49dde00640 1 -- 172.21.15.43:0/546956073 learned_addr learned my addr 172.21.15.43:0/546956073 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:11:58.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.765+0000 7f49de800640 1 --2- 172.21.15.43:0/546956073 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d5696cd70 0x563d56969ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:58.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.765+0000 7f49dde00640 1 -- 172.21.15.43:0/546956073 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d5696cd70 msgr2=0x563d56969ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:58.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.765+0000 7f49dde00640 1 --2- 172.21.15.43:0/546956073 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d5696cd70 0x563d56969ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:58.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.765+0000 7f49dde00640 1 -- 172.21.15.43:0/546956073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d569624b0 msgr2=0x563d569695a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:11:58.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.765+0000 7f49dde00640 1 --2- 172.21.15.43:0/546956073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d569624b0 0x563d569695a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:58.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.765+0000 7f49dde00640 1 -- 172.21.15.43:0/546956073 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d5695df50 con 0x563d56961b90 2024-09-19T05:11:58.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.765+0000 7f49dde00640 1 --2- 172.21.15.43:0/546956073 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d56961b90 0x563d569751c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f49d0003e30 tx=0x7f49d0005ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=839f619f1b08e784 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:11:58.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.765+0000 7f49bfe00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:11:58.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.766+0000 7f49bfe00640 1 -- 172.21.15.43:0/546956073 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d0027030 con 0x563d56961b90 2024-09-19T05:11:58.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.766+0000 7f49bfe00640 1 -- 172.21.15.43:0/546956073 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f49d0004450 con 0x563d56961b90 2024-09-19T05:11:58.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.766+0000 7f49e7a52740 1 -- 172.21.15.43:0/546956073 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49d0005620 con 0x563d56961b90 2024-09-19T05:11:58.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.766+0000 7f49bfe00640 1 -- 172.21.15.43:0/546956073 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f49d00099d0 con 0x563d56961b90 2024-09-19T05:11:58.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.766+0000 7f49e7a52740 10 client.26840 fetch_fsmap learned FSMap version 29 2024-09-19T05:11:58.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.766+0000 7f49e7a52740 1 -- 172.21.15.43:0/546956073 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563d569738f0 con 0x563d56961b90 2024-09-19T05:11:58.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.766+0000 7f49bfe00640 1 -- 172.21.15.43:0/546956073 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49d0005620 con 0x563d56961b90 2024-09-19T05:11:58.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.767+0000 7f49bfe00640 1 -- 172.21.15.43:0/546956073 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f49d0019aa0 con 0x563d56961b90 2024-09-19T05:11:58.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.767+0000 7f49e7a52740 10 client.26840 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:11:58.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.767+0000 7f49e7a52740 10 client.26840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:11:58.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.767+0000 7f49e7a52740 20 client.26840 populate_metadata read hostname 'smithi043' 2024-09-19T05:11:58.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.767+0000 7f49e7a52740 1 --2- 172.21.15.43:0/546956073 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x563d569e7720 0x563d569e9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:11:58.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.767+0000 7f49e7a52740 4 client.26840 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:11:58.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.767+0000 7f49e7a52740 1 -- 172.21.15.43:0/546956073 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563d568cfe60 con 0x563d569e7720 2024-09-19T05:11:58.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.767+0000 7f49dd400640 1 --2- 172.21.15.43:0/546956073 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x563d569e7720 0x563d569e9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:11:58.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.767+0000 7f49dd400640 1 --2- 172.21.15.43:0/546956073 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x563d569e7720 0x563d569e9b40 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f0aaaed0e8d630f server_cookie=d8a5bcaeef79864d in_seq=0 out_seq=0 2024-09-19T05:11:58.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.767+0000 7f49bfe00640 10 client.26840 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:11:58.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.768+0000 7f49bfe00640 1 -- 172.21.15.43:0/546956073 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563d568cfe60 con 0x563d569e7720 2024-09-19T05:11:58.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.768+0000 7f49bfe00640 10 client.26840 handle_command_reply: tid=0 2024-09-19T05:11:58.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.768+0000 7f49e7a52740 1 client.26840 shutdown 2024-09-19T05:11:58.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.768+0000 7f49e7a52740 1 -- 172.21.15.43:0/546956073 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d56961b90 msgr2=0x563d569751c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:58.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.768+0000 7f49e7a52740 1 --2- 172.21.15.43:0/546956073 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d56961b90 0x563d569751c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f49d0003e30 tx=0x7f49d0005ab0 comp rx=0 tx=0).stop 2024-09-19T05:11:58.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.768+0000 7f49e7a52740 1 -- 172.21.15.43:0/546956073 shutdown_connections 2024-09-19T05:11:58.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.768+0000 7f49e7a52740 1 --2- 172.21.15.43:0/546956073 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x563d569e7720 0x563d569e9b40 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:58.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.768+0000 7f49e7a52740 1 --2- 172.21.15.43:0/546956073 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d5696cd70 0x563d56969ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:58.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.768+0000 7f49e7a52740 1 --2- 172.21.15.43:0/546956073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d569624b0 0x563d569695a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:58.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.768+0000 7f49e7a52740 1 --2- 172.21.15.43:0/546956073 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d56961b90 0x563d569751c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:58.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.768+0000 7f49bfe00640 0 client.26840 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:11:58.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.768+0000 7f49e7a52740 1 -- 172.21.15.43:0/546956073 >> 172.21.15.43:0/546956073 conn(0x563d5695e120 msgr2=0x563d5695c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:58.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.768+0000 7f49e7a52740 1 -- 172.21.15.43:0/546956073 shutdown_connections 2024-09-19T05:11:58.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.768+0000 7f49e7a52740 1 -- 172.21.15.43:0/546956073 wait complete. 2024-09-19T05:11:58.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.768+0000 7f49e7a52740 20 client.26840 trim_cache size 0 max 16384 2024-09-19T05:11:58.773 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:11:58.773 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-09-19T05:11:58.773 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-09-19T05:11:58.773 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:11:58.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.769+0000 7f49e5000640 1 -- 172.21.15.43:0/1030983598 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f49c40a33b0 msgr2=0x7f49c40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:58.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.769+0000 7f49e5000640 1 --2- 172.21.15.43:0/1030983598 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f49c40a33b0 0x7f49c40a5870 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f49cc009230 tx=0x7f49cc000bc0 comp rx=0 tx=0).stop 2024-09-19T05:11:58.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.769+0000 7f49e5000640 1 -- 172.21.15.43:0/1030983598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e0109120 msgr2=0x7f49e019ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:11:58.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.769+0000 7f49e5000640 1 --2- 172.21.15.43:0/1030983598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e0109120 0x7f49e019ef70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f49d8009fd0 tx=0x7f49d800ef10 comp rx=0 tx=0).stop 2024-09-19T05:11:58.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.769+0000 7f49e5000640 1 -- 172.21.15.43:0/1030983598 shutdown_connections 2024-09-19T05:11:58.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.769+0000 7f49e5000640 1 --2- 172.21.15.43:0/1030983598 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f49c40a33b0 0x7f49c40a5870 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:58.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.769+0000 7f49e5000640 1 --2- 172.21.15.43:0/1030983598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e0109120 0x7f49e019ef70 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:58.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.769+0000 7f49e5000640 1 --2- 172.21.15.43:0/1030983598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49e01049a0 0x7f49e019abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:58.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.769+0000 7f49e5000640 1 --2- 172.21.15.43:0/1030983598 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49e0103ff0 0x7f49e019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:11:58.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.769+0000 7f49e5000640 1 -- 172.21.15.43:0/1030983598 >> 172.21.15.43:0/1030983598 conn(0x7f49e00ffa90 msgr2=0x7f49e0100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:11:58.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.769+0000 7f49e5000640 1 -- 172.21.15.43:0/1030983598 shutdown_connections 2024-09-19T05:11:58.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:11:58.769+0000 7f49e5000640 1 -- 172.21.15.43:0/1030983598 wait complete. 2024-09-19T05:11:59.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:11:59 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:11:59.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:11:59 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:11:59.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:11:59 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:11:59.612 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T05:11:59.612 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:11:59.991 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:12:00.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:00 smithi043 ceph-mon[30800]: pgmap v225: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:12:00.507 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:00 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:12:00.508 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:00 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:12:00.508 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:00 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:12:00.509 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:00 smithi096 ceph-mon[35085]: pgmap v225: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:12:00.509 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:00 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:12:00.509 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:00 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:12:00.509 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:00 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:12:00.509 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:00 smithi151 ceph-mon[35425]: pgmap v225: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:12:00.509 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:00 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:12:00.509 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:00 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:12:00.509 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:00 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:12:00.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.550+0000 7efef1c00640 1 -- 172.21.15.43:0/3456785632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efeec102790 msgr2=0x7efeec1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:00.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.551+0000 7efef1c00640 1 --2- 172.21.15.43:0/3456785632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efeec102790 0x7efeec1067e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7efed4009900 tx=0x7efed402a410 comp rx=0 tx=0).stop 2024-09-19T05:12:00.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.551+0000 7efef1c00640 1 -- 172.21.15.43:0/3456785632 shutdown_connections 2024-09-19T05:12:00.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.551+0000 7efef1c00640 1 --2- 172.21.15.43:0/3456785632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efeec106f10 0x7efeec10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:00.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.551+0000 7efef1c00640 1 --2- 172.21.15.43:0/3456785632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efeec102790 0x7efeec1067e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:00.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.551+0000 7efef1c00640 1 --2- 172.21.15.43:0/3456785632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efeec101de0 0x7efeec1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:00.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.551+0000 7efef1c00640 1 -- 172.21.15.43:0/3456785632 >> 172.21.15.43:0/3456785632 conn(0x7efeec0fd8c0 msgr2=0x7efeec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:00.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.551+0000 7efef1c00640 1 -- 172.21.15.43:0/3456785632 shutdown_connections 2024-09-19T05:12:00.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.551+0000 7efef1c00640 1 -- 172.21.15.43:0/3456785632 wait complete. 2024-09-19T05:12:00.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.552+0000 7efef1c00640 1 Processor -- start 2024-09-19T05:12:00.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.552+0000 7efef1c00640 1 -- start start 2024-09-19T05:12:00.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.552+0000 7efef1c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efeec101de0 0x7efeec19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:00.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.552+0000 7efef1c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efeec102790 0x7efeec19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:00.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.553+0000 7efef1c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efeec106f10 0x7efeec19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:00.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.553+0000 7efef1c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efeec111f80 con 0x7efeec102790 2024-09-19T05:12:00.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.553+0000 7efef1c00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efeec111e00 con 0x7efeec106f10 2024-09-19T05:12:00.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.553+0000 7efef1c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efeec112100 con 0x7efeec101de0 2024-09-19T05:12:00.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.553+0000 7efeeaa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efeec101de0 0x7efeec19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:00.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.553+0000 7efeeaa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efeec101de0 0x7efeec19a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:59838/0 (socket says 172.21.15.43:59838) 2024-09-19T05:12:00.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.553+0000 7efeeaa00640 1 -- 172.21.15.43:0/3683602156 learned_addr learned my addr 172.21.15.43:0/3683602156 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:00.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.553+0000 7efee2000640 1 --2- 172.21.15.43:0/3683602156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efeec102790 0x7efeec19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:00.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.553+0000 7efeeb400640 1 --2- 172.21.15.43:0/3683602156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efeec106f10 0x7efeec19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:00.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.554+0000 7efeeaa00640 1 -- 172.21.15.43:0/3683602156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efeec106f10 msgr2=0x7efeec19ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:00.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.554+0000 7efeeaa00640 1 --2- 172.21.15.43:0/3683602156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efeec106f10 0x7efeec19ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:00.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.554+0000 7efeeaa00640 1 -- 172.21.15.43:0/3683602156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efeec102790 msgr2=0x7efeec19aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:00.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.554+0000 7efeeaa00640 1 --2- 172.21.15.43:0/3683602156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efeec102790 0x7efeec19aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:00.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.554+0000 7efeeaa00640 1 -- 172.21.15.43:0/3683602156 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeec19f610 con 0x7efeec101de0 2024-09-19T05:12:00.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.554+0000 7efeeaa00640 1 --2- 172.21.15.43:0/3683602156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efeec101de0 0x7efeec19a660 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7efedc00e9f0 tx=0x7efedc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:00.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.554+0000 7efee3e00640 1 -- 172.21.15.43:0/3683602156 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efedc00cd80 con 0x7efeec101de0 2024-09-19T05:12:00.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.554+0000 7efee3e00640 1 -- 172.21.15.43:0/3683602156 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efedc010910 con 0x7efeec101de0 2024-09-19T05:12:00.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.554+0000 7efef1c00640 1 -- 172.21.15.43:0/3683602156 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeec19f900 con 0x7efeec101de0 2024-09-19T05:12:00.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.555+0000 7efef1c00640 1 -- 172.21.15.43:0/3683602156 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeec1a71e0 con 0x7efeec101de0 2024-09-19T05:12:00.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.555+0000 7efee3e00640 1 -- 172.21.15.43:0/3683602156 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efedc018990 con 0x7efeec101de0 2024-09-19T05:12:00.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.556+0000 7efef1c00640 1 -- 172.21.15.43:0/3683602156 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efec8005290 con 0x7efeec101de0 2024-09-19T05:12:00.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.558+0000 7efee3e00640 1 -- 172.21.15.43:0/3683602156 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7efedc018b30 con 0x7efeec101de0 2024-09-19T05:12:00.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.560+0000 7efee3e00640 1 --2- 172.21.15.43:0/3683602156 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7efed00a3430 0x7efed00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:00.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.560+0000 7efee3e00640 1 -- 172.21.15.43:0/3683602156 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7efedc0cff50 con 0x7efeec101de0 2024-09-19T05:12:00.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.560+0000 7efee2000640 1 --2- 172.21.15.43:0/3683602156 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7efed00a3430 0x7efed00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:00.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.561+0000 7efee2000640 1 --2- 172.21.15.43:0/3683602156 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7efed00a3430 0x7efed00a58f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efed4000c00 tx=0x7efed4005b50 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:00.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.564+0000 7efee3e00640 1 -- 172.21.15.43:0/3683602156 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7efedc09c270 con 0x7efeec101de0 2024-09-19T05:12:00.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.806+0000 7efef1c00640 1 -- 172.21.15.43:0/3683602156 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efec8005850 con 0x7efeec101de0 2024-09-19T05:12:00.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.808+0000 7efee3e00640 1 -- 172.21.15.43:0/3683602156 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11965 (secure 0 0 0) 0x7efedc0a0570 con 0x7efeec101de0 2024-09-19T05:12:00.809 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:12:00.810 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":29,"btime":"2024-09-19T05:04:38:785407+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/773801169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":773801169},{"type":"v1","addr":"172.21.15.43:6835","nonce":773801169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1799137902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1799137902},{"type":"v1","addr":"172.21.15.43:6837","nonce":1799137902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:02:25.463041+0000","modified":"2024-09-19T05:04:38.039809+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24518,"mds_2":24511,"mds_3":24506,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14541":{"gid":14541,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/794881135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":794881135},{"type":"v1","addr":"172.21.15.43:6841","nonce":794881135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/2836453158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2836453158},{"type":"v1","addr":"172.21.15.151:6833","nonce":2836453158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/178785812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":178785812},{"type":"v1","addr":"172.21.15.96:6833","nonce":178785812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1671043820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1671043820},{"type":"v1","addr":"172.21.15.96:6835","nonce":1671043820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2972514328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2972514328},{"type":"v1","addr":"172.21.15.151:6835","nonce":2972514328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.151:6837/3960378010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3960378010},{"type":"v1","addr":"172.21.15.151:6837","nonce":3960378010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6837/3978388286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3978388286},{"type":"v1","addr":"172.21.15.96:6837","nonce":3978388286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.151:6839/3794104313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3794104313},{"type":"v1","addr":"172.21.15.151:6839","nonce":3794104313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.43:6839/3404555030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3404555030},{"type":"v1","addr":"172.21.15.43:6839","nonce":3404555030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/605527621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":605527621},{"type":"v1","addr":"172.21.15.96:6839","nonce":605527621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24518,24511,24506,24493]},"id":1}]} 2024-09-19T05:12:00.810 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 29 2024-09-19T05:12:00.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.810+0000 7efef1c00640 1 -- 172.21.15.43:0/3683602156 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7efed00a3430 msgr2=0x7efed00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:00.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.810+0000 7efef1c00640 1 --2- 172.21.15.43:0/3683602156 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7efed00a3430 0x7efed00a58f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efed4000c00 tx=0x7efed4005b50 comp rx=0 tx=0).stop 2024-09-19T05:12:00.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.810+0000 7efef1c00640 1 -- 172.21.15.43:0/3683602156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efeec101de0 msgr2=0x7efeec19a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:00.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.810+0000 7efef1c00640 1 --2- 172.21.15.43:0/3683602156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efeec101de0 0x7efeec19a660 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7efedc00e9f0 tx=0x7efedc00eeb0 comp rx=0 tx=0).stop 2024-09-19T05:12:00.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.810+0000 7efef1c00640 1 -- 172.21.15.43:0/3683602156 shutdown_connections 2024-09-19T05:12:00.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.810+0000 7efef1c00640 1 --2- 172.21.15.43:0/3683602156 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7efed00a3430 0x7efed00a58f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:00.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.810+0000 7efef1c00640 1 --2- 172.21.15.43:0/3683602156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efeec106f10 0x7efeec19ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:00.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.810+0000 7efef1c00640 1 --2- 172.21.15.43:0/3683602156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efeec102790 0x7efeec19aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:00.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.810+0000 7efef1c00640 1 --2- 172.21.15.43:0/3683602156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efeec101de0 0x7efeec19a660 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:00.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.810+0000 7efef1c00640 1 -- 172.21.15.43:0/3683602156 >> 172.21.15.43:0/3683602156 conn(0x7efeec0fd8c0 msgr2=0x7efeec0ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:00.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.811+0000 7efef1c00640 1 -- 172.21.15.43:0/3683602156 shutdown_connections 2024-09-19T05:12:00.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:00.811+0000 7efef1c00640 1 -- 172.21.15.43:0/3683602156 wait complete. 2024-09-19T05:12:01.158 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:01 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/3683602156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:12:01.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:01 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/3683602156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:12:01.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:01 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/3683602156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:12:01.589 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T05:12:01.979 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:12:02.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:02 smithi043 ceph-mon[30800]: pgmap v226: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:12:02.415 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:02 smithi151 ceph-mon[35425]: pgmap v226: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:12:02.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:02 smithi096 ceph-mon[35085]: pgmap v226: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:12:02.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.576+0000 7f9b32600640 1 -- 172.21.15.43:0/1620700996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b2c109120 msgr2=0x7f9b2c110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:02.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.576+0000 7f9b32600640 1 --2- 172.21.15.43:0/1620700996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b2c109120 0x7f9b2c110ca0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f9b24009ca0 tx=0x7f9b2402a8c0 comp rx=0 tx=0).stop 2024-09-19T05:12:02.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.576+0000 7f9b32600640 1 -- 172.21.15.43:0/1620700996 shutdown_connections 2024-09-19T05:12:02.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.576+0000 7f9b32600640 1 --2- 172.21.15.43:0/1620700996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b2c109120 0x7f9b2c110ca0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:02.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.576+0000 7f9b32600640 1 --2- 172.21.15.43:0/1620700996 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b2c1049a0 0x7f9b2c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:02.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.576+0000 7f9b32600640 1 --2- 172.21.15.43:0/1620700996 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b2c103ff0 0x7f9b2c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:02.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.576+0000 7f9b32600640 1 -- 172.21.15.43:0/1620700996 >> 172.21.15.43:0/1620700996 conn(0x7f9b2c0ffa90 msgr2=0x7f9b2c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:02.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.576+0000 7f9b32600640 1 -- 172.21.15.43:0/1620700996 shutdown_connections 2024-09-19T05:12:02.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.577+0000 7f9b32600640 1 -- 172.21.15.43:0/1620700996 wait complete. 2024-09-19T05:12:02.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.577+0000 7f9b32600640 1 Processor -- start 2024-09-19T05:12:02.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.578+0000 7f9b32600640 1 -- start start 2024-09-19T05:12:02.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.578+0000 7f9b32600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b2c103ff0 0x7f9b2c19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:02.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.578+0000 7f9b32600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b2c1049a0 0x7f9b2c19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:02.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.578+0000 7f9b32600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b2c109120 0x7f9b2c19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:02.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.578+0000 7f9b32600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c114180 con 0x7f9b2c1049a0 2024-09-19T05:12:02.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.578+0000 7f9b32600640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c114000 con 0x7f9b2c109120 2024-09-19T05:12:02.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.578+0000 7f9b32600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c114300 con 0x7f9b2c103ff0 2024-09-19T05:12:02.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.578+0000 7f9b2aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b2c1049a0 0x7f9b2c19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:02.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.578+0000 7f9b2b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b2c103ff0 0x7f9b2c19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:02.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.578+0000 7f9b2b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b2c103ff0 0x7f9b2c19a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:59856/0 (socket says 172.21.15.43:59856) 2024-09-19T05:12:02.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.578+0000 7f9b2b400640 1 -- 172.21.15.43:0/2142276631 learned_addr learned my addr 172.21.15.43:0/2142276631 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:02.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.579+0000 7f9b2be00640 1 --2- 172.21.15.43:0/2142276631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b2c109120 0x7f9b2c19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:02.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.579+0000 7f9b2b400640 1 -- 172.21.15.43:0/2142276631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b2c109120 msgr2=0x7f9b2c19eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:02.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.579+0000 7f9b2b400640 1 --2- 172.21.15.43:0/2142276631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b2c109120 0x7f9b2c19eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:02.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.579+0000 7f9b2b400640 1 -- 172.21.15.43:0/2142276631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b2c1049a0 msgr2=0x7f9b2c19ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:02.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.579+0000 7f9b2b400640 1 --2- 172.21.15.43:0/2142276631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b2c1049a0 0x7f9b2c19ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:02.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.579+0000 7f9b2b400640 1 -- 172.21.15.43:0/2142276631 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b2c19f5d0 con 0x7f9b2c103ff0 2024-09-19T05:12:02.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.579+0000 7f9b2b400640 1 --2- 172.21.15.43:0/2142276631 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b2c103ff0 0x7f9b2c19a620 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c00e9f0 tx=0x7f9b1c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:02.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.579+0000 7f9b23e00640 1 -- 172.21.15.43:0/2142276631 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c00cd80 con 0x7f9b2c103ff0 2024-09-19T05:12:02.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.579+0000 7f9b32600640 1 -- 172.21.15.43:0/2142276631 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b2c19f8c0 con 0x7f9b2c103ff0 2024-09-19T05:12:02.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.579+0000 7f9b23e00640 1 -- 172.21.15.43:0/2142276631 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b1c010910 con 0x7f9b2c103ff0 2024-09-19T05:12:02.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.579+0000 7f9b23e00640 1 -- 172.21.15.43:0/2142276631 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c0188f0 con 0x7f9b2c103ff0 2024-09-19T05:12:02.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.579+0000 7f9b32600640 1 -- 172.21.15.43:0/2142276631 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b2c1a71a0 con 0x7f9b2c103ff0 2024-09-19T05:12:02.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.582+0000 7f9b3524a740 1 Processor -- start 2024-09-19T05:12:02.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.583+0000 7f9b23e00640 1 -- 172.21.15.43:0/2142276631 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f9b1c010ac0 con 0x7f9b2c103ff0 2024-09-19T05:12:02.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.583+0000 7f9b3524a740 1 -- start start 2024-09-19T05:12:02.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.583+0000 7f9b3524a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaf9ebec70 0x55aaf9ebf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:02.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.583+0000 7f9b3524a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaf9ebf590 0x55aaf9ec9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:02.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.583+0000 7f9b3524a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaf9ec9e50 0x55aaf9ecc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:02.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.583+0000 7f9b3524a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55aaf9e253e0 con 0x55aaf9ebf590 2024-09-19T05:12:02.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.583+0000 7f9b3524a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55aaf9dccd80 con 0x55aaf9ec9e50 2024-09-19T05:12:02.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.583+0000 7f9b3524a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55aaf9e213a0 con 0x55aaf9ebec70 2024-09-19T05:12:02.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.583+0000 7f9b2b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaf9ebf590 0x55aaf9ec9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:02.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.583+0000 7f9b2be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaf9ebec70 0x55aaf9ebf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:02.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.583+0000 7f9b2aa00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaf9ec9e50 0x55aaf9ecc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:02.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.583+0000 7f9b2be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaf9ebec70 0x55aaf9ebf050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:59870/0 (socket says 172.21.15.43:59870) 2024-09-19T05:12:02.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.583+0000 7f9b2aa00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaf9ec9e50 0x55aaf9ecc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:41316/0 (socket says 172.21.15.43:41316) 2024-09-19T05:12:02.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.583+0000 7f9b2be00640 1 -- 172.21.15.43:0/2644098921 learned_addr learned my addr 172.21.15.43:0/2644098921 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:02.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.583+0000 7f9b2b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaf9ebf590 0x55aaf9ec9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35214/0 (socket says 172.21.15.43:35214) 2024-09-19T05:12:02.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.584+0000 7f9b2be00640 1 -- 172.21.15.43:0/2644098921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaf9ec9e50 msgr2=0x55aaf9ecc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:02.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.584+0000 7f9b2be00640 1 --2- 172.21.15.43:0/2644098921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaf9ec9e50 0x55aaf9ecc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:02.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.584+0000 7f9b2be00640 1 -- 172.21.15.43:0/2644098921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaf9ebf590 msgr2=0x55aaf9ec9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:02.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.584+0000 7f9b2be00640 1 --2- 172.21.15.43:0/2644098921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaf9ebf590 0x55aaf9ec9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:02.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.584+0000 7f9b2be00640 1 -- 172.21.15.43:0/2644098921 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaf9debae0 con 0x55aaf9ebec70 2024-09-19T05:12:02.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.584+0000 7f9b2be00640 1 --2- 172.21.15.43:0/2644098921 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaf9ebec70 0x55aaf9ebf050 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9b24009870 tx=0x7f9b240098a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f690a844cbefc1ff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:02.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.584+0000 7f9b23e00640 1 --2- 172.21.15.43:0/2142276631 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f9b100a3430 0x7f9b100a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:02.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.584+0000 7f9b23e00640 1 -- 172.21.15.43:0/2142276631 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f9b1c0d0180 con 0x7f9b2c103ff0 2024-09-19T05:12:02.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.585+0000 7f9b20c00640 1 -- 172.21.15.43:0/2644098921 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b24031db0 con 0x55aaf9ebec70 2024-09-19T05:12:02.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.585+0000 7f9b2b400640 1 --2- 172.21.15.43:0/2142276631 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f9b100a3430 0x7f9b100a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:02.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.585+0000 7f9b20c00640 1 -- 172.21.15.43:0/2644098921 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b24036040 con 0x55aaf9ebec70 2024-09-19T05:12:02.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.585+0000 7f9b20c00640 1 -- 172.21.15.43:0/2644098921 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b24004380 con 0x55aaf9ebec70 2024-09-19T05:12:02.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.585+0000 7f9b3524a740 1 -- 172.21.15.43:0/2644098921 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaf9ebec70 msgr2=0x55aaf9ebf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:02.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.585+0000 7f9b2b400640 1 --2- 172.21.15.43:0/2142276631 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f9b100a3430 0x7f9b100a58f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x55aaf9ebc060 tx=0x7f9b1c0d3c30 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:02.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.585+0000 7f9b3524a740 1 --2- 172.21.15.43:0/2644098921 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaf9ebec70 0x55aaf9ebf050 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9b24009870 tx=0x7f9b240098a0 comp rx=0 tx=0).stop 2024-09-19T05:12:02.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.585+0000 7f9b3524a740 1 -- 172.21.15.43:0/2644098921 shutdown_connections 2024-09-19T05:12:02.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.585+0000 7f9b3524a740 1 --2- 172.21.15.43:0/2644098921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaf9ec9e50 0x55aaf9ecc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:02.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.585+0000 7f9b3524a740 1 --2- 172.21.15.43:0/2644098921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaf9ebf590 0x55aaf9ec9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:02.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.585+0000 7f9b3524a740 1 --2- 172.21.15.43:0/2644098921 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaf9ebec70 0x55aaf9ebf050 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:02.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.585+0000 7f9b3524a740 1 -- 172.21.15.43:0/2644098921 >> 172.21.15.43:0/2644098921 conn(0x55aaf9ebb200 msgr2=0x55aaf9ebb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:02.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.586+0000 7f9b3524a740 1 -- 172.21.15.43:0/2644098921 shutdown_connections 2024-09-19T05:12:02.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.586+0000 7f9b3524a740 1 -- 172.21.15.43:0/2644098921 wait complete. 2024-09-19T05:12:02.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.586+0000 7f9b3524a740 1 -- start start 2024-09-19T05:12:02.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.586+0000 7f9b3524a740 1 Processor -- start 2024-09-19T05:12:02.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.587+0000 7f9b3524a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaf9ebec70 0x55aaf9ec6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:02.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.587+0000 7f9b3524a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaf9ebf590 0x55aaf9ec7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:02.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.587+0000 7f9b3524a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaf9ec9e50 0x55aaf9ec77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:02.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.587+0000 7f9b3524a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55aaf9ecca20 con 0x55aaf9ebec70 2024-09-19T05:12:02.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.587+0000 7f9b3524a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55aaf9de00f0 con 0x55aaf9ec9e50 2024-09-19T05:12:02.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.587+0000 7f9b3524a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55aaf9eccba0 con 0x55aaf9ebf590 2024-09-19T05:12:02.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.587+0000 7f9b3524a740 10 client.0 _ll_register_callbacks cb 0x55aaf9dc7bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:12:02.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.587+0000 7f9b2be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaf9ebec70 0x55aaf9ec6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:02.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.587+0000 7f9b2aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaf9ebf590 0x55aaf9ec7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:02.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.587+0000 7f9b2be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaf9ebec70 0x55aaf9ec6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35218/0 (socket says 172.21.15.43:35218) 2024-09-19T05:12:02.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.587+0000 7f9b2be00640 1 -- 172.21.15.43:0/1206223294 learned_addr learned my addr 172.21.15.43:0/1206223294 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:02.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.587+0000 7f9b2be00640 1 --2- 172.21.15.43:0/1206223294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaf9ec9e50 0x55aaf9ec77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:02.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.588+0000 7f9b2aa00640 1 -- 172.21.15.43:0/1206223294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaf9ec9e50 msgr2=0x55aaf9ec77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:02.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.588+0000 7f9b2aa00640 1 --2- 172.21.15.43:0/1206223294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaf9ec9e50 0x55aaf9ec77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:02.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.588+0000 7f9b2aa00640 1 -- 172.21.15.43:0/1206223294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaf9ebec70 msgr2=0x55aaf9ec6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:02.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.588+0000 7f9b2aa00640 1 --2- 172.21.15.43:0/1206223294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaf9ebec70 0x55aaf9ec6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:02.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.588+0000 7f9b2aa00640 1 -- 172.21.15.43:0/1206223294 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aaf9b963c0 con 0x55aaf9ebf590 2024-09-19T05:12:02.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.588+0000 7f9b2be00640 1 --2- 172.21.15.43:0/1206223294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaf9ebec70 0x55aaf9ec6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:12:02.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.588+0000 7f9b2aa00640 1 --2- 172.21.15.43:0/1206223294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaf9ebf590 0x55aaf9ec7280 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9b180126d0 tx=0x7f9b18012b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e252bcd6428fd225 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:02.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.588+0000 7f9b15600640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:12:02.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.588+0000 7f9b15600640 1 -- 172.21.15.43:0/1206223294 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b18004030 con 0x55aaf9ebf590 2024-09-19T05:12:02.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.588+0000 7f9b3524a740 1 -- 172.21.15.43:0/1206223294 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b24036040 con 0x55aaf9ebf590 2024-09-19T05:12:02.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.588+0000 7f9b15600640 1 -- 172.21.15.43:0/1206223294 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f9b180041d0 con 0x55aaf9ebf590 2024-09-19T05:12:02.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.589+0000 7f9b15600640 1 -- 172.21.15.43:0/1206223294 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9b180043d0 con 0x55aaf9ebf590 2024-09-19T05:12:02.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.589+0000 7f9b3524a740 10 client.26791 fetch_fsmap learned FSMap version 29 2024-09-19T05:12:02.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.589+0000 7f9b3524a740 1 -- 172.21.15.43:0/1206223294 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55aaf9ed8b10 con 0x55aaf9ebf590 2024-09-19T05:12:02.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.589+0000 7f9b15600640 1 -- 172.21.15.43:0/1206223294 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b24036040 con 0x55aaf9ebf590 2024-09-19T05:12:02.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.589+0000 7f9b15600640 1 -- 172.21.15.43:0/1206223294 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9b18003ad0 con 0x55aaf9ebf590 2024-09-19T05:12:02.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.589+0000 7f9b3524a740 10 client.26791 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:12:02.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.589+0000 7f9b3524a740 10 client.26791 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-19T05:12:02.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.589+0000 7f9b3524a740 20 client.26791 populate_metadata read hostname 'smithi043' 2024-09-19T05:12:02.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.589+0000 7f9b3524a740 1 --2- 172.21.15.43:0/1206223294 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55aaf9ed9f20 0x55aaf9edc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:02.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.590+0000 7f9b3524a740 4 client.26791 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:12:02.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.590+0000 7f9b3524a740 1 -- 172.21.15.43:0/1206223294 --> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aaf9e32950 con 0x55aaf9ed9f20 2024-09-19T05:12:02.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.590+0000 7f9b2be00640 1 --2- 172.21.15.43:0/1206223294 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55aaf9ed9f20 0x55aaf9edc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:02.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.590+0000 7f9b2be00640 1 --2- 172.21.15.43:0/1206223294 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55aaf9ed9f20 0x55aaf9edc330 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9e0ec4373eaf8c37 server_cookie=2893445e968ecbbc in_seq=0 out_seq=0 2024-09-19T05:12:02.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.590+0000 7f9b15600640 10 client.26791 ms_handle_connect on v2:172.21.15.151:6836/3960378010 2024-09-19T05:12:02.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.591+0000 7f9b15600640 1 -- 172.21.15.43:0/1206223294 <== mds.4 v2:172.21.15.151:6836/3960378010 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aaf9e32950 con 0x55aaf9ed9f20 2024-09-19T05:12:02.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.591+0000 7f9b15600640 10 client.26791 handle_command_reply: tid=0 2024-09-19T05:12:02.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.591+0000 7f9b3524a740 1 client.26791 shutdown 2024-09-19T05:12:02.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.591+0000 7f9b3524a740 1 -- 172.21.15.43:0/1206223294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaf9ebf590 msgr2=0x55aaf9ec7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:02.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.591+0000 7f9b3524a740 1 --2- 172.21.15.43:0/1206223294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaf9ebf590 0x55aaf9ec7280 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9b180126d0 tx=0x7f9b18012b90 comp rx=0 tx=0).stop 2024-09-19T05:12:02.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.592+0000 7f9b3524a740 1 -- 172.21.15.43:0/1206223294 shutdown_connections 2024-09-19T05:12:02.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.592+0000 7f9b3524a740 1 --2- 172.21.15.43:0/1206223294 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55aaf9ed9f20 0x55aaf9edc330 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:02.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.592+0000 7f9b3524a740 1 --2- 172.21.15.43:0/1206223294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaf9ec9e50 0x55aaf9ec77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:02.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.592+0000 7f9b3524a740 1 --2- 172.21.15.43:0/1206223294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaf9ebf590 0x55aaf9ec7280 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:02.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.592+0000 7f9b3524a740 1 --2- 172.21.15.43:0/1206223294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaf9ebec70 0x55aaf9ec6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:02.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.592+0000 7f9b15600640 0 client.26791 ms_handle_reset on v2:172.21.15.151:6836/3960378010 2024-09-19T05:12:02.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.592+0000 7f9b3524a740 1 -- 172.21.15.43:0/1206223294 >> 172.21.15.43:0/1206223294 conn(0x55aaf9ebb200 msgr2=0x55aaf9eba380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:02.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.592+0000 7f9b3524a740 1 -- 172.21.15.43:0/1206223294 shutdown_connections 2024-09-19T05:12:02.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.592+0000 7f9b3524a740 1 -- 172.21.15.43:0/1206223294 wait complete. 2024-09-19T05:12:02.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.592+0000 7f9b3524a740 20 client.26791 trim_cache size 0 max 16384 2024-09-19T05:12:02.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.605+0000 7f9b3524a740 1 Processor -- start 2024-09-19T05:12:02.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.605+0000 7f9b3524a740 1 -- start start 2024-09-19T05:12:02.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.606+0000 7f9b3524a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaf9ebec70 0x55aaf9ec16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:02.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.606+0000 7f9b3524a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaf9ebf590 0x55aaf9ee1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:02.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.606+0000 7f9b3524a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaf9ec9e50 0x55aaf9ee2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:02.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.606+0000 7f9b3524a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c1afbb0 con 0x55aaf9ebf590 2024-09-19T05:12:02.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.606+0000 7f9b3524a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c1b16f0 con 0x55aaf9ec9e50 2024-09-19T05:12:02.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.606+0000 7f9b3524a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c1b0280 con 0x55aaf9ebec70 2024-09-19T05:12:02.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.606+0000 7f9b2aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaf9ebf590 0x55aaf9ee1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:02.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.606+0000 7f9b2aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaf9ebf590 0x55aaf9ee1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35220/0 (socket says 172.21.15.43:35220) 2024-09-19T05:12:02.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.606+0000 7f9b2aa00640 1 -- 172.21.15.43:0/3409872187 learned_addr learned my addr 172.21.15.43:0/3409872187 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:02.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.606+0000 7f9b2be00640 1 --2- 172.21.15.43:0/3409872187 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaf9ebec70 0x55aaf9ec16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:02.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.606+0000 7f9b2be00640 1 --2- 172.21.15.43:0/3409872187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaf9ec9e50 0x55aaf9ee2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:02.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.606+0000 7f9b2aa00640 1 -- 172.21.15.43:0/3409872187 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaf9ebec70 msgr2=0x55aaf9ec16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:02.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.606+0000 7f9b2aa00640 1 --2- 172.21.15.43:0/3409872187 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaf9ebec70 0x55aaf9ec16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:02.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.606+0000 7f9b2aa00640 1 -- 172.21.15.43:0/3409872187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaf9ec9e50 msgr2=0x55aaf9ee2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:02.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.606+0000 7f9b2aa00640 1 --2- 172.21.15.43:0/3409872187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaf9ec9e50 0x55aaf9ee2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:02.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.606+0000 7f9b2aa00640 1 -- 172.21.15.43:0/3409872187 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaf9eb6770 con 0x55aaf9ebf590 2024-09-19T05:12:02.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.606+0000 7f9b2be00640 1 --2- 172.21.15.43:0/3409872187 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaf9ebec70 0x55aaf9ec16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:12:02.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.607+0000 7f9b2aa00640 1 --2- 172.21.15.43:0/3409872187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaf9ebf590 0x55aaf9ee1c00 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f9b18001eb0 tx=0x7f9b1800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=367f293652d98874 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:02.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.607+0000 7f9b17e00640 1 -- 172.21.15.43:0/3409872187 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b18028070 con 0x55aaf9ebf590 2024-09-19T05:12:02.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.607+0000 7f9b17e00640 1 -- 172.21.15.43:0/3409872187 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b1800fe00 con 0x55aaf9ebf590 2024-09-19T05:12:02.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.607+0000 7f9b17e00640 1 -- 172.21.15.43:0/3409872187 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b18003c10 con 0x55aaf9ebf590 2024-09-19T05:12:02.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.608+0000 7f9b3524a740 1 -- 172.21.15.43:0/3409872187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaf9ebf590 msgr2=0x55aaf9ee1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:02.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.608+0000 7f9b3524a740 1 --2- 172.21.15.43:0/3409872187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaf9ebf590 0x55aaf9ee1c00 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f9b18001eb0 tx=0x7f9b1800f790 comp rx=0 tx=0).stop 2024-09-19T05:12:02.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.608+0000 7f9b3524a740 1 -- 172.21.15.43:0/3409872187 shutdown_connections 2024-09-19T05:12:02.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.608+0000 7f9b3524a740 1 --2- 172.21.15.43:0/3409872187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaf9ec9e50 0x55aaf9ee2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:02.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.608+0000 7f9b3524a740 1 --2- 172.21.15.43:0/3409872187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaf9ebf590 0x55aaf9ee1c00 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:02.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.608+0000 7f9b3524a740 1 --2- 172.21.15.43:0/3409872187 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaf9ebec70 0x55aaf9ec16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:02.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.608+0000 7f9b3524a740 1 -- 172.21.15.43:0/3409872187 >> 172.21.15.43:0/3409872187 conn(0x55aaf9ebb200 msgr2=0x55aaf9ec8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:02.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.608+0000 7f9b3524a740 1 -- 172.21.15.43:0/3409872187 shutdown_connections 2024-09-19T05:12:02.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.608+0000 7f9b3524a740 1 -- 172.21.15.43:0/3409872187 wait complete. 2024-09-19T05:12:02.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.608+0000 7f9b3524a740 1 -- start start 2024-09-19T05:12:02.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.608+0000 7f9b3524a740 1 Processor -- start 2024-09-19T05:12:02.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.609+0000 7f9b3524a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaf9ebec70 0x55aaf9ec3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:02.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.609+0000 7f9b3524a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaf9ebf590 0x55aaf9ec4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:02.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.609+0000 7f9b3524a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaf9ec9e50 0x55aaf9ec4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:02.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.609+0000 7f9b3524a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c1b1020 con 0x55aaf9ebec70 2024-09-19T05:12:02.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.609+0000 7f9b3524a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c1b0950 con 0x55aaf9ec9e50 2024-09-19T05:12:02.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.609+0000 7f9b3524a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c1adbb0 con 0x55aaf9ebf590 2024-09-19T05:12:02.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.609+0000 7f9b3524a740 10 client.0 _ll_register_callbacks cb 0x7f9b18009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:12:02.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.609+0000 7f9b2be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaf9ebec70 0x55aaf9ec3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:02.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.609+0000 7f9b2aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaf9ebf590 0x55aaf9ec4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:02.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.609+0000 7f9b2aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaf9ebf590 0x55aaf9ec4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:59900/0 (socket says 172.21.15.43:59900) 2024-09-19T05:12:02.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.609+0000 7f9b2be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaf9ebec70 0x55aaf9ec3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35228/0 (socket says 172.21.15.43:35228) 2024-09-19T05:12:02.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.609+0000 7f9b2aa00640 1 -- 172.21.15.43:0/1577206024 learned_addr learned my addr 172.21.15.43:0/1577206024 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:02.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.609+0000 7f9b2be00640 1 --2- 172.21.15.43:0/1577206024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaf9ec9e50 0x55aaf9ec4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:02.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.610+0000 7f9b2be00640 1 -- 172.21.15.43:0/1577206024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaf9ebf590 msgr2=0x55aaf9ec4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:02.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.610+0000 7f9b2be00640 1 --2- 172.21.15.43:0/1577206024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaf9ebf590 0x55aaf9ec4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:02.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.610+0000 7f9b2be00640 1 -- 172.21.15.43:0/1577206024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaf9ec9e50 msgr2=0x55aaf9ec4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:02.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.610+0000 7f9b2be00640 1 --2- 172.21.15.43:0/1577206024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaf9ec9e50 0x55aaf9ec4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:02.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.610+0000 7f9b2be00640 1 -- 172.21.15.43:0/1577206024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aaf9f43bd0 con 0x55aaf9ebec70 2024-09-19T05:12:02.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.610+0000 7f9b2aa00640 1 --2- 172.21.15.43:0/1577206024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaf9ebf590 0x55aaf9ec4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:12:02.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.610+0000 7f9b2be00640 1 --2- 172.21.15.43:0/1577206024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaf9ebec70 0x55aaf9ec3cf0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9b2402a900 tx=0x7f9b18009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5920193d3033680 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:02.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.610+0000 7f9b15600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:12:02.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.610+0000 7f9b15600640 1 -- 172.21.15.43:0/1577206024 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b24031bb0 con 0x55aaf9ebec70 2024-09-19T05:12:02.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.610+0000 7f9b15600640 1 -- 172.21.15.43:0/1577206024 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f9b24031d50 con 0x55aaf9ebec70 2024-09-19T05:12:02.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.610+0000 7f9b3524a740 1 -- 172.21.15.43:0/1577206024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b1800fe00 con 0x55aaf9ebec70 2024-09-19T05:12:02.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.610+0000 7f9b15600640 1 -- 172.21.15.43:0/1577206024 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9b24004b40 con 0x55aaf9ebec70 2024-09-19T05:12:02.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.610+0000 7f9b3524a740 10 client.16839 fetch_fsmap learned FSMap version 29 2024-09-19T05:12:02.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.610+0000 7f9b3524a740 1 -- 172.21.15.43:0/1577206024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55aaf9f44e10 con 0x55aaf9ebec70 2024-09-19T05:12:02.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.611+0000 7f9b15600640 1 -- 172.21.15.43:0/1577206024 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b1800fe00 con 0x55aaf9ebec70 2024-09-19T05:12:02.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.611+0000 7f9b15600640 1 -- 172.21.15.43:0/1577206024 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9b240042b0 con 0x55aaf9ebec70 2024-09-19T05:12:02.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.611+0000 7f9b3524a740 10 client.16839 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:12:02.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.611+0000 7f9b3524a740 10 client.16839 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-19T05:12:02.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.611+0000 7f9b3524a740 20 client.16839 populate_metadata read hostname 'smithi043' 2024-09-19T05:12:02.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.611+0000 7f9b3524a740 1 --2- 172.21.15.43:0/1577206024 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55aaf9f45030 0x55aaf9f47420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:02.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.611+0000 7f9b3524a740 4 client.16839 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:12:02.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.611+0000 7f9b3524a740 1 -- 172.21.15.43:0/1577206024 --> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aaf9e2ce60 con 0x55aaf9f45030 2024-09-19T05:12:02.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.611+0000 7f9b2aa00640 1 --2- 172.21.15.43:0/1577206024 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55aaf9f45030 0x55aaf9f47420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:02.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.612+0000 7f9b2aa00640 1 --2- 172.21.15.43:0/1577206024 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55aaf9f45030 0x55aaf9f47420 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=71ad32787ce5fd2e server_cookie=aa8e7bd8b6925238 in_seq=0 out_seq=0 2024-09-19T05:12:02.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.612+0000 7f9b15600640 10 client.16839 ms_handle_connect on v2:172.21.15.151:6836/3960378010 2024-09-19T05:12:02.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.612+0000 7f9b15600640 1 -- 172.21.15.43:0/1577206024 <== mds.4 v2:172.21.15.151:6836/3960378010 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aaf9e2ce60 con 0x55aaf9f45030 2024-09-19T05:12:02.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.612+0000 7f9b15600640 10 client.16839 handle_command_reply: tid=0 2024-09-19T05:12:02.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.612+0000 7f9b3524a740 1 client.16839 shutdown 2024-09-19T05:12:02.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.612+0000 7f9b3524a740 1 -- 172.21.15.43:0/1577206024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaf9ebec70 msgr2=0x55aaf9ec3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:02.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.612+0000 7f9b3524a740 1 --2- 172.21.15.43:0/1577206024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaf9ebec70 0x55aaf9ec3cf0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9b2402a900 tx=0x7f9b18009d50 comp rx=0 tx=0).stop 2024-09-19T05:12:02.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.613+0000 7f9b3524a740 1 -- 172.21.15.43:0/1577206024 shutdown_connections 2024-09-19T05:12:02.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.613+0000 7f9b3524a740 1 --2- 172.21.15.43:0/1577206024 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55aaf9f45030 0x55aaf9f47420 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:02.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.613+0000 7f9b3524a740 1 --2- 172.21.15.43:0/1577206024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaf9ec9e50 0x55aaf9ec4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:02.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.613+0000 7f9b3524a740 1 --2- 172.21.15.43:0/1577206024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaf9ebf590 0x55aaf9ec4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:02.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.613+0000 7f9b3524a740 1 --2- 172.21.15.43:0/1577206024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaf9ebec70 0x55aaf9ec3cf0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:02.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.613+0000 7f9b3524a740 1 -- 172.21.15.43:0/1577206024 >> 172.21.15.43:0/1577206024 conn(0x55aaf9ebb200 msgr2=0x55aaf9eb9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:02.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.613+0000 7f9b15600640 0 client.16839 ms_handle_reset on v2:172.21.15.151:6836/3960378010 2024-09-19T05:12:02.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.613+0000 7f9b3524a740 1 -- 172.21.15.43:0/1577206024 shutdown_connections 2024-09-19T05:12:02.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.613+0000 7f9b3524a740 1 -- 172.21.15.43:0/1577206024 wait complete. 2024-09-19T05:12:02.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.613+0000 7f9b3524a740 20 client.16839 trim_cache size 0 max 16384 2024-09-19T05:12:02.619 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:12:02.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.614+0000 7f9b32600640 1 -- 172.21.15.43:0/2142276631 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f9b100a3430 msgr2=0x7f9b100a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:02.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.614+0000 7f9b32600640 1 --2- 172.21.15.43:0/2142276631 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f9b100a3430 0x7f9b100a58f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x55aaf9ebc060 tx=0x7f9b1c0d3c30 comp rx=0 tx=0).stop 2024-09-19T05:12:02.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.614+0000 7f9b32600640 1 -- 172.21.15.43:0/2142276631 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b2c103ff0 msgr2=0x7f9b2c19a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:02.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.614+0000 7f9b32600640 1 --2- 172.21.15.43:0/2142276631 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b2c103ff0 0x7f9b2c19a620 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c00e9f0 tx=0x7f9b1c00eeb0 comp rx=0 tx=0).stop 2024-09-19T05:12:02.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.614+0000 7f9b32600640 1 -- 172.21.15.43:0/2142276631 shutdown_connections 2024-09-19T05:12:02.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.614+0000 7f9b32600640 1 --2- 172.21.15.43:0/2142276631 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f9b100a3430 0x7f9b100a58f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:02.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.614+0000 7f9b32600640 1 --2- 172.21.15.43:0/2142276631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b2c109120 0x7f9b2c19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:02.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.614+0000 7f9b32600640 1 --2- 172.21.15.43:0/2142276631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b2c1049a0 0x7f9b2c19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:02.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.614+0000 7f9b32600640 1 --2- 172.21.15.43:0/2142276631 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b2c103ff0 0x7f9b2c19a620 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:02.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.614+0000 7f9b32600640 1 -- 172.21.15.43:0/2142276631 >> 172.21.15.43:0/2142276631 conn(0x7f9b2c0ffa90 msgr2=0x7f9b2c100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:02.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.614+0000 7f9b32600640 1 -- 172.21.15.43:0/2142276631 shutdown_connections 2024-09-19T05:12:02.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:02.614+0000 7f9b32600640 1 -- 172.21.15.43:0/2142276631 wait complete. 2024-09-19T05:12:03.507 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T05:12:03.887 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:12:04.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:04 smithi043 ceph-mon[30800]: pgmap v227: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:12:04.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:04 smithi151 ceph-mon[35425]: pgmap v227: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:12:04.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:04 smithi096 ceph-mon[35085]: pgmap v227: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:12:04.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.495+0000 7ff009c00640 1 -- 172.21.15.43:0/548579250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff00410e260 msgr2=0x7ff004110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:04.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.495+0000 7ff009c00640 1 --2- 172.21.15.43:0/548579250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff00410e260 0x7ff004110650 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7feffc009ca0 tx=0x7feffc02a950 comp rx=0 tx=0).stop 2024-09-19T05:12:04.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.495+0000 7ff009c00640 1 -- 172.21.15.43:0/548579250 shutdown_connections 2024-09-19T05:12:04.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.495+0000 7ff009c00640 1 --2- 172.21.15.43:0/548579250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff00410e260 0x7ff004110650 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:04.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.495+0000 7ff009c00640 1 --2- 172.21.15.43:0/548579250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff004074ee0 0x7ff00410dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:04.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.495+0000 7ff009c00640 1 --2- 172.21.15.43:0/548579250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0040745c0 0x7ff0040749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:04.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.495+0000 7ff009c00640 1 -- 172.21.15.43:0/548579250 >> 172.21.15.43:0/548579250 conn(0x7ff0040fd740 msgr2=0x7ff0040ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:04.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.495+0000 7ff009c00640 1 -- 172.21.15.43:0/548579250 shutdown_connections 2024-09-19T05:12:04.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.495+0000 7ff009c00640 1 -- 172.21.15.43:0/548579250 wait complete. 2024-09-19T05:12:04.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.497+0000 7ff009c00640 1 Processor -- start 2024-09-19T05:12:04.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.497+0000 7ff009c00640 1 -- start start 2024-09-19T05:12:04.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.497+0000 7ff009c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0040745c0 0x7ff00419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:04.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.497+0000 7ff009c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff004074ee0 0x7ff00419a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:04.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.497+0000 7ff009c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff00410e260 0x7ff00419ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:04.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.497+0000 7ff009c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff004113f70 con 0x7ff00410e260 2024-09-19T05:12:04.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.497+0000 7ff009c00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff004113df0 con 0x7ff0040745c0 2024-09-19T05:12:04.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.497+0000 7ff009c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff0041140f0 con 0x7ff004074ee0 2024-09-19T05:12:04.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.498+0000 7ff003e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0040745c0 0x7ff00419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:04.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.498+0000 7ff003400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff004074ee0 0x7ff00419a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:04.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.498+0000 7ff003e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0040745c0 0x7ff00419a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:41376/0 (socket says 172.21.15.43:41376) 2024-09-19T05:12:04.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.498+0000 7ff003e00640 1 -- 172.21.15.43:0/885037609 learned_addr learned my addr 172.21.15.43:0/885037609 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:04.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.498+0000 7ff009200640 1 --2- 172.21.15.43:0/885037609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff00410e260 0x7ff00419ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:04.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.498+0000 7ff003400640 1 -- 172.21.15.43:0/885037609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0040745c0 msgr2=0x7ff00419a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:04.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.498+0000 7ff003400640 1 --2- 172.21.15.43:0/885037609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0040745c0 0x7ff00419a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:04.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.498+0000 7ff003400640 1 -- 172.21.15.43:0/885037609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff00410e260 msgr2=0x7ff00419ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:04.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.498+0000 7ff003400640 1 --2- 172.21.15.43:0/885037609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff00410e260 0x7ff00419ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:04.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.498+0000 7ff003400640 1 -- 172.21.15.43:0/885037609 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff00419f3e0 con 0x7ff004074ee0 2024-09-19T05:12:04.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.498+0000 7ff003e00640 1 --2- 172.21.15.43:0/885037609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0040745c0 0x7ff00419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:12:04.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.499+0000 7ff003400640 1 --2- 172.21.15.43:0/885037609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff004074ee0 0x7ff00419a970 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7feff000e9f0 tx=0x7feff000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:04.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.499+0000 7ff000c00640 1 -- 172.21.15.43:0/885037609 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff000ccc0 con 0x7ff004074ee0 2024-09-19T05:12:04.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.499+0000 7ff000c00640 1 -- 172.21.15.43:0/885037609 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feff0010910 con 0x7ff004074ee0 2024-09-19T05:12:04.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.499+0000 7ff009c00640 1 -- 172.21.15.43:0/885037609 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff00419f6d0 con 0x7ff004074ee0 2024-09-19T05:12:04.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.499+0000 7ff000c00640 1 -- 172.21.15.43:0/885037609 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff0018960 con 0x7ff004074ee0 2024-09-19T05:12:04.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.499+0000 7ff009c00640 1 -- 172.21.15.43:0/885037609 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0041a6fb0 con 0x7ff004074ee0 2024-09-19T05:12:04.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.502+0000 7ff00dc28740 1 Processor -- start 2024-09-19T05:12:04.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.502+0000 7ff00dc28740 1 -- start start 2024-09-19T05:12:04.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.503+0000 7ff00dc28740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee5608ee40 0x55ee5608f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:04.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.503+0000 7ff00dc28740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee5608f760 0x55ee56099ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:04.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.503+0000 7ff00dc28740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee5609a020 0x55ee5609c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:04.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.503+0000 7ff00dc28740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ee55ff53e0 con 0x55ee5609a020 2024-09-19T05:12:04.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.503+0000 7ff00dc28740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ee55f9cd80 con 0x55ee5608f760 2024-09-19T05:12:04.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.503+0000 7ff00dc28740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ee55ff13a0 con 0x55ee5608ee40 2024-09-19T05:12:04.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.503+0000 7ff000c00640 1 -- 172.21.15.43:0/885037609 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7feff0010ac0 con 0x7ff004074ee0 2024-09-19T05:12:04.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.503+0000 7ff009200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee5608ee40 0x55ee5608f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:04.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.503+0000 7ff009200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee5608ee40 0x55ee5608f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:59918/0 (socket says 172.21.15.43:59918) 2024-09-19T05:12:04.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.503+0000 7ff009200640 1 -- 172.21.15.43:0/1399105538 learned_addr learned my addr 172.21.15.43:0/1399105538 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:04.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.503+0000 7ff003e00640 1 --2- 172.21.15.43:0/1399105538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee5608f760 0x55ee56099ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:04.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.503+0000 7ff003400640 1 --2- 172.21.15.43:0/1399105538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee5609a020 0x55ee5609c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:04.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.503+0000 7ff009200640 1 -- 172.21.15.43:0/1399105538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee5608f760 msgr2=0x55ee56099ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:04.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.503+0000 7ff009200640 1 --2- 172.21.15.43:0/1399105538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee5608f760 0x55ee56099ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:04.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.503+0000 7ff009200640 1 -- 172.21.15.43:0/1399105538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee5609a020 msgr2=0x55ee5609c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:04.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.503+0000 7ff009200640 1 --2- 172.21.15.43:0/1399105538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee5609a020 0x55ee5609c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:04.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.503+0000 7ff009200640 1 -- 172.21.15.43:0/1399105538 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee55fbbae0 con 0x55ee5608ee40 2024-09-19T05:12:04.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.503+0000 7ff003400640 1 --2- 172.21.15.43:0/1399105538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee5609a020 0x55ee5609c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:12:04.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.504+0000 7ff009200640 1 --2- 172.21.15.43:0/1399105538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee5608ee40 0x55ee5608f220 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7feffc009ca0 tx=0x7feffc0093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39e9affad29b0e42 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:04.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.504+0000 7feff9600640 1 -- 172.21.15.43:0/1399105538 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feffc048070 con 0x55ee5608ee40 2024-09-19T05:12:04.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.504+0000 7feff9600640 1 -- 172.21.15.43:0/1399105538 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feffc02f940 con 0x55ee5608ee40 2024-09-19T05:12:04.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.504+0000 7feff9600640 1 -- 172.21.15.43:0/1399105538 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feffc034040 con 0x55ee5608ee40 2024-09-19T05:12:04.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.505+0000 7ff00dc28740 1 -- 172.21.15.43:0/1399105538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee5608ee40 msgr2=0x55ee5608f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:04.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.505+0000 7ff00dc28740 1 --2- 172.21.15.43:0/1399105538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee5608ee40 0x55ee5608f220 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7feffc009ca0 tx=0x7feffc0093e0 comp rx=0 tx=0).stop 2024-09-19T05:12:04.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.505+0000 7ff000c00640 1 --2- 172.21.15.43:0/885037609 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fefe80a3430 0x7fefe80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:04.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.505+0000 7ff00dc28740 1 -- 172.21.15.43:0/1399105538 shutdown_connections 2024-09-19T05:12:04.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.505+0000 7ff00dc28740 1 --2- 172.21.15.43:0/1399105538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee5609a020 0x55ee5609c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:04.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.505+0000 7ff00dc28740 1 --2- 172.21.15.43:0/1399105538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee5608f760 0x55ee56099ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:04.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.505+0000 7ff00dc28740 1 --2- 172.21.15.43:0/1399105538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee5608ee40 0x55ee5608f220 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:04.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.505+0000 7ff00dc28740 1 -- 172.21.15.43:0/1399105538 >> 172.21.15.43:0/1399105538 conn(0x55ee5608b3d0 msgr2=0x55ee5608b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:04.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.505+0000 7ff000c00640 1 -- 172.21.15.43:0/885037609 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7feff00cffe0 con 0x7ff004074ee0 2024-09-19T05:12:04.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.505+0000 7ff00dc28740 1 -- 172.21.15.43:0/1399105538 shutdown_connections 2024-09-19T05:12:04.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.505+0000 7ff003e00640 1 --2- 172.21.15.43:0/885037609 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fefe80a3430 0x7fefe80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:04.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.505+0000 7ff00dc28740 1 -- 172.21.15.43:0/1399105538 wait complete. 2024-09-19T05:12:04.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.506+0000 7ff00dc28740 1 -- start start 2024-09-19T05:12:04.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.506+0000 7ff003e00640 1 --2- 172.21.15.43:0/885037609 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fefe80a3430 0x7fefe80a58f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x55ee5608c230 tx=0x7feff4005f90 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:04.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.506+0000 7ff00dc28740 1 Processor -- start 2024-09-19T05:12:04.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.507+0000 7ff00dc28740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee5608ee40 0x55ee56096ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:04.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.507+0000 7ff00dc28740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee5608f760 0x55ee56097420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:04.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.507+0000 7ff00dc28740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee5609a020 0x55ee56097960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:04.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.507+0000 7ff00dc28740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ee5609cbf0 con 0x55ee5609a020 2024-09-19T05:12:04.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.507+0000 7ff00dc28740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ee55fb00f0 con 0x55ee5608ee40 2024-09-19T05:12:04.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.507+0000 7ff00dc28740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ee5609cd70 con 0x55ee5608f760 2024-09-19T05:12:04.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.507+0000 7ff00dc28740 10 client.0 _ll_register_callbacks cb 0x55ee55f97bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:12:04.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.507+0000 7ff003400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee5608f760 0x55ee56097420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:04.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.507+0000 7ff003400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee5608f760 0x55ee56097420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:59930/0 (socket says 172.21.15.43:59930) 2024-09-19T05:12:04.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.507+0000 7ff003e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee5608ee40 0x55ee56096ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:04.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.507+0000 7ff003400640 1 -- 172.21.15.43:0/674599048 learned_addr learned my addr 172.21.15.43:0/674599048 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:04.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.507+0000 7ff009200640 1 --2- 172.21.15.43:0/674599048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee5609a020 0x55ee56097960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:04.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.507+0000 7ff003400640 1 -- 172.21.15.43:0/674599048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee5608ee40 msgr2=0x55ee56096ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:04.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.507+0000 7ff003400640 1 --2- 172.21.15.43:0/674599048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee5608ee40 0x55ee56096ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:04.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.507+0000 7ff003400640 1 -- 172.21.15.43:0/674599048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee5609a020 msgr2=0x55ee56097960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:04.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.507+0000 7ff003400640 1 --2- 172.21.15.43:0/674599048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee5609a020 0x55ee56097960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:04.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.507+0000 7ff003400640 1 -- 172.21.15.43:0/674599048 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee55d663c0 con 0x55ee5608f760 2024-09-19T05:12:04.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.508+0000 7ff003400640 1 --2- 172.21.15.43:0/674599048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee5608f760 0x55ee56097420 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7feff000c6a0 tx=0x7feff009bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83252339e9c6772 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:04.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.508+0000 7fefee000640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:12:04.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.508+0000 7fefee000640 1 -- 172.21.15.43:0/674599048 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff00d2270 con 0x55ee5608f760 2024-09-19T05:12:04.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.508+0000 7fefee000640 1 -- 172.21.15.43:0/674599048 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7feff00bc0b0 con 0x55ee5608f760 2024-09-19T05:12:04.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.508+0000 7ff00dc28740 1 -- 172.21.15.43:0/674599048 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feffc02f940 con 0x55ee5608f760 2024-09-19T05:12:04.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.508+0000 7fefee000640 1 -- 172.21.15.43:0/674599048 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7feff00a4f20 con 0x55ee5608f760 2024-09-19T05:12:04.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.508+0000 7ff00dc28740 10 client.26809 fetch_fsmap learned FSMap version 29 2024-09-19T05:12:04.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.508+0000 7ff00dc28740 1 -- 172.21.15.43:0/674599048 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ee560a8bc0 con 0x55ee5608f760 2024-09-19T05:12:04.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.509+0000 7fefee000640 1 -- 172.21.15.43:0/674599048 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feffc02f940 con 0x55ee5608f760 2024-09-19T05:12:04.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.509+0000 7fefee000640 1 -- 172.21.15.43:0/674599048 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7feff00a0270 con 0x55ee5608f760 2024-09-19T05:12:04.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.509+0000 7ff00dc28740 10 client.26809 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:12:04.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.509+0000 7ff00dc28740 10 client.26809 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T05:12:04.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.509+0000 7ff00dc28740 20 client.26809 populate_metadata read hostname 'smithi043' 2024-09-19T05:12:04.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.509+0000 7ff00dc28740 1 --2- 172.21.15.43:0/674599048 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55ee560a9fd0 0x55ee560ac400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:04.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.509+0000 7ff00dc28740 4 client.26809 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:12:04.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.509+0000 7ff00dc28740 1 -- 172.21.15.43:0/674599048 --> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee56002950 con 0x55ee560a9fd0 2024-09-19T05:12:04.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.510+0000 7ff003e00640 1 --2- 172.21.15.43:0/674599048 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55ee560a9fd0 0x55ee560ac400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:04.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.510+0000 7ff003e00640 1 --2- 172.21.15.43:0/674599048 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55ee560a9fd0 0x55ee560ac400 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e292e36828d7ae9b server_cookie=9b8fdc2e1543dc0e in_seq=0 out_seq=0 2024-09-19T05:12:04.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.510+0000 7fefee000640 10 client.26809 ms_handle_connect on v2:172.21.15.96:6836/3978388286 2024-09-19T05:12:04.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.511+0000 7fefee000640 1 -- 172.21.15.43:0/674599048 <== mds.3 v2:172.21.15.96:6836/3978388286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ee56002950 con 0x55ee560a9fd0 2024-09-19T05:12:04.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.511+0000 7fefee000640 10 client.26809 handle_command_reply: tid=0 2024-09-19T05:12:04.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.511+0000 7ff00dc28740 1 client.26809 shutdown 2024-09-19T05:12:04.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.511+0000 7ff00dc28740 1 -- 172.21.15.43:0/674599048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee5608f760 msgr2=0x55ee56097420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:04.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.511+0000 7ff00dc28740 1 --2- 172.21.15.43:0/674599048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee5608f760 0x55ee56097420 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7feff000c6a0 tx=0x7feff009bf60 comp rx=0 tx=0).stop 2024-09-19T05:12:04.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.511+0000 7ff00dc28740 1 -- 172.21.15.43:0/674599048 shutdown_connections 2024-09-19T05:12:04.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.511+0000 7ff00dc28740 1 --2- 172.21.15.43:0/674599048 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55ee560a9fd0 0x55ee560ac400 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:04.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.511+0000 7ff00dc28740 1 --2- 172.21.15.43:0/674599048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee5609a020 0x55ee56097960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:04.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.511+0000 7ff00dc28740 1 --2- 172.21.15.43:0/674599048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee5608f760 0x55ee56097420 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:04.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.511+0000 7ff00dc28740 1 --2- 172.21.15.43:0/674599048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee5608ee40 0x55ee56096ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:04.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.511+0000 7ff00dc28740 1 -- 172.21.15.43:0/674599048 >> 172.21.15.43:0/674599048 conn(0x55ee5608b3d0 msgr2=0x55ee56091150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:04.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.511+0000 7fefee000640 0 client.26809 ms_handle_reset on v2:172.21.15.96:6836/3978388286 2024-09-19T05:12:04.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.511+0000 7ff00dc28740 1 -- 172.21.15.43:0/674599048 shutdown_connections 2024-09-19T05:12:04.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.511+0000 7ff00dc28740 1 -- 172.21.15.43:0/674599048 wait complete. 2024-09-19T05:12:04.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.511+0000 7ff00dc28740 20 client.26809 trim_cache size 0 max 16384 2024-09-19T05:12:04.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.525+0000 7ff00dc28740 1 Processor -- start 2024-09-19T05:12:04.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.525+0000 7ff00dc28740 1 -- start start 2024-09-19T05:12:04.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.526+0000 7ff00dc28740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee5608ee40 0x55ee560b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:04.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.526+0000 7ff00dc28740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee5608f760 0x55ee560b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:04.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.526+0000 7ff00dc28740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee5609a020 0x55ee56094160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:04.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.526+0000 7ff00dc28740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff0041af850 con 0x55ee5608ee40 2024-09-19T05:12:04.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.526+0000 7ff00dc28740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff0041b1390 con 0x55ee5609a020 2024-09-19T05:12:04.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.526+0000 7ff00dc28740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff0041aff20 con 0x55ee5608f760 2024-09-19T05:12:04.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.526+0000 7ff003e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee5608ee40 0x55ee560b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:04.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.526+0000 7ff003400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee5608f760 0x55ee560b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:04.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.526+0000 7ff003400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee5608f760 0x55ee560b2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:59946/0 (socket says 172.21.15.43:59946) 2024-09-19T05:12:04.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.526+0000 7ff003e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee5608ee40 0x55ee560b1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35262/0 (socket says 172.21.15.43:35262) 2024-09-19T05:12:04.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.526+0000 7ff003400640 1 -- 172.21.15.43:0/3757423632 learned_addr learned my addr 172.21.15.43:0/3757423632 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:04.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.526+0000 7ff009200640 1 --2- 172.21.15.43:0/3757423632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee5609a020 0x55ee56094160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:04.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.527+0000 7ff003400640 1 -- 172.21.15.43:0/3757423632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee5609a020 msgr2=0x55ee56094160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:04.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.527+0000 7ff003400640 1 --2- 172.21.15.43:0/3757423632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee5609a020 0x55ee56094160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:04.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.527+0000 7ff003400640 1 -- 172.21.15.43:0/3757423632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee5608ee40 msgr2=0x55ee560b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:04.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.527+0000 7ff003400640 1 --2- 172.21.15.43:0/3757423632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee5608ee40 0x55ee560b1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:04.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.527+0000 7ff003400640 1 -- 172.21.15.43:0/3757423632 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee5608c060 con 0x55ee5608f760 2024-09-19T05:12:04.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.527+0000 7ff003400640 1 --2- 172.21.15.43:0/3757423632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee5608f760 0x55ee560b2210 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7feff00987c0 tx=0x7feff009bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e053f2f2cd9a09d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:04.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.527+0000 7feff8c00640 1 -- 172.21.15.43:0/3757423632 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff009c130 con 0x55ee5608f760 2024-09-19T05:12:04.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.527+0000 7feff8c00640 1 -- 172.21.15.43:0/3757423632 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feff00bc0f0 con 0x55ee5608f760 2024-09-19T05:12:04.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.527+0000 7feff8c00640 1 -- 172.21.15.43:0/3757423632 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff00a03b0 con 0x55ee5608f760 2024-09-19T05:12:04.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.528+0000 7ff00dc28740 1 -- 172.21.15.43:0/3757423632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee5608f760 msgr2=0x55ee560b2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:04.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.528+0000 7ff00dc28740 1 --2- 172.21.15.43:0/3757423632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee5608f760 0x55ee560b2210 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7feff00987c0 tx=0x7feff009bfd0 comp rx=0 tx=0).stop 2024-09-19T05:12:04.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.528+0000 7ff00dc28740 1 -- 172.21.15.43:0/3757423632 shutdown_connections 2024-09-19T05:12:04.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.528+0000 7ff00dc28740 1 --2- 172.21.15.43:0/3757423632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee5609a020 0x55ee56094160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:04.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.528+0000 7ff00dc28740 1 --2- 172.21.15.43:0/3757423632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee5608f760 0x55ee560b2210 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:04.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.528+0000 7ff00dc28740 1 --2- 172.21.15.43:0/3757423632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee5608ee40 0x55ee560b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:04.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.528+0000 7ff00dc28740 1 -- 172.21.15.43:0/3757423632 >> 172.21.15.43:0/3757423632 conn(0x55ee5608b3d0 msgr2=0x55ee56098f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:04.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.528+0000 7ff00dc28740 1 -- 172.21.15.43:0/3757423632 shutdown_connections 2024-09-19T05:12:04.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.528+0000 7ff00dc28740 1 -- 172.21.15.43:0/3757423632 wait complete. 2024-09-19T05:12:04.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.528+0000 7ff00dc28740 1 -- start start 2024-09-19T05:12:04.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.529+0000 7ff00dc28740 1 Processor -- start 2024-09-19T05:12:04.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.529+0000 7ff00dc28740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee5608ee40 0x55ee56094020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:04.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.529+0000 7ff00dc28740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee5608f760 0x55ee56094560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:04.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.529+0000 7ff00dc28740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee5609a020 0x55ee56094aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:04.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.529+0000 7ff00dc28740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff0041b0cc0 con 0x55ee5608ee40 2024-09-19T05:12:04.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.529+0000 7ff00dc28740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff0041b05f0 con 0x55ee5608f760 2024-09-19T05:12:04.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.529+0000 7ff00dc28740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff0041ad850 con 0x55ee5609a020 2024-09-19T05:12:04.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.529+0000 7ff00dc28740 10 client.0 _ll_register_callbacks cb 0x7feff000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:12:04.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.529+0000 7ff003e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee5608ee40 0x55ee56094020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:04.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.529+0000 7ff003e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee5608ee40 0x55ee56094020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35274/0 (socket says 172.21.15.43:35274) 2024-09-19T05:12:04.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.529+0000 7ff003e00640 1 -- 172.21.15.43:0/1467941141 learned_addr learned my addr 172.21.15.43:0/1467941141 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:04.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.529+0000 7ff003400640 1 --2- 172.21.15.43:0/1467941141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee5608f760 0x55ee56094560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:04.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.529+0000 7ff009200640 1 --2- 172.21.15.43:0/1467941141 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee5609a020 0x55ee56094aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:04.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.530+0000 7ff003e00640 1 -- 172.21.15.43:0/1467941141 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee5609a020 msgr2=0x55ee56094aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:04.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.530+0000 7ff003e00640 1 --2- 172.21.15.43:0/1467941141 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee5609a020 0x55ee56094aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:04.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.530+0000 7ff003e00640 1 -- 172.21.15.43:0/1467941141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee5608f760 msgr2=0x55ee56094560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:04.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.530+0000 7ff003e00640 1 --2- 172.21.15.43:0/1467941141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee5608f760 0x55ee56094560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:04.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.530+0000 7ff003e00640 1 -- 172.21.15.43:0/1467941141 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee56113dd0 con 0x55ee5608ee40 2024-09-19T05:12:04.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.530+0000 7ff003e00640 1 --2- 172.21.15.43:0/1467941141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee5608ee40 0x55ee56094020 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x55ee560acc40 tx=0x7feff4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c477b1c20e476 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:04.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.530+0000 7fefee000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:12:04.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.530+0000 7fefee000640 1 -- 172.21.15.43:0/1467941141 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff4005020 con 0x55ee5608ee40 2024-09-19T05:12:04.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.530+0000 7ff00dc28740 1 -- 172.21.15.43:0/1467941141 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feff00bc0f0 con 0x55ee5608ee40 2024-09-19T05:12:04.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.530+0000 7fefee000640 1 -- 172.21.15.43:0/1467941141 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7feff40051c0 con 0x55ee5608ee40 2024-09-19T05:12:04.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.531+0000 7fefee000640 1 -- 172.21.15.43:0/1467941141 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7feff40053c0 con 0x55ee5608ee40 2024-09-19T05:12:04.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.531+0000 7ff00dc28740 10 client.16854 fetch_fsmap learned FSMap version 29 2024-09-19T05:12:04.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.531+0000 7ff00dc28740 1 -- 172.21.15.43:0/1467941141 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ee56115050 con 0x55ee5608ee40 2024-09-19T05:12:04.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.531+0000 7fefee000640 1 -- 172.21.15.43:0/1467941141 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feff00bc0f0 con 0x55ee5608ee40 2024-09-19T05:12:04.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.531+0000 7fefee000640 1 -- 172.21.15.43:0/1467941141 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7feff401d9e0 con 0x55ee5608ee40 2024-09-19T05:12:04.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.531+0000 7ff00dc28740 10 client.16854 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:12:04.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.531+0000 7ff00dc28740 10 client.16854 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T05:12:04.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.531+0000 7ff00dc28740 20 client.16854 populate_metadata read hostname 'smithi043' 2024-09-19T05:12:04.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.531+0000 7ff00dc28740 1 --2- 172.21.15.43:0/1467941141 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55ee561157d0 0x55ee56117bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:04.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.531+0000 7ff00dc28740 4 client.16854 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:12:04.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.531+0000 7ff00dc28740 1 -- 172.21.15.43:0/1467941141 --> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ee55ffce60 con 0x55ee561157d0 2024-09-19T05:12:04.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.532+0000 7ff003400640 1 --2- 172.21.15.43:0/1467941141 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55ee561157d0 0x55ee56117bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:04.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.532+0000 7ff003400640 1 --2- 172.21.15.43:0/1467941141 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55ee561157d0 0x55ee56117bf0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=16b62435c52f379 server_cookie=622c5d43148deb50 in_seq=0 out_seq=0 2024-09-19T05:12:04.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.532+0000 7fefee000640 10 client.16854 ms_handle_connect on v2:172.21.15.96:6836/3978388286 2024-09-19T05:12:04.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.533+0000 7fefee000640 1 -- 172.21.15.43:0/1467941141 <== mds.3 v2:172.21.15.96:6836/3978388286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ee55ffce60 con 0x55ee561157d0 2024-09-19T05:12:04.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.533+0000 7fefee000640 10 client.16854 handle_command_reply: tid=0 2024-09-19T05:12:04.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.533+0000 7ff00dc28740 1 client.16854 shutdown 2024-09-19T05:12:04.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.533+0000 7ff00dc28740 1 -- 172.21.15.43:0/1467941141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee5608ee40 msgr2=0x55ee56094020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:04.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.533+0000 7ff00dc28740 1 --2- 172.21.15.43:0/1467941141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee5608ee40 0x55ee56094020 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x55ee560acc40 tx=0x7feff4017000 comp rx=0 tx=0).stop 2024-09-19T05:12:04.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.533+0000 7ff00dc28740 1 -- 172.21.15.43:0/1467941141 shutdown_connections 2024-09-19T05:12:04.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.533+0000 7ff00dc28740 1 --2- 172.21.15.43:0/1467941141 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55ee561157d0 0x55ee56117bf0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:04.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.533+0000 7ff00dc28740 1 --2- 172.21.15.43:0/1467941141 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee5609a020 0x55ee56094aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:04.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.533+0000 7ff00dc28740 1 --2- 172.21.15.43:0/1467941141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee5608f760 0x55ee56094560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:04.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.533+0000 7ff00dc28740 1 --2- 172.21.15.43:0/1467941141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ee5608ee40 0x55ee56094020 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:04.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.533+0000 7ff00dc28740 1 -- 172.21.15.43:0/1467941141 >> 172.21.15.43:0/1467941141 conn(0x55ee5608b3d0 msgr2=0x55ee56098740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:04.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.533+0000 7fefee000640 0 client.16854 ms_handle_reset on v2:172.21.15.96:6836/3978388286 2024-09-19T05:12:04.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.533+0000 7ff00dc28740 1 -- 172.21.15.43:0/1467941141 shutdown_connections 2024-09-19T05:12:04.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.533+0000 7ff00dc28740 1 -- 172.21.15.43:0/1467941141 wait complete. 2024-09-19T05:12:04.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.533+0000 7ff00dc28740 20 client.16854 trim_cache size 0 max 16384 2024-09-19T05:12:04.539 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:12:04.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.534+0000 7ff009c00640 1 -- 172.21.15.43:0/885037609 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fefe80a3430 msgr2=0x7fefe80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:04.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.534+0000 7ff009c00640 1 --2- 172.21.15.43:0/885037609 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fefe80a3430 0x7fefe80a58f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x55ee5608c230 tx=0x7feff4005f90 comp rx=0 tx=0).stop 2024-09-19T05:12:04.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.534+0000 7ff009c00640 1 -- 172.21.15.43:0/885037609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff004074ee0 msgr2=0x7ff00419a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:04.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.534+0000 7ff009c00640 1 --2- 172.21.15.43:0/885037609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff004074ee0 0x7ff00419a970 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7feff000e9f0 tx=0x7feff000eeb0 comp rx=0 tx=0).stop 2024-09-19T05:12:04.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.534+0000 7ff009c00640 1 -- 172.21.15.43:0/885037609 shutdown_connections 2024-09-19T05:12:04.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.534+0000 7ff009c00640 1 --2- 172.21.15.43:0/885037609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff00410e260 0x7ff00419ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:04.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.534+0000 7ff009c00640 1 --2- 172.21.15.43:0/885037609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff004074ee0 0x7ff00419a970 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:04.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.534+0000 7ff009c00640 1 --2- 172.21.15.43:0/885037609 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fefe80a3430 0x7fefe80a58f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:04.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.534+0000 7ff009c00640 1 --2- 172.21.15.43:0/885037609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0040745c0 0x7ff00419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:04.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.534+0000 7ff009c00640 1 -- 172.21.15.43:0/885037609 >> 172.21.15.43:0/885037609 conn(0x7ff0040fd740 msgr2=0x7ff0040fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:04.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.534+0000 7ff009c00640 1 -- 172.21.15.43:0/885037609 shutdown_connections 2024-09-19T05:12:04.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:04.534+0000 7ff009c00640 1 -- 172.21.15.43:0/885037609 wait complete. 2024-09-19T05:12:05.373 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T05:12:05.752 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:12:06.182 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:06 smithi043 ceph-mon[30800]: pgmap v228: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-09-19T05:12:06.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.296+0000 7ff71e600640 1 -- 172.21.15.43:0/3089659645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7181079c0 msgr2=0x7ff718107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:06.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.296+0000 7ff71e600640 1 --2- 172.21.15.43:0/3089659645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7181079c0 0x7ff718107da0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff708009930 tx=0x7ff70802a440 comp rx=0 tx=0).stop 2024-09-19T05:12:06.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.296+0000 7ff71e600640 1 -- 172.21.15.43:0/3089659645 shutdown_connections 2024-09-19T05:12:06.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.296+0000 7ff71e600640 1 --2- 172.21.15.43:0/3089659645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff718102360 0x7ff71810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:06.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.296+0000 7ff71e600640 1 --2- 172.21.15.43:0/3089659645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7181019c0 0x7ff718101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:06.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.296+0000 7ff71e600640 1 --2- 172.21.15.43:0/3089659645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7181079c0 0x7ff718107da0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:06.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.296+0000 7ff71e600640 1 -- 172.21.15.43:0/3089659645 >> 172.21.15.43:0/3089659645 conn(0x7ff7180fd740 msgr2=0x7ff7180ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:06.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.296+0000 7ff71e600640 1 -- 172.21.15.43:0/3089659645 shutdown_connections 2024-09-19T05:12:06.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.296+0000 7ff71e600640 1 -- 172.21.15.43:0/3089659645 wait complete. 2024-09-19T05:12:06.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.297+0000 7ff71e600640 1 Processor -- start 2024-09-19T05:12:06.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.298+0000 7ff71e600640 1 -- start start 2024-09-19T05:12:06.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.298+0000 7ff71e600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff7181019c0 0x7ff7180744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:06.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.298+0000 7ff71e600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff718102360 0x7ff718074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:06.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.298+0000 7ff71e600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7181079c0 0x7ff718074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:06.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.298+0000 7ff71e600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff718111e10 con 0x7ff718102360 2024-09-19T05:12:06.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.298+0000 7ff71e600640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff718111c90 con 0x7ff7181079c0 2024-09-19T05:12:06.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.298+0000 7ff71e600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff718111f90 con 0x7ff7181019c0 2024-09-19T05:12:06.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.298+0000 7ff70fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff718102360 0x7ff718074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:06.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.298+0000 7ff70fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff718102360 0x7ff718074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34688/0 (socket says 172.21.15.43:34688) 2024-09-19T05:12:06.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.298+0000 7ff70fe00640 1 -- 172.21.15.43:0/888865074 learned_addr learned my addr 172.21.15.43:0/888865074 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:06.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.298+0000 7ff71dc00640 1 --2- 172.21.15.43:0/888865074 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7181079c0 0x7ff718074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:06.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.298+0000 7ff71d200640 1 --2- 172.21.15.43:0/888865074 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff7181019c0 0x7ff7180744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:06.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.299+0000 7ff70fe00640 1 -- 172.21.15.43:0/888865074 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff7181019c0 msgr2=0x7ff7180744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:06.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.299+0000 7ff70fe00640 1 --2- 172.21.15.43:0/888865074 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff7181019c0 0x7ff7180744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:06.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.299+0000 7ff70fe00640 1 -- 172.21.15.43:0/888865074 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7181079c0 msgr2=0x7ff718074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:06.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.299+0000 7ff70fe00640 1 --2- 172.21.15.43:0/888865074 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7181079c0 0x7ff718074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:06.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.299+0000 7ff70fe00640 1 -- 172.21.15.43:0/888865074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7181a66d0 con 0x7ff718102360 2024-09-19T05:12:06.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.299+0000 7ff70fe00640 1 --2- 172.21.15.43:0/888865074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff718102360 0x7ff718074a30 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff70000c920 tx=0x7ff70000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:06.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.299+0000 7ff716000640 1 -- 172.21.15.43:0/888865074 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff700007c70 con 0x7ff718102360 2024-09-19T05:12:06.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.299+0000 7ff716000640 1 -- 172.21.15.43:0/888865074 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff700007e10 con 0x7ff718102360 2024-09-19T05:12:06.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.300+0000 7ff71e600640 1 -- 172.21.15.43:0/888865074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7181a69c0 con 0x7ff718102360 2024-09-19T05:12:06.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.300+0000 7ff716000640 1 -- 172.21.15.43:0/888865074 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff700018890 con 0x7ff718102360 2024-09-19T05:12:06.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.300+0000 7ff71e600640 1 -- 172.21.15.43:0/888865074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7181a6f00 con 0x7ff718102360 2024-09-19T05:12:06.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.303+0000 7ff722640740 1 Processor -- start 2024-09-19T05:12:06.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.303+0000 7ff716000640 1 -- 172.21.15.43:0/888865074 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ff700018a70 con 0x7ff718102360 2024-09-19T05:12:06.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.303+0000 7ff722640740 1 -- start start 2024-09-19T05:12:06.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.303+0000 7ff722640740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6b41eee40 0x55b6b41ef220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:06.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.303+0000 7ff722640740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6b41ef760 0x55b6b41f9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:06.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.303+0000 7ff722640740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6b41fa020 0x55b6b41fc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:06.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.303+0000 7ff722640740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b6b41553e0 con 0x55b6b41fa020 2024-09-19T05:12:06.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.303+0000 7ff722640740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b6b40fcd80 con 0x55b6b41eee40 2024-09-19T05:12:06.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.303+0000 7ff722640740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b6b41513a0 con 0x55b6b41ef760 2024-09-19T05:12:06.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.304+0000 7ff71dc00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6b41eee40 0x55b6b41ef220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:06.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.304+0000 7ff71dc00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6b41eee40 0x55b6b41ef220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:54924/0 (socket says 172.21.15.43:54924) 2024-09-19T05:12:06.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.304+0000 7ff71dc00640 1 -- 172.21.15.43:0/4049008853 learned_addr learned my addr 172.21.15.43:0/4049008853 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:06.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.304+0000 7ff71d200640 1 --2- 172.21.15.43:0/4049008853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6b41ef760 0x55b6b41f9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:06.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.304+0000 7ff70fe00640 1 --2- 172.21.15.43:0/4049008853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6b41fa020 0x55b6b41fc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:06.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.304+0000 7ff71dc00640 1 -- 172.21.15.43:0/4049008853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6b41ef760 msgr2=0x55b6b41f9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:06.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.304+0000 7ff71dc00640 1 --2- 172.21.15.43:0/4049008853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6b41ef760 0x55b6b41f9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:06.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.304+0000 7ff71dc00640 1 -- 172.21.15.43:0/4049008853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6b41fa020 msgr2=0x55b6b41fc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:06.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.304+0000 7ff71dc00640 1 --2- 172.21.15.43:0/4049008853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6b41fa020 0x55b6b41fc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:06.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.304+0000 7ff71dc00640 1 -- 172.21.15.43:0/4049008853 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6b411bae0 con 0x55b6b41eee40 2024-09-19T05:12:06.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.304+0000 7ff71dc00640 1 --2- 172.21.15.43:0/4049008853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6b41eee40 0x55b6b41ef220 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff710009f70 tx=0x7ff710010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37d1ab2d6d348035 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:06.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.305+0000 7ff70e000640 1 -- 172.21.15.43:0/4049008853 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff710016070 con 0x55b6b41eee40 2024-09-19T05:12:06.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.305+0000 7ff70e000640 1 -- 172.21.15.43:0/4049008853 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff710004d20 con 0x55b6b41eee40 2024-09-19T05:12:06.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.305+0000 7ff716000640 1 --2- 172.21.15.43:0/888865074 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff6fc0a33b0 0x7ff6fc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:06.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.305+0000 7ff70e000640 1 -- 172.21.15.43:0/4049008853 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7100128f0 con 0x55b6b41eee40 2024-09-19T05:12:06.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.305+0000 7ff716000640 1 -- 172.21.15.43:0/888865074 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7ff7000cfe30 con 0x7ff718102360 2024-09-19T05:12:06.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.305+0000 7ff71d200640 1 --2- 172.21.15.43:0/888865074 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff6fc0a33b0 0x7ff6fc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:06.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.305+0000 7ff722640740 1 -- 172.21.15.43:0/4049008853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6b41eee40 msgr2=0x55b6b41ef220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:06.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.305+0000 7ff722640740 1 --2- 172.21.15.43:0/4049008853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6b41eee40 0x55b6b41ef220 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff710009f70 tx=0x7ff710010800 comp rx=0 tx=0).stop 2024-09-19T05:12:06.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.306+0000 7ff722640740 1 -- 172.21.15.43:0/4049008853 shutdown_connections 2024-09-19T05:12:06.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.306+0000 7ff722640740 1 --2- 172.21.15.43:0/4049008853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6b41fa020 0x55b6b41fc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:06.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.306+0000 7ff722640740 1 --2- 172.21.15.43:0/4049008853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6b41ef760 0x55b6b41f9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:06.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.306+0000 7ff722640740 1 --2- 172.21.15.43:0/4049008853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6b41eee40 0x55b6b41ef220 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:06.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.306+0000 7ff722640740 1 -- 172.21.15.43:0/4049008853 >> 172.21.15.43:0/4049008853 conn(0x55b6b41eb3d0 msgr2=0x55b6b41eb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:06.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.306+0000 7ff722640740 1 -- 172.21.15.43:0/4049008853 shutdown_connections 2024-09-19T05:12:06.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.306+0000 7ff722640740 1 -- 172.21.15.43:0/4049008853 wait complete. 2024-09-19T05:12:06.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.306+0000 7ff71d200640 1 --2- 172.21.15.43:0/888865074 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff6fc0a33b0 0x7ff6fc0a5870 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff708006010 tx=0x7ff708000bc0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:06.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.306+0000 7ff722640740 1 -- start start 2024-09-19T05:12:06.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.307+0000 7ff722640740 1 Processor -- start 2024-09-19T05:12:06.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.307+0000 7ff722640740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6b41eee40 0x55b6b41f6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:06.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.307+0000 7ff722640740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6b41ef760 0x55b6b41f7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:06.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.307+0000 7ff722640740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6b41fa020 0x55b6b41f7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:06.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.307+0000 7ff722640740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b6b41fcbf0 con 0x55b6b41ef760 2024-09-19T05:12:06.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.307+0000 7ff722640740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b6b41100f0 con 0x55b6b41fa020 2024-09-19T05:12:06.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.307+0000 7ff722640740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b6b41fcd70 con 0x55b6b41eee40 2024-09-19T05:12:06.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.307+0000 7ff722640740 10 client.0 _ll_register_callbacks cb 0x55b6b40f7bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:12:06.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.307+0000 7ff71d200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6b41eee40 0x55b6b41f6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:06.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.307+0000 7ff70fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6b41ef760 0x55b6b41f7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:06.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.307+0000 7ff71d200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6b41eee40 0x55b6b41f6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:50202/0 (socket says 172.21.15.43:50202) 2024-09-19T05:12:06.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.307+0000 7ff71d200640 1 -- 172.21.15.43:0/4073270439 learned_addr learned my addr 172.21.15.43:0/4073270439 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:06.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.307+0000 7ff71dc00640 1 --2- 172.21.15.43:0/4073270439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6b41fa020 0x55b6b41f7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:06.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.308+0000 7ff71d200640 1 -- 172.21.15.43:0/4073270439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6b41fa020 msgr2=0x55b6b41f7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:06.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.308+0000 7ff71d200640 1 --2- 172.21.15.43:0/4073270439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6b41fa020 0x55b6b41f7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:06.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.308+0000 7ff71d200640 1 -- 172.21.15.43:0/4073270439 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6b41ef760 msgr2=0x55b6b41f7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:06.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.308+0000 7ff71d200640 1 --2- 172.21.15.43:0/4073270439 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6b41ef760 0x55b6b41f7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:06.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.308+0000 7ff71d200640 1 -- 172.21.15.43:0/4073270439 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6b3ec63c0 con 0x55b6b41eee40 2024-09-19T05:12:06.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.308+0000 7ff71dc00640 1 --2- 172.21.15.43:0/4073270439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6b41fa020 0x55b6b41f7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:12:06.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.308+0000 7ff71d200640 1 --2- 172.21.15.43:0/4073270439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6b41eee40 0x55b6b41f6ed0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff708009930 tx=0x7ff708009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29f6b118ddefaa92 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:06.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.308+0000 7ff706a00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:12:06.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.309+0000 7ff706a00640 1 -- 172.21.15.43:0/4073270439 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70804e070 con 0x55b6b41eee40 2024-09-19T05:12:06.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.309+0000 7ff722640740 1 -- 172.21.15.43:0/4073270439 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff710004d20 con 0x55b6b41eee40 2024-09-19T05:12:06.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.309+0000 7ff706a00640 1 -- 172.21.15.43:0/4073270439 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7ff708004dd0 con 0x55b6b41eee40 2024-09-19T05:12:06.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.309+0000 7ff706a00640 1 -- 172.21.15.43:0/4073270439 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff70805a050 con 0x55b6b41eee40 2024-09-19T05:12:06.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.309+0000 7ff722640740 10 client.26827 fetch_fsmap learned FSMap version 29 2024-09-19T05:12:06.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.309+0000 7ff722640740 1 -- 172.21.15.43:0/4073270439 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b6b4208c50 con 0x55b6b41eee40 2024-09-19T05:12:06.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.309+0000 7ff706a00640 1 -- 172.21.15.43:0/4073270439 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff710004d20 con 0x55b6b41eee40 2024-09-19T05:12:06.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.310+0000 7ff706a00640 1 -- 172.21.15.43:0/4073270439 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff708003950 con 0x55b6b41eee40 2024-09-19T05:12:06.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.310+0000 7ff722640740 10 client.26827 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:12:06.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.310+0000 7ff722640740 10 client.26827 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T05:12:06.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.310+0000 7ff722640740 20 client.26827 populate_metadata read hostname 'smithi043' 2024-09-19T05:12:06.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.310+0000 7ff722640740 1 --2- 172.21.15.43:0/4073270439 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55b6b420a060 0x55b6b420c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:06.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.310+0000 7ff722640740 4 client.26827 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:12:06.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.310+0000 7ff722640740 1 -- 172.21.15.43:0/4073270439 --> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6b4162950 con 0x55b6b420a060 2024-09-19T05:12:06.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.310+0000 7ff70fe00640 1 --2- 172.21.15.43:0/4073270439 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55b6b420a060 0x55b6b420c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:06.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.311+0000 7ff70fe00640 1 --2- 172.21.15.43:0/4073270439 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55b6b420a060 0x55b6b420c4d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=11063e329da22536 server_cookie=9b467353ef63bd19 in_seq=0 out_seq=0 2024-09-19T05:12:06.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.311+0000 7ff706a00640 10 client.26827 ms_handle_connect on v2:172.21.15.151:6838/3794104313 2024-09-19T05:12:06.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.311+0000 7ff706a00640 1 -- 172.21.15.43:0/4073270439 <== mds.2 v2:172.21.15.151:6838/3794104313 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b6b4162950 con 0x55b6b420a060 2024-09-19T05:12:06.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.311+0000 7ff706a00640 10 client.26827 handle_command_reply: tid=0 2024-09-19T05:12:06.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.311+0000 7ff722640740 1 client.26827 shutdown 2024-09-19T05:12:06.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.312+0000 7ff722640740 1 -- 172.21.15.43:0/4073270439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6b41eee40 msgr2=0x55b6b41f6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:06.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.312+0000 7ff722640740 1 --2- 172.21.15.43:0/4073270439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6b41eee40 0x55b6b41f6ed0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff708009930 tx=0x7ff708009260 comp rx=0 tx=0).stop 2024-09-19T05:12:06.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.312+0000 7ff722640740 1 -- 172.21.15.43:0/4073270439 shutdown_connections 2024-09-19T05:12:06.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.312+0000 7ff722640740 1 --2- 172.21.15.43:0/4073270439 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55b6b420a060 0x55b6b420c4d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:06.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.312+0000 7ff722640740 1 --2- 172.21.15.43:0/4073270439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6b41fa020 0x55b6b41f7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:06.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.312+0000 7ff722640740 1 --2- 172.21.15.43:0/4073270439 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6b41ef760 0x55b6b41f7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:06.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.312+0000 7ff722640740 1 --2- 172.21.15.43:0/4073270439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6b41eee40 0x55b6b41f6ed0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:06.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.312+0000 7ff706a00640 0 client.26827 ms_handle_reset on v2:172.21.15.151:6838/3794104313 2024-09-19T05:12:06.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.312+0000 7ff722640740 1 -- 172.21.15.43:0/4073270439 >> 172.21.15.43:0/4073270439 conn(0x55b6b41eb3d0 msgr2=0x55b6b41f10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:06.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.312+0000 7ff722640740 1 -- 172.21.15.43:0/4073270439 shutdown_connections 2024-09-19T05:12:06.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.312+0000 7ff722640740 1 -- 172.21.15.43:0/4073270439 wait complete. 2024-09-19T05:12:06.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.312+0000 7ff722640740 20 client.26827 trim_cache size 0 max 16384 2024-09-19T05:12:06.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.326+0000 7ff722640740 1 Processor -- start 2024-09-19T05:12:06.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.326+0000 7ff722640740 1 -- start start 2024-09-19T05:12:06.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.326+0000 7ff722640740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6b41eee40 0x55b6b4211da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:06.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.326+0000 7ff722640740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6b41ef760 0x55b6b42122e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:06.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.326+0000 7ff722640740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6b41fa020 0x55b6b41f4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:06.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.326+0000 7ff722640740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff7181af850 con 0x55b6b41ef760 2024-09-19T05:12:06.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.326+0000 7ff722640740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff7181b1390 con 0x55b6b41fa020 2024-09-19T05:12:06.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.326+0000 7ff722640740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff7181aff20 con 0x55b6b41eee40 2024-09-19T05:12:06.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.326+0000 7ff70fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6b41ef760 0x55b6b42122e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:06.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.326+0000 7ff71d200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6b41eee40 0x55b6b4211da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:06.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.326+0000 7ff70fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6b41ef760 0x55b6b42122e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34716/0 (socket says 172.21.15.43:34716) 2024-09-19T05:12:06.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.326+0000 7ff70fe00640 1 -- 172.21.15.43:0/2487310350 learned_addr learned my addr 172.21.15.43:0/2487310350 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:06.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.327+0000 7ff71dc00640 1 --2- 172.21.15.43:0/2487310350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6b41fa020 0x55b6b41f4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:06.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.327+0000 7ff71d200640 1 -- 172.21.15.43:0/2487310350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6b41fa020 msgr2=0x55b6b41f4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:06.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.327+0000 7ff71d200640 1 --2- 172.21.15.43:0/2487310350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6b41fa020 0x55b6b41f4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:06.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.327+0000 7ff71d200640 1 -- 172.21.15.43:0/2487310350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6b41ef760 msgr2=0x55b6b42122e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:06.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.327+0000 7ff71d200640 1 --2- 172.21.15.43:0/2487310350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6b41ef760 0x55b6b42122e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:06.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.327+0000 7ff71d200640 1 -- 172.21.15.43:0/2487310350 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6b41eb200 con 0x55b6b41eee40 2024-09-19T05:12:06.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.327+0000 7ff70fe00640 1 --2- 172.21.15.43:0/2487310350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6b41ef760 0x55b6b42122e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:12:06.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.327+0000 7ff71d200640 1 --2- 172.21.15.43:0/2487310350 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6b41eee40 0x55b6b4211da0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff708003ec0 tx=0x7ff7080043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2356b656965fa00d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:06.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.327+0000 7ff70d600640 1 -- 172.21.15.43:0/2487310350 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff708062070 con 0x55b6b41eee40 2024-09-19T05:12:06.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.328+0000 7ff70d600640 1 -- 172.21.15.43:0/2487310350 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff708004dd0 con 0x55b6b41eee40 2024-09-19T05:12:06.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.328+0000 7ff70d600640 1 -- 172.21.15.43:0/2487310350 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70804e070 con 0x55b6b41eee40 2024-09-19T05:12:06.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.328+0000 7ff722640740 1 -- 172.21.15.43:0/2487310350 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6b41eee40 msgr2=0x55b6b4211da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:06.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.328+0000 7ff722640740 1 --2- 172.21.15.43:0/2487310350 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6b41eee40 0x55b6b4211da0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff708003ec0 tx=0x7ff7080043b0 comp rx=0 tx=0).stop 2024-09-19T05:12:06.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.328+0000 7ff722640740 1 -- 172.21.15.43:0/2487310350 shutdown_connections 2024-09-19T05:12:06.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.328+0000 7ff722640740 1 --2- 172.21.15.43:0/2487310350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6b41fa020 0x55b6b41f4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:06.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.328+0000 7ff722640740 1 --2- 172.21.15.43:0/2487310350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6b41ef760 0x55b6b42122e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:06.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.328+0000 7ff722640740 1 --2- 172.21.15.43:0/2487310350 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6b41eee40 0x55b6b4211da0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:06.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.328+0000 7ff722640740 1 -- 172.21.15.43:0/2487310350 >> 172.21.15.43:0/2487310350 conn(0x55b6b41eb3d0 msgr2=0x55b6b41f8f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:06.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.328+0000 7ff722640740 1 -- 172.21.15.43:0/2487310350 shutdown_connections 2024-09-19T05:12:06.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.328+0000 7ff722640740 1 -- 172.21.15.43:0/2487310350 wait complete. 2024-09-19T05:12:06.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.329+0000 7ff722640740 1 -- start start 2024-09-19T05:12:06.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.329+0000 7ff722640740 1 Processor -- start 2024-09-19T05:12:06.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.329+0000 7ff722640740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6b41eee40 0x55b6b41fddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:06.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.329+0000 7ff722640740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6b41ef760 0x55b6b41fe300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:06.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.329+0000 7ff722640740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6b41fa020 0x55b6b41fe840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:06.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.329+0000 7ff722640740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff7181b0cc0 con 0x55b6b41eee40 2024-09-19T05:12:06.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.329+0000 7ff722640740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff7181b05f0 con 0x55b6b41fa020 2024-09-19T05:12:06.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.329+0000 7ff722640740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff7181ad850 con 0x55b6b41ef760 2024-09-19T05:12:06.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.330+0000 7ff722640740 10 client.0 _ll_register_callbacks cb 0x7ff7080040e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:12:06.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.330+0000 7ff71d200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6b41eee40 0x55b6b41fddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:06.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.330+0000 7ff71d200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6b41eee40 0x55b6b41fddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34728/0 (socket says 172.21.15.43:34728) 2024-09-19T05:12:06.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.330+0000 7ff71d200640 1 -- 172.21.15.43:0/3934869414 learned_addr learned my addr 172.21.15.43:0/3934869414 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:06.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.330+0000 7ff71dc00640 1 --2- 172.21.15.43:0/3934869414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6b41fa020 0x55b6b41fe840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:06.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.330+0000 7ff70fe00640 1 --2- 172.21.15.43:0/3934869414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6b41ef760 0x55b6b41fe300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:06.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.330+0000 7ff71d200640 1 -- 172.21.15.43:0/3934869414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6b41ef760 msgr2=0x55b6b41fe300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:06.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.330+0000 7ff71d200640 1 --2- 172.21.15.43:0/3934869414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6b41ef760 0x55b6b41fe300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:06.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.330+0000 7ff71d200640 1 -- 172.21.15.43:0/3934869414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6b41fa020 msgr2=0x55b6b41fe840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:06.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.330+0000 7ff71d200640 1 --2- 172.21.15.43:0/3934869414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6b41fa020 0x55b6b41fe840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:06.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.330+0000 7ff71d200640 1 -- 172.21.15.43:0/3934869414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6b427c630 con 0x55b6b41eee40 2024-09-19T05:12:06.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.331+0000 7ff71d200640 1 --2- 172.21.15.43:0/3934869414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6b41eee40 0x55b6b41fddc0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff708003d50 tx=0x7ff708003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ced3b1c61fa996 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:06.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.331+0000 7ff706a00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:12:06.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.331+0000 7ff706a00640 1 -- 172.21.15.43:0/3934869414 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70806b060 con 0x55b6b41eee40 2024-09-19T05:12:06.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.331+0000 7ff722640740 1 -- 172.21.15.43:0/3934869414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff708004dd0 con 0x55b6b41eee40 2024-09-19T05:12:06.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.331+0000 7ff706a00640 1 -- 172.21.15.43:0/3934869414 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7ff708003a40 con 0x55b6b41eee40 2024-09-19T05:12:06.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.331+0000 7ff706a00640 1 -- 172.21.15.43:0/3934869414 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff708062040 con 0x55b6b41eee40 2024-09-19T05:12:06.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.331+0000 7ff722640740 10 client.16872 fetch_fsmap learned FSMap version 29 2024-09-19T05:12:06.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.331+0000 7ff722640740 1 -- 172.21.15.43:0/3934869414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b6b427d910 con 0x55b6b41eee40 2024-09-19T05:12:06.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.331+0000 7ff706a00640 1 -- 172.21.15.43:0/3934869414 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff708004dd0 con 0x55b6b41eee40 2024-09-19T05:12:06.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.332+0000 7ff706a00640 1 -- 172.21.15.43:0/3934869414 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff708065070 con 0x55b6b41eee40 2024-09-19T05:12:06.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.332+0000 7ff722640740 10 client.16872 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:12:06.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.332+0000 7ff722640740 10 client.16872 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T05:12:06.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.332+0000 7ff722640740 20 client.16872 populate_metadata read hostname 'smithi043' 2024-09-19T05:12:06.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.332+0000 7ff722640740 1 --2- 172.21.15.43:0/3934869414 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55b6b427df70 0x55b6b4280360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:06.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.332+0000 7ff722640740 4 client.16872 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:12:06.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.332+0000 7ff722640740 1 -- 172.21.15.43:0/3934869414 --> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b6b415ce60 con 0x55b6b427df70 2024-09-19T05:12:06.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.332+0000 7ff70fe00640 1 --2- 172.21.15.43:0/3934869414 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55b6b427df70 0x55b6b4280360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:06.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.333+0000 7ff70fe00640 1 --2- 172.21.15.43:0/3934869414 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55b6b427df70 0x55b6b4280360 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d2072e3fe2c1866d server_cookie=ad16b60fa7bd4db7 in_seq=0 out_seq=0 2024-09-19T05:12:06.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.333+0000 7ff706a00640 10 client.16872 ms_handle_connect on v2:172.21.15.151:6838/3794104313 2024-09-19T05:12:06.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.333+0000 7ff706a00640 1 -- 172.21.15.43:0/3934869414 <== mds.2 v2:172.21.15.151:6838/3794104313 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b6b415ce60 con 0x55b6b427df70 2024-09-19T05:12:06.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.333+0000 7ff706a00640 10 client.16872 handle_command_reply: tid=0 2024-09-19T05:12:06.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.333+0000 7ff722640740 1 client.16872 shutdown 2024-09-19T05:12:06.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.333+0000 7ff722640740 1 -- 172.21.15.43:0/3934869414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6b41eee40 msgr2=0x55b6b41fddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:06.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.333+0000 7ff722640740 1 --2- 172.21.15.43:0/3934869414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6b41eee40 0x55b6b41fddc0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff708003d50 tx=0x7ff708003820 comp rx=0 tx=0).stop 2024-09-19T05:12:06.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.333+0000 7ff722640740 1 -- 172.21.15.43:0/3934869414 shutdown_connections 2024-09-19T05:12:06.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.333+0000 7ff722640740 1 --2- 172.21.15.43:0/3934869414 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55b6b427df70 0x55b6b4280360 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:06.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.333+0000 7ff722640740 1 --2- 172.21.15.43:0/3934869414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6b41fa020 0x55b6b41fe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:06.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.333+0000 7ff722640740 1 --2- 172.21.15.43:0/3934869414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6b41ef760 0x55b6b41fe300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:06.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.333+0000 7ff722640740 1 --2- 172.21.15.43:0/3934869414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b6b41eee40 0x55b6b41fddc0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:06.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.333+0000 7ff706a00640 0 client.16872 ms_handle_reset on v2:172.21.15.151:6838/3794104313 2024-09-19T05:12:06.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.333+0000 7ff722640740 1 -- 172.21.15.43:0/3934869414 >> 172.21.15.43:0/3934869414 conn(0x55b6b41eb3d0 msgr2=0x55b6b41f0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:06.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.334+0000 7ff722640740 1 -- 172.21.15.43:0/3934869414 shutdown_connections 2024-09-19T05:12:06.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.334+0000 7ff722640740 1 -- 172.21.15.43:0/3934869414 wait complete. 2024-09-19T05:12:06.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.334+0000 7ff722640740 20 client.16872 trim_cache size 0 max 16384 2024-09-19T05:12:06.343 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:12:06.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.334+0000 7ff71e600640 1 -- 172.21.15.43:0/888865074 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff6fc0a33b0 msgr2=0x7ff6fc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:06.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.334+0000 7ff71e600640 1 --2- 172.21.15.43:0/888865074 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff6fc0a33b0 0x7ff6fc0a5870 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff708006010 tx=0x7ff708000bc0 comp rx=0 tx=0).stop 2024-09-19T05:12:06.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.334+0000 7ff71e600640 1 -- 172.21.15.43:0/888865074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff718102360 msgr2=0x7ff718074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:06.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.334+0000 7ff71e600640 1 --2- 172.21.15.43:0/888865074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff718102360 0x7ff718074a30 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff70000c920 tx=0x7ff70000cde0 comp rx=0 tx=0).stop 2024-09-19T05:12:06.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.334+0000 7ff71e600640 1 -- 172.21.15.43:0/888865074 shutdown_connections 2024-09-19T05:12:06.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.334+0000 7ff71e600640 1 --2- 172.21.15.43:0/888865074 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff6fc0a33b0 0x7ff6fc0a5870 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:06.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.334+0000 7ff71e600640 1 --2- 172.21.15.43:0/888865074 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7181079c0 0x7ff718074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:06.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.334+0000 7ff71e600640 1 --2- 172.21.15.43:0/888865074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff718102360 0x7ff718074a30 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:06.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.334+0000 7ff71e600640 1 --2- 172.21.15.43:0/888865074 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff7181019c0 0x7ff7180744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:06.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.334+0000 7ff71e600640 1 -- 172.21.15.43:0/888865074 >> 172.21.15.43:0/888865074 conn(0x7ff7180fd740 msgr2=0x7ff7180fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:06.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.334+0000 7ff71e600640 1 -- 172.21.15.43:0/888865074 shutdown_connections 2024-09-19T05:12:06.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:06.335+0000 7ff71e600640 1 -- 172.21.15.43:0/888865074 wait complete. 2024-09-19T05:12:06.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:06 smithi151 ceph-mon[35425]: pgmap v228: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-09-19T05:12:06.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:06 smithi096 ceph-mon[35085]: pgmap v228: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-09-19T05:12:07.078 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T05:12:07.456 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:12:08.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.006+0000 7f0479600640 1 -- 172.21.15.43:0/952499671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0474074eb0 msgr2=0x7f0474112490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:08.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.006+0000 7f0479600640 1 --2- 172.21.15.43:0/952499671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0474074eb0 0x7f0474112490 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f045c009860 tx=0x7f045c02a3d0 comp rx=0 tx=0).stop 2024-09-19T05:12:08.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.006+0000 7f0479600640 1 -- 172.21.15.43:0/952499671 shutdown_connections 2024-09-19T05:12:08.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.006+0000 7f0479600640 1 --2- 172.21.15.43:0/952499671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04741129d0 0x7f0474114dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:08.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.006+0000 7f0479600640 1 --2- 172.21.15.43:0/952499671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0474074eb0 0x7f0474112490 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:08.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.006+0000 7f0479600640 1 --2- 172.21.15.43:0/952499671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0474074590 0x7f0474074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:08.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.006+0000 7f0479600640 1 -- 172.21.15.43:0/952499671 >> 172.21.15.43:0/952499671 conn(0x7f04740fd6e0 msgr2=0x7f04740ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:08.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.006+0000 7f0479600640 1 -- 172.21.15.43:0/952499671 shutdown_connections 2024-09-19T05:12:08.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.006+0000 7f0479600640 1 -- 172.21.15.43:0/952499671 wait complete. 2024-09-19T05:12:08.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.007+0000 7f0479600640 1 Processor -- start 2024-09-19T05:12:08.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.007+0000 7f0479600640 1 -- start start 2024-09-19T05:12:08.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.008+0000 7f0479600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0474074590 0x7f04741a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:08.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.008+0000 7f0479600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0474074eb0 0x7f04741a4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:08.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.008+0000 7f0479600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04741129d0 0x7f047419e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:08.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.008+0000 7f0479600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f04741183f0 con 0x7f0474074590 2024-09-19T05:12:08.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.008+0000 7f0479600640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0474118270 con 0x7f0474074eb0 2024-09-19T05:12:08.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.008+0000 7f0479600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0474118570 con 0x7f04741129d0 2024-09-19T05:12:08.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.008+0000 7f0473e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0474074590 0x7f04741a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:08.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.008+0000 7f0478c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04741129d0 0x7f047419e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:08.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.008+0000 7f0473e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0474074590 0x7f04741a4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34744/0 (socket says 172.21.15.43:34744) 2024-09-19T05:12:08.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.008+0000 7f0473e00640 1 -- 172.21.15.43:0/2139259326 learned_addr learned my addr 172.21.15.43:0/2139259326 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:08.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.008+0000 7f046be00640 1 --2- 172.21.15.43:0/2139259326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0474074eb0 0x7f04741a4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:08.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.008+0000 7f0478c00640 1 -- 172.21.15.43:0/2139259326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0474074eb0 msgr2=0x7f04741a4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:08.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.008+0000 7f0478c00640 1 --2- 172.21.15.43:0/2139259326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0474074eb0 0x7f04741a4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:08.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.008+0000 7f0478c00640 1 -- 172.21.15.43:0/2139259326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0474074590 msgr2=0x7f04741a4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:08.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.008+0000 7f0478c00640 1 --2- 172.21.15.43:0/2139259326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0474074590 0x7f04741a4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:08.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.008+0000 7f0478c00640 1 -- 172.21.15.43:0/2139259326 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f047419ef50 con 0x7f04741129d0 2024-09-19T05:12:08.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.008+0000 7f0473e00640 1 --2- 172.21.15.43:0/2139259326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0474074590 0x7f04741a4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:12:08.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.009+0000 7f0478c00640 1 --2- 172.21.15.43:0/2139259326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04741129d0 0x7f047419e6e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f046c005fd0 tx=0x7f046c00c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:08.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.009+0000 7f0471600640 1 -- 172.21.15.43:0/2139259326 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046c003970 con 0x7f04741129d0 2024-09-19T05:12:08.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.009+0000 7f0479600640 1 -- 172.21.15.43:0/2139259326 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f047419f240 con 0x7f04741129d0 2024-09-19T05:12:08.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.009+0000 7f0471600640 1 -- 172.21.15.43:0/2139259326 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f046c004020 con 0x7f04741129d0 2024-09-19T05:12:08.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.009+0000 7f0471600640 1 -- 172.21.15.43:0/2139259326 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046c00ba50 con 0x7f04741129d0 2024-09-19T05:12:08.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.009+0000 7f0479600640 1 -- 172.21.15.43:0/2139259326 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04741ab590 con 0x7f04741129d0 2024-09-19T05:12:08.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.012+0000 7f047d652740 1 Processor -- start 2024-09-19T05:12:08.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.013+0000 7f0471600640 1 -- 172.21.15.43:0/2139259326 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f046c01d020 con 0x7f04741129d0 2024-09-19T05:12:08.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.013+0000 7f047d652740 1 -- start start 2024-09-19T05:12:08.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.013+0000 7f047d652740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff15222e40 0x55ff15223220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:08.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.013+0000 7f047d652740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff15223760 0x55ff1522dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:08.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.013+0000 7f047d652740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ff1522e020 0x55ff15230410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:08.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.013+0000 7f047d652740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ff151893e0 con 0x55ff15222e40 2024-09-19T05:12:08.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.013+0000 7f047d652740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ff15130d80 con 0x55ff15223760 2024-09-19T05:12:08.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.013+0000 7f047d652740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ff151853a0 con 0x55ff1522e020 2024-09-19T05:12:08.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.014+0000 7f046be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff15222e40 0x55ff15223220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:08.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.014+0000 7f046be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff15222e40 0x55ff15223220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34752/0 (socket says 172.21.15.43:34752) 2024-09-19T05:12:08.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.014+0000 7f046be00640 1 -- 172.21.15.43:0/3556449441 learned_addr learned my addr 172.21.15.43:0/3556449441 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:08.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.014+0000 7f0473e00640 1 --2- 172.21.15.43:0/3556449441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff15223760 0x55ff1522dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:08.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.014+0000 7f046be00640 1 --2- 172.21.15.43:0/3556449441 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ff1522e020 0x55ff15230410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:08.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.014+0000 7f046be00640 1 -- 172.21.15.43:0/3556449441 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ff1522e020 msgr2=0x55ff15230410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:08.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.014+0000 7f046be00640 1 --2- 172.21.15.43:0/3556449441 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ff1522e020 0x55ff15230410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:08.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.014+0000 7f046be00640 1 -- 172.21.15.43:0/3556449441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff15223760 msgr2=0x55ff1522dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:08.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.014+0000 7f046be00640 1 --2- 172.21.15.43:0/3556449441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff15223760 0x55ff1522dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:08.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.014+0000 7f046be00640 1 -- 172.21.15.43:0/3556449441 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff1514fae0 con 0x55ff15222e40 2024-09-19T05:12:08.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.014+0000 7f0471600640 1 --2- 172.21.15.43:0/2139259326 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f04580a3430 0x7f04580a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:08.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.014+0000 7f046be00640 1 --2- 172.21.15.43:0/3556449441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff15222e40 0x55ff15223220 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f045c032ff0 tx=0x7f045c032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a6f84a1554eb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:08.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.015+0000 7f0471600640 1 -- 172.21.15.43:0/2139259326 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f046c0cfa20 con 0x7f04741129d0 2024-09-19T05:12:08.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.015+0000 7f0473e00640 1 --2- 172.21.15.43:0/2139259326 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f04580a3430 0x7f04580a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:08.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.015+0000 7f0469200640 1 -- 172.21.15.43:0/3556449441 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045c004230 con 0x55ff15222e40 2024-09-19T05:12:08.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.015+0000 7f0469200640 1 -- 172.21.15.43:0/3556449441 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f045c047070 con 0x55ff15222e40 2024-09-19T05:12:08.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.015+0000 7f0469200640 1 -- 172.21.15.43:0/3556449441 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045c005620 con 0x55ff15222e40 2024-09-19T05:12:08.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.016+0000 7f047d652740 1 -- 172.21.15.43:0/3556449441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff15222e40 msgr2=0x55ff15223220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:08.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.016+0000 7f047d652740 1 --2- 172.21.15.43:0/3556449441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff15222e40 0x55ff15223220 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f045c032ff0 tx=0x7f045c032f60 comp rx=0 tx=0).stop 2024-09-19T05:12:08.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.016+0000 7f0473e00640 1 --2- 172.21.15.43:0/2139259326 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f04580a3430 0x7f04580a58f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0464009fd0 tx=0x7f0464005f90 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:08.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.016+0000 7f047d652740 1 -- 172.21.15.43:0/3556449441 shutdown_connections 2024-09-19T05:12:08.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.016+0000 7f047d652740 1 --2- 172.21.15.43:0/3556449441 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ff1522e020 0x55ff15230410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:08.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.016+0000 7f047d652740 1 --2- 172.21.15.43:0/3556449441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff15223760 0x55ff1522dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:08.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.016+0000 7f047d652740 1 --2- 172.21.15.43:0/3556449441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff15222e40 0x55ff15223220 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:08.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.016+0000 7f047d652740 1 -- 172.21.15.43:0/3556449441 >> 172.21.15.43:0/3556449441 conn(0x55ff1521f3d0 msgr2=0x55ff1521f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:08.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.016+0000 7f047d652740 1 -- 172.21.15.43:0/3556449441 shutdown_connections 2024-09-19T05:12:08.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.016+0000 7f047d652740 1 -- 172.21.15.43:0/3556449441 wait complete. 2024-09-19T05:12:08.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.016+0000 7f047d652740 1 -- start start 2024-09-19T05:12:08.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.017+0000 7f047d652740 1 Processor -- start 2024-09-19T05:12:08.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.017+0000 7f047d652740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ff15222e40 0x55ff1522af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:08.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.017+0000 7f047d652740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff15223760 0x55ff1522b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:08.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.017+0000 7f047d652740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff1522e020 0x55ff1522b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:08.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.017+0000 7f047d652740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ff15230bf0 con 0x55ff1522e020 2024-09-19T05:12:08.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.017+0000 7f047d652740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ff151440f0 con 0x55ff15223760 2024-09-19T05:12:08.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.017+0000 7f047d652740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ff15230d70 con 0x55ff15222e40 2024-09-19T05:12:08.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.017+0000 7f047d652740 10 client.0 _ll_register_callbacks cb 0x55ff1512bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:12:08.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.017+0000 7f0478c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff1522e020 0x55ff1522b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:08.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.017+0000 7f0478c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff1522e020 0x55ff1522b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34764/0 (socket says 172.21.15.43:34764) 2024-09-19T05:12:08.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.017+0000 7f0478c00640 1 -- 172.21.15.43:0/865588354 learned_addr learned my addr 172.21.15.43:0/865588354 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:08.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.017+0000 7f046be00640 1 --2- 172.21.15.43:0/865588354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff15223760 0x55ff1522b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:08.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.018+0000 7f0473e00640 1 --2- 172.21.15.43:0/865588354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ff15222e40 0x55ff1522af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:08.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.018+0000 7f0478c00640 1 -- 172.21.15.43:0/865588354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ff15222e40 msgr2=0x55ff1522af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:08.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.018+0000 7f0478c00640 1 --2- 172.21.15.43:0/865588354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ff15222e40 0x55ff1522af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:08.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.018+0000 7f0478c00640 1 -- 172.21.15.43:0/865588354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff15223760 msgr2=0x55ff1522b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:08.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.018+0000 7f0478c00640 1 --2- 172.21.15.43:0/865588354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff15223760 0x55ff1522b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:08.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.018+0000 7f0478c00640 1 -- 172.21.15.43:0/865588354 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff14efa3c0 con 0x55ff1522e020 2024-09-19T05:12:08.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.018+0000 7f046be00640 1 --2- 172.21.15.43:0/865588354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff15223760 0x55ff1522b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:12:08.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.018+0000 7f0478c00640 1 --2- 172.21.15.43:0/865588354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff1522e020 0x55ff1522b9d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f046c09b990 tx=0x7f046c09c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb742e808386cff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:08.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.018+0000 7f0461600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:12:08.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.018+0000 7f0461600640 1 -- 172.21.15.43:0/865588354 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046c0a4900 con 0x55ff1522e020 2024-09-19T05:12:08.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.018+0000 7f0461600640 1 -- 172.21.15.43:0/865588354 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f046c003b10 con 0x55ff1522e020 2024-09-19T05:12:08.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.018+0000 7f047d652740 1 -- 172.21.15.43:0/865588354 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f045c047070 con 0x55ff1522e020 2024-09-19T05:12:08.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.019+0000 7f0461600640 1 -- 172.21.15.43:0/865588354 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f046c0041d0 con 0x55ff1522e020 2024-09-19T05:12:08.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.019+0000 7f047d652740 10 client.16884 fetch_fsmap learned FSMap version 29 2024-09-19T05:12:08.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.019+0000 7f047d652740 1 -- 172.21.15.43:0/865588354 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ff1523cce0 con 0x55ff1522e020 2024-09-19T05:12:08.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.019+0000 7f0461600640 1 -- 172.21.15.43:0/865588354 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f045c047070 con 0x55ff1522e020 2024-09-19T05:12:08.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.019+0000 7f0461600640 1 -- 172.21.15.43:0/865588354 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f046c00b040 con 0x55ff1522e020 2024-09-19T05:12:08.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.019+0000 7f047d652740 10 client.16884 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:12:08.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.019+0000 7f047d652740 10 client.16884 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-19T05:12:08.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.019+0000 7f047d652740 20 client.16884 populate_metadata read hostname 'smithi043' 2024-09-19T05:12:08.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.019+0000 7f047d652740 1 --2- 172.21.15.43:0/865588354 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55ff1523e0f0 0x55ff15240520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:08.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.019+0000 7f047d652740 4 client.16884 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:12:08.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.019+0000 7f047d652740 1 -- 172.21.15.43:0/865588354 --> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff15196950 con 0x55ff1523e0f0 2024-09-19T05:12:08.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.020+0000 7f0473e00640 1 --2- 172.21.15.43:0/865588354 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55ff1523e0f0 0x55ff15240520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:08.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.020+0000 7f0473e00640 1 --2- 172.21.15.43:0/865588354 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55ff1523e0f0 0x55ff15240520 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9e9e69b5e9e51edb server_cookie=40eac60a6735a613 in_seq=0 out_seq=0 2024-09-19T05:12:08.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.020+0000 7f0461600640 10 client.16884 ms_handle_connect on v2:172.21.15.43:6838/3404555030 2024-09-19T05:12:08.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.021+0000 7f0461600640 1 -- 172.21.15.43:0/865588354 <== mds.1 v2:172.21.15.43:6838/3404555030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ff15196950 con 0x55ff1523e0f0 2024-09-19T05:12:08.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.021+0000 7f0461600640 10 client.16884 handle_command_reply: tid=0 2024-09-19T05:12:08.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.021+0000 7f047d652740 1 client.16884 shutdown 2024-09-19T05:12:08.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.021+0000 7f047d652740 1 -- 172.21.15.43:0/865588354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff1522e020 msgr2=0x55ff1522b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:08.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.021+0000 7f047d652740 1 --2- 172.21.15.43:0/865588354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff1522e020 0x55ff1522b9d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f046c09b990 tx=0x7f046c09c390 comp rx=0 tx=0).stop 2024-09-19T05:12:08.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.021+0000 7f047d652740 1 -- 172.21.15.43:0/865588354 shutdown_connections 2024-09-19T05:12:08.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.021+0000 7f047d652740 1 --2- 172.21.15.43:0/865588354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff1522e020 0x55ff1522b9d0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:08.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.021+0000 7f047d652740 1 --2- 172.21.15.43:0/865588354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff15223760 0x55ff1522b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:08.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.021+0000 7f047d652740 1 --2- 172.21.15.43:0/865588354 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55ff1523e0f0 0x55ff15240520 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:08.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.021+0000 7f047d652740 1 --2- 172.21.15.43:0/865588354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ff15222e40 0x55ff1522af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:08.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.021+0000 7f047d652740 1 -- 172.21.15.43:0/865588354 >> 172.21.15.43:0/865588354 conn(0x55ff1521f3d0 msgr2=0x55ff15225100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:08.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.021+0000 7f0461600640 0 client.16884 ms_handle_reset on v2:172.21.15.43:6838/3404555030 2024-09-19T05:12:08.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.022+0000 7f047d652740 1 -- 172.21.15.43:0/865588354 shutdown_connections 2024-09-19T05:12:08.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.022+0000 7f047d652740 1 -- 172.21.15.43:0/865588354 wait complete. 2024-09-19T05:12:08.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.022+0000 7f047d652740 20 client.16884 trim_cache size 0 max 16384 2024-09-19T05:12:08.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.035+0000 7f047d652740 1 Processor -- start 2024-09-19T05:12:08.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.036+0000 7f047d652740 1 -- start start 2024-09-19T05:12:08.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.036+0000 7f047d652740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ff15222e40 0x55ff152258f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:08.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.036+0000 7f047d652740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff15223760 0x55ff15245e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:08.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.036+0000 7f047d652740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff1522e020 0x55ff15246380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:08.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.036+0000 7f047d652740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f04741b3c60 con 0x55ff15223760 2024-09-19T05:12:08.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.036+0000 7f047d652740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f04741b57a0 con 0x55ff1522e020 2024-09-19T05:12:08.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.036+0000 7f047d652740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f04741b4330 con 0x55ff15222e40 2024-09-19T05:12:08.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.036+0000 7f046be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff15223760 0x55ff15245e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:08.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.036+0000 7f0473e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ff15222e40 0x55ff152258f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:08.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.036+0000 7f0478c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff1522e020 0x55ff15246380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:08.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.036+0000 7f046be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff15223760 0x55ff15245e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34768/0 (socket says 172.21.15.43:34768) 2024-09-19T05:12:08.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.036+0000 7f0473e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ff15222e40 0x55ff152258f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:50248/0 (socket says 172.21.15.43:50248) 2024-09-19T05:12:08.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.036+0000 7f046be00640 1 -- 172.21.15.43:0/1146678894 learned_addr learned my addr 172.21.15.43:0/1146678894 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:08.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.037+0000 7f046be00640 1 -- 172.21.15.43:0/1146678894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ff15222e40 msgr2=0x55ff152258f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:08.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.037+0000 7f046be00640 1 --2- 172.21.15.43:0/1146678894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ff15222e40 0x55ff152258f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:08.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.037+0000 7f046be00640 1 -- 172.21.15.43:0/1146678894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff1522e020 msgr2=0x55ff15246380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:08.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.037+0000 7f046be00640 1 --2- 172.21.15.43:0/1146678894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff1522e020 0x55ff15246380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:08.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.037+0000 7f046be00640 1 -- 172.21.15.43:0/1146678894 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff15220060 con 0x55ff15223760 2024-09-19T05:12:08.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.037+0000 7f0478c00640 1 --2- 172.21.15.43:0/1146678894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff1522e020 0x55ff15246380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:12:08.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.037+0000 7f046be00640 1 --2- 172.21.15.43:0/1146678894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff15223760 0x55ff15245e40 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f045c005d30 tx=0x7f045c005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69fa2174216dbc7b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:08.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.037+0000 7f0463e00640 1 -- 172.21.15.43:0/1146678894 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045c04f070 con 0x55ff15223760 2024-09-19T05:12:08.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.037+0000 7f0463e00640 1 -- 172.21.15.43:0/1146678894 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f045c047040 con 0x55ff15223760 2024-09-19T05:12:08.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.037+0000 7f0463e00640 1 -- 172.21.15.43:0/1146678894 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045c03c450 con 0x55ff15223760 2024-09-19T05:12:08.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.038+0000 7f047d652740 1 -- 172.21.15.43:0/1146678894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff15223760 msgr2=0x55ff15245e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:08.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.038+0000 7f047d652740 1 --2- 172.21.15.43:0/1146678894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff15223760 0x55ff15245e40 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f045c005d30 tx=0x7f045c005a90 comp rx=0 tx=0).stop 2024-09-19T05:12:08.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.038+0000 7f047d652740 1 -- 172.21.15.43:0/1146678894 shutdown_connections 2024-09-19T05:12:08.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.038+0000 7f047d652740 1 --2- 172.21.15.43:0/1146678894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff1522e020 0x55ff15246380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:08.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.038+0000 7f047d652740 1 --2- 172.21.15.43:0/1146678894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff15223760 0x55ff15245e40 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:08.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.038+0000 7f047d652740 1 --2- 172.21.15.43:0/1146678894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ff15222e40 0x55ff152258f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:08.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.038+0000 7f047d652740 1 -- 172.21.15.43:0/1146678894 >> 172.21.15.43:0/1146678894 conn(0x55ff1521f3d0 msgr2=0x55ff1522a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:08.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.038+0000 7f047d652740 1 -- 172.21.15.43:0/1146678894 shutdown_connections 2024-09-19T05:12:08.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.038+0000 7f047d652740 1 -- 172.21.15.43:0/1146678894 wait complete. 2024-09-19T05:12:08.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.039+0000 7f047d652740 1 -- start start 2024-09-19T05:12:08.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.039+0000 7f047d652740 1 Processor -- start 2024-09-19T05:12:08.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.039+0000 7f047d652740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff15222e40 0x55ff15231700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:08.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.039+0000 7f047d652740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff15223760 0x55ff15231c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:08.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.039+0000 7f047d652740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ff1522e020 0x55ff152a6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:08.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.039+0000 7f047d652740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f04741b50d0 con 0x55ff15222e40 2024-09-19T05:12:08.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.039+0000 7f047d652740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f04741b4a00 con 0x55ff15223760 2024-09-19T05:12:08.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.039+0000 7f047d652740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f04741b1c60 con 0x55ff1522e020 2024-09-19T05:12:08.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.039+0000 7f047d652740 10 client.0 _ll_register_callbacks cb 0x7f046c097f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:12:08.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.040+0000 7f0473e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff15222e40 0x55ff15231700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:08.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.040+0000 7f0473e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff15222e40 0x55ff15231700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34774/0 (socket says 172.21.15.43:34774) 2024-09-19T05:12:08.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.040+0000 7f0473e00640 1 -- 172.21.15.43:0/770126726 learned_addr learned my addr 172.21.15.43:0/770126726 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:08.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.040+0000 7f046be00640 1 --2- 172.21.15.43:0/770126726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff15223760 0x55ff15231c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:08.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.040+0000 7f0478c00640 1 --2- 172.21.15.43:0/770126726 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ff1522e020 0x55ff152a6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:08.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.040+0000 7f0473e00640 1 -- 172.21.15.43:0/770126726 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ff1522e020 msgr2=0x55ff152a6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:08.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.040+0000 7f0473e00640 1 --2- 172.21.15.43:0/770126726 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ff1522e020 0x55ff152a6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:08.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.040+0000 7f0473e00640 1 -- 172.21.15.43:0/770126726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff15223760 msgr2=0x55ff15231c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:08.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.040+0000 7f0473e00640 1 --2- 172.21.15.43:0/770126726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff15223760 0x55ff15231c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:08.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.040+0000 7f0473e00640 1 -- 172.21.15.43:0/770126726 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff152a7410 con 0x55ff15222e40 2024-09-19T05:12:08.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.040+0000 7f0473e00640 1 --2- 172.21.15.43:0/770126726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff15222e40 0x55ff15231700 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f046400f6b0 tx=0x7f04640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b30462aefe6139 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:08.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.040+0000 7f0461600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:12:08.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.041+0000 7f0461600640 1 -- 172.21.15.43:0/770126726 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046402b440 con 0x55ff15222e40 2024-09-19T05:12:08.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.041+0000 7f0461600640 1 -- 172.21.15.43:0/770126726 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f046402b5e0 con 0x55ff15222e40 2024-09-19T05:12:08.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.041+0000 7f047d652740 1 -- 172.21.15.43:0/770126726 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f045c047040 con 0x55ff15222e40 2024-09-19T05:12:08.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.041+0000 7f0461600640 1 -- 172.21.15.43:0/770126726 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f046402b820 con 0x55ff15222e40 2024-09-19T05:12:08.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.041+0000 7f047d652740 10 client.16890 fetch_fsmap learned FSMap version 29 2024-09-19T05:12:08.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.041+0000 7f047d652740 1 -- 172.21.15.43:0/770126726 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ff152a8730 con 0x55ff15222e40 2024-09-19T05:12:08.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.041+0000 7f0461600640 1 -- 172.21.15.43:0/770126726 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f045c047040 con 0x55ff15222e40 2024-09-19T05:12:08.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.042+0000 7f0461600640 1 -- 172.21.15.43:0/770126726 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f046401f420 con 0x55ff15222e40 2024-09-19T05:12:08.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.042+0000 7f047d652740 10 client.16890 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:12:08.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.042+0000 7f047d652740 10 client.16890 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-19T05:12:08.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.042+0000 7f047d652740 20 client.16890 populate_metadata read hostname 'smithi043' 2024-09-19T05:12:08.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.042+0000 7f047d652740 1 --2- 172.21.15.43:0/770126726 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55ff152a8e80 0x55ff152ab2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:08.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.042+0000 7f047d652740 4 client.16890 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:12:08.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.042+0000 7f047d652740 1 -- 172.21.15.43:0/770126726 --> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ff15190e60 con 0x55ff152a8e80 2024-09-19T05:12:08.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.042+0000 7f046be00640 1 --2- 172.21.15.43:0/770126726 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55ff152a8e80 0x55ff152ab2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:08.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.043+0000 7f046be00640 1 --2- 172.21.15.43:0/770126726 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55ff152a8e80 0x55ff152ab2a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b74c8e4f28f69ab1 server_cookie=87b32c016951d3ac in_seq=0 out_seq=0 2024-09-19T05:12:08.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.043+0000 7f0461600640 10 client.16890 ms_handle_connect on v2:172.21.15.43:6838/3404555030 2024-09-19T05:12:08.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.043+0000 7f0461600640 1 -- 172.21.15.43:0/770126726 <== mds.1 v2:172.21.15.43:6838/3404555030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ff15190e60 con 0x55ff152a8e80 2024-09-19T05:12:08.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.043+0000 7f0461600640 10 client.16890 handle_command_reply: tid=0 2024-09-19T05:12:08.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.043+0000 7f047d652740 1 client.16890 shutdown 2024-09-19T05:12:08.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.043+0000 7f047d652740 1 -- 172.21.15.43:0/770126726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff15222e40 msgr2=0x55ff15231700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:08.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.043+0000 7f047d652740 1 --2- 172.21.15.43:0/770126726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff15222e40 0x55ff15231700 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f046400f6b0 tx=0x7f04640046c0 comp rx=0 tx=0).stop 2024-09-19T05:12:08.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.043+0000 7f047d652740 1 -- 172.21.15.43:0/770126726 shutdown_connections 2024-09-19T05:12:08.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.043+0000 7f047d652740 1 --2- 172.21.15.43:0/770126726 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55ff152a8e80 0x55ff152ab2a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:08.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.043+0000 7f047d652740 1 --2- 172.21.15.43:0/770126726 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ff1522e020 0x55ff152a6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:08.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.043+0000 7f047d652740 1 --2- 172.21.15.43:0/770126726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff15223760 0x55ff15231c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:08.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.043+0000 7f047d652740 1 --2- 172.21.15.43:0/770126726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff15222e40 0x55ff15231700 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:08.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.043+0000 7f047d652740 1 -- 172.21.15.43:0/770126726 >> 172.21.15.43:0/770126726 conn(0x55ff1521f3d0 msgr2=0x55ff1521dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:08.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.043+0000 7f0461600640 0 client.16890 ms_handle_reset on v2:172.21.15.43:6838/3404555030 2024-09-19T05:12:08.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.044+0000 7f047d652740 1 -- 172.21.15.43:0/770126726 shutdown_connections 2024-09-19T05:12:08.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.044+0000 7f047d652740 1 -- 172.21.15.43:0/770126726 wait complete. 2024-09-19T05:12:08.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.044+0000 7f047d652740 20 client.16890 trim_cache size 0 max 16384 2024-09-19T05:12:08.052 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:12:08.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.045+0000 7f0479600640 1 -- 172.21.15.43:0/2139259326 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f04580a3430 msgr2=0x7f04580a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:08.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.045+0000 7f0479600640 1 --2- 172.21.15.43:0/2139259326 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f04580a3430 0x7f04580a58f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0464009fd0 tx=0x7f0464005f90 comp rx=0 tx=0).stop 2024-09-19T05:12:08.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.045+0000 7f0479600640 1 -- 172.21.15.43:0/2139259326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04741129d0 msgr2=0x7f047419e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:08.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.045+0000 7f0479600640 1 --2- 172.21.15.43:0/2139259326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04741129d0 0x7f047419e6e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f046c005fd0 tx=0x7f046c00c800 comp rx=0 tx=0).stop 2024-09-19T05:12:08.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.045+0000 7f0479600640 1 -- 172.21.15.43:0/2139259326 shutdown_connections 2024-09-19T05:12:08.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.045+0000 7f0479600640 1 --2- 172.21.15.43:0/2139259326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04741129d0 0x7f047419e6e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:08.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.045+0000 7f0479600640 1 --2- 172.21.15.43:0/2139259326 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f04580a3430 0x7f04580a58f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:08.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.045+0000 7f0479600640 1 --2- 172.21.15.43:0/2139259326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0474074eb0 0x7f04741a4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:08.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.045+0000 7f0479600640 1 --2- 172.21.15.43:0/2139259326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0474074590 0x7f04741a4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:08.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.045+0000 7f0479600640 1 -- 172.21.15.43:0/2139259326 >> 172.21.15.43:0/2139259326 conn(0x7f04740fd6e0 msgr2=0x7f04740fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:08.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.046+0000 7f0479600640 1 -- 172.21.15.43:0/2139259326 shutdown_connections 2024-09-19T05:12:08.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:08.046+0000 7f0479600640 1 -- 172.21.15.43:0/2139259326 wait complete. 2024-09-19T05:12:08.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:08 smithi043 ceph-mon[30800]: pgmap v229: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-19T05:12:08.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:08 smithi151 ceph-mon[35425]: pgmap v229: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-19T05:12:08.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:08 smithi096 ceph-mon[35085]: pgmap v229: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-19T05:12:08.918 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T05:12:09.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:09 smithi043 ceph-mon[30800]: osdmap e89: 12 total, 12 up, 12 in 2024-09-19T05:12:09.296 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:12:09.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:09 smithi151 ceph-mon[35425]: osdmap e89: 12 total, 12 up, 12 in 2024-09-19T05:12:09.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:09 smithi096 ceph-mon[35085]: osdmap e89: 12 total, 12 up, 12 in 2024-09-19T05:12:09.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.840+0000 7f2636200640 1 -- 172.21.15.43:0/229160053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26301019c0 msgr2=0x7f2630101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:09.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.840+0000 7f2636200640 1 --2- 172.21.15.43:0/229160053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26301019c0 0x7f2630101e20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f261c009860 tx=0x7f261c02a340 comp rx=0 tx=0).stop 2024-09-19T05:12:09.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.840+0000 7f2636200640 1 -- 172.21.15.43:0/229160053 shutdown_connections 2024-09-19T05:12:09.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.840+0000 7f2636200640 1 --2- 172.21.15.43:0/229160053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2630102360 0x7f263010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:09.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.840+0000 7f2636200640 1 --2- 172.21.15.43:0/229160053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26301019c0 0x7f2630101e20 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:09.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.840+0000 7f2636200640 1 --2- 172.21.15.43:0/229160053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26301079c0 0x7f2630107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:09.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.840+0000 7f2636200640 1 -- 172.21.15.43:0/229160053 >> 172.21.15.43:0/229160053 conn(0x7f26300fd6e0 msgr2=0x7f26300ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:09.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.840+0000 7f2636200640 1 -- 172.21.15.43:0/229160053 shutdown_connections 2024-09-19T05:12:09.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.840+0000 7f2636200640 1 -- 172.21.15.43:0/229160053 wait complete. 2024-09-19T05:12:09.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.841+0000 7f2636200640 1 Processor -- start 2024-09-19T05:12:09.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.841+0000 7f2636200640 1 -- start start 2024-09-19T05:12:09.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.841+0000 7f2636200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26301019c0 0x7f26301982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:09.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.841+0000 7f2636200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2630102360 0x7f26301987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:09.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.841+0000 7f2636200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26301079c0 0x7f263019cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:09.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.841+0000 7f2636200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2630111dc0 con 0x7f2630102360 2024-09-19T05:12:09.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.841+0000 7f2636200640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2630111c40 con 0x7f26301019c0 2024-09-19T05:12:09.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.841+0000 7f2636200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2630111f40 con 0x7f26301079c0 2024-09-19T05:12:09.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.842+0000 7f262fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2630102360 0x7f26301987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:09.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.842+0000 7f262fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2630102360 0x7f26301987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34786/0 (socket says 172.21.15.43:34786) 2024-09-19T05:12:09.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.842+0000 7f262fe00640 1 -- 172.21.15.43:0/365255286 learned_addr learned my addr 172.21.15.43:0/365255286 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:09.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.842+0000 7f2634e00640 1 --2- 172.21.15.43:0/365255286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26301019c0 0x7f26301982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:09.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.842+0000 7f2635800640 1 --2- 172.21.15.43:0/365255286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26301079c0 0x7f263019cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:09.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.842+0000 7f262fe00640 1 -- 172.21.15.43:0/365255286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26301079c0 msgr2=0x7f263019cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:09.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.842+0000 7f262fe00640 1 --2- 172.21.15.43:0/365255286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26301079c0 0x7f263019cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:09.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.842+0000 7f262fe00640 1 -- 172.21.15.43:0/365255286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26301019c0 msgr2=0x7f26301982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:09.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.842+0000 7f262fe00640 1 --2- 172.21.15.43:0/365255286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26301019c0 0x7f26301982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:09.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.842+0000 7f262fe00640 1 -- 172.21.15.43:0/365255286 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f263019d250 con 0x7f2630102360 2024-09-19T05:12:09.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.843+0000 7f262fe00640 1 --2- 172.21.15.43:0/365255286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2630102360 0x7f26301987e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f261c035040 tx=0x7f261c033b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:09.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.843+0000 7f262d600640 1 -- 172.21.15.43:0/365255286 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c03f020 con 0x7f2630102360 2024-09-19T05:12:09.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.843+0000 7f262d600640 1 -- 172.21.15.43:0/365255286 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f261c004040 con 0x7f2630102360 2024-09-19T05:12:09.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.843+0000 7f2636200640 1 -- 172.21.15.43:0/365255286 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f263019d4e0 con 0x7f2630102360 2024-09-19T05:12:09.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.843+0000 7f262d600640 1 -- 172.21.15.43:0/365255286 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c037040 con 0x7f2630102360 2024-09-19T05:12:09.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.843+0000 7f2636200640 1 -- 172.21.15.43:0/365255286 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26301a4e20 con 0x7f2630102360 2024-09-19T05:12:09.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.846+0000 7f262d600640 1 -- 172.21.15.43:0/365255286 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f261c042020 con 0x7f2630102360 2024-09-19T05:12:09.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.846+0000 7f263a1ec740 1 Processor -- start 2024-09-19T05:12:09.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.846+0000 7f263a1ec740 1 -- start start 2024-09-19T05:12:09.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.847+0000 7f263a1ec740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c118fdace0 0x55c118fdb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:09.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.847+0000 7f263a1ec740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c118fdb600 0x55c118fe9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:09.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.847+0000 7f263a1ec740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c118fea430 0x55c118fec820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:09.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.847+0000 7f263a1ec740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c118f413e0 con 0x55c118fdace0 2024-09-19T05:12:09.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.847+0000 7f263a1ec740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c118ee8d80 con 0x55c118fdb600 2024-09-19T05:12:09.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.847+0000 7f263a1ec740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c118f3d3a0 con 0x55c118fea430 2024-09-19T05:12:09.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.847+0000 7f2635800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c118fdace0 0x55c118fdb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:09.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.847+0000 7f2635800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c118fdace0 0x55c118fdb0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34790/0 (socket says 172.21.15.43:34790) 2024-09-19T05:12:09.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.847+0000 7f2635800640 1 -- 172.21.15.43:0/4022175923 learned_addr learned my addr 172.21.15.43:0/4022175923 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:09.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.847+0000 7f2634e00640 1 --2- 172.21.15.43:0/4022175923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c118fdb600 0x55c118fe9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:09.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.847+0000 7f262fe00640 1 --2- 172.21.15.43:0/4022175923 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c118fea430 0x55c118fec820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:09.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.848+0000 7f2634e00640 1 -- 172.21.15.43:0/4022175923 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c118fea430 msgr2=0x55c118fec820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:09.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.848+0000 7f2634e00640 1 --2- 172.21.15.43:0/4022175923 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c118fea430 0x55c118fec820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:09.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.848+0000 7f2634e00640 1 -- 172.21.15.43:0/4022175923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c118fdace0 msgr2=0x55c118fdb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:09.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.848+0000 7f2634e00640 1 --2- 172.21.15.43:0/4022175923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c118fdace0 0x55c118fdb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:09.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.848+0000 7f2634e00640 1 -- 172.21.15.43:0/4022175923 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c118f07ae0 con 0x55c118fdb600 2024-09-19T05:12:09.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.848+0000 7f262fe00640 1 --2- 172.21.15.43:0/4022175923 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c118fea430 0x55c118fec820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:12:09.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.848+0000 7f262d600640 1 --2- 172.21.15.43:0/365255286 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f26180a33b0 0x7f26180a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:09.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.848+0000 7f2634e00640 1 --2- 172.21.15.43:0/4022175923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c118fdb600 0x55c118fe9ef0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2624009c00 tx=0x7f2624011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8dd928d720e23ee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:09.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.848+0000 7f262d600640 1 -- 172.21.15.43:0/365255286 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8586+0+0 (secure 0 0 0) 0x7f261c0efdd0 con 0x7f2630102360 2024-09-19T05:12:09.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.848+0000 7f2622000640 1 -- 172.21.15.43:0/4022175923 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2624004050 con 0x55c118fdb600 2024-09-19T05:12:09.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.848+0000 7f2635800640 1 --2- 172.21.15.43:0/365255286 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f26180a33b0 0x7f26180a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:09.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.848+0000 7f2622000640 1 -- 172.21.15.43:0/4022175923 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2624005500 con 0x55c118fdb600 2024-09-19T05:12:09.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.848+0000 7f2622000640 1 -- 172.21.15.43:0/4022175923 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2624004e00 con 0x55c118fdb600 2024-09-19T05:12:09.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.849+0000 7f263a1ec740 1 -- 172.21.15.43:0/4022175923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c118fdb600 msgr2=0x55c118fe9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:09.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.849+0000 7f2635800640 1 --2- 172.21.15.43:0/365255286 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f26180a33b0 0x7f26180a5870 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f262800e660 tx=0x7f2628010040 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:09.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.849+0000 7f263a1ec740 1 --2- 172.21.15.43:0/4022175923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c118fdb600 0x55c118fe9ef0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2624009c00 tx=0x7f2624011810 comp rx=0 tx=0).stop 2024-09-19T05:12:09.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.849+0000 7f263a1ec740 1 -- 172.21.15.43:0/4022175923 shutdown_connections 2024-09-19T05:12:09.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.849+0000 7f263a1ec740 1 --2- 172.21.15.43:0/4022175923 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c118fea430 0x55c118fec820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:09.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.849+0000 7f263a1ec740 1 --2- 172.21.15.43:0/4022175923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c118fdb600 0x55c118fe9ef0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:09.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.849+0000 7f263a1ec740 1 --2- 172.21.15.43:0/4022175923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c118fdace0 0x55c118fdb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:09.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.849+0000 7f263a1ec740 1 -- 172.21.15.43:0/4022175923 >> 172.21.15.43:0/4022175923 conn(0x55c118fd73d0 msgr2=0x55c118fd77b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:09.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.849+0000 7f263a1ec740 1 -- 172.21.15.43:0/4022175923 shutdown_connections 2024-09-19T05:12:09.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.849+0000 7f263a1ec740 1 -- 172.21.15.43:0/4022175923 wait complete. 2024-09-19T05:12:09.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.850+0000 7f263a1ec740 1 -- start start 2024-09-19T05:12:09.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.850+0000 7f263a1ec740 1 Processor -- start 2024-09-19T05:12:09.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.851+0000 7f263a1ec740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c118fdace0 0x55c118fe73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:09.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.851+0000 7f263a1ec740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c118fdb600 0x55c118fe78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:09.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.851+0000 7f263a1ec740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c118fea430 0x55c118fe7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:09.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.851+0000 7f263a1ec740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c118fed000 con 0x55c118fea430 2024-09-19T05:12:09.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.851+0000 7f263a1ec740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c118efc0f0 con 0x55c118fdace0 2024-09-19T05:12:09.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.851+0000 7f263a1ec740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c118fed180 con 0x55c118fdb600 2024-09-19T05:12:09.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.851+0000 7f263a1ec740 10 client.0 _ll_register_callbacks cb 0x55c118ee3bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:12:09.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.851+0000 7f2634e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c118fdace0 0x55c118fe73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:09.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.851+0000 7f2634e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c118fdace0 0x55c118fe73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:55040/0 (socket says 172.21.15.43:55040) 2024-09-19T05:12:09.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.851+0000 7f2635800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c118fea430 0x55c118fe7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:09.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.851+0000 7f2634e00640 1 -- 172.21.15.43:0/47270962 learned_addr learned my addr 172.21.15.43:0/47270962 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:09.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.851+0000 7f262fe00640 1 --2- 172.21.15.43:0/47270962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c118fdb600 0x55c118fe78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:09.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.851+0000 7f2634e00640 1 -- 172.21.15.43:0/47270962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c118fdb600 msgr2=0x55c118fe78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:09.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.851+0000 7f2634e00640 1 --2- 172.21.15.43:0/47270962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c118fdb600 0x55c118fe78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:09.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.851+0000 7f2634e00640 1 -- 172.21.15.43:0/47270962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c118fea430 msgr2=0x55c118fe7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:09.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.851+0000 7f2634e00640 1 --2- 172.21.15.43:0/47270962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c118fea430 0x55c118fe7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:09.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.851+0000 7f2634e00640 1 -- 172.21.15.43:0/47270962 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c118cb23c0 con 0x55c118fdace0 2024-09-19T05:12:09.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.852+0000 7f262fe00640 1 --2- 172.21.15.43:0/47270962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c118fdb600 0x55c118fe78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:12:09.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.852+0000 7f2634e00640 1 --2- 172.21.15.43:0/47270962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c118fdace0 0x55c118fe73a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f26240045f0 tx=0x7f26240056b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d557d38f9cb1fb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:09.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.852+0000 7f2616a00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:12:09.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.852+0000 7f2616a00640 1 -- 172.21.15.43:0/47270962 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2624029070 con 0x55c118fdace0 2024-09-19T05:12:09.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.852+0000 7f263a1ec740 1 -- 172.21.15.43:0/47270962 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2624005500 con 0x55c118fdace0 2024-09-19T05:12:09.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.852+0000 7f2616a00640 1 -- 172.21.15.43:0/47270962 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8586+0+0 (secure 0 0 0) 0x7f2624004d80 con 0x55c118fdace0 2024-09-19T05:12:09.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.852+0000 7f2616a00640 1 -- 172.21.15.43:0/47270962 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2624035050 con 0x55c118fdace0 2024-09-19T05:12:09.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.852+0000 7f263a1ec740 10 client.26924 fetch_fsmap learned FSMap version 29 2024-09-19T05:12:09.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.852+0000 7f263a1ec740 1 -- 172.21.15.43:0/47270962 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c118ff9100 con 0x55c118fdace0 2024-09-19T05:12:09.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.853+0000 7f2616a00640 1 -- 172.21.15.43:0/47270962 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2624005500 con 0x55c118fdace0 2024-09-19T05:12:09.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.853+0000 7f2616a00640 1 -- 172.21.15.43:0/47270962 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2624023430 con 0x55c118fdace0 2024-09-19T05:12:09.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.853+0000 7f263a1ec740 10 client.26924 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:12:09.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.853+0000 7f263a1ec740 10 client.26924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:12:09.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.853+0000 7f263a1ec740 20 client.26924 populate_metadata read hostname 'smithi043' 2024-09-19T05:12:09.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.853+0000 7f263a1ec740 1 --2- 172.21.15.43:0/47270962 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55c118ffa4b0 0x55c118ffc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:09.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.853+0000 7f263a1ec740 4 client.26924 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:12:09.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.853+0000 7f263a1ec740 1 -- 172.21.15.43:0/47270962 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c118f4e950 con 0x55c118ffa4b0 2024-09-19T05:12:09.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.853+0000 7f262fe00640 1 --2- 172.21.15.43:0/47270962 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55c118ffa4b0 0x55c118ffc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:09.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.854+0000 7f262fe00640 1 --2- 172.21.15.43:0/47270962 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55c118ffa4b0 0x55c118ffc8a0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a275b00b2f43af5 server_cookie=87a51399aec18d8d in_seq=0 out_seq=0 2024-09-19T05:12:09.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.854+0000 7f2616a00640 10 client.26924 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:12:09.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.855+0000 7f2616a00640 1 -- 172.21.15.43:0/47270962 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c118f4e950 con 0x55c118ffa4b0 2024-09-19T05:12:09.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.855+0000 7f2616a00640 10 client.26924 handle_command_reply: tid=0 2024-09-19T05:12:09.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.855+0000 7f263a1ec740 1 client.26924 shutdown 2024-09-19T05:12:09.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.855+0000 7f263a1ec740 1 -- 172.21.15.43:0/47270962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c118fdace0 msgr2=0x55c118fe73a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:09.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.855+0000 7f263a1ec740 1 --2- 172.21.15.43:0/47270962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c118fdace0 0x55c118fe73a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f26240045f0 tx=0x7f26240056b0 comp rx=0 tx=0).stop 2024-09-19T05:12:09.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.855+0000 7f263a1ec740 1 -- 172.21.15.43:0/47270962 shutdown_connections 2024-09-19T05:12:09.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.855+0000 7f263a1ec740 1 --2- 172.21.15.43:0/47270962 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55c118ffa4b0 0x55c118ffc8a0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:09.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.855+0000 7f263a1ec740 1 --2- 172.21.15.43:0/47270962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c118fea430 0x55c118fe7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:09.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.855+0000 7f263a1ec740 1 --2- 172.21.15.43:0/47270962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c118fdb600 0x55c118fe78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:09.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.855+0000 7f263a1ec740 1 --2- 172.21.15.43:0/47270962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c118fdace0 0x55c118fe73a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:09.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.855+0000 7f2616a00640 0 client.26924 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:12:09.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.855+0000 7f263a1ec740 1 -- 172.21.15.43:0/47270962 >> 172.21.15.43:0/47270962 conn(0x55c118fd73d0 msgr2=0x55c118fdd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:09.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.855+0000 7f263a1ec740 1 -- 172.21.15.43:0/47270962 shutdown_connections 2024-09-19T05:12:09.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.855+0000 7f263a1ec740 1 -- 172.21.15.43:0/47270962 wait complete. 2024-09-19T05:12:09.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.855+0000 7f263a1ec740 20 client.26924 trim_cache size 0 max 16384 2024-09-19T05:12:09.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.869+0000 7f263a1ec740 1 Processor -- start 2024-09-19T05:12:09.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.869+0000 7f263a1ec740 1 -- start start 2024-09-19T05:12:09.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.870+0000 7f263a1ec740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c118fdace0 0x55c118fe6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:09.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.870+0000 7f263a1ec740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c118fdb600 0x55c119002080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:09.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.870+0000 7f263a1ec740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c118fea430 0x55c1190025c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:09.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.870+0000 7f263a1ec740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f26301ab530 con 0x55c118fdb600 2024-09-19T05:12:09.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.870+0000 7f263a1ec740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f26301ad070 con 0x55c118fea430 2024-09-19T05:12:09.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.870+0000 7f263a1ec740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f26301abc00 con 0x55c118fdace0 2024-09-19T05:12:09.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.870+0000 7f262fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c118fdb600 0x55c119002080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:09.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.870+0000 7f2635800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c118fea430 0x55c1190025c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:09.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.870+0000 7f262fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c118fdb600 0x55c119002080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34808/0 (socket says 172.21.15.43:34808) 2024-09-19T05:12:09.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.870+0000 7f262fe00640 1 -- 172.21.15.43:0/3638809174 learned_addr learned my addr 172.21.15.43:0/3638809174 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:09.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.870+0000 7f2634e00640 1 --2- 172.21.15.43:0/3638809174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c118fdace0 0x55c118fe6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:09.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.870+0000 7f262fe00640 1 -- 172.21.15.43:0/3638809174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c118fdace0 msgr2=0x55c118fe6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:09.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.870+0000 7f262fe00640 1 --2- 172.21.15.43:0/3638809174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c118fdace0 0x55c118fe6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:09.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.870+0000 7f262fe00640 1 -- 172.21.15.43:0/3638809174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c118fea430 msgr2=0x55c1190025c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:09.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.870+0000 7f262fe00640 1 --2- 172.21.15.43:0/3638809174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c118fea430 0x55c1190025c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:09.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.870+0000 7f262fe00640 1 -- 172.21.15.43:0/3638809174 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c118fd6c80 con 0x55c118fdb600 2024-09-19T05:12:09.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.870+0000 7f2635800640 1 --2- 172.21.15.43:0/3638809174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c118fea430 0x55c1190025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:12:09.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.871+0000 7f262fe00640 1 --2- 172.21.15.43:0/3638809174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c118fdb600 0x55c119002080 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f261c009830 tx=0x7f261c0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bec3e8d164df2b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:09.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.871+0000 7f2621600640 1 -- 172.21.15.43:0/3638809174 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c0bba80 con 0x55c118fdb600 2024-09-19T05:12:09.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.871+0000 7f2621600640 1 -- 172.21.15.43:0/3638809174 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f261c0bc820 con 0x55c118fdb600 2024-09-19T05:12:09.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.871+0000 7f2621600640 1 -- 172.21.15.43:0/3638809174 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c0dc0f0 con 0x55c118fdb600 2024-09-19T05:12:09.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.872+0000 7f263a1ec740 1 -- 172.21.15.43:0/3638809174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c118fdb600 msgr2=0x55c119002080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:09.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.872+0000 7f263a1ec740 1 --2- 172.21.15.43:0/3638809174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c118fdb600 0x55c119002080 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f261c009830 tx=0x7f261c0035f0 comp rx=0 tx=0).stop 2024-09-19T05:12:09.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.872+0000 7f263a1ec740 1 -- 172.21.15.43:0/3638809174 shutdown_connections 2024-09-19T05:12:09.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.872+0000 7f263a1ec740 1 --2- 172.21.15.43:0/3638809174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c118fea430 0x55c1190025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:09.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.872+0000 7f263a1ec740 1 --2- 172.21.15.43:0/3638809174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c118fdb600 0x55c119002080 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:09.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.872+0000 7f263a1ec740 1 --2- 172.21.15.43:0/3638809174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c118fdace0 0x55c118fe6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:09.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.872+0000 7f263a1ec740 1 -- 172.21.15.43:0/3638809174 >> 172.21.15.43:0/3638809174 conn(0x55c118fd73d0 msgr2=0x55c118fe27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:09.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.872+0000 7f263a1ec740 1 -- 172.21.15.43:0/3638809174 shutdown_connections 2024-09-19T05:12:09.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.872+0000 7f263a1ec740 1 -- 172.21.15.43:0/3638809174 wait complete. 2024-09-19T05:12:09.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.872+0000 7f263a1ec740 1 -- start start 2024-09-19T05:12:09.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.873+0000 7f263a1ec740 1 Processor -- start 2024-09-19T05:12:09.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.873+0000 7f263a1ec740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c118fdace0 0x55c118ff2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:09.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.873+0000 7f263a1ec740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c118fdb600 0x55c118ff2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:09.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.873+0000 7f263a1ec740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c118fea430 0x55c118fe3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:09.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.873+0000 7f263a1ec740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f26301ac9a0 con 0x55c118fea430 2024-09-19T05:12:09.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.873+0000 7f263a1ec740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f26301ac2d0 con 0x55c118fdb600 2024-09-19T05:12:09.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.873+0000 7f263a1ec740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f26301a9530 con 0x55c118fdace0 2024-09-19T05:12:09.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.873+0000 7f263a1ec740 10 client.0 _ll_register_callbacks cb 0x7f2628009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:12:09.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.873+0000 7f2635800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c118fea430 0x55c118fe3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:09.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.873+0000 7f2635800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c118fea430 0x55c118fe3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34816/0 (socket says 172.21.15.43:34816) 2024-09-19T05:12:09.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.873+0000 7f2635800640 1 -- 172.21.15.43:0/604882048 learned_addr learned my addr 172.21.15.43:0/604882048 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:09.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.873+0000 7f262fe00640 1 --2- 172.21.15.43:0/604882048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c118fdb600 0x55c118ff2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:09.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.873+0000 7f2634e00640 1 --2- 172.21.15.43:0/604882048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c118fdace0 0x55c118ff2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:09.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.874+0000 7f262fe00640 1 -- 172.21.15.43:0/604882048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c118fdace0 msgr2=0x55c118ff2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:09.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.874+0000 7f262fe00640 1 --2- 172.21.15.43:0/604882048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c118fdace0 0x55c118ff2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:09.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.874+0000 7f262fe00640 1 -- 172.21.15.43:0/604882048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c118fea430 msgr2=0x55c118fe3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:09.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.874+0000 7f262fe00640 1 --2- 172.21.15.43:0/604882048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c118fea430 0x55c118fe3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:09.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.874+0000 7f262fe00640 1 -- 172.21.15.43:0/604882048 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c118fe3fd0 con 0x55c118fdb600 2024-09-19T05:12:09.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.874+0000 7f262fe00640 1 --2- 172.21.15.43:0/604882048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c118fdb600 0x55c118ff2950 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f261c0c0690 tx=0x7f261c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8aaf79d5e23f6a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:09.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.874+0000 7f2616a00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:12:09.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.874+0000 7f2616a00640 1 -- 172.21.15.43:0/604882048 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c0bba80 con 0x55c118fdb600 2024-09-19T05:12:09.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.874+0000 7f2616a00640 1 -- 172.21.15.43:0/604882048 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8586+0+0 (secure 0 0 0) 0x7f261c0dc0f0 con 0x55c118fdb600 2024-09-19T05:12:09.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.874+0000 7f263a1ec740 1 -- 172.21.15.43:0/604882048 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f261c0bc820 con 0x55c118fdb600 2024-09-19T05:12:09.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.875+0000 7f2616a00640 1 -- 172.21.15.43:0/604882048 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f261c0bc120 con 0x55c118fdb600 2024-09-19T05:12:09.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.875+0000 7f263a1ec740 10 client.26930 fetch_fsmap learned FSMap version 29 2024-09-19T05:12:09.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.875+0000 7f263a1ec740 1 -- 172.21.15.43:0/604882048 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c1190648d0 con 0x55c118fdb600 2024-09-19T05:12:09.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.875+0000 7f2616a00640 1 -- 172.21.15.43:0/604882048 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f261c0bc820 con 0x55c118fdb600 2024-09-19T05:12:09.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.875+0000 7f2616a00640 1 -- 172.21.15.43:0/604882048 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f261c03a030 con 0x55c118fdb600 2024-09-19T05:12:09.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.875+0000 7f263a1ec740 10 client.26930 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:12:09.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.875+0000 7f263a1ec740 10 client.26930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:12:09.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.875+0000 7f263a1ec740 20 client.26930 populate_metadata read hostname 'smithi043' 2024-09-19T05:12:09.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.875+0000 7f263a1ec740 1 --2- 172.21.15.43:0/604882048 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55c119064ff0 0x55c119067410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:09.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.875+0000 7f263a1ec740 4 client.26930 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:12:09.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.875+0000 7f263a1ec740 1 -- 172.21.15.43:0/604882048 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c118f48e60 con 0x55c119064ff0 2024-09-19T05:12:09.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.875+0000 7f2634e00640 1 --2- 172.21.15.43:0/604882048 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55c119064ff0 0x55c119067410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:09.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.876+0000 7f2634e00640 1 --2- 172.21.15.43:0/604882048 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55c119064ff0 0x55c119067410 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=315634255f9bbe4 server_cookie=afd48ff4191e3a14 in_seq=0 out_seq=0 2024-09-19T05:12:09.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.876+0000 7f2616a00640 10 client.26930 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:12:09.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.876+0000 7f2616a00640 1 -- 172.21.15.43:0/604882048 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c118f48e60 con 0x55c119064ff0 2024-09-19T05:12:09.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.876+0000 7f2616a00640 10 client.26930 handle_command_reply: tid=0 2024-09-19T05:12:09.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.876+0000 7f263a1ec740 1 client.26930 shutdown 2024-09-19T05:12:09.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.876+0000 7f263a1ec740 1 -- 172.21.15.43:0/604882048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c118fdb600 msgr2=0x55c118ff2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:09.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.876+0000 7f263a1ec740 1 --2- 172.21.15.43:0/604882048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c118fdb600 0x55c118ff2950 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f261c0c0690 tx=0x7f261c005f90 comp rx=0 tx=0).stop 2024-09-19T05:12:09.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.876+0000 7f263a1ec740 1 -- 172.21.15.43:0/604882048 shutdown_connections 2024-09-19T05:12:09.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.876+0000 7f263a1ec740 1 --2- 172.21.15.43:0/604882048 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55c119064ff0 0x55c119067410 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:09.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.876+0000 7f263a1ec740 1 --2- 172.21.15.43:0/604882048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c118fea430 0x55c118fe3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:09.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.876+0000 7f263a1ec740 1 --2- 172.21.15.43:0/604882048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c118fdb600 0x55c118ff2950 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:09.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.876+0000 7f263a1ec740 1 --2- 172.21.15.43:0/604882048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c118fdace0 0x55c118ff2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:09.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.876+0000 7f2616a00640 0 client.26930 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:12:09.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.876+0000 7f263a1ec740 1 -- 172.21.15.43:0/604882048 >> 172.21.15.43:0/604882048 conn(0x55c118fd73d0 msgr2=0x55c118fd5c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:09.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.876+0000 7f263a1ec740 1 -- 172.21.15.43:0/604882048 shutdown_connections 2024-09-19T05:12:09.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.876+0000 7f263a1ec740 1 -- 172.21.15.43:0/604882048 wait complete. 2024-09-19T05:12:09.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.877+0000 7f263a1ec740 20 client.26930 trim_cache size 0 max 16384 2024-09-19T05:12:09.883 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:12:09.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.877+0000 7f2636200640 1 -- 172.21.15.43:0/365255286 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f26180a33b0 msgr2=0x7f26180a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:09.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.877+0000 7f2636200640 1 --2- 172.21.15.43:0/365255286 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f26180a33b0 0x7f26180a5870 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f262800e660 tx=0x7f2628010040 comp rx=0 tx=0).stop 2024-09-19T05:12:09.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.877+0000 7f2636200640 1 -- 172.21.15.43:0/365255286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2630102360 msgr2=0x7f26301987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:09.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.877+0000 7f2636200640 1 --2- 172.21.15.43:0/365255286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2630102360 0x7f26301987e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f261c035040 tx=0x7f261c033b60 comp rx=0 tx=0).stop 2024-09-19T05:12:09.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.877+0000 7f2636200640 1 -- 172.21.15.43:0/365255286 shutdown_connections 2024-09-19T05:12:09.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.877+0000 7f2636200640 1 --2- 172.21.15.43:0/365255286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26301079c0 0x7f263019cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:09.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.877+0000 7f2636200640 1 --2- 172.21.15.43:0/365255286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2630102360 0x7f26301987e0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:09.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.877+0000 7f2636200640 1 --2- 172.21.15.43:0/365255286 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f26180a33b0 0x7f26180a5870 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:09.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.877+0000 7f2636200640 1 --2- 172.21.15.43:0/365255286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26301019c0 0x7f26301982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:09.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.877+0000 7f2636200640 1 -- 172.21.15.43:0/365255286 >> 172.21.15.43:0/365255286 conn(0x7f26300fd6e0 msgr2=0x7f26300fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:09.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.877+0000 7f2636200640 1 -- 172.21.15.43:0/365255286 shutdown_connections 2024-09-19T05:12:09.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:09.877+0000 7f2636200640 1 -- 172.21.15.43:0/365255286 wait complete. 2024-09-19T05:12:10.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:10 smithi151 ceph-mon[35425]: pgmap v231: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T05:12:10.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:10 smithi096 ceph-mon[35085]: pgmap v231: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T05:12:10.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:10 smithi043 ceph-mon[30800]: pgmap v231: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T05:12:11.682 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T05:12:11.682 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T05:12:12.077 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:12:12.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:12 smithi043 ceph-mon[30800]: pgmap v232: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T05:12:12.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:12 smithi151 ceph-mon[35425]: pgmap v232: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T05:12:12.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:12 smithi096 ceph-mon[35085]: pgmap v232: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T05:12:12.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.619+0000 7fbef7c00640 1 -- 172.21.15.43:0/3727023245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbef0076720 msgr2=0x7fbef0076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:12.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.619+0000 7fbef7c00640 1 --2- 172.21.15.43:0/3727023245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbef0076720 0x7fbef0076b00 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fbee4009900 tx=0x7fbee402a3f0 comp rx=0 tx=0).stop 2024-09-19T05:12:12.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.619+0000 7fbef7c00640 1 -- 172.21.15.43:0/3727023245 shutdown_connections 2024-09-19T05:12:12.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.619+0000 7fbef7c00640 1 --2- 172.21.15.43:0/3727023245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbef0112a30 0x7fbef0114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:12.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.619+0000 7fbef7c00640 1 --2- 172.21.15.43:0/3727023245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbef0077040 0x7fbef01124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:12.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.619+0000 7fbef7c00640 1 --2- 172.21.15.43:0/3727023245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbef0076720 0x7fbef0076b00 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:12.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.619+0000 7fbef7c00640 1 -- 172.21.15.43:0/3727023245 >> 172.21.15.43:0/3727023245 conn(0x7fbef00ffa90 msgr2=0x7fbef0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:12.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.619+0000 7fbef7c00640 1 -- 172.21.15.43:0/3727023245 shutdown_connections 2024-09-19T05:12:12.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.620+0000 7fbef7c00640 1 -- 172.21.15.43:0/3727023245 wait complete. 2024-09-19T05:12:12.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.620+0000 7fbef7c00640 1 Processor -- start 2024-09-19T05:12:12.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.621+0000 7fbef7c00640 1 -- start start 2024-09-19T05:12:12.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.621+0000 7fbef7c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbef0076720 0x7fbef01a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:12.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.621+0000 7fbef7c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbef0077040 0x7fbef01a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:12.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.621+0000 7fbef7c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbef0112a30 0x7fbef019e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:12.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.621+0000 7fbef7c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbef01185a0 con 0x7fbef0076720 2024-09-19T05:12:12.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.621+0000 7fbef7c00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbef0118420 con 0x7fbef0112a30 2024-09-19T05:12:12.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.621+0000 7fbef7c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbef0118720 con 0x7fbef0077040 2024-09-19T05:12:12.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.621+0000 7fbef5200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbef0076720 0x7fbef01a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:12.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.621+0000 7fbef5200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbef0076720 0x7fbef01a46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34832/0 (socket says 172.21.15.43:34832) 2024-09-19T05:12:12.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.621+0000 7fbef5200640 1 -- 172.21.15.43:0/3387367713 learned_addr learned my addr 172.21.15.43:0/3387367713 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:12.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.621+0000 7fbeefe00640 1 --2- 172.21.15.43:0/3387367713 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbef0077040 0x7fbef01a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:12.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.622+0000 7fbef5c00640 1 --2- 172.21.15.43:0/3387367713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbef0112a30 0x7fbef019e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:12.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.622+0000 7fbef5200640 1 -- 172.21.15.43:0/3387367713 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbef0077040 msgr2=0x7fbef01a4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:12.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.622+0000 7fbef5200640 1 --2- 172.21.15.43:0/3387367713 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbef0077040 0x7fbef01a4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:12.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.622+0000 7fbef5200640 1 -- 172.21.15.43:0/3387367713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbef0112a30 msgr2=0x7fbef019e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:12.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.622+0000 7fbef5200640 1 --2- 172.21.15.43:0/3387367713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbef0112a30 0x7fbef019e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:12.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.622+0000 7fbef5200640 1 -- 172.21.15.43:0/3387367713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef019efe0 con 0x7fbef0076720 2024-09-19T05:12:12.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.622+0000 7fbeefe00640 1 --2- 172.21.15.43:0/3387367713 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbef0077040 0x7fbef01a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:12:12.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.622+0000 7fbef5c00640 1 --2- 172.21.15.43:0/3387367713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbef0112a30 0x7fbef019e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:12:12.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.622+0000 7fbef5200640 1 --2- 172.21.15.43:0/3387367713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbef0076720 0x7fbef01a46f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fbee4006010 tx=0x7fbee40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:12.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.623+0000 7fbeed600640 1 -- 172.21.15.43:0/3387367713 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee403f030 con 0x7fbef0076720 2024-09-19T05:12:12.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.623+0000 7fbeed600640 1 -- 172.21.15.43:0/3387367713 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbee4003ce0 con 0x7fbef0076720 2024-09-19T05:12:12.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.623+0000 7fbeed600640 1 -- 172.21.15.43:0/3387367713 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee4035070 con 0x7fbef0076720 2024-09-19T05:12:12.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.623+0000 7fbef7c00640 1 -- 172.21.15.43:0/3387367713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbef019f270 con 0x7fbef0076720 2024-09-19T05:12:12.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.623+0000 7fbef7c00640 1 -- 172.21.15.43:0/3387367713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbef01ab5c0 con 0x7fbef0076720 2024-09-19T05:12:12.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.626+0000 7fbefa6cb740 1 Processor -- start 2024-09-19T05:12:12.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.626+0000 7fbeed600640 1 -- 172.21.15.43:0/3387367713 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fbee4042030 con 0x7fbef0076720 2024-09-19T05:12:12.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.626+0000 7fbefa6cb740 1 -- start start 2024-09-19T05:12:12.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.626+0000 7fbefa6cb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580ec4f0ac0 0x5580ec4f0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:12.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.626+0000 7fbefa6cb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5580ec4f13e0 0x5580ec4fb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:12.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.626+0000 7fbefa6cb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580ec4fbca0 0x5580ec4fe090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:12.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.626+0000 7fbefa6cb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5580ec457290 con 0x5580ec4f13e0 2024-09-19T05:12:12.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.626+0000 7fbefa6cb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5580ec3feb40 con 0x5580ec4f0ac0 2024-09-19T05:12:12.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.626+0000 7fbefa6cb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5580ec453250 con 0x5580ec4fbca0 2024-09-19T05:12:12.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.627+0000 7fbef5200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5580ec4f13e0 0x5580ec4fb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:12.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.627+0000 7fbef5c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580ec4f0ac0 0x5580ec4f0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:12.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.627+0000 7fbef5c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580ec4f0ac0 0x5580ec4f0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:55088/0 (socket says 172.21.15.43:55088) 2024-09-19T05:12:12.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.627+0000 7fbef5c00640 1 -- 172.21.15.43:0/2423243811 learned_addr learned my addr 172.21.15.43:0/2423243811 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:12.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.627+0000 7fbeefe00640 1 --2- 172.21.15.43:0/2423243811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580ec4fbca0 0x5580ec4fe090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:12.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.627+0000 7fbef5c00640 1 -- 172.21.15.43:0/2423243811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580ec4fbca0 msgr2=0x5580ec4fe090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:12.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.627+0000 7fbef5c00640 1 --2- 172.21.15.43:0/2423243811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580ec4fbca0 0x5580ec4fe090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:12.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.627+0000 7fbef5c00640 1 -- 172.21.15.43:0/2423243811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5580ec4f13e0 msgr2=0x5580ec4fb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:12.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.627+0000 7fbef5c00640 1 --2- 172.21.15.43:0/2423243811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5580ec4f13e0 0x5580ec4fb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:12.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.627+0000 7fbef5c00640 1 -- 172.21.15.43:0/2423243811 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580ec1c86a0 con 0x5580ec4f0ac0 2024-09-19T05:12:12.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.627+0000 7fbeefe00640 1 --2- 172.21.15.43:0/2423243811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580ec4fbca0 0x5580ec4fe090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:12:12.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.628+0000 7fbef5c00640 1 --2- 172.21.15.43:0/2423243811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580ec4f0ac0 0x5580ec4f0ea0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbee80107a0 tx=0x7fbee8010ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5289568722fb7348 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:12.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.628+0000 7fbeed600640 1 --2- 172.21.15.43:0/3387367713 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fbed80a3430 0x7fbed80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:12.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.628+0000 7fbee2000640 1 -- 172.21.15.43:0/2423243811 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee800e930 con 0x5580ec4f0ac0 2024-09-19T05:12:12.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.628+0000 7fbee2000640 1 -- 172.21.15.43:0/2423243811 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbee8011440 con 0x5580ec4f0ac0 2024-09-19T05:12:12.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.628+0000 7fbee2000640 1 -- 172.21.15.43:0/2423243811 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee80195d0 con 0x5580ec4f0ac0 2024-09-19T05:12:12.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.628+0000 7fbeed600640 1 -- 172.21.15.43:0/3387367713 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8586+0+0 (secure 0 0 0) 0x7fbee40f0270 con 0x7fbef0076720 2024-09-19T05:12:12.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.628+0000 7fbef5200640 1 --2- 172.21.15.43:0/3387367713 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fbed80a3430 0x7fbed80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:12.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.629+0000 7fbefa6cb740 1 -- 172.21.15.43:0/2423243811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580ec4f0ac0 msgr2=0x5580ec4f0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:12.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.629+0000 7fbefa6cb740 1 --2- 172.21.15.43:0/2423243811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580ec4f0ac0 0x5580ec4f0ea0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbee80107a0 tx=0x7fbee8010ba0 comp rx=0 tx=0).stop 2024-09-19T05:12:12.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.629+0000 7fbef5200640 1 --2- 172.21.15.43:0/3387367713 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fbed80a3430 0x7fbed80a58f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x5580ec4edeb0 tx=0x7fbee40f3d20 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:12.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.629+0000 7fbefa6cb740 1 -- 172.21.15.43:0/2423243811 shutdown_connections 2024-09-19T05:12:12.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.629+0000 7fbefa6cb740 1 --2- 172.21.15.43:0/2423243811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580ec4fbca0 0x5580ec4fe090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:12.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.629+0000 7fbefa6cb740 1 --2- 172.21.15.43:0/2423243811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5580ec4f13e0 0x5580ec4fb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:12.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.629+0000 7fbefa6cb740 1 --2- 172.21.15.43:0/2423243811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580ec4f0ac0 0x5580ec4f0ea0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:12.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.629+0000 7fbefa6cb740 1 -- 172.21.15.43:0/2423243811 >> 172.21.15.43:0/2423243811 conn(0x5580ec4ed050 msgr2=0x5580ec4ed430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:12.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.629+0000 7fbefa6cb740 1 -- 172.21.15.43:0/2423243811 shutdown_connections 2024-09-19T05:12:12.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.629+0000 7fbefa6cb740 1 -- 172.21.15.43:0/2423243811 wait complete. 2024-09-19T05:12:12.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.629+0000 7fbefa6cb740 1 -- start start 2024-09-19T05:12:12.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.630+0000 7fbefa6cb740 1 Processor -- start 2024-09-19T05:12:12.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.630+0000 7fbefa6cb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5580ec4f0ac0 0x5580ec4fb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:12.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.630+0000 7fbefa6cb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580ec4f13e0 0x5580ec4f8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:12.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.630+0000 7fbefa6cb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580ec4fbca0 0x5580ec4f8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:12.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.630+0000 7fbefa6cb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5580ec4fe910 con 0x5580ec4f0ac0 2024-09-19T05:12:12.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.630+0000 7fbefa6cb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5580ec411fa0 con 0x5580ec4f13e0 2024-09-19T05:12:12.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.630+0000 7fbefa6cb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5580ec4fea90 con 0x5580ec4fbca0 2024-09-19T05:12:12.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.630+0000 7fbefa6cb740 10 client.0 _ll_register_callbacks cb 0x5580ec165ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:12:12.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.630+0000 7fbef5c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5580ec4f0ac0 0x5580ec4fb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:12.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.631+0000 7fbeefe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580ec4f13e0 0x5580ec4f8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:12.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.631+0000 7fbeefe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580ec4f13e0 0x5580ec4f8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:55102/0 (socket says 172.21.15.43:55102) 2024-09-19T05:12:12.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.631+0000 7fbeefe00640 1 -- 172.21.15.43:0/501856986 learned_addr learned my addr 172.21.15.43:0/501856986 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:12.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.631+0000 7fbef5c00640 1 --2- 172.21.15.43:0/501856986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580ec4fbca0 0x5580ec4f8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:12.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.631+0000 7fbeefe00640 1 -- 172.21.15.43:0/501856986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580ec4fbca0 msgr2=0x5580ec4f8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:12.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.631+0000 7fbeefe00640 1 --2- 172.21.15.43:0/501856986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580ec4fbca0 0x5580ec4f8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:12.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.631+0000 7fbeefe00640 1 -- 172.21.15.43:0/501856986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5580ec4f0ac0 msgr2=0x5580ec4fb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:12.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.631+0000 7fbeefe00640 1 --2- 172.21.15.43:0/501856986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5580ec4f0ac0 0x5580ec4fb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:12.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.631+0000 7fbeefe00640 1 -- 172.21.15.43:0/501856986 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580ec4f95e0 con 0x5580ec4f13e0 2024-09-19T05:12:12.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.631+0000 7fbef5c00640 1 --2- 172.21.15.43:0/501856986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5580ec4f0ac0 0x5580ec4fb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:12:12.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.631+0000 7fbeefe00640 1 --2- 172.21.15.43:0/501856986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580ec4f13e0 0x5580ec4f8850 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbedc012a10 tx=0x7fbedc012ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34fa7c5e7e0d0875 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:12.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.631+0000 7fbed6a00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:12:12.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.632+0000 7fbed6a00640 1 -- 172.21.15.43:0/501856986 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc00fa60 con 0x5580ec4f13e0 2024-09-19T05:12:12.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.632+0000 7fbefa6cb740 1 -- 172.21.15.43:0/501856986 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbee8011440 con 0x5580ec4f13e0 2024-09-19T05:12:12.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.632+0000 7fbed6a00640 1 -- 172.21.15.43:0/501856986 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8586+0+0 (secure 0 0 0) 0x7fbedc00fc00 con 0x5580ec4f13e0 2024-09-19T05:12:12.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.632+0000 7fbed6a00640 1 -- 172.21.15.43:0/501856986 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbedc00fe00 con 0x5580ec4f13e0 2024-09-19T05:12:12.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.632+0000 7fbefa6cb740 10 client.26942 fetch_fsmap learned FSMap version 29 2024-09-19T05:12:12.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.632+0000 7fbefa6cb740 1 -- 172.21.15.43:0/501856986 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5580ec50a8d0 con 0x5580ec4f13e0 2024-09-19T05:12:12.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.632+0000 7fbed6a00640 1 -- 172.21.15.43:0/501856986 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbee8011440 con 0x5580ec4f13e0 2024-09-19T05:12:12.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.632+0000 7fbed6a00640 1 -- 172.21.15.43:0/501856986 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbedc022030 con 0x5580ec4f13e0 2024-09-19T05:12:12.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.633+0000 7fbefa6cb740 10 client.26942 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:12:12.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.633+0000 7fbefa6cb740 10 client.26942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:12:12.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.633+0000 7fbefa6cb740 20 client.26942 populate_metadata read hostname 'smithi043' 2024-09-19T05:12:12.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.633+0000 7fbefa6cb740 1 --2- 172.21.15.43:0/501856986 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5580ec50bda0 0x5580ec50e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:12.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.633+0000 7fbefa6cb740 4 client.26942 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:12:12.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.633+0000 7fbefa6cb740 1 -- 172.21.15.43:0/501856986 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580ec464800 con 0x5580ec50bda0 2024-09-19T05:12:12.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.633+0000 7fbef5c00640 1 --2- 172.21.15.43:0/501856986 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5580ec50bda0 0x5580ec50e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:12.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.633+0000 7fbef5c00640 1 --2- 172.21.15.43:0/501856986 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5580ec50bda0 0x5580ec50e220 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4929c5e98694b69 server_cookie=8b0c64840906f454 in_seq=0 out_seq=0 2024-09-19T05:12:12.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.633+0000 7fbed6a00640 10 client.26942 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:12:12.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.634+0000 7fbed6a00640 1 -- 172.21.15.43:0/501856986 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5580ec464800 con 0x5580ec50bda0 2024-09-19T05:12:12.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.634+0000 7fbed6a00640 10 client.26942 handle_command_reply: tid=0 2024-09-19T05:12:12.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.634+0000 7fbefa6cb740 1 client.26942 shutdown 2024-09-19T05:12:12.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.634+0000 7fbefa6cb740 1 -- 172.21.15.43:0/501856986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580ec4f13e0 msgr2=0x5580ec4f8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:12.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.634+0000 7fbefa6cb740 1 --2- 172.21.15.43:0/501856986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580ec4f13e0 0x5580ec4f8850 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbedc012a10 tx=0x7fbedc012ed0 comp rx=0 tx=0).stop 2024-09-19T05:12:12.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.635+0000 7fbefa6cb740 1 -- 172.21.15.43:0/501856986 shutdown_connections 2024-09-19T05:12:12.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.635+0000 7fbefa6cb740 1 --2- 172.21.15.43:0/501856986 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5580ec50bda0 0x5580ec50e220 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:12.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.635+0000 7fbefa6cb740 1 --2- 172.21.15.43:0/501856986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580ec4fbca0 0x5580ec4f8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:12.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.635+0000 7fbefa6cb740 1 --2- 172.21.15.43:0/501856986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580ec4f13e0 0x5580ec4f8850 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:12.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.635+0000 7fbefa6cb740 1 --2- 172.21.15.43:0/501856986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5580ec4f0ac0 0x5580ec4fb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:12.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.635+0000 7fbefa6cb740 1 -- 172.21.15.43:0/501856986 >> 172.21.15.43:0/501856986 conn(0x5580ec4ed050 msgr2=0x5580ec4f2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:12.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.635+0000 7fbed6a00640 0 client.26942 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:12:12.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.635+0000 7fbefa6cb740 1 -- 172.21.15.43:0/501856986 shutdown_connections 2024-09-19T05:12:12.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.635+0000 7fbefa6cb740 1 -- 172.21.15.43:0/501856986 wait complete. 2024-09-19T05:12:12.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.635+0000 7fbefa6cb740 20 client.26942 trim_cache size 0 max 16384 2024-09-19T05:12:12.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.653+0000 7fbefa6cb740 1 Processor -- start 2024-09-19T05:12:12.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.653+0000 7fbefa6cb740 1 -- start start 2024-09-19T05:12:12.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.653+0000 7fbefa6cb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580ec4ed050 0x5580ec4fa7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:12.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.653+0000 7fbefa6cb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580ec4f0ac0 0x5580ec513af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:12.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.654+0000 7fbefa6cb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5580ec4f13e0 0x5580ec514030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:12.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.654+0000 7fbefa6cb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbef01b4020 con 0x5580ec4f13e0 2024-09-19T05:12:12.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.654+0000 7fbefa6cb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbef01b5b60 con 0x5580ec4ed050 2024-09-19T05:12:12.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.654+0000 7fbefa6cb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbef01b46f0 con 0x5580ec4f0ac0 2024-09-19T05:12:12.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.654+0000 7fbef5c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5580ec4f13e0 0x5580ec514030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:12.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.654+0000 7fbef5c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5580ec4f13e0 0x5580ec514030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34874/0 (socket says 172.21.15.43:34874) 2024-09-19T05:12:12.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.654+0000 7fbef5c00640 1 -- 172.21.15.43:0/1818388143 learned_addr learned my addr 172.21.15.43:0/1818388143 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:12.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.654+0000 7fbeefe00640 1 --2- 172.21.15.43:0/1818388143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580ec4f0ac0 0x5580ec513af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:12.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.654+0000 7fbef5c00640 1 --2- 172.21.15.43:0/1818388143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580ec4ed050 0x5580ec4fa7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:12.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.654+0000 7fbeefe00640 1 -- 172.21.15.43:0/1818388143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580ec4ed050 msgr2=0x5580ec4fa7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:12.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.654+0000 7fbeefe00640 1 --2- 172.21.15.43:0/1818388143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580ec4ed050 0x5580ec4fa7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:12.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.654+0000 7fbeefe00640 1 -- 172.21.15.43:0/1818388143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5580ec4f13e0 msgr2=0x5580ec514030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:12.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.654+0000 7fbeefe00640 1 --2- 172.21.15.43:0/1818388143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5580ec4f13e0 0x5580ec514030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:12.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.654+0000 7fbeefe00640 1 -- 172.21.15.43:0/1818388143 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580ec4edb90 con 0x5580ec4f0ac0 2024-09-19T05:12:12.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.654+0000 7fbef5c00640 1 --2- 172.21.15.43:0/1818388143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5580ec4f13e0 0x5580ec514030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:12:12.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.655+0000 7fbeefe00640 1 --2- 172.21.15.43:0/1818388143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580ec4f0ac0 0x5580ec513af0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0129e0 tx=0x7fbedc0123c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=594b5a13dee1a7f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:12.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.655+0000 7fbee1600640 1 -- 172.21.15.43:0/1818388143 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc00f790 con 0x5580ec4f0ac0 2024-09-19T05:12:12.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.655+0000 7fbee1600640 1 -- 172.21.15.43:0/1818388143 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbedc00fa60 con 0x5580ec4f0ac0 2024-09-19T05:12:12.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.655+0000 7fbee1600640 1 -- 172.21.15.43:0/1818388143 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc003a50 con 0x5580ec4f0ac0 2024-09-19T05:12:12.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.655+0000 7fbefa6cb740 1 -- 172.21.15.43:0/1818388143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580ec4f0ac0 msgr2=0x5580ec513af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:12.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.655+0000 7fbefa6cb740 1 --2- 172.21.15.43:0/1818388143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580ec4f0ac0 0x5580ec513af0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0129e0 tx=0x7fbedc0123c0 comp rx=0 tx=0).stop 2024-09-19T05:12:12.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.656+0000 7fbefa6cb740 1 -- 172.21.15.43:0/1818388143 shutdown_connections 2024-09-19T05:12:12.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.656+0000 7fbefa6cb740 1 --2- 172.21.15.43:0/1818388143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5580ec4f13e0 0x5580ec514030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:12.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.656+0000 7fbefa6cb740 1 --2- 172.21.15.43:0/1818388143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580ec4f0ac0 0x5580ec513af0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:12.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.656+0000 7fbefa6cb740 1 --2- 172.21.15.43:0/1818388143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580ec4ed050 0x5580ec4fa7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:12.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.656+0000 7fbefa6cb740 1 -- 172.21.15.43:0/1818388143 >> 172.21.15.43:0/1818388143 conn(0x5580ec4f8ef0 msgr2=0x5580ec4fa0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:12.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.656+0000 7fbefa6cb740 1 -- 172.21.15.43:0/1818388143 shutdown_connections 2024-09-19T05:12:12.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.656+0000 7fbefa6cb740 1 -- 172.21.15.43:0/1818388143 wait complete. 2024-09-19T05:12:12.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.656+0000 7fbefa6cb740 1 -- start start 2024-09-19T05:12:12.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.656+0000 7fbefa6cb740 1 Processor -- start 2024-09-19T05:12:12.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.657+0000 7fbefa6cb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5580ec4ed050 0x5580ec503d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:12.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.657+0000 7fbefa6cb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580ec4f0ac0 0x5580ec504250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:12.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.657+0000 7fbefa6cb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580ec4f13e0 0x5580ec500ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:12.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.657+0000 7fbefa6cb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbef01b5490 con 0x5580ec4ed050 2024-09-19T05:12:12.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.657+0000 7fbefa6cb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbef01b4dc0 con 0x5580ec4f13e0 2024-09-19T05:12:12.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.657+0000 7fbefa6cb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbef01b2020 con 0x5580ec4f0ac0 2024-09-19T05:12:12.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.657+0000 7fbefa6cb740 10 client.0 _ll_register_callbacks cb 0x5580ec4f97b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:12:12.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.657+0000 7fbef5c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5580ec4ed050 0x5580ec503d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:12.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.657+0000 7fbeefe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580ec4f0ac0 0x5580ec504250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:12.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.657+0000 7fbeefe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580ec4f0ac0 0x5580ec504250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:50382/0 (socket says 172.21.15.43:50382) 2024-09-19T05:12:12.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.657+0000 7fbef5c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5580ec4ed050 0x5580ec503d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34888/0 (socket says 172.21.15.43:34888) 2024-09-19T05:12:12.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.657+0000 7fbeefe00640 1 -- 172.21.15.43:0/65311831 learned_addr learned my addr 172.21.15.43:0/65311831 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:12.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.657+0000 7fbef5c00640 1 --2- 172.21.15.43:0/65311831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580ec4f13e0 0x5580ec500ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:12.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.658+0000 7fbef5c00640 1 -- 172.21.15.43:0/65311831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580ec4f0ac0 msgr2=0x5580ec504250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:12.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.658+0000 7fbef5c00640 1 --2- 172.21.15.43:0/65311831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580ec4f0ac0 0x5580ec504250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:12.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.658+0000 7fbef5c00640 1 -- 172.21.15.43:0/65311831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580ec4f13e0 msgr2=0x5580ec500ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:12.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.658+0000 7fbef5c00640 1 --2- 172.21.15.43:0/65311831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580ec4f13e0 0x5580ec500ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:12.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.658+0000 7fbef5c00640 1 -- 172.21.15.43:0/65311831 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580ec4f08f0 con 0x5580ec4ed050 2024-09-19T05:12:12.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.658+0000 7fbeefe00640 1 --2- 172.21.15.43:0/65311831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580ec4f0ac0 0x5580ec504250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:12:12.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.658+0000 7fbef5c00640 1 --2- 172.21.15.43:0/65311831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5580ec4ed050 0x5580ec503d10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbee8011e50 tx=0x7fbee8010ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74e9585533a9097 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:12.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.658+0000 7fbed6a00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:12:12.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.658+0000 7fbed6a00640 1 -- 172.21.15.43:0/65311831 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee800e9f0 con 0x5580ec4ed050 2024-09-19T05:12:12.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.658+0000 7fbed6a00640 1 -- 172.21.15.43:0/65311831 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8586+0+0 (secure 0 0 0) 0x7fbee800eb90 con 0x5580ec4ed050 2024-09-19T05:12:12.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.658+0000 7fbefa6cb740 1 -- 172.21.15.43:0/65311831 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbedc00fa60 con 0x5580ec4ed050 2024-09-19T05:12:12.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.658+0000 7fbed6a00640 1 -- 172.21.15.43:0/65311831 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbee800ed90 con 0x5580ec4ed050 2024-09-19T05:12:12.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.659+0000 7fbefa6cb740 10 client.16926 fetch_fsmap learned FSMap version 29 2024-09-19T05:12:12.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.659+0000 7fbefa6cb740 1 -- 172.21.15.43:0/65311831 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5580ec577110 con 0x5580ec4ed050 2024-09-19T05:12:12.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.659+0000 7fbed6a00640 1 -- 172.21.15.43:0/65311831 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbedc00fa60 con 0x5580ec4ed050 2024-09-19T05:12:12.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.659+0000 7fbed6a00640 1 -- 172.21.15.43:0/65311831 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbee8036020 con 0x5580ec4ed050 2024-09-19T05:12:12.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.659+0000 7fbefa6cb740 10 client.16926 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:12:12.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.659+0000 7fbefa6cb740 10 client.16926 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:12:12.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.659+0000 7fbefa6cb740 20 client.16926 populate_metadata read hostname 'smithi043' 2024-09-19T05:12:12.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.659+0000 7fbefa6cb740 1 --2- 172.21.15.43:0/65311831 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5580ec4fbca0 0x5580ec579800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:12.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.659+0000 7fbefa6cb740 4 client.16926 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T05:12:12.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.659+0000 7fbefa6cb740 1 -- 172.21.15.43:0/65311831 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5580ec45ed10 con 0x5580ec4fbca0 2024-09-19T05:12:12.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.660+0000 7fbeefe00640 1 --2- 172.21.15.43:0/65311831 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5580ec4fbca0 0x5580ec579800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:12.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.660+0000 7fbeefe00640 1 --2- 172.21.15.43:0/65311831 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5580ec4fbca0 0x5580ec579800 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f53a901b5aaf8821 server_cookie=6f993bdbabef615 in_seq=0 out_seq=0 2024-09-19T05:12:12.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.660+0000 7fbed6a00640 10 client.16926 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:12:12.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.661+0000 7fbed6a00640 1 -- 172.21.15.43:0/65311831 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5580ec45ed10 con 0x5580ec4fbca0 2024-09-19T05:12:12.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.661+0000 7fbed6a00640 10 client.16926 handle_command_reply: tid=0 2024-09-19T05:12:12.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.661+0000 7fbefa6cb740 1 client.16926 shutdown 2024-09-19T05:12:12.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.661+0000 7fbefa6cb740 1 -- 172.21.15.43:0/65311831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5580ec4ed050 msgr2=0x5580ec503d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:12.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.661+0000 7fbefa6cb740 1 --2- 172.21.15.43:0/65311831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5580ec4ed050 0x5580ec503d10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbee8011e50 tx=0x7fbee8010ba0 comp rx=0 tx=0).stop 2024-09-19T05:12:12.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.661+0000 7fbefa6cb740 1 -- 172.21.15.43:0/65311831 shutdown_connections 2024-09-19T05:12:12.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.661+0000 7fbefa6cb740 1 --2- 172.21.15.43:0/65311831 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5580ec4fbca0 0x5580ec579800 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:12.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.661+0000 7fbefa6cb740 1 --2- 172.21.15.43:0/65311831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580ec4f13e0 0x5580ec500ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:12.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.661+0000 7fbefa6cb740 1 --2- 172.21.15.43:0/65311831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580ec4f0ac0 0x5580ec504250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:12.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.661+0000 7fbefa6cb740 1 --2- 172.21.15.43:0/65311831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5580ec4ed050 0x5580ec503d10 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:12.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.662+0000 7fbefa6cb740 1 -- 172.21.15.43:0/65311831 >> 172.21.15.43:0/65311831 conn(0x5580ec4f8ef0 msgr2=0x5580ec4eb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:12.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.662+0000 7fbed6a00640 0 client.16926 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:12:12.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.662+0000 7fbefa6cb740 1 -- 172.21.15.43:0/65311831 shutdown_connections 2024-09-19T05:12:12.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.662+0000 7fbefa6cb740 1 -- 172.21.15.43:0/65311831 wait complete. 2024-09-19T05:12:12.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.662+0000 7fbefa6cb740 20 client.16926 trim_cache size 0 max 16384 2024-09-19T05:12:12.667 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:12:12.667 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-09-19T05:12:12.667 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "52093721-6a42-4fd5-a208-37c1c859e1f6", 2024-09-19T05:12:12.667 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-09-19T05:12:12.667 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:12:12.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.663+0000 7fbef7c00640 1 -- 172.21.15.43:0/3387367713 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fbed80a3430 msgr2=0x7fbed80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:12.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.663+0000 7fbef7c00640 1 --2- 172.21.15.43:0/3387367713 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fbed80a3430 0x7fbed80a58f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x5580ec4edeb0 tx=0x7fbee40f3d20 comp rx=0 tx=0).stop 2024-09-19T05:12:12.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.663+0000 7fbef7c00640 1 -- 172.21.15.43:0/3387367713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbef0076720 msgr2=0x7fbef01a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:12.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.663+0000 7fbef7c00640 1 --2- 172.21.15.43:0/3387367713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbef0076720 0x7fbef01a46f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fbee4006010 tx=0x7fbee40042f0 comp rx=0 tx=0).stop 2024-09-19T05:12:12.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.663+0000 7fbef7c00640 1 -- 172.21.15.43:0/3387367713 shutdown_connections 2024-09-19T05:12:12.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.663+0000 7fbef7c00640 1 --2- 172.21.15.43:0/3387367713 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fbed80a3430 0x7fbed80a58f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:12.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.663+0000 7fbef7c00640 1 --2- 172.21.15.43:0/3387367713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbef0112a30 0x7fbef019e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:12.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.663+0000 7fbef7c00640 1 --2- 172.21.15.43:0/3387367713 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbef0077040 0x7fbef01a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:12.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.663+0000 7fbef7c00640 1 --2- 172.21.15.43:0/3387367713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbef0076720 0x7fbef01a46f0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:12.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.663+0000 7fbef7c00640 1 -- 172.21.15.43:0/3387367713 >> 172.21.15.43:0/3387367713 conn(0x7fbef00ffa90 msgr2=0x7fbef0101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:12.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.663+0000 7fbef7c00640 1 -- 172.21.15.43:0/3387367713 shutdown_connections 2024-09-19T05:12:12.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:12.663+0000 7fbef7c00640 1 -- 172.21.15.43:0/3387367713 wait complete. 2024-09-19T05:12:13.383 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:13 smithi043 ceph-mon[30800]: scrub queued for path: / 2024-09-19T05:12:13.383 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:13 smithi043 ceph-mon[30800]: scrub summary: idle+waiting paths [/] 2024-09-19T05:12:13.383 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:13 smithi043 ceph-mon[30800]: scrub summary: active paths [/] 2024-09-19T05:12:13.383 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:13 smithi043 ceph-mon[30800]: scrub summary: idle+waiting paths [/] 2024-09-19T05:12:13.383 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:13 smithi043 ceph-mon[30800]: scrub summary: idle 2024-09-19T05:12:13.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:13 smithi151 ceph-mon[35425]: scrub queued for path: / 2024-09-19T05:12:13.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:13 smithi151 ceph-mon[35425]: scrub summary: idle+waiting paths [/] 2024-09-19T05:12:13.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:13 smithi151 ceph-mon[35425]: scrub summary: active paths [/] 2024-09-19T05:12:13.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:13 smithi151 ceph-mon[35425]: scrub summary: idle+waiting paths [/] 2024-09-19T05:12:13.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:13 smithi151 ceph-mon[35425]: scrub summary: idle 2024-09-19T05:12:13.423 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:12:13.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:13 smithi096 ceph-mon[35085]: scrub queued for path: / 2024-09-19T05:12:13.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:13 smithi096 ceph-mon[35085]: scrub summary: idle+waiting paths [/] 2024-09-19T05:12:13.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:13 smithi096 ceph-mon[35085]: scrub summary: active paths [/] 2024-09-19T05:12:13.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:13 smithi096 ceph-mon[35085]: scrub summary: idle+waiting paths [/] 2024-09-19T05:12:13.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:13 smithi096 ceph-mon[35085]: scrub summary: idle 2024-09-19T05:12:13.802 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:12:14.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:14 smithi043 ceph-mon[30800]: pgmap v233: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T05:12:14.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.403+0000 7ff52a000640 1 -- 172.21.15.43:0/3461914989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff524106f20 msgr2=0x7ff52410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:14.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.403+0000 7ff52a000640 1 --2- 172.21.15.43:0/3461914989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff524106f20 0x7ff52410eaa0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ff51c009ca0 tx=0x7ff51c02a8c0 comp rx=0 tx=0).stop 2024-09-19T05:12:14.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.403+0000 7ff52a000640 1 -- 172.21.15.43:0/3461914989 shutdown_connections 2024-09-19T05:12:14.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.403+0000 7ff52a000640 1 --2- 172.21.15.43:0/3461914989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff524106f20 0x7ff52410eaa0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:14.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.403+0000 7ff52a000640 1 --2- 172.21.15.43:0/3461914989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5241027a0 0x7ff5241067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:14.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.403+0000 7ff52a000640 1 --2- 172.21.15.43:0/3461914989 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff524101df0 0x7ff5241021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:14.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.403+0000 7ff52a000640 1 -- 172.21.15.43:0/3461914989 >> 172.21.15.43:0/3461914989 conn(0x7ff5240fd8b0 msgr2=0x7ff5240ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:14.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.403+0000 7ff52a000640 1 -- 172.21.15.43:0/3461914989 shutdown_connections 2024-09-19T05:12:14.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.403+0000 7ff52a000640 1 -- 172.21.15.43:0/3461914989 wait complete. 2024-09-19T05:12:14.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:14 smithi151 ceph-mon[35425]: pgmap v233: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T05:12:14.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.404+0000 7ff52a000640 1 Processor -- start 2024-09-19T05:12:14.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.404+0000 7ff52a000640 1 -- start start 2024-09-19T05:12:14.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.405+0000 7ff52a000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff524101df0 0x7ff52419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:14.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.405+0000 7ff52a000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5241027a0 0x7ff52419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:14.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.405+0000 7ff52a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff524106f20 0x7ff52419ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:14.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.405+0000 7ff52a000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff524111f80 con 0x7ff524106f20 2024-09-19T05:12:14.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.405+0000 7ff52a000640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff524111e00 con 0x7ff524101df0 2024-09-19T05:12:14.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.405+0000 7ff52a000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff524112100 con 0x7ff5241027a0 2024-09-19T05:12:14.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.405+0000 7ff523400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff524101df0 0x7ff52419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:14.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.405+0000 7ff523e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff524106f20 0x7ff52419ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:14.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.405+0000 7ff523400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff524101df0 0x7ff52419a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:55144/0 (socket says 172.21.15.43:55144) 2024-09-19T05:12:14.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.405+0000 7ff523400640 1 -- 172.21.15.43:0/1257297477 learned_addr learned my addr 172.21.15.43:0/1257297477 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:14.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.405+0000 7ff522a00640 1 --2- 172.21.15.43:0/1257297477 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5241027a0 0x7ff52419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:14.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.405+0000 7ff523400640 1 -- 172.21.15.43:0/1257297477 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5241027a0 msgr2=0x7ff52419aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:14.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.405+0000 7ff523400640 1 --2- 172.21.15.43:0/1257297477 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5241027a0 0x7ff52419aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:14.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.405+0000 7ff523400640 1 -- 172.21.15.43:0/1257297477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff524106f20 msgr2=0x7ff52419ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:14.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.405+0000 7ff523400640 1 --2- 172.21.15.43:0/1257297477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff524106f20 0x7ff52419ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:14.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.405+0000 7ff523400640 1 -- 172.21.15.43:0/1257297477 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff52419f560 con 0x7ff524101df0 2024-09-19T05:12:14.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.406+0000 7ff523400640 1 --2- 172.21.15.43:0/1257297477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff524101df0 0x7ff52419a5b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff51400e9f0 tx=0x7ff51400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:14.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.406+0000 7ff51be00640 1 -- 172.21.15.43:0/1257297477 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51400cd80 con 0x7ff524101df0 2024-09-19T05:12:14.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.406+0000 7ff52a000640 1 -- 172.21.15.43:0/1257297477 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff52419f850 con 0x7ff524101df0 2024-09-19T05:12:14.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.406+0000 7ff51be00640 1 -- 172.21.15.43:0/1257297477 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff514005500 con 0x7ff524101df0 2024-09-19T05:12:14.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.406+0000 7ff51be00640 1 -- 172.21.15.43:0/1257297477 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff514010690 con 0x7ff524101df0 2024-09-19T05:12:14.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.406+0000 7ff52a000640 1 -- 172.21.15.43:0/1257297477 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5241a7130 con 0x7ff524101df0 2024-09-19T05:12:14.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.410+0000 7ff51be00640 1 -- 172.21.15.43:0/1257297477 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ff5140046c0 con 0x7ff524101df0 2024-09-19T05:12:14.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.411+0000 7ff52c5fd740 1 Processor -- start 2024-09-19T05:12:14.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.411+0000 7ff52c5fd740 1 -- start start 2024-09-19T05:12:14.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.411+0000 7ff51be00640 1 --2- 172.21.15.43:0/1257297477 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff5080a33b0 0x7ff5080a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:14.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.411+0000 7ff51be00640 1 -- 172.21.15.43:0/1257297477 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8586+0+0 (secure 0 0 0) 0x7ff5140d02c0 con 0x7ff524101df0 2024-09-19T05:12:14.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.411+0000 7ff52c5fd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191ece3b90 0x56191ece3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:14.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.411+0000 7ff523e00640 1 --2- 172.21.15.43:0/1257297477 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff5080a33b0 0x7ff5080a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:14.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.411+0000 7ff52c5fd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191ece44b0 0x56191ecee830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:14.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.411+0000 7ff52c5fd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56191eceed70 0x56191ecf1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:14.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.411+0000 7ff52c5fd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56191ec4a3e0 con 0x56191eceed70 2024-09-19T05:12:14.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.411+0000 7ff52c5fd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56191ebf1d80 con 0x56191ece3b90 2024-09-19T05:12:14.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.411+0000 7ff52c5fd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56191ec463a0 con 0x56191ece44b0 2024-09-19T05:12:14.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.412+0000 7ff523e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56191eceed70 0x56191ecf1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:14.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.412+0000 7ff523400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191ece3b90 0x56191ece3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:14.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.412+0000 7ff523400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191ece3b90 0x56191ece3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:55152/0 (socket says 172.21.15.43:55152) 2024-09-19T05:12:14.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.412+0000 7ff523400640 1 -- 172.21.15.43:0/3181728245 learned_addr learned my addr 172.21.15.43:0/3181728245 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:14.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.412+0000 7ff522a00640 1 --2- 172.21.15.43:0/3181728245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191ece44b0 0x56191ecee830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:14.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.412+0000 7ff523400640 1 -- 172.21.15.43:0/3181728245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191ece44b0 msgr2=0x56191ecee830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:14.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.412+0000 7ff523400640 1 --2- 172.21.15.43:0/3181728245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191ece44b0 0x56191ecee830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:14.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.412+0000 7ff523400640 1 -- 172.21.15.43:0/3181728245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56191eceed70 msgr2=0x56191ecf1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:14.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.412+0000 7ff523400640 1 --2- 172.21.15.43:0/3181728245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56191eceed70 0x56191ecf1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:14.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.412+0000 7ff523400640 1 -- 172.21.15.43:0/3181728245 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56191ec10ae0 con 0x56191ece3b90 2024-09-19T05:12:14.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.412+0000 7ff523e00640 1 --2- 172.21.15.43:0/3181728245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56191eceed70 0x56191ecf1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:12:14.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.413+0000 7ff523e00640 1 --2- 172.21.15.43:0/1257297477 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff5080a33b0 0x7ff5080a5870 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff51c004af0 tx=0x7ff51c031c90 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:14.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.413+0000 7ff523400640 1 --2- 172.21.15.43:0/3181728245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191ece3b90 0x56191ece3f70 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff514098960 tx=0x7ff51409c100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efddf909ec9de312 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:14.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.413+0000 7ff50fe00640 1 -- 172.21.15.43:0/3181728245 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5140b4020 con 0x56191ece3b90 2024-09-19T05:12:14.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.413+0000 7ff50fe00640 1 -- 172.21.15.43:0/3181728245 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff51409c500 con 0x56191ece3b90 2024-09-19T05:12:14.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.413+0000 7ff50fe00640 1 -- 172.21.15.43:0/3181728245 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5140a4ce0 con 0x56191ece3b90 2024-09-19T05:12:14.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.414+0000 7ff52c5fd740 1 -- 172.21.15.43:0/3181728245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191ece3b90 msgr2=0x56191ece3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:14.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.414+0000 7ff52c5fd740 1 --2- 172.21.15.43:0/3181728245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191ece3b90 0x56191ece3f70 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff514098960 tx=0x7ff51409c100 comp rx=0 tx=0).stop 2024-09-19T05:12:14.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.414+0000 7ff52c5fd740 1 -- 172.21.15.43:0/3181728245 shutdown_connections 2024-09-19T05:12:14.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.414+0000 7ff52c5fd740 1 --2- 172.21.15.43:0/3181728245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56191eceed70 0x56191ecf1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:14.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.414+0000 7ff52c5fd740 1 --2- 172.21.15.43:0/3181728245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191ece44b0 0x56191ecee830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:14.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.414+0000 7ff52c5fd740 1 --2- 172.21.15.43:0/3181728245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191ece3b90 0x56191ece3f70 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:14.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.414+0000 7ff52c5fd740 1 -- 172.21.15.43:0/3181728245 >> 172.21.15.43:0/3181728245 conn(0x56191ece0120 msgr2=0x56191ece0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:14.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.414+0000 7ff52c5fd740 1 -- 172.21.15.43:0/3181728245 shutdown_connections 2024-09-19T05:12:14.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.414+0000 7ff52c5fd740 1 -- 172.21.15.43:0/3181728245 wait complete. 2024-09-19T05:12:14.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.415+0000 7ff52c5fd740 1 -- start start 2024-09-19T05:12:14.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.415+0000 7ff52c5fd740 1 Processor -- start 2024-09-19T05:12:14.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.415+0000 7ff52c5fd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191ece3b90 0x56191ecebca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:14.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.415+0000 7ff52c5fd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56191ece44b0 0x56191ecec1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:14.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.415+0000 7ff52c5fd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191eceed70 0x56191ecec720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:14.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.415+0000 7ff52c5fd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56191ecf19e0 con 0x56191ece44b0 2024-09-19T05:12:14.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.415+0000 7ff52c5fd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56191ec050f0 con 0x56191eceed70 2024-09-19T05:12:14.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.415+0000 7ff52c5fd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56191ecf1b60 con 0x56191ece3b90 2024-09-19T05:12:14.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.415+0000 7ff52c5fd740 10 client.0 _ll_register_callbacks cb 0x56191ebf69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:12:14.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.416+0000 7ff522a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56191ece44b0 0x56191ecec1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:14.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.416+0000 7ff522a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56191ece44b0 0x56191ecec1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34908/0 (socket says 172.21.15.43:34908) 2024-09-19T05:12:14.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.416+0000 7ff522a00640 1 -- 172.21.15.43:0/3078303248 learned_addr learned my addr 172.21.15.43:0/3078303248 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:14.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.416+0000 7ff523400640 1 --2- 172.21.15.43:0/3078303248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191ece3b90 0x56191ecebca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:14.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.416+0000 7ff523e00640 1 --2- 172.21.15.43:0/3078303248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191eceed70 0x56191ecec720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:14.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.416+0000 7ff522a00640 1 -- 172.21.15.43:0/3078303248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191ece3b90 msgr2=0x56191ecebca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:14.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.416+0000 7ff522a00640 1 --2- 172.21.15.43:0/3078303248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191ece3b90 0x56191ecebca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:14.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.416+0000 7ff522a00640 1 -- 172.21.15.43:0/3078303248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191eceed70 msgr2=0x56191ecec720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:14.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.416+0000 7ff522a00640 1 --2- 172.21.15.43:0/3078303248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191eceed70 0x56191ecec720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:14.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.416+0000 7ff522a00640 1 -- 172.21.15.43:0/3078303248 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56191e9bb3c0 con 0x56191ece44b0 2024-09-19T05:12:14.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.416+0000 7ff523400640 1 --2- 172.21.15.43:0/3078303248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191ece3b90 0x56191ecebca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:12:14.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.416+0000 7ff523e00640 1 --2- 172.21.15.43:0/3078303248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191eceed70 0x56191ecec720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:12:14.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.417+0000 7ff522a00640 1 --2- 172.21.15.43:0/3078303248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56191ece44b0 0x56191ecec1e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ff51000d9d0 tx=0x7ff51000de90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7953c98d725d7019 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:14.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.417+0000 7ff50cc00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:12:14.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.417+0000 7ff50cc00640 1 -- 172.21.15.43:0/3078303248 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff510003ea0 con 0x56191ece44b0 2024-09-19T05:12:14.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.417+0000 7ff50cc00640 1 -- 172.21.15.43:0/3078303248 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8586+0+0 (secure 0 0 0) 0x7ff510011440 con 0x56191ece44b0 2024-09-19T05:12:14.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.417+0000 7ff52c5fd740 1 -- 172.21.15.43:0/3078303248 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff51409c500 con 0x56191ece44b0 2024-09-19T05:12:14.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.417+0000 7ff50cc00640 1 -- 172.21.15.43:0/3078303248 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff51001b640 con 0x56191ece44b0 2024-09-19T05:12:14.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.417+0000 7ff52c5fd740 10 client.16938 fetch_fsmap learned FSMap version 29 2024-09-19T05:12:14.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.417+0000 7ff52c5fd740 1 -- 172.21.15.43:0/3078303248 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56191ecfd740 con 0x56191ece44b0 2024-09-19T05:12:14.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.418+0000 7ff50cc00640 1 -- 172.21.15.43:0/3078303248 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff51409c500 con 0x56191ece44b0 2024-09-19T05:12:14.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.418+0000 7ff50cc00640 1 -- 172.21.15.43:0/3078303248 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff510019020 con 0x56191ece44b0 2024-09-19T05:12:14.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.418+0000 7ff52c5fd740 10 client.16938 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:12:14.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.418+0000 7ff52c5fd740 10 client.16938 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:12:14.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.418+0000 7ff52c5fd740 20 client.16938 populate_metadata read hostname 'smithi043' 2024-09-19T05:12:14.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.418+0000 7ff52c5fd740 1 --2- 172.21.15.43:0/3078303248 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x56191ecfeca0 0x56191ed01090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:14.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.418+0000 7ff52c5fd740 4 client.16938 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:12:14.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.418+0000 7ff52c5fd740 1 -- 172.21.15.43:0/3078303248 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56191ec57950 con 0x56191ecfeca0 2024-09-19T05:12:14.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.418+0000 7ff523400640 1 --2- 172.21.15.43:0/3078303248 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x56191ecfeca0 0x56191ed01090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:14.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.419+0000 7ff523400640 1 --2- 172.21.15.43:0/3078303248 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x56191ecfeca0 0x56191ed01090 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28642dfaf07d6849 server_cookie=b66f247e1bcfcebe in_seq=0 out_seq=0 2024-09-19T05:12:14.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.419+0000 7ff50cc00640 10 client.16938 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:12:14.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.419+0000 7ff50cc00640 1 -- 172.21.15.43:0/3078303248 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56191ec57950 con 0x56191ecfeca0 2024-09-19T05:12:14.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.419+0000 7ff50cc00640 10 client.16938 handle_command_reply: tid=0 2024-09-19T05:12:14.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.419+0000 7ff52c5fd740 1 client.16938 shutdown 2024-09-19T05:12:14.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.420+0000 7ff52c5fd740 1 -- 172.21.15.43:0/3078303248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56191ece44b0 msgr2=0x56191ecec1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:14.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.420+0000 7ff52c5fd740 1 --2- 172.21.15.43:0/3078303248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56191ece44b0 0x56191ecec1e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ff51000d9d0 tx=0x7ff51000de90 comp rx=0 tx=0).stop 2024-09-19T05:12:14.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.420+0000 7ff52c5fd740 1 -- 172.21.15.43:0/3078303248 shutdown_connections 2024-09-19T05:12:14.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.420+0000 7ff52c5fd740 1 --2- 172.21.15.43:0/3078303248 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x56191ecfeca0 0x56191ed01090 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:14.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.420+0000 7ff52c5fd740 1 --2- 172.21.15.43:0/3078303248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191eceed70 0x56191ecec720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:14.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.420+0000 7ff52c5fd740 1 --2- 172.21.15.43:0/3078303248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56191ece44b0 0x56191ecec1e0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:14.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.420+0000 7ff52c5fd740 1 --2- 172.21.15.43:0/3078303248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191ece3b90 0x56191ecebca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:14.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.420+0000 7ff52c5fd740 1 -- 172.21.15.43:0/3078303248 >> 172.21.15.43:0/3078303248 conn(0x56191ece0120 msgr2=0x56191ece5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:14.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.420+0000 7ff50cc00640 0 client.16938 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:12:14.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.420+0000 7ff52c5fd740 1 -- 172.21.15.43:0/3078303248 shutdown_connections 2024-09-19T05:12:14.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.420+0000 7ff52c5fd740 1 -- 172.21.15.43:0/3078303248 wait complete. 2024-09-19T05:12:14.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.420+0000 7ff52c5fd740 20 client.16938 trim_cache size 0 max 16384 2024-09-19T05:12:14.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.438+0000 7ff52c5fd740 1 Processor -- start 2024-09-19T05:12:14.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.439+0000 7ff52c5fd740 1 -- start start 2024-09-19T05:12:14.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.439+0000 7ff52c5fd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56191ece3b90 0x56191ecedc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:14.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.439+0000 7ff52c5fd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191ece44b0 0x56191ecec570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:14.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.439+0000 7ff52c5fd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191eceed70 0x56191ed06950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:14.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.439+0000 7ff52c5fd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff5241afbb0 con 0x56191ece3b90 2024-09-19T05:12:14.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.439+0000 7ff52c5fd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff5241b16f0 con 0x56191ece44b0 2024-09-19T05:12:14.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.439+0000 7ff52c5fd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff5241b0280 con 0x56191eceed70 2024-09-19T05:12:14.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.439+0000 7ff523400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56191ece3b90 0x56191ecedc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:14.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.439+0000 7ff523400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56191ece3b90 0x56191ecedc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34920/0 (socket says 172.21.15.43:34920) 2024-09-19T05:12:14.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.439+0000 7ff522a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191ece44b0 0x56191ecec570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:14.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.439+0000 7ff523400640 1 -- 172.21.15.43:0/4263668461 learned_addr learned my addr 172.21.15.43:0/4263668461 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:14.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.439+0000 7ff523e00640 1 --2- 172.21.15.43:0/4263668461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191eceed70 0x56191ed06950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:14.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.440+0000 7ff522a00640 1 -- 172.21.15.43:0/4263668461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191eceed70 msgr2=0x56191ed06950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:14.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.440+0000 7ff522a00640 1 --2- 172.21.15.43:0/4263668461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191eceed70 0x56191ed06950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:14.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.440+0000 7ff522a00640 1 -- 172.21.15.43:0/4263668461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56191ece3b90 msgr2=0x56191ecedc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:14.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.440+0000 7ff522a00640 1 --2- 172.21.15.43:0/4263668461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56191ece3b90 0x56191ecedc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:14.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.440+0000 7ff522a00640 1 -- 172.21.15.43:0/4263668461 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56191ece42e0 con 0x56191ece44b0 2024-09-19T05:12:14.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.440+0000 7ff522a00640 1 --2- 172.21.15.43:0/4263668461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191ece44b0 0x56191ecec570 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff510011ca0 tx=0x7ff510012ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=770aebe1ffde97ae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:14.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.440+0000 7ff50f400640 1 -- 172.21.15.43:0/4263668461 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff510030070 con 0x56191ece44b0 2024-09-19T05:12:14.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.440+0000 7ff50f400640 1 -- 172.21.15.43:0/4263668461 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff510011920 con 0x56191ece44b0 2024-09-19T05:12:14.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.440+0000 7ff50f400640 1 -- 172.21.15.43:0/4263668461 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff510000e60 con 0x56191ece44b0 2024-09-19T05:12:14.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.441+0000 7ff52c5fd740 1 -- 172.21.15.43:0/4263668461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191ece44b0 msgr2=0x56191ecec570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:14.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.441+0000 7ff52c5fd740 1 --2- 172.21.15.43:0/4263668461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191ece44b0 0x56191ecec570 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff510011ca0 tx=0x7ff510012ea0 comp rx=0 tx=0).stop 2024-09-19T05:12:14.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.441+0000 7ff52c5fd740 1 -- 172.21.15.43:0/4263668461 shutdown_connections 2024-09-19T05:12:14.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.441+0000 7ff52c5fd740 1 --2- 172.21.15.43:0/4263668461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191eceed70 0x56191ed06950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:14.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.441+0000 7ff52c5fd740 1 --2- 172.21.15.43:0/4263668461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191ece44b0 0x56191ecec570 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:14.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.441+0000 7ff52c5fd740 1 --2- 172.21.15.43:0/4263668461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56191ece3b90 0x56191ecedc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:14.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.441+0000 7ff52c5fd740 1 -- 172.21.15.43:0/4263668461 >> 172.21.15.43:0/4263668461 conn(0x56191ece0120 msgr2=0x56191eced500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:14.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.441+0000 7ff52c5fd740 1 -- 172.21.15.43:0/4263668461 shutdown_connections 2024-09-19T05:12:14.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.441+0000 7ff52c5fd740 1 -- 172.21.15.43:0/4263668461 wait complete. 2024-09-19T05:12:14.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.442+0000 7ff52c5fd740 1 -- start start 2024-09-19T05:12:14.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.442+0000 7ff52c5fd740 1 Processor -- start 2024-09-19T05:12:14.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.442+0000 7ff52c5fd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191ece3b90 0x56191ecf2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:14.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.442+0000 7ff52c5fd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191ece44b0 0x56191ecf28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:14.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.442+0000 7ff52c5fd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56191eceed70 0x56191ed67910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:14.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.443+0000 7ff52c5fd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff5241b1020 con 0x56191eceed70 2024-09-19T05:12:14.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.443+0000 7ff52c5fd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff5241b0950 con 0x56191ece44b0 2024-09-19T05:12:14.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.443+0000 7ff52c5fd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff5241adbb0 con 0x56191ece3b90 2024-09-19T05:12:14.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.443+0000 7ff52c5fd740 10 client.0 _ll_register_callbacks cb 0x7ff5100042d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:12:14.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.443+0000 7ff523e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56191eceed70 0x56191ed67910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:14.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.443+0000 7ff523e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56191eceed70 0x56191ed67910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34926/0 (socket says 172.21.15.43:34926) 2024-09-19T05:12:14.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.443+0000 7ff523e00640 1 -- 172.21.15.43:0/2446934342 learned_addr learned my addr 172.21.15.43:0/2446934342 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:14.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.443+0000 7ff523400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191ece3b90 0x56191ecf2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:14.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.443+0000 7ff522a00640 1 --2- 172.21.15.43:0/2446934342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191ece44b0 0x56191ecf28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:14.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.443+0000 7ff523400640 1 -- 172.21.15.43:0/2446934342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191ece44b0 msgr2=0x56191ecf28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:14.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.443+0000 7ff523400640 1 --2- 172.21.15.43:0/2446934342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191ece44b0 0x56191ecf28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:14.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.443+0000 7ff523400640 1 -- 172.21.15.43:0/2446934342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56191eceed70 msgr2=0x56191ed67910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:14.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.443+0000 7ff523400640 1 --2- 172.21.15.43:0/2446934342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56191eceed70 0x56191ed67910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:14.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.443+0000 7ff523400640 1 -- 172.21.15.43:0/2446934342 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56191ed67e80 con 0x56191ece3b90 2024-09-19T05:12:14.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.444+0000 7ff523400640 1 --2- 172.21.15.43:0/2446934342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191ece3b90 0x56191ecf2360 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff51400c6a0 tx=0x7ff51409c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62b90bc1ba8317f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:14.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.444+0000 7ff50cc00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:12:14.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.444+0000 7ff50cc00640 1 -- 172.21.15.43:0/2446934342 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff514065030 con 0x56191ece3b90 2024-09-19T05:12:14.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.444+0000 7ff52c5fd740 1 -- 172.21.15.43:0/2446934342 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff510011920 con 0x56191ece3b90 2024-09-19T05:12:14.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.444+0000 7ff50cc00640 1 -- 172.21.15.43:0/2446934342 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8586+0+0 (secure 0 0 0) 0x7ff5140ba040 con 0x56191ece3b90 2024-09-19T05:12:14.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.444+0000 7ff50cc00640 1 -- 172.21.15.43:0/2446934342 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff5140b4040 con 0x56191ece3b90 2024-09-19T05:12:14.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.444+0000 7ff52c5fd740 10 client.26905 fetch_fsmap learned FSMap version 29 2024-09-19T05:12:14.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.444+0000 7ff52c5fd740 1 -- 172.21.15.43:0/2446934342 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56191ed69170 con 0x56191ece3b90 2024-09-19T05:12:14.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.444+0000 7ff50cc00640 1 -- 172.21.15.43:0/2446934342 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff510011920 con 0x56191ece3b90 2024-09-19T05:12:14.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.445+0000 7ff50cc00640 1 -- 172.21.15.43:0/2446934342 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff5140a4ce0 con 0x56191ece3b90 2024-09-19T05:12:14.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.445+0000 7ff52c5fd740 10 client.26905 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:12:14.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.445+0000 7ff52c5fd740 10 client.26905 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:12:14.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.445+0000 7ff52c5fd740 20 client.26905 populate_metadata read hostname 'smithi043' 2024-09-19T05:12:14.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.445+0000 7ff52c5fd740 1 --2- 172.21.15.43:0/2446934342 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x56191ed69400 0x56191ed6b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:14.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.445+0000 7ff52c5fd740 4 client.26905 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:12:14.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.445+0000 7ff52c5fd740 1 -- 172.21.15.43:0/2446934342 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56191ec51e60 con 0x56191ed69400 2024-09-19T05:12:14.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.445+0000 7ff522a00640 1 --2- 172.21.15.43:0/2446934342 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x56191ed69400 0x56191ed6b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:14.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.446+0000 7ff522a00640 1 --2- 172.21.15.43:0/2446934342 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x56191ed69400 0x56191ed6b7f0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acd4a314c4ed5fe5 server_cookie=f3846ece28ff3d78 in_seq=0 out_seq=0 2024-09-19T05:12:14.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.446+0000 7ff50cc00640 10 client.26905 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:12:14.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.446+0000 7ff50cc00640 1 -- 172.21.15.43:0/2446934342 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56191ec51e60 con 0x56191ed69400 2024-09-19T05:12:14.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.446+0000 7ff50cc00640 10 client.26905 handle_command_reply: tid=0 2024-09-19T05:12:14.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.446+0000 7ff52c5fd740 1 client.26905 shutdown 2024-09-19T05:12:14.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.446+0000 7ff52c5fd740 1 -- 172.21.15.43:0/2446934342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191ece3b90 msgr2=0x56191ecf2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:14.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.446+0000 7ff52c5fd740 1 --2- 172.21.15.43:0/2446934342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191ece3b90 0x56191ecf2360 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff51400c6a0 tx=0x7ff51409c380 comp rx=0 tx=0).stop 2024-09-19T05:12:14.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.446+0000 7ff52c5fd740 1 -- 172.21.15.43:0/2446934342 shutdown_connections 2024-09-19T05:12:14.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.446+0000 7ff52c5fd740 1 --2- 172.21.15.43:0/2446934342 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x56191ed69400 0x56191ed6b7f0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:14.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.446+0000 7ff52c5fd740 1 --2- 172.21.15.43:0/2446934342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56191eceed70 0x56191ed67910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:14.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.446+0000 7ff52c5fd740 1 --2- 172.21.15.43:0/2446934342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56191ece44b0 0x56191ecf28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:14.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.446+0000 7ff52c5fd740 1 --2- 172.21.15.43:0/2446934342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56191ece3b90 0x56191ecf2360 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:14.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.446+0000 7ff50cc00640 0 client.26905 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:12:14.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.446+0000 7ff52c5fd740 1 -- 172.21.15.43:0/2446934342 >> 172.21.15.43:0/2446934342 conn(0x56191ece0120 msgr2=0x56191ecedd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:14.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.446+0000 7ff52c5fd740 1 -- 172.21.15.43:0/2446934342 shutdown_connections 2024-09-19T05:12:14.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.446+0000 7ff52c5fd740 1 -- 172.21.15.43:0/2446934342 wait complete. 2024-09-19T05:12:14.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.447+0000 7ff52c5fd740 20 client.26905 trim_cache size 0 max 16384 2024-09-19T05:12:14.453 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:12:14.453 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-09-19T05:12:14.453 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-09-19T05:12:14.453 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:12:14.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.447+0000 7ff52a000640 1 -- 172.21.15.43:0/1257297477 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff5080a33b0 msgr2=0x7ff5080a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:14.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.447+0000 7ff52a000640 1 --2- 172.21.15.43:0/1257297477 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff5080a33b0 0x7ff5080a5870 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff51c004af0 tx=0x7ff51c031c90 comp rx=0 tx=0).stop 2024-09-19T05:12:14.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.447+0000 7ff52a000640 1 -- 172.21.15.43:0/1257297477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff524101df0 msgr2=0x7ff52419a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:14.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.447+0000 7ff52a000640 1 --2- 172.21.15.43:0/1257297477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff524101df0 0x7ff52419a5b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff51400e9f0 tx=0x7ff51400edf0 comp rx=0 tx=0).stop 2024-09-19T05:12:14.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.447+0000 7ff52a000640 1 -- 172.21.15.43:0/1257297477 shutdown_connections 2024-09-19T05:12:14.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.447+0000 7ff52a000640 1 --2- 172.21.15.43:0/1257297477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff524106f20 0x7ff52419ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:14.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.447+0000 7ff52a000640 1 --2- 172.21.15.43:0/1257297477 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5241027a0 0x7ff52419aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:14.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.447+0000 7ff52a000640 1 --2- 172.21.15.43:0/1257297477 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff5080a33b0 0x7ff5080a5870 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:14.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.447+0000 7ff52a000640 1 --2- 172.21.15.43:0/1257297477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff524101df0 0x7ff52419a5b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:14.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.447+0000 7ff52a000640 1 -- 172.21.15.43:0/1257297477 >> 172.21.15.43:0/1257297477 conn(0x7ff5240fd8b0 msgr2=0x7ff5240feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:14.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.448+0000 7ff52a000640 1 -- 172.21.15.43:0/1257297477 shutdown_connections 2024-09-19T05:12:14.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:14.448+0000 7ff52a000640 1 -- 172.21.15.43:0/1257297477 wait complete. 2024-09-19T05:12:14.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:14 smithi096 ceph-mon[35085]: pgmap v233: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T05:12:15.317 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T05:12:15.317 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:12:15.712 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:12:16.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:16 smithi043 ceph-mon[30800]: pgmap v234: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T05:12:16.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:16 smithi043 ceph-mon[30800]: osdmap e90: 12 total, 12 up, 12 in 2024-09-19T05:12:16.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.309+0000 7f49eac00640 1 -- 172.21.15.43:0/639974097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49e41019c0 msgr2=0x7f49e4101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:16.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.309+0000 7f49eac00640 1 --2- 172.21.15.43:0/639974097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49e41019c0 0x7f49e4101e20 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009930 tx=0x7f49d002a420 comp rx=0 tx=0).stop 2024-09-19T05:12:16.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.309+0000 7f49eac00640 1 -- 172.21.15.43:0/639974097 shutdown_connections 2024-09-19T05:12:16.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.309+0000 7f49eac00640 1 --2- 172.21.15.43:0/639974097 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e4102360 0x7f49e410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:16.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.309+0000 7f49eac00640 1 --2- 172.21.15.43:0/639974097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49e41019c0 0x7f49e4101e20 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:16.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.309+0000 7f49eac00640 1 --2- 172.21.15.43:0/639974097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49e41079c0 0x7f49e4107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:16.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.309+0000 7f49eac00640 1 -- 172.21.15.43:0/639974097 >> 172.21.15.43:0/639974097 conn(0x7f49e40fd740 msgr2=0x7f49e40ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:16.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.309+0000 7f49eac00640 1 -- 172.21.15.43:0/639974097 shutdown_connections 2024-09-19T05:12:16.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.309+0000 7f49eac00640 1 -- 172.21.15.43:0/639974097 wait complete. 2024-09-19T05:12:16.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.310+0000 7f49eac00640 1 Processor -- start 2024-09-19T05:12:16.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.311+0000 7f49eac00640 1 -- start start 2024-09-19T05:12:16.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.311+0000 7f49eac00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49e41019c0 0x7f49e41982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:16.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.311+0000 7f49eac00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e4102360 0x7f49e4198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:16.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.311+0000 7f49eac00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49e41079c0 0x7f49e419cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:16.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.311+0000 7f49eac00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f49e4111e10 con 0x7f49e41079c0 2024-09-19T05:12:16.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.311+0000 7f49eac00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f49e4111c90 con 0x7f49e4102360 2024-09-19T05:12:16.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.311+0000 7f49eac00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f49e4111f90 con 0x7f49e41019c0 2024-09-19T05:12:16.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.312+0000 7f49ea200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49e41079c0 0x7f49e419cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:16.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.312+0000 7f49e9800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49e41019c0 0x7f49e41982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:16.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.312+0000 7f49e8e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e4102360 0x7f49e4198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:16.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.312+0000 7f49ea200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49e41079c0 0x7f49e419cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37666/0 (socket says 172.21.15.43:37666) 2024-09-19T05:12:16.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.312+0000 7f49ea200640 1 -- 172.21.15.43:0/390167584 learned_addr learned my addr 172.21.15.43:0/390167584 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:16.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.312+0000 7f49e9800640 1 -- 172.21.15.43:0/390167584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e4102360 msgr2=0x7f49e4198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:16.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.312+0000 7f49e9800640 1 --2- 172.21.15.43:0/390167584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e4102360 0x7f49e4198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:16.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.312+0000 7f49e9800640 1 -- 172.21.15.43:0/390167584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49e41079c0 msgr2=0x7f49e419cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:16.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.312+0000 7f49e9800640 1 --2- 172.21.15.43:0/390167584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49e41079c0 0x7f49e419cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:16.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.312+0000 7f49e9800640 1 -- 172.21.15.43:0/390167584 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49e419d280 con 0x7f49e41019c0 2024-09-19T05:12:16.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.312+0000 7f49ea200640 1 --2- 172.21.15.43:0/390167584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49e41079c0 0x7f49e419cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:12:16.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.312+0000 7f49e9800640 1 --2- 172.21.15.43:0/390167584 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49e41019c0 0x7f49e41982d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f49d800e9f0 tx=0x7f49d800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:16.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.313+0000 7f49e2000640 1 -- 172.21.15.43:0/390167584 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d800ccc0 con 0x7f49e41019c0 2024-09-19T05:12:16.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.313+0000 7f49eac00640 1 -- 172.21.15.43:0/390167584 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49e419d570 con 0x7f49e41019c0 2024-09-19T05:12:16.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.313+0000 7f49eac00640 1 -- 172.21.15.43:0/390167584 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49e41a4e50 con 0x7f49e41019c0 2024-09-19T05:12:16.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.313+0000 7f49e2000640 1 -- 172.21.15.43:0/390167584 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49d8010910 con 0x7f49e41019c0 2024-09-19T05:12:16.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.313+0000 7f49e2000640 1 -- 172.21.15.43:0/390167584 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d8018880 con 0x7f49e41019c0 2024-09-19T05:12:16.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.314+0000 7f49eac00640 1 -- 172.21.15.43:0/390167584 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49c4005290 con 0x7f49e41019c0 2024-09-19T05:12:16.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.316+0000 7f49e2000640 1 -- 172.21.15.43:0/390167584 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f49d8010430 con 0x7f49e41019c0 2024-09-19T05:12:16.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.317+0000 7f49e2000640 1 --2- 172.21.15.43:0/390167584 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f49cc0a3430 0x7f49cc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:16.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.318+0000 7f49e2000640 1 -- 172.21.15.43:0/390167584 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f49d80cff30 con 0x7f49e41019c0 2024-09-19T05:12:16.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.318+0000 7f49e8e00640 1 --2- 172.21.15.43:0/390167584 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f49cc0a3430 0x7f49cc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:16.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.319+0000 7f49e8e00640 1 --2- 172.21.15.43:0/390167584 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f49cc0a3430 0x7f49cc0a58f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f49d0006010 tx=0x7f49d0000bc0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:16.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.320+0000 7f49e2000640 1 -- 172.21.15.43:0/390167584 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f49d809c250 con 0x7f49e41019c0 2024-09-19T05:12:16.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:16 smithi151 ceph-mon[35425]: pgmap v234: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T05:12:16.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:16 smithi151 ceph-mon[35425]: osdmap e90: 12 total, 12 up, 12 in 2024-09-19T05:12:16.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:16 smithi096 ceph-mon[35085]: pgmap v234: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T05:12:16.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:16 smithi096 ceph-mon[35085]: osdmap e90: 12 total, 12 up, 12 in 2024-09-19T05:12:16.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.546+0000 7f49eac00640 1 -- 172.21.15.43:0/390167584 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49c4005580 con 0x7f49e41019c0 2024-09-19T05:12:16.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.547+0000 7f49e2000640 1 -- 172.21.15.43:0/390167584 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11965 (secure 0 0 0) 0x7f49d80a0550 con 0x7f49e41019c0 2024-09-19T05:12:16.550 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:12:16.550 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":29,"btime":"2024-09-19T05:04:38:785407+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/773801169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":773801169},{"type":"v1","addr":"172.21.15.43:6835","nonce":773801169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1799137902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1799137902},{"type":"v1","addr":"172.21.15.43:6837","nonce":1799137902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:02:25.463041+0000","modified":"2024-09-19T05:04:38.039809+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24518,"mds_2":24511,"mds_3":24506,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14541":{"gid":14541,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/794881135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":794881135},{"type":"v1","addr":"172.21.15.43:6841","nonce":794881135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/2836453158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2836453158},{"type":"v1","addr":"172.21.15.151:6833","nonce":2836453158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/178785812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":178785812},{"type":"v1","addr":"172.21.15.96:6833","nonce":178785812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1671043820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1671043820},{"type":"v1","addr":"172.21.15.96:6835","nonce":1671043820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2972514328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2972514328},{"type":"v1","addr":"172.21.15.151:6835","nonce":2972514328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.151:6837/3960378010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3960378010},{"type":"v1","addr":"172.21.15.151:6837","nonce":3960378010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6837/3978388286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3978388286},{"type":"v1","addr":"172.21.15.96:6837","nonce":3978388286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.151:6839/3794104313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3794104313},{"type":"v1","addr":"172.21.15.151:6839","nonce":3794104313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.43:6839/3404555030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3404555030},{"type":"v1","addr":"172.21.15.43:6839","nonce":3404555030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/605527621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":605527621},{"type":"v1","addr":"172.21.15.96:6839","nonce":605527621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24518,24511,24506,24493]},"id":1}]} 2024-09-19T05:12:16.550 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 29 2024-09-19T05:12:16.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.550+0000 7f49eac00640 1 -- 172.21.15.43:0/390167584 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f49cc0a3430 msgr2=0x7f49cc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:16.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.550+0000 7f49eac00640 1 --2- 172.21.15.43:0/390167584 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f49cc0a3430 0x7f49cc0a58f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f49d0006010 tx=0x7f49d0000bc0 comp rx=0 tx=0).stop 2024-09-19T05:12:16.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.550+0000 7f49eac00640 1 -- 172.21.15.43:0/390167584 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49e41019c0 msgr2=0x7f49e41982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:16.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.550+0000 7f49eac00640 1 --2- 172.21.15.43:0/390167584 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49e41019c0 0x7f49e41982d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f49d800e9f0 tx=0x7f49d800eeb0 comp rx=0 tx=0).stop 2024-09-19T05:12:16.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.550+0000 7f49eac00640 1 -- 172.21.15.43:0/390167584 shutdown_connections 2024-09-19T05:12:16.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.550+0000 7f49eac00640 1 --2- 172.21.15.43:0/390167584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49e41079c0 0x7f49e419cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:16.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.550+0000 7f49eac00640 1 --2- 172.21.15.43:0/390167584 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f49cc0a3430 0x7f49cc0a58f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:16.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.550+0000 7f49eac00640 1 --2- 172.21.15.43:0/390167584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e4102360 0x7f49e4198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:16.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.550+0000 7f49eac00640 1 --2- 172.21.15.43:0/390167584 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49e41019c0 0x7f49e41982d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:16.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.550+0000 7f49eac00640 1 -- 172.21.15.43:0/390167584 >> 172.21.15.43:0/390167584 conn(0x7f49e40fd740 msgr2=0x7f49e40fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:16.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.550+0000 7f49eac00640 1 -- 172.21.15.43:0/390167584 shutdown_connections 2024-09-19T05:12:16.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:16.550+0000 7f49eac00640 1 -- 172.21.15.43:0/390167584 wait complete. 2024-09-19T05:12:17.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:17 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/390167584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:12:17.331 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T05:12:17.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:17 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/390167584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:12:17.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:17 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/390167584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:12:17.710 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:12:18.142 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:18 smithi043 ceph-mon[30800]: pgmap v236: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 245 B/s wr, 12 op/s 2024-09-19T05:12:18.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.255+0000 7f7e35600640 1 -- 172.21.15.43:0/2395688205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e30077040 msgr2=0x7f7e30112330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:18.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.255+0000 7f7e35600640 1 --2- 172.21.15.43:0/2395688205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e30077040 0x7f7e30112330 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f7e18009900 tx=0x7f7e1802a410 comp rx=0 tx=0).stop 2024-09-19T05:12:18.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.255+0000 7f7e35600640 1 -- 172.21.15.43:0/2395688205 shutdown_connections 2024-09-19T05:12:18.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.255+0000 7f7e35600640 1 --2- 172.21.15.43:0/2395688205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e30112870 0x7f7e30114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:18.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.255+0000 7f7e35600640 1 --2- 172.21.15.43:0/2395688205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e30077040 0x7f7e30112330 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:18.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.255+0000 7f7e35600640 1 --2- 172.21.15.43:0/2395688205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e30076720 0x7f7e30076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:18.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.255+0000 7f7e35600640 1 -- 172.21.15.43:0/2395688205 >> 172.21.15.43:0/2395688205 conn(0x7f7e300ff8d0 msgr2=0x7f7e30101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:18.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.255+0000 7f7e35600640 1 -- 172.21.15.43:0/2395688205 shutdown_connections 2024-09-19T05:12:18.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.255+0000 7f7e35600640 1 -- 172.21.15.43:0/2395688205 wait complete. 2024-09-19T05:12:18.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.256+0000 7f7e35600640 1 Processor -- start 2024-09-19T05:12:18.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.256+0000 7f7e35600640 1 -- start start 2024-09-19T05:12:18.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.256+0000 7f7e35600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e30076720 0x7f7e301a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:18.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.256+0000 7f7e35600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e30077040 0x7f7e301a4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:18.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.256+0000 7f7e35600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e30112870 0x7f7e3019e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:18.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.256+0000 7f7e35600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7e301183e0 con 0x7f7e30112870 2024-09-19T05:12:18.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.256+0000 7f7e35600640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7e30118260 con 0x7f7e30076720 2024-09-19T05:12:18.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.256+0000 7f7e35600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7e30118560 con 0x7f7e30077040 2024-09-19T05:12:18.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.257+0000 7f7e34c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e30112870 0x7f7e3019e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:18.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.257+0000 7f7e2fe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e30076720 0x7f7e301a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:18.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.257+0000 7f7e34c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e30112870 0x7f7e3019e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37670/0 (socket says 172.21.15.43:37670) 2024-09-19T05:12:18.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.257+0000 7f7e34c00640 1 -- 172.21.15.43:0/1564717921 learned_addr learned my addr 172.21.15.43:0/1564717921 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:18.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.257+0000 7f7e27e00640 1 --2- 172.21.15.43:0/1564717921 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e30077040 0x7f7e301a4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:18.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.257+0000 7f7e2fe00640 1 -- 172.21.15.43:0/1564717921 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e30077040 msgr2=0x7f7e301a4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:18.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.257+0000 7f7e2fe00640 1 --2- 172.21.15.43:0/1564717921 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e30077040 0x7f7e301a4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:18.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.257+0000 7f7e2fe00640 1 -- 172.21.15.43:0/1564717921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e30112870 msgr2=0x7f7e3019e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:18.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.257+0000 7f7e2fe00640 1 --2- 172.21.15.43:0/1564717921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e30112870 0x7f7e3019e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:18.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.257+0000 7f7e2fe00640 1 -- 172.21.15.43:0/1564717921 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e3019ef60 con 0x7f7e30076720 2024-09-19T05:12:18.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.257+0000 7f7e2fe00640 1 --2- 172.21.15.43:0/1564717921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e30076720 0x7f7e301a4620 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7e2000e9f0 tx=0x7f7e2000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:18.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.258+0000 7f7e2d600640 1 -- 172.21.15.43:0/1564717921 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2000cd80 con 0x7f7e30076720 2024-09-19T05:12:18.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.258+0000 7f7e35600640 1 -- 172.21.15.43:0/1564717921 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e3019f250 con 0x7f7e30076720 2024-09-19T05:12:18.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.258+0000 7f7e35600640 1 -- 172.21.15.43:0/1564717921 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e301ab5a0 con 0x7f7e30076720 2024-09-19T05:12:18.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.258+0000 7f7e2d600640 1 -- 172.21.15.43:0/1564717921 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e200035f0 con 0x7f7e30076720 2024-09-19T05:12:18.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.258+0000 7f7e2d600640 1 -- 172.21.15.43:0/1564717921 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e20018430 con 0x7f7e30076720 2024-09-19T05:12:18.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.261+0000 7f7e397c9740 1 Processor -- start 2024-09-19T05:12:18.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.261+0000 7f7e397c9740 1 -- start start 2024-09-19T05:12:18.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.261+0000 7f7e2d600640 1 -- 172.21.15.43:0/1564717921 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f7e20004020 con 0x7f7e30076720 2024-09-19T05:12:18.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.261+0000 7f7e397c9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83cbade40 0x55c83cbae220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:18.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.261+0000 7f7e397c9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c83cbae760 0x55c83cbb8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:18.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.261+0000 7f7e397c9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c83cbb9020 0x55c83cbbb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:18.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.261+0000 7f7e397c9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c83cb143e0 con 0x55c83cbb9020 2024-09-19T05:12:18.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.261+0000 7f7e397c9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c83cabbd80 con 0x55c83cbae760 2024-09-19T05:12:18.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.261+0000 7f7e397c9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c83cb103a0 con 0x55c83cbade40 2024-09-19T05:12:18.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.262+0000 7f7e34c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83cbade40 0x55c83cbae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:18.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.262+0000 7f7e34c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83cbade40 0x55c83cbae220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:38364/0 (socket says 172.21.15.43:38364) 2024-09-19T05:12:18.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.262+0000 7f7e34c00640 1 -- 172.21.15.43:0/2759736655 learned_addr learned my addr 172.21.15.43:0/2759736655 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:18.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.262+0000 7f7e2fe00640 1 --2- 172.21.15.43:0/2759736655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c83cbae760 0x55c83cbb8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:18.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.262+0000 7f7e27e00640 1 --2- 172.21.15.43:0/2759736655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c83cbb9020 0x55c83cbbb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:18.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.262+0000 7f7e34c00640 1 -- 172.21.15.43:0/2759736655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c83cbae760 msgr2=0x55c83cbb8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:18.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.262+0000 7f7e34c00640 1 --2- 172.21.15.43:0/2759736655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c83cbae760 0x55c83cbb8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:18.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.262+0000 7f7e34c00640 1 -- 172.21.15.43:0/2759736655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c83cbb9020 msgr2=0x55c83cbbb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:18.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.262+0000 7f7e34c00640 1 --2- 172.21.15.43:0/2759736655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c83cbb9020 0x55c83cbbb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:18.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.262+0000 7f7e34c00640 1 -- 172.21.15.43:0/2759736655 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c83cadaae0 con 0x55c83cbade40 2024-09-19T05:12:18.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.263+0000 7f7e34c00640 1 --2- 172.21.15.43:0/2759736655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83cbade40 0x55c83cbae220 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7e28009fb0 tx=0x7f7e280118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51e029abe0206a4d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:18.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.263+0000 7f7e2d600640 1 --2- 172.21.15.43:0/1564717921 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f7e140a33b0 0x7f7e140a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:18.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.263+0000 7f7e25200640 1 -- 172.21.15.43:0/2759736655 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e28004b20 con 0x55c83cbade40 2024-09-19T05:12:18.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.263+0000 7f7e25200640 1 -- 172.21.15.43:0/2759736655 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e28004cc0 con 0x55c83cbade40 2024-09-19T05:12:18.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.263+0000 7f7e25200640 1 -- 172.21.15.43:0/2759736655 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e280054b0 con 0x55c83cbade40 2024-09-19T05:12:18.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.263+0000 7f7e2d600640 1 -- 172.21.15.43:0/1564717921 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f7e200cff00 con 0x7f7e30076720 2024-09-19T05:12:18.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.263+0000 7f7e2fe00640 1 --2- 172.21.15.43:0/1564717921 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f7e140a33b0 0x7f7e140a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:18.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.264+0000 7f7e397c9740 1 -- 172.21.15.43:0/2759736655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83cbade40 msgr2=0x55c83cbae220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:18.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.264+0000 7f7e397c9740 1 --2- 172.21.15.43:0/2759736655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83cbade40 0x55c83cbae220 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7e28009fb0 tx=0x7f7e280118d0 comp rx=0 tx=0).stop 2024-09-19T05:12:18.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.264+0000 7f7e2fe00640 1 --2- 172.21.15.43:0/1564717921 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f7e140a33b0 0x7f7e140a5870 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x55c83cbab230 tx=0x7f7e200d39b0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:18.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.264+0000 7f7e397c9740 1 -- 172.21.15.43:0/2759736655 shutdown_connections 2024-09-19T05:12:18.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.264+0000 7f7e397c9740 1 --2- 172.21.15.43:0/2759736655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c83cbb9020 0x55c83cbbb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:18.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.264+0000 7f7e397c9740 1 --2- 172.21.15.43:0/2759736655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c83cbae760 0x55c83cbb8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:18.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.264+0000 7f7e397c9740 1 --2- 172.21.15.43:0/2759736655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83cbade40 0x55c83cbae220 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:18.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.264+0000 7f7e397c9740 1 -- 172.21.15.43:0/2759736655 >> 172.21.15.43:0/2759736655 conn(0x55c83cbaa3d0 msgr2=0x55c83cbaa7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:18.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.264+0000 7f7e397c9740 1 -- 172.21.15.43:0/2759736655 shutdown_connections 2024-09-19T05:12:18.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.264+0000 7f7e397c9740 1 -- 172.21.15.43:0/2759736655 wait complete. 2024-09-19T05:12:18.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.264+0000 7f7e397c9740 1 -- start start 2024-09-19T05:12:18.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.265+0000 7f7e397c9740 1 Processor -- start 2024-09-19T05:12:18.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.265+0000 7f7e397c9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c83cbade40 0x55c83cbb5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:18.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.265+0000 7f7e397c9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c83cbae760 0x55c83cbb6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:18.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.265+0000 7f7e397c9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83cbb9020 0x55c83cbb6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:18.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.265+0000 7f7e397c9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c83cbbbbf0 con 0x55c83cbae760 2024-09-19T05:12:18.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.265+0000 7f7e397c9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c83cacf0f0 con 0x55c83cbade40 2024-09-19T05:12:18.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.265+0000 7f7e397c9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c83cbbbd70 con 0x55c83cbb9020 2024-09-19T05:12:18.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.265+0000 7f7e397c9740 10 client.0 _ll_register_callbacks cb 0x55c83cab6bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:12:18.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.265+0000 7f7e27e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c83cbae760 0x55c83cbb6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:18.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.265+0000 7f7e34c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c83cbade40 0x55c83cbb5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:18.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.265+0000 7f7e34c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c83cbade40 0x55c83cbb5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:48730/0 (socket says 172.21.15.43:48730) 2024-09-19T05:12:18.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.265+0000 7f7e34c00640 1 -- 172.21.15.43:0/478865908 learned_addr learned my addr 172.21.15.43:0/478865908 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:18.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.266+0000 7f7e34c00640 1 --2- 172.21.15.43:0/478865908 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83cbb9020 0x55c83cbb6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:18.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.266+0000 7f7e27e00640 1 -- 172.21.15.43:0/478865908 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83cbb9020 msgr2=0x55c83cbb6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:18.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.266+0000 7f7e27e00640 1 --2- 172.21.15.43:0/478865908 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83cbb9020 0x55c83cbb6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:18.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.266+0000 7f7e27e00640 1 -- 172.21.15.43:0/478865908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c83cbade40 msgr2=0x55c83cbb5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:18.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.266+0000 7f7e27e00640 1 --2- 172.21.15.43:0/478865908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c83cbade40 0x55c83cbb5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:18.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.266+0000 7f7e27e00640 1 -- 172.21.15.43:0/478865908 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c83c8853c0 con 0x55c83cbae760 2024-09-19T05:12:18.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.266+0000 7f7e27e00640 1 --2- 172.21.15.43:0/478865908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c83cbae760 0x55c83cbb6410 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f7e18005940 tx=0x7f7e180317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18e00cc8574ea6a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:18.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.266+0000 7f7e1d600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:12:18.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.267+0000 7f7e1d600640 1 -- 172.21.15.43:0/478865908 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e18035040 con 0x55c83cbae760 2024-09-19T05:12:18.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.267+0000 7f7e397c9740 1 -- 172.21.15.43:0/478865908 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e28004cc0 con 0x55c83cbae760 2024-09-19T05:12:18.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.267+0000 7f7e1d600640 1 -- 172.21.15.43:0/478865908 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f7e18004460 con 0x55c83cbae760 2024-09-19T05:12:18.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.267+0000 7f7e1d600640 1 -- 172.21.15.43:0/478865908 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7e18031a10 con 0x55c83cbae760 2024-09-19T05:12:18.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.267+0000 7f7e397c9740 10 client.16962 fetch_fsmap learned FSMap version 29 2024-09-19T05:12:18.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.267+0000 7f7e397c9740 1 -- 172.21.15.43:0/478865908 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c83cbc7ce0 con 0x55c83cbae760 2024-09-19T05:12:18.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.267+0000 7f7e1d600640 1 -- 172.21.15.43:0/478865908 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e28004cc0 con 0x55c83cbae760 2024-09-19T05:12:18.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.267+0000 7f7e1d600640 1 -- 172.21.15.43:0/478865908 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7e18004c90 con 0x55c83cbae760 2024-09-19T05:12:18.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.267+0000 7f7e397c9740 10 client.16962 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:12:18.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.267+0000 7f7e397c9740 10 client.16962 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-19T05:12:18.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.268+0000 7f7e397c9740 20 client.16962 populate_metadata read hostname 'smithi043' 2024-09-19T05:12:18.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.268+0000 7f7e397c9740 1 --2- 172.21.15.43:0/478865908 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55c83cbc90f0 0x55c83cbcb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:18.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.268+0000 7f7e397c9740 4 client.16962 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:12:18.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.268+0000 7f7e397c9740 1 -- 172.21.15.43:0/478865908 --> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c83cb21950 con 0x55c83cbc90f0 2024-09-19T05:12:18.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.268+0000 7f7e34c00640 1 --2- 172.21.15.43:0/478865908 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55c83cbc90f0 0x55c83cbcb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:18.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.269+0000 7f7e34c00640 1 --2- 172.21.15.43:0/478865908 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55c83cbc90f0 0x55c83cbcb550 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=511a0452c1bd12c8 server_cookie=97e3ed458a4f0ab in_seq=0 out_seq=0 2024-09-19T05:12:18.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.269+0000 7f7e1d600640 10 client.16962 ms_handle_connect on v2:172.21.15.151:6836/3960378010 2024-09-19T05:12:18.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.269+0000 7f7e1d600640 1 -- 172.21.15.43:0/478865908 <== mds.4 v2:172.21.15.151:6836/3960378010 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c83cb21950 con 0x55c83cbc90f0 2024-09-19T05:12:18.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.269+0000 7f7e1d600640 10 client.16962 handle_command_reply: tid=0 2024-09-19T05:12:18.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.269+0000 7f7e397c9740 1 client.16962 shutdown 2024-09-19T05:12:18.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.270+0000 7f7e397c9740 1 -- 172.21.15.43:0/478865908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c83cbae760 msgr2=0x55c83cbb6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:18.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.270+0000 7f7e397c9740 1 --2- 172.21.15.43:0/478865908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c83cbae760 0x55c83cbb6410 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f7e18005940 tx=0x7f7e180317e0 comp rx=0 tx=0).stop 2024-09-19T05:12:18.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.270+0000 7f7e397c9740 1 -- 172.21.15.43:0/478865908 shutdown_connections 2024-09-19T05:12:18.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.270+0000 7f7e397c9740 1 --2- 172.21.15.43:0/478865908 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55c83cbc90f0 0x55c83cbcb550 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:18.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.270+0000 7f7e397c9740 1 --2- 172.21.15.43:0/478865908 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83cbb9020 0x55c83cbb6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:18.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.270+0000 7f7e397c9740 1 --2- 172.21.15.43:0/478865908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c83cbae760 0x55c83cbb6410 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:18.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.270+0000 7f7e397c9740 1 --2- 172.21.15.43:0/478865908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c83cbade40 0x55c83cbb5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:18.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.270+0000 7f7e1d600640 0 client.16962 ms_handle_reset on v2:172.21.15.151:6836/3960378010 2024-09-19T05:12:18.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.270+0000 7f7e397c9740 1 -- 172.21.15.43:0/478865908 >> 172.21.15.43:0/478865908 conn(0x55c83cbaa3d0 msgr2=0x55c83cbb00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:18.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.270+0000 7f7e397c9740 1 -- 172.21.15.43:0/478865908 shutdown_connections 2024-09-19T05:12:18.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.270+0000 7f7e397c9740 1 -- 172.21.15.43:0/478865908 wait complete. 2024-09-19T05:12:18.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.270+0000 7f7e397c9740 20 client.16962 trim_cache size 0 max 16384 2024-09-19T05:12:18.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.284+0000 7f7e397c9740 1 Processor -- start 2024-09-19T05:12:18.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.284+0000 7f7e397c9740 1 -- start start 2024-09-19T05:12:18.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.284+0000 7f7e397c9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c83cbade40 0x55c83cbb08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:18.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.284+0000 7f7e397c9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83cbae760 0x55c83cbb7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:18.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.284+0000 7f7e397c9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c83cbb9020 0x55c83cbd0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:18.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.284+0000 7f7e397c9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c00e7e0 con 0x55c83cbb9020 2024-09-19T05:12:18.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.284+0000 7f7e397c9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c010320 con 0x55c83cbade40 2024-09-19T05:12:18.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.284+0000 7f7e397c9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c00eeb0 con 0x55c83cbae760 2024-09-19T05:12:18.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.284+0000 7f7e27e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83cbae760 0x55c83cbb7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:18.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.284+0000 7f7e34c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c83cbb9020 0x55c83cbd0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:18.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.284+0000 7f7e27e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83cbae760 0x55c83cbb7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:38386/0 (socket says 172.21.15.43:38386) 2024-09-19T05:12:18.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.284+0000 7f7e27e00640 1 -- 172.21.15.43:0/4036599152 learned_addr learned my addr 172.21.15.43:0/4036599152 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:18.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.285+0000 7f7e34c00640 1 --2- 172.21.15.43:0/4036599152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c83cbade40 0x55c83cbb08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:18.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.285+0000 7f7e27e00640 1 -- 172.21.15.43:0/4036599152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c83cbade40 msgr2=0x55c83cbb08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:18.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.285+0000 7f7e27e00640 1 --2- 172.21.15.43:0/4036599152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c83cbade40 0x55c83cbb08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:18.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.285+0000 7f7e27e00640 1 -- 172.21.15.43:0/4036599152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c83cbb9020 msgr2=0x55c83cbd0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:18.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.285+0000 7f7e27e00640 1 --2- 172.21.15.43:0/4036599152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c83cbb9020 0x55c83cbd0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:18.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.285+0000 7f7e27e00640 1 -- 172.21.15.43:0/4036599152 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c83cbab060 con 0x55c83cbae760 2024-09-19T05:12:18.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.285+0000 7f7e27e00640 1 --2- 172.21.15.43:0/4036599152 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83cbae760 0x55c83cbb7e40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7e18006010 tx=0x7f7e18000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8157ed2d61fbe12 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:18.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.285+0000 7f7e1fe00640 1 -- 172.21.15.43:0/4036599152 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e18052020 con 0x55c83cbae760 2024-09-19T05:12:18.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.285+0000 7f7e1fe00640 1 -- 172.21.15.43:0/4036599152 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e18041070 con 0x55c83cbae760 2024-09-19T05:12:18.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.285+0000 7f7e1fe00640 1 -- 172.21.15.43:0/4036599152 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e18035070 con 0x55c83cbae760 2024-09-19T05:12:18.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.286+0000 7f7e397c9740 1 -- 172.21.15.43:0/4036599152 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83cbae760 msgr2=0x55c83cbb7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:18.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.286+0000 7f7e397c9740 1 --2- 172.21.15.43:0/4036599152 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83cbae760 0x55c83cbb7e40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7e18006010 tx=0x7f7e18000bc0 comp rx=0 tx=0).stop 2024-09-19T05:12:18.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.286+0000 7f7e397c9740 1 -- 172.21.15.43:0/4036599152 shutdown_connections 2024-09-19T05:12:18.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.286+0000 7f7e397c9740 1 --2- 172.21.15.43:0/4036599152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c83cbb9020 0x55c83cbd0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:18.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.286+0000 7f7e397c9740 1 --2- 172.21.15.43:0/4036599152 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83cbae760 0x55c83cbb7e40 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:18.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.286+0000 7f7e397c9740 1 --2- 172.21.15.43:0/4036599152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c83cbade40 0x55c83cbb08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:18.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.286+0000 7f7e397c9740 1 -- 172.21.15.43:0/4036599152 >> 172.21.15.43:0/4036599152 conn(0x55c83cbaa3d0 msgr2=0x55c83cbb7730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:18.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.286+0000 7f7e397c9740 1 -- 172.21.15.43:0/4036599152 shutdown_connections 2024-09-19T05:12:18.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.286+0000 7f7e397c9740 1 -- 172.21.15.43:0/4036599152 wait complete. 2024-09-19T05:12:18.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.287+0000 7f7e397c9740 1 -- start start 2024-09-19T05:12:18.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.287+0000 7f7e397c9740 1 Processor -- start 2024-09-19T05:12:18.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.287+0000 7f7e397c9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83cbade40 0x55c83cbbce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:18.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.287+0000 7f7e397c9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c83cbae760 0x55c83cbbd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:18.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.287+0000 7f7e397c9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c83cbb9020 0x55c83cbbd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:18.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.287+0000 7f7e397c9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c00fc50 con 0x55c83cbb9020 2024-09-19T05:12:18.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.287+0000 7f7e397c9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c00f580 con 0x55c83cbae760 2024-09-19T05:12:18.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.287+0000 7f7e397c9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c00c7e0 con 0x55c83cbade40 2024-09-19T05:12:18.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.287+0000 7f7e397c9740 10 client.0 _ll_register_callbacks cb 0x7f7e18009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:12:18.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.287+0000 7f7e34c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c83cbb9020 0x55c83cbbd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:18.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.288+0000 7f7e34c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c83cbb9020 0x55c83cbbd900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37694/0 (socket says 172.21.15.43:37694) 2024-09-19T05:12:18.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.288+0000 7f7e34c00640 1 -- 172.21.15.43:0/3714014453 learned_addr learned my addr 172.21.15.43:0/3714014453 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:18.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.288+0000 7f7e27e00640 1 --2- 172.21.15.43:0/3714014453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c83cbae760 0x55c83cbbd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:18.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.288+0000 7f7e34c00640 1 --2- 172.21.15.43:0/3714014453 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83cbade40 0x55c83cbbce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:18.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.288+0000 7f7e34c00640 1 -- 172.21.15.43:0/3714014453 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83cbade40 msgr2=0x55c83cbbce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:18.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.288+0000 7f7e34c00640 1 --2- 172.21.15.43:0/3714014453 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83cbade40 0x55c83cbbce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:18.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.288+0000 7f7e34c00640 1 -- 172.21.15.43:0/3714014453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c83cbae760 msgr2=0x55c83cbbd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:18.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.288+0000 7f7e34c00640 1 --2- 172.21.15.43:0/3714014453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c83cbae760 0x55c83cbbd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:18.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.288+0000 7f7e34c00640 1 -- 172.21.15.43:0/3714014453 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c83cc32f30 con 0x55c83cbb9020 2024-09-19T05:12:18.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.288+0000 7f7e27e00640 1 --2- 172.21.15.43:0/3714014453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c83cbae760 0x55c83cbbd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:12:18.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.288+0000 7f7e34c00640 1 --2- 172.21.15.43:0/3714014453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c83cbb9020 0x55c83cbbd900 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f7e28005c40 tx=0x7f7e28013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c21fb31155cecd86 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:18.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.288+0000 7f7e1d600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:12:18.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.289+0000 7f7e1d600640 1 -- 172.21.15.43:0/3714014453 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2800f9c0 con 0x55c83cbb9020 2024-09-19T05:12:18.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.289+0000 7f7e1d600640 1 -- 172.21.15.43:0/3714014453 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f7e28016040 con 0x55c83cbb9020 2024-09-19T05:12:18.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.289+0000 7f7e397c9740 1 -- 172.21.15.43:0/3714014453 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e18041070 con 0x55c83cbb9020 2024-09-19T05:12:18.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.289+0000 7f7e1d600640 1 -- 172.21.15.43:0/3714014453 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7e2800fb60 con 0x55c83cbb9020 2024-09-19T05:12:18.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.289+0000 7f7e397c9740 10 client.16968 fetch_fsmap learned FSMap version 29 2024-09-19T05:12:18.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.289+0000 7f7e397c9740 1 -- 172.21.15.43:0/3714014453 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c83cc34210 con 0x55c83cbb9020 2024-09-19T05:12:18.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.289+0000 7f7e1d600640 1 -- 172.21.15.43:0/3714014453 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e18041070 con 0x55c83cbb9020 2024-09-19T05:12:18.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.289+0000 7f7e1d600640 1 -- 172.21.15.43:0/3714014453 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7e2803b4d0 con 0x55c83cbb9020 2024-09-19T05:12:18.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.289+0000 7f7e397c9740 10 client.16968 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:12:18.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.289+0000 7f7e397c9740 10 client.16968 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-19T05:12:18.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.289+0000 7f7e397c9740 20 client.16968 populate_metadata read hostname 'smithi043' 2024-09-19T05:12:18.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.290+0000 7f7e397c9740 1 --2- 172.21.15.43:0/3714014453 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55c83cc34790 0x55c83cc36b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:18.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.290+0000 7f7e397c9740 4 client.16968 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:12:18.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.290+0000 7f7e397c9740 1 -- 172.21.15.43:0/3714014453 --> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c83cb1be60 con 0x55c83cc34790 2024-09-19T05:12:18.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.290+0000 7f7e27e00640 1 --2- 172.21.15.43:0/3714014453 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55c83cc34790 0x55c83cc36b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:18.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.290+0000 7f7e27e00640 1 --2- 172.21.15.43:0/3714014453 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55c83cc34790 0x55c83cc36b80 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d5ab0613c78f6971 server_cookie=7f4dd68ba14b734c in_seq=0 out_seq=0 2024-09-19T05:12:18.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.290+0000 7f7e1d600640 10 client.16968 ms_handle_connect on v2:172.21.15.151:6836/3960378010 2024-09-19T05:12:18.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.291+0000 7f7e1d600640 1 -- 172.21.15.43:0/3714014453 <== mds.4 v2:172.21.15.151:6836/3960378010 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c83cb1be60 con 0x55c83cc34790 2024-09-19T05:12:18.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.291+0000 7f7e1d600640 10 client.16968 handle_command_reply: tid=0 2024-09-19T05:12:18.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.291+0000 7f7e397c9740 1 client.16968 shutdown 2024-09-19T05:12:18.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.291+0000 7f7e397c9740 1 -- 172.21.15.43:0/3714014453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c83cbb9020 msgr2=0x55c83cbbd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:18.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.291+0000 7f7e397c9740 1 --2- 172.21.15.43:0/3714014453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c83cbb9020 0x55c83cbbd900 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f7e28005c40 tx=0x7f7e28013040 comp rx=0 tx=0).stop 2024-09-19T05:12:18.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.291+0000 7f7e397c9740 1 -- 172.21.15.43:0/3714014453 shutdown_connections 2024-09-19T05:12:18.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.291+0000 7f7e397c9740 1 --2- 172.21.15.43:0/3714014453 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55c83cc34790 0x55c83cc36b80 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:18.318 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:12:18.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.291+0000 7f7e397c9740 1 --2- 172.21.15.43:0/3714014453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c83cbb9020 0x55c83cbbd900 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:18.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.291+0000 7f7e397c9740 1 --2- 172.21.15.43:0/3714014453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c83cbae760 0x55c83cbbd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:18.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.291+0000 7f7e1d600640 0 client.16968 ms_handle_reset on v2:172.21.15.151:6836/3960378010 2024-09-19T05:12:18.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.291+0000 7f7e397c9740 1 --2- 172.21.15.43:0/3714014453 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c83cbade40 0x55c83cbbce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:18.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.291+0000 7f7e397c9740 1 -- 172.21.15.43:0/3714014453 >> 172.21.15.43:0/3714014453 conn(0x55c83cbaa3d0 msgr2=0x55c83cbb7f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:18.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.291+0000 7f7e397c9740 1 -- 172.21.15.43:0/3714014453 shutdown_connections 2024-09-19T05:12:18.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.291+0000 7f7e397c9740 1 -- 172.21.15.43:0/3714014453 wait complete. 2024-09-19T05:12:18.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.291+0000 7f7e397c9740 20 client.16968 trim_cache size 0 max 16384 2024-09-19T05:12:18.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.292+0000 7f7e35600640 1 -- 172.21.15.43:0/1564717921 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f7e140a33b0 msgr2=0x7f7e140a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:18.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.292+0000 7f7e35600640 1 --2- 172.21.15.43:0/1564717921 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f7e140a33b0 0x7f7e140a5870 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x55c83cbab230 tx=0x7f7e200d39b0 comp rx=0 tx=0).stop 2024-09-19T05:12:18.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.292+0000 7f7e35600640 1 -- 172.21.15.43:0/1564717921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e30076720 msgr2=0x7f7e301a4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:18.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.292+0000 7f7e35600640 1 --2- 172.21.15.43:0/1564717921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e30076720 0x7f7e301a4620 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7e2000e9f0 tx=0x7f7e2000eeb0 comp rx=0 tx=0).stop 2024-09-19T05:12:18.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.292+0000 7f7e35600640 1 -- 172.21.15.43:0/1564717921 shutdown_connections 2024-09-19T05:12:18.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.292+0000 7f7e35600640 1 --2- 172.21.15.43:0/1564717921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e30112870 0x7f7e3019e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:18.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.292+0000 7f7e35600640 1 --2- 172.21.15.43:0/1564717921 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e30077040 0x7f7e301a4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:18.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.292+0000 7f7e35600640 1 --2- 172.21.15.43:0/1564717921 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f7e140a33b0 0x7f7e140a5870 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:18.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.292+0000 7f7e35600640 1 --2- 172.21.15.43:0/1564717921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e30076720 0x7f7e301a4620 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:18.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.292+0000 7f7e35600640 1 -- 172.21.15.43:0/1564717921 >> 172.21.15.43:0/1564717921 conn(0x7f7e300ff8d0 msgr2=0x7f7e30101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:18.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.292+0000 7f7e35600640 1 -- 172.21.15.43:0/1564717921 shutdown_connections 2024-09-19T05:12:18.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:18.292+0000 7f7e35600640 1 -- 172.21.15.43:0/1564717921 wait complete. 2024-09-19T05:12:18.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:18 smithi151 ceph-mon[35425]: pgmap v236: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 245 B/s wr, 12 op/s 2024-09-19T05:12:18.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:18 smithi096 ceph-mon[35085]: pgmap v236: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 245 B/s wr, 12 op/s 2024-09-19T05:12:19.111 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T05:12:19.510 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:12:20.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.072+0000 7fcfaea00640 1 -- 172.21.15.43:0/1867911118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfa8104010 msgr2=0x7fcfa81043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:20.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.072+0000 7fcfaea00640 1 --2- 172.21.15.43:0/1867911118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfa8104010 0x7fcfa81043f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fcf98009900 tx=0x7fcf9802a410 comp rx=0 tx=0).stop 2024-09-19T05:12:20.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.072+0000 7fcfaea00640 1 -- 172.21.15.43:0/1867911118 shutdown_connections 2024-09-19T05:12:20.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.072+0000 7fcfaea00640 1 --2- 172.21.15.43:0/1867911118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfa8109140 0x7fcfa8110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:20.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.072+0000 7fcfaea00640 1 --2- 172.21.15.43:0/1867911118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfa81049c0 0x7fcfa8108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:20.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.072+0000 7fcfaea00640 1 --2- 172.21.15.43:0/1867911118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfa8104010 0x7fcfa81043f0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:20.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.072+0000 7fcfaea00640 1 -- 172.21.15.43:0/1867911118 >> 172.21.15.43:0/1867911118 conn(0x7fcfa80ffa50 msgr2=0x7fcfa8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:20.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.072+0000 7fcfaea00640 1 -- 172.21.15.43:0/1867911118 shutdown_connections 2024-09-19T05:12:20.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.072+0000 7fcfaea00640 1 -- 172.21.15.43:0/1867911118 wait complete. 2024-09-19T05:12:20.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.073+0000 7fcfaea00640 1 Processor -- start 2024-09-19T05:12:20.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.074+0000 7fcfaea00640 1 -- start start 2024-09-19T05:12:20.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.074+0000 7fcfaea00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfa8104010 0x7fcfa819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:20.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.074+0000 7fcfaea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfa81049c0 0x7fcfa819ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:20.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.074+0000 7fcfaea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfa8109140 0x7fcfa819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:20.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.074+0000 7fcfaea00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa8114210 con 0x7fcfa81049c0 2024-09-19T05:12:20.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.074+0000 7fcfaea00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa8114090 con 0x7fcfa8104010 2024-09-19T05:12:20.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.074+0000 7fcfaea00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa8114390 con 0x7fcfa8109140 2024-09-19T05:12:20.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.074+0000 7fcfa7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfa81049c0 0x7fcfa819ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:20.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.074+0000 7fcfa7e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfa8104010 0x7fcfa819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:20.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.074+0000 7fcfa7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfa81049c0 0x7fcfa819ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37710/0 (socket says 172.21.15.43:37710) 2024-09-19T05:12:20.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.074+0000 7fcfa7400640 1 -- 172.21.15.43:0/3456579196 learned_addr learned my addr 172.21.15.43:0/3456579196 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:20.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.075+0000 7fcfaca00640 1 --2- 172.21.15.43:0/3456579196 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfa8109140 0x7fcfa819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:20.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.075+0000 7fcfa7e00640 1 -- 172.21.15.43:0/3456579196 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfa8109140 msgr2=0x7fcfa819f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:20.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.075+0000 7fcfa7e00640 1 --2- 172.21.15.43:0/3456579196 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfa8109140 0x7fcfa819f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:20.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.075+0000 7fcfa7e00640 1 -- 172.21.15.43:0/3456579196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfa81049c0 msgr2=0x7fcfa819ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:20.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.075+0000 7fcfa7e00640 1 --2- 172.21.15.43:0/3456579196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfa81049c0 0x7fcfa819ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:20.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.075+0000 7fcfa7e00640 1 -- 172.21.15.43:0/3456579196 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa819f6e0 con 0x7fcfa8104010 2024-09-19T05:12:20.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.075+0000 7fcfa7400640 1 --2- 172.21.15.43:0/3456579196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfa81049c0 0x7fcfa819ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:12:20.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.075+0000 7fcfa7e00640 1 --2- 172.21.15.43:0/3456579196 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfa8104010 0x7fcfa819a730 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fcf9802a930 tx=0x7fcf98003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:20.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.075+0000 7fcfa4c00640 1 -- 172.21.15.43:0/3456579196 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9803f070 con 0x7fcfa8104010 2024-09-19T05:12:20.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.075+0000 7fcfaea00640 1 -- 172.21.15.43:0/3456579196 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfa819f970 con 0x7fcfa8104010 2024-09-19T05:12:20.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.076+0000 7fcfa4c00640 1 -- 172.21.15.43:0/3456579196 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf98031c90 con 0x7fcfa8104010 2024-09-19T05:12:20.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.076+0000 7fcfa4c00640 1 -- 172.21.15.43:0/3456579196 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf98035030 con 0x7fcfa8104010 2024-09-19T05:12:20.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.076+0000 7fcfaea00640 1 -- 172.21.15.43:0/3456579196 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfa81a72b0 con 0x7fcfa8104010 2024-09-19T05:12:20.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.078+0000 7fcfb1475740 1 Processor -- start 2024-09-19T05:12:20.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.079+0000 7fcfb1475740 1 -- start start 2024-09-19T05:12:20.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.079+0000 7fcfa4c00640 1 -- 172.21.15.43:0/3456579196 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fcf98042070 con 0x7fcfa8104010 2024-09-19T05:12:20.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.079+0000 7fcfb1475740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a595b8dc70 0x55a595b8e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:20.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.079+0000 7fcfb1475740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a595b8e590 0x55a595b98910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:20.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.079+0000 7fcfb1475740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a595b98e50 0x55a595b9b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:20.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.079+0000 7fcfb1475740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a595af43e0 con 0x55a595b8e590 2024-09-19T05:12:20.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.079+0000 7fcfb1475740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a595a9bd80 con 0x55a595b8dc70 2024-09-19T05:12:20.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.079+0000 7fcfb1475740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a595af03a0 con 0x55a595b98e50 2024-09-19T05:12:20.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.079+0000 7fcfaca00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a595b8dc70 0x55a595b8e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:20.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.079+0000 7fcfaca00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a595b8dc70 0x55a595b8e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:48784/0 (socket says 172.21.15.43:48784) 2024-09-19T05:12:20.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.079+0000 7fcfaca00640 1 -- 172.21.15.43:0/606855294 learned_addr learned my addr 172.21.15.43:0/606855294 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:20.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.079+0000 7fcfa7e00640 1 --2- 172.21.15.43:0/606855294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a595b8e590 0x55a595b98910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:20.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.080+0000 7fcfa7400640 1 --2- 172.21.15.43:0/606855294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a595b98e50 0x55a595b9b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:20.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.080+0000 7fcfaca00640 1 -- 172.21.15.43:0/606855294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a595b98e50 msgr2=0x55a595b9b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:20.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.080+0000 7fcfaca00640 1 --2- 172.21.15.43:0/606855294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a595b98e50 0x55a595b9b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:20.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.080+0000 7fcfaca00640 1 -- 172.21.15.43:0/606855294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a595b8e590 msgr2=0x55a595b98910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:20.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.080+0000 7fcfaca00640 1 --2- 172.21.15.43:0/606855294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a595b8e590 0x55a595b98910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:20.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.080+0000 7fcfaca00640 1 -- 172.21.15.43:0/606855294 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a595abaae0 con 0x55a595b8dc70 2024-09-19T05:12:20.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.080+0000 7fcfa7e00640 1 --2- 172.21.15.43:0/606855294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a595b8e590 0x55a595b98910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:12:20.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.080+0000 7fcfaca00640 1 --2- 172.21.15.43:0/606855294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a595b8dc70 0x55a595b8e050 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0009f20 tx=0x7fcfa000c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4310cedafcb727b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:20.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.081+0000 7fcf9d200640 1 -- 172.21.15.43:0/606855294 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa001a070 con 0x55a595b8dc70 2024-09-19T05:12:20.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.081+0000 7fcf9d200640 1 -- 172.21.15.43:0/606855294 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfa00158f0 con 0x55a595b8dc70 2024-09-19T05:12:20.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.081+0000 7fcf9d200640 1 -- 172.21.15.43:0/606855294 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa000a960 con 0x55a595b8dc70 2024-09-19T05:12:20.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.081+0000 7fcfa4c00640 1 --2- 172.21.15.43:0/3456579196 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fcf8c0a3430 0x7fcf8c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:20.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.081+0000 7fcfa4c00640 1 -- 172.21.15.43:0/3456579196 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7fcf980f0420 con 0x7fcfa8104010 2024-09-19T05:12:20.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.081+0000 7fcfa7e00640 1 --2- 172.21.15.43:0/3456579196 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fcf8c0a3430 0x7fcf8c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:20.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.081+0000 7fcfb1475740 1 -- 172.21.15.43:0/606855294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a595b8dc70 msgr2=0x55a595b8e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:20.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.081+0000 7fcfb1475740 1 --2- 172.21.15.43:0/606855294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a595b8dc70 0x55a595b8e050 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0009f20 tx=0x7fcfa000c8b0 comp rx=0 tx=0).stop 2024-09-19T05:12:20.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.082+0000 7fcfb1475740 1 -- 172.21.15.43:0/606855294 shutdown_connections 2024-09-19T05:12:20.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.082+0000 7fcfb1475740 1 --2- 172.21.15.43:0/606855294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a595b98e50 0x55a595b9b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:20.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.082+0000 7fcfb1475740 1 --2- 172.21.15.43:0/606855294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a595b8e590 0x55a595b98910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:20.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.082+0000 7fcfb1475740 1 --2- 172.21.15.43:0/606855294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a595b8dc70 0x55a595b8e050 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:20.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.082+0000 7fcfb1475740 1 -- 172.21.15.43:0/606855294 >> 172.21.15.43:0/606855294 conn(0x55a595b8a200 msgr2=0x55a595b8a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:20.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.082+0000 7fcfb1475740 1 -- 172.21.15.43:0/606855294 shutdown_connections 2024-09-19T05:12:20.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.082+0000 7fcfb1475740 1 -- 172.21.15.43:0/606855294 wait complete. 2024-09-19T05:12:20.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.082+0000 7fcfa7e00640 1 --2- 172.21.15.43:0/3456579196 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fcf8c0a3430 0x7fcf8c0a58f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x55a595b8b060 tx=0x7fcf98005dc0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:20.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.082+0000 7fcfb1475740 1 -- start start 2024-09-19T05:12:20.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.082+0000 7fcfb1475740 1 Processor -- start 2024-09-19T05:12:20.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.083+0000 7fcfb1475740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a595b8dc70 0x55a595b95d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:20.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.083+0000 7fcfb1475740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a595b8e590 0x55a595b96260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:20.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.083+0000 7fcfb1475740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a595b98e50 0x55a595b967a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:20.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.083+0000 7fcfb1475740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a595b9ba20 con 0x55a595b8dc70 2024-09-19T05:12:20.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.083+0000 7fcfb1475740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a595aaf0f0 con 0x55a595b98e50 2024-09-19T05:12:20.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.083+0000 7fcfb1475740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a595b9bba0 con 0x55a595b8e590 2024-09-19T05:12:20.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.083+0000 7fcfb1475740 10 client.0 _ll_register_callbacks cb 0x55a595a96bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:12:20.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.083+0000 7fcfaca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a595b8dc70 0x55a595b95d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:20.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.083+0000 7fcfa7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a595b8e590 0x55a595b96260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:20.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.083+0000 7fcfa7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a595b8e590 0x55a595b96260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:38434/0 (socket says 172.21.15.43:38434) 2024-09-19T05:12:20.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.083+0000 7fcfa7400640 1 -- 172.21.15.43:0/794906334 learned_addr learned my addr 172.21.15.43:0/794906334 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:20.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.084+0000 7fcfaca00640 1 --2- 172.21.15.43:0/794906334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a595b98e50 0x55a595b967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:20.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.084+0000 7fcfa7400640 1 -- 172.21.15.43:0/794906334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a595b98e50 msgr2=0x55a595b967a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:20.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.084+0000 7fcfa7400640 1 --2- 172.21.15.43:0/794906334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a595b98e50 0x55a595b967a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:20.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.084+0000 7fcfa7400640 1 -- 172.21.15.43:0/794906334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a595b8dc70 msgr2=0x55a595b95d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:20.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.084+0000 7fcfa7400640 1 --2- 172.21.15.43:0/794906334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a595b8dc70 0x55a595b95d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:20.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.084+0000 7fcfa7400640 1 -- 172.21.15.43:0/794906334 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5958653c0 con 0x55a595b8e590 2024-09-19T05:12:20.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.084+0000 7fcfa7400640 1 --2- 172.21.15.43:0/794906334 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a595b8e590 0x55a595b96260 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcf94012d50 tx=0x7fcf9400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9094fa94074c8aa1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:20.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.084+0000 7fcf91600640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:12:20.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.084+0000 7fcf91600640 1 -- 172.21.15.43:0/794906334 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf94012110 con 0x55a595b8e590 2024-09-19T05:12:20.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.084+0000 7fcf91600640 1 -- 172.21.15.43:0/794906334 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7fcf94004050 con 0x55a595b8e590 2024-09-19T05:12:20.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.084+0000 7fcfb1475740 1 -- 172.21.15.43:0/794906334 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfa00158f0 con 0x55a595b8e590 2024-09-19T05:12:20.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.090+0000 7fcf91600640 1 -- 172.21.15.43:0/794906334 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcf940122b0 con 0x55a595b8e590 2024-09-19T05:12:20.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.090+0000 7fcfb1475740 10 client.26938 fetch_fsmap learned FSMap version 29 2024-09-19T05:12:20.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.090+0000 7fcfb1475740 1 -- 172.21.15.43:0/794906334 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a595ba7ad0 con 0x55a595b8e590 2024-09-19T05:12:20.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.091+0000 7fcf91600640 1 -- 172.21.15.43:0/794906334 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfa00158f0 con 0x55a595b8e590 2024-09-19T05:12:20.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.091+0000 7fcf91600640 1 -- 172.21.15.43:0/794906334 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcf94010430 con 0x55a595b8e590 2024-09-19T05:12:20.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.091+0000 7fcfb1475740 10 client.26938 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:12:20.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.091+0000 7fcfb1475740 10 client.26938 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T05:12:20.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.091+0000 7fcfb1475740 20 client.26938 populate_metadata read hostname 'smithi043' 2024-09-19T05:12:20.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.091+0000 7fcfb1475740 1 --2- 172.21.15.43:0/794906334 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55a595ba8f90 0x55a595bab3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:20.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.091+0000 7fcfb1475740 4 client.26938 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:12:20.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.091+0000 7fcfb1475740 1 -- 172.21.15.43:0/794906334 --> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a595b01950 con 0x55a595ba8f90 2024-09-19T05:12:20.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.091+0000 7fcfaca00640 1 --2- 172.21.15.43:0/794906334 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55a595ba8f90 0x55a595bab3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:20.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.092+0000 7fcfaca00640 1 --2- 172.21.15.43:0/794906334 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55a595ba8f90 0x55a595bab3c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ee8d3c8dbad3b03b server_cookie=4d2267dae7186c06 in_seq=0 out_seq=0 2024-09-19T05:12:20.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.092+0000 7fcf91600640 10 client.26938 ms_handle_connect on v2:172.21.15.96:6836/3978388286 2024-09-19T05:12:20.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.093+0000 7fcf91600640 1 -- 172.21.15.43:0/794906334 <== mds.3 v2:172.21.15.96:6836/3978388286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a595b01950 con 0x55a595ba8f90 2024-09-19T05:12:20.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.093+0000 7fcf91600640 10 client.26938 handle_command_reply: tid=0 2024-09-19T05:12:20.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.093+0000 7fcfb1475740 1 client.26938 shutdown 2024-09-19T05:12:20.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.093+0000 7fcfb1475740 1 -- 172.21.15.43:0/794906334 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a595b8e590 msgr2=0x55a595b96260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:20.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.093+0000 7fcfb1475740 1 --2- 172.21.15.43:0/794906334 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a595b8e590 0x55a595b96260 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcf94012d50 tx=0x7fcf9400f790 comp rx=0 tx=0).stop 2024-09-19T05:12:20.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.093+0000 7fcfb1475740 1 -- 172.21.15.43:0/794906334 shutdown_connections 2024-09-19T05:12:20.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.093+0000 7fcfb1475740 1 --2- 172.21.15.43:0/794906334 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55a595ba8f90 0x55a595bab3c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:20.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.093+0000 7fcfb1475740 1 --2- 172.21.15.43:0/794906334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a595b98e50 0x55a595b967a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:20.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.093+0000 7fcfb1475740 1 --2- 172.21.15.43:0/794906334 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a595b8e590 0x55a595b96260 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:20.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.093+0000 7fcfb1475740 1 --2- 172.21.15.43:0/794906334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a595b8dc70 0x55a595b95d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:20.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.093+0000 7fcf91600640 0 client.26938 ms_handle_reset on v2:172.21.15.96:6836/3978388286 2024-09-19T05:12:20.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.093+0000 7fcfb1475740 1 -- 172.21.15.43:0/794906334 >> 172.21.15.43:0/794906334 conn(0x55a595b8a200 msgr2=0x55a595b89380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:20.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.093+0000 7fcfb1475740 1 -- 172.21.15.43:0/794906334 shutdown_connections 2024-09-19T05:12:20.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.093+0000 7fcfb1475740 1 -- 172.21.15.43:0/794906334 wait complete. 2024-09-19T05:12:20.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.094+0000 7fcfb1475740 20 client.26938 trim_cache size 0 max 16384 2024-09-19T05:12:20.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.107+0000 7fcfb1475740 1 Processor -- start 2024-09-19T05:12:20.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.107+0000 7fcfb1475740 1 -- start start 2024-09-19T05:12:20.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.108+0000 7fcfb1475740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a595b8dc70 0x55a595b97c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:20.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.108+0000 7fcfb1475740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a595b8e590 0x55a595bb0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:20.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.108+0000 7fcfb1475740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a595b98e50 0x55a595bb1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:20.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.108+0000 7fcfb1475740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa81afb20 con 0x55a595b8dc70 2024-09-19T05:12:20.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.108+0000 7fcfb1475740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa81b1660 con 0x55a595b8e590 2024-09-19T05:12:20.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.108+0000 7fcfb1475740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa81b01f0 con 0x55a595b98e50 2024-09-19T05:12:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.108+0000 7fcfaca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a595b8dc70 0x55a595b97c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.108+0000 7fcfaca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a595b8dc70 0x55a595b97c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37738/0 (socket says 172.21.15.43:37738) 2024-09-19T05:12:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.108+0000 7fcfaca00640 1 -- 172.21.15.43:0/703742085 learned_addr learned my addr 172.21.15.43:0/703742085 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.108+0000 7fcfa7400640 1 --2- 172.21.15.43:0/703742085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a595b8e590 0x55a595bb0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.109+0000 7fcfa7400640 1 -- 172.21.15.43:0/703742085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a595b98e50 msgr2=0x55a595bb1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.109+0000 7fcfa7400640 1 --2- 172.21.15.43:0/703742085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a595b98e50 0x55a595bb1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.109+0000 7fcfa7400640 1 -- 172.21.15.43:0/703742085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a595b8dc70 msgr2=0x55a595b97c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.109+0000 7fcfa7400640 1 --2- 172.21.15.43:0/703742085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a595b8dc70 0x55a595b97c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.109+0000 7fcfa7400640 1 -- 172.21.15.43:0/703742085 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a595b85770 con 0x55a595b8e590 2024-09-19T05:12:20.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.109+0000 7fcfaca00640 1 --2- 172.21.15.43:0/703742085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a595b8dc70 0x55a595b97c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:12:20.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.109+0000 7fcfa7400640 1 --2- 172.21.15.43:0/703742085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a595b8e590 0x55a595bb0cd0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fcf940108a0 tx=0x7fcf9400fc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2fe1e41a6f3fe68 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:20.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.109+0000 7fcf93e00640 1 -- 172.21.15.43:0/703742085 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf94029030 con 0x55a595b8e590 2024-09-19T05:12:20.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.109+0000 7fcf93e00640 1 -- 172.21.15.43:0/703742085 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf940055f0 con 0x55a595b8e590 2024-09-19T05:12:20.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.109+0000 7fcf93e00640 1 -- 172.21.15.43:0/703742085 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf94010e60 con 0x55a595b8e590 2024-09-19T05:12:20.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.110+0000 7fcfb1475740 1 -- 172.21.15.43:0/703742085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a595b8e590 msgr2=0x55a595bb0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:20.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.110+0000 7fcfb1475740 1 --2- 172.21.15.43:0/703742085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a595b8e590 0x55a595bb0cd0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fcf940108a0 tx=0x7fcf9400fc90 comp rx=0 tx=0).stop 2024-09-19T05:12:20.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.110+0000 7fcfb1475740 1 -- 172.21.15.43:0/703742085 shutdown_connections 2024-09-19T05:12:20.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.110+0000 7fcfb1475740 1 --2- 172.21.15.43:0/703742085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a595b98e50 0x55a595bb1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:20.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.110+0000 7fcfb1475740 1 --2- 172.21.15.43:0/703742085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a595b8e590 0x55a595bb0cd0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:20.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.110+0000 7fcfb1475740 1 --2- 172.21.15.43:0/703742085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a595b8dc70 0x55a595b97c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:20.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.110+0000 7fcfb1475740 1 -- 172.21.15.43:0/703742085 >> 172.21.15.43:0/703742085 conn(0x55a595b8a200 msgr2=0x55a595b97580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:20.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.110+0000 7fcfb1475740 1 -- 172.21.15.43:0/703742085 shutdown_connections 2024-09-19T05:12:20.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.110+0000 7fcfb1475740 1 -- 172.21.15.43:0/703742085 wait complete. 2024-09-19T05:12:20.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.110+0000 7fcfb1475740 1 -- start start 2024-09-19T05:12:20.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.111+0000 7fcfb1475740 1 Processor -- start 2024-09-19T05:12:20.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.111+0000 7fcfb1475740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a595b8dc70 0x55a595b9cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:20.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.111+0000 7fcfb1475740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a595b8e590 0x55a595b9d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:20.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.111+0000 7fcfb1475740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a595b98e50 0x55a595c12ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:20.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.111+0000 7fcfb1475740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa81b0f90 con 0x55a595b98e50 2024-09-19T05:12:20.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.111+0000 7fcfb1475740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa81b08c0 con 0x55a595b8dc70 2024-09-19T05:12:20.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.111+0000 7fcfb1475740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa81adb20 con 0x55a595b8e590 2024-09-19T05:12:20.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.111+0000 7fcfb1475740 10 client.0 _ll_register_callbacks cb 0x7fcf94009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:12:20.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.111+0000 7fcfa7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a595b8e590 0x55a595b9d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:20.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.112+0000 7fcfa7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a595b8e590 0x55a595b9d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:38460/0 (socket says 172.21.15.43:38460) 2024-09-19T05:12:20.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.112+0000 7fcfa7400640 1 -- 172.21.15.43:0/1217560776 learned_addr learned my addr 172.21.15.43:0/1217560776 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:20.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.112+0000 7fcfaca00640 1 --2- 172.21.15.43:0/1217560776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a595b98e50 0x55a595c12ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:20.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.112+0000 7fcfaca00640 1 --2- 172.21.15.43:0/1217560776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a595b8dc70 0x55a595b9cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:20.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.112+0000 7fcfa7400640 1 -- 172.21.15.43:0/1217560776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a595b8dc70 msgr2=0x55a595b9cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:20.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.112+0000 7fcfa7400640 1 --2- 172.21.15.43:0/1217560776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a595b8dc70 0x55a595b9cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:20.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.112+0000 7fcfa7400640 1 -- 172.21.15.43:0/1217560776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a595b98e50 msgr2=0x55a595c12ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:20.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.112+0000 7fcfa7400640 1 --2- 172.21.15.43:0/1217560776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a595b98e50 0x55a595c12ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:20.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.112+0000 7fcfa7400640 1 -- 172.21.15.43:0/1217560776 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a595b9dd60 con 0x55a595b8e590 2024-09-19T05:12:20.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.112+0000 7fcfa7400640 1 --2- 172.21.15.43:0/1217560776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a595b8e590 0x55a595b9d4b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcf9400f7d0 tx=0x7fcf94005340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67addbf96fd3f537 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:20.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.112+0000 7fcf91600640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:12:20.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.113+0000 7fcf91600640 1 -- 172.21.15.43:0/1217560776 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf94032020 con 0x55a595b8e590 2024-09-19T05:12:20.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.113+0000 7fcf91600640 1 -- 172.21.15.43:0/1217560776 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7fcf94010ca0 con 0x55a595b8e590 2024-09-19T05:12:20.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.113+0000 7fcfb1475740 1 -- 172.21.15.43:0/1217560776 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf940055f0 con 0x55a595b8e590 2024-09-19T05:12:20.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.113+0000 7fcf91600640 1 -- 172.21.15.43:0/1217560776 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcf9403d050 con 0x55a595b8e590 2024-09-19T05:12:20.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.113+0000 7fcfb1475740 10 client.26941 fetch_fsmap learned FSMap version 29 2024-09-19T05:12:20.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.113+0000 7fcfb1475740 1 -- 172.21.15.43:0/1217560776 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a595c142c0 con 0x55a595b8e590 2024-09-19T05:12:20.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.113+0000 7fcf91600640 1 -- 172.21.15.43:0/1217560776 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf940055f0 con 0x55a595b8e590 2024-09-19T05:12:20.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.114+0000 7fcf91600640 1 -- 172.21.15.43:0/1217560776 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcf94025560 con 0x55a595b8e590 2024-09-19T05:12:20.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.114+0000 7fcfb1475740 10 client.26941 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:12:20.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.114+0000 7fcfb1475740 10 client.26941 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T05:12:20.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.114+0000 7fcfb1475740 20 client.26941 populate_metadata read hostname 'smithi043' 2024-09-19T05:12:20.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.114+0000 7fcfb1475740 1 --2- 172.21.15.43:0/1217560776 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55a595c14970 0x55a595c16d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:20.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.114+0000 7fcfb1475740 4 client.26941 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:12:20.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.114+0000 7fcfb1475740 1 -- 172.21.15.43:0/1217560776 --> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a595afbe60 con 0x55a595c14970 2024-09-19T05:12:20.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.114+0000 7fcfaca00640 1 --2- 172.21.15.43:0/1217560776 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55a595c14970 0x55a595c16d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:20.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.115+0000 7fcfaca00640 1 --2- 172.21.15.43:0/1217560776 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55a595c14970 0x55a595c16d90 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=920e1bd7705844b server_cookie=59360391ed72204f in_seq=0 out_seq=0 2024-09-19T05:12:20.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.115+0000 7fcf91600640 10 client.26941 ms_handle_connect on v2:172.21.15.96:6836/3978388286 2024-09-19T05:12:20.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.115+0000 7fcf91600640 1 -- 172.21.15.43:0/1217560776 <== mds.3 v2:172.21.15.96:6836/3978388286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a595afbe60 con 0x55a595c14970 2024-09-19T05:12:20.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.115+0000 7fcf91600640 10 client.26941 handle_command_reply: tid=0 2024-09-19T05:12:20.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.115+0000 7fcfb1475740 1 client.26941 shutdown 2024-09-19T05:12:20.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.115+0000 7fcfb1475740 1 -- 172.21.15.43:0/1217560776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a595b8e590 msgr2=0x55a595b9d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:20.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.115+0000 7fcfb1475740 1 --2- 172.21.15.43:0/1217560776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a595b8e590 0x55a595b9d4b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcf9400f7d0 tx=0x7fcf94005340 comp rx=0 tx=0).stop 2024-09-19T05:12:20.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.115+0000 7fcfb1475740 1 -- 172.21.15.43:0/1217560776 shutdown_connections 2024-09-19T05:12:20.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.115+0000 7fcfb1475740 1 --2- 172.21.15.43:0/1217560776 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55a595c14970 0x55a595c16d90 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:20.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.115+0000 7fcfb1475740 1 --2- 172.21.15.43:0/1217560776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a595b98e50 0x55a595c12ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:20.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.115+0000 7fcfb1475740 1 --2- 172.21.15.43:0/1217560776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a595b8e590 0x55a595b9d4b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:20.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.115+0000 7fcf91600640 0 client.26941 ms_handle_reset on v2:172.21.15.96:6836/3978388286 2024-09-19T05:12:20.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.115+0000 7fcfb1475740 1 --2- 172.21.15.43:0/1217560776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a595b8dc70 0x55a595b9cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:20.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.115+0000 7fcfb1475740 1 -- 172.21.15.43:0/1217560776 >> 172.21.15.43:0/1217560776 conn(0x55a595b8a200 msgr2=0x55a595b89380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:20.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.116+0000 7fcfb1475740 1 -- 172.21.15.43:0/1217560776 shutdown_connections 2024-09-19T05:12:20.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.116+0000 7fcfb1475740 1 -- 172.21.15.43:0/1217560776 wait complete. 2024-09-19T05:12:20.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.116+0000 7fcfb1475740 20 client.26941 trim_cache size 0 max 16384 2024-09-19T05:12:20.124 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:12:20.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.116+0000 7fcfaea00640 1 -- 172.21.15.43:0/3456579196 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fcf8c0a3430 msgr2=0x7fcf8c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:20.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.116+0000 7fcfaea00640 1 --2- 172.21.15.43:0/3456579196 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fcf8c0a3430 0x7fcf8c0a58f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x55a595b8b060 tx=0x7fcf98005dc0 comp rx=0 tx=0).stop 2024-09-19T05:12:20.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.116+0000 7fcfaea00640 1 -- 172.21.15.43:0/3456579196 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfa8104010 msgr2=0x7fcfa819a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:20.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.116+0000 7fcfaea00640 1 --2- 172.21.15.43:0/3456579196 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfa8104010 0x7fcfa819a730 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fcf9802a930 tx=0x7fcf98003b90 comp rx=0 tx=0).stop 2024-09-19T05:12:20.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.117+0000 7fcfaea00640 1 -- 172.21.15.43:0/3456579196 shutdown_connections 2024-09-19T05:12:20.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.117+0000 7fcfaea00640 1 --2- 172.21.15.43:0/3456579196 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfa8109140 0x7fcfa819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:20.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.117+0000 7fcfaea00640 1 --2- 172.21.15.43:0/3456579196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfa81049c0 0x7fcfa819ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:20.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.117+0000 7fcfaea00640 1 --2- 172.21.15.43:0/3456579196 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fcf8c0a3430 0x7fcf8c0a58f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:20.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.117+0000 7fcfaea00640 1 --2- 172.21.15.43:0/3456579196 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfa8104010 0x7fcfa819a730 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:20.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.117+0000 7fcfaea00640 1 -- 172.21.15.43:0/3456579196 >> 172.21.15.43:0/3456579196 conn(0x7fcfa80ffa50 msgr2=0x7fcfa8100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:20.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.117+0000 7fcfaea00640 1 -- 172.21.15.43:0/3456579196 shutdown_connections 2024-09-19T05:12:20.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:20.117+0000 7fcfaea00640 1 -- 172.21.15.43:0/3456579196 wait complete. 2024-09-19T05:12:20.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:20 smithi043 ceph-mon[30800]: pgmap v237: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T05:12:20.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:20 smithi151 ceph-mon[35425]: pgmap v237: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T05:12:20.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:20 smithi096 ceph-mon[35085]: pgmap v237: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T05:12:20.946 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T05:12:21.325 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:12:21.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.933+0000 7f4bf3600640 1 -- 172.21.15.43:0/589054479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bec077040 msgr2=0x7f4bec1124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:21.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.933+0000 7f4bf3600640 1 --2- 172.21.15.43:0/589054479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bec077040 0x7f4bec1124f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8009900 tx=0x7f4bd802a410 comp rx=0 tx=0).stop 2024-09-19T05:12:21.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.933+0000 7f4bf3600640 1 -- 172.21.15.43:0/589054479 shutdown_connections 2024-09-19T05:12:21.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.933+0000 7f4bf3600640 1 --2- 172.21.15.43:0/589054479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4bec112a30 0x7f4bec114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:21.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.933+0000 7f4bf3600640 1 --2- 172.21.15.43:0/589054479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bec077040 0x7f4bec1124f0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:21.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.933+0000 7f4bf3600640 1 --2- 172.21.15.43:0/589054479 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4bec076720 0x7f4bec076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:21.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.933+0000 7f4bf3600640 1 -- 172.21.15.43:0/589054479 >> 172.21.15.43:0/589054479 conn(0x7f4bec0ffa90 msgr2=0x7f4bec101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:21.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.933+0000 7f4bf3600640 1 -- 172.21.15.43:0/589054479 shutdown_connections 2024-09-19T05:12:21.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.934+0000 7f4bf3600640 1 -- 172.21.15.43:0/589054479 wait complete. 2024-09-19T05:12:21.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.935+0000 7f4bf3600640 1 Processor -- start 2024-09-19T05:12:21.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.935+0000 7f4bf3600640 1 -- start start 2024-09-19T05:12:21.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.935+0000 7f4bf3600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bec076720 0x7f4bec1a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:21.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.935+0000 7f4bf3600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4bec077040 0x7f4bec1a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:21.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.935+0000 7f4bf3600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4bec112a30 0x7f4bec19e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:21.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.935+0000 7f4bf3600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4bec118600 con 0x7f4bec076720 2024-09-19T05:12:21.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.935+0000 7f4bf3600640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4bec118480 con 0x7f4bec077040 2024-09-19T05:12:21.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.935+0000 7f4bf3600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4bec118780 con 0x7f4bec112a30 2024-09-19T05:12:21.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.936+0000 7f4bf0c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bec076720 0x7f4bec1a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:21.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.936+0000 7f4bf0c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bec076720 0x7f4bec1a4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37750/0 (socket says 172.21.15.43:37750) 2024-09-19T05:12:21.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.936+0000 7f4bf0c00640 1 -- 172.21.15.43:0/3472901598 learned_addr learned my addr 172.21.15.43:0/3472901598 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:21.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.936+0000 7f4bebe00640 1 --2- 172.21.15.43:0/3472901598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4bec077040 0x7f4bec1a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:21.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.936+0000 7f4bf1600640 1 --2- 172.21.15.43:0/3472901598 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4bec112a30 0x7f4bec19e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:21.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.936+0000 7f4bf0c00640 1 -- 172.21.15.43:0/3472901598 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4bec112a30 msgr2=0x7f4bec19e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:21.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.936+0000 7f4bf0c00640 1 --2- 172.21.15.43:0/3472901598 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4bec112a30 0x7f4bec19e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:21.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.936+0000 7f4bf0c00640 1 -- 172.21.15.43:0/3472901598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4bec077040 msgr2=0x7f4bec1a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:21.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.936+0000 7f4bf0c00640 1 --2- 172.21.15.43:0/3472901598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4bec077040 0x7f4bec1a4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:21.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.936+0000 7f4bf0c00640 1 -- 172.21.15.43:0/3472901598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bec19eff0 con 0x7f4bec076720 2024-09-19T05:12:21.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.936+0000 7f4bf0c00640 1 --2- 172.21.15.43:0/3472901598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bec076720 0x7f4bec1a4700 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4be000e9f0 tx=0x7f4be000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:21.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.937+0000 7f4be9600640 1 -- 172.21.15.43:0/3472901598 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be000cd80 con 0x7f4bec076720 2024-09-19T05:12:21.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.937+0000 7f4be9600640 1 -- 172.21.15.43:0/3472901598 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4be0010910 con 0x7f4bec076720 2024-09-19T05:12:21.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.937+0000 7f4be9600640 1 -- 172.21.15.43:0/3472901598 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be0018990 con 0x7f4bec076720 2024-09-19T05:12:21.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.937+0000 7f4bf3600640 1 -- 172.21.15.43:0/3472901598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bec19f2e0 con 0x7f4bec076720 2024-09-19T05:12:21.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.937+0000 7f4bf3600640 1 -- 172.21.15.43:0/3472901598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bec1ab5d0 con 0x7f4bec076720 2024-09-19T05:12:21.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.940+0000 7f4be9600640 1 -- 172.21.15.43:0/3472901598 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4be0010ac0 con 0x7f4bec076720 2024-09-19T05:12:21.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.940+0000 7f4bf604b740 1 Processor -- start 2024-09-19T05:12:21.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.940+0000 7f4bf604b740 1 -- start start 2024-09-19T05:12:21.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.941+0000 7f4bf604b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aa1b50bc70 0x55aa1b50c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:21.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.941+0000 7f4bf604b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa1b50c590 0x55aa1b516910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:21.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.941+0000 7f4bf604b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1b516e50 0x55aa1b519240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:21.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.941+0000 7f4bf604b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55aa1b4723e0 con 0x55aa1b516e50 2024-09-19T05:12:21.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.941+0000 7f4bf604b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55aa1b419d80 con 0x55aa1b50bc70 2024-09-19T05:12:21.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.941+0000 7f4bf604b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55aa1b46e3a0 con 0x55aa1b50c590 2024-09-19T05:12:21.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.941+0000 7f4bebe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1b516e50 0x55aa1b519240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:21.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.941+0000 7f4bf0c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa1b50c590 0x55aa1b516910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:21.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.941+0000 7f4bf1600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aa1b50bc70 0x55aa1b50c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:21.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.941+0000 7f4bebe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1b516e50 0x55aa1b519240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37758/0 (socket says 172.21.15.43:37758) 2024-09-19T05:12:21.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.941+0000 7f4bf1600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aa1b50bc70 0x55aa1b50c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:48834/0 (socket says 172.21.15.43:48834) 2024-09-19T05:12:21.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.941+0000 7f4bebe00640 1 -- 172.21.15.43:0/1404596062 learned_addr learned my addr 172.21.15.43:0/1404596062 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:21.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.941+0000 7f4bebe00640 1 -- 172.21.15.43:0/1404596062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa1b50c590 msgr2=0x55aa1b516910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:21.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.941+0000 7f4bebe00640 1 --2- 172.21.15.43:0/1404596062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa1b50c590 0x55aa1b516910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:21.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.941+0000 7f4bebe00640 1 -- 172.21.15.43:0/1404596062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aa1b50bc70 msgr2=0x55aa1b50c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:21.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.941+0000 7f4bebe00640 1 --2- 172.21.15.43:0/1404596062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aa1b50bc70 0x55aa1b50c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:21.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.941+0000 7f4bebe00640 1 -- 172.21.15.43:0/1404596062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa1b438ae0 con 0x55aa1b516e50 2024-09-19T05:12:21.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.941+0000 7f4bf1600640 1 --2- 172.21.15.43:0/1404596062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aa1b50bc70 0x55aa1b50c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:12:21.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.942+0000 7f4bebe00640 1 --2- 172.21.15.43:0/1404596062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1b516e50 0x55aa1b519240 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8009900 tx=0x7f4bd8005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b8a4f2c3c800cdc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:21.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.942+0000 7f4bddc00640 1 -- 172.21.15.43:0/1404596062 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd8035040 con 0x55aa1b516e50 2024-09-19T05:12:21.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.942+0000 7f4bddc00640 1 -- 172.21.15.43:0/1404596062 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bd8004400 con 0x55aa1b516e50 2024-09-19T05:12:21.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.942+0000 7f4bddc00640 1 -- 172.21.15.43:0/1404596062 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd80035f0 con 0x55aa1b516e50 2024-09-19T05:12:21.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.943+0000 7f4be9600640 1 --2- 172.21.15.43:0/3472901598 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f4bd40a3430 0x7f4bd40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:21.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.943+0000 7f4be9600640 1 -- 172.21.15.43:0/3472901598 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f4be00cffe0 con 0x7f4bec076720 2024-09-19T05:12:21.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.943+0000 7f4bf604b740 1 -- 172.21.15.43:0/1404596062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1b516e50 msgr2=0x55aa1b519240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:21.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.943+0000 7f4bf604b740 1 --2- 172.21.15.43:0/1404596062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1b516e50 0x55aa1b519240 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8009900 tx=0x7f4bd8005de0 comp rx=0 tx=0).stop 2024-09-19T05:12:21.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.943+0000 7f4bf1600640 1 --2- 172.21.15.43:0/3472901598 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f4bd40a3430 0x7f4bd40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:21.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.943+0000 7f4bf604b740 1 -- 172.21.15.43:0/1404596062 shutdown_connections 2024-09-19T05:12:21.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.943+0000 7f4bf604b740 1 --2- 172.21.15.43:0/1404596062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1b516e50 0x55aa1b519240 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:21.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.943+0000 7f4bf604b740 1 --2- 172.21.15.43:0/1404596062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa1b50c590 0x55aa1b516910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:21.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.943+0000 7f4bf604b740 1 --2- 172.21.15.43:0/1404596062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aa1b50bc70 0x55aa1b50c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:21.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.943+0000 7f4bf604b740 1 -- 172.21.15.43:0/1404596062 >> 172.21.15.43:0/1404596062 conn(0x55aa1b508200 msgr2=0x55aa1b5085e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:21.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.943+0000 7f4bf604b740 1 -- 172.21.15.43:0/1404596062 shutdown_connections 2024-09-19T05:12:21.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.944+0000 7f4bf604b740 1 -- 172.21.15.43:0/1404596062 wait complete. 2024-09-19T05:12:21.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.944+0000 7f4bf1600640 1 --2- 172.21.15.43:0/3472901598 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f4bd40a3430 0x7f4bd40a58f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4bec06b460 tx=0x7f4be4009500 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:21.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.944+0000 7f4bf604b740 1 -- start start 2024-09-19T05:12:21.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.944+0000 7f4bf604b740 1 Processor -- start 2024-09-19T05:12:21.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.945+0000 7f4bf604b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa1b50bc70 0x55aa1b513d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:21.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.945+0000 7f4bf604b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1b50c590 0x55aa1b514280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:21.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.945+0000 7f4bf604b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aa1b516e50 0x55aa1b5147c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:21.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.945+0000 7f4bf604b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55aa1b519a20 con 0x55aa1b50c590 2024-09-19T05:12:21.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.945+0000 7f4bf604b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55aa1b42d0f0 con 0x55aa1b516e50 2024-09-19T05:12:21.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.945+0000 7f4bf604b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55aa1b519ba0 con 0x55aa1b50bc70 2024-09-19T05:12:21.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.945+0000 7f4bf604b740 10 client.0 _ll_register_callbacks cb 0x55aa1b414bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:12:21.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.945+0000 7f4bebe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1b50c590 0x55aa1b514280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:21.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.945+0000 7f4bebe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1b50c590 0x55aa1b514280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37766/0 (socket says 172.21.15.43:37766) 2024-09-19T05:12:21.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.945+0000 7f4bebe00640 1 -- 172.21.15.43:0/3157175108 learned_addr learned my addr 172.21.15.43:0/3157175108 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:21.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.945+0000 7f4bf0c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa1b50bc70 0x55aa1b513d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:21.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.946+0000 7f4bf1600640 1 --2- 172.21.15.43:0/3157175108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aa1b516e50 0x55aa1b5147c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:21.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.946+0000 7f4bebe00640 1 -- 172.21.15.43:0/3157175108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa1b50bc70 msgr2=0x55aa1b513d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:21.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.946+0000 7f4bebe00640 1 --2- 172.21.15.43:0/3157175108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa1b50bc70 0x55aa1b513d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:21.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.946+0000 7f4bebe00640 1 -- 172.21.15.43:0/3157175108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aa1b516e50 msgr2=0x55aa1b5147c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:21.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.946+0000 7f4bebe00640 1 --2- 172.21.15.43:0/3157175108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aa1b516e50 0x55aa1b5147c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:21.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.946+0000 7f4bebe00640 1 -- 172.21.15.43:0/3157175108 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa1b1e33c0 con 0x55aa1b50c590 2024-09-19T05:12:21.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.946+0000 7f4bf0c00640 1 --2- 172.21.15.43:0/3157175108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa1b50bc70 0x55aa1b513d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:12:21.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.946+0000 7f4bebe00640 1 --2- 172.21.15.43:0/3157175108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1b50c590 0x55aa1b514280 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8005c40 tx=0x7f4bd802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509d9d95f534a8a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:21.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.946+0000 7f4bd2000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:12:21.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.947+0000 7f4bd2000640 1 -- 172.21.15.43:0/3157175108 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd8052060 con 0x55aa1b50c590 2024-09-19T05:12:21.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.947+0000 7f4bd2000640 1 -- 172.21.15.43:0/3157175108 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f4bd8004200 con 0x55aa1b50c590 2024-09-19T05:12:21.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.947+0000 7f4bf604b740 1 -- 172.21.15.43:0/3157175108 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bd8004400 con 0x55aa1b50c590 2024-09-19T05:12:21.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.947+0000 7f4bd2000640 1 -- 172.21.15.43:0/3157175108 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4bd8035050 con 0x55aa1b50c590 2024-09-19T05:12:21.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.947+0000 7f4bf604b740 10 client.16998 fetch_fsmap learned FSMap version 29 2024-09-19T05:12:21.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.947+0000 7f4bf604b740 1 -- 172.21.15.43:0/3157175108 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55aa1b525a60 con 0x55aa1b50c590 2024-09-19T05:12:21.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.947+0000 7f4bd2000640 1 -- 172.21.15.43:0/3157175108 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bd8004400 con 0x55aa1b50c590 2024-09-19T05:12:21.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.948+0000 7f4bd2000640 1 -- 172.21.15.43:0/3157175108 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4bd8003a30 con 0x55aa1b50c590 2024-09-19T05:12:21.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.948+0000 7f4bf604b740 10 client.16998 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:12:21.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.948+0000 7f4bf604b740 10 client.16998 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T05:12:21.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.948+0000 7f4bf604b740 20 client.16998 populate_metadata read hostname 'smithi043' 2024-09-19T05:12:21.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.948+0000 7f4bf604b740 1 --2- 172.21.15.43:0/3157175108 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55aa1b526db0 0x55aa1b5291a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:21.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.948+0000 7f4bf604b740 4 client.16998 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:12:21.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.948+0000 7f4bf604b740 1 -- 172.21.15.43:0/3157175108 --> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa1b47f950 con 0x55aa1b526db0 2024-09-19T05:12:21.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.948+0000 7f4bf0c00640 1 --2- 172.21.15.43:0/3157175108 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55aa1b526db0 0x55aa1b5291a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:21.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.949+0000 7f4bf0c00640 1 --2- 172.21.15.43:0/3157175108 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55aa1b526db0 0x55aa1b5291a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=62e57140ff48ad7a server_cookie=b54f594603a16767 in_seq=0 out_seq=0 2024-09-19T05:12:21.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.949+0000 7f4bd2000640 10 client.16998 ms_handle_connect on v2:172.21.15.151:6838/3794104313 2024-09-19T05:12:21.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.949+0000 7f4bd2000640 1 -- 172.21.15.43:0/3157175108 <== mds.2 v2:172.21.15.151:6838/3794104313 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aa1b47f950 con 0x55aa1b526db0 2024-09-19T05:12:21.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.949+0000 7f4bd2000640 10 client.16998 handle_command_reply: tid=0 2024-09-19T05:12:21.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.949+0000 7f4bf604b740 1 client.16998 shutdown 2024-09-19T05:12:21.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.950+0000 7f4bf604b740 1 -- 172.21.15.43:0/3157175108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1b50c590 msgr2=0x55aa1b514280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:21.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.950+0000 7f4bf604b740 1 --2- 172.21.15.43:0/3157175108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1b50c590 0x55aa1b514280 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8005c40 tx=0x7f4bd802a410 comp rx=0 tx=0).stop 2024-09-19T05:12:21.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.950+0000 7f4bf604b740 1 -- 172.21.15.43:0/3157175108 shutdown_connections 2024-09-19T05:12:21.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.950+0000 7f4bf604b740 1 --2- 172.21.15.43:0/3157175108 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55aa1b526db0 0x55aa1b5291a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:21.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.950+0000 7f4bf604b740 1 --2- 172.21.15.43:0/3157175108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aa1b516e50 0x55aa1b5147c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:21.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.950+0000 7f4bf604b740 1 --2- 172.21.15.43:0/3157175108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1b50c590 0x55aa1b514280 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:21.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.950+0000 7f4bf604b740 1 --2- 172.21.15.43:0/3157175108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa1b50bc70 0x55aa1b513d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:21.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.950+0000 7f4bd2000640 0 client.16998 ms_handle_reset on v2:172.21.15.151:6838/3794104313 2024-09-19T05:12:21.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.950+0000 7f4bf604b740 1 -- 172.21.15.43:0/3157175108 >> 172.21.15.43:0/3157175108 conn(0x55aa1b508200 msgr2=0x55aa1b50dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:21.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.950+0000 7f4bf604b740 1 -- 172.21.15.43:0/3157175108 shutdown_connections 2024-09-19T05:12:21.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.950+0000 7f4bf604b740 1 -- 172.21.15.43:0/3157175108 wait complete. 2024-09-19T05:12:21.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.950+0000 7f4bf604b740 20 client.16998 trim_cache size 0 max 16384 2024-09-19T05:12:21.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.964+0000 7f4bf604b740 1 Processor -- start 2024-09-19T05:12:21.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.964+0000 7f4bf604b740 1 -- start start 2024-09-19T05:12:21.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.964+0000 7f4bf604b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1b50bc70 0x55aa1b515cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:21.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.964+0000 7f4bf604b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa1b50c590 0x55aa1b52c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:21.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.964+0000 7f4bf604b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aa1b516e50 0x55aa1b52c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:21.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.964+0000 7f4bf604b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4bec1b3ff0 con 0x55aa1b50bc70 2024-09-19T05:12:21.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.964+0000 7f4bf604b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4bec1b5b30 con 0x55aa1b516e50 2024-09-19T05:12:21.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.964+0000 7f4bf604b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4bec1b46c0 con 0x55aa1b50c590 2024-09-19T05:12:21.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.965+0000 7f4bf0c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1b50bc70 0x55aa1b515cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:21.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.965+0000 7f4bf0c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1b50bc70 0x55aa1b515cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37782/0 (socket says 172.21.15.43:37782) 2024-09-19T05:12:21.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.965+0000 7f4bf0c00640 1 -- 172.21.15.43:0/3043503108 learned_addr learned my addr 172.21.15.43:0/3043503108 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:21.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.965+0000 7f4bebe00640 1 --2- 172.21.15.43:0/3043503108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa1b50c590 0x55aa1b52c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:21.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.965+0000 7f4bf1600640 1 --2- 172.21.15.43:0/3043503108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aa1b516e50 0x55aa1b52c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:21.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.965+0000 7f4bf0c00640 1 -- 172.21.15.43:0/3043503108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa1b50c590 msgr2=0x55aa1b52c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:21.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.965+0000 7f4bf0c00640 1 --2- 172.21.15.43:0/3043503108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa1b50c590 0x55aa1b52c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:21.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.965+0000 7f4bf0c00640 1 -- 172.21.15.43:0/3043503108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aa1b516e50 msgr2=0x55aa1b52c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:21.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.965+0000 7f4bf0c00640 1 --2- 172.21.15.43:0/3043503108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aa1b516e50 0x55aa1b52c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:21.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.965+0000 7f4bf0c00640 1 -- 172.21.15.43:0/3043503108 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa1b50c3c0 con 0x55aa1b50bc70 2024-09-19T05:12:21.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.965+0000 7f4bf0c00640 1 --2- 172.21.15.43:0/3043503108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1b50bc70 0x55aa1b515cb0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4be009c2b0 tx=0x7f4be00d24e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f0744af9054bfe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:21.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.966+0000 7f4bdd200640 1 -- 172.21.15.43:0/3043503108 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be009c320 con 0x55aa1b50bc70 2024-09-19T05:12:21.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.966+0000 7f4bdd200640 1 -- 172.21.15.43:0/3043503108 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4be00a0620 con 0x55aa1b50bc70 2024-09-19T05:12:21.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.966+0000 7f4bdd200640 1 -- 172.21.15.43:0/3043503108 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be00a4a50 con 0x55aa1b50bc70 2024-09-19T05:12:21.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.966+0000 7f4bf604b740 1 -- 172.21.15.43:0/3043503108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1b50bc70 msgr2=0x55aa1b515cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:21.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.966+0000 7f4bf604b740 1 --2- 172.21.15.43:0/3043503108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1b50bc70 0x55aa1b515cb0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4be009c2b0 tx=0x7f4be00d24e0 comp rx=0 tx=0).stop 2024-09-19T05:12:21.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.966+0000 7f4bf604b740 1 -- 172.21.15.43:0/3043503108 shutdown_connections 2024-09-19T05:12:21.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.966+0000 7f4bf604b740 1 --2- 172.21.15.43:0/3043503108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aa1b516e50 0x55aa1b52c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:21.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.966+0000 7f4bf604b740 1 --2- 172.21.15.43:0/3043503108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa1b50c590 0x55aa1b52c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:21.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.966+0000 7f4bf604b740 1 --2- 172.21.15.43:0/3043503108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1b50bc70 0x55aa1b515cb0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:21.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.966+0000 7f4bf604b740 1 -- 172.21.15.43:0/3043503108 >> 172.21.15.43:0/3043503108 conn(0x55aa1b508200 msgr2=0x55aa1b5155a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:21.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.967+0000 7f4bf604b740 1 -- 172.21.15.43:0/3043503108 shutdown_connections 2024-09-19T05:12:21.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.967+0000 7f4bf604b740 1 -- 172.21.15.43:0/3043503108 wait complete. 2024-09-19T05:12:21.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.967+0000 7f4bf604b740 1 -- start start 2024-09-19T05:12:21.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.967+0000 7f4bf604b740 1 Processor -- start 2024-09-19T05:12:21.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.968+0000 7f4bf604b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1b50bc70 0x55aa1b52c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:21.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.968+0000 7f4bf604b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aa1b50c590 0x55aa1b529a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:21.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.968+0000 7f4bf604b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa1b516e50 0x55aa1b529fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:21.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.968+0000 7f4bf604b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4bec1b5460 con 0x55aa1b50bc70 2024-09-19T05:12:21.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.968+0000 7f4bf604b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4bec1b4d90 con 0x55aa1b50c590 2024-09-19T05:12:21.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.968+0000 7f4bf604b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4bec1b1ff0 con 0x55aa1b516e50 2024-09-19T05:12:21.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.968+0000 7f4bf604b740 10 client.0 _ll_register_callbacks cb 0x7f4be00d3980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:12:21.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.968+0000 7f4bf1600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa1b516e50 0x55aa1b529fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:21.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.968+0000 7f4bf0c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1b50bc70 0x55aa1b52c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:21.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.968+0000 7f4bebe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aa1b50c590 0x55aa1b529a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:21.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.968+0000 7f4bf1600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa1b516e50 0x55aa1b529fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:38492/0 (socket says 172.21.15.43:38492) 2024-09-19T05:12:21.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.968+0000 7f4bf1600640 1 -- 172.21.15.43:0/451543797 learned_addr learned my addr 172.21.15.43:0/451543797 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:21.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.968+0000 7f4bebe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aa1b50c590 0x55aa1b529a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:48868/0 (socket says 172.21.15.43:48868) 2024-09-19T05:12:21.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.969+0000 7f4bebe00640 1 -- 172.21.15.43:0/451543797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa1b516e50 msgr2=0x55aa1b529fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:21.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.969+0000 7f4bebe00640 1 --2- 172.21.15.43:0/451543797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa1b516e50 0x55aa1b529fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:21.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.969+0000 7f4bebe00640 1 -- 172.21.15.43:0/451543797 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1b50bc70 msgr2=0x55aa1b52c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:21.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.969+0000 7f4bebe00640 1 --2- 172.21.15.43:0/451543797 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1b50bc70 0x55aa1b52c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:21.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.969+0000 7f4bebe00640 1 -- 172.21.15.43:0/451543797 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa1b52a600 con 0x55aa1b50c590 2024-09-19T05:12:21.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.969+0000 7f4bf0c00640 1 --2- 172.21.15.43:0/451543797 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1b50bc70 0x55aa1b52c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:12:21.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.969+0000 7f4bebe00640 1 --2- 172.21.15.43:0/451543797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aa1b50c590 0x55aa1b529a80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8003e00 tx=0x7f4bd8004490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=273a35a6a048612a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:21.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.969+0000 7f4bd2000640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:12:21.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.969+0000 7f4bd2000640 1 -- 172.21.15.43:0/451543797 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd8052070 con 0x55aa1b50c590 2024-09-19T05:12:21.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.969+0000 7f4bf604b740 1 -- 172.21.15.43:0/451543797 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4be00a0620 con 0x55aa1b50c590 2024-09-19T05:12:21.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.969+0000 7f4bd2000640 1 -- 172.21.15.43:0/451543797 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f4bd8003800 con 0x55aa1b50c590 2024-09-19T05:12:21.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.969+0000 7f4bd2000640 1 -- 172.21.15.43:0/451543797 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4bd8064050 con 0x55aa1b50c590 2024-09-19T05:12:21.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.969+0000 7f4bf604b740 10 client.27026 fetch_fsmap learned FSMap version 29 2024-09-19T05:12:21.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.969+0000 7f4bf604b740 1 -- 172.21.15.43:0/451543797 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55aa1b591500 con 0x55aa1b50c590 2024-09-19T05:12:21.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.970+0000 7f4bd2000640 1 -- 172.21.15.43:0/451543797 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4be00a0620 con 0x55aa1b50c590 2024-09-19T05:12:21.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.970+0000 7f4bd2000640 1 -- 172.21.15.43:0/451543797 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4bd80315b0 con 0x55aa1b50c590 2024-09-19T05:12:21.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.970+0000 7f4bf604b740 10 client.27026 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:12:21.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.970+0000 7f4bf604b740 10 client.27026 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T05:12:21.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.970+0000 7f4bf604b740 20 client.27026 populate_metadata read hostname 'smithi043' 2024-09-19T05:12:21.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.970+0000 7f4bf604b740 1 --2- 172.21.15.43:0/451543797 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55aa1b591c40 0x55aa1b594060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:21.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.970+0000 7f4bf604b740 4 client.27026 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:12:21.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.970+0000 7f4bf604b740 1 -- 172.21.15.43:0/451543797 --> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aa1b479e60 con 0x55aa1b591c40 2024-09-19T05:12:21.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.970+0000 7f4bf0c00640 1 --2- 172.21.15.43:0/451543797 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55aa1b591c40 0x55aa1b594060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:21.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.971+0000 7f4bf0c00640 1 --2- 172.21.15.43:0/451543797 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55aa1b591c40 0x55aa1b594060 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=13cab8b2f51d52b2 server_cookie=1d2616f96ac4b02c in_seq=0 out_seq=0 2024-09-19T05:12:21.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.971+0000 7f4bd2000640 10 client.27026 ms_handle_connect on v2:172.21.15.151:6838/3794104313 2024-09-19T05:12:21.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.971+0000 7f4bd2000640 1 -- 172.21.15.43:0/451543797 <== mds.2 v2:172.21.15.151:6838/3794104313 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aa1b479e60 con 0x55aa1b591c40 2024-09-19T05:12:21.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.971+0000 7f4bd2000640 10 client.27026 handle_command_reply: tid=0 2024-09-19T05:12:21.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.971+0000 7f4bf604b740 1 client.27026 shutdown 2024-09-19T05:12:21.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.971+0000 7f4bf604b740 1 -- 172.21.15.43:0/451543797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aa1b50c590 msgr2=0x55aa1b529a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:21.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.971+0000 7f4bf604b740 1 --2- 172.21.15.43:0/451543797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aa1b50c590 0x55aa1b529a80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8003e00 tx=0x7f4bd8004490 comp rx=0 tx=0).stop 2024-09-19T05:12:21.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.971+0000 7f4bf604b740 1 -- 172.21.15.43:0/451543797 shutdown_connections 2024-09-19T05:12:21.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.971+0000 7f4bf604b740 1 --2- 172.21.15.43:0/451543797 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55aa1b591c40 0x55aa1b594060 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:21.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.971+0000 7f4bf604b740 1 --2- 172.21.15.43:0/451543797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa1b516e50 0x55aa1b529fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:21.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.971+0000 7f4bd2000640 0 client.27026 ms_handle_reset on v2:172.21.15.151:6838/3794104313 2024-09-19T05:12:21.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.971+0000 7f4bf604b740 1 --2- 172.21.15.43:0/451543797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aa1b50c590 0x55aa1b529a80 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:21.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.971+0000 7f4bf604b740 1 --2- 172.21.15.43:0/451543797 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1b50bc70 0x55aa1b52c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:21.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.971+0000 7f4bf604b740 1 -- 172.21.15.43:0/451543797 >> 172.21.15.43:0/451543797 conn(0x55aa1b508200 msgr2=0x55aa1b515dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:21.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.972+0000 7f4bf604b740 1 -- 172.21.15.43:0/451543797 shutdown_connections 2024-09-19T05:12:21.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.972+0000 7f4bf604b740 1 -- 172.21.15.43:0/451543797 wait complete. 2024-09-19T05:12:21.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.972+0000 7f4bf604b740 20 client.27026 trim_cache size 0 max 16384 2024-09-19T05:12:21.980 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:12:21.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.972+0000 7f4bf3600640 1 -- 172.21.15.43:0/3472901598 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f4bd40a3430 msgr2=0x7f4bd40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:21.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.972+0000 7f4bf3600640 1 --2- 172.21.15.43:0/3472901598 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f4bd40a3430 0x7f4bd40a58f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4bec06b460 tx=0x7f4be4009500 comp rx=0 tx=0).stop 2024-09-19T05:12:21.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.972+0000 7f4bf3600640 1 -- 172.21.15.43:0/3472901598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bec076720 msgr2=0x7f4bec1a4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:21.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.972+0000 7f4bf3600640 1 --2- 172.21.15.43:0/3472901598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bec076720 0x7f4bec1a4700 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4be000e9f0 tx=0x7f4be000eeb0 comp rx=0 tx=0).stop 2024-09-19T05:12:21.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.973+0000 7f4bf3600640 1 -- 172.21.15.43:0/3472901598 shutdown_connections 2024-09-19T05:12:21.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.973+0000 7f4bf3600640 1 --2- 172.21.15.43:0/3472901598 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4bec112a30 0x7f4bec19e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:21.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.973+0000 7f4bf3600640 1 --2- 172.21.15.43:0/3472901598 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f4bd40a3430 0x7f4bd40a58f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:21.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.973+0000 7f4bf3600640 1 --2- 172.21.15.43:0/3472901598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4bec077040 0x7f4bec1a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:21.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.973+0000 7f4bf3600640 1 --2- 172.21.15.43:0/3472901598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bec076720 0x7f4bec1a4700 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:21.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.973+0000 7f4bf3600640 1 -- 172.21.15.43:0/3472901598 >> 172.21.15.43:0/3472901598 conn(0x7f4bec0ffa90 msgr2=0x7f4bec101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:21.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.973+0000 7f4bf3600640 1 -- 172.21.15.43:0/3472901598 shutdown_connections 2024-09-19T05:12:21.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:21.973+0000 7f4bf3600640 1 -- 172.21.15.43:0/3472901598 wait complete. 2024-09-19T05:12:22.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:22 smithi151 ceph-mon[35425]: pgmap v238: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T05:12:22.423 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:22 smithi043 ceph-mon[30800]: pgmap v238: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T05:12:22.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:22 smithi096 ceph-mon[35085]: pgmap v238: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T05:12:22.779 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T05:12:23.159 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:12:23.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.728+0000 7f398d400640 1 -- 172.21.15.43:0/3024898181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3988103ff0 msgr2=0x7f39881043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:23.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.728+0000 7f398d400640 1 --2- 172.21.15.43:0/3024898181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3988103ff0 0x7f39881043d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3978009900 tx=0x7f397802a410 comp rx=0 tx=0).stop 2024-09-19T05:12:23.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.728+0000 7f398d400640 1 -- 172.21.15.43:0/3024898181 shutdown_connections 2024-09-19T05:12:23.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.728+0000 7f398d400640 1 --2- 172.21.15.43:0/3024898181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3988109120 0x7f3988110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:23.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.728+0000 7f398d400640 1 --2- 172.21.15.43:0/3024898181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39881049a0 0x7f39881089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:23.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.728+0000 7f398d400640 1 --2- 172.21.15.43:0/3024898181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3988103ff0 0x7f39881043d0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:23.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.728+0000 7f398d400640 1 -- 172.21.15.43:0/3024898181 >> 172.21.15.43:0/3024898181 conn(0x7f39880ffa90 msgr2=0x7f3988101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:23.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.728+0000 7f398d400640 1 -- 172.21.15.43:0/3024898181 shutdown_connections 2024-09-19T05:12:23.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.728+0000 7f398d400640 1 -- 172.21.15.43:0/3024898181 wait complete. 2024-09-19T05:12:23.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.729+0000 7f398d400640 1 Processor -- start 2024-09-19T05:12:23.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.729+0000 7f398d400640 1 -- start start 2024-09-19T05:12:23.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.730+0000 7f398d400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3988103ff0 0x7f398819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:23.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.730+0000 7f398d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39881049a0 0x7f398819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:23.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.730+0000 7f398d400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3988109120 0x7f398819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:23.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.730+0000 7f398d400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3988114150 con 0x7f39881049a0 2024-09-19T05:12:23.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.730+0000 7f398d400640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3988113fd0 con 0x7f3988109120 2024-09-19T05:12:23.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.730+0000 7f398d400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f39881142d0 con 0x7f3988103ff0 2024-09-19T05:12:23.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.730+0000 7f397fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39881049a0 0x7f398819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:23.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.730+0000 7f3987200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3988109120 0x7f398819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:23.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.730+0000 7f397fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39881049a0 0x7f398819ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37802/0 (socket says 172.21.15.43:37802) 2024-09-19T05:12:23.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.730+0000 7f3986800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3988103ff0 0x7f398819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:23.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.730+0000 7f397fe00640 1 -- 172.21.15.43:0/2422453917 learned_addr learned my addr 172.21.15.43:0/2422453917 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:23.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.731+0000 7f3987200640 1 -- 172.21.15.43:0/2422453917 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3988103ff0 msgr2=0x7f398819a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:23.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.731+0000 7f3987200640 1 --2- 172.21.15.43:0/2422453917 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3988103ff0 0x7f398819a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:23.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.731+0000 7f3987200640 1 -- 172.21.15.43:0/2422453917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39881049a0 msgr2=0x7f398819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:23.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.731+0000 7f3987200640 1 --2- 172.21.15.43:0/2422453917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39881049a0 0x7f398819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:23.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.731+0000 7f3987200640 1 -- 172.21.15.43:0/2422453917 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f398819f5a0 con 0x7f3988109120 2024-09-19T05:12:23.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.731+0000 7f397fe00640 1 --2- 172.21.15.43:0/2422453917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39881049a0 0x7f398819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:12:23.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.731+0000 7f3987200640 1 --2- 172.21.15.43:0/2422453917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3988109120 0x7f398819eec0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3980009fd0 tx=0x7f398000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:23.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.731+0000 7f397f400640 1 -- 172.21.15.43:0/2422453917 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f398000ce20 con 0x7f3988109120 2024-09-19T05:12:23.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.731+0000 7f398d400640 1 -- 172.21.15.43:0/2422453917 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f398819f890 con 0x7f3988109120 2024-09-19T05:12:23.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.731+0000 7f397f400640 1 -- 172.21.15.43:0/2422453917 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3980005020 con 0x7f3988109120 2024-09-19T05:12:23.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.731+0000 7f397f400640 1 -- 172.21.15.43:0/2422453917 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3980010720 con 0x7f3988109120 2024-09-19T05:12:23.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.732+0000 7f398d400640 1 -- 172.21.15.43:0/2422453917 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39881a7170 con 0x7f3988109120 2024-09-19T05:12:23.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.734+0000 7f398fea5740 1 Processor -- start 2024-09-19T05:12:23.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.735+0000 7f397f400640 1 -- 172.21.15.43:0/2422453917 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f3980005540 con 0x7f3988109120 2024-09-19T05:12:23.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.735+0000 7f398fea5740 1 -- start start 2024-09-19T05:12:23.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.735+0000 7f398fea5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eacda74c70 0x55eacda75050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:23.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.735+0000 7f398fea5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eacda75590 0x55eacda7f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:23.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.735+0000 7f398fea5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eacda7fe50 0x55eacda82240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:23.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.735+0000 7f398fea5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55eacd9db3e0 con 0x55eacda75590 2024-09-19T05:12:23.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.735+0000 7f398fea5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55eacd982d80 con 0x55eacda7fe50 2024-09-19T05:12:23.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.736+0000 7f398fea5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55eacd9d73a0 con 0x55eacda74c70 2024-09-19T05:12:23.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.736+0000 7f3986800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eacda75590 0x55eacda7f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:23.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.736+0000 7f3986800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eacda75590 0x55eacda7f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37814/0 (socket says 172.21.15.43:37814) 2024-09-19T05:12:23.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.736+0000 7f3986800640 1 -- 172.21.15.43:0/275998802 learned_addr learned my addr 172.21.15.43:0/275998802 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:23.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.736+0000 7f397fe00640 1 --2- 172.21.15.43:0/275998802 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eacda74c70 0x55eacda75050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:23.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.736+0000 7f397fe00640 1 --2- 172.21.15.43:0/275998802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eacda7fe50 0x55eacda82240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:23.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.736+0000 7f3986800640 1 -- 172.21.15.43:0/275998802 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eacda74c70 msgr2=0x55eacda75050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:23.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.736+0000 7f3986800640 1 --2- 172.21.15.43:0/275998802 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eacda74c70 0x55eacda75050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:23.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.736+0000 7f3986800640 1 -- 172.21.15.43:0/275998802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eacda7fe50 msgr2=0x55eacda82240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:23.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.736+0000 7f3986800640 1 --2- 172.21.15.43:0/275998802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eacda7fe50 0x55eacda82240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:23.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.736+0000 7f3986800640 1 -- 172.21.15.43:0/275998802 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eacd9a1ae0 con 0x55eacda75590 2024-09-19T05:12:23.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.737+0000 7f397f400640 1 --2- 172.21.15.43:0/2422453917 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f396c0a33b0 0x7f396c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:23.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.737+0000 7f3986800640 1 --2- 172.21.15.43:0/275998802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eacda75590 0x55eacda7f910 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f3978009900 tx=0x7f3978005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba2e3716f78f7dd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:23.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.737+0000 7f397f400640 1 -- 172.21.15.43:0/2422453917 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f39800d0320 con 0x7f3988109120 2024-09-19T05:12:23.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.737+0000 7f397fe00640 1 --2- 172.21.15.43:0/2422453917 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f396c0a33b0 0x7f396c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:23.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.737+0000 7f3977400640 1 -- 172.21.15.43:0/275998802 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3978040070 con 0x55eacda75590 2024-09-19T05:12:23.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.737+0000 7f3977400640 1 -- 172.21.15.43:0/275998802 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3978031870 con 0x55eacda75590 2024-09-19T05:12:23.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.737+0000 7f3977400640 1 -- 172.21.15.43:0/275998802 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3978035040 con 0x55eacda75590 2024-09-19T05:12:23.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.738+0000 7f398fea5740 1 -- 172.21.15.43:0/275998802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eacda75590 msgr2=0x55eacda7f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:23.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.738+0000 7f398fea5740 1 --2- 172.21.15.43:0/275998802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eacda75590 0x55eacda7f910 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f3978009900 tx=0x7f3978005be0 comp rx=0 tx=0).stop 2024-09-19T05:12:23.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.738+0000 7f397fe00640 1 --2- 172.21.15.43:0/2422453917 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f396c0a33b0 0x7f396c0a5870 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x55eacda72580 tx=0x7f3970005f90 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:23.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.738+0000 7f398fea5740 1 -- 172.21.15.43:0/275998802 shutdown_connections 2024-09-19T05:12:23.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.738+0000 7f398fea5740 1 --2- 172.21.15.43:0/275998802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eacda7fe50 0x55eacda82240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:23.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.738+0000 7f398fea5740 1 --2- 172.21.15.43:0/275998802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eacda75590 0x55eacda7f910 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:23.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.738+0000 7f398fea5740 1 --2- 172.21.15.43:0/275998802 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eacda74c70 0x55eacda75050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:23.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.738+0000 7f398fea5740 1 -- 172.21.15.43:0/275998802 >> 172.21.15.43:0/275998802 conn(0x55eacda71200 msgr2=0x55eacda715e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:23.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.738+0000 7f398fea5740 1 -- 172.21.15.43:0/275998802 shutdown_connections 2024-09-19T05:12:23.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.738+0000 7f398fea5740 1 -- 172.21.15.43:0/275998802 wait complete. 2024-09-19T05:12:23.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.739+0000 7f398fea5740 1 -- start start 2024-09-19T05:12:23.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.739+0000 7f398fea5740 1 Processor -- start 2024-09-19T05:12:23.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.740+0000 7f398fea5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eacda74c70 0x55eacda7cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:23.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.740+0000 7f398fea5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eacda75590 0x55eacda7d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:23.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.740+0000 7f398fea5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eacda7fe50 0x55eacda7d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:23.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.740+0000 7f398fea5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55eacda82a20 con 0x55eacda74c70 2024-09-19T05:12:23.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.740+0000 7f398fea5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55eacd9960f0 con 0x55eacda75590 2024-09-19T05:12:23.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.740+0000 7f398fea5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55eacda82ba0 con 0x55eacda7fe50 2024-09-19T05:12:23.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.740+0000 7f398fea5740 10 client.0 _ll_register_callbacks cb 0x55eacd97dbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:12:23.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.740+0000 7f3986800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eacda74c70 0x55eacda7cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:23.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.740+0000 7f397fe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eacda75590 0x55eacda7d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:23.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.740+0000 7f3987200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eacda7fe50 0x55eacda7d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:23.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.740+0000 7f3986800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eacda74c70 0x55eacda7cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37820/0 (socket says 172.21.15.43:37820) 2024-09-19T05:12:23.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.740+0000 7f3986800640 1 -- 172.21.15.43:0/428299592 learned_addr learned my addr 172.21.15.43:0/428299592 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:23.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.740+0000 7f3986800640 1 -- 172.21.15.43:0/428299592 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eacda7fe50 msgr2=0x55eacda7d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:23.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.740+0000 7f3986800640 1 --2- 172.21.15.43:0/428299592 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eacda7fe50 0x55eacda7d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:23.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.740+0000 7f3986800640 1 -- 172.21.15.43:0/428299592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eacda75590 msgr2=0x55eacda7d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:23.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.740+0000 7f3986800640 1 --2- 172.21.15.43:0/428299592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eacda75590 0x55eacda7d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:23.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.740+0000 7f3986800640 1 -- 172.21.15.43:0/428299592 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eacd74c3c0 con 0x55eacda74c70 2024-09-19T05:12:23.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.741+0000 7f3986800640 1 --2- 172.21.15.43:0/428299592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eacda74c70 0x55eacda7cc50 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f397802a930 tx=0x7f397802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f66e1f8315f8f3c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:23.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.741+0000 7f396be00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:12:23.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.741+0000 7f396be00640 1 -- 172.21.15.43:0/428299592 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3978040030 con 0x55eacda74c70 2024-09-19T05:12:23.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.741+0000 7f396be00640 1 -- 172.21.15.43:0/428299592 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f3978004050 con 0x55eacda74c70 2024-09-19T05:12:23.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.741+0000 7f398fea5740 1 -- 172.21.15.43:0/428299592 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3978031870 con 0x55eacda74c70 2024-09-19T05:12:23.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.741+0000 7f396be00640 1 -- 172.21.15.43:0/428299592 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3978056050 con 0x55eacda74c70 2024-09-19T05:12:23.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.741+0000 7f398fea5740 10 client.17016 fetch_fsmap learned FSMap version 29 2024-09-19T05:12:23.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.741+0000 7f398fea5740 1 -- 172.21.15.43:0/428299592 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55eacda8e9b0 con 0x55eacda74c70 2024-09-19T05:12:23.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.742+0000 7f396be00640 1 -- 172.21.15.43:0/428299592 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3978031870 con 0x55eacda74c70 2024-09-19T05:12:23.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.742+0000 7f396be00640 1 -- 172.21.15.43:0/428299592 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f39780035f0 con 0x55eacda74c70 2024-09-19T05:12:23.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.742+0000 7f398fea5740 10 client.17016 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:12:23.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.742+0000 7f398fea5740 10 client.17016 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-19T05:12:23.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.742+0000 7f398fea5740 20 client.17016 populate_metadata read hostname 'smithi043' 2024-09-19T05:12:23.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.742+0000 7f398fea5740 1 --2- 172.21.15.43:0/428299592 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55eacda8fd00 0x55eacda920f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:23.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.742+0000 7f398fea5740 4 client.17016 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:12:23.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.742+0000 7f398fea5740 1 -- 172.21.15.43:0/428299592 --> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eacd9e8950 con 0x55eacda8fd00 2024-09-19T05:12:23.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.742+0000 7f397fe00640 1 --2- 172.21.15.43:0/428299592 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55eacda8fd00 0x55eacda920f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:23.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.743+0000 7f397fe00640 1 --2- 172.21.15.43:0/428299592 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55eacda8fd00 0x55eacda920f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dc70d6d8f6929855 server_cookie=268fc6a838380f6a in_seq=0 out_seq=0 2024-09-19T05:12:23.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.743+0000 7f396be00640 10 client.17016 ms_handle_connect on v2:172.21.15.43:6838/3404555030 2024-09-19T05:12:23.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.744+0000 7f396be00640 1 -- 172.21.15.43:0/428299592 <== mds.1 v2:172.21.15.43:6838/3404555030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eacd9e8950 con 0x55eacda8fd00 2024-09-19T05:12:23.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.744+0000 7f396be00640 10 client.17016 handle_command_reply: tid=0 2024-09-19T05:12:23.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.744+0000 7f398fea5740 1 client.17016 shutdown 2024-09-19T05:12:23.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.744+0000 7f398fea5740 1 -- 172.21.15.43:0/428299592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eacda74c70 msgr2=0x55eacda7cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:23.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.744+0000 7f398fea5740 1 --2- 172.21.15.43:0/428299592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eacda74c70 0x55eacda7cc50 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f397802a930 tx=0x7f397802a410 comp rx=0 tx=0).stop 2024-09-19T05:12:23.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.744+0000 7f398fea5740 1 -- 172.21.15.43:0/428299592 shutdown_connections 2024-09-19T05:12:23.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.744+0000 7f398fea5740 1 --2- 172.21.15.43:0/428299592 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55eacda8fd00 0x55eacda920f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:23.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.744+0000 7f398fea5740 1 --2- 172.21.15.43:0/428299592 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eacda7fe50 0x55eacda7d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:23.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.744+0000 7f398fea5740 1 --2- 172.21.15.43:0/428299592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eacda75590 0x55eacda7d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:23.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.744+0000 7f398fea5740 1 --2- 172.21.15.43:0/428299592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eacda74c70 0x55eacda7cc50 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:23.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.744+0000 7f398fea5740 1 -- 172.21.15.43:0/428299592 >> 172.21.15.43:0/428299592 conn(0x55eacda71200 msgr2=0x55eacda76ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:23.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.744+0000 7f396be00640 0 client.17016 ms_handle_reset on v2:172.21.15.43:6838/3404555030 2024-09-19T05:12:23.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.744+0000 7f398fea5740 1 -- 172.21.15.43:0/428299592 shutdown_connections 2024-09-19T05:12:23.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.744+0000 7f398fea5740 1 -- 172.21.15.43:0/428299592 wait complete. 2024-09-19T05:12:23.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.744+0000 7f398fea5740 20 client.17016 trim_cache size 0 max 16384 2024-09-19T05:12:23.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.758+0000 7f398fea5740 1 Processor -- start 2024-09-19T05:12:23.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.758+0000 7f398fea5740 1 -- start start 2024-09-19T05:12:23.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.758+0000 7f398fea5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eacda74c70 0x55eacda8ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:23.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.758+0000 7f398fea5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eacda75590 0x55eacda79eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:23.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.759+0000 7f398fea5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eacda7fe50 0x55eacda7a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:23.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.759+0000 7f398fea5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f39881afab0 con 0x55eacda75590 2024-09-19T05:12:23.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.759+0000 7f398fea5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f39881b15f0 con 0x55eacda7fe50 2024-09-19T05:12:23.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.759+0000 7f398fea5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f39881b0180 con 0x55eacda74c70 2024-09-19T05:12:23.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.759+0000 7f397fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eacda75590 0x55eacda79eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:23.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.759+0000 7f397fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eacda75590 0x55eacda79eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37826/0 (socket says 172.21.15.43:37826) 2024-09-19T05:12:23.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.759+0000 7f397fe00640 1 -- 172.21.15.43:0/3598171871 learned_addr learned my addr 172.21.15.43:0/3598171871 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:23.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.759+0000 7f3987200640 1 --2- 172.21.15.43:0/3598171871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eacda7fe50 0x55eacda7a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:23.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.759+0000 7f397fe00640 1 -- 172.21.15.43:0/3598171871 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eacda74c70 msgr2=0x55eacda8ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:23.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.759+0000 7f397fe00640 1 --2- 172.21.15.43:0/3598171871 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eacda74c70 0x55eacda8ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:23.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.759+0000 7f397fe00640 1 -- 172.21.15.43:0/3598171871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eacda7fe50 msgr2=0x55eacda7a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:23.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.759+0000 7f397fe00640 1 --2- 172.21.15.43:0/3598171871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eacda7fe50 0x55eacda7a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:23.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.759+0000 7f397fe00640 1 -- 172.21.15.43:0/3598171871 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eacda71030 con 0x55eacda75590 2024-09-19T05:12:23.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.759+0000 7f3987200640 1 --2- 172.21.15.43:0/3598171871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eacda7fe50 0x55eacda7a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:12:23.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.760+0000 7f397fe00640 1 --2- 172.21.15.43:0/3598171871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eacda75590 0x55eacda79eb0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3970009620 tx=0x7f3970004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d099667ccbccfd98 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:23.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.760+0000 7f3976a00640 1 -- 172.21.15.43:0/3598171871 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3970011d90 con 0x55eacda75590 2024-09-19T05:12:23.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.760+0000 7f3976a00640 1 -- 172.21.15.43:0/3598171871 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3970005500 con 0x55eacda75590 2024-09-19T05:12:23.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.760+0000 7f3976a00640 1 -- 172.21.15.43:0/3598171871 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39700047a0 con 0x55eacda75590 2024-09-19T05:12:23.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.761+0000 7f398fea5740 1 -- 172.21.15.43:0/3598171871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eacda75590 msgr2=0x55eacda79eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:23.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.761+0000 7f398fea5740 1 --2- 172.21.15.43:0/3598171871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eacda75590 0x55eacda79eb0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3970009620 tx=0x7f3970004730 comp rx=0 tx=0).stop 2024-09-19T05:12:23.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.761+0000 7f398fea5740 1 -- 172.21.15.43:0/3598171871 shutdown_connections 2024-09-19T05:12:23.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.761+0000 7f398fea5740 1 --2- 172.21.15.43:0/3598171871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eacda7fe50 0x55eacda7a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:23.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.761+0000 7f398fea5740 1 --2- 172.21.15.43:0/3598171871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eacda75590 0x55eacda79eb0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:23.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.761+0000 7f398fea5740 1 --2- 172.21.15.43:0/3598171871 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eacda74c70 0x55eacda8ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:23.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.761+0000 7f398fea5740 1 -- 172.21.15.43:0/3598171871 >> 172.21.15.43:0/3598171871 conn(0x55eacda71200 msgr2=0x55eacda76ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:23.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.761+0000 7f398fea5740 1 -- 172.21.15.43:0/3598171871 shutdown_connections 2024-09-19T05:12:23.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.761+0000 7f398fea5740 1 -- 172.21.15.43:0/3598171871 wait complete. 2024-09-19T05:12:23.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.761+0000 7f398fea5740 1 -- start start 2024-09-19T05:12:23.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.761+0000 7f398fea5740 1 Processor -- start 2024-09-19T05:12:23.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.762+0000 7f398fea5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eacda74c70 0x55eacda91080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:23.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.762+0000 7f398fea5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eacda75590 0x55eacda915c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:23.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.762+0000 7f398fea5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eacda7fe50 0x55eacda91b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:23.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.762+0000 7f398fea5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f39881b0f20 con 0x55eacda75590 2024-09-19T05:12:23.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.762+0000 7f398fea5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f39881b0850 con 0x55eacda7fe50 2024-09-19T05:12:23.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.762+0000 7f398fea5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f39881adab0 con 0x55eacda74c70 2024-09-19T05:12:23.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.762+0000 7f398fea5740 10 client.0 _ll_register_callbacks cb 0x7f39700116d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:12:23.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.762+0000 7f3986800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eacda74c70 0x55eacda91080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:23.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.762+0000 7f397fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eacda75590 0x55eacda915c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:23.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.762+0000 7f3986800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eacda74c70 0x55eacda91080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:38550/0 (socket says 172.21.15.43:38550) 2024-09-19T05:12:23.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.762+0000 7f3986800640 1 -- 172.21.15.43:0/1836566590 learned_addr learned my addr 172.21.15.43:0/1836566590 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:23.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.762+0000 7f3987200640 1 --2- 172.21.15.43:0/1836566590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eacda7fe50 0x55eacda91b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:23.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.763+0000 7f3986800640 1 -- 172.21.15.43:0/1836566590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eacda7fe50 msgr2=0x55eacda91b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:23.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.763+0000 7f3986800640 1 --2- 172.21.15.43:0/1836566590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eacda7fe50 0x55eacda91b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:23.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.763+0000 7f3986800640 1 -- 172.21.15.43:0/1836566590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eacda75590 msgr2=0x55eacda915c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:23.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.763+0000 7f3986800640 1 --2- 172.21.15.43:0/1836566590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eacda75590 0x55eacda915c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:23.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.763+0000 7f3986800640 1 -- 172.21.15.43:0/1836566590 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eacdaf9830 con 0x55eacda74c70 2024-09-19T05:12:23.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.763+0000 7f397fe00640 1 --2- 172.21.15.43:0/1836566590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eacda75590 0x55eacda915c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:12:23.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.763+0000 7f3986800640 1 --2- 172.21.15.43:0/1836566590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eacda74c70 0x55eacda91080 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3978003990 tx=0x7f39780058d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97c4875c7015dd75 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:23.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.763+0000 7f396be00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:12:23.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.763+0000 7f396be00640 1 -- 172.21.15.43:0/1836566590 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3978040040 con 0x55eacda74c70 2024-09-19T05:12:23.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.763+0000 7f398fea5740 1 -- 172.21.15.43:0/1836566590 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3970005500 con 0x55eacda74c70 2024-09-19T05:12:23.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.763+0000 7f396be00640 1 -- 172.21.15.43:0/1836566590 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f3978031870 con 0x55eacda74c70 2024-09-19T05:12:23.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.764+0000 7f396be00640 1 -- 172.21.15.43:0/1836566590 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3978068050 con 0x55eacda74c70 2024-09-19T05:12:23.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.764+0000 7f398fea5740 10 client.26974 fetch_fsmap learned FSMap version 29 2024-09-19T05:12:23.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.764+0000 7f398fea5740 1 -- 172.21.15.43:0/1836566590 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55eacdafab10 con 0x55eacda74c70 2024-09-19T05:12:23.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.764+0000 7f396be00640 1 -- 172.21.15.43:0/1836566590 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3970005500 con 0x55eacda74c70 2024-09-19T05:12:23.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.764+0000 7f396be00640 1 -- 172.21.15.43:0/1836566590 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3978003bb0 con 0x55eacda74c70 2024-09-19T05:12:23.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.764+0000 7f398fea5740 10 client.26974 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:12:23.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.764+0000 7f398fea5740 10 client.26974 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-19T05:12:23.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.764+0000 7f398fea5740 20 client.26974 populate_metadata read hostname 'smithi043' 2024-09-19T05:12:23.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.764+0000 7f398fea5740 1 --2- 172.21.15.43:0/1836566590 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55eacdafb220 0x55eacdafd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:23.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.764+0000 7f398fea5740 4 client.26974 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:12:23.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.764+0000 7f398fea5740 1 -- 172.21.15.43:0/1836566590 --> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eacd9e2e60 con 0x55eacdafb220 2024-09-19T05:12:23.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.764+0000 7f397fe00640 1 --2- 172.21.15.43:0/1836566590 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55eacdafb220 0x55eacdafd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:23.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.765+0000 7f397fe00640 1 --2- 172.21.15.43:0/1836566590 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55eacdafb220 0x55eacdafd640 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e5db83b1c5dba3b8 server_cookie=94b1be01ef9f9ef in_seq=0 out_seq=0 2024-09-19T05:12:23.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.765+0000 7f396be00640 10 client.26974 ms_handle_connect on v2:172.21.15.43:6838/3404555030 2024-09-19T05:12:23.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.765+0000 7f396be00640 1 -- 172.21.15.43:0/1836566590 <== mds.1 v2:172.21.15.43:6838/3404555030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eacd9e2e60 con 0x55eacdafb220 2024-09-19T05:12:23.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.765+0000 7f396be00640 10 client.26974 handle_command_reply: tid=0 2024-09-19T05:12:23.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.765+0000 7f398fea5740 1 client.26974 shutdown 2024-09-19T05:12:23.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.765+0000 7f398fea5740 1 -- 172.21.15.43:0/1836566590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eacda74c70 msgr2=0x55eacda91080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:23.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.765+0000 7f398fea5740 1 --2- 172.21.15.43:0/1836566590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eacda74c70 0x55eacda91080 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3978003990 tx=0x7f39780058d0 comp rx=0 tx=0).stop 2024-09-19T05:12:23.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.766+0000 7f398fea5740 1 -- 172.21.15.43:0/1836566590 shutdown_connections 2024-09-19T05:12:23.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.766+0000 7f398fea5740 1 --2- 172.21.15.43:0/1836566590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eacda7fe50 0x55eacda91b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:23.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.766+0000 7f398fea5740 1 --2- 172.21.15.43:0/1836566590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eacda75590 0x55eacda915c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:23.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.766+0000 7f398fea5740 1 --2- 172.21.15.43:0/1836566590 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55eacdafb220 0x55eacdafd640 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:23.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.766+0000 7f398fea5740 1 --2- 172.21.15.43:0/1836566590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eacda74c70 0x55eacda91080 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:23.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.766+0000 7f396be00640 0 client.26974 ms_handle_reset on v2:172.21.15.43:6838/3404555030 2024-09-19T05:12:23.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.766+0000 7f398fea5740 1 -- 172.21.15.43:0/1836566590 >> 172.21.15.43:0/1836566590 conn(0x55eacda71200 msgr2=0x55eacda7b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:23.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.766+0000 7f398fea5740 1 -- 172.21.15.43:0/1836566590 shutdown_connections 2024-09-19T05:12:23.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.766+0000 7f398fea5740 1 -- 172.21.15.43:0/1836566590 wait complete. 2024-09-19T05:12:23.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.766+0000 7f398fea5740 20 client.26974 trim_cache size 0 max 16384 2024-09-19T05:12:23.773 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:12:23.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.766+0000 7f398d400640 1 -- 172.21.15.43:0/2422453917 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f396c0a33b0 msgr2=0x7f396c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:23.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.767+0000 7f398d400640 1 --2- 172.21.15.43:0/2422453917 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f396c0a33b0 0x7f396c0a5870 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x55eacda72580 tx=0x7f3970005f90 comp rx=0 tx=0).stop 2024-09-19T05:12:23.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.767+0000 7f398d400640 1 -- 172.21.15.43:0/2422453917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3988109120 msgr2=0x7f398819eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:23.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.767+0000 7f398d400640 1 --2- 172.21.15.43:0/2422453917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3988109120 0x7f398819eec0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3980009fd0 tx=0x7f398000ef10 comp rx=0 tx=0).stop 2024-09-19T05:12:23.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.767+0000 7f398d400640 1 -- 172.21.15.43:0/2422453917 shutdown_connections 2024-09-19T05:12:23.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.767+0000 7f398d400640 1 --2- 172.21.15.43:0/2422453917 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f396c0a33b0 0x7f396c0a5870 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:23.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.767+0000 7f398d400640 1 --2- 172.21.15.43:0/2422453917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3988109120 0x7f398819eec0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:23.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.767+0000 7f398d400640 1 --2- 172.21.15.43:0/2422453917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39881049a0 0x7f398819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:23.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.767+0000 7f398d400640 1 --2- 172.21.15.43:0/2422453917 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3988103ff0 0x7f398819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:23.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.767+0000 7f398d400640 1 -- 172.21.15.43:0/2422453917 >> 172.21.15.43:0/2422453917 conn(0x7f39880ffa90 msgr2=0x7f3988100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:23.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.767+0000 7f398d400640 1 -- 172.21.15.43:0/2422453917 shutdown_connections 2024-09-19T05:12:23.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:23.767+0000 7f398d400640 1 -- 172.21.15.43:0/2422453917 wait complete. 2024-09-19T05:12:24.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:24 smithi151 ceph-mon[35425]: pgmap v239: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T05:12:24.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:24 smithi096 ceph-mon[35085]: pgmap v239: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T05:12:24.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:24 smithi043 ceph-mon[30800]: pgmap v239: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T05:12:24.710 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T05:12:25.090 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:12:25.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.637+0000 7f2096000640 1 -- 172.21.15.43:0/4186159312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2090103e50 msgr2=0x7f2090104230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:25.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.637+0000 7f2096000640 1 --2- 172.21.15.43:0/4186159312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2090103e50 0x7f2090104230 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2084009900 tx=0x7f208402a410 comp rx=0 tx=0).stop 2024-09-19T05:12:25.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.637+0000 7f2096000640 1 -- 172.21.15.43:0/4186159312 shutdown_connections 2024-09-19T05:12:25.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.637+0000 7f2096000640 1 --2- 172.21.15.43:0/4186159312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2090109090 0x7f2090110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.637+0000 7f2096000640 1 --2- 172.21.15.43:0/4186159312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2090104800 0x7f2090108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.637+0000 7f2096000640 1 --2- 172.21.15.43:0/4186159312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2090103e50 0x7f2090104230 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.637+0000 7f2096000640 1 -- 172.21.15.43:0/4186159312 >> 172.21.15.43:0/4186159312 conn(0x7f20900ff8b0 msgr2=0x7f2090101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:25.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.637+0000 7f2096000640 1 -- 172.21.15.43:0/4186159312 shutdown_connections 2024-09-19T05:12:25.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.637+0000 7f2096000640 1 -- 172.21.15.43:0/4186159312 wait complete. 2024-09-19T05:12:25.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.638+0000 7f2096000640 1 Processor -- start 2024-09-19T05:12:25.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.639+0000 7f2096000640 1 -- start start 2024-09-19T05:12:25.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.639+0000 7f2096000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2090103e50 0x7f209019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:25.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.639+0000 7f2096000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2090104800 0x7f209019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:25.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.639+0000 7f2096000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2090109090 0x7f209019ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:25.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.639+0000 7f2096000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f20901140e0 con 0x7f2090103e50 2024-09-19T05:12:25.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.639+0000 7f2096000640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2090113f60 con 0x7f2090109090 2024-09-19T05:12:25.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.639+0000 7f2096000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2090114260 con 0x7f2090104800 2024-09-19T05:12:25.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.639+0000 7f2094c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2090103e50 0x7f209019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:25.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.639+0000 7f2094c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2090103e50 0x7f209019a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37866/0 (socket says 172.21.15.43:37866) 2024-09-19T05:12:25.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.639+0000 7f2094c00640 1 -- 172.21.15.43:0/3561936752 learned_addr learned my addr 172.21.15.43:0/3561936752 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:25.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.639+0000 7f208fe00640 1 --2- 172.21.15.43:0/3561936752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2090104800 0x7f209019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:25.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.639+0000 7f2095600640 1 --2- 172.21.15.43:0/3561936752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2090109090 0x7f209019ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:25.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.640+0000 7f208fe00640 1 -- 172.21.15.43:0/3561936752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2090109090 msgr2=0x7f209019ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:25.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.640+0000 7f208fe00640 1 --2- 172.21.15.43:0/3561936752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2090109090 0x7f209019ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.640+0000 7f208fe00640 1 -- 172.21.15.43:0/3561936752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2090103e50 msgr2=0x7f209019a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:25.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.640+0000 7f208fe00640 1 --2- 172.21.15.43:0/3561936752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2090103e50 0x7f209019a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.640+0000 7f208fe00640 1 -- 172.21.15.43:0/3561936752 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f209019f510 con 0x7f2090104800 2024-09-19T05:12:25.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.640+0000 7f2094c00640 1 --2- 172.21.15.43:0/3561936752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2090103e50 0x7f209019a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:12:25.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.640+0000 7f208fe00640 1 --2- 172.21.15.43:0/3561936752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2090104800 0x7f209019aaa0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f207c00e9f0 tx=0x7f207c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:25.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.640+0000 7f208d600640 1 -- 172.21.15.43:0/3561936752 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c00ccc0 con 0x7f2090104800 2024-09-19T05:12:25.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.641+0000 7f208d600640 1 -- 172.21.15.43:0/3561936752 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f207c010910 con 0x7f2090104800 2024-09-19T05:12:25.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.641+0000 7f2096000640 1 -- 172.21.15.43:0/3561936752 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f209019f800 con 0x7f2090104800 2024-09-19T05:12:25.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.641+0000 7f208d600640 1 -- 172.21.15.43:0/3561936752 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c018960 con 0x7f2090104800 2024-09-19T05:12:25.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.641+0000 7f2096000640 1 -- 172.21.15.43:0/3561936752 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20901a70e0 con 0x7f2090104800 2024-09-19T05:12:25.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.643+0000 7f209a1f9740 1 Processor -- start 2024-09-19T05:12:25.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.644+0000 7f209a1f9740 1 -- start start 2024-09-19T05:12:25.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.644+0000 7f209a1f9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561242d36e40 0x561242d37220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:25.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.644+0000 7f209a1f9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561242d37760 0x561242d41ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:25.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.644+0000 7f208d600640 1 -- 172.21.15.43:0/3561936752 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f207c010ac0 con 0x7f2090104800 2024-09-19T05:12:25.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.644+0000 7f209a1f9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561242d42020 0x561242d44410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:25.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.644+0000 7f209a1f9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561242c9d3e0 con 0x561242d42020 2024-09-19T05:12:25.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.644+0000 7f209a1f9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561242c44d80 con 0x561242d36e40 2024-09-19T05:12:25.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.644+0000 7f209a1f9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561242c993a0 con 0x561242d37760 2024-09-19T05:12:25.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.644+0000 7f2095600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561242d36e40 0x561242d37220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:25.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.644+0000 7f2095600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561242d36e40 0x561242d37220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:48950/0 (socket says 172.21.15.43:48950) 2024-09-19T05:12:25.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.644+0000 7f208fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561242d42020 0x561242d44410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:25.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.644+0000 7f2095600640 1 -- 172.21.15.43:0/1256340217 learned_addr learned my addr 172.21.15.43:0/1256340217 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:25.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.644+0000 7f2094c00640 1 --2- 172.21.15.43:0/1256340217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561242d37760 0x561242d41ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:25.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.645+0000 7f2094c00640 1 -- 172.21.15.43:0/1256340217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561242d36e40 msgr2=0x561242d37220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:25.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.645+0000 7f2094c00640 1 --2- 172.21.15.43:0/1256340217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561242d36e40 0x561242d37220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.645+0000 7f2094c00640 1 -- 172.21.15.43:0/1256340217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561242d42020 msgr2=0x561242d44410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:25.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.645+0000 7f2094c00640 1 --2- 172.21.15.43:0/1256340217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561242d42020 0x561242d44410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.645+0000 7f2094c00640 1 -- 172.21.15.43:0/1256340217 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561242c63ae0 con 0x561242d37760 2024-09-19T05:12:25.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.645+0000 7f2095600640 1 --2- 172.21.15.43:0/1256340217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561242d36e40 0x561242d37220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:12:25.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.645+0000 7f2094c00640 1 --2- 172.21.15.43:0/1256340217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561242d37760 0x561242d41ae0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2084031550 tx=0x7f2084005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd760027c633a528 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:25.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.646+0000 7f2081c00640 1 -- 172.21.15.43:0/1256340217 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2084040070 con 0x561242d37760 2024-09-19T05:12:25.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.646+0000 7f2081c00640 1 -- 172.21.15.43:0/1256340217 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20840047a0 con 0x561242d37760 2024-09-19T05:12:25.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.646+0000 7f2081c00640 1 -- 172.21.15.43:0/1256340217 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2084035040 con 0x561242d37760 2024-09-19T05:12:25.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.646+0000 7f209a1f9740 1 -- 172.21.15.43:0/1256340217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561242d37760 msgr2=0x561242d41ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:25.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.646+0000 7f209a1f9740 1 --2- 172.21.15.43:0/1256340217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561242d37760 0x561242d41ae0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2084031550 tx=0x7f2084005dd0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.646+0000 7f209a1f9740 1 -- 172.21.15.43:0/1256340217 shutdown_connections 2024-09-19T05:12:25.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.646+0000 7f209a1f9740 1 --2- 172.21.15.43:0/1256340217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561242d42020 0x561242d44410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.646+0000 7f209a1f9740 1 --2- 172.21.15.43:0/1256340217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561242d37760 0x561242d41ae0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.646+0000 7f209a1f9740 1 --2- 172.21.15.43:0/1256340217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561242d36e40 0x561242d37220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.646+0000 7f209a1f9740 1 -- 172.21.15.43:0/1256340217 >> 172.21.15.43:0/1256340217 conn(0x561242d333d0 msgr2=0x561242d337b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:25.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.646+0000 7f208d600640 1 --2- 172.21.15.43:0/3561936752 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f20780a33b0 0x7f20780a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:25.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.647+0000 7f209a1f9740 1 -- 172.21.15.43:0/1256340217 shutdown_connections 2024-09-19T05:12:25.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.647+0000 7f208d600640 1 -- 172.21.15.43:0/3561936752 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f207c0d0120 con 0x7f2090104800 2024-09-19T05:12:25.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.647+0000 7f209a1f9740 1 -- 172.21.15.43:0/1256340217 wait complete. 2024-09-19T05:12:25.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.647+0000 7f2095600640 1 --2- 172.21.15.43:0/3561936752 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f20780a33b0 0x7f20780a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:25.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.647+0000 7f209a1f9740 1 -- start start 2024-09-19T05:12:25.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.647+0000 7f2095600640 1 --2- 172.21.15.43:0/3561936752 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f20780a33b0 0x7f20780a5870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2088005fd0 tx=0x7f2088009500 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:25.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.648+0000 7f209a1f9740 1 Processor -- start 2024-09-19T05:12:25.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.648+0000 7f209a1f9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561242d36e40 0x561242d3ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:25.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.648+0000 7f209a1f9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561242d37760 0x561242d3f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:25.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.648+0000 7f209a1f9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561242d42020 0x561242d3f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:25.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.648+0000 7f209a1f9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561242d44bf0 con 0x561242d37760 2024-09-19T05:12:25.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.648+0000 7f209a1f9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561242c580f0 con 0x561242d36e40 2024-09-19T05:12:25.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.648+0000 7f209a1f9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561242d44d70 con 0x561242d42020 2024-09-19T05:12:25.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.648+0000 7f209a1f9740 10 client.0 _ll_register_callbacks cb 0x561242c3fbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:12:25.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.648+0000 7f208fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561242d37760 0x561242d3f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:25.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.648+0000 7f2094c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561242d36e40 0x561242d3ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:25.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.648+0000 7f208fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561242d37760 0x561242d3f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37878/0 (socket says 172.21.15.43:37878) 2024-09-19T05:12:25.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.648+0000 7f208fe00640 1 -- 172.21.15.43:0/1925884256 learned_addr learned my addr 172.21.15.43:0/1925884256 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:25.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.649+0000 7f2095600640 1 --2- 172.21.15.43:0/1925884256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561242d42020 0x561242d3f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:25.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.649+0000 7f2094c00640 1 -- 172.21.15.43:0/1925884256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561242d42020 msgr2=0x561242d3f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:25.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.649+0000 7f2094c00640 1 --2- 172.21.15.43:0/1925884256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561242d42020 0x561242d3f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.649+0000 7f2094c00640 1 -- 172.21.15.43:0/1925884256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561242d37760 msgr2=0x561242d3f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:25.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.649+0000 7f2094c00640 1 --2- 172.21.15.43:0/1925884256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561242d37760 0x561242d3f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.649+0000 7f2094c00640 1 -- 172.21.15.43:0/1925884256 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561242a0e3c0 con 0x561242d36e40 2024-09-19T05:12:25.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.649+0000 7f208fe00640 1 --2- 172.21.15.43:0/1925884256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561242d37760 0x561242d3f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:12:25.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.649+0000 7f2094c00640 1 --2- 172.21.15.43:0/1925884256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561242d36e40 0x561242d3ef00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2084004770 tx=0x7f2084003920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a75f29e713f60aa5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:25.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.649+0000 7f2076000640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:12:25.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.649+0000 7f2076000640 1 -- 172.21.15.43:0/1925884256 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208404c020 con 0x561242d36e40 2024-09-19T05:12:25.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.650+0000 7f2076000640 1 -- 172.21.15.43:0/1925884256 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f20840315b0 con 0x561242d36e40 2024-09-19T05:12:25.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.650+0000 7f209a1f9740 1 -- 172.21.15.43:0/1925884256 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20840047a0 con 0x561242d36e40 2024-09-19T05:12:25.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.650+0000 7f2076000640 1 -- 172.21.15.43:0/1925884256 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2084057050 con 0x561242d36e40 2024-09-19T05:12:25.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.650+0000 7f209a1f9740 10 client.27056 fetch_fsmap learned FSMap version 29 2024-09-19T05:12:25.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.650+0000 7f209a1f9740 1 -- 172.21.15.43:0/1925884256 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561242d50c60 con 0x561242d36e40 2024-09-19T05:12:25.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.650+0000 7f2076000640 1 -- 172.21.15.43:0/1925884256 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20840047a0 con 0x561242d36e40 2024-09-19T05:12:25.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.651+0000 7f2076000640 1 -- 172.21.15.43:0/1925884256 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f208403c740 con 0x561242d36e40 2024-09-19T05:12:25.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.651+0000 7f209a1f9740 10 client.27056 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:12:25.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.651+0000 7f209a1f9740 10 client.27056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:12:25.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.651+0000 7f209a1f9740 20 client.27056 populate_metadata read hostname 'smithi043' 2024-09-19T05:12:25.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.651+0000 7f209a1f9740 1 --2- 172.21.15.43:0/1925884256 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x561242d51fb0 0x561242d543a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:25.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.651+0000 7f209a1f9740 4 client.27056 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:12:25.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.651+0000 7f209a1f9740 1 -- 172.21.15.43:0/1925884256 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561242caa950 con 0x561242d51fb0 2024-09-19T05:12:25.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.651+0000 7f208fe00640 1 --2- 172.21.15.43:0/1925884256 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x561242d51fb0 0x561242d543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:25.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.652+0000 7f208fe00640 1 --2- 172.21.15.43:0/1925884256 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x561242d51fb0 0x561242d543a0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49b13e8c003f0aa8 server_cookie=bab1e8940173c379 in_seq=0 out_seq=0 2024-09-19T05:12:25.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.652+0000 7f2076000640 10 client.27056 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:12:25.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.652+0000 7f2076000640 1 -- 172.21.15.43:0/1925884256 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561242caa950 con 0x561242d51fb0 2024-09-19T05:12:25.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.652+0000 7f2076000640 10 client.27056 handle_command_reply: tid=0 2024-09-19T05:12:25.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.653+0000 7f209a1f9740 1 client.27056 shutdown 2024-09-19T05:12:25.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.653+0000 7f209a1f9740 1 -- 172.21.15.43:0/1925884256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561242d36e40 msgr2=0x561242d3ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:25.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.653+0000 7f209a1f9740 1 --2- 172.21.15.43:0/1925884256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561242d36e40 0x561242d3ef00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2084004770 tx=0x7f2084003920 comp rx=0 tx=0).stop 2024-09-19T05:12:25.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.653+0000 7f209a1f9740 1 -- 172.21.15.43:0/1925884256 shutdown_connections 2024-09-19T05:12:25.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.653+0000 7f209a1f9740 1 --2- 172.21.15.43:0/1925884256 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x561242d51fb0 0x561242d543a0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.653+0000 7f209a1f9740 1 --2- 172.21.15.43:0/1925884256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561242d42020 0x561242d3f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.653+0000 7f2076000640 0 client.27056 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:12:25.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.653+0000 7f209a1f9740 1 --2- 172.21.15.43:0/1925884256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561242d37760 0x561242d3f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.653+0000 7f209a1f9740 1 --2- 172.21.15.43:0/1925884256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561242d36e40 0x561242d3ef00 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.653+0000 7f209a1f9740 1 -- 172.21.15.43:0/1925884256 >> 172.21.15.43:0/1925884256 conn(0x561242d333d0 msgr2=0x561242d324b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:25.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.653+0000 7f209a1f9740 1 -- 172.21.15.43:0/1925884256 shutdown_connections 2024-09-19T05:12:25.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.653+0000 7f209a1f9740 1 -- 172.21.15.43:0/1925884256 wait complete. 2024-09-19T05:12:25.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.653+0000 7f209a1f9740 20 client.27056 trim_cache size 0 max 16384 2024-09-19T05:12:25.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.667+0000 7f209a1f9740 1 Processor -- start 2024-09-19T05:12:25.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.667+0000 7f209a1f9740 1 -- start start 2024-09-19T05:12:25.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.668+0000 7f209a1f9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561242d36e40 0x561242d3c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:25.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.668+0000 7f209a1f9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561242d37760 0x561242d3c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:25.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.668+0000 7f209a1f9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561242d42020 0x561242d40760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:25.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.668+0000 7f209a1f9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f20901af8e0 con 0x561242d37760 2024-09-19T05:12:25.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.668+0000 7f209a1f9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f20901b1420 con 0x561242d42020 2024-09-19T05:12:25.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.668+0000 7f209a1f9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f20901affb0 con 0x561242d36e40 2024-09-19T05:12:25.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.668+0000 7f208fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561242d37760 0x561242d3c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:25.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.668+0000 7f208fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561242d37760 0x561242d3c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37882/0 (socket says 172.21.15.43:37882) 2024-09-19T05:12:25.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.668+0000 7f208fe00640 1 -- 172.21.15.43:0/3169652804 learned_addr learned my addr 172.21.15.43:0/3169652804 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:25.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.668+0000 7f2095600640 1 --2- 172.21.15.43:0/3169652804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561242d42020 0x561242d40760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:25.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.668+0000 7f2094c00640 1 --2- 172.21.15.43:0/3169652804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561242d36e40 0x561242d3c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:25.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.669+0000 7f208fe00640 1 -- 172.21.15.43:0/3169652804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561242d36e40 msgr2=0x561242d3c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:25.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.669+0000 7f208fe00640 1 --2- 172.21.15.43:0/3169652804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561242d36e40 0x561242d3c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.669+0000 7f208fe00640 1 -- 172.21.15.43:0/3169652804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561242d42020 msgr2=0x561242d40760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:25.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.669+0000 7f208fe00640 1 --2- 172.21.15.43:0/3169652804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561242d42020 0x561242d40760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.669+0000 7f208fe00640 1 -- 172.21.15.43:0/3169652804 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561242d3d450 con 0x561242d37760 2024-09-19T05:12:25.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.669+0000 7f208fe00640 1 --2- 172.21.15.43:0/3169652804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561242d37760 0x561242d3c6a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f207c0988a0 tx=0x7f207c0d25e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=741c01c2e5010ab6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:25.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.669+0000 7f2081200640 1 -- 172.21.15.43:0/3169652804 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c09bf80 con 0x561242d37760 2024-09-19T05:12:25.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.669+0000 7f2081200640 1 -- 172.21.15.43:0/3169652804 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f207c09c440 con 0x561242d37760 2024-09-19T05:12:25.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.669+0000 7f2081200640 1 -- 172.21.15.43:0/3169652804 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c09bda0 con 0x561242d37760 2024-09-19T05:12:25.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.670+0000 7f209a1f9740 1 -- 172.21.15.43:0/3169652804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561242d37760 msgr2=0x561242d3c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:25.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.670+0000 7f209a1f9740 1 --2- 172.21.15.43:0/3169652804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561242d37760 0x561242d3c6a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f207c0988a0 tx=0x7f207c0d25e0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.670+0000 7f209a1f9740 1 -- 172.21.15.43:0/3169652804 shutdown_connections 2024-09-19T05:12:25.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.670+0000 7f209a1f9740 1 --2- 172.21.15.43:0/3169652804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561242d42020 0x561242d40760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.670+0000 7f209a1f9740 1 --2- 172.21.15.43:0/3169652804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561242d37760 0x561242d3c6a0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.670+0000 7f209a1f9740 1 --2- 172.21.15.43:0/3169652804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561242d36e40 0x561242d3c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.670+0000 7f209a1f9740 1 -- 172.21.15.43:0/3169652804 >> 172.21.15.43:0/3169652804 conn(0x561242d333d0 msgr2=0x561242d32490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:25.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.671+0000 7f209a1f9740 1 -- 172.21.15.43:0/3169652804 shutdown_connections 2024-09-19T05:12:25.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.671+0000 7f209a1f9740 1 -- 172.21.15.43:0/3169652804 wait complete. 2024-09-19T05:12:25.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.671+0000 7f209a1f9740 1 -- start start 2024-09-19T05:12:25.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.671+0000 7f209a1f9740 1 Processor -- start 2024-09-19T05:12:25.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.672+0000 7f209a1f9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561242d36e40 0x561242d57840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:25.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.672+0000 7f209a1f9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561242d37760 0x561242d54b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:25.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.672+0000 7f209a1f9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561242d42020 0x561242d550d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:25.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.672+0000 7f209a1f9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f20901b0d50 con 0x561242d36e40 2024-09-19T05:12:25.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.672+0000 7f209a1f9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f20901b0680 con 0x561242d37760 2024-09-19T05:12:25.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.672+0000 7f209a1f9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f20901ad8e0 con 0x561242d42020 2024-09-19T05:12:25.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.672+0000 7f209a1f9740 10 client.0 _ll_register_callbacks cb 0x7f207c0d3aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:12:25.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.672+0000 7f2094c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561242d36e40 0x561242d57840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:25.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.672+0000 7f2094c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561242d36e40 0x561242d57840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37898/0 (socket says 172.21.15.43:37898) 2024-09-19T05:12:25.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.672+0000 7f2094c00640 1 -- 172.21.15.43:0/264409674 learned_addr learned my addr 172.21.15.43:0/264409674 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:25.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.672+0000 7f208fe00640 1 --2- 172.21.15.43:0/264409674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561242d37760 0x561242d54b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:25.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.673+0000 7f2095600640 1 --2- 172.21.15.43:0/264409674 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561242d42020 0x561242d550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:25.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.673+0000 7f2094c00640 1 -- 172.21.15.43:0/264409674 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561242d42020 msgr2=0x561242d550d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:25.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.673+0000 7f2094c00640 1 --2- 172.21.15.43:0/264409674 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561242d42020 0x561242d550d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.673+0000 7f2094c00640 1 -- 172.21.15.43:0/264409674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561242d37760 msgr2=0x561242d54b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:25.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.673+0000 7f2094c00640 1 --2- 172.21.15.43:0/264409674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561242d37760 0x561242d54b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.673+0000 7f2094c00640 1 -- 172.21.15.43:0/264409674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561242d55610 con 0x561242d36e40 2024-09-19T05:12:25.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.673+0000 7f2095600640 1 --2- 172.21.15.43:0/264409674 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561242d42020 0x561242d550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:12:25.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.673+0000 7f2094c00640 1 --2- 172.21.15.43:0/264409674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561242d36e40 0x561242d57840 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f208403cb90 tx=0x7f20840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=345d0a203f1447e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:25.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.673+0000 7f2076000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:12:25.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.673+0000 7f2076000640 1 -- 172.21.15.43:0/264409674 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208404c040 con 0x561242d36e40 2024-09-19T05:12:25.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.673+0000 7f2076000640 1 -- 172.21.15.43:0/264409674 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f208403c850 con 0x561242d36e40 2024-09-19T05:12:25.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.673+0000 7f209a1f9740 1 -- 172.21.15.43:0/264409674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f207c09c440 con 0x561242d36e40 2024-09-19T05:12:25.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.674+0000 7f2076000640 1 -- 172.21.15.43:0/264409674 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2084000e60 con 0x561242d36e40 2024-09-19T05:12:25.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.674+0000 7f209a1f9740 10 client.17040 fetch_fsmap learned FSMap version 29 2024-09-19T05:12:25.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.674+0000 7f209a1f9740 1 -- 172.21.15.43:0/264409674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561242dc4e10 con 0x561242d36e40 2024-09-19T05:12:25.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.674+0000 7f2076000640 1 -- 172.21.15.43:0/264409674 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f207c09c440 con 0x561242d36e40 2024-09-19T05:12:25.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.674+0000 7f2076000640 1 -- 172.21.15.43:0/264409674 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2084063410 con 0x561242d36e40 2024-09-19T05:12:25.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.675+0000 7f209a1f9740 10 client.17040 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:12:25.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.675+0000 7f209a1f9740 10 client.17040 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:12:25.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.675+0000 7f209a1f9740 20 client.17040 populate_metadata read hostname 'smithi043' 2024-09-19T05:12:25.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.675+0000 7f209a1f9740 1 --2- 172.21.15.43:0/264409674 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x561242dc5520 0x561242dc7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:25.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.675+0000 7f209a1f9740 4 client.17040 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:12:25.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.675+0000 7f209a1f9740 1 -- 172.21.15.43:0/264409674 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561242ca4e60 con 0x561242dc5520 2024-09-19T05:12:25.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.675+0000 7f208fe00640 1 --2- 172.21.15.43:0/264409674 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x561242dc5520 0x561242dc7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:25.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.676+0000 7f2076000640 10 client.17040 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:12:25.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.676+0000 7f208fe00640 1 --2- 172.21.15.43:0/264409674 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x561242dc5520 0x561242dc7940 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28d85091ffc2ad66 server_cookie=5acd9dbe7729784e in_seq=0 out_seq=0 2024-09-19T05:12:25.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.676+0000 7f2076000640 1 -- 172.21.15.43:0/264409674 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561242ca4e60 con 0x561242dc5520 2024-09-19T05:12:25.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.676+0000 7f2076000640 10 client.17040 handle_command_reply: tid=0 2024-09-19T05:12:25.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.676+0000 7f209a1f9740 1 client.17040 shutdown 2024-09-19T05:12:25.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.676+0000 7f209a1f9740 1 -- 172.21.15.43:0/264409674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561242d36e40 msgr2=0x561242d57840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:25.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.676+0000 7f209a1f9740 1 --2- 172.21.15.43:0/264409674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561242d36e40 0x561242d57840 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f208403cb90 tx=0x7f20840046c0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.676+0000 7f209a1f9740 1 -- 172.21.15.43:0/264409674 shutdown_connections 2024-09-19T05:12:25.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.676+0000 7f209a1f9740 1 --2- 172.21.15.43:0/264409674 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x561242dc5520 0x561242dc7940 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.676+0000 7f209a1f9740 1 --2- 172.21.15.43:0/264409674 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561242d42020 0x561242d550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.676+0000 7f209a1f9740 1 --2- 172.21.15.43:0/264409674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561242d37760 0x561242d54b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.676+0000 7f209a1f9740 1 --2- 172.21.15.43:0/264409674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561242d36e40 0x561242d57840 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.676+0000 7f209a1f9740 1 -- 172.21.15.43:0/264409674 >> 172.21.15.43:0/264409674 conn(0x561242d333d0 msgr2=0x561242d3d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:25.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.676+0000 7f2076000640 0 client.17040 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:12:25.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.676+0000 7f209a1f9740 1 -- 172.21.15.43:0/264409674 shutdown_connections 2024-09-19T05:12:25.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.677+0000 7f209a1f9740 1 -- 172.21.15.43:0/264409674 wait complete. 2024-09-19T05:12:25.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.677+0000 7f209a1f9740 20 client.17040 trim_cache size 0 max 16384 2024-09-19T05:12:25.683 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:12:25.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.677+0000 7f2096000640 1 -- 172.21.15.43:0/3561936752 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f20780a33b0 msgr2=0x7f20780a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:25.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.677+0000 7f2096000640 1 --2- 172.21.15.43:0/3561936752 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f20780a33b0 0x7f20780a5870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2088005fd0 tx=0x7f2088009500 comp rx=0 tx=0).stop 2024-09-19T05:12:25.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.677+0000 7f2096000640 1 -- 172.21.15.43:0/3561936752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2090104800 msgr2=0x7f209019aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:25.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.677+0000 7f2096000640 1 --2- 172.21.15.43:0/3561936752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2090104800 0x7f209019aaa0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f207c00e9f0 tx=0x7f207c00eeb0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.677+0000 7f2096000640 1 -- 172.21.15.43:0/3561936752 shutdown_connections 2024-09-19T05:12:25.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.677+0000 7f2096000640 1 --2- 172.21.15.43:0/3561936752 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f20780a33b0 0x7f20780a5870 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.677+0000 7f2096000640 1 --2- 172.21.15.43:0/3561936752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2090109090 0x7f209019ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.677+0000 7f2096000640 1 --2- 172.21.15.43:0/3561936752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2090104800 0x7f209019aaa0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.677+0000 7f2096000640 1 --2- 172.21.15.43:0/3561936752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2090103e50 0x7f209019a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:25.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.678+0000 7f2096000640 1 -- 172.21.15.43:0/3561936752 >> 172.21.15.43:0/3561936752 conn(0x7f20900ff8b0 msgr2=0x7f2090100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:25.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.678+0000 7f2096000640 1 -- 172.21.15.43:0/3561936752 shutdown_connections 2024-09-19T05:12:25.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:25.678+0000 7f2096000640 1 -- 172.21.15.43:0/3561936752 wait complete. 2024-09-19T05:12:26.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:26 smithi151 ceph-mon[35425]: pgmap v240: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T05:12:26.413 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:26 smithi043 ceph-mon[30800]: pgmap v240: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T05:12:26.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:26 smithi096 ceph-mon[35085]: pgmap v240: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T05:12:27.453 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T05:12:27.453 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T05:12:27.846 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:12:28.267 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:28 smithi043 ceph-mon[30800]: pgmap v241: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-19T05:12:28.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.383+0000 7f9317400640 1 -- 172.21.15.43:0/733129977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9310076720 msgr2=0x7f9310076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:28.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.383+0000 7f9317400640 1 --2- 172.21.15.43:0/733129977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9310076720 0x7f9310076b00 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9304009900 tx=0x7f930402a410 comp rx=0 tx=0).stop 2024-09-19T05:12:28.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.384+0000 7f9317400640 1 -- 172.21.15.43:0/733129977 shutdown_connections 2024-09-19T05:12:28.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.384+0000 7f9317400640 1 --2- 172.21.15.43:0/733129977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9310112a30 0x7f9310114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.384+0000 7f9317400640 1 --2- 172.21.15.43:0/733129977 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9310077040 0x7f93101124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.384+0000 7f9317400640 1 --2- 172.21.15.43:0/733129977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9310076720 0x7f9310076b00 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.384+0000 7f9317400640 1 -- 172.21.15.43:0/733129977 >> 172.21.15.43:0/733129977 conn(0x7f93100ffa90 msgr2=0x7f9310101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:28.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.384+0000 7f9317400640 1 -- 172.21.15.43:0/733129977 shutdown_connections 2024-09-19T05:12:28.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.384+0000 7f9317400640 1 -- 172.21.15.43:0/733129977 wait complete. 2024-09-19T05:12:28.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.385+0000 7f9317400640 1 Processor -- start 2024-09-19T05:12:28.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.385+0000 7f9317400640 1 -- start start 2024-09-19T05:12:28.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.385+0000 7f9317400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9310076720 0x7f93101a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:28.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.385+0000 7f9317400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9310077040 0x7f93101a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:28.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.385+0000 7f9317400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9310112a30 0x7f931019e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:28.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.385+0000 7f9317400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9310118610 con 0x7f9310076720 2024-09-19T05:12:28.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.385+0000 7f9317400640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9310118490 con 0x7f9310112a30 2024-09-19T05:12:28.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.385+0000 7f9317400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9310118790 con 0x7f9310077040 2024-09-19T05:12:28.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.385+0000 7f9314a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9310076720 0x7f93101a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:28.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.385+0000 7f9314a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9310076720 0x7f93101a4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51108/0 (socket says 172.21.15.43:51108) 2024-09-19T05:12:28.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.385+0000 7f9314a00640 1 -- 172.21.15.43:0/4232071056 learned_addr learned my addr 172.21.15.43:0/4232071056 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:28.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.386+0000 7f930fe00640 1 --2- 172.21.15.43:0/4232071056 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9310077040 0x7f93101a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:28.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.386+0000 7f9315400640 1 --2- 172.21.15.43:0/4232071056 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9310112a30 0x7f931019e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:28.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.386+0000 7f9314a00640 1 -- 172.21.15.43:0/4232071056 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9310077040 msgr2=0x7f93101a4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:28.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.386+0000 7f9314a00640 1 --2- 172.21.15.43:0/4232071056 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9310077040 0x7f93101a4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.386+0000 7f9314a00640 1 -- 172.21.15.43:0/4232071056 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9310112a30 msgr2=0x7f931019e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:28.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.386+0000 7f9314a00640 1 --2- 172.21.15.43:0/4232071056 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9310112a30 0x7f931019e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.386+0000 7f9314a00640 1 -- 172.21.15.43:0/4232071056 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f931019f010 con 0x7f9310076720 2024-09-19T05:12:28.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.386+0000 7f9315400640 1 --2- 172.21.15.43:0/4232071056 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9310112a30 0x7f931019e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:12:28.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.386+0000 7f9314a00640 1 --2- 172.21.15.43:0/4232071056 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9310076720 0x7f93101a4720 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9304004770 tx=0x7f9304003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:28.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.387+0000 7f930d600640 1 -- 172.21.15.43:0/4232071056 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930403f070 con 0x7f9310076720 2024-09-19T05:12:28.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.387+0000 7f9317400640 1 -- 172.21.15.43:0/4232071056 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f931019f2a0 con 0x7f9310076720 2024-09-19T05:12:28.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.387+0000 7f930d600640 1 -- 172.21.15.43:0/4232071056 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9304031ca0 con 0x7f9310076720 2024-09-19T05:12:28.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.387+0000 7f930d600640 1 -- 172.21.15.43:0/4232071056 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9304035030 con 0x7f9310076720 2024-09-19T05:12:28.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.387+0000 7f9317400640 1 -- 172.21.15.43:0/4232071056 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93101ab5f0 con 0x7f9310076720 2024-09-19T05:12:28.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.390+0000 7f9319e75740 1 Processor -- start 2024-09-19T05:12:28.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.390+0000 7f930d600640 1 -- 172.21.15.43:0/4232071056 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f9304042070 con 0x7f9310076720 2024-09-19T05:12:28.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.390+0000 7f9319e75740 1 -- start start 2024-09-19T05:12:28.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.390+0000 7f9319e75740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556920126ac0 0x556920126ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:28.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.390+0000 7f9319e75740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569201273e0 0x556920131760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:28.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.390+0000 7f9319e75740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556920131ca0 0x556920134090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:28.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.390+0000 7f9319e75740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55692008d290 con 0x5569201273e0 2024-09-19T05:12:28.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.390+0000 7f9319e75740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556920034b40 con 0x556920131ca0 2024-09-19T05:12:28.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.390+0000 7f9319e75740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556920089250 con 0x556920126ac0 2024-09-19T05:12:28.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.391+0000 7f9315400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556920126ac0 0x556920126ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:28.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.391+0000 7f9315400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556920126ac0 0x556920126ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:42790/0 (socket says 172.21.15.43:42790) 2024-09-19T05:12:28.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.391+0000 7f9315400640 1 -- 172.21.15.43:0/2193842899 learned_addr learned my addr 172.21.15.43:0/2193842899 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:28.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.391+0000 7f930fe00640 1 --2- 172.21.15.43:0/2193842899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556920131ca0 0x556920134090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:28.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.391+0000 7f9314a00640 1 --2- 172.21.15.43:0/2193842899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569201273e0 0x556920131760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:28.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.391+0000 7f930fe00640 1 -- 172.21.15.43:0/2193842899 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556920126ac0 msgr2=0x556920126ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:28.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.391+0000 7f930fe00640 1 --2- 172.21.15.43:0/2193842899 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556920126ac0 0x556920126ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.391+0000 7f930fe00640 1 -- 172.21.15.43:0/2193842899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569201273e0 msgr2=0x556920131760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:28.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.391+0000 7f930fe00640 1 --2- 172.21.15.43:0/2193842899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569201273e0 0x556920131760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.391+0000 7f930fe00640 1 -- 172.21.15.43:0/2193842899 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55691fdfe6a0 con 0x556920131ca0 2024-09-19T05:12:28.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.391+0000 7f930fe00640 1 --2- 172.21.15.43:0/2193842899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556920131ca0 0x556920134090 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f93101a00d0 tx=0x7f92fc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30f106d4fc2bae12 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:28.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.392+0000 7f9301c00640 1 -- 172.21.15.43:0/2193842899 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92fc004230 con 0x556920131ca0 2024-09-19T05:12:28.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.392+0000 7f9301c00640 1 -- 172.21.15.43:0/2193842899 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92fc009e10 con 0x556920131ca0 2024-09-19T05:12:28.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.392+0000 7f9301c00640 1 -- 172.21.15.43:0/2193842899 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92fc004d70 con 0x556920131ca0 2024-09-19T05:12:28.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.392+0000 7f930d600640 1 --2- 172.21.15.43:0/4232071056 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f92f80a33b0 0x7f92f80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:28.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.392+0000 7f9319e75740 1 -- 172.21.15.43:0/2193842899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556920131ca0 msgr2=0x556920134090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:28.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.392+0000 7f9319e75740 1 --2- 172.21.15.43:0/2193842899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556920131ca0 0x556920134090 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f93101a00d0 tx=0x7f92fc011810 comp rx=0 tx=0).stop 2024-09-19T05:12:28.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.393+0000 7f930d600640 1 -- 172.21.15.43:0/4232071056 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f93040f0420 con 0x7f9310076720 2024-09-19T05:12:28.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.393+0000 7f9319e75740 1 -- 172.21.15.43:0/2193842899 shutdown_connections 2024-09-19T05:12:28.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.393+0000 7f9319e75740 1 --2- 172.21.15.43:0/2193842899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556920131ca0 0x556920134090 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.393+0000 7f9319e75740 1 --2- 172.21.15.43:0/2193842899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569201273e0 0x556920131760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.393+0000 7f9319e75740 1 --2- 172.21.15.43:0/2193842899 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556920126ac0 0x556920126ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.393+0000 7f9319e75740 1 -- 172.21.15.43:0/2193842899 >> 172.21.15.43:0/2193842899 conn(0x556920123050 msgr2=0x556920123430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:28.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.393+0000 7f9315400640 1 --2- 172.21.15.43:0/4232071056 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f92f80a33b0 0x7f92f80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:28.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.393+0000 7f9319e75740 1 -- 172.21.15.43:0/2193842899 shutdown_connections 2024-09-19T05:12:28.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.393+0000 7f9319e75740 1 -- 172.21.15.43:0/2193842899 wait complete. 2024-09-19T05:12:28.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.393+0000 7f9319e75740 1 -- start start 2024-09-19T05:12:28.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.394+0000 7f9315400640 1 --2- 172.21.15.43:0/4232071056 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f92f80a33b0 0x7f92f80a5870 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9308005fd0 tx=0x7f9308009500 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:28.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.394+0000 7f9319e75740 1 Processor -- start 2024-09-19T05:12:28.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.394+0000 7f9319e75740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556920126ac0 0x556920131550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:28.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.394+0000 7f9319e75740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569201273e0 0x55692012e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:28.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.394+0000 7f9319e75740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556920131ca0 0x55692012ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:28.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.394+0000 7f9319e75740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556920134910 con 0x556920126ac0 2024-09-19T05:12:28.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.394+0000 7f9319e75740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556920047fa0 con 0x5569201273e0 2024-09-19T05:12:28.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.394+0000 7f9319e75740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556920134a90 con 0x556920131ca0 2024-09-19T05:12:28.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.394+0000 7f9319e75740 10 client.0 _ll_register_callbacks cb 0x55691fd9bba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:12:28.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.395+0000 7f9314a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556920126ac0 0x556920131550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:28.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.395+0000 7f930fe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569201273e0 0x55692012e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:28.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.395+0000 7f9314a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556920126ac0 0x556920131550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51136/0 (socket says 172.21.15.43:51136) 2024-09-19T05:12:28.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.395+0000 7f9314a00640 1 -- 172.21.15.43:0/1183263719 learned_addr learned my addr 172.21.15.43:0/1183263719 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:28.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.395+0000 7f9315400640 1 --2- 172.21.15.43:0/1183263719 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556920131ca0 0x55692012ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:28.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.395+0000 7f930fe00640 1 -- 172.21.15.43:0/1183263719 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556920131ca0 msgr2=0x55692012ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:28.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.395+0000 7f930fe00640 1 --2- 172.21.15.43:0/1183263719 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556920131ca0 0x55692012ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.395+0000 7f930fe00640 1 -- 172.21.15.43:0/1183263719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556920126ac0 msgr2=0x556920131550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:28.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.395+0000 7f930fe00640 1 --2- 172.21.15.43:0/1183263719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556920126ac0 0x556920131550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.395+0000 7f930fe00640 1 -- 172.21.15.43:0/1183263719 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55692012f540 con 0x5569201273e0 2024-09-19T05:12:28.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.395+0000 7f9314a00640 1 --2- 172.21.15.43:0/1183263719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556920126ac0 0x556920131550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:12:28.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.395+0000 7f930fe00640 1 --2- 172.21.15.43:0/1183263719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569201273e0 0x55692012e800 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f92fc011d10 tx=0x7f92fc003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7daeb9b6bdc1b5a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:28.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.395+0000 7f92f6000640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:12:28.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.396+0000 7f92f6000640 1 -- 172.21.15.43:0/1183263719 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92fc027020 con 0x5569201273e0 2024-09-19T05:12:28.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.396+0000 7f92f6000640 1 -- 172.21.15.43:0/1183263719 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f92fc003c10 con 0x5569201273e0 2024-09-19T05:12:28.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.396+0000 7f9319e75740 1 -- 172.21.15.43:0/1183263719 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92fc009e10 con 0x5569201273e0 2024-09-19T05:12:28.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.396+0000 7f92f6000640 1 -- 172.21.15.43:0/1183263719 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f92fc032050 con 0x5569201273e0 2024-09-19T05:12:28.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.396+0000 7f9319e75740 10 client.27074 fetch_fsmap learned FSMap version 29 2024-09-19T05:12:28.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.396+0000 7f9319e75740 1 -- 172.21.15.43:0/1183263719 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556920140790 con 0x5569201273e0 2024-09-19T05:12:28.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.396+0000 7f92f6000640 1 -- 172.21.15.43:0/1183263719 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92fc009e10 con 0x5569201273e0 2024-09-19T05:12:28.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.397+0000 7f92f6000640 1 -- 172.21.15.43:0/1183263719 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f92fc018070 con 0x5569201273e0 2024-09-19T05:12:28.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.397+0000 7f9319e75740 10 client.27074 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:12:28.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.397+0000 7f9319e75740 10 client.27074 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:12:28.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.397+0000 7f9319e75740 20 client.27074 populate_metadata read hostname 'smithi043' 2024-09-19T05:12:28.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.397+0000 7f9319e75740 1 --2- 172.21.15.43:0/1183263719 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x556920141c50 0x556920144040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:28.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.397+0000 7f9319e75740 4 client.27074 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:12:28.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.397+0000 7f9319e75740 1 -- 172.21.15.43:0/1183263719 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55692009a800 con 0x556920141c50 2024-09-19T05:12:28.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.397+0000 7f9314a00640 1 --2- 172.21.15.43:0/1183263719 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x556920141c50 0x556920144040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:28.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:28 smithi151 ceph-mon[35425]: pgmap v241: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-19T05:12:28.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.398+0000 7f9314a00640 1 --2- 172.21.15.43:0/1183263719 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x556920141c50 0x556920144040 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=321d07e0d76d20be server_cookie=1780111253c1c09b in_seq=0 out_seq=0 2024-09-19T05:12:28.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.398+0000 7f92f6000640 10 client.27074 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:12:28.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.398+0000 7f92f6000640 1 -- 172.21.15.43:0/1183263719 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55692009a800 con 0x556920141c50 2024-09-19T05:12:28.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.398+0000 7f92f6000640 10 client.27074 handle_command_reply: tid=0 2024-09-19T05:12:28.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.398+0000 7f9319e75740 1 client.27074 shutdown 2024-09-19T05:12:28.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.399+0000 7f9319e75740 1 -- 172.21.15.43:0/1183263719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569201273e0 msgr2=0x55692012e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:28.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.399+0000 7f9319e75740 1 --2- 172.21.15.43:0/1183263719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569201273e0 0x55692012e800 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f92fc011d10 tx=0x7f92fc003820 comp rx=0 tx=0).stop 2024-09-19T05:12:28.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.399+0000 7f9319e75740 1 -- 172.21.15.43:0/1183263719 shutdown_connections 2024-09-19T05:12:28.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.399+0000 7f9319e75740 1 --2- 172.21.15.43:0/1183263719 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x556920141c50 0x556920144040 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.399+0000 7f9319e75740 1 --2- 172.21.15.43:0/1183263719 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556920131ca0 0x55692012ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.399+0000 7f9319e75740 1 --2- 172.21.15.43:0/1183263719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569201273e0 0x55692012e800 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.399+0000 7f92f6000640 0 client.27074 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:12:28.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.399+0000 7f9319e75740 1 --2- 172.21.15.43:0/1183263719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556920126ac0 0x556920131550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.399+0000 7f9319e75740 1 -- 172.21.15.43:0/1183263719 >> 172.21.15.43:0/1183263719 conn(0x556920123050 msgr2=0x556920128d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:28.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.399+0000 7f9319e75740 1 -- 172.21.15.43:0/1183263719 shutdown_connections 2024-09-19T05:12:28.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.399+0000 7f9319e75740 1 -- 172.21.15.43:0/1183263719 wait complete. 2024-09-19T05:12:28.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.399+0000 7f9319e75740 20 client.27074 trim_cache size 0 max 16384 2024-09-19T05:12:28.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.417+0000 7f9319e75740 1 Processor -- start 2024-09-19T05:12:28.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.417+0000 7f9319e75740 1 -- start start 2024-09-19T05:12:28.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.418+0000 7f9319e75740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556920126ac0 0x556920130770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:28.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.418+0000 7f9319e75740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569201273e0 0x5569201498a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:28.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.418+0000 7f9319e75740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556920131ca0 0x556920149de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:28.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.418+0000 7f9319e75740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f93101b3f50 con 0x556920126ac0 2024-09-19T05:12:28.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.418+0000 7f9319e75740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f93101b5a90 con 0x5569201273e0 2024-09-19T05:12:28.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.418+0000 7f9319e75740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f93101b4620 con 0x556920131ca0 2024-09-19T05:12:28.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.418+0000 7f9314a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556920126ac0 0x556920130770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:28.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.418+0000 7f9315400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556920131ca0 0x556920149de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:28.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.418+0000 7f9315400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556920131ca0 0x556920149de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:42806/0 (socket says 172.21.15.43:42806) 2024-09-19T05:12:28.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.418+0000 7f9315400640 1 -- 172.21.15.43:0/269461128 learned_addr learned my addr 172.21.15.43:0/269461128 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:28.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.418+0000 7f930fe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569201273e0 0x5569201498a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:28.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.418+0000 7f9314a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556920126ac0 0x556920130770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51148/0 (socket says 172.21.15.43:51148) 2024-09-19T05:12:28.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.419+0000 7f930fe00640 1 -- 172.21.15.43:0/269461128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556920131ca0 msgr2=0x556920149de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:28.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.419+0000 7f930fe00640 1 --2- 172.21.15.43:0/269461128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556920131ca0 0x556920149de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.419+0000 7f930fe00640 1 -- 172.21.15.43:0/269461128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556920126ac0 msgr2=0x556920130770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:28.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.419+0000 7f930fe00640 1 --2- 172.21.15.43:0/269461128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556920126ac0 0x556920130770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.419+0000 7f930fe00640 1 -- 172.21.15.43:0/269461128 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556920144670 con 0x5569201273e0 2024-09-19T05:12:28.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.419+0000 7f930fe00640 1 --2- 172.21.15.43:0/269461128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569201273e0 0x5569201498a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f92fc011850 tx=0x7f92fc001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12b5352422931d3f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:28.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.419+0000 7f9301200640 1 -- 172.21.15.43:0/269461128 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92fc027030 con 0x5569201273e0 2024-09-19T05:12:28.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.419+0000 7f9301200640 1 -- 172.21.15.43:0/269461128 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92fc009e10 con 0x5569201273e0 2024-09-19T05:12:28.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.419+0000 7f9301200640 1 -- 172.21.15.43:0/269461128 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92fc01e040 con 0x5569201273e0 2024-09-19T05:12:28.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.420+0000 7f9319e75740 1 -- 172.21.15.43:0/269461128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569201273e0 msgr2=0x5569201498a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:28.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.420+0000 7f9319e75740 1 --2- 172.21.15.43:0/269461128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569201273e0 0x5569201498a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f92fc011850 tx=0x7f92fc001e70 comp rx=0 tx=0).stop 2024-09-19T05:12:28.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.420+0000 7f9319e75740 1 -- 172.21.15.43:0/269461128 shutdown_connections 2024-09-19T05:12:28.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.420+0000 7f9319e75740 1 --2- 172.21.15.43:0/269461128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556920131ca0 0x556920149de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.420+0000 7f9319e75740 1 --2- 172.21.15.43:0/269461128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569201273e0 0x5569201498a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.420+0000 7f9319e75740 1 --2- 172.21.15.43:0/269461128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556920126ac0 0x556920130770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.420+0000 7f9319e75740 1 -- 172.21.15.43:0/269461128 >> 172.21.15.43:0/269461128 conn(0x556920123050 msgr2=0x556920130060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:28.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.420+0000 7f9319e75740 1 -- 172.21.15.43:0/269461128 shutdown_connections 2024-09-19T05:12:28.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.420+0000 7f9319e75740 1 -- 172.21.15.43:0/269461128 wait complete. 2024-09-19T05:12:28.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.421+0000 7f9319e75740 1 -- start start 2024-09-19T05:12:28.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.421+0000 7f9319e75740 1 Processor -- start 2024-09-19T05:12:28.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.421+0000 7f9319e75740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556920126ac0 0x556920135af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:28.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.421+0000 7f9319e75740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569201273e0 0x556920136030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:28.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.421+0000 7f9319e75740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556920131ca0 0x556920136570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:28.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.421+0000 7f9319e75740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f93101b53c0 con 0x556920131ca0 2024-09-19T05:12:28.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.421+0000 7f9319e75740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f93101b4cf0 con 0x5569201273e0 2024-09-19T05:12:28.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.421+0000 7f9319e75740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f93101b1f50 con 0x556920126ac0 2024-09-19T05:12:28.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.421+0000 7f9319e75740 10 client.0 _ll_register_callbacks cb 0x7f9308009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:12:28.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.422+0000 7f930fe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569201273e0 0x556920136030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:28.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.422+0000 7f9315400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556920131ca0 0x556920136570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:28.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.422+0000 7f930fe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569201273e0 0x556920136030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:40676/0 (socket says 172.21.15.43:40676) 2024-09-19T05:12:28.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.422+0000 7f930fe00640 1 -- 172.21.15.43:0/1472898414 learned_addr learned my addr 172.21.15.43:0/1472898414 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:28.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.422+0000 7f9314a00640 1 --2- 172.21.15.43:0/1472898414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556920126ac0 0x556920135af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:28.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.422+0000 7f930fe00640 1 -- 172.21.15.43:0/1472898414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556920126ac0 msgr2=0x556920135af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:28.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.422+0000 7f930fe00640 1 --2- 172.21.15.43:0/1472898414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556920126ac0 0x556920135af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.422+0000 7f930fe00640 1 -- 172.21.15.43:0/1472898414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556920131ca0 msgr2=0x556920136570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:28.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.422+0000 7f930fe00640 1 --2- 172.21.15.43:0/1472898414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556920131ca0 0x556920136570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.422+0000 7f930fe00640 1 -- 172.21.15.43:0/1472898414 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569201ab920 con 0x5569201273e0 2024-09-19T05:12:28.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.422+0000 7f9315400640 1 --2- 172.21.15.43:0/1472898414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556920131ca0 0x556920136570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:12:28.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.422+0000 7f9314a00640 1 --2- 172.21.15.43:0/1472898414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556920126ac0 0x556920135af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:12:28.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.422+0000 7f930fe00640 1 --2- 172.21.15.43:0/1472898414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569201273e0 0x556920136030 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f92fc000c00 tx=0x7f92fc0193f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edc63d952cd4d21a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:28.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.422+0000 7f92f6000640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:12:28.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.423+0000 7f92f6000640 1 -- 172.21.15.43:0/1472898414 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92fc03a030 con 0x5569201273e0 2024-09-19T05:12:28.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.423+0000 7f9319e75740 1 -- 172.21.15.43:0/1472898414 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92fc009e10 con 0x5569201273e0 2024-09-19T05:12:28.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.423+0000 7f92f6000640 1 -- 172.21.15.43:0/1472898414 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f92fc003800 con 0x5569201273e0 2024-09-19T05:12:28.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.423+0000 7f92f6000640 1 -- 172.21.15.43:0/1472898414 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f92fc027040 con 0x5569201273e0 2024-09-19T05:12:28.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.423+0000 7f9319e75740 10 client.27080 fetch_fsmap learned FSMap version 29 2024-09-19T05:12:28.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.423+0000 7f9319e75740 1 -- 172.21.15.43:0/1472898414 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5569201acc40 con 0x5569201273e0 2024-09-19T05:12:28.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.423+0000 7f92f6000640 1 -- 172.21.15.43:0/1472898414 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92fc009e10 con 0x5569201273e0 2024-09-19T05:12:28.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.424+0000 7f92f6000640 1 -- 172.21.15.43:0/1472898414 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f92fc018070 con 0x5569201273e0 2024-09-19T05:12:28.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.424+0000 7f9319e75740 10 client.27080 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:12:28.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.424+0000 7f9319e75740 10 client.27080 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:12:28.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.424+0000 7f9319e75740 20 client.27080 populate_metadata read hostname 'smithi043' 2024-09-19T05:12:28.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.424+0000 7f9319e75740 1 --2- 172.21.15.43:0/1472898414 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5569201ad280 0x5569201af6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:28.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.424+0000 7f9319e75740 4 client.27080 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T05:12:28.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.424+0000 7f9319e75740 1 -- 172.21.15.43:0/1472898414 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55692012f0d0 con 0x5569201ad280 2024-09-19T05:12:28.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.424+0000 7f9314a00640 1 --2- 172.21.15.43:0/1472898414 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5569201ad280 0x5569201af6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:28.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.425+0000 7f9314a00640 1 --2- 172.21.15.43:0/1472898414 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5569201ad280 0x5569201af6a0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5943f15f1228c49f server_cookie=9d5e02b89893e171 in_seq=0 out_seq=0 2024-09-19T05:12:28.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.425+0000 7f92f6000640 10 client.27080 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:12:28.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.426+0000 7f92f6000640 1 -- 172.21.15.43:0/1472898414 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55692012f0d0 con 0x5569201ad280 2024-09-19T05:12:28.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.426+0000 7f92f6000640 10 client.27080 handle_command_reply: tid=0 2024-09-19T05:12:28.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.426+0000 7f9319e75740 1 client.27080 shutdown 2024-09-19T05:12:28.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.426+0000 7f9319e75740 1 -- 172.21.15.43:0/1472898414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569201273e0 msgr2=0x556920136030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:28.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.426+0000 7f9319e75740 1 --2- 172.21.15.43:0/1472898414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569201273e0 0x556920136030 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f92fc000c00 tx=0x7f92fc0193f0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.426+0000 7f9319e75740 1 -- 172.21.15.43:0/1472898414 shutdown_connections 2024-09-19T05:12:28.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.426+0000 7f9319e75740 1 --2- 172.21.15.43:0/1472898414 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5569201ad280 0x5569201af6a0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.426+0000 7f9319e75740 1 --2- 172.21.15.43:0/1472898414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556920131ca0 0x556920136570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.426+0000 7f9319e75740 1 --2- 172.21.15.43:0/1472898414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569201273e0 0x556920136030 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.426+0000 7f9319e75740 1 --2- 172.21.15.43:0/1472898414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556920126ac0 0x556920135af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.426+0000 7f9319e75740 1 -- 172.21.15.43:0/1472898414 >> 172.21.15.43:0/1472898414 conn(0x556920123050 msgr2=0x5569201221c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:28.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.426+0000 7f92f6000640 0 client.27080 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:12:28.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.426+0000 7f9319e75740 1 -- 172.21.15.43:0/1472898414 shutdown_connections 2024-09-19T05:12:28.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.426+0000 7f9319e75740 1 -- 172.21.15.43:0/1472898414 wait complete. 2024-09-19T05:12:28.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.427+0000 7f9319e75740 20 client.27080 trim_cache size 0 max 16384 2024-09-19T05:12:28.433 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:12:28.433 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-09-19T05:12:28.433 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "250e5046-80b5-411d-bbb3-4b1eee19fa5d", 2024-09-19T05:12:28.433 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-09-19T05:12:28.433 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:12:28.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.427+0000 7f9317400640 1 -- 172.21.15.43:0/4232071056 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f92f80a33b0 msgr2=0x7f92f80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:28.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.427+0000 7f9317400640 1 --2- 172.21.15.43:0/4232071056 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f92f80a33b0 0x7f92f80a5870 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9308005fd0 tx=0x7f9308009500 comp rx=0 tx=0).stop 2024-09-19T05:12:28.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.427+0000 7f9317400640 1 -- 172.21.15.43:0/4232071056 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9310076720 msgr2=0x7f93101a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:28.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.427+0000 7f9317400640 1 --2- 172.21.15.43:0/4232071056 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9310076720 0x7f93101a4720 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9304004770 tx=0x7f9304003c10 comp rx=0 tx=0).stop 2024-09-19T05:12:28.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.427+0000 7f9317400640 1 -- 172.21.15.43:0/4232071056 shutdown_connections 2024-09-19T05:12:28.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.427+0000 7f9317400640 1 --2- 172.21.15.43:0/4232071056 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f92f80a33b0 0x7f92f80a5870 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.427+0000 7f9317400640 1 --2- 172.21.15.43:0/4232071056 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9310112a30 0x7f931019e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.427+0000 7f9317400640 1 --2- 172.21.15.43:0/4232071056 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9310077040 0x7f93101a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.427+0000 7f9317400640 1 --2- 172.21.15.43:0/4232071056 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9310076720 0x7f93101a4720 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:28.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.427+0000 7f9317400640 1 -- 172.21.15.43:0/4232071056 >> 172.21.15.43:0/4232071056 conn(0x7f93100ffa90 msgr2=0x7f9310101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:28.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.428+0000 7f9317400640 1 -- 172.21.15.43:0/4232071056 shutdown_connections 2024-09-19T05:12:28.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:28.428+0000 7f9317400640 1 -- 172.21.15.43:0/4232071056 wait complete. 2024-09-19T05:12:28.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:28 smithi096 ceph-mon[35085]: pgmap v241: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-19T05:12:29.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:29 smithi043 ceph-mon[30800]: scrub queued for path: / 2024-09-19T05:12:29.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:29 smithi043 ceph-mon[30800]: scrub summary: idle+waiting paths [/] 2024-09-19T05:12:29.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:29 smithi043 ceph-mon[30800]: scrub summary: active paths [/] 2024-09-19T05:12:29.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:29 smithi043 ceph-mon[30800]: scrub summary: idle+waiting paths [/] 2024-09-19T05:12:29.364 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:12:29.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:29 smithi151 ceph-mon[35425]: scrub queued for path: / 2024-09-19T05:12:29.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:29 smithi151 ceph-mon[35425]: scrub summary: idle+waiting paths [/] 2024-09-19T05:12:29.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:29 smithi151 ceph-mon[35425]: scrub summary: active paths [/] 2024-09-19T05:12:29.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:29 smithi151 ceph-mon[35425]: scrub summary: idle+waiting paths [/] 2024-09-19T05:12:29.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:29 smithi096 ceph-mon[35085]: scrub queued for path: / 2024-09-19T05:12:29.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:29 smithi096 ceph-mon[35085]: scrub summary: idle+waiting paths [/] 2024-09-19T05:12:29.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:29 smithi096 ceph-mon[35085]: scrub summary: active paths [/] 2024-09-19T05:12:29.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:29 smithi096 ceph-mon[35085]: scrub summary: idle+waiting paths [/] 2024-09-19T05:12:29.744 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:12:30.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:30 smithi043 ceph-mon[30800]: pgmap v242: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:12:30.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.366+0000 7f14b2200640 1 -- 172.21.15.43:0/1157857159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14ac106f10 msgr2=0x7f14ac10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:30.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.366+0000 7f14b2200640 1 --2- 172.21.15.43:0/1157857159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14ac106f10 0x7f14ac10ea90 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f14a4009ca0 tx=0x7f14a402a8c0 comp rx=0 tx=0).stop 2024-09-19T05:12:30.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.366+0000 7f14b2200640 1 -- 172.21.15.43:0/1157857159 shutdown_connections 2024-09-19T05:12:30.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.366+0000 7f14b2200640 1 --2- 172.21.15.43:0/1157857159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14ac106f10 0x7f14ac10ea90 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:30.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.366+0000 7f14b2200640 1 --2- 172.21.15.43:0/1157857159 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14ac102790 0x7f14ac1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:30.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.366+0000 7f14b2200640 1 --2- 172.21.15.43:0/1157857159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14ac101de0 0x7f14ac1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:30.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.367+0000 7f14b2200640 1 -- 172.21.15.43:0/1157857159 >> 172.21.15.43:0/1157857159 conn(0x7f14ac0fd8c0 msgr2=0x7f14ac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:30.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.367+0000 7f14b2200640 1 -- 172.21.15.43:0/1157857159 shutdown_connections 2024-09-19T05:12:30.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.367+0000 7f14b2200640 1 -- 172.21.15.43:0/1157857159 wait complete. 2024-09-19T05:12:30.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.368+0000 7f14b2200640 1 Processor -- start 2024-09-19T05:12:30.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.368+0000 7f14b2200640 1 -- start start 2024-09-19T05:12:30.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.368+0000 7f14b2200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14ac101de0 0x7f14ac0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:30.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.368+0000 7f14b2200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14ac102790 0x7f14ac074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:30.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.368+0000 7f14b2200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14ac106f10 0x7f14ac074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:30.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.368+0000 7f14b2200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac111fe0 con 0x7f14ac102790 2024-09-19T05:12:30.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.368+0000 7f14b2200640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac111e60 con 0x7f14ac106f10 2024-09-19T05:12:30.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.368+0000 7f14b2200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac112160 con 0x7f14ac101de0 2024-09-19T05:12:30.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.368+0000 7f14a3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14ac102790 0x7f14ac074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:30.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.368+0000 7f14a3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14ac102790 0x7f14ac074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51174/0 (socket says 172.21.15.43:51174) 2024-09-19T05:12:30.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.368+0000 7f14a3e00640 1 -- 172.21.15.43:0/270513356 learned_addr learned my addr 172.21.15.43:0/270513356 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:30.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.369+0000 7f14abe00640 1 --2- 172.21.15.43:0/270513356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14ac106f10 0x7f14ac074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:30.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.369+0000 7f14ab400640 1 --2- 172.21.15.43:0/270513356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14ac101de0 0x7f14ac0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:30.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.369+0000 7f14a3e00640 1 -- 172.21.15.43:0/270513356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14ac101de0 msgr2=0x7f14ac0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:30.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.369+0000 7f14a3e00640 1 --2- 172.21.15.43:0/270513356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14ac101de0 0x7f14ac0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:30.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.369+0000 7f14a3e00640 1 -- 172.21.15.43:0/270513356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14ac106f10 msgr2=0x7f14ac074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:30.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.369+0000 7f14a3e00640 1 --2- 172.21.15.43:0/270513356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14ac106f10 0x7f14ac074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:30.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.369+0000 7f14a3e00640 1 -- 172.21.15.43:0/270513356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14ac1a68e0 con 0x7f14ac102790 2024-09-19T05:12:30.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.369+0000 7f14a3e00640 1 --2- 172.21.15.43:0/270513356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14ac102790 0x7f14ac074a30 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f149400b4a0 tx=0x7f149400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:30.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.370+0000 7f14a8c00640 1 -- 172.21.15.43:0/270513356 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1494014070 con 0x7f14ac102790 2024-09-19T05:12:30.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.370+0000 7f14a8c00640 1 -- 172.21.15.43:0/270513356 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1494010af0 con 0x7f14ac102790 2024-09-19T05:12:30.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.370+0000 7f14a8c00640 1 -- 172.21.15.43:0/270513356 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1494017d10 con 0x7f14ac102790 2024-09-19T05:12:30.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.370+0000 7f14b2200640 1 -- 172.21.15.43:0/270513356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14ac1a6bd0 con 0x7f14ac102790 2024-09-19T05:12:30.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.370+0000 7f14b2200640 1 -- 172.21.15.43:0/270513356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14ac1a70c0 con 0x7f14ac102790 2024-09-19T05:12:30.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.374+0000 7f14b4c87740 1 Processor -- start 2024-09-19T05:12:30.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.374+0000 7f14b4c87740 1 -- start start 2024-09-19T05:12:30.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.374+0000 7f14a8c00640 1 -- 172.21.15.43:0/270513356 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f1494017070 con 0x7f14ac102790 2024-09-19T05:12:30.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.374+0000 7f14b4c87740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559336e61b90 0x559336e61f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:30.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.374+0000 7f14b4c87740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559336e624b0 0x559336e6c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:30.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.374+0000 7f14b4c87740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559336e6cd70 0x559336e6f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:30.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.375+0000 7f14b4c87740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559336dc83e0 con 0x559336e624b0 2024-09-19T05:12:30.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.375+0000 7f14b4c87740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559336d6fd80 con 0x559336e61b90 2024-09-19T05:12:30.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.375+0000 7f14b4c87740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559336dc43a0 con 0x559336e6cd70 2024-09-19T05:12:30.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.375+0000 7f14ab400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559336e624b0 0x559336e6c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:30.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.375+0000 7f14ab400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559336e624b0 0x559336e6c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51186/0 (socket says 172.21.15.43:51186) 2024-09-19T05:12:30.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.375+0000 7f14ab400640 1 -- 172.21.15.43:0/170585521 learned_addr learned my addr 172.21.15.43:0/170585521 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:30.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.375+0000 7f14abe00640 1 --2- 172.21.15.43:0/170585521 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559336e61b90 0x559336e61f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:30.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.375+0000 7f14abe00640 1 -- 172.21.15.43:0/170585521 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559336e6cd70 msgr2=0x559336e6f160 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:12:30.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.375+0000 7f14abe00640 1 --2- 172.21.15.43:0/170585521 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559336e6cd70 0x559336e6f160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:30.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.375+0000 7f14abe00640 1 -- 172.21.15.43:0/170585521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559336e624b0 msgr2=0x559336e6c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:30.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.375+0000 7f14abe00640 1 --2- 172.21.15.43:0/170585521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559336e624b0 0x559336e6c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:30.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.375+0000 7f14abe00640 1 -- 172.21.15.43:0/170585521 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559336d8eae0 con 0x559336e61b90 2024-09-19T05:12:30.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.375+0000 7f14ab400640 1 --2- 172.21.15.43:0/170585521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559336e624b0 0x559336e6c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:12:30.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.376+0000 7f14abe00640 1 --2- 172.21.15.43:0/170585521 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559336e61b90 0x559336e61f70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f14a4005e30 tx=0x7f14a402f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e03cf400827e81cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:30.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.376+0000 7f14a8c00640 1 --2- 172.21.15.43:0/270513356 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f14900a3430 0x7f14900a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:30.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.376+0000 7f149be00640 1 -- 172.21.15.43:0/170585521 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a4003970 con 0x559336e61b90 2024-09-19T05:12:30.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.376+0000 7f14a8c00640 1 -- 172.21.15.43:0/270513356 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f14940ceed0 con 0x7f14ac102790 2024-09-19T05:12:30.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.376+0000 7f149be00640 1 -- 172.21.15.43:0/170585521 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14a4036030 con 0x559336e61b90 2024-09-19T05:12:30.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.376+0000 7f14ab400640 1 --2- 172.21.15.43:0/270513356 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f14900a3430 0x7f14900a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:30.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.376+0000 7f149be00640 1 -- 172.21.15.43:0/170585521 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a4004440 con 0x559336e61b90 2024-09-19T05:12:30.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.377+0000 7f14b4c87740 1 -- 172.21.15.43:0/170585521 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559336e61b90 msgr2=0x559336e61f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:30.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.377+0000 7f14ab400640 1 --2- 172.21.15.43:0/270513356 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f14900a3430 0x7f14900a58f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x559336e5ef80 tx=0x7f149c005f90 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:30.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.377+0000 7f14b4c87740 1 --2- 172.21.15.43:0/170585521 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559336e61b90 0x559336e61f70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f14a4005e30 tx=0x7f14a402f3c0 comp rx=0 tx=0).stop 2024-09-19T05:12:30.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.377+0000 7f14b4c87740 1 -- 172.21.15.43:0/170585521 shutdown_connections 2024-09-19T05:12:30.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.377+0000 7f14b4c87740 1 --2- 172.21.15.43:0/170585521 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559336e6cd70 0x559336e6f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:30.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.377+0000 7f14b4c87740 1 --2- 172.21.15.43:0/170585521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559336e624b0 0x559336e6c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:30.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.377+0000 7f14b4c87740 1 --2- 172.21.15.43:0/170585521 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559336e61b90 0x559336e61f70 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:30.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.377+0000 7f14b4c87740 1 -- 172.21.15.43:0/170585521 >> 172.21.15.43:0/170585521 conn(0x559336e5e120 msgr2=0x559336e5e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:30.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.377+0000 7f14b4c87740 1 -- 172.21.15.43:0/170585521 shutdown_connections 2024-09-19T05:12:30.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.377+0000 7f14b4c87740 1 -- 172.21.15.43:0/170585521 wait complete. 2024-09-19T05:12:30.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.378+0000 7f14b4c87740 1 -- start start 2024-09-19T05:12:30.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.378+0000 7f14b4c87740 1 Processor -- start 2024-09-19T05:12:30.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.379+0000 7f14b4c87740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559336e61b90 0x559336e69d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:30.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.379+0000 7f14b4c87740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559336e624b0 0x559336e6a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:30.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.379+0000 7f14b4c87740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559336e6cd70 0x559336e6a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:30.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.379+0000 7f14b4c87740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559336e6f860 con 0x559336e624b0 2024-09-19T05:12:30.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.379+0000 7f14b4c87740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559336d830f0 con 0x559336e61b90 2024-09-19T05:12:30.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.379+0000 7f14b4c87740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559336e6f9e0 con 0x559336e6cd70 2024-09-19T05:12:30.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.379+0000 7f14b4c87740 10 client.0 _ll_register_callbacks cb 0x559336d749e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:12:30.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.379+0000 7f14ab400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559336e61b90 0x559336e69d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:30.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.379+0000 7f14a3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559336e624b0 0x559336e6a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:30.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.379+0000 7f14a3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559336e624b0 0x559336e6a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51190/0 (socket says 172.21.15.43:51190) 2024-09-19T05:12:30.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.379+0000 7f14a3e00640 1 -- 172.21.15.43:0/500188342 learned_addr learned my addr 172.21.15.43:0/500188342 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:30.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.379+0000 7f14ab400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559336e61b90 0x559336e69d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:40698/0 (socket says 172.21.15.43:40698) 2024-09-19T05:12:30.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.380+0000 7f14a3e00640 1 -- 172.21.15.43:0/500188342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559336e6cd70 msgr2=0x559336e6a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:30.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.380+0000 7f14a3e00640 1 --2- 172.21.15.43:0/500188342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559336e6cd70 0x559336e6a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:30.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.380+0000 7f14a3e00640 1 -- 172.21.15.43:0/500188342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559336e61b90 msgr2=0x559336e69d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:30.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.380+0000 7f14a3e00640 1 --2- 172.21.15.43:0/500188342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559336e61b90 0x559336e69d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:30.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.380+0000 7f14a3e00640 1 -- 172.21.15.43:0/500188342 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559336b393c0 con 0x559336e624b0 2024-09-19T05:12:30.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.380+0000 7f14a3e00640 1 --2- 172.21.15.43:0/500188342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559336e624b0 0x559336e6a2a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f14940ccc10 tx=0x7f149409ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d48ea01a6d4c566 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:30.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.380+0000 7f1498c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:12:30.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.380+0000 7f1498c00640 1 -- 172.21.15.43:0/500188342 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149401e070 con 0x559336e624b0 2024-09-19T05:12:30.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.380+0000 7f1498c00640 1 -- 172.21.15.43:0/500188342 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f14940979d0 con 0x559336e624b0 2024-09-19T05:12:30.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.380+0000 7f14b4c87740 1 -- 172.21.15.43:0/500188342 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14a4036030 con 0x559336e624b0 2024-09-19T05:12:30.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.380+0000 7f1498c00640 1 -- 172.21.15.43:0/500188342 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f14940b4270 con 0x559336e624b0 2024-09-19T05:12:30.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.380+0000 7f14b4c87740 10 client.17070 fetch_fsmap learned FSMap version 29 2024-09-19T05:12:30.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.380+0000 7f14b4c87740 1 -- 172.21.15.43:0/500188342 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559336e7b660 con 0x559336e624b0 2024-09-19T05:12:30.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.381+0000 7f1498c00640 1 -- 172.21.15.43:0/500188342 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14a4036030 con 0x559336e624b0 2024-09-19T05:12:30.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.381+0000 7f1498c00640 1 -- 172.21.15.43:0/500188342 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f14940bb170 con 0x559336e624b0 2024-09-19T05:12:30.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.381+0000 7f14b4c87740 10 client.17070 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:12:30.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.381+0000 7f14b4c87740 10 client.17070 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:12:30.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.381+0000 7f14b4c87740 20 client.17070 populate_metadata read hostname 'smithi043' 2024-09-19T05:12:30.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.381+0000 7f14b4c87740 1 --2- 172.21.15.43:0/500188342 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x559336e7cb20 0x559336e7ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:30.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.381+0000 7f14b4c87740 4 client.17070 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:12:30.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.381+0000 7f14b4c87740 1 -- 172.21.15.43:0/500188342 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559336dd5950 con 0x559336e7cb20 2024-09-19T05:12:30.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.382+0000 7f14ab400640 1 --2- 172.21.15.43:0/500188342 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x559336e7cb20 0x559336e7ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:30.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.382+0000 7f14ab400640 1 --2- 172.21.15.43:0/500188342 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x559336e7cb20 0x559336e7ef10 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e618e6b5414e073 server_cookie=6a2ecabca346c526 in_seq=0 out_seq=0 2024-09-19T05:12:30.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.382+0000 7f1498c00640 10 client.17070 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:12:30.456 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:30 smithi151 ceph-mon[35425]: pgmap v242: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:12:30.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.383+0000 7f1498c00640 1 -- 172.21.15.43:0/500188342 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559336dd5950 con 0x559336e7cb20 2024-09-19T05:12:30.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.383+0000 7f1498c00640 10 client.17070 handle_command_reply: tid=0 2024-09-19T05:12:30.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.383+0000 7f14b4c87740 1 client.17070 shutdown 2024-09-19T05:12:30.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.383+0000 7f14b4c87740 1 -- 172.21.15.43:0/500188342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559336e624b0 msgr2=0x559336e6a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:30.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.383+0000 7f14b4c87740 1 --2- 172.21.15.43:0/500188342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559336e624b0 0x559336e6a2a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f14940ccc10 tx=0x7f149409ad40 comp rx=0 tx=0).stop 2024-09-19T05:12:30.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.383+0000 7f14b4c87740 1 -- 172.21.15.43:0/500188342 shutdown_connections 2024-09-19T05:12:30.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.383+0000 7f14b4c87740 1 --2- 172.21.15.43:0/500188342 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x559336e7cb20 0x559336e7ef10 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:30.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.383+0000 7f14b4c87740 1 --2- 172.21.15.43:0/500188342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559336e6cd70 0x559336e6a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:30.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.383+0000 7f14b4c87740 1 --2- 172.21.15.43:0/500188342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559336e624b0 0x559336e6a2a0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:30.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.383+0000 7f14b4c87740 1 --2- 172.21.15.43:0/500188342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559336e61b90 0x559336e69d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:30.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.383+0000 7f14b4c87740 1 -- 172.21.15.43:0/500188342 >> 172.21.15.43:0/500188342 conn(0x559336e5e120 msgr2=0x559336e63e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:30.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.383+0000 7f1498c00640 0 client.17070 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:12:30.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.383+0000 7f14b4c87740 1 -- 172.21.15.43:0/500188342 shutdown_connections 2024-09-19T05:12:30.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.383+0000 7f14b4c87740 1 -- 172.21.15.43:0/500188342 wait complete. 2024-09-19T05:12:30.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.384+0000 7f14b4c87740 20 client.17070 trim_cache size 0 max 16384 2024-09-19T05:12:30.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.401+0000 7f14b4c87740 1 Processor -- start 2024-09-19T05:12:30.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.402+0000 7f14b4c87740 1 -- start start 2024-09-19T05:12:30.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.402+0000 7f14b4c87740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559336e61b90 0x559336e5e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:30.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.402+0000 7f14b4c87740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559336e624b0 0x559336e6a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:30.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.402+0000 7f14b4c87740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559336e6cd70 0x559336e847c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:30.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.402+0000 7f14b4c87740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac1afa40 con 0x559336e61b90 2024-09-19T05:12:30.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.402+0000 7f14b4c87740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac1b1580 con 0x559336e6cd70 2024-09-19T05:12:30.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.402+0000 7f14b4c87740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac1b0110 con 0x559336e624b0 2024-09-19T05:12:30.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.402+0000 7f14ab400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559336e61b90 0x559336e5e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:30.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.402+0000 7f14ab400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559336e61b90 0x559336e5e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51194/0 (socket says 172.21.15.43:51194) 2024-09-19T05:12:30.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.402+0000 7f14ab400640 1 -- 172.21.15.43:0/1581911545 learned_addr learned my addr 172.21.15.43:0/1581911545 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:30.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.402+0000 7f14abe00640 1 --2- 172.21.15.43:0/1581911545 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559336e6cd70 0x559336e847c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:30.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.402+0000 7f14a3e00640 1 --2- 172.21.15.43:0/1581911545 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559336e624b0 0x559336e6a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:30.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.403+0000 7f14ab400640 1 -- 172.21.15.43:0/1581911545 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559336e624b0 msgr2=0x559336e6a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:30.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.403+0000 7f14ab400640 1 --2- 172.21.15.43:0/1581911545 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559336e624b0 0x559336e6a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:30.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.403+0000 7f14ab400640 1 -- 172.21.15.43:0/1581911545 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559336e6cd70 msgr2=0x559336e847c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:30.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.403+0000 7f14ab400640 1 --2- 172.21.15.43:0/1581911545 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559336e6cd70 0x559336e847c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:30.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.403+0000 7f14ab400640 1 -- 172.21.15.43:0/1581911545 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559336e68d10 con 0x559336e61b90 2024-09-19T05:12:30.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.403+0000 7f14ab400640 1 --2- 172.21.15.43:0/1581911545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559336e61b90 0x559336e5e500 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f149c005b90 tx=0x7f149c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c5829fc6efa0fde server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:30.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.403+0000 7f149b400640 1 -- 172.21.15.43:0/1581911545 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149c00fe50 con 0x559336e61b90 2024-09-19T05:12:30.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.403+0000 7f149b400640 1 -- 172.21.15.43:0/1581911545 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f149c02a8f0 con 0x559336e61b90 2024-09-19T05:12:30.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.403+0000 7f149b400640 1 -- 172.21.15.43:0/1581911545 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149c01d880 con 0x559336e61b90 2024-09-19T05:12:30.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.404+0000 7f14b4c87740 1 -- 172.21.15.43:0/1581911545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559336e61b90 msgr2=0x559336e5e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:30.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.404+0000 7f14b4c87740 1 --2- 172.21.15.43:0/1581911545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559336e61b90 0x559336e5e500 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f149c005b90 tx=0x7f149c019000 comp rx=0 tx=0).stop 2024-09-19T05:12:30.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.404+0000 7f14b4c87740 1 -- 172.21.15.43:0/1581911545 shutdown_connections 2024-09-19T05:12:30.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.404+0000 7f14b4c87740 1 --2- 172.21.15.43:0/1581911545 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559336e6cd70 0x559336e847c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:30.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.404+0000 7f14b4c87740 1 --2- 172.21.15.43:0/1581911545 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559336e624b0 0x559336e6a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:30.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.404+0000 7f14b4c87740 1 --2- 172.21.15.43:0/1581911545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559336e61b90 0x559336e5e500 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:30.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.404+0000 7f14b4c87740 1 -- 172.21.15.43:0/1581911545 >> 172.21.15.43:0/1581911545 conn(0x559336e5e120 msgr2=0x559336e69d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:30.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.404+0000 7f14b4c87740 1 -- 172.21.15.43:0/1581911545 shutdown_connections 2024-09-19T05:12:30.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.404+0000 7f14b4c87740 1 -- 172.21.15.43:0/1581911545 wait complete. 2024-09-19T05:12:30.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.405+0000 7f14b4c87740 1 -- start start 2024-09-19T05:12:30.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.405+0000 7f14b4c87740 1 Processor -- start 2024-09-19T05:12:30.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.405+0000 7f14b4c87740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559336e61b90 0x559336e752e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:30.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.406+0000 7f14b4c87740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559336e624b0 0x559336e70610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:30.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.406+0000 7f14b4c87740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559336e6cd70 0x559336e70b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:30.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.406+0000 7f14b4c87740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac1b0eb0 con 0x559336e624b0 2024-09-19T05:12:30.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.406+0000 7f14b4c87740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac1b07e0 con 0x559336e61b90 2024-09-19T05:12:30.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.406+0000 7f14b4c87740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac1ada40 con 0x559336e6cd70 2024-09-19T05:12:30.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.406+0000 7f14b4c87740 10 client.0 _ll_register_callbacks cb 0x7f149409b130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:12:30.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.406+0000 7f14a3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559336e624b0 0x559336e70610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:30.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.406+0000 7f14a3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559336e624b0 0x559336e70610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51196/0 (socket says 172.21.15.43:51196) 2024-09-19T05:12:30.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.406+0000 7f14ab400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559336e61b90 0x559336e752e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:30.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.406+0000 7f14a3e00640 1 -- 172.21.15.43:0/2812533516 learned_addr learned my addr 172.21.15.43:0/2812533516 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:12:30.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.406+0000 7f14a3e00640 1 -- 172.21.15.43:0/2812533516 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559336e6cd70 msgr2=0x559336e70b50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:12:30.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.406+0000 7f14a3e00640 1 --2- 172.21.15.43:0/2812533516 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559336e6cd70 0x559336e70b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:30.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.406+0000 7f14a3e00640 1 -- 172.21.15.43:0/2812533516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559336e61b90 msgr2=0x559336e752e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:12:30.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.406+0000 7f14a3e00640 1 --2- 172.21.15.43:0/2812533516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559336e61b90 0x559336e752e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:30.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.406+0000 7f14a3e00640 1 -- 172.21.15.43:0/2812533516 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559336e71350 con 0x559336e624b0 2024-09-19T05:12:30.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.407+0000 7f14a3e00640 1 --2- 172.21.15.43:0/2812533516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559336e624b0 0x559336e70610 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f1494097d30 tx=0x7f1494017730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dc4941d92a045fd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:12:30.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.407+0000 7f1498c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:12:30.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.407+0000 7f1498c00640 1 -- 172.21.15.43:0/2812533516 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149401e070 con 0x559336e624b0 2024-09-19T05:12:30.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.407+0000 7f1498c00640 1 -- 172.21.15.43:0/2812533516 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f14940bb0b0 con 0x559336e624b0 2024-09-19T05:12:30.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.407+0000 7f14b4c87740 1 -- 172.21.15.43:0/2812533516 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f149c02a8f0 con 0x559336e624b0 2024-09-19T05:12:30.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.407+0000 7f1498c00640 1 -- 172.21.15.43:0/2812533516 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f14940bb2f0 con 0x559336e624b0 2024-09-19T05:12:30.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.407+0000 7f14b4c87740 10 client.17076 fetch_fsmap learned FSMap version 29 2024-09-19T05:12:30.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.407+0000 7f14b4c87740 1 -- 172.21.15.43:0/2812533516 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559336ee7680 con 0x559336e624b0 2024-09-19T05:12:30.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.407+0000 7f1498c00640 1 -- 172.21.15.43:0/2812533516 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f149c02a8f0 con 0x559336e624b0 2024-09-19T05:12:30.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.407+0000 7f1498c00640 1 -- 172.21.15.43:0/2812533516 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f14940970b0 con 0x559336e624b0 2024-09-19T05:12:30.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.407+0000 7f14b4c87740 10 client.17076 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:12:30.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.407+0000 7f14b4c87740 10 client.17076 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:12:30.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.407+0000 7f14b4c87740 20 client.17076 populate_metadata read hostname 'smithi043' 2024-09-19T05:12:30.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.408+0000 7f14b4c87740 1 --2- 172.21.15.43:0/2812533516 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x559336ee7c70 0x559336eea090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:12:30.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.408+0000 7f14b4c87740 4 client.17076 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:12:30.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.408+0000 7f14b4c87740 1 -- 172.21.15.43:0/2812533516 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559336dcfe60 con 0x559336ee7c70 2024-09-19T05:12:30.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.408+0000 7f14ab400640 1 --2- 172.21.15.43:0/2812533516 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x559336ee7c70 0x559336eea090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:12:30.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.408+0000 7f14ab400640 1 --2- 172.21.15.43:0/2812533516 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x559336ee7c70 0x559336eea090 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30570bb804294012 server_cookie=10a6a63f173e9df1 in_seq=0 out_seq=0 2024-09-19T05:12:30.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.408+0000 7f1498c00640 10 client.17076 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:12:30.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.409+0000 7f1498c00640 1 -- 172.21.15.43:0/2812533516 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559336dcfe60 con 0x559336ee7c70 2024-09-19T05:12:30.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.409+0000 7f1498c00640 10 client.17076 handle_command_reply: tid=0 2024-09-19T05:12:30.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.409+0000 7f14b4c87740 1 client.17076 shutdown 2024-09-19T05:12:30.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.409+0000 7f14b4c87740 1 -- 172.21.15.43:0/2812533516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559336e624b0 msgr2=0x559336e70610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:30.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.409+0000 7f14b4c87740 1 --2- 172.21.15.43:0/2812533516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559336e624b0 0x559336e70610 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f1494097d30 tx=0x7f1494017730 comp rx=0 tx=0).stop 2024-09-19T05:12:30.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.409+0000 7f14b4c87740 1 -- 172.21.15.43:0/2812533516 shutdown_connections 2024-09-19T05:12:30.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.409+0000 7f14b4c87740 1 --2- 172.21.15.43:0/2812533516 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x559336ee7c70 0x559336eea090 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:30.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.409+0000 7f14b4c87740 1 --2- 172.21.15.43:0/2812533516 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559336e6cd70 0x559336e70b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:30.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.409+0000 7f14b4c87740 1 --2- 172.21.15.43:0/2812533516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559336e624b0 0x559336e70610 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:30.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.409+0000 7f14b4c87740 1 --2- 172.21.15.43:0/2812533516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559336e61b90 0x559336e752e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:30.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.409+0000 7f14b4c87740 1 -- 172.21.15.43:0/2812533516 >> 172.21.15.43:0/2812533516 conn(0x559336e5e120 msgr2=0x559336e5d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:30.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.409+0000 7f1498c00640 0 client.17076 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:12:30.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.409+0000 7f14b4c87740 1 -- 172.21.15.43:0/2812533516 shutdown_connections 2024-09-19T05:12:30.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.409+0000 7f14b4c87740 1 -- 172.21.15.43:0/2812533516 wait complete. 2024-09-19T05:12:30.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.409+0000 7f14b4c87740 20 client.17076 trim_cache size 0 max 16384 2024-09-19T05:12:30.469 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:12:30.469 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T05:12:30.469 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-09-19T05:12:30.469 INFO:teuthology.orchestra.run.smithi043.stdout: "250e5046-80b5-411d-bbb3-4b1eee19fa5d": { 2024-09-19T05:12:30.469 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-09-19T05:12:30.469 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "250e5046-80b5-411d-bbb3-4b1eee19fa5d", 2024-09-19T05:12:30.469 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-09-19T05:12:30.470 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:12:30.470 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:12:30.470 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:12:30.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.410+0000 7f14b2200640 1 -- 172.21.15.43:0/270513356 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f14900a3430 msgr2=0x7f14900a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:30.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.410+0000 7f14b2200640 1 --2- 172.21.15.43:0/270513356 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f14900a3430 0x7f14900a58f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x559336e5ef80 tx=0x7f149c005f90 comp rx=0 tx=0).stop 2024-09-19T05:12:30.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.410+0000 7f14b2200640 1 -- 172.21.15.43:0/270513356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14ac102790 msgr2=0x7f14ac074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:12:30.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.410+0000 7f14b2200640 1 --2- 172.21.15.43:0/270513356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14ac102790 0x7f14ac074a30 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f149400b4a0 tx=0x7f149400b960 comp rx=0 tx=0).stop 2024-09-19T05:12:30.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.410+0000 7f14b2200640 1 -- 172.21.15.43:0/270513356 shutdown_connections 2024-09-19T05:12:30.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.410+0000 7f14b2200640 1 --2- 172.21.15.43:0/270513356 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f14900a3430 0x7f14900a58f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:30.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.410+0000 7f14b2200640 1 --2- 172.21.15.43:0/270513356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14ac106f10 0x7f14ac074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:30.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.410+0000 7f14b2200640 1 --2- 172.21.15.43:0/270513356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14ac102790 0x7f14ac074a30 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:30.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.410+0000 7f14b2200640 1 --2- 172.21.15.43:0/270513356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14ac101de0 0x7f14ac0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:12:30.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.410+0000 7f14b2200640 1 -- 172.21.15.43:0/270513356 >> 172.21.15.43:0/270513356 conn(0x7f14ac0fd8c0 msgr2=0x7f14ac0feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:12:30.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.410+0000 7f14b2200640 1 -- 172.21.15.43:0/270513356 shutdown_connections 2024-09-19T05:12:30.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:12:30.410+0000 7f14b2200640 1 -- 172.21.15.43:0/270513356 wait complete. 2024-09-19T05:12:30.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:30 smithi096 ceph-mon[35085]: pgmap v242: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:12:31.411 INFO:tasks.cephfs.filesystem:scrub status for tag:250e5046-80b5-411d-bbb3-4b1eee19fa5d - {'path': '/', 'tag': '250e5046-80b5-411d-bbb3-4b1eee19fa5d', 'options': 'recursive,force'} 2024-09-19T05:12:32.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:32 smithi151 ceph-mon[35425]: pgmap v243: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:12:32.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:32 smithi096 ceph-mon[35085]: pgmap v243: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:12:32.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:32 smithi043 ceph-mon[30800]: pgmap v243: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:12:33.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:33 smithi151 ceph-mon[35425]: scrub summary: idle 2024-09-19T05:12:33.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:33 smithi096 ceph-mon[35085]: scrub summary: idle 2024-09-19T05:12:33.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:33 smithi043 ceph-mon[30800]: scrub summary: idle 2024-09-19T05:12:34.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:34 smithi151 ceph-mon[35425]: pgmap v244: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:12:34.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:34 smithi096 ceph-mon[35085]: pgmap v244: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:12:34.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:34 smithi043 ceph-mon[30800]: pgmap v244: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:12:36.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:36 smithi151 ceph-mon[35425]: pgmap v245: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:12:36.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:36 smithi096 ceph-mon[35085]: pgmap v245: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:12:36.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:36 smithi043 ceph-mon[30800]: pgmap v245: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:12:38.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:38 smithi151 ceph-mon[35425]: pgmap v246: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:12:38.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:38 smithi096 ceph-mon[35085]: pgmap v246: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:12:38.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:38 smithi043 ceph-mon[30800]: pgmap v246: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:12:40.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:40 smithi151 ceph-mon[35425]: pgmap v247: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:12:40.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:40 smithi096 ceph-mon[35085]: pgmap v247: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:12:40.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:40 smithi043 ceph-mon[30800]: pgmap v247: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:12:42.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:42 smithi151 ceph-mon[35425]: pgmap v248: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:12:42.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:42 smithi096 ceph-mon[35085]: pgmap v248: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:12:42.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:42 smithi043 ceph-mon[30800]: pgmap v248: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:12:44.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:44 smithi151 ceph-mon[35425]: pgmap v249: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:12:44.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:44 smithi096 ceph-mon[35085]: pgmap v249: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:12:44.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:44 smithi043 ceph-mon[30800]: pgmap v249: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:12:46.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:46 smithi151 ceph-mon[35425]: pgmap v250: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:12:46.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:46 smithi096 ceph-mon[35085]: pgmap v250: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:12:46.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:46 smithi043 ceph-mon[30800]: pgmap v250: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:12:48.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:48 smithi151 ceph-mon[35425]: pgmap v251: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:12:48.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:48 smithi096 ceph-mon[35085]: pgmap v251: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:12:48.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:48 smithi043 ceph-mon[30800]: pgmap v251: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:12:50.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:50 smithi151 ceph-mon[35425]: pgmap v252: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:12:50.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:50 smithi096 ceph-mon[35085]: pgmap v252: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:12:50.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:50 smithi043 ceph-mon[30800]: pgmap v252: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:12:52.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:52 smithi151 ceph-mon[35425]: pgmap v253: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:12:52.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:52 smithi096 ceph-mon[35085]: pgmap v253: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:12:52.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:52 smithi043 ceph-mon[30800]: pgmap v253: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:12:54.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:54 smithi151 ceph-mon[35425]: pgmap v254: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:12:54.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:54 smithi096 ceph-mon[35085]: pgmap v254: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:12:54.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:54 smithi043 ceph-mon[30800]: pgmap v254: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:12:56.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:56 smithi096 ceph-mon[35085]: pgmap v255: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:12:56.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:56 smithi043 ceph-mon[30800]: pgmap v255: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:12:56.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:56 smithi151 ceph-mon[35425]: pgmap v255: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:12:58.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:12:58 smithi096 ceph-mon[35085]: pgmap v256: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:12:58.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:12:58 smithi043 ceph-mon[30800]: pgmap v256: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:12:58.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:12:58 smithi151 ceph-mon[35425]: pgmap v256: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:13:00.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:00 smithi096 ceph-mon[35085]: pgmap v257: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:13:00.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:00 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:13:00.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:00 smithi043 ceph-mon[30800]: pgmap v257: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:13:00.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:00 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:13:00.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:00 smithi151 ceph-mon[35425]: pgmap v257: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:13:00.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:00 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:13:01.412 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:13:01.462 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:01 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:13:01.462 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:01 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:13:01.462 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:01 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:13:01.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:01 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:13:01.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:01 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:13:01.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:01 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:13:01.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:01 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:13:01.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:01 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:13:01.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:01 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:13:01.794 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:13:02.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:02 smithi043 ceph-mon[30800]: pgmap v258: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:13:02.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.419+0000 7fa6e6800640 1 -- 172.21.15.43:0/1126781515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6e0103e30 msgr2=0x7fa6e0104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:02.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.419+0000 7fa6e6800640 1 --2- 172.21.15.43:0/1126781515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6e0103e30 0x7fa6e0104210 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4009900 tx=0x7fa6d402a410 comp rx=0 tx=0).stop 2024-09-19T05:13:02.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.419+0000 7fa6e6800640 1 -- 172.21.15.43:0/1126781515 shutdown_connections 2024-09-19T05:13:02.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.419+0000 7fa6e6800640 1 --2- 172.21.15.43:0/1126781515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6e0108f60 0x7fa6e0110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:02.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.419+0000 7fa6e6800640 1 --2- 172.21.15.43:0/1126781515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6e01047e0 0x7fa6e0108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:02.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.419+0000 7fa6e6800640 1 --2- 172.21.15.43:0/1126781515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6e0103e30 0x7fa6e0104210 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:02.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.419+0000 7fa6e6800640 1 -- 172.21.15.43:0/1126781515 >> 172.21.15.43:0/1126781515 conn(0x7fa6e00ff8d0 msgr2=0x7fa6e0101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:02.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.419+0000 7fa6e6800640 1 -- 172.21.15.43:0/1126781515 shutdown_connections 2024-09-19T05:13:02.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.419+0000 7fa6e6800640 1 -- 172.21.15.43:0/1126781515 wait complete. 2024-09-19T05:13:02.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.420+0000 7fa6e6800640 1 Processor -- start 2024-09-19T05:13:02.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.420+0000 7fa6e6800640 1 -- start start 2024-09-19T05:13:02.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.421+0000 7fa6e6800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6e0103e30 0x7fa6e019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:02.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.421+0000 7fa6e6800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6e01047e0 0x7fa6e019a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:02.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.421+0000 7fa6e6800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6e0108f60 0x7fa6e019ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:02.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.421+0000 7fa6e6800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e0113fc0 con 0x7fa6e0108f60 2024-09-19T05:13:02.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.421+0000 7fa6e6800640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e0113e40 con 0x7fa6e01047e0 2024-09-19T05:13:02.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.421+0000 7fa6e6800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e0114140 con 0x7fa6e0103e30 2024-09-19T05:13:02.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.421+0000 7fa6e5400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6e0103e30 0x7fa6e019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:02.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.421+0000 7fa6e4a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6e01047e0 0x7fa6e019a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:02.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.421+0000 7fa6e4a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6e01047e0 0x7fa6e019a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:34740/0 (socket says 172.21.15.43:34740) 2024-09-19T05:13:02.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.421+0000 7fa6e5400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6e0103e30 0x7fa6e019a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:34426/0 (socket says 172.21.15.43:34426) 2024-09-19T05:13:02.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.421+0000 7fa6e5400640 1 -- 172.21.15.43:0/3998858248 learned_addr learned my addr 172.21.15.43:0/3998858248 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:02.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.421+0000 7fa6e5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6e0108f60 0x7fa6e019ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:02.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.422+0000 7fa6e4a00640 1 -- 172.21.15.43:0/3998858248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6e0103e30 msgr2=0x7fa6e019a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:02.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.422+0000 7fa6e4a00640 1 --2- 172.21.15.43:0/3998858248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6e0103e30 0x7fa6e019a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:02.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.422+0000 7fa6e4a00640 1 -- 172.21.15.43:0/3998858248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6e0108f60 msgr2=0x7fa6e019ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:02.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.422+0000 7fa6e4a00640 1 --2- 172.21.15.43:0/3998858248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6e0108f60 0x7fa6e019ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:02.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.422+0000 7fa6e4a00640 1 -- 172.21.15.43:0/3998858248 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6e019f410 con 0x7fa6e01047e0 2024-09-19T05:13:02.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.422+0000 7fa6e4a00640 1 --2- 172.21.15.43:0/3998858248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6e01047e0 0x7fa6e019a9a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc005fd0 tx=0x7fa6cc00dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:02.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.422+0000 7fa6de000640 1 -- 172.21.15.43:0/3998858248 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc009c30 con 0x7fa6e01047e0 2024-09-19T05:13:02.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.422+0000 7fa6de000640 1 -- 172.21.15.43:0/3998858248 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6cc009dd0 con 0x7fa6e01047e0 2024-09-19T05:13:02.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.422+0000 7fa6e6800640 1 -- 172.21.15.43:0/3998858248 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6e019f700 con 0x7fa6e01047e0 2024-09-19T05:13:02.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.422+0000 7fa6de000640 1 -- 172.21.15.43:0/3998858248 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc010460 con 0x7fa6e01047e0 2024-09-19T05:13:02.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.423+0000 7fa6e6800640 1 -- 172.21.15.43:0/3998858248 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6e01a6fe0 con 0x7fa6e01047e0 2024-09-19T05:13:02.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.426+0000 7fa6ea881740 1 Processor -- start 2024-09-19T05:13:02.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.426+0000 7fa6de000640 1 -- 172.21.15.43:0/3998858248 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa6cc003f50 con 0x7fa6e01047e0 2024-09-19T05:13:02.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.426+0000 7fa6ea881740 1 -- start start 2024-09-19T05:13:02.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.426+0000 7fa6ea881740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f603399d50 0x55f60339a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:02.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.426+0000 7fa6ea881740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f60339a670 0x55f6033a49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:02.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.426+0000 7fa6ea881740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6033a4f30 0x55f6033a7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:02.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.426+0000 7fa6ea881740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f6033003e0 con 0x55f6033a4f30 2024-09-19T05:13:02.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.426+0000 7fa6ea881740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f6032a7d80 con 0x55f603399d50 2024-09-19T05:13:02.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.426+0000 7fa6ea881740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f6032fc3a0 con 0x55f60339a670 2024-09-19T05:13:02.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.427+0000 7fa6e4a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6033a4f30 0x55f6033a7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:02.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.427+0000 7fa6e5e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f603399d50 0x55f60339a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:02.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.427+0000 7fa6e5e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f603399d50 0x55f60339a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:34742/0 (socket says 172.21.15.43:34742) 2024-09-19T05:13:02.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.427+0000 7fa6e5e00640 1 -- 172.21.15.43:0/1755176956 learned_addr learned my addr 172.21.15.43:0/1755176956 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:02.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.427+0000 7fa6e5e00640 1 -- 172.21.15.43:0/1755176956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f60339a670 msgr2=0x55f6033a49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:02.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.427+0000 7fa6e5e00640 1 --2- 172.21.15.43:0/1755176956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f60339a670 0x55f6033a49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:02.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.427+0000 7fa6e5e00640 1 -- 172.21.15.43:0/1755176956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6033a4f30 msgr2=0x55f6033a7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:02.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.427+0000 7fa6e5e00640 1 --2- 172.21.15.43:0/1755176956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6033a4f30 0x55f6033a7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:02.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.427+0000 7fa6e5e00640 1 -- 172.21.15.43:0/1755176956 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6032c6ae0 con 0x55f603399d50 2024-09-19T05:13:02.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.427+0000 7fa6de000640 1 --2- 172.21.15.43:0/3998858248 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fa6c80a33b0 0x7fa6c80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:02.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.427+0000 7fa6e5e00640 1 --2- 172.21.15.43:0/1755176956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f603399d50 0x55f60339a130 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8009f70 tx=0x7fa6d8010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad2d0d239cc1b596 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:02.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.427+0000 7fa6de000640 1 -- 172.21.15.43:0/3998858248 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7fa6cc0cfc20 con 0x7fa6e01047e0 2024-09-19T05:13:02.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.428+0000 7fa6d2600640 1 -- 172.21.15.43:0/1755176956 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d8016070 con 0x55f603399d50 2024-09-19T05:13:02.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.428+0000 7fa6e5400640 1 --2- 172.21.15.43:0/3998858248 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fa6c80a33b0 0x7fa6c80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:02.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.428+0000 7fa6d2600640 1 -- 172.21.15.43:0/1755176956 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6d8004d20 con 0x55f603399d50 2024-09-19T05:13:02.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.428+0000 7fa6d2600640 1 -- 172.21.15.43:0/1755176956 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d8012950 con 0x55f603399d50 2024-09-19T05:13:02.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.428+0000 7fa6ea881740 1 -- 172.21.15.43:0/1755176956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f603399d50 msgr2=0x55f60339a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:02.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.428+0000 7fa6ea881740 1 --2- 172.21.15.43:0/1755176956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f603399d50 0x55f60339a130 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8009f70 tx=0x7fa6d8010800 comp rx=0 tx=0).stop 2024-09-19T05:13:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.428+0000 7fa6e5400640 1 --2- 172.21.15.43:0/3998858248 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fa6c80a33b0 0x7fa6c80a5870 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4031550 tx=0x7fa6d40095a0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.429+0000 7fa6ea881740 1 -- 172.21.15.43:0/1755176956 shutdown_connections 2024-09-19T05:13:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.429+0000 7fa6ea881740 1 --2- 172.21.15.43:0/1755176956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6033a4f30 0x55f6033a7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.429+0000 7fa6ea881740 1 --2- 172.21.15.43:0/1755176956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f60339a670 0x55f6033a49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.429+0000 7fa6ea881740 1 --2- 172.21.15.43:0/1755176956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f603399d50 0x55f60339a130 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.429+0000 7fa6ea881740 1 -- 172.21.15.43:0/1755176956 >> 172.21.15.43:0/1755176956 conn(0x55f6033962e0 msgr2=0x55f6033966c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.429+0000 7fa6ea881740 1 -- 172.21.15.43:0/1755176956 shutdown_connections 2024-09-19T05:13:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.429+0000 7fa6ea881740 1 -- 172.21.15.43:0/1755176956 wait complete. 2024-09-19T05:13:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.429+0000 7fa6ea881740 1 -- start start 2024-09-19T05:13:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.429+0000 7fa6ea881740 1 Processor -- start 2024-09-19T05:13:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.430+0000 7fa6ea881740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f603399d50 0x55f6033a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.430+0000 7fa6ea881740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f60339a670 0x55f6033a2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.430+0000 7fa6ea881740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f6033a4f30 0x55f6033a28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.430+0000 7fa6ea881740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f6033a7bf0 con 0x55f603399d50 2024-09-19T05:13:02.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.430+0000 7fa6ea881740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f6032bb0f0 con 0x55f60339a670 2024-09-19T05:13:02.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.430+0000 7fa6ea881740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f6033a7d70 con 0x55f6033a4f30 2024-09-19T05:13:02.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.430+0000 7fa6ea881740 10 client.0 _ll_register_callbacks cb 0x55f6032ac9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:02.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.430+0000 7fa6e5400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f603399d50 0x55f6033a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:02.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.430+0000 7fa6e5400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f603399d50 0x55f6033a1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60358/0 (socket says 172.21.15.43:60358) 2024-09-19T05:13:02.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.430+0000 7fa6e5400640 1 -- 172.21.15.43:0/1700000538 learned_addr learned my addr 172.21.15.43:0/1700000538 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:02.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.430+0000 7fa6e4a00640 1 --2- 172.21.15.43:0/1700000538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f60339a670 0x55f6033a2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:02.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.431+0000 7fa6e4a00640 1 -- 172.21.15.43:0/1700000538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f6033a4f30 msgr2=0x55f6033a28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:02.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.431+0000 7fa6e4a00640 1 --2- 172.21.15.43:0/1700000538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f6033a4f30 0x55f6033a28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:02.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.431+0000 7fa6e4a00640 1 -- 172.21.15.43:0/1700000538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f603399d50 msgr2=0x55f6033a1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:02.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.431+0000 7fa6e4a00640 1 --2- 172.21.15.43:0/1700000538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f603399d50 0x55f6033a1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:02.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.431+0000 7fa6e4a00640 1 -- 172.21.15.43:0/1700000538 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6030713c0 con 0x55f60339a670 2024-09-19T05:13:02.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.431+0000 7fa6e4a00640 1 --2- 172.21.15.43:0/1700000538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f60339a670 0x55f6033a2360 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc0cd300 tx=0x7fa6cc09bd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef5ac746243a33e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:02.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.431+0000 7fa6c6a00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:13:02.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.431+0000 7fa6c6a00640 1 -- 172.21.15.43:0/1700000538 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc020070 con 0x55f60339a670 2024-09-19T05:13:02.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.431+0000 7fa6ea881740 1 -- 172.21.15.43:0/1700000538 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6d8004d20 con 0x55f60339a670 2024-09-19T05:13:02.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.431+0000 7fa6c6a00640 1 -- 172.21.15.43:0/1700000538 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7fa6cc0cd580 con 0x55f60339a670 2024-09-19T05:13:02.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.432+0000 7fa6c6a00640 1 -- 172.21.15.43:0/1700000538 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa6cc0cd780 con 0x55f60339a670 2024-09-19T05:13:02.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.432+0000 7fa6ea881740 10 client.27110 fetch_fsmap learned FSMap version 29 2024-09-19T05:13:02.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.432+0000 7fa6ea881740 1 -- 172.21.15.43:0/1700000538 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f6033b3940 con 0x55f60339a670 2024-09-19T05:13:02.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.432+0000 7fa6c6a00640 1 -- 172.21.15.43:0/1700000538 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6d8004d20 con 0x55f60339a670 2024-09-19T05:13:02.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.432+0000 7fa6c6a00640 1 -- 172.21.15.43:0/1700000538 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa6cc09c230 con 0x55f60339a670 2024-09-19T05:13:02.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.432+0000 7fa6ea881740 10 client.27110 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:13:02.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.432+0000 7fa6ea881740 10 client.27110 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:13:02.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.432+0000 7fa6ea881740 20 client.27110 populate_metadata read hostname 'smithi043' 2024-09-19T05:13:02.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.432+0000 7fa6ea881740 1 --2- 172.21.15.43:0/1700000538 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55f6033b4e00 0x55f6033b7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:02.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.432+0000 7fa6ea881740 4 client.27110 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:13:02.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.432+0000 7fa6ea881740 1 -- 172.21.15.43:0/1700000538 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f60330d950 con 0x55f6033b4e00 2024-09-19T05:13:02.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.433+0000 7fa6e5400640 1 --2- 172.21.15.43:0/1700000538 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55f6033b4e00 0x55f6033b7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:02.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.433+0000 7fa6e5400640 1 --2- 172.21.15.43:0/1700000538 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55f6033b4e00 0x55f6033b7270 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7f8f61c33ead5b4 server_cookie=88eef73c5d99c4fe in_seq=0 out_seq=0 2024-09-19T05:13:02.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.433+0000 7fa6c6a00640 10 client.27110 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:13:02.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.434+0000 7fa6c6a00640 1 -- 172.21.15.43:0/1700000538 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f60330d950 con 0x55f6033b4e00 2024-09-19T05:13:02.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.434+0000 7fa6c6a00640 10 client.27110 handle_command_reply: tid=0 2024-09-19T05:13:02.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.434+0000 7fa6ea881740 1 client.27110 shutdown 2024-09-19T05:13:02.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.434+0000 7fa6ea881740 1 -- 172.21.15.43:0/1700000538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f60339a670 msgr2=0x55f6033a2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:02.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.434+0000 7fa6ea881740 1 --2- 172.21.15.43:0/1700000538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f60339a670 0x55f6033a2360 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc0cd300 tx=0x7fa6cc09bd10 comp rx=0 tx=0).stop 2024-09-19T05:13:02.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.434+0000 7fa6ea881740 1 -- 172.21.15.43:0/1700000538 shutdown_connections 2024-09-19T05:13:02.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.434+0000 7fa6ea881740 1 --2- 172.21.15.43:0/1700000538 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55f6033b4e00 0x55f6033b7270 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:02.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.434+0000 7fa6ea881740 1 --2- 172.21.15.43:0/1700000538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f6033a4f30 0x55f6033a28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:02.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.434+0000 7fa6ea881740 1 --2- 172.21.15.43:0/1700000538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f60339a670 0x55f6033a2360 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:02.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.434+0000 7fa6c6a00640 0 client.27110 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:13:02.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.434+0000 7fa6ea881740 1 --2- 172.21.15.43:0/1700000538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f603399d50 0x55f6033a1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:02.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.434+0000 7fa6ea881740 1 -- 172.21.15.43:0/1700000538 >> 172.21.15.43:0/1700000538 conn(0x55f6033962e0 msgr2=0x55f60339bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:02.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.434+0000 7fa6ea881740 1 -- 172.21.15.43:0/1700000538 shutdown_connections 2024-09-19T05:13:02.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.434+0000 7fa6ea881740 1 -- 172.21.15.43:0/1700000538 wait complete. 2024-09-19T05:13:02.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.435+0000 7fa6ea881740 20 client.27110 trim_cache size 0 max 16384 2024-09-19T05:13:02.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.453+0000 7fa6ea881740 1 Processor -- start 2024-09-19T05:13:02.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.453+0000 7fa6ea881740 1 -- start start 2024-09-19T05:13:02.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.453+0000 7fa6ea881740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f603399d50 0x55f6033a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:02.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.453+0000 7fa6ea881740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f60339a670 0x55f6033bcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:02.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.453+0000 7fa6ea881740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6033a4f30 0x55f6033bd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:02.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.453+0000 7fa6ea881740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e01af970 con 0x55f6033a4f30 2024-09-19T05:13:02.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.453+0000 7fa6ea881740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e01b14b0 con 0x55f60339a670 2024-09-19T05:13:02.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.453+0000 7fa6ea881740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e01b0040 con 0x55f603399d50 2024-09-19T05:13:02.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.454+0000 7fa6e5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6033a4f30 0x55f6033bd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:02.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.454+0000 7fa6e5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6033a4f30 0x55f6033bd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60364/0 (socket says 172.21.15.43:60364) 2024-09-19T05:13:02.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.454+0000 7fa6e5e00640 1 -- 172.21.15.43:0/3221420347 learned_addr learned my addr 172.21.15.43:0/3221420347 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:02.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.454+0000 7fa6e4a00640 1 --2- 172.21.15.43:0/3221420347 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f60339a670 0x55f6033bcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:02.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.454+0000 7fa6e5400640 1 --2- 172.21.15.43:0/3221420347 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f603399d50 0x55f6033a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:02.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.454+0000 7fa6e4a00640 1 -- 172.21.15.43:0/3221420347 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f603399d50 msgr2=0x55f6033a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:02.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.454+0000 7fa6e4a00640 1 --2- 172.21.15.43:0/3221420347 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f603399d50 0x55f6033a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:02.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.454+0000 7fa6e4a00640 1 -- 172.21.15.43:0/3221420347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6033a4f30 msgr2=0x55f6033bd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:02.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.454+0000 7fa6e4a00640 1 --2- 172.21.15.43:0/3221420347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6033a4f30 0x55f6033bd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:02.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.454+0000 7fa6e4a00640 1 -- 172.21.15.43:0/3221420347 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6033a0dd0 con 0x55f60339a670 2024-09-19T05:13:02.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.454+0000 7fa6e5e00640 1 --2- 172.21.15.43:0/3221420347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6033a4f30 0x55f6033bd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:13:02.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.454+0000 7fa6e4a00640 1 --2- 172.21.15.43:0/3221420347 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f60339a670 0x55f6033bcb40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc09bc40 tx=0x7fa6cc003800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aecfaaed4c81b1a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:02.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.455+0000 7fa6d1c00640 1 -- 172.21.15.43:0/3221420347 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc0d3020 con 0x55f60339a670 2024-09-19T05:13:02.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.455+0000 7fa6d1c00640 1 -- 172.21.15.43:0/3221420347 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6cc09bf40 con 0x55f60339a670 2024-09-19T05:13:02.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.455+0000 7fa6d1c00640 1 -- 172.21.15.43:0/3221420347 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc0cd760 con 0x55f60339a670 2024-09-19T05:13:02.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.455+0000 7fa6ea881740 1 -- 172.21.15.43:0/3221420347 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f60339a670 msgr2=0x55f6033bcb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:02.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.455+0000 7fa6ea881740 1 --2- 172.21.15.43:0/3221420347 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f60339a670 0x55f6033bcb40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc09bc40 tx=0x7fa6cc003800 comp rx=0 tx=0).stop 2024-09-19T05:13:02.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.455+0000 7fa6ea881740 1 -- 172.21.15.43:0/3221420347 shutdown_connections 2024-09-19T05:13:02.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.455+0000 7fa6ea881740 1 --2- 172.21.15.43:0/3221420347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6033a4f30 0x55f6033bd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:02.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.455+0000 7fa6ea881740 1 --2- 172.21.15.43:0/3221420347 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f60339a670 0x55f6033bcb40 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:02.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.455+0000 7fa6ea881740 1 --2- 172.21.15.43:0/3221420347 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f603399d50 0x55f6033a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:02.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.455+0000 7fa6ea881740 1 -- 172.21.15.43:0/3221420347 >> 172.21.15.43:0/3221420347 conn(0x55f6033962e0 msgr2=0x55f6033a3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:02.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.456+0000 7fa6ea881740 1 -- 172.21.15.43:0/3221420347 shutdown_connections 2024-09-19T05:13:02.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.456+0000 7fa6ea881740 1 -- 172.21.15.43:0/3221420347 wait complete. 2024-09-19T05:13:02.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.456+0000 7fa6ea881740 1 -- start start 2024-09-19T05:13:02.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.456+0000 7fa6ea881740 1 Processor -- start 2024-09-19T05:13:02.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.457+0000 7fa6ea881740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f603399d50 0x55f6033a8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:02.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.457+0000 7fa6ea881740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f60339a670 0x55f6033a9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:02.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.457+0000 7fa6ea881740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6033a4f30 0x55f6033a9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:02.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.457+0000 7fa6ea881740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e01b0de0 con 0x55f60339a670 2024-09-19T05:13:02.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.457+0000 7fa6ea881740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e01b0710 con 0x55f6033a4f30 2024-09-19T05:13:02.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.457+0000 7fa6ea881740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e01ad970 con 0x55f603399d50 2024-09-19T05:13:02.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.457+0000 7fa6ea881740 10 client.0 _ll_register_callbacks cb 0x7fa6cc0d33c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:02.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.457+0000 7fa6e4a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f60339a670 0x55f6033a9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:02.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.457+0000 7fa6e5400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f603399d50 0x55f6033a8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:02.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.457+0000 7fa6e5e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6033a4f30 0x55f6033a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:02.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.457+0000 7fa6e5400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f603399d50 0x55f6033a8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:34468/0 (socket says 172.21.15.43:34468) 2024-09-19T05:13:02.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.457+0000 7fa6e5400640 1 -- 172.21.15.43:0/2846739389 learned_addr learned my addr 172.21.15.43:0/2846739389 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:02.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.457+0000 7fa6e5400640 1 -- 172.21.15.43:0/2846739389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6033a4f30 msgr2=0x55f6033a9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:02.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.457+0000 7fa6e5400640 1 --2- 172.21.15.43:0/2846739389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6033a4f30 0x55f6033a9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:02.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.457+0000 7fa6e5400640 1 -- 172.21.15.43:0/2846739389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f60339a670 msgr2=0x55f6033a9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:02.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.457+0000 7fa6e5400640 1 --2- 172.21.15.43:0/2846739389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f60339a670 0x55f6033a9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:02.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.457+0000 7fa6e5400640 1 -- 172.21.15.43:0/2846739389 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f60341ebc0 con 0x55f603399d50 2024-09-19T05:13:02.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.458+0000 7fa6e5400640 1 --2- 172.21.15.43:0/2846739389 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f603399d50 0x55f6033a8e10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x55f6033b7ae0 tx=0x7fa6d403d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca82ab74e785cf09 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:02.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.458+0000 7fa6c6a00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:13:02.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.458+0000 7fa6c6a00640 1 -- 172.21.15.43:0/2846739389 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d4035040 con 0x55f603399d50 2024-09-19T05:13:02.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.458+0000 7fa6c6a00640 1 -- 172.21.15.43:0/2846739389 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7fa6d4004040 con 0x55f603399d50 2024-09-19T05:13:02.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.458+0000 7fa6ea881740 1 -- 172.21.15.43:0/2846739389 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6cc09bf40 con 0x55f603399d50 2024-09-19T05:13:02.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.458+0000 7fa6c6a00640 1 -- 172.21.15.43:0/2846739389 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa6d4004240 con 0x55f603399d50 2024-09-19T05:13:02.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.458+0000 7fa6ea881740 10 client.27025 fetch_fsmap learned FSMap version 29 2024-09-19T05:13:02.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.458+0000 7fa6ea881740 1 -- 172.21.15.43:0/2846739389 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f60341fef0 con 0x55f603399d50 2024-09-19T05:13:02.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.459+0000 7fa6c6a00640 1 -- 172.21.15.43:0/2846739389 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6cc09bf40 con 0x55f603399d50 2024-09-19T05:13:02.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.459+0000 7fa6c6a00640 1 -- 172.21.15.43:0/2846739389 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa6d4004aa0 con 0x55f603399d50 2024-09-19T05:13:02.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.459+0000 7fa6ea881740 10 client.27025 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:13:02.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.459+0000 7fa6ea881740 10 client.27025 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:13:02.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.459+0000 7fa6ea881740 20 client.27025 populate_metadata read hostname 'smithi043' 2024-09-19T05:13:02.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.459+0000 7fa6ea881740 1 --2- 172.21.15.43:0/2846739389 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55f6034205f0 0x55f603422a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:02.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.459+0000 7fa6ea881740 4 client.27025 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:13:02.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.459+0000 7fa6ea881740 1 -- 172.21.15.43:0/2846739389 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f603307e60 con 0x55f6034205f0 2024-09-19T05:13:02.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.460+0000 7fa6e4a00640 1 --2- 172.21.15.43:0/2846739389 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55f6034205f0 0x55f603422a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:02.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.460+0000 7fa6e4a00640 1 --2- 172.21.15.43:0/2846739389 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55f6034205f0 0x55f603422a10 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21cb1be6b3fdaa17 server_cookie=46e64d3d5819d9f4 in_seq=0 out_seq=0 2024-09-19T05:13:02.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.460+0000 7fa6c6a00640 10 client.27025 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:13:02.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.460+0000 7fa6c6a00640 1 -- 172.21.15.43:0/2846739389 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f603307e60 con 0x55f6034205f0 2024-09-19T05:13:02.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.460+0000 7fa6c6a00640 10 client.27025 handle_command_reply: tid=0 2024-09-19T05:13:02.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.460+0000 7fa6ea881740 1 client.27025 shutdown 2024-09-19T05:13:02.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.461+0000 7fa6ea881740 1 -- 172.21.15.43:0/2846739389 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f603399d50 msgr2=0x55f6033a8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:02.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.461+0000 7fa6ea881740 1 --2- 172.21.15.43:0/2846739389 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f603399d50 0x55f6033a8e10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x55f6033b7ae0 tx=0x7fa6d403d000 comp rx=0 tx=0).stop 2024-09-19T05:13:02.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.461+0000 7fa6ea881740 1 -- 172.21.15.43:0/2846739389 shutdown_connections 2024-09-19T05:13:02.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.461+0000 7fa6ea881740 1 --2- 172.21.15.43:0/2846739389 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55f6034205f0 0x55f603422a10 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:02.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.461+0000 7fa6ea881740 1 --2- 172.21.15.43:0/2846739389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6033a4f30 0x55f6033a9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:02.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.461+0000 7fa6ea881740 1 --2- 172.21.15.43:0/2846739389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f60339a670 0x55f6033a9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:02.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.461+0000 7fa6c6a00640 0 client.27025 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:13:02.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.461+0000 7fa6ea881740 1 --2- 172.21.15.43:0/2846739389 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f603399d50 0x55f6033a8e10 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:02.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.461+0000 7fa6ea881740 1 -- 172.21.15.43:0/2846739389 >> 172.21.15.43:0/2846739389 conn(0x55f6033962e0 msgr2=0x55f6033a3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:02.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.461+0000 7fa6ea881740 1 -- 172.21.15.43:0/2846739389 shutdown_connections 2024-09-19T05:13:02.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.461+0000 7fa6ea881740 1 -- 172.21.15.43:0/2846739389 wait complete. 2024-09-19T05:13:02.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.461+0000 7fa6ea881740 20 client.27025 trim_cache size 0 max 16384 2024-09-19T05:13:02.465 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:13:02.465 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-09-19T05:13:02.465 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-09-19T05:13:02.465 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:13:02.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.462+0000 7fa6e6800640 1 -- 172.21.15.43:0/3998858248 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fa6c80a33b0 msgr2=0x7fa6c80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:02.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.462+0000 7fa6e6800640 1 --2- 172.21.15.43:0/3998858248 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fa6c80a33b0 0x7fa6c80a5870 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4031550 tx=0x7fa6d40095a0 comp rx=0 tx=0).stop 2024-09-19T05:13:02.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.462+0000 7fa6e6800640 1 -- 172.21.15.43:0/3998858248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6e01047e0 msgr2=0x7fa6e019a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:02.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.462+0000 7fa6e6800640 1 --2- 172.21.15.43:0/3998858248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6e01047e0 0x7fa6e019a9a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc005fd0 tx=0x7fa6cc00dde0 comp rx=0 tx=0).stop 2024-09-19T05:13:02.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.462+0000 7fa6e6800640 1 -- 172.21.15.43:0/3998858248 shutdown_connections 2024-09-19T05:13:02.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.462+0000 7fa6e6800640 1 --2- 172.21.15.43:0/3998858248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6e0108f60 0x7fa6e019ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:02.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.462+0000 7fa6e6800640 1 --2- 172.21.15.43:0/3998858248 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fa6c80a33b0 0x7fa6c80a5870 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:02.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.462+0000 7fa6e6800640 1 --2- 172.21.15.43:0/3998858248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6e01047e0 0x7fa6e019a9a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:02.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.462+0000 7fa6e6800640 1 --2- 172.21.15.43:0/3998858248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6e0103e30 0x7fa6e019a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:02.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.462+0000 7fa6e6800640 1 -- 172.21.15.43:0/3998858248 >> 172.21.15.43:0/3998858248 conn(0x7fa6e00ff8d0 msgr2=0x7fa6e0100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:02.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.462+0000 7fa6e6800640 1 -- 172.21.15.43:0/3998858248 shutdown_connections 2024-09-19T05:13:02.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:02.462+0000 7fa6e6800640 1 -- 172.21.15.43:0/3998858248 wait complete. 2024-09-19T05:13:02.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:02 smithi096 ceph-mon[35085]: pgmap v258: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:13:02.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:02 smithi151 ceph-mon[35425]: pgmap v258: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:13:03.339 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T05:13:03.339 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:13:03.718 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:13:04.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.327+0000 7f8277000640 1 -- 172.21.15.43:0/242815494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82701019c0 msgr2=0x7f8270101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:04.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.327+0000 7f8277000640 1 --2- 172.21.15.43:0/242815494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82701019c0 0x7f8270101e20 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f825c009930 tx=0x7f825c02a440 comp rx=0 tx=0).stop 2024-09-19T05:13:04.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.327+0000 7f8277000640 1 -- 172.21.15.43:0/242815494 shutdown_connections 2024-09-19T05:13:04.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.327+0000 7f8277000640 1 --2- 172.21.15.43:0/242815494 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8270102360 0x7f827010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:04.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.327+0000 7f8277000640 1 --2- 172.21.15.43:0/242815494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82701019c0 0x7f8270101e20 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:04.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.327+0000 7f8277000640 1 --2- 172.21.15.43:0/242815494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82701079c0 0x7f8270107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:04.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.327+0000 7f8277000640 1 -- 172.21.15.43:0/242815494 >> 172.21.15.43:0/242815494 conn(0x7f82700fd740 msgr2=0x7f82700ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:04.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.327+0000 7f8277000640 1 -- 172.21.15.43:0/242815494 shutdown_connections 2024-09-19T05:13:04.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.327+0000 7f8277000640 1 -- 172.21.15.43:0/242815494 wait complete. 2024-09-19T05:13:04.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.328+0000 7f8277000640 1 Processor -- start 2024-09-19T05:13:04.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.328+0000 7f8277000640 1 -- start start 2024-09-19T05:13:04.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.328+0000 7f8277000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82701019c0 0x7f82700744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:04.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.328+0000 7f8277000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8270102360 0x7f8270074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:04.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.329+0000 7f8277000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82701079c0 0x7f8270074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:04.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.329+0000 7f8277000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8270111e50 con 0x7f82701079c0 2024-09-19T05:13:04.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.329+0000 7f8277000640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8270111cd0 con 0x7f8270102360 2024-09-19T05:13:04.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.329+0000 7f8277000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8270111fd0 con 0x7f82701019c0 2024-09-19T05:13:04.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.329+0000 7f8276600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82701079c0 0x7f8270074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:04.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.329+0000 7f8275200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8270102360 0x7f8270074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:04.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.329+0000 7f8275200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8270102360 0x7f8270074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:34794/0 (socket says 172.21.15.43:34794) 2024-09-19T05:13:04.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.329+0000 7f8275200640 1 -- 172.21.15.43:0/3816869275 learned_addr learned my addr 172.21.15.43:0/3816869275 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:04.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.329+0000 7f8275c00640 1 --2- 172.21.15.43:0/3816869275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82701019c0 0x7f82700744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:04.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.329+0000 7f8276600640 1 -- 172.21.15.43:0/3816869275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82701019c0 msgr2=0x7f82700744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:04.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.329+0000 7f8276600640 1 --2- 172.21.15.43:0/3816869275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82701019c0 0x7f82700744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:04.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.329+0000 7f8276600640 1 -- 172.21.15.43:0/3816869275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8270102360 msgr2=0x7f8270074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:04.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.329+0000 7f8276600640 1 --2- 172.21.15.43:0/3816869275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8270102360 0x7f8270074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:04.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.329+0000 7f8276600640 1 -- 172.21.15.43:0/3816869275 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82701a6830 con 0x7f82701079c0 2024-09-19T05:13:04.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.330+0000 7f8276600640 1 --2- 172.21.15.43:0/3816869275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82701079c0 0x7f8270074f70 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f826800be20 tx=0x7f826800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:04.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.330+0000 7f826e000640 1 -- 172.21.15.43:0/3816869275 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8268019070 con 0x7f82701079c0 2024-09-19T05:13:04.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.330+0000 7f826e000640 1 -- 172.21.15.43:0/3816869275 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8268014920 con 0x7f82701079c0 2024-09-19T05:13:04.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.330+0000 7f826e000640 1 -- 172.21.15.43:0/3816869275 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8268012940 con 0x7f82701079c0 2024-09-19T05:13:04.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.330+0000 7f8277000640 1 -- 172.21.15.43:0/3816869275 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82701a6b20 con 0x7f82701079c0 2024-09-19T05:13:04.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.330+0000 7f8277000640 1 -- 172.21.15.43:0/3816869275 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82701a70b0 con 0x7f82701079c0 2024-09-19T05:13:04.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.331+0000 7f8277000640 1 -- 172.21.15.43:0/3816869275 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8270101e20 con 0x7f82701079c0 2024-09-19T05:13:04.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.333+0000 7f826e000640 1 -- 172.21.15.43:0/3816869275 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f826801c070 con 0x7f82701079c0 2024-09-19T05:13:04.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.335+0000 7f826e000640 1 --2- 172.21.15.43:0/3816869275 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f82580a3430 0x7f82580a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:04.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.335+0000 7f826e000640 1 -- 172.21.15.43:0/3816869275 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f82680cfab0 con 0x7f82701079c0 2024-09-19T05:13:04.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.335+0000 7f8275c00640 1 --2- 172.21.15.43:0/3816869275 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f82580a3430 0x7f82580a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:04.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.336+0000 7f8275c00640 1 --2- 172.21.15.43:0/3816869275 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f82580a3430 0x7f82580a58f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8264009bd0 tx=0x7f8264005f90 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:04.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.337+0000 7f826e000640 1 -- 172.21.15.43:0/3816869275 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f826809bee0 con 0x7f82701079c0 2024-09-19T05:13:04.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:04 smithi096 ceph-mon[35085]: pgmap v259: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:13:04.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:04 smithi043 ceph-mon[30800]: pgmap v259: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:13:04.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.566+0000 7f8277000640 1 -- 172.21.15.43:0/3816869275 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8270075a20 con 0x7f82701079c0 2024-09-19T05:13:04.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.567+0000 7f826e000640 1 -- 172.21.15.43:0/3816869275 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11965 (secure 0 0 0) 0x7f82680a01e0 con 0x7f82701079c0 2024-09-19T05:13:04.571 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:13:04.571 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":29,"btime":"2024-09-19T05:04:38:785407+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/773801169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":773801169},{"type":"v1","addr":"172.21.15.43:6835","nonce":773801169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1799137902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1799137902},{"type":"v1","addr":"172.21.15.43:6837","nonce":1799137902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:02:25.463041+0000","modified":"2024-09-19T05:04:38.039809+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24518,"mds_2":24511,"mds_3":24506,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14541":{"gid":14541,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/794881135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":794881135},{"type":"v1","addr":"172.21.15.43:6841","nonce":794881135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/2836453158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2836453158},{"type":"v1","addr":"172.21.15.151:6833","nonce":2836453158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/178785812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":178785812},{"type":"v1","addr":"172.21.15.96:6833","nonce":178785812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1671043820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1671043820},{"type":"v1","addr":"172.21.15.96:6835","nonce":1671043820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2972514328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2972514328},{"type":"v1","addr":"172.21.15.151:6835","nonce":2972514328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.151:6837/3960378010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3960378010},{"type":"v1","addr":"172.21.15.151:6837","nonce":3960378010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6837/3978388286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3978388286},{"type":"v1","addr":"172.21.15.96:6837","nonce":3978388286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.151:6839/3794104313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3794104313},{"type":"v1","addr":"172.21.15.151:6839","nonce":3794104313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.43:6839/3404555030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3404555030},{"type":"v1","addr":"172.21.15.43:6839","nonce":3404555030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/605527621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":605527621},{"type":"v1","addr":"172.21.15.96:6839","nonce":605527621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24518,24511,24506,24493]},"id":1}]} 2024-09-19T05:13:04.572 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 29 2024-09-19T05:13:04.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.569+0000 7f8277000640 1 -- 172.21.15.43:0/3816869275 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f82580a3430 msgr2=0x7f82580a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:04.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.569+0000 7f8277000640 1 --2- 172.21.15.43:0/3816869275 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f82580a3430 0x7f82580a58f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8264009bd0 tx=0x7f8264005f90 comp rx=0 tx=0).stop 2024-09-19T05:13:04.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.569+0000 7f8277000640 1 -- 172.21.15.43:0/3816869275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82701079c0 msgr2=0x7f8270074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:04.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.569+0000 7f8277000640 1 --2- 172.21.15.43:0/3816869275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82701079c0 0x7f8270074f70 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f826800be20 tx=0x7f826800bf10 comp rx=0 tx=0).stop 2024-09-19T05:13:04.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.569+0000 7f8277000640 1 -- 172.21.15.43:0/3816869275 shutdown_connections 2024-09-19T05:13:04.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.569+0000 7f8277000640 1 --2- 172.21.15.43:0/3816869275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82701079c0 0x7f8270074f70 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:04.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.569+0000 7f8277000640 1 --2- 172.21.15.43:0/3816869275 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f82580a3430 0x7f82580a58f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:04.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.569+0000 7f8277000640 1 --2- 172.21.15.43:0/3816869275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8270102360 0x7f8270074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:04.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.569+0000 7f8277000640 1 --2- 172.21.15.43:0/3816869275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82701019c0 0x7f82700744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:04.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.569+0000 7f8277000640 1 -- 172.21.15.43:0/3816869275 >> 172.21.15.43:0/3816869275 conn(0x7f82700fd740 msgr2=0x7f82700fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:04.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.569+0000 7f8277000640 1 -- 172.21.15.43:0/3816869275 shutdown_connections 2024-09-19T05:13:04.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:04.569+0000 7f8277000640 1 -- 172.21.15.43:0/3816869275 wait complete. 2024-09-19T05:13:04.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:04 smithi151 ceph-mon[35425]: pgmap v259: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:13:05.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:05 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/3816869275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:13:05.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:05 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/3816869275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:13:05.487 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T05:13:05.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:05 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/3816869275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:13:05.866 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:13:06.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:06 smithi043 ceph-mon[30800]: pgmap v260: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-09-19T05:13:06.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.398+0000 7fd656e00640 1 -- 172.21.15.43:0/3972921714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd650076720 msgr2=0x7fd650076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:06.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.398+0000 7fd656e00640 1 --2- 172.21.15.43:0/3972921714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd650076720 0x7fd650076b00 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd640009900 tx=0x7fd64002a410 comp rx=0 tx=0).stop 2024-09-19T05:13:06.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.398+0000 7fd656e00640 1 -- 172.21.15.43:0/3972921714 shutdown_connections 2024-09-19T05:13:06.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.398+0000 7fd656e00640 1 --2- 172.21.15.43:0/3972921714 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd650112a30 0x7fd650114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:06.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.398+0000 7fd656e00640 1 --2- 172.21.15.43:0/3972921714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd650077040 0x7fd6501124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:06.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.398+0000 7fd656e00640 1 --2- 172.21.15.43:0/3972921714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd650076720 0x7fd650076b00 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:06.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.398+0000 7fd656e00640 1 -- 172.21.15.43:0/3972921714 >> 172.21.15.43:0/3972921714 conn(0x7fd6500ffa90 msgr2=0x7fd650101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:06.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.398+0000 7fd656e00640 1 -- 172.21.15.43:0/3972921714 shutdown_connections 2024-09-19T05:13:06.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.398+0000 7fd656e00640 1 -- 172.21.15.43:0/3972921714 wait complete. 2024-09-19T05:13:06.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.399+0000 7fd656e00640 1 Processor -- start 2024-09-19T05:13:06.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.399+0000 7fd656e00640 1 -- start start 2024-09-19T05:13:06.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.399+0000 7fd656e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd650076720 0x7fd6501a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:06.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.400+0000 7fd656e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd650077040 0x7fd6501a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:06.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.400+0000 7fd656e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd650112a30 0x7fd65019e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:06.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.400+0000 7fd656e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd6501185a0 con 0x7fd650077040 2024-09-19T05:13:06.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.400+0000 7fd656e00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd650118420 con 0x7fd650076720 2024-09-19T05:13:06.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.400+0000 7fd656e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd650118720 con 0x7fd650112a30 2024-09-19T05:13:06.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.400+0000 7fd647e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd650077040 0x7fd6501a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:06.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.400+0000 7fd64fe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd650076720 0x7fd6501a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:06.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.400+0000 7fd647e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd650077040 0x7fd6501a4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40140/0 (socket says 172.21.15.43:40140) 2024-09-19T05:13:06.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.400+0000 7fd647e00640 1 -- 172.21.15.43:0/3217627302 learned_addr learned my addr 172.21.15.43:0/3217627302 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:06.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.400+0000 7fd654e00640 1 --2- 172.21.15.43:0/3217627302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd650112a30 0x7fd65019e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:06.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.400+0000 7fd647e00640 1 -- 172.21.15.43:0/3217627302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd650112a30 msgr2=0x7fd65019e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:06.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.400+0000 7fd647e00640 1 --2- 172.21.15.43:0/3217627302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd650112a30 0x7fd65019e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:06.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.400+0000 7fd647e00640 1 -- 172.21.15.43:0/3217627302 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd650076720 msgr2=0x7fd6501a4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:06.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.400+0000 7fd647e00640 1 --2- 172.21.15.43:0/3217627302 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd650076720 0x7fd6501a4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:06.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.401+0000 7fd647e00640 1 -- 172.21.15.43:0/3217627302 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd65019f000 con 0x7fd650077040 2024-09-19T05:13:06.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.401+0000 7fd647e00640 1 --2- 172.21.15.43:0/3217627302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd650077040 0x7fd6501a4c50 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd63800e9f0 tx=0x7fd63800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:06.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.401+0000 7fd64d600640 1 -- 172.21.15.43:0/3217627302 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63800cd80 con 0x7fd650077040 2024-09-19T05:13:06.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.401+0000 7fd64d600640 1 -- 172.21.15.43:0/3217627302 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd638010910 con 0x7fd650077040 2024-09-19T05:13:06.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.401+0000 7fd64d600640 1 -- 172.21.15.43:0/3217627302 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd638018990 con 0x7fd650077040 2024-09-19T05:13:06.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.401+0000 7fd656e00640 1 -- 172.21.15.43:0/3217627302 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd65019f2f0 con 0x7fd650077040 2024-09-19T05:13:06.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.401+0000 7fd656e00640 1 -- 172.21.15.43:0/3217627302 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6501ab5e0 con 0x7fd650077040 2024-09-19T05:13:06.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.405+0000 7fd659a40740 1 Processor -- start 2024-09-19T05:13:06.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.405+0000 7fd64d600640 1 -- 172.21.15.43:0/3217627302 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fd638010ac0 con 0x7fd650077040 2024-09-19T05:13:06.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.405+0000 7fd659a40740 1 -- start start 2024-09-19T05:13:06.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.405+0000 7fd659a40740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564fc0853c70 0x564fc0854050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:06.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.405+0000 7fd659a40740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fc0854590 0x564fc085e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:06.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.405+0000 7fd659a40740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc085ee50 0x564fc0861240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:06.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.405+0000 7fd659a40740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564fc07ba3e0 con 0x564fc085ee50 2024-09-19T05:13:06.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.405+0000 7fd659a40740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564fc0761d80 con 0x564fc0854590 2024-09-19T05:13:06.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.405+0000 7fd659a40740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564fc07b63a0 con 0x564fc0853c70 2024-09-19T05:13:06.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.406+0000 7fd64fe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fc0854590 0x564fc085e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:06.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.406+0000 7fd654e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564fc0853c70 0x564fc0854050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:06.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.406+0000 7fd64fe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fc0854590 0x564fc085e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:52508/0 (socket says 172.21.15.43:52508) 2024-09-19T05:13:06.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.406+0000 7fd64fe00640 1 -- 172.21.15.43:0/4267255237 learned_addr learned my addr 172.21.15.43:0/4267255237 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:06.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.406+0000 7fd654e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564fc0853c70 0x564fc0854050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:57344/0 (socket says 172.21.15.43:57344) 2024-09-19T05:13:06.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.406+0000 7fd647e00640 1 --2- 172.21.15.43:0/4267255237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc085ee50 0x564fc0861240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:06.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.406+0000 7fd64fe00640 1 -- 172.21.15.43:0/4267255237 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564fc0853c70 msgr2=0x564fc0854050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:06.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.406+0000 7fd64fe00640 1 --2- 172.21.15.43:0/4267255237 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564fc0853c70 0x564fc0854050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:06.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.406+0000 7fd64fe00640 1 -- 172.21.15.43:0/4267255237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc085ee50 msgr2=0x564fc0861240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:06.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.406+0000 7fd64fe00640 1 --2- 172.21.15.43:0/4267255237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc085ee50 0x564fc0861240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:06.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.406+0000 7fd64fe00640 1 -- 172.21.15.43:0/4267255237 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fc0780ae0 con 0x564fc0854590 2024-09-19T05:13:06.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.407+0000 7fd64fe00640 1 --2- 172.21.15.43:0/4267255237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fc0854590 0x564fc085e910 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd640009900 tx=0x7fd640005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c41e6e52434eff3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:06.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.407+0000 7fd64d600640 1 --2- 172.21.15.43:0/3217627302 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fd6340a3430 0x7fd6340a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:06.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.407+0000 7fd645600640 1 -- 172.21.15.43:0/4267255237 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd640040070 con 0x564fc0854590 2024-09-19T05:13:06.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.407+0000 7fd645600640 1 -- 172.21.15.43:0/4267255237 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd640031870 con 0x564fc0854590 2024-09-19T05:13:06.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.407+0000 7fd645600640 1 -- 172.21.15.43:0/4267255237 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd640035040 con 0x564fc0854590 2024-09-19T05:13:06.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.407+0000 7fd64d600640 1 -- 172.21.15.43:0/3217627302 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7fd6380cffe0 con 0x7fd650077040 2024-09-19T05:13:06.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.407+0000 7fd654e00640 1 --2- 172.21.15.43:0/3217627302 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fd6340a3430 0x7fd6340a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:06.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.408+0000 7fd659a40740 1 -- 172.21.15.43:0/4267255237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fc0854590 msgr2=0x564fc085e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:06.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.408+0000 7fd659a40740 1 --2- 172.21.15.43:0/4267255237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fc0854590 0x564fc085e910 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd640009900 tx=0x7fd640005be0 comp rx=0 tx=0).stop 2024-09-19T05:13:06.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.408+0000 7fd659a40740 1 -- 172.21.15.43:0/4267255237 shutdown_connections 2024-09-19T05:13:06.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.408+0000 7fd659a40740 1 --2- 172.21.15.43:0/4267255237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc085ee50 0x564fc0861240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:06.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.408+0000 7fd659a40740 1 --2- 172.21.15.43:0/4267255237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fc0854590 0x564fc085e910 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:06.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.408+0000 7fd659a40740 1 --2- 172.21.15.43:0/4267255237 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564fc0853c70 0x564fc0854050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:06.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.408+0000 7fd659a40740 1 -- 172.21.15.43:0/4267255237 >> 172.21.15.43:0/4267255237 conn(0x564fc0850200 msgr2=0x564fc08505e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:06.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.408+0000 7fd654e00640 1 --2- 172.21.15.43:0/3217627302 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fd6340a3430 0x7fd6340a58f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd65006b460 tx=0x7fd648009500 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:06.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.408+0000 7fd659a40740 1 -- 172.21.15.43:0/4267255237 shutdown_connections 2024-09-19T05:13:06.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.408+0000 7fd659a40740 1 -- 172.21.15.43:0/4267255237 wait complete. 2024-09-19T05:13:06.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.408+0000 7fd659a40740 1 -- start start 2024-09-19T05:13:06.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.409+0000 7fd659a40740 1 Processor -- start 2024-09-19T05:13:06.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.409+0000 7fd659a40740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc0853c70 0x564fc085bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:06.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.409+0000 7fd659a40740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fc0854590 0x564fc085c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:06.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.409+0000 7fd659a40740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564fc085ee50 0x564fc085c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:06.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.409+0000 7fd659a40740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564fc0861a20 con 0x564fc0853c70 2024-09-19T05:13:06.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.409+0000 7fd659a40740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564fc07750f0 con 0x564fc0854590 2024-09-19T05:13:06.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.409+0000 7fd659a40740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564fc0861ba0 con 0x564fc085ee50 2024-09-19T05:13:06.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.409+0000 7fd659a40740 10 client.0 _ll_register_callbacks cb 0x564fc075cbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:06.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.409+0000 7fd64fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc0853c70 0x564fc085bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:06.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.409+0000 7fd64fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc0853c70 0x564fc085bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40158/0 (socket says 172.21.15.43:40158) 2024-09-19T05:13:06.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.409+0000 7fd64fe00640 1 -- 172.21.15.43:0/1193774537 learned_addr learned my addr 172.21.15.43:0/1193774537 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:06.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.410+0000 7fd647e00640 1 --2- 172.21.15.43:0/1193774537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fc0854590 0x564fc085c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:06.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.410+0000 7fd654e00640 1 --2- 172.21.15.43:0/1193774537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564fc085ee50 0x564fc085c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:06.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.410+0000 7fd647e00640 1 -- 172.21.15.43:0/1193774537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564fc085ee50 msgr2=0x564fc085c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:06.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.410+0000 7fd647e00640 1 --2- 172.21.15.43:0/1193774537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564fc085ee50 0x564fc085c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:06.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.410+0000 7fd647e00640 1 -- 172.21.15.43:0/1193774537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc0853c70 msgr2=0x564fc085bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:06.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.410+0000 7fd647e00640 1 --2- 172.21.15.43:0/1193774537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc0853c70 0x564fc085bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:06.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.410+0000 7fd647e00640 1 -- 172.21.15.43:0/1193774537 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fc052b3c0 con 0x564fc0854590 2024-09-19T05:13:06.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.410+0000 7fd64fe00640 1 --2- 172.21.15.43:0/1193774537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc0853c70 0x564fc085bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:13:06.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.410+0000 7fd647e00640 1 --2- 172.21.15.43:0/1193774537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fc0854590 0x564fc085c1f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd6380cd6e0 tx=0x7fd638009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8885a471d9f37ff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:06.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.410+0000 7fd63e000640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:13:06.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.410+0000 7fd63e000640 1 -- 172.21.15.43:0/1193774537 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63809bc80 con 0x564fc0854590 2024-09-19T05:13:06.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.411+0000 7fd63e000640 1 -- 172.21.15.43:0/1193774537 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7fd6380d3bc0 con 0x564fc0854590 2024-09-19T05:13:06.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.411+0000 7fd659a40740 1 -- 172.21.15.43:0/1193774537 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd640031870 con 0x564fc0854590 2024-09-19T05:13:06.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.411+0000 7fd63e000640 1 -- 172.21.15.43:0/1193774537 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd6380cdb60 con 0x564fc0854590 2024-09-19T05:13:06.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.411+0000 7fd659a40740 10 client.27134 fetch_fsmap learned FSMap version 29 2024-09-19T05:13:06.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.411+0000 7fd659a40740 1 -- 172.21.15.43:0/1193774537 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564fc086d9e0 con 0x564fc0854590 2024-09-19T05:13:06.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.411+0000 7fd63e000640 1 -- 172.21.15.43:0/1193774537 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd640031870 con 0x564fc0854590 2024-09-19T05:13:06.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.411+0000 7fd63e000640 1 -- 172.21.15.43:0/1193774537 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd6380bc140 con 0x564fc0854590 2024-09-19T05:13:06.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.411+0000 7fd659a40740 10 client.27134 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:13:06.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.411+0000 7fd659a40740 10 client.27134 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-19T05:13:06.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.411+0000 7fd659a40740 20 client.27134 populate_metadata read hostname 'smithi043' 2024-09-19T05:13:06.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.411+0000 7fd659a40740 1 --2- 172.21.15.43:0/1193774537 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x564fc086ed90 0x564fc0871200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:06.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.411+0000 7fd659a40740 4 client.27134 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:13:06.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.411+0000 7fd659a40740 1 -- 172.21.15.43:0/1193774537 --> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fc07c7950 con 0x564fc086ed90 2024-09-19T05:13:06.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.412+0000 7fd64fe00640 1 --2- 172.21.15.43:0/1193774537 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x564fc086ed90 0x564fc0871200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:06.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.412+0000 7fd64fe00640 1 --2- 172.21.15.43:0/1193774537 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x564fc086ed90 0x564fc0871200 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1eb551aaa9347d95 server_cookie=65312ae4692ac1f2 in_seq=0 out_seq=0 2024-09-19T05:13:06.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.412+0000 7fd63e000640 10 client.27134 ms_handle_connect on v2:172.21.15.151:6836/3960378010 2024-09-19T05:13:06.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.413+0000 7fd63e000640 1 -- 172.21.15.43:0/1193774537 <== mds.4 v2:172.21.15.151:6836/3960378010 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564fc07c7950 con 0x564fc086ed90 2024-09-19T05:13:06.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.413+0000 7fd63e000640 10 client.27134 handle_command_reply: tid=0 2024-09-19T05:13:06.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.413+0000 7fd659a40740 1 client.27134 shutdown 2024-09-19T05:13:06.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.413+0000 7fd659a40740 1 -- 172.21.15.43:0/1193774537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fc0854590 msgr2=0x564fc085c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:06.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.413+0000 7fd659a40740 1 --2- 172.21.15.43:0/1193774537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fc0854590 0x564fc085c1f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd6380cd6e0 tx=0x7fd638009310 comp rx=0 tx=0).stop 2024-09-19T05:13:06.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.414+0000 7fd659a40740 1 -- 172.21.15.43:0/1193774537 shutdown_connections 2024-09-19T05:13:06.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.414+0000 7fd659a40740 1 --2- 172.21.15.43:0/1193774537 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x564fc086ed90 0x564fc0871200 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:06.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.414+0000 7fd659a40740 1 --2- 172.21.15.43:0/1193774537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564fc085ee50 0x564fc085c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:06.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.414+0000 7fd659a40740 1 --2- 172.21.15.43:0/1193774537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fc0854590 0x564fc085c1f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:06.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.414+0000 7fd659a40740 1 --2- 172.21.15.43:0/1193774537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc0853c70 0x564fc085bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:06.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.414+0000 7fd63e000640 0 client.27134 ms_handle_reset on v2:172.21.15.151:6836/3960378010 2024-09-19T05:13:06.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.414+0000 7fd659a40740 1 -- 172.21.15.43:0/1193774537 >> 172.21.15.43:0/1193774537 conn(0x564fc0850200 msgr2=0x564fc084f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:06.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.414+0000 7fd659a40740 1 -- 172.21.15.43:0/1193774537 shutdown_connections 2024-09-19T05:13:06.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.414+0000 7fd659a40740 1 -- 172.21.15.43:0/1193774537 wait complete. 2024-09-19T05:13:06.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.414+0000 7fd659a40740 20 client.27134 trim_cache size 0 max 16384 2024-09-19T05:13:06.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.428+0000 7fd659a40740 1 Processor -- start 2024-09-19T05:13:06.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.428+0000 7fd659a40740 1 -- start start 2024-09-19T05:13:06.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.428+0000 7fd659a40740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fc0853c70 0x564fc084f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:06.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.428+0000 7fd659a40740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc0854590 0x564fc0876ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:06.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.428+0000 7fd659a40740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564fc085ee50 0x564fc0877010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:06.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.428+0000 7fd659a40740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd6501b3f70 con 0x564fc0854590 2024-09-19T05:13:06.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.428+0000 7fd659a40740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd6501b5ab0 con 0x564fc0853c70 2024-09-19T05:13:06.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.428+0000 7fd659a40740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd6501b4640 con 0x564fc085ee50 2024-09-19T05:13:06.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.428+0000 7fd647e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc0854590 0x564fc0876ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:06.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.428+0000 7fd647e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc0854590 0x564fc0876ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40168/0 (socket says 172.21.15.43:40168) 2024-09-19T05:13:06.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.428+0000 7fd647e00640 1 -- 172.21.15.43:0/267318085 learned_addr learned my addr 172.21.15.43:0/267318085 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:06.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.429+0000 7fd64fe00640 1 --2- 172.21.15.43:0/267318085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fc0853c70 0x564fc084f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:06.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.429+0000 7fd654e00640 1 --2- 172.21.15.43:0/267318085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564fc085ee50 0x564fc0877010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:06.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.429+0000 7fd647e00640 1 -- 172.21.15.43:0/267318085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564fc085ee50 msgr2=0x564fc0877010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:06.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.429+0000 7fd647e00640 1 --2- 172.21.15.43:0/267318085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564fc085ee50 0x564fc0877010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:06.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.429+0000 7fd647e00640 1 -- 172.21.15.43:0/267318085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fc0853c70 msgr2=0x564fc084f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:06.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.429+0000 7fd647e00640 1 --2- 172.21.15.43:0/267318085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fc0853c70 0x564fc084f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:06.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.429+0000 7fd647e00640 1 -- 172.21.15.43:0/267318085 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fc084b770 con 0x564fc0854590 2024-09-19T05:13:06.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.429+0000 7fd647e00640 1 --2- 172.21.15.43:0/267318085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc0854590 0x564fc0876ad0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd63800c6a0 tx=0x7fd6380a0350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1941e25403de8f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:06.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.429+0000 7fd644c00640 1 -- 172.21.15.43:0/267318085 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6380a0620 con 0x564fc0854590 2024-09-19T05:13:06.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.429+0000 7fd644c00640 1 -- 172.21.15.43:0/267318085 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd63809c320 con 0x564fc0854590 2024-09-19T05:13:06.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.429+0000 7fd644c00640 1 -- 172.21.15.43:0/267318085 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63809ba20 con 0x564fc0854590 2024-09-19T05:13:06.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.430+0000 7fd659a40740 1 -- 172.21.15.43:0/267318085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc0854590 msgr2=0x564fc0876ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:06.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.430+0000 7fd659a40740 1 --2- 172.21.15.43:0/267318085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc0854590 0x564fc0876ad0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd63800c6a0 tx=0x7fd6380a0350 comp rx=0 tx=0).stop 2024-09-19T05:13:06.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.430+0000 7fd659a40740 1 -- 172.21.15.43:0/267318085 shutdown_connections 2024-09-19T05:13:06.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.430+0000 7fd659a40740 1 --2- 172.21.15.43:0/267318085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564fc085ee50 0x564fc0877010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:06.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.430+0000 7fd659a40740 1 --2- 172.21.15.43:0/267318085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc0854590 0x564fc0876ad0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:06.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.430+0000 7fd659a40740 1 --2- 172.21.15.43:0/267318085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fc0853c70 0x564fc084f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:06.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.430+0000 7fd659a40740 1 -- 172.21.15.43:0/267318085 >> 172.21.15.43:0/267318085 conn(0x564fc0850200 msgr2=0x564fc084eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:06.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.430+0000 7fd659a40740 1 -- 172.21.15.43:0/267318085 shutdown_connections 2024-09-19T05:13:06.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.430+0000 7fd659a40740 1 -- 172.21.15.43:0/267318085 wait complete. 2024-09-19T05:13:06.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.431+0000 7fd659a40740 1 -- start start 2024-09-19T05:13:06.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.431+0000 7fd659a40740 1 Processor -- start 2024-09-19T05:13:06.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.431+0000 7fd659a40740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564fc0853c70 0x564fc0862fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:06.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.431+0000 7fd659a40740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fc0854590 0x564fc0863520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:06.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.431+0000 7fd659a40740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc085ee50 0x564fc08d8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:06.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.431+0000 7fd659a40740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd6501b53e0 con 0x564fc085ee50 2024-09-19T05:13:06.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.431+0000 7fd659a40740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd6501b4d10 con 0x564fc0854590 2024-09-19T05:13:06.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.431+0000 7fd659a40740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd6501b1f70 con 0x564fc0853c70 2024-09-19T05:13:06.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.431+0000 7fd659a40740 10 client.0 _ll_register_callbacks cb 0x7fd6380d37a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:06.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.432+0000 7fd654e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc085ee50 0x564fc08d8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:06.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.432+0000 7fd64fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564fc0853c70 0x564fc0862fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:06.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.432+0000 7fd654e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc085ee50 0x564fc08d8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40172/0 (socket says 172.21.15.43:40172) 2024-09-19T05:13:06.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.432+0000 7fd64fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564fc0853c70 0x564fc0862fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:57374/0 (socket says 172.21.15.43:57374) 2024-09-19T05:13:06.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.432+0000 7fd654e00640 1 -- 172.21.15.43:0/4277626896 learned_addr learned my addr 172.21.15.43:0/4277626896 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:06.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.432+0000 7fd647e00640 1 --2- 172.21.15.43:0/4277626896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fc0854590 0x564fc0863520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:06.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.432+0000 7fd654e00640 1 -- 172.21.15.43:0/4277626896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564fc0853c70 msgr2=0x564fc0862fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:06.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.432+0000 7fd654e00640 1 --2- 172.21.15.43:0/4277626896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564fc0853c70 0x564fc0862fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:06.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.432+0000 7fd654e00640 1 -- 172.21.15.43:0/4277626896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fc0854590 msgr2=0x564fc0863520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:06.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.432+0000 7fd654e00640 1 --2- 172.21.15.43:0/4277626896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fc0854590 0x564fc0863520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:06.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.432+0000 7fd654e00640 1 -- 172.21.15.43:0/4277626896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fc0856ae0 con 0x564fc085ee50 2024-09-19T05:13:06.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.432+0000 7fd647e00640 1 --2- 172.21.15.43:0/4277626896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fc0854590 0x564fc0863520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:13:06.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.433+0000 7fd654e00640 1 --2- 172.21.15.43:0/4277626896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc085ee50 0x564fc08d8aa0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd6480040c0 tx=0x7fd648004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd7f3eac03dd84ff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:06.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.433+0000 7fd63e000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:13:06.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.433+0000 7fd63e000640 1 -- 172.21.15.43:0/4277626896 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd648004b20 con 0x564fc085ee50 2024-09-19T05:13:06.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.433+0000 7fd659a40740 1 -- 172.21.15.43:0/4277626896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd63809c320 con 0x564fc085ee50 2024-09-19T05:13:06.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.433+0000 7fd63e000640 1 -- 172.21.15.43:0/4277626896 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7fd648004cc0 con 0x564fc085ee50 2024-09-19T05:13:06.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.433+0000 7fd63e000640 1 -- 172.21.15.43:0/4277626896 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd64800f910 con 0x564fc085ee50 2024-09-19T05:13:06.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.433+0000 7fd659a40740 10 client.17118 fetch_fsmap learned FSMap version 29 2024-09-19T05:13:06.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.433+0000 7fd659a40740 1 -- 172.21.15.43:0/4277626896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564fc08da050 con 0x564fc085ee50 2024-09-19T05:13:06.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.433+0000 7fd63e000640 1 -- 172.21.15.43:0/4277626896 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd63809c320 con 0x564fc085ee50 2024-09-19T05:13:06.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.434+0000 7fd63e000640 1 -- 172.21.15.43:0/4277626896 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd648005820 con 0x564fc085ee50 2024-09-19T05:13:06.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.434+0000 7fd659a40740 10 client.17118 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:13:06.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.434+0000 7fd659a40740 10 client.17118 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-19T05:13:06.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.434+0000 7fd659a40740 20 client.17118 populate_metadata read hostname 'smithi043' 2024-09-19T05:13:06.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.434+0000 7fd659a40740 1 --2- 172.21.15.43:0/4277626896 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x564fc08da820 0x564fc08dcc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:06.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.434+0000 7fd659a40740 4 client.17118 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:13:06.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.434+0000 7fd659a40740 1 -- 172.21.15.43:0/4277626896 --> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564fc07c1e60 con 0x564fc08da820 2024-09-19T05:13:06.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.434+0000 7fd64fe00640 1 --2- 172.21.15.43:0/4277626896 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x564fc08da820 0x564fc08dcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:06.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.435+0000 7fd64fe00640 1 --2- 172.21.15.43:0/4277626896 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x564fc08da820 0x564fc08dcc40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=12b8b49bedd05df3 server_cookie=aeaa7b383a58068a in_seq=0 out_seq=0 2024-09-19T05:13:06.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.435+0000 7fd63e000640 10 client.17118 ms_handle_connect on v2:172.21.15.151:6836/3960378010 2024-09-19T05:13:06.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.435+0000 7fd63e000640 1 -- 172.21.15.43:0/4277626896 <== mds.4 v2:172.21.15.151:6836/3960378010 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564fc07c1e60 con 0x564fc08da820 2024-09-19T05:13:06.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.435+0000 7fd63e000640 10 client.17118 handle_command_reply: tid=0 2024-09-19T05:13:06.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.435+0000 7fd659a40740 1 client.17118 shutdown 2024-09-19T05:13:06.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.435+0000 7fd659a40740 1 -- 172.21.15.43:0/4277626896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc085ee50 msgr2=0x564fc08d8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:06.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.435+0000 7fd659a40740 1 --2- 172.21.15.43:0/4277626896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc085ee50 0x564fc08d8aa0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd6480040c0 tx=0x7fd648004a40 comp rx=0 tx=0).stop 2024-09-19T05:13:06.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.435+0000 7fd659a40740 1 -- 172.21.15.43:0/4277626896 shutdown_connections 2024-09-19T05:13:06.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.435+0000 7fd659a40740 1 --2- 172.21.15.43:0/4277626896 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x564fc08da820 0x564fc08dcc40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:06.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.435+0000 7fd659a40740 1 --2- 172.21.15.43:0/4277626896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564fc085ee50 0x564fc08d8aa0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:06.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.435+0000 7fd659a40740 1 --2- 172.21.15.43:0/4277626896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fc0854590 0x564fc0863520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:06.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.435+0000 7fd659a40740 1 --2- 172.21.15.43:0/4277626896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564fc0853c70 0x564fc0862fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:06.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.435+0000 7fd659a40740 1 -- 172.21.15.43:0/4277626896 >> 172.21.15.43:0/4277626896 conn(0x564fc0850200 msgr2=0x564fc0855ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:06.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.435+0000 7fd63e000640 0 client.17118 ms_handle_reset on v2:172.21.15.151:6836/3960378010 2024-09-19T05:13:06.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.435+0000 7fd659a40740 1 -- 172.21.15.43:0/4277626896 shutdown_connections 2024-09-19T05:13:06.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.435+0000 7fd659a40740 1 -- 172.21.15.43:0/4277626896 wait complete. 2024-09-19T05:13:06.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.435+0000 7fd659a40740 20 client.17118 trim_cache size 0 max 16384 2024-09-19T05:13:06.441 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:13:06.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.436+0000 7fd656e00640 1 -- 172.21.15.43:0/3217627302 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fd6340a3430 msgr2=0x7fd6340a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:06.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.436+0000 7fd656e00640 1 --2- 172.21.15.43:0/3217627302 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fd6340a3430 0x7fd6340a58f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd65006b460 tx=0x7fd648009500 comp rx=0 tx=0).stop 2024-09-19T05:13:06.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.436+0000 7fd656e00640 1 -- 172.21.15.43:0/3217627302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd650077040 msgr2=0x7fd6501a4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:06.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.436+0000 7fd656e00640 1 --2- 172.21.15.43:0/3217627302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd650077040 0x7fd6501a4c50 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd63800e9f0 tx=0x7fd63800eeb0 comp rx=0 tx=0).stop 2024-09-19T05:13:06.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.436+0000 7fd656e00640 1 -- 172.21.15.43:0/3217627302 shutdown_connections 2024-09-19T05:13:06.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.436+0000 7fd656e00640 1 --2- 172.21.15.43:0/3217627302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd650112a30 0x7fd65019e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:06.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.436+0000 7fd656e00640 1 --2- 172.21.15.43:0/3217627302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd650077040 0x7fd6501a4c50 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:06.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.436+0000 7fd656e00640 1 --2- 172.21.15.43:0/3217627302 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fd6340a3430 0x7fd6340a58f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:06.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.436+0000 7fd656e00640 1 --2- 172.21.15.43:0/3217627302 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd650076720 0x7fd6501a4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:06.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.436+0000 7fd656e00640 1 -- 172.21.15.43:0/3217627302 >> 172.21.15.43:0/3217627302 conn(0x7fd6500ffa90 msgr2=0x7fd650101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:06.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.436+0000 7fd656e00640 1 -- 172.21.15.43:0/3217627302 shutdown_connections 2024-09-19T05:13:06.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:06.436+0000 7fd656e00640 1 -- 172.21.15.43:0/3217627302 wait complete. 2024-09-19T05:13:06.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:06 smithi096 ceph-mon[35085]: pgmap v260: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-09-19T05:13:06.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:06 smithi151 ceph-mon[35425]: pgmap v260: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-09-19T05:13:07.194 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T05:13:07.573 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:13:08.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.072+0000 7f1e83e00640 1 -- 172.21.15.43:0/841010955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e84108f60 msgr2=0x7f1e84110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:08.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.072+0000 7f1e83e00640 1 --2- 172.21.15.43:0/841010955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e84108f60 0x7f1e84110ae0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c009ca0 tx=0x7f1e7c02a8c0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.072+0000 7f1e83e00640 1 -- 172.21.15.43:0/841010955 shutdown_connections 2024-09-19T05:13:08.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.072+0000 7f1e83e00640 1 --2- 172.21.15.43:0/841010955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e84108f60 0x7f1e84110ae0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.072+0000 7f1e83e00640 1 --2- 172.21.15.43:0/841010955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e841047e0 0x7f1e84108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.072+0000 7f1e83e00640 1 --2- 172.21.15.43:0/841010955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e84103e30 0x7f1e84104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.072+0000 7f1e83e00640 1 -- 172.21.15.43:0/841010955 >> 172.21.15.43:0/841010955 conn(0x7f1e840ff8d0 msgr2=0x7f1e84101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:08.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.072+0000 7f1e83e00640 1 -- 172.21.15.43:0/841010955 shutdown_connections 2024-09-19T05:13:08.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.072+0000 7f1e83e00640 1 -- 172.21.15.43:0/841010955 wait complete. 2024-09-19T05:13:08.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.073+0000 7f1e83e00640 1 Processor -- start 2024-09-19T05:13:08.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.073+0000 7f1e83e00640 1 -- start start 2024-09-19T05:13:08.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.074+0000 7f1e83e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e84103e30 0x7f1e8419a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:08.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.074+0000 7f1e83e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e841047e0 0x7f1e8419a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:08.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.074+0000 7f1e83e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e84108f60 0x7f1e8419eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:08.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.074+0000 7f1e83e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1e84114010 con 0x7f1e84108f60 2024-09-19T05:13:08.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.074+0000 7f1e83e00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1e84113e90 con 0x7f1e84103e30 2024-09-19T05:13:08.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.074+0000 7f1e83e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1e84114190 con 0x7f1e841047e0 2024-09-19T05:13:08.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.074+0000 7f1e83400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e84108f60 0x7f1e8419eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:08.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.074+0000 7f1e82a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e84103e30 0x7f1e8419a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:08.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.074+0000 7f1e7a000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e841047e0 0x7f1e8419a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:08.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.074+0000 7f1e83400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e84108f60 0x7f1e8419eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40192/0 (socket says 172.21.15.43:40192) 2024-09-19T05:13:08.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.074+0000 7f1e83400640 1 -- 172.21.15.43:0/1720237042 learned_addr learned my addr 172.21.15.43:0/1720237042 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:08.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.074+0000 7f1e82a00640 1 -- 172.21.15.43:0/1720237042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e841047e0 msgr2=0x7f1e8419a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:08.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.074+0000 7f1e82a00640 1 --2- 172.21.15.43:0/1720237042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e841047e0 0x7f1e8419a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.074+0000 7f1e82a00640 1 -- 172.21.15.43:0/1720237042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e84108f60 msgr2=0x7f1e8419eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:08.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.074+0000 7f1e82a00640 1 --2- 172.21.15.43:0/1720237042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e84108f60 0x7f1e8419eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.074+0000 7f1e82a00640 1 -- 172.21.15.43:0/1720237042 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e8419f380 con 0x7f1e84103e30 2024-09-19T05:13:08.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.074+0000 7f1e7a000640 1 --2- 172.21.15.43:0/1720237042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e841047e0 0x7f1e8419a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:13:08.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.074+0000 7f1e83400640 1 --2- 172.21.15.43:0/1720237042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e84108f60 0x7f1e8419eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:13:08.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.075+0000 7f1e82a00640 1 --2- 172.21.15.43:0/1720237042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e84103e30 0x7f1e8419a3d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1e7400e9f0 tx=0x7f1e7400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:08.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.075+0000 7f1e88a00640 1 -- 172.21.15.43:0/1720237042 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7400ccc0 con 0x7f1e84103e30 2024-09-19T05:13:08.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.075+0000 7f1e88a00640 1 -- 172.21.15.43:0/1720237042 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e74010910 con 0x7f1e84103e30 2024-09-19T05:13:08.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.075+0000 7f1e83e00640 1 -- 172.21.15.43:0/1720237042 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e8419f670 con 0x7f1e84103e30 2024-09-19T05:13:08.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.075+0000 7f1e88a00640 1 -- 172.21.15.43:0/1720237042 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e74018960 con 0x7f1e84103e30 2024-09-19T05:13:08.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.075+0000 7f1e83e00640 1 -- 172.21.15.43:0/1720237042 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e841a6f50 con 0x7f1e84103e30 2024-09-19T05:13:08.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.078+0000 7f1e8ca31740 1 Processor -- start 2024-09-19T05:13:08.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.078+0000 7f1e88a00640 1 -- 172.21.15.43:0/1720237042 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f1e74010ac0 con 0x7f1e84103e30 2024-09-19T05:13:08.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.078+0000 7f1e8ca31740 1 -- start start 2024-09-19T05:13:08.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.079+0000 7f1e8ca31740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55590b9a3e40 0x55590b9a4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:08.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.079+0000 7f1e8ca31740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590b9a4760 0x55590b9aeae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:08.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.079+0000 7f1e8ca31740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55590b9af020 0x55590b9b1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:08.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.079+0000 7f1e8ca31740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55590b90a3e0 con 0x55590b9a3e40 2024-09-19T05:13:08.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.079+0000 7f1e8ca31740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55590b8b1d80 con 0x55590b9af020 2024-09-19T05:13:08.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.079+0000 7f1e8ca31740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55590b9063a0 con 0x55590b9a4760 2024-09-19T05:13:08.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.079+0000 7f1e83400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55590b9a3e40 0x55590b9a4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:08.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.079+0000 7f1e82a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590b9a4760 0x55590b9aeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:08.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.079+0000 7f1e83400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55590b9a3e40 0x55590b9a4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40200/0 (socket says 172.21.15.43:40200) 2024-09-19T05:13:08.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.079+0000 7f1e83400640 1 -- 172.21.15.43:0/2715472759 learned_addr learned my addr 172.21.15.43:0/2715472759 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:08.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.079+0000 7f1e7a000640 1 --2- 172.21.15.43:0/2715472759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55590b9af020 0x55590b9b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:08.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.079+0000 7f1e83400640 1 -- 172.21.15.43:0/2715472759 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590b9a4760 msgr2=0x55590b9aeae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:08.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.079+0000 7f1e83400640 1 --2- 172.21.15.43:0/2715472759 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590b9a4760 0x55590b9aeae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.079+0000 7f1e83400640 1 -- 172.21.15.43:0/2715472759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55590b9af020 msgr2=0x55590b9b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:08.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.079+0000 7f1e83400640 1 --2- 172.21.15.43:0/2715472759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55590b9af020 0x55590b9b1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.079+0000 7f1e83400640 1 -- 172.21.15.43:0/2715472759 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55590b8d0ae0 con 0x55590b9a3e40 2024-09-19T05:13:08.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.080+0000 7f1e83400640 1 --2- 172.21.15.43:0/2715472759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55590b9a3e40 0x55590b9a4220 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c009a10 tx=0x7f1e7c005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=663fe11d3eaa9a82 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:08.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.080+0000 7f1e73e00640 1 -- 172.21.15.43:0/2715472759 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c003b70 con 0x55590b9a3e40 2024-09-19T05:13:08.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.080+0000 7f1e73e00640 1 -- 172.21.15.43:0/2715472759 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e7c036040 con 0x55590b9a3e40 2024-09-19T05:13:08.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.080+0000 7f1e73e00640 1 -- 172.21.15.43:0/2715472759 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c004640 con 0x55590b9a3e40 2024-09-19T05:13:08.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.080+0000 7f1e88a00640 1 --2- 172.21.15.43:0/1720237042 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f1e680a3430 0x7f1e680a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:08.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.081+0000 7f1e88a00640 1 -- 172.21.15.43:0/1720237042 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f1e740d00a0 con 0x7f1e84103e30 2024-09-19T05:13:08.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.081+0000 7f1e82a00640 1 --2- 172.21.15.43:0/1720237042 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f1e680a3430 0x7f1e680a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:08.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.081+0000 7f1e8ca31740 1 -- 172.21.15.43:0/2715472759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55590b9a3e40 msgr2=0x55590b9a4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:08.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.081+0000 7f1e8ca31740 1 --2- 172.21.15.43:0/2715472759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55590b9a3e40 0x55590b9a4220 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c009a10 tx=0x7f1e7c005ef0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.081+0000 7f1e8ca31740 1 -- 172.21.15.43:0/2715472759 shutdown_connections 2024-09-19T05:13:08.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.081+0000 7f1e8ca31740 1 --2- 172.21.15.43:0/2715472759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55590b9af020 0x55590b9b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.081+0000 7f1e8ca31740 1 --2- 172.21.15.43:0/2715472759 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590b9a4760 0x55590b9aeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.081+0000 7f1e8ca31740 1 --2- 172.21.15.43:0/2715472759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55590b9a3e40 0x55590b9a4220 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.081+0000 7f1e8ca31740 1 -- 172.21.15.43:0/2715472759 >> 172.21.15.43:0/2715472759 conn(0x55590b9a03d0 msgr2=0x55590b9a07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:08.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.081+0000 7f1e8ca31740 1 -- 172.21.15.43:0/2715472759 shutdown_connections 2024-09-19T05:13:08.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.081+0000 7f1e8ca31740 1 -- 172.21.15.43:0/2715472759 wait complete. 2024-09-19T05:13:08.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.081+0000 7f1e82a00640 1 --2- 172.21.15.43:0/1720237042 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f1e680a3430 0x7f1e680a58f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x55590b9a1230 tx=0x7f1e740d3b50 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:08.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.082+0000 7f1e8ca31740 1 -- start start 2024-09-19T05:13:08.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.082+0000 7f1e8ca31740 1 Processor -- start 2024-09-19T05:13:08.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.082+0000 7f1e8ca31740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55590b9a3e40 0x55590b9abe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:08.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.082+0000 7f1e8ca31740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590b9a4760 0x55590b9ac3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:08.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.082+0000 7f1e8ca31740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55590b9af020 0x55590b9ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:08.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.082+0000 7f1e8ca31740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55590b9b1a70 con 0x55590b9af020 2024-09-19T05:13:08.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.082+0000 7f1e8ca31740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55590b8c50f0 con 0x55590b9a3e40 2024-09-19T05:13:08.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.082+0000 7f1e8ca31740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55590b9b1bf0 con 0x55590b9a4760 2024-09-19T05:13:08.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.082+0000 7f1e8ca31740 10 client.0 _ll_register_callbacks cb 0x55590b8acbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:08.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.083+0000 7f1e83400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55590b9a3e40 0x55590b9abe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:08.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.083+0000 7f1e7a000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590b9a4760 0x55590b9ac3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:08.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.083+0000 7f1e83400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55590b9a3e40 0x55590b9abe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:52580/0 (socket says 172.21.15.43:52580) 2024-09-19T05:13:08.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.083+0000 7f1e83400640 1 -- 172.21.15.43:0/950044372 learned_addr learned my addr 172.21.15.43:0/950044372 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:08.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.083+0000 7f1e83400640 1 --2- 172.21.15.43:0/950044372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55590b9af020 0x55590b9ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:08.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.083+0000 7f1e83400640 1 -- 172.21.15.43:0/950044372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590b9a4760 msgr2=0x55590b9ac3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:08.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.083+0000 7f1e83400640 1 --2- 172.21.15.43:0/950044372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590b9a4760 0x55590b9ac3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.083+0000 7f1e83400640 1 -- 172.21.15.43:0/950044372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55590b9a3e40 msgr2=0x55590b9abe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:08.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.083+0000 7f1e83400640 1 --2- 172.21.15.43:0/950044372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55590b9a3e40 0x55590b9abe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.083+0000 7f1e83400640 1 -- 172.21.15.43:0/950044372 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55590b67b3c0 con 0x55590b9af020 2024-09-19T05:13:08.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.084+0000 7f1e83400640 1 --2- 172.21.15.43:0/950044372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55590b9af020 0x55590b9ac8e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c003e30 tx=0x7f1e7c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c86862005855b20e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:08.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.084+0000 7f1e70c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:13:08.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.084+0000 7f1e70c00640 1 -- 172.21.15.43:0/950044372 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c059060 con 0x55590b9af020 2024-09-19T05:13:08.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.084+0000 7f1e70c00640 1 -- 172.21.15.43:0/950044372 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f1e7c0046d0 con 0x55590b9af020 2024-09-19T05:13:08.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.084+0000 7f1e8ca31740 1 -- 172.21.15.43:0/950044372 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e7c036040 con 0x55590b9af020 2024-09-19T05:13:08.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.084+0000 7f1e70c00640 1 -- 172.21.15.43:0/950044372 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1e7c031450 con 0x55590b9af020 2024-09-19T05:13:08.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.084+0000 7f1e8ca31740 10 client.17130 fetch_fsmap learned FSMap version 29 2024-09-19T05:13:08.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.084+0000 7f1e8ca31740 1 -- 172.21.15.43:0/950044372 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55590b9b1a70 con 0x55590b9af020 2024-09-19T05:13:08.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.085+0000 7f1e70c00640 1 -- 172.21.15.43:0/950044372 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e7c036040 con 0x55590b9af020 2024-09-19T05:13:08.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.085+0000 7f1e70c00640 1 -- 172.21.15.43:0/950044372 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1e7c051040 con 0x55590b9af020 2024-09-19T05:13:08.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.085+0000 7f1e8ca31740 10 client.17130 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:13:08.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.085+0000 7f1e8ca31740 10 client.17130 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T05:13:08.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.085+0000 7f1e8ca31740 20 client.17130 populate_metadata read hostname 'smithi043' 2024-09-19T05:13:08.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.085+0000 7f1e8ca31740 1 --2- 172.21.15.43:0/950044372 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55590b9bebb0 0x55590b9c0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:08.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.085+0000 7f1e8ca31740 4 client.17130 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:13:08.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.085+0000 7f1e8ca31740 1 -- 172.21.15.43:0/950044372 --> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55590b917950 con 0x55590b9bebb0 2024-09-19T05:13:08.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.085+0000 7f1e7a000640 1 --2- 172.21.15.43:0/950044372 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55590b9bebb0 0x55590b9c0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:08.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.086+0000 7f1e7a000640 1 --2- 172.21.15.43:0/950044372 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55590b9bebb0 0x55590b9c0fd0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c1675682c9272018 server_cookie=3b64fed2d9986237 in_seq=0 out_seq=0 2024-09-19T05:13:08.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.086+0000 7f1e70c00640 10 client.17130 ms_handle_connect on v2:172.21.15.96:6836/3978388286 2024-09-19T05:13:08.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.087+0000 7f1e70c00640 1 -- 172.21.15.43:0/950044372 <== mds.3 v2:172.21.15.96:6836/3978388286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55590b917950 con 0x55590b9bebb0 2024-09-19T05:13:08.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.087+0000 7f1e70c00640 10 client.17130 handle_command_reply: tid=0 2024-09-19T05:13:08.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.087+0000 7f1e8ca31740 1 client.17130 shutdown 2024-09-19T05:13:08.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.087+0000 7f1e8ca31740 1 -- 172.21.15.43:0/950044372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55590b9af020 msgr2=0x55590b9ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:08.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.087+0000 7f1e8ca31740 1 --2- 172.21.15.43:0/950044372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55590b9af020 0x55590b9ac8e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c003e30 tx=0x7f1e7c02a8c0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.087+0000 7f1e8ca31740 1 -- 172.21.15.43:0/950044372 shutdown_connections 2024-09-19T05:13:08.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.087+0000 7f1e8ca31740 1 --2- 172.21.15.43:0/950044372 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55590b9bebb0 0x55590b9c0fd0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.087+0000 7f1e8ca31740 1 --2- 172.21.15.43:0/950044372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55590b9af020 0x55590b9ac8e0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.087+0000 7f1e8ca31740 1 --2- 172.21.15.43:0/950044372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590b9a4760 0x55590b9ac3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.087+0000 7f1e8ca31740 1 --2- 172.21.15.43:0/950044372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55590b9a3e40 0x55590b9abe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.087+0000 7f1e70c00640 0 client.17130 ms_handle_reset on v2:172.21.15.96:6836/3978388286 2024-09-19T05:13:08.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.087+0000 7f1e8ca31740 1 -- 172.21.15.43:0/950044372 >> 172.21.15.43:0/950044372 conn(0x55590b9a03d0 msgr2=0x55590b9a60b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:08.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.087+0000 7f1e8ca31740 1 -- 172.21.15.43:0/950044372 shutdown_connections 2024-09-19T05:13:08.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.087+0000 7f1e8ca31740 1 -- 172.21.15.43:0/950044372 wait complete. 2024-09-19T05:13:08.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.087+0000 7f1e8ca31740 20 client.17130 trim_cache size 0 max 16384 2024-09-19T05:13:08.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.101+0000 7f1e8ca31740 1 Processor -- start 2024-09-19T05:13:08.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.101+0000 7f1e8ca31740 1 -- start start 2024-09-19T05:13:08.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.101+0000 7f1e8ca31740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590b9a3e40 0x55590b9ae190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:08.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.101+0000 7f1e8ca31740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55590b9a4760 0x55590b9c6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:08.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.101+0000 7f1e8ca31740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55590b9af020 0x55590b9c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:08.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.101+0000 7f1e8ca31740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1e841af860 con 0x55590b9a4760 2024-09-19T05:13:08.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.101+0000 7f1e8ca31740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1e841b13a0 con 0x55590b9af020 2024-09-19T05:13:08.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.101+0000 7f1e8ca31740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1e841aff30 con 0x55590b9a3e40 2024-09-19T05:13:08.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.102+0000 7f1e7a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55590b9a4760 0x55590b9c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:08.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.102+0000 7f1e83400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590b9a3e40 0x55590b9ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:08.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.102+0000 7f1e83400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590b9a3e40 0x55590b9ae190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:57424/0 (socket says 172.21.15.43:57424) 2024-09-19T05:13:08.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.102+0000 7f1e83400640 1 -- 172.21.15.43:0/1058665339 learned_addr learned my addr 172.21.15.43:0/1058665339 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:08.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.102+0000 7f1e7a000640 1 --2- 172.21.15.43:0/1058665339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55590b9a4760 0x55590b9c6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40216/0 (socket says 172.21.15.43:40216) 2024-09-19T05:13:08.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.102+0000 7f1e83400640 1 --2- 172.21.15.43:0/1058665339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55590b9af020 0x55590b9c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:08.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.102+0000 7f1e83400640 1 -- 172.21.15.43:0/1058665339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55590b9af020 msgr2=0x55590b9c6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:08.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.102+0000 7f1e83400640 1 --2- 172.21.15.43:0/1058665339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55590b9af020 0x55590b9c6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.102+0000 7f1e83400640 1 -- 172.21.15.43:0/1058665339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55590b9a4760 msgr2=0x55590b9c6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:08.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.102+0000 7f1e83400640 1 --2- 172.21.15.43:0/1058665339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55590b9a4760 0x55590b9c6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.102+0000 7f1e83400640 1 -- 172.21.15.43:0/1058665339 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55590b9a0200 con 0x55590b9a3e40 2024-09-19T05:13:08.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.102+0000 7f1e7a000640 1 --2- 172.21.15.43:0/1058665339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55590b9a4760 0x55590b9c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:13:08.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.102+0000 7f1e83400640 1 --2- 172.21.15.43:0/1058665339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590b9a3e40 0x55590b9ae190 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c002230 tx=0x7f1e7c0095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de32744d8c1cfcf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:08.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.103+0000 7f1e73400640 1 -- 172.21.15.43:0/1058665339 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c059070 con 0x55590b9a3e40 2024-09-19T05:13:08.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.103+0000 7f1e73400640 1 -- 172.21.15.43:0/1058665339 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e7c036070 con 0x55590b9a3e40 2024-09-19T05:13:08.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.103+0000 7f1e73400640 1 -- 172.21.15.43:0/1058665339 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c02f610 con 0x55590b9a3e40 2024-09-19T05:13:08.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.103+0000 7f1e8ca31740 1 -- 172.21.15.43:0/1058665339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590b9a3e40 msgr2=0x55590b9ae190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:08.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.103+0000 7f1e8ca31740 1 --2- 172.21.15.43:0/1058665339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590b9a3e40 0x55590b9ae190 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c002230 tx=0x7f1e7c0095c0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.103+0000 7f1e8ca31740 1 -- 172.21.15.43:0/1058665339 shutdown_connections 2024-09-19T05:13:08.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.103+0000 7f1e8ca31740 1 --2- 172.21.15.43:0/1058665339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55590b9af020 0x55590b9c6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.103+0000 7f1e8ca31740 1 --2- 172.21.15.43:0/1058665339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55590b9a4760 0x55590b9c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.103+0000 7f1e8ca31740 1 --2- 172.21.15.43:0/1058665339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590b9a3e40 0x55590b9ae190 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.103+0000 7f1e8ca31740 1 -- 172.21.15.43:0/1058665339 >> 172.21.15.43:0/1058665339 conn(0x55590b9a03d0 msgr2=0x55590b9aae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:08.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.103+0000 7f1e8ca31740 1 -- 172.21.15.43:0/1058665339 shutdown_connections 2024-09-19T05:13:08.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.104+0000 7f1e8ca31740 1 -- 172.21.15.43:0/1058665339 wait complete. 2024-09-19T05:13:08.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.104+0000 7f1e8ca31740 1 -- start start 2024-09-19T05:13:08.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.104+0000 7f1e8ca31740 1 Processor -- start 2024-09-19T05:13:08.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.105+0000 7f1e8ca31740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590b9a3e40 0x55590b9b2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:08.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.105+0000 7f1e8ca31740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55590b9a4760 0x55590b9b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:08.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.105+0000 7f1e8ca31740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55590b9af020 0x55590b9b36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:08.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.105+0000 7f1e8ca31740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1e841b0cd0 con 0x55590b9a4760 2024-09-19T05:13:08.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.105+0000 7f1e8ca31740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1e841b0600 con 0x55590b9af020 2024-09-19T05:13:08.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.105+0000 7f1e8ca31740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1e841ad860 con 0x55590b9a3e40 2024-09-19T05:13:08.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.105+0000 7f1e8ca31740 10 client.0 _ll_register_callbacks cb 0x7f1e740d3a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:08.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.105+0000 7f1e83400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590b9a3e40 0x55590b9b2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:08.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.105+0000 7f1e83400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590b9a3e40 0x55590b9b2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:57428/0 (socket says 172.21.15.43:57428) 2024-09-19T05:13:08.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.105+0000 7f1e83400640 1 -- 172.21.15.43:0/2706152675 learned_addr learned my addr 172.21.15.43:0/2706152675 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:08.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.105+0000 7f1e7a000640 1 --2- 172.21.15.43:0/2706152675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55590b9a4760 0x55590b9b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:08.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.105+0000 7f1e83400640 1 --2- 172.21.15.43:0/2706152675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55590b9af020 0x55590b9b36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:08.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.105+0000 7f1e83400640 1 -- 172.21.15.43:0/2706152675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55590b9af020 msgr2=0x55590b9b36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:08.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.105+0000 7f1e83400640 1 --2- 172.21.15.43:0/2706152675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55590b9af020 0x55590b9b36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.105+0000 7f1e83400640 1 -- 172.21.15.43:0/2706152675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55590b9a4760 msgr2=0x55590b9b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:08.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.105+0000 7f1e83400640 1 --2- 172.21.15.43:0/2706152675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55590b9a4760 0x55590b9b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.105+0000 7f1e83400640 1 -- 172.21.15.43:0/2706152675 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55590b9a1060 con 0x55590b9a3e40 2024-09-19T05:13:08.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.105+0000 7f1e7a000640 1 --2- 172.21.15.43:0/2706152675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55590b9a4760 0x55590b9b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:13:08.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.106+0000 7f1e83400640 1 --2- 172.21.15.43:0/2706152675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590b9a3e40 0x55590b9b2c60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c02ff60 tx=0x7f1e7c048040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd127e6abe0c6d20 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:08.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.106+0000 7f1e70c00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:13:08.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.106+0000 7f1e70c00640 1 -- 172.21.15.43:0/2706152675 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c06b070 con 0x55590b9a3e40 2024-09-19T05:13:08.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.106+0000 7f1e8ca31740 1 -- 172.21.15.43:0/2706152675 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e7c036070 con 0x55590b9a3e40 2024-09-19T05:13:08.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.106+0000 7f1e70c00640 1 -- 172.21.15.43:0/2706152675 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f1e7c0317f0 con 0x55590b9a3e40 2024-09-19T05:13:08.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.106+0000 7f1e70c00640 1 -- 172.21.15.43:0/2706152675 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1e7c040070 con 0x55590b9a3e40 2024-09-19T05:13:08.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.106+0000 7f1e8ca31740 10 client.27064 fetch_fsmap learned FSMap version 29 2024-09-19T05:13:08.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.106+0000 7f1e8ca31740 1 -- 172.21.15.43:0/2706152675 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55590ba29990 con 0x55590b9a3e40 2024-09-19T05:13:08.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.107+0000 7f1e70c00640 1 -- 172.21.15.43:0/2706152675 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e7c036070 con 0x55590b9a3e40 2024-09-19T05:13:08.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.107+0000 7f1e70c00640 1 -- 172.21.15.43:0/2706152675 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1e7c044070 con 0x55590b9a3e40 2024-09-19T05:13:08.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.107+0000 7f1e8ca31740 10 client.27064 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:13:08.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.107+0000 7f1e8ca31740 10 client.27064 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T05:13:08.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.107+0000 7f1e8ca31740 20 client.27064 populate_metadata read hostname 'smithi043' 2024-09-19T05:13:08.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.107+0000 7f1e8ca31740 1 --2- 172.21.15.43:0/2706152675 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55590ba29e70 0x55590ba2c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:08.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.107+0000 7f1e8ca31740 4 client.27064 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:13:08.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.107+0000 7f1e8ca31740 1 -- 172.21.15.43:0/2706152675 --> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55590b911e60 con 0x55590ba29e70 2024-09-19T05:13:08.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.107+0000 7f1e7a000640 1 --2- 172.21.15.43:0/2706152675 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55590ba29e70 0x55590ba2c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:08.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.108+0000 7f1e7a000640 1 --2- 172.21.15.43:0/2706152675 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55590ba29e70 0x55590ba2c260 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=da0c87e0f12ada31 server_cookie=4a3f6b696a4d5bc1 in_seq=0 out_seq=0 2024-09-19T05:13:08.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.108+0000 7f1e70c00640 10 client.27064 ms_handle_connect on v2:172.21.15.96:6836/3978388286 2024-09-19T05:13:08.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.108+0000 7f1e70c00640 1 -- 172.21.15.43:0/2706152675 <== mds.3 v2:172.21.15.96:6836/3978388286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55590b911e60 con 0x55590ba29e70 2024-09-19T05:13:08.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.108+0000 7f1e70c00640 10 client.27064 handle_command_reply: tid=0 2024-09-19T05:13:08.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.108+0000 7f1e8ca31740 1 client.27064 shutdown 2024-09-19T05:13:08.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.108+0000 7f1e8ca31740 1 -- 172.21.15.43:0/2706152675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590b9a3e40 msgr2=0x55590b9b2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:08.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.108+0000 7f1e8ca31740 1 --2- 172.21.15.43:0/2706152675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590b9a3e40 0x55590b9b2c60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c02ff60 tx=0x7f1e7c048040 comp rx=0 tx=0).stop 2024-09-19T05:13:08.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.108+0000 7f1e8ca31740 1 -- 172.21.15.43:0/2706152675 shutdown_connections 2024-09-19T05:13:08.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.108+0000 7f1e8ca31740 1 --2- 172.21.15.43:0/2706152675 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55590ba29e70 0x55590ba2c260 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.108+0000 7f1e8ca31740 1 --2- 172.21.15.43:0/2706152675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55590b9af020 0x55590b9b36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.108+0000 7f1e8ca31740 1 --2- 172.21.15.43:0/2706152675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55590b9a4760 0x55590b9b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.108+0000 7f1e8ca31740 1 --2- 172.21.15.43:0/2706152675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590b9a3e40 0x55590b9b2c60 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.108+0000 7f1e8ca31740 1 -- 172.21.15.43:0/2706152675 >> 172.21.15.43:0/2706152675 conn(0x55590b9a03d0 msgr2=0x55590b9abe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:08.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.108+0000 7f1e70c00640 0 client.27064 ms_handle_reset on v2:172.21.15.96:6836/3978388286 2024-09-19T05:13:08.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.108+0000 7f1e8ca31740 1 -- 172.21.15.43:0/2706152675 shutdown_connections 2024-09-19T05:13:08.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.109+0000 7f1e8ca31740 1 -- 172.21.15.43:0/2706152675 wait complete. 2024-09-19T05:13:08.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.109+0000 7f1e8ca31740 20 client.27064 trim_cache size 0 max 16384 2024-09-19T05:13:08.114 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:13:08.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.109+0000 7f1e83e00640 1 -- 172.21.15.43:0/1720237042 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f1e680a3430 msgr2=0x7f1e680a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:08.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.109+0000 7f1e83e00640 1 --2- 172.21.15.43:0/1720237042 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f1e680a3430 0x7f1e680a58f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x55590b9a1230 tx=0x7f1e740d3b50 comp rx=0 tx=0).stop 2024-09-19T05:13:08.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.109+0000 7f1e83e00640 1 -- 172.21.15.43:0/1720237042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e84103e30 msgr2=0x7f1e8419a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:08.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.109+0000 7f1e83e00640 1 --2- 172.21.15.43:0/1720237042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e84103e30 0x7f1e8419a3d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1e7400e9f0 tx=0x7f1e7400eeb0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.109+0000 7f1e83e00640 1 -- 172.21.15.43:0/1720237042 shutdown_connections 2024-09-19T05:13:08.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.109+0000 7f1e83e00640 1 --2- 172.21.15.43:0/1720237042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e84108f60 0x7f1e8419eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.109+0000 7f1e83e00640 1 --2- 172.21.15.43:0/1720237042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e841047e0 0x7f1e8419a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.109+0000 7f1e83e00640 1 --2- 172.21.15.43:0/1720237042 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f1e680a3430 0x7f1e680a58f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.109+0000 7f1e83e00640 1 --2- 172.21.15.43:0/1720237042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e84103e30 0x7f1e8419a3d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:08.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.109+0000 7f1e83e00640 1 -- 172.21.15.43:0/1720237042 >> 172.21.15.43:0/1720237042 conn(0x7f1e840ff8d0 msgr2=0x7f1e84100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:08.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.110+0000 7f1e83e00640 1 -- 172.21.15.43:0/1720237042 shutdown_connections 2024-09-19T05:13:08.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:08.110+0000 7f1e83e00640 1 -- 172.21.15.43:0/1720237042 wait complete. 2024-09-19T05:13:08.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:08 smithi043 ceph-mon[30800]: pgmap v261: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-19T05:13:08.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:08 smithi096 ceph-mon[35085]: pgmap v261: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-19T05:13:08.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:08 smithi151 ceph-mon[35425]: pgmap v261: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-19T05:13:09.169 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T05:13:09.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:09 smithi096 ceph-mon[35085]: osdmap e91: 12 total, 12 up, 12 in 2024-09-19T05:13:09.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:09 smithi096 ceph-mon[35085]: pgmap v263: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T05:13:09.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:09 smithi043 ceph-mon[30800]: osdmap e91: 12 total, 12 up, 12 in 2024-09-19T05:13:09.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:09 smithi043 ceph-mon[30800]: pgmap v263: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T05:13:09.548 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:13:09.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:09 smithi151 ceph-mon[35425]: osdmap e91: 12 total, 12 up, 12 in 2024-09-19T05:13:09.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:09 smithi151 ceph-mon[35425]: pgmap v263: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T05:13:10.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.153+0000 7f6ec5400640 1 -- 172.21.15.43:0/3323829250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ec0109120 msgr2=0x7f6ec0110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:10.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.153+0000 7f6ec5400640 1 --2- 172.21.15.43:0/3323829250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ec0109120 0x7f6ec0110ca0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8009ca0 tx=0x7f6eb802a8c0 comp rx=0 tx=0).stop 2024-09-19T05:13:10.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.153+0000 7f6ec5400640 1 -- 172.21.15.43:0/3323829250 shutdown_connections 2024-09-19T05:13:10.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.153+0000 7f6ec5400640 1 --2- 172.21.15.43:0/3323829250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ec0109120 0x7f6ec0110ca0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:10.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.153+0000 7f6ec5400640 1 --2- 172.21.15.43:0/3323829250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ec01049a0 0x7f6ec01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:10.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.153+0000 7f6ec5400640 1 --2- 172.21.15.43:0/3323829250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ec0103ff0 0x7f6ec01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:10.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.153+0000 7f6ec5400640 1 -- 172.21.15.43:0/3323829250 >> 172.21.15.43:0/3323829250 conn(0x7f6ec00ffa90 msgr2=0x7f6ec0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:10.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.153+0000 7f6ec5400640 1 -- 172.21.15.43:0/3323829250 shutdown_connections 2024-09-19T05:13:10.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.153+0000 7f6ec5400640 1 -- 172.21.15.43:0/3323829250 wait complete. 2024-09-19T05:13:10.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.154+0000 7f6ec5400640 1 Processor -- start 2024-09-19T05:13:10.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.154+0000 7f6ec5400640 1 -- start start 2024-09-19T05:13:10.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.154+0000 7f6ec5400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ec0103ff0 0x7f6ec019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:10.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.155+0000 7f6ec5400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ec01049a0 0x7f6ec019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:10.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.155+0000 7f6ec5400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ec0109120 0x7f6ec019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:10.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.155+0000 7f6ec5400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec0114130 con 0x7f6ec0103ff0 2024-09-19T05:13:10.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.155+0000 7f6ec5400640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec0113fb0 con 0x7f6ec0109120 2024-09-19T05:13:10.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.155+0000 7f6ec5400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec01142b0 con 0x7f6ec01049a0 2024-09-19T05:13:10.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.155+0000 7f6ebe800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ec0103ff0 0x7f6ec019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:10.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.155+0000 7f6ebe800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ec0103ff0 0x7f6ec019a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40248/0 (socket says 172.21.15.43:40248) 2024-09-19T05:13:10.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.155+0000 7f6ebe800640 1 -- 172.21.15.43:0/168123050 learned_addr learned my addr 172.21.15.43:0/168123050 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:10.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.155+0000 7f6ebf200640 1 --2- 172.21.15.43:0/168123050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ec0109120 0x7f6ec019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:10.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.155+0000 7f6ebde00640 1 --2- 172.21.15.43:0/168123050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ec01049a0 0x7f6ec019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:10.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.155+0000 7f6ebe800640 1 -- 172.21.15.43:0/168123050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ec01049a0 msgr2=0x7f6ec019ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:10.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.155+0000 7f6ebe800640 1 --2- 172.21.15.43:0/168123050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ec01049a0 0x7f6ec019ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:10.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.155+0000 7f6ebe800640 1 -- 172.21.15.43:0/168123050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ec0109120 msgr2=0x7f6ec019eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:10.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.155+0000 7f6ebe800640 1 --2- 172.21.15.43:0/168123050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ec0109120 0x7f6ec019eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:10.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.155+0000 7f6ebe800640 1 -- 172.21.15.43:0/168123050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ec019f5c0 con 0x7f6ec0103ff0 2024-09-19T05:13:10.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.155+0000 7f6ebf200640 1 --2- 172.21.15.43:0/168123050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ec0109120 0x7f6ec019eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:13:10.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.156+0000 7f6ebe800640 1 --2- 172.21.15.43:0/168123050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ec0103ff0 0x7f6ec019a610 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000d950 tx=0x7f6eb000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:10.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.156+0000 7f6eb7400640 1 -- 172.21.15.43:0/168123050 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0017070 con 0x7f6ec0103ff0 2024-09-19T05:13:10.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.156+0000 7f6eb7400640 1 -- 172.21.15.43:0/168123050 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6eb0009c30 con 0x7f6ec0103ff0 2024-09-19T05:13:10.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.156+0000 7f6ec5400640 1 -- 172.21.15.43:0/168123050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ec019f8b0 con 0x7f6ec0103ff0 2024-09-19T05:13:10.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.156+0000 7f6eb7400640 1 -- 172.21.15.43:0/168123050 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb000f6b0 con 0x7f6ec0103ff0 2024-09-19T05:13:10.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.156+0000 7f6ec5400640 1 -- 172.21.15.43:0/168123050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ec01a7190 con 0x7f6ec0103ff0 2024-09-19T05:13:10.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.159+0000 7f6ec7fce740 1 Processor -- start 2024-09-19T05:13:10.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.159+0000 7f6eb7400640 1 -- 172.21.15.43:0/168123050 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6eb00109d0 con 0x7f6ec0103ff0 2024-09-19T05:13:10.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.159+0000 7f6ec7fce740 1 -- start start 2024-09-19T05:13:10.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.160+0000 7f6ec7fce740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5c0002c70 0x55f5c0003050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:10.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.160+0000 7f6ec7fce740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c0003590 0x55f5c000d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:10.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.160+0000 7f6ec7fce740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c000de50 0x55f5c0010240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:10.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.160+0000 7f6ec7fce740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f5bff693e0 con 0x55f5c0002c70 2024-09-19T05:13:10.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.160+0000 7f6ec7fce740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f5bff10d80 con 0x55f5c000de50 2024-09-19T05:13:10.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.160+0000 7f6ec7fce740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f5bff653a0 con 0x55f5c0003590 2024-09-19T05:13:10.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.160+0000 7f6ebf200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5c0002c70 0x55f5c0003050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:10.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.160+0000 7f6ebe800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c0003590 0x55f5c000d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:10.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.160+0000 7f6ebde00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c000de50 0x55f5c0010240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:10.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.160+0000 7f6ebf200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5c0002c70 0x55f5c0003050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40264/0 (socket says 172.21.15.43:40264) 2024-09-19T05:13:10.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.160+0000 7f6ebf200640 1 -- 172.21.15.43:0/2179539287 learned_addr learned my addr 172.21.15.43:0/2179539287 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:10.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.161+0000 7f6ebde00640 1 -- 172.21.15.43:0/2179539287 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c0003590 msgr2=0x55f5c000d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:10.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.161+0000 7f6ebde00640 1 --2- 172.21.15.43:0/2179539287 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c0003590 0x55f5c000d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:10.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.161+0000 7f6ebde00640 1 -- 172.21.15.43:0/2179539287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5c0002c70 msgr2=0x55f5c0003050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:10.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.161+0000 7f6ebde00640 1 --2- 172.21.15.43:0/2179539287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5c0002c70 0x55f5c0003050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:10.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.161+0000 7f6ebde00640 1 -- 172.21.15.43:0/2179539287 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5bff2fae0 con 0x55f5c000de50 2024-09-19T05:13:10.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.161+0000 7f6eb7400640 1 --2- 172.21.15.43:0/168123050 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f6ea40a3430 0x7f6ea40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:10.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.161+0000 7f6ebde00640 1 --2- 172.21.15.43:0/2179539287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c000de50 0x55f5c0010240 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6eac005d00 tx=0x7f6eac00bf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2549903b6c4bfce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:10.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.161+0000 7f6eb7400640 1 -- 172.21.15.43:0/168123050 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8586+0+0 (secure 0 0 0) 0x7f6eb00cf900 con 0x7f6ec0103ff0 2024-09-19T05:13:10.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.161+0000 7f6eab400640 1 -- 172.21.15.43:0/2179539287 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eac010e20 con 0x55f5c000de50 2024-09-19T05:13:10.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.161+0000 7f6eab400640 1 -- 172.21.15.43:0/2179539287 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6eac01c960 con 0x55f5c000de50 2024-09-19T05:13:10.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.161+0000 7f6eab400640 1 -- 172.21.15.43:0/2179539287 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eac011a90 con 0x55f5c000de50 2024-09-19T05:13:10.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.162+0000 7f6ebf200640 1 --2- 172.21.15.43:0/168123050 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f6ea40a3430 0x7f6ea40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:10.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.162+0000 7f6ec7fce740 1 -- 172.21.15.43:0/2179539287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c000de50 msgr2=0x55f5c0010240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:10.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.162+0000 7f6ec7fce740 1 --2- 172.21.15.43:0/2179539287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c000de50 0x55f5c0010240 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6eac005d00 tx=0x7f6eac00bf70 comp rx=0 tx=0).stop 2024-09-19T05:13:10.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.162+0000 7f6ebf200640 1 --2- 172.21.15.43:0/168123050 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f6ea40a3430 0x7f6ea40a58f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8009430 tx=0x7f6eb8009360 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:10.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.162+0000 7f6ec7fce740 1 -- 172.21.15.43:0/2179539287 shutdown_connections 2024-09-19T05:13:10.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.162+0000 7f6ec7fce740 1 --2- 172.21.15.43:0/2179539287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c000de50 0x55f5c0010240 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:10.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.162+0000 7f6ec7fce740 1 --2- 172.21.15.43:0/2179539287 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c0003590 0x55f5c000d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:10.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.162+0000 7f6ec7fce740 1 --2- 172.21.15.43:0/2179539287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5c0002c70 0x55f5c0003050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:10.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.162+0000 7f6ec7fce740 1 -- 172.21.15.43:0/2179539287 >> 172.21.15.43:0/2179539287 conn(0x55f5bffff200 msgr2=0x55f5bffff5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:10.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.163+0000 7f6ec7fce740 1 -- 172.21.15.43:0/2179539287 shutdown_connections 2024-09-19T05:13:10.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.163+0000 7f6ec7fce740 1 -- 172.21.15.43:0/2179539287 wait complete. 2024-09-19T05:13:10.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.163+0000 7f6ec7fce740 1 -- start start 2024-09-19T05:13:10.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.163+0000 7f6ec7fce740 1 Processor -- start 2024-09-19T05:13:10.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.164+0000 7f6ec7fce740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c0002c70 0x55f5c000ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:10.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.164+0000 7f6ec7fce740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c0003590 0x55f5c000b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:10.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.164+0000 7f6ec7fce740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5c000de50 0x55f5c000b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:10.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.164+0000 7f6ec7fce740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f5c0010a20 con 0x55f5c000de50 2024-09-19T05:13:10.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.164+0000 7f6ec7fce740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f5bff240f0 con 0x55f5c0003590 2024-09-19T05:13:10.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.164+0000 7f6ec7fce740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f5c0010ba0 con 0x55f5c0002c70 2024-09-19T05:13:10.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.164+0000 7f6ec7fce740 10 client.0 _ll_register_callbacks cb 0x55f5bff0bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:10.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.164+0000 7f6ebe800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c0002c70 0x55f5c000ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:10.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.164+0000 7f6ebe800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c0002c70 0x55f5c000ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:57456/0 (socket says 172.21.15.43:57456) 2024-09-19T05:13:10.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.164+0000 7f6ebe800640 1 -- 172.21.15.43:0/3509664260 learned_addr learned my addr 172.21.15.43:0/3509664260 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:10.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.164+0000 7f6ebde00640 1 --2- 172.21.15.43:0/3509664260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c0003590 0x55f5c000b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:10.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.164+0000 7f6ebf200640 1 --2- 172.21.15.43:0/3509664260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5c000de50 0x55f5c000b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:10.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.164+0000 7f6ebe800640 1 -- 172.21.15.43:0/3509664260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c0003590 msgr2=0x55f5c000b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:10.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.164+0000 7f6ebe800640 1 --2- 172.21.15.43:0/3509664260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c0003590 0x55f5c000b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:10.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.164+0000 7f6ebe800640 1 -- 172.21.15.43:0/3509664260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5c000de50 msgr2=0x55f5c000b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:10.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.164+0000 7f6ebe800640 1 --2- 172.21.15.43:0/3509664260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5c000de50 0x55f5c000b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:10.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.164+0000 7f6ebe800640 1 -- 172.21.15.43:0/3509664260 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5bfcda3c0 con 0x55f5c0002c70 2024-09-19T05:13:10.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.165+0000 7f6ebe800640 1 --2- 172.21.15.43:0/3509664260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c0002c70 0x55f5c000ad60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6eb00cd010 tx=0x7f6eb00d3040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc26b13a4fe38c4f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:10.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.165+0000 7f6e9fe00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:13:10.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.165+0000 7f6e9fe00640 1 -- 172.21.15.43:0/3509664260 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0020030 con 0x55f5c0002c70 2024-09-19T05:13:10.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.165+0000 7f6ec7fce740 1 -- 172.21.15.43:0/3509664260 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6eac01c960 con 0x55f5c0002c70 2024-09-19T05:13:10.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.165+0000 7f6e9fe00640 1 -- 172.21.15.43:0/3509664260 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8586+0+0 (secure 0 0 0) 0x7f6eb00860b0 con 0x55f5c0002c70 2024-09-19T05:13:10.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.165+0000 7f6e9fe00640 1 -- 172.21.15.43:0/3509664260 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6eb009b550 con 0x55f5c0002c70 2024-09-19T05:13:10.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.165+0000 7f6ec7fce740 10 client.27076 fetch_fsmap learned FSMap version 29 2024-09-19T05:13:10.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.165+0000 7f6ec7fce740 1 -- 172.21.15.43:0/3509664260 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f5c001cae0 con 0x55f5c0002c70 2024-09-19T05:13:10.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.165+0000 7f6e9fe00640 1 -- 172.21.15.43:0/3509664260 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6eac01c960 con 0x55f5c0002c70 2024-09-19T05:13:10.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.166+0000 7f6e9fe00640 1 -- 172.21.15.43:0/3509664260 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6eb009b040 con 0x55f5c0002c70 2024-09-19T05:13:10.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.166+0000 7f6ec7fce740 10 client.27076 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:13:10.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.166+0000 7f6ec7fce740 10 client.27076 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T05:13:10.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.166+0000 7f6ec7fce740 20 client.27076 populate_metadata read hostname 'smithi043' 2024-09-19T05:13:10.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.166+0000 7f6ec7fce740 1 --2- 172.21.15.43:0/3509664260 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55f5c001dff0 0x55f5c0020440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:10.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.166+0000 7f6ec7fce740 4 client.27076 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:13:10.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.166+0000 7f6ec7fce740 1 -- 172.21.15.43:0/3509664260 --> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5bff76950 con 0x55f5c001dff0 2024-09-19T05:13:10.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.166+0000 7f6ebde00640 1 --2- 172.21.15.43:0/3509664260 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55f5c001dff0 0x55f5c0020440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:10.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.167+0000 7f6ebde00640 1 --2- 172.21.15.43:0/3509664260 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55f5c001dff0 0x55f5c0020440 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=602dd95043766c06 server_cookie=cf75d6f045109850 in_seq=0 out_seq=0 2024-09-19T05:13:10.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.167+0000 7f6e9fe00640 10 client.27076 ms_handle_connect on v2:172.21.15.151:6838/3794104313 2024-09-19T05:13:10.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.168+0000 7f6e9fe00640 1 -- 172.21.15.43:0/3509664260 <== mds.2 v2:172.21.15.151:6838/3794104313 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f5bff76950 con 0x55f5c001dff0 2024-09-19T05:13:10.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.168+0000 7f6e9fe00640 10 client.27076 handle_command_reply: tid=0 2024-09-19T05:13:10.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.168+0000 7f6ec7fce740 1 client.27076 shutdown 2024-09-19T05:13:10.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.168+0000 7f6ec7fce740 1 -- 172.21.15.43:0/3509664260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c0002c70 msgr2=0x55f5c000ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:10.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.168+0000 7f6ec7fce740 1 --2- 172.21.15.43:0/3509664260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c0002c70 0x55f5c000ad60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6eb00cd010 tx=0x7f6eb00d3040 comp rx=0 tx=0).stop 2024-09-19T05:13:10.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.168+0000 7f6ec7fce740 1 -- 172.21.15.43:0/3509664260 shutdown_connections 2024-09-19T05:13:10.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.168+0000 7f6ec7fce740 1 --2- 172.21.15.43:0/3509664260 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55f5c001dff0 0x55f5c0020440 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:10.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.168+0000 7f6ec7fce740 1 --2- 172.21.15.43:0/3509664260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5c000de50 0x55f5c000b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:10.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.168+0000 7f6ec7fce740 1 --2- 172.21.15.43:0/3509664260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c0003590 0x55f5c000b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:10.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.168+0000 7f6ec7fce740 1 --2- 172.21.15.43:0/3509664260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c0002c70 0x55f5c000ad60 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:10.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.168+0000 7f6ec7fce740 1 -- 172.21.15.43:0/3509664260 >> 172.21.15.43:0/3509664260 conn(0x55f5bffff200 msgr2=0x55f5c0004f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:10.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.169+0000 7f6e9fe00640 0 client.27076 ms_handle_reset on v2:172.21.15.151:6838/3794104313 2024-09-19T05:13:10.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.169+0000 7f6ec7fce740 1 -- 172.21.15.43:0/3509664260 shutdown_connections 2024-09-19T05:13:10.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.169+0000 7f6ec7fce740 1 -- 172.21.15.43:0/3509664260 wait complete. 2024-09-19T05:13:10.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.169+0000 7f6ec7fce740 20 client.27076 trim_cache size 0 max 16384 2024-09-19T05:13:10.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.182+0000 7f6ec7fce740 1 Processor -- start 2024-09-19T05:13:10.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.183+0000 7f6ec7fce740 1 -- start start 2024-09-19T05:13:10.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.183+0000 7f6ec7fce740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c0002c70 0x55f5c0005720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:10.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.183+0000 7f6ec7fce740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5c0003590 0x55f5c000ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:10.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.183+0000 7f6ec7fce740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c000de50 0x55f5c0025d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:10.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.183+0000 7f6ec7fce740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec01afb40 con 0x55f5c0003590 2024-09-19T05:13:10.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.183+0000 7f6ec7fce740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec01b1680 con 0x55f5c0002c70 2024-09-19T05:13:10.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.183+0000 7f6ec7fce740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec01b0210 con 0x55f5c000de50 2024-09-19T05:13:10.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.183+0000 7f6ebde00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5c0003590 0x55f5c000ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:10.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.183+0000 7f6ebf200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c000de50 0x55f5c0025d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:10.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.183+0000 7f6ebe800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c0002c70 0x55f5c0005720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:10.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.183+0000 7f6ebde00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5c0003590 0x55f5c000ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40294/0 (socket says 172.21.15.43:40294) 2024-09-19T05:13:10.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.183+0000 7f6ebde00640 1 -- 172.21.15.43:0/1421707892 learned_addr learned my addr 172.21.15.43:0/1421707892 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:10.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.183+0000 7f6ebf200640 1 --2- 172.21.15.43:0/1421707892 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c000de50 0x55f5c0025d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:57460/0 (socket says 172.21.15.43:57460) 2024-09-19T05:13:10.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.184+0000 7f6ebe800640 1 -- 172.21.15.43:0/1421707892 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c000de50 msgr2=0x55f5c0025d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:10.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.184+0000 7f6ebe800640 1 --2- 172.21.15.43:0/1421707892 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c000de50 0x55f5c0025d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:10.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.184+0000 7f6ebe800640 1 -- 172.21.15.43:0/1421707892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5c0003590 msgr2=0x55f5c000ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:10.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.184+0000 7f6ebe800640 1 --2- 172.21.15.43:0/1421707892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5c0003590 0x55f5c000ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:10.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.184+0000 7f6ebe800640 1 -- 172.21.15.43:0/1421707892 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5c0002aa0 con 0x55f5c0002c70 2024-09-19T05:13:10.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.184+0000 7f6ebe800640 1 --2- 172.21.15.43:0/1421707892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c0002c70 0x55f5c0005720 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6eb00058c0 tx=0x7f6eb000bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=375af2220a321fe8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:10.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.184+0000 7f6eaaa00640 1 -- 172.21.15.43:0/1421707892 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0020070 con 0x55f5c0002c70 2024-09-19T05:13:10.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.184+0000 7f6eaaa00640 1 -- 172.21.15.43:0/1421707892 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6eb009bbf0 con 0x55f5c0002c70 2024-09-19T05:13:10.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.184+0000 7f6eaaa00640 1 -- 172.21.15.43:0/1421707892 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb00862a0 con 0x55f5c0002c70 2024-09-19T05:13:10.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.185+0000 7f6ec7fce740 1 -- 172.21.15.43:0/1421707892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c0002c70 msgr2=0x55f5c0005720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:10.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.185+0000 7f6ec7fce740 1 --2- 172.21.15.43:0/1421707892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c0002c70 0x55f5c0005720 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6eb00058c0 tx=0x7f6eb000bf60 comp rx=0 tx=0).stop 2024-09-19T05:13:10.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.185+0000 7f6ec7fce740 1 -- 172.21.15.43:0/1421707892 shutdown_connections 2024-09-19T05:13:10.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.185+0000 7f6ec7fce740 1 --2- 172.21.15.43:0/1421707892 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c000de50 0x55f5c0025d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:10.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.185+0000 7f6ec7fce740 1 --2- 172.21.15.43:0/1421707892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5c0003590 0x55f5c000ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:10.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.185+0000 7f6ec7fce740 1 --2- 172.21.15.43:0/1421707892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c0002c70 0x55f5c0005720 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:10.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.185+0000 7f6ec7fce740 1 -- 172.21.15.43:0/1421707892 >> 172.21.15.43:0/1421707892 conn(0x55f5bffff200 msgr2=0x55f5c000c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:10.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.185+0000 7f6ec7fce740 1 -- 172.21.15.43:0/1421707892 shutdown_connections 2024-09-19T05:13:10.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.185+0000 7f6ec7fce740 1 -- 172.21.15.43:0/1421707892 wait complete. 2024-09-19T05:13:10.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.186+0000 7f6ec7fce740 1 -- start start 2024-09-19T05:13:10.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.186+0000 7f6ec7fce740 1 Processor -- start 2024-09-19T05:13:10.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.186+0000 7f6ec7fce740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5c0002c70 0x55f5c0016160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.186+0000 7f6ec7fce740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c0003590 0x55f5c00166a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.186+0000 7f6ec7fce740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c000de50 0x55f5c0007910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.186+0000 7f6ec7fce740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec01b0fb0 con 0x55f5c0002c70 2024-09-19T05:13:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.186+0000 7f6ec7fce740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec01b08e0 con 0x55f5c0003590 2024-09-19T05:13:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.186+0000 7f6ec7fce740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec01adb40 con 0x55f5c000de50 2024-09-19T05:13:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.186+0000 7f6ec7fce740 10 client.0 _ll_register_callbacks cb 0x7f6eb000df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.187+0000 7f6ebe800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5c0002c70 0x55f5c0016160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.187+0000 7f6ebf200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c000de50 0x55f5c0007910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.187+0000 7f6ebe800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5c0002c70 0x55f5c0016160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40296/0 (socket says 172.21.15.43:40296) 2024-09-19T05:13:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.187+0000 7f6ebe800640 1 -- 172.21.15.43:0/2675395874 learned_addr learned my addr 172.21.15.43:0/2675395874 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.187+0000 7f6ebde00640 1 --2- 172.21.15.43:0/2675395874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c0003590 0x55f5c00166a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.187+0000 7f6ebf200640 1 -- 172.21.15.43:0/2675395874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c0003590 msgr2=0x55f5c00166a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.187+0000 7f6ebf200640 1 --2- 172.21.15.43:0/2675395874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c0003590 0x55f5c00166a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:10.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.187+0000 7f6ebf200640 1 -- 172.21.15.43:0/2675395874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5c0002c70 msgr2=0x55f5c0016160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:10.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.187+0000 7f6ebf200640 1 --2- 172.21.15.43:0/2675395874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5c0002c70 0x55f5c0016160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:10.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.187+0000 7f6ebf200640 1 -- 172.21.15.43:0/2675395874 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5bfffa770 con 0x55f5c000de50 2024-09-19T05:13:10.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.187+0000 7f6ebe800640 1 --2- 172.21.15.43:0/2675395874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5c0002c70 0x55f5c0016160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:13:10.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.188+0000 7f6ebf200640 1 --2- 172.21.15.43:0/2675395874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c000de50 0x55f5c0007910 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8005e30 tx=0x7f6eb802a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a46f99d923ff3996 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:10.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.188+0000 7f6e9fe00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:13:10.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.189+0000 7f6e9fe00640 1 -- 172.21.15.43:0/2675395874 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb8059060 con 0x55f5c000de50 2024-09-19T05:13:10.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.189+0000 7f6e9fe00640 1 -- 172.21.15.43:0/2675395874 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8586+0+0 (secure 0 0 0) 0x7f6eb80307d0 con 0x55f5c000de50 2024-09-19T05:13:10.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.189+0000 7f6ec7fce740 1 -- 172.21.15.43:0/2675395874 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6eb009bbf0 con 0x55f5c000de50 2024-09-19T05:13:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.189+0000 7f6e9fe00640 1 -- 172.21.15.43:0/2675395874 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6eb80309d0 con 0x55f5c000de50 2024-09-19T05:13:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.189+0000 7f6ec7fce740 10 client.27082 fetch_fsmap learned FSMap version 29 2024-09-19T05:13:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.189+0000 7f6ec7fce740 1 -- 172.21.15.43:0/2675395874 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f5c0088760 con 0x55f5c000de50 2024-09-19T05:13:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.190+0000 7f6e9fe00640 1 -- 172.21.15.43:0/2675395874 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6eb009bbf0 con 0x55f5c000de50 2024-09-19T05:13:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.190+0000 7f6e9fe00640 1 -- 172.21.15.43:0/2675395874 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6eb8053070 con 0x55f5c000de50 2024-09-19T05:13:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.190+0000 7f6ec7fce740 10 client.27082 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:13:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.190+0000 7f6ec7fce740 10 client.27082 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T05:13:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.190+0000 7f6ec7fce740 20 client.27082 populate_metadata read hostname 'smithi043' 2024-09-19T05:13:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.190+0000 7f6ec7fce740 1 --2- 172.21.15.43:0/2675395874 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55f5c0088e80 0x55f5c008b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.190+0000 7f6ec7fce740 4 client.27082 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:13:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.190+0000 7f6ec7fce740 1 -- 172.21.15.43:0/2675395874 --> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f5bff70e60 con 0x55f5c0088e80 2024-09-19T05:13:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.190+0000 7f6ebe800640 1 --2- 172.21.15.43:0/2675395874 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55f5c0088e80 0x55f5c008b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.191+0000 7f6ebe800640 1 --2- 172.21.15.43:0/2675395874 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55f5c0088e80 0x55f5c008b2a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=be2fd52af3eb6fb4 server_cookie=9d90bbed1c034852 in_seq=0 out_seq=0 2024-09-19T05:13:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.191+0000 7f6e9fe00640 10 client.27082 ms_handle_connect on v2:172.21.15.151:6838/3794104313 2024-09-19T05:13:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.191+0000 7f6e9fe00640 1 -- 172.21.15.43:0/2675395874 <== mds.2 v2:172.21.15.151:6838/3794104313 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f5bff70e60 con 0x55f5c0088e80 2024-09-19T05:13:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.191+0000 7f6e9fe00640 10 client.27082 handle_command_reply: tid=0 2024-09-19T05:13:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.191+0000 7f6ec7fce740 1 client.27082 shutdown 2024-09-19T05:13:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.191+0000 7f6ec7fce740 1 -- 172.21.15.43:0/2675395874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c000de50 msgr2=0x55f5c0007910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.191+0000 7f6ec7fce740 1 --2- 172.21.15.43:0/2675395874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c000de50 0x55f5c0007910 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8005e30 tx=0x7f6eb802a8c0 comp rx=0 tx=0).stop 2024-09-19T05:13:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.191+0000 7f6ec7fce740 1 -- 172.21.15.43:0/2675395874 shutdown_connections 2024-09-19T05:13:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.191+0000 7f6ec7fce740 1 --2- 172.21.15.43:0/2675395874 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55f5c0088e80 0x55f5c008b2a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.191+0000 7f6ec7fce740 1 --2- 172.21.15.43:0/2675395874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c000de50 0x55f5c0007910 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.191+0000 7f6ec7fce740 1 --2- 172.21.15.43:0/2675395874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c0003590 0x55f5c00166a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:10.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.191+0000 7f6e9fe00640 0 client.27082 ms_handle_reset on v2:172.21.15.151:6838/3794104313 2024-09-19T05:13:10.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.191+0000 7f6ec7fce740 1 --2- 172.21.15.43:0/2675395874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5c0002c70 0x55f5c0016160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:10.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.191+0000 7f6ec7fce740 1 -- 172.21.15.43:0/2675395874 >> 172.21.15.43:0/2675395874 conn(0x55f5bffff200 msgr2=0x55f5bfffe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:10.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.191+0000 7f6ec7fce740 1 -- 172.21.15.43:0/2675395874 shutdown_connections 2024-09-19T05:13:10.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.191+0000 7f6ec7fce740 1 -- 172.21.15.43:0/2675395874 wait complete. 2024-09-19T05:13:10.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.192+0000 7f6ec7fce740 20 client.27082 trim_cache size 0 max 16384 2024-09-19T05:13:10.195 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:13:10.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.192+0000 7f6ec5400640 1 -- 172.21.15.43:0/168123050 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f6ea40a3430 msgr2=0x7f6ea40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:10.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.192+0000 7f6ec5400640 1 --2- 172.21.15.43:0/168123050 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f6ea40a3430 0x7f6ea40a58f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8009430 tx=0x7f6eb8009360 comp rx=0 tx=0).stop 2024-09-19T05:13:10.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.192+0000 7f6ec5400640 1 -- 172.21.15.43:0/168123050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ec0103ff0 msgr2=0x7f6ec019a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:10.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.192+0000 7f6ec5400640 1 --2- 172.21.15.43:0/168123050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ec0103ff0 0x7f6ec019a610 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000d950 tx=0x7f6eb000de10 comp rx=0 tx=0).stop 2024-09-19T05:13:10.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.192+0000 7f6ec5400640 1 -- 172.21.15.43:0/168123050 shutdown_connections 2024-09-19T05:13:10.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.192+0000 7f6ec5400640 1 --2- 172.21.15.43:0/168123050 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f6ea40a3430 0x7f6ea40a58f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:10.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.192+0000 7f6ec5400640 1 --2- 172.21.15.43:0/168123050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ec0109120 0x7f6ec019eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:10.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.192+0000 7f6ec5400640 1 --2- 172.21.15.43:0/168123050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ec01049a0 0x7f6ec019ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:10.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.192+0000 7f6ec5400640 1 --2- 172.21.15.43:0/168123050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ec0103ff0 0x7f6ec019a610 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:10.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.192+0000 7f6ec5400640 1 -- 172.21.15.43:0/168123050 >> 172.21.15.43:0/168123050 conn(0x7f6ec00ffa90 msgr2=0x7f6ec01005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:10.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.193+0000 7f6ec5400640 1 -- 172.21.15.43:0/168123050 shutdown_connections 2024-09-19T05:13:10.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:10.193+0000 7f6ec5400640 1 -- 172.21.15.43:0/168123050 wait complete. 2024-09-19T05:13:11.365 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T05:13:11.747 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:13:12.125 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:11 smithi043 ceph-mon[30800]: pgmap v264: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T05:13:12.125 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:11 smithi043 ceph-mon[30800]: osdmap e92: 12 total, 12 up, 12 in 2024-09-19T05:13:12.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:11 smithi151 ceph-mon[35425]: pgmap v264: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T05:13:12.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:11 smithi151 ceph-mon[35425]: osdmap e92: 12 total, 12 up, 12 in 2024-09-19T05:13:12.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:11 smithi096 ceph-mon[35085]: pgmap v264: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T05:13:12.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:11 smithi096 ceph-mon[35085]: osdmap e92: 12 total, 12 up, 12 in 2024-09-19T05:13:12.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.352+0000 7ffa87e00640 1 -- 172.21.15.43:0/1603692971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa80101de0 msgr2=0x7ffa801021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:12.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.352+0000 7ffa87e00640 1 --2- 172.21.15.43:0/1603692971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa80101de0 0x7ffa801021c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ffa74009900 tx=0x7ffa7402a410 comp rx=0 tx=0).stop 2024-09-19T05:13:12.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.353+0000 7ffa87e00640 1 -- 172.21.15.43:0/1603692971 shutdown_connections 2024-09-19T05:13:12.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.353+0000 7ffa87e00640 1 --2- 172.21.15.43:0/1603692971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa80106f10 0x7ffa8010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:12.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.353+0000 7ffa87e00640 1 --2- 172.21.15.43:0/1603692971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa80102790 0x7ffa801067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:12.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.353+0000 7ffa87e00640 1 --2- 172.21.15.43:0/1603692971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa80101de0 0x7ffa801021c0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:12.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.353+0000 7ffa87e00640 1 -- 172.21.15.43:0/1603692971 >> 172.21.15.43:0/1603692971 conn(0x7ffa800fd8c0 msgr2=0x7ffa800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:12.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.353+0000 7ffa87e00640 1 -- 172.21.15.43:0/1603692971 shutdown_connections 2024-09-19T05:13:12.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.353+0000 7ffa87e00640 1 -- 172.21.15.43:0/1603692971 wait complete. 2024-09-19T05:13:12.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.353+0000 7ffa87e00640 1 Processor -- start 2024-09-19T05:13:12.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.354+0000 7ffa87e00640 1 -- start start 2024-09-19T05:13:12.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.354+0000 7ffa87e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa80101de0 0x7ffa801984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:12.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.354+0000 7ffa87e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa80102790 0x7ffa80198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:12.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.354+0000 7ffa87e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa80106f10 0x7ffa8019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:12.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.354+0000 7ffa87e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ffa80111fe0 con 0x7ffa80106f10 2024-09-19T05:13:12.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.354+0000 7ffa87e00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffa80111e60 con 0x7ffa80102790 2024-09-19T05:13:12.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.354+0000 7ffa87e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffa80112160 con 0x7ffa80101de0 2024-09-19T05:13:12.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.354+0000 7ffa84a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa80102790 0x7ffa80198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:12.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.354+0000 7ffa85400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa80101de0 0x7ffa801984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:12.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.354+0000 7ffa85e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa80106f10 0x7ffa8019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:12.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.354+0000 7ffa84a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa80102790 0x7ffa80198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:52668/0 (socket says 172.21.15.43:52668) 2024-09-19T05:13:12.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.354+0000 7ffa85400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa80101de0 0x7ffa801984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:57496/0 (socket says 172.21.15.43:57496) 2024-09-19T05:13:12.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.354+0000 7ffa84a00640 1 -- 172.21.15.43:0/3790583139 learned_addr learned my addr 172.21.15.43:0/3790583139 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:12.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.355+0000 7ffa85e00640 1 -- 172.21.15.43:0/3790583139 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa80101de0 msgr2=0x7ffa801984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:12.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.355+0000 7ffa85e00640 1 --2- 172.21.15.43:0/3790583139 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa80101de0 0x7ffa801984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:12.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.355+0000 7ffa85e00640 1 -- 172.21.15.43:0/3790583139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa80102790 msgr2=0x7ffa80198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:12.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.355+0000 7ffa85e00640 1 --2- 172.21.15.43:0/3790583139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa80102790 0x7ffa80198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:12.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.355+0000 7ffa85e00640 1 -- 172.21.15.43:0/3790583139 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa8019d490 con 0x7ffa80106f10 2024-09-19T05:13:12.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.355+0000 7ffa84a00640 1 --2- 172.21.15.43:0/3790583139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa80102790 0x7ffa80198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:13:12.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.355+0000 7ffa85e00640 1 --2- 172.21.15.43:0/3790583139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa80106f10 0x7ffa8019cdb0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ffa7800da10 tx=0x7ffa7800ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:12.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.355+0000 7ffa7e000640 1 -- 172.21.15.43:0/3790583139 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7800bd70 con 0x7ffa80106f10 2024-09-19T05:13:12.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.355+0000 7ffa7e000640 1 -- 172.21.15.43:0/3790583139 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa78003e50 con 0x7ffa80106f10 2024-09-19T05:13:12.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.355+0000 7ffa7e000640 1 -- 172.21.15.43:0/3790583139 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa780106a0 con 0x7ffa80106f10 2024-09-19T05:13:12.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.356+0000 7ffa87e00640 1 -- 172.21.15.43:0/3790583139 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa8019d780 con 0x7ffa80106f10 2024-09-19T05:13:12.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.356+0000 7ffa87e00640 1 -- 172.21.15.43:0/3790583139 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa801a5060 con 0x7ffa80106f10 2024-09-19T05:13:12.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.359+0000 7ffa8a9d4740 1 Processor -- start 2024-09-19T05:13:12.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.359+0000 7ffa7e000640 1 -- 172.21.15.43:0/3790583139 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ffa78003970 con 0x7ffa80106f10 2024-09-19T05:13:12.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.359+0000 7ffa8a9d4740 1 -- start start 2024-09-19T05:13:12.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.359+0000 7ffa8a9d4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563332eceb70 0x563332ecef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:12.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.359+0000 7ffa8a9d4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563332ecf490 0x563332eddd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:12.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.359+0000 7ffa8a9d4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563332ede260 0x563332ee0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:12.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.359+0000 7ffa8a9d4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563332e353e0 con 0x563332ede260 2024-09-19T05:13:12.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.359+0000 7ffa8a9d4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563332ddcd80 con 0x563332ecf490 2024-09-19T05:13:12.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.359+0000 7ffa8a9d4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563332e313a0 con 0x563332eceb70 2024-09-19T05:13:12.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.360+0000 7ffa84a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563332eceb70 0x563332ecef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:12.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.360+0000 7ffa84a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563332eceb70 0x563332ecef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:57500/0 (socket says 172.21.15.43:57500) 2024-09-19T05:13:12.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.360+0000 7ffa84a00640 1 -- 172.21.15.43:0/101906511 learned_addr learned my addr 172.21.15.43:0/101906511 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:12.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.360+0000 7ffa85400640 1 --2- 172.21.15.43:0/101906511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563332ecf490 0x563332eddd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:12.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.360+0000 7ffa84a00640 1 --2- 172.21.15.43:0/101906511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563332ede260 0x563332ee0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:12.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.360+0000 7ffa85400640 1 -- 172.21.15.43:0/101906511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563332eceb70 msgr2=0x563332ecef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:12.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.360+0000 7ffa85400640 1 --2- 172.21.15.43:0/101906511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563332eceb70 0x563332ecef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:12.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.360+0000 7ffa85400640 1 -- 172.21.15.43:0/101906511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563332ede260 msgr2=0x563332ee0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:12.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.360+0000 7ffa85400640 1 --2- 172.21.15.43:0/101906511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563332ede260 0x563332ee0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:12.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.360+0000 7ffa85400640 1 -- 172.21.15.43:0/101906511 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563332dfbae0 con 0x563332ecf490 2024-09-19T05:13:12.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.361+0000 7ffa7e000640 1 --2- 172.21.15.43:0/3790583139 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ffa680a33b0 0x7ffa680a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:12.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.361+0000 7ffa85400640 1 --2- 172.21.15.43:0/101906511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563332ecf490 0x563332eddd20 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ffa7402aa00 tx=0x7ffa740041a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68677bb5223914a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:12.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.361+0000 7ffa7e000640 1 -- 172.21.15.43:0/3790583139 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7ffa780d0170 con 0x7ffa80106f10 2024-09-19T05:13:12.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.361+0000 7ffa84a00640 1 --2- 172.21.15.43:0/3790583139 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ffa680a33b0 0x7ffa680a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:12.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.361+0000 7ffa72600640 1 -- 172.21.15.43:0/101906511 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa74035040 con 0x563332ecf490 2024-09-19T05:13:12.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.361+0000 7ffa72600640 1 -- 172.21.15.43:0/101906511 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa74005520 con 0x563332ecf490 2024-09-19T05:13:12.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.361+0000 7ffa72600640 1 -- 172.21.15.43:0/101906511 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa740037f0 con 0x563332ecf490 2024-09-19T05:13:12.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.362+0000 7ffa8a9d4740 1 -- 172.21.15.43:0/101906511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563332ecf490 msgr2=0x563332eddd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:12.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.362+0000 7ffa8a9d4740 1 --2- 172.21.15.43:0/101906511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563332ecf490 0x563332eddd20 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ffa7402aa00 tx=0x7ffa740041a0 comp rx=0 tx=0).stop 2024-09-19T05:13:12.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.362+0000 7ffa84a00640 1 --2- 172.21.15.43:0/3790583139 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ffa680a33b0 0x7ffa680a5870 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x563332ed0710 tx=0x7ffa6c008040 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:12.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.362+0000 7ffa8a9d4740 1 -- 172.21.15.43:0/101906511 shutdown_connections 2024-09-19T05:13:12.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.362+0000 7ffa8a9d4740 1 --2- 172.21.15.43:0/101906511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563332ede260 0x563332ee0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:12.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.362+0000 7ffa8a9d4740 1 --2- 172.21.15.43:0/101906511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563332ecf490 0x563332eddd20 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:12.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.362+0000 7ffa8a9d4740 1 --2- 172.21.15.43:0/101906511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563332eceb70 0x563332ecef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:12.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.362+0000 7ffa8a9d4740 1 -- 172.21.15.43:0/101906511 >> 172.21.15.43:0/101906511 conn(0x563332ecb200 msgr2=0x563332ecb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:12.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.362+0000 7ffa8a9d4740 1 -- 172.21.15.43:0/101906511 shutdown_connections 2024-09-19T05:13:12.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.362+0000 7ffa8a9d4740 1 -- 172.21.15.43:0/101906511 wait complete. 2024-09-19T05:13:12.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.362+0000 7ffa8a9d4740 1 -- start start 2024-09-19T05:13:12.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.363+0000 7ffa8a9d4740 1 Processor -- start 2024-09-19T05:13:12.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.363+0000 7ffa8a9d4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563332eceb70 0x563332edb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:12.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.363+0000 7ffa8a9d4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563332ecf490 0x563332edb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:12.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.363+0000 7ffa8a9d4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563332ede260 0x563332edbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:12.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.363+0000 7ffa8a9d4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563332ee0e30 con 0x563332eceb70 2024-09-19T05:13:12.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.363+0000 7ffa8a9d4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563332df00f0 con 0x563332ede260 2024-09-19T05:13:12.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.363+0000 7ffa8a9d4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563332ee0fb0 con 0x563332ecf490 2024-09-19T05:13:12.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.363+0000 7ffa8a9d4740 10 client.0 _ll_register_callbacks cb 0x563332dd7bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:12.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.363+0000 7ffa85400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563332eceb70 0x563332edb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:12.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.363+0000 7ffa85400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563332eceb70 0x563332edb200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40326/0 (socket says 172.21.15.43:40326) 2024-09-19T05:13:12.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.363+0000 7ffa85400640 1 -- 172.21.15.43:0/644030534 learned_addr learned my addr 172.21.15.43:0/644030534 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:12.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.363+0000 7ffa85e00640 1 --2- 172.21.15.43:0/644030534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563332ede260 0x563332edbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:12.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.363+0000 7ffa84a00640 1 --2- 172.21.15.43:0/644030534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563332ecf490 0x563332edb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:12.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.364+0000 7ffa85400640 1 -- 172.21.15.43:0/644030534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563332ecf490 msgr2=0x563332edb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:12.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.364+0000 7ffa85400640 1 --2- 172.21.15.43:0/644030534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563332ecf490 0x563332edb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:12.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.364+0000 7ffa85400640 1 -- 172.21.15.43:0/644030534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563332ede260 msgr2=0x563332edbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:12.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.364+0000 7ffa85400640 1 --2- 172.21.15.43:0/644030534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563332ede260 0x563332edbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:12.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.364+0000 7ffa85400640 1 -- 172.21.15.43:0/644030534 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563332ba63c0 con 0x563332eceb70 2024-09-19T05:13:12.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.364+0000 7ffa84a00640 1 --2- 172.21.15.43:0/644030534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563332ecf490 0x563332edb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:13:12.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.364+0000 7ffa85e00640 1 --2- 172.21.15.43:0/644030534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563332ede260 0x563332edbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:13:12.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.364+0000 7ffa85400640 1 --2- 172.21.15.43:0/644030534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563332eceb70 0x563332edb200 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ffa74000c00 tx=0x7ffa74009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf5c24b3f76d8982 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:12.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.364+0000 7ffa66a00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:13:12.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.364+0000 7ffa66a00640 1 -- 172.21.15.43:0/644030534 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa74035070 con 0x563332eceb70 2024-09-19T05:13:12.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.364+0000 7ffa66a00640 1 -- 172.21.15.43:0/644030534 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7ffa740046c0 con 0x563332eceb70 2024-09-19T05:13:12.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.364+0000 7ffa8a9d4740 1 -- 172.21.15.43:0/644030534 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa74005520 con 0x563332eceb70 2024-09-19T05:13:12.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.365+0000 7ffa66a00640 1 -- 172.21.15.43:0/644030534 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffa74049070 con 0x563332eceb70 2024-09-19T05:13:12.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.365+0000 7ffa8a9d4740 10 client.17166 fetch_fsmap learned FSMap version 29 2024-09-19T05:13:12.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.365+0000 7ffa8a9d4740 1 -- 172.21.15.43:0/644030534 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563332eecf50 con 0x563332eceb70 2024-09-19T05:13:12.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.365+0000 7ffa66a00640 1 -- 172.21.15.43:0/644030534 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa74005520 con 0x563332eceb70 2024-09-19T05:13:12.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.365+0000 7ffa66a00640 1 -- 172.21.15.43:0/644030534 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ffa740315b0 con 0x563332eceb70 2024-09-19T05:13:12.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.365+0000 7ffa8a9d4740 10 client.17166 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:13:12.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.365+0000 7ffa8a9d4740 10 client.17166 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-19T05:13:12.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.365+0000 7ffa8a9d4740 20 client.17166 populate_metadata read hostname 'smithi043' 2024-09-19T05:13:12.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.365+0000 7ffa8a9d4740 1 --2- 172.21.15.43:0/644030534 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x563332eee2a0 0x563332ef0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:12.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.365+0000 7ffa8a9d4740 4 client.17166 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:13:12.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.365+0000 7ffa8a9d4740 1 -- 172.21.15.43:0/644030534 --> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563332e42950 con 0x563332eee2a0 2024-09-19T05:13:12.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.366+0000 7ffa84a00640 1 --2- 172.21.15.43:0/644030534 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x563332eee2a0 0x563332ef0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:12.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.366+0000 7ffa84a00640 1 --2- 172.21.15.43:0/644030534 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x563332eee2a0 0x563332ef0690 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b37718cb5220129a server_cookie=bbe6a21dd5a3049a in_seq=0 out_seq=0 2024-09-19T05:13:12.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.366+0000 7ffa66a00640 10 client.17166 ms_handle_connect on v2:172.21.15.43:6838/3404555030 2024-09-19T05:13:12.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.367+0000 7ffa66a00640 1 -- 172.21.15.43:0/644030534 <== mds.1 v2:172.21.15.43:6838/3404555030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563332e42950 con 0x563332eee2a0 2024-09-19T05:13:12.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.367+0000 7ffa66a00640 10 client.17166 handle_command_reply: tid=0 2024-09-19T05:13:12.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.367+0000 7ffa8a9d4740 1 client.17166 shutdown 2024-09-19T05:13:12.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.367+0000 7ffa8a9d4740 1 -- 172.21.15.43:0/644030534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563332eceb70 msgr2=0x563332edb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:12.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.367+0000 7ffa8a9d4740 1 --2- 172.21.15.43:0/644030534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563332eceb70 0x563332edb200 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ffa74000c00 tx=0x7ffa74009680 comp rx=0 tx=0).stop 2024-09-19T05:13:12.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.367+0000 7ffa8a9d4740 1 -- 172.21.15.43:0/644030534 shutdown_connections 2024-09-19T05:13:12.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.367+0000 7ffa8a9d4740 1 --2- 172.21.15.43:0/644030534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563332ede260 0x563332edbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:12.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.367+0000 7ffa8a9d4740 1 --2- 172.21.15.43:0/644030534 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x563332eee2a0 0x563332ef0690 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:12.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.367+0000 7ffa8a9d4740 1 --2- 172.21.15.43:0/644030534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563332ecf490 0x563332edb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:12.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.367+0000 7ffa8a9d4740 1 --2- 172.21.15.43:0/644030534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563332eceb70 0x563332edb200 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:12.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.367+0000 7ffa66a00640 0 client.17166 ms_handle_reset on v2:172.21.15.43:6838/3404555030 2024-09-19T05:13:12.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.367+0000 7ffa8a9d4740 1 -- 172.21.15.43:0/644030534 >> 172.21.15.43:0/644030534 conn(0x563332ecb200 msgr2=0x563332ed0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:12.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.367+0000 7ffa8a9d4740 1 -- 172.21.15.43:0/644030534 shutdown_connections 2024-09-19T05:13:12.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.367+0000 7ffa8a9d4740 1 -- 172.21.15.43:0/644030534 wait complete. 2024-09-19T05:13:12.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.368+0000 7ffa8a9d4740 20 client.17166 trim_cache size 0 max 16384 2024-09-19T05:13:12.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.381+0000 7ffa8a9d4740 1 Processor -- start 2024-09-19T05:13:12.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.381+0000 7ffa8a9d4740 1 -- start start 2024-09-19T05:13:12.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.382+0000 7ffa8a9d4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563332eceb70 0x563332ee9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:12.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.382+0000 7ffa8a9d4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563332ecf490 0x563332ed8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:12.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.382+0000 7ffa8a9d4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563332ede260 0x563332ed89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:12.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.382+0000 7ffa8a9d4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ffa801ab6a0 con 0x563332eceb70 2024-09-19T05:13:12.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.382+0000 7ffa8a9d4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffa801ad1e0 con 0x563332ede260 2024-09-19T05:13:12.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.382+0000 7ffa8a9d4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffa801abd70 con 0x563332ecf490 2024-09-19T05:13:12.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.382+0000 7ffa85400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563332eceb70 0x563332ee9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:12.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.382+0000 7ffa84a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563332ecf490 0x563332ed8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:12.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.382+0000 7ffa84a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563332ecf490 0x563332ed8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:57512/0 (socket says 172.21.15.43:57512) 2024-09-19T05:13:12.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.382+0000 7ffa85400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563332eceb70 0x563332ee9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40340/0 (socket says 172.21.15.43:40340) 2024-09-19T05:13:12.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.382+0000 7ffa84a00640 1 -- 172.21.15.43:0/1364293483 learned_addr learned my addr 172.21.15.43:0/1364293483 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:12.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.382+0000 7ffa85e00640 1 --2- 172.21.15.43:0/1364293483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563332ede260 0x563332ed89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:12.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.383+0000 7ffa84a00640 1 -- 172.21.15.43:0/1364293483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563332ede260 msgr2=0x563332ed89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:12.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.383+0000 7ffa84a00640 1 --2- 172.21.15.43:0/1364293483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563332ede260 0x563332ed89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:12.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.383+0000 7ffa84a00640 1 -- 172.21.15.43:0/1364293483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563332eceb70 msgr2=0x563332ee9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:12.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.383+0000 7ffa84a00640 1 --2- 172.21.15.43:0/1364293483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563332eceb70 0x563332ee9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:12.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.383+0000 7ffa84a00640 1 -- 172.21.15.43:0/1364293483 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563332ecf2c0 con 0x563332ecf490 2024-09-19T05:13:12.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.383+0000 7ffa84a00640 1 --2- 172.21.15.43:0/1364293483 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563332ecf490 0x563332ed8460 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x563332edaa90 tx=0x7ffa6c009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a07c8f1d213fe731 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:12.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.383+0000 7ffa71c00640 1 -- 172.21.15.43:0/1364293483 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c030030 con 0x563332ecf490 2024-09-19T05:13:12.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.383+0000 7ffa71c00640 1 -- 172.21.15.43:0/1364293483 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa6c025c70 con 0x563332ecf490 2024-09-19T05:13:12.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.383+0000 7ffa71c00640 1 -- 172.21.15.43:0/1364293483 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c01c040 con 0x563332ecf490 2024-09-19T05:13:12.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.384+0000 7ffa8a9d4740 1 -- 172.21.15.43:0/1364293483 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563332ecf490 msgr2=0x563332ed8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:12.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.384+0000 7ffa8a9d4740 1 --2- 172.21.15.43:0/1364293483 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563332ecf490 0x563332ed8460 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x563332edaa90 tx=0x7ffa6c009f90 comp rx=0 tx=0).stop 2024-09-19T05:13:12.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.384+0000 7ffa8a9d4740 1 -- 172.21.15.43:0/1364293483 shutdown_connections 2024-09-19T05:13:12.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.384+0000 7ffa8a9d4740 1 --2- 172.21.15.43:0/1364293483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563332ede260 0x563332ed89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:12.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.384+0000 7ffa8a9d4740 1 --2- 172.21.15.43:0/1364293483 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563332ecf490 0x563332ed8460 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:12.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.384+0000 7ffa8a9d4740 1 --2- 172.21.15.43:0/1364293483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563332eceb70 0x563332ee9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:12.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.384+0000 7ffa8a9d4740 1 -- 172.21.15.43:0/1364293483 >> 172.21.15.43:0/1364293483 conn(0x563332ecb200 msgr2=0x563332ee9240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:12.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.384+0000 7ffa8a9d4740 1 -- 172.21.15.43:0/1364293483 shutdown_connections 2024-09-19T05:13:12.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.384+0000 7ffa8a9d4740 1 -- 172.21.15.43:0/1364293483 wait complete. 2024-09-19T05:13:12.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.385+0000 7ffa8a9d4740 1 -- start start 2024-09-19T05:13:12.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.385+0000 7ffa8a9d4740 1 Processor -- start 2024-09-19T05:13:12.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.385+0000 7ffa8a9d4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563332eceb70 0x563332ef14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:12.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.385+0000 7ffa8a9d4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563332ecf490 0x563332ef1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:12.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.385+0000 7ffa8a9d4740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563332ede260 0x563332ef1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:12.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.385+0000 7ffa8a9d4740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ffa801acb10 con 0x563332ede260 2024-09-19T05:13:12.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.385+0000 7ffa8a9d4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffa801ac440 con 0x563332ecf490 2024-09-19T05:13:12.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.385+0000 7ffa8a9d4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffa801a96a0 con 0x563332eceb70 2024-09-19T05:13:12.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.385+0000 7ffa8a9d4740 10 client.0 _ll_register_callbacks cb 0x7ffa6c015480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:12.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.386+0000 7ffa85e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563332ede260 0x563332ef1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:12.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.386+0000 7ffa84a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563332ecf490 0x563332ef1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:12.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.386+0000 7ffa85e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563332ede260 0x563332ef1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40348/0 (socket says 172.21.15.43:40348) 2024-09-19T05:13:12.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.386+0000 7ffa85e00640 1 -- 172.21.15.43:0/2699745910 learned_addr learned my addr 172.21.15.43:0/2699745910 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:12.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.386+0000 7ffa85400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563332eceb70 0x563332ef14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:12.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.386+0000 7ffa84a00640 1 -- 172.21.15.43:0/2699745910 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563332eceb70 msgr2=0x563332ef14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:12.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.386+0000 7ffa84a00640 1 --2- 172.21.15.43:0/2699745910 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563332eceb70 0x563332ef14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:12.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.386+0000 7ffa84a00640 1 -- 172.21.15.43:0/2699745910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563332ede260 msgr2=0x563332ef1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:12.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.386+0000 7ffa84a00640 1 --2- 172.21.15.43:0/2699745910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563332ede260 0x563332ef1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:12.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.386+0000 7ffa84a00640 1 -- 172.21.15.43:0/2699745910 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563332ed6030 con 0x563332ecf490 2024-09-19T05:13:12.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.386+0000 7ffa84a00640 1 --2- 172.21.15.43:0/2699745910 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563332ecf490 0x563332ef1a00 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c008010 tx=0x7ffa6c009370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1bc33c7eeeb416 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:12.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.386+0000 7ffa66a00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:13:12.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.387+0000 7ffa66a00640 1 -- 172.21.15.43:0/2699745910 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c030040 con 0x563332ecf490 2024-09-19T05:13:12.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.387+0000 7ffa8a9d4740 1 -- 172.21.15.43:0/2699745910 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa6c025c70 con 0x563332ecf490 2024-09-19T05:13:12.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.387+0000 7ffa66a00640 1 -- 172.21.15.43:0/2699745910 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7ffa6c018450 con 0x563332ecf490 2024-09-19T05:13:12.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.387+0000 7ffa66a00640 1 -- 172.21.15.43:0/2699745910 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffa6c01c040 con 0x563332ecf490 2024-09-19T05:13:12.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.387+0000 7ffa8a9d4740 10 client.27194 fetch_fsmap learned FSMap version 29 2024-09-19T05:13:12.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.387+0000 7ffa8a9d4740 1 -- 172.21.15.43:0/2699745910 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563332f58d70 con 0x563332ecf490 2024-09-19T05:13:12.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.387+0000 7ffa66a00640 1 -- 172.21.15.43:0/2699745910 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa6c025c70 con 0x563332ecf490 2024-09-19T05:13:12.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.388+0000 7ffa66a00640 1 -- 172.21.15.43:0/2699745910 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ffa6c022040 con 0x563332ecf490 2024-09-19T05:13:12.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.388+0000 7ffa8a9d4740 10 client.27194 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:13:12.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.388+0000 7ffa8a9d4740 10 client.27194 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-19T05:13:12.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.388+0000 7ffa8a9d4740 20 client.27194 populate_metadata read hostname 'smithi043' 2024-09-19T05:13:12.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.388+0000 7ffa8a9d4740 1 --2- 172.21.15.43:0/2699745910 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x563332f59490 0x563332f5b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:12.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.388+0000 7ffa8a9d4740 4 client.27194 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:13:12.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.388+0000 7ffa8a9d4740 1 -- 172.21.15.43:0/2699745910 --> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563332e3ce60 con 0x563332f59490 2024-09-19T05:13:12.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.388+0000 7ffa85400640 1 --2- 172.21.15.43:0/2699745910 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x563332f59490 0x563332f5b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:12.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.389+0000 7ffa85400640 1 --2- 172.21.15.43:0/2699745910 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x563332f59490 0x563332f5b880 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2c80dc68ce0a224d server_cookie=afb6e1df60756a89 in_seq=0 out_seq=0 2024-09-19T05:13:12.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.389+0000 7ffa66a00640 10 client.27194 ms_handle_connect on v2:172.21.15.43:6838/3404555030 2024-09-19T05:13:12.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.389+0000 7ffa66a00640 1 -- 172.21.15.43:0/2699745910 <== mds.1 v2:172.21.15.43:6838/3404555030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563332e3ce60 con 0x563332f59490 2024-09-19T05:13:12.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.389+0000 7ffa66a00640 10 client.27194 handle_command_reply: tid=0 2024-09-19T05:13:12.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.389+0000 7ffa8a9d4740 1 client.27194 shutdown 2024-09-19T05:13:12.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.389+0000 7ffa8a9d4740 1 -- 172.21.15.43:0/2699745910 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563332ecf490 msgr2=0x563332ef1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:12.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.389+0000 7ffa8a9d4740 1 --2- 172.21.15.43:0/2699745910 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563332ecf490 0x563332ef1a00 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c008010 tx=0x7ffa6c009370 comp rx=0 tx=0).stop 2024-09-19T05:13:12.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.389+0000 7ffa8a9d4740 1 -- 172.21.15.43:0/2699745910 shutdown_connections 2024-09-19T05:13:12.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.389+0000 7ffa8a9d4740 1 --2- 172.21.15.43:0/2699745910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563332ede260 0x563332ef1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:12.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.389+0000 7ffa8a9d4740 1 --2- 172.21.15.43:0/2699745910 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563332ecf490 0x563332ef1a00 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:12.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.389+0000 7ffa8a9d4740 1 --2- 172.21.15.43:0/2699745910 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x563332f59490 0x563332f5b880 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:12.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.389+0000 7ffa8a9d4740 1 --2- 172.21.15.43:0/2699745910 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563332eceb70 0x563332ef14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:12.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.389+0000 7ffa66a00640 0 client.27194 ms_handle_reset on v2:172.21.15.43:6838/3404555030 2024-09-19T05:13:12.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.389+0000 7ffa8a9d4740 1 -- 172.21.15.43:0/2699745910 >> 172.21.15.43:0/2699745910 conn(0x563332ecb200 msgr2=0x563332ec9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:12.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.390+0000 7ffa8a9d4740 1 -- 172.21.15.43:0/2699745910 shutdown_connections 2024-09-19T05:13:12.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.390+0000 7ffa8a9d4740 1 -- 172.21.15.43:0/2699745910 wait complete. 2024-09-19T05:13:12.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.390+0000 7ffa8a9d4740 20 client.27194 trim_cache size 0 max 16384 2024-09-19T05:13:12.400 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:13:12.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.390+0000 7ffa87e00640 1 -- 172.21.15.43:0/3790583139 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ffa680a33b0 msgr2=0x7ffa680a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:12.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.390+0000 7ffa87e00640 1 --2- 172.21.15.43:0/3790583139 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ffa680a33b0 0x7ffa680a5870 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x563332ed0710 tx=0x7ffa6c008040 comp rx=0 tx=0).stop 2024-09-19T05:13:12.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.390+0000 7ffa87e00640 1 -- 172.21.15.43:0/3790583139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa80106f10 msgr2=0x7ffa8019cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:12.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.390+0000 7ffa87e00640 1 --2- 172.21.15.43:0/3790583139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa80106f10 0x7ffa8019cdb0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ffa7800da10 tx=0x7ffa7800ded0 comp rx=0 tx=0).stop 2024-09-19T05:13:12.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.391+0000 7ffa87e00640 1 -- 172.21.15.43:0/3790583139 shutdown_connections 2024-09-19T05:13:12.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.391+0000 7ffa87e00640 1 --2- 172.21.15.43:0/3790583139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa80106f10 0x7ffa8019cdb0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:12.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.391+0000 7ffa87e00640 1 --2- 172.21.15.43:0/3790583139 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ffa680a33b0 0x7ffa680a5870 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:12.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.391+0000 7ffa87e00640 1 --2- 172.21.15.43:0/3790583139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa80102790 0x7ffa80198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:12.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.391+0000 7ffa87e00640 1 --2- 172.21.15.43:0/3790583139 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa80101de0 0x7ffa801984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:12.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.391+0000 7ffa87e00640 1 -- 172.21.15.43:0/3790583139 >> 172.21.15.43:0/3790583139 conn(0x7ffa800fd8c0 msgr2=0x7ffa800fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:12.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.391+0000 7ffa87e00640 1 -- 172.21.15.43:0/3790583139 shutdown_connections 2024-09-19T05:13:12.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:12.391+0000 7ffa87e00640 1 -- 172.21.15.43:0/3790583139 wait complete. 2024-09-19T05:13:13.312 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T05:13:13.691 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:13:14.100 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:13 smithi043 ceph-mon[30800]: pgmap v266: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-09-19T05:13:14.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:13 smithi151 ceph-mon[35425]: pgmap v266: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-09-19T05:13:14.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.214+0000 7f13b9e00640 1 -- 172.21.15.43:0/3924103625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13b4106f10 msgr2=0x7f13b410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:14.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.214+0000 7f13b9e00640 1 --2- 172.21.15.43:0/3924103625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13b4106f10 0x7f13b410ea90 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f13ac009ca0 tx=0x7f13ac02a8c0 comp rx=0 tx=0).stop 2024-09-19T05:13:14.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.214+0000 7f13b9e00640 1 -- 172.21.15.43:0/3924103625 shutdown_connections 2024-09-19T05:13:14.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.214+0000 7f13b9e00640 1 --2- 172.21.15.43:0/3924103625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13b4106f10 0x7f13b410ea90 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:14.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.214+0000 7f13b9e00640 1 --2- 172.21.15.43:0/3924103625 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13b4102790 0x7f13b41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:14.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.214+0000 7f13b9e00640 1 --2- 172.21.15.43:0/3924103625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13b4101de0 0x7f13b41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:14.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.214+0000 7f13b9e00640 1 -- 172.21.15.43:0/3924103625 >> 172.21.15.43:0/3924103625 conn(0x7f13b40fd8c0 msgr2=0x7f13b40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:14.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.214+0000 7f13b9e00640 1 -- 172.21.15.43:0/3924103625 shutdown_connections 2024-09-19T05:13:14.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.214+0000 7f13b9e00640 1 -- 172.21.15.43:0/3924103625 wait complete. 2024-09-19T05:13:14.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.215+0000 7f13b9e00640 1 Processor -- start 2024-09-19T05:13:14.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.215+0000 7f13b9e00640 1 -- start start 2024-09-19T05:13:14.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.215+0000 7f13b9e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13b4101de0 0x7f13b419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:14.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.215+0000 7f13b9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13b4102790 0x7f13b419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:14.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.215+0000 7f13b9e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13b4106f10 0x7f13b419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:14.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.215+0000 7f13b9e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f13b4111fc0 con 0x7f13b4102790 2024-09-19T05:13:14.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.215+0000 7f13b9e00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f13b4111e40 con 0x7f13b4106f10 2024-09-19T05:13:14.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.215+0000 7f13b9e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f13b4112140 con 0x7f13b4101de0 2024-09-19T05:13:14.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.216+0000 7f13aa000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13b4102790 0x7f13b419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:14.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.216+0000 7f13aa000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13b4102790 0x7f13b419ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40380/0 (socket says 172.21.15.43:40380) 2024-09-19T05:13:14.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.216+0000 7f13aa000640 1 -- 172.21.15.43:0/3496118153 learned_addr learned my addr 172.21.15.43:0/3496118153 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:14.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.216+0000 7f13b2a00640 1 --2- 172.21.15.43:0/3496118153 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13b4101de0 0x7f13b419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:14.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.216+0000 7f13b3400640 1 --2- 172.21.15.43:0/3496118153 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13b4106f10 0x7f13b419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:14.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.216+0000 7f13b2a00640 1 -- 172.21.15.43:0/3496118153 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13b4106f10 msgr2=0x7f13b419eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:14.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.216+0000 7f13b2a00640 1 --2- 172.21.15.43:0/3496118153 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13b4106f10 0x7f13b419eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:14.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.216+0000 7f13b2a00640 1 -- 172.21.15.43:0/3496118153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13b4102790 msgr2=0x7f13b419ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:14.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.216+0000 7f13b2a00640 1 --2- 172.21.15.43:0/3496118153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13b4102790 0x7f13b419ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:14.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.216+0000 7f13b2a00640 1 -- 172.21.15.43:0/3496118153 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13b419f5b0 con 0x7f13b4101de0 2024-09-19T05:13:14.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.216+0000 7f13aa000640 1 --2- 172.21.15.43:0/3496118153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13b4102790 0x7f13b419ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:13:14.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.216+0000 7f13b3400640 1 --2- 172.21.15.43:0/3496118153 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13b4106f10 0x7f13b419eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:13:14.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.217+0000 7f13b2a00640 1 --2- 172.21.15.43:0/3496118153 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13b4101de0 0x7f13b419a600 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f13a400e9f0 tx=0x7f13a400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:14.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.217+0000 7f13b8a00640 1 -- 172.21.15.43:0/3496118153 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a400ccc0 con 0x7f13b4101de0 2024-09-19T05:13:14.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.217+0000 7f13b9e00640 1 -- 172.21.15.43:0/3496118153 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13b419f8a0 con 0x7f13b4101de0 2024-09-19T05:13:14.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.217+0000 7f13b8a00640 1 -- 172.21.15.43:0/3496118153 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13a4010910 con 0x7f13b4101de0 2024-09-19T05:13:14.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.217+0000 7f13b8a00640 1 -- 172.21.15.43:0/3496118153 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a4018960 con 0x7f13b4101de0 2024-09-19T05:13:14.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.217+0000 7f13b9e00640 1 -- 172.21.15.43:0/3496118153 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13b41a7180 con 0x7f13b4101de0 2024-09-19T05:13:14.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.220+0000 7f13b8a00640 1 -- 172.21.15.43:0/3496118153 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f13a4010ac0 con 0x7f13b4101de0 2024-09-19T05:13:14.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.220+0000 7f13bc8a6740 1 Processor -- start 2024-09-19T05:13:14.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.220+0000 7f13bc8a6740 1 -- start start 2024-09-19T05:13:14.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.220+0000 7f13bc8a6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d4708c70 0x5624d4709050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:14.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.220+0000 7f13bc8a6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624d4709590 0x5624d4713910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:14.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.220+0000 7f13bc8a6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d4713e50 0x5624d4716240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:14.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.220+0000 7f13bc8a6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5624d466f3e0 con 0x5624d4708c70 2024-09-19T05:13:14.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.220+0000 7f13bc8a6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5624d4616d80 con 0x5624d4713e50 2024-09-19T05:13:14.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.220+0000 7f13bc8a6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5624d466b3a0 con 0x5624d4709590 2024-09-19T05:13:14.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.221+0000 7f13b3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d4708c70 0x5624d4709050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:14.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.221+0000 7f13b2a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624d4709590 0x5624d4713910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:14.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.221+0000 7f13b2a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624d4709590 0x5624d4713910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:57526/0 (socket says 172.21.15.43:57526) 2024-09-19T05:13:14.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.221+0000 7f13b3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d4708c70 0x5624d4709050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40388/0 (socket says 172.21.15.43:40388) 2024-09-19T05:13:14.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.221+0000 7f13b2a00640 1 -- 172.21.15.43:0/1844283680 learned_addr learned my addr 172.21.15.43:0/1844283680 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:14.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.221+0000 7f13aa000640 1 --2- 172.21.15.43:0/1844283680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d4713e50 0x5624d4716240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:14.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:13 smithi096 ceph-mon[35085]: pgmap v266: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-09-19T05:13:14.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.221+0000 7f13aa000640 1 -- 172.21.15.43:0/1844283680 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624d4709590 msgr2=0x5624d4713910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:14.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.221+0000 7f13aa000640 1 --2- 172.21.15.43:0/1844283680 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624d4709590 0x5624d4713910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:14.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.221+0000 7f13aa000640 1 -- 172.21.15.43:0/1844283680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d4708c70 msgr2=0x5624d4709050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:14.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.221+0000 7f13aa000640 1 --2- 172.21.15.43:0/1844283680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d4708c70 0x5624d4709050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:14.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.221+0000 7f13aa000640 1 -- 172.21.15.43:0/1844283680 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624d4635ae0 con 0x5624d4713e50 2024-09-19T05:13:14.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.222+0000 7f13aa000640 1 --2- 172.21.15.43:0/1844283680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d4713e50 0x5624d4716240 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f139c0114c0 tx=0x7f139c011980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9848e3e751de50d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:14.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.222+0000 7f13a3e00640 1 -- 172.21.15.43:0/1844283680 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139c00f680 con 0x5624d4713e50 2024-09-19T05:13:14.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.222+0000 7f13a3e00640 1 -- 172.21.15.43:0/1844283680 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f139c00fd00 con 0x5624d4713e50 2024-09-19T05:13:14.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.222+0000 7f13b8a00640 1 --2- 172.21.15.43:0/3496118153 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f13980a33b0 0x7f13980a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:14.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.222+0000 7f13a3e00640 1 -- 172.21.15.43:0/1844283680 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139c019df0 con 0x5624d4713e50 2024-09-19T05:13:14.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.222+0000 7f13b8a00640 1 -- 172.21.15.43:0/3496118153 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f13a40d0170 con 0x7f13b4101de0 2024-09-19T05:13:14.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.222+0000 7f13b3400640 1 --2- 172.21.15.43:0/3496118153 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f13980a33b0 0x7f13980a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:14.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.223+0000 7f13bc8a6740 1 -- 172.21.15.43:0/1844283680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d4713e50 msgr2=0x5624d4716240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:14.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.223+0000 7f13bc8a6740 1 --2- 172.21.15.43:0/1844283680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d4713e50 0x5624d4716240 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f139c0114c0 tx=0x7f139c011980 comp rx=0 tx=0).stop 2024-09-19T05:13:14.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.223+0000 7f13bc8a6740 1 -- 172.21.15.43:0/1844283680 shutdown_connections 2024-09-19T05:13:14.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.223+0000 7f13bc8a6740 1 --2- 172.21.15.43:0/1844283680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d4713e50 0x5624d4716240 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:14.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.223+0000 7f13bc8a6740 1 --2- 172.21.15.43:0/1844283680 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624d4709590 0x5624d4713910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:14.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.223+0000 7f13bc8a6740 1 --2- 172.21.15.43:0/1844283680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d4708c70 0x5624d4709050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:14.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.223+0000 7f13bc8a6740 1 -- 172.21.15.43:0/1844283680 >> 172.21.15.43:0/1844283680 conn(0x5624d4705200 msgr2=0x5624d47055e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:14.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.223+0000 7f13bc8a6740 1 -- 172.21.15.43:0/1844283680 shutdown_connections 2024-09-19T05:13:14.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.223+0000 7f13b3400640 1 --2- 172.21.15.43:0/3496118153 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f13980a33b0 0x7f13980a5870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f13ac009430 tx=0x7f13ac004a40 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:14.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.223+0000 7f13bc8a6740 1 -- 172.21.15.43:0/1844283680 wait complete. 2024-09-19T05:13:14.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.224+0000 7f13bc8a6740 1 -- start start 2024-09-19T05:13:14.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.224+0000 7f13bc8a6740 1 Processor -- start 2024-09-19T05:13:14.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.224+0000 7f13bc8a6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d4708c70 0x5624d4710d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:14.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.224+0000 7f13bc8a6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d4709590 0x5624d47112a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:14.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.224+0000 7f13bc8a6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624d4713e50 0x5624d47117e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:14.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.224+0000 7f13bc8a6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5624d4716a20 con 0x5624d4708c70 2024-09-19T05:13:14.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.224+0000 7f13bc8a6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5624d462a0f0 con 0x5624d4709590 2024-09-19T05:13:14.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.224+0000 7f13bc8a6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5624d4716ba0 con 0x5624d4713e50 2024-09-19T05:13:14.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.224+0000 7f13bc8a6740 10 client.0 _ll_register_callbacks cb 0x5624d4611bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:14.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.225+0000 7f13aa000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d4709590 0x5624d47112a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:14.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.225+0000 7f13b2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d4708c70 0x5624d4710d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:14.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.225+0000 7f13aa000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d4709590 0x5624d47112a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:52752/0 (socket says 172.21.15.43:52752) 2024-09-19T05:13:14.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.225+0000 7f13aa000640 1 -- 172.21.15.43:0/3579784489 learned_addr learned my addr 172.21.15.43:0/3579784489 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:14.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.225+0000 7f13b2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d4708c70 0x5624d4710d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40392/0 (socket says 172.21.15.43:40392) 2024-09-19T05:13:14.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.225+0000 7f13b3400640 1 --2- 172.21.15.43:0/3579784489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624d4713e50 0x5624d47117e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:14.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.225+0000 7f13b2a00640 1 -- 172.21.15.43:0/3579784489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624d4713e50 msgr2=0x5624d47117e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:14.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.225+0000 7f13b2a00640 1 --2- 172.21.15.43:0/3579784489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624d4713e50 0x5624d47117e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:14.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.225+0000 7f13b2a00640 1 -- 172.21.15.43:0/3579784489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d4709590 msgr2=0x5624d47112a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:14.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.225+0000 7f13b2a00640 1 --2- 172.21.15.43:0/3579784489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d4709590 0x5624d47112a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:14.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.225+0000 7f13b2a00640 1 -- 172.21.15.43:0/3579784489 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624d43e03c0 con 0x5624d4708c70 2024-09-19T05:13:14.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.225+0000 7f13aa000640 1 --2- 172.21.15.43:0/3579784489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d4709590 0x5624d47112a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:13:14.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.225+0000 7f13b2a00640 1 --2- 172.21.15.43:0/3579784489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d4708c70 0x5624d4710d60 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f13a40cdeb0 tx=0x7f13a40d38b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f5ec084edea140d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:14.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.225+0000 7f13a0c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:13:14.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.226+0000 7f13a0c00640 1 -- 172.21.15.43:0/3579784489 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a40986e0 con 0x5624d4708c70 2024-09-19T05:13:14.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.226+0000 7f13a0c00640 1 -- 172.21.15.43:0/3579784489 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f13a409ba90 con 0x5624d4708c70 2024-09-19T05:13:14.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.226+0000 7f13bc8a6740 1 -- 172.21.15.43:0/3579784489 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f139c00fd00 con 0x5624d4708c70 2024-09-19T05:13:14.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.226+0000 7f13a0c00640 1 -- 172.21.15.43:0/3579784489 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f13a409bda0 con 0x5624d4708c70 2024-09-19T05:13:14.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.226+0000 7f13bc8a6740 10 client.17184 fetch_fsmap learned FSMap version 29 2024-09-19T05:13:14.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.226+0000 7f13bc8a6740 1 -- 172.21.15.43:0/3579784489 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5624d4722a90 con 0x5624d4708c70 2024-09-19T05:13:14.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.226+0000 7f13a0c00640 1 -- 172.21.15.43:0/3579784489 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f139c00fd00 con 0x5624d4708c70 2024-09-19T05:13:14.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.226+0000 7f13a0c00640 1 -- 172.21.15.43:0/3579784489 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f13a40b5240 con 0x5624d4708c70 2024-09-19T05:13:14.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.227+0000 7f13bc8a6740 10 client.17184 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:13:14.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.227+0000 7f13bc8a6740 10 client.17184 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:13:14.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.227+0000 7f13bc8a6740 20 client.17184 populate_metadata read hostname 'smithi043' 2024-09-19T05:13:14.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.227+0000 7f13bc8a6740 1 --2- 172.21.15.43:0/3579784489 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5624d4723ea0 0x5624d47262b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:14.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.227+0000 7f13bc8a6740 4 client.17184 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:13:14.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.227+0000 7f13bc8a6740 1 -- 172.21.15.43:0/3579784489 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624d467c950 con 0x5624d4723ea0 2024-09-19T05:13:14.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.227+0000 7f13aa000640 1 --2- 172.21.15.43:0/3579784489 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5624d4723ea0 0x5624d47262b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:14.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.227+0000 7f13aa000640 1 --2- 172.21.15.43:0/3579784489 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5624d4723ea0 0x5624d47262b0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0f301abc01ce68a server_cookie=cd0025cfd567c0e5 in_seq=0 out_seq=0 2024-09-19T05:13:14.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.227+0000 7f13a0c00640 10 client.17184 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:13:14.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.228+0000 7f13a0c00640 1 -- 172.21.15.43:0/3579784489 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5624d467c950 con 0x5624d4723ea0 2024-09-19T05:13:14.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.228+0000 7f13a0c00640 10 client.17184 handle_command_reply: tid=0 2024-09-19T05:13:14.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.228+0000 7f13bc8a6740 1 client.17184 shutdown 2024-09-19T05:13:14.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.228+0000 7f13bc8a6740 1 -- 172.21.15.43:0/3579784489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d4708c70 msgr2=0x5624d4710d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:14.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.228+0000 7f13bc8a6740 1 --2- 172.21.15.43:0/3579784489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d4708c70 0x5624d4710d60 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f13a40cdeb0 tx=0x7f13a40d38b0 comp rx=0 tx=0).stop 2024-09-19T05:13:14.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.229+0000 7f13bc8a6740 1 -- 172.21.15.43:0/3579784489 shutdown_connections 2024-09-19T05:13:14.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.229+0000 7f13bc8a6740 1 --2- 172.21.15.43:0/3579784489 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5624d4723ea0 0x5624d47262b0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:14.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.229+0000 7f13bc8a6740 1 --2- 172.21.15.43:0/3579784489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624d4713e50 0x5624d47117e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:14.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.229+0000 7f13bc8a6740 1 --2- 172.21.15.43:0/3579784489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d4709590 0x5624d47112a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:14.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.229+0000 7f13bc8a6740 1 --2- 172.21.15.43:0/3579784489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d4708c70 0x5624d4710d60 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:14.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.229+0000 7f13bc8a6740 1 -- 172.21.15.43:0/3579784489 >> 172.21.15.43:0/3579784489 conn(0x5624d4705200 msgr2=0x5624d470af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:14.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.229+0000 7f13a0c00640 0 client.17184 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:13:14.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.229+0000 7f13bc8a6740 1 -- 172.21.15.43:0/3579784489 shutdown_connections 2024-09-19T05:13:14.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.229+0000 7f13bc8a6740 1 -- 172.21.15.43:0/3579784489 wait complete. 2024-09-19T05:13:14.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.229+0000 7f13bc8a6740 20 client.17184 trim_cache size 0 max 16384 2024-09-19T05:13:14.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.243+0000 7f13bc8a6740 1 Processor -- start 2024-09-19T05:13:14.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.243+0000 7f13bc8a6740 1 -- start start 2024-09-19T05:13:14.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.243+0000 7f13bc8a6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d4708c70 0x5624d470b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:14.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.243+0000 7f13bc8a6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d4709590 0x5624d4712cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:14.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.243+0000 7f13bc8a6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624d4713e50 0x5624d472bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:14.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.243+0000 7f13bc8a6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f13b41afb30 con 0x5624d4708c70 2024-09-19T05:13:14.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.243+0000 7f13bc8a6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f13b41b1670 con 0x5624d4709590 2024-09-19T05:13:14.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.243+0000 7f13bc8a6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f13b41b0200 con 0x5624d4713e50 2024-09-19T05:13:14.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.243+0000 7f13b2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d4708c70 0x5624d470b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:14.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.243+0000 7f13aa000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d4709590 0x5624d4712cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:14.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.243+0000 7f13b2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d4708c70 0x5624d470b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40408/0 (socket says 172.21.15.43:40408) 2024-09-19T05:13:14.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.243+0000 7f13b2a00640 1 -- 172.21.15.43:0/4006232864 learned_addr learned my addr 172.21.15.43:0/4006232864 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:14.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.243+0000 7f13aa000640 1 --2- 172.21.15.43:0/4006232864 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d4709590 0x5624d4712cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:52758/0 (socket says 172.21.15.43:52758) 2024-09-19T05:13:14.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.244+0000 7f13b3400640 1 --2- 172.21.15.43:0/4006232864 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624d4713e50 0x5624d472bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:14.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.244+0000 7f13aa000640 1 -- 172.21.15.43:0/4006232864 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624d4713e50 msgr2=0x5624d472bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:14.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.244+0000 7f13aa000640 1 --2- 172.21.15.43:0/4006232864 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624d4713e50 0x5624d472bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:14.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.244+0000 7f13aa000640 1 -- 172.21.15.43:0/4006232864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d4708c70 msgr2=0x5624d470b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:14.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.244+0000 7f13aa000640 1 --2- 172.21.15.43:0/4006232864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d4708c70 0x5624d470b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:14.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.244+0000 7f13aa000640 1 -- 172.21.15.43:0/4006232864 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624d4708aa0 con 0x5624d4709590 2024-09-19T05:13:14.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.244+0000 7f13b2a00640 1 --2- 172.21.15.43:0/4006232864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d4708c70 0x5624d470b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:13:14.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.244+0000 7f13aa000640 1 --2- 172.21.15.43:0/4006232864 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d4709590 0x5624d4712cd0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f139c019970 tx=0x7f139c009600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4130652d1bc48f79 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:14.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.244+0000 7f13a3400640 1 -- 172.21.15.43:0/4006232864 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139c0197c0 con 0x5624d4709590 2024-09-19T05:13:14.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.244+0000 7f13a3400640 1 -- 172.21.15.43:0/4006232864 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f139c0199a0 con 0x5624d4709590 2024-09-19T05:13:14.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.244+0000 7f13a3400640 1 -- 172.21.15.43:0/4006232864 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139c004680 con 0x5624d4709590 2024-09-19T05:13:14.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.245+0000 7f13bc8a6740 1 -- 172.21.15.43:0/4006232864 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d4709590 msgr2=0x5624d4712cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:14.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.245+0000 7f13bc8a6740 1 --2- 172.21.15.43:0/4006232864 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d4709590 0x5624d4712cd0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f139c019970 tx=0x7f139c009600 comp rx=0 tx=0).stop 2024-09-19T05:13:14.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.245+0000 7f13bc8a6740 1 -- 172.21.15.43:0/4006232864 shutdown_connections 2024-09-19T05:13:14.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.245+0000 7f13bc8a6740 1 --2- 172.21.15.43:0/4006232864 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624d4713e50 0x5624d472bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:14.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.245+0000 7f13bc8a6740 1 --2- 172.21.15.43:0/4006232864 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d4709590 0x5624d4712cd0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:14.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.245+0000 7f13bc8a6740 1 --2- 172.21.15.43:0/4006232864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d4708c70 0x5624d470b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:14.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.245+0000 7f13bc8a6740 1 -- 172.21.15.43:0/4006232864 >> 172.21.15.43:0/4006232864 conn(0x5624d4705200 msgr2=0x5624d47125c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:14.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.245+0000 7f13bc8a6740 1 -- 172.21.15.43:0/4006232864 shutdown_connections 2024-09-19T05:13:14.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.245+0000 7f13bc8a6740 1 -- 172.21.15.43:0/4006232864 wait complete. 2024-09-19T05:13:14.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.246+0000 7f13bc8a6740 1 -- start start 2024-09-19T05:13:14.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.246+0000 7f13bc8a6740 1 Processor -- start 2024-09-19T05:13:14.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.246+0000 7f13bc8a6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624d4708c70 0x5624d471c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:14.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.246+0000 7f13bc8a6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d4709590 0x5624d470d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:14.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.246+0000 7f13bc8a6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d4713e50 0x5624d470dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:14.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.246+0000 7f13bc8a6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f13b41b0fa0 con 0x5624d4713e50 2024-09-19T05:13:14.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.246+0000 7f13bc8a6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f13b41b08d0 con 0x5624d4709590 2024-09-19T05:13:14.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.246+0000 7f13bc8a6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f13b41adb30 con 0x5624d4708c70 2024-09-19T05:13:14.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.246+0000 7f13bc8a6740 10 client.0 _ll_register_callbacks cb 0x7f13a40d39b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:14.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.247+0000 7f13b3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d4713e50 0x5624d470dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:14.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.247+0000 7f13aa000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d4709590 0x5624d470d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:14.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.247+0000 7f13aa000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d4709590 0x5624d470d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:52766/0 (socket says 172.21.15.43:52766) 2024-09-19T05:13:14.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.247+0000 7f13b3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d4713e50 0x5624d470dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40424/0 (socket says 172.21.15.43:40424) 2024-09-19T05:13:14.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.247+0000 7f13aa000640 1 -- 172.21.15.43:0/1410834274 learned_addr learned my addr 172.21.15.43:0/1410834274 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:14.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.247+0000 7f13b2a00640 1 --2- 172.21.15.43:0/1410834274 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624d4708c70 0x5624d471c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:14.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.247+0000 7f13aa000640 1 -- 172.21.15.43:0/1410834274 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624d4708c70 msgr2=0x5624d471c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:14.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.247+0000 7f13aa000640 1 --2- 172.21.15.43:0/1410834274 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624d4708c70 0x5624d471c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:14.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.247+0000 7f13aa000640 1 -- 172.21.15.43:0/1410834274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d4713e50 msgr2=0x5624d470dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:14.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.247+0000 7f13aa000640 1 --2- 172.21.15.43:0/1410834274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d4713e50 0x5624d470dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:14.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.247+0000 7f13aa000640 1 -- 172.21.15.43:0/1410834274 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624d4700770 con 0x5624d4709590 2024-09-19T05:13:14.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.247+0000 7f13aa000640 1 --2- 172.21.15.43:0/1410834274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d4709590 0x5624d470d840 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f139c006010 tx=0x7f139c005a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f55926840a7e7bf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:14.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.247+0000 7f13a0c00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:13:14.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.248+0000 7f13a0c00640 1 -- 172.21.15.43:0/1410834274 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139c00faa0 con 0x5624d4709590 2024-09-19T05:13:14.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.248+0000 7f13bc8a6740 1 -- 172.21.15.43:0/1410834274 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f139c0199a0 con 0x5624d4709590 2024-09-19T05:13:14.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.248+0000 7f13a0c00640 1 -- 172.21.15.43:0/1410834274 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f139c00fc40 con 0x5624d4709590 2024-09-19T05:13:14.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.248+0000 7f13a0c00640 1 -- 172.21.15.43:0/1410834274 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f139c039050 con 0x5624d4709590 2024-09-19T05:13:14.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.248+0000 7f13bc8a6740 10 client.27212 fetch_fsmap learned FSMap version 29 2024-09-19T05:13:14.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.248+0000 7f13bc8a6740 1 -- 172.21.15.43:0/1410834274 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5624d478e680 con 0x5624d4709590 2024-09-19T05:13:14.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.248+0000 7f13a0c00640 1 -- 172.21.15.43:0/1410834274 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f139c0199a0 con 0x5624d4709590 2024-09-19T05:13:14.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.248+0000 7f13a0c00640 1 -- 172.21.15.43:0/1410834274 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f139c004ba0 con 0x5624d4709590 2024-09-19T05:13:14.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.248+0000 7f13bc8a6740 10 client.27212 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:13:14.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.249+0000 7f13bc8a6740 10 client.27212 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:13:14.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.249+0000 7f13bc8a6740 20 client.27212 populate_metadata read hostname 'smithi043' 2024-09-19T05:13:14.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.249+0000 7f13bc8a6740 1 --2- 172.21.15.43:0/1410834274 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5624d478ebd0 0x5624d4790ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:14.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.249+0000 7f13bc8a6740 4 client.27212 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:13:14.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.249+0000 7f13bc8a6740 1 -- 172.21.15.43:0/1410834274 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5624d4676e60 con 0x5624d478ebd0 2024-09-19T05:13:14.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.249+0000 7f13b2a00640 1 --2- 172.21.15.43:0/1410834274 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5624d478ebd0 0x5624d4790ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:14.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.249+0000 7f13b2a00640 1 --2- 172.21.15.43:0/1410834274 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5624d478ebd0 0x5624d4790ff0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5c9f1ee0330142b server_cookie=64b474f474049bb8 in_seq=0 out_seq=0 2024-09-19T05:13:14.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.249+0000 7f13a0c00640 10 client.27212 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:13:14.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.250+0000 7f13a0c00640 1 -- 172.21.15.43:0/1410834274 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5624d4676e60 con 0x5624d478ebd0 2024-09-19T05:13:14.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.250+0000 7f13a0c00640 10 client.27212 handle_command_reply: tid=0 2024-09-19T05:13:14.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.250+0000 7f13bc8a6740 1 client.27212 shutdown 2024-09-19T05:13:14.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.250+0000 7f13bc8a6740 1 -- 172.21.15.43:0/1410834274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d4709590 msgr2=0x5624d470d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:14.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.250+0000 7f13bc8a6740 1 --2- 172.21.15.43:0/1410834274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d4709590 0x5624d470d840 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f139c006010 tx=0x7f139c005a70 comp rx=0 tx=0).stop 2024-09-19T05:13:14.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.250+0000 7f13bc8a6740 1 -- 172.21.15.43:0/1410834274 shutdown_connections 2024-09-19T05:13:14.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.250+0000 7f13bc8a6740 1 --2- 172.21.15.43:0/1410834274 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5624d478ebd0 0x5624d4790ff0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:14.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.250+0000 7f13bc8a6740 1 --2- 172.21.15.43:0/1410834274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624d4713e50 0x5624d470dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:14.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.250+0000 7f13bc8a6740 1 --2- 172.21.15.43:0/1410834274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5624d4709590 0x5624d470d840 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:14.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.250+0000 7f13a0c00640 0 client.27212 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:13:14.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.250+0000 7f13bc8a6740 1 --2- 172.21.15.43:0/1410834274 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624d4708c70 0x5624d471c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:14.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.250+0000 7f13bc8a6740 1 -- 172.21.15.43:0/1410834274 >> 172.21.15.43:0/1410834274 conn(0x5624d4705200 msgr2=0x5624d4704380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:14.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.250+0000 7f13bc8a6740 1 -- 172.21.15.43:0/1410834274 shutdown_connections 2024-09-19T05:13:14.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.250+0000 7f13bc8a6740 1 -- 172.21.15.43:0/1410834274 wait complete. 2024-09-19T05:13:14.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.250+0000 7f13bc8a6740 20 client.27212 trim_cache size 0 max 16384 2024-09-19T05:13:14.256 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:13:14.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.251+0000 7f13b9e00640 1 -- 172.21.15.43:0/3496118153 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f13980a33b0 msgr2=0x7f13980a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:14.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.251+0000 7f13b9e00640 1 --2- 172.21.15.43:0/3496118153 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f13980a33b0 0x7f13980a5870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f13ac009430 tx=0x7f13ac004a40 comp rx=0 tx=0).stop 2024-09-19T05:13:14.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.251+0000 7f13b9e00640 1 -- 172.21.15.43:0/3496118153 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13b4101de0 msgr2=0x7f13b419a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:14.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.251+0000 7f13b9e00640 1 --2- 172.21.15.43:0/3496118153 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13b4101de0 0x7f13b419a600 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f13a400e9f0 tx=0x7f13a400eeb0 comp rx=0 tx=0).stop 2024-09-19T05:13:14.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.251+0000 7f13b9e00640 1 -- 172.21.15.43:0/3496118153 shutdown_connections 2024-09-19T05:13:14.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.251+0000 7f13b9e00640 1 --2- 172.21.15.43:0/3496118153 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f13980a33b0 0x7f13980a5870 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:14.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.251+0000 7f13b9e00640 1 --2- 172.21.15.43:0/3496118153 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13b4106f10 0x7f13b419eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:14.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.251+0000 7f13b9e00640 1 --2- 172.21.15.43:0/3496118153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13b4102790 0x7f13b419ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:14.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.251+0000 7f13b9e00640 1 --2- 172.21.15.43:0/3496118153 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13b4101de0 0x7f13b419a600 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:14.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.251+0000 7f13b9e00640 1 -- 172.21.15.43:0/3496118153 >> 172.21.15.43:0/3496118153 conn(0x7f13b40fd8c0 msgr2=0x7f13b40fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:14.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.251+0000 7f13b9e00640 1 -- 172.21.15.43:0/3496118153 shutdown_connections 2024-09-19T05:13:14.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:14.251+0000 7f13b9e00640 1 -- 172.21.15.43:0/3496118153 wait complete. 2024-09-19T05:13:16.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:15 smithi151 ceph-mon[35425]: pgmap v267: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-19T05:13:16.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:15 smithi096 ceph-mon[35085]: pgmap v267: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-19T05:13:16.262 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T05:13:16.263 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T05:13:16.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:15 smithi043 ceph-mon[30800]: pgmap v267: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-19T05:13:16.653 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:13:17.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.202+0000 7f8e22c00640 1 -- 172.21.15.43:0/1032716999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e1c112a30 msgr2=0x7f8e1c114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:17.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.202+0000 7f8e22c00640 1 --2- 172.21.15.43:0/1032716999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e1c112a30 0x7f8e1c114e20 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f8e14009ca0 tx=0x7f8e1402a8c0 comp rx=0 tx=0).stop 2024-09-19T05:13:17.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.202+0000 7f8e22c00640 1 -- 172.21.15.43:0/1032716999 shutdown_connections 2024-09-19T05:13:17.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.202+0000 7f8e22c00640 1 --2- 172.21.15.43:0/1032716999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e1c112a30 0x7f8e1c114e20 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:17.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.202+0000 7f8e22c00640 1 --2- 172.21.15.43:0/1032716999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e1c077040 0x7f8e1c1124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:17.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.202+0000 7f8e22c00640 1 --2- 172.21.15.43:0/1032716999 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e1c076720 0x7f8e1c076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:17.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.202+0000 7f8e22c00640 1 -- 172.21.15.43:0/1032716999 >> 172.21.15.43:0/1032716999 conn(0x7f8e1c0ffa90 msgr2=0x7f8e1c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:17.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.202+0000 7f8e22c00640 1 -- 172.21.15.43:0/1032716999 shutdown_connections 2024-09-19T05:13:17.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.202+0000 7f8e22c00640 1 -- 172.21.15.43:0/1032716999 wait complete. 2024-09-19T05:13:17.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.203+0000 7f8e22c00640 1 Processor -- start 2024-09-19T05:13:17.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.203+0000 7f8e22c00640 1 -- start start 2024-09-19T05:13:17.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.203+0000 7f8e22c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e1c076720 0x7f8e1c1a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:17.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.203+0000 7f8e22c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e1c077040 0x7f8e1c1a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:17.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.203+0000 7f8e22c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e1c112a30 0x7f8e1c19e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:17.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.203+0000 7f8e22c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1c1185f0 con 0x7f8e1c077040 2024-09-19T05:13:17.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.203+0000 7f8e22c00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1c118470 con 0x7f8e1c076720 2024-09-19T05:13:17.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.203+0000 7f8e22c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1c118770 con 0x7f8e1c112a30 2024-09-19T05:13:17.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.204+0000 7f8e1b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e1c077040 0x7f8e1c1a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:17.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.204+0000 7f8e1be00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e1c076720 0x7f8e1c1a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:17.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.204+0000 7f8e1be00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e1c076720 0x7f8e1c1a4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:41608/0 (socket says 172.21.15.43:41608) 2024-09-19T05:13:17.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.204+0000 7f8e1be00640 1 -- 172.21.15.43:0/2693958466 learned_addr learned my addr 172.21.15.43:0/2693958466 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:17.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.204+0000 7f8e20c00640 1 --2- 172.21.15.43:0/2693958466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e1c112a30 0x7f8e1c19e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:17.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.204+0000 7f8e1be00640 1 -- 172.21.15.43:0/2693958466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e1c112a30 msgr2=0x7f8e1c19e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:17.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.204+0000 7f8e1be00640 1 --2- 172.21.15.43:0/2693958466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e1c112a30 0x7f8e1c19e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:17.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.204+0000 7f8e1be00640 1 -- 172.21.15.43:0/2693958466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e1c077040 msgr2=0x7f8e1c1a4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:17.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.204+0000 7f8e1be00640 1 --2- 172.21.15.43:0/2693958466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e1c077040 0x7f8e1c1a4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:17.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.204+0000 7f8e1be00640 1 -- 172.21.15.43:0/2693958466 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e1c19f050 con 0x7f8e1c076720 2024-09-19T05:13:17.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.204+0000 7f8e1be00640 1 --2- 172.21.15.43:0/2693958466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e1c076720 0x7f8e1c1a4760 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c00e9f0 tx=0x7f8e0c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:17.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.205+0000 7f8e18c00640 1 -- 172.21.15.43:0/2693958466 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c00cd80 con 0x7f8e1c076720 2024-09-19T05:13:17.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.205+0000 7f8e18c00640 1 -- 172.21.15.43:0/2693958466 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e0c005500 con 0x7f8e1c076720 2024-09-19T05:13:17.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.205+0000 7f8e18c00640 1 -- 172.21.15.43:0/2693958466 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c010690 con 0x7f8e1c076720 2024-09-19T05:13:17.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.205+0000 7f8e22c00640 1 -- 172.21.15.43:0/2693958466 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e1c19f340 con 0x7f8e1c076720 2024-09-19T05:13:17.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.205+0000 7f8e22c00640 1 -- 172.21.15.43:0/2693958466 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e1c1ab630 con 0x7f8e1c076720 2024-09-19T05:13:17.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.209+0000 7f8e257bb740 1 Processor -- start 2024-09-19T05:13:17.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.209+0000 7f8e257bb740 1 -- start start 2024-09-19T05:13:17.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.210+0000 7f8e257bb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650fffcaac0 0x5650fffcaea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:17.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.210+0000 7f8e257bb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650fffcb3e0 0x5650fffd5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:17.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.210+0000 7f8e257bb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650fffd5ca0 0x5650fffd8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:17.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.210+0000 7f8e257bb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5650fff31290 con 0x5650fffcb3e0 2024-09-19T05:13:17.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.210+0000 7f8e257bb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5650ffed8b40 con 0x5650fffcaac0 2024-09-19T05:13:17.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.210+0000 7f8e257bb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5650fff2d250 con 0x5650fffd5ca0 2024-09-19T05:13:17.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.210+0000 7f8e18c00640 1 -- 172.21.15.43:0/2693958466 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f8e0c0046c0 con 0x7f8e1c076720 2024-09-19T05:13:17.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.210+0000 7f8e20c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650fffcaac0 0x5650fffcaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:17.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.210+0000 7f8e20c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650fffcaac0 0x5650fffcaea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:41612/0 (socket says 172.21.15.43:41612) 2024-09-19T05:13:17.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.210+0000 7f8e20c00640 1 -- 172.21.15.43:0/2783009314 learned_addr learned my addr 172.21.15.43:0/2783009314 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:17.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.210+0000 7f8e1b400640 1 --2- 172.21.15.43:0/2783009314 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650fffd5ca0 0x5650fffd8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:17.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.210+0000 7f8e1be00640 1 --2- 172.21.15.43:0/2783009314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650fffcb3e0 0x5650fffd5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:17.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.210+0000 7f8e20c00640 1 -- 172.21.15.43:0/2783009314 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650fffd5ca0 msgr2=0x5650fffd8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:17.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.210+0000 7f8e20c00640 1 --2- 172.21.15.43:0/2783009314 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650fffd5ca0 0x5650fffd8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:17.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.210+0000 7f8e20c00640 1 -- 172.21.15.43:0/2783009314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650fffcb3e0 msgr2=0x5650fffd5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:17.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.210+0000 7f8e20c00640 1 --2- 172.21.15.43:0/2783009314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650fffcb3e0 0x5650fffd5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:17.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.210+0000 7f8e20c00640 1 -- 172.21.15.43:0/2783009314 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650ffca26a0 con 0x5650fffcaac0 2024-09-19T05:13:17.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.211+0000 7f8e20c00640 1 --2- 172.21.15.43:0/2783009314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650fffcaac0 0x5650fffcaea0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c07aa10 tx=0x7f8e14005f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f752cffede860b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:17.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.211+0000 7f8e11200640 1 -- 172.21.15.43:0/2783009314 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e14031db0 con 0x5650fffcaac0 2024-09-19T05:13:17.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.211+0000 7f8e11200640 1 -- 172.21.15.43:0/2783009314 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e14036030 con 0x5650fffcaac0 2024-09-19T05:13:17.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.211+0000 7f8e11200640 1 -- 172.21.15.43:0/2783009314 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e140042a0 con 0x5650fffcaac0 2024-09-19T05:13:17.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.212+0000 7f8e18c00640 1 --2- 172.21.15.43:0/2693958466 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f8e000a3430 0x7f8e000a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:17.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.212+0000 7f8e18c00640 1 -- 172.21.15.43:0/2693958466 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f8e0c0d0110 con 0x7f8e1c076720 2024-09-19T05:13:17.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.212+0000 7f8e257bb740 1 -- 172.21.15.43:0/2783009314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650fffcaac0 msgr2=0x5650fffcaea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:17.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.212+0000 7f8e257bb740 1 --2- 172.21.15.43:0/2783009314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650fffcaac0 0x5650fffcaea0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c07aa10 tx=0x7f8e14005f80 comp rx=0 tx=0).stop 2024-09-19T05:13:17.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.212+0000 7f8e1be00640 1 --2- 172.21.15.43:0/2693958466 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f8e000a3430 0x7f8e000a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:17.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.212+0000 7f8e257bb740 1 -- 172.21.15.43:0/2783009314 shutdown_connections 2024-09-19T05:13:17.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.212+0000 7f8e257bb740 1 --2- 172.21.15.43:0/2783009314 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650fffd5ca0 0x5650fffd8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:17.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.212+0000 7f8e257bb740 1 --2- 172.21.15.43:0/2783009314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650fffcb3e0 0x5650fffd5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:17.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.212+0000 7f8e257bb740 1 --2- 172.21.15.43:0/2783009314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650fffcaac0 0x5650fffcaea0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:17.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.212+0000 7f8e257bb740 1 -- 172.21.15.43:0/2783009314 >> 172.21.15.43:0/2783009314 conn(0x5650fffc7050 msgr2=0x5650fffc7430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:17.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.212+0000 7f8e257bb740 1 -- 172.21.15.43:0/2783009314 shutdown_connections 2024-09-19T05:13:17.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.212+0000 7f8e257bb740 1 -- 172.21.15.43:0/2783009314 wait complete. 2024-09-19T05:13:17.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.213+0000 7f8e1be00640 1 --2- 172.21.15.43:0/2693958466 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f8e000a3430 0x7f8e000a58f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x5650fffc7eb0 tx=0x7f8e0c0bc040 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:17.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.213+0000 7f8e257bb740 1 -- start start 2024-09-19T05:13:17.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.213+0000 7f8e257bb740 1 Processor -- start 2024-09-19T05:13:17.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.214+0000 7f8e257bb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650fffcaac0 0x5650fffd55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:17.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.214+0000 7f8e257bb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650fffcb3e0 0x5650fffd28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:17.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.214+0000 7f8e257bb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650fffd5ca0 0x5650fffd2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:17.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.214+0000 7f8e257bb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5650fffd8790 con 0x5650fffd5ca0 2024-09-19T05:13:17.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.214+0000 7f8e257bb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5650ffeebfa0 con 0x5650fffcb3e0 2024-09-19T05:13:17.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.214+0000 7f8e257bb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5650fffd8910 con 0x5650fffcaac0 2024-09-19T05:13:17.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.214+0000 7f8e257bb740 10 client.0 _ll_register_callbacks cb 0x5650ffc3fba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:17.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.214+0000 7f8e20c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650fffcaac0 0x5650fffd55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:17.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.214+0000 7f8e1b400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650fffcb3e0 0x5650fffd28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:17.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.214+0000 7f8e1b400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650fffcb3e0 0x5650fffd28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:41628/0 (socket says 172.21.15.43:41628) 2024-09-19T05:13:17.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.214+0000 7f8e1b400640 1 -- 172.21.15.43:0/2383561756 learned_addr learned my addr 172.21.15.43:0/2383561756 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:17.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.214+0000 7f8e20c00640 1 --2- 172.21.15.43:0/2383561756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650fffd5ca0 0x5650fffd2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:17.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.215+0000 7f8e1b400640 1 -- 172.21.15.43:0/2383561756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650fffcaac0 msgr2=0x5650fffd55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:17.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.215+0000 7f8e1b400640 1 --2- 172.21.15.43:0/2383561756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650fffcaac0 0x5650fffd55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:17.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.215+0000 7f8e1b400640 1 -- 172.21.15.43:0/2383561756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650fffd5ca0 msgr2=0x5650fffd2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:17.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.215+0000 7f8e1b400640 1 --2- 172.21.15.43:0/2383561756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650fffd5ca0 0x5650fffd2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:17.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.215+0000 7f8e1b400640 1 -- 172.21.15.43:0/2383561756 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650fffd3630 con 0x5650fffcb3e0 2024-09-19T05:13:17.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.215+0000 7f8e1b400640 1 --2- 172.21.15.43:0/2383561756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650fffcb3e0 0x5650fffd28a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8e08012700 tx=0x7f8e08012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79fffdc142dd311 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:17.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.215+0000 7f8e05600640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:13:17.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.215+0000 7f8e05600640 1 -- 172.21.15.43:0/2383561756 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0800f920 con 0x5650fffcb3e0 2024-09-19T05:13:17.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.215+0000 7f8e05600640 1 -- 172.21.15.43:0/2383561756 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f8e0800fac0 con 0x5650fffcb3e0 2024-09-19T05:13:17.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.215+0000 7f8e257bb740 1 -- 172.21.15.43:0/2383561756 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e14036030 con 0x5650fffcb3e0 2024-09-19T05:13:17.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.215+0000 7f8e05600640 1 -- 172.21.15.43:0/2383561756 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8e0800fcc0 con 0x5650fffcb3e0 2024-09-19T05:13:17.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.216+0000 7f8e257bb740 10 client.27224 fetch_fsmap learned FSMap version 29 2024-09-19T05:13:17.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.216+0000 7f8e257bb740 1 -- 172.21.15.43:0/2383561756 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5650fffe4760 con 0x5650fffcb3e0 2024-09-19T05:13:17.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.216+0000 7f8e05600640 1 -- 172.21.15.43:0/2383561756 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e14036030 con 0x5650fffcb3e0 2024-09-19T05:13:17.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.216+0000 7f8e05600640 1 -- 172.21.15.43:0/2383561756 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8e08010550 con 0x5650fffcb3e0 2024-09-19T05:13:17.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.216+0000 7f8e257bb740 10 client.27224 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:13:17.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.216+0000 7f8e257bb740 10 client.27224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:13:17.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.216+0000 7f8e257bb740 20 client.27224 populate_metadata read hostname 'smithi043' 2024-09-19T05:13:17.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.216+0000 7f8e257bb740 1 --2- 172.21.15.43:0/2383561756 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5650fffe5c20 0x5650fffe8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:17.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.216+0000 7f8e257bb740 4 client.27224 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:13:17.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.216+0000 7f8e257bb740 1 -- 172.21.15.43:0/2383561756 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650fff3e800 con 0x5650fffe5c20 2024-09-19T05:13:17.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.217+0000 7f8e20c00640 1 --2- 172.21.15.43:0/2383561756 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5650fffe5c20 0x5650fffe8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:17.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.217+0000 7f8e20c00640 1 --2- 172.21.15.43:0/2383561756 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5650fffe5c20 0x5650fffe8060 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75cb60d11f58ff6 server_cookie=582df20400216234 in_seq=0 out_seq=0 2024-09-19T05:13:17.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.217+0000 7f8e05600640 10 client.27224 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:13:17.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.218+0000 7f8e05600640 1 -- 172.21.15.43:0/2383561756 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5650fff3e800 con 0x5650fffe5c20 2024-09-19T05:13:17.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.218+0000 7f8e05600640 10 client.27224 handle_command_reply: tid=0 2024-09-19T05:13:17.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.218+0000 7f8e257bb740 1 client.27224 shutdown 2024-09-19T05:13:17.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.218+0000 7f8e257bb740 1 -- 172.21.15.43:0/2383561756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650fffcb3e0 msgr2=0x5650fffd28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:17.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.218+0000 7f8e257bb740 1 --2- 172.21.15.43:0/2383561756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650fffcb3e0 0x5650fffd28a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8e08012700 tx=0x7f8e08012bc0 comp rx=0 tx=0).stop 2024-09-19T05:13:17.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.218+0000 7f8e257bb740 1 -- 172.21.15.43:0/2383561756 shutdown_connections 2024-09-19T05:13:17.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.218+0000 7f8e257bb740 1 --2- 172.21.15.43:0/2383561756 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5650fffe5c20 0x5650fffe8060 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:17.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.218+0000 7f8e257bb740 1 --2- 172.21.15.43:0/2383561756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650fffd5ca0 0x5650fffd2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:17.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.218+0000 7f8e257bb740 1 --2- 172.21.15.43:0/2383561756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650fffcb3e0 0x5650fffd28a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:17.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.218+0000 7f8e257bb740 1 --2- 172.21.15.43:0/2383561756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650fffcaac0 0x5650fffd55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:17.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.218+0000 7f8e05600640 0 client.27224 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:13:17.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.218+0000 7f8e257bb740 1 -- 172.21.15.43:0/2383561756 >> 172.21.15.43:0/2383561756 conn(0x5650fffc7050 msgr2=0x5650fffc61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:17.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.218+0000 7f8e257bb740 1 -- 172.21.15.43:0/2383561756 shutdown_connections 2024-09-19T05:13:17.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.218+0000 7f8e257bb740 1 -- 172.21.15.43:0/2383561756 wait complete. 2024-09-19T05:13:17.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.219+0000 7f8e257bb740 20 client.27224 trim_cache size 0 max 16384 2024-09-19T05:13:17.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.237+0000 7f8e257bb740 1 Processor -- start 2024-09-19T05:13:17.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.237+0000 7f8e257bb740 1 -- start start 2024-09-19T05:13:17.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.237+0000 7f8e257bb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650fffcaac0 0x5650fffd1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:17.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.237+0000 7f8e257bb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650fffcb3e0 0x5650fffed930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:17.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.237+0000 7f8e257bb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650fffd5ca0 0x5650fffede70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:17.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.237+0000 7f8e257bb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1c1b3f50 con 0x5650fffcaac0 2024-09-19T05:13:17.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.237+0000 7f8e257bb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1c1b5a90 con 0x5650fffd5ca0 2024-09-19T05:13:17.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.237+0000 7f8e257bb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1c1b4620 con 0x5650fffcb3e0 2024-09-19T05:13:17.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.237+0000 7f8e20c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650fffcaac0 0x5650fffd1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:17.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.237+0000 7f8e20c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650fffcaac0 0x5650fffd1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50448/0 (socket says 172.21.15.43:50448) 2024-09-19T05:13:17.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.237+0000 7f8e20c00640 1 -- 172.21.15.43:0/601422863 learned_addr learned my addr 172.21.15.43:0/601422863 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:17.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.238+0000 7f8e1b400640 1 --2- 172.21.15.43:0/601422863 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650fffcb3e0 0x5650fffed930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:17.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.238+0000 7f8e20c00640 1 --2- 172.21.15.43:0/601422863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650fffd5ca0 0x5650fffede70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:17.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.238+0000 7f8e20c00640 1 -- 172.21.15.43:0/601422863 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650fffcb3e0 msgr2=0x5650fffed930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:17.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.238+0000 7f8e20c00640 1 --2- 172.21.15.43:0/601422863 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650fffcb3e0 0x5650fffed930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:17.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.238+0000 7f8e20c00640 1 -- 172.21.15.43:0/601422863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650fffd5ca0 msgr2=0x5650fffede70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:17.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.238+0000 7f8e20c00640 1 --2- 172.21.15.43:0/601422863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650fffd5ca0 0x5650fffede70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:17.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.238+0000 7f8e20c00640 1 -- 172.21.15.43:0/601422863 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650fffd33e0 con 0x5650fffcaac0 2024-09-19T05:13:17.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.238+0000 7f8e20c00640 1 --2- 172.21.15.43:0/601422863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650fffcaac0 0x5650fffd1880 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f8e1402f540 tx=0x7f8e14031980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=387611a53d29878e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:17.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.238+0000 7f8e07e00640 1 -- 172.21.15.43:0/601422863 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1404a040 con 0x5650fffcaac0 2024-09-19T05:13:17.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.239+0000 7f8e07e00640 1 -- 172.21.15.43:0/601422863 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e14036070 con 0x5650fffcaac0 2024-09-19T05:13:17.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.239+0000 7f8e07e00640 1 -- 172.21.15.43:0/601422863 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e14004670 con 0x5650fffcaac0 2024-09-19T05:13:17.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.239+0000 7f8e257bb740 1 -- 172.21.15.43:0/601422863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650fffcaac0 msgr2=0x5650fffd1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:17.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.239+0000 7f8e257bb740 1 --2- 172.21.15.43:0/601422863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650fffcaac0 0x5650fffd1880 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f8e1402f540 tx=0x7f8e14031980 comp rx=0 tx=0).stop 2024-09-19T05:13:17.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.239+0000 7f8e257bb740 1 -- 172.21.15.43:0/601422863 shutdown_connections 2024-09-19T05:13:17.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.239+0000 7f8e257bb740 1 --2- 172.21.15.43:0/601422863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650fffd5ca0 0x5650fffede70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:17.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.239+0000 7f8e257bb740 1 --2- 172.21.15.43:0/601422863 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650fffcb3e0 0x5650fffed930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:17.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.239+0000 7f8e257bb740 1 --2- 172.21.15.43:0/601422863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650fffcaac0 0x5650fffd1880 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:17.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.239+0000 7f8e257bb740 1 -- 172.21.15.43:0/601422863 >> 172.21.15.43:0/601422863 conn(0x5650fffc7050 msgr2=0x5650fffd1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:17.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.239+0000 7f8e257bb740 1 -- 172.21.15.43:0/601422863 shutdown_connections 2024-09-19T05:13:17.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.239+0000 7f8e257bb740 1 -- 172.21.15.43:0/601422863 wait complete. 2024-09-19T05:13:17.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.240+0000 7f8e257bb740 1 -- start start 2024-09-19T05:13:17.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.240+0000 7f8e257bb740 1 Processor -- start 2024-09-19T05:13:17.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.241+0000 7f8e257bb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650fffcaac0 0x5650fffd9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:17.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.241+0000 7f8e257bb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650fffcb3e0 0x5650fffd9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:17.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.241+0000 7f8e257bb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650fffd5ca0 0x5650fffda530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:17.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.241+0000 7f8e257bb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1c1b53c0 con 0x5650fffd5ca0 2024-09-19T05:13:17.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.241+0000 7f8e257bb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1c1b4cf0 con 0x5650fffcaac0 2024-09-19T05:13:17.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.241+0000 7f8e257bb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1c1b1f50 con 0x5650fffcb3e0 2024-09-19T05:13:17.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.241+0000 7f8e257bb740 10 client.0 _ll_register_callbacks cb 0x5650fffd3800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:17.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.241+0000 7f8e20c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650fffcaac0 0x5650fffd9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:17.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.241+0000 7f8e20c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650fffcaac0 0x5650fffd9ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:41650/0 (socket says 172.21.15.43:41650) 2024-09-19T05:13:17.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.241+0000 7f8e20c00640 1 -- 172.21.15.43:0/2946619068 learned_addr learned my addr 172.21.15.43:0/2946619068 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:17.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.241+0000 7f8e20c00640 1 --2- 172.21.15.43:0/2946619068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650fffd5ca0 0x5650fffda530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:17.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.241+0000 7f8e1b400640 1 --2- 172.21.15.43:0/2946619068 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650fffcb3e0 0x5650fffd9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:17.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.241+0000 7f8e20c00640 1 -- 172.21.15.43:0/2946619068 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650fffcb3e0 msgr2=0x5650fffd9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:17.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.241+0000 7f8e20c00640 1 --2- 172.21.15.43:0/2946619068 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650fffcb3e0 0x5650fffd9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:17.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.241+0000 7f8e20c00640 1 -- 172.21.15.43:0/2946619068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650fffd5ca0 msgr2=0x5650fffda530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:17.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.241+0000 7f8e20c00640 1 --2- 172.21.15.43:0/2946619068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650fffd5ca0 0x5650fffda530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:17.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.241+0000 7f8e20c00640 1 -- 172.21.15.43:0/2946619068 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56510004fb00 con 0x5650fffcaac0 2024-09-19T05:13:17.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.242+0000 7f8e20c00640 1 --2- 172.21.15.43:0/2946619068 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650fffcaac0 0x5650fffd9ab0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8e14004350 tx=0x7f8e14057040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8a48edcf4ae99d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:17.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.242+0000 7f8e05600640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:13:17.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.242+0000 7f8e05600640 1 -- 172.21.15.43:0/2946619068 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e140043f0 con 0x5650fffcaac0 2024-09-19T05:13:17.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.242+0000 7f8e05600640 1 -- 172.21.15.43:0/2946619068 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f8e14004590 con 0x5650fffcaac0 2024-09-19T05:13:17.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.242+0000 7f8e257bb740 1 -- 172.21.15.43:0/2946619068 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e14036070 con 0x5650fffcaac0 2024-09-19T05:13:17.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.242+0000 7f8e05600640 1 -- 172.21.15.43:0/2946619068 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8e14004790 con 0x5650fffcaac0 2024-09-19T05:13:17.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.242+0000 7f8e257bb740 10 client.27230 fetch_fsmap learned FSMap version 29 2024-09-19T05:13:17.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.242+0000 7f8e257bb740 1 -- 172.21.15.43:0/2946619068 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x565100050de0 con 0x5650fffcaac0 2024-09-19T05:13:17.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.243+0000 7f8e05600640 1 -- 172.21.15.43:0/2946619068 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e14036070 con 0x5650fffcaac0 2024-09-19T05:13:17.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.243+0000 7f8e05600640 1 -- 172.21.15.43:0/2946619068 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8e14043070 con 0x5650fffcaac0 2024-09-19T05:13:17.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.243+0000 7f8e257bb740 10 client.27230 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:13:17.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.243+0000 7f8e257bb740 10 client.27230 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:13:17.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.243+0000 7f8e257bb740 20 client.27230 populate_metadata read hostname 'smithi043' 2024-09-19T05:13:17.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.243+0000 7f8e257bb740 1 --2- 172.21.15.43:0/2946619068 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5651000511b0 0x5651000535d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:17.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.243+0000 7f8e257bb740 4 client.27230 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T05:13:17.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.243+0000 7f8e257bb740 1 -- 172.21.15.43:0/2946619068 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5650fff38d10 con 0x5651000511b0 2024-09-19T05:13:17.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.243+0000 7f8e1b400640 1 --2- 172.21.15.43:0/2946619068 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5651000511b0 0x5651000535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:17.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.244+0000 7f8e1b400640 1 --2- 172.21.15.43:0/2946619068 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5651000511b0 0x5651000535d0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d554eea7a0b01e3b server_cookie=e2792d56f93e5a97 in_seq=0 out_seq=0 2024-09-19T05:13:17.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.244+0000 7f8e05600640 10 client.27230 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:13:17.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.245+0000 7f8e05600640 1 -- 172.21.15.43:0/2946619068 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5650fff38d10 con 0x5651000511b0 2024-09-19T05:13:17.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.245+0000 7f8e05600640 10 client.27230 handle_command_reply: tid=0 2024-09-19T05:13:17.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.245+0000 7f8e257bb740 1 client.27230 shutdown 2024-09-19T05:13:17.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.245+0000 7f8e257bb740 1 -- 172.21.15.43:0/2946619068 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650fffcaac0 msgr2=0x5650fffd9ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:17.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.245+0000 7f8e257bb740 1 --2- 172.21.15.43:0/2946619068 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650fffcaac0 0x5650fffd9ab0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8e14004350 tx=0x7f8e14057040 comp rx=0 tx=0).stop 2024-09-19T05:13:17.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.245+0000 7f8e257bb740 1 -- 172.21.15.43:0/2946619068 shutdown_connections 2024-09-19T05:13:17.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.245+0000 7f8e257bb740 1 --2- 172.21.15.43:0/2946619068 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5651000511b0 0x5651000535d0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:17.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.245+0000 7f8e257bb740 1 --2- 172.21.15.43:0/2946619068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650fffd5ca0 0x5650fffda530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:17.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.245+0000 7f8e257bb740 1 --2- 172.21.15.43:0/2946619068 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650fffcb3e0 0x5650fffd9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:17.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.245+0000 7f8e257bb740 1 --2- 172.21.15.43:0/2946619068 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650fffcaac0 0x5650fffd9ab0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:17.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.245+0000 7f8e257bb740 1 -- 172.21.15.43:0/2946619068 >> 172.21.15.43:0/2946619068 conn(0x5650fffc7050 msgr2=0x5650fffd4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:17.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.245+0000 7f8e05600640 0 client.27230 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:13:17.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.245+0000 7f8e257bb740 1 -- 172.21.15.43:0/2946619068 shutdown_connections 2024-09-19T05:13:17.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.245+0000 7f8e257bb740 1 -- 172.21.15.43:0/2946619068 wait complete. 2024-09-19T05:13:17.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.246+0000 7f8e257bb740 20 client.27230 trim_cache size 0 max 16384 2024-09-19T05:13:17.252 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:13:17.252 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-09-19T05:13:17.252 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "eafc0cb3-16b3-4d86-8354-604e28a1e1b1", 2024-09-19T05:13:17.252 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-09-19T05:13:17.252 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:13:17.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.246+0000 7f8e22c00640 1 -- 172.21.15.43:0/2693958466 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f8e000a3430 msgr2=0x7f8e000a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:17.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.246+0000 7f8e22c00640 1 --2- 172.21.15.43:0/2693958466 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f8e000a3430 0x7f8e000a58f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x5650fffc7eb0 tx=0x7f8e0c0bc040 comp rx=0 tx=0).stop 2024-09-19T05:13:17.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.246+0000 7f8e22c00640 1 -- 172.21.15.43:0/2693958466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e1c076720 msgr2=0x7f8e1c1a4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:17.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.246+0000 7f8e22c00640 1 --2- 172.21.15.43:0/2693958466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e1c076720 0x7f8e1c1a4760 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c00e9f0 tx=0x7f8e0c00edf0 comp rx=0 tx=0).stop 2024-09-19T05:13:17.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.247+0000 7f8e22c00640 1 -- 172.21.15.43:0/2693958466 shutdown_connections 2024-09-19T05:13:17.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.247+0000 7f8e22c00640 1 --2- 172.21.15.43:0/2693958466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e1c112a30 0x7f8e1c19e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:17.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.247+0000 7f8e22c00640 1 --2- 172.21.15.43:0/2693958466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e1c077040 0x7f8e1c1a4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:17.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.247+0000 7f8e22c00640 1 --2- 172.21.15.43:0/2693958466 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f8e000a3430 0x7f8e000a58f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:17.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.247+0000 7f8e22c00640 1 --2- 172.21.15.43:0/2693958466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e1c076720 0x7f8e1c1a4760 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:17.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.247+0000 7f8e22c00640 1 -- 172.21.15.43:0/2693958466 >> 172.21.15.43:0/2693958466 conn(0x7f8e1c0ffa90 msgr2=0x7f8e1c113630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:17.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.247+0000 7f8e22c00640 1 -- 172.21.15.43:0/2693958466 shutdown_connections 2024-09-19T05:13:17.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:17.247+0000 7f8e22c00640 1 -- 172.21.15.43:0/2693958466 wait complete. 2024-09-19T05:13:18.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:17 smithi151 ceph-mon[35425]: pgmap v268: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 248 B/s wr, 12 op/s 2024-09-19T05:13:18.166 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:17 smithi043 ceph-mon[30800]: pgmap v268: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 248 B/s wr, 12 op/s 2024-09-19T05:13:18.206 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:13:18.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:17 smithi096 ceph-mon[35085]: pgmap v268: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 248 B/s wr, 12 op/s 2024-09-19T05:13:18.624 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:13:19.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:18 smithi043 ceph-mon[30800]: scrub queued for path: / 2024-09-19T05:13:19.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:18 smithi043 ceph-mon[30800]: scrub summary: idle+waiting paths [/] 2024-09-19T05:13:19.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:18 smithi043 ceph-mon[30800]: scrub summary: active paths [/] 2024-09-19T05:13:19.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:18 smithi043 ceph-mon[30800]: scrub summary: idle+waiting paths [/] 2024-09-19T05:13:19.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:18 smithi043 ceph-mon[30800]: scrub summary: idle 2024-09-19T05:13:19.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:18 smithi151 ceph-mon[35425]: scrub queued for path: / 2024-09-19T05:13:19.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:18 smithi151 ceph-mon[35425]: scrub summary: idle+waiting paths [/] 2024-09-19T05:13:19.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:18 smithi151 ceph-mon[35425]: scrub summary: active paths [/] 2024-09-19T05:13:19.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:18 smithi151 ceph-mon[35425]: scrub summary: idle+waiting paths [/] 2024-09-19T05:13:19.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:18 smithi151 ceph-mon[35425]: scrub summary: idle 2024-09-19T05:13:19.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.176+0000 7fb837c00640 1 -- 172.21.15.43:0/1015753616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb830101de0 msgr2=0x7fb8301021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:19.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.176+0000 7fb837c00640 1 --2- 172.21.15.43:0/1015753616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb830101de0 0x7fb8301021c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb820009900 tx=0x7fb82002a3f0 comp rx=0 tx=0).stop 2024-09-19T05:13:19.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.176+0000 7fb837c00640 1 -- 172.21.15.43:0/1015753616 shutdown_connections 2024-09-19T05:13:19.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.176+0000 7fb837c00640 1 --2- 172.21.15.43:0/1015753616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb830106f10 0x7fb83010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:19.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.176+0000 7fb837c00640 1 --2- 172.21.15.43:0/1015753616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb830102790 0x7fb8301067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:19.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.176+0000 7fb837c00640 1 --2- 172.21.15.43:0/1015753616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb830101de0 0x7fb8301021c0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:19.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.176+0000 7fb837c00640 1 -- 172.21.15.43:0/1015753616 >> 172.21.15.43:0/1015753616 conn(0x7fb8300fd8c0 msgr2=0x7fb8300ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:19.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.176+0000 7fb837c00640 1 -- 172.21.15.43:0/1015753616 shutdown_connections 2024-09-19T05:13:19.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.176+0000 7fb837c00640 1 -- 172.21.15.43:0/1015753616 wait complete. 2024-09-19T05:13:19.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.177+0000 7fb837c00640 1 Processor -- start 2024-09-19T05:13:19.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.177+0000 7fb837c00640 1 -- start start 2024-09-19T05:13:19.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.178+0000 7fb837c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb830101de0 0x7fb8300744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:19.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.178+0000 7fb837c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb830102790 0x7fb830074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:19.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.178+0000 7fb837c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb830106f10 0x7fb830074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:19.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.178+0000 7fb837c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb830111f70 con 0x7fb830101de0 2024-09-19T05:13:19.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.178+0000 7fb837c00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb830111df0 con 0x7fb830102790 2024-09-19T05:13:19.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.178+0000 7fb837c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb8301120f0 con 0x7fb830106f10 2024-09-19T05:13:19.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.178+0000 7fb835200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb830101de0 0x7fb8300744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:19.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.178+0000 7fb835200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb830101de0 0x7fb8300744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50472/0 (socket says 172.21.15.43:50472) 2024-09-19T05:13:19.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.178+0000 7fb835200640 1 -- 172.21.15.43:0/1524660879 learned_addr learned my addr 172.21.15.43:0/1524660879 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:19.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.178+0000 7fb835c00640 1 --2- 172.21.15.43:0/1524660879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb830106f10 0x7fb830074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:19.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.178+0000 7fb827e00640 1 --2- 172.21.15.43:0/1524660879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb830102790 0x7fb830074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:19.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.179+0000 7fb835200640 1 -- 172.21.15.43:0/1524660879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb830106f10 msgr2=0x7fb830074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:19.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.179+0000 7fb835200640 1 --2- 172.21.15.43:0/1524660879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb830106f10 0x7fb830074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:19.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.179+0000 7fb835200640 1 -- 172.21.15.43:0/1524660879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb830102790 msgr2=0x7fb830074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:19.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.179+0000 7fb835200640 1 --2- 172.21.15.43:0/1524660879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb830102790 0x7fb830074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:19.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.179+0000 7fb835200640 1 -- 172.21.15.43:0/1524660879 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8301a68d0 con 0x7fb830101de0 2024-09-19T05:13:19.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.179+0000 7fb835c00640 1 --2- 172.21.15.43:0/1524660879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb830106f10 0x7fb830074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:13:19.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.179+0000 7fb835200640 1 --2- 172.21.15.43:0/1524660879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb830101de0 0x7fb8300744f0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fb820006010 tx=0x7fb820004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:19.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.179+0000 7fb82e000640 1 -- 172.21.15.43:0/1524660879 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82003f030 con 0x7fb830101de0 2024-09-19T05:13:19.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.179+0000 7fb82e000640 1 -- 172.21.15.43:0/1524660879 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb820004440 con 0x7fb830101de0 2024-09-19T05:13:19.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.179+0000 7fb82e000640 1 -- 172.21.15.43:0/1524660879 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb820035070 con 0x7fb830101de0 2024-09-19T05:13:19.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.180+0000 7fb837c00640 1 -- 172.21.15.43:0/1524660879 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8301a6b60 con 0x7fb830101de0 2024-09-19T05:13:19.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.180+0000 7fb837c00640 1 -- 172.21.15.43:0/1524660879 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8301a6ff0 con 0x7fb830101de0 2024-09-19T05:13:19.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.183+0000 7fb82e000640 1 -- 172.21.15.43:0/1524660879 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb820042030 con 0x7fb830101de0 2024-09-19T05:13:19.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.183+0000 7fb83a686740 1 Processor -- start 2024-09-19T05:13:19.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.183+0000 7fb83a686740 1 -- start start 2024-09-19T05:13:19.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.183+0000 7fb83a686740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cc67d2b90 0x559cc67d2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:19.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.183+0000 7fb83a686740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cc67d34b0 0x559cc67dd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:19.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.183+0000 7fb83a686740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559cc67ddd70 0x559cc67e0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:19.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.183+0000 7fb83a686740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559cc67393e0 con 0x559cc67d34b0 2024-09-19T05:13:19.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.183+0000 7fb83a686740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559cc66e0d80 con 0x559cc67ddd70 2024-09-19T05:13:19.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.183+0000 7fb83a686740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559cc67353a0 con 0x559cc67d2b90 2024-09-19T05:13:19.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.184+0000 7fb835200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cc67d34b0 0x559cc67dd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:19.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.184+0000 7fb835200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cc67d34b0 0x559cc67dd830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50474/0 (socket says 172.21.15.43:50474) 2024-09-19T05:13:19.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.184+0000 7fb835200640 1 -- 172.21.15.43:0/3024763948 learned_addr learned my addr 172.21.15.43:0/3024763948 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:19.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.184+0000 7fb835c00640 1 --2- 172.21.15.43:0/3024763948 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cc67d2b90 0x559cc67d2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:19.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.184+0000 7fb827e00640 1 --2- 172.21.15.43:0/3024763948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559cc67ddd70 0x559cc67e0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:19.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.184+0000 7fb835200640 1 -- 172.21.15.43:0/3024763948 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cc67d2b90 msgr2=0x559cc67d2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:19.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.184+0000 7fb835200640 1 --2- 172.21.15.43:0/3024763948 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cc67d2b90 0x559cc67d2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:19.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.184+0000 7fb835200640 1 -- 172.21.15.43:0/3024763948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559cc67ddd70 msgr2=0x559cc67e0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:19.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.184+0000 7fb835200640 1 --2- 172.21.15.43:0/3024763948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559cc67ddd70 0x559cc67e0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:19.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.184+0000 7fb835200640 1 -- 172.21.15.43:0/3024763948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cc66ffae0 con 0x559cc67d34b0 2024-09-19T05:13:19.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.184+0000 7fb827e00640 1 --2- 172.21.15.43:0/3024763948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559cc67ddd70 0x559cc67e0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:13:19.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.184+0000 7fb82e000640 1 --2- 172.21.15.43:0/1524660879 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb8140a3430 0x7fb8140a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:19.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.185+0000 7fb82e000640 1 -- 172.21.15.43:0/1524660879 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7fb8200efff0 con 0x7fb830101de0 2024-09-19T05:13:19.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.185+0000 7fb835200640 1 --2- 172.21.15.43:0/3024763948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cc67d34b0 0x559cc67dd830 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fb8200ed6a0 tx=0x7fb8200f8320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44539b00887d5c54 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:19.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.185+0000 7fb835c00640 1 --2- 172.21.15.43:0/1524660879 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb8140a3430 0x7fb8140a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:19.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.185+0000 7fb826000640 1 -- 172.21.15.43:0/3024763948 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb820049070 con 0x559cc67d34b0 2024-09-19T05:13:19.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.185+0000 7fb826000640 1 -- 172.21.15.43:0/3024763948 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8200bc370 con 0x559cc67d34b0 2024-09-19T05:13:19.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.185+0000 7fb826000640 1 -- 172.21.15.43:0/3024763948 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8200bc140 con 0x559cc67d34b0 2024-09-19T05:13:19.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.186+0000 7fb83a686740 1 -- 172.21.15.43:0/3024763948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cc67d34b0 msgr2=0x559cc67dd830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:19.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.186+0000 7fb835c00640 1 --2- 172.21.15.43:0/1524660879 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb8140a3430 0x7fb8140a58f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb828009fd0 tx=0x7fb828009500 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:19.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.186+0000 7fb83a686740 1 --2- 172.21.15.43:0/3024763948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cc67d34b0 0x559cc67dd830 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fb8200ed6a0 tx=0x7fb8200f8320 comp rx=0 tx=0).stop 2024-09-19T05:13:19.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.186+0000 7fb83a686740 1 -- 172.21.15.43:0/3024763948 shutdown_connections 2024-09-19T05:13:19.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.186+0000 7fb83a686740 1 --2- 172.21.15.43:0/3024763948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559cc67ddd70 0x559cc67e0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:19.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.186+0000 7fb83a686740 1 --2- 172.21.15.43:0/3024763948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cc67d34b0 0x559cc67dd830 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:19.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.186+0000 7fb83a686740 1 --2- 172.21.15.43:0/3024763948 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cc67d2b90 0x559cc67d2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:19.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.186+0000 7fb83a686740 1 -- 172.21.15.43:0/3024763948 >> 172.21.15.43:0/3024763948 conn(0x559cc67cf120 msgr2=0x559cc67cf500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:19.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.186+0000 7fb83a686740 1 -- 172.21.15.43:0/3024763948 shutdown_connections 2024-09-19T05:13:19.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.186+0000 7fb83a686740 1 -- 172.21.15.43:0/3024763948 wait complete. 2024-09-19T05:13:19.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.186+0000 7fb83a686740 1 -- start start 2024-09-19T05:13:19.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.187+0000 7fb83a686740 1 Processor -- start 2024-09-19T05:13:19.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.187+0000 7fb83a686740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cc67d2b90 0x559cc67dacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:19.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.187+0000 7fb83a686740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559cc67d34b0 0x559cc67db210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:19.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.187+0000 7fb83a686740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cc67ddd70 0x559cc67db750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:19.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.187+0000 7fb83a686740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559cc67e0700 con 0x559cc67d2b90 2024-09-19T05:13:19.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.187+0000 7fb83a686740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559cc66f40f0 con 0x559cc67d34b0 2024-09-19T05:13:19.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.187+0000 7fb83a686740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559cc67e0880 con 0x559cc67ddd70 2024-09-19T05:13:19.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.187+0000 7fb835200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cc67d2b90 0x559cc67dacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:19.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.187+0000 7fb83a686740 10 client.0 _ll_register_callbacks cb 0x559cc66e59e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:19.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.187+0000 7fb835200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cc67d2b90 0x559cc67dacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50476/0 (socket says 172.21.15.43:50476) 2024-09-19T05:13:19.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.187+0000 7fb835200640 1 -- 172.21.15.43:0/410416777 learned_addr learned my addr 172.21.15.43:0/410416777 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:19.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.187+0000 7fb827e00640 1 --2- 172.21.15.43:0/410416777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559cc67d34b0 0x559cc67db210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:19.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.187+0000 7fb835c00640 1 --2- 172.21.15.43:0/410416777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cc67ddd70 0x559cc67db750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:19.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.188+0000 7fb835200640 1 -- 172.21.15.43:0/410416777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cc67ddd70 msgr2=0x559cc67db750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:19.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.188+0000 7fb835200640 1 --2- 172.21.15.43:0/410416777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cc67ddd70 0x559cc67db750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:19.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.188+0000 7fb835200640 1 -- 172.21.15.43:0/410416777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559cc67d34b0 msgr2=0x559cc67db210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:19.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.188+0000 7fb835200640 1 --2- 172.21.15.43:0/410416777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559cc67d34b0 0x559cc67db210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:19.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.188+0000 7fb835200640 1 -- 172.21.15.43:0/410416777 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cc64aa3c0 con 0x559cc67d2b90 2024-09-19T05:13:19.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.188+0000 7fb835200640 1 --2- 172.21.15.43:0/410416777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cc67d2b90 0x559cc67dacd0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb820005ab0 tx=0x7fb8200b8610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca562d8a963a6992 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:19.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.188+0000 7fb81ea00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:13:19.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.188+0000 7fb81ea00640 1 -- 172.21.15.43:0/410416777 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb820049040 con 0x559cc67d2b90 2024-09-19T05:13:19.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.188+0000 7fb81ea00640 1 -- 172.21.15.43:0/410416777 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7fb8200ed9b0 con 0x559cc67d2b90 2024-09-19T05:13:19.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.188+0000 7fb83a686740 1 -- 172.21.15.43:0/410416777 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8200bc370 con 0x559cc67d2b90 2024-09-19T05:13:19.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.188+0000 7fb81ea00640 1 -- 172.21.15.43:0/410416777 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb8200bbcd0 con 0x559cc67d2b90 2024-09-19T05:13:19.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.188+0000 7fb83a686740 10 client.17220 fetch_fsmap learned FSMap version 29 2024-09-19T05:13:19.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.188+0000 7fb83a686740 1 -- 172.21.15.43:0/410416777 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559cc67e0700 con 0x559cc67d2b90 2024-09-19T05:13:19.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.189+0000 7fb81ea00640 1 -- 172.21.15.43:0/410416777 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8200bc370 con 0x559cc67d2b90 2024-09-19T05:13:19.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.189+0000 7fb81ea00640 1 -- 172.21.15.43:0/410416777 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb8200b89e0 con 0x559cc67d2b90 2024-09-19T05:13:19.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.189+0000 7fb83a686740 10 client.17220 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:13:19.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.189+0000 7fb83a686740 10 client.17220 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:13:19.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.189+0000 7fb83a686740 20 client.17220 populate_metadata read hostname 'smithi043' 2024-09-19T05:13:19.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.189+0000 7fb83a686740 1 --2- 172.21.15.43:0/410416777 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x559cc67ed630 0x559cc67efa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:19.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.189+0000 7fb83a686740 4 client.17220 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:13:19.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.189+0000 7fb83a686740 1 -- 172.21.15.43:0/410416777 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559cc6746950 con 0x559cc67ed630 2024-09-19T05:13:19.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.189+0000 7fb827e00640 1 --2- 172.21.15.43:0/410416777 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x559cc67ed630 0x559cc67efa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:19.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.190+0000 7fb827e00640 1 --2- 172.21.15.43:0/410416777 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x559cc67ed630 0x559cc67efa80 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=997be64ec4b03b0 server_cookie=1a52ffa514cfb159 in_seq=0 out_seq=0 2024-09-19T05:13:19.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.190+0000 7fb81ea00640 10 client.17220 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:13:19.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.191+0000 7fb81ea00640 1 -- 172.21.15.43:0/410416777 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559cc6746950 con 0x559cc67ed630 2024-09-19T05:13:19.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.191+0000 7fb81ea00640 10 client.17220 handle_command_reply: tid=0 2024-09-19T05:13:19.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.191+0000 7fb83a686740 1 client.17220 shutdown 2024-09-19T05:13:19.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.191+0000 7fb83a686740 1 -- 172.21.15.43:0/410416777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cc67d2b90 msgr2=0x559cc67dacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:19.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.191+0000 7fb83a686740 1 --2- 172.21.15.43:0/410416777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cc67d2b90 0x559cc67dacd0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb820005ab0 tx=0x7fb8200b8610 comp rx=0 tx=0).stop 2024-09-19T05:13:19.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.191+0000 7fb83a686740 1 -- 172.21.15.43:0/410416777 shutdown_connections 2024-09-19T05:13:19.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.191+0000 7fb83a686740 1 --2- 172.21.15.43:0/410416777 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x559cc67ed630 0x559cc67efa80 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:19.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.191+0000 7fb83a686740 1 --2- 172.21.15.43:0/410416777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cc67ddd70 0x559cc67db750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:19.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.191+0000 7fb83a686740 1 --2- 172.21.15.43:0/410416777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559cc67d34b0 0x559cc67db210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:19.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.191+0000 7fb83a686740 1 --2- 172.21.15.43:0/410416777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cc67d2b90 0x559cc67dacd0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:19.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.191+0000 7fb83a686740 1 -- 172.21.15.43:0/410416777 >> 172.21.15.43:0/410416777 conn(0x559cc67cf120 msgr2=0x559cc67d4e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:19.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.191+0000 7fb81ea00640 0 client.17220 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:13:19.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.191+0000 7fb83a686740 1 -- 172.21.15.43:0/410416777 shutdown_connections 2024-09-19T05:13:19.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.191+0000 7fb83a686740 1 -- 172.21.15.43:0/410416777 wait complete. 2024-09-19T05:13:19.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.191+0000 7fb83a686740 20 client.17220 trim_cache size 0 max 16384 2024-09-19T05:13:19.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.209+0000 7fb83a686740 1 Processor -- start 2024-09-19T05:13:19.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.210+0000 7fb83a686740 1 -- start start 2024-09-19T05:13:19.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.210+0000 7fb83a686740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cc67d2b90 0x559cc67d5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:19.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.210+0000 7fb83a686740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559cc67d34b0 0x559cc67f5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:19.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.210+0000 7fb83a686740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cc67ddd70 0x559cc67f5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:19.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.210+0000 7fb83a686740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb8301afae0 con 0x559cc67ddd70 2024-09-19T05:13:19.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.210+0000 7fb83a686740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb8301b1620 con 0x559cc67d34b0 2024-09-19T05:13:19.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.210+0000 7fb83a686740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb8301b01b0 con 0x559cc67d2b90 2024-09-19T05:13:19.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.210+0000 7fb835c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cc67ddd70 0x559cc67f5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:19.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.210+0000 7fb827e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559cc67d34b0 0x559cc67f5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:19.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.210+0000 7fb835200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cc67d2b90 0x559cc67d5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:19.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.210+0000 7fb835c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cc67ddd70 0x559cc67f5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50484/0 (socket says 172.21.15.43:50484) 2024-09-19T05:13:19.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.210+0000 7fb835c00640 1 -- 172.21.15.43:0/3094218131 learned_addr learned my addr 172.21.15.43:0/3094218131 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:19.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.211+0000 7fb827e00640 1 -- 172.21.15.43:0/3094218131 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cc67d2b90 msgr2=0x559cc67d5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:19.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.211+0000 7fb827e00640 1 --2- 172.21.15.43:0/3094218131 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cc67d2b90 0x559cc67d5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:19.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.211+0000 7fb827e00640 1 -- 172.21.15.43:0/3094218131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cc67ddd70 msgr2=0x559cc67f5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:19.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.211+0000 7fb827e00640 1 --2- 172.21.15.43:0/3094218131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cc67ddd70 0x559cc67f5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:19.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.211+0000 7fb827e00640 1 -- 172.21.15.43:0/3094218131 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cc67ce8d0 con 0x559cc67d34b0 2024-09-19T05:13:19.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.211+0000 7fb835200640 1 --2- 172.21.15.43:0/3094218131 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cc67d2b90 0x559cc67d5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:13:19.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.211+0000 7fb827e00640 1 --2- 172.21.15.43:0/3094218131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559cc67d34b0 0x559cc67f5350 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x559cc67f02f0 tx=0x7fb818005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=929e37415806695 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:19.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.211+0000 7fb825600640 1 -- 172.21.15.43:0/3094218131 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb818003d60 con 0x559cc67d34b0 2024-09-19T05:13:19.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.211+0000 7fb825600640 1 -- 172.21.15.43:0/3094218131 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb818010910 con 0x559cc67d34b0 2024-09-19T05:13:19.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.211+0000 7fb825600640 1 -- 172.21.15.43:0/3094218131 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81801f860 con 0x559cc67d34b0 2024-09-19T05:13:19.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:18 smithi096 ceph-mon[35085]: scrub queued for path: / 2024-09-19T05:13:19.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:18 smithi096 ceph-mon[35085]: scrub summary: idle+waiting paths [/] 2024-09-19T05:13:19.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:18 smithi096 ceph-mon[35085]: scrub summary: active paths [/] 2024-09-19T05:13:19.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:18 smithi096 ceph-mon[35085]: scrub summary: idle+waiting paths [/] 2024-09-19T05:13:19.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:18 smithi096 ceph-mon[35085]: scrub summary: idle 2024-09-19T05:13:19.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.212+0000 7fb83a686740 1 -- 172.21.15.43:0/3094218131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559cc67d34b0 msgr2=0x559cc67f5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:19.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.212+0000 7fb83a686740 1 --2- 172.21.15.43:0/3094218131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559cc67d34b0 0x559cc67f5350 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x559cc67f02f0 tx=0x7fb818005f90 comp rx=0 tx=0).stop 2024-09-19T05:13:19.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.212+0000 7fb83a686740 1 -- 172.21.15.43:0/3094218131 shutdown_connections 2024-09-19T05:13:19.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.212+0000 7fb83a686740 1 --2- 172.21.15.43:0/3094218131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cc67ddd70 0x559cc67f5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:19.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.212+0000 7fb83a686740 1 --2- 172.21.15.43:0/3094218131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559cc67d34b0 0x559cc67f5350 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:19.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.212+0000 7fb83a686740 1 --2- 172.21.15.43:0/3094218131 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cc67d2b90 0x559cc67d5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:19.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.212+0000 7fb83a686740 1 -- 172.21.15.43:0/3094218131 >> 172.21.15.43:0/3094218131 conn(0x559cc67cf120 msgr2=0x559cc67ce1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:19.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.212+0000 7fb83a686740 1 -- 172.21.15.43:0/3094218131 shutdown_connections 2024-09-19T05:13:19.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.212+0000 7fb83a686740 1 -- 172.21.15.43:0/3094218131 wait complete. 2024-09-19T05:13:19.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.212+0000 7fb83a686740 1 -- start start 2024-09-19T05:13:19.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.213+0000 7fb83a686740 1 Processor -- start 2024-09-19T05:13:19.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.213+0000 7fb83a686740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cc67d2b90 0x559cc67e1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:19.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.213+0000 7fb83a686740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559cc67d34b0 0x559cc67e21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:19.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.213+0000 7fb83a686740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cc67ddd70 0x559cc685fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:19.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.213+0000 7fb83a686740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb8301b0f50 con 0x559cc67d2b90 2024-09-19T05:13:19.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.213+0000 7fb83a686740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb8301b0880 con 0x559cc67d34b0 2024-09-19T05:13:19.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.213+0000 7fb83a686740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb8301adae0 con 0x559cc67ddd70 2024-09-19T05:13:19.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.213+0000 7fb83a686740 10 client.0 _ll_register_callbacks cb 0x7fb8200f80e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:19.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.213+0000 7fb835200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cc67d2b90 0x559cc67e1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:19.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.213+0000 7fb835200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cc67d2b90 0x559cc67e1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50486/0 (socket says 172.21.15.43:50486) 2024-09-19T05:13:19.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.213+0000 7fb835200640 1 -- 172.21.15.43:0/3182331753 learned_addr learned my addr 172.21.15.43:0/3182331753 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:19.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.214+0000 7fb835c00640 1 --2- 172.21.15.43:0/3182331753 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cc67ddd70 0x559cc685fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:19.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.214+0000 7fb827e00640 1 --2- 172.21.15.43:0/3182331753 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559cc67d34b0 0x559cc67e21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:19.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.214+0000 7fb835200640 1 -- 172.21.15.43:0/3182331753 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cc67ddd70 msgr2=0x559cc685fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:19.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.214+0000 7fb835200640 1 --2- 172.21.15.43:0/3182331753 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cc67ddd70 0x559cc685fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:19.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.214+0000 7fb835200640 1 -- 172.21.15.43:0/3182331753 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559cc67d34b0 msgr2=0x559cc67e21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:19.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.214+0000 7fb835200640 1 --2- 172.21.15.43:0/3182331753 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559cc67d34b0 0x559cc67e21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:19.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.214+0000 7fb835200640 1 -- 172.21.15.43:0/3182331753 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cc6860070 con 0x559cc67d2b90 2024-09-19T05:13:19.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.214+0000 7fb835c00640 1 --2- 172.21.15.43:0/3182331753 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cc67ddd70 0x559cc685fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:13:19.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.214+0000 7fb835200640 1 --2- 172.21.15.43:0/3182331753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cc67d2b90 0x559cc67e1ca0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fb8200b87d0 tx=0x7fb8200c5020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ca7aef8972cefb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:19.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.214+0000 7fb81ea00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:13:19.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.214+0000 7fb81ea00640 1 -- 172.21.15.43:0/3182331753 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb820049050 con 0x559cc67d2b90 2024-09-19T05:13:19.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.214+0000 7fb81ea00640 1 -- 172.21.15.43:0/3182331753 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7fb8200c4aa0 con 0x559cc67d2b90 2024-09-19T05:13:19.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.214+0000 7fb83a686740 1 -- 172.21.15.43:0/3182331753 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb818010910 con 0x559cc67d2b90 2024-09-19T05:13:19.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.215+0000 7fb81ea00640 1 -- 172.21.15.43:0/3182331753 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb8200bbeb0 con 0x559cc67d2b90 2024-09-19T05:13:19.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.215+0000 7fb83a686740 10 client.17226 fetch_fsmap learned FSMap version 29 2024-09-19T05:13:19.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.215+0000 7fb83a686740 1 -- 172.21.15.43:0/3182331753 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559cc6860070 con 0x559cc67d2b90 2024-09-19T05:13:19.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.215+0000 7fb81ea00640 1 -- 172.21.15.43:0/3182331753 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb818010910 con 0x559cc67d2b90 2024-09-19T05:13:19.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.215+0000 7fb81ea00640 1 -- 172.21.15.43:0/3182331753 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb8200bc5d0 con 0x559cc67d2b90 2024-09-19T05:13:19.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.215+0000 7fb83a686740 10 client.17226 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:13:19.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.215+0000 7fb83a686740 10 client.17226 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:13:19.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.215+0000 7fb83a686740 20 client.17226 populate_metadata read hostname 'smithi043' 2024-09-19T05:13:19.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.215+0000 7fb83a686740 1 --2- 172.21.15.43:0/3182331753 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x559cc6861620 0x559cc6863a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:19.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.215+0000 7fb83a686740 4 client.17226 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:13:19.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.215+0000 7fb83a686740 1 -- 172.21.15.43:0/3182331753 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559cc6740e60 con 0x559cc6861620 2024-09-19T05:13:19.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.216+0000 7fb827e00640 1 --2- 172.21.15.43:0/3182331753 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x559cc6861620 0x559cc6863a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.216+0000 7fb827e00640 1 --2- 172.21.15.43:0/3182331753 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x559cc6861620 0x559cc6863a10 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69278f62a3d08e14 server_cookie=b7506b35a41e68d6 in_seq=0 out_seq=0 2024-09-19T05:13:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.216+0000 7fb81ea00640 10 client.17226 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:13:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.216+0000 7fb81ea00640 1 -- 172.21.15.43:0/3182331753 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559cc6740e60 con 0x559cc6861620 2024-09-19T05:13:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.216+0000 7fb81ea00640 10 client.17226 handle_command_reply: tid=0 2024-09-19T05:13:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.216+0000 7fb83a686740 1 client.17226 shutdown 2024-09-19T05:13:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.216+0000 7fb83a686740 1 -- 172.21.15.43:0/3182331753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cc67d2b90 msgr2=0x559cc67e1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.216+0000 7fb83a686740 1 --2- 172.21.15.43:0/3182331753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cc67d2b90 0x559cc67e1ca0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fb8200b87d0 tx=0x7fb8200c5020 comp rx=0 tx=0).stop 2024-09-19T05:13:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.217+0000 7fb83a686740 1 -- 172.21.15.43:0/3182331753 shutdown_connections 2024-09-19T05:13:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.217+0000 7fb83a686740 1 --2- 172.21.15.43:0/3182331753 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x559cc6861620 0x559cc6863a10 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.217+0000 7fb83a686740 1 --2- 172.21.15.43:0/3182331753 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559cc67ddd70 0x559cc685fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.217+0000 7fb83a686740 1 --2- 172.21.15.43:0/3182331753 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559cc67d34b0 0x559cc67e21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.217+0000 7fb83a686740 1 --2- 172.21.15.43:0/3182331753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cc67d2b90 0x559cc67e1ca0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.217+0000 7fb83a686740 1 -- 172.21.15.43:0/3182331753 >> 172.21.15.43:0/3182331753 conn(0x559cc67cf120 msgr2=0x559cc67d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:19.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.217+0000 7fb81ea00640 0 client.17226 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:13:19.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.217+0000 7fb83a686740 1 -- 172.21.15.43:0/3182331753 shutdown_connections 2024-09-19T05:13:19.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.217+0000 7fb83a686740 1 -- 172.21.15.43:0/3182331753 wait complete. 2024-09-19T05:13:19.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.217+0000 7fb83a686740 20 client.17226 trim_cache size 0 max 16384 2024-09-19T05:13:19.232 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:13:19.232 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-09-19T05:13:19.232 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-09-19T05:13:19.232 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:13:19.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.218+0000 7fb837c00640 1 -- 172.21.15.43:0/1524660879 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb8140a3430 msgr2=0x7fb8140a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:19.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.218+0000 7fb837c00640 1 --2- 172.21.15.43:0/1524660879 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb8140a3430 0x7fb8140a58f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb828009fd0 tx=0x7fb828009500 comp rx=0 tx=0).stop 2024-09-19T05:13:19.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.218+0000 7fb837c00640 1 -- 172.21.15.43:0/1524660879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb830101de0 msgr2=0x7fb8300744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:19.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.218+0000 7fb837c00640 1 --2- 172.21.15.43:0/1524660879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb830101de0 0x7fb8300744f0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fb820006010 tx=0x7fb820004040 comp rx=0 tx=0).stop 2024-09-19T05:13:19.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.218+0000 7fb837c00640 1 -- 172.21.15.43:0/1524660879 shutdown_connections 2024-09-19T05:13:19.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.218+0000 7fb837c00640 1 --2- 172.21.15.43:0/1524660879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb830106f10 0x7fb830074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:19.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.218+0000 7fb837c00640 1 --2- 172.21.15.43:0/1524660879 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fb8140a3430 0x7fb8140a58f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:19.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.218+0000 7fb837c00640 1 --2- 172.21.15.43:0/1524660879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb830102790 0x7fb830074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:19.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.218+0000 7fb837c00640 1 --2- 172.21.15.43:0/1524660879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb830101de0 0x7fb8300744f0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:19.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.218+0000 7fb837c00640 1 -- 172.21.15.43:0/1524660879 >> 172.21.15.43:0/1524660879 conn(0x7fb8300fd8c0 msgr2=0x7fb8300fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:19.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.218+0000 7fb837c00640 1 -- 172.21.15.43:0/1524660879 shutdown_connections 2024-09-19T05:13:19.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:19.218+0000 7fb837c00640 1 -- 172.21.15.43:0/1524660879 wait complete. 2024-09-19T05:13:19.988 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T05:13:19.989 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:13:20.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:19 smithi151 ceph-mon[35425]: pgmap v269: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T05:13:20.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:19 smithi096 ceph-mon[35085]: pgmap v269: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T05:13:20.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:19 smithi043 ceph-mon[30800]: pgmap v269: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T05:13:20.370 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:13:20.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:20.936+0000 7fdcc8400640 1 -- 172.21.15.43:0/3695714723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdcc0109120 msgr2=0x7fdcc0110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:20.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:20.936+0000 7fdcc8400640 1 --2- 172.21.15.43:0/3695714723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdcc0109120 0x7fdcc0110ca0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8009ca0 tx=0x7fdcb802a8c0 comp rx=0 tx=0).stop 2024-09-19T05:13:20.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:20.936+0000 7fdcc8400640 1 -- 172.21.15.43:0/3695714723 shutdown_connections 2024-09-19T05:13:20.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:20.936+0000 7fdcc8400640 1 --2- 172.21.15.43:0/3695714723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdcc0109120 0x7fdcc0110ca0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:20.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:20.936+0000 7fdcc8400640 1 --2- 172.21.15.43:0/3695714723 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcc01049a0 0x7fdcc01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:20.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:20.936+0000 7fdcc8400640 1 --2- 172.21.15.43:0/3695714723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcc0103ff0 0x7fdcc01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:20.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:20.936+0000 7fdcc8400640 1 -- 172.21.15.43:0/3695714723 >> 172.21.15.43:0/3695714723 conn(0x7fdcc00ffa90 msgr2=0x7fdcc0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:20.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:20.936+0000 7fdcc8400640 1 -- 172.21.15.43:0/3695714723 shutdown_connections 2024-09-19T05:13:20.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:20.936+0000 7fdcc8400640 1 -- 172.21.15.43:0/3695714723 wait complete. 2024-09-19T05:13:20.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:20.937+0000 7fdcc8400640 1 Processor -- start 2024-09-19T05:13:20.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:20.937+0000 7fdcc8400640 1 -- start start 2024-09-19T05:13:20.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:20.938+0000 7fdcc8400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcc0103ff0 0x7fdcc019a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:20.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:20.938+0000 7fdcc8400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdcc01049a0 0x7fdcc019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:20.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:20.938+0000 7fdcc8400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcc0109120 0x7fdcc019ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:20.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:20.938+0000 7fdcc8400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc0114150 con 0x7fdcc01049a0 2024-09-19T05:13:20.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:20.938+0000 7fdcc8400640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc0113fd0 con 0x7fdcc0103ff0 2024-09-19T05:13:20.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:20.938+0000 7fdcc8400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc01142d0 con 0x7fdcc0109120 2024-09-19T05:13:20.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:20.938+0000 7fdcc5000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdcc01049a0 0x7fdcc019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:20.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:20.938+0000 7fdcc5000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdcc01049a0 0x7fdcc019aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50496/0 (socket says 172.21.15.43:50496) 2024-09-19T05:13:20.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:20.938+0000 7fdcc5000640 1 -- 172.21.15.43:0/4227322424 learned_addr learned my addr 172.21.15.43:0/4227322424 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:20.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:20.938+0000 7fdcc5a00640 1 --2- 172.21.15.43:0/4227322424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcc0103ff0 0x7fdcc019a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:20.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:20.938+0000 7fdcc6400640 1 --2- 172.21.15.43:0/4227322424 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcc0109120 0x7fdcc019ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:20.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:20.939+0000 7fdcc5000640 1 -- 172.21.15.43:0/4227322424 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcc0109120 msgr2=0x7fdcc019ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:20.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:20.939+0000 7fdcc5000640 1 --2- 172.21.15.43:0/4227322424 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcc0109120 0x7fdcc019ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:20.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:20.939+0000 7fdcc5000640 1 -- 172.21.15.43:0/4227322424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcc0103ff0 msgr2=0x7fdcc019a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:20.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:20.939+0000 7fdcc5000640 1 --2- 172.21.15.43:0/4227322424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcc0103ff0 0x7fdcc019a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:20.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:20.939+0000 7fdcc5000640 1 -- 172.21.15.43:0/4227322424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc019f560 con 0x7fdcc01049a0 2024-09-19T05:13:20.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:20.939+0000 7fdcc5a00640 1 --2- 172.21.15.43:0/4227322424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcc0103ff0 0x7fdcc019a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:13:20.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:20.939+0000 7fdcc5000640 1 --2- 172.21.15.43:0/4227322424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdcc01049a0 0x7fdcc019aaf0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fdcac00e9f0 tx=0x7fdcac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:20.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:20.939+0000 7fdcbe000640 1 -- 172.21.15.43:0/4227322424 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcac00cd80 con 0x7fdcc01049a0 2024-09-19T05:13:20.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:20.939+0000 7fdcbe000640 1 -- 172.21.15.43:0/4227322424 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcac010910 con 0x7fdcc01049a0 2024-09-19T05:13:20.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:20.939+0000 7fdcc8400640 1 -- 172.21.15.43:0/4227322424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc019f850 con 0x7fdcc01049a0 2024-09-19T05:13:20.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:20.939+0000 7fdcbe000640 1 -- 172.21.15.43:0/4227322424 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcac018990 con 0x7fdcc01049a0 2024-09-19T05:13:20.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:20.939+0000 7fdcc8400640 1 -- 172.21.15.43:0/4227322424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc01a7130 con 0x7fdcc01049a0 2024-09-19T05:13:20.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:20.940+0000 7fdcc8400640 1 -- 172.21.15.43:0/4227322424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcc01043d0 con 0x7fdcc01049a0 2024-09-19T05:13:20.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:20.943+0000 7fdcbe000640 1 -- 172.21.15.43:0/4227322424 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fdcac010ac0 con 0x7fdcc01049a0 2024-09-19T05:13:20.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:20.944+0000 7fdcbe000640 1 --2- 172.21.15.43:0/4227322424 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fdca80a33b0 0x7fdca80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:20.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:20.944+0000 7fdcbe000640 1 -- 172.21.15.43:0/4227322424 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7fdcac0d0450 con 0x7fdcc01049a0 2024-09-19T05:13:20.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:20.944+0000 7fdcc5a00640 1 --2- 172.21.15.43:0/4227322424 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fdca80a33b0 0x7fdca80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:20.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:20.945+0000 7fdcc5a00640 1 --2- 172.21.15.43:0/4227322424 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fdca80a33b0 0x7fdca80a5870 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4009fc0 tx=0x7fdcb4005f90 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:20.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:20.946+0000 7fdcbe000640 1 -- 172.21.15.43:0/4227322424 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdcac09c770 con 0x7fdcc01049a0 2024-09-19T05:13:21.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:21.169+0000 7fdcc8400640 1 -- 172.21.15.43:0/4227322424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdcc01a7460 con 0x7fdcc01049a0 2024-09-19T05:13:21.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:21.170+0000 7fdcbe000640 1 -- 172.21.15.43:0/4227322424 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11965 (secure 0 0 0) 0x7fdcac0a0a70 con 0x7fdcc01049a0 2024-09-19T05:13:21.172 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:13:21.172 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":29,"btime":"2024-09-19T05:04:38:785407+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/773801169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":773801169},{"type":"v1","addr":"172.21.15.43:6835","nonce":773801169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1799137902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1799137902},{"type":"v1","addr":"172.21.15.43:6837","nonce":1799137902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:02:25.463041+0000","modified":"2024-09-19T05:04:38.039809+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24518,"mds_2":24511,"mds_3":24506,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14541":{"gid":14541,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/794881135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":794881135},{"type":"v1","addr":"172.21.15.43:6841","nonce":794881135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/2836453158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2836453158},{"type":"v1","addr":"172.21.15.151:6833","nonce":2836453158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/178785812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":178785812},{"type":"v1","addr":"172.21.15.96:6833","nonce":178785812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1671043820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1671043820},{"type":"v1","addr":"172.21.15.96:6835","nonce":1671043820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2972514328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2972514328},{"type":"v1","addr":"172.21.15.151:6835","nonce":2972514328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.151:6837/3960378010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3960378010},{"type":"v1","addr":"172.21.15.151:6837","nonce":3960378010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6837/3978388286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3978388286},{"type":"v1","addr":"172.21.15.96:6837","nonce":3978388286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.151:6839/3794104313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3794104313},{"type":"v1","addr":"172.21.15.151:6839","nonce":3794104313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.43:6839/3404555030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3404555030},{"type":"v1","addr":"172.21.15.43:6839","nonce":3404555030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/605527621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":605527621},{"type":"v1","addr":"172.21.15.96:6839","nonce":605527621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24518,24511,24506,24493]},"id":1}]} 2024-09-19T05:13:21.172 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 29 2024-09-19T05:13:21.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:21.172+0000 7fdcc8400640 1 -- 172.21.15.43:0/4227322424 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fdca80a33b0 msgr2=0x7fdca80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:21.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:21.172+0000 7fdcc8400640 1 --2- 172.21.15.43:0/4227322424 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fdca80a33b0 0x7fdca80a5870 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4009fc0 tx=0x7fdcb4005f90 comp rx=0 tx=0).stop 2024-09-19T05:13:21.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:21.172+0000 7fdcc8400640 1 -- 172.21.15.43:0/4227322424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdcc01049a0 msgr2=0x7fdcc019aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:21.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:21.172+0000 7fdcc8400640 1 --2- 172.21.15.43:0/4227322424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdcc01049a0 0x7fdcc019aaf0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fdcac00e9f0 tx=0x7fdcac00eeb0 comp rx=0 tx=0).stop 2024-09-19T05:13:21.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:21.173+0000 7fdcc8400640 1 -- 172.21.15.43:0/4227322424 shutdown_connections 2024-09-19T05:13:21.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:21.173+0000 7fdcc8400640 1 --2- 172.21.15.43:0/4227322424 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcc0109120 0x7fdcc019ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:21.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:21.173+0000 7fdcc8400640 1 --2- 172.21.15.43:0/4227322424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdcc01049a0 0x7fdcc019aaf0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:21.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:21.173+0000 7fdcc8400640 1 --2- 172.21.15.43:0/4227322424 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fdca80a33b0 0x7fdca80a5870 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:21.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:21.173+0000 7fdcc8400640 1 --2- 172.21.15.43:0/4227322424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcc0103ff0 0x7fdcc019a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:21.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:21.173+0000 7fdcc8400640 1 -- 172.21.15.43:0/4227322424 >> 172.21.15.43:0/4227322424 conn(0x7fdcc00ffa90 msgr2=0x7fdcc0101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:21.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:21.173+0000 7fdcc8400640 1 -- 172.21.15.43:0/4227322424 shutdown_connections 2024-09-19T05:13:21.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:21.173+0000 7fdcc8400640 1 -- 172.21.15.43:0/4227322424 wait complete. 2024-09-19T05:13:21.935 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T05:13:22.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:21 smithi151 ceph-mon[35425]: pgmap v270: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T05:13:22.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:21 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/4227322424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:13:22.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:21 smithi096 ceph-mon[35085]: pgmap v270: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T05:13:22.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:21 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/4227322424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:13:22.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:21 smithi043 ceph-mon[30800]: pgmap v270: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T05:13:22.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:21 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/4227322424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:13:22.313 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:13:23.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.031+0000 7fcd26200640 1 -- 172.21.15.43:0/3887113668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd201019c0 msgr2=0x7fcd20101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:23.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.031+0000 7fcd26200640 1 --2- 172.21.15.43:0/3887113668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd201019c0 0x7fcd20101e20 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fcd08009930 tx=0x7fcd0802a420 comp rx=0 tx=0).stop 2024-09-19T05:13:23.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.031+0000 7fcd26200640 1 -- 172.21.15.43:0/3887113668 shutdown_connections 2024-09-19T05:13:23.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.031+0000 7fcd26200640 1 --2- 172.21.15.43:0/3887113668 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd20102360 0x7fcd2010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:23.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.031+0000 7fcd26200640 1 --2- 172.21.15.43:0/3887113668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd201019c0 0x7fcd20101e20 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:23.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.031+0000 7fcd26200640 1 --2- 172.21.15.43:0/3887113668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd201079c0 0x7fcd20107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:23.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.031+0000 7fcd26200640 1 -- 172.21.15.43:0/3887113668 >> 172.21.15.43:0/3887113668 conn(0x7fcd200fd740 msgr2=0x7fcd200ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:23.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.031+0000 7fcd26200640 1 -- 172.21.15.43:0/3887113668 shutdown_connections 2024-09-19T05:13:23.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.031+0000 7fcd26200640 1 -- 172.21.15.43:0/3887113668 wait complete. 2024-09-19T05:13:23.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.032+0000 7fcd26200640 1 Processor -- start 2024-09-19T05:13:23.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.032+0000 7fcd26200640 1 -- start start 2024-09-19T05:13:23.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.033+0000 7fcd26200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd201019c0 0x7fcd2019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:23.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.033+0000 7fcd26200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd20102360 0x7fcd2019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:23.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.033+0000 7fcd26200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd201079c0 0x7fcd2019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:23.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.033+0000 7fcd26200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcd20111dc0 con 0x7fcd20102360 2024-09-19T05:13:23.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.033+0000 7fcd26200640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcd20111c40 con 0x7fcd201019c0 2024-09-19T05:13:23.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.033+0000 7fcd26200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcd20111f40 con 0x7fcd201079c0 2024-09-19T05:13:23.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.033+0000 7fcd17e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd20102360 0x7fcd2019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:23.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.033+0000 7fcd24e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd201019c0 0x7fcd2019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:23.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.033+0000 7fcd17e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd20102360 0x7fcd2019a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50514/0 (socket says 172.21.15.43:50514) 2024-09-19T05:13:23.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.033+0000 7fcd17e00640 1 -- 172.21.15.43:0/1063941001 learned_addr learned my addr 172.21.15.43:0/1063941001 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:23.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.033+0000 7fcd24e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd201019c0 0x7fcd2019a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:41760/0 (socket says 172.21.15.43:41760) 2024-09-19T05:13:23.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.033+0000 7fcd25800640 1 --2- 172.21.15.43:0/1063941001 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd201079c0 0x7fcd2019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:23.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.033+0000 7fcd24e00640 1 -- 172.21.15.43:0/1063941001 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd201079c0 msgr2=0x7fcd2019ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:23.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.033+0000 7fcd24e00640 1 --2- 172.21.15.43:0/1063941001 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd201079c0 0x7fcd2019ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:23.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.033+0000 7fcd24e00640 1 -- 172.21.15.43:0/1063941001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd20102360 msgr2=0x7fcd2019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:23.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.033+0000 7fcd24e00640 1 --2- 172.21.15.43:0/1063941001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd20102360 0x7fcd2019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:23.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.033+0000 7fcd24e00640 1 -- 172.21.15.43:0/1063941001 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd2019f460 con 0x7fcd201019c0 2024-09-19T05:13:23.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.033+0000 7fcd17e00640 1 --2- 172.21.15.43:0/1063941001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd20102360 0x7fcd2019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:13:23.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.034+0000 7fcd24e00640 1 --2- 172.21.15.43:0/1063941001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd201019c0 0x7fcd2019a4b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcd1000d6d0 tx=0x7fcd1000db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:23.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.034+0000 7fcd1e000640 1 -- 172.21.15.43:0/1063941001 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd10017070 con 0x7fcd201019c0 2024-09-19T05:13:23.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.034+0000 7fcd26200640 1 -- 172.21.15.43:0/1063941001 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd2019f750 con 0x7fcd201019c0 2024-09-19T05:13:23.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.034+0000 7fcd1e000640 1 -- 172.21.15.43:0/1063941001 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd10009820 con 0x7fcd201019c0 2024-09-19T05:13:23.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.034+0000 7fcd1e000640 1 -- 172.21.15.43:0/1063941001 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd10004cd0 con 0x7fcd201019c0 2024-09-19T05:13:23.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.034+0000 7fcd26200640 1 -- 172.21.15.43:0/1063941001 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd201a7030 con 0x7fcd201019c0 2024-09-19T05:13:23.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.037+0000 7fcd2a254740 1 Processor -- start 2024-09-19T05:13:23.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.037+0000 7fcd2a254740 1 -- start start 2024-09-19T05:13:23.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.037+0000 7fcd1e000640 1 -- 172.21.15.43:0/1063941001 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fcd100099d0 con 0x7fcd201019c0 2024-09-19T05:13:23.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.038+0000 7fcd2a254740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cba3c26e40 0x55cba3c27220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:23.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.038+0000 7fcd2a254740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cba3c27760 0x55cba3c31ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:23.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.038+0000 7fcd2a254740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cba3c32020 0x55cba3c34410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:23.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.038+0000 7fcd2a254740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cba3b8d3e0 con 0x55cba3c32020 2024-09-19T05:13:23.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.038+0000 7fcd2a254740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55cba3b34d80 con 0x55cba3c27760 2024-09-19T05:13:23.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.038+0000 7fcd2a254740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55cba3b893a0 con 0x55cba3c26e40 2024-09-19T05:13:23.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.038+0000 7fcd17e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cba3c32020 0x55cba3c34410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:23.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.038+0000 7fcd25800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cba3c26e40 0x55cba3c27220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:23.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.038+0000 7fcd25800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cba3c26e40 0x55cba3c27220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:57144/0 (socket says 172.21.15.43:57144) 2024-09-19T05:13:23.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.038+0000 7fcd17e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cba3c32020 0x55cba3c34410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50530/0 (socket says 172.21.15.43:50530) 2024-09-19T05:13:23.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.038+0000 7fcd25800640 1 -- 172.21.15.43:0/4268189796 learned_addr learned my addr 172.21.15.43:0/4268189796 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:23.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.038+0000 7fcd24e00640 1 --2- 172.21.15.43:0/4268189796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cba3c27760 0x55cba3c31ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:23.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.038+0000 7fcd17e00640 1 -- 172.21.15.43:0/4268189796 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cba3c26e40 msgr2=0x55cba3c27220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:23.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.038+0000 7fcd17e00640 1 --2- 172.21.15.43:0/4268189796 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cba3c26e40 0x55cba3c27220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:23.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.038+0000 7fcd17e00640 1 -- 172.21.15.43:0/4268189796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cba3c27760 msgr2=0x55cba3c31ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:23.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.038+0000 7fcd17e00640 1 --2- 172.21.15.43:0/4268189796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cba3c27760 0x55cba3c31ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:23.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.038+0000 7fcd17e00640 1 -- 172.21.15.43:0/4268189796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cba3b53ae0 con 0x55cba3c32020 2024-09-19T05:13:23.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.039+0000 7fcd17e00640 1 --2- 172.21.15.43:0/4268189796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cba3c32020 0x55cba3c34410 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fcd08009900 tx=0x7fcd08005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=873a1c605353aa00 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:23.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.039+0000 7fcd16000640 1 -- 172.21.15.43:0/4268189796 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd08035040 con 0x55cba3c32020 2024-09-19T05:13:23.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.039+0000 7fcd1e000640 1 --2- 172.21.15.43:0/1063941001 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fcd040a33b0 0x7fcd040a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:23.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.039+0000 7fcd16000640 1 -- 172.21.15.43:0/4268189796 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd080318b0 con 0x55cba3c32020 2024-09-19T05:13:23.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.039+0000 7fcd16000640 1 -- 172.21.15.43:0/4268189796 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd080049e0 con 0x55cba3c32020 2024-09-19T05:13:23.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.039+0000 7fcd1e000640 1 -- 172.21.15.43:0/1063941001 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7fcd100cf830 con 0x7fcd201019c0 2024-09-19T05:13:23.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.040+0000 7fcd25800640 1 --2- 172.21.15.43:0/1063941001 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fcd040a33b0 0x7fcd040a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:23.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.040+0000 7fcd2a254740 1 -- 172.21.15.43:0/4268189796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cba3c32020 msgr2=0x55cba3c34410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:23.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.040+0000 7fcd2a254740 1 --2- 172.21.15.43:0/4268189796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cba3c32020 0x55cba3c34410 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fcd08009900 tx=0x7fcd08005e70 comp rx=0 tx=0).stop 2024-09-19T05:13:23.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.040+0000 7fcd25800640 1 --2- 172.21.15.43:0/1063941001 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fcd040a33b0 0x7fcd040a5870 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcd180116b0 tx=0x7fcd18009500 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:23.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.040+0000 7fcd2a254740 1 -- 172.21.15.43:0/4268189796 shutdown_connections 2024-09-19T05:13:23.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.040+0000 7fcd2a254740 1 --2- 172.21.15.43:0/4268189796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cba3c32020 0x55cba3c34410 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:23.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.040+0000 7fcd2a254740 1 --2- 172.21.15.43:0/4268189796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cba3c27760 0x55cba3c31ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:23.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.040+0000 7fcd2a254740 1 --2- 172.21.15.43:0/4268189796 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cba3c26e40 0x55cba3c27220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:23.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.040+0000 7fcd2a254740 1 -- 172.21.15.43:0/4268189796 >> 172.21.15.43:0/4268189796 conn(0x55cba3c233d0 msgr2=0x55cba3c237b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:23.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.040+0000 7fcd2a254740 1 -- 172.21.15.43:0/4268189796 shutdown_connections 2024-09-19T05:13:23.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.040+0000 7fcd2a254740 1 -- 172.21.15.43:0/4268189796 wait complete. 2024-09-19T05:13:23.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.041+0000 7fcd2a254740 1 -- start start 2024-09-19T05:13:23.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.041+0000 7fcd2a254740 1 Processor -- start 2024-09-19T05:13:23.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.042+0000 7fcd2a254740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cba3c26e40 0x55cba3c2ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:23.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.042+0000 7fcd2a254740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cba3c27760 0x55cba3c2f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:23.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.042+0000 7fcd2a254740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cba3c32020 0x55cba3c2fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:23.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.042+0000 7fcd2a254740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cba3c34bf0 con 0x55cba3c32020 2024-09-19T05:13:23.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.042+0000 7fcd2a254740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55cba3b480f0 con 0x55cba3c26e40 2024-09-19T05:13:23.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.042+0000 7fcd2a254740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55cba3c34d70 con 0x55cba3c27760 2024-09-19T05:13:23.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.042+0000 7fcd2a254740 10 client.0 _ll_register_callbacks cb 0x55cba3b2fbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:23.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.042+0000 7fcd25800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cba3c32020 0x55cba3c2fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:23.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.042+0000 7fcd25800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cba3c32020 0x55cba3c2fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50538/0 (socket says 172.21.15.43:50538) 2024-09-19T05:13:23.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.042+0000 7fcd25800640 1 -- 172.21.15.43:0/3617381660 learned_addr learned my addr 172.21.15.43:0/3617381660 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:23.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.042+0000 7fcd24e00640 1 --2- 172.21.15.43:0/3617381660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cba3c26e40 0x55cba3c2ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:23.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.042+0000 7fcd25800640 1 -- 172.21.15.43:0/3617381660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cba3c27760 msgr2=0x55cba3c2f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:23.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.042+0000 7fcd25800640 1 --2- 172.21.15.43:0/3617381660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cba3c27760 0x55cba3c2f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:23.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.042+0000 7fcd25800640 1 -- 172.21.15.43:0/3617381660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cba3c26e40 msgr2=0x55cba3c2ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:23.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.042+0000 7fcd25800640 1 --2- 172.21.15.43:0/3617381660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cba3c26e40 0x55cba3c2ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:23.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.043+0000 7fcd25800640 1 -- 172.21.15.43:0/3617381660 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cba38fe3c0 con 0x55cba3c32020 2024-09-19T05:13:23.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.043+0000 7fcd24e00640 1 --2- 172.21.15.43:0/3617381660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cba3c26e40 0x55cba3c2ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:13:23.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.043+0000 7fcd25800640 1 --2- 172.21.15.43:0/3617381660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cba3c32020 0x55cba3c2fa00 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fcd1800f780 tx=0x7fcd18004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ea33cc3a349a6cd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:23.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.043+0000 7fcd0ea00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:13:23.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.043+0000 7fcd0ea00640 1 -- 172.21.15.43:0/3617381660 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd18003970 con 0x55cba3c32020 2024-09-19T05:13:23.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.043+0000 7fcd2a254740 1 -- 172.21.15.43:0/3617381660 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd080318b0 con 0x55cba3c32020 2024-09-19T05:13:23.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.043+0000 7fcd0ea00640 1 -- 172.21.15.43:0/3617381660 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7fcd18003b10 con 0x55cba3c32020 2024-09-19T05:13:23.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.043+0000 7fcd0ea00640 1 -- 172.21.15.43:0/3617381660 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcd18003d10 con 0x55cba3c32020 2024-09-19T05:13:23.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.044+0000 7fcd2a254740 10 client.17244 fetch_fsmap learned FSMap version 29 2024-09-19T05:13:23.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.044+0000 7fcd2a254740 1 -- 172.21.15.43:0/3617381660 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cba3c40d60 con 0x55cba3c32020 2024-09-19T05:13:23.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.044+0000 7fcd0ea00640 1 -- 172.21.15.43:0/3617381660 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd080318b0 con 0x55cba3c32020 2024-09-19T05:13:23.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.044+0000 7fcd0ea00640 1 -- 172.21.15.43:0/3617381660 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcd18026070 con 0x55cba3c32020 2024-09-19T05:13:23.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.044+0000 7fcd2a254740 10 client.17244 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:13:23.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.044+0000 7fcd2a254740 10 client.17244 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-19T05:13:23.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.044+0000 7fcd2a254740 20 client.17244 populate_metadata read hostname 'smithi043' 2024-09-19T05:13:23.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.044+0000 7fcd2a254740 1 --2- 172.21.15.43:0/3617381660 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55cba3c42230 0x55cba3c446f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:23.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.044+0000 7fcd2a254740 4 client.17244 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:13:23.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.044+0000 7fcd2a254740 1 -- 172.21.15.43:0/3617381660 --> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cba3b9a950 con 0x55cba3c42230 2024-09-19T05:13:23.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.045+0000 7fcd24e00640 1 --2- 172.21.15.43:0/3617381660 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55cba3c42230 0x55cba3c446f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:23.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.045+0000 7fcd24e00640 1 --2- 172.21.15.43:0/3617381660 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55cba3c42230 0x55cba3c446f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=aa7e6c65fa06438f server_cookie=f52d45dc09d99338 in_seq=0 out_seq=0 2024-09-19T05:13:23.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.045+0000 7fcd0ea00640 10 client.17244 ms_handle_connect on v2:172.21.15.151:6836/3960378010 2024-09-19T05:13:23.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.046+0000 7fcd0ea00640 1 -- 172.21.15.43:0/3617381660 <== mds.4 v2:172.21.15.151:6836/3960378010 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cba3b9a950 con 0x55cba3c42230 2024-09-19T05:13:23.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.046+0000 7fcd0ea00640 10 client.17244 handle_command_reply: tid=0 2024-09-19T05:13:23.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.046+0000 7fcd2a254740 1 client.17244 shutdown 2024-09-19T05:13:23.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.046+0000 7fcd2a254740 1 -- 172.21.15.43:0/3617381660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cba3c32020 msgr2=0x55cba3c2fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:23.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.046+0000 7fcd2a254740 1 --2- 172.21.15.43:0/3617381660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cba3c32020 0x55cba3c2fa00 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fcd1800f780 tx=0x7fcd18004ab0 comp rx=0 tx=0).stop 2024-09-19T05:13:23.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.046+0000 7fcd2a254740 1 -- 172.21.15.43:0/3617381660 shutdown_connections 2024-09-19T05:13:23.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.046+0000 7fcd2a254740 1 --2- 172.21.15.43:0/3617381660 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55cba3c42230 0x55cba3c446f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:23.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.046+0000 7fcd2a254740 1 --2- 172.21.15.43:0/3617381660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cba3c32020 0x55cba3c2fa00 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:23.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.046+0000 7fcd2a254740 1 --2- 172.21.15.43:0/3617381660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cba3c27760 0x55cba3c2f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:23.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.046+0000 7fcd2a254740 1 --2- 172.21.15.43:0/3617381660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cba3c26e40 0x55cba3c2ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:23.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.046+0000 7fcd2a254740 1 -- 172.21.15.43:0/3617381660 >> 172.21.15.43:0/3617381660 conn(0x55cba3c233d0 msgr2=0x55cba3c290b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:23.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.046+0000 7fcd0ea00640 0 client.17244 ms_handle_reset on v2:172.21.15.151:6836/3960378010 2024-09-19T05:13:23.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.046+0000 7fcd2a254740 1 -- 172.21.15.43:0/3617381660 shutdown_connections 2024-09-19T05:13:23.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.046+0000 7fcd2a254740 1 -- 172.21.15.43:0/3617381660 wait complete. 2024-09-19T05:13:23.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.047+0000 7fcd2a254740 20 client.17244 trim_cache size 0 max 16384 2024-09-19T05:13:23.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.061+0000 7fcd2a254740 1 Processor -- start 2024-09-19T05:13:23.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.061+0000 7fcd2a254740 1 -- start start 2024-09-19T05:13:23.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.061+0000 7fcd2a254740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cba3c26e40 0x55cba3c30ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:23.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.061+0000 7fcd2a254740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cba3c27760 0x55cba3c49fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:23.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.061+0000 7fcd2a254740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cba3c32020 0x55cba3c4a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:23.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.061+0000 7fcd2a254740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcd201afa20 con 0x55cba3c32020 2024-09-19T05:13:23.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.061+0000 7fcd2a254740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcd201b1560 con 0x55cba3c26e40 2024-09-19T05:13:23.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.061+0000 7fcd2a254740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcd201b00f0 con 0x55cba3c27760 2024-09-19T05:13:23.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.061+0000 7fcd25800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cba3c32020 0x55cba3c4a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:23.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.061+0000 7fcd24e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cba3c26e40 0x55cba3c30ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:23.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.061+0000 7fcd25800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cba3c32020 0x55cba3c4a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50548/0 (socket says 172.21.15.43:50548) 2024-09-19T05:13:23.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.062+0000 7fcd25800640 1 -- 172.21.15.43:0/3043986964 learned_addr learned my addr 172.21.15.43:0/3043986964 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:23.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.062+0000 7fcd17e00640 1 --2- 172.21.15.43:0/3043986964 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cba3c27760 0x55cba3c49fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:23.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.062+0000 7fcd25800640 1 -- 172.21.15.43:0/3043986964 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cba3c27760 msgr2=0x55cba3c49fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:23.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.062+0000 7fcd25800640 1 --2- 172.21.15.43:0/3043986964 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cba3c27760 0x55cba3c49fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:23.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.062+0000 7fcd25800640 1 -- 172.21.15.43:0/3043986964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cba3c26e40 msgr2=0x55cba3c30ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:23.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.062+0000 7fcd25800640 1 --2- 172.21.15.43:0/3043986964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cba3c26e40 0x55cba3c30ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:23.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.062+0000 7fcd25800640 1 -- 172.21.15.43:0/3043986964 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cba3c318f0 con 0x55cba3c32020 2024-09-19T05:13:23.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.062+0000 7fcd24e00640 1 --2- 172.21.15.43:0/3043986964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cba3c26e40 0x55cba3c30ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:13:23.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.062+0000 7fcd25800640 1 --2- 172.21.15.43:0/3043986964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cba3c32020 0x55cba3c4a500 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fcd180046c0 tx=0x7fcd1800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a20e360e1e050a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:23.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.062+0000 7fcd15600640 1 -- 172.21.15.43:0/3043986964 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd18003b70 con 0x55cba3c32020 2024-09-19T05:13:23.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.062+0000 7fcd15600640 1 -- 172.21.15.43:0/3043986964 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd18004390 con 0x55cba3c32020 2024-09-19T05:13:23.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.063+0000 7fcd15600640 1 -- 172.21.15.43:0/3043986964 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd180286a0 con 0x55cba3c32020 2024-09-19T05:13:23.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.063+0000 7fcd2a254740 1 -- 172.21.15.43:0/3043986964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cba3c32020 msgr2=0x55cba3c4a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:23.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.063+0000 7fcd2a254740 1 --2- 172.21.15.43:0/3043986964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cba3c32020 0x55cba3c4a500 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fcd180046c0 tx=0x7fcd1800f690 comp rx=0 tx=0).stop 2024-09-19T05:13:23.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.063+0000 7fcd2a254740 1 -- 172.21.15.43:0/3043986964 shutdown_connections 2024-09-19T05:13:23.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.063+0000 7fcd2a254740 1 --2- 172.21.15.43:0/3043986964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cba3c32020 0x55cba3c4a500 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:23.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.063+0000 7fcd2a254740 1 --2- 172.21.15.43:0/3043986964 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cba3c27760 0x55cba3c49fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:23.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.063+0000 7fcd2a254740 1 --2- 172.21.15.43:0/3043986964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cba3c26e40 0x55cba3c30ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:23.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.063+0000 7fcd2a254740 1 -- 172.21.15.43:0/3043986964 >> 172.21.15.43:0/3043986964 conn(0x55cba3c233d0 msgr2=0x55cba3c307e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:23.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.063+0000 7fcd2a254740 1 -- 172.21.15.43:0/3043986964 shutdown_connections 2024-09-19T05:13:23.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.063+0000 7fcd2a254740 1 -- 172.21.15.43:0/3043986964 wait complete. 2024-09-19T05:13:23.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.064+0000 7fcd2a254740 1 -- start start 2024-09-19T05:13:23.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.064+0000 7fcd2a254740 1 Processor -- start 2024-09-19T05:13:23.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.064+0000 7fcd2a254740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cba3c26e40 0x55cba3c3a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:23.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.064+0000 7fcd2a254740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cba3c27760 0x55cba3c3a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:23.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.064+0000 7fcd2a254740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cba3c32020 0x55cba3c2bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:23.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.064+0000 7fcd2a254740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcd201b0e90 con 0x55cba3c27760 2024-09-19T05:13:23.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.064+0000 7fcd2a254740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcd201b07c0 con 0x55cba3c32020 2024-09-19T05:13:23.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.064+0000 7fcd2a254740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcd201ada20 con 0x55cba3c26e40 2024-09-19T05:13:23.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.064+0000 7fcd2a254740 10 client.0 _ll_register_callbacks cb 0x7fcd18009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:23.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.065+0000 7fcd24e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cba3c26e40 0x55cba3c3a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:23.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.065+0000 7fcd17e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cba3c27760 0x55cba3c3a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:23.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.065+0000 7fcd24e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cba3c26e40 0x55cba3c3a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:57170/0 (socket says 172.21.15.43:57170) 2024-09-19T05:13:23.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.065+0000 7fcd24e00640 1 -- 172.21.15.43:0/2961471144 learned_addr learned my addr 172.21.15.43:0/2961471144 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:23.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.065+0000 7fcd25800640 1 --2- 172.21.15.43:0/2961471144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cba3c32020 0x55cba3c2bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:23.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.065+0000 7fcd24e00640 1 -- 172.21.15.43:0/2961471144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cba3c32020 msgr2=0x55cba3c2bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:23.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.065+0000 7fcd24e00640 1 --2- 172.21.15.43:0/2961471144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cba3c32020 0x55cba3c2bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:23.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.065+0000 7fcd24e00640 1 -- 172.21.15.43:0/2961471144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cba3c27760 msgr2=0x55cba3c3a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:23.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.065+0000 7fcd24e00640 1 --2- 172.21.15.43:0/2961471144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cba3c27760 0x55cba3c3a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:23.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.065+0000 7fcd24e00640 1 -- 172.21.15.43:0/2961471144 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cba3c2c3b0 con 0x55cba3c26e40 2024-09-19T05:13:23.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.065+0000 7fcd25800640 1 --2- 172.21.15.43:0/2961471144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cba3c32020 0x55cba3c2bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:13:23.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.065+0000 7fcd17e00640 1 --2- 172.21.15.43:0/2961471144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cba3c27760 0x55cba3c3a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:13:23.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.066+0000 7fcd24e00640 1 --2- 172.21.15.43:0/2961471144 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cba3c26e40 0x55cba3c3a330 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcd1009bae0 tx=0x7fcd100d5530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7edcd673adbce37 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:23.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.066+0000 7fcd0ea00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:13:23.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.066+0000 7fcd0ea00640 1 -- 172.21.15.43:0/2961471144 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd100b3020 con 0x55cba3c26e40 2024-09-19T05:13:23.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.066+0000 7fcd2a254740 1 -- 172.21.15.43:0/2961471144 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd18004390 con 0x55cba3c26e40 2024-09-19T05:13:23.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.066+0000 7fcd0ea00640 1 -- 172.21.15.43:0/2961471144 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7fcd100981c0 con 0x55cba3c26e40 2024-09-19T05:13:23.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.066+0000 7fcd0ea00640 1 -- 172.21.15.43:0/2961471144 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcd10020040 con 0x55cba3c26e40 2024-09-19T05:13:23.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.066+0000 7fcd2a254740 10 client.27172 fetch_fsmap learned FSMap version 29 2024-09-19T05:13:23.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.066+0000 7fcd2a254740 1 -- 172.21.15.43:0/2961471144 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cba3cacc80 con 0x55cba3c26e40 2024-09-19T05:13:23.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.066+0000 7fcd0ea00640 1 -- 172.21.15.43:0/2961471144 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd18004390 con 0x55cba3c26e40 2024-09-19T05:13:23.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.067+0000 7fcd0ea00640 1 -- 172.21.15.43:0/2961471144 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcd100a4310 con 0x55cba3c26e40 2024-09-19T05:13:23.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.067+0000 7fcd2a254740 10 client.27172 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:13:23.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.067+0000 7fcd2a254740 10 client.27172 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-19T05:13:23.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.067+0000 7fcd2a254740 20 client.27172 populate_metadata read hostname 'smithi043' 2024-09-19T05:13:23.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.067+0000 7fcd2a254740 1 --2- 172.21.15.43:0/2961471144 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55cba3cad350 0x55cba3caf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:23.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.067+0000 7fcd2a254740 4 client.27172 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:13:23.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.067+0000 7fcd2a254740 1 -- 172.21.15.43:0/2961471144 --> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cba3b94e60 con 0x55cba3cad350 2024-09-19T05:13:23.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.067+0000 7fcd17e00640 1 --2- 172.21.15.43:0/2961471144 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55cba3cad350 0x55cba3caf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:23.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.068+0000 7fcd17e00640 1 --2- 172.21.15.43:0/2961471144 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55cba3cad350 0x55cba3caf770 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fd4a78b653501e34 server_cookie=26ba76f71cc7736c in_seq=0 out_seq=0 2024-09-19T05:13:23.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.068+0000 7fcd0ea00640 10 client.27172 ms_handle_connect on v2:172.21.15.151:6836/3960378010 2024-09-19T05:13:23.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.068+0000 7fcd0ea00640 1 -- 172.21.15.43:0/2961471144 <== mds.4 v2:172.21.15.151:6836/3960378010 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cba3b94e60 con 0x55cba3cad350 2024-09-19T05:13:23.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.068+0000 7fcd0ea00640 10 client.27172 handle_command_reply: tid=0 2024-09-19T05:13:23.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.068+0000 7fcd2a254740 1 client.27172 shutdown 2024-09-19T05:13:23.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.068+0000 7fcd2a254740 1 -- 172.21.15.43:0/2961471144 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cba3c26e40 msgr2=0x55cba3c3a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:23.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.068+0000 7fcd2a254740 1 --2- 172.21.15.43:0/2961471144 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cba3c26e40 0x55cba3c3a330 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcd1009bae0 tx=0x7fcd100d5530 comp rx=0 tx=0).stop 2024-09-19T05:13:23.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.068+0000 7fcd2a254740 1 -- 172.21.15.43:0/2961471144 shutdown_connections 2024-09-19T05:13:23.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.068+0000 7fcd2a254740 1 --2- 172.21.15.43:0/2961471144 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55cba3cad350 0x55cba3caf770 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:23.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.068+0000 7fcd2a254740 1 --2- 172.21.15.43:0/2961471144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cba3c32020 0x55cba3c2bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:23.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.068+0000 7fcd2a254740 1 --2- 172.21.15.43:0/2961471144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cba3c27760 0x55cba3c3a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:23.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.068+0000 7fcd2a254740 1 --2- 172.21.15.43:0/2961471144 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cba3c26e40 0x55cba3c3a330 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:23.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.068+0000 7fcd2a254740 1 -- 172.21.15.43:0/2961471144 >> 172.21.15.43:0/2961471144 conn(0x55cba3c233d0 msgr2=0x55cba3c22490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:23.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.068+0000 7fcd0ea00640 0 client.27172 ms_handle_reset on v2:172.21.15.151:6836/3960378010 2024-09-19T05:13:23.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.068+0000 7fcd2a254740 1 -- 172.21.15.43:0/2961471144 shutdown_connections 2024-09-19T05:13:23.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.068+0000 7fcd2a254740 1 -- 172.21.15.43:0/2961471144 wait complete. 2024-09-19T05:13:23.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.069+0000 7fcd2a254740 20 client.27172 trim_cache size 0 max 16384 2024-09-19T05:13:23.075 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:13:23.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.069+0000 7fcd26200640 1 -- 172.21.15.43:0/1063941001 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fcd040a33b0 msgr2=0x7fcd040a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:23.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.069+0000 7fcd26200640 1 --2- 172.21.15.43:0/1063941001 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fcd040a33b0 0x7fcd040a5870 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcd180116b0 tx=0x7fcd18009500 comp rx=0 tx=0).stop 2024-09-19T05:13:23.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.069+0000 7fcd26200640 1 -- 172.21.15.43:0/1063941001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd201019c0 msgr2=0x7fcd2019a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:23.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.069+0000 7fcd26200640 1 --2- 172.21.15.43:0/1063941001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd201019c0 0x7fcd2019a4b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcd1000d6d0 tx=0x7fcd1000db90 comp rx=0 tx=0).stop 2024-09-19T05:13:23.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.069+0000 7fcd26200640 1 -- 172.21.15.43:0/1063941001 shutdown_connections 2024-09-19T05:13:23.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.069+0000 7fcd26200640 1 --2- 172.21.15.43:0/1063941001 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd201079c0 0x7fcd2019ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:23.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.069+0000 7fcd26200640 1 --2- 172.21.15.43:0/1063941001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd20102360 0x7fcd2019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:23.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.069+0000 7fcd26200640 1 --2- 172.21.15.43:0/1063941001 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fcd040a33b0 0x7fcd040a5870 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:23.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.069+0000 7fcd26200640 1 --2- 172.21.15.43:0/1063941001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd201019c0 0x7fcd2019a4b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:23.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.069+0000 7fcd26200640 1 -- 172.21.15.43:0/1063941001 >> 172.21.15.43:0/1063941001 conn(0x7fcd200fd740 msgr2=0x7fcd200fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:23.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.070+0000 7fcd26200640 1 -- 172.21.15.43:0/1063941001 shutdown_connections 2024-09-19T05:13:23.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:23.070+0000 7fcd26200640 1 -- 172.21.15.43:0/1063941001 wait complete. 2024-09-19T05:13:23.827 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T05:13:24.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:23 smithi151 ceph-mon[35425]: pgmap v271: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 88 B/s wr, 9 op/s 2024-09-19T05:13:24.207 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:13:24.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:23 smithi096 ceph-mon[35085]: pgmap v271: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 88 B/s wr, 9 op/s 2024-09-19T05:13:24.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:23 smithi043 ceph-mon[30800]: pgmap v271: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 88 B/s wr, 9 op/s 2024-09-19T05:13:24.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.765+0000 7fbaa5400640 1 -- 172.21.15.43:0/1412032918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbaa0103e30 msgr2=0x7fbaa0104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:24.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.765+0000 7fbaa5400640 1 --2- 172.21.15.43:0/1412032918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbaa0103e30 0x7fbaa0104210 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fba90009900 tx=0x7fba9002a410 comp rx=0 tx=0).stop 2024-09-19T05:13:24.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.765+0000 7fbaa5400640 1 -- 172.21.15.43:0/1412032918 shutdown_connections 2024-09-19T05:13:24.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.765+0000 7fbaa5400640 1 --2- 172.21.15.43:0/1412032918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbaa0108f60 0x7fbaa0110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:24.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.765+0000 7fbaa5400640 1 --2- 172.21.15.43:0/1412032918 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbaa01047e0 0x7fbaa0108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:24.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.765+0000 7fbaa5400640 1 --2- 172.21.15.43:0/1412032918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbaa0103e30 0x7fbaa0104210 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:24.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.765+0000 7fbaa5400640 1 -- 172.21.15.43:0/1412032918 >> 172.21.15.43:0/1412032918 conn(0x7fbaa00ff8d0 msgr2=0x7fbaa0101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:24.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.766+0000 7fbaa5400640 1 -- 172.21.15.43:0/1412032918 shutdown_connections 2024-09-19T05:13:24.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.766+0000 7fbaa5400640 1 -- 172.21.15.43:0/1412032918 wait complete. 2024-09-19T05:13:24.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.766+0000 7fbaa5400640 1 Processor -- start 2024-09-19T05:13:24.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.767+0000 7fbaa5400640 1 -- start start 2024-09-19T05:13:24.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.767+0000 7fbaa5400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbaa0103e30 0x7fbaa019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:24.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.767+0000 7fbaa5400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbaa01047e0 0x7fbaa019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:24.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.767+0000 7fbaa5400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbaa0108f60 0x7fbaa019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:24.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.767+0000 7fbaa5400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa0114010 con 0x7fbaa01047e0 2024-09-19T05:13:24.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.767+0000 7fbaa5400640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa0113e90 con 0x7fbaa0108f60 2024-09-19T05:13:24.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.767+0000 7fbaa5400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa0114190 con 0x7fbaa0103e30 2024-09-19T05:13:24.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.767+0000 7fba97e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbaa01047e0 0x7fbaa019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:24.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.767+0000 7fba9fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbaa0103e30 0x7fbaa019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:24.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.767+0000 7fba9fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbaa0103e30 0x7fbaa019a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:57184/0 (socket says 172.21.15.43:57184) 2024-09-19T05:13:24.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.767+0000 7fba9fe00640 1 -- 172.21.15.43:0/857554034 learned_addr learned my addr 172.21.15.43:0/857554034 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:24.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.767+0000 7fbaa4a00640 1 --2- 172.21.15.43:0/857554034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbaa0108f60 0x7fbaa019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:24.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.768+0000 7fba9fe00640 1 -- 172.21.15.43:0/857554034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbaa0108f60 msgr2=0x7fbaa019ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:24.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.768+0000 7fba9fe00640 1 --2- 172.21.15.43:0/857554034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbaa0108f60 0x7fbaa019ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:24.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.768+0000 7fba9fe00640 1 -- 172.21.15.43:0/857554034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbaa01047e0 msgr2=0x7fbaa019a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:24.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.768+0000 7fba9fe00640 1 --2- 172.21.15.43:0/857554034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbaa01047e0 0x7fbaa019a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:24.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.768+0000 7fba9fe00640 1 -- 172.21.15.43:0/857554034 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaa019f450 con 0x7fbaa0103e30 2024-09-19T05:13:24.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.768+0000 7fba9fe00640 1 --2- 172.21.15.43:0/857554034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbaa0103e30 0x7fbaa019a4a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fba90004770 tx=0x7fba90003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:24.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.769+0000 7fba9d600640 1 -- 172.21.15.43:0/857554034 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9003f070 con 0x7fbaa0103e30 2024-09-19T05:13:24.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.769+0000 7fbaa5400640 1 -- 172.21.15.43:0/857554034 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaa019f6e0 con 0x7fbaa0103e30 2024-09-19T05:13:24.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.769+0000 7fba9d600640 1 -- 172.21.15.43:0/857554034 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba90031ca0 con 0x7fbaa0103e30 2024-09-19T05:13:24.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.769+0000 7fba9d600640 1 -- 172.21.15.43:0/857554034 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba90035030 con 0x7fbaa0103e30 2024-09-19T05:13:24.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.769+0000 7fbaa5400640 1 -- 172.21.15.43:0/857554034 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaa01a7020 con 0x7fbaa0103e30 2024-09-19T05:13:24.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.772+0000 7fbaa9485740 1 Processor -- start 2024-09-19T05:13:24.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.772+0000 7fbaa9485740 1 -- start start 2024-09-19T05:13:24.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.772+0000 7fba9d600640 1 -- 172.21.15.43:0/857554034 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fba90042070 con 0x7fbaa0103e30 2024-09-19T05:13:24.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.772+0000 7fbaa9485740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573801e8e40 0x5573801e9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:24.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.772+0000 7fbaa9485740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573801e9760 0x5573801f3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:24.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.772+0000 7fbaa9485740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573801f4020 0x5573801f6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:24.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.772+0000 7fbaa9485740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55738014f3e0 con 0x5573801e9760 2024-09-19T05:13:24.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.772+0000 7fbaa9485740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5573800f6d80 con 0x5573801f4020 2024-09-19T05:13:24.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.772+0000 7fbaa9485740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55738014b3a0 con 0x5573801e8e40 2024-09-19T05:13:24.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.773+0000 7fba9fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573801e9760 0x5573801f3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:24.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.773+0000 7fbaa4a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573801e8e40 0x5573801e9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:24.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.773+0000 7fba9fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573801e9760 0x5573801f3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50588/0 (socket says 172.21.15.43:50588) 2024-09-19T05:13:24.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.773+0000 7fba9fe00640 1 -- 172.21.15.43:0/4018939340 learned_addr learned my addr 172.21.15.43:0/4018939340 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:24.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.773+0000 7fba97e00640 1 --2- 172.21.15.43:0/4018939340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573801f4020 0x5573801f6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:24.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.773+0000 7fbaa4a00640 1 -- 172.21.15.43:0/4018939340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573801f4020 msgr2=0x5573801f6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:24.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.773+0000 7fbaa4a00640 1 --2- 172.21.15.43:0/4018939340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573801f4020 0x5573801f6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:24.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.773+0000 7fbaa4a00640 1 -- 172.21.15.43:0/4018939340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573801e9760 msgr2=0x5573801f3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:24.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.773+0000 7fbaa4a00640 1 --2- 172.21.15.43:0/4018939340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573801e9760 0x5573801f3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:24.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.773+0000 7fbaa4a00640 1 -- 172.21.15.43:0/4018939340 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557380115ae0 con 0x5573801e8e40 2024-09-19T05:13:24.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.773+0000 7fba97e00640 1 --2- 172.21.15.43:0/4018939340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573801f4020 0x5573801f6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:13:24.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.774+0000 7fbaa4a00640 1 --2- 172.21.15.43:0/4018939340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573801e8e40 0x5573801e9220 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fba98009f10 tx=0x7fba98011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c719306b71121cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:24.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.774+0000 7fba95200640 1 -- 172.21.15.43:0/4018939340 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba98004d20 con 0x5573801e8e40 2024-09-19T05:13:24.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.774+0000 7fba95200640 1 -- 172.21.15.43:0/4018939340 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba98004050 con 0x5573801e8e40 2024-09-19T05:13:24.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.774+0000 7fba95200640 1 -- 172.21.15.43:0/4018939340 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba98005220 con 0x5573801e8e40 2024-09-19T05:13:24.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.774+0000 7fba9d600640 1 --2- 172.21.15.43:0/857554034 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fba840a33b0 0x7fba840a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:24.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.774+0000 7fba9d600640 1 -- 172.21.15.43:0/857554034 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7fba900f02f0 con 0x7fbaa0103e30 2024-09-19T05:13:24.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.774+0000 7fba9fe00640 1 --2- 172.21.15.43:0/857554034 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fba840a33b0 0x7fba840a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:24.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.774+0000 7fbaa9485740 1 -- 172.21.15.43:0/4018939340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573801e8e40 msgr2=0x5573801e9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:24.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.774+0000 7fbaa9485740 1 --2- 172.21.15.43:0/4018939340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573801e8e40 0x5573801e9220 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fba98009f10 tx=0x7fba98011810 comp rx=0 tx=0).stop 2024-09-19T05:13:24.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.775+0000 7fbaa9485740 1 -- 172.21.15.43:0/4018939340 shutdown_connections 2024-09-19T05:13:24.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.775+0000 7fbaa9485740 1 --2- 172.21.15.43:0/4018939340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573801f4020 0x5573801f6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:24.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.775+0000 7fbaa9485740 1 --2- 172.21.15.43:0/4018939340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573801e9760 0x5573801f3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:24.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.775+0000 7fbaa9485740 1 --2- 172.21.15.43:0/4018939340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573801e8e40 0x5573801e9220 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:24.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.775+0000 7fbaa9485740 1 -- 172.21.15.43:0/4018939340 >> 172.21.15.43:0/4018939340 conn(0x5573801e53d0 msgr2=0x5573801e57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:24.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.775+0000 7fbaa9485740 1 -- 172.21.15.43:0/4018939340 shutdown_connections 2024-09-19T05:13:24.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.775+0000 7fbaa9485740 1 -- 172.21.15.43:0/4018939340 wait complete. 2024-09-19T05:13:24.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.775+0000 7fba9fe00640 1 --2- 172.21.15.43:0/857554034 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fba840a33b0 0x7fba840a5870 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fba900c4d00 tx=0x7fba900f2800 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:24.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.775+0000 7fbaa9485740 1 -- start start 2024-09-19T05:13:24.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.776+0000 7fbaa9485740 1 Processor -- start 2024-09-19T05:13:24.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.776+0000 7fbaa9485740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573801e8e40 0x5573801f0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:24.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.776+0000 7fbaa9485740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573801e9760 0x5573801f1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:24.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.776+0000 7fbaa9485740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573801f4020 0x5573801f1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:24.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.776+0000 7fbaa9485740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5573801f6bf0 con 0x5573801f4020 2024-09-19T05:13:24.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.776+0000 7fbaa9485740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55738010a0f0 con 0x5573801e8e40 2024-09-19T05:13:24.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.776+0000 7fbaa9485740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5573801f6d70 con 0x5573801e9760 2024-09-19T05:13:24.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.776+0000 7fbaa9485740 10 client.0 _ll_register_callbacks cb 0x5573800f1bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:24.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.776+0000 7fbaa4a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573801f4020 0x5573801f1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:24.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.777+0000 7fba97e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573801e9760 0x5573801f1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:24.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.777+0000 7fbaa4a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573801f4020 0x5573801f1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50602/0 (socket says 172.21.15.43:50602) 2024-09-19T05:13:24.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.777+0000 7fbaa4a00640 1 -- 172.21.15.43:0/1799916882 learned_addr learned my addr 172.21.15.43:0/1799916882 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:24.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.777+0000 7fbaa4a00640 1 --2- 172.21.15.43:0/1799916882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573801e8e40 0x5573801f0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:24.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.777+0000 7fba97e00640 1 -- 172.21.15.43:0/1799916882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573801e8e40 msgr2=0x5573801f0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:24.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.777+0000 7fba97e00640 1 --2- 172.21.15.43:0/1799916882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573801e8e40 0x5573801f0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:24.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.777+0000 7fba97e00640 1 -- 172.21.15.43:0/1799916882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573801f4020 msgr2=0x5573801f1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:24.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.777+0000 7fba97e00640 1 --2- 172.21.15.43:0/1799916882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573801f4020 0x5573801f1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:24.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.777+0000 7fba97e00640 1 -- 172.21.15.43:0/1799916882 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55737fec03c0 con 0x5573801e9760 2024-09-19T05:13:24.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.777+0000 7fba97e00640 1 --2- 172.21.15.43:0/1799916882 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573801e9760 0x5573801f1430 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fba880126d0 tx=0x7fba88012b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a34e047da0dc397a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:24.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.777+0000 7fba8d600640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:13:24.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.778+0000 7fba8d600640 1 -- 172.21.15.43:0/1799916882 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba88004030 con 0x5573801e9760 2024-09-19T05:13:24.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.778+0000 7fba8d600640 1 -- 172.21.15.43:0/1799916882 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7fba880041d0 con 0x5573801e9760 2024-09-19T05:13:24.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.778+0000 7fbaa9485740 1 -- 172.21.15.43:0/1799916882 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba98004050 con 0x5573801e9760 2024-09-19T05:13:24.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.778+0000 7fba8d600640 1 -- 172.21.15.43:0/1799916882 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fba880043d0 con 0x5573801e9760 2024-09-19T05:13:24.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.778+0000 7fbaa9485740 10 client.27184 fetch_fsmap learned FSMap version 29 2024-09-19T05:13:24.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.778+0000 7fbaa9485740 1 -- 172.21.15.43:0/1799916882 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557380202c70 con 0x5573801e9760 2024-09-19T05:13:24.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.778+0000 7fba8d600640 1 -- 172.21.15.43:0/1799916882 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba98004050 con 0x5573801e9760 2024-09-19T05:13:24.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.779+0000 7fba8d600640 1 -- 172.21.15.43:0/1799916882 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fba88003bc0 con 0x5573801e9760 2024-09-19T05:13:24.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.779+0000 7fbaa9485740 10 client.27184 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:13:24.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.779+0000 7fbaa9485740 10 client.27184 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T05:13:24.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.779+0000 7fbaa9485740 20 client.27184 populate_metadata read hostname 'smithi043' 2024-09-19T05:13:24.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.779+0000 7fbaa9485740 1 --2- 172.21.15.43:0/1799916882 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x557380204180 0x5573802065a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:24.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.779+0000 7fbaa9485740 4 client.27184 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:13:24.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.779+0000 7fbaa9485740 1 -- 172.21.15.43:0/1799916882 --> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55738015c950 con 0x557380204180 2024-09-19T05:13:24.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.779+0000 7fbaa4a00640 1 --2- 172.21.15.43:0/1799916882 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x557380204180 0x5573802065a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:24.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.780+0000 7fbaa4a00640 1 --2- 172.21.15.43:0/1799916882 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x557380204180 0x5573802065a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=28d25637eeec415c server_cookie=5de0e39c377c47a2 in_seq=0 out_seq=0 2024-09-19T05:13:24.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.780+0000 7fba8d600640 10 client.27184 ms_handle_connect on v2:172.21.15.96:6836/3978388286 2024-09-19T05:13:24.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.780+0000 7fba8d600640 1 -- 172.21.15.43:0/1799916882 <== mds.3 v2:172.21.15.96:6836/3978388286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55738015c950 con 0x557380204180 2024-09-19T05:13:24.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.780+0000 7fba8d600640 10 client.27184 handle_command_reply: tid=0 2024-09-19T05:13:24.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.780+0000 7fbaa9485740 1 client.27184 shutdown 2024-09-19T05:13:24.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.781+0000 7fbaa9485740 1 -- 172.21.15.43:0/1799916882 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573801e9760 msgr2=0x5573801f1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:24.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.781+0000 7fbaa9485740 1 --2- 172.21.15.43:0/1799916882 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573801e9760 0x5573801f1430 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fba880126d0 tx=0x7fba88012b90 comp rx=0 tx=0).stop 2024-09-19T05:13:24.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.781+0000 7fbaa9485740 1 -- 172.21.15.43:0/1799916882 shutdown_connections 2024-09-19T05:13:24.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.781+0000 7fbaa9485740 1 --2- 172.21.15.43:0/1799916882 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x557380204180 0x5573802065a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:24.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.781+0000 7fbaa9485740 1 --2- 172.21.15.43:0/1799916882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573801f4020 0x5573801f1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:24.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.781+0000 7fbaa9485740 1 --2- 172.21.15.43:0/1799916882 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573801e9760 0x5573801f1430 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:24.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.781+0000 7fbaa9485740 1 --2- 172.21.15.43:0/1799916882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573801e8e40 0x5573801f0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:24.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.781+0000 7fba8d600640 0 client.27184 ms_handle_reset on v2:172.21.15.96:6836/3978388286 2024-09-19T05:13:24.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.781+0000 7fbaa9485740 1 -- 172.21.15.43:0/1799916882 >> 172.21.15.43:0/1799916882 conn(0x5573801e53d0 msgr2=0x5573801eb100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:24.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.781+0000 7fbaa9485740 1 -- 172.21.15.43:0/1799916882 shutdown_connections 2024-09-19T05:13:24.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.781+0000 7fbaa9485740 1 -- 172.21.15.43:0/1799916882 wait complete. 2024-09-19T05:13:24.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.781+0000 7fbaa9485740 20 client.27184 trim_cache size 0 max 16384 2024-09-19T05:13:24.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.795+0000 7fbaa9485740 1 Processor -- start 2024-09-19T05:13:24.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.795+0000 7fbaa9485740 1 -- start start 2024-09-19T05:13:24.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.795+0000 7fbaa9485740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573801e8e40 0x5573801ef7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:24.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.795+0000 7fbaa9485740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573801e9760 0x5573801f17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:24.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.795+0000 7fbaa9485740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573801f4020 0x55738020be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:24.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.795+0000 7fbaa9485740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa01af980 con 0x5573801e8e40 2024-09-19T05:13:24.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.795+0000 7fbaa9485740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa01b14c0 con 0x5573801f4020 2024-09-19T05:13:24.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.795+0000 7fbaa9485740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa01b0050 con 0x5573801e9760 2024-09-19T05:13:24.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.795+0000 7fbaa4a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573801e8e40 0x5573801ef7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:24.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.795+0000 7fbaa4a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573801e8e40 0x5573801ef7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50608/0 (socket says 172.21.15.43:50608) 2024-09-19T05:13:24.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.795+0000 7fbaa4a00640 1 -- 172.21.15.43:0/2226251123 learned_addr learned my addr 172.21.15.43:0/2226251123 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:24.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.796+0000 7fba97e00640 1 --2- 172.21.15.43:0/2226251123 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573801e9760 0x5573801f17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:24.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.796+0000 7fbaa4a00640 1 --2- 172.21.15.43:0/2226251123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573801f4020 0x55738020be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:24.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.796+0000 7fbaa4a00640 1 -- 172.21.15.43:0/2226251123 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573801e9760 msgr2=0x5573801f17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:24.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.796+0000 7fbaa4a00640 1 --2- 172.21.15.43:0/2226251123 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573801e9760 0x5573801f17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:24.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.796+0000 7fbaa4a00640 1 -- 172.21.15.43:0/2226251123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573801f4020 msgr2=0x55738020be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:24.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.796+0000 7fbaa4a00640 1 --2- 172.21.15.43:0/2226251123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573801f4020 0x55738020be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:24.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.796+0000 7fbaa4a00640 1 -- 172.21.15.43:0/2226251123 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573801e8c70 con 0x5573801e8e40 2024-09-19T05:13:24.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.796+0000 7fbaa4a00640 1 --2- 172.21.15.43:0/2226251123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573801e8e40 0x5573801ef7c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fba98005fd0 tx=0x7fba98009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=886ceaf62bba2497 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:24.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.796+0000 7fba8fe00640 1 -- 172.21.15.43:0/2226251123 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba98004000 con 0x5573801e8e40 2024-09-19T05:13:24.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.797+0000 7fba8fe00640 1 -- 172.21.15.43:0/2226251123 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba9802c020 con 0x5573801e8e40 2024-09-19T05:13:24.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.797+0000 7fba8fe00640 1 -- 172.21.15.43:0/2226251123 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba98028bd0 con 0x5573801e8e40 2024-09-19T05:13:24.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.797+0000 7fbaa9485740 1 -- 172.21.15.43:0/2226251123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573801e8e40 msgr2=0x5573801ef7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:24.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.797+0000 7fbaa9485740 1 --2- 172.21.15.43:0/2226251123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573801e8e40 0x5573801ef7c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fba98005fd0 tx=0x7fba98009500 comp rx=0 tx=0).stop 2024-09-19T05:13:24.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.797+0000 7fbaa9485740 1 -- 172.21.15.43:0/2226251123 shutdown_connections 2024-09-19T05:13:24.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.797+0000 7fbaa9485740 1 --2- 172.21.15.43:0/2226251123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573801f4020 0x55738020be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:24.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.797+0000 7fbaa9485740 1 --2- 172.21.15.43:0/2226251123 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573801e9760 0x5573801f17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:24.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.797+0000 7fbaa9485740 1 --2- 172.21.15.43:0/2226251123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573801e8e40 0x5573801ef7c0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:24.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.797+0000 7fbaa9485740 1 -- 172.21.15.43:0/2226251123 >> 172.21.15.43:0/2226251123 conn(0x5573801e53d0 msgr2=0x5573801f2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:24.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.797+0000 7fbaa9485740 1 -- 172.21.15.43:0/2226251123 shutdown_connections 2024-09-19T05:13:24.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.797+0000 7fbaa9485740 1 -- 172.21.15.43:0/2226251123 wait complete. 2024-09-19T05:13:24.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.798+0000 7fbaa9485740 1 -- start start 2024-09-19T05:13:24.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.798+0000 7fbaa9485740 1 Processor -- start 2024-09-19T05:13:24.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.798+0000 7fbaa9485740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573801e8e40 0x5573801fc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:24.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.798+0000 7fbaa9485740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573801e9760 0x5573801f7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:24.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.798+0000 7fbaa9485740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573801f4020 0x5573801f7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:24.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.798+0000 7fbaa9485740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa01b0df0 con 0x5573801f4020 2024-09-19T05:13:24.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.798+0000 7fbaa9485740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa01b0720 con 0x5573801e9760 2024-09-19T05:13:24.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.798+0000 7fbaa9485740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa01ad980 con 0x5573801e8e40 2024-09-19T05:13:24.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.799+0000 7fbaa9485740 10 client.0 _ll_register_callbacks cb 0x7fba88009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:24.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.799+0000 7fbaa4a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573801f4020 0x5573801f7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:24.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.799+0000 7fba97e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573801e9760 0x5573801f7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:24.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.799+0000 7fbaa4a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573801f4020 0x5573801f7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50620/0 (socket says 172.21.15.43:50620) 2024-09-19T05:13:24.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.799+0000 7fbaa4a00640 1 -- 172.21.15.43:0/1080173621 learned_addr learned my addr 172.21.15.43:0/1080173621 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:24.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.799+0000 7fbaa4a00640 1 --2- 172.21.15.43:0/1080173621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573801e8e40 0x5573801fc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:24.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.799+0000 7fba97e00640 1 -- 172.21.15.43:0/1080173621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573801e8e40 msgr2=0x5573801fc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:24.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.799+0000 7fba97e00640 1 --2- 172.21.15.43:0/1080173621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573801e8e40 0x5573801fc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:24.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.799+0000 7fba97e00640 1 -- 172.21.15.43:0/1080173621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573801f4020 msgr2=0x5573801f7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:24.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.799+0000 7fba97e00640 1 --2- 172.21.15.43:0/1080173621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573801f4020 0x5573801f7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:24.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.799+0000 7fba97e00640 1 -- 172.21.15.43:0/1080173621 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573801e6480 con 0x5573801e9760 2024-09-19T05:13:24.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.800+0000 7fba97e00640 1 --2- 172.21.15.43:0/1080173621 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573801e9760 0x5573801f7a00 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fba88000c00 tx=0x7fba88005d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=344142b400001cee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:24.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.800+0000 7fba8d600640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:13:24.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.800+0000 7fba8d600640 1 -- 172.21.15.43:0/1080173621 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba88028040 con 0x5573801e9760 2024-09-19T05:13:24.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.800+0000 7fbaa9485740 1 -- 172.21.15.43:0/1080173621 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba9802c020 con 0x5573801e9760 2024-09-19T05:13:24.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.800+0000 7fba8d600640 1 -- 172.21.15.43:0/1080173621 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7fba8800faa0 con 0x5573801e9760 2024-09-19T05:13:24.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.800+0000 7fba8d600640 1 -- 172.21.15.43:0/1080173621 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fba8800fca0 con 0x5573801e9760 2024-09-19T05:13:24.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.800+0000 7fbaa9485740 10 client.27290 fetch_fsmap learned FSMap version 29 2024-09-19T05:13:24.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.800+0000 7fbaa9485740 1 -- 172.21.15.43:0/1080173621 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55738026ec20 con 0x5573801e9760 2024-09-19T05:13:24.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.800+0000 7fba8d600640 1 -- 172.21.15.43:0/1080173621 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba9802c020 con 0x5573801e9760 2024-09-19T05:13:24.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.801+0000 7fba8d600640 1 -- 172.21.15.43:0/1080173621 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fba88010a40 con 0x5573801e9760 2024-09-19T05:13:24.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.801+0000 7fbaa9485740 10 client.27290 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:13:24.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.801+0000 7fbaa9485740 10 client.27290 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T05:13:24.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.801+0000 7fbaa9485740 20 client.27290 populate_metadata read hostname 'smithi043' 2024-09-19T05:13:24.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.801+0000 7fbaa9485740 1 --2- 172.21.15.43:0/1080173621 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55738026f340 0x557380271760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:24.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.801+0000 7fbaa9485740 4 client.27290 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:13:24.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.801+0000 7fbaa9485740 1 -- 172.21.15.43:0/1080173621 --> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557380156e60 con 0x55738026f340 2024-09-19T05:13:24.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.801+0000 7fbaa4a00640 1 --2- 172.21.15.43:0/1080173621 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55738026f340 0x557380271760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:24.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.802+0000 7fbaa4a00640 1 --2- 172.21.15.43:0/1080173621 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55738026f340 0x557380271760 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fee82b94fd6dafad server_cookie=59cc505a022eaab9 in_seq=0 out_seq=0 2024-09-19T05:13:24.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.802+0000 7fba8d600640 10 client.27290 ms_handle_connect on v2:172.21.15.96:6836/3978388286 2024-09-19T05:13:24.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.802+0000 7fba8d600640 1 -- 172.21.15.43:0/1080173621 <== mds.3 v2:172.21.15.96:6836/3978388286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557380156e60 con 0x55738026f340 2024-09-19T05:13:24.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.802+0000 7fba8d600640 10 client.27290 handle_command_reply: tid=0 2024-09-19T05:13:24.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.802+0000 7fbaa9485740 1 client.27290 shutdown 2024-09-19T05:13:24.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.802+0000 7fbaa9485740 1 -- 172.21.15.43:0/1080173621 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573801e9760 msgr2=0x5573801f7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:24.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.802+0000 7fbaa9485740 1 --2- 172.21.15.43:0/1080173621 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573801e9760 0x5573801f7a00 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fba88000c00 tx=0x7fba88005d20 comp rx=0 tx=0).stop 2024-09-19T05:13:24.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.802+0000 7fbaa9485740 1 -- 172.21.15.43:0/1080173621 shutdown_connections 2024-09-19T05:13:24.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.802+0000 7fbaa9485740 1 --2- 172.21.15.43:0/1080173621 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55738026f340 0x557380271760 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:24.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.802+0000 7fbaa9485740 1 --2- 172.21.15.43:0/1080173621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573801f4020 0x5573801f7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:24.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.802+0000 7fbaa9485740 1 --2- 172.21.15.43:0/1080173621 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5573801e9760 0x5573801f7a00 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:24.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.802+0000 7fbaa9485740 1 --2- 172.21.15.43:0/1080173621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573801e8e40 0x5573801fc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:24.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.802+0000 7fba8d600640 0 client.27290 ms_handle_reset on v2:172.21.15.96:6836/3978388286 2024-09-19T05:13:24.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.802+0000 7fbaa9485740 1 -- 172.21.15.43:0/1080173621 >> 172.21.15.43:0/1080173621 conn(0x5573801e53d0 msgr2=0x5573801e3c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:24.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.802+0000 7fbaa9485740 1 -- 172.21.15.43:0/1080173621 shutdown_connections 2024-09-19T05:13:24.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.802+0000 7fbaa9485740 1 -- 172.21.15.43:0/1080173621 wait complete. 2024-09-19T05:13:24.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.802+0000 7fbaa9485740 20 client.27290 trim_cache size 0 max 16384 2024-09-19T05:13:24.810 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:13:24.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.803+0000 7fbaa5400640 1 -- 172.21.15.43:0/857554034 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fba840a33b0 msgr2=0x7fba840a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:24.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.803+0000 7fbaa5400640 1 --2- 172.21.15.43:0/857554034 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fba840a33b0 0x7fba840a5870 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fba900c4d00 tx=0x7fba900f2800 comp rx=0 tx=0).stop 2024-09-19T05:13:24.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.803+0000 7fbaa5400640 1 -- 172.21.15.43:0/857554034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbaa0103e30 msgr2=0x7fbaa019a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:24.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.803+0000 7fbaa5400640 1 --2- 172.21.15.43:0/857554034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbaa0103e30 0x7fbaa019a4a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fba90004770 tx=0x7fba90003c10 comp rx=0 tx=0).stop 2024-09-19T05:13:24.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.803+0000 7fbaa5400640 1 -- 172.21.15.43:0/857554034 shutdown_connections 2024-09-19T05:13:24.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.803+0000 7fbaa5400640 1 --2- 172.21.15.43:0/857554034 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fba840a33b0 0x7fba840a5870 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:24.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.803+0000 7fbaa5400640 1 --2- 172.21.15.43:0/857554034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbaa0108f60 0x7fbaa019ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:24.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.803+0000 7fbaa5400640 1 --2- 172.21.15.43:0/857554034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbaa01047e0 0x7fbaa019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:24.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.803+0000 7fbaa5400640 1 --2- 172.21.15.43:0/857554034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbaa0103e30 0x7fbaa019a4a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:24.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.803+0000 7fbaa5400640 1 -- 172.21.15.43:0/857554034 >> 172.21.15.43:0/857554034 conn(0x7fbaa00ff8d0 msgr2=0x7fbaa0100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:24.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.803+0000 7fbaa5400640 1 -- 172.21.15.43:0/857554034 shutdown_connections 2024-09-19T05:13:24.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:24.803+0000 7fbaa5400640 1 -- 172.21.15.43:0/857554034 wait complete. 2024-09-19T05:13:25.562 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T05:13:25.942 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:13:26.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:25 smithi151 ceph-mon[35425]: pgmap v272: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-19T05:13:26.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:25 smithi096 ceph-mon[35085]: pgmap v272: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-19T05:13:26.231 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:25 smithi043 ceph-mon[30800]: pgmap v272: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-19T05:13:26.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.515+0000 7fa510200640 1 -- 172.21.15.43:0/2575715077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa508106f20 msgr2=0x7fa50810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:26.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.515+0000 7fa510200640 1 --2- 172.21.15.43:0/2575715077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa508106f20 0x7fa50810eaa0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa500009ca0 tx=0x7fa50002a8c0 comp rx=0 tx=0).stop 2024-09-19T05:13:26.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.515+0000 7fa510200640 1 -- 172.21.15.43:0/2575715077 shutdown_connections 2024-09-19T05:13:26.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.515+0000 7fa510200640 1 --2- 172.21.15.43:0/2575715077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa508106f20 0x7fa50810eaa0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:26.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.515+0000 7fa510200640 1 --2- 172.21.15.43:0/2575715077 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5081027a0 0x7fa5081067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:26.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.515+0000 7fa510200640 1 --2- 172.21.15.43:0/2575715077 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa508101df0 0x7fa5081021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:26.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.515+0000 7fa510200640 1 -- 172.21.15.43:0/2575715077 >> 172.21.15.43:0/2575715077 conn(0x7fa5080fd8b0 msgr2=0x7fa5080ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:26.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.515+0000 7fa510200640 1 -- 172.21.15.43:0/2575715077 shutdown_connections 2024-09-19T05:13:26.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.515+0000 7fa510200640 1 -- 172.21.15.43:0/2575715077 wait complete. 2024-09-19T05:13:26.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.516+0000 7fa510200640 1 Processor -- start 2024-09-19T05:13:26.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.517+0000 7fa510200640 1 -- start start 2024-09-19T05:13:26.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.517+0000 7fa510200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa508101df0 0x7fa5080744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:26.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.517+0000 7fa510200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5081027a0 0x7fa508074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:26.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.517+0000 7fa510200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa508106f20 0x7fa508074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:26.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.517+0000 7fa510200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa508111f80 con 0x7fa508106f20 2024-09-19T05:13:26.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.517+0000 7fa510200640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa508111e00 con 0x7fa508101df0 2024-09-19T05:13:26.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.517+0000 7fa510200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa508112100 con 0x7fa5081027a0 2024-09-19T05:13:26.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.517+0000 7fa50d800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa508101df0 0x7fa5080744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:26.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.517+0000 7fa50d800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa508101df0 0x7fa5080744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:54688/0 (socket says 172.21.15.43:54688) 2024-09-19T05:13:26.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.517+0000 7fa50d800640 1 -- 172.21.15.43:0/799666769 learned_addr learned my addr 172.21.15.43:0/799666769 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:26.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.517+0000 7fa50ce00640 1 --2- 172.21.15.43:0/799666769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5081027a0 0x7fa508074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:26.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.517+0000 7fa50e200640 1 --2- 172.21.15.43:0/799666769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa508106f20 0x7fa508074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:26.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.518+0000 7fa50d800640 1 -- 172.21.15.43:0/799666769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5081027a0 msgr2=0x7fa508074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:26.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.518+0000 7fa50d800640 1 --2- 172.21.15.43:0/799666769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5081027a0 0x7fa508074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:26.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.518+0000 7fa50d800640 1 -- 172.21.15.43:0/799666769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa508106f20 msgr2=0x7fa508074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:26.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.518+0000 7fa50d800640 1 --2- 172.21.15.43:0/799666769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa508106f20 0x7fa508074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:26.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.518+0000 7fa50d800640 1 -- 172.21.15.43:0/799666769 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5081a6840 con 0x7fa508101df0 2024-09-19T05:13:26.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.518+0000 7fa50d800640 1 --2- 172.21.15.43:0/799666769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa508101df0 0x7fa5080744f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc00e9f0 tx=0x7fa4fc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:26.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.518+0000 7fa506000640 1 -- 172.21.15.43:0/799666769 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc00cd80 con 0x7fa508101df0 2024-09-19T05:13:26.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.518+0000 7fa510200640 1 -- 172.21.15.43:0/799666769 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5081a6b30 con 0x7fa508101df0 2024-09-19T05:13:26.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.518+0000 7fa506000640 1 -- 172.21.15.43:0/799666769 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4fc010910 con 0x7fa508101df0 2024-09-19T05:13:26.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.518+0000 7fa506000640 1 -- 172.21.15.43:0/799666769 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc0188f0 con 0x7fa508101df0 2024-09-19T05:13:26.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.519+0000 7fa510200640 1 -- 172.21.15.43:0/799666769 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5081a7070 con 0x7fa508101df0 2024-09-19T05:13:26.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.522+0000 7fa512c12740 1 Processor -- start 2024-09-19T05:13:26.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.522+0000 7fa512c12740 1 -- start start 2024-09-19T05:13:26.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.522+0000 7fa506000640 1 -- 172.21.15.43:0/799666769 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa4fc010ac0 con 0x7fa508101df0 2024-09-19T05:13:26.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.522+0000 7fa512c12740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c5d97fc70 0x556c5d980050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:26.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.522+0000 7fa512c12740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c5d980590 0x556c5d98a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:26.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.522+0000 7fa512c12740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c5d98ae50 0x556c5d98d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:26.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.522+0000 7fa512c12740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556c5d8e63e0 con 0x556c5d980590 2024-09-19T05:13:26.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.522+0000 7fa512c12740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556c5d88dd80 con 0x556c5d97fc70 2024-09-19T05:13:26.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.522+0000 7fa512c12740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556c5d8e23a0 con 0x556c5d98ae50 2024-09-19T05:13:26.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.523+0000 7fa50e200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c5d97fc70 0x556c5d980050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:26.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.523+0000 7fa50d800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c5d980590 0x556c5d98a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:26.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.523+0000 7fa50e200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c5d97fc70 0x556c5d980050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:54694/0 (socket says 172.21.15.43:54694) 2024-09-19T05:13:26.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.523+0000 7fa50e200640 1 -- 172.21.15.43:0/2636343774 learned_addr learned my addr 172.21.15.43:0/2636343774 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:26.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.523+0000 7fa50ce00640 1 --2- 172.21.15.43:0/2636343774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c5d98ae50 0x556c5d98d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.523+0000 7fa50e200640 1 -- 172.21.15.43:0/2636343774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c5d98ae50 msgr2=0x556c5d98d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.523+0000 7fa50e200640 1 --2- 172.21.15.43:0/2636343774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c5d98ae50 0x556c5d98d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.523+0000 7fa50e200640 1 -- 172.21.15.43:0/2636343774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c5d980590 msgr2=0x556c5d98a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.523+0000 7fa50e200640 1 --2- 172.21.15.43:0/2636343774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c5d980590 0x556c5d98a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.523+0000 7fa50e200640 1 -- 172.21.15.43:0/2636343774 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c5d8acae0 con 0x556c5d97fc70 2024-09-19T05:13:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.523+0000 7fa50ce00640 1 --2- 172.21.15.43:0/2636343774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c5d98ae50 0x556c5d98d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:13:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.523+0000 7fa50e200640 1 --2- 172.21.15.43:0/2636343774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c5d97fc70 0x556c5d980050 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa500009ca0 tx=0x7fa500009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a4ac30e9b6dce6e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.524+0000 7fa4faa00640 1 -- 172.21.15.43:0/2636343774 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa500031db0 con 0x556c5d97fc70 2024-09-19T05:13:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.524+0000 7fa506000640 1 --2- 172.21.15.43:0/799666769 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fa4f00a33b0 0x7fa4f00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.524+0000 7fa4faa00640 1 -- 172.21.15.43:0/2636343774 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa500036040 con 0x556c5d97fc70 2024-09-19T05:13:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.524+0000 7fa4faa00640 1 -- 172.21.15.43:0/2636343774 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5000043c0 con 0x556c5d97fc70 2024-09-19T05:13:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.524+0000 7fa506000640 1 -- 172.21.15.43:0/799666769 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7fa4fc0d0140 con 0x7fa508101df0 2024-09-19T05:13:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.524+0000 7fa50d800640 1 --2- 172.21.15.43:0/799666769 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fa4f00a33b0 0x7fa4f00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:26.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.524+0000 7fa512c12740 1 -- 172.21.15.43:0/2636343774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c5d97fc70 msgr2=0x556c5d980050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:26.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.524+0000 7fa512c12740 1 --2- 172.21.15.43:0/2636343774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c5d97fc70 0x556c5d980050 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa500009ca0 tx=0x7fa500009800 comp rx=0 tx=0).stop 2024-09-19T05:13:26.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.525+0000 7fa512c12740 1 -- 172.21.15.43:0/2636343774 shutdown_connections 2024-09-19T05:13:26.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.525+0000 7fa512c12740 1 --2- 172.21.15.43:0/2636343774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c5d98ae50 0x556c5d98d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:26.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.525+0000 7fa512c12740 1 --2- 172.21.15.43:0/2636343774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c5d980590 0x556c5d98a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:26.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.525+0000 7fa512c12740 1 --2- 172.21.15.43:0/2636343774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c5d97fc70 0x556c5d980050 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:26.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.525+0000 7fa512c12740 1 -- 172.21.15.43:0/2636343774 >> 172.21.15.43:0/2636343774 conn(0x556c5d97c200 msgr2=0x556c5d97c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:26.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.525+0000 7fa512c12740 1 -- 172.21.15.43:0/2636343774 shutdown_connections 2024-09-19T05:13:26.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.525+0000 7fa512c12740 1 -- 172.21.15.43:0/2636343774 wait complete. 2024-09-19T05:13:26.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.525+0000 7fa50d800640 1 --2- 172.21.15.43:0/799666769 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fa4f00a33b0 0x7fa4f00a5870 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x556c5d97d060 tx=0x7fa4fc0d3bf0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:26.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.525+0000 7fa512c12740 1 -- start start 2024-09-19T05:13:26.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.525+0000 7fa512c12740 1 Processor -- start 2024-09-19T05:13:26.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.526+0000 7fa512c12740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c5d97fc70 0x556c5d987d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:26.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.526+0000 7fa512c12740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c5d980590 0x556c5d988280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:26.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.526+0000 7fa512c12740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c5d98ae50 0x556c5d9887c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:26.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.526+0000 7fa512c12740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556c5d98d8a0 con 0x556c5d980590 2024-09-19T05:13:26.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.526+0000 7fa512c12740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556c5d8a10f0 con 0x556c5d98ae50 2024-09-19T05:13:26.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.526+0000 7fa512c12740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556c5d98da20 con 0x556c5d97fc70 2024-09-19T05:13:26.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.526+0000 7fa512c12740 10 client.0 _ll_register_callbacks cb 0x556c5d888bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:26.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.526+0000 7fa50ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c5d980590 0x556c5d988280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:26.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.526+0000 7fa50ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c5d980590 0x556c5d988280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33028/0 (socket says 172.21.15.43:33028) 2024-09-19T05:13:26.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.526+0000 7fa50ce00640 1 -- 172.21.15.43:0/1509414714 learned_addr learned my addr 172.21.15.43:0/1509414714 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:26.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.526+0000 7fa50e200640 1 --2- 172.21.15.43:0/1509414714 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c5d97fc70 0x556c5d987d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:26.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.526+0000 7fa50e200640 1 --2- 172.21.15.43:0/1509414714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c5d98ae50 0x556c5d9887c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:26.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.527+0000 7fa50ce00640 1 -- 172.21.15.43:0/1509414714 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c5d97fc70 msgr2=0x556c5d987d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:26.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.527+0000 7fa50ce00640 1 --2- 172.21.15.43:0/1509414714 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c5d97fc70 0x556c5d987d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:26.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.527+0000 7fa50ce00640 1 -- 172.21.15.43:0/1509414714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c5d98ae50 msgr2=0x556c5d9887c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:26.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.527+0000 7fa50ce00640 1 --2- 172.21.15.43:0/1509414714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c5d98ae50 0x556c5d9887c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:26.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.527+0000 7fa50ce00640 1 -- 172.21.15.43:0/1509414714 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c5d6573c0 con 0x556c5d980590 2024-09-19T05:13:26.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.527+0000 7fa50ce00640 1 --2- 172.21.15.43:0/1509414714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c5d980590 0x556c5d988280 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa4f400bf00 tx=0x7fa4f4012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8110fcc30043294d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:26.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.527+0000 7fa4ef400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:13:26.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.527+0000 7fa4ef400640 1 -- 172.21.15.43:0/1509414714 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f40109d0 con 0x556c5d980590 2024-09-19T05:13:26.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.527+0000 7fa512c12740 1 -- 172.21.15.43:0/1509414714 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa500036040 con 0x556c5d980590 2024-09-19T05:13:26.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.527+0000 7fa4ef400640 1 -- 172.21.15.43:0/1509414714 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7fa4f4010b70 con 0x556c5d980590 2024-09-19T05:13:26.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.528+0000 7fa4ef400640 1 -- 172.21.15.43:0/1509414714 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa4f4010d70 con 0x556c5d980590 2024-09-19T05:13:26.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.528+0000 7fa512c12740 10 client.17280 fetch_fsmap learned FSMap version 29 2024-09-19T05:13:26.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.528+0000 7fa512c12740 1 -- 172.21.15.43:0/1509414714 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556c5d999990 con 0x556c5d980590 2024-09-19T05:13:26.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.528+0000 7fa4ef400640 1 -- 172.21.15.43:0/1509414714 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa500036040 con 0x556c5d980590 2024-09-19T05:13:26.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.528+0000 7fa4ef400640 1 -- 172.21.15.43:0/1509414714 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa4f401c510 con 0x556c5d980590 2024-09-19T05:13:26.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.528+0000 7fa512c12740 10 client.17280 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:13:26.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.528+0000 7fa512c12740 10 client.17280 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T05:13:26.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.528+0000 7fa512c12740 20 client.17280 populate_metadata read hostname 'smithi043' 2024-09-19T05:13:26.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.528+0000 7fa512c12740 1 --2- 172.21.15.43:0/1509414714 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x556c5d99aef0 0x556c5d99d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:26.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.528+0000 7fa512c12740 4 client.17280 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:13:26.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.528+0000 7fa512c12740 1 -- 172.21.15.43:0/1509414714 --> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c5d8f3950 con 0x556c5d99aef0 2024-09-19T05:13:26.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.529+0000 7fa50e200640 1 --2- 172.21.15.43:0/1509414714 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x556c5d99aef0 0x556c5d99d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:26.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.529+0000 7fa50e200640 1 --2- 172.21.15.43:0/1509414714 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x556c5d99aef0 0x556c5d99d310 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=69f03a808d625f8e server_cookie=be4ae1eb5343a0f5 in_seq=0 out_seq=0 2024-09-19T05:13:26.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.529+0000 7fa4ef400640 10 client.17280 ms_handle_connect on v2:172.21.15.151:6838/3794104313 2024-09-19T05:13:26.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.530+0000 7fa4ef400640 1 -- 172.21.15.43:0/1509414714 <== mds.2 v2:172.21.15.151:6838/3794104313 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556c5d8f3950 con 0x556c5d99aef0 2024-09-19T05:13:26.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.530+0000 7fa4ef400640 10 client.17280 handle_command_reply: tid=0 2024-09-19T05:13:26.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.530+0000 7fa512c12740 1 client.17280 shutdown 2024-09-19T05:13:26.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.531+0000 7fa512c12740 1 -- 172.21.15.43:0/1509414714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c5d980590 msgr2=0x556c5d988280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:26.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.531+0000 7fa512c12740 1 --2- 172.21.15.43:0/1509414714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c5d980590 0x556c5d988280 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa4f400bf00 tx=0x7fa4f4012820 comp rx=0 tx=0).stop 2024-09-19T05:13:26.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.531+0000 7fa512c12740 1 -- 172.21.15.43:0/1509414714 shutdown_connections 2024-09-19T05:13:26.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.531+0000 7fa512c12740 1 --2- 172.21.15.43:0/1509414714 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x556c5d99aef0 0x556c5d99d310 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:26.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.531+0000 7fa512c12740 1 --2- 172.21.15.43:0/1509414714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c5d98ae50 0x556c5d9887c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:26.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.531+0000 7fa512c12740 1 --2- 172.21.15.43:0/1509414714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c5d980590 0x556c5d988280 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:26.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.531+0000 7fa512c12740 1 --2- 172.21.15.43:0/1509414714 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c5d97fc70 0x556c5d987d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:26.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.531+0000 7fa4ef400640 0 client.17280 ms_handle_reset on v2:172.21.15.151:6838/3794104313 2024-09-19T05:13:26.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.531+0000 7fa512c12740 1 -- 172.21.15.43:0/1509414714 >> 172.21.15.43:0/1509414714 conn(0x556c5d97c200 msgr2=0x556c5d981f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:26.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.531+0000 7fa512c12740 1 -- 172.21.15.43:0/1509414714 shutdown_connections 2024-09-19T05:13:26.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.531+0000 7fa512c12740 1 -- 172.21.15.43:0/1509414714 wait complete. 2024-09-19T05:13:26.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.531+0000 7fa512c12740 20 client.17280 trim_cache size 0 max 16384 2024-09-19T05:13:26.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.545+0000 7fa512c12740 1 Processor -- start 2024-09-19T05:13:26.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.545+0000 7fa512c12740 1 -- start start 2024-09-19T05:13:26.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.545+0000 7fa512c12740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c5d97fc70 0x556c5d986610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:26.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.545+0000 7fa512c12740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c5d980590 0x556c5d988610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:26.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.545+0000 7fa512c12740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c5d98ae50 0x556c5d9a2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:26.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.545+0000 7fa512c12740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa5081afa10 con 0x556c5d98ae50 2024-09-19T05:13:26.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.545+0000 7fa512c12740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa5081b1550 con 0x556c5d980590 2024-09-19T05:13:26.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.545+0000 7fa512c12740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa5081b00e0 con 0x556c5d97fc70 2024-09-19T05:13:26.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.545+0000 7fa50e200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c5d97fc70 0x556c5d986610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:26.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.545+0000 7fa50e200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c5d97fc70 0x556c5d986610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:42122/0 (socket says 172.21.15.43:42122) 2024-09-19T05:13:26.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.545+0000 7fa50e200640 1 -- 172.21.15.43:0/18590074 learned_addr learned my addr 172.21.15.43:0/18590074 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:26.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.545+0000 7fa50e200640 1 --2- 172.21.15.43:0/18590074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c5d98ae50 0x556c5d9a2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:26.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.545+0000 7fa50ce00640 1 --2- 172.21.15.43:0/18590074 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c5d980590 0x556c5d988610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:26.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.546+0000 7fa50e200640 1 -- 172.21.15.43:0/18590074 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c5d980590 msgr2=0x556c5d988610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:26.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.546+0000 7fa50e200640 1 --2- 172.21.15.43:0/18590074 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c5d980590 0x556c5d988610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:26.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.546+0000 7fa50e200640 1 -- 172.21.15.43:0/18590074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c5d98ae50 msgr2=0x556c5d9a2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:26.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.546+0000 7fa50e200640 1 --2- 172.21.15.43:0/18590074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c5d98ae50 0x556c5d9a2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:26.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.546+0000 7fa50e200640 1 -- 172.21.15.43:0/18590074 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c5d9803c0 con 0x556c5d97fc70 2024-09-19T05:13:26.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.546+0000 7fa50e200640 1 --2- 172.21.15.43:0/18590074 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c5d97fc70 0x556c5d986610 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa500009a60 tx=0x7fa50002a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c432fbeecdce5a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:26.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.546+0000 7fa4fa000640 1 -- 172.21.15.43:0/18590074 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa500031830 con 0x556c5d97fc70 2024-09-19T05:13:26.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.546+0000 7fa4fa000640 1 -- 172.21.15.43:0/18590074 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa500036070 con 0x556c5d97fc70 2024-09-19T05:13:26.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.546+0000 7fa4fa000640 1 -- 172.21.15.43:0/18590074 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa500030820 con 0x556c5d97fc70 2024-09-19T05:13:26.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.547+0000 7fa512c12740 1 -- 172.21.15.43:0/18590074 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c5d97fc70 msgr2=0x556c5d986610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:26.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.547+0000 7fa512c12740 1 --2- 172.21.15.43:0/18590074 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c5d97fc70 0x556c5d986610 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa500009a60 tx=0x7fa50002a8c0 comp rx=0 tx=0).stop 2024-09-19T05:13:26.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.547+0000 7fa512c12740 1 -- 172.21.15.43:0/18590074 shutdown_connections 2024-09-19T05:13:26.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.547+0000 7fa512c12740 1 --2- 172.21.15.43:0/18590074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c5d98ae50 0x556c5d9a2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:26.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.547+0000 7fa512c12740 1 --2- 172.21.15.43:0/18590074 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c5d980590 0x556c5d988610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:26.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.547+0000 7fa512c12740 1 --2- 172.21.15.43:0/18590074 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c5d97fc70 0x556c5d986610 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:26.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.547+0000 7fa512c12740 1 -- 172.21.15.43:0/18590074 >> 172.21.15.43:0/18590074 conn(0x556c5d97c200 msgr2=0x556c5d989dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:26.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.547+0000 7fa512c12740 1 -- 172.21.15.43:0/18590074 shutdown_connections 2024-09-19T05:13:26.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.547+0000 7fa512c12740 1 -- 172.21.15.43:0/18590074 wait complete. 2024-09-19T05:13:26.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.548+0000 7fa512c12740 1 -- start start 2024-09-19T05:13:26.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.548+0000 7fa512c12740 1 Processor -- start 2024-09-19T05:13:26.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.548+0000 7fa512c12740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c5d97fc70 0x556c5d993160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:26.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.548+0000 7fa512c12740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c5d980590 0x556c5d9848b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:26.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.548+0000 7fa512c12740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c5d98ae50 0x556c5d984df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:26.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.548+0000 7fa512c12740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa5081b0e80 con 0x556c5d980590 2024-09-19T05:13:26.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.548+0000 7fa512c12740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa5081b07b0 con 0x556c5d98ae50 2024-09-19T05:13:26.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.548+0000 7fa512c12740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa5081ada10 con 0x556c5d97fc70 2024-09-19T05:13:26.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.548+0000 7fa512c12740 10 client.0 _ll_register_callbacks cb 0x7fa4f400ba60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:26.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.549+0000 7fa50e200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c5d97fc70 0x556c5d993160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:26.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.549+0000 7fa50e200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c5d97fc70 0x556c5d993160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:42126/0 (socket says 172.21.15.43:42126) 2024-09-19T05:13:26.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.549+0000 7fa50e200640 1 -- 172.21.15.43:0/2311493205 learned_addr learned my addr 172.21.15.43:0/2311493205 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:26.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.549+0000 7fa50ce00640 1 --2- 172.21.15.43:0/2311493205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c5d980590 0x556c5d9848b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:26.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.549+0000 7fa50e200640 1 --2- 172.21.15.43:0/2311493205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c5d98ae50 0x556c5d984df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:26.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.549+0000 7fa50ce00640 1 -- 172.21.15.43:0/2311493205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c5d97fc70 msgr2=0x556c5d993160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:26.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.549+0000 7fa50ce00640 1 --2- 172.21.15.43:0/2311493205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c5d97fc70 0x556c5d993160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:26.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.549+0000 7fa50ce00640 1 -- 172.21.15.43:0/2311493205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c5d98ae50 msgr2=0x556c5d984df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:26.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.549+0000 7fa50ce00640 1 --2- 172.21.15.43:0/2311493205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c5d98ae50 0x556c5d984df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:26.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.549+0000 7fa50ce00640 1 -- 172.21.15.43:0/2311493205 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c5d9853e0 con 0x556c5d980590 2024-09-19T05:13:26.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.549+0000 7fa50e200640 1 --2- 172.21.15.43:0/2311493205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c5d98ae50 0x556c5d984df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:13:26.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.550+0000 7fa50ce00640 1 --2- 172.21.15.43:0/2311493205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c5d980590 0x556c5d9848b0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa4f401c8b0 tx=0x7fa4f401dd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf2b93012edb8de server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:26.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.550+0000 7fa4ef400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:13:26.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.550+0000 7fa4ef400640 1 -- 172.21.15.43:0/2311493205 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f4012d20 con 0x556c5d980590 2024-09-19T05:13:26.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.550+0000 7fa4ef400640 1 -- 172.21.15.43:0/2311493205 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7fa4f4005040 con 0x556c5d980590 2024-09-19T05:13:26.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.550+0000 7fa512c12740 1 -- 172.21.15.43:0/2311493205 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa500036070 con 0x556c5d980590 2024-09-19T05:13:26.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.550+0000 7fa4ef400640 1 -- 172.21.15.43:0/2311493205 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa4f401cad0 con 0x556c5d980590 2024-09-19T05:13:26.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.550+0000 7fa512c12740 10 client.17286 fetch_fsmap learned FSMap version 29 2024-09-19T05:13:26.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.550+0000 7fa512c12740 1 -- 172.21.15.43:0/2311493205 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556c5da059b0 con 0x556c5d980590 2024-09-19T05:13:26.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.550+0000 7fa4ef400640 1 -- 172.21.15.43:0/2311493205 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa500036070 con 0x556c5d980590 2024-09-19T05:13:26.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.551+0000 7fa4ef400640 1 -- 172.21.15.43:0/2311493205 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa4f4004660 con 0x556c5d980590 2024-09-19T05:13:26.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.551+0000 7fa512c12740 10 client.17286 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:13:26.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.551+0000 7fa512c12740 10 client.17286 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T05:13:26.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.551+0000 7fa512c12740 20 client.17286 populate_metadata read hostname 'smithi043' 2024-09-19T05:13:26.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.551+0000 7fa512c12740 1 --2- 172.21.15.43:0/2311493205 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x556c5da06150 0x556c5da08570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:26.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.551+0000 7fa512c12740 4 client.17286 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:13:26.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.551+0000 7fa512c12740 1 -- 172.21.15.43:0/2311493205 --> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556c5d8ede60 con 0x556c5da06150 2024-09-19T05:13:26.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.551+0000 7fa50e200640 1 --2- 172.21.15.43:0/2311493205 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x556c5da06150 0x556c5da08570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:26.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.552+0000 7fa50e200640 1 --2- 172.21.15.43:0/2311493205 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x556c5da06150 0x556c5da08570 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=37415285e61d28b6 server_cookie=3f44277ca9e92554 in_seq=0 out_seq=0 2024-09-19T05:13:26.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.552+0000 7fa4ef400640 10 client.17286 ms_handle_connect on v2:172.21.15.151:6838/3794104313 2024-09-19T05:13:26.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.552+0000 7fa4ef400640 1 -- 172.21.15.43:0/2311493205 <== mds.2 v2:172.21.15.151:6838/3794104313 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556c5d8ede60 con 0x556c5da06150 2024-09-19T05:13:26.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.552+0000 7fa4ef400640 10 client.17286 handle_command_reply: tid=0 2024-09-19T05:13:26.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.552+0000 7fa512c12740 1 client.17286 shutdown 2024-09-19T05:13:26.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.552+0000 7fa512c12740 1 -- 172.21.15.43:0/2311493205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c5d980590 msgr2=0x556c5d9848b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:26.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.552+0000 7fa512c12740 1 --2- 172.21.15.43:0/2311493205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c5d980590 0x556c5d9848b0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa4f401c8b0 tx=0x7fa4f401dd60 comp rx=0 tx=0).stop 2024-09-19T05:13:26.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.552+0000 7fa512c12740 1 -- 172.21.15.43:0/2311493205 shutdown_connections 2024-09-19T05:13:26.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.552+0000 7fa512c12740 1 --2- 172.21.15.43:0/2311493205 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x556c5da06150 0x556c5da08570 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:26.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.552+0000 7fa512c12740 1 --2- 172.21.15.43:0/2311493205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556c5d98ae50 0x556c5d984df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:26.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.552+0000 7fa512c12740 1 --2- 172.21.15.43:0/2311493205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c5d980590 0x556c5d9848b0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:26.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.552+0000 7fa512c12740 1 --2- 172.21.15.43:0/2311493205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556c5d97fc70 0x556c5d993160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:26.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.552+0000 7fa512c12740 1 -- 172.21.15.43:0/2311493205 >> 172.21.15.43:0/2311493205 conn(0x556c5d97c200 msgr2=0x556c5d9895a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:26.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.552+0000 7fa4ef400640 0 client.17286 ms_handle_reset on v2:172.21.15.151:6838/3794104313 2024-09-19T05:13:26.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.552+0000 7fa512c12740 1 -- 172.21.15.43:0/2311493205 shutdown_connections 2024-09-19T05:13:26.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.552+0000 7fa512c12740 1 -- 172.21.15.43:0/2311493205 wait complete. 2024-09-19T05:13:26.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.553+0000 7fa512c12740 20 client.17286 trim_cache size 0 max 16384 2024-09-19T05:13:26.557 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:13:26.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.553+0000 7fa510200640 1 -- 172.21.15.43:0/799666769 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fa4f00a33b0 msgr2=0x7fa4f00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:26.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.553+0000 7fa510200640 1 --2- 172.21.15.43:0/799666769 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fa4f00a33b0 0x7fa4f00a5870 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x556c5d97d060 tx=0x7fa4fc0d3bf0 comp rx=0 tx=0).stop 2024-09-19T05:13:26.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.553+0000 7fa510200640 1 -- 172.21.15.43:0/799666769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa508101df0 msgr2=0x7fa5080744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:26.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.553+0000 7fa510200640 1 --2- 172.21.15.43:0/799666769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa508101df0 0x7fa5080744f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc00e9f0 tx=0x7fa4fc00eeb0 comp rx=0 tx=0).stop 2024-09-19T05:13:26.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.553+0000 7fa510200640 1 -- 172.21.15.43:0/799666769 shutdown_connections 2024-09-19T05:13:26.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.553+0000 7fa510200640 1 --2- 172.21.15.43:0/799666769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa508106f20 0x7fa508074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:26.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.553+0000 7fa510200640 1 --2- 172.21.15.43:0/799666769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5081027a0 0x7fa508074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:26.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.553+0000 7fa510200640 1 --2- 172.21.15.43:0/799666769 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fa4f00a33b0 0x7fa4f00a5870 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:26.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.553+0000 7fa510200640 1 --2- 172.21.15.43:0/799666769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa508101df0 0x7fa5080744f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:26.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.553+0000 7fa510200640 1 -- 172.21.15.43:0/799666769 >> 172.21.15.43:0/799666769 conn(0x7fa5080fd8b0 msgr2=0x7fa5080ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:26.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.554+0000 7fa510200640 1 -- 172.21.15.43:0/799666769 shutdown_connections 2024-09-19T05:13:26.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:26.554+0000 7fa510200640 1 -- 172.21.15.43:0/799666769 wait complete. 2024-09-19T05:13:27.311 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T05:13:27.687 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:13:28.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:27 smithi043 ceph-mon[30800]: pgmap v273: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:13:28.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:27 smithi151 ceph-mon[35425]: pgmap v273: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:13:28.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:27 smithi096 ceph-mon[35085]: pgmap v273: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:13:28.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.258+0000 7efc61200640 1 -- 172.21.15.43:0/1431358582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc5c10e450 msgr2=0x7efc5c110840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:28.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.258+0000 7efc61200640 1 --2- 172.21.15.43:0/1431358582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc5c10e450 0x7efc5c110840 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7efc54009ca0 tx=0x7efc5402a950 comp rx=0 tx=0).stop 2024-09-19T05:13:28.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.258+0000 7efc61200640 1 -- 172.21.15.43:0/1431358582 shutdown_connections 2024-09-19T05:13:28.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.258+0000 7efc61200640 1 --2- 172.21.15.43:0/1431358582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc5c10e450 0x7efc5c110840 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:28.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.258+0000 7efc61200640 1 --2- 172.21.15.43:0/1431358582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc5c074e30 0x7efc5c10df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:28.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.258+0000 7efc61200640 1 --2- 172.21.15.43:0/1431358582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc5c074510 0x7efc5c0748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:28.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.258+0000 7efc61200640 1 -- 172.21.15.43:0/1431358582 >> 172.21.15.43:0/1431358582 conn(0x7efc5c0fd8b0 msgr2=0x7efc5c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:28.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.258+0000 7efc61200640 1 -- 172.21.15.43:0/1431358582 shutdown_connections 2024-09-19T05:13:28.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.258+0000 7efc61200640 1 -- 172.21.15.43:0/1431358582 wait complete. 2024-09-19T05:13:28.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.259+0000 7efc61200640 1 Processor -- start 2024-09-19T05:13:28.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.259+0000 7efc61200640 1 -- start start 2024-09-19T05:13:28.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.260+0000 7efc61200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc5c074510 0x7efc5c19a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:28.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.260+0000 7efc61200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc5c074e30 0x7efc5c19abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:28.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.260+0000 7efc61200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc5c10e450 0x7efc5c19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:28.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.260+0000 7efc61200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efc5c1141c0 con 0x7efc5c10e450 2024-09-19T05:13:28.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.260+0000 7efc61200640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efc5c114040 con 0x7efc5c074510 2024-09-19T05:13:28.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.260+0000 7efc61200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efc5c114340 con 0x7efc5c074e30 2024-09-19T05:13:28.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.260+0000 7efc59e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc5c074510 0x7efc5c19a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:28.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.260+0000 7efc59400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc5c074e30 0x7efc5c19abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:28.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.260+0000 7efc59400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc5c074e30 0x7efc5c19abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:42148/0 (socket says 172.21.15.43:42148) 2024-09-19T05:13:28.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.260+0000 7efc59e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc5c074510 0x7efc5c19a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:54738/0 (socket says 172.21.15.43:54738) 2024-09-19T05:13:28.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.260+0000 7efc59400640 1 -- 172.21.15.43:0/514319405 learned_addr learned my addr 172.21.15.43:0/514319405 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:28.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.260+0000 7efc5a800640 1 --2- 172.21.15.43:0/514319405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc5c10e450 0x7efc5c19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:28.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.261+0000 7efc59e00640 1 -- 172.21.15.43:0/514319405 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc5c074e30 msgr2=0x7efc5c19abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:28.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.261+0000 7efc59e00640 1 --2- 172.21.15.43:0/514319405 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc5c074e30 0x7efc5c19abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:28.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.261+0000 7efc59e00640 1 -- 172.21.15.43:0/514319405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc5c10e450 msgr2=0x7efc5c19ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:28.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.261+0000 7efc59e00640 1 --2- 172.21.15.43:0/514319405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc5c10e450 0x7efc5c19ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:28.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.261+0000 7efc59e00640 1 -- 172.21.15.43:0/514319405 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc5c19f630 con 0x7efc5c074510 2024-09-19T05:13:28.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.261+0000 7efc59e00640 1 --2- 172.21.15.43:0/514319405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc5c074510 0x7efc5c19a680 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7efc4c00e9f0 tx=0x7efc4c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:28.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.261+0000 7efc52a00640 1 -- 172.21.15.43:0/514319405 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4c00ccc0 con 0x7efc5c074510 2024-09-19T05:13:28.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.261+0000 7efc52a00640 1 -- 172.21.15.43:0/514319405 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc4c005020 con 0x7efc5c074510 2024-09-19T05:13:28.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.261+0000 7efc52a00640 1 -- 172.21.15.43:0/514319405 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4c0049e0 con 0x7efc5c074510 2024-09-19T05:13:28.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.261+0000 7efc61200640 1 -- 172.21.15.43:0/514319405 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc5c19f920 con 0x7efc5c074510 2024-09-19T05:13:28.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.261+0000 7efc61200640 1 -- 172.21.15.43:0/514319405 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc5c1a7200 con 0x7efc5c074510 2024-09-19T05:13:28.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.264+0000 7efc63dcf740 1 Processor -- start 2024-09-19T05:13:28.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.264+0000 7efc52a00640 1 -- 172.21.15.43:0/514319405 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7efc4c005560 con 0x7efc5c074510 2024-09-19T05:13:28.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.264+0000 7efc63dcf740 1 -- start start 2024-09-19T05:13:28.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.265+0000 7efc63dcf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1cdd48c70 0x55d1cdd49050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:28.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.265+0000 7efc63dcf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1cdd49590 0x55d1cdd53910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:28.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.265+0000 7efc63dcf740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cdd53e50 0x55d1cdd56240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:28.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.265+0000 7efc63dcf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d1cdcaf3e0 con 0x55d1cdd49590 2024-09-19T05:13:28.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.265+0000 7efc63dcf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d1cdc56d80 con 0x55d1cdd48c70 2024-09-19T05:13:28.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.265+0000 7efc63dcf740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d1cdcab3a0 con 0x55d1cdd53e50 2024-09-19T05:13:28.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.265+0000 7efc59e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1cdd49590 0x55d1cdd53910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:28.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.265+0000 7efc5a800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1cdd48c70 0x55d1cdd49050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:28.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.265+0000 7efc5a800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1cdd48c70 0x55d1cdd49050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:54746/0 (socket says 172.21.15.43:54746) 2024-09-19T05:13:28.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.265+0000 7efc5a800640 1 -- 172.21.15.43:0/3356543299 learned_addr learned my addr 172.21.15.43:0/3356543299 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:28.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.265+0000 7efc59400640 1 --2- 172.21.15.43:0/3356543299 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cdd53e50 0x55d1cdd56240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:28.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.266+0000 7efc5a800640 1 -- 172.21.15.43:0/3356543299 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cdd53e50 msgr2=0x55d1cdd56240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:28.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.266+0000 7efc5a800640 1 --2- 172.21.15.43:0/3356543299 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cdd53e50 0x55d1cdd56240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:28.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.266+0000 7efc5a800640 1 -- 172.21.15.43:0/3356543299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1cdd49590 msgr2=0x55d1cdd53910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:28.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.266+0000 7efc5a800640 1 --2- 172.21.15.43:0/3356543299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1cdd49590 0x55d1cdd53910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:28.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.266+0000 7efc5a800640 1 -- 172.21.15.43:0/3356543299 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1cdc75ae0 con 0x55d1cdd48c70 2024-09-19T05:13:28.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.266+0000 7efc59e00640 1 --2- 172.21.15.43:0/3356543299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1cdd49590 0x55d1cdd53910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:13:28.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.266+0000 7efc52a00640 1 --2- 172.21.15.43:0/514319405 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7efc400a33b0 0x7efc400a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:28.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.266+0000 7efc5a800640 1 --2- 172.21.15.43:0/3356543299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1cdd48c70 0x55d1cdd49050 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7efc5c0754f0 tx=0x7efc54005f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e604c5faeeba4f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:28.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.266+0000 7efc52a00640 1 -- 172.21.15.43:0/514319405 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7efc4c0cfd10 con 0x7efc5c074510 2024-09-19T05:13:28.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.266+0000 7efc59e00640 1 --2- 172.21.15.43:0/514319405 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7efc400a33b0 0x7efc400a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:28.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.266+0000 7efc47400640 1 -- 172.21.15.43:0/3356543299 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc54048070 con 0x55d1cdd48c70 2024-09-19T05:13:28.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.266+0000 7efc47400640 1 -- 172.21.15.43:0/3356543299 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc54003e50 con 0x55d1cdd48c70 2024-09-19T05:13:28.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.266+0000 7efc47400640 1 -- 172.21.15.43:0/3356543299 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc54034040 con 0x55d1cdd48c70 2024-09-19T05:13:28.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.267+0000 7efc63dcf740 1 -- 172.21.15.43:0/3356543299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1cdd48c70 msgr2=0x55d1cdd49050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:28.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.267+0000 7efc59e00640 1 --2- 172.21.15.43:0/514319405 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7efc400a33b0 0x7efc400a5870 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x55d1cdd46060 tx=0x7efc4c0d37c0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:28.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.267+0000 7efc63dcf740 1 --2- 172.21.15.43:0/3356543299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1cdd48c70 0x55d1cdd49050 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7efc5c0754f0 tx=0x7efc54005f80 comp rx=0 tx=0).stop 2024-09-19T05:13:28.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.267+0000 7efc63dcf740 1 -- 172.21.15.43:0/3356543299 shutdown_connections 2024-09-19T05:13:28.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.267+0000 7efc63dcf740 1 --2- 172.21.15.43:0/3356543299 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cdd53e50 0x55d1cdd56240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:28.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.267+0000 7efc63dcf740 1 --2- 172.21.15.43:0/3356543299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1cdd49590 0x55d1cdd53910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:28.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.267+0000 7efc63dcf740 1 --2- 172.21.15.43:0/3356543299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1cdd48c70 0x55d1cdd49050 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:28.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.267+0000 7efc63dcf740 1 -- 172.21.15.43:0/3356543299 >> 172.21.15.43:0/3356543299 conn(0x55d1cdd45200 msgr2=0x55d1cdd455e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:28.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.267+0000 7efc63dcf740 1 -- 172.21.15.43:0/3356543299 shutdown_connections 2024-09-19T05:13:28.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.267+0000 7efc63dcf740 1 -- 172.21.15.43:0/3356543299 wait complete. 2024-09-19T05:13:28.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.268+0000 7efc63dcf740 1 -- start start 2024-09-19T05:13:28.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.268+0000 7efc63dcf740 1 Processor -- start 2024-09-19T05:13:28.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.268+0000 7efc63dcf740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cdd48c70 0x55d1cdd50d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:28.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.268+0000 7efc63dcf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1cdd49590 0x55d1cdd51280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:28.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.268+0000 7efc63dcf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1cdd53e50 0x55d1cdd517c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:28.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.268+0000 7efc63dcf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d1cdd56a20 con 0x55d1cdd53e50 2024-09-19T05:13:28.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.268+0000 7efc63dcf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d1cdc6a0f0 con 0x55d1cdd49590 2024-09-19T05:13:28.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.268+0000 7efc63dcf740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d1cdd56ba0 con 0x55d1cdd48c70 2024-09-19T05:13:28.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.269+0000 7efc63dcf740 10 client.0 _ll_register_callbacks cb 0x55d1cdc51bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:28.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.269+0000 7efc5a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1cdd53e50 0x55d1cdd517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:28.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.269+0000 7efc5a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1cdd53e50 0x55d1cdd517c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33068/0 (socket says 172.21.15.43:33068) 2024-09-19T05:13:28.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.269+0000 7efc5a800640 1 -- 172.21.15.43:0/1365066856 learned_addr learned my addr 172.21.15.43:0/1365066856 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:28.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.269+0000 7efc59400640 1 --2- 172.21.15.43:0/1365066856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1cdd49590 0x55d1cdd51280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:28.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.269+0000 7efc5a800640 1 --2- 172.21.15.43:0/1365066856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cdd48c70 0x55d1cdd50d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:28.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.269+0000 7efc59400640 1 -- 172.21.15.43:0/1365066856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cdd48c70 msgr2=0x55d1cdd50d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:28.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.269+0000 7efc59400640 1 --2- 172.21.15.43:0/1365066856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cdd48c70 0x55d1cdd50d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:28.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.269+0000 7efc59400640 1 -- 172.21.15.43:0/1365066856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1cdd53e50 msgr2=0x55d1cdd517c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:28.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.269+0000 7efc59400640 1 --2- 172.21.15.43:0/1365066856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1cdd53e50 0x55d1cdd517c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:28.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.269+0000 7efc59400640 1 -- 172.21.15.43:0/1365066856 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1cda203c0 con 0x55d1cdd49590 2024-09-19T05:13:28.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.270+0000 7efc59400640 1 --2- 172.21.15.43:0/1365066856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1cdd49590 0x55d1cdd51280 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7efc5c10f700 tx=0x7efc48012aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c244f744727a9d6a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:28.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.270+0000 7efc3be00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:13:28.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.270+0000 7efc3be00640 1 -- 172.21.15.43:0/1365066856 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc48004050 con 0x55d1cdd49590 2024-09-19T05:13:28.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.270+0000 7efc3be00640 1 -- 172.21.15.43:0/1365066856 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7efc480041f0 con 0x55d1cdd49590 2024-09-19T05:13:28.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.270+0000 7efc63dcf740 1 -- 172.21.15.43:0/1365066856 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc54003e50 con 0x55d1cdd49590 2024-09-19T05:13:28.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.270+0000 7efc3be00640 1 -- 172.21.15.43:0/1365066856 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efc480043f0 con 0x55d1cdd49590 2024-09-19T05:13:28.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.270+0000 7efc63dcf740 10 client.27320 fetch_fsmap learned FSMap version 29 2024-09-19T05:13:28.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.270+0000 7efc63dcf740 1 -- 172.21.15.43:0/1365066856 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d1cdd62af0 con 0x55d1cdd49590 2024-09-19T05:13:28.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.271+0000 7efc3be00640 1 -- 172.21.15.43:0/1365066856 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc54003e50 con 0x55d1cdd49590 2024-09-19T05:13:28.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.271+0000 7efc3be00640 1 -- 172.21.15.43:0/1365066856 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7efc480039c0 con 0x55d1cdd49590 2024-09-19T05:13:28.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.271+0000 7efc63dcf740 10 client.27320 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:13:28.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.271+0000 7efc63dcf740 10 client.27320 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-19T05:13:28.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.271+0000 7efc63dcf740 20 client.27320 populate_metadata read hostname 'smithi043' 2024-09-19T05:13:28.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.271+0000 7efc63dcf740 1 --2- 172.21.15.43:0/1365066856 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55d1cdd63fb0 0x55d1cdd663d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:28.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.271+0000 7efc63dcf740 4 client.27320 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:13:28.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.271+0000 7efc63dcf740 1 -- 172.21.15.43:0/1365066856 --> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1cdcbc950 con 0x55d1cdd63fb0 2024-09-19T05:13:28.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.272+0000 7efc5a800640 1 --2- 172.21.15.43:0/1365066856 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55d1cdd63fb0 0x55d1cdd663d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:28.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.272+0000 7efc5a800640 1 --2- 172.21.15.43:0/1365066856 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55d1cdd63fb0 0x55d1cdd663d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d11ab2591afe7fb4 server_cookie=7a507cc38b1de0b8 in_seq=0 out_seq=0 2024-09-19T05:13:28.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.272+0000 7efc3be00640 10 client.27320 ms_handle_connect on v2:172.21.15.43:6838/3404555030 2024-09-19T05:13:28.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.273+0000 7efc3be00640 1 -- 172.21.15.43:0/1365066856 <== mds.1 v2:172.21.15.43:6838/3404555030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d1cdcbc950 con 0x55d1cdd63fb0 2024-09-19T05:13:28.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.273+0000 7efc3be00640 10 client.27320 handle_command_reply: tid=0 2024-09-19T05:13:28.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.273+0000 7efc63dcf740 1 client.27320 shutdown 2024-09-19T05:13:28.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.273+0000 7efc63dcf740 1 -- 172.21.15.43:0/1365066856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1cdd49590 msgr2=0x55d1cdd51280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:28.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.273+0000 7efc63dcf740 1 --2- 172.21.15.43:0/1365066856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1cdd49590 0x55d1cdd51280 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7efc5c10f700 tx=0x7efc48012aa0 comp rx=0 tx=0).stop 2024-09-19T05:13:28.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.273+0000 7efc63dcf740 1 -- 172.21.15.43:0/1365066856 shutdown_connections 2024-09-19T05:13:28.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.273+0000 7efc63dcf740 1 --2- 172.21.15.43:0/1365066856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1cdd53e50 0x55d1cdd517c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:28.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.273+0000 7efc63dcf740 1 --2- 172.21.15.43:0/1365066856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1cdd49590 0x55d1cdd51280 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:28.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.273+0000 7efc63dcf740 1 --2- 172.21.15.43:0/1365066856 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55d1cdd63fb0 0x55d1cdd663d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:28.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.273+0000 7efc63dcf740 1 --2- 172.21.15.43:0/1365066856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cdd48c70 0x55d1cdd50d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:28.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.273+0000 7efc63dcf740 1 -- 172.21.15.43:0/1365066856 >> 172.21.15.43:0/1365066856 conn(0x55d1cdd45200 msgr2=0x55d1cdd44380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:28.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.273+0000 7efc3be00640 0 client.27320 ms_handle_reset on v2:172.21.15.43:6838/3404555030 2024-09-19T05:13:28.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.274+0000 7efc63dcf740 1 -- 172.21.15.43:0/1365066856 shutdown_connections 2024-09-19T05:13:28.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.274+0000 7efc63dcf740 1 -- 172.21.15.43:0/1365066856 wait complete. 2024-09-19T05:13:28.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.274+0000 7efc63dcf740 20 client.27320 trim_cache size 0 max 16384 2024-09-19T05:13:28.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.287+0000 7efc63dcf740 1 Processor -- start 2024-09-19T05:13:28.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.287+0000 7efc63dcf740 1 -- start start 2024-09-19T05:13:28.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.288+0000 7efc63dcf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1cdd48c70 0x55d1cdd6bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:28.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.288+0000 7efc63dcf740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cdd49590 0x55d1cdd6c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:28.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.288+0000 7efc63dcf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1cdd53e50 0x55d1cdd4dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:28.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.288+0000 7efc63dcf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efc5c1afb50 con 0x55d1cdd53e50 2024-09-19T05:13:28.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.288+0000 7efc63dcf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efc5c1b1690 con 0x55d1cdd48c70 2024-09-19T05:13:28.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.288+0000 7efc63dcf740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efc5c1b0220 con 0x55d1cdd49590 2024-09-19T05:13:28.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.288+0000 7efc5a800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1cdd48c70 0x55d1cdd6bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:28.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.288+0000 7efc59400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cdd49590 0x55d1cdd6c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:28.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.288+0000 7efc5a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1cdd53e50 0x55d1cdd4dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:28.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.288+0000 7efc59400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cdd49590 0x55d1cdd6c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:42170/0 (socket says 172.21.15.43:42170) 2024-09-19T05:13:28.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.288+0000 7efc59400640 1 -- 172.21.15.43:0/4224457926 learned_addr learned my addr 172.21.15.43:0/4224457926 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:28.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.288+0000 7efc5a800640 1 -- 172.21.15.43:0/4224457926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cdd49590 msgr2=0x55d1cdd6c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:28.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.288+0000 7efc5a800640 1 --2- 172.21.15.43:0/4224457926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cdd49590 0x55d1cdd6c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:28.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.288+0000 7efc5a800640 1 -- 172.21.15.43:0/4224457926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1cdd53e50 msgr2=0x55d1cdd4dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:28.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.288+0000 7efc5a800640 1 --2- 172.21.15.43:0/4224457926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1cdd53e50 0x55d1cdd4dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:28.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.289+0000 7efc5a800640 1 -- 172.21.15.43:0/4224457926 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1cdd40770 con 0x55d1cdd48c70 2024-09-19T05:13:28.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.289+0000 7efc59400640 1 --2- 172.21.15.43:0/4224457926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cdd49590 0x55d1cdd6c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:13:28.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.289+0000 7efc5a800640 1 --2- 172.21.15.43:0/4224457926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1cdd48c70 0x55d1cdd6bca0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7efc54009620 tx=0x7efc54005d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd80665fbb842b72 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:28.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.289+0000 7efc46a00640 1 -- 172.21.15.43:0/4224457926 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc54048040 con 0x55d1cdd48c70 2024-09-19T05:13:28.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.289+0000 7efc46a00640 1 -- 172.21.15.43:0/4224457926 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc54049040 con 0x55d1cdd48c70 2024-09-19T05:13:28.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.289+0000 7efc46a00640 1 -- 172.21.15.43:0/4224457926 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc540046c0 con 0x55d1cdd48c70 2024-09-19T05:13:28.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.290+0000 7efc63dcf740 1 -- 172.21.15.43:0/4224457926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1cdd48c70 msgr2=0x55d1cdd6bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:28.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.290+0000 7efc63dcf740 1 --2- 172.21.15.43:0/4224457926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1cdd48c70 0x55d1cdd6bca0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7efc54009620 tx=0x7efc54005d80 comp rx=0 tx=0).stop 2024-09-19T05:13:28.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.290+0000 7efc63dcf740 1 -- 172.21.15.43:0/4224457926 shutdown_connections 2024-09-19T05:13:28.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.290+0000 7efc63dcf740 1 --2- 172.21.15.43:0/4224457926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1cdd53e50 0x55d1cdd4dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:28.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.290+0000 7efc63dcf740 1 --2- 172.21.15.43:0/4224457926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cdd49590 0x55d1cdd6c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:28.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.290+0000 7efc63dcf740 1 --2- 172.21.15.43:0/4224457926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1cdd48c70 0x55d1cdd6bca0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:28.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.290+0000 7efc63dcf740 1 -- 172.21.15.43:0/4224457926 >> 172.21.15.43:0/4224457926 conn(0x55d1cdd45200 msgr2=0x55d1cdd52dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:28.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.290+0000 7efc63dcf740 1 -- 172.21.15.43:0/4224457926 shutdown_connections 2024-09-19T05:13:28.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.290+0000 7efc63dcf740 1 -- 172.21.15.43:0/4224457926 wait complete. 2024-09-19T05:13:28.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.290+0000 7efc63dcf740 1 -- start start 2024-09-19T05:13:28.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.290+0000 7efc63dcf740 1 Processor -- start 2024-09-19T05:13:28.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.291+0000 7efc63dcf740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cdd48c70 0x55d1cdd57e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:28.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.291+0000 7efc63dcf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1cdd49590 0x55d1cdd58350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:28.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.291+0000 7efc63dcf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1cdd53e50 0x55d1cdd58890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:28.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.291+0000 7efc63dcf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efc5c1b0fc0 con 0x55d1cdd49590 2024-09-19T05:13:28.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.291+0000 7efc63dcf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efc5c1b08f0 con 0x55d1cdd53e50 2024-09-19T05:13:28.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.291+0000 7efc63dcf740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efc5c1adb50 con 0x55d1cdd48c70 2024-09-19T05:13:28.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.291+0000 7efc63dcf740 10 client.0 _ll_register_callbacks cb 0x7efc48009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:28.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.291+0000 7efc59400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1cdd49590 0x55d1cdd58350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:28.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.291+0000 7efc59400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1cdd49590 0x55d1cdd58350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33080/0 (socket says 172.21.15.43:33080) 2024-09-19T05:13:28.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.291+0000 7efc59400640 1 -- 172.21.15.43:0/16698555 learned_addr learned my addr 172.21.15.43:0/16698555 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:28.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.291+0000 7efc5a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cdd48c70 0x55d1cdd57e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:28.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.291+0000 7efc5a800640 1 --2- 172.21.15.43:0/16698555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1cdd53e50 0x55d1cdd58890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:28.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.292+0000 7efc59400640 1 -- 172.21.15.43:0/16698555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cdd48c70 msgr2=0x55d1cdd57e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:28.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.292+0000 7efc59400640 1 --2- 172.21.15.43:0/16698555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cdd48c70 0x55d1cdd57e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:28.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.292+0000 7efc59400640 1 -- 172.21.15.43:0/16698555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1cdd53e50 msgr2=0x55d1cdd58890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:28.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.292+0000 7efc59400640 1 --2- 172.21.15.43:0/16698555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1cdd53e50 0x55d1cdd58890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:28.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.292+0000 7efc59400640 1 -- 172.21.15.43:0/16698555 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1cdd45e90 con 0x55d1cdd49590 2024-09-19T05:13:28.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.292+0000 7efc59400640 1 --2- 172.21.15.43:0/16698555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1cdd49590 0x55d1cdd58350 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7efc480097f0 tx=0x7efc48003fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b58be30d12f3e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:28.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.292+0000 7efc3be00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:13:28.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.292+0000 7efc3be00640 1 -- 172.21.15.43:0/16698555 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc48028040 con 0x55d1cdd49590 2024-09-19T05:13:28.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.292+0000 7efc3be00640 1 -- 172.21.15.43:0/16698555 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7efc480041c0 con 0x55d1cdd49590 2024-09-19T05:13:28.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.292+0000 7efc63dcf740 1 -- 172.21.15.43:0/16698555 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc54049040 con 0x55d1cdd49590 2024-09-19T05:13:28.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.293+0000 7efc3be00640 1 -- 172.21.15.43:0/16698555 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efc48039050 con 0x55d1cdd49590 2024-09-19T05:13:28.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.293+0000 7efc63dcf740 10 client.17304 fetch_fsmap learned FSMap version 29 2024-09-19T05:13:28.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.293+0000 7efc63dcf740 1 -- 172.21.15.43:0/16698555 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d1cddcf020 con 0x55d1cdd49590 2024-09-19T05:13:28.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.293+0000 7efc3be00640 1 -- 172.21.15.43:0/16698555 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc54049040 con 0x55d1cdd49590 2024-09-19T05:13:28.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.293+0000 7efc3be00640 1 -- 172.21.15.43:0/16698555 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7efc48034030 con 0x55d1cdd49590 2024-09-19T05:13:28.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.293+0000 7efc63dcf740 10 client.17304 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:13:28.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.293+0000 7efc63dcf740 10 client.17304 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-19T05:13:28.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.293+0000 7efc63dcf740 20 client.17304 populate_metadata read hostname 'smithi043' 2024-09-19T05:13:28.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.293+0000 7efc63dcf740 1 --2- 172.21.15.43:0/16698555 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55d1cddcf790 0x55d1cddd1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:28.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.293+0000 7efc63dcf740 4 client.17304 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:13:28.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.293+0000 7efc63dcf740 1 -- 172.21.15.43:0/16698555 --> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d1cdcb6e60 con 0x55d1cddcf790 2024-09-19T05:13:28.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.294+0000 7efc5a800640 1 --2- 172.21.15.43:0/16698555 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55d1cddcf790 0x55d1cddd1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:28.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.294+0000 7efc5a800640 1 --2- 172.21.15.43:0/16698555 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55d1cddcf790 0x55d1cddd1bb0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=81bd6fc370f567fd server_cookie=9004ada48637be52 in_seq=0 out_seq=0 2024-09-19T05:13:28.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.294+0000 7efc3be00640 10 client.17304 ms_handle_connect on v2:172.21.15.43:6838/3404555030 2024-09-19T05:13:28.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.294+0000 7efc3be00640 1 -- 172.21.15.43:0/16698555 <== mds.1 v2:172.21.15.43:6838/3404555030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d1cdcb6e60 con 0x55d1cddcf790 2024-09-19T05:13:28.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.294+0000 7efc3be00640 10 client.17304 handle_command_reply: tid=0 2024-09-19T05:13:28.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.294+0000 7efc63dcf740 1 client.17304 shutdown 2024-09-19T05:13:28.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.295+0000 7efc63dcf740 1 -- 172.21.15.43:0/16698555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1cdd49590 msgr2=0x55d1cdd58350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:28.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.295+0000 7efc63dcf740 1 --2- 172.21.15.43:0/16698555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1cdd49590 0x55d1cdd58350 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7efc480097f0 tx=0x7efc48003fa0 comp rx=0 tx=0).stop 2024-09-19T05:13:28.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.295+0000 7efc63dcf740 1 -- 172.21.15.43:0/16698555 shutdown_connections 2024-09-19T05:13:28.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.295+0000 7efc63dcf740 1 --2- 172.21.15.43:0/16698555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1cdd53e50 0x55d1cdd58890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:28.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.295+0000 7efc63dcf740 1 --2- 172.21.15.43:0/16698555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1cdd49590 0x55d1cdd58350 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:28.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.295+0000 7efc63dcf740 1 --2- 172.21.15.43:0/16698555 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55d1cddcf790 0x55d1cddd1bb0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:28.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.295+0000 7efc63dcf740 1 --2- 172.21.15.43:0/16698555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cdd48c70 0x55d1cdd57e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:28.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.295+0000 7efc63dcf740 1 -- 172.21.15.43:0/16698555 >> 172.21.15.43:0/16698555 conn(0x55d1cdd45200 msgr2=0x55d1cdd43b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:28.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.295+0000 7efc3be00640 0 client.17304 ms_handle_reset on v2:172.21.15.43:6838/3404555030 2024-09-19T05:13:28.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.295+0000 7efc63dcf740 1 -- 172.21.15.43:0/16698555 shutdown_connections 2024-09-19T05:13:28.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.295+0000 7efc63dcf740 1 -- 172.21.15.43:0/16698555 wait complete. 2024-09-19T05:13:28.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.295+0000 7efc63dcf740 20 client.17304 trim_cache size 0 max 16384 2024-09-19T05:13:28.302 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:13:28.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.296+0000 7efc61200640 1 -- 172.21.15.43:0/514319405 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7efc400a33b0 msgr2=0x7efc400a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:28.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.296+0000 7efc61200640 1 --2- 172.21.15.43:0/514319405 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7efc400a33b0 0x7efc400a5870 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x55d1cdd46060 tx=0x7efc4c0d37c0 comp rx=0 tx=0).stop 2024-09-19T05:13:28.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.296+0000 7efc61200640 1 -- 172.21.15.43:0/514319405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc5c074510 msgr2=0x7efc5c19a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:28.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.296+0000 7efc61200640 1 --2- 172.21.15.43:0/514319405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc5c074510 0x7efc5c19a680 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7efc4c00e9f0 tx=0x7efc4c00eeb0 comp rx=0 tx=0).stop 2024-09-19T05:13:28.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.296+0000 7efc61200640 1 -- 172.21.15.43:0/514319405 shutdown_connections 2024-09-19T05:13:28.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.296+0000 7efc61200640 1 --2- 172.21.15.43:0/514319405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc5c10e450 0x7efc5c19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:28.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.296+0000 7efc61200640 1 --2- 172.21.15.43:0/514319405 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc5c074e30 0x7efc5c19abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:28.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.296+0000 7efc61200640 1 --2- 172.21.15.43:0/514319405 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7efc400a33b0 0x7efc400a5870 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:28.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.296+0000 7efc61200640 1 --2- 172.21.15.43:0/514319405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc5c074510 0x7efc5c19a680 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:28.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.296+0000 7efc61200640 1 -- 172.21.15.43:0/514319405 >> 172.21.15.43:0/514319405 conn(0x7efc5c0fd8b0 msgr2=0x7efc5c0fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:28.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.296+0000 7efc61200640 1 -- 172.21.15.43:0/514319405 shutdown_connections 2024-09-19T05:13:28.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:28.296+0000 7efc61200640 1 -- 172.21.15.43:0/514319405 wait complete. 2024-09-19T05:13:29.289 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T05:13:29.667 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:13:30.138 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:29 smithi043 ceph-mon[30800]: pgmap v274: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:13:30.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:29 smithi151 ceph-mon[35425]: pgmap v274: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:13:30.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:29 smithi096 ceph-mon[35085]: pgmap v274: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:13:30.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.252+0000 7f254b600640 1 -- 172.21.15.43:0/2418412716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2544112b90 msgr2=0x7f2544114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:30.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.252+0000 7f254b600640 1 --2- 172.21.15.43:0/2418412716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2544112b90 0x7f2544114f80 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f253c009ca0 tx=0x7f253c02a950 comp rx=0 tx=0).stop 2024-09-19T05:13:30.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.252+0000 7f254b600640 1 -- 172.21.15.43:0/2418412716 shutdown_connections 2024-09-19T05:13:30.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.252+0000 7f254b600640 1 --2- 172.21.15.43:0/2418412716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2544112b90 0x7f2544114f80 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:30.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.252+0000 7f254b600640 1 --2- 172.21.15.43:0/2418412716 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2544074eb0 0x7f2544112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:30.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.252+0000 7f254b600640 1 --2- 172.21.15.43:0/2418412716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2544074590 0x7f2544074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:30.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.252+0000 7f254b600640 1 -- 172.21.15.43:0/2418412716 >> 172.21.15.43:0/2418412716 conn(0x7f25440fd8c0 msgr2=0x7f25440ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:30.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.252+0000 7f254b600640 1 -- 172.21.15.43:0/2418412716 shutdown_connections 2024-09-19T05:13:30.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.252+0000 7f254b600640 1 -- 172.21.15.43:0/2418412716 wait complete. 2024-09-19T05:13:30.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.253+0000 7f254b600640 1 Processor -- start 2024-09-19T05:13:30.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.253+0000 7f254b600640 1 -- start start 2024-09-19T05:13:30.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.254+0000 7f254b600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2544074590 0x7f25441a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:30.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.254+0000 7f254b600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2544074eb0 0x7f25441a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:30.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.254+0000 7f254b600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2544112b90 0x7f254419e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:30.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.254+0000 7f254b600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2544118580 con 0x7f2544112b90 2024-09-19T05:13:30.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.254+0000 7f254b600640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2544118400 con 0x7f2544074eb0 2024-09-19T05:13:30.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.254+0000 7f254b600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2544118700 con 0x7f2544074590 2024-09-19T05:13:30.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.254+0000 7f2548c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2544074590 0x7f25441a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:30.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.254+0000 7f2549600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2544112b90 0x7f254419e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:30.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.254+0000 7f2548c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2544074590 0x7f25441a4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:42186/0 (socket says 172.21.15.43:42186) 2024-09-19T05:13:30.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.254+0000 7f2548c00640 1 -- 172.21.15.43:0/348078862 learned_addr learned my addr 172.21.15.43:0/348078862 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:30.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.254+0000 7f2543e00640 1 --2- 172.21.15.43:0/348078862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2544074eb0 0x7f25441a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:30.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.255+0000 7f2548c00640 1 -- 172.21.15.43:0/348078862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2544074eb0 msgr2=0x7f25441a4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:30.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.255+0000 7f2548c00640 1 --2- 172.21.15.43:0/348078862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2544074eb0 0x7f25441a4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:30.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.255+0000 7f2548c00640 1 -- 172.21.15.43:0/348078862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2544112b90 msgr2=0x7f254419e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:30.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.255+0000 7f2548c00640 1 --2- 172.21.15.43:0/348078862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2544112b90 0x7f254419e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:30.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.255+0000 7f2548c00640 1 -- 172.21.15.43:0/348078862 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f254419f010 con 0x7f2544074590 2024-09-19T05:13:30.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.255+0000 7f2543e00640 1 --2- 172.21.15.43:0/348078862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2544074eb0 0x7f25441a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:13:30.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.255+0000 7f2548c00640 1 --2- 172.21.15.43:0/348078862 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2544074590 0x7f25441a4720 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f253800e9f0 tx=0x7f253800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:30.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.255+0000 7f2541600640 1 -- 172.21.15.43:0/348078862 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253800cd80 con 0x7f2544074590 2024-09-19T05:13:30.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.255+0000 7f254b600640 1 -- 172.21.15.43:0/348078862 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f254419f300 con 0x7f2544074590 2024-09-19T05:13:30.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.255+0000 7f2541600640 1 -- 172.21.15.43:0/348078862 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2538005500 con 0x7f2544074590 2024-09-19T05:13:30.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.255+0000 7f2541600640 1 -- 172.21.15.43:0/348078862 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2538010690 con 0x7f2544074590 2024-09-19T05:13:30.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.255+0000 7f254b600640 1 -- 172.21.15.43:0/348078862 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25441ab5f0 con 0x7f2544074590 2024-09-19T05:13:30.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.258+0000 7f254e1e0740 1 Processor -- start 2024-09-19T05:13:30.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.259+0000 7f2541600640 1 -- 172.21.15.43:0/348078862 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f25380046c0 con 0x7f2544074590 2024-09-19T05:13:30.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.259+0000 7f254e1e0740 1 -- start start 2024-09-19T05:13:30.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.259+0000 7f254e1e0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaed9dcc70 0x55aaed9dd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:30.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.259+0000 7f254e1e0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaed9dd590 0x55aaed9e7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:30.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.259+0000 7f254e1e0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaed9e7e50 0x55aaed9ea240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:30.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.259+0000 7f254e1e0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55aaed9433e0 con 0x55aaed9dcc70 2024-09-19T05:13:30.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.259+0000 7f254e1e0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55aaed8ead80 con 0x55aaed9dd590 2024-09-19T05:13:30.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.259+0000 7f254e1e0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55aaed93f3a0 con 0x55aaed9e7e50 2024-09-19T05:13:30.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.259+0000 7f2549600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaed9dcc70 0x55aaed9dd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:30.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.259+0000 7f2549600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaed9dcc70 0x55aaed9dd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33110/0 (socket says 172.21.15.43:33110) 2024-09-19T05:13:30.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.259+0000 7f2549600640 1 -- 172.21.15.43:0/1129417829 learned_addr learned my addr 172.21.15.43:0/1129417829 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:30.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.259+0000 7f2548c00640 1 --2- 172.21.15.43:0/1129417829 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaed9dd590 0x55aaed9e7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:30.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.259+0000 7f2543e00640 1 --2- 172.21.15.43:0/1129417829 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaed9e7e50 0x55aaed9ea240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:30.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.260+0000 7f2549600640 1 -- 172.21.15.43:0/1129417829 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaed9e7e50 msgr2=0x55aaed9ea240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:30.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.260+0000 7f2549600640 1 --2- 172.21.15.43:0/1129417829 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaed9e7e50 0x55aaed9ea240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:30.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.260+0000 7f2549600640 1 -- 172.21.15.43:0/1129417829 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaed9dd590 msgr2=0x55aaed9e7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:30.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.260+0000 7f2549600640 1 --2- 172.21.15.43:0/1129417829 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaed9dd590 0x55aaed9e7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:30.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.260+0000 7f2549600640 1 -- 172.21.15.43:0/1129417829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaed909ae0 con 0x55aaed9dcc70 2024-09-19T05:13:30.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.260+0000 7f2549600640 1 --2- 172.21.15.43:0/1129417829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaed9dcc70 0x55aaed9dd050 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f2544076600 tx=0x7f253c0093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb9b73d2673793b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:30.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.260+0000 7f2535c00640 1 -- 172.21.15.43:0/1129417829 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c048070 con 0x55aaed9dcc70 2024-09-19T05:13:30.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.261+0000 7f2535c00640 1 -- 172.21.15.43:0/1129417829 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f253c02f940 con 0x55aaed9dcc70 2024-09-19T05:13:30.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.261+0000 7f2535c00640 1 -- 172.21.15.43:0/1129417829 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c034040 con 0x55aaed9dcc70 2024-09-19T05:13:30.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.261+0000 7f2541600640 1 --2- 172.21.15.43:0/348078862 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f252c0a33b0 0x7f252c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:30.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.261+0000 7f254e1e0740 1 -- 172.21.15.43:0/1129417829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaed9dcc70 msgr2=0x55aaed9dd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:30.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.261+0000 7f254e1e0740 1 --2- 172.21.15.43:0/1129417829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaed9dcc70 0x55aaed9dd050 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f2544076600 tx=0x7f253c0093e0 comp rx=0 tx=0).stop 2024-09-19T05:13:30.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.261+0000 7f2541600640 1 -- 172.21.15.43:0/348078862 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f25380d01c0 con 0x7f2544074590 2024-09-19T05:13:30.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.261+0000 7f254e1e0740 1 -- 172.21.15.43:0/1129417829 shutdown_connections 2024-09-19T05:13:30.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.261+0000 7f254e1e0740 1 --2- 172.21.15.43:0/1129417829 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaed9e7e50 0x55aaed9ea240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:30.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.261+0000 7f254e1e0740 1 --2- 172.21.15.43:0/1129417829 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaed9dd590 0x55aaed9e7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:30.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.261+0000 7f254e1e0740 1 --2- 172.21.15.43:0/1129417829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaed9dcc70 0x55aaed9dd050 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:30.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.261+0000 7f254e1e0740 1 -- 172.21.15.43:0/1129417829 >> 172.21.15.43:0/1129417829 conn(0x55aaed9d9200 msgr2=0x55aaed9d95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:30.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.261+0000 7f2548c00640 1 --2- 172.21.15.43:0/348078862 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f252c0a33b0 0x7f252c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:30.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.262+0000 7f254e1e0740 1 -- 172.21.15.43:0/1129417829 shutdown_connections 2024-09-19T05:13:30.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.262+0000 7f254e1e0740 1 -- 172.21.15.43:0/1129417829 wait complete. 2024-09-19T05:13:30.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.262+0000 7f2548c00640 1 --2- 172.21.15.43:0/348078862 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f252c0a33b0 0x7f252c0a5870 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f25380cd9b0 tx=0x7f2538098040 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:30.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.262+0000 7f254e1e0740 1 -- start start 2024-09-19T05:13:30.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.263+0000 7f254e1e0740 1 Processor -- start 2024-09-19T05:13:30.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.263+0000 7f254e1e0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaed9dcc70 0x55aaed9e4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:30.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.263+0000 7f254e1e0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaed9dd590 0x55aaed9e52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:30.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.263+0000 7f254e1e0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaed9e7e50 0x55aaed9e5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:30.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.263+0000 7f254e1e0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55aaed9eaa20 con 0x55aaed9e7e50 2024-09-19T05:13:30.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.263+0000 7f254e1e0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55aaed8fe0f0 con 0x55aaed9dcc70 2024-09-19T05:13:30.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.263+0000 7f254e1e0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55aaed9eaba0 con 0x55aaed9dd590 2024-09-19T05:13:30.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.263+0000 7f254e1e0740 10 client.0 _ll_register_callbacks cb 0x55aaed8e5bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:30.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.263+0000 7f2549600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaed9dcc70 0x55aaed9e4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:30.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.263+0000 7f2549600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaed9e7e50 0x55aaed9e5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:30.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.263+0000 7f2543e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaed9dd590 0x55aaed9e52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:30.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.263+0000 7f2549600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaed9e7e50 0x55aaed9e5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33118/0 (socket says 172.21.15.43:33118) 2024-09-19T05:13:30.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.263+0000 7f2549600640 1 -- 172.21.15.43:0/612296313 learned_addr learned my addr 172.21.15.43:0/612296313 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:30.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.264+0000 7f2549600640 1 -- 172.21.15.43:0/612296313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaed9dd590 msgr2=0x55aaed9e52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:30.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.264+0000 7f2549600640 1 --2- 172.21.15.43:0/612296313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaed9dd590 0x55aaed9e52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:30.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.264+0000 7f2549600640 1 -- 172.21.15.43:0/612296313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaed9dcc70 msgr2=0x55aaed9e4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:30.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.264+0000 7f2549600640 1 --2- 172.21.15.43:0/612296313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaed9dcc70 0x55aaed9e4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:30.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.264+0000 7f2549600640 1 -- 172.21.15.43:0/612296313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aaed6b43c0 con 0x55aaed9e7e50 2024-09-19T05:13:30.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.264+0000 7f2543e00640 1 --2- 172.21.15.43:0/612296313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaed9dd590 0x55aaed9e52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:13:30.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.264+0000 7f2549600640 1 --2- 172.21.15.43:0/612296313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaed9e7e50 0x55aaed9e5800 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f253c005cc0 tx=0x7f253c003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d1a8063596a2f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:30.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.264+0000 7f252a000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:13:30.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.264+0000 7f252a000640 1 -- 172.21.15.43:0/612296313 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c048030 con 0x55aaed9e7e50 2024-09-19T05:13:30.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.264+0000 7f254e1e0740 1 -- 172.21.15.43:0/612296313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f253c02f940 con 0x55aaed9e7e50 2024-09-19T05:13:30.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.264+0000 7f252a000640 1 -- 172.21.15.43:0/612296313 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f253c004290 con 0x55aaed9e7e50 2024-09-19T05:13:30.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.265+0000 7f252a000640 1 -- 172.21.15.43:0/612296313 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f253c004490 con 0x55aaed9e7e50 2024-09-19T05:13:30.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.265+0000 7f254e1e0740 10 client.17316 fetch_fsmap learned FSMap version 29 2024-09-19T05:13:30.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.265+0000 7f254e1e0740 1 -- 172.21.15.43:0/612296313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55aaed9f6ab0 con 0x55aaed9e7e50 2024-09-19T05:13:30.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.265+0000 7f252a000640 1 -- 172.21.15.43:0/612296313 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f253c02f940 con 0x55aaed9e7e50 2024-09-19T05:13:30.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.265+0000 7f252a000640 1 -- 172.21.15.43:0/612296313 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f253c030b20 con 0x55aaed9e7e50 2024-09-19T05:13:30.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.265+0000 7f254e1e0740 10 client.17316 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:13:30.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.265+0000 7f254e1e0740 10 client.17316 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:13:30.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.265+0000 7f254e1e0740 20 client.17316 populate_metadata read hostname 'smithi043' 2024-09-19T05:13:30.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.265+0000 7f254e1e0740 1 --2- 172.21.15.43:0/612296313 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55aaed9f7e00 0x55aaed9fa1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:30.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.265+0000 7f254e1e0740 4 client.17316 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:13:30.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.265+0000 7f254e1e0740 1 -- 172.21.15.43:0/612296313 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aaed950950 con 0x55aaed9f7e00 2024-09-19T05:13:30.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.266+0000 7f2543e00640 1 --2- 172.21.15.43:0/612296313 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55aaed9f7e00 0x55aaed9fa1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:30.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.266+0000 7f2543e00640 1 --2- 172.21.15.43:0/612296313 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55aaed9f7e00 0x55aaed9fa1f0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44a086057afcdb15 server_cookie=9c2f3915cfdf2e1c in_seq=0 out_seq=0 2024-09-19T05:13:30.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.266+0000 7f252a000640 10 client.17316 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:13:30.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.267+0000 7f252a000640 1 -- 172.21.15.43:0/612296313 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aaed950950 con 0x55aaed9f7e00 2024-09-19T05:13:30.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.267+0000 7f252a000640 10 client.17316 handle_command_reply: tid=0 2024-09-19T05:13:30.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.267+0000 7f254e1e0740 1 client.17316 shutdown 2024-09-19T05:13:30.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.268+0000 7f254e1e0740 1 -- 172.21.15.43:0/612296313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaed9e7e50 msgr2=0x55aaed9e5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:30.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.268+0000 7f254e1e0740 1 --2- 172.21.15.43:0/612296313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaed9e7e50 0x55aaed9e5800 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f253c005cc0 tx=0x7f253c003cd0 comp rx=0 tx=0).stop 2024-09-19T05:13:30.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.268+0000 7f254e1e0740 1 -- 172.21.15.43:0/612296313 shutdown_connections 2024-09-19T05:13:30.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.268+0000 7f254e1e0740 1 --2- 172.21.15.43:0/612296313 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55aaed9f7e00 0x55aaed9fa1f0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:30.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.268+0000 7f254e1e0740 1 --2- 172.21.15.43:0/612296313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaed9e7e50 0x55aaed9e5800 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:30.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.268+0000 7f254e1e0740 1 --2- 172.21.15.43:0/612296313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaed9dd590 0x55aaed9e52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:30.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.268+0000 7f254e1e0740 1 --2- 172.21.15.43:0/612296313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaed9dcc70 0x55aaed9e4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:30.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.268+0000 7f252a000640 0 client.17316 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:13:30.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.268+0000 7f254e1e0740 1 -- 172.21.15.43:0/612296313 >> 172.21.15.43:0/612296313 conn(0x55aaed9d9200 msgr2=0x55aaed9d8380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:30.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.268+0000 7f254e1e0740 1 -- 172.21.15.43:0/612296313 shutdown_connections 2024-09-19T05:13:30.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.268+0000 7f254e1e0740 1 -- 172.21.15.43:0/612296313 wait complete. 2024-09-19T05:13:30.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.268+0000 7f254e1e0740 20 client.17316 trim_cache size 0 max 16384 2024-09-19T05:13:30.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.282+0000 7f254e1e0740 1 Processor -- start 2024-09-19T05:13:30.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.282+0000 7f254e1e0740 1 -- start start 2024-09-19T05:13:30.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.282+0000 7f254e1e0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaed9dcc70 0x55aaed9e6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:30.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.282+0000 7f254e1e0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaed9dd590 0x55aaed9fd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:30.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.282+0000 7f254e1e0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaed9e7e50 0x55aaed9fd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:30.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.282+0000 7f254e1e0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f25441b3f30 con 0x55aaed9dcc70 2024-09-19T05:13:30.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.282+0000 7f254e1e0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f25441b5a70 con 0x55aaed9e7e50 2024-09-19T05:13:30.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.282+0000 7f254e1e0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f25441b4600 con 0x55aaed9dd590 2024-09-19T05:13:30.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.282+0000 7f2549600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaed9dcc70 0x55aaed9e6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:30.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.282+0000 7f2549600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaed9dcc70 0x55aaed9e6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33124/0 (socket says 172.21.15.43:33124) 2024-09-19T05:13:30.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.282+0000 7f2549600640 1 -- 172.21.15.43:0/3806118803 learned_addr learned my addr 172.21.15.43:0/3806118803 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:30.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.282+0000 7f2543e00640 1 --2- 172.21.15.43:0/3806118803 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaed9dd590 0x55aaed9fd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:30.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.283+0000 7f2549600640 1 --2- 172.21.15.43:0/3806118803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaed9e7e50 0x55aaed9fd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:30.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.283+0000 7f2549600640 1 -- 172.21.15.43:0/3806118803 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaed9dd590 msgr2=0x55aaed9fd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:30.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.283+0000 7f2549600640 1 --2- 172.21.15.43:0/3806118803 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaed9dd590 0x55aaed9fd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:30.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.283+0000 7f2549600640 1 -- 172.21.15.43:0/3806118803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaed9e7e50 msgr2=0x55aaed9fd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:30.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.283+0000 7f2549600640 1 --2- 172.21.15.43:0/3806118803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaed9e7e50 0x55aaed9fd750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:30.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.283+0000 7f2549600640 1 -- 172.21.15.43:0/3806118803 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaed9da2b0 con 0x55aaed9dcc70 2024-09-19T05:13:30.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.283+0000 7f2549600640 1 --2- 172.21.15.43:0/3806118803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaed9dcc70 0x55aaed9e6cf0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f253c005de0 tx=0x7f253c0535e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=341960ff8bf98118 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:30.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.283+0000 7f2535200640 1 -- 172.21.15.43:0/3806118803 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c05d070 con 0x55aaed9dcc70 2024-09-19T05:13:30.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.283+0000 7f2535200640 1 -- 172.21.15.43:0/3806118803 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f253c049040 con 0x55aaed9dcc70 2024-09-19T05:13:30.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.283+0000 7f2535200640 1 -- 172.21.15.43:0/3806118803 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c048030 con 0x55aaed9dcc70 2024-09-19T05:13:30.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.284+0000 7f254e1e0740 1 -- 172.21.15.43:0/3806118803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaed9dcc70 msgr2=0x55aaed9e6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:30.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.284+0000 7f254e1e0740 1 --2- 172.21.15.43:0/3806118803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaed9dcc70 0x55aaed9e6cf0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f253c005de0 tx=0x7f253c0535e0 comp rx=0 tx=0).stop 2024-09-19T05:13:30.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.284+0000 7f254e1e0740 1 -- 172.21.15.43:0/3806118803 shutdown_connections 2024-09-19T05:13:30.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.284+0000 7f254e1e0740 1 --2- 172.21.15.43:0/3806118803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaed9e7e50 0x55aaed9fd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:30.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.284+0000 7f254e1e0740 1 --2- 172.21.15.43:0/3806118803 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaed9dd590 0x55aaed9fd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:30.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.284+0000 7f254e1e0740 1 --2- 172.21.15.43:0/3806118803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaed9dcc70 0x55aaed9e6cf0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:30.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.284+0000 7f254e1e0740 1 -- 172.21.15.43:0/3806118803 >> 172.21.15.43:0/3806118803 conn(0x55aaed9d9200 msgr2=0x55aaed9e65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:30.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.284+0000 7f254e1e0740 1 -- 172.21.15.43:0/3806118803 shutdown_connections 2024-09-19T05:13:30.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.284+0000 7f254e1e0740 1 -- 172.21.15.43:0/3806118803 wait complete. 2024-09-19T05:13:30.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.285+0000 7f254e1e0740 1 -- start start 2024-09-19T05:13:30.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.285+0000 7f254e1e0740 1 Processor -- start 2024-09-19T05:13:30.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.285+0000 7f254e1e0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaed9dcc70 0x55aaed9fdb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:30.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.285+0000 7f254e1e0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaed9dd590 0x55aaed9fae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:30.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.285+0000 7f254e1e0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaed9e7e50 0x55aaed9fb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:30.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.285+0000 7f254e1e0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f25441b53a0 con 0x55aaed9dcc70 2024-09-19T05:13:30.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.285+0000 7f254e1e0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f25441b4cd0 con 0x55aaed9e7e50 2024-09-19T05:13:30.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.285+0000 7f254e1e0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f25441b1f30 con 0x55aaed9dd590 2024-09-19T05:13:30.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.285+0000 7f254e1e0740 10 client.0 _ll_register_callbacks cb 0x7f253c046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:30.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.286+0000 7f2549600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaed9dcc70 0x55aaed9fdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:30.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.286+0000 7f2543e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaed9dd590 0x55aaed9fae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:30.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.286+0000 7f2543e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaed9dd590 0x55aaed9fae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:42248/0 (socket says 172.21.15.43:42248) 2024-09-19T05:13:30.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.286+0000 7f2543e00640 1 -- 172.21.15.43:0/781048497 learned_addr learned my addr 172.21.15.43:0/781048497 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:30.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.286+0000 7f2549600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaed9dcc70 0x55aaed9fdb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33132/0 (socket says 172.21.15.43:33132) 2024-09-19T05:13:30.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.286+0000 7f2549600640 1 --2- 172.21.15.43:0/781048497 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaed9e7e50 0x55aaed9fb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:30.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.286+0000 7f2543e00640 1 -- 172.21.15.43:0/781048497 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaed9e7e50 msgr2=0x55aaed9fb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:30.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.286+0000 7f2543e00640 1 --2- 172.21.15.43:0/781048497 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaed9e7e50 0x55aaed9fb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:30.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.286+0000 7f2543e00640 1 -- 172.21.15.43:0/781048497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaed9dcc70 msgr2=0x55aaed9fdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:30.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.286+0000 7f2543e00640 1 --2- 172.21.15.43:0/781048497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaed9dcc70 0x55aaed9fdb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:30.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.286+0000 7f2543e00640 1 -- 172.21.15.43:0/781048497 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aaed9d4770 con 0x55aaed9dd590 2024-09-19T05:13:30.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.286+0000 7f2549600640 1 --2- 172.21.15.43:0/781048497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaed9dcc70 0x55aaed9fdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:13:30.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.287+0000 7f2543e00640 1 --2- 172.21.15.43:0/781048497 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaed9dd590 0x55aaed9fae60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55aaed9da580 tx=0x7f2530005630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b5b3096923a734f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:30.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.287+0000 7f252a000640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:13:30.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.287+0000 7f252a000640 1 -- 172.21.15.43:0/781048497 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2530024030 con 0x55aaed9dd590 2024-09-19T05:13:30.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.287+0000 7f252a000640 1 -- 172.21.15.43:0/781048497 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f2530013890 con 0x55aaed9dd590 2024-09-19T05:13:30.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.287+0000 7f254e1e0740 1 -- 172.21.15.43:0/781048497 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f253c049040 con 0x55aaed9dd590 2024-09-19T05:13:30.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.287+0000 7f252a000640 1 -- 172.21.15.43:0/781048497 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2530004e50 con 0x55aaed9dd590 2024-09-19T05:13:30.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.287+0000 7f254e1e0740 10 client.27244 fetch_fsmap learned FSMap version 29 2024-09-19T05:13:30.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.287+0000 7f254e1e0740 1 -- 172.21.15.43:0/781048497 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55aaeda627a0 con 0x55aaed9dd590 2024-09-19T05:13:30.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.287+0000 7f252a000640 1 -- 172.21.15.43:0/781048497 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f253c049040 con 0x55aaed9dd590 2024-09-19T05:13:30.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.288+0000 7f252a000640 1 -- 172.21.15.43:0/781048497 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2530030410 con 0x55aaed9dd590 2024-09-19T05:13:30.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.288+0000 7f254e1e0740 10 client.27244 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:13:30.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.288+0000 7f254e1e0740 10 client.27244 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:13:30.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.288+0000 7f254e1e0740 20 client.27244 populate_metadata read hostname 'smithi043' 2024-09-19T05:13:30.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.288+0000 7f254e1e0740 1 --2- 172.21.15.43:0/781048497 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55aaeda62f50 0x55aaeda65370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:30.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.288+0000 7f254e1e0740 4 client.27244 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:13:30.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.288+0000 7f254e1e0740 1 -- 172.21.15.43:0/781048497 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aaed94ae60 con 0x55aaeda62f50 2024-09-19T05:13:30.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.288+0000 7f2549600640 1 --2- 172.21.15.43:0/781048497 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55aaeda62f50 0x55aaeda65370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:30.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.288+0000 7f2549600640 1 --2- 172.21.15.43:0/781048497 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55aaeda62f50 0x55aaeda65370 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efd98cd227f5f2dd server_cookie=b5a291d7b1a277f1 in_seq=0 out_seq=0 2024-09-19T05:13:30.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.288+0000 7f252a000640 10 client.27244 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:13:30.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.289+0000 7f252a000640 1 -- 172.21.15.43:0/781048497 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aaed94ae60 con 0x55aaeda62f50 2024-09-19T05:13:30.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.289+0000 7f252a000640 10 client.27244 handle_command_reply: tid=0 2024-09-19T05:13:30.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.289+0000 7f254e1e0740 1 client.27244 shutdown 2024-09-19T05:13:30.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.289+0000 7f254e1e0740 1 -- 172.21.15.43:0/781048497 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaed9dd590 msgr2=0x55aaed9fae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:30.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.289+0000 7f254e1e0740 1 --2- 172.21.15.43:0/781048497 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaed9dd590 0x55aaed9fae60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55aaed9da580 tx=0x7f2530005630 comp rx=0 tx=0).stop 2024-09-19T05:13:30.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.289+0000 7f254e1e0740 1 -- 172.21.15.43:0/781048497 shutdown_connections 2024-09-19T05:13:30.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.289+0000 7f254e1e0740 1 --2- 172.21.15.43:0/781048497 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55aaeda62f50 0x55aaeda65370 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:30.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.289+0000 7f254e1e0740 1 --2- 172.21.15.43:0/781048497 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aaed9e7e50 0x55aaed9fb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:30.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.289+0000 7f254e1e0740 1 --2- 172.21.15.43:0/781048497 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aaed9dd590 0x55aaed9fae60 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:30.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.289+0000 7f254e1e0740 1 --2- 172.21.15.43:0/781048497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aaed9dcc70 0x55aaed9fdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:30.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.289+0000 7f254e1e0740 1 -- 172.21.15.43:0/781048497 >> 172.21.15.43:0/781048497 conn(0x55aaed9d9200 msgr2=0x55aaed9e6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:30.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.289+0000 7f252a000640 0 client.27244 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:13:30.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.289+0000 7f254e1e0740 1 -- 172.21.15.43:0/781048497 shutdown_connections 2024-09-19T05:13:30.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.289+0000 7f254e1e0740 1 -- 172.21.15.43:0/781048497 wait complete. 2024-09-19T05:13:30.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.289+0000 7f254e1e0740 20 client.27244 trim_cache size 0 max 16384 2024-09-19T05:13:30.294 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:13:30.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.290+0000 7f254b600640 1 -- 172.21.15.43:0/348078862 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f252c0a33b0 msgr2=0x7f252c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:30.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.290+0000 7f254b600640 1 --2- 172.21.15.43:0/348078862 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f252c0a33b0 0x7f252c0a5870 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f25380cd9b0 tx=0x7f2538098040 comp rx=0 tx=0).stop 2024-09-19T05:13:30.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.290+0000 7f254b600640 1 -- 172.21.15.43:0/348078862 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2544074590 msgr2=0x7f25441a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:30.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.290+0000 7f254b600640 1 --2- 172.21.15.43:0/348078862 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2544074590 0x7f25441a4720 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f253800e9f0 tx=0x7f253800edf0 comp rx=0 tx=0).stop 2024-09-19T05:13:30.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.290+0000 7f254b600640 1 -- 172.21.15.43:0/348078862 shutdown_connections 2024-09-19T05:13:30.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.290+0000 7f254b600640 1 --2- 172.21.15.43:0/348078862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2544112b90 0x7f254419e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:30.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.290+0000 7f254b600640 1 --2- 172.21.15.43:0/348078862 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f252c0a33b0 0x7f252c0a5870 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:30.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.290+0000 7f254b600640 1 --2- 172.21.15.43:0/348078862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2544074eb0 0x7f25441a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:30.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.290+0000 7f254b600640 1 --2- 172.21.15.43:0/348078862 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2544074590 0x7f25441a4720 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:30.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.290+0000 7f254b600640 1 -- 172.21.15.43:0/348078862 >> 172.21.15.43:0/348078862 conn(0x7f25440fd8c0 msgr2=0x7f25440ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:30.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.290+0000 7f254b600640 1 -- 172.21.15.43:0/348078862 shutdown_connections 2024-09-19T05:13:30.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:30.290+0000 7f254b600640 1 -- 172.21.15.43:0/348078862 wait complete. 2024-09-19T05:13:32.109 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T05:13:32.110 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T05:13:32.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:31 smithi151 ceph-mon[35425]: pgmap v275: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:13:32.165 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:31 smithi043 ceph-mon[30800]: pgmap v275: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:13:32.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:31 smithi096 ceph-mon[35085]: pgmap v275: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:13:32.499 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:13:33.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.031+0000 7fd423200640 1 -- 172.21.15.43:0/1762636433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd424077040 msgr2=0x7fd424112330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:33.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.031+0000 7fd423200640 1 --2- 172.21.15.43:0/1762636433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd424077040 0x7fd424112330 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd40c009900 tx=0x7fd40c02a410 comp rx=0 tx=0).stop 2024-09-19T05:13:33.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.031+0000 7fd423200640 1 -- 172.21.15.43:0/1762636433 shutdown_connections 2024-09-19T05:13:33.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.031+0000 7fd423200640 1 --2- 172.21.15.43:0/1762636433 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd424112870 0x7fd424114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:33.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.031+0000 7fd423200640 1 --2- 172.21.15.43:0/1762636433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd424077040 0x7fd424112330 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:33.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.031+0000 7fd423200640 1 --2- 172.21.15.43:0/1762636433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd424076720 0x7fd424076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:33.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.031+0000 7fd423200640 1 -- 172.21.15.43:0/1762636433 >> 172.21.15.43:0/1762636433 conn(0x7fd4240ff8d0 msgr2=0x7fd424101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:33.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.031+0000 7fd423200640 1 -- 172.21.15.43:0/1762636433 shutdown_connections 2024-09-19T05:13:33.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.031+0000 7fd423200640 1 -- 172.21.15.43:0/1762636433 wait complete. 2024-09-19T05:13:33.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.032+0000 7fd423200640 1 Processor -- start 2024-09-19T05:13:33.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.033+0000 7fd423200640 1 -- start start 2024-09-19T05:13:33.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.033+0000 7fd423200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd424076720 0x7fd4241a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:33.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.033+0000 7fd423200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd424077040 0x7fd4241a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:33.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.033+0000 7fd423200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd424112870 0x7fd42419e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:33.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.033+0000 7fd423200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd424118440 con 0x7fd424112870 2024-09-19T05:13:33.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.033+0000 7fd423200640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd4241182c0 con 0x7fd424077040 2024-09-19T05:13:33.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.033+0000 7fd423200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd4241185c0 con 0x7fd424076720 2024-09-19T05:13:33.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.033+0000 7fd421e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd424076720 0x7fd4241a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:33.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.033+0000 7fd422800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd424112870 0x7fd42419e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:33.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.033+0000 7fd421e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd424076720 0x7fd4241a4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:42262/0 (socket says 172.21.15.43:42262) 2024-09-19T05:13:33.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.033+0000 7fd421e00640 1 -- 172.21.15.43:0/3811454236 learned_addr learned my addr 172.21.15.43:0/3811454236 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:33.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.033+0000 7fd419400640 1 --2- 172.21.15.43:0/3811454236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd424077040 0x7fd4241a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:33.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.034+0000 7fd421e00640 1 -- 172.21.15.43:0/3811454236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd424077040 msgr2=0x7fd4241a4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:33.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.034+0000 7fd421e00640 1 --2- 172.21.15.43:0/3811454236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd424077040 0x7fd4241a4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:33.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.034+0000 7fd421e00640 1 -- 172.21.15.43:0/3811454236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd424112870 msgr2=0x7fd42419e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:33.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.034+0000 7fd421e00640 1 --2- 172.21.15.43:0/3811454236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd424112870 0x7fd42419e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:33.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.034+0000 7fd421e00640 1 -- 172.21.15.43:0/3811454236 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd42419efa0 con 0x7fd424076720 2024-09-19T05:13:33.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.034+0000 7fd422800640 1 --2- 172.21.15.43:0/3811454236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd424112870 0x7fd42419e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:13:33.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.034+0000 7fd421e00640 1 --2- 172.21.15.43:0/3811454236 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd424076720 0x7fd4241a4660 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd41400ea60 tx=0x7fd41400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:33.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.034+0000 7fd41b400640 1 -- 172.21.15.43:0/3811454236 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41400cde0 con 0x7fd424076720 2024-09-19T05:13:33.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.035+0000 7fd41b400640 1 -- 172.21.15.43:0/3811454236 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4140035f0 con 0x7fd424076720 2024-09-19T05:13:33.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.035+0000 7fd423200640 1 -- 172.21.15.43:0/3811454236 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd42419f290 con 0x7fd424076720 2024-09-19T05:13:33.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.035+0000 7fd41b400640 1 -- 172.21.15.43:0/3811454236 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd414010670 con 0x7fd424076720 2024-09-19T05:13:33.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.035+0000 7fd423200640 1 -- 172.21.15.43:0/3811454236 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4241ab5e0 con 0x7fd424076720 2024-09-19T05:13:33.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.039+0000 7fd42b9e1740 1 Processor -- start 2024-09-19T05:13:33.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.039+0000 7fd42b9e1740 1 -- start start 2024-09-19T05:13:33.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.039+0000 7fd42b9e1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c50de61cb0 0x55c50de62090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:33.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.039+0000 7fd41b400640 1 -- 172.21.15.43:0/3811454236 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fd4140037a0 con 0x7fd424076720 2024-09-19T05:13:33.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.039+0000 7fd42b9e1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c50de625d0 0x55c50de6c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:33.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.039+0000 7fd42b9e1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c50de6ce90 0x55c50de6f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:33.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.039+0000 7fd42b9e1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c50ddc8290 con 0x55c50de625d0 2024-09-19T05:13:33.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.039+0000 7fd42b9e1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c50dd6fb40 con 0x55c50de61cb0 2024-09-19T05:13:33.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.039+0000 7fd42b9e1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c50ddc4250 con 0x55c50de6ce90 2024-09-19T05:13:33.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.040+0000 7fd422800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c50de61cb0 0x55c50de62090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:33.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.040+0000 7fd421e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c50de625d0 0x55c50de6c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:33.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.040+0000 7fd422800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c50de61cb0 0x55c50de62090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:54828/0 (socket says 172.21.15.43:54828) 2024-09-19T05:13:33.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.040+0000 7fd422800640 1 -- 172.21.15.43:0/1554625736 learned_addr learned my addr 172.21.15.43:0/1554625736 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:33.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.040+0000 7fd419400640 1 --2- 172.21.15.43:0/1554625736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c50de6ce90 0x55c50de6f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:33.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.040+0000 7fd422800640 1 -- 172.21.15.43:0/1554625736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c50de6ce90 msgr2=0x55c50de6f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:33.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.040+0000 7fd422800640 1 --2- 172.21.15.43:0/1554625736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c50de6ce90 0x55c50de6f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:33.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.040+0000 7fd422800640 1 -- 172.21.15.43:0/1554625736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c50de625d0 msgr2=0x55c50de6c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:33.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.040+0000 7fd422800640 1 --2- 172.21.15.43:0/1554625736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c50de625d0 0x55c50de6c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:33.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.040+0000 7fd422800640 1 -- 172.21.15.43:0/1554625736 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c50db396a0 con 0x55c50de61cb0 2024-09-19T05:13:33.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.041+0000 7fd422800640 1 --2- 172.21.15.43:0/1554625736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c50de61cb0 0x55c50de62090 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd41c011790 tx=0x7fd41c011c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cd9076fe58ad59e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:33.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.041+0000 7fd413400640 1 -- 172.21.15.43:0/1554625736 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c003c30 con 0x55c50de61cb0 2024-09-19T05:13:33.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.041+0000 7fd413400640 1 -- 172.21.15.43:0/1554625736 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd41c003dd0 con 0x55c50de61cb0 2024-09-19T05:13:33.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.041+0000 7fd413400640 1 -- 172.21.15.43:0/1554625736 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c0193e0 con 0x55c50de61cb0 2024-09-19T05:13:33.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.041+0000 7fd41b400640 1 --2- 172.21.15.43:0/3811454236 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fd4080a33b0 0x7fd4080a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:33.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.041+0000 7fd42b9e1740 1 -- 172.21.15.43:0/1554625736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c50de61cb0 msgr2=0x55c50de62090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:33.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.041+0000 7fd42b9e1740 1 --2- 172.21.15.43:0/1554625736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c50de61cb0 0x55c50de62090 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd41c011790 tx=0x7fd41c011c50 comp rx=0 tx=0).stop 2024-09-19T05:13:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.042+0000 7fd42b9e1740 1 -- 172.21.15.43:0/1554625736 shutdown_connections 2024-09-19T05:13:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.042+0000 7fd42b9e1740 1 --2- 172.21.15.43:0/1554625736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c50de6ce90 0x55c50de6f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.042+0000 7fd42b9e1740 1 --2- 172.21.15.43:0/1554625736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c50de625d0 0x55c50de6c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.042+0000 7fd42b9e1740 1 --2- 172.21.15.43:0/1554625736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c50de61cb0 0x55c50de62090 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.042+0000 7fd42b9e1740 1 -- 172.21.15.43:0/1554625736 >> 172.21.15.43:0/1554625736 conn(0x55c50de5e240 msgr2=0x55c50de5e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.042+0000 7fd41b400640 1 -- 172.21.15.43:0/3811454236 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7fd4140cffe0 con 0x7fd424076720 2024-09-19T05:13:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.042+0000 7fd421e00640 1 --2- 172.21.15.43:0/3811454236 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fd4080a33b0 0x7fd4080a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.042+0000 7fd42b9e1740 1 -- 172.21.15.43:0/1554625736 shutdown_connections 2024-09-19T05:13:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.042+0000 7fd42b9e1740 1 -- 172.21.15.43:0/1554625736 wait complete. 2024-09-19T05:13:33.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.042+0000 7fd42b9e1740 1 -- start start 2024-09-19T05:13:33.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.042+0000 7fd421e00640 1 --2- 172.21.15.43:0/3811454236 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fd4080a33b0 0x7fd4080a5870 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd4140cd7d0 tx=0x7fd4140d3b30 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:33.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.043+0000 7fd42b9e1740 1 Processor -- start 2024-09-19T05:13:33.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.043+0000 7fd42b9e1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c50de61cb0 0x55c50de6c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:33.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.043+0000 7fd42b9e1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c50de625d0 0x55c50de699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:33.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.043+0000 7fd42b9e1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c50de6ce90 0x55c50de69f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:33.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.043+0000 7fd42b9e1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c50de6fb00 con 0x55c50de6ce90 2024-09-19T05:13:33.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.043+0000 7fd42b9e1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c50dd82fa0 con 0x55c50de61cb0 2024-09-19T05:13:33.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.043+0000 7fd42b9e1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c50de6fc80 con 0x55c50de625d0 2024-09-19T05:13:33.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.043+0000 7fd42b9e1740 10 client.0 _ll_register_callbacks cb 0x55c50dad6ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:33.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.043+0000 7fd422800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c50de6ce90 0x55c50de69f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:33.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.044+0000 7fd422800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c50de6ce90 0x55c50de69f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33188/0 (socket says 172.21.15.43:33188) 2024-09-19T05:13:33.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.044+0000 7fd422800640 1 -- 172.21.15.43:0/1356694318 learned_addr learned my addr 172.21.15.43:0/1356694318 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:33.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.044+0000 7fd419400640 1 --2- 172.21.15.43:0/1356694318 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c50de625d0 0x55c50de699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:33.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.044+0000 7fd422800640 1 --2- 172.21.15.43:0/1356694318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c50de61cb0 0x55c50de6c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:33.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.044+0000 7fd422800640 1 -- 172.21.15.43:0/1356694318 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c50de625d0 msgr2=0x55c50de699e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:33.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.044+0000 7fd422800640 1 --2- 172.21.15.43:0/1356694318 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c50de625d0 0x55c50de699e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:33.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.044+0000 7fd422800640 1 -- 172.21.15.43:0/1356694318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c50de61cb0 msgr2=0x55c50de6c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:33.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.044+0000 7fd422800640 1 --2- 172.21.15.43:0/1356694318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c50de61cb0 0x55c50de6c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:33.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.044+0000 7fd422800640 1 -- 172.21.15.43:0/1356694318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c50de6a720 con 0x55c50de6ce90 2024-09-19T05:13:33.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.044+0000 7fd419400640 1 --2- 172.21.15.43:0/1356694318 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c50de625d0 0x55c50de699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:13:33.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.044+0000 7fd422800640 1 --2- 172.21.15.43:0/1356694318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c50de6ce90 0x55c50de69f20 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd41c011760 tx=0x7fd41c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=138cb99ba9a88f27 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:33.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.044+0000 7fd407e00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:13:33.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.045+0000 7fd407e00640 1 -- 172.21.15.43:0/1356694318 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c00fdb0 con 0x55c50de6ce90 2024-09-19T05:13:33.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.045+0000 7fd42b9e1740 1 -- 172.21.15.43:0/1356694318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd41c003dd0 con 0x55c50de6ce90 2024-09-19T05:13:33.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.045+0000 7fd407e00640 1 -- 172.21.15.43:0/1356694318 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7fd41c011050 con 0x55c50de6ce90 2024-09-19T05:13:33.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.045+0000 7fd407e00640 1 -- 172.21.15.43:0/1356694318 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd41c011250 con 0x55c50de6ce90 2024-09-19T05:13:33.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.045+0000 7fd42b9e1740 10 client.17334 fetch_fsmap learned FSMap version 29 2024-09-19T05:13:33.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.045+0000 7fd42b9e1740 1 -- 172.21.15.43:0/1356694318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c50de7b9e0 con 0x55c50de6ce90 2024-09-19T05:13:33.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.046+0000 7fd407e00640 1 -- 172.21.15.43:0/1356694318 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd41c003dd0 con 0x55c50de6ce90 2024-09-19T05:13:33.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.046+0000 7fd407e00640 1 -- 172.21.15.43:0/1356694318 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd41c018070 con 0x55c50de6ce90 2024-09-19T05:13:33.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.046+0000 7fd42b9e1740 10 client.17334 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:13:33.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.046+0000 7fd42b9e1740 10 client.17334 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:13:33.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.046+0000 7fd42b9e1740 20 client.17334 populate_metadata read hostname 'smithi043' 2024-09-19T05:13:33.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.046+0000 7fd42b9e1740 1 --2- 172.21.15.43:0/1356694318 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55c50de7cdf0 0x55c50de7f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:33.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.046+0000 7fd42b9e1740 4 client.17334 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:13:33.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.046+0000 7fd42b9e1740 1 -- 172.21.15.43:0/1356694318 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c50ddd5800 con 0x55c50de7cdf0 2024-09-19T05:13:33.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.046+0000 7fd419400640 1 --2- 172.21.15.43:0/1356694318 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55c50de7cdf0 0x55c50de7f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:33.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.047+0000 7fd419400640 1 --2- 172.21.15.43:0/1356694318 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55c50de7cdf0 0x55c50de7f1e0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1bf31a137ac7f0d server_cookie=7e5004917579905e in_seq=0 out_seq=0 2024-09-19T05:13:33.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.047+0000 7fd407e00640 10 client.17334 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:13:33.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.047+0000 7fd407e00640 1 -- 172.21.15.43:0/1356694318 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c50ddd5800 con 0x55c50de7cdf0 2024-09-19T05:13:33.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.047+0000 7fd407e00640 10 client.17334 handle_command_reply: tid=0 2024-09-19T05:13:33.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.047+0000 7fd42b9e1740 1 client.17334 shutdown 2024-09-19T05:13:33.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.048+0000 7fd42b9e1740 1 -- 172.21.15.43:0/1356694318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c50de6ce90 msgr2=0x55c50de69f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:33.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.048+0000 7fd42b9e1740 1 --2- 172.21.15.43:0/1356694318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c50de6ce90 0x55c50de69f20 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd41c011760 tx=0x7fd41c0045e0 comp rx=0 tx=0).stop 2024-09-19T05:13:33.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.048+0000 7fd42b9e1740 1 -- 172.21.15.43:0/1356694318 shutdown_connections 2024-09-19T05:13:33.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.048+0000 7fd42b9e1740 1 --2- 172.21.15.43:0/1356694318 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55c50de7cdf0 0x55c50de7f1e0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:33.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.048+0000 7fd42b9e1740 1 --2- 172.21.15.43:0/1356694318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c50de6ce90 0x55c50de69f20 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:33.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.048+0000 7fd42b9e1740 1 --2- 172.21.15.43:0/1356694318 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c50de625d0 0x55c50de699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:33.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.048+0000 7fd42b9e1740 1 --2- 172.21.15.43:0/1356694318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c50de61cb0 0x55c50de6c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:33.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.048+0000 7fd42b9e1740 1 -- 172.21.15.43:0/1356694318 >> 172.21.15.43:0/1356694318 conn(0x55c50de5e240 msgr2=0x55c50de63f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:33.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.048+0000 7fd407e00640 0 client.17334 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:13:33.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.048+0000 7fd42b9e1740 1 -- 172.21.15.43:0/1356694318 shutdown_connections 2024-09-19T05:13:33.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.048+0000 7fd42b9e1740 1 -- 172.21.15.43:0/1356694318 wait complete. 2024-09-19T05:13:33.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.048+0000 7fd42b9e1740 20 client.17334 trim_cache size 0 max 16384 2024-09-19T05:13:33.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.066+0000 7fd42b9e1740 1 Processor -- start 2024-09-19T05:13:33.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.066+0000 7fd42b9e1740 1 -- start start 2024-09-19T05:13:33.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.067+0000 7fd42b9e1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c50de5e240 0x55c50de6b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:33.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.067+0000 7fd42b9e1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c50de61cb0 0x55c50de84900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:33.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.067+0000 7fd42b9e1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c50de625d0 0x55c50de84e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:33.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.067+0000 7fd42b9e1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd4241b3d90 con 0x55c50de5e240 2024-09-19T05:13:33.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.067+0000 7fd42b9e1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd4241b58d0 con 0x55c50de625d0 2024-09-19T05:13:33.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.067+0000 7fd42b9e1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd4241b4460 con 0x55c50de61cb0 2024-09-19T05:13:33.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.067+0000 7fd422800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c50de5e240 0x55c50de6b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:33.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.067+0000 7fd422800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c50de5e240 0x55c50de6b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33202/0 (socket says 172.21.15.43:33202) 2024-09-19T05:13:33.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.067+0000 7fd422800640 1 -- 172.21.15.43:0/1606559437 learned_addr learned my addr 172.21.15.43:0/1606559437 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:33.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.067+0000 7fd419400640 1 --2- 172.21.15.43:0/1606559437 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c50de61cb0 0x55c50de84900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:33.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.067+0000 7fd422800640 1 -- 172.21.15.43:0/1606559437 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c50de61cb0 msgr2=0x55c50de84900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:33.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.067+0000 7fd422800640 1 --2- 172.21.15.43:0/1606559437 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c50de61cb0 0x55c50de84900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:33.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.067+0000 7fd422800640 1 -- 172.21.15.43:0/1606559437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c50de625d0 msgr2=0x55c50de84e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:33.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.067+0000 7fd422800640 1 --2- 172.21.15.43:0/1606559437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c50de625d0 0x55c50de84e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:33.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.067+0000 7fd422800640 1 -- 172.21.15.43:0/1606559437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c50de6a520 con 0x55c50de5e240 2024-09-19T05:13:33.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.068+0000 7fd422800640 1 --2- 172.21.15.43:0/1606559437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c50de5e240 0x55c50de6b950 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd41c005d70 tx=0x7fd41c004440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55f82cc4d8a71eba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:33.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.068+0000 7fd412a00640 1 -- 172.21.15.43:0/1606559437 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c015070 con 0x55c50de5e240 2024-09-19T05:13:33.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.068+0000 7fd412a00640 1 -- 172.21.15.43:0/1606559437 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd41c003a30 con 0x55c50de5e240 2024-09-19T05:13:33.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.068+0000 7fd412a00640 1 -- 172.21.15.43:0/1606559437 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c005770 con 0x55c50de5e240 2024-09-19T05:13:33.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.068+0000 7fd42b9e1740 1 -- 172.21.15.43:0/1606559437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c50de5e240 msgr2=0x55c50de6b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:33.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.068+0000 7fd42b9e1740 1 --2- 172.21.15.43:0/1606559437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c50de5e240 0x55c50de6b950 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd41c005d70 tx=0x7fd41c004440 comp rx=0 tx=0).stop 2024-09-19T05:13:33.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.069+0000 7fd42b9e1740 1 -- 172.21.15.43:0/1606559437 shutdown_connections 2024-09-19T05:13:33.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.069+0000 7fd42b9e1740 1 --2- 172.21.15.43:0/1606559437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c50de625d0 0x55c50de84e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:33.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.069+0000 7fd42b9e1740 1 --2- 172.21.15.43:0/1606559437 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c50de61cb0 0x55c50de84900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:33.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.069+0000 7fd42b9e1740 1 --2- 172.21.15.43:0/1606559437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c50de5e240 0x55c50de6b950 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:33.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.069+0000 7fd42b9e1740 1 -- 172.21.15.43:0/1606559437 >> 172.21.15.43:0/1606559437 conn(0x55c50de6a080 msgr2=0x55c50de6b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:33.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.069+0000 7fd42b9e1740 1 -- 172.21.15.43:0/1606559437 shutdown_connections 2024-09-19T05:13:33.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.069+0000 7fd42b9e1740 1 -- 172.21.15.43:0/1606559437 wait complete. 2024-09-19T05:13:33.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.069+0000 7fd42b9e1740 1 -- start start 2024-09-19T05:13:33.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.069+0000 7fd42b9e1740 1 Processor -- start 2024-09-19T05:13:33.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.070+0000 7fd42b9e1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c50de5e240 0x55c50de72da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:33.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.070+0000 7fd42b9e1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c50de61cb0 0x55c50de732e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:33.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.070+0000 7fd42b9e1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c50de625d0 0x55c50de73820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:33.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.070+0000 7fd42b9e1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd4241b5200 con 0x55c50de5e240 2024-09-19T05:13:33.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.070+0000 7fd42b9e1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd4241b4b30 con 0x55c50de61cb0 2024-09-19T05:13:33.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.070+0000 7fd42b9e1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd4241b1d90 con 0x55c50de625d0 2024-09-19T05:13:33.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.070+0000 7fd42b9e1740 10 client.0 _ll_register_callbacks cb 0x7fd4140d3a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:33.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.070+0000 7fd419400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c50de61cb0 0x55c50de732e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:33.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.070+0000 7fd422800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c50de5e240 0x55c50de72da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:33.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.070+0000 7fd419400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c50de61cb0 0x55c50de732e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:54856/0 (socket says 172.21.15.43:54856) 2024-09-19T05:13:33.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.070+0000 7fd419400640 1 -- 172.21.15.43:0/3668326517 learned_addr learned my addr 172.21.15.43:0/3668326517 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:33.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.070+0000 7fd422800640 1 --2- 172.21.15.43:0/3668326517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c50de625d0 0x55c50de73820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:33.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.070+0000 7fd419400640 1 -- 172.21.15.43:0/3668326517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c50de625d0 msgr2=0x55c50de73820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:33.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.070+0000 7fd419400640 1 --2- 172.21.15.43:0/3668326517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c50de625d0 0x55c50de73820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:33.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.070+0000 7fd419400640 1 -- 172.21.15.43:0/3668326517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c50de5e240 msgr2=0x55c50de72da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:33.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.070+0000 7fd419400640 1 --2- 172.21.15.43:0/3668326517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c50de5e240 0x55c50de72da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:33.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.070+0000 7fd419400640 1 -- 172.21.15.43:0/3668326517 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c50dee6a40 con 0x55c50de61cb0 2024-09-19T05:13:33.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.070+0000 7fd422800640 1 --2- 172.21.15.43:0/3668326517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c50de625d0 0x55c50de73820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:13:33.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.071+0000 7fd419400640 1 --2- 172.21.15.43:0/3668326517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c50de61cb0 0x55c50de732e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd40c0096c0 tx=0x7fd40c0044d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a584711f1e2e46ee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:33.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.071+0000 7fd407e00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:13:33.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.071+0000 7fd407e00640 1 -- 172.21.15.43:0/3668326517 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40c035040 con 0x55c50de61cb0 2024-09-19T05:13:33.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.071+0000 7fd407e00640 1 -- 172.21.15.43:0/3668326517 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7fd40c005510 con 0x55c50de61cb0 2024-09-19T05:13:33.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.071+0000 7fd42b9e1740 1 -- 172.21.15.43:0/3668326517 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd41c003a30 con 0x55c50de61cb0 2024-09-19T05:13:33.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.072+0000 7fd407e00640 1 -- 172.21.15.43:0/3668326517 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd40c0046c0 con 0x55c50de61cb0 2024-09-19T05:13:33.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.072+0000 7fd42b9e1740 10 client.27359 fetch_fsmap learned FSMap version 29 2024-09-19T05:13:33.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.072+0000 7fd42b9e1740 1 -- 172.21.15.43:0/3668326517 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c50dee7dd0 con 0x55c50de61cb0 2024-09-19T05:13:33.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.072+0000 7fd407e00640 1 -- 172.21.15.43:0/3668326517 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd41c003a30 con 0x55c50de61cb0 2024-09-19T05:13:33.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.072+0000 7fd407e00640 1 -- 172.21.15.43:0/3668326517 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd40c031b50 con 0x55c50de61cb0 2024-09-19T05:13:33.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.072+0000 7fd42b9e1740 10 client.27359 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:13:33.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.072+0000 7fd42b9e1740 10 client.27359 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:13:33.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.072+0000 7fd42b9e1740 20 client.27359 populate_metadata read hostname 'smithi043' 2024-09-19T05:13:33.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.072+0000 7fd42b9e1740 1 --2- 172.21.15.43:0/3668326517 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55c50de6ce90 0x55c50deea5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:33.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.072+0000 7fd42b9e1740 4 client.27359 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T05:13:33.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.072+0000 7fd42b9e1740 1 -- 172.21.15.43:0/3668326517 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c50ddcfd10 con 0x55c50de6ce90 2024-09-19T05:13:33.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.073+0000 7fd422800640 1 --2- 172.21.15.43:0/3668326517 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55c50de6ce90 0x55c50deea5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:33.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.073+0000 7fd422800640 1 --2- 172.21.15.43:0/3668326517 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55c50de6ce90 0x55c50deea5c0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e38155cf724fb7fb server_cookie=fa2495d48742dfa5 in_seq=0 out_seq=0 2024-09-19T05:13:33.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.073+0000 7fd407e00640 10 client.27359 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:13:33.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.074+0000 7fd407e00640 1 -- 172.21.15.43:0/3668326517 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c50ddcfd10 con 0x55c50de6ce90 2024-09-19T05:13:33.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.074+0000 7fd407e00640 10 client.27359 handle_command_reply: tid=0 2024-09-19T05:13:33.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.074+0000 7fd42b9e1740 1 client.27359 shutdown 2024-09-19T05:13:33.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.074+0000 7fd42b9e1740 1 -- 172.21.15.43:0/3668326517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c50de61cb0 msgr2=0x55c50de732e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:33.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.074+0000 7fd42b9e1740 1 --2- 172.21.15.43:0/3668326517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c50de61cb0 0x55c50de732e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd40c0096c0 tx=0x7fd40c0044d0 comp rx=0 tx=0).stop 2024-09-19T05:13:33.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.075+0000 7fd42b9e1740 1 -- 172.21.15.43:0/3668326517 shutdown_connections 2024-09-19T05:13:33.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.075+0000 7fd42b9e1740 1 --2- 172.21.15.43:0/3668326517 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55c50de6ce90 0x55c50deea5c0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:33.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.075+0000 7fd42b9e1740 1 --2- 172.21.15.43:0/3668326517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c50de625d0 0x55c50de73820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:33.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.075+0000 7fd42b9e1740 1 --2- 172.21.15.43:0/3668326517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c50de61cb0 0x55c50de732e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:33.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.075+0000 7fd42b9e1740 1 --2- 172.21.15.43:0/3668326517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c50de5e240 0x55c50de72da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:33.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.075+0000 7fd42b9e1740 1 -- 172.21.15.43:0/3668326517 >> 172.21.15.43:0/3668326517 conn(0x55c50de6a080 msgr2=0x55c50de6ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:33.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.075+0000 7fd407e00640 0 client.27359 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:13:33.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.075+0000 7fd42b9e1740 1 -- 172.21.15.43:0/3668326517 shutdown_connections 2024-09-19T05:13:33.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.075+0000 7fd42b9e1740 1 -- 172.21.15.43:0/3668326517 wait complete. 2024-09-19T05:13:33.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.075+0000 7fd42b9e1740 20 client.27359 trim_cache size 0 max 16384 2024-09-19T05:13:33.080 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:13:33.080 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-09-19T05:13:33.080 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "7ed96aac-c314-445c-983a-4a09305649f1", 2024-09-19T05:13:33.080 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-09-19T05:13:33.080 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:13:33.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.076+0000 7fd423200640 1 -- 172.21.15.43:0/3811454236 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fd4080a33b0 msgr2=0x7fd4080a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:33.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.076+0000 7fd423200640 1 --2- 172.21.15.43:0/3811454236 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fd4080a33b0 0x7fd4080a5870 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd4140cd7d0 tx=0x7fd4140d3b30 comp rx=0 tx=0).stop 2024-09-19T05:13:33.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.076+0000 7fd423200640 1 -- 172.21.15.43:0/3811454236 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd424076720 msgr2=0x7fd4241a4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:33.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.076+0000 7fd423200640 1 --2- 172.21.15.43:0/3811454236 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd424076720 0x7fd4241a4660 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd41400ea60 tx=0x7fd41400ef20 comp rx=0 tx=0).stop 2024-09-19T05:13:33.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.076+0000 7fd423200640 1 -- 172.21.15.43:0/3811454236 shutdown_connections 2024-09-19T05:13:33.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.076+0000 7fd423200640 1 --2- 172.21.15.43:0/3811454236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd424112870 0x7fd42419e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:33.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.076+0000 7fd423200640 1 --2- 172.21.15.43:0/3811454236 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fd4080a33b0 0x7fd4080a5870 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:33.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.076+0000 7fd423200640 1 --2- 172.21.15.43:0/3811454236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd424077040 0x7fd4241a4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:33.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.076+0000 7fd423200640 1 --2- 172.21.15.43:0/3811454236 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd424076720 0x7fd4241a4660 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:33.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.076+0000 7fd423200640 1 -- 172.21.15.43:0/3811454236 >> 172.21.15.43:0/3811454236 conn(0x7fd4240ff8d0 msgr2=0x7fd424101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:33.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.076+0000 7fd423200640 1 -- 172.21.15.43:0/3811454236 shutdown_connections 2024-09-19T05:13:33.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:33.076+0000 7fd423200640 1 -- 172.21.15.43:0/3811454236 wait complete. 2024-09-19T05:13:33.962 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:13:34.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:33 smithi096 ceph-mon[35085]: pgmap v276: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:13:34.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:33 smithi043 ceph-mon[30800]: pgmap v276: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:13:34.343 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:13:34.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:33 smithi151 ceph-mon[35425]: pgmap v276: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:13:34.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.896+0000 7f984a400640 1 -- 172.21.15.43:0/4021767279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9844106f10 msgr2=0x7f984410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:34.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.896+0000 7f984a400640 1 --2- 172.21.15.43:0/4021767279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9844106f10 0x7f984410ea90 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f983c009ca0 tx=0x7f983c02a8c0 comp rx=0 tx=0).stop 2024-09-19T05:13:34.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.896+0000 7f984a400640 1 -- 172.21.15.43:0/4021767279 shutdown_connections 2024-09-19T05:13:34.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.896+0000 7f984a400640 1 --2- 172.21.15.43:0/4021767279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9844106f10 0x7f984410ea90 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:34.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.896+0000 7f984a400640 1 --2- 172.21.15.43:0/4021767279 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9844102790 0x7f98441067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:34.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.896+0000 7f984a400640 1 --2- 172.21.15.43:0/4021767279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9844101de0 0x7f98441021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:34.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.896+0000 7f984a400640 1 -- 172.21.15.43:0/4021767279 >> 172.21.15.43:0/4021767279 conn(0x7f98440fd8c0 msgr2=0x7f98440ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:34.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.896+0000 7f984a400640 1 -- 172.21.15.43:0/4021767279 shutdown_connections 2024-09-19T05:13:34.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.896+0000 7f984a400640 1 -- 172.21.15.43:0/4021767279 wait complete. 2024-09-19T05:13:34.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.897+0000 7f984a400640 1 Processor -- start 2024-09-19T05:13:34.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.897+0000 7f984a400640 1 -- start start 2024-09-19T05:13:34.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.898+0000 7f984a400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9844101de0 0x7f984419a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:34.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.898+0000 7f984a400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9844102790 0x7f984419ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:34.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.898+0000 7f984a400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9844106f10 0x7f984419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:34.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.898+0000 7f984a400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9844111fc0 con 0x7f9844101de0 2024-09-19T05:13:34.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.898+0000 7f984a400640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9844111e40 con 0x7f9844106f10 2024-09-19T05:13:34.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.898+0000 7f984a400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9844112140 con 0x7f9844102790 2024-09-19T05:13:34.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.898+0000 7f9843400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9844101de0 0x7f984419a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:34.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.898+0000 7f9843400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9844101de0 0x7f984419a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33228/0 (socket says 172.21.15.43:33228) 2024-09-19T05:13:34.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.898+0000 7f9843400640 1 -- 172.21.15.43:0/3478527947 learned_addr learned my addr 172.21.15.43:0/3478527947 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:34.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.898+0000 7f983aa00640 1 --2- 172.21.15.43:0/3478527947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9844102790 0x7f984419ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:34.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.898+0000 7f9843e00640 1 --2- 172.21.15.43:0/3478527947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9844106f10 0x7f984419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:34.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.899+0000 7f9843400640 1 -- 172.21.15.43:0/3478527947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9844102790 msgr2=0x7f984419ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:34.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.899+0000 7f9843400640 1 --2- 172.21.15.43:0/3478527947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9844102790 0x7f984419ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:34.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.899+0000 7f9843400640 1 -- 172.21.15.43:0/3478527947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9844106f10 msgr2=0x7f984419eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:34.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.899+0000 7f9843400640 1 --2- 172.21.15.43:0/3478527947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9844106f10 0x7f984419eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:34.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.899+0000 7f9843400640 1 -- 172.21.15.43:0/3478527947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f984419f590 con 0x7f9844101de0 2024-09-19T05:13:34.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.899+0000 7f9843400640 1 --2- 172.21.15.43:0/3478527947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9844101de0 0x7f984419a5e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f983400e9f0 tx=0x7f983400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:34.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.899+0000 7f9840c00640 1 -- 172.21.15.43:0/3478527947 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983400cd80 con 0x7f9844101de0 2024-09-19T05:13:34.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.899+0000 7f9840c00640 1 -- 172.21.15.43:0/3478527947 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9834010910 con 0x7f9844101de0 2024-09-19T05:13:34.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.899+0000 7f984a400640 1 -- 172.21.15.43:0/3478527947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f984419f880 con 0x7f9844101de0 2024-09-19T05:13:34.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.899+0000 7f9840c00640 1 -- 172.21.15.43:0/3478527947 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9834018990 con 0x7f9844101de0 2024-09-19T05:13:34.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.899+0000 7f984a400640 1 -- 172.21.15.43:0/3478527947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98441a7160 con 0x7f9844101de0 2024-09-19T05:13:34.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.902+0000 7f984ce6b740 1 Processor -- start 2024-09-19T05:13:34.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.903+0000 7f9840c00640 1 -- 172.21.15.43:0/3478527947 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f9834010ac0 con 0x7f9844101de0 2024-09-19T05:13:34.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.903+0000 7f984ce6b740 1 -- start start 2024-09-19T05:13:34.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.903+0000 7f984ce6b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560680ffbb90 0x560680ffbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:34.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.903+0000 7f984ce6b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560680ffc4b0 0x560681006830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:34.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.903+0000 7f984ce6b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560681006d70 0x560681009160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:34.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.903+0000 7f984ce6b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560680f623e0 con 0x560680ffc4b0 2024-09-19T05:13:34.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.903+0000 7f984ce6b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560680f09d80 con 0x560680ffbb90 2024-09-19T05:13:34.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.903+0000 7f984ce6b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560680f5e3a0 con 0x560681006d70 2024-09-19T05:13:34.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.903+0000 7f9843e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560680ffbb90 0x560680ffbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:34.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.903+0000 7f9843e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560680ffbb90 0x560680ffbf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:54874/0 (socket says 172.21.15.43:54874) 2024-09-19T05:13:34.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.903+0000 7f9843e00640 1 -- 172.21.15.43:0/2045145300 learned_addr learned my addr 172.21.15.43:0/2045145300 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:34.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.903+0000 7f9843400640 1 --2- 172.21.15.43:0/2045145300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560680ffc4b0 0x560681006830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:34.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.904+0000 7f983aa00640 1 --2- 172.21.15.43:0/2045145300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560681006d70 0x560681009160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:34.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.904+0000 7f9843e00640 1 -- 172.21.15.43:0/2045145300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560681006d70 msgr2=0x560681009160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:34.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.904+0000 7f9843e00640 1 --2- 172.21.15.43:0/2045145300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560681006d70 0x560681009160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:34.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.904+0000 7f9843e00640 1 -- 172.21.15.43:0/2045145300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560680ffc4b0 msgr2=0x560681006830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:34.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.904+0000 7f9843e00640 1 --2- 172.21.15.43:0/2045145300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560680ffc4b0 0x560681006830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:34.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.904+0000 7f9843e00640 1 -- 172.21.15.43:0/2045145300 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560680f28ae0 con 0x560680ffbb90 2024-09-19T05:13:34.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.904+0000 7f9843400640 1 --2- 172.21.15.43:0/2045145300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560680ffc4b0 0x560681006830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:13:34.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.904+0000 7f9843e00640 1 --2- 172.21.15.43:0/2045145300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560680ffbb90 0x560680ffbf70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f983c009ca0 tx=0x7f983c02f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fcdd822a71ad5ce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:34.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.905+0000 7f9840c00640 1 --2- 172.21.15.43:0/3478527947 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f98280a3430 0x7f98280a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:34.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.905+0000 7f9838c00640 1 -- 172.21.15.43:0/2045145300 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c003970 con 0x560680ffbb90 2024-09-19T05:13:34.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.905+0000 7f9840c00640 1 -- 172.21.15.43:0/3478527947 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f98340cffe0 con 0x7f9844101de0 2024-09-19T05:13:34.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.905+0000 7f9843400640 1 --2- 172.21.15.43:0/3478527947 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f98280a3430 0x7f98280a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:34.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.905+0000 7f9838c00640 1 -- 172.21.15.43:0/2045145300 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f983c036030 con 0x560680ffbb90 2024-09-19T05:13:34.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.905+0000 7f9838c00640 1 -- 172.21.15.43:0/2045145300 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c004330 con 0x560680ffbb90 2024-09-19T05:13:34.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.906+0000 7f984ce6b740 1 -- 172.21.15.43:0/2045145300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560680ffbb90 msgr2=0x560680ffbf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:34.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.906+0000 7f9843400640 1 --2- 172.21.15.43:0/3478527947 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f98280a3430 0x7f98280a58f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x560680ff8f80 tx=0x7f9834000bc0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:34.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.906+0000 7f984ce6b740 1 --2- 172.21.15.43:0/2045145300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560680ffbb90 0x560680ffbf70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f983c009ca0 tx=0x7f983c02f3c0 comp rx=0 tx=0).stop 2024-09-19T05:13:34.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.906+0000 7f984ce6b740 1 -- 172.21.15.43:0/2045145300 shutdown_connections 2024-09-19T05:13:34.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.906+0000 7f984ce6b740 1 --2- 172.21.15.43:0/2045145300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560681006d70 0x560681009160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:34.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.906+0000 7f984ce6b740 1 --2- 172.21.15.43:0/2045145300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560680ffc4b0 0x560681006830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:34.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.906+0000 7f984ce6b740 1 --2- 172.21.15.43:0/2045145300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560680ffbb90 0x560680ffbf70 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:34.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.906+0000 7f984ce6b740 1 -- 172.21.15.43:0/2045145300 >> 172.21.15.43:0/2045145300 conn(0x560680ff8120 msgr2=0x560680ff8500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:34.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.906+0000 7f984ce6b740 1 -- 172.21.15.43:0/2045145300 shutdown_connections 2024-09-19T05:13:34.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.906+0000 7f984ce6b740 1 -- 172.21.15.43:0/2045145300 wait complete. 2024-09-19T05:13:34.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.907+0000 7f984ce6b740 1 -- start start 2024-09-19T05:13:34.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.907+0000 7f984ce6b740 1 Processor -- start 2024-09-19T05:13:34.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.908+0000 7f984ce6b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560680ffbb90 0x560681003c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:34.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.908+0000 7f984ce6b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560680ffc4b0 0x5606810041a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:34.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.908+0000 7f984ce6b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560681006d70 0x5606810046e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:34.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.908+0000 7f984ce6b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560681009860 con 0x560681006d70 2024-09-19T05:13:34.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.908+0000 7f984ce6b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560680f1d0f0 con 0x560680ffbb90 2024-09-19T05:13:34.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.908+0000 7f984ce6b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5606810099e0 con 0x560680ffc4b0 2024-09-19T05:13:34.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.908+0000 7f984ce6b740 10 client.0 _ll_register_callbacks cb 0x560680f0e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:34.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.908+0000 7f983aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560680ffc4b0 0x5606810041a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:34.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.908+0000 7f9843e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560680ffbb90 0x560681003c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:34.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.908+0000 7f983aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560680ffc4b0 0x5606810041a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:42374/0 (socket says 172.21.15.43:42374) 2024-09-19T05:13:34.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.908+0000 7f983aa00640 1 -- 172.21.15.43:0/1736296268 learned_addr learned my addr 172.21.15.43:0/1736296268 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:34.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.908+0000 7f9843e00640 1 --2- 172.21.15.43:0/1736296268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560681006d70 0x5606810046e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:34.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.909+0000 7f983aa00640 1 -- 172.21.15.43:0/1736296268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560680ffbb90 msgr2=0x560681003c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:34.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.909+0000 7f983aa00640 1 --2- 172.21.15.43:0/1736296268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560680ffbb90 0x560681003c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:34.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.909+0000 7f983aa00640 1 -- 172.21.15.43:0/1736296268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560681006d70 msgr2=0x5606810046e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:34.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.909+0000 7f983aa00640 1 --2- 172.21.15.43:0/1736296268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560681006d70 0x5606810046e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:34.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.909+0000 7f983aa00640 1 -- 172.21.15.43:0/1736296268 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560680cd33c0 con 0x560680ffc4b0 2024-09-19T05:13:34.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.909+0000 7f983aa00640 1 --2- 172.21.15.43:0/1736296268 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560680ffc4b0 0x5606810041a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f982c00be80 tx=0x7f982c012820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32ff263f4e544477 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:34.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.909+0000 7f9831600640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:13:34.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.910+0000 7f9831600640 1 -- 172.21.15.43:0/1736296268 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982c004220 con 0x560680ffc4b0 2024-09-19T05:13:34.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.910+0000 7f984ce6b740 1 -- 172.21.15.43:0/1736296268 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f983c036030 con 0x560680ffc4b0 2024-09-19T05:13:34.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.910+0000 7f9831600640 1 -- 172.21.15.43:0/1736296268 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f982c0043c0 con 0x560680ffc4b0 2024-09-19T05:13:34.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.913+0000 7f9831600640 1 -- 172.21.15.43:0/1736296268 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f982c010710 con 0x560680ffc4b0 2024-09-19T05:13:34.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.913+0000 7f984ce6b740 10 client.27271 fetch_fsmap learned FSMap version 29 2024-09-19T05:13:34.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.913+0000 7f984ce6b740 1 -- 172.21.15.43:0/1736296268 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5606810155c0 con 0x560680ffc4b0 2024-09-19T05:13:34.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.913+0000 7f9831600640 1 -- 172.21.15.43:0/1736296268 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f983c036030 con 0x560680ffc4b0 2024-09-19T05:13:34.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.914+0000 7f9831600640 1 -- 172.21.15.43:0/1736296268 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f982c003c20 con 0x560680ffc4b0 2024-09-19T05:13:34.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.914+0000 7f984ce6b740 10 client.27271 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:13:34.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.914+0000 7f984ce6b740 10 client.27271 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:13:34.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.914+0000 7f984ce6b740 20 client.27271 populate_metadata read hostname 'smithi043' 2024-09-19T05:13:34.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.914+0000 7f984ce6b740 1 --2- 172.21.15.43:0/1736296268 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x560681016b20 0x560681018f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:34.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.914+0000 7f984ce6b740 4 client.27271 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:13:34.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.914+0000 7f984ce6b740 1 -- 172.21.15.43:0/1736296268 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560680f6f950 con 0x560681016b20 2024-09-19T05:13:34.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.914+0000 7f9843e00640 1 --2- 172.21.15.43:0/1736296268 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x560681016b20 0x560681018f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:34.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.915+0000 7f9843e00640 1 --2- 172.21.15.43:0/1736296268 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x560681016b20 0x560681018f40 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44b9b56f65878c2d server_cookie=3989e044af9ee144 in_seq=0 out_seq=0 2024-09-19T05:13:34.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.915+0000 7f9831600640 10 client.27271 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:13:34.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.915+0000 7f9831600640 1 -- 172.21.15.43:0/1736296268 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560680f6f950 con 0x560681016b20 2024-09-19T05:13:34.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.915+0000 7f9831600640 10 client.27271 handle_command_reply: tid=0 2024-09-19T05:13:34.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.915+0000 7f984ce6b740 1 client.27271 shutdown 2024-09-19T05:13:34.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.916+0000 7f984ce6b740 1 -- 172.21.15.43:0/1736296268 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560680ffc4b0 msgr2=0x5606810041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:34.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.916+0000 7f984ce6b740 1 --2- 172.21.15.43:0/1736296268 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560680ffc4b0 0x5606810041a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f982c00be80 tx=0x7f982c012820 comp rx=0 tx=0).stop 2024-09-19T05:13:34.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.916+0000 7f984ce6b740 1 -- 172.21.15.43:0/1736296268 shutdown_connections 2024-09-19T05:13:34.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.916+0000 7f984ce6b740 1 --2- 172.21.15.43:0/1736296268 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x560681016b20 0x560681018f40 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:34.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.916+0000 7f984ce6b740 1 --2- 172.21.15.43:0/1736296268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560681006d70 0x5606810046e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:34.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.916+0000 7f984ce6b740 1 --2- 172.21.15.43:0/1736296268 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560680ffc4b0 0x5606810041a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:34.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.916+0000 7f984ce6b740 1 --2- 172.21.15.43:0/1736296268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560680ffbb90 0x560681003c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:34.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.916+0000 7f9831600640 0 client.27271 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:13:34.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.916+0000 7f984ce6b740 1 -- 172.21.15.43:0/1736296268 >> 172.21.15.43:0/1736296268 conn(0x560680ff8120 msgr2=0x560680ffde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:34.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.916+0000 7f984ce6b740 1 -- 172.21.15.43:0/1736296268 shutdown_connections 2024-09-19T05:13:34.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.916+0000 7f984ce6b740 1 -- 172.21.15.43:0/1736296268 wait complete. 2024-09-19T05:13:34.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.916+0000 7f984ce6b740 20 client.27271 trim_cache size 0 max 16384 2024-09-19T05:13:34.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.934+0000 7f984ce6b740 1 Processor -- start 2024-09-19T05:13:34.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.934+0000 7f984ce6b740 1 -- start start 2024-09-19T05:13:34.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.934+0000 7f984ce6b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560680ffbb90 0x560681004530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:34.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.934+0000 7f984ce6b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560680ffc4b0 0x56068101e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:34.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.934+0000 7f984ce6b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560681006d70 0x56068101ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:34.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.934+0000 7f984ce6b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f98441afa90 con 0x560680ffc4b0 2024-09-19T05:13:34.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.934+0000 7f984ce6b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f98441b15d0 con 0x560681006d70 2024-09-19T05:13:34.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.934+0000 7f984ce6b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f98441b0160 con 0x560680ffbb90 2024-09-19T05:13:34.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.935+0000 7f983aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560680ffc4b0 0x56068101e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:34.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.935+0000 7f983aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560680ffc4b0 0x56068101e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33254/0 (socket says 172.21.15.43:33254) 2024-09-19T05:13:34.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.935+0000 7f983aa00640 1 -- 172.21.15.43:0/3214703823 learned_addr learned my addr 172.21.15.43:0/3214703823 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:34.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.935+0000 7f9843e00640 1 --2- 172.21.15.43:0/3214703823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560680ffbb90 0x560681004530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:34.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.935+0000 7f9843e00640 1 --2- 172.21.15.43:0/3214703823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560681006d70 0x56068101ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:34.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.935+0000 7f983aa00640 1 -- 172.21.15.43:0/3214703823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560680ffbb90 msgr2=0x560681004530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:34.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.935+0000 7f983aa00640 1 --2- 172.21.15.43:0/3214703823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560680ffbb90 0x560681004530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:34.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.935+0000 7f983aa00640 1 -- 172.21.15.43:0/3214703823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560681006d70 msgr2=0x56068101ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:34.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.935+0000 7f983aa00640 1 --2- 172.21.15.43:0/3214703823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560681006d70 0x56068101ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:34.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.935+0000 7f983aa00640 1 -- 172.21.15.43:0/3214703823 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560680fff8c0 con 0x560680ffc4b0 2024-09-19T05:13:34.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.935+0000 7f9843e00640 1 --2- 172.21.15.43:0/3214703823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560680ffbb90 0x560681004530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:13:34.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.936+0000 7f983aa00640 1 --2- 172.21.15.43:0/3214703823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560680ffc4b0 0x56068101e7d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f982c024680 tx=0x7f982c00b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c965adf33f808ad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:34.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.936+0000 7f9833e00640 1 -- 172.21.15.43:0/3214703823 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982c004220 con 0x560680ffc4b0 2024-09-19T05:13:34.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.936+0000 7f9833e00640 1 -- 172.21.15.43:0/3214703823 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f982c0043c0 con 0x560680ffc4b0 2024-09-19T05:13:34.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.936+0000 7f9833e00640 1 -- 172.21.15.43:0/3214703823 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982c005630 con 0x560680ffc4b0 2024-09-19T05:13:34.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.937+0000 7f984ce6b740 1 -- 172.21.15.43:0/3214703823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560680ffc4b0 msgr2=0x56068101e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:34.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.937+0000 7f984ce6b740 1 --2- 172.21.15.43:0/3214703823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560680ffc4b0 0x56068101e7d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f982c024680 tx=0x7f982c00b710 comp rx=0 tx=0).stop 2024-09-19T05:13:34.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.937+0000 7f984ce6b740 1 -- 172.21.15.43:0/3214703823 shutdown_connections 2024-09-19T05:13:34.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.937+0000 7f984ce6b740 1 --2- 172.21.15.43:0/3214703823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560681006d70 0x56068101ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:34.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.937+0000 7f984ce6b740 1 --2- 172.21.15.43:0/3214703823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560680ffc4b0 0x56068101e7d0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:34.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.937+0000 7f984ce6b740 1 --2- 172.21.15.43:0/3214703823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560680ffbb90 0x560681004530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:34.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.937+0000 7f984ce6b740 1 -- 172.21.15.43:0/3214703823 >> 172.21.15.43:0/3214703823 conn(0x560680ff8120 msgr2=0x560681005ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:34.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.937+0000 7f984ce6b740 1 -- 172.21.15.43:0/3214703823 shutdown_connections 2024-09-19T05:13:34.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.937+0000 7f984ce6b740 1 -- 172.21.15.43:0/3214703823 wait complete. 2024-09-19T05:13:34.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.937+0000 7f984ce6b740 1 -- start start 2024-09-19T05:13:34.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.937+0000 7f984ce6b740 1 Processor -- start 2024-09-19T05:13:34.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.938+0000 7f984ce6b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560680ffbb90 0x56068100ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:34.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.938+0000 7f984ce6b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560680ffc4b0 0x56068100f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:34.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.938+0000 7f984ce6b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560681006d70 0x56068100bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:34.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.938+0000 7f984ce6b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f98441b0f00 con 0x560680ffc4b0 2024-09-19T05:13:34.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.938+0000 7f984ce6b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f98441b0830 con 0x560681006d70 2024-09-19T05:13:34.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.938+0000 7f984ce6b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f98441ada90 con 0x560680ffbb90 2024-09-19T05:13:34.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.938+0000 7f984ce6b740 10 client.0 _ll_register_callbacks cb 0x7f982c00b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:13:34.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.938+0000 7f983aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560680ffc4b0 0x56068100f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:34.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.938+0000 7f983aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560680ffc4b0 0x56068100f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33262/0 (socket says 172.21.15.43:33262) 2024-09-19T05:13:34.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.938+0000 7f983aa00640 1 -- 172.21.15.43:0/2185299570 learned_addr learned my addr 172.21.15.43:0/2185299570 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:13:34.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.938+0000 7f9843e00640 1 --2- 172.21.15.43:0/2185299570 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560680ffbb90 0x56068100ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:34.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.938+0000 7f9843e00640 1 --2- 172.21.15.43:0/2185299570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560681006d70 0x56068100bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:34.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.939+0000 7f983aa00640 1 -- 172.21.15.43:0/2185299570 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560680ffbb90 msgr2=0x56068100ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:34.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.939+0000 7f983aa00640 1 --2- 172.21.15.43:0/2185299570 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560680ffbb90 0x56068100ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:34.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.939+0000 7f983aa00640 1 -- 172.21.15.43:0/2185299570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560681006d70 msgr2=0x56068100bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:13:34.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.939+0000 7f983aa00640 1 --2- 172.21.15.43:0/2185299570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560681006d70 0x56068100bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:34.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.939+0000 7f983aa00640 1 -- 172.21.15.43:0/2185299570 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56068100c170 con 0x560680ffc4b0 2024-09-19T05:13:34.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.939+0000 7f9843e00640 1 --2- 172.21.15.43:0/2185299570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560681006d70 0x56068100bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:13:34.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.939+0000 7f983aa00640 1 --2- 172.21.15.43:0/2185299570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560680ffc4b0 0x56068100f170 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f982c024490 tx=0x7f982c0055e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f24a826c2c0e59 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:13:34.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.939+0000 7f9831600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:13:34.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.939+0000 7f9831600640 1 -- 172.21.15.43:0/2185299570 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982c03a060 con 0x560680ffc4b0 2024-09-19T05:13:34.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.939+0000 7f9831600640 1 -- 172.21.15.43:0/2185299570 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f982c010cc0 con 0x560680ffc4b0 2024-09-19T05:13:34.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.939+0000 7f984ce6b740 1 -- 172.21.15.43:0/2185299570 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f982c0043c0 con 0x560680ffc4b0 2024-09-19T05:13:34.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.940+0000 7f9831600640 1 -- 172.21.15.43:0/2185299570 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f982c0246b0 con 0x560680ffc4b0 2024-09-19T05:13:34.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.940+0000 7f984ce6b740 10 client.17358 fetch_fsmap learned FSMap version 29 2024-09-19T05:13:34.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.940+0000 7f984ce6b740 1 -- 172.21.15.43:0/2185299570 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560681082000 con 0x560680ffc4b0 2024-09-19T05:13:34.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.940+0000 7f9831600640 1 -- 172.21.15.43:0/2185299570 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f982c0043c0 con 0x560680ffc4b0 2024-09-19T05:13:34.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.940+0000 7f9831600640 1 -- 172.21.15.43:0/2185299570 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f982c02b070 con 0x560680ffc4b0 2024-09-19T05:13:34.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.940+0000 7f984ce6b740 10 client.17358 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:13:34.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.940+0000 7f984ce6b740 10 client.17358 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:13:34.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.940+0000 7f984ce6b740 20 client.17358 populate_metadata read hostname 'smithi043' 2024-09-19T05:13:34.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.940+0000 7f984ce6b740 1 --2- 172.21.15.43:0/2185299570 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x560681082630 0x560681084a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:13:34.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.940+0000 7f984ce6b740 4 client.17358 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:13:34.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.940+0000 7f984ce6b740 1 -- 172.21.15.43:0/2185299570 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560680f69e60 con 0x560681082630 2024-09-19T05:13:34.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.940+0000 7f9843e00640 1 --2- 172.21.15.43:0/2185299570 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x560681082630 0x560681084a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:13:34.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.941+0000 7f9843e00640 1 --2- 172.21.15.43:0/2185299570 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x560681082630 0x560681084a50 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3dc93625d31ce40 server_cookie=26121c5b45bf6da7 in_seq=0 out_seq=0 2024-09-19T05:13:34.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.941+0000 7f9831600640 10 client.17358 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:13:34.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.941+0000 7f9831600640 1 -- 172.21.15.43:0/2185299570 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560680f69e60 con 0x560681082630 2024-09-19T05:13:34.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.941+0000 7f9831600640 10 client.17358 handle_command_reply: tid=0 2024-09-19T05:13:34.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.941+0000 7f984ce6b740 1 client.17358 shutdown 2024-09-19T05:13:34.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.941+0000 7f984ce6b740 1 -- 172.21.15.43:0/2185299570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560680ffc4b0 msgr2=0x56068100f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:34.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.941+0000 7f984ce6b740 1 --2- 172.21.15.43:0/2185299570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560680ffc4b0 0x56068100f170 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f982c024490 tx=0x7f982c0055e0 comp rx=0 tx=0).stop 2024-09-19T05:13:34.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.941+0000 7f984ce6b740 1 -- 172.21.15.43:0/2185299570 shutdown_connections 2024-09-19T05:13:34.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.941+0000 7f984ce6b740 1 --2- 172.21.15.43:0/2185299570 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x560681082630 0x560681084a50 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:34.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.941+0000 7f984ce6b740 1 --2- 172.21.15.43:0/2185299570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560681006d70 0x56068100bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:34.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.941+0000 7f984ce6b740 1 --2- 172.21.15.43:0/2185299570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560680ffc4b0 0x56068100f170 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:34.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.941+0000 7f984ce6b740 1 --2- 172.21.15.43:0/2185299570 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560680ffbb90 0x56068100ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:34.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.941+0000 7f984ce6b740 1 -- 172.21.15.43:0/2185299570 >> 172.21.15.43:0/2185299570 conn(0x560680ff8120 msgr2=0x560680ff69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:34.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.941+0000 7f9831600640 0 client.17358 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:13:34.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.942+0000 7f984ce6b740 1 -- 172.21.15.43:0/2185299570 shutdown_connections 2024-09-19T05:13:34.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.942+0000 7f984ce6b740 1 -- 172.21.15.43:0/2185299570 wait complete. 2024-09-19T05:13:34.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.942+0000 7f984ce6b740 20 client.17358 trim_cache size 0 max 16384 2024-09-19T05:13:34.947 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:13:34.947 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T05:13:34.947 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-09-19T05:13:34.947 INFO:teuthology.orchestra.run.smithi043.stdout: "7ed96aac-c314-445c-983a-4a09305649f1": { 2024-09-19T05:13:34.947 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-09-19T05:13:34.947 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "7ed96aac-c314-445c-983a-4a09305649f1", 2024-09-19T05:13:34.947 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-09-19T05:13:34.947 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:13:34.947 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:13:34.948 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:13:34.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.942+0000 7f984a400640 1 -- 172.21.15.43:0/3478527947 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f98280a3430 msgr2=0x7f98280a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:34.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.942+0000 7f984a400640 1 --2- 172.21.15.43:0/3478527947 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f98280a3430 0x7f98280a58f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x560680ff8f80 tx=0x7f9834000bc0 comp rx=0 tx=0).stop 2024-09-19T05:13:34.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.942+0000 7f984a400640 1 -- 172.21.15.43:0/3478527947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9844101de0 msgr2=0x7f984419a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:13:34.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.942+0000 7f984a400640 1 --2- 172.21.15.43:0/3478527947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9844101de0 0x7f984419a5e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f983400e9f0 tx=0x7f983400eeb0 comp rx=0 tx=0).stop 2024-09-19T05:13:34.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.942+0000 7f984a400640 1 -- 172.21.15.43:0/3478527947 shutdown_connections 2024-09-19T05:13:34.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.942+0000 7f984a400640 1 --2- 172.21.15.43:0/3478527947 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f98280a3430 0x7f98280a58f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:34.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.942+0000 7f984a400640 1 --2- 172.21.15.43:0/3478527947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9844106f10 0x7f984419eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:34.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.942+0000 7f984a400640 1 --2- 172.21.15.43:0/3478527947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9844102790 0x7f984419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:34.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.942+0000 7f984a400640 1 --2- 172.21.15.43:0/3478527947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9844101de0 0x7f984419a5e0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:13:34.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.942+0000 7f984a400640 1 -- 172.21.15.43:0/3478527947 >> 172.21.15.43:0/3478527947 conn(0x7f98440fd8c0 msgr2=0x7f98440fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:13:34.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.943+0000 7f984a400640 1 -- 172.21.15.43:0/3478527947 shutdown_connections 2024-09-19T05:13:34.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:13:34.943+0000 7f984a400640 1 -- 172.21.15.43:0/3478527947 wait complete. 2024-09-19T05:13:34.950 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:34 smithi043 ceph-mon[30800]: scrub queued for path: / 2024-09-19T05:13:34.950 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:34 smithi043 ceph-mon[30800]: scrub summary: idle+waiting paths [/] 2024-09-19T05:13:34.950 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:34 smithi043 ceph-mon[30800]: scrub summary: active paths [/] 2024-09-19T05:13:34.950 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:34 smithi043 ceph-mon[30800]: scrub summary: idle+waiting paths [/] 2024-09-19T05:13:35.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:34 smithi096 ceph-mon[35085]: scrub queued for path: / 2024-09-19T05:13:35.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:34 smithi096 ceph-mon[35085]: scrub summary: idle+waiting paths [/] 2024-09-19T05:13:35.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:34 smithi096 ceph-mon[35085]: scrub summary: active paths [/] 2024-09-19T05:13:35.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:34 smithi096 ceph-mon[35085]: scrub summary: idle+waiting paths [/] 2024-09-19T05:13:35.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:34 smithi151 ceph-mon[35425]: scrub queued for path: / 2024-09-19T05:13:35.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:34 smithi151 ceph-mon[35425]: scrub summary: idle+waiting paths [/] 2024-09-19T05:13:35.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:34 smithi151 ceph-mon[35425]: scrub summary: active paths [/] 2024-09-19T05:13:35.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:34 smithi151 ceph-mon[35425]: scrub summary: idle+waiting paths [/] 2024-09-19T05:13:35.821 INFO:tasks.cephfs.filesystem:scrub status for tag:7ed96aac-c314-445c-983a-4a09305649f1 - {'path': '/', 'tag': '7ed96aac-c314-445c-983a-4a09305649f1', 'options': 'recursive,force'} 2024-09-19T05:13:36.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:35 smithi096 ceph-mon[35085]: pgmap v277: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:13:36.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:35 smithi043 ceph-mon[30800]: pgmap v277: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:13:36.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:35 smithi151 ceph-mon[35425]: pgmap v277: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:13:38.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:37 smithi096 ceph-mon[35085]: pgmap v278: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:13:38.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:37 smithi043 ceph-mon[30800]: pgmap v278: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:13:38.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:37 smithi151 ceph-mon[35425]: pgmap v278: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:13:39.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:38 smithi096 ceph-mon[35085]: scrub summary: idle 2024-09-19T05:13:39.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:38 smithi043 ceph-mon[30800]: scrub summary: idle 2024-09-19T05:13:39.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:38 smithi151 ceph-mon[35425]: scrub summary: idle 2024-09-19T05:13:40.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:39 smithi096 ceph-mon[35085]: pgmap v279: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:13:40.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:39 smithi043 ceph-mon[30800]: pgmap v279: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:13:40.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:39 smithi151 ceph-mon[35425]: pgmap v279: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:13:42.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:41 smithi096 ceph-mon[35085]: pgmap v280: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:13:42.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:41 smithi043 ceph-mon[30800]: pgmap v280: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:13:42.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:41 smithi151 ceph-mon[35425]: pgmap v280: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:13:44.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:43 smithi096 ceph-mon[35085]: pgmap v281: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:13:44.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:43 smithi043 ceph-mon[30800]: pgmap v281: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:13:44.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:43 smithi151 ceph-mon[35425]: pgmap v281: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:13:46.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:45 smithi096 ceph-mon[35085]: pgmap v282: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:13:46.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:45 smithi043 ceph-mon[30800]: pgmap v282: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:13:46.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:45 smithi151 ceph-mon[35425]: pgmap v282: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:13:48.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:47 smithi096 ceph-mon[35085]: pgmap v283: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:13:48.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:47 smithi043 ceph-mon[30800]: pgmap v283: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:13:48.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:47 smithi151 ceph-mon[35425]: pgmap v283: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:13:50.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:49 smithi096 ceph-mon[35085]: pgmap v284: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:13:50.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:49 smithi043 ceph-mon[30800]: pgmap v284: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:13:50.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:49 smithi151 ceph-mon[35425]: pgmap v284: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:13:52.225 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:51 smithi096 ceph-mon[35085]: pgmap v285: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:13:52.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:51 smithi043 ceph-mon[30800]: pgmap v285: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:13:52.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:51 smithi151 ceph-mon[35425]: pgmap v285: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:13:54.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:53 smithi096 ceph-mon[35085]: pgmap v286: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:13:54.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:53 smithi043 ceph-mon[30800]: pgmap v286: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:13:54.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:53 smithi151 ceph-mon[35425]: pgmap v286: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:13:56.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:55 smithi096 ceph-mon[35085]: pgmap v287: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:13:56.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:55 smithi043 ceph-mon[30800]: pgmap v287: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:13:56.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:55 smithi151 ceph-mon[35425]: pgmap v287: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:13:58.207 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:57 smithi151 ceph-mon[35425]: pgmap v288: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-19T05:13:58.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:57 smithi096 ceph-mon[35085]: pgmap v288: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-19T05:13:58.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:57 smithi043 ceph-mon[30800]: pgmap v288: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-19T05:14:00.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:13:59 smithi096 ceph-mon[35085]: pgmap v289: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-19T05:14:00.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:13:59 smithi043 ceph-mon[30800]: pgmap v289: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-19T05:14:00.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:13:59 smithi151 ceph-mon[35425]: pgmap v289: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-19T05:14:01.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:00 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:14:01.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:00 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:14:01.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:00 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:14:02.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:01 smithi096 ceph-mon[35085]: pgmap v290: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:14:02.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:01 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:14:02.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:01 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:14:02.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:01 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:14:02.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:01 smithi043 ceph-mon[30800]: pgmap v290: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:14:02.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:01 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:14:02.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:01 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:14:02.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:01 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:14:02.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:01 smithi151 ceph-mon[35425]: pgmap v290: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:14:02.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:01 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:14:02.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:01 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:14:02.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:01 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:14:04.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:03 smithi096 ceph-mon[35085]: pgmap v291: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-19T05:14:04.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:03 smithi043 ceph-mon[30800]: pgmap v291: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-19T05:14:04.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:03 smithi151 ceph-mon[35425]: pgmap v291: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-19T05:14:05.822 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:14:06.215 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:14:06.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:05 smithi043 ceph-mon[30800]: pgmap v292: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-09-19T05:14:06.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:05 smithi151 ceph-mon[35425]: pgmap v292: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-09-19T05:14:06.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:05 smithi096 ceph-mon[35085]: pgmap v292: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-09-19T05:14:06.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.839+0000 7f1893200640 1 -- 172.21.15.43:0/3233959248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f188c112a30 msgr2=0x7f188c114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:06.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.839+0000 7f1893200640 1 --2- 172.21.15.43:0/3233959248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f188c112a30 0x7f188c114e20 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f1884009ca0 tx=0x7f188402a8c0 comp rx=0 tx=0).stop 2024-09-19T05:14:06.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.839+0000 7f1893200640 1 -- 172.21.15.43:0/3233959248 shutdown_connections 2024-09-19T05:14:06.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.839+0000 7f1893200640 1 --2- 172.21.15.43:0/3233959248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f188c112a30 0x7f188c114e20 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:06.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.839+0000 7f1893200640 1 --2- 172.21.15.43:0/3233959248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f188c077040 0x7f188c1124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:06.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.839+0000 7f1893200640 1 --2- 172.21.15.43:0/3233959248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f188c076720 0x7f188c076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:06.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.839+0000 7f1893200640 1 -- 172.21.15.43:0/3233959248 >> 172.21.15.43:0/3233959248 conn(0x7f188c0ffa90 msgr2=0x7f188c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:06.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.839+0000 7f1893200640 1 -- 172.21.15.43:0/3233959248 shutdown_connections 2024-09-19T05:14:06.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.839+0000 7f1893200640 1 -- 172.21.15.43:0/3233959248 wait complete. 2024-09-19T05:14:06.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.840+0000 7f1893200640 1 Processor -- start 2024-09-19T05:14:06.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.840+0000 7f1893200640 1 -- start start 2024-09-19T05:14:06.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.841+0000 7f1893200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f188c076720 0x7f188c1a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:06.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.841+0000 7f1893200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f188c077040 0x7f188c1a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:06.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.841+0000 7f1893200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f188c112a30 0x7f188c19e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:06.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.841+0000 7f1893200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f188c1185c0 con 0x7f188c077040 2024-09-19T05:14:06.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.841+0000 7f1893200640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f188c118440 con 0x7f188c112a30 2024-09-19T05:14:06.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.841+0000 7f1893200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f188c118740 con 0x7f188c076720 2024-09-19T05:14:06.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.841+0000 7f188b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f188c077040 0x7f188c1a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:06.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.841+0000 7f188b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f188c077040 0x7f188c1a4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49912/0 (socket says 172.21.15.43:49912) 2024-09-19T05:14:06.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.841+0000 7f188b400640 1 -- 172.21.15.43:0/3137392462 learned_addr learned my addr 172.21.15.43:0/3137392462 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:06.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.841+0000 7f188be00640 1 --2- 172.21.15.43:0/3137392462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f188c076720 0x7f188c1a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:06.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.841+0000 7f1891200640 1 --2- 172.21.15.43:0/3137392462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f188c112a30 0x7f188c19e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:06.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.841+0000 7f188b400640 1 -- 172.21.15.43:0/3137392462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f188c076720 msgr2=0x7f188c1a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:06.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.841+0000 7f188b400640 1 --2- 172.21.15.43:0/3137392462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f188c076720 0x7f188c1a46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:06.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.841+0000 7f188b400640 1 -- 172.21.15.43:0/3137392462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f188c112a30 msgr2=0x7f188c19e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:06.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.841+0000 7f188b400640 1 --2- 172.21.15.43:0/3137392462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f188c112a30 0x7f188c19e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:06.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.841+0000 7f188b400640 1 -- 172.21.15.43:0/3137392462 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f188c19efd0 con 0x7f188c077040 2024-09-19T05:14:06.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.842+0000 7f188b400640 1 --2- 172.21.15.43:0/3137392462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f188c077040 0x7f188c1a4bf0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f187800e9f0 tx=0x7f187800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:06.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.842+0000 7f1888c00640 1 -- 172.21.15.43:0/3137392462 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187800cd80 con 0x7f188c077040 2024-09-19T05:14:06.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.842+0000 7f1888c00640 1 -- 172.21.15.43:0/3137392462 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1878010910 con 0x7f188c077040 2024-09-19T05:14:06.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.842+0000 7f1893200640 1 -- 172.21.15.43:0/3137392462 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f188c19f2c0 con 0x7f188c077040 2024-09-19T05:14:06.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.842+0000 7f1893200640 1 -- 172.21.15.43:0/3137392462 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f188c1ab510 con 0x7f188c077040 2024-09-19T05:14:06.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.842+0000 7f1888c00640 1 -- 172.21.15.43:0/3137392462 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1878018990 con 0x7f188c077040 2024-09-19T05:14:06.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.845+0000 7f1895c16740 1 Processor -- start 2024-09-19T05:14:06.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.845+0000 7f1895c16740 1 -- start start 2024-09-19T05:14:06.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.846+0000 7f1895c16740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5cd05bb90 0x55e5cd05bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:06.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.846+0000 7f1888c00640 1 -- 172.21.15.43:0/3137392462 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f1878010ac0 con 0x7f188c077040 2024-09-19T05:14:06.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.846+0000 7f1895c16740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5cd05c4b0 0x55e5cd066830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:06.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.846+0000 7f1895c16740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5cd066d70 0x55e5cd069160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:06.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.846+0000 7f1895c16740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e5ccfc23e0 con 0x55e5cd05c4b0 2024-09-19T05:14:06.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.846+0000 7f1895c16740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e5ccf69d80 con 0x55e5cd066d70 2024-09-19T05:14:06.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.846+0000 7f1895c16740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e5ccfbe3a0 con 0x55e5cd05bb90 2024-09-19T05:14:06.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.846+0000 7f1891200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5cd05bb90 0x55e5cd05bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:06.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.846+0000 7f188be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5cd05c4b0 0x55e5cd066830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:06.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.846+0000 7f1891200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5cd05bb90 0x55e5cd05bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:59362/0 (socket says 172.21.15.43:59362) 2024-09-19T05:14:06.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.846+0000 7f1891200640 1 -- 172.21.15.43:0/2837056965 learned_addr learned my addr 172.21.15.43:0/2837056965 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:06.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.846+0000 7f188b400640 1 --2- 172.21.15.43:0/2837056965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5cd066d70 0x55e5cd069160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:06.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.846+0000 7f1891200640 1 -- 172.21.15.43:0/2837056965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5cd066d70 msgr2=0x55e5cd069160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:06.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.846+0000 7f1891200640 1 --2- 172.21.15.43:0/2837056965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5cd066d70 0x55e5cd069160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:06.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.846+0000 7f1891200640 1 -- 172.21.15.43:0/2837056965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5cd05c4b0 msgr2=0x55e5cd066830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:06.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.846+0000 7f1891200640 1 --2- 172.21.15.43:0/2837056965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5cd05c4b0 0x55e5cd066830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:06.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.846+0000 7f1891200640 1 -- 172.21.15.43:0/2837056965 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5ccf88ae0 con 0x55e5cd05bb90 2024-09-19T05:14:06.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.847+0000 7f1891200640 1 --2- 172.21.15.43:0/2837056965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5cd05bb90 0x55e5cd05bf70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f188c07aa10 tx=0x7f1884005f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df606208640a2593 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:06.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.847+0000 7f1881600640 1 -- 172.21.15.43:0/2837056965 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1884031db0 con 0x55e5cd05bb90 2024-09-19T05:14:06.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.847+0000 7f1881600640 1 -- 172.21.15.43:0/2837056965 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1884036030 con 0x55e5cd05bb90 2024-09-19T05:14:06.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.847+0000 7f1881600640 1 -- 172.21.15.43:0/2837056965 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18840042a0 con 0x55e5cd05bb90 2024-09-19T05:14:06.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.847+0000 7f1888c00640 1 --2- 172.21.15.43:0/3137392462 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f18700a33b0 0x7f18700a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:06.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.848+0000 7f1888c00640 1 -- 172.21.15.43:0/3137392462 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f18780cfff0 con 0x7f188c077040 2024-09-19T05:14:06.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.848+0000 7f188be00640 1 --2- 172.21.15.43:0/3137392462 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f18700a33b0 0x7f18700a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:06.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.848+0000 7f1895c16740 1 -- 172.21.15.43:0/2837056965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5cd05bb90 msgr2=0x55e5cd05bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:06.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.848+0000 7f1895c16740 1 --2- 172.21.15.43:0/2837056965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5cd05bb90 0x55e5cd05bf70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f188c07aa10 tx=0x7f1884005f80 comp rx=0 tx=0).stop 2024-09-19T05:14:06.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.848+0000 7f1895c16740 1 -- 172.21.15.43:0/2837056965 shutdown_connections 2024-09-19T05:14:06.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.848+0000 7f1895c16740 1 --2- 172.21.15.43:0/2837056965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5cd066d70 0x55e5cd069160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:06.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.848+0000 7f1895c16740 1 --2- 172.21.15.43:0/2837056965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5cd05c4b0 0x55e5cd066830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:06.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.848+0000 7f1895c16740 1 --2- 172.21.15.43:0/2837056965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5cd05bb90 0x55e5cd05bf70 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:06.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.848+0000 7f1895c16740 1 -- 172.21.15.43:0/2837056965 >> 172.21.15.43:0/2837056965 conn(0x55e5cd058120 msgr2=0x55e5cd058500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:06.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.848+0000 7f1895c16740 1 -- 172.21.15.43:0/2837056965 shutdown_connections 2024-09-19T05:14:06.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.848+0000 7f1895c16740 1 -- 172.21.15.43:0/2837056965 wait complete. 2024-09-19T05:14:06.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.849+0000 7f188be00640 1 --2- 172.21.15.43:0/3137392462 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f18700a33b0 0x7f18700a5870 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55e5cd058f80 tx=0x7f187c005f90 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:06.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.849+0000 7f1895c16740 1 -- start start 2024-09-19T05:14:06.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.849+0000 7f1895c16740 1 Processor -- start 2024-09-19T05:14:06.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.850+0000 7f1895c16740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5cd05bb90 0x55e5cd063c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:06.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.850+0000 7f1895c16740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5cd05c4b0 0x55e5cd064150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:06.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.850+0000 7f1895c16740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5cd066d70 0x55e5cd064690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:06.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.850+0000 7f1895c16740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e5cd069860 con 0x55e5cd05bb90 2024-09-19T05:14:06.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.850+0000 7f1895c16740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e5ccf7d0f0 con 0x55e5cd066d70 2024-09-19T05:14:06.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.850+0000 7f1895c16740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e5cd0699e0 con 0x55e5cd05c4b0 2024-09-19T05:14:06.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.850+0000 7f1895c16740 10 client.0 _ll_register_callbacks cb 0x55e5ccf6e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:06.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.850+0000 7f188be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5cd05bb90 0x55e5cd063c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:06.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.850+0000 7f188b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5cd05c4b0 0x55e5cd064150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:06.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.850+0000 7f188be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5cd05bb90 0x55e5cd063c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49936/0 (socket says 172.21.15.43:49936) 2024-09-19T05:14:06.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.850+0000 7f188be00640 1 -- 172.21.15.43:0/3844872498 learned_addr learned my addr 172.21.15.43:0/3844872498 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:06.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.850+0000 7f1891200640 1 --2- 172.21.15.43:0/3844872498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5cd066d70 0x55e5cd064690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:06.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.850+0000 7f188b400640 1 -- 172.21.15.43:0/3844872498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5cd066d70 msgr2=0x55e5cd064690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:06.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.850+0000 7f188b400640 1 --2- 172.21.15.43:0/3844872498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5cd066d70 0x55e5cd064690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:06.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.850+0000 7f188b400640 1 -- 172.21.15.43:0/3844872498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5cd05bb90 msgr2=0x55e5cd063c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:06.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.850+0000 7f188b400640 1 --2- 172.21.15.43:0/3844872498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5cd05bb90 0x55e5cd063c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:06.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.850+0000 7f188b400640 1 -- 172.21.15.43:0/3844872498 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5ccd333c0 con 0x55e5cd05c4b0 2024-09-19T05:14:06.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.851+0000 7f188b400640 1 --2- 172.21.15.43:0/3844872498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5cd05c4b0 0x55e5cd064150 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f18780cd6f0 tx=0x7f1878009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=848294209f8dc65c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:06.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.851+0000 7f1876000640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:14:06.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.851+0000 7f1876000640 1 -- 172.21.15.43:0/3844872498 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187809bc90 con 0x55e5cd05c4b0 2024-09-19T05:14:06.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.851+0000 7f1876000640 1 -- 172.21.15.43:0/3844872498 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f18780cd970 con 0x55e5cd05c4b0 2024-09-19T05:14:06.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.851+0000 7f1895c16740 1 -- 172.21.15.43:0/3844872498 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1884036030 con 0x55e5cd05c4b0 2024-09-19T05:14:06.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.851+0000 7f1876000640 1 -- 172.21.15.43:0/3844872498 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f18780cdb70 con 0x55e5cd05c4b0 2024-09-19T05:14:06.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.851+0000 7f1895c16740 10 client.27289 fetch_fsmap learned FSMap version 29 2024-09-19T05:14:06.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.851+0000 7f1895c16740 1 -- 172.21.15.43:0/3844872498 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e5cd0754f0 con 0x55e5cd05c4b0 2024-09-19T05:14:06.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.852+0000 7f1876000640 1 -- 172.21.15.43:0/3844872498 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1884036030 con 0x55e5cd05c4b0 2024-09-19T05:14:06.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.852+0000 7f1876000640 1 -- 172.21.15.43:0/3844872498 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f18780d3c30 con 0x55e5cd05c4b0 2024-09-19T05:14:06.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.852+0000 7f1895c16740 10 client.27289 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:14:06.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.852+0000 7f1895c16740 10 client.27289 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:14:06.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.852+0000 7f1895c16740 20 client.27289 populate_metadata read hostname 'smithi043' 2024-09-19T05:14:06.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.852+0000 7f1895c16740 1 --2- 172.21.15.43:0/3844872498 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55e5cd0769b0 0x55e5cd078da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:06.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.852+0000 7f1895c16740 4 client.27289 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:14:06.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.852+0000 7f1895c16740 1 -- 172.21.15.43:0/3844872498 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5ccfcf950 con 0x55e5cd0769b0 2024-09-19T05:14:06.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.852+0000 7f188be00640 1 --2- 172.21.15.43:0/3844872498 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55e5cd0769b0 0x55e5cd078da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:06.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.853+0000 7f188be00640 1 --2- 172.21.15.43:0/3844872498 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55e5cd0769b0 0x55e5cd078da0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84dc7c24bb44e3e0 server_cookie=d2f01087d1ddc82c in_seq=0 out_seq=0 2024-09-19T05:14:06.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.853+0000 7f1876000640 10 client.27289 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:14:06.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.854+0000 7f1876000640 1 -- 172.21.15.43:0/3844872498 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e5ccfcf950 con 0x55e5cd0769b0 2024-09-19T05:14:06.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.854+0000 7f1876000640 10 client.27289 handle_command_reply: tid=0 2024-09-19T05:14:06.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.854+0000 7f1895c16740 1 client.27289 shutdown 2024-09-19T05:14:06.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.854+0000 7f1895c16740 1 -- 172.21.15.43:0/3844872498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5cd05c4b0 msgr2=0x55e5cd064150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:06.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.854+0000 7f1895c16740 1 --2- 172.21.15.43:0/3844872498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5cd05c4b0 0x55e5cd064150 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f18780cd6f0 tx=0x7f1878009310 comp rx=0 tx=0).stop 2024-09-19T05:14:06.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.854+0000 7f1895c16740 1 -- 172.21.15.43:0/3844872498 shutdown_connections 2024-09-19T05:14:06.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.854+0000 7f1895c16740 1 --2- 172.21.15.43:0/3844872498 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55e5cd0769b0 0x55e5cd078da0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:06.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.854+0000 7f1895c16740 1 --2- 172.21.15.43:0/3844872498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5cd066d70 0x55e5cd064690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:06.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.854+0000 7f1895c16740 1 --2- 172.21.15.43:0/3844872498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5cd05c4b0 0x55e5cd064150 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:06.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.854+0000 7f1876000640 0 client.27289 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:14:06.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.854+0000 7f1895c16740 1 --2- 172.21.15.43:0/3844872498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5cd05bb90 0x55e5cd063c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:06.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.854+0000 7f1895c16740 1 -- 172.21.15.43:0/3844872498 >> 172.21.15.43:0/3844872498 conn(0x55e5cd058120 msgr2=0x55e5cd05de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:06.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.855+0000 7f1895c16740 1 -- 172.21.15.43:0/3844872498 shutdown_connections 2024-09-19T05:14:06.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.855+0000 7f1895c16740 1 -- 172.21.15.43:0/3844872498 wait complete. 2024-09-19T05:14:06.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.855+0000 7f1895c16740 20 client.27289 trim_cache size 0 max 16384 2024-09-19T05:14:06.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.873+0000 7f1895c16740 1 Processor -- start 2024-09-19T05:14:06.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.873+0000 7f1895c16740 1 -- start start 2024-09-19T05:14:06.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.873+0000 7f1895c16740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5cd05bb90 0x55e5cd05e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:06.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.873+0000 7f1895c16740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5cd05c4b0 0x55e5cd07e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:06.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.873+0000 7f1895c16740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5cd066d70 0x55e5cd07ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:06.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.873+0000 7f1895c16740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f188c1b3f00 con 0x55e5cd05bb90 2024-09-19T05:14:06.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.873+0000 7f1895c16740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f188c1b5a40 con 0x55e5cd066d70 2024-09-19T05:14:06.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.873+0000 7f1895c16740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f188c1b45d0 con 0x55e5cd05c4b0 2024-09-19T05:14:06.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.873+0000 7f188be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5cd05bb90 0x55e5cd05e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:06.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.874+0000 7f188be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5cd05bb90 0x55e5cd05e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49944/0 (socket says 172.21.15.43:49944) 2024-09-19T05:14:06.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.874+0000 7f188be00640 1 -- 172.21.15.43:0/2364479389 learned_addr learned my addr 172.21.15.43:0/2364479389 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:06.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.874+0000 7f188b400640 1 --2- 172.21.15.43:0/2364479389 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5cd05c4b0 0x55e5cd07e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:06.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.874+0000 7f1891200640 1 --2- 172.21.15.43:0/2364479389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5cd066d70 0x55e5cd07ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:06.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.874+0000 7f188b400640 1 -- 172.21.15.43:0/2364479389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5cd066d70 msgr2=0x55e5cd07ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:06.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.874+0000 7f188b400640 1 --2- 172.21.15.43:0/2364479389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5cd066d70 0x55e5cd07ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:06.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.874+0000 7f188b400640 1 -- 172.21.15.43:0/2364479389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5cd05bb90 msgr2=0x55e5cd05e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:06.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.874+0000 7f188b400640 1 --2- 172.21.15.43:0/2364479389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5cd05bb90 0x55e5cd05e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:06.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.874+0000 7f188b400640 1 -- 172.21.15.43:0/2364479389 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5cd057f50 con 0x55e5cd05c4b0 2024-09-19T05:14:06.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.874+0000 7f188be00640 1 --2- 172.21.15.43:0/2364479389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5cd05bb90 0x55e5cd05e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:06.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.874+0000 7f188b400640 1 --2- 172.21.15.43:0/2364479389 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5cd05c4b0 0x55e5cd07e670 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x55e5cd065250 tx=0x7f1878000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2857914a78a9cbb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:06.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.875+0000 7f1880c00640 1 -- 172.21.15.43:0/2364479389 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18780d37b0 con 0x55e5cd05c4b0 2024-09-19T05:14:06.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.875+0000 7f1880c00640 1 -- 172.21.15.43:0/2364479389 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f187809c330 con 0x55e5cd05c4b0 2024-09-19T05:14:06.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.875+0000 7f1880c00640 1 -- 172.21.15.43:0/2364479389 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187809bab0 con 0x55e5cd05c4b0 2024-09-19T05:14:06.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.875+0000 7f1895c16740 1 -- 172.21.15.43:0/2364479389 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5cd05c4b0 msgr2=0x55e5cd07e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:06.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.875+0000 7f1895c16740 1 --2- 172.21.15.43:0/2364479389 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5cd05c4b0 0x55e5cd07e670 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x55e5cd065250 tx=0x7f1878000bc0 comp rx=0 tx=0).stop 2024-09-19T05:14:06.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.875+0000 7f1895c16740 1 -- 172.21.15.43:0/2364479389 shutdown_connections 2024-09-19T05:14:06.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.875+0000 7f1895c16740 1 --2- 172.21.15.43:0/2364479389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5cd066d70 0x55e5cd07ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:06.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.875+0000 7f1895c16740 1 --2- 172.21.15.43:0/2364479389 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5cd05c4b0 0x55e5cd07e670 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:06.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.875+0000 7f1895c16740 1 --2- 172.21.15.43:0/2364479389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5cd05bb90 0x55e5cd05e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:06.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.875+0000 7f1895c16740 1 -- 172.21.15.43:0/2364479389 >> 172.21.15.43:0/2364479389 conn(0x55e5cd058120 msgr2=0x55e5cd0571c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:06.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.875+0000 7f1895c16740 1 -- 172.21.15.43:0/2364479389 shutdown_connections 2024-09-19T05:14:06.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.875+0000 7f1895c16740 1 -- 172.21.15.43:0/2364479389 wait complete. 2024-09-19T05:14:06.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.876+0000 7f1895c16740 1 -- start start 2024-09-19T05:14:06.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.876+0000 7f1895c16740 1 Processor -- start 2024-09-19T05:14:06.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.876+0000 7f1895c16740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5cd05bb90 0x55e5cd06ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:06.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.876+0000 7f1895c16740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5cd05c4b0 0x55e5cd06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:06.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.876+0000 7f1895c16740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5cd066d70 0x55e5cd06bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:06.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.876+0000 7f1895c16740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f188c1b5370 con 0x55e5cd05c4b0 2024-09-19T05:14:06.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.876+0000 7f1895c16740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f188c1b4ca0 con 0x55e5cd05bb90 2024-09-19T05:14:06.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.876+0000 7f1895c16740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f188c1b1f00 con 0x55e5cd066d70 2024-09-19T05:14:06.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.876+0000 7f1895c16740 10 client.0 _ll_register_callbacks cb 0x7f1878009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:06.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.877+0000 7f188b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5cd05c4b0 0x55e5cd06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:06.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.877+0000 7f188b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5cd05c4b0 0x55e5cd06f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49954/0 (socket says 172.21.15.43:49954) 2024-09-19T05:14:06.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.877+0000 7f188b400640 1 -- 172.21.15.43:0/605685607 learned_addr learned my addr 172.21.15.43:0/605685607 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:06.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.877+0000 7f188be00640 1 --2- 172.21.15.43:0/605685607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5cd05bb90 0x55e5cd06ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:06.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.877+0000 7f1891200640 1 --2- 172.21.15.43:0/605685607 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5cd066d70 0x55e5cd06bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:06.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.877+0000 7f188b400640 1 -- 172.21.15.43:0/605685607 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5cd066d70 msgr2=0x55e5cd06bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:06.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.877+0000 7f188b400640 1 --2- 172.21.15.43:0/605685607 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5cd066d70 0x55e5cd06bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:06.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.877+0000 7f188b400640 1 -- 172.21.15.43:0/605685607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5cd05bb90 msgr2=0x55e5cd06ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:06.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.877+0000 7f188b400640 1 --2- 172.21.15.43:0/605685607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5cd05bb90 0x55e5cd06ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:06.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.877+0000 7f188b400640 1 -- 172.21.15.43:0/605685607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5cd0591d0 con 0x55e5cd05c4b0 2024-09-19T05:14:06.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.877+0000 7f188b400640 1 --2- 172.21.15.43:0/605685607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5cd05c4b0 0x55e5cd06f1a0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f187809a5e0 tx=0x7f18780a0770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be1ed350bb2c9c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:06.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.877+0000 7f1876000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:14:06.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.878+0000 7f1876000640 1 -- 172.21.15.43:0/605685607 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18780d3380 con 0x55e5cd05c4b0 2024-09-19T05:14:06.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.878+0000 7f1876000640 1 -- 172.21.15.43:0/605685607 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f18780a0120 con 0x55e5cd05c4b0 2024-09-19T05:14:06.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.878+0000 7f1895c16740 1 -- 172.21.15.43:0/605685607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f187809c330 con 0x55e5cd05c4b0 2024-09-19T05:14:06.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.878+0000 7f1876000640 1 -- 172.21.15.43:0/605685607 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f18780b0050 con 0x55e5cd05c4b0 2024-09-19T05:14:06.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.878+0000 7f1895c16740 10 client.17376 fetch_fsmap learned FSMap version 29 2024-09-19T05:14:06.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.878+0000 7f1895c16740 1 -- 172.21.15.43:0/605685607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e5cd0591d0 con 0x55e5cd05c4b0 2024-09-19T05:14:06.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.878+0000 7f1876000640 1 -- 172.21.15.43:0/605685607 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f187809c330 con 0x55e5cd05c4b0 2024-09-19T05:14:06.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.878+0000 7f1876000640 1 -- 172.21.15.43:0/605685607 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1878018b30 con 0x55e5cd05c4b0 2024-09-19T05:14:06.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.878+0000 7f1895c16740 10 client.17376 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:14:06.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.878+0000 7f1895c16740 10 client.17376 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:14:06.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.878+0000 7f1895c16740 20 client.17376 populate_metadata read hostname 'smithi043' 2024-09-19T05:14:06.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.879+0000 7f1895c16740 1 --2- 172.21.15.43:0/605685607 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55e5cd0e1d20 0x55e5cd0e4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:06.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.879+0000 7f1895c16740 4 client.17376 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:14:06.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.879+0000 7f1895c16740 1 -- 172.21.15.43:0/605685607 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e5ccfc9e60 con 0x55e5cd0e1d20 2024-09-19T05:14:06.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.879+0000 7f188be00640 1 --2- 172.21.15.43:0/605685607 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55e5cd0e1d20 0x55e5cd0e4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:06.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.879+0000 7f188be00640 1 --2- 172.21.15.43:0/605685607 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55e5cd0e1d20 0x55e5cd0e4140 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1305538fb94d2cca server_cookie=891ba9b149d6f858 in_seq=0 out_seq=0 2024-09-19T05:14:06.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.879+0000 7f1876000640 10 client.17376 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:14:06.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.880+0000 7f1876000640 1 -- 172.21.15.43:0/605685607 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e5ccfc9e60 con 0x55e5cd0e1d20 2024-09-19T05:14:06.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.880+0000 7f1876000640 10 client.17376 handle_command_reply: tid=0 2024-09-19T05:14:06.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.880+0000 7f1895c16740 1 client.17376 shutdown 2024-09-19T05:14:06.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.880+0000 7f1895c16740 1 -- 172.21.15.43:0/605685607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5cd05c4b0 msgr2=0x55e5cd06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:06.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.880+0000 7f1895c16740 1 --2- 172.21.15.43:0/605685607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5cd05c4b0 0x55e5cd06f1a0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f187809a5e0 tx=0x7f18780a0770 comp rx=0 tx=0).stop 2024-09-19T05:14:06.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.880+0000 7f1895c16740 1 -- 172.21.15.43:0/605685607 shutdown_connections 2024-09-19T05:14:06.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.880+0000 7f1895c16740 1 --2- 172.21.15.43:0/605685607 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55e5cd0e1d20 0x55e5cd0e4140 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:06.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.880+0000 7f1895c16740 1 --2- 172.21.15.43:0/605685607 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5cd066d70 0x55e5cd06bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:06.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.880+0000 7f1895c16740 1 --2- 172.21.15.43:0/605685607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e5cd05c4b0 0x55e5cd06f1a0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:06.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.880+0000 7f1895c16740 1 --2- 172.21.15.43:0/605685607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5cd05bb90 0x55e5cd06ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:06.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.880+0000 7f1895c16740 1 -- 172.21.15.43:0/605685607 >> 172.21.15.43:0/605685607 conn(0x55e5cd058120 msgr2=0x55e5cd065470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:06.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.880+0000 7f1876000640 0 client.17376 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:14:06.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.880+0000 7f1895c16740 1 -- 172.21.15.43:0/605685607 shutdown_connections 2024-09-19T05:14:06.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.880+0000 7f1895c16740 1 -- 172.21.15.43:0/605685607 wait complete. 2024-09-19T05:14:06.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.880+0000 7f1895c16740 20 client.17376 trim_cache size 0 max 16384 2024-09-19T05:14:06.887 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:14:06.887 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-09-19T05:14:06.887 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-09-19T05:14:06.887 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:14:06.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.881+0000 7f1893200640 1 -- 172.21.15.43:0/3137392462 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f18700a33b0 msgr2=0x7f18700a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:06.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.881+0000 7f1893200640 1 --2- 172.21.15.43:0/3137392462 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f18700a33b0 0x7f18700a5870 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55e5cd058f80 tx=0x7f187c005f90 comp rx=0 tx=0).stop 2024-09-19T05:14:06.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.881+0000 7f1893200640 1 -- 172.21.15.43:0/3137392462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f188c077040 msgr2=0x7f188c1a4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:06.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.881+0000 7f1893200640 1 --2- 172.21.15.43:0/3137392462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f188c077040 0x7f188c1a4bf0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f187800e9f0 tx=0x7f187800eeb0 comp rx=0 tx=0).stop 2024-09-19T05:14:06.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.881+0000 7f1893200640 1 -- 172.21.15.43:0/3137392462 shutdown_connections 2024-09-19T05:14:06.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.881+0000 7f1893200640 1 --2- 172.21.15.43:0/3137392462 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f18700a33b0 0x7f18700a5870 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:06.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.881+0000 7f1893200640 1 --2- 172.21.15.43:0/3137392462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f188c112a30 0x7f188c19e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:06.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.881+0000 7f1893200640 1 --2- 172.21.15.43:0/3137392462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f188c077040 0x7f188c1a4bf0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:06.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.881+0000 7f1893200640 1 --2- 172.21.15.43:0/3137392462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f188c076720 0x7f188c1a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:06.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.881+0000 7f1893200640 1 -- 172.21.15.43:0/3137392462 >> 172.21.15.43:0/3137392462 conn(0x7f188c0ffa90 msgr2=0x7f188c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:06.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.881+0000 7f1893200640 1 -- 172.21.15.43:0/3137392462 shutdown_connections 2024-09-19T05:14:06.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:06.881+0000 7f1893200640 1 -- 172.21.15.43:0/3137392462 wait complete. 2024-09-19T05:14:07.787 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T05:14:07.787 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:14:08.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:07 smithi043 ceph-mon[30800]: pgmap v293: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-19T05:14:08.352 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:14:08.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:07 smithi151 ceph-mon[35425]: pgmap v293: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-19T05:14:08.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:07 smithi096 ceph-mon[35085]: pgmap v293: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-19T05:14:09.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.086+0000 7f0b2aa00640 1 -- 172.21.15.43:0/999232178 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b24030d60 con 0x7f0b2c069770 2024-09-19T05:14:09.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.087+0000 7f0b33200640 1 -- 172.21.15.43:0/999232178 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b2c069770 msgr2=0x7f0b2c1003a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:09.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.088+0000 7f0b33200640 1 --2- 172.21.15.43:0/999232178 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b2c069770 0x7f0b2c1003a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0b24009be0 tx=0x7f0b2402a6f0 comp rx=0 tx=0).stop 2024-09-19T05:14:09.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.089+0000 7f0b33200640 1 -- 172.21.15.43:0/999232178 shutdown_connections 2024-09-19T05:14:09.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.089+0000 7f0b33200640 1 --2- 172.21.15.43:0/999232178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b2c100d20 0x7f0b2c105370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:09.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.090+0000 7f0b33200640 1 --2- 172.21.15.43:0/999232178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b2c1008e0 0x7f0b2c104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:09.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.090+0000 7f0b33200640 1 --2- 172.21.15.43:0/999232178 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b2c069770 0x7f0b2c1003a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:09.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.091+0000 7f0b33200640 1 -- 172.21.15.43:0/999232178 >> 172.21.15.43:0/999232178 conn(0x7f0b2c077230 msgr2=0x7f0b2c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:09.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.091+0000 7f0b33200640 1 -- 172.21.15.43:0/999232178 shutdown_connections 2024-09-19T05:14:09.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.091+0000 7f0b33200640 1 -- 172.21.15.43:0/999232178 wait complete. 2024-09-19T05:14:09.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.093+0000 7f0b33200640 1 Processor -- start 2024-09-19T05:14:09.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.096+0000 7f0b33200640 1 -- start start 2024-09-19T05:14:09.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.096+0000 7f0b33200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b2c069770 0x7f0b2c19a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:09.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.096+0000 7f0b33200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b2c1008e0 0x7f0b2c19ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:09.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.096+0000 7f0b33200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b2c100d20 0x7f0b2c19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:09.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.096+0000 7f0b33200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c109ef0 con 0x7f0b2c100d20 2024-09-19T05:14:09.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.096+0000 7f0b33200640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c109d70 con 0x7f0b2c069770 2024-09-19T05:14:09.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.096+0000 7f0b33200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c10a070 con 0x7f0b2c1008e0 2024-09-19T05:14:09.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.097+0000 7f0b2b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b2c1008e0 0x7f0b2c19ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:09.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.097+0000 7f0b2b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b2c1008e0 0x7f0b2c19ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:59404/0 (socket says 172.21.15.43:59404) 2024-09-19T05:14:09.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.097+0000 7f0b2be00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b2c069770 0x7f0b2c19a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:09.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.097+0000 7f0b2b400640 1 -- 172.21.15.43:0/2179473367 learned_addr learned my addr 172.21.15.43:0/2179473367 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:09.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.097+0000 7f0b31200640 1 --2- 172.21.15.43:0/2179473367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b2c100d20 0x7f0b2c19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:09.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.098+0000 7f0b2b400640 1 -- 172.21.15.43:0/2179473367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b2c069770 msgr2=0x7f0b2c19a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:09.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.098+0000 7f0b2b400640 1 --2- 172.21.15.43:0/2179473367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b2c069770 0x7f0b2c19a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:09.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.098+0000 7f0b2b400640 1 -- 172.21.15.43:0/2179473367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b2c100d20 msgr2=0x7f0b2c19f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:09.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.098+0000 7f0b2b400640 1 --2- 172.21.15.43:0/2179473367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b2c100d20 0x7f0b2c19f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:09.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.098+0000 7f0b2b400640 1 -- 172.21.15.43:0/2179473367 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b2c19f710 con 0x7f0b2c1008e0 2024-09-19T05:14:09.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.100+0000 7f0b2b400640 1 --2- 172.21.15.43:0/2179473367 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b2c1008e0 0x7f0b2c19ac80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0b1800b4a0 tx=0x7f0b1800b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:09.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.100+0000 7f0b28c00640 1 -- 172.21.15.43:0/2179473367 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18015020 con 0x7f0b2c1008e0 2024-09-19T05:14:09.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.100+0000 7f0b28c00640 1 -- 172.21.15.43:0/2179473367 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b18011d10 con 0x7f0b2c1008e0 2024-09-19T05:14:09.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.100+0000 7f0b28c00640 1 -- 172.21.15.43:0/2179473367 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1800f2e0 con 0x7f0b2c1008e0 2024-09-19T05:14:09.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.100+0000 7f0b33200640 1 -- 172.21.15.43:0/2179473367 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b2c19fa00 con 0x7f0b2c1008e0 2024-09-19T05:14:09.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.101+0000 7f0b33200640 1 -- 172.21.15.43:0/2179473367 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b2c06bc10 con 0x7f0b2c1008e0 2024-09-19T05:14:09.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.102+0000 7f0b33200640 1 -- 172.21.15.43:0/2179473367 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b2c101910 con 0x7f0b2c1008e0 2024-09-19T05:14:09.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.105+0000 7f0b28c00640 1 -- 172.21.15.43:0/2179473367 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0b18018020 con 0x7f0b2c1008e0 2024-09-19T05:14:09.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.107+0000 7f0b28c00640 1 --2- 172.21.15.43:0/2179473367 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f0b100a3430 0x7f0b100a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:09.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.108+0000 7f0b28c00640 1 -- 172.21.15.43:0/2179473367 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8586+0+0 (secure 0 0 0) 0x7f0b180cf420 con 0x7f0b2c1008e0 2024-09-19T05:14:09.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.109+0000 7f0b2be00640 1 --2- 172.21.15.43:0/2179473367 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f0b100a3430 0x7f0b100a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:09.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.111+0000 7f0b2be00640 1 --2- 172.21.15.43:0/2179473367 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f0b100a3430 0x7f0b100a58f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0b24004af0 tx=0x7f0b24005fd0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:09.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.112+0000 7f0b28c00640 1 -- 172.21.15.43:0/2179473367 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0b1801f020 con 0x7f0b2c1008e0 2024-09-19T05:14:09.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:08 smithi043 ceph-mon[30800]: osdmap e93: 12 total, 12 up, 12 in 2024-09-19T05:14:09.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:08 smithi151 ceph-mon[35425]: osdmap e93: 12 total, 12 up, 12 in 2024-09-19T05:14:09.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:08 smithi096 ceph-mon[35085]: osdmap e93: 12 total, 12 up, 12 in 2024-09-19T05:14:09.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.484+0000 7f0b33200640 1 -- 172.21.15.43:0/2179473367 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b2c06bf00 con 0x7f0b2c1008e0 2024-09-19T05:14:09.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.485+0000 7f0b28c00640 1 -- 172.21.15.43:0/2179473367 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11965 (secure 0 0 0) 0x7f0b1809bed0 con 0x7f0b2c1008e0 2024-09-19T05:14:09.487 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:14:09.488 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":29,"btime":"2024-09-19T05:04:38:785407+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/773801169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":773801169},{"type":"v1","addr":"172.21.15.43:6835","nonce":773801169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1799137902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1799137902},{"type":"v1","addr":"172.21.15.43:6837","nonce":1799137902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:02:25.463041+0000","modified":"2024-09-19T05:04:38.039809+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24518,"mds_2":24511,"mds_3":24506,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14541":{"gid":14541,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/794881135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":794881135},{"type":"v1","addr":"172.21.15.43:6841","nonce":794881135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/2836453158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2836453158},{"type":"v1","addr":"172.21.15.151:6833","nonce":2836453158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/178785812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":178785812},{"type":"v1","addr":"172.21.15.96:6833","nonce":178785812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1671043820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1671043820},{"type":"v1","addr":"172.21.15.96:6835","nonce":1671043820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2972514328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2972514328},{"type":"v1","addr":"172.21.15.151:6835","nonce":2972514328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.151:6837/3960378010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3960378010},{"type":"v1","addr":"172.21.15.151:6837","nonce":3960378010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6837/3978388286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3978388286},{"type":"v1","addr":"172.21.15.96:6837","nonce":3978388286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.151:6839/3794104313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3794104313},{"type":"v1","addr":"172.21.15.151:6839","nonce":3794104313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.43:6839/3404555030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3404555030},{"type":"v1","addr":"172.21.15.43:6839","nonce":3404555030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/605527621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":605527621},{"type":"v1","addr":"172.21.15.96:6839","nonce":605527621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24518,24511,24506,24493]},"id":1}]} 2024-09-19T05:14:09.488 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 29 2024-09-19T05:14:09.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.487+0000 7f0b33200640 1 -- 172.21.15.43:0/2179473367 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f0b100a3430 msgr2=0x7f0b100a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:09.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.487+0000 7f0b33200640 1 --2- 172.21.15.43:0/2179473367 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f0b100a3430 0x7f0b100a58f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0b24004af0 tx=0x7f0b24005fd0 comp rx=0 tx=0).stop 2024-09-19T05:14:09.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.487+0000 7f0b33200640 1 -- 172.21.15.43:0/2179473367 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b2c1008e0 msgr2=0x7f0b2c19ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:09.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.487+0000 7f0b33200640 1 --2- 172.21.15.43:0/2179473367 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b2c1008e0 0x7f0b2c19ac80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0b1800b4a0 tx=0x7f0b1800b8a0 comp rx=0 tx=0).stop 2024-09-19T05:14:09.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.487+0000 7f0b33200640 1 -- 172.21.15.43:0/2179473367 shutdown_connections 2024-09-19T05:14:09.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.487+0000 7f0b33200640 1 --2- 172.21.15.43:0/2179473367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b2c100d20 0x7f0b2c19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:09.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.487+0000 7f0b33200640 1 --2- 172.21.15.43:0/2179473367 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b2c1008e0 0x7f0b2c19ac80 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:09.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.487+0000 7f0b33200640 1 --2- 172.21.15.43:0/2179473367 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f0b100a3430 0x7f0b100a58f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:09.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.487+0000 7f0b33200640 1 --2- 172.21.15.43:0/2179473367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b2c069770 0x7f0b2c19a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:09.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.487+0000 7f0b33200640 1 -- 172.21.15.43:0/2179473367 >> 172.21.15.43:0/2179473367 conn(0x7f0b2c077230 msgr2=0x7f0b2c0fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:09.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.487+0000 7f0b33200640 1 -- 172.21.15.43:0/2179473367 shutdown_connections 2024-09-19T05:14:09.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:09.487+0000 7f0b33200640 1 -- 172.21.15.43:0/2179473367 wait complete. 2024-09-19T05:14:10.200 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:09 smithi043 ceph-mon[30800]: pgmap v295: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T05:14:10.200 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:09 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/2179473367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:14:10.238 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T05:14:10.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:09 smithi151 ceph-mon[35425]: pgmap v295: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T05:14:10.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:09 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/2179473367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:14:10.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:09 smithi096 ceph-mon[35085]: pgmap v295: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T05:14:10.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:09 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/2179473367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:14:10.614 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:14:11.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.199+0000 7f8636c00640 1 -- 172.21.15.43:0/2971409655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8630077040 msgr2=0x7f86301124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:11.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.199+0000 7f8636c00640 1 --2- 172.21.15.43:0/2971409655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8630077040 0x7f86301124f0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f861c009900 tx=0x7f861c02a3f0 comp rx=0 tx=0).stop 2024-09-19T05:14:11.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.199+0000 7f8636c00640 1 -- 172.21.15.43:0/2971409655 shutdown_connections 2024-09-19T05:14:11.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.199+0000 7f8636c00640 1 --2- 172.21.15.43:0/2971409655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8630112a30 0x7f8630114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:11.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.199+0000 7f8636c00640 1 --2- 172.21.15.43:0/2971409655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8630077040 0x7f86301124f0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:11.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.199+0000 7f8636c00640 1 --2- 172.21.15.43:0/2971409655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8630076720 0x7f8630076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:11.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.199+0000 7f8636c00640 1 -- 172.21.15.43:0/2971409655 >> 172.21.15.43:0/2971409655 conn(0x7f86300ffa90 msgr2=0x7f8630101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:11.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.199+0000 7f8636c00640 1 -- 172.21.15.43:0/2971409655 shutdown_connections 2024-09-19T05:14:11.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.199+0000 7f8636c00640 1 -- 172.21.15.43:0/2971409655 wait complete. 2024-09-19T05:14:11.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.200+0000 7f8636c00640 1 Processor -- start 2024-09-19T05:14:11.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.201+0000 7f8636c00640 1 -- start start 2024-09-19T05:14:11.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.201+0000 7f8636c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8630076720 0x7f86301a4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:11.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.201+0000 7f8636c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8630077040 0x7f86301a4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:11.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.201+0000 7f8636c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8630112a30 0x7f863019e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:11.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.201+0000 7f8636c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f86301185f0 con 0x7f8630076720 2024-09-19T05:14:11.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.201+0000 7f8636c00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8630118470 con 0x7f8630112a30 2024-09-19T05:14:11.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.201+0000 7f8636c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8630118770 con 0x7f8630077040 2024-09-19T05:14:11.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.201+0000 7f862f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8630077040 0x7f86301a4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:11.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.201+0000 7f862f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8630077040 0x7f86301a4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:59420/0 (socket says 172.21.15.43:59420) 2024-09-19T05:14:11.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.201+0000 7f8634c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8630112a30 0x7f863019e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:11.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.201+0000 7f862f400640 1 -- 172.21.15.43:0/871078837 learned_addr learned my addr 172.21.15.43:0/871078837 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:11.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.201+0000 7f862fe00640 1 --2- 172.21.15.43:0/871078837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8630076720 0x7f86301a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:11.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.202+0000 7f862f400640 1 -- 172.21.15.43:0/871078837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8630112a30 msgr2=0x7f863019e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:11.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.202+0000 7f862f400640 1 --2- 172.21.15.43:0/871078837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8630112a30 0x7f863019e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:11.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.202+0000 7f862f400640 1 -- 172.21.15.43:0/871078837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8630076720 msgr2=0x7f86301a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:11.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.202+0000 7f862f400640 1 --2- 172.21.15.43:0/871078837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8630076720 0x7f86301a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:11.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.202+0000 7f862f400640 1 -- 172.21.15.43:0/871078837 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f863019f030 con 0x7f8630077040 2024-09-19T05:14:11.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.202+0000 7f862f400640 1 --2- 172.21.15.43:0/871078837 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8630077040 0x7f86301a4c80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f861c004770 tx=0x7f861c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:11.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.202+0000 7f862cc00640 1 -- 172.21.15.43:0/871078837 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861c03f070 con 0x7f8630077040 2024-09-19T05:14:11.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.202+0000 7f8636c00640 1 -- 172.21.15.43:0/871078837 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f863019f2c0 con 0x7f8630077040 2024-09-19T05:14:11.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.202+0000 7f862cc00640 1 -- 172.21.15.43:0/871078837 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f861c031ca0 con 0x7f8630077040 2024-09-19T05:14:11.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.202+0000 7f862cc00640 1 -- 172.21.15.43:0/871078837 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861c035030 con 0x7f8630077040 2024-09-19T05:14:11.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.203+0000 7f8636c00640 1 -- 172.21.15.43:0/871078837 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86301ab610 con 0x7f8630077040 2024-09-19T05:14:11.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.205+0000 7f8639611740 1 Processor -- start 2024-09-19T05:14:11.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.206+0000 7f8639611740 1 -- start start 2024-09-19T05:14:11.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.206+0000 7f8639611740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d5dbbc70 0x55e4d5dbc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:11.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.206+0000 7f8639611740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d5dbc590 0x55e4d5dc6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:11.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.206+0000 7f8639611740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d5dc6e50 0x55e4d5dc9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:11.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.206+0000 7f8639611740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e4d5d223e0 con 0x55e4d5dc6e50 2024-09-19T05:14:11.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.206+0000 7f8639611740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e4d5cc9d80 con 0x55e4d5dbbc70 2024-09-19T05:14:11.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.206+0000 7f8639611740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e4d5d1e3a0 con 0x55e4d5dbc590 2024-09-19T05:14:11.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.206+0000 7f862cc00640 1 -- 172.21.15.43:0/871078837 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f861c042070 con 0x7f8630077040 2024-09-19T05:14:11.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.206+0000 7f8634c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d5dbbc70 0x55e4d5dbc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:11.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.206+0000 7f862fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d5dbc590 0x55e4d5dc6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:11.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.206+0000 7f8634c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d5dbbc70 0x55e4d5dbc050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:40072/0 (socket says 172.21.15.43:40072) 2024-09-19T05:14:11.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.206+0000 7f8634c00640 1 -- 172.21.15.43:0/1585163571 learned_addr learned my addr 172.21.15.43:0/1585163571 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:11.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.207+0000 7f862f400640 1 --2- 172.21.15.43:0/1585163571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d5dc6e50 0x55e4d5dc9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:11.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.207+0000 7f8634c00640 1 -- 172.21.15.43:0/1585163571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d5dbc590 msgr2=0x55e4d5dc6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:11.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.207+0000 7f8634c00640 1 --2- 172.21.15.43:0/1585163571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d5dbc590 0x55e4d5dc6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:11.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.207+0000 7f8634c00640 1 -- 172.21.15.43:0/1585163571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d5dc6e50 msgr2=0x55e4d5dc9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:11.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.207+0000 7f8634c00640 1 --2- 172.21.15.43:0/1585163571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d5dc6e50 0x55e4d5dc9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:11.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.207+0000 7f8634c00640 1 -- 172.21.15.43:0/1585163571 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4d5ce8ae0 con 0x55e4d5dbbc70 2024-09-19T05:14:11.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.207+0000 7f862f400640 1 --2- 172.21.15.43:0/1585163571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d5dc6e50 0x55e4d5dc9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:14:11.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.207+0000 7f8634c00640 1 --2- 172.21.15.43:0/1585163571 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d5dbbc70 0x55e4d5dbc050 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8628009f30 tx=0x7f86280118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d40f1d9188836a91 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:11.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.207+0000 7f8625200640 1 -- 172.21.15.43:0/1585163571 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8628004d20 con 0x55e4d5dbbc70 2024-09-19T05:14:11.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.207+0000 7f8625200640 1 -- 172.21.15.43:0/1585163571 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8628004050 con 0x55e4d5dbbc70 2024-09-19T05:14:11.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.207+0000 7f8625200640 1 -- 172.21.15.43:0/1585163571 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8628005260 con 0x55e4d5dbbc70 2024-09-19T05:14:11.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.208+0000 7f862cc00640 1 --2- 172.21.15.43:0/871078837 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f86140a33b0 0x7f86140a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:11.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.208+0000 7f862cc00640 1 -- 172.21.15.43:0/871078837 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8586+0+0 (secure 0 0 0) 0x7f861c0f03a0 con 0x7f8630077040 2024-09-19T05:14:11.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.208+0000 7f8639611740 1 -- 172.21.15.43:0/1585163571 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d5dbbc70 msgr2=0x55e4d5dbc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:11.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.208+0000 7f8639611740 1 --2- 172.21.15.43:0/1585163571 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d5dbbc70 0x55e4d5dbc050 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8628009f30 tx=0x7f86280118d0 comp rx=0 tx=0).stop 2024-09-19T05:14:11.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.208+0000 7f862fe00640 1 --2- 172.21.15.43:0/871078837 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f86140a33b0 0x7f86140a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:11.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.208+0000 7f8639611740 1 -- 172.21.15.43:0/1585163571 shutdown_connections 2024-09-19T05:14:11.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.208+0000 7f8639611740 1 --2- 172.21.15.43:0/1585163571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d5dc6e50 0x55e4d5dc9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:11.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.208+0000 7f8639611740 1 --2- 172.21.15.43:0/1585163571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d5dbc590 0x55e4d5dc6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:11.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.208+0000 7f8639611740 1 --2- 172.21.15.43:0/1585163571 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d5dbbc70 0x55e4d5dbc050 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:11.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.208+0000 7f8639611740 1 -- 172.21.15.43:0/1585163571 >> 172.21.15.43:0/1585163571 conn(0x55e4d5db8200 msgr2=0x55e4d5db85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:11.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.208+0000 7f8639611740 1 -- 172.21.15.43:0/1585163571 shutdown_connections 2024-09-19T05:14:11.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.209+0000 7f8639611740 1 -- 172.21.15.43:0/1585163571 wait complete. 2024-09-19T05:14:11.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.209+0000 7f862fe00640 1 --2- 172.21.15.43:0/871078837 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f86140a33b0 0x7f86140a5870 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x55e4d5db9060 tx=0x7f8620005f90 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:11.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.209+0000 7f8639611740 1 -- start start 2024-09-19T05:14:11.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.209+0000 7f8639611740 1 Processor -- start 2024-09-19T05:14:11.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.210+0000 7f8639611740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d5dbbc70 0x55e4d5dc3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:11.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.210+0000 7f8639611740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d5dbc590 0x55e4d5dc4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:11.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.210+0000 7f8639611740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d5dc6e50 0x55e4d5dc4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:11.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.210+0000 7f8639611740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e4d5dc9a20 con 0x55e4d5dc6e50 2024-09-19T05:14:11.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.210+0000 7f8639611740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e4d5cdd0f0 con 0x55e4d5dbbc70 2024-09-19T05:14:11.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.210+0000 7f8639611740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e4d5dc9ba0 con 0x55e4d5dbc590 2024-09-19T05:14:11.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.210+0000 7f8639611740 10 client.0 _ll_register_callbacks cb 0x55e4d5cc4bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:11.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.210+0000 7f862fe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d5dbbc70 0x55e4d5dc3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:11.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.210+0000 7f862fe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d5dbbc70 0x55e4d5dc3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:40078/0 (socket says 172.21.15.43:40078) 2024-09-19T05:14:11.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.210+0000 7f862fe00640 1 -- 172.21.15.43:0/566189429 learned_addr learned my addr 172.21.15.43:0/566189429 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:11.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.210+0000 7f8634c00640 1 --2- 172.21.15.43:0/566189429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d5dc6e50 0x55e4d5dc4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:11.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.210+0000 7f862f400640 1 --2- 172.21.15.43:0/566189429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d5dbc590 0x55e4d5dc4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:11.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.211+0000 7f862fe00640 1 -- 172.21.15.43:0/566189429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d5dbc590 msgr2=0x55e4d5dc4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:11.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.211+0000 7f862fe00640 1 --2- 172.21.15.43:0/566189429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d5dbc590 0x55e4d5dc4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:11.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.211+0000 7f862fe00640 1 -- 172.21.15.43:0/566189429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d5dc6e50 msgr2=0x55e4d5dc4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:11.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.211+0000 7f862fe00640 1 --2- 172.21.15.43:0/566189429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d5dc6e50 0x55e4d5dc4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:11.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.211+0000 7f862fe00640 1 -- 172.21.15.43:0/566189429 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4d5a933c0 con 0x55e4d5dbbc70 2024-09-19T05:14:11.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.211+0000 7f8634c00640 1 --2- 172.21.15.43:0/566189429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d5dc6e50 0x55e4d5dc4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:11.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.211+0000 7f862fe00640 1 --2- 172.21.15.43:0/566189429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d5dbbc70 0x55e4d5dc3cc0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8620004620 tx=0x7f8620004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f51c7a826cfb475 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:11.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.211+0000 7f8619600640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:14:11.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.211+0000 7f8619600640 1 -- 172.21.15.43:0/566189429 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862001f640 con 0x55e4d5dbbc70 2024-09-19T05:14:11.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.211+0000 7f8639611740 1 -- 172.21.15.43:0/566189429 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8628004050 con 0x55e4d5dbbc70 2024-09-19T05:14:11.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.211+0000 7f8619600640 1 -- 172.21.15.43:0/566189429 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8586+0+0 (secure 0 0 0) 0x7f862001f7e0 con 0x55e4d5dbbc70 2024-09-19T05:14:11.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.212+0000 7f8619600640 1 -- 172.21.15.43:0/566189429 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f862001f9e0 con 0x55e4d5dbbc70 2024-09-19T05:14:11.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.212+0000 7f8639611740 10 client.27413 fetch_fsmap learned FSMap version 29 2024-09-19T05:14:11.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.212+0000 7f8639611740 1 -- 172.21.15.43:0/566189429 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e4d5dd5a00 con 0x55e4d5dbbc70 2024-09-19T05:14:11.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.212+0000 7f8619600640 1 -- 172.21.15.43:0/566189429 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8628004050 con 0x55e4d5dbbc70 2024-09-19T05:14:11.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.212+0000 7f8619600640 1 -- 172.21.15.43:0/566189429 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8620027020 con 0x55e4d5dbbc70 2024-09-19T05:14:11.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.212+0000 7f8639611740 10 client.27413 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:14:11.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.212+0000 7f8639611740 10 client.27413 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-19T05:14:11.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.212+0000 7f8639611740 20 client.27413 populate_metadata read hostname 'smithi043' 2024-09-19T05:14:11.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.212+0000 7f8639611740 1 --2- 172.21.15.43:0/566189429 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55e4d5dd6e10 0x55e4d5dd9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:11.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.212+0000 7f8639611740 4 client.27413 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:14:11.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.212+0000 7f8639611740 1 -- 172.21.15.43:0/566189429 --> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4d5d2f950 con 0x55e4d5dd6e10 2024-09-19T05:14:11.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.213+0000 7f862f400640 1 --2- 172.21.15.43:0/566189429 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55e4d5dd6e10 0x55e4d5dd9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:11.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.213+0000 7f862f400640 1 --2- 172.21.15.43:0/566189429 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55e4d5dd6e10 0x55e4d5dd9200 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7877d53824c14fa6 server_cookie=c488e44e5c6942a1 in_seq=0 out_seq=0 2024-09-19T05:14:11.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.213+0000 7f8619600640 10 client.27413 ms_handle_connect on v2:172.21.15.151:6836/3960378010 2024-09-19T05:14:11.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.214+0000 7f8619600640 1 -- 172.21.15.43:0/566189429 <== mds.4 v2:172.21.15.151:6836/3960378010 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e4d5d2f950 con 0x55e4d5dd6e10 2024-09-19T05:14:11.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.214+0000 7f8619600640 10 client.27413 handle_command_reply: tid=0 2024-09-19T05:14:11.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.214+0000 7f8639611740 1 client.27413 shutdown 2024-09-19T05:14:11.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.214+0000 7f8639611740 1 -- 172.21.15.43:0/566189429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d5dbbc70 msgr2=0x55e4d5dc3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:11.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.214+0000 7f8639611740 1 --2- 172.21.15.43:0/566189429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d5dbbc70 0x55e4d5dc3cc0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8620004620 tx=0x7f8620004730 comp rx=0 tx=0).stop 2024-09-19T05:14:11.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.214+0000 7f8639611740 1 -- 172.21.15.43:0/566189429 shutdown_connections 2024-09-19T05:14:11.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.214+0000 7f8639611740 1 --2- 172.21.15.43:0/566189429 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55e4d5dd6e10 0x55e4d5dd9200 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:11.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.214+0000 7f8639611740 1 --2- 172.21.15.43:0/566189429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d5dc6e50 0x55e4d5dc4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:11.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.214+0000 7f8639611740 1 --2- 172.21.15.43:0/566189429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d5dbc590 0x55e4d5dc4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:11.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.214+0000 7f8639611740 1 --2- 172.21.15.43:0/566189429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d5dbbc70 0x55e4d5dc3cc0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:11.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.214+0000 7f8619600640 0 client.27413 ms_handle_reset on v2:172.21.15.151:6836/3960378010 2024-09-19T05:14:11.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.214+0000 7f8639611740 1 -- 172.21.15.43:0/566189429 >> 172.21.15.43:0/566189429 conn(0x55e4d5db8200 msgr2=0x55e4d5db7380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:11.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.215+0000 7f8639611740 1 -- 172.21.15.43:0/566189429 shutdown_connections 2024-09-19T05:14:11.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.215+0000 7f8639611740 1 -- 172.21.15.43:0/566189429 wait complete. 2024-09-19T05:14:11.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.215+0000 7f8639611740 20 client.27413 trim_cache size 0 max 16384 2024-09-19T05:14:11.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.228+0000 7f8639611740 1 Processor -- start 2024-09-19T05:14:11.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.228+0000 7f8639611740 1 -- start start 2024-09-19T05:14:11.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.229+0000 7f8639611740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d5dbbc70 0x55e4d5dbe6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:11.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.229+0000 7f8639611740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d5dbc590 0x55e4d5dc4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:11.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.229+0000 7f8639611740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d5dc6e50 0x55e4d5ddead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:11.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.229+0000 7f8639611740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f86301b4050 con 0x55e4d5dbc590 2024-09-19T05:14:11.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.229+0000 7f8639611740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f86301b5b90 con 0x55e4d5dc6e50 2024-09-19T05:14:11.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.229+0000 7f8639611740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f86301b4720 con 0x55e4d5dbbc70 2024-09-19T05:14:11.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.229+0000 7f862fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d5dbbc70 0x55e4d5dbe6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:11.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.229+0000 7f862fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d5dbbc70 0x55e4d5dbe6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:59446/0 (socket says 172.21.15.43:59446) 2024-09-19T05:14:11.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.229+0000 7f862f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d5dbc590 0x55e4d5dc4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:11.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.229+0000 7f862fe00640 1 -- 172.21.15.43:0/3744567462 learned_addr learned my addr 172.21.15.43:0/3744567462 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:11.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.229+0000 7f8634c00640 1 --2- 172.21.15.43:0/3744567462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d5dc6e50 0x55e4d5ddead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:11.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.230+0000 7f862fe00640 1 -- 172.21.15.43:0/3744567462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d5dc6e50 msgr2=0x55e4d5ddead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:11.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.230+0000 7f862fe00640 1 --2- 172.21.15.43:0/3744567462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d5dc6e50 0x55e4d5ddead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:11.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.230+0000 7f862fe00640 1 -- 172.21.15.43:0/3744567462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d5dbc590 msgr2=0x55e4d5dc4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:11.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.230+0000 7f862fe00640 1 --2- 172.21.15.43:0/3744567462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d5dbc590 0x55e4d5dc4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:11.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.230+0000 7f862fe00640 1 -- 172.21.15.43:0/3744567462 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4d5db3770 con 0x55e4d5dbbc70 2024-09-19T05:14:11.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.230+0000 7f862f400640 1 --2- 172.21.15.43:0/3744567462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d5dbc590 0x55e4d5dc4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:11.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.230+0000 7f862fe00640 1 --2- 172.21.15.43:0/3744567462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d5dbbc70 0x55e4d5dbe6d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f862001f010 tx=0x7f8620004660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9c4cf0cc8db0109 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:11.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.230+0000 7f861be00640 1 -- 172.21.15.43:0/3744567462 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862001f840 con 0x55e4d5dbbc70 2024-09-19T05:14:11.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.230+0000 7f861be00640 1 -- 172.21.15.43:0/3744567462 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8620005500 con 0x55e4d5dbbc70 2024-09-19T05:14:11.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.230+0000 7f861be00640 1 -- 172.21.15.43:0/3744567462 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8620004d10 con 0x55e4d5dbbc70 2024-09-19T05:14:11.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.231+0000 7f8639611740 1 -- 172.21.15.43:0/3744567462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d5dbbc70 msgr2=0x55e4d5dbe6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:11.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.231+0000 7f8639611740 1 --2- 172.21.15.43:0/3744567462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d5dbbc70 0x55e4d5dbe6d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f862001f010 tx=0x7f8620004660 comp rx=0 tx=0).stop 2024-09-19T05:14:11.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.231+0000 7f8639611740 1 -- 172.21.15.43:0/3744567462 shutdown_connections 2024-09-19T05:14:11.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.231+0000 7f8639611740 1 --2- 172.21.15.43:0/3744567462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d5dc6e50 0x55e4d5ddead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:11.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.231+0000 7f8639611740 1 --2- 172.21.15.43:0/3744567462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d5dbc590 0x55e4d5dc4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:11.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.231+0000 7f8639611740 1 --2- 172.21.15.43:0/3744567462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d5dbbc70 0x55e4d5dbe6d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:11.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.231+0000 7f8639611740 1 -- 172.21.15.43:0/3744567462 >> 172.21.15.43:0/3744567462 conn(0x55e4d5db8200 msgr2=0x55e4d5dc5d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:11.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.231+0000 7f8639611740 1 -- 172.21.15.43:0/3744567462 shutdown_connections 2024-09-19T05:14:11.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.231+0000 7f8639611740 1 -- 172.21.15.43:0/3744567462 wait complete. 2024-09-19T05:14:11.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.231+0000 7f8639611740 1 -- start start 2024-09-19T05:14:11.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.232+0000 7f8639611740 1 Processor -- start 2024-09-19T05:14:11.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.232+0000 7f8639611740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d5dbbc70 0x55e4d5dcedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:11.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.232+0000 7f8639611740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d5dbc590 0x55e4d5dcf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:11.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.232+0000 7f8639611740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d5dc6e50 0x55e4d5dc0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:11.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.232+0000 7f8639611740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f86301b54c0 con 0x55e4d5dbc590 2024-09-19T05:14:11.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.232+0000 7f8639611740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f86301b4df0 con 0x55e4d5dc6e50 2024-09-19T05:14:11.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.232+0000 7f8639611740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f86301b2050 con 0x55e4d5dbbc70 2024-09-19T05:14:11.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.232+0000 7f8639611740 10 client.0 _ll_register_callbacks cb 0x7f8620009680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:11.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.232+0000 7f8634c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d5dc6e50 0x55e4d5dc0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:11.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.232+0000 7f8634c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d5dc6e50 0x55e4d5dc0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:40092/0 (socket says 172.21.15.43:40092) 2024-09-19T05:14:11.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.233+0000 7f8634c00640 1 -- 172.21.15.43:0/727421166 learned_addr learned my addr 172.21.15.43:0/727421166 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:11.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.233+0000 7f862f400640 1 --2- 172.21.15.43:0/727421166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d5dbc590 0x55e4d5dcf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:11.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.233+0000 7f862fe00640 1 --2- 172.21.15.43:0/727421166 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d5dbbc70 0x55e4d5dcedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:11.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.233+0000 7f8634c00640 1 -- 172.21.15.43:0/727421166 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d5dbbc70 msgr2=0x55e4d5dcedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:11.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.233+0000 7f8634c00640 1 --2- 172.21.15.43:0/727421166 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d5dbbc70 0x55e4d5dcedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:11.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.233+0000 7f8634c00640 1 -- 172.21.15.43:0/727421166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d5dbc590 msgr2=0x55e4d5dcf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:11.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.233+0000 7f8634c00640 1 --2- 172.21.15.43:0/727421166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d5dbc590 0x55e4d5dcf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:11.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.233+0000 7f8634c00640 1 -- 172.21.15.43:0/727421166 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4d5dc0850 con 0x55e4d5dc6e50 2024-09-19T05:14:11.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.233+0000 7f862f400640 1 --2- 172.21.15.43:0/727421166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d5dbc590 0x55e4d5dcf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:14:11.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.233+0000 7f8634c00640 1 --2- 172.21.15.43:0/727421166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d5dc6e50 0x55e4d5dc0310 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8628011dd0 tx=0x7f8628009ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43ffb6d1027917c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:11.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.233+0000 7f8619600640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:14:11.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.234+0000 7f8619600640 1 -- 172.21.15.43:0/727421166 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8628004260 con 0x55e4d5dc6e50 2024-09-19T05:14:11.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.234+0000 7f8619600640 1 -- 172.21.15.43:0/727421166 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8586+0+0 (secure 0 0 0) 0x7f8628004400 con 0x55e4d5dc6e50 2024-09-19T05:14:11.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.234+0000 7f8639611740 1 -- 172.21.15.43:0/727421166 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8620005500 con 0x55e4d5dc6e50 2024-09-19T05:14:11.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.234+0000 7f8619600640 1 -- 172.21.15.43:0/727421166 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8628004600 con 0x55e4d5dc6e50 2024-09-19T05:14:11.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.234+0000 7f8639611740 10 client.27419 fetch_fsmap learned FSMap version 29 2024-09-19T05:14:11.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.234+0000 7f8639611740 1 -- 172.21.15.43:0/727421166 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e4d5e412a0 con 0x55e4d5dc6e50 2024-09-19T05:14:11.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.234+0000 7f8619600640 1 -- 172.21.15.43:0/727421166 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8620005500 con 0x55e4d5dc6e50 2024-09-19T05:14:11.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.234+0000 7f8619600640 1 -- 172.21.15.43:0/727421166 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f862802e6d0 con 0x55e4d5dc6e50 2024-09-19T05:14:11.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.234+0000 7f8639611740 10 client.27419 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:14:11.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.234+0000 7f8639611740 10 client.27419 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-19T05:14:11.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.234+0000 7f8639611740 20 client.27419 populate_metadata read hostname 'smithi043' 2024-09-19T05:14:11.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.234+0000 7f8639611740 1 --2- 172.21.15.43:0/727421166 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55e4d5e41820 0x55e4d5e43c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:11.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.234+0000 7f8639611740 4 client.27419 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:14:11.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.234+0000 7f8639611740 1 -- 172.21.15.43:0/727421166 --> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e4d5d29e60 con 0x55e4d5e41820 2024-09-19T05:14:11.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.235+0000 7f862fe00640 1 --2- 172.21.15.43:0/727421166 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55e4d5e41820 0x55e4d5e43c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:11.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.235+0000 7f862fe00640 1 --2- 172.21.15.43:0/727421166 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55e4d5e41820 0x55e4d5e43c40 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7fb2ac22d7bd7659 server_cookie=fd46feae8adcbfe0 in_seq=0 out_seq=0 2024-09-19T05:14:11.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.235+0000 7f8619600640 10 client.27419 ms_handle_connect on v2:172.21.15.151:6836/3960378010 2024-09-19T05:14:11.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.235+0000 7f8619600640 1 -- 172.21.15.43:0/727421166 <== mds.4 v2:172.21.15.151:6836/3960378010 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e4d5d29e60 con 0x55e4d5e41820 2024-09-19T05:14:11.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.235+0000 7f8619600640 10 client.27419 handle_command_reply: tid=0 2024-09-19T05:14:11.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.235+0000 7f8639611740 1 client.27419 shutdown 2024-09-19T05:14:11.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.236+0000 7f8639611740 1 -- 172.21.15.43:0/727421166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d5dc6e50 msgr2=0x55e4d5dc0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:11.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.236+0000 7f8639611740 1 --2- 172.21.15.43:0/727421166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d5dc6e50 0x55e4d5dc0310 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8628011dd0 tx=0x7f8628009ce0 comp rx=0 tx=0).stop 2024-09-19T05:14:11.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.236+0000 7f8639611740 1 -- 172.21.15.43:0/727421166 shutdown_connections 2024-09-19T05:14:11.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.236+0000 7f8639611740 1 --2- 172.21.15.43:0/727421166 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x55e4d5e41820 0x55e4d5e43c40 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:11.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.236+0000 7f8639611740 1 --2- 172.21.15.43:0/727421166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d5dc6e50 0x55e4d5dc0310 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:11.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.236+0000 7f8639611740 1 --2- 172.21.15.43:0/727421166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d5dbc590 0x55e4d5dcf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:11.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.236+0000 7f8639611740 1 --2- 172.21.15.43:0/727421166 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d5dbbc70 0x55e4d5dcedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:11.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.236+0000 7f8619600640 0 client.27419 ms_handle_reset on v2:172.21.15.151:6836/3960378010 2024-09-19T05:14:11.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.236+0000 7f8639611740 1 -- 172.21.15.43:0/727421166 >> 172.21.15.43:0/727421166 conn(0x55e4d5db8200 msgr2=0x55e4d5db6b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:11.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.236+0000 7f8639611740 1 -- 172.21.15.43:0/727421166 shutdown_connections 2024-09-19T05:14:11.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.236+0000 7f8639611740 1 -- 172.21.15.43:0/727421166 wait complete. 2024-09-19T05:14:11.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.236+0000 7f8639611740 20 client.27419 trim_cache size 0 max 16384 2024-09-19T05:14:11.242 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:14:11.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.237+0000 7f8636c00640 1 -- 172.21.15.43:0/871078837 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f86140a33b0 msgr2=0x7f86140a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:11.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.237+0000 7f8636c00640 1 --2- 172.21.15.43:0/871078837 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f86140a33b0 0x7f86140a5870 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x55e4d5db9060 tx=0x7f8620005f90 comp rx=0 tx=0).stop 2024-09-19T05:14:11.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.237+0000 7f8636c00640 1 -- 172.21.15.43:0/871078837 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8630077040 msgr2=0x7f86301a4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:11.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.237+0000 7f8636c00640 1 --2- 172.21.15.43:0/871078837 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8630077040 0x7f86301a4c80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f861c004770 tx=0x7f861c003c10 comp rx=0 tx=0).stop 2024-09-19T05:14:11.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.237+0000 7f8636c00640 1 -- 172.21.15.43:0/871078837 shutdown_connections 2024-09-19T05:14:11.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.237+0000 7f8636c00640 1 --2- 172.21.15.43:0/871078837 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f86140a33b0 0x7f86140a5870 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:11.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.237+0000 7f8636c00640 1 --2- 172.21.15.43:0/871078837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8630112a30 0x7f863019e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:11.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.237+0000 7f8636c00640 1 --2- 172.21.15.43:0/871078837 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8630077040 0x7f86301a4c80 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:11.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.237+0000 7f8636c00640 1 --2- 172.21.15.43:0/871078837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8630076720 0x7f86301a4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:11.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.237+0000 7f8636c00640 1 -- 172.21.15.43:0/871078837 >> 172.21.15.43:0/871078837 conn(0x7f86300ffa90 msgr2=0x7f8630101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:11.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.237+0000 7f8636c00640 1 -- 172.21.15.43:0/871078837 shutdown_connections 2024-09-19T05:14:11.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:11.237+0000 7f8636c00640 1 -- 172.21.15.43:0/871078837 wait complete. 2024-09-19T05:14:12.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:11 smithi043 ceph-mon[30800]: pgmap v296: 97 pgs: 12 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 80 active+clean; 294 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T05:14:12.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:11 smithi151 ceph-mon[35425]: pgmap v296: 97 pgs: 12 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 80 active+clean; 294 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T05:14:12.414 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T05:14:12.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:11 smithi096 ceph-mon[35085]: pgmap v296: 97 pgs: 12 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 80 active+clean; 294 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T05:14:12.797 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:14:14.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.016+0000 7f0004800640 1 -- 172.21.15.43:0/4261853354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efffc074ee0 msgr2=0x7efffc10def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:14.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.016+0000 7f0004800640 1 --2- 172.21.15.43:0/4261853354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efffc074ee0 0x7efffc10def0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7effec009860 tx=0x7effec02a3d0 comp rx=0 tx=0).stop 2024-09-19T05:14:14.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.016+0000 7f0004800640 1 -- 172.21.15.43:0/4261853354 shutdown_connections 2024-09-19T05:14:14.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.016+0000 7f0004800640 1 --2- 172.21.15.43:0/4261853354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efffc10e430 0x7efffc110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:14.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.016+0000 7f0004800640 1 --2- 172.21.15.43:0/4261853354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efffc074ee0 0x7efffc10def0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:14.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.016+0000 7f0004800640 1 --2- 172.21.15.43:0/4261853354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efffc0745c0 0x7efffc0749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:14.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.016+0000 7f0004800640 1 -- 172.21.15.43:0/4261853354 >> 172.21.15.43:0/4261853354 conn(0x7efffc0fd8b0 msgr2=0x7efffc0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:14.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.016+0000 7f0004800640 1 -- 172.21.15.43:0/4261853354 shutdown_connections 2024-09-19T05:14:14.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.016+0000 7f0004800640 1 -- 172.21.15.43:0/4261853354 wait complete. 2024-09-19T05:14:14.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.017+0000 7f0004800640 1 Processor -- start 2024-09-19T05:14:14.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.017+0000 7f0004800640 1 -- start start 2024-09-19T05:14:14.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.018+0000 7f0004800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efffc0745c0 0x7efffc19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:14.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.018+0000 7f0004800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efffc074ee0 0x7efffc19ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:14.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.018+0000 7f0004800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efffc10e430 0x7efffc19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:14.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.018+0000 7f0004800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efffc1141e0 con 0x7efffc10e430 2024-09-19T05:14:14.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.018+0000 7f0004800640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efffc114060 con 0x7efffc074ee0 2024-09-19T05:14:14.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.018+0000 7f0004800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efffc114360 con 0x7efffc0745c0 2024-09-19T05:14:14.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.018+0000 7f0002800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efffc10e430 0x7efffc19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:14.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.018+0000 7f0001400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efffc074ee0 0x7efffc19ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:14.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.018+0000 7f0001e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efffc0745c0 0x7efffc19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:14.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.018+0000 7f0001e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efffc0745c0 0x7efffc19a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:59468/0 (socket says 172.21.15.43:59468) 2024-09-19T05:14:14.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.018+0000 7f0001e00640 1 -- 172.21.15.43:0/779693697 learned_addr learned my addr 172.21.15.43:0/779693697 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:14.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.019+0000 7f0001e00640 1 -- 172.21.15.43:0/779693697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efffc074ee0 msgr2=0x7efffc19ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:14.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.019+0000 7f0001e00640 1 --2- 172.21.15.43:0/779693697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efffc074ee0 0x7efffc19ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:14.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.019+0000 7f0001e00640 1 -- 172.21.15.43:0/779693697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efffc10e430 msgr2=0x7efffc19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:14.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.019+0000 7f0001e00640 1 --2- 172.21.15.43:0/779693697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efffc10e430 0x7efffc19ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:14.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.019+0000 7f0001e00640 1 -- 172.21.15.43:0/779693697 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efffc19f670 con 0x7efffc0745c0 2024-09-19T05:14:14.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.019+0000 7f0002800640 1 --2- 172.21.15.43:0/779693697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efffc10e430 0x7efffc19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:14.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.019+0000 7f0001e00640 1 --2- 172.21.15.43:0/779693697 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efffc0745c0 0x7efffc19a6c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7efff000e9f0 tx=0x7efff000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:14.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.019+0000 7efff6a00640 1 -- 172.21.15.43:0/779693697 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff000cd80 con 0x7efffc0745c0 2024-09-19T05:14:14.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.019+0000 7efff6a00640 1 -- 172.21.15.43:0/779693697 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efff0005500 con 0x7efffc0745c0 2024-09-19T05:14:14.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.019+0000 7efff6a00640 1 -- 172.21.15.43:0/779693697 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff0010690 con 0x7efffc0745c0 2024-09-19T05:14:14.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.019+0000 7f0004800640 1 -- 172.21.15.43:0/779693697 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efffc19f960 con 0x7efffc0745c0 2024-09-19T05:14:14.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.019+0000 7f0004800640 1 -- 172.21.15.43:0/779693697 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efffc1a7240 con 0x7efffc0745c0 2024-09-19T05:14:14.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.022+0000 7efff6a00640 1 -- 172.21.15.43:0/779693697 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7efff00046c0 con 0x7efffc0745c0 2024-09-19T05:14:14.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.022+0000 7f000728b740 1 Processor -- start 2024-09-19T05:14:14.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.023+0000 7f000728b740 1 -- start start 2024-09-19T05:14:14.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.023+0000 7f000728b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d9cbbdc70 0x562d9cbbe050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:14.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.023+0000 7f000728b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d9cbbe590 0x562d9cbc8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:14.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.023+0000 7f000728b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d9cbc8e50 0x562d9cbcb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:14.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.023+0000 7f000728b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562d9cb243e0 con 0x562d9cbbdc70 2024-09-19T05:14:14.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.023+0000 7f000728b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562d9cacbd80 con 0x562d9cbc8e50 2024-09-19T05:14:14.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.023+0000 7f000728b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562d9cb203a0 con 0x562d9cbbe590 2024-09-19T05:14:14.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.023+0000 7f0002800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d9cbbdc70 0x562d9cbbe050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:14.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.023+0000 7f0002800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d9cbbdc70 0x562d9cbbe050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50070/0 (socket says 172.21.15.43:50070) 2024-09-19T05:14:14.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.023+0000 7f0002800640 1 -- 172.21.15.43:0/370164804 learned_addr learned my addr 172.21.15.43:0/370164804 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:14.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.023+0000 7f0001400640 1 --2- 172.21.15.43:0/370164804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d9cbc8e50 0x562d9cbcb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:14.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.024+0000 7f0002800640 1 -- 172.21.15.43:0/370164804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d9cbbe590 msgr2=0x562d9cbc8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:14.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.024+0000 7f0002800640 1 --2- 172.21.15.43:0/370164804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d9cbbe590 0x562d9cbc8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:14.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.024+0000 7f0002800640 1 -- 172.21.15.43:0/370164804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d9cbc8e50 msgr2=0x562d9cbcb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:14.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.024+0000 7f0002800640 1 --2- 172.21.15.43:0/370164804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d9cbc8e50 0x562d9cbcb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:14.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.024+0000 7f0002800640 1 -- 172.21.15.43:0/370164804 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d9caeaae0 con 0x562d9cbbdc70 2024-09-19T05:14:14.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.024+0000 7f0002800640 1 --2- 172.21.15.43:0/370164804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d9cbbdc70 0x562d9cbbe050 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7efff8009a60 tx=0x7efff800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4ae6a367ac8645f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:14.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.024+0000 7efff6a00640 1 --2- 172.21.15.43:0/779693697 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7effe40a33b0 0x7effe40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:14.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.024+0000 7efff6a00640 1 -- 172.21.15.43:0/779693697 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8586+0+0 (secure 0 0 0) 0x7efff00d0140 con 0x7efffc0745c0 2024-09-19T05:14:14.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.024+0000 7effeb000640 1 -- 172.21.15.43:0/370164804 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff800cce0 con 0x562d9cbbdc70 2024-09-19T05:14:14.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.025+0000 7effeb000640 1 -- 172.21.15.43:0/370164804 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efff801d070 con 0x562d9cbbdc70 2024-09-19T05:14:14.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.025+0000 7effeb000640 1 -- 172.21.15.43:0/370164804 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff800b5c0 con 0x562d9cbbdc70 2024-09-19T05:14:14.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.025+0000 7f0001e00640 1 --2- 172.21.15.43:0/779693697 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7effe40a33b0 0x7effe40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:14.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.025+0000 7f000728b740 1 -- 172.21.15.43:0/370164804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d9cbbdc70 msgr2=0x562d9cbbe050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:14.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.025+0000 7f000728b740 1 --2- 172.21.15.43:0/370164804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d9cbbdc70 0x562d9cbbe050 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7efff8009a60 tx=0x7efff800ec30 comp rx=0 tx=0).stop 2024-09-19T05:14:14.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.025+0000 7f0001e00640 1 --2- 172.21.15.43:0/779693697 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7effe40a33b0 0x7effe40a5870 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x562d9cbbb060 tx=0x7efff0009310 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:14.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.025+0000 7f000728b740 1 -- 172.21.15.43:0/370164804 shutdown_connections 2024-09-19T05:14:14.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.025+0000 7f000728b740 1 --2- 172.21.15.43:0/370164804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d9cbc8e50 0x562d9cbcb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:14.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.025+0000 7f000728b740 1 --2- 172.21.15.43:0/370164804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d9cbbe590 0x562d9cbc8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:14.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.025+0000 7f000728b740 1 --2- 172.21.15.43:0/370164804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d9cbbdc70 0x562d9cbbe050 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:14.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.025+0000 7f000728b740 1 -- 172.21.15.43:0/370164804 >> 172.21.15.43:0/370164804 conn(0x562d9cbba200 msgr2=0x562d9cbba5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:14.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.025+0000 7f000728b740 1 -- 172.21.15.43:0/370164804 shutdown_connections 2024-09-19T05:14:14.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.026+0000 7f000728b740 1 -- 172.21.15.43:0/370164804 wait complete. 2024-09-19T05:14:14.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.026+0000 7f000728b740 1 -- start start 2024-09-19T05:14:14.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.026+0000 7f000728b740 1 Processor -- start 2024-09-19T05:14:14.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.027+0000 7f000728b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d9cbbdc70 0x562d9cbc5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:14.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.027+0000 7f000728b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d9cbbe590 0x562d9cbc6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:14.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.027+0000 7f000728b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d9cbc8e50 0x562d9cbc67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:14.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.027+0000 7f000728b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562d9cbcba20 con 0x562d9cbbdc70 2024-09-19T05:14:14.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.027+0000 7f000728b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562d9cadf0f0 con 0x562d9cbc8e50 2024-09-19T05:14:14.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.027+0000 7f000728b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562d9cbcbba0 con 0x562d9cbbe590 2024-09-19T05:14:14.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.027+0000 7f000728b740 10 client.0 _ll_register_callbacks cb 0x562d9cac6bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:14.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.027+0000 7f0002800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d9cbbdc70 0x562d9cbc5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:14.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.027+0000 7f0001400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d9cbbe590 0x562d9cbc6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:14.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.027+0000 7f0002800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d9cbbdc70 0x562d9cbc5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50084/0 (socket says 172.21.15.43:50084) 2024-09-19T05:14:14.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.027+0000 7f0001400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d9cbbe590 0x562d9cbc6280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:59492/0 (socket says 172.21.15.43:59492) 2024-09-19T05:14:14.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.027+0000 7f0002800640 1 -- 172.21.15.43:0/189672208 learned_addr learned my addr 172.21.15.43:0/189672208 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:14.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.027+0000 7f0002800640 1 --2- 172.21.15.43:0/189672208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d9cbc8e50 0x562d9cbc67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:14.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.028+0000 7f0001400640 1 -- 172.21.15.43:0/189672208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d9cbc8e50 msgr2=0x562d9cbc67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:14.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.028+0000 7f0001400640 1 --2- 172.21.15.43:0/189672208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d9cbc8e50 0x562d9cbc67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:14.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.028+0000 7f0001400640 1 -- 172.21.15.43:0/189672208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d9cbbdc70 msgr2=0x562d9cbc5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:14.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.028+0000 7f0001400640 1 --2- 172.21.15.43:0/189672208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d9cbbdc70 0x562d9cbc5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:14.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.028+0000 7f0001400640 1 -- 172.21.15.43:0/189672208 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d9c8953c0 con 0x562d9cbbe590 2024-09-19T05:14:14.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.028+0000 7f0001400640 1 --2- 172.21.15.43:0/189672208 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d9cbbe590 0x562d9cbc6280 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7effec009830 tx=0x7effec0305d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7527b12f2d5b9d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:14.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.028+0000 7effdf400640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:14:14.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.028+0000 7effdf400640 1 -- 172.21.15.43:0/189672208 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec0308d0 con 0x562d9cbbe590 2024-09-19T05:14:14.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.028+0000 7effdf400640 1 -- 172.21.15.43:0/189672208 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8586+0+0 (secure 0 0 0) 0x7effec004050 con 0x562d9cbbe590 2024-09-19T05:14:14.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.028+0000 7f000728b740 1 -- 172.21.15.43:0/189672208 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efff801d070 con 0x562d9cbbe590 2024-09-19T05:14:14.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.029+0000 7effdf400640 1 -- 172.21.15.43:0/189672208 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7effec04e050 con 0x562d9cbbe590 2024-09-19T05:14:14.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.029+0000 7f000728b740 10 client.27328 fetch_fsmap learned FSMap version 29 2024-09-19T05:14:14.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.029+0000 7f000728b740 1 -- 172.21.15.43:0/189672208 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562d9cbd7a60 con 0x562d9cbbe590 2024-09-19T05:14:14.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.029+0000 7effdf400640 1 -- 172.21.15.43:0/189672208 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efff801d070 con 0x562d9cbbe590 2024-09-19T05:14:14.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.030+0000 7effdf400640 1 -- 172.21.15.43:0/189672208 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7effec004880 con 0x562d9cbbe590 2024-09-19T05:14:14.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.030+0000 7f000728b740 10 client.27328 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:14:14.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.030+0000 7f000728b740 10 client.27328 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T05:14:14.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.030+0000 7f000728b740 20 client.27328 populate_metadata read hostname 'smithi043' 2024-09-19T05:14:14.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.030+0000 7f000728b740 1 --2- 172.21.15.43:0/189672208 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x562d9cbd8e70 0x562d9cbdb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:14.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.030+0000 7f000728b740 4 client.27328 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:14:14.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.030+0000 7f000728b740 1 -- 172.21.15.43:0/189672208 --> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d9cb31950 con 0x562d9cbd8e70 2024-09-19T05:14:14.035 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:14 smithi043 ceph-mon[30800]: pgmap v297: 97 pgs: 12 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 80 active+clean; 294 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T05:14:14.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.031+0000 7f0002800640 1 --2- 172.21.15.43:0/189672208 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x562d9cbd8e70 0x562d9cbdb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:14.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.031+0000 7f0002800640 1 --2- 172.21.15.43:0/189672208 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x562d9cbd8e70 0x562d9cbdb260 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3366b87aec18fc94 server_cookie=19cc059e5bda176e in_seq=0 out_seq=0 2024-09-19T05:14:14.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.031+0000 7effdf400640 10 client.27328 ms_handle_connect on v2:172.21.15.96:6836/3978388286 2024-09-19T05:14:14.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.032+0000 7effdf400640 1 -- 172.21.15.43:0/189672208 <== mds.3 v2:172.21.15.96:6836/3978388286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562d9cb31950 con 0x562d9cbd8e70 2024-09-19T05:14:14.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.032+0000 7effdf400640 10 client.27328 handle_command_reply: tid=0 2024-09-19T05:14:14.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.032+0000 7f000728b740 1 client.27328 shutdown 2024-09-19T05:14:14.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.032+0000 7f000728b740 1 -- 172.21.15.43:0/189672208 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d9cbbe590 msgr2=0x562d9cbc6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:14.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.032+0000 7f000728b740 1 --2- 172.21.15.43:0/189672208 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d9cbbe590 0x562d9cbc6280 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7effec009830 tx=0x7effec0305d0 comp rx=0 tx=0).stop 2024-09-19T05:14:14.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.032+0000 7f000728b740 1 -- 172.21.15.43:0/189672208 shutdown_connections 2024-09-19T05:14:14.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.032+0000 7f000728b740 1 --2- 172.21.15.43:0/189672208 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x562d9cbd8e70 0x562d9cbdb260 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:14.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.032+0000 7f000728b740 1 --2- 172.21.15.43:0/189672208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d9cbc8e50 0x562d9cbc67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:14.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.032+0000 7f000728b740 1 --2- 172.21.15.43:0/189672208 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d9cbbe590 0x562d9cbc6280 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:14.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.032+0000 7f000728b740 1 --2- 172.21.15.43:0/189672208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d9cbbdc70 0x562d9cbc5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:14.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.032+0000 7f000728b740 1 -- 172.21.15.43:0/189672208 >> 172.21.15.43:0/189672208 conn(0x562d9cbba200 msgr2=0x562d9cbbfee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:14.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.032+0000 7effdf400640 0 client.27328 ms_handle_reset on v2:172.21.15.96:6836/3978388286 2024-09-19T05:14:14.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.033+0000 7f000728b740 1 -- 172.21.15.43:0/189672208 shutdown_connections 2024-09-19T05:14:14.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.033+0000 7f000728b740 1 -- 172.21.15.43:0/189672208 wait complete. 2024-09-19T05:14:14.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.033+0000 7f000728b740 20 client.27328 trim_cache size 0 max 16384 2024-09-19T05:14:14.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.047+0000 7f000728b740 1 Processor -- start 2024-09-19T05:14:14.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.047+0000 7f000728b740 1 -- start start 2024-09-19T05:14:14.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.047+0000 7f000728b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d9cbbdc70 0x562d9cbc06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:14.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.047+0000 7f000728b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d9cbbe590 0x562d9cbc7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:14.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.047+0000 7f000728b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d9cbc8e50 0x562d9cbe0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:14.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.047+0000 7f000728b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efffc1afbb0 con 0x562d9cbbe590 2024-09-19T05:14:14.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.048+0000 7f000728b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efffc1b16f0 con 0x562d9cbbdc70 2024-09-19T05:14:14.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.048+0000 7f000728b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efffc1b0280 con 0x562d9cbc8e50 2024-09-19T05:14:14.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.048+0000 7f0002800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d9cbbdc70 0x562d9cbc06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:14.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.048+0000 7f0002800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d9cbbdc70 0x562d9cbc06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:40130/0 (socket says 172.21.15.43:40130) 2024-09-19T05:14:14.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.048+0000 7f0002800640 1 -- 172.21.15.43:0/2226587204 learned_addr learned my addr 172.21.15.43:0/2226587204 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:14.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.048+0000 7f0002800640 1 --2- 172.21.15.43:0/2226587204 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d9cbc8e50 0x562d9cbe0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:14.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.048+0000 7f0001400640 1 --2- 172.21.15.43:0/2226587204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d9cbbe590 0x562d9cbc7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:14.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.049+0000 7f0002800640 1 -- 172.21.15.43:0/2226587204 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d9cbc8e50 msgr2=0x562d9cbe0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:14.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.049+0000 7f0002800640 1 --2- 172.21.15.43:0/2226587204 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d9cbc8e50 0x562d9cbe0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:14.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.049+0000 7f0002800640 1 -- 172.21.15.43:0/2226587204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d9cbbe590 msgr2=0x562d9cbc7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:14.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.049+0000 7f0002800640 1 --2- 172.21.15.43:0/2226587204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d9cbbe590 0x562d9cbc7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:14.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.049+0000 7f0002800640 1 -- 172.21.15.43:0/2226587204 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d9cbdb8c0 con 0x562d9cbbdc70 2024-09-19T05:14:14.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.049+0000 7f0001400640 1 --2- 172.21.15.43:0/2226587204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d9cbbe590 0x562d9cbc7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:14:14.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.049+0000 7f0002800640 1 --2- 172.21.15.43:0/2226587204 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d9cbbdc70 0x562d9cbc06d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7efff8005bd0 tx=0x7efff8018b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f85fb85cc706b2ca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:14.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.049+0000 7effea600640 1 -- 172.21.15.43:0/2226587204 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff8004c00 con 0x562d9cbbdc70 2024-09-19T05:14:14.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.050+0000 7effea600640 1 -- 172.21.15.43:0/2226587204 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efff801d040 con 0x562d9cbbdc70 2024-09-19T05:14:14.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.050+0000 7effea600640 1 -- 172.21.15.43:0/2226587204 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff80054e0 con 0x562d9cbbdc70 2024-09-19T05:14:14.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.050+0000 7f000728b740 1 -- 172.21.15.43:0/2226587204 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d9cbbdc70 msgr2=0x562d9cbc06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:14.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.050+0000 7f000728b740 1 --2- 172.21.15.43:0/2226587204 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d9cbbdc70 0x562d9cbc06d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7efff8005bd0 tx=0x7efff8018b70 comp rx=0 tx=0).stop 2024-09-19T05:14:14.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.050+0000 7f000728b740 1 -- 172.21.15.43:0/2226587204 shutdown_connections 2024-09-19T05:14:14.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.050+0000 7f000728b740 1 --2- 172.21.15.43:0/2226587204 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d9cbc8e50 0x562d9cbe0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:14.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.050+0000 7f000728b740 1 --2- 172.21.15.43:0/2226587204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d9cbbe590 0x562d9cbc7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:14.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.050+0000 7f000728b740 1 --2- 172.21.15.43:0/2226587204 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d9cbbdc70 0x562d9cbc06d0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:14.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.050+0000 7f000728b740 1 -- 172.21.15.43:0/2226587204 >> 172.21.15.43:0/2226587204 conn(0x562d9cbba200 msgr2=0x562d9cbc75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:14.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.051+0000 7f000728b740 1 -- 172.21.15.43:0/2226587204 shutdown_connections 2024-09-19T05:14:14.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.051+0000 7f000728b740 1 -- 172.21.15.43:0/2226587204 wait complete. 2024-09-19T05:14:14.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.051+0000 7f000728b740 1 -- start start 2024-09-19T05:14:14.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.051+0000 7f000728b740 1 Processor -- start 2024-09-19T05:14:14.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.052+0000 7f000728b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d9cbbdc70 0x562d9cbd0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:14.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.052+0000 7f000728b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d9cbbe590 0x562d9cbd1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:14.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.052+0000 7f000728b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d9cbc8e50 0x562d9cbcddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:14.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.052+0000 7f000728b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efffc1b1020 con 0x562d9cbbe590 2024-09-19T05:14:14.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.052+0000 7f000728b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efffc1b0950 con 0x562d9cbbdc70 2024-09-19T05:14:14.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.052+0000 7f000728b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efffc1adbb0 con 0x562d9cbc8e50 2024-09-19T05:14:14.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.052+0000 7f000728b740 10 client.0 _ll_register_callbacks cb 0x7effec0096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:14.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.052+0000 7f0002800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d9cbbdc70 0x562d9cbd0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:14.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.052+0000 7f0001400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d9cbbe590 0x562d9cbd1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:14.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.052+0000 7f0002800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d9cbbdc70 0x562d9cbd0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:40132/0 (socket says 172.21.15.43:40132) 2024-09-19T05:14:14.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.052+0000 7f0002800640 1 -- 172.21.15.43:0/2925086688 learned_addr learned my addr 172.21.15.43:0/2925086688 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:14.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.052+0000 7f0002800640 1 --2- 172.21.15.43:0/2925086688 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d9cbc8e50 0x562d9cbcddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:14.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.053+0000 7f0001400640 1 -- 172.21.15.43:0/2925086688 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d9cbc8e50 msgr2=0x562d9cbcddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:14.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.053+0000 7f0001400640 1 --2- 172.21.15.43:0/2925086688 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d9cbc8e50 0x562d9cbcddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:14.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.053+0000 7f0001400640 1 -- 172.21.15.43:0/2925086688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d9cbbdc70 msgr2=0x562d9cbd0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:14.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.053+0000 7f0001400640 1 --2- 172.21.15.43:0/2925086688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d9cbbdc70 0x562d9cbd0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:14.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.053+0000 7f0001400640 1 -- 172.21.15.43:0/2925086688 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d9cbb5770 con 0x562d9cbbe590 2024-09-19T05:14:14.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.053+0000 7f0002800640 1 --2- 172.21.15.43:0/2925086688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d9cbbdc70 0x562d9cbd0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:14.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.053+0000 7f0001400640 1 --2- 172.21.15.43:0/2925086688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d9cbbe590 0x562d9cbd1330 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x562d9cbdb770 tx=0x7effec0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6f82d8d431010cf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:14.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.053+0000 7effdf400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:14:14.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.053+0000 7effdf400640 1 -- 172.21.15.43:0/2925086688 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec057060 con 0x562d9cbbe590 2024-09-19T05:14:14.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.053+0000 7effdf400640 1 -- 172.21.15.43:0/2925086688 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8586+0+0 (secure 0 0 0) 0x7effec005000 con 0x562d9cbbe590 2024-09-19T05:14:14.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.053+0000 7f000728b740 1 -- 172.21.15.43:0/2925086688 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efff801d040 con 0x562d9cbbe590 2024-09-19T05:14:14.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.054+0000 7effdf400640 1 -- 172.21.15.43:0/2925086688 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7effec0035f0 con 0x562d9cbbe590 2024-09-19T05:14:14.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.054+0000 7f000728b740 10 client.17409 fetch_fsmap learned FSMap version 29 2024-09-19T05:14:14.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.054+0000 7f000728b740 1 -- 172.21.15.43:0/2925086688 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562d9cbbdaa0 con 0x562d9cbbe590 2024-09-19T05:14:14.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.054+0000 7effdf400640 1 -- 172.21.15.43:0/2925086688 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efff801d040 con 0x562d9cbbe590 2024-09-19T05:14:14.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.054+0000 7effdf400640 1 -- 172.21.15.43:0/2925086688 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7effec048480 con 0x562d9cbbe590 2024-09-19T05:14:14.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.054+0000 7f000728b740 10 client.17409 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:14:14.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.054+0000 7f000728b740 10 client.17409 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T05:14:14.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.054+0000 7f000728b740 20 client.17409 populate_metadata read hostname 'smithi043' 2024-09-19T05:14:14.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.054+0000 7f000728b740 1 --2- 172.21.15.43:0/2925086688 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x562d9cc442b0 0x562d9cc466d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:14.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.054+0000 7f000728b740 4 client.17409 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:14:14.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.054+0000 7f000728b740 1 -- 172.21.15.43:0/2925086688 --> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562d9cb2be60 con 0x562d9cc442b0 2024-09-19T05:14:14.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.055+0000 7f0002800640 1 --2- 172.21.15.43:0/2925086688 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x562d9cc442b0 0x562d9cc466d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:14.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.055+0000 7f0002800640 1 --2- 172.21.15.43:0/2925086688 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x562d9cc442b0 0x562d9cc466d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1c7d2b4cc9574a22 server_cookie=e9dea07d067e1d29 in_seq=0 out_seq=0 2024-09-19T05:14:14.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.055+0000 7effdf400640 10 client.17409 ms_handle_connect on v2:172.21.15.96:6836/3978388286 2024-09-19T05:14:14.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.056+0000 7effdf400640 1 -- 172.21.15.43:0/2925086688 <== mds.3 v2:172.21.15.96:6836/3978388286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562d9cb2be60 con 0x562d9cc442b0 2024-09-19T05:14:14.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.056+0000 7effdf400640 10 client.17409 handle_command_reply: tid=0 2024-09-19T05:14:14.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.056+0000 7f000728b740 1 client.17409 shutdown 2024-09-19T05:14:14.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.056+0000 7f000728b740 1 -- 172.21.15.43:0/2925086688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d9cbbe590 msgr2=0x562d9cbd1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:14.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.056+0000 7f000728b740 1 --2- 172.21.15.43:0/2925086688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d9cbbe590 0x562d9cbd1330 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x562d9cbdb770 tx=0x7effec0048f0 comp rx=0 tx=0).stop 2024-09-19T05:14:14.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.056+0000 7f000728b740 1 -- 172.21.15.43:0/2925086688 shutdown_connections 2024-09-19T05:14:14.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.056+0000 7f000728b740 1 --2- 172.21.15.43:0/2925086688 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x562d9cc442b0 0x562d9cc466d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:14.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.056+0000 7f000728b740 1 --2- 172.21.15.43:0/2925086688 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d9cbc8e50 0x562d9cbcddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:14.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.056+0000 7f000728b740 1 --2- 172.21.15.43:0/2925086688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d9cbbe590 0x562d9cbd1330 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:14.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.056+0000 7f000728b740 1 --2- 172.21.15.43:0/2925086688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d9cbbdc70 0x562d9cbd0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:14.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.056+0000 7f000728b740 1 -- 172.21.15.43:0/2925086688 >> 172.21.15.43:0/2925086688 conn(0x562d9cbba200 msgr2=0x562d9cbc7dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:14.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.056+0000 7effdf400640 0 client.17409 ms_handle_reset on v2:172.21.15.96:6836/3978388286 2024-09-19T05:14:14.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.056+0000 7f000728b740 1 -- 172.21.15.43:0/2925086688 shutdown_connections 2024-09-19T05:14:14.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.056+0000 7f000728b740 1 -- 172.21.15.43:0/2925086688 wait complete. 2024-09-19T05:14:14.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.056+0000 7f000728b740 20 client.17409 trim_cache size 0 max 16384 2024-09-19T05:14:14.062 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:14:14.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.057+0000 7f0004800640 1 -- 172.21.15.43:0/779693697 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7effe40a33b0 msgr2=0x7effe40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:14.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.057+0000 7f0004800640 1 --2- 172.21.15.43:0/779693697 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7effe40a33b0 0x7effe40a5870 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x562d9cbbb060 tx=0x7efff0009310 comp rx=0 tx=0).stop 2024-09-19T05:14:14.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.057+0000 7f0004800640 1 -- 172.21.15.43:0/779693697 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efffc0745c0 msgr2=0x7efffc19a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:14.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.057+0000 7f0004800640 1 --2- 172.21.15.43:0/779693697 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efffc0745c0 0x7efffc19a6c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7efff000e9f0 tx=0x7efff000edf0 comp rx=0 tx=0).stop 2024-09-19T05:14:14.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.057+0000 7f0004800640 1 -- 172.21.15.43:0/779693697 shutdown_connections 2024-09-19T05:14:14.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.057+0000 7f0004800640 1 --2- 172.21.15.43:0/779693697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efffc10e430 0x7efffc19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:14.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.057+0000 7f0004800640 1 --2- 172.21.15.43:0/779693697 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7effe40a33b0 0x7effe40a5870 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:14.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.057+0000 7f0004800640 1 --2- 172.21.15.43:0/779693697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efffc074ee0 0x7efffc19ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:14.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.057+0000 7f0004800640 1 --2- 172.21.15.43:0/779693697 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efffc0745c0 0x7efffc19a6c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:14.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.057+0000 7f0004800640 1 -- 172.21.15.43:0/779693697 >> 172.21.15.43:0/779693697 conn(0x7efffc0fd8b0 msgr2=0x7efffc0fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:14.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.057+0000 7f0004800640 1 -- 172.21.15.43:0/779693697 shutdown_connections 2024-09-19T05:14:14.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:14.057+0000 7f0004800640 1 -- 172.21.15.43:0/779693697 wait complete. 2024-09-19T05:14:14.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:14 smithi151 ceph-mon[35425]: pgmap v297: 97 pgs: 12 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 80 active+clean; 294 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T05:14:14.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:14 smithi096 ceph-mon[35085]: pgmap v297: 97 pgs: 12 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 80 active+clean; 294 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T05:14:14.808 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T05:14:15.186 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:14:15.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.761+0000 7f4091000640 1 -- 172.21.15.43:0/2047706549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f408c101de0 msgr2=0x7f408c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:15.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.761+0000 7f4091000640 1 --2- 172.21.15.43:0/2047706549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f408c101de0 0x7f408c1021c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f407c009900 tx=0x7f407c02a3f0 comp rx=0 tx=0).stop 2024-09-19T05:14:15.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.761+0000 7f4091000640 1 -- 172.21.15.43:0/2047706549 shutdown_connections 2024-09-19T05:14:15.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.761+0000 7f4091000640 1 --2- 172.21.15.43:0/2047706549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f408c106f10 0x7f408c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:15.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.761+0000 7f4091000640 1 --2- 172.21.15.43:0/2047706549 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f408c102790 0x7f408c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:15.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.761+0000 7f4091000640 1 --2- 172.21.15.43:0/2047706549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f408c101de0 0x7f408c1021c0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:15.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.761+0000 7f4091000640 1 -- 172.21.15.43:0/2047706549 >> 172.21.15.43:0/2047706549 conn(0x7f408c0fd8c0 msgr2=0x7f408c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:15.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.761+0000 7f4091000640 1 -- 172.21.15.43:0/2047706549 shutdown_connections 2024-09-19T05:14:15.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.761+0000 7f4091000640 1 -- 172.21.15.43:0/2047706549 wait complete. 2024-09-19T05:14:15.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.762+0000 7f4091000640 1 Processor -- start 2024-09-19T05:14:15.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.762+0000 7f4091000640 1 -- start start 2024-09-19T05:14:15.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.763+0000 7f4091000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f408c101de0 0x7f408c19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:15.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.763+0000 7f4091000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f408c102790 0x7f408c19ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:15.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.763+0000 7f4091000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f408c106f10 0x7f408c19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:15.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.763+0000 7f4091000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f408c111f30 con 0x7f408c101de0 2024-09-19T05:14:15.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.763+0000 7f4091000640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f408c111db0 con 0x7f408c106f10 2024-09-19T05:14:15.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.763+0000 7f4091000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f408c1120b0 con 0x7f408c102790 2024-09-19T05:14:15.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.763+0000 7f4089e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f408c101de0 0x7f408c19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:15.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.763+0000 7f4089e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f408c101de0 0x7f408c19a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44598/0 (socket says 172.21.15.43:44598) 2024-09-19T05:14:15.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.763+0000 7f4089e00640 1 -- 172.21.15.43:0/2469666678 learned_addr learned my addr 172.21.15.43:0/2469666678 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:15.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.763+0000 7f4089400640 1 --2- 172.21.15.43:0/2469666678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f408c102790 0x7f408c19ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:15.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.763+0000 7f408a800640 1 --2- 172.21.15.43:0/2469666678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f408c106f10 0x7f408c19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:15.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.763+0000 7f4089e00640 1 -- 172.21.15.43:0/2469666678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f408c102790 msgr2=0x7f408c19ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:15.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.764+0000 7f4089e00640 1 --2- 172.21.15.43:0/2469666678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f408c102790 0x7f408c19ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:15.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.764+0000 7f4089e00640 1 -- 172.21.15.43:0/2469666678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f408c106f10 msgr2=0x7f408c19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:15.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.764+0000 7f4089e00640 1 --2- 172.21.15.43:0/2469666678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f408c106f10 0x7f408c19ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:15.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.764+0000 7f4089e00640 1 -- 172.21.15.43:0/2469666678 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f408c19f600 con 0x7f408c101de0 2024-09-19T05:14:15.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.764+0000 7f4089e00640 1 --2- 172.21.15.43:0/2469666678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f408c101de0 0x7f408c19a650 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f407c006010 tx=0x7f407c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:15.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.764+0000 7f4082a00640 1 -- 172.21.15.43:0/2469666678 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f407c03f030 con 0x7f408c101de0 2024-09-19T05:14:15.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.764+0000 7f4082a00640 1 -- 172.21.15.43:0/2469666678 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f407c003ce0 con 0x7f408c101de0 2024-09-19T05:14:15.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.764+0000 7f4082a00640 1 -- 172.21.15.43:0/2469666678 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f407c035070 con 0x7f408c101de0 2024-09-19T05:14:15.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.764+0000 7f4091000640 1 -- 172.21.15.43:0/2469666678 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f408c19f890 con 0x7f408c101de0 2024-09-19T05:14:15.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.764+0000 7f4091000640 1 -- 172.21.15.43:0/2469666678 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f408c1a71d0 con 0x7f408c101de0 2024-09-19T05:14:15.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.767+0000 7f4093a7c740 1 Processor -- start 2024-09-19T05:14:15.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.768+0000 7f4082a00640 1 -- 172.21.15.43:0/2469666678 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f407c042030 con 0x7f408c101de0 2024-09-19T05:14:15.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.768+0000 7f4093a7c740 1 -- start start 2024-09-19T05:14:15.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.768+0000 7f4093a7c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b67822ec70 0x55b67822f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:15.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.768+0000 7f4093a7c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b67822f590 0x55b678239910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:15.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.768+0000 7f4093a7c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b678239e50 0x55b67823c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:15.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.768+0000 7f4093a7c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b6781953e0 con 0x55b67822ec70 2024-09-19T05:14:15.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.768+0000 7f4093a7c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b67813cd80 con 0x55b67822f590 2024-09-19T05:14:15.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.768+0000 7f4093a7c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b6781913a0 con 0x55b678239e50 2024-09-19T05:14:15.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.768+0000 7f408a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b67822ec70 0x55b67822f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:15.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.768+0000 7f408a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b67822ec70 0x55b67822f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44600/0 (socket says 172.21.15.43:44600) 2024-09-19T05:14:15.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.768+0000 7f408a800640 1 -- 172.21.15.43:0/3975284984 learned_addr learned my addr 172.21.15.43:0/3975284984 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:15.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.769+0000 7f4089e00640 1 --2- 172.21.15.43:0/3975284984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b67822f590 0x55b678239910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:15.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.769+0000 7f4089400640 1 --2- 172.21.15.43:0/3975284984 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b678239e50 0x55b67823c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:15.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.769+0000 7f408a800640 1 -- 172.21.15.43:0/3975284984 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b678239e50 msgr2=0x55b67823c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:15.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.769+0000 7f408a800640 1 --2- 172.21.15.43:0/3975284984 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b678239e50 0x55b67823c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:15.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.769+0000 7f408a800640 1 -- 172.21.15.43:0/3975284984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b67822f590 msgr2=0x55b678239910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:15.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.769+0000 7f408a800640 1 --2- 172.21.15.43:0/3975284984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b67822f590 0x55b678239910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:15.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.769+0000 7f408a800640 1 -- 172.21.15.43:0/3975284984 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b67815bae0 con 0x55b67822ec70 2024-09-19T05:14:15.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.769+0000 7f408a800640 1 --2- 172.21.15.43:0/3975284984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b67822ec70 0x55b67822f050 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f4084009f70 tx=0x7f4084010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49cbf521336610d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:15.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.770+0000 7f4082a00640 1 --2- 172.21.15.43:0/2469666678 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f40700a3430 0x7f40700a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:15.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.770+0000 7f4077400640 1 -- 172.21.15.43:0/3975284984 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4084016070 con 0x55b67822ec70 2024-09-19T05:14:15.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.770+0000 7f4077400640 1 -- 172.21.15.43:0/3975284984 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4084004d20 con 0x55b67822ec70 2024-09-19T05:14:15.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.770+0000 7f4082a00640 1 -- 172.21.15.43:0/2469666678 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8586+0+0 (secure 0 0 0) 0x7f407c0f02a0 con 0x7f408c101de0 2024-09-19T05:14:15.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.770+0000 7f4089e00640 1 --2- 172.21.15.43:0/2469666678 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f40700a3430 0x7f40700a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:15.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.770+0000 7f4077400640 1 -- 172.21.15.43:0/3975284984 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40840128f0 con 0x55b67822ec70 2024-09-19T05:14:15.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.771+0000 7f4093a7c740 1 -- 172.21.15.43:0/3975284984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b67822ec70 msgr2=0x55b67822f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:15.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.771+0000 7f4093a7c740 1 --2- 172.21.15.43:0/3975284984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b67822ec70 0x55b67822f050 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f4084009f70 tx=0x7f4084010800 comp rx=0 tx=0).stop 2024-09-19T05:14:15.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.771+0000 7f4089e00640 1 --2- 172.21.15.43:0/2469666678 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f40700a3430 0x7f40700a58f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x55b67822c060 tx=0x7f407c0f3d50 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:15.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.771+0000 7f4093a7c740 1 -- 172.21.15.43:0/3975284984 shutdown_connections 2024-09-19T05:14:15.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.771+0000 7f4093a7c740 1 --2- 172.21.15.43:0/3975284984 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b678239e50 0x55b67823c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:15.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.771+0000 7f4093a7c740 1 --2- 172.21.15.43:0/3975284984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b67822f590 0x55b678239910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:15.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.771+0000 7f4093a7c740 1 --2- 172.21.15.43:0/3975284984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b67822ec70 0x55b67822f050 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:15.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.771+0000 7f4093a7c740 1 -- 172.21.15.43:0/3975284984 >> 172.21.15.43:0/3975284984 conn(0x55b67822b200 msgr2=0x55b67822b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:15.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.771+0000 7f4093a7c740 1 -- 172.21.15.43:0/3975284984 shutdown_connections 2024-09-19T05:14:15.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.771+0000 7f4093a7c740 1 -- 172.21.15.43:0/3975284984 wait complete. 2024-09-19T05:14:15.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.771+0000 7f4093a7c740 1 -- start start 2024-09-19T05:14:15.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.772+0000 7f4093a7c740 1 Processor -- start 2024-09-19T05:14:15.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.772+0000 7f4093a7c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b67822ec70 0x55b678236c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:15.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.772+0000 7f4093a7c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b67822f590 0x55b678237190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:15.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.772+0000 7f4093a7c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b678239e50 0x55b6782376d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:15.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.772+0000 7f4093a7c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b67823ca70 con 0x55b67822ec70 2024-09-19T05:14:15.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.772+0000 7f4093a7c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b6781500f0 con 0x55b67822f590 2024-09-19T05:14:15.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.772+0000 7f4093a7c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b67823cbf0 con 0x55b678239e50 2024-09-19T05:14:15.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.772+0000 7f4093a7c740 10 client.0 _ll_register_callbacks cb 0x55b678137bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:15.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.772+0000 7f408a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b67822ec70 0x55b678236c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:15.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.772+0000 7f408a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b67822ec70 0x55b678236c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44606/0 (socket says 172.21.15.43:44606) 2024-09-19T05:14:15.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.772+0000 7f408a800640 1 -- 172.21.15.43:0/1853622528 learned_addr learned my addr 172.21.15.43:0/1853622528 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:15.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.772+0000 7f4089400640 1 --2- 172.21.15.43:0/1853622528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b67822f590 0x55b678237190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:15.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.773+0000 7f408a800640 1 --2- 172.21.15.43:0/1853622528 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b678239e50 0x55b6782376d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:15.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.773+0000 7f4089400640 1 -- 172.21.15.43:0/1853622528 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b678239e50 msgr2=0x55b6782376d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:15.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.773+0000 7f4089400640 1 --2- 172.21.15.43:0/1853622528 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b678239e50 0x55b6782376d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:15.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.773+0000 7f4089400640 1 -- 172.21.15.43:0/1853622528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b67822ec70 msgr2=0x55b678236c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:15.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.773+0000 7f4089400640 1 --2- 172.21.15.43:0/1853622528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b67822ec70 0x55b678236c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:15.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.773+0000 7f4089400640 1 -- 172.21.15.43:0/1853622528 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b677f063c0 con 0x55b67822f590 2024-09-19T05:14:15.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.773+0000 7f408a800640 1 --2- 172.21.15.43:0/1853622528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b67822ec70 0x55b678236c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:15.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.773+0000 7f4089400640 1 --2- 172.21.15.43:0/1853622528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b67822f590 0x55b678237190 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f407800baa0 tx=0x7f407800bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68638def1e6a0768 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:15.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.773+0000 7f406be00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:14:15.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.773+0000 7f406be00640 1 -- 172.21.15.43:0/1853622528 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4078003df0 con 0x55b67822f590 2024-09-19T05:14:15.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.774+0000 7f4093a7c740 1 -- 172.21.15.43:0/1853622528 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4084004d20 con 0x55b67822f590 2024-09-19T05:14:15.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.774+0000 7f406be00640 1 -- 172.21.15.43:0/1853622528 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8586+0+0 (secure 0 0 0) 0x7f4078005020 con 0x55b67822f590 2024-09-19T05:14:15.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.774+0000 7f406be00640 1 -- 172.21.15.43:0/1853622528 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4078005220 con 0x55b67822f590 2024-09-19T05:14:15.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.774+0000 7f4093a7c740 10 client.27449 fetch_fsmap learned FSMap version 29 2024-09-19T05:14:15.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.774+0000 7f4093a7c740 1 -- 172.21.15.43:0/1853622528 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b678248ad0 con 0x55b67822f590 2024-09-19T05:14:15.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.774+0000 7f406be00640 1 -- 172.21.15.43:0/1853622528 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4084004d20 con 0x55b67822f590 2024-09-19T05:14:15.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.775+0000 7f406be00640 1 -- 172.21.15.43:0/1853622528 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4078011560 con 0x55b67822f590 2024-09-19T05:14:15.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.775+0000 7f4093a7c740 10 client.27449 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:14:15.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.775+0000 7f4093a7c740 10 client.27449 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T05:14:15.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.775+0000 7f4093a7c740 20 client.27449 populate_metadata read hostname 'smithi043' 2024-09-19T05:14:15.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.775+0000 7f4093a7c740 1 --2- 172.21.15.43:0/1853622528 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55b678249ee0 0x55b67824c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:15.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.775+0000 7f4093a7c740 4 client.27449 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:14:15.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.775+0000 7f4093a7c740 1 -- 172.21.15.43:0/1853622528 --> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6781a2950 con 0x55b678249ee0 2024-09-19T05:14:15.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.775+0000 7f408a800640 1 --2- 172.21.15.43:0/1853622528 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55b678249ee0 0x55b67824c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:15.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.776+0000 7f408a800640 1 --2- 172.21.15.43:0/1853622528 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55b678249ee0 0x55b67824c2f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b911e2a426287c14 server_cookie=d62e0845778d60c6 in_seq=0 out_seq=0 2024-09-19T05:14:15.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.776+0000 7f406be00640 10 client.27449 ms_handle_connect on v2:172.21.15.151:6838/3794104313 2024-09-19T05:14:15.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.777+0000 7f406be00640 1 -- 172.21.15.43:0/1853622528 <== mds.2 v2:172.21.15.151:6838/3794104313 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b6781a2950 con 0x55b678249ee0 2024-09-19T05:14:15.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.777+0000 7f406be00640 10 client.27449 handle_command_reply: tid=0 2024-09-19T05:14:15.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.777+0000 7f4093a7c740 1 client.27449 shutdown 2024-09-19T05:14:15.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.777+0000 7f4093a7c740 1 -- 172.21.15.43:0/1853622528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b67822f590 msgr2=0x55b678237190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:15.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.777+0000 7f4093a7c740 1 --2- 172.21.15.43:0/1853622528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b67822f590 0x55b678237190 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f407800baa0 tx=0x7f407800bf60 comp rx=0 tx=0).stop 2024-09-19T05:14:15.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.777+0000 7f4093a7c740 1 -- 172.21.15.43:0/1853622528 shutdown_connections 2024-09-19T05:14:15.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.777+0000 7f4093a7c740 1 --2- 172.21.15.43:0/1853622528 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55b678249ee0 0x55b67824c2f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:15.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.777+0000 7f4093a7c740 1 --2- 172.21.15.43:0/1853622528 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b678239e50 0x55b6782376d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:15.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.777+0000 7f4093a7c740 1 --2- 172.21.15.43:0/1853622528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b67822f590 0x55b678237190 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:15.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.777+0000 7f4093a7c740 1 --2- 172.21.15.43:0/1853622528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b67822ec70 0x55b678236c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:15.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.777+0000 7f4093a7c740 1 -- 172.21.15.43:0/1853622528 >> 172.21.15.43:0/1853622528 conn(0x55b67822b200 msgr2=0x55b67822a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:15.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.777+0000 7f406be00640 0 client.27449 ms_handle_reset on v2:172.21.15.151:6838/3794104313 2024-09-19T05:14:15.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.777+0000 7f4093a7c740 1 -- 172.21.15.43:0/1853622528 shutdown_connections 2024-09-19T05:14:15.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.777+0000 7f4093a7c740 1 -- 172.21.15.43:0/1853622528 wait complete. 2024-09-19T05:14:15.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.777+0000 7f4093a7c740 20 client.27449 trim_cache size 0 max 16384 2024-09-19T05:14:15.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.791+0000 7f4093a7c740 1 Processor -- start 2024-09-19T05:14:15.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.792+0000 7f4093a7c740 1 -- start start 2024-09-19T05:14:15.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.792+0000 7f4093a7c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b67822ec70 0x55b6782316d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:15.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.792+0000 7f4093a7c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b678239e50 0x55b67822b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:15.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.792+0000 7f4093a7c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b678251bc0 0x55b678252020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:15.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.792+0000 7f4093a7c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f408c1afbe0 con 0x55b67822ec70 2024-09-19T05:14:15.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.792+0000 7f4093a7c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f408c1b1720 con 0x55b678239e50 2024-09-19T05:14:15.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.792+0000 7f4093a7c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f408c1b02b0 con 0x55b678251bc0 2024-09-19T05:14:15.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.792+0000 7f408a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b67822ec70 0x55b6782316d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:15.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.792+0000 7f4089400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b678239e50 0x55b67822b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:15.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.792+0000 7f408a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b67822ec70 0x55b6782316d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44612/0 (socket says 172.21.15.43:44612) 2024-09-19T05:14:15.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.792+0000 7f408a800640 1 -- 172.21.15.43:0/4265334946 learned_addr learned my addr 172.21.15.43:0/4265334946 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:15.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.793+0000 7f408a800640 1 --2- 172.21.15.43:0/4265334946 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b678251bc0 0x55b678252020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:15.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.793+0000 7f4089400640 1 -- 172.21.15.43:0/4265334946 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b678251bc0 msgr2=0x55b678252020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:15.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.793+0000 7f4089400640 1 --2- 172.21.15.43:0/4265334946 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b678251bc0 0x55b678252020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:15.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.793+0000 7f4089400640 1 -- 172.21.15.43:0/4265334946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b67822ec70 msgr2=0x55b6782316d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:15.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.793+0000 7f4089400640 1 --2- 172.21.15.43:0/4265334946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b67822ec70 0x55b6782316d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:15.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.793+0000 7f4089400640 1 -- 172.21.15.43:0/4265334946 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b67822b030 con 0x55b678239e50 2024-09-19T05:14:15.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.793+0000 7f408a800640 1 --2- 172.21.15.43:0/4265334946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b67822ec70 0x55b6782316d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:15.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.793+0000 7f4089400640 1 --2- 172.21.15.43:0/4265334946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b678239e50 0x55b67822b5e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x55b678238200 tx=0x7f4078001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33a980bcabe06447 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:15.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.793+0000 7f4076a00640 1 -- 172.21.15.43:0/4265334946 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4078003df0 con 0x55b678239e50 2024-09-19T05:14:15.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.793+0000 7f4076a00640 1 -- 172.21.15.43:0/4265334946 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4078011bf0 con 0x55b678239e50 2024-09-19T05:14:15.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.793+0000 7f4076a00640 1 -- 172.21.15.43:0/4265334946 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f407801b7f0 con 0x55b678239e50 2024-09-19T05:14:15.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.794+0000 7f4093a7c740 1 -- 172.21.15.43:0/4265334946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b678239e50 msgr2=0x55b67822b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:15.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.794+0000 7f4093a7c740 1 --2- 172.21.15.43:0/4265334946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b678239e50 0x55b67822b5e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x55b678238200 tx=0x7f4078001e70 comp rx=0 tx=0).stop 2024-09-19T05:14:15.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.794+0000 7f4093a7c740 1 -- 172.21.15.43:0/4265334946 shutdown_connections 2024-09-19T05:14:15.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.794+0000 7f4093a7c740 1 --2- 172.21.15.43:0/4265334946 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b678251bc0 0x55b678252020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:15.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.794+0000 7f4093a7c740 1 --2- 172.21.15.43:0/4265334946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b678239e50 0x55b67822b5e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:15.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.794+0000 7f4093a7c740 1 --2- 172.21.15.43:0/4265334946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b67822ec70 0x55b6782316d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:15.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.794+0000 7f4093a7c740 1 -- 172.21.15.43:0/4265334946 >> 172.21.15.43:0/4265334946 conn(0x55b67822b200 msgr2=0x55b678235d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:15.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.794+0000 7f4093a7c740 1 -- 172.21.15.43:0/4265334946 shutdown_connections 2024-09-19T05:14:15.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.794+0000 7f4093a7c740 1 -- 172.21.15.43:0/4265334946 wait complete. 2024-09-19T05:14:15.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.795+0000 7f4093a7c740 1 -- start start 2024-09-19T05:14:15.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.795+0000 7f4093a7c740 1 Processor -- start 2024-09-19T05:14:15.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.795+0000 7f4093a7c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b67822ec70 0x55b678242400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:15.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.795+0000 7f4093a7c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b678239e50 0x55b67823d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:15.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.795+0000 7f4093a7c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b678251bc0 0x55b67823dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:15.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.795+0000 7f4093a7c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f408c1b1050 con 0x55b678251bc0 2024-09-19T05:14:15.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.795+0000 7f4093a7c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f408c1b0980 con 0x55b678239e50 2024-09-19T05:14:15.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.795+0000 7f4093a7c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f408c1adbe0 con 0x55b67822ec70 2024-09-19T05:14:15.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.795+0000 7f4093a7c740 10 client.0 _ll_register_callbacks cb 0x7f4078004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:15.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.796+0000 7f408a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b67822ec70 0x55b678242400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:15.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.796+0000 7f408a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b67822ec70 0x55b678242400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:51194/0 (socket says 172.21.15.43:51194) 2024-09-19T05:14:15.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.796+0000 7f408a800640 1 -- 172.21.15.43:0/112484121 learned_addr learned my addr 172.21.15.43:0/112484121 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:15.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.796+0000 7f408a800640 1 --2- 172.21.15.43:0/112484121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b678251bc0 0x55b67823dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:15.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.796+0000 7f4089400640 1 --2- 172.21.15.43:0/112484121 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b678239e50 0x55b67823d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:15.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.796+0000 7f408a800640 1 -- 172.21.15.43:0/112484121 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b67822ec70 msgr2=0x55b678242400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:15.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.796+0000 7f408a800640 1 --2- 172.21.15.43:0/112484121 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b67822ec70 0x55b678242400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:15.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.796+0000 7f408a800640 1 -- 172.21.15.43:0/112484121 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b678239e50 msgr2=0x55b67823d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:15.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.796+0000 7f408a800640 1 --2- 172.21.15.43:0/112484121 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b678239e50 0x55b67823d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:15.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.796+0000 7f408a800640 1 -- 172.21.15.43:0/112484121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b67823e2b0 con 0x55b678251bc0 2024-09-19T05:14:15.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.796+0000 7f408a800640 1 --2- 172.21.15.43:0/112484121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b678251bc0 0x55b67823dc70 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f4084010e00 tx=0x7f4084012590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ece995f6034f7d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:15.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.796+0000 7f406be00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:14:15.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.797+0000 7f406be00640 1 -- 172.21.15.43:0/112484121 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4084016070 con 0x55b678251bc0 2024-09-19T05:14:15.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.797+0000 7f406be00640 1 -- 172.21.15.43:0/112484121 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8586+0+0 (secure 0 0 0) 0x7f4084013c90 con 0x55b678251bc0 2024-09-19T05:14:15.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.797+0000 7f4093a7c740 1 -- 172.21.15.43:0/112484121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4078011bf0 con 0x55b678251bc0 2024-09-19T05:14:15.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.797+0000 7f406be00640 1 -- 172.21.15.43:0/112484121 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4084003970 con 0x55b678251bc0 2024-09-19T05:14:15.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.797+0000 7f4093a7c740 10 client.17427 fetch_fsmap learned FSMap version 29 2024-09-19T05:14:15.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.797+0000 7f4093a7c740 1 -- 172.21.15.43:0/112484121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b6782b49a0 con 0x55b678251bc0 2024-09-19T05:14:15.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.797+0000 7f406be00640 1 -- 172.21.15.43:0/112484121 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4078011bf0 con 0x55b678251bc0 2024-09-19T05:14:15.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.797+0000 7f406be00640 1 -- 172.21.15.43:0/112484121 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f408402a040 con 0x55b678251bc0 2024-09-19T05:14:15.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.798+0000 7f4093a7c740 10 client.17427 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:14:15.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.798+0000 7f4093a7c740 10 client.17427 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T05:14:15.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.798+0000 7f4093a7c740 20 client.17427 populate_metadata read hostname 'smithi043' 2024-09-19T05:14:15.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.798+0000 7f4093a7c740 1 --2- 172.21.15.43:0/112484121 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55b6782b4df0 0x55b6782b71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:15.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.798+0000 7f4093a7c740 4 client.17427 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:14:15.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.798+0000 7f4093a7c740 1 -- 172.21.15.43:0/112484121 --> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b67819ce60 con 0x55b6782b4df0 2024-09-19T05:14:15.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.798+0000 7f4089400640 1 --2- 172.21.15.43:0/112484121 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55b6782b4df0 0x55b6782b71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:15.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.799+0000 7f4089400640 1 --2- 172.21.15.43:0/112484121 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55b6782b4df0 0x55b6782b71e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6d4a394a12c725bd server_cookie=39ae7266f0f8853d in_seq=0 out_seq=0 2024-09-19T05:14:15.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.799+0000 7f406be00640 10 client.17427 ms_handle_connect on v2:172.21.15.151:6838/3794104313 2024-09-19T05:14:15.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.799+0000 7f406be00640 1 -- 172.21.15.43:0/112484121 <== mds.2 v2:172.21.15.151:6838/3794104313 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b67819ce60 con 0x55b6782b4df0 2024-09-19T05:14:15.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.799+0000 7f406be00640 10 client.17427 handle_command_reply: tid=0 2024-09-19T05:14:15.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.799+0000 7f4093a7c740 1 client.17427 shutdown 2024-09-19T05:14:15.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.799+0000 7f4093a7c740 1 -- 172.21.15.43:0/112484121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b678251bc0 msgr2=0x55b67823dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:15.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.799+0000 7f4093a7c740 1 --2- 172.21.15.43:0/112484121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b678251bc0 0x55b67823dc70 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f4084010e00 tx=0x7f4084012590 comp rx=0 tx=0).stop 2024-09-19T05:14:15.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.799+0000 7f4093a7c740 1 -- 172.21.15.43:0/112484121 shutdown_connections 2024-09-19T05:14:15.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.799+0000 7f4093a7c740 1 --2- 172.21.15.43:0/112484121 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55b6782b4df0 0x55b6782b71e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:15.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.799+0000 7f4093a7c740 1 --2- 172.21.15.43:0/112484121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b678251bc0 0x55b67823dc70 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:15.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.799+0000 7f4093a7c740 1 --2- 172.21.15.43:0/112484121 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b678239e50 0x55b67823d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:15.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.799+0000 7f4093a7c740 1 --2- 172.21.15.43:0/112484121 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b67822ec70 0x55b678242400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:15.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.799+0000 7f406be00640 0 client.17427 ms_handle_reset on v2:172.21.15.151:6838/3794104313 2024-09-19T05:14:15.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.799+0000 7f4093a7c740 1 -- 172.21.15.43:0/112484121 >> 172.21.15.43:0/112484121 conn(0x55b67822b200 msgr2=0x55b678235d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:15.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.800+0000 7f4093a7c740 1 -- 172.21.15.43:0/112484121 shutdown_connections 2024-09-19T05:14:15.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.800+0000 7f4093a7c740 1 -- 172.21.15.43:0/112484121 wait complete. 2024-09-19T05:14:15.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.800+0000 7f4093a7c740 20 client.17427 trim_cache size 0 max 16384 2024-09-19T05:14:15.805 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:14:15.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.800+0000 7f4091000640 1 -- 172.21.15.43:0/2469666678 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f40700a3430 msgr2=0x7f40700a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:15.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.800+0000 7f4091000640 1 --2- 172.21.15.43:0/2469666678 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f40700a3430 0x7f40700a58f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x55b67822c060 tx=0x7f407c0f3d50 comp rx=0 tx=0).stop 2024-09-19T05:14:15.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.800+0000 7f4091000640 1 -- 172.21.15.43:0/2469666678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f408c101de0 msgr2=0x7f408c19a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:15.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.800+0000 7f4091000640 1 --2- 172.21.15.43:0/2469666678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f408c101de0 0x7f408c19a650 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f407c006010 tx=0x7f407c0042f0 comp rx=0 tx=0).stop 2024-09-19T05:14:15.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.800+0000 7f4091000640 1 -- 172.21.15.43:0/2469666678 shutdown_connections 2024-09-19T05:14:15.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.801+0000 7f4091000640 1 --2- 172.21.15.43:0/2469666678 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f40700a3430 0x7f40700a58f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:15.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.801+0000 7f4091000640 1 --2- 172.21.15.43:0/2469666678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f408c106f10 0x7f408c19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:15.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.801+0000 7f4091000640 1 --2- 172.21.15.43:0/2469666678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f408c102790 0x7f408c19ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:15.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.801+0000 7f4091000640 1 --2- 172.21.15.43:0/2469666678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f408c101de0 0x7f408c19a650 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:15.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.801+0000 7f4091000640 1 -- 172.21.15.43:0/2469666678 >> 172.21.15.43:0/2469666678 conn(0x7f408c0fd8c0 msgr2=0x7f408c0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:15.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.801+0000 7f4091000640 1 -- 172.21.15.43:0/2469666678 shutdown_connections 2024-09-19T05:14:15.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:15.801+0000 7f4091000640 1 -- 172.21.15.43:0/2469666678 wait complete. 2024-09-19T05:14:16.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:16 smithi043 ceph-mon[30800]: pgmap v298: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T05:14:16.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:16 smithi043 ceph-mon[30800]: osdmap e94: 12 total, 12 up, 12 in 2024-09-19T05:14:16.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:16 smithi151 ceph-mon[35425]: pgmap v298: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T05:14:16.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:16 smithi151 ceph-mon[35425]: osdmap e94: 12 total, 12 up, 12 in 2024-09-19T05:14:16.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:16 smithi096 ceph-mon[35085]: pgmap v298: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T05:14:16.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:16 smithi096 ceph-mon[35085]: osdmap e94: 12 total, 12 up, 12 in 2024-09-19T05:14:16.846 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T05:14:17.229 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:14:17.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.772+0000 7ff2cd000640 1 -- 172.21.15.43:0/355003962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2c8103ff0 msgr2=0x7ff2c81043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:18.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.772+0000 7ff2cd000640 1 --2- 172.21.15.43:0/355003962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2c8103ff0 0x7ff2c81043d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8009900 tx=0x7ff2b802a400 comp rx=0 tx=0).stop 2024-09-19T05:14:18.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.772+0000 7ff2cd000640 1 -- 172.21.15.43:0/355003962 shutdown_connections 2024-09-19T05:14:18.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.772+0000 7ff2cd000640 1 --2- 172.21.15.43:0/355003962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2c8109120 0x7ff2c8110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:18.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.772+0000 7ff2cd000640 1 --2- 172.21.15.43:0/355003962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2c81049a0 0x7ff2c81089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:18.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.772+0000 7ff2cd000640 1 --2- 172.21.15.43:0/355003962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2c8103ff0 0x7ff2c81043d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:18.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.772+0000 7ff2cd000640 1 -- 172.21.15.43:0/355003962 >> 172.21.15.43:0/355003962 conn(0x7ff2c80ffa90 msgr2=0x7ff2c8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:18.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.772+0000 7ff2cd000640 1 -- 172.21.15.43:0/355003962 shutdown_connections 2024-09-19T05:14:18.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.772+0000 7ff2cd000640 1 -- 172.21.15.43:0/355003962 wait complete. 2024-09-19T05:14:18.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.773+0000 7ff2cd000640 1 Processor -- start 2024-09-19T05:14:18.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.773+0000 7ff2cd000640 1 -- start start 2024-09-19T05:14:18.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.773+0000 7ff2cd000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2c8103ff0 0x7ff2c819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:18.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.773+0000 7ff2cd000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2c81049a0 0x7ff2c819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:18.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.773+0000 7ff2cd000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2c8109120 0x7ff2c819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:18.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.774+0000 7ff2cd000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c8114190 con 0x7ff2c8109120 2024-09-19T05:14:18.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.774+0000 7ff2cd000640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c8114010 con 0x7ff2c81049a0 2024-09-19T05:14:18.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.774+0000 7ff2cd000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c8114310 con 0x7ff2c8103ff0 2024-09-19T05:14:18.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.774+0000 7ff2c5e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2c8103ff0 0x7ff2c819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:18.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.774+0000 7ff2c6800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2c8109120 0x7ff2c819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:18.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.774+0000 7ff2c5e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2c8103ff0 0x7ff2c819a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:51226/0 (socket says 172.21.15.43:51226) 2024-09-19T05:14:18.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.774+0000 7ff2c5e00640 1 -- 172.21.15.43:0/3288807032 learned_addr learned my addr 172.21.15.43:0/3288807032 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:18.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.774+0000 7ff2c5400640 1 --2- 172.21.15.43:0/3288807032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2c81049a0 0x7ff2c819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:18.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.774+0000 7ff2c5400640 1 -- 172.21.15.43:0/3288807032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2c8103ff0 msgr2=0x7ff2c819a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:18.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.774+0000 7ff2c5400640 1 --2- 172.21.15.43:0/3288807032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2c8103ff0 0x7ff2c819a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:18.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.774+0000 7ff2c5400640 1 -- 172.21.15.43:0/3288807032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2c8109120 msgr2=0x7ff2c819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:18.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.774+0000 7ff2c5400640 1 --2- 172.21.15.43:0/3288807032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2c8109120 0x7ff2c819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:18.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.774+0000 7ff2c5400640 1 -- 172.21.15.43:0/3288807032 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c819f5d0 con 0x7ff2c81049a0 2024-09-19T05:14:18.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.775+0000 7ff2c5400640 1 --2- 172.21.15.43:0/3288807032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2c81049a0 0x7ff2c819ab60 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff2b400e9f0 tx=0x7ff2b400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:18.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.775+0000 7ff2bea00640 1 -- 172.21.15.43:0/3288807032 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b400ccc0 con 0x7ff2c81049a0 2024-09-19T05:14:18.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.775+0000 7ff2cd000640 1 -- 172.21.15.43:0/3288807032 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2c819f8c0 con 0x7ff2c81049a0 2024-09-19T05:14:18.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.775+0000 7ff2bea00640 1 -- 172.21.15.43:0/3288807032 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2b4005020 con 0x7ff2c81049a0 2024-09-19T05:14:18.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.775+0000 7ff2bea00640 1 -- 172.21.15.43:0/3288807032 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b4004a00 con 0x7ff2c81049a0 2024-09-19T05:14:18.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.775+0000 7ff2cd000640 1 -- 172.21.15.43:0/3288807032 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2c81a71a0 con 0x7ff2c81049a0 2024-09-19T05:14:18.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.778+0000 7ff2cfab9740 1 Processor -- start 2024-09-19T05:14:18.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.778+0000 7ff2cfab9740 1 -- start start 2024-09-19T05:14:18.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.778+0000 7ff2cfab9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584dc3c3c70 0x5584dc3c4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:18.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.778+0000 7ff2cfab9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584dc3c4590 0x5584dc3ce910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:18.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.778+0000 7ff2cfab9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584dc3cee50 0x5584dc3d1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:18.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.778+0000 7ff2cfab9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5584dc32a3e0 con 0x5584dc3c4590 2024-09-19T05:14:18.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.778+0000 7ff2cfab9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5584dc2d1d80 con 0x5584dc3cee50 2024-09-19T05:14:18.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.778+0000 7ff2cfab9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5584dc3263a0 con 0x5584dc3c3c70 2024-09-19T05:14:18.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.779+0000 7ff2c5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584dc3c4590 0x5584dc3ce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:18.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.779+0000 7ff2c6800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584dc3c3c70 0x5584dc3c4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:18.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.779+0000 7ff2c5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584dc3c4590 0x5584dc3ce910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44652/0 (socket says 172.21.15.43:44652) 2024-09-19T05:14:18.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.779+0000 7ff2c5e00640 1 -- 172.21.15.43:0/1653374576 learned_addr learned my addr 172.21.15.43:0/1653374576 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:18.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.779+0000 7ff2bea00640 1 -- 172.21.15.43:0/3288807032 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ff2b40051d0 con 0x7ff2c81049a0 2024-09-19T05:14:18.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.779+0000 7ff2c6800640 1 -- 172.21.15.43:0/1653374576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584dc3cee50 msgr2=0x5584dc3d1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:18.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.779+0000 7ff2c6800640 1 --2- 172.21.15.43:0/1653374576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584dc3cee50 0x5584dc3d1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:18.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.779+0000 7ff2c6800640 1 -- 172.21.15.43:0/1653374576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584dc3c4590 msgr2=0x5584dc3ce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:18.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.779+0000 7ff2c6800640 1 --2- 172.21.15.43:0/1653374576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584dc3c4590 0x5584dc3ce910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:18.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.779+0000 7ff2c6800640 1 -- 172.21.15.43:0/1653374576 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584dc2f0ae0 con 0x5584dc3c3c70 2024-09-19T05:14:18.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.780+0000 7ff2c6800640 1 --2- 172.21.15.43:0/1653374576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584dc3c3c70 0x5584dc3c4050 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0009f10 tx=0x7ff2c0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb32264c461b04d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:18.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.780+0000 7ff2b3400640 1 -- 172.21.15.43:0/1653374576 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0004d20 con 0x5584dc3c3c70 2024-09-19T05:14:18.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.780+0000 7ff2b3400640 1 -- 172.21.15.43:0/1653374576 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2c0004050 con 0x5584dc3c3c70 2024-09-19T05:14:18.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.780+0000 7ff2b3400640 1 -- 172.21.15.43:0/1653374576 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0005260 con 0x5584dc3c3c70 2024-09-19T05:14:18.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.781+0000 7ff2bea00640 1 --2- 172.21.15.43:0/3288807032 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff2ac0a3430 0x7ff2ac0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:18.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.781+0000 7ff2cfab9740 1 -- 172.21.15.43:0/1653374576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584dc3c3c70 msgr2=0x5584dc3c4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:18.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.781+0000 7ff2cfab9740 1 --2- 172.21.15.43:0/1653374576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584dc3c3c70 0x5584dc3c4050 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0009f10 tx=0x7ff2c0011810 comp rx=0 tx=0).stop 2024-09-19T05:14:18.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.781+0000 7ff2cfab9740 1 -- 172.21.15.43:0/1653374576 shutdown_connections 2024-09-19T05:14:18.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.781+0000 7ff2cfab9740 1 --2- 172.21.15.43:0/1653374576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584dc3cee50 0x5584dc3d1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:18.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.781+0000 7ff2cfab9740 1 --2- 172.21.15.43:0/1653374576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584dc3c4590 0x5584dc3ce910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:18.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.781+0000 7ff2cfab9740 1 --2- 172.21.15.43:0/1653374576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584dc3c3c70 0x5584dc3c4050 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:18.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.781+0000 7ff2cfab9740 1 -- 172.21.15.43:0/1653374576 >> 172.21.15.43:0/1653374576 conn(0x5584dc3c0200 msgr2=0x5584dc3c05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:18.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.781+0000 7ff2bea00640 1 -- 172.21.15.43:0/3288807032 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7ff2b40cfc50 con 0x7ff2c81049a0 2024-09-19T05:14:18.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.781+0000 7ff2c5e00640 1 --2- 172.21.15.43:0/3288807032 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff2ac0a3430 0x7ff2ac0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:18.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.781+0000 7ff2cfab9740 1 -- 172.21.15.43:0/1653374576 shutdown_connections 2024-09-19T05:14:18.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.781+0000 7ff2cfab9740 1 -- 172.21.15.43:0/1653374576 wait complete. 2024-09-19T05:14:18.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.781+0000 7ff2cfab9740 1 -- start start 2024-09-19T05:14:18.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.782+0000 7ff2cfab9740 1 Processor -- start 2024-09-19T05:14:18.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.782+0000 7ff2c5e00640 1 --2- 172.21.15.43:0/3288807032 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff2ac0a3430 0x7ff2ac0a58f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8004770 tx=0x7ff2b80046c0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:18.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.782+0000 7ff2cfab9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584dc3c3c70 0x5584dc3cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:18.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.782+0000 7ff2cfab9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584dc3c4590 0x5584dc3cc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:18.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.782+0000 7ff2cfab9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584dc3cee50 0x5584dc3cc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:18.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.782+0000 7ff2cfab9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5584dc3d1a20 con 0x5584dc3cee50 2024-09-19T05:14:18.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.782+0000 7ff2cfab9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5584dc2e50f0 con 0x5584dc3c4590 2024-09-19T05:14:18.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.782+0000 7ff2cfab9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5584dc3d1ba0 con 0x5584dc3c3c70 2024-09-19T05:14:18.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.782+0000 7ff2cfab9740 10 client.0 _ll_register_callbacks cb 0x5584dc2ccbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:18.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.783+0000 7ff2c5e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584dc3c3c70 0x5584dc3cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:18.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.783+0000 7ff2c5e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584dc3c3c70 0x5584dc3cbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:51246/0 (socket says 172.21.15.43:51246) 2024-09-19T05:14:18.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.783+0000 7ff2c5e00640 1 -- 172.21.15.43:0/1495391248 learned_addr learned my addr 172.21.15.43:0/1495391248 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:18.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.783+0000 7ff2c5400640 1 --2- 172.21.15.43:0/1495391248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584dc3c4590 0x5584dc3cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:18.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.783+0000 7ff2c6800640 1 --2- 172.21.15.43:0/1495391248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584dc3cee50 0x5584dc3cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:18.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.783+0000 7ff2c5e00640 1 -- 172.21.15.43:0/1495391248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584dc3c4590 msgr2=0x5584dc3cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:18.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.783+0000 7ff2c5e00640 1 --2- 172.21.15.43:0/1495391248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584dc3c4590 0x5584dc3cc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:18.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.783+0000 7ff2c5e00640 1 -- 172.21.15.43:0/1495391248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584dc3cee50 msgr2=0x5584dc3cc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:18.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.783+0000 7ff2c5e00640 1 --2- 172.21.15.43:0/1495391248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584dc3cee50 0x5584dc3cc870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:18.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.783+0000 7ff2c5e00640 1 -- 172.21.15.43:0/1495391248 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584dc09b3c0 con 0x5584dc3c3c70 2024-09-19T05:14:18.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.783+0000 7ff2c5400640 1 --2- 172.21.15.43:0/1495391248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584dc3c4590 0x5584dc3cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:18.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.783+0000 7ff2c5e00640 1 --2- 172.21.15.43:0/1495391248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584dc3c3c70 0x5584dc3cbdf0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff2b80045a0 tx=0x7ff2b8009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fbe27bf9b4f3020 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:18.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.783+0000 7ff2a7e00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:14:18.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.784+0000 7ff2a7e00640 1 -- 172.21.15.43:0/1495391248 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b804e070 con 0x5584dc3c3c70 2024-09-19T05:14:18.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.784+0000 7ff2cfab9740 1 -- 172.21.15.43:0/1495391248 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2c0004050 con 0x5584dc3c3c70 2024-09-19T05:14:18.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.784+0000 7ff2a7e00640 1 -- 172.21.15.43:0/1495391248 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7ff2b80037f0 con 0x5584dc3c3c70 2024-09-19T05:14:18.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.784+0000 7ff2a7e00640 1 -- 172.21.15.43:0/1495391248 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff2b805a050 con 0x5584dc3c3c70 2024-09-19T05:14:18.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.784+0000 7ff2cfab9740 10 client.27364 fetch_fsmap learned FSMap version 29 2024-09-19T05:14:18.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.784+0000 7ff2cfab9740 1 -- 172.21.15.43:0/1495391248 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5584dc3ddb30 con 0x5584dc3c3c70 2024-09-19T05:14:18.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.784+0000 7ff2a7e00640 1 -- 172.21.15.43:0/1495391248 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2c0004050 con 0x5584dc3c3c70 2024-09-19T05:14:18.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.784+0000 7ff2a7e00640 1 -- 172.21.15.43:0/1495391248 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff2b804aa30 con 0x5584dc3c3c70 2024-09-19T05:14:18.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.785+0000 7ff2cfab9740 10 client.27364 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:14:18.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.785+0000 7ff2cfab9740 10 client.27364 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-19T05:14:18.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.785+0000 7ff2cfab9740 20 client.27364 populate_metadata read hostname 'smithi043' 2024-09-19T05:14:18.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.785+0000 7ff2cfab9740 1 --2- 172.21.15.43:0/1495391248 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5584dc3def40 0x5584dc3e13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:18.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.785+0000 7ff2cfab9740 4 client.27364 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:14:18.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.785+0000 7ff2cfab9740 1 -- 172.21.15.43:0/1495391248 --> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584dc337950 con 0x5584dc3def40 2024-09-19T05:14:18.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.785+0000 7ff2c5400640 1 --2- 172.21.15.43:0/1495391248 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5584dc3def40 0x5584dc3e13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:18.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.786+0000 7ff2c5400640 1 --2- 172.21.15.43:0/1495391248 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5584dc3def40 0x5584dc3e13a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3047c8b3b0749335 server_cookie=888e3ecd434365df in_seq=0 out_seq=0 2024-09-19T05:14:18.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.786+0000 7ff2a7e00640 10 client.27364 ms_handle_connect on v2:172.21.15.43:6838/3404555030 2024-09-19T05:14:18.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.786+0000 7ff2a7e00640 1 -- 172.21.15.43:0/1495391248 <== mds.1 v2:172.21.15.43:6838/3404555030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5584dc337950 con 0x5584dc3def40 2024-09-19T05:14:18.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.786+0000 7ff2a7e00640 10 client.27364 handle_command_reply: tid=0 2024-09-19T05:14:18.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.786+0000 7ff2cfab9740 1 client.27364 shutdown 2024-09-19T05:14:18.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.787+0000 7ff2cfab9740 1 -- 172.21.15.43:0/1495391248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584dc3c3c70 msgr2=0x5584dc3cbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:18.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.787+0000 7ff2cfab9740 1 --2- 172.21.15.43:0/1495391248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584dc3c3c70 0x5584dc3cbdf0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff2b80045a0 tx=0x7ff2b8009260 comp rx=0 tx=0).stop 2024-09-19T05:14:18.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.787+0000 7ff2cfab9740 1 -- 172.21.15.43:0/1495391248 shutdown_connections 2024-09-19T05:14:18.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.787+0000 7ff2cfab9740 1 --2- 172.21.15.43:0/1495391248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584dc3cee50 0x5584dc3cc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:18.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.787+0000 7ff2cfab9740 1 --2- 172.21.15.43:0/1495391248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584dc3c4590 0x5584dc3cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:18.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.787+0000 7ff2cfab9740 1 --2- 172.21.15.43:0/1495391248 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5584dc3def40 0x5584dc3e13a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:18.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.787+0000 7ff2cfab9740 1 --2- 172.21.15.43:0/1495391248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584dc3c3c70 0x5584dc3cbdf0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:18.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.787+0000 7ff2cfab9740 1 -- 172.21.15.43:0/1495391248 >> 172.21.15.43:0/1495391248 conn(0x5584dc3c0200 msgr2=0x5584dc3c5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:18.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.787+0000 7ff2a7e00640 0 client.27364 ms_handle_reset on v2:172.21.15.43:6838/3404555030 2024-09-19T05:14:18.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.787+0000 7ff2cfab9740 1 -- 172.21.15.43:0/1495391248 shutdown_connections 2024-09-19T05:14:18.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.787+0000 7ff2cfab9740 1 -- 172.21.15.43:0/1495391248 wait complete. 2024-09-19T05:14:18.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.787+0000 7ff2cfab9740 20 client.27364 trim_cache size 0 max 16384 2024-09-19T05:14:18.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.801+0000 7ff2cfab9740 1 Processor -- start 2024-09-19T05:14:18.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.801+0000 7ff2cfab9740 1 -- start start 2024-09-19T05:14:18.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.801+0000 7ff2cfab9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584dc3c3c70 0x5584dc3e6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:18.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.801+0000 7ff2cfab9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584dc3c4590 0x5584dc3e7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:18.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.801+0000 7ff2cfab9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584dc3cee50 0x5584dc3cbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:18.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.801+0000 7ff2cfab9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c81afa90 con 0x5584dc3cee50 2024-09-19T05:14:18.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.801+0000 7ff2cfab9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c81b15d0 con 0x5584dc3c4590 2024-09-19T05:14:18.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.801+0000 7ff2cfab9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c81b0160 con 0x5584dc3c3c70 2024-09-19T05:14:18.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.801+0000 7ff2c5e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584dc3c3c70 0x5584dc3e6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:18.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.801+0000 7ff2c5e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584dc3c3c70 0x5584dc3e6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:51262/0 (socket says 172.21.15.43:51262) 2024-09-19T05:14:18.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.801+0000 7ff2c5e00640 1 -- 172.21.15.43:0/898921637 learned_addr learned my addr 172.21.15.43:0/898921637 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:18.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.802+0000 7ff2c5400640 1 --2- 172.21.15.43:0/898921637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584dc3c4590 0x5584dc3e7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:18.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.802+0000 7ff2c6800640 1 --2- 172.21.15.43:0/898921637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584dc3cee50 0x5584dc3cbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:18.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.802+0000 7ff2c5e00640 1 -- 172.21.15.43:0/898921637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584dc3c4590 msgr2=0x5584dc3e7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:18.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.802+0000 7ff2c5e00640 1 --2- 172.21.15.43:0/898921637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584dc3c4590 0x5584dc3e7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:18.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.802+0000 7ff2c5e00640 1 -- 172.21.15.43:0/898921637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584dc3cee50 msgr2=0x5584dc3cbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:18.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.802+0000 7ff2c5e00640 1 --2- 172.21.15.43:0/898921637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584dc3cee50 0x5584dc3cbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:18.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.802+0000 7ff2c5e00640 1 -- 172.21.15.43:0/898921637 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584dc3c3aa0 con 0x5584dc3c3c70 2024-09-19T05:14:18.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.802+0000 7ff2c5e00640 1 --2- 172.21.15.43:0/898921637 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584dc3c3c70 0x5584dc3e6cc0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff2b802abf0 tx=0x7ff2b8009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=160229a177f1194 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:18.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.802+0000 7ff2b2a00640 1 -- 172.21.15.43:0/898921637 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b8062070 con 0x5584dc3c3c70 2024-09-19T05:14:18.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.802+0000 7ff2b2a00640 1 -- 172.21.15.43:0/898921637 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2b803d6d0 con 0x5584dc3c3c70 2024-09-19T05:14:18.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.802+0000 7ff2b2a00640 1 -- 172.21.15.43:0/898921637 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b804e040 con 0x5584dc3c3c70 2024-09-19T05:14:18.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.803+0000 7ff2cfab9740 1 -- 172.21.15.43:0/898921637 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584dc3c3c70 msgr2=0x5584dc3e6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:18.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.803+0000 7ff2cfab9740 1 --2- 172.21.15.43:0/898921637 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584dc3c3c70 0x5584dc3e6cc0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff2b802abf0 tx=0x7ff2b8009620 comp rx=0 tx=0).stop 2024-09-19T05:14:18.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.803+0000 7ff2cfab9740 1 -- 172.21.15.43:0/898921637 shutdown_connections 2024-09-19T05:14:18.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.803+0000 7ff2cfab9740 1 --2- 172.21.15.43:0/898921637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584dc3cee50 0x5584dc3cbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:18.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.803+0000 7ff2cfab9740 1 --2- 172.21.15.43:0/898921637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584dc3c4590 0x5584dc3e7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:18.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.803+0000 7ff2cfab9740 1 --2- 172.21.15.43:0/898921637 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584dc3c3c70 0x5584dc3e6cc0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:18.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.803+0000 7ff2cfab9740 1 -- 172.21.15.43:0/898921637 >> 172.21.15.43:0/898921637 conn(0x5584dc3c0200 msgr2=0x5584dc3caf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:18.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.803+0000 7ff2cfab9740 1 -- 172.21.15.43:0/898921637 shutdown_connections 2024-09-19T05:14:18.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.803+0000 7ff2cfab9740 1 -- 172.21.15.43:0/898921637 wait complete. 2024-09-19T05:14:18.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.804+0000 7ff2cfab9740 1 -- start start 2024-09-19T05:14:18.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.804+0000 7ff2cfab9740 1 Processor -- start 2024-09-19T05:14:18.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.804+0000 7ff2cfab9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584dc3c3c70 0x5584dc3d7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:18.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.804+0000 7ff2cfab9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584dc3c4590 0x5584dc3d7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:18.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.804+0000 7ff2cfab9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584dc3cee50 0x5584dc3c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:18.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.804+0000 7ff2cfab9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c81b0f00 con 0x5584dc3c3c70 2024-09-19T05:14:18.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.804+0000 7ff2cfab9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c81b0830 con 0x5584dc3c4590 2024-09-19T05:14:18.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.804+0000 7ff2cfab9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c81ada90 con 0x5584dc3cee50 2024-09-19T05:14:18.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.804+0000 7ff2cfab9740 10 client.0 _ll_register_callbacks cb 0x7ff2b8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:18.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.805+0000 7ff2c5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584dc3c3c70 0x5584dc3d7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:18.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.805+0000 7ff2c5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584dc3c3c70 0x5584dc3d7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44676/0 (socket says 172.21.15.43:44676) 2024-09-19T05:14:18.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.805+0000 7ff2c5e00640 1 -- 172.21.15.43:0/1331095769 learned_addr learned my addr 172.21.15.43:0/1331095769 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:18.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.805+0000 7ff2c6800640 1 --2- 172.21.15.43:0/1331095769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584dc3cee50 0x5584dc3c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:18.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.805+0000 7ff2c5400640 1 --2- 172.21.15.43:0/1331095769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584dc3c4590 0x5584dc3d7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:18.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.805+0000 7ff2c5400640 1 -- 172.21.15.43:0/1331095769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584dc3cee50 msgr2=0x5584dc3c86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:18.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.805+0000 7ff2c5400640 1 --2- 172.21.15.43:0/1331095769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584dc3cee50 0x5584dc3c86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:18.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.805+0000 7ff2c5400640 1 -- 172.21.15.43:0/1331095769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584dc3c3c70 msgr2=0x5584dc3d7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:18.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.805+0000 7ff2c5400640 1 --2- 172.21.15.43:0/1331095769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584dc3c3c70 0x5584dc3d7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:18.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.805+0000 7ff2c5400640 1 -- 172.21.15.43:0/1331095769 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584dc3c8ed0 con 0x5584dc3c4590 2024-09-19T05:14:18.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.805+0000 7ff2c5e00640 1 --2- 172.21.15.43:0/1331095769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584dc3c3c70 0x5584dc3d7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:18.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.805+0000 7ff2c5400640 1 --2- 172.21.15.43:0/1331095769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584dc3c4590 0x5584dc3d7540 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff2b40cd330 tx=0x7ff2b4098260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=685157e3477e43dd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:18.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.805+0000 7ff2a7e00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:14:18.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.806+0000 7ff2a7e00640 1 -- 172.21.15.43:0/1331095769 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b401c040 con 0x5584dc3c4590 2024-09-19T05:14:18.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.806+0000 7ff2a7e00640 1 -- 172.21.15.43:0/1331095769 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7ff2b40cd5b0 con 0x5584dc3c4590 2024-09-19T05:14:18.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.806+0000 7ff2cfab9740 1 -- 172.21.15.43:0/1331095769 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2b803d6d0 con 0x5584dc3c4590 2024-09-19T05:14:18.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.806+0000 7ff2a7e00640 1 -- 172.21.15.43:0/1331095769 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff2b40a0270 con 0x5584dc3c4590 2024-09-19T05:14:18.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.806+0000 7ff2cfab9740 10 client.27470 fetch_fsmap learned FSMap version 29 2024-09-19T05:14:18.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.806+0000 7ff2cfab9740 1 -- 172.21.15.43:0/1331095769 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5584dc4496f0 con 0x5584dc3c4590 2024-09-19T05:14:18.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.806+0000 7ff2a7e00640 1 -- 172.21.15.43:0/1331095769 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2b803d6d0 con 0x5584dc3c4590 2024-09-19T05:14:18.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.806+0000 7ff2a7e00640 1 -- 172.21.15.43:0/1331095769 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff2b409b310 con 0x5584dc3c4590 2024-09-19T05:14:18.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.807+0000 7ff2cfab9740 10 client.27470 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:14:18.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.807+0000 7ff2cfab9740 10 client.27470 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-19T05:14:18.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.807+0000 7ff2cfab9740 20 client.27470 populate_metadata read hostname 'smithi043' 2024-09-19T05:14:18.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.807+0000 7ff2cfab9740 1 --2- 172.21.15.43:0/1331095769 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5584dc449e70 0x5584dc44c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:18.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.807+0000 7ff2cfab9740 4 client.27470 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:14:18.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.807+0000 7ff2cfab9740 1 -- 172.21.15.43:0/1331095769 --> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5584dc331e60 con 0x5584dc449e70 2024-09-19T05:14:18.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.807+0000 7ff2c5e00640 1 --2- 172.21.15.43:0/1331095769 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5584dc449e70 0x5584dc44c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:18.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.807+0000 7ff2c5e00640 1 --2- 172.21.15.43:0/1331095769 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5584dc449e70 0x5584dc44c290 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=667eb41df6cd1c0d server_cookie=91017799aaa27f27 in_seq=0 out_seq=0 2024-09-19T05:14:18.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.807+0000 7ff2a7e00640 10 client.27470 ms_handle_connect on v2:172.21.15.43:6838/3404555030 2024-09-19T05:14:18.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.808+0000 7ff2a7e00640 1 -- 172.21.15.43:0/1331095769 <== mds.1 v2:172.21.15.43:6838/3404555030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5584dc331e60 con 0x5584dc449e70 2024-09-19T05:14:18.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.808+0000 7ff2a7e00640 10 client.27470 handle_command_reply: tid=0 2024-09-19T05:14:18.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.808+0000 7ff2cfab9740 1 client.27470 shutdown 2024-09-19T05:14:18.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.808+0000 7ff2cfab9740 1 -- 172.21.15.43:0/1331095769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584dc3c4590 msgr2=0x5584dc3d7540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:18.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.808+0000 7ff2cfab9740 1 --2- 172.21.15.43:0/1331095769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584dc3c4590 0x5584dc3d7540 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff2b40cd330 tx=0x7ff2b4098260 comp rx=0 tx=0).stop 2024-09-19T05:14:18.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.808+0000 7ff2cfab9740 1 -- 172.21.15.43:0/1331095769 shutdown_connections 2024-09-19T05:14:18.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.808+0000 7ff2cfab9740 1 --2- 172.21.15.43:0/1331095769 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x5584dc449e70 0x5584dc44c290 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:18.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.808+0000 7ff2cfab9740 1 --2- 172.21.15.43:0/1331095769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584dc3cee50 0x5584dc3c86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:18.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.808+0000 7ff2cfab9740 1 --2- 172.21.15.43:0/1331095769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584dc3c4590 0x5584dc3d7540 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:18.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.808+0000 7ff2cfab9740 1 --2- 172.21.15.43:0/1331095769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584dc3c3c70 0x5584dc3d7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:18.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.808+0000 7ff2cfab9740 1 -- 172.21.15.43:0/1331095769 >> 172.21.15.43:0/1331095769 conn(0x5584dc3c0200 msgr2=0x5584dc3c5760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:18.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.808+0000 7ff2a7e00640 0 client.27470 ms_handle_reset on v2:172.21.15.43:6838/3404555030 2024-09-19T05:14:18.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.808+0000 7ff2cfab9740 1 -- 172.21.15.43:0/1331095769 shutdown_connections 2024-09-19T05:14:18.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.808+0000 7ff2cfab9740 1 -- 172.21.15.43:0/1331095769 wait complete. 2024-09-19T05:14:18.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.808+0000 7ff2cfab9740 20 client.27470 trim_cache size 0 max 16384 2024-09-19T05:14:18.131 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:14:18.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.809+0000 7ff2cd000640 1 -- 172.21.15.43:0/3288807032 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff2ac0a3430 msgr2=0x7ff2ac0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:18.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.809+0000 7ff2cd000640 1 --2- 172.21.15.43:0/3288807032 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff2ac0a3430 0x7ff2ac0a58f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8004770 tx=0x7ff2b80046c0 comp rx=0 tx=0).stop 2024-09-19T05:14:18.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.809+0000 7ff2cd000640 1 -- 172.21.15.43:0/3288807032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2c81049a0 msgr2=0x7ff2c819ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:18.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.809+0000 7ff2cd000640 1 --2- 172.21.15.43:0/3288807032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2c81049a0 0x7ff2c819ab60 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff2b400e9f0 tx=0x7ff2b400edf0 comp rx=0 tx=0).stop 2024-09-19T05:14:18.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.809+0000 7ff2cd000640 1 -- 172.21.15.43:0/3288807032 shutdown_connections 2024-09-19T05:14:18.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.809+0000 7ff2cd000640 1 --2- 172.21.15.43:0/3288807032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2c8109120 0x7ff2c819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:18.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.809+0000 7ff2cd000640 1 --2- 172.21.15.43:0/3288807032 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff2ac0a3430 0x7ff2ac0a58f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:18.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.809+0000 7ff2cd000640 1 --2- 172.21.15.43:0/3288807032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2c81049a0 0x7ff2c819ab60 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:18.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.809+0000 7ff2cd000640 1 --2- 172.21.15.43:0/3288807032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2c8103ff0 0x7ff2c819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:18.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.809+0000 7ff2cd000640 1 -- 172.21.15.43:0/3288807032 >> 172.21.15.43:0/3288807032 conn(0x7ff2c80ffa90 msgr2=0x7ff2c8100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:18.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.809+0000 7ff2cd000640 1 -- 172.21.15.43:0/3288807032 shutdown_connections 2024-09-19T05:14:18.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:17.809+0000 7ff2cd000640 1 -- 172.21.15.43:0/3288807032 wait complete. 2024-09-19T05:14:18.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:18 smithi043 ceph-mon[30800]: pgmap v300: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 242 B/s wr, 12 op/s 2024-09-19T05:14:18.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:18 smithi151 ceph-mon[35425]: pgmap v300: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 242 B/s wr, 12 op/s 2024-09-19T05:14:18.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:18 smithi096 ceph-mon[35085]: pgmap v300: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 242 B/s wr, 12 op/s 2024-09-19T05:14:18.576 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T05:14:18.954 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:14:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.505+0000 7fc2aa800640 1 -- 172.21.15.43:0/1609263528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2a4103e30 msgr2=0x7fc2a4104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.505+0000 7fc2aa800640 1 --2- 172.21.15.43:0/1609263528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2a4103e30 0x7fc2a4104210 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc298009ca0 tx=0x7fc29802a8c0 comp rx=0 tx=0).stop 2024-09-19T05:14:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.505+0000 7fc2aa800640 1 -- 172.21.15.43:0/1609263528 shutdown_connections 2024-09-19T05:14:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.505+0000 7fc2aa800640 1 --2- 172.21.15.43:0/1609263528 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2a4108f60 0x7fc2a4110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.505+0000 7fc2aa800640 1 --2- 172.21.15.43:0/1609263528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2a41047e0 0x7fc2a4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.505+0000 7fc2aa800640 1 --2- 172.21.15.43:0/1609263528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2a4103e30 0x7fc2a4104210 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.505+0000 7fc2aa800640 1 -- 172.21.15.43:0/1609263528 >> 172.21.15.43:0/1609263528 conn(0x7fc2a40ff8d0 msgr2=0x7fc2a4101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.505+0000 7fc2aa800640 1 -- 172.21.15.43:0/1609263528 shutdown_connections 2024-09-19T05:14:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.505+0000 7fc2aa800640 1 -- 172.21.15.43:0/1609263528 wait complete. 2024-09-19T05:14:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.506+0000 7fc2aa800640 1 Processor -- start 2024-09-19T05:14:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.506+0000 7fc2aa800640 1 -- start start 2024-09-19T05:14:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.506+0000 7fc2aa800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2a4103e30 0x7fc2a419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.507+0000 7fc2aa800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2a41047e0 0x7fc2a419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.507+0000 7fc2aa800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2a4108f60 0x7fc2a419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.507+0000 7fc2aa800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a4114010 con 0x7fc2a4103e30 2024-09-19T05:14:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.507+0000 7fc2aa800640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a4113e90 con 0x7fc2a4108f60 2024-09-19T05:14:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.507+0000 7fc2aa800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a4114190 con 0x7fc2a41047e0 2024-09-19T05:14:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.507+0000 7fc2a9400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2a4103e30 0x7fc2a419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.507+0000 7fc2a8a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2a41047e0 0x7fc2a419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.507+0000 7fc2a9400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2a4103e30 0x7fc2a419a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44700/0 (socket says 172.21.15.43:44700) 2024-09-19T05:14:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.507+0000 7fc2a9400640 1 -- 172.21.15.43:0/4046111754 learned_addr learned my addr 172.21.15.43:0/4046111754 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.507+0000 7fc2a9e00640 1 --2- 172.21.15.43:0/4046111754 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2a4108f60 0x7fc2a419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.507+0000 7fc2a8a00640 1 -- 172.21.15.43:0/4046111754 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2a4108f60 msgr2=0x7fc2a419ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.507+0000 7fc2a8a00640 1 --2- 172.21.15.43:0/4046111754 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2a4108f60 0x7fc2a419ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.507+0000 7fc2a8a00640 1 -- 172.21.15.43:0/4046111754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2a4103e30 msgr2=0x7fc2a419a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.507+0000 7fc2a8a00640 1 --2- 172.21.15.43:0/4046111754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2a4103e30 0x7fc2a419a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.507+0000 7fc2a8a00640 1 -- 172.21.15.43:0/4046111754 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2a419f400 con 0x7fc2a41047e0 2024-09-19T05:14:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.508+0000 7fc2a8a00640 1 --2- 172.21.15.43:0/4046111754 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2a41047e0 0x7fc2a419a990 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc29000e9f0 tx=0x7fc29000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.508+0000 7fc2a2000640 1 -- 172.21.15.43:0/4046111754 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29000ccc0 con 0x7fc2a41047e0 2024-09-19T05:14:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.508+0000 7fc2a2000640 1 -- 172.21.15.43:0/4046111754 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc290010910 con 0x7fc2a41047e0 2024-09-19T05:14:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.508+0000 7fc2aa800640 1 -- 172.21.15.43:0/4046111754 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2a419f6f0 con 0x7fc2a41047e0 2024-09-19T05:14:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.508+0000 7fc2a2000640 1 -- 172.21.15.43:0/4046111754 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc290018960 con 0x7fc2a41047e0 2024-09-19T05:14:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.508+0000 7fc2aa800640 1 -- 172.21.15.43:0/4046111754 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2a41a6fd0 con 0x7fc2a41047e0 2024-09-19T05:14:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.511+0000 7fc2ae7c1740 1 Processor -- start 2024-09-19T05:14:19.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.511+0000 7fc2a2000640 1 -- 172.21.15.43:0/4046111754 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fc290010ac0 con 0x7fc2a41047e0 2024-09-19T05:14:19.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.511+0000 7fc2ae7c1740 1 -- start start 2024-09-19T05:14:19.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.512+0000 7fc2ae7c1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558a3599e40 0x5558a359a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:19.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.512+0000 7fc2ae7c1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558a359a760 0x5558a35a4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:19.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.512+0000 7fc2ae7c1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558a35a5020 0x5558a35a7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:19.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.512+0000 7fc2ae7c1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5558a35003e0 con 0x5558a359a760 2024-09-19T05:14:19.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.512+0000 7fc2ae7c1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5558a34a7d80 con 0x5558a35a5020 2024-09-19T05:14:19.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.512+0000 7fc2ae7c1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5558a34fc3a0 con 0x5558a3599e40 2024-09-19T05:14:19.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.512+0000 7fc2a9400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558a359a760 0x5558a35a4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:19.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.512+0000 7fc2a9e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558a3599e40 0x5558a359a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:19.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.512+0000 7fc2a9e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558a3599e40 0x5558a359a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:51296/0 (socket says 172.21.15.43:51296) 2024-09-19T05:14:19.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.512+0000 7fc2a9e00640 1 -- 172.21.15.43:0/1215519008 learned_addr learned my addr 172.21.15.43:0/1215519008 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:19.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.512+0000 7fc2a8a00640 1 --2- 172.21.15.43:0/1215519008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558a35a5020 0x5558a35a7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:19.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.513+0000 7fc2a9400640 1 -- 172.21.15.43:0/1215519008 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558a3599e40 msgr2=0x5558a359a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.513+0000 7fc2a9400640 1 --2- 172.21.15.43:0/1215519008 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558a3599e40 0x5558a359a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.513+0000 7fc2a9400640 1 -- 172.21.15.43:0/1215519008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558a35a5020 msgr2=0x5558a35a7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.513+0000 7fc2a9400640 1 --2- 172.21.15.43:0/1215519008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558a35a5020 0x5558a35a7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.513+0000 7fc2a9400640 1 -- 172.21.15.43:0/1215519008 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558a34c6ae0 con 0x5558a359a760 2024-09-19T05:14:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.513+0000 7fc2a2000640 1 --2- 172.21.15.43:0/4046111754 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fc28c0a3430 0x7fc28c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.513+0000 7fc2a9400640 1 --2- 172.21.15.43:0/1215519008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558a359a760 0x5558a35a4ae0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fc298006010 tx=0x7fc29802f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a8d0166d47b8041 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.513+0000 7fc2a2000640 1 -- 172.21.15.43:0/4046111754 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7fc2900d0080 con 0x7fc2a41047e0 2024-09-19T05:14:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.513+0000 7fc2a9e00640 1 --2- 172.21.15.43:0/4046111754 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fc28c0a3430 0x7fc28c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.514+0000 7fc296600640 1 -- 172.21.15.43:0/1215519008 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc298031db0 con 0x5558a359a760 2024-09-19T05:14:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.514+0000 7fc296600640 1 -- 172.21.15.43:0/1215519008 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc298036030 con 0x5558a359a760 2024-09-19T05:14:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.514+0000 7fc296600640 1 -- 172.21.15.43:0/1215519008 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc298004280 con 0x5558a359a760 2024-09-19T05:14:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.514+0000 7fc2ae7c1740 1 -- 172.21.15.43:0/1215519008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558a359a760 msgr2=0x5558a35a4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.514+0000 7fc2a9e00640 1 --2- 172.21.15.43:0/4046111754 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fc28c0a3430 0x7fc28c0a58f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc29c009e60 tx=0x7fc29c005f90 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.514+0000 7fc2ae7c1740 1 --2- 172.21.15.43:0/1215519008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558a359a760 0x5558a35a4ae0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fc298006010 tx=0x7fc29802f310 comp rx=0 tx=0).stop 2024-09-19T05:14:19.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.514+0000 7fc2ae7c1740 1 -- 172.21.15.43:0/1215519008 shutdown_connections 2024-09-19T05:14:19.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.514+0000 7fc2ae7c1740 1 --2- 172.21.15.43:0/1215519008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558a35a5020 0x5558a35a7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:19.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.514+0000 7fc2ae7c1740 1 --2- 172.21.15.43:0/1215519008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558a359a760 0x5558a35a4ae0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:19.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.514+0000 7fc2ae7c1740 1 --2- 172.21.15.43:0/1215519008 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558a3599e40 0x5558a359a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:19.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.514+0000 7fc2ae7c1740 1 -- 172.21.15.43:0/1215519008 >> 172.21.15.43:0/1215519008 conn(0x5558a35963d0 msgr2=0x5558a35967b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:19.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.515+0000 7fc2ae7c1740 1 -- 172.21.15.43:0/1215519008 shutdown_connections 2024-09-19T05:14:19.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.515+0000 7fc2ae7c1740 1 -- 172.21.15.43:0/1215519008 wait complete. 2024-09-19T05:14:19.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.515+0000 7fc2ae7c1740 1 -- start start 2024-09-19T05:14:19.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.515+0000 7fc2ae7c1740 1 Processor -- start 2024-09-19T05:14:19.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.516+0000 7fc2ae7c1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558a3599e40 0x5558a35a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:19.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.516+0000 7fc2ae7c1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558a359a760 0x5558a35a2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:19.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.516+0000 7fc2ae7c1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558a35a5020 0x5558a35a2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:19.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.516+0000 7fc2ae7c1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5558a35a7a70 con 0x5558a359a760 2024-09-19T05:14:19.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.516+0000 7fc2ae7c1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5558a34bb0f0 con 0x5558a3599e40 2024-09-19T05:14:19.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.516+0000 7fc2ae7c1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5558a35a7bf0 con 0x5558a35a5020 2024-09-19T05:14:19.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.516+0000 7fc2ae7c1740 10 client.0 _ll_register_callbacks cb 0x5558a34a2bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:19.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.516+0000 7fc2a9400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558a3599e40 0x5558a35a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:19.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.516+0000 7fc2a8a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558a359a760 0x5558a35a2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:19.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.516+0000 7fc2a9400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558a3599e40 0x5558a35a1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:48878/0 (socket says 172.21.15.43:48878) 2024-09-19T05:14:19.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.516+0000 7fc2a9400640 1 -- 172.21.15.43:0/2497129478 learned_addr learned my addr 172.21.15.43:0/2497129478 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:19.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.516+0000 7fc2a8a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558a359a760 0x5558a35a2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44722/0 (socket says 172.21.15.43:44722) 2024-09-19T05:14:19.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.516+0000 7fc2a9e00640 1 --2- 172.21.15.43:0/2497129478 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558a35a5020 0x5558a35a2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:19.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.516+0000 7fc2a8a00640 1 -- 172.21.15.43:0/2497129478 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558a35a5020 msgr2=0x5558a35a2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:19.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.516+0000 7fc2a8a00640 1 --2- 172.21.15.43:0/2497129478 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558a35a5020 0x5558a35a2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:19.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.516+0000 7fc2a8a00640 1 -- 172.21.15.43:0/2497129478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558a3599e40 msgr2=0x5558a35a1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:19.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.516+0000 7fc2a8a00640 1 --2- 172.21.15.43:0/2497129478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558a3599e40 0x5558a35a1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:19.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.516+0000 7fc2a8a00640 1 -- 172.21.15.43:0/2497129478 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558a32713c0 con 0x5558a359a760 2024-09-19T05:14:19.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.517+0000 7fc2a8a00640 1 --2- 172.21.15.43:0/2497129478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558a359a760 0x5558a35a2420 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc2900cd760 tx=0x7fc290010ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=519a1dbcf3c5a9ef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:19.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.517+0000 7fc28aa00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:14:19.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.517+0000 7fc28aa00640 1 -- 172.21.15.43:0/2497129478 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29009bee0 con 0x5558a359a760 2024-09-19T05:14:19.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.517+0000 7fc28aa00640 1 -- 172.21.15.43:0/2497129478 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7fc2900d20d0 con 0x5558a359a760 2024-09-19T05:14:19.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.517+0000 7fc2ae7c1740 1 -- 172.21.15.43:0/2497129478 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc298036030 con 0x5558a359a760 2024-09-19T05:14:19.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.517+0000 7fc28aa00640 1 -- 172.21.15.43:0/2497129478 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc29009bd00 con 0x5558a359a760 2024-09-19T05:14:19.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.517+0000 7fc2ae7c1740 10 client.17457 fetch_fsmap learned FSMap version 29 2024-09-19T05:14:19.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.517+0000 7fc2ae7c1740 1 -- 172.21.15.43:0/2497129478 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5558a35b3ab0 con 0x5558a359a760 2024-09-19T05:14:19.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.517+0000 7fc28aa00640 1 -- 172.21.15.43:0/2497129478 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc298036030 con 0x5558a359a760 2024-09-19T05:14:19.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.518+0000 7fc28aa00640 1 -- 172.21.15.43:0/2497129478 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc29009b7f0 con 0x5558a359a760 2024-09-19T05:14:19.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.518+0000 7fc2ae7c1740 10 client.17457 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:14:19.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.518+0000 7fc2ae7c1740 10 client.17457 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:14:19.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.518+0000 7fc2ae7c1740 20 client.17457 populate_metadata read hostname 'smithi043' 2024-09-19T05:14:19.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.518+0000 7fc2ae7c1740 1 --2- 172.21.15.43:0/2497129478 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5558a35b4f10 0x5558a35b7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:19.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.518+0000 7fc2ae7c1740 4 client.17457 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:14:19.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.518+0000 7fc2ae7c1740 1 -- 172.21.15.43:0/2497129478 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558a350d950 con 0x5558a35b4f10 2024-09-19T05:14:19.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.518+0000 7fc2a9400640 1 --2- 172.21.15.43:0/2497129478 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5558a35b4f10 0x5558a35b7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:19.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.519+0000 7fc2a9400640 1 --2- 172.21.15.43:0/2497129478 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5558a35b4f10 0x5558a35b7330 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9359db7293476949 server_cookie=76191df77d9a116d in_seq=0 out_seq=0 2024-09-19T05:14:19.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.519+0000 7fc28aa00640 10 client.17457 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:14:19.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.519+0000 7fc28aa00640 1 -- 172.21.15.43:0/2497129478 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5558a350d950 con 0x5558a35b4f10 2024-09-19T05:14:19.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.519+0000 7fc28aa00640 10 client.17457 handle_command_reply: tid=0 2024-09-19T05:14:19.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.520+0000 7fc2ae7c1740 1 client.17457 shutdown 2024-09-19T05:14:19.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.520+0000 7fc2ae7c1740 1 -- 172.21.15.43:0/2497129478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558a359a760 msgr2=0x5558a35a2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:19.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.520+0000 7fc2ae7c1740 1 --2- 172.21.15.43:0/2497129478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558a359a760 0x5558a35a2420 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc2900cd760 tx=0x7fc290010ef0 comp rx=0 tx=0).stop 2024-09-19T05:14:19.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.520+0000 7fc2ae7c1740 1 -- 172.21.15.43:0/2497129478 shutdown_connections 2024-09-19T05:14:19.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.520+0000 7fc2ae7c1740 1 --2- 172.21.15.43:0/2497129478 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5558a35b4f10 0x5558a35b7330 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:19.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.520+0000 7fc2ae7c1740 1 --2- 172.21.15.43:0/2497129478 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558a35a5020 0x5558a35a2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:19.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.520+0000 7fc2ae7c1740 1 --2- 172.21.15.43:0/2497129478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558a359a760 0x5558a35a2420 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:19.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.520+0000 7fc2ae7c1740 1 --2- 172.21.15.43:0/2497129478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558a3599e40 0x5558a35a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:19.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.520+0000 7fc28aa00640 0 client.17457 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:14:19.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.520+0000 7fc2ae7c1740 1 -- 172.21.15.43:0/2497129478 >> 172.21.15.43:0/2497129478 conn(0x5558a35963d0 msgr2=0x5558a359c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:19.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.520+0000 7fc2ae7c1740 1 -- 172.21.15.43:0/2497129478 shutdown_connections 2024-09-19T05:14:19.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.520+0000 7fc2ae7c1740 1 -- 172.21.15.43:0/2497129478 wait complete. 2024-09-19T05:14:19.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.520+0000 7fc2ae7c1740 20 client.17457 trim_cache size 0 max 16384 2024-09-19T05:14:19.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.534+0000 7fc2ae7c1740 1 Processor -- start 2024-09-19T05:14:19.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.534+0000 7fc2ae7c1740 1 -- start start 2024-09-19T05:14:19.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.535+0000 7fc2ae7c1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558a3599e40 0x5558a359bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:19.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.535+0000 7fc2ae7c1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558a359a760 0x5558a35bcc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:19.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.535+0000 7fc2ae7c1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558a35a5020 0x5558a35bd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:19.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.535+0000 7fc2ae7c1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a41af900 con 0x5558a35a5020 2024-09-19T05:14:19.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.535+0000 7fc2ae7c1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a41b1440 con 0x5558a3599e40 2024-09-19T05:14:19.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.535+0000 7fc2ae7c1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a41affd0 con 0x5558a359a760 2024-09-19T05:14:19.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.535+0000 7fc2a9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558a35a5020 0x5558a35bd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:19.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.535+0000 7fc2a9400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558a3599e40 0x5558a359bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:19.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.535+0000 7fc2a9400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558a3599e40 0x5558a359bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:48886/0 (socket says 172.21.15.43:48886) 2024-09-19T05:14:19.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.535+0000 7fc2a9400640 1 -- 172.21.15.43:0/1116882989 learned_addr learned my addr 172.21.15.43:0/1116882989 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:19.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.535+0000 7fc2a8a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558a359a760 0x5558a35bcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:19.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.535+0000 7fc2a9400640 1 -- 172.21.15.43:0/1116882989 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558a359a760 msgr2=0x5558a35bcc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:19.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.535+0000 7fc2a9400640 1 --2- 172.21.15.43:0/1116882989 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558a359a760 0x5558a35bcc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:19.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.535+0000 7fc2a9400640 1 -- 172.21.15.43:0/1116882989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558a35a5020 msgr2=0x5558a35bd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:19.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.535+0000 7fc2a9400640 1 --2- 172.21.15.43:0/1116882989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558a35a5020 0x5558a35bd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:19.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.535+0000 7fc2a9400640 1 -- 172.21.15.43:0/1116882989 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558a3596200 con 0x5558a3599e40 2024-09-19T05:14:19.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.535+0000 7fc2a8a00640 1 --2- 172.21.15.43:0/1116882989 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558a359a760 0x5558a35bcc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:19.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.536+0000 7fc2a9400640 1 --2- 172.21.15.43:0/1116882989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558a3599e40 0x5558a359bfa0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc298009d90 tx=0x7fc298031b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4c73493b180bc00 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:19.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.536+0000 7fc295c00640 1 -- 172.21.15.43:0/1116882989 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc298054020 con 0x5558a3599e40 2024-09-19T05:14:19.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.536+0000 7fc295c00640 1 -- 172.21.15.43:0/1116882989 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc298036030 con 0x5558a3599e40 2024-09-19T05:14:19.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.536+0000 7fc295c00640 1 -- 172.21.15.43:0/1116882989 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29804a040 con 0x5558a3599e40 2024-09-19T05:14:19.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.537+0000 7fc2ae7c1740 1 -- 172.21.15.43:0/1116882989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558a3599e40 msgr2=0x5558a359bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:19.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.537+0000 7fc2ae7c1740 1 --2- 172.21.15.43:0/1116882989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558a3599e40 0x5558a359bfa0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc298009d90 tx=0x7fc298031b10 comp rx=0 tx=0).stop 2024-09-19T05:14:19.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.537+0000 7fc2ae7c1740 1 -- 172.21.15.43:0/1116882989 shutdown_connections 2024-09-19T05:14:19.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.537+0000 7fc2ae7c1740 1 --2- 172.21.15.43:0/1116882989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558a35a5020 0x5558a35bd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:19.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.537+0000 7fc2ae7c1740 1 --2- 172.21.15.43:0/1116882989 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558a359a760 0x5558a35bcc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:19.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.537+0000 7fc2ae7c1740 1 --2- 172.21.15.43:0/1116882989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558a3599e40 0x5558a359bfa0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:19.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.537+0000 7fc2ae7c1740 1 -- 172.21.15.43:0/1116882989 >> 172.21.15.43:0/1116882989 conn(0x5558a35963d0 msgr2=0x5558a359b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:19.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.537+0000 7fc2ae7c1740 1 -- 172.21.15.43:0/1116882989 shutdown_connections 2024-09-19T05:14:19.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.537+0000 7fc2ae7c1740 1 -- 172.21.15.43:0/1116882989 wait complete. 2024-09-19T05:14:19.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.537+0000 7fc2ae7c1740 1 -- start start 2024-09-19T05:14:19.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.537+0000 7fc2ae7c1740 1 Processor -- start 2024-09-19T05:14:19.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.538+0000 7fc2ae7c1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558a3599e40 0x5558a35ace70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:19.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.538+0000 7fc2ae7c1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558a359a760 0x5558a35ad3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:19.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.538+0000 7fc2ae7c1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558a35a5020 0x5558a35a9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:19.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.538+0000 7fc2ae7c1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a41b0d70 con 0x5558a35a5020 2024-09-19T05:14:19.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.538+0000 7fc2ae7c1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a41b06a0 con 0x5558a3599e40 2024-09-19T05:14:19.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.538+0000 7fc2ae7c1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a41ad900 con 0x5558a359a760 2024-09-19T05:14:19.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.538+0000 7fc2ae7c1740 10 client.0 _ll_register_callbacks cb 0x7fc29000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:19.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.538+0000 7fc2a9400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558a3599e40 0x5558a35ace70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:19.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.538+0000 7fc2a9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558a35a5020 0x5558a35a9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:19.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.538+0000 7fc2a9400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558a3599e40 0x5558a35ace70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:48900/0 (socket says 172.21.15.43:48900) 2024-09-19T05:14:19.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.538+0000 7fc2a9400640 1 -- 172.21.15.43:0/4061827 learned_addr learned my addr 172.21.15.43:0/4061827 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:19.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.538+0000 7fc2a8a00640 1 --2- 172.21.15.43:0/4061827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558a359a760 0x5558a35ad3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:19.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.539+0000 7fc2a9400640 1 -- 172.21.15.43:0/4061827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558a359a760 msgr2=0x5558a35ad3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:19.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.539+0000 7fc2a9400640 1 --2- 172.21.15.43:0/4061827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558a359a760 0x5558a35ad3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:19.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.539+0000 7fc2a9400640 1 -- 172.21.15.43:0/4061827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558a35a5020 msgr2=0x5558a35a9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:19.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.539+0000 7fc2a9400640 1 --2- 172.21.15.43:0/4061827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558a35a5020 0x5558a35a9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:19.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.539+0000 7fc2a9400640 1 -- 172.21.15.43:0/4061827 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558a35aa380 con 0x5558a3599e40 2024-09-19T05:14:19.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.539+0000 7fc2a9400640 1 --2- 172.21.15.43:0/4061827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558a3599e40 0x5558a35ace70 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc298009dc0 tx=0x7fc298031a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=599d7e237f140348 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:19.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.539+0000 7fc28aa00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:14:19.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.539+0000 7fc28aa00640 1 -- 172.21.15.43:0/4061827 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc298054030 con 0x5558a3599e40 2024-09-19T05:14:19.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.539+0000 7fc2ae7c1740 1 -- 172.21.15.43:0/4061827 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc298036030 con 0x5558a3599e40 2024-09-19T05:14:19.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.539+0000 7fc28aa00640 1 -- 172.21.15.43:0/4061827 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7fc298003970 con 0x5558a3599e40 2024-09-19T05:14:19.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.540+0000 7fc28aa00640 1 -- 172.21.15.43:0/4061827 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc298004b40 con 0x5558a3599e40 2024-09-19T05:14:19.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.540+0000 7fc2ae7c1740 10 client.27488 fetch_fsmap learned FSMap version 29 2024-09-19T05:14:19.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.540+0000 7fc2ae7c1740 1 -- 172.21.15.43:0/4061827 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5558a3620400 con 0x5558a3599e40 2024-09-19T05:14:19.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.540+0000 7fc28aa00640 1 -- 172.21.15.43:0/4061827 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc298036030 con 0x5558a3599e40 2024-09-19T05:14:19.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.540+0000 7fc28aa00640 1 -- 172.21.15.43:0/4061827 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc298069080 con 0x5558a3599e40 2024-09-19T05:14:19.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.540+0000 7fc2ae7c1740 10 client.27488 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:14:19.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.540+0000 7fc2ae7c1740 10 client.27488 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:14:19.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.540+0000 7fc2ae7c1740 20 client.27488 populate_metadata read hostname 'smithi043' 2024-09-19T05:14:19.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.540+0000 7fc2ae7c1740 1 --2- 172.21.15.43:0/4061827 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5558a36207e0 0x5558a3622bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:19.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.540+0000 7fc2ae7c1740 4 client.27488 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:14:19.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.540+0000 7fc2ae7c1740 1 -- 172.21.15.43:0/4061827 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5558a3507e60 con 0x5558a36207e0 2024-09-19T05:14:19.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.540+0000 7fc2a8a00640 1 --2- 172.21.15.43:0/4061827 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5558a36207e0 0x5558a3622bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:19.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.541+0000 7fc2a8a00640 1 --2- 172.21.15.43:0/4061827 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5558a36207e0 0x5558a3622bd0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71e0788cc894bcd0 server_cookie=8260aec4a9636394 in_seq=0 out_seq=0 2024-09-19T05:14:19.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.541+0000 7fc28aa00640 10 client.27488 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:14:19.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.541+0000 7fc28aa00640 1 -- 172.21.15.43:0/4061827 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5558a3507e60 con 0x5558a36207e0 2024-09-19T05:14:19.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.541+0000 7fc28aa00640 10 client.27488 handle_command_reply: tid=0 2024-09-19T05:14:19.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.541+0000 7fc2ae7c1740 1 client.27488 shutdown 2024-09-19T05:14:19.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.541+0000 7fc2ae7c1740 1 -- 172.21.15.43:0/4061827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558a3599e40 msgr2=0x5558a35ace70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:19.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.541+0000 7fc2ae7c1740 1 --2- 172.21.15.43:0/4061827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558a3599e40 0x5558a35ace70 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc298009dc0 tx=0x7fc298031a10 comp rx=0 tx=0).stop 2024-09-19T05:14:19.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.541+0000 7fc2ae7c1740 1 -- 172.21.15.43:0/4061827 shutdown_connections 2024-09-19T05:14:19.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.541+0000 7fc2ae7c1740 1 --2- 172.21.15.43:0/4061827 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5558a36207e0 0x5558a3622bd0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:19.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.541+0000 7fc2ae7c1740 1 --2- 172.21.15.43:0/4061827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558a35a5020 0x5558a35a9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:19.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.541+0000 7fc2ae7c1740 1 --2- 172.21.15.43:0/4061827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558a359a760 0x5558a35ad3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:19.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.541+0000 7fc2ae7c1740 1 --2- 172.21.15.43:0/4061827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558a3599e40 0x5558a35ace70 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:19.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.541+0000 7fc2ae7c1740 1 -- 172.21.15.43:0/4061827 >> 172.21.15.43:0/4061827 conn(0x5558a35963d0 msgr2=0x5558a3594c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:19.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.541+0000 7fc28aa00640 0 client.27488 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:14:19.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.542+0000 7fc2ae7c1740 1 -- 172.21.15.43:0/4061827 shutdown_connections 2024-09-19T05:14:19.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.542+0000 7fc2ae7c1740 1 -- 172.21.15.43:0/4061827 wait complete. 2024-09-19T05:14:19.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.542+0000 7fc2ae7c1740 20 client.27488 trim_cache size 0 max 16384 2024-09-19T05:14:19.547 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:14:19.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.542+0000 7fc2aa800640 1 -- 172.21.15.43:0/4046111754 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fc28c0a3430 msgr2=0x7fc28c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:19.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.542+0000 7fc2aa800640 1 --2- 172.21.15.43:0/4046111754 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fc28c0a3430 0x7fc28c0a58f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc29c009e60 tx=0x7fc29c005f90 comp rx=0 tx=0).stop 2024-09-19T05:14:19.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.542+0000 7fc2aa800640 1 -- 172.21.15.43:0/4046111754 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2a41047e0 msgr2=0x7fc2a419a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:19.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.542+0000 7fc2aa800640 1 --2- 172.21.15.43:0/4046111754 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2a41047e0 0x7fc2a419a990 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc29000e9f0 tx=0x7fc29000eeb0 comp rx=0 tx=0).stop 2024-09-19T05:14:19.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.543+0000 7fc2aa800640 1 -- 172.21.15.43:0/4046111754 shutdown_connections 2024-09-19T05:14:19.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.543+0000 7fc2aa800640 1 --2- 172.21.15.43:0/4046111754 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fc28c0a3430 0x7fc28c0a58f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:19.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.543+0000 7fc2aa800640 1 --2- 172.21.15.43:0/4046111754 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2a4108f60 0x7fc2a419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:19.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.543+0000 7fc2aa800640 1 --2- 172.21.15.43:0/4046111754 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2a41047e0 0x7fc2a419a990 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:19.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.543+0000 7fc2aa800640 1 --2- 172.21.15.43:0/4046111754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2a4103e30 0x7fc2a419a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:19.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.543+0000 7fc2aa800640 1 -- 172.21.15.43:0/4046111754 >> 172.21.15.43:0/4046111754 conn(0x7fc2a40ff8d0 msgr2=0x7fc2a4100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:19.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.543+0000 7fc2aa800640 1 -- 172.21.15.43:0/4046111754 shutdown_connections 2024-09-19T05:14:19.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:19.543+0000 7fc2aa800640 1 -- 172.21.15.43:0/4046111754 wait complete. 2024-09-19T05:14:20.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:20 smithi043 ceph-mon[30800]: pgmap v301: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T05:14:20.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:20 smithi151 ceph-mon[35425]: pgmap v301: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T05:14:20.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:20 smithi096 ceph-mon[35085]: pgmap v301: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T05:14:21.433 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T05:14:21.433 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T05:14:21.822 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:14:22.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:22 smithi043 ceph-mon[30800]: pgmap v302: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T05:14:22.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.401+0000 7fba20e00640 1 -- 172.21.15.43:0/348497513 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba1c101de0 msgr2=0x7fba1c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:22.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.401+0000 7fba20e00640 1 --2- 172.21.15.43:0/348497513 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba1c101de0 0x7fba1c1021c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fba0c009900 tx=0x7fba0c02a3f0 comp rx=0 tx=0).stop 2024-09-19T05:14:22.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.402+0000 7fba20e00640 1 -- 172.21.15.43:0/348497513 shutdown_connections 2024-09-19T05:14:22.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.402+0000 7fba20e00640 1 --2- 172.21.15.43:0/348497513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba1c106f10 0x7fba1c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:22.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.402+0000 7fba20e00640 1 --2- 172.21.15.43:0/348497513 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba1c102790 0x7fba1c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:22.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.402+0000 7fba20e00640 1 --2- 172.21.15.43:0/348497513 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba1c101de0 0x7fba1c1021c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:22.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.402+0000 7fba20e00640 1 -- 172.21.15.43:0/348497513 >> 172.21.15.43:0/348497513 conn(0x7fba1c0fd8c0 msgr2=0x7fba1c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:22.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.402+0000 7fba20e00640 1 -- 172.21.15.43:0/348497513 shutdown_connections 2024-09-19T05:14:22.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.402+0000 7fba20e00640 1 -- 172.21.15.43:0/348497513 wait complete. 2024-09-19T05:14:22.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:22 smithi151 ceph-mon[35425]: pgmap v302: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T05:14:22.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.403+0000 7fba20e00640 1 Processor -- start 2024-09-19T05:14:22.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.403+0000 7fba20e00640 1 -- start start 2024-09-19T05:14:22.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.403+0000 7fba20e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba1c101de0 0x7fba1c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:22.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.403+0000 7fba20e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba1c102790 0x7fba1c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:22.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.403+0000 7fba20e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba1c106f10 0x7fba1c19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:22.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.403+0000 7fba20e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c111f70 con 0x7fba1c102790 2024-09-19T05:14:22.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.403+0000 7fba20e00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c111df0 con 0x7fba1c101de0 2024-09-19T05:14:22.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.403+0000 7fba20e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c1120f0 con 0x7fba1c106f10 2024-09-19T05:14:22.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.404+0000 7fba11400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba1c102790 0x7fba1c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:22.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.404+0000 7fba19e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba1c101de0 0x7fba1c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:22.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.404+0000 7fba11400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba1c102790 0x7fba1c198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44756/0 (socket says 172.21.15.43:44756) 2024-09-19T05:14:22.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.404+0000 7fba11400640 1 -- 172.21.15.43:0/2230117876 learned_addr learned my addr 172.21.15.43:0/2230117876 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:22.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.404+0000 7fba1a800640 1 --2- 172.21.15.43:0/2230117876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba1c106f10 0x7fba1c19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:22.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.404+0000 7fba19e00640 1 -- 172.21.15.43:0/2230117876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba1c106f10 msgr2=0x7fba1c19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:22.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.404+0000 7fba19e00640 1 --2- 172.21.15.43:0/2230117876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba1c106f10 0x7fba1c19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:22.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.404+0000 7fba19e00640 1 -- 172.21.15.43:0/2230117876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba1c102790 msgr2=0x7fba1c198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:22.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.404+0000 7fba19e00640 1 --2- 172.21.15.43:0/2230117876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba1c102790 0x7fba1c198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:22.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.404+0000 7fba19e00640 1 -- 172.21.15.43:0/2230117876 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba1c19d3e0 con 0x7fba1c101de0 2024-09-19T05:14:22.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.404+0000 7fba19e00640 1 --2- 172.21.15.43:0/2230117876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba1c101de0 0x7fba1c198430 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fba0c005940 tx=0x7fba0c0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:22.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.405+0000 7fba13400640 1 -- 172.21.15.43:0/2230117876 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0c03f030 con 0x7fba1c101de0 2024-09-19T05:14:22.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.405+0000 7fba13400640 1 -- 172.21.15.43:0/2230117876 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba0c0044c0 con 0x7fba1c101de0 2024-09-19T05:14:22.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.405+0000 7fba13400640 1 -- 172.21.15.43:0/2230117876 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0c035070 con 0x7fba1c101de0 2024-09-19T05:14:22.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.405+0000 7fba20e00640 1 -- 172.21.15.43:0/2230117876 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba1c19d670 con 0x7fba1c101de0 2024-09-19T05:14:22.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.405+0000 7fba20e00640 1 -- 172.21.15.43:0/2230117876 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba1c19db00 con 0x7fba1c101de0 2024-09-19T05:14:22.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.408+0000 7fba238a5740 1 Processor -- start 2024-09-19T05:14:22.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.408+0000 7fba238a5740 1 -- start start 2024-09-19T05:14:22.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.408+0000 7fba13400640 1 -- 172.21.15.43:0/2230117876 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fba0c042030 con 0x7fba1c101de0 2024-09-19T05:14:22.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.408+0000 7fba238a5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555799937ac0 0x555799937ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:22.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.408+0000 7fba238a5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557999383e0 0x555799942760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:22.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.408+0000 7fba238a5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555799942ca0 0x555799945090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:22.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.408+0000 7fba238a5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55579989e290 con 0x5557999383e0 2024-09-19T05:14:22.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.408+0000 7fba238a5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555799845b40 con 0x555799937ac0 2024-09-19T05:14:22.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.408+0000 7fba238a5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55579989a250 con 0x555799942ca0 2024-09-19T05:14:22.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.409+0000 7fba19e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557999383e0 0x555799942760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:22.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.409+0000 7fba1a800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555799937ac0 0x555799937ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:22.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.409+0000 7fba1a800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555799937ac0 0x555799937ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:48920/0 (socket says 172.21.15.43:48920) 2024-09-19T05:14:22.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.409+0000 7fba1a800640 1 -- 172.21.15.43:0/3979005106 learned_addr learned my addr 172.21.15.43:0/3979005106 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:22.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.409+0000 7fba1a800640 1 -- 172.21.15.43:0/3979005106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555799942ca0 msgr2=0x555799945090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:22.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.409+0000 7fba11400640 1 --2- 172.21.15.43:0/3979005106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555799942ca0 0x555799945090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:22.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.409+0000 7fba1a800640 1 --2- 172.21.15.43:0/3979005106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555799942ca0 0x555799945090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:22.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.409+0000 7fba1a800640 1 -- 172.21.15.43:0/3979005106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557999383e0 msgr2=0x555799942760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:22.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.409+0000 7fba1a800640 1 --2- 172.21.15.43:0/3979005106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557999383e0 0x555799942760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:22.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.409+0000 7fba1a800640 1 -- 172.21.15.43:0/3979005106 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55579960f6a0 con 0x555799937ac0 2024-09-19T05:14:22.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.409+0000 7fba11400640 1 --2- 172.21.15.43:0/3979005106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555799942ca0 0x555799945090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:14:22.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.410+0000 7fba1a800640 1 --2- 172.21.15.43:0/3979005106 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555799937ac0 0x555799937ea0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fba14009f60 tx=0x7fba14009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a4a5af3060afca3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:22.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.410+0000 7fba0b400640 1 -- 172.21.15.43:0/3979005106 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1401e070 con 0x555799937ac0 2024-09-19T05:14:22.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.410+0000 7fba0b400640 1 -- 172.21.15.43:0/3979005106 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba14004030 con 0x555799937ac0 2024-09-19T05:14:22.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.410+0000 7fba13400640 1 --2- 172.21.15.43:0/2230117876 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fba000a3430 0x7fba000a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:22.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.410+0000 7fba0b400640 1 -- 172.21.15.43:0/3979005106 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1400f6c0 con 0x555799937ac0 2024-09-19T05:14:22.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.410+0000 7fba13400640 1 -- 172.21.15.43:0/2230117876 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7fba0c0effc0 con 0x7fba1c101de0 2024-09-19T05:14:22.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.411+0000 7fba19e00640 1 --2- 172.21.15.43:0/2230117876 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fba000a3430 0x7fba000a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:22.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.411+0000 7fba238a5740 1 -- 172.21.15.43:0/3979005106 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555799937ac0 msgr2=0x555799937ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:22.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.411+0000 7fba238a5740 1 --2- 172.21.15.43:0/3979005106 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555799937ac0 0x555799937ea0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fba14009f60 tx=0x7fba14009f90 comp rx=0 tx=0).stop 2024-09-19T05:14:22.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.411+0000 7fba238a5740 1 -- 172.21.15.43:0/3979005106 shutdown_connections 2024-09-19T05:14:22.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.411+0000 7fba238a5740 1 --2- 172.21.15.43:0/3979005106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555799942ca0 0x555799945090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:22.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.411+0000 7fba238a5740 1 --2- 172.21.15.43:0/3979005106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557999383e0 0x555799942760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:22.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.411+0000 7fba238a5740 1 --2- 172.21.15.43:0/3979005106 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555799937ac0 0x555799937ea0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:22.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.411+0000 7fba238a5740 1 -- 172.21.15.43:0/3979005106 >> 172.21.15.43:0/3979005106 conn(0x555799934050 msgr2=0x555799934430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:22.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.411+0000 7fba238a5740 1 -- 172.21.15.43:0/3979005106 shutdown_connections 2024-09-19T05:14:22.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.411+0000 7fba19e00640 1 --2- 172.21.15.43:0/2230117876 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fba000a3430 0x7fba000a58f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x555799934eb0 tx=0x7fba0c0f3a70 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:22.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.411+0000 7fba238a5740 1 -- 172.21.15.43:0/3979005106 wait complete. 2024-09-19T05:14:22.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.412+0000 7fba238a5740 1 -- start start 2024-09-19T05:14:22.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.412+0000 7fba238a5740 1 Processor -- start 2024-09-19T05:14:22.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.412+0000 7fba238a5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555799937ac0 0x555799947000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:22.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.412+0000 7fba238a5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557999383e0 0x555799947540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:22.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.412+0000 7fba238a5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555799942ca0 0x55579994b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:22.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.412+0000 7fba238a5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555799945960 con 0x5557999383e0 2024-09-19T05:14:22.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.412+0000 7fba238a5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555799858fa0 con 0x555799942ca0 2024-09-19T05:14:22.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.412+0000 7fba238a5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555799945ae0 con 0x555799937ac0 2024-09-19T05:14:22.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.412+0000 7fba238a5740 10 client.0 _ll_register_callbacks cb 0x5557995acba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:22.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.413+0000 7fba1a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555799937ac0 0x555799947000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:22.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.413+0000 7fba11400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557999383e0 0x555799947540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:22.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.413+0000 7fba1a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555799937ac0 0x555799947000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:51342/0 (socket says 172.21.15.43:51342) 2024-09-19T05:14:22.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.413+0000 7fba1a800640 1 -- 172.21.15.43:0/145059241 learned_addr learned my addr 172.21.15.43:0/145059241 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:22.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.413+0000 7fba1a800640 1 --2- 172.21.15.43:0/145059241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555799942ca0 0x55579994b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:22.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.413+0000 7fba1a800640 1 -- 172.21.15.43:0/145059241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555799942ca0 msgr2=0x55579994b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:22.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.413+0000 7fba1a800640 1 --2- 172.21.15.43:0/145059241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555799942ca0 0x55579994b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:22.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.413+0000 7fba1a800640 1 -- 172.21.15.43:0/145059241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557999383e0 msgr2=0x555799947540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:22.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.413+0000 7fba1a800640 1 --2- 172.21.15.43:0/145059241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557999383e0 0x555799947540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:22.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.413+0000 7fba1a800640 1 -- 172.21.15.43:0/145059241 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55579994bd50 con 0x555799937ac0 2024-09-19T05:14:22.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.413+0000 7fba11400640 1 --2- 172.21.15.43:0/145059241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557999383e0 0x555799947540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:22.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.414+0000 7fba1a800640 1 --2- 172.21.15.43:0/145059241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555799937ac0 0x555799947000 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fba14011d60 tx=0x7fba14004990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d15415548960fe3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:22.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.414+0000 7fb9ffe00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:14:22.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.414+0000 7fb9ffe00640 1 -- 172.21.15.43:0/145059241 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1402c060 con 0x555799937ac0 2024-09-19T05:14:22.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.414+0000 7fba238a5740 1 -- 172.21.15.43:0/145059241 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba14004030 con 0x555799937ac0 2024-09-19T05:14:22.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.414+0000 7fb9ffe00640 1 -- 172.21.15.43:0/145059241 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7fba1400f480 con 0x555799937ac0 2024-09-19T05:14:22.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.414+0000 7fb9ffe00640 1 -- 172.21.15.43:0/145059241 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fba1401e040 con 0x555799937ac0 2024-09-19T05:14:22.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.414+0000 7fba238a5740 10 client.27397 fetch_fsmap learned FSMap version 29 2024-09-19T05:14:22.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.414+0000 7fba238a5740 1 -- 172.21.15.43:0/145059241 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555799955cf0 con 0x555799937ac0 2024-09-19T05:14:22.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.414+0000 7fb9ffe00640 1 -- 172.21.15.43:0/145059241 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba14004030 con 0x555799937ac0 2024-09-19T05:14:22.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.415+0000 7fb9ffe00640 1 -- 172.21.15.43:0/145059241 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fba14024030 con 0x555799937ac0 2024-09-19T05:14:22.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.415+0000 7fba238a5740 10 client.27397 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:14:22.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.415+0000 7fba238a5740 10 client.27397 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:14:22.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.415+0000 7fba238a5740 20 client.27397 populate_metadata read hostname 'smithi043' 2024-09-19T05:14:22.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.415+0000 7fba238a5740 1 --2- 172.21.15.43:0/145059241 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x555799957100 0x5557999594f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:22.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.415+0000 7fba238a5740 4 client.27397 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:14:22.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.415+0000 7fba238a5740 1 -- 172.21.15.43:0/145059241 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557998ab800 con 0x555799957100 2024-09-19T05:14:22.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.415+0000 7fba11400640 1 --2- 172.21.15.43:0/145059241 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x555799957100 0x5557999594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:22.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.416+0000 7fba11400640 1 --2- 172.21.15.43:0/145059241 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x555799957100 0x5557999594f0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abd97b5476fca164 server_cookie=4cde20e4e30c06a4 in_seq=0 out_seq=0 2024-09-19T05:14:22.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.416+0000 7fb9ffe00640 10 client.27397 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:14:22.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.416+0000 7fb9ffe00640 1 -- 172.21.15.43:0/145059241 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5557998ab800 con 0x555799957100 2024-09-19T05:14:22.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.416+0000 7fb9ffe00640 10 client.27397 handle_command_reply: tid=0 2024-09-19T05:14:22.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.418+0000 7fba238a5740 1 client.27397 shutdown 2024-09-19T05:14:22.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.418+0000 7fba238a5740 1 -- 172.21.15.43:0/145059241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555799937ac0 msgr2=0x555799947000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:22.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.418+0000 7fba238a5740 1 --2- 172.21.15.43:0/145059241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555799937ac0 0x555799947000 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fba14011d60 tx=0x7fba14004990 comp rx=0 tx=0).stop 2024-09-19T05:14:22.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.418+0000 7fba238a5740 1 -- 172.21.15.43:0/145059241 shutdown_connections 2024-09-19T05:14:22.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.418+0000 7fba238a5740 1 --2- 172.21.15.43:0/145059241 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x555799957100 0x5557999594f0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:22.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.418+0000 7fba238a5740 1 --2- 172.21.15.43:0/145059241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555799942ca0 0x55579994b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:22.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.418+0000 7fba238a5740 1 --2- 172.21.15.43:0/145059241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557999383e0 0x555799947540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:22.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.418+0000 7fba238a5740 1 --2- 172.21.15.43:0/145059241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555799937ac0 0x555799947000 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:22.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.418+0000 7fba238a5740 1 -- 172.21.15.43:0/145059241 >> 172.21.15.43:0/145059241 conn(0x555799934050 msgr2=0x555799939d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:22.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.418+0000 7fb9ffe00640 0 client.27397 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:14:22.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.418+0000 7fba238a5740 1 -- 172.21.15.43:0/145059241 shutdown_connections 2024-09-19T05:14:22.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.418+0000 7fba238a5740 1 -- 172.21.15.43:0/145059241 wait complete. 2024-09-19T05:14:22.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.418+0000 7fba238a5740 20 client.27397 trim_cache size 0 max 16384 2024-09-19T05:14:22.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.435+0000 7fba238a5740 1 Processor -- start 2024-09-19T05:14:22.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.435+0000 7fba238a5740 1 -- start start 2024-09-19T05:14:22.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.436+0000 7fba238a5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555799934050 0x55579994b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:22.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.436+0000 7fba238a5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555799937ac0 0x55579995ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:22.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.436+0000 7fba238a5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557999383e0 0x55579995f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:22.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.436+0000 7fba238a5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c1ab720 con 0x555799934050 2024-09-19T05:14:22.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.436+0000 7fba238a5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c1ad260 con 0x555799937ac0 2024-09-19T05:14:22.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.436+0000 7fba238a5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c1abdf0 con 0x5557999383e0 2024-09-19T05:14:22.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.436+0000 7fba1a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555799934050 0x55579994b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:22.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.436+0000 7fba1a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555799934050 0x55579994b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44794/0 (socket says 172.21.15.43:44794) 2024-09-19T05:14:22.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.436+0000 7fba1a800640 1 -- 172.21.15.43:0/1669811682 learned_addr learned my addr 172.21.15.43:0/1669811682 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:22.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.436+0000 7fba11400640 1 --2- 172.21.15.43:0/1669811682 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555799937ac0 0x55579995ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:22.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.436+0000 7fba1a800640 1 --2- 172.21.15.43:0/1669811682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557999383e0 0x55579995f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:22.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.437+0000 7fba1a800640 1 -- 172.21.15.43:0/1669811682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557999383e0 msgr2=0x55579995f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:22.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.437+0000 7fba1a800640 1 --2- 172.21.15.43:0/1669811682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557999383e0 0x55579995f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:22.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.437+0000 7fba1a800640 1 -- 172.21.15.43:0/1669811682 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555799937ac0 msgr2=0x55579995ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:22.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.437+0000 7fba1a800640 1 --2- 172.21.15.43:0/1669811682 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555799937ac0 0x55579995ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:22.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.437+0000 7fba1a800640 1 -- 172.21.15.43:0/1669811682 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555799938210 con 0x555799934050 2024-09-19T05:14:22.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.437+0000 7fba1a800640 1 --2- 172.21.15.43:0/1669811682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555799934050 0x55579994b6a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fba14003b40 tx=0x7fba14004260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a34117047f9aa973 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:22.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.437+0000 7fba0aa00640 1 -- 172.21.15.43:0/1669811682 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba14004000 con 0x555799934050 2024-09-19T05:14:22.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.437+0000 7fba0aa00640 1 -- 172.21.15.43:0/1669811682 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba14004690 con 0x555799934050 2024-09-19T05:14:22.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.437+0000 7fba0aa00640 1 -- 172.21.15.43:0/1669811682 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba14005940 con 0x555799934050 2024-09-19T05:14:22.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.438+0000 7fba238a5740 1 -- 172.21.15.43:0/1669811682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555799934050 msgr2=0x55579994b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:22.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.438+0000 7fba238a5740 1 --2- 172.21.15.43:0/1669811682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555799934050 0x55579994b6a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fba14003b40 tx=0x7fba14004260 comp rx=0 tx=0).stop 2024-09-19T05:14:22.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.438+0000 7fba238a5740 1 -- 172.21.15.43:0/1669811682 shutdown_connections 2024-09-19T05:14:22.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.438+0000 7fba238a5740 1 --2- 172.21.15.43:0/1669811682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557999383e0 0x55579995f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:22.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.438+0000 7fba238a5740 1 --2- 172.21.15.43:0/1669811682 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555799937ac0 0x55579995ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:22.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.438+0000 7fba238a5740 1 --2- 172.21.15.43:0/1669811682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555799934050 0x55579994b6a0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:22.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.438+0000 7fba238a5740 1 -- 172.21.15.43:0/1669811682 >> 172.21.15.43:0/1669811682 conn(0x55579993f740 msgr2=0x555799939d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:22.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.438+0000 7fba238a5740 1 -- 172.21.15.43:0/1669811682 shutdown_connections 2024-09-19T05:14:22.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.438+0000 7fba238a5740 1 -- 172.21.15.43:0/1669811682 wait complete. 2024-09-19T05:14:22.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.438+0000 7fba238a5740 1 -- start start 2024-09-19T05:14:22.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.439+0000 7fba238a5740 1 Processor -- start 2024-09-19T05:14:22.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.439+0000 7fba238a5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555799934050 0x55579994f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:22.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.439+0000 7fba238a5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555799937ac0 0x55579994f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:22.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.439+0000 7fba238a5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557999383e0 0x5557999c0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:22.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.439+0000 7fba238a5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c1acb90 con 0x555799934050 2024-09-19T05:14:22.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.439+0000 7fba238a5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c1ac4c0 con 0x5557999383e0 2024-09-19T05:14:22.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.439+0000 7fba238a5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c1a9720 con 0x555799937ac0 2024-09-19T05:14:22.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.439+0000 7fba238a5740 10 client.0 _ll_register_callbacks cb 0x7fba1401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:22.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.439+0000 7fba1a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555799934050 0x55579994f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:22.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.439+0000 7fba1a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555799934050 0x55579994f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44798/0 (socket says 172.21.15.43:44798) 2024-09-19T05:14:22.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.439+0000 7fba1a800640 1 -- 172.21.15.43:0/2490589647 learned_addr learned my addr 172.21.15.43:0/2490589647 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:22.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.439+0000 7fba11400640 1 --2- 172.21.15.43:0/2490589647 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555799937ac0 0x55579994f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:22.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.440+0000 7fba1a800640 1 --2- 172.21.15.43:0/2490589647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557999383e0 0x5557999c0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:22.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.440+0000 7fba11400640 1 -- 172.21.15.43:0/2490589647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557999383e0 msgr2=0x5557999c0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:22.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.440+0000 7fba11400640 1 --2- 172.21.15.43:0/2490589647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557999383e0 0x5557999c0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:22.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.440+0000 7fba11400640 1 -- 172.21.15.43:0/2490589647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555799934050 msgr2=0x55579994f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:22.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.440+0000 7fba11400640 1 --2- 172.21.15.43:0/2490589647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555799934050 0x55579994f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:22.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.440+0000 7fba11400640 1 -- 172.21.15.43:0/2490589647 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55579994fdd0 con 0x555799937ac0 2024-09-19T05:14:22.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.440+0000 7fba1a800640 1 --2- 172.21.15.43:0/2490589647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555799934050 0x55579994f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:22.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.440+0000 7fba11400640 1 --2- 172.21.15.43:0/2490589647 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555799937ac0 0x55579994f540 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x555799959bb0 tx=0x7fba04005630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6605ffe88fdc354 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:22.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.440+0000 7fb9ffe00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:14:22.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.440+0000 7fb9ffe00640 1 -- 172.21.15.43:0/2490589647 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba04024070 con 0x555799937ac0 2024-09-19T05:14:22.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.441+0000 7fb9ffe00640 1 -- 172.21.15.43:0/2490589647 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7fba040138a0 con 0x555799937ac0 2024-09-19T05:14:22.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.441+0000 7fba238a5740 1 -- 172.21.15.43:0/2490589647 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba14004690 con 0x555799937ac0 2024-09-19T05:14:22.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.441+0000 7fb9ffe00640 1 -- 172.21.15.43:0/2490589647 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fba04013aa0 con 0x555799937ac0 2024-09-19T05:14:22.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.441+0000 7fba238a5740 10 client.27403 fetch_fsmap learned FSMap version 29 2024-09-19T05:14:22.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.441+0000 7fba238a5740 1 -- 172.21.15.43:0/2490589647 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5557999c2200 con 0x555799937ac0 2024-09-19T05:14:22.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.441+0000 7fb9ffe00640 1 -- 172.21.15.43:0/2490589647 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba14004690 con 0x555799937ac0 2024-09-19T05:14:22.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.441+0000 7fb9ffe00640 1 -- 172.21.15.43:0/2490589647 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fba04010430 con 0x555799937ac0 2024-09-19T05:14:22.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.442+0000 7fba238a5740 10 client.27403 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:14:22.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.442+0000 7fba238a5740 10 client.27403 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:14:22.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.442+0000 7fba238a5740 20 client.27403 populate_metadata read hostname 'smithi043' 2024-09-19T05:14:22.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.442+0000 7fba238a5740 1 --2- 172.21.15.43:0/2490589647 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x555799942ca0 0x5557999c49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:22.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.442+0000 7fba238a5740 4 client.27403 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T05:14:22.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.442+0000 7fba238a5740 1 -- 172.21.15.43:0/2490589647 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5557998a5d10 con 0x555799942ca0 2024-09-19T05:14:22.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.442+0000 7fba1a800640 1 --2- 172.21.15.43:0/2490589647 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x555799942ca0 0x5557999c49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:22.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.443+0000 7fba1a800640 1 --2- 172.21.15.43:0/2490589647 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x555799942ca0 0x5557999c49c0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d46936be3d70b8f0 server_cookie=c59afdd5eeb53d0b in_seq=0 out_seq=0 2024-09-19T05:14:22.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.443+0000 7fb9ffe00640 10 client.27403 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:14:22.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.444+0000 7fb9ffe00640 1 -- 172.21.15.43:0/2490589647 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5557998a5d10 con 0x555799942ca0 2024-09-19T05:14:22.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.444+0000 7fb9ffe00640 10 client.27403 handle_command_reply: tid=0 2024-09-19T05:14:22.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.444+0000 7fba238a5740 1 client.27403 shutdown 2024-09-19T05:14:22.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.444+0000 7fba238a5740 1 -- 172.21.15.43:0/2490589647 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555799937ac0 msgr2=0x55579994f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:22.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.444+0000 7fba238a5740 1 --2- 172.21.15.43:0/2490589647 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555799937ac0 0x55579994f540 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x555799959bb0 tx=0x7fba04005630 comp rx=0 tx=0).stop 2024-09-19T05:14:22.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.444+0000 7fba238a5740 1 -- 172.21.15.43:0/2490589647 shutdown_connections 2024-09-19T05:14:22.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.444+0000 7fba238a5740 1 --2- 172.21.15.43:0/2490589647 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x555799942ca0 0x5557999c49c0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:22.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.444+0000 7fba238a5740 1 --2- 172.21.15.43:0/2490589647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557999383e0 0x5557999c0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:22.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.444+0000 7fba238a5740 1 --2- 172.21.15.43:0/2490589647 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555799937ac0 0x55579994f540 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:22.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.444+0000 7fba238a5740 1 --2- 172.21.15.43:0/2490589647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555799934050 0x55579994f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:22.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.444+0000 7fba238a5740 1 -- 172.21.15.43:0/2490589647 >> 172.21.15.43:0/2490589647 conn(0x55579993f740 msgr2=0x555799939510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:22.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.444+0000 7fb9ffe00640 0 client.27403 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:14:22.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.444+0000 7fba238a5740 1 -- 172.21.15.43:0/2490589647 shutdown_connections 2024-09-19T05:14:22.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.444+0000 7fba238a5740 1 -- 172.21.15.43:0/2490589647 wait complete. 2024-09-19T05:14:22.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.444+0000 7fba238a5740 20 client.27403 trim_cache size 0 max 16384 2024-09-19T05:14:22.451 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:14:22.451 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-09-19T05:14:22.451 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "3779fc1c-1109-43b1-8cf3-1eba62b621a2", 2024-09-19T05:14:22.451 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-09-19T05:14:22.451 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:14:22.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.445+0000 7fba20e00640 1 -- 172.21.15.43:0/2230117876 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fba000a3430 msgr2=0x7fba000a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:22.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.445+0000 7fba20e00640 1 --2- 172.21.15.43:0/2230117876 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fba000a3430 0x7fba000a58f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x555799934eb0 tx=0x7fba0c0f3a70 comp rx=0 tx=0).stop 2024-09-19T05:14:22.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.445+0000 7fba20e00640 1 -- 172.21.15.43:0/2230117876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba1c101de0 msgr2=0x7fba1c198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:22.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.445+0000 7fba20e00640 1 --2- 172.21.15.43:0/2230117876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba1c101de0 0x7fba1c198430 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fba0c005940 tx=0x7fba0c0040c0 comp rx=0 tx=0).stop 2024-09-19T05:14:22.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.445+0000 7fba20e00640 1 -- 172.21.15.43:0/2230117876 shutdown_connections 2024-09-19T05:14:22.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.445+0000 7fba20e00640 1 --2- 172.21.15.43:0/2230117876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba1c106f10 0x7fba1c19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:22.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.445+0000 7fba20e00640 1 --2- 172.21.15.43:0/2230117876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba1c102790 0x7fba1c198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:22.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.445+0000 7fba20e00640 1 --2- 172.21.15.43:0/2230117876 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fba000a3430 0x7fba000a58f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:22.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.445+0000 7fba20e00640 1 --2- 172.21.15.43:0/2230117876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba1c101de0 0x7fba1c198430 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:22.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.445+0000 7fba20e00640 1 -- 172.21.15.43:0/2230117876 >> 172.21.15.43:0/2230117876 conn(0x7fba1c0fd8c0 msgr2=0x7fba1c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:22.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.445+0000 7fba20e00640 1 -- 172.21.15.43:0/2230117876 shutdown_connections 2024-09-19T05:14:22.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:22.445+0000 7fba20e00640 1 -- 172.21.15.43:0/2230117876 wait complete. 2024-09-19T05:14:22.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:22 smithi096 ceph-mon[35085]: pgmap v302: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T05:14:23.167 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:23 smithi043 ceph-mon[30800]: scrub queued for path: / 2024-09-19T05:14:23.167 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:23 smithi043 ceph-mon[30800]: scrub summary: idle+waiting paths [/] 2024-09-19T05:14:23.167 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:23 smithi043 ceph-mon[30800]: scrub summary: active paths [/] 2024-09-19T05:14:23.167 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:23 smithi043 ceph-mon[30800]: scrub summary: idle+waiting paths [/] 2024-09-19T05:14:23.167 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:23 smithi043 ceph-mon[30800]: scrub summary: idle 2024-09-19T05:14:23.204 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:14:23.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:23 smithi151 ceph-mon[35425]: scrub queued for path: / 2024-09-19T05:14:23.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:23 smithi151 ceph-mon[35425]: scrub summary: idle+waiting paths [/] 2024-09-19T05:14:23.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:23 smithi151 ceph-mon[35425]: scrub summary: active paths [/] 2024-09-19T05:14:23.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:23 smithi151 ceph-mon[35425]: scrub summary: idle+waiting paths [/] 2024-09-19T05:14:23.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:23 smithi151 ceph-mon[35425]: scrub summary: idle 2024-09-19T05:14:23.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:23 smithi096 ceph-mon[35085]: scrub queued for path: / 2024-09-19T05:14:23.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:23 smithi096 ceph-mon[35085]: scrub summary: idle+waiting paths [/] 2024-09-19T05:14:23.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:23 smithi096 ceph-mon[35085]: scrub summary: active paths [/] 2024-09-19T05:14:23.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:23 smithi096 ceph-mon[35085]: scrub summary: idle+waiting paths [/] 2024-09-19T05:14:23.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:23 smithi096 ceph-mon[35085]: scrub summary: idle 2024-09-19T05:14:23.586 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:14:24.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.112+0000 7f0a41e00640 1 -- 172.21.15.43:0/1452474703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3c112a50 msgr2=0x7f0a3c114e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:24.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.112+0000 7f0a41e00640 1 --2- 172.21.15.43:0/1452474703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3c112a50 0x7f0a3c114e40 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f0a34009ca0 tx=0x7f0a3402a8c0 comp rx=0 tx=0).stop 2024-09-19T05:14:24.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.112+0000 7f0a41e00640 1 -- 172.21.15.43:0/1452474703 shutdown_connections 2024-09-19T05:14:24.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.112+0000 7f0a41e00640 1 --2- 172.21.15.43:0/1452474703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3c112a50 0x7f0a3c114e40 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:24.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.112+0000 7f0a41e00640 1 --2- 172.21.15.43:0/1452474703 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a3c077040 0x7f0a3c112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:24.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.112+0000 7f0a41e00640 1 --2- 172.21.15.43:0/1452474703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a3c076720 0x7f0a3c076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:24.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.112+0000 7f0a41e00640 1 -- 172.21.15.43:0/1452474703 >> 172.21.15.43:0/1452474703 conn(0x7f0a3c0ffa50 msgr2=0x7f0a3c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:24.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.112+0000 7f0a41e00640 1 -- 172.21.15.43:0/1452474703 shutdown_connections 2024-09-19T05:14:24.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.112+0000 7f0a41e00640 1 -- 172.21.15.43:0/1452474703 wait complete. 2024-09-19T05:14:24.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.113+0000 7f0a41e00640 1 Processor -- start 2024-09-19T05:14:24.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.113+0000 7f0a41e00640 1 -- start start 2024-09-19T05:14:24.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.113+0000 7f0a41e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a3c076720 0x7f0a3c19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:24.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.113+0000 7f0a41e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3c077040 0x7f0a3c19f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:24.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.113+0000 7f0a41e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a3c112a50 0x7f0a3c1a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:24.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.113+0000 7f0a41e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c118600 con 0x7f0a3c077040 2024-09-19T05:14:24.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.113+0000 7f0a41e00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c118480 con 0x7f0a3c076720 2024-09-19T05:14:24.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.113+0000 7f0a41e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c118780 con 0x7f0a3c112a50 2024-09-19T05:14:24.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.114+0000 7f0a32000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3c077040 0x7f0a3c19f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:24.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.114+0000 7f0a3aa00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a3c076720 0x7f0a3c19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:24.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.114+0000 7f0a32000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3c077040 0x7f0a3c19f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44824/0 (socket says 172.21.15.43:44824) 2024-09-19T05:14:24.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.114+0000 7f0a32000640 1 -- 172.21.15.43:0/921784930 learned_addr learned my addr 172.21.15.43:0/921784930 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:24.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.114+0000 7f0a3b400640 1 --2- 172.21.15.43:0/921784930 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a3c112a50 0x7f0a3c1a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:24.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.114+0000 7f0a3aa00640 1 -- 172.21.15.43:0/921784930 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a3c112a50 msgr2=0x7f0a3c1a3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:24.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.114+0000 7f0a3aa00640 1 --2- 172.21.15.43:0/921784930 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a3c112a50 0x7f0a3c1a3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:24.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.114+0000 7f0a3aa00640 1 -- 172.21.15.43:0/921784930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3c077040 msgr2=0x7f0a3c19f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:24.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.114+0000 7f0a3aa00640 1 --2- 172.21.15.43:0/921784930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3c077040 0x7f0a3c19f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:24.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.114+0000 7f0a3aa00640 1 -- 172.21.15.43:0/921784930 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a3c1a3b90 con 0x7f0a3c076720 2024-09-19T05:14:24.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.114+0000 7f0a32000640 1 --2- 172.21.15.43:0/921784930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3c077040 0x7f0a3c19f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:14:24.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.114+0000 7f0a3b400640 1 --2- 172.21.15.43:0/921784930 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a3c112a50 0x7f0a3c1a3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:24.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.115+0000 7f0a3aa00640 1 --2- 172.21.15.43:0/921784930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a3c076720 0x7f0a3c19eb60 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c00e9f0 tx=0x7f0a2c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:24.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.115+0000 7f0a40a00640 1 -- 172.21.15.43:0/921784930 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c00ccc0 con 0x7f0a3c076720 2024-09-19T05:14:24.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.115+0000 7f0a40a00640 1 -- 172.21.15.43:0/921784930 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a2c010910 con 0x7f0a3c076720 2024-09-19T05:14:24.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.115+0000 7f0a40a00640 1 -- 172.21.15.43:0/921784930 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c018960 con 0x7f0a3c076720 2024-09-19T05:14:24.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.115+0000 7f0a41e00640 1 -- 172.21.15.43:0/921784930 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a3c1a3e80 con 0x7f0a3c076720 2024-09-19T05:14:24.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.115+0000 7f0a41e00640 1 -- 172.21.15.43:0/921784930 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a3c1ab710 con 0x7f0a3c076720 2024-09-19T05:14:24.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.118+0000 7f0a40a00640 1 -- 172.21.15.43:0/921784930 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0a2c010ac0 con 0x7f0a3c076720 2024-09-19T05:14:24.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.118+0000 7f0a44a60740 1 Processor -- start 2024-09-19T05:14:24.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.118+0000 7f0a44a60740 1 -- start start 2024-09-19T05:14:24.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.118+0000 7f0a44a60740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5618be42bb90 0x5618be42bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:24.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.119+0000 7f0a44a60740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5618be42c4b0 0x5618be436830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:24.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.119+0000 7f0a44a60740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5618be436d70 0x5618be439160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:24.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.119+0000 7f0a44a60740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5618be3923e0 con 0x5618be436d70 2024-09-19T05:14:24.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.119+0000 7f0a44a60740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5618be339d80 con 0x5618be42c4b0 2024-09-19T05:14:24.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.119+0000 7f0a44a60740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5618be38e3a0 con 0x5618be42bb90 2024-09-19T05:14:24.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.119+0000 7f0a32000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5618be436d70 0x5618be439160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:24.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.119+0000 7f0a3aa00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5618be42c4b0 0x5618be436830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:24.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.119+0000 7f0a3aa00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5618be42c4b0 0x5618be436830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:48966/0 (socket says 172.21.15.43:48966) 2024-09-19T05:14:24.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.119+0000 7f0a3b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5618be42bb90 0x5618be42bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:24.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.119+0000 7f0a3aa00640 1 -- 172.21.15.43:0/2355839377 learned_addr learned my addr 172.21.15.43:0/2355839377 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:24.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.119+0000 7f0a32000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5618be436d70 0x5618be439160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44836/0 (socket says 172.21.15.43:44836) 2024-09-19T05:14:24.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.119+0000 7f0a3aa00640 1 -- 172.21.15.43:0/2355839377 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5618be42bb90 msgr2=0x5618be42bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:24.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.119+0000 7f0a3aa00640 1 --2- 172.21.15.43:0/2355839377 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5618be42bb90 0x5618be42bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:24.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.119+0000 7f0a3aa00640 1 -- 172.21.15.43:0/2355839377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5618be436d70 msgr2=0x5618be439160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:24.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.119+0000 7f0a3aa00640 1 --2- 172.21.15.43:0/2355839377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5618be436d70 0x5618be439160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:24.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.119+0000 7f0a3aa00640 1 -- 172.21.15.43:0/2355839377 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618be358ae0 con 0x5618be42c4b0 2024-09-19T05:14:24.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.119+0000 7f0a32000640 1 --2- 172.21.15.43:0/2355839377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5618be436d70 0x5618be439160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:24.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.120+0000 7f0a3aa00640 1 --2- 172.21.15.43:0/2355839377 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5618be42c4b0 0x5618be436830 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c0cd820 tx=0x7f0a2c0d8860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf6bcc14fe9a8e9c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:24.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.120+0000 7f0a40a00640 1 --2- 172.21.15.43:0/921784930 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f0a200a33b0 0x7f0a200a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:24.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.120+0000 7f0a40a00640 1 -- 172.21.15.43:0/921784930 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f0a2c0d0170 con 0x7f0a3c076720 2024-09-19T05:14:24.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.120+0000 7f0a3b400640 1 --2- 172.21.15.43:0/921784930 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f0a200a33b0 0x7f0a200a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:24.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.120+0000 7f0a2be00640 1 -- 172.21.15.43:0/2355839377 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c0e3070 con 0x5618be42c4b0 2024-09-19T05:14:24.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.120+0000 7f0a2be00640 1 -- 172.21.15.43:0/2355839377 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a2c09cb40 con 0x5618be42c4b0 2024-09-19T05:14:24.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.120+0000 7f0a2be00640 1 -- 172.21.15.43:0/2355839377 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c09bda0 con 0x5618be42c4b0 2024-09-19T05:14:24.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.121+0000 7f0a44a60740 1 -- 172.21.15.43:0/2355839377 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5618be42c4b0 msgr2=0x5618be436830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:24.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.121+0000 7f0a44a60740 1 --2- 172.21.15.43:0/2355839377 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5618be42c4b0 0x5618be436830 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c0cd820 tx=0x7f0a2c0d8860 comp rx=0 tx=0).stop 2024-09-19T05:14:24.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.121+0000 7f0a3b400640 1 --2- 172.21.15.43:0/921784930 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f0a200a33b0 0x7f0a200a5870 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0a34005e30 tx=0x7f0a34005d80 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:24.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.121+0000 7f0a44a60740 1 -- 172.21.15.43:0/2355839377 shutdown_connections 2024-09-19T05:14:24.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.121+0000 7f0a44a60740 1 --2- 172.21.15.43:0/2355839377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5618be436d70 0x5618be439160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:24.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.121+0000 7f0a44a60740 1 --2- 172.21.15.43:0/2355839377 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5618be42c4b0 0x5618be436830 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:24.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.121+0000 7f0a44a60740 1 --2- 172.21.15.43:0/2355839377 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5618be42bb90 0x5618be42bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:24.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.121+0000 7f0a44a60740 1 -- 172.21.15.43:0/2355839377 >> 172.21.15.43:0/2355839377 conn(0x5618be428120 msgr2=0x5618be428500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:24.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.121+0000 7f0a44a60740 1 -- 172.21.15.43:0/2355839377 shutdown_connections 2024-09-19T05:14:24.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.121+0000 7f0a44a60740 1 -- 172.21.15.43:0/2355839377 wait complete. 2024-09-19T05:14:24.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.122+0000 7f0a44a60740 1 -- start start 2024-09-19T05:14:24.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.122+0000 7f0a44a60740 1 Processor -- start 2024-09-19T05:14:24.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.122+0000 7f0a44a60740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5618be42bb90 0x5618be433c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:24.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.122+0000 7f0a44a60740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5618be42c4b0 0x5618be4341b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:24.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.122+0000 7f0a44a60740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5618be436d70 0x5618be4346f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:24.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.122+0000 7f0a44a60740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5618be4399e0 con 0x5618be42c4b0 2024-09-19T05:14:24.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.122+0000 7f0a44a60740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5618be34d0f0 con 0x5618be42bb90 2024-09-19T05:14:24.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.122+0000 7f0a44a60740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5618be439b60 con 0x5618be436d70 2024-09-19T05:14:24.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.123+0000 7f0a44a60740 10 client.0 _ll_register_callbacks cb 0x5618be33e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:24.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.123+0000 7f0a3aa00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5618be42bb90 0x5618be433c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:24.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.123+0000 7f0a32000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5618be42c4b0 0x5618be4341b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:24.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.123+0000 7f0a3aa00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5618be42bb90 0x5618be433c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:48968/0 (socket says 172.21.15.43:48968) 2024-09-19T05:14:24.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.123+0000 7f0a3aa00640 1 -- 172.21.15.43:0/2118751838 learned_addr learned my addr 172.21.15.43:0/2118751838 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:24.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.123+0000 7f0a3b400640 1 --2- 172.21.15.43:0/2118751838 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5618be436d70 0x5618be4346f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:24.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.123+0000 7f0a3aa00640 1 -- 172.21.15.43:0/2118751838 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5618be436d70 msgr2=0x5618be4346f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:24.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.123+0000 7f0a3aa00640 1 --2- 172.21.15.43:0/2118751838 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5618be436d70 0x5618be4346f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:24.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.123+0000 7f0a3aa00640 1 -- 172.21.15.43:0/2118751838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5618be42c4b0 msgr2=0x5618be4341b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:24.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.123+0000 7f0a3aa00640 1 --2- 172.21.15.43:0/2118751838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5618be42c4b0 0x5618be4341b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:24.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.123+0000 7f0a3aa00640 1 -- 172.21.15.43:0/2118751838 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618be1033c0 con 0x5618be42bb90 2024-09-19T05:14:24.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.124+0000 7f0a3aa00640 1 --2- 172.21.15.43:0/2118751838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5618be42bb90 0x5618be433c70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c0cd960 tx=0x7f0a2c0d3810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86849265b93e975d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:24.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.124+0000 7f0a28c00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:14:24.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.124+0000 7f0a28c00640 1 -- 172.21.15.43:0/2118751838 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c09bda0 con 0x5618be42bb90 2024-09-19T05:14:24.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.124+0000 7f0a28c00640 1 -- 172.21.15.43:0/2118751838 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f0a2c098ab0 con 0x5618be42bb90 2024-09-19T05:14:24.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.124+0000 7f0a44a60740 1 -- 172.21.15.43:0/2118751838 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a2c09cb40 con 0x5618be42bb90 2024-09-19T05:14:24.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.124+0000 7f0a28c00640 1 -- 172.21.15.43:0/2118751838 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0a2c09c210 con 0x5618be42bb90 2024-09-19T05:14:24.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.124+0000 7f0a44a60740 10 client.27518 fetch_fsmap learned FSMap version 29 2024-09-19T05:14:24.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.124+0000 7f0a44a60740 1 -- 172.21.15.43:0/2118751838 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5618be4399e0 con 0x5618be42bb90 2024-09-19T05:14:24.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.124+0000 7f0a28c00640 1 -- 172.21.15.43:0/2118751838 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a2c09cb40 con 0x5618be42bb90 2024-09-19T05:14:24.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.125+0000 7f0a28c00640 1 -- 172.21.15.43:0/2118751838 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0a2c0d3350 con 0x5618be42bb90 2024-09-19T05:14:24.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.125+0000 7f0a44a60740 10 client.27518 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:14:24.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.125+0000 7f0a44a60740 10 client.27518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:14:24.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.125+0000 7f0a44a60740 20 client.27518 populate_metadata read hostname 'smithi043' 2024-09-19T05:14:24.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.125+0000 7f0a44a60740 1 --2- 172.21.15.43:0/2118751838 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5618be446850 0x5618be448c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:24.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.125+0000 7f0a44a60740 4 client.27518 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:14:24.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.125+0000 7f0a44a60740 1 -- 172.21.15.43:0/2118751838 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618be39f950 con 0x5618be446850 2024-09-19T05:14:24.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.125+0000 7f0a32000640 1 --2- 172.21.15.43:0/2118751838 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5618be446850 0x5618be448c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:24.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.126+0000 7f0a32000640 1 --2- 172.21.15.43:0/2118751838 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5618be446850 0x5618be448c40 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f590a2907f02bd4b server_cookie=8fcd1f720864cc00 in_seq=0 out_seq=0 2024-09-19T05:14:24.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.126+0000 7f0a28c00640 10 client.27518 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:14:24.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.127+0000 7f0a28c00640 1 -- 172.21.15.43:0/2118751838 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5618be39f950 con 0x5618be446850 2024-09-19T05:14:24.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.127+0000 7f0a28c00640 10 client.27518 handle_command_reply: tid=0 2024-09-19T05:14:24.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.127+0000 7f0a44a60740 1 client.27518 shutdown 2024-09-19T05:14:24.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.127+0000 7f0a44a60740 1 -- 172.21.15.43:0/2118751838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5618be42bb90 msgr2=0x5618be433c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:24.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.127+0000 7f0a44a60740 1 --2- 172.21.15.43:0/2118751838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5618be42bb90 0x5618be433c70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c0cd960 tx=0x7f0a2c0d3810 comp rx=0 tx=0).stop 2024-09-19T05:14:24.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.127+0000 7f0a44a60740 1 -- 172.21.15.43:0/2118751838 shutdown_connections 2024-09-19T05:14:24.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.127+0000 7f0a44a60740 1 --2- 172.21.15.43:0/2118751838 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5618be446850 0x5618be448c40 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:24.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.127+0000 7f0a44a60740 1 --2- 172.21.15.43:0/2118751838 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5618be436d70 0x5618be4346f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:24.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.127+0000 7f0a44a60740 1 --2- 172.21.15.43:0/2118751838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5618be42c4b0 0x5618be4341b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:24.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.127+0000 7f0a44a60740 1 --2- 172.21.15.43:0/2118751838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5618be42bb90 0x5618be433c70 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:24.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.127+0000 7f0a28c00640 0 client.27518 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:14:24.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.127+0000 7f0a44a60740 1 -- 172.21.15.43:0/2118751838 >> 172.21.15.43:0/2118751838 conn(0x5618be428120 msgr2=0x5618be42de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:24.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.127+0000 7f0a44a60740 1 -- 172.21.15.43:0/2118751838 shutdown_connections 2024-09-19T05:14:24.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.127+0000 7f0a44a60740 1 -- 172.21.15.43:0/2118751838 wait complete. 2024-09-19T05:14:24.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.127+0000 7f0a44a60740 20 client.27518 trim_cache size 0 max 16384 2024-09-19T05:14:24.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.145+0000 7f0a44a60740 1 Processor -- start 2024-09-19T05:14:24.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.145+0000 7f0a44a60740 1 -- start start 2024-09-19T05:14:24.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.146+0000 7f0a44a60740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5618be42bb90 0x5618be432540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:24.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.146+0000 7f0a44a60740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5618be42c4b0 0x5618be42dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:24.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.146+0000 7f0a44a60740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5618be436d70 0x5618be44e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:24.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.146+0000 7f0a44a60740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c1b40c0 con 0x5618be436d70 2024-09-19T05:14:24.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.146+0000 7f0a44a60740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c1b5c00 con 0x5618be42c4b0 2024-09-19T05:14:24.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.146+0000 7f0a44a60740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c1b4790 con 0x5618be42bb90 2024-09-19T05:14:24.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.146+0000 7f0a3b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5618be436d70 0x5618be44e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:24.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.146+0000 7f0a3b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5618be436d70 0x5618be44e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44864/0 (socket says 172.21.15.43:44864) 2024-09-19T05:14:24.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.146+0000 7f0a3b400640 1 -- 172.21.15.43:0/1438120229 learned_addr learned my addr 172.21.15.43:0/1438120229 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:24.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.146+0000 7f0a32000640 1 --2- 172.21.15.43:0/1438120229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5618be42c4b0 0x5618be42dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:24.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.146+0000 7f0a3aa00640 1 --2- 172.21.15.43:0/1438120229 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5618be42bb90 0x5618be432540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:24.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.147+0000 7f0a3b400640 1 -- 172.21.15.43:0/1438120229 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5618be42bb90 msgr2=0x5618be432540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:24.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.147+0000 7f0a3b400640 1 --2- 172.21.15.43:0/1438120229 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5618be42bb90 0x5618be432540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:24.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.147+0000 7f0a3b400640 1 -- 172.21.15.43:0/1438120229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5618be42c4b0 msgr2=0x5618be42dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:24.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.147+0000 7f0a3b400640 1 --2- 172.21.15.43:0/1438120229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5618be42c4b0 0x5618be42dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:24.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.147+0000 7f0a3b400640 1 -- 172.21.15.43:0/1438120229 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618be42b910 con 0x5618be436d70 2024-09-19T05:14:24.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.147+0000 7f0a3b400640 1 --2- 172.21.15.43:0/1438120229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5618be436d70 0x5618be44e270 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f0a34035000 tx=0x7f0a34042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e41655f45c7470a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:24.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.147+0000 7f0a2b400640 1 -- 172.21.15.43:0/1438120229 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a34031510 con 0x5618be436d70 2024-09-19T05:14:24.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.147+0000 7f0a2b400640 1 -- 172.21.15.43:0/1438120229 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a34036040 con 0x5618be436d70 2024-09-19T05:14:24.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.147+0000 7f0a2b400640 1 -- 172.21.15.43:0/1438120229 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a34003f90 con 0x5618be436d70 2024-09-19T05:14:24.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.148+0000 7f0a44a60740 1 -- 172.21.15.43:0/1438120229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5618be436d70 msgr2=0x5618be44e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:24.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.148+0000 7f0a44a60740 1 --2- 172.21.15.43:0/1438120229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5618be436d70 0x5618be44e270 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f0a34035000 tx=0x7f0a34042000 comp rx=0 tx=0).stop 2024-09-19T05:14:24.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.148+0000 7f0a44a60740 1 -- 172.21.15.43:0/1438120229 shutdown_connections 2024-09-19T05:14:24.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.148+0000 7f0a44a60740 1 --2- 172.21.15.43:0/1438120229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5618be436d70 0x5618be44e270 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:24.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.148+0000 7f0a44a60740 1 --2- 172.21.15.43:0/1438120229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5618be42c4b0 0x5618be42dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:24.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.148+0000 7f0a44a60740 1 --2- 172.21.15.43:0/1438120229 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5618be42bb90 0x5618be432540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:24.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.148+0000 7f0a44a60740 1 -- 172.21.15.43:0/1438120229 >> 172.21.15.43:0/1438120229 conn(0x5618be428120 msgr2=0x5618be42d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:24.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.148+0000 7f0a44a60740 1 -- 172.21.15.43:0/1438120229 shutdown_connections 2024-09-19T05:14:24.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.148+0000 7f0a44a60740 1 -- 172.21.15.43:0/1438120229 wait complete. 2024-09-19T05:14:24.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.149+0000 7f0a44a60740 1 -- start start 2024-09-19T05:14:24.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.149+0000 7f0a44a60740 1 Processor -- start 2024-09-19T05:14:24.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.149+0000 7f0a44a60740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5618be42bb90 0x5618be43ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:24.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.149+0000 7f0a44a60740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5618be42c4b0 0x5618be43b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:24.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.149+0000 7f0a44a60740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5618be436d70 0x5618be43b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:24.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.149+0000 7f0a44a60740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c1b5530 con 0x5618be436d70 2024-09-19T05:14:24.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.149+0000 7f0a44a60740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c1b4e60 con 0x5618be42bb90 2024-09-19T05:14:24.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.149+0000 7f0a44a60740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c1b20c0 con 0x5618be42c4b0 2024-09-19T05:14:24.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.149+0000 7f0a44a60740 10 client.0 _ll_register_callbacks cb 0x7f0a24005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:24.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.150+0000 7f0a32000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5618be42c4b0 0x5618be43b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:24.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.150+0000 7f0a3aa00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5618be42bb90 0x5618be43ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:24.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.150+0000 7f0a32000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5618be42c4b0 0x5618be43b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:51398/0 (socket says 172.21.15.43:51398) 2024-09-19T05:14:24.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.150+0000 7f0a32000640 1 -- 172.21.15.43:0/356204787 learned_addr learned my addr 172.21.15.43:0/356204787 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:24.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.150+0000 7f0a3b400640 1 --2- 172.21.15.43:0/356204787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5618be436d70 0x5618be43b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:24.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.150+0000 7f0a32000640 1 -- 172.21.15.43:0/356204787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5618be42bb90 msgr2=0x5618be43ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:24.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.150+0000 7f0a32000640 1 --2- 172.21.15.43:0/356204787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5618be42bb90 0x5618be43ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:24.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.150+0000 7f0a32000640 1 -- 172.21.15.43:0/356204787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5618be436d70 msgr2=0x5618be43b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:24.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.150+0000 7f0a32000640 1 --2- 172.21.15.43:0/356204787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5618be436d70 0x5618be43b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:24.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.150+0000 7f0a32000640 1 -- 172.21.15.43:0/356204787 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618be4b02f0 con 0x5618be42c4b0 2024-09-19T05:14:24.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.150+0000 7f0a3b400640 1 --2- 172.21.15.43:0/356204787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5618be436d70 0x5618be43b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:24.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.150+0000 7f0a32000640 1 --2- 172.21.15.43:0/356204787 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5618be42c4b0 0x5618be43b160 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0a24004700 tx=0x7f0a24005610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb7a50e1c8c2db6b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:24.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.150+0000 7f0a28c00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:14:24.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.151+0000 7f0a28c00640 1 -- 172.21.15.43:0/356204787 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2401e790 con 0x5618be42c4b0 2024-09-19T05:14:24.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.151+0000 7f0a44a60740 1 -- 172.21.15.43:0/356204787 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a34036040 con 0x5618be42c4b0 2024-09-19T05:14:24.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.151+0000 7f0a28c00640 1 -- 172.21.15.43:0/356204787 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f0a2401e930 con 0x5618be42c4b0 2024-09-19T05:14:24.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.151+0000 7f0a28c00640 1 -- 172.21.15.43:0/356204787 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0a24004e50 con 0x5618be42c4b0 2024-09-19T05:14:24.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.151+0000 7f0a44a60740 10 client.27421 fetch_fsmap learned FSMap version 29 2024-09-19T05:14:24.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.151+0000 7f0a44a60740 1 -- 172.21.15.43:0/356204787 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5618be4b1620 con 0x5618be42c4b0 2024-09-19T05:14:24.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.151+0000 7f0a28c00640 1 -- 172.21.15.43:0/356204787 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a34036040 con 0x5618be42c4b0 2024-09-19T05:14:24.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.151+0000 7f0a28c00640 1 -- 172.21.15.43:0/356204787 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0a24026400 con 0x5618be42c4b0 2024-09-19T05:14:24.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.151+0000 7f0a44a60740 10 client.27421 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:14:24.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.151+0000 7f0a44a60740 10 client.27421 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:14:24.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.151+0000 7f0a44a60740 20 client.27421 populate_metadata read hostname 'smithi043' 2024-09-19T05:14:24.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.151+0000 7f0a44a60740 1 --2- 172.21.15.43:0/356204787 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5618be4b1cf0 0x5618be4b4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:24.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.151+0000 7f0a44a60740 4 client.27421 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:14:24.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.151+0000 7f0a44a60740 1 -- 172.21.15.43:0/356204787 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5618be399e60 con 0x5618be4b1cf0 2024-09-19T05:14:24.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.152+0000 7f0a3aa00640 1 --2- 172.21.15.43:0/356204787 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5618be4b1cf0 0x5618be4b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:24.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.152+0000 7f0a3aa00640 1 --2- 172.21.15.43:0/356204787 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5618be4b1cf0 0x5618be4b4110 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b741c585a1193241 server_cookie=6fa94da169abcff7 in_seq=0 out_seq=0 2024-09-19T05:14:24.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.152+0000 7f0a28c00640 10 client.27421 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:14:24.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.152+0000 7f0a28c00640 1 -- 172.21.15.43:0/356204787 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5618be399e60 con 0x5618be4b1cf0 2024-09-19T05:14:24.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.152+0000 7f0a28c00640 10 client.27421 handle_command_reply: tid=0 2024-09-19T05:14:24.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.152+0000 7f0a44a60740 1 client.27421 shutdown 2024-09-19T05:14:24.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.153+0000 7f0a44a60740 1 -- 172.21.15.43:0/356204787 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5618be42c4b0 msgr2=0x5618be43b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:24.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.153+0000 7f0a44a60740 1 --2- 172.21.15.43:0/356204787 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5618be42c4b0 0x5618be43b160 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0a24004700 tx=0x7f0a24005610 comp rx=0 tx=0).stop 2024-09-19T05:14:24.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.153+0000 7f0a44a60740 1 -- 172.21.15.43:0/356204787 shutdown_connections 2024-09-19T05:14:24.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.153+0000 7f0a44a60740 1 --2- 172.21.15.43:0/356204787 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5618be4b1cf0 0x5618be4b4110 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:24.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.153+0000 7f0a44a60740 1 --2- 172.21.15.43:0/356204787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5618be436d70 0x5618be43b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:24.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.153+0000 7f0a44a60740 1 --2- 172.21.15.43:0/356204787 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5618be42c4b0 0x5618be43b160 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:24.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.153+0000 7f0a44a60740 1 --2- 172.21.15.43:0/356204787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5618be42bb90 0x5618be43ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:24.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.153+0000 7f0a28c00640 0 client.27421 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:14:24.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.153+0000 7f0a44a60740 1 -- 172.21.15.43:0/356204787 >> 172.21.15.43:0/356204787 conn(0x5618be428120 msgr2=0x5618be434450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:24.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.153+0000 7f0a44a60740 1 -- 172.21.15.43:0/356204787 shutdown_connections 2024-09-19T05:14:24.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.153+0000 7f0a44a60740 1 -- 172.21.15.43:0/356204787 wait complete. 2024-09-19T05:14:24.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.153+0000 7f0a44a60740 20 client.27421 trim_cache size 0 max 16384 2024-09-19T05:14:24.158 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:14:24.158 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-09-19T05:14:24.158 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-09-19T05:14:24.158 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:14:24.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.154+0000 7f0a41e00640 1 -- 172.21.15.43:0/921784930 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f0a200a33b0 msgr2=0x7f0a200a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:24.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.154+0000 7f0a41e00640 1 --2- 172.21.15.43:0/921784930 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f0a200a33b0 0x7f0a200a5870 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0a34005e30 tx=0x7f0a34005d80 comp rx=0 tx=0).stop 2024-09-19T05:14:24.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.154+0000 7f0a41e00640 1 -- 172.21.15.43:0/921784930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a3c076720 msgr2=0x7f0a3c19eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:24.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.154+0000 7f0a41e00640 1 --2- 172.21.15.43:0/921784930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a3c076720 0x7f0a3c19eb60 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c00e9f0 tx=0x7f0a2c00eeb0 comp rx=0 tx=0).stop 2024-09-19T05:14:24.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.154+0000 7f0a41e00640 1 -- 172.21.15.43:0/921784930 shutdown_connections 2024-09-19T05:14:24.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.154+0000 7f0a41e00640 1 --2- 172.21.15.43:0/921784930 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a3c112a50 0x7f0a3c1a3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:24.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.154+0000 7f0a41e00640 1 --2- 172.21.15.43:0/921784930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3c077040 0x7f0a3c19f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:24.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.154+0000 7f0a41e00640 1 --2- 172.21.15.43:0/921784930 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f0a200a33b0 0x7f0a200a5870 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:24.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.154+0000 7f0a41e00640 1 --2- 172.21.15.43:0/921784930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a3c076720 0x7f0a3c19eb60 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:24.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.154+0000 7f0a41e00640 1 -- 172.21.15.43:0/921784930 >> 172.21.15.43:0/921784930 conn(0x7f0a3c0ffa50 msgr2=0x7f0a3c101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:24.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.154+0000 7f0a41e00640 1 -- 172.21.15.43:0/921784930 shutdown_connections 2024-09-19T05:14:24.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:24.154+0000 7f0a41e00640 1 -- 172.21.15.43:0/921784930 wait complete. 2024-09-19T05:14:24.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:24 smithi043 ceph-mon[30800]: pgmap v303: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T05:14:24.374 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:24 smithi096 ceph-mon[35085]: pgmap v303: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T05:14:24.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:24 smithi151 ceph-mon[35425]: pgmap v303: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T05:14:24.981 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T05:14:24.981 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:14:25.359 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:14:25.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:25.858+0000 7f62ba600640 1 -- 172.21.15.43:0/3417975113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62b41047e0 msgr2=0x7f62b4108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:25.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:25.858+0000 7f62ba600640 1 --2- 172.21.15.43:0/3417975113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62b41047e0 0x7f62b4108830 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f62a0009900 tx=0x7f62a002a3f0 comp rx=0 tx=0).stop 2024-09-19T05:14:25.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:25.858+0000 7f62ba600640 1 -- 172.21.15.43:0/3417975113 shutdown_connections 2024-09-19T05:14:25.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:25.858+0000 7f62ba600640 1 --2- 172.21.15.43:0/3417975113 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62b4108f60 0x7f62b4110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:25.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:25.858+0000 7f62ba600640 1 --2- 172.21.15.43:0/3417975113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62b41047e0 0x7f62b4108830 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:25.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:25.858+0000 7f62ba600640 1 --2- 172.21.15.43:0/3417975113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62b4103e30 0x7f62b4104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:25.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:25.858+0000 7f62ba600640 1 -- 172.21.15.43:0/3417975113 >> 172.21.15.43:0/3417975113 conn(0x7f62b40ff8d0 msgr2=0x7f62b4101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:25.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:25.858+0000 7f62ba600640 1 -- 172.21.15.43:0/3417975113 shutdown_connections 2024-09-19T05:14:25.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:25.858+0000 7f62ba600640 1 -- 172.21.15.43:0/3417975113 wait complete. 2024-09-19T05:14:25.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:25.859+0000 7f62ba600640 1 Processor -- start 2024-09-19T05:14:25.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:25.859+0000 7f62ba600640 1 -- start start 2024-09-19T05:14:25.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:25.860+0000 7f62ba600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62b4103e30 0x7f62b419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:25.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:25.860+0000 7f62ba600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62b41047e0 0x7f62b419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:25.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:25.860+0000 7f62ba600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62b4108f60 0x7f62b419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:25.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:25.860+0000 7f62ba600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f62b4113fd0 con 0x7f62b4108f60 2024-09-19T05:14:25.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:25.860+0000 7f62ba600640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f62b4113e50 con 0x7f62b4103e30 2024-09-19T05:14:25.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:25.860+0000 7f62ba600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f62b4114150 con 0x7f62b41047e0 2024-09-19T05:14:25.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:25.860+0000 7f62b3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62b41047e0 0x7f62b419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:25.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:25.860+0000 7f62b3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62b41047e0 0x7f62b419a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:43150/0 (socket says 172.21.15.43:43150) 2024-09-19T05:14:25.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:25.860+0000 7f62b9200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62b4103e30 0x7f62b419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:25.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:25.860+0000 7f62b3e00640 1 -- 172.21.15.43:0/528611995 learned_addr learned my addr 172.21.15.43:0/528611995 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:25.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:25.860+0000 7f62b9c00640 1 --2- 172.21.15.43:0/528611995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62b4108f60 0x7f62b419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:25.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:25.861+0000 7f62b9200640 1 -- 172.21.15.43:0/528611995 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62b41047e0 msgr2=0x7f62b419a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:25.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:25.861+0000 7f62b9200640 1 --2- 172.21.15.43:0/528611995 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62b41047e0 0x7f62b419a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:25.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:25.861+0000 7f62b9200640 1 -- 172.21.15.43:0/528611995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62b4108f60 msgr2=0x7f62b419ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:25.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:25.861+0000 7f62b9200640 1 --2- 172.21.15.43:0/528611995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62b4108f60 0x7f62b419ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:25.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:25.861+0000 7f62b9200640 1 -- 172.21.15.43:0/528611995 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b419f460 con 0x7f62b4103e30 2024-09-19T05:14:25.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:25.861+0000 7f62b3e00640 1 --2- 172.21.15.43:0/528611995 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62b41047e0 0x7f62b419a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:25.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:25.861+0000 7f62b9c00640 1 --2- 172.21.15.43:0/528611995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62b4108f60 0x7f62b419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:14:25.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:25.861+0000 7f62b9200640 1 --2- 172.21.15.43:0/528611995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62b4103e30 0x7f62b419a4b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f62a800e9f0 tx=0x7f62a800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:25.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:25.861+0000 7f62b1600640 1 -- 172.21.15.43:0/528611995 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a800ccc0 con 0x7f62b4103e30 2024-09-19T05:14:25.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:25.861+0000 7f62b1600640 1 -- 172.21.15.43:0/528611995 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62a8010910 con 0x7f62b4103e30 2024-09-19T05:14:25.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:25.861+0000 7f62b1600640 1 -- 172.21.15.43:0/528611995 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a8018960 con 0x7f62b4103e30 2024-09-19T05:14:25.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:25.861+0000 7f62ba600640 1 -- 172.21.15.43:0/528611995 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62b419f750 con 0x7f62b4103e30 2024-09-19T05:14:25.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:25.861+0000 7f62ba600640 1 -- 172.21.15.43:0/528611995 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62b41a7030 con 0x7f62b4103e30 2024-09-19T05:14:25.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:25.862+0000 7f62ba600640 1 -- 172.21.15.43:0/528611995 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62b4104210 con 0x7f62b4103e30 2024-09-19T05:14:25.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:25.865+0000 7f62b1600640 1 -- 172.21.15.43:0/528611995 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f62a8010ac0 con 0x7f62b4103e30 2024-09-19T05:14:25.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:25.866+0000 7f62b1600640 1 --2- 172.21.15.43:0/528611995 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f629c0a3430 0x7f629c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:25.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:25.866+0000 7f62b1600640 1 -- 172.21.15.43:0/528611995 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f62a80d05e0 con 0x7f62b4103e30 2024-09-19T05:14:25.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:25.866+0000 7f62b3e00640 1 --2- 172.21.15.43:0/528611995 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f629c0a3430 0x7f629c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:25.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:25.867+0000 7f62b3e00640 1 --2- 172.21.15.43:0/528611995 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f629c0a3430 0x7f629c0a58f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f62a00096c0 tx=0x7f62a00046c0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:25.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:25.868+0000 7f62b1600640 1 -- 172.21.15.43:0/528611995 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f62a809cf50 con 0x7f62b4103e30 2024-09-19T05:14:26.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:26.092+0000 7f62ba600640 1 -- 172.21.15.43:0/528611995 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62b419b810 con 0x7f62b4103e30 2024-09-19T05:14:26.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:26.094+0000 7f62b1600640 1 -- 172.21.15.43:0/528611995 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11965 (secure 0 0 0) 0x7f62a809c850 con 0x7f62b4103e30 2024-09-19T05:14:26.096 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:14:26.096 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":29,"btime":"2024-09-19T05:04:38:785407+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/773801169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":773801169},{"type":"v1","addr":"172.21.15.43:6835","nonce":773801169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1799137902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1799137902},{"type":"v1","addr":"172.21.15.43:6837","nonce":1799137902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:02:25.463041+0000","modified":"2024-09-19T05:04:38.039809+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24518,"mds_2":24511,"mds_3":24506,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14541":{"gid":14541,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/794881135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":794881135},{"type":"v1","addr":"172.21.15.43:6841","nonce":794881135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/2836453158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2836453158},{"type":"v1","addr":"172.21.15.151:6833","nonce":2836453158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/178785812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":178785812},{"type":"v1","addr":"172.21.15.96:6833","nonce":178785812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1671043820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1671043820},{"type":"v1","addr":"172.21.15.96:6835","nonce":1671043820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2972514328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2972514328},{"type":"v1","addr":"172.21.15.151:6835","nonce":2972514328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.151:6837/3960378010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3960378010},{"type":"v1","addr":"172.21.15.151:6837","nonce":3960378010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6837/3978388286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3978388286},{"type":"v1","addr":"172.21.15.96:6837","nonce":3978388286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.151:6839/3794104313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3794104313},{"type":"v1","addr":"172.21.15.151:6839","nonce":3794104313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.43:6839/3404555030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3404555030},{"type":"v1","addr":"172.21.15.43:6839","nonce":3404555030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/605527621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":605527621},{"type":"v1","addr":"172.21.15.96:6839","nonce":605527621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24518,24511,24506,24493]},"id":1}]} 2024-09-19T05:14:26.096 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 29 2024-09-19T05:14:26.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:26.096+0000 7f62ba600640 1 -- 172.21.15.43:0/528611995 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f629c0a3430 msgr2=0x7f629c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:26.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:26.096+0000 7f62ba600640 1 --2- 172.21.15.43:0/528611995 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f629c0a3430 0x7f629c0a58f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f62a00096c0 tx=0x7f62a00046c0 comp rx=0 tx=0).stop 2024-09-19T05:14:26.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:26.096+0000 7f62ba600640 1 -- 172.21.15.43:0/528611995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62b4103e30 msgr2=0x7f62b419a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:26.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:26.096+0000 7f62ba600640 1 --2- 172.21.15.43:0/528611995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62b4103e30 0x7f62b419a4b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f62a800e9f0 tx=0x7f62a800eeb0 comp rx=0 tx=0).stop 2024-09-19T05:14:26.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:26.096+0000 7f62ba600640 1 -- 172.21.15.43:0/528611995 shutdown_connections 2024-09-19T05:14:26.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:26.096+0000 7f62ba600640 1 --2- 172.21.15.43:0/528611995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62b4108f60 0x7f62b419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:26.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:26.096+0000 7f62ba600640 1 --2- 172.21.15.43:0/528611995 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62b41047e0 0x7f62b419a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:26.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:26.096+0000 7f62ba600640 1 --2- 172.21.15.43:0/528611995 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f629c0a3430 0x7f629c0a58f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:26.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:26.096+0000 7f62ba600640 1 --2- 172.21.15.43:0/528611995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62b4103e30 0x7f62b419a4b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:26.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:26.096+0000 7f62ba600640 1 -- 172.21.15.43:0/528611995 >> 172.21.15.43:0/528611995 conn(0x7f62b40ff8d0 msgr2=0x7f62b4100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:26.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:26.096+0000 7f62ba600640 1 -- 172.21.15.43:0/528611995 shutdown_connections 2024-09-19T05:14:26.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:26.096+0000 7f62ba600640 1 -- 172.21.15.43:0/528611995 wait complete. 2024-09-19T05:14:26.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:26 smithi043 ceph-mon[30800]: pgmap v304: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-09-19T05:14:26.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:26 smithi151 ceph-mon[35425]: pgmap v304: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-09-19T05:14:26.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:26 smithi096 ceph-mon[35085]: pgmap v304: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-09-19T05:14:26.868 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T05:14:27.243 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:14:27.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:27 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/528611995' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:14:27.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:27 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/528611995' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:14:27.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:27 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/528611995' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:14:27.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.787+0000 7f94c6a00640 1 -- 172.21.15.43:0/3521180083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94c0074eb0 msgr2=0x7f94c0112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:27.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.787+0000 7f94c6a00640 1 --2- 172.21.15.43:0/3521180083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94c0074eb0 0x7f94c0112650 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f94a8009900 tx=0x7f94a802a480 comp rx=0 tx=0).stop 2024-09-19T05:14:27.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.787+0000 7f94c6a00640 1 -- 172.21.15.43:0/3521180083 shutdown_connections 2024-09-19T05:14:27.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.787+0000 7f94c6a00640 1 --2- 172.21.15.43:0/3521180083 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94c0112b90 0x7f94c0114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:27.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.787+0000 7f94c6a00640 1 --2- 172.21.15.43:0/3521180083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94c0074eb0 0x7f94c0112650 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:27.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.787+0000 7f94c6a00640 1 --2- 172.21.15.43:0/3521180083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94c0074590 0x7f94c0074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:27.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.787+0000 7f94c6a00640 1 -- 172.21.15.43:0/3521180083 >> 172.21.15.43:0/3521180083 conn(0x7f94c00fd8c0 msgr2=0x7f94c00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:27.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.787+0000 7f94c6a00640 1 -- 172.21.15.43:0/3521180083 shutdown_connections 2024-09-19T05:14:27.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.787+0000 7f94c6a00640 1 -- 172.21.15.43:0/3521180083 wait complete. 2024-09-19T05:14:27.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.788+0000 7f94c6a00640 1 Processor -- start 2024-09-19T05:14:27.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.788+0000 7f94c6a00640 1 -- start start 2024-09-19T05:14:27.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.788+0000 7f94c6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94c0074590 0x7f94c01a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:27.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.788+0000 7f94c6a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94c0074eb0 0x7f94c01a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:27.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.788+0000 7f94c6a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94c0112b90 0x7f94c019e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:27.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.788+0000 7f94c6a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f94c0118590 con 0x7f94c0074590 2024-09-19T05:14:27.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.788+0000 7f94c6a00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f94c0118410 con 0x7f94c0112b90 2024-09-19T05:14:27.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.788+0000 7f94c6a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f94c0118710 con 0x7f94c0074eb0 2024-09-19T05:14:27.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.789+0000 7f94bfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94c0074590 0x7f94c01a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:27.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.789+0000 7f94bfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94c0074590 0x7f94c01a4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44546/0 (socket says 172.21.15.43:44546) 2024-09-19T05:14:27.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.789+0000 7f94bfe00640 1 -- 172.21.15.43:0/1691935817 learned_addr learned my addr 172.21.15.43:0/1691935817 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:27.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.789+0000 7f94c4a00640 1 --2- 172.21.15.43:0/1691935817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94c0112b90 0x7f94c019e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:27.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.789+0000 7f94b7400640 1 --2- 172.21.15.43:0/1691935817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94c0074eb0 0x7f94c01a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:27.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.789+0000 7f94bfe00640 1 -- 172.21.15.43:0/1691935817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94c0074eb0 msgr2=0x7f94c01a4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:27.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.789+0000 7f94bfe00640 1 --2- 172.21.15.43:0/1691935817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94c0074eb0 0x7f94c01a4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:27.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.789+0000 7f94bfe00640 1 -- 172.21.15.43:0/1691935817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94c0112b90 msgr2=0x7f94c019e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:27.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.789+0000 7f94bfe00640 1 --2- 172.21.15.43:0/1691935817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94c0112b90 0x7f94c019e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:27.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.789+0000 7f94bfe00640 1 -- 172.21.15.43:0/1691935817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c019f000 con 0x7f94c0074590 2024-09-19T05:14:27.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.789+0000 7f94b7400640 1 --2- 172.21.15.43:0/1691935817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94c0074eb0 0x7f94c01a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:14:27.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.789+0000 7f94bfe00640 1 --2- 172.21.15.43:0/1691935817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94c0074590 0x7f94c01a4710 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f94b000d950 tx=0x7f94b000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:27.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.790+0000 7f94bd600640 1 -- 172.21.15.43:0/1691935817 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b0017030 con 0x7f94c0074590 2024-09-19T05:14:27.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.790+0000 7f94bd600640 1 -- 172.21.15.43:0/1691935817 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94b0004020 con 0x7f94c0074590 2024-09-19T05:14:27.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.790+0000 7f94bd600640 1 -- 172.21.15.43:0/1691935817 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b00047a0 con 0x7f94c0074590 2024-09-19T05:14:27.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.790+0000 7f94c6a00640 1 -- 172.21.15.43:0/1691935817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94c019f2f0 con 0x7f94c0074590 2024-09-19T05:14:27.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.790+0000 7f94c6a00640 1 -- 172.21.15.43:0/1691935817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94c01ab5e0 con 0x7f94c0074590 2024-09-19T05:14:27.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.793+0000 7f94bd600640 1 -- 172.21.15.43:0/1691935817 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f94b00041d0 con 0x7f94c0074590 2024-09-19T05:14:27.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.793+0000 7f94c9423740 1 Processor -- start 2024-09-19T05:14:27.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.793+0000 7f94c9423740 1 -- start start 2024-09-19T05:14:27.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.793+0000 7f94c9423740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567252afc70 0x5567252b0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:27.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.793+0000 7f94c9423740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567252b0590 0x5567252ba910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:27.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.793+0000 7f94c9423740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567252bae50 0x5567252bd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:27.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.793+0000 7f94c9423740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5567252163e0 con 0x5567252b0590 2024-09-19T05:14:27.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.793+0000 7f94c9423740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5567251bdd80 con 0x5567252afc70 2024-09-19T05:14:27.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.793+0000 7f94c9423740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5567252123a0 con 0x5567252bae50 2024-09-19T05:14:27.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.794+0000 7f94bfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567252b0590 0x5567252ba910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:27.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.794+0000 7f94c4a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567252afc70 0x5567252b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:27.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.794+0000 7f94c4a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567252afc70 0x5567252b0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:37550/0 (socket says 172.21.15.43:37550) 2024-09-19T05:14:27.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.794+0000 7f94c4a00640 1 -- 172.21.15.43:0/625542429 learned_addr learned my addr 172.21.15.43:0/625542429 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:27.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.794+0000 7f94b7400640 1 --2- 172.21.15.43:0/625542429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567252bae50 0x5567252bd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:27.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.794+0000 7f94c4a00640 1 -- 172.21.15.43:0/625542429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567252bae50 msgr2=0x5567252bd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:27.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.794+0000 7f94c4a00640 1 --2- 172.21.15.43:0/625542429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567252bae50 0x5567252bd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:27.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.794+0000 7f94c4a00640 1 -- 172.21.15.43:0/625542429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567252b0590 msgr2=0x5567252ba910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:27.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.794+0000 7f94c4a00640 1 --2- 172.21.15.43:0/625542429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567252b0590 0x5567252ba910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:27.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.794+0000 7f94c4a00640 1 -- 172.21.15.43:0/625542429 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567251dcae0 con 0x5567252afc70 2024-09-19T05:14:27.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.794+0000 7f94c4a00640 1 --2- 172.21.15.43:0/625542429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567252afc70 0x5567252b0050 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f94b8009f40 tx=0x7f94b8010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba61d7fd71d1f426 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:27.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.795+0000 7f94b5200640 1 -- 172.21.15.43:0/625542429 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b8016070 con 0x5567252afc70 2024-09-19T05:14:27.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.795+0000 7f94bd600640 1 --2- 172.21.15.43:0/1691935817 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f94a40a3430 0x7f94a40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:27.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.795+0000 7f94b5200640 1 -- 172.21.15.43:0/625542429 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94b8004d20 con 0x5567252afc70 2024-09-19T05:14:27.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.795+0000 7f94b5200640 1 -- 172.21.15.43:0/625542429 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b80128b0 con 0x5567252afc70 2024-09-19T05:14:27.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.795+0000 7f94bd600640 1 -- 172.21.15.43:0/1691935817 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f94b00cf700 con 0x7f94c0074590 2024-09-19T05:14:27.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.795+0000 7f94bfe00640 1 --2- 172.21.15.43:0/1691935817 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f94a40a3430 0x7f94a40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:27.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.795+0000 7f94c9423740 1 -- 172.21.15.43:0/625542429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567252afc70 msgr2=0x5567252b0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:27.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.795+0000 7f94c9423740 1 --2- 172.21.15.43:0/625542429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567252afc70 0x5567252b0050 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f94b8009f40 tx=0x7f94b8010800 comp rx=0 tx=0).stop 2024-09-19T05:14:27.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.796+0000 7f94bfe00640 1 --2- 172.21.15.43:0/1691935817 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f94a40a3430 0x7f94a40a58f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x5567252ad060 tx=0x7f94b00d5530 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:27.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.796+0000 7f94c9423740 1 -- 172.21.15.43:0/625542429 shutdown_connections 2024-09-19T05:14:27.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.796+0000 7f94c9423740 1 --2- 172.21.15.43:0/625542429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567252bae50 0x5567252bd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:27.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.796+0000 7f94c9423740 1 --2- 172.21.15.43:0/625542429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567252b0590 0x5567252ba910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:27.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.796+0000 7f94c9423740 1 --2- 172.21.15.43:0/625542429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567252afc70 0x5567252b0050 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:27.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.796+0000 7f94c9423740 1 -- 172.21.15.43:0/625542429 >> 172.21.15.43:0/625542429 conn(0x5567252ac200 msgr2=0x5567252ac5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:27.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.796+0000 7f94c9423740 1 -- 172.21.15.43:0/625542429 shutdown_connections 2024-09-19T05:14:27.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.796+0000 7f94c9423740 1 -- 172.21.15.43:0/625542429 wait complete. 2024-09-19T05:14:27.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.796+0000 7f94c9423740 1 -- start start 2024-09-19T05:14:27.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.796+0000 7f94c9423740 1 Processor -- start 2024-09-19T05:14:27.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.797+0000 7f94c9423740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567252afc70 0x5567252b7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:27.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.797+0000 7f94c9423740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567252b0590 0x5567252b82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:27.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.797+0000 7f94c9423740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567252bae50 0x5567252b8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:27.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.797+0000 7f94c9423740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5567252bda20 con 0x5567252bae50 2024-09-19T05:14:27.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.797+0000 7f94c9423740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5567251d10f0 con 0x5567252afc70 2024-09-19T05:14:27.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.797+0000 7f94c9423740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5567252bdba0 con 0x5567252b0590 2024-09-19T05:14:27.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.797+0000 7f94c9423740 10 client.0 _ll_register_callbacks cb 0x5567251b8bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:27.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.797+0000 7f94c4a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567252bae50 0x5567252b8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:27.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.797+0000 7f94c4a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567252bae50 0x5567252b8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44564/0 (socket says 172.21.15.43:44564) 2024-09-19T05:14:27.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.797+0000 7f94c4a00640 1 -- 172.21.15.43:0/2742613356 learned_addr learned my addr 172.21.15.43:0/2742613356 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:27.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.797+0000 7f94b7400640 1 --2- 172.21.15.43:0/2742613356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567252b0590 0x5567252b82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:27.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.797+0000 7f94c4a00640 1 --2- 172.21.15.43:0/2742613356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567252afc70 0x5567252b7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:27.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.798+0000 7f94c4a00640 1 -- 172.21.15.43:0/2742613356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567252b0590 msgr2=0x5567252b82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:27.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.798+0000 7f94c4a00640 1 --2- 172.21.15.43:0/2742613356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567252b0590 0x5567252b82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:27.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.798+0000 7f94c4a00640 1 -- 172.21.15.43:0/2742613356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567252afc70 msgr2=0x5567252b7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:27.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.798+0000 7f94c4a00640 1 --2- 172.21.15.43:0/2742613356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567252afc70 0x5567252b7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:27.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.798+0000 7f94c4a00640 1 -- 172.21.15.43:0/2742613356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556724f873c0 con 0x5567252bae50 2024-09-19T05:14:27.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.798+0000 7f94c4a00640 1 --2- 172.21.15.43:0/2742613356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567252bae50 0x5567252b8820 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f94b8005a30 tx=0x7f94b8012450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63748094dd7ad6af server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:27.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.798+0000 7f94ad600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:14:27.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.798+0000 7f94ad600640 1 -- 172.21.15.43:0/2742613356 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b8016030 con 0x5567252bae50 2024-09-19T05:14:27.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.798+0000 7f94ad600640 1 -- 172.21.15.43:0/2742613356 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f94b8012bc0 con 0x5567252bae50 2024-09-19T05:14:27.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.798+0000 7f94c9423740 1 -- 172.21.15.43:0/2742613356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94b8004d20 con 0x5567252bae50 2024-09-19T05:14:27.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.799+0000 7f94ad600640 1 -- 172.21.15.43:0/2742613356 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f94b8023040 con 0x5567252bae50 2024-09-19T05:14:27.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.799+0000 7f94c9423740 10 client.17517 fetch_fsmap learned FSMap version 29 2024-09-19T05:14:27.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.799+0000 7f94c9423740 1 -- 172.21.15.43:0/2742613356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5567252c9ac0 con 0x5567252bae50 2024-09-19T05:14:27.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.799+0000 7f94ad600640 1 -- 172.21.15.43:0/2742613356 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94b8004d20 con 0x5567252bae50 2024-09-19T05:14:27.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.799+0000 7f94ad600640 1 -- 172.21.15.43:0/2742613356 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f94b800f040 con 0x5567252bae50 2024-09-19T05:14:27.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.799+0000 7f94c9423740 10 client.17517 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:14:27.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.799+0000 7f94c9423740 10 client.17517 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-19T05:14:27.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.799+0000 7f94c9423740 20 client.17517 populate_metadata read hostname 'smithi043' 2024-09-19T05:14:27.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.799+0000 7f94c9423740 1 --2- 172.21.15.43:0/2742613356 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x5567252caed0 0x5567252cd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:27.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.799+0000 7f94c9423740 4 client.17517 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:14:27.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.799+0000 7f94c9423740 1 -- 172.21.15.43:0/2742613356 --> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556725223950 con 0x5567252caed0 2024-09-19T05:14:27.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.800+0000 7f94b7400640 1 --2- 172.21.15.43:0/2742613356 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x5567252caed0 0x5567252cd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:27.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.800+0000 7f94b7400640 1 --2- 172.21.15.43:0/2742613356 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x5567252caed0 0x5567252cd2c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=472a7106ca1c4ef9 server_cookie=765b871e0c9b2a52 in_seq=0 out_seq=0 2024-09-19T05:14:27.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.800+0000 7f94ad600640 10 client.17517 ms_handle_connect on v2:172.21.15.151:6836/3960378010 2024-09-19T05:14:27.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.801+0000 7f94ad600640 1 -- 172.21.15.43:0/2742613356 <== mds.4 v2:172.21.15.151:6836/3960378010 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556725223950 con 0x5567252caed0 2024-09-19T05:14:27.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.801+0000 7f94ad600640 10 client.17517 handle_command_reply: tid=0 2024-09-19T05:14:27.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.801+0000 7f94c9423740 1 client.17517 shutdown 2024-09-19T05:14:27.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.801+0000 7f94c9423740 1 -- 172.21.15.43:0/2742613356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567252bae50 msgr2=0x5567252b8820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:27.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.801+0000 7f94c9423740 1 --2- 172.21.15.43:0/2742613356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567252bae50 0x5567252b8820 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f94b8005a30 tx=0x7f94b8012450 comp rx=0 tx=0).stop 2024-09-19T05:14:27.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.802+0000 7f94c9423740 1 -- 172.21.15.43:0/2742613356 shutdown_connections 2024-09-19T05:14:27.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.802+0000 7f94c9423740 1 --2- 172.21.15.43:0/2742613356 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x5567252caed0 0x5567252cd2c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:27.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.802+0000 7f94c9423740 1 --2- 172.21.15.43:0/2742613356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567252bae50 0x5567252b8820 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:27.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.802+0000 7f94c9423740 1 --2- 172.21.15.43:0/2742613356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567252b0590 0x5567252b82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:27.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.802+0000 7f94c9423740 1 --2- 172.21.15.43:0/2742613356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567252afc70 0x5567252b7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:27.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.802+0000 7f94ad600640 0 client.17517 ms_handle_reset on v2:172.21.15.151:6836/3960378010 2024-09-19T05:14:27.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.802+0000 7f94c9423740 1 -- 172.21.15.43:0/2742613356 >> 172.21.15.43:0/2742613356 conn(0x5567252ac200 msgr2=0x5567252ab380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:27.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.802+0000 7f94c9423740 1 -- 172.21.15.43:0/2742613356 shutdown_connections 2024-09-19T05:14:27.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.802+0000 7f94c9423740 1 -- 172.21.15.43:0/2742613356 wait complete. 2024-09-19T05:14:27.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.802+0000 7f94c9423740 20 client.17517 trim_cache size 0 max 16384 2024-09-19T05:14:27.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.816+0000 7f94c9423740 1 Processor -- start 2024-09-19T05:14:27.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.816+0000 7f94c9423740 1 -- start start 2024-09-19T05:14:27.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.816+0000 7f94c9423740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567252afc70 0x5567252d2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:27.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.816+0000 7f94c9423740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567252b0590 0x5567252d2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:27.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.816+0000 7f94c9423740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567252bae50 0x5567252b5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:27.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.816+0000 7f94c9423740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f94c01b4030 con 0x5567252b0590 2024-09-19T05:14:27.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.816+0000 7f94c9423740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f94c01b5b70 con 0x5567252bae50 2024-09-19T05:14:27.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.816+0000 7f94c9423740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f94c01b4700 con 0x5567252afc70 2024-09-19T05:14:27.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.816+0000 7f94b7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567252b0590 0x5567252d2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:27.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.817+0000 7f94b7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567252b0590 0x5567252d2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44576/0 (socket says 172.21.15.43:44576) 2024-09-19T05:14:27.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.817+0000 7f94b7400640 1 -- 172.21.15.43:0/2111297335 learned_addr learned my addr 172.21.15.43:0/2111297335 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:27.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.817+0000 7f94c4a00640 1 --2- 172.21.15.43:0/2111297335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567252afc70 0x5567252d2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:27.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.817+0000 7f94c4a00640 1 --2- 172.21.15.43:0/2111297335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567252bae50 0x5567252b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:27.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.817+0000 7f94b7400640 1 -- 172.21.15.43:0/2111297335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567252afc70 msgr2=0x5567252d2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:27.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.817+0000 7f94b7400640 1 --2- 172.21.15.43:0/2111297335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567252afc70 0x5567252d2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:27.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.817+0000 7f94b7400640 1 -- 172.21.15.43:0/2111297335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567252bae50 msgr2=0x5567252b5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:27.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.817+0000 7f94b7400640 1 --2- 172.21.15.43:0/2111297335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567252bae50 0x5567252b5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:27.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.817+0000 7f94b7400640 1 -- 172.21.15.43:0/2111297335 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567252a7770 con 0x5567252b0590 2024-09-19T05:14:27.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.817+0000 7f94c4a00640 1 --2- 172.21.15.43:0/2111297335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567252bae50 0x5567252b5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:14:27.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.817+0000 7f94b7400640 1 --2- 172.21.15.43:0/2111297335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567252b0590 0x5567252d2eb0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f94a80095d0 tx=0x7f94a8005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2131a3fd7f44c520 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:27.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.818+0000 7f94afe00640 1 -- 172.21.15.43:0/2111297335 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a8035040 con 0x5567252b0590 2024-09-19T05:14:27.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.818+0000 7f94afe00640 1 -- 172.21.15.43:0/2111297335 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94a804a070 con 0x5567252b0590 2024-09-19T05:14:27.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.818+0000 7f94afe00640 1 -- 172.21.15.43:0/2111297335 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a8031810 con 0x5567252b0590 2024-09-19T05:14:27.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.818+0000 7f94c9423740 1 -- 172.21.15.43:0/2111297335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567252b0590 msgr2=0x5567252d2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:27.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.818+0000 7f94c9423740 1 --2- 172.21.15.43:0/2111297335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567252b0590 0x5567252d2eb0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f94a80095d0 tx=0x7f94a8005b90 comp rx=0 tx=0).stop 2024-09-19T05:14:27.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.818+0000 7f94c9423740 1 -- 172.21.15.43:0/2111297335 shutdown_connections 2024-09-19T05:14:27.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.818+0000 7f94c9423740 1 --2- 172.21.15.43:0/2111297335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567252bae50 0x5567252b5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:27.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.818+0000 7f94c9423740 1 --2- 172.21.15.43:0/2111297335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567252b0590 0x5567252d2eb0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:27.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.818+0000 7f94c9423740 1 --2- 172.21.15.43:0/2111297335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567252afc70 0x5567252d2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:27.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.818+0000 7f94c9423740 1 -- 172.21.15.43:0/2111297335 >> 172.21.15.43:0/2111297335 conn(0x5567252ac200 msgr2=0x5567252b9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:27.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.818+0000 7f94c9423740 1 -- 172.21.15.43:0/2111297335 shutdown_connections 2024-09-19T05:14:27.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.818+0000 7f94c9423740 1 -- 172.21.15.43:0/2111297335 wait complete. 2024-09-19T05:14:27.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.819+0000 7f94c9423740 1 -- start start 2024-09-19T05:14:27.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.819+0000 7f94c9423740 1 Processor -- start 2024-09-19T05:14:27.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.820+0000 7f94c9423740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567252afc70 0x5567252c3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:27.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.820+0000 7f94c9423740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567252b0590 0x5567252be550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:27.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.820+0000 7f94c9423740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567252bae50 0x5567252bea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:27.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.820+0000 7f94c9423740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f94c01b54a0 con 0x5567252afc70 2024-09-19T05:14:27.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.820+0000 7f94c9423740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f94c01b4dd0 con 0x5567252bae50 2024-09-19T05:14:27.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.820+0000 7f94c9423740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f94c01b2030 con 0x5567252b0590 2024-09-19T05:14:27.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.820+0000 7f94c9423740 10 client.0 _ll_register_callbacks cb 0x7f94b00d5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:27.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.820+0000 7f94c4a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567252afc70 0x5567252c3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:27.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.820+0000 7f94c4a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567252afc70 0x5567252c3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44592/0 (socket says 172.21.15.43:44592) 2024-09-19T05:14:27.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.820+0000 7f94c4a00640 1 -- 172.21.15.43:0/3282553460 learned_addr learned my addr 172.21.15.43:0/3282553460 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:27.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.820+0000 7f94b7400640 1 --2- 172.21.15.43:0/3282553460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567252b0590 0x5567252be550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:27.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.820+0000 7f94c4a00640 1 --2- 172.21.15.43:0/3282553460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567252bae50 0x5567252bea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:27.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.820+0000 7f94b7400640 1 -- 172.21.15.43:0/3282553460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567252bae50 msgr2=0x5567252bea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:27.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.820+0000 7f94b7400640 1 --2- 172.21.15.43:0/3282553460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567252bae50 0x5567252bea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:27.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.820+0000 7f94b7400640 1 -- 172.21.15.43:0/3282553460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567252afc70 msgr2=0x5567252c3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:27.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.820+0000 7f94b7400640 1 --2- 172.21.15.43:0/3282553460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567252afc70 0x5567252c3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:27.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.820+0000 7f94b7400640 1 -- 172.21.15.43:0/3282553460 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567252befd0 con 0x5567252b0590 2024-09-19T05:14:27.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.820+0000 7f94c4a00640 1 --2- 172.21.15.43:0/3282553460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567252afc70 0x5567252c3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:27.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.821+0000 7f94b7400640 1 --2- 172.21.15.43:0/3282553460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567252b0590 0x5567252be550 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f94a8006010 tx=0x7f94a8005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9aa7abcc819b1ce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:27.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.821+0000 7f94ad600640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:14:27.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.821+0000 7f94ad600640 1 -- 172.21.15.43:0/3282553460 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a8052070 con 0x5567252b0590 2024-09-19T05:14:27.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.821+0000 7f94c9423740 1 -- 172.21.15.43:0/3282553460 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94a804a070 con 0x5567252b0590 2024-09-19T05:14:27.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.821+0000 7f94ad600640 1 -- 172.21.15.43:0/3282553460 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f94a8005240 con 0x5567252b0590 2024-09-19T05:14:27.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.821+0000 7f94ad600640 1 -- 172.21.15.43:0/3282553460 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f94a8035050 con 0x5567252b0590 2024-09-19T05:14:27.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.821+0000 7f94c9423740 10 client.27445 fetch_fsmap learned FSMap version 29 2024-09-19T05:14:27.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.821+0000 7f94c9423740 1 -- 172.21.15.43:0/3282553460 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5567253355a0 con 0x5567252b0590 2024-09-19T05:14:27.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.822+0000 7f94ad600640 1 -- 172.21.15.43:0/3282553460 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94a804a070 con 0x5567252b0590 2024-09-19T05:14:27.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.822+0000 7f94ad600640 1 -- 172.21.15.43:0/3282553460 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f94a80035f0 con 0x5567252b0590 2024-09-19T05:14:27.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.822+0000 7f94c9423740 10 client.27445 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:14:27.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.822+0000 7f94c9423740 10 client.27445 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-19T05:14:27.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.822+0000 7f94c9423740 20 client.27445 populate_metadata read hostname 'smithi043' 2024-09-19T05:14:27.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.822+0000 7f94c9423740 1 --2- 172.21.15.43:0/3282553460 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x556725335c60 0x556725338050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:27.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.822+0000 7f94c9423740 4 client.27445 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:14:27.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.822+0000 7f94c9423740 1 -- 172.21.15.43:0/3282553460 --> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55672521de60 con 0x556725335c60 2024-09-19T05:14:27.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.822+0000 7f94c4a00640 1 --2- 172.21.15.43:0/3282553460 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x556725335c60 0x556725338050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:27.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.823+0000 7f94c4a00640 1 --2- 172.21.15.43:0/3282553460 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x556725335c60 0x556725338050 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8ab42165452ab003 server_cookie=4659f2815d196b0c in_seq=0 out_seq=0 2024-09-19T05:14:27.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.823+0000 7f94ad600640 10 client.27445 ms_handle_connect on v2:172.21.15.151:6836/3960378010 2024-09-19T05:14:27.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.823+0000 7f94ad600640 1 -- 172.21.15.43:0/3282553460 <== mds.4 v2:172.21.15.151:6836/3960378010 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55672521de60 con 0x556725335c60 2024-09-19T05:14:27.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.823+0000 7f94ad600640 10 client.27445 handle_command_reply: tid=0 2024-09-19T05:14:27.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.823+0000 7f94c9423740 1 client.27445 shutdown 2024-09-19T05:14:27.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.823+0000 7f94c9423740 1 -- 172.21.15.43:0/3282553460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567252b0590 msgr2=0x5567252be550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:27.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.823+0000 7f94c9423740 1 --2- 172.21.15.43:0/3282553460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567252b0590 0x5567252be550 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f94a8006010 tx=0x7f94a8005c00 comp rx=0 tx=0).stop 2024-09-19T05:14:27.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.823+0000 7f94c9423740 1 -- 172.21.15.43:0/3282553460 shutdown_connections 2024-09-19T05:14:27.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.823+0000 7f94c9423740 1 --2- 172.21.15.43:0/3282553460 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x556725335c60 0x556725338050 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:27.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.823+0000 7f94c9423740 1 --2- 172.21.15.43:0/3282553460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567252bae50 0x5567252bea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:27.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.823+0000 7f94c9423740 1 --2- 172.21.15.43:0/3282553460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567252b0590 0x5567252be550 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:27.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.823+0000 7f94c9423740 1 --2- 172.21.15.43:0/3282553460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567252afc70 0x5567252c3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:27.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.823+0000 7f94ad600640 0 client.27445 ms_handle_reset on v2:172.21.15.151:6836/3960378010 2024-09-19T05:14:27.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.823+0000 7f94c9423740 1 -- 172.21.15.43:0/3282553460 >> 172.21.15.43:0/3282553460 conn(0x5567252ac200 msgr2=0x5567252b7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:27.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.823+0000 7f94c9423740 1 -- 172.21.15.43:0/3282553460 shutdown_connections 2024-09-19T05:14:27.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.823+0000 7f94c9423740 1 -- 172.21.15.43:0/3282553460 wait complete. 2024-09-19T05:14:27.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.824+0000 7f94c9423740 20 client.27445 trim_cache size 0 max 16384 2024-09-19T05:14:27.829 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:14:27.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.824+0000 7f94c6a00640 1 -- 172.21.15.43:0/1691935817 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f94a40a3430 msgr2=0x7f94a40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:27.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.824+0000 7f94c6a00640 1 --2- 172.21.15.43:0/1691935817 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f94a40a3430 0x7f94a40a58f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x5567252ad060 tx=0x7f94b00d5530 comp rx=0 tx=0).stop 2024-09-19T05:14:27.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.824+0000 7f94c6a00640 1 -- 172.21.15.43:0/1691935817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94c0074590 msgr2=0x7f94c01a4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:27.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.824+0000 7f94c6a00640 1 --2- 172.21.15.43:0/1691935817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94c0074590 0x7f94c01a4710 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f94b000d950 tx=0x7f94b000de10 comp rx=0 tx=0).stop 2024-09-19T05:14:27.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.824+0000 7f94c6a00640 1 -- 172.21.15.43:0/1691935817 shutdown_connections 2024-09-19T05:14:27.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.824+0000 7f94c6a00640 1 --2- 172.21.15.43:0/1691935817 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f94a40a3430 0x7f94a40a58f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:27.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.824+0000 7f94c6a00640 1 --2- 172.21.15.43:0/1691935817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f94c0112b90 0x7f94c019e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:27.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.824+0000 7f94c6a00640 1 --2- 172.21.15.43:0/1691935817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94c0074eb0 0x7f94c01a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:27.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.824+0000 7f94c6a00640 1 --2- 172.21.15.43:0/1691935817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94c0074590 0x7f94c01a4710 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:27.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.824+0000 7f94c6a00640 1 -- 172.21.15.43:0/1691935817 >> 172.21.15.43:0/1691935817 conn(0x7f94c00fd8c0 msgr2=0x7f94c00ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:27.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.825+0000 7f94c6a00640 1 -- 172.21.15.43:0/1691935817 shutdown_connections 2024-09-19T05:14:27.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:27.825+0000 7f94c6a00640 1 -- 172.21.15.43:0/1691935817 wait complete. 2024-09-19T05:14:28.306 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:28 smithi151 ceph-mon[35425]: pgmap v305: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-09-19T05:14:28.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:28 smithi096 ceph-mon[35085]: pgmap v305: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-09-19T05:14:28.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:28 smithi043 ceph-mon[30800]: pgmap v305: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-09-19T05:14:28.606 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T05:14:28.985 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:14:29.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.522+0000 7f44eae00640 1 -- 172.21.15.43:0/1833477927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44e4103ff0 msgr2=0x7f44e41043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:29.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.522+0000 7f44eae00640 1 --2- 172.21.15.43:0/1833477927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44e4103ff0 0x7f44e41043d0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f44d4009900 tx=0x7f44d402a3f0 comp rx=0 tx=0).stop 2024-09-19T05:14:29.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.522+0000 7f44eae00640 1 -- 172.21.15.43:0/1833477927 shutdown_connections 2024-09-19T05:14:29.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.522+0000 7f44eae00640 1 --2- 172.21.15.43:0/1833477927 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44e4109120 0x7f44e4110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:29.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.522+0000 7f44eae00640 1 --2- 172.21.15.43:0/1833477927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f44e41049a0 0x7f44e41089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:29.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.522+0000 7f44eae00640 1 --2- 172.21.15.43:0/1833477927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44e4103ff0 0x7f44e41043d0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:29.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.522+0000 7f44eae00640 1 -- 172.21.15.43:0/1833477927 >> 172.21.15.43:0/1833477927 conn(0x7f44e40ffa90 msgr2=0x7f44e4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:29.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.522+0000 7f44eae00640 1 -- 172.21.15.43:0/1833477927 shutdown_connections 2024-09-19T05:14:29.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.523+0000 7f44eae00640 1 -- 172.21.15.43:0/1833477927 wait complete. 2024-09-19T05:14:29.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.523+0000 7f44eae00640 1 Processor -- start 2024-09-19T05:14:29.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.524+0000 7f44eae00640 1 -- start start 2024-09-19T05:14:29.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.524+0000 7f44eae00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44e4103ff0 0x7f44e419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:29.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.524+0000 7f44eae00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f44e41049a0 0x7f44e419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:29.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.524+0000 7f44eae00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44e4109120 0x7f44e419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:29.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.524+0000 7f44eae00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f44e4114190 con 0x7f44e4109120 2024-09-19T05:14:29.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.524+0000 7f44eae00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f44e4114010 con 0x7f44e4103ff0 2024-09-19T05:14:29.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.524+0000 7f44eae00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f44e4114310 con 0x7f44e41049a0 2024-09-19T05:14:29.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.524+0000 7f44e3e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44e4103ff0 0x7f44e419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:29.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.524+0000 7f44e8e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44e4109120 0x7f44e419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:29.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.524+0000 7f44dbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f44e41049a0 0x7f44e419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:29.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.524+0000 7f44e3e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44e4103ff0 0x7f44e419a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:37590/0 (socket says 172.21.15.43:37590) 2024-09-19T05:14:29.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.524+0000 7f44e3e00640 1 -- 172.21.15.43:0/3224448084 learned_addr learned my addr 172.21.15.43:0/3224448084 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:29.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.525+0000 7f44e3e00640 1 -- 172.21.15.43:0/3224448084 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f44e41049a0 msgr2=0x7f44e419abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:29.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.525+0000 7f44e3e00640 1 --2- 172.21.15.43:0/3224448084 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f44e41049a0 0x7f44e419abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:29.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.525+0000 7f44e3e00640 1 -- 172.21.15.43:0/3224448084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44e4109120 msgr2=0x7f44e419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:29.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.525+0000 7f44e3e00640 1 --2- 172.21.15.43:0/3224448084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44e4109120 0x7f44e419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:29.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.525+0000 7f44e3e00640 1 -- 172.21.15.43:0/3224448084 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e419f640 con 0x7f44e4103ff0 2024-09-19T05:14:29.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.525+0000 7f44dbe00640 1 --2- 172.21.15.43:0/3224448084 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f44e41049a0 0x7f44e419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:29.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.525+0000 7f44e3e00640 1 --2- 172.21.15.43:0/3224448084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44e4103ff0 0x7f44e419a690 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f44d4004770 tx=0x7f44d4003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:29.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.525+0000 7f44e1600640 1 -- 172.21.15.43:0/3224448084 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d403f070 con 0x7f44e4103ff0 2024-09-19T05:14:29.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.525+0000 7f44eae00640 1 -- 172.21.15.43:0/3224448084 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44e419f8d0 con 0x7f44e4103ff0 2024-09-19T05:14:29.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.525+0000 7f44e1600640 1 -- 172.21.15.43:0/3224448084 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44d4031ca0 con 0x7f44e4103ff0 2024-09-19T05:14:29.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.525+0000 7f44e1600640 1 -- 172.21.15.43:0/3224448084 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d4035030 con 0x7f44e4103ff0 2024-09-19T05:14:29.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.526+0000 7f44eae00640 1 -- 172.21.15.43:0/3224448084 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44e41a7210 con 0x7f44e4103ff0 2024-09-19T05:14:29.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.529+0000 7f44eda25740 1 Processor -- start 2024-09-19T05:14:29.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.529+0000 7f44e1600640 1 -- 172.21.15.43:0/3224448084 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f44d4042070 con 0x7f44e4103ff0 2024-09-19T05:14:29.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.529+0000 7f44eda25740 1 -- start start 2024-09-19T05:14:29.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.529+0000 7f44eda25740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a3d1ecc70 0x561a3d1ed050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:29.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.529+0000 7f44eda25740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a3d1ed590 0x561a3d1f7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:29.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.529+0000 7f44eda25740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a3d1f7e50 0x561a3d1fa240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:29.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.529+0000 7f44eda25740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561a3d1533e0 con 0x561a3d1ecc70 2024-09-19T05:14:29.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.529+0000 7f44eda25740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561a3d0fad80 con 0x561a3d1ed590 2024-09-19T05:14:29.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.529+0000 7f44eda25740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561a3d14f3a0 con 0x561a3d1f7e50 2024-09-19T05:14:29.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.529+0000 7f44e8e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a3d1ecc70 0x561a3d1ed050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:29.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.529+0000 7f44e8e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a3d1ecc70 0x561a3d1ed050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44612/0 (socket says 172.21.15.43:44612) 2024-09-19T05:14:29.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.529+0000 7f44e8e00640 1 -- 172.21.15.43:0/3946575157 learned_addr learned my addr 172.21.15.43:0/3946575157 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:29.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.530+0000 7f44e3e00640 1 --2- 172.21.15.43:0/3946575157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a3d1ed590 0x561a3d1f7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:29.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.530+0000 7f44dbe00640 1 --2- 172.21.15.43:0/3946575157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a3d1f7e50 0x561a3d1fa240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:29.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.530+0000 7f44e8e00640 1 -- 172.21.15.43:0/3946575157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a3d1f7e50 msgr2=0x561a3d1fa240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:29.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.530+0000 7f44e8e00640 1 --2- 172.21.15.43:0/3946575157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a3d1f7e50 0x561a3d1fa240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:29.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.530+0000 7f44e8e00640 1 -- 172.21.15.43:0/3946575157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a3d1ed590 msgr2=0x561a3d1f7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:29.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.530+0000 7f44e8e00640 1 --2- 172.21.15.43:0/3946575157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a3d1ed590 0x561a3d1f7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:29.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.530+0000 7f44e8e00640 1 -- 172.21.15.43:0/3946575157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a3d119ae0 con 0x561a3d1ecc70 2024-09-19T05:14:29.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.530+0000 7f44e8e00640 1 --2- 172.21.15.43:0/3946575157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a3d1ecc70 0x561a3d1ed050 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f44dc009f50 tx=0x7f44dc0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26cf2617385f72d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:29.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.531+0000 7f44d9600640 1 -- 172.21.15.43:0/3946575157 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44dc004d20 con 0x561a3d1ecc70 2024-09-19T05:14:29.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.531+0000 7f44e1600640 1 --2- 172.21.15.43:0/3224448084 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f44c80a33b0 0x7f44c80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:29.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.531+0000 7f44d9600640 1 -- 172.21.15.43:0/3946575157 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44dc004050 con 0x561a3d1ecc70 2024-09-19T05:14:29.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.531+0000 7f44d9600640 1 -- 172.21.15.43:0/3946575157 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44dc005280 con 0x561a3d1ecc70 2024-09-19T05:14:29.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.531+0000 7f44e1600640 1 -- 172.21.15.43:0/3224448084 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f44d40eff90 con 0x7f44e4103ff0 2024-09-19T05:14:29.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.531+0000 7f44e3e00640 1 --2- 172.21.15.43:0/3224448084 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f44c80a33b0 0x7f44c80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:29.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.531+0000 7f44eda25740 1 -- 172.21.15.43:0/3946575157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a3d1ecc70 msgr2=0x561a3d1ed050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:29.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.531+0000 7f44eda25740 1 --2- 172.21.15.43:0/3946575157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a3d1ecc70 0x561a3d1ed050 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f44dc009f50 tx=0x7f44dc0118d0 comp rx=0 tx=0).stop 2024-09-19T05:14:29.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.532+0000 7f44e3e00640 1 --2- 172.21.15.43:0/3224448084 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f44c80a33b0 0x7f44c80a5870 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f44d40ed780 tx=0x7f44d40b8040 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:29.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.532+0000 7f44eda25740 1 -- 172.21.15.43:0/3946575157 shutdown_connections 2024-09-19T05:14:29.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.532+0000 7f44eda25740 1 --2- 172.21.15.43:0/3946575157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a3d1f7e50 0x561a3d1fa240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:29.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.532+0000 7f44eda25740 1 --2- 172.21.15.43:0/3946575157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a3d1ed590 0x561a3d1f7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:29.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.532+0000 7f44eda25740 1 --2- 172.21.15.43:0/3946575157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a3d1ecc70 0x561a3d1ed050 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:29.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.532+0000 7f44eda25740 1 -- 172.21.15.43:0/3946575157 >> 172.21.15.43:0/3946575157 conn(0x561a3d1e9200 msgr2=0x561a3d1e95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:29.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.532+0000 7f44eda25740 1 -- 172.21.15.43:0/3946575157 shutdown_connections 2024-09-19T05:14:29.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.532+0000 7f44eda25740 1 -- 172.21.15.43:0/3946575157 wait complete. 2024-09-19T05:14:29.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.532+0000 7f44eda25740 1 -- start start 2024-09-19T05:14:29.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.533+0000 7f44eda25740 1 Processor -- start 2024-09-19T05:14:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.533+0000 7f44eda25740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a3d1ecc70 0x561a3d1f4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.533+0000 7f44eda25740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a3d1ed590 0x561a3d1f51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.533+0000 7f44eda25740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a3d1f7e50 0x561a3d1f5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.533+0000 7f44eda25740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561a3d1faa20 con 0x561a3d1f7e50 2024-09-19T05:14:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.533+0000 7f44eda25740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561a3d10e0f0 con 0x561a3d1ecc70 2024-09-19T05:14:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.533+0000 7f44eda25740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561a3d1faba0 con 0x561a3d1ed590 2024-09-19T05:14:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.533+0000 7f44eda25740 10 client.0 _ll_register_callbacks cb 0x561a3d0f5bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.533+0000 7f44e8e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a3d1f7e50 0x561a3d1f5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.533+0000 7f44e8e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a3d1f7e50 0x561a3d1f5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44624/0 (socket says 172.21.15.43:44624) 2024-09-19T05:14:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.533+0000 7f44e8e00640 1 -- 172.21.15.43:0/1107770041 learned_addr learned my addr 172.21.15.43:0/1107770041 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.533+0000 7f44e8e00640 1 --2- 172.21.15.43:0/1107770041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a3d1ecc70 0x561a3d1f4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.533+0000 7f44dbe00640 1 --2- 172.21.15.43:0/1107770041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a3d1ed590 0x561a3d1f51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.534+0000 7f44e8e00640 1 -- 172.21.15.43:0/1107770041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a3d1ed590 msgr2=0x561a3d1f51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:29.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.534+0000 7f44e8e00640 1 --2- 172.21.15.43:0/1107770041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a3d1ed590 0x561a3d1f51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:29.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.534+0000 7f44e8e00640 1 -- 172.21.15.43:0/1107770041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a3d1ecc70 msgr2=0x561a3d1f4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:29.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.534+0000 7f44e8e00640 1 --2- 172.21.15.43:0/1107770041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a3d1ecc70 0x561a3d1f4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:29.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.534+0000 7f44e8e00640 1 -- 172.21.15.43:0/1107770041 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a3cec43c0 con 0x561a3d1f7e50 2024-09-19T05:14:29.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.534+0000 7f44e8e00640 1 --2- 172.21.15.43:0/1107770041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a3d1f7e50 0x561a3d1f5720 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f44dc009250 tx=0x7f44dc005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7835a493b2bb1255 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:29.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.534+0000 7f44d2000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:14:29.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.534+0000 7f44d2000640 1 -- 172.21.15.43:0/1107770041 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44dc02b020 con 0x561a3d1f7e50 2024-09-19T05:14:29.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.534+0000 7f44d2000640 1 -- 172.21.15.43:0/1107770041 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f44dc027640 con 0x561a3d1f7e50 2024-09-19T05:14:29.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.534+0000 7f44eda25740 1 -- 172.21.15.43:0/1107770041 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44dc004050 con 0x561a3d1f7e50 2024-09-19T05:14:29.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.535+0000 7f44d2000640 1 -- 172.21.15.43:0/1107770041 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f44dc036050 con 0x561a3d1f7e50 2024-09-19T05:14:29.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.535+0000 7f44eda25740 10 client.17535 fetch_fsmap learned FSMap version 29 2024-09-19T05:14:29.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.535+0000 7f44eda25740 1 -- 172.21.15.43:0/1107770041 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561a3d2069c0 con 0x561a3d1f7e50 2024-09-19T05:14:29.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.535+0000 7f44d2000640 1 -- 172.21.15.43:0/1107770041 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44dc004050 con 0x561a3d1f7e50 2024-09-19T05:14:29.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.535+0000 7f44d2000640 1 -- 172.21.15.43:0/1107770041 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f44dc00f760 con 0x561a3d1f7e50 2024-09-19T05:14:29.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.535+0000 7f44eda25740 10 client.17535 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:14:29.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.535+0000 7f44eda25740 10 client.17535 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T05:14:29.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.535+0000 7f44eda25740 20 client.17535 populate_metadata read hostname 'smithi043' 2024-09-19T05:14:29.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.535+0000 7f44eda25740 1 --2- 172.21.15.43:0/1107770041 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x561a3d207e20 0x561a3d20a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:29.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.535+0000 7f44eda25740 4 client.17535 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:14:29.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.535+0000 7f44eda25740 1 -- 172.21.15.43:0/1107770041 --> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a3d160950 con 0x561a3d207e20 2024-09-19T05:14:29.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.536+0000 7f44dbe00640 1 --2- 172.21.15.43:0/1107770041 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x561a3d207e20 0x561a3d20a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:29.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.536+0000 7f44dbe00640 1 --2- 172.21.15.43:0/1107770041 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x561a3d207e20 0x561a3d20a210 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=48aa7c6ebef5d493 server_cookie=6f8dcd174ef209f7 in_seq=0 out_seq=0 2024-09-19T05:14:29.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.536+0000 7f44d2000640 10 client.17535 ms_handle_connect on v2:172.21.15.96:6836/3978388286 2024-09-19T05:14:29.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.537+0000 7f44d2000640 1 -- 172.21.15.43:0/1107770041 <== mds.3 v2:172.21.15.96:6836/3978388286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561a3d160950 con 0x561a3d207e20 2024-09-19T05:14:29.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.537+0000 7f44d2000640 10 client.17535 handle_command_reply: tid=0 2024-09-19T05:14:29.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.537+0000 7f44eda25740 1 client.17535 shutdown 2024-09-19T05:14:29.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.537+0000 7f44eda25740 1 -- 172.21.15.43:0/1107770041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a3d1f7e50 msgr2=0x561a3d1f5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:29.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.537+0000 7f44eda25740 1 --2- 172.21.15.43:0/1107770041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a3d1f7e50 0x561a3d1f5720 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f44dc009250 tx=0x7f44dc005bd0 comp rx=0 tx=0).stop 2024-09-19T05:14:29.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.537+0000 7f44eda25740 1 -- 172.21.15.43:0/1107770041 shutdown_connections 2024-09-19T05:14:29.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.537+0000 7f44eda25740 1 --2- 172.21.15.43:0/1107770041 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x561a3d207e20 0x561a3d20a210 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:29.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.537+0000 7f44eda25740 1 --2- 172.21.15.43:0/1107770041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a3d1f7e50 0x561a3d1f5720 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:29.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.537+0000 7f44eda25740 1 --2- 172.21.15.43:0/1107770041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a3d1ed590 0x561a3d1f51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:29.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.537+0000 7f44d2000640 0 client.17535 ms_handle_reset on v2:172.21.15.96:6836/3978388286 2024-09-19T05:14:29.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.537+0000 7f44eda25740 1 --2- 172.21.15.43:0/1107770041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a3d1ecc70 0x561a3d1f4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:29.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.537+0000 7f44eda25740 1 -- 172.21.15.43:0/1107770041 >> 172.21.15.43:0/1107770041 conn(0x561a3d1e9200 msgr2=0x561a3d1e8380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:29.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.538+0000 7f44eda25740 1 -- 172.21.15.43:0/1107770041 shutdown_connections 2024-09-19T05:14:29.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.538+0000 7f44eda25740 1 -- 172.21.15.43:0/1107770041 wait complete. 2024-09-19T05:14:29.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.538+0000 7f44eda25740 20 client.17535 trim_cache size 0 max 16384 2024-09-19T05:14:29.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.552+0000 7f44eda25740 1 Processor -- start 2024-09-19T05:14:29.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.552+0000 7f44eda25740 1 -- start start 2024-09-19T05:14:29.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.552+0000 7f44eda25740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a3d1ecc70 0x561a3d20f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:29.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.552+0000 7f44eda25740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a3d1ed590 0x561a3d20fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:29.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.552+0000 7f44eda25740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a3d1f7e50 0x561a3d1f6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:29.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.552+0000 7f44eda25740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f44e41afb70 con 0x561a3d1f7e50 2024-09-19T05:14:29.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.552+0000 7f44eda25740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f44e41b16b0 con 0x561a3d1ed590 2024-09-19T05:14:29.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.552+0000 7f44eda25740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f44e41b0240 con 0x561a3d1ecc70 2024-09-19T05:14:29.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.553+0000 7f44dbe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a3d1ed590 0x561a3d20fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:29.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.553+0000 7f44e8e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a3d1ecc70 0x561a3d20f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:29.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.553+0000 7f44dbe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a3d1ed590 0x561a3d20fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:37616/0 (socket says 172.21.15.43:37616) 2024-09-19T05:14:29.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.553+0000 7f44dbe00640 1 -- 172.21.15.43:0/4230077988 learned_addr learned my addr 172.21.15.43:0/4230077988 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:29.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.553+0000 7f44e8e00640 1 --2- 172.21.15.43:0/4230077988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a3d1f7e50 0x561a3d1f6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:29.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.553+0000 7f44dbe00640 1 -- 172.21.15.43:0/4230077988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a3d1ecc70 msgr2=0x561a3d20f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:29.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.553+0000 7f44dbe00640 1 --2- 172.21.15.43:0/4230077988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a3d1ecc70 0x561a3d20f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:29.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.553+0000 7f44dbe00640 1 -- 172.21.15.43:0/4230077988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a3d1f7e50 msgr2=0x561a3d1f6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:29.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.553+0000 7f44dbe00640 1 --2- 172.21.15.43:0/4230077988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a3d1f7e50 0x561a3d1f6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:29.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.553+0000 7f44dbe00640 1 -- 172.21.15.43:0/4230077988 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a3d1e9030 con 0x561a3d1ed590 2024-09-19T05:14:29.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.553+0000 7f44e8e00640 1 --2- 172.21.15.43:0/4230077988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a3d1f7e50 0x561a3d1f6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:14:29.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.554+0000 7f44dbe00640 1 --2- 172.21.15.43:0/4230077988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a3d1ed590 0x561a3d20fee0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x561a3d1ea580 tx=0x7f44cc005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbb53448fc2545d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:29.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.554+0000 7f44d8c00640 1 -- 172.21.15.43:0/4230077988 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44cc022070 con 0x561a3d1ed590 2024-09-19T05:14:29.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.554+0000 7f44d8c00640 1 -- 172.21.15.43:0/4230077988 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44cc0047a0 con 0x561a3d1ed590 2024-09-19T05:14:29.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.554+0000 7f44d8c00640 1 -- 172.21.15.43:0/4230077988 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44cc01f960 con 0x561a3d1ed590 2024-09-19T05:14:29.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.554+0000 7f44eda25740 1 -- 172.21.15.43:0/4230077988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a3d1ed590 msgr2=0x561a3d20fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:29.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.554+0000 7f44eda25740 1 --2- 172.21.15.43:0/4230077988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a3d1ed590 0x561a3d20fee0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x561a3d1ea580 tx=0x7f44cc005f90 comp rx=0 tx=0).stop 2024-09-19T05:14:29.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.555+0000 7f44eda25740 1 -- 172.21.15.43:0/4230077988 shutdown_connections 2024-09-19T05:14:29.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.555+0000 7f44eda25740 1 --2- 172.21.15.43:0/4230077988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a3d1f7e50 0x561a3d1f6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:29.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.555+0000 7f44eda25740 1 --2- 172.21.15.43:0/4230077988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a3d1ed590 0x561a3d20fee0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:29.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.555+0000 7f44eda25740 1 --2- 172.21.15.43:0/4230077988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a3d1ecc70 0x561a3d20f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:29.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.555+0000 7f44eda25740 1 -- 172.21.15.43:0/4230077988 >> 172.21.15.43:0/4230077988 conn(0x561a3d1e9200 msgr2=0x561a3d1f3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:29.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.555+0000 7f44eda25740 1 -- 172.21.15.43:0/4230077988 shutdown_connections 2024-09-19T05:14:29.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.555+0000 7f44eda25740 1 -- 172.21.15.43:0/4230077988 wait complete. 2024-09-19T05:14:29.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.555+0000 7f44eda25740 1 -- start start 2024-09-19T05:14:29.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.555+0000 7f44eda25740 1 Processor -- start 2024-09-19T05:14:29.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.556+0000 7f44eda25740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a3d1ecc70 0x561a3d1f1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:29.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.556+0000 7f44eda25740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a3d1ed590 0x561a3d1f2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:29.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.556+0000 7f44eda25740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a3d1f7e50 0x561a3d1f2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:29.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.556+0000 7f44eda25740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f44e41b0fe0 con 0x561a3d1ed590 2024-09-19T05:14:29.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.556+0000 7f44eda25740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f44e41b0910 con 0x561a3d1ecc70 2024-09-19T05:14:29.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.556+0000 7f44eda25740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f44e41adb70 con 0x561a3d1f7e50 2024-09-19T05:14:29.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.556+0000 7f44eda25740 10 client.0 _ll_register_callbacks cb 0x7f44d40f39b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:29.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.556+0000 7f44dbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a3d1ed590 0x561a3d1f2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:29.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.556+0000 7f44e8e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a3d1ecc70 0x561a3d1f1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:29.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.556+0000 7f44e8e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a3d1ecc70 0x561a3d1f1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:37622/0 (socket says 172.21.15.43:37622) 2024-09-19T05:14:29.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.556+0000 7f44e8e00640 1 -- 172.21.15.43:0/1548763820 learned_addr learned my addr 172.21.15.43:0/1548763820 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:29.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.556+0000 7f44e8e00640 1 --2- 172.21.15.43:0/1548763820 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a3d1f7e50 0x561a3d1f2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:29.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.557+0000 7f44e8e00640 1 -- 172.21.15.43:0/1548763820 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a3d1f7e50 msgr2=0x561a3d1f2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:29.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.557+0000 7f44e8e00640 1 --2- 172.21.15.43:0/1548763820 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a3d1f7e50 0x561a3d1f2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:29.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.557+0000 7f44e8e00640 1 -- 172.21.15.43:0/1548763820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a3d1ed590 msgr2=0x561a3d1f2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:29.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.557+0000 7f44e8e00640 1 --2- 172.21.15.43:0/1548763820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a3d1ed590 0x561a3d1f2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:29.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.557+0000 7f44e8e00640 1 -- 172.21.15.43:0/1548763820 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a3d1e9e90 con 0x561a3d1ecc70 2024-09-19T05:14:29.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.557+0000 7f44dbe00640 1 --2- 172.21.15.43:0/1548763820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a3d1ed590 0x561a3d1f2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:29.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.557+0000 7f44e8e00640 1 --2- 172.21.15.43:0/1548763820 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a3d1ecc70 0x561a3d1f1de0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f44dc009d60 tx=0x7f44dc004780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83e9ea8bd4a581ad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:29.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.557+0000 7f44d2000640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:14:29.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.557+0000 7f44d2000640 1 -- 172.21.15.43:0/1548763820 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44dc02b050 con 0x561a3d1ecc70 2024-09-19T05:14:29.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.557+0000 7f44d2000640 1 -- 172.21.15.43:0/1548763820 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f44dc004560 con 0x561a3d1ecc70 2024-09-19T05:14:29.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.557+0000 7f44eda25740 1 -- 172.21.15.43:0/1548763820 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44cc0047a0 con 0x561a3d1ecc70 2024-09-19T05:14:29.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.557+0000 7f44d2000640 1 -- 172.21.15.43:0/1548763820 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f44dc00f480 con 0x561a3d1ecc70 2024-09-19T05:14:29.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.558+0000 7f44eda25740 10 client.27563 fetch_fsmap learned FSMap version 29 2024-09-19T05:14:29.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.558+0000 7f44eda25740 1 -- 172.21.15.43:0/1548763820 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561a3d1e4770 con 0x561a3d1ecc70 2024-09-19T05:14:29.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.558+0000 7f44d2000640 1 -- 172.21.15.43:0/1548763820 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44cc0047a0 con 0x561a3d1ecc70 2024-09-19T05:14:29.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.558+0000 7f44d2000640 1 -- 172.21.15.43:0/1548763820 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f44dc044b10 con 0x561a3d1ecc70 2024-09-19T05:14:29.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.558+0000 7f44eda25740 10 client.27563 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:14:29.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.558+0000 7f44eda25740 10 client.27563 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T05:14:29.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.558+0000 7f44eda25740 20 client.27563 populate_metadata read hostname 'smithi043' 2024-09-19T05:14:29.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.558+0000 7f44eda25740 1 --2- 172.21.15.43:0/1548763820 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x561a3d272fe0 0x561a3d275400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:29.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.558+0000 7f44eda25740 4 client.27563 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:14:29.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.558+0000 7f44eda25740 1 -- 172.21.15.43:0/1548763820 --> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561a3d15ae60 con 0x561a3d272fe0 2024-09-19T05:14:29.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.558+0000 7f44dbe00640 1 --2- 172.21.15.43:0/1548763820 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x561a3d272fe0 0x561a3d275400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:29.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.559+0000 7f44dbe00640 1 --2- 172.21.15.43:0/1548763820 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x561a3d272fe0 0x561a3d275400 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2ef396143837db3 server_cookie=6e0c03c7f56cfb2 in_seq=0 out_seq=0 2024-09-19T05:14:29.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.559+0000 7f44d2000640 10 client.27563 ms_handle_connect on v2:172.21.15.96:6836/3978388286 2024-09-19T05:14:29.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.559+0000 7f44d2000640 1 -- 172.21.15.43:0/1548763820 <== mds.3 v2:172.21.15.96:6836/3978388286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561a3d15ae60 con 0x561a3d272fe0 2024-09-19T05:14:29.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.559+0000 7f44d2000640 10 client.27563 handle_command_reply: tid=0 2024-09-19T05:14:29.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.559+0000 7f44eda25740 1 client.27563 shutdown 2024-09-19T05:14:29.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.559+0000 7f44eda25740 1 -- 172.21.15.43:0/1548763820 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a3d1ecc70 msgr2=0x561a3d1f1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:29.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.559+0000 7f44eda25740 1 --2- 172.21.15.43:0/1548763820 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a3d1ecc70 0x561a3d1f1de0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f44dc009d60 tx=0x7f44dc004780 comp rx=0 tx=0).stop 2024-09-19T05:14:29.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.559+0000 7f44eda25740 1 -- 172.21.15.43:0/1548763820 shutdown_connections 2024-09-19T05:14:29.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.559+0000 7f44eda25740 1 --2- 172.21.15.43:0/1548763820 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x561a3d272fe0 0x561a3d275400 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:29.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.559+0000 7f44eda25740 1 --2- 172.21.15.43:0/1548763820 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a3d1f7e50 0x561a3d1f2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:29.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.559+0000 7f44eda25740 1 --2- 172.21.15.43:0/1548763820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a3d1ed590 0x561a3d1f2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:29.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.559+0000 7f44eda25740 1 --2- 172.21.15.43:0/1548763820 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a3d1ecc70 0x561a3d1f1de0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:29.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.559+0000 7f44eda25740 1 -- 172.21.15.43:0/1548763820 >> 172.21.15.43:0/1548763820 conn(0x561a3d1e9200 msgr2=0x561a3d1f6500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:29.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.559+0000 7f44d2000640 0 client.27563 ms_handle_reset on v2:172.21.15.96:6836/3978388286 2024-09-19T05:14:29.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.560+0000 7f44eda25740 1 -- 172.21.15.43:0/1548763820 shutdown_connections 2024-09-19T05:14:29.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.560+0000 7f44eda25740 1 -- 172.21.15.43:0/1548763820 wait complete. 2024-09-19T05:14:29.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.560+0000 7f44eda25740 20 client.27563 trim_cache size 0 max 16384 2024-09-19T05:14:29.574 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:14:29.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.560+0000 7f44eae00640 1 -- 172.21.15.43:0/3224448084 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f44c80a33b0 msgr2=0x7f44c80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:29.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.560+0000 7f44eae00640 1 --2- 172.21.15.43:0/3224448084 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f44c80a33b0 0x7f44c80a5870 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f44d40ed780 tx=0x7f44d40b8040 comp rx=0 tx=0).stop 2024-09-19T05:14:29.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.560+0000 7f44eae00640 1 -- 172.21.15.43:0/3224448084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44e4103ff0 msgr2=0x7f44e419a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:29.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.560+0000 7f44eae00640 1 --2- 172.21.15.43:0/3224448084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44e4103ff0 0x7f44e419a690 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f44d4004770 tx=0x7f44d4003c10 comp rx=0 tx=0).stop 2024-09-19T05:14:29.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.561+0000 7f44eae00640 1 -- 172.21.15.43:0/3224448084 shutdown_connections 2024-09-19T05:14:29.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.561+0000 7f44eae00640 1 --2- 172.21.15.43:0/3224448084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44e4109120 0x7f44e419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:29.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.561+0000 7f44eae00640 1 --2- 172.21.15.43:0/3224448084 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f44e41049a0 0x7f44e419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:29.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.561+0000 7f44eae00640 1 --2- 172.21.15.43:0/3224448084 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f44c80a33b0 0x7f44c80a5870 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:29.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.561+0000 7f44eae00640 1 --2- 172.21.15.43:0/3224448084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44e4103ff0 0x7f44e419a690 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:29.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.561+0000 7f44eae00640 1 -- 172.21.15.43:0/3224448084 >> 172.21.15.43:0/3224448084 conn(0x7f44e40ffa90 msgr2=0x7f44e4100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:29.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.561+0000 7f44eae00640 1 -- 172.21.15.43:0/3224448084 shutdown_connections 2024-09-19T05:14:29.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:29.561+0000 7f44eae00640 1 -- 172.21.15.43:0/3224448084 wait complete. 2024-09-19T05:14:30.263 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:30 smithi043 ceph-mon[30800]: pgmap v306: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:14:30.303 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T05:14:30.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:30 smithi151 ceph-mon[35425]: pgmap v306: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:14:30.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:30 smithi096 ceph-mon[35085]: pgmap v306: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:14:30.681 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:14:31.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.239+0000 7f37b6600640 1 -- 172.21.15.43:0/4135553583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37b0109120 msgr2=0x7f37b0110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:31.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.239+0000 7f37b6600640 1 --2- 172.21.15.43:0/4135553583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37b0109120 0x7f37b0110ca0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f37a8009ca0 tx=0x7f37a802a8c0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.239+0000 7f37b6600640 1 -- 172.21.15.43:0/4135553583 shutdown_connections 2024-09-19T05:14:31.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.239+0000 7f37b6600640 1 --2- 172.21.15.43:0/4135553583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37b0109120 0x7f37b0110ca0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.239+0000 7f37b6600640 1 --2- 172.21.15.43:0/4135553583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f37b01049a0 0x7f37b01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.239+0000 7f37b6600640 1 --2- 172.21.15.43:0/4135553583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37b0103ff0 0x7f37b01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.239+0000 7f37b6600640 1 -- 172.21.15.43:0/4135553583 >> 172.21.15.43:0/4135553583 conn(0x7f37b00ffa90 msgr2=0x7f37b0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:31.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.239+0000 7f37b6600640 1 -- 172.21.15.43:0/4135553583 shutdown_connections 2024-09-19T05:14:31.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.239+0000 7f37b6600640 1 -- 172.21.15.43:0/4135553583 wait complete. 2024-09-19T05:14:31.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.240+0000 7f37b6600640 1 Processor -- start 2024-09-19T05:14:31.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.240+0000 7f37b6600640 1 -- start start 2024-09-19T05:14:31.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.241+0000 7f37b6600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37b0103ff0 0x7f37b019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:31.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.241+0000 7f37b6600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37b01049a0 0x7f37b019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:31.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.241+0000 7f37b6600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f37b0109120 0x7f37b019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:31.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.241+0000 7f37b6600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f37b01141a0 con 0x7f37b01049a0 2024-09-19T05:14:31.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.241+0000 7f37b6600640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f37b0114020 con 0x7f37b0109120 2024-09-19T05:14:31.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.241+0000 7f37b6600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f37b0114320 con 0x7f37b0103ff0 2024-09-19T05:14:31.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.241+0000 7f37aea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37b01049a0 0x7f37b019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:31.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.241+0000 7f37aea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37b01049a0 0x7f37b019aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44652/0 (socket says 172.21.15.43:44652) 2024-09-19T05:14:31.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.241+0000 7f37afe00640 1 --2- 172.21.15.43:0/2632558426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f37b0109120 0x7f37b019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:31.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.241+0000 7f37aea00640 1 -- 172.21.15.43:0/2632558426 learned_addr learned my addr 172.21.15.43:0/2632558426 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:31.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.241+0000 7f37af400640 1 --2- 172.21.15.43:0/2632558426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37b0103ff0 0x7f37b019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:31.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.242+0000 7f37afe00640 1 -- 172.21.15.43:0/2632558426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37b0103ff0 msgr2=0x7f37b019a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:31.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.242+0000 7f37afe00640 1 --2- 172.21.15.43:0/2632558426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37b0103ff0 0x7f37b019a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.242+0000 7f37afe00640 1 -- 172.21.15.43:0/2632558426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37b01049a0 msgr2=0x7f37b019aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:31.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.242+0000 7f37afe00640 1 --2- 172.21.15.43:0/2632558426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37b01049a0 0x7f37b019aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.242+0000 7f37afe00640 1 -- 172.21.15.43:0/2632558426 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37b019f610 con 0x7f37b0109120 2024-09-19T05:14:31.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.242+0000 7f37aea00640 1 --2- 172.21.15.43:0/2632558426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37b01049a0 0x7f37b019aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:31.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.242+0000 7f37afe00640 1 --2- 172.21.15.43:0/2632558426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f37b0109120 0x7f37b019ef30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f37a8005e30 tx=0x7f37a8003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:31.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.242+0000 7f37a7e00640 1 -- 172.21.15.43:0/2632558426 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a8031650 con 0x7f37b0109120 2024-09-19T05:14:31.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.242+0000 7f37b6600640 1 -- 172.21.15.43:0/2632558426 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37b019f8a0 con 0x7f37b0109120 2024-09-19T05:14:31.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.242+0000 7f37a7e00640 1 -- 172.21.15.43:0/2632558426 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37a80317f0 con 0x7f37b0109120 2024-09-19T05:14:31.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.242+0000 7f37a7e00640 1 -- 172.21.15.43:0/2632558426 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a802f990 con 0x7f37b0109120 2024-09-19T05:14:31.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.242+0000 7f37b6600640 1 -- 172.21.15.43:0/2632558426 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37b01a71e0 con 0x7f37b0109120 2024-09-19T05:14:31.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.245+0000 7f37b9036740 1 Processor -- start 2024-09-19T05:14:31.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.245+0000 7f37a7e00640 1 -- 172.21.15.43:0/2632558426 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f37a802fb50 con 0x7f37b0109120 2024-09-19T05:14:31.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.246+0000 7f37b9036740 1 -- start start 2024-09-19T05:14:31.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.246+0000 7f37b9036740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1f7d09c70 0x55e1f7d0a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:31.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.246+0000 7f37b9036740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1f7d0a590 0x55e1f7d14910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:31.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.246+0000 7f37b9036740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1f7d14e50 0x55e1f7d17240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:31.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.246+0000 7f37b9036740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e1f7c703e0 con 0x55e1f7d0a590 2024-09-19T05:14:31.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.246+0000 7f37b9036740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e1f7c17d80 con 0x55e1f7d14e50 2024-09-19T05:14:31.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.246+0000 7f37b9036740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e1f7c6c3a0 con 0x55e1f7d09c70 2024-09-19T05:14:31.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.247+0000 7f37af400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1f7d0a590 0x55e1f7d14910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:31.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.247+0000 7f37aea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1f7d09c70 0x55e1f7d0a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:31.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.247+0000 7f37aea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1f7d09c70 0x55e1f7d0a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:43314/0 (socket says 172.21.15.43:43314) 2024-09-19T05:14:31.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.247+0000 7f37aea00640 1 -- 172.21.15.43:0/3085379595 learned_addr learned my addr 172.21.15.43:0/3085379595 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:31.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.247+0000 7f37af400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1f7d0a590 0x55e1f7d14910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44666/0 (socket says 172.21.15.43:44666) 2024-09-19T05:14:31.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.247+0000 7f37aea00640 1 --2- 172.21.15.43:0/3085379595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1f7d14e50 0x55e1f7d17240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:31.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.247+0000 7f37af400640 1 -- 172.21.15.43:0/3085379595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1f7d09c70 msgr2=0x55e1f7d0a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:31.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.247+0000 7f37af400640 1 --2- 172.21.15.43:0/3085379595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1f7d09c70 0x55e1f7d0a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.247+0000 7f37af400640 1 -- 172.21.15.43:0/3085379595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1f7d14e50 msgr2=0x55e1f7d17240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:31.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.247+0000 7f37af400640 1 --2- 172.21.15.43:0/3085379595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1f7d14e50 0x55e1f7d17240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.247+0000 7f37af400640 1 -- 172.21.15.43:0/3085379595 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1f7c36ae0 con 0x55e1f7d0a590 2024-09-19T05:14:31.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.247+0000 7f37aea00640 1 --2- 172.21.15.43:0/3085379595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1f7d09c70 0x55e1f7d0a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:31.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.247+0000 7f37a7e00640 1 --2- 172.21.15.43:0/2632558426 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f37940a33b0 0x7f37940a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:31.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.247+0000 7f37af400640 1 --2- 172.21.15.43:0/3085379595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1f7d0a590 0x55e1f7d14910 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f37a0009c00 tx=0x7f37a0011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5c4c58c1bfa75b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:31.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.247+0000 7f37a7e00640 1 -- 172.21.15.43:0/2632558426 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f37a80f0e30 con 0x7f37b0109120 2024-09-19T05:14:31.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.248+0000 7f37aea00640 1 --2- 172.21.15.43:0/2632558426 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f37940a33b0 0x7f37940a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:31.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.248+0000 7f37a4c00640 1 -- 172.21.15.43:0/3085379595 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a0004310 con 0x55e1f7d0a590 2024-09-19T05:14:31.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.248+0000 7f37a4c00640 1 -- 172.21.15.43:0/3085379595 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37a00044b0 con 0x55e1f7d0a590 2024-09-19T05:14:31.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.248+0000 7f37a4c00640 1 -- 172.21.15.43:0/3085379595 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a00035f0 con 0x55e1f7d0a590 2024-09-19T05:14:31.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.248+0000 7f37b9036740 1 -- 172.21.15.43:0/3085379595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1f7d0a590 msgr2=0x55e1f7d14910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:31.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.248+0000 7f37b9036740 1 --2- 172.21.15.43:0/3085379595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1f7d0a590 0x55e1f7d14910 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f37a0009c00 tx=0x7f37a0011440 comp rx=0 tx=0).stop 2024-09-19T05:14:31.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.248+0000 7f37aea00640 1 --2- 172.21.15.43:0/2632558426 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f37940a33b0 0x7f37940a5870 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x55e1f7d07580 tx=0x7f379c013040 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:31.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.249+0000 7f37b9036740 1 -- 172.21.15.43:0/3085379595 shutdown_connections 2024-09-19T05:14:31.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.249+0000 7f37b9036740 1 --2- 172.21.15.43:0/3085379595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1f7d14e50 0x55e1f7d17240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.249+0000 7f37b9036740 1 --2- 172.21.15.43:0/3085379595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1f7d0a590 0x55e1f7d14910 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.249+0000 7f37b9036740 1 --2- 172.21.15.43:0/3085379595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1f7d09c70 0x55e1f7d0a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.249+0000 7f37b9036740 1 -- 172.21.15.43:0/3085379595 >> 172.21.15.43:0/3085379595 conn(0x55e1f7d06200 msgr2=0x55e1f7d065e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:31.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.249+0000 7f37b9036740 1 -- 172.21.15.43:0/3085379595 shutdown_connections 2024-09-19T05:14:31.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.249+0000 7f37b9036740 1 -- 172.21.15.43:0/3085379595 wait complete. 2024-09-19T05:14:31.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.249+0000 7f37b9036740 1 -- start start 2024-09-19T05:14:31.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.249+0000 7f37b9036740 1 Processor -- start 2024-09-19T05:14:31.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.250+0000 7f37b9036740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1f7d09c70 0x55e1f7d11d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:31.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.250+0000 7f37b9036740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1f7d0a590 0x55e1f7d12260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:31.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.250+0000 7f37b9036740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1f7d14e50 0x55e1f7d127a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:31.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.250+0000 7f37b9036740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e1f7d17a20 con 0x55e1f7d14e50 2024-09-19T05:14:31.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.250+0000 7f37b9036740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e1f7c2b0f0 con 0x55e1f7d09c70 2024-09-19T05:14:31.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.250+0000 7f37b9036740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e1f7d17ba0 con 0x55e1f7d0a590 2024-09-19T05:14:31.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.250+0000 7f37b9036740 10 client.0 _ll_register_callbacks cb 0x55e1f7c12bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:31.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.250+0000 7f37afe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1f7d14e50 0x55e1f7d127a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:31.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.250+0000 7f37aea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1f7d0a590 0x55e1f7d12260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:31.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.250+0000 7f37aea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1f7d0a590 0x55e1f7d12260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:43324/0 (socket says 172.21.15.43:43324) 2024-09-19T05:14:31.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.250+0000 7f37aea00640 1 -- 172.21.15.43:0/3569756889 learned_addr learned my addr 172.21.15.43:0/3569756889 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:31.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.250+0000 7f37af400640 1 --2- 172.21.15.43:0/3569756889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1f7d09c70 0x55e1f7d11d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:31.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.251+0000 7f37aea00640 1 -- 172.21.15.43:0/3569756889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1f7d09c70 msgr2=0x55e1f7d11d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:31.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.251+0000 7f37aea00640 1 --2- 172.21.15.43:0/3569756889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1f7d09c70 0x55e1f7d11d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.251+0000 7f37aea00640 1 -- 172.21.15.43:0/3569756889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1f7d14e50 msgr2=0x55e1f7d127a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:31.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.251+0000 7f37aea00640 1 --2- 172.21.15.43:0/3569756889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1f7d14e50 0x55e1f7d127a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.251+0000 7f37aea00640 1 -- 172.21.15.43:0/3569756889 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1f79e13c0 con 0x55e1f7d0a590 2024-09-19T05:14:31.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.251+0000 7f37afe00640 1 --2- 172.21.15.43:0/3569756889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1f7d14e50 0x55e1f7d127a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:31.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.251+0000 7f37aea00640 1 --2- 172.21.15.43:0/3569756889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1f7d0a590 0x55e1f7d12260 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f379c021730 tx=0x7f379c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=132488813a6a0f39 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:31.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.251+0000 7f3799600640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:14:31.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.251+0000 7f3799600640 1 -- 172.21.15.43:0/3569756889 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379c01fb60 con 0x55e1f7d0a590 2024-09-19T05:14:31.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.251+0000 7f37b9036740 1 -- 172.21.15.43:0/3569756889 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37a00044b0 con 0x55e1f7d0a590 2024-09-19T05:14:31.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.251+0000 7f3799600640 1 -- 172.21.15.43:0/3569756889 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f379c01fd00 con 0x55e1f7d0a590 2024-09-19T05:14:31.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.251+0000 7f3799600640 1 -- 172.21.15.43:0/3569756889 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f379c034470 con 0x55e1f7d0a590 2024-09-19T05:14:31.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.252+0000 7f37b9036740 10 client.27475 fetch_fsmap learned FSMap version 29 2024-09-19T05:14:31.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.252+0000 7f37b9036740 1 -- 172.21.15.43:0/3569756889 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e1f7d23af0 con 0x55e1f7d0a590 2024-09-19T05:14:31.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.252+0000 7f3799600640 1 -- 172.21.15.43:0/3569756889 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37a00044b0 con 0x55e1f7d0a590 2024-09-19T05:14:31.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.252+0000 7f3799600640 1 -- 172.21.15.43:0/3569756889 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f379c00e3b0 con 0x55e1f7d0a590 2024-09-19T05:14:31.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.252+0000 7f37b9036740 10 client.27475 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:14:31.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.252+0000 7f37b9036740 10 client.27475 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T05:14:31.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.252+0000 7f37b9036740 20 client.27475 populate_metadata read hostname 'smithi043' 2024-09-19T05:14:31.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.252+0000 7f37b9036740 1 --2- 172.21.15.43:0/3569756889 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55e1f7d24ff0 0x55e1f7d273e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:31.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.252+0000 7f37b9036740 4 client.27475 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:14:31.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.252+0000 7f37b9036740 1 -- 172.21.15.43:0/3569756889 --> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1f7c7d950 con 0x55e1f7d24ff0 2024-09-19T05:14:31.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.253+0000 7f37af400640 1 --2- 172.21.15.43:0/3569756889 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55e1f7d24ff0 0x55e1f7d273e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:31.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.253+0000 7f37af400640 1 --2- 172.21.15.43:0/3569756889 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55e1f7d24ff0 0x55e1f7d273e0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=17a2e4462fe669c7 server_cookie=e301850a65884de0 in_seq=0 out_seq=0 2024-09-19T05:14:31.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.253+0000 7f3799600640 10 client.27475 ms_handle_connect on v2:172.21.15.151:6838/3794104313 2024-09-19T05:14:31.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.254+0000 7f3799600640 1 -- 172.21.15.43:0/3569756889 <== mds.2 v2:172.21.15.151:6838/3794104313 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e1f7c7d950 con 0x55e1f7d24ff0 2024-09-19T05:14:31.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.254+0000 7f3799600640 10 client.27475 handle_command_reply: tid=0 2024-09-19T05:14:31.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.254+0000 7f37b9036740 1 client.27475 shutdown 2024-09-19T05:14:31.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.254+0000 7f37b9036740 1 -- 172.21.15.43:0/3569756889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1f7d0a590 msgr2=0x55e1f7d12260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:31.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.254+0000 7f37b9036740 1 --2- 172.21.15.43:0/3569756889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1f7d0a590 0x55e1f7d12260 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f379c021730 tx=0x7f379c005f90 comp rx=0 tx=0).stop 2024-09-19T05:14:31.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.254+0000 7f37b9036740 1 -- 172.21.15.43:0/3569756889 shutdown_connections 2024-09-19T05:14:31.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.254+0000 7f37b9036740 1 --2- 172.21.15.43:0/3569756889 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55e1f7d24ff0 0x55e1f7d273e0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.254+0000 7f37b9036740 1 --2- 172.21.15.43:0/3569756889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1f7d14e50 0x55e1f7d127a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.254+0000 7f37b9036740 1 --2- 172.21.15.43:0/3569756889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1f7d0a590 0x55e1f7d12260 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.254+0000 7f37b9036740 1 --2- 172.21.15.43:0/3569756889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1f7d09c70 0x55e1f7d11d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.254+0000 7f3799600640 0 client.27475 ms_handle_reset on v2:172.21.15.151:6838/3794104313 2024-09-19T05:14:31.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.254+0000 7f37b9036740 1 -- 172.21.15.43:0/3569756889 >> 172.21.15.43:0/3569756889 conn(0x55e1f7d06200 msgr2=0x55e1f7d0bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:31.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.254+0000 7f37b9036740 1 -- 172.21.15.43:0/3569756889 shutdown_connections 2024-09-19T05:14:31.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.255+0000 7f37b9036740 1 -- 172.21.15.43:0/3569756889 wait complete. 2024-09-19T05:14:31.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.255+0000 7f37b9036740 20 client.27475 trim_cache size 0 max 16384 2024-09-19T05:14:31.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.269+0000 7f37b9036740 1 Processor -- start 2024-09-19T05:14:31.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.269+0000 7f37b9036740 1 -- start start 2024-09-19T05:14:31.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.269+0000 7f37b9036740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1f7d09c70 0x55e1f7d13c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:31.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.269+0000 7f37b9036740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1f7d0a590 0x55e1f7d2cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:31.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.269+0000 7f37b9036740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1f7d14e50 0x55e1f7d2d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:31.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.269+0000 7f37b9036740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f37b01afa90 con 0x55e1f7d0a590 2024-09-19T05:14:31.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.269+0000 7f37b9036740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f37b01b15d0 con 0x55e1f7d14e50 2024-09-19T05:14:31.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.269+0000 7f37b9036740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f37b01b0160 con 0x55e1f7d09c70 2024-09-19T05:14:31.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.269+0000 7f37aea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1f7d0a590 0x55e1f7d2cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:31.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.269+0000 7f37afe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1f7d14e50 0x55e1f7d2d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:31.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.269+0000 7f37af400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1f7d09c70 0x55e1f7d13c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:31.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.270+0000 7f37afe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1f7d14e50 0x55e1f7d2d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:37658/0 (socket says 172.21.15.43:37658) 2024-09-19T05:14:31.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.270+0000 7f37afe00640 1 -- 172.21.15.43:0/485206517 learned_addr learned my addr 172.21.15.43:0/485206517 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:31.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.270+0000 7f37aea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1f7d0a590 0x55e1f7d2cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44682/0 (socket says 172.21.15.43:44682) 2024-09-19T05:14:31.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.270+0000 7f37afe00640 1 -- 172.21.15.43:0/485206517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1f7d09c70 msgr2=0x55e1f7d13c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:31.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.270+0000 7f37afe00640 1 --2- 172.21.15.43:0/485206517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1f7d09c70 0x55e1f7d13c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.270+0000 7f37afe00640 1 -- 172.21.15.43:0/485206517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1f7d0a590 msgr2=0x55e1f7d2cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:31.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.270+0000 7f37afe00640 1 --2- 172.21.15.43:0/485206517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1f7d0a590 0x55e1f7d2cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.270+0000 7f37afe00640 1 -- 172.21.15.43:0/485206517 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1f7d279c0 con 0x55e1f7d14e50 2024-09-19T05:14:31.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.270+0000 7f37afe00640 1 --2- 172.21.15.43:0/485206517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1f7d14e50 0x55e1f7d2d130 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f37a80b9b80 tx=0x7f37a80dd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ef2e2c65c41d456 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:31.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.270+0000 7f379be00640 1 -- 172.21.15.43:0/485206517 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a80d5070 con 0x55e1f7d14e50 2024-09-19T05:14:31.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.270+0000 7f379be00640 1 -- 172.21.15.43:0/485206517 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37a8036050 con 0x55e1f7d14e50 2024-09-19T05:14:31.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.270+0000 7f379be00640 1 -- 172.21.15.43:0/485206517 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a80bb4b0 con 0x55e1f7d14e50 2024-09-19T05:14:31.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.271+0000 7f37b9036740 1 -- 172.21.15.43:0/485206517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1f7d14e50 msgr2=0x55e1f7d2d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:31.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.271+0000 7f37b9036740 1 --2- 172.21.15.43:0/485206517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1f7d14e50 0x55e1f7d2d130 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f37a80b9b80 tx=0x7f37a80dd040 comp rx=0 tx=0).stop 2024-09-19T05:14:31.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.271+0000 7f37b9036740 1 -- 172.21.15.43:0/485206517 shutdown_connections 2024-09-19T05:14:31.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.271+0000 7f37b9036740 1 --2- 172.21.15.43:0/485206517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1f7d14e50 0x55e1f7d2d130 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.271+0000 7f37b9036740 1 --2- 172.21.15.43:0/485206517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1f7d0a590 0x55e1f7d2cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.271+0000 7f37b9036740 1 --2- 172.21.15.43:0/485206517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1f7d09c70 0x55e1f7d13c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.271+0000 7f37b9036740 1 -- 172.21.15.43:0/485206517 >> 172.21.15.43:0/485206517 conn(0x55e1f7d06200 msgr2=0x55e1f7d13580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:31.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.271+0000 7f37b9036740 1 -- 172.21.15.43:0/485206517 shutdown_connections 2024-09-19T05:14:31.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.271+0000 7f37b9036740 1 -- 172.21.15.43:0/485206517 wait complete. 2024-09-19T05:14:31.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.272+0000 7f37b9036740 1 -- start start 2024-09-19T05:14:31.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.272+0000 7f37b9036740 1 Processor -- start 2024-09-19T05:14:31.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.272+0000 7f37b9036740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1f7d09c70 0x55e1f7d18d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:31.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.272+0000 7f37b9036740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1f7d0a590 0x55e1f7d19270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:31.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.272+0000 7f37b9036740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1f7d14e50 0x55e1f7d197b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:31.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.272+0000 7f37b9036740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f37b01b0f00 con 0x55e1f7d14e50 2024-09-19T05:14:31.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.272+0000 7f37b9036740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f37b01b0830 con 0x55e1f7d09c70 2024-09-19T05:14:31.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.272+0000 7f37b9036740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f37b01ada90 con 0x55e1f7d0a590 2024-09-19T05:14:31.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.272+0000 7f37b9036740 10 client.0 _ll_register_callbacks cb 0x7f379c009cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:31.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.273+0000 7f37afe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1f7d14e50 0x55e1f7d197b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:31.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.273+0000 7f37af400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1f7d09c70 0x55e1f7d18d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:31.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.273+0000 7f37afe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1f7d14e50 0x55e1f7d197b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44688/0 (socket says 172.21.15.43:44688) 2024-09-19T05:14:31.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.273+0000 7f37afe00640 1 -- 172.21.15.43:0/113519528 learned_addr learned my addr 172.21.15.43:0/113519528 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:31.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.273+0000 7f37aea00640 1 --2- 172.21.15.43:0/113519528 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1f7d0a590 0x55e1f7d19270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:31.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.273+0000 7f37af400640 1 -- 172.21.15.43:0/113519528 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1f7d0a590 msgr2=0x55e1f7d19270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:31.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.273+0000 7f37af400640 1 --2- 172.21.15.43:0/113519528 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1f7d0a590 0x55e1f7d19270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.273+0000 7f37af400640 1 -- 172.21.15.43:0/113519528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1f7d14e50 msgr2=0x55e1f7d197b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:31.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.273+0000 7f37af400640 1 --2- 172.21.15.43:0/113519528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1f7d14e50 0x55e1f7d197b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.273+0000 7f37af400640 1 -- 172.21.15.43:0/113519528 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1f7d0cae0 con 0x55e1f7d09c70 2024-09-19T05:14:31.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.273+0000 7f37afe00640 1 --2- 172.21.15.43:0/113519528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1f7d14e50 0x55e1f7d197b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:31.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.273+0000 7f37af400640 1 --2- 172.21.15.43:0/113519528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1f7d09c70 0x55e1f7d18d30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f37a0011940 tx=0x7f37a00098e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30fe20eb344e7a7b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:31.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.273+0000 7f3799600640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:14:31.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.274+0000 7f3799600640 1 -- 172.21.15.43:0/113519528 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a0034060 con 0x55e1f7d09c70 2024-09-19T05:14:31.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.274+0000 7f3799600640 1 -- 172.21.15.43:0/113519528 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f37a0005310 con 0x55e1f7d09c70 2024-09-19T05:14:31.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.274+0000 7f37b9036740 1 -- 172.21.15.43:0/113519528 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37a8036050 con 0x55e1f7d09c70 2024-09-19T05:14:31.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.274+0000 7f3799600640 1 -- 172.21.15.43:0/113519528 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f37a0005510 con 0x55e1f7d09c70 2024-09-19T05:14:31.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.274+0000 7f37b9036740 10 client.27581 fetch_fsmap learned FSMap version 29 2024-09-19T05:14:31.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.274+0000 7f37b9036740 1 -- 172.21.15.43:0/113519528 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e1f7d06e90 con 0x55e1f7d09c70 2024-09-19T05:14:31.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.274+0000 7f3799600640 1 -- 172.21.15.43:0/113519528 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37a8036050 con 0x55e1f7d09c70 2024-09-19T05:14:31.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.275+0000 7f3799600640 1 -- 172.21.15.43:0/113519528 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f37a0018070 con 0x55e1f7d09c70 2024-09-19T05:14:31.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.275+0000 7f37b9036740 10 client.27581 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:14:31.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.275+0000 7f37b9036740 10 client.27581 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T05:14:31.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.275+0000 7f37b9036740 20 client.27581 populate_metadata read hostname 'smithi043' 2024-09-19T05:14:31.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.275+0000 7f37b9036740 1 --2- 172.21.15.43:0/113519528 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55e1f7d90270 0x55e1f7d92660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:31.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.275+0000 7f37b9036740 4 client.27581 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:14:31.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.275+0000 7f37b9036740 1 -- 172.21.15.43:0/113519528 --> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e1f7c77e60 con 0x55e1f7d90270 2024-09-19T05:14:31.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.275+0000 7f37aea00640 1 --2- 172.21.15.43:0/113519528 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55e1f7d90270 0x55e1f7d92660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:31.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.276+0000 7f37aea00640 1 --2- 172.21.15.43:0/113519528 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55e1f7d90270 0x55e1f7d92660 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=840682257e7d224f server_cookie=8b4b96a2f1f14e8b in_seq=0 out_seq=0 2024-09-19T05:14:31.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.276+0000 7f3799600640 10 client.27581 ms_handle_connect on v2:172.21.15.151:6838/3794104313 2024-09-19T05:14:31.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.276+0000 7f3799600640 1 -- 172.21.15.43:0/113519528 <== mds.2 v2:172.21.15.151:6838/3794104313 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e1f7c77e60 con 0x55e1f7d90270 2024-09-19T05:14:31.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.276+0000 7f3799600640 10 client.27581 handle_command_reply: tid=0 2024-09-19T05:14:31.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.276+0000 7f37b9036740 1 client.27581 shutdown 2024-09-19T05:14:31.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.276+0000 7f37b9036740 1 -- 172.21.15.43:0/113519528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1f7d09c70 msgr2=0x55e1f7d18d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:31.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.276+0000 7f37b9036740 1 --2- 172.21.15.43:0/113519528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1f7d09c70 0x55e1f7d18d30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f37a0011940 tx=0x7f37a00098e0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.276+0000 7f37b9036740 1 -- 172.21.15.43:0/113519528 shutdown_connections 2024-09-19T05:14:31.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.276+0000 7f37b9036740 1 --2- 172.21.15.43:0/113519528 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x55e1f7d90270 0x55e1f7d92660 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.276+0000 7f37b9036740 1 --2- 172.21.15.43:0/113519528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1f7d14e50 0x55e1f7d197b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.276+0000 7f37b9036740 1 --2- 172.21.15.43:0/113519528 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1f7d0a590 0x55e1f7d19270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.276+0000 7f3799600640 0 client.27581 ms_handle_reset on v2:172.21.15.151:6838/3794104313 2024-09-19T05:14:31.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.276+0000 7f37b9036740 1 --2- 172.21.15.43:0/113519528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1f7d09c70 0x55e1f7d18d30 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.276+0000 7f37b9036740 1 -- 172.21.15.43:0/113519528 >> 172.21.15.43:0/113519528 conn(0x55e1f7d06200 msgr2=0x55e1f7d12500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:31.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.276+0000 7f37b9036740 1 -- 172.21.15.43:0/113519528 shutdown_connections 2024-09-19T05:14:31.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.276+0000 7f37b9036740 1 -- 172.21.15.43:0/113519528 wait complete. 2024-09-19T05:14:31.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.277+0000 7f37b9036740 20 client.27581 trim_cache size 0 max 16384 2024-09-19T05:14:31.290 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:14:31.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.277+0000 7f37b6600640 1 -- 172.21.15.43:0/2632558426 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f37940a33b0 msgr2=0x7f37940a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:31.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.277+0000 7f37b6600640 1 --2- 172.21.15.43:0/2632558426 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f37940a33b0 0x7f37940a5870 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x55e1f7d07580 tx=0x7f379c013040 comp rx=0 tx=0).stop 2024-09-19T05:14:31.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.277+0000 7f37b6600640 1 -- 172.21.15.43:0/2632558426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f37b0109120 msgr2=0x7f37b019ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:31.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.277+0000 7f37b6600640 1 --2- 172.21.15.43:0/2632558426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f37b0109120 0x7f37b019ef30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f37a8005e30 tx=0x7f37a8003f10 comp rx=0 tx=0).stop 2024-09-19T05:14:31.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.277+0000 7f37b6600640 1 -- 172.21.15.43:0/2632558426 shutdown_connections 2024-09-19T05:14:31.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.277+0000 7f37b6600640 1 --2- 172.21.15.43:0/2632558426 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f37940a33b0 0x7f37940a5870 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.277+0000 7f37b6600640 1 --2- 172.21.15.43:0/2632558426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f37b0109120 0x7f37b019ef30 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.277+0000 7f37b6600640 1 --2- 172.21.15.43:0/2632558426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37b01049a0 0x7f37b019aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.277+0000 7f37b6600640 1 --2- 172.21.15.43:0/2632558426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37b0103ff0 0x7f37b019a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:31.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.277+0000 7f37b6600640 1 -- 172.21.15.43:0/2632558426 >> 172.21.15.43:0/2632558426 conn(0x7f37b00ffa90 msgr2=0x7f37b0100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:31.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.278+0000 7f37b6600640 1 -- 172.21.15.43:0/2632558426 shutdown_connections 2024-09-19T05:14:31.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:31.278+0000 7f37b6600640 1 -- 172.21.15.43:0/2632558426 wait complete. 2024-09-19T05:14:32.044 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T05:14:32.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:32 smithi043 ceph-mon[30800]: pgmap v307: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:14:32.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:32 smithi151 ceph-mon[35425]: pgmap v307: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:14:32.421 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:14:32.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:32 smithi096 ceph-mon[35085]: pgmap v307: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:14:33.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.145+0000 7f9e59a00640 1 -- 172.21.15.43:0/2743436358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e541079c0 msgr2=0x7f9e54107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:33.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.145+0000 7f9e59a00640 1 --2- 172.21.15.43:0/2743436358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e541079c0 0x7f9e54107da0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f9e44009930 tx=0x7f9e4402a420 comp rx=0 tx=0).stop 2024-09-19T05:14:33.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.145+0000 7f9e59a00640 1 -- 172.21.15.43:0/2743436358 shutdown_connections 2024-09-19T05:14:33.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.145+0000 7f9e59a00640 1 --2- 172.21.15.43:0/2743436358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e54102360 0x7f9e5410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.145+0000 7f9e59a00640 1 --2- 172.21.15.43:0/2743436358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e541019c0 0x7f9e54101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.145+0000 7f9e59a00640 1 --2- 172.21.15.43:0/2743436358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e541079c0 0x7f9e54107da0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.145+0000 7f9e59a00640 1 -- 172.21.15.43:0/2743436358 >> 172.21.15.43:0/2743436358 conn(0x7f9e540fd740 msgr2=0x7f9e540ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:33.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.145+0000 7f9e59a00640 1 -- 172.21.15.43:0/2743436358 shutdown_connections 2024-09-19T05:14:33.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.145+0000 7f9e59a00640 1 -- 172.21.15.43:0/2743436358 wait complete. 2024-09-19T05:14:33.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.146+0000 7f9e59a00640 1 Processor -- start 2024-09-19T05:14:33.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.147+0000 7f9e59a00640 1 -- start start 2024-09-19T05:14:33.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.147+0000 7f9e59a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e541019c0 0x7f9e5419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:33.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.147+0000 7f9e59a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e54102360 0x7f9e5419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:33.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.147+0000 7f9e59a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e541079c0 0x7f9e5419ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:33.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.147+0000 7f9e59a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9e54111dc0 con 0x7f9e541019c0 2024-09-19T05:14:33.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.147+0000 7f9e59a00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9e54111c40 con 0x7f9e54102360 2024-09-19T05:14:33.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.147+0000 7f9e59a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9e54111f40 con 0x7f9e541079c0 2024-09-19T05:14:33.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.147+0000 7f9e53e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e541019c0 0x7f9e5419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:33.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.147+0000 7f9e4b400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e54102360 0x7f9e5419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:33.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.147+0000 7f9e53e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e541019c0 0x7f9e5419a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44706/0 (socket says 172.21.15.43:44706) 2024-09-19T05:14:33.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.147+0000 7f9e53e00640 1 -- 172.21.15.43:0/254419625 learned_addr learned my addr 172.21.15.43:0/254419625 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:33.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.148+0000 7f9e59000640 1 --2- 172.21.15.43:0/254419625 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e541079c0 0x7f9e5419ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:33.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.148+0000 7f9e53e00640 1 -- 172.21.15.43:0/254419625 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e541079c0 msgr2=0x7f9e5419ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:33.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.148+0000 7f9e53e00640 1 --2- 172.21.15.43:0/254419625 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e541079c0 0x7f9e5419ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.148+0000 7f9e53e00640 1 -- 172.21.15.43:0/254419625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e54102360 msgr2=0x7f9e5419aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:33.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.148+0000 7f9e53e00640 1 --2- 172.21.15.43:0/254419625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e54102360 0x7f9e5419aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.148+0000 7f9e53e00640 1 -- 172.21.15.43:0/254419625 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e5419f4f0 con 0x7f9e541019c0 2024-09-19T05:14:33.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.148+0000 7f9e53e00640 1 --2- 172.21.15.43:0/254419625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e541019c0 0x7f9e5419a540 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f9e44009230 tx=0x7f9e44004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:33.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.148+0000 7f9e51600640 1 -- 172.21.15.43:0/254419625 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4403f030 con 0x7f9e541019c0 2024-09-19T05:14:33.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.149+0000 7f9e51600640 1 -- 172.21.15.43:0/254419625 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e44003ad0 con 0x7f9e541019c0 2024-09-19T05:14:33.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.149+0000 7f9e59a00640 1 -- 172.21.15.43:0/254419625 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e5419f780 con 0x7f9e541019c0 2024-09-19T05:14:33.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.149+0000 7f9e51600640 1 -- 172.21.15.43:0/254419625 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e44035070 con 0x7f9e541019c0 2024-09-19T05:14:33.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.149+0000 7f9e59a00640 1 -- 172.21.15.43:0/254419625 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e541a70c0 con 0x7f9e541019c0 2024-09-19T05:14:33.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.153+0000 7f9e5da97740 1 Processor -- start 2024-09-19T05:14:33.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.153+0000 7f9e5da97740 1 -- start start 2024-09-19T05:14:33.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.153+0000 7f9e51600640 1 -- 172.21.15.43:0/254419625 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f9e44042030 con 0x7f9e541019c0 2024-09-19T05:14:33.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.153+0000 7f9e5da97740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562467a11e40 0x562467a12220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:33.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.153+0000 7f9e5da97740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562467a12760 0x562467a1cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:33.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.153+0000 7f9e5da97740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562467a1d020 0x562467a1f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:33.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.153+0000 7f9e5da97740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5624679783e0 con 0x562467a12760 2024-09-19T05:14:33.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.153+0000 7f9e5da97740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56246791fd80 con 0x562467a1d020 2024-09-19T05:14:33.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.153+0000 7f9e5da97740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5624679743a0 con 0x562467a11e40 2024-09-19T05:14:33.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.154+0000 7f9e53e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562467a12760 0x562467a1cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:33.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.154+0000 7f9e59000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562467a11e40 0x562467a12220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:33.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.154+0000 7f9e53e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562467a12760 0x562467a1cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44718/0 (socket says 172.21.15.43:44718) 2024-09-19T05:14:33.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.154+0000 7f9e53e00640 1 -- 172.21.15.43:0/445196274 learned_addr learned my addr 172.21.15.43:0/445196274 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:33.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.154+0000 7f9e4b400640 1 --2- 172.21.15.43:0/445196274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562467a1d020 0x562467a1f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:33.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.154+0000 7f9e59000640 1 -- 172.21.15.43:0/445196274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562467a1d020 msgr2=0x562467a1f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:33.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.154+0000 7f9e59000640 1 --2- 172.21.15.43:0/445196274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562467a1d020 0x562467a1f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.154+0000 7f9e59000640 1 -- 172.21.15.43:0/445196274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562467a12760 msgr2=0x562467a1cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:33.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.154+0000 7f9e59000640 1 --2- 172.21.15.43:0/445196274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562467a12760 0x562467a1cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.154+0000 7f9e59000640 1 -- 172.21.15.43:0/445196274 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56246793eae0 con 0x562467a11e40 2024-09-19T05:14:33.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.155+0000 7f9e59000640 1 --2- 172.21.15.43:0/445196274 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562467a11e40 0x562467a12220 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c009eb0 tx=0x7f9e4c0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13418b9b1b630135 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:33.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.155+0000 7f9e49600640 1 -- 172.21.15.43:0/445196274 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c003b70 con 0x562467a11e40 2024-09-19T05:14:33.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.155+0000 7f9e49600640 1 -- 172.21.15.43:0/445196274 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e4c015070 con 0x562467a11e40 2024-09-19T05:14:33.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.155+0000 7f9e49600640 1 -- 172.21.15.43:0/445196274 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c01a940 con 0x562467a11e40 2024-09-19T05:14:33.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.155+0000 7f9e51600640 1 --2- 172.21.15.43:0/254419625 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f9e380a33b0 0x7f9e380a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:33.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.155+0000 7f9e51600640 1 -- 172.21.15.43:0/254419625 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f9e440f02a0 con 0x7f9e541019c0 2024-09-19T05:14:33.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.155+0000 7f9e53e00640 1 --2- 172.21.15.43:0/254419625 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f9e380a33b0 0x7f9e380a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:33.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.156+0000 7f9e5da97740 1 -- 172.21.15.43:0/445196274 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562467a11e40 msgr2=0x562467a12220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:33.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.156+0000 7f9e5da97740 1 --2- 172.21.15.43:0/445196274 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562467a11e40 0x562467a12220 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c009eb0 tx=0x7f9e4c0107a0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.156+0000 7f9e5da97740 1 -- 172.21.15.43:0/445196274 shutdown_connections 2024-09-19T05:14:33.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.156+0000 7f9e5da97740 1 --2- 172.21.15.43:0/445196274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562467a1d020 0x562467a1f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.156+0000 7f9e5da97740 1 --2- 172.21.15.43:0/445196274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562467a12760 0x562467a1cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.156+0000 7f9e5da97740 1 --2- 172.21.15.43:0/445196274 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562467a11e40 0x562467a12220 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.156+0000 7f9e5da97740 1 -- 172.21.15.43:0/445196274 >> 172.21.15.43:0/445196274 conn(0x562467a0e3d0 msgr2=0x562467a0e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:33.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.156+0000 7f9e53e00640 1 --2- 172.21.15.43:0/254419625 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f9e380a33b0 0x7f9e380a5870 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9e440bc5b0 tx=0x7f9e440f3df0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:33.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.156+0000 7f9e5da97740 1 -- 172.21.15.43:0/445196274 shutdown_connections 2024-09-19T05:14:33.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.156+0000 7f9e5da97740 1 -- 172.21.15.43:0/445196274 wait complete. 2024-09-19T05:14:33.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.156+0000 7f9e5da97740 1 -- start start 2024-09-19T05:14:33.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.157+0000 7f9e5da97740 1 Processor -- start 2024-09-19T05:14:33.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.157+0000 7f9e5da97740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562467a11e40 0x562467a19f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:33.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.157+0000 7f9e5da97740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562467a12760 0x562467a1a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:33.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.157+0000 7f9e5da97740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562467a1d020 0x562467a1a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:33.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.157+0000 7f9e5da97740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562467a1fbf0 con 0x562467a11e40 2024-09-19T05:14:33.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.157+0000 7f9e5da97740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5624679330f0 con 0x562467a1d020 2024-09-19T05:14:33.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.157+0000 7f9e5da97740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562467a1fd70 con 0x562467a12760 2024-09-19T05:14:33.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.157+0000 7f9e5da97740 10 client.0 _ll_register_callbacks cb 0x56246791abd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:33.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.157+0000 7f9e59000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562467a11e40 0x562467a19f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:33.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.157+0000 7f9e4b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562467a12760 0x562467a1a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:33.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.157+0000 7f9e4b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562467a12760 0x562467a1a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:43386/0 (socket says 172.21.15.43:43386) 2024-09-19T05:14:33.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.157+0000 7f9e4b400640 1 -- 172.21.15.43:0/3220678032 learned_addr learned my addr 172.21.15.43:0/3220678032 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:33.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.158+0000 7f9e59000640 1 --2- 172.21.15.43:0/3220678032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562467a1d020 0x562467a1a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:33.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.158+0000 7f9e4b400640 1 -- 172.21.15.43:0/3220678032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562467a1d020 msgr2=0x562467a1a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:33.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.158+0000 7f9e4b400640 1 --2- 172.21.15.43:0/3220678032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562467a1d020 0x562467a1a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.158+0000 7f9e4b400640 1 -- 172.21.15.43:0/3220678032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562467a11e40 msgr2=0x562467a19f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:33.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.158+0000 7f9e4b400640 1 --2- 172.21.15.43:0/3220678032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562467a11e40 0x562467a19f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.158+0000 7f9e4b400640 1 -- 172.21.15.43:0/3220678032 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624676e93c0 con 0x562467a12760 2024-09-19T05:14:33.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.158+0000 7f9e4b400640 1 --2- 172.21.15.43:0/3220678032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562467a12760 0x562467a1a490 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c012710 tx=0x7f9e3c012bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c647f3cba290f733 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:33.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.158+0000 7f9e42000640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:14:33.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.158+0000 7f9e42000640 1 -- 172.21.15.43:0/3220678032 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c00f920 con 0x562467a12760 2024-09-19T05:14:33.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.159+0000 7f9e5da97740 1 -- 172.21.15.43:0/3220678032 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e4c015070 con 0x562467a12760 2024-09-19T05:14:33.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.159+0000 7f9e42000640 1 -- 172.21.15.43:0/3220678032 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f9e3c00fac0 con 0x562467a12760 2024-09-19T05:14:33.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.159+0000 7f9e42000640 1 -- 172.21.15.43:0/3220678032 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9e3c00fcc0 con 0x562467a12760 2024-09-19T05:14:33.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.159+0000 7f9e5da97740 10 client.27493 fetch_fsmap learned FSMap version 29 2024-09-19T05:14:33.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.159+0000 7f9e5da97740 1 -- 172.21.15.43:0/3220678032 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562467a2bd60 con 0x562467a12760 2024-09-19T05:14:33.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.159+0000 7f9e42000640 1 -- 172.21.15.43:0/3220678032 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e4c015070 con 0x562467a12760 2024-09-19T05:14:33.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.159+0000 7f9e42000640 1 -- 172.21.15.43:0/3220678032 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9e3c01c4b0 con 0x562467a12760 2024-09-19T05:14:33.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.159+0000 7f9e5da97740 10 client.27493 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:14:33.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.159+0000 7f9e5da97740 10 client.27493 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-19T05:14:33.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.159+0000 7f9e5da97740 20 client.27493 populate_metadata read hostname 'smithi043' 2024-09-19T05:14:33.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.160+0000 7f9e5da97740 1 --2- 172.21.15.43:0/3220678032 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x562467a2d1c0 0x562467a2f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:33.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.160+0000 7f9e5da97740 4 client.27493 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:14:33.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.160+0000 7f9e5da97740 1 -- 172.21.15.43:0/3220678032 --> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562467985950 con 0x562467a2d1c0 2024-09-19T05:14:33.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.160+0000 7f9e59000640 1 --2- 172.21.15.43:0/3220678032 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x562467a2d1c0 0x562467a2f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:33.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.160+0000 7f9e59000640 1 --2- 172.21.15.43:0/3220678032 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x562467a2d1c0 0x562467a2f5d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6178358730253a3f server_cookie=158e12b86ea62a98 in_seq=0 out_seq=0 2024-09-19T05:14:33.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.160+0000 7f9e42000640 10 client.27493 ms_handle_connect on v2:172.21.15.43:6838/3404555030 2024-09-19T05:14:33.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.161+0000 7f9e42000640 1 -- 172.21.15.43:0/3220678032 <== mds.1 v2:172.21.15.43:6838/3404555030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562467985950 con 0x562467a2d1c0 2024-09-19T05:14:33.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.161+0000 7f9e42000640 10 client.27493 handle_command_reply: tid=0 2024-09-19T05:14:33.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.161+0000 7f9e5da97740 1 client.27493 shutdown 2024-09-19T05:14:33.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.161+0000 7f9e5da97740 1 -- 172.21.15.43:0/3220678032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562467a12760 msgr2=0x562467a1a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:33.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.161+0000 7f9e5da97740 1 --2- 172.21.15.43:0/3220678032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562467a12760 0x562467a1a490 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c012710 tx=0x7f9e3c012bd0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.161+0000 7f9e5da97740 1 -- 172.21.15.43:0/3220678032 shutdown_connections 2024-09-19T05:14:33.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.161+0000 7f9e5da97740 1 --2- 172.21.15.43:0/3220678032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562467a1d020 0x562467a1a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.161+0000 7f9e5da97740 1 --2- 172.21.15.43:0/3220678032 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x562467a2d1c0 0x562467a2f5d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.161+0000 7f9e5da97740 1 --2- 172.21.15.43:0/3220678032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562467a12760 0x562467a1a490 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.161+0000 7f9e5da97740 1 --2- 172.21.15.43:0/3220678032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562467a11e40 0x562467a19f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.161+0000 7f9e42000640 0 client.27493 ms_handle_reset on v2:172.21.15.43:6838/3404555030 2024-09-19T05:14:33.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.161+0000 7f9e5da97740 1 -- 172.21.15.43:0/3220678032 >> 172.21.15.43:0/3220678032 conn(0x562467a0e3d0 msgr2=0x562467a14100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:33.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.162+0000 7f9e5da97740 1 -- 172.21.15.43:0/3220678032 shutdown_connections 2024-09-19T05:14:33.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.162+0000 7f9e5da97740 1 -- 172.21.15.43:0/3220678032 wait complete. 2024-09-19T05:14:33.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.162+0000 7f9e5da97740 20 client.27493 trim_cache size 0 max 16384 2024-09-19T05:14:33.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.175+0000 7f9e5da97740 1 Processor -- start 2024-09-19T05:14:33.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.176+0000 7f9e5da97740 1 -- start start 2024-09-19T05:14:33.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.176+0000 7f9e5da97740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562467a11e40 0x562467a148a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:33.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.176+0000 7f9e5da97740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562467a12760 0x562467a34ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:33.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.176+0000 7f9e5da97740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562467a1d020 0x562467a35420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:33.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.176+0000 7f9e5da97740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9e541afa00 con 0x562467a12760 2024-09-19T05:14:33.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.176+0000 7f9e5da97740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9e541b1540 con 0x562467a11e40 2024-09-19T05:14:33.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.176+0000 7f9e5da97740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9e541b00d0 con 0x562467a1d020 2024-09-19T05:14:33.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.176+0000 7f9e4b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562467a12760 0x562467a34ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:33.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.176+0000 7f9e4b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562467a12760 0x562467a34ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44742/0 (socket says 172.21.15.43:44742) 2024-09-19T05:14:33.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.176+0000 7f9e59000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562467a11e40 0x562467a148a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:33.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.176+0000 7f9e4b400640 1 -- 172.21.15.43:0/1549937097 learned_addr learned my addr 172.21.15.43:0/1549937097 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:33.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.176+0000 7f9e59000640 1 --2- 172.21.15.43:0/1549937097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562467a1d020 0x562467a35420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:33.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.177+0000 7f9e59000640 1 -- 172.21.15.43:0/1549937097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562467a1d020 msgr2=0x562467a35420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:33.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.177+0000 7f9e59000640 1 --2- 172.21.15.43:0/1549937097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562467a1d020 0x562467a35420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.177+0000 7f9e59000640 1 -- 172.21.15.43:0/1549937097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562467a12760 msgr2=0x562467a34ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:33.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.177+0000 7f9e59000640 1 --2- 172.21.15.43:0/1549937097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562467a12760 0x562467a34ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.177+0000 7f9e59000640 1 -- 172.21.15.43:0/1549937097 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562467a11c70 con 0x562467a11e40 2024-09-19T05:14:33.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.177+0000 7f9e4b400640 1 --2- 172.21.15.43:0/1549937097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562467a12760 0x562467a34ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:33.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.177+0000 7f9e59000640 1 --2- 172.21.15.43:0/1549937097 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562467a11e40 0x562467a148a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c006010 tx=0x7f9e4c004740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1904b3a7234b003c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:33.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.177+0000 7f9e48c00640 1 -- 172.21.15.43:0/1549937097 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c01a610 con 0x562467a11e40 2024-09-19T05:14:33.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.177+0000 7f9e48c00640 1 -- 172.21.15.43:0/1549937097 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e4c015040 con 0x562467a11e40 2024-09-19T05:14:33.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.177+0000 7f9e48c00640 1 -- 172.21.15.43:0/1549937097 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c02ea60 con 0x562467a11e40 2024-09-19T05:14:33.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.178+0000 7f9e5da97740 1 -- 172.21.15.43:0/1549937097 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562467a11e40 msgr2=0x562467a148a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:33.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.178+0000 7f9e5da97740 1 --2- 172.21.15.43:0/1549937097 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562467a11e40 0x562467a148a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c006010 tx=0x7f9e4c004740 comp rx=0 tx=0).stop 2024-09-19T05:14:33.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.178+0000 7f9e5da97740 1 -- 172.21.15.43:0/1549937097 shutdown_connections 2024-09-19T05:14:33.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.178+0000 7f9e5da97740 1 --2- 172.21.15.43:0/1549937097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562467a1d020 0x562467a35420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.178+0000 7f9e5da97740 1 --2- 172.21.15.43:0/1549937097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562467a12760 0x562467a34ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.178+0000 7f9e5da97740 1 --2- 172.21.15.43:0/1549937097 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562467a11e40 0x562467a148a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.178+0000 7f9e5da97740 1 -- 172.21.15.43:0/1549937097 >> 172.21.15.43:0/1549937097 conn(0x562467a0e3d0 msgr2=0x562467a0e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:33.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.178+0000 7f9e5da97740 1 -- 172.21.15.43:0/1549937097 shutdown_connections 2024-09-19T05:14:33.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.178+0000 7f9e5da97740 1 -- 172.21.15.43:0/1549937097 wait complete. 2024-09-19T05:14:33.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.178+0000 7f9e5da97740 1 -- start start 2024-09-19T05:14:33.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.179+0000 7f9e5da97740 1 Processor -- start 2024-09-19T05:14:33.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.179+0000 7f9e5da97740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562467a11e40 0x562467a25820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:33.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.179+0000 7f9e5da97740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562467a12760 0x562467a16d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:33.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.179+0000 7f9e5da97740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562467a1d020 0x562467a17340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:33.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.179+0000 7f9e5da97740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9e541b0e70 con 0x562467a12760 2024-09-19T05:14:33.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.179+0000 7f9e5da97740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9e541b07a0 con 0x562467a1d020 2024-09-19T05:14:33.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.179+0000 7f9e5da97740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9e541ada00 con 0x562467a11e40 2024-09-19T05:14:33.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.179+0000 7f9e5da97740 10 client.0 _ll_register_callbacks cb 0x7f9e3c009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:33.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.179+0000 7f9e4b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562467a12760 0x562467a16d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:33.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.179+0000 7f9e4b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562467a12760 0x562467a16d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44758/0 (socket says 172.21.15.43:44758) 2024-09-19T05:14:33.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.179+0000 7f9e4b400640 1 -- 172.21.15.43:0/58170051 learned_addr learned my addr 172.21.15.43:0/58170051 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:33.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.179+0000 7f9e59000640 1 --2- 172.21.15.43:0/58170051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562467a1d020 0x562467a17340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:33.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.180+0000 7f9e59000640 1 --2- 172.21.15.43:0/58170051 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562467a11e40 0x562467a25820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:33.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.180+0000 7f9e4b400640 1 -- 172.21.15.43:0/58170051 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562467a11e40 msgr2=0x562467a25820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:33.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.180+0000 7f9e4b400640 1 --2- 172.21.15.43:0/58170051 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562467a11e40 0x562467a25820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.180+0000 7f9e4b400640 1 -- 172.21.15.43:0/58170051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562467a1d020 msgr2=0x562467a17340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:33.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.180+0000 7f9e4b400640 1 --2- 172.21.15.43:0/58170051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562467a1d020 0x562467a17340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.180+0000 7f9e4b400640 1 -- 172.21.15.43:0/58170051 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562467a0f480 con 0x562467a12760 2024-09-19T05:14:33.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.180+0000 7f9e59000640 1 --2- 172.21.15.43:0/58170051 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562467a11e40 0x562467a25820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:14:33.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.180+0000 7f9e4b400640 1 --2- 172.21.15.43:0/58170051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562467a12760 0x562467a16d50 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c000c00 tx=0x7f9e3c01cdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db32a4aac6ea8bc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:33.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.180+0000 7f9e42000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:14:33.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.180+0000 7f9e42000640 1 -- 172.21.15.43:0/58170051 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c01c630 con 0x562467a12760 2024-09-19T05:14:33.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.180+0000 7f9e5da97740 1 -- 172.21.15.43:0/58170051 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e4c015040 con 0x562467a12760 2024-09-19T05:14:33.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.180+0000 7f9e42000640 1 -- 172.21.15.43:0/58170051 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f9e3c005030 con 0x562467a12760 2024-09-19T05:14:33.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.181+0000 7f9e42000640 1 -- 172.21.15.43:0/58170051 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9e3c033050 con 0x562467a12760 2024-09-19T05:14:33.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.181+0000 7f9e5da97740 10 client.17577 fetch_fsmap learned FSMap version 29 2024-09-19T05:14:33.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.181+0000 7f9e5da97740 1 -- 172.21.15.43:0/58170051 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562467a97db0 con 0x562467a12760 2024-09-19T05:14:33.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.181+0000 7f9e42000640 1 -- 172.21.15.43:0/58170051 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e4c015040 con 0x562467a12760 2024-09-19T05:14:33.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.181+0000 7f9e42000640 1 -- 172.21.15.43:0/58170051 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9e3c02b020 con 0x562467a12760 2024-09-19T05:14:33.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.181+0000 7f9e5da97740 10 client.17577 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:14:33.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.181+0000 7f9e5da97740 10 client.17577 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-19T05:14:33.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.181+0000 7f9e5da97740 20 client.17577 populate_metadata read hostname 'smithi043' 2024-09-19T05:14:33.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.181+0000 7f9e5da97740 1 --2- 172.21.15.43:0/58170051 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x562467a98550 0x562467a9a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:33.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.181+0000 7f9e5da97740 4 client.17577 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:14:33.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.181+0000 7f9e5da97740 1 -- 172.21.15.43:0/58170051 --> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56246797fe60 con 0x562467a98550 2024-09-19T05:14:33.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.182+0000 7f9e59000640 1 --2- 172.21.15.43:0/58170051 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x562467a98550 0x562467a9a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:33.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.182+0000 7f9e59000640 1 --2- 172.21.15.43:0/58170051 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x562467a98550 0x562467a9a970 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a813c98335d90276 server_cookie=866c9a29ee4a6392 in_seq=0 out_seq=0 2024-09-19T05:14:33.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.182+0000 7f9e42000640 10 client.17577 ms_handle_connect on v2:172.21.15.43:6838/3404555030 2024-09-19T05:14:33.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.182+0000 7f9e42000640 1 -- 172.21.15.43:0/58170051 <== mds.1 v2:172.21.15.43:6838/3404555030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56246797fe60 con 0x562467a98550 2024-09-19T05:14:33.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.182+0000 7f9e42000640 10 client.17577 handle_command_reply: tid=0 2024-09-19T05:14:33.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.182+0000 7f9e5da97740 1 client.17577 shutdown 2024-09-19T05:14:33.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.182+0000 7f9e5da97740 1 -- 172.21.15.43:0/58170051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562467a12760 msgr2=0x562467a16d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:33.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.182+0000 7f9e5da97740 1 --2- 172.21.15.43:0/58170051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562467a12760 0x562467a16d50 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c000c00 tx=0x7f9e3c01cdc0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.183+0000 7f9e5da97740 1 -- 172.21.15.43:0/58170051 shutdown_connections 2024-09-19T05:14:33.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.183+0000 7f9e5da97740 1 --2- 172.21.15.43:0/58170051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562467a1d020 0x562467a17340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.183+0000 7f9e5da97740 1 --2- 172.21.15.43:0/58170051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562467a12760 0x562467a16d50 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.183+0000 7f9e5da97740 1 --2- 172.21.15.43:0/58170051 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x562467a98550 0x562467a9a970 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.183+0000 7f9e5da97740 1 --2- 172.21.15.43:0/58170051 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562467a11e40 0x562467a25820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.183+0000 7f9e42000640 0 client.17577 ms_handle_reset on v2:172.21.15.43:6838/3404555030 2024-09-19T05:14:33.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.183+0000 7f9e5da97740 1 -- 172.21.15.43:0/58170051 >> 172.21.15.43:0/58170051 conn(0x562467a0e3d0 msgr2=0x562467a1b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:33.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.183+0000 7f9e5da97740 1 -- 172.21.15.43:0/58170051 shutdown_connections 2024-09-19T05:14:33.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.183+0000 7f9e5da97740 1 -- 172.21.15.43:0/58170051 wait complete. 2024-09-19T05:14:33.338 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:14:33.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.183+0000 7f9e5da97740 20 client.17577 trim_cache size 0 max 16384 2024-09-19T05:14:33.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.183+0000 7f9e59a00640 1 -- 172.21.15.43:0/254419625 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f9e380a33b0 msgr2=0x7f9e380a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:33.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.183+0000 7f9e59a00640 1 --2- 172.21.15.43:0/254419625 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f9e380a33b0 0x7f9e380a5870 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9e440bc5b0 tx=0x7f9e440f3df0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.183+0000 7f9e59a00640 1 -- 172.21.15.43:0/254419625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e541019c0 msgr2=0x7f9e5419a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:33.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.183+0000 7f9e59a00640 1 --2- 172.21.15.43:0/254419625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e541019c0 0x7f9e5419a540 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f9e44009230 tx=0x7f9e44004270 comp rx=0 tx=0).stop 2024-09-19T05:14:33.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.184+0000 7f9e59a00640 1 -- 172.21.15.43:0/254419625 shutdown_connections 2024-09-19T05:14:33.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.184+0000 7f9e59a00640 1 --2- 172.21.15.43:0/254419625 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e541079c0 0x7f9e5419ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.184+0000 7f9e59a00640 1 --2- 172.21.15.43:0/254419625 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f9e380a33b0 0x7f9e380a5870 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.184+0000 7f9e59a00640 1 --2- 172.21.15.43:0/254419625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e54102360 0x7f9e5419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.184+0000 7f9e59a00640 1 --2- 172.21.15.43:0/254419625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e541019c0 0x7f9e5419a540 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:33.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.184+0000 7f9e59a00640 1 -- 172.21.15.43:0/254419625 >> 172.21.15.43:0/254419625 conn(0x7f9e540fd740 msgr2=0x7f9e5410cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:33.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.184+0000 7f9e59a00640 1 -- 172.21.15.43:0/254419625 shutdown_connections 2024-09-19T05:14:33.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:33.184+0000 7f9e59a00640 1 -- 172.21.15.43:0/254419625 wait complete. 2024-09-19T05:14:34.190 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T05:14:34.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:34 smithi151 ceph-mon[35425]: pgmap v308: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:14:34.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:34 smithi096 ceph-mon[35085]: pgmap v308: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:14:34.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:34 smithi043 ceph-mon[30800]: pgmap v308: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:14:34.570 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:14:35.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.121+0000 7f1236000640 1 -- 172.21.15.43:0/2932153826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1230109120 msgr2=0x7f1230110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:35.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.121+0000 7f1236000640 1 --2- 172.21.15.43:0/2932153826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1230109120 0x7f1230110ca0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f1228009ca0 tx=0x7f122802a8c0 comp rx=0 tx=0).stop 2024-09-19T05:14:35.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.121+0000 7f1236000640 1 -- 172.21.15.43:0/2932153826 shutdown_connections 2024-09-19T05:14:35.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.121+0000 7f1236000640 1 --2- 172.21.15.43:0/2932153826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1230109120 0x7f1230110ca0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:35.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.121+0000 7f1236000640 1 --2- 172.21.15.43:0/2932153826 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12301049a0 0x7f12301089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:35.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.121+0000 7f1236000640 1 --2- 172.21.15.43:0/2932153826 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1230103ff0 0x7f12301043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:35.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.121+0000 7f1236000640 1 -- 172.21.15.43:0/2932153826 >> 172.21.15.43:0/2932153826 conn(0x7f12300ffa90 msgr2=0x7f1230101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:35.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.121+0000 7f1236000640 1 -- 172.21.15.43:0/2932153826 shutdown_connections 2024-09-19T05:14:35.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.121+0000 7f1236000640 1 -- 172.21.15.43:0/2932153826 wait complete. 2024-09-19T05:14:35.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.122+0000 7f1236000640 1 Processor -- start 2024-09-19T05:14:35.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.123+0000 7f1236000640 1 -- start start 2024-09-19T05:14:35.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.123+0000 7f1236000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1230103ff0 0x7f123019a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:35.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.123+0000 7f1236000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12301049a0 0x7f123019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:35.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.123+0000 7f1236000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1230109120 0x7f123019ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:35.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.123+0000 7f1236000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1230114150 con 0x7f1230103ff0 2024-09-19T05:14:35.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.123+0000 7f1236000640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1230113fd0 con 0x7f1230109120 2024-09-19T05:14:35.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.123+0000 7f1236000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f12301142d0 con 0x7f12301049a0 2024-09-19T05:14:35.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.123+0000 7f122f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1230103ff0 0x7f123019a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:35.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.123+0000 7f122f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1230103ff0 0x7f123019a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44774/0 (socket says 172.21.15.43:44774) 2024-09-19T05:14:35.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.123+0000 7f122f400640 1 -- 172.21.15.43:0/2428572387 learned_addr learned my addr 172.21.15.43:0/2428572387 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:35.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.123+0000 7f1226a00640 1 --2- 172.21.15.43:0/2428572387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12301049a0 0x7f123019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:35.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.123+0000 7f122fe00640 1 --2- 172.21.15.43:0/2428572387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1230109120 0x7f123019ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:35.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.124+0000 7f122f400640 1 -- 172.21.15.43:0/2428572387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12301049a0 msgr2=0x7f123019aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:35.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.124+0000 7f122f400640 1 --2- 172.21.15.43:0/2428572387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12301049a0 0x7f123019aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:35.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.124+0000 7f122f400640 1 -- 172.21.15.43:0/2428572387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1230109120 msgr2=0x7f123019ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:35.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.124+0000 7f122f400640 1 --2- 172.21.15.43:0/2428572387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1230109120 0x7f123019ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:35.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.124+0000 7f122f400640 1 -- 172.21.15.43:0/2428572387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f123019f560 con 0x7f1230103ff0 2024-09-19T05:14:35.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.124+0000 7f122fe00640 1 --2- 172.21.15.43:0/2428572387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1230109120 0x7f123019ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:14:35.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.124+0000 7f1226a00640 1 --2- 172.21.15.43:0/2428572387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12301049a0 0x7f123019aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:35.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.124+0000 7f122f400640 1 --2- 172.21.15.43:0/2428572387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1230103ff0 0x7f123019a5b0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f122000e9f0 tx=0x7f122000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:35.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.124+0000 7f122cc00640 1 -- 172.21.15.43:0/2428572387 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122000cd80 con 0x7f1230103ff0 2024-09-19T05:14:35.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.124+0000 7f122cc00640 1 -- 172.21.15.43:0/2428572387 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1220010910 con 0x7f1230103ff0 2024-09-19T05:14:35.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.124+0000 7f122cc00640 1 -- 172.21.15.43:0/2428572387 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1220018990 con 0x7f1230103ff0 2024-09-19T05:14:35.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.124+0000 7f1236000640 1 -- 172.21.15.43:0/2428572387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f123019f850 con 0x7f1230103ff0 2024-09-19T05:14:35.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.125+0000 7f1236000640 1 -- 172.21.15.43:0/2428572387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12301a7130 con 0x7f1230103ff0 2024-09-19T05:14:35.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.128+0000 7f122cc00640 1 -- 172.21.15.43:0/2428572387 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f1220010ac0 con 0x7f1230103ff0 2024-09-19T05:14:35.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.128+0000 7f1238a29740 1 Processor -- start 2024-09-19T05:14:35.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.128+0000 7f1238a29740 1 -- start start 2024-09-19T05:14:35.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.128+0000 7f1238a29740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f5a14ac70 0x555f5a14b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:35.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.128+0000 7f1238a29740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5a14b590 0x555f5a155910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:35.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.128+0000 7f1238a29740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f5a155e50 0x555f5a158240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:35.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.128+0000 7f1238a29740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555f5a0b13e0 con 0x555f5a155e50 2024-09-19T05:14:35.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.128+0000 7f1238a29740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555f5a058d80 con 0x555f5a14ac70 2024-09-19T05:14:35.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.128+0000 7f1238a29740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555f5a0ad3a0 con 0x555f5a14b590 2024-09-19T05:14:35.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.129+0000 7f122f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5a14b590 0x555f5a155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:35.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.129+0000 7f122f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5a14b590 0x555f5a155910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:43438/0 (socket says 172.21.15.43:43438) 2024-09-19T05:14:35.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.129+0000 7f122f400640 1 -- 172.21.15.43:0/3701972251 learned_addr learned my addr 172.21.15.43:0/3701972251 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:35.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.129+0000 7f1226a00640 1 --2- 172.21.15.43:0/3701972251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f5a155e50 0x555f5a158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:35.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.129+0000 7f122fe00640 1 --2- 172.21.15.43:0/3701972251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f5a14ac70 0x555f5a14b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:35.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.129+0000 7f122f400640 1 -- 172.21.15.43:0/3701972251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f5a14ac70 msgr2=0x555f5a14b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:35.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.129+0000 7f122f400640 1 --2- 172.21.15.43:0/3701972251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f5a14ac70 0x555f5a14b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:35.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.129+0000 7f122f400640 1 -- 172.21.15.43:0/3701972251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f5a155e50 msgr2=0x555f5a158240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:35.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.129+0000 7f122f400640 1 --2- 172.21.15.43:0/3701972251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f5a155e50 0x555f5a158240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:35.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.129+0000 7f122f400640 1 -- 172.21.15.43:0/3701972251 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f5a077ae0 con 0x555f5a14b590 2024-09-19T05:14:35.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.129+0000 7f1226a00640 1 --2- 172.21.15.43:0/3701972251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f5a155e50 0x555f5a158240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:14:35.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.129+0000 7f122cc00640 1 --2- 172.21.15.43:0/2428572387 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f12140a3430 0x7f12140a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:35.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.130+0000 7f122f400640 1 --2- 172.21.15.43:0/3701972251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5a14b590 0x555f5a155910 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f12200cd6d0 tx=0x7f12200d8700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbca0f91ce635615 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:35.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.130+0000 7f122cc00640 1 -- 172.21.15.43:0/2428572387 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f12200d00b0 con 0x7f1230103ff0 2024-09-19T05:14:35.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.130+0000 7f122fe00640 1 --2- 172.21.15.43:0/2428572387 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f12140a3430 0x7f12140a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:35.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.130+0000 7f121fe00640 1 -- 172.21.15.43:0/3701972251 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12200985c0 con 0x555f5a14b590 2024-09-19T05:14:35.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.130+0000 7f121fe00640 1 -- 172.21.15.43:0/3701972251 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f122009ca20 con 0x555f5a14b590 2024-09-19T05:14:35.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.130+0000 7f121fe00640 1 -- 172.21.15.43:0/3701972251 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122009bc80 con 0x555f5a14b590 2024-09-19T05:14:35.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.131+0000 7f1238a29740 1 -- 172.21.15.43:0/3701972251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5a14b590 msgr2=0x555f5a155910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:35.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.131+0000 7f122fe00640 1 --2- 172.21.15.43:0/2428572387 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f12140a3430 0x7f12140a58f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f122802f540 tx=0x7f122802f310 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:35.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.131+0000 7f1238a29740 1 --2- 172.21.15.43:0/3701972251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5a14b590 0x555f5a155910 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f12200cd6d0 tx=0x7f12200d8700 comp rx=0 tx=0).stop 2024-09-19T05:14:35.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.131+0000 7f1238a29740 1 -- 172.21.15.43:0/3701972251 shutdown_connections 2024-09-19T05:14:35.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.131+0000 7f1238a29740 1 --2- 172.21.15.43:0/3701972251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f5a155e50 0x555f5a158240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:35.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.131+0000 7f1238a29740 1 --2- 172.21.15.43:0/3701972251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5a14b590 0x555f5a155910 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:35.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.131+0000 7f1238a29740 1 --2- 172.21.15.43:0/3701972251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f5a14ac70 0x555f5a14b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:35.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.131+0000 7f1238a29740 1 -- 172.21.15.43:0/3701972251 >> 172.21.15.43:0/3701972251 conn(0x555f5a147200 msgr2=0x555f5a1475e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:35.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.131+0000 7f1238a29740 1 -- 172.21.15.43:0/3701972251 shutdown_connections 2024-09-19T05:14:35.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.131+0000 7f1238a29740 1 -- 172.21.15.43:0/3701972251 wait complete. 2024-09-19T05:14:35.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.132+0000 7f1238a29740 1 -- start start 2024-09-19T05:14:35.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.132+0000 7f1238a29740 1 Processor -- start 2024-09-19T05:14:35.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.132+0000 7f1238a29740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5a14ac70 0x555f5a152d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:35.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.132+0000 7f1238a29740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f5a14b590 0x555f5a1532a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:35.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.132+0000 7f1238a29740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f5a155e50 0x555f5a1537e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:35.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.132+0000 7f1238a29740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555f5a158a20 con 0x555f5a155e50 2024-09-19T05:14:35.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.132+0000 7f1238a29740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555f5a06c0f0 con 0x555f5a14b590 2024-09-19T05:14:35.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.132+0000 7f1238a29740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555f5a158ba0 con 0x555f5a14ac70 2024-09-19T05:14:35.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.132+0000 7f1238a29740 10 client.0 _ll_register_callbacks cb 0x555f5a053bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:35.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.133+0000 7f122fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f5a155e50 0x555f5a1537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:35.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.133+0000 7f122fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f5a155e50 0x555f5a1537e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44792/0 (socket says 172.21.15.43:44792) 2024-09-19T05:14:35.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.133+0000 7f122fe00640 1 -- 172.21.15.43:0/4166752837 learned_addr learned my addr 172.21.15.43:0/4166752837 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:35.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.133+0000 7f122f400640 1 --2- 172.21.15.43:0/4166752837 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5a14ac70 0x555f5a152d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:35.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.133+0000 7f1226a00640 1 --2- 172.21.15.43:0/4166752837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f5a14b590 0x555f5a1532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:35.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.133+0000 7f1226a00640 1 -- 172.21.15.43:0/4166752837 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5a14ac70 msgr2=0x555f5a152d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:35.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.133+0000 7f1226a00640 1 --2- 172.21.15.43:0/4166752837 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5a14ac70 0x555f5a152d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:35.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.133+0000 7f1226a00640 1 -- 172.21.15.43:0/4166752837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f5a155e50 msgr2=0x555f5a1537e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:35.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.133+0000 7f1226a00640 1 --2- 172.21.15.43:0/4166752837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f5a155e50 0x555f5a1537e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:35.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.133+0000 7f1226a00640 1 -- 172.21.15.43:0/4166752837 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f59e223c0 con 0x555f5a14b590 2024-09-19T05:14:35.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.133+0000 7f122fe00640 1 --2- 172.21.15.43:0/4166752837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f5a155e50 0x555f5a1537e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:35.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.133+0000 7f122f400640 1 --2- 172.21.15.43:0/4166752837 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5a14ac70 0x555f5a152d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:35.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.134+0000 7f1226a00640 1 --2- 172.21.15.43:0/4166752837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f5a14b590 0x555f5a1532a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1218009fd0 tx=0x7f1218012e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bc4f0f2ee3bb608 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:35.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.134+0000 7f121cc00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:14:35.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.134+0000 7f121cc00640 1 -- 172.21.15.43:0/4166752837 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121800f9e0 con 0x555f5a14b590 2024-09-19T05:14:35.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.134+0000 7f1238a29740 1 -- 172.21.15.43:0/4166752837 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f122009ca20 con 0x555f5a14b590 2024-09-19T05:14:35.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.134+0000 7f121cc00640 1 -- 172.21.15.43:0/4166752837 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f121800fb80 con 0x555f5a14b590 2024-09-19T05:14:35.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.134+0000 7f121cc00640 1 -- 172.21.15.43:0/4166752837 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f121800fd80 con 0x555f5a14b590 2024-09-19T05:14:35.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.134+0000 7f1238a29740 10 client.27611 fetch_fsmap learned FSMap version 29 2024-09-19T05:14:35.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.134+0000 7f1238a29740 1 -- 172.21.15.43:0/4166752837 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555f5a164b40 con 0x555f5a14b590 2024-09-19T05:14:35.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.134+0000 7f121cc00640 1 -- 172.21.15.43:0/4166752837 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f122009ca20 con 0x555f5a14b590 2024-09-19T05:14:35.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.135+0000 7f121cc00640 1 -- 172.21.15.43:0/4166752837 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1218010430 con 0x555f5a14b590 2024-09-19T05:14:35.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.135+0000 7f1238a29740 10 client.27611 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:14:35.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.135+0000 7f1238a29740 10 client.27611 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:14:35.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.135+0000 7f1238a29740 20 client.27611 populate_metadata read hostname 'smithi043' 2024-09-19T05:14:35.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.135+0000 7f1238a29740 1 --2- 172.21.15.43:0/4166752837 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x555f5a165fa0 0x555f5a1683c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:35.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.135+0000 7f1238a29740 4 client.27611 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:14:35.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.135+0000 7f1238a29740 1 -- 172.21.15.43:0/4166752837 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f5a0be950 con 0x555f5a165fa0 2024-09-19T05:14:35.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.135+0000 7f122f400640 1 --2- 172.21.15.43:0/4166752837 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x555f5a165fa0 0x555f5a1683c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:35.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.136+0000 7f122f400640 1 --2- 172.21.15.43:0/4166752837 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x555f5a165fa0 0x555f5a1683c0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=474873c62e3118f3 server_cookie=fb48350d960953f0 in_seq=0 out_seq=0 2024-09-19T05:14:35.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.136+0000 7f121cc00640 10 client.27611 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:14:35.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.136+0000 7f121cc00640 1 -- 172.21.15.43:0/4166752837 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555f5a0be950 con 0x555f5a165fa0 2024-09-19T05:14:35.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.136+0000 7f121cc00640 10 client.27611 handle_command_reply: tid=0 2024-09-19T05:14:35.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.137+0000 7f1238a29740 1 client.27611 shutdown 2024-09-19T05:14:35.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.137+0000 7f1238a29740 1 -- 172.21.15.43:0/4166752837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f5a14b590 msgr2=0x555f5a1532a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:35.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.137+0000 7f1238a29740 1 --2- 172.21.15.43:0/4166752837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f5a14b590 0x555f5a1532a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1218009fd0 tx=0x7f1218012e70 comp rx=0 tx=0).stop 2024-09-19T05:14:35.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.137+0000 7f1238a29740 1 -- 172.21.15.43:0/4166752837 shutdown_connections 2024-09-19T05:14:35.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.137+0000 7f1238a29740 1 --2- 172.21.15.43:0/4166752837 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x555f5a165fa0 0x555f5a1683c0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:35.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.137+0000 7f1238a29740 1 --2- 172.21.15.43:0/4166752837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f5a155e50 0x555f5a1537e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:35.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.137+0000 7f1238a29740 1 --2- 172.21.15.43:0/4166752837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f5a14b590 0x555f5a1532a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:35.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.137+0000 7f1238a29740 1 --2- 172.21.15.43:0/4166752837 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5a14ac70 0x555f5a152d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:35.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.137+0000 7f121cc00640 0 client.27611 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:14:35.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.137+0000 7f1238a29740 1 -- 172.21.15.43:0/4166752837 >> 172.21.15.43:0/4166752837 conn(0x555f5a147200 msgr2=0x555f5a14cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:35.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.137+0000 7f1238a29740 1 -- 172.21.15.43:0/4166752837 shutdown_connections 2024-09-19T05:14:35.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.137+0000 7f1238a29740 1 -- 172.21.15.43:0/4166752837 wait complete. 2024-09-19T05:14:35.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.137+0000 7f1238a29740 20 client.27611 trim_cache size 0 max 16384 2024-09-19T05:14:35.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.151+0000 7f1238a29740 1 Processor -- start 2024-09-19T05:14:35.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.151+0000 7f1238a29740 1 -- start start 2024-09-19T05:14:35.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.151+0000 7f1238a29740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5a14ac70 0x555f5a16dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:35.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.151+0000 7f1238a29740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f5a14b590 0x555f5a16e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:35.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.151+0000 7f1238a29740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f5a155e50 0x555f5a14ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:35.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.151+0000 7f1238a29740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f12301afb50 con 0x555f5a155e50 2024-09-19T05:14:35.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.151+0000 7f1238a29740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f12301b1690 con 0x555f5a14b590 2024-09-19T05:14:35.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.151+0000 7f1238a29740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f12301b0220 con 0x555f5a14ac70 2024-09-19T05:14:35.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.152+0000 7f122fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f5a155e50 0x555f5a14ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:35.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.152+0000 7f1226a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f5a14b590 0x555f5a16e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:35.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.152+0000 7f122fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f5a155e50 0x555f5a14ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44804/0 (socket says 172.21.15.43:44804) 2024-09-19T05:14:35.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.152+0000 7f122fe00640 1 -- 172.21.15.43:0/3380942292 learned_addr learned my addr 172.21.15.43:0/3380942292 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:35.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.152+0000 7f122f400640 1 --2- 172.21.15.43:0/3380942292 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5a14ac70 0x555f5a16dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:35.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.152+0000 7f1226a00640 1 -- 172.21.15.43:0/3380942292 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5a14ac70 msgr2=0x555f5a16dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:35.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.152+0000 7f1226a00640 1 --2- 172.21.15.43:0/3380942292 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5a14ac70 0x555f5a16dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:35.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.152+0000 7f1226a00640 1 -- 172.21.15.43:0/3380942292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f5a155e50 msgr2=0x555f5a14ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:35.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.152+0000 7f1226a00640 1 --2- 172.21.15.43:0/3380942292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f5a155e50 0x555f5a14ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:35.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.152+0000 7f1226a00640 1 -- 172.21.15.43:0/3380942292 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f5a14b3c0 con 0x555f5a14b590 2024-09-19T05:14:35.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.152+0000 7f122fe00640 1 --2- 172.21.15.43:0/3380942292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f5a155e50 0x555f5a14ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:35.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.152+0000 7f1226a00640 1 --2- 172.21.15.43:0/3380942292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f5a14b590 0x555f5a16e190 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f12180108a0 tx=0x7f121800f810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e6728ed53c5270e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:35.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.153+0000 7f121f400640 1 -- 172.21.15.43:0/3380942292 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1218028070 con 0x555f5a14b590 2024-09-19T05:14:35.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.153+0000 7f121f400640 1 -- 172.21.15.43:0/3380942292 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f121800f9e0 con 0x555f5a14b590 2024-09-19T05:14:35.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.153+0000 7f121f400640 1 -- 172.21.15.43:0/3380942292 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1218010e60 con 0x555f5a14b590 2024-09-19T05:14:35.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.153+0000 7f1238a29740 1 -- 172.21.15.43:0/3380942292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f5a14b590 msgr2=0x555f5a16e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:35.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.153+0000 7f1238a29740 1 --2- 172.21.15.43:0/3380942292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f5a14b590 0x555f5a16e190 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f12180108a0 tx=0x7f121800f810 comp rx=0 tx=0).stop 2024-09-19T05:14:35.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.153+0000 7f1238a29740 1 -- 172.21.15.43:0/3380942292 shutdown_connections 2024-09-19T05:14:35.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.153+0000 7f1238a29740 1 --2- 172.21.15.43:0/3380942292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f5a155e50 0x555f5a14ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:35.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.153+0000 7f1238a29740 1 --2- 172.21.15.43:0/3380942292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f5a14b590 0x555f5a16e190 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:35.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.153+0000 7f1238a29740 1 --2- 172.21.15.43:0/3380942292 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5a14ac70 0x555f5a16dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:35.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.153+0000 7f1238a29740 1 -- 172.21.15.43:0/3380942292 >> 172.21.15.43:0/3380942292 conn(0x555f5a147200 msgr2=0x555f5a154de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:35.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.154+0000 7f1238a29740 1 -- 172.21.15.43:0/3380942292 shutdown_connections 2024-09-19T05:14:35.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.154+0000 7f1238a29740 1 -- 172.21.15.43:0/3380942292 wait complete. 2024-09-19T05:14:35.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.154+0000 7f1238a29740 1 -- start start 2024-09-19T05:14:35.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.154+0000 7f1238a29740 1 Processor -- start 2024-09-19T05:14:35.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.155+0000 7f1238a29740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f5a14ac70 0x555f5a159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:35.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.155+0000 7f1238a29740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5a14b590 0x555f5a15a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:35.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.155+0000 7f1238a29740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f5a155e50 0x555f5a15a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:35.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.155+0000 7f1238a29740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f12301b0fc0 con 0x555f5a14ac70 2024-09-19T05:14:35.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.155+0000 7f1238a29740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f12301b08f0 con 0x555f5a155e50 2024-09-19T05:14:35.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.155+0000 7f1238a29740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f12301adb50 con 0x555f5a14b590 2024-09-19T05:14:35.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.155+0000 7f1238a29740 10 client.0 _ll_register_callbacks cb 0x7f1218009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:35.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.155+0000 7f122f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f5a14ac70 0x555f5a159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:35.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.155+0000 7f122fe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f5a155e50 0x555f5a15a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:35.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.155+0000 7f122fe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f5a155e50 0x555f5a15a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:37788/0 (socket says 172.21.15.43:37788) 2024-09-19T05:14:35.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.155+0000 7f122fe00640 1 -- 172.21.15.43:0/3297234114 learned_addr learned my addr 172.21.15.43:0/3297234114 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:35.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.155+0000 7f122f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f5a14ac70 0x555f5a159cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44814/0 (socket says 172.21.15.43:44814) 2024-09-19T05:14:35.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.155+0000 7f1226a00640 1 --2- 172.21.15.43:0/3297234114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5a14b590 0x555f5a15a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:35.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.156+0000 7f122fe00640 1 -- 172.21.15.43:0/3297234114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5a14b590 msgr2=0x555f5a15a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:35.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.156+0000 7f122fe00640 1 --2- 172.21.15.43:0/3297234114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5a14b590 0x555f5a15a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:35.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.156+0000 7f122fe00640 1 -- 172.21.15.43:0/3297234114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f5a14ac70 msgr2=0x555f5a159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:35.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.156+0000 7f122fe00640 1 --2- 172.21.15.43:0/3297234114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f5a14ac70 0x555f5a159cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:35.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.156+0000 7f122fe00640 1 -- 172.21.15.43:0/3297234114 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f5a1cfd80 con 0x555f5a155e50 2024-09-19T05:14:35.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.156+0000 7f122f400640 1 --2- 172.21.15.43:0/3297234114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f5a14ac70 0x555f5a159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:35.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.156+0000 7f122fe00640 1 --2- 172.21.15.43:0/3297234114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f5a155e50 0x555f5a15a750 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f122802ade0 tx=0x7f122802a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74a03532b18a399e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:35.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.156+0000 7f121cc00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:14:35.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.156+0000 7f121cc00640 1 -- 172.21.15.43:0/3297234114 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1228058060 con 0x555f5a155e50 2024-09-19T05:14:35.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.156+0000 7f1238a29740 1 -- 172.21.15.43:0/3297234114 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f121800f9e0 con 0x555f5a155e50 2024-09-19T05:14:35.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.156+0000 7f121cc00640 1 -- 172.21.15.43:0/3297234114 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f12280046a0 con 0x555f5a155e50 2024-09-19T05:14:35.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.156+0000 7f121cc00640 1 -- 172.21.15.43:0/3297234114 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f122802f610 con 0x555f5a155e50 2024-09-19T05:14:35.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.157+0000 7f1238a29740 10 client.27617 fetch_fsmap learned FSMap version 29 2024-09-19T05:14:35.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.157+0000 7f1238a29740 1 -- 172.21.15.43:0/3297234114 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555f5a1d1070 con 0x555f5a155e50 2024-09-19T05:14:35.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.157+0000 7f121cc00640 1 -- 172.21.15.43:0/3297234114 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f121800f9e0 con 0x555f5a155e50 2024-09-19T05:14:35.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.157+0000 7f121cc00640 1 -- 172.21.15.43:0/3297234114 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1228052070 con 0x555f5a155e50 2024-09-19T05:14:35.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.157+0000 7f1238a29740 10 client.27617 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:14:35.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.157+0000 7f1238a29740 10 client.27617 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:14:35.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.157+0000 7f1238a29740 20 client.27617 populate_metadata read hostname 'smithi043' 2024-09-19T05:14:35.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.157+0000 7f1238a29740 1 --2- 172.21.15.43:0/3297234114 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x555f5a1d17a0 0x555f5a1d3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:35.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.157+0000 7f1238a29740 4 client.27617 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:14:35.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.157+0000 7f1238a29740 1 -- 172.21.15.43:0/3297234114 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555f5a0b8e60 con 0x555f5a1d17a0 2024-09-19T05:14:35.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.157+0000 7f122f400640 1 --2- 172.21.15.43:0/3297234114 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x555f5a1d17a0 0x555f5a1d3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:35.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.158+0000 7f122f400640 1 --2- 172.21.15.43:0/3297234114 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x555f5a1d17a0 0x555f5a1d3bc0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c97e6ec9714081eb server_cookie=82ad53e1f80936ea in_seq=0 out_seq=0 2024-09-19T05:14:35.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.158+0000 7f121cc00640 10 client.27617 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:14:35.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.158+0000 7f121cc00640 1 -- 172.21.15.43:0/3297234114 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555f5a0b8e60 con 0x555f5a1d17a0 2024-09-19T05:14:35.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.158+0000 7f121cc00640 10 client.27617 handle_command_reply: tid=0 2024-09-19T05:14:35.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.158+0000 7f1238a29740 1 client.27617 shutdown 2024-09-19T05:14:35.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.158+0000 7f1238a29740 1 -- 172.21.15.43:0/3297234114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f5a155e50 msgr2=0x555f5a15a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:35.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.158+0000 7f1238a29740 1 --2- 172.21.15.43:0/3297234114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f5a155e50 0x555f5a15a750 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f122802ade0 tx=0x7f122802a8c0 comp rx=0 tx=0).stop 2024-09-19T05:14:35.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.158+0000 7f1238a29740 1 -- 172.21.15.43:0/3297234114 shutdown_connections 2024-09-19T05:14:35.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.158+0000 7f1238a29740 1 --2- 172.21.15.43:0/3297234114 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x555f5a1d17a0 0x555f5a1d3bc0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:35.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.158+0000 7f1238a29740 1 --2- 172.21.15.43:0/3297234114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f5a155e50 0x555f5a15a750 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:35.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.158+0000 7f1238a29740 1 --2- 172.21.15.43:0/3297234114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5a14b590 0x555f5a15a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:35.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.158+0000 7f1238a29740 1 --2- 172.21.15.43:0/3297234114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f5a14ac70 0x555f5a159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:35.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.158+0000 7f121cc00640 0 client.27617 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:14:35.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.158+0000 7f1238a29740 1 -- 172.21.15.43:0/3297234114 >> 172.21.15.43:0/3297234114 conn(0x555f5a147200 msgr2=0x555f5a145b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:35.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.159+0000 7f1238a29740 1 -- 172.21.15.43:0/3297234114 shutdown_connections 2024-09-19T05:14:35.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.159+0000 7f1238a29740 1 -- 172.21.15.43:0/3297234114 wait complete. 2024-09-19T05:14:35.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.159+0000 7f1238a29740 20 client.27617 trim_cache size 0 max 16384 2024-09-19T05:14:35.165 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:14:35.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.159+0000 7f1236000640 1 -- 172.21.15.43:0/2428572387 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f12140a3430 msgr2=0x7f12140a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:35.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.159+0000 7f1236000640 1 --2- 172.21.15.43:0/2428572387 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f12140a3430 0x7f12140a58f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f122802f540 tx=0x7f122802f310 comp rx=0 tx=0).stop 2024-09-19T05:14:35.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.159+0000 7f1236000640 1 -- 172.21.15.43:0/2428572387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1230103ff0 msgr2=0x7f123019a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:35.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.159+0000 7f1236000640 1 --2- 172.21.15.43:0/2428572387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1230103ff0 0x7f123019a5b0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f122000e9f0 tx=0x7f122000eeb0 comp rx=0 tx=0).stop 2024-09-19T05:14:35.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.159+0000 7f1236000640 1 -- 172.21.15.43:0/2428572387 shutdown_connections 2024-09-19T05:14:35.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.159+0000 7f1236000640 1 --2- 172.21.15.43:0/2428572387 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f12140a3430 0x7f12140a58f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:35.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.159+0000 7f1236000640 1 --2- 172.21.15.43:0/2428572387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1230109120 0x7f123019ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:35.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.159+0000 7f1236000640 1 --2- 172.21.15.43:0/2428572387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12301049a0 0x7f123019aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:35.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.160+0000 7f1236000640 1 --2- 172.21.15.43:0/2428572387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1230103ff0 0x7f123019a5b0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:35.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.160+0000 7f1236000640 1 -- 172.21.15.43:0/2428572387 >> 172.21.15.43:0/2428572387 conn(0x7f12300ffa90 msgr2=0x7f1230101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:35.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.160+0000 7f1236000640 1 -- 172.21.15.43:0/2428572387 shutdown_connections 2024-09-19T05:14:35.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:35.160+0000 7f1236000640 1 -- 172.21.15.43:0/2428572387 wait complete. 2024-09-19T05:14:36.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:36 smithi151 ceph-mon[35425]: pgmap v309: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:14:36.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:36 smithi096 ceph-mon[35085]: pgmap v309: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:14:36.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:36 smithi043 ceph-mon[30800]: pgmap v309: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:14:37.137 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T05:14:37.138 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T05:14:37.530 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:14:38.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.167+0000 7f87ea600640 1 -- 172.21.15.43:0/2879674815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87e41049c0 msgr2=0x7f87e4108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:38.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.167+0000 7f87ea600640 1 --2- 172.21.15.43:0/2879674815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87e41049c0 0x7f87e4108a10 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f87d0009900 tx=0x7f87d002a410 comp rx=0 tx=0).stop 2024-09-19T05:14:38.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.167+0000 7f87ea600640 1 -- 172.21.15.43:0/2879674815 shutdown_connections 2024-09-19T05:14:38.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.167+0000 7f87ea600640 1 --2- 172.21.15.43:0/2879674815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87e4109140 0x7f87e4110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.167+0000 7f87ea600640 1 --2- 172.21.15.43:0/2879674815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87e41049c0 0x7f87e4108a10 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.167+0000 7f87ea600640 1 --2- 172.21.15.43:0/2879674815 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87e4104010 0x7f87e41043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.167+0000 7f87ea600640 1 -- 172.21.15.43:0/2879674815 >> 172.21.15.43:0/2879674815 conn(0x7f87e40ffa50 msgr2=0x7f87e4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:38.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.167+0000 7f87ea600640 1 -- 172.21.15.43:0/2879674815 shutdown_connections 2024-09-19T05:14:38.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.168+0000 7f87ea600640 1 -- 172.21.15.43:0/2879674815 wait complete. 2024-09-19T05:14:38.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.168+0000 7f87ea600640 1 Processor -- start 2024-09-19T05:14:38.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.169+0000 7f87ea600640 1 -- start start 2024-09-19T05:14:38.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.169+0000 7f87ea600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87e4104010 0x7f87e419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:38.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.169+0000 7f87ea600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87e41049c0 0x7f87e419abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:38.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.169+0000 7f87ea600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87e4109140 0x7f87e419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:38.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.169+0000 7f87ea600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f87e41141b0 con 0x7f87e4109140 2024-09-19T05:14:38.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.169+0000 7f87ea600640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f87e4114030 con 0x7f87e41049c0 2024-09-19T05:14:38.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.169+0000 7f87ea600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f87e4114330 con 0x7f87e4104010 2024-09-19T05:14:38.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.169+0000 7f87e3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87e4109140 0x7f87e419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:38.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.169+0000 7f87e3400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87e4104010 0x7f87e419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:38.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.169+0000 7f87e2a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87e41049c0 0x7f87e419abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:38.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.169+0000 7f87e3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87e4109140 0x7f87e419ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40990/0 (socket says 172.21.15.43:40990) 2024-09-19T05:14:38.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.169+0000 7f87e3400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87e4104010 0x7f87e419a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:38236/0 (socket says 172.21.15.43:38236) 2024-09-19T05:14:38.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.169+0000 7f87e3400640 1 -- 172.21.15.43:0/2396077321 learned_addr learned my addr 172.21.15.43:0/2396077321 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:38.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.169+0000 7f87e2a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87e41049c0 0x7f87e419abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:49776/0 (socket says 172.21.15.43:49776) 2024-09-19T05:14:38.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.170+0000 7f87e2a00640 1 -- 172.21.15.43:0/2396077321 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87e4104010 msgr2=0x7f87e419a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:38.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.170+0000 7f87e2a00640 1 --2- 172.21.15.43:0/2396077321 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87e4104010 0x7f87e419a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.170+0000 7f87e2a00640 1 -- 172.21.15.43:0/2396077321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87e4109140 msgr2=0x7f87e419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:38.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.170+0000 7f87e2a00640 1 --2- 172.21.15.43:0/2396077321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87e4109140 0x7f87e419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.170+0000 7f87e2a00640 1 -- 172.21.15.43:0/2396077321 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87e419f660 con 0x7f87e41049c0 2024-09-19T05:14:38.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.170+0000 7f87e3e00640 1 --2- 172.21.15.43:0/2396077321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87e4109140 0x7f87e419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:38.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.170+0000 7f87e3400640 1 --2- 172.21.15.43:0/2396077321 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87e4104010 0x7f87e419a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:38.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.170+0000 7f87e2a00640 1 --2- 172.21.15.43:0/2396077321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87e41049c0 0x7f87e419abf0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f87d002a930 tx=0x7f87d0003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:38.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.171+0000 7f87dbe00640 1 -- 172.21.15.43:0/2396077321 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d003f070 con 0x7f87e41049c0 2024-09-19T05:14:38.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.171+0000 7f87dbe00640 1 -- 172.21.15.43:0/2396077321 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87d0031c90 con 0x7f87e41049c0 2024-09-19T05:14:38.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.171+0000 7f87dbe00640 1 -- 172.21.15.43:0/2396077321 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d0035030 con 0x7f87e41049c0 2024-09-19T05:14:38.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.171+0000 7f87ea600640 1 -- 172.21.15.43:0/2396077321 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87e419f8f0 con 0x7f87e41049c0 2024-09-19T05:14:38.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.171+0000 7f87ea600640 1 -- 172.21.15.43:0/2396077321 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87e41a7230 con 0x7f87e41049c0 2024-09-19T05:14:38.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.175+0000 7f87ed042740 1 Processor -- start 2024-09-19T05:14:38.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.175+0000 7f87ed042740 1 -- start start 2024-09-19T05:14:38.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.175+0000 7f87ed042740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d6188ac0 0x55e4d6188ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:38.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.175+0000 7f87ed042740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d61893e0 0x55e4d6193760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:38.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.175+0000 7f87ed042740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d6193ca0 0x55e4d6196090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:38.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.176+0000 7f87ed042740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e4d60ef290 con 0x55e4d61893e0 2024-09-19T05:14:38.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.176+0000 7f87ed042740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e4d6096b40 con 0x55e4d6193ca0 2024-09-19T05:14:38.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.176+0000 7f87ed042740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e4d60eb250 con 0x55e4d6188ac0 2024-09-19T05:14:38.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.176+0000 7f87e3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d6188ac0 0x55e4d6188ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:38.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.176+0000 7f87e3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d61893e0 0x55e4d6193760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:38.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.176+0000 7f87e3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d6188ac0 0x55e4d6188ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:38246/0 (socket says 172.21.15.43:38246) 2024-09-19T05:14:38.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.176+0000 7f87e3e00640 1 -- 172.21.15.43:0/2055038858 learned_addr learned my addr 172.21.15.43:0/2055038858 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:38.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.176+0000 7f87dbe00640 1 -- 172.21.15.43:0/2396077321 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f87d0042070 con 0x7f87e41049c0 2024-09-19T05:14:38.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.176+0000 7f87e3e00640 1 -- 172.21.15.43:0/2055038858 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d6193ca0 msgr2=0x55e4d6196090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:38.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.176+0000 7f87e3e00640 1 --2- 172.21.15.43:0/2055038858 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d6193ca0 0x55e4d6196090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.176+0000 7f87e3e00640 1 -- 172.21.15.43:0/2055038858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d61893e0 msgr2=0x55e4d6193760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:38.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.176+0000 7f87e3e00640 1 --2- 172.21.15.43:0/2055038858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d61893e0 0x55e4d6193760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.176+0000 7f87e3e00640 1 -- 172.21.15.43:0/2055038858 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4d5e606a0 con 0x55e4d6188ac0 2024-09-19T05:14:38.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.177+0000 7f87e3e00640 1 --2- 172.21.15.43:0/2055038858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d6188ac0 0x55e4d6188ea0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f87dc0107a0 tx=0x7f87dc010c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=835e5bb9c238c26a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:38.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.177+0000 7f87d8c00640 1 -- 172.21.15.43:0/2055038858 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87dc003b70 con 0x55e4d6188ac0 2024-09-19T05:14:38.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.177+0000 7f87d8c00640 1 -- 172.21.15.43:0/2055038858 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87dc010e50 con 0x55e4d6188ac0 2024-09-19T05:14:38.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.177+0000 7f87d8c00640 1 -- 172.21.15.43:0/2055038858 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87dc004730 con 0x55e4d6188ac0 2024-09-19T05:14:38.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.177+0000 7f87dbe00640 1 --2- 172.21.15.43:0/2396077321 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f87c80a3430 0x7f87c80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:38.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.178+0000 7f87ed042740 1 -- 172.21.15.43:0/2055038858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d6188ac0 msgr2=0x55e4d6188ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:38.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.178+0000 7f87ed042740 1 --2- 172.21.15.43:0/2055038858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d6188ac0 0x55e4d6188ea0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f87dc0107a0 tx=0x7f87dc010c60 comp rx=0 tx=0).stop 2024-09-19T05:14:38.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.178+0000 7f87dbe00640 1 -- 172.21.15.43:0/2396077321 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f87d00f0340 con 0x7f87e41049c0 2024-09-19T05:14:38.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.178+0000 7f87ed042740 1 -- 172.21.15.43:0/2055038858 shutdown_connections 2024-09-19T05:14:38.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.178+0000 7f87ed042740 1 --2- 172.21.15.43:0/2055038858 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d6193ca0 0x55e4d6196090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.178+0000 7f87ed042740 1 --2- 172.21.15.43:0/2055038858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d61893e0 0x55e4d6193760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.178+0000 7f87ed042740 1 --2- 172.21.15.43:0/2055038858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d6188ac0 0x55e4d6188ea0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.178+0000 7f87ed042740 1 -- 172.21.15.43:0/2055038858 >> 172.21.15.43:0/2055038858 conn(0x55e4d6185050 msgr2=0x55e4d6185430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:38.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.178+0000 7f87ed042740 1 -- 172.21.15.43:0/2055038858 shutdown_connections 2024-09-19T05:14:38.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.178+0000 7f87e3400640 1 --2- 172.21.15.43:0/2396077321 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f87c80a3430 0x7f87c80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:38.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.178+0000 7f87ed042740 1 -- 172.21.15.43:0/2055038858 wait complete. 2024-09-19T05:14:38.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.179+0000 7f87ed042740 1 -- start start 2024-09-19T05:14:38.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.179+0000 7f87e3400640 1 --2- 172.21.15.43:0/2396077321 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f87c80a3430 0x7f87c80a58f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x55e4d6185eb0 tx=0x7f87d4005f90 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:38.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.179+0000 7f87ed042740 1 Processor -- start 2024-09-19T05:14:38.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.179+0000 7f87ed042740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d6188ac0 0x55e4d61936d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:38.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.179+0000 7f87ed042740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d61893e0 0x55e4d6190980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:38.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.179+0000 7f87ed042740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d6193ca0 0x55e4d6191110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:38.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.179+0000 7f87ed042740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e4d6196910 con 0x55e4d6193ca0 2024-09-19T05:14:38.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.179+0000 7f87ed042740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e4d60a9fa0 con 0x55e4d6188ac0 2024-09-19T05:14:38.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.179+0000 7f87ed042740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e4d6196a90 con 0x55e4d61893e0 2024-09-19T05:14:38.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.180+0000 7f87ed042740 10 client.0 _ll_register_callbacks cb 0x55e4d5dfdba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:38.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.180+0000 7f87e2a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d61893e0 0x55e4d6190980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:38.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.180+0000 7f87e3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d6193ca0 0x55e4d6191110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:38.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.180+0000 7f87e3400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d6188ac0 0x55e4d61936d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:38.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.180+0000 7f87e2a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d61893e0 0x55e4d6190980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:38260/0 (socket says 172.21.15.43:38260) 2024-09-19T05:14:38.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.180+0000 7f87e2a00640 1 -- 172.21.15.43:0/2762472755 learned_addr learned my addr 172.21.15.43:0/2762472755 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:38.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.180+0000 7f87e2a00640 1 -- 172.21.15.43:0/2762472755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d6188ac0 msgr2=0x55e4d61936d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:38.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.180+0000 7f87e2a00640 1 --2- 172.21.15.43:0/2762472755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d6188ac0 0x55e4d61936d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.180+0000 7f87e2a00640 1 -- 172.21.15.43:0/2762472755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d6193ca0 msgr2=0x55e4d6191110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:38.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.180+0000 7f87e2a00640 1 --2- 172.21.15.43:0/2762472755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d6193ca0 0x55e4d6191110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.180+0000 7f87e2a00640 1 -- 172.21.15.43:0/2762472755 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4d619fdb0 con 0x55e4d61893e0 2024-09-19T05:14:38.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.180+0000 7f87e3e00640 1 --2- 172.21.15.43:0/2762472755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d6193ca0 0x55e4d6191110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:38.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.180+0000 7f87e2a00640 1 --2- 172.21.15.43:0/2762472755 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d61893e0 0x55e4d6190980 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f87d00bc360 tx=0x7f87d0003f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ab171e904979292 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:38.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.180+0000 7f87cd600640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:14:38.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.181+0000 7f87cd600640 1 -- 172.21.15.43:0/2762472755 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d00ef030 con 0x55e4d61893e0 2024-09-19T05:14:38.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.181+0000 7f87cd600640 1 -- 172.21.15.43:0/2762472755 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f87d00b8cd0 con 0x55e4d61893e0 2024-09-19T05:14:38.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.181+0000 7f87ed042740 1 -- 172.21.15.43:0/2762472755 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87dc010e50 con 0x55e4d61893e0 2024-09-19T05:14:38.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.181+0000 7f87cd600640 1 -- 172.21.15.43:0/2762472755 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f87d0049040 con 0x55e4d61893e0 2024-09-19T05:14:38.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.181+0000 7f87ed042740 10 client.27529 fetch_fsmap learned FSMap version 29 2024-09-19T05:14:38.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.181+0000 7f87ed042740 1 -- 172.21.15.43:0/2762472755 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e4d61969c0 con 0x55e4d61893e0 2024-09-19T05:14:38.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.181+0000 7f87cd600640 1 -- 172.21.15.43:0/2762472755 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87dc010e50 con 0x55e4d61893e0 2024-09-19T05:14:38.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.181+0000 7f87cd600640 1 -- 172.21.15.43:0/2762472755 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f87d00bc8c0 con 0x55e4d61893e0 2024-09-19T05:14:38.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.181+0000 7f87ed042740 10 client.27529 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:14:38.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.182+0000 7f87ed042740 10 client.27529 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:14:38.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.182+0000 7f87ed042740 20 client.27529 populate_metadata read hostname 'smithi043' 2024-09-19T05:14:38.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.182+0000 7f87ed042740 1 --2- 172.21.15.43:0/2762472755 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55e4d61a3d80 0x55e4d61a61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:38.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.182+0000 7f87ed042740 4 client.27529 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:14:38.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.182+0000 7f87ed042740 1 -- 172.21.15.43:0/2762472755 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4d60fc800 con 0x55e4d61a3d80 2024-09-19T05:14:38.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.182+0000 7f87e3400640 1 --2- 172.21.15.43:0/2762472755 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55e4d61a3d80 0x55e4d61a61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:38.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.183+0000 7f87e3400640 1 --2- 172.21.15.43:0/2762472755 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55e4d61a3d80 0x55e4d61a61a0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c93430065dac9441 server_cookie=651652b7a4c97f7a in_seq=0 out_seq=0 2024-09-19T05:14:38.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.183+0000 7f87cd600640 10 client.27529 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:14:38.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.183+0000 7f87cd600640 1 -- 172.21.15.43:0/2762472755 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e4d60fc800 con 0x55e4d61a3d80 2024-09-19T05:14:38.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.183+0000 7f87cd600640 10 client.27529 handle_command_reply: tid=0 2024-09-19T05:14:38.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.183+0000 7f87ed042740 1 client.27529 shutdown 2024-09-19T05:14:38.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.184+0000 7f87ed042740 1 -- 172.21.15.43:0/2762472755 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d61893e0 msgr2=0x55e4d6190980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:38.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.184+0000 7f87ed042740 1 --2- 172.21.15.43:0/2762472755 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d61893e0 0x55e4d6190980 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f87d00bc360 tx=0x7f87d0003f90 comp rx=0 tx=0).stop 2024-09-19T05:14:38.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.184+0000 7f87ed042740 1 -- 172.21.15.43:0/2762472755 shutdown_connections 2024-09-19T05:14:38.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.184+0000 7f87ed042740 1 --2- 172.21.15.43:0/2762472755 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55e4d61a3d80 0x55e4d61a61a0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.184+0000 7f87ed042740 1 --2- 172.21.15.43:0/2762472755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d6193ca0 0x55e4d6191110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.184+0000 7f87ed042740 1 --2- 172.21.15.43:0/2762472755 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d61893e0 0x55e4d6190980 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.184+0000 7f87cd600640 0 client.27529 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:14:38.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.184+0000 7f87ed042740 1 --2- 172.21.15.43:0/2762472755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d6188ac0 0x55e4d61936d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.184+0000 7f87ed042740 1 -- 172.21.15.43:0/2762472755 >> 172.21.15.43:0/2762472755 conn(0x55e4d6185050 msgr2=0x55e4d618add0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:38.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.184+0000 7f87ed042740 1 -- 172.21.15.43:0/2762472755 shutdown_connections 2024-09-19T05:14:38.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.184+0000 7f87ed042740 1 -- 172.21.15.43:0/2762472755 wait complete. 2024-09-19T05:14:38.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.184+0000 7f87ed042740 20 client.27529 trim_cache size 0 max 16384 2024-09-19T05:14:38.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.202+0000 7f87ed042740 1 Processor -- start 2024-09-19T05:14:38.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.202+0000 7f87ed042740 1 -- start start 2024-09-19T05:14:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.203+0000 7f87ed042740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d6188ac0 0x55e4d618bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.203+0000 7f87ed042740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d61893e0 0x55e4d61931f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.203+0000 7f87ed042740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d6193ca0 0x55e4d6193730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.203+0000 7f87ed042740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f87e41afa90 con 0x55e4d61893e0 2024-09-19T05:14:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.203+0000 7f87ed042740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f87e41b15d0 con 0x55e4d6193ca0 2024-09-19T05:14:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.203+0000 7f87ed042740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f87e41b0160 con 0x55e4d6188ac0 2024-09-19T05:14:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.203+0000 7f87e2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d61893e0 0x55e4d61931f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.203+0000 7f87e3400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d6188ac0 0x55e4d618bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.203+0000 7f87e3e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d6193ca0 0x55e4d6193730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.203+0000 7f87e3400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d6188ac0 0x55e4d618bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:38264/0 (socket says 172.21.15.43:38264) 2024-09-19T05:14:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.203+0000 7f87e3400640 1 -- 172.21.15.43:0/1257492216 learned_addr learned my addr 172.21.15.43:0/1257492216 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:38.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.203+0000 7f87e3400640 1 -- 172.21.15.43:0/1257492216 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d6193ca0 msgr2=0x55e4d6193730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:38.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.204+0000 7f87e3400640 1 --2- 172.21.15.43:0/1257492216 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d6193ca0 0x55e4d6193730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.204+0000 7f87e3400640 1 -- 172.21.15.43:0/1257492216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d61893e0 msgr2=0x55e4d61931f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:38.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.204+0000 7f87e3400640 1 --2- 172.21.15.43:0/1257492216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d61893e0 0x55e4d61931f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.204+0000 7f87e3400640 1 -- 172.21.15.43:0/1257492216 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4d6186100 con 0x55e4d6188ac0 2024-09-19T05:14:38.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.204+0000 7f87e3e00640 1 --2- 172.21.15.43:0/1257492216 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d6193ca0 0x55e4d6193730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:38.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.204+0000 7f87e2a00640 1 --2- 172.21.15.43:0/1257492216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d61893e0 0x55e4d61931f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:38.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.204+0000 7f87e3400640 1 --2- 172.21.15.43:0/1257492216 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d6188ac0 0x55e4d618bd80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x55e4d61a69f0 tx=0x7f87d4017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6eaad7f969e8d462 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:38.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.204+0000 7f87cfe00640 1 -- 172.21.15.43:0/1257492216 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d401d8e0 con 0x55e4d6188ac0 2024-09-19T05:14:38.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.204+0000 7f87cfe00640 1 -- 172.21.15.43:0/1257492216 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87d4005020 con 0x55e4d6188ac0 2024-09-19T05:14:38.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.204+0000 7f87cfe00640 1 -- 172.21.15.43:0/1257492216 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d4004a90 con 0x55e4d6188ac0 2024-09-19T05:14:38.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.205+0000 7f87ed042740 1 -- 172.21.15.43:0/1257492216 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d6188ac0 msgr2=0x55e4d618bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:38.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.205+0000 7f87ed042740 1 --2- 172.21.15.43:0/1257492216 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d6188ac0 0x55e4d618bd80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x55e4d61a69f0 tx=0x7f87d4017000 comp rx=0 tx=0).stop 2024-09-19T05:14:38.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.205+0000 7f87ed042740 1 -- 172.21.15.43:0/1257492216 shutdown_connections 2024-09-19T05:14:38.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.205+0000 7f87ed042740 1 --2- 172.21.15.43:0/1257492216 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d6193ca0 0x55e4d6193730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.205+0000 7f87ed042740 1 --2- 172.21.15.43:0/1257492216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d61893e0 0x55e4d61931f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.205+0000 7f87ed042740 1 --2- 172.21.15.43:0/1257492216 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d6188ac0 0x55e4d618bd80 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.205+0000 7f87ed042740 1 -- 172.21.15.43:0/1257492216 >> 172.21.15.43:0/1257492216 conn(0x55e4d6185050 msgr2=0x55e4d618b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:38.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.205+0000 7f87ed042740 1 -- 172.21.15.43:0/1257492216 shutdown_connections 2024-09-19T05:14:38.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.205+0000 7f87ed042740 1 -- 172.21.15.43:0/1257492216 wait complete. 2024-09-19T05:14:38.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.205+0000 7f87ed042740 1 -- start start 2024-09-19T05:14:38.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.206+0000 7f87ed042740 1 Processor -- start 2024-09-19T05:14:38.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.206+0000 7f87ed042740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d6188ac0 0x55e4d618dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:38.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.206+0000 7f87ed042740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d61893e0 0x55e4d618e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:38.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.206+0000 7f87ed042740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d6193ca0 0x55e4d618e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:38.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.206+0000 7f87ed042740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f87e41b0f00 con 0x55e4d6193ca0 2024-09-19T05:14:38.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.206+0000 7f87ed042740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f87e41b0830 con 0x55e4d61893e0 2024-09-19T05:14:38.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.206+0000 7f87ed042740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f87e41ada90 con 0x55e4d6188ac0 2024-09-19T05:14:38.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.206+0000 7f87ed042740 10 client.0 _ll_register_callbacks cb 0x7f87d0009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:38.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.206+0000 7f87e3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d6193ca0 0x55e4d618e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:38.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.206+0000 7f87e3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d6193ca0 0x55e4d618e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41014/0 (socket says 172.21.15.43:41014) 2024-09-19T05:14:38.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.206+0000 7f87e3e00640 1 -- 172.21.15.43:0/2685212928 learned_addr learned my addr 172.21.15.43:0/2685212928 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:38.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.206+0000 7f87e3400640 1 --2- 172.21.15.43:0/2685212928 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d6188ac0 0x55e4d618dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:38.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.207+0000 7f87e2a00640 1 --2- 172.21.15.43:0/2685212928 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d61893e0 0x55e4d618e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:38.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.207+0000 7f87e3400640 1 -- 172.21.15.43:0/2685212928 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d61893e0 msgr2=0x55e4d618e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:38.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.207+0000 7f87e3400640 1 --2- 172.21.15.43:0/2685212928 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d61893e0 0x55e4d618e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.207+0000 7f87e3400640 1 -- 172.21.15.43:0/2685212928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d6193ca0 msgr2=0x55e4d618e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:38.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.207+0000 7f87e3400640 1 --2- 172.21.15.43:0/2685212928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d6193ca0 0x55e4d618e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.207+0000 7f87e3400640 1 -- 172.21.15.43:0/2685212928 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4d6185ce0 con 0x55e4d6188ac0 2024-09-19T05:14:38.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.207+0000 7f87e3e00640 1 --2- 172.21.15.43:0/2685212928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d6193ca0 0x55e4d618e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:38.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.207+0000 7f87e3400640 1 --2- 172.21.15.43:0/2685212928 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d6188ac0 0x55e4d618dbb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f87d4009640 tx=0x7f87d40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36a6f76e3ce1cccd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:38.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.207+0000 7f87cd600640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:14:38.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.208+0000 7f87cd600640 1 -- 172.21.15.43:0/2685212928 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d40053d0 con 0x55e4d6188ac0 2024-09-19T05:14:38.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.208+0000 7f87cd600640 1 -- 172.21.15.43:0/2685212928 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f87d4005570 con 0x55e4d6188ac0 2024-09-19T05:14:38.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.208+0000 7f87ed042740 1 -- 172.21.15.43:0/2685212928 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87d4005020 con 0x55e4d6188ac0 2024-09-19T05:14:38.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.208+0000 7f87cd600640 1 -- 172.21.15.43:0/2685212928 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f87d401d8e0 con 0x55e4d6188ac0 2024-09-19T05:14:38.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.208+0000 7f87ed042740 10 client.27535 fetch_fsmap learned FSMap version 29 2024-09-19T05:14:38.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.208+0000 7f87ed042740 1 -- 172.21.15.43:0/2685212928 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e4d620ed30 con 0x55e4d6188ac0 2024-09-19T05:14:38.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.208+0000 7f87cd600640 1 -- 172.21.15.43:0/2685212928 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87d4005020 con 0x55e4d6188ac0 2024-09-19T05:14:38.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.208+0000 7f87cd600640 1 -- 172.21.15.43:0/2685212928 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f87d4033480 con 0x55e4d6188ac0 2024-09-19T05:14:38.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.208+0000 7f87ed042740 10 client.27535 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:14:38.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.208+0000 7f87ed042740 10 client.27535 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:14:38.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.208+0000 7f87ed042740 20 client.27535 populate_metadata read hostname 'smithi043' 2024-09-19T05:14:38.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.208+0000 7f87ed042740 1 --2- 172.21.15.43:0/2685212928 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55e4d620f390 0x55e4d6211780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:38.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.208+0000 7f87ed042740 4 client.27535 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T05:14:38.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.208+0000 7f87ed042740 1 -- 172.21.15.43:0/2685212928 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e4d60f6d10 con 0x55e4d620f390 2024-09-19T05:14:38.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.209+0000 7f87e2a00640 1 --2- 172.21.15.43:0/2685212928 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55e4d620f390 0x55e4d6211780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:38.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.209+0000 7f87e2a00640 1 --2- 172.21.15.43:0/2685212928 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55e4d620f390 0x55e4d6211780 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f264cf15a729662 server_cookie=5ed8e41ee76f35d9 in_seq=0 out_seq=0 2024-09-19T05:14:38.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.209+0000 7f87cd600640 10 client.27535 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:14:38.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.210+0000 7f87cd600640 1 -- 172.21.15.43:0/2685212928 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e4d60f6d10 con 0x55e4d620f390 2024-09-19T05:14:38.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.210+0000 7f87cd600640 10 client.27535 handle_command_reply: tid=0 2024-09-19T05:14:38.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.210+0000 7f87ed042740 1 client.27535 shutdown 2024-09-19T05:14:38.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.210+0000 7f87ed042740 1 -- 172.21.15.43:0/2685212928 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d6188ac0 msgr2=0x55e4d618dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:38.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.210+0000 7f87ed042740 1 --2- 172.21.15.43:0/2685212928 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d6188ac0 0x55e4d618dbb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f87d4009640 tx=0x7f87d40046c0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.211+0000 7f87ed042740 1 -- 172.21.15.43:0/2685212928 shutdown_connections 2024-09-19T05:14:38.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.211+0000 7f87ed042740 1 --2- 172.21.15.43:0/2685212928 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55e4d620f390 0x55e4d6211780 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.211+0000 7f87ed042740 1 --2- 172.21.15.43:0/2685212928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e4d6193ca0 0x55e4d618e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.211+0000 7f87ed042740 1 --2- 172.21.15.43:0/2685212928 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e4d61893e0 0x55e4d618e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.211+0000 7f87ed042740 1 --2- 172.21.15.43:0/2685212928 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4d6188ac0 0x55e4d618dbb0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.211+0000 7f87ed042740 1 -- 172.21.15.43:0/2685212928 >> 172.21.15.43:0/2685212928 conn(0x55e4d6185050 msgr2=0x55e4d61839a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:38.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.211+0000 7f87cd600640 0 client.27535 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:14:38.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.211+0000 7f87ed042740 1 -- 172.21.15.43:0/2685212928 shutdown_connections 2024-09-19T05:14:38.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.211+0000 7f87ed042740 1 -- 172.21.15.43:0/2685212928 wait complete. 2024-09-19T05:14:38.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.211+0000 7f87ed042740 20 client.27535 trim_cache size 0 max 16384 2024-09-19T05:14:38.217 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:14:38.217 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-09-19T05:14:38.217 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "2d6a5d03-444e-429a-abf7-bb07dd2cc899", 2024-09-19T05:14:38.217 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-09-19T05:14:38.217 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:14:38.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.212+0000 7f87ea600640 1 -- 172.21.15.43:0/2396077321 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f87c80a3430 msgr2=0x7f87c80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:38.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.212+0000 7f87ea600640 1 --2- 172.21.15.43:0/2396077321 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f87c80a3430 0x7f87c80a58f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x55e4d6185eb0 tx=0x7f87d4005f90 comp rx=0 tx=0).stop 2024-09-19T05:14:38.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.212+0000 7f87ea600640 1 -- 172.21.15.43:0/2396077321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87e41049c0 msgr2=0x7f87e419abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:38.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.212+0000 7f87ea600640 1 --2- 172.21.15.43:0/2396077321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87e41049c0 0x7f87e419abf0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f87d002a930 tx=0x7f87d0003b90 comp rx=0 tx=0).stop 2024-09-19T05:14:38.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.212+0000 7f87ea600640 1 -- 172.21.15.43:0/2396077321 shutdown_connections 2024-09-19T05:14:38.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.212+0000 7f87ea600640 1 --2- 172.21.15.43:0/2396077321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87e4109140 0x7f87e419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.212+0000 7f87ea600640 1 --2- 172.21.15.43:0/2396077321 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f87c80a3430 0x7f87c80a58f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.212+0000 7f87ea600640 1 --2- 172.21.15.43:0/2396077321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87e41049c0 0x7f87e419abf0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.212+0000 7f87ea600640 1 --2- 172.21.15.43:0/2396077321 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87e4104010 0x7f87e419a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:38.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.212+0000 7f87ea600640 1 -- 172.21.15.43:0/2396077321 >> 172.21.15.43:0/2396077321 conn(0x7f87e40ffa50 msgr2=0x7f87e4100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:38.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.212+0000 7f87ea600640 1 -- 172.21.15.43:0/2396077321 shutdown_connections 2024-09-19T05:14:38.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:38.212+0000 7f87ea600640 1 -- 172.21.15.43:0/2396077321 wait complete. 2024-09-19T05:14:38.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:38 smithi151 ceph-mon[35425]: pgmap v310: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:14:38.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:38 smithi096 ceph-mon[35085]: pgmap v310: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:14:38.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:38 smithi043 ceph-mon[30800]: pgmap v310: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:14:39.053 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:14:39.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:39 smithi043 ceph-mon[30800]: scrub queued for path: / 2024-09-19T05:14:39.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:39 smithi043 ceph-mon[30800]: scrub summary: idle+waiting paths [/] 2024-09-19T05:14:39.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:39 smithi043 ceph-mon[30800]: scrub summary: active paths [/] 2024-09-19T05:14:39.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:39 smithi043 ceph-mon[30800]: scrub summary: idle+waiting paths [/] 2024-09-19T05:14:39.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:39 smithi151 ceph-mon[35425]: scrub queued for path: / 2024-09-19T05:14:39.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:39 smithi151 ceph-mon[35425]: scrub summary: idle+waiting paths [/] 2024-09-19T05:14:39.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:39 smithi151 ceph-mon[35425]: scrub summary: active paths [/] 2024-09-19T05:14:39.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:39 smithi151 ceph-mon[35425]: scrub summary: idle+waiting paths [/] 2024-09-19T05:14:39.435 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:14:39.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:39 smithi096 ceph-mon[35085]: scrub queued for path: / 2024-09-19T05:14:39.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:39 smithi096 ceph-mon[35085]: scrub summary: idle+waiting paths [/] 2024-09-19T05:14:39.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:39 smithi096 ceph-mon[35085]: scrub summary: active paths [/] 2024-09-19T05:14:39.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:39 smithi096 ceph-mon[35085]: scrub summary: idle+waiting paths [/] 2024-09-19T05:14:39.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.982+0000 7fe944a00640 1 -- 172.21.15.43:0/4281386029 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe940105d90 msgr2=0x7fe940106220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:39.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.982+0000 7fe944a00640 1 --2- 172.21.15.43:0/4281386029 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe940105d90 0x7fe940106220 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe938009be0 tx=0x7fe93802a740 comp rx=0 tx=0).stop 2024-09-19T05:14:39.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.982+0000 7fe944a00640 1 -- 172.21.15.43:0/4281386029 shutdown_connections 2024-09-19T05:14:39.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.982+0000 7fe944a00640 1 --2- 172.21.15.43:0/4281386029 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe940105d90 0x7fe940106220 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:39.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.982+0000 7fe944a00640 1 --2- 172.21.15.43:0/4281386029 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9401012a0 0x7fe940105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:39.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.982+0000 7fe944a00640 1 --2- 172.21.15.43:0/4281386029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe940100980 0x7fe940100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:39.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.982+0000 7fe944a00640 1 -- 172.21.15.43:0/4281386029 >> 172.21.15.43:0/4281386029 conn(0x7fe9400fc610 msgr2=0x7fe9400fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:39.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.982+0000 7fe944a00640 1 -- 172.21.15.43:0/4281386029 shutdown_connections 2024-09-19T05:14:39.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.982+0000 7fe944a00640 1 -- 172.21.15.43:0/4281386029 wait complete. 2024-09-19T05:14:39.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.983+0000 7fe944a00640 1 Processor -- start 2024-09-19T05:14:39.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.983+0000 7fe944a00640 1 -- start start 2024-09-19T05:14:39.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.984+0000 7fe944a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe940100980 0x7fe9401a45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:39.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.984+0000 7fe944a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9401012a0 0x7fe9401a4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:39.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.984+0000 7fe944a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe940105d90 0x7fe94019e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:39.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.984+0000 7fe944a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe940118370 con 0x7fe940100980 2024-09-19T05:14:39.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.984+0000 7fe944a00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe9401181f0 con 0x7fe940105d90 2024-09-19T05:14:39.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.984+0000 7fe944a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe9401184f0 con 0x7fe9401012a0 2024-09-19T05:14:39.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.984+0000 7fe93e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe940100980 0x7fe9401a45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:39.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.984+0000 7fe93e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe940100980 0x7fe9401a45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41024/0 (socket says 172.21.15.43:41024) 2024-09-19T05:14:39.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.984+0000 7fe93e800640 1 -- 172.21.15.43:0/1276940201 learned_addr learned my addr 172.21.15.43:0/1276940201 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:39.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.984+0000 7fe93f200640 1 --2- 172.21.15.43:0/1276940201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe940105d90 0x7fe94019e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:39.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.984+0000 7fe93de00640 1 --2- 172.21.15.43:0/1276940201 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9401012a0 0x7fe9401a4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:39.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.984+0000 7fe93e800640 1 -- 172.21.15.43:0/1276940201 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9401012a0 msgr2=0x7fe9401a4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:39.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.984+0000 7fe93e800640 1 --2- 172.21.15.43:0/1276940201 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9401012a0 0x7fe9401a4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:39.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.984+0000 7fe93e800640 1 -- 172.21.15.43:0/1276940201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe940105d90 msgr2=0x7fe94019e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:39.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.984+0000 7fe93e800640 1 --2- 172.21.15.43:0/1276940201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe940105d90 0x7fe94019e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:39.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.984+0000 7fe93e800640 1 -- 172.21.15.43:0/1276940201 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe94019ef10 con 0x7fe940100980 2024-09-19T05:14:39.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.985+0000 7fe93e800640 1 --2- 172.21.15.43:0/1276940201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe940100980 0x7fe9401a45d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fe93000e9f0 tx=0x7fe93000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:39.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.985+0000 7fe937400640 1 -- 172.21.15.43:0/1276940201 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93000cd80 con 0x7fe940100980 2024-09-19T05:14:39.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.985+0000 7fe944a00640 1 -- 172.21.15.43:0/1276940201 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe94019f200 con 0x7fe940100980 2024-09-19T05:14:39.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.985+0000 7fe937400640 1 -- 172.21.15.43:0/1276940201 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe930005500 con 0x7fe940100980 2024-09-19T05:14:39.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.985+0000 7fe937400640 1 -- 172.21.15.43:0/1276940201 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe930010690 con 0x7fe940100980 2024-09-19T05:14:39.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.985+0000 7fe944a00640 1 -- 172.21.15.43:0/1276940201 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9401ab550 con 0x7fe940100980 2024-09-19T05:14:39.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.988+0000 7fe947fba740 1 Processor -- start 2024-09-19T05:14:39.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.988+0000 7fe947fba740 1 -- start start 2024-09-19T05:14:39.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.988+0000 7fe937400640 1 -- 172.21.15.43:0/1276940201 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fe9300046c0 con 0x7fe940100980 2024-09-19T05:14:39.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.989+0000 7fe947fba740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460e0e5d50 0x56460e0e6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:39.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.989+0000 7fe947fba740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460e0e6670 0x56460e0f09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:39.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.989+0000 7fe947fba740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56460e0f0f30 0x56460e0f3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:39.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.989+0000 7fe947fba740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56460e04c3e0 con 0x56460e0f0f30 2024-09-19T05:14:39.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.989+0000 7fe947fba740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56460dff3d80 con 0x56460e0e6670 2024-09-19T05:14:39.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.989+0000 7fe947fba740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56460e0483a0 con 0x56460e0e5d50 2024-09-19T05:14:39.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.989+0000 7fe93e800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460e0e6670 0x56460e0f09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:39.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.989+0000 7fe93e800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460e0e6670 0x56460e0f09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:49820/0 (socket says 172.21.15.43:49820) 2024-09-19T05:14:39.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.989+0000 7fe93e800640 1 -- 172.21.15.43:0/3183688096 learned_addr learned my addr 172.21.15.43:0/3183688096 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:39.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.989+0000 7fe93f200640 1 --2- 172.21.15.43:0/3183688096 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460e0e5d50 0x56460e0e6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:39.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.989+0000 7fe93de00640 1 --2- 172.21.15.43:0/3183688096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56460e0f0f30 0x56460e0f3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:39.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.990+0000 7fe93e800640 1 -- 172.21.15.43:0/3183688096 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460e0e5d50 msgr2=0x56460e0e6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:39.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.990+0000 7fe93e800640 1 --2- 172.21.15.43:0/3183688096 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460e0e5d50 0x56460e0e6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:39.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.990+0000 7fe93e800640 1 -- 172.21.15.43:0/3183688096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56460e0f0f30 msgr2=0x56460e0f3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:39.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.990+0000 7fe93e800640 1 --2- 172.21.15.43:0/3183688096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56460e0f0f30 0x56460e0f3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:39.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.990+0000 7fe93e800640 1 -- 172.21.15.43:0/3183688096 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56460e012ae0 con 0x56460e0e6670 2024-09-19T05:14:39.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.990+0000 7fe93e800640 1 --2- 172.21.15.43:0/3183688096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460e0e6670 0x56460e0f09f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe9300cd730 tx=0x7fe9300d8220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1b16e2f08ae99a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:39.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.990+0000 7fe92b400640 1 -- 172.21.15.43:0/3183688096 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9300e3070 con 0x56460e0e6670 2024-09-19T05:14:39.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.990+0000 7fe92b400640 1 -- 172.21.15.43:0/3183688096 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9300d0500 con 0x56460e0e6670 2024-09-19T05:14:39.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.990+0000 7fe92b400640 1 -- 172.21.15.43:0/3183688096 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9300de660 con 0x56460e0e6670 2024-09-19T05:14:39.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.991+0000 7fe937400640 1 --2- 172.21.15.43:0/1276940201 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fe9240a3430 0x7fe9240a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:39.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.991+0000 7fe937400640 1 -- 172.21.15.43:0/1276940201 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7fe9300d0110 con 0x7fe940100980 2024-09-19T05:14:39.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.991+0000 7fe93f200640 1 --2- 172.21.15.43:0/1276940201 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fe9240a3430 0x7fe9240a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:39.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.991+0000 7fe947fba740 1 -- 172.21.15.43:0/3183688096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460e0e6670 msgr2=0x56460e0f09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:39.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.991+0000 7fe947fba740 1 --2- 172.21.15.43:0/3183688096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460e0e6670 0x56460e0f09f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe9300cd730 tx=0x7fe9300d8220 comp rx=0 tx=0).stop 2024-09-19T05:14:39.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.991+0000 7fe947fba740 1 -- 172.21.15.43:0/3183688096 shutdown_connections 2024-09-19T05:14:39.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.991+0000 7fe947fba740 1 --2- 172.21.15.43:0/3183688096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56460e0f0f30 0x56460e0f3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:39.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.991+0000 7fe947fba740 1 --2- 172.21.15.43:0/3183688096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460e0e6670 0x56460e0f09f0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:39.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.991+0000 7fe947fba740 1 --2- 172.21.15.43:0/3183688096 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460e0e5d50 0x56460e0e6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:39.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.991+0000 7fe947fba740 1 -- 172.21.15.43:0/3183688096 >> 172.21.15.43:0/3183688096 conn(0x56460e0e22e0 msgr2=0x56460e0e26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:39.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.991+0000 7fe947fba740 1 -- 172.21.15.43:0/3183688096 shutdown_connections 2024-09-19T05:14:39.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.992+0000 7fe947fba740 1 -- 172.21.15.43:0/3183688096 wait complete. 2024-09-19T05:14:39.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.992+0000 7fe93f200640 1 --2- 172.21.15.43:0/1276940201 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fe9240a3430 0x7fe9240a58f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe938009370 tx=0x7fe9380098a0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:39.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.992+0000 7fe947fba740 1 -- start start 2024-09-19T05:14:39.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.992+0000 7fe947fba740 1 Processor -- start 2024-09-19T05:14:39.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.993+0000 7fe947fba740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56460e0e5d50 0x56460e0eddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:39.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.993+0000 7fe947fba740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460e0e6670 0x56460e0ee310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:39.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.993+0000 7fe947fba740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460e0f0f30 0x56460e0ee850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:39.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.993+0000 7fe947fba740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56460e0f3ba0 con 0x56460e0e5d50 2024-09-19T05:14:39.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.993+0000 7fe947fba740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56460e0070f0 con 0x56460e0f0f30 2024-09-19T05:14:39.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.993+0000 7fe947fba740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56460e0f3d20 con 0x56460e0e6670 2024-09-19T05:14:39.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.993+0000 7fe947fba740 10 client.0 _ll_register_callbacks cb 0x56460dff89e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:39.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.993+0000 7fe93de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460e0e6670 0x56460e0ee310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:39.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.993+0000 7fe93f200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460e0f0f30 0x56460e0ee850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:39.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.993+0000 7fe93de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460e0e6670 0x56460e0ee310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:38294/0 (socket says 172.21.15.43:38294) 2024-09-19T05:14:39.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.993+0000 7fe93de00640 1 -- 172.21.15.43:0/75937672 learned_addr learned my addr 172.21.15.43:0/75937672 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:39.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.993+0000 7fe93e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56460e0e5d50 0x56460e0eddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:39.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.994+0000 7fe93de00640 1 -- 172.21.15.43:0/75937672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460e0f0f30 msgr2=0x56460e0ee850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:39.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.994+0000 7fe93de00640 1 --2- 172.21.15.43:0/75937672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460e0f0f30 0x56460e0ee850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:39.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.994+0000 7fe93de00640 1 -- 172.21.15.43:0/75937672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56460e0e5d50 msgr2=0x56460e0eddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:39.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.994+0000 7fe93de00640 1 --2- 172.21.15.43:0/75937672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56460e0e5d50 0x56460e0eddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:39.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.994+0000 7fe93de00640 1 -- 172.21.15.43:0/75937672 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56460ddbd3c0 con 0x56460e0e6670 2024-09-19T05:14:39.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.994+0000 7fe93f200640 1 --2- 172.21.15.43:0/75937672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460e0f0f30 0x56460e0ee850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:39.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.994+0000 7fe93de00640 1 --2- 172.21.15.43:0/75937672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460e0e6670 0x56460e0ee310 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe92c0116f0 tx=0x7fe92c011bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9708b49c9cf2b134 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:39.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.994+0000 7fe91fe00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:14:39.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.994+0000 7fe91fe00640 1 -- 172.21.15.43:0/75937672 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c01f060 con 0x56460e0e6670 2024-09-19T05:14:39.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.994+0000 7fe947fba740 1 -- 172.21.15.43:0/75937672 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9300d0500 con 0x56460e0e6670 2024-09-19T05:14:39.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.995+0000 7fe91fe00640 1 -- 172.21.15.43:0/75937672 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7fe92c004320 con 0x56460e0e6670 2024-09-19T05:14:40.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.995+0000 7fe91fe00640 1 -- 172.21.15.43:0/75937672 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe92c00f920 con 0x56460e0e6670 2024-09-19T05:14:40.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.995+0000 7fe947fba740 10 client.27547 fetch_fsmap learned FSMap version 29 2024-09-19T05:14:40.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.995+0000 7fe947fba740 1 -- 172.21.15.43:0/75937672 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56460e0ff880 con 0x56460e0e6670 2024-09-19T05:14:40.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.995+0000 7fe91fe00640 1 -- 172.21.15.43:0/75937672 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9300d0500 con 0x56460e0e6670 2024-09-19T05:14:40.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.995+0000 7fe91fe00640 1 -- 172.21.15.43:0/75937672 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe92c01a410 con 0x56460e0e6670 2024-09-19T05:14:40.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.995+0000 7fe947fba740 10 client.27547 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:14:40.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.995+0000 7fe947fba740 10 client.27547 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:14:40.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.996+0000 7fe947fba740 20 client.27547 populate_metadata read hostname 'smithi043' 2024-09-19T05:14:40.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.996+0000 7fe947fba740 1 --2- 172.21.15.43:0/75937672 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x56460e100c90 0x56460e1030c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:40.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.996+0000 7fe947fba740 4 client.27547 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:14:40.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.996+0000 7fe947fba740 1 -- 172.21.15.43:0/75937672 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56460e059950 con 0x56460e100c90 2024-09-19T05:14:40.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.996+0000 7fe93e800640 1 --2- 172.21.15.43:0/75937672 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x56460e100c90 0x56460e1030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:40.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.996+0000 7fe93e800640 1 --2- 172.21.15.43:0/75937672 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x56460e100c90 0x56460e1030c0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f70efe9c16a33ca6 server_cookie=d6107feed23fb427 in_seq=0 out_seq=0 2024-09-19T05:14:40.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.996+0000 7fe91fe00640 10 client.27547 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:14:40.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.997+0000 7fe91fe00640 1 -- 172.21.15.43:0/75937672 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56460e059950 con 0x56460e100c90 2024-09-19T05:14:40.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.997+0000 7fe91fe00640 10 client.27547 handle_command_reply: tid=0 2024-09-19T05:14:40.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.997+0000 7fe947fba740 1 client.27547 shutdown 2024-09-19T05:14:40.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.997+0000 7fe947fba740 1 -- 172.21.15.43:0/75937672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460e0e6670 msgr2=0x56460e0ee310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:40.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.997+0000 7fe947fba740 1 --2- 172.21.15.43:0/75937672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460e0e6670 0x56460e0ee310 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe92c0116f0 tx=0x7fe92c011bb0 comp rx=0 tx=0).stop 2024-09-19T05:14:40.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.998+0000 7fe947fba740 1 -- 172.21.15.43:0/75937672 shutdown_connections 2024-09-19T05:14:40.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.998+0000 7fe947fba740 1 --2- 172.21.15.43:0/75937672 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x56460e100c90 0x56460e1030c0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:40.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.998+0000 7fe947fba740 1 --2- 172.21.15.43:0/75937672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460e0f0f30 0x56460e0ee850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:40.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.998+0000 7fe947fba740 1 --2- 172.21.15.43:0/75937672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460e0e6670 0x56460e0ee310 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:40.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.998+0000 7fe947fba740 1 --2- 172.21.15.43:0/75937672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56460e0e5d50 0x56460e0eddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:40.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.998+0000 7fe91fe00640 0 client.27547 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:14:40.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.998+0000 7fe947fba740 1 -- 172.21.15.43:0/75937672 >> 172.21.15.43:0/75937672 conn(0x56460e0e22e0 msgr2=0x56460e0e7fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:40.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.998+0000 7fe947fba740 1 -- 172.21.15.43:0/75937672 shutdown_connections 2024-09-19T05:14:40.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.998+0000 7fe947fba740 1 -- 172.21.15.43:0/75937672 wait complete. 2024-09-19T05:14:40.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:39.998+0000 7fe947fba740 20 client.27547 trim_cache size 0 max 16384 2024-09-19T05:14:40.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.016+0000 7fe947fba740 1 Processor -- start 2024-09-19T05:14:40.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.016+0000 7fe947fba740 1 -- start start 2024-09-19T05:14:40.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.016+0000 7fe947fba740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460e0e5d50 0x56460e0dd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:40.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.016+0000 7fe947fba740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56460e0e6670 0x56460e0e7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:40.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.016+0000 7fe947fba740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460e0f0f30 0x56460e108990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:40.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.016+0000 7fe947fba740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe9401b3dc0 con 0x56460e0e6670 2024-09-19T05:14:40.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.016+0000 7fe947fba740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe9401b5900 con 0x56460e0e5d50 2024-09-19T05:14:40.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.016+0000 7fe947fba740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe9401b4490 con 0x56460e0f0f30 2024-09-19T05:14:40.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.016+0000 7fe93de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56460e0e6670 0x56460e0e7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:40.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.016+0000 7fe93de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56460e0e6670 0x56460e0e7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41054/0 (socket says 172.21.15.43:41054) 2024-09-19T05:14:40.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.016+0000 7fe93de00640 1 -- 172.21.15.43:0/3403050388 learned_addr learned my addr 172.21.15.43:0/3403050388 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:40.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.016+0000 7fe93e800640 1 --2- 172.21.15.43:0/3403050388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460e0e5d50 0x56460e0dd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:40.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.017+0000 7fe93f200640 1 --2- 172.21.15.43:0/3403050388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460e0f0f30 0x56460e108990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:40.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.017+0000 7fe93e800640 1 -- 172.21.15.43:0/3403050388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460e0f0f30 msgr2=0x56460e108990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:40.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.017+0000 7fe93e800640 1 --2- 172.21.15.43:0/3403050388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460e0f0f30 0x56460e108990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:40.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.017+0000 7fe93e800640 1 -- 172.21.15.43:0/3403050388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56460e0e6670 msgr2=0x56460e0e7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:40.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.017+0000 7fe93e800640 1 --2- 172.21.15.43:0/3403050388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56460e0e6670 0x56460e0e7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:40.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.017+0000 7fe93e800640 1 -- 172.21.15.43:0/3403050388 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56460e0e1b70 con 0x56460e0e5d50 2024-09-19T05:14:40.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.017+0000 7fe93de00640 1 --2- 172.21.15.43:0/3403050388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56460e0e6670 0x56460e0e7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:40.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.017+0000 7fe93e800640 1 --2- 172.21.15.43:0/3403050388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460e0e5d50 0x56460e0dd060 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe93000e9c0 tx=0x7fe9300d0f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=606df7d8259ed798 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:40.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.017+0000 7fe92aa00640 1 -- 172.21.15.43:0/3403050388 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93009c150 con 0x56460e0e5d50 2024-09-19T05:14:40.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.017+0000 7fe92aa00640 1 -- 172.21.15.43:0/3403050388 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9300d36f0 con 0x56460e0e5d50 2024-09-19T05:14:40.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.017+0000 7fe92aa00640 1 -- 172.21.15.43:0/3403050388 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93009bce0 con 0x56460e0e5d50 2024-09-19T05:14:40.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.018+0000 7fe947fba740 1 -- 172.21.15.43:0/3403050388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460e0e5d50 msgr2=0x56460e0dd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:40.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.018+0000 7fe947fba740 1 --2- 172.21.15.43:0/3403050388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460e0e5d50 0x56460e0dd060 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe93000e9c0 tx=0x7fe9300d0f00 comp rx=0 tx=0).stop 2024-09-19T05:14:40.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.018+0000 7fe947fba740 1 -- 172.21.15.43:0/3403050388 shutdown_connections 2024-09-19T05:14:40.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.018+0000 7fe947fba740 1 --2- 172.21.15.43:0/3403050388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460e0f0f30 0x56460e108990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:40.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.018+0000 7fe947fba740 1 --2- 172.21.15.43:0/3403050388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56460e0e6670 0x56460e0e7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:40.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.018+0000 7fe947fba740 1 --2- 172.21.15.43:0/3403050388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460e0e5d50 0x56460e0dd060 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:40.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.018+0000 7fe947fba740 1 -- 172.21.15.43:0/3403050388 >> 172.21.15.43:0/3403050388 conn(0x56460e0e22e0 msgr2=0x56460e0e77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:40.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.018+0000 7fe947fba740 1 -- 172.21.15.43:0/3403050388 shutdown_connections 2024-09-19T05:14:40.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.018+0000 7fe947fba740 1 -- 172.21.15.43:0/3403050388 wait complete. 2024-09-19T05:14:40.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.019+0000 7fe947fba740 1 -- start start 2024-09-19T05:14:40.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.019+0000 7fe947fba740 1 Processor -- start 2024-09-19T05:14:40.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.020+0000 7fe947fba740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460e0e5d50 0x56460e0f5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:40.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.020+0000 7fe947fba740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56460e0e6670 0x56460e0f5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:40.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.020+0000 7fe947fba740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460e0f0f30 0x56460e16a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:40.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.020+0000 7fe947fba740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe9401b5230 con 0x56460e0e6670 2024-09-19T05:14:40.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.020+0000 7fe947fba740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe9401b4b60 con 0x56460e0e5d50 2024-09-19T05:14:40.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.020+0000 7fe947fba740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe9401b1dc0 con 0x56460e0f0f30 2024-09-19T05:14:40.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.020+0000 7fe947fba740 10 client.0 _ll_register_callbacks cb 0x7fe92c009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:14:40.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.020+0000 7fe93de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56460e0e6670 0x56460e0f5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:40.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.020+0000 7fe93f200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460e0f0f30 0x56460e16a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:40.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.020+0000 7fe93e800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460e0e5d50 0x56460e0f5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:40.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.020+0000 7fe93de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56460e0e6670 0x56460e0f5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41068/0 (socket says 172.21.15.43:41068) 2024-09-19T05:14:40.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.020+0000 7fe93de00640 1 -- 172.21.15.43:0/3347957704 learned_addr learned my addr 172.21.15.43:0/3347957704 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:40.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.020+0000 7fe93f200640 1 -- 172.21.15.43:0/3347957704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460e0e5d50 msgr2=0x56460e0f5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:40.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.020+0000 7fe93f200640 1 --2- 172.21.15.43:0/3347957704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460e0e5d50 0x56460e0f5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:40.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.020+0000 7fe93f200640 1 -- 172.21.15.43:0/3347957704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56460e0e6670 msgr2=0x56460e0f5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:14:40.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.020+0000 7fe93f200640 1 --2- 172.21.15.43:0/3347957704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56460e0e6670 0x56460e0f5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:40.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.020+0000 7fe93f200640 1 -- 172.21.15.43:0/3347957704 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56460e16afa0 con 0x56460e0f0f30 2024-09-19T05:14:40.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.020+0000 7fe93de00640 1 --2- 172.21.15.43:0/3347957704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56460e0e6670 0x56460e0f5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:40.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.021+0000 7fe93f200640 1 --2- 172.21.15.43:0/3347957704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460e0f0f30 0x56460e16a960 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe93802ac60 tx=0x7fe938009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59836a661f70978c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:40.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.021+0000 7fe91fe00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:14:40.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.021+0000 7fe91fe00640 1 -- 172.21.15.43:0/3347957704 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe938036050 con 0x56460e0f0f30 2024-09-19T05:14:40.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.021+0000 7fe947fba740 1 -- 172.21.15.43:0/3347957704 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56460e053e60 con 0x56460e0f0f30 2024-09-19T05:14:40.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.021+0000 7fe91fe00640 1 -- 172.21.15.43:0/3347957704 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7fe9380041d0 con 0x56460e0f0f30 2024-09-19T05:14:40.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.022+0000 7fe91fe00640 1 -- 172.21.15.43:0/3347957704 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe9380043d0 con 0x56460e0f0f30 2024-09-19T05:14:40.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.022+0000 7fe947fba740 10 client.27553 fetch_fsmap learned FSMap version 29 2024-09-19T05:14:40.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.022+0000 7fe947fba740 1 -- 172.21.15.43:0/3347957704 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56460e16c2e0 con 0x56460e0f0f30 2024-09-19T05:14:40.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.022+0000 7fe91fe00640 1 -- 172.21.15.43:0/3347957704 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56460e053e60 con 0x56460e0f0f30 2024-09-19T05:14:40.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.022+0000 7fe91fe00640 1 -- 172.21.15.43:0/3347957704 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe93803c040 con 0x56460e0f0f30 2024-09-19T05:14:40.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.022+0000 7fe947fba740 10 client.27553 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:14:40.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.022+0000 7fe947fba740 10 client.27553 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:14:40.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.022+0000 7fe947fba740 20 client.27553 populate_metadata read hostname 'smithi043' 2024-09-19T05:14:40.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.022+0000 7fe947fba740 1 --2- 172.21.15.43:0/3347957704 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x56460e16ca60 0x56460e16ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:40.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.022+0000 7fe947fba740 4 client.27553 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:14:40.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.022+0000 7fe947fba740 1 -- 172.21.15.43:0/3347957704 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56460dfd00c0 con 0x56460e16ca60 2024-09-19T05:14:40.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.023+0000 7fe93e800640 1 --2- 172.21.15.43:0/3347957704 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x56460e16ca60 0x56460e16ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:40.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.023+0000 7fe93e800640 1 --2- 172.21.15.43:0/3347957704 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x56460e16ca60 0x56460e16ee80 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=535ebe5194ca631 server_cookie=631ff271004d9596 in_seq=0 out_seq=0 2024-09-19T05:14:40.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.023+0000 7fe91fe00640 10 client.27553 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:14:40.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.023+0000 7fe91fe00640 1 -- 172.21.15.43:0/3347957704 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56460dfd00c0 con 0x56460e16ca60 2024-09-19T05:14:40.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.023+0000 7fe91fe00640 10 client.27553 handle_command_reply: tid=0 2024-09-19T05:14:40.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.023+0000 7fe947fba740 1 client.27553 shutdown 2024-09-19T05:14:40.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.024+0000 7fe947fba740 1 -- 172.21.15.43:0/3347957704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460e0f0f30 msgr2=0x56460e16a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:40.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.024+0000 7fe947fba740 1 --2- 172.21.15.43:0/3347957704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460e0f0f30 0x56460e16a960 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe93802ac60 tx=0x7fe938009500 comp rx=0 tx=0).stop 2024-09-19T05:14:40.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.024+0000 7fe947fba740 1 -- 172.21.15.43:0/3347957704 shutdown_connections 2024-09-19T05:14:40.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.024+0000 7fe947fba740 1 --2- 172.21.15.43:0/3347957704 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x56460e16ca60 0x56460e16ee80 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:40.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.024+0000 7fe947fba740 1 --2- 172.21.15.43:0/3347957704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460e0f0f30 0x56460e16a960 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:40.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.024+0000 7fe947fba740 1 --2- 172.21.15.43:0/3347957704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56460e0e6670 0x56460e0f5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:40.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.024+0000 7fe947fba740 1 --2- 172.21.15.43:0/3347957704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460e0e5d50 0x56460e0f5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:40.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.024+0000 7fe91fe00640 0 client.27553 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:14:40.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.024+0000 7fe947fba740 1 -- 172.21.15.43:0/3347957704 >> 172.21.15.43:0/3347957704 conn(0x56460e0e22e0 msgr2=0x56460e0e7fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:40.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.024+0000 7fe947fba740 1 -- 172.21.15.43:0/3347957704 shutdown_connections 2024-09-19T05:14:40.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.024+0000 7fe947fba740 1 -- 172.21.15.43:0/3347957704 wait complete. 2024-09-19T05:14:40.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.024+0000 7fe947fba740 20 client.27553 trim_cache size 0 max 16384 2024-09-19T05:14:40.028 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:14:40.028 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T05:14:40.029 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-09-19T05:14:40.029 INFO:teuthology.orchestra.run.smithi043.stdout: "2d6a5d03-444e-429a-abf7-bb07dd2cc899": { 2024-09-19T05:14:40.029 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-09-19T05:14:40.029 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "2d6a5d03-444e-429a-abf7-bb07dd2cc899", 2024-09-19T05:14:40.029 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-09-19T05:14:40.029 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:14:40.029 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:14:40.029 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:14:40.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.025+0000 7fe944a00640 1 -- 172.21.15.43:0/1276940201 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fe9240a3430 msgr2=0x7fe9240a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:40.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.025+0000 7fe944a00640 1 --2- 172.21.15.43:0/1276940201 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fe9240a3430 0x7fe9240a58f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe938009370 tx=0x7fe9380098a0 comp rx=0 tx=0).stop 2024-09-19T05:14:40.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.025+0000 7fe944a00640 1 -- 172.21.15.43:0/1276940201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe940100980 msgr2=0x7fe9401a45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:40.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.025+0000 7fe944a00640 1 --2- 172.21.15.43:0/1276940201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe940100980 0x7fe9401a45d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fe93000e9f0 tx=0x7fe93000edf0 comp rx=0 tx=0).stop 2024-09-19T05:14:40.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.025+0000 7fe944a00640 1 -- 172.21.15.43:0/1276940201 shutdown_connections 2024-09-19T05:14:40.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.025+0000 7fe944a00640 1 --2- 172.21.15.43:0/1276940201 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fe9240a3430 0x7fe9240a58f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:40.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.025+0000 7fe944a00640 1 --2- 172.21.15.43:0/1276940201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe940105d90 0x7fe94019e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:40.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.025+0000 7fe944a00640 1 --2- 172.21.15.43:0/1276940201 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9401012a0 0x7fe9401a4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:40.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.025+0000 7fe944a00640 1 --2- 172.21.15.43:0/1276940201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe940100980 0x7fe9401a45d0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:40.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.025+0000 7fe944a00640 1 -- 172.21.15.43:0/1276940201 >> 172.21.15.43:0/1276940201 conn(0x7fe9400fc610 msgr2=0x7fe94010f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:40.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.025+0000 7fe944a00640 1 -- 172.21.15.43:0/1276940201 shutdown_connections 2024-09-19T05:14:40.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:40.025+0000 7fe944a00640 1 -- 172.21.15.43:0/1276940201 wait complete. 2024-09-19T05:14:40.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:40 smithi043 ceph-mon[30800]: pgmap v311: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:14:40.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:40 smithi151 ceph-mon[35425]: pgmap v311: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:14:40.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:40 smithi096 ceph-mon[35085]: pgmap v311: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:14:40.969 INFO:tasks.cephfs.filesystem:scrub status for tag:2d6a5d03-444e-429a-abf7-bb07dd2cc899 - {'path': '/', 'tag': '2d6a5d03-444e-429a-abf7-bb07dd2cc899', 'options': 'recursive,force'} 2024-09-19T05:14:42.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:42 smithi151 ceph-mon[35425]: pgmap v312: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:14:42.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:42 smithi096 ceph-mon[35085]: pgmap v312: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:14:42.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:42 smithi043 ceph-mon[30800]: pgmap v312: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:14:43.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:43 smithi151 ceph-mon[35425]: scrub summary: idle 2024-09-19T05:14:43.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:43 smithi096 ceph-mon[35085]: scrub summary: idle 2024-09-19T05:14:43.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:43 smithi043 ceph-mon[30800]: scrub summary: idle 2024-09-19T05:14:44.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:44 smithi151 ceph-mon[35425]: pgmap v313: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:14:44.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:44 smithi096 ceph-mon[35085]: pgmap v313: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:14:44.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:44 smithi043 ceph-mon[30800]: pgmap v313: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:14:46.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:46 smithi151 ceph-mon[35425]: pgmap v314: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:14:46.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:46 smithi096 ceph-mon[35085]: pgmap v314: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:14:46.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:46 smithi043 ceph-mon[30800]: pgmap v314: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:14:48.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:48 smithi151 ceph-mon[35425]: pgmap v315: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:14:48.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:48 smithi096 ceph-mon[35085]: pgmap v315: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:14:48.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:48 smithi043 ceph-mon[30800]: pgmap v315: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:14:50.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:50 smithi151 ceph-mon[35425]: pgmap v316: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:14:50.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:50 smithi096 ceph-mon[35085]: pgmap v316: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:14:50.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:50 smithi043 ceph-mon[30800]: pgmap v316: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T05:14:51.936 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-19T05:14:52.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.104+0000 7f5474200640 1 -- 172.21.15.43:0/3660027093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f546c112af0 msgr2=0x7f546c114ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:52.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.104+0000 7f5474200640 1 --2- 172.21.15.43:0/3660027093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f546c112af0 0x7f546c114ee0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f5464009ca0 tx=0x7f546402a950 comp rx=0 tx=0).stop 2024-09-19T05:14:52.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.104+0000 7f5474200640 1 -- 172.21.15.43:0/3660027093 shutdown_connections 2024-09-19T05:14:52.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.104+0000 7f5474200640 1 --2- 172.21.15.43:0/3660027093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f546c112af0 0x7f546c114ee0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.104+0000 7f5474200640 1 --2- 172.21.15.43:0/3660027093 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f546c074eb0 0x7f546c1125b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.104+0000 7f5474200640 1 --2- 172.21.15.43:0/3660027093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f546c074590 0x7f546c074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.104+0000 7f5474200640 1 -- 172.21.15.43:0/3660027093 >> 172.21.15.43:0/3660027093 conn(0x7f546c0fd7e0 msgr2=0x7f546c0ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:52.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.104+0000 7f5474200640 1 -- 172.21.15.43:0/3660027093 shutdown_connections 2024-09-19T05:14:52.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.104+0000 7f5474200640 1 -- 172.21.15.43:0/3660027093 wait complete. 2024-09-19T05:14:52.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.105+0000 7f5474200640 1 Processor -- start 2024-09-19T05:14:52.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.105+0000 7f5474200640 1 -- start start 2024-09-19T05:14:52.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.105+0000 7f5474200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f546c074590 0x7f546c1a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:52.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.105+0000 7f5474200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f546c074eb0 0x7f546c1a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:52.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.105+0000 7f5474200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f546c112af0 0x7f546c19e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:52.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.105+0000 7f5474200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f546c118500 con 0x7f546c074eb0 2024-09-19T05:14:52.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.105+0000 7f5474200640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f546c118380 con 0x7f546c074590 2024-09-19T05:14:52.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.105+0000 7f5474200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f546c118680 con 0x7f546c112af0 2024-09-19T05:14:52.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.105+0000 7f5470e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f546c074eb0 0x7f546c1a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:52.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.105+0000 7f5470e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f546c074eb0 0x7f546c1a4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47684/0 (socket says 172.21.15.43:47684) 2024-09-19T05:14:52.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.105+0000 7f5470e00640 1 -- 172.21.15.43:0/1630220745 learned_addr learned my addr 172.21.15.43:0/1630220745 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:52.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.106+0000 7f5471800640 1 --2- 172.21.15.43:0/1630220745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f546c074590 0x7f546c1a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:52.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.106+0000 7f5472200640 1 --2- 172.21.15.43:0/1630220745 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f546c112af0 0x7f546c19e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:52.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.106+0000 7f5471800640 1 -- 172.21.15.43:0/1630220745 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f546c112af0 msgr2=0x7f546c19e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:52.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.106+0000 7f5471800640 1 --2- 172.21.15.43:0/1630220745 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f546c112af0 0x7f546c19e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.106+0000 7f5471800640 1 -- 172.21.15.43:0/1630220745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f546c074eb0 msgr2=0x7f546c1a4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:52.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.106+0000 7f5471800640 1 --2- 172.21.15.43:0/1630220745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f546c074eb0 0x7f546c1a4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.106+0000 7f5471800640 1 -- 172.21.15.43:0/1630220745 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f546c19f020 con 0x7f546c074590 2024-09-19T05:14:52.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.106+0000 7f5470e00640 1 --2- 172.21.15.43:0/1630220745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f546c074eb0 0x7f546c1a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:52.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.106+0000 7f5471800640 1 --2- 172.21.15.43:0/1630220745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f546c074590 0x7f546c1a46a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f546000e9f0 tx=0x7f546000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:52.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.106+0000 7f546a000640 1 -- 172.21.15.43:0/1630220745 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546000ccc0 con 0x7f546c074590 2024-09-19T05:14:52.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.107+0000 7f546a000640 1 -- 172.21.15.43:0/1630220745 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5460010910 con 0x7f546c074590 2024-09-19T05:14:52.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.107+0000 7f5474200640 1 -- 172.21.15.43:0/1630220745 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f546c19f310 con 0x7f546c074590 2024-09-19T05:14:52.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.107+0000 7f546a000640 1 -- 172.21.15.43:0/1630220745 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5460018960 con 0x7f546c074590 2024-09-19T05:14:52.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.107+0000 7f5474200640 1 -- 172.21.15.43:0/1630220745 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f546c1ab620 con 0x7f546c074590 2024-09-19T05:14:52.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.108+0000 7f5474200640 1 -- 172.21.15.43:0/1630220745 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f546c075cd0 con 0x7f546c074590 2024-09-19T05:14:52.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.110+0000 7f546a000640 1 -- 172.21.15.43:0/1630220745 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f5460010ac0 con 0x7f546c074590 2024-09-19T05:14:52.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.111+0000 7f546a000640 1 --2- 172.21.15.43:0/1630220745 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f54540a3460 0x7f54540a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:52.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.111+0000 7f546a000640 1 -- 172.21.15.43:0/1630220745 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f54600d01b0 con 0x7f546c074590 2024-09-19T05:14:52.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.111+0000 7f5470e00640 1 --2- 172.21.15.43:0/1630220745 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f54540a3460 0x7f54540a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:52.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.112+0000 7f5470e00640 1 --2- 172.21.15.43:0/1630220745 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f54540a3460 0x7f54540a5920 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f546c19ffb0 tx=0x7f5458005f90 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:52.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.113+0000 7f546a000640 1 -- 172.21.15.43:0/1630220745 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f546009c530 con 0x7f546c074590 2024-09-19T05:14:52.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:52 smithi043 ceph-mon[30800]: pgmap v317: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:14:52.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.319+0000 7f5474200640 1 -- 172.21.15.43:0/1630220745 --> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f546c0ff460 con 0x7f54540a3460 2024-09-19T05:14:52.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.328+0000 7f546a000640 1 -- 172.21.15.43:0/1630220745 <== mgr.25136 v2:172.21.15.43:6800/2120444093 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f546c0ff460 con 0x7f54540a3460 2024-09-19T05:14:52.329 INFO:teuthology.orchestra.run.smithi043.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-19T00:00:00", "created": "2024-09-19T05:04:50", "first": "2024-09-19T05:05:00", "last": "2024-09-19T05:14:00", "last_pruned": "2024-09-19T05:14:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-09-19T05:14:52.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.330+0000 7f5474200640 1 -- 172.21.15.43:0/1630220745 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f54540a3460 msgr2=0x7f54540a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:52.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.330+0000 7f5474200640 1 --2- 172.21.15.43:0/1630220745 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f54540a3460 0x7f54540a5920 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f546c19ffb0 tx=0x7f5458005f90 comp rx=0 tx=0).stop 2024-09-19T05:14:52.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.330+0000 7f5474200640 1 -- 172.21.15.43:0/1630220745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f546c074590 msgr2=0x7f546c1a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:52.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.330+0000 7f5474200640 1 --2- 172.21.15.43:0/1630220745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f546c074590 0x7f546c1a46a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f546000e9f0 tx=0x7f546000eeb0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.330+0000 7f5474200640 1 -- 172.21.15.43:0/1630220745 shutdown_connections 2024-09-19T05:14:52.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.330+0000 7f5474200640 1 --2- 172.21.15.43:0/1630220745 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f546c112af0 0x7f546c19e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.330+0000 7f5474200640 1 --2- 172.21.15.43:0/1630220745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f546c074eb0 0x7f546c1a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.330+0000 7f5474200640 1 --2- 172.21.15.43:0/1630220745 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f54540a3460 0x7f54540a5920 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.330+0000 7f5474200640 1 --2- 172.21.15.43:0/1630220745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f546c074590 0x7f546c1a46a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.330+0000 7f5474200640 1 -- 172.21.15.43:0/1630220745 >> 172.21.15.43:0/1630220745 conn(0x7f546c0fd7e0 msgr2=0x7f546c0fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:52.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.330+0000 7f5474200640 1 -- 172.21.15.43:0/1630220745 shutdown_connections 2024-09-19T05:14:52.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.330+0000 7f5474200640 1 -- 172.21.15.43:0/1630220745 wait complete. 2024-09-19T05:14:52.348 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-19T05:14:52.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:52 smithi151 ceph-mon[35425]: pgmap v317: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:14:52.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:52 smithi096 ceph-mon[35085]: pgmap v317: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-19T05:14:52.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.504+0000 7fcfea600640 1 -- 172.21.15.43:0/831482442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfe4103f50 msgr2=0x7fcfe4104330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:52.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.504+0000 7fcfea600640 1 --2- 172.21.15.43:0/831482442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfe4103f50 0x7fcfe4104330 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4009900 tx=0x7fcfd402a410 comp rx=0 tx=0).stop 2024-09-19T05:14:52.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.504+0000 7fcfea600640 1 -- 172.21.15.43:0/831482442 shutdown_connections 2024-09-19T05:14:52.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.504+0000 7fcfea600640 1 --2- 172.21.15.43:0/831482442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfe41090a0 0x7fcfe4110bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.504+0000 7fcfea600640 1 --2- 172.21.15.43:0/831482442 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfe4104900 0x7fcfe4108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.504+0000 7fcfea600640 1 --2- 172.21.15.43:0/831482442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfe4103f50 0x7fcfe4104330 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.504+0000 7fcfea600640 1 -- 172.21.15.43:0/831482442 >> 172.21.15.43:0/831482442 conn(0x7fcfe40ff9e0 msgr2=0x7fcfe4101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:52.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.504+0000 7fcfea600640 1 -- 172.21.15.43:0/831482442 shutdown_connections 2024-09-19T05:14:52.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.504+0000 7fcfea600640 1 -- 172.21.15.43:0/831482442 wait complete. 2024-09-19T05:14:52.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.505+0000 7fcfea600640 1 Processor -- start 2024-09-19T05:14:52.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.505+0000 7fcfea600640 1 -- start start 2024-09-19T05:14:52.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.505+0000 7fcfea600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfe4103f50 0x7fcfe419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:52.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.505+0000 7fcfea600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfe41090a0 0x7fcfe419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:52.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.505+0000 7fcfea600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfe419ef20 0x7fcfe419f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:52.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.505+0000 7fcfea600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe41140d0 con 0x7fcfe41090a0 2024-09-19T05:14:52.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.505+0000 7fcfea600640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe4113f50 con 0x7fcfe4103f50 2024-09-19T05:14:52.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.505+0000 7fcfea600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe4114250 con 0x7fcfe419ef20 2024-09-19T05:14:52.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.506+0000 7fcfdbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfe41090a0 0x7fcfe419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:52.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.506+0000 7fcfe3400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfe4103f50 0x7fcfe419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:52.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.506+0000 7fcfe3400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfe4103f50 0x7fcfe419a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:43542/0 (socket says 172.21.15.43:43542) 2024-09-19T05:14:52.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.506+0000 7fcfe3400640 1 -- 172.21.15.43:0/1570492823 learned_addr learned my addr 172.21.15.43:0/1570492823 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:14:52.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.506+0000 7fcfe3e00640 1 --2- 172.21.15.43:0/1570492823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfe419ef20 0x7fcfe419f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:52.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.506+0000 7fcfe3400640 1 -- 172.21.15.43:0/1570492823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfe419ef20 msgr2=0x7fcfe419f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:52.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.506+0000 7fcfe3400640 1 --2- 172.21.15.43:0/1570492823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfe419ef20 0x7fcfe419f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.506+0000 7fcfe3400640 1 -- 172.21.15.43:0/1570492823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfe41090a0 msgr2=0x7fcfe419aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:52.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.506+0000 7fcfe3400640 1 --2- 172.21.15.43:0/1570492823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfe41090a0 0x7fcfe419aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.506+0000 7fcfe3400640 1 -- 172.21.15.43:0/1570492823 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfe419fa10 con 0x7fcfe4103f50 2024-09-19T05:14:52.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.506+0000 7fcfdbe00640 1 --2- 172.21.15.43:0/1570492823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfe41090a0 0x7fcfe419aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:14:52.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.506+0000 7fcfe3400640 1 --2- 172.21.15.43:0/1570492823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfe4103f50 0x7fcfe419a660 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4006010 tx=0x7fcfd40042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:52.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.507+0000 7fcfe0c00640 1 -- 172.21.15.43:0/1570492823 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd403f030 con 0x7fcfe4103f50 2024-09-19T05:14:52.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.507+0000 7fcfea600640 1 -- 172.21.15.43:0/1570492823 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfe419fca0 con 0x7fcfe4103f50 2024-09-19T05:14:52.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.507+0000 7fcfe0c00640 1 -- 172.21.15.43:0/1570492823 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfd4003ce0 con 0x7fcfe4103f50 2024-09-19T05:14:52.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.507+0000 7fcfe0c00640 1 -- 172.21.15.43:0/1570492823 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd4035070 con 0x7fcfe4103f50 2024-09-19T05:14:52.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.507+0000 7fcfea600640 1 -- 172.21.15.43:0/1570492823 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfe41a74c0 con 0x7fcfe4103f50 2024-09-19T05:14:52.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.508+0000 7fcfea600640 1 -- 172.21.15.43:0/1570492823 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfe4104330 con 0x7fcfe4103f50 2024-09-19T05:14:52.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.510+0000 7fcfe0c00640 1 -- 172.21.15.43:0/1570492823 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fcfd4042030 con 0x7fcfe4103f50 2024-09-19T05:14:52.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.511+0000 7fcfe0c00640 1 --2- 172.21.15.43:0/1570492823 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fcfc80a34e0 0x7fcfc80a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:14:52.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.511+0000 7fcfe0c00640 1 -- 172.21.15.43:0/1570492823 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7fcfd40f0420 con 0x7fcfe4103f50 2024-09-19T05:14:52.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.511+0000 7fcfdbe00640 1 --2- 172.21.15.43:0/1570492823 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fcfc80a34e0 0x7fcfc80a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:14:52.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.512+0000 7fcfdbe00640 1 --2- 172.21.15.43:0/1570492823 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fcfc80a34e0 0x7fcfc80a59a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcfe410a900 tx=0x7fcfcc005f90 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:14:52.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.513+0000 7fcfe0c00640 1 -- 172.21.15.43:0/1570492823 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcfd40bc800 con 0x7fcfe4103f50 2024-09-19T05:14:52.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.717+0000 7fcfea600640 1 -- 172.21.15.43:0/1570492823 --> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fcfe410c2c0 con 0x7fcfc80a34e0 2024-09-19T05:14:52.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.725+0000 7fcfe0c00640 1 -- 172.21.15.43:0/1570492823 <== mgr.25136 v2:172.21.15.43:6800/2120444093 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fcfcc004030 con 0x7fcfc80a34e0 2024-09-19T05:14:52.726 INFO:teuthology.orchestra.run.smithi043.stdout:/ 1m 6m3h 2024-09-19T05:14:52.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.727+0000 7fcfea600640 1 -- 172.21.15.43:0/1570492823 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fcfc80a34e0 msgr2=0x7fcfc80a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:52.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.727+0000 7fcfea600640 1 --2- 172.21.15.43:0/1570492823 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fcfc80a34e0 0x7fcfc80a59a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcfe410a900 tx=0x7fcfcc005f90 comp rx=0 tx=0).stop 2024-09-19T05:14:52.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.727+0000 7fcfea600640 1 -- 172.21.15.43:0/1570492823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfe4103f50 msgr2=0x7fcfe419a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:14:52.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.727+0000 7fcfea600640 1 --2- 172.21.15.43:0/1570492823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfe4103f50 0x7fcfe419a660 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4006010 tx=0x7fcfd40042f0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.727+0000 7fcfea600640 1 -- 172.21.15.43:0/1570492823 shutdown_connections 2024-09-19T05:14:52.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.727+0000 7fcfea600640 1 --2- 172.21.15.43:0/1570492823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfe419ef20 0x7fcfe419f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.727+0000 7fcfea600640 1 --2- 172.21.15.43:0/1570492823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfe41090a0 0x7fcfe419aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.727+0000 7fcfea600640 1 --2- 172.21.15.43:0/1570492823 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fcfc80a34e0 0x7fcfc80a59a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.727+0000 7fcfea600640 1 --2- 172.21.15.43:0/1570492823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfe4103f50 0x7fcfe419a660 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:14:52.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.727+0000 7fcfea600640 1 -- 172.21.15.43:0/1570492823 >> 172.21.15.43:0/1570492823 conn(0x7fcfe40ff9e0 msgr2=0x7fcfe4100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:14:52.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.727+0000 7fcfea600640 1 -- 172.21.15.43:0/1570492823 shutdown_connections 2024-09-19T05:14:52.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:14:52.727+0000 7fcfea600640 1 -- 172.21.15.43:0/1570492823 wait complete. 2024-09-19T05:14:52.744 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-19T05:14:52.756 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-19T05:14:54.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:54 smithi151 ceph-mon[35425]: from='client.27656 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:14:54.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:54 smithi151 ceph-mon[35425]: pgmap v318: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:14:54.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:54 smithi151 ceph-mon[35425]: from='client.27662 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:14:54.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:54 smithi096 ceph-mon[35085]: from='client.27656 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:14:54.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:54 smithi096 ceph-mon[35085]: pgmap v318: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:14:54.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:54 smithi096 ceph-mon[35085]: from='client.27662 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:14:54.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:54 smithi043 ceph-mon[30800]: from='client.27656 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:14:54.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:54 smithi043 ceph-mon[30800]: pgmap v318: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:14:54.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:54 smithi043 ceph-mon[30800]: from='client.27662 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T05:14:56.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:56 smithi151 ceph-mon[35425]: pgmap v319: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-09-19T05:14:56.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:56 smithi096 ceph-mon[35085]: pgmap v319: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-09-19T05:14:56.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:56 smithi043 ceph-mon[30800]: pgmap v319: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-09-19T05:14:58.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:14:58 smithi151 ceph-mon[35425]: pgmap v320: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-19T05:14:58.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:14:58 smithi096 ceph-mon[35085]: pgmap v320: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-19T05:14:58.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:14:58 smithi043 ceph-mon[30800]: pgmap v320: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-19T05:15:00.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:00 smithi151 ceph-mon[35425]: pgmap v321: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-19T05:15:00.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:00 smithi096 ceph-mon[35085]: pgmap v321: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-19T05:15:00.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:00 smithi043 ceph-mon[30800]: pgmap v321: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-19T05:15:02.285 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:02 smithi151 ceph-mon[35425]: pgmap v322: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-09-19T05:15:02.286 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:02 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:15:02.381 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:02 smithi043 ceph-mon[30800]: pgmap v322: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-09-19T05:15:02.381 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:02 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:15:02.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:02 smithi096 ceph-mon[35085]: pgmap v322: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-09-19T05:15:02.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:02 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:15:04.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:03 smithi151 ceph-mon[35425]: pgmap v323: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-19T05:15:04.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:03 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:15:04.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:03 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:15:04.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:03 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:15:04.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:03 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:15:04.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:03 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:03 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:03 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:03 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:03 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:03 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:03 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:03 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:03 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:03 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:03 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:03 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:03 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:03 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:03 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:03 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:03 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:15:04.157 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:03 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:15:04.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:03 smithi096 ceph-mon[35085]: pgmap v323: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-19T05:15:04.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:03 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:15:04.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:03 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:15:04.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:03 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:15:04.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:03 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:15:04.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:03 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:03 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:03 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:03 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:03 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:03 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.225 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:03 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.225 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:03 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.225 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:03 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.225 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:03 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.225 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:03 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.225 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:03 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.225 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:03 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.225 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:03 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.225 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:03 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.225 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:03 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.225 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:03 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:15:04.225 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:03 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:15:04.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:03 smithi043 ceph-mon[30800]: pgmap v323: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-19T05:15:04.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:03 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:15:04.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:03 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:15:04.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:03 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:15:04.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:03 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:15:04.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:03 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:03 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:03 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:03 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:03 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:03 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:03 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:03 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:03 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:03 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:03 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:03 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:03 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.283 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:03 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.283 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:03 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.283 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:03 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:04.283 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:03 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:15:04.283 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:03 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:15:05.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:04 smithi151 ceph-mon[35425]: Adjusting osd_memory_target on smithi151 to 176.0M 2024-09-19T05:15:05.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:04 smithi151 ceph-mon[35425]: Adjusting osd_memory_target on smithi096 to 176.0M 2024-09-19T05:15:05.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:04 smithi151 ceph-mon[35425]: Unable to set osd_memory_target on smithi151 to 184591052: error parsing value: Value '184591052' is below minimum 939524096 2024-09-19T05:15:05.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:04 smithi151 ceph-mon[35425]: Unable to set osd_memory_target on smithi096 to 184587468: error parsing value: Value '184587468' is below minimum 939524096 2024-09-19T05:15:05.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:04 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:05.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:04 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:05.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:04 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:05.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:04 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:05.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:04 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:05.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:04 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:05.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:04 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:05.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:04 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:05.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:04 smithi151 ceph-mon[35425]: Adjusting osd_memory_target on smithi043 to 177.5M 2024-09-19T05:15:05.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:04 smithi151 ceph-mon[35425]: Unable to set osd_memory_target on smithi043 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-19T05:15:05.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:04 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:15:05.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:04 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:15:05.156 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:04 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:15:05.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:04 smithi096 ceph-mon[35085]: Adjusting osd_memory_target on smithi151 to 176.0M 2024-09-19T05:15:05.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:04 smithi096 ceph-mon[35085]: Adjusting osd_memory_target on smithi096 to 176.0M 2024-09-19T05:15:05.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:04 smithi096 ceph-mon[35085]: Unable to set osd_memory_target on smithi151 to 184591052: error parsing value: Value '184591052' is below minimum 939524096 2024-09-19T05:15:05.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:04 smithi096 ceph-mon[35085]: Unable to set osd_memory_target on smithi096 to 184587468: error parsing value: Value '184587468' is below minimum 939524096 2024-09-19T05:15:05.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:04 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:05.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:04 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:05.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:04 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:05.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:04 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:05.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:04 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:05.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:04 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:05.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:04 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:05.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:04 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:05.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:04 smithi096 ceph-mon[35085]: Adjusting osd_memory_target on smithi043 to 177.5M 2024-09-19T05:15:05.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:04 smithi096 ceph-mon[35085]: Unable to set osd_memory_target on smithi043 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-19T05:15:05.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:04 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:15:05.224 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:04 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:15:05.225 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:04 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:15:05.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:04 smithi043 ceph-mon[30800]: Adjusting osd_memory_target on smithi151 to 176.0M 2024-09-19T05:15:05.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:04 smithi043 ceph-mon[30800]: Adjusting osd_memory_target on smithi096 to 176.0M 2024-09-19T05:15:05.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:04 smithi043 ceph-mon[30800]: Unable to set osd_memory_target on smithi151 to 184591052: error parsing value: Value '184591052' is below minimum 939524096 2024-09-19T05:15:05.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:04 smithi043 ceph-mon[30800]: Unable to set osd_memory_target on smithi096 to 184587468: error parsing value: Value '184587468' is below minimum 939524096 2024-09-19T05:15:05.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:04 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:05.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:04 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:05.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:04 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:05.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:04 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:05.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:04 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:05.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:04 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:05.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:04 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:05.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:04 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T05:15:05.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:04 smithi043 ceph-mon[30800]: Adjusting osd_memory_target on smithi043 to 177.5M 2024-09-19T05:15:05.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:04 smithi043 ceph-mon[30800]: Unable to set osd_memory_target on smithi043 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-19T05:15:05.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:04 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:15:05.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:04 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:15:05.282 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:04 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:15:06.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:05 smithi151 ceph-mon[35425]: pgmap v324: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 32 op/s 2024-09-19T05:15:06.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:05 smithi096 ceph-mon[35085]: pgmap v324: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 32 op/s 2024-09-19T05:15:06.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:05 smithi043 ceph-mon[30800]: pgmap v324: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 32 op/s 2024-09-19T05:15:08.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:07 smithi151 ceph-mon[35425]: pgmap v325: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-19T05:15:08.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:07 smithi096 ceph-mon[35085]: pgmap v325: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-19T05:15:08.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:07 smithi043 ceph-mon[30800]: pgmap v325: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-19T05:15:10.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:09 smithi151 ceph-mon[35425]: pgmap v326: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-19T05:15:10.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:09 smithi151 ceph-mon[35425]: osdmap e95: 12 total, 12 up, 12 in 2024-09-19T05:15:10.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:09 smithi096 ceph-mon[35085]: pgmap v326: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-19T05:15:10.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:09 smithi096 ceph-mon[35085]: osdmap e95: 12 total, 12 up, 12 in 2024-09-19T05:15:10.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:09 smithi043 ceph-mon[30800]: pgmap v326: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-19T05:15:10.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:09 smithi043 ceph-mon[30800]: osdmap e95: 12 total, 12 up, 12 in 2024-09-19T05:15:11.010 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T05:15:11.407 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:15:12.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.005+0000 7f14a0600640 1 -- 172.21.15.43:0/4014526070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1498104010 msgr2=0x7f14981043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:12.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.005+0000 7f14a0600640 1 --2- 172.21.15.43:0/4014526070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1498104010 0x7f14981043f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f148c009900 tx=0x7f148c02a410 comp rx=0 tx=0).stop 2024-09-19T05:15:12.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.006+0000 7f14a0600640 1 -- 172.21.15.43:0/4014526070 shutdown_connections 2024-09-19T05:15:12.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.006+0000 7f14a0600640 1 --2- 172.21.15.43:0/4014526070 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1498109140 0x7f1498110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:12.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.006+0000 7f14a0600640 1 --2- 172.21.15.43:0/4014526070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14981049c0 0x7f1498108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:12.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.006+0000 7f14a0600640 1 --2- 172.21.15.43:0/4014526070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1498104010 0x7f14981043f0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:12.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.006+0000 7f14a0600640 1 -- 172.21.15.43:0/4014526070 >> 172.21.15.43:0/4014526070 conn(0x7f14980ffa50 msgr2=0x7f1498101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:12.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.006+0000 7f14a0600640 1 -- 172.21.15.43:0/4014526070 shutdown_connections 2024-09-19T05:15:12.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.006+0000 7f14a0600640 1 -- 172.21.15.43:0/4014526070 wait complete. 2024-09-19T05:15:12.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.007+0000 7f14a0600640 1 Processor -- start 2024-09-19T05:15:12.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.007+0000 7f14a0600640 1 -- start start 2024-09-19T05:15:12.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.007+0000 7f14a0600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1498104010 0x7f149819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:12.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.007+0000 7f14a0600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14981049c0 0x7f149819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:12.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.007+0000 7f14a0600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1498109140 0x7f149819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:12.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.007+0000 7f14a0600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f14981141c0 con 0x7f1498104010 2024-09-19T05:15:12.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.007+0000 7f14a0600640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1498114040 con 0x7f14981049c0 2024-09-19T05:15:12.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.007+0000 7f14a0600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1498114340 con 0x7f1498109140 2024-09-19T05:15:12.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.007+0000 7f149dc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1498104010 0x7f149819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:12.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.007+0000 7f149dc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1498104010 0x7f149819a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51040/0 (socket says 172.21.15.43:51040) 2024-09-19T05:15:12.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.007+0000 7f149dc00640 1 -- 172.21.15.43:0/1106913251 learned_addr learned my addr 172.21.15.43:0/1106913251 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:15:12.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.007+0000 7f149d200640 1 --2- 172.21.15.43:0/1106913251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14981049c0 0x7f149819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:12.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.008+0000 7f149e600640 1 --2- 172.21.15.43:0/1106913251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1498109140 0x7f149819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:12.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.008+0000 7f149dc00640 1 -- 172.21.15.43:0/1106913251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1498109140 msgr2=0x7f149819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:12.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.008+0000 7f149dc00640 1 --2- 172.21.15.43:0/1106913251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1498109140 0x7f149819ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:12.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.008+0000 7f149dc00640 1 -- 172.21.15.43:0/1106913251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14981049c0 msgr2=0x7f149819ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:12.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.008+0000 7f149dc00640 1 --2- 172.21.15.43:0/1106913251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14981049c0 0x7f149819ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:12.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.008+0000 7f149dc00640 1 -- 172.21.15.43:0/1106913251 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f149819f5f0 con 0x7f1498104010 2024-09-19T05:15:12.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.008+0000 7f149e600640 1 --2- 172.21.15.43:0/1106913251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1498109140 0x7f149819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:15:12.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.008+0000 7f149dc00640 1 --2- 172.21.15.43:0/1106913251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1498104010 0x7f149819a640 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f148c004770 tx=0x7f148c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:12.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.008+0000 7f1496000640 1 -- 172.21.15.43:0/1106913251 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148c03f070 con 0x7f1498104010 2024-09-19T05:15:12.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.008+0000 7f1496000640 1 -- 172.21.15.43:0/1106913251 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f148c031ca0 con 0x7f1498104010 2024-09-19T05:15:12.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.008+0000 7f1496000640 1 -- 172.21.15.43:0/1106913251 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148c035030 con 0x7f1498104010 2024-09-19T05:15:12.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.008+0000 7f14a0600640 1 -- 172.21.15.43:0/1106913251 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f149819f880 con 0x7f1498104010 2024-09-19T05:15:12.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.009+0000 7f14a0600640 1 -- 172.21.15.43:0/1106913251 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f149819fd10 con 0x7f1498104010 2024-09-19T05:15:12.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.012+0000 7f1496000640 1 -- 172.21.15.43:0/1106913251 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f148c042070 con 0x7f1498104010 2024-09-19T05:15:12.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.012+0000 7f14a300d740 1 Processor -- start 2024-09-19T05:15:12.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.012+0000 7f14a300d740 1 -- start start 2024-09-19T05:15:12.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.012+0000 7f14a300d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591ccafbb90 0x5591ccafbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:12.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.012+0000 7f14a300d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591ccafc4b0 0x5591ccb06830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:12.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.012+0000 7f14a300d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591ccb06d70 0x5591ccb09160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:12.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.012+0000 7f14a300d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5591cca623e0 con 0x5591ccafbb90 2024-09-19T05:15:12.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.012+0000 7f14a300d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5591cca09d80 con 0x5591ccafc4b0 2024-09-19T05:15:12.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.012+0000 7f14a300d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5591cca5e3a0 con 0x5591ccb06d70 2024-09-19T05:15:12.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.012+0000 7f149e600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591ccafbb90 0x5591ccafbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:12.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.012+0000 7f149e600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591ccafbb90 0x5591ccafbf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51044/0 (socket says 172.21.15.43:51044) 2024-09-19T05:15:12.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.012+0000 7f149e600640 1 -- 172.21.15.43:0/1969118414 learned_addr learned my addr 172.21.15.43:0/1969118414 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:15:12.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.012+0000 7f149dc00640 1 --2- 172.21.15.43:0/1969118414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591ccafc4b0 0x5591ccb06830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:12.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.013+0000 7f149d200640 1 --2- 172.21.15.43:0/1969118414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591ccb06d70 0x5591ccb09160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:12.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.013+0000 7f149e600640 1 -- 172.21.15.43:0/1969118414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591ccb06d70 msgr2=0x5591ccb09160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:12.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.013+0000 7f149e600640 1 --2- 172.21.15.43:0/1969118414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591ccb06d70 0x5591ccb09160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:12.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.013+0000 7f149e600640 1 -- 172.21.15.43:0/1969118414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591ccafc4b0 msgr2=0x5591ccb06830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:12.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.013+0000 7f149e600640 1 --2- 172.21.15.43:0/1969118414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591ccafc4b0 0x5591ccb06830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:12.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.013+0000 7f149e600640 1 -- 172.21.15.43:0/1969118414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591cca28ae0 con 0x5591ccafbb90 2024-09-19T05:15:12.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.013+0000 7f149e600640 1 --2- 172.21.15.43:0/1969118414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591ccafbb90 0x5591ccafbf70 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f1490009fd0 tx=0x7f1490010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db3987604eeaf4c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:12.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.013+0000 7f148aa00640 1 -- 172.21.15.43:0/1969118414 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1490011700 con 0x5591ccafbb90 2024-09-19T05:15:12.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.014+0000 7f148aa00640 1 -- 172.21.15.43:0/1969118414 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1490011d80 con 0x5591ccafbb90 2024-09-19T05:15:12.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.014+0000 7f148aa00640 1 -- 172.21.15.43:0/1969118414 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149000e3c0 con 0x5591ccafbb90 2024-09-19T05:15:12.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.014+0000 7f1496000640 1 --2- 172.21.15.43:0/1106913251 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f14800a3430 0x7f14800a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:12.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.014+0000 7f1496000640 1 -- 172.21.15.43:0/1106913251 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8586+0+0 (secure 0 0 0) 0x7f148c0f03b0 con 0x7f1498104010 2024-09-19T05:15:12.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.014+0000 7f14a300d740 1 -- 172.21.15.43:0/1969118414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591ccafbb90 msgr2=0x5591ccafbf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:12.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.014+0000 7f14a300d740 1 --2- 172.21.15.43:0/1969118414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591ccafbb90 0x5591ccafbf70 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f1490009fd0 tx=0x7f1490010860 comp rx=0 tx=0).stop 2024-09-19T05:15:12.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.014+0000 7f149dc00640 1 --2- 172.21.15.43:0/1106913251 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f14800a3430 0x7f14800a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:12.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.014+0000 7f14a300d740 1 -- 172.21.15.43:0/1969118414 shutdown_connections 2024-09-19T05:15:12.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.014+0000 7f14a300d740 1 --2- 172.21.15.43:0/1969118414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591ccb06d70 0x5591ccb09160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:12.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.014+0000 7f14a300d740 1 --2- 172.21.15.43:0/1969118414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591ccafc4b0 0x5591ccb06830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:12.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.014+0000 7f14a300d740 1 --2- 172.21.15.43:0/1969118414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591ccafbb90 0x5591ccafbf70 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:12.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.014+0000 7f14a300d740 1 -- 172.21.15.43:0/1969118414 >> 172.21.15.43:0/1969118414 conn(0x5591ccaf8120 msgr2=0x5591ccaf8500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:12.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.015+0000 7f14a300d740 1 -- 172.21.15.43:0/1969118414 shutdown_connections 2024-09-19T05:15:12.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.015+0000 7f14a300d740 1 -- 172.21.15.43:0/1969118414 wait complete. 2024-09-19T05:15:12.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.015+0000 7f149dc00640 1 --2- 172.21.15.43:0/1106913251 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f14800a3430 0x7f14800a58f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f148c0f3fd0 tx=0x7f148c0f3e60 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:12.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.015+0000 7f14a300d740 1 -- start start 2024-09-19T05:15:12.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.016+0000 7f14a300d740 1 Processor -- start 2024-09-19T05:15:12.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.016+0000 7f14a300d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591ccafbb90 0x5591ccb03c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:12.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.016+0000 7f14a300d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591ccafc4b0 0x5591ccb041d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:12.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.016+0000 7f14a300d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591ccb06d70 0x5591ccb04710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:12.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.016+0000 7f14a300d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5591ccb09a30 con 0x5591ccb06d70 2024-09-19T05:15:12.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.016+0000 7f14a300d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5591cca1d0f0 con 0x5591ccafc4b0 2024-09-19T05:15:12.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.016+0000 7f14a300d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5591ccb09bb0 con 0x5591ccafbb90 2024-09-19T05:15:12.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.016+0000 7f14a300d740 10 client.0 _ll_register_callbacks cb 0x5591cca0e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:15:12.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.016+0000 7f149e600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591ccb06d70 0x5591ccb04710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:12.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.016+0000 7f149e600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591ccafbb90 0x5591ccb03c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:12.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.016+0000 7f149e600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591ccafbb90 0x5591ccb03c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:43160/0 (socket says 172.21.15.43:43160) 2024-09-19T05:15:12.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.016+0000 7f149e600640 1 -- 172.21.15.43:0/251278984 learned_addr learned my addr 172.21.15.43:0/251278984 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:15:12.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.016+0000 7f149d200640 1 --2- 172.21.15.43:0/251278984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591ccafc4b0 0x5591ccb041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:12.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.017+0000 7f149e600640 1 -- 172.21.15.43:0/251278984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591ccafc4b0 msgr2=0x5591ccb041d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:12.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.017+0000 7f149e600640 1 --2- 172.21.15.43:0/251278984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591ccafc4b0 0x5591ccb041d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:12.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.017+0000 7f149e600640 1 -- 172.21.15.43:0/251278984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591ccb06d70 msgr2=0x5591ccb04710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:12.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.017+0000 7f149e600640 1 --2- 172.21.15.43:0/251278984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591ccb06d70 0x5591ccb04710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:12.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.017+0000 7f149e600640 1 -- 172.21.15.43:0/251278984 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591cc7d33c0 con 0x5591ccafbb90 2024-09-19T05:15:12.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.017+0000 7f149e600640 1 --2- 172.21.15.43:0/251278984 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591ccafbb90 0x5591ccb03c90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1490005bf0 tx=0x7f1490010fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=632db92f90fc8435 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:12.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.017+0000 7f147f400640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:15:12.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.017+0000 7f147f400640 1 -- 172.21.15.43:0/251278984 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1490024040 con 0x5591ccafbb90 2024-09-19T05:15:12.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.017+0000 7f14a300d740 1 -- 172.21.15.43:0/251278984 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1490011d80 con 0x5591ccafbb90 2024-09-19T05:15:12.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.017+0000 7f147f400640 1 -- 172.21.15.43:0/251278984 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8586+0+0 (secure 0 0 0) 0x7f1490011a20 con 0x5591ccafbb90 2024-09-19T05:15:12.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.018+0000 7f147f400640 1 -- 172.21.15.43:0/251278984 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f149002b430 con 0x5591ccafbb90 2024-09-19T05:15:12.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.018+0000 7f14a300d740 10 client.27577 fetch_fsmap learned FSMap version 29 2024-09-19T05:15:12.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.018+0000 7f14a300d740 1 -- 172.21.15.43:0/251278984 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5591ccb15740 con 0x5591ccafbb90 2024-09-19T05:15:12.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.018+0000 7f147f400640 1 -- 172.21.15.43:0/251278984 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1490011d80 con 0x5591ccafbb90 2024-09-19T05:15:12.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.018+0000 7f147f400640 1 -- 172.21.15.43:0/251278984 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1490031030 con 0x5591ccafbb90 2024-09-19T05:15:12.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.018+0000 7f14a300d740 10 client.27577 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:15:12.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.018+0000 7f14a300d740 10 client.27577 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:15:12.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.018+0000 7f14a300d740 20 client.27577 populate_metadata read hostname 'smithi043' 2024-09-19T05:15:12.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.018+0000 7f14a300d740 1 --2- 172.21.15.43:0/251278984 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5591ccb16b50 0x5591ccb18f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:12.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.019+0000 7f14a300d740 4 client.27577 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:15:12.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.019+0000 7f14a300d740 1 -- 172.21.15.43:0/251278984 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591cca6f950 con 0x5591ccb16b50 2024-09-19T05:15:12.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.019+0000 7f149d200640 1 --2- 172.21.15.43:0/251278984 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5591ccb16b50 0x5591ccb18f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:12.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.019+0000 7f149d200640 1 --2- 172.21.15.43:0/251278984 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5591ccb16b50 0x5591ccb18f40 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14b264d5452bc3c server_cookie=3774a84a39d78138 in_seq=0 out_seq=0 2024-09-19T05:15:12.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.019+0000 7f147f400640 10 client.27577 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:15:12.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.020+0000 7f147f400640 1 -- 172.21.15.43:0/251278984 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5591cca6f950 con 0x5591ccb16b50 2024-09-19T05:15:12.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.020+0000 7f147f400640 10 client.27577 handle_command_reply: tid=0 2024-09-19T05:15:12.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.020+0000 7f14a300d740 1 client.27577 shutdown 2024-09-19T05:15:12.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.020+0000 7f14a300d740 1 -- 172.21.15.43:0/251278984 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591ccafbb90 msgr2=0x5591ccb03c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:12.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.020+0000 7f14a300d740 1 --2- 172.21.15.43:0/251278984 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591ccafbb90 0x5591ccb03c90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1490005bf0 tx=0x7f1490010fd0 comp rx=0 tx=0).stop 2024-09-19T05:15:12.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.020+0000 7f14a300d740 1 -- 172.21.15.43:0/251278984 shutdown_connections 2024-09-19T05:15:12.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.020+0000 7f14a300d740 1 --2- 172.21.15.43:0/251278984 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5591ccb16b50 0x5591ccb18f40 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:12.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.020+0000 7f14a300d740 1 --2- 172.21.15.43:0/251278984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591ccb06d70 0x5591ccb04710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:12.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.020+0000 7f14a300d740 1 --2- 172.21.15.43:0/251278984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591ccafc4b0 0x5591ccb041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:12.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.020+0000 7f14a300d740 1 --2- 172.21.15.43:0/251278984 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591ccafbb90 0x5591ccb03c90 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:12.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.020+0000 7f14a300d740 1 -- 172.21.15.43:0/251278984 >> 172.21.15.43:0/251278984 conn(0x5591ccaf8120 msgr2=0x5591ccafde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:12.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.020+0000 7f147f400640 0 client.27577 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:15:12.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.021+0000 7f14a300d740 1 -- 172.21.15.43:0/251278984 shutdown_connections 2024-09-19T05:15:12.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.021+0000 7f14a300d740 1 -- 172.21.15.43:0/251278984 wait complete. 2024-09-19T05:15:12.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.021+0000 7f14a300d740 20 client.27577 trim_cache size 0 max 16384 2024-09-19T05:15:12.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.039+0000 7f14a300d740 1 Processor -- start 2024-09-19T05:15:12.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.039+0000 7f14a300d740 1 -- start start 2024-09-19T05:15:12.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.039+0000 7f14a300d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591ccafbb90 0x5591ccb1e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:12.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.039+0000 7f14a300d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591ccafc4b0 0x5591ccb1eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:12.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.039+0000 7f14a300d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591ccb06d70 0x5591ccb054f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:12.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.039+0000 7f14a300d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f14981afbb0 con 0x5591ccafc4b0 2024-09-19T05:15:12.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.039+0000 7f14a300d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f14981b16f0 con 0x5591ccb06d70 2024-09-19T05:15:12.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.039+0000 7f14a300d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f14981b0280 con 0x5591ccafbb90 2024-09-19T05:15:12.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.040+0000 7f149d200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591ccafc4b0 0x5591ccb1eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:12.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.040+0000 7f149d200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591ccafc4b0 0x5591ccb1eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51074/0 (socket says 172.21.15.43:51074) 2024-09-19T05:15:12.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.040+0000 7f149d200640 1 -- 172.21.15.43:0/1131970764 learned_addr learned my addr 172.21.15.43:0/1131970764 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:15:12.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.040+0000 7f149e600640 1 --2- 172.21.15.43:0/1131970764 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591ccafbb90 0x5591ccb1e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:12.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.040+0000 7f149e600640 1 --2- 172.21.15.43:0/1131970764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591ccb06d70 0x5591ccb054f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:12.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.040+0000 7f149d200640 1 -- 172.21.15.43:0/1131970764 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591ccafbb90 msgr2=0x5591ccb1e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:12.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.040+0000 7f149d200640 1 --2- 172.21.15.43:0/1131970764 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591ccafbb90 0x5591ccb1e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:12.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.040+0000 7f149d200640 1 -- 172.21.15.43:0/1131970764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591ccb06d70 msgr2=0x5591ccb054f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:12.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.040+0000 7f149d200640 1 --2- 172.21.15.43:0/1131970764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591ccb06d70 0x5591ccb054f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:12.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.040+0000 7f149d200640 1 -- 172.21.15.43:0/1131970764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591ccafca20 con 0x5591ccafc4b0 2024-09-19T05:15:12.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.041+0000 7f149d200640 1 --2- 172.21.15.43:0/1131970764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591ccafc4b0 0x5591ccb1eba0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f1484009dd0 tx=0x7f1484005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=216d6da7a8bf0e81 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:12.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.041+0000 7f148a000640 1 -- 172.21.15.43:0/1131970764 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1484022030 con 0x5591ccafc4b0 2024-09-19T05:15:12.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.041+0000 7f148a000640 1 -- 172.21.15.43:0/1131970764 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1484005500 con 0x5591ccafc4b0 2024-09-19T05:15:12.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.041+0000 7f148a000640 1 -- 172.21.15.43:0/1131970764 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148401e680 con 0x5591ccafc4b0 2024-09-19T05:15:12.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.042+0000 7f14a300d740 1 -- 172.21.15.43:0/1131970764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591ccafc4b0 msgr2=0x5591ccb1eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:12.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.042+0000 7f14a300d740 1 --2- 172.21.15.43:0/1131970764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591ccafc4b0 0x5591ccb1eba0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f1484009dd0 tx=0x7f1484005f90 comp rx=0 tx=0).stop 2024-09-19T05:15:12.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.042+0000 7f14a300d740 1 -- 172.21.15.43:0/1131970764 shutdown_connections 2024-09-19T05:15:12.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.042+0000 7f14a300d740 1 --2- 172.21.15.43:0/1131970764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591ccb06d70 0x5591ccb054f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:12.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.042+0000 7f14a300d740 1 --2- 172.21.15.43:0/1131970764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591ccafc4b0 0x5591ccb1eba0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:12.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.042+0000 7f14a300d740 1 --2- 172.21.15.43:0/1131970764 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591ccafbb90 0x5591ccb1e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:12.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.042+0000 7f14a300d740 1 -- 172.21.15.43:0/1131970764 >> 172.21.15.43:0/1131970764 conn(0x5591ccaf8120 msgr2=0x5591ccb02d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:12.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.042+0000 7f14a300d740 1 -- 172.21.15.43:0/1131970764 shutdown_connections 2024-09-19T05:15:12.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.042+0000 7f14a300d740 1 -- 172.21.15.43:0/1131970764 wait complete. 2024-09-19T05:15:12.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.042+0000 7f14a300d740 1 -- start start 2024-09-19T05:15:12.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.042+0000 7f14a300d740 1 Processor -- start 2024-09-19T05:15:12.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.043+0000 7f14a300d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591ccafbb90 0x5591ccb09f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:12.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.043+0000 7f14a300d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591ccafc4b0 0x5591ccb0a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:12.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.043+0000 7f14a300d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591ccb06d70 0x5591ccb7f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:12.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.043+0000 7f14a300d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f14981b1020 con 0x5591ccb06d70 2024-09-19T05:15:12.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.043+0000 7f14a300d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f14981b0950 con 0x5591ccafbb90 2024-09-19T05:15:12.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.043+0000 7f14a300d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f14981adbb0 con 0x5591ccafc4b0 2024-09-19T05:15:12.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.043+0000 7f14a300d740 10 client.0 _ll_register_callbacks cb 0x7f149001b7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:15:12.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.043+0000 7f149e600640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591ccafbb90 0x5591ccb09f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:12.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.043+0000 7f149d200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591ccafc4b0 0x5591ccb0a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:12.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.043+0000 7f149e600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591ccb06d70 0x5591ccb7f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:12.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.043+0000 7f149d200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591ccafc4b0 0x5591ccb0a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:43178/0 (socket says 172.21.15.43:43178) 2024-09-19T05:15:12.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.043+0000 7f149d200640 1 -- 172.21.15.43:0/2773528170 learned_addr learned my addr 172.21.15.43:0/2773528170 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:15:12.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.044+0000 7f149d200640 1 -- 172.21.15.43:0/2773528170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591ccafbb90 msgr2=0x5591ccb09f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:12.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.044+0000 7f149d200640 1 --2- 172.21.15.43:0/2773528170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591ccafbb90 0x5591ccb09f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:12.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.044+0000 7f149d200640 1 -- 172.21.15.43:0/2773528170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591ccb06d70 msgr2=0x5591ccb7f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:12.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.044+0000 7f149d200640 1 --2- 172.21.15.43:0/2773528170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591ccb06d70 0x5591ccb7f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:12.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.044+0000 7f149d200640 1 -- 172.21.15.43:0/2773528170 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591ccb7fb60 con 0x5591ccafc4b0 2024-09-19T05:15:12.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.044+0000 7f149e600640 1 --2- 172.21.15.43:0/2773528170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591ccb06d70 0x5591ccb7f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:15:12.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.044+0000 7f149d200640 1 --2- 172.21.15.43:0/2773528170 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591ccafc4b0 0x5591ccb0a450 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1484012fd0 tx=0x7f1484009b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd98fbe0b7215c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:12.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.044+0000 7f147f400640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:15:12.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.044+0000 7f147f400640 1 -- 172.21.15.43:0/2773528170 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1484022070 con 0x5591ccafc4b0 2024-09-19T05:15:12.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.044+0000 7f147f400640 1 -- 172.21.15.43:0/2773528170 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8586+0+0 (secure 0 0 0) 0x7f1484004eb0 con 0x5591ccafc4b0 2024-09-19T05:15:12.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.044+0000 7f14a300d740 1 -- 172.21.15.43:0/2773528170 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1484005500 con 0x5591ccafc4b0 2024-09-19T05:15:12.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.045+0000 7f147f400640 1 -- 172.21.15.43:0/2773528170 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1484004b90 con 0x5591ccafc4b0 2024-09-19T05:15:12.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.045+0000 7f14a300d740 10 client.27583 fetch_fsmap learned FSMap version 29 2024-09-19T05:15:12.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.045+0000 7f14a300d740 1 -- 172.21.15.43:0/2773528170 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5591ccb80d40 con 0x5591ccafc4b0 2024-09-19T05:15:12.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.045+0000 7f147f400640 1 -- 172.21.15.43:0/2773528170 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1484005500 con 0x5591ccafc4b0 2024-09-19T05:15:12.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.045+0000 7f147f400640 1 -- 172.21.15.43:0/2773528170 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f148402e020 con 0x5591ccafc4b0 2024-09-19T05:15:12.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.045+0000 7f14a300d740 10 client.27583 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:15:12.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.045+0000 7f14a300d740 10 client.27583 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:15:12.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.045+0000 7f14a300d740 20 client.27583 populate_metadata read hostname 'smithi043' 2024-09-19T05:15:12.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.045+0000 7f14a300d740 1 --2- 172.21.15.43:0/2773528170 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5591ccb81490 0x5591ccb838b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:12.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.045+0000 7f14a300d740 4 client.27583 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T05:15:12.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.045+0000 7f14a300d740 1 -- 172.21.15.43:0/2773528170 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5591cca69e60 con 0x5591ccb81490 2024-09-19T05:15:12.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.046+0000 7f149e600640 1 --2- 172.21.15.43:0/2773528170 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5591ccb81490 0x5591ccb838b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:12.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.046+0000 7f149e600640 1 --2- 172.21.15.43:0/2773528170 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5591ccb81490 0x5591ccb838b0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c938118edb349f3a server_cookie=6ad4f3e05a5c1639 in_seq=0 out_seq=0 2024-09-19T05:15:12.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.046+0000 7f147f400640 10 client.27583 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:15:12.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.046+0000 7f147f400640 1 -- 172.21.15.43:0/2773528170 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5591cca69e60 con 0x5591ccb81490 2024-09-19T05:15:12.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.046+0000 7f147f400640 10 client.27583 handle_command_reply: tid=0 2024-09-19T05:15:12.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.046+0000 7f14a300d740 1 client.27583 shutdown 2024-09-19T05:15:12.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.046+0000 7f14a300d740 1 -- 172.21.15.43:0/2773528170 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591ccafc4b0 msgr2=0x5591ccb0a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:12.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.046+0000 7f14a300d740 1 --2- 172.21.15.43:0/2773528170 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591ccafc4b0 0x5591ccb0a450 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1484012fd0 tx=0x7f1484009b00 comp rx=0 tx=0).stop 2024-09-19T05:15:12.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.046+0000 7f14a300d740 1 -- 172.21.15.43:0/2773528170 shutdown_connections 2024-09-19T05:15:12.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.046+0000 7f14a300d740 1 --2- 172.21.15.43:0/2773528170 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x5591ccb81490 0x5591ccb838b0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:12.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.046+0000 7f14a300d740 1 --2- 172.21.15.43:0/2773528170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5591ccb06d70 0x5591ccb7f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:12.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.046+0000 7f14a300d740 1 --2- 172.21.15.43:0/2773528170 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591ccafc4b0 0x5591ccb0a450 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:12.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.046+0000 7f14a300d740 1 --2- 172.21.15.43:0/2773528170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591ccafbb90 0x5591ccb09f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:12.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.046+0000 7f147f400640 0 client.27583 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:15:12.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.046+0000 7f14a300d740 1 -- 172.21.15.43:0/2773528170 >> 172.21.15.43:0/2773528170 conn(0x5591ccaf8120 msgr2=0x5591ccb054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:12.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.047+0000 7f14a300d740 1 -- 172.21.15.43:0/2773528170 shutdown_connections 2024-09-19T05:15:12.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.047+0000 7f14a300d740 1 -- 172.21.15.43:0/2773528170 wait complete. 2024-09-19T05:15:12.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.047+0000 7f14a300d740 20 client.27583 trim_cache size 0 max 16384 2024-09-19T05:15:12.054 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:15:12.054 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-09-19T05:15:12.054 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-09-19T05:15:12.054 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:15:12.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.047+0000 7f14a0600640 1 -- 172.21.15.43:0/1106913251 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f14800a3430 msgr2=0x7f14800a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:12.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.047+0000 7f14a0600640 1 --2- 172.21.15.43:0/1106913251 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f14800a3430 0x7f14800a58f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f148c0f3fd0 tx=0x7f148c0f3e60 comp rx=0 tx=0).stop 2024-09-19T05:15:12.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.047+0000 7f14a0600640 1 -- 172.21.15.43:0/1106913251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1498104010 msgr2=0x7f149819a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:12.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.047+0000 7f14a0600640 1 --2- 172.21.15.43:0/1106913251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1498104010 0x7f149819a640 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f148c004770 tx=0x7f148c003c10 comp rx=0 tx=0).stop 2024-09-19T05:15:12.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.048+0000 7f14a0600640 1 -- 172.21.15.43:0/1106913251 shutdown_connections 2024-09-19T05:15:12.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.048+0000 7f14a0600640 1 --2- 172.21.15.43:0/1106913251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1498109140 0x7f149819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:12.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.048+0000 7f14a0600640 1 --2- 172.21.15.43:0/1106913251 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f14800a3430 0x7f14800a58f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:12.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.048+0000 7f14a0600640 1 --2- 172.21.15.43:0/1106913251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14981049c0 0x7f149819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:12.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.048+0000 7f14a0600640 1 --2- 172.21.15.43:0/1106913251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1498104010 0x7f149819a640 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:12.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.048+0000 7f14a0600640 1 -- 172.21.15.43:0/1106913251 >> 172.21.15.43:0/1106913251 conn(0x7f14980ffa50 msgr2=0x7f1498100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:12.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.048+0000 7f14a0600640 1 -- 172.21.15.43:0/1106913251 shutdown_connections 2024-09-19T05:15:12.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:12.048+0000 7f14a0600640 1 -- 172.21.15.43:0/1106913251 wait complete. 2024-09-19T05:15:12.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:11 smithi151 ceph-mon[35425]: pgmap v328: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T05:15:12.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:11 smithi096 ceph-mon[35085]: pgmap v328: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T05:15:12.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:11 smithi043 ceph-mon[30800]: pgmap v328: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T05:15:12.963 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T05:15:12.963 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:15:13.343 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:15:14.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:13.910+0000 7f2658a00640 1 -- 172.21.15.43:0/2593740438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26541049a0 msgr2=0x7f26541089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:14.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:13.910+0000 7f2658a00640 1 --2- 172.21.15.43:0/2593740438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26541049a0 0x7f26541089f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f263c009860 tx=0x7f263c02a340 comp rx=0 tx=0).stop 2024-09-19T05:15:14.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:13.910+0000 7f2658a00640 1 -- 172.21.15.43:0/2593740438 shutdown_connections 2024-09-19T05:15:14.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:13.910+0000 7f2658a00640 1 --2- 172.21.15.43:0/2593740438 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2654109120 0x7f2654110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:14.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:13.910+0000 7f2658a00640 1 --2- 172.21.15.43:0/2593740438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26541049a0 0x7f26541089f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:14.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:13.910+0000 7f2658a00640 1 --2- 172.21.15.43:0/2593740438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2654103ff0 0x7f26541043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:14.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:13.910+0000 7f2658a00640 1 -- 172.21.15.43:0/2593740438 >> 172.21.15.43:0/2593740438 conn(0x7f26540ffa90 msgr2=0x7f2654101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:14.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:13.910+0000 7f2658a00640 1 -- 172.21.15.43:0/2593740438 shutdown_connections 2024-09-19T05:15:14.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:13.910+0000 7f2658a00640 1 -- 172.21.15.43:0/2593740438 wait complete. 2024-09-19T05:15:14.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:13.912+0000 7f2658a00640 1 Processor -- start 2024-09-19T05:15:14.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:13.912+0000 7f2658a00640 1 -- start start 2024-09-19T05:15:14.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:13.912+0000 7f2658a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2654103ff0 0x7f265419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:14.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:13.912+0000 7f2658a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26541049a0 0x7f265419abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:14.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:13.912+0000 7f2658a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2654109120 0x7f265419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:14.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:13.912+0000 7f2658a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2654114140 con 0x7f2654109120 2024-09-19T05:15:14.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:13.912+0000 7f2658a00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2654113fc0 con 0x7f2654103ff0 2024-09-19T05:15:14.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:13.912+0000 7f2658a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f26541142c0 con 0x7f26541049a0 2024-09-19T05:15:14.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:13.912+0000 7f2652800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2654109120 0x7f265419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:14.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:13.912+0000 7f2652800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2654109120 0x7f265419ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51096/0 (socket says 172.21.15.43:51096) 2024-09-19T05:15:14.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:13.912+0000 7f2652800640 1 -- 172.21.15.43:0/1694844318 learned_addr learned my addr 172.21.15.43:0/1694844318 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:15:14.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:13.913+0000 7f2651e00640 1 --2- 172.21.15.43:0/1694844318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2654103ff0 0x7f265419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:14.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:13.913+0000 7f264be00640 1 --2- 172.21.15.43:0/1694844318 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26541049a0 0x7f265419abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:14.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:13.913+0000 7f2652800640 1 -- 172.21.15.43:0/1694844318 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26541049a0 msgr2=0x7f265419abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:14.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:13.913+0000 7f2652800640 1 --2- 172.21.15.43:0/1694844318 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26541049a0 0x7f265419abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:14.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:13.913+0000 7f2652800640 1 -- 172.21.15.43:0/1694844318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2654103ff0 msgr2=0x7f265419a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:14.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:13.913+0000 7f2652800640 1 --2- 172.21.15.43:0/1694844318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2654103ff0 0x7f265419a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:14.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:13.913+0000 7f2652800640 1 -- 172.21.15.43:0/1694844318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f265419f650 con 0x7f2654109120 2024-09-19T05:15:14.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:13.913+0000 7f2652800640 1 --2- 172.21.15.43:0/1694844318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2654109120 0x7f265419ef70 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f264c00eb10 tx=0x7f264c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:14.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:13.913+0000 7f264aa00640 1 -- 172.21.15.43:0/1694844318 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c004b20 con 0x7f2654109120 2024-09-19T05:15:14.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:13.914+0000 7f2658a00640 1 -- 172.21.15.43:0/1694844318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f265419f940 con 0x7f2654109120 2024-09-19T05:15:14.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:13.914+0000 7f264aa00640 1 -- 172.21.15.43:0/1694844318 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f264c004cc0 con 0x7f2654109120 2024-09-19T05:15:14.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:13.914+0000 7f264aa00640 1 -- 172.21.15.43:0/1694844318 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c010760 con 0x7f2654109120 2024-09-19T05:15:14.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:13.914+0000 7f2658a00640 1 -- 172.21.15.43:0/1694844318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26541a7220 con 0x7f2654109120 2024-09-19T05:15:14.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:13.915+0000 7f2658a00640 1 -- 172.21.15.43:0/1694844318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26541043d0 con 0x7f2654109120 2024-09-19T05:15:14.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:13.917+0000 7f264aa00640 1 -- 172.21.15.43:0/1694844318 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f264c005020 con 0x7f2654109120 2024-09-19T05:15:14.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:13.918+0000 7f264aa00640 1 --2- 172.21.15.43:0/1694844318 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f26380a33b0 0x7f26380a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:14.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:13.918+0000 7f264aa00640 1 -- 172.21.15.43:0/1694844318 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8586+0+0 (secure 0 0 0) 0x7f264c0d03a0 con 0x7f2654109120 2024-09-19T05:15:14.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:13.918+0000 7f2651e00640 1 --2- 172.21.15.43:0/1694844318 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f26380a33b0 0x7f26380a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:14.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:13.919+0000 7f2651e00640 1 --2- 172.21.15.43:0/1694844318 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f26380a33b0 0x7f26380a5870 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2644009530 tx=0x7f2644009260 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:14.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:13.922+0000 7f264aa00640 1 -- 172.21.15.43:0/1694844318 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f264c09cd40 con 0x7f2654109120 2024-09-19T05:15:14.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:14.146+0000 7f2658a00640 1 -- 172.21.15.43:0/1694844318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f265410c3c0 con 0x7f2654109120 2024-09-19T05:15:14.149 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:13 smithi043 ceph-mon[30800]: pgmap v329: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T05:15:14.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:14.147+0000 7f264aa00640 1 -- 172.21.15.43:0/1694844318 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11965 (secure 0 0 0) 0x7f264c098020 con 0x7f2654109120 2024-09-19T05:15:14.150 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:15:14.150 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":29,"btime":"2024-09-19T05:04:38:785407+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/773801169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":773801169},{"type":"v1","addr":"172.21.15.43:6835","nonce":773801169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1799137902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1799137902},{"type":"v1","addr":"172.21.15.43:6837","nonce":1799137902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:02:25.463041+0000","modified":"2024-09-19T05:04:38.039809+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24518,"mds_2":24511,"mds_3":24506,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14541":{"gid":14541,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/794881135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":794881135},{"type":"v1","addr":"172.21.15.43:6841","nonce":794881135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/2836453158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2836453158},{"type":"v1","addr":"172.21.15.151:6833","nonce":2836453158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/178785812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":178785812},{"type":"v1","addr":"172.21.15.96:6833","nonce":178785812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1671043820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1671043820},{"type":"v1","addr":"172.21.15.96:6835","nonce":1671043820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2972514328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2972514328},{"type":"v1","addr":"172.21.15.151:6835","nonce":2972514328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.151:6837/3960378010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3960378010},{"type":"v1","addr":"172.21.15.151:6837","nonce":3960378010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6837/3978388286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3978388286},{"type":"v1","addr":"172.21.15.96:6837","nonce":3978388286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.151:6839/3794104313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3794104313},{"type":"v1","addr":"172.21.15.151:6839","nonce":3794104313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.43:6839/3404555030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3404555030},{"type":"v1","addr":"172.21.15.43:6839","nonce":3404555030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/605527621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":605527621},{"type":"v1","addr":"172.21.15.96:6839","nonce":605527621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24518,24511,24506,24493]},"id":1}]} 2024-09-19T05:15:14.150 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 29 2024-09-19T05:15:14.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:14.150+0000 7f2658a00640 1 -- 172.21.15.43:0/1694844318 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f26380a33b0 msgr2=0x7f26380a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:14.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:14.150+0000 7f2658a00640 1 --2- 172.21.15.43:0/1694844318 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f26380a33b0 0x7f26380a5870 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2644009530 tx=0x7f2644009260 comp rx=0 tx=0).stop 2024-09-19T05:15:14.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:14.150+0000 7f2658a00640 1 -- 172.21.15.43:0/1694844318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2654109120 msgr2=0x7f265419ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:14.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:14.150+0000 7f2658a00640 1 --2- 172.21.15.43:0/1694844318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2654109120 0x7f265419ef70 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f264c00eb10 tx=0x7f264c00c2d0 comp rx=0 tx=0).stop 2024-09-19T05:15:14.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:14.150+0000 7f2658a00640 1 -- 172.21.15.43:0/1694844318 shutdown_connections 2024-09-19T05:15:14.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:14.150+0000 7f2658a00640 1 --2- 172.21.15.43:0/1694844318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2654109120 0x7f265419ef70 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:14.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:14.150+0000 7f2658a00640 1 --2- 172.21.15.43:0/1694844318 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26541049a0 0x7f265419abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:14.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:14.150+0000 7f2658a00640 1 --2- 172.21.15.43:0/1694844318 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f26380a33b0 0x7f26380a5870 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:14.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:14.150+0000 7f2658a00640 1 --2- 172.21.15.43:0/1694844318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2654103ff0 0x7f265419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:14.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:14.150+0000 7f2658a00640 1 -- 172.21.15.43:0/1694844318 >> 172.21.15.43:0/1694844318 conn(0x7f26540ffa90 msgr2=0x7f2654100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:14.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:14.150+0000 7f2658a00640 1 -- 172.21.15.43:0/1694844318 shutdown_connections 2024-09-19T05:15:14.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:14.151+0000 7f2658a00640 1 -- 172.21.15.43:0/1694844318 wait complete. 2024-09-19T05:15:14.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:13 smithi096 ceph-mon[35085]: pgmap v329: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T05:15:14.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:13 smithi151 ceph-mon[35425]: pgmap v329: 97 pgs: 97 active+clean; 2.4 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-19T05:15:14.894 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T05:15:15.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:14 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/1694844318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:15:15.272 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:15:15.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:14 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/1694844318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:15:15.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:14 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/1694844318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:15:15.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.841+0000 7f9b67c00640 1 -- 172.21.15.43:0/2719488386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b60104010 msgr2=0x7f9b601043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:15.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.841+0000 7f9b67c00640 1 --2- 172.21.15.43:0/2719488386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b60104010 0x7f9b601043f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9b54009900 tx=0x7f9b5402a3f0 comp rx=0 tx=0).stop 2024-09-19T05:15:15.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.841+0000 7f9b67c00640 1 -- 172.21.15.43:0/2719488386 shutdown_connections 2024-09-19T05:15:15.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.841+0000 7f9b67c00640 1 --2- 172.21.15.43:0/2719488386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b60109140 0x7f9b60110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:15.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.841+0000 7f9b67c00640 1 --2- 172.21.15.43:0/2719488386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b601049c0 0x7f9b60108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:15.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.841+0000 7f9b67c00640 1 --2- 172.21.15.43:0/2719488386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b60104010 0x7f9b601043f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:15.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.841+0000 7f9b67c00640 1 -- 172.21.15.43:0/2719488386 >> 172.21.15.43:0/2719488386 conn(0x7f9b600ffa50 msgr2=0x7f9b60101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:15.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.841+0000 7f9b67c00640 1 -- 172.21.15.43:0/2719488386 shutdown_connections 2024-09-19T05:15:15.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.841+0000 7f9b67c00640 1 -- 172.21.15.43:0/2719488386 wait complete. 2024-09-19T05:15:15.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.842+0000 7f9b67c00640 1 Processor -- start 2024-09-19T05:15:15.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.842+0000 7f9b67c00640 1 -- start start 2024-09-19T05:15:15.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.842+0000 7f9b67c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b60104010 0x7f9b6019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:15.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.842+0000 7f9b67c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b601049c0 0x7f9b6019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:15.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.842+0000 7f9b67c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b60109140 0x7f9b6019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:15.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.842+0000 7f9b67c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9b601141c0 con 0x7f9b60104010 2024-09-19T05:15:15.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.842+0000 7f9b67c00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9b60114040 con 0x7f9b601049c0 2024-09-19T05:15:15.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.842+0000 7f9b67c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9b60114340 con 0x7f9b60109140 2024-09-19T05:15:15.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.843+0000 7f9b5fe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b601049c0 0x7f9b6019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:15.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.843+0000 7f9b65200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b60104010 0x7f9b6019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:15.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.843+0000 7f9b5fe00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b601049c0 0x7f9b6019ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:50056/0 (socket says 172.21.15.43:50056) 2024-09-19T05:15:15.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.843+0000 7f9b5fe00640 1 -- 172.21.15.43:0/2237499772 learned_addr learned my addr 172.21.15.43:0/2237499772 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:15:15.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.843+0000 7f9b65c00640 1 --2- 172.21.15.43:0/2237499772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b60109140 0x7f9b6019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:15.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.843+0000 7f9b5fe00640 1 -- 172.21.15.43:0/2237499772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b60109140 msgr2=0x7f9b6019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:15.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.843+0000 7f9b5fe00640 1 --2- 172.21.15.43:0/2237499772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b60109140 0x7f9b6019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:15.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.843+0000 7f9b5fe00640 1 -- 172.21.15.43:0/2237499772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b60104010 msgr2=0x7f9b6019a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:15.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.843+0000 7f9b5fe00640 1 --2- 172.21.15.43:0/2237499772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b60104010 0x7f9b6019a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:15.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.843+0000 7f9b5fe00640 1 -- 172.21.15.43:0/2237499772 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b6019f5e0 con 0x7f9b601049c0 2024-09-19T05:15:15.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.843+0000 7f9b65c00640 1 --2- 172.21.15.43:0/2237499772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b60109140 0x7f9b6019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:15:15.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.844+0000 7f9b5fe00640 1 --2- 172.21.15.43:0/2237499772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b601049c0 0x7f9b6019ab70 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c00e9f0 tx=0x7f9b4c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:15.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.844+0000 7f9b5d600640 1 -- 172.21.15.43:0/2237499772 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c00cd80 con 0x7f9b601049c0 2024-09-19T05:15:15.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.844+0000 7f9b67c00640 1 -- 172.21.15.43:0/2237499772 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b6019f8d0 con 0x7f9b601049c0 2024-09-19T05:15:15.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.844+0000 7f9b5d600640 1 -- 172.21.15.43:0/2237499772 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b4c010910 con 0x7f9b601049c0 2024-09-19T05:15:15.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.844+0000 7f9b5d600640 1 -- 172.21.15.43:0/2237499772 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c0188f0 con 0x7f9b601049c0 2024-09-19T05:15:15.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.844+0000 7f9b67c00640 1 -- 172.21.15.43:0/2237499772 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b601a71b0 con 0x7f9b601049c0 2024-09-19T05:15:15.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.847+0000 7f9b6a87c740 1 Processor -- start 2024-09-19T05:15:15.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.847+0000 7f9b6a87c740 1 -- start start 2024-09-19T05:15:15.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.847+0000 7f9b6a87c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558366b8cc70 0x558366b8d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:15.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.848+0000 7f9b6a87c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558366b8d590 0x558366b97910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:15.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.848+0000 7f9b6a87c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558366b97e50 0x558366b9a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:15.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.848+0000 7f9b6a87c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558366af33e0 con 0x558366b8cc70 2024-09-19T05:15:15.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.848+0000 7f9b6a87c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558366a9ad80 con 0x558366b97e50 2024-09-19T05:15:15.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.848+0000 7f9b6a87c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558366aef3a0 con 0x558366b8d590 2024-09-19T05:15:15.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.848+0000 7f9b65c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558366b8cc70 0x558366b8d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:15.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.848+0000 7f9b5d600640 1 -- 172.21.15.43:0/2237499772 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f9b4c010ac0 con 0x7f9b601049c0 2024-09-19T05:15:15.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.848+0000 7f9b65200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558366b8d590 0x558366b97910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:15.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.848+0000 7f9b65200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558366b8d590 0x558366b97910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:51340/0 (socket says 172.21.15.43:51340) 2024-09-19T05:15:15.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.848+0000 7f9b65200640 1 -- 172.21.15.43:0/638138407 learned_addr learned my addr 172.21.15.43:0/638138407 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:15:15.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.848+0000 7f9b5fe00640 1 --2- 172.21.15.43:0/638138407 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558366b97e50 0x558366b9a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:15.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.848+0000 7f9b65200640 1 -- 172.21.15.43:0/638138407 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558366b97e50 msgr2=0x558366b9a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:15.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.848+0000 7f9b65200640 1 --2- 172.21.15.43:0/638138407 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558366b97e50 0x558366b9a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:15.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.848+0000 7f9b65200640 1 -- 172.21.15.43:0/638138407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558366b8cc70 msgr2=0x558366b8d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:15.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.848+0000 7f9b65200640 1 --2- 172.21.15.43:0/638138407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558366b8cc70 0x558366b8d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:15.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.848+0000 7f9b65200640 1 -- 172.21.15.43:0/638138407 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558366ab9ae0 con 0x558366b8d590 2024-09-19T05:15:15.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.848+0000 7f9b5fe00640 1 --2- 172.21.15.43:0/638138407 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558366b97e50 0x558366b9a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:15:15.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.849+0000 7f9b65c00640 1 --2- 172.21.15.43:0/638138407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558366b8cc70 0x558366b8d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:15:15.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.849+0000 7f9b65200640 1 --2- 172.21.15.43:0/638138407 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558366b8d590 0x558366b97910 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9b54009900 tx=0x7f9b54005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d095e8a701661bb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:15.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.849+0000 7f9b52000640 1 -- 172.21.15.43:0/638138407 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b54035040 con 0x558366b8d590 2024-09-19T05:15:15.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.849+0000 7f9b52000640 1 -- 172.21.15.43:0/638138407 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b54004400 con 0x558366b8d590 2024-09-19T05:15:15.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.849+0000 7f9b52000640 1 -- 172.21.15.43:0/638138407 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b540035f0 con 0x558366b8d590 2024-09-19T05:15:15.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.850+0000 7f9b5d600640 1 --2- 172.21.15.43:0/2237499772 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f9b480a33b0 0x7f9b480a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:15.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.850+0000 7f9b6a87c740 1 -- 172.21.15.43:0/638138407 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558366b8d590 msgr2=0x558366b97910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:15.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.850+0000 7f9b6a87c740 1 --2- 172.21.15.43:0/638138407 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558366b8d590 0x558366b97910 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9b54009900 tx=0x7f9b54005de0 comp rx=0 tx=0).stop 2024-09-19T05:15:15.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.850+0000 7f9b5d600640 1 -- 172.21.15.43:0/2237499772 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8586+0+0 (secure 0 0 0) 0x7f9b4c0d0170 con 0x7f9b601049c0 2024-09-19T05:15:15.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.850+0000 7f9b65c00640 1 --2- 172.21.15.43:0/2237499772 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f9b480a33b0 0x7f9b480a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:15.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.850+0000 7f9b6a87c740 1 -- 172.21.15.43:0/638138407 shutdown_connections 2024-09-19T05:15:15.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.850+0000 7f9b6a87c740 1 --2- 172.21.15.43:0/638138407 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558366b97e50 0x558366b9a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:15.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.850+0000 7f9b6a87c740 1 --2- 172.21.15.43:0/638138407 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558366b8d590 0x558366b97910 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:15.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.850+0000 7f9b6a87c740 1 --2- 172.21.15.43:0/638138407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558366b8cc70 0x558366b8d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:15.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.850+0000 7f9b6a87c740 1 -- 172.21.15.43:0/638138407 >> 172.21.15.43:0/638138407 conn(0x558366b89200 msgr2=0x558366b895e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:15.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.850+0000 7f9b6a87c740 1 -- 172.21.15.43:0/638138407 shutdown_connections 2024-09-19T05:15:15.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.850+0000 7f9b6a87c740 1 -- 172.21.15.43:0/638138407 wait complete. 2024-09-19T05:15:15.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.851+0000 7f9b65c00640 1 --2- 172.21.15.43:0/2237499772 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f9b480a33b0 0x7f9b480a5870 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9b580115a0 tx=0x7f9b58009500 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:15.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.851+0000 7f9b6a87c740 1 -- start start 2024-09-19T05:15:15.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.851+0000 7f9b6a87c740 1 Processor -- start 2024-09-19T05:15:15.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.852+0000 7f9b6a87c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558366b8cc70 0x558366b94ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:15.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.852+0000 7f9b6a87c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558366b8d590 0x558366b95220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:15.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.852+0000 7f9b6a87c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558366b97e50 0x558366b95760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:15.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.852+0000 7f9b6a87c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558366b9aa20 con 0x558366b97e50 2024-09-19T05:15:15.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.852+0000 7f9b6a87c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558366aae0f0 con 0x558366b8d590 2024-09-19T05:15:15.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.852+0000 7f9b6a87c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558366b9aba0 con 0x558366b8cc70 2024-09-19T05:15:15.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.852+0000 7f9b6a87c740 10 client.0 _ll_register_callbacks cb 0x558366a95bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:15:15.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.852+0000 7f9b65200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558366b8cc70 0x558366b94ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:15.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.852+0000 7f9b65c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558366b97e50 0x558366b95760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:15.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.852+0000 7f9b65200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558366b8cc70 0x558366b94ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:51348/0 (socket says 172.21.15.43:51348) 2024-09-19T05:15:15.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.852+0000 7f9b65200640 1 -- 172.21.15.43:0/3844884620 learned_addr learned my addr 172.21.15.43:0/3844884620 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:15:15.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.852+0000 7f9b5fe00640 1 --2- 172.21.15.43:0/3844884620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558366b8d590 0x558366b95220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:15.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.852+0000 7f9b65c00640 1 -- 172.21.15.43:0/3844884620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558366b8cc70 msgr2=0x558366b94ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:15.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.852+0000 7f9b65c00640 1 --2- 172.21.15.43:0/3844884620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558366b8cc70 0x558366b94ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:15.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.852+0000 7f9b65c00640 1 -- 172.21.15.43:0/3844884620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558366b8d590 msgr2=0x558366b95220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:15.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.852+0000 7f9b65c00640 1 --2- 172.21.15.43:0/3844884620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558366b8d590 0x558366b95220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:15.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.852+0000 7f9b65c00640 1 -- 172.21.15.43:0/3844884620 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583668643c0 con 0x558366b97e50 2024-09-19T05:15:15.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.852+0000 7f9b65200640 1 --2- 172.21.15.43:0/3844884620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558366b8cc70 0x558366b94ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:15:15.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.853+0000 7f9b65c00640 1 --2- 172.21.15.43:0/3844884620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558366b97e50 0x558366b95760 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f9b58005fd0 tx=0x7f9b58004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff3a10267d341d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:15.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.853+0000 7f9b46a00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:15:15.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.853+0000 7f9b46a00640 1 -- 172.21.15.43:0/3844884620 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b58011ce0 con 0x558366b97e50 2024-09-19T05:15:15.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.853+0000 7f9b6a87c740 1 -- 172.21.15.43:0/3844884620 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b54004400 con 0x558366b97e50 2024-09-19T05:15:15.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.853+0000 7f9b46a00640 1 -- 172.21.15.43:0/3844884620 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8586+0+0 (secure 0 0 0) 0x7f9b5801f420 con 0x558366b97e50 2024-09-19T05:15:15.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.853+0000 7f9b46a00640 1 -- 172.21.15.43:0/3844884620 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9b5801f620 con 0x558366b97e50 2024-09-19T05:15:15.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.853+0000 7f9b6a87c740 10 client.17679 fetch_fsmap learned FSMap version 29 2024-09-19T05:15:15.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.853+0000 7f9b6a87c740 1 -- 172.21.15.43:0/3844884620 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558366ba6a40 con 0x558366b97e50 2024-09-19T05:15:15.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.854+0000 7f9b46a00640 1 -- 172.21.15.43:0/3844884620 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b54004400 con 0x558366b97e50 2024-09-19T05:15:15.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.854+0000 7f9b46a00640 1 -- 172.21.15.43:0/3844884620 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9b5802f030 con 0x558366b97e50 2024-09-19T05:15:15.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.854+0000 7f9b6a87c740 10 client.17679 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:15:15.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.854+0000 7f9b6a87c740 10 client.17679 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-19T05:15:15.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.854+0000 7f9b6a87c740 20 client.17679 populate_metadata read hostname 'smithi043' 2024-09-19T05:15:15.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.854+0000 7f9b6a87c740 1 --2- 172.21.15.43:0/3844884620 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x558366ba7e50 0x558366baa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:15.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.854+0000 7f9b6a87c740 4 client.17679 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:15:15.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.854+0000 7f9b6a87c740 1 -- 172.21.15.43:0/3844884620 --> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558366b00950 con 0x558366ba7e50 2024-09-19T05:15:15.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.855+0000 7f9b65200640 1 --2- 172.21.15.43:0/3844884620 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x558366ba7e50 0x558366baa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:15.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.855+0000 7f9b65200640 1 --2- 172.21.15.43:0/3844884620 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x558366ba7e50 0x558366baa2f0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3160db75d64d880f server_cookie=41fbfe8406166870 in_seq=0 out_seq=0 2024-09-19T05:15:15.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.855+0000 7f9b46a00640 10 client.17679 ms_handle_connect on v2:172.21.15.151:6836/3960378010 2024-09-19T05:15:15.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.856+0000 7f9b46a00640 1 -- 172.21.15.43:0/3844884620 <== mds.4 v2:172.21.15.151:6836/3960378010 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558366b00950 con 0x558366ba7e50 2024-09-19T05:15:15.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.856+0000 7f9b46a00640 10 client.17679 handle_command_reply: tid=0 2024-09-19T05:15:15.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.856+0000 7f9b6a87c740 1 client.17679 shutdown 2024-09-19T05:15:15.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.856+0000 7f9b6a87c740 1 -- 172.21.15.43:0/3844884620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558366b97e50 msgr2=0x558366b95760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:15.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.856+0000 7f9b6a87c740 1 --2- 172.21.15.43:0/3844884620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558366b97e50 0x558366b95760 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f9b58005fd0 tx=0x7f9b58004ab0 comp rx=0 tx=0).stop 2024-09-19T05:15:15.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.856+0000 7f9b6a87c740 1 -- 172.21.15.43:0/3844884620 shutdown_connections 2024-09-19T05:15:15.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.856+0000 7f9b6a87c740 1 --2- 172.21.15.43:0/3844884620 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x558366ba7e50 0x558366baa2f0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:15.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.856+0000 7f9b6a87c740 1 --2- 172.21.15.43:0/3844884620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558366b97e50 0x558366b95760 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:15.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.856+0000 7f9b6a87c740 1 --2- 172.21.15.43:0/3844884620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558366b8d590 0x558366b95220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:15.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.856+0000 7f9b6a87c740 1 --2- 172.21.15.43:0/3844884620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558366b8cc70 0x558366b94ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:15.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.856+0000 7f9b46a00640 0 client.17679 ms_handle_reset on v2:172.21.15.151:6836/3960378010 2024-09-19T05:15:15.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.856+0000 7f9b6a87c740 1 -- 172.21.15.43:0/3844884620 >> 172.21.15.43:0/3844884620 conn(0x558366b89200 msgr2=0x558366b8ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:15.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.856+0000 7f9b6a87c740 1 -- 172.21.15.43:0/3844884620 shutdown_connections 2024-09-19T05:15:15.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.857+0000 7f9b6a87c740 1 -- 172.21.15.43:0/3844884620 wait complete. 2024-09-19T05:15:15.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.857+0000 7f9b6a87c740 20 client.17679 trim_cache size 0 max 16384 2024-09-19T05:15:15.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.870+0000 7f9b6a87c740 1 Processor -- start 2024-09-19T05:15:15.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.870+0000 7f9b6a87c740 1 -- start start 2024-09-19T05:15:15.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.871+0000 7f9b6a87c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558366b8cc70 0x558366b8f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:15.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.871+0000 7f9b6a87c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558366b8d590 0x558366b96c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:15.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.871+0000 7f9b6a87c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558366b97e50 0x558366bafc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:15.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.871+0000 7f9b6a87c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9b601afbf0 con 0x558366b8d590 2024-09-19T05:15:15.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.871+0000 7f9b6a87c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9b601b1730 con 0x558366b97e50 2024-09-19T05:15:15.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.871+0000 7f9b6a87c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9b601b02c0 con 0x558366b8cc70 2024-09-19T05:15:15.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.871+0000 7f9b5fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558366b8d590 0x558366b96c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:15.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.871+0000 7f9b65200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558366b8cc70 0x558366b8f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:15.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.871+0000 7f9b5fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558366b8d590 0x558366b96c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34172/0 (socket says 172.21.15.43:34172) 2024-09-19T05:15:15.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.871+0000 7f9b5fe00640 1 -- 172.21.15.43:0/3359085252 learned_addr learned my addr 172.21.15.43:0/3359085252 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:15:15.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.871+0000 7f9b65c00640 1 --2- 172.21.15.43:0/3359085252 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558366b97e50 0x558366bafc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:15.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.871+0000 7f9b65200640 1 -- 172.21.15.43:0/3359085252 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558366b97e50 msgr2=0x558366bafc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:15.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.871+0000 7f9b65200640 1 --2- 172.21.15.43:0/3359085252 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558366b97e50 0x558366bafc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:15.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.871+0000 7f9b65200640 1 -- 172.21.15.43:0/3359085252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558366b8d590 msgr2=0x558366b96c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:15.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.871+0000 7f9b65200640 1 --2- 172.21.15.43:0/3359085252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558366b8d590 0x558366b96c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:15.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.871+0000 7f9b65200640 1 -- 172.21.15.43:0/3359085252 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558366b89030 con 0x558366b8cc70 2024-09-19T05:15:15.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.872+0000 7f9b65200640 1 --2- 172.21.15.43:0/3359085252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558366b8cc70 0x558366b8f720 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9b54005c40 tx=0x7f9b54004be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2289fea69933b81 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:15.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.872+0000 7f9b51600640 1 -- 172.21.15.43:0/3359085252 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b54035040 con 0x558366b8cc70 2024-09-19T05:15:15.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.872+0000 7f9b51600640 1 -- 172.21.15.43:0/3359085252 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b540046c0 con 0x558366b8cc70 2024-09-19T05:15:15.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.872+0000 7f9b51600640 1 -- 172.21.15.43:0/3359085252 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b540039e0 con 0x558366b8cc70 2024-09-19T05:15:15.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.873+0000 7f9b6a87c740 1 -- 172.21.15.43:0/3359085252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558366b8cc70 msgr2=0x558366b8f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:15.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.873+0000 7f9b6a87c740 1 --2- 172.21.15.43:0/3359085252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558366b8cc70 0x558366b8f720 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9b54005c40 tx=0x7f9b54004be0 comp rx=0 tx=0).stop 2024-09-19T05:15:15.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.873+0000 7f9b6a87c740 1 -- 172.21.15.43:0/3359085252 shutdown_connections 2024-09-19T05:15:15.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.873+0000 7f9b6a87c740 1 --2- 172.21.15.43:0/3359085252 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558366b97e50 0x558366bafc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:15.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.873+0000 7f9b6a87c740 1 --2- 172.21.15.43:0/3359085252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558366b8d590 0x558366b96c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:15.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.873+0000 7f9b6a87c740 1 --2- 172.21.15.43:0/3359085252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558366b8cc70 0x558366b8f720 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:15.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.873+0000 7f9b6a87c740 1 -- 172.21.15.43:0/3359085252 >> 172.21.15.43:0/3359085252 conn(0x558366b89200 msgr2=0x558366b96540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:15.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.873+0000 7f9b6a87c740 1 -- 172.21.15.43:0/3359085252 shutdown_connections 2024-09-19T05:15:15.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.873+0000 7f9b6a87c740 1 -- 172.21.15.43:0/3359085252 wait complete. 2024-09-19T05:15:15.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.873+0000 7f9b6a87c740 1 -- start start 2024-09-19T05:15:15.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.874+0000 7f9b6a87c740 1 Processor -- start 2024-09-19T05:15:15.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.874+0000 7f9b6a87c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558366b8cc70 0x558366b9ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:15.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.874+0000 7f9b6a87c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558366b8d590 0x558366ba0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:15.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.874+0000 7f9b6a87c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558366b97e50 0x558366b9b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:15.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.874+0000 7f9b6a87c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9b601b1060 con 0x558366b8d590 2024-09-19T05:15:15.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.874+0000 7f9b6a87c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9b601b0990 con 0x558366b97e50 2024-09-19T05:15:15.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.874+0000 7f9b6a87c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9b601adbf0 con 0x558366b8cc70 2024-09-19T05:15:15.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.874+0000 7f9b6a87c740 10 client.0 _ll_register_callbacks cb 0x7f9b58012050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:15:15.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.874+0000 7f9b65200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558366b8cc70 0x558366b9ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:15.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.874+0000 7f9b65200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558366b8cc70 0x558366b9ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:51364/0 (socket says 172.21.15.43:51364) 2024-09-19T05:15:15.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.874+0000 7f9b65200640 1 -- 172.21.15.43:0/1303165532 learned_addr learned my addr 172.21.15.43:0/1303165532 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:15:15.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.874+0000 7f9b5fe00640 1 --2- 172.21.15.43:0/1303165532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558366b8d590 0x558366ba0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:15.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.874+0000 7f9b65c00640 1 --2- 172.21.15.43:0/1303165532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558366b97e50 0x558366b9b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:15.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.875+0000 7f9b65200640 1 -- 172.21.15.43:0/1303165532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558366b97e50 msgr2=0x558366b9b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:15.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.875+0000 7f9b65200640 1 --2- 172.21.15.43:0/1303165532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558366b97e50 0x558366b9b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:15.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.875+0000 7f9b65200640 1 -- 172.21.15.43:0/1303165532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558366b8d590 msgr2=0x558366ba0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:15.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.875+0000 7f9b65200640 1 --2- 172.21.15.43:0/1303165532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558366b8d590 0x558366ba0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:15.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.875+0000 7f9b65200640 1 -- 172.21.15.43:0/1303165532 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558366b84770 con 0x558366b8cc70 2024-09-19T05:15:15.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.875+0000 7f9b65200640 1 --2- 172.21.15.43:0/1303165532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558366b8cc70 0x558366b9ffc0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9b5402a430 tx=0x7f9b54004290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2de2a0c63bc87d99 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:15.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.875+0000 7f9b46a00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:15:15.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.875+0000 7f9b46a00640 1 -- 172.21.15.43:0/1303165532 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b54035070 con 0x558366b8cc70 2024-09-19T05:15:15.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.875+0000 7f9b6a87c740 1 -- 172.21.15.43:0/1303165532 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b540046c0 con 0x558366b8cc70 2024-09-19T05:15:15.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.875+0000 7f9b46a00640 1 -- 172.21.15.43:0/1303165532 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8586+0+0 (secure 0 0 0) 0x7f9b540037f0 con 0x558366b8cc70 2024-09-19T05:15:15.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.876+0000 7f9b46a00640 1 -- 172.21.15.43:0/1303165532 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9b540039f0 con 0x558366b8cc70 2024-09-19T05:15:15.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.876+0000 7f9b6a87c740 10 client.27607 fetch_fsmap learned FSMap version 29 2024-09-19T05:15:15.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.876+0000 7f9b6a87c740 1 -- 172.21.15.43:0/1303165532 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558366c12510 con 0x558366b8cc70 2024-09-19T05:15:15.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.876+0000 7f9b46a00640 1 -- 172.21.15.43:0/1303165532 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b540046c0 con 0x558366b8cc70 2024-09-19T05:15:15.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.876+0000 7f9b46a00640 1 -- 172.21.15.43:0/1303165532 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9b54004c50 con 0x558366b8cc70 2024-09-19T05:15:15.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.876+0000 7f9b6a87c740 10 client.27607 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:15:15.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.876+0000 7f9b6a87c740 10 client.27607 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-19T05:15:15.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.876+0000 7f9b6a87c740 20 client.27607 populate_metadata read hostname 'smithi043' 2024-09-19T05:15:15.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.876+0000 7f9b6a87c740 1 --2- 172.21.15.43:0/1303165532 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x558366c126e0 0x558366c14ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:15.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.876+0000 7f9b6a87c740 4 client.27607 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:15:15.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.876+0000 7f9b6a87c740 1 -- 172.21.15.43:0/1303165532 --> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558366afae60 con 0x558366c126e0 2024-09-19T05:15:15.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.877+0000 7f9b5fe00640 1 --2- 172.21.15.43:0/1303165532 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x558366c126e0 0x558366c14ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:15.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.877+0000 7f9b5fe00640 1 --2- 172.21.15.43:0/1303165532 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x558366c126e0 0x558366c14ad0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f8e3779ad7f8b53d server_cookie=a14275d61f854476 in_seq=0 out_seq=0 2024-09-19T05:15:15.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.877+0000 7f9b46a00640 10 client.27607 ms_handle_connect on v2:172.21.15.151:6836/3960378010 2024-09-19T05:15:15.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.877+0000 7f9b46a00640 1 -- 172.21.15.43:0/1303165532 <== mds.4 v2:172.21.15.151:6836/3960378010 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558366afae60 con 0x558366c126e0 2024-09-19T05:15:15.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.877+0000 7f9b46a00640 10 client.27607 handle_command_reply: tid=0 2024-09-19T05:15:15.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.877+0000 7f9b6a87c740 1 client.27607 shutdown 2024-09-19T05:15:15.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.878+0000 7f9b6a87c740 1 -- 172.21.15.43:0/1303165532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558366b8cc70 msgr2=0x558366b9ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:15.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.878+0000 7f9b6a87c740 1 --2- 172.21.15.43:0/1303165532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558366b8cc70 0x558366b9ffc0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9b5402a430 tx=0x7f9b54004290 comp rx=0 tx=0).stop 2024-09-19T05:15:15.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.878+0000 7f9b6a87c740 1 -- 172.21.15.43:0/1303165532 shutdown_connections 2024-09-19T05:15:15.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.878+0000 7f9b6a87c740 1 --2- 172.21.15.43:0/1303165532 >> [v2:172.21.15.151:6836/3960378010,v1:172.21.15.151:6837/3960378010] conn(0x558366c126e0 0x558366c14ad0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:15.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.878+0000 7f9b6a87c740 1 --2- 172.21.15.43:0/1303165532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558366b97e50 0x558366b9b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:15.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.878+0000 7f9b6a87c740 1 --2- 172.21.15.43:0/1303165532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558366b8d590 0x558366ba0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:15.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.878+0000 7f9b6a87c740 1 --2- 172.21.15.43:0/1303165532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558366b8cc70 0x558366b9ffc0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:15.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.878+0000 7f9b46a00640 0 client.27607 ms_handle_reset on v2:172.21.15.151:6836/3960378010 2024-09-19T05:15:15.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.878+0000 7f9b6a87c740 1 -- 172.21.15.43:0/1303165532 >> 172.21.15.43:0/1303165532 conn(0x558366b89200 msgr2=0x558366b87b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:15.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.878+0000 7f9b6a87c740 1 -- 172.21.15.43:0/1303165532 shutdown_connections 2024-09-19T05:15:15.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.878+0000 7f9b6a87c740 1 -- 172.21.15.43:0/1303165532 wait complete. 2024-09-19T05:15:15.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.878+0000 7f9b6a87c740 20 client.27607 trim_cache size 0 max 16384 2024-09-19T05:15:15.884 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:15:15.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.878+0000 7f9b67c00640 1 -- 172.21.15.43:0/2237499772 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f9b480a33b0 msgr2=0x7f9b480a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:15.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.879+0000 7f9b67c00640 1 --2- 172.21.15.43:0/2237499772 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f9b480a33b0 0x7f9b480a5870 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9b580115a0 tx=0x7f9b58009500 comp rx=0 tx=0).stop 2024-09-19T05:15:15.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.879+0000 7f9b67c00640 1 -- 172.21.15.43:0/2237499772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b601049c0 msgr2=0x7f9b6019ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:15.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.879+0000 7f9b67c00640 1 --2- 172.21.15.43:0/2237499772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b601049c0 0x7f9b6019ab70 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c00e9f0 tx=0x7f9b4c00eeb0 comp rx=0 tx=0).stop 2024-09-19T05:15:15.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.879+0000 7f9b67c00640 1 -- 172.21.15.43:0/2237499772 shutdown_connections 2024-09-19T05:15:15.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.879+0000 7f9b67c00640 1 --2- 172.21.15.43:0/2237499772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b60109140 0x7f9b6019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:15.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.879+0000 7f9b67c00640 1 --2- 172.21.15.43:0/2237499772 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f9b480a33b0 0x7f9b480a5870 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:15.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.879+0000 7f9b67c00640 1 --2- 172.21.15.43:0/2237499772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b601049c0 0x7f9b6019ab70 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:15.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.879+0000 7f9b67c00640 1 --2- 172.21.15.43:0/2237499772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b60104010 0x7f9b6019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:15.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.879+0000 7f9b67c00640 1 -- 172.21.15.43:0/2237499772 >> 172.21.15.43:0/2237499772 conn(0x7f9b600ffa50 msgr2=0x7f9b60100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:15.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.879+0000 7f9b67c00640 1 -- 172.21.15.43:0/2237499772 shutdown_connections 2024-09-19T05:15:15.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:15.879+0000 7f9b67c00640 1 -- 172.21.15.43:0/2237499772 wait complete. 2024-09-19T05:15:16.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:15 smithi043 ceph-mon[30800]: pgmap v330: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T05:15:16.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:15 smithi043 ceph-mon[30800]: osdmap e96: 12 total, 12 up, 12 in 2024-09-19T05:15:16.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:15 smithi096 ceph-mon[35085]: pgmap v330: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T05:15:16.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:15 smithi096 ceph-mon[35085]: osdmap e96: 12 total, 12 up, 12 in 2024-09-19T05:15:16.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:15 smithi151 ceph-mon[35425]: pgmap v330: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-19T05:15:16.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:15 smithi151 ceph-mon[35425]: osdmap e96: 12 total, 12 up, 12 in 2024-09-19T05:15:16.664 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T05:15:17.045 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:15:17.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.600+0000 7fa5b5200640 1 -- 172.21.15.43:0/3812792169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5b0106f10 msgr2=0x7fa5b010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:17.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.600+0000 7fa5b5200640 1 --2- 172.21.15.43:0/3812792169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5b0106f10 0x7fa5b010ea90 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8009ca0 tx=0x7fa5a802a8b0 comp rx=0 tx=0).stop 2024-09-19T05:15:17.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.601+0000 7fa5b5200640 1 -- 172.21.15.43:0/3812792169 shutdown_connections 2024-09-19T05:15:17.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.601+0000 7fa5b5200640 1 --2- 172.21.15.43:0/3812792169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5b0106f10 0x7fa5b010ea90 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:17.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.601+0000 7fa5b5200640 1 --2- 172.21.15.43:0/3812792169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5b0102790 0x7fa5b01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:17.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.601+0000 7fa5b5200640 1 --2- 172.21.15.43:0/3812792169 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5b0101de0 0x7fa5b01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:17.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.601+0000 7fa5b5200640 1 -- 172.21.15.43:0/3812792169 >> 172.21.15.43:0/3812792169 conn(0x7fa5b00fd8c0 msgr2=0x7fa5b00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:17.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.601+0000 7fa5b5200640 1 -- 172.21.15.43:0/3812792169 shutdown_connections 2024-09-19T05:15:17.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.601+0000 7fa5b5200640 1 -- 172.21.15.43:0/3812792169 wait complete. 2024-09-19T05:15:17.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.602+0000 7fa5b5200640 1 Processor -- start 2024-09-19T05:15:17.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.602+0000 7fa5b5200640 1 -- start start 2024-09-19T05:15:17.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.603+0000 7fa5b5200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5b0101de0 0x7fa5b01983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:17.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.603+0000 7fa5b5200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5b0102790 0x7fa5b01988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:17.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.603+0000 7fa5b5200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5b0106f10 0x7fa5b019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:17.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.603+0000 7fa5b5200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b0111f70 con 0x7fa5b0106f10 2024-09-19T05:15:17.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.603+0000 7fa5b5200640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b0111df0 con 0x7fa5b0102790 2024-09-19T05:15:17.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.603+0000 7fa5b5200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b01120f0 con 0x7fa5b0101de0 2024-09-19T05:15:17.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.603+0000 7fa5ae800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5b0106f10 0x7fa5b019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:17.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.603+0000 7fa5ade00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5b0101de0 0x7fa5b01983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:17.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.603+0000 7fa5a7e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5b0102790 0x7fa5b01988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:17.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.603+0000 7fa5ade00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5b0101de0 0x7fa5b01983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:51368/0 (socket says 172.21.15.43:51368) 2024-09-19T05:15:17.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.603+0000 7fa5ade00640 1 -- 172.21.15.43:0/2165277779 learned_addr learned my addr 172.21.15.43:0/2165277779 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:15:17.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.604+0000 7fa5ade00640 1 -- 172.21.15.43:0/2165277779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5b0102790 msgr2=0x7fa5b01988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:17.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.604+0000 7fa5ade00640 1 --2- 172.21.15.43:0/2165277779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5b0102790 0x7fa5b01988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:17.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.604+0000 7fa5ade00640 1 -- 172.21.15.43:0/2165277779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5b0106f10 msgr2=0x7fa5b019cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:17.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.604+0000 7fa5ade00640 1 --2- 172.21.15.43:0/2165277779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5b0106f10 0x7fa5b019cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:17.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.604+0000 7fa5ade00640 1 -- 172.21.15.43:0/2165277779 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5b019d360 con 0x7fa5b0101de0 2024-09-19T05:15:17.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.604+0000 7fa5a7e00640 1 --2- 172.21.15.43:0/2165277779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5b0102790 0x7fa5b01988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:15:17.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.604+0000 7fa5ade00640 1 --2- 172.21.15.43:0/2165277779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5b0101de0 0x7fa5b01983b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa5a000e9f0 tx=0x7fa5a000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:17.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.604+0000 7fa5a6a00640 1 -- 172.21.15.43:0/2165277779 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a000cd80 con 0x7fa5b0101de0 2024-09-19T05:15:17.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.604+0000 7fa5b5200640 1 -- 172.21.15.43:0/2165277779 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5b019d650 con 0x7fa5b0101de0 2024-09-19T05:15:17.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.604+0000 7fa5a6a00640 1 -- 172.21.15.43:0/2165277779 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5a0005500 con 0x7fa5b0101de0 2024-09-19T05:15:17.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.604+0000 7fa5a6a00640 1 -- 172.21.15.43:0/2165277779 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a0010690 con 0x7fa5b0101de0 2024-09-19T05:15:17.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.604+0000 7fa5b5200640 1 -- 172.21.15.43:0/2165277779 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5b01a4f30 con 0x7fa5b0101de0 2024-09-19T05:15:17.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.608+0000 7fa5b7e70740 1 Processor -- start 2024-09-19T05:15:17.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.608+0000 7fa5b7e70740 1 -- start start 2024-09-19T05:15:17.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.608+0000 7fa5a6a00640 1 -- 172.21.15.43:0/2165277779 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa5a00046c0 con 0x7fa5b0101de0 2024-09-19T05:15:17.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.608+0000 7fa5b7e70740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55768e043b70 0x55768e043f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:17.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.608+0000 7fa5b7e70740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55768e044490 0x55768e052d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:17.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.608+0000 7fa5b7e70740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55768e053260 0x55768e055650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:17.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.608+0000 7fa5b7e70740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55768dfaa3e0 con 0x55768e043b70 2024-09-19T05:15:17.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.608+0000 7fa5b7e70740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55768df51d80 con 0x55768e044490 2024-09-19T05:15:17.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.608+0000 7fa5b7e70740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55768dfa63a0 con 0x55768e053260 2024-09-19T05:15:17.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.608+0000 7fa5ae800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55768e043b70 0x55768e043f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:17.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.608+0000 7fa5ae800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55768e043b70 0x55768e043f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34196/0 (socket says 172.21.15.43:34196) 2024-09-19T05:15:17.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.608+0000 7fa5ae800640 1 -- 172.21.15.43:0/1986274231 learned_addr learned my addr 172.21.15.43:0/1986274231 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:15:17.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.609+0000 7fa5ade00640 1 --2- 172.21.15.43:0/1986274231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55768e044490 0x55768e052d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:17.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.609+0000 7fa5a7e00640 1 --2- 172.21.15.43:0/1986274231 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55768e053260 0x55768e055650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:17.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.609+0000 7fa5ae800640 1 -- 172.21.15.43:0/1986274231 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55768e053260 msgr2=0x55768e055650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:17.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.609+0000 7fa5ae800640 1 --2- 172.21.15.43:0/1986274231 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55768e053260 0x55768e055650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:17.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.609+0000 7fa5ae800640 1 -- 172.21.15.43:0/1986274231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55768e044490 msgr2=0x55768e052d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:17.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.609+0000 7fa5ae800640 1 --2- 172.21.15.43:0/1986274231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55768e044490 0x55768e052d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:17.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.609+0000 7fa5ae800640 1 -- 172.21.15.43:0/1986274231 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55768df70ae0 con 0x55768e043b70 2024-09-19T05:15:17.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.609+0000 7fa5a7e00640 1 --2- 172.21.15.43:0/1986274231 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55768e053260 0x55768e055650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:15:17.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.609+0000 7fa5ade00640 1 --2- 172.21.15.43:0/1986274231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55768e044490 0x55768e052d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:15:17.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.609+0000 7fa5ae800640 1 --2- 172.21.15.43:0/1986274231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55768e043b70 0x55768e043f50 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8004ad0 tx=0x7fa5a8009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afe9477038ffb801 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:17.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.610+0000 7fa59f400640 1 -- 172.21.15.43:0/1986274231 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a8030660 con 0x55768e043b70 2024-09-19T05:15:17.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.610+0000 7fa59f400640 1 -- 172.21.15.43:0/1986274231 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5a8036030 con 0x55768e043b70 2024-09-19T05:15:17.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.610+0000 7fa59f400640 1 -- 172.21.15.43:0/1986274231 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a8004440 con 0x55768e043b70 2024-09-19T05:15:17.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.610+0000 7fa5a6a00640 1 --2- 172.21.15.43:0/2165277779 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fa5940a3430 0x7fa5940a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:17.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.610+0000 7fa5a6a00640 1 -- 172.21.15.43:0/2165277779 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7fa5a00d0110 con 0x7fa5b0101de0 2024-09-19T05:15:17.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.610+0000 7fa5ade00640 1 --2- 172.21.15.43:0/2165277779 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fa5940a3430 0x7fa5940a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:17.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.610+0000 7fa5b7e70740 1 -- 172.21.15.43:0/1986274231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55768e043b70 msgr2=0x55768e043f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:17.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.610+0000 7fa5b7e70740 1 --2- 172.21.15.43:0/1986274231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55768e043b70 0x55768e043f50 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8004ad0 tx=0x7fa5a8009360 comp rx=0 tx=0).stop 2024-09-19T05:15:17.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.611+0000 7fa5b7e70740 1 -- 172.21.15.43:0/1986274231 shutdown_connections 2024-09-19T05:15:17.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.611+0000 7fa5b7e70740 1 --2- 172.21.15.43:0/1986274231 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55768e053260 0x55768e055650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:17.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.611+0000 7fa5b7e70740 1 --2- 172.21.15.43:0/1986274231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55768e044490 0x55768e052d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:17.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.611+0000 7fa5b7e70740 1 --2- 172.21.15.43:0/1986274231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55768e043b70 0x55768e043f50 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:17.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.611+0000 7fa5b7e70740 1 -- 172.21.15.43:0/1986274231 >> 172.21.15.43:0/1986274231 conn(0x55768e040200 msgr2=0x55768e0405e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:17.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.611+0000 7fa5b7e70740 1 -- 172.21.15.43:0/1986274231 shutdown_connections 2024-09-19T05:15:17.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.611+0000 7fa5b7e70740 1 -- 172.21.15.43:0/1986274231 wait complete. 2024-09-19T05:15:17.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.611+0000 7fa5ade00640 1 --2- 172.21.15.43:0/2165277779 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fa5940a3430 0x7fa5940a58f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x55768e046860 tx=0x7fa5a00bc040 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:17.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.611+0000 7fa5b7e70740 1 -- start start 2024-09-19T05:15:17.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.612+0000 7fa5b7e70740 1 Processor -- start 2024-09-19T05:15:17.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.612+0000 7fa5b7e70740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55768e043b70 0x55768e052ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:17.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.612+0000 7fa5b7e70740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55768e044490 0x55768e04bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:17.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.612+0000 7fa5b7e70740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55768e053260 0x55768e04c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:17.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.612+0000 7fa5b7e70740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55768e055e30 con 0x55768e044490 2024-09-19T05:15:17.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.612+0000 7fa5b7e70740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55768df650f0 con 0x55768e053260 2024-09-19T05:15:17.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.612+0000 7fa5b7e70740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55768e055fb0 con 0x55768e043b70 2024-09-19T05:15:17.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.612+0000 7fa5b7e70740 10 client.0 _ll_register_callbacks cb 0x55768df4cbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:15:17.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.612+0000 7fa5a7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55768e044490 0x55768e04bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:17.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.612+0000 7fa5a7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55768e044490 0x55768e04bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34204/0 (socket says 172.21.15.43:34204) 2024-09-19T05:15:17.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.612+0000 7fa5a7e00640 1 -- 172.21.15.43:0/1656217154 learned_addr learned my addr 172.21.15.43:0/1656217154 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:15:17.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.613+0000 7fa5ae800640 1 --2- 172.21.15.43:0/1656217154 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55768e043b70 0x55768e052ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:17.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.613+0000 7fa5ae800640 1 --2- 172.21.15.43:0/1656217154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55768e053260 0x55768e04c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:17.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.613+0000 7fa5a7e00640 1 -- 172.21.15.43:0/1656217154 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55768e043b70 msgr2=0x55768e052ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:17.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.613+0000 7fa5a7e00640 1 --2- 172.21.15.43:0/1656217154 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55768e043b70 0x55768e052ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:17.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.613+0000 7fa5a7e00640 1 -- 172.21.15.43:0/1656217154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55768e053260 msgr2=0x55768e04c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:17.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.613+0000 7fa5a7e00640 1 --2- 172.21.15.43:0/1656217154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55768e053260 0x55768e04c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:17.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.613+0000 7fa5a7e00640 1 -- 172.21.15.43:0/1656217154 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55768dd1b3c0 con 0x55768e044490 2024-09-19T05:15:17.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.613+0000 7fa5ae800640 1 --2- 172.21.15.43:0/1656217154 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55768e043b70 0x55768e052ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:15:17.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.613+0000 7fa5a7e00640 1 --2- 172.21.15.43:0/1656217154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55768e044490 0x55768e04bd60 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa598012980 tx=0x7fa598012e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5917ee447fd94f49 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:17.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.613+0000 7fa593e00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:15:17.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.614+0000 7fa593e00640 1 -- 172.21.15.43:0/1656217154 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59800fa00 con 0x55768e044490 2024-09-19T05:15:17.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.614+0000 7fa5b7e70740 1 -- 172.21.15.43:0/1656217154 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5a8036030 con 0x55768e044490 2024-09-19T05:15:17.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.614+0000 7fa593e00640 1 -- 172.21.15.43:0/1656217154 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7fa59800fba0 con 0x55768e044490 2024-09-19T05:15:17.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.614+0000 7fa593e00640 1 -- 172.21.15.43:0/1656217154 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa59800fda0 con 0x55768e044490 2024-09-19T05:15:17.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.614+0000 7fa5b7e70740 10 client.17697 fetch_fsmap learned FSMap version 29 2024-09-19T05:15:17.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.614+0000 7fa5b7e70740 1 -- 172.21.15.43:0/1656217154 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55768e066400 con 0x55768e044490 2024-09-19T05:15:17.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.614+0000 7fa593e00640 1 -- 172.21.15.43:0/1656217154 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5a8036030 con 0x55768e044490 2024-09-19T05:15:17.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.614+0000 7fa593e00640 1 -- 172.21.15.43:0/1656217154 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa598010550 con 0x55768e044490 2024-09-19T05:15:17.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.615+0000 7fa5b7e70740 10 client.17697 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:15:17.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.615+0000 7fa5b7e70740 10 client.17697 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T05:15:17.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.615+0000 7fa5b7e70740 20 client.17697 populate_metadata read hostname 'smithi043' 2024-09-19T05:15:17.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.615+0000 7fa5b7e70740 1 --2- 172.21.15.43:0/1656217154 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55768e067860 0x55768e069c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:17.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.615+0000 7fa5b7e70740 4 client.17697 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:15:17.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.615+0000 7fa5b7e70740 1 -- 172.21.15.43:0/1656217154 --> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55768dfb7950 con 0x55768e067860 2024-09-19T05:15:17.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.615+0000 7fa5ae800640 1 --2- 172.21.15.43:0/1656217154 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55768e067860 0x55768e069c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:17.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.615+0000 7fa5ae800640 1 --2- 172.21.15.43:0/1656217154 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55768e067860 0x55768e069c80 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=60c6f12d46ed6985 server_cookie=e6795e783a70753b in_seq=0 out_seq=0 2024-09-19T05:15:17.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.615+0000 7fa593e00640 10 client.17697 ms_handle_connect on v2:172.21.15.96:6836/3978388286 2024-09-19T05:15:17.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.616+0000 7fa593e00640 1 -- 172.21.15.43:0/1656217154 <== mds.3 v2:172.21.15.96:6836/3978388286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55768dfb7950 con 0x55768e067860 2024-09-19T05:15:17.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.616+0000 7fa593e00640 10 client.17697 handle_command_reply: tid=0 2024-09-19T05:15:17.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.616+0000 7fa5b7e70740 1 client.17697 shutdown 2024-09-19T05:15:17.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.617+0000 7fa5b7e70740 1 -- 172.21.15.43:0/1656217154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55768e044490 msgr2=0x55768e04bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:17.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.617+0000 7fa5b7e70740 1 --2- 172.21.15.43:0/1656217154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55768e044490 0x55768e04bd60 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa598012980 tx=0x7fa598012e40 comp rx=0 tx=0).stop 2024-09-19T05:15:17.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.617+0000 7fa5b7e70740 1 -- 172.21.15.43:0/1656217154 shutdown_connections 2024-09-19T05:15:17.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.617+0000 7fa5b7e70740 1 --2- 172.21.15.43:0/1656217154 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55768e067860 0x55768e069c80 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:17.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.617+0000 7fa5b7e70740 1 --2- 172.21.15.43:0/1656217154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55768e053260 0x55768e04c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:17.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.617+0000 7fa5b7e70740 1 --2- 172.21.15.43:0/1656217154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55768e044490 0x55768e04bd60 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:17.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.617+0000 7fa593e00640 0 client.17697 ms_handle_reset on v2:172.21.15.96:6836/3978388286 2024-09-19T05:15:17.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.617+0000 7fa5b7e70740 1 --2- 172.21.15.43:0/1656217154 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55768e043b70 0x55768e052ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:17.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.617+0000 7fa5b7e70740 1 -- 172.21.15.43:0/1656217154 >> 172.21.15.43:0/1656217154 conn(0x55768e040200 msgr2=0x55768e045f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:17.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.617+0000 7fa5b7e70740 1 -- 172.21.15.43:0/1656217154 shutdown_connections 2024-09-19T05:15:17.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.617+0000 7fa5b7e70740 1 -- 172.21.15.43:0/1656217154 wait complete. 2024-09-19T05:15:17.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.617+0000 7fa5b7e70740 20 client.17697 trim_cache size 0 max 16384 2024-09-19T05:15:17.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.631+0000 7fa5b7e70740 1 Processor -- start 2024-09-19T05:15:17.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.631+0000 7fa5b7e70740 1 -- start start 2024-09-19T05:15:17.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.631+0000 7fa5b7e70740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55768e043b70 0x55768e06f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:17.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.631+0000 7fa5b7e70740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55768e044490 0x55768e06fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:17.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.631+0000 7fa5b7e70740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55768e053260 0x55768e0456a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:17.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.631+0000 7fa5b7e70740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b01ab610 con 0x55768e053260 2024-09-19T05:15:17.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.631+0000 7fa5b7e70740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b01ad150 con 0x55768e043b70 2024-09-19T05:15:17.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.631+0000 7fa5b7e70740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b01abce0 con 0x55768e044490 2024-09-19T05:15:17.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.631+0000 7fa5ae800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55768e053260 0x55768e0456a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:17.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.631+0000 7fa5ae800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55768e053260 0x55768e0456a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34220/0 (socket says 172.21.15.43:34220) 2024-09-19T05:15:17.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.631+0000 7fa5ae800640 1 -- 172.21.15.43:0/13518411 learned_addr learned my addr 172.21.15.43:0/13518411 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:15:17.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.632+0000 7fa5a7e00640 1 --2- 172.21.15.43:0/13518411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55768e044490 0x55768e06fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:17.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.632+0000 7fa5ae800640 1 --2- 172.21.15.43:0/13518411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55768e043b70 0x55768e06f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:17.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.632+0000 7fa5ae800640 1 -- 172.21.15.43:0/13518411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55768e044490 msgr2=0x55768e06fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:17.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.632+0000 7fa5ae800640 1 --2- 172.21.15.43:0/13518411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55768e044490 0x55768e06fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:17.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.632+0000 7fa5ae800640 1 -- 172.21.15.43:0/13518411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55768e043b70 msgr2=0x55768e06f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:17.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.632+0000 7fa5ae800640 1 --2- 172.21.15.43:0/13518411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55768e043b70 0x55768e06f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:17.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.632+0000 7fa5ae800640 1 -- 172.21.15.43:0/13518411 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55768e069ff0 con 0x55768e053260 2024-09-19T05:15:17.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.632+0000 7fa5ae800640 1 --2- 172.21.15.43:0/13518411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55768e053260 0x55768e0456a0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8000f80 tx=0x7fa5a8009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc05a6125da0da9d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:17.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.632+0000 7fa59ea00640 1 -- 172.21.15.43:0/13518411 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a80306a0 con 0x55768e053260 2024-09-19T05:15:17.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.633+0000 7fa59ea00640 1 -- 172.21.15.43:0/13518411 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5a8036040 con 0x55768e053260 2024-09-19T05:15:17.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.633+0000 7fa59ea00640 1 -- 172.21.15.43:0/13518411 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a8031510 con 0x55768e053260 2024-09-19T05:15:17.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.633+0000 7fa5b7e70740 1 -- 172.21.15.43:0/13518411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55768e053260 msgr2=0x55768e0456a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:17.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.633+0000 7fa5b7e70740 1 --2- 172.21.15.43:0/13518411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55768e053260 0x55768e0456a0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8000f80 tx=0x7fa5a8009360 comp rx=0 tx=0).stop 2024-09-19T05:15:17.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.633+0000 7fa5b7e70740 1 -- 172.21.15.43:0/13518411 shutdown_connections 2024-09-19T05:15:17.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.633+0000 7fa5b7e70740 1 --2- 172.21.15.43:0/13518411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55768e053260 0x55768e0456a0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:17.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.633+0000 7fa5b7e70740 1 --2- 172.21.15.43:0/13518411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55768e044490 0x55768e06fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:17.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.633+0000 7fa5b7e70740 1 --2- 172.21.15.43:0/13518411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55768e043b70 0x55768e06f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:17.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.633+0000 7fa5b7e70740 1 -- 172.21.15.43:0/13518411 >> 172.21.15.43:0/13518411 conn(0x55768e040200 msgr2=0x55768e04a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:17.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.633+0000 7fa5b7e70740 1 -- 172.21.15.43:0/13518411 shutdown_connections 2024-09-19T05:15:17.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.633+0000 7fa5b7e70740 1 -- 172.21.15.43:0/13518411 wait complete. 2024-09-19T05:15:17.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.634+0000 7fa5b7e70740 1 -- start start 2024-09-19T05:15:17.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.634+0000 7fa5b7e70740 1 Processor -- start 2024-09-19T05:15:17.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.634+0000 7fa5b7e70740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55768e043b70 0x55768e05ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:17.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.634+0000 7fa5b7e70740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55768e044490 0x55768e05f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:17.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.634+0000 7fa5b7e70740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55768e053260 0x55768e0d04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:17.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.634+0000 7fa5b7e70740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b01aca80 con 0x55768e044490 2024-09-19T05:15:17.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.634+0000 7fa5b7e70740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b01ac3b0 con 0x55768e053260 2024-09-19T05:15:17.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.634+0000 7fa5b7e70740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b01a9610 con 0x55768e043b70 2024-09-19T05:15:17.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.635+0000 7fa5b7e70740 10 client.0 _ll_register_callbacks cb 0x7fa598009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:15:17.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.635+0000 7fa5a7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55768e044490 0x55768e05f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:17.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.635+0000 7fa5a7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55768e044490 0x55768e05f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34230/0 (socket says 172.21.15.43:34230) 2024-09-19T05:15:17.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.635+0000 7fa5a7e00640 1 -- 172.21.15.43:0/2761811851 learned_addr learned my addr 172.21.15.43:0/2761811851 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:15:17.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.635+0000 7fa5ae800640 1 --2- 172.21.15.43:0/2761811851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55768e043b70 0x55768e05ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:17.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.635+0000 7fa5ae800640 1 --2- 172.21.15.43:0/2761811851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55768e053260 0x55768e0d04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:17.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.635+0000 7fa5a7e00640 1 -- 172.21.15.43:0/2761811851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55768e043b70 msgr2=0x55768e05ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:17.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.635+0000 7fa5a7e00640 1 --2- 172.21.15.43:0/2761811851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55768e043b70 0x55768e05ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:17.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.635+0000 7fa5a7e00640 1 -- 172.21.15.43:0/2761811851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55768e053260 msgr2=0x55768e0d04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:17.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.635+0000 7fa5a7e00640 1 --2- 172.21.15.43:0/2761811851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55768e053260 0x55768e0d04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:17.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.635+0000 7fa5a7e00640 1 -- 172.21.15.43:0/2761811851 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55768e05fc60 con 0x55768e044490 2024-09-19T05:15:17.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.635+0000 7fa5ae800640 1 --2- 172.21.15.43:0/2761811851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55768e053260 0x55768e0d04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:15:17.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.636+0000 7fa5a7e00640 1 --2- 172.21.15.43:0/2761811851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55768e044490 0x55768e05f3b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa598010970 tx=0x7fa59800fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e73265c99ebc34e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:17.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.636+0000 7fa593e00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:15:17.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.636+0000 7fa593e00640 1 -- 172.21.15.43:0/2761811851 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa598028040 con 0x55768e044490 2024-09-19T05:15:17.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.636+0000 7fa5b7e70740 1 -- 172.21.15.43:0/2761811851 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5a8036040 con 0x55768e044490 2024-09-19T05:15:17.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.636+0000 7fa593e00640 1 -- 172.21.15.43:0/2761811851 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7fa598010430 con 0x55768e044490 2024-09-19T05:15:17.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.636+0000 7fa593e00640 1 -- 172.21.15.43:0/2761811851 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa598010650 con 0x55768e044490 2024-09-19T05:15:17.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.636+0000 7fa5b7e70740 10 client.17703 fetch_fsmap learned FSMap version 29 2024-09-19T05:15:17.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.636+0000 7fa5b7e70740 1 -- 172.21.15.43:0/2761811851 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55768e0d1af0 con 0x55768e044490 2024-09-19T05:15:17.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.636+0000 7fa593e00640 1 -- 172.21.15.43:0/2761811851 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5a8036040 con 0x55768e044490 2024-09-19T05:15:17.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.637+0000 7fa593e00640 1 -- 172.21.15.43:0/2761811851 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa598004640 con 0x55768e044490 2024-09-19T05:15:17.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.637+0000 7fa5b7e70740 10 client.17703 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:15:17.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.637+0000 7fa5b7e70740 10 client.17703 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-19T05:15:17.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.637+0000 7fa5b7e70740 20 client.17703 populate_metadata read hostname 'smithi043' 2024-09-19T05:15:17.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.637+0000 7fa5b7e70740 1 --2- 172.21.15.43:0/2761811851 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55768e067860 0x55768e0d4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:17.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.637+0000 7fa5b7e70740 4 client.17703 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:15:17.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.637+0000 7fa5b7e70740 1 -- 172.21.15.43:0/2761811851 --> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55768dfb1e60 con 0x55768e067860 2024-09-19T05:15:17.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.637+0000 7fa5ae800640 1 --2- 172.21.15.43:0/2761811851 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55768e067860 0x55768e0d4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:17.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.638+0000 7fa5ae800640 1 --2- 172.21.15.43:0/2761811851 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55768e067860 0x55768e0d4280 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9712eea8db9bde5c server_cookie=f239a9f886a915ee in_seq=0 out_seq=0 2024-09-19T05:15:17.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.638+0000 7fa593e00640 10 client.17703 ms_handle_connect on v2:172.21.15.96:6836/3978388286 2024-09-19T05:15:17.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.638+0000 7fa593e00640 1 -- 172.21.15.43:0/2761811851 <== mds.3 v2:172.21.15.96:6836/3978388286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55768dfb1e60 con 0x55768e067860 2024-09-19T05:15:17.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.638+0000 7fa593e00640 10 client.17703 handle_command_reply: tid=0 2024-09-19T05:15:17.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.638+0000 7fa5b7e70740 1 client.17703 shutdown 2024-09-19T05:15:17.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.638+0000 7fa5b7e70740 1 -- 172.21.15.43:0/2761811851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55768e044490 msgr2=0x55768e05f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:17.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.638+0000 7fa5b7e70740 1 --2- 172.21.15.43:0/2761811851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55768e044490 0x55768e05f3b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa598010970 tx=0x7fa59800fc30 comp rx=0 tx=0).stop 2024-09-19T05:15:17.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.638+0000 7fa5b7e70740 1 -- 172.21.15.43:0/2761811851 shutdown_connections 2024-09-19T05:15:17.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.638+0000 7fa5b7e70740 1 --2- 172.21.15.43:0/2761811851 >> [v2:172.21.15.96:6836/3978388286,v1:172.21.15.96:6837/3978388286] conn(0x55768e067860 0x55768e0d4280 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:17.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.638+0000 7fa5b7e70740 1 --2- 172.21.15.43:0/2761811851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55768e053260 0x55768e0d04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:17.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.638+0000 7fa5b7e70740 1 --2- 172.21.15.43:0/2761811851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55768e044490 0x55768e05f3b0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:17.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.638+0000 7fa5b7e70740 1 --2- 172.21.15.43:0/2761811851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55768e043b70 0x55768e05ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:17.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.638+0000 7fa593e00640 0 client.17703 ms_handle_reset on v2:172.21.15.96:6836/3978388286 2024-09-19T05:15:17.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.638+0000 7fa5b7e70740 1 -- 172.21.15.43:0/2761811851 >> 172.21.15.43:0/2761811851 conn(0x55768e040200 msgr2=0x55768e04f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:17.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.638+0000 7fa5b7e70740 1 -- 172.21.15.43:0/2761811851 shutdown_connections 2024-09-19T05:15:17.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.638+0000 7fa5b7e70740 1 -- 172.21.15.43:0/2761811851 wait complete. 2024-09-19T05:15:17.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.639+0000 7fa5b7e70740 20 client.17703 trim_cache size 0 max 16384 2024-09-19T05:15:17.644 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:15:17.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.639+0000 7fa5b5200640 1 -- 172.21.15.43:0/2165277779 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fa5940a3430 msgr2=0x7fa5940a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:17.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.639+0000 7fa5b5200640 1 --2- 172.21.15.43:0/2165277779 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fa5940a3430 0x7fa5940a58f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x55768e046860 tx=0x7fa5a00bc040 comp rx=0 tx=0).stop 2024-09-19T05:15:17.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.639+0000 7fa5b5200640 1 -- 172.21.15.43:0/2165277779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5b0101de0 msgr2=0x7fa5b01983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:17.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.639+0000 7fa5b5200640 1 --2- 172.21.15.43:0/2165277779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5b0101de0 0x7fa5b01983b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa5a000e9f0 tx=0x7fa5a000edf0 comp rx=0 tx=0).stop 2024-09-19T05:15:17.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.639+0000 7fa5b5200640 1 -- 172.21.15.43:0/2165277779 shutdown_connections 2024-09-19T05:15:17.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.639+0000 7fa5b5200640 1 --2- 172.21.15.43:0/2165277779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5b0106f10 0x7fa5b019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:17.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.639+0000 7fa5b5200640 1 --2- 172.21.15.43:0/2165277779 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fa5940a3430 0x7fa5940a58f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:17.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.639+0000 7fa5b5200640 1 --2- 172.21.15.43:0/2165277779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5b0102790 0x7fa5b01988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:17.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.639+0000 7fa5b5200640 1 --2- 172.21.15.43:0/2165277779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5b0101de0 0x7fa5b01983b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:17.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.639+0000 7fa5b5200640 1 -- 172.21.15.43:0/2165277779 >> 172.21.15.43:0/2165277779 conn(0x7fa5b00fd8c0 msgr2=0x7fa5b00feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:17.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.639+0000 7fa5b5200640 1 -- 172.21.15.43:0/2165277779 shutdown_connections 2024-09-19T05:15:17.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:17.640+0000 7fa5b5200640 1 -- 172.21.15.43:0/2165277779 wait complete. 2024-09-19T05:15:18.221 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:17 smithi043 ceph-mon[30800]: pgmap v332: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-19T05:15:18.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:17 smithi096 ceph-mon[35085]: pgmap v332: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-19T05:15:18.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:17 smithi151 ceph-mon[35425]: pgmap v332: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-19T05:15:18.516 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T05:15:18.894 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:15:19.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.511+0000 7f577b400640 1 -- 172.21.15.43:0/771392962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5774076720 msgr2=0x7f5774076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:19.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.511+0000 7f577b400640 1 --2- 172.21.15.43:0/771392962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5774076720 0x7f5774076b00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5768009900 tx=0x7f576802a3f0 comp rx=0 tx=0).stop 2024-09-19T05:15:19.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.511+0000 7f577b400640 1 -- 172.21.15.43:0/771392962 shutdown_connections 2024-09-19T05:15:19.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.511+0000 7f577b400640 1 --2- 172.21.15.43:0/771392962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5774112a50 0x7f5774114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:19.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.511+0000 7f577b400640 1 --2- 172.21.15.43:0/771392962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5774077040 0x7f5774112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:19.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.511+0000 7f577b400640 1 --2- 172.21.15.43:0/771392962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5774076720 0x7f5774076b00 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:19.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.511+0000 7f577b400640 1 -- 172.21.15.43:0/771392962 >> 172.21.15.43:0/771392962 conn(0x7f57740ffa50 msgr2=0x7f5774101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:19.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.511+0000 7f577b400640 1 -- 172.21.15.43:0/771392962 shutdown_connections 2024-09-19T05:15:19.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.511+0000 7f577b400640 1 -- 172.21.15.43:0/771392962 wait complete. 2024-09-19T05:15:19.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.512+0000 7f577b400640 1 Processor -- start 2024-09-19T05:15:19.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.512+0000 7f577b400640 1 -- start start 2024-09-19T05:15:19.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.512+0000 7f577b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5774076720 0x7f57741a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:19.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.512+0000 7f577b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5774077040 0x7f57741a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:19.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.512+0000 7f577b400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5774112a50 0x7f577419e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:19.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.512+0000 7f577b400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f57741185e0 con 0x7f5774076720 2024-09-19T05:15:19.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.512+0000 7f577b400640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5774118460 con 0x7f5774112a50 2024-09-19T05:15:19.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.512+0000 7f577b400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5774118760 con 0x7f5774077040 2024-09-19T05:15:19.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.513+0000 7f5778a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5774076720 0x7f57741a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:19.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.513+0000 7f5778a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5774076720 0x7f57741a4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34238/0 (socket says 172.21.15.43:34238) 2024-09-19T05:15:19.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.513+0000 7f5778a00640 1 -- 172.21.15.43:0/1857979464 learned_addr learned my addr 172.21.15.43:0/1857979464 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:15:19.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.513+0000 7f5773e00640 1 --2- 172.21.15.43:0/1857979464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5774077040 0x7f57741a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:19.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.513+0000 7f5779400640 1 --2- 172.21.15.43:0/1857979464 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5774112a50 0x7f577419e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:19.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.513+0000 7f5778a00640 1 -- 172.21.15.43:0/1857979464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5774077040 msgr2=0x7f57741a4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:19.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.513+0000 7f5778a00640 1 --2- 172.21.15.43:0/1857979464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5774077040 0x7f57741a4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:19.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.513+0000 7f5778a00640 1 -- 172.21.15.43:0/1857979464 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5774112a50 msgr2=0x7f577419e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:19.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.513+0000 7f5778a00640 1 --2- 172.21.15.43:0/1857979464 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5774112a50 0x7f577419e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:19.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.513+0000 7f5778a00640 1 -- 172.21.15.43:0/1857979464 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f577419f020 con 0x7f5774076720 2024-09-19T05:15:19.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.513+0000 7f5778a00640 1 --2- 172.21.15.43:0/1857979464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5774076720 0x7f57741a4730 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f5768006010 tx=0x7f5768004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:19.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.514+0000 7f5771600640 1 -- 172.21.15.43:0/1857979464 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576803f030 con 0x7f5774076720 2024-09-19T05:15:19.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.514+0000 7f5771600640 1 -- 172.21.15.43:0/1857979464 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5768004440 con 0x7f5774076720 2024-09-19T05:15:19.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.514+0000 7f577b400640 1 -- 172.21.15.43:0/1857979464 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f577419f2b0 con 0x7f5774076720 2024-09-19T05:15:19.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.514+0000 7f5771600640 1 -- 172.21.15.43:0/1857979464 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5768035070 con 0x7f5774076720 2024-09-19T05:15:19.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.514+0000 7f577b400640 1 -- 172.21.15.43:0/1857979464 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57741ab600 con 0x7f5774076720 2024-09-19T05:15:19.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.517+0000 7f5771600640 1 -- 172.21.15.43:0/1857979464 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f5768042030 con 0x7f5774076720 2024-09-19T05:15:19.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.517+0000 7f577de6e740 1 Processor -- start 2024-09-19T05:15:19.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.517+0000 7f577de6e740 1 -- start start 2024-09-19T05:15:19.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.517+0000 7f577de6e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557adecdc70 0x5557adece050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:19.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.517+0000 7f577de6e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557adece590 0x5557aded8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:19.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.517+0000 7f577de6e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557aded8e50 0x5557adedb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:19.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.517+0000 7f577de6e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5557ade343e0 con 0x5557aded8e50 2024-09-19T05:15:19.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.517+0000 7f577de6e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5557adddbd80 con 0x5557adece590 2024-09-19T05:15:19.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.517+0000 7f577de6e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5557ade303a0 con 0x5557adecdc70 2024-09-19T05:15:19.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.518+0000 7f5773e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557aded8e50 0x5557adedb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:19.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.518+0000 7f5779400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557adecdc70 0x5557adece050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:19.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.518+0000 7f5773e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557aded8e50 0x5557adedb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34240/0 (socket says 172.21.15.43:34240) 2024-09-19T05:15:19.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.518+0000 7f5773e00640 1 -- 172.21.15.43:0/3557599663 learned_addr learned my addr 172.21.15.43:0/3557599663 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:15:19.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.518+0000 7f5778a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557adece590 0x5557aded8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:19.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.518+0000 7f5779400640 1 -- 172.21.15.43:0/3557599663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557adece590 msgr2=0x5557aded8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:19.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.518+0000 7f5779400640 1 --2- 172.21.15.43:0/3557599663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557adece590 0x5557aded8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:19.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.518+0000 7f5779400640 1 -- 172.21.15.43:0/3557599663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557aded8e50 msgr2=0x5557adedb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:19.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.518+0000 7f5779400640 1 --2- 172.21.15.43:0/3557599663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557aded8e50 0x5557adedb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:19.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.518+0000 7f5779400640 1 -- 172.21.15.43:0/3557599663 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557addfaae0 con 0x5557adecdc70 2024-09-19T05:15:19.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.518+0000 7f5773e00640 1 --2- 172.21.15.43:0/3557599663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557aded8e50 0x5557adedb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:15:19.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.519+0000 7f5779400640 1 --2- 172.21.15.43:0/3557599663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557adecdc70 0x5557adece050 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f577407aa10 tx=0x7f576c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12fe6b622e7e8576 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:19.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.519+0000 7f5765c00640 1 -- 172.21.15.43:0/3557599663 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c004d20 con 0x5557adecdc70 2024-09-19T05:15:19.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.519+0000 7f5765c00640 1 -- 172.21.15.43:0/3557599663 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f576c004050 con 0x5557adecdc70 2024-09-19T05:15:19.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.519+0000 7f5765c00640 1 -- 172.21.15.43:0/3557599663 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c005260 con 0x5557adecdc70 2024-09-19T05:15:19.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.519+0000 7f5771600640 1 --2- 172.21.15.43:0/1857979464 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f575c0a33b0 0x7f575c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:19.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.519+0000 7f5771600640 1 -- 172.21.15.43:0/1857979464 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7f57680efff0 con 0x7f5774076720 2024-09-19T05:15:19.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.519+0000 7f5778a00640 1 --2- 172.21.15.43:0/1857979464 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f575c0a33b0 0x7f575c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:19.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.520+0000 7f577de6e740 1 -- 172.21.15.43:0/3557599663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557adecdc70 msgr2=0x5557adece050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:19.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.520+0000 7f577de6e740 1 --2- 172.21.15.43:0/3557599663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557adecdc70 0x5557adece050 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f577407aa10 tx=0x7f576c011810 comp rx=0 tx=0).stop 2024-09-19T05:15:19.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.520+0000 7f577de6e740 1 -- 172.21.15.43:0/3557599663 shutdown_connections 2024-09-19T05:15:19.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.520+0000 7f577de6e740 1 --2- 172.21.15.43:0/3557599663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557aded8e50 0x5557adedb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:19.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.520+0000 7f577de6e740 1 --2- 172.21.15.43:0/3557599663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557adece590 0x5557aded8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:19.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.520+0000 7f577de6e740 1 --2- 172.21.15.43:0/3557599663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557adecdc70 0x5557adece050 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:19.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.520+0000 7f577de6e740 1 -- 172.21.15.43:0/3557599663 >> 172.21.15.43:0/3557599663 conn(0x5557adeca200 msgr2=0x5557adeca5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:19.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.520+0000 7f577de6e740 1 -- 172.21.15.43:0/3557599663 shutdown_connections 2024-09-19T05:15:19.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.520+0000 7f577de6e740 1 -- 172.21.15.43:0/3557599663 wait complete. 2024-09-19T05:15:19.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.520+0000 7f5778a00640 1 --2- 172.21.15.43:0/1857979464 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f575c0a33b0 0x7f575c0a5870 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x5557adecb060 tx=0x7f57680f3aa0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:19.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.520+0000 7f577de6e740 1 -- start start 2024-09-19T05:15:19.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.521+0000 7f577de6e740 1 Processor -- start 2024-09-19T05:15:19.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.521+0000 7f577de6e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557adecdc70 0x5557aded5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:19.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.521+0000 7f577de6e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557adece590 0x5557aded62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:19.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.521+0000 7f577de6e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557aded8e50 0x5557aded6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:19.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.521+0000 7f577de6e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5557adedba20 con 0x5557adecdc70 2024-09-19T05:15:19.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.521+0000 7f577de6e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5557addef0f0 con 0x5557adece590 2024-09-19T05:15:19.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.521+0000 7f577de6e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5557adedbba0 con 0x5557aded8e50 2024-09-19T05:15:19.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.521+0000 7f577de6e740 10 client.0 _ll_register_callbacks cb 0x5557addd6bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:15:19.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.521+0000 7f5779400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557adecdc70 0x5557aded5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:19.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.522+0000 7f5779400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557adecdc70 0x5557aded5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34254/0 (socket says 172.21.15.43:34254) 2024-09-19T05:15:19.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.522+0000 7f5779400640 1 -- 172.21.15.43:0/2456517857 learned_addr learned my addr 172.21.15.43:0/2456517857 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:15:19.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.522+0000 7f5773e00640 1 --2- 172.21.15.43:0/2456517857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557adece590 0x5557aded62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:19.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.522+0000 7f5779400640 1 --2- 172.21.15.43:0/2456517857 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557aded8e50 0x5557aded6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:19.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.522+0000 7f5779400640 1 -- 172.21.15.43:0/2456517857 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557aded8e50 msgr2=0x5557aded6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:19.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.522+0000 7f5779400640 1 --2- 172.21.15.43:0/2456517857 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557aded8e50 0x5557aded6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:19.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.522+0000 7f5779400640 1 -- 172.21.15.43:0/2456517857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557adece590 msgr2=0x5557aded62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:19.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.522+0000 7f5779400640 1 --2- 172.21.15.43:0/2456517857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557adece590 0x5557aded62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:19.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.522+0000 7f5779400640 1 -- 172.21.15.43:0/2456517857 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557adba53c0 con 0x5557adecdc70 2024-09-19T05:15:19.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.522+0000 7f5779400640 1 --2- 172.21.15.43:0/2456517857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557adecdc70 0x5557aded5da0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f576c003eb0 tx=0x7f576c009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d366f050c8db0c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:19.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.522+0000 7f575a000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:15:19.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.523+0000 7f575a000640 1 -- 172.21.15.43:0/2456517857 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c01d040 con 0x5557adecdc70 2024-09-19T05:15:19.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.523+0000 7f575a000640 1 -- 172.21.15.43:0/2456517857 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7f576c004a40 con 0x5557adecdc70 2024-09-19T05:15:19.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.523+0000 7f577de6e740 1 -- 172.21.15.43:0/2456517857 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f576c004050 con 0x5557adecdc70 2024-09-19T05:15:19.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.523+0000 7f575a000640 1 -- 172.21.15.43:0/2456517857 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f576c034050 con 0x5557adecdc70 2024-09-19T05:15:19.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.523+0000 7f577de6e740 10 client.17715 fetch_fsmap learned FSMap version 29 2024-09-19T05:15:19.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.523+0000 7f577de6e740 1 -- 172.21.15.43:0/2456517857 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5557adee7af0 con 0x5557adecdc70 2024-09-19T05:15:19.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.523+0000 7f575a000640 1 -- 172.21.15.43:0/2456517857 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f576c004050 con 0x5557adecdc70 2024-09-19T05:15:19.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.523+0000 7f575a000640 1 -- 172.21.15.43:0/2456517857 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f576c005830 con 0x5557adecdc70 2024-09-19T05:15:19.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.524+0000 7f577de6e740 10 client.17715 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:15:19.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.524+0000 7f577de6e740 10 client.17715 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T05:15:19.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.524+0000 7f577de6e740 20 client.17715 populate_metadata read hostname 'smithi043' 2024-09-19T05:15:19.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.524+0000 7f577de6e740 1 --2- 172.21.15.43:0/2456517857 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x5557adee8f50 0x5557adeeb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:19.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.524+0000 7f577de6e740 4 client.17715 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:15:19.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.524+0000 7f577de6e740 1 -- 172.21.15.43:0/2456517857 --> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557ade41950 con 0x5557adee8f50 2024-09-19T05:15:19.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.524+0000 7f5773e00640 1 --2- 172.21.15.43:0/2456517857 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x5557adee8f50 0x5557adeeb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:19.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.525+0000 7f5773e00640 1 --2- 172.21.15.43:0/2456517857 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x5557adee8f50 0x5557adeeb340 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=57681e4c0b9b0639 server_cookie=e690a44429e2064b in_seq=0 out_seq=0 2024-09-19T05:15:19.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.525+0000 7f575a000640 10 client.17715 ms_handle_connect on v2:172.21.15.151:6838/3794104313 2024-09-19T05:15:19.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.525+0000 7f575a000640 1 -- 172.21.15.43:0/2456517857 <== mds.2 v2:172.21.15.151:6838/3794104313 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5557ade41950 con 0x5557adee8f50 2024-09-19T05:15:19.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.525+0000 7f575a000640 10 client.17715 handle_command_reply: tid=0 2024-09-19T05:15:19.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.525+0000 7f577de6e740 1 client.17715 shutdown 2024-09-19T05:15:19.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.526+0000 7f577de6e740 1 -- 172.21.15.43:0/2456517857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557adecdc70 msgr2=0x5557aded5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:19.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.526+0000 7f577de6e740 1 --2- 172.21.15.43:0/2456517857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557adecdc70 0x5557aded5da0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f576c003eb0 tx=0x7f576c009c90 comp rx=0 tx=0).stop 2024-09-19T05:15:19.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.526+0000 7f577de6e740 1 -- 172.21.15.43:0/2456517857 shutdown_connections 2024-09-19T05:15:19.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.526+0000 7f577de6e740 1 --2- 172.21.15.43:0/2456517857 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x5557adee8f50 0x5557adeeb340 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:19.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.526+0000 7f577de6e740 1 --2- 172.21.15.43:0/2456517857 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557aded8e50 0x5557aded6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:19.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.526+0000 7f575a000640 0 client.17715 ms_handle_reset on v2:172.21.15.151:6838/3794104313 2024-09-19T05:15:19.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.526+0000 7f577de6e740 1 --2- 172.21.15.43:0/2456517857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557adece590 0x5557aded62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:19.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.526+0000 7f577de6e740 1 --2- 172.21.15.43:0/2456517857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557adecdc70 0x5557aded5da0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:19.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.526+0000 7f577de6e740 1 -- 172.21.15.43:0/2456517857 >> 172.21.15.43:0/2456517857 conn(0x5557adeca200 msgr2=0x5557adecfee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:19.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.526+0000 7f577de6e740 1 -- 172.21.15.43:0/2456517857 shutdown_connections 2024-09-19T05:15:19.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.526+0000 7f577de6e740 1 -- 172.21.15.43:0/2456517857 wait complete. 2024-09-19T05:15:19.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.526+0000 7f577de6e740 20 client.17715 trim_cache size 0 max 16384 2024-09-19T05:15:19.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.540+0000 7f577de6e740 1 Processor -- start 2024-09-19T05:15:19.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.540+0000 7f577de6e740 1 -- start start 2024-09-19T05:15:19.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.540+0000 7f577de6e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557adecdc70 0x5557adecfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:19.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.541+0000 7f577de6e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557adece590 0x5557adef0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:19.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.541+0000 7f577de6e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557aded8e50 0x5557adef1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:19.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.541+0000 7f577de6e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f57741b3f40 con 0x5557adece590 2024-09-19T05:15:19.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.541+0000 7f577de6e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f57741b5a80 con 0x5557aded8e50 2024-09-19T05:15:19.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.541+0000 7f577de6e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f57741b4610 con 0x5557adecdc70 2024-09-19T05:15:19.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.541+0000 7f5773e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557adece590 0x5557adef0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:19.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.541+0000 7f5779400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557adecdc70 0x5557adecfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:19.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.541+0000 7f5779400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557adecdc70 0x5557adecfdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:51464/0 (socket says 172.21.15.43:51464) 2024-09-19T05:15:19.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.541+0000 7f5779400640 1 -- 172.21.15.43:0/510807012 learned_addr learned my addr 172.21.15.43:0/510807012 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:15:19.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.541+0000 7f5779400640 1 --2- 172.21.15.43:0/510807012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557aded8e50 0x5557adef1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:19.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.541+0000 7f5779400640 1 -- 172.21.15.43:0/510807012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557aded8e50 msgr2=0x5557adef1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:19.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.541+0000 7f5779400640 1 --2- 172.21.15.43:0/510807012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557aded8e50 0x5557adef1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:19.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.541+0000 7f5779400640 1 -- 172.21.15.43:0/510807012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557adece590 msgr2=0x5557adef0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:19.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.541+0000 7f5779400640 1 --2- 172.21.15.43:0/510807012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557adece590 0x5557adef0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:19.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.541+0000 7f5779400640 1 -- 172.21.15.43:0/510807012 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557adecdaa0 con 0x5557adecdc70 2024-09-19T05:15:19.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.542+0000 7f5779400640 1 --2- 172.21.15.43:0/510807012 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557adecdc70 0x5557adecfdd0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f576c005d70 tx=0x7f576c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cf39ddea83635d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:19.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.542+0000 7f5765200640 1 -- 172.21.15.43:0/510807012 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c0277f0 con 0x5557adecdc70 2024-09-19T05:15:19.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.542+0000 7f5765200640 1 -- 172.21.15.43:0/510807012 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f576c003ee0 con 0x5557adecdc70 2024-09-19T05:15:19.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.542+0000 7f5765200640 1 -- 172.21.15.43:0/510807012 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c005030 con 0x5557adecdc70 2024-09-19T05:15:19.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.543+0000 7f577de6e740 1 -- 172.21.15.43:0/510807012 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557adecdc70 msgr2=0x5557adecfdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:19.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.543+0000 7f577de6e740 1 --2- 172.21.15.43:0/510807012 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557adecdc70 0x5557adecfdd0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f576c005d70 tx=0x7f576c011810 comp rx=0 tx=0).stop 2024-09-19T05:15:19.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.543+0000 7f577de6e740 1 -- 172.21.15.43:0/510807012 shutdown_connections 2024-09-19T05:15:19.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.543+0000 7f577de6e740 1 --2- 172.21.15.43:0/510807012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557aded8e50 0x5557adef1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:19.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.543+0000 7f577de6e740 1 --2- 172.21.15.43:0/510807012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557adece590 0x5557adef0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:19.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.543+0000 7f577de6e740 1 --2- 172.21.15.43:0/510807012 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557adecdc70 0x5557adecfdd0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:19.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.543+0000 7f577de6e740 1 -- 172.21.15.43:0/510807012 >> 172.21.15.43:0/510807012 conn(0x5557adeca200 msgr2=0x5557adecf6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:19.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.543+0000 7f577de6e740 1 -- 172.21.15.43:0/510807012 shutdown_connections 2024-09-19T05:15:19.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.543+0000 7f577de6e740 1 -- 172.21.15.43:0/510807012 wait complete. 2024-09-19T05:15:19.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.543+0000 7f577de6e740 1 -- start start 2024-09-19T05:15:19.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.543+0000 7f577de6e740 1 Processor -- start 2024-09-19T05:15:19.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.544+0000 7f577de6e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557adecdc70 0x5557adee0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:19.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.544+0000 7f577de6e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557adece590 0x5557adee1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:19.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.544+0000 7f577de6e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557aded8e50 0x5557adedddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:19.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.544+0000 7f577de6e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f57741b53b0 con 0x5557adece590 2024-09-19T05:15:19.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.544+0000 7f577de6e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f57741b4ce0 con 0x5557adecdc70 2024-09-19T05:15:19.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.544+0000 7f577de6e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f57741b1f40 con 0x5557aded8e50 2024-09-19T05:15:19.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.544+0000 7f577de6e740 10 client.0 _ll_register_callbacks cb 0x7f5760009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:15:19.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.544+0000 7f5779400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557adecdc70 0x5557adee0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:19.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.544+0000 7f5779400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557adecdc70 0x5557adee0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:50208/0 (socket says 172.21.15.43:50208) 2024-09-19T05:15:19.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.544+0000 7f5779400640 1 -- 172.21.15.43:0/2370211194 learned_addr learned my addr 172.21.15.43:0/2370211194 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:15:19.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.544+0000 7f5773e00640 1 --2- 172.21.15.43:0/2370211194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557adece590 0x5557adee1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:19.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.544+0000 7f5779400640 1 --2- 172.21.15.43:0/2370211194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557aded8e50 0x5557adedddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:19.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.545+0000 7f5773e00640 1 -- 172.21.15.43:0/2370211194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557aded8e50 msgr2=0x5557adedddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:19.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.545+0000 7f5773e00640 1 --2- 172.21.15.43:0/2370211194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557aded8e50 0x5557adedddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:19.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.545+0000 7f5773e00640 1 -- 172.21.15.43:0/2370211194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557adecdc70 msgr2=0x5557adee0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:19.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.545+0000 7f5773e00640 1 --2- 172.21.15.43:0/2370211194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557adecdc70 0x5557adee0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:19.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.545+0000 7f5773e00640 1 -- 172.21.15.43:0/2370211194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557adec5770 con 0x5557adece590 2024-09-19T05:15:19.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.545+0000 7f5779400640 1 --2- 172.21.15.43:0/2370211194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557adecdc70 0x5557adee0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:15:19.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.545+0000 7f5773e00640 1 --2- 172.21.15.43:0/2370211194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557adece590 0x5557adee1360 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x5557adeeba60 tx=0x7f5760004260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13a44af0c013129 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:19.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.545+0000 7f575a000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:15:19.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.545+0000 7f575a000640 1 -- 172.21.15.43:0/2370211194 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5760005020 con 0x5557adece590 2024-09-19T05:15:19.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.545+0000 7f575a000640 1 -- 172.21.15.43:0/2370211194 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7f57600051c0 con 0x5557adece590 2024-09-19T05:15:19.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.545+0000 7f577de6e740 1 -- 172.21.15.43:0/2370211194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f576c003ee0 con 0x5557adece590 2024-09-19T05:15:19.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.546+0000 7f575a000640 1 -- 172.21.15.43:0/2370211194 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f57600053e0 con 0x5557adece590 2024-09-19T05:15:19.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.546+0000 7f577de6e740 10 client.17721 fetch_fsmap learned FSMap version 29 2024-09-19T05:15:19.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.546+0000 7f577de6e740 1 -- 172.21.15.43:0/2370211194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5557adf541b0 con 0x5557adece590 2024-09-19T05:15:19.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.546+0000 7f575a000640 1 -- 172.21.15.43:0/2370211194 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f576c003ee0 con 0x5557adece590 2024-09-19T05:15:19.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.546+0000 7f575a000640 1 -- 172.21.15.43:0/2370211194 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f57600204a0 con 0x5557adece590 2024-09-19T05:15:19.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.546+0000 7f577de6e740 10 client.17721 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:15:19.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.546+0000 7f577de6e740 10 client.17721 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-19T05:15:19.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.546+0000 7f577de6e740 20 client.17721 populate_metadata read hostname 'smithi043' 2024-09-19T05:15:19.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.546+0000 7f577de6e740 1 --2- 172.21.15.43:0/2370211194 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x5557adf54950 0x5557adf56d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:19.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.546+0000 7f577de6e740 4 client.17721 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:15:19.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.546+0000 7f577de6e740 1 -- 172.21.15.43:0/2370211194 --> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5557ade3be60 con 0x5557adf54950 2024-09-19T05:15:19.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.547+0000 7f5779400640 1 --2- 172.21.15.43:0/2370211194 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x5557adf54950 0x5557adf56d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:19.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.547+0000 7f5779400640 1 --2- 172.21.15.43:0/2370211194 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x5557adf54950 0x5557adf56d70 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3a544dad801efbc8 server_cookie=8dbe79a571513747 in_seq=0 out_seq=0 2024-09-19T05:15:19.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.547+0000 7f575a000640 10 client.17721 ms_handle_connect on v2:172.21.15.151:6838/3794104313 2024-09-19T05:15:19.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.548+0000 7f575a000640 1 -- 172.21.15.43:0/2370211194 <== mds.2 v2:172.21.15.151:6838/3794104313 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5557ade3be60 con 0x5557adf54950 2024-09-19T05:15:19.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.548+0000 7f575a000640 10 client.17721 handle_command_reply: tid=0 2024-09-19T05:15:19.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.548+0000 7f577de6e740 1 client.17721 shutdown 2024-09-19T05:15:19.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.548+0000 7f577de6e740 1 -- 172.21.15.43:0/2370211194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557adece590 msgr2=0x5557adee1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:19.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.548+0000 7f577de6e740 1 --2- 172.21.15.43:0/2370211194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557adece590 0x5557adee1360 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x5557adeeba60 tx=0x7f5760004260 comp rx=0 tx=0).stop 2024-09-19T05:15:19.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.548+0000 7f577de6e740 1 -- 172.21.15.43:0/2370211194 shutdown_connections 2024-09-19T05:15:19.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.548+0000 7f577de6e740 1 --2- 172.21.15.43:0/2370211194 >> [v2:172.21.15.151:6838/3794104313,v1:172.21.15.151:6839/3794104313] conn(0x5557adf54950 0x5557adf56d70 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:19.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.548+0000 7f577de6e740 1 --2- 172.21.15.43:0/2370211194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557aded8e50 0x5557adedddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:19.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.548+0000 7f577de6e740 1 --2- 172.21.15.43:0/2370211194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557adece590 0x5557adee1360 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:19.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.548+0000 7f577de6e740 1 --2- 172.21.15.43:0/2370211194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557adecdc70 0x5557adee0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:19.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.548+0000 7f577de6e740 1 -- 172.21.15.43:0/2370211194 >> 172.21.15.43:0/2370211194 conn(0x5557adeca200 msgr2=0x5557aded6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:19.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.548+0000 7f575a000640 0 client.17721 ms_handle_reset on v2:172.21.15.151:6838/3794104313 2024-09-19T05:15:19.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.548+0000 7f577de6e740 1 -- 172.21.15.43:0/2370211194 shutdown_connections 2024-09-19T05:15:19.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.548+0000 7f577de6e740 1 -- 172.21.15.43:0/2370211194 wait complete. 2024-09-19T05:15:19.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.548+0000 7f577de6e740 20 client.17721 trim_cache size 0 max 16384 2024-09-19T05:15:19.567 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:15:19.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.549+0000 7f577b400640 1 -- 172.21.15.43:0/1857979464 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f575c0a33b0 msgr2=0x7f575c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:19.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.549+0000 7f577b400640 1 --2- 172.21.15.43:0/1857979464 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f575c0a33b0 0x7f575c0a5870 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x5557adecb060 tx=0x7f57680f3aa0 comp rx=0 tx=0).stop 2024-09-19T05:15:19.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.549+0000 7f577b400640 1 -- 172.21.15.43:0/1857979464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5774076720 msgr2=0x7f57741a4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:19.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.549+0000 7f577b400640 1 --2- 172.21.15.43:0/1857979464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5774076720 0x7f57741a4730 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f5768006010 tx=0x7f5768004040 comp rx=0 tx=0).stop 2024-09-19T05:15:19.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.549+0000 7f577b400640 1 -- 172.21.15.43:0/1857979464 shutdown_connections 2024-09-19T05:15:19.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.549+0000 7f577b400640 1 --2- 172.21.15.43:0/1857979464 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f575c0a33b0 0x7f575c0a5870 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:19.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.549+0000 7f577b400640 1 --2- 172.21.15.43:0/1857979464 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5774112a50 0x7f577419e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:19.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.549+0000 7f577b400640 1 --2- 172.21.15.43:0/1857979464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5774077040 0x7f57741a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:19.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.549+0000 7f577b400640 1 --2- 172.21.15.43:0/1857979464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5774076720 0x7f57741a4730 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:19.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.549+0000 7f577b400640 1 -- 172.21.15.43:0/1857979464 >> 172.21.15.43:0/1857979464 conn(0x7f57740ffa50 msgr2=0x7f5774101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:19.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.550+0000 7f577b400640 1 -- 172.21.15.43:0/1857979464 shutdown_connections 2024-09-19T05:15:19.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:19.550+0000 7f577b400640 1 -- 172.21.15.43:0/1857979464 wait complete. 2024-09-19T05:15:20.219 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:19 smithi043 ceph-mon[30800]: pgmap v333: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 214 B/s wr, 10 op/s 2024-09-19T05:15:20.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:19 smithi096 ceph-mon[35085]: pgmap v333: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 214 B/s wr, 10 op/s 2024-09-19T05:15:20.257 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T05:15:20.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:19 smithi151 ceph-mon[35425]: pgmap v333: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 214 B/s wr, 10 op/s 2024-09-19T05:15:20.638 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:15:21.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.226+0000 7f7fb9200640 1 -- 172.21.15.43:0/1670899875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fb4076720 msgr2=0x7f7fb4076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:21.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.226+0000 7f7fb9200640 1 --2- 172.21.15.43:0/1670899875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fb4076720 0x7f7fb4076b00 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4009900 tx=0x7f7fa402a3f0 comp rx=0 tx=0).stop 2024-09-19T05:15:21.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.226+0000 7f7fb9200640 1 -- 172.21.15.43:0/1670899875 shutdown_connections 2024-09-19T05:15:21.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.226+0000 7f7fb9200640 1 --2- 172.21.15.43:0/1670899875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fb4112a30 0x7f7fb4114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:21.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.226+0000 7f7fb9200640 1 --2- 172.21.15.43:0/1670899875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fb4077040 0x7f7fb41124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:21.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.226+0000 7f7fb9200640 1 --2- 172.21.15.43:0/1670899875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fb4076720 0x7f7fb4076b00 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:21.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.226+0000 7f7fb9200640 1 -- 172.21.15.43:0/1670899875 >> 172.21.15.43:0/1670899875 conn(0x7f7fb40ffa90 msgr2=0x7f7fb4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:21.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.226+0000 7f7fb9200640 1 -- 172.21.15.43:0/1670899875 shutdown_connections 2024-09-19T05:15:21.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.226+0000 7f7fb9200640 1 -- 172.21.15.43:0/1670899875 wait complete. 2024-09-19T05:15:21.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.227+0000 7f7fb9200640 1 Processor -- start 2024-09-19T05:15:21.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.228+0000 7f7fb9200640 1 -- start start 2024-09-19T05:15:21.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.228+0000 7f7fb9200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fb4076720 0x7f7fb41a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:21.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.228+0000 7f7fb9200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fb4077040 0x7f7fb41a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:21.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.228+0000 7f7fb9200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fb4112a30 0x7f7fb419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:21.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.228+0000 7f7fb9200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb41185f0 con 0x7f7fb4077040 2024-09-19T05:15:21.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.228+0000 7f7fb9200640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb4118470 con 0x7f7fb4112a30 2024-09-19T05:15:21.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.228+0000 7f7fb9200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb4118770 con 0x7f7fb4076720 2024-09-19T05:15:21.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.228+0000 7f7fb1400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fb4077040 0x7f7fb41a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:21.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.228+0000 7f7fb1e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fb4076720 0x7f7fb41a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:21.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.228+0000 7f7fb1400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fb4077040 0x7f7fb41a4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34288/0 (socket says 172.21.15.43:34288) 2024-09-19T05:15:21.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.228+0000 7f7fb1e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fb4076720 0x7f7fb41a46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:51482/0 (socket says 172.21.15.43:51482) 2024-09-19T05:15:21.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.228+0000 7f7fb1400640 1 -- 172.21.15.43:0/4066915488 learned_addr learned my addr 172.21.15.43:0/4066915488 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:15:21.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.228+0000 7f7fb2800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fb4112a30 0x7f7fb419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:21.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.229+0000 7f7fb1e00640 1 -- 172.21.15.43:0/4066915488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fb4112a30 msgr2=0x7f7fb419e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:21.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.229+0000 7f7fb1e00640 1 --2- 172.21.15.43:0/4066915488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fb4112a30 0x7f7fb419e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:21.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.229+0000 7f7fb1e00640 1 -- 172.21.15.43:0/4066915488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fb4077040 msgr2=0x7f7fb41a4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:21.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.229+0000 7f7fb1e00640 1 --2- 172.21.15.43:0/4066915488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fb4077040 0x7f7fb41a4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:21.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.229+0000 7f7fb1e00640 1 -- 172.21.15.43:0/4066915488 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fb419ef90 con 0x7f7fb4076720 2024-09-19T05:15:21.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.229+0000 7f7fb1400640 1 --2- 172.21.15.43:0/4066915488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fb4077040 0x7f7fb41a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:15:21.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.229+0000 7f7fb1e00640 1 --2- 172.21.15.43:0/4066915488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fb4076720 0x7f7fb41a46a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7fa402a910 tx=0x7f7fa4003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:21.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.230+0000 7f7faaa00640 1 -- 172.21.15.43:0/4066915488 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa403f070 con 0x7f7fb4076720 2024-09-19T05:15:21.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.230+0000 7f7faaa00640 1 -- 172.21.15.43:0/4066915488 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fa4031c90 con 0x7f7fb4076720 2024-09-19T05:15:21.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.230+0000 7f7fb9200640 1 -- 172.21.15.43:0/4066915488 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fb419f220 con 0x7f7fb4076720 2024-09-19T05:15:21.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.230+0000 7f7faaa00640 1 -- 172.21.15.43:0/4066915488 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4035030 con 0x7f7fb4076720 2024-09-19T05:15:21.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.230+0000 7f7fb9200640 1 -- 172.21.15.43:0/4066915488 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fb41ab570 con 0x7f7fb4076720 2024-09-19T05:15:21.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.233+0000 7f7fbbe0a740 1 Processor -- start 2024-09-19T05:15:21.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.233+0000 7f7faaa00640 1 -- 172.21.15.43:0/4066915488 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f7fa4042070 con 0x7f7fb4076720 2024-09-19T05:15:21.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.233+0000 7f7fbbe0a740 1 -- start start 2024-09-19T05:15:21.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.233+0000 7f7fbbe0a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d5b8c70 0x55b02d5b9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:21.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.233+0000 7f7fbbe0a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b02d5b9590 0x55b02d5c3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:21.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.233+0000 7f7fbbe0a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b02d5c3e50 0x55b02d5c6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:21.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.234+0000 7f7fbbe0a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b02d51f3e0 con 0x55b02d5b9590 2024-09-19T05:15:21.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.234+0000 7f7fbbe0a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b02d4c6d80 con 0x55b02d5b8c70 2024-09-19T05:15:21.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.234+0000 7f7fbbe0a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b02d51b3a0 con 0x55b02d5c3e50 2024-09-19T05:15:21.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.234+0000 7f7fb2800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d5b8c70 0x55b02d5b9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:21.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.234+0000 7f7fb2800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d5b8c70 0x55b02d5b9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:50230/0 (socket says 172.21.15.43:50230) 2024-09-19T05:15:21.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.234+0000 7f7fb2800640 1 -- 172.21.15.43:0/670784380 learned_addr learned my addr 172.21.15.43:0/670784380 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:15:21.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.234+0000 7f7fb1400640 1 --2- 172.21.15.43:0/670784380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b02d5c3e50 0x55b02d5c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:21.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.234+0000 7f7fb1e00640 1 --2- 172.21.15.43:0/670784380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b02d5b9590 0x55b02d5c3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:21.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.234+0000 7f7fb1400640 1 -- 172.21.15.43:0/670784380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d5b8c70 msgr2=0x55b02d5b9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:21.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.234+0000 7f7fb1400640 1 --2- 172.21.15.43:0/670784380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d5b8c70 0x55b02d5b9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:21.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.234+0000 7f7fb1400640 1 -- 172.21.15.43:0/670784380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b02d5b9590 msgr2=0x55b02d5c3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:21.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.234+0000 7f7fb1400640 1 --2- 172.21.15.43:0/670784380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b02d5b9590 0x55b02d5c3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:21.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.234+0000 7f7fb1400640 1 -- 172.21.15.43:0/670784380 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b02d4e5ae0 con 0x55b02d5c3e50 2024-09-19T05:15:21.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.235+0000 7f7fb1400640 1 --2- 172.21.15.43:0/670784380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b02d5c3e50 0x55b02d5c6240 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7fb41a0050 tx=0x7f7fa00109e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cad9273e41dd727b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:21.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.235+0000 7f7faaa00640 1 --2- 172.21.15.43:0/4066915488 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f7f980a3430 0x7f7f980a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:21.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.235+0000 7f7f9f400640 1 -- 172.21.15.43:0/670784380 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa0011820 con 0x55b02d5c3e50 2024-09-19T05:15:21.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.235+0000 7f7f9f400640 1 -- 172.21.15.43:0/670784380 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fa000e3b0 con 0x55b02d5c3e50 2024-09-19T05:15:21.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.235+0000 7f7f9f400640 1 -- 172.21.15.43:0/670784380 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa000d5a0 con 0x55b02d5c3e50 2024-09-19T05:15:21.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.235+0000 7f7faaa00640 1 -- 172.21.15.43:0/4066915488 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7f7fa40f03f0 con 0x7f7fb4076720 2024-09-19T05:15:21.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.235+0000 7f7fb2800640 1 --2- 172.21.15.43:0/4066915488 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f7f980a3430 0x7f7f980a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:21.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.236+0000 7f7fbbe0a740 1 -- 172.21.15.43:0/670784380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b02d5c3e50 msgr2=0x55b02d5c6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:21.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.236+0000 7f7fbbe0a740 1 --2- 172.21.15.43:0/670784380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b02d5c3e50 0x55b02d5c6240 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7fb41a0050 tx=0x7f7fa00109e0 comp rx=0 tx=0).stop 2024-09-19T05:15:21.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.236+0000 7f7fbbe0a740 1 -- 172.21.15.43:0/670784380 shutdown_connections 2024-09-19T05:15:21.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.236+0000 7f7fbbe0a740 1 --2- 172.21.15.43:0/670784380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b02d5c3e50 0x55b02d5c6240 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:21.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.236+0000 7f7fbbe0a740 1 --2- 172.21.15.43:0/670784380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b02d5b9590 0x55b02d5c3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:21.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.236+0000 7f7fbbe0a740 1 --2- 172.21.15.43:0/670784380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d5b8c70 0x55b02d5b9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:21.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.236+0000 7f7fbbe0a740 1 -- 172.21.15.43:0/670784380 >> 172.21.15.43:0/670784380 conn(0x55b02d5b5200 msgr2=0x55b02d5b55e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:21.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.236+0000 7f7fbbe0a740 1 -- 172.21.15.43:0/670784380 shutdown_connections 2024-09-19T05:15:21.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.236+0000 7f7fb2800640 1 --2- 172.21.15.43:0/4066915488 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f7f980a3430 0x7f7f980a58f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7fac0049d0 tx=0x7f7fac009600 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:21.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.236+0000 7f7fbbe0a740 1 -- 172.21.15.43:0/670784380 wait complete. 2024-09-19T05:15:21.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.237+0000 7f7fbbe0a740 1 -- start start 2024-09-19T05:15:21.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.237+0000 7f7fbbe0a740 1 Processor -- start 2024-09-19T05:15:21.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.237+0000 7f7fbbe0a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b02d5b8c70 0x55b02d5c0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:21.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.237+0000 7f7fbbe0a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b02d5b9590 0x55b02d5c1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:21.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.237+0000 7f7fbbe0a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d5c3e50 0x55b02d5c17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:21.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.237+0000 7f7fbbe0a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b02d5c6a20 con 0x55b02d5b8c70 2024-09-19T05:15:21.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.237+0000 7f7fbbe0a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b02d4da0f0 con 0x55b02d5c3e50 2024-09-19T05:15:21.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.237+0000 7f7fbbe0a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b02d5c6ba0 con 0x55b02d5b9590 2024-09-19T05:15:21.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.237+0000 7f7fbbe0a740 10 client.0 _ll_register_callbacks cb 0x55b02d4c1bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:15:21.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.238+0000 7f7fb1400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b02d5b9590 0x55b02d5c1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:21.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.238+0000 7f7fb1400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b02d5b9590 0x55b02d5c1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:51512/0 (socket says 172.21.15.43:51512) 2024-09-19T05:15:21.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.238+0000 7f7fb1400640 1 -- 172.21.15.43:0/2709344958 learned_addr learned my addr 172.21.15.43:0/2709344958 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:15:21.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.238+0000 7f7fb1e00640 1 --2- 172.21.15.43:0/2709344958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b02d5b8c70 0x55b02d5c0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:21.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.238+0000 7f7fb2800640 1 --2- 172.21.15.43:0/2709344958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d5c3e50 0x55b02d5c17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:21.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.238+0000 7f7fb1400640 1 -- 172.21.15.43:0/2709344958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d5c3e50 msgr2=0x55b02d5c17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:21.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.238+0000 7f7fb1400640 1 --2- 172.21.15.43:0/2709344958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d5c3e50 0x55b02d5c17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:21.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.238+0000 7f7fb1400640 1 -- 172.21.15.43:0/2709344958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b02d5b8c70 msgr2=0x55b02d5c0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:21.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.238+0000 7f7fb1400640 1 --2- 172.21.15.43:0/2709344958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b02d5b8c70 0x55b02d5c0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:21.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.238+0000 7f7fb1400640 1 -- 172.21.15.43:0/2709344958 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b02d2903c0 con 0x55b02d5b9590 2024-09-19T05:15:21.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.238+0000 7f7fb1400640 1 --2- 172.21.15.43:0/2709344958 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b02d5b9590 0x55b02d5c1280 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7fa000e7c0 tx=0x7f7fa000efd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3173aab27bc9651f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:21.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.238+0000 7f7f93e00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T05:15:21.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.239+0000 7f7f93e00640 1 -- 172.21.15.43:0/2709344958 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa000bd20 con 0x55b02d5b9590 2024-09-19T05:15:21.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.239+0000 7f7f93e00640 1 -- 172.21.15.43:0/2709344958 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7f7fa0004030 con 0x55b02d5b9590 2024-09-19T05:15:21.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.239+0000 7f7fbbe0a740 1 -- 172.21.15.43:0/2709344958 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fa000e3b0 con 0x55b02d5b9590 2024-09-19T05:15:21.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.239+0000 7f7f93e00640 1 -- 172.21.15.43:0/2709344958 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7fa0004230 con 0x55b02d5b9590 2024-09-19T05:15:21.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.239+0000 7f7fbbe0a740 10 client.27655 fetch_fsmap learned FSMap version 29 2024-09-19T05:15:21.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.239+0000 7f7fbbe0a740 1 -- 172.21.15.43:0/2709344958 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b02d5d2a60 con 0x55b02d5b9590 2024-09-19T05:15:21.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.239+0000 7f7f93e00640 1 -- 172.21.15.43:0/2709344958 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fa000e3b0 con 0x55b02d5b9590 2024-09-19T05:15:21.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.240+0000 7f7f93e00640 1 -- 172.21.15.43:0/2709344958 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7fa0004920 con 0x55b02d5b9590 2024-09-19T05:15:21.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.240+0000 7f7fbbe0a740 10 client.27655 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:15:21.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.240+0000 7f7fbbe0a740 10 client.27655 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-19T05:15:21.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.240+0000 7f7fbbe0a740 20 client.27655 populate_metadata read hostname 'smithi043' 2024-09-19T05:15:21.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.240+0000 7f7fbbe0a740 1 --2- 172.21.15.43:0/2709344958 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55b02d5d3f20 0x55b02d5d6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:21.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.240+0000 7f7fbbe0a740 4 client.27655 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:15:21.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.240+0000 7f7fbbe0a740 1 -- 172.21.15.43:0/2709344958 --> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b02d52c950 con 0x55b02d5d3f20 2024-09-19T05:15:21.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.240+0000 7f7fb1e00640 1 --2- 172.21.15.43:0/2709344958 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55b02d5d3f20 0x55b02d5d6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:21.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.240+0000 7f7fb1e00640 1 --2- 172.21.15.43:0/2709344958 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55b02d5d3f20 0x55b02d5d6310 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e65f5aab62f974de server_cookie=d5be1af810c41f6c in_seq=0 out_seq=0 2024-09-19T05:15:21.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.240+0000 7f7f93e00640 10 client.27655 ms_handle_connect on v2:172.21.15.43:6838/3404555030 2024-09-19T05:15:21.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.241+0000 7f7f93e00640 1 -- 172.21.15.43:0/2709344958 <== mds.1 v2:172.21.15.43:6838/3404555030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b02d52c950 con 0x55b02d5d3f20 2024-09-19T05:15:21.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.241+0000 7f7f93e00640 10 client.27655 handle_command_reply: tid=0 2024-09-19T05:15:21.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.241+0000 7f7fbbe0a740 1 client.27655 shutdown 2024-09-19T05:15:21.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.241+0000 7f7fbbe0a740 1 -- 172.21.15.43:0/2709344958 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b02d5b9590 msgr2=0x55b02d5c1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:21.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.241+0000 7f7fbbe0a740 1 --2- 172.21.15.43:0/2709344958 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b02d5b9590 0x55b02d5c1280 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7fa000e7c0 tx=0x7f7fa000efd0 comp rx=0 tx=0).stop 2024-09-19T05:15:21.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.241+0000 7f7fbbe0a740 1 -- 172.21.15.43:0/2709344958 shutdown_connections 2024-09-19T05:15:21.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.241+0000 7f7fbbe0a740 1 --2- 172.21.15.43:0/2709344958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d5c3e50 0x55b02d5c17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:21.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.241+0000 7f7fbbe0a740 1 --2- 172.21.15.43:0/2709344958 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55b02d5d3f20 0x55b02d5d6310 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:21.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.242+0000 7f7fbbe0a740 1 --2- 172.21.15.43:0/2709344958 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b02d5b9590 0x55b02d5c1280 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:21.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.242+0000 7f7f93e00640 0 client.27655 ms_handle_reset on v2:172.21.15.43:6838/3404555030 2024-09-19T05:15:21.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.242+0000 7f7fbbe0a740 1 --2- 172.21.15.43:0/2709344958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b02d5b8c70 0x55b02d5c0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:21.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.242+0000 7f7fbbe0a740 1 -- 172.21.15.43:0/2709344958 >> 172.21.15.43:0/2709344958 conn(0x55b02d5b5200 msgr2=0x55b02d5baf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:21.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.242+0000 7f7fbbe0a740 1 -- 172.21.15.43:0/2709344958 shutdown_connections 2024-09-19T05:15:21.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.242+0000 7f7fbbe0a740 1 -- 172.21.15.43:0/2709344958 wait complete. 2024-09-19T05:15:21.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.242+0000 7f7fbbe0a740 20 client.27655 trim_cache size 0 max 16384 2024-09-19T05:15:21.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.256+0000 7f7fbbe0a740 1 Processor -- start 2024-09-19T05:15:21.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.256+0000 7f7fbbe0a740 1 -- start start 2024-09-19T05:15:21.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.256+0000 7f7fbbe0a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b02d5b8c70 0x55b02d5c2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:21.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.256+0000 7f7fbbe0a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b02d5b9590 0x55b02d5db8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:21.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.256+0000 7f7fbbe0a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d5c3e50 0x55b02d5dbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:21.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.256+0000 7f7fbbe0a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb41b3ef0 con 0x55b02d5b9590 2024-09-19T05:15:21.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.256+0000 7f7fbbe0a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb41b5a30 con 0x55b02d5c3e50 2024-09-19T05:15:21.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.256+0000 7f7fbbe0a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb41b45c0 con 0x55b02d5b8c70 2024-09-19T05:15:21.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.256+0000 7f7fb1400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b02d5b9590 0x55b02d5db8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:21.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.256+0000 7f7fb1400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b02d5b9590 0x55b02d5db8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34326/0 (socket says 172.21.15.43:34326) 2024-09-19T05:15:21.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.256+0000 7f7fb1400640 1 -- 172.21.15.43:0/1369754222 learned_addr learned my addr 172.21.15.43:0/1369754222 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:15:21.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.256+0000 7f7fb1e00640 1 --2- 172.21.15.43:0/1369754222 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b02d5b8c70 0x55b02d5c2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:21.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.257+0000 7f7fb2800640 1 --2- 172.21.15.43:0/1369754222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d5c3e50 0x55b02d5dbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:21.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.257+0000 7f7fb1400640 1 -- 172.21.15.43:0/1369754222 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b02d5b8c70 msgr2=0x55b02d5c2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:21.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.257+0000 7f7fb1400640 1 --2- 172.21.15.43:0/1369754222 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b02d5b8c70 0x55b02d5c2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:21.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.257+0000 7f7fb1400640 1 -- 172.21.15.43:0/1369754222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d5c3e50 msgr2=0x55b02d5dbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:21.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.257+0000 7f7fb1400640 1 --2- 172.21.15.43:0/1369754222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d5c3e50 0x55b02d5dbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:21.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.257+0000 7f7fb1400640 1 -- 172.21.15.43:0/1369754222 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b02d5b0770 con 0x55b02d5b9590 2024-09-19T05:15:21.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.257+0000 7f7fb1400640 1 --2- 172.21.15.43:0/1369754222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b02d5b9590 0x55b02d5db8b0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0004eb0 tx=0x7f7fa000bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec32a85232caa3d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:21.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.257+0000 7f7f9ea00640 1 -- 172.21.15.43:0/1369754222 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa0033070 con 0x55b02d5b9590 2024-09-19T05:15:21.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.257+0000 7f7f9ea00640 1 -- 172.21.15.43:0/1369754222 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fa0005e50 con 0x55b02d5b9590 2024-09-19T05:15:21.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.257+0000 7f7f9ea00640 1 -- 172.21.15.43:0/1369754222 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa00043d0 con 0x55b02d5b9590 2024-09-19T05:15:21.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.258+0000 7f7fbbe0a740 1 -- 172.21.15.43:0/1369754222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b02d5b9590 msgr2=0x55b02d5db8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:21.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.258+0000 7f7fbbe0a740 1 --2- 172.21.15.43:0/1369754222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b02d5b9590 0x55b02d5db8b0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0004eb0 tx=0x7f7fa000bf60 comp rx=0 tx=0).stop 2024-09-19T05:15:21.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.258+0000 7f7fbbe0a740 1 -- 172.21.15.43:0/1369754222 shutdown_connections 2024-09-19T05:15:21.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.258+0000 7f7fbbe0a740 1 --2- 172.21.15.43:0/1369754222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d5c3e50 0x55b02d5dbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:21.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.258+0000 7f7fbbe0a740 1 --2- 172.21.15.43:0/1369754222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b02d5b9590 0x55b02d5db8b0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:21.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.258+0000 7f7fbbe0a740 1 --2- 172.21.15.43:0/1369754222 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b02d5b8c70 0x55b02d5c2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:21.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.258+0000 7f7fbbe0a740 1 -- 172.21.15.43:0/1369754222 >> 172.21.15.43:0/1369754222 conn(0x55b02d5b5200 msgr2=0x55b02d5c25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:21.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.258+0000 7f7fbbe0a740 1 -- 172.21.15.43:0/1369754222 shutdown_connections 2024-09-19T05:15:21.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.258+0000 7f7fbbe0a740 1 -- 172.21.15.43:0/1369754222 wait complete. 2024-09-19T05:15:21.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.259+0000 7f7fbbe0a740 1 -- start start 2024-09-19T05:15:21.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.259+0000 7f7fbbe0a740 1 Processor -- start 2024-09-19T05:15:21.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.259+0000 7f7fbbe0a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b02d5b8c70 0x55b02d5bdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:21.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.259+0000 7f7fbbe0a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d5c3e50 0x55b02d5be270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:21.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.259+0000 7f7fbbe0a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b02d5be7b0 0x55b02d63d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:21.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.259+0000 7f7fbbe0a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb41b5360 con 0x55b02d5b8c70 2024-09-19T05:15:21.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.259+0000 7f7fbbe0a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb41b4c90 con 0x55b02d5c3e50 2024-09-19T05:15:21.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.259+0000 7f7fbbe0a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb41b1ef0 con 0x55b02d5be7b0 2024-09-19T05:15:21.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.259+0000 7f7fbbe0a740 10 client.0 _ll_register_callbacks cb 0x7f7fa40f3d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:15:21.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.260+0000 7f7fb1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b02d5b8c70 0x55b02d5bdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:21.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.260+0000 7f7fb1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b02d5b8c70 0x55b02d5bdd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34340/0 (socket says 172.21.15.43:34340) 2024-09-19T05:15:21.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.260+0000 7f7fb1e00640 1 -- 172.21.15.43:0/127183621 learned_addr learned my addr 172.21.15.43:0/127183621 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:15:21.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.260+0000 7f7fb1400640 1 --2- 172.21.15.43:0/127183621 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d5c3e50 0x55b02d5be270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:21.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.260+0000 7f7fb2800640 1 --2- 172.21.15.43:0/127183621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b02d5be7b0 0x55b02d63d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:21.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.260+0000 7f7fb1e00640 1 -- 172.21.15.43:0/127183621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b02d5be7b0 msgr2=0x55b02d63d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:21.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.260+0000 7f7fb1e00640 1 --2- 172.21.15.43:0/127183621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b02d5be7b0 0x55b02d63d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:21.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.260+0000 7f7fb1e00640 1 -- 172.21.15.43:0/127183621 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d5c3e50 msgr2=0x55b02d5be270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:21.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.260+0000 7f7fb1e00640 1 --2- 172.21.15.43:0/127183621 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d5c3e50 0x55b02d5be270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:21.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.260+0000 7f7fb1e00640 1 -- 172.21.15.43:0/127183621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b02d63ddc0 con 0x55b02d5b8c70 2024-09-19T05:15:21.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.260+0000 7f7fb1e00640 1 --2- 172.21.15.43:0/127183621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b02d5b8c70 0x55b02d5bdd30 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f7fa40b8b70 tx=0x7f7fa40f3f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=127ad08a30373a7f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:21.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.260+0000 7f7f93e00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-19T05:15:21.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.261+0000 7f7f93e00640 1 -- 172.21.15.43:0/127183621 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4049040 con 0x55b02d5b8c70 2024-09-19T05:15:21.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.261+0000 7f7f93e00640 1 -- 172.21.15.43:0/127183621 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7f7fa403b4f0 con 0x55b02d5b8c70 2024-09-19T05:15:21.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.261+0000 7f7fbbe0a740 1 -- 172.21.15.43:0/127183621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fa0005e50 con 0x55b02d5b8c70 2024-09-19T05:15:21.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.261+0000 7f7f93e00640 1 -- 172.21.15.43:0/127183621 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7fa40bc4e0 con 0x55b02d5b8c70 2024-09-19T05:15:21.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.261+0000 7f7fbbe0a740 10 client.17739 fetch_fsmap learned FSMap version 29 2024-09-19T05:15:21.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.261+0000 7f7fbbe0a740 1 -- 172.21.15.43:0/127183621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b02d63efb0 con 0x55b02d5b8c70 2024-09-19T05:15:21.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.261+0000 7f7f93e00640 1 -- 172.21.15.43:0/127183621 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fa0005e50 con 0x55b02d5b8c70 2024-09-19T05:15:21.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.261+0000 7f7f93e00640 1 -- 172.21.15.43:0/127183621 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7fa40a1e80 con 0x55b02d5b8c70 2024-09-19T05:15:21.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.261+0000 7f7fbbe0a740 10 client.17739 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:15:21.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.261+0000 7f7fbbe0a740 10 client.17739 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-09-19T05:15:21.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.262+0000 7f7fbbe0a740 20 client.17739 populate_metadata read hostname 'smithi043' 2024-09-19T05:15:21.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.262+0000 7f7fbbe0a740 1 --2- 172.21.15.43:0/127183621 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55b02d63f680 0x55b02d641aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:21.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.262+0000 7f7fbbe0a740 4 client.17739 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:15:21.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.262+0000 7f7fbbe0a740 1 -- 172.21.15.43:0/127183621 --> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b02d526e60 con 0x55b02d63f680 2024-09-19T05:15:21.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.262+0000 7f7fb1400640 1 --2- 172.21.15.43:0/127183621 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55b02d63f680 0x55b02d641aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:21.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.262+0000 7f7fb1400640 1 --2- 172.21.15.43:0/127183621 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55b02d63f680 0x55b02d641aa0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=970b62f5fcf0e396 server_cookie=34a3271eb42ee541 in_seq=0 out_seq=0 2024-09-19T05:15:21.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.262+0000 7f7f93e00640 10 client.17739 ms_handle_connect on v2:172.21.15.43:6838/3404555030 2024-09-19T05:15:21.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.263+0000 7f7f93e00640 1 -- 172.21.15.43:0/127183621 <== mds.1 v2:172.21.15.43:6838/3404555030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b02d526e60 con 0x55b02d63f680 2024-09-19T05:15:21.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.263+0000 7f7f93e00640 10 client.17739 handle_command_reply: tid=0 2024-09-19T05:15:21.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.263+0000 7f7fbbe0a740 1 client.17739 shutdown 2024-09-19T05:15:21.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.263+0000 7f7fbbe0a740 1 -- 172.21.15.43:0/127183621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b02d5b8c70 msgr2=0x55b02d5bdd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:21.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.263+0000 7f7fbbe0a740 1 --2- 172.21.15.43:0/127183621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b02d5b8c70 0x55b02d5bdd30 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f7fa40b8b70 tx=0x7f7fa40f3f90 comp rx=0 tx=0).stop 2024-09-19T05:15:21.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.263+0000 7f7fbbe0a740 1 -- 172.21.15.43:0/127183621 shutdown_connections 2024-09-19T05:15:21.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.263+0000 7f7fbbe0a740 1 --2- 172.21.15.43:0/127183621 >> [v2:172.21.15.43:6838/3404555030,v1:172.21.15.43:6839/3404555030] conn(0x55b02d63f680 0x55b02d641aa0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:21.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.263+0000 7f7fbbe0a740 1 --2- 172.21.15.43:0/127183621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b02d5be7b0 0x55b02d63d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:21.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.263+0000 7f7fbbe0a740 1 --2- 172.21.15.43:0/127183621 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b02d5c3e50 0x55b02d5be270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:21.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.263+0000 7f7fbbe0a740 1 --2- 172.21.15.43:0/127183621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b02d5b8c70 0x55b02d5bdd30 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:21.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.263+0000 7f7fbbe0a740 1 -- 172.21.15.43:0/127183621 >> 172.21.15.43:0/127183621 conn(0x55b02d5b5200 msgr2=0x55b02d5c1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:21.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.263+0000 7f7f93e00640 0 client.17739 ms_handle_reset on v2:172.21.15.43:6838/3404555030 2024-09-19T05:15:21.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.263+0000 7f7fbbe0a740 1 -- 172.21.15.43:0/127183621 shutdown_connections 2024-09-19T05:15:21.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.263+0000 7f7fbbe0a740 1 -- 172.21.15.43:0/127183621 wait complete. 2024-09-19T05:15:21.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.264+0000 7f7fbbe0a740 20 client.17739 trim_cache size 0 max 16384 2024-09-19T05:15:21.281 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:15:21.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.264+0000 7f7fb9200640 1 -- 172.21.15.43:0/4066915488 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f7f980a3430 msgr2=0x7f7f980a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:21.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.264+0000 7f7fb9200640 1 --2- 172.21.15.43:0/4066915488 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f7f980a3430 0x7f7f980a58f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7fac0049d0 tx=0x7f7fac009600 comp rx=0 tx=0).stop 2024-09-19T05:15:21.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.264+0000 7f7fb9200640 1 -- 172.21.15.43:0/4066915488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fb4076720 msgr2=0x7f7fb41a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:21.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.264+0000 7f7fb9200640 1 --2- 172.21.15.43:0/4066915488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fb4076720 0x7f7fb41a46a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7fa402a910 tx=0x7f7fa4003b90 comp rx=0 tx=0).stop 2024-09-19T05:15:21.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.264+0000 7f7fb9200640 1 -- 172.21.15.43:0/4066915488 shutdown_connections 2024-09-19T05:15:21.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.264+0000 7f7fb9200640 1 --2- 172.21.15.43:0/4066915488 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f7f980a3430 0x7f7f980a58f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:21.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.265+0000 7f7fb9200640 1 --2- 172.21.15.43:0/4066915488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fb4112a30 0x7f7fb419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:21.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.265+0000 7f7fb9200640 1 --2- 172.21.15.43:0/4066915488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fb4077040 0x7f7fb41a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:21.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.265+0000 7f7fb9200640 1 --2- 172.21.15.43:0/4066915488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fb4076720 0x7f7fb41a46a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:21.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.265+0000 7f7fb9200640 1 -- 172.21.15.43:0/4066915488 >> 172.21.15.43:0/4066915488 conn(0x7f7fb40ffa90 msgr2=0x7f7fb4101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:21.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.265+0000 7f7fb9200640 1 -- 172.21.15.43:0/4066915488 shutdown_connections 2024-09-19T05:15:21.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:21.265+0000 7f7fb9200640 1 -- 172.21.15.43:0/4066915488 wait complete. 2024-09-19T05:15:22.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:21 smithi043 ceph-mon[30800]: pgmap v334: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T05:15:22.102 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T05:15:22.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:21 smithi096 ceph-mon[35085]: pgmap v334: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T05:15:22.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:21 smithi151 ceph-mon[35425]: pgmap v334: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T05:15:22.478 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:15:23.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.047+0000 7fc4edc00640 1 -- 172.21.15.43:0/3481646839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4e810e440 msgr2=0x7fc4e8110830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:23.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.047+0000 7fc4edc00640 1 --2- 172.21.15.43:0/3481646839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4e810e440 0x7fc4e8110830 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0009ca0 tx=0x7fc4e002a950 comp rx=0 tx=0).stop 2024-09-19T05:15:23.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.047+0000 7fc4edc00640 1 -- 172.21.15.43:0/3481646839 shutdown_connections 2024-09-19T05:15:23.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.047+0000 7fc4edc00640 1 --2- 172.21.15.43:0/3481646839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4e810e440 0x7fc4e8110830 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:23.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.047+0000 7fc4edc00640 1 --2- 172.21.15.43:0/3481646839 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4e8074e30 0x7fc4e810df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:23.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.047+0000 7fc4edc00640 1 --2- 172.21.15.43:0/3481646839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4e8074510 0x7fc4e80748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:23.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.047+0000 7fc4edc00640 1 -- 172.21.15.43:0/3481646839 >> 172.21.15.43:0/3481646839 conn(0x7fc4e80fd8c0 msgr2=0x7fc4e80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:23.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.047+0000 7fc4edc00640 1 -- 172.21.15.43:0/3481646839 shutdown_connections 2024-09-19T05:15:23.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.047+0000 7fc4edc00640 1 -- 172.21.15.43:0/3481646839 wait complete. 2024-09-19T05:15:23.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.048+0000 7fc4edc00640 1 Processor -- start 2024-09-19T05:15:23.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.048+0000 7fc4edc00640 1 -- start start 2024-09-19T05:15:23.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.048+0000 7fc4edc00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4e8074510 0x7fc4e819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:23.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.048+0000 7fc4edc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4e8074e30 0x7fc4e819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:23.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.049+0000 7fc4edc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4e810e440 0x7fc4e819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:23.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.049+0000 7fc4edc00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e81141c0 con 0x7fc4e8074e30 2024-09-19T05:15:23.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.049+0000 7fc4edc00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e8114040 con 0x7fc4e8074510 2024-09-19T05:15:23.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.049+0000 7fc4edc00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e8114340 con 0x7fc4e810e440 2024-09-19T05:15:23.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.049+0000 7fc4e6000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4e8074e30 0x7fc4e819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:23.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.049+0000 7fc4e6000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4e8074e30 0x7fc4e819ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34364/0 (socket says 172.21.15.43:34364) 2024-09-19T05:15:23.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.049+0000 7fc4e6000640 1 -- 172.21.15.43:0/2759726964 learned_addr learned my addr 172.21.15.43:0/2759726964 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:15:23.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.049+0000 7fc4e6a00640 1 --2- 172.21.15.43:0/2759726964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4e8074510 0x7fc4e819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:23.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.049+0000 7fc4e7400640 1 --2- 172.21.15.43:0/2759726964 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4e810e440 0x7fc4e819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:23.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.049+0000 7fc4e6000640 1 -- 172.21.15.43:0/2759726964 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4e810e440 msgr2=0x7fc4e819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:23.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.049+0000 7fc4e6000640 1 --2- 172.21.15.43:0/2759726964 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4e810e440 0x7fc4e819ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:23.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.049+0000 7fc4e6000640 1 -- 172.21.15.43:0/2759726964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4e8074510 msgr2=0x7fc4e819a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:23.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.049+0000 7fc4e6000640 1 --2- 172.21.15.43:0/2759726964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4e8074510 0x7fc4e819a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:23.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.049+0000 7fc4e6000640 1 -- 172.21.15.43:0/2759726964 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e819f5f0 con 0x7fc4e8074e30 2024-09-19T05:15:23.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.050+0000 7fc4e6000640 1 --2- 172.21.15.43:0/2759726964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4e8074e30 0x7fc4e819ab80 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400e9f0 tx=0x7fc4d400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:23.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.050+0000 7fc4df400640 1 -- 172.21.15.43:0/2759726964 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d400cd80 con 0x7fc4e8074e30 2024-09-19T05:15:23.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.050+0000 7fc4df400640 1 -- 172.21.15.43:0/2759726964 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4d4005500 con 0x7fc4e8074e30 2024-09-19T05:15:23.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.050+0000 7fc4df400640 1 -- 172.21.15.43:0/2759726964 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4010690 con 0x7fc4e8074e30 2024-09-19T05:15:23.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.050+0000 7fc4edc00640 1 -- 172.21.15.43:0/2759726964 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4e819f8e0 con 0x7fc4e8074e30 2024-09-19T05:15:23.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.050+0000 7fc4edc00640 1 -- 172.21.15.43:0/2759726964 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4e81a71c0 con 0x7fc4e8074e30 2024-09-19T05:15:23.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.053+0000 7fc4df400640 1 -- 172.21.15.43:0/2759726964 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fc4d40046c0 con 0x7fc4e8074e30 2024-09-19T05:15:23.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.053+0000 7fc4f080a740 1 Processor -- start 2024-09-19T05:15:23.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.053+0000 7fc4f080a740 1 -- start start 2024-09-19T05:15:23.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.054+0000 7fc4f080a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb71a3ec70 0x55fb71a3f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:23.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.054+0000 7fc4f080a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb71a3f590 0x55fb71a49910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:23.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.054+0000 7fc4f080a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb71a49e50 0x55fb71a4c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:23.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.054+0000 7fc4f080a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fb719a53e0 con 0x55fb71a3ec70 2024-09-19T05:15:23.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.054+0000 7fc4f080a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fb7194cd80 con 0x55fb71a49e50 2024-09-19T05:15:23.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.054+0000 7fc4f080a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55fb719a13a0 con 0x55fb71a3f590 2024-09-19T05:15:23.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.054+0000 7fc4e6a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb71a3f590 0x55fb71a49910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:23.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.054+0000 7fc4e7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb71a3ec70 0x55fb71a3f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:23.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.054+0000 7fc4e6a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb71a3f590 0x55fb71a49910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:51562/0 (socket says 172.21.15.43:51562) 2024-09-19T05:15:23.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.054+0000 7fc4e6a00640 1 -- 172.21.15.43:0/1584904307 learned_addr learned my addr 172.21.15.43:0/1584904307 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:15:23.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.054+0000 7fc4e6000640 1 --2- 172.21.15.43:0/1584904307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb71a49e50 0x55fb71a4c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:23.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.055+0000 7fc4e6a00640 1 -- 172.21.15.43:0/1584904307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb71a49e50 msgr2=0x55fb71a4c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:23.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.055+0000 7fc4e6a00640 1 --2- 172.21.15.43:0/1584904307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb71a49e50 0x55fb71a4c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:23.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.055+0000 7fc4e6a00640 1 -- 172.21.15.43:0/1584904307 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb71a3ec70 msgr2=0x55fb71a3f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:23.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.055+0000 7fc4e6a00640 1 --2- 172.21.15.43:0/1584904307 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb71a3ec70 0x55fb71a3f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:23.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.055+0000 7fc4e6a00640 1 -- 172.21.15.43:0/1584904307 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb7196bae0 con 0x55fb71a3f590 2024-09-19T05:15:23.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.055+0000 7fc4df400640 1 --2- 172.21.15.43:0/2759726964 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fc4cc0a3430 0x7fc4cc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:23.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.055+0000 7fc4e6a00640 1 --2- 172.21.15.43:0/1584904307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb71a3f590 0x55fb71a49910 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8009cf0 tx=0x7fc4d8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc6cd20446f02355 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:23.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.055+0000 7fc4df400640 1 -- 172.21.15.43:0/2759726964 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7fc4d40d00e0 con 0x7fc4e8074e30 2024-09-19T05:15:23.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.055+0000 7fc4e7400640 1 --2- 172.21.15.43:0/2759726964 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fc4cc0a3430 0x7fc4cc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:23.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.056+0000 7fc4d3e00640 1 -- 172.21.15.43:0/1584904307 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8004230 con 0x55fb71a3f590 2024-09-19T05:15:23.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.056+0000 7fc4d3e00640 1 -- 172.21.15.43:0/1584904307 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4d80043d0 con 0x55fb71a3f590 2024-09-19T05:15:23.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.056+0000 7fc4d3e00640 1 -- 172.21.15.43:0/1584904307 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8019960 con 0x55fb71a3f590 2024-09-19T05:15:23.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.057+0000 7fc4f080a740 1 -- 172.21.15.43:0/1584904307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb71a3f590 msgr2=0x55fb71a49910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:23.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.057+0000 7fc4f080a740 1 --2- 172.21.15.43:0/1584904307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb71a3f590 0x55fb71a49910 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8009cf0 tx=0x7fc4d8011810 comp rx=0 tx=0).stop 2024-09-19T05:15:23.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.057+0000 7fc4e7400640 1 --2- 172.21.15.43:0/2759726964 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fc4cc0a3430 0x7fc4cc0a58f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0004af0 tx=0x7fc4e0005d80 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:23.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.057+0000 7fc4f080a740 1 -- 172.21.15.43:0/1584904307 shutdown_connections 2024-09-19T05:15:23.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.057+0000 7fc4f080a740 1 --2- 172.21.15.43:0/1584904307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb71a49e50 0x55fb71a4c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:23.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.057+0000 7fc4f080a740 1 --2- 172.21.15.43:0/1584904307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb71a3f590 0x55fb71a49910 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:23.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.057+0000 7fc4f080a740 1 --2- 172.21.15.43:0/1584904307 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb71a3ec70 0x55fb71a3f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:23.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.057+0000 7fc4f080a740 1 -- 172.21.15.43:0/1584904307 >> 172.21.15.43:0/1584904307 conn(0x55fb71a3b200 msgr2=0x55fb71a3b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:23.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.057+0000 7fc4f080a740 1 -- 172.21.15.43:0/1584904307 shutdown_connections 2024-09-19T05:15:23.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.057+0000 7fc4f080a740 1 -- 172.21.15.43:0/1584904307 wait complete. 2024-09-19T05:15:23.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.057+0000 7fc4f080a740 1 -- start start 2024-09-19T05:15:23.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.058+0000 7fc4f080a740 1 Processor -- start 2024-09-19T05:15:23.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.058+0000 7fc4f080a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb71a3ec70 0x55fb71a46ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:23.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.058+0000 7fc4f080a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb71a3f590 0x55fb71a47220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:23.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.058+0000 7fc4f080a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb71a49e50 0x55fb71a47760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:23.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.058+0000 7fc4f080a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fb71a4ca20 con 0x55fb71a3f590 2024-09-19T05:15:23.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.058+0000 7fc4f080a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fb719600f0 con 0x55fb71a3ec70 2024-09-19T05:15:23.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.058+0000 7fc4f080a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55fb71a4cba0 con 0x55fb71a49e50 2024-09-19T05:15:23.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.058+0000 7fc4f080a740 10 client.0 _ll_register_callbacks cb 0x55fb71947bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:15:23.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.058+0000 7fc4e6000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb71a3f590 0x55fb71a47220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:23.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.058+0000 7fc4e6a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb71a3ec70 0x55fb71a46ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:23.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.058+0000 7fc4e6a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb71a3ec70 0x55fb71a46ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:50284/0 (socket says 172.21.15.43:50284) 2024-09-19T05:15:23.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.058+0000 7fc4e6a00640 1 -- 172.21.15.43:0/1017194007 learned_addr learned my addr 172.21.15.43:0/1017194007 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:15:23.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.059+0000 7fc4e7400640 1 --2- 172.21.15.43:0/1017194007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb71a49e50 0x55fb71a47760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:23.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.059+0000 7fc4e6a00640 1 -- 172.21.15.43:0/1017194007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb71a49e50 msgr2=0x55fb71a47760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:23.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.059+0000 7fc4e6a00640 1 --2- 172.21.15.43:0/1017194007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb71a49e50 0x55fb71a47760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:23.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.059+0000 7fc4e6a00640 1 -- 172.21.15.43:0/1017194007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb71a3f590 msgr2=0x55fb71a47220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:23.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.059+0000 7fc4e6a00640 1 --2- 172.21.15.43:0/1017194007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb71a3f590 0x55fb71a47220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:23.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.059+0000 7fc4e6a00640 1 -- 172.21.15.43:0/1017194007 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb717163c0 con 0x55fb71a3ec70 2024-09-19T05:15:23.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.059+0000 7fc4e7400640 1 --2- 172.21.15.43:0/1017194007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb71a49e50 0x55fb71a47760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:15:23.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.059+0000 7fc4e6000640 1 --2- 172.21.15.43:0/1017194007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb71a3f590 0x55fb71a47220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:15:23.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.059+0000 7fc4e6a00640 1 --2- 172.21.15.43:0/1017194007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb71a3ec70 0x55fb71a46ce0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8000c00 tx=0x7fc4d8003dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4d8817790d859bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:23.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.059+0000 7fc4d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:15:23.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.059+0000 7fc4d0c00640 1 -- 172.21.15.43:0/1017194007 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8027070 con 0x55fb71a3ec70 2024-09-19T05:15:23.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.060+0000 7fc4f080a740 1 -- 172.21.15.43:0/1017194007 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4d80043d0 con 0x55fb71a3ec70 2024-09-19T05:15:23.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.060+0000 7fc4d0c00640 1 -- 172.21.15.43:0/1017194007 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7fc4d8005240 con 0x55fb71a3ec70 2024-09-19T05:15:23.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.060+0000 7fc4d0c00640 1 -- 172.21.15.43:0/1017194007 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc4d8033050 con 0x55fb71a3ec70 2024-09-19T05:15:23.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.060+0000 7fc4f080a740 10 client.27767 fetch_fsmap learned FSMap version 29 2024-09-19T05:15:23.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.060+0000 7fc4f080a740 1 -- 172.21.15.43:0/1017194007 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fb71a589f0 con 0x55fb71a3ec70 2024-09-19T05:15:23.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.060+0000 7fc4d0c00640 1 -- 172.21.15.43:0/1017194007 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4d80043d0 con 0x55fb71a3ec70 2024-09-19T05:15:23.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.060+0000 7fc4d0c00640 1 -- 172.21.15.43:0/1017194007 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc4d8004ba0 con 0x55fb71a3ec70 2024-09-19T05:15:23.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.060+0000 7fc4f080a740 10 client.27767 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:15:23.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.060+0000 7fc4f080a740 10 client.27767 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:15:23.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.060+0000 7fc4f080a740 20 client.27767 populate_metadata read hostname 'smithi043' 2024-09-19T05:15:23.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.060+0000 7fc4f080a740 1 --2- 172.21.15.43:0/1017194007 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55fb71a5a1c0 0x55fb71a5c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:23.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.060+0000 7fc4f080a740 4 client.27767 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:15:23.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.060+0000 7fc4f080a740 1 -- 172.21.15.43:0/1017194007 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb719b2950 con 0x55fb71a5a1c0 2024-09-19T05:15:23.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.061+0000 7fc4e6000640 1 --2- 172.21.15.43:0/1017194007 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55fb71a5a1c0 0x55fb71a5c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:23.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.061+0000 7fc4e6000640 1 --2- 172.21.15.43:0/1017194007 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55fb71a5a1c0 0x55fb71a5c5b0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b73a9973f39011c server_cookie=ea5d4306784575ea in_seq=0 out_seq=0 2024-09-19T05:15:23.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.061+0000 7fc4d0c00640 10 client.27767 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:15:23.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.062+0000 7fc4d0c00640 1 -- 172.21.15.43:0/1017194007 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fb719b2950 con 0x55fb71a5a1c0 2024-09-19T05:15:23.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.062+0000 7fc4d0c00640 10 client.27767 handle_command_reply: tid=0 2024-09-19T05:15:23.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.062+0000 7fc4f080a740 1 client.27767 shutdown 2024-09-19T05:15:23.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.062+0000 7fc4f080a740 1 -- 172.21.15.43:0/1017194007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb71a3ec70 msgr2=0x55fb71a46ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:23.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.062+0000 7fc4f080a740 1 --2- 172.21.15.43:0/1017194007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb71a3ec70 0x55fb71a46ce0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8000c00 tx=0x7fc4d8003dc0 comp rx=0 tx=0).stop 2024-09-19T05:15:23.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.062+0000 7fc4f080a740 1 -- 172.21.15.43:0/1017194007 shutdown_connections 2024-09-19T05:15:23.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.062+0000 7fc4f080a740 1 --2- 172.21.15.43:0/1017194007 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55fb71a5a1c0 0x55fb71a5c5b0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:23.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.062+0000 7fc4f080a740 1 --2- 172.21.15.43:0/1017194007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb71a49e50 0x55fb71a47760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:23.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.062+0000 7fc4d0c00640 0 client.27767 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:15:23.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.062+0000 7fc4f080a740 1 --2- 172.21.15.43:0/1017194007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb71a3f590 0x55fb71a47220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:23.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.062+0000 7fc4f080a740 1 --2- 172.21.15.43:0/1017194007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb71a3ec70 0x55fb71a46ce0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:23.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.062+0000 7fc4f080a740 1 -- 172.21.15.43:0/1017194007 >> 172.21.15.43:0/1017194007 conn(0x55fb71a3b200 msgr2=0x55fb71a40f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:23.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.062+0000 7fc4f080a740 1 -- 172.21.15.43:0/1017194007 shutdown_connections 2024-09-19T05:15:23.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.062+0000 7fc4f080a740 1 -- 172.21.15.43:0/1017194007 wait complete. 2024-09-19T05:15:23.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.063+0000 7fc4f080a740 20 client.27767 trim_cache size 0 max 16384 2024-09-19T05:15:23.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.076+0000 7fc4f080a740 1 Processor -- start 2024-09-19T05:15:23.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.076+0000 7fc4f080a740 1 -- start start 2024-09-19T05:15:23.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.077+0000 7fc4f080a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb71a3ec70 0x55fb71a416d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:23.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.077+0000 7fc4f080a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb71a3f590 0x55fb71a3a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:23.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.077+0000 7fc4f080a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb71a49e50 0x55fb71a5f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:23.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.077+0000 7fc4f080a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e81afb80 con 0x55fb71a3f590 2024-09-19T05:15:23.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.077+0000 7fc4f080a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e81b16c0 con 0x55fb71a3ec70 2024-09-19T05:15:23.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.077+0000 7fc4f080a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e81b0250 con 0x55fb71a49e50 2024-09-19T05:15:23.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.077+0000 7fc4e6a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb71a3ec70 0x55fb71a416d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:23.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.077+0000 7fc4e6a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb71a3ec70 0x55fb71a416d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:50286/0 (socket says 172.21.15.43:50286) 2024-09-19T05:15:23.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.077+0000 7fc4e6a00640 1 -- 172.21.15.43:0/4043456678 learned_addr learned my addr 172.21.15.43:0/4043456678 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:15:23.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.077+0000 7fc4e6000640 1 --2- 172.21.15.43:0/4043456678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb71a3f590 0x55fb71a3a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:23.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.077+0000 7fc4e7400640 1 --2- 172.21.15.43:0/4043456678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb71a49e50 0x55fb71a5f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:23.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.077+0000 7fc4e6a00640 1 -- 172.21.15.43:0/4043456678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb71a49e50 msgr2=0x55fb71a5f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:23.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.077+0000 7fc4e6a00640 1 --2- 172.21.15.43:0/4043456678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb71a49e50 0x55fb71a5f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:23.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.077+0000 7fc4e6a00640 1 -- 172.21.15.43:0/4043456678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb71a3f590 msgr2=0x55fb71a3a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:23.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.077+0000 7fc4e6a00640 1 --2- 172.21.15.43:0/4043456678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb71a3f590 0x55fb71a3a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:23.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.077+0000 7fc4e6a00640 1 -- 172.21.15.43:0/4043456678 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb71a3b030 con 0x55fb71a3ec70 2024-09-19T05:15:23.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.077+0000 7fc4e7400640 1 --2- 172.21.15.43:0/4043456678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb71a49e50 0x55fb71a5f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:15:23.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.078+0000 7fc4e6a00640 1 --2- 172.21.15.43:0/4043456678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb71a3ec70 0x55fb71a416d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8003fc0 tx=0x7fc4d80097c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59e65f0ddf3d06dc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:23.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.078+0000 7fc4d3400640 1 -- 172.21.15.43:0/4043456678 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8030020 con 0x55fb71a3ec70 2024-09-19T05:15:23.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.078+0000 7fc4d3400640 1 -- 172.21.15.43:0/4043456678 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4d8009910 con 0x55fb71a3ec70 2024-09-19T05:15:23.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.078+0000 7fc4d3400640 1 -- 172.21.15.43:0/4043456678 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8027070 con 0x55fb71a3ec70 2024-09-19T05:15:23.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.078+0000 7fc4f080a740 1 -- 172.21.15.43:0/4043456678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb71a3ec70 msgr2=0x55fb71a416d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:23.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.078+0000 7fc4f080a740 1 --2- 172.21.15.43:0/4043456678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb71a3ec70 0x55fb71a416d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8003fc0 tx=0x7fc4d80097c0 comp rx=0 tx=0).stop 2024-09-19T05:15:23.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.079+0000 7fc4f080a740 1 -- 172.21.15.43:0/4043456678 shutdown_connections 2024-09-19T05:15:23.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.079+0000 7fc4f080a740 1 --2- 172.21.15.43:0/4043456678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb71a49e50 0x55fb71a5f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:23.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.079+0000 7fc4f080a740 1 --2- 172.21.15.43:0/4043456678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb71a3f590 0x55fb71a3a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:23.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.079+0000 7fc4f080a740 1 --2- 172.21.15.43:0/4043456678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb71a3ec70 0x55fb71a416d0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:23.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.079+0000 7fc4f080a740 1 -- 172.21.15.43:0/4043456678 >> 172.21.15.43:0/4043456678 conn(0x55fb71a3b200 msgr2=0x55fb71a39b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:23.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.079+0000 7fc4f080a740 1 -- 172.21.15.43:0/4043456678 shutdown_connections 2024-09-19T05:15:23.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.079+0000 7fc4f080a740 1 -- 172.21.15.43:0/4043456678 wait complete. 2024-09-19T05:15:23.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.079+0000 7fc4f080a740 1 -- start start 2024-09-19T05:15:23.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.079+0000 7fc4f080a740 1 Processor -- start 2024-09-19T05:15:23.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.080+0000 7fc4f080a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb71a3ec70 0x55fb71a5cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:23.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.080+0000 7fc4f080a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb71a3f590 0x55fb71a5d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:23.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.080+0000 7fc4f080a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb71a49e50 0x55fb71ac2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:23.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.080+0000 7fc4f080a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e81b0ff0 con 0x55fb71a3f590 2024-09-19T05:15:23.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.080+0000 7fc4f080a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e81b0920 con 0x55fb71a3ec70 2024-09-19T05:15:23.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.080+0000 7fc4f080a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e81adb80 con 0x55fb71a49e50 2024-09-19T05:15:23.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.080+0000 7fc4f080a740 10 client.0 _ll_register_callbacks cb 0x7fc4d4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:15:23.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.080+0000 7fc4e6a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb71a3ec70 0x55fb71a5cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:23.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.080+0000 7fc4e6000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb71a3f590 0x55fb71a5d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:23.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.080+0000 7fc4e6a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb71a3ec70 0x55fb71a5cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:50300/0 (socket says 172.21.15.43:50300) 2024-09-19T05:15:23.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.080+0000 7fc4e6a00640 1 -- 172.21.15.43:0/3591905772 learned_addr learned my addr 172.21.15.43:0/3591905772 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:15:23.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.080+0000 7fc4e7400640 1 --2- 172.21.15.43:0/3591905772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb71a49e50 0x55fb71ac2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:23.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.080+0000 7fc4e6a00640 1 -- 172.21.15.43:0/3591905772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb71a49e50 msgr2=0x55fb71ac2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:23.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.080+0000 7fc4e6a00640 1 --2- 172.21.15.43:0/3591905772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb71a49e50 0x55fb71ac2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:23.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.080+0000 7fc4e6a00640 1 -- 172.21.15.43:0/3591905772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb71a3f590 msgr2=0x55fb71a5d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:15:23.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.080+0000 7fc4e6a00640 1 --2- 172.21.15.43:0/3591905772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb71a3f590 0x55fb71a5d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:23.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.080+0000 7fc4e6a00640 1 -- 172.21.15.43:0/3591905772 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb71a36770 con 0x55fb71a3ec70 2024-09-19T05:15:23.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.080+0000 7fc4e6000640 1 --2- 172.21.15.43:0/3591905772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb71a3f590 0x55fb71a5d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:15:23.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.081+0000 7fc4e6a00640 1 --2- 172.21.15.43:0/3591905772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb71a3ec70 0x55fb71a5cb50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8009cf0 tx=0x7fc4d8003e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b24186e48efaaf96 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:23.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.081+0000 7fc4d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T05:15:23.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.081+0000 7fc4d0c00640 1 -- 172.21.15.43:0/3591905772 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8027040 con 0x55fb71a3ec70 2024-09-19T05:15:23.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.081+0000 7fc4d0c00640 1 -- 172.21.15.43:0/3591905772 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7fc4d8004380 con 0x55fb71a3ec70 2024-09-19T05:15:23.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.081+0000 7fc4f080a740 1 -- 172.21.15.43:0/3591905772 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4d8009910 con 0x55fb71a3ec70 2024-09-19T05:15:23.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.081+0000 7fc4d0c00640 1 -- 172.21.15.43:0/3591905772 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc4d8019420 con 0x55fb71a3ec70 2024-09-19T05:15:23.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.081+0000 7fc4f080a740 10 client.27773 fetch_fsmap learned FSMap version 29 2024-09-19T05:15:23.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.081+0000 7fc4f080a740 1 -- 172.21.15.43:0/3591905772 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fb71ac4060 con 0x55fb71a3ec70 2024-09-19T05:15:23.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.082+0000 7fc4d0c00640 1 -- 172.21.15.43:0/3591905772 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4d8009910 con 0x55fb71a3ec70 2024-09-19T05:15:23.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.082+0000 7fc4d0c00640 1 -- 172.21.15.43:0/3591905772 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc4d8004640 con 0x55fb71a3ec70 2024-09-19T05:15:23.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.082+0000 7fc4f080a740 10 client.27773 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T05:15:23.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.082+0000 7fc4f080a740 10 client.27773 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T05:15:23.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.082+0000 7fc4f080a740 20 client.27773 populate_metadata read hostname 'smithi043' 2024-09-19T05:15:23.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.082+0000 7fc4f080a740 1 --2- 172.21.15.43:0/3591905772 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55fb71ac46b0 0x55fb71ac6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:23.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.082+0000 7fc4f080a740 4 client.27773 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T05:15:23.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.082+0000 7fc4f080a740 1 -- 172.21.15.43:0/3591905772 --> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fb719ace60 con 0x55fb71ac46b0 2024-09-19T05:15:23.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.082+0000 7fc4e6000640 1 --2- 172.21.15.43:0/3591905772 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55fb71ac46b0 0x55fb71ac6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:23.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.083+0000 7fc4e6000640 1 --2- 172.21.15.43:0/3591905772 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55fb71ac46b0 0x55fb71ac6ac0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38fbcf1ad3c56c8e server_cookie=f0c4253f77c60bf5 in_seq=0 out_seq=0 2024-09-19T05:15:23.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.083+0000 7fc4d0c00640 10 client.27773 ms_handle_connect on v2:172.21.15.96:6838/605527621 2024-09-19T05:15:23.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.083+0000 7fc4d0c00640 1 -- 172.21.15.43:0/3591905772 <== mds.0 v2:172.21.15.96:6838/605527621 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fb719ace60 con 0x55fb71ac46b0 2024-09-19T05:15:23.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.083+0000 7fc4d0c00640 10 client.27773 handle_command_reply: tid=0 2024-09-19T05:15:23.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.083+0000 7fc4f080a740 1 client.27773 shutdown 2024-09-19T05:15:23.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.083+0000 7fc4f080a740 1 -- 172.21.15.43:0/3591905772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb71a3ec70 msgr2=0x55fb71a5cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:23.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.083+0000 7fc4f080a740 1 --2- 172.21.15.43:0/3591905772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb71a3ec70 0x55fb71a5cb50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8009cf0 tx=0x7fc4d8003e10 comp rx=0 tx=0).stop 2024-09-19T05:15:23.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.083+0000 7fc4f080a740 1 -- 172.21.15.43:0/3591905772 shutdown_connections 2024-09-19T05:15:23.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.083+0000 7fc4f080a740 1 --2- 172.21.15.43:0/3591905772 >> [v2:172.21.15.96:6838/605527621,v1:172.21.15.96:6839/605527621] conn(0x55fb71ac46b0 0x55fb71ac6ac0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:23.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.083+0000 7fc4f080a740 1 --2- 172.21.15.43:0/3591905772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fb71a49e50 0x55fb71ac2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:23.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.083+0000 7fc4f080a740 1 --2- 172.21.15.43:0/3591905772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fb71a3f590 0x55fb71a5d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:23.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.083+0000 7fc4f080a740 1 --2- 172.21.15.43:0/3591905772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fb71a3ec70 0x55fb71a5cb50 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:23.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.083+0000 7fc4d0c00640 0 client.27773 ms_handle_reset on v2:172.21.15.96:6838/605527621 2024-09-19T05:15:23.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.083+0000 7fc4f080a740 1 -- 172.21.15.43:0/3591905772 >> 172.21.15.43:0/3591905772 conn(0x55fb71a3b200 msgr2=0x55fb71a3a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:23.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.084+0000 7fc4f080a740 1 -- 172.21.15.43:0/3591905772 shutdown_connections 2024-09-19T05:15:23.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.084+0000 7fc4f080a740 1 -- 172.21.15.43:0/3591905772 wait complete. 2024-09-19T05:15:23.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.084+0000 7fc4f080a740 20 client.27773 trim_cache size 0 max 16384 2024-09-19T05:15:23.091 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-19T05:15:23.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.084+0000 7fc4edc00640 1 -- 172.21.15.43:0/2759726964 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fc4cc0a3430 msgr2=0x7fc4cc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:23.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.084+0000 7fc4edc00640 1 --2- 172.21.15.43:0/2759726964 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fc4cc0a3430 0x7fc4cc0a58f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0004af0 tx=0x7fc4e0005d80 comp rx=0 tx=0).stop 2024-09-19T05:15:23.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.084+0000 7fc4edc00640 1 -- 172.21.15.43:0/2759726964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4e8074e30 msgr2=0x7fc4e819ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:23.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.084+0000 7fc4edc00640 1 --2- 172.21.15.43:0/2759726964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4e8074e30 0x7fc4e819ab80 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400e9f0 tx=0x7fc4d400edf0 comp rx=0 tx=0).stop 2024-09-19T05:15:23.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.085+0000 7fc4edc00640 1 -- 172.21.15.43:0/2759726964 shutdown_connections 2024-09-19T05:15:23.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.085+0000 7fc4edc00640 1 --2- 172.21.15.43:0/2759726964 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4e810e440 0x7fc4e819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:23.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.085+0000 7fc4edc00640 1 --2- 172.21.15.43:0/2759726964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4e8074e30 0x7fc4e819ab80 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:23.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.085+0000 7fc4edc00640 1 --2- 172.21.15.43:0/2759726964 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fc4cc0a3430 0x7fc4cc0a58f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:23.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.085+0000 7fc4edc00640 1 --2- 172.21.15.43:0/2759726964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4e8074510 0x7fc4e819a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:23.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.085+0000 7fc4edc00640 1 -- 172.21.15.43:0/2759726964 >> 172.21.15.43:0/2759726964 conn(0x7fc4e80fd8c0 msgr2=0x7fc4e80fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:23.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.085+0000 7fc4edc00640 1 -- 172.21.15.43:0/2759726964 shutdown_connections 2024-09-19T05:15:23.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:23.085+0000 7fc4edc00640 1 -- 172.21.15.43:0/2759726964 wait complete. 2024-09-19T05:15:23.921 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-19T05:15:23.922 INFO:tasks.fwd_scrub:done joining 2024-09-19T05:15:23.923 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-19T05:15:23.931 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T05:15:24.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.090+0000 7f6472c00640 1 -- 172.21.15.43:0/3861340710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f646c1048c0 msgr2=0x7f646c108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:24.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.090+0000 7f6472c00640 1 --2- 172.21.15.43:0/3861340710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f646c1048c0 0x7f646c108900 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f6454009860 tx=0x7f645402a340 comp rx=0 tx=0).stop 2024-09-19T05:15:24.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.090+0000 7f6472c00640 1 -- 172.21.15.43:0/3861340710 shutdown_connections 2024-09-19T05:15:24.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.090+0000 7f6472c00640 1 --2- 172.21.15.43:0/3861340710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f646c109060 0x7f646c110bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:24.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.090+0000 7f6472c00640 1 --2- 172.21.15.43:0/3861340710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f646c1048c0 0x7f646c108900 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:24.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.090+0000 7f6472c00640 1 --2- 172.21.15.43:0/3861340710 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f646c103f10 0x7f646c1042f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:24.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.090+0000 7f6472c00640 1 -- 172.21.15.43:0/3861340710 >> 172.21.15.43:0/3861340710 conn(0x7f646c0ff980 msgr2=0x7f646c101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:24.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.090+0000 7f6472c00640 1 -- 172.21.15.43:0/3861340710 shutdown_connections 2024-09-19T05:15:24.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.090+0000 7f6472c00640 1 -- 172.21.15.43:0/3861340710 wait complete. 2024-09-19T05:15:24.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.091+0000 7f6472c00640 1 Processor -- start 2024-09-19T05:15:24.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.091+0000 7f6472c00640 1 -- start start 2024-09-19T05:15:24.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.091+0000 7f6472c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f646c103f10 0x7f646c19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:24.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.091+0000 7f6472c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f646c109060 0x7f646c19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:24.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.091+0000 7f6472c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f646c19eeb0 0x7f646c19f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:24.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.091+0000 7f6472c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f646c114060 con 0x7f646c19eeb0 2024-09-19T05:15:24.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.091+0000 7f6472c00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f646c113ee0 con 0x7f646c103f10 2024-09-19T05:15:24.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.091+0000 7f6472c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f646c1141e0 con 0x7f646c109060 2024-09-19T05:15:24.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.092+0000 7f646be00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f646c103f10 0x7f646c19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:24.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.092+0000 7f6470c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f646c19eeb0 0x7f646c19f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:24.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.092+0000 7f646be00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f646c103f10 0x7f646c19a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:50316/0 (socket says 172.21.15.43:50316) 2024-09-19T05:15:24.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.092+0000 7f646be00640 1 -- 172.21.15.43:0/1961521128 learned_addr learned my addr 172.21.15.43:0/1961521128 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:15:24.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.092+0000 7f6463400640 1 --2- 172.21.15.43:0/1961521128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f646c109060 0x7f646c19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:24.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.092+0000 7f646be00640 1 -- 172.21.15.43:0/1961521128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f646c109060 msgr2=0x7f646c19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:24.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.092+0000 7f646be00640 1 --2- 172.21.15.43:0/1961521128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f646c109060 0x7f646c19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:24.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.092+0000 7f646be00640 1 -- 172.21.15.43:0/1961521128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f646c19eeb0 msgr2=0x7f646c19f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:24.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.092+0000 7f646be00640 1 --2- 172.21.15.43:0/1961521128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f646c19eeb0 0x7f646c19f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:24.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.092+0000 7f646be00640 1 -- 172.21.15.43:0/1961521128 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f646c19f9a0 con 0x7f646c103f10 2024-09-19T05:15:24.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.092+0000 7f646be00640 1 --2- 172.21.15.43:0/1961521128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f646c103f10 0x7f646c19a5f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f645c00e9c0 tx=0x7f645c00edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:24.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.093+0000 7f6469600640 1 -- 172.21.15.43:0/1961521128 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645c00cd80 con 0x7f646c103f10 2024-09-19T05:15:24.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.093+0000 7f6469600640 1 -- 172.21.15.43:0/1961521128 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f645c0035f0 con 0x7f646c103f10 2024-09-19T05:15:24.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.093+0000 7f6469600640 1 -- 172.21.15.43:0/1961521128 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645c010670 con 0x7f646c103f10 2024-09-19T05:15:24.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.093+0000 7f6472c00640 1 -- 172.21.15.43:0/1961521128 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f646c19fc90 con 0x7f646c103f10 2024-09-19T05:15:24.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.093+0000 7f6472c00640 1 -- 172.21.15.43:0/1961521128 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f646c1a7400 con 0x7f646c103f10 2024-09-19T05:15:24.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.094+0000 7f6472c00640 1 -- 172.21.15.43:0/1961521128 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f646c1042f0 con 0x7f646c103f10 2024-09-19T05:15:24.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.096+0000 7f6469600640 1 -- 172.21.15.43:0/1961521128 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f645c0037a0 con 0x7f646c103f10 2024-09-19T05:15:24.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.097+0000 7f6469600640 1 --2- 172.21.15.43:0/1961521128 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f64500a3440 0x7f64500a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:24.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.097+0000 7f6469600640 1 -- 172.21.15.43:0/1961521128 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7f645c0cfff0 con 0x7f646c103f10 2024-09-19T05:15:24.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.097+0000 7f6463400640 1 --2- 172.21.15.43:0/1961521128 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f64500a3440 0x7f64500a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:24.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.098+0000 7f6463400640 1 --2- 172.21.15.43:0/1961521128 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f64500a3440 0x7f64500a5900 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6454009830 tx=0x7f64540056d0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:24.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.099+0000 7f6469600640 1 -- 172.21.15.43:0/1961521128 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f645c09c9c0 con 0x7f646c103f10 2024-09-19T05:15:24.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:23 smithi096 ceph-mon[35085]: pgmap v335: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T05:15:24.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:23 smithi043 ceph-mon[30800]: pgmap v335: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T05:15:24.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.288+0000 7f6472c00640 1 -- 172.21.15.43:0/1961521128 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f646c19b940 con 0x7f646c103f10 2024-09-19T05:15:24.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.290+0000 7f6469600640 1 -- 172.21.15.43:0/1961521128 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+23128 (secure 0 0 0) 0x7f646c19b940 con 0x7f646c103f10 2024-09-19T05:15:24.293 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:15:24.293 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":96,"fsid":"1456a810-7643-11ef-bceb-c7b262605968","created":"2024-09-19T04:54:14.706644+0000","modified":"2024-09-19T05:15:15.893134+0000","last_up_change":"2024-09-19T05:01:27.348740+0000","last_in_change":"2024-09-19T05:01:03.396477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:57:27.642705+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:02:20.685123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:02:23.393596+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":95,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"81a317a8-7ec8-45c3-bc10-7da023283635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6803","nonce":825351486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6805","nonce":825351486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6809","nonce":825351486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6807","nonce":825351486}]},"public_addr":"172.21.15.43:6803/825351486","cluster_addr":"172.21.15.43:6805/825351486","heartbeat_back_addr":"172.21.15.43:6809/825351486","heartbeat_front_addr":"172.21.15.43:6807/825351486","state":["exists","up"]},{"osd":1,"uuid":"e96fcfac-4fff-4a36-b38b-aebf57842740","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6801","nonce":2140095130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6803","nonce":2140095130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6807","nonce":2140095130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6805","nonce":2140095130}]},"public_addr":"172.21.15.96:6801/2140095130","cluster_addr":"172.21.15.96:6803/2140095130","heartbeat_back_addr":"172.21.15.96:6807/2140095130","heartbeat_front_addr":"172.21.15.96:6805/2140095130","state":["exists","up"]},{"osd":2,"uuid":"b659869e-114a-4f56-8aac-a2ed602b4058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6801","nonce":2047147996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6803","nonce":2047147996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6807","nonce":2047147996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6805","nonce":2047147996}]},"public_addr":"172.21.15.151:6801/2047147996","cluster_addr":"172.21.15.151:6803/2047147996","heartbeat_back_addr":"172.21.15.151:6807/2047147996","heartbeat_front_addr":"172.21.15.151:6805/2047147996","state":["exists","up"]},{"osd":3,"uuid":"7be64b2c-e45c-493d-a610-4ce41196061f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6811","nonce":1734673762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6813","nonce":1734673762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6817","nonce":1734673762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6815","nonce":1734673762}]},"public_addr":"172.21.15.43:6811/1734673762","cluster_addr":"172.21.15.43:6813/1734673762","heartbeat_back_addr":"172.21.15.43:6817/1734673762","heartbeat_front_addr":"172.21.15.43:6815/1734673762","state":["exists","up"]},{"osd":4,"uuid":"196d5af1-a6e5-496c-8933-4e58892f4360","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6809","nonce":3184785362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6811","nonce":3184785362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6815","nonce":3184785362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6813","nonce":3184785362}]},"public_addr":"172.21.15.96:6809/3184785362","cluster_addr":"172.21.15.96:6811/3184785362","heartbeat_back_addr":"172.21.15.96:6815/3184785362","heartbeat_front_addr":"172.21.15.96:6813/3184785362","state":["exists","up"]},{"osd":5,"uuid":"5492d474-8509-4865-bc2e-d08e31558a75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6809","nonce":2878451924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6811","nonce":2878451924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6815","nonce":2878451924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6813","nonce":2878451924}]},"public_addr":"172.21.15.151:6809/2878451924","cluster_addr":"172.21.15.151:6811/2878451924","heartbeat_back_addr":"172.21.15.151:6815/2878451924","heartbeat_front_addr":"172.21.15.151:6813/2878451924","state":["exists","up"]},{"osd":6,"uuid":"fb8457ed-7bb9-4435-9d15-c1cbfbddfcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6819","nonce":2008630290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6821","nonce":2008630290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6825","nonce":2008630290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6823","nonce":2008630290}]},"public_addr":"172.21.15.43:6819/2008630290","cluster_addr":"172.21.15.43:6821/2008630290","heartbeat_back_addr":"172.21.15.43:6825/2008630290","heartbeat_front_addr":"172.21.15.43:6823/2008630290","state":["exists","up"]},{"osd":7,"uuid":"02046db8-30de-4e92-b25a-1f07d6190e91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6817","nonce":3588713275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6819","nonce":3588713275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6823","nonce":3588713275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6821","nonce":3588713275}]},"public_addr":"172.21.15.96:6817/3588713275","cluster_addr":"172.21.15.96:6819/3588713275","heartbeat_back_addr":"172.21.15.96:6823/3588713275","heartbeat_front_addr":"172.21.15.96:6821/3588713275","state":["exists","up"]},{"osd":8,"uuid":"a5bb4218-51f3-4044-b43e-ebfc764f49ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6817","nonce":725316393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6819","nonce":725316393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6823","nonce":725316393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6821","nonce":725316393}]},"public_addr":"172.21.15.151:6817/725316393","cluster_addr":"172.21.15.151:6819/725316393","heartbeat_back_addr":"172.21.15.151:6823/725316393","heartbeat_front_addr":"172.21.15.151:6821/725316393","state":["exists","up"]},{"osd":9,"uuid":"6c19f65c-9750-4baa-9d63-7b8bd03e6955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6827","nonce":4053438621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6829","nonce":4053438621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6833","nonce":4053438621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6831","nonce":4053438621}]},"public_addr":"172.21.15.43:6827/4053438621","cluster_addr":"172.21.15.43:6829/4053438621","heartbeat_back_addr":"172.21.15.43:6833/4053438621","heartbeat_front_addr":"172.21.15.43:6831/4053438621","state":["exists","up"]},{"osd":10,"uuid":"a7aa7da3-8eab-47fe-a351-298b6ff75f14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6825","nonce":2403210086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6827","nonce":2403210086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6831","nonce":2403210086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6829","nonce":2403210086}]},"public_addr":"172.21.15.96:6825/2403210086","cluster_addr":"172.21.15.96:6827/2403210086","heartbeat_back_addr":"172.21.15.96:6831/2403210086","heartbeat_front_addr":"172.21.15.96:6829/2403210086","state":["exists","up"]},{"osd":11,"uuid":"760b6afc-253d-44dd-88e9-dcc97f67c9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6825","nonce":292202881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6827","nonce":292202881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6831","nonce":292202881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6829","nonce":292202881}]},"public_addr":"172.21.15.151:6825/292202881","cluster_addr":"172.21.15.151:6827/292202881","heartbeat_back_addr":"172.21.15.151:6831/292202881","heartbeat_front_addr":"172.21.15.151:6829/292202881","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:56:34.716991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:56:59.321174+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:57:24.281696+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:57:53.443880+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:58:19.205873+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:58:44.504873+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:13.650731+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:39.137876+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:04.579893+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:34.337312+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:59.434658+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:01:24.930529+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/3819455723":"2024-09-20T05:04:38.237695+0000","172.21.15.43:0/3712369735":"2024-09-20T05:04:38.237695+0000","172.21.15.43:0/330854185":"2024-09-20T04:54:39.548065+0000","172.21.15.43:0/6132665":"2024-09-20T04:54:39.548065+0000","172.21.15.43:0/3730884138":"2024-09-20T05:04:38.237695+0000","172.21.15.43:0/632163478":"2024-09-20T04:54:39.548065+0000","172.21.15.43:0/4271788315":"2024-09-20T04:55:07.037117+0000","172.21.15.43:6800/587463881":"2024-09-20T04:54:39.548065+0000","172.21.15.43:6800/3241799377":"2024-09-20T04:55:07.037117+0000","172.21.15.43:6801/3241799377":"2024-09-20T04:55:07.037117+0000","172.21.15.43:0/898247657":"2024-09-20T04:55:07.037117+0000","172.21.15.43:6801/3167108925":"2024-09-20T05:04:38.237695+0000","172.21.15.43:0/515912755":"2024-09-20T05:04:38.237695+0000","172.21.15.43:6801/587463881":"2024-09-20T04:54:39.548065+0000","172.21.15.43:0/3256350596":"2024-09-20T04:55:07.037117+0000","172.21.15.43:6800/3167108925":"2024-09-20T05:04:38.237695+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:15:24.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.292+0000 7f6472c00640 1 -- 172.21.15.43:0/1961521128 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f64500a3440 msgr2=0x7f64500a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:24.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.292+0000 7f6472c00640 1 --2- 172.21.15.43:0/1961521128 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f64500a3440 0x7f64500a5900 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6454009830 tx=0x7f64540056d0 comp rx=0 tx=0).stop 2024-09-19T05:15:24.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.292+0000 7f6472c00640 1 -- 172.21.15.43:0/1961521128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f646c103f10 msgr2=0x7f646c19a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:24.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.292+0000 7f6472c00640 1 --2- 172.21.15.43:0/1961521128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f646c103f10 0x7f646c19a5f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f645c00e9c0 tx=0x7f645c00edc0 comp rx=0 tx=0).stop 2024-09-19T05:15:24.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.292+0000 7f6472c00640 1 -- 172.21.15.43:0/1961521128 shutdown_connections 2024-09-19T05:15:24.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.292+0000 7f6472c00640 1 --2- 172.21.15.43:0/1961521128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f646c19eeb0 0x7f646c19f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:24.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.292+0000 7f6472c00640 1 --2- 172.21.15.43:0/1961521128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f646c109060 0x7f646c19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:24.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.292+0000 7f6472c00640 1 --2- 172.21.15.43:0/1961521128 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f64500a3440 0x7f64500a5900 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:24.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.292+0000 7f6472c00640 1 --2- 172.21.15.43:0/1961521128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f646c103f10 0x7f646c19a5f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:24.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.292+0000 7f6472c00640 1 -- 172.21.15.43:0/1961521128 >> 172.21.15.43:0/1961521128 conn(0x7f646c0ff980 msgr2=0x7f646c100df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:24.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.292+0000 7f6472c00640 1 -- 172.21.15.43:0/1961521128 shutdown_connections 2024-09-19T05:15:24.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.292+0000 7f6472c00640 1 -- 172.21.15.43:0/1961521128 wait complete. 2024-09-19T05:15:24.313 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T04:57:27.642705+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-19T05:02:20.685123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-19T05:02:23.393596+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 95, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T05:15:24.313 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T05:15:24.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:23 smithi151 ceph-mon[35425]: pgmap v335: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-19T05:15:24.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.475+0000 7ff9cda00640 1 -- 172.21.15.43:0/840647477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9c81048c0 msgr2=0x7ff9c8108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:24.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.475+0000 7ff9cda00640 1 --2- 172.21.15.43:0/840647477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9c81048c0 0x7ff9c8108900 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0009900 tx=0x7ff9b002a410 comp rx=0 tx=0).stop 2024-09-19T05:15:24.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.475+0000 7ff9cda00640 1 -- 172.21.15.43:0/840647477 shutdown_connections 2024-09-19T05:15:24.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.476+0000 7ff9cda00640 1 --2- 172.21.15.43:0/840647477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9c8109060 0x7ff9c8110bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:24.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.476+0000 7ff9cda00640 1 --2- 172.21.15.43:0/840647477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9c81048c0 0x7ff9c8108900 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:24.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.476+0000 7ff9cda00640 1 --2- 172.21.15.43:0/840647477 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9c8103f10 0x7ff9c81042f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:24.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.476+0000 7ff9cda00640 1 -- 172.21.15.43:0/840647477 >> 172.21.15.43:0/840647477 conn(0x7ff9c80ff940 msgr2=0x7ff9c8101d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:24.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.476+0000 7ff9cda00640 1 -- 172.21.15.43:0/840647477 shutdown_connections 2024-09-19T05:15:24.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.476+0000 7ff9cda00640 1 -- 172.21.15.43:0/840647477 wait complete. 2024-09-19T05:15:24.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.476+0000 7ff9cda00640 1 Processor -- start 2024-09-19T05:15:24.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.477+0000 7ff9cda00640 1 -- start start 2024-09-19T05:15:24.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.477+0000 7ff9cda00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9c8103f10 0x7ff9c819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:24.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.477+0000 7ff9cda00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9c8109060 0x7ff9c819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:24.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.477+0000 7ff9cda00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9c819ef30 0x7ff9c819f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:24.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.477+0000 7ff9cda00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c8114060 con 0x7ff9c8109060 2024-09-19T05:15:24.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.477+0000 7ff9cda00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c8113ee0 con 0x7ff9c8103f10 2024-09-19T05:15:24.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.477+0000 7ff9cda00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c81141e0 con 0x7ff9c819ef30 2024-09-19T05:15:24.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.477+0000 7ff9c6a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9c8103f10 0x7ff9c819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:24.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.477+0000 7ff9be000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9c8109060 0x7ff9c819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:24.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.477+0000 7ff9c6a00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9c8103f10 0x7ff9c819a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:50332/0 (socket says 172.21.15.43:50332) 2024-09-19T05:15:24.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.477+0000 7ff9c6a00640 1 -- 172.21.15.43:0/2432137501 learned_addr learned my addr 172.21.15.43:0/2432137501 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:15:24.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.477+0000 7ff9c7400640 1 --2- 172.21.15.43:0/2432137501 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9c819ef30 0x7ff9c819f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:24.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.478+0000 7ff9c6a00640 1 -- 172.21.15.43:0/2432137501 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9c819ef30 msgr2=0x7ff9c819f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:24.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.478+0000 7ff9c6a00640 1 --2- 172.21.15.43:0/2432137501 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9c819ef30 0x7ff9c819f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:24.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.478+0000 7ff9c6a00640 1 -- 172.21.15.43:0/2432137501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9c8109060 msgr2=0x7ff9c819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:24.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.478+0000 7ff9c6a00640 1 --2- 172.21.15.43:0/2432137501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9c8109060 0x7ff9c819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:24.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.478+0000 7ff9c6a00640 1 -- 172.21.15.43:0/2432137501 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c819fa20 con 0x7ff9c8103f10 2024-09-19T05:15:24.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.478+0000 7ff9be000640 1 --2- 172.21.15.43:0/2432137501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9c8109060 0x7ff9c819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:15:24.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.478+0000 7ff9c6a00640 1 --2- 172.21.15.43:0/2432137501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9c8103f10 0x7ff9c819a670 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff9b800e9f0 tx=0x7ff9b800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:24.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.478+0000 7ff9bfe00640 1 -- 172.21.15.43:0/2432137501 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b800ccc0 con 0x7ff9c8103f10 2024-09-19T05:15:24.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.478+0000 7ff9cda00640 1 -- 172.21.15.43:0/2432137501 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9c819fd10 con 0x7ff9c8103f10 2024-09-19T05:15:24.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.478+0000 7ff9bfe00640 1 -- 172.21.15.43:0/2432137501 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9b8010910 con 0x7ff9c8103f10 2024-09-19T05:15:24.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.478+0000 7ff9bfe00640 1 -- 172.21.15.43:0/2432137501 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b80188a0 con 0x7ff9c8103f10 2024-09-19T05:15:24.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.479+0000 7ff9cda00640 1 -- 172.21.15.43:0/2432137501 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9c81a74d0 con 0x7ff9c8103f10 2024-09-19T05:15:24.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.479+0000 7ff9cda00640 1 -- 172.21.15.43:0/2432137501 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9c81042f0 con 0x7ff9c8103f10 2024-09-19T05:15:24.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.481+0000 7ff9bfe00640 1 -- 172.21.15.43:0/2432137501 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ff9b8010ac0 con 0x7ff9c8103f10 2024-09-19T05:15:24.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.483+0000 7ff9bfe00640 1 --2- 172.21.15.43:0/2432137501 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff9ac0a33c0 0x7ff9ac0a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:24.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.483+0000 7ff9bfe00640 1 -- 172.21.15.43:0/2432137501 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7ff9b80d0660 con 0x7ff9c8103f10 2024-09-19T05:15:24.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.483+0000 7ff9be000640 1 --2- 172.21.15.43:0/2432137501 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff9ac0a33c0 0x7ff9ac0a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:24.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.484+0000 7ff9be000640 1 --2- 172.21.15.43:0/2432137501 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff9ac0a33c0 0x7ff9ac0a5880 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0004770 tx=0x7ff9b00058d0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:24.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.485+0000 7ff9bfe00640 1 -- 172.21.15.43:0/2432137501 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff9b809cfd0 con 0x7ff9c8103f10 2024-09-19T05:15:24.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.672+0000 7ff9cda00640 1 -- 172.21.15.43:0/2432137501 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff9c810c2d0 con 0x7ff9c8103f10 2024-09-19T05:15:24.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.673+0000 7ff9bfe00640 1 -- 172.21.15.43:0/2432137501 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) ==== 93+0+10 (secure 0 0 0) 0x7ff9b809c8d0 con 0x7ff9c8103f10 2024-09-19T05:15:24.675 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-09-19T05:15:24.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.675+0000 7ff9cda00640 1 -- 172.21.15.43:0/2432137501 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff9ac0a33c0 msgr2=0x7ff9ac0a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:24.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.675+0000 7ff9cda00640 1 --2- 172.21.15.43:0/2432137501 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff9ac0a33c0 0x7ff9ac0a5880 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0004770 tx=0x7ff9b00058d0 comp rx=0 tx=0).stop 2024-09-19T05:15:24.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.675+0000 7ff9cda00640 1 -- 172.21.15.43:0/2432137501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9c8103f10 msgr2=0x7ff9c819a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:24.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.675+0000 7ff9cda00640 1 --2- 172.21.15.43:0/2432137501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9c8103f10 0x7ff9c819a670 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff9b800e9f0 tx=0x7ff9b800edf0 comp rx=0 tx=0).stop 2024-09-19T05:15:24.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.675+0000 7ff9cda00640 1 -- 172.21.15.43:0/2432137501 shutdown_connections 2024-09-19T05:15:24.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.675+0000 7ff9cda00640 1 --2- 172.21.15.43:0/2432137501 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9c819ef30 0x7ff9c819f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:24.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.675+0000 7ff9cda00640 1 --2- 172.21.15.43:0/2432137501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9c8109060 0x7ff9c819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:24.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.675+0000 7ff9cda00640 1 --2- 172.21.15.43:0/2432137501 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff9ac0a33c0 0x7ff9ac0a5880 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:24.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.675+0000 7ff9cda00640 1 --2- 172.21.15.43:0/2432137501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9c8103f10 0x7ff9c819a670 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:24.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.675+0000 7ff9cda00640 1 -- 172.21.15.43:0/2432137501 >> 172.21.15.43:0/2432137501 conn(0x7ff9c80ff940 msgr2=0x7ff9c8100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:24.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.675+0000 7ff9cda00640 1 -- 172.21.15.43:0/2432137501 shutdown_connections 2024-09-19T05:15:24.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.675+0000 7ff9cda00640 1 -- 172.21.15.43:0/2432137501 wait complete. 2024-09-19T05:15:24.694 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-19T05:15:24.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.854+0000 7f4f7a000640 1 -- 172.21.15.43:0/4232024616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f741047a0 msgr2=0x7f4f741087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:24.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.854+0000 7f4f7a000640 1 --2- 172.21.15.43:0/4232024616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f741047a0 0x7f4f741087e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4f60009900 tx=0x7f4f6002a3f0 comp rx=0 tx=0).stop 2024-09-19T05:15:24.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.854+0000 7f4f7a000640 1 -- 172.21.15.43:0/4232024616 shutdown_connections 2024-09-19T05:15:24.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.854+0000 7f4f7a000640 1 --2- 172.21.15.43:0/4232024616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f74108f40 0x7f4f74110a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:24.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.854+0000 7f4f7a000640 1 --2- 172.21.15.43:0/4232024616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f741047a0 0x7f4f741087e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:24.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.854+0000 7f4f7a000640 1 --2- 172.21.15.43:0/4232024616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f74103df0 0x7f4f741041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:24.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.854+0000 7f4f7a000640 1 -- 172.21.15.43:0/4232024616 >> 172.21.15.43:0/4232024616 conn(0x7f4f740ff840 msgr2=0x7f4f74101c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:24.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.854+0000 7f4f7a000640 1 -- 172.21.15.43:0/4232024616 shutdown_connections 2024-09-19T05:15:24.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.854+0000 7f4f7a000640 1 -- 172.21.15.43:0/4232024616 wait complete. 2024-09-19T05:15:24.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.855+0000 7f4f7a000640 1 Processor -- start 2024-09-19T05:15:24.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.855+0000 7f4f7a000640 1 -- start start 2024-09-19T05:15:24.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.856+0000 7f4f7a000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f74103df0 0x7f4f7419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:24.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.856+0000 7f4f7a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f74108f40 0x7f4f7419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:24.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.856+0000 7f4f7a000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f7419ed80 0x7f4f7419f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:24.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.856+0000 7f4f7a000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4f74113ff0 con 0x7f4f74108f40 2024-09-19T05:15:24.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.856+0000 7f4f7a000640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4f74113e70 con 0x7f4f74103df0 2024-09-19T05:15:24.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.856+0000 7f4f7a000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4f74114170 con 0x7f4f7419ed80 2024-09-19T05:15:24.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.856+0000 7f4f78c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f74103df0 0x7f4f7419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:24.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.856+0000 7f4f73e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f74108f40 0x7f4f7419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:24.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.856+0000 7f4f78c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f74103df0 0x7f4f7419a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:50348/0 (socket says 172.21.15.43:50348) 2024-09-19T05:15:24.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.856+0000 7f4f78c00640 1 -- 172.21.15.43:0/2573401014 learned_addr learned my addr 172.21.15.43:0/2573401014 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:15:24.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.856+0000 7f4f79600640 1 --2- 172.21.15.43:0/2573401014 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f7419ed80 0x7f4f7419f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:24.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.856+0000 7f4f78c00640 1 -- 172.21.15.43:0/2573401014 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f7419ed80 msgr2=0x7f4f7419f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:24.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.857+0000 7f4f78c00640 1 --2- 172.21.15.43:0/2573401014 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f7419ed80 0x7f4f7419f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:24.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.857+0000 7f4f78c00640 1 -- 172.21.15.43:0/2573401014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f74108f40 msgr2=0x7f4f7419aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:24.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.857+0000 7f4f78c00640 1 --2- 172.21.15.43:0/2573401014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f74108f40 0x7f4f7419aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:24.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.857+0000 7f4f78c00640 1 -- 172.21.15.43:0/2573401014 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f7419f870 con 0x7f4f74103df0 2024-09-19T05:15:24.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.857+0000 7f4f78c00640 1 --2- 172.21.15.43:0/2573401014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f74103df0 0x7f4f7419a4c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4f6800e9f0 tx=0x7f4f6800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:24.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.857+0000 7f4f71600640 1 -- 172.21.15.43:0/2573401014 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6800cd80 con 0x7f4f74103df0 2024-09-19T05:15:24.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.857+0000 7f4f7a000640 1 -- 172.21.15.43:0/2573401014 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f7419fb60 con 0x7f4f74103df0 2024-09-19T05:15:24.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.857+0000 7f4f71600640 1 -- 172.21.15.43:0/2573401014 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f68010910 con 0x7f4f74103df0 2024-09-19T05:15:24.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.857+0000 7f4f71600640 1 -- 172.21.15.43:0/2573401014 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f680188f0 con 0x7f4f74103df0 2024-09-19T05:15:24.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.857+0000 7f4f7a000640 1 -- 172.21.15.43:0/2573401014 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f741a7370 con 0x7f4f74103df0 2024-09-19T05:15:24.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.858+0000 7f4f7a000640 1 -- 172.21.15.43:0/2573401014 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f741041d0 con 0x7f4f74103df0 2024-09-19T05:15:24.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.860+0000 7f4f71600640 1 -- 172.21.15.43:0/2573401014 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4f68010ac0 con 0x7f4f74103df0 2024-09-19T05:15:24.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.861+0000 7f4f71600640 1 --2- 172.21.15.43:0/2573401014 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f4f5c0a3440 0x7f4f5c0a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:24.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.862+0000 7f4f71600640 1 -- 172.21.15.43:0/2573401014 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7f4f680d0370 con 0x7f4f74103df0 2024-09-19T05:15:24.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.862+0000 7f4f73e00640 1 --2- 172.21.15.43:0/2573401014 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f4f5c0a3440 0x7f4f5c0a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:24.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.862+0000 7f4f73e00640 1 --2- 172.21.15.43:0/2573401014 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f4f5c0a3440 0x7f4f5c0a5900 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4f60006010 tx=0x7f4f60000bc0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:24.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:24.863+0000 7f4f71600640 1 -- 172.21.15.43:0/2573401014 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4f6809c690 con 0x7f4f74103df0 2024-09-19T05:15:25.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:24 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/1961521128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:15:25.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:24 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/2432137501' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T05:15:25.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.050+0000 7f4f7a000640 1 -- 172.21.15.43:0/2573401014 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4f7410c0f0 con 0x7f4f74103df0 2024-09-19T05:15:25.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.051+0000 7f4f71600640 1 -- 172.21.15.43:0/2573401014 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) ==== 104+0+11 (secure 0 0 0) 0x7f4f680a0990 con 0x7f4f74103df0 2024-09-19T05:15:25.053 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-09-19T05:15:25.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.053+0000 7f4f7a000640 1 -- 172.21.15.43:0/2573401014 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f4f5c0a3440 msgr2=0x7f4f5c0a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:25.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.053+0000 7f4f7a000640 1 --2- 172.21.15.43:0/2573401014 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f4f5c0a3440 0x7f4f5c0a5900 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4f60006010 tx=0x7f4f60000bc0 comp rx=0 tx=0).stop 2024-09-19T05:15:25.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.053+0000 7f4f7a000640 1 -- 172.21.15.43:0/2573401014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f74103df0 msgr2=0x7f4f7419a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:25.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.053+0000 7f4f7a000640 1 --2- 172.21.15.43:0/2573401014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f74103df0 0x7f4f7419a4c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4f6800e9f0 tx=0x7f4f6800eeb0 comp rx=0 tx=0).stop 2024-09-19T05:15:25.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.054+0000 7f4f7a000640 1 -- 172.21.15.43:0/2573401014 shutdown_connections 2024-09-19T05:15:25.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.054+0000 7f4f7a000640 1 --2- 172.21.15.43:0/2573401014 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f7419ed80 0x7f4f7419f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:25.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.054+0000 7f4f7a000640 1 --2- 172.21.15.43:0/2573401014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f74108f40 0x7f4f7419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:25.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.054+0000 7f4f7a000640 1 --2- 172.21.15.43:0/2573401014 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f4f5c0a3440 0x7f4f5c0a5900 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:25.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.054+0000 7f4f7a000640 1 --2- 172.21.15.43:0/2573401014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f74103df0 0x7f4f7419a4c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:25.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.054+0000 7f4f7a000640 1 -- 172.21.15.43:0/2573401014 >> 172.21.15.43:0/2573401014 conn(0x7f4f740ff840 msgr2=0x7f4f74100b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:25.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.054+0000 7f4f7a000640 1 -- 172.21.15.43:0/2573401014 shutdown_connections 2024-09-19T05:15:25.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.054+0000 7f4f7a000640 1 -- 172.21.15.43:0/2573401014 wait complete. 2024-09-19T05:15:25.072 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-19T05:15:25.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:24 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/1961521128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:15:25.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:24 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/2432137501' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T05:15:25.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.234+0000 7ff61d400640 1 -- 172.21.15.43:0/2307627806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff618107a90 msgr2=0x7ff618107e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:25.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.234+0000 7ff61d400640 1 --2- 172.21.15.43:0/2307627806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff618107a90 0x7ff618107e70 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff608009930 tx=0x7ff60802a420 comp rx=0 tx=0).stop 2024-09-19T05:15:25.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.234+0000 7ff61d400640 1 -- 172.21.15.43:0/2307627806 shutdown_connections 2024-09-19T05:15:25.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.234+0000 7ff61d400640 1 --2- 172.21.15.43:0/2307627806 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff618102420 0x7ff61810e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:25.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.234+0000 7ff61d400640 1 --2- 172.21.15.43:0/2307627806 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff618101a80 0x7ff618101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:25.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.234+0000 7ff61d400640 1 --2- 172.21.15.43:0/2307627806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff618107a90 0x7ff618107e70 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:25.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.234+0000 7ff61d400640 1 -- 172.21.15.43:0/2307627806 >> 172.21.15.43:0/2307627806 conn(0x7ff6180fd7c0 msgr2=0x7ff6180ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:25.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.234+0000 7ff61d400640 1 -- 172.21.15.43:0/2307627806 shutdown_connections 2024-09-19T05:15:25.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.234+0000 7ff61d400640 1 -- 172.21.15.43:0/2307627806 wait complete. 2024-09-19T05:15:25.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.235+0000 7ff61d400640 1 Processor -- start 2024-09-19T05:15:25.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.235+0000 7ff61d400640 1 -- start start 2024-09-19T05:15:25.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.235+0000 7ff61d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff618101a80 0x7ff61819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:25.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.235+0000 7ff61d400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff618102420 0x7ff61819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:25.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.235+0000 7ff61d400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff618107a90 0x7ff61819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:25.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.235+0000 7ff61d400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff618111f20 con 0x7ff618101a80 2024-09-19T05:15:25.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.235+0000 7ff61d400640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff618111da0 con 0x7ff618102420 2024-09-19T05:15:25.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.235+0000 7ff61d400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff6181120a0 con 0x7ff618107a90 2024-09-19T05:15:25.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.235+0000 7ff616800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff618101a80 0x7ff61819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:25.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.235+0000 7ff60de00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff618102420 0x7ff61819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:25.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.235+0000 7ff60de00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff618102420 0x7ff61819ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:50370/0 (socket says 172.21.15.43:50370) 2024-09-19T05:15:25.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.235+0000 7ff60de00640 1 -- 172.21.15.43:0/650246046 learned_addr learned my addr 172.21.15.43:0/650246046 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:15:25.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.236+0000 7ff617200640 1 --2- 172.21.15.43:0/650246046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff618107a90 0x7ff61819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:25.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.236+0000 7ff60de00640 1 -- 172.21.15.43:0/650246046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff618107a90 msgr2=0x7ff61819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:25.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.236+0000 7ff60de00640 1 --2- 172.21.15.43:0/650246046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff618107a90 0x7ff61819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:25.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.236+0000 7ff60de00640 1 -- 172.21.15.43:0/650246046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff618101a80 msgr2=0x7ff61819a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:25.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.236+0000 7ff60de00640 1 --2- 172.21.15.43:0/650246046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff618101a80 0x7ff61819a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:25.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.236+0000 7ff60de00640 1 -- 172.21.15.43:0/650246046 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff61819f620 con 0x7ff618102420 2024-09-19T05:15:25.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.236+0000 7ff60de00640 1 --2- 172.21.15.43:0/650246046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff618102420 0x7ff61819ab70 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff60000ea60 tx=0x7ff60000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:25.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.236+0000 7ff60fe00640 1 -- 172.21.15.43:0/650246046 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60000cde0 con 0x7ff618102420 2024-09-19T05:15:25.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.236+0000 7ff60fe00640 1 -- 172.21.15.43:0/650246046 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6000035f0 con 0x7ff618102420 2024-09-19T05:15:25.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.236+0000 7ff61d400640 1 -- 172.21.15.43:0/650246046 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff61819f910 con 0x7ff618102420 2024-09-19T05:15:25.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.236+0000 7ff60fe00640 1 -- 172.21.15.43:0/650246046 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff600010670 con 0x7ff618102420 2024-09-19T05:15:25.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.236+0000 7ff61d400640 1 -- 172.21.15.43:0/650246046 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6181a71a0 con 0x7ff618102420 2024-09-19T05:15:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.238+0000 7ff61d400640 1 -- 172.21.15.43:0/650246046 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff618101ee0 con 0x7ff618102420 2024-09-19T05:15:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.239+0000 7ff60fe00640 1 -- 172.21.15.43:0/650246046 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ff6000037a0 con 0x7ff618102420 2024-09-19T05:15:25.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.240+0000 7ff60fe00640 1 --2- 172.21.15.43:0/650246046 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff5fc0a33c0 0x7ff5fc0a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:25.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.241+0000 7ff60fe00640 1 -- 172.21.15.43:0/650246046 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7ff6000d00e0 con 0x7ff618102420 2024-09-19T05:15:25.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.241+0000 7ff616800640 1 --2- 172.21.15.43:0/650246046 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff5fc0a33c0 0x7ff5fc0a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:25.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.241+0000 7ff616800640 1 --2- 172.21.15.43:0/650246046 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff5fc0a33c0 0x7ff5fc0a5880 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff608006010 tx=0x7ff608000bc0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:25.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.244+0000 7ff60fe00640 1 -- 172.21.15.43:0/650246046 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff60009ca50 con 0x7ff618102420 2024-09-19T05:15:25.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:24 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/1961521128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:15:25.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:24 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/2432137501' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T05:15:25.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.431+0000 7ff61d400640 1 -- 172.21.15.43:0/650246046 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff6181a74a0 con 0x7ff618102420 2024-09-19T05:15:25.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.432+0000 7ff60fe00640 1 -- 172.21.15.43:0/650246046 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) ==== 100+0+11 (secure 0 0 0) 0x7ff60009c350 con 0x7ff618102420 2024-09-19T05:15:25.434 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-09-19T05:15:25.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.434+0000 7ff61d400640 1 -- 172.21.15.43:0/650246046 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff5fc0a33c0 msgr2=0x7ff5fc0a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:25.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.434+0000 7ff61d400640 1 --2- 172.21.15.43:0/650246046 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff5fc0a33c0 0x7ff5fc0a5880 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff608006010 tx=0x7ff608000bc0 comp rx=0 tx=0).stop 2024-09-19T05:15:25.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.434+0000 7ff61d400640 1 -- 172.21.15.43:0/650246046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff618102420 msgr2=0x7ff61819ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:25.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.434+0000 7ff61d400640 1 --2- 172.21.15.43:0/650246046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff618102420 0x7ff61819ab70 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff60000ea60 tx=0x7ff60000ef20 comp rx=0 tx=0).stop 2024-09-19T05:15:25.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.435+0000 7ff61d400640 1 -- 172.21.15.43:0/650246046 shutdown_connections 2024-09-19T05:15:25.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.435+0000 7ff61d400640 1 --2- 172.21.15.43:0/650246046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff618107a90 0x7ff61819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:25.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.435+0000 7ff61d400640 1 --2- 172.21.15.43:0/650246046 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff5fc0a33c0 0x7ff5fc0a5880 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:25.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.435+0000 7ff61d400640 1 --2- 172.21.15.43:0/650246046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff618102420 0x7ff61819ab70 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:25.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.435+0000 7ff61d400640 1 --2- 172.21.15.43:0/650246046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff618101a80 0x7ff61819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:25.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.435+0000 7ff61d400640 1 -- 172.21.15.43:0/650246046 >> 172.21.15.43:0/650246046 conn(0x7ff6180fd7c0 msgr2=0x7ff61810d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:25.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.435+0000 7ff61d400640 1 -- 172.21.15.43:0/650246046 shutdown_connections 2024-09-19T05:15:25.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.435+0000 7ff61d400640 1 -- 172.21.15.43:0/650246046 wait complete. 2024-09-19T05:15:25.454 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-19T05:15:25.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.610+0000 7fa0e6200640 1 -- 172.21.15.43:0/1608788857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0e0102440 msgr2=0x7fa0e010e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:25.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.610+0000 7fa0e6200640 1 --2- 172.21.15.43:0/1608788857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0e0102440 0x7fa0e010e970 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8009be0 tx=0x7fa0d802a6c0 comp rx=0 tx=0).stop 2024-09-19T05:15:25.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.611+0000 7fa0e6200640 1 -- 172.21.15.43:0/1608788857 shutdown_connections 2024-09-19T05:15:25.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.611+0000 7fa0e6200640 1 --2- 172.21.15.43:0/1608788857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0e0102440 0x7fa0e010e970 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:25.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.611+0000 7fa0e6200640 1 --2- 172.21.15.43:0/1608788857 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0e0101aa0 0x7fa0e0101f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:25.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.611+0000 7fa0e6200640 1 --2- 172.21.15.43:0/1608788857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0e0107ab0 0x7fa0e0107e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:25.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.611+0000 7fa0e6200640 1 -- 172.21.15.43:0/1608788857 >> 172.21.15.43:0/1608788857 conn(0x7fa0e00fd7c0 msgr2=0x7fa0e00ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:25.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.611+0000 7fa0e6200640 1 -- 172.21.15.43:0/1608788857 shutdown_connections 2024-09-19T05:15:25.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.611+0000 7fa0e6200640 1 -- 172.21.15.43:0/1608788857 wait complete. 2024-09-19T05:15:25.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.612+0000 7fa0e6200640 1 Processor -- start 2024-09-19T05:15:25.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.612+0000 7fa0e6200640 1 -- start start 2024-09-19T05:15:25.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.612+0000 7fa0e6200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0e0101aa0 0x7fa0e019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:25.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.612+0000 7fa0e6200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0e0102440 0x7fa0e019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:25.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.612+0000 7fa0e6200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0e0107ab0 0x7fa0e019ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:25.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.612+0000 7fa0e6200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e0111ef0 con 0x7fa0e0107ab0 2024-09-19T05:15:25.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.612+0000 7fa0e6200640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e0111d70 con 0x7fa0e0101aa0 2024-09-19T05:15:25.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.612+0000 7fa0e6200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e0112070 con 0x7fa0e0102440 2024-09-19T05:15:25.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.612+0000 7fa0dfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0e0107ab0 0x7fa0e019ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:25.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.612+0000 7fa0d7e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0e0102440 0x7fa0e019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:25.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.612+0000 7fa0dfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0e0107ab0 0x7fa0e019ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34492/0 (socket says 172.21.15.43:34492) 2024-09-19T05:15:25.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.612+0000 7fa0dfe00640 1 -- 172.21.15.43:0/997785415 learned_addr learned my addr 172.21.15.43:0/997785415 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:15:25.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.612+0000 7fa0df400640 1 --2- 172.21.15.43:0/997785415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0e0101aa0 0x7fa0e019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:25.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.613+0000 7fa0dfe00640 1 -- 172.21.15.43:0/997785415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0e0102440 msgr2=0x7fa0e019aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:25.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.613+0000 7fa0dfe00640 1 --2- 172.21.15.43:0/997785415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0e0102440 0x7fa0e019aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:25.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.613+0000 7fa0dfe00640 1 -- 172.21.15.43:0/997785415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0e0101aa0 msgr2=0x7fa0e019a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:25.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.613+0000 7fa0dfe00640 1 --2- 172.21.15.43:0/997785415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0e0101aa0 0x7fa0e019a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:25.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.613+0000 7fa0dfe00640 1 -- 172.21.15.43:0/997785415 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0e019f560 con 0x7fa0e0107ab0 2024-09-19T05:15:25.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.613+0000 7fa0dfe00640 1 --2- 172.21.15.43:0/997785415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0e0107ab0 0x7fa0e019ee30 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8034040 tx=0x7fa0d8030c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:25.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.613+0000 7fa0dcc00640 1 -- 172.21.15.43:0/997785415 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8040020 con 0x7fa0e0107ab0 2024-09-19T05:15:25.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.613+0000 7fa0dcc00640 1 -- 172.21.15.43:0/997785415 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0d8003e50 con 0x7fa0e0107ab0 2024-09-19T05:15:25.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.613+0000 7fa0dcc00640 1 -- 172.21.15.43:0/997785415 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8036040 con 0x7fa0e0107ab0 2024-09-19T05:15:25.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.613+0000 7fa0e6200640 1 -- 172.21.15.43:0/997785415 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0e019f7f0 con 0x7fa0e0107ab0 2024-09-19T05:15:25.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.614+0000 7fa0e6200640 1 -- 172.21.15.43:0/997785415 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0e01a70e0 con 0x7fa0e0107ab0 2024-09-19T05:15:25.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.614+0000 7fa0e6200640 1 -- 172.21.15.43:0/997785415 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0e0101f00 con 0x7fa0e0107ab0 2024-09-19T05:15:25.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.616+0000 7fa0dcc00640 1 -- 172.21.15.43:0/997785415 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa0d8043020 con 0x7fa0e0107ab0 2024-09-19T05:15:25.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.618+0000 7fa0dcc00640 1 --2- 172.21.15.43:0/997785415 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fa0c40a3440 0x7fa0c40a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:25.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.618+0000 7fa0dcc00640 1 -- 172.21.15.43:0/997785415 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7fa0d80f08c0 con 0x7fa0e0107ab0 2024-09-19T05:15:25.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.618+0000 7fa0df400640 1 --2- 172.21.15.43:0/997785415 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fa0c40a3440 0x7fa0c40a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:25.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.619+0000 7fa0df400640 1 --2- 172.21.15.43:0/997785415 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fa0c40a3440 0x7fa0c40a5900 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0005fd0 tx=0x7fa0d0009260 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:25.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.622+0000 7fa0dcc00640 1 -- 172.21.15.43:0/997785415 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa0d804a020 con 0x7fa0e0107ab0 2024-09-19T05:15:25.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.856+0000 7fa0e6200640 1 -- 172.21.15.43:0/997785415 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fa0e01a7340 con 0x7fa0e0107ab0 2024-09-19T05:15:25.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.866+0000 7fa0dcc00640 1 -- 172.21.15.43:0/997785415 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+742871 (secure 0 0 0) 0x7fa0d80bd230 con 0x7fa0e0107ab0 2024-09-19T05:15:25.879 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:15:25.880 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:15:25.880 INFO:teuthology.orchestra.run.smithi043.stdout: "epoch": 26, 2024-09-19T05:15:25.880 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.880 INFO:teuthology.orchestra.run.smithi043.stdout: "active_gid": 25136, 2024-09-19T05:15:25.880 INFO:teuthology.orchestra.run.smithi043.stdout: "active_name": "x", 2024-09-19T05:15:25.880 INFO:teuthology.orchestra.run.smithi043.stdout: "active_addrs": { 2024-09-19T05:15:25.880 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-19T05:15:25.880 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:25.880 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-19T05:15:25.880 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6800", 2024-09-19T05:15:25.880 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 2120444093 2024-09-19T05:15:25.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.881 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:25.881 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-09-19T05:15:25.881 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6801", 2024-09-19T05:15:25.881 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 2120444093 2024-09-19T05:15:25.881 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:25.881 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-19T05:15:25.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.881 INFO:teuthology.orchestra.run.smithi043.stdout: "active_addr": "172.21.15.43:6801/2120444093", 2024-09-19T05:15:25.881 INFO:teuthology.orchestra.run.smithi043.stdout: "active_change": "2024-09-19T05:04:38.238084+0000", 2024-09-19T05:15:25.881 INFO:teuthology.orchestra.run.smithi043.stdout: "active_mgr_features": 4540701547738038271, 2024-09-19T05:15:25.881 INFO:teuthology.orchestra.run.smithi043.stdout: "available": true, 2024-09-19T05:15:25.881 INFO:teuthology.orchestra.run.smithi043.stdout: "standbys": [ 2024-09-19T05:15:25.881 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:25.881 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 25111, 2024-09-19T05:15:25.882 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "z", 2024-09-19T05:15:25.882 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr_features": 4540701547738038271, 2024-09-19T05:15:25.882 INFO:teuthology.orchestra.run.smithi043.stdout: "available_modules": [ 2024-09-19T05:15:25.882 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:25.882 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "alerts", 2024-09-19T05:15:25.882 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:25.882 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:25.882 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:25.882 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-09-19T05:15:25.882 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-09-19T05:15:25.882 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:25.882 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.882 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:25.882 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-09-19T05:15:25.883 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.883 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.883 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.883 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T05:15:25.883 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.883 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.883 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.883 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:25.883 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:25.883 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.883 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.883 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:25.884 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:25.884 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.884 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.884 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:25.884 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:25.884 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:25.884 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:25.884 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:25.884 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:25.884 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:25.884 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:25.884 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:25.884 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.884 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.885 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.885 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:25.885 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:25.885 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:25.885 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.885 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:25.885 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:25.885 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.885 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.885 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.885 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:25.885 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.886 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.886 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.886 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:25.886 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:25.886 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.886 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.886 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:25.886 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:25.886 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.886 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.886 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:25.886 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:25.886 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:25.887 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:25.887 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:25.887 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:25.887 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:25.887 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:25.887 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:25.887 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.887 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.887 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.887 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:25.887 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:25.887 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:25.887 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.888 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:25.888 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:25.888 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.888 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.888 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.888 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:25.888 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.888 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.888 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.888 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_destination": { 2024-09-19T05:15:25.888 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_destination", 2024-09-19T05:15:25.889 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.889 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.889 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:25.889 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:25.889 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.889 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.889 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.889 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Email address to send alerts to", 2024-09-19T05:15:25.889 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.889 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.889 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.889 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_from_name": { 2024-09-19T05:15:25.889 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_from_name", 2024-09-19T05:15:25.890 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.890 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.890 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:25.890 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "Ceph", 2024-09-19T05:15:25.890 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.890 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.890 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.890 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Email From: name", 2024-09-19T05:15:25.890 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.890 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.890 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.890 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_host": { 2024-09-19T05:15:25.890 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_host", 2024-09-19T05:15:25.891 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.891 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.891 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:25.891 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:25.891 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.891 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.891 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.891 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP server", 2024-09-19T05:15:25.891 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.891 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.891 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.891 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_password": { 2024-09-19T05:15:25.891 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_password", 2024-09-19T05:15:25.891 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.892 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.892 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:25.892 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:25.892 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.892 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.892 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.892 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Password to authenticate with", 2024-09-19T05:15:25.892 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.892 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.892 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.892 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_port": { 2024-09-19T05:15:25.892 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_port", 2024-09-19T05:15:25.892 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:25.893 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.893 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:25.893 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "465", 2024-09-19T05:15:25.893 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.893 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.893 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.893 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP port", 2024-09-19T05:15:25.893 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.893 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.893 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.893 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_sender": { 2024-09-19T05:15:25.893 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_sender", 2024-09-19T05:15:25.893 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.894 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.894 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:25.894 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:25.894 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.894 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.894 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.894 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP envelope sender", 2024-09-19T05:15:25.894 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.894 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.894 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.894 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_ssl": { 2024-09-19T05:15:25.894 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_ssl", 2024-09-19T05:15:25.895 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:25.895 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.895 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:25.895 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:25.895 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.895 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.896 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.896 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T05:15:25.897 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.897 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.897 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.897 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_user": { 2024-09-19T05:15:25.897 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_user", 2024-09-19T05:15:25.897 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.897 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.897 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:25.897 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:25.897 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.897 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.898 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.898 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "User to authenticate as", 2024-09-19T05:15:25.898 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.898 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.898 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.898 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:25.898 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:25.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:25.898 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "balancer", 2024-09-19T05:15:25.898 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:25.898 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:25.898 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:25.898 INFO:teuthology.orchestra.run.smithi043.stdout: "active": { 2024-09-19T05:15:25.899 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "active", 2024-09-19T05:15:25.899 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:25.899 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.899 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:25.899 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:25.899 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.899 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.899 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.899 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T05:15:25.899 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.899 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.899 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.900 INFO:teuthology.orchestra.run.smithi043.stdout: "begin_time": { 2024-09-19T05:15:25.900 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "begin_time", 2024-09-19T05:15:25.900 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.900 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.900 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:25.900 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0000", 2024-09-19T05:15:25.900 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.900 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.900 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.900 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T05:15:25.900 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T05:15:25.900 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.900 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.901 INFO:teuthology.orchestra.run.smithi043.stdout: "begin_weekday": { 2024-09-19T05:15:25.901 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "begin_weekday", 2024-09-19T05:15:25.901 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-09-19T05:15:25.901 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.901 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:25.901 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-09-19T05:15:25.901 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0", 2024-09-19T05:15:25.901 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "6", 2024-09-19T05:15:25.901 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.901 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T05:15:25.901 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T05:15:25.901 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.901 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.902 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_max_iterations": { 2024-09-19T05:15:25.902 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_max_iterations", 2024-09-19T05:15:25.902 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-09-19T05:15:25.902 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.902 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:25.902 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "25", 2024-09-19T05:15:25.902 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-09-19T05:15:25.902 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "250", 2024-09-19T05:15:25.902 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.902 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T05:15:25.902 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.902 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.902 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.903 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_metrics": { 2024-09-19T05:15:25.903 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_metrics", 2024-09-19T05:15:25.903 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.903 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.903 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:25.903 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T05:15:25.903 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.903 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.903 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.903 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T05:15:25.903 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T05:15:25.903 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.903 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.904 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_step": { 2024-09-19T05:15:25.904 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_step", 2024-09-19T05:15:25.904 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-09-19T05:15:25.904 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.904 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:25.904 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0.5", 2024-09-19T05:15:25.904 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0.001", 2024-09-19T05:15:25.904 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "0.999", 2024-09-19T05:15:25.904 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.904 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "aggressiveness of optimization", 2024-09-19T05:15:25.904 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T05:15:25.904 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.904 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.905 INFO:teuthology.orchestra.run.smithi043.stdout: "end_time": { 2024-09-19T05:15:25.905 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "end_time", 2024-09-19T05:15:25.905 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.905 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.905 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:25.905 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "2359", 2024-09-19T05:15:25.905 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.905 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.905 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.905 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T05:15:25.905 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T05:15:25.905 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.905 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.906 INFO:teuthology.orchestra.run.smithi043.stdout: "end_weekday": { 2024-09-19T05:15:25.906 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "end_weekday", 2024-09-19T05:15:25.906 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-09-19T05:15:25.906 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.906 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:25.906 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-09-19T05:15:25.906 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0", 2024-09-19T05:15:25.906 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "6", 2024-09-19T05:15:25.906 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.906 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T05:15:25.906 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T05:15:25.906 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.906 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.907 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:25.907 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:25.907 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.907 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.907 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:25.907 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:25.907 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.907 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.907 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:25.907 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:25.907 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:25.907 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:25.908 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:25.908 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:25.908 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:25.908 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:25.908 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:25.908 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.908 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.908 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.908 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:25.908 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:25.908 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:25.908 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.908 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:25.909 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:25.909 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.909 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.909 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.909 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:25.909 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.909 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.909 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.909 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:25.909 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:25.909 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.910 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.910 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:25.910 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:25.910 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.910 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.910 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:25.910 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:25.910 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:25.910 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:25.910 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:25.910 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:25.910 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:25.910 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:25.910 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:25.911 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.911 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.911 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.911 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:25.911 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:25.911 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:25.911 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.911 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:25.911 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:25.911 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.911 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.911 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.912 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:25.912 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.912 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.912 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.912 INFO:teuthology.orchestra.run.smithi043.stdout: "min_score": { 2024-09-19T05:15:25.912 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "min_score", 2024-09-19T05:15:25.912 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-09-19T05:15:25.912 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.912 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:25.912 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-09-19T05:15:25.912 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.912 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.913 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.913 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T05:15:25.913 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.913 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.913 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.913 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": { 2024-09-19T05:15:25.913 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mode", 2024-09-19T05:15:25.913 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.913 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.913 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:25.913 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "upmap", 2024-09-19T05:15:25.913 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.914 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.914 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:25.914 INFO:teuthology.orchestra.run.smithi043.stdout: "crush-compat", 2024-09-19T05:15:25.914 INFO:teuthology.orchestra.run.smithi043.stdout: "none", 2024-09-19T05:15:25.914 INFO:teuthology.orchestra.run.smithi043.stdout: "read", 2024-09-19T05:15:25.914 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap", 2024-09-19T05:15:25.914 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap-read" 2024-09-19T05:15:25.914 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:25.914 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Balancer mode", 2024-09-19T05:15:25.914 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.914 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.914 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.914 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ids": { 2024-09-19T05:15:25.915 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pool_ids", 2024-09-19T05:15:25.915 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.915 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.915 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:25.915 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:25.915 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.915 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.915 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.915 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T05:15:25.915 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.915 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.915 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.915 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-09-19T05:15:25.916 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-09-19T05:15:25.916 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:25.916 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.916 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:25.916 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-09-19T05:15:25.916 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.916 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.916 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.916 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T05:15:25.916 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.916 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.916 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.917 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap_max_deviation": { 2024-09-19T05:15:25.917 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "upmap_max_deviation", 2024-09-19T05:15:25.917 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:25.917 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.917 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:25.917 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-09-19T05:15:25.917 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-09-19T05:15:25.917 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.917 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.917 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T05:15:25.917 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T05:15:25.917 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.917 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.918 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap_max_optimizations": { 2024-09-19T05:15:25.918 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "upmap_max_optimizations", 2024-09-19T05:15:25.918 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-09-19T05:15:25.918 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.918 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:25.918 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-19T05:15:25.918 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.918 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.918 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.918 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T05:15:25.918 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.918 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.918 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.919 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:25.919 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:25.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:25.919 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cephadm", 2024-09-19T05:15:25.919 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:25.919 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:25.919 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:25.919 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_down_multiplier": { 2024-09-19T05:15:25.919 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_down_multiplier", 2024-09-19T05:15:25.919 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-09-19T05:15:25.919 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.919 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.919 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3.0", 2024-09-19T05:15:25.920 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.920 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.920 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.920 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T05:15:25.920 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.920 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.920 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.920 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_refresh_rate": { 2024-09-19T05:15:25.920 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_refresh_rate", 2024-09-19T05:15:25.920 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:25.920 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.920 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.920 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "20", 2024-09-19T05:15:25.921 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.921 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.921 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.921 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T05:15:25.921 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.921 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.921 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.921 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_starting_port": { 2024-09-19T05:15:25.921 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_starting_port", 2024-09-19T05:15:25.921 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:25.921 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.921 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.921 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "4721", 2024-09-19T05:15:25.922 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.922 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.922 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.922 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T05:15:25.922 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.922 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.922 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.922 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_ptrace": { 2024-09-19T05:15:25.922 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_ptrace", 2024-09-19T05:15:25.922 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:25.922 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.922 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.923 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:25.923 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.923 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.923 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.923 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T05:15:25.923 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T05:15:25.923 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.923 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.923 INFO:teuthology.orchestra.run.smithi043.stdout: "autotune_interval": { 2024-09-19T05:15:25.923 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "autotune_interval", 2024-09-19T05:15:25.923 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:25.923 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.923 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.924 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-09-19T05:15:25.924 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.924 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.924 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.924 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T05:15:25.924 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.924 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.924 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.924 INFO:teuthology.orchestra.run.smithi043.stdout: "autotune_memory_target_ratio": { 2024-09-19T05:15:25.924 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T05:15:25.924 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-09-19T05:15:25.924 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.925 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.925 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0.7", 2024-09-19T05:15:25.925 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.925 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.925 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.925 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T05:15:25.925 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.925 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.925 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.926 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm_log_destination": { 2024-09-19T05:15:25.926 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cephadm_log_destination", 2024-09-19T05:15:25.926 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.926 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.926 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.926 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:25.926 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.926 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.926 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:25.926 INFO:teuthology.orchestra.run.smithi043.stdout: "file", 2024-09-19T05:15:25.926 INFO:teuthology.orchestra.run.smithi043.stdout: "file,syslog", 2024-09-19T05:15:25.926 INFO:teuthology.orchestra.run.smithi043.stdout: "syslog" 2024-09-19T05:15:25.926 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:25.926 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T05:15:25.927 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.927 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.927 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.927 INFO:teuthology.orchestra.run.smithi043.stdout: "cgroups_split": { 2024-09-19T05:15:25.927 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cgroups_split", 2024-09-19T05:15:25.927 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:25.927 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.927 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.927 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:25.927 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.927 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.927 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.927 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T05:15:25.927 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.928 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.928 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.928 INFO:teuthology.orchestra.run.smithi043.stdout: "config_checks_enabled": { 2024-09-19T05:15:25.928 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_checks_enabled", 2024-09-19T05:15:25.928 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:25.928 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.928 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.928 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:25.928 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.928 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.928 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.928 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T05:15:25.929 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.929 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.929 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.929 INFO:teuthology.orchestra.run.smithi043.stdout: "config_dashboard": { 2024-09-19T05:15:25.929 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_dashboard", 2024-09-19T05:15:25.929 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:25.929 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.929 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.929 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:25.929 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.929 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.929 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.929 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T05:15:25.930 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.930 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.930 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.930 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_alertmanager": { 2024-09-19T05:15:25.930 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_alertmanager", 2024-09-19T05:15:25.930 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.930 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.930 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.930 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T05:15:25.930 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.930 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.930 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.930 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-09-19T05:15:25.931 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.931 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.931 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.931 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_base": { 2024-09-19T05:15:25.931 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_base", 2024-09-19T05:15:25.931 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.931 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.931 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:25.931 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T05:15:25.931 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.931 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.931 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.931 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Container image name, without the tag", 2024-09-19T05:15:25.931 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.932 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.932 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.932 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_elasticsearch": { 2024-09-19T05:15:25.932 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_elasticsearch", 2024-09-19T05:15:25.932 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.932 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.932 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.932 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T05:15:25.932 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.932 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.932 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.932 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "elasticsearch container image", 2024-09-19T05:15:25.932 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.933 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.933 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.933 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_grafana": { 2024-09-19T05:15:25.933 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_grafana", 2024-09-19T05:15:25.933 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.933 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.933 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.933 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T05:15:25.933 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.933 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.933 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.933 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-09-19T05:15:25.934 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.934 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.934 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.934 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_haproxy": { 2024-09-19T05:15:25.934 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_haproxy", 2024-09-19T05:15:25.934 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.934 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.934 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.934 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T05:15:25.934 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.934 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.935 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.935 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "HAproxy container image", 2024-09-19T05:15:25.935 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.935 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.935 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.935 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_agent": { 2024-09-19T05:15:25.935 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_agent", 2024-09-19T05:15:25.935 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.935 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.935 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.935 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T05:15:25.935 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.936 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.936 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.936 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger agent container image", 2024-09-19T05:15:25.936 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.936 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.936 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.936 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_collector": { 2024-09-19T05:15:25.936 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_collector", 2024-09-19T05:15:25.936 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.936 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.936 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.937 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T05:15:25.937 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.937 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.937 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.937 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger collector container image", 2024-09-19T05:15:25.937 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.937 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.937 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.937 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_query": { 2024-09-19T05:15:25.937 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_query", 2024-09-19T05:15:25.937 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.937 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.938 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.938 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T05:15:25.938 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.938 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.938 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.938 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger query container image", 2024-09-19T05:15:25.938 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.938 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.938 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.938 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_keepalived": { 2024-09-19T05:15:25.938 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_keepalived", 2024-09-19T05:15:25.938 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.938 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.939 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.939 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T05:15:25.939 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.939 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.939 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.939 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Keepalived container image", 2024-09-19T05:15:25.939 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.939 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.939 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.939 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_loki": { 2024-09-19T05:15:25.939 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_loki", 2024-09-19T05:15:25.939 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.939 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.940 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.940 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T05:15:25.940 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.940 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.940 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.940 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Loki container image", 2024-09-19T05:15:25.940 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.940 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.940 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.940 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_node_exporter": { 2024-09-19T05:15:25.940 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_node_exporter", 2024-09-19T05:15:25.940 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.941 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.941 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.941 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T05:15:25.941 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.941 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.941 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.941 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-09-19T05:15:25.941 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.941 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.941 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.941 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_nvmeof": { 2024-09-19T05:15:25.941 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_nvmeof", 2024-09-19T05:15:25.942 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.942 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.942 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.942 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T05:15:25.942 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.942 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.942 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.942 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Nvme-of container image", 2024-09-19T05:15:25.942 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.942 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.942 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.942 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_prometheus": { 2024-09-19T05:15:25.943 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_prometheus", 2024-09-19T05:15:25.943 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.943 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.943 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.943 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T05:15:25.943 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.943 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.943 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.943 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-09-19T05:15:25.943 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.943 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.943 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.943 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_promtail": { 2024-09-19T05:15:25.944 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_promtail", 2024-09-19T05:15:25.944 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.944 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.944 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.944 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T05:15:25.944 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.944 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.944 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.944 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Promtail container image", 2024-09-19T05:15:25.944 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.944 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.944 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.945 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_samba": { 2024-09-19T05:15:25.945 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_samba", 2024-09-19T05:15:25.945 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.945 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.945 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.945 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T05:15:25.945 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.945 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.945 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.945 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Samba/SMB container image", 2024-09-19T05:15:25.945 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.945 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.945 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.946 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_snmp_gateway": { 2024-09-19T05:15:25.946 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_snmp_gateway", 2024-09-19T05:15:25.946 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.946 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.946 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.946 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T05:15:25.946 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.946 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.946 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.946 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SNMP Gateway container image", 2024-09-19T05:15:25.946 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.946 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.946 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.947 INFO:teuthology.orchestra.run.smithi043.stdout: "container_init": { 2024-09-19T05:15:25.947 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_init", 2024-09-19T05:15:25.947 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:25.947 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.947 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.947 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:25.947 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.947 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.947 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.947 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T05:15:25.947 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.947 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.948 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.948 INFO:teuthology.orchestra.run.smithi043.stdout: "daemon_cache_timeout": { 2024-09-19T05:15:25.948 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "daemon_cache_timeout", 2024-09-19T05:15:25.948 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:25.948 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.948 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.948 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-09-19T05:15:25.948 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.948 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.948 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.948 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T05:15:25.948 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.949 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.949 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.949 INFO:teuthology.orchestra.run.smithi043.stdout: "default_cephadm_command_timeout": { 2024-09-19T05:15:25.949 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T05:15:25.949 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:25.949 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.949 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.949 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "900", 2024-09-19T05:15:25.949 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.949 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.949 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.949 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T05:15:25.949 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.950 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.950 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.950 INFO:teuthology.orchestra.run.smithi043.stdout: "default_registry": { 2024-09-19T05:15:25.950 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "default_registry", 2024-09-19T05:15:25.950 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.950 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.950 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.950 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io", 2024-09-19T05:15:25.950 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.950 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.950 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.950 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T05:15:25.950 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.951 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.951 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.951 INFO:teuthology.orchestra.run.smithi043.stdout: "device_cache_timeout": { 2024-09-19T05:15:25.951 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_cache_timeout", 2024-09-19T05:15:25.951 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:25.951 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.951 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.951 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1800", 2024-09-19T05:15:25.951 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.951 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.951 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.951 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache device inventory", 2024-09-19T05:15:25.951 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.952 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.952 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.952 INFO:teuthology.orchestra.run.smithi043.stdout: "device_enhanced_scan": { 2024-09-19T05:15:25.952 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_enhanced_scan", 2024-09-19T05:15:25.952 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:25.952 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.952 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.952 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:25.952 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.952 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.952 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.953 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T05:15:25.953 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.953 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.953 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.953 INFO:teuthology.orchestra.run.smithi043.stdout: "facts_cache_timeout": { 2024-09-19T05:15:25.953 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "facts_cache_timeout", 2024-09-19T05:15:25.953 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:25.953 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.953 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.953 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-09-19T05:15:25.953 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.953 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.954 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.954 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache host facts data", 2024-09-19T05:15:25.954 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.954 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.954 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.954 INFO:teuthology.orchestra.run.smithi043.stdout: "grafana_dashboards_path": { 2024-09-19T05:15:25.954 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "grafana_dashboards_path", 2024-09-19T05:15:25.954 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.954 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.954 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.954 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T05:15:25.954 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.954 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.955 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.955 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T05:15:25.955 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.955 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.955 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.955 INFO:teuthology.orchestra.run.smithi043.stdout: "host_check_interval": { 2024-09-19T05:15:25.955 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "host_check_interval", 2024-09-19T05:15:25.955 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:25.955 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.955 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.955 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-09-19T05:15:25.956 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.956 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.956 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.956 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to perform a host check", 2024-09-19T05:15:25.956 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.956 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.956 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.956 INFO:teuthology.orchestra.run.smithi043.stdout: "hw_monitoring": { 2024-09-19T05:15:25.956 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "hw_monitoring", 2024-09-19T05:15:25.956 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:25.956 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.956 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.956 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:25.957 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.957 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.957 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.957 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T05:15:25.957 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.957 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.957 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.957 INFO:teuthology.orchestra.run.smithi043.stdout: "inventory_list_all": { 2024-09-19T05:15:25.957 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "inventory_list_all", 2024-09-19T05:15:25.957 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:25.957 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.957 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.958 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:25.958 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.958 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.958 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.958 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T05:15:25.958 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.958 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.958 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.958 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:25.958 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:25.958 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.958 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.958 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:25.959 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:25.959 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.959 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.959 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:25.959 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:25.959 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:25.959 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:25.959 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:25.959 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:25.959 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:25.959 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:25.959 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:25.959 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.959 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.960 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.960 INFO:teuthology.orchestra.run.smithi043.stdout: "log_refresh_metadata": { 2024-09-19T05:15:25.960 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_refresh_metadata", 2024-09-19T05:15:25.960 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:25.960 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.960 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.960 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:25.960 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.960 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.960 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.960 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T05:15:25.960 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.961 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.961 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.961 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:25.961 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:25.961 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:25.961 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.961 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.961 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:25.961 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.961 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.961 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.961 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T05:15:25.962 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.962 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.962 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.962 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:25.962 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:25.962 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.962 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.962 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:25.962 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:25.962 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.962 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.962 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:25.963 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:25.963 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:25.963 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:25.963 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:25.963 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:25.963 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:25.963 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:25.963 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:25.963 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.963 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.963 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.963 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:25.963 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:25.963 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:25.964 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.964 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:25.964 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:25.964 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.964 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.964 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.964 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:25.964 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.964 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.964 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.964 INFO:teuthology.orchestra.run.smithi043.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T05:15:25.964 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T05:15:25.965 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:25.965 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.965 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.965 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:25.965 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.965 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.965 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.965 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T05:15:25.965 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.965 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.965 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.965 INFO:teuthology.orchestra.run.smithi043.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T05:15:25.966 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T05:15:25.966 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.966 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.966 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.966 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "*", 2024-09-19T05:15:25.966 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.966 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.966 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.966 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T05:15:25.966 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.966 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.966 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.966 INFO:teuthology.orchestra.run.smithi043.stdout: "max_count_per_host": { 2024-09-19T05:15:25.967 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_count_per_host", 2024-09-19T05:15:25.967 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:25.967 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.967 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.967 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-19T05:15:25.967 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.967 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.967 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.967 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "max number of daemons per service per host", 2024-09-19T05:15:25.967 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.967 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.967 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.967 INFO:teuthology.orchestra.run.smithi043.stdout: "max_osd_draining_count": { 2024-09-19T05:15:25.968 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_osd_draining_count", 2024-09-19T05:15:25.968 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:25.968 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.968 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.968 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-19T05:15:25.968 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.968 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.968 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.968 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T05:15:25.968 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.968 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.968 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.968 INFO:teuthology.orchestra.run.smithi043.stdout: "migration_current": { 2024-09-19T05:15:25.969 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "migration_current", 2024-09-19T05:15:25.969 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:25.969 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.969 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.969 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:25.969 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.969 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.969 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.969 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "internal - do not modify", 2024-09-19T05:15:25.969 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.969 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.969 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.969 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": { 2024-09-19T05:15:25.970 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mode", 2024-09-19T05:15:25.970 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.970 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.970 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.970 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "root", 2024-09-19T05:15:25.970 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.970 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.970 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:25.970 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm-package", 2024-09-19T05:15:25.970 INFO:teuthology.orchestra.run.smithi043.stdout: "root" 2024-09-19T05:15:25.970 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:25.970 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T05:15:25.970 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.971 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.971 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.971 INFO:teuthology.orchestra.run.smithi043.stdout: "oob_default_addr": { 2024-09-19T05:15:25.971 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "oob_default_addr", 2024-09-19T05:15:25.971 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.971 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.971 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.971 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "169.254.1.1", 2024-09-19T05:15:25.971 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.971 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.971 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.971 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T05:15:25.971 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.971 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.972 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.972 INFO:teuthology.orchestra.run.smithi043.stdout: "prometheus_alerts_path": { 2024-09-19T05:15:25.972 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prometheus_alerts_path", 2024-09-19T05:15:25.972 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.972 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.972 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.972 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T05:15:25.972 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.972 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.972 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.972 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T05:15:25.972 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.973 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.973 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.973 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_insecure": { 2024-09-19T05:15:25.973 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_insecure", 2024-09-19T05:15:25.973 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:25.973 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.973 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.973 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:25.973 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.973 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.973 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.974 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T05:15:25.974 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.974 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.974 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.974 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_password": { 2024-09-19T05:15:25.974 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_password", 2024-09-19T05:15:25.974 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.974 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.974 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.974 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:25.974 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.974 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.975 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.975 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T05:15:25.975 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.975 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.975 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.975 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_url": { 2024-09-19T05:15:25.975 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_url", 2024-09-19T05:15:25.975 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.975 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.975 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.975 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:25.975 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.976 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.976 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.976 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T05:15:25.976 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.976 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.976 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.976 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_username": { 2024-09-19T05:15:25.976 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_username", 2024-09-19T05:15:25.976 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.976 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.976 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.976 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:25.976 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.977 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.977 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.977 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T05:15:25.977 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.977 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.977 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.977 INFO:teuthology.orchestra.run.smithi043.stdout: "secure_monitoring_stack": { 2024-09-19T05:15:25.977 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "secure_monitoring_stack", 2024-09-19T05:15:25.977 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:25.977 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.977 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.977 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:25.977 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.978 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.978 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.978 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T05:15:25.978 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.978 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.978 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.978 INFO:teuthology.orchestra.run.smithi043.stdout: "service_discovery_port": { 2024-09-19T05:15:25.978 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "service_discovery_port", 2024-09-19T05:15:25.978 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:25.978 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.978 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.978 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8765", 2024-09-19T05:15:25.978 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.979 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.979 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.979 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "cephadm service discovery port", 2024-09-19T05:15:25.979 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.979 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.979 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.979 INFO:teuthology.orchestra.run.smithi043.stdout: "ssh_config_file": { 2024-09-19T05:15:25.979 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssh_config_file", 2024-09-19T05:15:25.979 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.979 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.980 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.980 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:25.980 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.980 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.980 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.980 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T05:15:25.980 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.980 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.980 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.980 INFO:teuthology.orchestra.run.smithi043.stdout: "use_agent": { 2024-09-19T05:15:25.980 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "use_agent", 2024-09-19T05:15:25.980 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:25.981 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.981 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.981 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:25.981 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.981 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.981 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.981 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T05:15:25.981 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.981 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.981 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.981 INFO:teuthology.orchestra.run.smithi043.stdout: "use_repo_digest": { 2024-09-19T05:15:25.981 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "use_repo_digest", 2024-09-19T05:15:25.982 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:25.982 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.982 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.982 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:25.982 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.982 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.982 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.982 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T05:15:25.982 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.982 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.982 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.982 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_failed_host_check": { 2024-09-19T05:15:25.982 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_failed_host_check", 2024-09-19T05:15:25.983 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:25.983 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.983 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.983 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:25.983 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.983 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.983 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.983 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T05:15:25.983 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.983 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.983 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.983 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_stray_daemons": { 2024-09-19T05:15:25.983 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_stray_daemons", 2024-09-19T05:15:25.984 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:25.984 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.984 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.984 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:25.984 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.984 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.984 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.984 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T05:15:25.984 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.984 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.985 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.985 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_stray_hosts": { 2024-09-19T05:15:25.985 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_stray_hosts", 2024-09-19T05:15:25.985 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:25.985 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.985 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.985 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:25.985 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.985 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.985 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.985 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T05:15:25.985 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.986 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.986 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.986 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:25.986 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:25.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:25.986 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crash", 2024-09-19T05:15:25.986 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:25.986 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:25.986 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:25.986 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:25.986 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:25.986 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.987 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.987 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:25.987 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:25.987 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.987 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.988 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:25.988 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:25.988 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:25.988 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:25.988 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:25.988 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:25.988 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:25.988 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:25.988 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:25.988 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.988 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.989 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.989 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:25.989 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:25.989 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:25.989 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.989 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:25.989 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:25.989 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.989 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.989 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.989 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:25.989 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.990 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.990 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.990 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:25.990 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:25.990 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.990 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.990 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:25.990 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:25.990 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.990 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.990 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:25.990 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:25.991 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:25.991 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:25.991 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:25.991 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:25.991 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:25.991 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:25.991 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:25.991 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.991 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.991 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.991 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:25.991 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:25.991 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:25.992 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.992 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:25.992 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:25.992 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.992 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.992 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.992 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:25.992 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.992 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.992 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.992 INFO:teuthology.orchestra.run.smithi043.stdout: "retain_interval": { 2024-09-19T05:15:25.992 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "retain_interval", 2024-09-19T05:15:25.992 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:25.993 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.993 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:25.993 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "31536000", 2024-09-19T05:15:25.993 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.993 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.993 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.993 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T05:15:25.993 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.993 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.993 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.993 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_recent_interval": { 2024-09-19T05:15:25.993 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_recent_interval", 2024-09-19T05:15:25.994 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:25.994 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.994 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:25.994 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1209600", 2024-09-19T05:15:25.994 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.994 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.994 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.994 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T05:15:25.994 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.994 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.994 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.994 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:25.994 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:25.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:25.995 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "dashboard", 2024-09-19T05:15:25.995 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:25.995 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:25.995 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:25.995 INFO:teuthology.orchestra.run.smithi043.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T05:15:25.995 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T05:15:25.995 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:25.995 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.995 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.995 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-19T05:15:25.995 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.995 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.995 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.996 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:25.996 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:25.996 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:25.996 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:25.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:25.996 INFO:teuthology.orchestra.run.smithi043.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T05:15:25.996 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T05:15:25.996 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:25.996 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:25.996 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:25.996 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:25.996 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:25.996 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:25.996 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:25.997 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:25.997 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.000 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.000 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.001 INFO:teuthology.orchestra.run.smithi043.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T05:15:26.001 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T05:15:26.001 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.001 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.001 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.001 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.001 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.001 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.001 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.001 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.001 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.001 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.001 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.002 INFO:teuthology.orchestra.run.smithi043.stdout: "AUDIT_API_ENABLED": { 2024-09-19T05:15:26.002 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T05:15:26.002 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.002 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.002 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.002 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.002 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.002 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.002 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.002 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.002 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.002 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.002 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.003 INFO:teuthology.orchestra.run.smithi043.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T05:15:26.003 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T05:15:26.003 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.003 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.003 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.003 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.003 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.003 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.003 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.003 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.003 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.003 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.003 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.004 INFO:teuthology.orchestra.run.smithi043.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T05:15:26.004 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T05:15:26.004 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.004 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.004 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.004 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.004 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.004 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.004 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.004 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.004 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.004 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.004 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.005 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T05:15:26.005 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T05:15:26.005 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.005 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.005 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.005 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.005 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.005 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.005 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.005 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.005 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.005 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.006 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.006 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T05:15:26.006 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T05:15:26.006 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.006 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.006 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.006 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.006 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.006 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.006 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.006 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.006 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.006 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.007 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.007 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T05:15:26.007 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T05:15:26.007 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.007 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.007 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.007 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.007 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.007 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.007 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.007 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.007 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.007 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.008 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.008 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T05:15:26.008 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T05:15:26.008 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.008 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.008 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.008 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.008 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.008 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.008 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.008 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.008 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.008 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.009 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.009 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T05:15:26.009 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T05:15:26.009 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.009 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.009 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.009 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.009 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.009 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.009 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.009 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.009 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.009 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.009 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.010 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T05:15:26.010 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T05:15:26.010 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.010 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.010 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.010 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.010 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.010 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.010 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.010 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.010 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.010 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.010 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.011 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T05:15:26.011 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T05:15:26.011 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.011 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.011 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.011 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.011 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.011 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.011 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.011 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.011 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.011 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.011 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.012 INFO:teuthology.orchestra.run.smithi043.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T05:15:26.012 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T05:15:26.012 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.012 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.012 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.012 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.012 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.012 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.012 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.012 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.012 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.012 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.012 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.013 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T05:15:26.013 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T05:15:26.013 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.013 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.013 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.013 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-09-19T05:15:26.013 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.013 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.013 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.013 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.013 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.013 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.013 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.014 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T05:15:26.014 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T05:15:26.014 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.014 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.014 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.014 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.014 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.014 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.014 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.014 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.014 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.014 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.014 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.015 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_URL": { 2024-09-19T05:15:26.015 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_URL", 2024-09-19T05:15:26.015 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.015 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.015 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.015 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.015 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.015 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.015 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.015 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.015 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.015 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.015 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.016 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T05:15:26.016 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T05:15:26.016 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.016 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.016 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.016 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-09-19T05:15:26.016 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.016 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.016 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.016 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.016 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.016 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.016 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.017 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T05:15:26.017 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T05:15:26.017 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.017 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.017 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.017 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.017 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.017 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.017 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.017 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.017 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.017 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.017 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.018 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T05:15:26.018 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T05:15:26.018 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.018 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.018 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.018 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.018 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.018 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.018 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.018 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.018 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.018 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.019 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.019 INFO:teuthology.orchestra.run.smithi043.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T05:15:26.019 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T05:15:26.019 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.019 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.019 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.019 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.019 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.019 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.019 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.019 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.019 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.019 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.019 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.020 INFO:teuthology.orchestra.run.smithi043.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T05:15:26.020 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T05:15:26.020 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.020 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.020 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.020 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.020 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.020 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.020 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.020 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.020 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.020 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.020 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.021 INFO:teuthology.orchestra.run.smithi043.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T05:15:26.021 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T05:15:26.021 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.021 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.021 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.021 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.021 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.021 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.021 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.021 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.021 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.021 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.022 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.022 INFO:teuthology.orchestra.run.smithi043.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T05:15:26.022 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T05:15:26.022 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.022 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.022 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.022 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.022 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.022 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.022 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.022 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.022 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.022 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.023 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.023 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T05:15:26.023 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T05:15:26.023 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.023 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.023 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.023 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.023 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.023 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.023 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.023 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.023 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.023 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.024 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.024 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T05:15:26.024 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T05:15:26.024 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.024 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.024 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.024 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.024 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.024 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.024 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.024 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.024 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.024 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.025 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.025 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T05:15:26.025 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T05:15:26.025 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.025 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.025 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.025 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.025 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.025 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.025 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.025 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.025 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.026 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.026 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.026 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T05:15:26.026 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T05:15:26.026 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.026 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.026 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.026 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.026 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.026 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.026 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.026 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.026 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.026 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.027 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.027 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T05:15:26.027 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T05:15:26.027 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.027 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.027 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.027 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.027 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.027 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.027 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.027 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.027 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.027 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.028 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.028 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T05:15:26.028 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T05:15:26.028 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.028 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.028 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.028 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.028 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.028 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.028 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.028 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.028 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.028 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.029 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.029 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T05:15:26.029 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T05:15:26.029 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.029 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.029 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.029 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.029 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.029 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.029 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.029 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.029 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.029 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.030 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.030 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T05:15:26.030 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T05:15:26.030 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.030 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.030 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.030 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.030 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.030 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.030 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.030 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.030 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.030 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.030 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.031 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T05:15:26.031 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T05:15:26.031 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.031 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.031 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.031 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T05:15:26.031 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.031 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.031 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.031 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.031 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.031 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.032 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.032 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T05:15:26.032 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T05:15:26.032 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.032 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.032 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.032 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-19T05:15:26.032 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.032 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.032 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.032 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.032 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.032 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.033 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.033 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T05:15:26.033 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T05:15:26.033 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.033 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.033 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.033 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8", 2024-09-19T05:15:26.033 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.033 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.033 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.033 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.033 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.033 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.034 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.034 INFO:teuthology.orchestra.run.smithi043.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T05:15:26.034 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T05:15:26.034 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.034 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.034 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.034 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "45", 2024-09-19T05:15:26.034 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.034 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.034 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.034 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.034 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.034 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.035 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.035 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T05:15:26.035 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T05:15:26.035 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.035 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.035 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.035 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.035 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.035 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.035 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.035 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.035 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.035 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.036 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.036 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T05:15:26.036 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T05:15:26.036 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.036 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.036 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.036 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-09-19T05:15:26.036 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.036 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.036 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.036 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.036 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.036 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.037 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.037 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T05:15:26.037 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T05:15:26.037 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.037 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.037 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.037 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.037 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.037 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.037 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.037 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.037 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.037 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.038 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.038 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T05:15:26.038 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T05:15:26.038 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.038 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.038 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.038 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.038 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.038 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.038 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.038 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.038 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.038 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.039 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.039 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T05:15:26.039 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T05:15:26.039 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.039 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.039 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.039 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-09-19T05:15:26.039 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.039 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.039 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.039 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.039 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.039 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.040 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.040 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T05:15:26.040 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T05:15:26.040 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.040 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.040 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.040 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-19T05:15:26.040 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.040 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.040 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.040 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.040 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.040 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.041 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.041 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T05:15:26.041 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T05:15:26.041 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.041 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.041 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.041 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-09-19T05:15:26.041 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.041 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.041 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.041 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.041 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.041 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.041 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.042 INFO:teuthology.orchestra.run.smithi043.stdout: "cross_origin_url": { 2024-09-19T05:15:26.042 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cross_origin_url", 2024-09-19T05:15:26.042 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.042 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.042 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.042 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.042 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.042 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.042 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.042 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.042 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.042 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.042 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.043 INFO:teuthology.orchestra.run.smithi043.stdout: "crt_file": { 2024-09-19T05:15:26.043 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crt_file", 2024-09-19T05:15:26.043 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.043 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.043 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.043 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.043 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.043 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.043 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.043 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.043 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.043 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.043 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.044 INFO:teuthology.orchestra.run.smithi043.stdout: "debug": { 2024-09-19T05:15:26.044 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "debug", 2024-09-19T05:15:26.044 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.044 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.044 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.044 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.044 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.044 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.044 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.044 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable/disable debug options", 2024-09-19T05:15:26.044 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.044 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.044 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.045 INFO:teuthology.orchestra.run.smithi043.stdout: "jwt_token_ttl": { 2024-09-19T05:15:26.045 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "jwt_token_ttl", 2024-09-19T05:15:26.045 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.045 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.045 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.045 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "28800", 2024-09-19T05:15:26.045 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.045 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.045 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.045 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.045 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.045 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.046 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.046 INFO:teuthology.orchestra.run.smithi043.stdout: "key_file": { 2024-09-19T05:15:26.046 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "key_file", 2024-09-19T05:15:26.046 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.046 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.046 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.046 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.046 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.046 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.046 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.046 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.046 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.046 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.046 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.047 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.047 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.047 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.047 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.047 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.047 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.047 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.047 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.047 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.047 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.047 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.047 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.047 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.048 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.048 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.048 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.048 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.048 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.048 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.048 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.048 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.048 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.048 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.048 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.048 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.048 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.049 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.049 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.049 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.049 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.049 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.049 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.049 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.049 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.049 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.049 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.049 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.049 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.049 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.050 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.050 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.050 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.050 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.050 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.050 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.050 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.050 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.050 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.050 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.050 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.050 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.050 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.050 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.051 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.051 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.051 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.051 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.051 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.051 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.051 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.051 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.051 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.051 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.051 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.051 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.051 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.052 INFO:teuthology.orchestra.run.smithi043.stdout: "motd": { 2024-09-19T05:15:26.052 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "motd", 2024-09-19T05:15:26.052 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.052 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.052 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.052 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.052 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.052 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.052 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.052 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "The message of the day", 2024-09-19T05:15:26.052 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.052 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.052 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.053 INFO:teuthology.orchestra.run.smithi043.stdout: "redirect_resolve_ip_addr": { 2024-09-19T05:15:26.053 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T05:15:26.053 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.053 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.053 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.053 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.053 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.053 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.053 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.053 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.053 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.053 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.053 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.053 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-09-19T05:15:26.054 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-09-19T05:15:26.054 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.054 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.054 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.054 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "::", 2024-09-19T05:15:26.054 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.054 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.054 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.054 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.054 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.054 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.054 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.054 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-09-19T05:15:26.055 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-09-19T05:15:26.055 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.055 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.055 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.055 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8080", 2024-09-19T05:15:26.055 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.055 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.055 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.055 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.055 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.055 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.055 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.055 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl": { 2024-09-19T05:15:26.056 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl", 2024-09-19T05:15:26.056 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.056 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.056 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.056 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.056 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.056 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.056 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.056 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.056 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.056 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.056 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.056 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl_server_port": { 2024-09-19T05:15:26.057 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl_server_port", 2024-09-19T05:15:26.057 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.057 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.057 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.057 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8443", 2024-09-19T05:15:26.057 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.057 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.057 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.057 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.057 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.057 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.057 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.058 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_behaviour": { 2024-09-19T05:15:26.058 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_behaviour", 2024-09-19T05:15:26.058 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.058 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.058 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.058 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "redirect", 2024-09-19T05:15:26.058 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.058 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.058 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.058 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.058 INFO:teuthology.orchestra.run.smithi043.stdout: "redirect" 2024-09-19T05:15:26.058 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.059 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.059 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.059 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.059 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.059 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_error_status_code": { 2024-09-19T05:15:26.059 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_error_status_code", 2024-09-19T05:15:26.059 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.059 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.059 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.059 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "500", 2024-09-19T05:15:26.059 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "400", 2024-09-19T05:15:26.059 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "599", 2024-09-19T05:15:26.060 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.060 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.060 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.060 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.060 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.060 INFO:teuthology.orchestra.run.smithi043.stdout: "url_prefix": { 2024-09-19T05:15:26.060 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "url_prefix", 2024-09-19T05:15:26.060 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.060 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.060 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.060 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.060 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.060 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.061 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.061 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.061 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.061 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.061 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.061 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.061 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.061 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "devicehealth", 2024-09-19T05:15:26.061 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.061 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.061 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.061 INFO:teuthology.orchestra.run.smithi043.stdout: "enable_monitoring": { 2024-09-19T05:15:26.062 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enable_monitoring", 2024-09-19T05:15:26.062 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.062 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.062 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.062 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.062 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.062 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.062 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.062 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "monitor device health metrics", 2024-09-19T05:15:26.062 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.062 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.062 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.062 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.063 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.063 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.063 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.063 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.063 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.063 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.063 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.063 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.063 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.063 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.063 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.063 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.063 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.063 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.063 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.064 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.064 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.064 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.064 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.064 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.064 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.064 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.064 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.064 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.064 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.064 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.064 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.064 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.065 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.065 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.065 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.065 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.065 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.065 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.065 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.065 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.065 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.065 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.065 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.065 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.065 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.066 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.066 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.066 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.066 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.066 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.066 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.066 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.066 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.066 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.066 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.066 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.066 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.066 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.066 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.067 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.067 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.067 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.067 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.067 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.067 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.067 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.067 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.067 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.067 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.067 INFO:teuthology.orchestra.run.smithi043.stdout: "mark_out_threshold": { 2024-09-19T05:15:26.067 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mark_out_threshold", 2024-09-19T05:15:26.067 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:26.067 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.068 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.068 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "2419200", 2024-09-19T05:15:26.068 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.068 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.068 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.068 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T05:15:26.068 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.068 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.068 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.068 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": { 2024-09-19T05:15:26.068 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pool_name", 2024-09-19T05:15:26.068 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.068 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.069 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.069 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "device_health_metrics", 2024-09-19T05:15:26.069 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.069 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.069 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.069 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T05:15:26.069 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.069 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.069 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.069 INFO:teuthology.orchestra.run.smithi043.stdout: "retention_period": { 2024-09-19T05:15:26.069 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "retention_period", 2024-09-19T05:15:26.069 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:26.069 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.069 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.070 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15552000", 2024-09-19T05:15:26.070 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.070 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.070 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.070 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long to retain device health metrics", 2024-09-19T05:15:26.070 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.070 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.070 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.070 INFO:teuthology.orchestra.run.smithi043.stdout: "scrape_frequency": { 2024-09-19T05:15:26.070 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "scrape_frequency", 2024-09-19T05:15:26.070 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:26.070 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.070 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.071 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "86400", 2024-09-19T05:15:26.071 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.071 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.071 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.071 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T05:15:26.071 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.071 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.071 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.071 INFO:teuthology.orchestra.run.smithi043.stdout: "self_heal": { 2024-09-19T05:15:26.071 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "self_heal", 2024-09-19T05:15:26.071 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.071 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.072 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.072 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.072 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.072 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.072 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.072 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T05:15:26.072 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.072 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.072 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.072 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-09-19T05:15:26.072 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-09-19T05:15:26.072 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:26.072 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.073 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.073 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-09-19T05:15:26.073 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.073 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.073 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.073 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T05:15:26.073 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.073 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.073 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.073 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_threshold": { 2024-09-19T05:15:26.073 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_threshold", 2024-09-19T05:15:26.073 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:26.073 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.073 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.074 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "7257600", 2024-09-19T05:15:26.074 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.074 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.074 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.074 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T05:15:26.074 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.074 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.074 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.074 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.074 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.074 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "diskprediction_local", 2024-09-19T05:15:26.074 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.075 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.075 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.075 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.075 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.075 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.075 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.075 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.075 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.075 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.075 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.075 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.075 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.075 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.075 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.075 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.076 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.076 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.076 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.076 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.076 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.076 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.076 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.076 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.076 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.076 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.076 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.076 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.076 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.077 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.077 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.077 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.077 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.077 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.077 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.077 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.077 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.077 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.077 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.077 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.077 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.077 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.078 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.078 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.078 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.078 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.078 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.078 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.078 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.078 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.078 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.078 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.078 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.078 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.078 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.078 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.079 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.079 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.079 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.079 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.079 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.079 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.079 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.079 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.079 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.079 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.079 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.079 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.079 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.080 INFO:teuthology.orchestra.run.smithi043.stdout: "predict_interval": { 2024-09-19T05:15:26.080 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "predict_interval", 2024-09-19T05:15:26.080 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.080 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.080 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.080 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "86400", 2024-09-19T05:15:26.080 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.080 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.080 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.080 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.080 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.080 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.080 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.081 INFO:teuthology.orchestra.run.smithi043.stdout: "predictor_model": { 2024-09-19T05:15:26.081 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "predictor_model", 2024-09-19T05:15:26.081 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.081 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.081 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.081 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "prophetstor", 2024-09-19T05:15:26.081 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.081 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.081 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.081 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.081 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.081 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.081 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.081 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-09-19T05:15:26.082 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-09-19T05:15:26.082 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.082 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.082 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.082 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-09-19T05:15:26.082 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.082 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.082 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.082 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.082 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.082 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.082 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.082 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.082 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.083 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "influx", 2024-09-19T05:15:26.083 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": false, 2024-09-19T05:15:26.083 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "influxdb python module not found", 2024-09-19T05:15:26.083 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.083 INFO:teuthology.orchestra.run.smithi043.stdout: "batch_size": { 2024-09-19T05:15:26.083 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "batch_size", 2024-09-19T05:15:26.083 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.083 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.083 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.083 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5000", 2024-09-19T05:15:26.083 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.083 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.083 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.084 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T05:15:26.084 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.084 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.084 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.084 INFO:teuthology.orchestra.run.smithi043.stdout: "database": { 2024-09-19T05:15:26.084 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "database", 2024-09-19T05:15:26.084 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.084 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.084 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.085 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "ceph", 2024-09-19T05:15:26.085 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.085 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.085 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.085 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T05:15:26.085 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.085 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.085 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.085 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": { 2024-09-19T05:15:26.085 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "hostname", 2024-09-19T05:15:26.085 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.086 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.086 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.086 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.086 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.086 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.086 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.086 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB server hostname", 2024-09-19T05:15:26.086 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.086 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.086 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.086 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-09-19T05:15:26.086 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-09-19T05:15:26.086 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:26.087 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.087 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.087 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "30", 2024-09-19T05:15:26.087 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "5", 2024-09-19T05:15:26.087 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.087 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.087 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T05:15:26.087 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.087 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.087 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.088 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.088 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.088 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.088 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.088 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.088 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.089 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.089 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.089 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.089 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.089 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.089 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.089 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.089 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.089 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.089 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.089 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.089 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.089 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.089 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.090 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.090 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.090 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.090 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.090 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.090 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.090 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.090 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.090 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.090 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.090 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.090 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.090 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.091 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.091 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.091 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.091 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.091 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.091 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.091 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.091 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.091 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.091 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.091 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.091 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.091 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.092 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.092 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.092 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.092 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.092 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.092 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.092 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.092 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.092 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.092 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.092 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.092 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.092 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.092 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.093 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.093 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.093 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.093 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.093 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.093 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.093 INFO:teuthology.orchestra.run.smithi043.stdout: "password": { 2024-09-19T05:15:26.093 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "password", 2024-09-19T05:15:26.093 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.093 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.093 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.093 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.093 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.094 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.094 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.094 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "password of InfluxDB server user", 2024-09-19T05:15:26.094 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.094 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.094 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.094 INFO:teuthology.orchestra.run.smithi043.stdout: "port": { 2024-09-19T05:15:26.094 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "port", 2024-09-19T05:15:26.094 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.094 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.094 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.094 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8086", 2024-09-19T05:15:26.094 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.095 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.095 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.095 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB server port", 2024-09-19T05:15:26.095 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.096 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.096 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.096 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl": { 2024-09-19T05:15:26.096 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl", 2024-09-19T05:15:26.096 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.096 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.096 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.096 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "false", 2024-09-19T05:15:26.096 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.096 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.096 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.096 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T05:15:26.097 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.097 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.097 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.097 INFO:teuthology.orchestra.run.smithi043.stdout: "threads": { 2024-09-19T05:15:26.097 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "threads", 2024-09-19T05:15:26.097 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.097 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.097 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.098 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-09-19T05:15:26.098 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-09-19T05:15:26.098 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "32", 2024-09-19T05:15:26.098 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.098 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T05:15:26.098 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.098 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.098 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.098 INFO:teuthology.orchestra.run.smithi043.stdout: "username": { 2024-09-19T05:15:26.098 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "username", 2024-09-19T05:15:26.098 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.098 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.098 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.099 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.099 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.099 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.099 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.099 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "username of InfluxDB server user", 2024-09-19T05:15:26.099 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.099 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.099 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.099 INFO:teuthology.orchestra.run.smithi043.stdout: "verify_ssl": { 2024-09-19T05:15:26.100 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "verify_ssl", 2024-09-19T05:15:26.100 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.100 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.100 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.100 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "true", 2024-09-19T05:15:26.100 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.100 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.100 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.100 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T05:15:26.100 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.100 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.100 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.100 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.101 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.101 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "insights", 2024-09-19T05:15:26.101 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.101 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.101 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.101 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.101 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.101 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.102 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.102 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.102 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.102 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.102 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.102 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.102 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.102 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.102 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.102 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.102 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.102 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.102 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.103 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.103 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.103 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.103 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.103 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.103 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.103 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.103 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.103 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.103 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.103 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.103 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.103 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.104 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.104 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.104 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.104 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.104 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.104 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.104 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.104 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.104 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.104 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.104 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.104 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.105 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.105 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.105 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.105 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.105 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.105 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.105 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.105 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.106 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.106 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.106 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.106 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.106 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.106 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.106 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.106 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.106 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.107 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.107 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.107 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.107 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.107 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.107 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.107 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.107 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.107 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.107 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.108 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "iostat", 2024-09-19T05:15:26.108 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.108 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.108 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.108 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.108 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.108 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.108 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.109 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.109 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.109 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.109 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.109 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.109 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.109 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.109 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.109 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.109 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.110 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.110 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.110 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.110 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.110 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.110 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.110 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.110 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.110 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.110 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.110 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.110 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.110 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.110 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.111 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.111 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.111 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.111 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.111 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.111 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.111 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.111 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.111 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.111 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.111 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.111 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.111 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.112 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.112 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.112 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.112 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.112 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.112 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.112 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.112 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.112 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.112 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.112 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.112 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.112 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.113 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.113 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.113 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.113 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.113 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.113 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.113 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.113 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.113 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.113 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.113 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.113 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.113 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.113 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.114 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "k8sevents", 2024-09-19T05:15:26.114 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.114 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.114 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.114 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_event_retention_days": { 2024-09-19T05:15:26.114 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ceph_event_retention_days", 2024-09-19T05:15:26.114 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.114 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.114 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.114 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "7", 2024-09-19T05:15:26.114 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.114 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.114 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.115 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T05:15:26.115 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.115 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.115 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.115 INFO:teuthology.orchestra.run.smithi043.stdout: "config_check_secs": { 2024-09-19T05:15:26.115 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_check_secs", 2024-09-19T05:15:26.115 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.115 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.115 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.115 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-19T05:15:26.115 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "10", 2024-09-19T05:15:26.115 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.116 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.116 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T05:15:26.116 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.116 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.116 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.116 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.116 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.116 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.116 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.116 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.116 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.116 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.116 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.117 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.117 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.117 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.117 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.117 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.117 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.117 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.117 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.117 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.117 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.117 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.117 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.117 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.117 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.118 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.118 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.118 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.118 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.118 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.118 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.118 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.118 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.118 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.118 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.118 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.118 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.119 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.119 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.119 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.119 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.119 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.119 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.119 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.119 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.119 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.119 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.119 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.119 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.119 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.119 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.119 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.120 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.120 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.120 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.120 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.120 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.120 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.120 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.120 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.120 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.120 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.120 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.120 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.120 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.121 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.121 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.121 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.121 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.121 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.121 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.121 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "localpool", 2024-09-19T05:15:26.121 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.121 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.121 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.121 INFO:teuthology.orchestra.run.smithi043.stdout: "failure_domain": { 2024-09-19T05:15:26.121 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "failure_domain", 2024-09-19T05:15:26.121 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.122 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.122 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.122 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "host", 2024-09-19T05:15:26.122 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.122 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.122 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.122 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "failure domain for any created local pool", 2024-09-19T05:15:26.122 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T05:15:26.122 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.122 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.122 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.122 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.122 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.123 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.123 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.123 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.123 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.123 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.123 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.123 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.123 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.123 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.123 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.123 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.123 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.123 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.123 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.123 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.124 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.124 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.124 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.124 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.124 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.124 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.124 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.124 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.124 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.124 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.124 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.124 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.124 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.125 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.125 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.125 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.125 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.125 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.125 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.125 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.125 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.125 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.125 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.125 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.125 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.125 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.126 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.126 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.126 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.126 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.126 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.126 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.126 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.126 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.126 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.126 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.126 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.126 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.126 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.126 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.127 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.127 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.127 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.127 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.127 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.127 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.127 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.127 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.127 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": { 2024-09-19T05:15:26.127 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "min_size", 2024-09-19T05:15:26.127 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.127 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.127 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.127 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.128 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.128 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.128 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.128 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default min_size for any created local pool", 2024-09-19T05:15:26.128 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T05:15:26.128 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.128 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.128 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rep": { 2024-09-19T05:15:26.128 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "num_rep", 2024-09-19T05:15:26.128 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.128 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.128 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.129 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3", 2024-09-19T05:15:26.129 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.129 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.129 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.129 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default replica count for any created local pool", 2024-09-19T05:15:26.129 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.129 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.129 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.129 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": { 2024-09-19T05:15:26.129 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pg_num", 2024-09-19T05:15:26.129 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.129 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.130 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.130 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "128", 2024-09-19T05:15:26.130 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.130 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.130 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.130 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T05:15:26.130 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.130 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.130 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.130 INFO:teuthology.orchestra.run.smithi043.stdout: "prefix": { 2024-09-19T05:15:26.130 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prefix", 2024-09-19T05:15:26.130 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.131 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.131 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.131 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.131 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.131 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.131 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.131 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "name prefix for any created local pool", 2024-09-19T05:15:26.131 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.131 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.131 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.131 INFO:teuthology.orchestra.run.smithi043.stdout: "subtree": { 2024-09-19T05:15:26.131 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "subtree", 2024-09-19T05:15:26.131 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.131 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.132 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.132 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "rack", 2024-09-19T05:15:26.132 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.132 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.132 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.132 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T05:15:26.132 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T05:15:26.132 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.132 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.133 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.133 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.133 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mds_autoscaler", 2024-09-19T05:15:26.133 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.133 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.133 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.133 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.133 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.133 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.133 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.133 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.133 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.134 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.134 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.134 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.134 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.134 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.134 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.134 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.134 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.134 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.134 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.135 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.135 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.135 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.135 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.135 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.135 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.135 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.135 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.135 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.135 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.135 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.136 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.136 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.136 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.136 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.136 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.136 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.136 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.136 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.137 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.137 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.137 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.137 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.137 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.137 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.137 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.137 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.137 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.137 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.137 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.137 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.137 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.138 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.138 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.138 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.138 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.138 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.138 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.138 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.138 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.138 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.139 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.139 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.139 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.139 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.139 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.139 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.139 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.139 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.139 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.139 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.140 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.140 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mirroring", 2024-09-19T05:15:26.140 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.140 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.140 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.140 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.141 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.141 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.141 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.141 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.141 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.141 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.141 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.141 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.141 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.142 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.142 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.142 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.142 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.142 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.142 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.142 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.142 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.142 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.142 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.142 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.143 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.143 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.143 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.143 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.143 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.143 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.143 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.143 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.143 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.143 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.143 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.143 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.143 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.144 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.144 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.144 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.144 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.144 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.144 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.144 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.144 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.144 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.144 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.144 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.144 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.144 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.145 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.145 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.145 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.145 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.145 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.145 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.145 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.145 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.145 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.145 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.145 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.145 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.146 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.146 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.146 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.146 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.146 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.146 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.146 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.146 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.146 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.146 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "nfs", 2024-09-19T05:15:26.146 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.146 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.147 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.147 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.147 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.147 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.147 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.147 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.147 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.147 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.147 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.147 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.147 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.147 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.147 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.147 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.148 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.148 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.148 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.148 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.148 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.148 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.148 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.148 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.148 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.148 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.148 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.148 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.149 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.149 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.149 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.149 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.149 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.149 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.149 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.149 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.149 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.149 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.149 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.149 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.150 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.150 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.150 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.150 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.150 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.150 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.150 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.150 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.150 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.150 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.150 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.150 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.150 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.150 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.150 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.151 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.151 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.151 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.151 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.151 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.151 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.151 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.151 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.151 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.151 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.151 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.151 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.151 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.151 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.152 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.152 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.152 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "orchestrator", 2024-09-19T05:15:26.152 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.152 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.152 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.152 INFO:teuthology.orchestra.run.smithi043.stdout: "fail_fs": { 2024-09-19T05:15:26.152 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "fail_fs", 2024-09-19T05:15:26.152 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.152 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.152 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.153 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.153 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.153 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.153 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.153 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T05:15:26.153 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.153 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.153 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.153 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.153 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.153 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.153 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.153 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.154 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.154 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.154 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.154 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.154 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.154 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.154 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.154 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.154 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.154 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.154 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.154 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.154 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.154 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.154 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.155 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.155 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.155 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.155 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.155 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.155 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.155 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.155 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.155 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.155 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.155 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.155 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.155 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.156 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.156 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.156 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.156 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.156 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.156 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.156 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.156 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.156 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.156 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.156 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.156 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.156 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.156 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.156 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.157 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.157 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.157 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.157 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.157 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.157 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.157 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.157 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.157 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.157 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.157 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.158 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.158 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.158 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.158 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.158 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.158 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.158 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.158 INFO:teuthology.orchestra.run.smithi043.stdout: "orchestrator": { 2024-09-19T05:15:26.158 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "orchestrator", 2024-09-19T05:15:26.158 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.158 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.158 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.158 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.158 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.159 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.159 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.159 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm", 2024-09-19T05:15:26.159 INFO:teuthology.orchestra.run.smithi043.stdout: "rook", 2024-09-19T05:15:26.159 INFO:teuthology.orchestra.run.smithi043.stdout: "test_orchestrator" 2024-09-19T05:15:26.159 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.159 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Orchestrator backend", 2024-09-19T05:15:26.159 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.159 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.159 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.159 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.159 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.160 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "osd_perf_query", 2024-09-19T05:15:26.160 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.160 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.160 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.160 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.160 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.160 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.160 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.160 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.160 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.160 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.160 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.160 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.161 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.161 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.161 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.161 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.161 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.161 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.161 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.161 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.161 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.161 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.161 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.161 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.161 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.162 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.162 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.162 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.162 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.162 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.162 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.162 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.162 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.162 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.162 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.163 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.163 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.163 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.163 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.163 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.163 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.163 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.163 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.163 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.163 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.163 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.163 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.163 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.164 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.164 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.164 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.164 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.164 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.164 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.164 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.164 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.164 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.164 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.164 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.164 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.165 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.165 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.165 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.165 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.165 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.165 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.165 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.165 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.165 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.165 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.165 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.166 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "osd_support", 2024-09-19T05:15:26.166 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.166 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.166 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.166 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.166 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.166 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.166 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.166 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.166 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.167 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.167 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.167 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.167 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.167 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.167 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.167 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.167 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.167 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.167 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.167 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.168 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.168 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.168 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.168 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.168 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.168 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.168 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.168 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.168 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.168 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.168 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.168 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.169 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.169 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.169 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.169 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.169 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.169 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.169 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.169 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.169 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.169 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.169 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.170 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.170 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.170 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.170 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.170 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.170 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.170 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.170 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.170 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.170 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.170 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.171 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.171 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.171 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.171 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.171 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.171 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.171 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.171 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.171 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.171 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.171 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.171 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.172 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.172 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.172 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.172 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.172 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.172 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pg_autoscaler", 2024-09-19T05:15:26.172 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.172 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.172 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.172 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.172 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.172 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.173 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.173 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.173 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.173 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.173 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.173 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.173 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.173 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.173 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.173 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.173 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.173 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.173 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.173 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.173 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.174 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.174 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.174 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.174 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.174 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.174 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.174 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.174 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.174 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.174 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.175 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.175 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.175 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.175 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.175 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.175 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.175 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.175 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.176 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.176 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.176 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.176 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.176 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.176 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.176 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.176 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.177 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.177 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.177 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.177 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.177 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.177 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.177 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.177 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.177 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.178 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.178 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.178 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.178 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.178 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.178 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.178 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.178 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.178 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.178 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.178 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.178 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.178 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.179 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-09-19T05:15:26.179 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-09-19T05:15:26.179 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:26.179 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.179 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.179 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-09-19T05:15:26.179 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.179 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.179 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.179 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.179 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.179 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.179 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.180 INFO:teuthology.orchestra.run.smithi043.stdout: "threshold": { 2024-09-19T05:15:26.180 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "threshold", 2024-09-19T05:15:26.180 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-09-19T05:15:26.180 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.180 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.180 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3.0", 2024-09-19T05:15:26.180 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1.0", 2024-09-19T05:15:26.180 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.180 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.180 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "scaling threshold", 2024-09-19T05:15:26.180 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T05:15:26.180 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.180 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.181 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.181 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.181 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "progress", 2024-09-19T05:15:26.181 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.181 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.181 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.181 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_pg_recovery_event": { 2024-09-19T05:15:26.181 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_pg_recovery_event", 2024-09-19T05:15:26.181 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.181 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.181 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.181 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.182 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.182 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.182 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.182 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T05:15:26.182 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.182 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.182 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.182 INFO:teuthology.orchestra.run.smithi043.stdout: "enabled": { 2024-09-19T05:15:26.182 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enabled", 2024-09-19T05:15:26.182 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.182 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.182 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.182 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.182 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.183 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.183 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.183 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.183 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.183 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.183 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.183 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.183 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.183 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.183 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.183 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.183 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.183 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.183 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.184 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.184 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.184 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.184 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.184 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.184 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.184 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.184 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.184 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.184 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.184 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.184 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.184 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.185 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.185 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.185 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.185 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.185 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.185 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.185 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.185 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.185 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.185 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.185 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.185 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.185 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.186 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.186 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.186 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.186 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.186 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.186 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.186 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.186 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.186 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.186 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.186 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.186 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.186 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.186 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.187 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.187 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.187 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.187 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.187 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.187 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.187 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.187 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.187 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.187 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.187 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.187 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.187 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.188 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.188 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.188 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.188 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.188 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.188 INFO:teuthology.orchestra.run.smithi043.stdout: "max_completed_events": { 2024-09-19T05:15:26.188 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_completed_events", 2024-09-19T05:15:26.188 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.188 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.188 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.188 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "50", 2024-09-19T05:15:26.188 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.188 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.189 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.189 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "number of past completed events to remember", 2024-09-19T05:15:26.189 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.189 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.189 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.189 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-09-19T05:15:26.189 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-09-19T05:15:26.189 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:26.189 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.189 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.189 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-09-19T05:15:26.189 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.189 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.189 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.190 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long the module is going to sleep", 2024-09-19T05:15:26.190 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.190 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.190 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.190 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.190 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.190 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prometheus", 2024-09-19T05:15:26.190 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.190 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.190 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.190 INFO:teuthology.orchestra.run.smithi043.stdout: "cache": { 2024-09-19T05:15:26.190 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cache", 2024-09-19T05:15:26.191 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.191 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.191 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.191 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.191 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.191 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.191 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.191 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.191 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.191 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.191 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.191 INFO:teuthology.orchestra.run.smithi043.stdout: "exclude_perf_counters": { 2024-09-19T05:15:26.191 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "exclude_perf_counters", 2024-09-19T05:15:26.192 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.192 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.192 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.192 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.192 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.192 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.192 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.192 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T05:15:26.192 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T05:15:26.192 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.192 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.192 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.192 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.193 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.193 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.193 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.193 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.193 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.193 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.193 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.193 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.193 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.193 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.193 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.193 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.193 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.193 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.193 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.194 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.194 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.194 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.194 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.194 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.194 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.194 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.194 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.194 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.194 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.194 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.194 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.194 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.195 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.195 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.195 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.195 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.195 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.195 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.195 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.195 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.195 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.195 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.195 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.195 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.195 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.195 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.196 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.196 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.196 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.196 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.196 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.196 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.196 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.196 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.196 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.196 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.196 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.196 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.196 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.197 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.197 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.197 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.197 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.197 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.197 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.197 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.197 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.197 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.197 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_stats_pools": { 2024-09-19T05:15:26.197 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_stats_pools", 2024-09-19T05:15:26.197 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.197 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.197 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.198 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.198 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.198 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.198 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.198 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.198 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.198 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.198 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.199 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T05:15:26.199 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T05:15:26.199 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.199 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.199 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.199 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "300", 2024-09-19T05:15:26.199 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.199 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.199 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.199 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.199 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.199 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.199 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.199 INFO:teuthology.orchestra.run.smithi043.stdout: "scrape_interval": { 2024-09-19T05:15:26.200 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "scrape_interval", 2024-09-19T05:15:26.200 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-09-19T05:15:26.200 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.200 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.200 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15.0", 2024-09-19T05:15:26.200 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.200 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.200 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.200 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.200 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.200 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.201 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.201 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-09-19T05:15:26.201 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-09-19T05:15:26.201 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.201 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.206 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.206 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "::", 2024-09-19T05:15:26.206 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.206 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.206 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.206 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T05:15:26.206 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.206 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.206 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.206 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-09-19T05:15:26.206 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-09-19T05:15:26.206 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.207 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.207 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.207 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "9283", 2024-09-19T05:15:26.207 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.207 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.207 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.207 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T05:15:26.207 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.207 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.207 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.208 INFO:teuthology.orchestra.run.smithi043.stdout: "stale_cache_strategy": { 2024-09-19T05:15:26.208 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "stale_cache_strategy", 2024-09-19T05:15:26.208 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.208 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.208 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.208 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "log", 2024-09-19T05:15:26.208 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.208 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.208 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.208 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.209 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.209 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.209 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.209 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_behaviour": { 2024-09-19T05:15:26.209 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_behaviour", 2024-09-19T05:15:26.209 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.209 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.209 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.209 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "default", 2024-09-19T05:15:26.210 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.210 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.210 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.210 INFO:teuthology.orchestra.run.smithi043.stdout: "default", 2024-09-19T05:15:26.210 INFO:teuthology.orchestra.run.smithi043.stdout: "error" 2024-09-19T05:15:26.210 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.210 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.210 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.210 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.210 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.211 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_error_status_code": { 2024-09-19T05:15:26.211 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_error_status_code", 2024-09-19T05:15:26.211 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.211 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.211 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.211 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "500", 2024-09-19T05:15:26.211 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "400", 2024-09-19T05:15:26.211 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "599", 2024-09-19T05:15:26.211 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.211 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.212 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.212 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.212 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.212 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.212 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.212 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_support", 2024-09-19T05:15:26.212 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.212 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.213 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.213 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.213 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.213 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.213 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.213 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.213 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.213 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.213 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.213 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.213 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.214 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.214 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.214 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.214 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.214 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.214 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.214 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.214 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.214 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.214 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.214 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.214 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.214 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.214 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.215 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.215 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.215 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.215 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.215 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.215 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.215 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.215 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.215 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.215 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.215 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.215 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.215 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.216 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.216 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.216 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.216 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.216 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.216 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.216 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.216 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.216 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.216 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.216 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.216 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.216 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.216 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.216 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.217 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.217 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.217 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.217 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.217 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.217 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.217 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.217 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.217 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.217 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.217 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.217 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.217 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.217 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.218 INFO:teuthology.orchestra.run.smithi043.stdout: "max_concurrent_snap_create": { 2024-09-19T05:15:26.218 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_concurrent_snap_create", 2024-09-19T05:15:26.218 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.218 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.218 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.218 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-19T05:15:26.218 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.218 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.218 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.218 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.218 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.218 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.218 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.218 INFO:teuthology.orchestra.run.smithi043.stdout: "mirror_snapshot_schedule": { 2024-09-19T05:15:26.219 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T05:15:26.219 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.219 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.219 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.219 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.219 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.219 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.219 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.219 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.219 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.219 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.219 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.219 INFO:teuthology.orchestra.run.smithi043.stdout: "trash_purge_schedule": { 2024-09-19T05:15:26.219 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "trash_purge_schedule", 2024-09-19T05:15:26.219 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.219 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.220 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.220 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.220 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.220 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.220 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.220 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.220 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.220 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.220 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.220 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.220 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.220 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "restful", 2024-09-19T05:15:26.220 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.221 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.221 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.221 INFO:teuthology.orchestra.run.smithi043.stdout: "enable_auth": { 2024-09-19T05:15:26.221 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enable_auth", 2024-09-19T05:15:26.221 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.221 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.221 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.221 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.221 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.221 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.221 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.221 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.221 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.221 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.221 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.222 INFO:teuthology.orchestra.run.smithi043.stdout: "key_file": { 2024-09-19T05:15:26.222 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "key_file", 2024-09-19T05:15:26.222 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.222 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.222 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.222 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.222 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.222 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.222 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.222 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.222 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.223 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.223 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.223 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.223 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.223 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.223 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.223 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.223 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.223 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.223 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.223 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.224 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.224 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.224 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.224 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.224 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.224 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.224 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.224 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.224 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.224 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.224 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.224 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.224 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.224 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.225 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.225 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.225 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.225 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.225 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.225 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.225 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.225 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.225 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.225 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.225 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.225 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.225 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.225 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.226 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.226 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.226 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.226 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.226 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.226 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.226 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.226 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.226 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.226 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.226 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.226 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.226 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.226 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.226 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.227 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.227 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.227 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.227 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.227 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.227 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.227 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.227 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.227 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.227 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.227 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.227 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.227 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.227 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.228 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-09-19T05:15:26.228 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-09-19T05:15:26.228 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.228 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.228 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.228 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.228 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.228 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.228 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.228 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.228 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.228 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.228 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.228 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-09-19T05:15:26.228 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-09-19T05:15:26.229 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.229 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.229 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.229 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.229 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.229 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.229 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.229 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.229 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.229 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.229 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.229 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.229 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.230 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rgw", 2024-09-19T05:15:26.230 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.230 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.230 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.230 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.230 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.230 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.230 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.230 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.230 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.230 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.230 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.230 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.230 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.231 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.231 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.231 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.231 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.231 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.231 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.231 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.231 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.231 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.231 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.231 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.231 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.231 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.231 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.231 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.232 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.232 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.232 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.232 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.232 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.232 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.232 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.232 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.232 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.232 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.232 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.232 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.232 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.232 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.233 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.233 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.233 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.233 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.233 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.233 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.233 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.233 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.233 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.234 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.234 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.234 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.234 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.234 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.234 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.234 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.234 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.234 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.234 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.234 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.234 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.234 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.235 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.235 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.235 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.235 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.235 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.235 INFO:teuthology.orchestra.run.smithi043.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T05:15:26.235 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T05:15:26.235 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.235 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.235 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.235 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-09-19T05:15:26.235 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.236 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.236 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.236 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T05:15:26.236 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.236 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.236 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.236 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.236 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.236 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rook", 2024-09-19T05:15:26.237 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.237 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.237 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.237 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.237 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.237 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.237 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.237 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.237 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.237 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.237 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.237 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.237 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.238 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.238 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.238 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.238 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.238 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.238 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.238 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.238 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.238 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.238 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.238 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.239 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.239 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.239 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.239 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.239 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.239 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.239 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.239 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.239 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.239 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.240 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.240 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.240 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.240 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.240 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.240 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.240 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.240 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.240 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.240 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.240 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.240 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.240 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.241 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.241 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.241 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.241 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.241 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.241 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.241 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.241 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.241 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.242 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.242 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.242 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.242 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.242 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.242 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.242 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.242 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.242 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.242 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.243 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.243 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.243 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.243 INFO:teuthology.orchestra.run.smithi043.stdout: "storage_class": { 2024-09-19T05:15:26.243 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "storage_class", 2024-09-19T05:15:26.243 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.243 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.243 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.243 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "local", 2024-09-19T05:15:26.243 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.244 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.244 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.244 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T05:15:26.244 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.244 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.244 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.244 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.244 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.244 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "selftest", 2024-09-19T05:15:26.244 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.245 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.245 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.245 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.245 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.245 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.245 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.245 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.245 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.245 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.245 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.245 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.245 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.245 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.246 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.246 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.246 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.246 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.246 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.246 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.246 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.246 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.246 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.246 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.246 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.246 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.246 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.246 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.247 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.247 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.247 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.247 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.247 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.247 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.247 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.247 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.248 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.248 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.248 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.248 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.248 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.248 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.248 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.248 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.248 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.248 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.249 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.249 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.249 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.249 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.249 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.249 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.249 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.249 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.249 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.249 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.249 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.249 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.249 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.249 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.250 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.250 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.250 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.250 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.250 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.250 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.250 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.250 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.250 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.250 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.250 INFO:teuthology.orchestra.run.smithi043.stdout: "roption1": { 2024-09-19T05:15:26.250 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "roption1", 2024-09-19T05:15:26.251 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.251 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.251 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.251 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.251 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.251 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.251 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.251 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.251 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.251 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.251 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.251 INFO:teuthology.orchestra.run.smithi043.stdout: "roption2": { 2024-09-19T05:15:26.252 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "roption2", 2024-09-19T05:15:26.252 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.252 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.252 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.252 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "xyz", 2024-09-19T05:15:26.252 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.252 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.252 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.252 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.252 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.252 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.252 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.252 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption1": { 2024-09-19T05:15:26.253 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption1", 2024-09-19T05:15:26.253 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.253 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.253 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.253 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.253 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.253 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.253 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.253 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.253 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.253 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.253 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.254 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption2": { 2024-09-19T05:15:26.254 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption2", 2024-09-19T05:15:26.254 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.254 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.254 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.254 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.254 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.254 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.254 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.254 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.254 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.254 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.254 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.255 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption3": { 2024-09-19T05:15:26.255 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption3", 2024-09-19T05:15:26.255 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-09-19T05:15:26.255 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.255 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.255 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.255 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.255 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.255 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.255 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.255 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.255 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.255 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.256 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption4": { 2024-09-19T05:15:26.256 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption4", 2024-09-19T05:15:26.256 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.256 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.256 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.256 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.256 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.256 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.256 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.256 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.256 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.256 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.256 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.257 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption5": { 2024-09-19T05:15:26.257 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption5", 2024-09-19T05:15:26.257 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.257 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.257 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.257 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.257 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.257 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.257 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.257 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.257 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.257 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.258 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.258 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption6": { 2024-09-19T05:15:26.258 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption6", 2024-09-19T05:15:26.258 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.258 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.258 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.258 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.258 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.258 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.258 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.258 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.258 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.259 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.259 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.259 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption7": { 2024-09-19T05:15:26.259 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption7", 2024-09-19T05:15:26.259 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.259 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.259 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.259 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.259 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-09-19T05:15:26.259 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "42", 2024-09-19T05:15:26.259 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.259 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.259 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.260 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.260 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.260 INFO:teuthology.orchestra.run.smithi043.stdout: "testkey": { 2024-09-19T05:15:26.260 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testkey", 2024-09-19T05:15:26.260 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.260 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.260 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.260 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.260 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.260 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.260 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.260 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.260 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.261 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.261 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.261 INFO:teuthology.orchestra.run.smithi043.stdout: "testlkey": { 2024-09-19T05:15:26.261 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testlkey", 2024-09-19T05:15:26.261 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.261 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.261 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.261 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.261 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.261 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.261 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.261 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.261 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.261 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.262 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.262 INFO:teuthology.orchestra.run.smithi043.stdout: "testnewline": { 2024-09-19T05:15:26.262 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testnewline", 2024-09-19T05:15:26.262 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.262 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.262 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.262 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.262 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.262 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.262 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.262 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.262 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.262 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.262 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.263 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.263 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.263 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snap_schedule", 2024-09-19T05:15:26.263 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.263 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.263 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.263 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_m_granularity": { 2024-09-19T05:15:26.263 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_m_granularity", 2024-09-19T05:15:26.263 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.263 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.263 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.263 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.263 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.264 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.264 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.264 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T05:15:26.264 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.264 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.264 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.264 INFO:teuthology.orchestra.run.smithi043.stdout: "dump_on_update": { 2024-09-19T05:15:26.264 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "dump_on_update", 2024-09-19T05:15:26.264 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.264 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.264 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.264 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.264 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.265 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.265 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.265 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "dump database to debug log on update", 2024-09-19T05:15:26.265 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.265 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.265 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.265 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.265 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.265 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.265 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.265 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.265 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.266 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.266 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.266 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.266 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.266 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.266 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.266 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.266 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.266 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.266 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.266 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.266 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.266 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.266 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.267 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.267 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.267 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.267 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.267 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.267 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.267 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.267 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.267 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.267 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.267 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.267 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.268 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.268 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.268 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.268 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.268 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.268 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.268 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.268 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.268 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.268 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.268 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.269 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.269 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.269 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.269 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.269 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.269 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.269 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.269 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.269 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.269 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.270 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.270 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.270 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.270 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.270 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.270 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.270 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.270 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.270 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.270 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.270 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.270 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.270 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.271 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.271 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.271 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "stats", 2024-09-19T05:15:26.271 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.271 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.271 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.271 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.271 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.272 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.272 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.272 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.272 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.272 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.272 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.272 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.272 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.272 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.272 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.272 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.272 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.273 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.273 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.273 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.273 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.273 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.273 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.273 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.273 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.273 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.273 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.274 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.274 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.274 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.274 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.274 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.274 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.274 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.274 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.274 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.274 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.275 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.275 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.275 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.275 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.275 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.275 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.275 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.275 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.275 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.275 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.275 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.275 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.276 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.276 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.276 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.276 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.276 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.276 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.276 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.276 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.276 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.277 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.277 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.277 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.277 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.277 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.277 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.277 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.277 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.277 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.277 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.277 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.278 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.278 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.278 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "status", 2024-09-19T05:15:26.278 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.278 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.278 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.278 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.278 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.278 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.279 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.279 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.279 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.279 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.279 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.279 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.279 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.279 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.279 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.279 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.280 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.280 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.280 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.280 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.280 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.280 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.280 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.280 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.280 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.281 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.281 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.281 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.281 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.281 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.281 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.281 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.281 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.281 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.281 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.281 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.281 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.282 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.282 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.282 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.282 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.282 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.282 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.282 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.282 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.282 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.282 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.282 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.282 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.282 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.282 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.283 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.283 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.283 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.283 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.283 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.283 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.283 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.283 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.283 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.283 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.283 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.283 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.283 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.284 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.284 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.284 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.284 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.284 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.284 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.284 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.284 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "telegraf", 2024-09-19T05:15:26.285 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.285 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.285 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.285 INFO:teuthology.orchestra.run.smithi043.stdout: "address": { 2024-09-19T05:15:26.285 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "address", 2024-09-19T05:15:26.285 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.285 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.285 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.285 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T05:15:26.285 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.285 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.285 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.285 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.286 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.286 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.286 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.286 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-09-19T05:15:26.286 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-09-19T05:15:26.286 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:26.286 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.286 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.286 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15", 2024-09-19T05:15:26.286 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.286 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.286 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.286 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.286 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.287 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.287 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.287 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.287 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.287 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.287 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.287 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.287 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.287 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.287 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.287 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.287 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.287 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.287 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.288 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.288 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.288 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.288 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.288 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.288 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.288 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.288 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.288 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.288 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.288 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.288 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.288 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.289 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.289 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.289 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.289 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.289 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.289 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.289 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.289 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.289 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.289 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.289 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.289 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.289 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.290 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.290 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.290 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.290 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.290 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.290 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.290 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.290 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.290 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.290 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.290 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.290 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.290 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.290 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.290 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.291 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.291 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.291 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.291 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.291 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.291 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.291 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.291 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.291 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.291 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.291 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.291 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.291 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.291 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.292 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.292 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "telemetry", 2024-09-19T05:15:26.292 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.292 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.292 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.292 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_basic": { 2024-09-19T05:15:26.292 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_basic", 2024-09-19T05:15:26.292 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.292 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.292 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.292 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.292 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.293 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.293 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.293 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T05:15:26.293 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.293 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.293 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.293 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_crash": { 2024-09-19T05:15:26.293 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_crash", 2024-09-19T05:15:26.293 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.293 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.293 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.293 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.293 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.294 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.294 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.294 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T05:15:26.294 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.294 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.294 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.294 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_device": { 2024-09-19T05:15:26.294 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_device", 2024-09-19T05:15:26.294 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.294 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.294 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.294 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.294 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.295 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.295 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.295 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T05:15:26.295 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.295 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.295 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.295 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_ident": { 2024-09-19T05:15:26.295 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_ident", 2024-09-19T05:15:26.295 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.295 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.295 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.295 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.295 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.296 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.296 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.296 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T05:15:26.296 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.296 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.296 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.296 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_perf": { 2024-09-19T05:15:26.296 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_perf", 2024-09-19T05:15:26.296 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.296 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.296 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.296 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.296 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.297 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.297 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.297 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T05:15:26.297 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.297 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.297 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.297 INFO:teuthology.orchestra.run.smithi043.stdout: "contact": { 2024-09-19T05:15:26.297 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "contact", 2024-09-19T05:15:26.297 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.297 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.297 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.297 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.298 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.298 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.298 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.298 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.298 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.298 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.298 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.298 INFO:teuthology.orchestra.run.smithi043.stdout: "description": { 2024-09-19T05:15:26.298 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "description", 2024-09-19T05:15:26.298 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.298 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.298 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.298 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.298 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.299 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.299 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.299 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.299 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.299 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.299 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.299 INFO:teuthology.orchestra.run.smithi043.stdout: "device_url": { 2024-09-19T05:15:26.299 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_url", 2024-09-19T05:15:26.299 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.299 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.299 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.299 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T05:15:26.299 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.299 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.300 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.300 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.300 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.300 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.300 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.300 INFO:teuthology.orchestra.run.smithi043.stdout: "enabled": { 2024-09-19T05:15:26.300 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enabled", 2024-09-19T05:15:26.300 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.300 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.300 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.300 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.300 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.300 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.301 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.301 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.301 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.301 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.301 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.301 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-09-19T05:15:26.301 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-09-19T05:15:26.301 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.302 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.302 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.302 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "24", 2024-09-19T05:15:26.302 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "8", 2024-09-19T05:15:26.302 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.302 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.302 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.302 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.302 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.302 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.302 INFO:teuthology.orchestra.run.smithi043.stdout: "last_opt_revision": { 2024-09-19T05:15:26.302 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "last_opt_revision", 2024-09-19T05:15:26.302 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.303 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.303 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.303 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1", 2024-09-19T05:15:26.303 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.303 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.303 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.303 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.303 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.303 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.304 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.304 INFO:teuthology.orchestra.run.smithi043.stdout: "leaderboard": { 2024-09-19T05:15:26.304 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "leaderboard", 2024-09-19T05:15:26.304 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.304 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.304 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.304 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.304 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.304 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.304 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.304 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.304 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.304 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.305 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.305 INFO:teuthology.orchestra.run.smithi043.stdout: "leaderboard_description": { 2024-09-19T05:15:26.305 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "leaderboard_description", 2024-09-19T05:15:26.305 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.305 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.305 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.305 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.305 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.305 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.305 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.306 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.306 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.306 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.306 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.306 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.306 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.306 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.306 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.306 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.306 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.306 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.306 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.307 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.307 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.307 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.307 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.307 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.307 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.307 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.307 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.307 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.307 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.308 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.308 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.308 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.308 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.308 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.308 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.308 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.308 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.308 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.308 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.308 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.308 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.309 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.309 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.309 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.309 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.309 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.309 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.309 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.309 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.309 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.309 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.310 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.310 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.310 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.310 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.310 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.310 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.310 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.310 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.310 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.310 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.310 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.311 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.311 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.311 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.311 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.311 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.311 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.311 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.311 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.312 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.312 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.312 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.312 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.312 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.312 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.312 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.312 INFO:teuthology.orchestra.run.smithi043.stdout: "organization": { 2024-09-19T05:15:26.312 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "organization", 2024-09-19T05:15:26.313 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.313 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.313 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.313 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.313 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.313 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.313 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.313 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.313 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.313 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.313 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.314 INFO:teuthology.orchestra.run.smithi043.stdout: "proxy": { 2024-09-19T05:15:26.314 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "proxy", 2024-09-19T05:15:26.314 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.314 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.314 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.314 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.314 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.314 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.314 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.314 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.314 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.314 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.314 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.314 INFO:teuthology.orchestra.run.smithi043.stdout: "url": { 2024-09-19T05:15:26.315 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "url", 2024-09-19T05:15:26.315 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.315 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.315 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.315 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T05:15:26.315 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.315 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.315 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.316 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.316 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.316 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.316 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.316 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.316 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.316 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "test_orchestrator", 2024-09-19T05:15:26.316 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.316 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.316 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.316 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.316 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.317 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.317 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.317 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.317 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.317 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.317 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.317 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.317 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.317 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.317 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.317 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.317 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.317 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.317 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.318 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.318 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.318 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.318 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.318 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.318 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.318 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.318 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.318 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.318 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.318 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.318 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.318 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.319 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.319 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.319 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.319 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.319 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.319 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.319 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.319 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.319 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.319 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.319 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.319 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.319 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.319 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.320 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.320 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.320 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.320 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.320 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.320 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.320 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.320 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.320 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.320 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.320 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.320 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.320 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.320 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.321 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.321 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.321 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.321 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.321 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.321 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.321 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.321 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.321 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.321 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.321 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.321 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "volumes", 2024-09-19T05:15:26.322 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.322 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.322 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.322 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.322 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.322 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.322 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.322 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.322 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.322 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.322 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.322 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.322 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.322 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.322 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.323 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.323 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.323 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.323 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.323 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.323 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.323 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.323 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.323 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.323 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.323 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.323 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.323 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.324 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.324 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.324 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.324 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.324 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.324 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.324 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.324 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.324 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.324 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.324 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.324 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.324 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.325 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.325 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.325 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.325 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.325 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.325 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.325 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.325 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.325 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.325 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.325 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.325 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.325 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.325 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.325 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.326 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.326 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.326 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.326 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.326 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.326 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.326 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.326 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.326 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.326 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.326 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.326 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.327 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.327 INFO:teuthology.orchestra.run.smithi043.stdout: "max_concurrent_clones": { 2024-09-19T05:15:26.327 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_concurrent_clones", 2024-09-19T05:15:26.327 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.327 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.327 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.327 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "4", 2024-09-19T05:15:26.327 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.327 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.327 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.327 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T05:15:26.327 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.328 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.328 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.328 INFO:teuthology.orchestra.run.smithi043.stdout: "periodic_async_work": { 2024-09-19T05:15:26.328 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "periodic_async_work", 2024-09-19T05:15:26.328 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.328 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.328 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.328 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.328 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.328 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.328 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.328 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Periodically check for async work", 2024-09-19T05:15:26.329 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.329 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.329 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.329 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.329 INFO:teuthology.orchestra.run.smithi043.stdout: "snapshot_clone_delay": { 2024-09-19T05:15:26.329 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snapshot_clone_delay", 2024-09-19T05:15:26.329 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.329 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.329 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.329 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-09-19T05:15:26.329 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.329 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.329 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.329 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T05:15:26.330 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.330 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.330 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.330 INFO:teuthology.orchestra.run.smithi043.stdout: "snapshot_clone_no_wait": { 2024-09-19T05:15:26.330 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T05:15:26.330 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.330 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.330 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.330 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.330 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.330 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.330 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.331 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T05:15:26.331 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.331 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.331 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.331 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.331 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.331 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix", 2024-09-19T05:15:26.331 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.331 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.331 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.331 INFO:teuthology.orchestra.run.smithi043.stdout: "discovery_interval": { 2024-09-19T05:15:26.331 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "discovery_interval", 2024-09-19T05:15:26.331 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-09-19T05:15:26.332 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.332 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.332 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "100", 2024-09-19T05:15:26.332 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.332 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.332 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.332 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.332 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.332 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.332 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.332 INFO:teuthology.orchestra.run.smithi043.stdout: "identifier": { 2024-09-19T05:15:26.332 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "identifier", 2024-09-19T05:15:26.332 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.333 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.333 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.333 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.333 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.333 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.333 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.333 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.333 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.333 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.333 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.333 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-09-19T05:15:26.333 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-09-19T05:15:26.334 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:26.334 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.334 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.334 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-09-19T05:15:26.334 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.334 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.334 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.334 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.334 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.334 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.334 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.334 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.334 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.335 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.335 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.335 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.335 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.335 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.335 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.335 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.335 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.335 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.335 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.335 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.335 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.335 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.335 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.336 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.336 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.336 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.336 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.336 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.336 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.336 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.336 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.336 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.336 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.336 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.336 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.337 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.337 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.337 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.337 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.337 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.337 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.337 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.337 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.337 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.337 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.337 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.337 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.337 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.337 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.337 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.338 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.338 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.338 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.338 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.338 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.338 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.338 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.338 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.338 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.338 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.339 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.339 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.339 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.339 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.339 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.339 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.339 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.339 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.339 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.339 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.339 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.339 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.339 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.340 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_host": { 2024-09-19T05:15:26.340 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_host", 2024-09-19T05:15:26.340 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.340 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.340 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.340 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.340 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.340 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.340 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.340 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.341 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.341 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.341 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.341 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_port": { 2024-09-19T05:15:26.341 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_port", 2024-09-19T05:15:26.341 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.341 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.341 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.341 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10051", 2024-09-19T05:15:26.341 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.341 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.342 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.342 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.342 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.342 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.342 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.342 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_sender": { 2024-09-19T05:15:26.342 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_sender", 2024-09-19T05:15:26.342 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.342 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.343 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.343 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T05:15:26.343 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.343 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.343 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.343 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.343 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.343 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.343 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.343 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.343 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.343 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.344 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-19T05:15:26.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.344 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 25133, 2024-09-19T05:15:26.344 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "y", 2024-09-19T05:15:26.344 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr_features": 4540701547738038271, 2024-09-19T05:15:26.344 INFO:teuthology.orchestra.run.smithi043.stdout: "available_modules": [ 2024-09-19T05:15:26.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.344 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "alerts", 2024-09-19T05:15:26.344 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.345 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.345 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.345 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-09-19T05:15:26.345 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-09-19T05:15:26.345 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:26.345 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.345 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.345 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-09-19T05:15:26.345 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.345 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.345 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.346 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T05:15:26.346 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.346 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.346 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.346 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.346 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.346 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.346 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.347 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.347 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.347 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.347 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.347 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.347 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.347 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.347 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.347 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.347 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.348 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.348 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.348 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.348 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.348 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.348 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.348 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.348 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.348 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.349 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.349 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.349 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.349 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.349 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.349 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.349 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.349 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.349 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.349 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.349 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.349 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.349 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.350 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.350 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.350 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.350 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.350 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.350 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.350 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.350 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.350 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.350 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.350 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.350 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.350 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.351 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.351 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.351 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.351 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.351 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.351 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.351 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.351 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.351 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.351 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.351 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.351 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.351 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.352 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.352 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.352 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.352 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.352 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.352 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_destination": { 2024-09-19T05:15:26.352 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_destination", 2024-09-19T05:15:26.352 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.352 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.352 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.352 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.352 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.352 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.352 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.353 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Email address to send alerts to", 2024-09-19T05:15:26.353 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.353 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.353 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.353 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_from_name": { 2024-09-19T05:15:26.353 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_from_name", 2024-09-19T05:15:26.353 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.353 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.353 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.353 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "Ceph", 2024-09-19T05:15:26.353 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.353 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.354 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.354 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Email From: name", 2024-09-19T05:15:26.354 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.354 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.354 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.354 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_host": { 2024-09-19T05:15:26.354 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_host", 2024-09-19T05:15:26.354 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.354 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.354 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.354 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.354 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.354 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.355 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.355 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP server", 2024-09-19T05:15:26.355 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.355 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.355 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.355 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_password": { 2024-09-19T05:15:26.355 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_password", 2024-09-19T05:15:26.355 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.355 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.355 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.355 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.355 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.356 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.356 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.356 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Password to authenticate with", 2024-09-19T05:15:26.356 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.356 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.356 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.356 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_port": { 2024-09-19T05:15:26.356 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_port", 2024-09-19T05:15:26.356 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.356 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.356 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.356 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "465", 2024-09-19T05:15:26.356 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.357 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.357 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.357 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP port", 2024-09-19T05:15:26.357 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.357 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.357 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.357 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_sender": { 2024-09-19T05:15:26.357 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_sender", 2024-09-19T05:15:26.357 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.357 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.357 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.357 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.357 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.357 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.358 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.358 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP envelope sender", 2024-09-19T05:15:26.358 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.358 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.358 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.358 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_ssl": { 2024-09-19T05:15:26.358 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_ssl", 2024-09-19T05:15:26.358 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.358 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.358 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.358 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.358 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.359 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.359 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.359 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T05:15:26.359 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.359 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.359 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.359 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_user": { 2024-09-19T05:15:26.359 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_user", 2024-09-19T05:15:26.359 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.359 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.359 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.359 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.359 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.360 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.360 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.360 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "User to authenticate as", 2024-09-19T05:15:26.360 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.360 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.360 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.360 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.360 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.360 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.360 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "balancer", 2024-09-19T05:15:26.360 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.360 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.360 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.361 INFO:teuthology.orchestra.run.smithi043.stdout: "active": { 2024-09-19T05:15:26.361 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "active", 2024-09-19T05:15:26.361 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.361 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.361 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.361 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.361 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.361 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.361 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.361 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T05:15:26.361 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.361 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.362 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.362 INFO:teuthology.orchestra.run.smithi043.stdout: "begin_time": { 2024-09-19T05:15:26.362 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "begin_time", 2024-09-19T05:15:26.362 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.362 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.362 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.362 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0000", 2024-09-19T05:15:26.362 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.362 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.362 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.362 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T05:15:26.362 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T05:15:26.363 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.363 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.363 INFO:teuthology.orchestra.run.smithi043.stdout: "begin_weekday": { 2024-09-19T05:15:26.363 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "begin_weekday", 2024-09-19T05:15:26.363 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-09-19T05:15:26.363 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.363 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.363 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-09-19T05:15:26.363 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0", 2024-09-19T05:15:26.363 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "6", 2024-09-19T05:15:26.363 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.363 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T05:15:26.364 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T05:15:26.364 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.364 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.364 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_max_iterations": { 2024-09-19T05:15:26.364 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_max_iterations", 2024-09-19T05:15:26.364 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-09-19T05:15:26.364 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.364 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.364 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "25", 2024-09-19T05:15:26.364 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-09-19T05:15:26.364 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "250", 2024-09-19T05:15:26.364 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.364 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T05:15:26.364 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.365 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.365 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.365 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.365 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_metrics": { 2024-09-19T05:15:26.365 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_metrics", 2024-09-19T05:15:26.365 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.365 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.365 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.365 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T05:15:26.365 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.365 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.365 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.365 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T05:15:26.366 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T05:15:26.366 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.366 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.366 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.366 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_step": { 2024-09-19T05:15:26.366 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_step", 2024-09-19T05:15:26.366 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-09-19T05:15:26.366 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.366 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.366 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0.5", 2024-09-19T05:15:26.366 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0.001", 2024-09-19T05:15:26.366 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "0.999", 2024-09-19T05:15:26.366 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.366 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "aggressiveness of optimization", 2024-09-19T05:15:26.367 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T05:15:26.367 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.367 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.367 INFO:teuthology.orchestra.run.smithi043.stdout: "end_time": { 2024-09-19T05:15:26.367 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "end_time", 2024-09-19T05:15:26.367 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.367 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.367 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.367 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "2359", 2024-09-19T05:15:26.367 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.367 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.367 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.367 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T05:15:26.367 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T05:15:26.368 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.368 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.368 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.368 INFO:teuthology.orchestra.run.smithi043.stdout: "end_weekday": { 2024-09-19T05:15:26.368 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "end_weekday", 2024-09-19T05:15:26.368 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-09-19T05:15:26.368 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.368 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.368 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-09-19T05:15:26.368 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0", 2024-09-19T05:15:26.368 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "6", 2024-09-19T05:15:26.368 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.368 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T05:15:26.369 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T05:15:26.369 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.369 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.369 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.369 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.369 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.369 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.369 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.369 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.369 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.369 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.370 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.370 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.370 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.370 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.370 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.370 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.370 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.370 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.370 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.370 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.370 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.370 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.370 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.370 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.371 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.371 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.371 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.371 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.371 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.371 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.371 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.371 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.371 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.372 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.372 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.372 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.372 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.372 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.372 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.372 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.372 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.372 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.372 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.372 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.373 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.373 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.373 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.373 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.373 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.373 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.373 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.373 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.373 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.373 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.374 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.374 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.374 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.374 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.374 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.374 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.374 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.374 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.374 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.374 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.374 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.374 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.375 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.375 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.375 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.375 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.375 INFO:teuthology.orchestra.run.smithi043.stdout: "min_score": { 2024-09-19T05:15:26.375 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "min_score", 2024-09-19T05:15:26.375 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-09-19T05:15:26.375 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.375 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.376 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-09-19T05:15:26.376 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.376 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.376 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.376 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T05:15:26.376 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.376 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.376 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.376 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": { 2024-09-19T05:15:26.376 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mode", 2024-09-19T05:15:26.376 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.376 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.377 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.377 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "upmap", 2024-09-19T05:15:26.377 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.377 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.377 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.377 INFO:teuthology.orchestra.run.smithi043.stdout: "crush-compat", 2024-09-19T05:15:26.377 INFO:teuthology.orchestra.run.smithi043.stdout: "none", 2024-09-19T05:15:26.377 INFO:teuthology.orchestra.run.smithi043.stdout: "read", 2024-09-19T05:15:26.377 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap", 2024-09-19T05:15:26.378 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap-read" 2024-09-19T05:15:26.378 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.378 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Balancer mode", 2024-09-19T05:15:26.378 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.378 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.378 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.378 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.378 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ids": { 2024-09-19T05:15:26.378 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pool_ids", 2024-09-19T05:15:26.378 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.379 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.379 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.379 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.379 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.379 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.379 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.379 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T05:15:26.379 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.379 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.379 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.380 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.380 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-09-19T05:15:26.380 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-09-19T05:15:26.380 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:26.380 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.380 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.380 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-09-19T05:15:26.380 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.380 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.381 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.381 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T05:15:26.381 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.381 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.381 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.381 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap_max_deviation": { 2024-09-19T05:15:26.381 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "upmap_max_deviation", 2024-09-19T05:15:26.381 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.381 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.382 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.382 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-09-19T05:15:26.382 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-09-19T05:15:26.382 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.382 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.382 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T05:15:26.382 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T05:15:26.382 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.382 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.382 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap_max_optimizations": { 2024-09-19T05:15:26.382 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "upmap_max_optimizations", 2024-09-19T05:15:26.382 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-09-19T05:15:26.382 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.383 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.383 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-19T05:15:26.383 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.383 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.383 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.383 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T05:15:26.383 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.383 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.383 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.383 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.383 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.383 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cephadm", 2024-09-19T05:15:26.384 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.384 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.384 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.384 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_down_multiplier": { 2024-09-19T05:15:26.384 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_down_multiplier", 2024-09-19T05:15:26.384 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-09-19T05:15:26.384 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.384 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.384 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3.0", 2024-09-19T05:15:26.384 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.384 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.384 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.385 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T05:15:26.385 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.385 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.385 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.385 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_refresh_rate": { 2024-09-19T05:15:26.385 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_refresh_rate", 2024-09-19T05:15:26.385 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:26.385 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.385 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.385 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "20", 2024-09-19T05:15:26.385 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.385 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.386 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.386 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T05:15:26.386 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.386 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.386 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.386 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_starting_port": { 2024-09-19T05:15:26.386 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_starting_port", 2024-09-19T05:15:26.386 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.386 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.386 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.386 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "4721", 2024-09-19T05:15:26.386 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.386 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.387 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.387 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T05:15:26.387 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.387 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.387 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.387 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_ptrace": { 2024-09-19T05:15:26.387 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_ptrace", 2024-09-19T05:15:26.387 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.387 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.387 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.387 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.387 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.387 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.388 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.388 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T05:15:26.388 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T05:15:26.388 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.388 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.388 INFO:teuthology.orchestra.run.smithi043.stdout: "autotune_interval": { 2024-09-19T05:15:26.388 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "autotune_interval", 2024-09-19T05:15:26.388 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:26.388 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.388 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.388 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-09-19T05:15:26.388 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.388 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.389 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.389 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T05:15:26.389 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.389 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.389 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.389 INFO:teuthology.orchestra.run.smithi043.stdout: "autotune_memory_target_ratio": { 2024-09-19T05:15:26.389 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T05:15:26.389 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-09-19T05:15:26.389 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.389 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.389 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0.7", 2024-09-19T05:15:26.389 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.389 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.390 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.390 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T05:15:26.390 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.390 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.390 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.390 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm_log_destination": { 2024-09-19T05:15:26.390 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cephadm_log_destination", 2024-09-19T05:15:26.390 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.390 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.390 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.390 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.390 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.390 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.391 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.391 INFO:teuthology.orchestra.run.smithi043.stdout: "file", 2024-09-19T05:15:26.391 INFO:teuthology.orchestra.run.smithi043.stdout: "file,syslog", 2024-09-19T05:15:26.391 INFO:teuthology.orchestra.run.smithi043.stdout: "syslog" 2024-09-19T05:15:26.391 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.391 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T05:15:26.391 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.391 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.391 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.391 INFO:teuthology.orchestra.run.smithi043.stdout: "cgroups_split": { 2024-09-19T05:15:26.391 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cgroups_split", 2024-09-19T05:15:26.391 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.391 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.392 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.392 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.392 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.392 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.392 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.392 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T05:15:26.392 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.392 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.392 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.392 INFO:teuthology.orchestra.run.smithi043.stdout: "config_checks_enabled": { 2024-09-19T05:15:26.392 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_checks_enabled", 2024-09-19T05:15:26.392 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.392 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.393 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.393 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.393 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.393 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.393 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.393 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T05:15:26.393 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.393 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.393 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.393 INFO:teuthology.orchestra.run.smithi043.stdout: "config_dashboard": { 2024-09-19T05:15:26.393 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_dashboard", 2024-09-19T05:15:26.393 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.394 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.394 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.394 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.394 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.394 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.394 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.394 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T05:15:26.394 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.394 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.394 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.394 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_alertmanager": { 2024-09-19T05:15:26.394 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_alertmanager", 2024-09-19T05:15:26.395 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.395 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.395 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.395 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T05:15:26.395 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.395 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.395 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.395 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-09-19T05:15:26.395 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.395 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.395 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.395 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_base": { 2024-09-19T05:15:26.395 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_base", 2024-09-19T05:15:26.396 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.396 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.396 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.396 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T05:15:26.396 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.396 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.396 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.396 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Container image name, without the tag", 2024-09-19T05:15:26.396 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.396 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.396 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.397 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_elasticsearch": { 2024-09-19T05:15:26.397 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_elasticsearch", 2024-09-19T05:15:26.397 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.397 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.397 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.397 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T05:15:26.397 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.397 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.397 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.397 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "elasticsearch container image", 2024-09-19T05:15:26.397 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.397 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.397 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.398 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_grafana": { 2024-09-19T05:15:26.398 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_grafana", 2024-09-19T05:15:26.398 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.398 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.398 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.398 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T05:15:26.398 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.398 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.398 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.398 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-09-19T05:15:26.398 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.398 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.399 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.399 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_haproxy": { 2024-09-19T05:15:26.399 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_haproxy", 2024-09-19T05:15:26.399 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.399 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.399 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.399 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T05:15:26.399 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.399 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.399 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.399 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "HAproxy container image", 2024-09-19T05:15:26.399 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.400 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.400 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.400 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_agent": { 2024-09-19T05:15:26.400 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_agent", 2024-09-19T05:15:26.400 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.400 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.400 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.400 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T05:15:26.400 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.400 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.400 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.400 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger agent container image", 2024-09-19T05:15:26.400 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.401 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.401 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.401 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_collector": { 2024-09-19T05:15:26.401 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_collector", 2024-09-19T05:15:26.401 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.401 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.401 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.401 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T05:15:26.401 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.401 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.401 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.401 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger collector container image", 2024-09-19T05:15:26.402 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.402 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.402 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.402 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_query": { 2024-09-19T05:15:26.402 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_query", 2024-09-19T05:15:26.402 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.402 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.402 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.402 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T05:15:26.402 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.402 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.402 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.402 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger query container image", 2024-09-19T05:15:26.402 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.403 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.403 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.403 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_keepalived": { 2024-09-19T05:15:26.403 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_keepalived", 2024-09-19T05:15:26.403 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.403 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.403 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.403 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T05:15:26.403 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.403 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.403 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.403 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Keepalived container image", 2024-09-19T05:15:26.403 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.404 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.404 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.404 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_loki": { 2024-09-19T05:15:26.404 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_loki", 2024-09-19T05:15:26.404 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.404 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.404 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.404 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T05:15:26.404 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.404 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.404 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.404 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Loki container image", 2024-09-19T05:15:26.405 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.405 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.405 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.405 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_node_exporter": { 2024-09-19T05:15:26.405 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_node_exporter", 2024-09-19T05:15:26.405 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.405 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.405 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.405 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T05:15:26.405 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.405 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.405 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.406 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-09-19T05:15:26.406 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.406 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.406 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.406 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_nvmeof": { 2024-09-19T05:15:26.406 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_nvmeof", 2024-09-19T05:15:26.406 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.406 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.406 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.406 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T05:15:26.406 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.406 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.406 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.407 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Nvme-of container image", 2024-09-19T05:15:26.407 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.407 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.407 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.407 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_prometheus": { 2024-09-19T05:15:26.407 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_prometheus", 2024-09-19T05:15:26.407 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.407 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.407 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.407 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T05:15:26.407 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.407 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.408 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.408 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-09-19T05:15:26.408 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.408 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.408 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.408 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_promtail": { 2024-09-19T05:15:26.408 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_promtail", 2024-09-19T05:15:26.408 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.408 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.408 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.408 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T05:15:26.408 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.409 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.409 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.409 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Promtail container image", 2024-09-19T05:15:26.409 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.409 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.409 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.409 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_samba": { 2024-09-19T05:15:26.409 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_samba", 2024-09-19T05:15:26.409 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.409 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.409 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.409 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T05:15:26.409 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.410 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.410 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.410 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Samba/SMB container image", 2024-09-19T05:15:26.410 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.410 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.410 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.410 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_snmp_gateway": { 2024-09-19T05:15:26.410 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_snmp_gateway", 2024-09-19T05:15:26.411 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.411 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.411 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.411 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T05:15:26.411 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.411 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.411 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.411 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SNMP Gateway container image", 2024-09-19T05:15:26.411 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.411 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.411 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.411 INFO:teuthology.orchestra.run.smithi043.stdout: "container_init": { 2024-09-19T05:15:26.412 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_init", 2024-09-19T05:15:26.412 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.412 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.412 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.412 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.412 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.412 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.412 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.412 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T05:15:26.413 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.413 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.413 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.413 INFO:teuthology.orchestra.run.smithi043.stdout: "daemon_cache_timeout": { 2024-09-19T05:15:26.413 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "daemon_cache_timeout", 2024-09-19T05:15:26.413 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:26.413 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.413 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.413 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-09-19T05:15:26.413 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.413 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.414 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.414 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T05:15:26.414 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.414 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.414 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.414 INFO:teuthology.orchestra.run.smithi043.stdout: "default_cephadm_command_timeout": { 2024-09-19T05:15:26.414 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T05:15:26.414 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.414 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.414 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.415 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "900", 2024-09-19T05:15:26.415 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.415 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.415 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.415 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T05:15:26.415 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.415 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.415 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.415 INFO:teuthology.orchestra.run.smithi043.stdout: "default_registry": { 2024-09-19T05:15:26.415 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "default_registry", 2024-09-19T05:15:26.416 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.416 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.416 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.416 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io", 2024-09-19T05:15:26.416 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.416 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.416 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.416 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T05:15:26.416 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.416 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.417 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.417 INFO:teuthology.orchestra.run.smithi043.stdout: "device_cache_timeout": { 2024-09-19T05:15:26.417 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_cache_timeout", 2024-09-19T05:15:26.417 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:26.417 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.417 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.417 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1800", 2024-09-19T05:15:26.417 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.417 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.418 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.418 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache device inventory", 2024-09-19T05:15:26.418 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.418 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.418 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.418 INFO:teuthology.orchestra.run.smithi043.stdout: "device_enhanced_scan": { 2024-09-19T05:15:26.418 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_enhanced_scan", 2024-09-19T05:15:26.418 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.418 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.418 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.418 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.418 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.419 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.419 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.419 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T05:15:26.419 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.419 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.419 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.419 INFO:teuthology.orchestra.run.smithi043.stdout: "facts_cache_timeout": { 2024-09-19T05:15:26.419 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "facts_cache_timeout", 2024-09-19T05:15:26.419 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:26.419 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.419 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.419 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-09-19T05:15:26.419 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.420 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.420 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.420 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache host facts data", 2024-09-19T05:15:26.420 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.420 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.420 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.420 INFO:teuthology.orchestra.run.smithi043.stdout: "grafana_dashboards_path": { 2024-09-19T05:15:26.420 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "grafana_dashboards_path", 2024-09-19T05:15:26.420 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.420 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.420 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.420 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T05:15:26.420 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.420 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.421 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.421 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T05:15:26.421 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.421 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.421 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.421 INFO:teuthology.orchestra.run.smithi043.stdout: "host_check_interval": { 2024-09-19T05:15:26.421 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "host_check_interval", 2024-09-19T05:15:26.421 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:26.421 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.421 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.421 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-09-19T05:15:26.421 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.421 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.421 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.422 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to perform a host check", 2024-09-19T05:15:26.422 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.422 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.422 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.422 INFO:teuthology.orchestra.run.smithi043.stdout: "hw_monitoring": { 2024-09-19T05:15:26.422 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "hw_monitoring", 2024-09-19T05:15:26.422 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.422 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.422 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.422 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.422 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.422 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.422 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.423 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T05:15:26.423 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.423 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.423 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.423 INFO:teuthology.orchestra.run.smithi043.stdout: "inventory_list_all": { 2024-09-19T05:15:26.423 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "inventory_list_all", 2024-09-19T05:15:26.423 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.423 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.423 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.423 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.423 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.423 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.423 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.423 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T05:15:26.424 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.424 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.424 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.424 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.424 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.424 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.424 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.424 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.424 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.424 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.424 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.424 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.424 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.425 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.425 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.425 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.425 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.425 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.425 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.425 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.425 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.425 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.425 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.425 INFO:teuthology.orchestra.run.smithi043.stdout: "log_refresh_metadata": { 2024-09-19T05:15:26.425 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_refresh_metadata", 2024-09-19T05:15:26.426 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.426 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.426 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.426 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.426 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.426 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.426 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.426 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T05:15:26.426 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.426 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.426 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.426 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.426 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.427 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.427 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.427 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.427 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.427 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.427 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.427 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.427 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T05:15:26.427 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.427 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.427 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.427 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.427 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.427 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.428 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.428 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.428 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.428 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.428 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.428 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.428 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.428 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.428 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.428 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.428 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.428 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.428 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.429 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.429 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.429 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.429 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.429 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.429 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.429 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.429 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.429 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.429 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.429 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.429 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.429 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.429 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.430 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.430 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.430 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.430 INFO:teuthology.orchestra.run.smithi043.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T05:15:26.430 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T05:15:26.430 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.430 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.430 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.430 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.430 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.430 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.430 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.430 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T05:15:26.431 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.431 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.431 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.431 INFO:teuthology.orchestra.run.smithi043.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T05:15:26.431 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T05:15:26.431 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.431 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.431 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.431 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "*", 2024-09-19T05:15:26.431 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.431 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.431 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.431 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T05:15:26.431 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.432 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.432 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.432 INFO:teuthology.orchestra.run.smithi043.stdout: "max_count_per_host": { 2024-09-19T05:15:26.432 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_count_per_host", 2024-09-19T05:15:26.432 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.432 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.432 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.432 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-19T05:15:26.432 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.432 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.432 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.432 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "max number of daemons per service per host", 2024-09-19T05:15:26.432 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.433 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.433 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.433 INFO:teuthology.orchestra.run.smithi043.stdout: "max_osd_draining_count": { 2024-09-19T05:15:26.433 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_osd_draining_count", 2024-09-19T05:15:26.433 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.433 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.433 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.433 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-19T05:15:26.433 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.433 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.433 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.433 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T05:15:26.433 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.434 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.434 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.434 INFO:teuthology.orchestra.run.smithi043.stdout: "migration_current": { 2024-09-19T05:15:26.434 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "migration_current", 2024-09-19T05:15:26.434 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.434 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.434 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.434 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.434 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.434 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.434 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.434 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "internal - do not modify", 2024-09-19T05:15:26.434 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.435 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.435 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.435 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": { 2024-09-19T05:15:26.435 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mode", 2024-09-19T05:15:26.435 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.435 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.435 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.435 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "root", 2024-09-19T05:15:26.435 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.435 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.435 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.435 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm-package", 2024-09-19T05:15:26.435 INFO:teuthology.orchestra.run.smithi043.stdout: "root" 2024-09-19T05:15:26.436 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.436 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T05:15:26.436 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.436 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.436 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.436 INFO:teuthology.orchestra.run.smithi043.stdout: "oob_default_addr": { 2024-09-19T05:15:26.436 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "oob_default_addr", 2024-09-19T05:15:26.436 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.436 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.436 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.436 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "169.254.1.1", 2024-09-19T05:15:26.436 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.436 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.437 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.437 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T05:15:26.437 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.437 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.437 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.437 INFO:teuthology.orchestra.run.smithi043.stdout: "prometheus_alerts_path": { 2024-09-19T05:15:26.437 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prometheus_alerts_path", 2024-09-19T05:15:26.437 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.437 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.437 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.437 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T05:15:26.437 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.438 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.438 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.438 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T05:15:26.438 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.438 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.438 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.438 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_insecure": { 2024-09-19T05:15:26.438 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_insecure", 2024-09-19T05:15:26.438 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.439 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.439 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.439 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.439 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.439 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.439 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.439 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T05:15:26.439 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.439 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.439 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.439 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_password": { 2024-09-19T05:15:26.439 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_password", 2024-09-19T05:15:26.440 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.440 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.440 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.440 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.440 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.440 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.440 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.440 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T05:15:26.440 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.440 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.440 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.441 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_url": { 2024-09-19T05:15:26.441 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_url", 2024-09-19T05:15:26.441 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.441 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.441 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.441 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.441 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.441 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.441 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.441 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T05:15:26.441 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.442 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.442 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.442 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_username": { 2024-09-19T05:15:26.442 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_username", 2024-09-19T05:15:26.442 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.442 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.442 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.442 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.442 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.442 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.442 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.443 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T05:15:26.443 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.443 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.443 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.443 INFO:teuthology.orchestra.run.smithi043.stdout: "secure_monitoring_stack": { 2024-09-19T05:15:26.443 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "secure_monitoring_stack", 2024-09-19T05:15:26.443 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.443 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.443 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.443 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.444 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.444 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.444 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.444 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T05:15:26.444 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.444 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.444 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.444 INFO:teuthology.orchestra.run.smithi043.stdout: "service_discovery_port": { 2024-09-19T05:15:26.444 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "service_discovery_port", 2024-09-19T05:15:26.445 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.445 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.445 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.445 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8765", 2024-09-19T05:15:26.445 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.445 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.445 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.445 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "cephadm service discovery port", 2024-09-19T05:15:26.445 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.445 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.445 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.445 INFO:teuthology.orchestra.run.smithi043.stdout: "ssh_config_file": { 2024-09-19T05:15:26.446 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssh_config_file", 2024-09-19T05:15:26.446 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.446 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.446 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.446 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.446 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.446 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.446 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.446 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T05:15:26.446 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.446 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.447 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.447 INFO:teuthology.orchestra.run.smithi043.stdout: "use_agent": { 2024-09-19T05:15:26.447 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "use_agent", 2024-09-19T05:15:26.447 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.447 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.447 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.447 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.447 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.447 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.447 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.448 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T05:15:26.448 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.448 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.448 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.448 INFO:teuthology.orchestra.run.smithi043.stdout: "use_repo_digest": { 2024-09-19T05:15:26.448 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "use_repo_digest", 2024-09-19T05:15:26.448 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.448 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.448 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.449 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.449 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.449 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.449 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.449 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T05:15:26.449 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.449 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.449 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.449 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_failed_host_check": { 2024-09-19T05:15:26.449 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_failed_host_check", 2024-09-19T05:15:26.450 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.450 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.450 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.450 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.450 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.450 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.450 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.450 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T05:15:26.450 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.450 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.451 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.451 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_stray_daemons": { 2024-09-19T05:15:26.451 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_stray_daemons", 2024-09-19T05:15:26.451 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.451 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.451 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.451 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.451 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.451 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.451 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.451 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T05:15:26.451 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.451 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.452 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.452 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_stray_hosts": { 2024-09-19T05:15:26.452 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_stray_hosts", 2024-09-19T05:15:26.452 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.452 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.452 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.452 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.452 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.452 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.452 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.452 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T05:15:26.452 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.452 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.453 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.453 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.453 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.453 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crash", 2024-09-19T05:15:26.453 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.453 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.453 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.453 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.453 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.453 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.454 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.454 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.454 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.454 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.454 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.454 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.454 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.454 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.454 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.454 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.454 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.454 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.454 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.454 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.455 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.455 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.455 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.455 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.455 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.455 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.455 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.455 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.455 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.455 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.455 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.455 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.455 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.455 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.456 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.456 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.456 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.456 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.456 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.456 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.456 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.456 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.456 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.456 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.456 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.456 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.457 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.457 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.457 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.457 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.457 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.457 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.457 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.457 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.457 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.457 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.457 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.457 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.457 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.458 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.458 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.458 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.458 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.458 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.458 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.458 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.458 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.458 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.458 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.458 INFO:teuthology.orchestra.run.smithi043.stdout: "retain_interval": { 2024-09-19T05:15:26.458 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "retain_interval", 2024-09-19T05:15:26.458 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:26.459 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.459 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.459 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "31536000", 2024-09-19T05:15:26.459 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.459 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.459 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.459 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T05:15:26.459 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.459 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.459 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.459 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_recent_interval": { 2024-09-19T05:15:26.459 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_recent_interval", 2024-09-19T05:15:26.459 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:26.459 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.460 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.460 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1209600", 2024-09-19T05:15:26.460 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.460 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.460 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.460 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T05:15:26.460 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.460 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.460 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.460 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.460 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.460 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "dashboard", 2024-09-19T05:15:26.461 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.461 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.461 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.461 INFO:teuthology.orchestra.run.smithi043.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T05:15:26.461 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T05:15:26.461 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.461 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.461 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.461 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-19T05:15:26.461 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.461 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.461 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.461 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.462 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.462 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.462 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.462 INFO:teuthology.orchestra.run.smithi043.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T05:15:26.462 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T05:15:26.462 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.462 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.462 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.462 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.462 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.462 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.462 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.462 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.463 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.463 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.463 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.463 INFO:teuthology.orchestra.run.smithi043.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T05:15:26.463 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T05:15:26.463 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.463 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.463 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.463 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.463 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.463 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.463 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.463 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.464 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.464 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.464 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.464 INFO:teuthology.orchestra.run.smithi043.stdout: "AUDIT_API_ENABLED": { 2024-09-19T05:15:26.464 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T05:15:26.464 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.464 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.464 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.464 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.464 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.464 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.464 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.464 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.465 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.465 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.465 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.465 INFO:teuthology.orchestra.run.smithi043.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T05:15:26.465 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T05:15:26.465 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.465 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.465 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.465 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.465 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.465 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.465 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.465 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.466 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.466 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.466 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.466 INFO:teuthology.orchestra.run.smithi043.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T05:15:26.466 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T05:15:26.466 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.466 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.466 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.466 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.466 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.466 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.466 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.467 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.467 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.467 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.467 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.467 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T05:15:26.467 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T05:15:26.467 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.467 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.467 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.467 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.467 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.467 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.467 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.468 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.468 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.468 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.468 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.468 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T05:15:26.468 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T05:15:26.468 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.468 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.468 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.468 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.468 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.468 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.468 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.469 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.469 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.469 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.469 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.469 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T05:15:26.469 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T05:15:26.469 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.469 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.469 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.469 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.469 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.469 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.470 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.470 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.470 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.470 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.470 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.470 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T05:15:26.470 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T05:15:26.470 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.470 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.470 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.470 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.470 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.471 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.471 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.471 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.471 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.471 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.471 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.471 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T05:15:26.471 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T05:15:26.471 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.471 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.472 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.472 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.472 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.472 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.472 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.472 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.472 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.472 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.472 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.472 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T05:15:26.472 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T05:15:26.472 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.473 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.473 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.473 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.473 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.473 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.473 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.473 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.473 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.473 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.473 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.474 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T05:15:26.474 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T05:15:26.474 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.474 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.474 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.474 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.474 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.474 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.474 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.474 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.474 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.474 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.474 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.475 INFO:teuthology.orchestra.run.smithi043.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T05:15:26.475 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T05:15:26.475 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.475 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.475 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.475 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.475 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.475 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.475 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.475 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.475 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.475 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.475 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.476 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T05:15:26.476 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T05:15:26.476 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.476 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.476 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.476 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-09-19T05:15:26.476 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.476 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.476 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.476 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.476 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.476 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.476 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.477 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T05:15:26.477 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T05:15:26.477 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.477 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.477 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.477 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.477 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.477 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.477 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.477 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.477 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.477 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.477 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.478 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_URL": { 2024-09-19T05:15:26.478 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_URL", 2024-09-19T05:15:26.478 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.478 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.478 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.478 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.478 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.478 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.478 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.478 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.478 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.479 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.479 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.479 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T05:15:26.479 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T05:15:26.479 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.479 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.479 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.479 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-09-19T05:15:26.479 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.479 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.479 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.479 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.480 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.480 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.480 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.480 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T05:15:26.480 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T05:15:26.480 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.480 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.480 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.480 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.480 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.480 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.481 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.481 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.481 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.481 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.481 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.481 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T05:15:26.481 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T05:15:26.481 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.481 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.482 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.482 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.482 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.482 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.482 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.482 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.482 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.482 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.482 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.482 INFO:teuthology.orchestra.run.smithi043.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T05:15:26.482 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T05:15:26.482 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.483 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.483 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.483 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.483 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.483 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.483 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.483 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.483 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.484 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.484 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.484 INFO:teuthology.orchestra.run.smithi043.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T05:15:26.484 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T05:15:26.484 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.484 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.484 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.484 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.484 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.485 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.485 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.485 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.485 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.485 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.485 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.485 INFO:teuthology.orchestra.run.smithi043.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T05:15:26.485 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T05:15:26.485 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.486 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.486 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.486 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.486 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.486 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.486 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.486 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.486 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.486 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.486 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.486 INFO:teuthology.orchestra.run.smithi043.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T05:15:26.486 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T05:15:26.486 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.487 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.487 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.487 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.487 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.487 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.487 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.487 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.487 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.487 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.487 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.487 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T05:15:26.487 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T05:15:26.487 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.487 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.488 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.488 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.488 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.488 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.488 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.488 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.488 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.488 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.488 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.488 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T05:15:26.488 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T05:15:26.488 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.489 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.489 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.489 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.489 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.489 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.489 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.489 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.489 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.489 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.489 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.489 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T05:15:26.489 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T05:15:26.489 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.490 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.490 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.490 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.490 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.490 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.490 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.490 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.490 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.490 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.490 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.490 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T05:15:26.490 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T05:15:26.490 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.490 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.491 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.491 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.491 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.491 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.491 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.491 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.491 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.491 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.491 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.491 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T05:15:26.491 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T05:15:26.491 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.491 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.491 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.492 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.492 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.492 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.492 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.492 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.492 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.492 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.492 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.492 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T05:15:26.492 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T05:15:26.492 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.492 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.492 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.493 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.493 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.493 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.493 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.493 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.493 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.493 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.493 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.493 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T05:15:26.493 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T05:15:26.493 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.493 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.493 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.494 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.494 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.494 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.494 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.494 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.494 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.494 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.494 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.494 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T05:15:26.494 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T05:15:26.494 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.494 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.494 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.494 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.495 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.495 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.495 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.495 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.495 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.495 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.495 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.495 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T05:15:26.495 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T05:15:26.495 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.495 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.495 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.496 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T05:15:26.496 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.496 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.496 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.496 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.496 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.496 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.496 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.496 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T05:15:26.496 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T05:15:26.496 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.496 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.497 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.497 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-19T05:15:26.497 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.497 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.497 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.497 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.497 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.497 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.497 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.497 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T05:15:26.497 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T05:15:26.497 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.497 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.497 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.498 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8", 2024-09-19T05:15:26.498 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.498 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.498 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.498 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.498 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.498 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.498 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.498 INFO:teuthology.orchestra.run.smithi043.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T05:15:26.498 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T05:15:26.498 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.498 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.499 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.499 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "45", 2024-09-19T05:15:26.499 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.499 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.499 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.499 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.499 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.499 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.499 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.499 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T05:15:26.499 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T05:15:26.499 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.500 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.500 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.500 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.500 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.500 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.500 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.500 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.500 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.500 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.500 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.500 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T05:15:26.500 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T05:15:26.501 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.501 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.501 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.501 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-09-19T05:15:26.501 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.501 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.501 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.501 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.501 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.501 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.501 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.501 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T05:15:26.502 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T05:15:26.502 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.502 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.502 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.502 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.502 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.502 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.502 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.502 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.502 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.502 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.502 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.502 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T05:15:26.503 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T05:15:26.503 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.503 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.503 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.503 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.503 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.503 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.503 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.503 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.503 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.503 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.503 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.504 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T05:15:26.504 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T05:15:26.504 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.504 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.504 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.504 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-09-19T05:15:26.504 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.504 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.504 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.504 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.504 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.504 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.504 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.505 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T05:15:26.505 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T05:15:26.505 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.505 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.505 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.505 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-19T05:15:26.505 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.505 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.505 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.505 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.505 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.505 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.505 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.505 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T05:15:26.506 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T05:15:26.506 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.506 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.506 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.506 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-09-19T05:15:26.506 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.506 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.506 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.506 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.506 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.507 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.507 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.507 INFO:teuthology.orchestra.run.smithi043.stdout: "cross_origin_url": { 2024-09-19T05:15:26.507 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cross_origin_url", 2024-09-19T05:15:26.507 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.507 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.507 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.507 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.507 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.507 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.507 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.507 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.507 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.508 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.508 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.508 INFO:teuthology.orchestra.run.smithi043.stdout: "crt_file": { 2024-09-19T05:15:26.508 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crt_file", 2024-09-19T05:15:26.508 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.508 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.508 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.508 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.508 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.508 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.508 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.508 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.509 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.509 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.509 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.509 INFO:teuthology.orchestra.run.smithi043.stdout: "debug": { 2024-09-19T05:15:26.509 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "debug", 2024-09-19T05:15:26.509 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.509 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.509 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.509 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.509 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.510 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.510 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.510 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable/disable debug options", 2024-09-19T05:15:26.510 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.510 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.510 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.510 INFO:teuthology.orchestra.run.smithi043.stdout: "jwt_token_ttl": { 2024-09-19T05:15:26.510 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "jwt_token_ttl", 2024-09-19T05:15:26.510 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.510 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.510 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.511 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "28800", 2024-09-19T05:15:26.511 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.511 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.511 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.511 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.511 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.511 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.511 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.511 INFO:teuthology.orchestra.run.smithi043.stdout: "key_file": { 2024-09-19T05:15:26.511 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "key_file", 2024-09-19T05:15:26.511 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.512 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.512 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.512 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.512 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.512 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.512 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.512 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.512 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.512 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.512 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.513 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.513 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.513 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.513 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.513 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.513 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.513 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.513 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.513 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.513 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.513 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.513 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.513 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.514 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.514 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.514 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.514 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.514 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.514 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.514 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.514 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.514 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.515 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.515 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.515 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.515 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.515 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.515 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.515 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.515 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.515 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.515 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.515 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.516 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.516 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.516 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.516 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.516 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.516 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.516 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.517 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.517 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.517 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.517 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.517 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.517 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.517 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.517 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.517 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.517 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.518 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.518 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.518 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.518 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.518 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.518 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.518 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.518 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.518 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.519 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.519 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.519 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.519 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.519 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.519 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.519 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.519 INFO:teuthology.orchestra.run.smithi043.stdout: "motd": { 2024-09-19T05:15:26.519 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "motd", 2024-09-19T05:15:26.519 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.519 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.519 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.519 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.520 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.520 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.520 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.520 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "The message of the day", 2024-09-19T05:15:26.520 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.520 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.520 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.520 INFO:teuthology.orchestra.run.smithi043.stdout: "redirect_resolve_ip_addr": { 2024-09-19T05:15:26.520 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T05:15:26.520 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.520 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.520 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.520 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.520 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.521 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.521 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.521 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.521 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.521 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.521 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.521 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-09-19T05:15:26.521 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-09-19T05:15:26.521 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.521 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.521 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.521 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "::", 2024-09-19T05:15:26.521 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.522 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.522 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.522 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.522 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.522 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.522 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.522 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-09-19T05:15:26.522 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-09-19T05:15:26.522 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.522 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.522 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.522 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8080", 2024-09-19T05:15:26.522 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.522 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.523 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.523 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.523 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.523 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.523 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.523 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl": { 2024-09-19T05:15:26.523 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl", 2024-09-19T05:15:26.523 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.523 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.523 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.523 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.523 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.523 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.524 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.524 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.524 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.524 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.524 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.524 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl_server_port": { 2024-09-19T05:15:26.524 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl_server_port", 2024-09-19T05:15:26.524 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.524 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.524 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.524 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8443", 2024-09-19T05:15:26.524 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.525 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.525 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.525 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.525 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.525 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.525 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.525 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_behaviour": { 2024-09-19T05:15:26.525 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_behaviour", 2024-09-19T05:15:26.525 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.525 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.525 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.526 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "redirect", 2024-09-19T05:15:26.526 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.526 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.526 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.526 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.526 INFO:teuthology.orchestra.run.smithi043.stdout: "redirect" 2024-09-19T05:15:26.526 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.526 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.526 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.526 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.526 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.526 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_error_status_code": { 2024-09-19T05:15:26.526 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_error_status_code", 2024-09-19T05:15:26.527 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.527 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.527 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.527 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "500", 2024-09-19T05:15:26.527 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "400", 2024-09-19T05:15:26.527 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "599", 2024-09-19T05:15:26.527 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.527 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.527 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.527 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.527 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.527 INFO:teuthology.orchestra.run.smithi043.stdout: "url_prefix": { 2024-09-19T05:15:26.528 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "url_prefix", 2024-09-19T05:15:26.528 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.528 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.528 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.528 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.528 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.528 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.528 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.528 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.528 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.528 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.528 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.528 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.528 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.529 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "devicehealth", 2024-09-19T05:15:26.529 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.529 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.529 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.529 INFO:teuthology.orchestra.run.smithi043.stdout: "enable_monitoring": { 2024-09-19T05:15:26.529 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enable_monitoring", 2024-09-19T05:15:26.529 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.529 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.529 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.529 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.529 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.529 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.530 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.530 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "monitor device health metrics", 2024-09-19T05:15:26.530 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.530 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.530 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.530 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.530 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.530 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.530 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.530 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.530 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.530 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.530 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.530 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.531 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.531 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.531 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.531 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.531 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.531 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.531 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.531 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.531 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.531 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.531 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.531 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.531 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.531 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.532 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.532 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.532 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.532 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.532 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.532 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.532 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.532 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.532 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.532 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.532 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.532 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.532 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.533 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.533 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.533 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.533 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.533 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.533 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.533 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.533 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.533 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.533 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.533 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.533 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.533 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.533 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.534 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.534 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.534 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.534 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.534 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.534 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.534 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.534 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.534 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.534 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.534 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.534 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.535 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.535 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.535 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.535 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.535 INFO:teuthology.orchestra.run.smithi043.stdout: "mark_out_threshold": { 2024-09-19T05:15:26.535 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mark_out_threshold", 2024-09-19T05:15:26.535 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:26.535 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.535 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.535 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "2419200", 2024-09-19T05:15:26.535 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.535 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.535 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.536 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T05:15:26.536 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.536 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.536 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.536 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": { 2024-09-19T05:15:26.536 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pool_name", 2024-09-19T05:15:26.536 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.536 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.536 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.536 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "device_health_metrics", 2024-09-19T05:15:26.536 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.536 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.536 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.537 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T05:15:26.537 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.537 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.537 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.537 INFO:teuthology.orchestra.run.smithi043.stdout: "retention_period": { 2024-09-19T05:15:26.537 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "retention_period", 2024-09-19T05:15:26.537 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:26.537 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.537 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.537 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15552000", 2024-09-19T05:15:26.537 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.537 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.537 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.538 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long to retain device health metrics", 2024-09-19T05:15:26.538 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.538 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.538 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.538 INFO:teuthology.orchestra.run.smithi043.stdout: "scrape_frequency": { 2024-09-19T05:15:26.538 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "scrape_frequency", 2024-09-19T05:15:26.538 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:26.538 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.538 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.538 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "86400", 2024-09-19T05:15:26.538 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.539 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.539 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.539 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T05:15:26.539 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.539 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.539 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.539 INFO:teuthology.orchestra.run.smithi043.stdout: "self_heal": { 2024-09-19T05:15:26.539 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "self_heal", 2024-09-19T05:15:26.539 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.539 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.540 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.540 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.540 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.540 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.540 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.540 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T05:15:26.540 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.540 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.540 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.540 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-09-19T05:15:26.540 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-09-19T05:15:26.540 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:26.541 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.541 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.541 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-09-19T05:15:26.541 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.541 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.541 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.541 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T05:15:26.541 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.541 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.541 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.542 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_threshold": { 2024-09-19T05:15:26.542 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_threshold", 2024-09-19T05:15:26.542 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:26.542 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.542 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.542 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "7257600", 2024-09-19T05:15:26.542 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.542 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.542 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.542 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T05:15:26.542 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.542 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.542 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.542 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.543 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.543 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "diskprediction_local", 2024-09-19T05:15:26.543 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.543 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.543 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.543 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.543 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.543 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.544 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.544 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.544 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.544 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.544 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.544 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.544 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.544 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.544 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.544 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.544 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.544 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.545 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.545 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.545 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.545 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.545 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.545 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.545 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.545 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.545 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.545 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.545 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.546 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.546 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.546 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.546 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.546 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.546 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.546 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.546 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.546 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.546 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.546 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.546 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.546 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.547 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.547 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.547 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.547 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.547 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.547 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.547 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.547 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.547 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.547 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.547 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.547 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.547 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.547 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.548 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.548 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.548 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.548 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.548 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.548 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.548 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.548 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.548 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.548 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.548 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.548 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.548 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.549 INFO:teuthology.orchestra.run.smithi043.stdout: "predict_interval": { 2024-09-19T05:15:26.549 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "predict_interval", 2024-09-19T05:15:26.549 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.549 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.549 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.549 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "86400", 2024-09-19T05:15:26.549 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.549 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.549 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.549 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.549 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.549 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.549 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.550 INFO:teuthology.orchestra.run.smithi043.stdout: "predictor_model": { 2024-09-19T05:15:26.550 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "predictor_model", 2024-09-19T05:15:26.550 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.550 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.550 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.550 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "prophetstor", 2024-09-19T05:15:26.550 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.550 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.550 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.550 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.551 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.551 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.551 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.551 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-09-19T05:15:26.551 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-09-19T05:15:26.551 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.551 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.551 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.551 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-09-19T05:15:26.551 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.552 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.552 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.552 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.552 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.552 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.552 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.552 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.552 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.553 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "influx", 2024-09-19T05:15:26.553 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": false, 2024-09-19T05:15:26.553 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "influxdb python module not found", 2024-09-19T05:15:26.553 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.553 INFO:teuthology.orchestra.run.smithi043.stdout: "batch_size": { 2024-09-19T05:15:26.553 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "batch_size", 2024-09-19T05:15:26.553 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.553 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.554 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.554 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5000", 2024-09-19T05:15:26.554 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.554 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.554 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.554 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T05:15:26.554 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.554 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.554 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.555 INFO:teuthology.orchestra.run.smithi043.stdout: "database": { 2024-09-19T05:15:26.555 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "database", 2024-09-19T05:15:26.555 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.555 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.555 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.555 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "ceph", 2024-09-19T05:15:26.555 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.555 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.555 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.555 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T05:15:26.555 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.555 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.555 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.556 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": { 2024-09-19T05:15:26.556 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "hostname", 2024-09-19T05:15:26.556 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.556 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.556 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.556 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.556 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.556 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.556 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.556 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB server hostname", 2024-09-19T05:15:26.556 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.556 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.556 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.557 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-09-19T05:15:26.557 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-09-19T05:15:26.557 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:26.557 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.557 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.557 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "30", 2024-09-19T05:15:26.557 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "5", 2024-09-19T05:15:26.557 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.557 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.557 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T05:15:26.558 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.558 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.558 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.558 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.558 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.558 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.558 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.558 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.558 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.558 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.558 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.558 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.559 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.559 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.559 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.559 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.559 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.559 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.559 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.559 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.559 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.559 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.559 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.559 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.559 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.560 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.560 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.560 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.560 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.560 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.560 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.560 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.560 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.560 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.560 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.560 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.560 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.560 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.561 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.561 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.561 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.561 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.561 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.561 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.561 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.561 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.561 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.561 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.561 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.561 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.561 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.561 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.562 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.562 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.562 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.562 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.562 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.562 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.562 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.562 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.562 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.562 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.562 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.562 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.562 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.563 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.563 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.563 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.563 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.563 INFO:teuthology.orchestra.run.smithi043.stdout: "password": { 2024-09-19T05:15:26.563 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "password", 2024-09-19T05:15:26.563 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.563 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.563 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.563 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.563 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.563 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.563 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.563 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "password of InfluxDB server user", 2024-09-19T05:15:26.564 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.564 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.564 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.564 INFO:teuthology.orchestra.run.smithi043.stdout: "port": { 2024-09-19T05:15:26.564 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "port", 2024-09-19T05:15:26.564 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.564 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.564 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.564 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8086", 2024-09-19T05:15:26.564 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.564 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.564 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.564 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB server port", 2024-09-19T05:15:26.565 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.565 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.565 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.565 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl": { 2024-09-19T05:15:26.565 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl", 2024-09-19T05:15:26.565 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.565 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.565 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.565 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "false", 2024-09-19T05:15:26.565 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.565 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.565 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.565 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T05:15:26.566 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.566 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.566 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.566 INFO:teuthology.orchestra.run.smithi043.stdout: "threads": { 2024-09-19T05:15:26.566 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "threads", 2024-09-19T05:15:26.566 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.566 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.566 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.566 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-09-19T05:15:26.566 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-09-19T05:15:26.566 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "32", 2024-09-19T05:15:26.566 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.566 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T05:15:26.566 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.567 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.567 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.567 INFO:teuthology.orchestra.run.smithi043.stdout: "username": { 2024-09-19T05:15:26.567 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "username", 2024-09-19T05:15:26.567 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.567 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.567 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.567 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.567 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.567 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.567 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.567 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "username of InfluxDB server user", 2024-09-19T05:15:26.567 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.568 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.568 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.568 INFO:teuthology.orchestra.run.smithi043.stdout: "verify_ssl": { 2024-09-19T05:15:26.568 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "verify_ssl", 2024-09-19T05:15:26.568 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.568 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.568 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.568 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "true", 2024-09-19T05:15:26.568 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.568 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.568 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.568 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T05:15:26.569 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.569 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.569 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.569 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.569 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.569 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "insights", 2024-09-19T05:15:26.569 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.569 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.569 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.569 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.569 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.569 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.570 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.570 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.570 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.570 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.570 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.570 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.570 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.570 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.570 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.570 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.570 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.570 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.570 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.570 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.571 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.571 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.571 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.571 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.571 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.571 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.571 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.571 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.571 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.571 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.571 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.571 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.571 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.572 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.572 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.572 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.572 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.572 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.572 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.572 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.572 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.572 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.572 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.572 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.572 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.572 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.572 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.573 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.573 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.573 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.573 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.573 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.573 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.573 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.573 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.573 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.573 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.573 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.573 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.573 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.574 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.574 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.574 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.574 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.574 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.574 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.574 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.574 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.574 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.574 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.574 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.574 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "iostat", 2024-09-19T05:15:26.575 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.575 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.575 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.575 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.575 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.575 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.575 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.575 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.575 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.575 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.576 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.576 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.576 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.576 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.576 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.576 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.576 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.576 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.576 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.576 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.576 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.576 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.576 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.577 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.577 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.577 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.577 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.577 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.577 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.577 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.577 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.577 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.577 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.577 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.578 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.578 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.578 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.578 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.578 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.578 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.578 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.578 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.578 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.578 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.578 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.578 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.578 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.579 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.579 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.579 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.579 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.579 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.579 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.579 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.579 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.579 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.579 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.580 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.580 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.580 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.580 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.580 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.580 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.580 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.580 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.580 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.580 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.580 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.580 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.580 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.581 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.581 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "k8sevents", 2024-09-19T05:15:26.581 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.581 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.581 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.581 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_event_retention_days": { 2024-09-19T05:15:26.581 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ceph_event_retention_days", 2024-09-19T05:15:26.581 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.581 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.582 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.582 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "7", 2024-09-19T05:15:26.582 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.582 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.582 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.582 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T05:15:26.582 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.582 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.582 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.582 INFO:teuthology.orchestra.run.smithi043.stdout: "config_check_secs": { 2024-09-19T05:15:26.582 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_check_secs", 2024-09-19T05:15:26.582 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.583 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.583 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.583 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-19T05:15:26.583 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "10", 2024-09-19T05:15:26.583 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.583 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.583 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T05:15:26.583 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.583 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.583 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.584 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.584 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.584 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.584 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.584 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.584 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.584 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.584 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.584 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.585 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.585 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.585 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.585 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.585 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.585 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.585 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.585 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.585 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.586 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.586 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.586 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.586 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.586 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.586 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.586 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.586 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.586 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.587 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.587 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.587 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.587 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.587 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.587 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.587 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.587 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.587 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.587 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.588 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.588 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.588 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.588 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.588 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.588 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.588 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.588 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.588 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.588 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.588 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.588 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.588 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.588 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.589 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.589 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.589 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.589 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.589 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.589 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.589 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.589 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.589 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.589 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.589 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.589 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.589 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.590 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.590 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.590 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.590 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.590 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "localpool", 2024-09-19T05:15:26.590 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.590 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.590 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.590 INFO:teuthology.orchestra.run.smithi043.stdout: "failure_domain": { 2024-09-19T05:15:26.590 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "failure_domain", 2024-09-19T05:15:26.590 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.590 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.590 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.591 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "host", 2024-09-19T05:15:26.591 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.591 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.591 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.591 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "failure domain for any created local pool", 2024-09-19T05:15:26.591 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T05:15:26.591 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.591 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.591 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.591 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.591 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.592 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.592 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.592 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.592 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.592 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.592 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.592 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.592 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.592 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.592 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.592 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.592 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.592 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.592 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.593 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.593 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.593 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.593 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.593 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.593 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.593 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.593 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.593 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.593 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.593 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.593 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.593 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.593 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.594 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.594 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.594 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.594 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.594 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.594 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.594 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.594 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.594 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.594 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.594 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.594 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.594 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.595 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.595 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.595 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.595 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.595 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.595 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.595 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.595 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.595 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.595 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.595 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.595 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.595 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.596 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.596 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.596 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.596 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.596 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.596 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.596 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.596 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.596 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.596 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": { 2024-09-19T05:15:26.596 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "min_size", 2024-09-19T05:15:26.596 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.597 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.597 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.597 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.597 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.597 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.597 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.597 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default min_size for any created local pool", 2024-09-19T05:15:26.597 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T05:15:26.597 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.597 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.597 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rep": { 2024-09-19T05:15:26.597 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "num_rep", 2024-09-19T05:15:26.597 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.598 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.598 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.598 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3", 2024-09-19T05:15:26.598 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.598 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.598 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.598 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default replica count for any created local pool", 2024-09-19T05:15:26.598 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.598 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.598 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.598 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": { 2024-09-19T05:15:26.599 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pg_num", 2024-09-19T05:15:26.599 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.599 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.599 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.599 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "128", 2024-09-19T05:15:26.599 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.599 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.599 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.599 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T05:15:26.599 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.599 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.599 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.599 INFO:teuthology.orchestra.run.smithi043.stdout: "prefix": { 2024-09-19T05:15:26.600 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prefix", 2024-09-19T05:15:26.600 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.600 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.600 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.600 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.600 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.600 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.600 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.600 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "name prefix for any created local pool", 2024-09-19T05:15:26.600 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.600 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.600 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.600 INFO:teuthology.orchestra.run.smithi043.stdout: "subtree": { 2024-09-19T05:15:26.601 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "subtree", 2024-09-19T05:15:26.601 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.601 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.601 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.601 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "rack", 2024-09-19T05:15:26.601 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.601 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.601 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.601 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T05:15:26.601 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T05:15:26.601 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.601 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.601 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.602 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.602 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mds_autoscaler", 2024-09-19T05:15:26.602 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.602 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.602 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.602 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.602 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.602 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.602 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.602 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.602 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.603 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.603 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.603 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.603 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.603 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.603 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.603 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.603 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.603 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.603 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.603 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.603 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.603 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.603 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.604 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.604 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.604 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.604 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.604 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.604 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.604 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.604 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.604 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.604 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.604 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.604 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.605 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.605 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.605 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.605 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.605 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.605 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.605 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.605 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.605 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.605 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.605 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.605 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.605 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.605 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.606 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.606 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.606 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.606 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.606 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.606 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.606 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.606 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.606 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.606 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.606 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.606 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.606 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.607 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.607 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.607 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.607 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.607 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.607 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.607 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.607 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.607 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.607 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mirroring", 2024-09-19T05:15:26.607 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.608 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.608 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.608 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.608 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.608 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.608 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.608 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.608 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.608 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.608 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.608 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.608 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.608 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.608 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.608 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.609 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.609 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.609 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.609 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.609 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.609 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.609 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.609 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.609 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.609 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.609 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.609 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.609 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.610 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.610 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.610 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.610 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.610 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.610 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.610 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.610 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.610 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.611 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.611 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.611 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.611 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.611 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.611 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.611 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.611 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.611 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.611 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.611 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.611 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.611 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.611 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.612 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.612 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.612 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.612 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.612 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.612 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.612 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.612 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.612 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.613 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.613 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.613 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.613 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.613 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.613 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.613 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.613 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.613 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.613 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.613 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "nfs", 2024-09-19T05:15:26.613 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.614 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.614 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.614 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.614 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.614 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.614 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.614 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.614 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.614 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.614 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.615 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.615 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.615 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.615 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.615 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.615 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.615 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.615 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.615 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.615 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.615 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.615 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.616 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.616 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.616 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.616 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.616 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.616 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.616 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.616 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.616 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.616 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.616 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.617 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.617 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.617 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.617 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.617 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.617 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.617 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.617 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.617 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.617 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.617 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.617 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.617 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.617 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.618 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.618 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.618 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.618 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.618 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.618 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.618 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.618 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.618 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.618 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.618 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.618 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.618 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.619 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.619 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.619 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.619 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.619 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.619 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.619 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.619 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.619 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.619 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.619 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "orchestrator", 2024-09-19T05:15:26.619 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.620 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.620 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.620 INFO:teuthology.orchestra.run.smithi043.stdout: "fail_fs": { 2024-09-19T05:15:26.620 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "fail_fs", 2024-09-19T05:15:26.620 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.620 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.620 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.620 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.620 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.620 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.620 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.620 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T05:15:26.620 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.621 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.621 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.621 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.621 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.621 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.621 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.621 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.621 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.621 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.621 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.622 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.622 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.622 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.622 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.622 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.622 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.622 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.622 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.622 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.623 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.623 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.623 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.623 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.623 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.623 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.623 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.623 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.623 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.624 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.624 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.624 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.624 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.624 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.624 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.624 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.624 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.624 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.624 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.625 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.625 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.625 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.625 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.625 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.625 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.625 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.625 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.625 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.625 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.625 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.625 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.625 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.625 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.625 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.626 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.626 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.626 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.626 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.626 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.626 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.626 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.626 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.626 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.626 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.626 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.626 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.626 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.626 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.627 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.627 INFO:teuthology.orchestra.run.smithi043.stdout: "orchestrator": { 2024-09-19T05:15:26.627 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "orchestrator", 2024-09-19T05:15:26.627 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.627 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.627 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.627 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.627 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.627 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.627 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.627 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm", 2024-09-19T05:15:26.627 INFO:teuthology.orchestra.run.smithi043.stdout: "rook", 2024-09-19T05:15:26.627 INFO:teuthology.orchestra.run.smithi043.stdout: "test_orchestrator" 2024-09-19T05:15:26.628 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.628 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Orchestrator backend", 2024-09-19T05:15:26.628 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.628 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.628 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.628 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.628 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.628 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "osd_perf_query", 2024-09-19T05:15:26.628 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.628 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.628 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.629 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.629 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.629 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.629 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.629 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.629 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.629 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.629 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.629 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.629 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.629 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.629 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.629 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.629 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.630 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.630 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.630 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.630 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.630 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.630 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.630 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.630 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.630 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.630 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.630 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.630 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.630 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.631 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.631 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.631 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.631 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.631 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.631 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.631 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.631 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.631 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.631 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.631 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.631 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.631 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.631 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.632 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.632 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.632 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.632 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.632 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.632 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.632 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.632 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.632 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.632 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.632 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.632 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.633 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.633 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.633 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.633 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.633 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.633 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.633 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.633 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.633 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.633 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.633 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.633 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.633 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.634 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.634 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.634 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "osd_support", 2024-09-19T05:15:26.634 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.634 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.634 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.634 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.634 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.634 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.634 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.634 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.634 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.634 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.635 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.635 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.635 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.635 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.635 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.635 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.635 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.635 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.635 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.635 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.635 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.635 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.635 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.636 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.636 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.636 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.636 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.636 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.636 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.636 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.636 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.636 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.636 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.636 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.636 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.636 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.637 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.637 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.637 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.637 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.637 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.637 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.637 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.637 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.637 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.637 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.637 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.637 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.637 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.637 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.638 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.638 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.638 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.638 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.638 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.638 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.638 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.638 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.638 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.638 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.638 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.638 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.638 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.639 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.639 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.639 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.639 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.639 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.639 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.639 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.639 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.639 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pg_autoscaler", 2024-09-19T05:15:26.639 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.639 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.639 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.640 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.640 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.640 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.640 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.640 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.640 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.640 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.640 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.640 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.640 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.640 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.640 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.640 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.640 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.641 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.641 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.641 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.641 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.641 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.641 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.641 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.641 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.641 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.641 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.641 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.641 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.641 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.641 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.642 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.642 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.642 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.642 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.642 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.642 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.642 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.642 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.642 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.642 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.642 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.642 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.642 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.643 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.643 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.643 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.643 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.643 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.643 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.643 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.643 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.643 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.643 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.643 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.643 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.643 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.644 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.644 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.644 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.644 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.644 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.644 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.644 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.644 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.644 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.644 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.644 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.644 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.645 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-09-19T05:15:26.645 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-09-19T05:15:26.645 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:26.645 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.645 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.645 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-09-19T05:15:26.645 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.645 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.645 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.645 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.645 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.645 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.645 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.646 INFO:teuthology.orchestra.run.smithi043.stdout: "threshold": { 2024-09-19T05:15:26.646 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "threshold", 2024-09-19T05:15:26.646 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-09-19T05:15:26.646 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.646 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.646 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3.0", 2024-09-19T05:15:26.646 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1.0", 2024-09-19T05:15:26.646 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.646 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.646 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "scaling threshold", 2024-09-19T05:15:26.646 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T05:15:26.646 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.646 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.646 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.647 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.647 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "progress", 2024-09-19T05:15:26.647 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.647 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.647 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.647 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_pg_recovery_event": { 2024-09-19T05:15:26.647 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_pg_recovery_event", 2024-09-19T05:15:26.647 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.648 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.648 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.648 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.648 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.648 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.648 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.648 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T05:15:26.648 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.648 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.648 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.648 INFO:teuthology.orchestra.run.smithi043.stdout: "enabled": { 2024-09-19T05:15:26.648 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enabled", 2024-09-19T05:15:26.649 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.649 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.649 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.649 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.649 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.649 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.649 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.649 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.649 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.649 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.649 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.650 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.650 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.650 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.650 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.650 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.650 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.650 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.650 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.650 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.650 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.650 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.650 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.651 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.651 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.651 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.651 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.651 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.651 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.651 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.651 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.651 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.651 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.651 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.652 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.652 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.652 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.652 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.652 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.652 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.652 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.652 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.652 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.652 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.652 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.653 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.653 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.653 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.653 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.653 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.653 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.653 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.653 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.653 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.653 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.654 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.654 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.654 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.654 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.654 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.654 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.654 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.654 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.654 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.655 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.655 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.655 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.657 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.657 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.657 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.657 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.657 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.657 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.657 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.657 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.657 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.658 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.658 INFO:teuthology.orchestra.run.smithi043.stdout: "max_completed_events": { 2024-09-19T05:15:26.658 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_completed_events", 2024-09-19T05:15:26.658 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.658 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.658 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.658 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "50", 2024-09-19T05:15:26.658 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.658 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.659 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.659 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "number of past completed events to remember", 2024-09-19T05:15:26.659 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.659 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.659 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.659 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-09-19T05:15:26.659 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-09-19T05:15:26.659 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:26.660 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.660 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.660 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-09-19T05:15:26.660 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.660 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.660 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.660 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long the module is going to sleep", 2024-09-19T05:15:26.660 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.660 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.660 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.660 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.660 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.661 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prometheus", 2024-09-19T05:15:26.661 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.661 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.661 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.661 INFO:teuthology.orchestra.run.smithi043.stdout: "cache": { 2024-09-19T05:15:26.661 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cache", 2024-09-19T05:15:26.661 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.661 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.661 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.661 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.661 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.661 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.661 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.661 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.662 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.662 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.662 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.662 INFO:teuthology.orchestra.run.smithi043.stdout: "exclude_perf_counters": { 2024-09-19T05:15:26.662 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "exclude_perf_counters", 2024-09-19T05:15:26.662 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.662 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.662 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.662 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.662 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.662 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.662 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.662 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T05:15:26.663 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T05:15:26.663 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.663 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.663 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.663 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.663 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.663 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.663 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.663 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.663 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.663 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.663 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.663 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.664 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.664 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.664 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.664 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.664 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.664 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.664 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.664 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.664 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.664 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.664 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.664 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.664 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.665 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.665 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.665 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.665 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.665 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.665 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.665 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.665 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.665 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.665 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.665 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.665 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.666 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.666 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.666 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.666 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.666 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.666 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.666 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.666 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.666 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.666 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.666 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.666 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.666 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.666 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.666 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.667 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.667 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.667 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.667 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.667 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.667 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.667 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.667 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.667 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.667 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.667 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.667 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.667 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.668 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.668 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.668 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.668 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_stats_pools": { 2024-09-19T05:15:26.668 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_stats_pools", 2024-09-19T05:15:26.668 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.668 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.668 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.668 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.668 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.668 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.668 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.669 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.669 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.669 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.669 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.669 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T05:15:26.669 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T05:15:26.669 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.669 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.669 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.669 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "300", 2024-09-19T05:15:26.669 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.669 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.669 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.670 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.670 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.670 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.670 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.670 INFO:teuthology.orchestra.run.smithi043.stdout: "scrape_interval": { 2024-09-19T05:15:26.670 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "scrape_interval", 2024-09-19T05:15:26.670 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-09-19T05:15:26.670 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.670 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.670 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15.0", 2024-09-19T05:15:26.670 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.670 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.670 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.670 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.671 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.671 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.671 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.671 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-09-19T05:15:26.671 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-09-19T05:15:26.671 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.671 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.671 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.671 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "::", 2024-09-19T05:15:26.671 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.671 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.671 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.671 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T05:15:26.671 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.672 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.672 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.672 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-09-19T05:15:26.672 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-09-19T05:15:26.672 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.672 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.672 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.672 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "9283", 2024-09-19T05:15:26.672 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.672 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.672 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.672 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T05:15:26.672 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.673 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.673 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.673 INFO:teuthology.orchestra.run.smithi043.stdout: "stale_cache_strategy": { 2024-09-19T05:15:26.673 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "stale_cache_strategy", 2024-09-19T05:15:26.673 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.673 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.673 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.673 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "log", 2024-09-19T05:15:26.673 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.673 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.673 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.673 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.674 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.674 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.674 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.674 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_behaviour": { 2024-09-19T05:15:26.674 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_behaviour", 2024-09-19T05:15:26.674 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.674 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.674 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.674 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "default", 2024-09-19T05:15:26.674 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.674 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.674 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.674 INFO:teuthology.orchestra.run.smithi043.stdout: "default", 2024-09-19T05:15:26.675 INFO:teuthology.orchestra.run.smithi043.stdout: "error" 2024-09-19T05:15:26.675 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.675 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.675 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.675 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.675 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.675 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_error_status_code": { 2024-09-19T05:15:26.675 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_error_status_code", 2024-09-19T05:15:26.675 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.675 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.675 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.675 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "500", 2024-09-19T05:15:26.675 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "400", 2024-09-19T05:15:26.675 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "599", 2024-09-19T05:15:26.676 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.676 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.676 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.676 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.676 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.676 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.676 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.676 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_support", 2024-09-19T05:15:26.676 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.676 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.676 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.676 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.677 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.677 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.677 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.677 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.677 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.677 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.677 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.677 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.677 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.677 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.677 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.677 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.677 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.677 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.677 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.678 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.678 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.678 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.678 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.678 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.678 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.678 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.678 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.678 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.678 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.678 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.678 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.678 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.678 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.679 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.679 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.679 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.679 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.679 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.679 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.679 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.679 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.679 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.679 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.679 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.679 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.679 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.680 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.680 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.680 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.680 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.680 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.680 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.680 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.680 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.680 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.680 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.680 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.680 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.681 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.681 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.681 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.681 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.681 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.681 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.681 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.681 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.681 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.681 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.681 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.681 INFO:teuthology.orchestra.run.smithi043.stdout: "max_concurrent_snap_create": { 2024-09-19T05:15:26.681 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_concurrent_snap_create", 2024-09-19T05:15:26.681 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.682 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.682 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.682 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-19T05:15:26.682 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.682 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.682 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.682 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.682 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.682 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.682 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.683 INFO:teuthology.orchestra.run.smithi043.stdout: "mirror_snapshot_schedule": { 2024-09-19T05:15:26.683 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T05:15:26.683 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.683 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.683 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.683 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.683 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.683 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.683 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.683 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.683 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.683 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.684 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.684 INFO:teuthology.orchestra.run.smithi043.stdout: "trash_purge_schedule": { 2024-09-19T05:15:26.684 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "trash_purge_schedule", 2024-09-19T05:15:26.684 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.684 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.684 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.684 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.684 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.684 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.684 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.684 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.684 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.685 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.685 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.685 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.685 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.685 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "restful", 2024-09-19T05:15:26.685 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.685 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.685 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.685 INFO:teuthology.orchestra.run.smithi043.stdout: "enable_auth": { 2024-09-19T05:15:26.686 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enable_auth", 2024-09-19T05:15:26.686 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.686 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.686 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.686 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.686 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.686 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.686 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.686 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.686 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.686 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.687 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.687 INFO:teuthology.orchestra.run.smithi043.stdout: "key_file": { 2024-09-19T05:15:26.687 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "key_file", 2024-09-19T05:15:26.687 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.687 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.687 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.687 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.687 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.687 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.687 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.687 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.687 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.688 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.688 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.688 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.688 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.688 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.688 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.688 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.688 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.688 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.689 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.689 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.689 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.689 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.689 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.689 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.689 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.689 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.689 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.689 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.689 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.690 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.690 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.690 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.690 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.690 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.690 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.690 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.690 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.690 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.690 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.691 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.691 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.691 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.691 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.691 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.691 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.691 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.691 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.691 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.691 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.691 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.691 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.692 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.692 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.692 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.692 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.692 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.692 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.692 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.692 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.692 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.692 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.692 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.692 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.692 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.693 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.693 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.693 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.693 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.693 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.693 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.693 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.693 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.693 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.693 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.693 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.693 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.693 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.694 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-09-19T05:15:26.694 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-09-19T05:15:26.694 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.694 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.694 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.694 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.694 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.694 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.694 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.695 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.695 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.695 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.695 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.695 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-09-19T05:15:26.695 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-09-19T05:15:26.695 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.695 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.695 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.696 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.696 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.696 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.696 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.696 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.696 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.696 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.696 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.696 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.696 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.696 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rgw", 2024-09-19T05:15:26.696 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.697 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.697 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.697 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.697 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.697 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.697 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.697 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.697 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.697 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.697 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.697 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.697 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.697 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.697 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.698 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.698 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.698 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.698 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.698 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.698 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.698 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.698 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.698 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.698 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.698 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.698 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.698 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.698 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.699 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.699 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.699 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.699 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.699 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.699 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.699 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.699 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.699 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.699 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.699 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.699 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.699 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.699 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.700 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.700 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.700 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.700 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.700 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.700 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.700 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.700 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.700 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.700 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.700 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.700 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.700 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.701 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.701 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.701 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.701 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.701 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.701 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.701 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.701 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.701 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.701 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.701 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.701 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.701 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.701 INFO:teuthology.orchestra.run.smithi043.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T05:15:26.702 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T05:15:26.702 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.702 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.702 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.702 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-09-19T05:15:26.702 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.702 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.702 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.702 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T05:15:26.702 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.702 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.702 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.702 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.702 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.703 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rook", 2024-09-19T05:15:26.703 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.703 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.703 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.703 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.703 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.703 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.703 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.703 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.703 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.703 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.703 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.704 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.704 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.704 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.704 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.704 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.704 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.704 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.704 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.704 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.704 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.704 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.704 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.704 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.704 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.705 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.705 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.705 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.705 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.705 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.705 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.705 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.705 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.705 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.705 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.705 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.705 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.705 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.706 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.706 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.706 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.706 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.706 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.706 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.706 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.706 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.706 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.706 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.706 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.706 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.706 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.706 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.707 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.707 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.707 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.707 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.707 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.707 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.707 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.707 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.707 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.707 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.707 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.707 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.707 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.707 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.708 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.708 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.708 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.708 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.708 INFO:teuthology.orchestra.run.smithi043.stdout: "storage_class": { 2024-09-19T05:15:26.708 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "storage_class", 2024-09-19T05:15:26.708 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.708 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.708 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.708 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "local", 2024-09-19T05:15:26.708 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.708 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.708 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.709 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T05:15:26.709 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.709 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.709 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.709 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.709 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.709 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "selftest", 2024-09-19T05:15:26.709 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.709 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.709 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.709 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.709 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.710 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.710 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.710 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.710 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.710 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.710 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.710 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.710 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.710 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.710 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.710 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.710 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.710 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.710 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.710 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.711 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.711 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.711 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.711 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.711 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.711 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.711 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.711 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.711 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.711 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.711 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.711 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.711 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.712 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.712 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.712 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.712 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.712 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.712 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.712 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.712 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.712 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.712 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.712 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.712 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.712 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.713 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.713 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.713 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.713 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.713 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.713 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.713 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.713 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.713 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.713 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.713 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.713 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.713 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.714 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.714 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.714 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.714 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.714 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.714 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.714 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.714 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.714 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.714 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.714 INFO:teuthology.orchestra.run.smithi043.stdout: "roption1": { 2024-09-19T05:15:26.714 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "roption1", 2024-09-19T05:15:26.715 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.715 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.715 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.715 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.715 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.715 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.715 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.715 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.715 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.715 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.715 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.715 INFO:teuthology.orchestra.run.smithi043.stdout: "roption2": { 2024-09-19T05:15:26.716 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "roption2", 2024-09-19T05:15:26.716 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.716 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.716 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.716 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "xyz", 2024-09-19T05:15:26.716 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.716 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.716 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.716 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.717 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.717 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.717 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.717 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption1": { 2024-09-19T05:15:26.717 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption1", 2024-09-19T05:15:26.717 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.717 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.717 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.717 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.717 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.717 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.717 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.717 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.718 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.718 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.718 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.718 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption2": { 2024-09-19T05:15:26.718 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption2", 2024-09-19T05:15:26.718 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.718 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.718 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.718 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.719 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.719 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.719 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.719 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.719 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.719 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.719 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.719 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption3": { 2024-09-19T05:15:26.719 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption3", 2024-09-19T05:15:26.719 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-09-19T05:15:26.719 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.719 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.719 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.720 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.720 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.720 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.720 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.720 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.720 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.720 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.720 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption4": { 2024-09-19T05:15:26.720 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption4", 2024-09-19T05:15:26.720 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.721 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.721 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.721 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.721 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.721 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.721 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.721 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.721 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.721 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.721 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.721 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption5": { 2024-09-19T05:15:26.722 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption5", 2024-09-19T05:15:26.722 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.722 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.722 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.722 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.722 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.722 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.722 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.722 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.722 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.722 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.723 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.723 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.723 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption6": { 2024-09-19T05:15:26.723 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption6", 2024-09-19T05:15:26.723 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.723 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.723 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.723 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.723 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.723 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.723 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.724 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.724 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.724 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.724 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.724 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption7": { 2024-09-19T05:15:26.724 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption7", 2024-09-19T05:15:26.724 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.724 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.724 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.724 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.724 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-09-19T05:15:26.725 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "42", 2024-09-19T05:15:26.725 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.725 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.725 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.725 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.725 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.725 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.725 INFO:teuthology.orchestra.run.smithi043.stdout: "testkey": { 2024-09-19T05:15:26.725 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testkey", 2024-09-19T05:15:26.726 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.726 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.726 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.726 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.726 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.726 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.726 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.726 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.726 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.727 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.727 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.727 INFO:teuthology.orchestra.run.smithi043.stdout: "testlkey": { 2024-09-19T05:15:26.727 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testlkey", 2024-09-19T05:15:26.727 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.727 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.727 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.727 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.727 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.728 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.728 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.728 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.728 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.728 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.728 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.728 INFO:teuthology.orchestra.run.smithi043.stdout: "testnewline": { 2024-09-19T05:15:26.728 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testnewline", 2024-09-19T05:15:26.729 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.729 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.729 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.729 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.729 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.729 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.729 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.729 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.729 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.729 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.729 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.729 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.729 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.729 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.730 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snap_schedule", 2024-09-19T05:15:26.730 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.730 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.730 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.730 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_m_granularity": { 2024-09-19T05:15:26.730 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_m_granularity", 2024-09-19T05:15:26.730 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.730 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.730 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.730 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.730 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.730 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.730 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.730 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T05:15:26.731 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.731 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.731 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.731 INFO:teuthology.orchestra.run.smithi043.stdout: "dump_on_update": { 2024-09-19T05:15:26.731 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "dump_on_update", 2024-09-19T05:15:26.731 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.731 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.731 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.731 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.731 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.731 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.731 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.731 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "dump database to debug log on update", 2024-09-19T05:15:26.731 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.731 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.732 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.732 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.732 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.732 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.732 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.732 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.732 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.732 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.732 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.732 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.732 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.732 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.732 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.733 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.733 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.733 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.733 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.733 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.733 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.733 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.733 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.733 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.733 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.733 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.733 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.734 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.734 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.734 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.734 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.734 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.734 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.734 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.734 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.734 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.734 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.734 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.734 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.734 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.735 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.735 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.735 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.735 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.735 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.735 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.735 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.735 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.735 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.735 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.735 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.735 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.735 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.735 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.735 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.735 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.736 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.736 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.736 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.736 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.736 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.736 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.736 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.736 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.736 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.736 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.736 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.737 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.737 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.737 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.737 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.737 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "stats", 2024-09-19T05:15:26.737 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.737 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.737 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.737 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.737 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.737 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.738 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.738 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.738 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.738 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.738 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.738 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.738 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.738 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.738 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.738 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.738 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.738 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.738 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.739 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.739 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.739 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.739 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.739 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.739 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.739 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.739 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.739 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.739 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.739 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.739 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.739 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.740 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.740 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.740 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.740 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.740 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.740 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.740 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.740 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.740 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.740 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.740 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.740 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.741 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.741 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.741 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.741 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.741 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.741 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.741 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.741 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.741 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.741 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.741 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.741 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.741 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.742 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.742 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.742 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.742 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.742 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.742 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.742 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.742 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.742 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.742 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.742 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.742 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.742 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.742 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.743 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.743 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "status", 2024-09-19T05:15:26.743 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.743 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.743 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.743 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.743 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.743 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.743 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.743 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.743 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.743 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.744 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.744 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.744 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.744 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.744 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.744 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.744 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.744 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.744 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.744 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.744 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.744 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.744 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.745 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.745 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.745 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.745 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.745 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.745 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.745 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.745 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.745 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.745 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.745 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.745 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.745 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.746 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.746 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.746 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.746 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.746 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.746 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.746 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.746 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.746 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.746 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.746 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.746 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.746 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.746 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.747 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.747 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.747 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.747 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.747 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.747 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.747 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.747 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.747 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.747 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.747 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.747 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.747 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.747 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.748 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.748 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.748 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.748 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.748 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.748 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.748 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.748 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "telegraf", 2024-09-19T05:15:26.748 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.748 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.749 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.749 INFO:teuthology.orchestra.run.smithi043.stdout: "address": { 2024-09-19T05:15:26.749 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "address", 2024-09-19T05:15:26.749 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.749 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.749 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.749 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T05:15:26.749 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.749 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.749 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.749 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.749 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.749 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.749 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.750 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-09-19T05:15:26.750 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-09-19T05:15:26.750 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:26.750 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.750 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.750 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15", 2024-09-19T05:15:26.750 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.750 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.750 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.750 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.750 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.750 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.750 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.751 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.751 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.751 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.751 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.751 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.751 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.751 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.751 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.751 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.752 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.752 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.752 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.752 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.752 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.752 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.752 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.752 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.752 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.752 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.752 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.752 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.753 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.753 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.753 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.753 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.753 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.753 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.753 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.753 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.753 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.754 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.754 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.754 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.754 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.754 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.754 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.754 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.754 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.754 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.754 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.754 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.754 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.755 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.755 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.755 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.755 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.755 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.755 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.755 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.755 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.755 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.755 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.755 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.756 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.756 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.756 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.756 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.756 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.756 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.756 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.756 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.756 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.756 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.756 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.757 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.757 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.757 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.757 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.757 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.757 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "telemetry", 2024-09-19T05:15:26.757 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.757 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.757 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.758 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_basic": { 2024-09-19T05:15:26.758 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_basic", 2024-09-19T05:15:26.758 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.758 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.758 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.758 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.758 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.758 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.758 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.758 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T05:15:26.758 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.759 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.759 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.759 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_crash": { 2024-09-19T05:15:26.759 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_crash", 2024-09-19T05:15:26.759 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.759 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.759 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.759 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.760 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.760 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.760 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.760 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T05:15:26.760 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.760 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.760 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.760 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_device": { 2024-09-19T05:15:26.760 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_device", 2024-09-19T05:15:26.760 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.760 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.761 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.761 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.761 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.761 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.761 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.761 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T05:15:26.761 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.761 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.761 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.761 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_ident": { 2024-09-19T05:15:26.761 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_ident", 2024-09-19T05:15:26.761 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.761 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.762 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.762 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.762 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.762 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.762 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.762 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T05:15:26.762 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.762 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.762 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.762 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_perf": { 2024-09-19T05:15:26.762 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_perf", 2024-09-19T05:15:26.762 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.763 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.763 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.763 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.763 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.763 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.763 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.763 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T05:15:26.763 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.763 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.764 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.764 INFO:teuthology.orchestra.run.smithi043.stdout: "contact": { 2024-09-19T05:15:26.764 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "contact", 2024-09-19T05:15:26.764 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.764 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.764 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.764 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.764 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.764 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.764 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.764 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.764 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.764 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.765 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.765 INFO:teuthology.orchestra.run.smithi043.stdout: "description": { 2024-09-19T05:15:26.765 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "description", 2024-09-19T05:15:26.765 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.765 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.765 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.765 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.765 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.765 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.765 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.765 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.765 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.765 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.766 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.766 INFO:teuthology.orchestra.run.smithi043.stdout: "device_url": { 2024-09-19T05:15:26.766 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_url", 2024-09-19T05:15:26.766 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.766 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.766 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.766 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T05:15:26.766 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.766 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.766 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.766 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.766 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.766 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.767 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.767 INFO:teuthology.orchestra.run.smithi043.stdout: "enabled": { 2024-09-19T05:15:26.767 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enabled", 2024-09-19T05:15:26.767 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.767 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.767 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.767 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.767 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.767 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.767 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.767 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.767 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.767 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.768 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.768 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-09-19T05:15:26.768 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-09-19T05:15:26.768 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.768 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.768 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.768 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "24", 2024-09-19T05:15:26.768 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "8", 2024-09-19T05:15:26.768 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.768 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.768 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.768 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.768 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.769 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.769 INFO:teuthology.orchestra.run.smithi043.stdout: "last_opt_revision": { 2024-09-19T05:15:26.769 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "last_opt_revision", 2024-09-19T05:15:26.769 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.769 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.769 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.769 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1", 2024-09-19T05:15:26.769 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.769 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.769 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.769 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.769 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.769 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.770 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.770 INFO:teuthology.orchestra.run.smithi043.stdout: "leaderboard": { 2024-09-19T05:15:26.770 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "leaderboard", 2024-09-19T05:15:26.770 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.770 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.770 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.770 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.770 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.770 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.770 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.770 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.770 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.771 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.771 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.771 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.771 INFO:teuthology.orchestra.run.smithi043.stdout: "leaderboard_description": { 2024-09-19T05:15:26.771 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "leaderboard_description", 2024-09-19T05:15:26.771 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.771 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.771 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.771 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.771 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.771 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.771 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.771 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.771 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.772 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.772 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.772 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.772 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.772 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.772 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.772 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.772 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.772 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.772 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.772 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.772 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.773 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.773 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.773 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.773 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.773 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.773 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.773 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.773 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.773 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.773 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.773 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.773 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.774 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.774 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.774 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.774 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.774 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.774 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.774 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.774 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.774 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.774 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.774 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.774 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.774 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.775 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.775 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.775 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.775 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.775 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.775 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.775 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.775 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.775 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.775 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.775 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.775 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.775 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.775 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.776 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.776 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.776 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.776 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.776 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.776 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.776 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.776 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.776 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.776 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.776 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.776 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.776 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.777 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.777 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.777 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.777 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.777 INFO:teuthology.orchestra.run.smithi043.stdout: "organization": { 2024-09-19T05:15:26.777 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "organization", 2024-09-19T05:15:26.777 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.777 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.777 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.777 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.777 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.777 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.777 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.777 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.778 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.778 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.778 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.778 INFO:teuthology.orchestra.run.smithi043.stdout: "proxy": { 2024-09-19T05:15:26.778 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "proxy", 2024-09-19T05:15:26.778 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.778 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.778 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.778 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.778 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.778 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.778 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.778 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.779 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.779 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.779 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.779 INFO:teuthology.orchestra.run.smithi043.stdout: "url": { 2024-09-19T05:15:26.779 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "url", 2024-09-19T05:15:26.779 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.779 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.779 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.779 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T05:15:26.779 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.779 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.780 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.780 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.780 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.780 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.780 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.780 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.780 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.780 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "test_orchestrator", 2024-09-19T05:15:26.780 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.780 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.780 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.780 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.781 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.781 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.781 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.781 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.781 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.781 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.781 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.781 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.781 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.781 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.781 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.781 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.781 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.782 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.782 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.782 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.782 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.782 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.782 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.782 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.782 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.782 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.782 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.782 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.782 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.782 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.783 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.783 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.783 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.783 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.783 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.783 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.783 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.783 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.783 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.783 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.783 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.783 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.783 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.784 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.784 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.784 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.784 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.784 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.784 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.784 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.784 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.784 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.784 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.784 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.784 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.784 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.785 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.785 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.785 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.785 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.785 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.785 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.785 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.785 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.785 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.785 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.785 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.785 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.785 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.785 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.786 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.786 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "volumes", 2024-09-19T05:15:26.786 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.786 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.786 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.786 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.786 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.786 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.787 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.787 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.787 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.787 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.787 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.787 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.787 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.787 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.787 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.787 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.787 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.787 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.787 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.787 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.787 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.788 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.788 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.788 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.788 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.788 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.788 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.788 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.788 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.788 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.789 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.789 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.789 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.789 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.789 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.789 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.789 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.789 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.789 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.789 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.789 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.789 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.789 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.789 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.790 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.790 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.790 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.790 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.790 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.790 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.790 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.790 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.790 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.790 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.791 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.791 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.791 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.791 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.791 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.791 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.791 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.791 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.791 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.791 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.791 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.791 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.791 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.792 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.792 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.792 INFO:teuthology.orchestra.run.smithi043.stdout: "max_concurrent_clones": { 2024-09-19T05:15:26.792 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_concurrent_clones", 2024-09-19T05:15:26.792 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.792 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.792 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.792 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "4", 2024-09-19T05:15:26.792 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.793 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.793 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.793 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T05:15:26.793 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.793 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.793 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.793 INFO:teuthology.orchestra.run.smithi043.stdout: "periodic_async_work": { 2024-09-19T05:15:26.793 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "periodic_async_work", 2024-09-19T05:15:26.793 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.793 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.794 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.794 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.794 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.794 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.794 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.794 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Periodically check for async work", 2024-09-19T05:15:26.794 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.794 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.794 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.795 INFO:teuthology.orchestra.run.smithi043.stdout: "snapshot_clone_delay": { 2024-09-19T05:15:26.795 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snapshot_clone_delay", 2024-09-19T05:15:26.795 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.795 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.795 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.795 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-09-19T05:15:26.795 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.795 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.796 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.796 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T05:15:26.796 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.796 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.796 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.796 INFO:teuthology.orchestra.run.smithi043.stdout: "snapshot_clone_no_wait": { 2024-09-19T05:15:26.796 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T05:15:26.796 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.796 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.796 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.797 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.797 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.797 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.797 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.797 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T05:15:26.797 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.797 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.797 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.797 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.797 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.797 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix", 2024-09-19T05:15:26.798 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.798 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.798 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.798 INFO:teuthology.orchestra.run.smithi043.stdout: "discovery_interval": { 2024-09-19T05:15:26.798 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "discovery_interval", 2024-09-19T05:15:26.798 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-09-19T05:15:26.798 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.798 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.798 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "100", 2024-09-19T05:15:26.798 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.798 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.798 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.798 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.798 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.799 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.799 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.799 INFO:teuthology.orchestra.run.smithi043.stdout: "identifier": { 2024-09-19T05:15:26.799 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "identifier", 2024-09-19T05:15:26.799 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.799 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.799 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.799 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.799 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.799 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.799 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.799 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.799 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.800 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.800 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.800 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-09-19T05:15:26.800 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-09-19T05:15:26.800 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:26.800 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.800 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.800 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-09-19T05:15:26.800 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.800 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.800 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.800 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.801 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.801 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.801 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.801 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.801 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.801 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.801 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.801 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.801 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.801 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.801 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.801 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.801 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.801 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.802 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.802 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.802 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.802 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.802 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.802 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.802 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.802 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.802 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.802 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.802 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.802 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.802 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.803 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.803 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.803 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.803 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.803 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.803 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.803 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.803 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.803 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.803 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.803 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.803 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.803 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.804 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.804 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.804 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.804 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.804 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.804 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.804 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.804 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.804 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.804 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.804 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.804 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.804 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.804 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.805 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.805 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.805 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.805 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.805 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.805 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.805 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.805 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.805 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.805 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.805 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.805 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.805 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.806 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.806 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.806 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_host": { 2024-09-19T05:15:26.806 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_host", 2024-09-19T05:15:26.806 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.806 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.806 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.806 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.806 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.806 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.806 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.806 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.806 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.806 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.807 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.807 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_port": { 2024-09-19T05:15:26.807 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_port", 2024-09-19T05:15:26.807 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.807 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.807 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.807 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10051", 2024-09-19T05:15:26.807 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.807 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.807 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.807 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.807 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.807 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.808 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.808 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_sender": { 2024-09-19T05:15:26.808 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_sender", 2024-09-19T05:15:26.808 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.808 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.808 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.808 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T05:15:26.808 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.808 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.808 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.808 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.808 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.809 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.809 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.809 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.809 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.809 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.809 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-19T05:15:26.809 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.809 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.809 INFO:teuthology.orchestra.run.smithi043.stdout: "modules": [ 2024-09-19T05:15:26.809 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm", 2024-09-19T05:15:26.809 INFO:teuthology.orchestra.run.smithi043.stdout: "dashboard", 2024-09-19T05:15:26.809 INFO:teuthology.orchestra.run.smithi043.stdout: "iostat", 2024-09-19T05:15:26.809 INFO:teuthology.orchestra.run.smithi043.stdout: "nfs", 2024-09-19T05:15:26.809 INFO:teuthology.orchestra.run.smithi043.stdout: "restful", 2024-09-19T05:15:26.810 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_schedule" 2024-09-19T05:15:26.810 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.810 INFO:teuthology.orchestra.run.smithi043.stdout: "available_modules": [ 2024-09-19T05:15:26.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.810 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "alerts", 2024-09-19T05:15:26.810 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.810 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.810 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.810 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-09-19T05:15:26.810 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-09-19T05:15:26.810 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:26.810 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.810 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.810 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-09-19T05:15:26.810 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.811 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.811 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.811 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T05:15:26.811 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.811 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.811 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.811 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.811 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.811 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.811 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.811 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.811 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.811 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.811 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.811 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.812 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.812 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.812 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.812 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.812 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.812 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.812 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.812 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.812 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.812 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.812 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.812 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.812 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.812 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.813 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.813 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.813 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.813 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.813 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.813 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.813 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.813 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.813 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.813 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.813 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.813 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.813 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.814 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.814 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.814 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.814 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.814 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.814 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.814 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.814 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.814 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.814 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.814 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.814 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.814 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.814 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.814 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.815 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.815 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.815 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.815 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.815 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.815 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.815 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.815 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.815 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.815 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.815 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.815 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.815 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.815 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.816 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.816 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_destination": { 2024-09-19T05:15:26.816 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_destination", 2024-09-19T05:15:26.816 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.816 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.816 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.816 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.816 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.816 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.816 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.817 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Email address to send alerts to", 2024-09-19T05:15:26.817 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.817 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.817 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.817 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_from_name": { 2024-09-19T05:15:26.817 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_from_name", 2024-09-19T05:15:26.817 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.817 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.817 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.817 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "Ceph", 2024-09-19T05:15:26.817 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.818 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.818 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.818 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Email From: name", 2024-09-19T05:15:26.818 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.818 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.818 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.818 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_host": { 2024-09-19T05:15:26.818 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_host", 2024-09-19T05:15:26.818 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.818 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.818 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.818 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.818 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.819 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.819 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.819 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP server", 2024-09-19T05:15:26.819 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.819 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.819 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.819 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.819 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_password": { 2024-09-19T05:15:26.819 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_password", 2024-09-19T05:15:26.819 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.820 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.820 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.820 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.820 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.820 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.820 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.820 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Password to authenticate with", 2024-09-19T05:15:26.820 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.820 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.820 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.820 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.820 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_port": { 2024-09-19T05:15:26.820 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_port", 2024-09-19T05:15:26.820 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.820 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.821 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.821 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "465", 2024-09-19T05:15:26.821 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.821 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.821 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.821 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP port", 2024-09-19T05:15:26.821 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.821 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.821 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.821 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_sender": { 2024-09-19T05:15:26.822 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_sender", 2024-09-19T05:15:26.822 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.822 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.822 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.822 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.822 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.822 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.822 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.822 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP envelope sender", 2024-09-19T05:15:26.822 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.822 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.822 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.823 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_ssl": { 2024-09-19T05:15:26.823 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_ssl", 2024-09-19T05:15:26.823 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.823 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.823 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.823 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.823 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.823 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.823 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.823 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T05:15:26.824 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.824 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.824 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.824 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.824 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_user": { 2024-09-19T05:15:26.824 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_user", 2024-09-19T05:15:26.824 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.824 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.824 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.824 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.824 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.824 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.824 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.825 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "User to authenticate as", 2024-09-19T05:15:26.825 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.825 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.825 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.825 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.825 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.825 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "balancer", 2024-09-19T05:15:26.825 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.825 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.826 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.826 INFO:teuthology.orchestra.run.smithi043.stdout: "active": { 2024-09-19T05:15:26.826 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "active", 2024-09-19T05:15:26.826 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.826 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.826 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.826 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.826 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.826 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.826 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.826 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T05:15:26.827 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.827 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.827 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.827 INFO:teuthology.orchestra.run.smithi043.stdout: "begin_time": { 2024-09-19T05:15:26.827 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "begin_time", 2024-09-19T05:15:26.827 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.827 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.827 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.827 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0000", 2024-09-19T05:15:26.828 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.828 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.828 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.828 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T05:15:26.828 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T05:15:26.828 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.828 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.828 INFO:teuthology.orchestra.run.smithi043.stdout: "begin_weekday": { 2024-09-19T05:15:26.828 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "begin_weekday", 2024-09-19T05:15:26.829 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-09-19T05:15:26.829 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.829 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.829 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-09-19T05:15:26.829 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0", 2024-09-19T05:15:26.829 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "6", 2024-09-19T05:15:26.829 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.829 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T05:15:26.829 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T05:15:26.829 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.829 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.829 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_max_iterations": { 2024-09-19T05:15:26.830 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_max_iterations", 2024-09-19T05:15:26.830 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-09-19T05:15:26.830 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.830 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.830 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "25", 2024-09-19T05:15:26.830 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-09-19T05:15:26.830 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "250", 2024-09-19T05:15:26.830 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.830 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T05:15:26.830 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.830 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.830 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.830 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_metrics": { 2024-09-19T05:15:26.830 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_metrics", 2024-09-19T05:15:26.831 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.831 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.831 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.831 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T05:15:26.831 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.831 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.831 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.831 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T05:15:26.831 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T05:15:26.831 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.831 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.831 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_step": { 2024-09-19T05:15:26.831 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_step", 2024-09-19T05:15:26.831 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-09-19T05:15:26.832 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.832 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.832 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0.5", 2024-09-19T05:15:26.832 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0.001", 2024-09-19T05:15:26.832 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "0.999", 2024-09-19T05:15:26.832 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.832 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "aggressiveness of optimization", 2024-09-19T05:15:26.832 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T05:15:26.832 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.832 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.832 INFO:teuthology.orchestra.run.smithi043.stdout: "end_time": { 2024-09-19T05:15:26.832 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "end_time", 2024-09-19T05:15:26.832 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.832 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.833 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.833 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "2359", 2024-09-19T05:15:26.833 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.833 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.833 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.833 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T05:15:26.833 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T05:15:26.833 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.833 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.833 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.833 INFO:teuthology.orchestra.run.smithi043.stdout: "end_weekday": { 2024-09-19T05:15:26.833 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "end_weekday", 2024-09-19T05:15:26.834 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-09-19T05:15:26.834 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.834 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.834 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-09-19T05:15:26.834 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0", 2024-09-19T05:15:26.834 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "6", 2024-09-19T05:15:26.834 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.834 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T05:15:26.834 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T05:15:26.834 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.834 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.834 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.834 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.834 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.835 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.835 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.835 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.835 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.835 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.835 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.835 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.835 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.835 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.835 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.835 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.835 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.835 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.835 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.835 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.836 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.836 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.836 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.836 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.836 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.836 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.836 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.836 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.836 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.836 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.836 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.836 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.836 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.837 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.837 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.837 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.837 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.837 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.837 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.837 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.837 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.837 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.837 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.837 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.837 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.838 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.838 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.838 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.838 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.838 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.838 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.838 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.838 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.838 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.838 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.838 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.838 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.838 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.838 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.838 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.839 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.839 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.839 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.839 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.839 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.839 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.839 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.839 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.839 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.839 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.839 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.839 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.839 INFO:teuthology.orchestra.run.smithi043.stdout: "min_score": { 2024-09-19T05:15:26.839 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "min_score", 2024-09-19T05:15:26.839 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-09-19T05:15:26.840 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.840 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.840 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-09-19T05:15:26.840 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.840 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.840 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.840 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T05:15:26.840 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.840 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.840 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.840 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.840 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": { 2024-09-19T05:15:26.840 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mode", 2024-09-19T05:15:26.840 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.840 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.841 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.841 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "upmap", 2024-09-19T05:15:26.841 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.841 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.841 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.841 INFO:teuthology.orchestra.run.smithi043.stdout: "crush-compat", 2024-09-19T05:15:26.841 INFO:teuthology.orchestra.run.smithi043.stdout: "none", 2024-09-19T05:15:26.841 INFO:teuthology.orchestra.run.smithi043.stdout: "read", 2024-09-19T05:15:26.841 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap", 2024-09-19T05:15:26.841 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap-read" 2024-09-19T05:15:26.841 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.841 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Balancer mode", 2024-09-19T05:15:26.841 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.841 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.842 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.842 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.842 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ids": { 2024-09-19T05:15:26.842 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pool_ids", 2024-09-19T05:15:26.842 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.842 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.842 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.842 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.842 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.842 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.842 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.842 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T05:15:26.842 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.842 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.843 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.843 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.843 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-09-19T05:15:26.843 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-09-19T05:15:26.843 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:26.843 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.843 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.843 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-09-19T05:15:26.843 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.843 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.843 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.843 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T05:15:26.843 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.843 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.843 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.844 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.844 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap_max_deviation": { 2024-09-19T05:15:26.844 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "upmap_max_deviation", 2024-09-19T05:15:26.844 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.844 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.844 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.844 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-09-19T05:15:26.844 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-09-19T05:15:26.844 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.844 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.844 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T05:15:26.844 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T05:15:26.844 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.844 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.845 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.845 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap_max_optimizations": { 2024-09-19T05:15:26.845 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "upmap_max_optimizations", 2024-09-19T05:15:26.845 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-09-19T05:15:26.845 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.845 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.845 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-19T05:15:26.845 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.845 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.845 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.845 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T05:15:26.845 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.845 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.845 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.846 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.846 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.846 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.846 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.846 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cephadm", 2024-09-19T05:15:26.846 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.846 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.846 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.846 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_down_multiplier": { 2024-09-19T05:15:26.846 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_down_multiplier", 2024-09-19T05:15:26.846 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-09-19T05:15:26.846 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.846 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.846 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3.0", 2024-09-19T05:15:26.847 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.847 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.847 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.847 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T05:15:26.847 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.847 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.847 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.847 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_refresh_rate": { 2024-09-19T05:15:26.847 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_refresh_rate", 2024-09-19T05:15:26.847 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:26.847 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.847 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.848 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "20", 2024-09-19T05:15:26.848 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.848 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.848 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.848 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T05:15:26.848 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.848 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.848 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.848 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.848 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_starting_port": { 2024-09-19T05:15:26.848 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_starting_port", 2024-09-19T05:15:26.848 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.848 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.848 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.848 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "4721", 2024-09-19T05:15:26.849 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.849 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.849 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.849 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T05:15:26.849 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.849 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.849 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.849 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.849 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_ptrace": { 2024-09-19T05:15:26.849 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_ptrace", 2024-09-19T05:15:26.849 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.850 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.850 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.850 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.850 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.850 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.850 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.850 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T05:15:26.850 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T05:15:26.850 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.850 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.850 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.850 INFO:teuthology.orchestra.run.smithi043.stdout: "autotune_interval": { 2024-09-19T05:15:26.851 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "autotune_interval", 2024-09-19T05:15:26.851 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:26.851 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.851 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.851 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-09-19T05:15:26.851 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.851 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.851 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.851 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T05:15:26.851 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.851 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.851 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.851 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.851 INFO:teuthology.orchestra.run.smithi043.stdout: "autotune_memory_target_ratio": { 2024-09-19T05:15:26.851 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T05:15:26.852 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-09-19T05:15:26.852 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.852 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.852 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0.7", 2024-09-19T05:15:26.852 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.852 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.852 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.852 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T05:15:26.852 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.852 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.852 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.852 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.852 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm_log_destination": { 2024-09-19T05:15:26.853 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cephadm_log_destination", 2024-09-19T05:15:26.853 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.853 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.853 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.853 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.853 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.853 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.853 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.853 INFO:teuthology.orchestra.run.smithi043.stdout: "file", 2024-09-19T05:15:26.853 INFO:teuthology.orchestra.run.smithi043.stdout: "file,syslog", 2024-09-19T05:15:26.853 INFO:teuthology.orchestra.run.smithi043.stdout: "syslog" 2024-09-19T05:15:26.853 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.853 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T05:15:26.853 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.854 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.854 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.854 INFO:teuthology.orchestra.run.smithi043.stdout: "cgroups_split": { 2024-09-19T05:15:26.854 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cgroups_split", 2024-09-19T05:15:26.854 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.854 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.854 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.854 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.854 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.854 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.854 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.854 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T05:15:26.855 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.855 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.855 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.855 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.855 INFO:teuthology.orchestra.run.smithi043.stdout: "config_checks_enabled": { 2024-09-19T05:15:26.855 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_checks_enabled", 2024-09-19T05:15:26.855 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.855 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.855 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.855 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.855 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.856 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.856 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.856 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T05:15:26.856 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.856 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.856 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.856 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.856 INFO:teuthology.orchestra.run.smithi043.stdout: "config_dashboard": { 2024-09-19T05:15:26.856 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_dashboard", 2024-09-19T05:15:26.856 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.856 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.856 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.856 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.856 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.857 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.857 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.857 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T05:15:26.857 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.857 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.857 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.857 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.857 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_alertmanager": { 2024-09-19T05:15:26.857 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_alertmanager", 2024-09-19T05:15:26.858 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.858 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.858 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.858 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T05:15:26.858 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.858 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.858 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.858 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-09-19T05:15:26.858 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.858 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.858 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.858 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.858 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_base": { 2024-09-19T05:15:26.859 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_base", 2024-09-19T05:15:26.859 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.859 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.859 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.859 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T05:15:26.859 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.859 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.859 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.859 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Container image name, without the tag", 2024-09-19T05:15:26.859 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.860 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.860 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.860 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.860 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_elasticsearch": { 2024-09-19T05:15:26.860 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_elasticsearch", 2024-09-19T05:15:26.860 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.860 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.860 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.860 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T05:15:26.860 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.860 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.861 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.861 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "elasticsearch container image", 2024-09-19T05:15:26.861 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.861 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.861 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.861 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.861 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_grafana": { 2024-09-19T05:15:26.861 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_grafana", 2024-09-19T05:15:26.861 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.861 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.861 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.862 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T05:15:26.862 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.862 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.862 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.862 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-09-19T05:15:26.862 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.862 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.862 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.862 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.862 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_haproxy": { 2024-09-19T05:15:26.862 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_haproxy", 2024-09-19T05:15:26.863 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.863 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.863 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.863 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T05:15:26.863 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.863 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.863 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.863 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "HAproxy container image", 2024-09-19T05:15:26.863 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.863 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.863 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.864 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.864 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_agent": { 2024-09-19T05:15:26.864 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_agent", 2024-09-19T05:15:26.864 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.864 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.864 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.864 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T05:15:26.864 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.864 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.865 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.865 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger agent container image", 2024-09-19T05:15:26.865 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.865 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.865 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.865 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.865 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_collector": { 2024-09-19T05:15:26.865 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_collector", 2024-09-19T05:15:26.865 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.865 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.865 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.865 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T05:15:26.866 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.866 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.866 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.866 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger collector container image", 2024-09-19T05:15:26.866 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.866 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.866 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.866 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.866 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_query": { 2024-09-19T05:15:26.866 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_query", 2024-09-19T05:15:26.866 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.866 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.866 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.866 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T05:15:26.866 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.867 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.867 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.867 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger query container image", 2024-09-19T05:15:26.867 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.867 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.867 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.867 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_keepalived": { 2024-09-19T05:15:26.867 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_keepalived", 2024-09-19T05:15:26.867 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.867 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.867 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.868 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T05:15:26.868 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.868 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.868 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.868 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Keepalived container image", 2024-09-19T05:15:26.868 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.868 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.868 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.868 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.868 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_loki": { 2024-09-19T05:15:26.868 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_loki", 2024-09-19T05:15:26.868 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.868 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.868 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.868 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T05:15:26.869 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.869 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.869 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.869 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Loki container image", 2024-09-19T05:15:26.869 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.869 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.869 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.869 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.869 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_node_exporter": { 2024-09-19T05:15:26.869 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_node_exporter", 2024-09-19T05:15:26.869 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.869 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.869 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.869 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T05:15:26.869 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.870 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.870 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.870 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-09-19T05:15:26.870 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.870 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.870 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.870 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.870 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_nvmeof": { 2024-09-19T05:15:26.870 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_nvmeof", 2024-09-19T05:15:26.870 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.870 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.870 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.870 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T05:15:26.870 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.870 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.871 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.871 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Nvme-of container image", 2024-09-19T05:15:26.871 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.871 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.871 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.871 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.871 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_prometheus": { 2024-09-19T05:15:26.871 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_prometheus", 2024-09-19T05:15:26.871 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.871 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.871 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.871 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T05:15:26.871 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.871 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.871 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.872 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-09-19T05:15:26.872 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.872 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.872 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.872 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_promtail": { 2024-09-19T05:15:26.872 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_promtail", 2024-09-19T05:15:26.872 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.872 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.872 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.872 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T05:15:26.872 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.872 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.873 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.873 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Promtail container image", 2024-09-19T05:15:26.873 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.873 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.873 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.873 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.873 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_samba": { 2024-09-19T05:15:26.873 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_samba", 2024-09-19T05:15:26.873 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.873 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.873 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.873 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T05:15:26.873 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.873 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.874 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.874 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Samba/SMB container image", 2024-09-19T05:15:26.874 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.874 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.874 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.874 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.874 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_snmp_gateway": { 2024-09-19T05:15:26.874 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_snmp_gateway", 2024-09-19T05:15:26.874 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.874 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.874 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.874 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T05:15:26.874 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.874 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.875 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.875 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SNMP Gateway container image", 2024-09-19T05:15:26.875 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.875 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.875 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.875 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.875 INFO:teuthology.orchestra.run.smithi043.stdout: "container_init": { 2024-09-19T05:15:26.875 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_init", 2024-09-19T05:15:26.875 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.875 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.875 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.875 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.875 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.875 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.876 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.876 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T05:15:26.876 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.876 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.876 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.876 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.876 INFO:teuthology.orchestra.run.smithi043.stdout: "daemon_cache_timeout": { 2024-09-19T05:15:26.876 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "daemon_cache_timeout", 2024-09-19T05:15:26.876 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:26.876 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.876 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.876 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-09-19T05:15:26.876 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.876 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.877 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.877 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T05:15:26.877 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.877 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.877 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.877 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.877 INFO:teuthology.orchestra.run.smithi043.stdout: "default_cephadm_command_timeout": { 2024-09-19T05:15:26.877 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T05:15:26.877 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.877 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.877 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.877 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "900", 2024-09-19T05:15:26.877 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.877 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.877 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.878 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T05:15:26.878 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.878 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.878 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.878 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.878 INFO:teuthology.orchestra.run.smithi043.stdout: "default_registry": { 2024-09-19T05:15:26.878 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "default_registry", 2024-09-19T05:15:26.878 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.878 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.878 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.878 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io", 2024-09-19T05:15:26.878 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.878 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.878 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.879 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T05:15:26.879 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.879 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.879 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.879 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.879 INFO:teuthology.orchestra.run.smithi043.stdout: "device_cache_timeout": { 2024-09-19T05:15:26.879 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_cache_timeout", 2024-09-19T05:15:26.879 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:26.879 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.879 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.879 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1800", 2024-09-19T05:15:26.879 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.879 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.879 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.879 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache device inventory", 2024-09-19T05:15:26.880 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.880 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.880 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.880 INFO:teuthology.orchestra.run.smithi043.stdout: "device_enhanced_scan": { 2024-09-19T05:15:26.880 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_enhanced_scan", 2024-09-19T05:15:26.880 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.880 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.880 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.880 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.880 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.880 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.880 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.880 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T05:15:26.881 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.881 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.881 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.881 INFO:teuthology.orchestra.run.smithi043.stdout: "facts_cache_timeout": { 2024-09-19T05:15:26.881 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "facts_cache_timeout", 2024-09-19T05:15:26.881 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:26.881 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.881 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.881 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-09-19T05:15:26.881 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.881 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.881 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.881 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache host facts data", 2024-09-19T05:15:26.881 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.882 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.882 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.882 INFO:teuthology.orchestra.run.smithi043.stdout: "grafana_dashboards_path": { 2024-09-19T05:15:26.882 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "grafana_dashboards_path", 2024-09-19T05:15:26.882 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.882 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.882 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.882 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T05:15:26.882 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.882 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.882 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.882 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T05:15:26.882 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.882 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.883 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.883 INFO:teuthology.orchestra.run.smithi043.stdout: "host_check_interval": { 2024-09-19T05:15:26.883 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "host_check_interval", 2024-09-19T05:15:26.883 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:26.883 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.883 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.883 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-09-19T05:15:26.883 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.883 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.883 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.883 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to perform a host check", 2024-09-19T05:15:26.883 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.883 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.883 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.884 INFO:teuthology.orchestra.run.smithi043.stdout: "hw_monitoring": { 2024-09-19T05:15:26.884 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "hw_monitoring", 2024-09-19T05:15:26.884 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.884 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.884 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.884 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.884 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.884 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.884 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.884 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T05:15:26.884 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.884 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.884 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.885 INFO:teuthology.orchestra.run.smithi043.stdout: "inventory_list_all": { 2024-09-19T05:15:26.885 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "inventory_list_all", 2024-09-19T05:15:26.885 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.885 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.885 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.885 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.885 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.885 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.885 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.885 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T05:15:26.885 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.886 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.886 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.886 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.886 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.886 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.886 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.886 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.886 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.886 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.886 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.886 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.886 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.886 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.886 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.887 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.887 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.887 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.887 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.887 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.887 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.887 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.887 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.887 INFO:teuthology.orchestra.run.smithi043.stdout: "log_refresh_metadata": { 2024-09-19T05:15:26.888 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_refresh_metadata", 2024-09-19T05:15:26.888 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.888 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.888 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.888 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.888 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.888 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.888 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.888 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T05:15:26.888 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.888 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.888 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.889 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.889 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.889 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.889 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.889 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.889 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.889 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.889 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.889 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.889 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T05:15:26.889 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.890 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.890 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.890 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.890 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.890 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.890 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.890 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.890 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.890 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.890 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.890 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.891 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.891 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.891 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.891 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.891 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.891 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.891 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.891 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.891 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.891 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.891 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.892 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.892 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.892 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.892 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.892 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.892 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.892 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.892 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.892 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.892 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.892 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.892 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.892 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.893 INFO:teuthology.orchestra.run.smithi043.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T05:15:26.893 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T05:15:26.893 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.893 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.893 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.893 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.893 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.893 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.894 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.894 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T05:15:26.894 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.894 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.894 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.894 INFO:teuthology.orchestra.run.smithi043.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T05:15:26.894 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T05:15:26.894 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.894 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.894 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.894 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "*", 2024-09-19T05:15:26.894 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.895 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.895 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.895 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T05:15:26.895 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.895 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.895 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.895 INFO:teuthology.orchestra.run.smithi043.stdout: "max_count_per_host": { 2024-09-19T05:15:26.895 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_count_per_host", 2024-09-19T05:15:26.895 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.896 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.896 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.896 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-19T05:15:26.896 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.896 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.896 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.896 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "max number of daemons per service per host", 2024-09-19T05:15:26.896 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.896 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.896 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.897 INFO:teuthology.orchestra.run.smithi043.stdout: "max_osd_draining_count": { 2024-09-19T05:15:26.897 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_osd_draining_count", 2024-09-19T05:15:26.897 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.897 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.897 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.897 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-19T05:15:26.897 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.897 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.897 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.898 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T05:15:26.898 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.898 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.898 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.898 INFO:teuthology.orchestra.run.smithi043.stdout: "migration_current": { 2024-09-19T05:15:26.898 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "migration_current", 2024-09-19T05:15:26.898 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.898 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.898 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.898 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.898 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.898 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.898 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.898 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "internal - do not modify", 2024-09-19T05:15:26.899 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.899 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.899 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.899 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": { 2024-09-19T05:15:26.899 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mode", 2024-09-19T05:15:26.899 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.899 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.899 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.899 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "root", 2024-09-19T05:15:26.899 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.899 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.899 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.899 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm-package", 2024-09-19T05:15:26.899 INFO:teuthology.orchestra.run.smithi043.stdout: "root" 2024-09-19T05:15:26.899 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.899 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T05:15:26.900 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.900 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.900 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.900 INFO:teuthology.orchestra.run.smithi043.stdout: "oob_default_addr": { 2024-09-19T05:15:26.900 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "oob_default_addr", 2024-09-19T05:15:26.900 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.900 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.900 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.900 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "169.254.1.1", 2024-09-19T05:15:26.900 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.900 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.900 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.900 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T05:15:26.900 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.901 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.901 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.901 INFO:teuthology.orchestra.run.smithi043.stdout: "prometheus_alerts_path": { 2024-09-19T05:15:26.901 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prometheus_alerts_path", 2024-09-19T05:15:26.901 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.901 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.901 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.901 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T05:15:26.901 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.901 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.901 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.901 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T05:15:26.901 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.901 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.901 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.902 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_insecure": { 2024-09-19T05:15:26.902 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_insecure", 2024-09-19T05:15:26.902 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.902 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.902 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.902 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.902 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.902 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.902 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.902 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T05:15:26.902 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.902 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.902 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.903 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_password": { 2024-09-19T05:15:26.903 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_password", 2024-09-19T05:15:26.903 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.903 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.903 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.903 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.903 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.903 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.903 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.903 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T05:15:26.903 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.903 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.903 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.904 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_url": { 2024-09-19T05:15:26.904 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_url", 2024-09-19T05:15:26.904 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.904 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.904 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.904 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.904 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.904 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.904 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.904 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T05:15:26.904 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.904 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.904 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.904 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_username": { 2024-09-19T05:15:26.905 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_username", 2024-09-19T05:15:26.905 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.905 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.905 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.905 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.905 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.905 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.905 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.905 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T05:15:26.905 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.905 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.905 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.905 INFO:teuthology.orchestra.run.smithi043.stdout: "secure_monitoring_stack": { 2024-09-19T05:15:26.905 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "secure_monitoring_stack", 2024-09-19T05:15:26.905 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.906 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.906 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.906 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.906 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.906 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.906 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.906 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T05:15:26.906 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.906 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.906 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.906 INFO:teuthology.orchestra.run.smithi043.stdout: "service_discovery_port": { 2024-09-19T05:15:26.906 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "service_discovery_port", 2024-09-19T05:15:26.906 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.906 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.907 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.907 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8765", 2024-09-19T05:15:26.907 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.907 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.907 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.907 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "cephadm service discovery port", 2024-09-19T05:15:26.907 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.907 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.907 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.907 INFO:teuthology.orchestra.run.smithi043.stdout: "ssh_config_file": { 2024-09-19T05:15:26.907 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssh_config_file", 2024-09-19T05:15:26.907 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.907 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.907 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.908 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.908 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.908 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.908 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.908 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T05:15:26.908 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.908 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.908 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.908 INFO:teuthology.orchestra.run.smithi043.stdout: "use_agent": { 2024-09-19T05:15:26.908 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "use_agent", 2024-09-19T05:15:26.908 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.908 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.908 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.908 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.909 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.909 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.909 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.909 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T05:15:26.909 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.909 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.909 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.909 INFO:teuthology.orchestra.run.smithi043.stdout: "use_repo_digest": { 2024-09-19T05:15:26.909 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "use_repo_digest", 2024-09-19T05:15:26.909 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.909 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.909 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.909 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.910 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.910 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.910 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.910 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T05:15:26.910 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.910 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.910 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.910 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_failed_host_check": { 2024-09-19T05:15:26.910 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_failed_host_check", 2024-09-19T05:15:26.910 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.910 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.910 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.910 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.911 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.911 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.911 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.911 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T05:15:26.911 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.911 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.911 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.911 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_stray_daemons": { 2024-09-19T05:15:26.911 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_stray_daemons", 2024-09-19T05:15:26.911 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.911 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.911 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.912 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.912 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.912 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.912 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.912 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T05:15:26.912 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.912 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.912 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.912 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_stray_hosts": { 2024-09-19T05:15:26.912 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_stray_hosts", 2024-09-19T05:15:26.912 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.912 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.913 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.913 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.913 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.913 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.913 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.913 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T05:15:26.913 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.913 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.913 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.913 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.913 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.913 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crash", 2024-09-19T05:15:26.914 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.914 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.914 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.914 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.914 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.914 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.914 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.914 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.914 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.914 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.914 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.914 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.914 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.914 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.914 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.915 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.915 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.915 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.915 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.915 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.915 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.915 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.915 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.915 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.915 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.915 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.915 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.915 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.915 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.915 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.916 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.916 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.916 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.916 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.916 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.916 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.916 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.916 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.916 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.916 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.916 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.916 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.916 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.917 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.917 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.917 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.917 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.917 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.917 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.917 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.917 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.917 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.917 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.917 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.917 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.917 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.918 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.918 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.918 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.918 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.918 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.918 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.918 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.918 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.918 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.918 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.918 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.918 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.919 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.919 INFO:teuthology.orchestra.run.smithi043.stdout: "retain_interval": { 2024-09-19T05:15:26.919 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "retain_interval", 2024-09-19T05:15:26.919 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:26.919 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.919 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.919 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "31536000", 2024-09-19T05:15:26.919 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.919 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.919 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.919 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T05:15:26.919 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.919 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.920 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.920 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_recent_interval": { 2024-09-19T05:15:26.920 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_recent_interval", 2024-09-19T05:15:26.920 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:26.920 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.920 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.920 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1209600", 2024-09-19T05:15:26.920 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.920 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.920 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.920 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T05:15:26.921 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.921 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.921 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.921 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.921 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.921 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "dashboard", 2024-09-19T05:15:26.921 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.921 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.921 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.922 INFO:teuthology.orchestra.run.smithi043.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T05:15:26.922 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T05:15:26.922 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.922 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.922 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.922 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-19T05:15:26.922 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.922 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.922 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.922 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.922 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.922 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.922 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.923 INFO:teuthology.orchestra.run.smithi043.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T05:15:26.923 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T05:15:26.923 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.923 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.923 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.923 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.923 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.923 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.923 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.923 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.923 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.923 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.923 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.924 INFO:teuthology.orchestra.run.smithi043.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T05:15:26.924 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T05:15:26.924 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.924 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.924 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.924 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.924 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.924 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.924 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.924 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.924 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.924 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.924 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.925 INFO:teuthology.orchestra.run.smithi043.stdout: "AUDIT_API_ENABLED": { 2024-09-19T05:15:26.925 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T05:15:26.925 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.925 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.925 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.925 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.925 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.925 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.925 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.925 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.925 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.925 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.925 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.926 INFO:teuthology.orchestra.run.smithi043.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T05:15:26.926 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T05:15:26.926 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.926 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.926 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.926 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.926 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.926 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.926 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.926 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.926 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.926 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.926 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.927 INFO:teuthology.orchestra.run.smithi043.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T05:15:26.927 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T05:15:26.927 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.927 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.927 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.927 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.927 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.927 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.927 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.927 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.927 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.927 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.927 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.928 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T05:15:26.928 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T05:15:26.928 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.928 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.928 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.928 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.928 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.928 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.928 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.928 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.929 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.929 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.929 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.929 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T05:15:26.929 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T05:15:26.929 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.929 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.929 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.929 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.929 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.929 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.929 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.930 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.930 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.930 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.930 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.930 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T05:15:26.930 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T05:15:26.930 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.930 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.930 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.930 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.931 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.931 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.931 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.931 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.931 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.931 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.931 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.931 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T05:15:26.931 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T05:15:26.932 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.932 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.932 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.932 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.932 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.932 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.932 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.932 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.932 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.932 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.932 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.933 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T05:15:26.933 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T05:15:26.933 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.933 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.933 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.933 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.933 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.933 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.933 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.934 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.934 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.934 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.934 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.934 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T05:15:26.934 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T05:15:26.934 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.934 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.934 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.934 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.935 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.935 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.935 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.935 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.935 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.935 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.935 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.935 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T05:15:26.935 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T05:15:26.935 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.935 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.935 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.935 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.936 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.936 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.936 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.936 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.936 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.936 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.936 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.936 INFO:teuthology.orchestra.run.smithi043.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T05:15:26.936 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T05:15:26.936 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.936 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.936 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.936 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.937 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.937 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.937 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.937 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.937 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.937 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.937 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.937 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T05:15:26.937 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T05:15:26.937 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.937 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.937 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.937 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-09-19T05:15:26.938 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.938 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.938 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.938 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.938 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.938 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.938 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.938 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T05:15:26.938 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T05:15:26.938 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.938 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.938 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.938 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.939 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.939 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.939 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.939 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.939 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.939 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.939 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.939 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_URL": { 2024-09-19T05:15:26.939 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_URL", 2024-09-19T05:15:26.939 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.939 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.939 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.939 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.940 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.940 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.940 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.940 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.940 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.940 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.940 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.940 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T05:15:26.940 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T05:15:26.940 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.940 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.940 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.940 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-09-19T05:15:26.941 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.941 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.941 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.941 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.941 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.941 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.941 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.941 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T05:15:26.941 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T05:15:26.941 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.941 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.941 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.941 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.942 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.942 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.942 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.942 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.942 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.942 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.942 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.942 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T05:15:26.942 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T05:15:26.942 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.942 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.943 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.943 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.943 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.943 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.943 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.943 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.943 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.943 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.943 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.943 INFO:teuthology.orchestra.run.smithi043.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T05:15:26.944 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T05:15:26.944 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.944 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.944 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.944 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.944 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.944 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.944 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.944 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.944 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.944 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.944 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.945 INFO:teuthology.orchestra.run.smithi043.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T05:15:26.945 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T05:15:26.945 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.945 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.945 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.945 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.945 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.945 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.945 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.945 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.946 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.946 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.946 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.946 INFO:teuthology.orchestra.run.smithi043.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T05:15:26.946 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T05:15:26.946 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.946 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.946 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.946 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.946 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.946 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.947 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.947 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.947 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.947 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.947 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.947 INFO:teuthology.orchestra.run.smithi043.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T05:15:26.947 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T05:15:26.947 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.947 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.947 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.947 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.947 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.947 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.948 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.948 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.948 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.948 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.948 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.948 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T05:15:26.948 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T05:15:26.948 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.948 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.948 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.948 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.948 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.949 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.949 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.949 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.949 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.949 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.949 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.949 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T05:15:26.949 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T05:15:26.949 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.949 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.949 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.949 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.950 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.950 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.950 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.950 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.950 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.950 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.950 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.950 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T05:15:26.950 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T05:15:26.950 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.950 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.950 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.950 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.950 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T05:15:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T05:15:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.952 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.952 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.952 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.952 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.952 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.952 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T05:15:26.952 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T05:15:26.952 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.952 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.952 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.952 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.952 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.952 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.953 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.953 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.953 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.953 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.953 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.953 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T05:15:26.953 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T05:15:26.953 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.953 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.953 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.953 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.953 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.954 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.954 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.954 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.954 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.954 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.954 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.954 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T05:15:26.954 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T05:15:26.954 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.954 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.954 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.954 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.954 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.955 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.955 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.955 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.955 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.955 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.955 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.955 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T05:15:26.955 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T05:15:26.955 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.955 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.955 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.955 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.955 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.956 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.956 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.956 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.956 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.956 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.956 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.956 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T05:15:26.956 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T05:15:26.956 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.956 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.956 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.956 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T05:15:26.956 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.957 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.957 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.957 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.957 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.957 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.957 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.957 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T05:15:26.957 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T05:15:26.957 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.958 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.958 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.958 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-19T05:15:26.958 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.958 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.958 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.958 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.958 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.958 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.958 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.958 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T05:15:26.958 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T05:15:26.958 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.959 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.959 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.959 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8", 2024-09-19T05:15:26.959 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.959 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.959 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.959 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.959 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.959 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.959 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.960 INFO:teuthology.orchestra.run.smithi043.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T05:15:26.960 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T05:15:26.960 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.960 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.960 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.960 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "45", 2024-09-19T05:15:26.960 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.960 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.960 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.960 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.960 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.961 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.961 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.961 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T05:15:26.961 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T05:15:26.961 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.961 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.961 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.961 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.961 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.961 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.962 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.962 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.962 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.962 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.962 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.962 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T05:15:26.962 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T05:15:26.962 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.962 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.962 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.963 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-09-19T05:15:26.963 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.963 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.963 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.963 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.963 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.963 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.963 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.963 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T05:15:26.964 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T05:15:26.964 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.964 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.964 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.964 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.964 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.964 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.964 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.964 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.964 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.965 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.965 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.965 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T05:15:26.965 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T05:15:26.965 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.965 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.965 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.965 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.966 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.966 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.966 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.966 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.966 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.966 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.966 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.966 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T05:15:26.967 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T05:15:26.967 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.967 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.967 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.967 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-09-19T05:15:26.967 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.967 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.967 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.967 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.967 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.967 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.968 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.968 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T05:15:26.968 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T05:15:26.968 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.968 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.968 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.968 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-19T05:15:26.968 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.968 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.968 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.969 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.969 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.969 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.969 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.969 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T05:15:26.969 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T05:15:26.969 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.969 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.969 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.969 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-09-19T05:15:26.970 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.970 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.970 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.970 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.970 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.970 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.970 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.970 INFO:teuthology.orchestra.run.smithi043.stdout: "cross_origin_url": { 2024-09-19T05:15:26.970 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cross_origin_url", 2024-09-19T05:15:26.970 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.970 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.970 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.971 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.971 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.971 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.971 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.971 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.971 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.971 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.971 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.971 INFO:teuthology.orchestra.run.smithi043.stdout: "crt_file": { 2024-09-19T05:15:26.971 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crt_file", 2024-09-19T05:15:26.971 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.971 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.971 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.971 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.972 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.972 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.972 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.972 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.972 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.972 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.972 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.972 INFO:teuthology.orchestra.run.smithi043.stdout: "debug": { 2024-09-19T05:15:26.972 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "debug", 2024-09-19T05:15:26.972 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.972 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.972 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.972 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.973 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.973 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.973 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.973 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable/disable debug options", 2024-09-19T05:15:26.973 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.973 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.973 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.973 INFO:teuthology.orchestra.run.smithi043.stdout: "jwt_token_ttl": { 2024-09-19T05:15:26.973 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "jwt_token_ttl", 2024-09-19T05:15:26.973 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.973 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.974 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.974 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "28800", 2024-09-19T05:15:26.974 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.974 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.974 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.974 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.974 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.974 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.974 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.974 INFO:teuthology.orchestra.run.smithi043.stdout: "key_file": { 2024-09-19T05:15:26.974 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "key_file", 2024-09-19T05:15:26.974 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.975 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.975 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.975 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.975 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.975 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.975 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.975 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.975 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.975 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.975 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.975 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.975 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.975 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.976 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.976 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.976 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.976 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.976 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.976 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.976 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.976 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.976 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.976 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.976 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.976 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.976 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.976 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.976 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.977 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.977 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.977 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.977 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.977 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.977 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.977 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.977 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.977 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.977 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.977 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.977 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.977 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.978 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.978 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.978 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.978 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.978 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.978 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.978 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.978 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.978 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.978 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.978 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.978 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.978 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.978 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.979 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.979 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.979 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.979 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.979 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.979 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.979 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.979 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.979 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.979 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.979 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.979 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.980 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.980 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.980 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.980 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.980 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.980 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.980 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.980 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.980 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.980 INFO:teuthology.orchestra.run.smithi043.stdout: "motd": { 2024-09-19T05:15:26.980 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "motd", 2024-09-19T05:15:26.980 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.980 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.981 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.981 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.981 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.981 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.981 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.981 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "The message of the day", 2024-09-19T05:15:26.981 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.981 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.981 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.981 INFO:teuthology.orchestra.run.smithi043.stdout: "redirect_resolve_ip_addr": { 2024-09-19T05:15:26.981 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T05:15:26.981 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.981 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-09-19T05:15:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-09-19T05:15:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "::", 2024-09-19T05:15:26.983 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.983 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.983 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.983 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.983 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.983 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.983 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.983 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-09-19T05:15:26.983 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-09-19T05:15:26.983 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.983 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.983 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.983 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8080", 2024-09-19T05:15:26.984 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.984 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.984 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.984 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.984 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.984 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.984 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.984 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl": { 2024-09-19T05:15:26.984 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl", 2024-09-19T05:15:26.984 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.984 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.984 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.984 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.985 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.985 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.985 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.985 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.985 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.985 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.985 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.985 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl_server_port": { 2024-09-19T05:15:26.985 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl_server_port", 2024-09-19T05:15:26.985 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.985 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.985 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.986 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8443", 2024-09-19T05:15:26.986 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.986 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.986 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.986 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.986 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.986 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.986 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.986 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_behaviour": { 2024-09-19T05:15:26.986 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_behaviour", 2024-09-19T05:15:26.986 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.986 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.986 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.987 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "redirect", 2024-09-19T05:15:26.987 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.987 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.987 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.987 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.987 INFO:teuthology.orchestra.run.smithi043.stdout: "redirect" 2024-09-19T05:15:26.987 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.987 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.987 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.988 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.988 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.988 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_error_status_code": { 2024-09-19T05:15:26.988 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_error_status_code", 2024-09-19T05:15:26.988 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:26.988 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.988 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.988 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "500", 2024-09-19T05:15:26.988 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "400", 2024-09-19T05:15:26.988 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "599", 2024-09-19T05:15:26.988 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.989 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.989 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.989 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.989 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.989 INFO:teuthology.orchestra.run.smithi043.stdout: "url_prefix": { 2024-09-19T05:15:26.989 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "url_prefix", 2024-09-19T05:15:26.989 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.989 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.989 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:26.989 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "devicehealth", 2024-09-19T05:15:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: "enable_monitoring": { 2024-09-19T05:15:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enable_monitoring", 2024-09-19T05:15:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "monitor device health metrics", 2024-09-19T05:15:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: "mark_out_threshold": { 2024-09-19T05:15:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mark_out_threshold", 2024-09-19T05:15:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "2419200", 2024-09-19T05:15:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T05:15:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": { 2024-09-19T05:15:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pool_name", 2024-09-19T05:15:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "device_health_metrics", 2024-09-19T05:15:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T05:15:27.000 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.000 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.000 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.000 INFO:teuthology.orchestra.run.smithi043.stdout: "retention_period": { 2024-09-19T05:15:27.000 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "retention_period", 2024-09-19T05:15:27.000 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:27.000 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.000 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.000 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15552000", 2024-09-19T05:15:27.001 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.001 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.001 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.001 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long to retain device health metrics", 2024-09-19T05:15:27.001 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.001 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.001 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.001 INFO:teuthology.orchestra.run.smithi043.stdout: "scrape_frequency": { 2024-09-19T05:15:27.001 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "scrape_frequency", 2024-09-19T05:15:27.001 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:27.001 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.002 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.002 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "86400", 2024-09-19T05:15:27.002 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.002 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.002 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.002 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T05:15:27.002 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.002 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.002 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: "self_heal": { 2024-09-19T05:15:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "self_heal", 2024-09-19T05:15:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.004 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T05:15:27.004 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.004 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.004 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.004 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-09-19T05:15:27.004 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-09-19T05:15:27.004 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:27.004 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.004 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.004 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-09-19T05:15:27.004 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.004 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.004 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.004 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T05:15:27.005 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.005 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.005 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.005 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_threshold": { 2024-09-19T05:15:27.005 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_threshold", 2024-09-19T05:15:27.005 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:27.005 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.005 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.005 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "7257600", 2024-09-19T05:15:27.005 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.005 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.005 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.005 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T05:15:27.006 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.006 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.006 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.006 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.006 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:27.006 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "diskprediction_local", 2024-09-19T05:15:27.006 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:27.006 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:27.006 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:27.006 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:27.006 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:27.006 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.006 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.007 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.007 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.007 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.007 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.007 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.007 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.007 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.007 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.007 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.007 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.007 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.007 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.007 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.007 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.008 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.008 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.008 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:27.008 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:27.008 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.008 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.008 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.008 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.008 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.008 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.008 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.008 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.008 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.008 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.009 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.009 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:27.009 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:27.009 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.009 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.009 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.009 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:27.009 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.009 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.010 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.010 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.010 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.010 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.010 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.010 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.010 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.010 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.010 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.010 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.010 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.010 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.010 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:27.011 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:27.011 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.011 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.011 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.011 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.011 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.011 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.011 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.011 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.011 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.011 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.011 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.011 INFO:teuthology.orchestra.run.smithi043.stdout: "predict_interval": { 2024-09-19T05:15:27.011 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "predict_interval", 2024-09-19T05:15:27.012 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.012 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.012 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.012 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "86400", 2024-09-19T05:15:27.012 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.012 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.012 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.012 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.012 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.012 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.012 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.012 INFO:teuthology.orchestra.run.smithi043.stdout: "predictor_model": { 2024-09-19T05:15:27.012 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "predictor_model", 2024-09-19T05:15:27.013 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.013 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.013 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.013 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "prophetstor", 2024-09-19T05:15:27.013 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.013 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.013 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.013 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.013 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.013 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.013 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.013 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-09-19T05:15:27.013 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-09-19T05:15:27.014 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.014 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.014 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.014 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-09-19T05:15:27.014 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.014 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.014 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.014 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.014 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.014 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.014 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.014 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.014 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:27.015 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "influx", 2024-09-19T05:15:27.015 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": false, 2024-09-19T05:15:27.015 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "influxdb python module not found", 2024-09-19T05:15:27.015 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:27.015 INFO:teuthology.orchestra.run.smithi043.stdout: "batch_size": { 2024-09-19T05:15:27.015 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "batch_size", 2024-09-19T05:15:27.015 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:27.015 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.015 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.015 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5000", 2024-09-19T05:15:27.015 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.015 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.016 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.016 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T05:15:27.016 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.016 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.016 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.016 INFO:teuthology.orchestra.run.smithi043.stdout: "database": { 2024-09-19T05:15:27.016 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "database", 2024-09-19T05:15:27.016 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.016 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.016 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.016 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "ceph", 2024-09-19T05:15:27.016 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.016 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.017 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.017 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T05:15:27.017 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.017 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.017 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.017 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": { 2024-09-19T05:15:27.017 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "hostname", 2024-09-19T05:15:27.017 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.017 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.017 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.017 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.017 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.017 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.018 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.018 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB server hostname", 2024-09-19T05:15:27.020 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.020 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.020 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.021 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-09-19T05:15:27.021 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-09-19T05:15:27.021 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:27.021 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.021 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.021 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "30", 2024-09-19T05:15:27.021 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "5", 2024-09-19T05:15:27.021 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.021 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.021 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T05:15:27.021 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.021 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.021 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.021 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:27.022 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:27.022 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.022 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.022 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.022 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.022 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.022 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.022 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.022 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.022 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.022 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.022 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.022 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.022 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.022 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.023 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.023 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.023 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.023 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.023 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:27.023 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:27.023 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.023 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.023 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.023 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.023 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.023 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.023 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.024 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.024 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.024 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.024 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.024 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:27.024 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:27.024 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.024 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.024 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.024 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:27.024 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.024 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.024 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.025 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.025 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.025 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.025 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.025 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.025 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.025 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.025 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.025 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.025 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.025 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.025 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:27.025 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:27.026 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.026 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.026 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.026 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.026 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.026 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.026 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.026 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.026 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.026 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.026 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.026 INFO:teuthology.orchestra.run.smithi043.stdout: "password": { 2024-09-19T05:15:27.027 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "password", 2024-09-19T05:15:27.027 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.027 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.027 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.027 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.027 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.027 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.027 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.027 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "password of InfluxDB server user", 2024-09-19T05:15:27.027 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.027 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.028 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.028 INFO:teuthology.orchestra.run.smithi043.stdout: "port": { 2024-09-19T05:15:27.028 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "port", 2024-09-19T05:15:27.028 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:27.028 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.028 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.028 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8086", 2024-09-19T05:15:27.028 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.028 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.028 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.029 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB server port", 2024-09-19T05:15:27.029 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.029 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.029 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.029 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl": { 2024-09-19T05:15:27.029 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl", 2024-09-19T05:15:27.029 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.029 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.029 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.029 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "false", 2024-09-19T05:15:27.029 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.029 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.030 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.030 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T05:15:27.030 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.030 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.030 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.030 INFO:teuthology.orchestra.run.smithi043.stdout: "threads": { 2024-09-19T05:15:27.030 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "threads", 2024-09-19T05:15:27.030 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:27.030 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.030 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.031 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-09-19T05:15:27.031 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-09-19T05:15:27.031 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "32", 2024-09-19T05:15:27.031 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.031 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T05:15:27.031 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.031 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.031 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.031 INFO:teuthology.orchestra.run.smithi043.stdout: "username": { 2024-09-19T05:15:27.031 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "username", 2024-09-19T05:15:27.031 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.031 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.031 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.032 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.032 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.032 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.032 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.032 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "username of InfluxDB server user", 2024-09-19T05:15:27.032 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.032 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.032 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.032 INFO:teuthology.orchestra.run.smithi043.stdout: "verify_ssl": { 2024-09-19T05:15:27.033 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "verify_ssl", 2024-09-19T05:15:27.033 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.033 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.033 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.033 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "true", 2024-09-19T05:15:27.033 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.033 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.033 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.033 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T05:15:27.033 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.033 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.033 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.033 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.034 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:27.034 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "insights", 2024-09-19T05:15:27.034 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:27.034 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:27.034 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:27.034 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:27.034 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:27.034 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.035 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.035 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.035 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.035 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.035 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.035 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.035 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.035 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.035 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.035 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.036 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.036 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.036 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.036 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.036 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.036 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.036 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.036 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:27.036 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:27.036 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.036 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.036 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.037 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.037 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.037 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.037 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.037 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.037 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.037 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.037 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.038 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:27.038 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:27.038 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.038 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.038 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.038 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:27.038 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.038 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.038 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.038 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.038 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.038 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.038 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.038 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.038 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.039 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.039 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.039 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.039 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.039 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.039 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:27.039 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:27.039 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.039 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.039 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.039 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.039 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.039 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.039 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.040 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.040 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.040 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.040 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.040 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.040 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:27.040 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "iostat", 2024-09-19T05:15:27.040 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:27.040 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:27.040 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:27.040 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:27.040 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:27.041 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.041 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.041 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.041 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.041 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.041 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.041 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.041 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.041 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.041 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.041 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.041 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.041 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.041 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.041 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.042 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.042 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.042 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.042 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:27.042 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:27.042 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.042 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.042 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.042 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.042 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.042 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.042 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.042 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.042 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.043 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.043 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.043 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:27.043 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:27.043 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.043 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.043 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.043 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:27.043 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.043 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.043 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.043 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.043 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.044 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.044 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.044 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.044 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.044 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.044 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.044 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.044 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.044 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.044 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:27.044 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:27.044 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.044 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.044 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.045 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.045 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.045 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.045 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.045 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.045 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.045 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.045 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.045 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.045 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:27.045 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "k8sevents", 2024-09-19T05:15:27.046 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:27.046 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:27.046 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:27.046 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_event_retention_days": { 2024-09-19T05:15:27.046 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ceph_event_retention_days", 2024-09-19T05:15:27.046 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:27.046 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.046 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.046 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "7", 2024-09-19T05:15:27.046 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.046 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.046 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.046 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T05:15:27.047 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.047 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.047 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.047 INFO:teuthology.orchestra.run.smithi043.stdout: "config_check_secs": { 2024-09-19T05:15:27.047 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_check_secs", 2024-09-19T05:15:27.047 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:27.047 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.047 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.047 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-19T05:15:27.047 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "10", 2024-09-19T05:15:27.047 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.047 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.047 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T05:15:27.048 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.048 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.048 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.048 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:27.048 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:27.048 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.048 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.048 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.048 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.048 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.048 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.049 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.049 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.049 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.049 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.049 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.049 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.049 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.049 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.049 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.049 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.049 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.049 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.049 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:27.050 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:27.050 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.050 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.050 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.050 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.050 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.050 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.050 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.050 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.050 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.050 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.050 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.050 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:27.051 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:27.051 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.051 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.051 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.051 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:27.051 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.051 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.051 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.051 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.051 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.051 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.051 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.051 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.051 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.051 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.052 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.052 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.052 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.052 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.052 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:27.052 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:27.052 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.052 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.052 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.052 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.052 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.052 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.052 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.053 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.053 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.053 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.053 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.053 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.053 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:27.053 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "localpool", 2024-09-19T05:15:27.053 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:27.053 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:27.053 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:27.053 INFO:teuthology.orchestra.run.smithi043.stdout: "failure_domain": { 2024-09-19T05:15:27.053 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "failure_domain", 2024-09-19T05:15:27.054 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.054 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.054 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.054 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "host", 2024-09-19T05:15:27.054 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.054 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.054 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.054 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "failure domain for any created local pool", 2024-09-19T05:15:27.054 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T05:15:27.054 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.054 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.054 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:27.054 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:27.054 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.055 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.055 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.055 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.055 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.055 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.055 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.055 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.055 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.055 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.055 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.055 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.055 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.055 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.055 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.056 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.056 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.056 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.056 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:27.056 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:27.056 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.056 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.056 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.056 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.056 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.056 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.057 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.057 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.057 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.057 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.057 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.057 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:27.057 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:27.057 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.057 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.057 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.057 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:27.057 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.057 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.058 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.058 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.058 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.058 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.058 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.058 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.058 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.058 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.058 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.058 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.058 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.058 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.058 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:27.058 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:27.059 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.059 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.059 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.059 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.059 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.059 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.059 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.059 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.059 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.059 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.059 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.059 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": { 2024-09-19T05:15:27.059 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "min_size", 2024-09-19T05:15:27.059 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:27.060 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.060 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.060 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.060 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.060 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.060 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.060 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default min_size for any created local pool", 2024-09-19T05:15:27.060 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T05:15:27.060 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.060 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.060 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rep": { 2024-09-19T05:15:27.060 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "num_rep", 2024-09-19T05:15:27.060 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:27.060 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.060 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.061 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3", 2024-09-19T05:15:27.061 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.061 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.061 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.061 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default replica count for any created local pool", 2024-09-19T05:15:27.061 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.061 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.061 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.061 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": { 2024-09-19T05:15:27.061 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pg_num", 2024-09-19T05:15:27.061 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:27.061 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.061 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.061 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "128", 2024-09-19T05:15:27.061 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.062 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.062 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.062 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T05:15:27.062 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.062 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.062 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.062 INFO:teuthology.orchestra.run.smithi043.stdout: "prefix": { 2024-09-19T05:15:27.062 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prefix", 2024-09-19T05:15:27.062 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.062 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.062 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.062 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.062 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.063 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.063 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.063 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "name prefix for any created local pool", 2024-09-19T05:15:27.063 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.063 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.063 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.063 INFO:teuthology.orchestra.run.smithi043.stdout: "subtree": { 2024-09-19T05:15:27.063 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "subtree", 2024-09-19T05:15:27.063 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.063 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.063 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.063 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "rack", 2024-09-19T05:15:27.064 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.064 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.064 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.064 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T05:15:27.064 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T05:15:27.064 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.064 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.064 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.064 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:27.064 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mds_autoscaler", 2024-09-19T05:15:27.064 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:27.064 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:27.065 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:27.065 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:27.065 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:27.065 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.065 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.065 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.065 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.065 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.065 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.065 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.065 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.065 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.065 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.066 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.066 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.066 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.066 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.066 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.066 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.066 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.066 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.066 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:27.066 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:27.066 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.066 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.066 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.067 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.067 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.067 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.067 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.067 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.067 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.067 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.067 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.067 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:27.067 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:27.067 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.067 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.067 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.068 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:27.068 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.068 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.068 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.068 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.068 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.068 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.068 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.068 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.068 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.068 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.068 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.068 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.069 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.069 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.069 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:27.069 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:27.069 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.069 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.069 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.069 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.069 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.069 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.069 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.069 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.069 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.070 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.070 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.070 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.070 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:27.070 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mirroring", 2024-09-19T05:15:27.070 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:27.070 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:27.070 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:27.070 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:27.070 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:27.070 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.070 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.071 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.071 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.071 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.071 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.071 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.071 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.071 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.071 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.071 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.071 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.071 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.071 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.071 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.071 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.071 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.072 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.072 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:27.072 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:27.072 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.072 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.072 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.072 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.072 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.072 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.072 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.072 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.072 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.072 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.073 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.073 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:27.073 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:27.073 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.073 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.073 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.073 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:27.073 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.073 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.073 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.073 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.073 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.074 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.074 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.074 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.074 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.074 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.074 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.074 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.074 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.074 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.074 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:27.074 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:27.074 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.074 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.075 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.075 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.075 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.075 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.075 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.075 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.075 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.075 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.075 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.075 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.075 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:27.075 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "nfs", 2024-09-19T05:15:27.076 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:27.076 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:27.076 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:27.076 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:27.076 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:27.076 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.076 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.076 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.076 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.076 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.076 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.076 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.076 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.076 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.077 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.077 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.077 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.077 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.077 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.077 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.077 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.077 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.077 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.077 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:27.077 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:27.077 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.077 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.078 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.078 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.078 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.078 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.078 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.078 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.078 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.078 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.078 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.078 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:27.078 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:27.078 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.079 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.079 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.079 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:27.079 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.079 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.079 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.079 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.079 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.079 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.079 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.079 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.079 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.079 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.079 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.080 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.080 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.080 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.080 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:27.080 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:27.080 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.080 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.080 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.080 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.080 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.080 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.080 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.080 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.081 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.081 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.081 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.081 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.081 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:27.081 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "orchestrator", 2024-09-19T05:15:27.081 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:27.081 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:27.081 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:27.081 INFO:teuthology.orchestra.run.smithi043.stdout: "fail_fs": { 2024-09-19T05:15:27.081 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "fail_fs", 2024-09-19T05:15:27.081 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.082 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.082 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.082 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.082 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.082 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.082 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.082 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T05:15:27.082 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.082 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.082 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.082 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:27.082 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:27.082 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.083 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.083 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.083 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.083 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.083 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.083 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.083 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.083 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.083 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.083 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.083 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.083 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.083 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.084 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.084 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.084 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.084 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.084 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:27.084 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:27.084 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.084 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.084 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.084 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.084 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.084 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.084 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.085 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.085 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.085 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.085 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.085 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:27.085 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:27.085 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.085 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.085 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.085 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:27.085 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.085 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.085 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.085 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.086 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.086 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.086 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.086 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.086 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.086 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.086 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.086 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.086 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.086 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.086 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:27.086 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:27.086 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.087 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.087 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.087 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.087 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.087 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.087 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.087 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.087 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.087 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.087 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.087 INFO:teuthology.orchestra.run.smithi043.stdout: "orchestrator": { 2024-09-19T05:15:27.087 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "orchestrator", 2024-09-19T05:15:27.088 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.088 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.088 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.088 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.088 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.088 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.088 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.088 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm", 2024-09-19T05:15:27.088 INFO:teuthology.orchestra.run.smithi043.stdout: "rook", 2024-09-19T05:15:27.088 INFO:teuthology.orchestra.run.smithi043.stdout: "test_orchestrator" 2024-09-19T05:15:27.088 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.088 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Orchestrator backend", 2024-09-19T05:15:27.088 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.088 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.089 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.089 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.089 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:27.089 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "osd_perf_query", 2024-09-19T05:15:27.089 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:27.089 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:27.089 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:27.089 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:27.089 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:27.089 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.089 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.089 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.089 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.090 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.090 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.090 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.090 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.090 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.090 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.090 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.090 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.090 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.090 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.090 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.090 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.090 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.090 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.091 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:27.091 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:27.091 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.091 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.091 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.091 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.091 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.091 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.091 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.091 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.091 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.091 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.091 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.091 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:27.092 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:27.092 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.092 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.092 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.092 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:27.092 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.092 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.092 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.092 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.092 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.092 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.092 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.092 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.092 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.093 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.093 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.093 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.093 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.093 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.093 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:27.093 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:27.093 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.093 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.093 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.093 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.093 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.093 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.093 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.094 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.094 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.094 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.094 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.094 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.094 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:27.094 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "osd_support", 2024-09-19T05:15:27.094 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:27.094 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:27.094 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:27.094 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:27.094 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:27.094 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.095 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.095 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.095 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.095 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.095 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.095 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.095 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.095 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.095 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.095 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.095 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.095 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.095 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.095 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.095 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.096 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.096 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.096 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:27.096 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:27.096 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.096 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.096 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.096 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.096 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.096 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.096 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.096 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.096 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.096 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.096 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.097 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:27.097 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:27.097 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.097 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.097 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.097 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:27.097 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.097 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.097 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.097 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.097 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.097 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.097 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.098 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.098 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.098 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.098 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.098 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.098 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.098 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.098 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:27.098 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:27.098 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.098 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.098 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.098 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.098 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.099 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.099 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.099 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.099 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.099 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.099 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.099 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.099 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:27.099 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pg_autoscaler", 2024-09-19T05:15:27.099 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:27.099 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:27.099 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:27.100 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:27.100 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:27.100 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.100 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.100 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.100 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.100 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.100 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.100 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.100 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.100 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.100 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.100 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.100 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.100 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.101 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.101 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.101 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.101 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.101 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.101 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:27.101 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:27.101 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.101 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.101 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.101 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.101 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.101 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.101 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.102 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.102 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.102 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.102 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.102 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:27.102 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:27.102 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.102 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.102 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.102 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:27.102 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.102 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.102 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.103 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.103 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.103 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.103 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.103 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.103 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.103 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.103 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.103 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.103 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.103 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.103 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:27.103 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:27.104 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.104 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.104 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.104 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.104 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.104 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.104 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.104 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.104 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.104 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.105 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.105 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-09-19T05:15:27.105 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-09-19T05:15:27.105 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:27.105 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.105 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.105 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-09-19T05:15:27.105 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.105 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.105 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.105 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.105 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.105 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.106 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.106 INFO:teuthology.orchestra.run.smithi043.stdout: "threshold": { 2024-09-19T05:15:27.106 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "threshold", 2024-09-19T05:15:27.106 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-09-19T05:15:27.106 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.106 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.106 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3.0", 2024-09-19T05:15:27.106 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1.0", 2024-09-19T05:15:27.106 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.106 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.106 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "scaling threshold", 2024-09-19T05:15:27.106 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T05:15:27.106 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.106 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.107 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.107 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:27.107 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "progress", 2024-09-19T05:15:27.107 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:27.107 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:27.107 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:27.107 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_pg_recovery_event": { 2024-09-19T05:15:27.107 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_pg_recovery_event", 2024-09-19T05:15:27.107 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.107 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.107 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.107 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.107 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.108 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.108 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.108 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T05:15:27.108 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.108 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.108 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.108 INFO:teuthology.orchestra.run.smithi043.stdout: "enabled": { 2024-09-19T05:15:27.108 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enabled", 2024-09-19T05:15:27.108 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.108 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.108 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.108 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:27.108 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.108 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.109 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.109 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.109 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.109 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.109 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.109 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:27.109 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:27.109 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.109 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.109 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.109 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.109 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.109 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.109 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.110 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.110 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.110 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.110 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.110 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.110 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.110 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.110 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.110 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.110 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.110 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.110 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:27.110 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:27.110 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.111 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.111 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.111 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.111 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.111 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.111 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.111 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.111 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.111 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.111 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.111 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:27.111 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:27.111 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.111 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.112 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.112 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:27.112 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.112 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.112 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.112 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.112 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.112 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.112 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.112 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.112 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.112 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.112 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.112 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.112 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.113 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.113 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:27.113 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:27.113 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.113 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.113 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.113 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.113 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.113 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.113 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.113 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.113 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.113 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.114 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.114 INFO:teuthology.orchestra.run.smithi043.stdout: "max_completed_events": { 2024-09-19T05:15:27.114 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_completed_events", 2024-09-19T05:15:27.114 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:27.114 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.114 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.114 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "50", 2024-09-19T05:15:27.114 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.114 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.114 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.114 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "number of past completed events to remember", 2024-09-19T05:15:27.114 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.114 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.115 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.115 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-09-19T05:15:27.115 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-09-19T05:15:27.115 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:27.115 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.115 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.115 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-09-19T05:15:27.115 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.115 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.115 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.115 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long the module is going to sleep", 2024-09-19T05:15:27.115 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.115 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.115 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.116 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.116 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:27.116 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prometheus", 2024-09-19T05:15:27.116 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:27.116 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:27.116 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:27.116 INFO:teuthology.orchestra.run.smithi043.stdout: "cache": { 2024-09-19T05:15:27.116 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cache", 2024-09-19T05:15:27.116 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.116 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.116 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.116 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:27.117 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.117 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.117 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.117 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.117 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.117 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.117 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.117 INFO:teuthology.orchestra.run.smithi043.stdout: "exclude_perf_counters": { 2024-09-19T05:15:27.117 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "exclude_perf_counters", 2024-09-19T05:15:27.117 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.117 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.117 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.117 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:27.117 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.118 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.118 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.118 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T05:15:27.118 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T05:15:27.118 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.118 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.118 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:27.118 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:27.118 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.118 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.118 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.118 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.118 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.119 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.119 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.119 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.119 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.119 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.119 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.119 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.119 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.119 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.119 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.119 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.119 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.119 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.120 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:27.120 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:27.120 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.120 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.120 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.120 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.120 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.120 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.120 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.120 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.120 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.120 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.120 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.121 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:27.121 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:27.121 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.121 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.121 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.121 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:27.121 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.121 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.121 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.121 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.121 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.121 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.121 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.121 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.122 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.122 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.122 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.122 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.122 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.122 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.122 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:27.122 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:27.122 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.122 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.122 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.122 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.122 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.122 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.123 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.123 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.123 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.123 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.123 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.123 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_stats_pools": { 2024-09-19T05:15:27.123 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_stats_pools", 2024-09-19T05:15:27.123 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.123 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.123 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.123 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.123 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.123 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.124 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.124 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.124 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.124 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.124 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.124 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T05:15:27.124 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T05:15:27.124 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:27.124 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.124 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.124 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "300", 2024-09-19T05:15:27.124 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.124 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.125 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.125 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.125 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.125 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.125 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.125 INFO:teuthology.orchestra.run.smithi043.stdout: "scrape_interval": { 2024-09-19T05:15:27.125 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "scrape_interval", 2024-09-19T05:15:27.125 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-09-19T05:15:27.125 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.125 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.125 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15.0", 2024-09-19T05:15:27.125 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.125 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.126 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.126 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.126 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.126 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.126 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.126 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-09-19T05:15:27.126 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-09-19T05:15:27.126 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.126 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.126 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.126 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "::", 2024-09-19T05:15:27.126 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.126 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.126 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.126 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T05:15:27.127 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.127 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.127 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.127 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-09-19T05:15:27.127 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-09-19T05:15:27.127 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:27.127 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.127 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.127 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "9283", 2024-09-19T05:15:27.127 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.127 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.127 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.127 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T05:15:27.127 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.128 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.128 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.128 INFO:teuthology.orchestra.run.smithi043.stdout: "stale_cache_strategy": { 2024-09-19T05:15:27.128 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "stale_cache_strategy", 2024-09-19T05:15:27.128 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.128 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.128 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.128 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "log", 2024-09-19T05:15:27.128 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.128 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.128 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.128 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.129 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.129 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.129 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.129 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_behaviour": { 2024-09-19T05:15:27.129 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_behaviour", 2024-09-19T05:15:27.129 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.129 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.129 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.129 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "default", 2024-09-19T05:15:27.129 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.129 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.130 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.130 INFO:teuthology.orchestra.run.smithi043.stdout: "default", 2024-09-19T05:15:27.130 INFO:teuthology.orchestra.run.smithi043.stdout: "error" 2024-09-19T05:15:27.130 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.130 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.130 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.130 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.130 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.130 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_error_status_code": { 2024-09-19T05:15:27.130 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_error_status_code", 2024-09-19T05:15:27.130 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:27.130 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.130 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.130 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "500", 2024-09-19T05:15:27.131 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "400", 2024-09-19T05:15:27.131 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "599", 2024-09-19T05:15:27.131 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.131 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.131 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.131 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.131 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.131 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.131 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:27.131 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_support", 2024-09-19T05:15:27.131 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:27.131 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:27.131 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:27.131 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:27.132 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:27.132 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.132 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.132 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.132 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.132 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.132 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.132 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.132 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.132 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.132 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.132 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.132 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.132 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.132 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.133 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.133 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.133 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.133 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.133 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:27.133 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:27.133 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.133 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.133 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.133 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.133 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.133 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.133 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.133 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.134 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.134 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.134 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.134 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:27.134 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:27.134 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.134 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.134 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.134 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:27.134 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.134 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.134 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.134 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.134 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.135 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.135 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.135 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.135 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.135 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.135 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.135 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.135 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.135 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.135 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:27.135 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:27.135 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.135 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.135 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.136 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.136 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.136 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.136 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.136 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.136 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.136 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.136 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.136 INFO:teuthology.orchestra.run.smithi043.stdout: "max_concurrent_snap_create": { 2024-09-19T05:15:27.136 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_concurrent_snap_create", 2024-09-19T05:15:27.136 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:27.136 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.136 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.137 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-19T05:15:27.137 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.137 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.137 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.137 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.137 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.137 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.137 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.137 INFO:teuthology.orchestra.run.smithi043.stdout: "mirror_snapshot_schedule": { 2024-09-19T05:15:27.137 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T05:15:27.137 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.137 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.138 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.138 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.138 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.138 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.138 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.138 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.138 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.138 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.138 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.138 INFO:teuthology.orchestra.run.smithi043.stdout: "trash_purge_schedule": { 2024-09-19T05:15:27.138 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "trash_purge_schedule", 2024-09-19T05:15:27.138 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.138 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.138 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.138 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.139 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.139 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.139 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.139 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.139 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.139 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.139 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.139 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.139 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:27.139 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "restful", 2024-09-19T05:15:27.139 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:27.139 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:27.139 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:27.140 INFO:teuthology.orchestra.run.smithi043.stdout: "enable_auth": { 2024-09-19T05:15:27.140 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enable_auth", 2024-09-19T05:15:27.140 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.140 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.140 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.140 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:27.140 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.140 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.140 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.140 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.140 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.140 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.140 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.141 INFO:teuthology.orchestra.run.smithi043.stdout: "key_file": { 2024-09-19T05:15:27.141 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "key_file", 2024-09-19T05:15:27.141 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.141 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.141 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.141 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.141 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.141 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.141 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.141 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.141 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.141 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.141 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.142 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:27.142 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:27.142 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.142 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.142 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.142 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.142 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.142 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.142 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.142 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.142 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.142 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.142 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.142 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.143 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.143 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.143 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.143 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.143 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.143 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.143 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:27.143 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:27.143 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.143 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.143 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.143 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.143 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.143 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.144 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.144 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.144 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.144 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.144 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.144 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:27.144 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:27.144 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.144 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.144 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.144 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:27.144 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.144 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.144 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.145 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.145 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.145 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.145 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.145 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.145 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.145 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.145 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.145 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.145 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.145 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.145 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:27.145 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:27.146 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.146 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.146 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.146 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.146 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.146 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.146 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.146 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.146 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.146 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.146 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.146 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-09-19T05:15:27.146 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-09-19T05:15:27.147 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.147 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.147 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.147 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.147 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.147 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.147 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.147 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.147 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.147 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.147 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.147 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-09-19T05:15:27.147 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-09-19T05:15:27.148 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.148 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.148 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.148 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.148 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.148 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.148 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.148 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.148 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.148 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.148 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.148 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.148 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:27.149 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rgw", 2024-09-19T05:15:27.149 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:27.149 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:27.149 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:27.149 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:27.149 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:27.149 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.149 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.149 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.149 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.149 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.149 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.149 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.150 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.150 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.150 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.150 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.150 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.150 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.150 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.150 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.150 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.150 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.150 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.150 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:27.151 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:27.151 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.151 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.151 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.151 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.151 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.151 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.151 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.151 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.151 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.151 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.151 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.151 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:27.152 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:27.152 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.152 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.152 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.152 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:27.152 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.152 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.152 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.152 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.152 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.152 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.152 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.152 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.152 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.153 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.153 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.153 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.153 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.153 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.153 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:27.153 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:27.153 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.153 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.153 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.153 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.153 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.153 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.153 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.154 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.154 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.154 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.154 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.154 INFO:teuthology.orchestra.run.smithi043.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T05:15:27.154 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T05:15:27.154 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:27.154 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.154 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.154 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-09-19T05:15:27.154 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.154 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.154 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.155 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T05:15:27.155 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.155 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.155 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.155 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.155 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:27.155 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rook", 2024-09-19T05:15:27.155 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:27.155 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:27.155 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:27.155 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:27.155 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:27.155 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.156 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.156 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.156 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.156 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.156 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.156 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.156 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.156 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.156 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.156 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.156 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.156 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.156 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.156 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.156 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.157 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.157 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.157 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:27.157 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:27.157 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.157 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.157 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.157 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.157 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.157 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.157 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.157 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.157 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.157 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.158 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.158 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:27.158 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:27.158 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.158 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.158 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.158 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:27.158 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.158 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.158 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.158 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.158 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.158 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.159 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.159 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.159 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.159 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.159 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.159 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.159 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.159 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.159 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:27.159 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:27.159 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.159 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.159 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.160 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.160 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.160 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.160 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.160 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.160 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.160 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.160 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.160 INFO:teuthology.orchestra.run.smithi043.stdout: "storage_class": { 2024-09-19T05:15:27.160 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "storage_class", 2024-09-19T05:15:27.160 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.160 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.160 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.161 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "local", 2024-09-19T05:15:27.161 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.161 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.161 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.161 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T05:15:27.161 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.161 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.161 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.161 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.161 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:27.161 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "selftest", 2024-09-19T05:15:27.162 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:27.162 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:27.162 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:27.162 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:27.162 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:27.162 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.162 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.162 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.162 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.162 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.162 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.162 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.162 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.162 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.163 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.163 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.163 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.163 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.163 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.163 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.163 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.163 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.163 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.163 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:27.163 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:27.163 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.164 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.164 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.164 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.164 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.164 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.164 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.164 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.164 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.164 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.164 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.164 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:27.164 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:27.164 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.165 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.165 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.165 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:27.165 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.165 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.165 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.165 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.165 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.165 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.165 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.165 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.165 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.165 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.165 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.166 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.166 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.166 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.166 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:27.166 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:27.166 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.166 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.166 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.166 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.166 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.166 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.166 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.166 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.166 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.167 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.167 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.167 INFO:teuthology.orchestra.run.smithi043.stdout: "roption1": { 2024-09-19T05:15:27.167 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "roption1", 2024-09-19T05:15:27.167 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.167 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.167 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.167 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.167 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.167 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.167 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.167 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.167 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.167 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.168 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.168 INFO:teuthology.orchestra.run.smithi043.stdout: "roption2": { 2024-09-19T05:15:27.168 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "roption2", 2024-09-19T05:15:27.168 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.168 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.168 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.168 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "xyz", 2024-09-19T05:15:27.168 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.168 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.168 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.168 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.168 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.168 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.169 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.169 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption1": { 2024-09-19T05:15:27.169 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption1", 2024-09-19T05:15:27.169 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.169 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.169 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.169 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.169 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.169 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.169 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.169 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.169 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.169 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.170 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.170 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption2": { 2024-09-19T05:15:27.170 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption2", 2024-09-19T05:15:27.170 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:27.170 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.170 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.170 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.170 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.170 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.170 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.170 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.170 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.170 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.170 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.171 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption3": { 2024-09-19T05:15:27.171 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption3", 2024-09-19T05:15:27.171 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-09-19T05:15:27.171 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.171 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.171 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.171 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.171 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.171 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.171 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.171 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.171 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.171 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.171 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption4": { 2024-09-19T05:15:27.172 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption4", 2024-09-19T05:15:27.172 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.172 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.172 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.172 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.172 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.172 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.172 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.172 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.172 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.172 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.172 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.173 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption5": { 2024-09-19T05:15:27.173 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption5", 2024-09-19T05:15:27.173 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.173 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.173 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.173 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.173 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.173 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.173 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.173 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.173 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.173 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.174 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.174 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption6": { 2024-09-19T05:15:27.174 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption6", 2024-09-19T05:15:27.174 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.174 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.174 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.174 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:27.174 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.174 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.174 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.174 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.174 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.174 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.174 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.175 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption7": { 2024-09-19T05:15:27.175 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption7", 2024-09-19T05:15:27.175 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:27.175 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.175 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.175 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.175 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-09-19T05:15:27.175 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "42", 2024-09-19T05:15:27.175 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.175 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.175 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.175 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.176 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.176 INFO:teuthology.orchestra.run.smithi043.stdout: "testkey": { 2024-09-19T05:15:27.176 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testkey", 2024-09-19T05:15:27.176 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.176 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.176 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.176 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.176 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.176 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.176 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.176 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.176 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.176 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.177 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.177 INFO:teuthology.orchestra.run.smithi043.stdout: "testlkey": { 2024-09-19T05:15:27.177 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testlkey", 2024-09-19T05:15:27.177 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.177 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.177 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.177 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.177 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.177 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.177 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.177 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.177 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.177 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.177 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.178 INFO:teuthology.orchestra.run.smithi043.stdout: "testnewline": { 2024-09-19T05:15:27.178 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testnewline", 2024-09-19T05:15:27.178 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.178 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.178 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.178 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.178 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.178 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.178 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.178 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.178 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.178 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.178 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.178 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.179 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:27.179 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snap_schedule", 2024-09-19T05:15:27.179 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:27.179 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:27.179 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:27.179 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_m_granularity": { 2024-09-19T05:15:27.179 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_m_granularity", 2024-09-19T05:15:27.179 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.179 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.179 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.179 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.179 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.180 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.180 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.180 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T05:15:27.180 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.180 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.180 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.180 INFO:teuthology.orchestra.run.smithi043.stdout: "dump_on_update": { 2024-09-19T05:15:27.180 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "dump_on_update", 2024-09-19T05:15:27.180 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.180 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.180 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.180 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.181 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.181 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.181 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.181 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "dump database to debug log on update", 2024-09-19T05:15:27.181 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.181 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.181 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.181 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:27.181 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:27.181 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.181 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.181 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.182 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.182 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.182 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.182 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.182 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.182 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.182 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.182 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.182 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.182 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.182 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.182 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.182 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.183 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.183 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.183 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:27.183 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:27.183 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.183 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.183 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.183 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.183 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.183 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.183 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.183 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.184 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.184 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.184 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.184 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:27.184 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:27.184 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.184 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.184 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.184 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:27.184 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.184 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.185 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.185 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.185 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.185 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.185 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.185 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.185 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.185 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.185 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.185 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.186 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.186 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.186 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:27.186 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:27.186 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.186 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.186 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.186 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.186 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.186 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.186 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.186 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.186 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.187 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.187 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.187 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.187 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:27.187 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "stats", 2024-09-19T05:15:27.187 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:27.187 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:27.187 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:27.187 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:27.187 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:27.187 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.187 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.188 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.188 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.188 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.188 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.188 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.188 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.188 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.188 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.188 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.188 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.188 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.188 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.188 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.188 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.189 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.189 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.189 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:27.189 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:27.189 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.189 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.189 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.189 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.189 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.189 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.189 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.189 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.189 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.190 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.190 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.190 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:27.190 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:27.190 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.190 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.190 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.190 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:27.190 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.190 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.190 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.190 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.190 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.191 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.191 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.191 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.191 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.191 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.191 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.191 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.191 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.191 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.191 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:27.191 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:27.191 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.192 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.192 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.192 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.192 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.192 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.192 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.192 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.192 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.192 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.192 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.192 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.192 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:27.193 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "status", 2024-09-19T05:15:27.193 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:27.193 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:27.193 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:27.193 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:27.193 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:27.193 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.193 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.193 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.193 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.193 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.193 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.193 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.193 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.194 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.194 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.194 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.194 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.194 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.194 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.194 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.194 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.194 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.194 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.194 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:27.194 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:27.194 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.194 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.195 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.195 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.195 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.195 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.195 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.195 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.195 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.195 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.195 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.195 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:27.195 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:27.195 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.195 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.196 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.196 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:27.196 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.196 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.196 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.196 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.196 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.196 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.196 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.196 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.196 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.196 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.196 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.196 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.197 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.197 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.197 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:27.197 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:27.197 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.197 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.197 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.197 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.197 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.197 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.197 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.197 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.197 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.197 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.198 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.198 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.198 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:27.198 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "telegraf", 2024-09-19T05:15:27.198 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:27.198 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:27.198 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:27.198 INFO:teuthology.orchestra.run.smithi043.stdout: "address": { 2024-09-19T05:15:27.198 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "address", 2024-09-19T05:15:27.198 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.198 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.199 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.199 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T05:15:27.199 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.199 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.199 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.199 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.199 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.199 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.199 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.199 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-09-19T05:15:27.199 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-09-19T05:15:27.199 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:27.199 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.200 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.200 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15", 2024-09-19T05:15:27.200 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.200 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.200 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.200 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.200 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.200 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.200 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.200 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:27.200 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:27.200 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.200 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.201 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.201 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.201 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.201 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.201 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.201 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.201 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.201 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.201 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.201 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.201 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.201 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.201 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.201 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.202 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.202 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.202 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:27.202 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:27.202 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.202 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.202 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.202 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.202 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.202 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.202 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.202 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.202 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.203 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.203 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.203 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:27.203 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:27.203 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.203 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.203 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.203 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:27.203 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.203 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.203 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.203 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.203 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.203 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.204 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.204 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.204 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.204 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.204 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.204 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.204 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.204 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.204 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:27.204 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:27.204 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.204 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.204 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.205 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.205 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.205 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.205 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.205 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.205 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.205 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.205 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.205 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.205 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:27.205 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "telemetry", 2024-09-19T05:15:27.205 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:27.206 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:27.206 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:27.206 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_basic": { 2024-09-19T05:15:27.206 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_basic", 2024-09-19T05:15:27.206 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.206 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.206 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.206 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:27.206 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.206 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.206 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.206 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T05:15:27.206 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.206 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.206 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.207 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_crash": { 2024-09-19T05:15:27.207 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_crash", 2024-09-19T05:15:27.207 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.207 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.207 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.207 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:27.207 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.207 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.207 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.207 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T05:15:27.207 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.207 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.207 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.208 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_device": { 2024-09-19T05:15:27.208 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_device", 2024-09-19T05:15:27.208 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.208 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.208 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.208 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:27.208 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.208 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.208 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.208 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T05:15:27.208 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.208 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.208 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.209 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_ident": { 2024-09-19T05:15:27.209 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_ident", 2024-09-19T05:15:27.209 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.209 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.209 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.209 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.209 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.209 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.209 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.209 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T05:15:27.209 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.209 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.209 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.210 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_perf": { 2024-09-19T05:15:27.210 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_perf", 2024-09-19T05:15:27.210 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.210 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.210 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.210 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.210 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.210 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.210 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.210 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T05:15:27.210 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.210 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.210 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.211 INFO:teuthology.orchestra.run.smithi043.stdout: "contact": { 2024-09-19T05:15:27.211 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "contact", 2024-09-19T05:15:27.211 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.211 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.211 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.211 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.211 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.211 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.211 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.211 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.211 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.211 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.211 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.212 INFO:teuthology.orchestra.run.smithi043.stdout: "description": { 2024-09-19T05:15:27.212 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "description", 2024-09-19T05:15:27.212 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.212 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.212 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.212 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.212 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.212 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.212 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.212 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.212 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.212 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.212 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.213 INFO:teuthology.orchestra.run.smithi043.stdout: "device_url": { 2024-09-19T05:15:27.213 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_url", 2024-09-19T05:15:27.213 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.213 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.213 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.213 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T05:15:27.213 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.213 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.213 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.213 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.213 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.213 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.213 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.213 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.214 INFO:teuthology.orchestra.run.smithi043.stdout: "enabled": { 2024-09-19T05:15:27.214 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enabled", 2024-09-19T05:15:27.214 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.214 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.214 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.214 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.214 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.214 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.214 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.214 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.214 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.214 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.214 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.215 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-09-19T05:15:27.215 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-09-19T05:15:27.215 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:27.215 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.215 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.215 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "24", 2024-09-19T05:15:27.215 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "8", 2024-09-19T05:15:27.215 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.215 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.215 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.215 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.215 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.215 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.216 INFO:teuthology.orchestra.run.smithi043.stdout: "last_opt_revision": { 2024-09-19T05:15:27.216 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "last_opt_revision", 2024-09-19T05:15:27.216 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:27.216 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.216 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.216 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1", 2024-09-19T05:15:27.216 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.216 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.216 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.216 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.216 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.216 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.216 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.217 INFO:teuthology.orchestra.run.smithi043.stdout: "leaderboard": { 2024-09-19T05:15:27.217 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "leaderboard", 2024-09-19T05:15:27.217 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.217 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.217 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.217 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.217 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.217 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.217 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.217 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.217 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.217 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.217 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.218 INFO:teuthology.orchestra.run.smithi043.stdout: "leaderboard_description": { 2024-09-19T05:15:27.218 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "leaderboard_description", 2024-09-19T05:15:27.218 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.218 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.218 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.218 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.218 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.218 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.218 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.218 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.218 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.218 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.218 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.218 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:27.219 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:27.219 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.219 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.219 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.219 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.219 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.219 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.219 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.219 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.219 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.219 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.219 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.219 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.219 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.220 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.220 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.220 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.220 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.220 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.220 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:27.220 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:27.220 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.220 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.220 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.220 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.220 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.220 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.221 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.221 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.221 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.221 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.221 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.221 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:27.221 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:27.221 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.221 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.221 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.222 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:27.222 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.222 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.222 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.222 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.222 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.222 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.222 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.222 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.222 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.222 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.222 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.222 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.223 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.223 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.223 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:27.223 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:27.223 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.223 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.223 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.223 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.223 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.223 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.223 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.223 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.223 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.223 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.224 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.224 INFO:teuthology.orchestra.run.smithi043.stdout: "organization": { 2024-09-19T05:15:27.224 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "organization", 2024-09-19T05:15:27.224 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.226 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.226 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.226 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.226 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.226 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.226 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.226 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.226 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.226 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.226 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.227 INFO:teuthology.orchestra.run.smithi043.stdout: "proxy": { 2024-09-19T05:15:27.227 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "proxy", 2024-09-19T05:15:27.227 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.227 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.227 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.227 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.227 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.227 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.227 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.227 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.227 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.227 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.227 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.228 INFO:teuthology.orchestra.run.smithi043.stdout: "url": { 2024-09-19T05:15:27.228 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "url", 2024-09-19T05:15:27.228 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.228 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.228 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.228 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T05:15:27.228 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.228 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.228 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.228 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.228 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.228 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.228 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.228 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.228 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:27.229 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "test_orchestrator", 2024-09-19T05:15:27.229 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:27.229 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:27.229 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:27.229 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:27.229 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:27.229 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.229 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.229 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.229 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.229 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.230 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.230 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.230 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.230 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.230 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.230 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.230 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.230 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.230 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.230 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.230 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.230 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.230 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.230 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:27.231 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:27.231 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.231 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.231 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.231 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.231 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.231 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.231 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.231 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.231 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.231 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.231 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.231 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:27.231 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:27.232 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.232 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.232 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.232 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:27.232 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.232 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.232 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.232 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.232 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.232 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.232 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.232 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.232 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.232 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.233 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.233 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.233 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.233 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.233 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:27.233 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:27.233 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.233 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.233 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.233 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.233 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.233 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.234 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.234 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.234 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.234 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.234 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.234 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.234 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:27.234 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "volumes", 2024-09-19T05:15:27.234 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:27.234 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:27.234 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:27.234 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:27.234 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:27.235 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.235 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.235 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.235 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.235 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.235 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.235 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.235 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.235 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.235 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.235 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.235 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.235 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.235 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.236 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.236 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.236 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.236 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.236 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:27.236 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:27.236 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.236 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.236 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.236 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.236 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.236 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.236 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.237 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.237 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.237 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.237 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.237 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:27.237 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:27.237 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.237 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.237 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.237 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:27.237 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.237 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.237 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.237 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.238 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.238 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.238 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.238 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.238 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.238 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.238 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.238 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.238 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.238 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.238 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:27.238 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:27.238 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.239 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.239 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.239 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.239 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.239 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.239 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.239 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.239 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.239 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.239 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.239 INFO:teuthology.orchestra.run.smithi043.stdout: "max_concurrent_clones": { 2024-09-19T05:15:27.239 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_concurrent_clones", 2024-09-19T05:15:27.239 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:27.239 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.240 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.240 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "4", 2024-09-19T05:15:27.240 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.240 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.240 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.240 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T05:15:27.240 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.240 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.240 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.240 INFO:teuthology.orchestra.run.smithi043.stdout: "periodic_async_work": { 2024-09-19T05:15:27.240 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "periodic_async_work", 2024-09-19T05:15:27.240 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.240 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.241 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.241 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.241 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.241 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.241 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.241 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Periodically check for async work", 2024-09-19T05:15:27.241 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.241 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.241 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.241 INFO:teuthology.orchestra.run.smithi043.stdout: "snapshot_clone_delay": { 2024-09-19T05:15:27.241 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snapshot_clone_delay", 2024-09-19T05:15:27.241 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:27.241 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.242 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.242 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-09-19T05:15:27.242 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.242 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.242 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.242 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T05:15:27.242 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.242 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.242 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.242 INFO:teuthology.orchestra.run.smithi043.stdout: "snapshot_clone_no_wait": { 2024-09-19T05:15:27.242 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T05:15:27.242 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.242 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.242 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.243 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-19T05:15:27.243 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.243 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.243 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.243 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T05:15:27.243 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.243 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.243 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.243 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.243 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:27.243 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix", 2024-09-19T05:15:27.243 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-19T05:15:27.244 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-19T05:15:27.244 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-19T05:15:27.244 INFO:teuthology.orchestra.run.smithi043.stdout: "discovery_interval": { 2024-09-19T05:15:27.244 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "discovery_interval", 2024-09-19T05:15:27.244 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-09-19T05:15:27.244 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.244 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.244 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "100", 2024-09-19T05:15:27.244 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.244 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.244 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.244 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.244 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.244 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.245 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.245 INFO:teuthology.orchestra.run.smithi043.stdout: "identifier": { 2024-09-19T05:15:27.245 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "identifier", 2024-09-19T05:15:27.245 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.245 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.245 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.245 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.245 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.245 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.245 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.245 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.245 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.245 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.245 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.246 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-09-19T05:15:27.246 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-09-19T05:15:27.246 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-19T05:15:27.246 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.246 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.246 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-09-19T05:15:27.246 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.246 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.246 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.246 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.246 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.246 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.246 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.247 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-19T05:15:27.247 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-19T05:15:27.247 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.247 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.247 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.247 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.247 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.247 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.247 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.247 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.247 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.247 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.247 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.247 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.248 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.248 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.248 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.248 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.248 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.248 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.248 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-19T05:15:27.248 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-19T05:15:27.248 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.248 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.248 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.248 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.248 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.248 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.249 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.249 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.249 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.249 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.249 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.249 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.249 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-19T05:15:27.249 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-19T05:15:27.249 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.249 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.249 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.249 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-19T05:15:27.249 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.249 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.250 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-19T05:15:27.250 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-19T05:15:27.250 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-19T05:15:27.250 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-19T05:15:27.250 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-19T05:15:27.250 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-19T05:15:27.250 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-19T05:15:27.250 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.250 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.250 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.250 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.250 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.250 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-19T05:15:27.250 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-19T05:15:27.251 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-19T05:15:27.251 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.251 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-19T05:15:27.251 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-19T05:15:27.251 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.251 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.251 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.251 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.251 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.251 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.251 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.251 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_host": { 2024-09-19T05:15:27.251 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_host", 2024-09-19T05:15:27.252 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.252 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.252 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.252 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-19T05:15:27.252 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.252 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.252 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.252 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.252 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.252 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.252 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.252 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_port": { 2024-09-19T05:15:27.252 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_port", 2024-09-19T05:15:27.252 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-19T05:15:27.253 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.253 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.253 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10051", 2024-09-19T05:15:27.253 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.253 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.253 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.253 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.253 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.253 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.253 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.253 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_sender": { 2024-09-19T05:15:27.253 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_sender", 2024-09-19T05:15:27.253 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-19T05:15:27.254 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-19T05:15:27.254 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-19T05:15:27.254 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T05:15:27.254 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-19T05:15:27.254 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-19T05:15:27.254 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-19T05:15:27.254 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-19T05:15:27.254 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-19T05:15:27.254 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-19T05:15:27.254 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-19T05:15:27.254 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.254 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.254 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.254 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.254 INFO:teuthology.orchestra.run.smithi043.stdout: "services": { 2024-09-19T05:15:27.255 INFO:teuthology.orchestra.run.smithi043.stdout: "dashboard": "https://172.21.15.43:8443/" 2024-09-19T05:15:27.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.255 INFO:teuthology.orchestra.run.smithi043.stdout: "always_on_modules": { 2024-09-19T05:15:27.255 INFO:teuthology.orchestra.run.smithi043.stdout: "octopus": [ 2024-09-19T05:15:27.255 INFO:teuthology.orchestra.run.smithi043.stdout: "balancer", 2024-09-19T05:15:27.255 INFO:teuthology.orchestra.run.smithi043.stdout: "crash", 2024-09-19T05:15:27.255 INFO:teuthology.orchestra.run.smithi043.stdout: "devicehealth", 2024-09-19T05:15:27.255 INFO:teuthology.orchestra.run.smithi043.stdout: "orchestrator", 2024-09-19T05:15:27.255 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscaler", 2024-09-19T05:15:27.255 INFO:teuthology.orchestra.run.smithi043.stdout: "progress", 2024-09-19T05:15:27.255 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_support", 2024-09-19T05:15:27.255 INFO:teuthology.orchestra.run.smithi043.stdout: "status", 2024-09-19T05:15:27.255 INFO:teuthology.orchestra.run.smithi043.stdout: "telemetry", 2024-09-19T05:15:27.255 INFO:teuthology.orchestra.run.smithi043.stdout: "volumes" 2024-09-19T05:15:27.256 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.256 INFO:teuthology.orchestra.run.smithi043.stdout: "pacific": [ 2024-09-19T05:15:27.256 INFO:teuthology.orchestra.run.smithi043.stdout: "balancer", 2024-09-19T05:15:27.256 INFO:teuthology.orchestra.run.smithi043.stdout: "crash", 2024-09-19T05:15:27.256 INFO:teuthology.orchestra.run.smithi043.stdout: "devicehealth", 2024-09-19T05:15:27.256 INFO:teuthology.orchestra.run.smithi043.stdout: "orchestrator", 2024-09-19T05:15:27.256 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscaler", 2024-09-19T05:15:27.256 INFO:teuthology.orchestra.run.smithi043.stdout: "progress", 2024-09-19T05:15:27.256 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_support", 2024-09-19T05:15:27.256 INFO:teuthology.orchestra.run.smithi043.stdout: "status", 2024-09-19T05:15:27.256 INFO:teuthology.orchestra.run.smithi043.stdout: "telemetry", 2024-09-19T05:15:27.256 INFO:teuthology.orchestra.run.smithi043.stdout: "volumes" 2024-09-19T05:15:27.256 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.257 INFO:teuthology.orchestra.run.smithi043.stdout: "quincy": [ 2024-09-19T05:15:27.257 INFO:teuthology.orchestra.run.smithi043.stdout: "balancer", 2024-09-19T05:15:27.257 INFO:teuthology.orchestra.run.smithi043.stdout: "crash", 2024-09-19T05:15:27.257 INFO:teuthology.orchestra.run.smithi043.stdout: "devicehealth", 2024-09-19T05:15:27.257 INFO:teuthology.orchestra.run.smithi043.stdout: "orchestrator", 2024-09-19T05:15:27.257 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscaler", 2024-09-19T05:15:27.257 INFO:teuthology.orchestra.run.smithi043.stdout: "progress", 2024-09-19T05:15:27.257 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_support", 2024-09-19T05:15:27.257 INFO:teuthology.orchestra.run.smithi043.stdout: "status", 2024-09-19T05:15:27.257 INFO:teuthology.orchestra.run.smithi043.stdout: "telemetry", 2024-09-19T05:15:27.257 INFO:teuthology.orchestra.run.smithi043.stdout: "volumes" 2024-09-19T05:15:27.257 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.257 INFO:teuthology.orchestra.run.smithi043.stdout: "reef": [ 2024-09-19T05:15:27.257 INFO:teuthology.orchestra.run.smithi043.stdout: "balancer", 2024-09-19T05:15:27.258 INFO:teuthology.orchestra.run.smithi043.stdout: "crash", 2024-09-19T05:15:27.258 INFO:teuthology.orchestra.run.smithi043.stdout: "devicehealth", 2024-09-19T05:15:27.258 INFO:teuthology.orchestra.run.smithi043.stdout: "orchestrator", 2024-09-19T05:15:27.258 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscaler", 2024-09-19T05:15:27.258 INFO:teuthology.orchestra.run.smithi043.stdout: "progress", 2024-09-19T05:15:27.258 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_support", 2024-09-19T05:15:27.258 INFO:teuthology.orchestra.run.smithi043.stdout: "status", 2024-09-19T05:15:27.258 INFO:teuthology.orchestra.run.smithi043.stdout: "telemetry", 2024-09-19T05:15:27.258 INFO:teuthology.orchestra.run.smithi043.stdout: "volumes" 2024-09-19T05:15:27.258 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-19T05:15:27.258 INFO:teuthology.orchestra.run.smithi043.stdout: "squid": [ 2024-09-19T05:15:27.258 INFO:teuthology.orchestra.run.smithi043.stdout: "balancer", 2024-09-19T05:15:27.258 INFO:teuthology.orchestra.run.smithi043.stdout: "crash", 2024-09-19T05:15:27.258 INFO:teuthology.orchestra.run.smithi043.stdout: "devicehealth", 2024-09-19T05:15:27.259 INFO:teuthology.orchestra.run.smithi043.stdout: "orchestrator", 2024-09-19T05:15:27.259 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscaler", 2024-09-19T05:15:27.259 INFO:teuthology.orchestra.run.smithi043.stdout: "progress", 2024-09-19T05:15:27.259 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_support", 2024-09-19T05:15:27.259 INFO:teuthology.orchestra.run.smithi043.stdout: "status", 2024-09-19T05:15:27.259 INFO:teuthology.orchestra.run.smithi043.stdout: "telemetry", 2024-09-19T05:15:27.259 INFO:teuthology.orchestra.run.smithi043.stdout: "volumes" 2024-09-19T05:15:27.259 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-19T05:15:27.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.259 INFO:teuthology.orchestra.run.smithi043.stdout: "last_failure_osd_epoch": 86, 2024-09-19T05:15:27.259 INFO:teuthology.orchestra.run.smithi043.stdout: "active_clients": [ 2024-09-19T05:15:27.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:27.260 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "devicehealth", 2024-09-19T05:15:27.260 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-19T05:15:27.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:27.260 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-19T05:15:27.260 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:0", 2024-09-19T05:15:27.260 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1684838741 2024-09-19T05:15:27.260 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.260 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-19T05:15:27.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:27.260 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "libcephsqlite", 2024-09-19T05:15:27.261 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-19T05:15:27.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:27.261 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-19T05:15:27.261 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:0", 2024-09-19T05:15:27.261 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1289795822 2024-09-19T05:15:27.261 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.261 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-19T05:15:27.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:27.261 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_support", 2024-09-19T05:15:27.261 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-19T05:15:27.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:27.261 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-19T05:15:27.261 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:0", 2024-09-19T05:15:27.262 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 959457925 2024-09-19T05:15:27.262 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.262 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-19T05:15:27.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:27.262 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snap_schedule", 2024-09-19T05:15:27.262 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-19T05:15:27.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:27.262 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-19T05:15:27.262 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:0", 2024-09-19T05:15:27.262 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3239713862 2024-09-19T05:15:27.262 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.262 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-19T05:15:27.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:27.263 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snap_schedule", 2024-09-19T05:15:27.263 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-19T05:15:27.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:27.263 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-19T05:15:27.263 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:0", 2024-09-19T05:15:27.263 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 2052189613 2024-09-19T05:15:27.263 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.263 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-19T05:15:27.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:27.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:27.263 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "volumes", 2024-09-19T05:15:27.263 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-19T05:15:27.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-19T05:15:27.264 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-19T05:15:27.264 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:0", 2024-09-19T05:15:27.264 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 608851215 2024-09-19T05:15:27.264 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.264 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-19T05:15:27.264 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:27.264 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-19T05:15:27.264 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:15:27.266 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:25 smithi151 ceph-mon[35425]: pgmap v336: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T05:15:27.266 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:25 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/2573401014' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T05:15:27.266 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:25 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/650246046' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T05:15:27.266 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:25 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/997785415' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T05:15:27.266 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:25 smithi096 ceph-mon[35085]: pgmap v336: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T05:15:27.266 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:25 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/2573401014' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T05:15:27.266 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:25 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/650246046' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T05:15:27.266 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:25 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/997785415' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T05:15:27.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.874+0000 7fa0e6200640 1 -- 172.21.15.43:0/997785415 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fa0c40a3440 msgr2=0x7fa0c40a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:27.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.874+0000 7fa0e6200640 1 --2- 172.21.15.43:0/997785415 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fa0c40a3440 0x7fa0c40a5900 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0005fd0 tx=0x7fa0d0009260 comp rx=0 tx=0).stop 2024-09-19T05:15:27.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.874+0000 7fa0e6200640 1 -- 172.21.15.43:0/997785415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0e0107ab0 msgr2=0x7fa0e019ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:27.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.874+0000 7fa0e6200640 1 --2- 172.21.15.43:0/997785415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0e0107ab0 0x7fa0e019ee30 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8034040 tx=0x7fa0d8030c50 comp rx=0 tx=0).stop 2024-09-19T05:15:27.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.874+0000 7fa0e6200640 1 -- 172.21.15.43:0/997785415 shutdown_connections 2024-09-19T05:15:27.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.874+0000 7fa0e6200640 1 --2- 172.21.15.43:0/997785415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0e0107ab0 0x7fa0e019ee30 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:27.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.874+0000 7fa0e6200640 1 --2- 172.21.15.43:0/997785415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0e0102440 0x7fa0e019aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:27.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.874+0000 7fa0e6200640 1 --2- 172.21.15.43:0/997785415 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fa0c40a3440 0x7fa0c40a5900 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:27.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.874+0000 7fa0e6200640 1 --2- 172.21.15.43:0/997785415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0e0101aa0 0x7fa0e019a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:27.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.874+0000 7fa0e6200640 1 -- 172.21.15.43:0/997785415 >> 172.21.15.43:0/997785415 conn(0x7fa0e00fd7c0 msgr2=0x7fa0e010d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:27.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.874+0000 7fa0e6200640 1 -- 172.21.15.43:0/997785415 shutdown_connections 2024-09-19T05:15:27.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:25.874+0000 7fa0e6200640 1 -- 172.21.15.43:0/997785415 wait complete. 2024-09-19T05:15:27.325 DEBUG:tasks.check_counter:Getting stats from a 2024-09-19T05:15:27.325 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-19T05:15:27.326 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:25 smithi043 ceph-mon[30800]: pgmap v336: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T05:15:27.326 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:25 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/2573401014' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T05:15:27.326 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:25 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/650246046' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T05:15:27.326 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:25 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/997785415' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T05:15:27.714 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:15:28.170 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:27 smithi043 ceph-mon[30800]: pgmap v337: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-19T05:15:28.207 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:27 smithi151 ceph-mon[35425]: pgmap v337: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-19T05:15:28.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:27 smithi096 ceph-mon[35085]: pgmap v337: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-19T05:15:28.317 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:15:28.317 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T05:15:28.317 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 1014, 2024-09-19T05:15:28.317 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 1001, 2024-09-19T05:15:28.317 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 1883200, 2024-09-19T05:15:28.317 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 379248, 2024-09-19T05:15:28.318 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 5, 2024-09-19T05:15:28.318 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 1, 2024-09-19T05:15:28.318 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.179353639, 2024-09-19T05:15:28.318 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.080186097, 2024-09-19T05:15:28.318 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.076477147, 2024-09-19T05:15:28.318 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.027194449, 2024-09-19T05:15:28.318 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:15:28.318 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 1000, 2024-09-19T05:15:28.318 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.103666578, 2024-09-19T05:15:28.318 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000103666 2024-09-19T05:15:28.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.319 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:15:28.319 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:28.319 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:28.319 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:28.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.319 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 1625152, 2024-09-19T05:15:28.319 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 135024 2024-09-19T05:15:28.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.319 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T05:15:28.319 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 1456, 2024-09-19T05:15:28.320 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 1618, 2024-09-19T05:15:28.320 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 372012, 2024-09-19T05:15:28.320 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 914333, 2024-09-19T05:15:28.320 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 7, 2024-09-19T05:15:28.320 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 5, 2024-09-19T05:15:28.320 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.153543630, 2024-09-19T05:15:28.320 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.050997113, 2024-09-19T05:15:28.320 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.037803729, 2024-09-19T05:15:28.320 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.053170314, 2024-09-19T05:15:28.320 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:15:28.321 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 1615, 2024-09-19T05:15:28.321 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.095093877, 2024-09-19T05:15:28.321 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000058881 2024-09-19T05:15:28.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.321 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:15:28.321 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:28.321 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:28.321 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:28.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.321 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T05:15:28.322 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 451904 2024-09-19T05:15:28.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.322 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T05:15:28.322 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 8, 2024-09-19T05:15:28.322 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 8, 2024-09-19T05:15:28.322 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 4653, 2024-09-19T05:15:28.322 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 1732, 2024-09-19T05:15:28.322 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 5, 2024-09-19T05:15:28.322 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 4, 2024-09-19T05:15:28.322 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.007518424, 2024-09-19T05:15:28.322 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.000195902, 2024-09-19T05:15:28.323 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.007094870, 2024-09-19T05:15:28.323 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.000098391, 2024-09-19T05:15:28.323 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:15:28.323 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 7, 2024-09-19T05:15:28.323 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.013145824, 2024-09-19T05:15:28.323 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.001877974 2024-09-19T05:15:28.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.323 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:15:28.323 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:28.324 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:28.324 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:28.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.324 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-19T05:15:28.324 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T05:15:28.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.324 INFO:teuthology.orchestra.run.smithi043.stdout: "cct": { 2024-09-19T05:15:28.324 INFO:teuthology.orchestra.run.smithi043.stdout: "total_workers": 1, 2024-09-19T05:15:28.324 INFO:teuthology.orchestra.run.smithi043.stdout: "unhealthy_workers": 0 2024-09-19T05:15:28.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.324 INFO:teuthology.orchestra.run.smithi043.stdout: "finisher-MDSRank": { 2024-09-19T05:15:28.324 INFO:teuthology.orchestra.run.smithi043.stdout: "queue_len": 0, 2024-09-19T05:15:28.324 INFO:teuthology.orchestra.run.smithi043.stdout: "complete_latency": { 2024-09-19T05:15:28.325 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 3723, 2024-09-19T05:15:28.325 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.100672310, 2024-09-19T05:15:28.325 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000027040 2024-09-19T05:15:28.325 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:28.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.325 INFO:teuthology.orchestra.run.smithi043.stdout: "finisher-PurgeQueue": { 2024-09-19T05:15:28.325 INFO:teuthology.orchestra.run.smithi043.stdout: "queue_len": 0, 2024-09-19T05:15:28.325 INFO:teuthology.orchestra.run.smithi043.stdout: "complete_latency": { 2024-09-19T05:15:28.325 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:28.326 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:28.326 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:28.326 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:28.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.326 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": { 2024-09-19T05:15:28.326 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-09-19T05:15:28.326 INFO:teuthology.orchestra.run.smithi043.stdout: "reply": 0, 2024-09-19T05:15:28.326 INFO:teuthology.orchestra.run.smithi043.stdout: "reply_latency": { 2024-09-19T05:15:28.326 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:28.326 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:28.327 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:28.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.327 INFO:teuthology.orchestra.run.smithi043.stdout: "slow_reply": 0, 2024-09-19T05:15:28.327 INFO:teuthology.orchestra.run.smithi043.stdout: "forward": 0, 2024-09-19T05:15:28.327 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_fetch_complete": 0, 2024-09-19T05:15:28.327 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_fetch_keys": 0, 2024-09-19T05:15:28.327 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_commit": 0, 2024-09-19T05:15:28.327 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_split": 0, 2024-09-19T05:15:28.328 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_merge": 0, 2024-09-19T05:15:28.328 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes": 0, 2024-09-19T05:15:28.328 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_top": 0, 2024-09-19T05:15:28.328 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_bottom": 0, 2024-09-19T05:15:28.328 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_pin_tail": 0, 2024-09-19T05:15:28.328 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_pinned": 0, 2024-09-19T05:15:28.329 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_expired": 0, 2024-09-19T05:15:28.329 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_with_caps": 0, 2024-09-19T05:15:28.329 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 0, 2024-09-19T05:15:28.329 INFO:teuthology.orchestra.run.smithi043.stdout: "subtrees": 1, 2024-09-19T05:15:28.329 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse": 0, 2024-09-19T05:15:28.329 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_hit": 0, 2024-09-19T05:15:28.330 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_forward": 0, 2024-09-19T05:15:28.330 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_discover": 0, 2024-09-19T05:15:28.330 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_dir_fetch": 0, 2024-09-19T05:15:28.330 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_remote_ino": 0, 2024-09-19T05:15:28.330 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_lock": 0, 2024-09-19T05:15:28.330 INFO:teuthology.orchestra.run.smithi043.stdout: "load_cent": 0, 2024-09-19T05:15:28.330 INFO:teuthology.orchestra.run.smithi043.stdout: "q": 0, 2024-09-19T05:15:28.330 INFO:teuthology.orchestra.run.smithi043.stdout: "exported": 0, 2024-09-19T05:15:28.331 INFO:teuthology.orchestra.run.smithi043.stdout: "exported_inodes": 0, 2024-09-19T05:15:28.331 INFO:teuthology.orchestra.run.smithi043.stdout: "imported": 0, 2024-09-19T05:15:28.331 INFO:teuthology.orchestra.run.smithi043.stdout: "imported_inodes": 0, 2024-09-19T05:15:28.331 INFO:teuthology.orchestra.run.smithi043.stdout: "openino_dir_fetch": 0, 2024-09-19T05:15:28.331 INFO:teuthology.orchestra.run.smithi043.stdout: "openino_backtrace_fetch": 0, 2024-09-19T05:15:28.331 INFO:teuthology.orchestra.run.smithi043.stdout: "openino_peer_discover": 0, 2024-09-19T05:15:28.331 INFO:teuthology.orchestra.run.smithi043.stdout: "root_rfiles": 0, 2024-09-19T05:15:28.331 INFO:teuthology.orchestra.run.smithi043.stdout: "root_rbytes": 0, 2024-09-19T05:15:28.331 INFO:teuthology.orchestra.run.smithi043.stdout: "root_rsnaps": 0, 2024-09-19T05:15:28.331 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T05:15:28.331 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_set_tag": 0, 2024-09-19T05:15:28.332 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T05:15:28.332 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_inotable_repaired": 0, 2024-09-19T05:15:28.332 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_dir_inodes": 0, 2024-09-19T05:15:28.332 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T05:15:28.332 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T05:15:28.332 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_file_inodes": 0, 2024-09-19T05:15:28.333 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_inode_file_caps": 0, 2024-09-19T05:15:28.333 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T05:15:28.333 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_grant": 0, 2024-09-19T05:15:28.333 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T05:15:28.333 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T05:15:28.333 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T05:15:28.333 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_caps": 0, 2024-09-19T05:15:28.333 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_caps_dirty": 0, 2024-09-19T05:15:28.333 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_cap_release": 0, 2024-09-19T05:15:28.333 INFO:teuthology.orchestra.run.smithi043.stdout: "process_request_cap_release": 0 2024-09-19T05:15:28.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.333 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_cache": { 2024-09-19T05:15:28.333 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_update": 0, 2024-09-19T05:15:28.333 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_update_receipt": 0, 2024-09-19T05:15:28.334 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_try_discover": 0, 2024-09-19T05:15:28.334 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_send_discover": 0, 2024-09-19T05:15:28.334 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_handle_discover": 0, 2024-09-19T05:15:28.334 INFO:teuthology.orchestra.run.smithi043.stdout: "num_strays": 0, 2024-09-19T05:15:28.334 INFO:teuthology.orchestra.run.smithi043.stdout: "num_strays_delayed": 0, 2024-09-19T05:15:28.334 INFO:teuthology.orchestra.run.smithi043.stdout: "num_strays_enqueuing": 0, 2024-09-19T05:15:28.334 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_created": 0, 2024-09-19T05:15:28.334 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_enqueued": 0, 2024-09-19T05:15:28.334 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_reintegrated": 0, 2024-09-19T05:15:28.334 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_migrated": 0, 2024-09-19T05:15:28.334 INFO:teuthology.orchestra.run.smithi043.stdout: "num_recovering_processing": 0, 2024-09-19T05:15:28.334 INFO:teuthology.orchestra.run.smithi043.stdout: "num_recovering_enqueued": 0, 2024-09-19T05:15:28.334 INFO:teuthology.orchestra.run.smithi043.stdout: "num_recovering_prioritized": 0, 2024-09-19T05:15:28.334 INFO:teuthology.orchestra.run.smithi043.stdout: "recovery_started": 0, 2024-09-19T05:15:28.334 INFO:teuthology.orchestra.run.smithi043.stdout: "recovery_completed": 0, 2024-09-19T05:15:28.335 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_quiesce_path": 0, 2024-09-19T05:15:28.335 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_quiesce_inode": 0, 2024-09-19T05:15:28.335 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T05:15:28.335 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_exportdir": 0, 2024-09-19T05:15:28.335 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_flush": 0, 2024-09-19T05:15:28.335 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_fragmentdir": 0, 2024-09-19T05:15:28.335 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_fragstats": 0, 2024-09-19T05:15:28.335 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_inodestats": 0 2024-09-19T05:15:28.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.335 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_log": { 2024-09-19T05:15:28.335 INFO:teuthology.orchestra.run.smithi043.stdout: "evlrg": 0, 2024-09-19T05:15:28.335 INFO:teuthology.orchestra.run.smithi043.stdout: "evadd": 0, 2024-09-19T05:15:28.335 INFO:teuthology.orchestra.run.smithi043.stdout: "evex": 0, 2024-09-19T05:15:28.335 INFO:teuthology.orchestra.run.smithi043.stdout: "evtrm": 0, 2024-09-19T05:15:28.336 INFO:teuthology.orchestra.run.smithi043.stdout: "ev": 6, 2024-09-19T05:15:28.336 INFO:teuthology.orchestra.run.smithi043.stdout: "evexg": 0, 2024-09-19T05:15:28.336 INFO:teuthology.orchestra.run.smithi043.stdout: "evexd": 0, 2024-09-19T05:15:28.336 INFO:teuthology.orchestra.run.smithi043.stdout: "segadd": 0, 2024-09-19T05:15:28.336 INFO:teuthology.orchestra.run.smithi043.stdout: "segex": 0, 2024-09-19T05:15:28.336 INFO:teuthology.orchestra.run.smithi043.stdout: "segtrm": 0, 2024-09-19T05:15:28.336 INFO:teuthology.orchestra.run.smithi043.stdout: "seg": 2, 2024-09-19T05:15:28.336 INFO:teuthology.orchestra.run.smithi043.stdout: "segmjr": 2, 2024-09-19T05:15:28.336 INFO:teuthology.orchestra.run.smithi043.stdout: "segexg": 0, 2024-09-19T05:15:28.336 INFO:teuthology.orchestra.run.smithi043.stdout: "segexd": 0, 2024-09-19T05:15:28.336 INFO:teuthology.orchestra.run.smithi043.stdout: "expos": 4194304, 2024-09-19T05:15:28.336 INFO:teuthology.orchestra.run.smithi043.stdout: "wrpos": 4195960, 2024-09-19T05:15:28.336 INFO:teuthology.orchestra.run.smithi043.stdout: "rdpos": 4195819, 2024-09-19T05:15:28.336 INFO:teuthology.orchestra.run.smithi043.stdout: "jlat": { 2024-09-19T05:15:28.336 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:28.337 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:28.337 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:28.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.337 INFO:teuthology.orchestra.run.smithi043.stdout: "replayed": 6 2024-09-19T05:15:28.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.337 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_mem": { 2024-09-19T05:15:28.337 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 3, 2024-09-19T05:15:28.337 INFO:teuthology.orchestra.run.smithi043.stdout: "ino+": 3, 2024-09-19T05:15:28.337 INFO:teuthology.orchestra.run.smithi043.stdout: "ino-": 0, 2024-09-19T05:15:28.337 INFO:teuthology.orchestra.run.smithi043.stdout: "dir": 1, 2024-09-19T05:15:28.337 INFO:teuthology.orchestra.run.smithi043.stdout: "dir+": 1, 2024-09-19T05:15:28.337 INFO:teuthology.orchestra.run.smithi043.stdout: "dir-": 0, 2024-09-19T05:15:28.337 INFO:teuthology.orchestra.run.smithi043.stdout: "dn": 0, 2024-09-19T05:15:28.337 INFO:teuthology.orchestra.run.smithi043.stdout: "dn+": 0, 2024-09-19T05:15:28.338 INFO:teuthology.orchestra.run.smithi043.stdout: "dn-": 0, 2024-09-19T05:15:28.338 INFO:teuthology.orchestra.run.smithi043.stdout: "cap": 0, 2024-09-19T05:15:28.338 INFO:teuthology.orchestra.run.smithi043.stdout: "cap+": 0, 2024-09-19T05:15:28.338 INFO:teuthology.orchestra.run.smithi043.stdout: "cap-": 0, 2024-09-19T05:15:28.338 INFO:teuthology.orchestra.run.smithi043.stdout: "rss": 47384, 2024-09-19T05:15:28.338 INFO:teuthology.orchestra.run.smithi043.stdout: "heap": 215328 2024-09-19T05:15:28.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.338 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_server": { 2024-09-19T05:15:28.338 INFO:teuthology.orchestra.run.smithi043.stdout: "dispatch_client_request": 0, 2024-09-19T05:15:28.338 INFO:teuthology.orchestra.run.smithi043.stdout: "dispatch_server_request": 0, 2024-09-19T05:15:28.338 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_request": 0, 2024-09-19T05:15:28.338 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_session": 0, 2024-09-19T05:15:28.338 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_peer_request": 0, 2024-09-19T05:15:28.339 INFO:teuthology.orchestra.run.smithi043.stdout: "req_create_latency": { 2024-09-19T05:15:28.339 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:28.339 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:28.339 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:28.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.339 INFO:teuthology.orchestra.run.smithi043.stdout: "req_getattr_latency": { 2024-09-19T05:15:28.339 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:28.339 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:28.339 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:28.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.339 INFO:teuthology.orchestra.run.smithi043.stdout: "req_getfilelock_latency": { 2024-09-19T05:15:28.339 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:28.339 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:28.339 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:28.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.340 INFO:teuthology.orchestra.run.smithi043.stdout: "req_link_latency": { 2024-09-19T05:15:28.340 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:28.340 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:28.340 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:28.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.340 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookup_latency": { 2024-09-19T05:15:28.340 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:28.340 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:28.340 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:28.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.340 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookuphash_latency": { 2024-09-19T05:15:28.340 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:28.340 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:28.341 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:28.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.341 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupino_latency": { 2024-09-19T05:15:28.341 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:28.341 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:28.341 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:28.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.341 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupname_latency": { 2024-09-19T05:15:28.341 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:28.341 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:28.341 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:28.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.341 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupparent_latency": { 2024-09-19T05:15:28.341 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:28.341 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:28.342 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:28.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.342 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupsnap_latency": { 2024-09-19T05:15:28.342 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:28.342 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:28.342 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:28.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.342 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lssnap_latency": { 2024-09-19T05:15:28.342 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:28.342 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:28.342 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:28.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.343 INFO:teuthology.orchestra.run.smithi043.stdout: "req_mkdir_latency": { 2024-09-19T05:15:28.343 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:28.343 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:28.343 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:28.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.344 INFO:teuthology.orchestra.run.smithi043.stdout: "req_mknod_latency": { 2024-09-19T05:15:28.344 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:28.344 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:28.344 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:28.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.344 INFO:teuthology.orchestra.run.smithi043.stdout: "req_mksnap_latency": { 2024-09-19T05:15:28.344 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:28.344 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:28.344 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:28.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.344 INFO:teuthology.orchestra.run.smithi043.stdout: "req_open_latency": { 2024-09-19T05:15:28.344 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:28.345 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:28.345 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:28.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.345 INFO:teuthology.orchestra.run.smithi043.stdout: "req_readdir_latency": { 2024-09-19T05:15:28.345 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:28.345 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:28.345 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:28.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.345 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rename_latency": { 2024-09-19T05:15:28.345 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:28.345 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:28.345 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:28.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.346 INFO:teuthology.orchestra.run.smithi043.stdout: "req_renamesnap_latency": { 2024-09-19T05:15:28.346 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:28.346 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:28.346 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:28.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.346 INFO:teuthology.orchestra.run.smithi043.stdout: "req_snapdiff_latency": { 2024-09-19T05:15:28.346 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:28.346 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:28.346 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:28.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.346 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rmdir_latency": { 2024-09-19T05:15:28.346 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:28.346 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:28.347 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:28.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.347 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rmsnap_latency": { 2024-09-19T05:15:28.347 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:28.347 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:28.347 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:28.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.347 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rmxattr_latency": { 2024-09-19T05:15:28.347 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:28.347 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:28.347 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:28.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.347 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setattr_latency": { 2024-09-19T05:15:28.347 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:28.348 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:28.348 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:28.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.348 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setdirlayout_latency": { 2024-09-19T05:15:28.348 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:28.348 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:28.348 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:28.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.348 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setfilelock_latency": { 2024-09-19T05:15:28.348 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:28.348 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:28.348 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:28.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.348 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setlayout_latency": { 2024-09-19T05:15:28.349 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:28.349 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:28.349 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:28.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.349 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setxattr_latency": { 2024-09-19T05:15:28.349 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:28.349 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:28.349 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:28.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.349 INFO:teuthology.orchestra.run.smithi043.stdout: "req_symlink_latency": { 2024-09-19T05:15:28.349 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:28.349 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:28.349 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:28.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.350 INFO:teuthology.orchestra.run.smithi043.stdout: "req_unlink_latency": { 2024-09-19T05:15:28.350 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:28.350 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:28.350 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:28.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.350 INFO:teuthology.orchestra.run.smithi043.stdout: "cap_revoke_eviction": 0, 2024-09-19T05:15:28.350 INFO:teuthology.orchestra.run.smithi043.stdout: "cap_acquisition_throttle": 0, 2024-09-19T05:15:28.350 INFO:teuthology.orchestra.run.smithi043.stdout: "req_getvxattr_latency": { 2024-09-19T05:15:28.350 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:28.350 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:28.350 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:28.351 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:28.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.351 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_sessions": { 2024-09-19T05:15:28.351 INFO:teuthology.orchestra.run.smithi043.stdout: "session_count": 0, 2024-09-19T05:15:28.351 INFO:teuthology.orchestra.run.smithi043.stdout: "session_add": 2, 2024-09-19T05:15:28.351 INFO:teuthology.orchestra.run.smithi043.stdout: "session_remove": 2, 2024-09-19T05:15:28.351 INFO:teuthology.orchestra.run.smithi043.stdout: "sessions_open": 1, 2024-09-19T05:15:28.351 INFO:teuthology.orchestra.run.smithi043.stdout: "sessions_stale": 0, 2024-09-19T05:15:28.351 INFO:teuthology.orchestra.run.smithi043.stdout: "total_load": 0, 2024-09-19T05:15:28.351 INFO:teuthology.orchestra.run.smithi043.stdout: "average_load": 0, 2024-09-19T05:15:28.351 INFO:teuthology.orchestra.run.smithi043.stdout: "avg_session_uptime": 1192, 2024-09-19T05:15:28.351 INFO:teuthology.orchestra.run.smithi043.stdout: "mdthresh_evicted": 0 2024-09-19T05:15:28.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.352 INFO:teuthology.orchestra.run.smithi043.stdout: "mempool": { 2024-09-19T05:15:28.352 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_bytes": 0, 2024-09-19T05:15:28.352 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_items": 0, 2024-09-19T05:15:28.352 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T05:15:28.352 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_items": 0, 2024-09-19T05:15:28.352 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T05:15:28.352 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_items": 0, 2024-09-19T05:15:28.352 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T05:15:28.352 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T05:15:28.353 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T05:15:28.353 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T05:15:28.353 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T05:15:28.353 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_items": 0, 2024-09-19T05:15:28.353 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T05:15:28.353 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T05:15:28.353 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_bytes": 0, 2024-09-19T05:15:28.353 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_items": 0, 2024-09-19T05:15:28.353 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_bytes": 0, 2024-09-19T05:15:28.353 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_items": 0, 2024-09-19T05:15:28.353 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T05:15:28.354 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T05:15:28.354 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T05:15:28.354 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T05:15:28.354 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T05:15:28.354 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_items": 0, 2024-09-19T05:15:28.354 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_bytes": 0, 2024-09-19T05:15:28.355 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_items": 0, 2024-09-19T05:15:28.355 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T05:15:28.355 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T05:15:28.355 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_bytes": 0, 2024-09-19T05:15:28.355 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_items": 0, 2024-09-19T05:15:28.355 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_bytes": 0, 2024-09-19T05:15:28.356 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_items": 0, 2024-09-19T05:15:28.356 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T05:15:28.356 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_items": 0, 2024-09-19T05:15:28.356 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T05:15:28.356 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_items": 0, 2024-09-19T05:15:28.356 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_bytes": 198504, 2024-09-19T05:15:28.357 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_items": 56, 2024-09-19T05:15:28.357 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_bytes": 0, 2024-09-19T05:15:28.357 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_items": 0, 2024-09-19T05:15:28.357 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_bytes": 0, 2024-09-19T05:15:28.357 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_items": 0, 2024-09-19T05:15:28.357 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_bytes": 0, 2024-09-19T05:15:28.358 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_items": 0, 2024-09-19T05:15:28.358 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_bytes": 0, 2024-09-19T05:15:28.358 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_items": 0, 2024-09-19T05:15:28.358 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_bytes": 8224, 2024-09-19T05:15:28.358 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_items": 181, 2024-09-19T05:15:28.358 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T05:15:28.358 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_items": 0, 2024-09-19T05:15:28.358 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_bytes": 0, 2024-09-19T05:15:28.358 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_items": 0, 2024-09-19T05:15:28.359 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_bytes": 8512, 2024-09-19T05:15:28.359 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_items": 13, 2024-09-19T05:15:28.359 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_bytes": 0, 2024-09-19T05:15:28.359 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_items": 0, 2024-09-19T05:15:28.359 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_bytes": 0, 2024-09-19T05:15:28.359 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_items": 0 2024-09-19T05:15:28.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.359 INFO:teuthology.orchestra.run.smithi043.stdout: "objecter": { 2024-09-19T05:15:28.359 INFO:teuthology.orchestra.run.smithi043.stdout: "op_active": 0, 2024-09-19T05:15:28.360 INFO:teuthology.orchestra.run.smithi043.stdout: "op_laggy": 0, 2024-09-19T05:15:28.360 INFO:teuthology.orchestra.run.smithi043.stdout: "op_send": 2226, 2024-09-19T05:15:28.360 INFO:teuthology.orchestra.run.smithi043.stdout: "op_send_bytes": 21, 2024-09-19T05:15:28.360 INFO:teuthology.orchestra.run.smithi043.stdout: "op_resend": 3, 2024-09-19T05:15:28.360 INFO:teuthology.orchestra.run.smithi043.stdout: "op_reply": 2226, 2024-09-19T05:15:28.360 INFO:teuthology.orchestra.run.smithi043.stdout: "op_latency": { 2024-09-19T05:15:28.360 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 2226, 2024-09-19T05:15:28.360 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 1.573981158, 2024-09-19T05:15:28.360 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000707089 2024-09-19T05:15:28.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.361 INFO:teuthology.orchestra.run.smithi043.stdout: "op_inflight": 0, 2024-09-19T05:15:28.361 INFO:teuthology.orchestra.run.smithi043.stdout: "oplen_avg": { 2024-09-19T05:15:28.361 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 2226, 2024-09-19T05:15:28.361 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 2227 2024-09-19T05:15:28.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.361 INFO:teuthology.orchestra.run.smithi043.stdout: "op": 2226, 2024-09-19T05:15:28.361 INFO:teuthology.orchestra.run.smithi043.stdout: "op_r": 2226, 2024-09-19T05:15:28.361 INFO:teuthology.orchestra.run.smithi043.stdout: "op_w": 0, 2024-09-19T05:15:28.361 INFO:teuthology.orchestra.run.smithi043.stdout: "op_rmw": 0, 2024-09-19T05:15:28.361 INFO:teuthology.orchestra.run.smithi043.stdout: "op_pg": 0, 2024-09-19T05:15:28.361 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_stat": 1448, 2024-09-19T05:15:28.362 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_create": 0, 2024-09-19T05:15:28.362 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_read": 776, 2024-09-19T05:15:28.362 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_write": 0, 2024-09-19T05:15:28.362 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_writefull": 0, 2024-09-19T05:15:28.362 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_writesame": 0, 2024-09-19T05:15:28.362 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_append": 0, 2024-09-19T05:15:28.362 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_zero": 0, 2024-09-19T05:15:28.362 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_truncate": 0, 2024-09-19T05:15:28.362 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_delete": 0, 2024-09-19T05:15:28.362 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_mapext": 0, 2024-09-19T05:15:28.362 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_sparse_read": 0, 2024-09-19T05:15:28.362 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_clonerange": 0, 2024-09-19T05:15:28.362 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_getxattr": 1, 2024-09-19T05:15:28.362 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_setxattr": 0, 2024-09-19T05:15:28.362 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_cmpxattr": 0, 2024-09-19T05:15:28.363 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_rmxattr": 0, 2024-09-19T05:15:28.363 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_resetxattrs": 0, 2024-09-19T05:15:28.363 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_call": 0, 2024-09-19T05:15:28.363 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_watch": 0, 2024-09-19T05:15:28.363 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_notify": 0, 2024-09-19T05:15:28.363 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T05:15:28.363 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_pgls": 0, 2024-09-19T05:15:28.363 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_pgls_filter": 0, 2024-09-19T05:15:28.363 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_other": 0, 2024-09-19T05:15:28.363 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_active": 0, 2024-09-19T05:15:28.363 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_send": 0, 2024-09-19T05:15:28.363 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_resend": 0, 2024-09-19T05:15:28.363 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_ping": 0, 2024-09-19T05:15:28.363 INFO:teuthology.orchestra.run.smithi043.stdout: "poolop_active": 0, 2024-09-19T05:15:28.363 INFO:teuthology.orchestra.run.smithi043.stdout: "poolop_send": 0, 2024-09-19T05:15:28.364 INFO:teuthology.orchestra.run.smithi043.stdout: "poolop_resend": 0, 2024-09-19T05:15:28.364 INFO:teuthology.orchestra.run.smithi043.stdout: "poolstat_active": 0, 2024-09-19T05:15:28.364 INFO:teuthology.orchestra.run.smithi043.stdout: "poolstat_send": 0, 2024-09-19T05:15:28.364 INFO:teuthology.orchestra.run.smithi043.stdout: "poolstat_resend": 0, 2024-09-19T05:15:28.364 INFO:teuthology.orchestra.run.smithi043.stdout: "statfs_active": 0, 2024-09-19T05:15:28.364 INFO:teuthology.orchestra.run.smithi043.stdout: "statfs_send": 0, 2024-09-19T05:15:28.364 INFO:teuthology.orchestra.run.smithi043.stdout: "statfs_resend": 0, 2024-09-19T05:15:28.364 INFO:teuthology.orchestra.run.smithi043.stdout: "command_active": 0, 2024-09-19T05:15:28.364 INFO:teuthology.orchestra.run.smithi043.stdout: "command_send": 0, 2024-09-19T05:15:28.364 INFO:teuthology.orchestra.run.smithi043.stdout: "command_resend": 0, 2024-09-19T05:15:28.364 INFO:teuthology.orchestra.run.smithi043.stdout: "map_epoch": 96, 2024-09-19T05:15:28.365 INFO:teuthology.orchestra.run.smithi043.stdout: "map_full": 0, 2024-09-19T05:15:28.365 INFO:teuthology.orchestra.run.smithi043.stdout: "map_inc": 13, 2024-09-19T05:15:28.365 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_sessions": 6, 2024-09-19T05:15:28.365 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_session_open": 6, 2024-09-19T05:15:28.365 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_session_close": 0, 2024-09-19T05:15:28.365 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_laggy": 0, 2024-09-19T05:15:28.365 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_wr": 0, 2024-09-19T05:15:28.365 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_rd": 2, 2024-09-19T05:15:28.365 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_del": 0 2024-09-19T05:15:28.365 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.366 INFO:teuthology.orchestra.run.smithi043.stdout: "oft": { 2024-09-19T05:15:28.366 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_objs": 0, 2024-09-19T05:15:28.366 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_kv_pairs": 0, 2024-09-19T05:15:28.366 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_updates": 0, 2024-09-19T05:15:28.366 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_removes": 0 2024-09-19T05:15:28.366 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.366 INFO:teuthology.orchestra.run.smithi043.stdout: "osd-slow-ops": { 2024-09-19T05:15:28.366 INFO:teuthology.orchestra.run.smithi043.stdout: "slow_ops_count": 0 2024-09-19T05:15:28.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.367 INFO:teuthology.orchestra.run.smithi043.stdout: "purge_queue": { 2024-09-19T05:15:28.367 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing_ops": 0, 2024-09-19T05:15:28.367 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T05:15:28.367 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing": 0, 2024-09-19T05:15:28.367 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing_high_water": 0, 2024-09-19T05:15:28.367 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executed_ops": 0, 2024-09-19T05:15:28.367 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executed": 0, 2024-09-19T05:15:28.367 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_item_in_journal": 0 2024-09-19T05:15:28.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.367 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T05:15:28.367 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-09-19T05:15:28.367 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 104857600, 2024-09-19T05:15:28.367 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-09-19T05:15:28.368 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 2474, 2024-09-19T05:15:28.368 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 2021567, 2024-09-19T05:15:28.368 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:28.368 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 2474, 2024-09-19T05:15:28.368 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-09-19T05:15:28.368 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-09-19T05:15:28.368 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 2474, 2024-09-19T05:15:28.368 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 2021567, 2024-09-19T05:15:28.368 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-09-19T05:15:28.368 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:28.368 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:28.368 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:28.368 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:28.368 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.369 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-objecter_bytes": { 2024-09-19T05:15:28.369 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-09-19T05:15:28.369 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 104857600, 2024-09-19T05:15:28.369 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-09-19T05:15:28.369 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 0, 2024-09-19T05:15:28.369 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 0, 2024-09-19T05:15:28.369 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:28.369 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 0, 2024-09-19T05:15:28.369 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 2226, 2024-09-19T05:15:28.369 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 1661, 2024-09-19T05:15:28.369 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 2226, 2024-09-19T05:15:28.369 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 1661, 2024-09-19T05:15:28.369 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-09-19T05:15:28.369 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:28.369 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:28.370 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:28.370 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:28.370 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.370 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-objecter_ops": { 2024-09-19T05:15:28.370 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-09-19T05:15:28.370 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 1024, 2024-09-19T05:15:28.370 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-09-19T05:15:28.370 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 0, 2024-09-19T05:15:28.370 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 0, 2024-09-19T05:15:28.370 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:28.370 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 0, 2024-09-19T05:15:28.370 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 2226, 2024-09-19T05:15:28.370 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 2226, 2024-09-19T05:15:28.370 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 2226, 2024-09-19T05:15:28.371 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 2226, 2024-09-19T05:15:28.371 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-09-19T05:15:28.371 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:28.371 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:28.371 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:28.371 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:28.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.371 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-write_buf_throttle": { 2024-09-19T05:15:28.371 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-09-19T05:15:28.371 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 3758096384, 2024-09-19T05:15:28.371 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-09-19T05:15:28.371 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 0, 2024-09-19T05:15:28.371 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 0, 2024-09-19T05:15:28.371 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:28.371 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 0, 2024-09-19T05:15:28.372 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-09-19T05:15:28.372 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-09-19T05:15:28.372 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 0, 2024-09-19T05:15:28.372 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 0, 2024-09-19T05:15:28.372 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-09-19T05:15:28.372 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:28.372 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:28.372 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:28.372 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:28.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:28.372 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-write_buf_throttle-0x555eb5c66be0": { 2024-09-19T05:15:28.372 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-09-19T05:15:28.372 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 3758096384, 2024-09-19T05:15:28.372 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-09-19T05:15:28.373 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 0, 2024-09-19T05:15:28.373 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 0, 2024-09-19T05:15:28.373 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:28.373 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 0, 2024-09-19T05:15:28.373 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-09-19T05:15:28.373 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-09-19T05:15:28.373 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 0, 2024-09-19T05:15:28.373 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 0, 2024-09-19T05:15:28.373 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-09-19T05:15:28.373 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:28.373 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:28.373 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:28.373 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:28.373 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:28.374 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:15:29.209 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-19T05:15:29.210 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-19T05:15:29.210 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-09-19T05:15:29.210 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-09-19T05:15:29.210 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T05:15:29.210 DEBUG:tasks.check_counter:Getting stats from d 2024-09-19T05:15:29.210 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-19T05:15:29.590 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:15:30.014 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:29 smithi043 ceph-mon[30800]: pgmap v338: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:15:30.160 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:15:30.161 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T05:15:30.161 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 0, 2024-09-19T05:15:30.161 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 2, 2024-09-19T05:15:30.161 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 0, 2024-09-19T05:15:30.161 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 0, 2024-09-19T05:15:30.161 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 4, 2024-09-19T05:15:30.161 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 0, 2024-09-19T05:15:30.161 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.003537966, 2024-09-19T05:15:30.161 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.000000000, 2024-09-19T05:15:30.161 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.000739450, 2024-09-19T05:15:30.161 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T05:15:30.161 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:15:30.161 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:30.161 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:30.162 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:30.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:30.162 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:15:30.162 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:30.162 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:30.162 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:30.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:30.162 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T05:15:30.162 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T05:15:30.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:30.162 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T05:15:30.162 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 236, 2024-09-19T05:15:30.162 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 223, 2024-09-19T05:15:30.162 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 1592512, 2024-09-19T05:15:30.163 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 136192, 2024-09-19T05:15:30.163 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 2, 2024-09-19T05:15:30.163 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 1, 2024-09-19T05:15:30.163 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.057532076, 2024-09-19T05:15:30.163 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.022513007, 2024-09-19T05:15:30.163 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.051547754, 2024-09-19T05:15:30.163 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T05:15:30.163 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:15:30.163 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 223, 2024-09-19T05:15:30.163 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.022364359, 2024-09-19T05:15:30.163 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000100288 2024-09-19T05:15:30.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:30.163 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:15:30.163 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:30.164 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:30.164 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:30.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:30.164 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 1592512, 2024-09-19T05:15:30.164 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 136192 2024-09-19T05:15:30.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:30.164 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T05:15:30.164 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 2, 2024-09-19T05:15:30.164 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 169, 2024-09-19T05:15:30.164 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 320, 2024-09-19T05:15:30.164 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 186976, 2024-09-19T05:15:30.164 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 5, 2024-09-19T05:15:30.164 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 3, 2024-09-19T05:15:30.164 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.021907580, 2024-09-19T05:15:30.165 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.015329970, 2024-09-19T05:15:30.165 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.002848333, 2024-09-19T05:15:30.165 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T05:15:30.165 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:15:30.165 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 166, 2024-09-19T05:15:30.165 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.023102585, 2024-09-19T05:15:30.165 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000139172 2024-09-19T05:15:30.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:30.165 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:15:30.165 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:30.165 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:30.165 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:30.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:30.165 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T05:15:30.166 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 186976 2024-09-19T05:15:30.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:30.166 INFO:teuthology.orchestra.run.smithi043.stdout: "cct": { 2024-09-19T05:15:30.166 INFO:teuthology.orchestra.run.smithi043.stdout: "total_workers": 0, 2024-09-19T05:15:30.166 INFO:teuthology.orchestra.run.smithi043.stdout: "unhealthy_workers": 0 2024-09-19T05:15:30.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:30.166 INFO:teuthology.orchestra.run.smithi043.stdout: "mempool": { 2024-09-19T05:15:30.166 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_bytes": 0, 2024-09-19T05:15:30.166 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_items": 0, 2024-09-19T05:15:30.166 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T05:15:30.166 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_items": 0, 2024-09-19T05:15:30.166 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T05:15:30.166 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_items": 0, 2024-09-19T05:15:30.166 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T05:15:30.166 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T05:15:30.167 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T05:15:30.167 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T05:15:30.167 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T05:15:30.167 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_items": 0, 2024-09-19T05:15:30.167 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T05:15:30.167 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T05:15:30.167 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_bytes": 0, 2024-09-19T05:15:30.167 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_items": 0, 2024-09-19T05:15:30.167 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_bytes": 0, 2024-09-19T05:15:30.167 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_items": 0, 2024-09-19T05:15:30.167 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T05:15:30.167 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T05:15:30.167 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T05:15:30.167 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T05:15:30.167 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T05:15:30.168 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_items": 0, 2024-09-19T05:15:30.168 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_bytes": 0, 2024-09-19T05:15:30.168 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_items": 0, 2024-09-19T05:15:30.168 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T05:15:30.168 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T05:15:30.168 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_bytes": 0, 2024-09-19T05:15:30.168 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_items": 0, 2024-09-19T05:15:30.168 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_bytes": 0, 2024-09-19T05:15:30.168 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_items": 0, 2024-09-19T05:15:30.168 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T05:15:30.169 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_items": 0, 2024-09-19T05:15:30.169 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T05:15:30.169 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_items": 0, 2024-09-19T05:15:30.169 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_bytes": 148977, 2024-09-19T05:15:30.169 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_items": 30, 2024-09-19T05:15:30.169 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_bytes": 0, 2024-09-19T05:15:30.169 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_items": 0, 2024-09-19T05:15:30.169 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_bytes": 0, 2024-09-19T05:15:30.169 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_items": 0, 2024-09-19T05:15:30.169 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_bytes": 0, 2024-09-19T05:15:30.169 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_items": 0, 2024-09-19T05:15:30.169 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_bytes": 0, 2024-09-19T05:15:30.169 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_items": 0, 2024-09-19T05:15:30.169 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_bytes": 0, 2024-09-19T05:15:30.170 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_items": 0, 2024-09-19T05:15:30.170 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T05:15:30.170 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_items": 0, 2024-09-19T05:15:30.170 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_bytes": 0, 2024-09-19T05:15:30.170 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_items": 0, 2024-09-19T05:15:30.170 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_bytes": 928, 2024-09-19T05:15:30.170 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_items": 2, 2024-09-19T05:15:30.170 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_bytes": 0, 2024-09-19T05:15:30.170 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_items": 0, 2024-09-19T05:15:30.170 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_bytes": 0, 2024-09-19T05:15:30.170 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_items": 0 2024-09-19T05:15:30.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:30.170 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T05:15:30.170 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-09-19T05:15:30.171 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 104857600, 2024-09-19T05:15:30.171 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-09-19T05:15:30.171 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 234, 2024-09-19T05:15:30.171 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 1556762, 2024-09-19T05:15:30.171 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:30.171 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 234, 2024-09-19T05:15:30.171 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-09-19T05:15:30.171 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-09-19T05:15:30.171 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 234, 2024-09-19T05:15:30.171 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 1556762, 2024-09-19T05:15:30.171 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-09-19T05:15:30.171 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:30.171 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:30.171 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:30.172 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:30.172 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:30.172 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:15:30.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:29 smithi096 ceph-mon[35085]: pgmap v338: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:15:30.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:29 smithi151 ceph-mon[35425]: pgmap v338: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:15:31.139 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-19T05:15:31.140 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-19T05:15:31.140 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-19T05:15:31.140 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-19T05:15:31.140 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-19T05:15:31.140 DEBUG:tasks.check_counter:Getting stats from g 2024-09-19T05:15:31.140 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-19T05:15:31.520 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:15:32.147 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:15:32.147 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T05:15:32.147 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 685, 2024-09-19T05:15:32.147 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 791, 2024-09-19T05:15:32.147 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 1765066, 2024-09-19T05:15:32.147 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 257621, 2024-09-19T05:15:32.147 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 7, 2024-09-19T05:15:32.147 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 3, 2024-09-19T05:15:32.148 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.136392704, 2024-09-19T05:15:32.148 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.048385184, 2024-09-19T05:15:32.148 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.084745463, 2024-09-19T05:15:32.148 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.005309396, 2024-09-19T05:15:32.148 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:15:32.148 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 791, 2024-09-19T05:15:32.148 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.088814584, 2024-09-19T05:15:32.148 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000112281 2024-09-19T05:15:32.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.148 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:15:32.148 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 890, 2024-09-19T05:15:32.148 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000197033, 2024-09-19T05:15:32.149 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000221 2024-09-19T05:15:32.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.149 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 1656224, 2024-09-19T05:15:32.149 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 141392 2024-09-19T05:15:32.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.149 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T05:15:32.149 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 62, 2024-09-19T05:15:32.149 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 229, 2024-09-19T05:15:32.149 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 12075, 2024-09-19T05:15:32.149 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 903426, 2024-09-19T05:15:32.149 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 47, 2024-09-19T05:15:32.149 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 4, 2024-09-19T05:15:32.149 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.056112821, 2024-09-19T05:15:32.150 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.019332137, 2024-09-19T05:15:32.150 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.034594029, 2024-09-19T05:15:32.150 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.000235437, 2024-09-19T05:15:32.150 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:15:32.150 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 226, 2024-09-19T05:15:32.150 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.037392944, 2024-09-19T05:15:32.150 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000165455 2024-09-19T05:15:32.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.150 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:15:32.150 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 64, 2024-09-19T05:15:32.150 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000008550, 2024-09-19T05:15:32.151 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000133 2024-09-19T05:15:32.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.151 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 4080, 2024-09-19T05:15:32.151 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 459424 2024-09-19T05:15:32.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.151 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T05:15:32.151 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 22, 2024-09-19T05:15:32.151 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 24, 2024-09-19T05:15:32.151 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 5522, 2024-09-19T05:15:32.152 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 62880, 2024-09-19T05:15:32.152 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 10, 2024-09-19T05:15:32.152 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 4, 2024-09-19T05:15:32.152 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.008199961, 2024-09-19T05:15:32.152 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.000715537, 2024-09-19T05:15:32.152 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.006270740, 2024-09-19T05:15:32.152 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.000563503, 2024-09-19T05:15:32.152 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:15:32.152 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 22, 2024-09-19T05:15:32.152 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.028885529, 2024-09-19T05:15:32.152 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.001312978 2024-09-19T05:15:32.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.152 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:15:32.152 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 4, 2024-09-19T05:15:32.153 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000520, 2024-09-19T05:15:32.153 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000130 2024-09-19T05:15:32.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.153 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T05:15:32.153 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T05:15:32.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.153 INFO:teuthology.orchestra.run.smithi043.stdout: "cct": { 2024-09-19T05:15:32.153 INFO:teuthology.orchestra.run.smithi043.stdout: "total_workers": 1, 2024-09-19T05:15:32.153 INFO:teuthology.orchestra.run.smithi043.stdout: "unhealthy_workers": 0 2024-09-19T05:15:32.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.153 INFO:teuthology.orchestra.run.smithi043.stdout: "finisher-MDSRank": { 2024-09-19T05:15:32.153 INFO:teuthology.orchestra.run.smithi043.stdout: "queue_len": 0, 2024-09-19T05:15:32.153 INFO:teuthology.orchestra.run.smithi043.stdout: "complete_latency": { 2024-09-19T05:15:32.154 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 51, 2024-09-19T05:15:32.154 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.001603834, 2024-09-19T05:15:32.154 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000031447 2024-09-19T05:15:32.154 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:32.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.154 INFO:teuthology.orchestra.run.smithi043.stdout: "finisher-PurgeQueue": { 2024-09-19T05:15:32.154 INFO:teuthology.orchestra.run.smithi043.stdout: "queue_len": 0, 2024-09-19T05:15:32.154 INFO:teuthology.orchestra.run.smithi043.stdout: "complete_latency": { 2024-09-19T05:15:32.154 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 2, 2024-09-19T05:15:32.154 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000018235, 2024-09-19T05:15:32.154 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000009117 2024-09-19T05:15:32.154 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:32.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.155 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": { 2024-09-19T05:15:32.155 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 6, 2024-09-19T05:15:32.155 INFO:teuthology.orchestra.run.smithi043.stdout: "reply": 4, 2024-09-19T05:15:32.155 INFO:teuthology.orchestra.run.smithi043.stdout: "reply_latency": { 2024-09-19T05:15:32.155 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 4, 2024-09-19T05:15:32.155 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.001300933, 2024-09-19T05:15:32.155 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000325233 2024-09-19T05:15:32.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.155 INFO:teuthology.orchestra.run.smithi043.stdout: "slow_reply": 0, 2024-09-19T05:15:32.155 INFO:teuthology.orchestra.run.smithi043.stdout: "forward": 2, 2024-09-19T05:15:32.155 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_fetch_complete": 0, 2024-09-19T05:15:32.155 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_fetch_keys": 0, 2024-09-19T05:15:32.155 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_commit": 11, 2024-09-19T05:15:32.155 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_split": 0, 2024-09-19T05:15:32.156 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_merge": 0, 2024-09-19T05:15:32.156 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes": 12, 2024-09-19T05:15:32.156 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_top": 0, 2024-09-19T05:15:32.156 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_bottom": 12, 2024-09-19T05:15:32.156 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_pin_tail": 0, 2024-09-19T05:15:32.156 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_pinned": 11, 2024-09-19T05:15:32.156 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_expired": 0, 2024-09-19T05:15:32.156 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_with_caps": 0, 2024-09-19T05:15:32.156 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 0, 2024-09-19T05:15:32.156 INFO:teuthology.orchestra.run.smithi043.stdout: "subtrees": 2, 2024-09-19T05:15:32.156 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse": 11, 2024-09-19T05:15:32.156 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_hit": 8, 2024-09-19T05:15:32.156 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_forward": 0, 2024-09-19T05:15:32.157 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_discover": 2, 2024-09-19T05:15:32.157 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_dir_fetch": 0, 2024-09-19T05:15:32.157 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_remote_ino": 0, 2024-09-19T05:15:32.157 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_lock": 0, 2024-09-19T05:15:32.157 INFO:teuthology.orchestra.run.smithi043.stdout: "load_cent": 0, 2024-09-19T05:15:32.157 INFO:teuthology.orchestra.run.smithi043.stdout: "q": 0, 2024-09-19T05:15:32.157 INFO:teuthology.orchestra.run.smithi043.stdout: "exported": 0, 2024-09-19T05:15:32.157 INFO:teuthology.orchestra.run.smithi043.stdout: "exported_inodes": 0, 2024-09-19T05:15:32.157 INFO:teuthology.orchestra.run.smithi043.stdout: "imported": 0, 2024-09-19T05:15:32.157 INFO:teuthology.orchestra.run.smithi043.stdout: "imported_inodes": 0, 2024-09-19T05:15:32.157 INFO:teuthology.orchestra.run.smithi043.stdout: "openino_dir_fetch": 0, 2024-09-19T05:15:32.157 INFO:teuthology.orchestra.run.smithi043.stdout: "openino_backtrace_fetch": 0, 2024-09-19T05:15:32.157 INFO:teuthology.orchestra.run.smithi043.stdout: "openino_peer_discover": 0, 2024-09-19T05:15:32.158 INFO:teuthology.orchestra.run.smithi043.stdout: "root_rfiles": 4, 2024-09-19T05:15:32.158 INFO:teuthology.orchestra.run.smithi043.stdout: "root_rbytes": 234, 2024-09-19T05:15:32.158 INFO:teuthology.orchestra.run.smithi043.stdout: "root_rsnaps": 6, 2024-09-19T05:15:32.158 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T05:15:32.158 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_set_tag": 0, 2024-09-19T05:15:32.158 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T05:15:32.158 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_inotable_repaired": 0, 2024-09-19T05:15:32.158 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_dir_inodes": 0, 2024-09-19T05:15:32.158 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T05:15:32.158 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T05:15:32.158 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_file_inodes": 0, 2024-09-19T05:15:32.158 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_inode_file_caps": 0, 2024-09-19T05:15:32.158 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T05:15:32.159 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_grant": 0, 2024-09-19T05:15:32.159 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T05:15:32.159 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T05:15:32.159 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T05:15:32.159 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_caps": 0, 2024-09-19T05:15:32.159 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_caps_dirty": 0, 2024-09-19T05:15:32.159 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_cap_release": 0, 2024-09-19T05:15:32.159 INFO:teuthology.orchestra.run.smithi043.stdout: "process_request_cap_release": 1 2024-09-19T05:15:32.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.159 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_cache": { 2024-09-19T05:15:32.160 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_update": 0, 2024-09-19T05:15:32.160 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_update_receipt": 0, 2024-09-19T05:15:32.160 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_try_discover": 0, 2024-09-19T05:15:32.160 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_send_discover": 3, 2024-09-19T05:15:32.160 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_handle_discover": 0, 2024-09-19T05:15:32.160 INFO:teuthology.orchestra.run.smithi043.stdout: "num_strays": 0, 2024-09-19T05:15:32.160 INFO:teuthology.orchestra.run.smithi043.stdout: "num_strays_delayed": 0, 2024-09-19T05:15:32.160 INFO:teuthology.orchestra.run.smithi043.stdout: "num_strays_enqueuing": 0, 2024-09-19T05:15:32.160 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_created": 0, 2024-09-19T05:15:32.160 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_enqueued": 0, 2024-09-19T05:15:32.160 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_reintegrated": 0, 2024-09-19T05:15:32.160 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_migrated": 0, 2024-09-19T05:15:32.160 INFO:teuthology.orchestra.run.smithi043.stdout: "num_recovering_processing": 0, 2024-09-19T05:15:32.160 INFO:teuthology.orchestra.run.smithi043.stdout: "num_recovering_enqueued": 0, 2024-09-19T05:15:32.161 INFO:teuthology.orchestra.run.smithi043.stdout: "num_recovering_prioritized": 0, 2024-09-19T05:15:32.161 INFO:teuthology.orchestra.run.smithi043.stdout: "recovery_started": 0, 2024-09-19T05:15:32.161 INFO:teuthology.orchestra.run.smithi043.stdout: "recovery_completed": 0, 2024-09-19T05:15:32.161 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_quiesce_path": 0, 2024-09-19T05:15:32.161 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_quiesce_inode": 0, 2024-09-19T05:15:32.161 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T05:15:32.161 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_exportdir": 0, 2024-09-19T05:15:32.161 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_flush": 0, 2024-09-19T05:15:32.161 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_fragmentdir": 0, 2024-09-19T05:15:32.162 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_fragstats": 0, 2024-09-19T05:15:32.162 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_inodestats": 0 2024-09-19T05:15:32.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.162 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_log": { 2024-09-19T05:15:32.162 INFO:teuthology.orchestra.run.smithi043.stdout: "evlrg": 0, 2024-09-19T05:15:32.162 INFO:teuthology.orchestra.run.smithi043.stdout: "evadd": 6, 2024-09-19T05:15:32.163 INFO:teuthology.orchestra.run.smithi043.stdout: "evex": 0, 2024-09-19T05:15:32.163 INFO:teuthology.orchestra.run.smithi043.stdout: "evtrm": 0, 2024-09-19T05:15:32.163 INFO:teuthology.orchestra.run.smithi043.stdout: "ev": 6, 2024-09-19T05:15:32.163 INFO:teuthology.orchestra.run.smithi043.stdout: "evexg": 0, 2024-09-19T05:15:32.163 INFO:teuthology.orchestra.run.smithi043.stdout: "evexd": 0, 2024-09-19T05:15:32.163 INFO:teuthology.orchestra.run.smithi043.stdout: "segadd": 2, 2024-09-19T05:15:32.163 INFO:teuthology.orchestra.run.smithi043.stdout: "segex": 0, 2024-09-19T05:15:32.164 INFO:teuthology.orchestra.run.smithi043.stdout: "segtrm": 0, 2024-09-19T05:15:32.164 INFO:teuthology.orchestra.run.smithi043.stdout: "seg": 2, 2024-09-19T05:15:32.164 INFO:teuthology.orchestra.run.smithi043.stdout: "segmjr": 2, 2024-09-19T05:15:32.164 INFO:teuthology.orchestra.run.smithi043.stdout: "segexg": 0, 2024-09-19T05:15:32.164 INFO:teuthology.orchestra.run.smithi043.stdout: "segexd": 0, 2024-09-19T05:15:32.164 INFO:teuthology.orchestra.run.smithi043.stdout: "expos": 4194304, 2024-09-19T05:15:32.165 INFO:teuthology.orchestra.run.smithi043.stdout: "wrpos": 4195960, 2024-09-19T05:15:32.165 INFO:teuthology.orchestra.run.smithi043.stdout: "rdpos": 0, 2024-09-19T05:15:32.165 INFO:teuthology.orchestra.run.smithi043.stdout: "jlat": { 2024-09-19T05:15:32.165 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 5, 2024-09-19T05:15:32.165 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.010436762, 2024-09-19T05:15:32.165 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.002087352 2024-09-19T05:15:32.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.166 INFO:teuthology.orchestra.run.smithi043.stdout: "replayed": 0 2024-09-19T05:15:32.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.166 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_mem": { 2024-09-19T05:15:32.166 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 15, 2024-09-19T05:15:32.166 INFO:teuthology.orchestra.run.smithi043.stdout: "ino+": 15, 2024-09-19T05:15:32.167 INFO:teuthology.orchestra.run.smithi043.stdout: "ino-": 0, 2024-09-19T05:15:32.167 INFO:teuthology.orchestra.run.smithi043.stdout: "dir": 13, 2024-09-19T05:15:32.167 INFO:teuthology.orchestra.run.smithi043.stdout: "dir+": 14, 2024-09-19T05:15:32.167 INFO:teuthology.orchestra.run.smithi043.stdout: "dir-": 1, 2024-09-19T05:15:32.167 INFO:teuthology.orchestra.run.smithi043.stdout: "dn": 12, 2024-09-19T05:15:32.167 INFO:teuthology.orchestra.run.smithi043.stdout: "dn+": 12, 2024-09-19T05:15:32.167 INFO:teuthology.orchestra.run.smithi043.stdout: "dn-": 0, 2024-09-19T05:15:32.168 INFO:teuthology.orchestra.run.smithi043.stdout: "cap": 0, 2024-09-19T05:15:32.168 INFO:teuthology.orchestra.run.smithi043.stdout: "cap+": 4, 2024-09-19T05:15:32.168 INFO:teuthology.orchestra.run.smithi043.stdout: "cap-": 4, 2024-09-19T05:15:32.168 INFO:teuthology.orchestra.run.smithi043.stdout: "rss": 48652, 2024-09-19T05:15:32.168 INFO:teuthology.orchestra.run.smithi043.stdout: "heap": 223520 2024-09-19T05:15:32.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.169 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_server": { 2024-09-19T05:15:32.169 INFO:teuthology.orchestra.run.smithi043.stdout: "dispatch_client_request": 8, 2024-09-19T05:15:32.169 INFO:teuthology.orchestra.run.smithi043.stdout: "dispatch_server_request": 0, 2024-09-19T05:15:32.169 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_request": 6, 2024-09-19T05:15:32.169 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_session": 8, 2024-09-19T05:15:32.170 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_peer_request": 0, 2024-09-19T05:15:32.170 INFO:teuthology.orchestra.run.smithi043.stdout: "req_create_latency": { 2024-09-19T05:15:32.170 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:32.170 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:32.170 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:32.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.171 INFO:teuthology.orchestra.run.smithi043.stdout: "req_getattr_latency": { 2024-09-19T05:15:32.171 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 2, 2024-09-19T05:15:32.171 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000319620, 2024-09-19T05:15:32.171 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000159810 2024-09-19T05:15:32.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.172 INFO:teuthology.orchestra.run.smithi043.stdout: "req_getfilelock_latency": { 2024-09-19T05:15:32.172 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:32.172 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:32.172 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:32.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.172 INFO:teuthology.orchestra.run.smithi043.stdout: "req_link_latency": { 2024-09-19T05:15:32.173 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:32.173 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:32.173 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:32.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.174 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookup_latency": { 2024-09-19T05:15:32.174 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 2, 2024-09-19T05:15:32.174 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000981313, 2024-09-19T05:15:32.174 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000490656 2024-09-19T05:15:32.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.175 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookuphash_latency": { 2024-09-19T05:15:32.175 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:32.175 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:32.175 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:32.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.175 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupino_latency": { 2024-09-19T05:15:32.176 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:32.176 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:32.176 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:32.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.176 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupname_latency": { 2024-09-19T05:15:32.176 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:32.177 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:32.177 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:32.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.177 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupparent_latency": { 2024-09-19T05:15:32.177 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:32.177 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:32.178 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:32.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.178 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupsnap_latency": { 2024-09-19T05:15:32.178 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:32.178 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:32.178 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:32.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.179 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lssnap_latency": { 2024-09-19T05:15:32.179 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:32.179 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:32.179 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:32.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.180 INFO:teuthology.orchestra.run.smithi043.stdout: "req_mkdir_latency": { 2024-09-19T05:15:32.180 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:32.181 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:32.181 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:32.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.181 INFO:teuthology.orchestra.run.smithi043.stdout: "req_mknod_latency": { 2024-09-19T05:15:32.181 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:32.182 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:32.182 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:32.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.182 INFO:teuthology.orchestra.run.smithi043.stdout: "req_mksnap_latency": { 2024-09-19T05:15:32.182 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:32.183 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:32.183 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:32.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.183 INFO:teuthology.orchestra.run.smithi043.stdout: "req_open_latency": { 2024-09-19T05:15:32.183 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:32.183 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:32.184 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:32.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.184 INFO:teuthology.orchestra.run.smithi043.stdout: "req_readdir_latency": { 2024-09-19T05:15:32.184 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:32.184 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:32.184 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:32.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.185 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rename_latency": { 2024-09-19T05:15:32.185 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:32.185 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:32.185 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:32.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.185 INFO:teuthology.orchestra.run.smithi043.stdout: "req_renamesnap_latency": { 2024-09-19T05:15:32.186 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:32.186 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:32.186 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:32.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.186 INFO:teuthology.orchestra.run.smithi043.stdout: "req_snapdiff_latency": { 2024-09-19T05:15:32.186 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:32.186 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:32.186 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:32.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.187 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rmdir_latency": { 2024-09-19T05:15:32.187 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:32.187 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:32.187 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:32.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.187 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rmsnap_latency": { 2024-09-19T05:15:32.187 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:32.187 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:32.187 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:32.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.187 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rmxattr_latency": { 2024-09-19T05:15:32.187 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:32.188 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:32.188 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:32.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.188 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setattr_latency": { 2024-09-19T05:15:32.188 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:32.188 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:32.188 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:32.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.188 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setdirlayout_latency": { 2024-09-19T05:15:32.188 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:32.188 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:32.188 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:32.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.189 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setfilelock_latency": { 2024-09-19T05:15:32.189 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:32.189 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:32.189 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:32.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.189 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setlayout_latency": { 2024-09-19T05:15:32.189 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:32.189 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:32.189 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:32.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.189 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setxattr_latency": { 2024-09-19T05:15:32.189 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:32.189 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:32.189 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:32.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.190 INFO:teuthology.orchestra.run.smithi043.stdout: "req_symlink_latency": { 2024-09-19T05:15:32.190 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:32.190 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:32.190 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:32.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.190 INFO:teuthology.orchestra.run.smithi043.stdout: "req_unlink_latency": { 2024-09-19T05:15:32.190 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:32.190 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:32.190 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:32.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.190 INFO:teuthology.orchestra.run.smithi043.stdout: "cap_revoke_eviction": 0, 2024-09-19T05:15:32.190 INFO:teuthology.orchestra.run.smithi043.stdout: "cap_acquisition_throttle": 0, 2024-09-19T05:15:32.190 INFO:teuthology.orchestra.run.smithi043.stdout: "req_getvxattr_latency": { 2024-09-19T05:15:32.191 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:32.191 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:32.191 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:32.191 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:32.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.191 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_sessions": { 2024-09-19T05:15:32.191 INFO:teuthology.orchestra.run.smithi043.stdout: "session_count": 0, 2024-09-19T05:15:32.191 INFO:teuthology.orchestra.run.smithi043.stdout: "session_add": 2, 2024-09-19T05:15:32.191 INFO:teuthology.orchestra.run.smithi043.stdout: "session_remove": 2, 2024-09-19T05:15:32.191 INFO:teuthology.orchestra.run.smithi043.stdout: "sessions_open": 0, 2024-09-19T05:15:32.191 INFO:teuthology.orchestra.run.smithi043.stdout: "sessions_stale": 0, 2024-09-19T05:15:32.191 INFO:teuthology.orchestra.run.smithi043.stdout: "total_load": 3, 2024-09-19T05:15:32.191 INFO:teuthology.orchestra.run.smithi043.stdout: "average_load": 3, 2024-09-19T05:15:32.192 INFO:teuthology.orchestra.run.smithi043.stdout: "avg_session_uptime": 87, 2024-09-19T05:15:32.192 INFO:teuthology.orchestra.run.smithi043.stdout: "mdthresh_evicted": 0 2024-09-19T05:15:32.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.192 INFO:teuthology.orchestra.run.smithi043.stdout: "mempool": { 2024-09-19T05:15:32.192 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_bytes": 0, 2024-09-19T05:15:32.192 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_items": 0, 2024-09-19T05:15:32.192 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T05:15:32.192 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_items": 0, 2024-09-19T05:15:32.192 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T05:15:32.192 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_items": 0, 2024-09-19T05:15:32.192 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T05:15:32.192 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T05:15:32.193 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T05:15:32.193 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T05:15:32.193 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T05:15:32.193 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_items": 0, 2024-09-19T05:15:32.193 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T05:15:32.193 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T05:15:32.193 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_bytes": 0, 2024-09-19T05:15:32.193 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_items": 0, 2024-09-19T05:15:32.193 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_bytes": 0, 2024-09-19T05:15:32.193 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_items": 0, 2024-09-19T05:15:32.193 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T05:15:32.193 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T05:15:32.193 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T05:15:32.194 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T05:15:32.194 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T05:15:32.194 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_items": 0, 2024-09-19T05:15:32.194 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_bytes": 0, 2024-09-19T05:15:32.194 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_items": 0, 2024-09-19T05:15:32.194 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T05:15:32.194 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T05:15:32.194 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_bytes": 0, 2024-09-19T05:15:32.194 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_items": 0, 2024-09-19T05:15:32.194 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_bytes": 0, 2024-09-19T05:15:32.194 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_items": 0, 2024-09-19T05:15:32.194 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T05:15:32.194 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_items": 0, 2024-09-19T05:15:32.194 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T05:15:32.195 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_items": 0, 2024-09-19T05:15:32.195 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_bytes": 255479, 2024-09-19T05:15:32.195 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_items": 88, 2024-09-19T05:15:32.195 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_bytes": 88, 2024-09-19T05:15:32.195 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_items": 1, 2024-09-19T05:15:32.195 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_bytes": 0, 2024-09-19T05:15:32.195 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_items": 0, 2024-09-19T05:15:32.195 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_bytes": 0, 2024-09-19T05:15:32.195 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_items": 0, 2024-09-19T05:15:32.195 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_bytes": 0, 2024-09-19T05:15:32.195 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_items": 0, 2024-09-19T05:15:32.195 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_bytes": 8224, 2024-09-19T05:15:32.195 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_items": 181, 2024-09-19T05:15:32.195 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T05:15:32.196 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_items": 0, 2024-09-19T05:15:32.196 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_bytes": 0, 2024-09-19T05:15:32.196 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_items": 0, 2024-09-19T05:15:32.196 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_bytes": 59336, 2024-09-19T05:15:32.196 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_items": 234, 2024-09-19T05:15:32.196 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_bytes": 0, 2024-09-19T05:15:32.196 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_items": 0, 2024-09-19T05:15:32.196 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_bytes": 0, 2024-09-19T05:15:32.196 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_items": 0 2024-09-19T05:15:32.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.196 INFO:teuthology.orchestra.run.smithi043.stdout: "objecter": { 2024-09-19T05:15:32.197 INFO:teuthology.orchestra.run.smithi043.stdout: "op_active": 0, 2024-09-19T05:15:32.197 INFO:teuthology.orchestra.run.smithi043.stdout: "op_laggy": 0, 2024-09-19T05:15:32.197 INFO:teuthology.orchestra.run.smithi043.stdout: "op_send": 44, 2024-09-19T05:15:32.197 INFO:teuthology.orchestra.run.smithi043.stdout: "op_send_bytes": 12273, 2024-09-19T05:15:32.197 INFO:teuthology.orchestra.run.smithi043.stdout: "op_resend": 28, 2024-09-19T05:15:32.197 INFO:teuthology.orchestra.run.smithi043.stdout: "op_reply": 44, 2024-09-19T05:15:32.197 INFO:teuthology.orchestra.run.smithi043.stdout: "op_latency": { 2024-09-19T05:15:32.197 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 44, 2024-09-19T05:15:32.197 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.257996928, 2024-09-19T05:15:32.197 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.005863566 2024-09-19T05:15:32.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.198 INFO:teuthology.orchestra.run.smithi043.stdout: "op_inflight": 0, 2024-09-19T05:15:32.198 INFO:teuthology.orchestra.run.smithi043.stdout: "oplen_avg": { 2024-09-19T05:15:32.198 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 44, 2024-09-19T05:15:32.198 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 68 2024-09-19T05:15:32.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.198 INFO:teuthology.orchestra.run.smithi043.stdout: "op": 44, 2024-09-19T05:15:32.198 INFO:teuthology.orchestra.run.smithi043.stdout: "op_r": 0, 2024-09-19T05:15:32.198 INFO:teuthology.orchestra.run.smithi043.stdout: "op_w": 44, 2024-09-19T05:15:32.198 INFO:teuthology.orchestra.run.smithi043.stdout: "op_rmw": 0, 2024-09-19T05:15:32.198 INFO:teuthology.orchestra.run.smithi043.stdout: "op_pg": 0, 2024-09-19T05:15:32.198 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_stat": 0, 2024-09-19T05:15:32.198 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_create": 10, 2024-09-19T05:15:32.198 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_read": 0, 2024-09-19T05:15:32.199 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_write": 5, 2024-09-19T05:15:32.199 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_writefull": 8, 2024-09-19T05:15:32.199 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_writesame": 0, 2024-09-19T05:15:32.199 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_append": 0, 2024-09-19T05:15:32.199 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_zero": 0, 2024-09-19T05:15:32.199 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_truncate": 0, 2024-09-19T05:15:32.199 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_delete": 6, 2024-09-19T05:15:32.199 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_mapext": 0, 2024-09-19T05:15:32.199 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_sparse_read": 0, 2024-09-19T05:15:32.199 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_clonerange": 0, 2024-09-19T05:15:32.199 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_getxattr": 0, 2024-09-19T05:15:32.199 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_setxattr": 20, 2024-09-19T05:15:32.200 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_cmpxattr": 0, 2024-09-19T05:15:32.200 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_rmxattr": 0, 2024-09-19T05:15:32.200 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_resetxattrs": 0, 2024-09-19T05:15:32.200 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_call": 0, 2024-09-19T05:15:32.200 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_watch": 0, 2024-09-19T05:15:32.200 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_notify": 0, 2024-09-19T05:15:32.200 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T05:15:32.200 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_pgls": 0, 2024-09-19T05:15:32.200 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_pgls_filter": 0, 2024-09-19T05:15:32.200 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_other": 0, 2024-09-19T05:15:32.201 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_active": 0, 2024-09-19T05:15:32.201 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_send": 0, 2024-09-19T05:15:32.201 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_resend": 0, 2024-09-19T05:15:32.201 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_ping": 0, 2024-09-19T05:15:32.201 INFO:teuthology.orchestra.run.smithi043.stdout: "poolop_active": 0, 2024-09-19T05:15:32.201 INFO:teuthology.orchestra.run.smithi043.stdout: "poolop_send": 0, 2024-09-19T05:15:32.201 INFO:teuthology.orchestra.run.smithi043.stdout: "poolop_resend": 0, 2024-09-19T05:15:32.201 INFO:teuthology.orchestra.run.smithi043.stdout: "poolstat_active": 0, 2024-09-19T05:15:32.201 INFO:teuthology.orchestra.run.smithi043.stdout: "poolstat_send": 0, 2024-09-19T05:15:32.201 INFO:teuthology.orchestra.run.smithi043.stdout: "poolstat_resend": 0, 2024-09-19T05:15:32.202 INFO:teuthology.orchestra.run.smithi043.stdout: "statfs_active": 0, 2024-09-19T05:15:32.202 INFO:teuthology.orchestra.run.smithi043.stdout: "statfs_send": 0, 2024-09-19T05:15:32.202 INFO:teuthology.orchestra.run.smithi043.stdout: "statfs_resend": 0, 2024-09-19T05:15:32.202 INFO:teuthology.orchestra.run.smithi043.stdout: "command_active": 0, 2024-09-19T05:15:32.202 INFO:teuthology.orchestra.run.smithi043.stdout: "command_send": 0, 2024-09-19T05:15:32.202 INFO:teuthology.orchestra.run.smithi043.stdout: "command_resend": 0, 2024-09-19T05:15:32.202 INFO:teuthology.orchestra.run.smithi043.stdout: "map_epoch": 96, 2024-09-19T05:15:32.202 INFO:teuthology.orchestra.run.smithi043.stdout: "map_full": 0, 2024-09-19T05:15:32.202 INFO:teuthology.orchestra.run.smithi043.stdout: "map_inc": 13, 2024-09-19T05:15:32.202 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_sessions": 9, 2024-09-19T05:15:32.202 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_session_open": 9, 2024-09-19T05:15:32.202 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_session_close": 0, 2024-09-19T05:15:32.203 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_laggy": 0, 2024-09-19T05:15:32.203 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_wr": 17, 2024-09-19T05:15:32.203 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_rd": 0, 2024-09-19T05:15:32.203 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_del": 2 2024-09-19T05:15:32.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.203 INFO:teuthology.orchestra.run.smithi043.stdout: "oft": { 2024-09-19T05:15:32.203 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_objs": 1, 2024-09-19T05:15:32.203 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_kv_pairs": 0, 2024-09-19T05:15:32.203 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_updates": 0, 2024-09-19T05:15:32.203 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_removes": 0 2024-09-19T05:15:32.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.203 INFO:teuthology.orchestra.run.smithi043.stdout: "osd-slow-ops": { 2024-09-19T05:15:32.203 INFO:teuthology.orchestra.run.smithi043.stdout: "slow_ops_count": 0 2024-09-19T05:15:32.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.204 INFO:teuthology.orchestra.run.smithi043.stdout: "purge_queue": { 2024-09-19T05:15:32.204 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing_ops": 0, 2024-09-19T05:15:32.204 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T05:15:32.204 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing": 0, 2024-09-19T05:15:32.204 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing_high_water": 0, 2024-09-19T05:15:32.204 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executed_ops": 0, 2024-09-19T05:15:32.204 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executed": 0, 2024-09-19T05:15:32.204 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_item_in_journal": 0 2024-09-19T05:15:32.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.204 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T05:15:32.204 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-09-19T05:15:32.205 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 104857600, 2024-09-19T05:15:32.205 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-09-19T05:15:32.205 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 765, 2024-09-19T05:15:32.205 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 1695891, 2024-09-19T05:15:32.205 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:32.205 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 765, 2024-09-19T05:15:32.205 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-09-19T05:15:32.205 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-09-19T05:15:32.205 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 765, 2024-09-19T05:15:32.205 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 1695891, 2024-09-19T05:15:32.205 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-09-19T05:15:32.205 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:32.205 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:32.205 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:32.206 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:32.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.206 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-objecter_bytes": { 2024-09-19T05:15:32.206 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-09-19T05:15:32.206 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 104857600, 2024-09-19T05:15:32.206 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-09-19T05:15:32.206 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 0, 2024-09-19T05:15:32.206 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 0, 2024-09-19T05:15:32.206 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:32.206 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 0, 2024-09-19T05:15:32.206 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 44, 2024-09-19T05:15:32.206 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 12273, 2024-09-19T05:15:32.206 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 44, 2024-09-19T05:15:32.206 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 12273, 2024-09-19T05:15:32.207 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-09-19T05:15:32.207 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:32.207 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:32.207 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:32.207 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:32.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.207 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-objecter_ops": { 2024-09-19T05:15:32.207 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-09-19T05:15:32.207 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 1024, 2024-09-19T05:15:32.207 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-09-19T05:15:32.208 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 0, 2024-09-19T05:15:32.208 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 0, 2024-09-19T05:15:32.208 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:32.208 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 0, 2024-09-19T05:15:32.208 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 44, 2024-09-19T05:15:32.208 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 44, 2024-09-19T05:15:32.208 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 44, 2024-09-19T05:15:32.208 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 44, 2024-09-19T05:15:32.208 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-09-19T05:15:32.208 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:32.208 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:32.209 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:32.209 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:32.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.209 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-write_buf_throttle": { 2024-09-19T05:15:32.209 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-09-19T05:15:32.210 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 3758096384, 2024-09-19T05:15:32.210 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-09-19T05:15:32.210 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 0, 2024-09-19T05:15:32.210 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 0, 2024-09-19T05:15:32.210 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:32.210 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 0, 2024-09-19T05:15:32.210 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-09-19T05:15:32.210 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-09-19T05:15:32.210 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 0, 2024-09-19T05:15:32.210 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 0, 2024-09-19T05:15:32.211 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-09-19T05:15:32.211 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:32.211 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:32.211 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:32.211 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:32.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:32.211 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-write_buf_throttle-0x56243c202960": { 2024-09-19T05:15:32.211 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-09-19T05:15:32.211 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 3758096384, 2024-09-19T05:15:32.211 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-09-19T05:15:32.211 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 6, 2024-09-19T05:15:32.211 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 1656, 2024-09-19T05:15:32.211 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:32.211 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 6, 2024-09-19T05:15:32.212 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-09-19T05:15:32.212 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-09-19T05:15:32.212 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 5, 2024-09-19T05:15:32.212 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 1656, 2024-09-19T05:15:32.212 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-09-19T05:15:32.212 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:32.212 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:32.212 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:32.212 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:32.212 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:32.212 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:15:32.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:31 smithi096 ceph-mon[35085]: pgmap v339: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:15:32.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:31 smithi043 ceph-mon[30800]: pgmap v339: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:15:32.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:31 smithi151 ceph-mon[35425]: pgmap v339: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:15:33.095 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-19T05:15:33.095 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-19T05:15:33.095 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=6 2024-09-19T05:15:33.095 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-09-19T05:15:33.095 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T05:15:33.095 DEBUG:tasks.check_counter:Getting stats from j 2024-09-19T05:15:33.096 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-19T05:15:33.474 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:15:34.069 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-19T05:15:34.069 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T05:15:34.069 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 4, 2024-09-19T05:15:34.069 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 4, 2024-09-19T05:15:34.069 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 3024, 2024-09-19T05:15:34.069 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 448, 2024-09-19T05:15:34.069 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 4, 2024-09-19T05:15:34.069 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 0, 2024-09-19T05:15:34.070 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.006110349, 2024-09-19T05:15:34.070 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.000079716, 2024-09-19T05:15:34.070 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.003515519, 2024-09-19T05:15:34.070 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T05:15:34.070 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:15:34.070 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 3, 2024-09-19T05:15:34.070 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.006189513, 2024-09-19T05:15:34.070 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.002063171 2024-09-19T05:15:34.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:34.070 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:15:34.070 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:34.070 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:34.070 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:34.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:34.071 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-19T05:15:34.071 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T05:15:34.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:34.071 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T05:15:34.071 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 2, 2024-09-19T05:15:34.071 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 170, 2024-09-19T05:15:34.071 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 320, 2024-09-19T05:15:34.071 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 187152, 2024-09-19T05:15:34.071 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 5, 2024-09-19T05:15:34.071 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 3, 2024-09-19T05:15:34.071 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.021733216, 2024-09-19T05:15:34.071 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.015601820, 2024-09-19T05:15:34.071 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.002475575, 2024-09-19T05:15:34.071 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T05:15:34.072 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:15:34.072 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 167, 2024-09-19T05:15:34.072 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.019317164, 2024-09-19T05:15:34.072 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000115671 2024-09-19T05:15:34.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:34.072 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:15:34.072 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:34.072 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:34.072 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:34.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:34.072 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T05:15:34.072 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 187152 2024-09-19T05:15:34.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:34.073 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T05:15:34.073 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 234, 2024-09-19T05:15:34.073 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 223, 2024-09-19T05:15:34.073 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 1590032, 2024-09-19T05:15:34.073 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 137024, 2024-09-19T05:15:34.073 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 2, 2024-09-19T05:15:34.073 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 1, 2024-09-19T05:15:34.073 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.059387214, 2024-09-19T05:15:34.073 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.024008382, 2024-09-19T05:15:34.074 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.054269838, 2024-09-19T05:15:34.074 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T05:15:34.074 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:15:34.074 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 222, 2024-09-19T05:15:34.074 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.016638677, 2024-09-19T05:15:34.074 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000074948 2024-09-19T05:15:34.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:34.074 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:15:34.074 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:34.074 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:34.074 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:34.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:34.074 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 1590032, 2024-09-19T05:15:34.075 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 137024 2024-09-19T05:15:34.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:34.075 INFO:teuthology.orchestra.run.smithi043.stdout: "cct": { 2024-09-19T05:15:34.075 INFO:teuthology.orchestra.run.smithi043.stdout: "total_workers": 0, 2024-09-19T05:15:34.075 INFO:teuthology.orchestra.run.smithi043.stdout: "unhealthy_workers": 0 2024-09-19T05:15:34.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:34.075 INFO:teuthology.orchestra.run.smithi043.stdout: "mempool": { 2024-09-19T05:15:34.075 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_bytes": 0, 2024-09-19T05:15:34.075 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_items": 0, 2024-09-19T05:15:34.075 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T05:15:34.075 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_items": 0, 2024-09-19T05:15:34.075 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T05:15:34.075 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_items": 0, 2024-09-19T05:15:34.076 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T05:15:34.076 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T05:15:34.076 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T05:15:34.076 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T05:15:34.077 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T05:15:34.077 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_items": 0, 2024-09-19T05:15:34.077 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T05:15:34.077 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T05:15:34.077 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_bytes": 0, 2024-09-19T05:15:34.077 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_items": 0, 2024-09-19T05:15:34.077 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_bytes": 0, 2024-09-19T05:15:34.077 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_items": 0, 2024-09-19T05:15:34.077 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T05:15:34.077 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T05:15:34.077 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T05:15:34.077 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T05:15:34.077 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T05:15:34.077 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_items": 0, 2024-09-19T05:15:34.078 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_bytes": 0, 2024-09-19T05:15:34.078 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_items": 0, 2024-09-19T05:15:34.078 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T05:15:34.078 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T05:15:34.078 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_bytes": 0, 2024-09-19T05:15:34.078 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_items": 0, 2024-09-19T05:15:34.078 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_bytes": 0, 2024-09-19T05:15:34.078 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_items": 0, 2024-09-19T05:15:34.078 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T05:15:34.078 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_items": 0, 2024-09-19T05:15:34.078 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T05:15:34.078 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_items": 0, 2024-09-19T05:15:34.078 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_bytes": 148977, 2024-09-19T05:15:34.078 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_items": 30, 2024-09-19T05:15:34.079 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_bytes": 0, 2024-09-19T05:15:34.079 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_items": 0, 2024-09-19T05:15:34.079 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_bytes": 0, 2024-09-19T05:15:34.079 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_items": 0, 2024-09-19T05:15:34.079 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_bytes": 0, 2024-09-19T05:15:34.079 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_items": 0, 2024-09-19T05:15:34.079 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_bytes": 0, 2024-09-19T05:15:34.079 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_items": 0, 2024-09-19T05:15:34.079 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_bytes": 0, 2024-09-19T05:15:34.079 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_items": 0, 2024-09-19T05:15:34.079 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T05:15:34.079 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_items": 0, 2024-09-19T05:15:34.079 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_bytes": 0, 2024-09-19T05:15:34.079 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_items": 0, 2024-09-19T05:15:34.080 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_bytes": 928, 2024-09-19T05:15:34.080 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_items": 2, 2024-09-19T05:15:34.080 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_bytes": 0, 2024-09-19T05:15:34.080 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_items": 0, 2024-09-19T05:15:34.080 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_bytes": 0, 2024-09-19T05:15:34.080 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_items": 0 2024-09-19T05:15:34.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-19T05:15:34.080 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T05:15:34.080 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-09-19T05:15:34.080 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 104857600, 2024-09-19T05:15:34.080 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-09-19T05:15:34.080 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 236, 2024-09-19T05:15:34.080 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 1557022, 2024-09-19T05:15:34.081 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:34.081 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 236, 2024-09-19T05:15:34.081 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-09-19T05:15:34.081 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-09-19T05:15:34.081 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 236, 2024-09-19T05:15:34.081 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 1557022, 2024-09-19T05:15:34.081 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-09-19T05:15:34.081 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-19T05:15:34.081 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-19T05:15:34.081 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-19T05:15:34.081 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:34.081 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-19T05:15:34.081 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-19T05:15:34.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:33 smithi096 ceph-mon[35085]: pgmap v340: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:15:34.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:33 smithi043 ceph-mon[30800]: pgmap v340: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:15:34.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:33 smithi151 ceph-mon[35425]: pgmap v340: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:15:34.928 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-19T05:15:34.929 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-19T05:15:34.929 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-19T05:15:34.929 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-19T05:15:34.929 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-19T05:15:34.929 DEBUG:tasks.check_counter:Getting stats from b 2024-09-19T05:15:34.929 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-19T05:15:35.186 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T05:15:35.846 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-19T05:15:35.846 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T05:15:35.847 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 2945, 2024-09-19T05:15:35.847 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 2788, 2024-09-19T05:15:35.847 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 763492, 2024-09-19T05:15:35.847 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 3947921, 2024-09-19T05:15:35.847 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 140, 2024-09-19T05:15:35.847 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 6, 2024-09-19T05:15:35.847 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.265730768, 2024-09-19T05:15:35.847 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.073797178, 2024-09-19T05:15:35.847 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.118944888, 2024-09-19T05:15:35.847 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.058653384, 2024-09-19T05:15:35.848 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:15:35.848 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 2789, 2024-09-19T05:15:35.848 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.220054808, 2024-09-19T05:15:35.848 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000078900 2024-09-19T05:15:35.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.849 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:15:35.849 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1304, 2024-09-19T05:15:35.849 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000279724, 2024-09-19T05:15:35.849 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000214 2024-09-19T05:15:35.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.849 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 101712, 2024-09-19T05:15:35.849 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 361344 2024-09-19T05:15:35.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.849 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T05:15:35.850 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 3988, 2024-09-19T05:15:35.850 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 3261, 2024-09-19T05:15:35.850 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 1129053, 2024-09-19T05:15:35.850 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 1227206, 2024-09-19T05:15:35.850 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 15, 2024-09-19T05:15:35.850 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 7, 2024-09-19T05:15:35.850 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.303791837, 2024-09-19T05:15:35.850 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.083283963, 2024-09-19T05:15:35.850 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.117543810, 2024-09-19T05:15:35.850 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.100941656, 2024-09-19T05:15:35.850 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:15:35.850 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 3256, 2024-09-19T05:15:35.850 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.215455450, 2024-09-19T05:15:35.851 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000066171 2024-09-19T05:15:35.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.851 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:15:35.851 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1158, 2024-09-19T05:15:35.851 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000350082, 2024-09-19T05:15:35.851 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000302 2024-09-19T05:15:35.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.851 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 42720, 2024-09-19T05:15:35.851 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 136496 2024-09-19T05:15:35.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.851 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T05:15:35.851 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 3271, 2024-09-19T05:15:35.851 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 2248, 2024-09-19T05:15:35.852 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 2885189, 2024-09-19T05:15:35.852 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 871560, 2024-09-19T05:15:35.852 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 14, 2024-09-19T05:15:35.852 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 7, 2024-09-19T05:15:35.852 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.307275334, 2024-09-19T05:15:35.852 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.101476779, 2024-09-19T05:15:35.852 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.168675144, 2024-09-19T05:15:35.852 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.065608999, 2024-09-19T05:15:35.852 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:15:35.852 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 2247, 2024-09-19T05:15:35.852 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.200336357, 2024-09-19T05:15:35.853 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000089157 2024-09-19T05:15:35.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.853 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:15:35.853 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 2689, 2024-09-19T05:15:35.853 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000527943, 2024-09-19T05:15:35.853 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000196 2024-09-19T05:15:35.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.853 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 2043216, 2024-09-19T05:15:35.853 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 264208 2024-09-19T05:15:35.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.853 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-09-19T05:15:35.853 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 1, 2024-09-19T05:15:35.853 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-09-19T05:15:35.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.854 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-MDSRank": { 2024-09-19T05:15:35.854 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-09-19T05:15:35.854 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-09-19T05:15:35.854 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 737, 2024-09-19T05:15:35.854 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.150423064, 2024-09-19T05:15:35.854 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000204101 2024-09-19T05:15:35.854 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T05:15:35.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.854 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-PurgeQueue": { 2024-09-19T05:15:35.854 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-09-19T05:15:35.854 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-09-19T05:15:35.854 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 4445, 2024-09-19T05:15:35.854 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.116639422, 2024-09-19T05:15:35.855 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000026240 2024-09-19T05:15:35.855 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T05:15:35.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.855 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": { 2024-09-19T05:15:35.855 INFO:teuthology.orchestra.run.smithi096.stdout: "request": 176, 2024-09-19T05:15:35.855 INFO:teuthology.orchestra.run.smithi096.stdout: "reply": 176, 2024-09-19T05:15:35.855 INFO:teuthology.orchestra.run.smithi096.stdout: "reply_latency": { 2024-09-19T05:15:35.855 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 176, 2024-09-19T05:15:35.855 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.164336367, 2024-09-19T05:15:35.855 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000933729 2024-09-19T05:15:35.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.855 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_reply": 0, 2024-09-19T05:15:35.855 INFO:teuthology.orchestra.run.smithi096.stdout: "forward": 0, 2024-09-19T05:15:35.856 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_complete": 0, 2024-09-19T05:15:35.856 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_keys": 0, 2024-09-19T05:15:35.856 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_commit": 12, 2024-09-19T05:15:35.856 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_split": 0, 2024-09-19T05:15:35.856 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_merge": 0, 2024-09-19T05:15:35.856 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes": 36, 2024-09-19T05:15:35.856 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_top": 2, 2024-09-19T05:15:35.856 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_bottom": 34, 2024-09-19T05:15:35.856 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pin_tail": 0, 2024-09-19T05:15:35.856 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pinned": 33, 2024-09-19T05:15:35.856 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_expired": 0, 2024-09-19T05:15:35.856 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_with_caps": 3, 2024-09-19T05:15:35.856 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": 3, 2024-09-19T05:15:35.857 INFO:teuthology.orchestra.run.smithi096.stdout: "subtrees": 2, 2024-09-19T05:15:35.857 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse": 234, 2024-09-19T05:15:35.857 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_hit": 223, 2024-09-19T05:15:35.857 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_forward": 0, 2024-09-19T05:15:35.857 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_discover": 0, 2024-09-19T05:15:35.857 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_dir_fetch": 0, 2024-09-19T05:15:35.857 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_remote_ino": 0, 2024-09-19T05:15:35.857 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_lock": 0, 2024-09-19T05:15:35.857 INFO:teuthology.orchestra.run.smithi096.stdout: "load_cent": 0, 2024-09-19T05:15:35.857 INFO:teuthology.orchestra.run.smithi096.stdout: "q": 0, 2024-09-19T05:15:35.857 INFO:teuthology.orchestra.run.smithi096.stdout: "exported": 0, 2024-09-19T05:15:35.857 INFO:teuthology.orchestra.run.smithi096.stdout: "exported_inodes": 0, 2024-09-19T05:15:35.857 INFO:teuthology.orchestra.run.smithi096.stdout: "imported": 0, 2024-09-19T05:15:35.858 INFO:teuthology.orchestra.run.smithi096.stdout: "imported_inodes": 0, 2024-09-19T05:15:35.858 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_dir_fetch": 0, 2024-09-19T05:15:35.858 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_backtrace_fetch": 210, 2024-09-19T05:15:35.858 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_peer_discover": 0, 2024-09-19T05:15:35.858 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rfiles": 4, 2024-09-19T05:15:35.858 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rbytes": 234, 2024-09-19T05:15:35.858 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rsnaps": 6, 2024-09-19T05:15:35.858 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_fetch": 210, 2024-09-19T05:15:35.858 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_set_tag": 0, 2024-09-19T05:15:35.858 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T05:15:35.858 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_inotable_repaired": 0, 2024-09-19T05:15:35.858 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_inodes": 135, 2024-09-19T05:15:35.858 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_base_inodes": 18, 2024-09-19T05:15:35.859 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dirfrag_rstats": 117, 2024-09-19T05:15:35.859 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_file_inodes": 75, 2024-09-19T05:15:35.859 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_inode_file_caps": 0, 2024-09-19T05:15:35.859 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_revoke": 13, 2024-09-19T05:15:35.859 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_grant": 109, 2024-09-19T05:15:35.859 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T05:15:35.859 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T05:15:35.859 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T05:15:35.859 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps": 153, 2024-09-19T05:15:35.859 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps_dirty": 135, 2024-09-19T05:15:35.859 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_cap_release": 4, 2024-09-19T05:15:35.859 INFO:teuthology.orchestra.run.smithi096.stdout: "process_request_cap_release": 24 2024-09-19T05:15:35.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.860 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache": { 2024-09-19T05:15:35.860 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update": 0, 2024-09-19T05:15:35.860 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update_receipt": 0, 2024-09-19T05:15:35.860 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_try_discover": 0, 2024-09-19T05:15:35.860 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_send_discover": 0, 2024-09-19T05:15:35.860 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_handle_discover": 6, 2024-09-19T05:15:35.860 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays": 2, 2024-09-19T05:15:35.860 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_delayed": 0, 2024-09-19T05:15:35.860 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_enqueuing": 0, 2024-09-19T05:15:35.860 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_created": 5, 2024-09-19T05:15:35.860 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_enqueued": 3, 2024-09-19T05:15:35.860 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_reintegrated": 0, 2024-09-19T05:15:35.860 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_migrated": 0, 2024-09-19T05:15:35.861 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_processing": 0, 2024-09-19T05:15:35.861 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_enqueued": 0, 2024-09-19T05:15:35.861 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_prioritized": 0, 2024-09-19T05:15:35.861 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_started": 0, 2024-09-19T05:15:35.861 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_completed": 0, 2024-09-19T05:15:35.861 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_path": 0, 2024-09-19T05:15:35.861 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_inode": 0, 2024-09-19T05:15:35.861 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_enqueue_scrub": 18, 2024-09-19T05:15:35.861 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_exportdir": 0, 2024-09-19T05:15:35.861 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_flush": 0, 2024-09-19T05:15:35.861 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragmentdir": 0, 2024-09-19T05:15:35.862 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragstats": 0, 2024-09-19T05:15:35.862 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_inodestats": 0 2024-09-19T05:15:35.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.862 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log": { 2024-09-19T05:15:35.862 INFO:teuthology.orchestra.run.smithi096.stdout: "evlrg": 0, 2024-09-19T05:15:35.862 INFO:teuthology.orchestra.run.smithi096.stdout: "evadd": 288, 2024-09-19T05:15:35.862 INFO:teuthology.orchestra.run.smithi096.stdout: "evex": 0, 2024-09-19T05:15:35.862 INFO:teuthology.orchestra.run.smithi096.stdout: "evtrm": 0, 2024-09-19T05:15:35.862 INFO:teuthology.orchestra.run.smithi096.stdout: "ev": 288, 2024-09-19T05:15:35.862 INFO:teuthology.orchestra.run.smithi096.stdout: "evexg": 0, 2024-09-19T05:15:35.862 INFO:teuthology.orchestra.run.smithi096.stdout: "evexd": 0, 2024-09-19T05:15:35.862 INFO:teuthology.orchestra.run.smithi096.stdout: "segadd": 2, 2024-09-19T05:15:35.862 INFO:teuthology.orchestra.run.smithi096.stdout: "segex": 0, 2024-09-19T05:15:35.862 INFO:teuthology.orchestra.run.smithi096.stdout: "segtrm": 0, 2024-09-19T05:15:35.863 INFO:teuthology.orchestra.run.smithi096.stdout: "seg": 2, 2024-09-19T05:15:35.863 INFO:teuthology.orchestra.run.smithi096.stdout: "segmjr": 2, 2024-09-19T05:15:35.863 INFO:teuthology.orchestra.run.smithi096.stdout: "segexg": 0, 2024-09-19T05:15:35.863 INFO:teuthology.orchestra.run.smithi096.stdout: "segexd": 0, 2024-09-19T05:15:35.863 INFO:teuthology.orchestra.run.smithi096.stdout: "expos": 4194304, 2024-09-19T05:15:35.863 INFO:teuthology.orchestra.run.smithi096.stdout: "wrpos": 5433086, 2024-09-19T05:15:35.863 INFO:teuthology.orchestra.run.smithi096.stdout: "rdpos": 0, 2024-09-19T05:15:35.863 INFO:teuthology.orchestra.run.smithi096.stdout: "jlat": { 2024-09-19T05:15:35.863 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 256, 2024-09-19T05:15:35.863 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.484419185, 2024-09-19T05:15:35.863 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.001892262 2024-09-19T05:15:35.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.863 INFO:teuthology.orchestra.run.smithi096.stdout: "replayed": 0 2024-09-19T05:15:35.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.864 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_mem": { 2024-09-19T05:15:35.864 INFO:teuthology.orchestra.run.smithi096.stdout: "ino": 27, 2024-09-19T05:15:35.864 INFO:teuthology.orchestra.run.smithi096.stdout: "ino+": 48, 2024-09-19T05:15:35.864 INFO:teuthology.orchestra.run.smithi096.stdout: "ino-": 21, 2024-09-19T05:15:35.864 INFO:teuthology.orchestra.run.smithi096.stdout: "dir": 20, 2024-09-19T05:15:35.864 INFO:teuthology.orchestra.run.smithi096.stdout: "dir+": 20, 2024-09-19T05:15:35.864 INFO:teuthology.orchestra.run.smithi096.stdout: "dir-": 0, 2024-09-19T05:15:35.864 INFO:teuthology.orchestra.run.smithi096.stdout: "dn": 36, 2024-09-19T05:15:35.864 INFO:teuthology.orchestra.run.smithi096.stdout: "dn+": 39, 2024-09-19T05:15:35.864 INFO:teuthology.orchestra.run.smithi096.stdout: "dn-": 3, 2024-09-19T05:15:35.864 INFO:teuthology.orchestra.run.smithi096.stdout: "cap": 3, 2024-09-19T05:15:35.864 INFO:teuthology.orchestra.run.smithi096.stdout: "cap+": 24, 2024-09-19T05:15:35.865 INFO:teuthology.orchestra.run.smithi096.stdout: "cap-": 21, 2024-09-19T05:15:35.865 INFO:teuthology.orchestra.run.smithi096.stdout: "rss": 58316, 2024-09-19T05:15:35.865 INFO:teuthology.orchestra.run.smithi096.stdout: "heap": 231712 2024-09-19T05:15:35.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.865 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_server": { 2024-09-19T05:15:35.865 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_client_request": 226, 2024-09-19T05:15:35.865 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_server_request": 0, 2024-09-19T05:15:35.865 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_request": 176, 2024-09-19T05:15:35.865 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_session": 39, 2024-09-19T05:15:35.865 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_peer_request": 0, 2024-09-19T05:15:35.865 INFO:teuthology.orchestra.run.smithi096.stdout: "req_create_latency": { 2024-09-19T05:15:35.865 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 7, 2024-09-19T05:15:35.865 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.006976063, 2024-09-19T05:15:35.866 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000996580 2024-09-19T05:15:35.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.866 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getattr_latency": { 2024-09-19T05:15:35.866 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 110, 2024-09-19T05:15:35.866 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.019969976, 2024-09-19T05:15:35.866 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000181545 2024-09-19T05:15:35.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.866 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getfilelock_latency": { 2024-09-19T05:15:35.866 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:35.866 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:35.866 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:35.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.866 INFO:teuthology.orchestra.run.smithi096.stdout: "req_link_latency": { 2024-09-19T05:15:35.866 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:35.867 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:35.867 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:35.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.867 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookup_latency": { 2024-09-19T05:15:35.867 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 4, 2024-09-19T05:15:35.867 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000743356, 2024-09-19T05:15:35.867 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000185839 2024-09-19T05:15:35.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.868 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookuphash_latency": { 2024-09-19T05:15:35.868 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:35.868 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:35.868 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:35.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.868 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupino_latency": { 2024-09-19T05:15:35.868 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1, 2024-09-19T05:15:35.868 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000121679, 2024-09-19T05:15:35.868 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000121679 2024-09-19T05:15:35.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.868 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupname_latency": { 2024-09-19T05:15:35.869 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:35.869 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:35.869 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:35.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.869 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupparent_latency": { 2024-09-19T05:15:35.869 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:35.869 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:35.870 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:35.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.870 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupsnap_latency": { 2024-09-19T05:15:35.870 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:35.870 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:35.870 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:35.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.871 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lssnap_latency": { 2024-09-19T05:15:35.871 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 11, 2024-09-19T05:15:35.871 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.001527152, 2024-09-19T05:15:35.871 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000138832 2024-09-19T05:15:35.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.872 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mkdir_latency": { 2024-09-19T05:15:35.872 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 8, 2024-09-19T05:15:35.872 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.012692217, 2024-09-19T05:15:35.872 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.001586527 2024-09-19T05:15:35.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.872 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mknod_latency": { 2024-09-19T05:15:35.872 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:35.872 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:35.872 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:35.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.872 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mksnap_latency": { 2024-09-19T05:15:35.872 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 11, 2024-09-19T05:15:35.873 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.056041433, 2024-09-19T05:15:35.873 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.005094675 2024-09-19T05:15:35.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.873 INFO:teuthology.orchestra.run.smithi096.stdout: "req_open_latency": { 2024-09-19T05:15:35.873 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1, 2024-09-19T05:15:35.873 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000229100, 2024-09-19T05:15:35.873 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000229100 2024-09-19T05:15:35.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.873 INFO:teuthology.orchestra.run.smithi096.stdout: "req_readdir_latency": { 2024-09-19T05:15:35.873 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 2, 2024-09-19T05:15:35.873 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.005726013, 2024-09-19T05:15:35.873 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.002863006 2024-09-19T05:15:35.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.873 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rename_latency": { 2024-09-19T05:15:35.874 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 2, 2024-09-19T05:15:35.874 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.002630589, 2024-09-19T05:15:35.874 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.001315294 2024-09-19T05:15:35.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.874 INFO:teuthology.orchestra.run.smithi096.stdout: "req_renamesnap_latency": { 2024-09-19T05:15:35.874 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:35.874 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:35.874 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:35.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.874 INFO:teuthology.orchestra.run.smithi096.stdout: "req_snapdiff_latency": { 2024-09-19T05:15:35.874 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:35.874 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:35.874 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:35.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.875 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmdir_latency": { 2024-09-19T05:15:35.875 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 2, 2024-09-19T05:15:35.875 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.008038650, 2024-09-19T05:15:35.875 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.004019325 2024-09-19T05:15:35.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.875 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmsnap_latency": { 2024-09-19T05:15:35.875 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 5, 2024-09-19T05:15:35.875 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.021172781, 2024-09-19T05:15:35.875 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.004234556 2024-09-19T05:15:35.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.875 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmxattr_latency": { 2024-09-19T05:15:35.875 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:35.875 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:35.875 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:35.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.876 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setattr_latency": { 2024-09-19T05:15:35.876 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1, 2024-09-19T05:15:35.876 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000912379, 2024-09-19T05:15:35.876 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000912379 2024-09-19T05:15:35.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.876 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setdirlayout_latency": { 2024-09-19T05:15:35.876 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:35.876 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:35.876 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:35.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.876 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setfilelock_latency": { 2024-09-19T05:15:35.876 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:35.876 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:35.877 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:35.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.877 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setlayout_latency": { 2024-09-19T05:15:35.877 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:35.877 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:35.877 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:35.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.877 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setxattr_latency": { 2024-09-19T05:15:35.877 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 8, 2024-09-19T05:15:35.877 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.020574227, 2024-09-19T05:15:35.877 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.002571778 2024-09-19T05:15:35.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.877 INFO:teuthology.orchestra.run.smithi096.stdout: "req_symlink_latency": { 2024-09-19T05:15:35.878 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:35.878 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:35.878 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:35.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.878 INFO:teuthology.orchestra.run.smithi096.stdout: "req_unlink_latency": { 2024-09-19T05:15:35.878 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 3, 2024-09-19T05:15:35.878 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.006980752, 2024-09-19T05:15:35.878 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.002326917 2024-09-19T05:15:35.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.878 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_revoke_eviction": 0, 2024-09-19T05:15:35.878 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_acquisition_throttle": 0, 2024-09-19T05:15:35.878 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getvxattr_latency": { 2024-09-19T05:15:35.878 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:35.879 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:35.879 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:35.879 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T05:15:35.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.879 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_sessions": { 2024-09-19T05:15:35.879 INFO:teuthology.orchestra.run.smithi096.stdout: "session_count": 2, 2024-09-19T05:15:35.879 INFO:teuthology.orchestra.run.smithi096.stdout: "session_add": 6, 2024-09-19T05:15:35.879 INFO:teuthology.orchestra.run.smithi096.stdout: "session_remove": 4, 2024-09-19T05:15:35.879 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_open": 2, 2024-09-19T05:15:35.879 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_stale": 0, 2024-09-19T05:15:35.879 INFO:teuthology.orchestra.run.smithi096.stdout: "total_load": 27, 2024-09-19T05:15:35.879 INFO:teuthology.orchestra.run.smithi096.stdout: "average_load": 13, 2024-09-19T05:15:35.879 INFO:teuthology.orchestra.run.smithi096.stdout: "avg_session_uptime": 643, 2024-09-19T05:15:35.879 INFO:teuthology.orchestra.run.smithi096.stdout: "mdthresh_evicted": 0 2024-09-19T05:15:35.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.880 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-09-19T05:15:35.880 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 0, 2024-09-19T05:15:35.880 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 0, 2024-09-19T05:15:35.880 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T05:15:35.880 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-09-19T05:15:35.880 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T05:15:35.880 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-09-19T05:15:35.880 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T05:15:35.880 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T05:15:35.880 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T05:15:35.880 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T05:15:35.880 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T05:15:35.881 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-09-19T05:15:35.881 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T05:15:35.881 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T05:15:35.881 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-09-19T05:15:35.881 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-09-19T05:15:35.881 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-09-19T05:15:35.881 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-09-19T05:15:35.881 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T05:15:35.881 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T05:15:35.881 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T05:15:35.881 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T05:15:35.881 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T05:15:35.881 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-09-19T05:15:35.882 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-09-19T05:15:35.882 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-09-19T05:15:35.882 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T05:15:35.882 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T05:15:35.882 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-09-19T05:15:35.882 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-09-19T05:15:35.882 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-09-19T05:15:35.882 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-09-19T05:15:35.882 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T05:15:35.882 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-09-19T05:15:35.882 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T05:15:35.882 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-09-19T05:15:35.882 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 363426, 2024-09-19T05:15:35.882 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 152, 2024-09-19T05:15:35.883 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 792, 2024-09-19T05:15:35.883 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 9, 2024-09-19T05:15:35.883 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-09-19T05:15:35.883 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-09-19T05:15:35.883 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-09-19T05:15:35.883 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-09-19T05:15:35.883 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-09-19T05:15:35.883 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-09-19T05:15:35.883 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 8224, 2024-09-19T05:15:35.883 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 181, 2024-09-19T05:15:35.883 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T05:15:35.883 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-09-19T05:15:35.883 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-09-19T05:15:35.884 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-09-19T05:15:35.884 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 136565, 2024-09-19T05:15:35.884 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 1063, 2024-09-19T05:15:35.884 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-09-19T05:15:35.884 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-09-19T05:15:35.884 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-09-19T05:15:35.884 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-09-19T05:15:35.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.884 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter": { 2024-09-19T05:15:35.884 INFO:teuthology.orchestra.run.smithi096.stdout: "op_active": 0, 2024-09-19T05:15:35.884 INFO:teuthology.orchestra.run.smithi096.stdout: "op_laggy": 0, 2024-09-19T05:15:35.884 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send": 5714, 2024-09-19T05:15:35.884 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send_bytes": 1258662, 2024-09-19T05:15:35.884 INFO:teuthology.orchestra.run.smithi096.stdout: "op_resend": 37, 2024-09-19T05:15:35.885 INFO:teuthology.orchestra.run.smithi096.stdout: "op_reply": 5714, 2024-09-19T05:15:35.885 INFO:teuthology.orchestra.run.smithi096.stdout: "op_latency": { 2024-09-19T05:15:35.885 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 5714, 2024-09-19T05:15:35.885 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 60.277584671, 2024-09-19T05:15:35.885 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.010549104 2024-09-19T05:15:35.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.885 INFO:teuthology.orchestra.run.smithi096.stdout: "op_inflight": 0, 2024-09-19T05:15:35.885 INFO:teuthology.orchestra.run.smithi096.stdout: "oplen_avg": { 2024-09-19T05:15:35.885 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 5714, 2024-09-19T05:15:35.885 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 5767 2024-09-19T05:15:35.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.886 INFO:teuthology.orchestra.run.smithi096.stdout: "op": 5714, 2024-09-19T05:15:35.886 INFO:teuthology.orchestra.run.smithi096.stdout: "op_r": 249, 2024-09-19T05:15:35.886 INFO:teuthology.orchestra.run.smithi096.stdout: "op_w": 5465, 2024-09-19T05:15:35.887 INFO:teuthology.orchestra.run.smithi096.stdout: "op_rmw": 0, 2024-09-19T05:15:35.887 INFO:teuthology.orchestra.run.smithi096.stdout: "op_pg": 0, 2024-09-19T05:15:35.887 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_stat": 0, 2024-09-19T05:15:35.887 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_create": 11, 2024-09-19T05:15:35.887 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_read": 21, 2024-09-19T05:15:35.887 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_write": 259, 2024-09-19T05:15:35.888 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writefull": 32, 2024-09-19T05:15:35.888 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writesame": 0, 2024-09-19T05:15:35.888 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_append": 0, 2024-09-19T05:15:35.888 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_zero": 0, 2024-09-19T05:15:35.888 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_truncate": 0, 2024-09-19T05:15:35.888 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_delete": 5122, 2024-09-19T05:15:35.888 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_mapext": 0, 2024-09-19T05:15:35.889 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_sparse_read": 0, 2024-09-19T05:15:35.889 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_clonerange": 0, 2024-09-19T05:15:35.889 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_getxattr": 228, 2024-09-19T05:15:35.889 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_setxattr": 22, 2024-09-19T05:15:35.889 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_cmpxattr": 0, 2024-09-19T05:15:35.889 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_rmxattr": 0, 2024-09-19T05:15:35.890 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_resetxattrs": 0, 2024-09-19T05:15:35.890 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_call": 0, 2024-09-19T05:15:35.890 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_watch": 0, 2024-09-19T05:15:35.890 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_notify": 0, 2024-09-19T05:15:35.890 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T05:15:35.890 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls": 0, 2024-09-19T05:15:35.891 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls_filter": 0, 2024-09-19T05:15:35.891 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_other": 0, 2024-09-19T05:15:35.891 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_active": 0, 2024-09-19T05:15:35.891 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_send": 0, 2024-09-19T05:15:35.891 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_resend": 0, 2024-09-19T05:15:35.891 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_ping": 0, 2024-09-19T05:15:35.892 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_active": 0, 2024-09-19T05:15:35.892 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_send": 0, 2024-09-19T05:15:35.892 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_resend": 0, 2024-09-19T05:15:35.892 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_active": 0, 2024-09-19T05:15:35.892 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_send": 0, 2024-09-19T05:15:35.893 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_resend": 0, 2024-09-19T05:15:35.893 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_active": 0, 2024-09-19T05:15:35.893 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_send": 0, 2024-09-19T05:15:35.893 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_resend": 0, 2024-09-19T05:15:35.893 INFO:teuthology.orchestra.run.smithi096.stdout: "command_active": 0, 2024-09-19T05:15:35.893 INFO:teuthology.orchestra.run.smithi096.stdout: "command_send": 0, 2024-09-19T05:15:35.894 INFO:teuthology.orchestra.run.smithi096.stdout: "command_resend": 0, 2024-09-19T05:15:35.894 INFO:teuthology.orchestra.run.smithi096.stdout: "map_epoch": 96, 2024-09-19T05:15:35.894 INFO:teuthology.orchestra.run.smithi096.stdout: "map_full": 0, 2024-09-19T05:15:35.894 INFO:teuthology.orchestra.run.smithi096.stdout: "map_inc": 13, 2024-09-19T05:15:35.894 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_sessions": 12, 2024-09-19T05:15:35.894 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_open": 12, 2024-09-19T05:15:35.895 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_close": 0, 2024-09-19T05:15:35.895 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_laggy": 0, 2024-09-19T05:15:35.895 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_wr": 62, 2024-09-19T05:15:35.895 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_rd": 0, 2024-09-19T05:15:35.895 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_del": 10 2024-09-19T05:15:35.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.896 INFO:teuthology.orchestra.run.smithi096.stdout: "oft": { 2024-09-19T05:15:35.896 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_objs": 1, 2024-09-19T05:15:35.896 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_kv_pairs": 0, 2024-09-19T05:15:35.896 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_updates": 0, 2024-09-19T05:15:35.896 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_removes": 0 2024-09-19T05:15:35.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.897 INFO:teuthology.orchestra.run.smithi096.stdout: "osd-slow-ops": { 2024-09-19T05:15:35.897 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_ops_count": 0 2024-09-19T05:15:35.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.897 INFO:teuthology.orchestra.run.smithi096.stdout: "purge_queue": { 2024-09-19T05:15:35.897 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops": 0, 2024-09-19T05:15:35.898 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops_high_water": 3584, 2024-09-19T05:15:35.898 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing": 0, 2024-09-19T05:15:35.898 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_high_water": 1, 2024-09-19T05:15:35.898 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed_ops": 4610, 2024-09-19T05:15:35.898 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed": 3, 2024-09-19T05:15:35.898 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_item_in_journal": 0 2024-09-19T05:15:35.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.899 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T05:15:35.899 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-19T05:15:35.899 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-09-19T05:15:35.899 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-19T05:15:35.899 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 10200, 2024-09-19T05:15:35.899 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 3799461, 2024-09-19T05:15:35.900 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:35.900 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 10200, 2024-09-19T05:15:35.900 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-19T05:15:35.900 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-19T05:15:35.900 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 10200, 2024-09-19T05:15:35.901 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 3799461, 2024-09-19T05:15:35.901 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-19T05:15:35.901 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:35.901 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:35.901 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:35.901 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T05:15:35.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.902 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_bytes": { 2024-09-19T05:15:35.902 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-19T05:15:35.902 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-09-19T05:15:35.902 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-19T05:15:35.902 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-19T05:15:35.903 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-19T05:15:35.903 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:35.903 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-19T05:15:35.903 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 5714, 2024-09-19T05:15:35.903 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 1258981, 2024-09-19T05:15:35.904 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 5714, 2024-09-19T05:15:35.904 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 1258981, 2024-09-19T05:15:35.904 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-19T05:15:35.904 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:35.904 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:35.904 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:35.905 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T05:15:35.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.905 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_ops": { 2024-09-19T05:15:35.905 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-19T05:15:35.905 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 1024, 2024-09-19T05:15:35.906 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-19T05:15:35.906 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-19T05:15:35.906 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-19T05:15:35.906 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:35.906 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-19T05:15:35.906 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 5714, 2024-09-19T05:15:35.907 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 5714, 2024-09-19T05:15:35.907 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 5714, 2024-09-19T05:15:35.907 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 5714, 2024-09-19T05:15:35.907 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-19T05:15:35.907 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:35.907 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:35.908 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:35.908 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T05:15:35.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.908 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle": { 2024-09-19T05:15:35.908 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-19T05:15:35.908 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-09-19T05:15:35.909 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-19T05:15:35.909 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 3, 2024-09-19T05:15:35.909 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 319, 2024-09-19T05:15:35.909 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:35.909 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 3, 2024-09-19T05:15:35.910 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-19T05:15:35.910 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-19T05:15:35.910 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 3, 2024-09-19T05:15:35.910 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 319, 2024-09-19T05:15:35.910 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-19T05:15:35.910 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:35.911 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:35.911 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:35.911 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T05:15:35.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:35.911 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle-0x5587b8d6a960": { 2024-09-19T05:15:35.911 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-19T05:15:35.912 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-09-19T05:15:35.912 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-19T05:15:35.912 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 288, 2024-09-19T05:15:35.912 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 1238782, 2024-09-19T05:15:35.912 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:35.913 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 288, 2024-09-19T05:15:35.913 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-19T05:15:35.913 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-19T05:15:35.913 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 256, 2024-09-19T05:15:35.913 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 1238782, 2024-09-19T05:15:35.913 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-19T05:15:35.914 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:35.914 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:35.914 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:35.914 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T05:15:35.914 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T05:15:35.914 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-19T05:15:36.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:35 smithi096 ceph-mon[35085]: pgmap v341: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:15:36.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:35 smithi043 ceph-mon[30800]: pgmap v341: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:15:36.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:35 smithi151 ceph-mon[35425]: pgmap v341: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:15:36.804 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-09-19T05:15:36.804 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-09-19T05:15:36.805 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-09-19T05:15:36.805 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2024-09-19T05:15:36.805 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=5 2024-09-19T05:15:36.805 DEBUG:tasks.check_counter:Getting stats from e 2024-09-19T05:15:36.805 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-19T05:15:37.052 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T05:15:37.591 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-19T05:15:37.591 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T05:15:37.591 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 2250, 2024-09-19T05:15:37.591 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 2252, 2024-09-19T05:15:37.591 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 638141, 2024-09-19T05:15:37.591 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 715507, 2024-09-19T05:15:37.591 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 6, 2024-09-19T05:15:37.591 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 2, 2024-09-19T05:15:37.591 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.168570240, 2024-09-19T05:15:37.591 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.055574294, 2024-09-19T05:15:37.592 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.042532217, 2024-09-19T05:15:37.592 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.056731609, 2024-09-19T05:15:37.592 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:15:37.592 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 2250, 2024-09-19T05:15:37.592 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.068737616, 2024-09-19T05:15:37.592 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000030550 2024-09-19T05:15:37.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.592 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:15:37.592 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:37.592 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:37.592 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:37.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.593 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T05:15:37.593 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T05:15:37.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.593 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T05:15:37.593 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 255, 2024-09-19T05:15:37.593 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 242, 2024-09-19T05:15:37.593 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 1627612, 2024-09-19T05:15:37.593 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 140202, 2024-09-19T05:15:37.593 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 3, 2024-09-19T05:15:37.593 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 2, 2024-09-19T05:15:37.593 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.045954695, 2024-09-19T05:15:37.593 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.016816269, 2024-09-19T05:15:37.594 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.042703665, 2024-09-19T05:15:37.594 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000067972, 2024-09-19T05:15:37.594 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:15:37.594 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 241, 2024-09-19T05:15:37.594 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.014921812, 2024-09-19T05:15:37.594 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000061916 2024-09-19T05:15:37.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.594 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:15:37.594 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:37.594 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:37.594 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:37.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.594 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 1627056, 2024-09-19T05:15:37.595 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 139504 2024-09-19T05:15:37.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.595 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T05:15:37.595 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 7, 2024-09-19T05:15:37.595 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 172, 2024-09-19T05:15:37.595 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 3612, 2024-09-19T05:15:37.595 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 455166, 2024-09-19T05:15:37.595 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 6, 2024-09-19T05:15:37.595 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 4, 2024-09-19T05:15:37.595 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.012024108, 2024-09-19T05:15:37.595 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.006628881, 2024-09-19T05:15:37.595 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.003454201, 2024-09-19T05:15:37.595 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000019855, 2024-09-19T05:15:37.596 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:15:37.596 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 170, 2024-09-19T05:15:37.596 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.023360584, 2024-09-19T05:15:37.596 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000137415 2024-09-19T05:15:37.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.596 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:15:37.596 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:37.596 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:37.596 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:37.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.596 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-09-19T05:15:37.596 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 454848 2024-09-19T05:15:37.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.597 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-09-19T05:15:37.597 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 1, 2024-09-19T05:15:37.597 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-09-19T05:15:37.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.597 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-MDSRank": { 2024-09-19T05:15:37.597 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-09-19T05:15:37.597 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-09-19T05:15:37.597 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 3792, 2024-09-19T05:15:37.597 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.086768252, 2024-09-19T05:15:37.597 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000022881 2024-09-19T05:15:37.597 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T05:15:37.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.597 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-PurgeQueue": { 2024-09-19T05:15:37.598 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-09-19T05:15:37.598 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-09-19T05:15:37.598 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:37.598 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:37.598 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:37.598 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T05:15:37.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.598 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": { 2024-09-19T05:15:37.598 INFO:teuthology.orchestra.run.smithi096.stdout: "request": 0, 2024-09-19T05:15:37.598 INFO:teuthology.orchestra.run.smithi096.stdout: "reply": 0, 2024-09-19T05:15:37.598 INFO:teuthology.orchestra.run.smithi096.stdout: "reply_latency": { 2024-09-19T05:15:37.598 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:37.598 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:37.599 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:37.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.599 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_reply": 0, 2024-09-19T05:15:37.599 INFO:teuthology.orchestra.run.smithi096.stdout: "forward": 0, 2024-09-19T05:15:37.599 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_complete": 0, 2024-09-19T05:15:37.599 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_keys": 0, 2024-09-19T05:15:37.599 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_commit": 0, 2024-09-19T05:15:37.599 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_split": 0, 2024-09-19T05:15:37.599 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_merge": 0, 2024-09-19T05:15:37.599 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes": 0, 2024-09-19T05:15:37.599 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_top": 0, 2024-09-19T05:15:37.599 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_bottom": 0, 2024-09-19T05:15:37.599 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pin_tail": 0, 2024-09-19T05:15:37.600 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pinned": 0, 2024-09-19T05:15:37.600 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_expired": 0, 2024-09-19T05:15:37.600 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_with_caps": 0, 2024-09-19T05:15:37.600 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": 0, 2024-09-19T05:15:37.600 INFO:teuthology.orchestra.run.smithi096.stdout: "subtrees": 1, 2024-09-19T05:15:37.600 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse": 0, 2024-09-19T05:15:37.600 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_hit": 0, 2024-09-19T05:15:37.600 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_forward": 0, 2024-09-19T05:15:37.600 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_discover": 0, 2024-09-19T05:15:37.600 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_dir_fetch": 0, 2024-09-19T05:15:37.600 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_remote_ino": 0, 2024-09-19T05:15:37.600 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_lock": 0, 2024-09-19T05:15:37.600 INFO:teuthology.orchestra.run.smithi096.stdout: "load_cent": 0, 2024-09-19T05:15:37.601 INFO:teuthology.orchestra.run.smithi096.stdout: "q": 0, 2024-09-19T05:15:37.601 INFO:teuthology.orchestra.run.smithi096.stdout: "exported": 0, 2024-09-19T05:15:37.601 INFO:teuthology.orchestra.run.smithi096.stdout: "exported_inodes": 0, 2024-09-19T05:15:37.601 INFO:teuthology.orchestra.run.smithi096.stdout: "imported": 0, 2024-09-19T05:15:37.601 INFO:teuthology.orchestra.run.smithi096.stdout: "imported_inodes": 0, 2024-09-19T05:15:37.601 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_dir_fetch": 0, 2024-09-19T05:15:37.601 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_backtrace_fetch": 0, 2024-09-19T05:15:37.601 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_peer_discover": 0, 2024-09-19T05:15:37.601 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rfiles": 0, 2024-09-19T05:15:37.601 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rbytes": 0, 2024-09-19T05:15:37.601 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rsnaps": 0, 2024-09-19T05:15:37.601 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T05:15:37.601 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_set_tag": 0, 2024-09-19T05:15:37.602 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T05:15:37.602 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_inotable_repaired": 0, 2024-09-19T05:15:37.602 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_inodes": 0, 2024-09-19T05:15:37.602 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T05:15:37.602 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T05:15:37.602 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_file_inodes": 0, 2024-09-19T05:15:37.602 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_inode_file_caps": 0, 2024-09-19T05:15:37.602 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T05:15:37.602 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_grant": 0, 2024-09-19T05:15:37.602 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T05:15:37.602 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T05:15:37.602 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T05:15:37.602 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps": 0, 2024-09-19T05:15:37.602 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps_dirty": 0, 2024-09-19T05:15:37.603 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_cap_release": 0, 2024-09-19T05:15:37.603 INFO:teuthology.orchestra.run.smithi096.stdout: "process_request_cap_release": 0 2024-09-19T05:15:37.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.603 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache": { 2024-09-19T05:15:37.603 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update": 0, 2024-09-19T05:15:37.603 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update_receipt": 0, 2024-09-19T05:15:37.603 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_try_discover": 0, 2024-09-19T05:15:37.603 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_send_discover": 0, 2024-09-19T05:15:37.603 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_handle_discover": 0, 2024-09-19T05:15:37.603 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays": 0, 2024-09-19T05:15:37.603 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_delayed": 0, 2024-09-19T05:15:37.603 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_enqueuing": 0, 2024-09-19T05:15:37.603 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_created": 0, 2024-09-19T05:15:37.603 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_enqueued": 0, 2024-09-19T05:15:37.604 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_reintegrated": 0, 2024-09-19T05:15:37.604 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_migrated": 0, 2024-09-19T05:15:37.604 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_processing": 0, 2024-09-19T05:15:37.604 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_enqueued": 0, 2024-09-19T05:15:37.604 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_prioritized": 0, 2024-09-19T05:15:37.604 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_started": 0, 2024-09-19T05:15:37.604 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_completed": 0, 2024-09-19T05:15:37.604 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_path": 0, 2024-09-19T05:15:37.604 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_inode": 0, 2024-09-19T05:15:37.604 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T05:15:37.604 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_exportdir": 0, 2024-09-19T05:15:37.604 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_flush": 0, 2024-09-19T05:15:37.604 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragmentdir": 0, 2024-09-19T05:15:37.605 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragstats": 0, 2024-09-19T05:15:37.605 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_inodestats": 0 2024-09-19T05:15:37.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.605 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log": { 2024-09-19T05:15:37.605 INFO:teuthology.orchestra.run.smithi096.stdout: "evlrg": 0, 2024-09-19T05:15:37.605 INFO:teuthology.orchestra.run.smithi096.stdout: "evadd": 0, 2024-09-19T05:15:37.605 INFO:teuthology.orchestra.run.smithi096.stdout: "evex": 0, 2024-09-19T05:15:37.605 INFO:teuthology.orchestra.run.smithi096.stdout: "evtrm": 0, 2024-09-19T05:15:37.605 INFO:teuthology.orchestra.run.smithi096.stdout: "ev": 6, 2024-09-19T05:15:37.605 INFO:teuthology.orchestra.run.smithi096.stdout: "evexg": 0, 2024-09-19T05:15:37.605 INFO:teuthology.orchestra.run.smithi096.stdout: "evexd": 0, 2024-09-19T05:15:37.605 INFO:teuthology.orchestra.run.smithi096.stdout: "segadd": 0, 2024-09-19T05:15:37.605 INFO:teuthology.orchestra.run.smithi096.stdout: "segex": 0, 2024-09-19T05:15:37.605 INFO:teuthology.orchestra.run.smithi096.stdout: "segtrm": 0, 2024-09-19T05:15:37.606 INFO:teuthology.orchestra.run.smithi096.stdout: "seg": 2, 2024-09-19T05:15:37.606 INFO:teuthology.orchestra.run.smithi096.stdout: "segmjr": 2, 2024-09-19T05:15:37.606 INFO:teuthology.orchestra.run.smithi096.stdout: "segexg": 0, 2024-09-19T05:15:37.606 INFO:teuthology.orchestra.run.smithi096.stdout: "segexd": 0, 2024-09-19T05:15:37.606 INFO:teuthology.orchestra.run.smithi096.stdout: "expos": 4194304, 2024-09-19T05:15:37.606 INFO:teuthology.orchestra.run.smithi096.stdout: "wrpos": 4195960, 2024-09-19T05:15:37.606 INFO:teuthology.orchestra.run.smithi096.stdout: "rdpos": 4195819, 2024-09-19T05:15:37.606 INFO:teuthology.orchestra.run.smithi096.stdout: "jlat": { 2024-09-19T05:15:37.606 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:37.606 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:37.606 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:37.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.606 INFO:teuthology.orchestra.run.smithi096.stdout: "replayed": 6 2024-09-19T05:15:37.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.607 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_mem": { 2024-09-19T05:15:37.607 INFO:teuthology.orchestra.run.smithi096.stdout: "ino": 3, 2024-09-19T05:15:37.607 INFO:teuthology.orchestra.run.smithi096.stdout: "ino+": 3, 2024-09-19T05:15:37.607 INFO:teuthology.orchestra.run.smithi096.stdout: "ino-": 0, 2024-09-19T05:15:37.607 INFO:teuthology.orchestra.run.smithi096.stdout: "dir": 1, 2024-09-19T05:15:37.607 INFO:teuthology.orchestra.run.smithi096.stdout: "dir+": 1, 2024-09-19T05:15:37.607 INFO:teuthology.orchestra.run.smithi096.stdout: "dir-": 0, 2024-09-19T05:15:37.607 INFO:teuthology.orchestra.run.smithi096.stdout: "dn": 0, 2024-09-19T05:15:37.607 INFO:teuthology.orchestra.run.smithi096.stdout: "dn+": 0, 2024-09-19T05:15:37.607 INFO:teuthology.orchestra.run.smithi096.stdout: "dn-": 0, 2024-09-19T05:15:37.607 INFO:teuthology.orchestra.run.smithi096.stdout: "cap": 0, 2024-09-19T05:15:37.607 INFO:teuthology.orchestra.run.smithi096.stdout: "cap+": 0, 2024-09-19T05:15:37.607 INFO:teuthology.orchestra.run.smithi096.stdout: "cap-": 0, 2024-09-19T05:15:37.608 INFO:teuthology.orchestra.run.smithi096.stdout: "rss": 51016, 2024-09-19T05:15:37.608 INFO:teuthology.orchestra.run.smithi096.stdout: "heap": 215328 2024-09-19T05:15:37.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.608 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_server": { 2024-09-19T05:15:37.608 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_client_request": 0, 2024-09-19T05:15:37.608 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_server_request": 0, 2024-09-19T05:15:37.608 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_request": 0, 2024-09-19T05:15:37.608 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_session": 0, 2024-09-19T05:15:37.608 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_peer_request": 0, 2024-09-19T05:15:37.608 INFO:teuthology.orchestra.run.smithi096.stdout: "req_create_latency": { 2024-09-19T05:15:37.608 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:37.608 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:37.608 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:37.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.609 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getattr_latency": { 2024-09-19T05:15:37.609 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:37.609 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:37.609 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:37.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.609 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getfilelock_latency": { 2024-09-19T05:15:37.609 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:37.609 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:37.609 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:37.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.609 INFO:teuthology.orchestra.run.smithi096.stdout: "req_link_latency": { 2024-09-19T05:15:37.609 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:37.609 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:37.609 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:37.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.610 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookup_latency": { 2024-09-19T05:15:37.610 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:37.610 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:37.610 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:37.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.610 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookuphash_latency": { 2024-09-19T05:15:37.610 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:37.610 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:37.610 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:37.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.610 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupino_latency": { 2024-09-19T05:15:37.610 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:37.610 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:37.610 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:37.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.611 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupname_latency": { 2024-09-19T05:15:37.611 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:37.611 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:37.611 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:37.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.611 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupparent_latency": { 2024-09-19T05:15:37.611 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:37.611 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:37.611 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:37.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.611 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupsnap_latency": { 2024-09-19T05:15:37.611 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:37.611 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:37.612 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:37.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.612 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lssnap_latency": { 2024-09-19T05:15:37.612 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:37.612 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:37.612 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:37.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.612 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mkdir_latency": { 2024-09-19T05:15:37.612 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:37.613 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:37.613 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:37.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.613 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mknod_latency": { 2024-09-19T05:15:37.613 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:37.613 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:37.613 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:37.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.613 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mksnap_latency": { 2024-09-19T05:15:37.613 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:37.613 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:37.614 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:37.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.614 INFO:teuthology.orchestra.run.smithi096.stdout: "req_open_latency": { 2024-09-19T05:15:37.614 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:37.614 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:37.614 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:37.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.614 INFO:teuthology.orchestra.run.smithi096.stdout: "req_readdir_latency": { 2024-09-19T05:15:37.614 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:37.614 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:37.614 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:37.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.614 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rename_latency": { 2024-09-19T05:15:37.614 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:37.615 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:37.615 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:37.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.615 INFO:teuthology.orchestra.run.smithi096.stdout: "req_renamesnap_latency": { 2024-09-19T05:15:37.615 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:37.615 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:37.615 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:37.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.615 INFO:teuthology.orchestra.run.smithi096.stdout: "req_snapdiff_latency": { 2024-09-19T05:15:37.615 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:37.615 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:37.615 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:37.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.615 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmdir_latency": { 2024-09-19T05:15:37.616 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:37.616 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:37.616 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:37.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.616 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmsnap_latency": { 2024-09-19T05:15:37.616 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:37.616 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:37.616 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:37.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.616 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmxattr_latency": { 2024-09-19T05:15:37.616 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:37.616 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:37.616 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:37.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.617 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setattr_latency": { 2024-09-19T05:15:37.617 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:37.617 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:37.617 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:37.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.617 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setdirlayout_latency": { 2024-09-19T05:15:37.617 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:37.617 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:37.617 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:37.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.617 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setfilelock_latency": { 2024-09-19T05:15:37.617 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:37.617 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:37.618 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:37.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.618 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setlayout_latency": { 2024-09-19T05:15:37.618 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:37.618 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:37.618 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:37.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.618 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setxattr_latency": { 2024-09-19T05:15:37.618 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:37.618 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:37.618 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:37.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.618 INFO:teuthology.orchestra.run.smithi096.stdout: "req_symlink_latency": { 2024-09-19T05:15:37.618 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:37.619 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:37.619 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:37.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.619 INFO:teuthology.orchestra.run.smithi096.stdout: "req_unlink_latency": { 2024-09-19T05:15:37.619 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:37.619 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:37.619 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:37.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.619 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_revoke_eviction": 0, 2024-09-19T05:15:37.619 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_acquisition_throttle": 0, 2024-09-19T05:15:37.619 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getvxattr_latency": { 2024-09-19T05:15:37.619 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:37.619 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:37.619 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:37.620 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T05:15:37.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.620 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_sessions": { 2024-09-19T05:15:37.620 INFO:teuthology.orchestra.run.smithi096.stdout: "session_count": 0, 2024-09-19T05:15:37.620 INFO:teuthology.orchestra.run.smithi096.stdout: "session_add": 2, 2024-09-19T05:15:37.620 INFO:teuthology.orchestra.run.smithi096.stdout: "session_remove": 2, 2024-09-19T05:15:37.620 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_open": 1, 2024-09-19T05:15:37.620 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_stale": 0, 2024-09-19T05:15:37.620 INFO:teuthology.orchestra.run.smithi096.stdout: "total_load": 0, 2024-09-19T05:15:37.620 INFO:teuthology.orchestra.run.smithi096.stdout: "average_load": 0, 2024-09-19T05:15:37.620 INFO:teuthology.orchestra.run.smithi096.stdout: "avg_session_uptime": 1467, 2024-09-19T05:15:37.620 INFO:teuthology.orchestra.run.smithi096.stdout: "mdthresh_evicted": 0 2024-09-19T05:15:37.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.621 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-09-19T05:15:37.621 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 0, 2024-09-19T05:15:37.621 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 0, 2024-09-19T05:15:37.621 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T05:15:37.621 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-09-19T05:15:37.621 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T05:15:37.621 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-09-19T05:15:37.621 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T05:15:37.621 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T05:15:37.621 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T05:15:37.621 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T05:15:37.621 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T05:15:37.621 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-09-19T05:15:37.622 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T05:15:37.622 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T05:15:37.622 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-09-19T05:15:37.622 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-09-19T05:15:37.622 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-09-19T05:15:37.622 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-09-19T05:15:37.622 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T05:15:37.622 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T05:15:37.622 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T05:15:37.622 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T05:15:37.622 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T05:15:37.622 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-09-19T05:15:37.622 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-09-19T05:15:37.622 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-09-19T05:15:37.623 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T05:15:37.623 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T05:15:37.623 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-09-19T05:15:37.623 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-09-19T05:15:37.623 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-09-19T05:15:37.623 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-09-19T05:15:37.623 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T05:15:37.623 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-09-19T05:15:37.623 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T05:15:37.623 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-09-19T05:15:37.623 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 181269, 2024-09-19T05:15:37.623 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 47, 2024-09-19T05:15:37.623 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 0, 2024-09-19T05:15:37.623 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 0, 2024-09-19T05:15:37.624 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-09-19T05:15:37.624 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-09-19T05:15:37.624 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-09-19T05:15:37.624 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-09-19T05:15:37.624 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-09-19T05:15:37.624 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-09-19T05:15:37.624 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 8224, 2024-09-19T05:15:37.624 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 181, 2024-09-19T05:15:37.624 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T05:15:37.624 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-09-19T05:15:37.624 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-09-19T05:15:37.624 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-09-19T05:15:37.624 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 8512, 2024-09-19T05:15:37.624 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 13, 2024-09-19T05:15:37.624 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-09-19T05:15:37.625 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-09-19T05:15:37.625 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-09-19T05:15:37.625 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-09-19T05:15:37.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.625 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter": { 2024-09-19T05:15:37.625 INFO:teuthology.orchestra.run.smithi096.stdout: "op_active": 0, 2024-09-19T05:15:37.625 INFO:teuthology.orchestra.run.smithi096.stdout: "op_laggy": 0, 2024-09-19T05:15:37.625 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send": 2253, 2024-09-19T05:15:37.625 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send_bytes": 21, 2024-09-19T05:15:37.625 INFO:teuthology.orchestra.run.smithi096.stdout: "op_resend": 3, 2024-09-19T05:15:37.625 INFO:teuthology.orchestra.run.smithi096.stdout: "op_reply": 2253, 2024-09-19T05:15:37.625 INFO:teuthology.orchestra.run.smithi096.stdout: "op_latency": { 2024-09-19T05:15:37.625 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 2253, 2024-09-19T05:15:37.625 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 1.203971695, 2024-09-19T05:15:37.626 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000534386 2024-09-19T05:15:37.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.626 INFO:teuthology.orchestra.run.smithi096.stdout: "op_inflight": 0, 2024-09-19T05:15:37.626 INFO:teuthology.orchestra.run.smithi096.stdout: "oplen_avg": { 2024-09-19T05:15:37.626 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 2253, 2024-09-19T05:15:37.626 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 2254 2024-09-19T05:15:37.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.626 INFO:teuthology.orchestra.run.smithi096.stdout: "op": 2253, 2024-09-19T05:15:37.626 INFO:teuthology.orchestra.run.smithi096.stdout: "op_r": 2253, 2024-09-19T05:15:37.626 INFO:teuthology.orchestra.run.smithi096.stdout: "op_w": 0, 2024-09-19T05:15:37.626 INFO:teuthology.orchestra.run.smithi096.stdout: "op_rmw": 0, 2024-09-19T05:15:37.626 INFO:teuthology.orchestra.run.smithi096.stdout: "op_pg": 0, 2024-09-19T05:15:37.626 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_stat": 1466, 2024-09-19T05:15:37.626 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_create": 0, 2024-09-19T05:15:37.627 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_read": 785, 2024-09-19T05:15:37.627 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_write": 0, 2024-09-19T05:15:37.627 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writefull": 0, 2024-09-19T05:15:37.627 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writesame": 0, 2024-09-19T05:15:37.627 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_append": 0, 2024-09-19T05:15:37.627 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_zero": 0, 2024-09-19T05:15:37.627 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_truncate": 0, 2024-09-19T05:15:37.627 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_delete": 0, 2024-09-19T05:15:37.627 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_mapext": 0, 2024-09-19T05:15:37.627 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_sparse_read": 0, 2024-09-19T05:15:37.628 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_clonerange": 0, 2024-09-19T05:15:37.628 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_getxattr": 1, 2024-09-19T05:15:37.628 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_setxattr": 0, 2024-09-19T05:15:37.628 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_cmpxattr": 0, 2024-09-19T05:15:37.628 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_rmxattr": 0, 2024-09-19T05:15:37.628 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_resetxattrs": 0, 2024-09-19T05:15:37.628 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_call": 0, 2024-09-19T05:15:37.628 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_watch": 0, 2024-09-19T05:15:37.628 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_notify": 0, 2024-09-19T05:15:37.629 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T05:15:37.629 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls": 0, 2024-09-19T05:15:37.629 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls_filter": 0, 2024-09-19T05:15:37.629 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_other": 0, 2024-09-19T05:15:37.629 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_active": 0, 2024-09-19T05:15:37.629 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_send": 0, 2024-09-19T05:15:37.629 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_resend": 0, 2024-09-19T05:15:37.629 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_ping": 0, 2024-09-19T05:15:37.629 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_active": 0, 2024-09-19T05:15:37.629 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_send": 0, 2024-09-19T05:15:37.629 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_resend": 0, 2024-09-19T05:15:37.629 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_active": 0, 2024-09-19T05:15:37.629 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_send": 0, 2024-09-19T05:15:37.629 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_resend": 0, 2024-09-19T05:15:37.630 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_active": 0, 2024-09-19T05:15:37.630 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_send": 0, 2024-09-19T05:15:37.630 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_resend": 0, 2024-09-19T05:15:37.630 INFO:teuthology.orchestra.run.smithi096.stdout: "command_active": 0, 2024-09-19T05:15:37.630 INFO:teuthology.orchestra.run.smithi096.stdout: "command_send": 0, 2024-09-19T05:15:37.630 INFO:teuthology.orchestra.run.smithi096.stdout: "command_resend": 0, 2024-09-19T05:15:37.630 INFO:teuthology.orchestra.run.smithi096.stdout: "map_epoch": 96, 2024-09-19T05:15:37.630 INFO:teuthology.orchestra.run.smithi096.stdout: "map_full": 0, 2024-09-19T05:15:37.630 INFO:teuthology.orchestra.run.smithi096.stdout: "map_inc": 13, 2024-09-19T05:15:37.630 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_sessions": 4, 2024-09-19T05:15:37.630 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_open": 4, 2024-09-19T05:15:37.630 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_close": 0, 2024-09-19T05:15:37.630 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_laggy": 0, 2024-09-19T05:15:37.630 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_wr": 0, 2024-09-19T05:15:37.631 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_rd": 2, 2024-09-19T05:15:37.631 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_del": 0 2024-09-19T05:15:37.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.631 INFO:teuthology.orchestra.run.smithi096.stdout: "oft": { 2024-09-19T05:15:37.631 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_objs": 0, 2024-09-19T05:15:37.631 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_kv_pairs": 0, 2024-09-19T05:15:37.631 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_updates": 0, 2024-09-19T05:15:37.631 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_removes": 0 2024-09-19T05:15:37.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.631 INFO:teuthology.orchestra.run.smithi096.stdout: "osd-slow-ops": { 2024-09-19T05:15:37.631 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_ops_count": 0 2024-09-19T05:15:37.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.631 INFO:teuthology.orchestra.run.smithi096.stdout: "purge_queue": { 2024-09-19T05:15:37.631 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops": 0, 2024-09-19T05:15:37.631 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T05:15:37.632 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing": 0, 2024-09-19T05:15:37.632 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_high_water": 0, 2024-09-19T05:15:37.632 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed_ops": 0, 2024-09-19T05:15:37.632 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed": 0, 2024-09-19T05:15:37.632 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_item_in_journal": 0 2024-09-19T05:15:37.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.632 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T05:15:37.632 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-19T05:15:37.632 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-09-19T05:15:37.632 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-19T05:15:37.632 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 2508, 2024-09-19T05:15:37.632 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 2027643, 2024-09-19T05:15:37.632 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:37.633 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 2508, 2024-09-19T05:15:37.633 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-19T05:15:37.633 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-19T05:15:37.633 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 2508, 2024-09-19T05:15:37.633 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 2027643, 2024-09-19T05:15:37.633 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-19T05:15:37.633 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:37.633 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:37.633 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:37.633 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T05:15:37.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.634 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_bytes": { 2024-09-19T05:15:37.634 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-19T05:15:37.634 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-09-19T05:15:37.634 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-19T05:15:37.634 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-19T05:15:37.634 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-19T05:15:37.634 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:37.634 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-19T05:15:37.634 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 2253, 2024-09-19T05:15:37.634 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 1661, 2024-09-19T05:15:37.634 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 2253, 2024-09-19T05:15:37.634 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 1661, 2024-09-19T05:15:37.634 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-19T05:15:37.635 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:37.635 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:37.635 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:37.635 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T05:15:37.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.635 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_ops": { 2024-09-19T05:15:37.635 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-19T05:15:37.635 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 1024, 2024-09-19T05:15:37.635 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-19T05:15:37.635 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-19T05:15:37.635 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-19T05:15:37.635 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:37.635 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-19T05:15:37.635 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 2253, 2024-09-19T05:15:37.636 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 2253, 2024-09-19T05:15:37.636 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 2253, 2024-09-19T05:15:37.636 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 2253, 2024-09-19T05:15:37.636 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-19T05:15:37.636 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:37.636 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:37.636 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:37.636 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T05:15:37.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.636 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle": { 2024-09-19T05:15:37.636 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-19T05:15:37.636 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-09-19T05:15:37.636 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-19T05:15:37.637 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-19T05:15:37.637 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-19T05:15:37.637 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:37.637 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-19T05:15:37.637 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-19T05:15:37.637 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-19T05:15:37.637 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 0, 2024-09-19T05:15:37.637 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 0, 2024-09-19T05:15:37.637 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-19T05:15:37.637 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:37.637 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:37.637 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:37.637 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T05:15:37.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:37.638 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle-0x564cf3586b40": { 2024-09-19T05:15:37.638 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-19T05:15:37.638 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-09-19T05:15:37.638 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-19T05:15:37.638 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-19T05:15:37.638 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-19T05:15:37.638 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:37.638 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-19T05:15:37.638 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-19T05:15:37.638 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-19T05:15:37.638 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 0, 2024-09-19T05:15:37.638 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 0, 2024-09-19T05:15:37.638 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-19T05:15:37.638 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:37.639 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:37.639 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:37.639 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T05:15:37.639 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T05:15:37.639 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-19T05:15:38.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:37 smithi096 ceph-mon[35085]: pgmap v342: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:15:38.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:37 smithi043 ceph-mon[30800]: pgmap v342: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:15:38.398 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-19T05:15:38.398 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-19T05:15:38.398 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-09-19T05:15:38.398 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-09-19T05:15:38.398 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T05:15:38.398 DEBUG:tasks.check_counter:Getting stats from h 2024-09-19T05:15:38.399 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-19T05:15:38.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:37 smithi151 ceph-mon[35425]: pgmap v342: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:15:38.643 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T05:15:39.231 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-19T05:15:39.231 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T05:15:39.231 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 50, 2024-09-19T05:15:39.232 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 216, 2024-09-19T05:15:39.232 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 12271, 2024-09-19T05:15:39.232 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 377452, 2024-09-19T05:15:39.232 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 13, 2024-09-19T05:15:39.232 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 3, 2024-09-19T05:15:39.232 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.019131799, 2024-09-19T05:15:39.232 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.007318101, 2024-09-19T05:15:39.232 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.005705457, 2024-09-19T05:15:39.232 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000653002, 2024-09-19T05:15:39.232 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:15:39.232 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 214, 2024-09-19T05:15:39.232 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.025205938, 2024-09-19T05:15:39.232 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000117784 2024-09-19T05:15:39.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.233 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:15:39.233 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 58, 2024-09-19T05:15:39.233 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000010891, 2024-09-19T05:15:39.233 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000187 2024-09-19T05:15:39.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.233 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 4384, 2024-09-19T05:15:39.233 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 337808 2024-09-19T05:15:39.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.234 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T05:15:39.234 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 339, 2024-09-19T05:15:39.234 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 322, 2024-09-19T05:15:39.234 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 1657579, 2024-09-19T05:15:39.234 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 613553, 2024-09-19T05:15:39.234 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 44, 2024-09-19T05:15:39.234 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 6, 2024-09-19T05:15:39.234 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.065344737, 2024-09-19T05:15:39.234 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.017489828, 2024-09-19T05:15:39.234 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.063778905, 2024-09-19T05:15:39.234 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000256881, 2024-09-19T05:15:39.234 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:15:39.235 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 322, 2024-09-19T05:15:39.235 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.032404431, 2024-09-19T05:15:39.235 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000100634 2024-09-19T05:15:39.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.235 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:15:39.235 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 47, 2024-09-19T05:15:39.235 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000004573, 2024-09-19T05:15:39.235 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000097 2024-09-19T05:15:39.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.235 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 1648800, 2024-09-19T05:15:39.235 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 144528 2024-09-19T05:15:39.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.235 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T05:15:39.235 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 364, 2024-09-19T05:15:39.236 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 590, 2024-09-19T05:15:39.236 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 100551, 2024-09-19T05:15:39.236 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 486313, 2024-09-19T05:15:39.236 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 9, 2024-09-19T05:15:39.236 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 4, 2024-09-19T05:15:39.236 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.065991592, 2024-09-19T05:15:39.236 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.031142635, 2024-09-19T05:15:39.236 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.027726418, 2024-09-19T05:15:39.236 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.003791725, 2024-09-19T05:15:39.236 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:15:39.236 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 587, 2024-09-19T05:15:39.236 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.036197424, 2024-09-19T05:15:39.237 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000061665 2024-09-19T05:15:39.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.237 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:15:39.237 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 879, 2024-09-19T05:15:39.237 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000219356, 2024-09-19T05:15:39.237 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000249 2024-09-19T05:15:39.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.237 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-09-19T05:15:39.237 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 132176 2024-09-19T05:15:39.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.237 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-09-19T05:15:39.237 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 1, 2024-09-19T05:15:39.237 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-09-19T05:15:39.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.238 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-MDSRank": { 2024-09-19T05:15:39.238 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-09-19T05:15:39.238 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-09-19T05:15:39.238 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 63, 2024-09-19T05:15:39.238 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.001787518, 2024-09-19T05:15:39.238 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000028373 2024-09-19T05:15:39.238 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T05:15:39.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.238 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-PurgeQueue": { 2024-09-19T05:15:39.238 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-09-19T05:15:39.238 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-09-19T05:15:39.238 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 2, 2024-09-19T05:15:39.238 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000010524, 2024-09-19T05:15:39.239 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000005262 2024-09-19T05:15:39.239 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T05:15:39.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.239 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": { 2024-09-19T05:15:39.239 INFO:teuthology.orchestra.run.smithi096.stdout: "request": 7, 2024-09-19T05:15:39.239 INFO:teuthology.orchestra.run.smithi096.stdout: "reply": 4, 2024-09-19T05:15:39.239 INFO:teuthology.orchestra.run.smithi096.stdout: "reply_latency": { 2024-09-19T05:15:39.239 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 4, 2024-09-19T05:15:39.239 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000539874, 2024-09-19T05:15:39.240 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000134968 2024-09-19T05:15:39.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.240 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_reply": 0, 2024-09-19T05:15:39.240 INFO:teuthology.orchestra.run.smithi096.stdout: "forward": 3, 2024-09-19T05:15:39.240 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_complete": 0, 2024-09-19T05:15:39.240 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_keys": 0, 2024-09-19T05:15:39.240 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_commit": 11, 2024-09-19T05:15:39.240 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_split": 0, 2024-09-19T05:15:39.240 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_merge": 0, 2024-09-19T05:15:39.240 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes": 10, 2024-09-19T05:15:39.240 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_top": 0, 2024-09-19T05:15:39.240 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_bottom": 10, 2024-09-19T05:15:39.240 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pin_tail": 0, 2024-09-19T05:15:39.240 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pinned": 10, 2024-09-19T05:15:39.241 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_expired": 0, 2024-09-19T05:15:39.241 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_with_caps": 1, 2024-09-19T05:15:39.241 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": 2, 2024-09-19T05:15:39.241 INFO:teuthology.orchestra.run.smithi096.stdout: "subtrees": 1, 2024-09-19T05:15:39.241 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse": 7, 2024-09-19T05:15:39.241 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_hit": 6, 2024-09-19T05:15:39.241 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_forward": 0, 2024-09-19T05:15:39.241 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_discover": 0, 2024-09-19T05:15:39.241 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_dir_fetch": 0, 2024-09-19T05:15:39.241 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_remote_ino": 0, 2024-09-19T05:15:39.241 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_lock": 0, 2024-09-19T05:15:39.241 INFO:teuthology.orchestra.run.smithi096.stdout: "load_cent": 0, 2024-09-19T05:15:39.241 INFO:teuthology.orchestra.run.smithi096.stdout: "q": 0, 2024-09-19T05:15:39.241 INFO:teuthology.orchestra.run.smithi096.stdout: "exported": 0, 2024-09-19T05:15:39.242 INFO:teuthology.orchestra.run.smithi096.stdout: "exported_inodes": 0, 2024-09-19T05:15:39.242 INFO:teuthology.orchestra.run.smithi096.stdout: "imported": 0, 2024-09-19T05:15:39.242 INFO:teuthology.orchestra.run.smithi096.stdout: "imported_inodes": 0, 2024-09-19T05:15:39.242 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_dir_fetch": 0, 2024-09-19T05:15:39.242 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_backtrace_fetch": 0, 2024-09-19T05:15:39.242 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_peer_discover": 1, 2024-09-19T05:15:39.242 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rfiles": 4, 2024-09-19T05:15:39.242 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rbytes": 234, 2024-09-19T05:15:39.242 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rsnaps": 6, 2024-09-19T05:15:39.242 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T05:15:39.242 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_set_tag": 0, 2024-09-19T05:15:39.242 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T05:15:39.242 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_inotable_repaired": 0, 2024-09-19T05:15:39.242 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_inodes": 0, 2024-09-19T05:15:39.243 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T05:15:39.243 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T05:15:39.243 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_file_inodes": 0, 2024-09-19T05:15:39.243 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_inode_file_caps": 0, 2024-09-19T05:15:39.243 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T05:15:39.243 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_grant": 0, 2024-09-19T05:15:39.243 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T05:15:39.243 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T05:15:39.243 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T05:15:39.243 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps": 0, 2024-09-19T05:15:39.243 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps_dirty": 0, 2024-09-19T05:15:39.243 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_cap_release": 0, 2024-09-19T05:15:39.243 INFO:teuthology.orchestra.run.smithi096.stdout: "process_request_cap_release": 0 2024-09-19T05:15:39.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.244 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache": { 2024-09-19T05:15:39.244 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update": 0, 2024-09-19T05:15:39.244 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update_receipt": 0, 2024-09-19T05:15:39.244 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_try_discover": 0, 2024-09-19T05:15:39.244 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_send_discover": 1, 2024-09-19T05:15:39.244 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_handle_discover": 0, 2024-09-19T05:15:39.244 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays": 0, 2024-09-19T05:15:39.244 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_delayed": 0, 2024-09-19T05:15:39.244 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_enqueuing": 0, 2024-09-19T05:15:39.244 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_created": 0, 2024-09-19T05:15:39.244 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_enqueued": 0, 2024-09-19T05:15:39.244 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_reintegrated": 0, 2024-09-19T05:15:39.244 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_migrated": 0, 2024-09-19T05:15:39.244 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_processing": 0, 2024-09-19T05:15:39.244 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_enqueued": 0, 2024-09-19T05:15:39.245 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_prioritized": 0, 2024-09-19T05:15:39.245 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_started": 0, 2024-09-19T05:15:39.245 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_completed": 0, 2024-09-19T05:15:39.245 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_path": 0, 2024-09-19T05:15:39.245 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_inode": 0, 2024-09-19T05:15:39.245 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T05:15:39.245 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_exportdir": 0, 2024-09-19T05:15:39.245 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_flush": 0, 2024-09-19T05:15:39.245 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragmentdir": 0, 2024-09-19T05:15:39.245 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragstats": 0, 2024-09-19T05:15:39.245 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_inodestats": 0 2024-09-19T05:15:39.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.245 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log": { 2024-09-19T05:15:39.245 INFO:teuthology.orchestra.run.smithi096.stdout: "evlrg": 0, 2024-09-19T05:15:39.246 INFO:teuthology.orchestra.run.smithi096.stdout: "evadd": 10, 2024-09-19T05:15:39.246 INFO:teuthology.orchestra.run.smithi096.stdout: "evex": 0, 2024-09-19T05:15:39.246 INFO:teuthology.orchestra.run.smithi096.stdout: "evtrm": 0, 2024-09-19T05:15:39.246 INFO:teuthology.orchestra.run.smithi096.stdout: "ev": 10, 2024-09-19T05:15:39.246 INFO:teuthology.orchestra.run.smithi096.stdout: "evexg": 0, 2024-09-19T05:15:39.246 INFO:teuthology.orchestra.run.smithi096.stdout: "evexd": 0, 2024-09-19T05:15:39.246 INFO:teuthology.orchestra.run.smithi096.stdout: "segadd": 2, 2024-09-19T05:15:39.246 INFO:teuthology.orchestra.run.smithi096.stdout: "segex": 0, 2024-09-19T05:15:39.246 INFO:teuthology.orchestra.run.smithi096.stdout: "segtrm": 0, 2024-09-19T05:15:39.246 INFO:teuthology.orchestra.run.smithi096.stdout: "seg": 2, 2024-09-19T05:15:39.246 INFO:teuthology.orchestra.run.smithi096.stdout: "segmjr": 2, 2024-09-19T05:15:39.246 INFO:teuthology.orchestra.run.smithi096.stdout: "segexg": 0, 2024-09-19T05:15:39.246 INFO:teuthology.orchestra.run.smithi096.stdout: "segexd": 0, 2024-09-19T05:15:39.247 INFO:teuthology.orchestra.run.smithi096.stdout: "expos": 4194304, 2024-09-19T05:15:39.247 INFO:teuthology.orchestra.run.smithi096.stdout: "wrpos": 4197198, 2024-09-19T05:15:39.247 INFO:teuthology.orchestra.run.smithi096.stdout: "rdpos": 0, 2024-09-19T05:15:39.247 INFO:teuthology.orchestra.run.smithi096.stdout: "jlat": { 2024-09-19T05:15:39.247 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 9, 2024-09-19T05:15:39.247 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.019711288, 2024-09-19T05:15:39.247 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.002190143 2024-09-19T05:15:39.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.247 INFO:teuthology.orchestra.run.smithi096.stdout: "replayed": 0 2024-09-19T05:15:39.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.247 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_mem": { 2024-09-19T05:15:39.247 INFO:teuthology.orchestra.run.smithi096.stdout: "ino": 13, 2024-09-19T05:15:39.247 INFO:teuthology.orchestra.run.smithi096.stdout: "ino+": 13, 2024-09-19T05:15:39.247 INFO:teuthology.orchestra.run.smithi096.stdout: "ino-": 0, 2024-09-19T05:15:39.247 INFO:teuthology.orchestra.run.smithi096.stdout: "dir": 11, 2024-09-19T05:15:39.248 INFO:teuthology.orchestra.run.smithi096.stdout: "dir+": 12, 2024-09-19T05:15:39.248 INFO:teuthology.orchestra.run.smithi096.stdout: "dir-": 1, 2024-09-19T05:15:39.248 INFO:teuthology.orchestra.run.smithi096.stdout: "dn": 10, 2024-09-19T05:15:39.248 INFO:teuthology.orchestra.run.smithi096.stdout: "dn+": 10, 2024-09-19T05:15:39.248 INFO:teuthology.orchestra.run.smithi096.stdout: "dn-": 0, 2024-09-19T05:15:39.248 INFO:teuthology.orchestra.run.smithi096.stdout: "cap": 2, 2024-09-19T05:15:39.248 INFO:teuthology.orchestra.run.smithi096.stdout: "cap+": 2, 2024-09-19T05:15:39.248 INFO:teuthology.orchestra.run.smithi096.stdout: "cap-": 0, 2024-09-19T05:15:39.248 INFO:teuthology.orchestra.run.smithi096.stdout: "rss": 49852, 2024-09-19T05:15:39.248 INFO:teuthology.orchestra.run.smithi096.stdout: "heap": 223520 2024-09-19T05:15:39.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.248 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_server": { 2024-09-19T05:15:39.248 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_client_request": 7, 2024-09-19T05:15:39.249 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_server_request": 0, 2024-09-19T05:15:39.249 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_request": 7, 2024-09-19T05:15:39.249 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_session": 30, 2024-09-19T05:15:39.249 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_peer_request": 0, 2024-09-19T05:15:39.249 INFO:teuthology.orchestra.run.smithi096.stdout: "req_create_latency": { 2024-09-19T05:15:39.249 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:39.249 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:39.249 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:39.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.249 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getattr_latency": { 2024-09-19T05:15:39.249 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 3, 2024-09-19T05:15:39.249 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000463196, 2024-09-19T05:15:39.249 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000154398 2024-09-19T05:15:39.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.249 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getfilelock_latency": { 2024-09-19T05:15:39.250 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:39.250 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:39.250 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:39.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.250 INFO:teuthology.orchestra.run.smithi096.stdout: "req_link_latency": { 2024-09-19T05:15:39.250 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:39.250 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:39.250 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:39.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.250 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookup_latency": { 2024-09-19T05:15:39.250 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:39.250 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:39.250 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:39.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.251 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookuphash_latency": { 2024-09-19T05:15:39.251 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:39.251 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:39.251 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:39.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.251 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupino_latency": { 2024-09-19T05:15:39.251 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1, 2024-09-19T05:15:39.251 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000076678, 2024-09-19T05:15:39.251 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000076678 2024-09-19T05:15:39.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.251 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupname_latency": { 2024-09-19T05:15:39.251 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:39.251 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:39.251 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:39.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.252 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupparent_latency": { 2024-09-19T05:15:39.252 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:39.252 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:39.252 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:39.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.252 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupsnap_latency": { 2024-09-19T05:15:39.252 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:39.252 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:39.252 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:39.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.252 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lssnap_latency": { 2024-09-19T05:15:39.252 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:39.252 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:39.252 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:39.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.253 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mkdir_latency": { 2024-09-19T05:15:39.253 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:39.253 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:39.253 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:39.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.253 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mknod_latency": { 2024-09-19T05:15:39.253 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:39.253 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:39.253 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:39.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.253 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mksnap_latency": { 2024-09-19T05:15:39.253 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:39.254 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:39.254 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:39.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.254 INFO:teuthology.orchestra.run.smithi096.stdout: "req_open_latency": { 2024-09-19T05:15:39.254 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:39.254 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:39.254 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:39.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.254 INFO:teuthology.orchestra.run.smithi096.stdout: "req_readdir_latency": { 2024-09-19T05:15:39.254 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:39.254 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:39.254 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:39.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.254 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rename_latency": { 2024-09-19T05:15:39.254 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:39.255 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:39.255 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:39.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.255 INFO:teuthology.orchestra.run.smithi096.stdout: "req_renamesnap_latency": { 2024-09-19T05:15:39.255 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:39.255 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:39.255 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:39.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.255 INFO:teuthology.orchestra.run.smithi096.stdout: "req_snapdiff_latency": { 2024-09-19T05:15:39.255 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:39.255 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:39.255 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:39.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.255 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmdir_latency": { 2024-09-19T05:15:39.256 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:39.256 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:39.256 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:39.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.256 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmsnap_latency": { 2024-09-19T05:15:39.256 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:39.256 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:39.256 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:39.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.257 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmxattr_latency": { 2024-09-19T05:15:39.257 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:39.257 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:39.257 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:39.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.257 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setattr_latency": { 2024-09-19T05:15:39.257 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:39.257 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:39.257 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:39.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.257 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setdirlayout_latency": { 2024-09-19T05:15:39.257 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:39.257 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:39.257 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:39.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.258 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setfilelock_latency": { 2024-09-19T05:15:39.258 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:39.258 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:39.258 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:39.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.258 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setlayout_latency": { 2024-09-19T05:15:39.258 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:39.258 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:39.258 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:39.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.258 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setxattr_latency": { 2024-09-19T05:15:39.258 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:39.259 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:39.259 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:39.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.259 INFO:teuthology.orchestra.run.smithi096.stdout: "req_symlink_latency": { 2024-09-19T05:15:39.259 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:39.259 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:39.259 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:39.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.259 INFO:teuthology.orchestra.run.smithi096.stdout: "req_unlink_latency": { 2024-09-19T05:15:39.259 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:39.259 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:39.259 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:39.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.259 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_revoke_eviction": 0, 2024-09-19T05:15:39.260 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_acquisition_throttle": 0, 2024-09-19T05:15:39.260 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getvxattr_latency": { 2024-09-19T05:15:39.260 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:39.260 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:39.260 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:39.260 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T05:15:39.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.260 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_sessions": { 2024-09-19T05:15:39.260 INFO:teuthology.orchestra.run.smithi096.stdout: "session_count": 2, 2024-09-19T05:15:39.260 INFO:teuthology.orchestra.run.smithi096.stdout: "session_add": 5, 2024-09-19T05:15:39.260 INFO:teuthology.orchestra.run.smithi096.stdout: "session_remove": 3, 2024-09-19T05:15:39.260 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_open": 2, 2024-09-19T05:15:39.260 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_stale": 0, 2024-09-19T05:15:39.260 INFO:teuthology.orchestra.run.smithi096.stdout: "total_load": 3, 2024-09-19T05:15:39.261 INFO:teuthology.orchestra.run.smithi096.stdout: "average_load": 0, 2024-09-19T05:15:39.261 INFO:teuthology.orchestra.run.smithi096.stdout: "avg_session_uptime": 639, 2024-09-19T05:15:39.261 INFO:teuthology.orchestra.run.smithi096.stdout: "mdthresh_evicted": 0 2024-09-19T05:15:39.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.261 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-09-19T05:15:39.261 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 0, 2024-09-19T05:15:39.261 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 0, 2024-09-19T05:15:39.261 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T05:15:39.261 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-09-19T05:15:39.261 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T05:15:39.261 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-09-19T05:15:39.261 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T05:15:39.261 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T05:15:39.262 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T05:15:39.262 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T05:15:39.262 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T05:15:39.262 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-09-19T05:15:39.262 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T05:15:39.262 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T05:15:39.262 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-09-19T05:15:39.262 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-09-19T05:15:39.262 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-09-19T05:15:39.262 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-09-19T05:15:39.262 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T05:15:39.262 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T05:15:39.262 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T05:15:39.262 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T05:15:39.263 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T05:15:39.263 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-09-19T05:15:39.263 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-09-19T05:15:39.263 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-09-19T05:15:39.263 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T05:15:39.263 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T05:15:39.263 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-09-19T05:15:39.263 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-09-19T05:15:39.263 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-09-19T05:15:39.263 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-09-19T05:15:39.263 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T05:15:39.263 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-09-19T05:15:39.263 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T05:15:39.263 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-09-19T05:15:39.264 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 247797, 2024-09-19T05:15:39.264 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 89, 2024-09-19T05:15:39.264 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 88, 2024-09-19T05:15:39.264 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 1, 2024-09-19T05:15:39.264 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-09-19T05:15:39.264 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-09-19T05:15:39.264 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-09-19T05:15:39.264 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-09-19T05:15:39.264 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-09-19T05:15:39.264 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-09-19T05:15:39.264 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 8224, 2024-09-19T05:15:39.264 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 181, 2024-09-19T05:15:39.264 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T05:15:39.265 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-09-19T05:15:39.265 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-09-19T05:15:39.265 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-09-19T05:15:39.265 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 51608, 2024-09-19T05:15:39.265 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 202, 2024-09-19T05:15:39.265 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-09-19T05:15:39.265 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-09-19T05:15:39.265 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-09-19T05:15:39.265 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-09-19T05:15:39.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.265 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter": { 2024-09-19T05:15:39.265 INFO:teuthology.orchestra.run.smithi096.stdout: "op_active": 0, 2024-09-19T05:15:39.265 INFO:teuthology.orchestra.run.smithi096.stdout: "op_laggy": 0, 2024-09-19T05:15:39.265 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send": 47, 2024-09-19T05:15:39.266 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send_bytes": 13376, 2024-09-19T05:15:39.266 INFO:teuthology.orchestra.run.smithi096.stdout: "op_resend": 28, 2024-09-19T05:15:39.266 INFO:teuthology.orchestra.run.smithi096.stdout: "op_reply": 47, 2024-09-19T05:15:39.266 INFO:teuthology.orchestra.run.smithi096.stdout: "op_latency": { 2024-09-19T05:15:39.266 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 47, 2024-09-19T05:15:39.266 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.240994360, 2024-09-19T05:15:39.266 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.005127539 2024-09-19T05:15:39.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.266 INFO:teuthology.orchestra.run.smithi096.stdout: "op_inflight": 0, 2024-09-19T05:15:39.266 INFO:teuthology.orchestra.run.smithi096.stdout: "oplen_avg": { 2024-09-19T05:15:39.266 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 47, 2024-09-19T05:15:39.266 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 69 2024-09-19T05:15:39.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.266 INFO:teuthology.orchestra.run.smithi096.stdout: "op": 47, 2024-09-19T05:15:39.267 INFO:teuthology.orchestra.run.smithi096.stdout: "op_r": 0, 2024-09-19T05:15:39.267 INFO:teuthology.orchestra.run.smithi096.stdout: "op_w": 47, 2024-09-19T05:15:39.267 INFO:teuthology.orchestra.run.smithi096.stdout: "op_rmw": 0, 2024-09-19T05:15:39.267 INFO:teuthology.orchestra.run.smithi096.stdout: "op_pg": 0, 2024-09-19T05:15:39.267 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_stat": 0, 2024-09-19T05:15:39.267 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_create": 10, 2024-09-19T05:15:39.267 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_read": 0, 2024-09-19T05:15:39.267 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_write": 9, 2024-09-19T05:15:39.267 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writefull": 9, 2024-09-19T05:15:39.267 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writesame": 0, 2024-09-19T05:15:39.267 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_append": 0, 2024-09-19T05:15:39.267 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_zero": 0, 2024-09-19T05:15:39.267 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_truncate": 0, 2024-09-19T05:15:39.267 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_delete": 6, 2024-09-19T05:15:39.268 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_mapext": 0, 2024-09-19T05:15:39.268 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_sparse_read": 0, 2024-09-19T05:15:39.268 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_clonerange": 0, 2024-09-19T05:15:39.268 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_getxattr": 0, 2024-09-19T05:15:39.268 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_setxattr": 20, 2024-09-19T05:15:39.268 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_cmpxattr": 0, 2024-09-19T05:15:39.268 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_rmxattr": 0, 2024-09-19T05:15:39.268 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_resetxattrs": 0, 2024-09-19T05:15:39.268 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_call": 0, 2024-09-19T05:15:39.268 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_watch": 0, 2024-09-19T05:15:39.268 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_notify": 0, 2024-09-19T05:15:39.268 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T05:15:39.268 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls": 0, 2024-09-19T05:15:39.268 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls_filter": 0, 2024-09-19T05:15:39.269 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_other": 0, 2024-09-19T05:15:39.269 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_active": 0, 2024-09-19T05:15:39.269 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_send": 0, 2024-09-19T05:15:39.269 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_resend": 0, 2024-09-19T05:15:39.269 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_ping": 0, 2024-09-19T05:15:39.269 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_active": 0, 2024-09-19T05:15:39.269 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_send": 0, 2024-09-19T05:15:39.269 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_resend": 0, 2024-09-19T05:15:39.269 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_active": 0, 2024-09-19T05:15:39.269 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_send": 0, 2024-09-19T05:15:39.269 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_resend": 0, 2024-09-19T05:15:39.269 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_active": 0, 2024-09-19T05:15:39.269 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_send": 0, 2024-09-19T05:15:39.269 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_resend": 0, 2024-09-19T05:15:39.269 INFO:teuthology.orchestra.run.smithi096.stdout: "command_active": 0, 2024-09-19T05:15:39.270 INFO:teuthology.orchestra.run.smithi096.stdout: "command_send": 0, 2024-09-19T05:15:39.270 INFO:teuthology.orchestra.run.smithi096.stdout: "command_resend": 0, 2024-09-19T05:15:39.270 INFO:teuthology.orchestra.run.smithi096.stdout: "map_epoch": 96, 2024-09-19T05:15:39.270 INFO:teuthology.orchestra.run.smithi096.stdout: "map_full": 0, 2024-09-19T05:15:39.270 INFO:teuthology.orchestra.run.smithi096.stdout: "map_inc": 13, 2024-09-19T05:15:39.270 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_sessions": 8, 2024-09-19T05:15:39.270 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_open": 8, 2024-09-19T05:15:39.270 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_close": 0, 2024-09-19T05:15:39.270 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_laggy": 0, 2024-09-19T05:15:39.270 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_wr": 14, 2024-09-19T05:15:39.270 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_rd": 0, 2024-09-19T05:15:39.270 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_del": 1 2024-09-19T05:15:39.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.271 INFO:teuthology.orchestra.run.smithi096.stdout: "oft": { 2024-09-19T05:15:39.271 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_objs": 1, 2024-09-19T05:15:39.271 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_kv_pairs": 0, 2024-09-19T05:15:39.271 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_updates": 0, 2024-09-19T05:15:39.271 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_removes": 0 2024-09-19T05:15:39.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.271 INFO:teuthology.orchestra.run.smithi096.stdout: "osd-slow-ops": { 2024-09-19T05:15:39.271 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_ops_count": 0 2024-09-19T05:15:39.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.271 INFO:teuthology.orchestra.run.smithi096.stdout: "purge_queue": { 2024-09-19T05:15:39.271 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops": 0, 2024-09-19T05:15:39.271 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T05:15:39.271 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing": 0, 2024-09-19T05:15:39.271 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_high_water": 0, 2024-09-19T05:15:39.271 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed_ops": 0, 2024-09-19T05:15:39.272 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed": 0, 2024-09-19T05:15:39.272 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_item_in_journal": 0 2024-09-19T05:15:39.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.272 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T05:15:39.272 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-19T05:15:39.272 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-09-19T05:15:39.272 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-19T05:15:39.272 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 749, 2024-09-19T05:15:39.272 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 1684765, 2024-09-19T05:15:39.272 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:39.272 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 749, 2024-09-19T05:15:39.272 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-19T05:15:39.272 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-19T05:15:39.272 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 749, 2024-09-19T05:15:39.273 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 1684765, 2024-09-19T05:15:39.273 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-19T05:15:39.273 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:39.273 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:39.273 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:39.273 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T05:15:39.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.273 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_bytes": { 2024-09-19T05:15:39.273 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-19T05:15:39.273 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-09-19T05:15:39.273 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-19T05:15:39.273 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-19T05:15:39.273 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-19T05:15:39.273 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:39.274 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-19T05:15:39.274 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 47, 2024-09-19T05:15:39.274 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 13376, 2024-09-19T05:15:39.274 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 47, 2024-09-19T05:15:39.274 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 13376, 2024-09-19T05:15:39.274 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-19T05:15:39.274 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:39.274 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:39.274 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:39.274 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T05:15:39.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.274 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_ops": { 2024-09-19T05:15:39.274 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-19T05:15:39.274 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 1024, 2024-09-19T05:15:39.275 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-19T05:15:39.275 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-19T05:15:39.275 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-19T05:15:39.275 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:39.275 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-19T05:15:39.275 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 47, 2024-09-19T05:15:39.275 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 47, 2024-09-19T05:15:39.275 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 47, 2024-09-19T05:15:39.275 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 47, 2024-09-19T05:15:39.275 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-19T05:15:39.275 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:39.275 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:39.275 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:39.275 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T05:15:39.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.276 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle": { 2024-09-19T05:15:39.276 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-19T05:15:39.276 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-09-19T05:15:39.276 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-19T05:15:39.276 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-19T05:15:39.276 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-19T05:15:39.276 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:39.276 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-19T05:15:39.276 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-19T05:15:39.276 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-19T05:15:39.276 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 0, 2024-09-19T05:15:39.276 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 0, 2024-09-19T05:15:39.277 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-19T05:15:39.277 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:39.277 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:39.277 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:39.277 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T05:15:39.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:39.277 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle-0x5603bc1f2960": { 2024-09-19T05:15:39.277 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-19T05:15:39.277 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-09-19T05:15:39.277 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-19T05:15:39.277 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 10, 2024-09-19T05:15:39.277 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 2894, 2024-09-19T05:15:39.277 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:39.277 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 10, 2024-09-19T05:15:39.278 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-19T05:15:39.278 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-19T05:15:39.278 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 9, 2024-09-19T05:15:39.278 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 2894, 2024-09-19T05:15:39.278 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-19T05:15:39.278 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:39.278 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:39.278 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:39.278 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T05:15:39.278 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T05:15:39.278 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-19T05:15:40.014 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-19T05:15:40.014 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-19T05:15:40.014 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-09-19T05:15:40.015 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-19T05:15:40.015 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T05:15:40.015 DEBUG:tasks.check_counter:Getting stats from k 2024-09-19T05:15:40.015 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-19T05:15:40.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:39 smithi096 ceph-mon[35085]: pgmap v343: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:15:40.261 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T05:15:40.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:39 smithi043 ceph-mon[30800]: pgmap v343: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:15:40.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:39 smithi151 ceph-mon[35425]: pgmap v343: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:15:40.868 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-19T05:15:40.868 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T05:15:40.868 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 795, 2024-09-19T05:15:40.868 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 794, 2024-09-19T05:15:40.869 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 213548, 2024-09-19T05:15:40.869 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 251732, 2024-09-19T05:15:40.869 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 6, 2024-09-19T05:15:40.869 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 2, 2024-09-19T05:15:40.870 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.073147813, 2024-09-19T05:15:40.870 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.030021328, 2024-09-19T05:15:40.870 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.018360339, 2024-09-19T05:15:40.870 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.017204640, 2024-09-19T05:15:40.870 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:15:40.871 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 793, 2024-09-19T05:15:40.871 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.037367659, 2024-09-19T05:15:40.871 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000047121 2024-09-19T05:15:40.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.872 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:15:40.872 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:40.872 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:40.872 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:40.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.873 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-19T05:15:40.873 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T05:15:40.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.873 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T05:15:40.874 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 950, 2024-09-19T05:15:40.874 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 937, 2024-09-19T05:15:40.874 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 1798618, 2024-09-19T05:15:40.874 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 362121, 2024-09-19T05:15:40.874 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 4, 2024-09-19T05:15:40.875 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 3, 2024-09-19T05:15:40.875 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.107227349, 2024-09-19T05:15:40.875 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.036011978, 2024-09-19T05:15:40.875 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.056832229, 2024-09-19T05:15:40.875 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.022686928, 2024-09-19T05:15:40.876 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:15:40.876 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 936, 2024-09-19T05:15:40.876 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.040697720, 2024-09-19T05:15:40.876 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000043480 2024-09-19T05:15:40.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.876 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:15:40.877 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:40.877 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:40.877 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:40.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.877 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 1627872, 2024-09-19T05:15:40.877 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 141424 2024-09-19T05:15:40.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.878 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T05:15:40.878 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 782, 2024-09-19T05:15:40.878 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 952, 2024-09-19T05:15:40.878 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 262400, 2024-09-19T05:15:40.878 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 704584, 2024-09-19T05:15:40.878 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 6, 2024-09-19T05:15:40.879 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 4, 2024-09-19T05:15:40.879 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.103530818, 2024-09-19T05:15:40.879 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.035121957, 2024-09-19T05:15:40.879 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.027032962, 2024-09-19T05:15:40.879 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.030530114, 2024-09-19T05:15:40.879 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:15:40.879 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 949, 2024-09-19T05:15:40.880 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.059553447, 2024-09-19T05:15:40.880 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000062753 2024-09-19T05:15:40.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.880 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:15:40.880 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:40.881 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:40.881 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:40.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.881 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T05:15:40.881 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 456544 2024-09-19T05:15:40.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.882 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-09-19T05:15:40.882 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 1, 2024-09-19T05:15:40.882 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-09-19T05:15:40.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.882 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-MDSRank": { 2024-09-19T05:15:40.883 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-09-19T05:15:40.883 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-09-19T05:15:40.883 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 3785, 2024-09-19T05:15:40.883 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.109323039, 2024-09-19T05:15:40.883 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000028883 2024-09-19T05:15:40.883 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T05:15:40.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.884 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-PurgeQueue": { 2024-09-19T05:15:40.884 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-09-19T05:15:40.884 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-09-19T05:15:40.884 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:40.885 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:40.885 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:40.885 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T05:15:40.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.885 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": { 2024-09-19T05:15:40.885 INFO:teuthology.orchestra.run.smithi096.stdout: "request": 0, 2024-09-19T05:15:40.886 INFO:teuthology.orchestra.run.smithi096.stdout: "reply": 0, 2024-09-19T05:15:40.886 INFO:teuthology.orchestra.run.smithi096.stdout: "reply_latency": { 2024-09-19T05:15:40.886 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:40.886 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:40.886 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:40.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.887 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_reply": 0, 2024-09-19T05:15:40.887 INFO:teuthology.orchestra.run.smithi096.stdout: "forward": 0, 2024-09-19T05:15:40.887 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_complete": 0, 2024-09-19T05:15:40.887 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_keys": 0, 2024-09-19T05:15:40.887 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_commit": 0, 2024-09-19T05:15:40.887 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_split": 0, 2024-09-19T05:15:40.888 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_merge": 0, 2024-09-19T05:15:40.888 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes": 0, 2024-09-19T05:15:40.888 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_top": 0, 2024-09-19T05:15:40.888 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_bottom": 0, 2024-09-19T05:15:40.888 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pin_tail": 0, 2024-09-19T05:15:40.888 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pinned": 0, 2024-09-19T05:15:40.889 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_expired": 0, 2024-09-19T05:15:40.889 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_with_caps": 0, 2024-09-19T05:15:40.889 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": 0, 2024-09-19T05:15:40.889 INFO:teuthology.orchestra.run.smithi096.stdout: "subtrees": 1, 2024-09-19T05:15:40.889 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse": 0, 2024-09-19T05:15:40.890 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_hit": 0, 2024-09-19T05:15:40.890 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_forward": 0, 2024-09-19T05:15:40.890 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_discover": 0, 2024-09-19T05:15:40.890 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_dir_fetch": 0, 2024-09-19T05:15:40.890 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_remote_ino": 0, 2024-09-19T05:15:40.890 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_lock": 0, 2024-09-19T05:15:40.891 INFO:teuthology.orchestra.run.smithi096.stdout: "load_cent": 0, 2024-09-19T05:15:40.891 INFO:teuthology.orchestra.run.smithi096.stdout: "q": 0, 2024-09-19T05:15:40.891 INFO:teuthology.orchestra.run.smithi096.stdout: "exported": 0, 2024-09-19T05:15:40.891 INFO:teuthology.orchestra.run.smithi096.stdout: "exported_inodes": 0, 2024-09-19T05:15:40.891 INFO:teuthology.orchestra.run.smithi096.stdout: "imported": 0, 2024-09-19T05:15:40.891 INFO:teuthology.orchestra.run.smithi096.stdout: "imported_inodes": 0, 2024-09-19T05:15:40.892 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_dir_fetch": 0, 2024-09-19T05:15:40.892 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_backtrace_fetch": 0, 2024-09-19T05:15:40.892 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_peer_discover": 0, 2024-09-19T05:15:40.892 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rfiles": 0, 2024-09-19T05:15:40.892 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rbytes": 0, 2024-09-19T05:15:40.893 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rsnaps": 0, 2024-09-19T05:15:40.893 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T05:15:40.893 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_set_tag": 0, 2024-09-19T05:15:40.893 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T05:15:40.893 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_inotable_repaired": 0, 2024-09-19T05:15:40.893 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_inodes": 0, 2024-09-19T05:15:40.893 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T05:15:40.894 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T05:15:40.894 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_file_inodes": 0, 2024-09-19T05:15:40.894 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_inode_file_caps": 0, 2024-09-19T05:15:40.894 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T05:15:40.894 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_grant": 0, 2024-09-19T05:15:40.894 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T05:15:40.895 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T05:15:40.895 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T05:15:40.895 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps": 0, 2024-09-19T05:15:40.895 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps_dirty": 0, 2024-09-19T05:15:40.895 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_cap_release": 0, 2024-09-19T05:15:40.895 INFO:teuthology.orchestra.run.smithi096.stdout: "process_request_cap_release": 0 2024-09-19T05:15:40.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.896 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache": { 2024-09-19T05:15:40.896 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update": 0, 2024-09-19T05:15:40.896 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update_receipt": 0, 2024-09-19T05:15:40.896 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_try_discover": 0, 2024-09-19T05:15:40.896 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_send_discover": 0, 2024-09-19T05:15:40.897 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_handle_discover": 0, 2024-09-19T05:15:40.897 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays": 0, 2024-09-19T05:15:40.897 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_delayed": 0, 2024-09-19T05:15:40.897 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_enqueuing": 0, 2024-09-19T05:15:40.897 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_created": 0, 2024-09-19T05:15:40.897 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_enqueued": 0, 2024-09-19T05:15:40.898 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_reintegrated": 0, 2024-09-19T05:15:40.898 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_migrated": 0, 2024-09-19T05:15:40.898 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_processing": 0, 2024-09-19T05:15:40.898 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_enqueued": 0, 2024-09-19T05:15:40.898 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_prioritized": 0, 2024-09-19T05:15:40.898 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_started": 0, 2024-09-19T05:15:40.899 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_completed": 0, 2024-09-19T05:15:40.899 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_path": 0, 2024-09-19T05:15:40.899 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_inode": 0, 2024-09-19T05:15:40.899 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T05:15:40.899 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_exportdir": 0, 2024-09-19T05:15:40.899 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_flush": 0, 2024-09-19T05:15:40.900 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragmentdir": 0, 2024-09-19T05:15:40.900 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragstats": 0, 2024-09-19T05:15:40.900 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_inodestats": 0 2024-09-19T05:15:40.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.900 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log": { 2024-09-19T05:15:40.900 INFO:teuthology.orchestra.run.smithi096.stdout: "evlrg": 0, 2024-09-19T05:15:40.901 INFO:teuthology.orchestra.run.smithi096.stdout: "evadd": 0, 2024-09-19T05:15:40.901 INFO:teuthology.orchestra.run.smithi096.stdout: "evex": 0, 2024-09-19T05:15:40.901 INFO:teuthology.orchestra.run.smithi096.stdout: "evtrm": 0, 2024-09-19T05:15:40.901 INFO:teuthology.orchestra.run.smithi096.stdout: "ev": 9, 2024-09-19T05:15:40.901 INFO:teuthology.orchestra.run.smithi096.stdout: "evexg": 0, 2024-09-19T05:15:40.901 INFO:teuthology.orchestra.run.smithi096.stdout: "evexd": 0, 2024-09-19T05:15:40.902 INFO:teuthology.orchestra.run.smithi096.stdout: "segadd": 0, 2024-09-19T05:15:40.902 INFO:teuthology.orchestra.run.smithi096.stdout: "segex": 0, 2024-09-19T05:15:40.902 INFO:teuthology.orchestra.run.smithi096.stdout: "segtrm": 0, 2024-09-19T05:15:40.902 INFO:teuthology.orchestra.run.smithi096.stdout: "seg": 2, 2024-09-19T05:15:40.902 INFO:teuthology.orchestra.run.smithi096.stdout: "segmjr": 2, 2024-09-19T05:15:40.902 INFO:teuthology.orchestra.run.smithi096.stdout: "segexg": 0, 2024-09-19T05:15:40.902 INFO:teuthology.orchestra.run.smithi096.stdout: "segexd": 0, 2024-09-19T05:15:40.903 INFO:teuthology.orchestra.run.smithi096.stdout: "expos": 4194304, 2024-09-19T05:15:40.903 INFO:teuthology.orchestra.run.smithi096.stdout: "wrpos": 4196887, 2024-09-19T05:15:40.903 INFO:teuthology.orchestra.run.smithi096.stdout: "rdpos": 4196746, 2024-09-19T05:15:40.903 INFO:teuthology.orchestra.run.smithi096.stdout: "jlat": { 2024-09-19T05:15:40.903 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:40.903 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:40.904 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:40.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.904 INFO:teuthology.orchestra.run.smithi096.stdout: "replayed": 9 2024-09-19T05:15:40.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.904 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_mem": { 2024-09-19T05:15:40.904 INFO:teuthology.orchestra.run.smithi096.stdout: "ino": 3, 2024-09-19T05:15:40.905 INFO:teuthology.orchestra.run.smithi096.stdout: "ino+": 3, 2024-09-19T05:15:40.905 INFO:teuthology.orchestra.run.smithi096.stdout: "ino-": 0, 2024-09-19T05:15:40.905 INFO:teuthology.orchestra.run.smithi096.stdout: "dir": 1, 2024-09-19T05:15:40.905 INFO:teuthology.orchestra.run.smithi096.stdout: "dir+": 1, 2024-09-19T05:15:40.905 INFO:teuthology.orchestra.run.smithi096.stdout: "dir-": 0, 2024-09-19T05:15:40.905 INFO:teuthology.orchestra.run.smithi096.stdout: "dn": 0, 2024-09-19T05:15:40.906 INFO:teuthology.orchestra.run.smithi096.stdout: "dn+": 0, 2024-09-19T05:15:40.906 INFO:teuthology.orchestra.run.smithi096.stdout: "dn-": 0, 2024-09-19T05:15:40.906 INFO:teuthology.orchestra.run.smithi096.stdout: "cap": 0, 2024-09-19T05:15:40.906 INFO:teuthology.orchestra.run.smithi096.stdout: "cap+": 0, 2024-09-19T05:15:40.906 INFO:teuthology.orchestra.run.smithi096.stdout: "cap-": 0, 2024-09-19T05:15:40.906 INFO:teuthology.orchestra.run.smithi096.stdout: "rss": 48628, 2024-09-19T05:15:40.907 INFO:teuthology.orchestra.run.smithi096.stdout: "heap": 215328 2024-09-19T05:15:40.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.907 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_server": { 2024-09-19T05:15:40.907 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_client_request": 0, 2024-09-19T05:15:40.907 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_server_request": 0, 2024-09-19T05:15:40.907 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_request": 0, 2024-09-19T05:15:40.908 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_session": 0, 2024-09-19T05:15:40.908 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_peer_request": 0, 2024-09-19T05:15:40.908 INFO:teuthology.orchestra.run.smithi096.stdout: "req_create_latency": { 2024-09-19T05:15:40.908 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:40.908 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:40.908 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:40.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.909 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getattr_latency": { 2024-09-19T05:15:40.909 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:40.909 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:40.909 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:40.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.910 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getfilelock_latency": { 2024-09-19T05:15:40.910 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:40.910 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:40.910 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:40.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.910 INFO:teuthology.orchestra.run.smithi096.stdout: "req_link_latency": { 2024-09-19T05:15:40.911 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:40.911 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:40.911 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:40.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.911 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookup_latency": { 2024-09-19T05:15:40.911 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:40.912 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:40.912 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:40.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.912 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookuphash_latency": { 2024-09-19T05:15:40.912 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:40.912 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:40.912 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:40.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.913 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupino_latency": { 2024-09-19T05:15:40.913 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:40.913 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:40.913 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:40.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.914 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupname_latency": { 2024-09-19T05:15:40.914 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:40.914 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:40.914 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:40.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.914 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupparent_latency": { 2024-09-19T05:15:40.915 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:40.915 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:40.915 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:40.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.915 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupsnap_latency": { 2024-09-19T05:15:40.915 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:40.916 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:40.916 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:40.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.916 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lssnap_latency": { 2024-09-19T05:15:40.916 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:40.916 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:40.917 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:40.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.917 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mkdir_latency": { 2024-09-19T05:15:40.917 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:40.918 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:40.918 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:40.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.918 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mknod_latency": { 2024-09-19T05:15:40.918 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:40.918 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:40.918 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:40.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.919 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mksnap_latency": { 2024-09-19T05:15:40.919 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:40.919 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:40.919 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:40.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.920 INFO:teuthology.orchestra.run.smithi096.stdout: "req_open_latency": { 2024-09-19T05:15:40.920 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:40.920 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:40.920 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:40.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.920 INFO:teuthology.orchestra.run.smithi096.stdout: "req_readdir_latency": { 2024-09-19T05:15:40.921 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:40.921 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:40.921 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:40.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.921 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rename_latency": { 2024-09-19T05:15:40.921 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:40.922 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:40.922 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:40.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.922 INFO:teuthology.orchestra.run.smithi096.stdout: "req_renamesnap_latency": { 2024-09-19T05:15:40.922 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:40.922 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:40.922 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:40.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.923 INFO:teuthology.orchestra.run.smithi096.stdout: "req_snapdiff_latency": { 2024-09-19T05:15:40.923 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:40.923 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:40.923 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:40.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.924 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmdir_latency": { 2024-09-19T05:15:40.924 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:40.924 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:40.925 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:40.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.925 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmsnap_latency": { 2024-09-19T05:15:40.925 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:40.925 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:40.926 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:40.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.926 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmxattr_latency": { 2024-09-19T05:15:40.926 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:40.927 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:40.927 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:40.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.927 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setattr_latency": { 2024-09-19T05:15:40.927 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:40.928 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:40.928 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:40.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.928 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setdirlayout_latency": { 2024-09-19T05:15:40.928 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:40.929 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:40.929 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:40.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.929 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setfilelock_latency": { 2024-09-19T05:15:40.930 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:40.930 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:40.930 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:40.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.930 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setlayout_latency": { 2024-09-19T05:15:40.931 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:40.931 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:40.931 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:40.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.931 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setxattr_latency": { 2024-09-19T05:15:40.932 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:40.932 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:40.932 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:40.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.932 INFO:teuthology.orchestra.run.smithi096.stdout: "req_symlink_latency": { 2024-09-19T05:15:40.933 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:40.933 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:40.933 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:40.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.933 INFO:teuthology.orchestra.run.smithi096.stdout: "req_unlink_latency": { 2024-09-19T05:15:40.933 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:40.933 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:40.933 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:40.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.934 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_revoke_eviction": 0, 2024-09-19T05:15:40.934 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_acquisition_throttle": 0, 2024-09-19T05:15:40.934 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getvxattr_latency": { 2024-09-19T05:15:40.934 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:40.934 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:40.934 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:40.934 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T05:15:40.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.936 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_sessions": { 2024-09-19T05:15:40.937 INFO:teuthology.orchestra.run.smithi096.stdout: "session_count": 1, 2024-09-19T05:15:40.937 INFO:teuthology.orchestra.run.smithi096.stdout: "session_add": 4, 2024-09-19T05:15:40.937 INFO:teuthology.orchestra.run.smithi096.stdout: "session_remove": 3, 2024-09-19T05:15:40.937 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_open": 3, 2024-09-19T05:15:40.937 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_stale": 0, 2024-09-19T05:15:40.937 INFO:teuthology.orchestra.run.smithi096.stdout: "total_load": 0, 2024-09-19T05:15:40.937 INFO:teuthology.orchestra.run.smithi096.stdout: "average_load": 0, 2024-09-19T05:15:40.937 INFO:teuthology.orchestra.run.smithi096.stdout: "avg_session_uptime": 4794, 2024-09-19T05:15:40.937 INFO:teuthology.orchestra.run.smithi096.stdout: "mdthresh_evicted": 0 2024-09-19T05:15:40.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.937 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-09-19T05:15:40.937 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 0, 2024-09-19T05:15:40.937 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 0, 2024-09-19T05:15:40.938 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T05:15:40.938 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-09-19T05:15:40.938 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T05:15:40.938 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-09-19T05:15:40.938 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T05:15:40.938 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T05:15:40.938 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T05:15:40.938 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T05:15:40.938 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T05:15:40.938 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-09-19T05:15:40.938 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T05:15:40.938 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T05:15:40.938 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-09-19T05:15:40.938 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-09-19T05:15:40.939 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-09-19T05:15:40.939 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-09-19T05:15:40.939 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T05:15:40.939 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T05:15:40.939 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T05:15:40.939 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T05:15:40.939 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T05:15:40.939 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-09-19T05:15:40.939 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-09-19T05:15:40.939 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-09-19T05:15:40.940 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T05:15:40.940 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T05:15:40.940 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-09-19T05:15:40.940 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-09-19T05:15:40.940 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-09-19T05:15:40.940 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-09-19T05:15:40.940 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T05:15:40.940 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-09-19T05:15:40.940 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T05:15:40.940 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-09-19T05:15:40.940 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 189433, 2024-09-19T05:15:40.940 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 52, 2024-09-19T05:15:40.940 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 0, 2024-09-19T05:15:40.941 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 0, 2024-09-19T05:15:40.941 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-09-19T05:15:40.941 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-09-19T05:15:40.941 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-09-19T05:15:40.941 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-09-19T05:15:40.941 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-09-19T05:15:40.941 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-09-19T05:15:40.941 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 8224, 2024-09-19T05:15:40.941 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 181, 2024-09-19T05:15:40.941 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T05:15:40.941 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-09-19T05:15:40.941 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-09-19T05:15:40.941 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-09-19T05:15:40.941 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 8512, 2024-09-19T05:15:40.941 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 13, 2024-09-19T05:15:40.942 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-09-19T05:15:40.942 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-09-19T05:15:40.942 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-09-19T05:15:40.942 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-09-19T05:15:40.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.942 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter": { 2024-09-19T05:15:40.942 INFO:teuthology.orchestra.run.smithi096.stdout: "op_active": 0, 2024-09-19T05:15:40.942 INFO:teuthology.orchestra.run.smithi096.stdout: "op_laggy": 0, 2024-09-19T05:15:40.942 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send": 2264, 2024-09-19T05:15:40.942 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send_bytes": 21, 2024-09-19T05:15:40.942 INFO:teuthology.orchestra.run.smithi096.stdout: "op_resend": 3, 2024-09-19T05:15:40.942 INFO:teuthology.orchestra.run.smithi096.stdout: "op_reply": 2264, 2024-09-19T05:15:40.942 INFO:teuthology.orchestra.run.smithi096.stdout: "op_latency": { 2024-09-19T05:15:40.942 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 2264, 2024-09-19T05:15:40.943 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 1.874956000, 2024-09-19T05:15:40.943 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000828160 2024-09-19T05:15:40.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.943 INFO:teuthology.orchestra.run.smithi096.stdout: "op_inflight": 0, 2024-09-19T05:15:40.943 INFO:teuthology.orchestra.run.smithi096.stdout: "oplen_avg": { 2024-09-19T05:15:40.943 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 2264, 2024-09-19T05:15:40.943 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 2265 2024-09-19T05:15:40.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.943 INFO:teuthology.orchestra.run.smithi096.stdout: "op": 2264, 2024-09-19T05:15:40.943 INFO:teuthology.orchestra.run.smithi096.stdout: "op_r": 2264, 2024-09-19T05:15:40.943 INFO:teuthology.orchestra.run.smithi096.stdout: "op_w": 0, 2024-09-19T05:15:40.943 INFO:teuthology.orchestra.run.smithi096.stdout: "op_rmw": 0, 2024-09-19T05:15:40.943 INFO:teuthology.orchestra.run.smithi096.stdout: "op_pg": 0, 2024-09-19T05:15:40.943 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_stat": 1472, 2024-09-19T05:15:40.943 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_create": 0, 2024-09-19T05:15:40.944 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_read": 790, 2024-09-19T05:15:40.944 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_write": 0, 2024-09-19T05:15:40.944 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writefull": 0, 2024-09-19T05:15:40.944 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writesame": 0, 2024-09-19T05:15:40.944 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_append": 0, 2024-09-19T05:15:40.944 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_zero": 0, 2024-09-19T05:15:40.944 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_truncate": 0, 2024-09-19T05:15:40.944 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_delete": 0, 2024-09-19T05:15:40.944 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_mapext": 0, 2024-09-19T05:15:40.944 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_sparse_read": 0, 2024-09-19T05:15:40.944 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_clonerange": 0, 2024-09-19T05:15:40.944 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_getxattr": 1, 2024-09-19T05:15:40.944 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_setxattr": 0, 2024-09-19T05:15:40.944 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_cmpxattr": 0, 2024-09-19T05:15:40.945 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_rmxattr": 0, 2024-09-19T05:15:40.945 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_resetxattrs": 0, 2024-09-19T05:15:40.945 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_call": 0, 2024-09-19T05:15:40.945 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_watch": 0, 2024-09-19T05:15:40.945 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_notify": 0, 2024-09-19T05:15:40.945 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T05:15:40.945 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls": 0, 2024-09-19T05:15:40.945 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls_filter": 0, 2024-09-19T05:15:40.945 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_other": 0, 2024-09-19T05:15:40.945 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_active": 0, 2024-09-19T05:15:40.945 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_send": 0, 2024-09-19T05:15:40.945 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_resend": 0, 2024-09-19T05:15:40.945 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_ping": 0, 2024-09-19T05:15:40.945 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_active": 0, 2024-09-19T05:15:40.945 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_send": 0, 2024-09-19T05:15:40.946 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_resend": 0, 2024-09-19T05:15:40.946 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_active": 0, 2024-09-19T05:15:40.946 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_send": 0, 2024-09-19T05:15:40.946 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_resend": 0, 2024-09-19T05:15:40.946 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_active": 0, 2024-09-19T05:15:40.946 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_send": 0, 2024-09-19T05:15:40.946 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_resend": 0, 2024-09-19T05:15:40.946 INFO:teuthology.orchestra.run.smithi096.stdout: "command_active": 0, 2024-09-19T05:15:40.946 INFO:teuthology.orchestra.run.smithi096.stdout: "command_send": 0, 2024-09-19T05:15:40.946 INFO:teuthology.orchestra.run.smithi096.stdout: "command_resend": 0, 2024-09-19T05:15:40.946 INFO:teuthology.orchestra.run.smithi096.stdout: "map_epoch": 96, 2024-09-19T05:15:40.946 INFO:teuthology.orchestra.run.smithi096.stdout: "map_full": 0, 2024-09-19T05:15:40.946 INFO:teuthology.orchestra.run.smithi096.stdout: "map_inc": 13, 2024-09-19T05:15:40.946 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_sessions": 5, 2024-09-19T05:15:40.947 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_open": 5, 2024-09-19T05:15:40.947 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_close": 0, 2024-09-19T05:15:40.947 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_laggy": 0, 2024-09-19T05:15:40.947 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_wr": 0, 2024-09-19T05:15:40.947 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_rd": 2, 2024-09-19T05:15:40.947 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_del": 0 2024-09-19T05:15:40.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.947 INFO:teuthology.orchestra.run.smithi096.stdout: "oft": { 2024-09-19T05:15:40.947 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_objs": 0, 2024-09-19T05:15:40.947 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_kv_pairs": 0, 2024-09-19T05:15:40.947 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_updates": 0, 2024-09-19T05:15:40.947 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_removes": 0 2024-09-19T05:15:40.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.947 INFO:teuthology.orchestra.run.smithi096.stdout: "osd-slow-ops": { 2024-09-19T05:15:40.947 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_ops_count": 0 2024-09-19T05:15:40.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.948 INFO:teuthology.orchestra.run.smithi096.stdout: "purge_queue": { 2024-09-19T05:15:40.948 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops": 0, 2024-09-19T05:15:40.948 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T05:15:40.948 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing": 0, 2024-09-19T05:15:40.948 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_high_water": 0, 2024-09-19T05:15:40.948 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed_ops": 0, 2024-09-19T05:15:40.948 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed": 0, 2024-09-19T05:15:40.948 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_item_in_journal": 0 2024-09-19T05:15:40.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.948 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T05:15:40.948 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-19T05:15:40.948 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-09-19T05:15:40.948 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-19T05:15:40.949 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 2523, 2024-09-19T05:15:40.949 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 2031338, 2024-09-19T05:15:40.949 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:40.949 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 2523, 2024-09-19T05:15:40.949 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-19T05:15:40.949 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-19T05:15:40.949 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 2523, 2024-09-19T05:15:40.949 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 2031338, 2024-09-19T05:15:40.949 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-19T05:15:40.949 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:40.949 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:40.949 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:40.949 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T05:15:40.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.950 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_bytes": { 2024-09-19T05:15:40.950 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-19T05:15:40.950 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-09-19T05:15:40.950 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-19T05:15:40.950 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-19T05:15:40.950 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-19T05:15:40.950 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:40.950 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-19T05:15:40.950 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 2264, 2024-09-19T05:15:40.950 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 2588, 2024-09-19T05:15:40.950 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 2264, 2024-09-19T05:15:40.950 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 2588, 2024-09-19T05:15:40.950 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-19T05:15:40.950 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:40.951 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:40.951 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:40.951 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T05:15:40.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.951 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_ops": { 2024-09-19T05:15:40.951 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-19T05:15:40.951 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 1024, 2024-09-19T05:15:40.951 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-19T05:15:40.951 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-19T05:15:40.951 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-19T05:15:40.951 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:40.951 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-19T05:15:40.951 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 2264, 2024-09-19T05:15:40.951 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 2264, 2024-09-19T05:15:40.951 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 2264, 2024-09-19T05:15:40.952 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 2264, 2024-09-19T05:15:40.952 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-19T05:15:40.952 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:40.952 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:40.952 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:40.952 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T05:15:40.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.952 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle": { 2024-09-19T05:15:40.952 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-19T05:15:40.952 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-09-19T05:15:40.952 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-19T05:15:40.952 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-19T05:15:40.952 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-19T05:15:40.952 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:40.953 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-19T05:15:40.953 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-19T05:15:40.953 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-19T05:15:40.953 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 0, 2024-09-19T05:15:40.953 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 0, 2024-09-19T05:15:40.953 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-19T05:15:40.953 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:40.953 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:40.953 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:40.953 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T05:15:40.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T05:15:40.953 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle-0x55ddcba02b40": { 2024-09-19T05:15:40.953 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-19T05:15:40.953 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-09-19T05:15:40.954 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-19T05:15:40.954 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-19T05:15:40.954 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-19T05:15:40.954 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:40.954 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-19T05:15:40.954 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-19T05:15:40.954 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-19T05:15:40.954 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 0, 2024-09-19T05:15:40.954 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 0, 2024-09-19T05:15:40.954 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-19T05:15:40.954 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T05:15:40.954 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T05:15:40.954 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T05:15:40.954 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T05:15:40.954 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T05:15:40.955 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-19T05:15:41.735 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-19T05:15:41.736 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-19T05:15:41.736 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-09-19T05:15:41.736 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-09-19T05:15:41.736 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T05:15:41.736 DEBUG:tasks.check_counter:Getting stats from c 2024-09-19T05:15:41.736 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-19T05:15:41.998 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:15:42.118 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:41 smithi151 ceph-mon[35425]: pgmap v344: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:15:42.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:41 smithi096 ceph-mon[35085]: pgmap v344: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:15:42.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:41 smithi043 ceph-mon[30800]: pgmap v344: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:15:42.485 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-19T05:15:42.485 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T05:15:42.485 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 66, 2024-09-19T05:15:42.485 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 71, 2024-09-19T05:15:42.485 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 14759, 2024-09-19T05:15:42.485 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 450486, 2024-09-19T05:15:42.485 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 47, 2024-09-19T05:15:42.485 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 4, 2024-09-19T05:15:42.486 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.023684138, 2024-09-19T05:15:42.486 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.001598200, 2024-09-19T05:15:42.486 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.016576411, 2024-09-19T05:15:42.486 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.000551602, 2024-09-19T05:15:42.486 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:15:42.486 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 67, 2024-09-19T05:15:42.486 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.015946563, 2024-09-19T05:15:42.486 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000238008 2024-09-19T05:15:42.486 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.486 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:15:42.486 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 52, 2024-09-19T05:15:42.487 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000005768, 2024-09-19T05:15:42.487 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000110 2024-09-19T05:15:42.487 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.487 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 4880, 2024-09-19T05:15:42.487 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 2400 2024-09-19T05:15:42.487 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.487 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T05:15:42.487 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 351, 2024-09-19T05:15:42.488 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 673, 2024-09-19T05:15:42.488 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 97660, 2024-09-19T05:15:42.488 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 578931, 2024-09-19T05:15:42.488 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 8, 2024-09-19T05:15:42.488 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 4, 2024-09-19T05:15:42.489 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.059665075, 2024-09-19T05:15:42.489 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.026815999, 2024-09-19T05:15:42.489 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.023292058, 2024-09-19T05:15:42.489 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.005062641, 2024-09-19T05:15:42.489 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:15:42.489 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 673, 2024-09-19T05:15:42.490 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.053121471, 2024-09-19T05:15:42.490 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000078932 2024-09-19T05:15:42.490 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.490 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:15:42.490 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 834, 2024-09-19T05:15:42.490 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000212180, 2024-09-19T05:15:42.490 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000254 2024-09-19T05:15:42.490 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.490 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T05:15:42.491 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 456016 2024-09-19T05:15:42.491 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.491 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T05:15:42.491 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 303, 2024-09-19T05:15:42.491 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 288, 2024-09-19T05:15:42.491 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 1644000, 2024-09-19T05:15:42.491 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 200578, 2024-09-19T05:15:42.491 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 9, 2024-09-19T05:15:42.491 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 4, 2024-09-19T05:15:42.491 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.049243519, 2024-09-19T05:15:42.491 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.017618239, 2024-09-19T05:15:42.492 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.045427241, 2024-09-19T05:15:42.492 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.000325484, 2024-09-19T05:15:42.492 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:15:42.492 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 287, 2024-09-19T05:15:42.492 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.030659959, 2024-09-19T05:15:42.492 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000106829 2024-09-19T05:15:42.492 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.492 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:15:42.492 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 4, 2024-09-19T05:15:42.492 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000526, 2024-09-19T05:15:42.492 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000131 2024-09-19T05:15:42.492 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.493 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 1639344, 2024-09-19T05:15:42.493 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 144144 2024-09-19T05:15:42.493 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.493 INFO:teuthology.orchestra.run.smithi151.stdout: "cct": { 2024-09-19T05:15:42.493 INFO:teuthology.orchestra.run.smithi151.stdout: "total_workers": 1, 2024-09-19T05:15:42.493 INFO:teuthology.orchestra.run.smithi151.stdout: "unhealthy_workers": 0 2024-09-19T05:15:42.493 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.493 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-MDSRank": { 2024-09-19T05:15:42.493 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-09-19T05:15:42.493 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-09-19T05:15:42.494 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 47, 2024-09-19T05:15:42.494 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.002245721, 2024-09-19T05:15:42.494 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000047781 2024-09-19T05:15:42.494 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T05:15:42.494 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.494 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-PurgeQueue": { 2024-09-19T05:15:42.494 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-09-19T05:15:42.494 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-09-19T05:15:42.494 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 2, 2024-09-19T05:15:42.494 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000012074, 2024-09-19T05:15:42.494 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000006037 2024-09-19T05:15:42.494 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T05:15:42.494 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.494 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": { 2024-09-19T05:15:42.495 INFO:teuthology.orchestra.run.smithi151.stdout: "request": 0, 2024-09-19T05:15:42.495 INFO:teuthology.orchestra.run.smithi151.stdout: "reply": 0, 2024-09-19T05:15:42.495 INFO:teuthology.orchestra.run.smithi151.stdout: "reply_latency": { 2024-09-19T05:15:42.495 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:42.496 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:42.496 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:42.496 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.496 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_reply": 0, 2024-09-19T05:15:42.496 INFO:teuthology.orchestra.run.smithi151.stdout: "forward": 0, 2024-09-19T05:15:42.496 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_complete": 0, 2024-09-19T05:15:42.496 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_keys": 0, 2024-09-19T05:15:42.496 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_commit": 11, 2024-09-19T05:15:42.496 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_split": 0, 2024-09-19T05:15:42.496 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_merge": 0, 2024-09-19T05:15:42.496 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes": 10, 2024-09-19T05:15:42.496 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_top": 0, 2024-09-19T05:15:42.497 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_bottom": 10, 2024-09-19T05:15:42.497 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pin_tail": 0, 2024-09-19T05:15:42.497 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pinned": 10, 2024-09-19T05:15:42.497 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_expired": 0, 2024-09-19T05:15:42.497 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_with_caps": 0, 2024-09-19T05:15:42.497 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": 0, 2024-09-19T05:15:42.497 INFO:teuthology.orchestra.run.smithi151.stdout: "subtrees": 1, 2024-09-19T05:15:42.497 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse": 0, 2024-09-19T05:15:42.497 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_hit": 0, 2024-09-19T05:15:42.497 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_forward": 0, 2024-09-19T05:15:42.497 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_discover": 0, 2024-09-19T05:15:42.497 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_dir_fetch": 0, 2024-09-19T05:15:42.498 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_remote_ino": 0, 2024-09-19T05:15:42.498 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_lock": 0, 2024-09-19T05:15:42.498 INFO:teuthology.orchestra.run.smithi151.stdout: "load_cent": 0, 2024-09-19T05:15:42.498 INFO:teuthology.orchestra.run.smithi151.stdout: "q": 0, 2024-09-19T05:15:42.498 INFO:teuthology.orchestra.run.smithi151.stdout: "exported": 0, 2024-09-19T05:15:42.498 INFO:teuthology.orchestra.run.smithi151.stdout: "exported_inodes": 0, 2024-09-19T05:15:42.498 INFO:teuthology.orchestra.run.smithi151.stdout: "imported": 0, 2024-09-19T05:15:42.498 INFO:teuthology.orchestra.run.smithi151.stdout: "imported_inodes": 0, 2024-09-19T05:15:42.498 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_dir_fetch": 0, 2024-09-19T05:15:42.498 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_backtrace_fetch": 0, 2024-09-19T05:15:42.498 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_peer_discover": 0, 2024-09-19T05:15:42.498 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rfiles": 4, 2024-09-19T05:15:42.498 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rbytes": 234, 2024-09-19T05:15:42.499 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rsnaps": 6, 2024-09-19T05:15:42.499 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T05:15:42.499 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_set_tag": 0, 2024-09-19T05:15:42.499 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T05:15:42.499 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_inotable_repaired": 0, 2024-09-19T05:15:42.499 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_inodes": 0, 2024-09-19T05:15:42.499 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T05:15:42.499 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T05:15:42.499 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_file_inodes": 0, 2024-09-19T05:15:42.499 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_inode_file_caps": 0, 2024-09-19T05:15:42.499 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T05:15:42.499 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_grant": 0, 2024-09-19T05:15:42.499 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T05:15:42.499 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T05:15:42.500 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T05:15:42.500 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps": 0, 2024-09-19T05:15:42.500 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps_dirty": 0, 2024-09-19T05:15:42.500 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_cap_release": 0, 2024-09-19T05:15:42.500 INFO:teuthology.orchestra.run.smithi151.stdout: "process_request_cap_release": 0 2024-09-19T05:15:42.500 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.500 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_cache": { 2024-09-19T05:15:42.500 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update": 0, 2024-09-19T05:15:42.500 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update_receipt": 0, 2024-09-19T05:15:42.500 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_try_discover": 0, 2024-09-19T05:15:42.501 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_send_discover": 1, 2024-09-19T05:15:42.501 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_handle_discover": 0, 2024-09-19T05:15:42.501 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays": 0, 2024-09-19T05:15:42.501 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_delayed": 0, 2024-09-19T05:15:42.501 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_enqueuing": 0, 2024-09-19T05:15:42.501 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_created": 0, 2024-09-19T05:15:42.501 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_enqueued": 0, 2024-09-19T05:15:42.501 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_reintegrated": 0, 2024-09-19T05:15:42.501 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_migrated": 0, 2024-09-19T05:15:42.501 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_processing": 0, 2024-09-19T05:15:42.501 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_enqueued": 0, 2024-09-19T05:15:42.501 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_prioritized": 0, 2024-09-19T05:15:42.501 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_started": 0, 2024-09-19T05:15:42.502 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_completed": 0, 2024-09-19T05:15:42.502 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_path": 0, 2024-09-19T05:15:42.502 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_inode": 0, 2024-09-19T05:15:42.502 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T05:15:42.502 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_exportdir": 0, 2024-09-19T05:15:42.502 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_flush": 0, 2024-09-19T05:15:42.502 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragmentdir": 0, 2024-09-19T05:15:42.502 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragstats": 0, 2024-09-19T05:15:42.502 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_inodestats": 0 2024-09-19T05:15:42.502 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.502 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_log": { 2024-09-19T05:15:42.502 INFO:teuthology.orchestra.run.smithi151.stdout: "evlrg": 0, 2024-09-19T05:15:42.502 INFO:teuthology.orchestra.run.smithi151.stdout: "evadd": 6, 2024-09-19T05:15:42.502 INFO:teuthology.orchestra.run.smithi151.stdout: "evex": 0, 2024-09-19T05:15:42.503 INFO:teuthology.orchestra.run.smithi151.stdout: "evtrm": 0, 2024-09-19T05:15:42.503 INFO:teuthology.orchestra.run.smithi151.stdout: "ev": 6, 2024-09-19T05:15:42.503 INFO:teuthology.orchestra.run.smithi151.stdout: "evexg": 0, 2024-09-19T05:15:42.503 INFO:teuthology.orchestra.run.smithi151.stdout: "evexd": 0, 2024-09-19T05:15:42.503 INFO:teuthology.orchestra.run.smithi151.stdout: "segadd": 2, 2024-09-19T05:15:42.503 INFO:teuthology.orchestra.run.smithi151.stdout: "segex": 0, 2024-09-19T05:15:42.503 INFO:teuthology.orchestra.run.smithi151.stdout: "segtrm": 0, 2024-09-19T05:15:42.503 INFO:teuthology.orchestra.run.smithi151.stdout: "seg": 2, 2024-09-19T05:15:42.503 INFO:teuthology.orchestra.run.smithi151.stdout: "segmjr": 2, 2024-09-19T05:15:42.503 INFO:teuthology.orchestra.run.smithi151.stdout: "segexg": 0, 2024-09-19T05:15:42.503 INFO:teuthology.orchestra.run.smithi151.stdout: "segexd": 0, 2024-09-19T05:15:42.503 INFO:teuthology.orchestra.run.smithi151.stdout: "expos": 4194304, 2024-09-19T05:15:42.503 INFO:teuthology.orchestra.run.smithi151.stdout: "wrpos": 4195960, 2024-09-19T05:15:42.503 INFO:teuthology.orchestra.run.smithi151.stdout: "rdpos": 0, 2024-09-19T05:15:42.504 INFO:teuthology.orchestra.run.smithi151.stdout: "jlat": { 2024-09-19T05:15:42.504 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 5, 2024-09-19T05:15:42.504 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.011777945, 2024-09-19T05:15:42.504 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.002355589 2024-09-19T05:15:42.504 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.504 INFO:teuthology.orchestra.run.smithi151.stdout: "replayed": 0 2024-09-19T05:15:42.504 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.504 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_mem": { 2024-09-19T05:15:42.504 INFO:teuthology.orchestra.run.smithi151.stdout: "ino": 13, 2024-09-19T05:15:42.504 INFO:teuthology.orchestra.run.smithi151.stdout: "ino+": 13, 2024-09-19T05:15:42.504 INFO:teuthology.orchestra.run.smithi151.stdout: "ino-": 0, 2024-09-19T05:15:42.504 INFO:teuthology.orchestra.run.smithi151.stdout: "dir": 11, 2024-09-19T05:15:42.505 INFO:teuthology.orchestra.run.smithi151.stdout: "dir+": 12, 2024-09-19T05:15:42.505 INFO:teuthology.orchestra.run.smithi151.stdout: "dir-": 1, 2024-09-19T05:15:42.505 INFO:teuthology.orchestra.run.smithi151.stdout: "dn": 10, 2024-09-19T05:15:42.505 INFO:teuthology.orchestra.run.smithi151.stdout: "dn+": 10, 2024-09-19T05:15:42.505 INFO:teuthology.orchestra.run.smithi151.stdout: "dn-": 0, 2024-09-19T05:15:42.505 INFO:teuthology.orchestra.run.smithi151.stdout: "cap": 0, 2024-09-19T05:15:42.505 INFO:teuthology.orchestra.run.smithi151.stdout: "cap+": 0, 2024-09-19T05:15:42.505 INFO:teuthology.orchestra.run.smithi151.stdout: "cap-": 0, 2024-09-19T05:15:42.505 INFO:teuthology.orchestra.run.smithi151.stdout: "rss": 50424, 2024-09-19T05:15:42.505 INFO:teuthology.orchestra.run.smithi151.stdout: "heap": 223520 2024-09-19T05:15:42.505 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.505 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_server": { 2024-09-19T05:15:42.505 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_client_request": 0, 2024-09-19T05:15:42.505 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_server_request": 0, 2024-09-19T05:15:42.506 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_request": 0, 2024-09-19T05:15:42.506 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_session": 8, 2024-09-19T05:15:42.506 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_peer_request": 0, 2024-09-19T05:15:42.506 INFO:teuthology.orchestra.run.smithi151.stdout: "req_create_latency": { 2024-09-19T05:15:42.506 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:42.506 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:42.506 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:42.506 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.506 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getattr_latency": { 2024-09-19T05:15:42.506 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:42.506 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:42.506 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:42.506 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.506 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getfilelock_latency": { 2024-09-19T05:15:42.507 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:42.507 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:42.507 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:42.507 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.507 INFO:teuthology.orchestra.run.smithi151.stdout: "req_link_latency": { 2024-09-19T05:15:42.507 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:42.507 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:42.507 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:42.507 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.507 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookup_latency": { 2024-09-19T05:15:42.507 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:42.507 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:42.507 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:42.507 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.508 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookuphash_latency": { 2024-09-19T05:15:42.508 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:42.508 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:42.508 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:42.508 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.508 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupino_latency": { 2024-09-19T05:15:42.508 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:42.508 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:42.508 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:42.508 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.508 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupname_latency": { 2024-09-19T05:15:42.508 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:42.508 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:42.508 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:42.509 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.509 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupparent_latency": { 2024-09-19T05:15:42.509 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:42.509 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:42.509 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:42.509 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.509 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupsnap_latency": { 2024-09-19T05:15:42.509 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:42.509 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:42.509 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:42.509 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.509 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lssnap_latency": { 2024-09-19T05:15:42.509 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:42.510 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:42.510 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:42.510 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.510 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mkdir_latency": { 2024-09-19T05:15:42.510 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:42.510 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:42.511 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:42.511 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.511 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mknod_latency": { 2024-09-19T05:15:42.511 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:42.511 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:42.511 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:42.511 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.511 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mksnap_latency": { 2024-09-19T05:15:42.511 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:42.511 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:42.511 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:42.512 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.512 INFO:teuthology.orchestra.run.smithi151.stdout: "req_open_latency": { 2024-09-19T05:15:42.512 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:42.512 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:42.512 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:42.512 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.512 INFO:teuthology.orchestra.run.smithi151.stdout: "req_readdir_latency": { 2024-09-19T05:15:42.512 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:42.512 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:42.513 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:42.513 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.513 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rename_latency": { 2024-09-19T05:15:42.513 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:42.513 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:42.513 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:42.513 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.513 INFO:teuthology.orchestra.run.smithi151.stdout: "req_renamesnap_latency": { 2024-09-19T05:15:42.513 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:42.513 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:42.513 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:42.514 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.514 INFO:teuthology.orchestra.run.smithi151.stdout: "req_snapdiff_latency": { 2024-09-19T05:15:42.514 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:42.514 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:42.514 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:42.514 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.514 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmdir_latency": { 2024-09-19T05:15:42.514 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:42.514 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:42.514 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:42.514 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.514 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmsnap_latency": { 2024-09-19T05:15:42.514 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:42.515 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:42.515 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:42.515 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.515 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmxattr_latency": { 2024-09-19T05:15:42.515 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:42.515 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:42.515 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:42.515 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.515 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setattr_latency": { 2024-09-19T05:15:42.515 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:42.515 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:42.516 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:42.516 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.516 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setdirlayout_latency": { 2024-09-19T05:15:42.516 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:42.516 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:42.516 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:42.516 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.516 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setfilelock_latency": { 2024-09-19T05:15:42.516 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:42.516 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:42.516 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:42.517 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.517 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setlayout_latency": { 2024-09-19T05:15:42.517 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:42.517 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:42.517 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:42.517 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.517 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setxattr_latency": { 2024-09-19T05:15:42.517 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:42.517 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:42.517 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:42.517 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.517 INFO:teuthology.orchestra.run.smithi151.stdout: "req_symlink_latency": { 2024-09-19T05:15:42.518 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:42.518 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:42.518 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:42.518 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.518 INFO:teuthology.orchestra.run.smithi151.stdout: "req_unlink_latency": { 2024-09-19T05:15:42.518 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:42.518 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:42.518 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:42.518 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.518 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_revoke_eviction": 0, 2024-09-19T05:15:42.518 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_acquisition_throttle": 0, 2024-09-19T05:15:42.518 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getvxattr_latency": { 2024-09-19T05:15:42.518 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:42.519 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:42.519 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:42.519 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T05:15:42.519 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.519 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_sessions": { 2024-09-19T05:15:42.519 INFO:teuthology.orchestra.run.smithi151.stdout: "session_count": 0, 2024-09-19T05:15:42.519 INFO:teuthology.orchestra.run.smithi151.stdout: "session_add": 2, 2024-09-19T05:15:42.519 INFO:teuthology.orchestra.run.smithi151.stdout: "session_remove": 2, 2024-09-19T05:15:42.519 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_open": 0, 2024-09-19T05:15:42.519 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_stale": 0, 2024-09-19T05:15:42.519 INFO:teuthology.orchestra.run.smithi151.stdout: "total_load": 0, 2024-09-19T05:15:42.519 INFO:teuthology.orchestra.run.smithi151.stdout: "average_load": 0, 2024-09-19T05:15:42.519 INFO:teuthology.orchestra.run.smithi151.stdout: "avg_session_uptime": 89, 2024-09-19T05:15:42.519 INFO:teuthology.orchestra.run.smithi151.stdout: "mdthresh_evicted": 0 2024-09-19T05:15:42.520 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.520 INFO:teuthology.orchestra.run.smithi151.stdout: "mempool": { 2024-09-19T05:15:42.520 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_bytes": 0, 2024-09-19T05:15:42.520 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_items": 0, 2024-09-19T05:15:42.520 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T05:15:42.520 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_items": 0, 2024-09-19T05:15:42.520 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T05:15:42.520 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_items": 0, 2024-09-19T05:15:42.520 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T05:15:42.520 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T05:15:42.520 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T05:15:42.520 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T05:15:42.520 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T05:15:42.521 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_items": 0, 2024-09-19T05:15:42.521 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T05:15:42.521 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T05:15:42.521 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_bytes": 0, 2024-09-19T05:15:42.521 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_items": 0, 2024-09-19T05:15:42.521 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_bytes": 0, 2024-09-19T05:15:42.521 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_items": 0, 2024-09-19T05:15:42.521 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T05:15:42.521 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T05:15:42.521 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T05:15:42.521 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T05:15:42.521 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T05:15:42.521 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_items": 0, 2024-09-19T05:15:42.522 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_bytes": 0, 2024-09-19T05:15:42.522 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_items": 0, 2024-09-19T05:15:42.522 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T05:15:42.522 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T05:15:42.522 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_bytes": 0, 2024-09-19T05:15:42.522 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_items": 0, 2024-09-19T05:15:42.522 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_bytes": 0, 2024-09-19T05:15:42.522 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_items": 0, 2024-09-19T05:15:42.522 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T05:15:42.522 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_items": 0, 2024-09-19T05:15:42.522 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T05:15:42.522 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_items": 0, 2024-09-19T05:15:42.522 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_bytes": 237847, 2024-09-19T05:15:42.522 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_items": 71, 2024-09-19T05:15:42.523 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_bytes": 0, 2024-09-19T05:15:42.523 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_items": 0, 2024-09-19T05:15:42.523 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_bytes": 0, 2024-09-19T05:15:42.523 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_items": 0, 2024-09-19T05:15:42.523 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_bytes": 0, 2024-09-19T05:15:42.523 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_items": 0, 2024-09-19T05:15:42.523 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_bytes": 0, 2024-09-19T05:15:42.523 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_items": 0, 2024-09-19T05:15:42.523 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_bytes": 8224, 2024-09-19T05:15:42.523 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_items": 181, 2024-09-19T05:15:42.523 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T05:15:42.523 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_items": 0, 2024-09-19T05:15:42.523 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_bytes": 0, 2024-09-19T05:15:42.523 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_items": 0, 2024-09-19T05:15:42.524 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_bytes": 50872, 2024-09-19T05:15:42.524 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_items": 198, 2024-09-19T05:15:42.524 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_bytes": 0, 2024-09-19T05:15:42.524 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_items": 0, 2024-09-19T05:15:42.524 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_bytes": 0, 2024-09-19T05:15:42.524 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_items": 0 2024-09-19T05:15:42.524 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.524 INFO:teuthology.orchestra.run.smithi151.stdout: "objecter": { 2024-09-19T05:15:42.524 INFO:teuthology.orchestra.run.smithi151.stdout: "op_active": 0, 2024-09-19T05:15:42.524 INFO:teuthology.orchestra.run.smithi151.stdout: "op_laggy": 0, 2024-09-19T05:15:42.524 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send": 42, 2024-09-19T05:15:42.524 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send_bytes": 12048, 2024-09-19T05:15:42.524 INFO:teuthology.orchestra.run.smithi151.stdout: "op_resend": 32, 2024-09-19T05:15:42.525 INFO:teuthology.orchestra.run.smithi151.stdout: "op_reply": 42, 2024-09-19T05:15:42.525 INFO:teuthology.orchestra.run.smithi151.stdout: "op_latency": { 2024-09-19T05:15:42.525 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 42, 2024-09-19T05:15:42.525 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.225997204, 2024-09-19T05:15:42.525 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.005380885 2024-09-19T05:15:42.525 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.525 INFO:teuthology.orchestra.run.smithi151.stdout: "op_inflight": 0, 2024-09-19T05:15:42.525 INFO:teuthology.orchestra.run.smithi151.stdout: "oplen_avg": { 2024-09-19T05:15:42.525 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 42, 2024-09-19T05:15:42.525 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 64 2024-09-19T05:15:42.525 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.526 INFO:teuthology.orchestra.run.smithi151.stdout: "op": 42, 2024-09-19T05:15:42.526 INFO:teuthology.orchestra.run.smithi151.stdout: "op_r": 0, 2024-09-19T05:15:42.526 INFO:teuthology.orchestra.run.smithi151.stdout: "op_w": 42, 2024-09-19T05:15:42.526 INFO:teuthology.orchestra.run.smithi151.stdout: "op_rmw": 0, 2024-09-19T05:15:42.526 INFO:teuthology.orchestra.run.smithi151.stdout: "op_pg": 0, 2024-09-19T05:15:42.526 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_stat": 0, 2024-09-19T05:15:42.526 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_create": 10, 2024-09-19T05:15:42.526 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_read": 0, 2024-09-19T05:15:42.526 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_write": 5, 2024-09-19T05:15:42.526 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writefull": 8, 2024-09-19T05:15:42.526 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writesame": 0, 2024-09-19T05:15:42.526 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_append": 0, 2024-09-19T05:15:42.526 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_zero": 0, 2024-09-19T05:15:42.526 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_truncate": 0, 2024-09-19T05:15:42.527 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_delete": 6, 2024-09-19T05:15:42.527 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_mapext": 0, 2024-09-19T05:15:42.527 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_sparse_read": 0, 2024-09-19T05:15:42.527 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_clonerange": 0, 2024-09-19T05:15:42.527 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_getxattr": 0, 2024-09-19T05:15:42.527 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_setxattr": 20, 2024-09-19T05:15:42.527 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_cmpxattr": 0, 2024-09-19T05:15:42.527 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_rmxattr": 0, 2024-09-19T05:15:42.527 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_resetxattrs": 0, 2024-09-19T05:15:42.527 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_call": 0, 2024-09-19T05:15:42.527 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_watch": 0, 2024-09-19T05:15:42.527 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_notify": 0, 2024-09-19T05:15:42.527 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T05:15:42.527 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls": 0, 2024-09-19T05:15:42.527 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls_filter": 0, 2024-09-19T05:15:42.528 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_other": 0, 2024-09-19T05:15:42.528 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_active": 0, 2024-09-19T05:15:42.528 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_send": 0, 2024-09-19T05:15:42.528 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_resend": 0, 2024-09-19T05:15:42.528 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_ping": 0, 2024-09-19T05:15:42.528 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_active": 0, 2024-09-19T05:15:42.528 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_send": 0, 2024-09-19T05:15:42.528 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_resend": 0, 2024-09-19T05:15:42.528 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_active": 0, 2024-09-19T05:15:42.528 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_send": 0, 2024-09-19T05:15:42.528 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_resend": 0, 2024-09-19T05:15:42.528 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_active": 0, 2024-09-19T05:15:42.528 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_send": 0, 2024-09-19T05:15:42.529 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_resend": 0, 2024-09-19T05:15:42.529 INFO:teuthology.orchestra.run.smithi151.stdout: "command_active": 0, 2024-09-19T05:15:42.529 INFO:teuthology.orchestra.run.smithi151.stdout: "command_send": 0, 2024-09-19T05:15:42.529 INFO:teuthology.orchestra.run.smithi151.stdout: "command_resend": 0, 2024-09-19T05:15:42.529 INFO:teuthology.orchestra.run.smithi151.stdout: "map_epoch": 96, 2024-09-19T05:15:42.529 INFO:teuthology.orchestra.run.smithi151.stdout: "map_full": 0, 2024-09-19T05:15:42.529 INFO:teuthology.orchestra.run.smithi151.stdout: "map_inc": 13, 2024-09-19T05:15:42.529 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_sessions": 9, 2024-09-19T05:15:42.529 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_open": 9, 2024-09-19T05:15:42.529 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_close": 0, 2024-09-19T05:15:42.529 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_laggy": 0, 2024-09-19T05:15:42.529 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_wr": 14, 2024-09-19T05:15:42.529 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_rd": 0, 2024-09-19T05:15:42.530 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_del": 1 2024-09-19T05:15:42.530 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.530 INFO:teuthology.orchestra.run.smithi151.stdout: "oft": { 2024-09-19T05:15:42.530 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_objs": 1, 2024-09-19T05:15:42.530 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_kv_pairs": 0, 2024-09-19T05:15:42.530 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_updates": 0, 2024-09-19T05:15:42.530 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_removes": 0 2024-09-19T05:15:42.530 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.530 INFO:teuthology.orchestra.run.smithi151.stdout: "osd-slow-ops": { 2024-09-19T05:15:42.530 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_ops_count": 0 2024-09-19T05:15:42.530 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.530 INFO:teuthology.orchestra.run.smithi151.stdout: "purge_queue": { 2024-09-19T05:15:42.530 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops": 0, 2024-09-19T05:15:42.530 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T05:15:42.531 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing": 0, 2024-09-19T05:15:42.531 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_high_water": 0, 2024-09-19T05:15:42.531 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed_ops": 0, 2024-09-19T05:15:42.531 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed": 0, 2024-09-19T05:15:42.531 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_item_in_journal": 0 2024-09-19T05:15:42.531 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.531 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T05:15:42.531 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-19T05:15:42.531 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-09-19T05:15:42.531 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-19T05:15:42.531 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 716, 2024-09-19T05:15:42.531 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 1674211, 2024-09-19T05:15:42.531 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:42.532 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 716, 2024-09-19T05:15:42.532 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-09-19T05:15:42.532 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-09-19T05:15:42.532 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 716, 2024-09-19T05:15:42.532 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 1674211, 2024-09-19T05:15:42.532 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-19T05:15:42.532 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:42.532 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:42.532 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:42.532 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T05:15:42.532 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.532 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_bytes": { 2024-09-19T05:15:42.532 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-19T05:15:42.533 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-09-19T05:15:42.533 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-19T05:15:42.533 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-09-19T05:15:42.533 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-09-19T05:15:42.533 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:42.533 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-09-19T05:15:42.533 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 42, 2024-09-19T05:15:42.533 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 12048, 2024-09-19T05:15:42.533 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 42, 2024-09-19T05:15:42.533 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 12048, 2024-09-19T05:15:42.533 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-19T05:15:42.533 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:42.533 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:42.533 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:42.534 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T05:15:42.534 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.534 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_ops": { 2024-09-19T05:15:42.534 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-19T05:15:42.534 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1024, 2024-09-19T05:15:42.534 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-19T05:15:42.534 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-09-19T05:15:42.534 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-09-19T05:15:42.534 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:42.534 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-09-19T05:15:42.534 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 42, 2024-09-19T05:15:42.535 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 42, 2024-09-19T05:15:42.535 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 42, 2024-09-19T05:15:42.535 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 42, 2024-09-19T05:15:42.535 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-19T05:15:42.535 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:42.535 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:42.535 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:42.535 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T05:15:42.535 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.535 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle": { 2024-09-19T05:15:42.535 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-19T05:15:42.535 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-09-19T05:15:42.535 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-19T05:15:42.535 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-09-19T05:15:42.536 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-09-19T05:15:42.536 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:42.536 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-09-19T05:15:42.536 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-09-19T05:15:42.536 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-09-19T05:15:42.536 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 0, 2024-09-19T05:15:42.536 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 0, 2024-09-19T05:15:42.536 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-19T05:15:42.536 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:42.536 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:42.536 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:42.536 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T05:15:42.536 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:42.536 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle-0x5561519e2960": { 2024-09-19T05:15:42.537 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-19T05:15:42.537 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-09-19T05:15:42.537 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-19T05:15:42.537 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 6, 2024-09-19T05:15:42.537 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 1656, 2024-09-19T05:15:42.537 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:42.537 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 6, 2024-09-19T05:15:42.537 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-09-19T05:15:42.537 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-09-19T05:15:42.537 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 5, 2024-09-19T05:15:42.537 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 1656, 2024-09-19T05:15:42.537 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-19T05:15:42.537 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:42.538 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:42.538 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:42.538 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T05:15:42.538 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T05:15:42.538 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-19T05:15:43.295 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-19T05:15:43.295 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-19T05:15:43.295 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-09-19T05:15:43.295 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-19T05:15:43.295 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T05:15:43.295 DEBUG:tasks.check_counter:Getting stats from f 2024-09-19T05:15:43.295 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-19T05:15:43.551 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:15:44.063 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-19T05:15:44.063 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T05:15:44.064 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 1683, 2024-09-19T05:15:44.064 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 1685, 2024-09-19T05:15:44.064 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 1736525, 2024-09-19T05:15:44.064 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 535204, 2024-09-19T05:15:44.064 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 6, 2024-09-19T05:15:44.064 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 2, 2024-09-19T05:15:44.064 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.157527648, 2024-09-19T05:15:44.064 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.051048968, 2024-09-19T05:15:44.064 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.042472030, 2024-09-19T05:15:44.064 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.048924134, 2024-09-19T05:15:44.064 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:15:44.064 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 1683, 2024-09-19T05:15:44.064 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.066570441, 2024-09-19T05:15:44.065 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000039554 2024-09-19T05:15:44.065 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.065 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:15:44.065 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:44.065 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:44.065 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:44.065 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.065 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T05:15:44.065 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T05:15:44.065 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.065 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T05:15:44.065 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 259, 2024-09-19T05:15:44.065 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 246, 2024-09-19T05:15:44.066 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 1629144, 2024-09-19T05:15:44.066 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 141642, 2024-09-19T05:15:44.066 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 4, 2024-09-19T05:15:44.066 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 3, 2024-09-19T05:15:44.066 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.047480312, 2024-09-19T05:15:44.066 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.016618899, 2024-09-19T05:15:44.066 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.047035968, 2024-09-19T05:15:44.066 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.000087251, 2024-09-19T05:15:44.066 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:15:44.066 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 245, 2024-09-19T05:15:44.066 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.019331216, 2024-09-19T05:15:44.066 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000078902 2024-09-19T05:15:44.066 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.067 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:15:44.067 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:44.067 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:44.067 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:44.067 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.067 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 1627488, 2024-09-19T05:15:44.067 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 140304 2024-09-19T05:15:44.067 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.067 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T05:15:44.067 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 787, 2024-09-19T05:15:44.067 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 953, 2024-09-19T05:15:44.067 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 195541, 2024-09-19T05:15:44.067 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 706615, 2024-09-19T05:15:44.067 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 6, 2024-09-19T05:15:44.068 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 4, 2024-09-19T05:15:44.068 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.086735186, 2024-09-19T05:15:44.068 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.034037971, 2024-09-19T05:15:44.068 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.019187753, 2024-09-19T05:15:44.068 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.024300660, 2024-09-19T05:15:44.068 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:15:44.068 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 951, 2024-09-19T05:15:44.068 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.046941094, 2024-09-19T05:15:44.068 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000049359 2024-09-19T05:15:44.068 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.068 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:15:44.068 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:44.068 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:44.069 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:44.069 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.069 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-09-19T05:15:44.069 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 458256 2024-09-19T05:15:44.069 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.069 INFO:teuthology.orchestra.run.smithi151.stdout: "cct": { 2024-09-19T05:15:44.069 INFO:teuthology.orchestra.run.smithi151.stdout: "total_workers": 1, 2024-09-19T05:15:44.069 INFO:teuthology.orchestra.run.smithi151.stdout: "unhealthy_workers": 0 2024-09-19T05:15:44.069 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.069 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-MDSRank": { 2024-09-19T05:15:44.069 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-09-19T05:15:44.070 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-09-19T05:15:44.070 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 4066, 2024-09-19T05:15:44.070 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.106243060, 2024-09-19T05:15:44.070 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000026129 2024-09-19T05:15:44.070 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T05:15:44.070 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.070 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-PurgeQueue": { 2024-09-19T05:15:44.070 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-09-19T05:15:44.070 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-09-19T05:15:44.070 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:44.070 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:44.070 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:44.071 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T05:15:44.071 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.071 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": { 2024-09-19T05:15:44.071 INFO:teuthology.orchestra.run.smithi151.stdout: "request": 0, 2024-09-19T05:15:44.071 INFO:teuthology.orchestra.run.smithi151.stdout: "reply": 0, 2024-09-19T05:15:44.071 INFO:teuthology.orchestra.run.smithi151.stdout: "reply_latency": { 2024-09-19T05:15:44.071 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:44.071 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:44.071 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:44.071 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.071 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_reply": 0, 2024-09-19T05:15:44.071 INFO:teuthology.orchestra.run.smithi151.stdout: "forward": 0, 2024-09-19T05:15:44.071 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_complete": 0, 2024-09-19T05:15:44.071 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_keys": 0, 2024-09-19T05:15:44.072 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_commit": 0, 2024-09-19T05:15:44.072 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_split": 0, 2024-09-19T05:15:44.072 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_merge": 0, 2024-09-19T05:15:44.072 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes": 27, 2024-09-19T05:15:44.072 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_top": 0, 2024-09-19T05:15:44.072 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_bottom": 27, 2024-09-19T05:15:44.072 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pin_tail": 0, 2024-09-19T05:15:44.072 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pinned": 27, 2024-09-19T05:15:44.072 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_expired": 0, 2024-09-19T05:15:44.072 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_with_caps": 0, 2024-09-19T05:15:44.072 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": 0, 2024-09-19T05:15:44.072 INFO:teuthology.orchestra.run.smithi151.stdout: "subtrees": 2, 2024-09-19T05:15:44.072 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse": 0, 2024-09-19T05:15:44.073 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_hit": 0, 2024-09-19T05:15:44.073 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_forward": 0, 2024-09-19T05:15:44.073 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_discover": 0, 2024-09-19T05:15:44.073 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_dir_fetch": 0, 2024-09-19T05:15:44.073 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_remote_ino": 0, 2024-09-19T05:15:44.073 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_lock": 0, 2024-09-19T05:15:44.073 INFO:teuthology.orchestra.run.smithi151.stdout: "load_cent": 0, 2024-09-19T05:15:44.073 INFO:teuthology.orchestra.run.smithi151.stdout: "q": 0, 2024-09-19T05:15:44.073 INFO:teuthology.orchestra.run.smithi151.stdout: "exported": 0, 2024-09-19T05:15:44.073 INFO:teuthology.orchestra.run.smithi151.stdout: "exported_inodes": 0, 2024-09-19T05:15:44.073 INFO:teuthology.orchestra.run.smithi151.stdout: "imported": 0, 2024-09-19T05:15:44.073 INFO:teuthology.orchestra.run.smithi151.stdout: "imported_inodes": 0, 2024-09-19T05:15:44.073 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_dir_fetch": 0, 2024-09-19T05:15:44.074 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_backtrace_fetch": 0, 2024-09-19T05:15:44.074 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_peer_discover": 0, 2024-09-19T05:15:44.074 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rfiles": 4, 2024-09-19T05:15:44.074 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rbytes": 234, 2024-09-19T05:15:44.074 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rsnaps": 6, 2024-09-19T05:15:44.074 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T05:15:44.074 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_set_tag": 0, 2024-09-19T05:15:44.074 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T05:15:44.074 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_inotable_repaired": 0, 2024-09-19T05:15:44.074 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_inodes": 0, 2024-09-19T05:15:44.075 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T05:15:44.075 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T05:15:44.075 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_file_inodes": 0, 2024-09-19T05:15:44.075 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_inode_file_caps": 0, 2024-09-19T05:15:44.075 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T05:15:44.075 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_grant": 0, 2024-09-19T05:15:44.075 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T05:15:44.075 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T05:15:44.075 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T05:15:44.075 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps": 0, 2024-09-19T05:15:44.075 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps_dirty": 0, 2024-09-19T05:15:44.075 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_cap_release": 0, 2024-09-19T05:15:44.075 INFO:teuthology.orchestra.run.smithi151.stdout: "process_request_cap_release": 0 2024-09-19T05:15:44.076 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.076 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_cache": { 2024-09-19T05:15:44.076 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update": 0, 2024-09-19T05:15:44.076 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update_receipt": 0, 2024-09-19T05:15:44.076 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_try_discover": 0, 2024-09-19T05:15:44.076 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_send_discover": 0, 2024-09-19T05:15:44.076 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_handle_discover": 0, 2024-09-19T05:15:44.076 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays": 0, 2024-09-19T05:15:44.076 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_delayed": 0, 2024-09-19T05:15:44.076 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_enqueuing": 0, 2024-09-19T05:15:44.076 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_created": 0, 2024-09-19T05:15:44.076 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_enqueued": 0, 2024-09-19T05:15:44.077 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_reintegrated": 0, 2024-09-19T05:15:44.077 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_migrated": 0, 2024-09-19T05:15:44.077 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_processing": 0, 2024-09-19T05:15:44.077 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_enqueued": 0, 2024-09-19T05:15:44.077 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_prioritized": 0, 2024-09-19T05:15:44.077 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_started": 0, 2024-09-19T05:15:44.077 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_completed": 0, 2024-09-19T05:15:44.077 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_path": 0, 2024-09-19T05:15:44.077 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_inode": 0, 2024-09-19T05:15:44.077 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T05:15:44.077 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_exportdir": 0, 2024-09-19T05:15:44.077 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_flush": 0, 2024-09-19T05:15:44.077 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragmentdir": 0, 2024-09-19T05:15:44.077 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragstats": 0, 2024-09-19T05:15:44.078 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_inodestats": 0 2024-09-19T05:15:44.078 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.078 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_log": { 2024-09-19T05:15:44.078 INFO:teuthology.orchestra.run.smithi151.stdout: "evlrg": 0, 2024-09-19T05:15:44.078 INFO:teuthology.orchestra.run.smithi151.stdout: "evadd": 0, 2024-09-19T05:15:44.078 INFO:teuthology.orchestra.run.smithi151.stdout: "evex": 0, 2024-09-19T05:15:44.078 INFO:teuthology.orchestra.run.smithi151.stdout: "evtrm": 0, 2024-09-19T05:15:44.078 INFO:teuthology.orchestra.run.smithi151.stdout: "ev": 288, 2024-09-19T05:15:44.078 INFO:teuthology.orchestra.run.smithi151.stdout: "evexg": 0, 2024-09-19T05:15:44.078 INFO:teuthology.orchestra.run.smithi151.stdout: "evexd": 0, 2024-09-19T05:15:44.078 INFO:teuthology.orchestra.run.smithi151.stdout: "segadd": 0, 2024-09-19T05:15:44.078 INFO:teuthology.orchestra.run.smithi151.stdout: "segex": 0, 2024-09-19T05:15:44.078 INFO:teuthology.orchestra.run.smithi151.stdout: "segtrm": 0, 2024-09-19T05:15:44.078 INFO:teuthology.orchestra.run.smithi151.stdout: "seg": 2, 2024-09-19T05:15:44.078 INFO:teuthology.orchestra.run.smithi151.stdout: "segmjr": 2, 2024-09-19T05:15:44.079 INFO:teuthology.orchestra.run.smithi151.stdout: "segexg": 0, 2024-09-19T05:15:44.079 INFO:teuthology.orchestra.run.smithi151.stdout: "segexd": 0, 2024-09-19T05:15:44.079 INFO:teuthology.orchestra.run.smithi151.stdout: "expos": 4194304, 2024-09-19T05:15:44.079 INFO:teuthology.orchestra.run.smithi151.stdout: "wrpos": 5433086, 2024-09-19T05:15:44.079 INFO:teuthology.orchestra.run.smithi151.stdout: "rdpos": 5432974, 2024-09-19T05:15:44.079 INFO:teuthology.orchestra.run.smithi151.stdout: "jlat": { 2024-09-19T05:15:44.079 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:44.079 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:44.079 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:44.079 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.079 INFO:teuthology.orchestra.run.smithi151.stdout: "replayed": 288 2024-09-19T05:15:44.079 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.079 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_mem": { 2024-09-19T05:15:44.079 INFO:teuthology.orchestra.run.smithi151.stdout: "ino": 18, 2024-09-19T05:15:44.080 INFO:teuthology.orchestra.run.smithi151.stdout: "ino+": 21, 2024-09-19T05:15:44.080 INFO:teuthology.orchestra.run.smithi151.stdout: "ino-": 3, 2024-09-19T05:15:44.080 INFO:teuthology.orchestra.run.smithi151.stdout: "dir": 11, 2024-09-19T05:15:44.080 INFO:teuthology.orchestra.run.smithi151.stdout: "dir+": 11, 2024-09-19T05:15:44.080 INFO:teuthology.orchestra.run.smithi151.stdout: "dir-": 0, 2024-09-19T05:15:44.080 INFO:teuthology.orchestra.run.smithi151.stdout: "dn": 27, 2024-09-19T05:15:44.080 INFO:teuthology.orchestra.run.smithi151.stdout: "dn+": 27, 2024-09-19T05:15:44.080 INFO:teuthology.orchestra.run.smithi151.stdout: "dn-": 0, 2024-09-19T05:15:44.080 INFO:teuthology.orchestra.run.smithi151.stdout: "cap": 0, 2024-09-19T05:15:44.080 INFO:teuthology.orchestra.run.smithi151.stdout: "cap+": 0, 2024-09-19T05:15:44.080 INFO:teuthology.orchestra.run.smithi151.stdout: "cap-": 0, 2024-09-19T05:15:44.080 INFO:teuthology.orchestra.run.smithi151.stdout: "rss": 51584, 2024-09-19T05:15:44.080 INFO:teuthology.orchestra.run.smithi151.stdout: "heap": 215328 2024-09-19T05:15:44.080 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.081 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_server": { 2024-09-19T05:15:44.081 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_client_request": 0, 2024-09-19T05:15:44.081 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_server_request": 0, 2024-09-19T05:15:44.081 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_request": 0, 2024-09-19T05:15:44.081 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_session": 0, 2024-09-19T05:15:44.081 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_peer_request": 0, 2024-09-19T05:15:44.081 INFO:teuthology.orchestra.run.smithi151.stdout: "req_create_latency": { 2024-09-19T05:15:44.081 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:44.081 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:44.081 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:44.081 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.081 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getattr_latency": { 2024-09-19T05:15:44.081 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:44.082 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:44.082 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:44.082 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.082 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getfilelock_latency": { 2024-09-19T05:15:44.082 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:44.082 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:44.082 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:44.082 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.082 INFO:teuthology.orchestra.run.smithi151.stdout: "req_link_latency": { 2024-09-19T05:15:44.082 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:44.082 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:44.082 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:44.082 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.082 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookup_latency": { 2024-09-19T05:15:44.082 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:44.083 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:44.083 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:44.083 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.083 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookuphash_latency": { 2024-09-19T05:15:44.083 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:44.083 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:44.083 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:44.083 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.083 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupino_latency": { 2024-09-19T05:15:44.083 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:44.083 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:44.083 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:44.083 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.083 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupname_latency": { 2024-09-19T05:15:44.084 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:44.084 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:44.084 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:44.084 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.084 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupparent_latency": { 2024-09-19T05:15:44.084 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:44.084 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:44.084 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:44.084 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.084 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupsnap_latency": { 2024-09-19T05:15:44.084 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:44.084 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:44.084 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:44.085 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.085 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lssnap_latency": { 2024-09-19T05:15:44.085 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:44.085 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:44.085 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:44.085 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.085 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mkdir_latency": { 2024-09-19T05:15:44.085 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:44.085 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:44.085 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:44.085 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.085 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mknod_latency": { 2024-09-19T05:15:44.085 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:44.086 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:44.086 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:44.086 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.086 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mksnap_latency": { 2024-09-19T05:15:44.086 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:44.086 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:44.086 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:44.086 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.086 INFO:teuthology.orchestra.run.smithi151.stdout: "req_open_latency": { 2024-09-19T05:15:44.086 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:44.086 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:44.086 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:44.086 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.086 INFO:teuthology.orchestra.run.smithi151.stdout: "req_readdir_latency": { 2024-09-19T05:15:44.087 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:44.087 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:44.087 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:44.087 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.087 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rename_latency": { 2024-09-19T05:15:44.087 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:44.087 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:44.087 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:44.087 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.087 INFO:teuthology.orchestra.run.smithi151.stdout: "req_renamesnap_latency": { 2024-09-19T05:15:44.087 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:44.087 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:44.087 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:44.087 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.088 INFO:teuthology.orchestra.run.smithi151.stdout: "req_snapdiff_latency": { 2024-09-19T05:15:44.088 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:44.088 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:44.088 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:44.088 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.088 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmdir_latency": { 2024-09-19T05:15:44.088 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:44.088 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:44.088 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:44.088 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.088 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmsnap_latency": { 2024-09-19T05:15:44.088 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:44.088 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:44.088 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:44.089 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.089 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmxattr_latency": { 2024-09-19T05:15:44.089 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:44.089 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:44.089 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:44.089 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.089 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setattr_latency": { 2024-09-19T05:15:44.089 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:44.089 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:44.089 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:44.089 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.089 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setdirlayout_latency": { 2024-09-19T05:15:44.089 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:44.089 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:44.090 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:44.090 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.090 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setfilelock_latency": { 2024-09-19T05:15:44.090 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:44.090 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:44.090 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:44.090 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.090 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setlayout_latency": { 2024-09-19T05:15:44.090 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:44.090 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:44.090 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:44.090 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.090 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setxattr_latency": { 2024-09-19T05:15:44.091 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:44.091 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:44.091 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:44.091 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.091 INFO:teuthology.orchestra.run.smithi151.stdout: "req_symlink_latency": { 2024-09-19T05:15:44.091 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:44.091 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:44.091 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:44.091 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.091 INFO:teuthology.orchestra.run.smithi151.stdout: "req_unlink_latency": { 2024-09-19T05:15:44.091 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:44.091 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:44.091 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:44.092 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.092 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_revoke_eviction": 0, 2024-09-19T05:15:44.092 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_acquisition_throttle": 0, 2024-09-19T05:15:44.092 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getvxattr_latency": { 2024-09-19T05:15:44.092 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:44.092 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:44.092 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:44.092 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T05:15:44.092 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.092 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_sessions": { 2024-09-19T05:15:44.092 INFO:teuthology.orchestra.run.smithi151.stdout: "session_count": 2, 2024-09-19T05:15:44.092 INFO:teuthology.orchestra.run.smithi151.stdout: "session_add": 6, 2024-09-19T05:15:44.092 INFO:teuthology.orchestra.run.smithi151.stdout: "session_remove": 4, 2024-09-19T05:15:44.092 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_open": 5, 2024-09-19T05:15:44.093 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_stale": 0, 2024-09-19T05:15:44.093 INFO:teuthology.orchestra.run.smithi151.stdout: "total_load": 0, 2024-09-19T05:15:44.093 INFO:teuthology.orchestra.run.smithi151.stdout: "average_load": 0, 2024-09-19T05:15:44.093 INFO:teuthology.orchestra.run.smithi151.stdout: "avg_session_uptime": 4052, 2024-09-19T05:15:44.093 INFO:teuthology.orchestra.run.smithi151.stdout: "mdthresh_evicted": 0 2024-09-19T05:15:44.093 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.093 INFO:teuthology.orchestra.run.smithi151.stdout: "mempool": { 2024-09-19T05:15:44.093 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_bytes": 0, 2024-09-19T05:15:44.093 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_items": 0, 2024-09-19T05:15:44.093 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T05:15:44.093 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_items": 0, 2024-09-19T05:15:44.093 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T05:15:44.093 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_items": 0, 2024-09-19T05:15:44.094 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T05:15:44.094 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T05:15:44.094 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T05:15:44.094 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T05:15:44.094 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T05:15:44.094 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_items": 0, 2024-09-19T05:15:44.094 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T05:15:44.094 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T05:15:44.094 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_bytes": 0, 2024-09-19T05:15:44.094 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_items": 0, 2024-09-19T05:15:44.094 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_bytes": 0, 2024-09-19T05:15:44.094 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_items": 0, 2024-09-19T05:15:44.094 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T05:15:44.094 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T05:15:44.095 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T05:15:44.095 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T05:15:44.095 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T05:15:44.095 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_items": 0, 2024-09-19T05:15:44.095 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_bytes": 0, 2024-09-19T05:15:44.095 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_items": 0, 2024-09-19T05:15:44.095 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T05:15:44.095 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T05:15:44.095 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_bytes": 0, 2024-09-19T05:15:44.095 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_items": 0, 2024-09-19T05:15:44.095 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_bytes": 0, 2024-09-19T05:15:44.095 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_items": 0, 2024-09-19T05:15:44.095 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T05:15:44.095 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_items": 0, 2024-09-19T05:15:44.096 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T05:15:44.096 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_items": 0, 2024-09-19T05:15:44.096 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_bytes": 189780, 2024-09-19T05:15:44.096 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_items": 62, 2024-09-19T05:15:44.096 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_bytes": 0, 2024-09-19T05:15:44.096 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_items": 0, 2024-09-19T05:15:44.096 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_bytes": 0, 2024-09-19T05:15:44.096 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_items": 0, 2024-09-19T05:15:44.096 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_bytes": 0, 2024-09-19T05:15:44.096 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_items": 0, 2024-09-19T05:15:44.096 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_bytes": 0, 2024-09-19T05:15:44.096 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_items": 0, 2024-09-19T05:15:44.096 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_bytes": 8224, 2024-09-19T05:15:44.097 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_items": 181, 2024-09-19T05:15:44.097 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T05:15:44.097 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_items": 0, 2024-09-19T05:15:44.097 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_bytes": 0, 2024-09-19T05:15:44.097 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_items": 0, 2024-09-19T05:15:44.097 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_bytes": 94790, 2024-09-19T05:15:44.097 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_items": 671, 2024-09-19T05:15:44.097 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_bytes": 0, 2024-09-19T05:15:44.097 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_items": 0, 2024-09-19T05:15:44.097 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_bytes": 0, 2024-09-19T05:15:44.097 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_items": 0 2024-09-19T05:15:44.097 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.097 INFO:teuthology.orchestra.run.smithi151.stdout: "objecter": { 2024-09-19T05:15:44.098 INFO:teuthology.orchestra.run.smithi151.stdout: "op_active": 0, 2024-09-19T05:15:44.098 INFO:teuthology.orchestra.run.smithi151.stdout: "op_laggy": 0, 2024-09-19T05:15:44.098 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send": 2468, 2024-09-19T05:15:44.098 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send_bytes": 26, 2024-09-19T05:15:44.098 INFO:teuthology.orchestra.run.smithi151.stdout: "op_resend": 4, 2024-09-19T05:15:44.098 INFO:teuthology.orchestra.run.smithi151.stdout: "op_reply": 2468, 2024-09-19T05:15:44.098 INFO:teuthology.orchestra.run.smithi151.stdout: "op_latency": { 2024-09-19T05:15:44.098 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 2468, 2024-09-19T05:15:44.098 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 1.134985853, 2024-09-19T05:15:44.098 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000459880 2024-09-19T05:15:44.098 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.098 INFO:teuthology.orchestra.run.smithi151.stdout: "op_inflight": 0, 2024-09-19T05:15:44.098 INFO:teuthology.orchestra.run.smithi151.stdout: "oplen_avg": { 2024-09-19T05:15:44.098 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 2468, 2024-09-19T05:15:44.098 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 2469 2024-09-19T05:15:44.099 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.099 INFO:teuthology.orchestra.run.smithi151.stdout: "op": 2468, 2024-09-19T05:15:44.099 INFO:teuthology.orchestra.run.smithi151.stdout: "op_r": 2468, 2024-09-19T05:15:44.099 INFO:teuthology.orchestra.run.smithi151.stdout: "op_w": 0, 2024-09-19T05:15:44.099 INFO:teuthology.orchestra.run.smithi151.stdout: "op_rmw": 0, 2024-09-19T05:15:44.099 INFO:teuthology.orchestra.run.smithi151.stdout: "op_pg": 0, 2024-09-19T05:15:44.099 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_stat": 1564, 2024-09-19T05:15:44.099 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_create": 0, 2024-09-19T05:15:44.099 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_read": 901, 2024-09-19T05:15:44.099 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_write": 0, 2024-09-19T05:15:44.099 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writefull": 0, 2024-09-19T05:15:44.099 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writesame": 0, 2024-09-19T05:15:44.099 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_append": 0, 2024-09-19T05:15:44.100 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_zero": 0, 2024-09-19T05:15:44.100 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_truncate": 0, 2024-09-19T05:15:44.100 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_delete": 0, 2024-09-19T05:15:44.100 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_mapext": 0, 2024-09-19T05:15:44.100 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_sparse_read": 0, 2024-09-19T05:15:44.100 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_clonerange": 0, 2024-09-19T05:15:44.100 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_getxattr": 2, 2024-09-19T05:15:44.100 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_setxattr": 0, 2024-09-19T05:15:44.100 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_cmpxattr": 0, 2024-09-19T05:15:44.100 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_rmxattr": 0, 2024-09-19T05:15:44.100 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_resetxattrs": 0, 2024-09-19T05:15:44.100 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_call": 0, 2024-09-19T05:15:44.100 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_watch": 0, 2024-09-19T05:15:44.100 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_notify": 0, 2024-09-19T05:15:44.101 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T05:15:44.101 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls": 0, 2024-09-19T05:15:44.101 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls_filter": 0, 2024-09-19T05:15:44.101 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_other": 0, 2024-09-19T05:15:44.101 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_active": 0, 2024-09-19T05:15:44.101 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_send": 0, 2024-09-19T05:15:44.101 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_resend": 0, 2024-09-19T05:15:44.101 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_ping": 0, 2024-09-19T05:15:44.101 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_active": 0, 2024-09-19T05:15:44.101 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_send": 0, 2024-09-19T05:15:44.101 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_resend": 0, 2024-09-19T05:15:44.101 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_active": 0, 2024-09-19T05:15:44.101 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_send": 0, 2024-09-19T05:15:44.101 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_resend": 0, 2024-09-19T05:15:44.102 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_active": 0, 2024-09-19T05:15:44.102 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_send": 0, 2024-09-19T05:15:44.102 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_resend": 0, 2024-09-19T05:15:44.102 INFO:teuthology.orchestra.run.smithi151.stdout: "command_active": 0, 2024-09-19T05:15:44.102 INFO:teuthology.orchestra.run.smithi151.stdout: "command_send": 0, 2024-09-19T05:15:44.102 INFO:teuthology.orchestra.run.smithi151.stdout: "command_resend": 0, 2024-09-19T05:15:44.102 INFO:teuthology.orchestra.run.smithi151.stdout: "map_epoch": 96, 2024-09-19T05:15:44.102 INFO:teuthology.orchestra.run.smithi151.stdout: "map_full": 0, 2024-09-19T05:15:44.102 INFO:teuthology.orchestra.run.smithi151.stdout: "map_inc": 13, 2024-09-19T05:15:44.102 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_sessions": 5, 2024-09-19T05:15:44.102 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_open": 5, 2024-09-19T05:15:44.102 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_close": 0, 2024-09-19T05:15:44.102 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_laggy": 0, 2024-09-19T05:15:44.102 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_wr": 0, 2024-09-19T05:15:44.102 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_rd": 2, 2024-09-19T05:15:44.103 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_del": 0 2024-09-19T05:15:44.103 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.103 INFO:teuthology.orchestra.run.smithi151.stdout: "oft": { 2024-09-19T05:15:44.103 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_objs": 0, 2024-09-19T05:15:44.103 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_kv_pairs": 0, 2024-09-19T05:15:44.103 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_updates": 0, 2024-09-19T05:15:44.103 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_removes": 0 2024-09-19T05:15:44.103 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.103 INFO:teuthology.orchestra.run.smithi151.stdout: "osd-slow-ops": { 2024-09-19T05:15:44.103 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_ops_count": 0 2024-09-19T05:15:44.103 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.103 INFO:teuthology.orchestra.run.smithi151.stdout: "purge_queue": { 2024-09-19T05:15:44.103 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops": 0, 2024-09-19T05:15:44.103 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T05:15:44.104 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing": 0, 2024-09-19T05:15:44.104 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_high_water": 0, 2024-09-19T05:15:44.104 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed_ops": 0, 2024-09-19T05:15:44.104 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed": 0, 2024-09-19T05:15:44.104 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_item_in_journal": 0 2024-09-19T05:15:44.104 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.104 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T05:15:44.104 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-19T05:15:44.104 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-09-19T05:15:44.104 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-19T05:15:44.104 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 2725, 2024-09-19T05:15:44.104 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 3299860, 2024-09-19T05:15:44.104 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:44.104 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 2725, 2024-09-19T05:15:44.105 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-09-19T05:15:44.105 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-09-19T05:15:44.105 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 2725, 2024-09-19T05:15:44.105 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 3299860, 2024-09-19T05:15:44.105 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-19T05:15:44.105 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:44.105 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:44.105 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:44.105 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T05:15:44.105 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.105 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_bytes": { 2024-09-19T05:15:44.105 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-19T05:15:44.105 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-09-19T05:15:44.105 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-19T05:15:44.106 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-09-19T05:15:44.106 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-09-19T05:15:44.106 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:44.106 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-09-19T05:15:44.106 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 2468, 2024-09-19T05:15:44.106 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 1238792, 2024-09-19T05:15:44.106 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 2468, 2024-09-19T05:15:44.106 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 1238792, 2024-09-19T05:15:44.106 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-19T05:15:44.107 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:44.107 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:44.107 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:44.107 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T05:15:44.107 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.107 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_ops": { 2024-09-19T05:15:44.107 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-19T05:15:44.107 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1024, 2024-09-19T05:15:44.107 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-19T05:15:44.107 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-09-19T05:15:44.107 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-09-19T05:15:44.108 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:44.108 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-09-19T05:15:44.108 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 2468, 2024-09-19T05:15:44.108 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 2468, 2024-09-19T05:15:44.108 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 2468, 2024-09-19T05:15:44.108 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 2468, 2024-09-19T05:15:44.108 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-19T05:15:44.108 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:44.108 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:44.108 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:44.108 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T05:15:44.108 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.108 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle": { 2024-09-19T05:15:44.109 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-19T05:15:44.109 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-09-19T05:15:44.109 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-19T05:15:44.109 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-09-19T05:15:44.109 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-09-19T05:15:44.109 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:44.109 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-09-19T05:15:44.109 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-09-19T05:15:44.109 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-09-19T05:15:44.109 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 0, 2024-09-19T05:15:44.109 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 0, 2024-09-19T05:15:44.109 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-19T05:15:44.109 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:44.110 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:44.110 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:44.110 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T05:15:44.110 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:44.110 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle-0x563caf358b40": { 2024-09-19T05:15:44.110 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-19T05:15:44.110 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-09-19T05:15:44.110 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-19T05:15:44.110 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-09-19T05:15:44.110 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-09-19T05:15:44.110 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:44.110 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-09-19T05:15:44.110 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-09-19T05:15:44.110 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-09-19T05:15:44.111 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 0, 2024-09-19T05:15:44.111 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 0, 2024-09-19T05:15:44.111 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-19T05:15:44.111 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:44.111 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:44.111 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:44.111 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T05:15:44.111 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T05:15:44.111 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-19T05:15:44.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:43 smithi096 ceph-mon[35085]: pgmap v345: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:15:44.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:43 smithi043 ceph-mon[30800]: pgmap v345: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:15:44.384 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:43 smithi151 ceph-mon[35425]: pgmap v345: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:15:44.885 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-19T05:15:44.885 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-19T05:15:44.885 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-09-19T05:15:44.885 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-19T05:15:44.885 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T05:15:44.885 DEBUG:tasks.check_counter:Getting stats from i 2024-09-19T05:15:44.886 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-19T05:15:45.133 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:15:45.691 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-19T05:15:45.691 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T05:15:45.691 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 652, 2024-09-19T05:15:45.691 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 792, 2024-09-19T05:15:45.691 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 1750098, 2024-09-19T05:15:45.691 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 409646, 2024-09-19T05:15:45.692 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 9, 2024-09-19T05:15:45.692 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 2, 2024-09-19T05:15:45.692 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.102357658, 2024-09-19T05:15:45.692 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.038556564, 2024-09-19T05:15:45.692 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.067085015, 2024-09-19T05:15:45.692 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.005296921, 2024-09-19T05:15:45.692 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:15:45.692 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 793, 2024-09-19T05:15:45.692 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.051469975, 2024-09-19T05:15:45.692 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000064905 2024-09-19T05:15:45.692 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.692 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:15:45.692 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 847, 2024-09-19T05:15:45.693 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000222958, 2024-09-19T05:15:45.693 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000263 2024-09-19T05:15:45.693 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.693 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 1650960, 2024-09-19T05:15:45.693 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 144144 2024-09-19T05:15:45.693 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.693 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T05:15:45.693 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 63, 2024-09-19T05:15:45.693 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 98, 2024-09-19T05:15:45.693 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 14029, 2024-09-19T05:15:45.693 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 584757, 2024-09-19T05:15:45.693 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 45, 2024-09-19T05:15:45.693 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 5, 2024-09-19T05:15:45.694 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.022583756, 2024-09-19T05:15:45.694 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.002864395, 2024-09-19T05:15:45.694 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.017411119, 2024-09-19T05:15:45.694 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.000331777, 2024-09-19T05:15:45.694 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:15:45.694 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 96, 2024-09-19T05:15:45.694 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.016048604, 2024-09-19T05:15:45.694 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000167172 2024-09-19T05:15:45.694 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.694 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:15:45.694 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 54, 2024-09-19T05:15:45.694 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000005235, 2024-09-19T05:15:45.694 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000096 2024-09-19T05:15:45.694 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.695 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 5040, 2024-09-19T05:15:45.695 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 133664 2024-09-19T05:15:45.695 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.695 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T05:15:45.695 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 30, 2024-09-19T05:15:45.695 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 184, 2024-09-19T05:15:45.695 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 6882, 2024-09-19T05:15:45.695 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 366484, 2024-09-19T05:15:45.695 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 11, 2024-09-19T05:15:45.695 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 6, 2024-09-19T05:15:45.695 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.013185510, 2024-09-19T05:15:45.695 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.006904390, 2024-09-19T05:15:45.695 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.004581439, 2024-09-19T05:15:45.696 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.000363275, 2024-09-19T05:15:45.696 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:15:45.696 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 180, 2024-09-19T05:15:45.696 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.020247198, 2024-09-19T05:15:45.696 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000112484 2024-09-19T05:15:45.696 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.696 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:15:45.696 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 43, 2024-09-19T05:15:45.696 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000008162, 2024-09-19T05:15:45.696 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000189 2024-09-19T05:15:45.696 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.696 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 2816, 2024-09-19T05:15:45.696 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 335072 2024-09-19T05:15:45.696 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.697 INFO:teuthology.orchestra.run.smithi151.stdout: "cct": { 2024-09-19T05:15:45.697 INFO:teuthology.orchestra.run.smithi151.stdout: "total_workers": 1, 2024-09-19T05:15:45.697 INFO:teuthology.orchestra.run.smithi151.stdout: "unhealthy_workers": 0 2024-09-19T05:15:45.697 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.697 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-MDSRank": { 2024-09-19T05:15:45.697 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-09-19T05:15:45.697 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-09-19T05:15:45.697 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 60, 2024-09-19T05:15:45.697 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.001694345, 2024-09-19T05:15:45.697 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000028239 2024-09-19T05:15:45.697 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T05:15:45.697 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.697 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-PurgeQueue": { 2024-09-19T05:15:45.697 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-09-19T05:15:45.697 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-09-19T05:15:45.698 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 2, 2024-09-19T05:15:45.698 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000013550, 2024-09-19T05:15:45.698 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000006775 2024-09-19T05:15:45.698 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T05:15:45.698 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.698 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": { 2024-09-19T05:15:45.698 INFO:teuthology.orchestra.run.smithi151.stdout: "request": 0, 2024-09-19T05:15:45.698 INFO:teuthology.orchestra.run.smithi151.stdout: "reply": 0, 2024-09-19T05:15:45.698 INFO:teuthology.orchestra.run.smithi151.stdout: "reply_latency": { 2024-09-19T05:15:45.698 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:45.698 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:45.698 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:45.698 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.699 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_reply": 0, 2024-09-19T05:15:45.699 INFO:teuthology.orchestra.run.smithi151.stdout: "forward": 0, 2024-09-19T05:15:45.699 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_complete": 0, 2024-09-19T05:15:45.699 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_keys": 0, 2024-09-19T05:15:45.699 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_commit": 11, 2024-09-19T05:15:45.699 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_split": 0, 2024-09-19T05:15:45.699 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_merge": 0, 2024-09-19T05:15:45.699 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes": 10, 2024-09-19T05:15:45.699 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_top": 0, 2024-09-19T05:15:45.699 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_bottom": 10, 2024-09-19T05:15:45.699 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pin_tail": 0, 2024-09-19T05:15:45.699 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pinned": 10, 2024-09-19T05:15:45.699 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_expired": 0, 2024-09-19T05:15:45.699 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_with_caps": 0, 2024-09-19T05:15:45.699 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": 0, 2024-09-19T05:15:45.699 INFO:teuthology.orchestra.run.smithi151.stdout: "subtrees": 1, 2024-09-19T05:15:45.700 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse": 0, 2024-09-19T05:15:45.700 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_hit": 0, 2024-09-19T05:15:45.700 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_forward": 0, 2024-09-19T05:15:45.700 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_discover": 0, 2024-09-19T05:15:45.700 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_dir_fetch": 0, 2024-09-19T05:15:45.700 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_remote_ino": 0, 2024-09-19T05:15:45.700 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_lock": 0, 2024-09-19T05:15:45.700 INFO:teuthology.orchestra.run.smithi151.stdout: "load_cent": 0, 2024-09-19T05:15:45.700 INFO:teuthology.orchestra.run.smithi151.stdout: "q": 0, 2024-09-19T05:15:45.700 INFO:teuthology.orchestra.run.smithi151.stdout: "exported": 0, 2024-09-19T05:15:45.700 INFO:teuthology.orchestra.run.smithi151.stdout: "exported_inodes": 0, 2024-09-19T05:15:45.700 INFO:teuthology.orchestra.run.smithi151.stdout: "imported": 0, 2024-09-19T05:15:45.700 INFO:teuthology.orchestra.run.smithi151.stdout: "imported_inodes": 0, 2024-09-19T05:15:45.700 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_dir_fetch": 0, 2024-09-19T05:15:45.701 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_backtrace_fetch": 0, 2024-09-19T05:15:45.701 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_peer_discover": 0, 2024-09-19T05:15:45.701 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rfiles": 4, 2024-09-19T05:15:45.701 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rbytes": 234, 2024-09-19T05:15:45.701 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rsnaps": 6, 2024-09-19T05:15:45.701 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T05:15:45.701 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_set_tag": 0, 2024-09-19T05:15:45.701 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T05:15:45.701 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_inotable_repaired": 0, 2024-09-19T05:15:45.701 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_inodes": 0, 2024-09-19T05:15:45.701 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T05:15:45.701 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T05:15:45.701 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_file_inodes": 0, 2024-09-19T05:15:45.701 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_inode_file_caps": 0, 2024-09-19T05:15:45.701 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T05:15:45.701 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_grant": 0, 2024-09-19T05:15:45.702 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T05:15:45.702 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T05:15:45.702 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T05:15:45.702 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps": 0, 2024-09-19T05:15:45.702 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps_dirty": 0, 2024-09-19T05:15:45.702 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_cap_release": 0, 2024-09-19T05:15:45.702 INFO:teuthology.orchestra.run.smithi151.stdout: "process_request_cap_release": 0 2024-09-19T05:15:45.702 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.702 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_cache": { 2024-09-19T05:15:45.702 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update": 0, 2024-09-19T05:15:45.702 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update_receipt": 0, 2024-09-19T05:15:45.702 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_try_discover": 0, 2024-09-19T05:15:45.702 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_send_discover": 1, 2024-09-19T05:15:45.702 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_handle_discover": 0, 2024-09-19T05:15:45.702 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays": 0, 2024-09-19T05:15:45.702 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_delayed": 0, 2024-09-19T05:15:45.703 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_enqueuing": 0, 2024-09-19T05:15:45.703 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_created": 0, 2024-09-19T05:15:45.703 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_enqueued": 0, 2024-09-19T05:15:45.703 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_reintegrated": 0, 2024-09-19T05:15:45.703 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_migrated": 0, 2024-09-19T05:15:45.703 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_processing": 0, 2024-09-19T05:15:45.703 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_enqueued": 0, 2024-09-19T05:15:45.703 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_prioritized": 0, 2024-09-19T05:15:45.703 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_started": 0, 2024-09-19T05:15:45.703 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_completed": 0, 2024-09-19T05:15:45.703 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_path": 0, 2024-09-19T05:15:45.703 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_inode": 0, 2024-09-19T05:15:45.703 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T05:15:45.703 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_exportdir": 0, 2024-09-19T05:15:45.704 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_flush": 0, 2024-09-19T05:15:45.704 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragmentdir": 0, 2024-09-19T05:15:45.704 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragstats": 0, 2024-09-19T05:15:45.704 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_inodestats": 0 2024-09-19T05:15:45.704 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.704 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_log": { 2024-09-19T05:15:45.704 INFO:teuthology.orchestra.run.smithi151.stdout: "evlrg": 0, 2024-09-19T05:15:45.704 INFO:teuthology.orchestra.run.smithi151.stdout: "evadd": 9, 2024-09-19T05:15:45.704 INFO:teuthology.orchestra.run.smithi151.stdout: "evex": 0, 2024-09-19T05:15:45.704 INFO:teuthology.orchestra.run.smithi151.stdout: "evtrm": 0, 2024-09-19T05:15:45.704 INFO:teuthology.orchestra.run.smithi151.stdout: "ev": 9, 2024-09-19T05:15:45.704 INFO:teuthology.orchestra.run.smithi151.stdout: "evexg": 0, 2024-09-19T05:15:45.705 INFO:teuthology.orchestra.run.smithi151.stdout: "evexd": 0, 2024-09-19T05:15:45.705 INFO:teuthology.orchestra.run.smithi151.stdout: "segadd": 2, 2024-09-19T05:15:45.705 INFO:teuthology.orchestra.run.smithi151.stdout: "segex": 0, 2024-09-19T05:15:45.705 INFO:teuthology.orchestra.run.smithi151.stdout: "segtrm": 0, 2024-09-19T05:15:45.705 INFO:teuthology.orchestra.run.smithi151.stdout: "seg": 2, 2024-09-19T05:15:45.705 INFO:teuthology.orchestra.run.smithi151.stdout: "segmjr": 2, 2024-09-19T05:15:45.705 INFO:teuthology.orchestra.run.smithi151.stdout: "segexg": 0, 2024-09-19T05:15:45.705 INFO:teuthology.orchestra.run.smithi151.stdout: "segexd": 0, 2024-09-19T05:15:45.705 INFO:teuthology.orchestra.run.smithi151.stdout: "expos": 4194304, 2024-09-19T05:15:45.705 INFO:teuthology.orchestra.run.smithi151.stdout: "wrpos": 4196887, 2024-09-19T05:15:45.705 INFO:teuthology.orchestra.run.smithi151.stdout: "rdpos": 0, 2024-09-19T05:15:45.705 INFO:teuthology.orchestra.run.smithi151.stdout: "jlat": { 2024-09-19T05:15:45.705 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 8, 2024-09-19T05:15:45.705 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.063157763, 2024-09-19T05:15:45.706 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.007894720 2024-09-19T05:15:45.706 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.706 INFO:teuthology.orchestra.run.smithi151.stdout: "replayed": 0 2024-09-19T05:15:45.706 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.706 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_mem": { 2024-09-19T05:15:45.706 INFO:teuthology.orchestra.run.smithi151.stdout: "ino": 13, 2024-09-19T05:15:45.706 INFO:teuthology.orchestra.run.smithi151.stdout: "ino+": 13, 2024-09-19T05:15:45.706 INFO:teuthology.orchestra.run.smithi151.stdout: "ino-": 0, 2024-09-19T05:15:45.706 INFO:teuthology.orchestra.run.smithi151.stdout: "dir": 11, 2024-09-19T05:15:45.706 INFO:teuthology.orchestra.run.smithi151.stdout: "dir+": 12, 2024-09-19T05:15:45.707 INFO:teuthology.orchestra.run.smithi151.stdout: "dir-": 1, 2024-09-19T05:15:45.707 INFO:teuthology.orchestra.run.smithi151.stdout: "dn": 10, 2024-09-19T05:15:45.707 INFO:teuthology.orchestra.run.smithi151.stdout: "dn+": 10, 2024-09-19T05:15:45.707 INFO:teuthology.orchestra.run.smithi151.stdout: "dn-": 0, 2024-09-19T05:15:45.707 INFO:teuthology.orchestra.run.smithi151.stdout: "cap": 0, 2024-09-19T05:15:45.707 INFO:teuthology.orchestra.run.smithi151.stdout: "cap+": 0, 2024-09-19T05:15:45.707 INFO:teuthology.orchestra.run.smithi151.stdout: "cap-": 0, 2024-09-19T05:15:45.707 INFO:teuthology.orchestra.run.smithi151.stdout: "rss": 50784, 2024-09-19T05:15:45.707 INFO:teuthology.orchestra.run.smithi151.stdout: "heap": 223520 2024-09-19T05:15:45.707 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.707 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_server": { 2024-09-19T05:15:45.707 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_client_request": 0, 2024-09-19T05:15:45.707 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_server_request": 0, 2024-09-19T05:15:45.707 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_request": 0, 2024-09-19T05:15:45.707 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_session": 21, 2024-09-19T05:15:45.707 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_peer_request": 0, 2024-09-19T05:15:45.708 INFO:teuthology.orchestra.run.smithi151.stdout: "req_create_latency": { 2024-09-19T05:15:45.708 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:45.708 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:45.708 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:45.708 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.708 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getattr_latency": { 2024-09-19T05:15:45.708 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:45.708 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:45.708 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:45.708 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.708 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getfilelock_latency": { 2024-09-19T05:15:45.708 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:45.708 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:45.708 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:45.709 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.709 INFO:teuthology.orchestra.run.smithi151.stdout: "req_link_latency": { 2024-09-19T05:15:45.709 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:45.709 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:45.709 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:45.709 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.709 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookup_latency": { 2024-09-19T05:15:45.709 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:45.709 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:45.709 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:45.709 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.709 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookuphash_latency": { 2024-09-19T05:15:45.709 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:45.709 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:45.709 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:45.710 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.710 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupino_latency": { 2024-09-19T05:15:45.710 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:45.710 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:45.710 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:45.710 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.710 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupname_latency": { 2024-09-19T05:15:45.710 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:45.710 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:45.710 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:45.710 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.710 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupparent_latency": { 2024-09-19T05:15:45.710 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:45.710 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:45.711 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:45.711 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.711 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupsnap_latency": { 2024-09-19T05:15:45.711 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:45.711 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:45.711 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:45.711 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.711 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lssnap_latency": { 2024-09-19T05:15:45.711 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:45.711 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:45.711 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:45.711 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.711 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mkdir_latency": { 2024-09-19T05:15:45.711 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:45.712 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:45.712 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:45.712 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.712 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mknod_latency": { 2024-09-19T05:15:45.712 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:45.712 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:45.712 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:45.713 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.713 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mksnap_latency": { 2024-09-19T05:15:45.713 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:45.713 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:45.713 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:45.713 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.713 INFO:teuthology.orchestra.run.smithi151.stdout: "req_open_latency": { 2024-09-19T05:15:45.713 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:45.713 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:45.713 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:45.713 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.713 INFO:teuthology.orchestra.run.smithi151.stdout: "req_readdir_latency": { 2024-09-19T05:15:45.713 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:45.713 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:45.714 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:45.714 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.714 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rename_latency": { 2024-09-19T05:15:45.714 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:45.714 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:45.714 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:45.714 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.714 INFO:teuthology.orchestra.run.smithi151.stdout: "req_renamesnap_latency": { 2024-09-19T05:15:45.714 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:45.714 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:45.714 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:45.715 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.715 INFO:teuthology.orchestra.run.smithi151.stdout: "req_snapdiff_latency": { 2024-09-19T05:15:45.715 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:45.715 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:45.715 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:45.715 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.715 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmdir_latency": { 2024-09-19T05:15:45.715 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:45.715 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:45.715 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:45.715 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.715 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmsnap_latency": { 2024-09-19T05:15:45.715 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:45.716 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:45.716 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:45.716 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.716 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmxattr_latency": { 2024-09-19T05:15:45.716 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:45.716 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:45.716 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:45.716 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.716 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setattr_latency": { 2024-09-19T05:15:45.716 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:45.716 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:45.716 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:45.716 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.717 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setdirlayout_latency": { 2024-09-19T05:15:45.717 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:45.717 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:45.717 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:45.717 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.717 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setfilelock_latency": { 2024-09-19T05:15:45.717 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:45.717 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:45.717 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:45.717 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.717 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setlayout_latency": { 2024-09-19T05:15:45.717 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:45.717 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:45.717 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:45.718 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.718 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setxattr_latency": { 2024-09-19T05:15:45.718 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:45.718 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:45.718 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:45.718 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.718 INFO:teuthology.orchestra.run.smithi151.stdout: "req_symlink_latency": { 2024-09-19T05:15:45.718 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:45.718 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:45.718 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:45.718 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.718 INFO:teuthology.orchestra.run.smithi151.stdout: "req_unlink_latency": { 2024-09-19T05:15:45.718 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:45.718 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:45.719 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:45.719 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.719 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_revoke_eviction": 0, 2024-09-19T05:15:45.719 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_acquisition_throttle": 0, 2024-09-19T05:15:45.719 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getvxattr_latency": { 2024-09-19T05:15:45.719 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:45.719 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:45.719 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:45.719 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T05:15:45.719 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.719 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_sessions": { 2024-09-19T05:15:45.719 INFO:teuthology.orchestra.run.smithi151.stdout: "session_count": 1, 2024-09-19T05:15:45.719 INFO:teuthology.orchestra.run.smithi151.stdout: "session_add": 4, 2024-09-19T05:15:45.720 INFO:teuthology.orchestra.run.smithi151.stdout: "session_remove": 3, 2024-09-19T05:15:45.720 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_open": 1, 2024-09-19T05:15:45.720 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_stale": 0, 2024-09-19T05:15:45.720 INFO:teuthology.orchestra.run.smithi151.stdout: "total_load": 0, 2024-09-19T05:15:45.720 INFO:teuthology.orchestra.run.smithi151.stdout: "average_load": 0, 2024-09-19T05:15:45.720 INFO:teuthology.orchestra.run.smithi151.stdout: "avg_session_uptime": 642, 2024-09-19T05:15:45.720 INFO:teuthology.orchestra.run.smithi151.stdout: "mdthresh_evicted": 0 2024-09-19T05:15:45.720 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.720 INFO:teuthology.orchestra.run.smithi151.stdout: "mempool": { 2024-09-19T05:15:45.720 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_bytes": 0, 2024-09-19T05:15:45.721 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_items": 0, 2024-09-19T05:15:45.721 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T05:15:45.721 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_items": 0, 2024-09-19T05:15:45.721 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T05:15:45.721 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_items": 0, 2024-09-19T05:15:45.721 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T05:15:45.721 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T05:15:45.721 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T05:15:45.721 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T05:15:45.721 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T05:15:45.721 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_items": 0, 2024-09-19T05:15:45.722 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T05:15:45.722 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T05:15:45.722 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_bytes": 0, 2024-09-19T05:15:45.722 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_items": 0, 2024-09-19T05:15:45.722 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_bytes": 0, 2024-09-19T05:15:45.722 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_items": 0, 2024-09-19T05:15:45.722 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T05:15:45.722 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T05:15:45.722 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T05:15:45.722 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T05:15:45.722 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T05:15:45.722 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_items": 0, 2024-09-19T05:15:45.722 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_bytes": 0, 2024-09-19T05:15:45.722 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_items": 0, 2024-09-19T05:15:45.722 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T05:15:45.723 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T05:15:45.723 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_bytes": 0, 2024-09-19T05:15:45.723 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_items": 0, 2024-09-19T05:15:45.723 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_bytes": 0, 2024-09-19T05:15:45.723 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_items": 0, 2024-09-19T05:15:45.723 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T05:15:45.723 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_items": 0, 2024-09-19T05:15:45.723 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T05:15:45.723 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_items": 0, 2024-09-19T05:15:45.723 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_bytes": 230135, 2024-09-19T05:15:45.723 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_items": 72, 2024-09-19T05:15:45.723 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_bytes": 0, 2024-09-19T05:15:45.723 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_items": 0, 2024-09-19T05:15:45.723 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_bytes": 0, 2024-09-19T05:15:45.723 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_items": 0, 2024-09-19T05:15:45.723 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_bytes": 0, 2024-09-19T05:15:45.724 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_items": 0, 2024-09-19T05:15:45.724 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_bytes": 0, 2024-09-19T05:15:45.724 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_items": 0, 2024-09-19T05:15:45.724 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_bytes": 8224, 2024-09-19T05:15:45.724 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_items": 181, 2024-09-19T05:15:45.724 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T05:15:45.725 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_items": 0, 2024-09-19T05:15:45.725 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_bytes": 0, 2024-09-19T05:15:45.725 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_items": 0, 2024-09-19T05:15:45.725 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_bytes": 50872, 2024-09-19T05:15:45.725 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_items": 198, 2024-09-19T05:15:45.725 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_bytes": 0, 2024-09-19T05:15:45.725 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_items": 0, 2024-09-19T05:15:45.725 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_bytes": 0, 2024-09-19T05:15:45.725 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_items": 0 2024-09-19T05:15:45.725 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.725 INFO:teuthology.orchestra.run.smithi151.stdout: "objecter": { 2024-09-19T05:15:45.726 INFO:teuthology.orchestra.run.smithi151.stdout: "op_active": 0, 2024-09-19T05:15:45.726 INFO:teuthology.orchestra.run.smithi151.stdout: "op_laggy": 0, 2024-09-19T05:15:45.726 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send": 46, 2024-09-19T05:15:45.726 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send_bytes": 13065, 2024-09-19T05:15:45.726 INFO:teuthology.orchestra.run.smithi151.stdout: "op_resend": 32, 2024-09-19T05:15:45.726 INFO:teuthology.orchestra.run.smithi151.stdout: "op_reply": 46, 2024-09-19T05:15:45.726 INFO:teuthology.orchestra.run.smithi151.stdout: "op_latency": { 2024-09-19T05:15:45.726 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 46, 2024-09-19T05:15:45.726 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.295996363, 2024-09-19T05:15:45.726 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.006434703 2024-09-19T05:15:45.726 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.727 INFO:teuthology.orchestra.run.smithi151.stdout: "op_inflight": 0, 2024-09-19T05:15:45.727 INFO:teuthology.orchestra.run.smithi151.stdout: "oplen_avg": { 2024-09-19T05:15:45.727 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 46, 2024-09-19T05:15:45.727 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 68 2024-09-19T05:15:45.727 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.727 INFO:teuthology.orchestra.run.smithi151.stdout: "op": 46, 2024-09-19T05:15:45.727 INFO:teuthology.orchestra.run.smithi151.stdout: "op_r": 0, 2024-09-19T05:15:45.727 INFO:teuthology.orchestra.run.smithi151.stdout: "op_w": 46, 2024-09-19T05:15:45.727 INFO:teuthology.orchestra.run.smithi151.stdout: "op_rmw": 0, 2024-09-19T05:15:45.727 INFO:teuthology.orchestra.run.smithi151.stdout: "op_pg": 0, 2024-09-19T05:15:45.727 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_stat": 0, 2024-09-19T05:15:45.727 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_create": 10, 2024-09-19T05:15:45.727 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_read": 0, 2024-09-19T05:15:45.727 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_write": 8, 2024-09-19T05:15:45.728 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writefull": 9, 2024-09-19T05:15:45.728 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writesame": 0, 2024-09-19T05:15:45.728 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_append": 0, 2024-09-19T05:15:45.728 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_zero": 0, 2024-09-19T05:15:45.728 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_truncate": 0, 2024-09-19T05:15:45.728 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_delete": 6, 2024-09-19T05:15:45.728 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_mapext": 0, 2024-09-19T05:15:45.728 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_sparse_read": 0, 2024-09-19T05:15:45.728 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_clonerange": 0, 2024-09-19T05:15:45.728 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_getxattr": 0, 2024-09-19T05:15:45.728 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_setxattr": 20, 2024-09-19T05:15:45.728 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_cmpxattr": 0, 2024-09-19T05:15:45.729 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_rmxattr": 0, 2024-09-19T05:15:45.729 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_resetxattrs": 0, 2024-09-19T05:15:45.729 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_call": 0, 2024-09-19T05:15:45.729 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_watch": 0, 2024-09-19T05:15:45.729 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_notify": 0, 2024-09-19T05:15:45.729 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T05:15:45.729 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls": 0, 2024-09-19T05:15:45.729 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls_filter": 0, 2024-09-19T05:15:45.729 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_other": 0, 2024-09-19T05:15:45.729 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_active": 0, 2024-09-19T05:15:45.729 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_send": 0, 2024-09-19T05:15:45.729 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_resend": 0, 2024-09-19T05:15:45.729 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_ping": 0, 2024-09-19T05:15:45.729 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_active": 0, 2024-09-19T05:15:45.730 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_send": 0, 2024-09-19T05:15:45.730 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_resend": 0, 2024-09-19T05:15:45.730 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_active": 0, 2024-09-19T05:15:45.730 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_send": 0, 2024-09-19T05:15:45.730 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_resend": 0, 2024-09-19T05:15:45.730 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_active": 0, 2024-09-19T05:15:45.730 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_send": 0, 2024-09-19T05:15:45.730 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_resend": 0, 2024-09-19T05:15:45.730 INFO:teuthology.orchestra.run.smithi151.stdout: "command_active": 0, 2024-09-19T05:15:45.730 INFO:teuthology.orchestra.run.smithi151.stdout: "command_send": 0, 2024-09-19T05:15:45.730 INFO:teuthology.orchestra.run.smithi151.stdout: "command_resend": 0, 2024-09-19T05:15:45.730 INFO:teuthology.orchestra.run.smithi151.stdout: "map_epoch": 96, 2024-09-19T05:15:45.730 INFO:teuthology.orchestra.run.smithi151.stdout: "map_full": 0, 2024-09-19T05:15:45.730 INFO:teuthology.orchestra.run.smithi151.stdout: "map_inc": 13, 2024-09-19T05:15:45.730 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_sessions": 8, 2024-09-19T05:15:45.731 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_open": 8, 2024-09-19T05:15:45.731 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_close": 0, 2024-09-19T05:15:45.731 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_laggy": 0, 2024-09-19T05:15:45.731 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_wr": 14, 2024-09-19T05:15:45.731 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_rd": 0, 2024-09-19T05:15:45.731 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_del": 1 2024-09-19T05:15:45.731 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.731 INFO:teuthology.orchestra.run.smithi151.stdout: "oft": { 2024-09-19T05:15:45.731 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_objs": 1, 2024-09-19T05:15:45.731 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_kv_pairs": 0, 2024-09-19T05:15:45.731 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_updates": 0, 2024-09-19T05:15:45.731 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_removes": 0 2024-09-19T05:15:45.731 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.731 INFO:teuthology.orchestra.run.smithi151.stdout: "osd-slow-ops": { 2024-09-19T05:15:45.731 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_ops_count": 0 2024-09-19T05:15:45.732 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.732 INFO:teuthology.orchestra.run.smithi151.stdout: "purge_queue": { 2024-09-19T05:15:45.732 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops": 0, 2024-09-19T05:15:45.732 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T05:15:45.732 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing": 0, 2024-09-19T05:15:45.732 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_high_water": 0, 2024-09-19T05:15:45.732 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed_ops": 0, 2024-09-19T05:15:45.732 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed": 0, 2024-09-19T05:15:45.732 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_item_in_journal": 0 2024-09-19T05:15:45.732 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.732 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T05:15:45.732 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-19T05:15:45.732 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-09-19T05:15:45.732 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-19T05:15:45.732 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 741, 2024-09-19T05:15:45.733 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 1685819, 2024-09-19T05:15:45.733 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:45.733 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 741, 2024-09-19T05:15:45.733 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-09-19T05:15:45.733 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-09-19T05:15:45.733 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 741, 2024-09-19T05:15:45.733 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 1685819, 2024-09-19T05:15:45.733 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-19T05:15:45.733 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:45.733 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:45.733 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:45.733 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T05:15:45.733 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.733 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_bytes": { 2024-09-19T05:15:45.734 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-19T05:15:45.734 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-09-19T05:15:45.734 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-19T05:15:45.734 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-09-19T05:15:45.734 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-09-19T05:15:45.734 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:45.734 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-09-19T05:15:45.734 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 46, 2024-09-19T05:15:45.734 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 13065, 2024-09-19T05:15:45.734 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 46, 2024-09-19T05:15:45.734 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 13065, 2024-09-19T05:15:45.734 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-19T05:15:45.734 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:45.734 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:45.735 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:45.735 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T05:15:45.735 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.735 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_ops": { 2024-09-19T05:15:45.735 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-19T05:15:45.735 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1024, 2024-09-19T05:15:45.735 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-19T05:15:45.735 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-09-19T05:15:45.735 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-09-19T05:15:45.735 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:45.735 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-09-19T05:15:45.735 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 46, 2024-09-19T05:15:45.735 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 46, 2024-09-19T05:15:45.735 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 46, 2024-09-19T05:15:45.736 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 46, 2024-09-19T05:15:45.736 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-19T05:15:45.736 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:45.736 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:45.736 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:45.736 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T05:15:45.736 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.736 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle": { 2024-09-19T05:15:45.736 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-19T05:15:45.736 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-09-19T05:15:45.736 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-19T05:15:45.736 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-09-19T05:15:45.736 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-09-19T05:15:45.736 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:45.737 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-09-19T05:15:45.737 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-09-19T05:15:45.737 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-09-19T05:15:45.737 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 0, 2024-09-19T05:15:45.737 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 0, 2024-09-19T05:15:45.737 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-19T05:15:45.737 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:45.737 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:45.737 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:45.737 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T05:15:45.737 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:45.737 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle-0x55d91631a960": { 2024-09-19T05:15:45.737 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-19T05:15:45.737 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-09-19T05:15:45.737 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-19T05:15:45.738 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 9, 2024-09-19T05:15:45.738 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 2583, 2024-09-19T05:15:45.738 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:45.738 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 9, 2024-09-19T05:15:45.738 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-09-19T05:15:45.738 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-09-19T05:15:45.738 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 8, 2024-09-19T05:15:45.738 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 2583, 2024-09-19T05:15:45.738 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-19T05:15:45.738 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:45.738 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:45.738 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:45.738 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T05:15:45.738 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T05:15:45.738 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-19T05:15:46.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:45 smithi096 ceph-mon[35085]: pgmap v346: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:15:46.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:45 smithi043 ceph-mon[30800]: pgmap v346: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:15:46.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:45 smithi151 ceph-mon[35425]: pgmap v346: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:15:46.524 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-09-19T05:15:46.524 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-09-19T05:15:46.524 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-09-19T05:15:46.524 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-09-19T05:15:46.524 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T05:15:46.525 DEBUG:tasks.check_counter:Getting stats from l 2024-09-19T05:15:46.525 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-19T05:15:46.771 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T05:15:47.291 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-19T05:15:47.291 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T05:15:47.291 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 1583, 2024-09-19T05:15:47.291 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 1585, 2024-09-19T05:15:47.291 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 475769, 2024-09-19T05:15:47.291 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 503395, 2024-09-19T05:15:47.291 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 6, 2024-09-19T05:15:47.291 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 2, 2024-09-19T05:15:47.292 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.167833938, 2024-09-19T05:15:47.292 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.058608422, 2024-09-19T05:15:47.292 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.042815649, 2024-09-19T05:15:47.292 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.050944944, 2024-09-19T05:15:47.292 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:15:47.292 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 1583, 2024-09-19T05:15:47.292 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.067234730, 2024-09-19T05:15:47.292 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000042472 2024-09-19T05:15:47.292 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.292 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:15:47.292 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:47.292 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:47.292 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:47.293 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.293 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T05:15:47.293 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T05:15:47.293 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.293 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T05:15:47.293 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 704, 2024-09-19T05:15:47.293 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 876, 2024-09-19T05:15:47.293 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 173664, 2024-09-19T05:15:47.293 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 681816, 2024-09-19T05:15:47.293 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 7, 2024-09-19T05:15:47.293 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 5, 2024-09-19T05:15:47.293 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.083828688, 2024-09-19T05:15:47.293 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.032191533, 2024-09-19T05:15:47.293 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.017682745, 2024-09-19T05:15:47.294 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.025648032, 2024-09-19T05:15:47.294 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:15:47.294 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 873, 2024-09-19T05:15:47.294 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.050619556, 2024-09-19T05:15:47.294 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000057983 2024-09-19T05:15:47.294 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.294 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:15:47.294 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:47.294 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:47.294 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:47.294 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.294 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T05:15:47.294 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 458512 2024-09-19T05:15:47.295 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.295 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T05:15:47.295 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 263, 2024-09-19T05:15:47.296 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 248, 2024-09-19T05:15:47.296 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 1631708, 2024-09-19T05:15:47.296 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 143470, 2024-09-19T05:15:47.296 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 3, 2024-09-19T05:15:47.296 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 2, 2024-09-19T05:15:47.296 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.047760941, 2024-09-19T05:15:47.296 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.016851388, 2024-09-19T05:15:47.296 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.044957063, 2024-09-19T05:15:47.296 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.000039569, 2024-09-19T05:15:47.296 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T05:15:47.296 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 248, 2024-09-19T05:15:47.296 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.019012387, 2024-09-19T05:15:47.296 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000076662 2024-09-19T05:15:47.296 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.297 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-09-19T05:15:47.297 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:47.297 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:47.297 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:47.297 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.297 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 1631440, 2024-09-19T05:15:47.297 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 143152 2024-09-19T05:15:47.297 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.297 INFO:teuthology.orchestra.run.smithi151.stdout: "cct": { 2024-09-19T05:15:47.297 INFO:teuthology.orchestra.run.smithi151.stdout: "total_workers": 1, 2024-09-19T05:15:47.297 INFO:teuthology.orchestra.run.smithi151.stdout: "unhealthy_workers": 0 2024-09-19T05:15:47.297 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.297 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-MDSRank": { 2024-09-19T05:15:47.298 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-09-19T05:15:47.298 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-09-19T05:15:47.298 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 3766, 2024-09-19T05:15:47.298 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.115033802, 2024-09-19T05:15:47.298 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000030545 2024-09-19T05:15:47.298 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T05:15:47.298 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.298 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-PurgeQueue": { 2024-09-19T05:15:47.298 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-09-19T05:15:47.298 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-09-19T05:15:47.298 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:47.298 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:47.298 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:47.298 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T05:15:47.299 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.299 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": { 2024-09-19T05:15:47.299 INFO:teuthology.orchestra.run.smithi151.stdout: "request": 0, 2024-09-19T05:15:47.299 INFO:teuthology.orchestra.run.smithi151.stdout: "reply": 0, 2024-09-19T05:15:47.299 INFO:teuthology.orchestra.run.smithi151.stdout: "reply_latency": { 2024-09-19T05:15:47.299 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:47.299 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:47.299 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:47.299 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.299 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_reply": 0, 2024-09-19T05:15:47.299 INFO:teuthology.orchestra.run.smithi151.stdout: "forward": 0, 2024-09-19T05:15:47.299 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_complete": 0, 2024-09-19T05:15:47.299 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_keys": 0, 2024-09-19T05:15:47.300 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_commit": 0, 2024-09-19T05:15:47.300 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_split": 0, 2024-09-19T05:15:47.300 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_merge": 0, 2024-09-19T05:15:47.300 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes": 0, 2024-09-19T05:15:47.300 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_top": 0, 2024-09-19T05:15:47.300 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_bottom": 0, 2024-09-19T05:15:47.300 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pin_tail": 0, 2024-09-19T05:15:47.300 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pinned": 0, 2024-09-19T05:15:47.300 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_expired": 0, 2024-09-19T05:15:47.300 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_with_caps": 0, 2024-09-19T05:15:47.300 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": 0, 2024-09-19T05:15:47.300 INFO:teuthology.orchestra.run.smithi151.stdout: "subtrees": 1, 2024-09-19T05:15:47.300 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse": 0, 2024-09-19T05:15:47.300 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_hit": 0, 2024-09-19T05:15:47.301 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_forward": 0, 2024-09-19T05:15:47.301 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_discover": 0, 2024-09-19T05:15:47.301 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_dir_fetch": 0, 2024-09-19T05:15:47.301 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_remote_ino": 0, 2024-09-19T05:15:47.301 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_lock": 0, 2024-09-19T05:15:47.301 INFO:teuthology.orchestra.run.smithi151.stdout: "load_cent": 0, 2024-09-19T05:15:47.301 INFO:teuthology.orchestra.run.smithi151.stdout: "q": 0, 2024-09-19T05:15:47.301 INFO:teuthology.orchestra.run.smithi151.stdout: "exported": 0, 2024-09-19T05:15:47.301 INFO:teuthology.orchestra.run.smithi151.stdout: "exported_inodes": 0, 2024-09-19T05:15:47.301 INFO:teuthology.orchestra.run.smithi151.stdout: "imported": 0, 2024-09-19T05:15:47.301 INFO:teuthology.orchestra.run.smithi151.stdout: "imported_inodes": 0, 2024-09-19T05:15:47.301 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_dir_fetch": 0, 2024-09-19T05:15:47.302 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_backtrace_fetch": 0, 2024-09-19T05:15:47.302 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_peer_discover": 0, 2024-09-19T05:15:47.302 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rfiles": 0, 2024-09-19T05:15:47.302 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rbytes": 0, 2024-09-19T05:15:47.302 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rsnaps": 0, 2024-09-19T05:15:47.302 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T05:15:47.302 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_set_tag": 0, 2024-09-19T05:15:47.302 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T05:15:47.302 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_inotable_repaired": 0, 2024-09-19T05:15:47.302 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_inodes": 0, 2024-09-19T05:15:47.302 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T05:15:47.302 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T05:15:47.302 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_file_inodes": 0, 2024-09-19T05:15:47.302 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_inode_file_caps": 0, 2024-09-19T05:15:47.303 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T05:15:47.303 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_grant": 0, 2024-09-19T05:15:47.303 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T05:15:47.303 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T05:15:47.303 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T05:15:47.303 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps": 0, 2024-09-19T05:15:47.303 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps_dirty": 0, 2024-09-19T05:15:47.303 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_cap_release": 0, 2024-09-19T05:15:47.303 INFO:teuthology.orchestra.run.smithi151.stdout: "process_request_cap_release": 0 2024-09-19T05:15:47.303 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.303 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_cache": { 2024-09-19T05:15:47.303 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update": 0, 2024-09-19T05:15:47.303 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update_receipt": 0, 2024-09-19T05:15:47.303 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_try_discover": 0, 2024-09-19T05:15:47.304 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_send_discover": 0, 2024-09-19T05:15:47.304 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_handle_discover": 0, 2024-09-19T05:15:47.304 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays": 0, 2024-09-19T05:15:47.304 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_delayed": 0, 2024-09-19T05:15:47.304 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_enqueuing": 0, 2024-09-19T05:15:47.304 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_created": 0, 2024-09-19T05:15:47.304 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_enqueued": 0, 2024-09-19T05:15:47.304 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_reintegrated": 0, 2024-09-19T05:15:47.304 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_migrated": 0, 2024-09-19T05:15:47.304 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_processing": 0, 2024-09-19T05:15:47.304 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_enqueued": 0, 2024-09-19T05:15:47.304 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_prioritized": 0, 2024-09-19T05:15:47.304 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_started": 0, 2024-09-19T05:15:47.304 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_completed": 0, 2024-09-19T05:15:47.305 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_path": 0, 2024-09-19T05:15:47.305 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_inode": 0, 2024-09-19T05:15:47.305 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T05:15:47.305 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_exportdir": 0, 2024-09-19T05:15:47.305 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_flush": 0, 2024-09-19T05:15:47.305 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragmentdir": 0, 2024-09-19T05:15:47.305 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragstats": 0, 2024-09-19T05:15:47.305 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_inodestats": 0 2024-09-19T05:15:47.305 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.305 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_log": { 2024-09-19T05:15:47.305 INFO:teuthology.orchestra.run.smithi151.stdout: "evlrg": 0, 2024-09-19T05:15:47.305 INFO:teuthology.orchestra.run.smithi151.stdout: "evadd": 0, 2024-09-19T05:15:47.305 INFO:teuthology.orchestra.run.smithi151.stdout: "evex": 0, 2024-09-19T05:15:47.305 INFO:teuthology.orchestra.run.smithi151.stdout: "evtrm": 0, 2024-09-19T05:15:47.305 INFO:teuthology.orchestra.run.smithi151.stdout: "ev": 10, 2024-09-19T05:15:47.306 INFO:teuthology.orchestra.run.smithi151.stdout: "evexg": 0, 2024-09-19T05:15:47.306 INFO:teuthology.orchestra.run.smithi151.stdout: "evexd": 0, 2024-09-19T05:15:47.306 INFO:teuthology.orchestra.run.smithi151.stdout: "segadd": 0, 2024-09-19T05:15:47.306 INFO:teuthology.orchestra.run.smithi151.stdout: "segex": 0, 2024-09-19T05:15:47.306 INFO:teuthology.orchestra.run.smithi151.stdout: "segtrm": 0, 2024-09-19T05:15:47.306 INFO:teuthology.orchestra.run.smithi151.stdout: "seg": 2, 2024-09-19T05:15:47.306 INFO:teuthology.orchestra.run.smithi151.stdout: "segmjr": 2, 2024-09-19T05:15:47.306 INFO:teuthology.orchestra.run.smithi151.stdout: "segexg": 0, 2024-09-19T05:15:47.306 INFO:teuthology.orchestra.run.smithi151.stdout: "segexd": 0, 2024-09-19T05:15:47.306 INFO:teuthology.orchestra.run.smithi151.stdout: "expos": 4194304, 2024-09-19T05:15:47.306 INFO:teuthology.orchestra.run.smithi151.stdout: "wrpos": 4197198, 2024-09-19T05:15:47.306 INFO:teuthology.orchestra.run.smithi151.stdout: "rdpos": 4197057, 2024-09-19T05:15:47.306 INFO:teuthology.orchestra.run.smithi151.stdout: "jlat": { 2024-09-19T05:15:47.306 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:47.307 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:47.307 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:47.307 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.307 INFO:teuthology.orchestra.run.smithi151.stdout: "replayed": 10 2024-09-19T05:15:47.307 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.307 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_mem": { 2024-09-19T05:15:47.307 INFO:teuthology.orchestra.run.smithi151.stdout: "ino": 3, 2024-09-19T05:15:47.307 INFO:teuthology.orchestra.run.smithi151.stdout: "ino+": 3, 2024-09-19T05:15:47.307 INFO:teuthology.orchestra.run.smithi151.stdout: "ino-": 0, 2024-09-19T05:15:47.307 INFO:teuthology.orchestra.run.smithi151.stdout: "dir": 1, 2024-09-19T05:15:47.307 INFO:teuthology.orchestra.run.smithi151.stdout: "dir+": 1, 2024-09-19T05:15:47.307 INFO:teuthology.orchestra.run.smithi151.stdout: "dir-": 0, 2024-09-19T05:15:47.307 INFO:teuthology.orchestra.run.smithi151.stdout: "dn": 0, 2024-09-19T05:15:47.307 INFO:teuthology.orchestra.run.smithi151.stdout: "dn+": 0, 2024-09-19T05:15:47.308 INFO:teuthology.orchestra.run.smithi151.stdout: "dn-": 0, 2024-09-19T05:15:47.308 INFO:teuthology.orchestra.run.smithi151.stdout: "cap": 0, 2024-09-19T05:15:47.308 INFO:teuthology.orchestra.run.smithi151.stdout: "cap+": 0, 2024-09-19T05:15:47.308 INFO:teuthology.orchestra.run.smithi151.stdout: "cap-": 0, 2024-09-19T05:15:47.308 INFO:teuthology.orchestra.run.smithi151.stdout: "rss": 48816, 2024-09-19T05:15:47.308 INFO:teuthology.orchestra.run.smithi151.stdout: "heap": 215328 2024-09-19T05:15:47.308 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.308 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_server": { 2024-09-19T05:15:47.308 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_client_request": 0, 2024-09-19T05:15:47.308 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_server_request": 0, 2024-09-19T05:15:47.308 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_request": 0, 2024-09-19T05:15:47.308 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_session": 0, 2024-09-19T05:15:47.308 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_peer_request": 0, 2024-09-19T05:15:47.308 INFO:teuthology.orchestra.run.smithi151.stdout: "req_create_latency": { 2024-09-19T05:15:47.309 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:47.309 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:47.309 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:47.309 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.309 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getattr_latency": { 2024-09-19T05:15:47.309 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:47.309 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:47.309 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:47.309 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.309 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getfilelock_latency": { 2024-09-19T05:15:47.309 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:47.309 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:47.309 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:47.309 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.309 INFO:teuthology.orchestra.run.smithi151.stdout: "req_link_latency": { 2024-09-19T05:15:47.310 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:47.310 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:47.310 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:47.310 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.310 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookup_latency": { 2024-09-19T05:15:47.310 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:47.310 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:47.310 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:47.310 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.310 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookuphash_latency": { 2024-09-19T05:15:47.310 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:47.310 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:47.310 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:47.310 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.311 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupino_latency": { 2024-09-19T05:15:47.311 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:47.311 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:47.311 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:47.311 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.311 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupname_latency": { 2024-09-19T05:15:47.311 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:47.311 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:47.311 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:47.311 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.311 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupparent_latency": { 2024-09-19T05:15:47.311 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:47.311 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:47.311 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:47.311 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.312 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupsnap_latency": { 2024-09-19T05:15:47.312 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:47.312 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:47.312 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:47.312 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.312 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lssnap_latency": { 2024-09-19T05:15:47.312 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:47.312 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:47.312 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:47.312 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.312 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mkdir_latency": { 2024-09-19T05:15:47.312 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:47.313 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:47.313 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:47.313 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.313 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mknod_latency": { 2024-09-19T05:15:47.314 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:47.314 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:47.314 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:47.314 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.314 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mksnap_latency": { 2024-09-19T05:15:47.314 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:47.314 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:47.314 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:47.314 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.314 INFO:teuthology.orchestra.run.smithi151.stdout: "req_open_latency": { 2024-09-19T05:15:47.314 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:47.314 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:47.314 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:47.314 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.315 INFO:teuthology.orchestra.run.smithi151.stdout: "req_readdir_latency": { 2024-09-19T05:15:47.315 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:47.315 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:47.315 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:47.315 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.315 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rename_latency": { 2024-09-19T05:15:47.315 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:47.315 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:47.315 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:47.315 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.315 INFO:teuthology.orchestra.run.smithi151.stdout: "req_renamesnap_latency": { 2024-09-19T05:15:47.315 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:47.315 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:47.315 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:47.316 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.316 INFO:teuthology.orchestra.run.smithi151.stdout: "req_snapdiff_latency": { 2024-09-19T05:15:47.316 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:47.316 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:47.316 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:47.316 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.316 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmdir_latency": { 2024-09-19T05:15:47.316 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:47.316 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:47.316 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:47.316 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.316 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmsnap_latency": { 2024-09-19T05:15:47.316 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:47.316 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:47.317 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:47.317 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.317 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmxattr_latency": { 2024-09-19T05:15:47.317 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:47.317 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:47.317 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:47.317 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.317 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setattr_latency": { 2024-09-19T05:15:47.317 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:47.317 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:47.317 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:47.317 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.317 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setdirlayout_latency": { 2024-09-19T05:15:47.317 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:47.318 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:47.318 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:47.318 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.318 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setfilelock_latency": { 2024-09-19T05:15:47.318 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:47.318 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:47.318 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:47.318 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.318 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setlayout_latency": { 2024-09-19T05:15:47.318 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:47.318 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:47.318 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:47.318 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.318 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setxattr_latency": { 2024-09-19T05:15:47.318 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:47.319 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:47.319 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:47.319 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.319 INFO:teuthology.orchestra.run.smithi151.stdout: "req_symlink_latency": { 2024-09-19T05:15:47.319 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:47.319 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:47.319 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:47.319 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.319 INFO:teuthology.orchestra.run.smithi151.stdout: "req_unlink_latency": { 2024-09-19T05:15:47.319 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:47.319 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:47.319 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:47.319 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.319 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_revoke_eviction": 0, 2024-09-19T05:15:47.320 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_acquisition_throttle": 0, 2024-09-19T05:15:47.320 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getvxattr_latency": { 2024-09-19T05:15:47.320 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:47.320 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:47.320 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:47.320 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T05:15:47.320 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.320 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_sessions": { 2024-09-19T05:15:47.320 INFO:teuthology.orchestra.run.smithi151.stdout: "session_count": 2, 2024-09-19T05:15:47.320 INFO:teuthology.orchestra.run.smithi151.stdout: "session_add": 5, 2024-09-19T05:15:47.320 INFO:teuthology.orchestra.run.smithi151.stdout: "session_remove": 3, 2024-09-19T05:15:47.320 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_open": 4, 2024-09-19T05:15:47.320 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_stale": 0, 2024-09-19T05:15:47.320 INFO:teuthology.orchestra.run.smithi151.stdout: "total_load": 0, 2024-09-19T05:15:47.321 INFO:teuthology.orchestra.run.smithi151.stdout: "average_load": 0, 2024-09-19T05:15:47.321 INFO:teuthology.orchestra.run.smithi151.stdout: "avg_session_uptime": 3378, 2024-09-19T05:15:47.321 INFO:teuthology.orchestra.run.smithi151.stdout: "mdthresh_evicted": 0 2024-09-19T05:15:47.321 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.321 INFO:teuthology.orchestra.run.smithi151.stdout: "mempool": { 2024-09-19T05:15:47.321 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_bytes": 0, 2024-09-19T05:15:47.321 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_items": 0, 2024-09-19T05:15:47.321 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T05:15:47.321 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_items": 0, 2024-09-19T05:15:47.321 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T05:15:47.321 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_items": 0, 2024-09-19T05:15:47.321 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T05:15:47.321 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T05:15:47.321 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T05:15:47.322 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T05:15:47.322 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T05:15:47.322 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_items": 0, 2024-09-19T05:15:47.322 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T05:15:47.322 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T05:15:47.322 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_bytes": 0, 2024-09-19T05:15:47.322 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_items": 0, 2024-09-19T05:15:47.322 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_bytes": 0, 2024-09-19T05:15:47.322 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_items": 0, 2024-09-19T05:15:47.322 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T05:15:47.322 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T05:15:47.322 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T05:15:47.322 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T05:15:47.322 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T05:15:47.322 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_items": 0, 2024-09-19T05:15:47.323 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_bytes": 0, 2024-09-19T05:15:47.323 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_items": 0, 2024-09-19T05:15:47.323 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T05:15:47.323 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T05:15:47.323 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_bytes": 0, 2024-09-19T05:15:47.323 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_items": 0, 2024-09-19T05:15:47.323 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_bytes": 0, 2024-09-19T05:15:47.323 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_items": 0, 2024-09-19T05:15:47.323 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T05:15:47.323 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_items": 0, 2024-09-19T05:15:47.323 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T05:15:47.323 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_items": 0, 2024-09-19T05:15:47.323 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_bytes": 190179, 2024-09-19T05:15:47.323 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_items": 53, 2024-09-19T05:15:47.324 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_bytes": 0, 2024-09-19T05:15:47.324 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_items": 0, 2024-09-19T05:15:47.324 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_bytes": 0, 2024-09-19T05:15:47.324 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_items": 0, 2024-09-19T05:15:47.324 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_bytes": 0, 2024-09-19T05:15:47.324 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_items": 0, 2024-09-19T05:15:47.324 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_bytes": 0, 2024-09-19T05:15:47.324 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_items": 0, 2024-09-19T05:15:47.324 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_bytes": 8224, 2024-09-19T05:15:47.324 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_items": 181, 2024-09-19T05:15:47.324 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T05:15:47.324 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_items": 0, 2024-09-19T05:15:47.324 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_bytes": 0, 2024-09-19T05:15:47.324 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_items": 0, 2024-09-19T05:15:47.325 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_bytes": 8512, 2024-09-19T05:15:47.325 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_items": 13, 2024-09-19T05:15:47.325 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_bytes": 0, 2024-09-19T05:15:47.325 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_items": 0, 2024-09-19T05:15:47.325 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_bytes": 0, 2024-09-19T05:15:47.325 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_items": 0 2024-09-19T05:15:47.325 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.325 INFO:teuthology.orchestra.run.smithi151.stdout: "objecter": { 2024-09-19T05:15:47.325 INFO:teuthology.orchestra.run.smithi151.stdout: "op_active": 0, 2024-09-19T05:15:47.325 INFO:teuthology.orchestra.run.smithi151.stdout: "op_laggy": 0, 2024-09-19T05:15:47.325 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send": 2286, 2024-09-19T05:15:47.325 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send_bytes": 21, 2024-09-19T05:15:47.325 INFO:teuthology.orchestra.run.smithi151.stdout: "op_resend": 3, 2024-09-19T05:15:47.325 INFO:teuthology.orchestra.run.smithi151.stdout: "op_reply": 2286, 2024-09-19T05:15:47.326 INFO:teuthology.orchestra.run.smithi151.stdout: "op_latency": { 2024-09-19T05:15:47.326 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 2286, 2024-09-19T05:15:47.326 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 1.059986775, 2024-09-19T05:15:47.326 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000463686 2024-09-19T05:15:47.326 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.326 INFO:teuthology.orchestra.run.smithi151.stdout: "op_inflight": 0, 2024-09-19T05:15:47.326 INFO:teuthology.orchestra.run.smithi151.stdout: "oplen_avg": { 2024-09-19T05:15:47.326 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 2286, 2024-09-19T05:15:47.326 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 2287 2024-09-19T05:15:47.326 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.326 INFO:teuthology.orchestra.run.smithi151.stdout: "op": 2286, 2024-09-19T05:15:47.326 INFO:teuthology.orchestra.run.smithi151.stdout: "op_r": 2286, 2024-09-19T05:15:47.326 INFO:teuthology.orchestra.run.smithi151.stdout: "op_w": 0, 2024-09-19T05:15:47.326 INFO:teuthology.orchestra.run.smithi151.stdout: "op_rmw": 0, 2024-09-19T05:15:47.327 INFO:teuthology.orchestra.run.smithi151.stdout: "op_pg": 0, 2024-09-19T05:15:47.327 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_stat": 1486, 2024-09-19T05:15:47.327 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_create": 0, 2024-09-19T05:15:47.327 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_read": 798, 2024-09-19T05:15:47.327 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_write": 0, 2024-09-19T05:15:47.327 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writefull": 0, 2024-09-19T05:15:47.327 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writesame": 0, 2024-09-19T05:15:47.327 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_append": 0, 2024-09-19T05:15:47.327 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_zero": 0, 2024-09-19T05:15:47.327 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_truncate": 0, 2024-09-19T05:15:47.327 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_delete": 0, 2024-09-19T05:15:47.327 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_mapext": 0, 2024-09-19T05:15:47.327 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_sparse_read": 0, 2024-09-19T05:15:47.328 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_clonerange": 0, 2024-09-19T05:15:47.328 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_getxattr": 1, 2024-09-19T05:15:47.328 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_setxattr": 0, 2024-09-19T05:15:47.328 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_cmpxattr": 0, 2024-09-19T05:15:47.328 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_rmxattr": 0, 2024-09-19T05:15:47.328 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_resetxattrs": 0, 2024-09-19T05:15:47.328 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_call": 0, 2024-09-19T05:15:47.328 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_watch": 0, 2024-09-19T05:15:47.328 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_notify": 0, 2024-09-19T05:15:47.328 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T05:15:47.328 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls": 0, 2024-09-19T05:15:47.328 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls_filter": 0, 2024-09-19T05:15:47.328 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_other": 0, 2024-09-19T05:15:47.328 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_active": 0, 2024-09-19T05:15:47.329 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_send": 0, 2024-09-19T05:15:47.329 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_resend": 0, 2024-09-19T05:15:47.329 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_ping": 0, 2024-09-19T05:15:47.329 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_active": 0, 2024-09-19T05:15:47.329 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_send": 0, 2024-09-19T05:15:47.329 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_resend": 0, 2024-09-19T05:15:47.329 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_active": 0, 2024-09-19T05:15:47.329 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_send": 0, 2024-09-19T05:15:47.329 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_resend": 0, 2024-09-19T05:15:47.329 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_active": 0, 2024-09-19T05:15:47.329 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_send": 0, 2024-09-19T05:15:47.329 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_resend": 0, 2024-09-19T05:15:47.329 INFO:teuthology.orchestra.run.smithi151.stdout: "command_active": 0, 2024-09-19T05:15:47.330 INFO:teuthology.orchestra.run.smithi151.stdout: "command_send": 0, 2024-09-19T05:15:47.330 INFO:teuthology.orchestra.run.smithi151.stdout: "command_resend": 0, 2024-09-19T05:15:47.330 INFO:teuthology.orchestra.run.smithi151.stdout: "map_epoch": 96, 2024-09-19T05:15:47.330 INFO:teuthology.orchestra.run.smithi151.stdout: "map_full": 0, 2024-09-19T05:15:47.330 INFO:teuthology.orchestra.run.smithi151.stdout: "map_inc": 13, 2024-09-19T05:15:47.330 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_sessions": 5, 2024-09-19T05:15:47.330 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_open": 5, 2024-09-19T05:15:47.330 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_close": 0, 2024-09-19T05:15:47.330 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_laggy": 0, 2024-09-19T05:15:47.330 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_wr": 0, 2024-09-19T05:15:47.330 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_rd": 2, 2024-09-19T05:15:47.330 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_del": 0 2024-09-19T05:15:47.330 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.331 INFO:teuthology.orchestra.run.smithi151.stdout: "oft": { 2024-09-19T05:15:47.331 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_objs": 0, 2024-09-19T05:15:47.331 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_kv_pairs": 0, 2024-09-19T05:15:47.331 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_updates": 0, 2024-09-19T05:15:47.331 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_removes": 0 2024-09-19T05:15:47.331 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.331 INFO:teuthology.orchestra.run.smithi151.stdout: "osd-slow-ops": { 2024-09-19T05:15:47.331 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_ops_count": 0 2024-09-19T05:15:47.331 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.331 INFO:teuthology.orchestra.run.smithi151.stdout: "purge_queue": { 2024-09-19T05:15:47.331 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops": 0, 2024-09-19T05:15:47.331 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T05:15:47.331 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing": 0, 2024-09-19T05:15:47.331 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_high_water": 0, 2024-09-19T05:15:47.332 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed_ops": 0, 2024-09-19T05:15:47.332 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed": 0, 2024-09-19T05:15:47.332 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_item_in_journal": 0 2024-09-19T05:15:47.332 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.332 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T05:15:47.332 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-19T05:15:47.332 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-09-19T05:15:47.332 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-19T05:15:47.332 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 2546, 2024-09-19T05:15:47.332 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 2035739, 2024-09-19T05:15:47.332 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:47.332 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 2546, 2024-09-19T05:15:47.332 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-09-19T05:15:47.333 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-09-19T05:15:47.333 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 2546, 2024-09-19T05:15:47.333 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 2035739, 2024-09-19T05:15:47.333 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-19T05:15:47.333 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:47.333 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:47.333 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:47.333 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T05:15:47.333 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.333 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_bytes": { 2024-09-19T05:15:47.333 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-19T05:15:47.333 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-09-19T05:15:47.333 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-19T05:15:47.333 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-09-19T05:15:47.333 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-09-19T05:15:47.334 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:47.334 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-09-19T05:15:47.334 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 2286, 2024-09-19T05:15:47.334 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 2899, 2024-09-19T05:15:47.334 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 2286, 2024-09-19T05:15:47.334 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 2899, 2024-09-19T05:15:47.334 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-19T05:15:47.334 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:47.334 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:47.334 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:47.334 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T05:15:47.334 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.335 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_ops": { 2024-09-19T05:15:47.335 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-19T05:15:47.335 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1024, 2024-09-19T05:15:47.335 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-19T05:15:47.335 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-09-19T05:15:47.335 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-09-19T05:15:47.335 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:47.335 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-09-19T05:15:47.335 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 2286, 2024-09-19T05:15:47.335 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 2286, 2024-09-19T05:15:47.335 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 2286, 2024-09-19T05:15:47.335 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 2286, 2024-09-19T05:15:47.335 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-19T05:15:47.335 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:47.335 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:47.336 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:47.336 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T05:15:47.336 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.336 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle": { 2024-09-19T05:15:47.336 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-19T05:15:47.336 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-09-19T05:15:47.336 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-19T05:15:47.336 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-09-19T05:15:47.336 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-09-19T05:15:47.336 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:47.336 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-09-19T05:15:47.336 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-09-19T05:15:47.336 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-09-19T05:15:47.336 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 0, 2024-09-19T05:15:47.337 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 0, 2024-09-19T05:15:47.337 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-19T05:15:47.337 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:47.337 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:47.337 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:47.337 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T05:15:47.337 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T05:15:47.337 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle-0x55edb7bd4b40": { 2024-09-19T05:15:47.337 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-19T05:15:47.337 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-09-19T05:15:47.337 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-19T05:15:47.337 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-09-19T05:15:47.337 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-09-19T05:15:47.337 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-19T05:15:47.337 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-09-19T05:15:47.338 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-09-19T05:15:47.338 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-09-19T05:15:47.338 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 0, 2024-09-19T05:15:47.338 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 0, 2024-09-19T05:15:47.338 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-19T05:15:47.338 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T05:15:47.338 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T05:15:47.338 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T05:15:47.338 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T05:15:47.338 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T05:15:47.338 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-19T05:15:48.176 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-19T05:15:48.176 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-19T05:15:48.176 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-09-19T05:15:48.176 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-09-19T05:15:48.176 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T05:15:48.176 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-19T05:15:48.186 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-19T05:15:48.187 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-19T05:15:48.187 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T05:15:48.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:47 smithi096 ceph-mon[35085]: pgmap v347: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:15:48.229 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-19T05:15:48.229 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T05:15:48.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:47 smithi043 ceph-mon[30800]: pgmap v347: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:15:48.308 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-19T05:15:48.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:47 smithi151 ceph-mon[35425]: pgmap v347: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:15:48.733 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:15:49.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.320+0000 7f833c800640 1 -- 172.21.15.43:0/2101483093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8334112a30 msgr2=0x7f8334114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:49.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.320+0000 7f833c800640 1 --2- 172.21.15.43:0/2101483093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8334112a30 0x7f8334114e20 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f8330009ca0 tx=0x7f833002a8c0 comp rx=0 tx=0).stop 2024-09-19T05:15:49.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.320+0000 7f833c800640 1 -- 172.21.15.43:0/2101483093 shutdown_connections 2024-09-19T05:15:49.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.320+0000 7f833c800640 1 --2- 172.21.15.43:0/2101483093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8334112a30 0x7f8334114e20 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:49.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.320+0000 7f833c800640 1 --2- 172.21.15.43:0/2101483093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8334077040 0x7f83341124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:49.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.320+0000 7f833c800640 1 --2- 172.21.15.43:0/2101483093 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8334076720 0x7f8334076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:49.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.320+0000 7f833c800640 1 -- 172.21.15.43:0/2101483093 >> 172.21.15.43:0/2101483093 conn(0x7f83340ffa90 msgr2=0x7f8334101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:49.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.320+0000 7f833c800640 1 -- 172.21.15.43:0/2101483093 shutdown_connections 2024-09-19T05:15:49.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.320+0000 7f833c800640 1 -- 172.21.15.43:0/2101483093 wait complete. 2024-09-19T05:15:49.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.321+0000 7f833c800640 1 Processor -- start 2024-09-19T05:15:49.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.321+0000 7f833c800640 1 -- start start 2024-09-19T05:15:49.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.322+0000 7f833c800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8334076720 0x7f83341a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:49.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.322+0000 7f833c800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8334077040 0x7f83341a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:49.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.322+0000 7f833c800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8334112a30 0x7f833419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:49.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.322+0000 7f833c800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f83341185c0 con 0x7f8334112a30 2024-09-19T05:15:49.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.322+0000 7f833c800640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8334118440 con 0x7f8334076720 2024-09-19T05:15:49.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.322+0000 7f833c800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8334118740 con 0x7f8334077040 2024-09-19T05:15:49.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.322+0000 7f833a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8334112a30 0x7f833419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:49.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.322+0000 7f8339400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8334077040 0x7f83341a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:49.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.322+0000 7f833a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8334112a30 0x7f833419e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36622/0 (socket says 172.21.15.43:36622) 2024-09-19T05:15:49.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.322+0000 7f8339e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8334076720 0x7f83341a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:49.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.322+0000 7f833a800640 1 -- 172.21.15.43:0/2281635525 learned_addr learned my addr 172.21.15.43:0/2281635525 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:15:49.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.322+0000 7f8339400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8334077040 0x7f83341a4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:38888/0 (socket says 172.21.15.43:38888) 2024-09-19T05:15:49.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.323+0000 7f8339400640 1 -- 172.21.15.43:0/2281635525 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8334076720 msgr2=0x7f83341a46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:49.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.323+0000 7f8339400640 1 --2- 172.21.15.43:0/2281635525 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8334076720 0x7f83341a46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:49.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.323+0000 7f8339400640 1 -- 172.21.15.43:0/2281635525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8334112a30 msgr2=0x7f833419e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:49.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.323+0000 7f8339400640 1 --2- 172.21.15.43:0/2281635525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8334112a30 0x7f833419e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:49.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.323+0000 7f8339400640 1 -- 172.21.15.43:0/2281635525 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f833419ef90 con 0x7f8334077040 2024-09-19T05:15:49.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.323+0000 7f833a800640 1 --2- 172.21.15.43:0/2281635525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8334112a30 0x7f833419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:15:49.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.323+0000 7f8339400640 1 --2- 172.21.15.43:0/2281635525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8334077040 0x7f83341a4be0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f832400d6d0 tx=0x7f832400db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:49.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.323+0000 7f832ea00640 1 -- 172.21.15.43:0/2281635525 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8324017070 con 0x7f8334077040 2024-09-19T05:15:49.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.323+0000 7f832ea00640 1 -- 172.21.15.43:0/2281635525 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8324009820 con 0x7f8334077040 2024-09-19T05:15:49.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.323+0000 7f833c800640 1 -- 172.21.15.43:0/2281635525 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f833419f280 con 0x7f8334077040 2024-09-19T05:15:49.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.323+0000 7f832ea00640 1 -- 172.21.15.43:0/2281635525 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8324004d00 con 0x7f8334077040 2024-09-19T05:15:49.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.324+0000 7f833c800640 1 -- 172.21.15.43:0/2281635525 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83341ab570 con 0x7f8334077040 2024-09-19T05:15:49.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.325+0000 7f833c800640 1 -- 172.21.15.43:0/2281635525 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8334076b00 con 0x7f8334077040 2024-09-19T05:15:49.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.327+0000 7f832ea00640 1 -- 172.21.15.43:0/2281635525 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f83240099d0 con 0x7f8334077040 2024-09-19T05:15:49.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.328+0000 7f832ea00640 1 --2- 172.21.15.43:0/2281635525 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f831c0a33b0 0x7f831c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:49.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.328+0000 7f832ea00640 1 -- 172.21.15.43:0/2281635525 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7f83240cf7d0 con 0x7f8334077040 2024-09-19T05:15:49.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.328+0000 7f8339e00640 1 --2- 172.21.15.43:0/2281635525 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f831c0a33b0 0x7f831c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:49.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.329+0000 7f8339e00640 1 --2- 172.21.15.43:0/2281635525 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f831c0a33b0 0x7f831c0a5870 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f833419f9a0 tx=0x7f8328005f90 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:49.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.330+0000 7f832ea00640 1 -- 172.21.15.43:0/2281635525 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f832409baf0 con 0x7f8334077040 2024-09-19T05:15:49.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.531+0000 7f833c800640 1 -- 172.21.15.43:0/2281635525 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f833402d7c0 con 0x7f8334077040 2024-09-19T05:15:49.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.532+0000 7f832ea00640 1 -- 172.21.15.43:0/2281635525 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v96) ==== 81+0+937 (secure 0 0 0) 0x7f832409fdf0 con 0x7f8334077040 2024-09-19T05:15:49.535 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/3819455723 2024-09-20T05:04:38.237695+0000 2024-09-19T05:15:49.535 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/3712369735 2024-09-20T05:04:38.237695+0000 2024-09-19T05:15:49.535 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/330854185 2024-09-20T04:54:39.548065+0000 2024-09-19T05:15:49.535 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/6132665 2024-09-20T04:54:39.548065+0000 2024-09-19T05:15:49.535 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/3730884138 2024-09-20T05:04:38.237695+0000 2024-09-19T05:15:49.535 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/632163478 2024-09-20T04:54:39.548065+0000 2024-09-19T05:15:49.535 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/4271788315 2024-09-20T04:55:07.037117+0000 2024-09-19T05:15:49.535 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6800/587463881 2024-09-20T04:54:39.548065+0000 2024-09-19T05:15:49.535 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6800/3241799377 2024-09-20T04:55:07.037117+0000 2024-09-19T05:15:49.535 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6801/3241799377 2024-09-20T04:55:07.037117+0000 2024-09-19T05:15:49.535 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/898247657 2024-09-20T04:55:07.037117+0000 2024-09-19T05:15:49.536 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6801/3167108925 2024-09-20T05:04:38.237695+0000 2024-09-19T05:15:49.536 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/515912755 2024-09-20T05:04:38.237695+0000 2024-09-19T05:15:49.536 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6801/587463881 2024-09-20T04:54:39.548065+0000 2024-09-19T05:15:49.536 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/3256350596 2024-09-20T04:55:07.037117+0000 2024-09-19T05:15:49.536 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6800/3167108925 2024-09-20T05:04:38.237695+0000 2024-09-19T05:15:49.536 INFO:teuthology.orchestra.run.smithi043.stderr:listed 16 entries 2024-09-19T05:15:49.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.535+0000 7f833c800640 1 -- 172.21.15.43:0/2281635525 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f831c0a33b0 msgr2=0x7f831c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:49.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.535+0000 7f833c800640 1 --2- 172.21.15.43:0/2281635525 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f831c0a33b0 0x7f831c0a5870 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f833419f9a0 tx=0x7f8328005f90 comp rx=0 tx=0).stop 2024-09-19T05:15:49.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.535+0000 7f833c800640 1 -- 172.21.15.43:0/2281635525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8334077040 msgr2=0x7f83341a4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:49.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.535+0000 7f833c800640 1 --2- 172.21.15.43:0/2281635525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8334077040 0x7f83341a4be0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f832400d6d0 tx=0x7f832400db90 comp rx=0 tx=0).stop 2024-09-19T05:15:49.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.535+0000 7f833c800640 1 -- 172.21.15.43:0/2281635525 shutdown_connections 2024-09-19T05:15:49.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.535+0000 7f833c800640 1 --2- 172.21.15.43:0/2281635525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8334112a30 0x7f833419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:49.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.535+0000 7f833c800640 1 --2- 172.21.15.43:0/2281635525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8334077040 0x7f83341a4be0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:49.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.535+0000 7f833c800640 1 --2- 172.21.15.43:0/2281635525 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f831c0a33b0 0x7f831c0a5870 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:49.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.535+0000 7f833c800640 1 --2- 172.21.15.43:0/2281635525 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8334076720 0x7f83341a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:49.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.535+0000 7f833c800640 1 -- 172.21.15.43:0/2281635525 >> 172.21.15.43:0/2281635525 conn(0x7f83340ffa90 msgr2=0x7f8334101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:49.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.535+0000 7f833c800640 1 -- 172.21.15.43:0/2281635525 shutdown_connections 2024-09-19T05:15:49.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:49.535+0000 7f833c800640 1 -- 172.21.15.43:0/2281635525 wait complete. 2024-09-19T05:15:50.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:49 smithi096 ceph-mon[35085]: pgmap v348: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:15:50.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:49 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/2281635525' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T05:15:50.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:49 smithi043 ceph-mon[30800]: pgmap v348: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:15:50.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:49 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/2281635525' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T05:15:50.373 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-19T05:15:50.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:15:50.374 DEBUG:teuthology.orchestra.run.smithi043:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-19T05:15:50.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:49 smithi151 ceph-mon[35425]: pgmap v348: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:15:50.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:49 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/2281635525' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T05:15:50.456 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-19T05:15:50.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:15:50.456 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T05:15:50.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:15:50.524 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-09-19T05:15:50.560 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T05:15:50.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:15:50.561 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T05:15:50.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:15:50.659 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link delete ceph-brx 2024-09-19T05:15:50.993 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-19T05:15:51.004 INFO:tasks.cephadm:Teardown begin 2024-09-19T05:15:51.004 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T05:15:51.004 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:15:51.385 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:15:51.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:51.934+0000 7f0828e00640 1 -- 172.21.15.43:0/1029969918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08241047e0 msgr2=0x7f0824108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:51.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:51.934+0000 7f0828e00640 1 --2- 172.21.15.43:0/1029969918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08241047e0 0x7f0824108830 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f0810009900 tx=0x7f081002a3f0 comp rx=0 tx=0).stop 2024-09-19T05:15:51.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:51.934+0000 7f0828e00640 1 -- 172.21.15.43:0/1029969918 shutdown_connections 2024-09-19T05:15:51.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:51.934+0000 7f0828e00640 1 --2- 172.21.15.43:0/1029969918 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0824108f60 0x7f0824110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:51.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:51.934+0000 7f0828e00640 1 --2- 172.21.15.43:0/1029969918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08241047e0 0x7f0824108830 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:51.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:51.934+0000 7f0828e00640 1 --2- 172.21.15.43:0/1029969918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0824103e30 0x7f0824104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:51.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:51.934+0000 7f0828e00640 1 -- 172.21.15.43:0/1029969918 >> 172.21.15.43:0/1029969918 conn(0x7f08240ff8d0 msgr2=0x7f0824101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:51.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:51.934+0000 7f0828e00640 1 -- 172.21.15.43:0/1029969918 shutdown_connections 2024-09-19T05:15:51.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:51.934+0000 7f0828e00640 1 -- 172.21.15.43:0/1029969918 wait complete. 2024-09-19T05:15:51.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:51.935+0000 7f0828e00640 1 Processor -- start 2024-09-19T05:15:51.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:51.935+0000 7f0828e00640 1 -- start start 2024-09-19T05:15:51.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:51.936+0000 7f0828e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0824103e30 0x7f082419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:51.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:51.936+0000 7f0828e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08241047e0 0x7f082419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:51.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:51.936+0000 7f0828e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0824108f60 0x7f082419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:51.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:51.936+0000 7f0828e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0824114010 con 0x7f08241047e0 2024-09-19T05:15:51.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:51.936+0000 7f0828e00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0824113e90 con 0x7f0824108f60 2024-09-19T05:15:51.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:51.936+0000 7f0828e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0824114190 con 0x7f0824103e30 2024-09-19T05:15:51.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:51.936+0000 7f0822a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08241047e0 0x7f082419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:51.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:51.936+0000 7f0823400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0824103e30 0x7f082419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:51.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:51.936+0000 7f0822a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08241047e0 0x7f082419a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36634/0 (socket says 172.21.15.43:36634) 2024-09-19T05:15:51.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:51.936+0000 7f0822a00640 1 -- 172.21.15.43:0/3118886498 learned_addr learned my addr 172.21.15.43:0/3118886498 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:15:51.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:51.936+0000 7f0823e00640 1 --2- 172.21.15.43:0/3118886498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0824108f60 0x7f082419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:51.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:51.937+0000 7f0823400640 1 -- 172.21.15.43:0/3118886498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0824108f60 msgr2=0x7f082419ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:51.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:51.937+0000 7f0823400640 1 --2- 172.21.15.43:0/3118886498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0824108f60 0x7f082419ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:51.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:51.937+0000 7f0823400640 1 -- 172.21.15.43:0/3118886498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08241047e0 msgr2=0x7f082419a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:51.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:51.937+0000 7f0823400640 1 --2- 172.21.15.43:0/3118886498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08241047e0 0x7f082419a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:51.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:51.937+0000 7f0823400640 1 -- 172.21.15.43:0/3118886498 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f082419f420 con 0x7f0824103e30 2024-09-19T05:15:51.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:51.937+0000 7f0823400640 1 --2- 172.21.15.43:0/3118886498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0824103e30 0x7f082419a470 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f081400e9f0 tx=0x7f081400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:51.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:51.937+0000 7f081be00640 1 -- 172.21.15.43:0/3118886498 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081400ccc0 con 0x7f0824103e30 2024-09-19T05:15:51.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:51.937+0000 7f081be00640 1 -- 172.21.15.43:0/3118886498 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0814010910 con 0x7f0824103e30 2024-09-19T05:15:51.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:51.937+0000 7f0828e00640 1 -- 172.21.15.43:0/3118886498 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f082419f710 con 0x7f0824103e30 2024-09-19T05:15:51.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:51.937+0000 7f081be00640 1 -- 172.21.15.43:0/3118886498 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0814018960 con 0x7f0824103e30 2024-09-19T05:15:51.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:51.937+0000 7f0828e00640 1 -- 172.21.15.43:0/3118886498 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08241a6ff0 con 0x7f0824103e30 2024-09-19T05:15:51.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:51.938+0000 7f0828e00640 1 -- 172.21.15.43:0/3118886498 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0824104210 con 0x7f0824103e30 2024-09-19T05:15:51.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:51.941+0000 7f081be00640 1 -- 172.21.15.43:0/3118886498 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0814010ac0 con 0x7f0824103e30 2024-09-19T05:15:51.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:51.942+0000 7f081be00640 1 --2- 172.21.15.43:0/3118886498 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f08080a3430 0x7f08080a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:51.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:51.942+0000 7f081be00640 1 -- 172.21.15.43:0/3118886498 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7f08140d05e0 con 0x7f0824103e30 2024-09-19T05:15:51.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:51.942+0000 7f0822a00640 1 --2- 172.21.15.43:0/3118886498 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f08080a3430 0x7f08080a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:51.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:51.943+0000 7f0822a00640 1 --2- 172.21.15.43:0/3118886498 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f08080a3430 0x7f08080a58f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f08100096c0 tx=0x7f08100046c0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:51.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:51.944+0000 7f081be00640 1 -- 172.21.15.43:0/3118886498 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f081409cf50 con 0x7f0824103e30 2024-09-19T05:15:52.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:52.168+0000 7f0828e00640 1 -- 172.21.15.43:0/3118886498 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f082410c140 con 0x7f0824103e30 2024-09-19T05:15:52.169 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:51 smithi043 ceph-mon[30800]: pgmap v349: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:15:52.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:52.169+0000 7f081be00640 1 -- 172.21.15.43:0/3118886498 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11965 (secure 0 0 0) 0x7f081409c850 con 0x7f0824103e30 2024-09-19T05:15:52.170 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:15:52.170 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":29,"btime":"2024-09-19T05:04:38:785407+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/773801169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":773801169},{"type":"v1","addr":"172.21.15.43:6835","nonce":773801169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1799137902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1799137902},{"type":"v1","addr":"172.21.15.43:6837","nonce":1799137902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:02:25.463041+0000","modified":"2024-09-19T05:04:38.039809+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24518,"mds_2":24511,"mds_3":24506,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14541":{"gid":14541,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/794881135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":794881135},{"type":"v1","addr":"172.21.15.43:6841","nonce":794881135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/2836453158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2836453158},{"type":"v1","addr":"172.21.15.151:6833","nonce":2836453158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/178785812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":178785812},{"type":"v1","addr":"172.21.15.96:6833","nonce":178785812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1671043820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1671043820},{"type":"v1","addr":"172.21.15.96:6835","nonce":1671043820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2972514328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2972514328},{"type":"v1","addr":"172.21.15.151:6835","nonce":2972514328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.151:6837/3960378010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3960378010},{"type":"v1","addr":"172.21.15.151:6837","nonce":3960378010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6837/3978388286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3978388286},{"type":"v1","addr":"172.21.15.96:6837","nonce":3978388286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.151:6839/3794104313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3794104313},{"type":"v1","addr":"172.21.15.151:6839","nonce":3794104313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.43:6839/3404555030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3404555030},{"type":"v1","addr":"172.21.15.43:6839","nonce":3404555030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/605527621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":605527621},{"type":"v1","addr":"172.21.15.96:6839","nonce":605527621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24518,24511,24506,24493]},"id":1}]} 2024-09-19T05:15:52.171 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 29 2024-09-19T05:15:52.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:52.171+0000 7f0828e00640 1 -- 172.21.15.43:0/3118886498 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f08080a3430 msgr2=0x7f08080a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:52.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:52.171+0000 7f0828e00640 1 --2- 172.21.15.43:0/3118886498 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f08080a3430 0x7f08080a58f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f08100096c0 tx=0x7f08100046c0 comp rx=0 tx=0).stop 2024-09-19T05:15:52.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:52.171+0000 7f0828e00640 1 -- 172.21.15.43:0/3118886498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0824103e30 msgr2=0x7f082419a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:52.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:52.171+0000 7f0828e00640 1 --2- 172.21.15.43:0/3118886498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0824103e30 0x7f082419a470 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f081400e9f0 tx=0x7f081400eeb0 comp rx=0 tx=0).stop 2024-09-19T05:15:52.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:52.171+0000 7f0828e00640 1 -- 172.21.15.43:0/3118886498 shutdown_connections 2024-09-19T05:15:52.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:52.171+0000 7f0828e00640 1 --2- 172.21.15.43:0/3118886498 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f08080a3430 0x7f08080a58f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:52.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:52.171+0000 7f0828e00640 1 --2- 172.21.15.43:0/3118886498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0824108f60 0x7f082419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:52.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:52.171+0000 7f0828e00640 1 --2- 172.21.15.43:0/3118886498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08241047e0 0x7f082419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:52.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:52.171+0000 7f0828e00640 1 --2- 172.21.15.43:0/3118886498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0824103e30 0x7f082419a470 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:52.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:52.171+0000 7f0828e00640 1 -- 172.21.15.43:0/3118886498 >> 172.21.15.43:0/3118886498 conn(0x7f08240ff8d0 msgr2=0x7f0824100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:52.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:52.172+0000 7f0828e00640 1 -- 172.21.15.43:0/3118886498 shutdown_connections 2024-09-19T05:15:52.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:52.172+0000 7f0828e00640 1 -- 172.21.15.43:0/3118886498 wait complete. 2024-09-19T05:15:52.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:51 smithi151 ceph-mon[35425]: pgmap v349: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:15:52.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:51 smithi096 ceph-mon[35085]: pgmap v349: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T05:15:53.053 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T05:15:53.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:52 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/3118886498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:15:53.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:52 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/3118886498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:15:53.435 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:15:53.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:52 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/3118886498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:15:54.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.020+0000 7f3908c00640 1 -- 172.21.15.43:0/1815209928 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3904102d40 msgr2=0x7f3904103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:54.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.020+0000 7f3908c00640 1 --2- 172.21.15.43:0/1815209928 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3904102d40 0x7f3904103120 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f38f4009900 tx=0x7f38f402a480 comp rx=0 tx=0).stop 2024-09-19T05:15:54.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.020+0000 7f3908c00640 1 -- 172.21.15.43:0/1815209928 shutdown_connections 2024-09-19T05:15:54.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.020+0000 7f3908c00640 1 --2- 172.21.15.43:0/1815209928 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f390410c170 0x7f390410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:54.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.020+0000 7f3908c00640 1 --2- 172.21.15.43:0/1815209928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3904103660 0x7f390410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:54.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.020+0000 7f3908c00640 1 --2- 172.21.15.43:0/1815209928 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3904102d40 0x7f3904103120 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:54.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.020+0000 7f3908c00640 1 -- 172.21.15.43:0/1815209928 >> 172.21.15.43:0/1815209928 conn(0x7f39040fc7a0 msgr2=0x7f39040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:54.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.020+0000 7f3908c00640 1 -- 172.21.15.43:0/1815209928 shutdown_connections 2024-09-19T05:15:54.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.020+0000 7f3908c00640 1 -- 172.21.15.43:0/1815209928 wait complete. 2024-09-19T05:15:54.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.021+0000 7f3908c00640 1 Processor -- start 2024-09-19T05:15:54.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.022+0000 7f3908c00640 1 -- start start 2024-09-19T05:15:54.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.022+0000 7f3908c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3904102d40 0x7f3904198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:54.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.022+0000 7f3908c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3904103660 0x7f3904198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:54.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.022+0000 7f3908c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f390410c170 0x7f390419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:54.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.022+0000 7f3908c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3904111fc0 con 0x7f390410c170 2024-09-19T05:15:54.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.022+0000 7f3908c00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3904111e40 con 0x7f3904103660 2024-09-19T05:15:54.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.022+0000 7f3908c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3904112140 con 0x7f3904102d40 2024-09-19T05:15:54.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.022+0000 7f3901e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3904102d40 0x7f3904198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:54.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.022+0000 7f3902800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f390410c170 0x7f390419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:54.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.022+0000 7f3901e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3904102d40 0x7f3904198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.43:38902/0 (socket says 172.21.15.43:38902) 2024-09-19T05:15:54.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.022+0000 7f3901e00640 1 -- 172.21.15.43:0/448450015 learned_addr learned my addr 172.21.15.43:0/448450015 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:15:54.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.023+0000 7f38fbe00640 1 --2- 172.21.15.43:0/448450015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3904103660 0x7f3904198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:54.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.023+0000 7f3901e00640 1 -- 172.21.15.43:0/448450015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3904103660 msgr2=0x7f3904198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:54.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.023+0000 7f3901e00640 1 --2- 172.21.15.43:0/448450015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3904103660 0x7f3904198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:54.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.023+0000 7f3901e00640 1 -- 172.21.15.43:0/448450015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f390410c170 msgr2=0x7f390419cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:54.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.023+0000 7f3901e00640 1 --2- 172.21.15.43:0/448450015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f390410c170 0x7f390419cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:54.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.023+0000 7f3901e00640 1 -- 172.21.15.43:0/448450015 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f390419d3f0 con 0x7f3904102d40 2024-09-19T05:15:54.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.023+0000 7f3902800640 1 --2- 172.21.15.43:0/448450015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f390410c170 0x7f390419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:15:54.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.023+0000 7f3901e00640 1 --2- 172.21.15.43:0/448450015 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3904102d40 0x7f3904198440 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f38f4004770 tx=0x7f38f4003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:54.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.024+0000 7f38faa00640 1 -- 172.21.15.43:0/448450015 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f403f070 con 0x7f3904102d40 2024-09-19T05:15:54.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.024+0000 7f38faa00640 1 -- 172.21.15.43:0/448450015 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38f4031ca0 con 0x7f3904102d40 2024-09-19T05:15:54.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.024+0000 7f3908c00640 1 -- 172.21.15.43:0/448450015 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f390419d680 con 0x7f3904102d40 2024-09-19T05:15:54.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.024+0000 7f38faa00640 1 -- 172.21.15.43:0/448450015 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f4035030 con 0x7f3904102d40 2024-09-19T05:15:54.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.024+0000 7f3908c00640 1 -- 172.21.15.43:0/448450015 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39041a4fc0 con 0x7f3904102d40 2024-09-19T05:15:54.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.025+0000 7f3908c00640 1 -- 172.21.15.43:0/448450015 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3904103120 con 0x7f3904102d40 2024-09-19T05:15:54.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.027+0000 7f38faa00640 1 -- 172.21.15.43:0/448450015 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f38f4042070 con 0x7f3904102d40 2024-09-19T05:15:54.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.028+0000 7f38faa00640 1 --2- 172.21.15.43:0/448450015 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f38e80a3430 0x7f38e80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:54.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.028+0000 7f38faa00640 1 -- 172.21.15.43:0/448450015 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7f38f40f0870 con 0x7f3904102d40 2024-09-19T05:15:54.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.028+0000 7f38fbe00640 1 --2- 172.21.15.43:0/448450015 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f38e80a3430 0x7f38e80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:54.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.029+0000 7f38fbe00640 1 --2- 172.21.15.43:0/448450015 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f38e80a3430 0x7f38e80a58f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f390410d5b0 tx=0x7f38ec005f90 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:54.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.030+0000 7f38faa00640 1 -- 172.21.15.43:0/448450015 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f38f40bcc50 con 0x7f3904102d40 2024-09-19T05:15:54.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.252+0000 7f3908c00640 1 -- 172.21.15.43:0/448450015 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39041997a0 con 0x7f3904102d40 2024-09-19T05:15:54.254 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:53 smithi043 ceph-mon[30800]: pgmap v350: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:15:54.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.253+0000 7f38faa00640 1 -- 172.21.15.43:0/448450015 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11965 (secure 0 0 0) 0x7f39041997a0 con 0x7f3904102d40 2024-09-19T05:15:54.255 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:15:54.255 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":29,"btime":"2024-09-19T05:04:38:785407+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/773801169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":773801169},{"type":"v1","addr":"172.21.15.43:6835","nonce":773801169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14493,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1799137902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":1799137902},{"type":"v1","addr":"172.21.15.43:6837","nonce":1799137902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T05:02:25.463041+0000","modified":"2024-09-19T05:04:38.039809+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24518,"mds_2":24511,"mds_3":24506,"mds_4":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14541":{"gid":14541,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6841/794881135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":794881135},{"type":"v1","addr":"172.21.15.43:6841","nonce":794881135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/2836453158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2836453158},{"type":"v1","addr":"172.21.15.151:6833","nonce":2836453158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/178785812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":178785812},{"type":"v1","addr":"172.21.15.96:6833","nonce":178785812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6835/1671043820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1671043820},{"type":"v1","addr":"172.21.15.96:6835","nonce":1671043820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/2972514328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2972514328},{"type":"v1","addr":"172.21.15.151:6835","nonce":2972514328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.151:6837/3960378010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3960378010},{"type":"v1","addr":"172.21.15.151:6837","nonce":3960378010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6837/3978388286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3978388286},{"type":"v1","addr":"172.21.15.96:6837","nonce":3978388286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.151:6839/3794104313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3794104313},{"type":"v1","addr":"172.21.15.151:6839","nonce":3794104313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.43:6839/3404555030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":3404555030},{"type":"v1","addr":"172.21.15.43:6839","nonce":3404555030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/605527621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":605527621},{"type":"v1","addr":"172.21.15.96:6839","nonce":605527621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24518,24511,24506,24493]},"id":1}]} 2024-09-19T05:15:54.255 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 29 2024-09-19T05:15:54.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.256+0000 7f3908c00640 1 -- 172.21.15.43:0/448450015 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f38e80a3430 msgr2=0x7f38e80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:54.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.256+0000 7f3908c00640 1 --2- 172.21.15.43:0/448450015 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f38e80a3430 0x7f38e80a58f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f390410d5b0 tx=0x7f38ec005f90 comp rx=0 tx=0).stop 2024-09-19T05:15:54.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.256+0000 7f3908c00640 1 -- 172.21.15.43:0/448450015 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3904102d40 msgr2=0x7f3904198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:54.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.256+0000 7f3908c00640 1 --2- 172.21.15.43:0/448450015 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3904102d40 0x7f3904198440 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f38f4004770 tx=0x7f38f4003c10 comp rx=0 tx=0).stop 2024-09-19T05:15:54.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.256+0000 7f3908c00640 1 -- 172.21.15.43:0/448450015 shutdown_connections 2024-09-19T05:15:54.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.256+0000 7f3908c00640 1 --2- 172.21.15.43:0/448450015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f390410c170 0x7f390419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:54.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.256+0000 7f3908c00640 1 --2- 172.21.15.43:0/448450015 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f38e80a3430 0x7f38e80a58f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:54.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.256+0000 7f3908c00640 1 --2- 172.21.15.43:0/448450015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3904103660 0x7f3904198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:54.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.256+0000 7f3908c00640 1 --2- 172.21.15.43:0/448450015 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3904102d40 0x7f3904198440 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:54.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.256+0000 7f3908c00640 1 -- 172.21.15.43:0/448450015 >> 172.21.15.43:0/448450015 conn(0x7f39040fc7a0 msgr2=0x7f39040fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:54.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.256+0000 7f3908c00640 1 -- 172.21.15.43:0/448450015 shutdown_connections 2024-09-19T05:15:54.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:54.256+0000 7f3908c00640 1 -- 172.21.15.43:0/448450015 wait complete. 2024-09-19T05:15:54.375 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:53 smithi096 ceph-mon[35085]: pgmap v350: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:15:54.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:53 smithi151 ceph-mon[35425]: pgmap v350: 97 pgs: 97 active+clean; 2.4 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T05:15:55.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:54 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/448450015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:15:55.336 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T05:15:55.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:54 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/448450015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:15:55.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:54 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/448450015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T05:15:55.715 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:15:56.184 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:55 smithi043 ceph-mon[30800]: pgmap v351: 97 pgs: 97 active+clean; 2.4 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-09-19T05:15:56.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.297+0000 7ff581000640 1 -- 172.21.15.43:0/2004928427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff57c1027a0 msgr2=0x7ff57c1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:56.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.297+0000 7ff581000640 1 --2- 172.21.15.43:0/2004928427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff57c1027a0 0x7ff57c1067f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff564009900 tx=0x7ff56402a3f0 comp rx=0 tx=0).stop 2024-09-19T05:15:56.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.297+0000 7ff581000640 1 -- 172.21.15.43:0/2004928427 shutdown_connections 2024-09-19T05:15:56.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.297+0000 7ff581000640 1 --2- 172.21.15.43:0/2004928427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff57c106f20 0x7ff57c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:56.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.297+0000 7ff581000640 1 --2- 172.21.15.43:0/2004928427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff57c1027a0 0x7ff57c1067f0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:56.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.297+0000 7ff581000640 1 --2- 172.21.15.43:0/2004928427 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff57c101df0 0x7ff57c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:56.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.297+0000 7ff581000640 1 -- 172.21.15.43:0/2004928427 >> 172.21.15.43:0/2004928427 conn(0x7ff57c0fd8b0 msgr2=0x7ff57c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:56.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.297+0000 7ff581000640 1 -- 172.21.15.43:0/2004928427 shutdown_connections 2024-09-19T05:15:56.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.297+0000 7ff581000640 1 -- 172.21.15.43:0/2004928427 wait complete. 2024-09-19T05:15:56.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.298+0000 7ff581000640 1 Processor -- start 2024-09-19T05:15:56.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.298+0000 7ff581000640 1 -- start start 2024-09-19T05:15:56.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.299+0000 7ff581000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff57c101df0 0x7ff57c19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:56.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.299+0000 7ff581000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff57c1027a0 0x7ff57c19abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:56.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.299+0000 7ff581000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff57c106f20 0x7ff57c19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:56.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.299+0000 7ff581000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff57c111fd0 con 0x7ff57c1027a0 2024-09-19T05:15:56.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.299+0000 7ff581000640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff57c111e50 con 0x7ff57c101df0 2024-09-19T05:15:56.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.299+0000 7ff581000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff57c112150 con 0x7ff57c106f20 2024-09-19T05:15:56.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.299+0000 7ff573e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff57c1027a0 0x7ff57c19abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:56.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.299+0000 7ff579e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff57c101df0 0x7ff57c19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:56.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.299+0000 7ff573e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff57c1027a0 0x7ff57c19abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50772/0 (socket says 172.21.15.43:50772) 2024-09-19T05:15:56.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.299+0000 7ff573e00640 1 -- 172.21.15.43:0/3280676899 learned_addr learned my addr 172.21.15.43:0/3280676899 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:15:56.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.299+0000 7ff57a800640 1 --2- 172.21.15.43:0/3280676899 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff57c106f20 0x7ff57c19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:56.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.299+0000 7ff579e00640 1 -- 172.21.15.43:0/3280676899 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff57c106f20 msgr2=0x7ff57c19ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:56.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.299+0000 7ff579e00640 1 --2- 172.21.15.43:0/3280676899 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff57c106f20 0x7ff57c19ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:56.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.299+0000 7ff579e00640 1 -- 172.21.15.43:0/3280676899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff57c1027a0 msgr2=0x7ff57c19abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:56.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.299+0000 7ff579e00640 1 --2- 172.21.15.43:0/3280676899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff57c1027a0 0x7ff57c19abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:56.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.299+0000 7ff579e00640 1 -- 172.21.15.43:0/3280676899 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff57c19f640 con 0x7ff57c101df0 2024-09-19T05:15:56.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.300+0000 7ff579e00640 1 --2- 172.21.15.43:0/3280676899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff57c101df0 0x7ff57c19a690 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff56c00e9f0 tx=0x7ff56c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:56.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.300+0000 7ff572a00640 1 -- 172.21.15.43:0/3280676899 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56c00ccc0 con 0x7ff57c101df0 2024-09-19T05:15:56.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.300+0000 7ff581000640 1 -- 172.21.15.43:0/3280676899 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff57c19f930 con 0x7ff57c101df0 2024-09-19T05:15:56.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.300+0000 7ff572a00640 1 -- 172.21.15.43:0/3280676899 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff56c005020 con 0x7ff57c101df0 2024-09-19T05:15:56.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.300+0000 7ff572a00640 1 -- 172.21.15.43:0/3280676899 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56c004a00 con 0x7ff57c101df0 2024-09-19T05:15:56.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.300+0000 7ff581000640 1 -- 172.21.15.43:0/3280676899 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff57c1a7210 con 0x7ff57c101df0 2024-09-19T05:15:56.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.301+0000 7ff581000640 1 -- 172.21.15.43:0/3280676899 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff57c1021d0 con 0x7ff57c101df0 2024-09-19T05:15:56.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.303+0000 7ff572a00640 1 -- 172.21.15.43:0/3280676899 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ff56c0051d0 con 0x7ff57c101df0 2024-09-19T05:15:56.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.304+0000 7ff572a00640 1 --2- 172.21.15.43:0/3280676899 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff5600a33b0 0x7ff5600a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:56.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.304+0000 7ff572a00640 1 -- 172.21.15.43:0/3280676899 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7ff56c0cfcc0 con 0x7ff57c101df0 2024-09-19T05:15:56.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.305+0000 7ff573e00640 1 --2- 172.21.15.43:0/3280676899 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff5600a33b0 0x7ff5600a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:56.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.305+0000 7ff573e00640 1 --2- 172.21.15.43:0/3280676899 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff5600a33b0 0x7ff5600a5870 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff5640096c0 tx=0x7ff5640046c0 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:56.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.308+0000 7ff572a00640 1 -- 172.21.15.43:0/3280676899 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff56c0d6080 con 0x7ff57c101df0 2024-09-19T05:15:56.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:55 smithi151 ceph-mon[35425]: pgmap v351: 97 pgs: 97 active+clean; 2.4 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-09-19T05:15:56.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:55 smithi096 ceph-mon[35085]: pgmap v351: 97 pgs: 97 active+clean; 2.4 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-09-19T05:15:56.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.497+0000 7ff581000640 1 -- 172.21.15.43:0/3280676899 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff57c1a74b0 con 0x7ff57c101df0 2024-09-19T05:15:56.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.499+0000 7ff572a00640 1 -- 172.21.15.43:0/3280676899 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+23128 (secure 0 0 0) 0x7ff56c09c630 con 0x7ff57c101df0 2024-09-19T05:15:56.500 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-19T05:15:56.501 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":96,"fsid":"1456a810-7643-11ef-bceb-c7b262605968","created":"2024-09-19T04:54:14.706644+0000","modified":"2024-09-19T05:15:15.893134+0000","last_up_change":"2024-09-19T05:01:27.348740+0000","last_in_change":"2024-09-19T05:01:03.396477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:57:27.642705+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:02:20.685123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:02:23.393596+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":95,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"81a317a8-7ec8-45c3-bc10-7da023283635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6803","nonce":825351486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6805","nonce":825351486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6809","nonce":825351486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":825351486},{"type":"v1","addr":"172.21.15.43:6807","nonce":825351486}]},"public_addr":"172.21.15.43:6803/825351486","cluster_addr":"172.21.15.43:6805/825351486","heartbeat_back_addr":"172.21.15.43:6809/825351486","heartbeat_front_addr":"172.21.15.43:6807/825351486","state":["exists","up"]},{"osd":1,"uuid":"e96fcfac-4fff-4a36-b38b-aebf57842740","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6801","nonce":2140095130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6803","nonce":2140095130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6807","nonce":2140095130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2140095130},{"type":"v1","addr":"172.21.15.96:6805","nonce":2140095130}]},"public_addr":"172.21.15.96:6801/2140095130","cluster_addr":"172.21.15.96:6803/2140095130","heartbeat_back_addr":"172.21.15.96:6807/2140095130","heartbeat_front_addr":"172.21.15.96:6805/2140095130","state":["exists","up"]},{"osd":2,"uuid":"b659869e-114a-4f56-8aac-a2ed602b4058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6801","nonce":2047147996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6803","nonce":2047147996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6807","nonce":2047147996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":2047147996},{"type":"v1","addr":"172.21.15.151:6805","nonce":2047147996}]},"public_addr":"172.21.15.151:6801/2047147996","cluster_addr":"172.21.15.151:6803/2047147996","heartbeat_back_addr":"172.21.15.151:6807/2047147996","heartbeat_front_addr":"172.21.15.151:6805/2047147996","state":["exists","up"]},{"osd":3,"uuid":"7be64b2c-e45c-493d-a610-4ce41196061f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6811","nonce":1734673762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6813","nonce":1734673762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6817","nonce":1734673762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1734673762},{"type":"v1","addr":"172.21.15.43:6815","nonce":1734673762}]},"public_addr":"172.21.15.43:6811/1734673762","cluster_addr":"172.21.15.43:6813/1734673762","heartbeat_back_addr":"172.21.15.43:6817/1734673762","heartbeat_front_addr":"172.21.15.43:6815/1734673762","state":["exists","up"]},{"osd":4,"uuid":"196d5af1-a6e5-496c-8933-4e58892f4360","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6809","nonce":3184785362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6811","nonce":3184785362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6815","nonce":3184785362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3184785362},{"type":"v1","addr":"172.21.15.96:6813","nonce":3184785362}]},"public_addr":"172.21.15.96:6809/3184785362","cluster_addr":"172.21.15.96:6811/3184785362","heartbeat_back_addr":"172.21.15.96:6815/3184785362","heartbeat_front_addr":"172.21.15.96:6813/3184785362","state":["exists","up"]},{"osd":5,"uuid":"5492d474-8509-4865-bc2e-d08e31558a75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6809","nonce":2878451924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6811","nonce":2878451924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6815","nonce":2878451924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":2878451924},{"type":"v1","addr":"172.21.15.151:6813","nonce":2878451924}]},"public_addr":"172.21.15.151:6809/2878451924","cluster_addr":"172.21.15.151:6811/2878451924","heartbeat_back_addr":"172.21.15.151:6815/2878451924","heartbeat_front_addr":"172.21.15.151:6813/2878451924","state":["exists","up"]},{"osd":6,"uuid":"fb8457ed-7bb9-4435-9d15-c1cbfbddfcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6819","nonce":2008630290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6821","nonce":2008630290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6825","nonce":2008630290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2008630290},{"type":"v1","addr":"172.21.15.43:6823","nonce":2008630290}]},"public_addr":"172.21.15.43:6819/2008630290","cluster_addr":"172.21.15.43:6821/2008630290","heartbeat_back_addr":"172.21.15.43:6825/2008630290","heartbeat_front_addr":"172.21.15.43:6823/2008630290","state":["exists","up"]},{"osd":7,"uuid":"02046db8-30de-4e92-b25a-1f07d6190e91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6817","nonce":3588713275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6819","nonce":3588713275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6823","nonce":3588713275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3588713275},{"type":"v1","addr":"172.21.15.96:6821","nonce":3588713275}]},"public_addr":"172.21.15.96:6817/3588713275","cluster_addr":"172.21.15.96:6819/3588713275","heartbeat_back_addr":"172.21.15.96:6823/3588713275","heartbeat_front_addr":"172.21.15.96:6821/3588713275","state":["exists","up"]},{"osd":8,"uuid":"a5bb4218-51f3-4044-b43e-ebfc764f49ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6817","nonce":725316393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6819","nonce":725316393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6823","nonce":725316393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":725316393},{"type":"v1","addr":"172.21.15.151:6821","nonce":725316393}]},"public_addr":"172.21.15.151:6817/725316393","cluster_addr":"172.21.15.151:6819/725316393","heartbeat_back_addr":"172.21.15.151:6823/725316393","heartbeat_front_addr":"172.21.15.151:6821/725316393","state":["exists","up"]},{"osd":9,"uuid":"6c19f65c-9750-4baa-9d63-7b8bd03e6955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6827","nonce":4053438621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6829","nonce":4053438621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6833","nonce":4053438621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":4053438621},{"type":"v1","addr":"172.21.15.43:6831","nonce":4053438621}]},"public_addr":"172.21.15.43:6827/4053438621","cluster_addr":"172.21.15.43:6829/4053438621","heartbeat_back_addr":"172.21.15.43:6833/4053438621","heartbeat_front_addr":"172.21.15.43:6831/4053438621","state":["exists","up"]},{"osd":10,"uuid":"a7aa7da3-8eab-47fe-a351-298b6ff75f14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6825","nonce":2403210086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6827","nonce":2403210086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6831","nonce":2403210086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":2403210086},{"type":"v1","addr":"172.21.15.96:6829","nonce":2403210086}]},"public_addr":"172.21.15.96:6825/2403210086","cluster_addr":"172.21.15.96:6827/2403210086","heartbeat_back_addr":"172.21.15.96:6831/2403210086","heartbeat_front_addr":"172.21.15.96:6829/2403210086","state":["exists","up"]},{"osd":11,"uuid":"760b6afc-253d-44dd-88e9-dcc97f67c9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6825","nonce":292202881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6827","nonce":292202881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6831","nonce":292202881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":292202881},{"type":"v1","addr":"172.21.15.151:6829","nonce":292202881}]},"public_addr":"172.21.15.151:6825/292202881","cluster_addr":"172.21.15.151:6827/292202881","heartbeat_back_addr":"172.21.15.151:6831/292202881","heartbeat_front_addr":"172.21.15.151:6829/292202881","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:56:34.716991+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:56:59.321174+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:57:24.281696+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:57:53.443880+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:58:19.205873+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:58:44.504873+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:13.650731+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:59:39.137876+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:04.579893+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:34.337312+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:00:59.434658+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:01:24.930529+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/3819455723":"2024-09-20T05:04:38.237695+0000","172.21.15.43:0/3712369735":"2024-09-20T05:04:38.237695+0000","172.21.15.43:0/330854185":"2024-09-20T04:54:39.548065+0000","172.21.15.43:0/6132665":"2024-09-20T04:54:39.548065+0000","172.21.15.43:0/3730884138":"2024-09-20T05:04:38.237695+0000","172.21.15.43:0/632163478":"2024-09-20T04:54:39.548065+0000","172.21.15.43:0/4271788315":"2024-09-20T04:55:07.037117+0000","172.21.15.43:6800/587463881":"2024-09-20T04:54:39.548065+0000","172.21.15.43:6800/3241799377":"2024-09-20T04:55:07.037117+0000","172.21.15.43:6801/3241799377":"2024-09-20T04:55:07.037117+0000","172.21.15.43:0/898247657":"2024-09-20T04:55:07.037117+0000","172.21.15.43:6801/3167108925":"2024-09-20T05:04:38.237695+0000","172.21.15.43:0/515912755":"2024-09-20T05:04:38.237695+0000","172.21.15.43:6801/587463881":"2024-09-20T04:54:39.548065+0000","172.21.15.43:0/3256350596":"2024-09-20T04:55:07.037117+0000","172.21.15.43:6800/3167108925":"2024-09-20T05:04:38.237695+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:15:56.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.501+0000 7ff581000640 1 -- 172.21.15.43:0/3280676899 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff5600a33b0 msgr2=0x7ff5600a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:56.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.501+0000 7ff581000640 1 --2- 172.21.15.43:0/3280676899 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff5600a33b0 0x7ff5600a5870 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff5640096c0 tx=0x7ff5640046c0 comp rx=0 tx=0).stop 2024-09-19T05:15:56.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.501+0000 7ff581000640 1 -- 172.21.15.43:0/3280676899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff57c101df0 msgr2=0x7ff57c19a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:56.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.501+0000 7ff581000640 1 --2- 172.21.15.43:0/3280676899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff57c101df0 0x7ff57c19a690 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff56c00e9f0 tx=0x7ff56c00edf0 comp rx=0 tx=0).stop 2024-09-19T05:15:56.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.501+0000 7ff581000640 1 -- 172.21.15.43:0/3280676899 shutdown_connections 2024-09-19T05:15:56.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.501+0000 7ff581000640 1 --2- 172.21.15.43:0/3280676899 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff57c106f20 0x7ff57c19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:56.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.501+0000 7ff581000640 1 --2- 172.21.15.43:0/3280676899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff57c1027a0 0x7ff57c19abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:56.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.501+0000 7ff581000640 1 --2- 172.21.15.43:0/3280676899 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff5600a33b0 0x7ff5600a5870 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:56.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.501+0000 7ff581000640 1 --2- 172.21.15.43:0/3280676899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff57c101df0 0x7ff57c19a690 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:56.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.501+0000 7ff581000640 1 -- 172.21.15.43:0/3280676899 >> 172.21.15.43:0/3280676899 conn(0x7ff57c0fd8b0 msgr2=0x7ff57c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:56.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.501+0000 7ff581000640 1 -- 172.21.15.43:0/3280676899 shutdown_connections 2024-09-19T05:15:56.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:56.502+0000 7ff581000640 1 -- 172.21.15.43:0/3280676899 wait complete. 2024-09-19T05:15:57.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:57 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/3280676899' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:15:57.337 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs fail cephfs 2024-09-19T05:15:57.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:57 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/3280676899' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:15:57.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:57 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/3280676899' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T05:15:57.716 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:15:58.169 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:58 smithi043 ceph-mon[30800]: pgmap v352: 97 pgs: 97 active+clean; 2.4 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-19T05:15:58.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.282+0000 7fe245400640 1 -- 172.21.15.43:0/3501752686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe240103ff0 msgr2=0x7fe2401043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:58.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.282+0000 7fe245400640 1 --2- 172.21.15.43:0/3501752686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe240103ff0 0x7fe2401043d0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fe230009900 tx=0x7fe23002a410 comp rx=0 tx=0).stop 2024-09-19T05:15:58.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.282+0000 7fe245400640 1 -- 172.21.15.43:0/3501752686 shutdown_connections 2024-09-19T05:15:58.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.282+0000 7fe245400640 1 --2- 172.21.15.43:0/3501752686 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe240109120 0x7fe240110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:58.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.282+0000 7fe245400640 1 --2- 172.21.15.43:0/3501752686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2401049a0 0x7fe2401089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:58.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.282+0000 7fe245400640 1 --2- 172.21.15.43:0/3501752686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe240103ff0 0x7fe2401043d0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:58.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.282+0000 7fe245400640 1 -- 172.21.15.43:0/3501752686 >> 172.21.15.43:0/3501752686 conn(0x7fe2400ffa90 msgr2=0x7fe240101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:58.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.282+0000 7fe245400640 1 -- 172.21.15.43:0/3501752686 shutdown_connections 2024-09-19T05:15:58.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.282+0000 7fe245400640 1 -- 172.21.15.43:0/3501752686 wait complete. 2024-09-19T05:15:58.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.283+0000 7fe245400640 1 Processor -- start 2024-09-19T05:15:58.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.283+0000 7fe245400640 1 -- start start 2024-09-19T05:15:58.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.284+0000 7fe245400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe240103ff0 0x7fe24019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:58.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.284+0000 7fe245400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2401049a0 0x7fe24019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:58.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.284+0000 7fe245400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe240109120 0x7fe24019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:58.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.284+0000 7fe245400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe240114120 con 0x7fe240109120 2024-09-19T05:15:58.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.284+0000 7fe245400640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe240113fa0 con 0x7fe240103ff0 2024-09-19T05:15:58.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.284+0000 7fe245400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe2401142a0 con 0x7fe2401049a0 2024-09-19T05:15:58.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.284+0000 7fe23f200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe240109120 0x7fe24019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:58.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.284+0000 7fe23e800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe240103ff0 0x7fe24019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:58.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.284+0000 7fe23f200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe240109120 0x7fe24019ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50786/0 (socket says 172.21.15.43:50786) 2024-09-19T05:15:58.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.284+0000 7fe23e800640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe240103ff0 0x7fe24019a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:48116/0 (socket says 172.21.15.43:48116) 2024-09-19T05:15:58.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.284+0000 7fe23f200640 1 -- 172.21.15.43:0/1041721037 learned_addr learned my addr 172.21.15.43:0/1041721037 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:15:58.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.284+0000 7fe237e00640 1 --2- 172.21.15.43:0/1041721037 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2401049a0 0x7fe24019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:58.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.285+0000 7fe23e800640 1 -- 172.21.15.43:0/1041721037 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2401049a0 msgr2=0x7fe24019ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:58.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.285+0000 7fe23e800640 1 --2- 172.21.15.43:0/1041721037 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2401049a0 0x7fe24019ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:58.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.285+0000 7fe23e800640 1 -- 172.21.15.43:0/1041721037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe240109120 msgr2=0x7fe24019ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:58.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.285+0000 7fe23e800640 1 --2- 172.21.15.43:0/1041721037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe240109120 0x7fe24019ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:58.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.285+0000 7fe23e800640 1 -- 172.21.15.43:0/1041721037 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe24019f5f0 con 0x7fe240103ff0 2024-09-19T05:15:58.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.285+0000 7fe23f200640 1 --2- 172.21.15.43:0/1041721037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe240109120 0x7fe24019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:15:58.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.285+0000 7fe23e800640 1 --2- 172.21.15.43:0/1041721037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe240103ff0 0x7fe24019a640 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe230005940 tx=0x7fe230004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:58.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.285+0000 7fe237400640 1 -- 172.21.15.43:0/1041721037 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23003f030 con 0x7fe240103ff0 2024-09-19T05:15:58.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.286+0000 7fe245400640 1 -- 172.21.15.43:0/1041721037 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe24019f880 con 0x7fe240103ff0 2024-09-19T05:15:58.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.286+0000 7fe237400640 1 -- 172.21.15.43:0/1041721037 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe230003ad0 con 0x7fe240103ff0 2024-09-19T05:15:58.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.286+0000 7fe237400640 1 -- 172.21.15.43:0/1041721037 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe230035070 con 0x7fe240103ff0 2024-09-19T05:15:58.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.286+0000 7fe245400640 1 -- 172.21.15.43:0/1041721037 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2401a71c0 con 0x7fe240103ff0 2024-09-19T05:15:58.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.287+0000 7fe245400640 1 -- 172.21.15.43:0/1041721037 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2401054d0 con 0x7fe240103ff0 2024-09-19T05:15:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.289+0000 7fe237400640 1 -- 172.21.15.43:0/1041721037 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fe230042030 con 0x7fe240103ff0 2024-09-19T05:15:58.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.290+0000 7fe237400640 1 --2- 172.21.15.43:0/1041721037 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fe2240a33b0 0x7fe2240a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:15:58.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.290+0000 7fe237400640 1 -- 172.21.15.43:0/1041721037 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7fe2300f0580 con 0x7fe240103ff0 2024-09-19T05:15:58.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.290+0000 7fe237e00640 1 --2- 172.21.15.43:0/1041721037 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fe2240a33b0 0x7fe2240a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:15:58.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.291+0000 7fe237e00640 1 --2- 172.21.15.43:0/1041721037 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fe2240a33b0 0x7fe2240a5870 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe2400ffeb0 tx=0x7fe228005f90 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:15:58.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.292+0000 7fe237400640 1 -- 172.21.15.43:0/1041721037 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe2300bc8a0 con 0x7fe240103ff0 2024-09-19T05:15:58.306 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:58 smithi151 ceph-mon[35425]: pgmap v352: 97 pgs: 97 active+clean; 2.4 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-19T05:15:58.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:58 smithi096 ceph-mon[35085]: pgmap v352: 97 pgs: 97 active+clean; 2.4 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-19T05:15:58.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.515+0000 7fe245400640 1 -- 172.21.15.43:0/1041721037 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe24010c300 con 0x7fe240103ff0 2024-09-19T05:15:58.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.526+0000 7fe237400640 1 -- 172.21.15.43:0/1041721037 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v30) ==== 161+0+0 (secure 0 0 0) 0x7fe2300c0ba0 con 0x7fe240103ff0 2024-09-19T05:15:58.528 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T05:15:58.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.529+0000 7fe245400640 1 -- 172.21.15.43:0/1041721037 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fe2240a33b0 msgr2=0x7fe2240a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:58.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.529+0000 7fe245400640 1 --2- 172.21.15.43:0/1041721037 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fe2240a33b0 0x7fe2240a5870 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe2400ffeb0 tx=0x7fe228005f90 comp rx=0 tx=0).stop 2024-09-19T05:15:58.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.529+0000 7fe245400640 1 -- 172.21.15.43:0/1041721037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe240103ff0 msgr2=0x7fe24019a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:15:58.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.529+0000 7fe245400640 1 --2- 172.21.15.43:0/1041721037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe240103ff0 0x7fe24019a640 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe230005940 tx=0x7fe230004270 comp rx=0 tx=0).stop 2024-09-19T05:15:58.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.529+0000 7fe245400640 1 -- 172.21.15.43:0/1041721037 shutdown_connections 2024-09-19T05:15:58.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.529+0000 7fe245400640 1 --2- 172.21.15.43:0/1041721037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe240109120 0x7fe24019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:58.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.529+0000 7fe245400640 1 --2- 172.21.15.43:0/1041721037 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2401049a0 0x7fe24019ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:58.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.529+0000 7fe245400640 1 --2- 172.21.15.43:0/1041721037 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7fe2240a33b0 0x7fe2240a5870 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:58.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.529+0000 7fe245400640 1 --2- 172.21.15.43:0/1041721037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe240103ff0 0x7fe24019a640 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:15:58.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.529+0000 7fe245400640 1 -- 172.21.15.43:0/1041721037 >> 172.21.15.43:0/1041721037 conn(0x7fe2400ffa90 msgr2=0x7fe240100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:15:58.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.529+0000 7fe245400640 1 -- 172.21.15.43:0/1041721037 shutdown_connections 2024-09-19T05:15:58.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:15:58.529+0000 7fe245400640 1 -- 172.21.15.43:0/1041721037 wait complete. 2024-09-19T05:15:58.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:58 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mon-a[30796]: 2024-09-19T05:15:58.516+0000 7ff063400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T05:15:58.781 INFO:journalctl@ceph.mds.a.smithi043.stdout:Sep 19 05:15:58 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-a[69678]: ignoring --setuser ceph since I am not root 2024-09-19T05:15:58.781 INFO:journalctl@ceph.mds.a.smithi043.stdout:Sep 19 05:15:58 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-a[69678]: ignoring --setgroup ceph since I am not root 2024-09-19T05:15:58.781 INFO:journalctl@ceph.mds.a.smithi043.stdout:Sep 19 05:15:58 smithi043 ceph-mds[69684]: -- 172.21.15.43:0/766467575 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e5a1cf5c20 con 0x55e5a1cd3000 2024-09-19T05:15:58.781 INFO:journalctl@ceph.mds.a.smithi043.stdout:Sep 19 05:15:58 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-a[69678]: starting mds.a at 2024-09-19T05:15:58.781 INFO:journalctl@ceph.mds.g.smithi043.stdout:Sep 19 05:15:58 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-g[69296]: ignoring --setuser ceph since I am not root 2024-09-19T05:15:58.782 INFO:journalctl@ceph.mds.g.smithi043.stdout:Sep 19 05:15:58 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-g[69296]: ignoring --setgroup ceph since I am not root 2024-09-19T05:15:58.782 INFO:journalctl@ceph.mds.g.smithi043.stdout:Sep 19 05:15:58 smithi043 ceph-mds[69300]: -- 172.21.15.43:0/3084498519 <== mon.0 v2:172.21.15.43:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5574da0a9c20 con 0x5574da086800 2024-09-19T05:15:58.782 INFO:journalctl@ceph.mds.g.smithi043.stdout:Sep 19 05:15:58 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-g[69296]: starting mds.g at 2024-09-19T05:15:58.905 INFO:journalctl@ceph.mds.f.smithi151.stdout:Sep 19 05:15:58 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-f[65183]: ignoring --setuser ceph since I am not root 2024-09-19T05:15:58.905 INFO:journalctl@ceph.mds.f.smithi151.stdout:Sep 19 05:15:58 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-f[65183]: ignoring --setgroup ceph since I am not root 2024-09-19T05:15:58.905 INFO:journalctl@ceph.mds.f.smithi151.stdout:Sep 19 05:15:58 smithi151 ceph-mds[65187]: -- 172.21.15.151:0/2426055295 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e37e58d860 con 0x55e37e56a800 2024-09-19T05:15:58.905 INFO:journalctl@ceph.mds.f.smithi151.stdout:Sep 19 05:15:58 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-f[65183]: starting mds.f at 2024-09-19T05:15:58.906 INFO:journalctl@ceph.mds.l.smithi151.stdout:Sep 19 05:15:58 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-l[64870]: ignoring --setuser ceph since I am not root 2024-09-19T05:15:58.906 INFO:journalctl@ceph.mds.l.smithi151.stdout:Sep 19 05:15:58 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-l[64870]: ignoring --setgroup ceph since I am not root 2024-09-19T05:15:58.906 INFO:journalctl@ceph.mds.l.smithi151.stdout:Sep 19 05:15:58 smithi151 ceph-mds[64874]: -- 172.21.15.151:0/294999765 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561b967c9860 con 0x561b967a7400 2024-09-19T05:15:58.906 INFO:journalctl@ceph.mds.l.smithi151.stdout:Sep 19 05:15:58 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-l[64870]: starting mds.l at 2024-09-19T05:15:58.906 INFO:journalctl@ceph.mds.c.smithi151.stdout:Sep 19 05:15:58 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-c[65488]: ignoring --setuser ceph since I am not root 2024-09-19T05:15:58.906 INFO:journalctl@ceph.mds.c.smithi151.stdout:Sep 19 05:15:58 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-c[65488]: ignoring --setgroup ceph since I am not root 2024-09-19T05:15:58.906 INFO:journalctl@ceph.mds.c.smithi151.stdout:Sep 19 05:15:58 smithi151 ceph-mds[65492]: -- 172.21.15.151:0/3853625947 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55daaa2b1860 con 0x55daaa28f400 2024-09-19T05:15:58.906 INFO:journalctl@ceph.mds.c.smithi151.stdout:Sep 19 05:15:58 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-c[65488]: starting mds.c at 2024-09-19T05:15:58.907 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 05:15:58 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-i[65787]: ignoring --setuser ceph since I am not root 2024-09-19T05:15:58.907 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 05:15:58 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-i[65787]: ignoring --setgroup ceph since I am not root 2024-09-19T05:15:58.907 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 05:15:58 smithi151 ceph-mds[65791]: -- 172.21.15.151:0/2849666148 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55697035d860 con 0x55697033b000 2024-09-19T05:15:58.907 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 05:15:58 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-i[65787]: starting mds.i at 2024-09-19T05:15:58.973 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 05:15:58 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-b[65164]: ignoring --setuser ceph since I am not root 2024-09-19T05:15:58.974 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 05:15:58 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-b[65164]: ignoring --setgroup ceph since I am not root 2024-09-19T05:15:58.974 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 05:15:58 smithi096 ceph-mds[65168]: -- 172.21.15.96:0/3557951544 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c3c78d1860 con 0x55c3c78ae800 2024-09-19T05:15:58.974 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 05:15:58 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-b[65164]: starting mds.b at 2024-09-19T05:15:58.974 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 19 05:15:58 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-h[64865]: ignoring --setuser ceph since I am not root 2024-09-19T05:15:58.974 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 19 05:15:58 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-h[64865]: ignoring --setgroup ceph since I am not root 2024-09-19T05:15:58.974 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 19 05:15:58 smithi096 ceph-mds[64869]: -- 172.21.15.96:0/4189988916 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5579c703b860 con 0x5579c7019400 2024-09-19T05:15:58.974 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 19 05:15:58 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-h[64865]: starting mds.h at 2024-09-19T05:15:58.975 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 19 05:15:58 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-e[64565]: ignoring --setuser ceph since I am not root 2024-09-19T05:15:58.975 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 19 05:15:58 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-e[64565]: ignoring --setgroup ceph since I am not root 2024-09-19T05:15:58.975 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 19 05:15:58 smithi096 ceph-mds[64569]: -- 172.21.15.96:0/710971975 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564064696680 con 0x56406466b000 2024-09-19T05:15:58.975 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 19 05:15:58 smithi096 ceph-mds[64569]: -- 172.21.15.96:0/710971975 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56406468d860 con 0x56406466b000 2024-09-19T05:15:58.975 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 19 05:15:58 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-e[64565]: starting mds.e at 2024-09-19T05:15:58.975 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 19 05:15:58 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-k[64252]: ignoring --setuser ceph since I am not root 2024-09-19T05:15:58.975 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 19 05:15:58 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-k[64252]: ignoring --setgroup ceph since I am not root 2024-09-19T05:15:58.975 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 19 05:15:58 smithi096 ceph-mds[64256]: -- 172.21.15.96:0/2817958077 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d021149860 con 0x55d021126800 2024-09-19T05:15:58.975 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 19 05:15:58 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-k[64252]: starting mds.k at 2024-09-19T05:15:59.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:59 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/1041721037' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T05:15:59.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:59 smithi043 ceph-mon[30800]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T05:15:59.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:59 smithi043 ceph-mon[30800]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T05:15:59.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:59 smithi043 ceph-mon[30800]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T05:15:59.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:59 smithi043 ceph-mon[30800]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T05:15:59.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:59 smithi043 ceph-mon[30800]: osdmap e97: 12 total, 12 up, 12 in 2024-09-19T05:15:59.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:15:59 smithi043 ceph-mon[30800]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-19T05:15:59.321 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:59 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/1041721037' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T05:15:59.321 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:59 smithi151 ceph-mon[35425]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T05:15:59.321 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:59 smithi151 ceph-mon[35425]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T05:15:59.321 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:59 smithi151 ceph-mon[35425]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T05:15:59.321 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:59 smithi151 ceph-mon[35425]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T05:15:59.321 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:59 smithi151 ceph-mon[35425]: osdmap e97: 12 total, 12 up, 12 in 2024-09-19T05:15:59.321 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:15:59 smithi151 ceph-mon[35425]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-19T05:15:59.357 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T05:15:59.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:59 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/1041721037' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T05:15:59.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:59 smithi096 ceph-mon[35085]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T05:15:59.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:59 smithi096 ceph-mon[35085]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T05:15:59.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:59 smithi096 ceph-mon[35085]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T05:15:59.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:59 smithi096 ceph-mon[35085]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T05:15:59.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:59 smithi096 ceph-mon[35085]: osdmap e97: 12 total, 12 up, 12 in 2024-09-19T05:15:59.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:15:59 smithi096 ceph-mon[35085]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-19T05:15:59.742 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:16:00.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:00 smithi043 ceph-mon[30800]: pgmap v353: 97 pgs: 97 active+clean; 2.4 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-19T05:16:00.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:00 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T05:16:00.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:00 smithi043 ceph-mon[30800]: mds.? [v2:172.21.15.96:6832/3085644718,v1:172.21.15.96:6836/3085644718] up:boot 2024-09-19T05:16:00.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:00 smithi043 ceph-mon[30800]: mds.? [v2:172.21.15.151:6832/1508086446,v1:172.21.15.151:6833/1508086446] up:boot 2024-09-19T05:16:00.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:00 smithi043 ceph-mon[30800]: mds.? [v2:172.21.15.96:6834/911297807,v1:172.21.15.96:6837/911297807] up:boot 2024-09-19T05:16:00.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:00 smithi043 ceph-mon[30800]: mds.? [v2:172.21.15.96:6833/1242449779,v1:172.21.15.96:6838/1242449779] up:boot 2024-09-19T05:16:00.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:00 smithi043 ceph-mon[30800]: mds.? [v2:172.21.15.96:6835/2443373034,v1:172.21.15.96:6839/2443373034] up:boot 2024-09-19T05:16:00.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:00 smithi043 ceph-mon[30800]: mds.? [v2:172.21.15.151:6834/3381482340,v1:172.21.15.151:6835/3381482340] up:boot 2024-09-19T05:16:00.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:00 smithi043 ceph-mon[30800]: mds.? [v2:172.21.15.151:6836/1280245351,v1:172.21.15.151:6837/1280245351] up:boot 2024-09-19T05:16:00.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:00 smithi043 ceph-mon[30800]: mds.? [v2:172.21.15.151:6838/2223201935,v1:172.21.15.151:6839/2223201935] up:boot 2024-09-19T05:16:00.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:00 smithi043 ceph-mon[30800]: mds.? [v2:172.21.15.43:6838/2538098443,v1:172.21.15.43:6840/2538098443] up:boot 2024-09-19T05:16:00.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:00 smithi043 ceph-mon[30800]: mds.? [v2:172.21.15.43:6839/4182094011,v1:172.21.15.43:6841/4182094011] up:boot 2024-09-19T05:16:00.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:00 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T05:16:00.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:00 smithi043 ceph-mon[30800]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-19T05:16:00.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:00 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T05:16:00.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:00 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T05:16:00.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:00 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T05:16:00.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:00 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T05:16:00.156 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:00 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T05:16:00.156 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:00 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T05:16:00.156 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:00 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T05:16:00.156 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:00 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T05:16:00.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:00 smithi151 ceph-mon[35425]: pgmap v353: 97 pgs: 97 active+clean; 2.4 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-19T05:16:00.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:00 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T05:16:00.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:00 smithi151 ceph-mon[35425]: mds.? [v2:172.21.15.96:6832/3085644718,v1:172.21.15.96:6836/3085644718] up:boot 2024-09-19T05:16:00.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:00 smithi151 ceph-mon[35425]: mds.? [v2:172.21.15.151:6832/1508086446,v1:172.21.15.151:6833/1508086446] up:boot 2024-09-19T05:16:00.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:00 smithi151 ceph-mon[35425]: mds.? [v2:172.21.15.96:6834/911297807,v1:172.21.15.96:6837/911297807] up:boot 2024-09-19T05:16:00.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:00 smithi151 ceph-mon[35425]: mds.? [v2:172.21.15.96:6833/1242449779,v1:172.21.15.96:6838/1242449779] up:boot 2024-09-19T05:16:00.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:00 smithi151 ceph-mon[35425]: mds.? [v2:172.21.15.96:6835/2443373034,v1:172.21.15.96:6839/2443373034] up:boot 2024-09-19T05:16:00.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:00 smithi151 ceph-mon[35425]: mds.? [v2:172.21.15.151:6834/3381482340,v1:172.21.15.151:6835/3381482340] up:boot 2024-09-19T05:16:00.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:00 smithi151 ceph-mon[35425]: mds.? [v2:172.21.15.151:6836/1280245351,v1:172.21.15.151:6837/1280245351] up:boot 2024-09-19T05:16:00.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:00 smithi151 ceph-mon[35425]: mds.? [v2:172.21.15.151:6838/2223201935,v1:172.21.15.151:6839/2223201935] up:boot 2024-09-19T05:16:00.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:00 smithi151 ceph-mon[35425]: mds.? [v2:172.21.15.43:6838/2538098443,v1:172.21.15.43:6840/2538098443] up:boot 2024-09-19T05:16:00.406 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:00 smithi151 ceph-mon[35425]: mds.? [v2:172.21.15.43:6839/4182094011,v1:172.21.15.43:6841/4182094011] up:boot 2024-09-19T05:16:00.407 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:00 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T05:16:00.407 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:00 smithi151 ceph-mon[35425]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-19T05:16:00.407 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:00 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T05:16:00.407 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:00 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T05:16:00.407 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:00 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T05:16:00.407 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:00 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T05:16:00.407 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:00 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T05:16:00.407 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:00 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T05:16:00.407 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:00 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T05:16:00.407 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:00 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T05:16:00.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:00.409+0000 7f71ae200640 1 -- 172.21.15.43:0/2604210215 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71a8112b90 msgr2=0x7f71a8114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:00.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:00.409+0000 7f71ae200640 1 --2- 172.21.15.43:0/2604210215 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71a8112b90 0x7f71a8114f80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f71a0009ca0 tx=0x7f71a002a940 comp rx=0 tx=0).stop 2024-09-19T05:16:00.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:00.409+0000 7f71ae200640 1 -- 172.21.15.43:0/2604210215 shutdown_connections 2024-09-19T05:16:00.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:00.409+0000 7f71ae200640 1 --2- 172.21.15.43:0/2604210215 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71a8112b90 0x7f71a8114f80 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:00.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:00.409+0000 7f71ae200640 1 --2- 172.21.15.43:0/2604210215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71a8074eb0 0x7f71a8112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:00.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:00.409+0000 7f71ae200640 1 --2- 172.21.15.43:0/2604210215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71a8074590 0x7f71a8074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:00.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:00.409+0000 7f71ae200640 1 -- 172.21.15.43:0/2604210215 >> 172.21.15.43:0/2604210215 conn(0x7f71a80fd8c0 msgr2=0x7f71a80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:00.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:00.410+0000 7f71ae200640 1 -- 172.21.15.43:0/2604210215 shutdown_connections 2024-09-19T05:16:00.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:00.410+0000 7f71ae200640 1 -- 172.21.15.43:0/2604210215 wait complete. 2024-09-19T05:16:00.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:00.411+0000 7f71ae200640 1 Processor -- start 2024-09-19T05:16:00.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:00.411+0000 7f71ae200640 1 -- start start 2024-09-19T05:16:00.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:00.411+0000 7f71ae200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71a8074590 0x7f71a81a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:00.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:00.411+0000 7f71ae200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71a8074eb0 0x7f71a81a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:00.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:00.411+0000 7f71ae200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71a8112b90 0x7f71a819e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:00.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:00.411+0000 7f71ae200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f71a81185a0 con 0x7f71a8074590 2024-09-19T05:16:00.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:00.411+0000 7f71ae200640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f71a8118420 con 0x7f71a8112b90 2024-09-19T05:16:00.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:00.411+0000 7f71ae200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f71a8118720 con 0x7f71a8074eb0 2024-09-19T05:16:00.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:00.411+0000 7f71a7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71a8074590 0x7f71a81a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:00.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:00.411+0000 7f719fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71a8074eb0 0x7f71a81a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:00.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:00.411+0000 7f71a7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71a8074590 0x7f71a81a46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50834/0 (socket says 172.21.15.43:50834) 2024-09-19T05:16:00.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:00.411+0000 7f71a7400640 1 -- 172.21.15.43:0/2699415311 learned_addr learned my addr 172.21.15.43:0/2699415311 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:16:00.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:00.412+0000 7f71a7e00640 1 --2- 172.21.15.43:0/2699415311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71a8112b90 0x7f71a819e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:00.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:00.412+0000 7f719fe00640 1 -- 172.21.15.43:0/2699415311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71a8112b90 msgr2=0x7f71a819e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:00.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:00.412+0000 7f719fe00640 1 --2- 172.21.15.43:0/2699415311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71a8112b90 0x7f71a819e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:00.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:00.412+0000 7f719fe00640 1 -- 172.21.15.43:0/2699415311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71a8074590 msgr2=0x7f71a81a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:00.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:00.412+0000 7f719fe00640 1 --2- 172.21.15.43:0/2699415311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71a8074590 0x7f71a81a46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:00.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:00.412+0000 7f719fe00640 1 -- 172.21.15.43:0/2699415311 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71a819efc0 con 0x7f71a8074eb0 2024-09-19T05:16:00.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:00.412+0000 7f71a7400640 1 --2- 172.21.15.43:0/2699415311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71a8074590 0x7f71a81a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:16:00.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:00.412+0000 7f719fe00640 1 --2- 172.21.15.43:0/2699415311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71a8074eb0 0x7f71a81a4c10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f719000d6d0 tx=0x7f719000db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:00.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:00.412+0000 7f71a4c00640 1 -- 172.21.15.43:0/2699415311 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7190017070 con 0x7f71a8074eb0 2024-09-19T05:16:00.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:00.412+0000 7f71ae200640 1 -- 172.21.15.43:0/2699415311 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71a819f2b0 con 0x7f71a8074eb0 2024-09-19T05:16:00.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:00.412+0000 7f71a4c00640 1 -- 172.21.15.43:0/2699415311 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7190009820 con 0x7f71a8074eb0 2024-09-19T05:16:00.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:00.412+0000 7f71a4c00640 1 -- 172.21.15.43:0/2699415311 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7190004cd0 con 0x7f71a8074eb0 2024-09-19T05:16:00.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:00.413+0000 7f71ae200640 1 -- 172.21.15.43:0/2699415311 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71a81ab5a0 con 0x7f71a8074eb0 2024-09-19T05:16:00.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:00.414+0000 7f71ae200640 1 -- 172.21.15.43:0/2699415311 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71a8074970 con 0x7f71a8074eb0 2024-09-19T05:16:00.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:00.415+0000 7f71a4c00640 1 -- 172.21.15.43:0/2699415311 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f71900099d0 con 0x7f71a8074eb0 2024-09-19T05:16:00.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:00.417+0000 7f71a4c00640 1 --2- 172.21.15.43:0/2699415311 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f718c0a33b0 0x7f718c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:00.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:00.417+0000 7f71a4c00640 1 -- 172.21.15.43:0/2699415311 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8928+0+0 (secure 0 0 0) 0x7f71900cf990 con 0x7f71a8074eb0 2024-09-19T05:16:00.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:00.417+0000 7f71a7400640 1 --2- 172.21.15.43:0/2699415311 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f718c0a33b0 0x7f718c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:00.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:00.417+0000 7f71a7400640 1 --2- 172.21.15.43:0/2699415311 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f718c0a33b0 0x7f718c0a5870 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f71a819f9d0 tx=0x7f7198005f90 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:00.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:00.420+0000 7f71a4c00640 1 -- 172.21.15.43:0/2699415311 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f719009bb50 con 0x7f71a8074eb0 2024-09-19T05:16:00.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:00 smithi096 ceph-mon[35085]: pgmap v353: 97 pgs: 97 active+clean; 2.4 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-19T05:16:00.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:00 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T05:16:00.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:00 smithi096 ceph-mon[35085]: mds.? [v2:172.21.15.96:6832/3085644718,v1:172.21.15.96:6836/3085644718] up:boot 2024-09-19T05:16:00.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:00 smithi096 ceph-mon[35085]: mds.? [v2:172.21.15.151:6832/1508086446,v1:172.21.15.151:6833/1508086446] up:boot 2024-09-19T05:16:00.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:00 smithi096 ceph-mon[35085]: mds.? [v2:172.21.15.96:6834/911297807,v1:172.21.15.96:6837/911297807] up:boot 2024-09-19T05:16:00.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:00 smithi096 ceph-mon[35085]: mds.? [v2:172.21.15.96:6833/1242449779,v1:172.21.15.96:6838/1242449779] up:boot 2024-09-19T05:16:00.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:00 smithi096 ceph-mon[35085]: mds.? [v2:172.21.15.96:6835/2443373034,v1:172.21.15.96:6839/2443373034] up:boot 2024-09-19T05:16:00.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:00 smithi096 ceph-mon[35085]: mds.? [v2:172.21.15.151:6834/3381482340,v1:172.21.15.151:6835/3381482340] up:boot 2024-09-19T05:16:00.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:00 smithi096 ceph-mon[35085]: mds.? [v2:172.21.15.151:6836/1280245351,v1:172.21.15.151:6837/1280245351] up:boot 2024-09-19T05:16:00.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:00 smithi096 ceph-mon[35085]: mds.? [v2:172.21.15.151:6838/2223201935,v1:172.21.15.151:6839/2223201935] up:boot 2024-09-19T05:16:00.475 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:00 smithi096 ceph-mon[35085]: mds.? [v2:172.21.15.43:6838/2538098443,v1:172.21.15.43:6840/2538098443] up:boot 2024-09-19T05:16:00.475 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:00 smithi096 ceph-mon[35085]: mds.? [v2:172.21.15.43:6839/4182094011,v1:172.21.15.43:6841/4182094011] up:boot 2024-09-19T05:16:00.475 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:00 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T05:16:00.475 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:00 smithi096 ceph-mon[35085]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-19T05:16:00.475 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:00 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T05:16:00.475 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:00 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T05:16:00.475 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:00 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T05:16:00.475 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:00 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T05:16:00.475 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:00 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T05:16:00.475 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:00 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T05:16:00.475 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:00 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T05:16:00.476 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:00 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T05:16:00.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:00.646+0000 7f71ae200640 1 -- 172.21.15.43:0/2699415311 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f71a802d7c0 con 0x7f71a8074eb0 2024-09-19T05:16:01.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:01.022+0000 7f71a4c00640 1 -- 172.21.15.43:0/2699415311 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v32) ==== 104+0+0 (secure 0 0 0) 0x7f719009fe50 con 0x7f71a8074eb0 2024-09-19T05:16:01.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:01.025+0000 7f71ae200640 1 -- 172.21.15.43:0/2699415311 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f718c0a33b0 msgr2=0x7f718c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:01.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:01.025+0000 7f71ae200640 1 --2- 172.21.15.43:0/2699415311 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f718c0a33b0 0x7f718c0a5870 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f71a819f9d0 tx=0x7f7198005f90 comp rx=0 tx=0).stop 2024-09-19T05:16:01.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:01.025+0000 7f71ae200640 1 -- 172.21.15.43:0/2699415311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71a8074eb0 msgr2=0x7f71a81a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:01.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:01.025+0000 7f71ae200640 1 --2- 172.21.15.43:0/2699415311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71a8074eb0 0x7f71a81a4c10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f719000d6d0 tx=0x7f719000db90 comp rx=0 tx=0).stop 2024-09-19T05:16:01.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:01.025+0000 7f71ae200640 1 -- 172.21.15.43:0/2699415311 shutdown_connections 2024-09-19T05:16:01.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:01.025+0000 7f71ae200640 1 --2- 172.21.15.43:0/2699415311 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f718c0a33b0 0x7f718c0a5870 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:01.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:01.025+0000 7f71ae200640 1 --2- 172.21.15.43:0/2699415311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71a8112b90 0x7f71a819e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:01.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:01.025+0000 7f71ae200640 1 --2- 172.21.15.43:0/2699415311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71a8074eb0 0x7f71a81a4c10 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:01.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:01.025+0000 7f71ae200640 1 --2- 172.21.15.43:0/2699415311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71a8074590 0x7f71a81a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:01.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:01.025+0000 7f71ae200640 1 -- 172.21.15.43:0/2699415311 >> 172.21.15.43:0/2699415311 conn(0x7f71a80fd8c0 msgr2=0x7f71a80fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:01.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:01.025+0000 7f71ae200640 1 -- 172.21.15.43:0/2699415311 shutdown_connections 2024-09-19T05:16:01.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:01.025+0000 7f71ae200640 1 -- 172.21.15.43:0/2699415311 wait complete. 2024-09-19T05:16:01.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:01 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/2699415311' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T05:16:01.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:01 smithi043 ceph-mon[30800]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T05:16:01.338 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:01 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/2699415311' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T05:16:01.338 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:01 smithi151 ceph-mon[35425]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T05:16:01.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:01 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/2699415311' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T05:16:01.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:01 smithi096 ceph-mon[35085]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T05:16:01.872 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T05:16:02.253 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:16:02.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:02 smithi043 ceph-mon[30800]: pgmap v355: 97 pgs: 97 active+clean; 2.4 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-19T05:16:02.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:02 smithi043 ceph-mon[30800]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T05:16:02.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:02 smithi043 ceph-mon[30800]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T05:16:02.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:02 smithi043 ceph-mon[30800]: Cluster is now healthy 2024-09-19T05:16:02.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:02 smithi043 ceph-mon[30800]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T05:16:02.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:02 smithi043 ceph-mon[30800]: fsmap 12 up:standby 2024-09-19T05:16:02.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:02 smithi151 ceph-mon[35425]: pgmap v355: 97 pgs: 97 active+clean; 2.4 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-19T05:16:02.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:02 smithi151 ceph-mon[35425]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T05:16:02.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:02 smithi151 ceph-mon[35425]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T05:16:02.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:02 smithi151 ceph-mon[35425]: Cluster is now healthy 2024-09-19T05:16:02.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:02 smithi151 ceph-mon[35425]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T05:16:02.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:02 smithi151 ceph-mon[35425]: fsmap 12 up:standby 2024-09-19T05:16:02.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:02 smithi096 ceph-mon[35085]: pgmap v355: 97 pgs: 97 active+clean; 2.4 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-19T05:16:02.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:02 smithi096 ceph-mon[35085]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T05:16:02.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:02 smithi096 ceph-mon[35085]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T05:16:02.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:02 smithi096 ceph-mon[35085]: Cluster is now healthy 2024-09-19T05:16:02.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:02 smithi096 ceph-mon[35085]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T05:16:02.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:02 smithi096 ceph-mon[35085]: fsmap 12 up:standby 2024-09-19T05:16:02.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:02.781+0000 7f5b5ce00640 1 -- 172.21.15.43:0/2225297841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b58076720 msgr2=0x7f5b58076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:02.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:02.781+0000 7f5b5ce00640 1 --2- 172.21.15.43:0/2225297841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b58076720 0x7f5b58076b00 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f5b48009ca0 tx=0x7f5b4802a8c0 comp rx=0 tx=0).stop 2024-09-19T05:16:02.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:02.781+0000 7f5b5ce00640 1 -- 172.21.15.43:0/2225297841 shutdown_connections 2024-09-19T05:16:02.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:02.781+0000 7f5b5ce00640 1 --2- 172.21.15.43:0/2225297841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b58112a30 0x7f5b58114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:02.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:02.781+0000 7f5b5ce00640 1 --2- 172.21.15.43:0/2225297841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b58077040 0x7f5b581124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:02.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:02.781+0000 7f5b5ce00640 1 --2- 172.21.15.43:0/2225297841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b58076720 0x7f5b58076b00 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:02.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:02.781+0000 7f5b5ce00640 1 -- 172.21.15.43:0/2225297841 >> 172.21.15.43:0/2225297841 conn(0x7f5b580ffa90 msgr2=0x7f5b58101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:02.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:02.781+0000 7f5b5ce00640 1 -- 172.21.15.43:0/2225297841 shutdown_connections 2024-09-19T05:16:02.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:02.781+0000 7f5b5ce00640 1 -- 172.21.15.43:0/2225297841 wait complete. 2024-09-19T05:16:02.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:02.782+0000 7f5b5ce00640 1 Processor -- start 2024-09-19T05:16:02.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:02.783+0000 7f5b5ce00640 1 -- start start 2024-09-19T05:16:02.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:02.783+0000 7f5b5ce00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b58076720 0x7f5b581a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:02.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:02.783+0000 7f5b5ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b58077040 0x7f5b581a4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:02.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:02.783+0000 7f5b5ce00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b58112a30 0x7f5b5819e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:02.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:02.783+0000 7f5b5ce00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5b581185f0 con 0x7f5b58077040 2024-09-19T05:16:02.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:02.783+0000 7f5b5ce00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5b58118470 con 0x7f5b58076720 2024-09-19T05:16:02.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:02.783+0000 7f5b5ce00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5b58118770 con 0x7f5b58112a30 2024-09-19T05:16:02.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:02.783+0000 7f5b4d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b58077040 0x7f5b581a4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:02.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:02.783+0000 7f5b55e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b58076720 0x7f5b581a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:02.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:02.783+0000 7f5b55e00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b58076720 0x7f5b581a47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.43:48184/0 (socket says 172.21.15.43:48184) 2024-09-19T05:16:02.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:02.783+0000 7f5b55e00640 1 -- 172.21.15.43:0/3473568071 learned_addr learned my addr 172.21.15.43:0/3473568071 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:16:02.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:02.783+0000 7f5b4d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b58077040 0x7f5b581a4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50840/0 (socket says 172.21.15.43:50840) 2024-09-19T05:16:02.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:02.784+0000 7f5b56800640 1 --2- 172.21.15.43:0/3473568071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b58112a30 0x7f5b5819e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:02.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:02.784+0000 7f5b55e00640 1 -- 172.21.15.43:0/3473568071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b58112a30 msgr2=0x7f5b5819e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:02.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:02.784+0000 7f5b55e00640 1 --2- 172.21.15.43:0/3473568071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b58112a30 0x7f5b5819e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:02.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:02.784+0000 7f5b55e00640 1 -- 172.21.15.43:0/3473568071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b58077040 msgr2=0x7f5b581a4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:02.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:02.784+0000 7f5b55e00640 1 --2- 172.21.15.43:0/3473568071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b58077040 0x7f5b581a4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:02.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:02.784+0000 7f5b55e00640 1 -- 172.21.15.43:0/3473568071 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b5819f090 con 0x7f5b58076720 2024-09-19T05:16:02.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:02.784+0000 7f5b4d400640 1 --2- 172.21.15.43:0/3473568071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b58077040 0x7f5b581a4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:16:02.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:02.784+0000 7f5b55e00640 1 --2- 172.21.15.43:0/3473568071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b58076720 0x7f5b581a47a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5b48005e30 tx=0x7f5b48003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:02.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:02.785+0000 7f5b4f400640 1 -- 172.21.15.43:0/3473568071 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b480316d0 con 0x7f5b58076720 2024-09-19T05:16:02.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:02.785+0000 7f5b5ce00640 1 -- 172.21.15.43:0/3473568071 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b5819f320 con 0x7f5b58076720 2024-09-19T05:16:02.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:02.785+0000 7f5b4f400640 1 -- 172.21.15.43:0/3473568071 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b48031870 con 0x7f5b58076720 2024-09-19T05:16:02.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:02.785+0000 7f5b4f400640 1 -- 172.21.15.43:0/3473568071 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4802fc00 con 0x7f5b58076720 2024-09-19T05:16:02.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:02.785+0000 7f5b5ce00640 1 -- 172.21.15.43:0/3473568071 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b581ab670 con 0x7f5b58076720 2024-09-19T05:16:02.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:02.786+0000 7f5b5ce00640 1 -- 172.21.15.43:0/3473568071 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b58076b00 con 0x7f5b58076720 2024-09-19T05:16:02.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:02.788+0000 7f5b4f400640 1 -- 172.21.15.43:0/3473568071 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f5b480046b0 con 0x7f5b58076720 2024-09-19T05:16:02.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:02.789+0000 7f5b4f400640 1 --2- 172.21.15.43:0/3473568071 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f5b3c0a3430 0x7f5b3c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:02.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:02.789+0000 7f5b4f400640 1 -- 172.21.15.43:0/3473568071 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8928+0+0 (secure 0 0 0) 0x7f5b480f1400 con 0x7f5b58076720 2024-09-19T05:16:02.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:02.790+0000 7f5b4d400640 1 --2- 172.21.15.43:0/3473568071 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f5b3c0a3430 0x7f5b3c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:02.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:02.790+0000 7f5b4d400640 1 --2- 172.21.15.43:0/3473568071 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f5b3c0a3430 0x7f5b3c0a58f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5b58079320 tx=0x7f5b4000b390 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:02.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:02.791+0000 7f5b4f400640 1 -- 172.21.15.43:0/3473568071 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5b480bdd20 con 0x7f5b58076720 2024-09-19T05:16:02.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:02.978+0000 7f5b5ce00640 1 -- 172.21.15.43:0/3473568071 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5b5810dc20 con 0x7f5b58076720 2024-09-19T05:16:03.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:03.024+0000 7f5b4f400640 1 -- 172.21.15.43:0/3473568071 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v98) ==== 181+0+0 (secure 0 0 0) 0x7f5b480bd620 con 0x7f5b58076720 2024-09-19T05:16:03.026 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' removed 2024-09-19T05:16:03.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:03.027+0000 7f5b5ce00640 1 -- 172.21.15.43:0/3473568071 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f5b3c0a3430 msgr2=0x7f5b3c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:03.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:03.027+0000 7f5b5ce00640 1 --2- 172.21.15.43:0/3473568071 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f5b3c0a3430 0x7f5b3c0a58f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5b58079320 tx=0x7f5b4000b390 comp rx=0 tx=0).stop 2024-09-19T05:16:03.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:03.027+0000 7f5b5ce00640 1 -- 172.21.15.43:0/3473568071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b58076720 msgr2=0x7f5b581a47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:03.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:03.027+0000 7f5b5ce00640 1 --2- 172.21.15.43:0/3473568071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b58076720 0x7f5b581a47a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5b48005e30 tx=0x7f5b48003f90 comp rx=0 tx=0).stop 2024-09-19T05:16:03.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:03.027+0000 7f5b5ce00640 1 -- 172.21.15.43:0/3473568071 shutdown_connections 2024-09-19T05:16:03.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:03.027+0000 7f5b5ce00640 1 --2- 172.21.15.43:0/3473568071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b58112a30 0x7f5b5819e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:03.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:03.027+0000 7f5b5ce00640 1 --2- 172.21.15.43:0/3473568071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b58077040 0x7f5b581a4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:03.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:03.027+0000 7f5b5ce00640 1 --2- 172.21.15.43:0/3473568071 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7f5b3c0a3430 0x7f5b3c0a58f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:03.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:03.027+0000 7f5b5ce00640 1 --2- 172.21.15.43:0/3473568071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b58076720 0x7f5b581a47a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:03.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:03.027+0000 7f5b5ce00640 1 -- 172.21.15.43:0/3473568071 >> 172.21.15.43:0/3473568071 conn(0x7f5b580ffa90 msgr2=0x7f5b58101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:03.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:03.027+0000 7f5b5ce00640 1 -- 172.21.15.43:0/3473568071 shutdown_connections 2024-09-19T05:16:03.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:03.027+0000 7f5b5ce00640 1 -- 172.21.15.43:0/3473568071 wait complete. 2024-09-19T05:16:03.280 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:03 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/3473568071' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T05:16:03.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:03 smithi043 ceph-mon[30800]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T05:16:03.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:03 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/3473568071' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T05:16:03.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:03 smithi151 ceph-mon[35425]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T05:16:03.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:03 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/3473568071' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T05:16:03.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:03 smithi096 ceph-mon[35085]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T05:16:03.930 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 1456a810-7643-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T05:16:04.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:04 smithi043 ceph-mon[30800]: pgmap v356: 97 pgs: 97 active+clean; 2.4 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-19T05:16:04.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:04 smithi043 ceph-mon[30800]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T05:16:04.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:04 smithi043 ceph-mon[30800]: osdmap e98: 12 total, 12 up, 12 in 2024-09-19T05:16:04.313 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T05:16:04.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:04 smithi151 ceph-mon[35425]: pgmap v356: 97 pgs: 97 active+clean; 2.4 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-19T05:16:04.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:04 smithi151 ceph-mon[35425]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T05:16:04.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:04 smithi151 ceph-mon[35425]: osdmap e98: 12 total, 12 up, 12 in 2024-09-19T05:16:04.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:04 smithi096 ceph-mon[35085]: pgmap v356: 97 pgs: 97 active+clean; 2.4 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-19T05:16:04.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:04 smithi096 ceph-mon[35085]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T05:16:04.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:04 smithi096 ceph-mon[35085]: osdmap e98: 12 total, 12 up, 12 in 2024-09-19T05:16:04.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:04.938+0000 7ff8cee00640 1 -- 172.21.15.43:0/3254168081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8c8103ff0 msgr2=0x7ff8c81043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:04.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:04.938+0000 7ff8cee00640 1 --2- 172.21.15.43:0/3254168081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8c8103ff0 0x7ff8c81043d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009900 tx=0x7ff8b802a410 comp rx=0 tx=0).stop 2024-09-19T05:16:04.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:04.938+0000 7ff8cee00640 1 -- 172.21.15.43:0/3254168081 shutdown_connections 2024-09-19T05:16:04.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:04.938+0000 7ff8cee00640 1 --2- 172.21.15.43:0/3254168081 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8c8109120 0x7ff8c8110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:04.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:04.938+0000 7ff8cee00640 1 --2- 172.21.15.43:0/3254168081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8c81049a0 0x7ff8c81089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:04.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:04.938+0000 7ff8cee00640 1 --2- 172.21.15.43:0/3254168081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8c8103ff0 0x7ff8c81043d0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:04.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:04.938+0000 7ff8cee00640 1 -- 172.21.15.43:0/3254168081 >> 172.21.15.43:0/3254168081 conn(0x7ff8c80ffa90 msgr2=0x7ff8c8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:04.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:04.938+0000 7ff8cee00640 1 -- 172.21.15.43:0/3254168081 shutdown_connections 2024-09-19T05:16:04.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:04.938+0000 7ff8cee00640 1 -- 172.21.15.43:0/3254168081 wait complete. 2024-09-19T05:16:04.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:04.939+0000 7ff8cee00640 1 Processor -- start 2024-09-19T05:16:04.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:04.939+0000 7ff8cee00640 1 -- start start 2024-09-19T05:16:04.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:04.940+0000 7ff8cee00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8c8103ff0 0x7ff8c819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:04.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:04.940+0000 7ff8cee00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8c81049a0 0x7ff8c819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:04.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:04.940+0000 7ff8cee00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8c8109120 0x7ff8c819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:04.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:04.940+0000 7ff8cee00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c8114150 con 0x7ff8c8103ff0 2024-09-19T05:16:04.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:04.940+0000 7ff8cee00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c8113fd0 con 0x7ff8c8109120 2024-09-19T05:16:04.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:04.940+0000 7ff8cee00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c81142d0 con 0x7ff8c81049a0 2024-09-19T05:16:04.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:04.940+0000 7ff8c7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8c8103ff0 0x7ff8c819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:04.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:04.940+0000 7ff8bfe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8c81049a0 0x7ff8c819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:04.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:04.940+0000 7ff8c7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8c8103ff0 0x7ff8c819a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50848/0 (socket says 172.21.15.43:50848) 2024-09-19T05:16:04.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:04.940+0000 7ff8c7e00640 1 -- 172.21.15.43:0/2167110461 learned_addr learned my addr 172.21.15.43:0/2167110461 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-19T05:16:04.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:04.940+0000 7ff8cce00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8c8109120 0x7ff8c819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:04.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:04.940+0000 7ff8bfe00640 1 -- 172.21.15.43:0/2167110461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8c8109120 msgr2=0x7ff8c819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:04.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:04.940+0000 7ff8bfe00640 1 --2- 172.21.15.43:0/2167110461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8c8109120 0x7ff8c819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:04.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:04.940+0000 7ff8bfe00640 1 -- 172.21.15.43:0/2167110461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8c8103ff0 msgr2=0x7ff8c819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:04.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:04.940+0000 7ff8bfe00640 1 --2- 172.21.15.43:0/2167110461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8c8103ff0 0x7ff8c819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:04.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:04.940+0000 7ff8bfe00640 1 -- 172.21.15.43:0/2167110461 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8c819f620 con 0x7ff8c81049a0 2024-09-19T05:16:04.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:04.941+0000 7ff8bfe00640 1 --2- 172.21.15.43:0/2167110461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8c81049a0 0x7ff8c819abb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff8b000e9f0 tx=0x7ff8b000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:04.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:04.941+0000 7ff8c5600640 1 -- 172.21.15.43:0/2167110461 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b000ccc0 con 0x7ff8c81049a0 2024-09-19T05:16:04.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:04.941+0000 7ff8c5600640 1 -- 172.21.15.43:0/2167110461 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8b0010910 con 0x7ff8c81049a0 2024-09-19T05:16:04.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:04.941+0000 7ff8c5600640 1 -- 172.21.15.43:0/2167110461 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b0018960 con 0x7ff8c81049a0 2024-09-19T05:16:04.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:04.941+0000 7ff8cee00640 1 -- 172.21.15.43:0/2167110461 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8c819f910 con 0x7ff8c81049a0 2024-09-19T05:16:04.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:04.941+0000 7ff8cee00640 1 -- 172.21.15.43:0/2167110461 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8c81a71f0 con 0x7ff8c81049a0 2024-09-19T05:16:04.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:04.943+0000 7ff8cee00640 1 -- 172.21.15.43:0/2167110461 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8c81043d0 con 0x7ff8c81049a0 2024-09-19T05:16:04.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:04.944+0000 7ff8c5600640 1 -- 172.21.15.43:0/2167110461 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ff8b0010ac0 con 0x7ff8c81049a0 2024-09-19T05:16:04.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:04.946+0000 7ff8c5600640 1 --2- 172.21.15.43:0/2167110461 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff8ac0a3430 0x7ff8ac0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:16:04.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:04.946+0000 7ff8c5600640 1 -- 172.21.15.43:0/2167110461 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8345+0+0 (secure 0 0 0) 0x7ff8b00cffc0 con 0x7ff8c81049a0 2024-09-19T05:16:04.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:04.946+0000 7ff8c7e00640 1 --2- 172.21.15.43:0/2167110461 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff8ac0a3430 0x7ff8ac0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:16:04.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:04.947+0000 7ff8c7e00640 1 --2- 172.21.15.43:0/2167110461 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff8ac0a3430 0x7ff8ac0a58f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff8b80096c0 tx=0x7ff8b8005a20 comp rx=0 tx=0).ready entity=mgr.25136 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:16:04.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:04.949+0000 7ff8c5600640 1 -- 172.21.15.43:0/2167110461 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff8b009c3a0 con 0x7ff8c81049a0 2024-09-19T05:16:05.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:05.143+0000 7ff8cee00640 1 -- 172.21.15.43:0/2167110461 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff8c810c300 con 0x7ff8c81049a0 2024-09-19T05:16:05.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:05 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:16:05.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:05 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:16:05.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:05 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:16:05.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:05 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:16:05.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:05 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:16:05.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:05 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:16:05.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:05 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:16:05.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:05 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:16:05.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:05 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:16:05.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:05 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:16:05.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:05 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:16:05.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:05 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:16:05.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:05 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:16:05.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:05 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:16:05.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:05 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T05:16:05.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:05 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:16:05.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:05 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:16:05.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:05 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:16:05.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:05 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:16:05.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:05 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:16:05.531 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:05 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:16:06.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:06.039+0000 7ff8c5600640 1 -- 172.21.15.43:0/2167110461 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v99) ==== 169+0+0 (secure 0 0 0) 0x7ff8b00a06a0 con 0x7ff8c81049a0 2024-09-19T05:16:06.041 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' removed 2024-09-19T05:16:06.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:06.043+0000 7ff8cee00640 1 -- 172.21.15.43:0/2167110461 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff8ac0a3430 msgr2=0x7ff8ac0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:06.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:06.043+0000 7ff8cee00640 1 --2- 172.21.15.43:0/2167110461 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff8ac0a3430 0x7ff8ac0a58f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff8b80096c0 tx=0x7ff8b8005a20 comp rx=0 tx=0).stop 2024-09-19T05:16:06.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:06.043+0000 7ff8cee00640 1 -- 172.21.15.43:0/2167110461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8c81049a0 msgr2=0x7ff8c819abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:16:06.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:06.043+0000 7ff8cee00640 1 --2- 172.21.15.43:0/2167110461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8c81049a0 0x7ff8c819abb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff8b000e9f0 tx=0x7ff8b000eeb0 comp rx=0 tx=0).stop 2024-09-19T05:16:06.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:06.043+0000 7ff8cee00640 1 -- 172.21.15.43:0/2167110461 shutdown_connections 2024-09-19T05:16:06.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:06.043+0000 7ff8cee00640 1 --2- 172.21.15.43:0/2167110461 >> [v2:172.21.15.43:6800/2120444093,v1:172.21.15.43:6801/2120444093] conn(0x7ff8ac0a3430 0x7ff8ac0a58f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:06.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:06.043+0000 7ff8cee00640 1 --2- 172.21.15.43:0/2167110461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8c8109120 0x7ff8c819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:06.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:06.043+0000 7ff8cee00640 1 --2- 172.21.15.43:0/2167110461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8c81049a0 0x7ff8c819abb0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:06.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:06.043+0000 7ff8cee00640 1 --2- 172.21.15.43:0/2167110461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8c8103ff0 0x7ff8c819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:16:06.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:06.043+0000 7ff8cee00640 1 -- 172.21.15.43:0/2167110461 >> 172.21.15.43:0/2167110461 conn(0x7ff8c80ffa90 msgr2=0x7ff8c8100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:16:06.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:06.043+0000 7ff8cee00640 1 -- 172.21.15.43:0/2167110461 shutdown_connections 2024-09-19T05:16:06.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-19T05:16:06.044+0000 7ff8cee00640 1 -- 172.21.15.43:0/2167110461 wait complete. 2024-09-19T05:16:06.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:06 smithi151 ceph-mon[35425]: pgmap v358: 65 pgs: 65 active+clean; 577 KiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:16:06.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:06 smithi151 ceph-mon[35425]: from='client.? 172.21.15.43:0/2167110461' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T05:16:06.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:06 smithi151 ceph-mon[35425]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T05:16:06.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:06 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:16:06.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:06 smithi151 ceph-mon[35425]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:16:06.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:06 smithi151 ceph-mon[35425]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:16:06.439 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:06 smithi043 ceph-mon[30800]: pgmap v358: 65 pgs: 65 active+clean; 577 KiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:16:06.439 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:06 smithi043 ceph-mon[30800]: from='client.? 172.21.15.43:0/2167110461' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T05:16:06.439 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:06 smithi043 ceph-mon[30800]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T05:16:06.439 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:06 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:16:06.440 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:06 smithi043 ceph-mon[30800]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:16:06.440 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:06 smithi043 ceph-mon[30800]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:16:06.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:06 smithi096 ceph-mon[35085]: pgmap v358: 65 pgs: 65 active+clean; 577 KiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T05:16:06.473 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:06 smithi096 ceph-mon[35085]: from='client.? 172.21.15.43:0/2167110461' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T05:16:06.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:06 smithi096 ceph-mon[35085]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T05:16:06.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:06 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T05:16:06.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:06 smithi096 ceph-mon[35085]: from='mgr.25136 172.21.15.43:0/1435390139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T05:16:06.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:06 smithi096 ceph-mon[35085]: from='mgr.25136 ' entity='mgr.x' 2024-09-19T05:16:06.835 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T05:16:06.893 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T05:16:06.926 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T05:16:06.960 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-19T05:16:06.960 DEBUG:teuthology.orchestra.run.smithi043:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T05:16:06.995 DEBUG:teuthology.orchestra.run.smithi096:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T05:16:07.013 DEBUG:teuthology.orchestra.run.smithi151:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T05:16:07.031 INFO:tasks.cephadm:Stopping all daemons... 2024-09-19T05:16:07.032 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-19T05:16:07.032 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-1456a810-7643-11ef-bceb-c7b262605968@mon.a 2024-09-19T05:16:07.085 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:07 smithi043 ceph-mon[30800]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T05:16:07.085 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:07 smithi043 ceph-mon[30800]: osdmap e99: 12 total, 12 up, 12 in 2024-09-19T05:16:07.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:07 smithi096 ceph-mon[35085]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T05:16:07.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:07 smithi096 ceph-mon[35085]: osdmap e99: 12 total, 12 up, 12 in 2024-09-19T05:16:07.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:07 smithi151 ceph-mon[35425]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T05:16:07.405 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:07 smithi151 ceph-mon[35425]: osdmap e99: 12 total, 12 up, 12 in 2024-09-19T05:16:07.464 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:07 smithi043 systemd[1]: Stopping Ceph mon.a for 1456a810-7643-11ef-bceb-c7b262605968... 2024-09-19T05:16:07.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:07 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mon-a[30796]: 2024-09-19T05:16:07.462+0000 7ff06a200640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:16:07.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:07 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mon-a[30796]: 2024-09-19T05:16:07.462+0000 7ff06a200640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-19T05:16:07.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:07 smithi043 podman[108009]: 2024-09-19 05:16:07.657742363 +0000 UTC m=+0.331400918 container died cd395eb8a2c34a2d7fedb2538263765d6b841bfe0aeab36999c549b57daf5a67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mon-a, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-19T05:16:08.265 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:08 smithi043 podman[108009]: 2024-09-19 05:16:08.234644963 +0000 UTC m=+0.908303518 container cleanup cd395eb8a2c34a2d7fedb2538263765d6b841bfe0aeab36999c549b57daf5a67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-19T05:16:08.265 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:08 smithi043 bash[108009]: ceph-1456a810-7643-11ef-bceb-c7b262605968-mon-a 2024-09-19T05:16:08.672 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 19 05:16:08 smithi043 podman[108021]: 2024-09-19 05:16:08.411205553 +0000 UTC m=+0.749189224 container remove cd395eb8a2c34a2d7fedb2538263765d6b841bfe0aeab36999c549b57daf5a67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mon-a, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-19T05:16:08.691 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@mon.a.service' 2024-09-19T05:16:08.786 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:16:08.786 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-19T05:16:08.786 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-19T05:16:08.786 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-1456a810-7643-11ef-bceb-c7b262605968@mon.b 2024-09-19T05:16:09.223 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:08 smithi096 systemd[1]: Stopping Ceph mon.b for 1456a810-7643-11ef-bceb-c7b262605968... 2024-09-19T05:16:09.532 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:09 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mon-b[35081]: 2024-09-19T05:16:09.264+0000 7fc4f3201640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:16:09.761 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:09 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mon-b[35081]: 2024-09-19T05:16:09.264+0000 7fc4f3201640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-19T05:16:09.834 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:09 smithi096 podman[70585]: 2024-09-19 05:16:09.530521048 +0000 UTC m=+0.542489148 container died 7a4dd79a850d74485604503418bd7b9013735a9d00f6879df263ebb92d5dc2a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mon-b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, ceph=True, GIT_BRANCH=HEAD) 2024-09-19T05:16:10.184 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:09 smithi096 podman[70585]: 2024-09-19 05:16:09.939829835 +0000 UTC m=+0.951797935 container cleanup 7a4dd79a850d74485604503418bd7b9013735a9d00f6879df263ebb92d5dc2a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mon-b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2) 2024-09-19T05:16:10.184 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 05:16:09 smithi096 bash[70585]: ceph-1456a810-7643-11ef-bceb-c7b262605968-mon-b 2024-09-19T05:16:10.313 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@mon.b.service' 2024-09-19T05:16:10.361 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:16:10.361 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-19T05:16:10.361 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-19T05:16:10.361 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-1456a810-7643-11ef-bceb-c7b262605968@mon.c 2024-09-19T05:16:10.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:10 smithi151 systemd[1]: Stopping Ceph mon.c for 1456a810-7643-11ef-bceb-c7b262605968... 2024-09-19T05:16:11.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:10 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mon-c[35421]: 2024-09-19T05:16:10.694+0000 7f84aa614640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:16:11.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:10 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mon-c[35421]: 2024-09-19T05:16:10.694+0000 7f84aa614640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-19T05:16:11.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:10 smithi151 podman[71408]: 2024-09-19 05:16:10.919605304 +0000 UTC m=+0.362007671 container died 92705e2799e7cc20a1ac0925b0f6ff3a51a0330bb6d9dc505889143a6264327c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mon-c, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-19T05:16:11.581 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:11 smithi151 podman[71408]: 2024-09-19 05:16:11.329026274 +0000 UTC m=+0.771428675 container cleanup 92705e2799e7cc20a1ac0925b0f6ff3a51a0330bb6d9dc505889143a6264327c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-19T05:16:11.582 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:11 smithi151 bash[71408]: ceph-1456a810-7643-11ef-bceb-c7b262605968-mon-c 2024-09-19T05:16:11.814 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@mon.c.service' 2024-09-19T05:16:11.836 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:11 smithi151 podman[71421]: 2024-09-19 05:16:11.580851752 +0000 UTC m=+0.658774618 container remove 92705e2799e7cc20a1ac0925b0f6ff3a51a0330bb6d9dc505889143a6264327c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mon-c, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-19T05:16:11.836 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:11 smithi151 systemd[1]: ceph-1456a810-7643-11ef-bceb-c7b262605968@mon.c.service: Deactivated successfully. 2024-09-19T05:16:11.836 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:11 smithi151 systemd[1]: Stopped Ceph mon.c for 1456a810-7643-11ef-bceb-c7b262605968. 2024-09-19T05:16:11.837 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 05:16:11 smithi151 systemd[1]: ceph-1456a810-7643-11ef-bceb-c7b262605968@mon.c.service: Consumed 14.511s CPU time. 2024-09-19T05:16:11.866 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:16:11.866 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-19T05:16:11.866 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-19T05:16:11.867 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-1456a810-7643-11ef-bceb-c7b262605968@mgr.x 2024-09-19T05:16:12.281 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:16:11 smithi043 systemd[1]: Stopping Ceph mgr.x for 1456a810-7643-11ef-bceb-c7b262605968... 2024-09-19T05:16:12.684 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:16:12 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:16:12.292+0000 7f8296a00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:16:12.684 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:16:12 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x[31026]: 2024-09-19T05:16:12.292+0000 7f8296a00640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-19T05:16:12.685 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:16:12 smithi043 podman[108131]: 2024-09-19 05:16:12.375127872 +0000 UTC m=+0.210842377 container died 185a2dbed7db0f598ffc0d6685cfceb3b2ccf3610b2e2972c06173250d00123d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T05:16:13.277 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:16:12 smithi043 podman[108131]: 2024-09-19 05:16:12.990196383 +0000 UTC m=+0.825910882 container cleanup 185a2dbed7db0f598ffc0d6685cfceb3b2ccf3610b2e2972c06173250d00123d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-19T05:16:13.277 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:16:12 smithi043 bash[108131]: ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x 2024-09-19T05:16:13.489 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-19T05:16:13.531 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:16:13 smithi043 podman[108143]: 2024-09-19 05:16:13.275724676 +0000 UTC m=+0.888745915 container remove 185a2dbed7db0f598ffc0d6685cfceb3b2ccf3610b2e2972c06173250d00123d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-x, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-19T05:16:13.532 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:16:13 smithi043 systemd[1]: ceph-1456a810-7643-11ef-bceb-c7b262605968@mgr.x.service: Deactivated successfully. 2024-09-19T05:16:13.532 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:16:13 smithi043 systemd[1]: Stopped Ceph mgr.x for 1456a810-7643-11ef-bceb-c7b262605968. 2024-09-19T05:16:13.532 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 19 05:16:13 smithi043 systemd[1]: ceph-1456a810-7643-11ef-bceb-c7b262605968@mgr.x.service: Consumed 1min 3.723s CPU time. 2024-09-19T05:16:13.565 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:16:13.566 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-19T05:16:13.567 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-19T05:16:13.568 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-1456a810-7643-11ef-bceb-c7b262605968@mgr.y 2024-09-19T05:16:13.917 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 05:16:13 smithi096 systemd[1]: Stopping Ceph mgr.y for 1456a810-7643-11ef-bceb-c7b262605968... 2024-09-19T05:16:14.179 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 05:16:13 smithi096 podman[70708]: 2024-09-19 05:16:13.914929495 +0000 UTC m=+0.157445391 container died 97a90af90dc6cd3617342bc404baae1e68f917691bb326280f741eb9daee1cbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD) 2024-09-19T05:16:14.473 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 05:16:14 smithi096 podman[70708]: 2024-09-19 05:16:14.278065883 +0000 UTC m=+0.520581778 container cleanup 97a90af90dc6cd3617342bc404baae1e68f917691bb326280f741eb9daee1cbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-19T05:16:14.473 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 05:16:14 smithi096 bash[70708]: ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-y 2024-09-19T05:16:14.861 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-19T05:16:14.897 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:16:14.897 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-19T05:16:14.897 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-19T05:16:14.898 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-1456a810-7643-11ef-bceb-c7b262605968@mgr.z 2024-09-19T05:16:15.240 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 05:16:14 smithi151 systemd[1]: Stopping Ceph mgr.z for 1456a810-7643-11ef-bceb-c7b262605968... 2024-09-19T05:16:15.546 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 05:16:15 smithi151 podman[71532]: 2024-09-19 05:16:15.238501386 +0000 UTC m=+0.138742832 container died 1e3c4197daf2699d5423f0bd348688b12841ab2a1270a9562ccf4ecb38c511d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-19T05:16:15.905 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 05:16:15 smithi151 podman[71532]: 2024-09-19 05:16:15.67767103 +0000 UTC m=+0.577912464 container cleanup 1e3c4197daf2699d5423f0bd348688b12841ab2a1270a9562ccf4ecb38c511d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T05:16:15.905 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 05:16:15 smithi151 bash[71532]: ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z 2024-09-19T05:16:16.146 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@mgr.z.service' 2024-09-19T05:16:16.167 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 05:16:15 smithi151 podman[71544]: 2024-09-19 05:16:15.91301364 +0000 UTC m=+0.668706285 container remove 1e3c4197daf2699d5423f0bd348688b12841ab2a1270a9562ccf4ecb38c511d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mgr-z, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:16:16.167 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 05:16:15 smithi151 systemd[1]: ceph-1456a810-7643-11ef-bceb-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-09-19T05:16:16.167 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 05:16:16 smithi151 systemd[1]: ceph-1456a810-7643-11ef-bceb-c7b262605968@mgr.z.service: Failed with result 'exit-code'. 2024-09-19T05:16:16.167 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 05:16:16 smithi151 systemd[1]: Stopped Ceph mgr.z for 1456a810-7643-11ef-bceb-c7b262605968. 2024-09-19T05:16:16.168 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 05:16:16 smithi151 systemd[1]: ceph-1456a810-7643-11ef-bceb-c7b262605968@mgr.z.service: Consumed 11.554s CPU time. 2024-09-19T05:16:16.181 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:16:16.181 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-19T05:16:16.181 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-19T05:16:16.181 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-1456a810-7643-11ef-bceb-c7b262605968@osd.0 2024-09-19T05:16:16.531 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 19 05:16:16 smithi043 systemd[1]: Stopping Ceph osd.0 for 1456a810-7643-11ef-bceb-c7b262605968... 2024-09-19T05:16:17.031 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 19 05:16:16 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-0[41934]: 2024-09-19T05:16:16.598+0000 7f176c400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:16:17.031 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 19 05:16:16 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-0[41934]: 2024-09-19T05:16:16.598+0000 7f176c400640 -1 osd.0 99 *** Got signal Terminated *** 2024-09-19T05:16:17.031 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 19 05:16:16 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-0[41934]: 2024-09-19T05:16:16.598+0000 7f176c400640 -1 osd.0 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T05:16:22.449 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 19 05:16:22 smithi043 podman[108253]: 2024-09-19 05:16:22.145329844 +0000 UTC m=+5.677986642 container died d6d47d99edac48381be5c6008d0a8fdf548a298d4c286a92e801099f727b7ea3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-0, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-19T05:16:22.781 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 19 05:16:22 smithi043 podman[108253]: 2024-09-19 05:16:22.561942765 +0000 UTC m=+6.094599571 container cleanup d6d47d99edac48381be5c6008d0a8fdf548a298d4c286a92e801099f727b7ea3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-0, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-19T05:16:22.781 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 19 05:16:22 smithi043 bash[108253]: ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-0 2024-09-19T05:16:23.280 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 19 05:16:22 smithi043 podman[108271]: 2024-09-19 05:16:22.847183407 +0000 UTC m=+0.695596750 container remove d6d47d99edac48381be5c6008d0a8fdf548a298d4c286a92e801099f727b7ea3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-0, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, io.buildah.version=1.37.2) 2024-09-19T05:16:23.681 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 19 05:16:23 smithi043 podman[108367]: 2024-09-19 05:16:23.292718292 +0000 UTC m=+0.027935483 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:16:23.681 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 19 05:16:23 smithi043 podman[108367]: 2024-09-19 05:16:23.423424118 +0000 UTC m=+0.158641308 container create cbd521fbfe9c674c4eb7545555b38d4b359a0f8b6c781cc18424a4ce9df5ec15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-19T05:16:23.681 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 19 05:16:23 smithi043 podman[108367]: 2024-09-19 05:16:23.676213746 +0000 UTC m=+0.411430938 container init cbd521fbfe9c674c4eb7545555b38d4b359a0f8b6c781cc18424a4ce9df5ec15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0) 2024-09-19T05:16:23.948 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 19 05:16:23 smithi043 podman[108367]: 2024-09-19 05:16:23.683107253 +0000 UTC m=+0.418324445 container start cbd521fbfe9c674c4eb7545555b38d4b359a0f8b6c781cc18424a4ce9df5ec15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-19T05:16:23.948 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 19 05:16:23 smithi043 podman[108367]: 2024-09-19 05:16:23.809401322 +0000 UTC m=+0.544618526 container attach cbd521fbfe9c674c4eb7545555b38d4b359a0f8b6c781cc18424a4ce9df5ec15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T05:16:24.280 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 19 05:16:23 smithi043 podman[108367]: 2024-09-19 05:16:23.947564614 +0000 UTC m=+0.682781805 container died cbd521fbfe9c674c4eb7545555b38d4b359a0f8b6c781cc18424a4ce9df5ec15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-19T05:16:24.668 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@osd.0.service' 2024-09-19T05:16:24.745 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:16:24.745 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-19T05:16:24.745 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-19T05:16:24.745 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-1456a810-7643-11ef-bceb-c7b262605968@osd.1 2024-09-19T05:16:25.058 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 05:16:24 smithi096 systemd[1]: Stopping Ceph osd.1 for 1456a810-7643-11ef-bceb-c7b262605968... 2024-09-19T05:16:25.473 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 05:16:25 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-1[39579]: 2024-09-19T05:16:25.056+0000 7fbee352b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:16:25.473 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 05:16:25 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-1[39579]: 2024-09-19T05:16:25.056+0000 7fbee352b640 -1 osd.1 99 *** Got signal Terminated *** 2024-09-19T05:16:25.474 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 05:16:25 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-1[39579]: 2024-09-19T05:16:25.056+0000 7fbee352b640 -1 osd.1 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T05:16:30.835 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 05:16:30 smithi096 podman[70832]: 2024-09-19 05:16:30.50585903 +0000 UTC m=+5.560922886 container died 74367a52818f3145f38adbca494644ffee5ada5d42c67a4b441c89980543a182 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-1, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-19T05:16:31.159 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 05:16:30 smithi096 podman[70832]: 2024-09-19 05:16:30.931888112 +0000 UTC m=+5.986951973 container cleanup 74367a52818f3145f38adbca494644ffee5ada5d42c67a4b441c89980543a182 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-1, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T05:16:31.159 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 05:16:30 smithi096 bash[70832]: ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-1 2024-09-19T05:16:31.473 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 05:16:31 smithi096 podman[70858]: 2024-09-19 05:16:31.158420581 +0000 UTC m=+0.650983638 container remove 74367a52818f3145f38adbca494644ffee5ada5d42c67a4b441c89980543a182 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-1, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-2f599a4, io.buildah.version=1.37.2) 2024-09-19T05:16:31.905 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 05:16:31 smithi096 podman[70950]: 2024-09-19 05:16:31.500122122 +0000 UTC m=+0.017028370 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:16:31.905 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 05:16:31 smithi096 podman[70950]: 2024-09-19 05:16:31.626114491 +0000 UTC m=+0.143020801 container create 718893caac6dce37f15ec6f4099269698f3c639dfc9951dd2f8e60c4800807e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T05:16:31.905 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 05:16:31 smithi096 podman[70950]: 2024-09-19 05:16:31.90352417 +0000 UTC m=+0.420430420 container init 718893caac6dce37f15ec6f4099269698f3c639dfc9951dd2f8e60c4800807e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-1-deactivate, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T05:16:32.223 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 05:16:31 smithi096 podman[70950]: 2024-09-19 05:16:31.90712125 +0000 UTC m=+0.424027510 container start 718893caac6dce37f15ec6f4099269698f3c639dfc9951dd2f8e60c4800807e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-19T05:16:32.224 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 05:16:31 smithi096 podman[70950]: 2024-09-19 05:16:31.98695823 +0000 UTC m=+0.503864489 container attach 718893caac6dce37f15ec6f4099269698f3c639dfc9951dd2f8e60c4800807e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD) 2024-09-19T05:16:32.224 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 05:16:32 smithi096 podman[70950]: 2024-09-19 05:16:32.107943322 +0000 UTC m=+0.624849571 container died 718893caac6dce37f15ec6f4099269698f3c639dfc9951dd2f8e60c4800807e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-19T05:16:32.778 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@osd.1.service' 2024-09-19T05:16:32.818 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:16:32.818 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-19T05:16:32.818 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-19T05:16:32.818 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-1456a810-7643-11ef-bceb-c7b262605968@osd.2 2024-09-19T05:16:33.155 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 05:16:32 smithi151 systemd[1]: Stopping Ceph osd.2 for 1456a810-7643-11ef-bceb-c7b262605968... 2024-09-19T05:16:33.905 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 05:16:33 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-2[40388]: 2024-09-19T05:16:33.423+0000 7ff6c4c29640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:16:33.905 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 05:16:33 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-2[40388]: 2024-09-19T05:16:33.423+0000 7ff6c4c29640 -1 osd.2 99 *** Got signal Terminated *** 2024-09-19T05:16:33.905 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 05:16:33 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-2[40388]: 2024-09-19T05:16:33.423+0000 7ff6c4c29640 -1 osd.2 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T05:16:39.155 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 05:16:38 smithi151 podman[71658]: 2024-09-19 05:16:38.880595557 +0000 UTC m=+5.866525968 container died 81f63e983b994fed5d951d88a3b1f6c8ec904ce43b83487045e4d69864ce6581 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-2, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-19T05:16:39.553 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 05:16:39 smithi151 podman[71658]: 2024-09-19 05:16:39.308459121 +0000 UTC m=+6.294389531 container cleanup 81f63e983b994fed5d951d88a3b1f6c8ec904ce43b83487045e4d69864ce6581 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-19T05:16:39.553 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 05:16:39 smithi151 bash[71658]: ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-2 2024-09-19T05:16:39.905 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 05:16:39 smithi151 podman[71675]: 2024-09-19 05:16:39.551963285 +0000 UTC m=+0.668399076 container remove 81f63e983b994fed5d951d88a3b1f6c8ec904ce43b83487045e4d69864ce6581 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:16:40.258 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 05:16:39 smithi151 podman[71762]: 2024-09-19 05:16:39.87381338 +0000 UTC m=+0.015937828 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:16:40.258 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 05:16:39 smithi151 podman[71762]: 2024-09-19 05:16:39.986646985 +0000 UTC m=+0.128771429 container create 77fbb138dac62656f9c1332d0f2eb675933aec3a9c8b64114976870ba860754c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-19T05:16:40.655 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 05:16:40 smithi151 podman[71762]: 2024-09-19 05:16:40.256973679 +0000 UTC m=+0.399098132 container init 77fbb138dac62656f9c1332d0f2eb675933aec3a9c8b64114976870ba860754c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T05:16:40.655 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 05:16:40 smithi151 podman[71762]: 2024-09-19 05:16:40.260688865 +0000 UTC m=+0.402813320 container start 77fbb138dac62656f9c1332d0f2eb675933aec3a9c8b64114976870ba860754c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-19T05:16:40.656 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 05:16:40 smithi151 podman[71762]: 2024-09-19 05:16:40.35686641 +0000 UTC m=+0.498990874 container attach 77fbb138dac62656f9c1332d0f2eb675933aec3a9c8b64114976870ba860754c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-19T05:16:40.656 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 05:16:40 smithi151 podman[71762]: 2024-09-19 05:16:40.4522412 +0000 UTC m=+0.594365646 container died 77fbb138dac62656f9c1332d0f2eb675933aec3a9c8b64114976870ba860754c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_CLEAN=True) 2024-09-19T05:16:41.073 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@osd.2.service' 2024-09-19T05:16:41.114 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:16:41.114 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-19T05:16:41.114 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-19T05:16:41.114 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-1456a810-7643-11ef-bceb-c7b262605968@osd.3 2024-09-19T05:16:41.531 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 19 05:16:41 smithi043 systemd[1]: Stopping Ceph osd.3 for 1456a810-7643-11ef-bceb-c7b262605968... 2024-09-19T05:16:42.030 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 19 05:16:41 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-3[48868]: 2024-09-19T05:16:41.534+0000 7fdacba00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:16:42.031 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 19 05:16:41 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-3[48868]: 2024-09-19T05:16:41.534+0000 7fdacba00640 -1 osd.3 99 *** Got signal Terminated *** 2024-09-19T05:16:42.031 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 19 05:16:41 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-3[48868]: 2024-09-19T05:16:41.534+0000 7fdacba00640 -1 osd.3 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T05:16:42.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:16:42 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:16:42.626+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:16:43.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:16:43 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:16:43.629+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:16:44.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:16:43 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:16:43.595+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:16:44.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:16:44 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:16:44.594+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:16:45.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:16:44 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:16:44.617+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:16:45.548 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:16:45 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5[46846]: 2024-09-19T05:16:45.264+0000 7fa5ccbf1640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.779197+0000 front 2024-09-19T05:16:20.779097+0000 (oldest deadline 2024-09-19T05:16:44.878870+0000) 2024-09-19T05:16:45.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:16:45 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:16:45.547+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:16:46.033 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 19 05:16:45 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-3[48868]: 2024-09-19T05:16:45.719+0000 7fdac2400640 -1 osd.3 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.190761+0000 front 2024-09-19T05:16:19.190837+0000 (oldest deadline 2024-09-19T05:16:45.090870+0000) 2024-09-19T05:16:46.034 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:16:45 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:16:45.603+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:16:46.034 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:16:45 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:16:45.845+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.347642+0000 front 2024-09-19T05:16:20.347562+0000 (oldest deadline 2024-09-19T05:16:45.047862+0000) 2024-09-19T05:16:46.223 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:16:45 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:16:45.816+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:16:46.578 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:16:46 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5[46846]: 2024-09-19T05:16:46.235+0000 7fa5ccbf1640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.779197+0000 front 2024-09-19T05:16:20.779097+0000 (oldest deadline 2024-09-19T05:16:44.878870+0000) 2024-09-19T05:16:46.881 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:16:46 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:16:46.623+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:16:46.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:16:46 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:16:46.577+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:16:47.155 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:16:46 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:16:46.878+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.347642+0000 front 2024-09-19T05:16:20.347562+0000 (oldest deadline 2024-09-19T05:16:45.047862+0000) 2024-09-19T05:16:47.223 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:16:46 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:16:46.838+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:16:47.467 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 19 05:16:47 smithi043 podman[108470]: 2024-09-19 05:16:47.15460023 +0000 UTC m=+5.754055340 container died 12571aea2f41b458205711619c2da6db74a2647ece9580b6e3e6f7f5eb3226a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-3, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-19T05:16:47.562 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:16:47 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5[46846]: 2024-09-19T05:16:47.213+0000 7fa5ccbf1640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.779197+0000 front 2024-09-19T05:16:20.779097+0000 (oldest deadline 2024-09-19T05:16:44.878870+0000) 2024-09-19T05:16:47.719 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 19 05:16:47 smithi043 podman[108470]: 2024-09-19 05:16:47.699360885 +0000 UTC m=+6.298815994 container cleanup 12571aea2f41b458205711619c2da6db74a2647ece9580b6e3e6f7f5eb3226a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-3, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-2f599a4) 2024-09-19T05:16:47.720 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 19 05:16:47 smithi043 bash[108470]: ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-3 2024-09-19T05:16:47.720 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:16:47 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:16:47.612+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:16:47.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:16:47 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:16:47.560+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:16:48.030 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:16:47 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:16:47.843+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.347642+0000 front 2024-09-19T05:16:20.347562+0000 (oldest deadline 2024-09-19T05:16:45.047862+0000) 2024-09-19T05:16:48.031 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 19 05:16:47 smithi043 podman[108488]: 2024-09-19 05:16:47.967688663 +0000 UTC m=+0.801637077 container remove 12571aea2f41b458205711619c2da6db74a2647ece9580b6e3e6f7f5eb3226a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-3, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True) 2024-09-19T05:16:48.223 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:16:47 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:16:47.867+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:16:48.580 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:16:48 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5[46846]: 2024-09-19T05:16:48.193+0000 7fa5ccbf1640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.779197+0000 front 2024-09-19T05:16:20.779097+0000 (oldest deadline 2024-09-19T05:16:44.878870+0000) 2024-09-19T05:16:48.814 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 19 05:16:48 smithi043 podman[108570]: 2024-09-19 05:16:48.439181176 +0000 UTC m=+0.027768792 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:16:48.814 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 19 05:16:48 smithi043 podman[108570]: 2024-09-19 05:16:48.552020693 +0000 UTC m=+0.140608278 container create a891a8fc51a2390298f3d9f4ae3dcfcfb04ba880011473c918e3987f0d4363f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-19T05:16:48.815 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:16:48 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:16:48.599+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:16:48.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:16:48 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:16:48.578+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:16:49.086 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 19 05:16:48 smithi043 podman[108570]: 2024-09-19 05:16:48.813041215 +0000 UTC m=+0.401628831 container init a891a8fc51a2390298f3d9f4ae3dcfcfb04ba880011473c918e3987f0d4363f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4) 2024-09-19T05:16:49.087 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 19 05:16:48 smithi043 podman[108570]: 2024-09-19 05:16:48.819980157 +0000 UTC m=+0.408567743 container start a891a8fc51a2390298f3d9f4ae3dcfcfb04ba880011473c918e3987f0d4363f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-19T05:16:49.087 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 19 05:16:48 smithi043 podman[108570]: 2024-09-19 05:16:48.913005047 +0000 UTC m=+0.501592633 container attach a891a8fc51a2390298f3d9f4ae3dcfcfb04ba880011473c918e3987f0d4363f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-3-deactivate, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-19T05:16:49.087 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:16:48 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:16:48.841+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.347642+0000 front 2024-09-19T05:16:20.347562+0000 (oldest deadline 2024-09-19T05:16:45.047862+0000) 2024-09-19T05:16:49.223 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:16:48 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:16:48.865+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:16:49.429 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 19 05:16:49 smithi043 podman[108570]: 2024-09-19 05:16:49.086304753 +0000 UTC m=+0.674892352 container died a891a8fc51a2390298f3d9f4ae3dcfcfb04ba880011473c918e3987f0d4363f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-3-deactivate, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-19T05:16:49.567 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:16:49 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5[46846]: 2024-09-19T05:16:49.161+0000 7fa5ccbf1640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.779197+0000 front 2024-09-19T05:16:20.779097+0000 (oldest deadline 2024-09-19T05:16:44.878870+0000) 2024-09-19T05:16:49.770 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:16:49 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:16:49.603+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:16:49.888 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@osd.3.service' 2024-09-19T05:16:49.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:16:49 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:16:49.566+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:16:49.965 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:16:49.965 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-19T05:16:49.965 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-19T05:16:49.965 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-1456a810-7643-11ef-bceb-c7b262605968@osd.4 2024-09-19T05:16:50.031 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:16:49 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:16:49.853+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.347642+0000 front 2024-09-19T05:16:20.347562+0000 (oldest deadline 2024-09-19T05:16:45.047862+0000) 2024-09-19T05:16:50.223 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 19 05:16:50 smithi096 systemd[1]: Stopping Ceph osd.4 for 1456a810-7643-11ef-bceb-c7b262605968... 2024-09-19T05:16:50.224 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:16:49 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:16:49.857+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:16:50.568 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:16:50 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5[46846]: 2024-09-19T05:16:50.199+0000 7fa5ccbf1640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.779197+0000 front 2024-09-19T05:16:20.779097+0000 (oldest deadline 2024-09-19T05:16:44.878870+0000) 2024-09-19T05:16:50.813 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 19 05:16:50 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-4[46023]: 2024-09-19T05:16:50.559+0000 7f7149ddf640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:16:50.813 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 19 05:16:50 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-4[46023]: 2024-09-19T05:16:50.559+0000 7f7149ddf640 -1 osd.4 99 *** Got signal Terminated *** 2024-09-19T05:16:50.813 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 19 05:16:50 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-4[46023]: 2024-09-19T05:16:50.559+0000 7f7149ddf640 -1 osd.4 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T05:16:50.846 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:16:50 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:16:50.593+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:16:50.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:16:50 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:16:50.567+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:16:51.223 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:16:50 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:16:50.811+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:16:51.280 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:16:50 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:16:50.844+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.347642+0000 front 2024-09-19T05:16:20.347562+0000 (oldest deadline 2024-09-19T05:16:45.047862+0000) 2024-09-19T05:16:51.577 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:16:51 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5[46846]: 2024-09-19T05:16:51.208+0000 7fa5ccbf1640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.779197+0000 front 2024-09-19T05:16:20.779097+0000 (oldest deadline 2024-09-19T05:16:44.878870+0000) 2024-09-19T05:16:51.865 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:16:51 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:16:51.560+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:16:51.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:16:51 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:16:51.575+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:16:52.223 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:16:51 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:16:51.845+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:16:52.281 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:16:51 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:16:51.864+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.347642+0000 front 2024-09-19T05:16:20.347562+0000 (oldest deadline 2024-09-19T05:16:45.047862+0000) 2024-09-19T05:16:52.610 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:16:52 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5[46846]: 2024-09-19T05:16:52.212+0000 7fa5ccbf1640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.779197+0000 front 2024-09-19T05:16:20.779097+0000 (oldest deadline 2024-09-19T05:16:44.878870+0000) 2024-09-19T05:16:52.610 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:16:52 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5[46846]: 2024-09-19T05:16:52.212+0000 7fa5ccbf1640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.480388+0000 front 2024-09-19T05:16:29.480333+0000 (oldest deadline 2024-09-19T05:16:51.780349+0000) 2024-09-19T05:16:52.830 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:16:52 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:16:52.569+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:16:52.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:16:52 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:16:52.608+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:16:53.147 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:16:52 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:16:52.848+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:16:53.147 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:16:52 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:16:52.848+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:16:53.281 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:16:52 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:16:52.827+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.347642+0000 front 2024-09-19T05:16:20.347562+0000 (oldest deadline 2024-09-19T05:16:45.047862+0000) 2024-09-19T05:16:53.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:16:53 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:16:53.145+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:27.546438+0000 front 2024-09-19T05:16:27.546400+0000 (oldest deadline 2024-09-19T05:16:52.846815+0000) 2024-09-19T05:16:53.610 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:16:53 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5[46846]: 2024-09-19T05:16:53.241+0000 7fa5ccbf1640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.779197+0000 front 2024-09-19T05:16:20.779097+0000 (oldest deadline 2024-09-19T05:16:44.878870+0000) 2024-09-19T05:16:53.610 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:16:53 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5[46846]: 2024-09-19T05:16:53.241+0000 7fa5ccbf1640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.480388+0000 front 2024-09-19T05:16:29.480333+0000 (oldest deadline 2024-09-19T05:16:51.780349+0000) 2024-09-19T05:16:53.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:16:53 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:16:53.608+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:16:54.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:16:53 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:16:53.593+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:16:54.031 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:16:53 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:16:53.826+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.347642+0000 front 2024-09-19T05:16:20.347562+0000 (oldest deadline 2024-09-19T05:16:45.047862+0000) 2024-09-19T05:16:54.172 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:16:53 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:16:53.839+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:16:54.172 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:16:53 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:16:53.839+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:16:54.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:16:54 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:16:54.170+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:27.546438+0000 front 2024-09-19T05:16:27.546400+0000 (oldest deadline 2024-09-19T05:16:52.846815+0000) 2024-09-19T05:16:54.591 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:16:54 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5[46846]: 2024-09-19T05:16:54.195+0000 7fa5ccbf1640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.779197+0000 front 2024-09-19T05:16:20.779097+0000 (oldest deadline 2024-09-19T05:16:44.878870+0000) 2024-09-19T05:16:54.591 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:16:54 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5[46846]: 2024-09-19T05:16:54.195+0000 7fa5ccbf1640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.480388+0000 front 2024-09-19T05:16:29.480333+0000 (oldest deadline 2024-09-19T05:16:51.780349+0000) 2024-09-19T05:16:54.857 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:16:54 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:16:54.590+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:16:54.857 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:16:54 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:16:54.590+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.976294+0000 front 2024-09-19T05:16:28.976202+0000 (oldest deadline 2024-09-19T05:16:54.276323+0000) 2024-09-19T05:16:54.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:16:54 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:16:54.590+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:16:54.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:16:54 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:16:54.590+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:16:55.151 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:16:54 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:16:54.846+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:16:55.151 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:16:54 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:16:54.846+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:16:55.281 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:16:54 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:16:54.855+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.347642+0000 front 2024-09-19T05:16:20.347562+0000 (oldest deadline 2024-09-19T05:16:45.047862+0000) 2024-09-19T05:16:55.448 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 19 05:16:55 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-4[46023]: 2024-09-19T05:16:55.218+0000 7f71463f5640 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.765109+0000 front 2024-09-19T05:16:29.765079+0000 (oldest deadline 2024-09-19T05:16:55.065280+0000) 2024-09-19T05:16:55.448 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:16:55 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:16:55.149+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:27.546438+0000 front 2024-09-19T05:16:27.546400+0000 (oldest deadline 2024-09-19T05:16:52.846815+0000) 2024-09-19T05:16:55.597 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:16:55 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5[46846]: 2024-09-19T05:16:55.188+0000 7fa5ccbf1640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.779197+0000 front 2024-09-19T05:16:20.779097+0000 (oldest deadline 2024-09-19T05:16:44.878870+0000) 2024-09-19T05:16:55.597 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:16:55 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5[46846]: 2024-09-19T05:16:55.188+0000 7fa5ccbf1640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.480388+0000 front 2024-09-19T05:16:29.480333+0000 (oldest deadline 2024-09-19T05:16:51.780349+0000) 2024-09-19T05:16:55.597 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:16:55 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:16:55.379+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.631981+0000 front 2024-09-19T05:16:29.631823+0000 (oldest deadline 2024-09-19T05:16:54.931686+0000) 2024-09-19T05:16:55.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:16:55 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:16:55.595+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:16:55.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:16:55 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:16:55.595+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:16:56.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:16:55 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:16:55.628+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:16:56.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:16:55 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:16:55.628+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.976294+0000 front 2024-09-19T05:16:28.976202+0000 (oldest deadline 2024-09-19T05:16:54.276323+0000) 2024-09-19T05:16:56.031 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:16:55 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:16:55.868+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.347642+0000 front 2024-09-19T05:16:20.347562+0000 (oldest deadline 2024-09-19T05:16:45.047862+0000) 2024-09-19T05:16:56.031 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:16:55 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:16:55.868+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.749196+0000 front 2024-09-19T05:16:29.749016+0000 (oldest deadline 2024-09-19T05:16:55.648755+0000) 2024-09-19T05:16:56.170 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 19 05:16:56 smithi096 podman[71052]: 2024-09-19 05:16:56.085245436 +0000 UTC m=+5.927879212 container died 39519f99e72fd6226d776c9d3ce101fdf97731ac40840f8f387cf49aab7b4d87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-4, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-19T05:16:56.171 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:16:55 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:16:55.881+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:16:56.171 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:16:55 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:16:55.881+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:16:56.426 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:16:56 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:16:56.168+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:27.546438+0000 front 2024-09-19T05:16:27.546400+0000 (oldest deadline 2024-09-19T05:16:52.846815+0000) 2024-09-19T05:16:56.560 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:16:56 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:16:56.368+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.631981+0000 front 2024-09-19T05:16:29.631823+0000 (oldest deadline 2024-09-19T05:16:54.931686+0000) 2024-09-19T05:16:56.560 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:16:56 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5[46846]: 2024-09-19T05:16:56.232+0000 7fa5ccbf1640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.779197+0000 front 2024-09-19T05:16:20.779097+0000 (oldest deadline 2024-09-19T05:16:44.878870+0000) 2024-09-19T05:16:56.561 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:16:56 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5[46846]: 2024-09-19T05:16:56.232+0000 7fa5ccbf1640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.480388+0000 front 2024-09-19T05:16:29.480333+0000 (oldest deadline 2024-09-19T05:16:51.780349+0000) 2024-09-19T05:16:56.723 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 19 05:16:56 smithi096 podman[71052]: 2024-09-19 05:16:56.52544655 +0000 UTC m=+6.368080334 container cleanup 39519f99e72fd6226d776c9d3ce101fdf97731ac40840f8f387cf49aab7b4d87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-4, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4) 2024-09-19T05:16:56.723 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 19 05:16:56 smithi096 bash[71052]: ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-4 2024-09-19T05:16:56.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:16:56 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:16:56.558+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:16:56.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:16:56 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:16:56.558+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:16:57.030 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:16:56 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:16:56.891+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.347642+0000 front 2024-09-19T05:16:20.347562+0000 (oldest deadline 2024-09-19T05:16:45.047862+0000) 2024-09-19T05:16:57.031 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:16:56 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:16:56.891+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.749196+0000 front 2024-09-19T05:16:29.749016+0000 (oldest deadline 2024-09-19T05:16:55.648755+0000) 2024-09-19T05:16:57.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:16:56 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:16:56.644+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:16:57.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:16:56 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:16:56.644+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.976294+0000 front 2024-09-19T05:16:28.976202+0000 (oldest deadline 2024-09-19T05:16:54.276323+0000) 2024-09-19T05:16:57.179 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 19 05:16:56 smithi096 podman[71081]: 2024-09-19 05:16:56.75184381 +0000 UTC m=+0.663938303 container remove 39519f99e72fd6226d776c9d3ce101fdf97731ac40840f8f387cf49aab7b4d87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-4, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-19T05:16:57.179 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:16:56 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:16:56.833+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:16:57.179 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:16:56 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:16:56.833+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:16:57.447 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:16:57 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:16:57.177+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:27.546438+0000 front 2024-09-19T05:16:27.546400+0000 (oldest deadline 2024-09-19T05:16:52.846815+0000) 2024-09-19T05:16:57.447 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 19 05:16:57 smithi096 podman[71166]: 2024-09-19 05:16:57.088231 +0000 UTC m=+0.015728298 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:16:57.447 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 19 05:16:57 smithi096 podman[71166]: 2024-09-19 05:16:57.194299695 +0000 UTC m=+0.121797026 container create ac67e11d4306aafd16169be5840d00ea1c1c369f95eb488ac0ebe335220ee465 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-19T05:16:57.447 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 19 05:16:57 smithi096 podman[71166]: 2024-09-19 05:16:57.446685236 +0000 UTC m=+0.374182525 container init ac67e11d4306aafd16169be5840d00ea1c1c369f95eb488ac0ebe335220ee465 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-19T05:16:57.557 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:16:57 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:16:57.347+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.631981+0000 front 2024-09-19T05:16:29.631823+0000 (oldest deadline 2024-09-19T05:16:54.931686+0000) 2024-09-19T05:16:57.558 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:16:57 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5[46846]: 2024-09-19T05:16:57.233+0000 7fa5ccbf1640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.779197+0000 front 2024-09-19T05:16:20.779097+0000 (oldest deadline 2024-09-19T05:16:44.878870+0000) 2024-09-19T05:16:57.558 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:16:57 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5[46846]: 2024-09-19T05:16:57.233+0000 7fa5ccbf1640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.480388+0000 front 2024-09-19T05:16:29.480333+0000 (oldest deadline 2024-09-19T05:16:51.780349+0000) 2024-09-19T05:16:57.724 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 19 05:16:57 smithi096 podman[71166]: 2024-09-19 05:16:57.449964944 +0000 UTC m=+0.377462234 container start ac67e11d4306aafd16169be5840d00ea1c1c369f95eb488ac0ebe335220ee465 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True) 2024-09-19T05:16:57.724 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 19 05:16:57 smithi096 podman[71166]: 2024-09-19 05:16:57.530093794 +0000 UTC m=+0.457591106 container attach ac67e11d4306aafd16169be5840d00ea1c1c369f95eb488ac0ebe335220ee465 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-19T05:16:57.724 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 19 05:16:57 smithi096 podman[71166]: 2024-09-19 05:16:57.636895197 +0000 UTC m=+0.564392500 container died ac67e11d4306aafd16169be5840d00ea1c1c369f95eb488ac0ebe335220ee465 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-19T05:16:57.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:16:57 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:16:57.556+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:16:57.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:16:57 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:16:57.556+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:16:58.031 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:16:57 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:16:57.890+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.347642+0000 front 2024-09-19T05:16:20.347562+0000 (oldest deadline 2024-09-19T05:16:45.047862+0000) 2024-09-19T05:16:58.031 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:16:57 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:16:57.890+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.749196+0000 front 2024-09-19T05:16:29.749016+0000 (oldest deadline 2024-09-19T05:16:55.648755+0000) 2024-09-19T05:16:58.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:16:57 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:16:57.669+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:16:58.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:16:57 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:16:57.669+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.976294+0000 front 2024-09-19T05:16:28.976202+0000 (oldest deadline 2024-09-19T05:16:54.276323+0000) 2024-09-19T05:16:58.206 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:16:57 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:16:57.809+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:16:58.206 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:16:57 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:16:57.809+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:16:58.295 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@osd.4.service' 2024-09-19T05:16:58.337 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:16:58.338 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-19T05:16:58.338 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-19T05:16:58.338 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-1456a810-7643-11ef-bceb-c7b262605968@osd.5 2024-09-19T05:16:58.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:16:58 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:16:58.204+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:27.546438+0000 front 2024-09-19T05:16:27.546400+0000 (oldest deadline 2024-09-19T05:16:52.846815+0000) 2024-09-19T05:16:58.562 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:16:58 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:16:58.351+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.631981+0000 front 2024-09-19T05:16:29.631823+0000 (oldest deadline 2024-09-19T05:16:54.931686+0000) 2024-09-19T05:16:58.563 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:16:58 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5[46846]: 2024-09-19T05:16:58.262+0000 7fa5ccbf1640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.779197+0000 front 2024-09-19T05:16:20.779097+0000 (oldest deadline 2024-09-19T05:16:44.878870+0000) 2024-09-19T05:16:58.563 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:16:58 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5[46846]: 2024-09-19T05:16:58.262+0000 7fa5ccbf1640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.480388+0000 front 2024-09-19T05:16:29.480333+0000 (oldest deadline 2024-09-19T05:16:51.780349+0000) 2024-09-19T05:16:58.563 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:16:58 smithi151 systemd[1]: Stopping Ceph osd.5 for 1456a810-7643-11ef-bceb-c7b262605968... 2024-09-19T05:16:58.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:16:58 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:16:58.561+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:16:58.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:16:58 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:16:58.561+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:16:58.905 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:16:58 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5[46846]: 2024-09-19T05:16:58.665+0000 7fa5d45e3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:16:58.905 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:16:58 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5[46846]: 2024-09-19T05:16:58.665+0000 7fa5d45e3640 -1 osd.5 99 *** Got signal Terminated *** 2024-09-19T05:16:58.906 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:16:58 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5[46846]: 2024-09-19T05:16:58.665+0000 7fa5d45e3640 -1 osd.5 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T05:16:58.917 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:16:58 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:16:58.653+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:16:58.917 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:16:58 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:16:58.653+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.976294+0000 front 2024-09-19T05:16:28.976202+0000 (oldest deadline 2024-09-19T05:16:54.276323+0000) 2024-09-19T05:16:59.185 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:16:58 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:16:58.783+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:16:59.185 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:16:58 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:16:58.783+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:16:59.280 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:16:58 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:16:58.915+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.347642+0000 front 2024-09-19T05:16:20.347562+0000 (oldest deadline 2024-09-19T05:16:45.047862+0000) 2024-09-19T05:16:59.281 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:16:58 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:16:58.915+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.749196+0000 front 2024-09-19T05:16:29.749016+0000 (oldest deadline 2024-09-19T05:16:55.648755+0000) 2024-09-19T05:16:59.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:16:59 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:16:59.184+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:27.546438+0000 front 2024-09-19T05:16:27.546400+0000 (oldest deadline 2024-09-19T05:16:52.846815+0000) 2024-09-19T05:16:59.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:16:59 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:16:59.184+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:38.048448+0000 front 2024-09-19T05:16:38.048358+0000 (oldest deadline 2024-09-19T05:16:59.148332+0000) 2024-09-19T05:16:59.527 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:16:59 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:16:59.341+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.631981+0000 front 2024-09-19T05:16:29.631823+0000 (oldest deadline 2024-09-19T05:16:54.931686+0000) 2024-09-19T05:16:59.527 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:16:59 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5[46846]: 2024-09-19T05:16:59.256+0000 7fa5ccbf1640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.779197+0000 front 2024-09-19T05:16:20.779097+0000 (oldest deadline 2024-09-19T05:16:44.878870+0000) 2024-09-19T05:16:59.527 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:16:59 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5[46846]: 2024-09-19T05:16:59.256+0000 7fa5ccbf1640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.480388+0000 front 2024-09-19T05:16:29.480333+0000 (oldest deadline 2024-09-19T05:16:51.780349+0000) 2024-09-19T05:16:59.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:16:59 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:16:59.526+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:16:59.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:16:59 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:16:59.526+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:16:59.962 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:16:59 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:16:59.677+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:16:59.962 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:16:59 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:16:59.677+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.976294+0000 front 2024-09-19T05:16:28.976202+0000 (oldest deadline 2024-09-19T05:16:54.276323+0000) 2024-09-19T05:16:59.962 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:16:59 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:16:59.677+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.678064+0000 front 2024-09-19T05:16:37.678149+0000 (oldest deadline 2024-09-19T05:16:59.278673+0000) 2024-09-19T05:17:00.139 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:16:59 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:16:59.758+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:00.139 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:16:59 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:16:59.758+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:00.139 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:16:59 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:16:59.758+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:00.281 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:16:59 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:16:59.960+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.347642+0000 front 2024-09-19T05:16:20.347562+0000 (oldest deadline 2024-09-19T05:16:45.047862+0000) 2024-09-19T05:17:00.281 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:16:59 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:16:59.960+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.749196+0000 front 2024-09-19T05:16:29.749016+0000 (oldest deadline 2024-09-19T05:16:55.648755+0000) 2024-09-19T05:17:00.281 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:16:59 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:16:59.960+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.649370+0000 front 2024-09-19T05:16:35.649302+0000 (oldest deadline 2024-09-19T05:16:59.749160+0000) 2024-09-19T05:17:00.475 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:00 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:00.137+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:27.546438+0000 front 2024-09-19T05:16:27.546400+0000 (oldest deadline 2024-09-19T05:16:52.846815+0000) 2024-09-19T05:17:00.475 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:00 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:00.137+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:38.048448+0000 front 2024-09-19T05:16:38.048358+0000 (oldest deadline 2024-09-19T05:16:59.148332+0000) 2024-09-19T05:17:00.555 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:17:00 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5[46846]: 2024-09-19T05:17:00.220+0000 7fa5ccbf1640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.779197+0000 front 2024-09-19T05:16:20.779097+0000 (oldest deadline 2024-09-19T05:16:44.878870+0000) 2024-09-19T05:17:00.555 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:17:00 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5[46846]: 2024-09-19T05:17:00.220+0000 7fa5ccbf1640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.480388+0000 front 2024-09-19T05:16:29.480333+0000 (oldest deadline 2024-09-19T05:16:51.780349+0000) 2024-09-19T05:17:00.555 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:17:00 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5[46846]: 2024-09-19T05:17:00.220+0000 7fa5ccbf1640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.781956+0000 front 2024-09-19T05:16:35.781879+0000 (oldest deadline 2024-09-19T05:16:59.281953+0000) 2024-09-19T05:17:00.555 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:00 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:00.383+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.631981+0000 front 2024-09-19T05:16:29.631823+0000 (oldest deadline 2024-09-19T05:16:54.931686+0000) 2024-09-19T05:17:00.555 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:00 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:00.383+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.232773+0000 front 2024-09-19T05:16:37.232811+0000 (oldest deadline 2024-09-19T05:17:00.132921+0000) 2024-09-19T05:17:00.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:00 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:00.553+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:00.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:00 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:00.553+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:00.956 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:00 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:00.670+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:17:00.956 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:00 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:00.670+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.976294+0000 front 2024-09-19T05:16:28.976202+0000 (oldest deadline 2024-09-19T05:16:54.276323+0000) 2024-09-19T05:17:00.956 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:00 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:00.670+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.678064+0000 front 2024-09-19T05:16:37.678149+0000 (oldest deadline 2024-09-19T05:16:59.278673+0000) 2024-09-19T05:17:01.110 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:00 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:00.769+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:01.110 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:00 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:00.769+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:01.111 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:00 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:00.769+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:01.280 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:00 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:17:00.953+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.347642+0000 front 2024-09-19T05:16:20.347562+0000 (oldest deadline 2024-09-19T05:16:45.047862+0000) 2024-09-19T05:17:01.281 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:00 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:17:00.954+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.749196+0000 front 2024-09-19T05:16:29.749016+0000 (oldest deadline 2024-09-19T05:16:55.648755+0000) 2024-09-19T05:17:01.281 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:00 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:17:00.954+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.649370+0000 front 2024-09-19T05:16:35.649302+0000 (oldest deadline 2024-09-19T05:16:59.749160+0000) 2024-09-19T05:17:01.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:01 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:01.109+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:27.546438+0000 front 2024-09-19T05:16:27.546400+0000 (oldest deadline 2024-09-19T05:16:52.846815+0000) 2024-09-19T05:17:01.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:01 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:01.109+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:38.048448+0000 front 2024-09-19T05:16:38.048358+0000 (oldest deadline 2024-09-19T05:16:59.148332+0000) 2024-09-19T05:17:01.576 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:17:01 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5[46846]: 2024-09-19T05:17:01.173+0000 7fa5ccbf1640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.779197+0000 front 2024-09-19T05:16:20.779097+0000 (oldest deadline 2024-09-19T05:16:44.878870+0000) 2024-09-19T05:17:01.576 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:17:01 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5[46846]: 2024-09-19T05:17:01.173+0000 7fa5ccbf1640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.480388+0000 front 2024-09-19T05:16:29.480333+0000 (oldest deadline 2024-09-19T05:16:51.780349+0000) 2024-09-19T05:17:01.576 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:17:01 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5[46846]: 2024-09-19T05:17:01.173+0000 7fa5ccbf1640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.781956+0000 front 2024-09-19T05:16:35.781879+0000 (oldest deadline 2024-09-19T05:16:59.281953+0000) 2024-09-19T05:17:01.576 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:01 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:01.422+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.631981+0000 front 2024-09-19T05:16:29.631823+0000 (oldest deadline 2024-09-19T05:16:54.931686+0000) 2024-09-19T05:17:01.576 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:01 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:01.422+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.232773+0000 front 2024-09-19T05:16:37.232811+0000 (oldest deadline 2024-09-19T05:17:00.132921+0000) 2024-09-19T05:17:01.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:01 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:01.574+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:01.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:01 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:01.574+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:02.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:01 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:01.702+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:17:02.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:01 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:01.702+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.976294+0000 front 2024-09-19T05:16:28.976202+0000 (oldest deadline 2024-09-19T05:16:54.276323+0000) 2024-09-19T05:17:02.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:01 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:01.702+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.678064+0000 front 2024-09-19T05:16:37.678149+0000 (oldest deadline 2024-09-19T05:16:59.278673+0000) 2024-09-19T05:17:02.031 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:01 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:17:01.937+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.347642+0000 front 2024-09-19T05:16:20.347562+0000 (oldest deadline 2024-09-19T05:16:45.047862+0000) 2024-09-19T05:17:02.031 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:01 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:17:01.938+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.749196+0000 front 2024-09-19T05:16:29.749016+0000 (oldest deadline 2024-09-19T05:16:55.648755+0000) 2024-09-19T05:17:02.031 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:01 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:17:01.938+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.649370+0000 front 2024-09-19T05:16:35.649302+0000 (oldest deadline 2024-09-19T05:16:59.749160+0000) 2024-09-19T05:17:02.105 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:01 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:01.745+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:02.105 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:01 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:01.745+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:02.105 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:01 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:01.745+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:02.405 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:17:02 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5[46846]: 2024-09-19T05:17:02.145+0000 7fa5ccbf1640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.779197+0000 front 2024-09-19T05:16:20.779097+0000 (oldest deadline 2024-09-19T05:16:44.878870+0000) 2024-09-19T05:17:02.405 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:17:02 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5[46846]: 2024-09-19T05:17:02.145+0000 7fa5ccbf1640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.480388+0000 front 2024-09-19T05:16:29.480333+0000 (oldest deadline 2024-09-19T05:16:51.780349+0000) 2024-09-19T05:17:02.405 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:17:02 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5[46846]: 2024-09-19T05:17:02.145+0000 7fa5ccbf1640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.781956+0000 front 2024-09-19T05:16:35.781879+0000 (oldest deadline 2024-09-19T05:16:59.281953+0000) 2024-09-19T05:17:02.406 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:02 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:02.384+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.631981+0000 front 2024-09-19T05:16:29.631823+0000 (oldest deadline 2024-09-19T05:16:54.931686+0000) 2024-09-19T05:17:02.406 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:02 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:02.384+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.232773+0000 front 2024-09-19T05:16:37.232811+0000 (oldest deadline 2024-09-19T05:17:00.132921+0000) 2024-09-19T05:17:02.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:02 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:02.103+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:27.546438+0000 front 2024-09-19T05:16:27.546400+0000 (oldest deadline 2024-09-19T05:16:52.846815+0000) 2024-09-19T05:17:02.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:02 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:02.103+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:38.048448+0000 front 2024-09-19T05:16:38.048358+0000 (oldest deadline 2024-09-19T05:16:59.148332+0000) 2024-09-19T05:17:02.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:02 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:02.576+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:02.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:02 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:02.576+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:02.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:02 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:02.713+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:02.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:02 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:02.713+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:02.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:02 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:02.713+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:03.030 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:02 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:02.726+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:17:03.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:02 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:02.726+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.976294+0000 front 2024-09-19T05:16:28.976202+0000 (oldest deadline 2024-09-19T05:16:54.276323+0000) 2024-09-19T05:17:03.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:02 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:02.726+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.678064+0000 front 2024-09-19T05:16:37.678149+0000 (oldest deadline 2024-09-19T05:16:59.278673+0000) 2024-09-19T05:17:03.031 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:02 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:17:02.948+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.347642+0000 front 2024-09-19T05:16:20.347562+0000 (oldest deadline 2024-09-19T05:16:45.047862+0000) 2024-09-19T05:17:03.031 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:02 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:17:02.948+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.749196+0000 front 2024-09-19T05:16:29.749016+0000 (oldest deadline 2024-09-19T05:16:55.648755+0000) 2024-09-19T05:17:03.031 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:02 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:17:02.948+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.649370+0000 front 2024-09-19T05:16:35.649302+0000 (oldest deadline 2024-09-19T05:16:59.749160+0000) 2024-09-19T05:17:03.372 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:17:03 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5[46846]: 2024-09-19T05:17:03.119+0000 7fa5ccbf1640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.779197+0000 front 2024-09-19T05:16:20.779097+0000 (oldest deadline 2024-09-19T05:16:44.878870+0000) 2024-09-19T05:17:03.372 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:17:03 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5[46846]: 2024-09-19T05:17:03.119+0000 7fa5ccbf1640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.480388+0000 front 2024-09-19T05:16:29.480333+0000 (oldest deadline 2024-09-19T05:16:51.780349+0000) 2024-09-19T05:17:03.372 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:17:03 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5[46846]: 2024-09-19T05:17:03.119+0000 7fa5ccbf1640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.781956+0000 front 2024-09-19T05:16:35.781879+0000 (oldest deadline 2024-09-19T05:16:59.281953+0000) 2024-09-19T05:17:03.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:03 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:03.122+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:27.546438+0000 front 2024-09-19T05:16:27.546400+0000 (oldest deadline 2024-09-19T05:16:52.846815+0000) 2024-09-19T05:17:03.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:03 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:03.122+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:38.048448+0000 front 2024-09-19T05:16:38.048358+0000 (oldest deadline 2024-09-19T05:16:59.148332+0000) 2024-09-19T05:17:03.655 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:03 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:03.546+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:03.655 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:03 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:03.546+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:03.655 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:03 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:03.546+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:03.655 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:03 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:03.370+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.631981+0000 front 2024-09-19T05:16:29.631823+0000 (oldest deadline 2024-09-19T05:16:54.931686+0000) 2024-09-19T05:17:03.655 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:03 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:03.370+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.232773+0000 front 2024-09-19T05:16:37.232811+0000 (oldest deadline 2024-09-19T05:17:00.132921+0000) 2024-09-19T05:17:03.984 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:03 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:03.692+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:17:03.984 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:03 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:03.692+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.976294+0000 front 2024-09-19T05:16:28.976202+0000 (oldest deadline 2024-09-19T05:16:54.276323+0000) 2024-09-19T05:17:03.984 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:03 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:03.692+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.678064+0000 front 2024-09-19T05:16:37.678149+0000 (oldest deadline 2024-09-19T05:16:59.278673+0000) 2024-09-19T05:17:04.093 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:03 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:03.747+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:04.093 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:03 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:03.747+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:04.093 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:03 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:03.747+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:04.281 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:03 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:17:03.982+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.347642+0000 front 2024-09-19T05:16:20.347562+0000 (oldest deadline 2024-09-19T05:16:45.047862+0000) 2024-09-19T05:17:04.281 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:03 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:17:03.982+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.749196+0000 front 2024-09-19T05:16:29.749016+0000 (oldest deadline 2024-09-19T05:16:55.648755+0000) 2024-09-19T05:17:04.281 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:03 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:17:03.982+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.649370+0000 front 2024-09-19T05:16:35.649302+0000 (oldest deadline 2024-09-19T05:16:59.749160+0000) 2024-09-19T05:17:04.405 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:17:04 smithi151 podman[71864]: 2024-09-19 05:17:04.123628957 +0000 UTC m=+5.590183753 container died a48b682ae76f12b27c7206f03f6a0d0f2398d3492828126d7187edd876b300a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-19T05:17:04.405 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:04 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:04.344+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.631981+0000 front 2024-09-19T05:16:29.631823+0000 (oldest deadline 2024-09-19T05:16:54.931686+0000) 2024-09-19T05:17:04.405 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:04 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:04.344+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.232773+0000 front 2024-09-19T05:16:37.232811+0000 (oldest deadline 2024-09-19T05:17:00.132921+0000) 2024-09-19T05:17:04.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:04 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:04.092+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:27.546438+0000 front 2024-09-19T05:16:27.546400+0000 (oldest deadline 2024-09-19T05:16:52.846815+0000) 2024-09-19T05:17:04.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:04 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:04.092+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:38.048448+0000 front 2024-09-19T05:16:38.048358+0000 (oldest deadline 2024-09-19T05:16:59.148332+0000) 2024-09-19T05:17:04.786 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:17:04 smithi151 podman[71864]: 2024-09-19 05:17:04.516848065 +0000 UTC m=+5.983402861 container cleanup a48b682ae76f12b27c7206f03f6a0d0f2398d3492828126d7187edd876b300a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-19T05:17:04.786 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:17:04 smithi151 bash[71864]: ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5 2024-09-19T05:17:04.786 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:04 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:04.544+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:04.786 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:04 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:04.544+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:04.786 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:04 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:04.544+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:05.014 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:04 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:04.703+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:17:05.014 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:04 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:04.703+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.976294+0000 front 2024-09-19T05:16:28.976202+0000 (oldest deadline 2024-09-19T05:16:54.276323+0000) 2024-09-19T05:17:05.014 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:04 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:04.703+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.678064+0000 front 2024-09-19T05:16:37.678149+0000 (oldest deadline 2024-09-19T05:16:59.278673+0000) 2024-09-19T05:17:05.091 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:17:04 smithi151 podman[71879]: 2024-09-19 05:17:04.785153697 +0000 UTC m=+0.655377048 container remove a48b682ae76f12b27c7206f03f6a0d0f2398d3492828126d7187edd876b300a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-19T05:17:05.135 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:04 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:04.764+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:05.135 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:04 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:04.764+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:05.135 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:04 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:04.764+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:05.281 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:05 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:17:05.012+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.347642+0000 front 2024-09-19T05:16:20.347562+0000 (oldest deadline 2024-09-19T05:16:45.047862+0000) 2024-09-19T05:17:05.281 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:05 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:17:05.012+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.749196+0000 front 2024-09-19T05:16:29.749016+0000 (oldest deadline 2024-09-19T05:16:55.648755+0000) 2024-09-19T05:17:05.281 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:05 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:17:05.012+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.649370+0000 front 2024-09-19T05:16:35.649302+0000 (oldest deadline 2024-09-19T05:16:59.749160+0000) 2024-09-19T05:17:05.382 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:17:05 smithi151 podman[71967]: 2024-09-19 05:17:04.994253285 +0000 UTC m=+0.015911498 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:17:05.382 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:17:05 smithi151 podman[71967]: 2024-09-19 05:17:05.103016638 +0000 UTC m=+0.124674856 container create 9f06e31a497ae789ab6c49fceac8c15b4b1efa89892ad94ec05496d0a64884aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-19T05:17:05.382 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:05 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:05.332+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.631981+0000 front 2024-09-19T05:16:29.631823+0000 (oldest deadline 2024-09-19T05:16:54.931686+0000) 2024-09-19T05:17:05.383 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:05 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:05.332+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.232773+0000 front 2024-09-19T05:16:37.232811+0000 (oldest deadline 2024-09-19T05:17:00.132921+0000) 2024-09-19T05:17:05.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:05 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:05.133+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:27.546438+0000 front 2024-09-19T05:16:27.546400+0000 (oldest deadline 2024-09-19T05:16:52.846815+0000) 2024-09-19T05:17:05.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:05 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:05.133+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:38.048448+0000 front 2024-09-19T05:16:38.048358+0000 (oldest deadline 2024-09-19T05:16:59.148332+0000) 2024-09-19T05:17:05.655 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:05 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:05.587+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:05.655 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:05 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:05.587+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:05.655 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:05 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:05.587+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:05.655 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:17:05 smithi151 podman[71967]: 2024-09-19 05:17:05.381420646 +0000 UTC m=+0.403078859 container init 9f06e31a497ae789ab6c49fceac8c15b4b1efa89892ad94ec05496d0a64884aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True) 2024-09-19T05:17:05.655 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:17:05 smithi151 podman[71967]: 2024-09-19 05:17:05.384822562 +0000 UTC m=+0.406480782 container start 9f06e31a497ae789ab6c49fceac8c15b4b1efa89892ad94ec05496d0a64884aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-19T05:17:05.656 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:17:05 smithi151 podman[71967]: 2024-09-19 05:17:05.489862041 +0000 UTC m=+0.511520255 container attach 9f06e31a497ae789ab6c49fceac8c15b4b1efa89892ad94ec05496d0a64884aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-19T05:17:05.656 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 05:17:05 smithi151 podman[71967]: 2024-09-19 05:17:05.570154504 +0000 UTC m=+0.591812725 container died 9f06e31a497ae789ab6c49fceac8c15b4b1efa89892ad94ec05496d0a64884aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD) 2024-09-19T05:17:06.030 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:05 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:05.746+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:17:06.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:05 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:05.746+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.976294+0000 front 2024-09-19T05:16:28.976202+0000 (oldest deadline 2024-09-19T05:16:54.276323+0000) 2024-09-19T05:17:06.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:05 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:05.746+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.678064+0000 front 2024-09-19T05:16:37.678149+0000 (oldest deadline 2024-09-19T05:16:59.278673+0000) 2024-09-19T05:17:06.031 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:05 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:17:05.986+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.347642+0000 front 2024-09-19T05:16:20.347562+0000 (oldest deadline 2024-09-19T05:16:45.047862+0000) 2024-09-19T05:17:06.031 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:05 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:17:05.986+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.749196+0000 front 2024-09-19T05:16:29.749016+0000 (oldest deadline 2024-09-19T05:16:55.648755+0000) 2024-09-19T05:17:06.031 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:05 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:17:05.986+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.649370+0000 front 2024-09-19T05:16:35.649302+0000 (oldest deadline 2024-09-19T05:16:59.749160+0000) 2024-09-19T05:17:06.109 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:05 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:05.790+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:06.109 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:05 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:05.790+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:06.109 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:05 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:05.790+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:06.281 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@osd.5.service' 2024-09-19T05:17:06.321 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:17:06.321 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-19T05:17:06.321 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-19T05:17:06.321 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-1456a810-7643-11ef-bceb-c7b262605968@osd.6 2024-09-19T05:17:06.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:06 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:06.107+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:27.546438+0000 front 2024-09-19T05:16:27.546400+0000 (oldest deadline 2024-09-19T05:16:52.846815+0000) 2024-09-19T05:17:06.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:06 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:06.107+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:38.048448+0000 front 2024-09-19T05:16:38.048358+0000 (oldest deadline 2024-09-19T05:16:59.148332+0000) 2024-09-19T05:17:06.624 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:06 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:06.286+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.631981+0000 front 2024-09-19T05:16:29.631823+0000 (oldest deadline 2024-09-19T05:16:54.931686+0000) 2024-09-19T05:17:06.624 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:06 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:06.286+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.232773+0000 front 2024-09-19T05:16:37.232811+0000 (oldest deadline 2024-09-19T05:17:00.132921+0000) 2024-09-19T05:17:06.720 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:06 smithi043 systemd[1]: Stopping Ceph osd.6 for 1456a810-7643-11ef-bceb-c7b262605968... 2024-09-19T05:17:06.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:06 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:06.622+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:06.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:06 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:06.622+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:06.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:06 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:06.622+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:06.971 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:06 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:17:06.763+0000 7fdd3b200640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:17:06.971 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:06 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:17:06.763+0000 7fdd3b200640 -1 osd.6 99 *** Got signal Terminated *** 2024-09-19T05:17:06.971 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:06 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:17:06.763+0000 7fdd3b200640 -1 osd.6 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T05:17:06.971 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:06 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:17:06.968+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.347642+0000 front 2024-09-19T05:16:20.347562+0000 (oldest deadline 2024-09-19T05:16:45.047862+0000) 2024-09-19T05:17:06.972 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:06 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:06.718+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:17:06.972 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:06 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:06.718+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.976294+0000 front 2024-09-19T05:16:28.976202+0000 (oldest deadline 2024-09-19T05:16:54.276323+0000) 2024-09-19T05:17:06.972 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:06 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:06.718+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.678064+0000 front 2024-09-19T05:16:37.678149+0000 (oldest deadline 2024-09-19T05:16:59.278673+0000) 2024-09-19T05:17:07.089 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:06 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:06.744+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:07.089 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:06 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:06.744+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:07.089 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:06 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:06.744+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:07.281 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:06 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:17:06.968+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.749196+0000 front 2024-09-19T05:16:29.749016+0000 (oldest deadline 2024-09-19T05:16:55.648755+0000) 2024-09-19T05:17:07.281 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:06 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:17:06.968+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.649370+0000 front 2024-09-19T05:16:35.649302+0000 (oldest deadline 2024-09-19T05:16:59.749160+0000) 2024-09-19T05:17:07.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:07 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:07.087+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:27.546438+0000 front 2024-09-19T05:16:27.546400+0000 (oldest deadline 2024-09-19T05:16:52.846815+0000) 2024-09-19T05:17:07.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:07 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:07.087+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:38.048448+0000 front 2024-09-19T05:16:38.048358+0000 (oldest deadline 2024-09-19T05:16:59.148332+0000) 2024-09-19T05:17:07.619 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:07 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:07.250+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.631981+0000 front 2024-09-19T05:16:29.631823+0000 (oldest deadline 2024-09-19T05:16:54.931686+0000) 2024-09-19T05:17:07.619 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:07 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:07.250+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.232773+0000 front 2024-09-19T05:16:37.232811+0000 (oldest deadline 2024-09-19T05:17:00.132921+0000) 2024-09-19T05:17:07.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:07 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:07.618+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:07.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:07 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:07.618+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:07.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:07 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:07.618+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:07.963 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:07 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:07.690+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:17:07.963 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:07 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:07.690+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.976294+0000 front 2024-09-19T05:16:28.976202+0000 (oldest deadline 2024-09-19T05:16:54.276323+0000) 2024-09-19T05:17:07.964 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:07 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:07.690+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.678064+0000 front 2024-09-19T05:16:37.678149+0000 (oldest deadline 2024-09-19T05:16:59.278673+0000) 2024-09-19T05:17:07.964 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:07 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:07.690+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:45.680188+0000 front 2024-09-19T05:16:45.680274+0000 (oldest deadline 2024-09-19T05:17:07.380336+0000) 2024-09-19T05:17:07.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:07 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:07.698+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:07.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:07 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:07.698+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:07.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:07 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:07.698+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:08.281 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:07 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:17:07.961+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.347642+0000 front 2024-09-19T05:16:20.347562+0000 (oldest deadline 2024-09-19T05:16:45.047862+0000) 2024-09-19T05:17:08.281 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:07 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:17:07.961+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.749196+0000 front 2024-09-19T05:16:29.749016+0000 (oldest deadline 2024-09-19T05:16:55.648755+0000) 2024-09-19T05:17:08.281 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:07 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:17:07.961+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.649370+0000 front 2024-09-19T05:16:35.649302+0000 (oldest deadline 2024-09-19T05:16:59.749160+0000) 2024-09-19T05:17:08.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:08 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:08.106+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:27.546438+0000 front 2024-09-19T05:16:27.546400+0000 (oldest deadline 2024-09-19T05:16:52.846815+0000) 2024-09-19T05:17:08.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:08 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:08.106+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:38.048448+0000 front 2024-09-19T05:16:38.048358+0000 (oldest deadline 2024-09-19T05:16:59.148332+0000) 2024-09-19T05:17:08.654 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:08 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:08.244+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.631981+0000 front 2024-09-19T05:16:29.631823+0000 (oldest deadline 2024-09-19T05:16:54.931686+0000) 2024-09-19T05:17:08.654 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:08 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:08.244+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.232773+0000 front 2024-09-19T05:16:37.232811+0000 (oldest deadline 2024-09-19T05:17:00.132921+0000) 2024-09-19T05:17:08.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:08 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:08.652+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:08.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:08 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:08.652+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:08.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:08 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:08.652+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:08.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:08 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:08.662+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:08.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:08 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:08.662+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:08.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:08 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:08.662+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:08.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:08 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:08.662+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:43.370861+0000 front 2024-09-19T05:16:43.370887+0000 (oldest deadline 2024-09-19T05:17:08.070492+0000) 2024-09-19T05:17:08.994 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:08 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:08.706+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:17:08.994 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:08 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:08.706+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.976294+0000 front 2024-09-19T05:16:28.976202+0000 (oldest deadline 2024-09-19T05:16:54.276323+0000) 2024-09-19T05:17:08.994 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:08 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:08.706+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.678064+0000 front 2024-09-19T05:16:37.678149+0000 (oldest deadline 2024-09-19T05:16:59.278673+0000) 2024-09-19T05:17:08.995 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:08 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:08.706+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:45.680188+0000 front 2024-09-19T05:16:45.680274+0000 (oldest deadline 2024-09-19T05:17:07.380336+0000) 2024-09-19T05:17:09.281 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:08 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:17:08.992+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.347642+0000 front 2024-09-19T05:16:20.347562+0000 (oldest deadline 2024-09-19T05:16:45.047862+0000) 2024-09-19T05:17:09.281 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:08 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:17:08.992+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.749196+0000 front 2024-09-19T05:16:29.749016+0000 (oldest deadline 2024-09-19T05:16:55.648755+0000) 2024-09-19T05:17:09.281 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:08 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:17:08.992+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.649370+0000 front 2024-09-19T05:16:35.649302+0000 (oldest deadline 2024-09-19T05:16:59.749160+0000) 2024-09-19T05:17:09.281 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:08 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:17:08.992+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:44.851122+0000 front 2024-09-19T05:16:44.851102+0000 (oldest deadline 2024-09-19T05:17:08.152032+0000) 2024-09-19T05:17:09.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:09 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:09.086+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:27.546438+0000 front 2024-09-19T05:16:27.546400+0000 (oldest deadline 2024-09-19T05:16:52.846815+0000) 2024-09-19T05:17:09.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:09 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:09.086+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:38.048448+0000 front 2024-09-19T05:16:38.048358+0000 (oldest deadline 2024-09-19T05:16:59.148332+0000) 2024-09-19T05:17:09.655 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:09 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:09.288+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.631981+0000 front 2024-09-19T05:16:29.631823+0000 (oldest deadline 2024-09-19T05:16:54.931686+0000) 2024-09-19T05:17:09.655 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:09 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:09.288+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.232773+0000 front 2024-09-19T05:16:37.232811+0000 (oldest deadline 2024-09-19T05:17:00.132921+0000) 2024-09-19T05:17:09.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:09 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:09.615+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:09.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:09 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:09.615+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:09.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:09 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:09.615+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:09.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:09 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:09.615+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:43.370861+0000 front 2024-09-19T05:16:43.370887+0000 (oldest deadline 2024-09-19T05:17:08.070492+0000) 2024-09-19T05:17:10.031 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:09 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:17:09.945+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.347642+0000 front 2024-09-19T05:16:20.347562+0000 (oldest deadline 2024-09-19T05:16:45.047862+0000) 2024-09-19T05:17:10.031 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:09 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:17:09.945+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.749196+0000 front 2024-09-19T05:16:29.749016+0000 (oldest deadline 2024-09-19T05:16:55.648755+0000) 2024-09-19T05:17:10.031 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:09 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:17:09.945+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.649370+0000 front 2024-09-19T05:16:35.649302+0000 (oldest deadline 2024-09-19T05:16:59.749160+0000) 2024-09-19T05:17:10.031 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:09 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:17:09.945+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:44.851122+0000 front 2024-09-19T05:16:44.851102+0000 (oldest deadline 2024-09-19T05:17:08.152032+0000) 2024-09-19T05:17:10.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:09 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:09.698+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:17:10.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:09 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:09.698+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.976294+0000 front 2024-09-19T05:16:28.976202+0000 (oldest deadline 2024-09-19T05:16:54.276323+0000) 2024-09-19T05:17:10.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:09 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:09.698+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.678064+0000 front 2024-09-19T05:16:37.678149+0000 (oldest deadline 2024-09-19T05:16:59.278673+0000) 2024-09-19T05:17:10.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:09 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:09.698+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:45.680188+0000 front 2024-09-19T05:16:45.680274+0000 (oldest deadline 2024-09-19T05:17:07.380336+0000) 2024-09-19T05:17:10.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:09 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:09.663+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:10.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:09 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:09.663+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:10.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:09 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:09.663+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:10.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:10 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:10.136+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:27.546438+0000 front 2024-09-19T05:16:27.546400+0000 (oldest deadline 2024-09-19T05:16:52.846815+0000) 2024-09-19T05:17:10.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:10 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:10.136+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:38.048448+0000 front 2024-09-19T05:16:38.048358+0000 (oldest deadline 2024-09-19T05:16:59.148332+0000) 2024-09-19T05:17:10.627 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:10 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:10.271+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.631981+0000 front 2024-09-19T05:16:29.631823+0000 (oldest deadline 2024-09-19T05:16:54.931686+0000) 2024-09-19T05:17:10.860 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:10 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:10.271+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.232773+0000 front 2024-09-19T05:16:37.232811+0000 (oldest deadline 2024-09-19T05:17:00.132921+0000) 2024-09-19T05:17:10.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:10 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:10.626+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:10.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:10 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:10.626+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:10.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:10 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:10.626+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:10.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:10 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:10.618+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:10.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:10 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:10.618+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:10.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:10 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:10.618+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:10.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:10 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:10.618+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:43.370861+0000 front 2024-09-19T05:16:43.370887+0000 (oldest deadline 2024-09-19T05:17:08.070492+0000) 2024-09-19T05:17:11.031 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:10 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:17:10.937+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:20.347642+0000 front 2024-09-19T05:16:20.347562+0000 (oldest deadline 2024-09-19T05:16:45.047862+0000) 2024-09-19T05:17:11.031 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:10 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:17:10.937+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.749196+0000 front 2024-09-19T05:16:29.749016+0000 (oldest deadline 2024-09-19T05:16:55.648755+0000) 2024-09-19T05:17:11.031 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:10 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:17:10.937+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.649370+0000 front 2024-09-19T05:16:35.649302+0000 (oldest deadline 2024-09-19T05:16:59.749160+0000) 2024-09-19T05:17:11.031 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:10 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6[55774]: 2024-09-19T05:17:10.937+0000 7fdd36c00640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:44.851122+0000 front 2024-09-19T05:16:44.851102+0000 (oldest deadline 2024-09-19T05:17:08.152032+0000) 2024-09-19T05:17:11.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:10 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:10.698+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:17:11.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:10 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:10.698+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.976294+0000 front 2024-09-19T05:16:28.976202+0000 (oldest deadline 2024-09-19T05:16:54.276323+0000) 2024-09-19T05:17:11.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:10 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:10.698+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.678064+0000 front 2024-09-19T05:16:37.678149+0000 (oldest deadline 2024-09-19T05:16:59.278673+0000) 2024-09-19T05:17:11.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:10 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:10.698+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:45.680188+0000 front 2024-09-19T05:16:45.680274+0000 (oldest deadline 2024-09-19T05:17:07.380336+0000) 2024-09-19T05:17:11.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:11 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:11.148+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:27.546438+0000 front 2024-09-19T05:16:27.546400+0000 (oldest deadline 2024-09-19T05:16:52.846815+0000) 2024-09-19T05:17:11.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:11 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:11.148+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:38.048448+0000 front 2024-09-19T05:16:38.048358+0000 (oldest deadline 2024-09-19T05:16:59.148332+0000) 2024-09-19T05:17:11.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:11 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:11.148+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:46.050679+0000 front 2024-09-19T05:16:46.050775+0000 (oldest deadline 2024-09-19T05:17:10.150418+0000) 2024-09-19T05:17:11.592 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:11 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:11.295+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.631981+0000 front 2024-09-19T05:16:29.631823+0000 (oldest deadline 2024-09-19T05:16:54.931686+0000) 2024-09-19T05:17:11.592 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:11 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:11.295+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.232773+0000 front 2024-09-19T05:16:37.232811+0000 (oldest deadline 2024-09-19T05:17:00.132921+0000) 2024-09-19T05:17:11.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:11 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:11.591+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:11.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:11 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:11.591+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:11.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:11 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:11.591+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:11.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:11 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:11.662+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:11.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:11 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:11.662+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:11.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:11 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:11.662+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:11.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:11 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:11.662+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:43.370861+0000 front 2024-09-19T05:16:43.370887+0000 (oldest deadline 2024-09-19T05:17:08.070492+0000) 2024-09-19T05:17:12.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:11 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:11.652+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:17:12.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:11 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:11.652+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.976294+0000 front 2024-09-19T05:16:28.976202+0000 (oldest deadline 2024-09-19T05:16:54.276323+0000) 2024-09-19T05:17:12.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:11 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:11.652+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.678064+0000 front 2024-09-19T05:16:37.678149+0000 (oldest deadline 2024-09-19T05:16:59.278673+0000) 2024-09-19T05:17:12.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:11 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:11.652+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:45.680188+0000 front 2024-09-19T05:16:45.680274+0000 (oldest deadline 2024-09-19T05:17:07.380336+0000) 2024-09-19T05:17:12.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:12 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:12.123+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:27.546438+0000 front 2024-09-19T05:16:27.546400+0000 (oldest deadline 2024-09-19T05:16:52.846815+0000) 2024-09-19T05:17:12.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:12 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:12.123+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:38.048448+0000 front 2024-09-19T05:16:38.048358+0000 (oldest deadline 2024-09-19T05:16:59.148332+0000) 2024-09-19T05:17:12.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:12 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:12.123+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:46.050679+0000 front 2024-09-19T05:16:46.050775+0000 (oldest deadline 2024-09-19T05:17:10.150418+0000) 2024-09-19T05:17:12.522 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:12 smithi043 podman[108682]: 2024-09-19 05:17:12.211559748 +0000 UTC m=+5.601972606 container died 1abd2be77e5b0704cfd2f495a0d9f3f9ef8e559b2d2211157d9541b89b9129a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-19T05:17:12.583 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:12 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:12.291+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.631981+0000 front 2024-09-19T05:16:29.631823+0000 (oldest deadline 2024-09-19T05:16:54.931686+0000) 2024-09-19T05:17:12.583 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:12 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:12.291+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.232773+0000 front 2024-09-19T05:16:37.232811+0000 (oldest deadline 2024-09-19T05:17:00.132921+0000) 2024-09-19T05:17:12.780 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:12 smithi043 podman[108682]: 2024-09-19 05:17:12.753031267 +0000 UTC m=+6.143444122 container cleanup 1abd2be77e5b0704cfd2f495a0d9f3f9ef8e559b2d2211157d9541b89b9129a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-19T05:17:12.781 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:12 smithi043 bash[108682]: ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6 2024-09-19T05:17:12.781 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:12 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:12.696+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:17:12.781 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:12 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:12.696+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.976294+0000 front 2024-09-19T05:16:28.976202+0000 (oldest deadline 2024-09-19T05:16:54.276323+0000) 2024-09-19T05:17:12.781 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:12 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:12.696+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.678064+0000 front 2024-09-19T05:16:37.678149+0000 (oldest deadline 2024-09-19T05:16:59.278673+0000) 2024-09-19T05:17:12.781 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:12 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:12.696+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:45.680188+0000 front 2024-09-19T05:16:45.680274+0000 (oldest deadline 2024-09-19T05:17:07.380336+0000) 2024-09-19T05:17:12.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:12 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:12.582+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:12.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:12 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:12.582+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:12.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:12 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:12.582+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:12.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:12 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:12.703+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:12.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:12 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:12.703+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:12.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:12 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:12.703+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:12.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:12 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:12.703+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:43.370861+0000 front 2024-09-19T05:16:43.370887+0000 (oldest deadline 2024-09-19T05:17:08.070492+0000) 2024-09-19T05:17:13.280 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:13 smithi043 podman[108702]: 2024-09-19 05:17:13.012996922 +0000 UTC m=+0.787165463 container remove 1abd2be77e5b0704cfd2f495a0d9f3f9ef8e559b2d2211157d9541b89b9129a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-19T05:17:13.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:13 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:13.083+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:27.546438+0000 front 2024-09-19T05:16:27.546400+0000 (oldest deadline 2024-09-19T05:16:52.846815+0000) 2024-09-19T05:17:13.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:13 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:13.083+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:38.048448+0000 front 2024-09-19T05:16:38.048358+0000 (oldest deadline 2024-09-19T05:16:59.148332+0000) 2024-09-19T05:17:13.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:13 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:13.083+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:46.050679+0000 front 2024-09-19T05:16:46.050775+0000 (oldest deadline 2024-09-19T05:17:10.150418+0000) 2024-09-19T05:17:13.655 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:13 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:13.322+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.631981+0000 front 2024-09-19T05:16:29.631823+0000 (oldest deadline 2024-09-19T05:16:54.931686+0000) 2024-09-19T05:17:13.656 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:13 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:13.322+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.232773+0000 front 2024-09-19T05:16:37.232811+0000 (oldest deadline 2024-09-19T05:17:00.132921+0000) 2024-09-19T05:17:13.656 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:13 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:13.322+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:46.534659+0000 front 2024-09-19T05:16:46.534699+0000 (oldest deadline 2024-09-19T05:17:12.434425+0000) 2024-09-19T05:17:13.656 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:13 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:13.563+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:13.656 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:13 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:13.563+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:13.656 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:13 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:13.563+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:13.867 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:13 smithi043 podman[108784]: 2024-09-19 05:17:13.478211412 +0000 UTC m=+0.027532801 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:17:13.868 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:13 smithi043 podman[108784]: 2024-09-19 05:17:13.580689896 +0000 UTC m=+0.130011325 container create 0aaddcdc2073232cd2ea402c7197ed8284e04cad5442e64820c6c478c9069969 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6-deactivate, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-19T05:17:13.868 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:13 smithi043 podman[108784]: 2024-09-19 05:17:13.866640294 +0000 UTC m=+0.415961684 container init 0aaddcdc2073232cd2ea402c7197ed8284e04cad5442e64820c6c478c9069969 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:17:13.868 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:13 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:13.698+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:17:13.868 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:13 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:13.698+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.976294+0000 front 2024-09-19T05:16:28.976202+0000 (oldest deadline 2024-09-19T05:16:54.276323+0000) 2024-09-19T05:17:13.868 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:13 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:13.698+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.678064+0000 front 2024-09-19T05:16:37.678149+0000 (oldest deadline 2024-09-19T05:16:59.278673+0000) 2024-09-19T05:17:13.868 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:13 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:13.698+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:45.680188+0000 front 2024-09-19T05:16:45.680274+0000 (oldest deadline 2024-09-19T05:17:07.380336+0000) 2024-09-19T05:17:13.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:13 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:13.703+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:13.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:13 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:13.703+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:13.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:13 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:13.703+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:13.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:13 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:13.703+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:43.370861+0000 front 2024-09-19T05:16:43.370887+0000 (oldest deadline 2024-09-19T05:17:08.070492+0000) 2024-09-19T05:17:14.145 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:13 smithi043 podman[108784]: 2024-09-19 05:17:13.873792869 +0000 UTC m=+0.423114258 container start 0aaddcdc2073232cd2ea402c7197ed8284e04cad5442e64820c6c478c9069969 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True) 2024-09-19T05:17:14.145 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:13 smithi043 podman[108784]: 2024-09-19 05:17:13.966508533 +0000 UTC m=+0.515829937 container attach 0aaddcdc2073232cd2ea402c7197ed8284e04cad5442e64820c6c478c9069969 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-19T05:17:14.420 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:14 smithi043 podman[108784]: 2024-09-19 05:17:14.144311261 +0000 UTC m=+0.693632660 container died 0aaddcdc2073232cd2ea402c7197ed8284e04cad5442e64820c6c478c9069969 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6-deactivate, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T05:17:14.474 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:14 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:14.096+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:27.546438+0000 front 2024-09-19T05:16:27.546400+0000 (oldest deadline 2024-09-19T05:16:52.846815+0000) 2024-09-19T05:17:14.474 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:14 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:14.096+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:38.048448+0000 front 2024-09-19T05:16:38.048358+0000 (oldest deadline 2024-09-19T05:16:59.148332+0000) 2024-09-19T05:17:14.474 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:14 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:14.096+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:46.050679+0000 front 2024-09-19T05:16:46.050775+0000 (oldest deadline 2024-09-19T05:17:10.150418+0000) 2024-09-19T05:17:14.581 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:14 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:14.278+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.631981+0000 front 2024-09-19T05:16:29.631823+0000 (oldest deadline 2024-09-19T05:16:54.931686+0000) 2024-09-19T05:17:14.581 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:14 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:14.278+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.232773+0000 front 2024-09-19T05:16:37.232811+0000 (oldest deadline 2024-09-19T05:17:00.132921+0000) 2024-09-19T05:17:14.581 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:14 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:14.278+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:46.534659+0000 front 2024-09-19T05:16:46.534699+0000 (oldest deadline 2024-09-19T05:17:12.434425+0000) 2024-09-19T05:17:14.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:14 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:14.579+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:14.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:14 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:14.579+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:14.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:14 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:14.579+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:14.907 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@osd.6.service' 2024-09-19T05:17:14.957 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:14 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:14.689+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:17:14.958 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:14 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:14.689+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.976294+0000 front 2024-09-19T05:16:28.976202+0000 (oldest deadline 2024-09-19T05:16:54.276323+0000) 2024-09-19T05:17:14.958 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:14 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:14.689+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.678064+0000 front 2024-09-19T05:16:37.678149+0000 (oldest deadline 2024-09-19T05:16:59.278673+0000) 2024-09-19T05:17:14.958 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:14 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:14.689+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:45.680188+0000 front 2024-09-19T05:16:45.680274+0000 (oldest deadline 2024-09-19T05:17:07.380336+0000) 2024-09-19T05:17:14.958 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:14 smithi043 podman[108806]: 2024-09-19 05:17:14.879039998 +0000 UTC m=+0.720799754 container remove 0aaddcdc2073232cd2ea402c7197ed8284e04cad5442e64820c6c478c9069969 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-19T05:17:14.958 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:14 smithi043 systemd[1]: ceph-1456a810-7643-11ef-bceb-c7b262605968@osd.6.service: Deactivated successfully. 2024-09-19T05:17:14.958 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:14 smithi043 systemd[1]: Stopped Ceph osd.6 for 1456a810-7643-11ef-bceb-c7b262605968. 2024-09-19T05:17:14.958 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 19 05:17:14 smithi043 systemd[1]: ceph-1456a810-7643-11ef-bceb-c7b262605968@osd.6.service: Consumed 57.091s CPU time. 2024-09-19T05:17:14.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:14 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:14.678+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:14.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:14 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:14.678+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:14.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:14 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:14.678+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:14.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:14 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:14.678+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:43.370861+0000 front 2024-09-19T05:16:43.370887+0000 (oldest deadline 2024-09-19T05:17:08.070492+0000) 2024-09-19T05:17:14.983 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:17:14.984 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-19T05:17:14.984 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-19T05:17:14.984 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-1456a810-7643-11ef-bceb-c7b262605968@osd.7 2024-09-19T05:17:15.328 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:15 smithi096 systemd[1]: Stopping Ceph osd.7 for 1456a810-7643-11ef-bceb-c7b262605968... 2024-09-19T05:17:15.328 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:15 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:15.101+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:27.546438+0000 front 2024-09-19T05:16:27.546400+0000 (oldest deadline 2024-09-19T05:16:52.846815+0000) 2024-09-19T05:17:15.328 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:15 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:15.101+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:38.048448+0000 front 2024-09-19T05:16:38.048358+0000 (oldest deadline 2024-09-19T05:16:59.148332+0000) 2024-09-19T05:17:15.328 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:15 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:15.101+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:46.050679+0000 front 2024-09-19T05:16:46.050775+0000 (oldest deadline 2024-09-19T05:17:10.150418+0000) 2024-09-19T05:17:15.329 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:15 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:15.326+0000 7fae1ec2e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:17:15.329 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:15 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:15.326+0000 7fae1ec2e640 -1 osd.7 99 *** Got signal Terminated *** 2024-09-19T05:17:15.329 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:15 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:15.326+0000 7fae1ec2e640 -1 osd.7 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T05:17:15.627 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:15 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:15.304+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.631981+0000 front 2024-09-19T05:16:29.631823+0000 (oldest deadline 2024-09-19T05:16:54.931686+0000) 2024-09-19T05:17:15.627 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:15 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:15.304+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.232773+0000 front 2024-09-19T05:16:37.232811+0000 (oldest deadline 2024-09-19T05:17:00.132921+0000) 2024-09-19T05:17:15.627 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:15 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:15.304+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:46.534659+0000 front 2024-09-19T05:16:46.534699+0000 (oldest deadline 2024-09-19T05:17:12.434425+0000) 2024-09-19T05:17:15.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:15 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:15.625+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:15.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:15 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:15.625+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:15.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:15 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:15.625+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:15.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:15 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:15.700+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:15.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:15 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:15.700+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:15.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:15 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:15.700+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:15.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:15 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:15.700+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:43.370861+0000 front 2024-09-19T05:16:43.370887+0000 (oldest deadline 2024-09-19T05:17:08.070492+0000) 2024-09-19T05:17:16.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:15 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:15.688+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:17:16.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:15 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:15.688+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.976294+0000 front 2024-09-19T05:16:28.976202+0000 (oldest deadline 2024-09-19T05:16:54.276323+0000) 2024-09-19T05:17:16.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:15 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:15.688+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.678064+0000 front 2024-09-19T05:16:37.678149+0000 (oldest deadline 2024-09-19T05:16:59.278673+0000) 2024-09-19T05:17:16.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:15 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:15.688+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:45.680188+0000 front 2024-09-19T05:16:45.680274+0000 (oldest deadline 2024-09-19T05:17:07.380336+0000) 2024-09-19T05:17:16.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:16 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:16.078+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:27.546438+0000 front 2024-09-19T05:16:27.546400+0000 (oldest deadline 2024-09-19T05:16:52.846815+0000) 2024-09-19T05:17:16.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:16 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:16.078+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:38.048448+0000 front 2024-09-19T05:16:38.048358+0000 (oldest deadline 2024-09-19T05:16:59.148332+0000) 2024-09-19T05:17:16.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:16 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:16.078+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:46.050679+0000 front 2024-09-19T05:16:46.050775+0000 (oldest deadline 2024-09-19T05:17:10.150418+0000) 2024-09-19T05:17:16.620 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:16 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:16.277+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.631981+0000 front 2024-09-19T05:16:29.631823+0000 (oldest deadline 2024-09-19T05:16:54.931686+0000) 2024-09-19T05:17:16.620 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:16 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:16.277+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.232773+0000 front 2024-09-19T05:16:37.232811+0000 (oldest deadline 2024-09-19T05:17:00.132921+0000) 2024-09-19T05:17:16.620 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:16 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:16.277+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:46.534659+0000 front 2024-09-19T05:16:46.534699+0000 (oldest deadline 2024-09-19T05:17:12.434425+0000) 2024-09-19T05:17:16.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:16 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:16.618+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:16.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:16 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:16.618+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:16.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:16 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:16.618+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:16.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:16 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:16.677+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:16.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:16 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:16.677+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:16.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:16 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:16.677+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:16.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:16 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:16.677+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:43.370861+0000 front 2024-09-19T05:16:43.370887+0000 (oldest deadline 2024-09-19T05:17:08.070492+0000) 2024-09-19T05:17:17.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:16 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:16.657+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:17:17.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:16 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:16.657+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.976294+0000 front 2024-09-19T05:16:28.976202+0000 (oldest deadline 2024-09-19T05:16:54.276323+0000) 2024-09-19T05:17:17.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:16 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:16.657+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.678064+0000 front 2024-09-19T05:16:37.678149+0000 (oldest deadline 2024-09-19T05:16:59.278673+0000) 2024-09-19T05:17:17.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:16 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:16.657+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:45.680188+0000 front 2024-09-19T05:16:45.680274+0000 (oldest deadline 2024-09-19T05:17:07.380336+0000) 2024-09-19T05:17:17.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:16 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:16.657+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:53.782102+0000 front 2024-09-19T05:16:53.782082+0000 (oldest deadline 2024-09-19T05:17:16.082229+0000) 2024-09-19T05:17:17.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:17 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:17.051+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:27.546438+0000 front 2024-09-19T05:16:27.546400+0000 (oldest deadline 2024-09-19T05:16:52.846815+0000) 2024-09-19T05:17:17.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:17 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:17.051+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:38.048448+0000 front 2024-09-19T05:16:38.048358+0000 (oldest deadline 2024-09-19T05:16:59.148332+0000) 2024-09-19T05:17:17.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:17 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:17.051+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:46.050679+0000 front 2024-09-19T05:16:46.050775+0000 (oldest deadline 2024-09-19T05:17:10.150418+0000) 2024-09-19T05:17:17.589 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:17 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:17.317+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.631981+0000 front 2024-09-19T05:16:29.631823+0000 (oldest deadline 2024-09-19T05:16:54.931686+0000) 2024-09-19T05:17:17.589 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:17 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:17.317+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.232773+0000 front 2024-09-19T05:16:37.232811+0000 (oldest deadline 2024-09-19T05:17:00.132921+0000) 2024-09-19T05:17:17.589 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:17 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:17.317+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:46.534659+0000 front 2024-09-19T05:16:46.534699+0000 (oldest deadline 2024-09-19T05:17:12.434425+0000) 2024-09-19T05:17:17.589 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:17 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:17.317+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.036000+0000 front 2024-09-19T05:16:54.036137+0000 (oldest deadline 2024-09-19T05:17:16.335906+0000) 2024-09-19T05:17:17.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:17 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:17.587+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:17.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:17 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:17.587+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:17.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:17 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:17.587+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:17.932 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:17 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:17.661+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:17:17.932 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:17 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:17.661+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.976294+0000 front 2024-09-19T05:16:28.976202+0000 (oldest deadline 2024-09-19T05:16:54.276323+0000) 2024-09-19T05:17:17.932 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:17 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:17.661+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.678064+0000 front 2024-09-19T05:16:37.678149+0000 (oldest deadline 2024-09-19T05:16:59.278673+0000) 2024-09-19T05:17:17.932 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:17 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:17.661+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:45.680188+0000 front 2024-09-19T05:16:45.680274+0000 (oldest deadline 2024-09-19T05:17:07.380336+0000) 2024-09-19T05:17:17.933 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:17 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:17.661+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:53.782102+0000 front 2024-09-19T05:16:53.782082+0000 (oldest deadline 2024-09-19T05:17:16.082229+0000) 2024-09-19T05:17:17.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:17 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:17.633+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:17.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:17 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:17.633+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:17.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:17 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:17.633+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:17.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:17 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:17.633+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:43.370861+0000 front 2024-09-19T05:16:43.370887+0000 (oldest deadline 2024-09-19T05:17:08.070492+0000) 2024-09-19T05:17:17.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:17 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:17.633+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:50.971438+0000 front 2024-09-19T05:16:50.971421+0000 (oldest deadline 2024-09-19T05:17:16.871470+0000) 2024-09-19T05:17:18.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:18 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:18.030+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:27.546438+0000 front 2024-09-19T05:16:27.546400+0000 (oldest deadline 2024-09-19T05:16:52.846815+0000) 2024-09-19T05:17:18.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:18 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:18.030+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:38.048448+0000 front 2024-09-19T05:16:38.048358+0000 (oldest deadline 2024-09-19T05:16:59.148332+0000) 2024-09-19T05:17:18.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:18 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:18.030+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:46.050679+0000 front 2024-09-19T05:16:46.050775+0000 (oldest deadline 2024-09-19T05:17:10.150418+0000) 2024-09-19T05:17:18.618 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:18 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:18.345+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.631981+0000 front 2024-09-19T05:16:29.631823+0000 (oldest deadline 2024-09-19T05:16:54.931686+0000) 2024-09-19T05:17:18.618 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:18 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:18.345+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.232773+0000 front 2024-09-19T05:16:37.232811+0000 (oldest deadline 2024-09-19T05:17:00.132921+0000) 2024-09-19T05:17:18.618 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:18 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:18.345+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:46.534659+0000 front 2024-09-19T05:16:46.534699+0000 (oldest deadline 2024-09-19T05:17:12.434425+0000) 2024-09-19T05:17:18.618 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:18 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:18.345+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.036000+0000 front 2024-09-19T05:16:54.036137+0000 (oldest deadline 2024-09-19T05:17:16.335906+0000) 2024-09-19T05:17:18.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:18 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:18.616+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:18.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:18 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:18.616+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:18.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:18 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:18.616+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:18.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:18 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:18.616+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.171184+0000 front 2024-09-19T05:16:54.171329+0000 (oldest deadline 2024-09-19T05:17:18.271276+0000) 2024-09-19T05:17:18.945 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:18 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:18.627+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:17:18.945 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:18 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:18.627+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.976294+0000 front 2024-09-19T05:16:28.976202+0000 (oldest deadline 2024-09-19T05:16:54.276323+0000) 2024-09-19T05:17:18.946 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:18 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:18.627+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.678064+0000 front 2024-09-19T05:16:37.678149+0000 (oldest deadline 2024-09-19T05:16:59.278673+0000) 2024-09-19T05:17:18.946 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:18 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:18.627+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:45.680188+0000 front 2024-09-19T05:16:45.680274+0000 (oldest deadline 2024-09-19T05:17:07.380336+0000) 2024-09-19T05:17:18.946 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:18 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:18.627+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:53.782102+0000 front 2024-09-19T05:16:53.782082+0000 (oldest deadline 2024-09-19T05:17:16.082229+0000) 2024-09-19T05:17:18.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:18 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:18.610+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:18.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:18 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:18.610+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:18.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:18 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:18.610+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:18.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:18 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:18.610+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:43.370861+0000 front 2024-09-19T05:16:43.370887+0000 (oldest deadline 2024-09-19T05:17:08.070492+0000) 2024-09-19T05:17:18.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:18 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:18.610+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:50.971438+0000 front 2024-09-19T05:16:50.971421+0000 (oldest deadline 2024-09-19T05:17:16.871470+0000) 2024-09-19T05:17:19.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:19 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:19.024+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:27.546438+0000 front 2024-09-19T05:16:27.546400+0000 (oldest deadline 2024-09-19T05:16:52.846815+0000) 2024-09-19T05:17:19.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:19 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:19.024+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:38.048448+0000 front 2024-09-19T05:16:38.048358+0000 (oldest deadline 2024-09-19T05:16:59.148332+0000) 2024-09-19T05:17:19.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:19 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:19.024+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:46.050679+0000 front 2024-09-19T05:16:46.050775+0000 (oldest deadline 2024-09-19T05:17:10.150418+0000) 2024-09-19T05:17:19.653 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:19 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:19.359+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.631981+0000 front 2024-09-19T05:16:29.631823+0000 (oldest deadline 2024-09-19T05:16:54.931686+0000) 2024-09-19T05:17:19.653 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:19 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:19.359+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.232773+0000 front 2024-09-19T05:16:37.232811+0000 (oldest deadline 2024-09-19T05:17:00.132921+0000) 2024-09-19T05:17:19.654 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:19 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:19.359+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:46.534659+0000 front 2024-09-19T05:16:46.534699+0000 (oldest deadline 2024-09-19T05:17:12.434425+0000) 2024-09-19T05:17:19.654 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:19 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:19.359+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.036000+0000 front 2024-09-19T05:16:54.036137+0000 (oldest deadline 2024-09-19T05:17:16.335906+0000) 2024-09-19T05:17:19.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:19 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:19.652+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:19.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:19 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:19.652+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:19.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:19 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:19.652+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:19.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:19 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:19.652+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.171184+0000 front 2024-09-19T05:16:54.171329+0000 (oldest deadline 2024-09-19T05:17:18.271276+0000) 2024-09-19T05:17:19.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:19 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:19.571+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:19.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:19 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:19.571+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:19.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:19 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:19.571+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:19.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:19 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:19.571+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:43.370861+0000 front 2024-09-19T05:16:43.370887+0000 (oldest deadline 2024-09-19T05:17:08.070492+0000) 2024-09-19T05:17:19.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:19 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:19.571+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:50.971438+0000 front 2024-09-19T05:16:50.971421+0000 (oldest deadline 2024-09-19T05:17:16.871470+0000) 2024-09-19T05:17:20.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:19 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:19.607+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:17:20.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:19 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:19.607+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.976294+0000 front 2024-09-19T05:16:28.976202+0000 (oldest deadline 2024-09-19T05:16:54.276323+0000) 2024-09-19T05:17:20.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:19 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:19.607+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.678064+0000 front 2024-09-19T05:16:37.678149+0000 (oldest deadline 2024-09-19T05:16:59.278673+0000) 2024-09-19T05:17:20.032 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:19 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:19.607+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:45.680188+0000 front 2024-09-19T05:16:45.680274+0000 (oldest deadline 2024-09-19T05:17:07.380336+0000) 2024-09-19T05:17:20.032 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:19 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:19.607+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:53.782102+0000 front 2024-09-19T05:16:53.782082+0000 (oldest deadline 2024-09-19T05:17:16.082229+0000) 2024-09-19T05:17:20.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:20 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:20.073+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:27.546438+0000 front 2024-09-19T05:16:27.546400+0000 (oldest deadline 2024-09-19T05:16:52.846815+0000) 2024-09-19T05:17:20.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:20 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:20.073+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:38.048448+0000 front 2024-09-19T05:16:38.048358+0000 (oldest deadline 2024-09-19T05:16:59.148332+0000) 2024-09-19T05:17:20.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:20 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:20.073+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:46.050679+0000 front 2024-09-19T05:16:46.050775+0000 (oldest deadline 2024-09-19T05:17:10.150418+0000) 2024-09-19T05:17:20.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:20 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7[52803]: 2024-09-19T05:17:20.073+0000 7fae1b244640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:52.451248+0000 front 2024-09-19T05:16:52.451291+0000 (oldest deadline 2024-09-19T05:17:20.051747+0000) 2024-09-19T05:17:20.655 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:20 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:20.325+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.631981+0000 front 2024-09-19T05:16:29.631823+0000 (oldest deadline 2024-09-19T05:16:54.931686+0000) 2024-09-19T05:17:20.655 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:20 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:20.325+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.232773+0000 front 2024-09-19T05:16:37.232811+0000 (oldest deadline 2024-09-19T05:17:00.132921+0000) 2024-09-19T05:17:20.655 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:20 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:20.325+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:46.534659+0000 front 2024-09-19T05:16:46.534699+0000 (oldest deadline 2024-09-19T05:17:12.434425+0000) 2024-09-19T05:17:20.655 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:20 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:20.325+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.036000+0000 front 2024-09-19T05:16:54.036137+0000 (oldest deadline 2024-09-19T05:17:16.335906+0000) 2024-09-19T05:17:20.781 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:20 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:20.583+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:17:20.781 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:20 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:20.583+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.976294+0000 front 2024-09-19T05:16:28.976202+0000 (oldest deadline 2024-09-19T05:16:54.276323+0000) 2024-09-19T05:17:20.781 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:20 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:20.583+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.678064+0000 front 2024-09-19T05:16:37.678149+0000 (oldest deadline 2024-09-19T05:16:59.278673+0000) 2024-09-19T05:17:20.781 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:20 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:20.583+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:45.680188+0000 front 2024-09-19T05:16:45.680274+0000 (oldest deadline 2024-09-19T05:17:07.380336+0000) 2024-09-19T05:17:20.781 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:20 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:20.583+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:53.782102+0000 front 2024-09-19T05:16:53.782082+0000 (oldest deadline 2024-09-19T05:17:16.082229+0000) 2024-09-19T05:17:20.899 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:20 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:20.575+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:20.899 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:20 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:20.575+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:20.899 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:20 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:20.575+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:20.899 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:20 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:20.575+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:43.370861+0000 front 2024-09-19T05:16:43.370887+0000 (oldest deadline 2024-09-19T05:17:08.070492+0000) 2024-09-19T05:17:20.899 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:20 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:20.575+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:50.971438+0000 front 2024-09-19T05:16:50.971421+0000 (oldest deadline 2024-09-19T05:17:16.871470+0000) 2024-09-19T05:17:21.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:20 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:20.666+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:21.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:20 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:20.666+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:21.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:20 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:20.666+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:21.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:20 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:20.666+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.171184+0000 front 2024-09-19T05:16:54.171329+0000 (oldest deadline 2024-09-19T05:17:18.271276+0000) 2024-09-19T05:17:21.178 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:20 smithi096 podman[71268]: 2024-09-19 05:17:20.898019399 +0000 UTC m=+5.715432400 container died 179eb0c5de23e1adae0db644666c4d515e9bcd9426d5bb0d4bfe06a4fff93794 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True) 2024-09-19T05:17:21.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:21 smithi096 podman[71268]: 2024-09-19 05:17:21.251018502 +0000 UTC m=+6.068431511 container cleanup 179eb0c5de23e1adae0db644666c4d515e9bcd9426d5bb0d4bfe06a4fff93794 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-19T05:17:21.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:21 smithi096 bash[71268]: ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7 2024-09-19T05:17:21.655 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:21 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:21.350+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.631981+0000 front 2024-09-19T05:16:29.631823+0000 (oldest deadline 2024-09-19T05:16:54.931686+0000) 2024-09-19T05:17:21.655 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:21 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:21.350+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.232773+0000 front 2024-09-19T05:16:37.232811+0000 (oldest deadline 2024-09-19T05:17:00.132921+0000) 2024-09-19T05:17:21.655 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:21 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:21.350+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:46.534659+0000 front 2024-09-19T05:16:46.534699+0000 (oldest deadline 2024-09-19T05:17:12.434425+0000) 2024-09-19T05:17:21.655 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:21 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:21.350+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.036000+0000 front 2024-09-19T05:16:54.036137+0000 (oldest deadline 2024-09-19T05:17:16.335906+0000) 2024-09-19T05:17:21.867 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:21 smithi096 podman[71283]: 2024-09-19 05:17:21.510929411 +0000 UTC m=+0.614015595 container remove 179eb0c5de23e1adae0db644666c4d515e9bcd9426d5bb0d4bfe06a4fff93794 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-19T05:17:21.868 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:21 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:21.597+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:21.868 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:21 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:21.597+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:21.868 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:21 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:21.597+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:21.868 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:21 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:21.597+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:43.370861+0000 front 2024-09-19T05:16:43.370887+0000 (oldest deadline 2024-09-19T05:17:08.070492+0000) 2024-09-19T05:17:21.868 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:21 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:21.597+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:50.971438+0000 front 2024-09-19T05:16:50.971421+0000 (oldest deadline 2024-09-19T05:17:16.871470+0000) 2024-09-19T05:17:22.030 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:21 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:21.567+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:17:22.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:21 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:21.567+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.976294+0000 front 2024-09-19T05:16:28.976202+0000 (oldest deadline 2024-09-19T05:16:54.276323+0000) 2024-09-19T05:17:22.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:21 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:21.567+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.678064+0000 front 2024-09-19T05:16:37.678149+0000 (oldest deadline 2024-09-19T05:16:59.278673+0000) 2024-09-19T05:17:22.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:21 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:21.567+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:45.680188+0000 front 2024-09-19T05:16:45.680274+0000 (oldest deadline 2024-09-19T05:17:07.380336+0000) 2024-09-19T05:17:22.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:21 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:21.567+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:53.782102+0000 front 2024-09-19T05:16:53.782082+0000 (oldest deadline 2024-09-19T05:17:16.082229+0000) 2024-09-19T05:17:22.123 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:21 smithi096 podman[71370]: 2024-09-19 05:17:21.76915839 +0000 UTC m=+0.016235946 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:17:22.123 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:21 smithi096 podman[71370]: 2024-09-19 05:17:21.878199058 +0000 UTC m=+0.125276599 container create e0b0359e37498b813b1e0b8e16a605f73689dce9d999d59ae8b816bb11b139cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-19T05:17:22.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:21 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:21.705+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:22.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:21 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:21.705+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:22.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:21 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:21.705+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:22.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:21 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:21.705+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.171184+0000 front 2024-09-19T05:16:54.171329+0000 (oldest deadline 2024-09-19T05:17:18.271276+0000) 2024-09-19T05:17:22.473 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:22 smithi096 podman[71370]: 2024-09-19 05:17:22.12231434 +0000 UTC m=+0.369391881 container init e0b0359e37498b813b1e0b8e16a605f73689dce9d999d59ae8b816bb11b139cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T05:17:22.474 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:22 smithi096 podman[71370]: 2024-09-19 05:17:22.125684779 +0000 UTC m=+0.372762335 container start e0b0359e37498b813b1e0b8e16a605f73689dce9d999d59ae8b816bb11b139cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-19T05:17:22.474 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:22 smithi096 podman[71370]: 2024-09-19 05:17:22.247376894 +0000 UTC m=+0.494454446 container attach e0b0359e37498b813b1e0b8e16a605f73689dce9d999d59ae8b816bb11b139cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-19T05:17:22.474 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:22 smithi096 podman[71370]: 2024-09-19 05:17:22.319245084 +0000 UTC m=+0.566322640 container died e0b0359e37498b813b1e0b8e16a605f73689dce9d999d59ae8b816bb11b139cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-19T05:17:22.655 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:22 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:22.380+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.631981+0000 front 2024-09-19T05:16:29.631823+0000 (oldest deadline 2024-09-19T05:16:54.931686+0000) 2024-09-19T05:17:22.655 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:22 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:22.380+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.232773+0000 front 2024-09-19T05:16:37.232811+0000 (oldest deadline 2024-09-19T05:17:00.132921+0000) 2024-09-19T05:17:22.655 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:22 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:22.380+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:46.534659+0000 front 2024-09-19T05:16:46.534699+0000 (oldest deadline 2024-09-19T05:17:12.434425+0000) 2024-09-19T05:17:22.655 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:22 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:22.380+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.036000+0000 front 2024-09-19T05:16:54.036137+0000 (oldest deadline 2024-09-19T05:17:16.335906+0000) 2024-09-19T05:17:22.875 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 05:17:22 smithi096 podman[71393]: 2024-09-19 05:17:22.859206575 +0000 UTC m=+0.532603406 container remove e0b0359e37498b813b1e0b8e16a605f73689dce9d999d59ae8b816bb11b139cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T05:17:22.875 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:22 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:22.619+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:22.875 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:22 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:22.619+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:22.875 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:22 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:22.619+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:22.875 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:22 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:22.619+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:43.370861+0000 front 2024-09-19T05:16:43.370887+0000 (oldest deadline 2024-09-19T05:17:08.070492+0000) 2024-09-19T05:17:22.875 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:22 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:22.619+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:50.971438+0000 front 2024-09-19T05:16:50.971421+0000 (oldest deadline 2024-09-19T05:17:16.871470+0000) 2024-09-19T05:17:22.880 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@osd.7.service' 2024-09-19T05:17:22.919 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:17:22.919 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-19T05:17:22.919 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-19T05:17:22.920 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-1456a810-7643-11ef-bceb-c7b262605968@osd.8 2024-09-19T05:17:22.947 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:22 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:22.695+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:22.948 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:22 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:22.695+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:22.948 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:22 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:22.695+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:22.948 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:22 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:22.695+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.171184+0000 front 2024-09-19T05:16:54.171329+0000 (oldest deadline 2024-09-19T05:17:18.271276+0000) 2024-09-19T05:17:23.030 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:22 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:22.545+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:17:23.030 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:22 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:22.545+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.976294+0000 front 2024-09-19T05:16:28.976202+0000 (oldest deadline 2024-09-19T05:16:54.276323+0000) 2024-09-19T05:17:23.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:22 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:22.545+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.678064+0000 front 2024-09-19T05:16:37.678149+0000 (oldest deadline 2024-09-19T05:16:59.278673+0000) 2024-09-19T05:17:23.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:22 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:22.545+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:45.680188+0000 front 2024-09-19T05:16:45.680274+0000 (oldest deadline 2024-09-19T05:17:07.380336+0000) 2024-09-19T05:17:23.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:22 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:22.545+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:53.782102+0000 front 2024-09-19T05:16:53.782082+0000 (oldest deadline 2024-09-19T05:17:16.082229+0000) 2024-09-19T05:17:23.240 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:22 smithi151 systemd[1]: Stopping Ceph osd.8 for 1456a810-7643-11ef-bceb-c7b262605968... 2024-09-19T05:17:23.655 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:23 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:23.238+0000 7f9296c41640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:17:23.655 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:23 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:23.238+0000 7f9296c41640 -1 osd.8 99 *** Got signal Terminated *** 2024-09-19T05:17:23.655 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:23 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:23.238+0000 7f9296c41640 -1 osd.8 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T05:17:23.655 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:23 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:23.348+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.631981+0000 front 2024-09-19T05:16:29.631823+0000 (oldest deadline 2024-09-19T05:16:54.931686+0000) 2024-09-19T05:17:23.655 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:23 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:23.348+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.232773+0000 front 2024-09-19T05:16:37.232811+0000 (oldest deadline 2024-09-19T05:17:00.132921+0000) 2024-09-19T05:17:23.655 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:23 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:23.348+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:46.534659+0000 front 2024-09-19T05:16:46.534699+0000 (oldest deadline 2024-09-19T05:17:12.434425+0000) 2024-09-19T05:17:23.656 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:23 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:23.348+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.036000+0000 front 2024-09-19T05:16:54.036137+0000 (oldest deadline 2024-09-19T05:17:16.335906+0000) 2024-09-19T05:17:23.972 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:23 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:23.602+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:23.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:23 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:23.602+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:23.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:23 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:23.602+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:23.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:23 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:23.602+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:43.370861+0000 front 2024-09-19T05:16:43.370887+0000 (oldest deadline 2024-09-19T05:17:08.070492+0000) 2024-09-19T05:17:23.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:23 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:23.602+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:50.971438+0000 front 2024-09-19T05:16:50.971421+0000 (oldest deadline 2024-09-19T05:17:16.871470+0000) 2024-09-19T05:17:24.030 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:23 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:23.585+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:17:24.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:23 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:23.585+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.976294+0000 front 2024-09-19T05:16:28.976202+0000 (oldest deadline 2024-09-19T05:16:54.276323+0000) 2024-09-19T05:17:24.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:23 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:23.585+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.678064+0000 front 2024-09-19T05:16:37.678149+0000 (oldest deadline 2024-09-19T05:16:59.278673+0000) 2024-09-19T05:17:24.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:23 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:23.585+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:45.680188+0000 front 2024-09-19T05:16:45.680274+0000 (oldest deadline 2024-09-19T05:17:07.380336+0000) 2024-09-19T05:17:24.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:23 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:23.585+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:53.782102+0000 front 2024-09-19T05:16:53.782082+0000 (oldest deadline 2024-09-19T05:17:16.082229+0000) 2024-09-19T05:17:24.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:23 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:23.738+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:24.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:23 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:23.738+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:24.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:23 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:23.738+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:24.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:23 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:23.738+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.171184+0000 front 2024-09-19T05:16:54.171329+0000 (oldest deadline 2024-09-19T05:17:18.271276+0000) 2024-09-19T05:17:24.655 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:24 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:24.342+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.631981+0000 front 2024-09-19T05:16:29.631823+0000 (oldest deadline 2024-09-19T05:16:54.931686+0000) 2024-09-19T05:17:24.655 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:24 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:24.342+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.232773+0000 front 2024-09-19T05:16:37.232811+0000 (oldest deadline 2024-09-19T05:17:00.132921+0000) 2024-09-19T05:17:24.655 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:24 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:24.342+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:46.534659+0000 front 2024-09-19T05:16:46.534699+0000 (oldest deadline 2024-09-19T05:17:12.434425+0000) 2024-09-19T05:17:24.655 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:24 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:24.342+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.036000+0000 front 2024-09-19T05:16:54.036137+0000 (oldest deadline 2024-09-19T05:17:16.335906+0000) 2024-09-19T05:17:24.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:24 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:24.630+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:24.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:24 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:24.630+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:24.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:24 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:24.630+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:24.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:24 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:24.630+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:43.370861+0000 front 2024-09-19T05:16:43.370887+0000 (oldest deadline 2024-09-19T05:17:08.070492+0000) 2024-09-19T05:17:24.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:24 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:24.630+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:50.971438+0000 front 2024-09-19T05:16:50.971421+0000 (oldest deadline 2024-09-19T05:17:16.871470+0000) 2024-09-19T05:17:25.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:24 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:24.628+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:17:25.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:24 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:24.628+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.976294+0000 front 2024-09-19T05:16:28.976202+0000 (oldest deadline 2024-09-19T05:16:54.276323+0000) 2024-09-19T05:17:25.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:24 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:24.628+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.678064+0000 front 2024-09-19T05:16:37.678149+0000 (oldest deadline 2024-09-19T05:16:59.278673+0000) 2024-09-19T05:17:25.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:24 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:24.628+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:45.680188+0000 front 2024-09-19T05:16:45.680274+0000 (oldest deadline 2024-09-19T05:17:07.380336+0000) 2024-09-19T05:17:25.032 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:24 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:24.628+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:53.782102+0000 front 2024-09-19T05:16:53.782082+0000 (oldest deadline 2024-09-19T05:17:16.082229+0000) 2024-09-19T05:17:25.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:24 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:24.783+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:25.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:24 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:24.783+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:25.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:24 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:24.783+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:25.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:24 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:24.783+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.171184+0000 front 2024-09-19T05:16:54.171329+0000 (oldest deadline 2024-09-19T05:17:18.271276+0000) 2024-09-19T05:17:25.655 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:25 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:25.342+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.631981+0000 front 2024-09-19T05:16:29.631823+0000 (oldest deadline 2024-09-19T05:16:54.931686+0000) 2024-09-19T05:17:25.655 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:25 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:25.342+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.232773+0000 front 2024-09-19T05:16:37.232811+0000 (oldest deadline 2024-09-19T05:17:00.132921+0000) 2024-09-19T05:17:25.655 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:25 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:25.342+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:46.534659+0000 front 2024-09-19T05:16:46.534699+0000 (oldest deadline 2024-09-19T05:17:12.434425+0000) 2024-09-19T05:17:25.655 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:25 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:25.342+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.036000+0000 front 2024-09-19T05:16:54.036137+0000 (oldest deadline 2024-09-19T05:17:16.335906+0000) 2024-09-19T05:17:25.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:25 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:25.590+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:25.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:25 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:25.590+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:25.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:25 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:25.590+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:25.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:25 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:25.590+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:43.370861+0000 front 2024-09-19T05:16:43.370887+0000 (oldest deadline 2024-09-19T05:17:08.070492+0000) 2024-09-19T05:17:25.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:25 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:25.590+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:50.971438+0000 front 2024-09-19T05:16:50.971421+0000 (oldest deadline 2024-09-19T05:17:16.871470+0000) 2024-09-19T05:17:26.030 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:25 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:25.613+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:17:26.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:25 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:25.613+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.976294+0000 front 2024-09-19T05:16:28.976202+0000 (oldest deadline 2024-09-19T05:16:54.276323+0000) 2024-09-19T05:17:26.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:25 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:25.613+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.678064+0000 front 2024-09-19T05:16:37.678149+0000 (oldest deadline 2024-09-19T05:16:59.278673+0000) 2024-09-19T05:17:26.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:25 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:25.613+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:45.680188+0000 front 2024-09-19T05:16:45.680274+0000 (oldest deadline 2024-09-19T05:17:07.380336+0000) 2024-09-19T05:17:26.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:25 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:25.613+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:53.782102+0000 front 2024-09-19T05:16:53.782082+0000 (oldest deadline 2024-09-19T05:17:16.082229+0000) 2024-09-19T05:17:26.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:25 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:25.796+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:26.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:25 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:25.796+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:26.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:25 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:25.796+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:26.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:25 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:25.796+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.171184+0000 front 2024-09-19T05:16:54.171329+0000 (oldest deadline 2024-09-19T05:17:18.271276+0000) 2024-09-19T05:17:26.655 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:26 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:26.366+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.631981+0000 front 2024-09-19T05:16:29.631823+0000 (oldest deadline 2024-09-19T05:16:54.931686+0000) 2024-09-19T05:17:26.655 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:26 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:26.366+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.232773+0000 front 2024-09-19T05:16:37.232811+0000 (oldest deadline 2024-09-19T05:17:00.132921+0000) 2024-09-19T05:17:26.655 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:26 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:26.366+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:46.534659+0000 front 2024-09-19T05:16:46.534699+0000 (oldest deadline 2024-09-19T05:17:12.434425+0000) 2024-09-19T05:17:26.655 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:26 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:26.366+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.036000+0000 front 2024-09-19T05:16:54.036137+0000 (oldest deadline 2024-09-19T05:17:16.335906+0000) 2024-09-19T05:17:26.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:26 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:26.577+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:26.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:26 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:26.577+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:26.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:26 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:26.577+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:26.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:26 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:26.577+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:43.370861+0000 front 2024-09-19T05:16:43.370887+0000 (oldest deadline 2024-09-19T05:17:08.070492+0000) 2024-09-19T05:17:26.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:26 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:26.577+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:50.971438+0000 front 2024-09-19T05:16:50.971421+0000 (oldest deadline 2024-09-19T05:17:16.871470+0000) 2024-09-19T05:17:27.030 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:26 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:26.646+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:17:27.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:26 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:26.646+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.976294+0000 front 2024-09-19T05:16:28.976202+0000 (oldest deadline 2024-09-19T05:16:54.276323+0000) 2024-09-19T05:17:27.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:26 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:26.646+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.678064+0000 front 2024-09-19T05:16:37.678149+0000 (oldest deadline 2024-09-19T05:16:59.278673+0000) 2024-09-19T05:17:27.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:26 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:26.646+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:45.680188+0000 front 2024-09-19T05:16:45.680274+0000 (oldest deadline 2024-09-19T05:17:07.380336+0000) 2024-09-19T05:17:27.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:26 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:26.646+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:53.782102+0000 front 2024-09-19T05:16:53.782082+0000 (oldest deadline 2024-09-19T05:17:16.082229+0000) 2024-09-19T05:17:27.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:26 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:26.823+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:27.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:26 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:26.823+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:27.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:26 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:26.823+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:27.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:26 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:26.823+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.171184+0000 front 2024-09-19T05:16:54.171329+0000 (oldest deadline 2024-09-19T05:17:18.271276+0000) 2024-09-19T05:17:27.655 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:27 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:27.349+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.631981+0000 front 2024-09-19T05:16:29.631823+0000 (oldest deadline 2024-09-19T05:16:54.931686+0000) 2024-09-19T05:17:27.655 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:27 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:27.349+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.232773+0000 front 2024-09-19T05:16:37.232811+0000 (oldest deadline 2024-09-19T05:17:00.132921+0000) 2024-09-19T05:17:27.655 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:27 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:27.349+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:46.534659+0000 front 2024-09-19T05:16:46.534699+0000 (oldest deadline 2024-09-19T05:17:12.434425+0000) 2024-09-19T05:17:27.655 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:27 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:27.349+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.036000+0000 front 2024-09-19T05:16:54.036137+0000 (oldest deadline 2024-09-19T05:17:16.335906+0000) 2024-09-19T05:17:27.655 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:27 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8[53790]: 2024-09-19T05:17:27.349+0000 7f928f24f640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:17:01.938014+0000 front 2024-09-19T05:17:01.937979+0000 (oldest deadline 2024-09-19T05:17:27.238055+0000) 2024-09-19T05:17:27.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:27 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:27.607+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:27.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:27 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:27.607+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:27.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:27 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:27.607+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:27.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:27 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:27.607+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:43.370861+0000 front 2024-09-19T05:16:43.370887+0000 (oldest deadline 2024-09-19T05:17:08.070492+0000) 2024-09-19T05:17:27.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:27 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:27.607+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:50.971438+0000 front 2024-09-19T05:16:50.971421+0000 (oldest deadline 2024-09-19T05:17:16.871470+0000) 2024-09-19T05:17:27.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:27 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:27.607+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:16:59.772595+0000 front 2024-09-19T05:16:59.772482+0000 (oldest deadline 2024-09-19T05:17:27.372759+0000) 2024-09-19T05:17:28.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:27 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:27.661+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:17:28.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:27 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:27.661+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.976294+0000 front 2024-09-19T05:16:28.976202+0000 (oldest deadline 2024-09-19T05:16:54.276323+0000) 2024-09-19T05:17:28.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:27 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:27.661+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.678064+0000 front 2024-09-19T05:16:37.678149+0000 (oldest deadline 2024-09-19T05:16:59.278673+0000) 2024-09-19T05:17:28.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:27 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:27.661+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:45.680188+0000 front 2024-09-19T05:16:45.680274+0000 (oldest deadline 2024-09-19T05:17:07.380336+0000) 2024-09-19T05:17:28.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:27 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:27.661+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:53.782102+0000 front 2024-09-19T05:16:53.782082+0000 (oldest deadline 2024-09-19T05:17:16.082229+0000) 2024-09-19T05:17:28.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:27 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:27.803+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:28.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:27 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:27.803+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:28.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:27 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:27.803+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:28.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:27 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:27.803+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.171184+0000 front 2024-09-19T05:16:54.171329+0000 (oldest deadline 2024-09-19T05:17:18.271276+0000) 2024-09-19T05:17:28.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:27 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:27.803+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:16:58.772220+0000 front 2024-09-19T05:16:58.772177+0000 (oldest deadline 2024-09-19T05:17:26.972713+0000) 2024-09-19T05:17:28.949 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:28 smithi151 podman[72069]: 2024-09-19 05:17:28.684502668 +0000 UTC m=+5.574019981 container died 3567e6bc34cd96f98137dd418b68bbc3bded8d272215263cd61351003b05b381 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_CLEAN=True) 2024-09-19T05:17:28.950 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:28 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:28.846+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:28.950 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:28 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:28.846+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:28.950 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:28 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:28.846+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:28.950 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:28 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:28.846+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.171184+0000 front 2024-09-19T05:16:54.171329+0000 (oldest deadline 2024-09-19T05:17:18.271276+0000) 2024-09-19T05:17:28.950 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:28 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:28.846+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:16:58.772220+0000 front 2024-09-19T05:16:58.772177+0000 (oldest deadline 2024-09-19T05:17:26.972713+0000) 2024-09-19T05:17:28.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:28 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:28.625+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:28.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:28 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:28.625+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:28.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:28 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:28.625+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:28.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:28 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:28.625+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:43.370861+0000 front 2024-09-19T05:16:43.370887+0000 (oldest deadline 2024-09-19T05:17:08.070492+0000) 2024-09-19T05:17:28.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:28 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:28.625+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:50.971438+0000 front 2024-09-19T05:16:50.971421+0000 (oldest deadline 2024-09-19T05:17:16.871470+0000) 2024-09-19T05:17:28.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:28 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:28.625+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:16:59.772595+0000 front 2024-09-19T05:16:59.772482+0000 (oldest deadline 2024-09-19T05:17:27.372759+0000) 2024-09-19T05:17:29.030 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:28 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:28.652+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:17:29.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:28 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:28.652+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.976294+0000 front 2024-09-19T05:16:28.976202+0000 (oldest deadline 2024-09-19T05:16:54.276323+0000) 2024-09-19T05:17:29.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:28 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:28.652+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.678064+0000 front 2024-09-19T05:16:37.678149+0000 (oldest deadline 2024-09-19T05:16:59.278673+0000) 2024-09-19T05:17:29.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:28 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:28.652+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:45.680188+0000 front 2024-09-19T05:16:45.680274+0000 (oldest deadline 2024-09-19T05:17:07.380336+0000) 2024-09-19T05:17:29.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:28 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:28.652+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:53.782102+0000 front 2024-09-19T05:16:53.782082+0000 (oldest deadline 2024-09-19T05:17:16.082229+0000) 2024-09-19T05:17:29.359 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:29 smithi151 podman[72069]: 2024-09-19 05:17:29.048117999 +0000 UTC m=+5.937635311 container cleanup 3567e6bc34cd96f98137dd418b68bbc3bded8d272215263cd61351003b05b381 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:17:29.359 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:29 smithi151 bash[72069]: ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8 2024-09-19T05:17:29.655 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:29 smithi151 podman[72086]: 2024-09-19 05:17:29.358148634 +0000 UTC m=+0.667851583 container remove 3567e6bc34cd96f98137dd418b68bbc3bded8d272215263cd61351003b05b381 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS) 2024-09-19T05:17:29.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:29 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:29.623+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:29.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:29 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:29.623+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:29.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:29 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:29.623+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:29.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:29 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:29.623+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:43.370861+0000 front 2024-09-19T05:16:43.370887+0000 (oldest deadline 2024-09-19T05:17:08.070492+0000) 2024-09-19T05:17:29.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:29 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:29.623+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:50.971438+0000 front 2024-09-19T05:16:50.971421+0000 (oldest deadline 2024-09-19T05:17:16.871470+0000) 2024-09-19T05:17:29.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:29 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:29.623+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:16:59.772595+0000 front 2024-09-19T05:16:59.772482+0000 (oldest deadline 2024-09-19T05:17:27.372759+0000) 2024-09-19T05:17:30.014 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:29 smithi151 podman[72173]: 2024-09-19 05:17:29.618655877 +0000 UTC m=+0.015614599 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:17:30.014 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:29 smithi151 podman[72173]: 2024-09-19 05:17:29.743113653 +0000 UTC m=+0.140072406 container create 618dfef76e071cf319a7211bae7271f22b7f998385f5ffd8d584c76d7f30d422 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-19T05:17:30.014 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:29 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:29.850+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:30.014 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:29 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:29.850+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:30.014 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:29 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:29.850+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:30.014 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:29 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:29.850+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.171184+0000 front 2024-09-19T05:16:54.171329+0000 (oldest deadline 2024-09-19T05:17:18.271276+0000) 2024-09-19T05:17:30.014 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:29 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:29.850+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:16:58.772220+0000 front 2024-09-19T05:16:58.772177+0000 (oldest deadline 2024-09-19T05:17:26.972713+0000) 2024-09-19T05:17:30.032 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:29 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:29.681+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:17:30.032 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:29 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:29.681+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.976294+0000 front 2024-09-19T05:16:28.976202+0000 (oldest deadline 2024-09-19T05:16:54.276323+0000) 2024-09-19T05:17:30.032 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:29 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:29.681+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.678064+0000 front 2024-09-19T05:16:37.678149+0000 (oldest deadline 2024-09-19T05:16:59.278673+0000) 2024-09-19T05:17:30.032 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:29 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:29.681+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:45.680188+0000 front 2024-09-19T05:16:45.680274+0000 (oldest deadline 2024-09-19T05:17:07.380336+0000) 2024-09-19T05:17:30.032 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:29 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:29.681+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:53.782102+0000 front 2024-09-19T05:16:53.782082+0000 (oldest deadline 2024-09-19T05:17:16.082229+0000) 2024-09-19T05:17:30.405 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:30 smithi151 podman[72173]: 2024-09-19 05:17:30.012651191 +0000 UTC m=+0.409609904 container init 618dfef76e071cf319a7211bae7271f22b7f998385f5ffd8d584c76d7f30d422 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-19T05:17:30.405 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:30 smithi151 podman[72173]: 2024-09-19 05:17:30.016278123 +0000 UTC m=+0.413236837 container start 618dfef76e071cf319a7211bae7271f22b7f998385f5ffd8d584c76d7f30d422 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-19T05:17:30.406 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:30 smithi151 podman[72173]: 2024-09-19 05:17:30.104649051 +0000 UTC m=+0.501607778 container attach 618dfef76e071cf319a7211bae7271f22b7f998385f5ffd8d584c76d7f30d422 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=) 2024-09-19T05:17:30.406 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 05:17:30 smithi151 podman[72173]: 2024-09-19 05:17:30.207075257 +0000 UTC m=+0.604033974 container died 618dfef76e071cf319a7211bae7271f22b7f998385f5ffd8d584c76d7f30d422 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:17:30.875 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@osd.8.service' 2024-09-19T05:17:30.953 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:17:30.954 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-19T05:17:30.954 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-19T05:17:30.954 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-1456a810-7643-11ef-bceb-c7b262605968@osd.9 2024-09-19T05:17:30.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:30 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:30.598+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:30.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:30 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:30.598+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:30.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:30 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:30.598+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:30.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:30 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:30.598+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:43.370861+0000 front 2024-09-19T05:16:43.370887+0000 (oldest deadline 2024-09-19T05:17:08.070492+0000) 2024-09-19T05:17:30.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:30 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:30.599+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:50.971438+0000 front 2024-09-19T05:16:50.971421+0000 (oldest deadline 2024-09-19T05:17:16.871470+0000) 2024-09-19T05:17:30.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:30 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:30.599+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:16:59.772595+0000 front 2024-09-19T05:16:59.772482+0000 (oldest deadline 2024-09-19T05:17:27.372759+0000) 2024-09-19T05:17:31.008 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:30 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:30.699+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:17:31.008 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:30 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:30.699+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.976294+0000 front 2024-09-19T05:16:28.976202+0000 (oldest deadline 2024-09-19T05:16:54.276323+0000) 2024-09-19T05:17:31.008 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:30 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:30.699+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.678064+0000 front 2024-09-19T05:16:37.678149+0000 (oldest deadline 2024-09-19T05:16:59.278673+0000) 2024-09-19T05:17:31.008 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:30 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:30.699+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:45.680188+0000 front 2024-09-19T05:16:45.680274+0000 (oldest deadline 2024-09-19T05:17:07.380336+0000) 2024-09-19T05:17:31.008 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:30 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:30.699+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:53.782102+0000 front 2024-09-19T05:16:53.782082+0000 (oldest deadline 2024-09-19T05:17:16.082229+0000) 2024-09-19T05:17:31.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:30 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:30.887+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:31.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:30 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:30.887+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:31.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:30 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:30.887+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:31.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:30 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:30.887+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.171184+0000 front 2024-09-19T05:16:54.171329+0000 (oldest deadline 2024-09-19T05:17:18.271276+0000) 2024-09-19T05:17:31.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:30 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:30.887+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:16:58.772220+0000 front 2024-09-19T05:16:58.772177+0000 (oldest deadline 2024-09-19T05:17:26.972713+0000) 2024-09-19T05:17:31.281 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:31 smithi043 systemd[1]: Stopping Ceph osd.9 for 1456a810-7643-11ef-bceb-c7b262605968... 2024-09-19T05:17:31.689 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:31 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:31.384+0000 7f5bc2c00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:17:31.689 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:31 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:31.384+0000 7f5bc2c00640 -1 osd.9 99 *** Got signal Terminated *** 2024-09-19T05:17:31.689 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:31 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:31.384+0000 7f5bc2c00640 -1 osd.9 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T05:17:31.689 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:31 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:31.687+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:17:31.689 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:31 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:31.687+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.976294+0000 front 2024-09-19T05:16:28.976202+0000 (oldest deadline 2024-09-19T05:16:54.276323+0000) 2024-09-19T05:17:31.689 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:31 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:31.687+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.678064+0000 front 2024-09-19T05:16:37.678149+0000 (oldest deadline 2024-09-19T05:16:59.278673+0000) 2024-09-19T05:17:31.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:31 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:31.588+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:31.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:31 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:31.588+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:31.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:31 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:31.588+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:31.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:31 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:31.588+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:43.370861+0000 front 2024-09-19T05:16:43.370887+0000 (oldest deadline 2024-09-19T05:17:08.070492+0000) 2024-09-19T05:17:31.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:31 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:31.588+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:50.971438+0000 front 2024-09-19T05:16:50.971421+0000 (oldest deadline 2024-09-19T05:17:16.871470+0000) 2024-09-19T05:17:31.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:31 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:31.588+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:16:59.772595+0000 front 2024-09-19T05:16:59.772482+0000 (oldest deadline 2024-09-19T05:17:27.372759+0000) 2024-09-19T05:17:32.030 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:31 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:31.687+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:45.680188+0000 front 2024-09-19T05:16:45.680274+0000 (oldest deadline 2024-09-19T05:17:07.380336+0000) 2024-09-19T05:17:32.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:31 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:31.687+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:53.782102+0000 front 2024-09-19T05:16:53.782082+0000 (oldest deadline 2024-09-19T05:17:16.082229+0000) 2024-09-19T05:17:32.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:31 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:31.888+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:32.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:31 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:31.888+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:32.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:31 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:31.888+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:32.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:31 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:31.888+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.171184+0000 front 2024-09-19T05:16:54.171329+0000 (oldest deadline 2024-09-19T05:17:18.271276+0000) 2024-09-19T05:17:32.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:31 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:31.888+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:16:58.772220+0000 front 2024-09-19T05:16:58.772177+0000 (oldest deadline 2024-09-19T05:17:26.972713+0000) 2024-09-19T05:17:32.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:32 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:32.551+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:32.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:32 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:32.551+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:32.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:32 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:32.551+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:32.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:32 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:32.551+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:43.370861+0000 front 2024-09-19T05:16:43.370887+0000 (oldest deadline 2024-09-19T05:17:08.070492+0000) 2024-09-19T05:17:32.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:32 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:32.551+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:50.971438+0000 front 2024-09-19T05:16:50.971421+0000 (oldest deadline 2024-09-19T05:17:16.871470+0000) 2024-09-19T05:17:32.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:32 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:32.551+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:16:59.772595+0000 front 2024-09-19T05:16:59.772482+0000 (oldest deadline 2024-09-19T05:17:27.372759+0000) 2024-09-19T05:17:33.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:32 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:32.689+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:17:33.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:32 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:32.689+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.976294+0000 front 2024-09-19T05:16:28.976202+0000 (oldest deadline 2024-09-19T05:16:54.276323+0000) 2024-09-19T05:17:33.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:32 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:32.689+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.678064+0000 front 2024-09-19T05:16:37.678149+0000 (oldest deadline 2024-09-19T05:16:59.278673+0000) 2024-09-19T05:17:33.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:32 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:32.689+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:45.680188+0000 front 2024-09-19T05:16:45.680274+0000 (oldest deadline 2024-09-19T05:17:07.380336+0000) 2024-09-19T05:17:33.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:32 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:32.689+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:53.782102+0000 front 2024-09-19T05:16:53.782082+0000 (oldest deadline 2024-09-19T05:17:16.082229+0000) 2024-09-19T05:17:33.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:32 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:32.878+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:33.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:32 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:32.878+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:33.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:32 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:32.878+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:33.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:32 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:32.878+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.171184+0000 front 2024-09-19T05:16:54.171329+0000 (oldest deadline 2024-09-19T05:17:18.271276+0000) 2024-09-19T05:17:33.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:32 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:32.878+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:16:58.772220+0000 front 2024-09-19T05:16:58.772177+0000 (oldest deadline 2024-09-19T05:17:26.972713+0000) 2024-09-19T05:17:33.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:32 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:32.878+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-09-19T05:17:08.673539+0000 front 2024-09-19T05:17:08.673631+0000 (oldest deadline 2024-09-19T05:17:32.773449+0000) 2024-09-19T05:17:33.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:33 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:33.520+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:33.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:33 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:33.520+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:33.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:33 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:33.520+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:33.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:33 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:33.520+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:43.370861+0000 front 2024-09-19T05:16:43.370887+0000 (oldest deadline 2024-09-19T05:17:08.070492+0000) 2024-09-19T05:17:33.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:33 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:33.520+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:50.971438+0000 front 2024-09-19T05:16:50.971421+0000 (oldest deadline 2024-09-19T05:17:16.871470+0000) 2024-09-19T05:17:33.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:33 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:33.520+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:16:59.772595+0000 front 2024-09-19T05:16:59.772482+0000 (oldest deadline 2024-09-19T05:17:27.372759+0000) 2024-09-19T05:17:33.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:33 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:33.520+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-09-19T05:17:10.873848+0000 front 2024-09-19T05:17:10.873916+0000 (oldest deadline 2024-09-19T05:17:32.573687+0000) 2024-09-19T05:17:34.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:33 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:33.715+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:17:34.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:33 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:33.715+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.976294+0000 front 2024-09-19T05:16:28.976202+0000 (oldest deadline 2024-09-19T05:16:54.276323+0000) 2024-09-19T05:17:34.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:33 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:33.715+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.678064+0000 front 2024-09-19T05:16:37.678149+0000 (oldest deadline 2024-09-19T05:16:59.278673+0000) 2024-09-19T05:17:34.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:33 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:33.715+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:45.680188+0000 front 2024-09-19T05:16:45.680274+0000 (oldest deadline 2024-09-19T05:17:07.380336+0000) 2024-09-19T05:17:34.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:33 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:33.715+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:53.782102+0000 front 2024-09-19T05:16:53.782082+0000 (oldest deadline 2024-09-19T05:17:16.082229+0000) 2024-09-19T05:17:34.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:33 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:33.875+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:34.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:33 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:33.875+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:34.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:33 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:33.875+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:34.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:33 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:33.875+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.171184+0000 front 2024-09-19T05:16:54.171329+0000 (oldest deadline 2024-09-19T05:17:18.271276+0000) 2024-09-19T05:17:34.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:33 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:33.875+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:16:58.772220+0000 front 2024-09-19T05:16:58.772177+0000 (oldest deadline 2024-09-19T05:17:26.972713+0000) 2024-09-19T05:17:34.155 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:33 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:33.875+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-09-19T05:17:08.673539+0000 front 2024-09-19T05:17:08.673631+0000 (oldest deadline 2024-09-19T05:17:32.773449+0000) 2024-09-19T05:17:34.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:34 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:34.507+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:34.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:34 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:34.507+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:34.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:34 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:34.507+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:34.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:34 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:34.507+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:43.370861+0000 front 2024-09-19T05:16:43.370887+0000 (oldest deadline 2024-09-19T05:17:08.070492+0000) 2024-09-19T05:17:34.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:34 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:34.507+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:50.971438+0000 front 2024-09-19T05:16:50.971421+0000 (oldest deadline 2024-09-19T05:17:16.871470+0000) 2024-09-19T05:17:34.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:34 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:34.507+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:16:59.772595+0000 front 2024-09-19T05:16:59.772482+0000 (oldest deadline 2024-09-19T05:17:27.372759+0000) 2024-09-19T05:17:34.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:34 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:34.507+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-09-19T05:17:10.873848+0000 front 2024-09-19T05:17:10.873916+0000 (oldest deadline 2024-09-19T05:17:32.573687+0000) 2024-09-19T05:17:35.030 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:34 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:34.731+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:17:35.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:34 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:34.731+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.976294+0000 front 2024-09-19T05:16:28.976202+0000 (oldest deadline 2024-09-19T05:16:54.276323+0000) 2024-09-19T05:17:35.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:34 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:34.731+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.678064+0000 front 2024-09-19T05:16:37.678149+0000 (oldest deadline 2024-09-19T05:16:59.278673+0000) 2024-09-19T05:17:35.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:34 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:34.731+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:45.680188+0000 front 2024-09-19T05:16:45.680274+0000 (oldest deadline 2024-09-19T05:17:07.380336+0000) 2024-09-19T05:17:35.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:34 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:34.731+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:53.782102+0000 front 2024-09-19T05:16:53.782082+0000 (oldest deadline 2024-09-19T05:17:16.082229+0000) 2024-09-19T05:17:35.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:34 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:34.731+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-09-19T05:17:09.985252+0000 front 2024-09-19T05:17:09.985378+0000 (oldest deadline 2024-09-19T05:17:34.685220+0000) 2024-09-19T05:17:35.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:34 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:34.904+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:35.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:34 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:34.904+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:35.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:34 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:34.904+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:35.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:34 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:34.904+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.171184+0000 front 2024-09-19T05:16:54.171329+0000 (oldest deadline 2024-09-19T05:17:18.271276+0000) 2024-09-19T05:17:35.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:34 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:34.904+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:16:58.772220+0000 front 2024-09-19T05:16:58.772177+0000 (oldest deadline 2024-09-19T05:17:26.972713+0000) 2024-09-19T05:17:35.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:34 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:34.904+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-09-19T05:17:08.673539+0000 front 2024-09-19T05:17:08.673631+0000 (oldest deadline 2024-09-19T05:17:32.773449+0000) 2024-09-19T05:17:35.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:35 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:35.494+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:35.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:35 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:35.494+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:35.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:35 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:35.494+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:35.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:35 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:35.494+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:43.370861+0000 front 2024-09-19T05:16:43.370887+0000 (oldest deadline 2024-09-19T05:17:08.070492+0000) 2024-09-19T05:17:35.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:35 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:35.494+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:50.971438+0000 front 2024-09-19T05:16:50.971421+0000 (oldest deadline 2024-09-19T05:17:16.871470+0000) 2024-09-19T05:17:35.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:35 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:35.494+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:16:59.772595+0000 front 2024-09-19T05:16:59.772482+0000 (oldest deadline 2024-09-19T05:17:27.372759+0000) 2024-09-19T05:17:35.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:35 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:35.494+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-09-19T05:17:10.873848+0000 front 2024-09-19T05:17:10.873916+0000 (oldest deadline 2024-09-19T05:17:32.573687+0000) 2024-09-19T05:17:36.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:35 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:35.753+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.574904+0000 front 2024-09-19T05:16:19.574950+0000 (oldest deadline 2024-09-19T05:16:43.075052+0000) 2024-09-19T05:17:36.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:35 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:35.753+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.976294+0000 front 2024-09-19T05:16:28.976202+0000 (oldest deadline 2024-09-19T05:16:54.276323+0000) 2024-09-19T05:17:36.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:35 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:35.753+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.678064+0000 front 2024-09-19T05:16:37.678149+0000 (oldest deadline 2024-09-19T05:16:59.278673+0000) 2024-09-19T05:17:36.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:35 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:35.753+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:45.680188+0000 front 2024-09-19T05:16:45.680274+0000 (oldest deadline 2024-09-19T05:17:07.380336+0000) 2024-09-19T05:17:36.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:35 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:35.753+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:53.782102+0000 front 2024-09-19T05:16:53.782082+0000 (oldest deadline 2024-09-19T05:17:16.082229+0000) 2024-09-19T05:17:36.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:35 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9[62490]: 2024-09-19T05:17:35.753+0000 7f5bbe600640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-09-19T05:17:09.985252+0000 front 2024-09-19T05:17:09.985378+0000 (oldest deadline 2024-09-19T05:17:34.685220+0000) 2024-09-19T05:17:36.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:35 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:35.918+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:36.406 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:35 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:35.918+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:36.406 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:35 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:35.918+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:36.406 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:35 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:35.918+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.171184+0000 front 2024-09-19T05:16:54.171329+0000 (oldest deadline 2024-09-19T05:17:18.271276+0000) 2024-09-19T05:17:36.406 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:35 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:35.918+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:16:58.772220+0000 front 2024-09-19T05:16:58.772177+0000 (oldest deadline 2024-09-19T05:17:26.972713+0000) 2024-09-19T05:17:36.406 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:35 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:35.918+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-09-19T05:17:08.673539+0000 front 2024-09-19T05:17:08.673631+0000 (oldest deadline 2024-09-19T05:17:32.773449+0000) 2024-09-19T05:17:36.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:36 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:36.533+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:36.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:36 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:36.533+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:36.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:36 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:36.533+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:36.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:36 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:36.533+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:43.370861+0000 front 2024-09-19T05:16:43.370887+0000 (oldest deadline 2024-09-19T05:17:08.070492+0000) 2024-09-19T05:17:36.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:36 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:36.533+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:50.971438+0000 front 2024-09-19T05:16:50.971421+0000 (oldest deadline 2024-09-19T05:17:16.871470+0000) 2024-09-19T05:17:36.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:36 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:36.533+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:16:59.772595+0000 front 2024-09-19T05:16:59.772482+0000 (oldest deadline 2024-09-19T05:17:27.372759+0000) 2024-09-19T05:17:36.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:36 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:36.533+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-09-19T05:17:10.873848+0000 front 2024-09-19T05:17:10.873916+0000 (oldest deadline 2024-09-19T05:17:32.573687+0000) 2024-09-19T05:17:37.241 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:36 smithi043 podman[108897]: 2024-09-19 05:17:36.917609743 +0000 UTC m=+5.671787822 container died b3dc20aefb076766043fbf04f267fa980246da2b793596ce345bd42faa201bbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-19T05:17:37.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:36 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:36.918+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:37.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:36 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:36.918+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:37.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:36 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:36.918+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:37.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:36 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:36.918+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.171184+0000 front 2024-09-19T05:16:54.171329+0000 (oldest deadline 2024-09-19T05:17:18.271276+0000) 2024-09-19T05:17:37.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:36 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:36.918+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:16:58.772220+0000 front 2024-09-19T05:16:58.772177+0000 (oldest deadline 2024-09-19T05:17:26.972713+0000) 2024-09-19T05:17:37.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:36 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:36.918+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-09-19T05:17:08.673539+0000 front 2024-09-19T05:17:08.673631+0000 (oldest deadline 2024-09-19T05:17:32.773449+0000) 2024-09-19T05:17:37.531 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:37 smithi043 podman[108897]: 2024-09-19 05:17:37.439448105 +0000 UTC m=+6.193626183 container cleanup b3dc20aefb076766043fbf04f267fa980246da2b793596ce345bd42faa201bbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4) 2024-09-19T05:17:37.531 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:37 smithi043 bash[108897]: ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9 2024-09-19T05:17:37.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:37 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:37.569+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:37.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:37 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:37.569+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:37.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:37 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:37.569+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:37.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:37 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:37.569+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:43.370861+0000 front 2024-09-19T05:16:43.370887+0000 (oldest deadline 2024-09-19T05:17:08.070492+0000) 2024-09-19T05:17:37.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:37 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:37.569+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:50.971438+0000 front 2024-09-19T05:16:50.971421+0000 (oldest deadline 2024-09-19T05:17:16.871470+0000) 2024-09-19T05:17:37.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:37 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:37.569+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:16:59.772595+0000 front 2024-09-19T05:16:59.772482+0000 (oldest deadline 2024-09-19T05:17:27.372759+0000) 2024-09-19T05:17:37.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:37 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:37.569+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-09-19T05:17:10.873848+0000 front 2024-09-19T05:17:10.873916+0000 (oldest deadline 2024-09-19T05:17:32.573687+0000) 2024-09-19T05:17:38.030 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:37 smithi043 podman[108914]: 2024-09-19 05:17:37.657783644 +0000 UTC m=+0.731495195 container remove b3dc20aefb076766043fbf04f267fa980246da2b793596ce345bd42faa201bbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-19T05:17:38.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:37 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:37.956+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:38.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:37 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:37.956+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:38.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:37 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:37.956+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:38.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:37 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:37.956+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.171184+0000 front 2024-09-19T05:16:54.171329+0000 (oldest deadline 2024-09-19T05:17:18.271276+0000) 2024-09-19T05:17:38.406 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:37 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:37.956+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:16:58.772220+0000 front 2024-09-19T05:16:58.772177+0000 (oldest deadline 2024-09-19T05:17:26.972713+0000) 2024-09-19T05:17:38.406 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:37 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:37.956+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-09-19T05:17:08.673539+0000 front 2024-09-19T05:17:08.673631+0000 (oldest deadline 2024-09-19T05:17:32.773449+0000) 2024-09-19T05:17:38.531 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:38 smithi043 podman[109000]: 2024-09-19 05:17:38.225187634 +0000 UTC m=+0.119275638 container create d24b1e98f3c8684c15fc30be4616d347770fdbe4da522907ea798b1bcc7528f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-19T05:17:38.531 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:38 smithi043 podman[109000]: 2024-09-19 05:17:38.133809182 +0000 UTC m=+0.027897195 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:17:38.531 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:38 smithi043 podman[109000]: 2024-09-19 05:17:38.444198976 +0000 UTC m=+0.338286979 container init d24b1e98f3c8684c15fc30be4616d347770fdbe4da522907ea798b1bcc7528f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-19T05:17:38.531 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:38 smithi043 podman[109000]: 2024-09-19 05:17:38.450912798 +0000 UTC m=+0.345000801 container start d24b1e98f3c8684c15fc30be4616d347770fdbe4da522907ea798b1bcc7528f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-19T05:17:38.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:38 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:38.590+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:38.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:38 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:38.590+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:38.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:38 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:38.590+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:38.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:38 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:38.590+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:43.370861+0000 front 2024-09-19T05:16:43.370887+0000 (oldest deadline 2024-09-19T05:17:08.070492+0000) 2024-09-19T05:17:38.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:38 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:38.590+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:50.971438+0000 front 2024-09-19T05:16:50.971421+0000 (oldest deadline 2024-09-19T05:17:16.871470+0000) 2024-09-19T05:17:38.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:38 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:38.590+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:16:59.772595+0000 front 2024-09-19T05:16:59.772482+0000 (oldest deadline 2024-09-19T05:17:27.372759+0000) 2024-09-19T05:17:38.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:38 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:38.590+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-09-19T05:17:10.873848+0000 front 2024-09-19T05:17:10.873916+0000 (oldest deadline 2024-09-19T05:17:32.573687+0000) 2024-09-19T05:17:39.030 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:38 smithi043 podman[109000]: 2024-09-19 05:17:38.535669775 +0000 UTC m=+0.429757808 container attach d24b1e98f3c8684c15fc30be4616d347770fdbe4da522907ea798b1bcc7528f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-19T05:17:39.031 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 19 05:17:38 smithi043 podman[109023]: 2024-09-19 05:17:38.746680525 +0000 UTC m=+0.026505364 container died d24b1e98f3c8684c15fc30be4616d347770fdbe4da522907ea798b1bcc7528f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-19T05:17:39.320 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@osd.9.service' 2024-09-19T05:17:39.395 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:17:39.395 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-19T05:17:39.395 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-19T05:17:39.396 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-1456a810-7643-11ef-bceb-c7b262605968@osd.10 2024-09-19T05:17:39.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:38 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:38.997+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:39.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:38 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:38.997+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:39.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:38 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:38.997+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:39.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:38 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:38.997+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.171184+0000 front 2024-09-19T05:16:54.171329+0000 (oldest deadline 2024-09-19T05:17:18.271276+0000) 2024-09-19T05:17:39.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:38 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:38.997+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:16:58.772220+0000 front 2024-09-19T05:16:58.772177+0000 (oldest deadline 2024-09-19T05:17:26.972713+0000) 2024-09-19T05:17:39.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:38 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:38.997+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-09-19T05:17:08.673539+0000 front 2024-09-19T05:17:08.673631+0000 (oldest deadline 2024-09-19T05:17:32.773449+0000) 2024-09-19T05:17:39.703 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:39 smithi096 systemd[1]: Stopping Ceph osd.10 for 1456a810-7643-11ef-bceb-c7b262605968... 2024-09-19T05:17:39.703 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:39 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:39.596+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:39.704 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:39 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:39.596+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:39.704 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:39 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:39.596+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:39.704 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:39 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:39.596+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:43.370861+0000 front 2024-09-19T05:16:43.370887+0000 (oldest deadline 2024-09-19T05:17:08.070492+0000) 2024-09-19T05:17:39.704 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:39 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:39.596+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:50.971438+0000 front 2024-09-19T05:16:50.971421+0000 (oldest deadline 2024-09-19T05:17:16.871470+0000) 2024-09-19T05:17:39.704 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:39 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:39.596+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:16:59.772595+0000 front 2024-09-19T05:16:59.772482+0000 (oldest deadline 2024-09-19T05:17:27.372759+0000) 2024-09-19T05:17:39.704 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:39 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:39.596+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-09-19T05:17:10.873848+0000 front 2024-09-19T05:17:10.873916+0000 (oldest deadline 2024-09-19T05:17:32.573687+0000) 2024-09-19T05:17:39.704 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:39 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:39.701+0000 7faabfcd8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:17:39.704 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:39 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:39.701+0000 7faabfcd8640 -1 osd.10 99 *** Got signal Terminated *** 2024-09-19T05:17:39.704 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:39 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:39.701+0000 7faabfcd8640 -1 osd.10 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T05:17:40.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:39 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:39.998+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:40.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:39 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:39.998+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:40.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:39 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:39.998+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:40.406 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:39 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:39.998+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.171184+0000 front 2024-09-19T05:16:54.171329+0000 (oldest deadline 2024-09-19T05:17:18.271276+0000) 2024-09-19T05:17:40.406 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:39 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:39.998+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:16:58.772220+0000 front 2024-09-19T05:16:58.772177+0000 (oldest deadline 2024-09-19T05:17:26.972713+0000) 2024-09-19T05:17:40.406 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:39 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:39.998+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-09-19T05:17:08.673539+0000 front 2024-09-19T05:17:08.673631+0000 (oldest deadline 2024-09-19T05:17:32.773449+0000) 2024-09-19T05:17:40.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:40 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:40.636+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:40.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:40 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:40.636+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:40.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:40 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:40.636+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:40.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:40 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:40.637+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:43.370861+0000 front 2024-09-19T05:16:43.370887+0000 (oldest deadline 2024-09-19T05:17:08.070492+0000) 2024-09-19T05:17:40.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:40 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:40.637+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:50.971438+0000 front 2024-09-19T05:16:50.971421+0000 (oldest deadline 2024-09-19T05:17:16.871470+0000) 2024-09-19T05:17:40.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:40 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:40.637+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:16:59.772595+0000 front 2024-09-19T05:16:59.772482+0000 (oldest deadline 2024-09-19T05:17:27.372759+0000) 2024-09-19T05:17:40.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:40 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:40.637+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-09-19T05:17:10.873848+0000 front 2024-09-19T05:17:10.873916+0000 (oldest deadline 2024-09-19T05:17:32.573687+0000) 2024-09-19T05:17:41.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:41 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:41.004+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:41.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:41 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:41.004+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:41.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:41 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:41.004+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:41.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:41 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:41.004+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.171184+0000 front 2024-09-19T05:16:54.171329+0000 (oldest deadline 2024-09-19T05:17:18.271276+0000) 2024-09-19T05:17:41.406 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:41 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:41.004+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:16:58.772220+0000 front 2024-09-19T05:16:58.772177+0000 (oldest deadline 2024-09-19T05:17:26.972713+0000) 2024-09-19T05:17:41.406 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:41 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:41.004+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-09-19T05:17:08.673539+0000 front 2024-09-19T05:17:08.673631+0000 (oldest deadline 2024-09-19T05:17:32.773449+0000) 2024-09-19T05:17:41.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:41 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:41.626+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:41.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:41 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:41.626+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:41.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:41 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:41.626+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:41.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:41 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:41.626+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:43.370861+0000 front 2024-09-19T05:16:43.370887+0000 (oldest deadline 2024-09-19T05:17:08.070492+0000) 2024-09-19T05:17:41.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:41 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:41.626+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:50.971438+0000 front 2024-09-19T05:16:50.971421+0000 (oldest deadline 2024-09-19T05:17:16.871470+0000) 2024-09-19T05:17:41.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:41 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:41.626+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:16:59.772595+0000 front 2024-09-19T05:16:59.772482+0000 (oldest deadline 2024-09-19T05:17:27.372759+0000) 2024-09-19T05:17:41.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:41 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:41.626+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-09-19T05:17:10.873848+0000 front 2024-09-19T05:17:10.873916+0000 (oldest deadline 2024-09-19T05:17:32.573687+0000) 2024-09-19T05:17:42.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:41 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:41.988+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:42.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:41 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:41.988+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:42.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:41 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:41.988+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:42.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:41 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:41.988+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.171184+0000 front 2024-09-19T05:16:54.171329+0000 (oldest deadline 2024-09-19T05:17:18.271276+0000) 2024-09-19T05:17:42.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:41 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:41.988+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:16:58.772220+0000 front 2024-09-19T05:16:58.772177+0000 (oldest deadline 2024-09-19T05:17:26.972713+0000) 2024-09-19T05:17:42.406 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:41 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:41.988+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-09-19T05:17:08.673539+0000 front 2024-09-19T05:17:08.673631+0000 (oldest deadline 2024-09-19T05:17:32.773449+0000) 2024-09-19T05:17:42.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:42 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:42.598+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:42.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:42 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:42.598+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:42.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:42 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:42.598+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:42.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:42 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:42.598+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:43.370861+0000 front 2024-09-19T05:16:43.370887+0000 (oldest deadline 2024-09-19T05:17:08.070492+0000) 2024-09-19T05:17:42.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:42 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:42.598+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:50.971438+0000 front 2024-09-19T05:16:50.971421+0000 (oldest deadline 2024-09-19T05:17:16.871470+0000) 2024-09-19T05:17:42.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:42 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:42.598+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:16:59.772595+0000 front 2024-09-19T05:16:59.772482+0000 (oldest deadline 2024-09-19T05:17:27.372759+0000) 2024-09-19T05:17:42.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:42 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:42.598+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-09-19T05:17:10.873848+0000 front 2024-09-19T05:17:10.873916+0000 (oldest deadline 2024-09-19T05:17:32.573687+0000) 2024-09-19T05:17:43.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:43 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:43.025+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:43.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:43 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:43.025+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:43.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:43 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:43.025+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:43.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:43 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:43.025+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.171184+0000 front 2024-09-19T05:16:54.171329+0000 (oldest deadline 2024-09-19T05:17:18.271276+0000) 2024-09-19T05:17:43.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:43 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:43.025+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:16:58.772220+0000 front 2024-09-19T05:16:58.772177+0000 (oldest deadline 2024-09-19T05:17:26.972713+0000) 2024-09-19T05:17:43.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:43 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:43.025+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-09-19T05:17:08.673539+0000 front 2024-09-19T05:17:08.673631+0000 (oldest deadline 2024-09-19T05:17:32.773449+0000) 2024-09-19T05:17:43.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:43 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:43.571+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:43.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:43 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:43.571+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:43.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:43 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:43.571+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:43.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:43 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:43.571+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:43.370861+0000 front 2024-09-19T05:16:43.370887+0000 (oldest deadline 2024-09-19T05:17:08.070492+0000) 2024-09-19T05:17:43.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:43 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:43.571+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:50.971438+0000 front 2024-09-19T05:16:50.971421+0000 (oldest deadline 2024-09-19T05:17:16.871470+0000) 2024-09-19T05:17:43.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:43 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:43.571+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:16:59.772595+0000 front 2024-09-19T05:16:59.772482+0000 (oldest deadline 2024-09-19T05:17:27.372759+0000) 2024-09-19T05:17:43.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:43 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:43.571+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-09-19T05:17:10.873848+0000 front 2024-09-19T05:17:10.873916+0000 (oldest deadline 2024-09-19T05:17:32.573687+0000) 2024-09-19T05:17:44.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:44 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:44.047+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:44.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:44 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:44.047+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:44.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:44 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:44.047+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:44.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:44 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:44.047+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.171184+0000 front 2024-09-19T05:16:54.171329+0000 (oldest deadline 2024-09-19T05:17:18.271276+0000) 2024-09-19T05:17:44.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:44 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:44.047+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:16:58.772220+0000 front 2024-09-19T05:16:58.772177+0000 (oldest deadline 2024-09-19T05:17:26.972713+0000) 2024-09-19T05:17:44.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:44 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:44.047+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-09-19T05:17:08.673539+0000 front 2024-09-19T05:17:08.673631+0000 (oldest deadline 2024-09-19T05:17:32.773449+0000) 2024-09-19T05:17:44.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:44 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:44.588+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:19.566723+0000 front 2024-09-19T05:16:19.566603+0000 (oldest deadline 2024-09-19T05:16:44.866778+0000) 2024-09-19T05:17:44.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:44 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:44.588+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:28.267804+0000 front 2024-09-19T05:16:28.267841+0000 (oldest deadline 2024-09-19T05:16:52.367973+0000) 2024-09-19T05:17:44.973 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:44 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:44.588+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:35.269116+0000 front 2024-09-19T05:16:35.269017+0000 (oldest deadline 2024-09-19T05:16:59.368967+0000) 2024-09-19T05:17:44.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:44 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:44.588+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-19T05:16:43.370861+0000 front 2024-09-19T05:16:43.370887+0000 (oldest deadline 2024-09-19T05:17:08.070492+0000) 2024-09-19T05:17:44.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:44 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:44.588+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:50.971438+0000 front 2024-09-19T05:16:50.971421+0000 (oldest deadline 2024-09-19T05:17:16.871470+0000) 2024-09-19T05:17:44.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:44 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:44.588+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:16:59.772595+0000 front 2024-09-19T05:16:59.772482+0000 (oldest deadline 2024-09-19T05:17:27.372759+0000) 2024-09-19T05:17:44.974 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:44 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10[59812]: 2024-09-19T05:17:44.588+0000 7faab82e6640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-09-19T05:17:10.873848+0000 front 2024-09-19T05:17:10.873916+0000 (oldest deadline 2024-09-19T05:17:32.573687+0000) 2024-09-19T05:17:45.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:45 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:45.032+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:45.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:45 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:45.032+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:45.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:45 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:45.032+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:45.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:45 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:45.032+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.171184+0000 front 2024-09-19T05:16:54.171329+0000 (oldest deadline 2024-09-19T05:17:18.271276+0000) 2024-09-19T05:17:45.406 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:45 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:45.032+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:16:58.772220+0000 front 2024-09-19T05:16:58.772177+0000 (oldest deadline 2024-09-19T05:17:26.972713+0000) 2024-09-19T05:17:45.406 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:45 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:45.032+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-09-19T05:17:08.673539+0000 front 2024-09-19T05:17:08.673631+0000 (oldest deadline 2024-09-19T05:17:32.773449+0000) 2024-09-19T05:17:45.453 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:45 smithi096 podman[71483]: 2024-09-19 05:17:45.181034494 +0000 UTC m=+5.592639917 container died fce4e0704c19e88f055ab3989de85453153c0171e183086246801a8952b56130 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-19T05:17:45.723 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:45 smithi096 podman[71483]: 2024-09-19 05:17:45.542894567 +0000 UTC m=+5.954499999 container cleanup fce4e0704c19e88f055ab3989de85453153c0171e183086246801a8952b56130 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-19T05:17:45.723 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:45 smithi096 bash[71483]: ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10 2024-09-19T05:17:46.153 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:45 smithi096 podman[71497]: 2024-09-19 05:17:45.810621695 +0000 UTC m=+0.626750999 container remove fce4e0704c19e88f055ab3989de85453153c0171e183086246801a8952b56130 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731) 2024-09-19T05:17:46.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:46 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:46.003+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:46.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:46 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:46.003+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:46.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:46 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:46.003+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:46.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:46 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:46.003+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.171184+0000 front 2024-09-19T05:16:54.171329+0000 (oldest deadline 2024-09-19T05:17:18.271276+0000) 2024-09-19T05:17:46.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:46 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:46.003+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:16:58.772220+0000 front 2024-09-19T05:16:58.772177+0000 (oldest deadline 2024-09-19T05:17:26.972713+0000) 2024-09-19T05:17:46.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:46 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:46.003+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-09-19T05:17:08.673539+0000 front 2024-09-19T05:17:08.673631+0000 (oldest deadline 2024-09-19T05:17:32.773449+0000) 2024-09-19T05:17:46.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:46 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:46.003+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-09-19T05:17:19.774919+0000 front 2024-09-19T05:17:19.774989+0000 (oldest deadline 2024-09-19T05:17:45.674763+0000) 2024-09-19T05:17:46.456 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:46 smithi096 podman[71587]: 2024-09-19 05:17:46.152920766 +0000 UTC m=+0.104909322 container create ee7e194eb6247dcdd4b574a27a8f9fb4edb22246ca7fb4e367c6aabd5a2564fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T05:17:46.456 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:46 smithi096 podman[71587]: 2024-09-19 05:17:46.063760198 +0000 UTC m=+0.015748763 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:17:46.456 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:46 smithi096 podman[71587]: 2024-09-19 05:17:46.380180595 +0000 UTC m=+0.332169151 container init ee7e194eb6247dcdd4b574a27a8f9fb4edb22246ca7fb4e367c6aabd5a2564fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-19T05:17:46.456 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:46 smithi096 podman[71587]: 2024-09-19 05:17:46.38345587 +0000 UTC m=+0.335444426 container start ee7e194eb6247dcdd4b574a27a8f9fb4edb22246ca7fb4e367c6aabd5a2564fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10-deactivate, ceph=True, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-19T05:17:46.456 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:46 smithi096 podman[71587]: 2024-09-19 05:17:46.454909051 +0000 UTC m=+0.406897622 container attach ee7e194eb6247dcdd4b574a27a8f9fb4edb22246ca7fb4e367c6aabd5a2564fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True) 2024-09-19T05:17:46.723 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:46 smithi096 podman[71587]: 2024-09-19 05:17:46.568697757 +0000 UTC m=+0.520686337 container died ee7e194eb6247dcdd4b574a27a8f9fb4edb22246ca7fb4e367c6aabd5a2564fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-19T05:17:47.100 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 05:17:47 smithi096 podman[71611]: 2024-09-19 05:17:47.074689413 +0000 UTC m=+0.499661180 container remove ee7e194eb6247dcdd4b574a27a8f9fb4edb22246ca7fb4e367c6aabd5a2564fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-19T05:17:47.106 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@osd.10.service' 2024-09-19T05:17:47.146 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:17:47.146 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-19T05:17:47.147 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-19T05:17:47.147 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-1456a810-7643-11ef-bceb-c7b262605968@osd.11 2024-09-19T05:17:47.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:47 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:47.018+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:47.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:47 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:47.018+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:47.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:47 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:47.018+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:47.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:47 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:47.018+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.171184+0000 front 2024-09-19T05:16:54.171329+0000 (oldest deadline 2024-09-19T05:17:18.271276+0000) 2024-09-19T05:17:47.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:47 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:47.018+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:16:58.772220+0000 front 2024-09-19T05:16:58.772177+0000 (oldest deadline 2024-09-19T05:17:26.972713+0000) 2024-09-19T05:17:47.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:47 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:47.018+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-09-19T05:17:08.673539+0000 front 2024-09-19T05:17:08.673631+0000 (oldest deadline 2024-09-19T05:17:32.773449+0000) 2024-09-19T05:17:47.406 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:47 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:47.018+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-09-19T05:17:19.774919+0000 front 2024-09-19T05:17:19.774989+0000 (oldest deadline 2024-09-19T05:17:45.674763+0000) 2024-09-19T05:17:47.406 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:47 smithi151 systemd[1]: Stopping Ceph osd.11 for 1456a810-7643-11ef-bceb-c7b262605968... 2024-09-19T05:17:47.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:47 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:47.461+0000 7fda8c5d8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:17:47.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:47 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:47.461+0000 7fda8c5d8640 -1 osd.11 99 *** Got signal Terminated *** 2024-09-19T05:17:47.905 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:47 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:47.461+0000 7fda8c5d8640 -1 osd.11 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T05:17:48.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:48 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:48.052+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:48.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:48 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:48.052+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:48.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:48 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:48.052+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:48.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:48 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:48.052+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.171184+0000 front 2024-09-19T05:16:54.171329+0000 (oldest deadline 2024-09-19T05:17:18.271276+0000) 2024-09-19T05:17:48.406 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:48 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:48.052+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:16:58.772220+0000 front 2024-09-19T05:16:58.772177+0000 (oldest deadline 2024-09-19T05:17:26.972713+0000) 2024-09-19T05:17:48.406 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:48 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:48.052+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-09-19T05:17:08.673539+0000 front 2024-09-19T05:17:08.673631+0000 (oldest deadline 2024-09-19T05:17:32.773449+0000) 2024-09-19T05:17:48.406 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:48 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:48.052+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-09-19T05:17:19.774919+0000 front 2024-09-19T05:17:19.774989+0000 (oldest deadline 2024-09-19T05:17:45.674763+0000) 2024-09-19T05:17:49.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:49 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:49.046+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:49.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:49 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:49.046+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:49.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:49 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:49.046+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:49.406 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:49 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:49.046+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.171184+0000 front 2024-09-19T05:16:54.171329+0000 (oldest deadline 2024-09-19T05:17:18.271276+0000) 2024-09-19T05:17:49.406 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:49 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:49.046+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:16:58.772220+0000 front 2024-09-19T05:16:58.772177+0000 (oldest deadline 2024-09-19T05:17:26.972713+0000) 2024-09-19T05:17:49.406 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:49 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:49.046+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-09-19T05:17:08.673539+0000 front 2024-09-19T05:17:08.673631+0000 (oldest deadline 2024-09-19T05:17:32.773449+0000) 2024-09-19T05:17:49.406 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:49 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:49.046+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-09-19T05:17:19.774919+0000 front 2024-09-19T05:17:19.774989+0000 (oldest deadline 2024-09-19T05:17:45.674763+0000) 2024-09-19T05:17:50.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:50 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:50.009+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:50.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:50 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:50.009+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:50.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:50 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:50.009+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:50.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:50 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:50.009+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.171184+0000 front 2024-09-19T05:16:54.171329+0000 (oldest deadline 2024-09-19T05:17:18.271276+0000) 2024-09-19T05:17:50.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:50 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:50.009+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:16:58.772220+0000 front 2024-09-19T05:16:58.772177+0000 (oldest deadline 2024-09-19T05:17:26.972713+0000) 2024-09-19T05:17:50.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:50 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:50.009+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-09-19T05:17:08.673539+0000 front 2024-09-19T05:17:08.673631+0000 (oldest deadline 2024-09-19T05:17:32.773449+0000) 2024-09-19T05:17:50.406 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:50 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:50.009+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-09-19T05:17:19.774919+0000 front 2024-09-19T05:17:19.774989+0000 (oldest deadline 2024-09-19T05:17:45.674763+0000) 2024-09-19T05:17:51.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:51 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:51.001+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:51.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:51 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:51.001+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:51.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:51 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:51.001+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:51.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:51 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:51.001+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.171184+0000 front 2024-09-19T05:16:54.171329+0000 (oldest deadline 2024-09-19T05:17:18.271276+0000) 2024-09-19T05:17:51.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:51 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:51.001+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:16:58.772220+0000 front 2024-09-19T05:16:58.772177+0000 (oldest deadline 2024-09-19T05:17:26.972713+0000) 2024-09-19T05:17:51.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:51 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:51.001+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-09-19T05:17:08.673539+0000 front 2024-09-19T05:17:08.673631+0000 (oldest deadline 2024-09-19T05:17:32.773449+0000) 2024-09-19T05:17:51.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:51 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:51.001+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-09-19T05:17:19.774919+0000 front 2024-09-19T05:17:19.774989+0000 (oldest deadline 2024-09-19T05:17:45.674763+0000) 2024-09-19T05:17:51.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:51 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:51.001+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6820 osd.8 since back 2024-09-19T05:17:25.675262+0000 front 2024-09-19T05:17:25.675278+0000 (oldest deadline 2024-09-19T05:17:50.975278+0000) 2024-09-19T05:17:52.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:52 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:52.051+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-19T05:16:18.764445+0000 front 2024-09-19T05:16:18.764556+0000 (oldest deadline 2024-09-19T05:16:42.264606+0000) 2024-09-19T05:17:52.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:52 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:52.051+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T05:16:29.766984+0000 front 2024-09-19T05:16:29.766969+0000 (oldest deadline 2024-09-19T05:16:54.466984+0000) 2024-09-19T05:17:52.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:52 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:52.051+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T05:16:37.368024+0000 front 2024-09-19T05:16:37.368072+0000 (oldest deadline 2024-09-19T05:17:02.668249+0000) 2024-09-19T05:17:52.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:52 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:52.051+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T05:16:54.171184+0000 front 2024-09-19T05:16:54.171329+0000 (oldest deadline 2024-09-19T05:17:18.271276+0000) 2024-09-19T05:17:52.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:52 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:52.051+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T05:16:58.772220+0000 front 2024-09-19T05:16:58.772177+0000 (oldest deadline 2024-09-19T05:17:26.972713+0000) 2024-09-19T05:17:52.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:52 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:52.051+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-09-19T05:17:08.673539+0000 front 2024-09-19T05:17:08.673631+0000 (oldest deadline 2024-09-19T05:17:32.773449+0000) 2024-09-19T05:17:52.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:52 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:52.051+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-09-19T05:17:19.774919+0000 front 2024-09-19T05:17:19.774989+0000 (oldest deadline 2024-09-19T05:17:45.674763+0000) 2024-09-19T05:17:52.406 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:52 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11[60631]: 2024-09-19T05:17:52.051+0000 7fda84be6640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.151:6820 osd.8 since back 2024-09-19T05:17:25.675262+0000 front 2024-09-19T05:17:25.675278+0000 (oldest deadline 2024-09-19T05:17:50.975278+0000) 2024-09-19T05:17:53.073 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:52 smithi151 podman[72287]: 2024-09-19 05:17:52.807363524 +0000 UTC m=+5.461551956 container died bb8d55cff5fc812fb9a54c5868ff5c712ff79a95e71da0137532ff818c533441 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-19T05:17:53.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:53 smithi151 podman[72287]: 2024-09-19 05:17:53.17098226 +0000 UTC m=+5.825170683 container cleanup bb8d55cff5fc812fb9a54c5868ff5c712ff79a95e71da0137532ff818c533441 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11, RELEASE=squid-2f599a4, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-19T05:17:53.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:53 smithi151 bash[72287]: ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11 2024-09-19T05:17:53.788 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:53 smithi151 podman[72304]: 2024-09-19 05:17:53.473034822 +0000 UTC m=+0.655870486 container remove bb8d55cff5fc812fb9a54c5868ff5c712ff79a95e71da0137532ff818c533441 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T05:17:54.094 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:53 smithi151 podman[72389]: 2024-09-19 05:17:53.690204386 +0000 UTC m=+0.016070702 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T05:17:54.094 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:53 smithi151 podman[72389]: 2024-09-19 05:17:53.798870682 +0000 UTC m=+0.124736987 container create c3d6db5e58667d162e33ad778db66c170c552002b97a85102d831e7d7a0e1418 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True) 2024-09-19T05:17:54.094 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:54 smithi151 podman[72389]: 2024-09-19 05:17:54.010014721 +0000 UTC m=+0.335881028 container init c3d6db5e58667d162e33ad778db66c170c552002b97a85102d831e7d7a0e1418 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11-deactivate, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-19T05:17:54.095 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:54 smithi151 podman[72389]: 2024-09-19 05:17:54.013364031 +0000 UTC m=+0.339230341 container start c3d6db5e58667d162e33ad778db66c170c552002b97a85102d831e7d7a0e1418 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-19T05:17:54.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:54 smithi151 podman[72389]: 2024-09-19 05:17:54.093333543 +0000 UTC m=+0.419199852 container attach c3d6db5e58667d162e33ad778db66c170c552002b97a85102d831e7d7a0e1418 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:17:54.405 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:54 smithi151 podman[72389]: 2024-09-19 05:17:54.201731491 +0000 UTC m=+0.527597798 container died c3d6db5e58667d162e33ad778db66c170c552002b97a85102d831e7d7a0e1418 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-19T05:17:54.779 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 05:17:54 smithi151 podman[72413]: 2024-09-19 05:17:54.756441545 +0000 UTC m=+0.549425010 container remove c3d6db5e58667d162e33ad778db66c170c552002b97a85102d831e7d7a0e1418 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-osd-11-deactivate, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-19T05:17:54.784 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@osd.11.service' 2024-09-19T05:17:54.824 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:17:54.824 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-19T05:17:54.824 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-19T05:17:54.824 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-1456a810-7643-11ef-bceb-c7b262605968@mds.a 2024-09-19T05:17:55.237 INFO:journalctl@ceph.mds.a.smithi043.stdout:Sep 19 05:17:54 smithi043 systemd[1]: Stopping Ceph mds.a for 1456a810-7643-11ef-bceb-c7b262605968... 2024-09-19T05:17:55.530 INFO:journalctl@ceph.mds.a.smithi043.stdout:Sep 19 05:17:55 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-a[69678]: 2024-09-19T05:17:55.234+0000 7fab63a00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:17:55.530 INFO:journalctl@ceph.mds.a.smithi043.stdout:Sep 19 05:17:55 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-a[69678]: 2024-09-19T05:17:55.234+0000 7fab63a00640 -1 mds.a *** got signal Terminated *** 2024-09-19T05:17:58.918 INFO:journalctl@ceph.mds.a.smithi043.stdout:Sep 19 05:17:58 smithi043 podman[109102]: 2024-09-19 05:17:58.620563661 +0000 UTC m=+3.504688347 container died d6a90fee2ab115c5f79e1f4af0de8da4550119b6cadb392c90dc37da2f4a72f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-a, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4) 2024-09-19T05:17:59.276 INFO:journalctl@ceph.mds.a.smithi043.stdout:Sep 19 05:17:59 smithi043 podman[109102]: 2024-09-19 05:17:59.014875129 +0000 UTC m=+3.898999825 container cleanup d6a90fee2ab115c5f79e1f4af0de8da4550119b6cadb392c90dc37da2f4a72f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-a, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T05:17:59.277 INFO:journalctl@ceph.mds.a.smithi043.stdout:Sep 19 05:17:59 smithi043 bash[109102]: ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-a 2024-09-19T05:17:59.459 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@mds.a.service' 2024-09-19T05:17:59.533 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:17:59.534 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-19T05:17:59.534 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-19T05:17:59.534 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-1456a810-7643-11ef-bceb-c7b262605968@mds.d 2024-09-19T05:17:59.940 INFO:journalctl@ceph.mds.d.smithi043.stdout:Sep 19 05:17:59 smithi043 systemd[1]: Stopping Ceph mds.d for 1456a810-7643-11ef-bceb-c7b262605968... 2024-09-19T05:18:00.281 INFO:journalctl@ceph.mds.d.smithi043.stdout:Sep 19 05:17:59 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-d[68495]: 2024-09-19T05:17:59.937+0000 7f2e03800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:18:00.281 INFO:journalctl@ceph.mds.d.smithi043.stdout:Sep 19 05:17:59 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-d[68495]: 2024-09-19T05:17:59.937+0000 7f2e03800640 -1 mds.d *** got signal Terminated *** 2024-09-19T05:18:02.246 INFO:journalctl@ceph.mds.d.smithi043.stdout:Sep 19 05:18:01 smithi043 podman[109235]: 2024-09-19 05:18:01.974176101 +0000 UTC m=+2.166551804 container died d91e4fed3c4c6c4ff379235adfb37e7906daf1fc90de45d45116b44893cf90ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-d, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-19T05:18:02.531 INFO:journalctl@ceph.mds.d.smithi043.stdout:Sep 19 05:18:02 smithi043 podman[109235]: 2024-09-19 05:18:02.418941911 +0000 UTC m=+2.611317613 container cleanup d91e4fed3c4c6c4ff379235adfb37e7906daf1fc90de45d45116b44893cf90ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-d, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-19T05:18:02.531 INFO:journalctl@ceph.mds.d.smithi043.stdout:Sep 19 05:18:02 smithi043 bash[109235]: ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-d 2024-09-19T05:18:02.903 INFO:journalctl@ceph.mds.d.smithi043.stdout:Sep 19 05:18:02 smithi043 podman[109247]: 2024-09-19 05:18:02.628743342 +0000 UTC m=+0.643959058 container remove d91e4fed3c4c6c4ff379235adfb37e7906daf1fc90de45d45116b44893cf90ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-d, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-19T05:18:02.912 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@mds.d.service' 2024-09-19T05:18:02.982 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:18:02.983 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-19T05:18:02.983 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-19T05:18:02.983 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-1456a810-7643-11ef-bceb-c7b262605968@mds.g 2024-09-19T05:18:03.280 INFO:journalctl@ceph.mds.g.smithi043.stdout:Sep 19 05:18:03 smithi043 systemd[1]: Stopping Ceph mds.g for 1456a810-7643-11ef-bceb-c7b262605968... 2024-09-19T05:18:03.780 INFO:journalctl@ceph.mds.g.smithi043.stdout:Sep 19 05:18:03 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-g[69296]: 2024-09-19T05:18:03.433+0000 7f47b0a00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:18:03.781 INFO:journalctl@ceph.mds.g.smithi043.stdout:Sep 19 05:18:03 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-g[69296]: 2024-09-19T05:18:03.433+0000 7f47b0a00640 -1 mds.g *** got signal Terminated *** 2024-09-19T05:18:06.931 INFO:journalctl@ceph.mds.g.smithi043.stdout:Sep 19 05:18:06 smithi043 podman[109358]: 2024-09-19 05:18:06.623189126 +0000 UTC m=+3.311123829 container died c1495dc06331c24efa0347774080a4dd469b732e38411afe17185eed9faae0b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:18:07.280 INFO:journalctl@ceph.mds.g.smithi043.stdout:Sep 19 05:18:07 smithi043 podman[109358]: 2024-09-19 05:18:07.130995547 +0000 UTC m=+3.818930247 container cleanup c1495dc06331c24efa0347774080a4dd469b732e38411afe17185eed9faae0b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-g, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T05:18:07.281 INFO:journalctl@ceph.mds.g.smithi043.stdout:Sep 19 05:18:07 smithi043 bash[109358]: ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-g 2024-09-19T05:18:07.547 INFO:journalctl@ceph.mds.g.smithi043.stdout:Sep 19 05:18:07 smithi043 podman[109371]: 2024-09-19 05:18:07.399460889 +0000 UTC m=+0.766083210 container remove c1495dc06331c24efa0347774080a4dd469b732e38411afe17185eed9faae0b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-g, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-19T05:18:07.570 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@mds.g.service' 2024-09-19T05:18:07.639 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:18:07.640 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-19T05:18:07.640 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-19T05:18:07.640 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-1456a810-7643-11ef-bceb-c7b262605968@mds.j 2024-09-19T05:18:08.031 INFO:journalctl@ceph.mds.j.smithi043.stdout:Sep 19 05:18:07 smithi043 systemd[1]: Stopping Ceph mds.j for 1456a810-7643-11ef-bceb-c7b262605968... 2024-09-19T05:18:08.531 INFO:journalctl@ceph.mds.j.smithi043.stdout:Sep 19 05:18:08 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-j[68120]: 2024-09-19T05:18:08.070+0000 7f56c6e00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:18:08.531 INFO:journalctl@ceph.mds.j.smithi043.stdout:Sep 19 05:18:08 smithi043 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-j[68120]: 2024-09-19T05:18:08.070+0000 7f56c6e00640 -1 mds.j *** got signal Terminated *** 2024-09-19T05:18:10.059 INFO:journalctl@ceph.mds.j.smithi043.stdout:Sep 19 05:18:09 smithi043 podman[109479]: 2024-09-19 05:18:09.807653947 +0000 UTC m=+1.885597053 container died 80bca36d1ddde68859eade052168b5a31ad0dfd4cc5f442cd9a9ad2dbad1d6aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-j, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-19T05:18:10.471 INFO:journalctl@ceph.mds.j.smithi043.stdout:Sep 19 05:18:10 smithi043 podman[109479]: 2024-09-19 05:18:10.269245638 +0000 UTC m=+2.347188743 container cleanup 80bca36d1ddde68859eade052168b5a31ad0dfd4cc5f442cd9a9ad2dbad1d6aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-j, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:18:10.471 INFO:journalctl@ceph.mds.j.smithi043.stdout:Sep 19 05:18:10 smithi043 bash[109479]: ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-j 2024-09-19T05:18:10.727 INFO:journalctl@ceph.mds.j.smithi043.stdout:Sep 19 05:18:10 smithi043 podman[109501]: 2024-09-19 05:18:10.470181664 +0000 UTC m=+0.653119324 container remove 80bca36d1ddde68859eade052168b5a31ad0dfd4cc5f442cd9a9ad2dbad1d6aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:18:10.736 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@mds.j.service' 2024-09-19T05:18:10.809 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:18:10.809 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-19T05:18:10.809 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-19T05:18:10.810 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-1456a810-7643-11ef-bceb-c7b262605968@mds.b 2024-09-19T05:18:11.108 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 05:18:10 smithi096 systemd[1]: Stopping Ceph mds.b for 1456a810-7643-11ef-bceb-c7b262605968... 2024-09-19T05:18:11.363 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 05:18:11 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-b[65164]: 2024-09-19T05:18:11.105+0000 7f5a46b0e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:18:11.363 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 05:18:11 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-b[65164]: 2024-09-19T05:18:11.105+0000 7f5a46b0e640 -1 mds.b *** got signal Terminated *** 2024-09-19T05:18:14.866 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 05:18:14 smithi096 podman[71694]: 2024-09-19 05:18:14.581995804 +0000 UTC m=+3.572377728 container died 1102336ec1069b3325bc3a082bdb99c2d8b7f553d7840070373005006062688c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-b, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-19T05:18:14.866 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 05:18:14 smithi096 podman[71694]: 2024-09-19 05:18:14.864470012 +0000 UTC m=+3.854851939 container cleanup 1102336ec1069b3325bc3a082bdb99c2d8b7f553d7840070373005006062688c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-b, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-19T05:18:15.116 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 05:18:14 smithi096 bash[71694]: ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-b 2024-09-19T05:18:15.248 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@mds.b.service' 2024-09-19T05:18:15.283 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:18:15.284 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-19T05:18:15.284 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-19T05:18:15.284 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-1456a810-7643-11ef-bceb-c7b262605968@mds.e 2024-09-19T05:18:15.473 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 19 05:18:15 smithi096 systemd[1]: Stopping Ceph mds.e for 1456a810-7643-11ef-bceb-c7b262605968... 2024-09-19T05:18:15.973 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 19 05:18:15 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-e[64565]: 2024-09-19T05:18:15.601+0000 7f4c9eedd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:18:15.973 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 19 05:18:15 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-e[64565]: 2024-09-19T05:18:15.601+0000 7f4c9eedd640 -1 mds.e *** got signal Terminated *** 2024-09-19T05:18:18.862 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 19 05:18:18 smithi096 podman[71828]: 2024-09-19 05:18:18.582148711 +0000 UTC m=+3.081551133 container died 06f3d3813f3932e9b4d91d0172fc96dc1e180ed5de878eff54d45b7862a44c48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-e, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-19T05:18:19.203 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 19 05:18:18 smithi096 podman[71828]: 2024-09-19 05:18:18.942725498 +0000 UTC m=+3.442127914 container cleanup 06f3d3813f3932e9b4d91d0172fc96dc1e180ed5de878eff54d45b7862a44c48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0) 2024-09-19T05:18:19.203 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 19 05:18:18 smithi096 bash[71828]: ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-e 2024-09-19T05:18:19.376 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@mds.e.service' 2024-09-19T05:18:19.410 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:18:19.410 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-19T05:18:19.410 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-19T05:18:19.410 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-1456a810-7643-11ef-bceb-c7b262605968@mds.h 2024-09-19T05:18:19.723 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 19 05:18:19 smithi096 systemd[1]: Stopping Ceph mds.h for 1456a810-7643-11ef-bceb-c7b262605968... 2024-09-19T05:18:20.223 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 19 05:18:19 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-h[64865]: 2024-09-19T05:18:19.729+0000 7f1825423640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:18:20.223 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 19 05:18:19 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-h[64865]: 2024-09-19T05:18:19.729+0000 7f1825423640 -1 mds.h *** got signal Terminated *** 2024-09-19T05:18:22.888 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 19 05:18:22 smithi096 podman[71949]: 2024-09-19 05:18:22.58505279 +0000 UTC m=+2.954615776 container died 3bd8c04d61ff9f3d9ac7ab0c52ea54f582e55d3ac025a72c58ee5ddcc5b7676e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-h, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:18:22.888 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 19 05:18:22 smithi096 podman[71949]: 2024-09-19 05:18:22.887331307 +0000 UTC m=+3.256894276 container cleanup 3bd8c04d61ff9f3d9ac7ab0c52ea54f582e55d3ac025a72c58ee5ddcc5b7676e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-h, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731) 2024-09-19T05:18:23.223 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 19 05:18:22 smithi096 bash[71949]: ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-h 2024-09-19T05:18:23.223 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 19 05:18:23 smithi096 podman[71961]: 2024-09-19 05:18:23.130037369 +0000 UTC m=+0.537991782 container remove 3bd8c04d61ff9f3d9ac7ab0c52ea54f582e55d3ac025a72c58ee5ddcc5b7676e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:18:23.272 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@mds.h.service' 2024-09-19T05:18:23.305 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:18:23.305 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-19T05:18:23.306 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-19T05:18:23.306 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-1456a810-7643-11ef-bceb-c7b262605968@mds.k 2024-09-19T05:18:23.617 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 19 05:18:23 smithi096 systemd[1]: Stopping Ceph mds.k for 1456a810-7643-11ef-bceb-c7b262605968... 2024-09-19T05:18:23.973 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 19 05:18:23 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-k[64252]: 2024-09-19T05:18:23.615+0000 7f81bd818640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:18:23.973 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 19 05:18:23 smithi096 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-k[64252]: 2024-09-19T05:18:23.615+0000 7f81bd818640 -1 mds.k *** got signal Terminated *** 2024-09-19T05:18:26.875 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 19 05:18:26 smithi096 podman[72070]: 2024-09-19 05:18:26.587071917 +0000 UTC m=+3.069186398 container died 843ac97aa1db88033d069b41a1d43f3dc11f82d507562afc9dd5c431e03918d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-k, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-19T05:18:27.209 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 19 05:18:26 smithi096 podman[72070]: 2024-09-19 05:18:26.957408686 +0000 UTC m=+3.439523173 container cleanup 843ac97aa1db88033d069b41a1d43f3dc11f82d507562afc9dd5c431e03918d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-k, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-19T05:18:27.209 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 19 05:18:26 smithi096 bash[72070]: ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-k 2024-09-19T05:18:27.342 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@mds.k.service' 2024-09-19T05:18:27.373 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:18:27.374 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-19T05:18:27.374 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-19T05:18:27.374 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-1456a810-7643-11ef-bceb-c7b262605968@mds.c 2024-09-19T05:18:27.655 INFO:journalctl@ceph.mds.c.smithi151.stdout:Sep 19 05:18:27 smithi151 systemd[1]: Stopping Ceph mds.c for 1456a810-7643-11ef-bceb-c7b262605968... 2024-09-19T05:18:28.055 INFO:journalctl@ceph.mds.c.smithi151.stdout:Sep 19 05:18:27 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-c[65488]: 2024-09-19T05:18:27.710+0000 7f05739bd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:18:28.055 INFO:journalctl@ceph.mds.c.smithi151.stdout:Sep 19 05:18:27 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-c[65488]: 2024-09-19T05:18:27.710+0000 7f05739bd640 -1 mds.c *** got signal Terminated *** 2024-09-19T05:18:30.905 INFO:journalctl@ceph.mds.c.smithi151.stdout:Sep 19 05:18:30 smithi151 podman[72493]: 2024-09-19 05:18:30.60130827 +0000 UTC m=+3.034865887 container died b2c85bce4d908712ab79c6be5738175ffb5194f947d9065bc9ccbcd7023a88c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-c, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-19T05:18:31.359 INFO:journalctl@ceph.mds.c.smithi151.stdout:Sep 19 05:18:31 smithi151 podman[72493]: 2024-09-19 05:18:31.090005909 +0000 UTC m=+3.523563524 container cleanup b2c85bce4d908712ab79c6be5738175ffb5194f947d9065bc9ccbcd7023a88c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-c, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, ceph=True) 2024-09-19T05:18:31.359 INFO:journalctl@ceph.mds.c.smithi151.stdout:Sep 19 05:18:31 smithi151 bash[72493]: ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-c 2024-09-19T05:18:31.526 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@mds.c.service' 2024-09-19T05:18:31.563 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:18:31.563 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-19T05:18:31.563 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-19T05:18:31.563 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-1456a810-7643-11ef-bceb-c7b262605968@mds.f 2024-09-19T05:18:31.888 INFO:journalctl@ceph.mds.f.smithi151.stdout:Sep 19 05:18:31 smithi151 systemd[1]: Stopping Ceph mds.f for 1456a810-7643-11ef-bceb-c7b262605968... 2024-09-19T05:18:32.155 INFO:journalctl@ceph.mds.f.smithi151.stdout:Sep 19 05:18:31 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-f[65183]: 2024-09-19T05:18:31.886+0000 7fd192768640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:18:32.155 INFO:journalctl@ceph.mds.f.smithi151.stdout:Sep 19 05:18:31 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-f[65183]: 2024-09-19T05:18:31.886+0000 7fd192768640 -1 mds.f *** got signal Terminated *** 2024-09-19T05:18:34.846 INFO:journalctl@ceph.mds.f.smithi151.stdout:Sep 19 05:18:34 smithi151 podman[72625]: 2024-09-19 05:18:34.590926505 +0000 UTC m=+2.814671928 container died 3bdb1ef911fa3ac241c97ee037bf051fa668866ee0287b3ea03ed09fb7bdee00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:18:35.155 INFO:journalctl@ceph.mds.f.smithi151.stdout:Sep 19 05:18:34 smithi151 podman[72625]: 2024-09-19 05:18:34.945151392 +0000 UTC m=+3.168896814 container cleanup 3bdb1ef911fa3ac241c97ee037bf051fa668866ee0287b3ea03ed09fb7bdee00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4) 2024-09-19T05:18:35.155 INFO:journalctl@ceph.mds.f.smithi151.stdout:Sep 19 05:18:34 smithi151 bash[72625]: ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-f 2024-09-19T05:18:35.378 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@mds.f.service' 2024-09-19T05:18:35.411 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:18:35.412 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-19T05:18:35.412 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-19T05:18:35.412 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-1456a810-7643-11ef-bceb-c7b262605968@mds.i 2024-09-19T05:18:35.733 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 05:18:35 smithi151 systemd[1]: Stopping Ceph mds.i for 1456a810-7643-11ef-bceb-c7b262605968... 2024-09-19T05:18:36.155 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 05:18:35 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-i[65787]: 2024-09-19T05:18:35.732+0000 7f098eec8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:18:36.155 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 05:18:35 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-i[65787]: 2024-09-19T05:18:35.732+0000 7f098eec8640 -1 mds.i *** got signal Terminated *** 2024-09-19T05:18:38.876 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 05:18:38 smithi151 podman[72749]: 2024-09-19 05:18:38.590446556 +0000 UTC m=+2.965179174 container died 17ab155834124caaf0e12d97e98808ec5bfb213853248f1f254006a4fb83b975 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-i, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-2f599a4) 2024-09-19T05:18:39.155 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 05:18:38 smithi151 podman[72749]: 2024-09-19 05:18:38.966877565 +0000 UTC m=+3.341610181 container cleanup 17ab155834124caaf0e12d97e98808ec5bfb213853248f1f254006a4fb83b975 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-i, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-19T05:18:39.155 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 05:18:38 smithi151 bash[72749]: ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-i 2024-09-19T05:18:39.344 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@mds.i.service' 2024-09-19T05:18:39.376 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:18:39.376 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-19T05:18:39.376 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-19T05:18:39.376 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-1456a810-7643-11ef-bceb-c7b262605968@mds.l 2024-09-19T05:18:39.655 INFO:journalctl@ceph.mds.l.smithi151.stdout:Sep 19 05:18:39 smithi151 systemd[1]: Stopping Ceph mds.l for 1456a810-7643-11ef-bceb-c7b262605968... 2024-09-19T05:18:40.155 INFO:journalctl@ceph.mds.l.smithi151.stdout:Sep 19 05:18:39 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-l[64870]: 2024-09-19T05:18:39.695+0000 7f1f6669b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T05:18:40.155 INFO:journalctl@ceph.mds.l.smithi151.stdout:Sep 19 05:18:39 smithi151 ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-l[64870]: 2024-09-19T05:18:39.695+0000 7f1f6669b640 -1 mds.l *** got signal Terminated *** 2024-09-19T05:18:42.848 INFO:journalctl@ceph.mds.l.smithi151.stdout:Sep 19 05:18:42 smithi151 podman[72871]: 2024-09-19 05:18:42.584637134 +0000 UTC m=+2.993930952 container died f4341073aa7ac136530694e85b28ad8df89b0ea7d5cf7080676490e82081d478 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T05:18:43.155 INFO:journalctl@ceph.mds.l.smithi151.stdout:Sep 19 05:18:42 smithi151 podman[72871]: 2024-09-19 05:18:42.946818363 +0000 UTC m=+3.356112168 container cleanup f4341073aa7ac136530694e85b28ad8df89b0ea7d5cf7080676490e82081d478 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-l, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T05:18:43.155 INFO:journalctl@ceph.mds.l.smithi151.stdout:Sep 19 05:18:42 smithi151 bash[72871]: ceph-1456a810-7643-11ef-bceb-c7b262605968-mds-l 2024-09-19T05:18:43.374 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1456a810-7643-11ef-bceb-c7b262605968@mds.l.service' 2024-09-19T05:18:43.405 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T05:18:43.406 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-19T05:18:43.406 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1456a810-7643-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T05:18:43.689 INFO:teuthology.orchestra.run.smithi043.stdout:Deleting cluster with fsid: 1456a810-7643-11ef-bceb-c7b262605968 2024-09-19T05:18:47.769 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1456a810-7643-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T05:18:47.968 INFO:teuthology.orchestra.run.smithi096.stdout:Deleting cluster with fsid: 1456a810-7643-11ef-bceb-c7b262605968 2024-09-19T05:18:50.426 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1456a810-7643-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T05:18:50.628 INFO:teuthology.orchestra.run.smithi151.stdout:Deleting cluster with fsid: 1456a810-7643-11ef-bceb-c7b262605968 2024-09-19T05:18:53.117 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T05:18:53.182 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T05:18:53.215 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T05:18:53.242 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-19T05:18:53.244 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911648/remote/smithi043/crash 2024-09-19T05:18:53.245 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/crash -- . 2024-09-19T05:18:53.312 INFO:teuthology.orchestra.run.smithi043.stderr:tar: /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T05:18:53.312 INFO:teuthology.orchestra.run.smithi043.stderr:tar: Error is not recoverable: exiting now 2024-09-19T05:18:53.315 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911648/remote/smithi096/crash 2024-09-19T05:18:53.316 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/crash -- . 2024-09-19T05:18:53.357 INFO:teuthology.orchestra.run.smithi096.stderr:tar: /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T05:18:53.357 INFO:teuthology.orchestra.run.smithi096.stderr:tar: Error is not recoverable: exiting now 2024-09-19T05:18:53.360 DEBUG:teuthology.misc:Transferring archived files from smithi151:/var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911648/remote/smithi151/crash 2024-09-19T05:18:53.361 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/crash -- . 2024-09-19T05:18:53.430 INFO:teuthology.orchestra.run.smithi151.stderr:tar: /var/lib/ceph/1456a810-7643-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T05:18:53.430 INFO:teuthology.orchestra.run.smithi151.stderr:tar: Error is not recoverable: exiting now 2024-09-19T05:18:53.431 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-19T05:18:53.432 DEBUG:teuthology.orchestra.run.smithi043:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-19T05:18:53.512 INFO:tasks.cephadm:Compressing logs... 2024-09-19T05:18:53.513 DEBUG:teuthology.orchestra.run.smithi043:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T05:18:53.517 DEBUG:teuthology.orchestra.run.smithi096:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T05:18:53.520 DEBUG:teuthology.orchestra.run.smithi151:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T05:18:53.548 INFO:teuthology.orchestra.run.smithi151.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T05:18:53.548 INFO:teuthology.orchestra.run.smithi151.stderr:: No such file or directory 2024-09-19T05:18:53.548 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T05:18:53.548 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mds.i.log 2024-09-19T05:18:53.548 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mon.c.log 2024-09-19T05:18:53.548 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T05:18:53.548 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-osd.11.log 2024-09-19T05:18:53.548 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mon.c.log: /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-osd.2.log 2024-09-19T05:18:53.549 INFO:teuthology.orchestra.run.smithi151.stderr: 81.9% -- replaced with /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T05:18:53.549 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-osd.5.log 2024-09-19T05:18:53.549 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5/var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-osd.2.log: --verbose -- /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph.log 2024-09-19T05:18:53.549 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mds.l.log 2024-09-19T05:18:53.549 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph.log: 90.5% -- replaced with /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T05:18:53.551 INFO:teuthology.orchestra.run.smithi096.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-osd.10.log 2024-09-19T05:18:53.551 INFO:teuthology.orchestra.run.smithi096.stderr:: No such file or directory 2024-09-19T05:18:53.551 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-osd.1.log 2024-09-19T05:18:53.551 INFO:teuthology.orchestra.run.smithi096.stderr:gzip/var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-osd.10.log: -5 --verbose -- /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-osd.7.log 2024-09-19T05:18:53.551 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T05:18:53.551 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mds.b.log 2024-09-19T05:18:53.551 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mds.k.log 2024-09-19T05:18:53.551 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-volume.log: /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-osd.4.log 2024-09-19T05:18:53.555 INFO:teuthology.orchestra.run.smithi151.stderr: 87.4% -- replaced with /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T05:18:53.555 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-osd.8.log 2024-09-19T05:18:53.555 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mgr.z.log 2024-09-19T05:18:53.557 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T05:18:53.557 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-osd.4.log: /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T05:18:53.557 INFO:teuthology.orchestra.run.smithi096.stderr: 90.4% -- replaced with /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T05:18:53.557 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mgr.y.log 2024-09-19T05:18:53.557 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph.cephadm.log: 82.0% -- replaced with /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T05:18:53.558 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph.log 2024-09-19T05:18:53.559 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-osd.8.log: 90.8% -- replaced with /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T05:18:53.564 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mgr.y.log: 90.8% -- replaced with /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T05:18:53.564 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mds.h.log 2024-09-19T05:18:53.566 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mds.c.log 2024-09-19T05:18:53.567 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph.log: 91.9% 87.4% -- replaced with /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mgr.y.log.gz 2024-09-19T05:18:53.567 INFO:teuthology.orchestra.run.smithi096.stderr: -- replaced with /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T05:18:53.568 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mon.b.log 2024-09-19T05:18:53.568 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mds.e.log 2024-09-19T05:18:53.568 INFO:teuthology.orchestra.run.smithi043.stderr:find: ‘/var/log/rbd-target-api’gzip -5: No such file or directory --verbose 2024-09-19T05:18:53.568 INFO:teuthology.orchestra.run.smithi043.stderr: -- /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-osd.0.log 2024-09-19T05:18:53.569 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-osd.9.log 2024-09-19T05:18:53.569 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T05:18:53.570 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T05:18:53.571 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mds.a.log 2024-09-19T05:18:53.572 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T05:18:53.574 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mon.a.log 2024-09-19T05:18:53.576 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mgr.z.log: 92.0% -- replaced with /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mgr.z.log.gz 2024-09-19T05:18:53.576 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mds.g.log 2024-09-19T05:18:53.576 INFO:teuthology.orchestra.run.smithi043.stderr: 84.5% -- replaced with /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T05:18:53.576 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mon.a.log: 90.3% -- replaced with /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T05:18:53.577 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-osd.6.log 2024-09-19T05:18:53.578 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph.log 2024-09-19T05:18:53.578 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mds.f.log 2024-09-19T05:18:53.580 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-osd.6.log: /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mds.d.log 2024-09-19T05:18:53.583 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T05:18:53.584 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mgr.x.log 2024-09-19T05:18:53.586 INFO:teuthology.orchestra.run.smithi043.stderr: 90.8% -- replaced with /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T05:18:53.588 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mgr.x.log: 87.4% -- replaced with /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T05:18:53.589 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mds.j.log 2024-09-19T05:18:53.589 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-osd.3.log 2024-09-19T05:18:53.591 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T05:18:53.592 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-osd.3.log: 94.5% -- replaced with /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mds.d.log.gz 2024-09-19T05:18:53.594 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T05:18:53.605 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 93.0% -- replaced with /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mds.i.log.gz 2024-09-19T05:18:53.606 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T05:18:53.606 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/cephadm.log: 94.6% -- replaced with /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mds.j.log.gz 2024-09-19T05:18:53.612 INFO:teuthology.orchestra.run.smithi151.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T05:18:53.613 INFO:teuthology.orchestra.run.smithi043.stderr: 94.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T05:18:53.617 INFO:teuthology.orchestra.run.smithi151.stderr: 94.4% -- replaced with /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mds.l.log.gz 2024-09-19T05:18:53.617 INFO:teuthology.orchestra.run.smithi096.stderr: 94.5% -- replaced with /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mds.k.log.gz 2024-09-19T05:18:53.631 INFO:teuthology.orchestra.run.smithi096.stderr: 92.8% -- replaced with /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mds.h.log.gz 2024-09-19T05:18:53.637 INFO:teuthology.orchestra.run.smithi151.stderr: 93.1% -- replaced with /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mds.c.log.gz 2024-09-19T05:18:53.645 INFO:teuthology.orchestra.run.smithi043.stderr: 93.0% -- replaced with /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mds.g.log.gz 2024-09-19T05:18:53.647 INFO:teuthology.orchestra.run.smithi043.stderr: 94.5% -- replaced with /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mds.a.log.gz 2024-09-19T05:18:53.664 INFO:teuthology.orchestra.run.smithi096.stderr: 94.6% -- replaced with /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mds.e.log.gz 2024-09-19T05:18:53.690 INFO:teuthology.orchestra.run.smithi151.stderr: 93.8% -- replaced with /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mds.f.log.gz 2024-09-19T05:18:53.922 INFO:teuthology.orchestra.run.smithi096.stderr: 92.8% -- replaced with /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mds.b.log.gz 2024-09-19T05:18:53.957 INFO:teuthology.orchestra.run.smithi151.stderr: 91.6% -- replaced with /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mon.c.log.gz 2024-09-19T05:18:54.009 INFO:teuthology.orchestra.run.smithi043.stderr: 90.2% -- replaced with /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mgr.x.log.gz 2024-09-19T05:18:54.061 INFO:teuthology.orchestra.run.smithi096.stderr: 91.6% -- replaced with /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mon.b.log.gz 2024-09-19T05:18:54.937 INFO:teuthology.orchestra.run.smithi043.stderr: 90.4% -- replaced with /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-mon.a.log.gz 2024-09-19T05:18:58.981 INFO:teuthology.orchestra.run.smithi043.stderr: 93.5% -- replaced with /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-osd.9.log.gz 2024-09-19T05:18:59.013 INFO:teuthology.orchestra.run.smithi096.stderr: 93.5% -- replaced with /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-osd.7.log.gz 2024-09-19T05:18:59.202 INFO:teuthology.orchestra.run.smithi151.stderr: 93.6% -- replaced with /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-osd.8.log.gz 2024-09-19T05:18:59.690 INFO:teuthology.orchestra.run.smithi151.stderr: 93.5% -- replaced with /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-osd.2.log.gz 2024-09-19T05:18:59.968 INFO:teuthology.orchestra.run.smithi043.stderr: 93.5% -- replaced with /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-osd.6.log.gz 2024-09-19T05:19:00.028 INFO:teuthology.orchestra.run.smithi151.stderr: 93.6% -- replaced with /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-osd.11.log.gz 2024-09-19T05:19:00.055 INFO:teuthology.orchestra.run.smithi096.stderr: 93.7% -- replaced with /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-osd.4.log.gz 2024-09-19T05:19:00.601 INFO:teuthology.orchestra.run.smithi096.stderr: 93.5% -- replaced with /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-osd.10.log.gz 2024-09-19T05:19:00.859 INFO:teuthology.orchestra.run.smithi151.stderr: 93.6% -- replaced with /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-osd.5.log.gz 2024-09-19T05:19:00.861 INFO:teuthology.orchestra.run.smithi151.stderr: 2024-09-19T05:19:00.861 INFO:teuthology.orchestra.run.smithi151.stderr:real 0m7.327s 2024-09-19T05:19:00.861 INFO:teuthology.orchestra.run.smithi151.stderr:user 0m25.358s 2024-09-19T05:19:00.861 INFO:teuthology.orchestra.run.smithi151.stderr:sys 0m0.919s 2024-09-19T05:19:00.938 INFO:teuthology.orchestra.run.smithi096.stderr: 93.6% -- replaced with /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-osd.1.log.gz 2024-09-19T05:19:00.939 INFO:teuthology.orchestra.run.smithi096.stderr: 2024-09-19T05:19:00.940 INFO:teuthology.orchestra.run.smithi096.stderr:real 0m7.405s 2024-09-19T05:19:00.940 INFO:teuthology.orchestra.run.smithi096.stderr:user 0m26.539s 2024-09-19T05:19:00.940 INFO:teuthology.orchestra.run.smithi096.stderr:sys 0m0.935s 2024-09-19T05:19:01.525 INFO:teuthology.orchestra.run.smithi043.stderr: 93.5% -- replaced with /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-osd.0.log.gz 2024-09-19T05:19:01.782 INFO:teuthology.orchestra.run.smithi043.stderr: 93.5% -- replaced with /var/log/ceph/1456a810-7643-11ef-bceb-c7b262605968/ceph-osd.3.log.gz 2024-09-19T05:19:01.786 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-09-19T05:19:01.786 INFO:teuthology.orchestra.run.smithi043.stderr:real 0m8.240s 2024-09-19T05:19:01.786 INFO:teuthology.orchestra.run.smithi043.stderr:user 0m27.518s 2024-09-19T05:19:01.786 INFO:teuthology.orchestra.run.smithi043.stderr:sys 0m2.448s 2024-09-19T05:19:01.787 INFO:tasks.cephadm:Archiving logs... 2024-09-19T05:19:01.787 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911648/remote/smithi043/log 2024-09-19T05:19:01.788 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T05:19:06.485 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911648/remote/smithi096/log 2024-09-19T05:19:06.486 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T05:19:10.638 DEBUG:teuthology.misc:Transferring archived files from smithi151:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911648/remote/smithi151/log 2024-09-19T05:19:10.639 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T05:19:14.545 INFO:tasks.cephadm:Removing cluster... 2024-09-19T05:19:14.546 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1456a810-7643-11ef-bceb-c7b262605968 --force 2024-09-19T05:19:14.831 INFO:teuthology.orchestra.run.smithi043.stdout:Deleting cluster with fsid: 1456a810-7643-11ef-bceb-c7b262605968 2024-09-19T05:19:15.383 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1456a810-7643-11ef-bceb-c7b262605968 --force 2024-09-19T05:19:15.583 INFO:teuthology.orchestra.run.smithi096.stdout:Deleting cluster with fsid: 1456a810-7643-11ef-bceb-c7b262605968 2024-09-19T05:19:15.867 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1456a810-7643-11ef-bceb-c7b262605968 --force 2024-09-19T05:19:16.071 INFO:teuthology.orchestra.run.smithi151.stdout:Deleting cluster with fsid: 1456a810-7643-11ef-bceb-c7b262605968 2024-09-19T05:19:16.360 INFO:tasks.cephadm:Removing cephadm ... 2024-09-19T05:19:16.360 DEBUG:teuthology.orchestra.run.smithi043:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T05:19:16.397 DEBUG:teuthology.orchestra.run.smithi096:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T05:19:16.414 DEBUG:teuthology.orchestra.run.smithi151:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T05:19:16.430 INFO:tasks.cephadm:Teardown complete 2024-09-19T05:19:16.430 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-19T05:19:16.443 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-19T05:19:16.443 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T05:19:16.448 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T05:19:16.456 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T05:19:16.571 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-19T05:19:16.582 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-19T05:19:16.582 DEBUG:teuthology.orchestra.run.smithi043:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T05:19:16.586 DEBUG:teuthology.orchestra.run.smithi096:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T05:19:16.589 DEBUG:teuthology.orchestra.run.smithi151:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T05:19:16.601 INFO:teuthology.orchestra.run.smithi096.stderr:bash: line 1: ntpq: command not found 2024-09-19T05:19:16.604 INFO:teuthology.orchestra.run.smithi151.stderr:bash: line 1: ntpq: command not found 2024-09-19T05:19:16.606 INFO:teuthology.orchestra.run.smithi096.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T05:19:16.606 INFO:teuthology.orchestra.run.smithi096.stdout:=============================================================================== 2024-09-19T05:19:16.606 INFO:teuthology.orchestra.run.smithi096.stdout:^* hv01.front.sepia.ceph.com 3 7 377 110 -40us[ -32us] +/- 46ms 2024-09-19T05:19:16.606 INFO:teuthology.orchestra.run.smithi096.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 48 +373us[ +373us] +/- 76ms 2024-09-19T05:19:16.606 INFO:teuthology.orchestra.run.smithi096.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 46 -232us[ -232us] +/- 69ms 2024-09-19T05:19:16.606 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T05:19:16.608 INFO:teuthology.orchestra.run.smithi151.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T05:19:16.608 INFO:teuthology.orchestra.run.smithi151.stdout:=============================================================================== 2024-09-19T05:19:16.608 INFO:teuthology.orchestra.run.smithi151.stdout:^* hv01.front.sepia.ceph.com 3 6 377 47 -70us[ -67us] +/- 47ms 2024-09-19T05:19:16.608 INFO:teuthology.orchestra.run.smithi151.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 45 +385us[ +385us] +/- 76ms 2024-09-19T05:19:16.608 INFO:teuthology.orchestra.run.smithi151.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 46 -222us[ -222us] +/- 69ms 2024-09-19T05:19:16.608 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T05:19:16.616 INFO:teuthology.orchestra.run.smithi043.stderr:bash: line 1: ntpq: command not found 2024-09-19T05:19:16.623 INFO:teuthology.orchestra.run.smithi043.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T05:19:16.623 INFO:teuthology.orchestra.run.smithi043.stdout:=============================================================================== 2024-09-19T05:19:16.624 INFO:teuthology.orchestra.run.smithi043.stdout:^* hv01.front.sepia.ceph.com 3 6 377 47 -64us[ -49us] +/- 47ms 2024-09-19T05:19:16.624 INFO:teuthology.orchestra.run.smithi043.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 48 +362us[ +378us] +/- 76ms 2024-09-19T05:19:16.624 INFO:teuthology.orchestra.run.smithi043.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 45 -227us[ -227us] +/- 69ms 2024-09-19T05:19:16.624 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T05:19:16.625 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-19T05:19:16.635 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-19T05:19:16.635 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-19T05:19:16.731 DEBUG:teuthology.orchestra.run.smithi043:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T05:19:16.868 DEBUG:teuthology.orchestra.run.smithi096:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T05:19:16.937 DEBUG:teuthology.orchestra.run.smithi151:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T05:19:17.006 DEBUG:teuthology.orchestra.run.smithi043:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T05:19:17.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T05:19:17.067 DEBUG:teuthology.orchestra.run.smithi096:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T05:19:17.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T05:19:17.096 DEBUG:teuthology.orchestra.run.smithi151:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T05:19:17.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T05:19:17.125 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-19T05:19:17.136 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-19T05:19:17.216 INFO:teuthology.task.internal:Duration was 1954.369770 seconds 2024-09-19T05:19:17.216 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-19T05:19:17.257 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-19T05:19:17.257 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T05:19:17.262 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T05:19:17.264 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T05:19:17.302 INFO:teuthology.orchestra.run.smithi096.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T05:19:17.303 INFO:teuthology.orchestra.run.smithi151.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T05:19:17.346 INFO:teuthology.orchestra.run.smithi043.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T05:19:17.839 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-19T05:19:17.840 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi043.front.sepia.ceph.com 2024-09-19T05:19:17.840 DEBUG:teuthology.orchestra.run.smithi043:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T05:19:17.890 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi096.front.sepia.ceph.com 2024-09-19T05:19:17.890 DEBUG:teuthology.orchestra.run.smithi096:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T05:19:17.915 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi151.front.sepia.ceph.com 2024-09-19T05:19:17.915 DEBUG:teuthology.orchestra.run.smithi151:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T05:19:17.939 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-19T05:19:17.939 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T05:19:17.943 DEBUG:teuthology.orchestra.run.smithi096:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T05:19:17.958 DEBUG:teuthology.orchestra.run.smithi151:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T05:19:18.007 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-19T05:19:18.007 DEBUG:teuthology.orchestra.run.smithi043:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T05:19:18.011 DEBUG:teuthology.orchestra.run.smithi096:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T05:19:18.026 DEBUG:teuthology.orchestra.run.smithi151:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T05:19:19.328 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-19T05:19:19.345 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-19T05:19:19.346 DEBUG:teuthology.orchestra.run.smithi043:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T05:19:19.402 DEBUG:teuthology.orchestra.run.smithi096:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T05:19:19.432 DEBUG:teuthology.orchestra.run.smithi151:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T05:19:19.463 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-19T05:19:19.474 DEBUG:teuthology.orchestra.run.smithi043:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T05:19:19.478 DEBUG:teuthology.orchestra.run.smithi096:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T05:19:19.481 DEBUG:teuthology.orchestra.run.smithi151:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T05:19:19.505 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern = core 2024-09-19T05:19:19.531 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern = core 2024-09-19T05:19:19.545 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern = core 2024-09-19T05:19:19.575 DEBUG:teuthology.orchestra.run.smithi043:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T05:19:19.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T05:19:19.609 DEBUG:teuthology.orchestra.run.smithi096:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T05:19:19.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T05:19:19.626 DEBUG:teuthology.orchestra.run.smithi151:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T05:19:19.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T05:19:19.642 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-19T05:19:19.658 INFO:teuthology.task.internal:Transferring archived files... 2024-09-19T05:19:19.660 DEBUG:teuthology.misc:Transferring archived files from smithi043:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911648/remote/smithi043 2024-09-19T05:19:19.660 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T05:19:19.756 DEBUG:teuthology.misc:Transferring archived files from smithi096:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911648/remote/smithi096 2024-09-19T05:19:19.756 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T05:19:19.797 DEBUG:teuthology.misc:Transferring archived files from smithi151:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911648/remote/smithi151 2024-09-19T05:19:19.797 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T05:19:19.838 INFO:teuthology.task.internal:Removing archive directory... 2024-09-19T05:19:19.838 DEBUG:teuthology.orchestra.run.smithi043:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T05:19:19.842 DEBUG:teuthology.orchestra.run.smithi096:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T05:19:19.844 DEBUG:teuthology.orchestra.run.smithi151:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T05:19:19.888 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-19T05:19:19.901 INFO:teuthology.task.internal:Not uploading archives. 2024-09-19T05:19:19.901 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-19T05:19:19.961 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-19T05:19:19.961 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T05:19:19.965 DEBUG:teuthology.orchestra.run.smithi096:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T05:19:19.968 DEBUG:teuthology.orchestra.run.smithi151:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T05:19:19.983 INFO:teuthology.orchestra.run.smithi096.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 05:19 /home/ubuntu/cephtest 2024-09-19T05:19:19.984 INFO:teuthology.orchestra.run.smithi151.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 05:19 /home/ubuntu/cephtest 2024-09-19T05:19:19.996 INFO:teuthology.orchestra.run.smithi043.stdout: 393255 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 05:19 /home/ubuntu/cephtest 2024-09-19T05:19:19.998 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-19T05:19:20.010 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-19T05:19:20.223 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/suites/iozone}} duration: 1954.3697698116302 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-19T05:19:20.223 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T05:19:20.313 INFO:teuthology.run:pass